2024-04-25T13:24:25.564 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:24:25.564 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/orig.config.yaml 2024-04-25T13:24:25.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:24:25.700 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673159' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGu8IIzB5+xAZg31N4rryMTZ1MkHDagRf1tlakphRpJuB73FdZKaQhPQbYdznH0JiqNF2rhU6qufQsQkrzSm1g8= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK57oJeXj48zBOm7XcNb7pSP0YqLmJdMo/6K0024P9V+FFlQJz/W34RtgnWRGRvgHdffIsx/7m2Qqq5ffqeZvT8= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWSLpcmBG5e7EyRynntsoDR2YeP89oDm0UMRz8bwfgBST/Uvnls78NP5VkNhE48wgPgoRur7HHlqJUQFVxno0c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:24:25.700 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T13:24:25.701 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T13:24:25.701 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:24:25.703 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:24:25.724 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T13:24:25.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:24:25.724 INFO:teuthology.packaging:ref: None 2024-04-25T13:24:25.724 INFO:teuthology.packaging:tag: None 2024-04-25T13:24:25.724 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:24:25.724 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:24:25.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:24:25.868 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T13:24:25.869 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:24:25.875 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:24:25.876 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:24:25.894 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:24:25.913 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:24:25.919 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:24:25.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:19:28.337651', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGu8IIzB5+xAZg31N4rryMTZ1MkHDagRf1tlakphRpJuB73FdZKaQhPQbYdznH0JiqNF2rhU6qufQsQkrzSm1g8='} 2024-04-25T13:24:25.958 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:19:28.336245', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK57oJeXj48zBOm7XcNb7pSP0YqLmJdMo/6K0024P9V+FFlQJz/W34RtgnWRGRvgHdffIsx/7m2Qqq5ffqeZvT8='} 2024-04-25T13:24:25.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:19:28.339023', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWSLpcmBG5e7EyRynntsoDR2YeP89oDm0UMRz8bwfgBST/Uvnls78NP5VkNhE48wgPgoRur7HHlqJUQFVxno0c='} 2024-04-25T13:24:25.976 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:24:25.982 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T13:24:25.983 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T13:24:25.983 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T13:24:25.983 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:24:26.102 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd317be40d0>, signals=[15]) 2024-04-25T13:24:26.103 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:24:26.109 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:24:26.109 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T13:24:26.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:24:26.186 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-04-25T13:24:26.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:24:26.260 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T13:24:26.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:24:26.334 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:24:26.341 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-25T13:24:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-25T13:24:26.359 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-25T13:24:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:24:26.416 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:24:26.416 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:24:26.416 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:24:26.416 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:24:26.416 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:24:26.416 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-25T13:24:26.436 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-04-25T13:24:26.454 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-04-25T13:24:26.454 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:24:26.511 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:24:26.512 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:24:26.512 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:24:26.512 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:24:26.512 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:24:26.512 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-04-25T13:24:26.527 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T13:24:26.544 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T13:24:26.545 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T13:24:26.601 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:24:26.602 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:24:26.603 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T13:24:26.619 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:24:26.630 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:24:26.636 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:24:26.636 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:24:26.638 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:24:26.641 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:24:26.657 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:24:26.663 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:24:26.664 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:24:26.694 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:24:26.697 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:24:26.811 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:24:26.874 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:24:26.889 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:24:26.889 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T13:24:26.889 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T13:24:26.890 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:24:26.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:26.906 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T13:24:26.962 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-04-25T13:24:26.962 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-437.el9.x86_64 2024-04-25T13:24:26.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-04-25T13:24:29.575 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.0 MB 00:01 2024-04-25T13:24:31.526 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T13:24:35.603 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T13:24:37.251 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-25T13:24:38.478 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T13:24:43.578 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T13:24:45.476 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T13:24:45.476 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T13:24:45.558 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T13:24:45.559 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-25T13:24:45.559 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T13:24:45.672 DEBUG:teuthology.orchestra.run.smithi094:> echo no | sudo yum reinstall kernel || true 2024-04-25T13:24:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:24:43 PM UTC. 2024-04-25T13:24:46.291 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalling: 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:46.310 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 255 k 2024-04-25T13:24:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 0 2024-04-25T13:24:46.311 INFO:teuthology.orchestra.run.smithi094.stderr:Operation aborted. 2024-04-25T13:24:46.375 INFO:teuthology.orchestra.run.smithi094.stdout:Is this ok [y/N]: 2024-04-25T13:24:46.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum reinstall -y kernel || true 2024-04-25T13:24:46.867 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:24:43 PM UTC. 2024-04-25T13:24:46.985 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalling: 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:46.986 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T13:24:46.987 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-25T13:24:46.987 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:46.987 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 255 k 2024-04-25T13:24:46.987 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 0 2024-04-25T13:24:46.987 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T13:24:47.755 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-437.el9.x86_64.rpm 520 kB/s | 255 kB 00:00 2024-04-25T13:24:47.756 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:24:47.756 INFO:teuthology.orchestra.run.smithi094.stdout:Total 331 kB/s | 255 kB 00:00 2024-04-25T13:24:47.774 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T13:24:47.934 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T13:24:47.934 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T13:24:47.939 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T13:24:47.939 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T13:24:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T13:24:48.173 INFO:teuthology.orchestra.run.smithi094.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:24:48.173 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:24:48.848 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:24:48.848 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout:Reinstalled: 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:24:49.544 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T13:24:49.729 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T13:24:49.775 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T13:24:49.775 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T13:24:49.776 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T13:24:49.776 DEBUG:teuthology.parallel:result is None 2024-04-25T13:24:49.776 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:24:49.784 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:24:49.784 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:24:49.787 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:24:49.789 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:24:49.806 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:24:49.813 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:24:49.824 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:24:49.824 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:24:49.841 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:24:49.846 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:24:49.958 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:24:49.964 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:24:49.964 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:24:49.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:49.981 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:24:49.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:49.998 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:24:50.000 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:24:50.024 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:24:50.044 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.062 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.068 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.081 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.103 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.107 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:24:50.109 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:24:50.117 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:24:50.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:24:50.119 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:24:50.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:24:50.176 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:24:50.184 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:24:50.184 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:24:50.187 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:24:50.189 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:24:50.232 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.328 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:50.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:50.449 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:24:50.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:24:50.512 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.600 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:50.681 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:50.725 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:24:50.725 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:24:50.791 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.835 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:24:50.891 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:50.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:24:51.015 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:24:51.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:24:51.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-25T13:24:51.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-04-25T13:24:51.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T13:24:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:24:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:24:51.159 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:24:51.532 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:24:51.540 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:24:51.540 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:24:51.550 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:24:51.558 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T13:24:51.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-25T13:24:51.604 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-25T13:24:51.606 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-04-25T13:24:51.678 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-04-25T13:24:51.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-25T13:24:51.744 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-25T13:24:51.745 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:24:51.746 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-25T13:24:51.776 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-25T13:24:51.776 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-04-25T13:24:51.803 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-04-25T13:24:51.803 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-25T13:24:51.823 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-25T13:24:51.824 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-25T13:24:51.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:24:51.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:51.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:24:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:51.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:24:51.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:24:51.901 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:24:51.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:24:51.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:24:51.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:24:51.988 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:24:51.997 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:24:52.012 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:24:52.014 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:28:42.944 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T13:28:42.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T13:28:42.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:43.027 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T13:28:43.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T13:28:43.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-25T13:28:43.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:43.197 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-25T13:28:43.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-25T13:28:43.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T13:28:43.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:43.356 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T13:28:43.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T13:28:43.447 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:28:43.459 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:28:43.459 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:28:43.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:28:43.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:28:43.462 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:28:43.464 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:28:43.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:28:43.488 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:28:43.492 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:28:43.501 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:28:43.506 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:28:43.516 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:28:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-25T13:28:43.529 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-04-25T13:28:43.530 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:28:43.535 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-25T13:28:43.540 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-04-25T13:28:43.548 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:28:43.553 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:28:43.554 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-25T13:28:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:28:43.564 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-25T13:28:43.565 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:28:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:28:43.591 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:28:43.621 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.625 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:28:43.632 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:28:43.632 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T13:28:43.632 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.632 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.633 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.633 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.647 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:28:43.648 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:28:43.648 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T13:28:43.648 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.649 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.649 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.649 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:28:43.649 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:28:43.657 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:28:43.657 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T13:28:43.657 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T13:28:43.658 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:28:43.663 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:28:43.663 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:28:43.664 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:28:43.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:28:43.667 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:28:43.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:28:43.668 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:28:43.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:28:43.819 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:28:43.820 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:28:43.824 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:28:43.824 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:28:43.831 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:28:43.831 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:28:43.940 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:28:43.941 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:28:43.941 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:28:43.945 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:28:43.946 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:28:43.946 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:28:43.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:28:43.966 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:28:43.971 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:28:43.971 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:28:44.001 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:28:44.002 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:28:44.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:28:44.026 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:28:44.027 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:28:44.031 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:28:44.035 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:28:44.093 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:28:44.104 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:28:44.111 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:28:44.168 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:28:44.168 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-25T13:28:44.169 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-25T13:28:44.175 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-04-25T13:28:44.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-04-25T13:28:44.231 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-25T13:28:44.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-25T13:28:44.392 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-25T13:28:44.400 INFO:teuthology.orchestra.run.smithi137.stdout:48 files removed 2024-04-25T13:28:44.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:28:44.428 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:28:44.457 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-25T13:28:44.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:28:45.027 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 243 kB/s | 84 kB 00:00 2024-04-25T13:28:45.040 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 238 kB/s | 84 kB 00:00 2024-04-25T13:28:45.165 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 200 kB/s | 84 kB 00:00 2024-04-25T13:28:45.427 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T13:28:45.457 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T13:28:45.540 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-25T13:28:45.811 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-25T13:28:45.850 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T13:28:45.999 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-25T13:28:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T13:28:46.726 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T13:28:46.862 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-04-25T13:28:48.499 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T13:28:48.770 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-25T13:28:48.987 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T13:28:52.792 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-04-25T13:28:53.004 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T13:28:53.154 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-04-25T13:28:54.586 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-25T13:28:54.602 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-25T13:28:54.608 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-25T13:28:56.259 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T13:28:56.606 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:05 2024-04-25T13:29:01.334 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-04-25T13:29:01.753 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T13:29:03.478 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.479 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.489 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:29:03.490 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.490 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:29:03.491 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:29:03.493 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:29:03.495 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:29:03.496 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:29:03.557 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:29:03.566 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:29:03.567 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:29:03.568 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:29:03.569 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:29:03.570 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:29:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:29:03.572 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:29:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:29:03.574 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:29:03.575 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:29:03.576 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:29:03.577 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T13:29:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T13:29:03.579 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-25T13:29:03.579 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-25T13:29:03.579 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:03.579 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 214 M 2024-04-25T13:29:03.580 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T13:29:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.889 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.900 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:29:03.901 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:29:03.901 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:29:03.902 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:29:03.904 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:29:03.906 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:29:03.907 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:29:03.969 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-04-25T13:29:03.988 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:29:03.989 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:29:03.990 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:29:03.991 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:29:03.992 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:29:03.993 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:29:03.994 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:29:03.995 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:29:03.996 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:29:03.997 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:29:03.998 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:29:03.999 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:29:04.000 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:29:04.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:29:04.002 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 4 Packages 2024-04-25T13:29:04.003 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:29:04.004 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 214 M 2024-04-25T13:29:04.004 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-04-25T13:29:04.386 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T13:29:04.603 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 3.9 MB/s | 863 kB 00:00 2024-04-25T13:29:04.653 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.9 MB/s | 147 kB 00:00 2024-04-25T13:29:04.937 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 8.0 MB/s | 2.3 MB 00:00 2024-04-25T13:29:04.955 INFO:teuthology.orchestra.run.smithi137.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-25T13:29:05.012 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 6.8 MB/s | 5.4 MB 00:00 2024-04-25T13:29:05.187 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-04-25T13:29:05.554 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 9.0 MB/s | 4.9 MB 00:00 2024-04-25T13:29:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 863 kB 00:00 2024-04-25T13:29:05.795 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-25T13:29:05.930 INFO:teuthology.orchestra.run.smithi137.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 735 kB/s | 147 kB 00:00 2024-04-25T13:29:06.614 INFO:teuthology.orchestra.run.smithi137.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-25T13:29:06.813 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 8.0 MB/s | 10 MB 00:01 2024-04-25T13:29:06.856 INFO:teuthology.orchestra.run.smithi137.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-04-25T13:29:06.888 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 334 kB/s | 25 kB 00:00 2024-04-25T13:29:07.148 INFO:teuthology.orchestra.run.smithi137.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.9 MB/s | 2.3 MB 00:01 2024-04-25T13:29:07.722 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 7.0 MB/s | 18 MB 00:02 2024-04-25T13:29:07.773 INFO:teuthology.orchestra.run.smithi137.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.3 MB/s | 4.9 MB 00:00 2024-04-25T13:29:07.822 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.2 MB/s | 224 kB 00:00 2024-04-25T13:29:07.898 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 361 kB/s | 27 kB 00:00 2024-04-25T13:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 422 kB/s | 32 kB 00:00 2024-04-25T13:29:07.990 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:07.990 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:29:08.000 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:29:08.001 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:29:08.002 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:29:08.003 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:29:08.004 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:29:08.006 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:29:08.007 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:29:08.068 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:29:08.087 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:29:08.088 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:29:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:29:08.090 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:29:08.091 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:29:08.092 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:29:08.093 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:29:08.094 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:29:08.095 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:29:08.096 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:29:08.097 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:29:08.098 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 4 Packages 2024-04-25T13:29:08.099 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:29:08.100 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 214 M 2024-04-25T13:29:08.100 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T13:29:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.4 MB/s | 756 kB 00:00 2024-04-25T13:29:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.8 MB/s | 158 kB 00:00 2024-04-25T13:29:08.307 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.5 MB/s | 127 kB 00:00 2024-04-25T13:29:08.474 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.9 MB/s | 497 kB 00:00 2024-04-25T13:29:09.041 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): ceph-common-19.0.0-3291.g994e2f20.el9 4.1 MB/s | 20 MB 00:04 2024-04-25T13:29:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 417 kB/s | 45 kB 00:00 2024-04-25T13:29:09.259 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.2 MB/s | 135 kB 00:00 2024-04-25T13:29:09.375 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.4 MB/s | 161 kB 00:00 2024-04-25T13:29:09.459 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.2 MB/s | 5.2 MB 00:00 2024-04-25T13:29:09.534 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.9 MB/s | 298 kB 00:00 2024-04-25T13:29:09.568 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rados-19.0.0-3291.g994e2f20.e 1.6 MB/s | 321 kB 00:00 2024-04-25T13:29:09.580 INFO:teuthology.orchestra.run.smithi138.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T13:29:09.589 INFO:teuthology.orchestra.run.smithi137.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 5.6 MB/s | 10 MB 00:01 2024-04-25T13:29:09.602 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T13:29:09.644 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T13:29:09.648 INFO:teuthology.orchestra.run.smithi137.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 433 kB/s | 25 kB 00:00 2024-04-25T13:29:09.736 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 169 kB 00:00 2024-04-25T13:29:09.853 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 292 kB/s | 34 kB 00:00 2024-04-25T13:29:09.962 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.3 MB/s | 142 kB 00:00 2024-04-25T13:29:10.362 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.0 MB/s | 3.0 MB 00:00 2024-04-25T13:29:10.546 INFO:teuthology.orchestra.run.smithi138.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 893 kB/s | 863 kB 00:00 2024-04-25T13:29:10.746 INFO:teuthology.orchestra.run.smithi138.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 735 kB/s | 147 kB 00:00 2024-04-25T13:29:11.047 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.3 MB/s | 3.5 MB 00:01 2024-04-25T13:29:11.155 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T13:29:11.622 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 9.1 MB/s | 43 MB 00:04 2024-04-25T13:29:11.649 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 100 kB/s | 49 kB 00:00 2024-04-25T13:29:11.698 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 219 kB/s | 16 kB 00:00 2024-04-25T13:29:11.815 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.6 MB/s | 273 kB 00:00 2024-04-25T13:29:11.830 INFO:teuthology.orchestra.run.smithi138.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-04-25T13:29:11.899 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T13:29:11.915 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 182 kB/s | 18 kB 00:00 2024-04-25T13:29:12.082 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-04-25T13:29:12.108 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 384 kB/s | 80 kB 00:00 2024-04-25T13:29:12.141 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2024-04-25T13:29:12.148 INFO:teuthology.orchestra.run.smithi137.stdout:(10/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.6 MB/s | 20 MB 00:07 2024-04-25T13:29:12.172 INFO:teuthology.orchestra.run.smithi138.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-04-25T13:29:12.175 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-25T13:29:12.225 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T13:29:12.250 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T13:29:12.276 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-25T13:29:12.281 INFO:teuthology.orchestra.run.smithi137.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.6 MB/s | 224 kB 00:00 2024-04-25T13:29:12.301 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-25T13:29:12.326 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T13:29:12.352 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T13:29:12.355 INFO:teuthology.orchestra.run.smithi138.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-25T13:29:12.369 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-25T13:29:12.373 INFO:teuthology.orchestra.run.smithi137.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 295 kB/s | 27 kB 00:00 2024-04-25T13:29:12.411 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-25T13:29:12.436 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-04-25T13:29:12.457 INFO:teuthology.orchestra.run.smithi137.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T13:29:12.461 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-04-25T13:29:12.487 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-25T13:29:12.512 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T13:29:12.537 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-25T13:29:12.596 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-25T13:29:12.630 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 97 kB/s | 11 kB 00:00 2024-04-25T13:29:12.655 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T13:29:12.680 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T13:29:12.705 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-25T13:29:12.739 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 582 kB 00:00 2024-04-25T13:29:12.749 INFO:teuthology.orchestra.run.smithi137.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.5 MB/s | 756 kB 00:00 2024-04-25T13:29:12.765 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 825 kB/s | 47 kB 00:00 2024-04-25T13:29:12.790 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 306 kB/s | 25 kB 00:00 2024-04-25T13:29:12.815 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T13:29:12.840 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T13:29:12.841 INFO:teuthology.orchestra.run.smithi137.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T13:29:12.866 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T13:29:12.958 INFO:teuthology.orchestra.run.smithi137.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T13:29:12.991 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-25T13:29:13.033 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.6 MB/s | 1.6 MB 00:00 2024-04-25T13:29:13.100 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T13:29:13.106 INFO:teuthology.orchestra.run.smithi138.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.2 MB/s | 4.9 MB 00:00 2024-04-25T13:29:13.125 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-25T13:29:13.151 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T13:29:13.192 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T13:29:13.200 INFO:teuthology.orchestra.run.smithi137.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.0 MB/s | 497 kB 00:00 2024-04-25T13:29:13.218 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T13:29:13.360 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T13:29:13.435 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-04-25T13:29:13.460 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-25T13:29:13.486 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-04-25T13:29:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-04-25T13:29:13.536 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-25T13:29:13.562 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T13:29:13.587 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-04-25T13:29:13.612 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T13:29:13.637 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.5 MB/s | 222 kB 00:00 2024-04-25T13:29:13.704 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-04-25T13:29:13.730 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 672 kB/s | 77 kB 00:00 2024-04-25T13:29:13.734 INFO:teuthology.orchestra.run.smithi137.stdout:(18/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.7 MB/s | 18 MB 00:06 2024-04-25T13:29:13.747 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 396 kB/s | 42 kB 00:00 2024-04-25T13:29:13.772 INFO:teuthology.orchestra.run.smithi138.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.2 MB/s | 20 MB 00:04 2024-04-25T13:29:13.789 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T13:29:13.815 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-25T13:29:13.840 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-25T13:29:13.842 INFO:teuthology.orchestra.run.smithi137.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 417 kB/s | 45 kB 00:00 2024-04-25T13:29:13.865 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-04-25T13:29:13.881 INFO:teuthology.orchestra.run.smithi138.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 233 kB/s | 25 kB 00:00 2024-04-25T13:29:13.898 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-04-25T13:29:13.959 INFO:teuthology.orchestra.run.smithi137.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.1 MB/s | 135 kB 00:00 2024-04-25T13:29:14.024 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 42 kB 00:00 2024-04-25T13:29:14.068 INFO:teuthology.orchestra.run.smithi137.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.5 MB/s | 161 kB 00:00 2024-04-25T13:29:14.174 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 160 kB/s | 44 kB 00:00 2024-04-25T13:29:14.310 INFO:teuthology.orchestra.run.smithi137.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.3 MB/s | 321 kB 00:00 2024-04-25T13:29:14.325 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T13:29:14.350 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 683 kB/s | 221 kB 00:00 2024-04-25T13:29:14.375 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 179 kB/s | 36 kB 00:00 2024-04-25T13:29:14.400 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T13:29:14.406 INFO:teuthology.orchestra.run.smithi138.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 7.8 MB/s | 10 MB 00:01 2024-04-25T13:29:14.426 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T13:29:14.435 INFO:teuthology.orchestra.run.smithi137.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 298 kB 00:00 2024-04-25T13:29:14.450 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 772 kB/s | 58 kB 00:00 2024-04-25T13:29:14.476 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-25T13:29:14.498 INFO:teuthology.orchestra.run.smithi138.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.4 MB/s | 224 kB 00:00 2024-04-25T13:29:14.501 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-25T13:29:14.552 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-25T13:29:14.560 INFO:teuthology.orchestra.run.smithi137.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 798 kB/s | 100 kB 00:00 2024-04-25T13:29:14.565 INFO:teuthology.orchestra.run.smithi138.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 406 kB/s | 27 kB 00:00 2024-04-25T13:29:14.577 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T13:29:14.602 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-04-25T13:29:14.627 INFO:teuthology.orchestra.run.smithi137.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T13:29:14.640 INFO:teuthology.orchestra.run.smithi138.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 424 kB/s | 32 kB 00:00 2024-04-25T13:29:14.757 INFO:teuthology.orchestra.run.smithi138.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 6.3 MB/s | 756 kB 00:00 2024-04-25T13:29:14.849 INFO:teuthology.orchestra.run.smithi138.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T13:29:14.933 INFO:teuthology.orchestra.run.smithi138.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.5 MB/s | 127 kB 00:00 2024-04-25T13:29:14.961 INFO:teuthology.orchestra.run.smithi137.stdout:(26/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 5.2 MB 00:01 2024-04-25T13:29:15.010 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 48 kB/s | 19 kB 00:00 2024-04-25T13:29:15.025 INFO:teuthology.orchestra.run.smithi138.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 5.3 MB/s | 497 kB 00:00 2024-04-25T13:29:15.061 INFO:teuthology.orchestra.run.smithi137.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T13:29:15.184 INFO:teuthology.orchestra.run.smithi137.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 279 kB/s | 34 kB 00:00 2024-04-25T13:29:15.253 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 475 kB/s | 309 kB 00:00 2024-04-25T13:29:15.286 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 747 kB/s | 548 kB 00:00 2024-04-25T13:29:15.295 INFO:teuthology.orchestra.run.smithi137.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.3 MB/s | 142 kB 00:00 2024-04-25T13:29:15.353 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T13:29:15.387 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-25T13:29:15.487 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 504 kB/s | 67 kB 00:00 2024-04-25T13:29:15.512 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T13:29:15.533 INFO:teuthology.orchestra.run.smithi138.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-25T13:29:15.592 INFO:teuthology.orchestra.run.smithi138.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 774 kB/s | 45 kB 00:00 2024-04-25T13:29:15.663 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-25T13:29:15.684 INFO:teuthology.orchestra.run.smithi138.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.4 MB/s | 135 kB 00:00 2024-04-25T13:29:15.729 INFO:teuthology.orchestra.run.smithi137.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 2.8 MB/s | 3.0 MB 00:01 2024-04-25T13:29:15.751 INFO:teuthology.orchestra.run.smithi138.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.4 MB/s | 161 kB 00:00 2024-04-25T13:29:15.755 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-04-25T13:29:15.780 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-25T13:29:15.814 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-25T13:29:15.851 INFO:teuthology.orchestra.run.smithi138.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.1 MB/s | 321 kB 00:00 2024-04-25T13:29:15.856 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-04-25T13:29:15.882 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T13:29:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T13:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T13:29:15.982 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T13:29:16.007 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T13:29:16.033 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-25T13:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T13:29:16.100 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T13:29:16.111 INFO:teuthology.orchestra.run.smithi138.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T13:29:16.138 INFO:teuthology.orchestra.run.smithi137.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.2 MB/s | 3.5 MB 00:00 2024-04-25T13:29:16.158 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T13:29:16.184 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 960 kB/s | 144 kB 00:00 2024-04-25T13:29:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-25T13:29:16.246 INFO:teuthology.orchestra.run.smithi137.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T13:29:16.260 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-04-25T13:29:16.285 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-25T13:29:16.310 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-25T13:29:16.338 INFO:teuthology.orchestra.run.smithi137.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 539 kB/s | 49 kB 00:00 2024-04-25T13:29:16.352 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T13:29:16.402 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-04-25T13:29:16.422 INFO:teuthology.orchestra.run.smithi137.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 197 kB/s | 16 kB 00:00 2024-04-25T13:29:16.453 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-04-25T13:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 8.7 MB/s | 3.0 MB 00:00 2024-04-25T13:29:16.486 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-25T13:29:16.503 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-04-25T13:29:16.538 INFO:teuthology.orchestra.run.smithi137.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 273 kB 00:00 2024-04-25T13:29:16.544 INFO:teuthology.orchestra.run.smithi138.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T13:29:16.553 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-25T13:29:16.579 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-25T13:29:16.604 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-25T13:29:16.628 INFO:teuthology.orchestra.run.smithi138.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 410 kB/s | 34 kB 00:00 2024-04-25T13:29:16.654 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-25T13:29:16.680 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-25T13:29:16.712 INFO:teuthology.orchestra.run.smithi138.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.7 MB/s | 142 kB 00:00 2024-04-25T13:29:16.714 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T13:29:16.722 INFO:teuthology.orchestra.run.smithi137.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-25T13:29:16.756 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 585 kB/s | 59 kB 00:00 2024-04-25T13:29:16.781 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-25T13:29:16.815 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-25T13:29:16.831 INFO:teuthology.orchestra.run.smithi137.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 169 kB/s | 18 kB 00:00 2024-04-25T13:29:16.857 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T13:29:16.890 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-04-25T13:29:16.931 INFO:teuthology.orchestra.run.smithi137.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 804 kB/s | 80 kB 00:00 2024-04-25T13:29:16.941 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T13:29:16.966 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T13:29:16.991 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T13:29:17.006 INFO:teuthology.orchestra.run.smithi137.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2024-04-25T13:29:17.048 INFO:teuthology.orchestra.run.smithi137.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T13:29:17.098 INFO:teuthology.orchestra.run.smithi137.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T13:29:17.120 INFO:teuthology.orchestra.run.smithi138.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.7 MB/s | 3.5 MB 00:00 2024-04-25T13:29:17.142 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-25T13:29:17.157 INFO:teuthology.orchestra.run.smithi137.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-25T13:29:17.191 INFO:teuthology.orchestra.run.smithi137.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T13:29:17.209 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T13:29:17.224 INFO:teuthology.orchestra.run.smithi137.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T13:29:17.258 INFO:teuthology.orchestra.run.smithi137.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T13:29:17.291 INFO:teuthology.orchestra.run.smithi137.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 6.3 MB/s | 215 kB 00:00 2024-04-25T13:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T13:29:17.333 INFO:teuthology.orchestra.run.smithi137.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 16 MB/s | 689 kB 00:00 2024-04-25T13:29:17.384 INFO:teuthology.orchestra.run.smithi137.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 943 kB 00:00 2024-04-25T13:29:17.409 INFO:teuthology.orchestra.run.smithi137.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 4.2 MB/s | 107 kB 00:00 2024-04-25T13:29:17.443 INFO:teuthology.orchestra.run.smithi137.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 643 kB/s | 21 kB 00:00 2024-04-25T13:29:17.493 INFO:teuthology.orchestra.run.smithi137.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:29:17.527 INFO:teuthology.orchestra.run.smithi137.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 981 kB/s | 33 kB 00:00 2024-04-25T13:29:17.627 INFO:teuthology.orchestra.run.smithi137.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T13:29:17.661 INFO:teuthology.orchestra.run.smithi137.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-04-25T13:29:17.694 INFO:teuthology.orchestra.run.smithi137.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-04-25T13:29:17.728 INFO:teuthology.orchestra.run.smithi137.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-25T13:29:17.762 INFO:teuthology.orchestra.run.smithi137.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T13:29:17.814 INFO:teuthology.orchestra.run.smithi137.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:29:17.862 INFO:teuthology.orchestra.run.smithi137.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-25T13:29:17.896 INFO:teuthology.orchestra.run.smithi137.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 765 kB/s | 25 kB 00:00 2024-04-25T13:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-25T13:29:17.938 INFO:teuthology.orchestra.run.smithi137.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T13:29:17.980 INFO:teuthology.orchestra.run.smithi137.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-04-25T13:29:17.984 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 3.3 MB/s | 3.3 MB 00:01 2024-04-25T13:29:17.992 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:29:17.992 INFO:teuthology.orchestra.run.smithi094.stdout:Total 15 MB/s | 214 MB 00:14 2024-04-25T13:29:18.022 INFO:teuthology.orchestra.run.smithi137.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T13:29:18.064 INFO:teuthology.orchestra.run.smithi137.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 846 kB/s | 35 kB 00:00 2024-04-25T13:29:18.206 INFO:teuthology.orchestra.run.smithi137.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-25T13:29:18.246 INFO:teuthology.orchestra.run.smithi138.stdout:(31/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 9.9 MB/s | 43 MB 00:04 2024-04-25T13:29:18.363 INFO:teuthology.orchestra.run.smithi138.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-04-25T13:29:18.388 INFO:teuthology.orchestra.run.smithi138.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.7 MB/s | 249 kB 00:00 2024-04-25T13:29:18.438 INFO:teuthology.orchestra.run.smithi138.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 658 kB/s | 49 kB 00:00 2024-04-25T13:29:18.463 INFO:teuthology.orchestra.run.smithi138.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 218 kB/s | 16 kB 00:00 2024-04-25T13:29:18.556 INFO:teuthology.orchestra.run.smithi138.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 273 kB 00:00 2024-04-25T13:29:18.615 INFO:teuthology.orchestra.run.smithi137.stdout:(66/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 4.8 MB/s | 43 MB 00:08 2024-04-25T13:29:18.615 INFO:teuthology.orchestra.run.smithi138.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.9 MB/s | 762 kB 00:00 2024-04-25T13:29:18.640 INFO:teuthology.orchestra.run.smithi138.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 217 kB/s | 18 kB 00:00 2024-04-25T13:29:18.681 INFO:teuthology.orchestra.run.smithi137.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-25T13:29:18.723 INFO:teuthology.orchestra.run.smithi137.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T13:29:18.757 INFO:teuthology.orchestra.run.smithi138.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-25T13:29:18.786 INFO:teuthology.orchestra.run.smithi137.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 911 kB 00:00 2024-04-25T13:29:18.807 INFO:teuthology.orchestra.run.smithi138.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T13:29:18.815 INFO:teuthology.orchestra.run.smithi137.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T13:29:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-25T13:29:18.866 INFO:teuthology.orchestra.run.smithi138.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T13:29:18.908 INFO:teuthology.orchestra.run.smithi137.stdout:(71/152): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-04-25T13:29:18.917 INFO:teuthology.orchestra.run.smithi138.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T13:29:18.958 INFO:teuthology.orchestra.run.smithi138.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T13:29:18.983 INFO:teuthology.orchestra.run.smithi137.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T13:29:18.992 INFO:teuthology.orchestra.run.smithi138.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-25T13:29:19.016 INFO:teuthology.orchestra.run.smithi137.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-04-25T13:29:19.017 INFO:teuthology.orchestra.run.smithi138.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T13:29:19.042 INFO:teuthology.orchestra.run.smithi137.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 669 kB/s | 87 kB 00:00 2024-04-25T13:29:19.043 INFO:teuthology.orchestra.run.smithi138.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T13:29:19.068 INFO:teuthology.orchestra.run.smithi138.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T13:29:19.089 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T13:29:19.110 INFO:teuthology.orchestra.run.smithi138.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T13:29:19.135 INFO:teuthology.orchestra.run.smithi138.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T13:29:19.150 INFO:teuthology.orchestra.run.smithi137.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-04-25T13:29:19.161 INFO:teuthology.orchestra.run.smithi138.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T13:29:19.176 INFO:teuthology.orchestra.run.smithi137.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T13:29:19.186 INFO:teuthology.orchestra.run.smithi138.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:29:19.200 INFO:teuthology.orchestra.run.smithi137.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T13:29:19.211 INFO:teuthology.orchestra.run.smithi138.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-04-25T13:29:19.226 INFO:teuthology.orchestra.run.smithi137.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-04-25T13:29:19.232 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T13:29:19.232 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T13:29:19.251 INFO:teuthology.orchestra.run.smithi137.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T13:29:19.276 INFO:teuthology.orchestra.run.smithi137.stdout:(80/152): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-04-25T13:29:19.295 INFO:teuthology.orchestra.run.smithi138.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T13:29:19.301 INFO:teuthology.orchestra.run.smithi137.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-25T13:29:19.320 INFO:teuthology.orchestra.run.smithi138.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 18 kB 00:00 2024-04-25T13:29:19.345 INFO:teuthology.orchestra.run.smithi138.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T13:29:19.371 INFO:teuthology.orchestra.run.smithi138.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T13:29:19.396 INFO:teuthology.orchestra.run.smithi138.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-25T13:29:19.410 INFO:teuthology.orchestra.run.smithi137.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T13:29:19.421 INFO:teuthology.orchestra.run.smithi138.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:29:19.435 INFO:teuthology.orchestra.run.smithi137.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 250 kB 00:00 2024-04-25T13:29:19.469 INFO:teuthology.orchestra.run.smithi137.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-25T13:29:19.494 INFO:teuthology.orchestra.run.smithi137.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 930 kB/s | 77 kB 00:00 2024-04-25T13:29:19.520 INFO:teuthology.orchestra.run.smithi137.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 849 kB/s | 42 kB 00:00 2024-04-25T13:29:19.545 INFO:teuthology.orchestra.run.smithi137.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T13:29:19.570 INFO:teuthology.orchestra.run.smithi137.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T13:29:19.595 INFO:teuthology.orchestra.run.smithi137.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T13:29:19.605 INFO:teuthology.orchestra.run.smithi138.stdout:(60/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.4 MB/s | 18 MB 00:07 2024-04-25T13:29:19.620 INFO:teuthology.orchestra.run.smithi137.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T13:29:19.631 INFO:teuthology.orchestra.run.smithi138.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.4 MB/s | 582 kB 00:00 2024-04-25T13:29:19.646 INFO:teuthology.orchestra.run.smithi137.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-04-25T13:29:19.658 INFO:teuthology.orchestra.run.smithi138.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 109 kB/s | 25 kB 00:00 2024-04-25T13:29:19.681 INFO:teuthology.orchestra.run.smithi138.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 952 kB/s | 47 kB 00:00 2024-04-25T13:29:19.706 INFO:teuthology.orchestra.run.smithi138.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T13:29:19.712 INFO:teuthology.orchestra.run.smithi137.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 653 kB/s | 42 kB 00:00 2024-04-25T13:29:19.731 INFO:teuthology.orchestra.run.smithi138.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T13:29:19.738 INFO:teuthology.orchestra.run.smithi137.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 496 kB/s | 44 kB 00:00 2024-04-25T13:29:19.756 INFO:teuthology.orchestra.run.smithi138.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T13:29:19.805 INFO:teuthology.orchestra.run.smithi137.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T13:29:19.855 INFO:teuthology.orchestra.run.smithi137.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 314 kB/s | 36 kB 00:00 2024-04-25T13:29:19.882 INFO:teuthology.orchestra.run.smithi138.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-25T13:29:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T13:29:20.024 INFO:teuthology.orchestra.run.smithi138.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 6.0 MB/s | 1.6 MB 00:00 2024-04-25T13:29:20.055 INFO:teuthology.orchestra.run.smithi137.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T13:29:20.057 INFO:teuthology.orchestra.run.smithi138.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.2 MB/s | 911 kB 00:00 2024-04-25T13:29:20.081 INFO:teuthology.orchestra.run.smithi137.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 173 kB/s | 47 kB 00:00 2024-04-25T13:29:20.083 INFO:teuthology.orchestra.run.smithi138.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T13:29:20.106 INFO:teuthology.orchestra.run.smithi137.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-25T13:29:20.131 INFO:teuthology.orchestra.run.smithi137.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 776 kB/s | 58 kB 00:00 2024-04-25T13:29:20.133 INFO:teuthology.orchestra.run.smithi138.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-25T13:29:20.156 INFO:teuthology.orchestra.run.smithi137.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T13:29:20.181 INFO:teuthology.orchestra.run.smithi137.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T13:29:20.207 INFO:teuthology.orchestra.run.smithi137.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-04-25T13:29:20.232 INFO:teuthology.orchestra.run.smithi137.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T13:29:20.242 INFO:teuthology.orchestra.run.smithi138.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-04-25T13:29:20.266 INFO:teuthology.orchestra.run.smithi137.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T13:29:20.267 INFO:teuthology.orchestra.run.smithi138.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-04-25T13:29:20.359 INFO:teuthology.orchestra.run.smithi138.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-04-25T13:29:20.384 INFO:teuthology.orchestra.run.smithi138.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T13:29:20.410 INFO:teuthology.orchestra.run.smithi138.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-25T13:29:20.426 INFO:teuthology.orchestra.run.smithi138.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 387 kB/s | 26 kB 00:00 2024-04-25T13:29:20.443 INFO:teuthology.orchestra.run.smithi138.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-04-25T13:29:20.466 INFO:teuthology.orchestra.run.smithi137.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-04-25T13:29:20.468 INFO:teuthology.orchestra.run.smithi138.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T13:29:20.485 INFO:teuthology.orchestra.run.smithi138.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T13:29:20.533 INFO:teuthology.orchestra.run.smithi137.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-25T13:29:20.558 INFO:teuthology.orchestra.run.smithi137.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-25T13:29:20.603 INFO:teuthology.orchestra.run.smithi138.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-04-25T13:29:20.628 INFO:teuthology.orchestra.run.smithi138.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-25T13:29:20.633 INFO:teuthology.orchestra.run.smithi137.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-25T13:29:20.653 INFO:teuthology.orchestra.run.smithi138.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-25T13:29:20.658 INFO:teuthology.orchestra.run.smithi137.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 490 kB/s | 49 kB 00:00 2024-04-25T13:29:20.678 INFO:teuthology.orchestra.run.smithi138.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T13:29:20.704 INFO:teuthology.orchestra.run.smithi138.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-25T13:29:20.725 INFO:teuthology.orchestra.run.smithi137.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T13:29:20.728 INFO:teuthology.orchestra.run.smithi138.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T13:29:20.750 INFO:teuthology.orchestra.run.smithi137.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 749 kB/s | 67 kB 00:00 2024-04-25T13:29:20.754 INFO:teuthology.orchestra.run.smithi138.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T13:29:20.768 INFO:teuthology.orchestra.run.smithi137.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T13:29:20.779 INFO:teuthology.orchestra.run.smithi138.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T13:29:20.801 INFO:teuthology.orchestra.run.smithi137.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-25T13:29:20.804 INFO:teuthology.orchestra.run.smithi138.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-04-25T13:29:20.830 INFO:teuthology.orchestra.run.smithi138.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 716 kB/s | 54 kB 00:00 2024-04-25T13:29:20.834 INFO:teuthology.orchestra.run.smithi137.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-25T13:29:20.852 INFO:teuthology.orchestra.run.smithi137.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 528 kB/s | 43 kB 00:00 2024-04-25T13:29:20.855 INFO:teuthology.orchestra.run.smithi138.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-25T13:29:20.868 INFO:teuthology.orchestra.run.smithi137.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-04-25T13:29:20.896 INFO:teuthology.orchestra.run.smithi137.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 246 kB/s | 15 kB 00:00 2024-04-25T13:29:20.897 INFO:teuthology.orchestra.run.smithi138.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 659 kB/s | 44 kB 00:00 2024-04-25T13:29:20.947 INFO:teuthology.orchestra.run.smithi137.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T13:29:20.947 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T13:29:20.947 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T13:29:20.947 INFO:teuthology.orchestra.run.smithi138.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T13:29:20.971 INFO:teuthology.orchestra.run.smithi137.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.4 MB/s | 358 kB 00:00 2024-04-25T13:29:20.997 INFO:teuthology.orchestra.run.smithi137.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-25T13:29:21.022 INFO:teuthology.orchestra.run.smithi137.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 475 kB/s | 35 kB 00:00 2024-04-25T13:29:21.048 INFO:teuthology.orchestra.run.smithi138.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 242 kB/s | 36 kB 00:00 2024-04-25T13:29:21.131 INFO:teuthology.orchestra.run.smithi138.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 260 kB/s | 47 kB 00:00 2024-04-25T13:29:21.290 INFO:teuthology.orchestra.run.smithi138.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T13:29:21.315 INFO:teuthology.orchestra.run.smithi138.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-25T13:29:21.340 INFO:teuthology.orchestra.run.smithi138.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 279 kB/s | 58 kB 00:00 2024-04-25T13:29:21.366 INFO:teuthology.orchestra.run.smithi138.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T13:29:21.391 INFO:teuthology.orchestra.run.smithi138.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-25T13:29:21.416 INFO:teuthology.orchestra.run.smithi138.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-04-25T13:29:21.441 INFO:teuthology.orchestra.run.smithi138.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T13:29:21.475 INFO:teuthology.orchestra.run.smithi138.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T13:29:21.514 INFO:teuthology.orchestra.run.smithi137.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 398 kB/s | 206 kB 00:00 2024-04-25T13:29:21.647 INFO:teuthology.orchestra.run.smithi137.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-04-25T13:29:21.697 INFO:teuthology.orchestra.run.smithi137.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 214 kB/s | 144 kB 00:00 2024-04-25T13:29:21.702 INFO:teuthology.orchestra.run.smithi138.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 86 kB/s | 19 kB 00:00 2024-04-25T13:29:21.723 INFO:teuthology.orchestra.run.smithi137.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 51 kB/s | 11 kB 00:00 2024-04-25T13:29:21.748 INFO:teuthology.orchestra.run.smithi137.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T13:29:21.773 INFO:teuthology.orchestra.run.smithi137.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-25T13:29:21.798 INFO:teuthology.orchestra.run.smithi137.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-25T13:29:21.809 INFO:teuthology.orchestra.run.smithi138.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 844 kB/s | 309 kB 00:00 2024-04-25T13:29:21.849 INFO:teuthology.orchestra.run.smithi137.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-25T13:29:21.909 INFO:teuthology.orchestra.run.smithi138.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T13:29:21.924 INFO:teuthology.orchestra.run.smithi137.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-04-25T13:29:21.949 INFO:teuthology.orchestra.run.smithi137.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 307 kB/s | 46 kB 00:00 2024-04-25T13:29:21.951 INFO:teuthology.orchestra.run.smithi138.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T13:29:21.975 INFO:teuthology.orchestra.run.smithi137.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 629 kB/s | 79 kB 00:00 2024-04-25T13:29:22.026 INFO:teuthology.orchestra.run.smithi138.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-25T13:29:22.083 INFO:teuthology.orchestra.run.smithi137.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 364 kB/s | 58 kB 00:00 2024-04-25T13:29:22.093 INFO:teuthology.orchestra.run.smithi138.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T13:29:22.108 INFO:teuthology.orchestra.run.smithi137.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-25T13:29:22.118 INFO:teuthology.orchestra.run.smithi138.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-04-25T13:29:22.134 INFO:teuthology.orchestra.run.smithi137.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 103 kB/s | 16 kB 00:00 2024-04-25T13:29:22.135 INFO:teuthology.orchestra.run.smithi138.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T13:29:22.159 INFO:teuthology.orchestra.run.smithi137.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T13:29:22.184 INFO:teuthology.orchestra.run.smithi137.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-04-25T13:29:22.194 INFO:teuthology.orchestra.run.smithi138.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-25T13:29:22.209 INFO:teuthology.orchestra.run.smithi137.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T13:29:22.219 INFO:teuthology.orchestra.run.smithi138.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-25T13:29:22.235 INFO:teuthology.orchestra.run.smithi137.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T13:29:22.244 INFO:teuthology.orchestra.run.smithi138.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-04-25T13:29:22.270 INFO:teuthology.orchestra.run.smithi138.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 431 kB/s | 32 kB 00:00 2024-04-25T13:29:22.293 INFO:teuthology.orchestra.run.smithi137.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 711 kB/s | 59 kB 00:00 2024-04-25T13:29:22.312 INFO:teuthology.orchestra.run.smithi138.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-25T13:29:22.318 INFO:teuthology.orchestra.run.smithi137.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-25T13:29:22.337 INFO:teuthology.orchestra.run.smithi138.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T13:29:22.343 INFO:teuthology.orchestra.run.smithi137.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-25T13:29:22.362 INFO:teuthology.orchestra.run.smithi138.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T13:29:22.369 INFO:teuthology.orchestra.run.smithi137.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T13:29:22.387 INFO:teuthology.orchestra.run.smithi138.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-25T13:29:22.394 INFO:teuthology.orchestra.run.smithi137.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T13:29:22.420 INFO:teuthology.orchestra.run.smithi137.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T13:29:22.438 INFO:teuthology.orchestra.run.smithi138.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 473 kB/s | 35 kB 00:00 2024-04-25T13:29:22.444 INFO:teuthology.orchestra.run.smithi137.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-25T13:29:22.463 INFO:teuthology.orchestra.run.smithi138.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T13:29:22.470 INFO:teuthology.orchestra.run.smithi137.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T13:29:22.530 INFO:teuthology.orchestra.run.smithi138.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T13:29:22.537 INFO:teuthology.orchestra.run.smithi137.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T13:29:22.555 INFO:teuthology.orchestra.run.smithi138.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T13:29:22.581 INFO:teuthology.orchestra.run.smithi138.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T13:29:22.604 INFO:teuthology.orchestra.run.smithi137.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T13:29:22.606 INFO:teuthology.orchestra.run.smithi138.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T13:29:22.623 INFO:teuthology.orchestra.run.smithi138.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 344 kB/s | 23 kB 00:00 2024-04-25T13:29:22.640 INFO:teuthology.orchestra.run.smithi138.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-04-25T13:29:22.665 INFO:teuthology.orchestra.run.smithi138.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-25T13:29:22.696 INFO:teuthology.orchestra.run.smithi137.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T13:29:22.707 INFO:teuthology.orchestra.run.smithi138.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T13:29:22.724 INFO:teuthology.orchestra.run.smithi138.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-25T13:29:22.757 INFO:teuthology.orchestra.run.smithi138.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 862 kB/s | 79 kB 00:00 2024-04-25T13:29:22.783 INFO:teuthology.orchestra.run.smithi138.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-25T13:29:22.824 INFO:teuthology.orchestra.run.smithi138.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T13:29:22.841 INFO:teuthology.orchestra.run.smithi138.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-25T13:29:22.858 INFO:teuthology.orchestra.run.smithi138.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T13:29:22.892 INFO:teuthology.orchestra.run.smithi138.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-04-25T13:29:22.917 INFO:teuthology.orchestra.run.smithi138.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-04-25T13:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T13:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 780 kB/s | 59 kB 00:00 2024-04-25T13:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-25T13:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-25T13:29:23.053 INFO:teuthology.orchestra.run.smithi138.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T13:29:23.078 INFO:teuthology.orchestra.run.smithi138.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-25T13:29:23.113 INFO:teuthology.orchestra.run.smithi137.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-25T13:29:23.157 INFO:teuthology.orchestra.run.smithi138.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-04-25T13:29:23.203 INFO:teuthology.orchestra.run.smithi138.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-04-25T13:29:23.228 INFO:teuthology.orchestra.run.smithi138.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-25T13:29:23.279 INFO:teuthology.orchestra.run.smithi138.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T13:29:23.346 INFO:teuthology.orchestra.run.smithi138.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T13:29:23.437 INFO:teuthology.orchestra.run.smithi138.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T13:29:23.805 INFO:teuthology.orchestra.run.smithi138.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 5.8 MB/s | 3.3 MB 00:00 2024-04-25T13:29:23.921 INFO:teuthology.orchestra.run.smithi138.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-25T13:29:23.929 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:29:23.929 INFO:teuthology.orchestra.run.smithi138.stdout:Total 14 MB/s | 214 MB 00:15 2024-04-25T13:29:25.057 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T13:29:25.198 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T13:29:25.198 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T13:29:25.854 INFO:teuthology.orchestra.run.smithi137.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 998 kB/s | 3.3 MB 00:03 2024-04-25T13:29:25.862 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:29:25.863 INFO:teuthology.orchestra.run.smithi137.stdout:Total 9.8 MB/s | 214 MB 00:21 2024-04-25T13:29:26.907 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T13:29:26.907 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T13:29:26.982 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-04-25T13:29:27.124 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-04-25T13:29:27.124 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-04-25T13:29:27.129 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T13:29:27.222 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:29:27.269 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:29:27.323 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:29:27.366 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:29:27.391 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:29:27.434 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:29:27.771 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:29:27.884 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:28.009 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:28.036 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:28.126 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:28.157 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:29:28.307 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:29:28.371 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:29:28.399 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:28.504 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:28.533 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:28.596 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:28.618 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:28.644 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:28.835 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:28.860 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-04-25T13:29:28.860 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-04-25T13:29:28.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:28.984 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:29:29.080 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:29:29.109 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:29:29.144 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:29:29.209 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:29:29.252 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:29:29.305 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:29:29.337 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:29:29.365 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:29:29.435 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:29:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:29:29.519 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:29:29.563 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:29:29.640 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:29:29.682 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:29:29.726 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:29:29.754 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:29:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:29:29.869 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:29:29.944 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:29:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:29:30.139 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:29:30.674 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:29:30.752 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:29:30.781 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:29:30.824 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:29:30.862 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:29:30.903 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:29:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:29:31.132 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:29:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:29:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:29:31.300 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:29:31.337 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:29:31.384 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:29:31.419 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:29:31.463 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:29:31.505 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:29:31.641 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:29:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:29:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:29:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:29:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:29:31.958 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:29:31.990 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:29:32.016 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:29:32.049 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:29:32.123 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:29:32.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:29:32.291 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:29:32.319 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:29:32.353 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:29:32.392 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:29:32.424 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:29:32.596 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:29:32.782 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:29:32.815 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:29:32.819 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T13:29:32.858 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:29:32.891 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:29:32.908 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:29:32.946 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:29:32.982 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:29:33.000 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:29:33.030 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:29:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:29:33.060 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:29:33.061 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:29:33.102 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:29:33.109 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:29:33.217 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:29:33.271 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:29:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:29:33.561 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:33.685 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:33.712 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:33.803 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:33.841 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:29:33.986 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:29:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:29:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:29:34.046 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:29:34.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:29:34.171 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:34.174 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:29:34.201 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:34.236 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:29:34.255 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:34.276 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:34.298 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:34.323 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:29:34.397 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:29:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:34.600 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:29:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:29:34.728 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:29:34.729 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:29:34.767 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:29:34.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:29:34.806 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:29:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:29:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:29:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:29:35.084 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:29:35.098 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-04-25T13:29:35.145 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:29:35.174 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:29:35.183 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:29:35.213 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:29:35.227 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:29:35.266 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:29:35.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:29:35.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:29:35.326 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:29:35.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:29:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:29:35.479 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:29:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:29:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:29:35.608 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:29:35.694 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:29:35.742 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:29:35.794 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:29:35.810 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:35.935 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:29:35.961 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:36.052 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:29:36.090 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:29:36.240 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:29:36.287 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:29:36.315 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:36.316 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:29:36.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:29:36.419 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:29:36.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:29:36.441 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:36.470 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:29:36.487 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:29:36.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:29:36.509 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:36.532 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:29:36.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:29:36.683 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:36.709 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:29:36.725 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:29:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:29:36.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:29:36.832 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:29:36.840 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:29:36.888 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:29:36.919 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:29:36.920 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:29:36.927 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:29:36.948 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:29:36.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:29:36.957 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:29:36.981 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:29:36.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:29:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:29:37.039 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:29:37.047 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:29:37.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:29:37.082 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:29:37.115 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:29:37.126 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:29:37.159 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:29:37.186 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:29:37.251 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:29:37.254 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:29:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:29:37.302 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:29:37.316 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:29:37.326 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:29:37.347 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:29:37.368 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:29:37.378 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:37.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:29:37.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:29:37.462 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:29:37.469 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:29:37.499 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:29:37.502 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:29:37.533 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:29:37.535 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:29:37.536 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:29:37.559 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:29:37.566 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:29:37.583 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:29:37.636 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:29:37.658 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:29:37.705 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:29:37.785 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:29:37.788 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:29:37.816 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:29:37.844 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:29:37.878 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:29:37.917 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:29:37.930 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:29:37.949 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:29:37.979 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:29:37.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:29:38.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:29:38.025 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:29:38.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:29:38.065 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:38.088 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:29:38.123 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:29:38.136 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:29:38.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:29:38.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:29:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:29:38.331 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:29:38.429 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:29:38.513 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:29:38.569 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:29:38.595 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:29:38.632 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:29:38.675 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:29:38.714 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:29:38.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:29:38.917 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:29:38.967 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:29:39.020 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:29:39.094 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:29:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:29:39.161 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:29:39.208 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:29:39.244 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:29:39.287 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:29:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:29:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:29:39.463 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:29:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:29:39.517 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:29:39.557 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:29:39.597 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:29:39.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:29:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:29:39.690 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:29:39.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:29:39.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:29:39.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:29:39.777 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:29:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:29:39.813 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:29:39.887 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:29:39.954 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:29:39.984 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:29:40.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:29:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:29:40.069 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:29:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:29:40.116 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:29:40.155 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:29:40.186 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:29:40.216 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:29:40.251 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:29:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:29:40.325 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:29:40.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:29:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:42.629 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:42.659 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:29:42.696 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:29:42.724 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:29:42.843 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:29:43.183 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:29:43.208 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:43.310 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:29:44.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:29:44.149 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:29:44.177 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:29:44.211 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:29:44.531 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:29:44.573 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:44.665 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:29:44.694 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:29:44.730 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:29:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:29:45.259 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:45.313 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:29:45.335 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:29:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:46.204 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:46.243 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:46.334 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:46.334 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:29:46.334 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:29:46.334 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:29:46.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:47.547 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:48.146 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:29:48.185 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:48.274 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:29:48.274 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:29:48.274 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:29:48.275 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:29:48.312 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-25T13:29:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-25T13:29:52.748 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-25T13:29:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:29:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:29:52.948 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:52.948 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:29:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:54.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:29:54.570 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:29:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:29:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:29:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:29:54.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:29:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:29:54.818 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:54.818 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:29:54.818 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:29:54.819 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:29:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:55.279 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:29:55.303 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:29:55.303 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:55.303 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:29:55.304 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:29:55.304 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:29:55.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:55.409 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:29:55.459 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:29:55.514 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:29:55.700 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:29:55.756 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:29:55.833 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:29:55.922 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:29:56.062 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:29:56.115 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:29:56.148 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:29:56.175 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:29:56.620 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:29:56.650 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:29:56.681 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:29:56.742 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:29:56.781 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:29:57.814 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:29:57.850 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-25T13:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-25T13:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:29:58.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:29:58.458 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:29:58.459 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:58.459 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:29:58.459 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:29:58.818 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:29:58.842 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:29:58.942 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:29:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:29:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:29:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:29:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:29:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:29:59.946 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-04-25T13:29:59.947 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:00.054 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:30:00.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:00.117 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:30:00.149 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:30:00.149 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.149 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:30:00.149 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:00.321 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:30:00.328 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:30:00.345 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:30:00.345 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.345 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:30:00.345 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:30:00.346 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:30:00.346 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:00.357 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:00.391 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:00.392 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.392 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:30:00.392 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:00.392 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:00.392 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:00.433 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:00.456 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:00.456 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.456 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:30:00.456 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:00.814 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:30:00.837 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:30:00.837 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.837 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:30:00.837 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:30:00.837 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:30:00.838 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:00.843 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:00.866 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:00.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:30:01.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:30:01.056 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:30:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:30:01.290 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:30:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:30:01.456 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:30:01.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:30:01.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:30:01.665 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:30:01.692 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:30:01.745 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:30:01.772 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:30:01.772 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:01.772 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:30:01.772 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:30:01.773 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:30:01.773 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:01.995 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:30:02.018 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:30:02.018 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:30:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:30:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:30:02.019 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:02.160 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:30:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:30:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:30:02.284 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:30:02.314 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:30:02.488 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:30:02.511 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:02.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:30:02.666 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:30:02.730 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:30:02.917 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:30:02.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:30:03.040 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:30:03.131 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:30:03.272 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:30:03.313 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:30:03.346 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:30:03.351 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:30:03.373 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:30:03.390 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:30:03.824 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:30:03.854 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:30:03.887 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:30:03.949 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:30:03.986 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:30:04.357 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:30:04.391 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:30:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:30:04.667 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:30:04.693 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:30:04.716 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:30:04.716 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:04.716 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:30:04.717 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:30:04.717 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:30:04.717 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:04.761 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:30:04.772 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:30:04.870 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:30:04.917 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:30:05.031 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:30:05.071 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:30:05.134 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:30:05.174 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:30:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:30:05.247 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:30:05.274 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:30:05.461 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:30:05.512 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:30:05.542 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:30:05.851 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:30:05.897 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:05.920 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:05.964 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:05.986 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:05.986 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:05.987 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:30:05.987 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:06.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:06.043 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:06.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:06.072 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:30:06.155 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:06.174 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:30:06.197 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:06.217 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:06.363 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:30:06.365 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:06.388 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:06.390 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:30:06.414 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:06.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:30:06.469 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:30:06.570 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:30:06.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:30:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:30:07.135 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:30:07.175 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:30:07.256 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:30:07.289 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:30:07.289 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:07.373 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:07.559 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:30:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:07.594 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:30:07.617 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:30:07.692 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:07.692 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:30:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:30:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:08.079 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:08.102 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:30:08.103 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:30:08.103 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:30:08.103 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:08.103 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:30:08.103 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:10.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:30:10.710 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:30:10.751 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:30:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:30:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:30:10.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:30:11.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:30:11.070 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:30:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:30:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:30:12.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:30:12.533 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:30:12.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:30:12.718 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:30:12.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:30:12.773 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:30:12.792 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:30:12.812 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:30:12.812 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:12.896 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:13.021 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:13.086 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:13.393 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:30:13.684 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:30:13.685 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:30:13.686 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:30:13.687 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:30:13.688 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:30:13.689 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:30:13.690 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:30:13.691 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:30:13.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:30:13.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:30:13.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:30:13.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:30:13.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:30:13.693 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:30:13.694 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:30:13.695 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:30:13.696 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:13.697 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:30:14.296 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:30:14.377 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:30:14.418 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:30:14.501 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:30:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:30:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:14.633 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:30:14.665 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:14.666 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:14.758 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:30:14.822 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.792 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.793 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.794 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:30:15.795 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:30:15.796 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:30:15.799 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:30:15.800 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:30:15.801 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:30:15.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:30:15.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:30:15.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:30:15.806 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:30:15.807 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:30:15.808 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:30:15.808 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:30:15.808 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T13:30:16.212 DEBUG:teuthology.parallel:result is None 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:30:19.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:30:19.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:30:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:30:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:30:19.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:30:19.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:30:19.393 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:30:19.394 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:30:19.395 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:30:19.396 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:30:19.397 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:30:19.398 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:19.399 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:30:20.753 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:20.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:30:20.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:30:20.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:30:20.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:30:20.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:30:20.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:30:20.767 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:30:20.768 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:30:20.769 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:30:20.770 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:30:20.771 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:30:20.772 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:30:20.773 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:30:20.774 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:30:20.775 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:30:20.776 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:21.489 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.490 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.491 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:30:21.492 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:30:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:30:21.494 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:30:21.495 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:30:21.496 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:30:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:21.498 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:30:21.499 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:30:21.500 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.501 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:30:21.502 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:30:21.503 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:30:21.503 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T13:30:21.900 DEBUG:teuthology.parallel:result is None 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:22.878 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:30:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:30:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:30:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:30:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:30:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:30:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:30:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:30:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:30:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:30:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:30:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:30:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:30:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:30:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T13:30:22.891 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-04-25T13:30:23.249 DEBUG:teuthology.parallel:result is None 2024-04-25T13:30:23.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:23.389 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:30:23.414 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.415 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.415 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:30:23.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:23.645 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:30:23.666 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.666 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.666 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:30:23.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:23.890 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:30:23.911 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.911 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:30:23.911 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:30:23.913 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:30:23.913 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:30:23.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:30:23.945 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:30:23.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:30:23.971 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:30:23.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:30:23.997 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:30:23.997 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:30:23.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:30:24.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:30:24.090 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:30:24.090 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:30:24.114 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:30:24.179 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:30:24.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:30:24.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:30:24.267 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:30:24.268 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:30:24.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:30:24.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:30:24.362 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:30:24.363 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:30:24.387 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:30:24.452 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:30:24.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:30:24.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:30:24.538 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:30:24.539 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:30:24.539 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:30:24.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:30:24.631 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:30:24.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:30:24.654 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:30:24.718 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:30:24.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:30:24.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:30:24.806 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T13:30:24.908 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:30:24.908 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:24.909 INFO:tasks.cephadm:Cluster fsid is f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:30:24.909 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T13:30:24.909 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.138'} 2024-04-25T13:30:24.909 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-04-25T13:30:24.909 INFO:tasks.cephadm:First mgr is x 2024-04-25T13:30:24.909 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T13:30:24.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-25T13:30:24.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-04-25T13:30:24.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-04-25T13:30:24.984 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T13:30:24.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:25.238 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T13:30:25.480 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:30:25.482 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T13:30:25.482 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:30 /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.009 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.583 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:30 /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.583 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:30 /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.979 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:26.997 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:27.015 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:30:27.050 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T13:30:27.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:30:27.052 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:30:27.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:30:28.665 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:30:28.702 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:30:28.734 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:31:00.858 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T13:31:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:31:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:31:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-04-25T13:31:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:31:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-25T13:31:00.859 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:31:20.760 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T13:31:20.761 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:31:23.856 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-25T13:31:23.857 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T13:31:23.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-25T13:31:23.913 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-04-25T13:31:23.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-04-25T13:31:23.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-25T13:31:23.998 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-04-25T13:31:24.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-04-25T13:31:24.049 INFO:tasks.cephadm:Writing seed config... 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T13:31:24.051 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T13:31:24.052 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T13:31:24.053 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T13:31:24.054 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T13:31:24.054 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T13:31:24.054 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T13:31:24.054 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T13:31:24.054 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T13:31:24.055 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:31:24.055 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T13:31:24.071 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f9021454-0307-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T13:31:24.072 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T13:31:24.114 DEBUG:teuthology.orchestra.run.smithi094:mgr.x> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T13:31:24.157 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T13:31:24.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid f9021454-0307-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:31:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:31:24.347 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', 'f9021454-0307-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-04-25T13:31:24.348 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T13:31:24.348 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-04-25T13:31:24.382 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:31:24.383 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-04-25T13:31:24.383 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-04-25T13:31:24.390 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:31:24.390 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:31:24.395 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:31:24.396 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:31:24.402 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-25T13:31:24.407 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-25T13:31:24.408 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:31:24.408 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-04-25T13:31:24.431 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:31:24.431 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T13:31:24.431 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-04-25T13:31:24.431 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-04-25T13:31:24.437 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:31:24.438 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:31:24.443 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:31:24.443 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:31:24.449 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 139985936231584 on /run/cephadm/f9021454-0307-11ef-bc93-c7b262605968.lock 2024-04-25T13:31:24.456 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 139985936231584 acquired on /run/cephadm/f9021454-0307-11ef-bc93-c7b262605968.lock 2024-04-25T13:31:24.457 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-04-25T13:31:24.457 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-04-25T13:31:24.457 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-25T13:31:24.460 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-04-25T13:31:24.460 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-04-25T13:31:24.462 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T13:31:24.463 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T13:31:24.463 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T13:31:24.465 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::9d70:8b0e:721a:ae75/64 scope link noprefixroute 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-25T13:31:24.466 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T13:31:24.467 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T13:31:24.467 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T13:31:24.955 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T13:31:29.685 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T13:31:29.685 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T13:31:29.685 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T13:31:39.230 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-04-25T13:31:39.231 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-04-25T13:31:40.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQA8WypmXNf/EBAA7QAv/OJbtFtVE/pxDtunvA== 2024-04-25T13:31:41.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQA9Wypmpy+OBBAAQEzwxcSvgvuFnWBJ43JO5A== 2024-04-25T13:31:42.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQA+Wypmak8pAhAAF9lW4SN04yBidzq9NQqzzg== 2024-04-25T13:31:42.586 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-04-25T13:31:43.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:31:43.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T13:31:43.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:43.507 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:31:43.508 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-04-25T13:31:44.422 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-04-25T13:31:44.617 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T13:31:44.797 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T13:31:44.979 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f9021454-0307-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968.target. 2024-04-25T13:31:44.979 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f9021454-0307-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968.target. 2024-04-25T13:31:45.259 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:31:45.260 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T13:31:45.507 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968.target.wants/ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968@.service. 2024-04-25T13:31:45.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:45 smithi094 podman[28799]: 2024-04-25 13:31:45.65670807 +0000 UTC m=+0.017702537 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:31:46.035 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:31:46.035 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T13:31:46.043 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:31:46.043 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:31:46.043 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T13:31:46.043 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:31:46.043 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-04-25T13:31:46.044 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-04-25T13:31:46.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:45 smithi094 podman[28799]: 2024-04-25 13:31:45.938692748 +0000 UTC m=+0.299687215 container init 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:31:46.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:45 smithi094 podman[28799]: 2024-04-25 13:31:45.941477447 +0000 UTC m=+0.302471915 container start 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T13:31:46.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:45 smithi094 ceph-mon[28818]: mkfs f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:46.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:45 smithi094 ceph-mon[28818]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:31:46.166 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 bash[28799]: 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee 2024-04-25T13:31:46.166 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 systemd[1]: Started Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.569585s) 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T13:31:47.114 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.482+0000 7fd446023700 1 Processor -- start 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.482+0000 7fd446023700 1 -- start start 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.482+0000 7fd446023700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.482+0000 7fd446023700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd440077000 con 0x7fd44010a6d0 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.483+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.483+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53146/0 (socket says 172.21.15.94:53146) 2024-04-25T13:31:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.483+0000 7fd43f7fe700 1 -- 172.21.15.94:0/1191354207 learned_addr learned my addr 172.21.15.94:0/1191354207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.483+0000 7fd43f7fe700 1 -- 172.21.15.94:0/1191354207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44010d030 con 0x7fd44010a6d0 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.483+0000 7fd43f7fe700 1 --2- 172.21.15.94:0/1191354207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098a0 tx=0x7fd430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ddd40d37a2ebe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd43effd700 1 -- 172.21.15.94:0/1191354207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd430004160 con 0x7fd44010a6d0 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd43effd700 1 -- 172.21.15.94:0/1191354207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd430004300 con 0x7fd44010a6d0 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd43effd700 1 -- 172.21.15.94:0/1191354207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4300036a0 con 0x7fd44010a6d0 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 -- 172.21.15.94:0/1191354207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 msgr2=0x7fd44010caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 --2- 172.21.15.94:0/1191354207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd4300098a0 tx=0x7fd430009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 -- 172.21.15.94:0/1191354207 shutdown_connections 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 --2- 172.21.15.94:0/1191354207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44010caf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 -- 172.21.15.94:0/1191354207 >> 172.21.15.94:0/1191354207 conn(0x7fd4401040d0 msgr2=0x7fd4401065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 -- 172.21.15.94:0/1191354207 shutdown_connections 2024-04-25T13:31:47.116 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.484+0000 7fd446023700 1 -- 172.21.15.94:0/1191354207 wait complete. 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.485+0000 7fd446023700 1 Processor -- start 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.485+0000 7fd446023700 1 -- start start 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.485+0000 7fd446023700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.485+0000 7fd446023700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010d440 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53160/0 (socket says 172.21.15.94:53160) 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43f7fe700 1 -- 172.21.15.94:0/2398706524 learned_addr learned my addr 172.21.15.94:0/2398706524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43f7fe700 1 -- 172.21.15.94:0/2398706524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44019db90 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43f7fe700 1 --2- 172.21.15.94:0/2398706524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd430000c00 tx=0x7fd430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd430003db0 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd430011630 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44019de20 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd430011950 con 0x7fd44010a6d0 2024-04-25T13:31:47.117 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a0a30 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd43001a460 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.486+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd43001b780 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.487+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd440064cf0 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.490+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd430031020 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.560+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd44002e8d0 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.560+0000 7fd43d7fa700 1 -- 172.21.15.94:0/2398706524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd43001b980 con 0x7fd44010a6d0 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.560+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 msgr2=0x7fd44019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.560+0000 7fd446023700 1 --2- 172.21.15.94:0/2398706524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd430000c00 tx=0x7fd430004750 comp rx=0 tx=0).stop 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.561+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 shutdown_connections 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.561+0000 7fd446023700 1 --2- 172.21.15.94:0/2398706524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd44010a6d0 0x7fd44019d650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.561+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 >> 172.21.15.94:0/2398706524 conn(0x7fd4401040d0 msgr2=0x7fd4401062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.561+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 shutdown_connections 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:46.561+0000 7fd446023700 1 -- 172.21.15.94:0/2398706524 wait complete. 2024-04-25T13:31:47.118 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-04-25T13:31:47.119 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: monmap epoch 1 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: last_changed 2024-04-25T13:31:42.925512+0000 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: min_mon_release 19 (squid) 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: election_strategy: 1 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: fsmap 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:31:47.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: mgrmap e1: no daemons active 2024-04-25T13:31:47.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:46 smithi094 ceph-mon[28818]: from='client.? 172.21.15.94:0/2398706524' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T13:31:48.182 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:47 smithi094 ceph-mon[28818]: from='client.? 172.21.15.94:0/429380357' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:31:48.182 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:47 smithi094 ceph-mon[28818]: from='client.? 172.21.15.94:0/429380357' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:31:48.190 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.522+0000 7f32b91f6700 1 Processor -- start 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.522+0000 7f32b91f6700 1 -- start start 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.522+0000 7f32b91f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.522+0000 7f32b91f6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410a590 con 0x7f32b4109b90 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53172/0 (socket says 172.21.15.94:53172) 2024-04-25T13:31:48.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b2d9d700 1 -- 172.21.15.94:0/97365988 learned_addr learned my addr 172.21.15.94:0/97365988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b2d9d700 1 -- 172.21.15.94:0/97365988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b410ad60 con 0x7f32b4109b90 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b2d9d700 1 --2- 172.21.15.94:0/97365988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f32a80098a0 tx=0x7f32a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba6b2ea8c166e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b1d9b700 1 -- 172.21.15.94:0/97365988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a8004160 con 0x7f32b4109b90 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b1d9b700 1 -- 172.21.15.94:0/97365988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a8004300 con 0x7f32b4109b90 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.523+0000 7f32b1d9b700 1 -- 172.21.15.94:0/97365988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a80036a0 con 0x7f32b4109b90 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 -- 172.21.15.94:0/97365988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 msgr2=0x7f32b4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 --2- 172.21.15.94:0/97365988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f32a80098a0 tx=0x7f32a8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 -- 172.21.15.94:0/97365988 shutdown_connections 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 --2- 172.21.15.94:0/97365988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b4109fa0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 -- 172.21.15.94:0/97365988 >> 172.21.15.94:0/97365988 conn(0x7f32b4105210 msgr2=0x7f32b4107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 -- 172.21.15.94:0/97365988 shutdown_connections 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.524+0000 7f32b91f6700 1 -- 172.21.15.94:0/97365988 wait complete. 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b91f6700 1 Processor -- start 2024-04-25T13:31:48.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b91f6700 1 -- start start 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b91f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b91f6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410b290 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53186/0 (socket says 172.21.15.94:53186) 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.525+0000 7f32b2d9d700 1 -- 172.21.15.94:0/429380357 learned_addr learned my addr 172.21.15.94:0/429380357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f32b2d9d700 1 -- 172.21.15.94:0/429380357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b419db70 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f32b2d9d700 1 --2- 172.21.15.94:0/429380357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a8003db0 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a8011630 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a8011950 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b419de00 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41a0a10 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32a8011af0 con 0x7f32b4109b90 2024-04-25T13:31:48.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.526+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32a801c450 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.527+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b4064cf0 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.529+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f32a8032020 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.596+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f32b402e810 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.600+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f32a8004260 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f329bfff700 1 -- 172.21.15.94:0/429380357 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f32a801c6a0 con 0x7f32b4109b90 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 msgr2=0x7f32b419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 --2- 172.21.15.94:0/429380357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8004750 comp rx=0 tx=0).stop 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 shutdown_connections 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 --2- 172.21.15.94:0/429380357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32b4109b90 0x7f32b419d630 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 >> 172.21.15.94:0/429380357 conn(0x7f32b4105210 msgr2=0x7f32b4193d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 shutdown_connections 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:47.601+0000 7f32b91f6700 1 -- 172.21.15.94:0/429380357 wait complete. 2024-04-25T13:31:48.194 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-04-25T13:31:49.280 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:48 smithi094 ceph-mon[28818]: from='client.? 172.21.15.94:0/3377067247' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:31:49.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f98995700 1 Processor -- start 2024-04-25T13:31:49.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f98995700 1 -- start start 2024-04-25T13:31:49.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f98995700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:49.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f98995700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9407cdb0 con 0x7f6f9410a740 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53196/0 (socket says 172.21.15.94:53196) 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f9259c700 1 -- 172.21.15.94:0/3701706323 learned_addr learned my addr 172.21.15.94:0/3701706323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.633+0000 7f6f9259c700 1 -- 172.21.15.94:0/3701706323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9410d0a0 con 0x7f6f9410a740 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f9259c700 1 --2- 172.21.15.94:0/3701706323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f840098a0 tx=0x7f6f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dd51809cb012d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f9159a700 1 -- 172.21.15.94:0/3701706323 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f84004160 con 0x7f6f9410a740 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f9159a700 1 -- 172.21.15.94:0/3701706323 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f84004300 con 0x7f6f9410a740 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f9159a700 1 -- 172.21.15.94:0/3701706323 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f84011af0 con 0x7f6f9410a740 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f98995700 1 -- 172.21.15.94:0/3701706323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 msgr2=0x7f6f9410cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.634+0000 7f6f98995700 1 --2- 172.21.15.94:0/3701706323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6f840098a0 tx=0x7f6f84009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 -- 172.21.15.94:0/3701706323 shutdown_connections 2024-04-25T13:31:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 --2- 172.21.15.94:0/3701706323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9410cb60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 -- 172.21.15.94:0/3701706323 >> 172.21.15.94:0/3701706323 conn(0x7f6f94104180 msgr2=0x7f6f941065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 -- 172.21.15.94:0/3701706323 shutdown_connections 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 -- 172.21.15.94:0/3701706323 wait complete. 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 Processor -- start 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.635+0000 7f6f98995700 1 -- start start 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f98995700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f98995700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9410dec0 con 0x7f6f9410a740 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53198/0 (socket says 172.21.15.94:53198) 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f9259c700 1 -- 172.21.15.94:0/3377067247 learned_addr learned my addr 172.21.15.94:0/3377067247 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f9259c700 1 -- 172.21.15.94:0/3377067247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9419e400 con 0x7f6f9410a740 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f9259c700 1 --2- 172.21.15.94:0/3377067247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009fb0 tx=0x7f6f84011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f84011bc0 con 0x7f6f9410a740 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f84024440 con 0x7f6f9410a740 2024-04-25T13:31:49.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8401a5c0 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9419e690 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.636+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941a12a0 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.637+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6f8401ac10 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.637+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f84019700 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.637+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f94064cf0 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.640+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6f8402dde0 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.707+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6f941a1470 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f837fe700 1 -- 172.21.15.94:0/3377067247 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6f840245f0 con 0x7f6f9410a740 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 msgr2=0x7f6f9419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 --2- 172.21.15.94:0/3377067247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009fb0 tx=0x7f6f84011890 comp rx=0 tx=0).stop 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 shutdown_connections 2024-04-25T13:31:49.290 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 --2- 172.21.15.94:0/3377067247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f9410a740 0x7f6f9419dec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:49.291 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 >> 172.21.15.94:0/3377067247 conn(0x7f6f94104180 msgr2=0x7f6f94105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:49.291 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 shutdown_connections 2024-04-25T13:31:49.291 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:48.708+0000 7f6f98995700 1 -- 172.21.15.94:0/3377067247 wait complete. 2024-04-25T13:31:49.291 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-04-25T13:31:49.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:49 smithi094 systemd[1]: Stopping Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:31:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:49 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a[28814]: 2024-04-25T13:31:49.565+0000 7fa437945700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:49.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:49 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a[28814]: 2024-04-25T13:31:49.565+0000 7fa437945700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T13:31:49.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:49 smithi094 podman[29066]: 2024-04-25 13:31:49.771053166 +0000 UTC m=+0.313877209 container died 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T13:31:50.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:50 smithi094 podman[29066]: 2024-04-25 13:31:50.107664559 +0000 UTC m=+0.650488603 container cleanup 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:31:50.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:50 smithi094 bash[29066]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:31:51.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:50 smithi094 podman[29080]: 2024-04-25 13:31:50.783917331 +0000 UTC m=+1.008247229 container remove 44a05c448864f20caaa494d80420f6aa7a2df42378f21f9bd4046a1461dc5eee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:31:51.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T13:31:51.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 systemd[1]: Stopped Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:31:51.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 systemd[1]: Starting Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:31:51.532 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 podman[29174]: 2024-04-25 13:31:51.153160189 +0000 UTC m=+0.018140584 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:31:51.532 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 podman[29174]: 2024-04-25 13:31:51.285188371 +0000 UTC m=+0.150168757 container create 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T13:31:51.532 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 podman[29174]: 2024-04-25 13:31:51.49599859 +0000 UTC m=+0.360978971 container init 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T13:31:51.532 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 podman[29174]: 2024-04-25 13:31:51.498916037 +0000 UTC m=+0.363896419 container start 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:31:51.584 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T13:31:51.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: pidfile_write: ignore empty --pid-file 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: load: jerasure load: lrc 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Git sha 0 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: DB SUMMARY 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: DB Session ID: XG4YZLN7WKKQOYQVFV1C 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T13:31:51.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105020 ; 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.env: 0x556324141500 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.info_log: 0x556325069fc0 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:31:51.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.statistics: (nil) 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.use_fsync: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:31:51.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:31:51.810 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:31:51.810 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.db_log_dir: 2024-04-25T13:31:51.810 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.wal_dir: 2024-04-25T13:31:51.810 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.write_buffer_manager: 0x55632503a8c0 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:31:51.812 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.unordered_write: 0 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.row_cache: None 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.wal_filter: None 2024-04-25T13:31:51.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.wal_compression: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:31:51.814 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:31:51.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_open_files: -1 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Compression algorithms supported: 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kZSTD supported: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:31:51.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.merge_operator: 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_filter: None 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:31:51.817 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5563251701a0) 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x55632506d1f0 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:31:51.818 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-04-25T13:31:51.819 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:31:51.820 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression: NoCompression 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.num_levels: 7 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:31:51.822 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:31:51.823 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:31:51.824 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:31:51.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.ttl: 2592000 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:31:51.826 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6798e131-4510-4adf-8768-6d37f1f3f37b 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714051911538363, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714051911539501, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100197, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98081, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90364, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714051911, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6798e131-4510-4adf-8768-6d37f1f3f37b", "db_session_id": "XG4YZLN7WKKQOYQVFV1C", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714051911539582, "job": 1, "event": "recovery_finished"} 2024-04-25T13:31:51.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556325051c00 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: DB pointer 0x556329e44000 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:31:51.828 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 99.71 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 99.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:31:51.829 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 30.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 30.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x55632506d1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:31:51.830 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???) e1 preinit fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).mds e1 new map 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).mds e1 print_map 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:31:51.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 bash[29174]: 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 systemd[1]: Started Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: monmap epoch 1 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: last_changed 2024-04-25T13:31:42.925512+0000 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: election_strategy: 1 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: fsmap 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:31:51.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:51 smithi094 ceph-mon[29195]: mgrmap e1: no daemons active 2024-04-25T13:31:52.728 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.069+0000 7f6c22977700 1 Processor -- start 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.069+0000 7f6c22977700 1 -- start start 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.069+0000 7f6c22977700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.069+0000 7f6c22977700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c07d7d0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53212/0 (socket says 172.21.15.94:53212) 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1bfff700 1 -- 172.21.15.94:0/2556188923 learned_addr learned my addr 172.21.15.94:0/2556188923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1bfff700 1 -- 172.21.15.94:0/2556188923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c07d950 con 0x7f6c1c07edc0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1bfff700 1 --2- 172.21.15.94:0/2556188923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f293e8dbd1c119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1affd700 1 -- 172.21.15.94:0/2556188923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c004160 con 0x7f6c1c07edc0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1affd700 1 -- 172.21.15.94:0/2556188923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c0c004300 con 0x7f6c1c07edc0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.070+0000 7f6c1affd700 1 -- 172.21.15.94:0/2556188923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c011af0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 -- 172.21.15.94:0/2556188923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 msgr2=0x7f6c1c07d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:52.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 --2- 172.21.15.94:0/2556188923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 -- 172.21.15.94:0/2556188923 shutdown_connections 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 --2- 172.21.15.94:0/2556188923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c07d290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 -- 172.21.15.94:0/2556188923 >> 172.21.15.94:0/2556188923 conn(0x7f6c1c105260 msgr2=0x7f6c1c107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 -- 172.21.15.94:0/2556188923 shutdown_connections 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.071+0000 7f6c22977700 1 -- 172.21.15.94:0/2556188923 wait complete. 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c22977700 1 Processor -- start 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c22977700 1 -- start start 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c22977700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c22977700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c10bcb0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53216/0 (socket says 172.21.15.94:53216) 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.072+0000 7f6c1bfff700 1 -- 172.21.15.94:0/3050951817 learned_addr learned my addr 172.21.15.94:0/3050951817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:52.730 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c1bfff700 1 -- 172.21.15.94:0/3050951817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c1a49d0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c1bfff700 1 --2- 172.21.15.94:0/3050951817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0117b0 tx=0x7f6c0c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c003af0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c1a4c60 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6c0c020070 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c019630 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c1a7870 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c0c019c80 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.073+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6c0c00b720 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.074+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c064cf0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.077+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6c0c0042a0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.144+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f6c1c02e510 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.146+0000 7f6c197fa700 1 -- 172.21.15.94:0/3050951817 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6c0c01a9c0 con 0x7f6c1c07edc0 2024-04-25T13:31:52.731 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 msgr2=0x7f6c1c1a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 --2- 172.21.15.94:0/3050951817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0117b0 tx=0x7f6c0c011b70 comp rx=0 tx=0).stop 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 shutdown_connections 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 --2- 172.21.15.94:0/3050951817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c1c07edc0 0x7f6c1c1a4490 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 >> 172.21.15.94:0/3050951817 conn(0x7f6c1c105260 msgr2=0x7f6c1c19af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 shutdown_connections 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:52.147+0000 7f6c22977700 1 -- 172.21.15.94:0/3050951817 wait complete. 2024-04-25T13:31:52.732 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T13:31:52.778 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:31:52.778 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-04-25T13:31:52.778 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T13:31:52.779 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T13:31:53.046 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:31:53.046 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T13:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968.target.wants/ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f9021454-0307-11ef-bc93-c7b262605968@.service. 2024-04-25T13:31:53.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:53 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3050951817' entity='client.admin' 2024-04-25T13:31:53.427 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 systemd[1]: Starting Ceph mgr.x for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:31:53.700 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 podman[29409]: 2024-04-25 13:31:53.329736447 +0000 UTC m=+0.017974881 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:31:53.700 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 podman[29409]: 2024-04-25 13:31:53.430771159 +0000 UTC m=+0.119009625 container create fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:31:53.700 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 podman[29409]: 2024-04-25 13:31:53.624998119 +0000 UTC m=+0.313236539 container init fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T13:31:53.700 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 podman[29409]: 2024-04-25 13:31:53.627451501 +0000 UTC m=+0.315689922 container start fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T13:31:53.700 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 bash[29409]: fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 2024-04-25T13:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T13:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T13:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T13:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T13:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T13:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-04-25T13:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-04-25T13:31:53.970 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 systemd[1]: Started Ceph mgr.x for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:31:53.970 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:53.792+0000 7f273a679200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:31:54.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:53 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:53.980+0000 7f273a679200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:31:54.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:54 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:54.082+0000 7f273a679200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:31:54.582 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:54 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:54.403+0000 7f273a679200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:31:54.582 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:54 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:54.486+0000 7f273a679200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:31:54.582 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:54 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4077132536' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:31:45.993268+0000", 2024-04-25T13:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.204+0000 7fcc937fb700 1 Processor -- start 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.204+0000 7fcc937fb700 1 -- start start 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.204+0000 7fcc937fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.204+0000 7fcc937fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c1094b0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc91597700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc91597700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35944/0 (socket says 172.21.15.94:35944) 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc91597700 1 -- 172.21.15.94:0/581203473 learned_addr learned my addr 172.21.15.94:0/581203473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc91597700 1 -- 172.21.15.94:0/581203473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c109c80 con 0x7fcc8c108ab0 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc91597700 1 --2- 172.21.15.94:0/581203473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098a0 tx=0x7fcc7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6555eb64c24c28bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc83fff700 1 -- 172.21.15.94:0/581203473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7c004160 con 0x7fcc8c108ab0 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc83fff700 1 -- 172.21.15.94:0/581203473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc7c004300 con 0x7fcc8c108ab0 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.205+0000 7fcc83fff700 1 -- 172.21.15.94:0/581203473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7c011af0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 -- 172.21.15.94:0/581203473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 msgr2=0x7fcc8c108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 --2- 172.21.15.94:0/581203473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098a0 tx=0x7fcc7c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 -- 172.21.15.94:0/581203473 shutdown_connections 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 --2- 172.21.15.94:0/581203473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c108ec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 -- 172.21.15.94:0/581203473 >> 172.21.15.94:0/581203473 conn(0x7fcc8c1040d0 msgr2=0x7fcc8c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.206+0000 7fcc937fb700 1 -- 172.21.15.94:0/581203473 shutdown_connections 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc937fb700 1 -- 172.21.15.94:0/581203473 wait complete. 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc937fb700 1 Processor -- start 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc937fb700 1 -- start start 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc937fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc937fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c10ceb0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc91597700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc91597700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35948/0 (socket says 172.21.15.94:35948) 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.207+0000 7fcc91597700 1 -- 172.21.15.94:0/4077132536 learned_addr learned my addr 172.21.15.94:0/4077132536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc91597700 1 -- 172.21.15.94:0/4077132536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c077070 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc91597700 1 --2- 172.21.15.94:0/4077132536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c011670 tx=0x7fcc7c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7c0036a0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c1a0cf0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc7c01a430 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7c019550 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c1a1170 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.208+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc7c019b50 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.209+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc7c024730 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.209+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c064cf0 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.212+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc7c043e50 con 0x7fcc8c108ab0 2024-04-25T13:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.284+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc8c1a1510 con 0x7fcc8c108ab0 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.285+0000 7fcc827fc700 1 -- 172.21.15.94:0/4077132536 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcc7c032020 con 0x7fcc8c108ab0 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 msgr2=0x7fcc8c076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 --2- 172.21.15.94:0/4077132536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c011670 tx=0x7fcc7c004280 comp rx=0 tx=0).stop 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 shutdown_connections 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 --2- 172.21.15.94:0/4077132536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc8c108ab0 0x7fcc8c076b30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 >> 172.21.15.94:0/4077132536 conn(0x7fcc8c1040d0 msgr2=0x7fcc8c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 shutdown_connections 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:54.286+0000 7fcc937fb700 1 -- 172.21.15.94:0/4077132536 wait complete. 2024-04-25T13:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-04-25T13:31:55.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:55 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:55.692+0000 7f273a679200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:31:55.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:55 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:55.859+0000 7f273a679200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:31:56.309 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:56.025+0000 7f273a679200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:31:56.309 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:56.307+0000 7f273a679200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:31:56.615 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:56.398+0000 7f273a679200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:31:56.615 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:56.614+0000 7f273a679200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:31:56.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:56.807+0000 7f273a679200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:31:57.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:31:57 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2098593486' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:31:57.726 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:57 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:57.421+0000 7f273a679200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:31:58.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:31:58.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:31:58.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:31:58.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:31:58.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:31:58.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:31:58.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:31:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:31:45.993268+0000", 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:31:58.081 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5ca159700 1 Processor -- start 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5ca159700 1 -- start start 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5ca159700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5ca159700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41094e0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35958/0 (socket says 172.21.15.94:35958) 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5c37fe700 1 -- 172.21.15.94:0/2962286955 learned_addr learned my addr 172.21.15.94:0/2962286955 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5c37fe700 1 -- 172.21.15.94:0/2962286955 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c4109cb0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.444+0000 7fa5c37fe700 1 --2- 172.21.15.94:0/2962286955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098a0 tx=0x7fa5b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2372abfded29dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.445+0000 7fa5c27fc700 1 -- 172.21.15.94:0/2962286955 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b4004160 con 0x7fa5c4108ae0 2024-04-25T13:31:58.082 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.445+0000 7fa5c27fc700 1 -- 172.21.15.94:0/2962286955 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5b4004300 con 0x7fa5c4108ae0 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.445+0000 7fa5c27fc700 1 -- 172.21.15.94:0/2962286955 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b4011af0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.445+0000 7fa5ca159700 1 -- 172.21.15.94:0/2962286955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 msgr2=0x7fa5c4108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.445+0000 7fa5ca159700 1 --2- 172.21.15.94:0/2962286955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098a0 tx=0x7fa5b4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 -- 172.21.15.94:0/2962286955 shutdown_connections 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 --2- 172.21.15.94:0/2962286955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c4108ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 -- 172.21.15.94:0/2962286955 >> 172.21.15.94:0/2962286955 conn(0x7fa5c4104120 msgr2=0x7fa5c41065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 -- 172.21.15.94:0/2962286955 shutdown_connections 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 -- 172.21.15.94:0/2962286955 wait complete. 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.446+0000 7fa5ca159700 1 Processor -- start 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5ca159700 1 -- start start 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5ca159700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5ca159700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410cee0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:31:58.083 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35960/0 (socket says 172.21.15.94:35960) 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c37fe700 1 -- 172.21.15.94:0/2098593486 learned_addr learned my addr 172.21.15.94:0/2098593486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c37fe700 1 -- 172.21.15.94:0/2098593486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419e4e0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c37fe700 1 --2- 172.21.15.94:0/2098593486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40090e0 tx=0x7fa5b4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.447+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b40039e0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa5b4003b80 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419e770 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5b40246e0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a1380 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa5b4022020 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.448+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5b402dd70 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.449+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4064cf0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.451+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa5b401aae0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.523+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa5c402e780 con 0x7fa5c4108ae0 2024-04-25T13:31:58.084 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.523+0000 7fa5c0ff9700 1 -- 172.21.15.94:0/2098593486 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa5b402d3f0 con 0x7fa5c4108ae0 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 msgr2=0x7fa5c419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 --2- 172.21.15.94:0/2098593486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40090e0 tx=0x7fa5b4011940 comp rx=0 tx=0).stop 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 shutdown_connections 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 --2- 172.21.15.94:0/2098593486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c4108ae0 0x7fa5c419dfa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 >> 172.21.15.94:0/2098593486 conn(0x7fa5c4104120 msgr2=0x7fa5c41944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 shutdown_connections 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:31:57.524+0000 7fa5ca159700 1 -- 172.21.15.94:0/2098593486 wait complete. 2024-04-25T13:31:58.085 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-04-25T13:31:58.135 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:57 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:57.858+0000 7f273a679200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:31:58.135 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:57 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:57.955+0000 7f273a679200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:31:58.475 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:58 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:58.160+0000 7f273a679200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:31:58.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:58 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:58.690+0000 7f273a679200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:31:58.977 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:58 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:58.783+0000 7f273a679200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:31:59.374 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:59.028+0000 7f273a679200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:31:59.375 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:59.161+0000 7f273a679200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:31:59.375 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:59.257+0000 7f273a679200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:31:59.375 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:59.373+0000 7f273a679200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:31:59.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:31:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:31:59.675+0000 7f273a679200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:32:00.466 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:00.278+0000 7f273a679200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:32:00.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:00.505+0000 7f273a679200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: Activating manager daemon x 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: mgrmap e2: x(active, starting, since 0.00507396s) 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T13:32:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:32:00.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:32:00.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' 2024-04-25T13:32:00.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' 2024-04-25T13:32:00.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:00 smithi094 ceph-mon[29195]: from='mgr.14100 172.21.15.94:0/1642121904' entity='mgr.x' 2024-04-25T13:32:01.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:32:01.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:01.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:32:01.191 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:32:01.192 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:32:01.193 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:32:01.194 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:31:45.993268+0000", 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:01.195 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f498e4fe700 1 Processor -- start 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f498e4fe700 1 -- start start 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f498e4fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f498e4fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4988109450 con 0x7f4988108a50 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f4987fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f4987fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35966/0 (socket says 172.21.15.94:35966) 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.491+0000 7f4987fff700 1 -- 172.21.15.94:0/3408884206 learned_addr learned my addr 172.21.15.94:0/3408884206 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.492+0000 7f4987fff700 1 -- 172.21.15.94:0/3408884206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4988109c20 con 0x7f4988108a50 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.492+0000 7f4987fff700 1 --2- 172.21.15.94:0/3408884206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49780098a0 tx=0x7f4978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be29607f2459f39b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.492+0000 7f4986ffd700 1 -- 172.21.15.94:0/3408884206 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4978004160 con 0x7f4988108a50 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.492+0000 7f4986ffd700 1 -- 172.21.15.94:0/3408884206 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4978004300 con 0x7f4988108a50 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.492+0000 7f4986ffd700 1 -- 172.21.15.94:0/3408884206 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4978011af0 con 0x7f4988108a50 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 -- 172.21.15.94:0/3408884206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 msgr2=0x7f4988108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 --2- 172.21.15.94:0/3408884206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49780098a0 tx=0x7f4978009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:01.196 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 -- 172.21.15.94:0/3408884206 shutdown_connections 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 --2- 172.21.15.94:0/3408884206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f4988108e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 -- 172.21.15.94:0/3408884206 >> 172.21.15.94:0/3408884206 conn(0x7f49881040f0 msgr2=0x7f4988106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 -- 172.21.15.94:0/3408884206 shutdown_connections 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.493+0000 7f498e4fe700 1 -- 172.21.15.94:0/3408884206 wait complete. 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f498e4fe700 1 Processor -- start 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f498e4fe700 1 -- start start 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f498e4fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f498e4fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f498810ce50 con 0x7f4988108a50 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f4987fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f4987fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35982/0 (socket says 172.21.15.94:35982) 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f4987fff700 1 -- 172.21.15.94:0/1000964014 learned_addr learned my addr 172.21.15.94:0/1000964014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:01.197 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.494+0000 7f4987fff700 1 -- 172.21.15.94:0/1000964014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881a06a0 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f4987fff700 1 --2- 172.21.15.94:0/1000964014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49780090e0 tx=0x7f4978011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49780039e0 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4978003b80 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f498e4fe700 1 -- 172.21.15.94:0/1000964014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881a0930 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49780246e0 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f498e4fe700 1 -- 172.21.15.94:0/1000964014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a3540 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4978022020 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.495+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4978019450 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.496+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968005330 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.498+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f497801aae0 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.509+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 2) ==== 49272+0+0 (secure 0 0 0) 0x7f497801a430 con 0x7f4988108a50 2024-04-25T13:32:01.198 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.584+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f49680058f0 con 0x7f4988108a50 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.584+0000 7f49857fa700 1 -- 172.21.15.94:0/1000964014 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4978018350 con 0x7f4988108a50 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 msgr2=0x7f49881a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 --2- 172.21.15.94:0/1000964014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49780090e0 tx=0x7f4978011940 comp rx=0 tx=0).stop 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 shutdown_connections 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 --2- 172.21.15.94:0/1000964014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4988108a50 0x7f49881a0160 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 >> 172.21.15.94:0/1000964014 conn(0x7f49881040f0 msgr2=0x7f4988104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 shutdown_connections 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:00.585+0000 7f496effd700 1 -- 172.21.15.94:0/1000964014 wait complete. 2024-04-25T13:32:01.199 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-04-25T13:32:01.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:01 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1000964014' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:32:02.045 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:01 smithi094 ceph-mon[29195]: mgrmap e3: x(active, since 1.0072s) 2024-04-25T13:32:03.862 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:03 smithi094 ceph-mon[29195]: mgrmap e4: x(active, since 2s) 2024-04-25T13:32:04.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:32:04.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:32:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:32:04.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:32:04.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:32:04.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:32:04.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:32:04.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:32:04.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:32:04.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:31:45.993268+0000", 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.568+0000 7f1773855700 1 Processor -- start 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f1773855700 1 -- start start 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f1773855700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f1773855700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f176c076fe0 con 0x7f176c10a6f0 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f17715f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f17715f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38254/0 (socket says 172.21.15.94:38254) 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f17715f1700 1 -- 172.21.15.94:0/2286261411 learned_addr learned my addr 172.21.15.94:0/2286261411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f17715f1700 1 -- 172.21.15.94:0/2286261411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c10d050 con 0x7f176c10a6f0 2024-04-25T13:32:04.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.569+0000 7f17715f1700 1 --2- 172.21.15.94:0/2286261411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f17680098a0 tx=0x7f1768009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8299f1a0875159c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1763fff700 1 -- 172.21.15.94:0/2286261411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1768004160 con 0x7f176c10a6f0 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1763fff700 1 -- 172.21.15.94:0/2286261411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1768004300 con 0x7f176c10a6f0 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1763fff700 1 -- 172.21.15.94:0/2286261411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1768011af0 con 0x7f176c10a6f0 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 -- 172.21.15.94:0/2286261411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 msgr2=0x7f176c10cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 --2- 172.21.15.94:0/2286261411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f17680098a0 tx=0x7f1768009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 -- 172.21.15.94:0/2286261411 shutdown_connections 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 --2- 172.21.15.94:0/2286261411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c10cb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 -- 172.21.15.94:0/2286261411 >> 172.21.15.94:0/2286261411 conn(0x7f176c1040d0 msgr2=0x7f176c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.570+0000 7f1773855700 1 -- 172.21.15.94:0/2286261411 shutdown_connections 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f1773855700 1 -- 172.21.15.94:0/2286261411 wait complete. 2024-04-25T13:32:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f1773855700 1 Processor -- start 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f1773855700 1 -- start start 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f1773855700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f1773855700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10ded0 con 0x7f176c10a6f0 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f17715f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f17715f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38262/0 (socket says 172.21.15.94:38262) 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.571+0000 7f17715f1700 1 -- 172.21.15.94:0/1174876180 learned_addr learned my addr 172.21.15.94:0/1174876180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f17715f1700 1 -- 172.21.15.94:0/1174876180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c10a3b0 con 0x7f176c10a6f0 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f17715f1700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1768009fb0 tx=0x7f1768011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1768011bc0 con 0x7f176c10a6f0 2024-04-25T13:32:04.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1768024440 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c108600 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f176801a5c0 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.572+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c108ae0 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.573+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f176801a880 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.573+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c064cf0 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.573+0000 7f17627fc700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 0x7f175803f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.573+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f17680525e0 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.573+0000 7f1770df0700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 0x7f175803f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.574+0000 7f1770df0700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 0x7f175803f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f175c006fd0 tx=0x7f175c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.579+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f176800b3e0 con 0x7f176c10a6f0 2024-04-25T13:32:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.862+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f176c109260 con 0x7f176c10a6f0 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.863+0000 7f17627fc700 1 -- 172.21.15.94:0/1174876180 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1768029df0 con 0x7f176c10a6f0 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 msgr2=0x7f175803f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 0x7f175803f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f175c006fd0 tx=0x7f175c006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 msgr2=0x7f176c109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1768009fb0 tx=0x7f1768011890 comp rx=0 tx=0).stop 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 shutdown_connections 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f175803d4d0 0x7f175803f9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 --2- 172.21.15.94:0/1174876180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f176c10a6f0 0x7f176c109e70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 >> 172.21.15.94:0/1174876180 conn(0x7f176c1040d0 msgr2=0x7f176c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 shutdown_connections 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:03.865+0000 7f1773855700 1 -- 172.21.15.94:0/1174876180 wait complete. 2024-04-25T13:32:04.516 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-04-25T13:32:04.829 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:04 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1174876180' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:32:05.650 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:05 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/100295203' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:32:05.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:32:05.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:32:05.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:05.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:32:05.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:32:05.661 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.935+0000 7f1e5eefa700 1 Processor -- start 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.935+0000 7f1e5eefa700 1 -- start start 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5eefa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5eefa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58109460 con 0x7f1e58108a60 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5cc96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5cc96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38272/0 (socket says 172.21.15.94:38272) 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5cc96700 1 -- 172.21.15.94:0/2852824115 learned_addr learned my addr 172.21.15.94:0/2852824115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5cc96700 1 -- 172.21.15.94:0/2852824115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e58109c30 con 0x7f1e58108a60 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e5cc96700 1 --2- 172.21.15.94:0/2852824115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098a0 tx=0x7f1e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256a4d68b6578362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e577fe700 1 -- 172.21.15.94:0/2852824115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c004160 con 0x7f1e58108a60 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e577fe700 1 -- 172.21.15.94:0/2852824115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1e4c004300 con 0x7f1e58108a60 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.936+0000 7f1e577fe700 1 -- 172.21.15.94:0/2852824115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c011af0 con 0x7f1e58108a60 2024-04-25T13:32:05.662 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 -- 172.21.15.94:0/2852824115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 msgr2=0x7f1e58108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/2852824115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0098a0 tx=0x7f1e4c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 -- 172.21.15.94:0/2852824115 shutdown_connections 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/2852824115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e58108e70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 -- 172.21.15.94:0/2852824115 >> 172.21.15.94:0/2852824115 conn(0x7f1e58104140 msgr2=0x7f1e58106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 -- 172.21.15.94:0/2852824115 shutdown_connections 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.937+0000 7f1e5eefa700 1 -- 172.21.15.94:0/2852824115 wait complete. 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5eefa700 1 Processor -- start 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5eefa700 1 -- start start 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5eefa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5eefa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810ce60 con 0x7f1e58108a60 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5cc96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5cc96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38288/0 (socket says 172.21.15.94:38288) 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5cc96700 1 -- 172.21.15.94:0/100295203 learned_addr learned my addr 172.21.15.94:0/100295203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5cc96700 1 -- 172.21.15.94:0/100295203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5819d330 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.938+0000 7f1e5cc96700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0090e0 tx=0x7f1e4c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.939+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c0039e0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.939+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5819d5c0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.939+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1e4c003b80 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.939+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e4c0246e0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.939+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5807bfc0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f1e4c022020 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e58064cf0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e55ffb700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 0x7f1e48041c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e4c0522e0 con 0x7f1e58108a60 2024-04-25T13:32:05.664 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e57fff700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 0x7f1e48041c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.940+0000 7f1e57fff700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 0x7f1e48041c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e44006fd0 tx=0x7f1e44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:04.945+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e4c052040 con 0x7f1e58108a60 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.143+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1e5807c710 con 0x7f1e58108a60 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.144+0000 7f1e55ffb700 1 -- 172.21.15.94:0/100295203 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f1e4c04c1d0 con 0x7f1e58108a60 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 msgr2=0x7f1e48041c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 0x7f1e48041c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e44006fd0 tx=0x7f1e44006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 msgr2=0x7f1e5819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0090e0 tx=0x7f1e4c011940 comp rx=0 tx=0).stop 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 shutdown_connections 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f1e4803f730 0x7f1e48041c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 --2- 172.21.15.94:0/100295203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58108a60 0x7f1e5819cdf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 >> 172.21.15.94:0/100295203 conn(0x7f1e58104140 msgr2=0x7f1e58105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 shutdown_connections 2024-04-25T13:32:05.665 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:05.147+0000 7f1e5eefa700 1 -- 172.21.15.94:0/100295203 wait complete. 2024-04-25T13:32:05.666 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-04-25T13:32:06.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:06 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1573674043' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T13:32:06.946 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setuser ceph since I am not root 2024-04-25T13:32:06.946 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setgroup ceph since I am not root 2024-04-25T13:32:06.947 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:06.758+0000 7fc0215b8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:32:06.947 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:06.946+0000 7fc0215b8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:32:07.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.088+0000 7fb948530700 1 Processor -- start 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.088+0000 7fb948530700 1 -- start start 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.088+0000 7fb948530700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.088+0000 7fb948530700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb940108830 con 0x7fb940107e50 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9462cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9462cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38294/0 (socket says 172.21.15.94:38294) 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9462cc700 1 -- 172.21.15.94:0/1211860895 learned_addr learned my addr 172.21.15.94:0/1211860895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9462cc700 1 -- 172.21.15.94:0/1211860895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94010ae50 con 0x7fb940107e50 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9462cc700 1 --2- 172.21.15.94:0/1211860895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009e80 tx=0x7fb93c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e176989eddae38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9452ca700 1 -- 172.21.15.94:0/1211860895 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb93c017070 con 0x7fb940107e50 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9452ca700 1 -- 172.21.15.94:0/1211860895 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb93c00b8c0 con 0x7fb940107e50 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.089+0000 7fb9452ca700 1 -- 172.21.15.94:0/1211860895 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb93c004490 con 0x7fb940107e50 2024-04-25T13:32:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 -- 172.21.15.94:0/1211860895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 msgr2=0x7fb940108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 --2- 172.21.15.94:0/1211860895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009e80 tx=0x7fb93c00b330 comp rx=0 tx=0).stop 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 -- 172.21.15.94:0/1211860895 shutdown_connections 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 --2- 172.21.15.94:0/1211860895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb940108260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 -- 172.21.15.94:0/1211860895 >> 172.21.15.94:0/1211860895 conn(0x7fb940103840 msgr2=0x7fb940105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 -- 172.21.15.94:0/1211860895 shutdown_connections 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.090+0000 7fb948530700 1 -- 172.21.15.94:0/1211860895 wait complete. 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb948530700 1 Processor -- start 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb948530700 1 -- start start 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb948530700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb948530700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010bc70 con 0x7fb940107e50 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb9462cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb9462cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38296/0 (socket says 172.21.15.94:38296) 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb9462cc700 1 -- 172.21.15.94:0/1573674043 learned_addr learned my addr 172.21.15.94:0/1573674043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:07.149 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.091+0000 7fb9462cc700 1 -- 172.21.15.94:0/1573674043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94010ab10 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb9462cc700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00bca0 tx=0x7fb93c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb93c020020 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb93c012430 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb93c017040 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb940108d60 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.092+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9401092a0 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb940064cf0 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fb93c023020 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb9377fe700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 0x7fb92c03f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb93c00f050 con 0x7fb940107e50 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb945acb700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 0x7fb92c03f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:07.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.093+0000 7fb945acb700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 0x7fb92c03f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb930006fd0 tx=0x7fb930006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.098+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93c02a020 con 0x7fb940107e50 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.351+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb940109990 con 0x7fb940107e50 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.565+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb93c02b260 con 0x7fb940107e50 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.567+0000 7fb9377fe700 1 -- 172.21.15.94:0/1573674043 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fb93c057020 con 0x7fb940107e50 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 msgr2=0x7fb92c03f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 0x7fb92c03f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb930006fd0 tx=0x7fb930006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 msgr2=0x7fb94010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00bca0 tx=0x7fb93c00bd80 comp rx=0 tx=0).stop 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 shutdown_connections 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb92c03d4d0 0x7fb92c03f9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 --2- 172.21.15.94:0/1573674043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb940107e50 0x7fb94010a5d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 >> 172.21.15.94:0/1573674043 conn(0x7fb940103840 msgr2=0x7fb940104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 shutdown_connections 2024-04-25T13:32:07.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:06.568+0000 7fb948530700 1 -- 172.21.15.94:0/1573674043 wait complete. 2024-04-25T13:32:07.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:07.050+0000 7fc0215b8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:32:07.534 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:07.366+0000 7fc0215b8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:32:07.535 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:07.456+0000 7fc0215b8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:32:07.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:07 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1573674043' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T13:32:07.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:07 smithi094 ceph-mon[29195]: mgrmap e5: x(active, since 6s) 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:08.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e9af93700 1 Processor -- start 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e9af93700 1 -- start start 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e9af93700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e9af93700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e94076f80 con 0x7f5e9410a780 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e98d2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e98d2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38320/0 (socket says 172.21.15.94:38320) 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.646+0000 7f5e98d2f700 1 -- 172.21.15.94:0/793264685 learned_addr learned my addr 172.21.15.94:0/793264685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.647+0000 7f5e98d2f700 1 -- 172.21.15.94:0/793264685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9410d0e0 con 0x7f5e9410a780 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.647+0000 7f5e98d2f700 1 --2- 172.21.15.94:0/793264685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e880098a0 tx=0x7f5e88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dcbda61adc872d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.647+0000 7f5e937fe700 1 -- 172.21.15.94:0/793264685 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88004160 con 0x7f5e9410a780 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.647+0000 7f5e937fe700 1 -- 172.21.15.94:0/793264685 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5e88004300 con 0x7f5e9410a780 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.647+0000 7f5e937fe700 1 -- 172.21.15.94:0/793264685 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88011af0 con 0x7f5e9410a780 2024-04-25T13:32:08.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- 172.21.15.94:0/793264685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 msgr2=0x7f5e9410cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 --2- 172.21.15.94:0/793264685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e880098a0 tx=0x7f5e88009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- 172.21.15.94:0/793264685 shutdown_connections 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 --2- 172.21.15.94:0/793264685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9410cba0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- 172.21.15.94:0/793264685 >> 172.21.15.94:0/793264685 conn(0x7f5e94104120 msgr2=0x7f5e941065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- 172.21.15.94:0/793264685 shutdown_connections 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- 172.21.15.94:0/793264685 wait complete. 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 Processor -- start 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.648+0000 7f5e9af93700 1 -- start start 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e9af93700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e9af93700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9410df00 con 0x7f5e9410a780 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e98d2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e98d2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38322/0 (socket says 172.21.15.94:38322) 2024-04-25T13:32:08.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e98d2f700 1 -- 172.21.15.94:0/2447618582 learned_addr learned my addr 172.21.15.94:0/2447618582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e98d2f700 1 -- 172.21.15.94:0/2447618582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9419e480 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e98d2f700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009fb0 tx=0x7f5e88011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88011bc0 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9419e710 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5e88024440 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.649+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e8801a5c0 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.650+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e941a1320 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.650+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e94064cf0 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.650+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f5e8801a880 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.651+0000 7f5e91ffb700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 0x7f5e8403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.651+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e88052670 con 0x7f5e9410a780 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.651+0000 7f5e93fff700 1 -- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 msgr2=0x7f5e8403fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:08.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.651+0000 7f5e93fff700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 0x7f5e8403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.656+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e8800b3e0 con 0x7f5e9410a780 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.852+0000 7f5e93fff700 1 -- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 msgr2=0x7f5e8403fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.852+0000 7f5e93fff700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 0x7f5e8403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.952+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5e941a1540 con 0x7f5e9410a780 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.953+0000 7f5e91ffb700 1 -- 172.21.15.94:0/2447618582 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5e941a1540 con 0x7f5e9410a780 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 msgr2=0x7f5e8403fa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 0x7f5e8403fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 msgr2=0x7f5e9419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009fb0 tx=0x7f5e88011890 comp rx=0 tx=0).stop 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 shutdown_connections 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7f5e8403d520 0x7f5e8403fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 --2- 172.21.15.94:0/2447618582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e9410a780 0x7f5e9419df40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:08.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 >> 172.21.15.94:0/2447618582 conn(0x7f5e94104120 msgr2=0x7f5e94104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:08.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 shutdown_connections 2024-04-25T13:32:08.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:07.955+0000 7f5e9af93700 1 -- 172.21.15.94:0/2447618582 wait complete. 2024-04-25T13:32:08.503 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-25T13:32:08.503 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-04-25T13:32:08.870 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2447618582' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:32:08.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:08.689+0000 7fc0215b8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:32:08.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:08.870+0000 7fc0215b8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:32:09.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:09.040+0000 7fc0215b8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:32:09.617 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:09.327+0000 7fc0215b8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:32:09.617 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:09.418+0000 7fc0215b8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:32:09.617 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:09.617+0000 7fc0215b8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:32:09.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:09.808+0000 7fc0215b8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:32:10.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:10.396+0000 7fc0215b8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:32:10.993 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:10.827+0000 7fc0215b8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:32:10.994 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:10.928+0000 7fc0215b8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:32:11.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:11.150+0000 7fc0215b8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:32:11.964 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:11.690+0000 7fc0215b8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:32:11.964 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:11.786+0000 7fc0215b8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:32:12.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:12.038+0000 7fc0215b8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:32:12.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:12.170+0000 7fc0215b8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:32:12.679 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:12.266+0000 7fc0215b8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:32:12.679 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:12.382+0000 7fc0215b8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:32:12.680 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:12.679+0000 7fc0215b8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:32:13.550 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:13.280+0000 7fc0215b8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:32:13.550 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:13.497+0000 7fc0215b8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:32:13.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: Activating manager daemon x 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: mgrmap e6: x(active, starting, since 0.00376105s) 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:13.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:13 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:14.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:14 smithi094 ceph-mon[29195]: Found migration_current of "None". Setting to last migration. 2024-04-25T13:32:14.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:14 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:32:14.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:14 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:32:14.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:14 smithi094 ceph-mon[29195]: mgrmap e7: x(active, since 1.00636s) 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.902+0000 7fb33bcd0700 1 Processor -- start 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.902+0000 7fb33bcd0700 1 -- start start 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.902+0000 7fb33bcd0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:15.040 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.902+0000 7fb33bcd0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb334076fe0 con 0x7fb33410a720 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb339a6c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb339a6c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38336/0 (socket says 172.21.15.94:38336) 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb339a6c700 1 -- 172.21.15.94:0/1317242546 learned_addr learned my addr 172.21.15.94:0/1317242546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb339a6c700 1 -- 172.21.15.94:0/1317242546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33410d080 con 0x7fb33410a720 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb339a6c700 1 --2- 172.21.15.94:0/1317242546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3240098a0 tx=0x7fb324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7e1cd225485b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb338a6a700 1 -- 172.21.15.94:0/1317242546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb324004160 con 0x7fb33410a720 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb338a6a700 1 -- 172.21.15.94:0/1317242546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb324004300 con 0x7fb33410a720 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.903+0000 7fb338a6a700 1 -- 172.21.15.94:0/1317242546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb324011af0 con 0x7fb33410a720 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- 172.21.15.94:0/1317242546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 msgr2=0x7fb33410cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 --2- 172.21.15.94:0/1317242546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3240098a0 tx=0x7fb324009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- 172.21.15.94:0/1317242546 shutdown_connections 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 --2- 172.21.15.94:0/1317242546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33410cb40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.041 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- 172.21.15.94:0/1317242546 >> 172.21.15.94:0/1317242546 conn(0x7fb334104120 msgr2=0x7fb3341065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- 172.21.15.94:0/1317242546 shutdown_connections 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- 172.21.15.94:0/1317242546 wait complete. 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 Processor -- start 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.905+0000 7fb33bcd0700 1 -- start start 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb33bcd0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb33bcd0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb33410df00 con 0x7fb33410a720 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb339a6c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb339a6c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38340/0 (socket says 172.21.15.94:38340) 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb339a6c700 1 -- 172.21.15.94:0/2292819626 learned_addr learned my addr 172.21.15.94:0/2292819626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb339a6c700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33419e4a0 con 0x7fb33410a720 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb339a6c700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fb0 tx=0x7fb324011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb324011bc0 con 0x7fb33410a720 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb33bcd0700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33419e730 con 0x7fb33410a720 2024-04-25T13:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb324024440 con 0x7fb33410a720 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.906+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb32401a5c0 con 0x7fb33410a720 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.907+0000 7fb33bcd0700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341a1340 con 0x7fb33410a720 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.907+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fb32401a880 con 0x7fb33410a720 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.908+0000 7fb32affd700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.908+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb31c000d10 con 0x7fb32003d520 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.908+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3240245f0 con 0x7fb33410a720 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.908+0000 7fb33926b700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:08.908+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:09.109+0000 7fb33926b700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:15.043 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:09.109+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:09.510+0000 7fb33926b700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:09.510+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:10.311+0000 7fb33926b700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:10.311+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:11.912+0000 7fb33926b700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2945875633 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:11.912+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:13.501+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7fb324021d40 con 0x7fb33410a720 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:13.501+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 msgr2=0x7fb32003fa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:13.501+0000 7fb32affd700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.502+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7fb3240188a0 con 0x7fb33410a720 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.502+0000 7fb32affd700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 0x7fb320043090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.502+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb31c000d10 con 0x7fb320040c70 2024-04-25T13:32:15.044 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.503+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 0x7fb320043090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.504+0000 7fb33926b700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 0x7fb320043090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb330003a10 tx=0x7fb330009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.504+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb31c000d10 con 0x7fb320040c70 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.512+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb31c002810 con 0x7fb320040c70 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb32affd700 1 -- 172.21.15.94:0/2292819626 <== mgr.14120 v2:172.21.15.94:6800/1021554684 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb31c002810 con 0x7fb320040c70 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 msgr2=0x7fb320043090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 0x7fb320043090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb330003a10 tx=0x7fb330009290 comp rx=0 tx=0).stop 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 msgr2=0x7fb33419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fb0 tx=0x7fb324011890 comp rx=0 tx=0).stop 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 shutdown_connections 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fb320040c70 0x7fb320043090 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb33410a720 0x7fb33419df60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 --2- 172.21.15.94:0/2292819626 >> [v2:172.21.15.94:6800/2945875633,v1:172.21.15.94:6801/2945875633] conn(0x7fb32003d520 0x7fb32003fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:15.045 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 >> 172.21.15.94:0/2292819626 conn(0x7fb334104120 msgr2=0x7fb334104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:15.046 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 shutdown_connections 2024-04-25T13:32:15.046 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:14.513+0000 7fb328ff9700 1 -- 172.21.15.94:0/2292819626 wait complete. 2024-04-25T13:32:15.046 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-04-25T13:32:15.046 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-04-25T13:32:15.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:15 smithi094 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:32:15.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:15 smithi094 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:32:15.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:15 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:15.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:15 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.460+0000 7fd36a5d4700 1 Processor -- start 2024-04-25T13:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.460+0000 7fd36a5d4700 1 -- start start 2024-04-25T13:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.460+0000 7fd36a5d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.460+0000 7fd36a5d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd364077040 con 0x7fd36410a6c0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd363fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd363fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59812/0 (socket says 172.21.15.94:59812) 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd363fff700 1 -- 172.21.15.94:0/2735552518 learned_addr learned my addr 172.21.15.94:0/2735552518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd363fff700 1 -- 172.21.15.94:0/2735552518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36410d020 con 0x7fd36410a6c0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd363fff700 1 --2- 172.21.15.94:0/2735552518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3580098a0 tx=0x7fd358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6693123d68da6fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd362ffd700 1 -- 172.21.15.94:0/2735552518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd358004160 con 0x7fd36410a6c0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd362ffd700 1 -- 172.21.15.94:0/2735552518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd358004300 con 0x7fd36410a6c0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.461+0000 7fd362ffd700 1 -- 172.21.15.94:0/2735552518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd358011af0 con 0x7fd36410a6c0 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 -- 172.21.15.94:0/2735552518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 msgr2=0x7fd36410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/2735552518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3580098a0 tx=0x7fd358009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 -- 172.21.15.94:0/2735552518 shutdown_connections 2024-04-25T13:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/2735552518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36410cae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 -- 172.21.15.94:0/2735552518 >> 172.21.15.94:0/2735552518 conn(0x7fd364104120 msgr2=0x7fd3641065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 -- 172.21.15.94:0/2735552518 shutdown_connections 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.462+0000 7fd36a5d4700 1 -- 172.21.15.94:0/2735552518 wait complete. 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd36a5d4700 1 Processor -- start 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd36a5d4700 1 -- start start 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd36a5d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd36a5d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd364108e60 con 0x7fd36410a6c0 2024-04-25T13:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd363fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd363fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59818/0 (socket says 172.21.15.94:59818) 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.463+0000 7fd363fff700 1 -- 172.21.15.94:0/501681795 learned_addr learned my addr 172.21.15.94:0/501681795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.464+0000 7fd363fff700 1 -- 172.21.15.94:0/501681795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36419c2a0 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.464+0000 7fd363fff700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd358009fb0 tx=0x7fd358011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.464+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd358011bc0 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.464+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd358024440 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.464+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd35801a5c0 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.465+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419c530 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.465+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36419f130 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.466+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7fd3580245f0 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.466+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd364064cf0 con 0x7fd36410a6c0 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.466+0000 7fd3617fa700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 0x7fd34c043d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.466+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd358052460 con 0x7fd36410a6c0 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.466+0000 7fd3637fe700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 0x7fd34c043d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.467+0000 7fd3637fe700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 0x7fd34c043d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd354006fd0 tx=0x7fd354006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.472+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd358029060 con 0x7fd36410a6c0 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.579+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fd3580038b0 con 0x7fd36410a6c0 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.694+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd36402e8c0 con 0x7fd34c041860 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.704+0000 7fd3617fa700 1 -- 172.21.15.94:0/501681795 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd36402e8c0 con 0x7fd34c041860 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 msgr2=0x7fd34c043d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 0x7fd34c043d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd354006fd0 tx=0x7fd354006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 msgr2=0x7fd36419bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd358009fb0 tx=0x7fd358011890 comp rx=0 tx=0).stop 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 shutdown_connections 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fd34c041860 0x7fd34c043d70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 --2- 172.21.15.94:0/501681795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd36410a6c0 0x7fd36419bd60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 >> 172.21.15.94:0/501681795 conn(0x7fd364104120 msgr2=0x7fd364192260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 shutdown_connections 2024-04-25T13:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:15.707+0000 7fd36a5d4700 1 -- 172.21.15.94:0/501681795 wait complete. 2024-04-25T13:32:16.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:16 smithi094 ceph-mon[29195]: mgrmap e8: x(active, since 2s) 2024-04-25T13:32:16.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:16 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:15] ENGINE Bus STARTING 2024-04-25T13:32:16.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:16 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:16.656 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:16 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:16.656 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:16 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.689+0000 7fcc44e9d700 1 Processor -- start 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.689+0000 7fcc44e9d700 1 -- start start 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc44e9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc44e9d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010bba0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59828/0 (socket says 172.21.15.94:59828) 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc3e59c700 1 -- 172.21.15.94:0/1906628280 learned_addr learned my addr 172.21.15.94:0/1906628280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.690+0000 7fcc3e59c700 1 -- 172.21.15.94:0/1906628280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4010bd20 con 0x7fcc4010b1c0 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc3e59c700 1 --2- 172.21.15.94:0/1906628280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc340098a0 tx=0x7fcc34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4e34e47faf978e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc3d59a700 1 -- 172.21.15.94:0/1906628280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc34004160 con 0x7fcc4010b1c0 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc3d59a700 1 -- 172.21.15.94:0/1906628280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc34004300 con 0x7fcc4010b1c0 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc3d59a700 1 -- 172.21.15.94:0/1906628280 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc34011af0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1906628280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 msgr2=0x7fcc4010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.691+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1906628280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc340098a0 tx=0x7fcc34009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1906628280 shutdown_connections 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1906628280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4010b5d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1906628280 >> 172.21.15.94:0/1906628280 conn(0x7fcc401038a0 msgr2=0x7fcc40105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1906628280 shutdown_connections 2024-04-25T13:32:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1906628280 wait complete. 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 Processor -- start 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.692+0000 7fcc44e9d700 1 -- start start 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc44e9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc44e9d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40108990 con 0x7fcc4010b1c0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc3e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59838/0 (socket says 172.21.15.94:59838) 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc3e59c700 1 -- 172.21.15.94:0/1108117229 learned_addr learned my addr 172.21.15.94:0/1108117229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc3e59c700 1 -- 172.21.15.94:0/1108117229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4019e550 con 0x7fcc4010b1c0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc3e59c700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc34011670 tx=0x7fcc34004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc340036a0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc3401a430 con 0x7fcc4010b1c0 2024-04-25T13:32:17.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc34019550 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4019e7e0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.693+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a13f0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.694+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc40064cf0 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.695+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fcc34019810 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.695+0000 7fcc2f7fe700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 0x7fcc2803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.695+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc34052760 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.695+0000 7fcc3dd9b700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 0x7fcc2803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.695+0000 7fcc3dd9b700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 0x7fcc2803fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.700+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc3402ce20 con 0x7fcc4010b1c0 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.899+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fcc401049c0 con 0x7fcc2803d520 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.900+0000 7fcc2f7fe700 1 -- 172.21.15.94:0/1108117229 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fcc401049c0 con 0x7fcc2803d520 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 msgr2=0x7fcc2803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 0x7fcc2803fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 msgr2=0x7fcc4019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc34011670 tx=0x7fcc34004280 comp rx=0 tx=0).stop 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 shutdown_connections 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcc2803d520 0x7fcc2803fa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 --2- 172.21.15.94:0/1108117229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc4010b1c0 0x7fcc4019e010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 >> 172.21.15.94:0/1108117229 conn(0x7fcc401038a0 msgr2=0x7fcc40104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 shutdown_connections 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:16.902+0000 7fcc44e9d700 1 -- 172.21.15.94:0/1108117229 wait complete. 2024-04-25T13:32:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-04-25T13:32:17.795 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:17 smithi094 ceph-mon[29195]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:17.796 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:17 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:15] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T13:32:17.796 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:17 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:15] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T13:32:17.796 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:17 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:15] ENGINE Bus STARTED 2024-04-25T13:32:18.940 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-mon[29195]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: Generating public/private rsa key pair. 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: Your identification has been saved in /tmp/tmp3o_8klla/key. 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: Your public key has been saved in /tmp/tmp3o_8klla/key.pub. 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: The key fingerprint is: 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: SHA256:EnCec8MGm7ZD20o+gBKo8EOIfSOFL9bXMIAjpfNnoyk ceph-f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: The key's randomart image is: 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: +---[RSA 3072]----+ 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: | ..o+.o | 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: |+++. +o* | 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: |O.++o X+= | 2024-04-25T13:32:18.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: |o*o+o+.O.. | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: |o.=.=.* S | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: | . * = + | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: |E o + | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: | . . | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: | | 2024-04-25T13:32:18.942 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: +----[SHA256]-----+ 2024-04-25T13:32:19.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.903+0000 7f86cf714700 1 Processor -- start 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.903+0000 7f86cf714700 1 -- start start 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.903+0000 7f86cf714700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.903+0000 7f86cf714700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86c810a6d0 con 0x7f86c8109cd0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86cd4b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86cd4b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59850/0 (socket says 172.21.15.94:59850) 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86cd4b0700 1 -- 172.21.15.94:0/3374572809 learned_addr learned my addr 172.21.15.94:0/3374572809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86cd4b0700 1 -- 172.21.15.94:0/3374572809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c810ae00 con 0x7f86c8109cd0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86cd4b0700 1 --2- 172.21.15.94:0/3374572809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009e80 tx=0x7f86c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf88224ca6ea74fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86bffff700 1 -- 172.21.15.94:0/3374572809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c4017070 con 0x7f86c8109cd0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86bffff700 1 -- 172.21.15.94:0/3374572809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f86c400b8c0 con 0x7f86c8109cd0 2024-04-25T13:32:19.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.904+0000 7f86bffff700 1 -- 172.21.15.94:0/3374572809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c4004490 con 0x7f86c8109cd0 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 -- 172.21.15.94:0/3374572809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 msgr2=0x7f86c810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 --2- 172.21.15.94:0/3374572809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009e80 tx=0x7f86c400b330 comp rx=0 tx=0).stop 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 -- 172.21.15.94:0/3374572809 shutdown_connections 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 --2- 172.21.15.94:0/3374572809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c810a0e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 -- 172.21.15.94:0/3374572809 >> 172.21.15.94:0/3374572809 conn(0x7f86c81052f0 msgr2=0x7f86c81077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 -- 172.21.15.94:0/3374572809 shutdown_connections 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.905+0000 7f86cf714700 1 -- 172.21.15.94:0/3374572809 wait complete. 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 Processor -- start 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 -- start start 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86c810e030 con 0x7f86c8109cd0 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cd4b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cd4b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59862/0 (socket says 172.21.15.94:59862) 2024-04-25T13:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cd4b0700 1 -- 172.21.15.94:0/641779938 learned_addr learned my addr 172.21.15.94:0/641779938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cd4b0700 1 -- 172.21.15.94:0/641779938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c807efa0 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cd4b0700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86c400bca0 tx=0x7f86c400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c4020020 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c807d1f0 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f86c4012430 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86c4017040 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.906+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c807d730 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.907+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c8064cf0 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.907+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f86c4023020 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.908+0000 7f86be7fc700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 0x7f86b403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.908+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86c400f050 con 0x7f86c8109cd0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.908+0000 7f86cccaf700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 0x7f86b403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:19.235 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.908+0000 7f86cccaf700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 0x7f86b403fa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:17.913+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86c402a020 con 0x7f86c8109cd0 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.108+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f86c807de90 con 0x7f86b403d520 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.638+0000 7f86be7fc700 1 -- 172.21.15.94:0/641779938 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86c807de90 con 0x7f86b403d520 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 msgr2=0x7f86b403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 0x7f86b403fa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 msgr2=0x7f86c807ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86c400bca0 tx=0x7f86c400bd80 comp rx=0 tx=0).stop 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 shutdown_connections 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f86b403d520 0x7f86b403fa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 --2- 172.21.15.94:0/641779938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86c8109cd0 0x7f86c807ea60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.640+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 >> 172.21.15.94:0/641779938 conn(0x7f86c81052f0 msgr2=0x7f86c8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.641+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 shutdown_connections 2024-04-25T13:32:19.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:18.641+0000 7f86cf714700 1 -- 172.21.15.94:0/641779938 wait complete. 2024-04-25T13:32:19.882 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:19 smithi094 ceph-mon[29195]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:19.882 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:19 smithi094 ceph-mon[29195]: Generating ssh key... 2024-04-25T13:32:19.882 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:19 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:19.883 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:19 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.671+0000 7f9eeeb82700 1 Processor -- start 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.671+0000 7f9eeeb82700 1 -- start start 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.671+0000 7f9eeeb82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.671+0000 7f9eeeb82700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81094e0 con 0x7f9ee8108ae0 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9eec91e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9eec91e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59876/0 (socket says 172.21.15.94:59876) 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9eec91e700 1 -- 172.21.15.94:0/4014912964 learned_addr learned my addr 172.21.15.94:0/4014912964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9eec91e700 1 -- 172.21.15.94:0/4014912964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee8109cb0 con 0x7f9ee8108ae0 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9eec91e700 1 --2- 172.21.15.94:0/4014912964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009e80 tx=0x7f9ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eaf4cfc53123a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9edf7fe700 1 -- 172.21.15.94:0/4014912964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4017070 con 0x7f9ee8108ae0 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9edf7fe700 1 -- 172.21.15.94:0/4014912964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9ee400b8c0 con 0x7f9ee8108ae0 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.672+0000 7f9edf7fe700 1 -- 172.21.15.94:0/4014912964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4004490 con 0x7f9ee8108ae0 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 -- 172.21.15.94:0/4014912964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 msgr2=0x7f9ee8108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/4014912964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009e80 tx=0x7f9ee400b330 comp rx=0 tx=0).stop 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 -- 172.21.15.94:0/4014912964 shutdown_connections 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/4014912964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee8108ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 -- 172.21.15.94:0/4014912964 >> 172.21.15.94:0/4014912964 conn(0x7f9ee8104120 msgr2=0x7f9ee81065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 -- 172.21.15.94:0/4014912964 shutdown_connections 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.673+0000 7f9eeeb82700 1 -- 172.21.15.94:0/4014912964 wait complete. 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eeeb82700 1 Processor -- start 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eeeb82700 1 -- start start 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eeeb82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eeeb82700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee810cee0 con 0x7f9ee8108ae0 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eec91e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eec91e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59888/0 (socket says 172.21.15.94:59888) 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eec91e700 1 -- 172.21.15.94:0/422265384 learned_addr learned my addr 172.21.15.94:0/422265384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.674+0000 7f9eec91e700 1 -- 172.21.15.94:0/422265384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee819d390 con 0x7f9ee8108ae0 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eec91e700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400bca0 tx=0x7f9ee400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:20.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4020020 con 0x7f9ee8108ae0 2024-04-25T13:32:20.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9ee4012430 con 0x7f9ee8108ae0 2024-04-25T13:32:20.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee819d620 con 0x7f9ee8108ae0 2024-04-25T13:32:20.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4017040 con 0x7f9ee8108ae0 2024-04-25T13:32:20.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.675+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee81a0230 con 0x7f9ee8108ae0 2024-04-25T13:32:20.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.676+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f9ee4023020 con 0x7f9ee8108ae0 2024-04-25T13:32:20.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.676+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee8064cf0 con 0x7f9ee8108ae0 2024-04-25T13:32:20.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.676+0000 7f9eddffb700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 0x7f9ed003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:20.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.676+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ee400f050 con 0x7f9ee8108ae0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.676+0000 7f9edffff700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 0x7f9ed003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.677+0000 7f9edffff700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 0x7f9ed003fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006fd0 tx=0x7f9ed8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.682+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ee402a020 con 0x7f9ee8108ae0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.881+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9ee8105150 con 0x7f9ed003d5c0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.882+0000 7f9eddffb700 1 -- 172.21.15.94:0/422265384 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9ee8105150 con 0x7f9ed003d5c0 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 msgr2=0x7f9ed003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 0x7f9ed003fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006fd0 tx=0x7f9ed8006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 msgr2=0x7f9ee819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400bca0 tx=0x7f9ee400bd80 comp rx=0 tx=0).stop 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 shutdown_connections 2024-04-25T13:32:20.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9ed003d5c0 0x7f9ed003fad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 --2- 172.21.15.94:0/422265384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee8108ae0 0x7f9ee819ce50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 >> 172.21.15.94:0/422265384 conn(0x7f9ee8104120 msgr2=0x7f9ee8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 shutdown_connections 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:19.884+0000 7f9eeeb82700 1 -- 172.21.15.94:0/422265384 wait complete. 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T13:32:20.355 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-04-25T13:32:20.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:20 smithi094 ceph-mon[29195]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:21.792 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:21 smithi094 ceph-mon[29195]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:21.793 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:21 smithi094 ceph-mon[29195]: Deploying cephadm binary to smithi094 2024-04-25T13:32:25.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-04-25T13:32:25.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.728+0000 7f132d7fd700 1 Processor -- start 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.728+0000 7f132d7fd700 1 -- start start 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f132d7fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f132d7fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1328076f80 con 0x7f132810a780 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59904/0 (socket says 172.21.15.94:59904) 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1326ffd700 1 -- 172.21.15.94:0/4029297371 learned_addr learned my addr 172.21.15.94:0/4029297371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1326ffd700 1 -- 172.21.15.94:0/4029297371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132810d0e0 con 0x7f132810a780 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1326ffd700 1 --2- 172.21.15.94:0/4029297371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463a814fbd165e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1325ffb700 1 -- 172.21.15.94:0/4029297371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318004160 con 0x7f132810a780 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1325ffb700 1 -- 172.21.15.94:0/4029297371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1318004300 con 0x7f132810a780 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.729+0000 7f1325ffb700 1 -- 172.21.15.94:0/4029297371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318011af0 con 0x7f132810a780 2024-04-25T13:32:25.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 -- 172.21.15.94:0/4029297371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 msgr2=0x7f132810cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4029297371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f1318009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 -- 172.21.15.94:0/4029297371 shutdown_connections 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4029297371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132810cba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 -- 172.21.15.94:0/4029297371 >> 172.21.15.94:0/4029297371 conn(0x7f1328104120 msgr2=0x7f13281065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 -- 172.21.15.94:0/4029297371 shutdown_connections 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.730+0000 7f132d7fd700 1 -- 172.21.15.94:0/4029297371 wait complete. 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f132d7fd700 1 Processor -- start 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f132d7fd700 1 -- start start 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f132d7fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f132d7fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f132810df00 con 0x7f132810a780 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59918/0 (socket says 172.21.15.94:59918) 2024-04-25T13:32:25.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.731+0000 7f1326ffd700 1 -- 172.21.15.94:0/4080166321 learned_addr learned my addr 172.21.15.94:0/4080166321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f1326ffd700 1 -- 172.21.15.94:0/4080166321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132819e480 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f1326ffd700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1318009fb0 tx=0x7f1318011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1318011bc0 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1318024440 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f131801a5c0 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132819e710 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.732+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281a1320 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.733+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f131801a830 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.733+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1328064cf0 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.733+0000 7f130ffff700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 0x7f131003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.733+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13180525c0 con 0x7f132810a780 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.733+0000 7f13267fc700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 0x7f131003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:25.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.734+0000 7f13267fc700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 0x7f131003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f131c006fd0 tx=0x7f131c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.738+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f131800b3e0 con 0x7f132810a780 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:20.944+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) -- 0x7f1328105240 con 0x7f131003d570 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.976+0000 7f130ffff700 1 -- 172.21.15.94:0/4080166321 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1328105240 con 0x7f131003d570 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 msgr2=0x7f131003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 0x7f131003fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f131c006fd0 tx=0x7f131c006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 msgr2=0x7f132819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1318009fb0 tx=0x7f1318011890 comp rx=0 tx=0).stop 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 shutdown_connections 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f131003d570 0x7f131003fa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 --2- 172.21.15.94:0/4080166321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f132810a780 0x7f132819df40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 >> 172.21.15.94:0/4080166321 conn(0x7f1328104120 msgr2=0x7f1328104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 shutdown_connections 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:24.978+0000 7f132d7fd700 1 -- 172.21.15.94:0/4080166321 wait complete. 2024-04-25T13:32:25.610 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-04-25T13:32:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:25 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:25 smithi094 ceph-mon[29195]: Added host smithi094 2024-04-25T13:32:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:25 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:27.063 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T13:32:27.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e11407700 1 Processor -- start 2024-04-25T13:32:27.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e11407700 1 -- start start 2024-04-25T13:32:27.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e11407700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e11407700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10bbc0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59356/0 (socket says 172.21.15.94:59356) 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e0affd700 1 -- 172.21.15.94:0/2100797221 learned_addr learned my addr 172.21.15.94:0/2100797221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.030+0000 7f5e0affd700 1 -- 172.21.15.94:0/2100797221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c10bd40 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e0affd700 1 --2- 172.21.15.94:0/2100797221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098a0 tx=0x7f5dfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a03bf3b3024c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e09ffb700 1 -- 172.21.15.94:0/2100797221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc004160 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e09ffb700 1 -- 172.21.15.94:0/2100797221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5dfc004300 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e09ffb700 1 -- 172.21.15.94:0/2100797221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc011af0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 -- 172.21.15.94:0/2100797221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 msgr2=0x7f5e0c10b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 --2- 172.21.15.94:0/2100797221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098a0 tx=0x7f5dfc009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 -- 172.21.15.94:0/2100797221 shutdown_connections 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 --2- 172.21.15.94:0/2100797221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c10b5f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:27.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 -- 172.21.15.94:0/2100797221 >> 172.21.15.94:0/2100797221 conn(0x7f5e0c1038a0 msgr2=0x7f5e0c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 -- 172.21.15.94:0/2100797221 shutdown_connections 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.031+0000 7f5e11407700 1 -- 172.21.15.94:0/2100797221 wait complete. 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.032+0000 7f5e11407700 1 Processor -- start 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.032+0000 7f5e11407700 1 -- start start 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.032+0000 7f5e11407700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.032+0000 7f5e11407700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1089b0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59360/0 (socket says 172.21.15.94:59360) 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e0affd700 1 -- 172.21.15.94:0/1087707969 learned_addr learned my addr 172.21.15.94:0/1087707969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e0affd700 1 -- 172.21.15.94:0/1087707969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19e470 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e0affd700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009fb0 tx=0x7f5dfc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc011bc0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5dfc024440 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc01a5c0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19e700 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.033+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a1310 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.034+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f5dfc01a830 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.034+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c064cf0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.034+0000 7f5df3fff700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 0x7f5df403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.034+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5dfc0525e0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.034+0000 7f5e0a7fc700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 0x7f5df403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.035+0000 7f5e0a7fc700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 0x7f5df403fa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006fd0 tx=0x7f5e00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.040+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dfc00b3e0 con 0x7f5e0c10b1e0 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.256+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5e0c104980 con 0x7f5df403d570 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.262+0000 7f5df3fff700 1 -- 172.21.15.94:0/1087707969 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5e0c104980 con 0x7f5df403d570 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 msgr2=0x7f5df403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:27.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 0x7f5df403fa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006fd0 tx=0x7f5e00006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 msgr2=0x7f5e0c19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009fb0 tx=0x7f5dfc011890 comp rx=0 tx=0).stop 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 shutdown_connections 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f5df403d570 0x7f5df403fa80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 --2- 172.21.15.94:0/1087707969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c10b1e0 0x7f5e0c19df30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 >> 172.21.15.94:0/1087707969 conn(0x7f5e0c1038a0 msgr2=0x7f5e0c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 shutdown_connections 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:26.265+0000 7f5e11407700 1 -- 172.21.15.94:0/1087707969 wait complete. 2024-04-25T13:32:27.072 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-04-25T13:32:27.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:27 smithi094 ceph-mon[29195]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:27.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:27 smithi094 ceph-mon[29195]: Saving service mon spec with placement count:5 2024-04-25T13:32:27.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:27 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:28.771 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T13:32:28.771 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee302b700 1 Processor -- start 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee302b700 1 -- start start 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee302b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee302b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc10aaa0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee0dc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee0dc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59366/0 (socket says 172.21.15.94:59366) 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee0dc7700 1 -- 172.21.15.94:0/2109273251 learned_addr learned my addr 172.21.15.94:0/2109273251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.832+0000 7f0ee0dc7700 1 -- 172.21.15.94:0/2109273251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc108d20 con 0x7f0edc10a0a0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0ee0dc7700 1 --2- 172.21.15.94:0/2109273251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00098a0 tx=0x7f0ed0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfa0b2df4038a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0edb7fe700 1 -- 172.21.15.94:0/2109273251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed0004160 con 0x7f0edc10a0a0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0edb7fe700 1 -- 172.21.15.94:0/2109273251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ed0004300 con 0x7f0edc10a0a0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0edb7fe700 1 -- 172.21.15.94:0/2109273251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed0011af0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0ee302b700 1 -- 172.21.15.94:0/2109273251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 msgr2=0x7f0edc10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:28.772 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.833+0000 7f0ee302b700 1 --2- 172.21.15.94:0/2109273251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00098a0 tx=0x7f0ed0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- 172.21.15.94:0/2109273251 shutdown_connections 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 --2- 172.21.15.94:0/2109273251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc10a4b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- 172.21.15.94:0/2109273251 >> 172.21.15.94:0/2109273251 conn(0x7f0edc1038a0 msgr2=0x7f0edc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- 172.21.15.94:0/2109273251 shutdown_connections 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- 172.21.15.94:0/2109273251 wait complete. 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 Processor -- start 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- start start 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee302b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc10cf60 con 0x7f0edc10a0a0 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee0dc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee0dc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59370/0 (socket says 172.21.15.94:59370) 2024-04-25T13:32:28.773 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.834+0000 7f0ee0dc7700 1 -- 172.21.15.94:0/4097419307 learned_addr learned my addr 172.21.15.94:0/4097419307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ee0dc7700 1 -- 172.21.15.94:0/4097419307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc105f80 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ee0dc7700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0011670 tx=0x7f0ed0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed00036a0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ed001a430 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed0019550 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc1041d0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.835+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc1046b0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.836+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f0ed001a5e0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.836+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc064cf0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.837+0000 7f0ed9ffb700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 0x7f0ecc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.837+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ed00525e0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.774 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.837+0000 7f0edbfff700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 0x7f0ecc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.837+0000 7f0edbfff700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 0x7f0ecc03fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8006fd0 tx=0x7f0ec8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:27.842+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ed000b3e0 con 0x7f0edc10a0a0 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.055+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0edc1970f0 con 0x7f0ecc03d570 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.060+0000 7f0ed9ffb700 1 -- 172.21.15.94:0/4097419307 <== mgr.14120 v2:172.21.15.94:6800/1021554684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0edc1970f0 con 0x7f0ecc03d570 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 msgr2=0x7f0ecc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 0x7f0ecc03fa80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8006fd0 tx=0x7f0ec8006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 msgr2=0x7f0edc105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0011670 tx=0x7f0ed0004280 comp rx=0 tx=0).stop 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 shutdown_connections 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f0ecc03d570 0x7f0ecc03fa80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 --2- 172.21.15.94:0/4097419307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0edc10a0a0 0x7f0edc105a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 >> 172.21.15.94:0/4097419307 conn(0x7f0edc1038a0 msgr2=0x7f0edc1944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 shutdown_connections 2024-04-25T13:32:28.775 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:28.063+0000 7f0ee302b700 1 -- 172.21.15.94:0/4097419307 wait complete. 2024-04-25T13:32:29.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:29 smithi094 ceph-mon[29195]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:29.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:29 smithi094 ceph-mon[29195]: Saving service mgr spec with placement count:2 2024-04-25T13:32:29.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:29 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:29.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:29 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:29.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:29 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.222+0000 7f9b5ddbe700 1 Processor -- start 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.222+0000 7f9b5ddbe700 1 -- start start 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.222+0000 7f9b5ddbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.222+0000 7f9b5ddbe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58077040 con 0x7f9b5810a690 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59384/0 (socket says 172.21.15.94:59384) 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b577fe700 1 -- 172.21.15.94:0/3235881400 learned_addr learned my addr 172.21.15.94:0/3235881400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b577fe700 1 -- 172.21.15.94:0/3235881400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5810cff0 con 0x7f9b5810a690 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b577fe700 1 --2- 172.21.15.94:0/3235881400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b480098a0 tx=0x7f9b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7cf5d303ecc516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b567fc700 1 -- 172.21.15.94:0/3235881400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48004160 con 0x7f9b5810a690 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b567fc700 1 -- 172.21.15.94:0/3235881400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b48004300 con 0x7f9b5810a690 2024-04-25T13:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.223+0000 7f9b567fc700 1 -- 172.21.15.94:0/3235881400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48011a20 con 0x7f9b5810a690 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/3235881400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 msgr2=0x7f9b5810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/3235881400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b480098a0 tx=0x7f9b48009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/3235881400 shutdown_connections 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/3235881400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5810cab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/3235881400 >> 172.21.15.94:0/3235881400 conn(0x7f9b581040d0 msgr2=0x7f9b581065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/3235881400 shutdown_connections 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.224+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/3235881400 wait complete. 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b5ddbe700 1 Processor -- start 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b5ddbe700 1 -- start start 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b5ddbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b5ddbe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5810dff0 con 0x7f9b5810a690 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b577fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59396/0 (socket says 172.21.15.94:59396) 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.225+0000 7f9b577fe700 1 -- 172.21.15.94:0/1593926876 learned_addr learned my addr 172.21.15.94:0/1593926876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b577fe700 1 -- 172.21.15.94:0/1593926876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5819e410 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b577fe700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009520 tx=0x7f9b48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b480039e0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5819e6a0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581a12b0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4801b8f0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.226+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b480197d0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.227+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f9b48019ac0 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.227+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b38005360 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.227+0000 7f9b54ff9700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 0x7f9b4403fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.227+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b48052d50 con 0x7f9b5810a690 2024-04-25T13:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.228+0000 7f9b56ffd700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 0x7f9b4403fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.228+0000 7f9b56ffd700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 0x7f9b4403fad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006fd0 tx=0x7f9b40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.233+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b4801f030 con 0x7f9b5810a690 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.463+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9b38005650 con 0x7f9b5810a690 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.466+0000 7f9b54ff9700 1 -- 172.21.15.94:0/1593926876 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f9b4802be20 con 0x7f9b5810a690 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.468+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 msgr2=0x7f9b4403fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.468+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 0x7f9b4403fad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006fd0 tx=0x7f9b40006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.468+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 msgr2=0x7f9b5819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.468+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009520 tx=0x7f9b48011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 shutdown_connections 2024-04-25T13:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f9b4403d5c0 0x7f9b4403fad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 --2- 172.21.15.94:0/1593926876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5810a690 0x7f9b5819ded0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 >> 172.21.15.94:0/1593926876 conn(0x7f9b581040d0 msgr2=0x7f9b58104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 shutdown_connections 2024-04-25T13:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:29.469+0000 7f9b5ddbe700 1 -- 172.21.15.94:0/1593926876 wait complete. 2024-04-25T13:32:30.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:30 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1593926876' entity='client.admin' 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.001+0000 7f3b9a260700 1 Processor -- start 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.001+0000 7f3b9a260700 1 -- start start 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b9a260700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b9a260700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941094e0 con 0x7f3b94108ae0 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b937fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b937fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59404/0 (socket says 172.21.15.94:59404) 2024-04-25T13:32:31.815 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b937fe700 1 -- 172.21.15.94:0/2625336576 learned_addr learned my addr 172.21.15.94:0/2625336576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b937fe700 1 -- 172.21.15.94:0/2625336576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b94109cb0 con 0x7f3b94108ae0 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b937fe700 1 --2- 172.21.15.94:0/2625336576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b840098a0 tx=0x7f3b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83156a87d9e1055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b92ffd700 1 -- 172.21.15.94:0/2625336576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b84004160 con 0x7f3b94108ae0 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b92ffd700 1 -- 172.21.15.94:0/2625336576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b84004300 con 0x7f3b94108ae0 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.002+0000 7f3b92ffd700 1 -- 172.21.15.94:0/2625336576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b84011a20 con 0x7f3b94108ae0 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 -- 172.21.15.94:0/2625336576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 msgr2=0x7f3b94108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2625336576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3b840098a0 tx=0x7f3b84009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 -- 172.21.15.94:0/2625336576 shutdown_connections 2024-04-25T13:32:31.816 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2625336576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b94108ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 -- 172.21.15.94:0/2625336576 >> 172.21.15.94:0/2625336576 conn(0x7f3b94104120 msgr2=0x7f3b941065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 -- 172.21.15.94:0/2625336576 shutdown_connections 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.003+0000 7f3b9a260700 1 -- 172.21.15.94:0/2625336576 wait complete. 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b9a260700 1 Processor -- start 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b9a260700 1 -- start start 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b9a260700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b9a260700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9410d060 con 0x7f3b94108ae0 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b937fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b937fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59406/0 (socket says 172.21.15.94:59406) 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.004+0000 7f3b937fe700 1 -- 172.21.15.94:0/2895597481 learned_addr learned my addr 172.21.15.94:0/2895597481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:31.817 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b937fe700 1 -- 172.21.15.94:0/2895597481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9419d410 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b937fe700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009130 tx=0x7f3b84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b84003aa0 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9419d6a0 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8401a8e0 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b840248f0 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.005+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b941a02b0 con 0x7f3b94108ae0 2024-04-25T13:32:31.818 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.006+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b94064cf0 con 0x7f3b94108ae0 2024-04-25T13:32:31.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.006+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f3b84022020 con 0x7f3b94108ae0 2024-04-25T13:32:31.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.006+0000 7f3b917fa700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 0x7f3b8003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:31.819 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.006+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b84052590 con 0x7f3b94108ae0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.006+0000 7f3b8affd700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 0x7f3b8003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.007+0000 7f3b8affd700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 0x7f3b8003fad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006fd0 tx=0x7f3b7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.012+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b84029df0 con 0x7f3b94108ae0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.232+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3b941a0480 con 0x7f3b94108ae0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.235+0000 7f3b917fa700 1 -- 172.21.15.94:0/2895597481 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3b8401aa90 con 0x7f3b94108ae0 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 msgr2=0x7f3b8003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 0x7f3b8003fad0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006fd0 tx=0x7f3b7c006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:31.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 msgr2=0x7f3b9419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009130 tx=0x7f3b84011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 shutdown_connections 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3b8003d5c0 0x7f3b8003fad0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 --2- 172.21.15.94:0/2895597481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b94108ae0 0x7f3b9419ced0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 >> 172.21.15.94:0/2895597481 conn(0x7f3b94104120 msgr2=0x7f3b94104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 shutdown_connections 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:31.238+0000 7f3b9a260700 1 -- 172.21.15.94:0/2895597481 wait complete. 2024-04-25T13:32:31.822 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-04-25T13:32:32.394 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:32 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:32.394 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:32 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2895597481' entity='client.admin' 2024-04-25T13:32:33.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2595447100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T13:32:33.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-mon[29195]: from='mgr.14120 172.21.15.94:0/619023692' entity='mgr.x' 2024-04-25T13:32:33.457 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setuser ceph since I am not root 2024-04-25T13:32:33.457 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setgroup ceph since I am not root 2024-04-25T13:32:33.457 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:33.347+0000 7f3f35752200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:32:33.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:33.547+0000 7f3f35752200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:32:33.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:33.651+0000 7f3f35752200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.279+0000 7fcf7f4ac700 1 Processor -- start 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.279+0000 7fcf7f4ac700 1 -- start start 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7f4ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7f4ac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781094b0 con 0x7fcf78108ab0 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7d248700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:33.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7d248700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59422/0 (socket says 172.21.15.94:59422) 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7d248700 1 -- 172.21.15.94:0/2137102493 learned_addr learned my addr 172.21.15.94:0/2137102493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7d248700 1 -- 172.21.15.94:0/2137102493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf78109c80 con 0x7fcf78108ab0 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf7d248700 1 --2- 172.21.15.94:0/2137102493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf740098a0 tx=0x7fcf74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8639bb9a57daa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf6ffff700 1 -- 172.21.15.94:0/2137102493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf74004160 con 0x7fcf78108ab0 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf6ffff700 1 -- 172.21.15.94:0/2137102493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf74004300 con 0x7fcf78108ab0 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.280+0000 7fcf6ffff700 1 -- 172.21.15.94:0/2137102493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf74011a20 con 0x7fcf78108ab0 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2137102493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 msgr2=0x7fcf78108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2137102493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf740098a0 tx=0x7fcf74009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2137102493 shutdown_connections 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2137102493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf78108ec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2137102493 >> 172.21.15.94:0/2137102493 conn(0x7fcf781040d0 msgr2=0x7fcf781065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2137102493 shutdown_connections 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.281+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2137102493 wait complete. 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7f4ac700 1 Processor -- start 2024-04-25T13:32:33.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7f4ac700 1 -- start start 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7f4ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7f4ac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7810d030 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7d248700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7d248700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59432/0 (socket says 172.21.15.94:59432) 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.282+0000 7fcf7d248700 1 -- 172.21.15.94:0/2595447100 learned_addr learned my addr 172.21.15.94:0/2595447100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf7d248700 1 -- 172.21.15.94:0/2595447100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7819e490 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf7d248700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf74011a20 tx=0x7fcf74011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf740039c0 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf74003b60 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7819e720 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf74019700 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.283+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781a1330 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.284+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78064cf0 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.284+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fcf74019a00 con 0x7fcf78108ab0 2024-04-25T13:32:33.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.284+0000 7fcf6e7fc700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 0x7fcf6403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.284+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf74052800 con 0x7fcf78108ab0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.284+0000 7fcf7ca47700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 0x7fcf6403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.285+0000 7fcf7ca47700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 0x7fcf6403fa30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.290+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf74020e20 con 0x7fcf78108ab0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:32.557+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcf7802e7d0 con 0x7fcf78108ab0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.124+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcf7400b3e0 con 0x7fcf78108ab0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.126+0000 7fcf6e7fc700 1 -- 172.21.15.94:0/2595447100 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fcf74054cc0 con 0x7fcf78108ab0 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 msgr2=0x7fcf6403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 0x7fcf6403fa30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 msgr2=0x7fcf7819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf74011a20 tx=0x7fcf74011b00 comp rx=0 tx=0).stop 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 shutdown_connections 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7fcf6403d520 0x7fcf6403fa30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 --2- 172.21.15.94:0/2595447100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78108ab0 0x7fcf7819df50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 >> 172.21.15.94:0/2595447100 conn(0x7fcf781040d0 msgr2=0x7fcf781944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:33.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 shutdown_connections 2024-04-25T13:32:33.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:33.128+0000 7fcf7f4ac700 1 -- 172.21.15.94:0/2595447100 wait complete. 2024-04-25T13:32:34.066 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:33.970+0000 7f3f35752200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:32:34.066 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:34.057+0000 7f3f35752200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:32:34.436 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2595447100' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T13:32:34.436 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:34 smithi094 ceph-mon[29195]: mgrmap e9: x(active, since 19s) 2024-04-25T13:32:35.407 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3286255794' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:32:35.407 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:35.293+0000 7f3f35752200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:32:35.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c2d81700 1 Processor -- start 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c2d81700 1 -- start start 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c2d81700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c2d81700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc07ce70 con 0x7f73bc10a610 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c0b1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c0b1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55106/0 (socket says 172.21.15.94:55106) 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c0b1d700 1 -- 172.21.15.94:0/3906646152 learned_addr learned my addr 172.21.15.94:0/3906646152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.280+0000 7f73c0b1d700 1 -- 172.21.15.94:0/3906646152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc10cf70 con 0x7f73bc10a610 2024-04-25T13:32:35.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.281+0000 7f73c0b1d700 1 --2- 172.21.15.94:0/3906646152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73b00098a0 tx=0x7f73b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1896a45c31d9f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.281+0000 7f73bb7fe700 1 -- 172.21.15.94:0/3906646152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b0004160 con 0x7f73bc10a610 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.281+0000 7f73bb7fe700 1 -- 172.21.15.94:0/3906646152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b0004300 con 0x7f73bc10a610 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.281+0000 7f73bb7fe700 1 -- 172.21.15.94:0/3906646152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b0011a20 con 0x7f73bc10a610 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 -- 172.21.15.94:0/3906646152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 msgr2=0x7f73bc10ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3906646152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73b00098a0 tx=0x7f73b0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 -- 172.21.15.94:0/3906646152 shutdown_connections 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3906646152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc10ca30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 -- 172.21.15.94:0/3906646152 >> 172.21.15.94:0/3906646152 conn(0x7f73bc1040d0 msgr2=0x7f73bc106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 -- 172.21.15.94:0/3906646152 shutdown_connections 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.282+0000 7f73c2d81700 1 -- 172.21.15.94:0/3906646152 wait complete. 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.283+0000 7f73c2d81700 1 Processor -- start 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.283+0000 7f73c2d81700 1 -- start start 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.283+0000 7f73c2d81700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:35.417 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.283+0000 7f73c2d81700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc10df70 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c0b1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c0b1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55116/0 (socket says 172.21.15.94:55116) 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c0b1d700 1 -- 172.21.15.94:0/3286255794 learned_addr learned my addr 172.21.15.94:0/3286255794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c0b1d700 1 -- 172.21.15.94:0/3286255794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc19e3b0 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c0b1d700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009520 tx=0x7f73b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b00039e0 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc19e640 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b001b8f0 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.284+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b00197d0 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.285+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc1a1250 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73bc064cf0 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f73b0019a40 con 0x7f73bc10a610 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73b9ffb700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 0x7f73ac03fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:35.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f73b0052da0 con 0x7f73bc10a610 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73bbfff700 1 -- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 msgr2=0x7f73ac03fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.286+0000 7f73bbfff700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 0x7f73ac03fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.292+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73b001f030 con 0x7f73bc10a610 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.487+0000 7f73bbfff700 1 -- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 msgr2=0x7f73ac03fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.487+0000 7f73bbfff700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 0x7f73ac03fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.607+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f73bc1a1420 con 0x7f73bc10a610 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.607+0000 7f73b9ffb700 1 -- 172.21.15.94:0/3286255794 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f73b002a350 con 0x7f73bc10a610 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 msgr2=0x7f73ac03fb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 0x7f73ac03fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 msgr2=0x7f73bc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009520 tx=0x7f73b0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 shutdown_connections 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f73ac03d620 0x7f73ac03fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 --2- 172.21.15.94:0/3286255794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73bc10a610 0x7f73bc19de70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:35.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.609+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 >> 172.21.15.94:0/3286255794 conn(0x7f73bc1040d0 msgr2=0x7f73bc1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:35.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.610+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 shutdown_connections 2024-04-25T13:32:35.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:34.610+0000 7f73c2d81700 1 -- 172.21.15.94:0/3286255794 wait complete. 2024-04-25T13:32:35.420 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-25T13:32:35.420 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-04-25T13:32:35.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:35.475+0000 7f3f35752200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:32:35.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:35.648+0000 7f3f35752200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:32:36.047 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:35.948+0000 7f3f35752200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:32:36.048 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:36.046+0000 7f3f35752200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:32:36.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:36.258+0000 7f3f35752200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:32:36.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:36.461+0000 7f3f35752200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:32:37.225 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:37.066+0000 7f3f35752200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:32:37.804 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:37.498+0000 7f3f35752200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:32:37.808 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:37.597+0000 7f3f35752200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:32:37.809 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:37.802+0000 7f3f35752200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:32:38.671 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:38.324+0000 7f3f35752200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:32:38.672 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:38.420+0000 7f3f35752200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:32:38.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:38.670+0000 7f3f35752200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:32:38.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:38.806+0000 7f3f35752200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:32:38.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:38.904+0000 7f3f35752200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:32:39.302 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:39.024+0000 7f3f35752200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:32:39.725 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:39.335+0000 7f3f35752200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:32:40.219 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:39.941+0000 7f3f35752200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:32:40.219 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:32:40.165+0000 7f3f35752200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:32:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: Activating manager daemon x 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: mgrmap e10: x(active, starting, since 0.00414979s) 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T13:32:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:41.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:32:41.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:32:41.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:40] ENGINE Bus STARTING 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:40] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:40] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: [25/Apr/2024:13:32:40] ENGINE Bus STARTED 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: mgrmap e11: x(active, since 1.00747s) 2024-04-25T13:32:41.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:41 smithi094 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.289+0000 7f3a21232700 1 Processor -- start 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.289+0000 7f3a21232700 1 -- start start 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.289+0000 7f3a21232700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.289+0000 7f3a21232700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c077000 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a1ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a1ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55128/0 (socket says 172.21.15.94:55128) 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a1ad9d700 1 -- 172.21.15.94:0/4131038216 learned_addr learned my addr 172.21.15.94:0/4131038216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a1ad9d700 1 -- 172.21.15.94:0/4131038216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c10d030 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a1ad9d700 1 --2- 172.21.15.94:0/4131038216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f51bf72629b5d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.290+0000 7f3a19d9b700 1 -- 172.21.15.94:0/4131038216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c004160 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a19d9b700 1 -- 172.21.15.94:0/4131038216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c004300 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a19d9b700 1 -- 172.21.15.94:0/4131038216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c011a20 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 -- 172.21.15.94:0/4131038216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 msgr2=0x7f3a1c10caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 --2- 172.21.15.94:0/4131038216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 -- 172.21.15.94:0/4131038216 shutdown_connections 2024-04-25T13:32:42.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 --2- 172.21.15.94:0/4131038216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c10caf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 -- 172.21.15.94:0/4131038216 >> 172.21.15.94:0/4131038216 conn(0x7f3a1c1040d0 msgr2=0x7f3a1c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 -- 172.21.15.94:0/4131038216 shutdown_connections 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.291+0000 7f3a21232700 1 -- 172.21.15.94:0/4131038216 wait complete. 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.292+0000 7f3a21232700 1 Processor -- start 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a21232700 1 -- start start 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a21232700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a21232700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c10dfd0 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a1ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a1ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55144/0 (socket says 172.21.15.94:55144) 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a1ad9d700 1 -- 172.21.15.94:0/2530281160 learned_addr learned my addr 172.21.15.94:0/2530281160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.293+0000 7f3a1ad9d700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c19e3f0 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a1ad9d700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c004660 tx=0x7f3a0c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c003a60 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c01b8f0 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c19e680 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c023890 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.294+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c1a1290 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f3a0c02c3f0 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a03fff700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3a04040190 con 0x7f3a0403d570 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a0c052df0 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a1a59c700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.295+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.496+0000 7f3a1a59c700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.496+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:32:42.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.897+0000 7f3a1a59c700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:36.897+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:37.698+0000 7f3a1a59c700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:37.698+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:39.299+0000 7f3a1a59c700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1021554684 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:39.299+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:40.169+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7f3a0c02cb00 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:40.169+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 msgr2=0x7f3a0403fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:40.169+0000 7f3a03fff700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.171+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f3a0c00fd80 con 0x7f3a1c10a6d0 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.171+0000 7f3a03fff700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 0x7f3a04043000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.171+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3a04040190 con 0x7f3a0c052980 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.171+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 0x7f3a04043000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.172+0000 7f3a1a59c700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 0x7f3a04043000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003a10 tx=0x7f3a100092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.173+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3a04040190 con 0x7f3a0c052980 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.181+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3a1c064cf0 con 0x7f3a0c052980 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a03fff700 1 -- 172.21.15.94:0/2530281160 <== mgr.14152 v2:172.21.15.94:6800/449370799 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3a1c064cf0 con 0x7f3a0c052980 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 msgr2=0x7f3a04043000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 0x7f3a04043000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003a10 tx=0x7f3a100092b0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 msgr2=0x7f3a1c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c004660 tx=0x7f3a0c011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 shutdown_connections 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a0c052980 0x7f3a04043000 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a1c10a6d0 0x7f3a1c19deb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 --2- 172.21.15.94:0/2530281160 >> [v2:172.21.15.94:6800/1021554684,v1:172.21.15.94:6801/1021554684] conn(0x7f3a0403d570 0x7f3a0403fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 >> 172.21.15.94:0/2530281160 conn(0x7f3a1c1040d0 msgr2=0x7f3a1c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 shutdown_connections 2024-04-25T13:32:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:41.182+0000 7f3a21232700 1 -- 172.21.15.94:0/2530281160 wait complete. 2024-04-25T13:32:42.177 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-04-25T13:32:42.177 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-04-25T13:32:43.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d9299700 1 Processor -- start 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d9299700 1 -- start start 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d9299700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d9299700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41094e0 con 0x7f17d4108ae0 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:43.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55210/0 (socket says 172.21.15.94:55210) 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.630+0000 7f17d2ffd700 1 -- 172.21.15.94:0/3322749709 learned_addr learned my addr 172.21.15.94:0/3322749709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.631+0000 7f17d2ffd700 1 -- 172.21.15.94:0/3322749709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d4109cb0 con 0x7f17d4108ae0 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.631+0000 7f17d2ffd700 1 --2- 172.21.15.94:0/3322749709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17c40098a0 tx=0x7f17c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27f3f22761a4fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.631+0000 7f17d1ffb700 1 -- 172.21.15.94:0/3322749709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4004160 con 0x7f17d4108ae0 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.631+0000 7f17d1ffb700 1 -- 172.21.15.94:0/3322749709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c4004300 con 0x7f17d4108ae0 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.631+0000 7f17d1ffb700 1 -- 172.21.15.94:0/3322749709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4011a20 con 0x7f17d4108ae0 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- 172.21.15.94:0/3322749709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 msgr2=0x7f17d4108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 --2- 172.21.15.94:0/3322749709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17c40098a0 tx=0x7f17c4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:43.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- 172.21.15.94:0/3322749709 shutdown_connections 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 --2- 172.21.15.94:0/3322749709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4108ef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- 172.21.15.94:0/3322749709 >> 172.21.15.94:0/3322749709 conn(0x7f17d4104120 msgr2=0x7f17d41065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- 172.21.15.94:0/3322749709 shutdown_connections 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- 172.21.15.94:0/3322749709 wait complete. 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 Processor -- start 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.632+0000 7f17d9299700 1 -- start start 2024-04-25T13:32:43.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d9299700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d9299700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17d410d060 con 0x7f17d4108ae0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55212/0 (socket says 172.21.15.94:55212) 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d2ffd700 1 -- 172.21.15.94:0/280760635 learned_addr learned my addr 172.21.15.94:0/280760635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d2ffd700 1 -- 172.21.15.94:0/280760635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d4077050 con 0x7f17d4108ae0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d2ffd700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17c4011a20 tx=0x7f17c4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c40039c0 con 0x7f17d4108ae0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c4003b60 con 0x7f17d4108ae0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41a0d80 con 0x7f17d4108ae0 2024-04-25T13:32:43.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.633+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4019700 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.634+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a1200 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.634+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d4064cf0 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.634+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f17c4019a00 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.635+0000 7f17bbfff700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 0x7f17bc03f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.635+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17c4052a50 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.635+0000 7f17d27fc700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 0x7f17bc03f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.635+0000 7f17d27fc700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 0x7f17bc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.640+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17c4020e20 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.827+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f17c402d430 con 0x7f17d4108ae0 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.863+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f17d4105220 con 0x7f17bc03d460 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.922+0000 7f17bbfff700 1 -- 172.21.15.94:0/280760635 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f17d4105220 con 0x7f17bc03d460 2024-04-25T13:32:43.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 msgr2=0x7f17bc03f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 0x7f17bc03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 msgr2=0x7f17d4076b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17c4011a20 tx=0x7f17c4011b00 comp rx=0 tx=0).stop 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 shutdown_connections 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17bc03d460 0x7f17bc03f970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 --2- 172.21.15.94:0/280760635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17d4108ae0 0x7f17d4076b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 >> 172.21.15.94:0/280760635 conn(0x7f17d4104120 msgr2=0x7f17d4104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:43.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 shutdown_connections 2024-04-25T13:32:43.505 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:42.924+0000 7f17d9299700 1 -- 172.21.15.94:0/280760635 wait complete. 2024-04-25T13:32:43.505 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-04-25T13:32:43.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:43 smithi094 ceph-mon[29195]: mgrmap e12: x(active, since 2s) 2024-04-25T13:32:43.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:43 smithi094 ceph-mon[29195]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:43.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:43.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:43.920 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:44 smithi094 ceph-mon[29195]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.ozHLqv3lXChaY.7VGE7iOXXaOhsGzyTbj3FQx7JKwCOmR3hREUxS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714051964, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.949+0000 7f039c0c6700 1 Processor -- start 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f039c0c6700 1 -- start start 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f039c0c6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f039c0c6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0394076fe0 con 0x7f039410a720 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f0399e62700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:45.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f0399e62700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43602/0 (socket says 172.21.15.94:43602) 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f0399e62700 1 -- 172.21.15.94:0/3848393865 learned_addr learned my addr 172.21.15.94:0/3848393865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.950+0000 7f0399e62700 1 -- 172.21.15.94:0/3848393865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039410d080 con 0x7f039410a720 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.951+0000 7f0399e62700 1 --2- 172.21.15.94:0/3848393865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03900098a0 tx=0x7f0390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0f671ba8383617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.951+0000 7f0398e60700 1 -- 172.21.15.94:0/3848393865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390004160 con 0x7f039410a720 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.951+0000 7f0398e60700 1 -- 172.21.15.94:0/3848393865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0390004300 con 0x7f039410a720 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.951+0000 7f0398e60700 1 -- 172.21.15.94:0/3848393865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390011a20 con 0x7f039410a720 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 -- 172.21.15.94:0/3848393865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 msgr2=0x7f039410cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 --2- 172.21.15.94:0/3848393865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03900098a0 tx=0x7f0390009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 -- 172.21.15.94:0/3848393865 shutdown_connections 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 --2- 172.21.15.94:0/3848393865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039410cb40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 -- 172.21.15.94:0/3848393865 >> 172.21.15.94:0/3848393865 conn(0x7f0394104120 msgr2=0x7f03941065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 -- 172.21.15.94:0/3848393865 shutdown_connections 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 -- 172.21.15.94:0/3848393865 wait complete. 2024-04-25T13:32:45.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.952+0000 7f039c0c6700 1 Processor -- start 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f039c0c6700 1 -- start start 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f039c0c6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f039c0c6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f039410e020 con 0x7f039410a720 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f0399e62700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f0399e62700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43606/0 (socket says 172.21.15.94:43606) 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f0399e62700 1 -- 172.21.15.94:0/2296843064 learned_addr learned my addr 172.21.15.94:0/2296843064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f0399e62700 1 -- 172.21.15.94:0/2296843064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039419e440 con 0x7f039410a720 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f0399e62700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0390004660 tx=0x7f0390011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390003a60 con 0x7f039410a720 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039001b8f0 con 0x7f039410a720 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.953+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039419e6d0 con 0x7f039410a720 2024-04-25T13:32:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.954+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0390023890 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.954+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03941a12e0 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f039001b440 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0394064cf0 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f038affd700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 0x7f038003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0390052e30 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f0399661700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 0x7f038003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.955+0000 7f0399661700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 0x7f038003fae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0384006fd0 tx=0x7f0384006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:43.965+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f039001f030 con 0x7f039410a720 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.192+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f03941a1570 con 0x7f038003d5d0 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.417+0000 7f038affd700 1 -- 172.21.15.94:0/2296843064 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f03941a1570 con 0x7f038003d5d0 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 msgr2=0x7f038003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 0x7f038003fae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0384006fd0 tx=0x7f0384006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 msgr2=0x7f039419df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:45.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0390004660 tx=0x7f0390011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 shutdown_connections 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f038003d5d0 0x7f038003fae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 --2- 172.21.15.94:0/2296843064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f039410a720 0x7f039419df00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 >> 172.21.15.94:0/2296843064 conn(0x7f0394104120 msgr2=0x7f0394104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 shutdown_connections 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:44.419+0000 7f039c0c6700 1 -- 172.21.15.94:0/2296843064 wait complete. 2024-04-25T13:32:45.240 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-04-25T13:32:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/990742773' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T13:32:46.685 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T13:32:46.685 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed18b6b700 1 Processor -- start 2024-04-25T13:32:46.685 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed18b6b700 1 -- start start 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed18b6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed18b6b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed1407d840 con 0x7fed1407ee40 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43622/0 (socket says 172.21.15.94:43622) 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.686+0000 7fed1259c700 1 -- 172.21.15.94:0/2780188334 learned_addr learned my addr 172.21.15.94:0/2780188334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.687+0000 7fed1259c700 1 -- 172.21.15.94:0/2780188334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1407d9c0 con 0x7fed1407ee40 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.687+0000 7fed1259c700 1 --2- 172.21.15.94:0/2780188334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed040098a0 tx=0x7fed04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2509265466bd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.687+0000 7fed1159a700 1 -- 172.21.15.94:0/2780188334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed04004160 con 0x7fed1407ee40 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.687+0000 7fed1159a700 1 -- 172.21.15.94:0/2780188334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed04004300 con 0x7fed1407ee40 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.687+0000 7fed1159a700 1 -- 172.21.15.94:0/2780188334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed04011a20 con 0x7fed1407ee40 2024-04-25T13:32:46.686 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 -- 172.21.15.94:0/2780188334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 msgr2=0x7fed1407d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 --2- 172.21.15.94:0/2780188334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed040098a0 tx=0x7fed04009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 -- 172.21.15.94:0/2780188334 shutdown_connections 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 --2- 172.21.15.94:0/2780188334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed1407d270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 -- 172.21.15.94:0/2780188334 >> 172.21.15.94:0/2780188334 conn(0x7fed14105210 msgr2=0x7fed14107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 -- 172.21.15.94:0/2780188334 shutdown_connections 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.688+0000 7fed18b6b700 1 -- 172.21.15.94:0/2780188334 wait complete. 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed18b6b700 1 Processor -- start 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed18b6b700 1 -- start start 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed18b6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed18b6b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed141101c0 con 0x7fed1407ee40 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed1259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43626/0 (socket says 172.21.15.94:43626) 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed1259c700 1 -- 172.21.15.94:0/990742773 learned_addr learned my addr 172.21.15.94:0/990742773 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed1259c700 1 -- 172.21.15.94:0/990742773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed141a0570 con 0x7fed1407ee40 2024-04-25T13:32:46.687 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.689+0000 7fed1259c700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fed040119f0 tx=0x7fed04011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.690+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed040039e0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.690+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed0401b8f0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.690+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141a0800 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.690+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed040237a0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.690+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a3410 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14064cf0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7fed0402c3f0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed037fe700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 0x7fecfc03fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed04052d70 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed11d9b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 0x7fecfc03fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.691+0000 7fed11d9b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 0x7fecfc03fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed08006fd0 tx=0x7fed08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.696+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed04057070 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.898+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fed141a35e0 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.899+0000 7fed037fe700 1 -- 172.21.15.94:0/990742773 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fed0401f030 con 0x7fed1407ee40 2024-04-25T13:32:46.688 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 msgr2=0x7fecfc03fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 0x7fecfc03fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed08006fd0 tx=0x7fed08006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 msgr2=0x7fed141a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fed040119f0 tx=0x7fed04011ad0 comp rx=0 tx=0).stop 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 shutdown_connections 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fecfc03d5d0 0x7fecfc03fae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 --2- 172.21.15.94:0/990742773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed1407ee40 0x7fed141a0030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 >> 172.21.15.94:0/990742773 conn(0x7fed14105210 msgr2=0x7fed14105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 shutdown_connections 2024-04-25T13:32:46.689 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:45.901+0000 7fed18b6b700 1 -- 172.21.15.94:0/990742773 wait complete. 2024-04-25T13:32:46.691 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:32:46.691 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T13:32:46.699 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:32:46.699 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T13:32:46.699 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T13:32:46.699 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout: Password: udb69ceomt 2024-04-25T13:32:46.700 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:46.701 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config directory 2024-04-25T13:32:46.701 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-04-25T13:32:47.849 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.115+0000 7f7a93d39700 1 Processor -- start 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.115+0000 7f7a93d39700 1 -- start start 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.115+0000 7f7a93d39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.115+0000 7f7a93d39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c07ce10 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a91ad5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a91ad5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43632/0 (socket says 172.21.15.94:43632) 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a91ad5700 1 -- 172.21.15.94:0/636559296 learned_addr learned my addr 172.21.15.94:0/636559296 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a91ad5700 1 -- 172.21.15.94:0/636559296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c10d000 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a91ad5700 1 --2- 172.21.15.94:0/636559296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e80 tx=0x7f7a8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb8c3ff6fb7c7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a90ad3700 1 -- 172.21.15.94:0/636559296 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a88017070 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a90ad3700 1 -- 172.21.15.94:0/636559296 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8800b8c0 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.116+0000 7f7a90ad3700 1 -- 172.21.15.94:0/636559296 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a88004490 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 -- 172.21.15.94:0/636559296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 msgr2=0x7f7a8c10cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 --2- 172.21.15.94:0/636559296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e80 tx=0x7f7a8800b330 comp rx=0 tx=0).stop 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 -- 172.21.15.94:0/636559296 shutdown_connections 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 --2- 172.21.15.94:0/636559296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c10cac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 -- 172.21.15.94:0/636559296 >> 172.21.15.94:0/636559296 conn(0x7f7a8c104140 msgr2=0x7f7a8c106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 -- 172.21.15.94:0/636559296 shutdown_connections 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.117+0000 7f7a93d39700 1 -- 172.21.15.94:0/636559296 wait complete. 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a93d39700 1 Processor -- start 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a93d39700 1 -- start start 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a93d39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a93d39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10dfa0 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.851 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a91ad5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a91ad5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43644/0 (socket says 172.21.15.94:43644) 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a91ad5700 1 -- 172.21.15.94:0/2994172691 learned_addr learned my addr 172.21.15.94:0/2994172691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.118+0000 7f7a91ad5700 1 -- 172.21.15.94:0/2994172691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a05d0 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a91ad5700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800b7b0 tx=0x7f7a8800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a88020020 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a88012430 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a0860 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a3470 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.119+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a88017040 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.120+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f7a88023020 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.120+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a70005330 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.852 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.120+0000 7f7a82ffd700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 0x7f7a7803fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.120+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a8800f050 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.120+0000 7f7a912d4700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 0x7f7a7803fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.121+0000 7f7a912d4700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 0x7f7a7803fb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.126+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a8802a020 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.321+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f7a70005620 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.322+0000 7f7a82ffd700 1 -- 172.21.15.94:0/2994172691 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7a8802a200 con 0x7f7a8c10a6a0 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 msgr2=0x7f7a7803fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 0x7f7a7803fb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 msgr2=0x7f7a8c1a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800b7b0 tx=0x7f7a8800bd30 comp rx=0 tx=0).stop 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 shutdown_connections 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a7803d660 0x7f7a7803fb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:47.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 --2- 172.21.15.94:0/2994172691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c10a6a0 0x7f7a8c1a0090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:47.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 >> 172.21.15.94:0/2994172691 conn(0x7f7a8c104140 msgr2=0x7f7a8c108600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:47.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 shutdown_connections 2024-04-25T13:32:47.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:47.325+0000 7f7a93d39700 1 -- 172.21.15.94:0/2994172691 wait complete. 2024-04-25T13:32:48.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:47 smithi094 ceph-mon[29195]: mgrmap e13: x(active, since 6s) 2024-04-25T13:32:49.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ef3cf700 1 Processor -- start 2024-04-25T13:32:49.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ef3cf700 1 -- start start 2024-04-25T13:32:49.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ef3cf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ef3cf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48e807ce10 con 0x7f48e810a6e0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ed16b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ed16b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43648/0 (socket says 172.21.15.94:43648) 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ed16b700 1 -- 172.21.15.94:0/712719341 learned_addr learned my addr 172.21.15.94:0/712719341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.287+0000 7f48ed16b700 1 -- 172.21.15.94:0/712719341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e810d040 con 0x7f48e810a6e0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.288+0000 7f48ed16b700 1 --2- 172.21.15.94:0/712719341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48e40098a0 tx=0x7f48e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61baeca867240d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.288+0000 7f48dffff700 1 -- 172.21.15.94:0/712719341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4004160 con 0x7f48e810a6e0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.288+0000 7f48dffff700 1 -- 172.21.15.94:0/712719341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e4004300 con 0x7f48e810a6e0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.288+0000 7f48dffff700 1 -- 172.21.15.94:0/712719341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4011a20 con 0x7f48e810a6e0 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.288+0000 7f48ef3cf700 1 -- 172.21.15.94:0/712719341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 msgr2=0x7f48e810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/712719341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48e40098a0 tx=0x7f48e4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- 172.21.15.94:0/712719341 shutdown_connections 2024-04-25T13:32:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/712719341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e810cb00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- 172.21.15.94:0/712719341 >> 172.21.15.94:0/712719341 conn(0x7f48e8104140 msgr2=0x7f48e8106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- 172.21.15.94:0/712719341 shutdown_connections 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- 172.21.15.94:0/712719341 wait complete. 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 Processor -- start 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- start start 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.289+0000 7f48ef3cf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48e810dfe0 con 0x7f48e810a6e0 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ed16b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ed16b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43664/0 (socket says 172.21.15.94:43664) 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ed16b700 1 -- 172.21.15.94:0/3821932730 learned_addr learned my addr 172.21.15.94:0/3821932730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ed16b700 1 -- 172.21.15.94:0/3821932730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e819e3a0 con 0x7f48e810a6e0 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ed16b700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48e4004660 tx=0x7f48e4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:49.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4003a60 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e819e630 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e401b8f0 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4023890 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.290+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e81a1240 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.291+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f48e402c3f0 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.291+0000 7f48de7fc700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 0x7f48d403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.291+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e8064cf0 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.291+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48e4053f10 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.291+0000 7f48ec96a700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 0x7f48d403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.292+0000 7f48ec96a700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 0x7f48d403fb20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.297+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48e401f030 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.609+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f48e81a1530 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.612+0000 7f48de7fc700 1 -- 172.21.15.94:0/3821932730 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f48e81a1530 con 0x7f48e810a6e0 2024-04-25T13:32:49.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 msgr2=0x7f48d403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 0x7f48d403fb20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 msgr2=0x7f48e819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48e4004660 tx=0x7f48e4011aa0 comp rx=0 tx=0).stop 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 shutdown_connections 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48d403d610 0x7f48d403fb20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 --2- 172.21.15.94:0/3821932730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e810a6e0 0x7f48e819de60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 >> 172.21.15.94:0/3821932730 conn(0x7f48e8104140 msgr2=0x7f48e8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 shutdown_connections 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T13:32:48.614+0000 7f48ef3cf700 1 -- 172.21.15.94:0/3821932730 wait complete. 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:32:49.176 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:32:49.177 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-04-25T13:32:49.207 INFO:tasks.cephadm:Fetching config... 2024-04-25T13:32:49.207 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:32:49.207 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T13:32:49.222 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T13:32:49.222 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:32:49.223 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T13:32:49.278 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T13:32:49.278 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:32:49.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T13:32:49.341 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T13:32:49.341 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:32:49.342 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T13:32:49.397 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T13:32:49.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:32:49.470 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:49.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:32:49.528 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:49.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:32:49.580 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLpMgmIcqO26DDoFNxUsSh3JSkW5ov4Me+tZu7CMLWbj+1MUpk42tEdrhpHjp0KpbrxDgzd3pdSM+Gr/xFGhfial0iVj7sQAgN7nMB+zRdraASQ4QDXpHSOLt/ljpboORZiLO6IZVTU9WZXYtYpQTIfi5HTMc0DscZcDwWU/SZLU5CyfclyyE1Jc1eFOpAm+lR9KxYvOzJ5hTrwG5WZD41AHWLc0kf2zRinmOHa7fquvndW+gLchR3Mz+LAIyXc3lcEgywikhLmIjVraqyKa0TeugNZvAis7Pz7KFe6XcgX36aiAoN/QLzn+0QEsFn5k+OPuDx0HFZhYMjulhG93L38RE38KIGWaNl3ci5fia4VoEul3Aty8shZl0Z9zY4fIXfYLjxIZ0yN5HV2akpLqxut5mHrUXF5iaZTxYJWzws0gDmt+c2OMJIZVA9XWJvY90+r3ZFbSc5bOLbLnByoiSmS7uuySQsP2d7/p3H7O61Kbm2FF14kGro5o4UPFdbZck= ceph-f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:32:49.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T13:32:49.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3821932730' entity='client.admin' 2024-04-25T13:32:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:32:52.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 -- 172.21.15.94:0/1222581598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 msgr2=0x7fdf0c103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:52.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 --2- 172.21.15.94:0/1222581598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c103ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009910 tx=0x7fdefc009c20 comp rx=0 tx=0).stop 2024-04-25T13:32:52.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 -- 172.21.15.94:0/1222581598 shutdown_connections 2024-04-25T13:32:52.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 --2- 172.21.15.94:0/1222581598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c103ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:52.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 -- 172.21.15.94:0/1222581598 >> 172.21.15.94:0/1222581598 conn(0x7fdf0c0ff0d0 msgr2=0x7fdf0c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 -- 172.21.15.94:0/1222581598 shutdown_connections 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.196+0000 7fdf1346e700 1 -- 172.21.15.94:0/1222581598 wait complete. 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1346e700 1 Processor -- start 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1346e700 1 -- start start 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1346e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1346e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c109a50 con 0x7fdf0c103a90 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1120a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1120a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43690/0 (socket says 172.21.15.94:43690) 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1120a700 1 -- 172.21.15.94:0/3673174945 learned_addr learned my addr 172.21.15.94:0/3673174945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.197+0000 7fdf1120a700 1 -- 172.21.15.94:0/3673174945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c19bd10 con 0x7fdf0c103a90 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf1120a700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0057d0 tx=0x7fdefc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc027070 con 0x7fdf0c103a90 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc00be10 con 0x7fdf0c103a90 2024-04-25T13:32:52.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c19bfa0 con 0x7fdf0c103a90 2024-04-25T13:32:52.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc022dd0 con 0x7fdf0c103a90 2024-04-25T13:32:52.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.198+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c19ebb0 con 0x7fdf0c103a90 2024-04-25T13:32:52.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdefc02a070 con 0x7fdf0c103a90 2024-04-25T13:32:52.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c10a1d0 con 0x7fdf0c103a90 2024-04-25T13:32:52.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf027fc700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 0x7fdef803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:52.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdefc01e030 con 0x7fdf0c103a90 2024-04-25T13:32:52.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf10a09700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 0x7fdef803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:52.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.199+0000 7fdf10a09700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 0x7fdef803fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:52.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.205+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdefc013070 con 0x7fdf0c103a90 2024-04-25T13:32:52.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.410+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdf0c02e620 con 0x7fdf0c103a90 2024-04-25T13:32:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.413+0000 7fdf027fc700 1 -- 172.21.15.94:0/3673174945 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdefc0292c0 con 0x7fdf0c103a90 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 msgr2=0x7fdef803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 0x7fdef803fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 msgr2=0x7fdf0c19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0057d0 tx=0x7fdefc005800 comp rx=0 tx=0).stop 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 shutdown_connections 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdef803d610 0x7fdef803fb20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 --2- 172.21.15.94:0/3673174945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf0c103a90 0x7fdf0c19b7d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.415+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 >> 172.21.15.94:0/3673174945 conn(0x7fdf0c0ff0d0 msgr2=0x7fdf0c0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.416+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 shutdown_connections 2024-04-25T13:32:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:52.416+0000 7fdf1346e700 1 -- 172.21.15.94:0/3673174945 wait complete. 2024-04-25T13:32:52.988 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T13:32:52.988 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T13:32:53.607 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:32:53.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:53 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3673174945' entity='client.admin' 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:54.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 -- 172.21.15.94:0/505597333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 msgr2=0x7fabcc102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 --2- 172.21.15.94:0/505597333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc102dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009910 tx=0x7fabbc009c20 comp rx=0 tx=0).stop 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 -- 172.21.15.94:0/505597333 shutdown_connections 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 --2- 172.21.15.94:0/505597333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc102dd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 -- 172.21.15.94:0/505597333 >> 172.21.15.94:0/505597333 conn(0x7fabcc0fdfe0 msgr2=0x7fabcc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 -- 172.21.15.94:0/505597333 shutdown_connections 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.130+0000 7fabd17fd700 1 -- 172.21.15.94:0/505597333 wait complete. 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabd17fd700 1 Processor -- start 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabd17fd700 1 -- start start 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabd17fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabd17fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc108980 con 0x7fabcc1029c0 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabcaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabcaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43258/0 (socket says 172.21.15.94:43258) 2024-04-25T13:32:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabcaffd700 1 -- 172.21.15.94:0/4180740889 learned_addr learned my addr 172.21.15.94:0/4180740889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:55.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.131+0000 7fabcaffd700 1 -- 172.21.15.94:0/4180740889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc06bba0 con 0x7fabcc1029c0 2024-04-25T13:32:55.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabcaffd700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0057d0 tx=0x7fabbc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:55.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc027070 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabbc00be10 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc19c360 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc022dd0 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.132+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc19c7e0 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabcc109100 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fabbc02a070 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabc8ff9700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 0x7fabb803fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabbc01e030 con 0x7fabcc1029c0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabc27fc700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 0x7fabb803fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.133+0000 7fabc27fc700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 0x7fabb803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.138+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabbc013070 con 0x7fabcc1029c0 2024-04-25T13:32:55.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.339+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fabcc02e840 con 0x7fabb803d5c0 2024-04-25T13:32:55.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.342+0000 7fabc8ff9700 1 -- 172.21.15.94:0/4180740889 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fabcc02e840 con 0x7fabb803d5c0 2024-04-25T13:32:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 msgr2=0x7fabb803fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 0x7fabb803fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4006e40 comp rx=0 tx=0).stop 2024-04-25T13:32:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 msgr2=0x7fabcc06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0057d0 tx=0x7fabbc005800 comp rx=0 tx=0).stop 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 shutdown_connections 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fabb803d5c0 0x7fabb803fad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 --2- 172.21.15.94:0/4180740889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc1029c0 0x7fabcc06b660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 >> 172.21.15.94:0/4180740889 conn(0x7fabcc0fdfe0 msgr2=0x7fabcc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 shutdown_connections 2024-04-25T13:32:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:55.345+0000 7fabd17fd700 1 -- 172.21.15.94:0/4180740889 wait complete. 2024-04-25T13:32:56.103 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-04-25T13:32:56.104 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:32:56.104 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:32:56.123 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:32:56.123 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:32:56.180 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-04-25T13:32:56.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch host add smithi137 2024-04-25T13:32:56.418 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T13:32:56.620 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: Updating smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:32:56.621 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:32:56.621 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: Updating smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:32:56.621 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:56.621 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:56.621 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:32:57.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 -- 172.21.15.94:0/1686393669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 msgr2=0x7f23b8105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:32:57.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 --2- 172.21.15.94:0/1686393669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b8105010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009910 tx=0x7f23ac009c20 comp rx=0 tx=0).stop 2024-04-25T13:32:57.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 -- 172.21.15.94:0/1686393669 shutdown_connections 2024-04-25T13:32:57.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 --2- 172.21.15.94:0/1686393669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b8105010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:32:57.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 -- 172.21.15.94:0/1686393669 >> 172.21.15.94:0/1686393669 conn(0x7f23b8078460 msgr2=0x7f23b80788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:32:57.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 -- 172.21.15.94:0/1686393669 shutdown_connections 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.723+0000 7f23bef53700 1 -- 172.21.15.94:0/1686393669 wait complete. 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bef53700 1 Processor -- start 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bef53700 1 -- start start 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bef53700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bef53700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b810e000 con 0x7f23b8104c00 2024-04-25T13:32:57.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bccef700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bccef700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43268/0 (socket says 172.21.15.94:43268) 2024-04-25T13:32:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bccef700 1 -- 172.21.15.94:0/1921632255 learned_addr learned my addr 172.21.15.94:0/1921632255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:32:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bccef700 1 -- 172.21.15.94:0/1921632255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81a1b90 con 0x7f23b8104c00 2024-04-25T13:32:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.724+0000 7f23bccef700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0056d0 tx=0x7f23ac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:57.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.725+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac026070 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.725+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ac004370 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.725+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b81a1e20 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.725+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac0039b0 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.725+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b819b870 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.726+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f23ac029070 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.726+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b806b4e0 con 0x7f23b8104c00 2024-04-25T13:32:57.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.726+0000 7f23b5ffb700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 0x7f23a803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:32:57.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.726+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23ac01e070 con 0x7f23b8104c00 2024-04-25T13:32:57.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.726+0000 7f23b7fff700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 0x7f23a803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:32:57.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.727+0000 7f23b7fff700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 0x7f23a803fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006fd0 tx=0x7f23a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:32:57.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.731+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23ac013070 con 0x7f23b8104c00 2024-04-25T13:32:57.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:32:57.935+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) -- 0x7f23b819cda0 con 0x7f23a803d670 2024-04-25T13:32:58.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:58 smithi094 ceph-mon[29195]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:32:59.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:32:59 smithi094 ceph-mon[29195]: Deploying cephadm binary to smithi137 2024-04-25T13:33:01.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:01 smithi094 ceph-mon[29195]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.128+0000 7f23b5ffb700 1 -- 172.21.15.94:0/1921632255 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f23b819cda0 con 0x7f23a803d670 2024-04-25T13:33:02.129 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 msgr2=0x7f23a803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 0x7f23a803fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23a4006fd0 tx=0x7f23a4006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 msgr2=0x7f23b81a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0056d0 tx=0x7f23ac0057b0 comp rx=0 tx=0).stop 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 shutdown_connections 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23a803d670 0x7f23a803fb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 --2- 172.21.15.94:0/1921632255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8104c00 0x7f23b81a1650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.130+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 >> 172.21.15.94:0/1921632255 conn(0x7f23b8078460 msgr2=0x7f23b8109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.131+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 shutdown_connections 2024-04-25T13:33:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:02.131+0000 7f23bef53700 1 -- 172.21.15.94:0/1921632255 wait complete. 2024-04-25T13:33:02.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:33:03.035 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:03.402 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:03.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:03 smithi094 ceph-mon[29195]: Added host smithi137 2024-04-25T13:33:03.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:03.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:03 smithi094 ceph-mon[29195]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:03.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:04.424 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:04.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 -- 172.21.15.94:0/3133983373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 msgr2=0x7eff64105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 --2- 172.21.15.94:0/3133983373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff64105000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff54009910 tx=0x7eff54009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 -- 172.21.15.94:0/3133983373 shutdown_connections 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 --2- 172.21.15.94:0/3133983373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff64105000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 -- 172.21.15.94:0/3133983373 >> 172.21.15.94:0/3133983373 conn(0x7eff640fe440 msgr2=0x7eff64100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 -- 172.21.15.94:0/3133983373 shutdown_connections 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.460+0000 7eff699e8700 1 -- 172.21.15.94:0/3133983373 wait complete. 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff699e8700 1 Processor -- start 2024-04-25T13:33:04.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff699e8700 1 -- start start 2024-04-25T13:33:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff699e8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff699e8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff6410dfe0 con 0x7eff64104bf0 2024-04-25T13:33:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43252/0 (socket says 172.21.15.94:43252) 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.461+0000 7eff62ffd700 1 -- 172.21.15.94:0/91753457 learned_addr learned my addr 172.21.15.94:0/91753457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff62ffd700 1 -- 172.21.15.94:0/91753457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419fa60 con 0x7eff64104bf0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff62ffd700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff540056d0 tx=0x7eff540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff54026070 con 0x7eff64104bf0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff54004370 con 0x7eff64104bf0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419fcf0 con 0x7eff64104bf0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff540039b0 con 0x7eff64104bf0 2024-04-25T13:33:04.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.462+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641996b0 con 0x7eff64104bf0 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.463+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7eff54029070 con 0x7eff64104bf0 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.463+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6406c2f0 con 0x7eff64104bf0 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.463+0000 7eff689e6700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 0x7eff4c03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.463+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff5401e070 con 0x7eff64104bf0 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.463+0000 7eff627fc700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 0x7eff4c03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:04.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.464+0000 7eff627fc700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 0x7eff4c03fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff58006fd0 tx=0x7eff58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:04.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.469+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff54013070 con 0x7eff64104bf0 2024-04-25T13:33:04.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.670+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff6419abc0 con 0x7eff4c03d670 2024-04-25T13:33:04.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.671+0000 7eff689e6700 1 -- 172.21.15.94:0/91753457 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7eff6419abc0 con 0x7eff4c03d670 2024-04-25T13:33:04.672 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:33:04.672 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.673+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 msgr2=0x7eff4c03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.673+0000 7eff699e8700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 0x7eff4c03fb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff58006fd0 tx=0x7eff58006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 msgr2=0x7eff6419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff540056d0 tx=0x7eff540057b0 comp rx=0 tx=0).stop 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 shutdown_connections 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff4c03d670 0x7eff4c03fb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 --2- 172.21.15.94:0/91753457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff64104bf0 0x7eff6419f520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 >> 172.21.15.94:0/91753457 conn(0x7eff640fe440 msgr2=0x7eff64109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 shutdown_connections 2024-04-25T13:33:04.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:04.674+0000 7eff699e8700 1 -- 172.21.15.94:0/91753457 wait complete. 2024-04-25T13:33:05.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:05 smithi094 ceph-mon[29195]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:05.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:05.314 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-04-25T13:33:05.314 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:33:05.314 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:33:05.332 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:33:05.333 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:05.390 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-04-25T13:33:05.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch host add smithi138 2024-04-25T13:33:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:06 smithi094 ceph-mon[29195]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 -- 172.21.15.94:0/3653312283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 msgr2=0x7f8188108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3653312283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f8188108d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8178009910 tx=0x7f8178009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 -- 172.21.15.94:0/3653312283 shutdown_connections 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3653312283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f8188108d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 -- 172.21.15.94:0/3653312283 >> 172.21.15.94:0/3653312283 conn(0x7f81880fe440 msgr2=0x7f8188100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 -- 172.21.15.94:0/3653312283 shutdown_connections 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.148+0000 7f818d5ad700 1 -- 172.21.15.94:0/3653312283 wait complete. 2024-04-25T13:33:07.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.149+0000 7f818d5ad700 1 Processor -- start 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.149+0000 7f818d5ad700 1 -- start start 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.149+0000 7f818d5ad700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.149+0000 7f818d5ad700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f818810ce80 con 0x7f8188108930 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f8186ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f8186ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43270/0 (socket says 172.21.15.94:43270) 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f8186ffd700 1 -- 172.21.15.94:0/3024669404 learned_addr learned my addr 172.21.15.94:0/3024669404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f8186ffd700 1 -- 172.21.15.94:0/3024669404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818806c940 con 0x7f8188108930 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f8186ffd700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81780056d0 tx=0x7f81780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178027070 con 0x7f8188108930 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819ac60 con 0x7f8188108930 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817800be10 con 0x7f8188108930 2024-04-25T13:33:07.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8178022d20 con 0x7f8188108930 2024-04-25T13:33:07.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.150+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818819b120 con 0x7f8188108930 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.151+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8188104040 con 0x7f8188108930 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.151+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f817802a070 con 0x7f8188108930 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.152+0000 7f816ffff700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 0x7f817003fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.152+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f817801e030 con 0x7f8188108930 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.152+0000 7f81867fc700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 0x7f817003fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.152+0000 7f81867fc700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 0x7f817003fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:07.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.157+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f818806c940 con 0x7f8188108930 2024-04-25T13:33:07.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:07 smithi094 ceph-mon[29195]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:07.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:07.358+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) -- 0x7f818806c0c0 con 0x7f817003d670 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: Deploying cephadm binary to smithi138 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:33:09.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:09.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:09.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T13:33:09.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:09 smithi094 ceph-mon[29195]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:10 smithi094 ceph-mon[29195]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:11.464 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:11 smithi094 ceph-mon[29195]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:11.464 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-04-25T13:33:11.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.463+0000 7f816ffff700 1 -- 172.21.15.94:0/3024669404 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f818806c0c0 con 0x7f817003d670 2024-04-25T13:33:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 msgr2=0x7f817003fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 0x7f817003fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 msgr2=0x7f818819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:11.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81780056d0 tx=0x7f81780057b0 comp rx=0 tx=0).stop 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 shutdown_connections 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f817003d670 0x7f817003fb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 --2- 172.21.15.94:0/3024669404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8188108930 0x7f818819a660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 >> 172.21.15.94:0/3024669404 conn(0x7f81880fe440 msgr2=0x7f81880ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 shutdown_connections 2024-04-25T13:33:11.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:11.466+0000 7f818d5ad700 1 -- 172.21.15.94:0/3024669404 wait complete. 2024-04-25T13:33:12.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:33:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:12.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:12.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:13.404 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:13 smithi094 ceph-mon[29195]: Added host smithi138 2024-04-25T13:33:13.404 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:13 smithi094 ceph-mon[29195]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:13.404 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1727307966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 msgr2=0x7fdd84108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1727307966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84108d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000b0a0 tx=0x7fdd8000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1727307966 shutdown_connections 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1727307966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84108d70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1727307966 >> 172.21.15.94:0/1727307966 conn(0x7fdd84078480 msgr2=0x7fdd840788e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1727307966 shutdown_connections 2024-04-25T13:33:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.742+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1727307966 wait complete. 2024-04-25T13:33:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd8b3dc700 1 Processor -- start 2024-04-25T13:33:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd8b3dc700 1 -- start start 2024-04-25T13:33:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd8b3dc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd8b3dc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8410ceb0 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd89178700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd89178700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37042/0 (socket says 172.21.15.94:37042) 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.743+0000 7fdd89178700 1 -- 172.21.15.94:0/1877434346 learned_addr learned my addr 172.21.15.94:0/1877434346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd89178700 1 -- 172.21.15.94:0/1877434346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd84199ca0 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd89178700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd80015040 tx=0x7fdd80009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd80021020 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd84199f30 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80007970 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd80017040 con 0x7fdd84108960 2024-04-25T13:33:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.744+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8419df90 con 0x7fdd84108960 2024-04-25T13:33:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.745+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdd80024020 con 0x7fdd84108960 2024-04-25T13:33:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.745+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd84104070 con 0x7fdd84108960 2024-04-25T13:33:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.745+0000 7fdd7a7fc700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 0x7fdd7003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.745+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd8000f050 con 0x7fdd84108960 2024-04-25T13:33:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.745+0000 7fdd88977700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 0x7fdd7003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.746+0000 7fdd88977700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 0x7fdd7003fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdd74006fd0 tx=0x7fdd74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.751+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd80059080 con 0x7fdd84108960 2024-04-25T13:33:13.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.950+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd8419aac0 con 0x7fdd7003d610 2024-04-25T13:33:13.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.952+0000 7fdd7a7fc700 1 -- 172.21.15.94:0/1877434346 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fdd8419aac0 con 0x7fdd7003d610 2024-04-25T13:33:13.952 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:33:13.952 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 msgr2=0x7fdd7003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 0x7fdd7003fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdd74006fd0 tx=0x7fdd74006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 msgr2=0x7fdd84199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd80015040 tx=0x7fdd80009790 comp rx=0 tx=0).stop 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 shutdown_connections 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd7003d610 0x7fdd7003fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 --2- 172.21.15.94:0/1877434346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd84108960 0x7fdd84199760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 >> 172.21.15.94:0/1877434346 conn(0x7fdd84078480 msgr2=0x7fdd8406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 shutdown_connections 2024-04-25T13:33:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:13.954+0000 7fdd8b3dc700 1 -- 172.21.15.94:0/1877434346 wait complete. 2024-04-25T13:33:14.597 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T13:33:14.597 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T13:33:14.830 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:15.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:15 smithi094 ceph-mon[29195]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:33:15.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:15.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:15 smithi094 ceph-mon[29195]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:16.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 -- 172.21.15.94:0/3096399296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 msgr2=0x7f9150104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:16.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 --2- 172.21.15.94:0/3096399296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f9150104fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f914c009910 tx=0x7f914c009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 -- 172.21.15.94:0/3096399296 shutdown_connections 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 --2- 172.21.15.94:0/3096399296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f9150104fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 -- 172.21.15.94:0/3096399296 >> 172.21.15.94:0/3096399296 conn(0x7f91500fe420 msgr2=0x7f9150100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 -- 172.21.15.94:0/3096399296 shutdown_connections 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.388+0000 7f91584a4700 1 -- 172.21.15.94:0/3096399296 wait complete. 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f91584a4700 1 Processor -- start 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f91584a4700 1 -- start start 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f91584a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f91584a4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f915010dfc0 con 0x7f9150104bd0 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f9156240700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f9156240700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37064/0 (socket says 172.21.15.94:37064) 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f9156240700 1 -- 172.21.15.94:0/927150508 learned_addr learned my addr 172.21.15.94:0/927150508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:16.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.389+0000 7f9156240700 1 -- 172.21.15.94:0/927150508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915019fa60 con 0x7f9150104bd0 2024-04-25T13:33:16.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f9156240700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f914c0056d0 tx=0x7f914c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f914c026070 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915019fcf0 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f914c004370 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f914c0039b0 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.390+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501996b0 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f914c029070 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915006c2f0 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f91477fe700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 0x7f913c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f914c01e070 con 0x7f9150104bd0 2024-04-25T13:33:16.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f9155a3f700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 0x7f913c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:16.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.391+0000 7f9155a3f700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 0x7f913c03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9140006fd0 tx=0x7f9140006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:16.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.396+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f914c013070 con 0x7f9150104bd0 2024-04-25T13:33:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:16 smithi094 ceph-mon[29195]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:16.598+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f915019abc0 con 0x7f9150104bd0 2024-04-25T13:33:17.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.270+0000 7f91477fe700 1 -- 172.21.15.94:0/927150508 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f914c00d340 con 0x7f9150104bd0 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 msgr2=0x7f913c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 0x7f913c03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9140006fd0 tx=0x7f9140006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 msgr2=0x7f915019f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f914c0056d0 tx=0x7f914c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 shutdown_connections 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f913c03d610 0x7f913c03fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 --2- 172.21.15.94:0/927150508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9150104bd0 0x7f915019f520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.272+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 >> 172.21.15.94:0/927150508 conn(0x7f91500fe420 msgr2=0x7f9150109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.273+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 shutdown_connections 2024-04-25T13:33:17.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:17.273+0000 7f91584a4700 1 -- 172.21.15.94:0/927150508 wait complete. 2024-04-25T13:33:17.274 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-04-25T13:33:17.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/927150508' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:33:17.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:17.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:17.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:17.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:17.589 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:33:17.589 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:17.589 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:17.933 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-04-25T13:33:17.933 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-04-25T13:33:17.934 INFO:tasks.cephadm:Adding mon.c on smithi138 2024-04-25T13:33:17.934 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c' 2024-04-25T13:33:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:18.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/927150508' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:18.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:18 smithi094 ceph-mon[29195]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.634+0000 7fc0bc31b700 1 -- 172.21.15.138:0/2665151066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 msgr2=0x7fc0b4108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.634+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/2665151066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b4108d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009910 tx=0x7fc0b0009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 -- 172.21.15.138:0/2665151066 shutdown_connections 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/2665151066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b4108d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 -- 172.21.15.138:0/2665151066 >> 172.21.15.138:0/2665151066 conn(0x7fc0b40fe430 msgr2=0x7fc0b4100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 -- 172.21.15.138:0/2665151066 shutdown_connections 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 -- 172.21.15.138:0/2665151066 wait complete. 2024-04-25T13:33:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 Processor -- start 2024-04-25T13:33:19.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.635+0000 7fc0bc31b700 1 -- start start 2024-04-25T13:33:19.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0bc31b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:19.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0bc31b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b410ce90 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0ba0b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0ba0b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.138:44918/0 (socket says 172.21.15.138:44918) 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0ba0b7700 1 -- 172.21.15.138:0/1959503098 learned_addr learned my addr 172.21.15.138:0/1959503098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0ba0b7700 1 -- 172.21.15.138:0/1959503098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b4101220 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.636+0000 7fc0ba0b7700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00092e0 tx=0x7fc0b0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.637+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0b0026070 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.637+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b000b6f0 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.637+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b40ff910 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.637+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0b00037f0 con 0x7fc0b4108940 2024-04-25T13:33:19.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.637+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b40ffdd0 con 0x7fc0b4108940 2024-04-25T13:33:19.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.638+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b4104050 con 0x7fc0b4108940 2024-04-25T13:33:19.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.638+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fc0b0029070 con 0x7fc0b4108940 2024-04-25T13:33:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.638+0000 7fc0ab7fe700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 0x7fc0a003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.638+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc0b001e070 con 0x7fc0b4108940 2024-04-25T13:33:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.638+0000 7fc0b98b6700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 0x7fc0a003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.639+0000 7fc0b98b6700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 0x7fc0a003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.644+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0b0013070 con 0x7fc0b4108940 2024-04-25T13:33:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.852+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}) -- 0x7fc0b4100970 con 0x7fc0a003d610 2024-04-25T13:33:19.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.860+0000 7fc0ab7fe700 1 -- 172.21.15.138:0/1959503098 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc0b4100970 con 0x7fc0a003d610 2024-04-25T13:33:19.860 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 msgr2=0x7fc0a003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 0x7fc0a003fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 msgr2=0x7fc0b40ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00092e0 tx=0x7fc0b0005780 comp rx=0 tx=0).stop 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 shutdown_connections 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0a003d610 0x7fc0a003fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 --2- 172.21.15.138:0/1959503098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0b4108940 0x7fc0b40ff3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 >> 172.21.15.138:0/1959503098 conn(0x7fc0b40fe430 msgr2=0x7fc0b4106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 shutdown_connections 2024-04-25T13:33:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:19.862+0000 7fc0bc31b700 1 -- 172.21.15.138:0/1959503098 wait complete. 2024-04-25T13:33:20.434 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T13:33:20.436 DEBUG:teuthology.orchestra.run.smithi138:mon.c> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T13:33:20.438 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:33:20.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: Deploying daemon mon.c on smithi138 2024-04-25T13:33:21.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:20 smithi094 ceph-mon[29195]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:22.199 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:22.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:22 smithi094 ceph-mon[29195]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:23 smithi138 systemd[1]: Starting Ceph mon.c for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 -- 172.21.15.138:0/3029408263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 msgr2=0x7f6934108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 --2- 172.21.15.138:0/3029408263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f6934108d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6924009910 tx=0x7f6924009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 -- 172.21.15.138:0/3029408263 shutdown_connections 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 --2- 172.21.15.138:0/3029408263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f6934108d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 -- 172.21.15.138:0/3029408263 >> 172.21.15.138:0/3029408263 conn(0x7f69340fe440 msgr2=0x7f6934100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 -- 172.21.15.138:0/3029408263 shutdown_connections 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.350+0000 7f693c723700 1 -- 172.21.15.138:0/3029408263 wait complete. 2024-04-25T13:33:24.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693c723700 1 Processor -- start 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693c723700 1 -- start start 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693c723700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693c723700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f693410ce80 con 0x7f6934108930 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693a4bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693a4bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.138:44934/0 (socket says 172.21.15.138:44934) 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.351+0000 7f693a4bf700 1 -- 172.21.15.138:0/767929398 learned_addr learned my addr 172.21.15.138:0/767929398 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:33:24.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f693a4bf700 1 -- 172.21.15.138:0/767929398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693406c920 con 0x7f6934108930 2024-04-25T13:33:24.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f693a4bf700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69240092e0 tx=0x7f6924005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6924026070 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692400b750 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693419acd0 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69240038a0 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.352+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693419b190 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.353+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6934104040 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.353+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f6924029070 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.353+0000 7f692b7fe700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 0x7f692003fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.353+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f692401e070 con 0x7f6934108930 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.354+0000 7f6939cbe700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 0x7f692003fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:24.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.354+0000 7f6939cbe700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 0x7f692003fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6930006fd0 tx=0x7f6930006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:24.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.359+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6924013070 con 0x7f6934108930 2024-04-25T13:33:24.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.659+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f693419b470 con 0x7f6934108930 2024-04-25T13:33:24.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.660+0000 7f692b7fe700 1 -- 172.21.15.138:0/767929398 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6924030300 con 0x7f6934108930 2024-04-25T13:33:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:33:24.661 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","modified":"2024-04-25T13:31:42.925512Z","created":"2024-04-25T13:31:42.925512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 msgr2=0x7f692003fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 0x7f692003fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6930006fd0 tx=0x7f6930006e40 comp rx=0 tx=0).stop 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 msgr2=0x7f693419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69240092e0 tx=0x7f6924005780 comp rx=0 tx=0).stop 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 shutdown_connections 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f692003d670 0x7f692003fb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 --2- 172.21.15.138:0/767929398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6934108930 0x7f693419a6d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 >> 172.21.15.138:0/767929398 conn(0x7f69340fe440 msgr2=0x7f69340ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 shutdown_connections 2024-04-25T13:33:24.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:24.662+0000 7f693c723700 1 -- 172.21.15.138:0/767929398 wait complete. 2024-04-25T13:33:24.664 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T13:33:24.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:24 smithi094 ceph-mon[29195]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:24.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:24 smithi138 podman[32944]: 2024-04-25 13:33:24.475036513 +0000 UTC m=+0.018033897 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:33:24.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:24 smithi138 podman[32944]: 2024-04-25 13:33:24.634031176 +0000 UTC m=+0.177028592 container create 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 podman[32944]: 2024-04-25 13:33:25.055107882 +0000 UTC m=+0.598105250 container init 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 podman[32944]: 2024-04-25 13:33:25.058280661 +0000 UTC m=+0.601278033 container start 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pidfile_write: ignore empty --pid-file 2024-04-25T13:33:25.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: load: jerasure load: lrc 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Git sha 0 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: DB SUMMARY 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: DB Session ID: X05E4RIJKVUJ8QXY5MGG 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:33:25.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.env: 0x5654f300b500 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.info_log: 0x5654f4d35fc0 2024-04-25T13:33:25.504 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.statistics: (nil) 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.use_fsync: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:33:25.505 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.db_log_dir: 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.wal_dir: 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:33:25.506 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.write_buffer_manager: 0x5654f4d068c0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.unordered_write: 0 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:33:25.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.row_cache: None 2024-04-25T13:33:25.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.wal_filter: None 2024-04-25T13:33:25.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:33:25.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:33:25.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:33:25.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.wal_compression: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:33:25.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_open_files: -1 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:33:25.511 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Compression algorithms supported: 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kZSTD supported: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:33:25.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.merge_operator: 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_filter: None 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5654f4e3c1a0) 2024-04-25T13:33:25.513 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:33:25.514 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:33:25.514 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_type: 0 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_index_type: 0 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_shortening: 1 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: checksum: 4 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: no_block_cache: 0 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache: 0x5654f4d391f0 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_options: 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: capacity : 536870912 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_shard_bits : 4 2024-04-25T13:33:25.515 INFO:journalctl@ceph.mon.c.smithi138.stdout: strict_capacity_limit : 0 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_compressed: (nil) 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: persistent_cache: (nil) 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size: 4096 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size_deviation: 10 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_restart_interval: 16 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_block_restart_interval: 1 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: metadata_block_size: 4096 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: partition_filters: 0 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: use_delta_encoding: 1 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: filter_policy: bloomfilter 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: whole_key_filtering: 1 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: verify_compression: 0 2024-04-25T13:33:25.516 INFO:journalctl@ceph.mon.c.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: format_version: 5 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_index_compression: 1 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_align: 0 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: max_auto_readahead_size: 262144 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: prepopulate_block_cache: 0 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression: NoCompression 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:33:25.517 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.num_levels: 7 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:33:25.518 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:33:25.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:33:25.520 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:33:25.521 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.ttl: 2592000 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f8717b80-85d1-4082-8d9b-6f02bdfbc0cd 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052005098365, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052005098799, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714052005, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f8717b80-85d1-4082-8d9b-6f02bdfbc0cd", "db_session_id": "X05E4RIJKVUJ8QXY5MGG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052005098888, "job": 1, "event": "recovery_finished"} 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5654f4d1dc00 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: DB pointer 0x5654f9b10000 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:33:25.524 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** DB Stats ** 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T13:33:25.525 INFO:journalctl@ceph.mon.c.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.526 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:33:25.527 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache BinnedLRUCache@0x5654f4d391f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: starting mon.c rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(???) e0 preinit fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T13:33:25.528 INFO:journalctl@ceph.mon.c.smithi138.stdout: e1 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout: legacy client fscid: -1 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout: No filesystems configured 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:33:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Deploying cephadm binary to smithi138 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T13:33:25.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Added host smithi138 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.531 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/927150508' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/927150508' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:25.532 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:25.533 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: Deploying daemon mon.c on smithi138 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 ceph-mon[32977]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T13:33:25.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 bash[32944]: 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c 2024-04-25T13:33:25.535 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:25 smithi138 systemd[1]: Started Ceph mon.c for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:33:26.635 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:33:26.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:33:26.873 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:30.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: monmap epoch 2 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: last_changed 2024-04-25T13:33:25.123758+0000 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: election_strategy: 1 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: fsmap 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: mgrmap e13: x(active, since 49s) 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T13:33:30.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: mon.a calling monitor election 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: mon.c calling monitor election 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: monmap epoch 2 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: last_changed 2024-04-25T13:33:25.123758+0000 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: min_mon_release 19 (squid) 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: election_strategy: 1 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: fsmap 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: mgrmap e13: x(active, since 49s) 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T13:33:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:30.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:30.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:31.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.315+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/2280932603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb000035c0 msgr2=0x7ffb00005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.315+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/2280932603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb000035c0 0x7ffb00005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb14102950 tx=0x7ffb0400feb0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.316+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/2280932603 shutdown_connections 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.316+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/2280932603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb000035c0 0x7ffb00005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.316+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/2280932603 >> 172.21.15.138:0/2280932603 conn(0x7ffb140fe500 msgr2=0x7ffb14100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.316+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/2280932603 shutdown_connections 2024-04-25T13:33:31.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.316+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/2280932603 wait complete. 2024-04-25T13:33:31.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 Processor -- start 2024-04-25T13:33:31.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 -- start start 2024-04-25T13:33:31.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:31.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 0x7ffb1419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:31.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1410aa00 con 0x7ffb1419acc0 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.317+0000 7ffb1b6eb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1410a880 con 0x7ffb000035c0 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59214/0 (socket says 172.21.15.138:59214) 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 -- 172.21.15.138:0/3900896107 learned_addr learned my addr 172.21.15.138:0/3900896107 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb18c86700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 0x7ffb1419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 -- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 msgr2=0x7ffb1419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 -- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 msgr2=0x7ffb1419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb19487700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:33:31.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb18c86700 1 -- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 msgr2=0x7ffb1419a780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb18c86700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb18c86700 1 -- 172.21.15.138:0/3900896107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141a37f0 con 0x7ffb1419acc0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.318+0000 7ffb18c86700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 0x7ffb1419b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000b4d0 tx=0x7ffb1000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.319+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffb10011630 con 0x7ffb1419acc0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.319+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb10011c80 con 0x7ffb1419acc0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.319+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb141a3a20 con 0x7ffb1419acc0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.319+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffb10010da0 con 0x7ffb1419acc0 2024-04-25T13:33:31.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.319+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a3f80 con 0x7ffb1419acc0 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.320+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7ffb10018020 con 0x7ffb1419acc0 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.320+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb14104160 con 0x7ffb1419acc0 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.320+0000 7ffb0a7fc700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 0x7ffaf803f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.320+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffb10050ff0 con 0x7ffb1419acc0 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.320+0000 7ffb19487700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 0x7ffaf803f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.321+0000 7ffb19487700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 0x7ffaf803f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400fa60 tx=0x7ffb0400f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:31.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.326+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb1001f020 con 0x7ffb1419acc0 2024-04-25T13:33:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:31 smithi094 ceph-mon[29195]: Deploying daemon mon.b on smithi137 2024-04-25T13:33:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:31 smithi094 ceph-mon[29195]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:31.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:31 smithi138 ceph-mon[32977]: Deploying daemon mon.b on smithi137 2024-04-25T13:33:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:31 smithi138 ceph-mon[32977]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:31.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.609+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffb1402e510 con 0x7ffb1419acc0 2024-04-25T13:33:31.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.610+0000 7ffb0a7fc700 1 -- 172.21.15.138:0/3900896107 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ffb10010410 con 0x7ffb1419acc0 2024-04-25T13:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:33:31.611 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","modified":"2024-04-25T13:33:25.123758Z","created":"2024-04-25T13:31:42.925512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 msgr2=0x7ffaf803f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 0x7ffaf803f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400fa60 tx=0x7ffb0400f4a0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 msgr2=0x7ffb1419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 0x7ffb1419b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000b4d0 tx=0x7ffb1000b890 comp rx=0 tx=0).stop 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 shutdown_connections 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb000035c0 0x7ffb1419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffaf803d400 0x7ffaf803f910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 --2- 172.21.15.138:0/3900896107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb1419acc0 0x7ffb1419b0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:31.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 >> 172.21.15.138:0/3900896107 conn(0x7ffb140fe500 msgr2=0x7ffb14101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:31.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 shutdown_connections 2024-04-25T13:33:31.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:31.612+0000 7ffb1b6eb700 1 -- 172.21.15.138:0/3900896107 wait complete. 2024-04-25T13:33:31.614 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-04-25T13:33:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:32 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/3900896107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:33:32.476 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:33:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:33:32.123+0000 7f3efb1cd700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T13:33:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:32 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3900896107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:33:33.222 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:33:33.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:33:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:33:33.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:33 smithi094 ceph-mon[29195]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:33 smithi138 ceph-mon[32977]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 -- 172.21.15.138:0/2065963695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34100035c0 msgr2=0x7f3410005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 --2- 172.21.15.138:0/2065963695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34100035c0 0x7f3410005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f342806bc30 tx=0x7f3420009e80 comp rx=0 tx=0).stop 2024-04-25T13:33:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 -- 172.21.15.138:0/2065963695 shutdown_connections 2024-04-25T13:33:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 --2- 172.21.15.138:0/2065963695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34100035c0 0x7f3410005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 -- 172.21.15.138:0/2065963695 >> 172.21.15.138:0/2065963695 conn(0x7f34280fe520 msgr2=0x7f3428100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 -- 172.21.15.138:0/2065963695 shutdown_connections 2024-04-25T13:33:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.362+0000 7f342caa8700 1 -- 172.21.15.138:0/2065963695 wait complete. 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.363+0000 7f342caa8700 1 Processor -- start 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.363+0000 7f342caa8700 1 -- start start 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 0x7f34281a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f342810ba90 con 0x7f34281a5ca0 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f342810b910 con 0x7f3428102ec0 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342caa8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f342810bc10 con 0x7f34100035c0 2024-04-25T13:33:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59218/0 (socket says 172.21.15.138:59218) 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 -- 172.21.15.138:0/1868492082 learned_addr learned my addr 172.21.15.138:0/1868492082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 msgr2=0x7f3428102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 msgr2=0x7f3428102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 0x7f34281a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f342659c700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3425d9b700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3425d9b700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 msgr2=0x7f34281032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3425d9b700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 msgr2=0x7f34281032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3425d9b700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:33:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3425d9b700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 msgr2=0x7f3428102980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 msgr2=0x7f34281032d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.364+0000 7f3426d9d700 1 -- 172.21.15.138:0/1868492082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34281a8740 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f3426d9d700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 0x7f34281a80c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3418006fd0 tx=0x7f341800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418015020 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3418011ce0 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34281a8a30 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418010d40 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.365+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a8f20 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.366+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f3418018020 con 0x7f34281a5ca0 2024-04-25T13:33:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.366+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3428064cf0 con 0x7f34281a5ca0 2024-04-25T13:33:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.366+0000 7f34177fe700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 0x7f340803ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.366+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3418051070 con 0x7f34281a5ca0 2024-04-25T13:33:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.367+0000 7f342659c700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 0x7f340803ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.367+0000 7f342659c700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 0x7f340803ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34200098a0 tx=0x7f342000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.372+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f341801f020 con 0x7f34281a5ca0 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: monmap epoch 3 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: last_changed 2024-04-25T13:33:34.156502+0000 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: election_strategy: 1 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: fsmap 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: mgrmap e13: x(active, since 58s) 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T13:33:39.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: mon.c calling monitor election 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: mon.a calling monitor election 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: monmap epoch 3 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: last_changed 2024-04-25T13:33:34.156502+0000 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: min_mon_release 19 (squid) 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: election_strategy: 1 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T13:33:39.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: fsmap 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: mgrmap e13: x(active, since 58s) 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:39.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.667+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34281a9330 con 0x7f34281a5ca0 2024-04-25T13:33:39.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.668+0000 7f34177fe700 1 -- 172.21.15.138:0/1868492082 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f34180187b0 con 0x7f34281a5ca0 2024-04-25T13:33:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T13:33:39.669 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":3,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","modified":"2024-04-25T13:33:34.156502Z","created":"2024-04-25T13:31:42.925512Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 msgr2=0x7f340803ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 0x7f340803ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34200098a0 tx=0x7f342000b040 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 msgr2=0x7f34281a80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 0x7f34281a80c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3418006fd0 tx=0x7f341800b7e0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 shutdown_connections 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f340803daa0 0x7f340803ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34281a5ca0 0x7f34281a80c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34100035c0 0x7f3428102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 --2- 172.21.15.138:0/1868492082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3428102ec0 0x7f34281032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 >> 172.21.15.138:0/1868492082 conn(0x7f34280fe520 msgr2=0x7f3428101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 shutdown_connections 2024-04-25T13:33:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:39.671+0000 7f342caa8700 1 -- 172.21.15.138:0/1868492082 wait complete. 2024-04-25T13:33:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 3 2024-04-25T13:33:40.295 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T13:33:40.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T13:33:40.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:40.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:40 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/1868492082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:33:40.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:40.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:40 smithi138 ceph-mon[32977]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:40.550 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:40.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:40.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:40 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/1868492082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:33:40.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:40.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:40 smithi094 ceph-mon[29195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:41 smithi137 ceph-mon[32568]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:42.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mon.b calling monitor election 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mon.b calling monitor election 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: monmap epoch 3 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: last_changed 2024-04-25T13:33:34.156502+0000 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: election_strategy: 1 2024-04-25T13:33:42.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: fsmap 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: mgrmap e13: x(active, since 61s) 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:42.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:42.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 -- 172.21.15.94:0/2486843503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8109a10 msgr2=0x7fcad8109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 --2- 172.21.15.94:0/2486843503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8109a10 0x7fcad8109e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009910 tx=0x7fcac8009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 -- 172.21.15.94:0/2486843503 shutdown_connections 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 --2- 172.21.15.94:0/2486843503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8109a10 0x7fcad8109e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 -- 172.21.15.94:0/2486843503 >> 172.21.15.94:0/2486843503 conn(0x7fcad80ff560 msgr2=0x7fcad81019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 -- 172.21.15.94:0/2486843503 shutdown_connections 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.310+0000 7fcadcff1700 1 -- 172.21.15.94:0/2486843503 wait complete. 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 Processor -- start 2024-04-25T13:33:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 -- start start 2024-04-25T13:33:42.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad8109a10 0x7fcad8075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:42.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad81a5ce0 0x7fcad81a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810df60 con 0x7fcad8075ed0 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810dde0 con 0x7fcad81a5ce0 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcadcff1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810e0e0 con 0x7fcad8109a10 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46756/0 (socket says 172.21.15.94:46756) 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 -- 172.21.15.94:0/3767686126 learned_addr learned my addr 172.21.15.94:0/3767686126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 -- 172.21.15.94:0/3767686126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad8109a10 msgr2=0x7fcad8075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad8109a10 0x7fcad8075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 -- 172.21.15.94:0/3767686126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad81a5ce0 msgr2=0x7fcad81a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:42.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad81a5ce0 0x7fcad81a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.311+0000 7fcad5d9b700 1 -- 172.21.15.94:0/3767686126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81a8780 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcad5d9b700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00b780 tx=0x7fcacc00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc004230 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcacc0043d0 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81a8a70 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc011bd0 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.312+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81a9000 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.313+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fcacc01a400 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.313+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8064cf0 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.313+0000 7fcac77fe700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 0x7fcac003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.313+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcacc051200 con 0x7fcad8075ed0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.313+0000 7fcad659c700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 0x7fcac003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:42.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.314+0000 7fcad659c700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 0x7fcac003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:42.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.319+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcacc020740 con 0x7fcad8075ed0 2024-04-25T13:33:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mon.b calling monitor election 2024-04-25T13:33:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T13:33:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T13:33:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T13:33:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mon.b calling monitor election 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mon.c calling monitor election 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mon.a calling monitor election 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: monmap epoch 3 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: last_changed 2024-04-25T13:33:34.156502+0000 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: min_mon_release 19 (squid) 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: election_strategy: 1 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:42.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: fsmap 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: mgrmap e13: x(active, since 61s) 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: overall HEALTH_OK 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:42.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:42.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mon.b calling monitor election 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mon.b calling monitor election 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mon.c calling monitor election 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mon.a calling monitor election 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: monmap epoch 3 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:42.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: last_changed 2024-04-25T13:33:34.156502+0000 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: created 2024-04-25T13:31:42.925512+0000 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: min_mon_release 19 (squid) 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: election_strategy: 1 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: fsmap 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: mgrmap e13: x(active, since 61s) 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:42.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:42.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.519+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcad81a9460 con 0x7fcad8075ed0 2024-04-25T13:33:42.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.520+0000 7fcac77fe700 1 -- 172.21.15.94:0/3767686126 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fcacc015df0 con 0x7fcad8075ed0 2024-04-25T13:33:42.521 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:42.521 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-04-25T13:33:42.521 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:33:42.521 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-25T13:33:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 msgr2=0x7fcac003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 0x7fcac003ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac800b560 comp rx=0 tx=0).stop 2024-04-25T13:33:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 msgr2=0x7fcad80762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00b780 tx=0x7fcacc00bb40 comp rx=0 tx=0).stop 2024-04-25T13:33:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 shutdown_connections 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad81a5ce0 0x7fcad81a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad8109a10 0x7fcad8075990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcac003daa0 0x7fcac003ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 --2- 172.21.15.94:0/3767686126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcad8075ed0 0x7fcad80762e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 >> 172.21.15.94:0/3767686126 conn(0x7fcad80ff560 msgr2=0x7fcad8100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 shutdown_connections 2024-04-25T13:33:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:42.522+0000 7fcadcff1700 1 -- 172.21.15.94:0/3767686126 wait complete. 2024-04-25T13:33:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: Updating smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: Reconfiguring daemon mon.a on smithi094 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3767686126' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:43.475 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:33:43.155+0000 7f3efb1cd700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: Updating smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: Reconfiguring daemon mon.a on smithi094 2024-04-25T13:33:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:43.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:43.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3767686126' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: Updating smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: Updating smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: Reconfiguring daemon mon.a on smithi094 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3767686126' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:43.969 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T13:33:43.970 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:33:43.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:33:43.997 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:33:43.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:44.061 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:33:44.061 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:33:44.088 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:33:44.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:44.152 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:33:44.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:33:44.183 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:33:44.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:44.251 INFO:tasks.cephadm:Adding mgr.x on smithi094 2024-04-25T13:33:44.251 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-04-25T13:33:44.251 INFO:tasks.cephadm:Adding mgr.z on smithi138 2024-04-25T13:33:44.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi094=x;smithi137=y;smithi138=z' 2024-04-25T13:33:44.529 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: Reconfiguring daemon mon.b on smithi137 2024-04-25T13:33:44.703 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:44 smithi137 ceph-mon[32568]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:44.865 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:44.866 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: Reconfiguring daemon mon.b on smithi137 2024-04-25T13:33:44.866 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:44 smithi138 ceph-mon[32977]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: Reconfiguring daemon mon.b on smithi137 2024-04-25T13:33:44.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:44 smithi094 ceph-mon[29195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.105+0000 7f9eccc48700 1 -- 172.21.15.138:0/3356030821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8102ac0 msgr2=0x7f9ec8102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.105+0000 7f9eccc48700 1 --2- 172.21.15.138:0/3356030821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8102ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009910 tx=0x7f9eb8009c20 comp rx=0 tx=0).stop 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 -- 172.21.15.138:0/3356030821 shutdown_connections 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 --2- 172.21.15.138:0/3356030821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8102ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 -- 172.21.15.138:0/3356030821 >> 172.21.15.138:0/3356030821 conn(0x7f9ec80fe120 msgr2=0x7f9ec81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 -- 172.21.15.138:0/3356030821 shutdown_connections 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 -- 172.21.15.138:0/3356030821 wait complete. 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 Processor -- start 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.106+0000 7f9eccc48700 1 -- start start 2024-04-25T13:33:46.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 0x7f9ec81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8108a80 con 0x7f9ec8199d70 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8108900 con 0x7f9ec8102ac0 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9eccc48700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8108c00 con 0x7f9ec819f240 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 0x7f9ec81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.138:48456/0 (socket says 172.21.15.138:48456) 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 0x7f9ec81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38722/0 (socket says 172.21.15.138:38722) 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 -- 172.21.15.138:0/1569602292 learned_addr learned my addr 172.21.15.138:0/1569602292 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec659c700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 -- 172.21.15.138:0/1569602292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 msgr2=0x7f9ec81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 0x7f9ec81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 -- 172.21.15.138:0/1569602292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec8102ac0 msgr2=0x7f9ec8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.107+0000 7f9ec5d9b700 1 -- 172.21.15.138:0/1569602292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec81a1bd0 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9ec5d9b700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b780 tx=0x7f9ebc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc016070 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec81a1ec0 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec81a23f0 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ebc0036a0 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.108+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011990 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.109+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea8005360 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.109+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f9ebc019070 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.109+0000 7f9eb77fe700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 0x7f9eb003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.109+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ebc050080 con 0x7f9ec8199d70 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.110+0000 7f9ec659c700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 0x7f9eb003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:46.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.110+0000 7f9ec659c700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 0x7f9eb003ffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8006010 tx=0x7f9eb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:46.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.115+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ebc0176c0 con 0x7f9ec8199d70 2024-04-25T13:33:46.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.331+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}) -- 0x7f9ea8000bf0 con 0x7f9eb003daa0 2024-04-25T13:33:46.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.338+0000 7f9eb77fe700 1 -- 172.21.15.138:0/1569602292 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ea8000bf0 con 0x7f9eb003daa0 2024-04-25T13:33:46.339 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mgr update... 2024-04-25T13:33:46.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 msgr2=0x7f9eb003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:46.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 0x7f9eb003ffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8006010 tx=0x7f9eb801a040 comp rx=0 tx=0).stop 2024-04-25T13:33:46.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 msgr2=0x7f9ec819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:46.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b780 tx=0x7f9ebc00ba90 comp rx=0 tx=0).stop 2024-04-25T13:33:46.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 shutdown_connections 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ec819f240 0x7f9ec81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ec8102ac0 0x7f9ec8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9eb003daa0 0x7f9eb003ffb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 --2- 172.21.15.138:0/1569602292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec8199d70 0x7f9ec819ed00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 >> 172.21.15.138:0/1569602292 conn(0x7f9ec80fe120 msgr2=0x7f9ec806bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 shutdown_connections 2024-04-25T13:33:46.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:33:46.340+0000 7f9eccc48700 1 -- 172.21.15.138:0/1569602292 wait complete. 2024-04-25T13:33:46.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:46.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:46.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:46.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:46.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:46.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:33:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:33:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:47.270 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:47 smithi138 ceph-mon[32977]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:33:47.270 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:47 smithi138 ceph-mon[32977]: Reconfiguring daemon mon.c on smithi138 2024-04-25T13:33:47.271 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:47 smithi138 ceph-mon[32977]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:47.271 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:47.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:47 smithi094 ceph-mon[29195]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:33:47.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:47 smithi094 ceph-mon[29195]: Reconfiguring daemon mon.c on smithi138 2024-04-25T13:33:47.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:47 smithi094 ceph-mon[29195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:47.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:47 smithi137 ceph-mon[32568]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:33:47.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:47 smithi137 ceph-mon[32568]: Reconfiguring daemon mon.c on smithi138 2024-04-25T13:33:47.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:47 smithi137 ceph-mon[32568]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:47.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:47.712 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T13:33:47.715 DEBUG:teuthology.orchestra.run.smithi138:mgr.z> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T13:33:47.718 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T13:33:47.718 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:33:47.718 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:33:47.734 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:33:47.734 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-25T13:33:47.791 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 13:32:52.412438900 +0000 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 13:28:02.156250978 +0000 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 13:28:02.156250978 +0000 2024-04-25T13:33:47.792 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 13:28:02.156250978 +0000 2024-04-25T13:33:47.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:33:47.855 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T13:33:47.855 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T13:33:47.855 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.175e-05 s, 5.6 MB/s 2024-04-25T13:33:47.856 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:33:47.913 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 13:32:52.452438257 +0000 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 13:28:02.484245286 +0000 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 13:28:02.484245286 +0000 2024-04-25T13:33:47.968 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 13:28:02.484245286 +0000 2024-04-25T13:33:47.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:33:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T13:33:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T13:33:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000113487 s, 4.5 MB/s 2024-04-25T13:33:48.031 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:33:48.088 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-25T13:33:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 13:32:52.485437728 +0000 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 13:28:02.812239596 +0000 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 13:28:02.812239596 +0000 2024-04-25T13:33:48.143 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 13:28:02.812239596 +0000 2024-04-25T13:33:48.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:33:48.206 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T13:33:48.206 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T13:33:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.0297e-05 s, 5.7 MB/s 2024-04-25T13:33:48.207 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:33:48.263 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 13:32:52.518437197 +0000 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 13:28:03.161233540 +0000 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 13:28:03.161233540 +0000 2024-04-25T13:33:48.319 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 13:28:03.161233540 +0000 2024-04-25T13:33:48.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:33:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T13:33:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T13:33:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000128443 s, 4.0 MB/s 2024-04-25T13:33:48.382 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:33:48.438 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T13:33:48.439 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:33:48.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:33:48.458 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-04-25T13:33:48.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:48.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:48.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:48.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 13:33:33.598344998 +0000 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 13:28:20.136590351 +0000 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 13:28:20.136590351 +0000 2024-04-25T13:33:48.515 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 13:28:20.136590351 +0000 2024-04-25T13:33:48.516 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:48.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:48.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:48.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:48.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:48.581 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T13:33:48.581 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T13:33:48.581 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000274525 s, 1.9 MB/s 2024-04-25T13:33:48.583 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:33:48.640 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-04-25T13:33:48.697 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 13:33:33.598344998 +0000 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 13:28:20.471583647 +0000 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 13:28:20.471583647 +0000 2024-04-25T13:33:48.698 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 13:28:20.471583647 +0000 2024-04-25T13:33:48.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:33:48.762 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T13:33:48.762 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T13:33:48.763 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000203376 s, 2.5 MB/s 2024-04-25T13:33:48.764 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:33:48.821 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 13:33:33.598344998 +0000 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 13:28:20.823576604 +0000 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 13:28:20.823576604 +0000 2024-04-25T13:33:48.879 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 13:28:20.823576604 +0000 2024-04-25T13:33:48.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:33:48.942 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T13:33:48.943 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T13:33:48.943 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000228254 s, 2.2 MB/s 2024-04-25T13:33:48.944 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:33:49.000 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 13:33:33.599344979 +0000 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 13:28:21.162569820 +0000 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 13:28:21.162569820 +0000 2024-04-25T13:33:49.056 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 13:28:21.162569820 +0000 2024-04-25T13:33:49.056 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:33:49.119 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T13:33:49.119 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T13:33:49.119 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000225796 s, 2.3 MB/s 2024-04-25T13:33:49.120 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:33:49.175 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T13:33:49.176 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:33:49.191 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:33:49.192 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 13:33:23.553558937 +0000 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 13:28:22.769742900 +0000 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 13:28:22.769742900 +0000 2024-04-25T13:33:49.247 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 13:28:22.769742900 +0000 2024-04-25T13:33:49.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:33:49.311 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T13:33:49.312 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T13:33:49.312 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000278723 s, 1.8 MB/s 2024-04-25T13:33:49.313 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:33:49.368 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T13:33:49.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:49 smithi137 ceph-mon[32568]: Deploying daemon mgr.z on smithi138 2024-04-25T13:33:49.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:49 smithi137 ceph-mon[32568]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:33:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:49.423 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T13:33:49.423 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:49.423 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:49.424 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 13:33:23.553558937 +0000 2024-04-25T13:33:49.424 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 13:28:23.101735577 +0000 2024-04-25T13:33:49.424 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 13:28:23.101735577 +0000 2024-04-25T13:33:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 13:28:23.101735577 +0000 2024-04-25T13:33:49.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:33:49.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:49 smithi094 ceph-mon[29195]: Deploying daemon mgr.z on smithi138 2024-04-25T13:33:49.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:49 smithi094 ceph-mon[29195]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:49.483 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:49 smithi138 ceph-mon[32977]: Deploying daemon mgr.z on smithi138 2024-04-25T13:33:49.483 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:49 smithi138 ceph-mon[32977]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:49.485 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T13:33:49.485 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T13:33:49.485 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000185993 s, 2.8 MB/s 2024-04-25T13:33:49.486 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:33:49.542 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 13:33:23.553558937 +0000 2024-04-25T13:33:49.597 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 13:28:23.434728232 +0000 2024-04-25T13:33:49.598 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 13:28:23.434728232 +0000 2024-04-25T13:33:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 13:28:23.434728232 +0000 2024-04-25T13:33:49.598 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:33:49.660 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T13:33:49.660 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T13:33:49.660 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000182178 s, 2.8 MB/s 2024-04-25T13:33:49.661 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:33:49.723 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T13:33:49.779 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:33:49.779 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:33:49.779 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 13:33:23.554558917 +0000 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 13:28:23.766720910 +0000 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 13:28:23.766720910 +0000 2024-04-25T13:33:49.780 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 13:28:23.766720910 +0000 2024-04-25T13:33:49.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:33:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T13:33:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T13:33:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000188 s, 2.7 MB/s 2024-04-25T13:33:49.848 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:33:49.910 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-25T13:33:49.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:33:50.166 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:50.246 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 systemd[1]: Starting Ceph mgr.z for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:33:50.498 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 podman[34818]: 2024-04-25 13:33:50.150457751 +0000 UTC m=+0.021265216 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:33:50.498 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 podman[34818]: 2024-04-25 13:33:50.278144218 +0000 UTC m=+0.148951682 container create 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:33:50.879 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 podman[34818]: 2024-04-25 13:33:50.504896192 +0000 UTC m=+0.375703669 container init 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:33:50.879 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 podman[34818]: 2024-04-25 13:33:50.508803238 +0000 UTC m=+0.379610704 container start 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:33:50.879 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 ceph-mgr[34845]: -- 172.21.15.138:0/1466367193 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55587bdd25a0 con 0x555880b5c000 2024-04-25T13:33:50.879 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 bash[34818]: 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 2024-04-25T13:33:50.880 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 systemd[1]: Started Ceph mgr.z for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:33:50.880 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:50.689+0000 7f02c7599200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:33:50.880 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:50.877+0000 7f02c7599200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:33:51.189 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:50.981+0000 7f02c7599200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:51.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:51.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:51.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:51.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:51.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:51.497 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:51.300+0000 7f02c7599200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:33:51.497 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:51.384+0000 7f02c7599200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:51.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0086063 s, 1.2 GB/s 2024-04-25T13:33:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T13:33:52.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:52 smithi094 ceph-mon[29195]: Deploying daemon mgr.y on smithi137 2024-04-25T13:33:52.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:52 smithi138 ceph-mon[32977]: Deploying daemon mgr.y on smithi137 2024-04-25T13:33:52.561 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:52 smithi137 ceph-mon[32568]: Deploying daemon mgr.y on smithi137 2024-04-25T13:33:52.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-25T13:33:52.933 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:52.584+0000 7f02c7599200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:33:52.933 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:52.760+0000 7f02c7599200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:33:52.933 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:52.931+0000 7f02c7599200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:33:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:33:53.144 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:52 smithi137 systemd[1]: Starting Ceph mgr.y for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:33:53.362 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:53 smithi094 ceph-mon[29195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:53.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:53 smithi137 ceph-mon[32568]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:53.461 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 podman[33954]: 2024-04-25 13:33:53.18827449 +0000 UTC m=+0.092745463 container create 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:33:53.461 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 podman[33954]: 2024-04-25 13:33:53.113374769 +0000 UTC m=+0.017845760 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:33:53.461 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 podman[33954]: 2024-04-25 13:33:53.424059682 +0000 UTC m=+0.328530656 container init 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:33:53.461 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 podman[33954]: 2024-04-25 13:33:53.42670321 +0000 UTC m=+0.331174185 container start 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:33:53.461 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 ceph-mgr[33973]: -- 172.21.15.137:0/780605577 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b8fdb25a0 con 0x558b94b3c480 2024-04-25T13:33:53.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:53 smithi138 ceph-mon[32977]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:53.497 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:53.215+0000 7f02c7599200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:33:53.498 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:53.306+0000 7f02c7599200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:33:53.786 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 bash[33954]: 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 2024-04-25T13:33:53.786 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 systemd[1]: Started Ceph mgr.y for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:33:53.786 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:53.594+0000 7f2934e6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:33:53.786 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:53.785+0000 7f2934e6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:33:53.997 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:53.511+0000 7f02c7599200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:33:53.997 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:53.702+0000 7f02c7599200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:33:54.143 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:53.891+0000 7f2934e6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:33:54.570 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:54.225+0000 7f2934e6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:33:54.570 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:54.316+0000 7f2934e6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:33:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/2740904622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 msgr2=0x7fdbc4103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/2740904622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4103530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009990 tx=0x7fdbc0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/2740904622 shutdown_connections 2024-04-25T13:33:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/2740904622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104bf0 0x7fdbc410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/2740904622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4103530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/2740904622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104250 0x7fdbc4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.646+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/2740904622 >> 172.21.15.94:0/2740904622 conn(0x7fdbc40fe760 msgr2=0x7fdbc4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:33:54.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:54.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:54 smithi094 ceph-mon[29195]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/2740904622 shutdown_connections 2024-04-25T13:33:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/2740904622 wait complete. 2024-04-25T13:33:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 Processor -- start 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 -- start start 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104250 0x7fdbc419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.647+0000 7fdbcbdf2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104bf0 0x7fdbc41a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbcbdf2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc410eb60 con 0x7fdbc4103120 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbcbdf2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc410e9e0 con 0x7fdbc4104bf0 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbcbdf2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc410ece0 con 0x7fdbc4104250 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42674/0 (socket says 172.21.15.94:42674) 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 -- 172.21.15.94:0/3748110150 learned_addr learned my addr 172.21.15.94:0/3748110150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc938d700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104250 0x7fdbc419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbca38f700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104bf0 0x7fdbc41a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 -- 172.21.15.94:0/3748110150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104250 msgr2=0x7fdbc419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104250 0x7fdbc419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 -- 172.21.15.94:0/3748110150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104bf0 msgr2=0x7fdbc41a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104bf0 0x7fdbc41a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 -- 172.21.15.94:0/3748110150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41a2390 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbc9b8e700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000c00 tx=0x7fdbc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0004830 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc41a2620 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc0005020 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003d90 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.648+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc41a2a10 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.650+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4051190 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.650+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdbc0003f30 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.650+0000 7fdbbaffd700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 0x7fdbb003fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.650+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdbc00526d0 con 0x7fdbc4103120 2024-04-25T13:33:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.650+0000 7fdbc938d700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 0x7fdbb003fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:33:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.651+0000 7fdbc938d700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 0x7fdbb003fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4075cd0 tx=0x7fdbb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:33:54.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.656+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbc0013070 con 0x7fdbc4103120 2024-04-25T13:33:54.747 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:54.342+0000 7f02c7599200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:33:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:54.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-mon[32977]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:54.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:54.860+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdbc4063ad0 con 0x7fdbb003d850 2024-04-25T13:33:54.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:54.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:54.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:54 smithi137 ceph-mon[32568]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:55.087 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:54.776+0000 7f02c7599200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:33:55.087 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:54.873+0000 7f02c7599200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:33:55.087 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:55.085+0000 7f02c7599200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:33:55.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:55.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:33:55.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:33:55.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T13:33:55.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.864 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:55.528+0000 7f2934e6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:33:55.864 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:55.694+0000 7f2934e6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:33:55.864 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:55.863+0000 7f2934e6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.865 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:33:55.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:55.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:55.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T13:33:55.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.927 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:55.601+0000 7f02c7599200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:55.690+0000 7f02c7599200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:55.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:33:55.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:33:55.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:33:55.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:55.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T13:33:55.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:56.247 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:55.925+0000 7f02c7599200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:33:56.248 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:56.055+0000 7f02c7599200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:33:56.248 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:56.147+0000 7f02c7599200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:33:56.435 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:56 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:56.143+0000 7f2934e6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:33:56.435 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:56 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:56.233+0000 7f2934e6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:33:56.435 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:56 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:56.434+0000 7f2934e6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:33:56.562 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:56.260+0000 7f02c7599200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:33:56.562 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:56.560+0000 7f02c7599200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:33:56.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:56 smithi094 ceph-mon[29195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:56.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:56 smithi137 ceph-mon[32568]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:56.894 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:56 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:56.622+0000 7f2934e6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:33:56.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:56 smithi138 ceph-mon[32977]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:57.497 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:57.147+0000 7f02c7599200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:33:57.497 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z[34836]: 2024-04-25T13:33:57.356+0000 7f02c7599200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:33:57.593 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:57.211+0000 7f2934e6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:33:57.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:33:57.599+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7fdbc00051d0 con 0x7fdbc4103120 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: Standby manager daemon z started 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:33:57.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:57.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:57.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:57.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:57.626+0000 7f2934e6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:57.722+0000 7f2934e6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: Standby manager daemon z started 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:57.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: Standby manager daemon z started 2024-04-25T13:33:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:33:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3746321657' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:57.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:33:58.393 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:57.917+0000 7f2934e6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:33:58.758 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:58.430+0000 7f2934e6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:33:58.759 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:58.520+0000 7f2934e6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:33:58.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-mon[32568]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:33:58.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:33:58.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-mon[32568]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:58 smithi094 ceph-mon[29195]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:33:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:33:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:58 smithi094 ceph-mon[29195]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:58.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:58 smithi138 ceph-mon[32977]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:33:58.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:33:58.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:58 smithi138 ceph-mon[32977]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:33:59.091 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:58.757+0000 7f2934e6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:33:59.091 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:58.886+0000 7f2934e6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:33:59.091 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:58.977+0000 7f2934e6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:33:59.091 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:59.090+0000 7f2934e6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:33:59.643 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:59.390+0000 7f2934e6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:33:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]: dispatch 2024-04-25T13:33:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]': finished 2024-04-25T13:33:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-mon[32568]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:33:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:33:59.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3530433721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:33:59.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:59 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]: dispatch 2024-04-25T13:33:59.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:59 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]': finished 2024-04-25T13:33:59.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:59 smithi094 ceph-mon[29195]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:33:59.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:59 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:33:59.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:33:59 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3530433721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:33:59.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:59 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]: dispatch 2024-04-25T13:33:59.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:59 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3785404625' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71e93888-dbdc-44cc-b0c5-bd211fc1fbed"}]': finished 2024-04-25T13:33:59.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:59 smithi138 ceph-mon[32977]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:33:59.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:59 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:33:59.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:33:59 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3530433721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:00.393 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:33:59 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:33:59.976+0000 7f2934e6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:34:00.394 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y[33969]: 2024-04-25T13:34:00.186+0000 7f2934e6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:34:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:00.615+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fdbc00054f0 con 0x7fdbc4103120 2024-04-25T13:34:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: Standby manager daemon y started 2024-04-25T13:34:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:34:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:34:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:34:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:00 smithi137 ceph-mon[32568]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:34:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: Standby manager daemon y started 2024-04-25T13:34:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:34:00.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:34:00.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:34:00.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:00 smithi094 ceph-mon[29195]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:34:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: Standby manager daemon y started 2024-04-25T13:34:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:34:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:34:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:34:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:00 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.137:0/1585483343' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:34:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:01 smithi137 ceph-mon[32568]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T13:34:01.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:01 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:34:01.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:01 smithi094 ceph-mon[29195]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T13:34:01.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:01 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:34:01.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:01 smithi138 ceph-mon[32977]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T13:34:01.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:01 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:34:02.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:02 smithi137 ceph-mon[32568]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:02.907 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:02 smithi094 ceph-mon[29195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:02.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:02 smithi138 ceph-mon[32977]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:04.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:04 smithi094 ceph-mon[29195]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:04 smithi137 ceph-mon[32568]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:04 smithi138 ceph-mon[32977]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:05.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:34:05.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:06.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:34:06.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:06.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:34:06.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:06.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:06 smithi094 ceph-mon[29195]: Deploying daemon osd.0 on smithi094 2024-04-25T13:34:06.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:06 smithi094 ceph-mon[29195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:07.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:06 smithi137 ceph-mon[32568]: Deploying daemon osd.0 on smithi094 2024-04-25T13:34:07.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:06 smithi137 ceph-mon[32568]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:07.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:06 smithi138 ceph-mon[32977]: Deploying daemon osd.0 on smithi094 2024-04-25T13:34:07.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:06 smithi138 ceph-mon[32977]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:08.364 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:08 smithi094 ceph-mon[29195]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:08.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:08 smithi137 ceph-mon[32568]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:08 smithi138 ceph-mon[32977]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:10.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:10 smithi094 ceph-mon[29195]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:10 smithi137 ceph-mon[32568]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:10 smithi138 ceph-mon[32977]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:11.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:11.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:11.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:12.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:12 smithi094 ceph-mon[29195]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:12 smithi138 ceph-mon[32977]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:12 smithi137 ceph-mon[32568]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:14.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:14 smithi094 ceph-mon[29195]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:14 smithi137 ceph-mon[32568]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:14 smithi138 ceph-mon[32977]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.272+0000 7fdbbaffd700 1 -- 172.21.15.94:0/3748110150 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbc4063ad0 con 0x7fdbb003d850 2024-04-25T13:34:15.273 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 msgr2=0x7fdbb003fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 0x7fdbb003fd60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4075cd0 tx=0x7fdbb400b410 comp rx=0 tx=0).stop 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 msgr2=0x7fdbc4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000c00 tx=0x7fdbc00047b0 comp rx=0 tx=0).stop 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 shutdown_connections 2024-04-25T13:34:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbc4104bf0 0x7fdbc41a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbb003d850 0x7fdbb003fd60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbc4103120 0x7fdbc4076320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 --2- 172.21.15.94:0/3748110150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc4104250 0x7fdbc419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 >> 172.21.15.94:0/3748110150 conn(0x7fdbc40fe760 msgr2=0x7fdbc4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 shutdown_connections 2024-04-25T13:34:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:34:15.276+0000 7fdbcbdf2700 1 -- 172.21.15.94:0/3748110150 wait complete. 2024-04-25T13:34:15.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:15.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:15 smithi094 ceph-mon[29195]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:34:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:15 smithi137 ceph-mon[32568]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:34:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:15 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:34:15.877 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T13:34:15.880 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-04-25T13:34:15.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:34:16.121 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:34:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:16 smithi094 ceph-mon[29195]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:16.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:16 smithi094 ceph-mon[29195]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:34:16.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:16 smithi094 ceph-mon[29195]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:34:16.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:16 smithi094 ceph-mon[29195]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:34:16.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:16 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:16.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:16 smithi137 ceph-mon[32568]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:16.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:16 smithi137 ceph-mon[32568]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:34:16.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:16 smithi137 ceph-mon[32568]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:34:16.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:16 smithi137 ceph-mon[32568]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:34:16.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:16 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:16 smithi138 ceph-mon[32977]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:16.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:16 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:34:16.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:16 smithi138 ceph-mon[32977]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:34:16.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:16 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:34:16.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:16 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119218 s, 880 MB/s 2024-04-25T13:34:18.062 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T13:34:18.226 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:34:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0[44411]: 2024-04-25T13:34:17.928+0000 7fc405f53700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:34:18.226 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:34:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0[44411]: 2024-04-25T13:34:17.932+0000 7fc3fdd00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' 2024-04-25T13:34:18.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:18 smithi138 ceph-mon[32977]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:18.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:34:18.521 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:34:18.521 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.521 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.521 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' 2024-04-25T13:34:18.521 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:18 smithi094 ceph-mon[29195]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: from='osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455]' entity='osd.0' 2024-04-25T13:34:18.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:18 smithi137 ceph-mon[32568]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:34:18.764 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-04-25T13:34:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi094 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:19.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] boot 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:34:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:19.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi094 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:19.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] boot 2024-04-25T13:34:19.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:34:19.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:19.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:34:19.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:34:19.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:19.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi094 2024-04-25T13:34:19.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: osd.0 [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] boot 2024-04-25T13:34:19.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:34:19.728 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:34:20.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:20 smithi137 ceph-mon[32568]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:20.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:20 smithi138 ceph-mon[32977]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:20.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 -- 172.21.15.137:0/1823904817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8104bd0 msgr2=0x7ff9d810b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:20.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/1823904817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8104bd0 0x7ff9d810b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b0a0 tx=0x7ff9d000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 -- 172.21.15.137:0/1823904817 shutdown_connections 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/1823904817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8104bd0 0x7ff9d810b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/1823904817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8103100 0x7ff9d8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/1823904817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d8104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 -- 172.21.15.137:0/1823904817 >> 172.21.15.137:0/1823904817 conn(0x7ff9d80fe740 msgr2=0x7ff9d8100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 -- 172.21.15.137:0/1823904817 shutdown_connections 2024-04-25T13:34:20.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.606+0000 7ff9dce1a700 1 -- 172.21.15.137:0/1823904817 wait complete. 2024-04-25T13:34:20.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 Processor -- start 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 -- start start 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 0x7ff9d81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810ebd0 con 0x7ff9d8104bd0 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810ea50 con 0x7ff9d8103100 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9dce1a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810ed50 con 0x7ff9d8104230 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d5d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53830/0 (socket says 172.21.15.137:53830) 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 -- 172.21.15.137:0/839982802 learned_addr learned my addr 172.21.15.137:0/839982802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d6d9d700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 0x7ff9d81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 -- 172.21.15.137:0/839982802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 msgr2=0x7ff9d819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:20.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 -- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 msgr2=0x7ff9d81a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 0x7ff9d81a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d659c700 1 -- 172.21.15.137:0/839982802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d81a2310 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d6d9d700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 0x7ff9d81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.607+0000 7ff9d5d9b700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9d659c700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800d950 tx=0x7ff9c800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8017070 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c8009730 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d81a2600 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8003a50 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.608+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a2b40 con 0x7ff9d8103100 2024-04-25T13:34:20.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.609+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d8051190 con 0x7ff9d8103100 2024-04-25T13:34:20.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.611+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7ff9c8003bf0 con 0x7ff9d8103100 2024-04-25T13:34:20.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.612+0000 7ff9c77fe700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 0x7ff9c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:20.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.612+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff9c80dd470 con 0x7ff9d8103100 2024-04-25T13:34:20.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.612+0000 7ff9d5d9b700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 0x7ff9c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:20.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.613+0000 7ff9d5d9b700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 0x7ff9c00b2000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d819b860 tx=0x7ff9cc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:34:20.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.616+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9c8020030 con 0x7ff9d8103100 2024-04-25T13:34:20.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:20 smithi094 ceph-mon[29195]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:20.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:20.820+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff9d81a2e60 con 0x7ff9c00afaf0 2024-04-25T13:34:21.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:21 smithi138 ceph-mon[32977]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:34:21.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:21 smithi138 ceph-mon[32977]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:21.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:21.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:21.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:21.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:21 smithi137 ceph-mon[32568]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:34:21.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:21 smithi137 ceph-mon[32568]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:21.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:21.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:21.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:21.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:21 smithi094 ceph-mon[29195]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:34:21.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:21 smithi094 ceph-mon[29195]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:21.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:21.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:21.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:22 smithi138 ceph-mon[32977]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:22.519 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:22 smithi137 ceph-mon[32568]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:22 smithi094 ceph-mon[29195]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:23.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:23 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/3014494857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:23 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:23 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]': finished 2024-04-25T13:34:23.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:23 smithi137 ceph-mon[32568]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:34:23.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:23.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:23 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/3014494857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:23 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:23 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]': finished 2024-04-25T13:34:23.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:23 smithi094 ceph-mon[29195]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:34:23.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:23.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:23 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/3014494857' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:23 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]: dispatch 2024-04-25T13:34:23.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:23 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "814d5805-282b-4231-9570-1ff9e7e41fec"}]': finished 2024-04-25T13:34:23.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:23 smithi138 ceph-mon[32977]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:34:23.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:24.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:24 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/3754309225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:24.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:24 smithi137 ceph-mon[32568]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:24.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:24 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/3754309225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:24.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:24 smithi094 ceph-mon[29195]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:24.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:24 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/3754309225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:24.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:24 smithi138 ceph-mon[32977]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:26 smithi137 ceph-mon[32568]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:26 smithi094 ceph-mon[29195]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:26 smithi138 ceph-mon[32977]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:28.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:28 smithi137 ceph-mon[32568]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:28.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:28 smithi094 ceph-mon[29195]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:28.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:28 smithi138 ceph-mon[32977]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:30 smithi138 ceph-mon[32977]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:30.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:30 smithi137 ceph-mon[32568]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:30.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:30 smithi094 ceph-mon[29195]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:31.587 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:34:31.587 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:31.587 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:31 smithi137 ceph-mon[32568]: Deploying daemon osd.1 on smithi137 2024-04-25T13:34:31.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:34:31.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:31.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:31 smithi094 ceph-mon[29195]: Deploying daemon osd.1 on smithi137 2024-04-25T13:34:31.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:34:31.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:31.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:31 smithi138 ceph-mon[32977]: Deploying daemon osd.1 on smithi137 2024-04-25T13:34:32.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:32 smithi137 ceph-mon[32568]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:32.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:32 smithi094 ceph-mon[29195]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:32.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:32 smithi138 ceph-mon[32977]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:34.550 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:34 smithi137 ceph-mon[32568]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:34 smithi094 ceph-mon[29195]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:34 smithi138 ceph-mon[32977]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:36 smithi137 ceph-mon[32568]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:36.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:36 smithi094 ceph-mon[29195]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:36 smithi138 ceph-mon[32977]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:38.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:38 smithi137 ceph-mon[32568]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:38.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:38 smithi094 ceph-mon[29195]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:38 smithi138 ceph-mon[32977]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:40.237 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-04-25T13:34:40.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.236+0000 7ff9c77fe700 1 -- 172.21.15.137:0/839982802 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9d81a2e60 con 0x7ff9c00afaf0 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 msgr2=0x7ff9c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 0x7ff9c00b2000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d819b860 tx=0x7ff9cc0073d0 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 msgr2=0x7ff9d819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800d950 tx=0x7ff9c800dc60 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 shutdown_connections 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9c00afaf0 0x7ff9c00b2000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8104bd0 0x7ff9d81a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8103100 0x7ff9d819a7b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 --2- 172.21.15.137:0/839982802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9d8104230 0x7ff9d819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.239+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 >> 172.21.15.137:0/839982802 conn(0x7ff9d80fe740 msgr2=0x7ff9d80fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.240+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 shutdown_connections 2024-04-25T13:34:40.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:34:40.240+0000 7ff9dce1a700 1 -- 172.21.15.137:0/839982802 wait complete. 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:40.493 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:40.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:40.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:40.894 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T13:34:40.897 INFO:tasks.cephadm:Deploying osd.2 on smithi138 with /dev/vg_nvme/lv_4... 2024-04-25T13:34:40.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:34:41.142 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:34:41.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:41 smithi137 ceph-mon[32568]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:41.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:41 smithi137 ceph-mon[32568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:41.616 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:41 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:41.617 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:41 smithi138 ceph-mon[32977]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:41 smithi094 ceph-mon[29195]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:41 smithi094 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:34:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:34:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:34:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-mon[32568]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:42.394 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1[37198]: 2024-04-25T13:34:42.339+0000 7fa3c254f700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:34:42.394 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:34:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1[37198]: 2024-04-25T13:34:42.343+0000 7fa3be304700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:42 smithi138 ceph-mon[32977]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:34:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:34:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:42.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: from='osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:34:42.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:42 smithi094 ceph-mon[29195]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126885 s, 826 MB/s 2024-04-25T13:34:43.130 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T13:34:43.596 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:43 smithi137 ceph-mon[32568]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:34:43.596 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:43 smithi137 ceph-mon[32568]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:34:43.596 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.596 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.597 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:43 smithi138 ceph-mon[32977]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:34:43.597 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:43 smithi138 ceph-mon[32977]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:34:43.597 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.598 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:43 smithi094 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:34:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:43 smithi094 ceph-mon[29195]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:34:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:43.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-04-25T13:34:43.995 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:34:44.400 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] boot 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi137 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.401 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:44 smithi138 ceph-mon[32977]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] boot 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi137 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:44 smithi137 ceph-mon[32568]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:44.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: osd.1 [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] boot 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi137 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:44.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:34:44.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:44 smithi094 ceph-mon[29195]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 -- 172.21.15.138:0/4186463299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8105340 msgr2=0x7faee8105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 --2- 172.21.15.138:0/4186463299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8105340 0x7faee8105750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faed8009990 tx=0x7faed8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 -- 172.21.15.138:0/4186463299 shutdown_connections 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 --2- 172.21.15.138:0/4186463299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105ce0 0x7faee810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 --2- 172.21.15.138:0/4186463299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8104210 0x7faee8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 --2- 172.21.15.138:0/4186463299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8105340 0x7faee8105750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 -- 172.21.15.138:0/4186463299 >> 172.21.15.138:0/4186463299 conn(0x7faee80ff890 msgr2=0x7faee8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 -- 172.21.15.138:0/4186463299 shutdown_connections 2024-04-25T13:34:45.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.554+0000 7faeefbb4700 1 -- 172.21.15.138:0/4186463299 wait complete. 2024-04-25T13:34:45.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 Processor -- start 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 -- start start 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105340 0x7faee819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8105ce0 0x7faee81a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faee810fcc0 con 0x7faee8105340 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faee810fb40 con 0x7faee8105ce0 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeefbb4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faee810fe40 con 0x7faee8104210 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43808/0 (socket says 172.21.15.138:43808) 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 -- 172.21.15.138:0/3456030201 learned_addr learned my addr 172.21.15.138:0/3456030201 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:34:45.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed14f700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105340 0x7faee819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeee151700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8105ce0 0x7faee81a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 -- 172.21.15.138:0/3456030201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8105ce0 msgr2=0x7faee81a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8105ce0 0x7faee81a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 -- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105340 msgr2=0x7faee819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105340 0x7faee819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.555+0000 7faeed950700 1 -- 172.21.15.138:0/3456030201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee81a2370 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faeed950700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faee400d950 tx=0x7faee400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4017070 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee400f690 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee81a2660 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4003aa0 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.556+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee81a2bf0 con 0x7faee8104210 2024-04-25T13:34:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.557+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee8051190 con 0x7faee8104210 2024-04-25T13:34:45.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.558+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7faee400f840 con 0x7faee8104210 2024-04-25T13:34:45.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.559+0000 7faedeffd700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 0x7faed40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:34:45.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.559+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7faee40dd590 con 0x7faee8104210 2024-04-25T13:34:45.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.559+0000 7faeed14f700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 0x7faed40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:34:45.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.560+0000 7faeed14f700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 0x7faed40b2080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faed80038a0 tx=0x7faed801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:34:45.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.563+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faee4020030 con 0x7faee8104210 2024-04-25T13:34:45.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:45 smithi137 ceph-mon[32568]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:34:45.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:45 smithi094 ceph-mon[29195]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:34:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:45 smithi138 ceph-mon[32977]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:34:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:34:45.767+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7faee8063ad0 con 0x7faed40afb70 2024-04-25T13:34:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:46 smithi137 ceph-mon[32568]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:46.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:46 smithi137 ceph-mon[32568]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:46.661 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:46 smithi138 ceph-mon[32977]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:46.662 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:46.662 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:46.662 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:46.662 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:46 smithi138 ceph-mon[32977]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:46.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:46 smithi094 ceph-mon[29195]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:34:46.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:34:46.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:34:46.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:46.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:46 smithi094 ceph-mon[29195]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:48 smithi137 ceph-mon[32568]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:48.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:48 smithi094 ceph-mon[29195]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:48 smithi138 ceph-mon[32977]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:49.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/3582669019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]': finished 2024-04-25T13:34:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:34:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:34:49.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/2758196386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/3582669019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]': finished 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:34:49.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/2758196386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:49.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3582669019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]: dispatch 2024-04-25T13:34:49.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d09b14-8c4c-48b2-b295-b6581d524d75"}]': finished 2024-04-25T13:34:49.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:34:49.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:34:49.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/2758196386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:34:50.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:50 smithi137 ceph-mon[32568]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:50 smithi094 ceph-mon[29195]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:50.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:50 smithi138 ceph-mon[32977]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:52 smithi137 ceph-mon[32568]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:52 smithi138 ceph-mon[32977]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:52.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:52 smithi094 ceph-mon[29195]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:54.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:54 smithi138 ceph-mon[32977]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:54.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:54 smithi137 ceph-mon[32568]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:54 smithi094 ceph-mon[29195]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:55.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:34:55.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:56.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:34:56.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:56.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:34:56.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:56.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:56 smithi138 ceph-mon[32977]: Deploying daemon osd.2 on smithi138 2024-04-25T13:34:56.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:56 smithi138 ceph-mon[32977]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:57.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:56 smithi094 ceph-mon[29195]: Deploying daemon osd.2 on smithi138 2024-04-25T13:34:57.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:56 smithi094 ceph-mon[29195]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:57.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:56 smithi137 ceph-mon[32568]: Deploying daemon osd.2 on smithi138 2024-04-25T13:34:57.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:56 smithi137 ceph-mon[32568]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:58.572 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:34:58 smithi138 ceph-mon[32977]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:58.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:34:58 smithi137 ceph-mon[32568]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:34:58.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:34:58 smithi094 ceph-mon[29195]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:00.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:00 smithi138 ceph-mon[32977]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:00.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:00 smithi137 ceph-mon[32568]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:00 smithi094 ceph-mon[29195]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:02.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:02 smithi138 ceph-mon[32977]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:02 smithi137 ceph-mon[32568]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:02 smithi094 ceph-mon[29195]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:04.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.331+0000 7faedeffd700 1 -- 172.21.15.138:0/3456030201 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faee8063ad0 con 0x7faed40afb70 2024-04-25T13:35:04.332 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 2 on host 'smithi138' 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 msgr2=0x7faed40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 0x7faed40b2080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faed80038a0 tx=0x7faed801a040 comp rx=0 tx=0).stop 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 msgr2=0x7faee819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faee400d950 tx=0x7faee400dc60 comp rx=0 tx=0).stop 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 shutdown_connections 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faee8105ce0 0x7faee81a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee8104210 0x7faee819a810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faed40afb70 0x7faed40b2080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:04.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 --2- 172.21.15.138:0/3456030201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faee8105340 0x7faee819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:04.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 >> 172.21.15.138:0/3456030201 conn(0x7faee80ff890 msgr2=0x7faee8108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:04.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 shutdown_connections 2024-04-25T13:35:04.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:04.333+0000 7faeefbb4700 1 -- 172.21.15.138:0/3456030201 wait complete. 2024-04-25T13:35:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:04 smithi137 ceph-mon[32568]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:04.650 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:04 smithi138 ceph-mon[32977]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:04 smithi094 ceph-mon[29195]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:04.950 DEBUG:teuthology.orchestra.run.smithi138:osd.2> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T13:35:04.953 INFO:tasks.cephadm:Deploying osd.3 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-25T13:35:04.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:35:05.200 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:35:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:35:05.498 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T13:35:05.193+0000 7f0b2bfdf7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T13:35:05.577 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.577 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.578 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.578 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.578 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:05 smithi094 ceph-mon[29195]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:35:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:05.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:05 smithi137 ceph-mon[32568]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:35:07.153 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:35:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:35:07.154 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T13:35:07.154 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T13:35:07.155 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842649 s, 1.2 GB/s 2024-04-25T13:35:07.155 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T13:35:07.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:35:07.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-mon[32977]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:35:07.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:35:07.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:07.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-mon[32977]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:07.249 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T13:35:07.057+0000 7f0b21644700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:35:07.249 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T13:35:07.062+0000 7f0b1d3f9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:35:07.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:07 smithi137 ceph-mon[32568]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:35:07.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:07 smithi137 ceph-mon[32568]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:35:07.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:07 smithi137 ceph-mon[32568]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:35:07.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:07.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:07 smithi137 ceph-mon[32568]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:07.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:07 smithi094 ceph-mon[29195]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:35:07.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:07 smithi094 ceph-mon[29195]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:35:07.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:07 smithi094 ceph-mon[29195]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:35:07.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:07.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:07 smithi094 ceph-mon[29195]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:07.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-25T13:35:08.107 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:35:08.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:08 smithi094 ceph-mon[29195]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:35:08.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:08 smithi094 ceph-mon[29195]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:35:08.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:08.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:08.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:08 smithi137 ceph-mon[32568]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:35:08.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:08 smithi137 ceph-mon[32568]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:35:08.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:08.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:08 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:35:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:08 smithi138 ceph-mon[32977]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:35:08.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:08.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:09.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] boot 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi138 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.175 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] boot 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi138 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: osd.2 [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] boot 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:09.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 -- 172.21.15.94:0/3887301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878103660 msgr2=0x7f6878103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 --2- 172.21.15.94:0/3887301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878103660 0x7f6878103a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f687000b0a0 tx=0x7f687000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 -- 172.21.15.94:0/3887301424 shutdown_connections 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 --2- 172.21.15.94:0/3887301424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 0x7f687810b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 --2- 172.21.15.94:0/3887301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878103660 0x7f6878103a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 --2- 172.21.15.94:0/3887301424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878104040 0x7f6878104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 -- 172.21.15.94:0/3887301424 >> 172.21.15.94:0/3887301424 conn(0x7f68780fded0 msgr2=0x7f6878100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 -- 172.21.15.94:0/3887301424 shutdown_connections 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.707+0000 7f687d5c7700 1 -- 172.21.15.94:0/3887301424 wait complete. 2024-04-25T13:35:09.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 Processor -- start 2024-04-25T13:35:09.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 -- start start 2024-04-25T13:35:09.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878103660 0x7f68781996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:09.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 0x7f68781a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f687810eb50 con 0x7f6878104040 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f687810e9d0 con 0x7f6878104ba0 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f687d5c7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f687810ecd0 con 0x7f6878103660 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.708+0000 7f68767fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36434/0 (socket says 172.21.15.94:36434) 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 -- 172.21.15.94:0/895714436 learned_addr learned my addr 172.21.15.94:0/895714436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f6876ffd700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878103660 0x7f68781996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68777fe700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 0x7f68781a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 -- 172.21.15.94:0/895714436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878103660 msgr2=0x7f68781996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878103660 0x7f68781996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 -- 172.21.15.94:0/895714436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 msgr2=0x7f68781a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 0x7f68781a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 -- 172.21.15.94:0/895714436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68781a1230 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f68767fc700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f686000e9a0 tx=0x7f686000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686000ca10 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686000cbb0 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68781a1520 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860018680 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.709+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a1a60 con 0x7f6878104040 2024-04-25T13:35:09.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.710+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6878051190 con 0x7f6878104040 2024-04-25T13:35:09.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.712+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f6860010430 con 0x7f6878104040 2024-04-25T13:35:09.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.712+0000 7f686ffff700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 0x7f68640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:09.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.712+0000 7f6876ffd700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 0x7f68640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:09.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.712+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f68600ddc10 con 0x7f6878104040 2024-04-25T13:35:09.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.713+0000 7f6876ffd700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 0x7f68640b2000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f687000b070 tx=0x7f68700060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:09.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.716+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68600a7cb0 con 0x7f6878104040 2024-04-25T13:35:09.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:09.921+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6878102320 con 0x7f68640afaf0 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:10.425 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:10.425 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:35:10.425 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:10.425 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:10.425 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:10.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:10.493 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:10.493 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:35:10.493 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:35:10.493 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:10.494 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:10.494 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48924]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48924]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.547 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.548 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48937]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.548 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48937]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.747 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41720]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41720]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41720]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41723]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41723]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.748 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41723]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41726]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41726]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41726]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41726]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41729]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41729]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:10 smithi138 sudo[41729]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.873 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48975]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.873 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48975]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.873 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48975]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.873 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48975]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.874 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48978]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.874 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.874 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48978]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.874 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:10 smithi094 sudo[48978]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.893 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41326]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.893 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.893 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41326]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.894 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41326]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.894 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.894 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.894 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41329]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.894 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41329]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41332]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:35:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41332]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41332]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41335]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:35:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:35:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41335]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:35:10.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:10 smithi137 sudo[41335]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:35:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.477 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:11 smithi094 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:11 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:35:11.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:11 smithi137 ceph-mon[32568]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:35:12.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:12 smithi094 ceph-mon[29195]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:35:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:12 smithi138 ceph-mon[32977]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:35:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:12 smithi137 ceph-mon[32568]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:35:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:13.179+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f68600a3840 con 0x7f6878104040 2024-04-25T13:35:13.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:13 smithi094 ceph-mon[29195]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:13.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]: dispatch 2024-04-25T13:35:13.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]': finished 2024-04-25T13:35:13.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:13 smithi094 ceph-mon[29195]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:35:13.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:13.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:13 smithi138 ceph-mon[32977]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:13.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]: dispatch 2024-04-25T13:35:13.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]': finished 2024-04-25T13:35:13.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:13 smithi138 ceph-mon[32977]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:35:13.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:13.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:13 smithi137 ceph-mon[32568]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:13.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]: dispatch 2024-04-25T13:35:13.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4238054107' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "046f0d40-5505-42d2-961f-4cf5250b1eda"}]': finished 2024-04-25T13:35:13.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:13 smithi137 ceph-mon[32568]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:35:13.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:14.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:14 smithi094 ceph-mon[29195]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:35:14.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:14 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3148641342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:14.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:14 smithi138 ceph-mon[32977]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:35:14.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:14 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3148641342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:14 smithi137 ceph-mon[32568]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:35:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:14 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3148641342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:15.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:15 smithi094 ceph-mon[29195]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:15.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:15 smithi138 ceph-mon[32977]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:15 smithi137 ceph-mon[32568]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:16.615 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:16 smithi094 ceph-mon[29195]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:16.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:16 smithi137 ceph-mon[32568]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:16 smithi138 ceph-mon[32977]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:18.552 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:18 smithi094 ceph-mon[29195]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:18.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:18 smithi137 ceph-mon[32568]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:18.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:18 smithi138 ceph-mon[32977]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:20.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:35:20.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:35:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:35:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:20 smithi094 ceph-mon[29195]: Deploying daemon osd.3 on smithi094 2024-04-25T13:35:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:20 smithi094 ceph-mon[29195]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:20 smithi137 ceph-mon[32568]: Deploying daemon osd.3 on smithi094 2024-04-25T13:35:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:20 smithi137 ceph-mon[32568]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:21.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:20 smithi138 ceph-mon[32977]: Deploying daemon osd.3 on smithi094 2024-04-25T13:35:21.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:20 smithi138 ceph-mon[32977]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:22 smithi094 ceph-mon[29195]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:22 smithi137 ceph-mon[32568]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:22 smithi138 ceph-mon[32977]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:24.640 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:24 smithi094 ceph-mon[29195]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:24.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:24 smithi137 ceph-mon[32568]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:24.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:24 smithi138 ceph-mon[32977]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:26 smithi094 ceph-mon[29195]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:26.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:26.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:26 smithi137 ceph-mon[32568]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:26 smithi138 ceph-mon[32977]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:29.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.227+0000 7f686ffff700 1 -- 172.21.15.94:0/895714436 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6878102320 con 0x7f68640afaf0 2024-04-25T13:35:29.229 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 3 on host 'smithi094' 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 msgr2=0x7f68640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 0x7f68640b2000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f687000b070 tx=0x7f68700060c0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 msgr2=0x7f6878199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f686000e9a0 tx=0x7f686000ecb0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 shutdown_connections 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6878104ba0 0x7f68781a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878103660 0x7f68781996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68640afaf0 0x7f68640b2000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 --2- 172.21.15.94:0/895714436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6878104040 0x7f6878199c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:29.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 >> 172.21.15.94:0/895714436 conn(0x7f68780fded0 msgr2=0x7f6878107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:29.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 shutdown_connections 2024-04-25T13:35:29.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:35:29.230+0000 7f687d5c7700 1 -- 172.21.15.94:0/895714436 wait complete. 2024-04-25T13:35:29.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:29.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:29.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:29.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:29.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:29.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:29.830 DEBUG:teuthology.orchestra.run.smithi094:osd.3> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T13:35:29.833 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-04-25T13:35:29.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:35:30.076 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:35:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:30 smithi094 ceph-mon[29195]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:35:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:30 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:35:30.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:30 smithi137 ceph-mon[32568]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:35:31.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:31 smithi137 ceph-mon[32568]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:31.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:31 smithi137 ceph-mon[32568]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:35:31.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:31 smithi137 ceph-mon[32568]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:35:31.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:31 smithi137 ceph-mon[32568]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:35:31.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-mon[29195]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-mon[29195]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:35:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-mon[29195]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:35:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-mon[29195]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:35:31.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:31 smithi138 ceph-mon[32977]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:31 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:35:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:31 smithi138 ceph-mon[32977]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:35:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:31 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:35:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112406 s, 933 MB/s 2024-04-25T13:35:32.018 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T13:35:32.252 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T13:35:31.977+0000 7f31e9f88700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:35:32.252 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 13:35:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T13:35:31.984+0000 7f31e1534700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' 2024-04-25T13:35:32.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:32 smithi137 ceph-mon[32568]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:32.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-04-25T13:35:32.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' 2024-04-25T13:35:32.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:32 smithi094 ceph-mon[29195]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:35:32.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:32.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692]' entity='osd.3' 2024-04-25T13:35:32.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:32 smithi138 ceph-mon[32977]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:35:32.925 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:35:33.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:35:33.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:35:33.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] boot 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi094 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:33.368 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:35:33.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] boot 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi094 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:33.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:33.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:35:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:35:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: osd.3 [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] boot 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi094 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:33.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:34.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 -- 172.21.15.137:0/2276701190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c4103e30 msgr2=0x7fb0c4104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:34.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/2276701190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c4103e30 0x7fb0c4104240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009990 tx=0x7fb0b8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 -- 172.21.15.137:0/2276701190 shutdown_connections 2024-04-25T13:35:34.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/2276701190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c41048b0 0x7fb0c410c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/2276701190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 0x7fb0c4105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/2276701190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c4103e30 0x7fb0c4104240 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 -- 172.21.15.137:0/2276701190 >> 172.21.15.137:0/2276701190 conn(0x7fb0c40ff8a0 msgr2=0x7fb0c4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 -- 172.21.15.137:0/2276701190 shutdown_connections 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 -- 172.21.15.137:0/2276701190 wait complete. 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.454+0000 7fb0c97f2700 1 Processor -- start 2024-04-25T13:35:34.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 -- start start 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 0x7fb0c419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 0x7fb0c41a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410fdb0 con 0x7fb0c4103e30 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410fc30 con 0x7fb0c41048b0 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c97f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410ff30 con 0x7fb0c4105880 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:34.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44404/0 (socket says 172.21.15.137:44404) 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 -- 172.21.15.137:0/4128598103 learned_addr learned my addr 172.21.15.137:0/4128598103 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c2ffd700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 0x7fb0c419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 -- 172.21.15.137:0/4128598103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 msgr2=0x7fb0c41a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c37fe700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 0x7fb0c41a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 0x7fb0c41a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 -- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 msgr2=0x7fb0c419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 0x7fb0c419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c27fc700 1 -- 172.21.15.137:0/4128598103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41a23d0 con 0x7fb0c41048b0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.455+0000 7fb0c2ffd700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 0x7fb0c419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0c27fc700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80092e0 tx=0x7fb0b8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8004830 con 0x7fb0c41048b0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41a2660 con 0x7fb0c41048b0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8003c00 con 0x7fb0c41048b0 2024-04-25T13:35:34.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8005220 con 0x7fb0c41048b0 2024-04-25T13:35:34.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.456+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c41a2b30 con 0x7fb0c41048b0 2024-04-25T13:35:34.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.457+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c4051190 con 0x7fb0c41048b0 2024-04-25T13:35:34.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.459+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb0b8029060 con 0x7fb0c41048b0 2024-04-25T13:35:34.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.460+0000 7fb0abfff700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 0x7fb0ac0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:34.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.460+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fb0b80de910 con 0x7fb0c41048b0 2024-04-25T13:35:34.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.460+0000 7fb0c2ffd700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 0x7fb0ac0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:34.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.461+0000 7fb0c2ffd700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 0x7fb0ac0b2090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005950 tx=0x7fb0b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:34.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.464+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0b8013070 con 0x7fb0c41048b0 2024-04-25T13:35:34.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:34.675+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb0c402e8c0 con 0x7fb0ac0afb80 2024-04-25T13:35:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:34 smithi137 ceph-mon[32568]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:35:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:34 smithi137 ceph-mon[32568]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:34.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:34 smithi094 ceph-mon[29195]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:35:34.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:34 smithi094 ceph-mon[29195]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:34 smithi138 ceph-mon[32977]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:35:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:34 smithi138 ceph-mon[32977]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:35 smithi137 ceph-mon[32568]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:35:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:35 smithi137 ceph-mon[32568]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:35.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:35 smithi094 ceph-mon[29195]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:35:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:35 smithi094 ceph-mon[29195]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:35.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:35.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:35.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:35 smithi138 ceph-mon[32977]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:35:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:35 smithi138 ceph-mon[32977]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:35:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:35:35.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:35:35.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:36.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:36 smithi137 ceph-mon[32568]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:36.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:36 smithi094 ceph-mon[29195]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:36.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:36 smithi138 ceph-mon[32977]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:37 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/476860487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:37 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:37 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]': finished 2024-04-25T13:35:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:37 smithi137 ceph-mon[32568]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:35:37.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:37 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/476860487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:37 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:37 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]': finished 2024-04-25T13:35:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:37 smithi094 ceph-mon[29195]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:35:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:37 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/476860487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:37 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]: dispatch 2024-04-25T13:35:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:37 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d740f16-7559-45f6-bd33-f834183b06f3"}]': finished 2024-04-25T13:35:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:37 smithi138 ceph-mon[32977]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:35:37.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:38.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:38 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/4188552542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:38.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:38 smithi137 ceph-mon[32568]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:38.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:38 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/4188552542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:38.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:38 smithi094 ceph-mon[29195]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:38 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/4188552542' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:35:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:38 smithi138 ceph-mon[32977]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:40 smithi137 ceph-mon[32568]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:35:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:40 smithi094 ceph-mon[29195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:35:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:40 smithi138 ceph-mon[32977]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:35:42.629 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:42 smithi137 ceph-mon[32568]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:35:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:42 smithi094 ceph-mon[29195]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:35:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:42 smithi138 ceph-mon[32977]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:35:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:44 smithi137 ceph-mon[32568]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:35:44.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:44 smithi094 ceph-mon[29195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:35:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:44 smithi138 ceph-mon[32977]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:35:45.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:35:45.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:45.558 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:45 smithi137 ceph-mon[32568]: Deploying daemon osd.4 on smithi137 2024-04-25T13:35:45.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:35:45.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:45.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:45 smithi094 ceph-mon[29195]: Deploying daemon osd.4 on smithi137 2024-04-25T13:35:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:35:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:45 smithi138 ceph-mon[32977]: Deploying daemon osd.4 on smithi137 2024-04-25T13:35:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:46 smithi137 ceph-mon[32568]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:35:46.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:46 smithi094 ceph-mon[29195]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:35:46.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:46 smithi138 ceph-mon[32977]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:35:48.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:48 smithi137 ceph-mon[32568]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-25T13:35:48.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:48 smithi094 ceph-mon[29195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-25T13:35:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:48 smithi138 ceph-mon[32977]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-25T13:35:50.549 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:50 smithi138 ceph-mon[32977]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:35:50.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:50 smithi137 ceph-mon[32568]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:35:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:50 smithi094 ceph-mon[29195]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:35:51.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:51.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:52 smithi094 ceph-mon[29195]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:52.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:52 smithi137 ceph-mon[32568]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:52.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:52 smithi138 ceph-mon[32977]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:54.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.489+0000 7fb0abfff700 1 -- 172.21.15.137:0/4128598103 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb0c402e8c0 con 0x7fb0ac0afb80 2024-04-25T13:35:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.491+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 msgr2=0x7fb0ac0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.491+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 0x7fb0ac0b2090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005950 tx=0x7fb0b40058e0 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 msgr2=0x7fb0c419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80092e0 tx=0x7fb0b8009f90 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 shutdown_connections 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4105880 0x7fb0c41a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb0ac0afb80 0x7fb0ac0b2090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0c4103e30 0x7fb0c419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 --2- 172.21.15.137:0/4128598103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0c41048b0 0x7fb0c419ae90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 >> 172.21.15.137:0/4128598103 conn(0x7fb0c40ff8a0 msgr2=0x7fb0c41085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 shutdown_connections 2024-04-25T13:35:54.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:35:54.492+0000 7fb0c97f2700 1 -- 172.21.15.137:0/4128598103 wait complete. 2024-04-25T13:35:54.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:54.714 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:54.714 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.714 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:54 smithi137 ceph-mon[32568]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:54 smithi094 ceph-mon[29195]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:54 smithi138 ceph-mon[32977]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:55.151 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T13:35:55.154 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_3... 2024-04-25T13:35:55.154 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:35:55.400 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:35:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:55 smithi137 ceph-mon[32568]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:55 smithi137 ceph-mon[32568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:55 smithi094 ceph-mon[29195]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:55 smithi094 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:55 smithi138 ceph-mon[32977]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:55 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:35:56.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:35:56.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:35:56.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:56.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:56 smithi137 ceph-mon[32568]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.734 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:56 smithi138 ceph-mon[32977]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:35:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:35:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: from='osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:35:56.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:56 smithi094 ceph-mon[29195]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122832 s, 854 MB/s 2024-04-25T13:35:57.273 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T13:35:57.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:57 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:35:57.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:57 smithi138 ceph-mon[32977]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:35:57.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.567 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:57 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' 2024-04-25T13:35:57.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-mon[32568]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:35:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-mon[32568]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:35:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-mon[32568]: from='osd.4 ' entity='osd.4' 2024-04-25T13:35:57.779 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T13:35:57.397+0000 7fcbc0181700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:35:57.779 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 13:35:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T13:35:57.521+0000 7fcbb7f2e700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:35:57.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-04-25T13:35:57.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:57 smithi094 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:35:57.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:57 smithi094 ceph-mon[29195]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T13:35:57.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:57.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:57 smithi094 ceph-mon[29195]: from='osd.4 ' entity='osd.4' 2024-04-25T13:35:58.197 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi137 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T13:35:58.574 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.575 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:58.575 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:58.575 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.575 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:58 smithi138 ceph-mon[32977]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:58.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi137 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:58 smithi137 ceph-mon[32568]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi137 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:35:58.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:58 smithi094 ceph-mon[29195]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:35:59.554 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:59 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:59 smithi138 ceph-mon[32977]: osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] boot 2024-04-25T13:35:59.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:59 smithi138 ceph-mon[32977]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:35:59.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:35:59 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.767+0000 7fcd342a5700 1 -- 172.21.15.138:0/580189093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c104ce0 msgr2=0x7fcd2c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.767+0000 7fcd342a5700 1 --2- 172.21.15.138:0/580189093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c107100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009990 tx=0x7fcd28009ca0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.767+0000 7fcd342a5700 1 -- 172.21.15.138:0/580189093 shutdown_connections 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.767+0000 7fcd342a5700 1 --2- 172.21.15.138:0/580189093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 0x7fcd2c10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.767+0000 7fcd342a5700 1 --2- 172.21.15.138:0/580189093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c107100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 --2- 172.21.15.138:0/580189093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c107640 0x7fcd2c109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- 172.21.15.138:0/580189093 >> 172.21.15.138:0/580189093 conn(0x7fcd2c0fe740 msgr2=0x7fcd2c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- 172.21.15.138:0/580189093 shutdown_connections 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- 172.21.15.138:0/580189093 wait complete. 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 Processor -- start 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- start start 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 0x7fcd2c1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10fc20 con 0x7fcd2c109fa0 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10faa0 con 0x7fcd2c104ce0 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.768+0000 7fcd342a5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10fda0 con 0x7fcd2c107640 2024-04-25T13:35:59.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58364/0 (socket says 172.21.15.138:58364) 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 -- 172.21.15.138:0/3854097023 learned_addr learned my addr 172.21.15.138:0/3854097023 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd32041700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd32842700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 0x7fcd2c1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 -- 172.21.15.138:0/3854097023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c104ce0 msgr2=0x7fcd2c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 -- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 msgr2=0x7fcd2c1a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 0x7fcd2c1a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 -- 172.21.15.138:0/3854097023 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c1a22f0 con 0x7fcd2c107640 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd31840700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d950 tx=0x7fcd1c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c016070 con 0x7fcd2c107640 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd1c010460 con 0x7fcd2c107640 2024-04-25T13:35:59.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1a25e0 con 0x7fcd2c107640 2024-04-25T13:35:59.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00f620 con 0x7fcd2c107640 2024-04-25T13:35:59.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.769+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1a2b20 con 0x7fcd2c107640 2024-04-25T13:35:59.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.770+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c051190 con 0x7fcd2c107640 2024-04-25T13:35:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.772+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcd1c019070 con 0x7fcd2c107640 2024-04-25T13:35:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.772+0000 7fcd237fe700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 0x7fcd180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:35:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.772+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fcd1c0dd7d0 con 0x7fcd2c107640 2024-04-25T13:35:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.772+0000 7fcd32041700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 0x7fcd180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:35:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.773+0000 7fcd32041700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 0x7fcd180b2090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd280038a0 tx=0x7fcd2801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:35:59.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.776+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd1c0a7420 con 0x7fcd2c107640 2024-04-25T13:35:59.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:59 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:59 smithi137 ceph-mon[32568]: osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] boot 2024-04-25T13:35:59.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:59 smithi137 ceph-mon[32568]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:35:59.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:35:59 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:59 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:59 smithi094 ceph-mon[29195]: osd.4 [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] boot 2024-04-25T13:35:59.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:59 smithi094 ceph-mon[29195]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:35:59.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:35:59 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:35:59.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:35:59.984+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcd2c063ad0 con 0x7fcd180afb80 2024-04-25T13:36:00.782 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:36:00.782 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:00.783 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:00.783 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:00.783 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:00.783 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:00 smithi138 ceph-mon[32977]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:36:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:00 smithi137 ceph-mon[32568]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T13:36:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:00.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:00.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:00 smithi094 ceph-mon[29195]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:02 smithi137 ceph-mon[32568]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:02 smithi094 ceph-mon[29195]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:02 smithi138 ceph-mon[32977]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:03 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3177595962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:03 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:03 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]': finished 2024-04-25T13:36:03.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:03 smithi138 ceph-mon[32977]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:36:03.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:03 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:03.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:03 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/3177595962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:03 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:03 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]': finished 2024-04-25T13:36:03.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:03 smithi137 ceph-mon[32568]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:36:03.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:03 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:03.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:03 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/3177595962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:03 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]: dispatch 2024-04-25T13:36:03.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:03 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eeda2336-9862-4bd4-b776-454204034ebb"}]': finished 2024-04-25T13:36:03.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:03 smithi094 ceph-mon[29195]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:36:03.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:04 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/2361771822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:04 smithi137 ceph-mon[32568]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:04 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/2361771822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:04 smithi094 ceph-mon[29195]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:04 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/2361771822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:04 smithi138 ceph-mon[32977]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:06 smithi137 ceph-mon[32568]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:06.653 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:06 smithi138 ceph-mon[32977]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:06 smithi094 ceph-mon[29195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:08 smithi138 ceph-mon[32977]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:08 smithi137 ceph-mon[32568]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:08 smithi094 ceph-mon[29195]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:10 smithi138 ceph-mon[32977]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:10.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:36:10.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:10.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:10 smithi138 ceph-mon[32977]: Deploying daemon osd.5 on smithi138 2024-04-25T13:36:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:10 smithi137 ceph-mon[32568]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:36:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:10 smithi137 ceph-mon[32568]: Deploying daemon osd.5 on smithi138 2024-04-25T13:36:10.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:10 smithi094 ceph-mon[29195]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:10.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:36:10.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:10.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:10 smithi094 ceph-mon[29195]: Deploying daemon osd.5 on smithi138 2024-04-25T13:36:12.384 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:12 smithi138 ceph-mon[32977]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:12 smithi137 ceph-mon[32568]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:12.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:12 smithi094 ceph-mon[29195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:14 smithi137 ceph-mon[32568]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:14.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:14 smithi094 ceph-mon[29195]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:14 smithi138 ceph-mon[32977]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:16 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:16 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:16 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:16 smithi138 ceph-mon[32977]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:16.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:16 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:16 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:16 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:16.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:16 smithi137 ceph-mon[32568]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:16 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:16 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:16 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:16 smithi094 ceph-mon[29195]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:18.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:18.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:18.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:18.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:18.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:18.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:18 smithi138 ceph-mon[32977]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:18 smithi094 ceph-mon[29195]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:19.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:18 smithi137 ceph-mon[32568]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:19.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.402+0000 7fcd237fe700 1 -- 172.21.15.138:0/3854097023 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd2c063ad0 con 0x7fcd180afb80 2024-04-25T13:36:19.403 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 msgr2=0x7fcd180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 0x7fcd180b2090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd280038a0 tx=0x7fcd2801a040 comp rx=0 tx=0).stop 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 msgr2=0x7fcd2c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d950 tx=0x7fcd1c00dd10 comp rx=0 tx=0).stop 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 shutdown_connections 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd180afb80 0x7fcd180b2090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd2c109fa0 0x7fcd2c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2c104ce0 0x7fcd2c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 --2- 172.21.15.138:0/3854097023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2c107640 0x7fcd2c19acd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:19.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 >> 172.21.15.138:0/3854097023 conn(0x7fcd2c0fe740 msgr2=0x7fcd2c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:19.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 shutdown_connections 2024-04-25T13:36:19.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:36:19.404+0000 7fcd342a5700 1 -- 172.21.15.138:0/3854097023 wait complete. 2024-04-25T13:36:20.049 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T13:36:20.052 INFO:tasks.cephadm:Deploying osd.6 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-25T13:36:20.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:36:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:20 smithi138 ceph-mon[32977]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:20 smithi094 ceph-mon[29195]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:36:20.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:20 smithi137 ceph-mon[32568]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:21.442 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:21 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:36:21.443 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:21 smithi138 ceph-mon[32977]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:36:21.443 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:21.443 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:21 smithi138 ceph-mon[32977]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:21.443 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:21 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:21.673 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:21 smithi094 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:36:21.673 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:21 smithi094 ceph-mon[29195]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:36:21.673 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:21.673 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:21 smithi094 ceph-mon[29195]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:21.673 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:21 smithi094 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:21.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:21 smithi137 ceph-mon[32568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:36:21.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:21 smithi137 ceph-mon[32568]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:36:21.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:21.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:21 smithi137 ceph-mon[32568]: from='osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:21.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:21 smithi137 ceph-mon[32568]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847715 s, 1.2 GB/s 2024-04-25T13:36:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T13:36:22.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:36:22.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:36:22.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:36:22.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:22 smithi094 ceph-mon[29195]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-mon[32977]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:22.748 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T13:36:22.404+0000 7f5d93bfd700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:36:22.748 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 13:36:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T13:36:22.454+0000 7f5d8b9aa700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:22.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:22 smithi137 ceph-mon[32568]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:36:22.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-25T13:36:23.239 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='osd.5 ' entity='osd.5' 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] boot 2024-04-25T13:36:23.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi138 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:23.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:36:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:36:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' 2024-04-25T13:36:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] boot 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:23.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:36:23.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='osd.5 ' entity='osd.5' 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: osd.5 [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] boot 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi138 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:23.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:24.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:24 smithi094 ceph-mon[29195]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:36:24.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:24 smithi094 ceph-mon[29195]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:24.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:24 smithi137 ceph-mon[32568]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:36:24.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:24 smithi137 ceph-mon[32568]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:24.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- 172.21.15.94:0/1847852347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3341048b0 msgr2=0x7fc33410c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:24.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1847852347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3341048b0 0x7fc33410c5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00b0a0 tx=0x7fc32c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- 172.21.15.94:0/1847852347 shutdown_connections 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1847852347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3341048b0 0x7fc33410c5c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1847852347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc334105880 0x7fc334105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1847852347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334103e30 0x7fc334104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- 172.21.15.94:0/1847852347 >> 172.21.15.94:0/1847852347 conn(0x7fc3340ff8a0 msgr2=0x7fc334101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- 172.21.15.94:0/1847852347 shutdown_connections 2024-04-25T13:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- 172.21.15.94:0/1847852347 wait complete. 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 Processor -- start 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.927+0000 7fc33a21d700 1 -- start start 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3341048b0 0x7fc334076af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334105880 0x7fc334077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410fde0 con 0x7fc334103e30 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410fc60 con 0x7fc334105880 2024-04-25T13:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc33a21d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410ff60 con 0x7fc3341048b0 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49610/0 (socket says 172.21.15.94:49610) 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 -- 172.21.15.94:0/1004442786 learned_addr learned my addr 172.21.15.94:0/1004442786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc333fff700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334105880 0x7fc334077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc332ffd700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3341048b0 0x7fc334076af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 -- 172.21.15.94:0/1004442786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3341048b0 msgr2=0x7fc334076af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3341048b0 0x7fc334076af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 -- 172.21.15.94:0/1004442786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334105880 msgr2=0x7fc334077030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334105880 0x7fc334077030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 -- 172.21.15.94:0/1004442786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc334077790 con 0x7fc334103e30 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc3337fe700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c8c0 tx=0x7fc32800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.928+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328004230 con 0x7fc334103e30 2024-04-25T13:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.929+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3280043d0 con 0x7fc334103e30 2024-04-25T13:36:24.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.929+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341a4960 con 0x7fc334103e30 2024-04-25T13:36:24.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.929+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328007710 con 0x7fc334103e30 2024-04-25T13:36:24.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.929+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341a4df0 con 0x7fc334103e30 2024-04-25T13:36:24.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.930+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334051190 con 0x7fc334103e30 2024-04-25T13:36:24.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.932+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc328004750 con 0x7fc334103e30 2024-04-25T13:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.933+0000 7fc330ff9700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 0x7fc31c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.933+0000 7fc332ffd700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 0x7fc31c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.933+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7fc3280ddd20 con 0x7fc334103e30 2024-04-25T13:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.933+0000 7fc332ffd700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 0x7fc31c0b2090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc324005950 tx=0x7fc324005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:24.937+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc328014070 con 0x7fc334103e30 2024-04-25T13:36:24.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:24 smithi138 ceph-mon[32977]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T13:36:24.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:24 smithi138 ceph-mon[32977]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:25.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:25.145+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc33402e8e0 con 0x7fc31c0afb80 2024-04-25T13:36:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:25 smithi137 ceph-mon[32568]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:36:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:25 smithi137 ceph-mon[32568]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:25.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:25.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:25.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:25 smithi094 ceph-mon[29195]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:36:25.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:25 smithi094 ceph-mon[29195]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:25.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:25.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:25.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:25.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:25 smithi138 ceph-mon[32977]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:36:25.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:25 smithi138 ceph-mon[32977]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:25.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:25.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:25.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:26 smithi094 ceph-mon[29195]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:26 smithi137 ceph-mon[32568]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:26.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:26 smithi138 ceph-mon[32977]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:27 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]: dispatch 2024-04-25T13:36:28.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:27 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]': finished 2024-04-25T13:36:28.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:27 smithi094 ceph-mon[29195]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:36:28.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]: dispatch 2024-04-25T13:36:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]': finished 2024-04-25T13:36:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:27 smithi138 ceph-mon[32977]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:36:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:28.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:27 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]: dispatch 2024-04-25T13:36:28.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:27 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/621728779' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a884764f-45af-4c83-b7d3-65ebd314f012"}]': finished 2024-04-25T13:36:28.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:27 smithi137 ceph-mon[32568]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:36:28.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:29.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:28 smithi094 ceph-mon[29195]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:29.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:28 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3165251127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:29.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:28 smithi138 ceph-mon[32977]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:29.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:28 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3165251127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:29.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:28 smithi137 ceph-mon[32568]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:29.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:28 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3165251127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:30.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:30 smithi137 ceph-mon[32568]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:36:30.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:30 smithi094 ceph-mon[29195]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:36:30.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:30 smithi138 ceph-mon[32977]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:36:32.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:32 smithi094 ceph-mon[29195]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:36:32.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:32 smithi137 ceph-mon[32568]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:36:32.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:32 smithi138 ceph-mon[32977]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:36:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:34 smithi137 ceph-mon[32568]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:36:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:34 smithi094 ceph-mon[29195]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:36:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:34 smithi138 ceph-mon[32977]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:36:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:36:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:36:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:36:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:36 smithi137 ceph-mon[32568]: Deploying daemon osd.6 on smithi094 2024-04-25T13:36:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:36 smithi137 ceph-mon[32568]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:36:36.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:36 smithi094 ceph-mon[29195]: Deploying daemon osd.6 on smithi094 2024-04-25T13:36:36.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:36 smithi094 ceph-mon[29195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:36:36.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:36 smithi138 ceph-mon[32977]: Deploying daemon osd.6 on smithi094 2024-04-25T13:36:36.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:36 smithi138 ceph-mon[32977]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:36:38.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:38 smithi094 ceph-mon[29195]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T13:36:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:38 smithi137 ceph-mon[32568]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T13:36:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:38 smithi138 ceph-mon[32977]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T13:36:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:40 smithi137 ceph-mon[32568]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:36:40.669 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:40 smithi094 ceph-mon[29195]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:36:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:40 smithi138 ceph-mon[32977]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:36:42.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:43.320 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:43 smithi094 ceph-mon[29195]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:43 smithi138 ceph-mon[32977]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:43 smithi137 ceph-mon[32568]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:44.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:44 smithi094 ceph-mon[29195]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:44 smithi137 ceph-mon[32568]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:44 smithi138 ceph-mon[32977]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:44.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.919+0000 7fc330ff9700 1 -- 172.21.15.94:0/1004442786 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc33402e8e0 con 0x7fc31c0afb80 2024-04-25T13:36:44.920 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 6 on host 'smithi094' 2024-04-25T13:36:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 msgr2=0x7fc31c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:44.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 0x7fc31c0b2090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc324005950 tx=0x7fc324005ed0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 msgr2=0x7fc334078440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c8c0 tx=0x7fc32800cbd0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 shutdown_connections 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc334105880 0x7fc334077030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc31c0afb80 0x7fc31c0b2090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc334103e30 0x7fc334078440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 --2- 172.21.15.94:0/1004442786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3341048b0 0x7fc334076af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 >> 172.21.15.94:0/1004442786 conn(0x7fc3340ff8a0 msgr2=0x7fc3341085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 shutdown_connections 2024-04-25T13:36:44.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:36:44.922+0000 7fc33a21d700 1 -- 172.21.15.94:0/1004442786 wait complete. 2024-04-25T13:36:45.586 DEBUG:teuthology.orchestra.run.smithi094:osd.6> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T13:36:45.589 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-04-25T13:36:45.589 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:36:45.835 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:46.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:45 smithi137 ceph-mon[32568]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:36:46.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:45 smithi094 ceph-mon[29195]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:36:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:45 smithi138 ceph-mon[32977]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:36:46.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:46.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:47.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:46 smithi137 ceph-mon[32568]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:36:47.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:46 smithi137 ceph-mon[32568]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T13:36:47.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:46 smithi137 ceph-mon[32568]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:36:47.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:46 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:47.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:46 smithi137 ceph-mon[32568]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:47.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:46 smithi094 ceph-mon[29195]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:36:47.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:46 smithi094 ceph-mon[29195]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T13:36:47.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:46 smithi094 ceph-mon[29195]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:36:47.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:46 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:47.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:46 smithi094 ceph-mon[29195]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:47.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:46 smithi138 ceph-mon[32977]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:36:47.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:46 smithi138 ceph-mon[32977]: osdmap e42: 7 total, 6 up, 7 in 2024-04-25T13:36:47.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:46 smithi138 ceph-mon[32977]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:36:47.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:46 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:47.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:46 smithi138 ceph-mon[32977]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:47.684 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T13:36:47.561+0000 7f910c0a4700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:36:47.684 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T13:36:47.571+0000 7f9103e51700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:36:47.843 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:36:47.843 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:36:47.844 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T13:36:47.844 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T13:36:47.844 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112469 s, 932 MB/s 2024-04-25T13:36:47.844 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T13:36:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-mon[29195]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:36:47.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-mon[29195]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T13:36:47.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:47.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:47.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:47 smithi094 ceph-mon[29195]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' 2024-04-25T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:47 smithi137 ceph-mon[32568]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:47 smithi137 ceph-mon[32568]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:48.137 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:47 smithi137 ceph-mon[32568]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' 2024-04-25T13:36:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:47 smithi138 ceph-mon[32977]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:36:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:47 smithi138 ceph-mon[32977]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T13:36:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:47 smithi138 ceph-mon[32977]: from='osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162]' entity='osd.6' 2024-04-25T13:36:48.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-04-25T13:36:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] boot 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T13:36:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] boot 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T13:36:49.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: osd.6 [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] boot 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: osdmap e44: 7 total, 7 up, 7 in 2024-04-25T13:36:49.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 -- 172.21.15.137:0/2349174661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c076540 msgr2=0x7f855c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 --2- 172.21.15.137:0/2349174661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c076540 0x7f855c106d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 -- 172.21.15.137:0/2349174661 shutdown_connections 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 --2- 172.21.15.137:0/2349174661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c109e90 0x7f855c10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 --2- 172.21.15.137:0/2349174661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c076540 0x7f855c106d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 --2- 172.21.15.137:0/2349174661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 -- 172.21.15.137:0/2349174661 >> 172.21.15.137:0/2349174661 conn(0x7f855c0fe740 msgr2=0x7f855c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 -- 172.21.15.137:0/2349174661 shutdown_connections 2024-04-25T13:36:50.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.271+0000 7f85610e6700 1 -- 172.21.15.137:0/2349174661 wait complete. 2024-04-25T13:36:50.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 Processor -- start 2024-04-25T13:36:50.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 -- start start 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c076540 0x7f855c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f855c10fc20 con 0x7f855c107360 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f855c10faa0 con 0x7f855c109e90 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f85610e6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855c10fda0 con 0x7f855c076540 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f855b59e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f855b59e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43750/0 (socket says 172.21.15.137:43750) 2024-04-25T13:36:50.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f855b59e700 1 -- 172.21.15.137:0/179703632 learned_addr learned my addr 172.21.15.137:0/179703632 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f855ad9d700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c076540 0x7f855c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.272+0000 7f855a59c700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 -- 172.21.15.137:0/179703632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c076540 msgr2=0x7f855c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c076540 0x7f855c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 -- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 msgr2=0x7f855c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 -- 172.21.15.137:0/179703632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1a2230 con 0x7f855c109e90 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855a59c700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f855b59e700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8554007ee0 tx=0x7f855400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855400dbd0 con 0x7f855c109e90 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c1a2520 con 0x7f855c109e90 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855400dd70 con 0x7f855c109e90 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554005990 con 0x7f855c109e90 2024-04-25T13:36:50.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.273+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c1a2a60 con 0x7f855c109e90 2024-04-25T13:36:50.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.274+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855c051190 con 0x7f855c109e90 2024-04-25T13:36:50.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.275+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f855401a020 con 0x7f855c109e90 2024-04-25T13:36:50.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.276+0000 7f8543fff700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 0x7f85440b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:50.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.276+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f85540dd7b0 con 0x7f855c109e90 2024-04-25T13:36:50.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.276+0000 7f855ad9d700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 0x7f85440b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:50.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.277+0000 7f855ad9d700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 0x7f85440b2090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f854c0038a0 tx=0x7f854c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:50.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.281+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8554010070 con 0x7f855c109e90 2024-04-25T13:36:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi094 2024-04-25T13:36:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:50 smithi137 ceph-mon[32568]: osdmap e45: 7 total, 7 up, 7 in 2024-04-25T13:36:50.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:36:50.488+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f855c063ad0 con 0x7f85440afb80 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi094 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:50 smithi138 ceph-mon[32977]: osdmap e45: 7 total, 7 up, 7 in 2024-04-25T13:36:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi094 2024-04-25T13:36:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:50.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:50.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:50.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:50.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:36:50.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:50 smithi094 ceph-mon[29195]: osdmap e45: 7 total, 7 up, 7 in 2024-04-25T13:36:51.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:51 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T13:36:51.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:51 smithi137 ceph-mon[32568]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:51.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:51.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:51.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:51.688 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:51 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T13:36:51.688 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:51 smithi138 ceph-mon[32977]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:51.688 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:51.689 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:51.689 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:51.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:51 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T13:36:51.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:51 smithi094 ceph-mon[29195]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:51.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:36:51.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:36:51.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:52.344 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:52 smithi137 ceph-mon[32568]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:52 smithi094 ceph-mon[29195]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:52 smithi138 ceph-mon[32977]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/2774252863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]': finished 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:36:53.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:53 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/2774252863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]': finished 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:36:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:53 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:36:53.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:53.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/2774252863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]: dispatch 2024-04-25T13:36:53.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73b89099-7f29-4114-b29f-c0b33124e04e"}]': finished 2024-04-25T13:36:53.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:36:53.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:53 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:36:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:54 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/3646566478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:54 smithi137 ceph-mon[32568]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:54 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/3646566478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:54.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:54 smithi094 ceph-mon[29195]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:54 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/3646566478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:36:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:54 smithi138 ceph-mon[32977]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:56 smithi137 ceph-mon[32568]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:56 smithi094 ceph-mon[29195]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:56.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:56 smithi138 ceph-mon[32977]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:58.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:36:58 smithi137 ceph-mon[32568]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:58.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:36:58 smithi094 ceph-mon[29195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:36:58.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:36:58 smithi138 ceph-mon[32977]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:00.592 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:00 smithi137 ceph-mon[32568]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:00 smithi094 ceph-mon[29195]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:00.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:00 smithi138 ceph-mon[32977]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:01 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:37:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:01 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:01 smithi138 ceph-mon[32977]: Deploying daemon osd.7 on smithi137 2024-04-25T13:37:01.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:01 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:37:01.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:01 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:01.544 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:01 smithi137 ceph-mon[32568]: Deploying daemon osd.7 on smithi137 2024-04-25T13:37:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:01 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:37:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:01 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:01 smithi094 ceph-mon[29195]: Deploying daemon osd.7 on smithi137 2024-04-25T13:37:02.592 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:02 smithi137 ceph-mon[32568]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:02 smithi094 ceph-mon[29195]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:02 smithi138 ceph-mon[32977]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:04 smithi137 ceph-mon[32568]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:04 smithi094 ceph-mon[29195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:04 smithi138 ceph-mon[32977]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:07.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:07.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:07 smithi137 ceph-mon[32568]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:07 smithi094 ceph-mon[29195]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:07 smithi138 ceph-mon[32977]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:08 smithi137 ceph-mon[32568]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:08 smithi094 ceph-mon[29195]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:08 smithi138 ceph-mon[32977]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:09.956 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-04-25T13:37:09.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.955+0000 7f8543fff700 1 -- 172.21.15.137:0/179703632 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f855c063ad0 con 0x7f85440afb80 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 msgr2=0x7f85440b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 0x7f85440b2090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f854c0038a0 tx=0x7f854c01a040 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 msgr2=0x7f855c1a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8554007ee0 tx=0x7f855400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 shutdown_connections 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f855c109e90 0x7f855c1a1c40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c076540 0x7f855c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f85440afb80 0x7f85440b2090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 --2- 172.21.15.137:0/179703632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f855c107360 0x7f855c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:09.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 >> 172.21.15.137:0/179703632 conn(0x7f855c0fe740 msgr2=0x7f855c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:37:09.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 shutdown_connections 2024-04-25T13:37:09.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:37:09.957+0000 7f85610e6700 1 -- 172.21.15.137:0/179703632 wait complete. 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:10 smithi137 ceph-mon[32568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:10.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:10 smithi094 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:10 smithi138 ceph-mon[32977]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:37:10.618 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:37:10.620 INFO:tasks.cephadm:Deploying osd.8 on smithi138 with /dev/vg_nvme/lv_2... 2024-04-25T13:37:10.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:37:10.867 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:37:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:37:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T13:37:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:11.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:11 smithi138 ceph-mon[32977]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-mon[32568]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: from='osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:11.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:11 smithi094 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:37:12.105 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T13:37:11.959+0000 7feadfb3e700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:37:12.105 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 13:37:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T13:37:11.969+0000 7feadb8f3700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125118 s, 838 MB/s 2024-04-25T13:37:12.787 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T13:37:13.100 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:12 smithi138 ceph-mon[32977]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:37:13.100 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:12 smithi138 ceph-mon[32977]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T13:37:13.100 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.100 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.100 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:12 smithi138 ceph-mon[32977]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:13.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:12 smithi137 ceph-mon[32568]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:37:13.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:12 smithi137 ceph-mon[32568]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T13:37:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:12 smithi137 ceph-mon[32568]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:13.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:12 smithi094 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:37:13.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:12 smithi094 ceph-mon[29195]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T13:37:13.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:13.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:12 smithi094 ceph-mon[29195]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:37:13.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-04-25T13:37:13.595 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:37:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:37:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] boot 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:37:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:37:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] boot 2024-04-25T13:37:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:37:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: osd.7 [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] boot 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:15.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 -- 172.21.15.138:0/1434046680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4103120 msgr2=0x7fafe4103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:15.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 --2- 172.21.15.138:0/1434046680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4103120 0x7fafe4103530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009990 tx=0x7fafd8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.186 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:14 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi137 2024-04-25T13:37:15.186 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:14 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T13:37:15.186 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:14 smithi138 ceph-mon[32977]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T13:37:15.186 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:14 smithi138 ceph-mon[32977]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:15.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 -- 172.21.15.138:0/1434046680 shutdown_connections 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 --2- 172.21.15.138:0/1434046680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4104bf0 0x7fafe410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 --2- 172.21.15.138:0/1434046680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4103120 0x7fafe4103530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 --2- 172.21.15.138:0/1434046680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104250 0x7fafe4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 -- 172.21.15.138:0/1434046680 >> 172.21.15.138:0/1434046680 conn(0x7fafe40fe760 msgr2=0x7fafe4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 -- 172.21.15.138:0/1434046680 shutdown_connections 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 -- 172.21.15.138:0/1434046680 wait complete. 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.185+0000 7fafeb213700 1 Processor -- start 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 -- start start 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4103120 0x7fafe4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104bf0 0x7fafe41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410ebf0 con 0x7fafe4104bf0 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410ea70 con 0x7fafe4103120 2024-04-25T13:37:15.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafeb213700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafe410ed70 con 0x7fafe4104250 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39596/0 (socket says 172.21.15.138:39596) 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 -- 172.21.15.138:0/2422500059 learned_addr learned my addr 172.21.15.138:0/2422500059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe97b0700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104bf0 0x7fafe41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe8faf700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4103120 0x7fafe4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 -- 172.21.15.138:0/2422500059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4103120 msgr2=0x7fafe4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4103120 0x7fafe4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 -- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104bf0 msgr2=0x7fafe41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104bf0 0x7fafe41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.186+0000 7fafe3fff700 1 -- 172.21.15.138:0/2422500059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe41a1230 con 0x7fafe4104250 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafe3fff700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c9b0 tx=0x7fafd000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0007b00 con 0x7fafe4104250 2024-04-25T13:37:15.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd000ce50 con 0x7fafe4104250 2024-04-25T13:37:15.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018660 con 0x7fafe4104250 2024-04-25T13:37:15.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe4075920 con 0x7fafe4104250 2024-04-25T13:37:15.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.187+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe4075e90 con 0x7fafe4104250 2024-04-25T13:37:15.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.188+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe4051190 con 0x7fafe4104250 2024-04-25T13:37:15.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.190+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fafd000f450 con 0x7fafe4104250 2024-04-25T13:37:15.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.191+0000 7fafe1ffb700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 0x7fafd40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:15.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.191+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7fafd00de270 con 0x7fafe4104250 2024-04-25T13:37:15.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.191+0000 7fafe8faf700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 0x7fafd40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:15.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.192+0000 7fafe8faf700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 0x7fafd40b2090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafd80038a0 tx=0x7fafd801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:37:15.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.195+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafd0014030 con 0x7fafe4104250 2024-04-25T13:37:15.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:14 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi137 2024-04-25T13:37:15.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:14 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T13:37:15.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:14 smithi094 ceph-mon[29195]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T13:37:15.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:14 smithi094 ceph-mon[29195]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:14 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi137 2024-04-25T13:37:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:14 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T13:37:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:14 smithi137 ceph-mon[32568]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T13:37:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:14 smithi137 ceph-mon[32568]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:15.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:15.401+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fafe4076150 con 0x7fafd40afb80 2024-04-25T13:37:16.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:15 smithi138 ceph-mon[32977]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T13:37:16.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:16.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:16.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:15 smithi137 ceph-mon[32568]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T13:37:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:15 smithi094 ceph-mon[29195]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T13:37:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:16.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:17.178 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:16 smithi138 ceph-mon[32977]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:17.179 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:16 smithi138 ceph-mon[32977]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:16 smithi137 ceph-mon[32568]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:16 smithi137 ceph-mon[32568]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:17.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:16 smithi094 ceph-mon[29195]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:17.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:16 smithi094 ceph-mon[29195]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:18 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/306225926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:18 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:18 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]': finished 2024-04-25T13:37:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:18 smithi137 ceph-mon[32568]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:37:18.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:18.451 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:18 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/306225926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.451 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:18 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.451 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:18 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]': finished 2024-04-25T13:37:18.451 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:18 smithi138 ceph-mon[32977]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:37:18.451 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:18.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:18 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/306225926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:18 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]: dispatch 2024-04-25T13:37:18.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:18 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e5d4bf-c849-4b0c-bb6b-f4c14675072a"}]': finished 2024-04-25T13:37:18.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:18 smithi094 ceph-mon[29195]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:37:18.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:19.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:19 smithi094 ceph-mon[29195]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:19.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:19 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/2728583650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:19.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:19 smithi138 ceph-mon[32977]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:19.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:19 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/2728583650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:19.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:19 smithi137 ceph-mon[32568]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:19.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:19 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/2728583650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:20 smithi137 ceph-mon[32568]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 184 KiB/s, 0 objects/s recovering 2024-04-25T13:37:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:20 smithi094 ceph-mon[29195]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 184 KiB/s, 0 objects/s recovering 2024-04-25T13:37:20.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:20 smithi138 ceph-mon[32977]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 184 KiB/s, 0 objects/s recovering 2024-04-25T13:37:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:22 smithi137 ceph-mon[32568]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:37:22.668 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:22 smithi138 ceph-mon[32977]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:37:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:22 smithi094 ceph-mon[29195]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:37:24.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:24 smithi138 ceph-mon[32977]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:37:24.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:24 smithi137 ceph-mon[32568]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:37:24.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:24 smithi094 ceph-mon[29195]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T13:37:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:37:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:37:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:25.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:37:25.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:26.837 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:26 smithi138 ceph-mon[32977]: Deploying daemon osd.8 on smithi138 2024-04-25T13:37:26.837 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:26 smithi138 ceph-mon[32977]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:37:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:26 smithi137 ceph-mon[32568]: Deploying daemon osd.8 on smithi138 2024-04-25T13:37:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:26 smithi137 ceph-mon[32568]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:37:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:26 smithi094 ceph-mon[29195]: Deploying daemon osd.8 on smithi138 2024-04-25T13:37:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:26 smithi094 ceph-mon[29195]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:37:28.352 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:28 smithi138 ceph-mon[32977]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T13:37:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:28 smithi137 ceph-mon[32568]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T13:37:28.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:28 smithi094 ceph-mon[29195]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T13:37:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:30 smithi138 ceph-mon[32977]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:30.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:30 smithi137 ceph-mon[32568]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:30.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:30 smithi094 ceph-mon[29195]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:32.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:33.226 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:33 smithi138 ceph-mon[32977]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:33.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:33 smithi137 ceph-mon[32568]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:33.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:33 smithi094 ceph-mon[29195]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:34.480 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:34 smithi138 ceph-mon[32977]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:34.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.566+0000 7fafe1ffb700 1 -- 172.21.15.138:0/2422500059 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fafe4076150 con 0x7fafd40afb80 2024-04-25T13:37:34.567 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 8 on host 'smithi138' 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 msgr2=0x7fafd40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 0x7fafd40b2090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafd80038a0 tx=0x7fafd801a040 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 msgr2=0x7fafe4199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c9b0 tx=0x7fafd000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 shutdown_connections 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafd40afb80 0x7fafd40b2090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4104bf0 0x7fafe41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe4103120 0x7fafe4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 --2- 172.21.15.138:0/2422500059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe4104250 0x7fafe4199ca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 >> 172.21.15.138:0/2422500059 conn(0x7fafe40fe760 msgr2=0x7fafe4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 shutdown_connections 2024-04-25T13:37:34.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:37:34.569+0000 7fafeb213700 1 -- 172.21.15.138:0/2422500059 wait complete. 2024-04-25T13:37:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:34 smithi137 ceph-mon[32568]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:34 smithi094 ceph-mon[29195]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:35.127 DEBUG:teuthology.orchestra.run.smithi138:osd.8> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:37:35.129 INFO:tasks.cephadm:Deploying osd.9 on smithi094 with /dev/vg_nvme/lv_1... 2024-04-25T13:37:35.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:37:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.823 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.847 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.847 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.847 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.682 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:36 smithi138 ceph-mon[32977]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:36 smithi094 ceph-mon[29195]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:37:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T13:37:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:36.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: from='osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:37:36.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:36 smithi137 ceph-mon[32568]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00843418 s, 1.2 GB/s 2024-04-25T13:37:37.396 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T13:37:37.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: osdmap e54: 9 total, 8 up, 9 in 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: osdmap e55: 9 total, 8 up, 9 in 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-mon[32977]: from='osd.8 ' entity='osd.8' 2024-04-25T13:37:37.748 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T13:37:37.428+0000 7fea5d44f700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:37:37.748 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 13:37:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T13:37:37.534+0000 7fea551fc700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: osdmap e54: 9 total, 8 up, 9 in 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: osdmap e55: 9 total, 8 up, 9 in 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:37 smithi094 ceph-mon[29195]: from='osd.8 ' entity='osd.8' 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: osdmap e54: 9 total, 8 up, 9 in 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: osdmap e55: 9 total, 8 up, 9 in 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:37 smithi137 ceph-mon[32568]: from='osd.8 ' entity='osd.8' 2024-04-25T13:37:38.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-04-25T13:37:38.364 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:37:38.737 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] boot 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:38 smithi094 ceph-mon[29195]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:38.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:37:38.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:37:38.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] boot 2024-04-25T13:37:38.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T13:37:38.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:38 smithi137 ceph-mon[32568]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: osd.8 [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] boot 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:37:38.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:38 smithi138 ceph-mon[32977]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:37:39.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi138 2024-04-25T13:37:39.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: osdmap e57: 9 total, 9 up, 9 in 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi138 2024-04-25T13:37:39.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: osdmap e57: 9 total, 9 up, 9 in 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.951+0000 7f7013554700 1 -- 172.21.15.94:0/1922138131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 msgr2=0x7f700c105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.951+0000 7f7013554700 1 --2- 172.21.15.94:0/1922138131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c105c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.951+0000 7f7013554700 1 -- 172.21.15.94:0/1922138131 shutdown_connections 2024-04-25T13:37:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.951+0000 7f7013554700 1 --2- 172.21.15.94:0/1922138131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c104880 0x7f700c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 --2- 172.21.15.94:0/1922138131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c105c60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 --2- 172.21.15.94:0/1922138131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c103e00 0x7f700c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- 172.21.15.94:0/1922138131 >> 172.21.15.94:0/1922138131 conn(0x7f700c0ff870 msgr2=0x7f700c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:37:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- 172.21.15.94:0/1922138131 shutdown_connections 2024-04-25T13:37:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- 172.21.15.94:0/1922138131 wait complete. 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 Processor -- start 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- start start 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 0x7f700c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c104880 0x7f700c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10fe10 con 0x7f700c105850 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10fc90 con 0x7f700c104880 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.952+0000 7f7013554700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10ff90 con 0x7f700c103e00 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44460/0 (socket says 172.21.15.94:44460) 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 -- 172.21.15.94:0/427648778 learned_addr learned my addr 172.21.15.94:0/427648778 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7010aef700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c104880 0x7f700c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f70112f0700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 0x7f700c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 -- 172.21.15.94:0/427648778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 msgr2=0x7f700c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 0x7f700c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 -- 172.21.15.94:0/427648778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c104880 msgr2=0x7f700c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c104880 0x7f700c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 -- 172.21.15.94:0/427648778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c1a2480 con 0x7f700c105850 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f70112f0700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 0x7f700c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7011af1700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f700400b970 tx=0x7f700400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:37:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004019070 con 0x7f700c105850 2024-04-25T13:37:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f700400cde0 con 0x7f700c105850 2024-04-25T13:37:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c1a2770 con 0x7f700c105850 2024-04-25T13:37:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004007370 con 0x7f700c105850 2024-04-25T13:37:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.953+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c1a2d30 con 0x7f700c105850 2024-04-25T13:37:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.954+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c051190 con 0x7f700c105850 2024-04-25T13:37:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.956+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f700401c070 con 0x7f700c105850 2024-04-25T13:37:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.956+0000 7f6ffe7fc700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 0x7f6ff80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:37:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.957+0000 7f70112f0700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 0x7f6ff80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:37:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.957+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5829+0+0 (secure 0 0 0) 0x7f70040dde40 con 0x7f700c105850 2024-04-25T13:37:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.957+0000 7f70112f0700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 0x7f6ff80b2110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70000038a0 tx=0x7f700001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:37:39.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:39.960+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7004023070 con 0x7f700c105850 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: osdmap e57: 9 total, 9 up, 9 in 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:37:39.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:40.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:37:40.168+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f700c02e880 con 0x7f6ff80afc00 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: osdmap e58: 9 total, 9 up, 9 in 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:40 smithi094 ceph-mon[29195]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: osdmap e58: 9 total, 9 up, 9 in 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:40.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:40 smithi137 ceph-mon[32568]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: osdmap e58: 9 total, 9 up, 9 in 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:40.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:40 smithi138 ceph-mon[32977]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:42.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:42 smithi094 ceph-mon[29195]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:42 smithi137 ceph-mon[32568]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:42 smithi138 ceph-mon[32977]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]: dispatch 2024-04-25T13:37:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]': finished 2024-04-25T13:37:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:43 smithi137 ceph-mon[32568]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T13:37:43.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:37:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]: dispatch 2024-04-25T13:37:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]': finished 2024-04-25T13:37:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:43 smithi094 ceph-mon[29195]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T13:37:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:37:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]: dispatch 2024-04-25T13:37:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1568740764' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4cec5f6-fc9d-4166-9979-a7dad7ce3585"}]': finished 2024-04-25T13:37:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:43 smithi138 ceph-mon[32977]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T13:37:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:37:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/140987743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:44 smithi137 ceph-mon[32568]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:44.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/140987743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:44.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:44 smithi094 ceph-mon[29195]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/140987743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:37:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:44 smithi138 ceph-mon[32977]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:37:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:46 smithi137 ceph-mon[32568]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:37:46.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:46 smithi094 ceph-mon[29195]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:37:46.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:46 smithi138 ceph-mon[32977]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:37:48.408 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:48 smithi094 ceph-mon[29195]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:37:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:48 smithi137 ceph-mon[32568]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:37:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:48 smithi138 ceph-mon[32977]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:37:50.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:50 smithi094 ceph-mon[29195]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:50.564 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:50 smithi138 ceph-mon[32977]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:50.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:50 smithi137 ceph-mon[32568]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:37:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:51 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:51.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:51 smithi094 ceph-mon[29195]: Deploying daemon osd.9 on smithi094 2024-04-25T13:37:51.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:37:51.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:51 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:51.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:51 smithi137 ceph-mon[32568]: Deploying daemon osd.9 on smithi094 2024-04-25T13:37:51.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:37:51.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:51 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:37:51.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:51 smithi138 ceph-mon[32977]: Deploying daemon osd.9 on smithi094 2024-04-25T13:37:52.444 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:52 smithi094 ceph-mon[29195]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:52 smithi137 ceph-mon[32568]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:52 smithi138 ceph-mon[32977]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:37:54.604 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:54 smithi094 ceph-mon[29195]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:37:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:54 smithi137 ceph-mon[32568]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:37:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:54 smithi138 ceph-mon[32977]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-25T13:37:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:56 smithi137 ceph-mon[32568]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:56 smithi094 ceph-mon[29195]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:56.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:56 smithi138 ceph-mon[32977]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:57.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:57.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:57.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:57.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:57.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:57.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:37:57.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:37:58.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:37:58 smithi137 ceph-mon[32568]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:37:58 smithi094 ceph-mon[29195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:37:58.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:37:58 smithi138 ceph-mon[32977]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:00.442 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 9 on host 'smithi094' 2024-04-25T13:38:00.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.441+0000 7f6ffe7fc700 1 -- 172.21.15.94:0/427648778 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f700c02e880 con 0x7f6ff80afc00 2024-04-25T13:38:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 msgr2=0x7f6ff80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 0x7f6ff80b2110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70000038a0 tx=0x7f700001a040 comp rx=0 tx=0).stop 2024-04-25T13:38:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 msgr2=0x7f700c1a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f700400b970 tx=0x7f700400bc80 comp rx=0 tx=0).stop 2024-04-25T13:38:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 shutdown_connections 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ff80afc00 0x7f6ff80b2110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f700c105850 0x7f700c1a1e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c103e00 0x7f700c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 --2- 172.21.15.94:0/427648778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c104880 0x7f700c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 >> 172.21.15.94:0/427648778 conn(0x7f700c0ff870 msgr2=0x7f700c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 shutdown_connections 2024-04-25T13:38:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:00.443+0000 7f7013554700 1 -- 172.21.15.94:0/427648778 wait complete. 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:00 smithi094 ceph-mon[29195]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:38:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:00.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:00.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:00 smithi137 ceph-mon[32568]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:00.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:00 smithi138 ceph-mon[32977]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:38:01.056 DEBUG:teuthology.orchestra.run.smithi094:osd.9> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:38:01.058 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-04-25T13:38:01.058 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:38:01.304 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:38:02.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:02 smithi094 ceph-mon[29195]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:38:02.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:02 smithi094 ceph-mon[29195]: osdmap e60: 10 total, 9 up, 10 in 2024-04-25T13:38:02.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:02 smithi094 ceph-mon[29195]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:38:02.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:02.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:02 smithi094 ceph-mon[29195]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:02 smithi138 ceph-mon[32977]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:38:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:02 smithi138 ceph-mon[32977]: osdmap e60: 10 total, 9 up, 10 in 2024-04-25T13:38:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:02 smithi138 ceph-mon[32977]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:38:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:02 smithi138 ceph-mon[32977]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:02.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:02 smithi137 ceph-mon[32568]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:38:02.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:02 smithi137 ceph-mon[32568]: osdmap e60: 10 total, 9 up, 10 in 2024-04-25T13:38:02.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:02 smithi137 ceph-mon[32568]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T13:38:02.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:02.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:02 smithi137 ceph-mon[32568]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130426 s, 804 MB/s 2024-04-25T13:38:03.320 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T13:38:03.407 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T13:38:03.116+0000 7fafbb980700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:38:03.408 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T13:38:03.129+0000 7fafb372d700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:38:03.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:03 smithi137 ceph-mon[32568]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:38:03.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:03 smithi137 ceph-mon[32568]: osdmap e61: 10 total, 9 up, 10 in 2024-04-25T13:38:03.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:03 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:03 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:03 smithi137 ceph-mon[32568]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' 2024-04-25T13:38:03.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-mon[29195]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:38:03.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-mon[29195]: osdmap e61: 10 total, 9 up, 10 in 2024-04-25T13:38:03.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:03 smithi094 ceph-mon[29195]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' 2024-04-25T13:38:03.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:03 smithi138 ceph-mon[32977]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T13:38:03.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:03 smithi138 ceph-mon[32977]: osdmap e61: 10 total, 9 up, 10 in 2024-04-25T13:38:03.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:03 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:03 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:03.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:03 smithi138 ceph-mon[32977]: from='osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868]' entity='osd.9' 2024-04-25T13:38:03.966 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-04-25T13:38:04.215 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:38:04.625 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:38:04.625 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] boot 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: osdmap e62: 10 total, 10 up, 10 in 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi094 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T13:38:04.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:04.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:04.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.627 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:04 smithi137 ceph-mon[32568]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] boot 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: osdmap e62: 10 total, 10 up, 10 in 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi094 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:04 smithi094 ceph-mon[29195]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: osd.9 [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] boot 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: osdmap e62: 10 total, 10 up, 10 in 2024-04-25T13:38:04.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi094 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:04.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:04 smithi138 ceph-mon[32977]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:38:05.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 -- 172.21.15.137:0/145001106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105340 msgr2=0x7f05f8105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:05.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/145001106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105340 0x7f05f8105750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009990 tx=0x7f05e0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 -- 172.21.15.137:0/145001106 shutdown_connections 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/145001106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105ce0 0x7f05f810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/145001106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/145001106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105340 0x7f05f8105750 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 -- 172.21.15.137:0/145001106 >> 172.21.15.137:0/145001106 conn(0x7f05f80ff870 msgr2=0x7f05f8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 -- 172.21.15.137:0/145001106 shutdown_connections 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.885+0000 7f05fdaa0700 1 -- 172.21.15.137:0/145001106 wait complete. 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 Processor -- start 2024-04-25T13:38:05.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 -- start start 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8078440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105340 0x7f05f8076af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810fcc0 con 0x7f05f8104210 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810fb40 con 0x7f05f8105ce0 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05fdaa0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f810fe40 con 0x7f05f8105340 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05f7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05f7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39078/0 (socket says 172.21.15.137:39078) 2024-04-25T13:38:05.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05f7fff700 1 -- 172.21.15.137:0/1850514465 learned_addr learned my addr 172.21.15.137:0/1850514465 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05eeffd700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105340 0x7f05f8076af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.886+0000 7f05f77fe700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8078440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 -- 172.21.15.137:0/1850514465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105340 msgr2=0x7f05f8076af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105340 0x7f05f8076af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 -- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 msgr2=0x7f05f8078440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8078440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 -- 172.21.15.137:0/1850514465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f8077790 con 0x7f05f8105ce0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f77fe700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8078440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f7fff700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007ee0 tx=0x7f05f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f000dbd0 con 0x7f05f8105ce0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f81a4880 con 0x7f05f8105ce0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f000dd70 con 0x7f05f8105ce0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0005990 con 0x7f05f8105ce0 2024-04-25T13:38:05.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.887+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81a4d10 con 0x7f05f8105ce0 2024-04-25T13:38:05.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.888+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f8051190 con 0x7f05f8105ce0 2024-04-25T13:38:05.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.889+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f05f001a020 con 0x7f05f8105ce0 2024-04-25T13:38:05.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.890+0000 7f05f57fa700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 0x7f05e40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:05.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.890+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7f05f00de300 con 0x7f05f8105ce0 2024-04-25T13:38:05.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.890+0000 7f05f77fe700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 0x7f05e40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:05.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.891+0000 7f05f77fe700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 0x7f05e40b2090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005950 tx=0x7f05e80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:05.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:05.894+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05f0010070 con 0x7f05f8105ce0 2024-04-25T13:38:06.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:06.099+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f05f8063ad0 con 0x7f05e40afb80 2024-04-25T13:38:06.709 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: osdmap e63: 10 total, 10 up, 10 in 2024-04-25T13:38:06.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:06.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:06.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:06.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:06.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:06 smithi137 ceph-mon[32568]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: osdmap e63: 10 total, 10 up, 10 in 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:06 smithi094 ceph-mon[29195]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: osdmap e63: 10 total, 10 up, 10 in 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:06 smithi138 ceph-mon[32977]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:08 smithi137 ceph-mon[32568]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:08 smithi094 ceph-mon[29195]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:08 smithi138 ceph-mon[32977]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:09 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/1580595482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:09 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:09 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]': finished 2024-04-25T13:38:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:09 smithi137 ceph-mon[32568]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T13:38:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:09 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/1580595482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:09 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:09 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]': finished 2024-04-25T13:38:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:09 smithi094 ceph-mon[29195]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T13:38:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/1580595482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:09 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]: dispatch 2024-04-25T13:38:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:09 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "110b4c5b-fa58-48a4-9567-c88bd908afa7"}]': finished 2024-04-25T13:38:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:09 smithi138 ceph-mon[32977]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T13:38:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:10.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:10 smithi094 ceph-mon[29195]: from='client.? 172.21.15.137:0/3322081972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:10.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:10 smithi094 ceph-mon[29195]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:10 smithi137 ceph-mon[32568]: from='client.? 172.21.15.137:0/3322081972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:10 smithi137 ceph-mon[32568]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:10 smithi138 ceph-mon[32977]: from='client.? 172.21.15.137:0/3322081972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:10 smithi138 ceph-mon[32977]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:12 smithi137 ceph-mon[32568]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:12.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:12 smithi094 ceph-mon[29195]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:12 smithi138 ceph-mon[32977]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:14.583 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:14 smithi137 ceph-mon[32568]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:14.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:14 smithi094 ceph-mon[29195]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:14 smithi138 ceph-mon[32977]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:16.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:16 smithi137 ceph-mon[32568]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:16.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:16 smithi094 ceph-mon[29195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:16 smithi138 ceph-mon[32977]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:17.470 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:38:17.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:17.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:17 smithi137 ceph-mon[32568]: Deploying daemon osd.10 on smithi137 2024-04-25T13:38:17.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:38:17.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:17.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:17 smithi094 ceph-mon[29195]: Deploying daemon osd.10 on smithi137 2024-04-25T13:38:17.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:38:17.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:17.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:17 smithi138 ceph-mon[32977]: Deploying daemon osd.10 on smithi137 2024-04-25T13:38:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:18 smithi137 ceph-mon[32568]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:18.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:18 smithi094 ceph-mon[29195]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:18.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:18 smithi138 ceph-mon[32977]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:20.569 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:20 smithi138 ceph-mon[32977]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:20 smithi137 ceph-mon[32568]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:20 smithi094 ceph-mon[29195]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:22 smithi137 ceph-mon[32568]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:22 smithi094 ceph-mon[29195]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:22 smithi138 ceph-mon[32977]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:24.369 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:24 smithi137 ceph-mon[32568]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:24.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:24 smithi094 ceph-mon[29195]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:24.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:24 smithi138 ceph-mon[32977]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.615+0000 7f05f57fa700 1 -- 172.21.15.137:0/1850514465 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f05f8063ad0 con 0x7f05e40afb80 2024-04-25T13:38:25.615 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 msgr2=0x7f05e40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 0x7f05e40b2090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005950 tx=0x7f05e80058e0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 msgr2=0x7f05f8077030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007ee0 tx=0x7f05f000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 shutdown_connections 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f8105ce0 0x7f05f8077030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05e40afb80 0x7f05e40b2090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f8104210 0x7f05f8078440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 --2- 172.21.15.137:0/1850514465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f8105340 0x7f05f8076af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.617+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 >> 172.21.15.137:0/1850514465 conn(0x7f05f80ff870 msgr2=0x7f05f8108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.618+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 shutdown_connections 2024-04-25T13:38:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T13:38:25.618+0000 7f05fdaa0700 1 -- 172.21.15.137:0/1850514465 wait complete. 2024-04-25T13:38:26.329 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:38:26.332 INFO:tasks.cephadm:Deploying osd.11 on smithi138 with /dev/vg_nvme/lv_1... 2024-04-25T13:38:26.332 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:38:26.588 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:38:26.703 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.703 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.703 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.703 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.704 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:26.704 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.704 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:26.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:27.855 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:27 smithi138 ceph-mon[32977]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:38:27.855 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:27 smithi138 ceph-mon[32977]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T13:38:27.855 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:27.855 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:27 smithi138 ceph-mon[32977]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:27.855 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:27 smithi138 ceph-mon[32977]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:27.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:27 smithi137 ceph-mon[32568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:38:27.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:27 smithi137 ceph-mon[32568]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T13:38:27.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:27.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:27 smithi137 ceph-mon[32568]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:27.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:27 smithi137 ceph-mon[32568]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:27 smithi094 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:38:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:27 smithi094 ceph-mon[29195]: osdmap e65: 11 total, 10 up, 11 in 2024-04-25T13:38:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:27 smithi094 ceph-mon[29195]: from='osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:27 smithi094 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T13:38:28.526 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:38:28.527 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:38:28.527 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T13:38:28.527 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T13:38:28.527 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119827 s, 875 MB/s 2024-04-25T13:38:28.527 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T13:38:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:38:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: osdmap e66: 11 total, 10 up, 11 in 2024-04-25T13:38:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:28.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-mon[32568]: from='osd.10 ' entity='osd.10' 2024-04-25T13:38:28.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T13:38:28.530+0000 7f5528d5f700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: osdmap e66: 11 total, 10 up, 11 in 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:28.901 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:28 smithi138 ceph-mon[32977]: from='osd.10 ' entity='osd.10' 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: osdmap e66: 11 total, 10 up, 11 in 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:28 smithi094 ceph-mon[29195]: from='osd.10 ' entity='osd.10' 2024-04-25T13:38:29.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 13:38:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T13:38:28.652+0000 7f5520b0c700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:38:29.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-04-25T13:38:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: osdmap e67: 11 total, 10 up, 11 in 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: osdmap e67: 11 total, 10 up, 11 in 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.866 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: osdmap e67: 11 total, 10 up, 11 in 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:29.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] boot 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi137 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T13:38:30.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:30 smithi137 ceph-mon[32568]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] boot 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi137 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.912 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.913 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:30 smithi138 ceph-mon[32977]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: osd.10 [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] boot 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: osdmap e68: 11 total, 11 up, 11 in 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi137 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:30 smithi094 ceph-mon[29195]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- 172.21.15.138:0/3467737085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9848103e30 msgr2=0x7f9848104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 --2- 172.21.15.138:0/3467737085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9848103e30 0x7f9848104240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9844009990 tx=0x7f9844009ca0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- 172.21.15.138:0/3467737085 shutdown_connections 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 --2- 172.21.15.138:0/3467737085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98481048b0 0x7f984810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 --2- 172.21.15.138:0/3467737085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 0x7f9848105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 --2- 172.21.15.138:0/3467737085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9848103e30 0x7f9848104240 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- 172.21.15.138:0/3467737085 >> 172.21.15.138:0/3467737085 conn(0x7f98480ff8a0 msgr2=0x7f9848101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- 172.21.15.138:0/3467737085 shutdown_connections 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- 172.21.15.138:0/3467737085 wait complete. 2024-04-25T13:38:31.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 Processor -- start 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.023+0000 7f984fbe0700 1 -- start start 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 0x7f984819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 0x7f98481a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f984810fdc0 con 0x7f9848103e30 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f984810fc40 con 0x7f9848105880 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f984810ff40 con 0x7f98481048b0 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34682/0 (socket says 172.21.15.138:34682) 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 -- 172.21.15.138:0/139608978 learned_addr learned my addr 172.21.15.138:0/139608978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984e17d700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 0x7f98481a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d97c700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 0x7f984819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 -- 172.21.15.138:0/139608978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 msgr2=0x7f98481a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:31.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 0x7f98481a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 -- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 msgr2=0x7f984819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 0x7f984819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 -- 172.21.15.138:0/139608978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98481a2430 con 0x7f98481048b0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d97c700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 0x7f984819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984d17b700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f98440057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844027070 con 0x7f98481048b0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481a26c0 con 0x7f98481048b0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98440040c0 con 0x7f98481048b0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.024+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844022da0 con 0x7f98481048b0 2024-04-25T13:38:31.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.025+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a2b70 con 0x7f98481048b0 2024-04-25T13:38:31.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.026+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848051190 con 0x7f98481048b0 2024-04-25T13:38:31.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.027+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f984402a070 con 0x7f98481048b0 2024-04-25T13:38:31.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.027+0000 7f983effd700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 0x7f98340b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:31.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.028+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6694+0+0 (secure 0 0 0) 0x7f98440df470 con 0x7f98481048b0 2024-04-25T13:38:31.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.028+0000 7f984d97c700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 0x7f98340b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:31.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.028+0000 7f984d97c700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 0x7f98340b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9838005d90 tx=0x7f9838005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:31.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.032+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9844013070 con 0x7f98481048b0 2024-04-25T13:38:31.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:31.237+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f984802e890 con 0x7f98340afb80 2024-04-25T13:38:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:31 smithi137 ceph-mon[32568]: osdmap e69: 11 total, 11 up, 11 in 2024-04-25T13:38:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:31 smithi137 ceph-mon[32568]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:31 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:31.944 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:31 smithi138 ceph-mon[32977]: osdmap e69: 11 total, 11 up, 11 in 2024-04-25T13:38:31.944 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:31 smithi138 ceph-mon[32977]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:31.944 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:31.944 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:31.944 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:31 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:31.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:31 smithi094 ceph-mon[29195]: osdmap e69: 11 total, 11 up, 11 in 2024-04-25T13:38:31.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:31 smithi094 ceph-mon[29195]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:38:31.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:38:31.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:38:31.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:31 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:32.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:32 smithi137 ceph-mon[32568]: osdmap e70: 11 total, 11 up, 11 in 2024-04-25T13:38:32.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:32 smithi137 ceph-mon[32568]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:32.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:32 smithi094 ceph-mon[29195]: osdmap e70: 11 total, 11 up, 11 in 2024-04-25T13:38:32.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:32 smithi094 ceph-mon[29195]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:32.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:32 smithi138 ceph-mon[32977]: osdmap e70: 11 total, 11 up, 11 in 2024-04-25T13:38:32.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:32 smithi138 ceph-mon[32977]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/3342346482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:34 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:34 smithi137 ceph-mon[32568]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]': finished 2024-04-25T13:38:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:34 smithi137 ceph-mon[32568]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T13:38:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:34.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3342346482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.459 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:34 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.459 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:34 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]': finished 2024-04-25T13:38:34.459 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:34 smithi138 ceph-mon[32977]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T13:38:34.459 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/3342346482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:34 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]: dispatch 2024-04-25T13:38:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:34 smithi094 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af033e4-3bf8-4a1d-a355-55edad2c8226"}]': finished 2024-04-25T13:38:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:34 smithi094 ceph-mon[29195]: osdmap e71: 12 total, 11 up, 12 in 2024-04-25T13:38:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:35.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:35 smithi094 ceph-mon[29195]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T13:38:35.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.138:0/3016431759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:35.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:35 smithi138 ceph-mon[32977]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T13:38:35.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3016431759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:35.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:35 smithi137 ceph-mon[32568]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T13:38:35.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.138:0/3016431759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:38:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:36 smithi137 ceph-mon[32568]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:38:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:36 smithi094 ceph-mon[29195]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:38:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:36 smithi138 ceph-mon[32977]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:38:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:38 smithi137 ceph-mon[32568]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:38:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:38 smithi094 ceph-mon[29195]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:38:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:38 smithi138 ceph-mon[32977]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T13:38:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:40 smithi137 ceph-mon[32568]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:38:40.692 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:40 smithi138 ceph-mon[32977]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:38:40.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:40 smithi094 ceph-mon[29195]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T13:38:42.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:38:42.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:42.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:38:42.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:38:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:43.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:42 smithi094 ceph-mon[29195]: Deploying daemon osd.11 on smithi138 2024-04-25T13:38:43.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:42 smithi094 ceph-mon[29195]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:38:43.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:42 smithi138 ceph-mon[32977]: Deploying daemon osd.11 on smithi138 2024-04-25T13:38:43.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:42 smithi138 ceph-mon[32977]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:38:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:42 smithi137 ceph-mon[32568]: Deploying daemon osd.11 on smithi138 2024-04-25T13:38:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:42 smithi137 ceph-mon[32568]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:38:44.563 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:44 smithi138 ceph-mon[32977]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:44 smithi137 ceph-mon[32568]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:44.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:44 smithi094 ceph-mon[29195]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:46 smithi137 ceph-mon[32568]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:46.705 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:46 smithi138 ceph-mon[32977]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:46.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:46 smithi094 ceph-mon[29195]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:48 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:48 smithi138 ceph-mon[32977]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:48 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:48 smithi094 ceph-mon[29195]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:49.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:49.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:49.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:48 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:49.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:48 smithi137 ceph-mon[32568]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:50.541 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:50 smithi138 ceph-mon[32977]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:50 smithi137 ceph-mon[32568]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:50 smithi094 ceph-mon[29195]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 11 on host 'smithi138' 2024-04-25T13:38:51.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.408+0000 7f983effd700 1 -- 172.21.15.138:0/139608978 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f984802e890 con 0x7f98340afb80 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 msgr2=0x7f98340b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 0x7f98340b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9838005d90 tx=0x7f9838005d00 comp rx=0 tx=0).stop 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 msgr2=0x7f984819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f98440057c0 comp rx=0 tx=0).stop 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 shutdown_connections 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9848105880 0x7f98481a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98340afb80 0x7f98340b2090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:51.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9848103e30 0x7f984819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:51.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 --2- 172.21.15.138:0/139608978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98481048b0 0x7f984819ae10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:51.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 >> 172.21.15.138:0/139608978 conn(0x7f98480ff8a0 msgr2=0x7f98481085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:51.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 shutdown_connections 2024-04-25T13:38:51.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:38:51.411+0000 7f984fbe0700 1 -- 172.21.15.138:0/139608978 wait complete. 2024-04-25T13:38:52.106 DEBUG:teuthology.orchestra.run.smithi138:osd.11> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:38:52.109 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:38:52.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:38:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:38:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:52 smithi094 ceph-mon[29195]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:52 smithi137 ceph-mon[32568]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:52.654 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:52 smithi138 ceph-mon[32977]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:53.535 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-mon[32977]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:38:53.535 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-mon[32977]: osdmap e72: 12 total, 11 up, 12 in 2024-04-25T13:38:53.536 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:53.536 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-mon[32977]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:53.536 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-mon[32977]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:53.536 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T13:38:53.372+0000 7fefb5993700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:38:53.536 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 13:38:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T13:38:53.387+0000 7fefb1748700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:38:53.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:53 smithi137 ceph-mon[32568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:38:53.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:53 smithi137 ceph-mon[32568]: osdmap e72: 12 total, 11 up, 12 in 2024-04-25T13:38:53.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:53 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:53.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:53 smithi137 ceph-mon[32568]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:53.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:53 smithi137 ceph-mon[32568]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:53.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:53 smithi094 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:38:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:53 smithi094 ceph-mon[29195]: osdmap e72: 12 total, 11 up, 12 in 2024-04-25T13:38:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:53 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:53 smithi094 ceph-mon[29195]: from='osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:53.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:53 smithi094 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T13:38:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- 172.21.15.94:0/1187120759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580104360 msgr2=0x7ff58010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 --2- 172.21.15.94:0/1187120759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580104360 0x7ff58010ac10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff57800b0a0 tx=0x7ff57800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- 172.21.15.94:0/1187120759 shutdown_connections 2024-04-25T13:38:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 --2- 172.21.15.94:0/1187120759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580104360 0x7ff58010ac10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 --2- 172.21.15.94:0/1187120759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff580102890 0x7ff580102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 --2- 172.21.15.94:0/1187120759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5801039c0 0x7ff580103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- 172.21.15.94:0/1187120759 >> 172.21.15.94:0/1187120759 conn(0x7ff5800fded0 msgr2=0x7ff580100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- 172.21.15.94:0/1187120759 shutdown_connections 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- 172.21.15.94:0/1187120759 wait complete. 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 Processor -- start 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.077+0000 7ff587d47700 1 -- start start 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5801039c0 0x7ff58019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580104360 0x7ff5801a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010fd50 con 0x7ff580102890 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010fbd0 con 0x7ff580104360 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010fed0 con 0x7ff5801039c0 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42258/0 (socket says 172.21.15.94:42258) 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 -- 172.21.15.94:0/2217393207 learned_addr learned my addr 172.21.15.94:0/2217393207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff5862e4700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580104360 0x7ff5801a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff5852e2700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5801039c0 0x7ff58019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 -- 172.21.15.94:0/2217393207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5801039c0 msgr2=0x7ff58019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5801039c0 0x7ff58019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 -- 172.21.15.94:0/2217393207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580104360 msgr2=0x7ff5801a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580104360 0x7ff5801a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 -- 172.21.15.94:0/2217393207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801a2330 con 0x7ff580102890 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff585ae3700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e940 tx=0x7ff57c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004220 con 0x7ff580102890 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a2620 con 0x7ff580102890 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c0043c0 con 0x7ff580102890 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.078+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003c60 con 0x7ff580102890 2024-04-25T13:38:54.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.079+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a2bb0 con 0x7ff580102890 2024-04-25T13:38:54.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.079+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff580051190 con 0x7ff580102890 2024-04-25T13:38:54.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.082+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff57c00c7e0 con 0x7ff580102890 2024-04-25T13:38:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.082+0000 7ff576ffd700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 0x7ff56c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.082+0000 7ff5852e2700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 0x7ff56c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.083+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7ff57c0de8c0 con 0x7ff580102890 2024-04-25T13:38:54.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.083+0000 7ff5852e2700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 0x7ff56c0b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff58019b880 tx=0x7ff5700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:54.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.086+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff57c0a7af0 con 0x7ff580102890 2024-04-25T13:38:54.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.285+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff58002e510 con 0x7ff580102890 2024-04-25T13:38:54.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.285+0000 7ff576ffd700 1 -- 172.21.15.94:0/2217393207 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7ff57c0a73f0 con 0x7ff580102890 2024-04-25T13:38:54.286 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:38:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.287+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 msgr2=0x7ff56c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.287+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 0x7ff56c0b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff58019b880 tx=0x7ff5700073d0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.287+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 msgr2=0x7ff58019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.287+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e940 tx=0x7ff57c00ec50 comp rx=0 tx=0).stop 2024-04-25T13:38:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 shutdown_connections 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff580104360 0x7ff5801a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff56c0afb80 0x7ff56c0b2090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580102890 0x7ff58019a7d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 --2- 172.21.15.94:0/2217393207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5801039c0 0x7ff58019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 >> 172.21.15.94:0/2217393207 conn(0x7ff5800fded0 msgr2=0x7ff580106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 shutdown_connections 2024-04-25T13:38:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:54.288+0000 7ff587d47700 1 -- 172.21.15.94:0/2217393207 wait complete. 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: osdmap e73: 12 total, 11 up, 12 in 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:54.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:54 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2217393207' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: osdmap e73: 12 total, 11 up, 12 in 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:54 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2217393207' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:54.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T13:38:54.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: osdmap e73: 12 total, 11 up, 12 in 2024-04-25T13:38:54.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:38:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:54 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2217393207' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:54.933 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1714052309,"num_in_osds":12,"osd_in_since":1714052314,"num_remapped_pgs":0} 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: purged_snaps scrub starts 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: purged_snaps scrub ok 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] boot 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: Detected new or changed devices on smithi138 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:55 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] boot 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: Detected new or changed devices on smithi138 2024-04-25T13:38:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:55 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: osd.11 [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] boot 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:38:55.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:55 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:38:55.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:38:56.172 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:38:56.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:56 smithi094 ceph-mon[29195]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:38:56.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:56 smithi094 ceph-mon[29195]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:38:56.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:56 smithi138 ceph-mon[32977]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:38:56.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:56 smithi138 ceph-mon[32977]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:38:56.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:56 smithi137 ceph-mon[32568]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:38:56.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:56 smithi137 ceph-mon[32568]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 -- 172.21.15.94:0/3858346340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c105cb0 msgr2=0x7f3a8c10c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 --2- 172.21.15.94:0/3858346340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c10c560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400b0a0 tx=0x7f3a8400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 -- 172.21.15.94:0/3858346340 shutdown_connections 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 --2- 172.21.15.94:0/3858346340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c10c560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 --2- 172.21.15.94:0/3858346340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 --2- 172.21.15.94:0/3858346340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 0x7f3a8c105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 -- 172.21.15.94:0/3858346340 >> 172.21.15.94:0/3858346340 conn(0x7f3a8c0ff840 msgr2=0x7f3a8c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 -- 172.21.15.94:0/3858346340 shutdown_connections 2024-04-25T13:38:57.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.801+0000 7f3a916b9700 1 -- 172.21.15.94:0/3858346340 wait complete. 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 Processor -- start 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 -- start start 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 0x7f3a8c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fc40 con 0x7f3a8c1041e0 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.802+0000 7f3a916b9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fac0 con 0x7f3a8c105310 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a916b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fdc0 con 0x7f3a8c105cb0 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42272/0 (socket says 172.21.15.94:42272) 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 -- 172.21.15.94:0/2285070088 learned_addr learned my addr 172.21.15.94:0/2285070088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8a7fc700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 0x7f3a8c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8b7fe700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:57.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 -- 172.21.15.94:0/2285070088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c105cb0 msgr2=0x7f3a8c1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 -- 172.21.15.94:0/2285070088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 msgr2=0x7f3a8c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 0x7f3a8c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 -- 172.21.15.94:0/2285070088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c1a2280 con 0x7f3a8c1041e0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a8affd700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c970 tx=0x7f3a8000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004220 con 0x7f3a8c1041e0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1a2570 con 0x7f3a8c1041e0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a80007db0 con 0x7f3a8c1041e0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80003d00 con 0x7f3a8c1041e0 2024-04-25T13:38:57.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.803+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1a2b00 con 0x7f3a8c1041e0 2024-04-25T13:38:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.804+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c051190 con 0x7f3a8c1041e0 2024-04-25T13:38:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.806+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3a80007900 con 0x7f3a8c1041e0 2024-04-25T13:38:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.806+0000 7f3a73fff700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 0x7f3a740b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:38:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.807+0000 7f3a8a7fc700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 0x7f3a740b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:38:57.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.807+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3a800de7b0 con 0x7f3a8c1041e0 2024-04-25T13:38:57.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.807+0000 7f3a8a7fc700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 0x7f3a740b2090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:38:57.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:57.811+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a80014030 con 0x7f3a8c1041e0 2024-04-25T13:38:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.010+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3a8c02e510 con 0x7f3a8c1041e0 2024-04-25T13:38:58.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.010+0000 7f3a73fff700 1 -- 172.21.15.94:0/2285070088 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f3a800a7830 con 0x7f3a8c1041e0 2024-04-25T13:38:58.011 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 msgr2=0x7f3a740b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 0x7f3a740b2090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c005f60 comp rx=0 tx=0).stop 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 msgr2=0x7f3a8c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c970 tx=0x7f3a8000cd30 comp rx=0 tx=0).stop 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 shutdown_connections 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c105cb0 0x7f3a8c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a740afb80 0x7f3a740b2090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a8c1041e0 0x7f3a8c19a720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 --2- 172.21.15.94:0/2285070088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a8c105310 0x7f3a8c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:38:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.012+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 >> 172.21.15.94:0/2285070088 conn(0x7f3a8c0ff840 msgr2=0x7f3a8c108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:38:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.013+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 shutdown_connections 2024-04-25T13:38:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:38:58.013+0000 7f3a916b9700 1 -- 172.21.15.94:0/2285070088 wait complete. 2024-04-25T13:38:58.354 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:58 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2285070088' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:58.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:58 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2285070088' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:58.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:58 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2285070088' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:38:58.672 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1714052334,"num_in_osds":12,"osd_in_since":1714052314,"num_remapped_pgs":0} 2024-04-25T13:38:58.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:38:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:38:59.291 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:38:59 smithi094 ceph-mon[29195]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:38:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:38:59 smithi137 ceph-mon[32568]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:38:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:38:59 smithi138 ceph-mon[32977]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 -- 172.21.15.94:0/163175847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528108dd0 msgr2=0x7fc5281091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 --2- 172.21.15.94:0/163175847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528108dd0 0x7fc5281091e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc5180099c0 tx=0x7fc518009cd0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 -- 172.21.15.94:0/163175847 shutdown_connections 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 --2- 172.21.15.94:0/163175847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc528109720 0x7fc528111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 --2- 172.21.15.94:0/163175847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5281044a0 0x7fc5281048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 --2- 172.21.15.94:0/163175847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528108dd0 0x7fc5281091e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 -- 172.21.15.94:0/163175847 >> 172.21.15.94:0/163175847 conn(0x7fc5280ffd70 msgr2=0x7fc5281021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:00.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 -- 172.21.15.94:0/163175847 shutdown_connections 2024-04-25T13:39:00.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:00 smithi094 ceph-mon[29195]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.451+0000 7fc52fb01700 1 -- 172.21.15.94:0/163175847 wait complete. 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.452+0000 7fc52fb01700 1 Processor -- start 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.452+0000 7fc52fb01700 1 -- start start 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5281044a0 0x7fc528076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc528108dd0 0x7fc528076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:00.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281143a0 con 0x7fc528109720 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc528114220 con 0x7fc5281044a0 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc528114520 con 0x7fc528108dd0 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42276/0 (socket says 172.21.15.94:42276) 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 -- 172.21.15.94:0/3631989978 learned_addr learned my addr 172.21.15.94:0/3631989978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52d89d700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5281044a0 0x7fc528076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52d09c700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc528108dd0 0x7fc528076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 -- 172.21.15.94:0/3631989978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc528108dd0 msgr2=0x7fc528076fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc528108dd0 0x7fc528076fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 -- 172.21.15.94:0/3631989978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5281044a0 msgr2=0x7fc528076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5281044a0 0x7fc528076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 -- 172.21.15.94:0/3631989978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281a9100 con 0x7fc528109720 2024-04-25T13:39:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52e09e700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc52000b970 tx=0x7fc52000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000d860 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281a93f0 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.453+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52000e430 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.454+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520017660 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.454+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a9900 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.454+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc528105950 con 0x7fc528109720 2024-04-25T13:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.457+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc52000da00 con 0x7fc528109720 2024-04-25T13:39:00.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.458+0000 7fc51effd700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 0x7fc5140b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:00.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.458+0000 7fc52d89d700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 0x7fc5140b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:00.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.458+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc5200de700 con 0x7fc528109720 2024-04-25T13:39:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.458+0000 7fc52d89d700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 0x7fc5140b2110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc524005950 tx=0x7fc52400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.461+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc520013030 con 0x7fc528109720 2024-04-25T13:39:00.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:00 smithi137 ceph-mon[32568]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.665+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc52802e510 con 0x7fc528109720 2024-04-25T13:39:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.666+0000 7fc51effd700 1 -- 172.21.15.94:0/3631989978 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18481 (secure 0 0 0) 0x7fc5200a7800 con 0x7fc528109720 2024-04-25T13:39:00.667 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:00.667 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":75,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:38:55.378345+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 msgr2=0x7fc5140b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 0x7fc5140b2110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc524005950 tx=0x7fc52400a300 comp rx=0 tx=0).stop 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 msgr2=0x7fc528077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc52000b970 tx=0x7fc52000bc80 comp rx=0 tx=0).stop 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 shutdown_connections 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc5140afc00 0x7fc5140b2110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc528109720 0x7fc528077520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5281044a0 0x7fc528076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 --2- 172.21.15.94:0/3631989978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc528108dd0 0x7fc528076fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 >> 172.21.15.94:0/3631989978 conn(0x7fc5280ffd70 msgr2=0x7fc52810a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.668+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 shutdown_connections 2024-04-25T13:39:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:00.669+0000 7fc52fb01700 1 -- 172.21.15.94:0/3631989978 wait complete. 2024-04-25T13:39:00.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:00 smithi138 ceph-mon[32977]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:01.316 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:39:01.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:39:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:39:01.593 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:01 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3631989978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:01 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3631989978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:01.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:01 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3631989978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:02 smithi137 ceph-mon[32568]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:02.674 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:02 smithi094 ceph-mon[29195]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:02 smithi138 ceph-mon[32977]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:03.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 -- 172.21.15.94:0/3322106573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f401085e0 msgr2=0x7f6f4010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:03.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 --2- 172.21.15.94:0/3322106573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f401085e0 0x7f6f4010ff40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b0d0 tx=0x7f6f3800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 -- 172.21.15.94:0/3322106573 shutdown_connections 2024-04-25T13:39:03.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 --2- 172.21.15.94:0/3322106573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f401085e0 0x7f6f4010ff40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 --2- 172.21.15.94:0/3322106573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40103360 0x7f6f40103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 --2- 172.21.15.94:0/3322106573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f40107c90 0x7f6f401080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 -- 172.21.15.94:0/3322106573 >> 172.21.15.94:0/3322106573 conn(0x7f6f400fec90 msgr2=0x7f6f401010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 -- 172.21.15.94:0/3322106573 shutdown_connections 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.239+0000 7f6f46613700 1 -- 172.21.15.94:0/3322106573 wait complete. 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 Processor -- start 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 -- start start 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 0x7f6f40110920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40107c90 0x7f6f40110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 0x7f6f401113a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40114360 con 0x7f6f40103360 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401141e0 con 0x7f6f40107c90 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.240+0000 7f6f46613700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401144e0 con 0x7f6f401085e0 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f44bb0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 0x7f6f401113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:03.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f44bb0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 0x7f6f401113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35760/0 (socket says 172.21.15.94:35760) 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f44bb0700 1 -- 172.21.15.94:0/265907524 learned_addr learned my addr 172.21.15.94:0/265907524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 0x7f6f40110920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3f7fe700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40107c90 0x7f6f40110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 -- 172.21.15.94:0/265907524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 msgr2=0x7f6f401113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 0x7f6f401113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 -- 172.21.15.94:0/265907524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40107c90 msgr2=0x7f6f40110e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40107c90 0x7f6f40110e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 -- 172.21.15.94:0/265907524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f401a8f40 con 0x7f6f40103360 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3ffff700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 0x7f6f40110920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c970 tx=0x7f6f3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30014070 con 0x7f6f40103360 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f401a9230 con 0x7f6f40103360 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401a97c0 con 0x7f6f40103360 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.241+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f30004440 con 0x7f6f40103360 2024-04-25T13:39:03.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.242+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30007a30 con 0x7f6f40103360 2024-04-25T13:39:03.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.242+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2c005330 con 0x7f6f40103360 2024-04-25T13:39:03.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.244+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6f30017070 con 0x7f6f40103360 2024-04-25T13:39:03.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.245+0000 7f6f3d7fa700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 0x7f6f280b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:03.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.245+0000 7f6f3f7fe700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 0x7f6f280b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:03.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.245+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6f300de6d0 con 0x7f6f40103360 2024-04-25T13:39:03.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.246+0000 7f6f3f7fe700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 0x7f6f280b2090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f40106560 tx=0x7f6f3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:03.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.249+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f3001e030 con 0x7f6f40103360 2024-04-25T13:39:03.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.447+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f2c0058f0 con 0x7f6f40103360 2024-04-25T13:39:03.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.447+0000 7f6f3d7fa700 1 -- 172.21.15.94:0/265907524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f6f300a7750 con 0x7f6f40103360 2024-04-25T13:39:03.448 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:39:03.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 msgr2=0x7f6f280b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:03.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 0x7f6f280b2090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f40106560 tx=0x7f6f3400b410 comp rx=0 tx=0).stop 2024-04-25T13:39:03.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 msgr2=0x7f6f40110920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:03.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 0x7f6f40110920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c970 tx=0x7f6f3000cc80 comp rx=0 tx=0).stop 2024-04-25T13:39:03.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 shutdown_connections 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f401085e0 0x7f6f401113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f280afb80 0x7f6f280b2090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f40103360 0x7f6f40110920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 --2- 172.21.15.94:0/265907524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f40107c90 0x7f6f40110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.449+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 >> 172.21.15.94:0/265907524 conn(0x7f6f400fec90 msgr2=0x7f6f40107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.450+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 shutdown_connections 2024-04-25T13:39:03.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:03.450+0000 7f6f46613700 1 -- 172.21.15.94:0/265907524 wait complete. 2024-04-25T13:39:03.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:03 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/265907524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:03.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:03 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/265907524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:03.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:03 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/265907524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.a on smithi094 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.d on smithi094 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.g on smithi094 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.j on smithi094 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-04-25T13:39:04.131 INFO:tasks.cephadm:Adding mds.c on smithi138 2024-04-25T13:39:04.132 INFO:tasks.cephadm:Adding mds.f on smithi138 2024-04-25T13:39:04.132 INFO:tasks.cephadm:Adding mds.i on smithi138 2024-04-25T13:39:04.132 INFO:tasks.cephadm:Adding mds.l on smithi138 2024-04-25T13:39:04.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l' 2024-04-25T13:39:04.382 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:39:04.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:04 smithi138 ceph-mon[32977]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:04.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:04 smithi137 ceph-mon[32568]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:04.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:04 smithi094 ceph-mon[29195]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:06.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 -- 172.21.15.138:0/3868497529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c1030d0 msgr2=0x7fb89c1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:06.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/3868497529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c1030d0 0x7fb89c1034e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009990 tx=0x7fb88c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 -- 172.21.15.138:0/3868497529 shutdown_connections 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/3868497529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 0x7fb89c10b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/3868497529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c1030d0 0x7fb89c1034e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/3868497529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c104200 0x7fb89c104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 -- 172.21.15.138:0/3868497529 >> 172.21.15.138:0/3868497529 conn(0x7fb89c0fe710 msgr2=0x7fb89c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 -- 172.21.15.138:0/3868497529 shutdown_connections 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.075+0000 7fb8a15c8700 1 -- 172.21.15.138:0/3868497529 wait complete. 2024-04-25T13:39:06.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 Processor -- start 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 -- start start 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c1030d0 0x7fb89c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 0x7fb89c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10eb20 con 0x7fb89c1030d0 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10e9a0 con 0x7fb89c104ba0 2024-04-25T13:39:06.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb8a15c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10eca0 con 0x7fb89c104200 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb89a7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb89a7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36582/0 (socket says 172.21.15.138:36582) 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb89a7fc700 1 -- 172.21.15.138:0/1119907661 learned_addr learned my addr 172.21.15.138:0/1119907661 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb89affd700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c1030d0 0x7fb89c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.076+0000 7fb89b7fe700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 0x7fb89c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 -- 172.21.15.138:0/1119907661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 msgr2=0x7fb89c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 0x7fb89c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 -- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c1030d0 msgr2=0x7fb89c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c1030d0 0x7fb89c06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 -- 172.21.15.138:0/1119907661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c1a2330 con 0x7fb89c104200 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb89a7fc700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c920 tx=0x7fb89000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890004a30 con 0x7fb89c104200 2024-04-25T13:39:06.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c1a2620 con 0x7fb89c104200 2024-04-25T13:39:06.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8900054d0 con 0x7fb89c104200 2024-04-25T13:39:06.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890007bd0 con 0x7fb89c104200 2024-04-25T13:39:06.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.077+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a2b60 con 0x7fb89c104200 2024-04-25T13:39:06.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.078+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c051190 con 0x7fb89c104200 2024-04-25T13:39:06.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.080+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb890004bd0 con 0x7fb89c104200 2024-04-25T13:39:06.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.080+0000 7fb883fff700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 0x7fb8840b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:06.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.080+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb8900de6a0 con 0x7fb89c104200 2024-04-25T13:39:06.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.081+0000 7fb89affd700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 0x7fb8840b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:06.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.081+0000 7fb89affd700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 0x7fb8840b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0038a0 tx=0x7fb88c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:06.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.084+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb890014030 con 0x7fb89c104200 2024-04-25T13:39:06.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.292+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}) -- 0x7fb89c063cd0 con 0x7fb8840afb80 2024-04-25T13:39:06.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.301+0000 7fb883fff700 1 -- 172.21.15.138:0/1119907661 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb89c063cd0 con 0x7fb8840afb80 2024-04-25T13:39:06.301 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mds.all update... 2024-04-25T13:39:06.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 msgr2=0x7fb8840b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:06.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 0x7fb8840b2090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0038a0 tx=0x7fb88c01a040 comp rx=0 tx=0).stop 2024-04-25T13:39:06.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 msgr2=0x7fb89c19f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:06.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c920 tx=0x7fb89000cc30 comp rx=0 tx=0).stop 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 shutdown_connections 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb89c104ba0 0x7fb89c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8840afb80 0x7fb8840b2090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb89c1030d0 0x7fb89c06c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 --2- 172.21.15.138:0/1119907661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c104200 0x7fb89c19f590 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.303+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 >> 172.21.15.138:0/1119907661 conn(0x7fb89c0fe710 msgr2=0x7fb89c107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.304+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 shutdown_connections 2024-04-25T13:39:06.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T13:39:06.304+0000 7fb8a15c8700 1 -- 172.21.15.138:0/1119907661 wait complete. 2024-04-25T13:39:06.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:06 smithi138 ceph-mon[32977]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:06.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:06.530 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:06 smithi137 ceph-mon[32568]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:06 smithi094 ceph-mon[29195]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:06.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:07.010 DEBUG:teuthology.orchestra.run.smithi094:mds.a> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:39:07.013 DEBUG:teuthology.orchestra.run.smithi094:mds.d> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:39:07.055 DEBUG:teuthology.orchestra.run.smithi094:mds.g> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:39:07.098 DEBUG:teuthology.orchestra.run.smithi094:mds.j> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:39:07.141 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:39:07.143 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:39:07.186 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:39:07.230 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:39:07.273 DEBUG:teuthology.orchestra.run.smithi138:mds.c> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:39:07.275 DEBUG:teuthology.orchestra.run.smithi138:mds.f> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:39:07.318 DEBUG:teuthology.orchestra.run.smithi138:mds.i> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:39:07.360 DEBUG:teuthology.orchestra.run.smithi138:mds.l> sudo journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:39:07.404 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:39:07.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:39:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 -- 172.21.15.94:0/1609767970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 msgr2=0x7fd644105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/1609767970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd644105160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd634009ca0 tx=0x7fd63402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 -- 172.21.15.94:0/1609767970 shutdown_connections 2024-04-25T13:39:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/1609767970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644109a50 0x7fd64410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/1609767970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd644105160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/1609767970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644103b10 0x7fd644104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 -- 172.21.15.94:0/1609767970 >> 172.21.15.94:0/1609767970 conn(0x7fd6440fed40 msgr2=0x7fd6441011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 -- 172.21.15.94:0/1609767970 shutdown_connections 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.525+0000 7fd64ac6e640 1 -- 172.21.15.94:0/1609767970 wait complete. 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 Processor -- start 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 -- start start 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644103b10 0x7fd64419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644109a50 0x7fd6441a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410f640 con 0x7fd644104d50 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410f4c0 con 0x7fd644109a50 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64ac6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410f7c0 con 0x7fd644103b10 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44400/0 (socket says 172.21.15.94:44400) 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd643fff640 1 -- 172.21.15.94:0/3268738634 learned_addr learned my addr 172.21.15.94:0/3268738634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd648a0a640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644103b10 0x7fd64419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.526+0000 7fd64920b640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644109a50 0x7fd6441a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 -- 172.21.15.94:0/3268738634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644103b10 msgr2=0x7fd64419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644103b10 0x7fd64419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 -- 172.21.15.94:0/3268738634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644109a50 msgr2=0x7fd6441a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644109a50 0x7fd6441a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 -- 172.21.15.94:0/3268738634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6441a1d80 con 0x7fd644104d50 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd643fff640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd63402ade0 tx=0x7fd634003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63402f380 con 0x7fd644104d50 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6340317c0 con 0x7fd644104d50 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6441a2010 con 0x7fd644104d50 2024-04-25T13:39:07.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63402fc60 con 0x7fd644104d50 2024-04-25T13:39:07.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.527+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6441a2560 con 0x7fd644104d50 2024-04-25T13:39:07.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.528+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644108860 con 0x7fd644104d50 2024-04-25T13:39:07.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.530+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd634004040 con 0x7fd644104d50 2024-04-25T13:39:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.530+0000 7fd641ffb640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 0x7fd6300b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.530+0000 7fd648a0a640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 0x7fd6300b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.530+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd6340ffcd0 con 0x7fd644104d50 2024-04-25T13:39:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.531+0000 7fd648a0a640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 0x7fd6300b21a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd644104710 tx=0x7fd63c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:07.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.535+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6340c8d50 con 0x7fd644104d50 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:07.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:07 smithi137 ceph-mon[32568]: Deploying daemon mds.l on smithi138 2024-04-25T13:39:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.716+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6440637a0 con 0x7fd644104d50 2024-04-25T13:39:07.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.717+0000 7fd641ffb640 1 -- 172.21.15.94:0/3268738634 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18481 (secure 0 0 0) 0x7fd6340c8650 con 0x7fd644104d50 2024-04-25T13:39:07.718 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:07.718 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":75,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:38:55.378345+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:07.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 msgr2=0x7fd6300b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 0x7fd6300b21a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd644104710 tx=0x7fd63c00b390 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 msgr2=0x7fd64419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd63402ade0 tx=0x7fd634003ba0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 shutdown_connections 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6300afc90 0x7fd6300b21a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd644109a50 0x7fd6441a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd644104d50 0x7fd64419a780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 --2- 172.21.15.94:0/3268738634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd644103b10 0x7fd64419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 >> 172.21.15.94:0/3268738634 conn(0x7fd6440fed40 msgr2=0x7fd644105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 shutdown_connections 2024-04-25T13:39:07.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.720+0000 7fd64ac6e640 1 -- 172.21.15.94:0/3268738634 wait complete. 2024-04-25T13:39:07.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:07 smithi094 ceph-mon[29195]: Deploying daemon mds.l on smithi138 2024-04-25T13:39:07.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:39:07.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:07.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:07 smithi138 ceph-mon[32977]: Deploying daemon mds.l on smithi138 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 -- 172.21.15.94:0/961671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d01033b0 msgr2=0x7f94d01037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 --2- 172.21.15.94:0/961671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d01033b0 0x7f94d01037c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94c400aa10 tx=0x7f94c402b720 comp rx=0 tx=0).stop 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 -- 172.21.15.94:0/961671860 shutdown_connections 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 --2- 172.21.15.94:0/961671860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01080b0 0x7f94d010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 --2- 172.21.15.94:0/961671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d01033b0 0x7f94d01037c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 --2- 172.21.15.94:0/961671860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d0102170 0x7f94d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 -- 172.21.15.94:0/961671860 >> 172.21.15.94:0/961671860 conn(0x7f94d00fd380 msgr2=0x7f94d00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:07.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 -- 172.21.15.94:0/961671860 shutdown_connections 2024-04-25T13:39:07.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.841+0000 7f94d4da8640 1 -- 172.21.15.94:0/961671860 wait complete. 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 Processor -- start 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 -- start start 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01033b0 0x7f94d019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d01080b0 0x7f94d01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94d010d400 con 0x7f94d0102170 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94d010d280 con 0x7f94d01033b0 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94d4da8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94d010d580 con 0x7f94d01080b0 2024-04-25T13:39:07.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.842+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 -- 172.21.15.94:0/188044310 learned_addr learned my addr 172.21.15.94:0/188044310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94cdd9b640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01033b0 0x7f94d019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ced9d640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d01080b0 0x7f94d01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 -- 172.21.15.94:0/188044310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d01080b0 msgr2=0x7f94d01a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d01080b0 0x7f94d01a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 -- 172.21.15.94:0/188044310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01033b0 msgr2=0x7f94d019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01033b0 0x7f94d019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 -- 172.21.15.94:0/188044310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d01a1ca0 con 0x7f94d0102170 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94ce59c640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94c000d740 tx=0x7f94c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0017070 con 0x7f94d0102170 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94c000de50 con 0x7f94d0102170 2024-04-25T13:39:07.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d01a1f90 con 0x7f94d0102170 2024-04-25T13:39:07.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0003c20 con 0x7f94d0102170 2024-04-25T13:39:07.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.843+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d01a23f0 con 0x7f94d0102170 2024-04-25T13:39:07.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.844+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d006a190 con 0x7f94d0102170 2024-04-25T13:39:07.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.846+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f94c0003dc0 con 0x7f94d0102170 2024-04-25T13:39:07.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.846+0000 7f94bf7fe640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 0x7f94b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:07.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.846+0000 7f94cdd9b640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 0x7f94b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:07.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.846+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f94c00de600 con 0x7f94d0102170 2024-04-25T13:39:07.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.847+0000 7f94cdd9b640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 0x7f94b80b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94c400a340 tx=0x7f94c4030510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:07.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:07.850+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94d006a190 con 0x7f94d0102170 2024-04-25T13:39:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.028+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94d01a28f0 con 0x7f94d0102170 2024-04-25T13:39:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.028+0000 7f94bf7fe640 1 -- 172.21.15.94:0/188044310 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f94c00a7680 con 0x7f94d0102170 2024-04-25T13:39:08.029 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 msgr2=0x7f94b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 0x7f94b80b2000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94c400a340 tx=0x7f94c4030510 comp rx=0 tx=0).stop 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 msgr2=0x7f94d019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94c000d740 tx=0x7f94c000dc00 comp rx=0 tx=0).stop 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 shutdown_connections 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94b80afaf0 0x7f94b80b2000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94d01080b0 0x7f94d01a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94d01033b0 0x7f94d019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 --2- 172.21.15.94:0/188044310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d0102170 0x7f94d019a210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 >> 172.21.15.94:0/188044310 conn(0x7f94d00fd380 msgr2=0x7f94d01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 shutdown_connections 2024-04-25T13:39:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.030+0000 7f94d4da8640 1 -- 172.21.15.94:0/188044310 wait complete. 2024-04-25T13:39:08.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 -- 172.21.15.94:0/319471724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88108a20 msgr2=0x7fae8810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 --2- 172.21.15.94:0/319471724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88108a20 0x7fae8810ae40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ca0 tx=0x7fae8002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 -- 172.21.15.94:0/319471724 shutdown_connections 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 --2- 172.21.15.94:0/319471724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88108a20 0x7fae8810ae40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 --2- 172.21.15.94:0/319471724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae881043d0 0x7fae88102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 --2- 172.21.15.94:0/319471724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88103ef0 0x7fae881027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.149+0000 7fae8dc67640 1 -- 172.21.15.94:0/319471724 >> 172.21.15.94:0/319471724 conn(0x7fae880fd370 msgr2=0x7fae880ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:08.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- 172.21.15.94:0/319471724 shutdown_connections 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- 172.21.15.94:0/319471724 wait complete. 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 Processor -- start 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- start start 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae88108a20 0x7fae881991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae8819e100 0x7fae881a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810e530 con 0x7fae88103ef0 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810e3b0 con 0x7fae88108a20 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.150+0000 7fae8dc67640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810e6b0 con 0x7fae8819e100 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44434/0 (socket says 172.21.15.94:44434) 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 -- 172.21.15.94:0/407200292 learned_addr learned my addr 172.21.15.94:0/407200292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:08.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae86ffd640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae88108a20 0x7fae881991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae87fff640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae8819e100 0x7fae881a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 -- 172.21.15.94:0/407200292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae8819e100 msgr2=0x7fae881a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae8819e100 0x7fae881a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 -- 172.21.15.94:0/407200292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae88108a20 msgr2=0x7fae881991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae88108a20 0x7fae881991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 -- 172.21.15.94:0/407200292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae881a0b90 con 0x7fae88103ef0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae877fe640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9f0 tx=0x7fae7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7800ccc0 con 0x7fae88103ef0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881a0e80 con 0x7fae88103ef0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae780054d0 con 0x7fae88103ef0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78010430 con 0x7fae88103ef0 2024-04-25T13:39:08.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.151+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a12e0 con 0x7fae88103ef0 2024-04-25T13:39:08.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.152+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88107120 con 0x7fae88103ef0 2024-04-25T13:39:08.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.154+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fae780046c0 con 0x7fae88103ef0 2024-04-25T13:39:08.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.154+0000 7fae84ff9640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 0x7fae700b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:08.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.154+0000 7fae86ffd640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 0x7fae700b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:08.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.154+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fae780deab0 con 0x7fae88103ef0 2024-04-25T13:39:08.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.155+0000 7fae86ffd640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 0x7fae700b2030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00bbe0 tx=0x7fae7c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.159+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae780a7b30 con 0x7fae88103ef0 2024-04-25T13:39:08.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.378+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae8802db50 con 0x7fae88103ef0 2024-04-25T13:39:08.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.379+0000 7fae84ff9640 1 -- 172.21.15.94:0/407200292 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fae780a7430 con 0x7fae88103ef0 2024-04-25T13:39:08.380 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:08.380 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:39:08.380 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 1 2024-04-25T13:39:08.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 msgr2=0x7fae700b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 0x7fae700b2030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00bbe0 tx=0x7fae7c00b390 comp rx=0 tx=0).stop 2024-04-25T13:39:08.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 msgr2=0x7fae88198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:08.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9f0 tx=0x7fae7800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 shutdown_connections 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae700afb20 0x7fae700b2030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae8819e100 0x7fae881a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae88108a20 0x7fae881991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 --2- 172.21.15.94:0/407200292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88103ef0 0x7fae88198c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 >> 172.21.15.94:0/407200292 conn(0x7fae880fd370 msgr2=0x7fae880ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 shutdown_connections 2024-04-25T13:39:08.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:08.381+0000 7fae8dc67640 1 -- 172.21.15.94:0/407200292 wait complete. 2024-04-25T13:39:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3268738634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/188044310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:08 smithi138 ceph-mon[32977]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:08 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3268738634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:08 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/188044310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:08 smithi137 ceph-mon[32568]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3268738634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/188044310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:08 smithi094 ceph-mon[29195]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/407200292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:09 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/407200292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:09 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/407200292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:09.796 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 13:39:09 smithi138 podman[62993]: 2024-04-25 13:39:09.763796588 +0000 UTC m=+0.356982636 container start a002a84e219607ec4d0a0ef667bf7493809d98528a372e135055d3cafb2b45fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T13:39:10.247 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 13:39:09 smithi138 ceph-mds[63013]: -- 172.21.15.138:0/3593802539 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cac2b1ba40 con 0x55cac2b63180 2024-04-25T13:39:10.247 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 13:39:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: starting mds.l at 2024-04-25T13:39:10.247 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 13:39:09 smithi138 bash[62993]: a002a84e219607ec4d0a0ef667bf7493809d98528a372e135055d3cafb2b45fd 2024-04-25T13:39:10.247 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 13:39:09 smithi138 systemd[1]: Started Ceph mds.l for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:10.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:10.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 -- 172.21.15.94:0/1269445269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c108920 msgr2=0x7f650c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 --2- 172.21.15.94:0/1269445269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c108920 0x7f650c10ad40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009900 tx=0x7f64fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 -- 172.21.15.94:0/1269445269 shutdown_connections 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 --2- 172.21.15.94:0/1269445269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c108920 0x7f650c10ad40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 --2- 172.21.15.94:0/1269445269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c103c20 0x7f650c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 --2- 172.21.15.94:0/1269445269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f650c1029e0 0x7f650c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 -- 172.21.15.94:0/1269445269 >> 172.21.15.94:0/1269445269 conn(0x7f650c0fdbf0 msgr2=0x7f650c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 -- 172.21.15.94:0/1269445269 shutdown_connections 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.513+0000 7f651430d640 1 -- 172.21.15.94:0/1269445269 wait complete. 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 Processor -- start 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 -- start start 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 0x7f650c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f650c108920 0x7f650c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:10.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10e4f0 con 0x7f650c103c20 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10e370 con 0x7f650c1029e0 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f651430d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f650c10e670 con 0x7f650c108920 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44444/0 (socket says 172.21.15.94:44444) 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 -- 172.21.15.94:0/338498813 learned_addr learned my addr 172.21.15.94:0/338498813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65120a9640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 0x7f650c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 -- 172.21.15.94:0/338498813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f650c108920 msgr2=0x7f650c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f650c108920 0x7f650c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 -- 172.21.15.94:0/338498813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 msgr2=0x7f650c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 0x7f650c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 -- 172.21.15.94:0/338498813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c1a0c50 con 0x7f650c103c20 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65120a9640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 0x7f650c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f65118a8640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9f0 tx=0x7f650000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.514+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000cd80 con 0x7f650c103c20 2024-04-25T13:39:10.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.515+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65000108e0 con 0x7f650c103c20 2024-04-25T13:39:10.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.515+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c1a0f40 con 0x7f650c103c20 2024-04-25T13:39:10.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.515+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500018980 con 0x7f650c103c20 2024-04-25T13:39:10.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.515+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1a13b0 con 0x7f650c103c20 2024-04-25T13:39:10.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.515+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c107730 con 0x7f650c103c20 2024-04-25T13:39:10.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.518+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6500010430 con 0x7f650c103c20 2024-04-25T13:39:10.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.518+0000 7f650b7fe640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 0x7f64f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:10.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.519+0000 7f65120a9640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 0x7f64f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:10.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.519+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f65000df050 con 0x7f650c103c20 2024-04-25T13:39:10.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.519+0000 7f65120a9640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 0x7f64f40b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f650c1035e0 tx=0x7f65040095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:10.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.522+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65000a80d0 con 0x7f650c103c20 2024-04-25T13:39:10.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.742+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f650c02db50 con 0x7f650c103c20 2024-04-25T13:39:10.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.742+0000 7f650b7fe640 1 -- 172.21.15.94:0/338498813 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f65000b0150 con 0x7f650c103c20 2024-04-25T13:39:10.743 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:10.743 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:39:10.743 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-25T13:39:10.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.744+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 msgr2=0x7f64f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:10.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.744+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 0x7f64f40b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f650c1035e0 tx=0x7f65040095c0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.744+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 msgr2=0x7f650c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.744+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f650000e9f0 tx=0x7f650000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 shutdown_connections 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64f40afaf0 0x7f64f40b2000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f650c108920 0x7f650c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f650c103c20 0x7f650c1996c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 --2- 172.21.15.94:0/338498813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c1029e0 0x7f650c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 >> 172.21.15.94:0/338498813 conn(0x7f650c0fdbf0 msgr2=0x7f650c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 shutdown_connections 2024-04-25T13:39:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:10.745+0000 7f651430d640 1 -- 172.21.15.94:0/338498813 wait complete. 2024-04-25T13:39:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6832/4012763239,v1:172.21.15.138:6833/4012763239] up:boot 2024-04-25T13:39:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: fsmap 1 up:standby 2024-04-25T13:39:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:39:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: Deploying daemon mds.k on smithi137 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:10.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:10 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/338498813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:11.066 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6832/4012763239,v1:172.21.15.138:6833/4012763239] up:boot 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: fsmap 1 up:standby 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: Deploying daemon mds.k on smithi137 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:11.067 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:10 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/338498813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6832/4012763239,v1:172.21.15.138:6833/4012763239] up:boot 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: fsmap 1 up:standby 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:11.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:11.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: Deploying daemon mds.k on smithi137 2024-04-25T13:39:11.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:11.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:10 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/338498813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:12.322 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 systemd[1]: Starting Ceph mds.k for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:12.625 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:12 smithi137 ceph-mon[32568]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:12.625 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 podman[62552]: 2024-04-25 13:39:12.277117943 +0000 UTC m=+0.017908734 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:12.625 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 podman[62552]: 2024-04-25 13:39:12.390475541 +0000 UTC m=+0.131266357 container create c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T13:39:12.625 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 podman[62552]: 2024-04-25 13:39:12.625908154 +0000 UTC m=+0.366698928 container init c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:12 smithi094 ceph-mon[29195]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:12 smithi138 ceph-mon[32977]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:12.762 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:12.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 podman[62552]: 2024-04-25 13:39:12.628954532 +0000 UTC m=+0.369745310 container start c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-25T13:39:12.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 ceph-mds[62571]: -- 172.21.15.137:0/2168196897 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3a160fa40 con 0x55e3a1657180 2024-04-25T13:39:12.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: starting mds.k at 2024-04-25T13:39:12.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 bash[62552]: c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 2024-04-25T13:39:12.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 13:39:12 smithi137 systemd[1]: Started Ceph mds.k for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 -- 172.21.15.94:0/1508663151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 msgr2=0x7fb1881033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 --2- 172.21.15.94:0/1508663151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb1881033f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb178009860 tx=0x7fb17802a340 comp rx=0 tx=0).stop 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 -- 172.21.15.94:0/1508663151 shutdown_connections 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 --2- 172.21.15.94:0/1508663151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1881046a0 0x7fb18810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 --2- 172.21.15.94:0/1508663151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb188104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:12.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 --2- 172.21.15.94:0/1508663151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb1881033f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:12.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.896+0000 7fb18cd72640 1 -- 172.21.15.94:0/1508663151 >> 172.21.15.94:0/1508663151 conn(0x7fb1880fd360 msgr2=0x7fb1880ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:12.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.897+0000 7fb18cd72640 1 -- 172.21.15.94:0/1508663151 shutdown_connections 2024-04-25T13:39:12.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.897+0000 7fb18cd72640 1 -- 172.21.15.94:0/1508663151 wait complete. 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.897+0000 7fb18cd72640 1 Processor -- start 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.897+0000 7fb18cd72640 1 -- start start 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb188102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb18819f2d0 0x7fb1881a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb18810e720 con 0x7fb188103b40 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb18810e5a0 con 0x7fb188102fe0 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18cd72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb18810e8a0 con 0x7fb18819f2d0 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:12.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44462/0 (socket says 172.21.15.94:44462) 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 -- 172.21.15.94:0/3686487593 learned_addr learned my addr 172.21.15.94:0/3686487593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb18659c640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb188102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb186d9d640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb18819f2d0 0x7fb1881a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 -- 172.21.15.94:0/3686487593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb18819f2d0 msgr2=0x7fb1881a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb18819f2d0 0x7fb1881a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 -- 172.21.15.94:0/3686487593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 msgr2=0x7fb188102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb188102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 -- 172.21.15.94:0/3686487593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1881a1f10 con 0x7fb188103b40 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.898+0000 7fb185d9b640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00e9c0 tx=0x7fb17c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.899+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00cde0 con 0x7fb188103b40 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.899+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1881a2200 con 0x7fb188103b40 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.899+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb17c0035f0 con 0x7fb188103b40 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.899+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c010690 con 0x7fb188103b40 2024-04-25T13:39:12.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.899+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1881a2750 con 0x7fb188103b40 2024-04-25T13:39:12.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.900+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18806b380 con 0x7fb188103b40 2024-04-25T13:39:12.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.901+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb17c004000 con 0x7fb188103b40 2024-04-25T13:39:12.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.902+0000 7fb1777fe640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 0x7fb1700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:12.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.902+0000 7fb18659c640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 0x7fb1700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:12.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.902+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb17c0dec10 con 0x7fb188103b40 2024-04-25T13:39:12.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.903+0000 7fb18659c640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 0x7fb1700b2000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb17802a930 tx=0x7fb178030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:12.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:12.907+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb17c0a7c80 con 0x7fb188103b40 2024-04-25T13:39:13.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.130+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1881073f0 con 0x7fb188103b40 2024-04-25T13:39:13.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.130+0000 7fb1777fe640 1 -- 172.21.15.94:0/3686487593 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb17c0a7580 con 0x7fb188103b40 2024-04-25T13:39:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:13.131 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:39:13.131 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 msgr2=0x7fb1700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 0x7fb1700b2000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb17802a930 tx=0x7fb178030450 comp rx=0 tx=0).stop 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 msgr2=0x7fb18819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00e9c0 tx=0x7fb17c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 shutdown_connections 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1700afaf0 0x7fb1700b2000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb18819f2d0 0x7fb1881a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb188103b40 0x7fb18819ed90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:13.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 --2- 172.21.15.94:0/3686487593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb188102fe0 0x7fb188102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:13.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 >> 172.21.15.94:0/3686487593 conn(0x7fb1880fd360 msgr2=0x7fb1880ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:13.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 shutdown_connections 2024-04-25T13:39:13.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:13.132+0000 7fb18cd72640 1 -- 172.21.15.94:0/3686487593 wait complete. 2024-04-25T13:39:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:14.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: Deploying daemon mds.j on smithi094 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3686487593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6832/3830518540,v1:172.21.15.137:6833/3830518540] up:boot 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: fsmap 2 up:standby 2024-04-25T13:39:14.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: Deploying daemon mds.j on smithi094 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3686487593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6832/3830518540,v1:172.21.15.137:6833/3830518540] up:boot 2024-04-25T13:39:14.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: fsmap 2 up:standby 2024-04-25T13:39:14.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: Deploying daemon mds.j on smithi094 2024-04-25T13:39:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3686487593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6832/3830518540,v1:172.21.15.137:6833/3830518540] up:boot 2024-04-25T13:39:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: fsmap 2 up:standby 2024-04-25T13:39:14.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:39:15.065 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:14 smithi137 ceph-mon[32568]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:15.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:14 smithi094 ceph-mon[29195]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:15.086 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 systemd[1]: Starting Ceph mds.j for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:15.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:15.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:14 smithi138 ceph-mon[32977]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:15.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.266+0000 7f81bd267640 1 -- 172.21.15.94:0/2746514295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8108900 msgr2=0x7f81b810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:15.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.266+0000 7f81bd267640 1 --2- 172.21.15.94:0/2746514295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8108900 0x7f81b810ad20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009ca0 tx=0x7f81b002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 -- 172.21.15.94:0/2746514295 shutdown_connections 2024-04-25T13:39:15.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 --2- 172.21.15.94:0/2746514295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8108900 0x7f81b810ad20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 --2- 172.21.15.94:0/2746514295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8103c00 0x7f81b8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 --2- 172.21.15.94:0/2746514295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b81029c0 0x7f81b81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 -- 172.21.15.94:0/2746514295 >> 172.21.15.94:0/2746514295 conn(0x7f81b80fdbd0 msgr2=0x7f81b8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 -- 172.21.15.94:0/2746514295 shutdown_connections 2024-04-25T13:39:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 -- 172.21.15.94:0/2746514295 wait complete. 2024-04-25T13:39:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 Processor -- start 2024-04-25T13:39:15.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.267+0000 7f81bd267640 1 -- start start 2024-04-25T13:39:15.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b81029c0 0x7f81b806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:15.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:15.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 0x7f81b81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81b810e4f0 con 0x7f81b8103c00 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81b810e370 con 0x7f81b81029c0 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81b810e670 con 0x7f81b8108900 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45922/0 (socket says 172.21.15.94:45922) 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 -- 172.21.15.94:0/3716888131 learned_addr learned my addr 172.21.15.94:0/3716888131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b6ffd640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b81029c0 0x7f81b806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b77fe640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 0x7f81b81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 -- 172.21.15.94:0/3716888131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 msgr2=0x7f81b81a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 0x7f81b81a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 -- 172.21.15.94:0/3716888131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b81029c0 msgr2=0x7f81b806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:15.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b81029c0 0x7f81b806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 -- 172.21.15.94:0/3716888131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b81a1ce0 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b77fe640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 0x7f81b81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81b67fc640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e9f0 tx=0x7f81a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a800cd80 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a80108e0 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018980 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b81a1fd0 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.268+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b81a2490 con 0x7f81b8103c00 2024-04-25T13:39:15.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.269+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b8107710 con 0x7f81b8103c00 2024-04-25T13:39:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.271+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f81a8010430 con 0x7f81b8103c00 2024-04-25T13:39:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.271+0000 7f819ffff640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 0x7f81a00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.271+0000 7f81b6ffd640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 0x7f81a00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.271+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f81a80def50 con 0x7f81b8103c00 2024-04-25T13:39:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.272+0000 7f81b6ffd640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 0x7f81a00b2080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0058c0 tx=0x7f81ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.275+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a80a7fd0 con 0x7f81b8103c00 2024-04-25T13:39:15.432 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 podman[69528]: 2024-04-25 13:39:15.214445219 +0000 UTC m=+0.022545682 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:15.432 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 podman[69528]: 2024-04-25 13:39:15.332196941 +0000 UTC m=+0.140297393 container create abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:39:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.499+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81b802db50 con 0x7f81b8103c00 2024-04-25T13:39:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.499+0000 7f819ffff640 1 -- 172.21.15.94:0/3716888131 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f81a80a78d0 con 0x7f81b8103c00 2024-04-25T13:39:15.500 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:15.500 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:39:15.500 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 msgr2=0x7f81a00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 0x7f81a00b2080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0058c0 tx=0x7f81ac00a2d0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 msgr2=0x7f81b819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e9f0 tx=0x7f81a800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 shutdown_connections 2024-04-25T13:39:15.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81a00afb70 0x7f81a00b2080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b8108900 0x7f81b81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81b8103c00 0x7f81b819ef70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 --2- 172.21.15.94:0/3716888131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b81029c0 0x7f81b806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.501+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 >> 172.21.15.94:0/3716888131 conn(0x7f81b80fdbd0 msgr2=0x7f81b81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.502+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 shutdown_connections 2024-04-25T13:39:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:15.502+0000 7f81bd267640 1 -- 172.21.15.94:0/3716888131 wait complete. 2024-04-25T13:39:15.811 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 podman[69528]: 2024-04-25 13:39:15.568432528 +0000 UTC m=+0.376532984 container init abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T13:39:15.812 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 podman[69528]: 2024-04-25 13:39:15.571013721 +0000 UTC m=+0.379114176 container start abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:39:15.812 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mds[69567]: -- 172.21.15.94:0/1970346400 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf79605a40 con 0x55cf7964da80 2024-04-25T13:39:15.812 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j[69561]: starting mds.j at 2024-04-25T13:39:15.812 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 bash[69528]: abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 2024-04-25T13:39:15.812 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 13:39:15 smithi094 systemd[1]: Started Ceph mds.j for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3716888131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:16.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:16.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3716888131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3716888131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:16 smithi094 ceph-mon[29195]: Deploying daemon mds.d on smithi094 2024-04-25T13:39:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:16 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6834/4024796429,v1:172.21.15.94:6835/4024796429] up:boot 2024-04-25T13:39:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:16 smithi094 ceph-mon[29195]: fsmap 3 up:standby 2024-04-25T13:39:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:16 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:39:16.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:16 smithi094 ceph-mon[29195]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:17.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:16 smithi137 ceph-mon[32568]: Deploying daemon mds.d on smithi094 2024-04-25T13:39:17.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:16 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6834/4024796429,v1:172.21.15.94:6835/4024796429] up:boot 2024-04-25T13:39:17.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:16 smithi137 ceph-mon[32568]: fsmap 3 up:standby 2024-04-25T13:39:17.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:16 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:39:17.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:16 smithi137 ceph-mon[32568]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:16 smithi138 ceph-mon[32977]: Deploying daemon mds.d on smithi094 2024-04-25T13:39:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:16 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6834/4024796429,v1:172.21.15.94:6835/4024796429] up:boot 2024-04-25T13:39:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:16 smithi138 ceph-mon[32977]: fsmap 3 up:standby 2024-04-25T13:39:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:16 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:39:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:16 smithi138 ceph-mon[32977]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:17.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- 172.21.15.94:0/1451339226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe01039b0 msgr2=0x7fdbe0103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/1451339226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe01039b0 0x7fdbe0103e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009900 tx=0x7fdbd002a410 comp rx=0 tx=0).stop 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- 172.21.15.94:0/1451339226 shutdown_connections 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/1451339226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/1451339226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe01039b0 0x7fdbe0103e40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/1451339226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe0102fd0 0x7fdbe01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- 172.21.15.94:0/1451339226 >> 172.21.15.94:0/1451339226 conn(0x7fdbe00fd3b0 msgr2=0x7fdbe00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- 172.21.15.94:0/1451339226 shutdown_connections 2024-04-25T13:39:17.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- 172.21.15.94:0/1451339226 wait complete. 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 Processor -- start 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.647+0000 7fdbe79e3640 1 -- start start 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe01039b0 0x7fdbe019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010f680 con 0x7fdbe0102fd0 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010f500 con 0x7fdbe01044d0 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010f800 con 0x7fdbe01039b0 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45964/0 (socket says 172.21.15.94:45964) 2024-04-25T13:39:17.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 -- 172.21.15.94:0/352963864 learned_addr learned my addr 172.21.15.94:0/352963864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe5f80640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe4f7e640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe01039b0 0x7fdbe019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 -- 172.21.15.94:0/352963864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe01039b0 msgr2=0x7fdbe019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe01039b0 0x7fdbe019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 -- 172.21.15.94:0/352963864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 msgr2=0x7fdbe01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 -- 172.21.15.94:0/352963864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe01a1d60 con 0x7fdbe0102fd0 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe5f80640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe577f640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00b450 tx=0x7fdbdc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc015020 con 0x7fdbe0102fd0 2024-04-25T13:39:17.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbdc011d40 con 0x7fdbe0102fd0 2024-04-25T13:39:17.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe01a1ff0 con 0x7fdbe0102fd0 2024-04-25T13:39:17.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.648+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00f340 con 0x7fdbe0102fd0 2024-04-25T13:39:17.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.649+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01a2530 con 0x7fdbe0102fd0 2024-04-25T13:39:17.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.649+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe0107970 con 0x7fdbe0102fd0 2024-04-25T13:39:17.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.651+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fdbdc018020 con 0x7fdbe0102fd0 2024-04-25T13:39:17.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.652+0000 7fdbd67fc640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 0x7fdbcc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:17.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.652+0000 7fdbe4f7e640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 0x7fdbcc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:17.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.652+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fdbdc0de810 con 0x7fdbe0102fd0 2024-04-25T13:39:17.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.652+0000 7fdbe4f7e640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 0x7fdbcc0b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbd002a930 tx=0x7fdbd0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:17.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.656+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbdc01f020 con 0x7fdbe0102fd0 2024-04-25T13:39:17.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.888+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbe002db50 con 0x7fdbe0102fd0 2024-04-25T13:39:17.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.893+0000 7fdbd67fc640 1 -- 172.21.15.94:0/352963864 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fdbdc00d040 con 0x7fdbe0102fd0 2024-04-25T13:39:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:17.893 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-04-25T13:39:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 msgr2=0x7fdbcc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 0x7fdbcc0b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbd002a930 tx=0x7fdbd0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 msgr2=0x7fdbe019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00b450 tx=0x7fdbdc00b910 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 shutdown_connections 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdbcc0afaf0 0x7fdbcc0b2000 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbe01044d0 0x7fdbe01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe01039b0 0x7fdbe019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 --2- 172.21.15.94:0/352963864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbe0102fd0 0x7fdbe019a220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 >> 172.21.15.94:0/352963864 conn(0x7fdbe00fd3b0 msgr2=0x7fdbe0104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 shutdown_connections 2024-04-25T13:39:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:17.896+0000 7fdbe79e3640 1 -- 172.21.15.94:0/352963864 wait complete. 2024-04-25T13:39:18.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/352963864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:18.226 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 systemd[1]: Starting Ceph mds.d for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:18.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/352963864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:17 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/352963864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:18.516 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 podman[69894]: 2024-04-25 13:39:18.149812524 +0000 UTC m=+0.017731492 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:18.517 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 podman[69894]: 2024-04-25 13:39:18.289004854 +0000 UTC m=+0.156923816 container create 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T13:39:18.517 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 podman[69894]: 2024-04-25 13:39:18.517546373 +0000 UTC m=+0.385465333 container init 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:39:18.950 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 podman[69894]: 2024-04-25 13:39:18.519992202 +0000 UTC m=+0.387911164 container start 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T13:39:18.950 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mds[69913]: -- 172.21.15.94:0/4052137246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab3b485a40 con 0x55ab3b4cd600 2024-04-25T13:39:18.950 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d[69909]: starting mds.d at 2024-04-25T13:39:18.950 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 bash[69894]: 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 2024-04-25T13:39:18.951 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 13:39:18 smithi094 systemd[1]: Started Ceph mds.d for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:19.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:19.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:18 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:19.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:18 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:19.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:18 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:19.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 -- 172.21.15.94:0/473448162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8104d50 msgr2=0x7f6af8105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/473448162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8104d50 0x7f6af8105160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae802a410 comp rx=0 tx=0).stop 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 -- 172.21.15.94:0/473448162 shutdown_connections 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/473448162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 0x7f6af810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/473448162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8104d50 0x7f6af8105160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/473448162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8103b10 0x7f6af8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 -- 172.21.15.94:0/473448162 >> 172.21.15.94:0/473448162 conn(0x7f6af80fed40 msgr2=0x7f6af81011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 -- 172.21.15.94:0/473448162 shutdown_connections 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.034+0000 7f6aff9ba640 1 -- 172.21.15.94:0/473448162 wait complete. 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 Processor -- start 2024-04-25T13:39:20.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 -- start start 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 0x7f6af819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 0x7f6af81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810f630 con 0x7f6af8103b10 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810f4b0 con 0x7f6af8109a50 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6aff9ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810f7b0 con 0x7f6af8104d50 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6afd756640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6afd756640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45990/0 (socket says 172.21.15.94:45990) 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6afd756640 1 -- 172.21.15.94:0/3128935976 learned_addr learned my addr 172.21.15.94:0/3128935976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.035+0000 7f6afdf57640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 0x7f6af81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afcf55640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 0x7f6af819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:20.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 -- 172.21.15.94:0/3128935976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 msgr2=0x7f6af819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 0x7f6af819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 -- 172.21.15.94:0/3128935976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 msgr2=0x7f6af81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 0x7f6af81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 -- 172.21.15.94:0/3128935976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af81a1da0 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afcf55640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 0x7f6af819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6afd756640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005d00 tx=0x7f6af400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4017070 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af4009c30 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81a2090 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400f620 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.036+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a2590 con 0x7f6af8103b10 2024-04-25T13:39:20.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.037+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af8108860 con 0x7f6af8103b10 2024-04-25T13:39:20.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.039+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6af401a070 con 0x7f6af8103b10 2024-04-25T13:39:20.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.039+0000 7f6aee7fc640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 0x7f6ae40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:20.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.040+0000 7f6afcf55640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 0x7f6ae40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:20.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.040+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6af40de210 con 0x7f6af8103b10 2024-04-25T13:39:20.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.040+0000 7f6afcf55640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 0x7f6ae40b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006010 tx=0x7f6ae8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:20.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.044+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6af4009de0 con 0x7f6af8103b10 2024-04-25T13:39:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:19 smithi094 ceph-mon[29195]: Deploying daemon mds.e on smithi137 2024-04-25T13:39:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:19 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6836/2786321466,v1:172.21.15.94:6837/2786321466] up:boot 2024-04-25T13:39:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:19 smithi094 ceph-mon[29195]: fsmap 4 up:standby 2024-04-25T13:39:20.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:39:20.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:19 smithi138 ceph-mon[32977]: Deploying daemon mds.e on smithi137 2024-04-25T13:39:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:19 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6836/2786321466,v1:172.21.15.94:6837/2786321466] up:boot 2024-04-25T13:39:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:19 smithi138 ceph-mon[32977]: fsmap 4 up:standby 2024-04-25T13:39:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:39:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.262+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6af802db50 con 0x7f6af8103b10 2024-04-25T13:39:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.262+0000 7f6aee7fc640 1 -- 172.21.15.94:0/3128935976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f6af40c7e20 con 0x7f6af8103b10 2024-04-25T13:39:20.263 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:20.263 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:39:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 5 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 msgr2=0x7f6ae40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 0x7f6ae40b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006010 tx=0x7f6ae8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 msgr2=0x7f6af819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005d00 tx=0x7f6af400dde0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 shutdown_connections 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6ae40afaf0 0x7f6ae40b2000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6af8109a50 0x7f6af81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8104d50 0x7f6af819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 --2- 172.21.15.94:0/3128935976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6af8103b10 0x7f6af819a260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 >> 172.21.15.94:0/3128935976 conn(0x7f6af80fed40 msgr2=0x7f6af8105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 shutdown_connections 2024-04-25T13:39:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:20.265+0000 7f6aff9ba640 1 -- 172.21.15.94:0/3128935976 wait complete. 2024-04-25T13:39:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:19 smithi137 ceph-mon[32568]: Deploying daemon mds.e on smithi137 2024-04-25T13:39:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:19 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6836/2786321466,v1:172.21.15.94:6837/2786321466] up:boot 2024-04-25T13:39:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:19 smithi137 ceph-mon[32568]: fsmap 4 up:standby 2024-04-25T13:39:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:39:21.126 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:20 smithi137 ceph-mon[32568]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:21.126 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:20 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3128935976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:21.126 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 podman[62867]: 2024-04-25 13:39:20.991773002 +0000 UTC m=+0.018702462 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:21.127 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 podman[62867]: 2024-04-25 13:39:21.126576438 +0000 UTC m=+0.153505917 container create 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:21.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:20 smithi094 ceph-mon[29195]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:21.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:20 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3128935976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:21.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:20 smithi138 ceph-mon[32977]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:21.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:20 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3128935976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:21.381 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 podman[62867]: 2024-04-25 13:39:21.346056038 +0000 UTC m=+0.372985480 container init 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T13:39:21.381 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 podman[62867]: 2024-04-25 13:39:21.348634502 +0000 UTC m=+0.375563942 container start 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T13:39:21.643 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 ceph-mds[62886]: -- 172.21.15.137:0/935248275 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fdf5055a40 con 0x55fdf509da80 2024-04-25T13:39:21.643 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: starting mds.e at 2024-04-25T13:39:21.643 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 bash[62867]: 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 2024-04-25T13:39:21.644 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 13:39:21 smithi137 systemd[1]: Started Ceph mds.e for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:22.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 -- 172.21.15.94:0/1681692130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 msgr2=0x7f074c1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 --2- 172.21.15.94:0/1681692130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c1069e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0734009900 tx=0x7f073402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 -- 172.21.15.94:0/1681692130 shutdown_connections 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 --2- 172.21.15.94:0/1681692130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c109880 0x7f074c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 --2- 172.21.15.94:0/1681692130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c106f20 0x7f074c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 --2- 172.21.15.94:0/1681692130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c1069e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 -- 172.21.15.94:0/1681692130 >> 172.21.15.94:0/1681692130 conn(0x7f074c0fdba0 msgr2=0x7f074c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 -- 172.21.15.94:0/1681692130 shutdown_connections 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.404+0000 7f0752a97640 1 -- 172.21.15.94:0/1681692130 wait complete. 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 Processor -- start 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 -- start start 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c106f20 0x7f074c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 0x7f074c1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f5d0 con 0x7f074c1045c0 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f450 con 0x7f074c109880 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0752a97640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f074c10f750 con 0x7f074c106f20 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46000/0 (socket says 172.21.15.94:46000) 2024-04-25T13:39:22.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 -- 172.21.15.94:0/355297921 learned_addr learned my addr 172.21.15.94:0/355297921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f074bfff640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c106f20 0x7f074c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0751034640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 0x7f074c1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 -- 172.21.15.94:0/355297921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c106f20 msgr2=0x7f074c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c106f20 0x7f074c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 -- 172.21.15.94:0/355297921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 msgr2=0x7f074c1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 0x7f074c1a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 -- 172.21.15.94:0/355297921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c1a3f80 con 0x7f074c1045c0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0751034640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 0x7f074c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.405+0000 7f0750833640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0734006010 tx=0x7f0734031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.406+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073403f030 con 0x7f074c1045c0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.406+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c1a4210 con 0x7f074c1045c0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.406+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0734003aa0 con 0x7f074c1045c0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.406+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734035070 con 0x7f074c1045c0 2024-04-25T13:39:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.406+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c1a4700 con 0x7f074c1045c0 2024-04-25T13:39:22.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.407+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c1a4a20 con 0x7f074c1045c0 2024-04-25T13:39:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.409+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0734042030 con 0x7f074c1045c0 2024-04-25T13:39:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.410+0000 7f0749ffb640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.410+0000 7f074bfff640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.410+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f07340ff180 con 0x7f074c1045c0 2024-04-25T13:39:22.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.410+0000 7f074bfff640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 0x7f07380b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f074c19d4e0 tx=0x7f073c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:22.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.414+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07340c8200 con 0x7f074c1045c0 2024-04-25T13:39:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.631+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f074c1a4bf0 con 0x7f074c1045c0 2024-04-25T13:39:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.632+0000 7f0749ffb640 1 -- 172.21.15.94:0/355297921 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f07340c7b00 con 0x7f074c1045c0 2024-04-25T13:39:22.632 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:22.632 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:39:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 msgr2=0x7f07380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 0x7f07380b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f074c19d4e0 tx=0x7f073c009680 comp rx=0 tx=0).stop 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 msgr2=0x7f074c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0734006010 tx=0x7f0734031a50 comp rx=0 tx=0).stop 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 shutdown_connections 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07380afaf0 0x7f07380b2000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f074c109880 0x7f074c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c106f20 0x7f074c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 --2- 172.21.15.94:0/355297921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f074c1045c0 0x7f074c19c440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 >> 172.21.15.94:0/355297921 conn(0x7f074c0fdba0 msgr2=0x7f074c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 shutdown_connections 2024-04-25T13:39:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:22.634+0000 7f0752a97640 1 -- 172.21.15.94:0/355297921 wait complete. 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:22.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: Deploying daemon mds.f on smithi138 2024-04-25T13:39:22.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:boot 2024-04-25T13:39:22.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: fsmap 5 up:standby 2024-04-25T13:39:22.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:39:22.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:22 smithi137 ceph-mon[32568]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: Deploying daemon mds.f on smithi138 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:boot 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: fsmap 5 up:standby 2024-04-25T13:39:22.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:39:22.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:22 smithi094 ceph-mon[29195]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:22.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: Deploying daemon mds.f on smithi138 2024-04-25T13:39:22.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:boot 2024-04-25T13:39:22.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: fsmap 5 up:standby 2024-04-25T13:39:22.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:39:22.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:22 smithi138 ceph-mon[32977]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:23 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/355297921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:23.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:23 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/355297921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:23.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:23 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/355297921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:24.497 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:39:24 smithi138 systemd[1]: Started Ceph mds.f for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:24.646 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:24.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 -- 172.21.15.94:0/2603507888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 msgr2=0x7f53d01069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 --2- 172.21.15.94:0/2603507888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d01069e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009ca0 tx=0x7f53cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 -- 172.21.15.94:0/2603507888 shutdown_connections 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 --2- 172.21.15.94:0/2603507888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0109880 0x7f53d010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 --2- 172.21.15.94:0/2603507888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0106f20 0x7f53d0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 --2- 172.21.15.94:0/2603507888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d01069e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 -- 172.21.15.94:0/2603507888 >> 172.21.15.94:0/2603507888 conn(0x7f53d00fdba0 msgr2=0x7f53d0100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 -- 172.21.15.94:0/2603507888 shutdown_connections 2024-04-25T13:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.774+0000 7f53d80f2640 1 -- 172.21.15.94:0/2603507888 wait complete. 2024-04-25T13:39:24.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 Processor -- start 2024-04-25T13:39:24.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 -- start start 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 0x7f53d019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0109880 0x7f53d01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010f5d0 con 0x7f53d01045c0 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010f450 con 0x7f53d0106f20 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d80f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010f750 con 0x7f53d0109880 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34606/0 (socket says 172.21.15.94:34606) 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 -- 172.21.15.94:0/4047698630 learned_addr learned my addr 172.21.15.94:0/4047698630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d668f640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0109880 0x7f53d01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d568d640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 0x7f53d019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 -- 172.21.15.94:0/4047698630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0109880 msgr2=0x7f53d01a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0109880 0x7f53d01a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 -- 172.21.15.94:0/4047698630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 msgr2=0x7f53d019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 0x7f53d019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d5e8e640 1 -- 172.21.15.94:0/4047698630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d01a1d40 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.775+0000 7f53d568d640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 0x7f53d019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53d5e8e640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53cc02ade0 tx=0x7f53cc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0317c0 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc0044f0 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01a1fd0 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc02fd10 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.776+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a2490 con 0x7f53d01045c0 2024-04-25T13:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.777+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d00ff6c0 con 0x7f53d01045c0 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.778+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f53cc004040 con 0x7f53d01045c0 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.779+0000 7f53c6ffd640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 0x7f53bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.779+0000 7f53d568d640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 0x7f53bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.779+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f53cc0ffe30 con 0x7f53d01045c0 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.779+0000 7f53d568d640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 0x7f53bc0b2080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53d019b2a0 tx=0x7f53c0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:24.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.784+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53cc0c8eb0 con 0x7f53d01045c0 2024-04-25T13:39:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.999+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d002dd70 con 0x7f53d01045c0 2024-04-25T13:39:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:24.999+0000 7f53c6ffd640 1 -- 172.21.15.94:0/4047698630 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f53cc0c87b0 con 0x7f53d01045c0 2024-04-25T13:39:24.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:25.000 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:39:25.000 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T13:39:25.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 msgr2=0x7f53bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:25.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 0x7f53bc0b2080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53d019b2a0 tx=0x7f53c0005e70 comp rx=0 tx=0).stop 2024-04-25T13:39:25.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 msgr2=0x7f53d019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:25.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53cc02ade0 tx=0x7f53cc003ba0 comp rx=0 tx=0).stop 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 shutdown_connections 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afb70 0x7f53bc0b2080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53d0109880 0x7f53d01a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53d0106f20 0x7f53d019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 --2- 172.21.15.94:0/4047698630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d01045c0 0x7f53d019a200 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 >> 172.21.15.94:0/4047698630 conn(0x7f53d00fdba0 msgr2=0x7f53d01051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 shutdown_connections 2024-04-25T13:39:25.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:25.002+0000 7f53d80f2640 1 -- 172.21.15.94:0/4047698630 wait complete. 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: Deploying daemon mds.c on smithi138 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/1621301649,v1:172.21.15.138:6835/1621301649] up:boot 2024-04-25T13:39:25.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: fsmap 6 up:standby 2024-04-25T13:39:25.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:39:25.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:25 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4047698630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: Deploying daemon mds.c on smithi138 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/1621301649,v1:172.21.15.138:6835/1621301649] up:boot 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: fsmap 6 up:standby 2024-04-25T13:39:25.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:39:25.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4047698630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: Deploying daemon mds.c on smithi138 2024-04-25T13:39:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/1621301649,v1:172.21.15.138:6835/1621301649] up:boot 2024-04-25T13:39:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: fsmap 6 up:standby 2024-04-25T13:39:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:39:25.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:25 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4047698630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:26.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:26 smithi138 ceph-mon[32977]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:26 smithi137 ceph-mon[32568]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:26 smithi094 ceph-mon[29195]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:26.824 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 13:39:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: starting mds.c at 2024-04-25T13:39:27.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 -- 172.21.15.94:0/3777704755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 msgr2=0x7fcfd4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/3777704755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd4101bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009900 tx=0x7fcfc402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 -- 172.21.15.94:0/3777704755 shutdown_connections 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/3777704755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 0x7fcfd410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/3777704755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd4101bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/3777704755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 0x7fcfd41054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 -- 172.21.15.94:0/3777704755 >> 172.21.15.94:0/3777704755 conn(0x7fcfd40fd370 msgr2=0x7fcfd40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:27.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 -- 172.21.15.94:0/3777704755 shutdown_connections 2024-04-25T13:39:27.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.136+0000 7fcfda6b6640 1 -- 172.21.15.94:0/3777704755 wait complete. 2024-04-25T13:39:27.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 Processor -- start 2024-04-25T13:39:27.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 -- start start 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 0x7fcfd419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 0x7fcfd41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410f630 con 0x7fcfd4101750 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410f4b0 con 0x7fcfd4102250 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfda6b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410f7b0 con 0x7fcfd41050a0 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34618/0 (socket says 172.21.15.94:34618) 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 -- 172.21.15.94:0/484051897 learned_addr learned my addr 172.21.15.94:0/484051897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd37fe640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 0x7fcfd419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:27.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd8c53640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 0x7fcfd41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 -- 172.21.15.94:0/484051897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 msgr2=0x7fcfd41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 0x7fcfd41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 -- 172.21.15.94:0/484051897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 msgr2=0x7fcfd419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 0x7fcfd419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.137+0000 7fcfd3fff640 1 -- 172.21.15.94:0/484051897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd41a1d50 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfd3fff640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9f0 tx=0x7fcfc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800ccc0 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc80035f0 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800ce60 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd41a1fe0 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.138+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41a2450 con 0x7fcfd4101750 2024-04-25T13:39:27.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.139+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd4108a30 con 0x7fcfd4101750 2024-04-25T13:39:27.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.140+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcfc8003aa0 con 0x7fcfd4101750 2024-04-25T13:39:27.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.141+0000 7fcfd17fa640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 0x7fcfbc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:27.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.141+0000 7fcfd37fe640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 0x7fcfbc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:27.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.141+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fcfc80deb30 con 0x7fcfd4101750 2024-04-25T13:39:27.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.142+0000 7fcfd37fe640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 0x7fcfbc0b2000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009230 tx=0x7fcfc4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:27.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.145+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfc80a7bb0 con 0x7fcfd4101750 2024-04-25T13:39:27.247 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 13:39:26 smithi138 bash[63624]: 0cf929995c85c4c43d0d3e726bcd6345626477426cfc11f7c77146a58cb73caa 2024-04-25T13:39:27.247 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 13:39:26 smithi138 systemd[1]: Started Ceph mds.c for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:27.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.360+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfd402db50 con 0x7fcfd4101750 2024-04-25T13:39:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.360+0000 7fcfd17fa640 1 -- 172.21.15.94:0/484051897 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fcfc80a74b0 con 0x7fcfd4101750 2024-04-25T13:39:27.360 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:27.360 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:39:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-04-25T13:39:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.362+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 msgr2=0x7fcfbc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.362+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 0x7fcfbc0b2000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009230 tx=0x7fcfc4031400 comp rx=0 tx=0).stop 2024-04-25T13:39:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.362+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 msgr2=0x7fcfd419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.362+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9f0 tx=0x7fcfc800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 shutdown_connections 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfbc0afaf0 0x7fcfbc0b2000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfd41050a0 0x7fcfd41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfd4102250 0x7fcfd419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 --2- 172.21.15.94:0/484051897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd4101750 0x7fcfd419a210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 >> 172.21.15.94:0/484051897 conn(0x7fcfd40fd370 msgr2=0x7fcfd4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 shutdown_connections 2024-04-25T13:39:27.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:27.363+0000 7fcfda6b6640 1 -- 172.21.15.94:0/484051897 wait complete. 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: Deploying daemon mds.h on smithi137 2024-04-25T13:39:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:boot 2024-04-25T13:39:28.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: fsmap 7 up:standby 2024-04-25T13:39:28.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:39:28.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:27 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/484051897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: Deploying daemon mds.h on smithi137 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:boot 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: fsmap 7 up:standby 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:39:28.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:27 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/484051897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: Deploying daemon mds.h on smithi137 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:boot 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: fsmap 7 up:standby 2024-04-25T13:39:28.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:39:28.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/484051897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:29.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:28 smithi137 ceph-mon[32568]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:29.143 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 systemd[1]: Starting Ceph mds.h for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:29.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:28 smithi094 ceph-mon[29195]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:29.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:28 smithi138 ceph-mon[32977]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:29.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:29.495 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 podman[63177]: 2024-04-25 13:39:29.128474888 +0000 UTC m=+0.017908790 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:29.495 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 podman[63177]: 2024-04-25 13:39:29.251509827 +0000 UTC m=+0.140943708 container create 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:39:29.495 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 podman[63177]: 2024-04-25 13:39:29.495558616 +0000 UTC m=+0.384992499 container init 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:29.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- 172.21.15.94:0/3571296103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 msgr2=0x7f55dc107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:29.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 --2- 172.21.15.94:0/3571296103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc107140 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009900 tx=0x7f55cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:39:29.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- 172.21.15.94:0/3571296103 shutdown_connections 2024-04-25T13:39:29.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 --2- 172.21.15.94:0/3571296103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc107680 0x7f55dc107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 --2- 172.21.15.94:0/3571296103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc107140 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 --2- 172.21.15.94:0/3571296103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc1023c0 0x7f55dc1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- 172.21.15.94:0/3571296103 >> 172.21.15.94:0/3571296103 conn(0x7f55dc0fdbf0 msgr2=0x7f55dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- 172.21.15.94:0/3571296103 shutdown_connections 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- 172.21.15.94:0/3571296103 wait complete. 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 Processor -- start 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.498+0000 7f55e3364640 1 -- start start 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc1023c0 0x7f55dc1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc107680 0x7f55dc1a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:29.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc115c00 con 0x7f55dc104d20 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc115a80 con 0x7f55dc107680 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc115d80 con 0x7f55dc1023c0 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34638/0 (socket says 172.21.15.94:34638) 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 -- 172.21.15.94:0/632145326 learned_addr learned my addr 172.21.15.94:0/632145326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e1901640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc107680 0x7f55dc1a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e1100640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc1023c0 0x7f55dc1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 -- 172.21.15.94:0/632145326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc1023c0 msgr2=0x7f55dc1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc1023c0 0x7f55dc1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 -- 172.21.15.94:0/632145326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc107680 msgr2=0x7f55dc1a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc107680 0x7f55dc1a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 -- 172.21.15.94:0/632145326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1a8340 con 0x7f55dc104d20 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e08ff640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f55cc006010 tx=0x7f55cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:29.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc03f030 con 0x7f55dc104d20 2024-04-25T13:39:29.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55cc003aa0 con 0x7f55dc104d20 2024-04-25T13:39:29.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1a85d0 con 0x7f55dc104d20 2024-04-25T13:39:29.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc035070 con 0x7f55dc104d20 2024-04-25T13:39:29.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.499+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a8ac0 con 0x7f55dc104d20 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.501+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc0ff710 con 0x7f55dc104d20 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.502+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f55cc042030 con 0x7f55dc104d20 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.503+0000 7f55d27fc640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 0x7f55c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.503+0000 7f55e1100640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 0x7f55c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.503+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f55cc0fefc0 con 0x7f55dc104d20 2024-04-25T13:39:29.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.503+0000 7f55e1100640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 0x7f55c80b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0757f0 tx=0x7f55d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:29.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.506+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55cc0c8040 con 0x7f55dc104d20 2024-04-25T13:39:29.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.728+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55dc02db50 con 0x7f55dc104d20 2024-04-25T13:39:29.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.729+0000 7f55d27fc640 1 -- 172.21.15.94:0/632145326 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f55cc0c7940 con 0x7f55dc104d20 2024-04-25T13:39:29.728 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:29.729 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:39:29.729 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 msgr2=0x7f55c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 0x7f55c80b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0757f0 tx=0x7f55d8005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 msgr2=0x7f55dc1a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f55cc006010 tx=0x7f55cc031a50 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 shutdown_connections 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afaf0 0x7f55c80b2000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55dc107680 0x7f55dc1a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55dc104d20 0x7f55dc1a0df0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 --2- 172.21.15.94:0/632145326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55dc1023c0 0x7f55dc1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 >> 172.21.15.94:0/632145326 conn(0x7f55dc0fdbf0 msgr2=0x7f55dc103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 shutdown_connections 2024-04-25T13:39:29.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:29.731+0000 7f55e3364640 1 -- 172.21.15.94:0/632145326 wait complete. 2024-04-25T13:39:29.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 podman[63177]: 2024-04-25 13:39:29.498435093 +0000 UTC m=+0.387868989 container start 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:39:29.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 ceph-mds[63196]: -- 172.21.15.137:0/2232127021 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56026356ba40 con 0x5602635b3600 2024-04-25T13:39:29.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: starting mds.h at 2024-04-25T13:39:29.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 bash[63177]: 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 2024-04-25T13:39:29.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 13:39:29 smithi137 systemd[1]: Started Ceph mds.h for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:30.850 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: Deploying daemon mds.i on smithi138 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/632145326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:boot 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: fsmap 8 up:standby 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:39:30.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:30 smithi138 ceph-mon[32977]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: Deploying daemon mds.i on smithi138 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/632145326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:boot 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: fsmap 8 up:standby 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:39:30.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:30 smithi137 ceph-mon[32568]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: Deploying daemon mds.i on smithi138 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/632145326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:boot 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: fsmap 8 up:standby 2024-04-25T13:39:30.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:39:30.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:30 smithi094 ceph-mon[29195]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:31.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:31.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 -- 172.21.15.94:0/2519293741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014108940 msgr2=0x7f301410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 --2- 172.21.15.94:0/2519293741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014108940 0x7f301410ad60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f300c009ca0 tx=0x7f300c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 -- 172.21.15.94:0/2519293741 shutdown_connections 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 --2- 172.21.15.94:0/2519293741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014108940 0x7f301410ad60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 --2- 172.21.15.94:0/2519293741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 0x7f3014104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 --2- 172.21.15.94:0/2519293741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014102a00 0x7f3014103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 -- 172.21.15.94:0/2519293741 >> 172.21.15.94:0/2519293741 conn(0x7f30140fdbf0 msgr2=0x7f30141000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 -- 172.21.15.94:0/2519293741 shutdown_connections 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.869+0000 7f301a8a8640 1 -- 172.21.15.94:0/2519293741 wait complete. 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.870+0000 7f301a8a8640 1 Processor -- start 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.870+0000 7f301a8a8640 1 -- start start 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 0x7f30141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014108940 0x7f30141a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f301410e530 con 0x7f3014102a00 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f301410e3b0 con 0x7f3014108940 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f301410e6b0 con 0x7f3014103c40 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34658/0 (socket says 172.21.15.94:34658) 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 -- 172.21.15.94:0/3447612391 learned_addr learned my addr 172.21.15.94:0/3447612391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f30137fe640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 0x7f30141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3018e45640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014108940 0x7f30141a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 -- 172.21.15.94:0/3447612391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 msgr2=0x7f30141995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 0x7f30141995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 -- 172.21.15.94:0/3447612391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014108940 msgr2=0x7f30141a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014108940 0x7f30141a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 -- 172.21.15.94:0/3447612391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30141a0be0 con 0x7f3014102a00 2024-04-25T13:39:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f3013fff640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3004009830 tx=0x7f3004009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004004020 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3014074e30 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3004007c50 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004003a10 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.871+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3014075380 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.872+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301406a190 con 0x7f3014102a00 2024-04-25T13:39:31.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.875+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3004003bb0 con 0x7f3014102a00 2024-04-25T13:39:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.875+0000 7f30117fa640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 0x7f2ffc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.876+0000 7f30137fe640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 0x7f2ffc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.876+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f30040e0500 con 0x7f3014102a00 2024-04-25T13:39:31.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.876+0000 7f30137fe640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 0x7f2ffc0b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f301419a140 tx=0x7f3008005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:31.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:31.880+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30040a94e0 con 0x7f3014102a00 2024-04-25T13:39:31.997 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:31 smithi138 systemd[1]: Starting Ceph mds.i for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:32.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.094+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30141071c0 con 0x7f3014102a00 2024-04-25T13:39:32.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.094+0000 7f30117fa640 1 -- 172.21.15.94:0/3447612391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f30040a8de0 con 0x7f3014102a00 2024-04-25T13:39:32.094 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:32.094 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:39:32.094 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 msgr2=0x7f2ffc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 0x7f2ffc0b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f301419a140 tx=0x7f3008005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 msgr2=0x7f30141990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3004009830 tx=0x7f3004009cf0 comp rx=0 tx=0).stop 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 shutdown_connections 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ffc0afaf0 0x7f2ffc0b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3014108940 0x7f30141a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3014103c40 0x7f30141995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 --2- 172.21.15.94:0/3447612391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3014102a00 0x7f30141990a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 >> 172.21.15.94:0/3447612391 conn(0x7f30140fdbf0 msgr2=0x7f3014104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 shutdown_connections 2024-04-25T13:39:32.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:32.096+0000 7f301a8a8640 1 -- 172.21.15.94:0/3447612391 wait complete. 2024-04-25T13:39:32.321 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:32 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3447612391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:32.322 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 podman[63935]: 2024-04-25 13:39:31.973515481 +0000 UTC m=+0.017924899 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:32.322 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 podman[63935]: 2024-04-25 13:39:32.092601045 +0000 UTC m=+0.137010508 container create a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T13:39:32.322 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 podman[63935]: 2024-04-25 13:39:32.320022793 +0000 UTC m=+0.364432212 container init a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:39:32.322 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 podman[63935]: 2024-04-25 13:39:32.322628977 +0000 UTC m=+0.367038398 container start a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T13:39:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:32 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3447612391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:32.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:32 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3447612391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:32.747 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 ceph-mds[63953]: -- 172.21.15.138:0/3640999009 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559212ec7a40 con 0x559212f0fa80 2024-04-25T13:39:32.747 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: starting mds.i at 2024-04-25T13:39:32.747 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 bash[63935]: a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd 2024-04-25T13:39:32.747 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 13:39:32 smithi138 systemd[1]: Started Ceph mds.i for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:33.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:33.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:33.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:33.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:33.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:33.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:34.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 -- 172.21.15.94:0/736730758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434106f10 msgr2=0x7fe434109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/736730758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434106f10 0x7fe434109330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe42402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 -- 172.21.15.94:0/736730758 shutdown_connections 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/736730758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434109870 0x7fe43410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/736730758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434106f10 0x7fe434109330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/736730758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 0x7fe4341069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 -- 172.21.15.94:0/736730758 >> 172.21.15.94:0/736730758 conn(0x7fe4340fdbd0 msgr2=0x7fe434100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 -- 172.21.15.94:0/736730758 shutdown_connections 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 -- 172.21.15.94:0/736730758 wait complete. 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.230+0000 7fe43a1d7640 1 Processor -- start 2024-04-25T13:39:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 -- start start 2024-04-25T13:39:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 0x7fe43419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434106f10 0x7fe43419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410f5f0 con 0x7fe434109870 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410f470 con 0x7fe434106f10 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe43a1d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410f770 con 0x7fe4341045b0 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34152/0 (socket says 172.21.15.94:34152) 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 -- 172.21.15.94:0/2606031180 learned_addr learned my addr 172.21.15.94:0/2606031180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe4337fe640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 0x7fe43419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe432ffd640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434106f10 0x7fe43419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 -- 172.21.15.94:0/2606031180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 msgr2=0x7fe43419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 0x7fe43419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 -- 172.21.15.94:0/2606031180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434106f10 msgr2=0x7fe43419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434106f10 0x7fe43419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 -- 172.21.15.94:0/2606031180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4341a1d70 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.231+0000 7fe433fff640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eb40 tx=0x7fe42c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.232+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c004b20 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.232+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c004cc0 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.232+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4341a2060 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.232+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c010720 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.232+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a2560 con 0x7fe434109870 2024-04-25T13:39:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.233+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4340ff6f0 con 0x7fe434109870 2024-04-25T13:39:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.234+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe42c005560 con 0x7fe434109870 2024-04-25T13:39:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.235+0000 7fe430ff9640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 0x7fe4200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.235+0000 7fe4337fe640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 0x7fe4200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.235+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe42c0defc0 con 0x7fe434109870 2024-04-25T13:39:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.235+0000 7fe4337fe640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 0x7fe4200b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0058c0 tx=0x7fe41c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:34.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.240+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe42c0a8040 con 0x7fe434109870 2024-04-25T13:39:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:34 smithi094 ceph-mon[29195]: Deploying daemon mds.g on smithi094 2024-04-25T13:39:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:34 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:boot 2024-04-25T13:39:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:34 smithi094 ceph-mon[29195]: fsmap 9 up:standby 2024-04-25T13:39:34.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:39:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:34 smithi138 ceph-mon[32977]: Deploying daemon mds.g on smithi094 2024-04-25T13:39:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:34 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:boot 2024-04-25T13:39:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:34 smithi138 ceph-mon[32977]: fsmap 9 up:standby 2024-04-25T13:39:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:39:34.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.516+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4340637a0 con 0x7fe434109870 2024-04-25T13:39:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.517+0000 7fe430ff9640 1 -- 172.21.15.94:0/2606031180 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7fe42c0a7940 con 0x7fe434109870 2024-04-25T13:39:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:34.518 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:39:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 10 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 msgr2=0x7fe4200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 0x7fe4200b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0058c0 tx=0x7fe41c00a250 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 msgr2=0x7fe4341a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eb40 tx=0x7fe42c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 shutdown_connections 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4200afaf0 0x7fe4200b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe434109870 0x7fe4341a1700 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe434106f10 0x7fe43419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 --2- 172.21.15.94:0/2606031180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4341045b0 0x7fe43419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 >> 172.21.15.94:0/2606031180 conn(0x7fe4340fdbd0 msgr2=0x7fe4341051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 shutdown_connections 2024-04-25T13:39:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:34.519+0000 7fe43a1d7640 1 -- 172.21.15.94:0/2606031180 wait complete. 2024-04-25T13:39:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:34 smithi137 ceph-mon[32568]: Deploying daemon mds.g on smithi094 2024-04-25T13:39:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:34 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:boot 2024-04-25T13:39:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:34 smithi137 ceph-mon[32568]: fsmap 9 up:standby 2024-04-25T13:39:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:39:34.930 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:34 smithi094 systemd[1]: Starting Ceph mds.g for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:35.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:35 smithi094 ceph-mon[29195]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2606031180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:34 smithi094 podman[70496]: 2024-04-25 13:39:34.93066811 +0000 UTC m=+0.113579650 container create 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:34 smithi094 podman[70496]: 2024-04-25 13:39:34.834978266 +0000 UTC m=+0.017889813 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 podman[70496]: 2024-04-25 13:39:35.133014908 +0000 UTC m=+0.315926450 container init 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 podman[70496]: 2024-04-25 13:39:35.135568433 +0000 UTC m=+0.318479973 container start 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 ceph-mds[70516]: -- 172.21.15.94:0/2204950250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e44e93a40 con 0x555e44edba80 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: starting mds.g at 2024-04-25T13:39:35.225 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 bash[70496]: 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 2024-04-25T13:39:35.475 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 13:39:35 smithi094 systemd[1]: Started Ceph mds.g for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:35.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:35 smithi138 ceph-mon[32977]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:35.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2606031180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:35.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:35 smithi137 ceph-mon[32568]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:35.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2606031180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6838/3883623467,v1:172.21.15.94:6839/3883623467] up:boot 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: fsmap 10 up:standby 2024-04-25T13:39:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:39:36.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6838/3883623467,v1:172.21.15.94:6839/3883623467] up:boot 2024-04-25T13:39:36.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: fsmap 10 up:standby 2024-04-25T13:39:36.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:39:36.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 -- 172.21.15.94:0/4093763393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40181041f0 msgr2=0x7f4018106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 --2- 172.21.15.94:0/4093763393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40181041f0 0x7f4018106610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f401402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 -- 172.21.15.94:0/4093763393 shutdown_connections 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 --2- 172.21.15.94:0/4093763393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018109750 0x7f401810bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 --2- 172.21.15.94:0/4093763393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018106be0 0x7f40181090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 --2- 172.21.15.94:0/4093763393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40181041f0 0x7f4018106610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 -- 172.21.15.94:0/4093763393 >> 172.21.15.94:0/4093763393 conn(0x7f40180fdbd0 msgr2=0x7f4018100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 -- 172.21.15.94:0/4093763393 shutdown_connections 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 -- 172.21.15.94:0/4093763393 wait complete. 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.647+0000 7f401f2fa640 1 Processor -- start 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 -- start start 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 0x7f401819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 0x7f401819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:36.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f401810f5e0 con 0x7f4018109750 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f401810f460 con 0x7f4018106be0 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401f2fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f401810f760 con 0x7f40181041f0 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34176/0 (socket says 172.21.15.94:34176) 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 -- 172.21.15.94:0/1671403171 learned_addr learned my addr 172.21.15.94:0/1671403171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401c895640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 0x7f401819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d096640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 0x7f401819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 -- 172.21.15.94:0/1671403171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 msgr2=0x7f401819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 0x7f401819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 -- 172.21.15.94:0/1671403171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 msgr2=0x7f401819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 0x7f401819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 -- 172.21.15.94:0/1671403171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40181a1c80 con 0x7f4018109750 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401c895640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 0x7f401819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d096640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 0x7f401819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f401d897640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4010009fd0 tx=0x7f401000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.648+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010017070 con 0x7f4018109750 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.649+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4010003e20 con 0x7f4018109750 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.649+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010010460 con 0x7f4018109750 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.649+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40181a1f70 con 0x7f4018109750 2024-04-25T13:39:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.649+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40181a2380 con 0x7f4018109750 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.650+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401806afd0 con 0x7f4018109750 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.651+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4010003970 con 0x7f4018109750 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.652+0000 7f400e7fc640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 0x7f40040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.652+0000 7f401d096640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 0x7f40040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.652+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f40100de5b0 con 0x7f4018109750 2024-04-25T13:39:36.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.652+0000 7f401d096640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 0x7f40040b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:36.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.656+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f401806afd0 con 0x7f4018109750 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6838/3883623467,v1:172.21.15.94:6839/3883623467] up:boot 2024-04-25T13:39:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: fsmap 10 up:standby 2024-04-25T13:39:36.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:39:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.879+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f401802dc10 con 0x7f4018109750 2024-04-25T13:39:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.880+0000 7f400e7fc640 1 -- 172.21.15.94:0/1671403171 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f40100a7630 con 0x7f4018109750 2024-04-25T13:39:36.881 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:39:36.881 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 msgr2=0x7f40040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 0x7f40040b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014005fd0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 msgr2=0x7f40181a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4010009fd0 tx=0x7f401000b260 comp rx=0 tx=0).stop 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 shutdown_connections 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40040afaf0 0x7f40040b2000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4018109750 0x7f40181a16c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4018106be0 0x7f401819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 --2- 172.21.15.94:0/1671403171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40181041f0 0x7f401819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 >> 172.21.15.94:0/1671403171 conn(0x7f40180fdbd0 msgr2=0x7f4018104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 shutdown_connections 2024-04-25T13:39:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:36.882+0000 7f401f2fa640 1 -- 172.21.15.94:0/1671403171 wait complete. 2024-04-25T13:39:37.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:37 smithi094 ceph-mon[29195]: Deploying daemon mds.a on smithi094 2024-04-25T13:39:37.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:37 smithi094 ceph-mon[29195]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:37.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:37 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1671403171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:37.475 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 systemd[1]: Starting Ceph mds.a for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:37 smithi137 ceph-mon[32568]: Deploying daemon mds.a on smithi094 2024-04-25T13:39:37.662 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:37 smithi137 ceph-mon[32568]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:37.662 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:37 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1671403171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:37.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:37 smithi138 ceph-mon[32977]: Deploying daemon mds.a on smithi094 2024-04-25T13:39:37.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:37 smithi138 ceph-mon[32977]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:37.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:37 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1671403171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:37.845 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 podman[70858]: 2024-04-25 13:39:37.486392482 +0000 UTC m=+0.019143002 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:37.846 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 podman[70858]: 2024-04-25 13:39:37.593471657 +0000 UTC m=+0.126222172 container create d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:37.846 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 podman[70858]: 2024-04-25 13:39:37.846310379 +0000 UTC m=+0.379060892 container init d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T13:39:38.225 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 podman[70858]: 2024-04-25 13:39:37.848879106 +0000 UTC m=+0.381629628 container start d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T13:39:38.225 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 ceph-mds[70876]: -- 172.21.15.94:0/2772558894 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b8bb8ba40 con 0x555b8bbd3600 2024-04-25T13:39:38.225 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: starting mds.a at 2024-04-25T13:39:38.225 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 bash[70858]: d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb 2024-04-25T13:39:38.225 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 13:39:37 smithi094 systemd[1]: Started Ceph mds.a for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:38.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 -- 172.21.15.94:0/1083827106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 msgr2=0x7fe804104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 --2- 172.21.15.94:0/1083827106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe804104000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009900 tx=0x7fe7f402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 -- 172.21.15.94:0/1083827106 shutdown_connections 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 --2- 172.21.15.94:0/1083827106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804104690 0x7fe80410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 --2- 172.21.15.94:0/1083827106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe804104000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 --2- 172.21.15.94:0/1083827106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe804102fd0 0x7fe8041033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 -- 172.21.15.94:0/1083827106 >> 172.21.15.94:0/1083827106 conn(0x7fe8040fd3b0 msgr2=0x7fe8040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 -- 172.21.15.94:0/1083827106 shutdown_connections 2024-04-25T13:39:39.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.015+0000 7fe808a70640 1 -- 172.21.15.94:0/1083827106 wait complete. 2024-04-25T13:39:39.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.016+0000 7fe808a70640 1 Processor -- start 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.016+0000 7fe808a70640 1 -- start start 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804102fd0 0x7fe804199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe80419f4b0 0x7fe8041a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe80410f860 con 0x7fe804103b30 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe80410f6e0 con 0x7fe804102fd0 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe808a70640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe80410f9e0 con 0x7fe80419f4b0 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34214/0 (socket says 172.21.15.94:34214) 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 -- 172.21.15.94:0/261197985 learned_addr learned my addr 172.21.15.94:0/261197985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe80259c640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804102fd0 0x7fe804199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe802d9d640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe80419f4b0 0x7fe8041a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 -- 172.21.15.94:0/261197985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe80419f4b0 msgr2=0x7fe8041a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe80419f4b0 0x7fe8041a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 -- 172.21.15.94:0/261197985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804102fd0 msgr2=0x7fe804199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804102fd0 0x7fe804199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 -- 172.21.15.94:0/261197985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8041a1f40 con 0x7fe804103b30 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.017+0000 7fe801d9b640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006010 tx=0x7fe7f4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.019+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f403f030 con 0x7fe804103b30 2024-04-25T13:39:39.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.019+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8041a21d0 con 0x7fe804103b30 2024-04-25T13:39:39.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.019+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f4003cb0 con 0x7fe804103b30 2024-04-25T13:39:39.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.019+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4035070 con 0x7fe804103b30 2024-04-25T13:39:39.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.019+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a26c0 con 0x7fe804103b30 2024-04-25T13:39:39.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.020+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80406b3c0 con 0x7fe804103b30 2024-04-25T13:39:39.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.022+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe7f4042030 con 0x7fe804103b30 2024-04-25T13:39:39.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.022+0000 7fe7fb7fe640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 0x7fe7f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:39.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.022+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe7f40fed50 con 0x7fe804103b30 2024-04-25T13:39:39.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.022+0000 7fe80259c640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 0x7fe7f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.023+0000 7fe80259c640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 0x7fe7f00b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe804103a30 tx=0x7fe7ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:39.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.027+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7f4102030 con 0x7fe804103b30 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: Deploying daemon mds.b on smithi137 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6840/2301018309,v1:172.21.15.94:6841/2301018309] up:boot 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: fsmap 11 up:standby 2024-04-25T13:39:39.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:39:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.244+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8041073e0 con 0x7fe804103b30 2024-04-25T13:39:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.245+0000 7fe7fb7fe640 1 -- 172.21.15.94:0/261197985 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7fe7f4003e60 con 0x7fe804103b30 2024-04-25T13:39:39.246 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:39.246 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:39:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:39.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: Deploying daemon mds.b on smithi137 2024-04-25T13:39:39.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:39.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6840/2301018309,v1:172.21.15.94:6841/2301018309] up:boot 2024-04-25T13:39:39.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: fsmap 11 up:standby 2024-04-25T13:39:39.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:39:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 msgr2=0x7fe7f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 0x7fe7f00b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe804103a30 tx=0x7fe7ec00b390 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 msgr2=0x7fe80419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006010 tx=0x7fe7f4004340 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 shutdown_connections 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7f00afaf0 0x7fe7f00b2000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe80419f4b0 0x7fe8041a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe804103b30 0x7fe80419a530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 --2- 172.21.15.94:0/261197985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe804102fd0 0x7fe804199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 >> 172.21.15.94:0/261197985 conn(0x7fe8040fd3b0 msgr2=0x7fe8040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 shutdown_connections 2024-04-25T13:39:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:39.247+0000 7fe808a70640 1 -- 172.21.15.94:0/261197985 wait complete. 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: Deploying daemon mds.b on smithi137 2024-04-25T13:39:39.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:39.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6840/2301018309,v1:172.21.15.94:6841/2301018309] up:boot 2024-04-25T13:39:39.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: fsmap 11 up:standby 2024-04-25T13:39:39.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:39:40.343 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:40 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/261197985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:40.343 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 systemd[1]: Starting Ceph mds.b for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T13:39:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:40 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/261197985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:40.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:40 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/261197985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:40.617 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 podman[63481]: 2024-04-25 13:39:40.247149159 +0000 UTC m=+0.018221306 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:39:40.617 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 podman[63481]: 2024-04-25 13:39:40.34993121 +0000 UTC m=+0.121003358 container create a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:39:40.617 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 podman[63481]: 2024-04-25 13:39:40.581552122 +0000 UTC m=+0.352624266 container init a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:39:40.617 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 podman[63481]: 2024-04-25 13:39:40.584155563 +0000 UTC m=+0.355227713 container start a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T13:39:40.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 ceph-mds[63500]: -- 172.21.15.137:0/3794457046 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c06120ba40 con 0x55c061253600 2024-04-25T13:39:40.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: starting mds.b at 2024-04-25T13:39:40.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 bash[63481]: a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 2024-04-25T13:39:40.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:40 smithi137 systemd[1]: Started Ceph mds.b for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T13:39:41.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:41.281 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.282 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 -- 172.21.15.94:0/929355665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa41029e0 msgr2=0x7f6fa41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/929355665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa41029e0 0x7f6fa41036e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009900 tx=0x7f6f9402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 -- 172.21.15.94:0/929355665 shutdown_connections 2024-04-25T13:39:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/929355665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4108920 0x7f6fa410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/929355665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 0x7f6fa4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/929355665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa41029e0 0x7f6fa41036e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 -- 172.21.15.94:0/929355665 >> 172.21.15.94:0/929355665 conn(0x7f6fa40fdbd0 msgr2=0x7f6fa4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 -- 172.21.15.94:0/929355665 shutdown_connections 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.370+0000 7f6fa9a51640 1 -- 172.21.15.94:0/929355665 wait complete. 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 Processor -- start 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 -- start start 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 0x7f6fa4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 0x7f6fa41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e4c0 con 0x7f6fa41029e0 2024-04-25T13:39:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e340 con 0x7f6fa4103c20 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa9a51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e640 con 0x7f6fa4108920 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34232/0 (socket says 172.21.15.94:34232) 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa2ffd640 1 -- 172.21.15.94:0/2951599930 learned_addr learned my addr 172.21.15.94:0/2951599930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 0x7f6fa41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.371+0000 7f6f9bfff640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 0x7f6fa4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 -- 172.21.15.94:0/2951599930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 msgr2=0x7f6fa4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 0x7f6fa4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 -- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 msgr2=0x7f6fa4199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 -- 172.21.15.94:0/2951599930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41a0c40 con 0x7f6fa4108920 2024-04-25T13:39:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa2ffd640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa37fe640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 0x7f6fa41a05d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ec50 tx=0x7f6f9c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c003b70 con 0x7f6fa4108920 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa41a0f30 con 0x7f6fa4108920 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c003d10 con 0x7f6fa4108920 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c010830 con 0x7f6fa4108920 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.372+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a1390 con 0x7f6fa4108920 2024-04-25T13:39:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.373+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4107730 con 0x7f6fa4108920 2024-04-25T13:39:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.375+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6f9c004b40 con 0x7f6fa4108920 2024-04-25T13:39:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.375+0000 7f6fa8a4f640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.375+0000 7f6fa2ffd640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.375+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6f9c0ded20 con 0x7f6fa4108920 2024-04-25T13:39:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.376+0000 7f6fa2ffd640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 0x7f6f900b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f94031500 tx=0x7f6f940046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.379+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f9c0a7e10 con 0x7f6fa4108920 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:41.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.611+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fa402db50 con 0x7f6fa4108920 2024-04-25T13:39:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.612+0000 7f6fa8a4f640 1 -- 172.21.15.94:0/2951599930 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f6f9c0a7710 con 0x7f6fa4108920 2024-04-25T13:39:41.613 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:41.613 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:39:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-04-25T13:39:41.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.614+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 msgr2=0x7f6f900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.614+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 0x7f6f900b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f94031500 tx=0x7f6f940046c0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.614+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 msgr2=0x7f6fa41a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.614+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 0x7f6fa41a05d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00ec50 tx=0x7f6f9c00c6a0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 shutdown_connections 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa4108920 0x7f6fa41a05d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa4103c20 0x7f6fa4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 --2- 172.21.15.94:0/2951599930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa41029e0 0x7f6fa4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 >> 172.21.15.94:0/2951599930 conn(0x7f6fa40fdbd0 msgr2=0x7f6fa4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 shutdown_connections 2024-04-25T13:39:41.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.615+0000 7f6fa9a51640 1 -- 172.21.15.94:0/2951599930 wait complete. 2024-04-25T13:39:41.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 -- 172.21.15.94:0/2717139967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694109a20 msgr2=0x7f269410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 --2- 172.21.15.94:0/2717139967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694109a20 0x7f269410be40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f268c009ca0 tx=0x7f268c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 -- 172.21.15.94:0/2717139967 shutdown_connections 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 --2- 172.21.15.94:0/2717139967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694109a20 0x7f269410be40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 --2- 172.21.15.94:0/2717139967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 0x7f2694105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 --2- 172.21.15.94:0/2717139967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694103ae0 0x7f26941047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 -- 172.21.15.94:0/2717139967 >> 172.21.15.94:0/2717139967 conn(0x7f26940fed10 msgr2=0x7f2694101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 -- 172.21.15.94:0/2717139967 shutdown_connections 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.777+0000 7f2699146640 1 -- 172.21.15.94:0/2717139967 wait complete. 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 Processor -- start 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 -- start start 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 0x7f2694076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 0x7f2694076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f269410f610 con 0x7f2694103ae0 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f269410f490 con 0x7f2694109a20 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2699146640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f269410f790 con 0x7f2694104d20 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34246/0 (socket says 172.21.15.94:34246) 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 -- 172.21.15.94:0/411491513 learned_addr learned my addr 172.21.15.94:0/411491513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f269259c640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 0x7f2694076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f269359e640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 0x7f2694076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 -- 172.21.15.94:0/411491513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 msgr2=0x7f2694076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 0x7f2694076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 -- 172.21.15.94:0/411491513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 msgr2=0x7f2694076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 0x7f2694076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f2692d9d640 1 -- 172.21.15.94:0/411491513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2694076c70 con 0x7f2694103ae0 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.778+0000 7f269359e640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 0x7f2694076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f2692d9d640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9f0 tx=0x7f268800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800ccc0 con 0x7f2694103ae0 2024-04-25T13:39:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26941a4270 con 0x7f2694103ae0 2024-04-25T13:39:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26880035f0 con 0x7f2694103ae0 2024-04-25T13:39:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800ce60 con 0x7f2694103ae0 2024-04-25T13:39:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.779+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26941a4730 con 0x7f2694103ae0 2024-04-25T13:39:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.780+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2694108830 con 0x7f2694103ae0 2024-04-25T13:39:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.781+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2688003aa0 con 0x7f2694103ae0 2024-04-25T13:39:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.782+0000 7f267bfff640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 0x7f267c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.782+0000 7f269259c640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 0x7f267c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.782+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f26880dec10 con 0x7f2694103ae0 2024-04-25T13:39:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.783+0000 7f269259c640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 0x7f267c0b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2694077300 tx=0x7f2684005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.786+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26880a7c90 con 0x7f2694103ae0 2024-04-25T13:39:41.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.971+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26940637a0 con 0x7f2694103ae0 2024-04-25T13:39:41.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.972+0000 7f267bfff640 1 -- 172.21.15.94:0/411491513 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18481 (secure 0 0 0) 0x7f26880a7590 con 0x7f2694103ae0 2024-04-25T13:39:41.973 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:41.973 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":75,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:38:55.378345+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 msgr2=0x7f267c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 0x7f267c0b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2694077300 tx=0x7f2684005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 msgr2=0x7f2694077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9f0 tx=0x7f268800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 shutdown_connections 2024-04-25T13:39:41.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f267c0afaf0 0x7f267c0b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2694109a20 0x7f2694076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2694104d20 0x7f2694076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 --2- 172.21.15.94:0/411491513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2694103ae0 0x7f2694077960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.974+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 >> 172.21.15.94:0/411491513 conn(0x7f26940fed10 msgr2=0x7f2694105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.975+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 shutdown_connections 2024-04-25T13:39:41.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:41.975+0000 7f2699146640 1 -- 172.21.15.94:0/411491513 wait complete. 2024-04-25T13:39:41.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:39:41.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:39:42.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 -- 172.21.15.94:0/3866013520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 msgr2=0x7fbd5c103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 --2- 172.21.15.94:0/3866013520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c103dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009900 tx=0x7fbd5002a410 comp rx=0 tx=0).stop 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 -- 172.21.15.94:0/3866013520 shutdown_connections 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 --2- 172.21.15.94:0/3866013520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 0x7fbd5c10aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 --2- 172.21.15.94:0/3866013520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c103dd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 --2- 172.21.15.94:0/3866013520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 -- 172.21.15.94:0/3866013520 >> 172.21.15.94:0/3866013520 conn(0x7fbd5c0fd340 msgr2=0x7fbd5c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 -- 172.21.15.94:0/3866013520 shutdown_connections 2024-04-25T13:39:42.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.101+0000 7fbd61138640 1 -- 172.21.15.94:0/3866013520 wait complete. 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 Processor -- start 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 -- start start 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 0x7fbd5c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10e530 con 0x7fbd5c103920 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10e3b0 con 0x7fbd5c102f40 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd61138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10e6b0 con 0x7fbd5c104460 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34272/0 (socket says 172.21.15.94:34272) 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd5a59c640 1 -- 172.21.15.94:0/692972933 learned_addr learned my addr 172.21.15.94:0/692972933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd5ad9d640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.102+0000 7fbd5b59e640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 0x7fbd5c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 -- 172.21.15.94:0/692972933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 msgr2=0x7fbd5c1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 0x7fbd5c1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 -- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 msgr2=0x7fbd5c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 -- 172.21.15.94:0/692972933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c1a1d00 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5a59c640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd5ad9d640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c102240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c069bf0 tx=0x7fbd4c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00cc40 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c1a1ff0 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd4c00cde0 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c018760 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.103+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1a2460 con 0x7fbd5c102f40 2024-04-25T13:39:42.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.104+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c0692e0 con 0x7fbd5c102f40 2024-04-25T13:39:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.106+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbd4c010a90 con 0x7fbd5c102f40 2024-04-25T13:39:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.107+0000 7fbd43fff640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 0x7fbd440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.107+0000 7fbd5a59c640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 0x7fbd440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.107+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbd4c0defa0 con 0x7fbd5c102f40 2024-04-25T13:39:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.107+0000 7fbd5a59c640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 0x7fbd440b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd50031440 tx=0x7fbd500314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.111+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd4c0a7930 con 0x7fbd5c102f40 2024-04-25T13:39:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.305+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd5c02dce0 con 0x7fbd5c102f40 2024-04-25T13:39:42.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.305+0000 7fbd43fff640 1 -- 172.21.15.94:0/692972933 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fbd4c0abc30 con 0x7fbd5c102f40 2024-04-25T13:39:42.306 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 msgr2=0x7fbd440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 0x7fbd440b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd50031440 tx=0x7fbd500314a0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 msgr2=0x7fbd5c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c102240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c069bf0 tx=0x7fbd4c00ee50 comp rx=0 tx=0).stop 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 shutdown_connections 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbd440afaf0 0x7fbd440b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c104460 0x7fbd5c1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd5c103920 0x7fbd5c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 --2- 172.21.15.94:0/692972933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd5c102f40 0x7fbd5c102240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 >> 172.21.15.94:0/692972933 conn(0x7fbd5c0fd340 msgr2=0x7fbd5c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.307+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 shutdown_connections 2024-04-25T13:39:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.308+0000 7fbd61138640 1 -- 172.21.15.94:0/692972933 wait complete. 2024-04-25T13:39:42.319 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:42 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:boot 2024-04-25T13:39:42.319 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:42 smithi094 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T13:39:42.319 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:39:42.319 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:42 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2951599930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.320 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:42 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/411491513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:39:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:42 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:boot 2024-04-25T13:39:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:42 smithi137 ceph-mon[32568]: fsmap 12 up:standby 2024-04-25T13:39:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:39:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:42 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2951599930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:42 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/411491513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 -- 172.21.15.94:0/3100878783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c104450 msgr2=0x7f9b0c10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 --2- 172.21.15.94:0/3100878783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c104450 0x7f9b0c10ab90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ca0 tx=0x7f9b0402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 -- 172.21.15.94:0/3100878783 shutdown_connections 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 --2- 172.21.15.94:0/3100878783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c104450 0x7f9b0c10ab90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 --2- 172.21.15.94:0/3100878783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c103950 0x7f9b0c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 --2- 172.21.15.94:0/3100878783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c102f70 0x7f9b0c103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 -- 172.21.15.94:0/3100878783 >> 172.21.15.94:0/3100878783 conn(0x7f9b0c0fd330 msgr2=0x7f9b0c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 -- 172.21.15.94:0/3100878783 shutdown_connections 2024-04-25T13:39:42.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.444+0000 7f9b11114640 1 -- 172.21.15.94:0/3100878783 wait complete. 2024-04-25T13:39:42.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.445+0000 7f9b11114640 1 Processor -- start 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.445+0000 7f9b11114640 1 -- start start 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.445+0000 7f9b11114640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 0x7f9b0c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.445+0000 7f9b11114640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.445+0000 7f9b11114640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c104450 0x7f9b0c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b11114640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e510 con 0x7f9b0c103950 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b11114640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e390 con 0x7f9b0c104450 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b11114640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10e690 con 0x7f9b0c102f70 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34302/0 (socket says 172.21.15.94:34302) 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 -- 172.21.15.94:0/3446279572 learned_addr learned my addr 172.21.15.94:0/3446279572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0ad9d640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 0x7f9b0c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0b59e640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c104450 0x7f9b0c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 -- 172.21.15.94:0/3446279572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 msgr2=0x7f9b0c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 0x7f9b0c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 -- 172.21.15.94:0/3446279572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c104450 msgr2=0x7f9b0c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c104450 0x7f9b0c1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 -- 172.21.15.94:0/3446279572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1a0c00 con 0x7f9b0c103950 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0ad9d640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 0x7f9b0c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.446+0000 7f9b0a59c640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e9f0 tx=0x7f9b0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.447+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000cd80 con 0x7f9b0c103950 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.447+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1a0ef0 con 0x7f9b0c103950 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.447+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1a1360 con 0x7f9b0c103950 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.447+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b000218f0 con 0x7f9b0c103950 2024-04-25T13:39:42.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.447+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00010900 con 0x7f9b0c103950 2024-04-25T13:39:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.448+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af80052c0 con 0x7f9b0c103950 2024-04-25T13:39:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.450+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9b00021440 con 0x7f9b0c103950 2024-04-25T13:39:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.450+0000 7f9af3fff640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 0x7f9af40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.451+0000 7f9b0ad9d640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 0x7f9af40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.451+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9b000de9b0 con 0x7f9b0c103950 2024-04-25T13:39:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.451+0000 7f9b0ad9d640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 0x7f9af40b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c069bb0 tx=0x7f9afc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.454+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b000a7ac0 con 0x7f9b0c103950 2024-04-25T13:39:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:42 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:boot 2024-04-25T13:39:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:42 smithi138 ceph-mon[32977]: fsmap 12 up:standby 2024-04-25T13:39:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:39:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:42 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2951599930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:42 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/411491513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.648+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9af8005880 con 0x7f9b0c103950 2024-04-25T13:39:42.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.649+0000 7f9af3fff640 1 -- 172.21.15.94:0/3446279572 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f9b000a73c0 con 0x7f9b0c103950 2024-04-25T13:39:42.649 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:42.649 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-04-25T13:39:42.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T13:39:42.650 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-04-25T13:39:42.650 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-04-25T13:39:42.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:39:42.650 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-04-25T13:39:42.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 msgr2=0x7f9af40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 0x7f9af40b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c069bb0 tx=0x7f9afc005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 msgr2=0x7f9b0c199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000e9f0 tx=0x7f9b0000edf0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 shutdown_connections 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9af40afaf0 0x7f9af40b2000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b0c104450 0x7f9b0c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b0c103950 0x7f9b0c199600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 --2- 172.21.15.94:0/3446279572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b0c102f70 0x7f9b0c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 >> 172.21.15.94:0/3446279572 conn(0x7f9b0c0fd330 msgr2=0x7f9b0c106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 shutdown_connections 2024-04-25T13:39:42.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.651+0000 7f9b11114640 1 -- 172.21.15.94:0/3446279572 wait complete. 2024-04-25T13:39:42.663 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:39:42.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:39:42.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.779+0000 7f0fe608c640 1 -- 172.21.15.94:0/667000771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe01045d0 msgr2=0x7f0fe01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 --2- 172.21.15.94:0/667000771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe01045d0 0x7f0fe01069f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009900 tx=0x7f0fc802a410 comp rx=0 tx=0).stop 2024-04-25T13:39:42.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 -- 172.21.15.94:0/667000771 shutdown_connections 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 --2- 172.21.15.94:0/667000771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0109890 0x7f0fe010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 --2- 172.21.15.94:0/667000771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0106f30 0x7f0fe0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 --2- 172.21.15.94:0/667000771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe01045d0 0x7f0fe01069f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 -- 172.21.15.94:0/667000771 >> 172.21.15.94:0/667000771 conn(0x7f0fe00fdbd0 msgr2=0x7f0fe0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 -- 172.21.15.94:0/667000771 shutdown_connections 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 -- 172.21.15.94:0/667000771 wait complete. 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 Processor -- start 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.780+0000 7f0fe608c640 1 -- start start 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 0x7f0fe019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0109890 0x7f0fe01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f5c0 con 0x7f0fe0106f30 2024-04-25T13:39:42.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f440 con 0x7f0fe0109890 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fe608c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f740 con 0x7f0fe01045d0 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34314/0 (socket says 172.21.15.94:34314) 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 0x7f0fe019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 -- 172.21.15.94:0/4218903277 learned_addr learned my addr 172.21.15.94:0/4218903277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdffff640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0109890 0x7f0fe01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 -- 172.21.15.94:0/4218903277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 msgr2=0x7f0fe019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 0x7f0fe019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 -- 172.21.15.94:0/4218903277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0109890 msgr2=0x7f0fe01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0109890 0x7f0fe01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 -- 172.21.15.94:0/4218903277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe01a2030 con 0x7f0fe0106f30 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdf7fe640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 0x7f0fe019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdeffd640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d950 tx=0x7f0fd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.781+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0017070 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.782+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fd0009c30 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.782+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000f620 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.782+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe01a2320 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.782+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe01a28c0 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.783+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe006c2a0 con 0x7f0fe0106f30 2024-04-25T13:39:42.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.785+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0fd0010460 con 0x7f0fe0106f30 2024-04-25T13:39:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.786+0000 7f0fdcff9640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 0x7f0fcc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.786+0000 7f0fdf7fe640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 0x7f0fcc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:42.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.786+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0fd00de230 con 0x7f0fe0106f30 2024-04-25T13:39:42.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.786+0000 7f0fdf7fe640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 0x7f0fcc0b2080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009230 tx=0x7f0fc8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:42.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.790+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fd0009de0 con 0x7f0fe0106f30 2024-04-25T13:39:42.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:42.979+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0fe01a2b70 con 0x7f0fe0106f30 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/692972933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3446279572' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:43.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/692972933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3446279572' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:39:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:43.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:39:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/692972933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:39:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3446279572' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:39:43.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:39:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.598+0000 7f0fdcff9640 1 -- 172.21.15.94:0/4218903277 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f0fd00a6ce0 con 0x7f0fe0106f30 2024-04-25T13:39:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-04-25T13:39:43.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 msgr2=0x7f0fcc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:43.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 0x7f0fcc0b2080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009230 tx=0x7f0fc8031400 comp rx=0 tx=0).stop 2024-04-25T13:39:43.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 msgr2=0x7f0fe019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:43.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d950 tx=0x7f0fd000de10 comp rx=0 tx=0).stop 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 shutdown_connections 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0fcc0afb70 0x7f0fcc0b2080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe0109890 0x7f0fe01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fe0106f30 0x7f0fe019aae0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 --2- 172.21.15.94:0/4218903277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe01045d0 0x7f0fe019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 >> 172.21.15.94:0/4218903277 conn(0x7f0fe00fdbd0 msgr2=0x7f0fe0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 shutdown_connections 2024-04-25T13:39:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.601+0000 7f0fe608c640 1 -- 172.21.15.94:0/4218903277 wait complete. 2024-04-25T13:39:43.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 -- 172.21.15.94:0/1252626610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c108900 msgr2=0x7fbc3c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 --2- 172.21.15.94:0/1252626610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c108900 0x7fbc3c10ad20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009900 tx=0x7fbc3402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 -- 172.21.15.94:0/1252626610 shutdown_connections 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 --2- 172.21.15.94:0/1252626610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c108900 0x7fbc3c10ad20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 --2- 172.21.15.94:0/1252626610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c103c00 0x7fbc3c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 --2- 172.21.15.94:0/1252626610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 -- 172.21.15.94:0/1252626610 >> 172.21.15.94:0/1252626610 conn(0x7fbc3c0fdbd0 msgr2=0x7fbc3c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 -- 172.21.15.94:0/1252626610 shutdown_connections 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 -- 172.21.15.94:0/1252626610 wait complete. 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.736+0000 7fbc414ee640 1 Processor -- start 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 -- start start 2024-04-25T13:39:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c108900 0x7fbc3c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10e4f0 con 0x7fbc3c103c00 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10e370 con 0x7fbc3c1029c0 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc414ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10e670 con 0x7fbc3c108900 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42646/0 (socket says 172.21.15.94:42646) 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 -- 172.21.15.94:0/3089529601 learned_addr learned my addr 172.21.15.94:0/3089529601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3affd640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3b7fe640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c108900 0x7fbc3c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 -- 172.21.15.94:0/3089529601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c108900 msgr2=0x7fbc3c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c108900 0x7fbc3c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 -- 172.21.15.94:0/3089529601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 msgr2=0x7fbc3c06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:43.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 -- 172.21.15.94:0/3089529601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c1a1cb0 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc3a7fc640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d770 tx=0x7fbc3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.737+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30017070 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.738+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc300054d0 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.738+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c1a1fa0 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.738+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004c30 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.738+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a2410 con 0x7fbc3c103c00 2024-04-25T13:39:43.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.739+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c107710 con 0x7fbc3c103c00 2024-04-25T13:39:43.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.740+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbc30004dd0 con 0x7fbc3c103c00 2024-04-25T13:39:43.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.741+0000 7fbc23fff640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 0x7fbc240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:43.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.741+0000 7fbc3affd640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 0x7fbc240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:43.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.741+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fbc300de730 con 0x7fbc3c103c00 2024-04-25T13:39:43.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.741+0000 7fbc3affd640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 0x7fbc240b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c1035c0 tx=0x7fbc2c009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.745+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc300a6fd0 con 0x7fbc3c103c00 2024-04-25T13:39:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:43.929+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbc3c02dd10 con 0x7fbc3c103c00 2024-04-25T13:39:44.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.604+0000 7fbc23fff640 1 -- 172.21.15.94:0/3089529601 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fbc3c02dd10 con 0x7fbc3c103c00 2024-04-25T13:39:44.604 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-04-25T13:39:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 msgr2=0x7fbc240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 0x7fbc240b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c1035c0 tx=0x7fbc2c009670 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 msgr2=0x7fbc3c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d770 tx=0x7fbc3000dc30 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 shutdown_connections 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc240afaf0 0x7fbc240b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc3c108900 0x7fbc3c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc3c103c00 0x7fbc3c19ef40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 --2- 172.21.15.94:0/3089529601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 >> 172.21.15.94:0/3089529601 conn(0x7fbc3c0fdbd0 msgr2=0x7fbc3c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 shutdown_connections 2024-04-25T13:39:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.607+0000 7fbc414ee640 1 -- 172.21.15.94:0/3089529601 wait complete. 2024-04-25T13:39:44.626 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 -- 172.21.15.94:0/2678879705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828103c40 msgr2=0x7f0828104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 --2- 172.21.15.94:0/2678879705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828103c40 0x7f0828104050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0818009900 tx=0x7f081802a410 comp rx=0 tx=0).stop 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 -- 172.21.15.94:0/2678879705 shutdown_connections 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 --2- 172.21.15.94:0/2678879705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828108940 0x7f082810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 --2- 172.21.15.94:0/2678879705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828103c40 0x7f0828104050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 --2- 172.21.15.94:0/2678879705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828102a00 0x7f0828103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 -- 172.21.15.94:0/2678879705 >> 172.21.15.94:0/2678879705 conn(0x7f08280fdbf0 msgr2=0x7f08281000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 -- 172.21.15.94:0/2678879705 shutdown_connections 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.738+0000 7f082f70d640 1 -- 172.21.15.94:0/2678879705 wait complete. 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 Processor -- start 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 -- start start 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828102a00 0x7f082819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 0x7f082819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f082810e520 con 0x7f0828108940 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f082810e3a0 con 0x7f0828102a00 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082f70d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f082810e6a0 con 0x7f0828103c40 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082dcaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:44.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082dcaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42664/0 (socket says 172.21.15.94:42664) 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082dcaa640 1 -- 172.21.15.94:0/2281072908 learned_addr learned my addr 172.21.15.94:0/2281072908 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082d4a9640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828102a00 0x7f082819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.739+0000 7f082cca8640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 0x7f082819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 -- 172.21.15.94:0/2281072908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 msgr2=0x7f082819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 0x7f082819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 -- 172.21.15.94:0/2281072908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828102a00 msgr2=0x7f082819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828102a00 0x7f082819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 -- 172.21.15.94:0/2281072908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08281a1d60 con 0x7f0828108940 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082cca8640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 0x7f082819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082dcaa640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f082000eaa0 tx=0x7f082000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082000cdb0 con 0x7f0828108940 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0820005020 con 0x7f0828108940 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200116d0 con 0x7f0828108940 2024-04-25T13:39:44.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08281a1ff0 con 0x7f0828108940 2024-04-25T13:39:44.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.740+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281a2460 con 0x7f0828108940 2024-04-25T13:39:44.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.741+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08281a2720 con 0x7f0828108940 2024-04-25T13:39:44.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.742+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f08200051d0 con 0x7f0828108940 2024-04-25T13:39:44.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.743+0000 7f081e7fc640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 0x7f08140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:44.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.743+0000 7f082d4a9640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 0x7f08140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:44.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.743+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f08200deff0 con 0x7f0828108940 2024-04-25T13:39:44.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.744+0000 7f082d4a9640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 0x7f08140b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0828103600 tx=0x7f0824005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.747+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0820016030 con 0x7f0828108940 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: pgmap v278: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:44.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:44.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:44 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:44.964+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f08281a2b20 con 0x7f0828108940 2024-04-25T13:39:44.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: pgmap v278: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:44.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:44 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4218903277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: pgmap v278: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:44.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:39:44.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:39:44.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:44 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:45.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.605+0000 7f081e7fc640 1 -- 172.21.15.94:0/2281072908 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f08200a7d80 con 0x7f0828108940 2024-04-25T13:39:45.606 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:39:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 msgr2=0x7f08140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 0x7f08140b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0828103600 tx=0x7f0824005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 msgr2=0x7f08281a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f082000eaa0 tx=0x7f082000ef60 comp rx=0 tx=0).stop 2024-04-25T13:39:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 shutdown_connections 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08140afaf0 0x7f08140b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0828108940 0x7f08281a16f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0828103c40 0x7f082819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 --2- 172.21.15.94:0/2281072908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0828102a00 0x7f082819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 >> 172.21.15.94:0/2281072908 conn(0x7f08280fdbf0 msgr2=0x7f0828109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 shutdown_connections 2024-04-25T13:39:45.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.608+0000 7f082f70d640 1 -- 172.21.15.94:0/2281072908 wait complete. 2024-04-25T13:39:45.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:39:45.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 -- 172.21.15.94:0/3297870986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88103c00 msgr2=0x7fad88104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 --2- 172.21.15.94:0/3297870986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88103c00 0x7fad88104010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009900 tx=0x7fad7c02a410 comp rx=0 tx=0).stop 2024-04-25T13:39:45.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 -- 172.21.15.94:0/3297870986 shutdown_connections 2024-04-25T13:39:45.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 --2- 172.21.15.94:0/3297870986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad88108900 0x7fad8810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 --2- 172.21.15.94:0/3297870986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88103c00 0x7fad88104010 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 --2- 172.21.15.94:0/3297870986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad881029c0 0x7fad881036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 -- 172.21.15.94:0/3297870986 >> 172.21.15.94:0/3297870986 conn(0x7fad880fdbd0 msgr2=0x7fad88100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 -- 172.21.15.94:0/3297870986 shutdown_connections 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.733+0000 7fad8e53f640 1 -- 172.21.15.94:0/3297870986 wait complete. 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 Processor -- start 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 -- start start 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad881029c0 0x7fad881990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 0x7fad88199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810e4f0 con 0x7fad88108900 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810e370 con 0x7fad88103c00 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8e53f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810e670 con 0x7fad881029c0 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8cadc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:45.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad87fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad881029c0 0x7fad881990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8cadc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42684/0 (socket says 172.21.15.94:42684) 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad8cadc640 1 -- 172.21.15.94:0/1336858760 learned_addr learned my addr 172.21.15.94:0/1336858760 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.734+0000 7fad877fe640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 0x7fad88199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 -- 172.21.15.94:0/1336858760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad881029c0 msgr2=0x7fad881990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad881029c0 0x7fad881990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 -- 172.21.15.94:0/1336858760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 msgr2=0x7fad88199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 0x7fad88199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 -- 172.21.15.94:0/1336858760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad881a0c30 con 0x7fad88108900 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad877fe640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 0x7fad88199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8cadc640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad8000cad0 tx=0x7fad8000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8000ae20 con 0x7fad88108900 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad80005020 con 0x7fad88108900 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8000f6b0 con 0x7fad88108900 2024-04-25T13:39:45.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad881a0f20 con 0x7fad88108900 2024-04-25T13:39:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.735+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad881a13e0 con 0x7fad88108900 2024-04-25T13:39:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.736+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad88107710 con 0x7fad88108900 2024-04-25T13:39:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.737+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fad800051d0 con 0x7fad88108900 2024-04-25T13:39:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.738+0000 7fad857fa640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 0x7fad700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:45.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.738+0000 7fad87fff640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 0x7fad700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:45.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.738+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fad800dee50 con 0x7fad88108900 2024-04-25T13:39:45.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.738+0000 7fad87fff640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 0x7fad700b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fad881035c0 tx=0x7fad7800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.742+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad80014030 con 0x7fad88108900 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:45.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:45 smithi137 ceph-mon[32568]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T13:39:45.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.920+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad8802dda0 con 0x7fad88108900 2024-04-25T13:39:45.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.921+0000 7fad857fa640 1 -- 172.21.15.94:0/1336858760 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22257 (secure 0 0 0) 0x7fad800a7c00 con 0x7fad88108900 2024-04-25T13:39:45.922 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:45.922 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":78,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:39:44.966151+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 msgr2=0x7fad700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 0x7fad700b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fad881035c0 tx=0x7fad7800b390 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 msgr2=0x7fad881a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad8000cad0 tx=0x7fad8000cf90 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 shutdown_connections 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad700afb70 0x7fad700b2080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad88108900 0x7fad881a05c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad88103c00 0x7fad88199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 --2- 172.21.15.94:0/1336858760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad881029c0 0x7fad881990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:45.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 >> 172.21.15.94:0/1336858760 conn(0x7fad880fdbd0 msgr2=0x7fad881048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 shutdown_connections 2024-04-25T13:39:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:45.923+0000 7fad8e53f640 1 -- 172.21.15.94:0/1336858760 wait complete. 2024-04-25T13:39:45.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:39:45.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:39:45.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:39:45.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:39:45.934 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:39:45.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:45.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:45 smithi094 ceph-mon[29195]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T13:39:45.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3089529601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:39:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:45 smithi138 ceph-mon[32977]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 -- 172.21.15.94:0/3805012885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64109a40 msgr2=0x7f1d6410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 --2- 172.21.15.94:0/3805012885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64109a40 0x7f1d6410be60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 -- 172.21.15.94:0/3805012885 shutdown_connections 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 --2- 172.21.15.94:0/3805012885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64109a40 0x7f1d6410be60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 --2- 172.21.15.94:0/3805012885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64104d40 0x7f1d64105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 --2- 172.21.15.94:0/3805012885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64103b00 0x7f1d64104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 -- 172.21.15.94:0/3805012885 >> 172.21.15.94:0/3805012885 conn(0x7f1d640fed10 msgr2=0x7f1d641011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 -- 172.21.15.94:0/3805012885 shutdown_connections 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.044+0000 7f1d6c596640 1 -- 172.21.15.94:0/3805012885 wait complete. 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 Processor -- start 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 -- start start 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 0x7f1d6419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64109a40 0x7f1d641a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410f660 con 0x7f1d64104d40 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410f4e0 con 0x7f1d64103b00 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6c596640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410f7e0 con 0x7f1d64109a40 2024-04-25T13:39:46.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42696/0 (socket says 172.21.15.94:42696) 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 -- 172.21.15.94:0/1320744043 learned_addr learned my addr 172.21.15.94:0/1320744043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6a332640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 0x7f1d6419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6ab33640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64109a40 0x7f1d641a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 -- 172.21.15.94:0/1320744043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64109a40 msgr2=0x7f1d641a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64109a40 0x7f1d641a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 -- 172.21.15.94:0/1320744043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 msgr2=0x7f1d6419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 0x7f1d6419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d69b31640 1 -- 172.21.15.94:0/1320744043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a1d40 con 0x7f1d64104d40 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.045+0000 7f1d6a332640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 0x7f1d6419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d69b31640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400d950 tx=0x7f1d5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54017070 con 0x7f1d64104d40 2024-04-25T13:39:46.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54009c30 con 0x7f1d64104d40 2024-04-25T13:39:46.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a1fd0 con 0x7f1d64104d40 2024-04-25T13:39:46.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400f620 con 0x7f1d64104d40 2024-04-25T13:39:46.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.046+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a2510 con 0x7f1d64104d40 2024-04-25T13:39:46.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.047+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d64108850 con 0x7f1d64104d40 2024-04-25T13:39:46.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.048+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1d54010460 con 0x7f1d64104d40 2024-04-25T13:39:46.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.049+0000 7f1d5b7fe640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 0x7f1d500b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.049+0000 7f1d6a332640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 0x7f1d500b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.049+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1d540de6e0 con 0x7f1d64104d40 2024-04-25T13:39:46.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.049+0000 7f1d6a332640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 0x7f1d500b2080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d64104700 tx=0x7f1d6000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.053+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d540a7490 con 0x7f1d64104d40 2024-04-25T13:39:46.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.232+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d640637a0 con 0x7f1d64104d40 2024-04-25T13:39:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.233+0000 7f1d5b7fe640 1 -- 172.21.15.94:0/1320744043 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22257 (secure 0 0 0) 0x7f1d54009de0 con 0x7f1d64104d40 2024-04-25T13:39:46.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":78,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:39:44.966151+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 msgr2=0x7f1d500b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 0x7f1d500b2080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d64104700 tx=0x7f1d6000a560 comp rx=0 tx=0).stop 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 msgr2=0x7f1d6419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400d950 tx=0x7f1d5400de10 comp rx=0 tx=0).stop 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 shutdown_connections 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d500afb70 0x7f1d500b2080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d64109a40 0x7f1d641a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d64104d40 0x7f1d6419a740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 --2- 172.21.15.94:0/1320744043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d64103b00 0x7f1d6419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.235+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 >> 172.21.15.94:0/1320744043 conn(0x7f1d640fed10 msgr2=0x7f1d64105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.236+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 shutdown_connections 2024-04-25T13:39:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.236+0000 7f1d6c596640 1 -- 172.21.15.94:0/1320744043 wait complete. 2024-04-25T13:39:46.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:39:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- 172.21.15.94:0/3930347973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 msgr2=0x7f51c810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3930347973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c810be50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009900 tx=0x7f51c002a410 comp rx=0 tx=0).stop 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- 172.21.15.94:0/3930347973 shutdown_connections 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3930347973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c810be50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3930347973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3930347973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 0x7f51c81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- 172.21.15.94:0/3930347973 >> 172.21.15.94:0/3930347973 conn(0x7f51c80fece0 msgr2=0x7f51c8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- 172.21.15.94:0/3930347973 shutdown_connections 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- 172.21.15.94:0/3930347973 wait complete. 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 Processor -- start 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.359+0000 7f51d02f2640 1 -- start start 2024-04-25T13:39:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 0x7f51c819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51c810f600 con 0x7f51c8109a30 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51c810f480 con 0x7f51c8104d30 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51c810f780 con 0x7f51c8103af0 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42716/0 (socket says 172.21.15.94:42716) 2024-04-25T13:39:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 -- 172.21.15.94:0/3138928964 learned_addr learned my addr 172.21.15.94:0/3138928964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51cd88d640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce08e640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 0x7f51c819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 -- 172.21.15.94:0/3138928964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 msgr2=0x7f51c819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 0x7f51c819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 -- 172.21.15.94:0/3138928964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 msgr2=0x7f51c819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 -- 172.21.15.94:0/3138928964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81a1d20 con 0x7f51c8109a30 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51cd88d640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51ce88f640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009230 tx=0x7f51c0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c003f030 con 0x7f51c8109a30 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c0003aa0 con 0x7f51c8109a30 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0035070 con 0x7f51c8109a30 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81a1fb0 con 0x7f51c8109a30 2024-04-25T13:39:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.360+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81a2410 con 0x7f51c8109a30 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.361+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c8108840 con 0x7f51c8109a30 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.363+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f51c0042030 con 0x7f51c8109a30 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.363+0000 7f51bf7fe640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 0x7f51b40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.363+0000 7f51ce08e640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 0x7f51b40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.363+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f51c00ff430 con 0x7f51c8109a30 2024-04-25T13:39:46.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.364+0000 7f51ce08e640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 0x7f51b40b2080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51c81046f0 tx=0x7f51c4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.368+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51c00c8160 con 0x7f51c8109a30 2024-04-25T13:39:46.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.585+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51c802db50 con 0x7f51c8109a30 2024-04-25T13:39:46.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.612+0000 7f51bf7fe640 1 -- 172.21.15.94:0/3138928964 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f51c00c7a60 con 0x7f51c8109a30 2024-04-25T13:39:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 msgr2=0x7f51b40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 0x7f51b40b2080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f51c81046f0 tx=0x7f51c4009870 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 msgr2=0x7f51c81a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009230 tx=0x7f51c0031a50 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 shutdown_connections 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51b40afb70 0x7f51b40b2080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51c8109a30 0x7f51c81a16b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d30 0x7f51c819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 --2- 172.21.15.94:0/3138928964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8103af0 0x7f51c819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 >> 172.21.15.94:0/3138928964 conn(0x7f51c80fece0 msgr2=0x7f51c8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 shutdown_connections 2024-04-25T13:39:46.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.614+0000 7f51d02f2640 1 -- 172.21.15.94:0/3138928964 wait complete. 2024-04-25T13:39:46.626 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:39:46.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T13:39:46.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.738+0000 7fcc1757a640 1 -- 172.21.15.94:0/1098394124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc10108920 msgr2=0x7fcc1010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.738+0000 7fcc1757a640 1 --2- 172.21.15.94:0/1098394124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc10108920 0x7fcc1010ad40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009ca0 tx=0x7fcc0802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- 172.21.15.94:0/1098394124 shutdown_connections 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- 172.21.15.94:0/1098394124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc10108920 0x7fcc1010ad40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- 172.21.15.94:0/1098394124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 0x7fcc10104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- 172.21.15.94:0/1098394124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc101029e0 0x7fcc101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- 172.21.15.94:0/1098394124 >> 172.21.15.94:0/1098394124 conn(0x7fcc100fdbd0 msgr2=0x7fcc10100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- 172.21.15.94:0/1098394124 shutdown_connections 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- 172.21.15.94:0/1098394124 wait complete. 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 Processor -- start 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- start start 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 0x7fcc1019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc10108920 0x7fcc101a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010e510 con 0x7fcc101029e0 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010e390 con 0x7fcc10103c20 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.739+0000 7fcc1757a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1010e690 con 0x7fcc10108920 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42738/0 (socket says 172.21.15.94:42738) 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 -- 172.21.15.94:0/3067721941 learned_addr learned my addr 172.21.15.94:0/3067721941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc14b15640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 0x7fcc1019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15b17640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc10108920 0x7fcc101a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 -- 172.21.15.94:0/3067721941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc10108920 msgr2=0x7fcc101a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc10108920 0x7fcc101a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 -- 172.21.15.94:0/3067721941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 msgr2=0x7fcc1019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 0x7fcc1019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:46.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 -- 172.21.15.94:0/3067721941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc101a2060 con 0x7fcc101029e0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc15316640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c970 tx=0x7fcc0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c004220 con 0x7fcc101029e0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc0c0043c0 con 0x7fcc101029e0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc101a2350 con 0x7fcc101029e0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c004a00 con 0x7fcc101029e0 2024-04-25T13:39:46.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.740+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc101a2850 con 0x7fcc101029e0 2024-04-25T13:39:46.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.741+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1006c2a0 con 0x7fcc101029e0 2024-04-25T13:39:46.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.743+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcc0c0079b0 con 0x7fcc101029e0 2024-04-25T13:39:46.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.744+0000 7fcc067fc640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 0x7fcbfc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:46.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.744+0000 7fcc14b15640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 0x7fcbfc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:46.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.744+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcc0c0dec20 con 0x7fcc101029e0 2024-04-25T13:39:46.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.744+0000 7fcc14b15640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 0x7fcbfc0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc000058c0 tx=0x7fcc0000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:46.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.748+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc0c014030 con 0x7fcc101029e0 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1336858760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1320744043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: pgmap v282: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:46.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:39:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:46.964+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fcc101a2b10 con 0x7fcc101029e0 2024-04-25T13:39:46.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1336858760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1320744043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: pgmap v282: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:46.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2281072908' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1336858760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1320744043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: pgmap v282: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:46.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:39:47.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.644+0000 7fcc067fc640 1 -- 172.21.15.94:0/3067721941 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7fcc0c0a72e0 con 0x7fcc101029e0 2024-04-25T13:39:47.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 msgr2=0x7fcbfc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:47.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 0x7fcbfc0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc000058c0 tx=0x7fcc0000a2d0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 msgr2=0x7fcc1019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c970 tx=0x7fcc0c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 shutdown_connections 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbfc0afaf0 0x7fcbfc0b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc10108920 0x7fcc101a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc10103c20 0x7fcc1019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 --2- 172.21.15.94:0/3067721941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc101029e0 0x7fcc1019a520 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 >> 172.21.15.94:0/3067721941 conn(0x7fcc100fdbd0 msgr2=0x7fcc10069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 shutdown_connections 2024-04-25T13:39:47.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.647+0000 7fcc1757a640 1 -- 172.21.15.94:0/3067721941 wait complete. 2024-04-25T13:39:47.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 -- 172.21.15.94:0/4130710744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 msgr2=0x7f87f410ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 --2- 172.21.15.94:0/4130710744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f410ad50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009ca0 tx=0x7f87ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 -- 172.21.15.94:0/4130710744 shutdown_connections 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 --2- 172.21.15.94:0/4130710744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f410ad50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 --2- 172.21.15.94:0/4130710744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 0x7f87f4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 --2- 172.21.15.94:0/4130710744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 0x7f87f41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 -- 172.21.15.94:0/4130710744 >> 172.21.15.94:0/4130710744 conn(0x7f87f40fdbc0 msgr2=0x7f87f4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 -- 172.21.15.94:0/4130710744 shutdown_connections 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.774+0000 7f87f9784640 1 -- 172.21.15.94:0/4130710744 wait complete. 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 Processor -- start 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 -- start start 2024-04-25T13:39:47.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 0x7f87f4075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 0x7f87f419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e590 con 0x7f87f4108930 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e410 con 0x7f87f4103c30 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f9784640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e710 con 0x7f87f41029f0 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42766/0 (socket says 172.21.15.94:42766) 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f37fe640 1 -- 172.21.15.94:0/2894355803 learned_addr learned my addr 172.21.15.94:0/2894355803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.775+0000 7f87f2ffd640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 0x7f87f4075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f27fc640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 0x7f87f419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:47.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 -- 172.21.15.94:0/2894355803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 msgr2=0x7f87f4075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 0x7f87f4075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 -- 172.21.15.94:0/2894355803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 msgr2=0x7f87f419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 0x7f87f419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 -- 172.21.15.94:0/2894355803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f41a1d80 con 0x7f87f4108930 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f37fe640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097b0 tx=0x7f87ec003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec004240 con 0x7f87f4108930 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f41a2010 con 0x7f87f4108930 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec0317c0 con 0x7f87f4108930 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec02fde0 con 0x7f87f4108930 2024-04-25T13:39:47.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.776+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f41a25e0 con 0x7f87f4108930 2024-04-25T13:39:47.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.777+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f4107740 con 0x7f87f4108930 2024-04-25T13:39:47.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.778+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f87ec0043e0 con 0x7f87f4108930 2024-04-25T13:39:47.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.779+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 0x7f87dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:47.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.779+0000 7f87f2ffd640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 0x7f87dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:47.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.779+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f87ec100410 con 0x7f87f4108930 2024-04-25T13:39:47.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.779+0000 7f87f2ffd640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 0x7f87dc0b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87e40058c0 tx=0x7f87e400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:47.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:47.784+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87ec0c9140 con 0x7f87f4108930 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:active 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:39:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:47 smithi137 ceph-mon[32568]: Cluster is now healthy 2024-04-25T13:39:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:39:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:active 2024-04-25T13:39:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:39:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:39:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:47 smithi094 ceph-mon[29195]: Cluster is now healthy 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3138928964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] up:active 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:39:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:47 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T13:39:48.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.000+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f87f40637a0 con 0x7f87f4108930 2024-04-25T13:39:48.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.656+0000 7f87dbfff640 1 -- 172.21.15.94:0/2894355803 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f87ec0c8a40 con 0x7f87f4108930 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 msgr2=0x7f87dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 0x7f87dc0b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87e40058c0 tx=0x7f87e400a2d0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 msgr2=0x7f87f41a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097b0 tx=0x7f87ec003c20 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 shutdown_connections 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87dc0afaf0 0x7f87dc0b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108930 0x7f87f41a1560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c30 0x7f87f419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 --2- 172.21.15.94:0/2894355803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029f0 0x7f87f4075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 >> 172.21.15.94:0/2894355803 conn(0x7f87f40fdbc0 msgr2=0x7f87f4104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:48.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.659+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 shutdown_connections 2024-04-25T13:39:48.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.660+0000 7f87f9784640 1 -- 172.21.15.94:0/2894355803 wait complete. 2024-04-25T13:39:48.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:39:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- 172.21.15.94:0/3408246318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268102f60 msgr2=0x7f7268103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 --2- 172.21.15.94:0/3408246318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268102f60 0x7f7268103370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7264009ca0 tx=0x7f726402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- 172.21.15.94:0/3408246318 shutdown_connections 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 --2- 172.21.15.94:0/3408246318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7268104460 0x7f726810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 --2- 172.21.15.94:0/3408246318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268103940 0x7f7268103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 --2- 172.21.15.94:0/3408246318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268102f60 0x7f7268103370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- 172.21.15.94:0/3408246318 >> 172.21.15.94:0/3408246318 conn(0x7f72680fd340 msgr2=0x7f72680ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- 172.21.15.94:0/3408246318 shutdown_connections 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- 172.21.15.94:0/3408246318 wait complete. 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 Processor -- start 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.792+0000 7f72700b1640 1 -- start start 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268102f60 0x7f7268199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7268104460 0x7f72681a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e500 con 0x7f7268103940 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e380 con 0x7f7268102f60 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f726810e680 con 0x7f7268104460 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42796/0 (socket says 172.21.15.94:42796) 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 -- 172.21.15.94:0/2550315287 learned_addr learned my addr 172.21.15.94:0/2550315287 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726de4d640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268102f60 0x7f7268199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 -- 172.21.15.94:0/2550315287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7268104460 msgr2=0x7f72681a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7268104460 0x7f72681a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 -- 172.21.15.94:0/2550315287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268102f60 msgr2=0x7f7268199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268102f60 0x7f7268199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 -- 172.21.15.94:0/2550315287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72681a0c70 con 0x7f7268103940 2024-04-25T13:39:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f726d64c640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f725800e9f0 tx=0x7f725800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800cd80 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72580108e0 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72681a0f60 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258018980 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.793+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681a13d0 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.794+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7268107900 con 0x7f7268103940 2024-04-25T13:39:48.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.796+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7258010430 con 0x7f7268103940 2024-04-25T13:39:48.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.796+0000 7f725effd640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 0x7f72540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:48.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.796+0000 7f726de4d640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 0x7f72540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:48.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.796+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f72580df330 con 0x7f7268103940 2024-04-25T13:39:48.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.797+0000 7f726de4d640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 0x7f72540b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72640094e0 tx=0x7f7264004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:48.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:48.800+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72580a8060 con 0x7f7268103940 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:39:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:48 smithi137 ceph-mon[32568]: pgmap v283: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:48.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:39:48.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:39:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:48 smithi094 ceph-mon[29195]: pgmap v283: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3067721941' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:39:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:48 smithi138 ceph-mon[32977]: pgmap v283: 97 pgs: 15 creating+peering, 67 unknown, 15 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:39:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.015+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f726802db50 con 0x7f7268103940 2024-04-25T13:39:49.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.663+0000 7f725effd640 1 -- 172.21.15.94:0/2550315287 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f72580b00e0 con 0x7f7268103940 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 msgr2=0x7f72540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 0x7f72540b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72640094e0 tx=0x7f7264004a40 comp rx=0 tx=0).stop 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 msgr2=0x7f7268199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f725800e9f0 tx=0x7f725800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 shutdown_connections 2024-04-25T13:39:49.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72540afaf0 0x7f72540b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7268104460 0x7f72681a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7268103940 0x7f7268199670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 --2- 172.21.15.94:0/2550315287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7268102f60 0x7f7268199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 >> 172.21.15.94:0/2550315287 conn(0x7f72680fd340 msgr2=0x7f7268104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 shutdown_connections 2024-04-25T13:39:49.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.666+0000 7f72700b1640 1 -- 172.21.15.94:0/2550315287 wait complete. 2024-04-25T13:39:49.682 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T13:39:49.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 -- 172.21.15.94:0/3194719539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad10108920 msgr2=0x7fad1010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 --2- 172.21.15.94:0/3194719539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad10108920 0x7fad1010ad40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad08009ca0 tx=0x7fad0802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 -- 172.21.15.94:0/3194719539 shutdown_connections 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 --2- 172.21.15.94:0/3194719539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad10108920 0x7fad1010ad40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 --2- 172.21.15.94:0/3194719539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103c20 0x7fad10104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 --2- 172.21.15.94:0/3194719539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad101029e0 0x7fad101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 -- 172.21.15.94:0/3194719539 >> 172.21.15.94:0/3194719539 conn(0x7fad100fdbd0 msgr2=0x7fad10100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:49.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 -- 172.21.15.94:0/3194719539 shutdown_connections 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.795+0000 7fad180bd640 1 -- 172.21.15.94:0/3194719539 wait complete. 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 Processor -- start 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 -- start start 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103c20 0x7fad1019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10108920 0x7fad101a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010e4f0 con 0x7fad101029e0 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010e370 con 0x7fad10108920 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad180bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad1010e670 con 0x7fad10103c20 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42812/0 (socket says 172.21.15.94:42812) 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 -- 172.21.15.94:0/2217942157 learned_addr learned my addr 172.21.15.94:0/2217942157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad1665a640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10108920 0x7fad101a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 -- 172.21.15.94:0/2217942157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103c20 msgr2=0x7fad1019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:49.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103c20 0x7fad1019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 -- 172.21.15.94:0/2217942157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10108920 msgr2=0x7fad101a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10108920 0x7fad101a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.796+0000 7fad15e59640 1 -- 172.21.15.94:0/2217942157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad101a1d00 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad15e59640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e9f0 tx=0x7fad0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00ccc0 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad0c0035f0 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad101a1ff0 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00ce60 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.797+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a2500 con 0x7fad101029e0 2024-04-25T13:39:49.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.798+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad101a27c0 con 0x7fad101029e0 2024-04-25T13:39:49.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.799+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fad0c003aa0 con 0x7fad101029e0 2024-04-25T13:39:49.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.800+0000 7fad06ffd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 0x7facfc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:49.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.800+0000 7fad15658640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 0x7facfc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:49.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.800+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fad0c0dee60 con 0x7fad101029e0 2024-04-25T13:39:49.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.800+0000 7fad15658640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 0x7facfc0b2080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad1019b310 tx=0x7fad00005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:49.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:49.804+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad0c0a7b90 con 0x7fad101029e0 2024-04-25T13:39:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:active 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T13:39:49.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:active 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T13:39:49.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:39:50.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.007+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fad101a2990 con 0x7facfc0afb70 2024-04-25T13:39:50.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.030+0000 7fad06ffd640 1 -- 172.21.15.94:0/2217942157 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad101a2990 con 0x7facfc0afb70 2024-04-25T13:39:50.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 msgr2=0x7facfc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 0x7facfc0b2080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad1019b310 tx=0x7fad00005c70 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 msgr2=0x7fad1019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e9f0 tx=0x7fad0c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 shutdown_connections 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facfc0afb70 0x7facfc0b2080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10108920 0x7fad101a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103c20 0x7fad1019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 --2- 172.21.15.94:0/2217942157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad101029e0 0x7fad1019a270 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 >> 172.21.15.94:0/2217942157 conn(0x7fad100fdbd0 msgr2=0x7fad10074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 shutdown_connections 2024-04-25T13:39:50.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.032+0000 7fad180bd640 1 -- 172.21.15.94:0/2217942157 wait complete. 2024-04-25T13:39:50.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2894355803' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] up:active 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T13:39:50.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 -- 172.21.15.94:0/448773722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 msgr2=0x7f431810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 --2- 172.21.15.94:0/448773722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f431810ad40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ca0 tx=0x7f431002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 -- 172.21.15.94:0/448773722 shutdown_connections 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 --2- 172.21.15.94:0/448773722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f431810ad40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 --2- 172.21.15.94:0/448773722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318103c20 0x7f4318104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 --2- 172.21.15.94:0/448773722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43181029e0 0x7f43181036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 -- 172.21.15.94:0/448773722 >> 172.21.15.94:0/448773722 conn(0x7f43180fdbd0 msgr2=0x7f4318100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 -- 172.21.15.94:0/448773722 shutdown_connections 2024-04-25T13:39:50.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.157+0000 7f431d106640 1 -- 172.21.15.94:0/448773722 wait complete. 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.158+0000 7f431d106640 1 Processor -- start 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.158+0000 7f431d106640 1 -- start start 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 0x7f431819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 0x7f431819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f431810f610 con 0x7f4318108920 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f431810f490 con 0x7f4318103c20 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f431810f790 con 0x7f43181029e0 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 0x7f431819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42842/0 (socket says 172.21.15.94:42842) 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 -- 172.21.15.94:0/1154612427 learned_addr learned my addr 172.21.15.94:0/1154612427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f4316d9d640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 0x7f431819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 -- 172.21.15.94:0/1154612427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 msgr2=0x7f431819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 0x7f431819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 -- 172.21.15.94:0/1154612427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 msgr2=0x7f431819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 0x7f431819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 -- 172.21.15.94:0/1154612427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181a2040 con 0x7f4318108920 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431659c640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 0x7f431819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f4316d9d640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 0x7f431819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431759e640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43100097b0 tx=0x7f4310031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310003bf0 con 0x7f4318108920 2024-04-25T13:39:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310003d90 con 0x7f4318108920 2024-04-25T13:39:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431002fc30 con 0x7f4318108920 2024-04-25T13:39:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181a22d0 con 0x7f4318108920 2024-04-25T13:39:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.159+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a2820 con 0x7f4318108920 2024-04-25T13:39:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.161+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4318107730 con 0x7f4318108920 2024-04-25T13:39:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.162+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4310004040 con 0x7f4318108920 2024-04-25T13:39:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.163+0000 7f42fffff640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 0x7f43000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.163+0000 7f4316d9d640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 0x7f43000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.163+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f4310100340 con 0x7f4318108920 2024-04-25T13:39:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.163+0000 7f4316d9d640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 0x7f43000b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43181035e0 tx=0x7f4308005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.167+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43100c9070 con 0x7f4318108920 2024-04-25T13:39:50.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.373+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f431802dda0 con 0x7f43000afaf0 2024-04-25T13:39:50.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.396+0000 7f42fffff640 1 -- 172.21.15.94:0/1154612427 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f431802dda0 con 0x7f43000afaf0 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 msgr2=0x7f43000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 0x7f43000b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43181035e0 tx=0x7f4308005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 msgr2=0x7f43181a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43100097b0 tx=0x7f4310031840 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 shutdown_connections 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f43000afaf0 0x7f43000b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4318108920 0x7f43181a19d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4318103c20 0x7f431819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 --2- 172.21.15.94:0/1154612427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43181029e0 0x7f431819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 >> 172.21.15.94:0/1154612427 conn(0x7f43180fdbd0 msgr2=0x7f4318104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 shutdown_connections 2024-04-25T13:39:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.398+0000 7f431d106640 1 -- 172.21.15.94:0/1154612427 wait complete. 2024-04-25T13:39:50.408 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:39:50.012+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:39:50.409 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:39:50.012+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:39:50.409 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:39:50.012+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:39:50.409 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:39:50.012+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:39:50.409 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:39:50.012+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:39:50.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 -- 172.21.15.94:0/208340390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff44101f10 msgr2=0x7eff44106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 --2- 172.21.15.94:0/208340390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff44101f10 0x7eff44106b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7eff34009900 tx=0x7eff3402a410 comp rx=0 tx=0).stop 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 -- 172.21.15.94:0/208340390 shutdown_connections 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 --2- 172.21.15.94:0/208340390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44109a20 0x7eff4410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 --2- 172.21.15.94:0/208340390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff441070c0 0x7eff441094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 --2- 172.21.15.94:0/208340390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff44101f10 0x7eff44106b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 -- 172.21.15.94:0/208340390 >> 172.21.15.94:0/208340390 conn(0x7eff440fd350 msgr2=0x7eff440ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 -- 172.21.15.94:0/208340390 shutdown_connections 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.522+0000 7eff4b627640 1 -- 172.21.15.94:0/208340390 wait complete. 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 Processor -- start 2024-04-25T13:39:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 -- start start 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44101f10 0x7eff44104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff44109a20 0x7eff44102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410f690 con 0x7eff441070c0 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410f510 con 0x7eff44101f10 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff4b627640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410f810 con 0x7eff44109a20 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42856/0 (socket says 172.21.15.94:42856) 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 -- 172.21.15.94:0/2053285698 learned_addr learned my addr 172.21.15.94:0/2053285698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff493c3640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44101f10 0x7eff44104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 -- 172.21.15.94:0/2053285698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff44109a20 msgr2=0x7eff44102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff44109a20 0x7eff44102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 -- 172.21.15.94:0/2053285698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44101f10 msgr2=0x7eff44104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44101f10 0x7eff44104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 -- 172.21.15.94:0/2053285698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff44103660 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff48bc2640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d770 tx=0x7eff4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.523+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40017070 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.524+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff400054d0 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.524+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff441a4330 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.524+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40004c30 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.524+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff441a47f0 con 0x7eff441070c0 2024-04-25T13:39:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.525+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff440fee70 con 0x7eff441070c0 2024-04-25T13:39:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.526+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7eff40004dd0 con 0x7eff441070c0 2024-04-25T13:39:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.526+0000 7eff3a7fc640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 0x7eff300b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.527+0000 7eff493c3640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 0x7eff300b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.527+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7eff400de930 con 0x7eff441070c0 2024-04-25T13:39:50.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.527+0000 7eff493c3640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 0x7eff300b2080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff34031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.530+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff400a76e0 con 0x7eff441070c0 2024-04-25T13:39:50.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.686+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7eff400a3270 con 0x7eff441070c0 2024-04-25T13:39:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.753+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7eff44062fe0 con 0x7eff300afb70 2024-04-25T13:39:50.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.758+0000 7eff3a7fc640 1 -- 172.21.15.94:0/2053285698 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7eff44062fe0 con 0x7eff300afb70 2024-04-25T13:39:50.759 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_0/438ad488-c693-4d06-86fe-55fab9149042 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 msgr2=0x7eff300b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 0x7eff300b2080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff34031400 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 msgr2=0x7eff441029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d770 tx=0x7eff4000dc30 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 shutdown_connections 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7eff300afb70 0x7eff300b2080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff44109a20 0x7eff44102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff441070c0 0x7eff441029f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 --2- 172.21.15.94:0/2053285698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff44101f10 0x7eff44104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 >> 172.21.15.94:0/2053285698 conn(0x7eff440fd350 msgr2=0x7eff441056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 shutdown_connections 2024-04-25T13:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.760+0000 7eff4b627640 1 -- 172.21.15.94:0/2053285698 wait complete. 2024-04-25T13:39:50.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:39:50.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 -- 172.21.15.94:0/2207864399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968103c20 msgr2=0x7f0968104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 --2- 172.21.15.94:0/2207864399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968103c20 0x7f0968104030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0950009900 tx=0x7f095002a410 comp rx=0 tx=0).stop 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 -- 172.21.15.94:0/2207864399 shutdown_connections 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 --2- 172.21.15.94:0/2207864399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968108920 0x7f096810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 --2- 172.21.15.94:0/2207864399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968103c20 0x7f0968104030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 --2- 172.21.15.94:0/2207864399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 0x7f09681036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 -- 172.21.15.94:0/2207864399 >> 172.21.15.94:0/2207864399 conn(0x7f09680fdbd0 msgr2=0x7f0968100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 -- 172.21.15.94:0/2207864399 shutdown_connections 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.890+0000 7f096d230640 1 -- 172.21.15.94:0/2207864399 wait complete. 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 Processor -- start 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 -- start start 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 0x7f096819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968103c20 0x7f096819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f096810f610 con 0x7f0968108920 2024-04-25T13:39:50.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f096810f490 con 0x7f09681029e0 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096d230640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f096810f790 con 0x7f0968103c20 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f0966d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 0x7f096819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42872/0 (socket says 172.21.15.94:42872) 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 -- 172.21.15.94:0/553363276 learned_addr learned my addr 172.21.15.94:0/553363276 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f095ffff640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968103c20 0x7f096819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 -- 172.21.15.94:0/553363276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968103c20 msgr2=0x7f096819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968103c20 0x7f096819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 -- 172.21.15.94:0/553363276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 msgr2=0x7f096819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 0x7f096819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.891+0000 7f096759e640 1 -- 172.21.15.94:0/553363276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681a2080 con 0x7f0968108920 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f096759e640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f096000eaa0 tx=0x7f096000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096000cdb0 con 0x7f0968108920 2024-04-25T13:39:50.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681a2370 con 0x7f0968108920 2024-04-25T13:39:50.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0960005020 con 0x7f0968108920 2024-04-25T13:39:50.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600116d0 con 0x7f0968108920 2024-04-25T13:39:50.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.892+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a2910 con 0x7f0968108920 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.893+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0968107730 con 0x7f0968108920 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.894+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f09600054a0 con 0x7f0968108920 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.895+0000 7f0964d99640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 0x7f09540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.895+0000 7f0966d9d640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 0x7f09540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.895+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f09600df1a0 con 0x7f0968108920 2024-04-25T13:39:50.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.895+0000 7f0966d9d640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 0x7f09540b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09681035e0 tx=0x7f0958005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:50.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:50.899+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0960016030 con 0x7f0968108920 2024-04-25T13:39:50.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:active 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:50.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:50 smithi094 ceph-mon[29195]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 16 op/s 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:active 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:50.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:50 smithi138 ceph-mon[32977]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 16 op/s 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2550315287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] up:active 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:50 smithi137 ceph-mon[32568]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s wr, 16 op/s 2024-04-25T13:39:51.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.106+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f096802dd80 con 0x7f09540afb80 2024-04-25T13:39:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.129+0000 7f0964d99640 1 -- 172.21.15.94:0/553363276 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f096802dd80 con 0x7f09540afb80 2024-04-25T13:39:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 msgr2=0x7f09540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 0x7f09540b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09681035e0 tx=0x7f0958005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 msgr2=0x7f09681a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f096000eaa0 tx=0x7f096000ef60 comp rx=0 tx=0).stop 2024-04-25T13:39:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 shutdown_connections 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09540afb80 0x7f09540b2090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0968108920 0x7f09681a1a10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0968103c20 0x7f096819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 --2- 172.21.15.94:0/553363276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09681029e0 0x7f096819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 >> 172.21.15.94:0/553363276 conn(0x7f09680fdbd0 msgr2=0x7f0968104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 shutdown_connections 2024-04-25T13:39:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.132+0000 7f096d230640 1 -- 172.21.15.94:0/553363276 wait complete. 2024-04-25T13:39:51.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:39:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 -- 172.21.15.94:0/2187754107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1740103c20 msgr2=0x7f1740104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 --2- 172.21.15.94:0/2187754107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1740103c20 0x7f1740104030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f173c009900 tx=0x7f173c02a410 comp rx=0 tx=0).stop 2024-04-25T13:39:51.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 -- 172.21.15.94:0/2187754107 shutdown_connections 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 --2- 172.21.15.94:0/2187754107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740108920 0x7f174010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 --2- 172.21.15.94:0/2187754107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1740103c20 0x7f1740104030 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 --2- 172.21.15.94:0/2187754107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17401029e0 0x7f17401036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 -- 172.21.15.94:0/2187754107 >> 172.21.15.94:0/2187754107 conn(0x7f17400fdbd0 msgr2=0x7f1740100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 -- 172.21.15.94:0/2187754107 shutdown_connections 2024-04-25T13:39:51.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.257+0000 7f174818f640 1 -- 172.21.15.94:0/2187754107 wait complete. 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 Processor -- start 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 -- start start 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740103c20 0x7f174019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 0x7f17401a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f174010e4e0 con 0x7f17401029e0 2024-04-25T13:39:51.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f174010e360 con 0x7f1740103c20 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f174010e660 con 0x7f1740108920 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42898/0 (socket says 172.21.15.94:42898) 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 -- 172.21.15.94:0/568602016 learned_addr learned my addr 172.21.15.94:0/568602016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174572a640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740103c20 0x7f174019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174672c640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 0x7f17401a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 -- 172.21.15.94:0/568602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 msgr2=0x7f17401a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 0x7f17401a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 -- 172.21.15.94:0/568602016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740103c20 msgr2=0x7f174019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740103c20 0x7f174019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 -- 172.21.15.94:0/568602016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401a2050 con 0x7f17401029e0 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174672c640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 0x7f17401a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:51.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1745f2b640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1734005fd0 tx=0x7f173400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734017070 con 0x7f17401029e0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1734009c30 con 0x7f17401029e0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401a2340 con 0x7f17401029e0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.258+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734003d10 con 0x7f17401029e0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.259+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401a2890 con 0x7f17401029e0 2024-04-25T13:39:51.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.259+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740107730 con 0x7f17401029e0 2024-04-25T13:39:51.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.261+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1734003eb0 con 0x7f17401029e0 2024-04-25T13:39:51.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.261+0000 7f1732ffd640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 0x7f172c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.262+0000 7f174572a640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 0x7f172c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.262+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f17340df0e0 con 0x7f17401029e0 2024-04-25T13:39:51.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.262+0000 7f174572a640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 0x7f172c0b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173c009230 tx=0x7f173c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.265+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1734020030 con 0x7f17401029e0 2024-04-25T13:39:51.393 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T13:39:51.030+0000 7fbaffd29700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:39:51.393 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T13:39:51.030+0000 7fbaffd29700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T13:39:51.393 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T13:39:51.030+0000 7fbaffd29700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T13:39:51.393 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T13:39:51.030+0000 7fbaffd29700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T13:39:51.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.474+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f174002dda0 con 0x7f172c0afb80 2024-04-25T13:39:51.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.479+0000 7f1732ffd640 1 -- 172.21.15.94:0/568602016 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f174002dda0 con 0x7f172c0afb80 2024-04-25T13:39:51.479 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_1/76de3e12-c8d1-4ba0-beb9-97fd0823e7ee 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 msgr2=0x7f172c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 0x7f172c0b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173c009230 tx=0x7f173c031400 comp rx=0 tx=0).stop 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 msgr2=0x7f174006a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1734005fd0 tx=0x7f173400dde0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 shutdown_connections 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f172c0afb80 0x7f172c0b2090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1740108920 0x7f17401a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1740103c20 0x7f174019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 --2- 172.21.15.94:0/568602016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17401029e0 0x7f174006a8b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 >> 172.21.15.94:0/568602016 conn(0x7f17400fdbd0 msgr2=0x7f1740104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 shutdown_connections 2024-04-25T13:39:51.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.481+0000 7f174818f640 1 -- 172.21.15.94:0/568602016 wait complete. 2024-04-25T13:39:51.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:39:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 -- 172.21.15.94:0/1418286178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254109a40 msgr2=0x7fe25410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 --2- 172.21.15.94:0/1418286178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254109a40 0x7fe25410be60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009ca0 tx=0x7fe24c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 -- 172.21.15.94:0/1418286178 shutdown_connections 2024-04-25T13:39:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 --2- 172.21.15.94:0/1418286178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254109a40 0x7fe25410be60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 --2- 172.21.15.94:0/1418286178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254104d40 0x7fe254105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 --2- 172.21.15.94:0/1418286178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254103b00 0x7fe254104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 -- 172.21.15.94:0/1418286178 >> 172.21.15.94:0/1418286178 conn(0x7fe2540fed10 msgr2=0x7fe2541011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 -- 172.21.15.94:0/1418286178 shutdown_connections 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 -- 172.21.15.94:0/1418286178 wait complete. 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.600+0000 7fe25b746640 1 Processor -- start 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 -- start start 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254104d40 0x7fe25419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254109a40 0x7fe2541a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410f610 con 0x7fe254103b00 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410f490 con 0x7fe254109a40 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe25410f790 con 0x7fe254104d40 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42910/0 (socket says 172.21.15.94:42910) 2024-04-25T13:39:51.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 -- 172.21.15.94:0/436400871 learned_addr learned my addr 172.21.15.94:0/436400871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe258ce1640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254104d40 0x7fe25419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe259ce3640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254109a40 0x7fe2541a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 -- 172.21.15.94:0/436400871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254104d40 msgr2=0x7fe25419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254104d40 0x7fe25419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 -- 172.21.15.94:0/436400871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254109a40 msgr2=0x7fe2541a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254109a40 0x7fe2541a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 -- 172.21.15.94:0/436400871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2541a1d10 con 0x7fe254103b00 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe2594e2640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe244005fd0 tx=0x7fe24400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244017070 con 0x7fe254103b00 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244009c30 con 0x7fe254103b00 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.601+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2541a1fa0 con 0x7fe254103b00 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.602+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244003d10 con 0x7fe254103b00 2024-04-25T13:39:51.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.602+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2541a2460 con 0x7fe254103b00 2024-04-25T13:39:51.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.603+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe254108850 con 0x7fe254103b00 2024-04-25T13:39:51.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.604+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe244003eb0 con 0x7fe254103b00 2024-04-25T13:39:51.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.604+0000 7fe24a7fc640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 0x7fe2400b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.605+0000 7fe258ce1640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 0x7fe2400b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.605+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fe2440defe0 con 0x7fe254103b00 2024-04-25T13:39:51.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.605+0000 7fe258ce1640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 0x7fe2400b2090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe25419b270 tx=0x7fe250005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.608+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe244020030 con 0x7fe254103b00 2024-04-25T13:39:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.830+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe25402db50 con 0x7fe254103b00 2024-04-25T13:39:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.831+0000 7fe24a7fc640 1 -- 172.21.15.94:0/436400871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7fe244009de0 con 0x7fe254103b00 2024-04-25T13:39:51.831 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:51.832 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:39:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 msgr2=0x7fe2400b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 0x7fe2400b2090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe25419b270 tx=0x7fe250005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 msgr2=0x7fe25419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe244005fd0 tx=0x7fe24400dde0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 shutdown_connections 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe2400afb80 0x7fe2400b2090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe254109a40 0x7fe2541a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe254104d40 0x7fe25419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 --2- 172.21.15.94:0/436400871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe254103b00 0x7fe25419a1d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 >> 172.21.15.94:0/436400871 conn(0x7fe2540fed10 msgr2=0x7fe254105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 shutdown_connections 2024-04-25T13:39:51.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.833+0000 7fe25b746640 1 -- 172.21.15.94:0/436400871 wait complete. 2024-04-25T13:39:51.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:active 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: Cluster is now healthy 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:active 2024-04-25T13:39:51.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:51 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:39:51.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 -- 172.21.15.94:0/1489398570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 msgr2=0x7f573410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1489398570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f573410ad20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f572c009ca0 tx=0x7f572c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 -- 172.21.15.94:0/1489398570 shutdown_connections 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1489398570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f573410ad20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1489398570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 0x7f5734104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1489398570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 0x7f57341036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 -- 172.21.15.94:0/1489398570 >> 172.21.15.94:0/1489398570 conn(0x7f57340fdbd0 msgr2=0x7f5734100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 -- 172.21.15.94:0/1489398570 shutdown_connections 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 -- 172.21.15.94:0/1489398570 wait complete. 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.953+0000 7f5739fd7640 1 Processor -- start 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 -- start start 2024-04-25T13:39:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 0x7f573419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 0x7f573419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f573410f5f0 con 0x7f5734108900 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f573410f470 con 0x7f57341029c0 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5739fd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f573410f770 con 0x7f5734103c00 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42928/0 (socket says 172.21.15.94:42928) 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f57337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 0x7f573419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 -- 172.21.15.94:0/1301517747 learned_addr learned my addr 172.21.15.94:0/1301517747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f572bfff640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 0x7f573419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 -- 172.21.15.94:0/1301517747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 msgr2=0x7f573419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 0x7f573419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 -- 172.21.15.94:0/1301517747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 msgr2=0x7f573419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:51.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 0x7f573419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 -- 172.21.15.94:0/1301517747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341a1c50 con 0x7f5734108900 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f5733fff640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f572c0097b0 tx=0x7f572c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.954+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c003bf0 con 0x7f5734108900 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.955+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f572c003d90 con 0x7f5734108900 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.955+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a1ee0 con 0x7f5734108900 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.955+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c02fc30 con 0x7f5734108900 2024-04-25T13:39:51.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.955+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a2340 con 0x7f5734108900 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.956+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573406afa0 con 0x7f5734108900 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.957+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f572c004040 con 0x7f5734108900 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.957+0000 7f57317fa640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 0x7f57200b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.957+0000 7f57337fe640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 0x7f57200b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.958+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f572c100180 con 0x7f5734108900 2024-04-25T13:39:51.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.958+0000 7f57337fe640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 0x7f57200b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57341035c0 tx=0x7f5724005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:51.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:51.963+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f572c0c8ec0 con 0x7f5734108900 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:active 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:active 2024-04-25T13:39:51.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:51 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:39:52.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.141+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57340637a0 con 0x7f5734108900 2024-04-25T13:39:52.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.142+0000 7f57317fa640 1 -- 172.21.15.94:0/1301517747 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22244 (secure 0 0 0) 0x7f572c0c87c0 con 0x7f5734108900 2024-04-25T13:39:52.143 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:52.143 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:39:46.244890+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] up:active 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: Cluster is now healthy 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:52.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] up:active 2024-04-25T13:39:52.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:51 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:39:52.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 msgr2=0x7f57200b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 0x7f57200b2090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57341035c0 tx=0x7f5724005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 msgr2=0x7f57341a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f572c0097b0 tx=0x7f572c031840 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 shutdown_connections 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57200afb80 0x7f57200b2090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5734108900 0x7f57341a1690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5734103c00 0x7f573419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 --2- 172.21.15.94:0/1301517747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57341029c0 0x7f573419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 >> 172.21.15.94:0/1301517747 conn(0x7f57340fdbd0 msgr2=0x7f57341048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 shutdown_connections 2024-04-25T13:39:52.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.144+0000 7f5739fd7640 1 -- 172.21.15.94:0/1301517747 wait complete. 2024-04-25T13:39:52.157 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:39:52.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:39:52.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.264+0000 7f058969f640 1 -- 172.21.15.94:0/1481114567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584109a20 msgr2=0x7f058410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.264+0000 7f058969f640 1 --2- 172.21.15.94:0/1481114567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584109a20 0x7f058410be40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f057c009ca0 tx=0x7f057c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 -- 172.21.15.94:0/1481114567 shutdown_connections 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 --2- 172.21.15.94:0/1481114567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584109a20 0x7f058410be40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 --2- 172.21.15.94:0/1481114567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584104d20 0x7f0584105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 --2- 172.21.15.94:0/1481114567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584103ae0 0x7f05841047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 -- 172.21.15.94:0/1481114567 >> 172.21.15.94:0/1481114567 conn(0x7f05840fecd0 msgr2=0x7f0584101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 -- 172.21.15.94:0/1481114567 shutdown_connections 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 -- 172.21.15.94:0/1481114567 wait complete. 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 Processor -- start 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.265+0000 7f058969f640 1 -- start start 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 0x7f058419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584109a20 0x7f05841a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f058410f610 con 0x7f0584104d20 2024-04-25T13:39:52.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f058410f490 con 0x7f0584109a20 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f058410f790 con 0x7f0584103ae0 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42954/0 (socket says 172.21.15.94:42954) 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 -- 172.21.15.94:0/583497716 learned_addr learned my addr 172.21.15.94:0/583497716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05837fe640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584109a20 0x7f05841a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f0582ffd640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 0x7f058419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 -- 172.21.15.94:0/583497716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 msgr2=0x7f058419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 0x7f058419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 -- 172.21.15.94:0/583497716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584109a20 msgr2=0x7f05841a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584109a20 0x7f05841a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 -- 172.21.15.94:0/583497716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05841a1c90 con 0x7f0584104d20 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f0582ffd640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 0x7f058419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:39:52.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f05827fc640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f057400d770 tx=0x7f057400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:52.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574017070 con 0x7f0584104d20 2024-04-25T13:39:52.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05740054d0 con 0x7f0584104d20 2024-04-25T13:39:52.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841a1f80 con 0x7f0584104d20 2024-04-25T13:39:52.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574004c30 con 0x7f0584104d20 2024-04-25T13:39:52.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.266+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841a23e0 con 0x7f0584104d20 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.268+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584108830 con 0x7f0584104d20 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.269+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0574004dd0 con 0x7f0584104d20 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.269+0000 7f056bfff640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 0x7f056c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.270+0000 7f0582ffd640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 0x7f056c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.270+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f05740dea50 con 0x7f0584104d20 2024-04-25T13:39:52.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.270+0000 7f0582ffd640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 0x7f056c0b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0578005d00 tx=0x7f0578005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:52.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.273+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05740a7680 con 0x7f0584104d20 2024-04-25T13:39:52.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.448+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0584062fe0 con 0x7f056c0afb80 2024-04-25T13:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.456+0000 7f056bfff640 1 -- 172.21.15.94:0/583497716 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7f0584062fe0 con 0x7f056c0afb80 2024-04-25T13:39:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:52.461 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 msgr2=0x7f056c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 0x7f056c0b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0578005d00 tx=0x7f0578005ef0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 msgr2=0x7f058419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f057400d770 tx=0x7f057400dc30 comp rx=0 tx=0).stop 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 shutdown_connections 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f056c0afb80 0x7f056c0b2090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0584109a20 0x7f05841a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0584104d20 0x7f058419a690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 --2- 172.21.15.94:0/583497716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584103ae0 0x7f058419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 >> 172.21.15.94:0/583497716 conn(0x7f05840fecd0 msgr2=0x7f0584105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 shutdown_connections 2024-04-25T13:39:52.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.459+0000 7f058969f640 1 -- 172.21.15.94:0/583497716 wait complete. 2024-04-25T13:39:52.471 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-04-25T13:39:52.257169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614968,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":254,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":249,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398488,"kb_used_data":11864,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124543336,"statfs":{"total":1151940427776,"available":1151532376064,"internally_reserved":0,"allocated":12148736,"data_stored":6656692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24600,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.651227"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664300+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:39:47.664300+0000","last_peered":"2024-04-25T13:39:47.664300+0000","last_clean":"2024-04-25T13:39:47.664300+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:39:47.664300+0000","last_undegraded":"2024-04-25T13:39:47.664300+0000","last_fullsized":"2024-04-25T13:39:47.664300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660335+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:39:47.660335+0000","last_peered":"2024-04-25T13:39:47.660335+0000","last_clean":"2024-04-25T13:39:47.660335+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:39:47.660335+0000","last_undegraded":"2024-04-25T13:39:47.660335+0000","last_fullsized":"2024-04-25T13:39:47.660335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660313+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:39:47.660313+0000","last_peered":"2024-04-25T13:39:47.660313+0000","last_clean":"2024-04-25T13:39:47.660313+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:39:47.660313+0000","last_undegraded":"2024-04-25T13:39:47.660313+0000","last_fullsized":"2024-04-25T13:39:47.660313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660340+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:39:47.660340+0000","last_peered":"2024-04-25T13:39:47.660340+0000","last_clean":"2024-04-25T13:39:47.660340+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:39:47.660340+0000","last_undegraded":"2024-04-25T13:39:47.660340+0000","last_fullsized":"2024-04-25T13:39:47.660340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616318+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:39:47.616318+0000","last_peered":"2024-04-25T13:39:47.616318+0000","last_clean":"2024-04-25T13:39:47.616318+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:39:47.616318+0000","last_undegraded":"2024-04-25T13:39:47.616318+0000","last_fullsized":"2024-04-25T13:39:47.616318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689172+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:39:46.689172+0000","last_peered":"2024-04-25T13:39:46.689172+0000","last_clean":"2024-04-25T13:39:46.689172+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:39:46.689172+0000","last_undegraded":"2024-04-25T13:39:46.689172+0000","last_fullsized":"2024-04-25T13:39:46.689172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251560+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:39:46.251560+0000","last_peered":"2024-04-25T13:39:46.251560+0000","last_clean":"2024-04-25T13:39:46.251560+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:39:46.251560+0000","last_undegraded":"2024-04-25T13:39:46.251560+0000","last_fullsized":"2024-04-25T13:39:46.251560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615391+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:39:47.615391+0000","last_peered":"2024-04-25T13:39:47.615391+0000","last_clean":"2024-04-25T13:39:47.615391+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:39:47.615391+0000","last_undegraded":"2024-04-25T13:39:47.615391+0000","last_fullsized":"2024-04-25T13:39:47.615391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539267+0000","last_change":"2024-04-25T13:39:45.265065+0000","last_active":"2024-04-25T13:39:46.539267+0000","last_peered":"2024-04-25T13:39:46.539267+0000","last_clean":"2024-04-25T13:39:46.539267+0000","last_became_active":"2024-04-25T13:39:45.264734+0000","last_became_peered":"2024-04-25T13:39:45.264734+0000","last_unstale":"2024-04-25T13:39:46.539267+0000","last_undegraded":"2024-04-25T13:39:46.539267+0000","last_fullsized":"2024-04-25T13:39:46.539267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689607+0000","last_change":"2024-04-25T13:39:45.268786+0000","last_active":"2024-04-25T13:39:46.689607+0000","last_peered":"2024-04-25T13:39:46.689607+0000","last_clean":"2024-04-25T13:39:46.689607+0000","last_became_active":"2024-04-25T13:39:45.268384+0000","last_became_peered":"2024-04-25T13:39:45.268384+0000","last_unstale":"2024-04-25T13:39:46.689607+0000","last_undegraded":"2024-04-25T13:39:46.689607+0000","last_fullsized":"2024-04-25T13:39:46.689607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:14.179367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251972+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:39:46.251972+0000","last_peered":"2024-04-25T13:39:46.251972+0000","last_clean":"2024-04-25T13:39:46.251972+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:39:46.251972+0000","last_undegraded":"2024-04-25T13:39:46.251972+0000","last_fullsized":"2024-04-25T13:39:46.251972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.661336+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:39:47.661336+0000","last_peered":"2024-04-25T13:39:47.661336+0000","last_clean":"2024-04-25T13:39:47.661336+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:39:47.661336+0000","last_undegraded":"2024-04-25T13:39:47.661336+0000","last_fullsized":"2024-04-25T13:39:47.661336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248359+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:39:46.248359+0000","last_peered":"2024-04-25T13:39:46.248359+0000","last_clean":"2024-04-25T13:39:46.248359+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:39:46.248359+0000","last_undegraded":"2024-04-25T13:39:46.248359+0000","last_fullsized":"2024-04-25T13:39:46.248359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.659790+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:39:47.659790+0000","last_peered":"2024-04-25T13:39:47.659790+0000","last_clean":"2024-04-25T13:39:47.659790+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:39:47.659790+0000","last_undegraded":"2024-04-25T13:39:47.659790+0000","last_fullsized":"2024-04-25T13:39:47.659790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.659501+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:39:47.659501+0000","last_peered":"2024-04-25T13:39:47.659501+0000","last_clean":"2024-04-25T13:39:47.659501+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:39:47.659501+0000","last_undegraded":"2024-04-25T13:39:47.659501+0000","last_fullsized":"2024-04-25T13:39:47.659501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538737+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:39:46.538737+0000","last_peered":"2024-04-25T13:39:46.538737+0000","last_clean":"2024-04-25T13:39:46.538737+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:39:46.538737+0000","last_undegraded":"2024-04-25T13:39:46.538737+0000","last_fullsized":"2024-04-25T13:39:46.538737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664374+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:39:47.664374+0000","last_peered":"2024-04-25T13:39:47.664374+0000","last_clean":"2024-04-25T13:39:47.664374+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:39:47.664374+0000","last_undegraded":"2024-04-25T13:39:47.664374+0000","last_fullsized":"2024-04-25T13:39:47.664374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613658+0000","last_change":"2024-04-25T13:39:45.259538+0000","last_active":"2024-04-25T13:39:47.613658+0000","last_peered":"2024-04-25T13:39:47.613658+0000","last_clean":"2024-04-25T13:39:47.613658+0000","last_became_active":"2024-04-25T13:39:45.259097+0000","last_became_peered":"2024-04-25T13:39:45.259097+0000","last_unstale":"2024-04-25T13:39:47.613658+0000","last_undegraded":"2024-04-25T13:39:47.613658+0000","last_fullsized":"2024-04-25T13:39:47.613658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:10.901989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538905+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:39:46.538905+0000","last_peered":"2024-04-25T13:39:46.538905+0000","last_clean":"2024-04-25T13:39:46.538905+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:39:46.538905+0000","last_undegraded":"2024-04-25T13:39:46.538905+0000","last_fullsized":"2024-04-25T13:39:46.538905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.678534+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:39:48.678534+0000","last_peered":"2024-04-25T13:39:48.678534+0000","last_clean":"2024-04-25T13:39:48.678534+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:39:48.678534+0000","last_undegraded":"2024-04-25T13:39:48.678534+0000","last_fullsized":"2024-04-25T13:39:48.678534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615946+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:39:47.615946+0000","last_peered":"2024-04-25T13:39:47.615946+0000","last_clean":"2024-04-25T13:39:47.615946+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:39:47.615946+0000","last_undegraded":"2024-04-25T13:39:47.615946+0000","last_fullsized":"2024-04-25T13:39:47.615946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615961+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:39:47.615961+0000","last_peered":"2024-04-25T13:39:47.615961+0000","last_clean":"2024-04-25T13:39:47.615961+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:39:47.615961+0000","last_undegraded":"2024-04-25T13:39:47.615961+0000","last_fullsized":"2024-04-25T13:39:47.615961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660430+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:39:47.660430+0000","last_peered":"2024-04-25T13:39:47.660430+0000","last_clean":"2024-04-25T13:39:47.660430+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:39:47.660430+0000","last_undegraded":"2024-04-25T13:39:47.660430+0000","last_fullsized":"2024-04-25T13:39:47.660430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616562+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:39:47.616562+0000","last_peered":"2024-04-25T13:39:47.616562+0000","last_clean":"2024-04-25T13:39:47.616562+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:39:47.616562+0000","last_undegraded":"2024-04-25T13:39:47.616562+0000","last_fullsized":"2024-04-25T13:39:47.616562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538947+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:39:46.538947+0000","last_peered":"2024-04-25T13:39:46.538947+0000","last_clean":"2024-04-25T13:39:46.538947+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:39:46.538947+0000","last_undegraded":"2024-04-25T13:39:46.538947+0000","last_fullsized":"2024-04-25T13:39:46.538947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616517+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:39:47.616517+0000","last_peered":"2024-04-25T13:39:47.616517+0000","last_clean":"2024-04-25T13:39:47.616517+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:39:47.616517+0000","last_undegraded":"2024-04-25T13:39:47.616517+0000","last_fullsized":"2024-04-25T13:39:47.616517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249743+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:39:46.249743+0000","last_peered":"2024-04-25T13:39:46.249743+0000","last_clean":"2024-04-25T13:39:46.249743+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:39:46.249743+0000","last_undegraded":"2024-04-25T13:39:46.249743+0000","last_fullsized":"2024-04-25T13:39:46.249743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664521+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:39:47.664521+0000","last_peered":"2024-04-25T13:39:47.664521+0000","last_clean":"2024-04-25T13:39:47.664521+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:39:47.664521+0000","last_undegraded":"2024-04-25T13:39:47.664521+0000","last_fullsized":"2024-04-25T13:39:47.664521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616189+0000","last_change":"2024-04-25T13:39:45.262172+0000","last_active":"2024-04-25T13:39:47.616189+0000","last_peered":"2024-04-25T13:39:47.616189+0000","last_clean":"2024-04-25T13:39:47.616189+0000","last_became_active":"2024-04-25T13:39:45.261838+0000","last_became_peered":"2024-04-25T13:39:45.261838+0000","last_unstale":"2024-04-25T13:39:47.616189+0000","last_undegraded":"2024-04-25T13:39:47.616189+0000","last_fullsized":"2024-04-25T13:39:47.616189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:09.938897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:49.679774+0000","last_change":"2024-04-25T13:39:44.625364+0000","last_active":"2024-04-25T13:39:49.679774+0000","last_peered":"2024-04-25T13:39:49.679774+0000","last_clean":"2024-04-25T13:39:49.679774+0000","last_became_active":"2024-04-25T13:39:44.625014+0000","last_became_peered":"2024-04-25T13:39:44.625014+0000","last_unstale":"2024-04-25T13:39:49.679774+0000","last_undegraded":"2024-04-25T13:39:49.679774+0000","last_fullsized":"2024-04-25T13:39:49.679774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:36.507525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249607+0000","last_change":"2024-04-25T13:39:45.268193+0000","last_active":"2024-04-25T13:39:46.249607+0000","last_peered":"2024-04-25T13:39:46.249607+0000","last_clean":"2024-04-25T13:39:46.249607+0000","last_became_active":"2024-04-25T13:39:45.267842+0000","last_became_peered":"2024-04-25T13:39:45.267842+0000","last_unstale":"2024-04-25T13:39:46.249607+0000","last_undegraded":"2024-04-25T13:39:46.249607+0000","last_fullsized":"2024-04-25T13:39:46.249607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:56.308127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.675645+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:39:48.675645+0000","last_peered":"2024-04-25T13:39:48.675645+0000","last_clean":"2024-04-25T13:39:48.675645+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:39:48.675645+0000","last_undegraded":"2024-04-25T13:39:48.675645+0000","last_fullsized":"2024-04-25T13:39:48.675645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615344+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:39:47.615344+0000","last_peered":"2024-04-25T13:39:47.615344+0000","last_clean":"2024-04-25T13:39:47.615344+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:39:47.615344+0000","last_undegraded":"2024-04-25T13:39:47.615344+0000","last_fullsized":"2024-04-25T13:39:47.615344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660704+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:39:47.660704+0000","last_peered":"2024-04-25T13:39:47.660704+0000","last_clean":"2024-04-25T13:39:47.660704+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:39:47.660704+0000","last_undegraded":"2024-04-25T13:39:47.660704+0000","last_fullsized":"2024-04-25T13:39:47.660704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613513+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:39:47.613513+0000","last_peered":"2024-04-25T13:39:47.613513+0000","last_clean":"2024-04-25T13:39:47.613513+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:39:47.613513+0000","last_undegraded":"2024-04-25T13:39:47.613513+0000","last_fullsized":"2024-04-25T13:39:47.613513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662962+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:39:47.662962+0000","last_peered":"2024-04-25T13:39:47.662962+0000","last_clean":"2024-04-25T13:39:47.662962+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:39:47.662962+0000","last_undegraded":"2024-04-25T13:39:47.662962+0000","last_fullsized":"2024-04-25T13:39:47.662962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539290+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:39:46.539290+0000","last_peered":"2024-04-25T13:39:46.539290+0000","last_clean":"2024-04-25T13:39:46.539290+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:39:46.539290+0000","last_undegraded":"2024-04-25T13:39:46.539290+0000","last_fullsized":"2024-04-25T13:39:46.539290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662925+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:39:47.662925+0000","last_peered":"2024-04-25T13:39:47.662925+0000","last_clean":"2024-04-25T13:39:47.662925+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:39:47.662925+0000","last_undegraded":"2024-04-25T13:39:47.662925+0000","last_fullsized":"2024-04-25T13:39:47.662925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660412+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:39:47.660412+0000","last_peered":"2024-04-25T13:39:47.660412+0000","last_clean":"2024-04-25T13:39:47.660412+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:39:47.660412+0000","last_undegraded":"2024-04-25T13:39:47.660412+0000","last_fullsized":"2024-04-25T13:39:47.660412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662087+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:39:47.662087+0000","last_peered":"2024-04-25T13:39:47.662087+0000","last_clean":"2024-04-25T13:39:47.662087+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:39:47.662087+0000","last_undegraded":"2024-04-25T13:39:47.662087+0000","last_fullsized":"2024-04-25T13:39:47.662087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539104+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:39:46.539104+0000","last_peered":"2024-04-25T13:39:46.539104+0000","last_clean":"2024-04-25T13:39:46.539104+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:39:46.539104+0000","last_undegraded":"2024-04-25T13:39:46.539104+0000","last_fullsized":"2024-04-25T13:39:46.539104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'15","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:50.389148+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:39:50.389148+0000","last_peered":"2024-04-25T13:39:50.389148+0000","last_clean":"2024-04-25T13:39:50.389148+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:39:50.389148+0000","last_undegraded":"2024-04-25T13:39:50.389148+0000","last_fullsized":"2024-04-25T13:39:50.389148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616425+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:39:47.616425+0000","last_peered":"2024-04-25T13:39:47.616425+0000","last_clean":"2024-04-25T13:39:47.616425+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:39:47.616425+0000","last_undegraded":"2024-04-25T13:39:47.616425+0000","last_fullsized":"2024-04-25T13:39:47.616425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.676062+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:39:48.676062+0000","last_peered":"2024-04-25T13:39:48.676062+0000","last_clean":"2024-04-25T13:39:48.676062+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:39:48.676062+0000","last_undegraded":"2024-04-25T13:39:48.676062+0000","last_fullsized":"2024-04-25T13:39:48.676062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248183+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:39:46.248183+0000","last_peered":"2024-04-25T13:39:46.248183+0000","last_clean":"2024-04-25T13:39:46.248183+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:39:46.248183+0000","last_undegraded":"2024-04-25T13:39:46.248183+0000","last_fullsized":"2024-04-25T13:39:46.248183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.661193+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:39:47.661193+0000","last_peered":"2024-04-25T13:39:47.661193+0000","last_clean":"2024-04-25T13:39:47.661193+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:39:47.661193+0000","last_undegraded":"2024-04-25T13:39:47.661193+0000","last_fullsized":"2024-04-25T13:39:47.661193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251832+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:39:46.251832+0000","last_peered":"2024-04-25T13:39:46.251832+0000","last_clean":"2024-04-25T13:39:46.251832+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:39:46.251832+0000","last_undegraded":"2024-04-25T13:39:46.251832+0000","last_fullsized":"2024-04-25T13:39:46.251832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.675014+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:39:48.675014+0000","last_peered":"2024-04-25T13:39:48.675014+0000","last_clean":"2024-04-25T13:39:48.675014+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:39:48.675014+0000","last_undegraded":"2024-04-25T13:39:48.675014+0000","last_fullsized":"2024-04-25T13:39:48.675014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251347+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:39:46.251347+0000","last_peered":"2024-04-25T13:39:46.251347+0000","last_clean":"2024-04-25T13:39:46.251347+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:39:46.251347+0000","last_undegraded":"2024-04-25T13:39:46.251347+0000","last_fullsized":"2024-04-25T13:39:46.251347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'10","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663300+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:39:47.663300+0000","last_peered":"2024-04-25T13:39:47.663300+0000","last_clean":"2024-04-25T13:39:47.663300+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:39:47.663300+0000","last_undegraded":"2024-04-25T13:39:47.663300+0000","last_fullsized":"2024-04-25T13:39:47.663300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249214+0000","last_change":"2024-04-25T13:39:45.273219+0000","last_active":"2024-04-25T13:39:46.249214+0000","last_peered":"2024-04-25T13:39:46.249214+0000","last_clean":"2024-04-25T13:39:46.249214+0000","last_became_active":"2024-04-25T13:39:45.272891+0000","last_became_peered":"2024-04-25T13:39:45.272891+0000","last_unstale":"2024-04-25T13:39:46.249214+0000","last_undegraded":"2024-04-25T13:39:46.249214+0000","last_fullsized":"2024-04-25T13:39:46.249214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:40.531758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616513+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:39:47.616513+0000","last_peered":"2024-04-25T13:39:47.616513+0000","last_clean":"2024-04-25T13:39:47.616513+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:39:47.616513+0000","last_undegraded":"2024-04-25T13:39:47.616513+0000","last_fullsized":"2024-04-25T13:39:47.616513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663853+0000","last_change":"2024-04-25T13:39:45.274393+0000","last_active":"2024-04-25T13:39:47.663853+0000","last_peered":"2024-04-25T13:39:47.663853+0000","last_clean":"2024-04-25T13:39:47.663853+0000","last_became_active":"2024-04-25T13:39:45.273994+0000","last_became_peered":"2024-04-25T13:39:45.273994+0000","last_unstale":"2024-04-25T13:39:47.663853+0000","last_undegraded":"2024-04-25T13:39:47.663853+0000","last_fullsized":"2024-04-25T13:39:47.663853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:28.084491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613444+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:39:47.613444+0000","last_peered":"2024-04-25T13:39:47.613444+0000","last_clean":"2024-04-25T13:39:47.613444+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:39:47.613444+0000","last_undegraded":"2024-04-25T13:39:47.613444+0000","last_fullsized":"2024-04-25T13:39:47.613444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689556+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:39:46.689556+0000","last_peered":"2024-04-25T13:39:46.689556+0000","last_clean":"2024-04-25T13:39:46.689556+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:39:46.689556+0000","last_undegraded":"2024-04-25T13:39:46.689556+0000","last_fullsized":"2024-04-25T13:39:46.689556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251633+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:39:46.251633+0000","last_peered":"2024-04-25T13:39:46.251633+0000","last_clean":"2024-04-25T13:39:46.251633+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:39:46.251633+0000","last_undegraded":"2024-04-25T13:39:46.251633+0000","last_fullsized":"2024-04-25T13:39:46.251633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689300+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:39:46.689300+0000","last_peered":"2024-04-25T13:39:46.689300+0000","last_clean":"2024-04-25T13:39:46.689300+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:39:46.689300+0000","last_undegraded":"2024-04-25T13:39:46.689300+0000","last_fullsized":"2024-04-25T13:39:46.689300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249203+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:39:46.249203+0000","last_peered":"2024-04-25T13:39:46.249203+0000","last_clean":"2024-04-25T13:39:46.249203+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:39:46.249203+0000","last_undegraded":"2024-04-25T13:39:46.249203+0000","last_fullsized":"2024-04-25T13:39:46.249203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.676863+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:39:48.676863+0000","last_peered":"2024-04-25T13:39:48.676863+0000","last_clean":"2024-04-25T13:39:48.676863+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:39:48.676863+0000","last_undegraded":"2024-04-25T13:39:48.676863+0000","last_fullsized":"2024-04-25T13:39:48.676863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615746+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:39:47.615746+0000","last_peered":"2024-04-25T13:39:47.615746+0000","last_clean":"2024-04-25T13:39:47.615746+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:39:47.615746+0000","last_undegraded":"2024-04-25T13:39:47.615746+0000","last_fullsized":"2024-04-25T13:39:47.615746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664028+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:39:47.664028+0000","last_peered":"2024-04-25T13:39:47.664028+0000","last_clean":"2024-04-25T13:39:47.664028+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:39:47.664028+0000","last_undegraded":"2024-04-25T13:39:47.664028+0000","last_fullsized":"2024-04-25T13:39:47.664028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":92,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251238+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:39:46.251238+0000","last_peered":"2024-04-25T13:39:46.251238+0000","last_clean":"2024-04-25T13:39:46.251238+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:39:46.251238+0000","last_undegraded":"2024-04-25T13:39:46.251238+0000","last_fullsized":"2024-04-25T13:39:46.251238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248065+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:39:46.248065+0000","last_peered":"2024-04-25T13:39:46.248065+0000","last_clean":"2024-04-25T13:39:46.248065+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:39:46.248065+0000","last_undegraded":"2024-04-25T13:39:46.248065+0000","last_fullsized":"2024-04-25T13:39:46.248065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677164+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:39:48.677164+0000","last_peered":"2024-04-25T13:39:48.677164+0000","last_clean":"2024-04-25T13:39:48.677164+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:39:48.677164+0000","last_undegraded":"2024-04-25T13:39:48.677164+0000","last_fullsized":"2024-04-25T13:39:48.677164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662372+0000","last_change":"2024-04-25T13:39:44.625497+0000","last_active":"2024-04-25T13:39:47.662372+0000","last_peered":"2024-04-25T13:39:47.662372+0000","last_clean":"2024-04-25T13:39:47.662372+0000","last_became_active":"2024-04-25T13:39:44.624438+0000","last_became_peered":"2024-04-25T13:39:44.624438+0000","last_unstale":"2024-04-25T13:39:47.662372+0000","last_undegraded":"2024-04-25T13:39:47.662372+0000","last_fullsized":"2024-04-25T13:39:47.662372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:24.719826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248549+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:39:46.248549+0000","last_peered":"2024-04-25T13:39:46.248549+0000","last_clean":"2024-04-25T13:39:46.248549+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:39:46.248549+0000","last_undegraded":"2024-04-25T13:39:46.248549+0000","last_fullsized":"2024-04-25T13:39:46.248549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677205+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:39:48.677205+0000","last_peered":"2024-04-25T13:39:48.677205+0000","last_clean":"2024-04-25T13:39:48.677205+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:39:48.677205+0000","last_undegraded":"2024-04-25T13:39:48.677205+0000","last_fullsized":"2024-04-25T13:39:48.677205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615828+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:39:47.615828+0000","last_peered":"2024-04-25T13:39:47.615828+0000","last_clean":"2024-04-25T13:39:47.615828+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:39:47.615828+0000","last_undegraded":"2024-04-25T13:39:47.615828+0000","last_fullsized":"2024-04-25T13:39:47.615828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663263+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:39:47.663263+0000","last_peered":"2024-04-25T13:39:47.663263+0000","last_clean":"2024-04-25T13:39:47.663263+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:39:47.663263+0000","last_undegraded":"2024-04-25T13:39:47.663263+0000","last_fullsized":"2024-04-25T13:39:47.663263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660237+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:39:47.660237+0000","last_peered":"2024-04-25T13:39:47.660237+0000","last_clean":"2024-04-25T13:39:47.660237+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:39:47.660237+0000","last_undegraded":"2024-04-25T13:39:47.660237+0000","last_fullsized":"2024-04-25T13:39:47.660237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662949+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:39:47.662949+0000","last_peered":"2024-04-25T13:39:47.662949+0000","last_clean":"2024-04-25T13:39:47.662949+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:39:47.662949+0000","last_undegraded":"2024-04-25T13:39:47.662949+0000","last_fullsized":"2024-04-25T13:39:47.662949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616250+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:39:47.616250+0000","last_peered":"2024-04-25T13:39:47.616250+0000","last_clean":"2024-04-25T13:39:47.616250+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:39:47.616250+0000","last_undegraded":"2024-04-25T13:39:47.616250+0000","last_fullsized":"2024-04-25T13:39:47.616250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662815+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:39:47.662815+0000","last_peered":"2024-04-25T13:39:47.662815+0000","last_clean":"2024-04-25T13:39:47.662815+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:39:47.662815+0000","last_undegraded":"2024-04-25T13:39:47.662815+0000","last_fullsized":"2024-04-25T13:39:47.662815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663598+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:39:47.663598+0000","last_peered":"2024-04-25T13:39:47.663598+0000","last_clean":"2024-04-25T13:39:47.663598+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:39:47.663598+0000","last_undegraded":"2024-04-25T13:39:47.663598+0000","last_fullsized":"2024-04-25T13:39:47.663598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538349+0000","last_change":"2024-04-25T13:39:44.626339+0000","last_active":"2024-04-25T13:39:46.538349+0000","last_peered":"2024-04-25T13:39:46.538349+0000","last_clean":"2024-04-25T13:39:46.538349+0000","last_became_active":"2024-04-25T13:39:44.626026+0000","last_became_peered":"2024-04-25T13:39:44.626026+0000","last_unstale":"2024-04-25T13:39:46.538349+0000","last_undegraded":"2024-04-25T13:39:46.538349+0000","last_fullsized":"2024-04-25T13:39:46.538349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:22.415965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615820+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:39:47.615820+0000","last_peered":"2024-04-25T13:39:47.615820+0000","last_clean":"2024-04-25T13:39:47.615820+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:39:47.615820+0000","last_undegraded":"2024-04-25T13:39:47.615820+0000","last_fullsized":"2024-04-25T13:39:47.615820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663316+0000","last_change":"2024-04-25T13:39:44.626101+0000","last_active":"2024-04-25T13:39:47.663316+0000","last_peered":"2024-04-25T13:39:47.663316+0000","last_clean":"2024-04-25T13:39:47.663316+0000","last_became_active":"2024-04-25T13:39:44.625735+0000","last_became_peered":"2024-04-25T13:39:44.625735+0000","last_unstale":"2024-04-25T13:39:47.663316+0000","last_undegraded":"2024-04-25T13:39:47.663316+0000","last_fullsized":"2024-04-25T13:39:47.663316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.402845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251813+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:39:46.251813+0000","last_peered":"2024-04-25T13:39:46.251813+0000","last_clean":"2024-04-25T13:39:46.251813+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:39:46.251813+0000","last_undegraded":"2024-04-25T13:39:46.251813+0000","last_fullsized":"2024-04-25T13:39:46.251813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677267+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:39:48.677267+0000","last_peered":"2024-04-25T13:39:48.677267+0000","last_clean":"2024-04-25T13:39:48.677267+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:39:48.677267+0000","last_undegraded":"2024-04-25T13:39:48.677267+0000","last_fullsized":"2024-04-25T13:39:48.677267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251999+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:39:46.251999+0000","last_peered":"2024-04-25T13:39:46.251999+0000","last_clean":"2024-04-25T13:39:46.251999+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:39:46.251999+0000","last_undegraded":"2024-04-25T13:39:46.251999+0000","last_fullsized":"2024-04-25T13:39:46.251999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249674+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:39:46.249674+0000","last_peered":"2024-04-25T13:39:46.249674+0000","last_clean":"2024-04-25T13:39:46.249674+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:39:46.249674+0000","last_undegraded":"2024-04-25T13:39:46.249674+0000","last_fullsized":"2024-04-25T13:39:46.249674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251826+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:39:46.251826+0000","last_peered":"2024-04-25T13:39:46.251826+0000","last_clean":"2024-04-25T13:39:46.251826+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:39:46.251826+0000","last_undegraded":"2024-04-25T13:39:46.251826+0000","last_fullsized":"2024-04-25T13:39:46.251826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248464+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:39:46.248464+0000","last_peered":"2024-04-25T13:39:46.248464+0000","last_clean":"2024-04-25T13:39:46.248464+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:39:46.248464+0000","last_undegraded":"2024-04-25T13:39:46.248464+0000","last_fullsized":"2024-04-25T13:39:46.248464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251966+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:39:46.251966+0000","last_peered":"2024-04-25T13:39:46.251966+0000","last_clean":"2024-04-25T13:39:46.251966+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:39:46.251966+0000","last_undegraded":"2024-04-25T13:39:46.251966+0000","last_fullsized":"2024-04-25T13:39:46.251966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663603+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:39:47.663603+0000","last_peered":"2024-04-25T13:39:47.663603+0000","last_clean":"2024-04-25T13:39:47.663603+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:39:47.663603+0000","last_undegraded":"2024-04-25T13:39:47.663603+0000","last_fullsized":"2024-04-25T13:39:47.663603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538589+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:39:46.538589+0000","last_peered":"2024-04-25T13:39:46.538589+0000","last_clean":"2024-04-25T13:39:46.538589+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:39:46.538589+0000","last_undegraded":"2024-04-25T13:39:46.538589+0000","last_fullsized":"2024-04-25T13:39:46.538589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538577+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:39:46.538577+0000","last_peered":"2024-04-25T13:39:46.538577+0000","last_clean":"2024-04-25T13:39:46.538577+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:39:46.538577+0000","last_undegraded":"2024-04-25T13:39:46.538577+0000","last_fullsized":"2024-04-25T13:39:46.538577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660063+0000","last_change":"2024-04-25T13:39:45.269647+0000","last_active":"2024-04-25T13:39:47.660063+0000","last_peered":"2024-04-25T13:39:47.660063+0000","last_clean":"2024-04-25T13:39:47.660063+0000","last_became_active":"2024-04-25T13:39:45.269314+0000","last_became_peered":"2024-04-25T13:39:45.269314+0000","last_unstale":"2024-04-25T13:39:47.660063+0000","last_undegraded":"2024-04-25T13:39:47.660063+0000","last_fullsized":"2024-04-25T13:39:47.660063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:26.667263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248413+0000","last_change":"2024-04-25T13:39:45.261199+0000","last_active":"2024-04-25T13:39:46.248413+0000","last_peered":"2024-04-25T13:39:46.248413+0000","last_clean":"2024-04-25T13:39:46.248413+0000","last_became_active":"2024-04-25T13:39:45.260867+0000","last_became_peered":"2024-04-25T13:39:45.260867+0000","last_unstale":"2024-04-25T13:39:46.248413+0000","last_undegraded":"2024-04-25T13:39:46.248413+0000","last_fullsized":"2024-04-25T13:39:46.248413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:47.043364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663983+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:39:47.663983+0000","last_peered":"2024-04-25T13:39:47.663983+0000","last_clean":"2024-04-25T13:39:47.663983+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:39:47.663983+0000","last_undegraded":"2024-04-25T13:39:47.663983+0000","last_fullsized":"2024-04-25T13:39:47.663983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539065+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:39:46.539065+0000","last_peered":"2024-04-25T13:39:46.539065+0000","last_clean":"2024-04-25T13:39:46.539065+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:39:46.539065+0000","last_undegraded":"2024-04-25T13:39:46.539065+0000","last_fullsized":"2024-04-25T13:39:46.539065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615861+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:39:47.615861+0000","last_peered":"2024-04-25T13:39:47.615861+0000","last_clean":"2024-04-25T13:39:47.615861+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:39:47.615861+0000","last_undegraded":"2024-04-25T13:39:47.615861+0000","last_fullsized":"2024-04-25T13:39:47.615861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249256+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:39:46.249256+0000","last_peered":"2024-04-25T13:39:46.249256+0000","last_clean":"2024-04-25T13:39:46.249256+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:39:46.249256+0000","last_undegraded":"2024-04-25T13:39:46.249256+0000","last_fullsized":"2024-04-25T13:39:46.249256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615480+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:39:47.615480+0000","last_peered":"2024-04-25T13:39:47.615480+0000","last_clean":"2024-04-25T13:39:47.615480+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:39:47.615480+0000","last_undegraded":"2024-04-25T13:39:47.615480+0000","last_fullsized":"2024-04-25T13:39:47.615480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538877+0000","last_change":"2024-04-25T13:39:45.259273+0000","last_active":"2024-04-25T13:39:46.538877+0000","last_peered":"2024-04-25T13:39:46.538877+0000","last_clean":"2024-04-25T13:39:46.538877+0000","last_became_active":"2024-04-25T13:39:45.258905+0000","last_became_peered":"2024-04-25T13:39:45.258905+0000","last_unstale":"2024-04-25T13:39:46.538877+0000","last_undegraded":"2024-04-25T13:39:46.538877+0000","last_fullsized":"2024-04-25T13:39:46.538877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:25.749581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248440+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:39:46.248440+0000","last_peered":"2024-04-25T13:39:46.248440+0000","last_clean":"2024-04-25T13:39:46.248440+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:39:46.248440+0000","last_undegraded":"2024-04-25T13:39:46.248440+0000","last_fullsized":"2024-04-25T13:39:46.248440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663935+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:39:47.663935+0000","last_peered":"2024-04-25T13:39:47.663935+0000","last_clean":"2024-04-25T13:39:47.663935+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:39:47.663935+0000","last_undegraded":"2024-04-25T13:39:47.663935+0000","last_fullsized":"2024-04-25T13:39:47.663935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24483,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":35481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":404913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1017413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972376,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40340,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704812,"statfs":{"total":95995035648,"available":95953727488,"internally_reserved":0,"allocated":864256,"data_stored":405149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168605,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":404957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1449984,"data_stored":995903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":843776,"data_stored":404331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":856064,"data_stored":405943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":405809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":406599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":856064,"data_stored":404525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":404975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36772,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708380,"statfs":{"total":95995035648,"available":95957381120,"internally_reserved":0,"allocated":1466368,"data_stored":996175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":22597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:39:52.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:39:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- 172.21.15.94:0/1880250772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 msgr2=0x7f15e4103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 --2- 172.21.15.94:0/1880250772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e4103710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009900 tx=0x7f15e002a410 comp rx=0 tx=0).stop 2024-04-25T13:39:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- 172.21.15.94:0/1880250772 shutdown_connections 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 --2- 172.21.15.94:0/1880250772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 0x7f15e410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 --2- 172.21.15.94:0/1880250772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 0x7f15e4104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 --2- 172.21.15.94:0/1880250772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e4103710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- 172.21.15.94:0/1880250772 >> 172.21.15.94:0/1880250772 conn(0x7f15e40fdc00 msgr2=0x7f15e41000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- 172.21.15.94:0/1880250772 shutdown_connections 2024-04-25T13:39:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- 172.21.15.94:0/1880250772 wait complete. 2024-04-25T13:39:52.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 Processor -- start 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.619+0000 7f15ebccd640 1 -- start start 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 0x7f15e419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 0x7f15e41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15e410e510 con 0x7f15e4102a10 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15e410e390 con 0x7f15e4108950 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15e410e690 con 0x7f15e4103c50 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42970/0 (socket says 172.21.15.94:42970) 2024-04-25T13:39:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 -- 172.21.15.94:0/3449918204 learned_addr learned my addr 172.21.15.94:0/3449918204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9268640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 0x7f15e419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ea26a640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 0x7f15e41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 -- 172.21.15.94:0/3449918204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 msgr2=0x7f15e419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 0x7f15e419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 -- 172.21.15.94:0/3449918204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 msgr2=0x7f15e41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 0x7f15e41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 -- 172.21.15.94:0/3449918204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41a1e00 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15e9a69640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15e0006010 tx=0x7f15e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e003f030 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e0031910 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41a2090 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.620+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0035070 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.621+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a25d0 con 0x7f15e4102a10 2024-04-25T13:39:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.622+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e41a2830 con 0x7f15e4102a10 2024-04-25T13:39:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.623+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f15e0042030 con 0x7f15e4102a10 2024-04-25T13:39:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.624+0000 7f15daffd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 0x7f15d00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.624+0000 7f15e9268640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 0x7f15d00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:52.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.624+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f15e00ff250 con 0x7f15e4102a10 2024-04-25T13:39:52.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.624+0000 7f15e9268640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 0x7f15d00b2110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15e419b360 tx=0x7f15d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.628+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15e00c7830 con 0x7f15e4102a10 2024-04-25T13:39:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:52 smithi094 ceph-mon[29195]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:52 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/436400871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:52 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1301517747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:52 smithi094 ceph-mon[29195]: pgmap v285: 97 pgs: 97 active+clean; 601 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 17 op/s 2024-04-25T13:39:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.807+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15e410a300 con 0x7f15d00afc00 2024-04-25T13:39:52.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.814+0000 7f15daffd640 1 -- 172.21.15.94:0/3449918204 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7f15e410a300 con 0x7f15d00afc00 2024-04-25T13:39:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:39:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 msgr2=0x7f15d00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 0x7f15d00b2110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15e419b360 tx=0x7f15d4005f90 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 msgr2=0x7f15e419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15e0006010 tx=0x7f15e0004270 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 shutdown_connections 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15d00afc00 0x7f15d00b2110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15e4108950 0x7f15e41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15e4103c50 0x7f15e419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 --2- 172.21.15.94:0/3449918204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15e4102a10 0x7f15e419a2c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 >> 172.21.15.94:0/3449918204 conn(0x7f15e40fdc00 msgr2=0x7f15e4109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 shutdown_connections 2024-04-25T13:39:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:52.817+0000 7f15ebccd640 1 -- 172.21.15.94:0/3449918204 wait complete. 2024-04-25T13:39:52.829 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-04-25T13:39:52.257169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614968,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":254,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":249,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398488,"kb_used_data":11864,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124543336,"statfs":{"total":1151940427776,"available":1151532376064,"internally_reserved":0,"allocated":12148736,"data_stored":6656692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24600,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.651227"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664300+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:39:47.664300+0000","last_peered":"2024-04-25T13:39:47.664300+0000","last_clean":"2024-04-25T13:39:47.664300+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:39:47.664300+0000","last_undegraded":"2024-04-25T13:39:47.664300+0000","last_fullsized":"2024-04-25T13:39:47.664300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660335+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:39:47.660335+0000","last_peered":"2024-04-25T13:39:47.660335+0000","last_clean":"2024-04-25T13:39:47.660335+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:39:47.660335+0000","last_undegraded":"2024-04-25T13:39:47.660335+0000","last_fullsized":"2024-04-25T13:39:47.660335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660313+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:39:47.660313+0000","last_peered":"2024-04-25T13:39:47.660313+0000","last_clean":"2024-04-25T13:39:47.660313+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:39:47.660313+0000","last_undegraded":"2024-04-25T13:39:47.660313+0000","last_fullsized":"2024-04-25T13:39:47.660313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660340+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:39:47.660340+0000","last_peered":"2024-04-25T13:39:47.660340+0000","last_clean":"2024-04-25T13:39:47.660340+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:39:47.660340+0000","last_undegraded":"2024-04-25T13:39:47.660340+0000","last_fullsized":"2024-04-25T13:39:47.660340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616318+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:39:47.616318+0000","last_peered":"2024-04-25T13:39:47.616318+0000","last_clean":"2024-04-25T13:39:47.616318+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:39:47.616318+0000","last_undegraded":"2024-04-25T13:39:47.616318+0000","last_fullsized":"2024-04-25T13:39:47.616318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689172+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:39:46.689172+0000","last_peered":"2024-04-25T13:39:46.689172+0000","last_clean":"2024-04-25T13:39:46.689172+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:39:46.689172+0000","last_undegraded":"2024-04-25T13:39:46.689172+0000","last_fullsized":"2024-04-25T13:39:46.689172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251560+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:39:46.251560+0000","last_peered":"2024-04-25T13:39:46.251560+0000","last_clean":"2024-04-25T13:39:46.251560+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:39:46.251560+0000","last_undegraded":"2024-04-25T13:39:46.251560+0000","last_fullsized":"2024-04-25T13:39:46.251560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615391+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:39:47.615391+0000","last_peered":"2024-04-25T13:39:47.615391+0000","last_clean":"2024-04-25T13:39:47.615391+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:39:47.615391+0000","last_undegraded":"2024-04-25T13:39:47.615391+0000","last_fullsized":"2024-04-25T13:39:47.615391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539267+0000","last_change":"2024-04-25T13:39:45.265065+0000","last_active":"2024-04-25T13:39:46.539267+0000","last_peered":"2024-04-25T13:39:46.539267+0000","last_clean":"2024-04-25T13:39:46.539267+0000","last_became_active":"2024-04-25T13:39:45.264734+0000","last_became_peered":"2024-04-25T13:39:45.264734+0000","last_unstale":"2024-04-25T13:39:46.539267+0000","last_undegraded":"2024-04-25T13:39:46.539267+0000","last_fullsized":"2024-04-25T13:39:46.539267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689607+0000","last_change":"2024-04-25T13:39:45.268786+0000","last_active":"2024-04-25T13:39:46.689607+0000","last_peered":"2024-04-25T13:39:46.689607+0000","last_clean":"2024-04-25T13:39:46.689607+0000","last_became_active":"2024-04-25T13:39:45.268384+0000","last_became_peered":"2024-04-25T13:39:45.268384+0000","last_unstale":"2024-04-25T13:39:46.689607+0000","last_undegraded":"2024-04-25T13:39:46.689607+0000","last_fullsized":"2024-04-25T13:39:46.689607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:14.179367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251972+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:39:46.251972+0000","last_peered":"2024-04-25T13:39:46.251972+0000","last_clean":"2024-04-25T13:39:46.251972+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:39:46.251972+0000","last_undegraded":"2024-04-25T13:39:46.251972+0000","last_fullsized":"2024-04-25T13:39:46.251972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.661336+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:39:47.661336+0000","last_peered":"2024-04-25T13:39:47.661336+0000","last_clean":"2024-04-25T13:39:47.661336+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:39:47.661336+0000","last_undegraded":"2024-04-25T13:39:47.661336+0000","last_fullsized":"2024-04-25T13:39:47.661336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248359+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:39:46.248359+0000","last_peered":"2024-04-25T13:39:46.248359+0000","last_clean":"2024-04-25T13:39:46.248359+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:39:46.248359+0000","last_undegraded":"2024-04-25T13:39:46.248359+0000","last_fullsized":"2024-04-25T13:39:46.248359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.659790+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:39:47.659790+0000","last_peered":"2024-04-25T13:39:47.659790+0000","last_clean":"2024-04-25T13:39:47.659790+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:39:47.659790+0000","last_undegraded":"2024-04-25T13:39:47.659790+0000","last_fullsized":"2024-04-25T13:39:47.659790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.659501+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:39:47.659501+0000","last_peered":"2024-04-25T13:39:47.659501+0000","last_clean":"2024-04-25T13:39:47.659501+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:39:47.659501+0000","last_undegraded":"2024-04-25T13:39:47.659501+0000","last_fullsized":"2024-04-25T13:39:47.659501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538737+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:39:46.538737+0000","last_peered":"2024-04-25T13:39:46.538737+0000","last_clean":"2024-04-25T13:39:46.538737+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:39:46.538737+0000","last_undegraded":"2024-04-25T13:39:46.538737+0000","last_fullsized":"2024-04-25T13:39:46.538737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664374+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:39:47.664374+0000","last_peered":"2024-04-25T13:39:47.664374+0000","last_clean":"2024-04-25T13:39:47.664374+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:39:47.664374+0000","last_undegraded":"2024-04-25T13:39:47.664374+0000","last_fullsized":"2024-04-25T13:39:47.664374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613658+0000","last_change":"2024-04-25T13:39:45.259538+0000","last_active":"2024-04-25T13:39:47.613658+0000","last_peered":"2024-04-25T13:39:47.613658+0000","last_clean":"2024-04-25T13:39:47.613658+0000","last_became_active":"2024-04-25T13:39:45.259097+0000","last_became_peered":"2024-04-25T13:39:45.259097+0000","last_unstale":"2024-04-25T13:39:47.613658+0000","last_undegraded":"2024-04-25T13:39:47.613658+0000","last_fullsized":"2024-04-25T13:39:47.613658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:10.901989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538905+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:39:46.538905+0000","last_peered":"2024-04-25T13:39:46.538905+0000","last_clean":"2024-04-25T13:39:46.538905+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:39:46.538905+0000","last_undegraded":"2024-04-25T13:39:46.538905+0000","last_fullsized":"2024-04-25T13:39:46.538905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.678534+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:39:48.678534+0000","last_peered":"2024-04-25T13:39:48.678534+0000","last_clean":"2024-04-25T13:39:48.678534+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:39:48.678534+0000","last_undegraded":"2024-04-25T13:39:48.678534+0000","last_fullsized":"2024-04-25T13:39:48.678534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615946+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:39:47.615946+0000","last_peered":"2024-04-25T13:39:47.615946+0000","last_clean":"2024-04-25T13:39:47.615946+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:39:47.615946+0000","last_undegraded":"2024-04-25T13:39:47.615946+0000","last_fullsized":"2024-04-25T13:39:47.615946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615961+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:39:47.615961+0000","last_peered":"2024-04-25T13:39:47.615961+0000","last_clean":"2024-04-25T13:39:47.615961+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:39:47.615961+0000","last_undegraded":"2024-04-25T13:39:47.615961+0000","last_fullsized":"2024-04-25T13:39:47.615961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660430+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:39:47.660430+0000","last_peered":"2024-04-25T13:39:47.660430+0000","last_clean":"2024-04-25T13:39:47.660430+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:39:47.660430+0000","last_undegraded":"2024-04-25T13:39:47.660430+0000","last_fullsized":"2024-04-25T13:39:47.660430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616562+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:39:47.616562+0000","last_peered":"2024-04-25T13:39:47.616562+0000","last_clean":"2024-04-25T13:39:47.616562+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:39:47.616562+0000","last_undegraded":"2024-04-25T13:39:47.616562+0000","last_fullsized":"2024-04-25T13:39:47.616562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538947+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:39:46.538947+0000","last_peered":"2024-04-25T13:39:46.538947+0000","last_clean":"2024-04-25T13:39:46.538947+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:39:46.538947+0000","last_undegraded":"2024-04-25T13:39:46.538947+0000","last_fullsized":"2024-04-25T13:39:46.538947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616517+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:39:47.616517+0000","last_peered":"2024-04-25T13:39:47.616517+0000","last_clean":"2024-04-25T13:39:47.616517+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:39:47.616517+0000","last_undegraded":"2024-04-25T13:39:47.616517+0000","last_fullsized":"2024-04-25T13:39:47.616517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249743+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:39:46.249743+0000","last_peered":"2024-04-25T13:39:46.249743+0000","last_clean":"2024-04-25T13:39:46.249743+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:39:46.249743+0000","last_undegraded":"2024-04-25T13:39:46.249743+0000","last_fullsized":"2024-04-25T13:39:46.249743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664521+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:39:47.664521+0000","last_peered":"2024-04-25T13:39:47.664521+0000","last_clean":"2024-04-25T13:39:47.664521+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:39:47.664521+0000","last_undegraded":"2024-04-25T13:39:47.664521+0000","last_fullsized":"2024-04-25T13:39:47.664521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616189+0000","last_change":"2024-04-25T13:39:45.262172+0000","last_active":"2024-04-25T13:39:47.616189+0000","last_peered":"2024-04-25T13:39:47.616189+0000","last_clean":"2024-04-25T13:39:47.616189+0000","last_became_active":"2024-04-25T13:39:45.261838+0000","last_became_peered":"2024-04-25T13:39:45.261838+0000","last_unstale":"2024-04-25T13:39:47.616189+0000","last_undegraded":"2024-04-25T13:39:47.616189+0000","last_fullsized":"2024-04-25T13:39:47.616189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:09.938897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:49.679774+0000","last_change":"2024-04-25T13:39:44.625364+0000","last_active":"2024-04-25T13:39:49.679774+0000","last_peered":"2024-04-25T13:39:49.679774+0000","last_clean":"2024-04-25T13:39:49.679774+0000","last_became_active":"2024-04-25T13:39:44.625014+0000","last_became_peered":"2024-04-25T13:39:44.625014+0000","last_unstale":"2024-04-25T13:39:49.679774+0000","last_undegraded":"2024-04-25T13:39:49.679774+0000","last_fullsized":"2024-04-25T13:39:49.679774+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:36.507525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249607+0000","last_change":"2024-04-25T13:39:45.268193+0000","last_active":"2024-04-25T13:39:46.249607+0000","last_peered":"2024-04-25T13:39:46.249607+0000","last_clean":"2024-04-25T13:39:46.249607+0000","last_became_active":"2024-04-25T13:39:45.267842+0000","last_became_peered":"2024-04-25T13:39:45.267842+0000","last_unstale":"2024-04-25T13:39:46.249607+0000","last_undegraded":"2024-04-25T13:39:46.249607+0000","last_fullsized":"2024-04-25T13:39:46.249607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:56.308127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.675645+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:39:48.675645+0000","last_peered":"2024-04-25T13:39:48.675645+0000","last_clean":"2024-04-25T13:39:48.675645+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:39:48.675645+0000","last_undegraded":"2024-04-25T13:39:48.675645+0000","last_fullsized":"2024-04-25T13:39:48.675645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615344+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:39:47.615344+0000","last_peered":"2024-04-25T13:39:47.615344+0000","last_clean":"2024-04-25T13:39:47.615344+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:39:47.615344+0000","last_undegraded":"2024-04-25T13:39:47.615344+0000","last_fullsized":"2024-04-25T13:39:47.615344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660704+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:39:47.660704+0000","last_peered":"2024-04-25T13:39:47.660704+0000","last_clean":"2024-04-25T13:39:47.660704+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:39:47.660704+0000","last_undegraded":"2024-04-25T13:39:47.660704+0000","last_fullsized":"2024-04-25T13:39:47.660704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613513+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:39:47.613513+0000","last_peered":"2024-04-25T13:39:47.613513+0000","last_clean":"2024-04-25T13:39:47.613513+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:39:47.613513+0000","last_undegraded":"2024-04-25T13:39:47.613513+0000","last_fullsized":"2024-04-25T13:39:47.613513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662962+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:39:47.662962+0000","last_peered":"2024-04-25T13:39:47.662962+0000","last_clean":"2024-04-25T13:39:47.662962+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:39:47.662962+0000","last_undegraded":"2024-04-25T13:39:47.662962+0000","last_fullsized":"2024-04-25T13:39:47.662962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539290+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:39:46.539290+0000","last_peered":"2024-04-25T13:39:46.539290+0000","last_clean":"2024-04-25T13:39:46.539290+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:39:46.539290+0000","last_undegraded":"2024-04-25T13:39:46.539290+0000","last_fullsized":"2024-04-25T13:39:46.539290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662925+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:39:47.662925+0000","last_peered":"2024-04-25T13:39:47.662925+0000","last_clean":"2024-04-25T13:39:47.662925+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:39:47.662925+0000","last_undegraded":"2024-04-25T13:39:47.662925+0000","last_fullsized":"2024-04-25T13:39:47.662925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660412+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:39:47.660412+0000","last_peered":"2024-04-25T13:39:47.660412+0000","last_clean":"2024-04-25T13:39:47.660412+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:39:47.660412+0000","last_undegraded":"2024-04-25T13:39:47.660412+0000","last_fullsized":"2024-04-25T13:39:47.660412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662087+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:39:47.662087+0000","last_peered":"2024-04-25T13:39:47.662087+0000","last_clean":"2024-04-25T13:39:47.662087+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:39:47.662087+0000","last_undegraded":"2024-04-25T13:39:47.662087+0000","last_fullsized":"2024-04-25T13:39:47.662087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539104+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:39:46.539104+0000","last_peered":"2024-04-25T13:39:46.539104+0000","last_clean":"2024-04-25T13:39:46.539104+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:39:46.539104+0000","last_undegraded":"2024-04-25T13:39:46.539104+0000","last_fullsized":"2024-04-25T13:39:46.539104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'15","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:50.389148+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:39:50.389148+0000","last_peered":"2024-04-25T13:39:50.389148+0000","last_clean":"2024-04-25T13:39:50.389148+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:39:50.389148+0000","last_undegraded":"2024-04-25T13:39:50.389148+0000","last_fullsized":"2024-04-25T13:39:50.389148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616425+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:39:47.616425+0000","last_peered":"2024-04-25T13:39:47.616425+0000","last_clean":"2024-04-25T13:39:47.616425+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:39:47.616425+0000","last_undegraded":"2024-04-25T13:39:47.616425+0000","last_fullsized":"2024-04-25T13:39:47.616425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.676062+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:39:48.676062+0000","last_peered":"2024-04-25T13:39:48.676062+0000","last_clean":"2024-04-25T13:39:48.676062+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:39:48.676062+0000","last_undegraded":"2024-04-25T13:39:48.676062+0000","last_fullsized":"2024-04-25T13:39:48.676062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248183+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:39:46.248183+0000","last_peered":"2024-04-25T13:39:46.248183+0000","last_clean":"2024-04-25T13:39:46.248183+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:39:46.248183+0000","last_undegraded":"2024-04-25T13:39:46.248183+0000","last_fullsized":"2024-04-25T13:39:46.248183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.661193+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:39:47.661193+0000","last_peered":"2024-04-25T13:39:47.661193+0000","last_clean":"2024-04-25T13:39:47.661193+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:39:47.661193+0000","last_undegraded":"2024-04-25T13:39:47.661193+0000","last_fullsized":"2024-04-25T13:39:47.661193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251832+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:39:46.251832+0000","last_peered":"2024-04-25T13:39:46.251832+0000","last_clean":"2024-04-25T13:39:46.251832+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:39:46.251832+0000","last_undegraded":"2024-04-25T13:39:46.251832+0000","last_fullsized":"2024-04-25T13:39:46.251832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.675014+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:39:48.675014+0000","last_peered":"2024-04-25T13:39:48.675014+0000","last_clean":"2024-04-25T13:39:48.675014+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:39:48.675014+0000","last_undegraded":"2024-04-25T13:39:48.675014+0000","last_fullsized":"2024-04-25T13:39:48.675014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251347+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:39:46.251347+0000","last_peered":"2024-04-25T13:39:46.251347+0000","last_clean":"2024-04-25T13:39:46.251347+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:39:46.251347+0000","last_undegraded":"2024-04-25T13:39:46.251347+0000","last_fullsized":"2024-04-25T13:39:46.251347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'10","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663300+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:39:47.663300+0000","last_peered":"2024-04-25T13:39:47.663300+0000","last_clean":"2024-04-25T13:39:47.663300+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:39:47.663300+0000","last_undegraded":"2024-04-25T13:39:47.663300+0000","last_fullsized":"2024-04-25T13:39:47.663300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249214+0000","last_change":"2024-04-25T13:39:45.273219+0000","last_active":"2024-04-25T13:39:46.249214+0000","last_peered":"2024-04-25T13:39:46.249214+0000","last_clean":"2024-04-25T13:39:46.249214+0000","last_became_active":"2024-04-25T13:39:45.272891+0000","last_became_peered":"2024-04-25T13:39:45.272891+0000","last_unstale":"2024-04-25T13:39:46.249214+0000","last_undegraded":"2024-04-25T13:39:46.249214+0000","last_fullsized":"2024-04-25T13:39:46.249214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:40.531758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616513+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:39:47.616513+0000","last_peered":"2024-04-25T13:39:47.616513+0000","last_clean":"2024-04-25T13:39:47.616513+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:39:47.616513+0000","last_undegraded":"2024-04-25T13:39:47.616513+0000","last_fullsized":"2024-04-25T13:39:47.616513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663853+0000","last_change":"2024-04-25T13:39:45.274393+0000","last_active":"2024-04-25T13:39:47.663853+0000","last_peered":"2024-04-25T13:39:47.663853+0000","last_clean":"2024-04-25T13:39:47.663853+0000","last_became_active":"2024-04-25T13:39:45.273994+0000","last_became_peered":"2024-04-25T13:39:45.273994+0000","last_unstale":"2024-04-25T13:39:47.663853+0000","last_undegraded":"2024-04-25T13:39:47.663853+0000","last_fullsized":"2024-04-25T13:39:47.663853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:28.084491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.613444+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:39:47.613444+0000","last_peered":"2024-04-25T13:39:47.613444+0000","last_clean":"2024-04-25T13:39:47.613444+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:39:47.613444+0000","last_undegraded":"2024-04-25T13:39:47.613444+0000","last_fullsized":"2024-04-25T13:39:47.613444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689556+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:39:46.689556+0000","last_peered":"2024-04-25T13:39:46.689556+0000","last_clean":"2024-04-25T13:39:46.689556+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:39:46.689556+0000","last_undegraded":"2024-04-25T13:39:46.689556+0000","last_fullsized":"2024-04-25T13:39:46.689556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251633+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:39:46.251633+0000","last_peered":"2024-04-25T13:39:46.251633+0000","last_clean":"2024-04-25T13:39:46.251633+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:39:46.251633+0000","last_undegraded":"2024-04-25T13:39:46.251633+0000","last_fullsized":"2024-04-25T13:39:46.251633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.689300+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:39:46.689300+0000","last_peered":"2024-04-25T13:39:46.689300+0000","last_clean":"2024-04-25T13:39:46.689300+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:39:46.689300+0000","last_undegraded":"2024-04-25T13:39:46.689300+0000","last_fullsized":"2024-04-25T13:39:46.689300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249203+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:39:46.249203+0000","last_peered":"2024-04-25T13:39:46.249203+0000","last_clean":"2024-04-25T13:39:46.249203+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:39:46.249203+0000","last_undegraded":"2024-04-25T13:39:46.249203+0000","last_fullsized":"2024-04-25T13:39:46.249203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.676863+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:39:48.676863+0000","last_peered":"2024-04-25T13:39:48.676863+0000","last_clean":"2024-04-25T13:39:48.676863+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:39:48.676863+0000","last_undegraded":"2024-04-25T13:39:48.676863+0000","last_fullsized":"2024-04-25T13:39:48.676863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615746+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:39:47.615746+0000","last_peered":"2024-04-25T13:39:47.615746+0000","last_clean":"2024-04-25T13:39:47.615746+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:39:47.615746+0000","last_undegraded":"2024-04-25T13:39:47.615746+0000","last_fullsized":"2024-04-25T13:39:47.615746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.664028+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:39:47.664028+0000","last_peered":"2024-04-25T13:39:47.664028+0000","last_clean":"2024-04-25T13:39:47.664028+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:39:47.664028+0000","last_undegraded":"2024-04-25T13:39:47.664028+0000","last_fullsized":"2024-04-25T13:39:47.664028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":92,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251238+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:39:46.251238+0000","last_peered":"2024-04-25T13:39:46.251238+0000","last_clean":"2024-04-25T13:39:46.251238+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:39:46.251238+0000","last_undegraded":"2024-04-25T13:39:46.251238+0000","last_fullsized":"2024-04-25T13:39:46.251238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248065+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:39:46.248065+0000","last_peered":"2024-04-25T13:39:46.248065+0000","last_clean":"2024-04-25T13:39:46.248065+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:39:46.248065+0000","last_undegraded":"2024-04-25T13:39:46.248065+0000","last_fullsized":"2024-04-25T13:39:46.248065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677164+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:39:48.677164+0000","last_peered":"2024-04-25T13:39:48.677164+0000","last_clean":"2024-04-25T13:39:48.677164+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:39:48.677164+0000","last_undegraded":"2024-04-25T13:39:48.677164+0000","last_fullsized":"2024-04-25T13:39:48.677164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662372+0000","last_change":"2024-04-25T13:39:44.625497+0000","last_active":"2024-04-25T13:39:47.662372+0000","last_peered":"2024-04-25T13:39:47.662372+0000","last_clean":"2024-04-25T13:39:47.662372+0000","last_became_active":"2024-04-25T13:39:44.624438+0000","last_became_peered":"2024-04-25T13:39:44.624438+0000","last_unstale":"2024-04-25T13:39:47.662372+0000","last_undegraded":"2024-04-25T13:39:47.662372+0000","last_fullsized":"2024-04-25T13:39:47.662372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:24.719826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248549+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:39:46.248549+0000","last_peered":"2024-04-25T13:39:46.248549+0000","last_clean":"2024-04-25T13:39:46.248549+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:39:46.248549+0000","last_undegraded":"2024-04-25T13:39:46.248549+0000","last_fullsized":"2024-04-25T13:39:46.248549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677205+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:39:48.677205+0000","last_peered":"2024-04-25T13:39:48.677205+0000","last_clean":"2024-04-25T13:39:48.677205+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:39:48.677205+0000","last_undegraded":"2024-04-25T13:39:48.677205+0000","last_fullsized":"2024-04-25T13:39:48.677205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615828+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:39:47.615828+0000","last_peered":"2024-04-25T13:39:47.615828+0000","last_clean":"2024-04-25T13:39:47.615828+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:39:47.615828+0000","last_undegraded":"2024-04-25T13:39:47.615828+0000","last_fullsized":"2024-04-25T13:39:47.615828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663263+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:39:47.663263+0000","last_peered":"2024-04-25T13:39:47.663263+0000","last_clean":"2024-04-25T13:39:47.663263+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:39:47.663263+0000","last_undegraded":"2024-04-25T13:39:47.663263+0000","last_fullsized":"2024-04-25T13:39:47.663263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660237+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:39:47.660237+0000","last_peered":"2024-04-25T13:39:47.660237+0000","last_clean":"2024-04-25T13:39:47.660237+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:39:47.660237+0000","last_undegraded":"2024-04-25T13:39:47.660237+0000","last_fullsized":"2024-04-25T13:39:47.660237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662949+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:39:47.662949+0000","last_peered":"2024-04-25T13:39:47.662949+0000","last_clean":"2024-04-25T13:39:47.662949+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:39:47.662949+0000","last_undegraded":"2024-04-25T13:39:47.662949+0000","last_fullsized":"2024-04-25T13:39:47.662949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.616250+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:39:47.616250+0000","last_peered":"2024-04-25T13:39:47.616250+0000","last_clean":"2024-04-25T13:39:47.616250+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:39:47.616250+0000","last_undegraded":"2024-04-25T13:39:47.616250+0000","last_fullsized":"2024-04-25T13:39:47.616250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.662815+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:39:47.662815+0000","last_peered":"2024-04-25T13:39:47.662815+0000","last_clean":"2024-04-25T13:39:47.662815+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:39:47.662815+0000","last_undegraded":"2024-04-25T13:39:47.662815+0000","last_fullsized":"2024-04-25T13:39:47.662815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663598+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:39:47.663598+0000","last_peered":"2024-04-25T13:39:47.663598+0000","last_clean":"2024-04-25T13:39:47.663598+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:39:47.663598+0000","last_undegraded":"2024-04-25T13:39:47.663598+0000","last_fullsized":"2024-04-25T13:39:47.663598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538349+0000","last_change":"2024-04-25T13:39:44.626339+0000","last_active":"2024-04-25T13:39:46.538349+0000","last_peered":"2024-04-25T13:39:46.538349+0000","last_clean":"2024-04-25T13:39:46.538349+0000","last_became_active":"2024-04-25T13:39:44.626026+0000","last_became_peered":"2024-04-25T13:39:44.626026+0000","last_unstale":"2024-04-25T13:39:46.538349+0000","last_undegraded":"2024-04-25T13:39:46.538349+0000","last_fullsized":"2024-04-25T13:39:46.538349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:22.415965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615820+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:39:47.615820+0000","last_peered":"2024-04-25T13:39:47.615820+0000","last_clean":"2024-04-25T13:39:47.615820+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:39:47.615820+0000","last_undegraded":"2024-04-25T13:39:47.615820+0000","last_fullsized":"2024-04-25T13:39:47.615820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663316+0000","last_change":"2024-04-25T13:39:44.626101+0000","last_active":"2024-04-25T13:39:47.663316+0000","last_peered":"2024-04-25T13:39:47.663316+0000","last_clean":"2024-04-25T13:39:47.663316+0000","last_became_active":"2024-04-25T13:39:44.625735+0000","last_became_peered":"2024-04-25T13:39:44.625735+0000","last_unstale":"2024-04-25T13:39:47.663316+0000","last_undegraded":"2024-04-25T13:39:47.663316+0000","last_fullsized":"2024-04-25T13:39:47.663316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.402845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251813+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:39:46.251813+0000","last_peered":"2024-04-25T13:39:46.251813+0000","last_clean":"2024-04-25T13:39:46.251813+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:39:46.251813+0000","last_undegraded":"2024-04-25T13:39:46.251813+0000","last_fullsized":"2024-04-25T13:39:46.251813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:48.677267+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:39:48.677267+0000","last_peered":"2024-04-25T13:39:48.677267+0000","last_clean":"2024-04-25T13:39:48.677267+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:39:48.677267+0000","last_undegraded":"2024-04-25T13:39:48.677267+0000","last_fullsized":"2024-04-25T13:39:48.677267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251999+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:39:46.251999+0000","last_peered":"2024-04-25T13:39:46.251999+0000","last_clean":"2024-04-25T13:39:46.251999+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:39:46.251999+0000","last_undegraded":"2024-04-25T13:39:46.251999+0000","last_fullsized":"2024-04-25T13:39:46.251999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249674+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:39:46.249674+0000","last_peered":"2024-04-25T13:39:46.249674+0000","last_clean":"2024-04-25T13:39:46.249674+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:39:46.249674+0000","last_undegraded":"2024-04-25T13:39:46.249674+0000","last_fullsized":"2024-04-25T13:39:46.249674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251826+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:39:46.251826+0000","last_peered":"2024-04-25T13:39:46.251826+0000","last_clean":"2024-04-25T13:39:46.251826+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:39:46.251826+0000","last_undegraded":"2024-04-25T13:39:46.251826+0000","last_fullsized":"2024-04-25T13:39:46.251826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248464+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:39:46.248464+0000","last_peered":"2024-04-25T13:39:46.248464+0000","last_clean":"2024-04-25T13:39:46.248464+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:39:46.248464+0000","last_undegraded":"2024-04-25T13:39:46.248464+0000","last_fullsized":"2024-04-25T13:39:46.248464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.251966+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:39:46.251966+0000","last_peered":"2024-04-25T13:39:46.251966+0000","last_clean":"2024-04-25T13:39:46.251966+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:39:46.251966+0000","last_undegraded":"2024-04-25T13:39:46.251966+0000","last_fullsized":"2024-04-25T13:39:46.251966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663603+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:39:47.663603+0000","last_peered":"2024-04-25T13:39:47.663603+0000","last_clean":"2024-04-25T13:39:47.663603+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:39:47.663603+0000","last_undegraded":"2024-04-25T13:39:47.663603+0000","last_fullsized":"2024-04-25T13:39:47.663603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538589+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:39:46.538589+0000","last_peered":"2024-04-25T13:39:46.538589+0000","last_clean":"2024-04-25T13:39:46.538589+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:39:46.538589+0000","last_undegraded":"2024-04-25T13:39:46.538589+0000","last_fullsized":"2024-04-25T13:39:46.538589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538577+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:39:46.538577+0000","last_peered":"2024-04-25T13:39:46.538577+0000","last_clean":"2024-04-25T13:39:46.538577+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:39:46.538577+0000","last_undegraded":"2024-04-25T13:39:46.538577+0000","last_fullsized":"2024-04-25T13:39:46.538577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.660063+0000","last_change":"2024-04-25T13:39:45.269647+0000","last_active":"2024-04-25T13:39:47.660063+0000","last_peered":"2024-04-25T13:39:47.660063+0000","last_clean":"2024-04-25T13:39:47.660063+0000","last_became_active":"2024-04-25T13:39:45.269314+0000","last_became_peered":"2024-04-25T13:39:45.269314+0000","last_unstale":"2024-04-25T13:39:47.660063+0000","last_undegraded":"2024-04-25T13:39:47.660063+0000","last_fullsized":"2024-04-25T13:39:47.660063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:26.667263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248413+0000","last_change":"2024-04-25T13:39:45.261199+0000","last_active":"2024-04-25T13:39:46.248413+0000","last_peered":"2024-04-25T13:39:46.248413+0000","last_clean":"2024-04-25T13:39:46.248413+0000","last_became_active":"2024-04-25T13:39:45.260867+0000","last_became_peered":"2024-04-25T13:39:45.260867+0000","last_unstale":"2024-04-25T13:39:46.248413+0000","last_undegraded":"2024-04-25T13:39:46.248413+0000","last_fullsized":"2024-04-25T13:39:46.248413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:47.043364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663983+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:39:47.663983+0000","last_peered":"2024-04-25T13:39:47.663983+0000","last_clean":"2024-04-25T13:39:47.663983+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:39:47.663983+0000","last_undegraded":"2024-04-25T13:39:47.663983+0000","last_fullsized":"2024-04-25T13:39:47.663983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.539065+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:39:46.539065+0000","last_peered":"2024-04-25T13:39:46.539065+0000","last_clean":"2024-04-25T13:39:46.539065+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:39:46.539065+0000","last_undegraded":"2024-04-25T13:39:46.539065+0000","last_fullsized":"2024-04-25T13:39:46.539065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615861+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:39:47.615861+0000","last_peered":"2024-04-25T13:39:47.615861+0000","last_clean":"2024-04-25T13:39:47.615861+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:39:47.615861+0000","last_undegraded":"2024-04-25T13:39:47.615861+0000","last_fullsized":"2024-04-25T13:39:47.615861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.249256+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:39:46.249256+0000","last_peered":"2024-04-25T13:39:46.249256+0000","last_clean":"2024-04-25T13:39:46.249256+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:39:46.249256+0000","last_undegraded":"2024-04-25T13:39:46.249256+0000","last_fullsized":"2024-04-25T13:39:46.249256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.615480+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:39:47.615480+0000","last_peered":"2024-04-25T13:39:47.615480+0000","last_clean":"2024-04-25T13:39:47.615480+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:39:47.615480+0000","last_undegraded":"2024-04-25T13:39:47.615480+0000","last_fullsized":"2024-04-25T13:39:47.615480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.538877+0000","last_change":"2024-04-25T13:39:45.259273+0000","last_active":"2024-04-25T13:39:46.538877+0000","last_peered":"2024-04-25T13:39:46.538877+0000","last_clean":"2024-04-25T13:39:46.538877+0000","last_became_active":"2024-04-25T13:39:45.258905+0000","last_became_peered":"2024-04-25T13:39:45.258905+0000","last_unstale":"2024-04-25T13:39:46.538877+0000","last_undegraded":"2024-04-25T13:39:46.538877+0000","last_fullsized":"2024-04-25T13:39:46.538877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:25.749581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:46.248440+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:39:46.248440+0000","last_peered":"2024-04-25T13:39:46.248440+0000","last_clean":"2024-04-25T13:39:46.248440+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:39:46.248440+0000","last_undegraded":"2024-04-25T13:39:46.248440+0000","last_fullsized":"2024-04-25T13:39:46.248440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T13:39:47.663935+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:39:47.663935+0000","last_peered":"2024-04-25T13:39:47.663935+0000","last_clean":"2024-04-25T13:39:47.663935+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:39:47.663935+0000","last_undegraded":"2024-04-25T13:39:47.663935+0000","last_fullsized":"2024-04-25T13:39:47.663935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24483,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":35481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579918,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":404913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1017413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972376,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40340,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704812,"statfs":{"total":95995035648,"available":95953727488,"internally_reserved":0,"allocated":864256,"data_stored":405149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168605,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":404957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1449984,"data_stored":995903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":843776,"data_stored":404331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":856064,"data_stored":405943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":405809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":406599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40336,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704816,"statfs":{"total":95995035648,"available":95953731584,"internally_reserved":0,"allocated":856064,"data_stored":404525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":856064,"data_stored":404975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36772,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708380,"statfs":{"total":95995035648,"available":95957381120,"internally_reserved":0,"allocated":1466368,"data_stored":996175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":22597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:39:52.833 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:39:52.834 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:39:52.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:39:52.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:52 smithi138 ceph-mon[32977]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:52.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:52 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/436400871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:52.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:52 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1301517747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:52.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:52 smithi138 ceph-mon[32977]: pgmap v285: 97 pgs: 97 active+clean; 601 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 17 op/s 2024-04-25T13:39:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:39:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:52 smithi137 ceph-mon[32568]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:39:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:52 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/436400871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:39:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:52 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1301517747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:39:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:52 smithi137 ceph-mon[32568]: pgmap v285: 97 pgs: 97 active+clean; 601 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 17 op/s 2024-04-25T13:39:53.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:53 smithi094 ceph-mon[29195]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:53.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:53 smithi094 ceph-mon[29195]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:53.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:53 smithi138 ceph-mon[32977]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:53.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:53 smithi138 ceph-mon[32977]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:54.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:53 smithi137 ceph-mon[32568]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:54.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:53 smithi137 ceph-mon[32568]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- 172.21.15.94:0/2306221495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104200 msgr2=0x7fafdc104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 --2- 172.21.15.94:0/2306221495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104200 0x7fafdc104610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009990 tx=0x7fafcc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- 172.21.15.94:0/2306221495 shutdown_connections 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 --2- 172.21.15.94:0/2306221495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104ba0 0x7fafdc10b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 --2- 172.21.15.94:0/2306221495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 0x7fafdc1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 --2- 172.21.15.94:0/2306221495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104200 0x7fafdc104610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- 172.21.15.94:0/2306221495 >> 172.21.15.94:0/2306221495 conn(0x7fafdc0fe710 msgr2=0x7fafdc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- 172.21.15.94:0/2306221495 shutdown_connections 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- 172.21.15.94:0/2306221495 wait complete. 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 Processor -- start 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.878+0000 7fafe4062700 1 -- start start 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 0x7fafdc10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104200 0x7fafdc10c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10fcb0 con 0x7fafdc104ba0 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10fb30 con 0x7fafdc1030d0 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10fe30 con 0x7fafdc104200 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:54.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60974/0 (socket says 172.21.15.94:60974) 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 -- 172.21.15.94:0/629151526 learned_addr learned my addr 172.21.15.94:0/629151526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe1dfe700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 0x7fafdc10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe15fd700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104200 0x7fafdc10c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 -- 172.21.15.94:0/629151526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104200 msgr2=0x7fafdc10c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104200 0x7fafdc10c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 -- 172.21.15.94:0/629151526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 msgr2=0x7fafdc10da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 0x7fafdc10da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 -- 172.21.15.94:0/629151526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc10cd60 con 0x7fafdc104ba0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe25ff700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fafd400b550 tx=0x7fafd400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4015020 con 0x7fafdc104ba0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd4011ce0 con 0x7fafdc104ba0 2024-04-25T13:39:54.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4010c90 con 0x7fafdc104ba0 2024-04-25T13:39:54.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.879+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1a48c0 con 0x7fafdc104ba0 2024-04-25T13:39:54.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.880+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a4e20 con 0x7fafdc104ba0 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.881+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc051190 con 0x7fafdc104ba0 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.882+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fafd4018020 con 0x7fafdc104ba0 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.883+0000 7fafd2ffd700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 0x7fafc80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.883+0000 7fafe1dfe700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 0x7fafc80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.883+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fafd40de670 con 0x7fafdc104ba0 2024-04-25T13:39:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.883+0000 7fafe1dfe700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 0x7fafc80b21a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009550 tx=0x7fafd80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:54.886+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafd401f020 con 0x7fafdc104ba0 2024-04-25T13:39:54.887 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:54 smithi094 ceph-mon[29195]: pgmap v286: 97 pgs: 97 active+clean; 604 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-25T13:39:54.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:54 smithi138 ceph-mon[32977]: pgmap v286: 97 pgs: 97 active+clean; 604 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-25T13:39:55.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:54 smithi137 ceph-mon[32568]: pgmap v286: 97 pgs: 97 active+clean; 604 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-25T13:39:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.172+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fafdc02e510 con 0x7fafdc104ba0 2024-04-25T13:39:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.175+0000 7fafd2ffd700 1 -- 172.21.15.94:0/629151526 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fafd40a73b0 con 0x7fafdc104ba0 2024-04-25T13:39:55.175 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-25T13:39:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQArXSpmDj5fChAA55EkistFwzB3RrcP8x3IYg== 2024-04-25T13:39:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 msgr2=0x7fafc80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 0x7fafc80b21a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009550 tx=0x7fafd80091b0 comp rx=0 tx=0).stop 2024-04-25T13:39:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 msgr2=0x7fafdc10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fafd400b550 tx=0x7fafd400b860 comp rx=0 tx=0).stop 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 shutdown_connections 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafc80afc90 0x7fafc80b21a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafdc104ba0 0x7fafdc10c600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafdc1030d0 0x7fafdc10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 --2- 172.21.15.94:0/629151526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafdc104200 0x7fafdc10c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 >> 172.21.15.94:0/629151526 conn(0x7fafdc0fe710 msgr2=0x7fafdc107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 shutdown_connections 2024-04-25T13:39:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:55.177+0000 7fafe4062700 1 -- 172.21.15.94:0/629151526 wait complete. 2024-04-25T13:39:55.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:39:55.787 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:39:55.813 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:39:55.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:39:55.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:39:55.848 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:39:55.848 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:39:55.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:39:55.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:39:55.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:39:56.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:39:56.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/629151526' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:39:56.125 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:39:56.393 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 13:39:56 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T13:39:56.031+0000 7fbaffd29700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T13:39:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:56 smithi094 ceph-mon[29195]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T13:39:56.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:56 smithi138 ceph-mon[32977]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T13:39:57.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:56 smithi137 ceph-mon[32568]: pgmap v287: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T13:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- 172.21.15.94:0/1544832776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c108650 msgr2=0x7fe98c10fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 --2- 172.21.15.94:0/1544832776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c108650 0x7fe98c10fff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b0d0 tx=0x7fe98400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- 172.21.15.94:0/1544832776 shutdown_connections 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 --2- 172.21.15.94:0/1544832776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c108650 0x7fe98c10fff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 --2- 172.21.15.94:0/1544832776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c103b00 0x7fe98c103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 --2- 172.21.15.94:0/1544832776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c107d00 0x7fe98c108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- 172.21.15.94:0/1544832776 >> 172.21.15.94:0/1544832776 conn(0x7fe98c0fe370 msgr2=0x7fe98c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- 172.21.15.94:0/1544832776 shutdown_connections 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- 172.21.15.94:0/1544832776 wait complete. 2024-04-25T13:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 Processor -- start 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.825+0000 7fe992291700 1 -- start start 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c103b00 0x7fe98c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 0x7fe98c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113340 con 0x7fe98c107d00 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1131c0 con 0x7fe98c103b00 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe992291700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1134c0 con 0x7fe98c108650 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60992/0 (socket says 172.21.15.94:60992) 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 -- 172.21.15.94:0/4156920505 learned_addr learned my addr 172.21.15.94:0/4156920505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98bfff700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c103b00 0x7fe98c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe99082e700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 0x7fe98c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 -- 172.21.15.94:0/4156920505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 msgr2=0x7fe98c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 0x7fe98c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 -- 172.21.15.94:0/4156920505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c103b00 msgr2=0x7fe98c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c103b00 0x7fe98c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe98b7fe700 1 -- 172.21.15.94:0/4156920505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1a6ed0 con 0x7fe98c107d00 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.826+0000 7fe99082e700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 0x7fe98c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe98b7fe700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe98000d980 tx=0x7fe98000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980017070 con 0x7fe98c107d00 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe980009de0 con 0x7fe98c107d00 2024-04-25T13:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a71c0 con 0x7fe98c107d00 2024-04-25T13:39:57.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f3c0 con 0x7fe98c107d00 2024-04-25T13:39:57.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.827+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a7700 con 0x7fe98c107d00 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.828+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c051190 con 0x7fe98c107d00 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.830+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe980009930 con 0x7fe98c107d00 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.830+0000 7fe9897fa700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 0x7fe9740b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.831+0000 7fe98bfff700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 0x7fe9740b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.831+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fe9800de700 con 0x7fe98c107d00 2024-04-25T13:39:57.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.831+0000 7fe98bfff700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 0x7fe9740b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009a10 tx=0x7fe97c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:39:57.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:57.834+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9800a7330 con 0x7fe98c107d00 2024-04-25T13:39:58.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.091+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe98c02e510 con 0x7fe98c107d00 2024-04-25T13:39:58.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.096+0000 7fe9897fa700 1 -- 172.21.15.94:0/4156920505 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284476 (secure 0 0 0) 0x7fe9800a6c30 con 0x7fe98c107d00 2024-04-25T13:39:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.100+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 msgr2=0x7fe9740b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 0x7fe9740b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009a10 tx=0x7fe97c009190 comp rx=0 tx=0).stop 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 msgr2=0x7fe98c19c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe98000d980 tx=0x7fe98000dc90 comp rx=0 tx=0).stop 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 shutdown_connections 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe98c108650 0x7fe98c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c103b00 0x7fe98c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9740afc10 0x7fe9740b2120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 --2- 172.21.15.94:0/4156920505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c107d00 0x7fe98c19c0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:39:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 >> 172.21.15.94:0/4156920505 conn(0x7fe98c0fe370 msgr2=0x7fe98c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:39:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 shutdown_connections 2024-04-25T13:39:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:39:58.101+0000 7fe992291700 1 -- 172.21.15.94:0/4156920505 wait complete. 2024-04-25T13:39:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:58 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4156920505' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:39:58.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:58 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4156920505' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:39:58.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:58 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4156920505' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:39:58.874 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":449370799},{"type":"v1","addr":"172.21.15.94:6801","nonce":449370799}]},"active_addr":"172.21.15.94:6801/449370799","active_change":"2024-04-25T13:32:40.168036+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3351813029}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1155370168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3124826398}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2188866588}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":909352381}]}]} 2024-04-25T13:39:58.880 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:39:58.881 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:39:58.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:39:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:39:59.465 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:39:59 smithi094 ceph-mon[29195]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T13:39:59.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:39:59 smithi138 ceph-mon[32977]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T13:39:59.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:39:59 smithi137 ceph-mon[32568]: pgmap v288: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T13:40:00.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:00 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T13:40:00.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:00 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T13:40:00.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:00 smithi137 ceph-mon[32568]: overall HEALTH_OK 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 -- 172.21.15.94:0/1718512524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130107cb0 msgr2=0x7f61301080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 --2- 172.21.15.94:0/1718512524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130107cb0 0x7f61301080c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61240099c0 tx=0x7f6124009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 -- 172.21.15.94:0/1718512524 shutdown_connections 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 --2- 172.21.15.94:0/1718512524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130108600 0x7f613010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 --2- 172.21.15.94:0/1718512524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130103380 0x7f6130103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 --2- 172.21.15.94:0/1718512524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130107cb0 0x7f61301080c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 -- 172.21.15.94:0/1718512524 >> 172.21.15.94:0/1718512524 conn(0x7f61300fecb0 msgr2=0x7f61301010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:00.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 -- 172.21.15.94:0/1718512524 shutdown_connections 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.727+0000 7f6134c07700 1 -- 172.21.15.94:0/1718512524 wait complete. 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 Processor -- start 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 -- start start 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 0x7f61301999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130107cb0 0x7f6130199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6130113210 con 0x7f6130108600 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6130113090 con 0x7f6130103380 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f6134c07700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6130113390 con 0x7f6130107cb0 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f612ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f612ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32780/0 (socket says 172.21.15.94:32780) 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.728+0000 7f612ed9d700 1 -- 172.21.15.94:0/2418933762 learned_addr learned my addr 172.21.15.94:0/2418933762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612dd9b700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130107cb0 0x7f6130199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612e59c700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 0x7f61301999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 -- 172.21.15.94:0/2418933762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130107cb0 msgr2=0x7f6130199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130107cb0 0x7f6130199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 -- 172.21.15.94:0/2418933762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 msgr2=0x7f61301999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 0x7f61301999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 -- 172.21.15.94:0/2418933762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61301a90c0 con 0x7f6130108600 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612e59c700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 0x7f61301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f612ed9d700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f612800ef30 tx=0x7f612800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280092e0 con 0x7f6130108600 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61301a9350 con 0x7f6130108600 2024-04-25T13:40:00.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6128004bb0 con 0x7f6130108600 2024-04-25T13:40:00.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128004280 con 0x7f6130108600 2024-04-25T13:40:00.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.729+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61301a9810 con 0x7f6130108600 2024-04-25T13:40:00.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.730+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6130104830 con 0x7f6130108600 2024-04-25T13:40:00.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.732+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f612801b030 con 0x7f6130108600 2024-04-25T13:40:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.733+0000 7f611f7fe700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 0x7f61180b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.733+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f61280df060 con 0x7f6130108600 2024-04-25T13:40:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.733+0000 7f612e59c700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 0x7f61180b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.733+0000 7f612e59c700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 0x7f61180b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6120009be0 tx=0x7f6120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:00.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.737+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6128022020 con 0x7f6130108600 2024-04-25T13:40:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.935+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f613002e510 con 0x7f6130108600 2024-04-25T13:40:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.936+0000 7f611f7fe700 1 -- 172.21.15.94:0/2418933762 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22244 (secure 0 0 0) 0x7f6128010040 con 0x7f6130108600 2024-04-25T13:40:00.937 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:40:00.937 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:39:46.244890+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 msgr2=0x7f61180b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 0x7f61180b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6120009be0 tx=0x7f6120009260 comp rx=0 tx=0).stop 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 msgr2=0x7f613019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f612800ef30 tx=0x7f612800c3a0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 shutdown_connections 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61180afc10 0x7f61180b2120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6130108600 0x7f613019a440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130103380 0x7f61301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 --2- 172.21.15.94:0/2418933762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6130107cb0 0x7f6130199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 >> 172.21.15.94:0/2418933762 conn(0x7f61300fecb0 msgr2=0x7f6130109560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 shutdown_connections 2024-04-25T13:40:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:00.938+0000 7f6134c07700 1 -- 172.21.15.94:0/2418933762 wait complete. 2024-04-25T13:40:01.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:01 smithi094 ceph-mon[29195]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-04-25T13:40:01.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:01 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2418933762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:01.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:01 smithi138 ceph-mon[32977]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-04-25T13:40:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:01 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2418933762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:01.598 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:40:01.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:40:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:01 smithi137 ceph-mon[32568]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 26 op/s 2024-04-25T13:40:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:01 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2418933762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:01.839 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:02 smithi138 ceph-mon[32977]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2024-04-25T13:40:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:02 smithi137 ceph-mon[32568]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2024-04-25T13:40:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:02 smithi094 ceph-mon[29195]: pgmap v290: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2024-04-25T13:40:03.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 -- 172.21.15.94:0/202342209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65301053c0 msgr2=0x7f65301057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:03.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 --2- 172.21.15.94:0/202342209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65301053c0 0x7f65301057d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6520009990 tx=0x7f6520009ca0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 -- 172.21.15.94:0/202342209 shutdown_connections 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 --2- 172.21.15.94:0/202342209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f653010e7f0 0x7f6530110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 --2- 172.21.15.94:0/202342209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65301053c0 0x7f65301057d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 --2- 172.21.15.94:0/202342209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6530105d10 0x7f653010e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 -- 172.21.15.94:0/202342209 >> 172.21.15.94:0/202342209 conn(0x7f65300fedb0 msgr2=0x7f65301011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 -- 172.21.15.94:0/202342209 shutdown_connections 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 -- 172.21.15.94:0/202342209 wait complete. 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.440+0000 7f6535690700 1 Processor -- start 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 -- start start 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 0x7f653019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 0x7f653019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:03.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6530114440 con 0x7f653010e7f0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65301142c0 con 0x7f6530105d10 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f6535690700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65301145c0 con 0x7f65301053c0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32806/0 (socket says 172.21.15.94:32806) 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 -- 172.21.15.94:0/3106604051 learned_addr learned my addr 172.21.15.94:0/3106604051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652e7fc700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 0x7f653019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652effd700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 0x7f653019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 -- 172.21.15.94:0/3106604051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 msgr2=0x7f653019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 0x7f653019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 -- 172.21.15.94:0/3106604051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 msgr2=0x7f653019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 0x7f653019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 -- 172.21.15.94:0/3106604051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653019fce0 con 0x7f653010e7f0 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652effd700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 0x7f653019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:03.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652e7fc700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 0x7f653019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.441+0000 7f652f7fe700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f652800b970 tx=0x7f652800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.442+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528019070 con 0x7f653010e7f0 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.442+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6528014440 con 0x7f653010e7f0 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.442+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653019ffd0 con 0x7f653010e7f0 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.442+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800ce60 con 0x7f653010e7f0 2024-04-25T13:40:03.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.442+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a78f0 con 0x7f653010e7f0 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.443+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6530051190 con 0x7f653010e7f0 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.444+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f652801c070 con 0x7f653010e7f0 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.445+0000 7f6517fff700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 0x7f65180b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.445+0000 7f652effd700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 0x7f65180b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.445+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f65280de3e0 con 0x7f653010e7f0 2024-04-25T13:40:03.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.445+0000 7f652effd700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 0x7f65180b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6520003800 tx=0x7f6520005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:03.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.449+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65280a7120 con 0x7f653010e7f0 2024-04-25T13:40:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.649+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f653019c170 con 0x7f653010e7f0 2024-04-25T13:40:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.651+0000 7f6517fff700 1 -- 172.21.15.94:0/3106604051 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22244 (secure 0 0 0) 0x7f65280a2030 con 0x7f653010e7f0 2024-04-25T13:40:03.652 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:40:03.652 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:39:46.244890+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 msgr2=0x7f65180b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 0x7f65180b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6520003800 tx=0x7f6520005fb0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 msgr2=0x7f653019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f652800b970 tx=0x7f652800bd30 comp rx=0 tx=0).stop 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 shutdown_connections 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65180afc10 0x7f65180b2120 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653010e7f0 0x7f653019f640 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65301053c0 0x7f653019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 --2- 172.21.15.94:0/3106604051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6530105d10 0x7f653019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 >> 172.21.15.94:0/3106604051 conn(0x7f65300fedb0 msgr2=0x7f65300ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 shutdown_connections 2024-04-25T13:40:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:03.653+0000 7f6535690700 1 -- 172.21.15.94:0/3106604051 wait complete. 2024-04-25T13:40:03.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:03 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3106604051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:03.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:03 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3106604051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:04.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:03 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3106604051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:40:04.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:40:04.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:40:04.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:40:04.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:40:04.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:40:04.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:40:04.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:40:04.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:40:04.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:40:04.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:40:04.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:40:04.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:40:04.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:04 smithi094 ceph-mon[29195]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-04-25T13:40:04.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:04 smithi138 ceph-mon[32977]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-04-25T13:40:05.063 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:04 smithi137 ceph-mon[32568]: pgmap v291: 97 pgs: 97 active+clean; 632 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-04-25T13:40:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.150 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.151 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.153 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.153 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:05.154 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:06.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:06 smithi094 ceph-mon[29195]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T13:40:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:06 smithi137 ceph-mon[32568]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T13:40:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:06 smithi138 ceph-mon[32977]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T13:40:08.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:08 smithi094 ceph-mon[29195]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T13:40:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:08 smithi137 ceph-mon[32568]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T13:40:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:08 smithi138 ceph-mon[32977]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T13:40:10.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:10 smithi094 ceph-mon[29195]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T13:40:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:10 smithi137 ceph-mon[32568]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T13:40:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:10 smithi138 ceph-mon[32977]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T13:40:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:12 smithi094 ceph-mon[29195]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T13:40:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:12 smithi137 ceph-mon[32568]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T13:40:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:12 smithi138 ceph-mon[32977]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T13:40:14.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:14 smithi094 ceph-mon[29195]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:40:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:14 smithi137 ceph-mon[32568]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:40:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:14 smithi138 ceph-mon[32977]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 -- 172.21.15.94:0/1924772093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 msgr2=0x7f69d410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/1924772093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d410ff70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b0d0 tx=0x7f69cc00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 -- 172.21.15.94:0/1924772093 shutdown_connections 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/1924772093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d410ff70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/1924772093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69d4103390 0x7f69d41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/1924772093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69d4107cc0 0x7f69d41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 -- 172.21.15.94:0/1924772093 >> 172.21.15.94:0/1924772093 conn(0x7f69d40fecc0 msgr2=0x7f69d41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 -- 172.21.15.94:0/1924772093 shutdown_connections 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.033+0000 7f69d8eb5700 1 -- 172.21.15.94:0/1924772093 wait complete. 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 Processor -- start 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 -- start start 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69d4103390 0x7f69d40ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69d4107cc0 0x7f69d40ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d40ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4113260 con 0x7f69d4108610 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41130e0 con 0x7f69d4103390 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.034+0000 7f69d8eb5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41133e0 con 0x7f69d4107cc0 2024-04-25T13:40:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d40ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d40ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45974/0 (socket says 172.21.15.94:45974) 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 -- 172.21.15.94:0/4138673959 learned_addr learned my addr 172.21.15.94:0/4138673959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d1d9b700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69d4107cc0 0x7f69d40ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d259c700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69d4103390 0x7f69d40ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 -- 172.21.15.94:0/4138673959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69d4107cc0 msgr2=0x7f69d40ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69d4107cc0 0x7f69d40ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 -- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69d4103390 msgr2=0x7f69d40ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69d4103390 0x7f69d40ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d41a9060 con 0x7f69d4108610 2024-04-25T13:40:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d2d9d700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d40ffc20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0095c0 tx=0x7f69cc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc01f070 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69cc007e20 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d41a92f0 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc01b4a0 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.035+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d41a9740 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.036+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69d4063da0 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.038+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f69cc022070 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.038+0000 7f69c37fe700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69bc0afc10 0x7f69bc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.038+0000 7f69d259c700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69bc0afc10 0x7f69bc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.038+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f69cc00f020 con 0x7f69d4108610 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.039+0000 7f69c37fe700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] conn(0x7f69bc0b9440 0x7f69bc0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.039+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69cc031ea0 con 0x7f69bc0b9440 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.039+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f69cc0e0700 con 0x7f69d4108610 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.039+0000 7f69d259c700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69bc0afc10 0x7f69bc0b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69c4005950 tx=0x7f69c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.039+0000 7f69d1d9b700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] conn(0x7f69bc0b9440 0x7f69bc0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.040+0000 7f69d1d9b700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] conn(0x7f69bc0b9440 0x7f69bc0bb890 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.040+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== osd.4 v2:172.21.15.137:6808/3127701916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69cc031ea0 con 0x7f69bc0b9440 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 --> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69d40641a0 con 0x7f69bc0b9440 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69c37fe700 1 -- 172.21.15.94:0/4138673959 <== osd.4 v2:172.21.15.137:6808/3127701916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f69d40641a0 con 0x7f69bc0b9440 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] conn(0x7f69bc0b9440 msgr2=0x7f69bc0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] conn(0x7f69bc0b9440 0x7f69bc0bb890 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69bc0afc10 msgr2=0x7f69bc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69bc0afc10 0x7f69bc0b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69c4005950 tx=0x7f69c400b410 comp rx=0 tx=0).stop 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 msgr2=0x7f69d40ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 --2- 172.21.15.94:0/4138673959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69d4108610 0x7f69d40ffc20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0095c0 tx=0x7f69cc007a20 comp rx=0 tx=0).stop 2024-04-25T13:40:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d2d9d700 1 -- 172.21.15.94:0/4138673959 reap_dead start 2024-04-25T13:40:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 shutdown_connections 2024-04-25T13:40:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.057+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 >> 172.21.15.94:0/4138673959 conn(0x7f69d40fecc0 msgr2=0x7f69d4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.058+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 shutdown_connections 2024-04-25T13:40:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:13.058+0000 7f69d8eb5700 1 -- 172.21.15.94:0/4138673959 wait complete. 2024-04-25T13:40:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 -- 172.21.15.94:0/212057805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 msgr2=0x7f0f8410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/212057805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f8410ff70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b0d0 tx=0x7f0f7c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 -- 172.21.15.94:0/212057805 shutdown_connections 2024-04-25T13:40:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/212057805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f8410ff70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/212057805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 0x7f0f841037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.334+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/212057805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84107cc0 0x7f0f841080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.335+0000 7f0f8bde1700 1 -- 172.21.15.94:0/212057805 >> 172.21.15.94:0/212057805 conn(0x7f0f840fec90 msgr2=0x7f0f841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.335+0000 7f0f8bde1700 1 -- 172.21.15.94:0/212057805 shutdown_connections 2024-04-25T13:40:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.335+0000 7f0f8bde1700 1 -- 172.21.15.94:0/212057805 wait complete. 2024-04-25T13:40:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.335+0000 7f0f8bde1700 1 Processor -- start 2024-04-25T13:40:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.335+0000 7f0f8bde1700 1 -- start start 2024-04-25T13:40:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 0x7f0f8419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84107cc0 0x7f0f8419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f84199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84113240 con 0x7f0f84108610 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841130c0 con 0x7f0f84103390 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8bde1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841133c0 con 0x7f0f84107cc0 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f84199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f84199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60256/0 (socket says 172.21.15.94:60256) 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 -- 172.21.15.94:0/2728482418 learned_addr learned my addr 172.21.15.94:0/2728482418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f89b7d700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 0x7f0f8419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8937c700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84107cc0 0x7f0f8419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 -- 172.21.15.94:0/2728482418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84107cc0 msgr2=0x7f0f8419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84107cc0 0x7f0f8419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 -- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 msgr2=0x7f0f8419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 0x7f0f8419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f8a37e700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8419a030 con 0x7f0f84108610 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.336+0000 7f0f89b7d700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103390 0x7f0f8419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.337+0000 7f0f8a37e700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f84199980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009850 tx=0x7f0f7c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.339+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c01f070 con 0x7f0f84108610 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.339+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c004c50 con 0x7f0f84108610 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.339+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8419a2c0 con 0x7f0f84108610 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.339+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c01b690 con 0x7f0f84108610 2024-04-25T13:40:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.339+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8419a7a0 con 0x7f0f84108610 2024-04-25T13:40:16.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.340+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f8406b830 con 0x7f0f84108610 2024-04-25T13:40:16.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.341+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0f7c022070 con 0x7f0f84108610 2024-04-25T13:40:16.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f7affd700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afc10 0x7f0f700b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:16.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f89b7d700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afc10 0x7f0f700b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f0f7c00f020 con 0x7f0f84108610 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f7affd700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] conn(0x7f0f700b9440 0x7f0f700bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f7c0075c0 con 0x7f0f700b9440 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f0f7c0e42c0 con 0x7f0f84108610 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f89b7d700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afc10 0x7f0f700b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419aac0 tx=0x7f0f80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.342+0000 7f0f8937c700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] conn(0x7f0f700b9440 0x7f0f700bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:16.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.343+0000 7f0f8937c700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] conn(0x7f0f700b9440 0x7f0f700bb890 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:16.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.344+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== osd.10 v2:172.21.15.137:6824/3471315400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f7c0075c0 con 0x7f0f700b9440 2024-04-25T13:40:16.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.360+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 --> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f8406bcb0 con 0x7f0f700b9440 2024-04-25T13:40:16.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f7affd700 1 -- 172.21.15.94:0/2728482418 <== osd.10 v2:172.21.15.137:6824/3471315400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f8406bcb0 con 0x7f0f700b9440 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] conn(0x7f0f700b9440 msgr2=0x7f0f700bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] conn(0x7f0f700b9440 0x7f0f700bb890 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afc10 msgr2=0x7f0f700b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afc10 0x7f0f700b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419aac0 tx=0x7f0f80009260 comp rx=0 tx=0).stop 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 msgr2=0x7f0f84199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 --2- 172.21.15.94:0/2728482418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108610 0x7f0f84199980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009850 tx=0x7f0f7c007a20 comp rx=0 tx=0).stop 2024-04-25T13:40:16.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8a37e700 1 -- 172.21.15.94:0/2728482418 reap_dead start 2024-04-25T13:40:16.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 shutdown_connections 2024-04-25T13:40:16.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.361+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 >> 172.21.15.94:0/2728482418 conn(0x7f0f840fec90 msgr2=0x7f0f84107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:16.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.362+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 shutdown_connections 2024-04-25T13:40:16.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:16.362+0000 7f0f8bde1700 1 -- 172.21.15.94:0/2728482418 wait complete. 2024-04-25T13:40:16.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:16 smithi094 ceph-mon[29195]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T13:40:16.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:16 smithi137 ceph-mon[32568]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T13:40:16.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:16 smithi138 ceph-mon[32977]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T13:40:18.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:18 smithi094 ceph-mon[29195]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:40:18.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:18 smithi137 ceph-mon[32568]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:40:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:18 smithi138 ceph-mon[32977]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 -- 172.21.15.94:0/3731431714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 msgr2=0x7fcf4410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3731431714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4410ff70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b0d0 tx=0x7fcf3c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 -- 172.21.15.94:0/3731431714 shutdown_connections 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3731431714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4410ff70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3731431714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf44103390 0x7fcf441037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3731431714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44107cc0 0x7fcf441080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 -- 172.21.15.94:0/3731431714 >> 172.21.15.94:0/3731431714 conn(0x7fcf440fec90 msgr2=0x7fcf441010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 -- 172.21.15.94:0/3731431714 shutdown_connections 2024-04-25T13:40:19.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.500+0000 7fcf492f4700 1 -- 172.21.15.94:0/3731431714 wait complete. 2024-04-25T13:40:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 Processor -- start 2024-04-25T13:40:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 -- start start 2024-04-25T13:40:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44103390 0x7fcf4419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf44107cc0 0x7fcf4419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44114400 con 0x7fcf44108610 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44114280 con 0x7fcf44103390 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf492f4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44114580 con 0x7fcf44107cc0 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf437fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf437fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60274/0 (socket says 172.21.15.94:60274) 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.501+0000 7fcf437fe700 1 -- 172.21.15.94:0/3089093409 learned_addr learned my addr 172.21.15.94:0/3089093409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf42ffd700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44103390 0x7fcf4419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf427fc700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf44107cc0 0x7fcf4419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 -- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf44107cc0 msgr2=0x7fcf4419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf44107cc0 0x7fcf4419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 -- 172.21.15.94:0/3089093409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44103390 msgr2=0x7fcf4419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44103390 0x7fcf4419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4419fc80 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf42ffd700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44103390 0x7fcf4419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf437fe700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4419f5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00d010 tx=0x7fcf3c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c01f070 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4419ff10 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c003c00 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c01b6a0 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.502+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a7890 con 0x7fcf44108610 2024-04-25T13:40:19.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.503+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf441a0440 con 0x7fcf44108610 2024-04-25T13:40:19.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.504+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcf3c022070 con 0x7fcf44108610 2024-04-25T13:40:19.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf2bfff700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc10 0x7fcf2c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:19.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf42ffd700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc10 0x7fcf2c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:19.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fcf3c00f020 con 0x7fcf44108610 2024-04-25T13:40:19.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf2bfff700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] conn(0x7fcf2c0b9440 0x7fcf2c0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf3c025e50 con 0x7fcf2c0b9440 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.505+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c0e0520 con 0x7fcf44108610 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.506+0000 7fcf42ffd700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc10 0x7fcf2c0b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf34005fd0 tx=0x7fcf34005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.506+0000 7fcf427fc700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] conn(0x7fcf2c0b9440 0x7fcf2c0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.506+0000 7fcf427fc700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] conn(0x7fcf2c0b9440 0x7fcf2c0bb890 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:19.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.507+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== osd.5 v2:172.21.15.138:6808/699823626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf3c025e50 con 0x7fcf2c0b9440 2024-04-25T13:40:19.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.524+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 --> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf44063fb0 con 0x7fcf2c0b9440 2024-04-25T13:40:19.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.524+0000 7fcf2bfff700 1 -- 172.21.15.94:0/3089093409 <== osd.5 v2:172.21.15.138:6808/699823626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf44063fb0 con 0x7fcf2c0b9440 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] conn(0x7fcf2c0b9440 msgr2=0x7fcf2c0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] conn(0x7fcf2c0b9440 0x7fcf2c0bb890 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc10 msgr2=0x7fcf2c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc10 0x7fcf2c0b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf34005fd0 tx=0x7fcf34005ee0 comp rx=0 tx=0).stop 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 msgr2=0x7fcf4419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 --2- 172.21.15.94:0/3089093409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf44108610 0x7fcf4419f5e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00d010 tx=0x7fcf3c007a50 comp rx=0 tx=0).stop 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf437fe700 1 -- 172.21.15.94:0/3089093409 reap_dead start 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 shutdown_connections 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 >> 172.21.15.94:0/3089093409 conn(0x7fcf440fec90 msgr2=0x7fcf44107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 shutdown_connections 2024-04-25T13:40:19.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:19.525+0000 7fcf492f4700 1 -- 172.21.15.94:0/3089093409 wait complete. 2024-04-25T13:40:20.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:20 smithi094 ceph-mon[29195]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 -- 172.21.15.94:0/3497408361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 msgr2=0x7fae84103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 --2- 172.21.15.94:0/3497408361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84103550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fae740099c0 tx=0x7fae74009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 -- 172.21.15.94:0/3497408361 shutdown_connections 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 --2- 172.21.15.94:0/3497408361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae841085e0 0x7fae84108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 --2- 172.21.15.94:0/3497408361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84103550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.574+0000 7fae88968700 1 --2- 172.21.15.94:0/3497408361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae84103b20 0x7fae84107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.575+0000 7fae88968700 1 -- 172.21.15.94:0/3497408361 >> 172.21.15.94:0/3497408361 conn(0x7fae840fee20 msgr2=0x7fae84101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.575+0000 7fae88968700 1 -- 172.21.15.94:0/3497408361 shutdown_connections 2024-04-25T13:40:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.575+0000 7fae88968700 1 -- 172.21.15.94:0/3497408361 wait complete. 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 Processor -- start 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 -- start start 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84117520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae84103b20 0x7fae84111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae841085e0 0x7fae84111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae84109b60 con 0x7fae84103140 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae841099e0 con 0x7fae841085e0 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae88968700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae84109ce0 con 0x7fae84103b20 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae8259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84117520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae8259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84117520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60292/0 (socket says 172.21.15.94:60292) 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae8259c700 1 -- 172.21.15.94:0/320725694 learned_addr learned my addr 172.21.15.94:0/320725694 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae81d9b700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae84103b20 0x7fae84111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.576+0000 7fae82d9d700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae841085e0 0x7fae84111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 -- 172.21.15.94:0/320725694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae84103b20 msgr2=0x7fae84111640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae84103b20 0x7fae84111640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 -- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae841085e0 msgr2=0x7fae84111b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae841085e0 0x7fae84111b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841121c0 con 0x7fae84103140 2024-04-25T13:40:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae82d9d700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae841085e0 0x7fae84111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.577+0000 7fae8259c700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84117520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b580 tx=0x7fae740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.578+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74027070 con 0x7fae84103140 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.578+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae740040a0 con 0x7fae84103140 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.578+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae84112450 con 0x7fae84103140 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.578+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7400f460 con 0x7fae84103140 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.578+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841b63b0 con 0x7fae84103140 2024-04-25T13:40:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.579+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae84107910 con 0x7fae84103140 2024-04-25T13:40:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.581+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fae7402a070 con 0x7fae84103140 2024-04-25T13:40:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.581+0000 7fae737fe700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae6c0afd60 0x7fae6c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fae740e03e0 con 0x7fae84103140 2024-04-25T13:40:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae81d9b700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae6c0afd60 0x7fae6c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae737fe700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] conn(0x7fae6c0b9590 0x7fae6c0bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae7402de50 con 0x7fae6c0b9590 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fae740e0840 con 0x7fae84103140 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae81d9b700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae6c0afd60 0x7fae6c0b2270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fae84101800 tx=0x7fae7800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.582+0000 7fae82d9d700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] conn(0x7fae6c0b9590 0x7fae6c0bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:20.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.583+0000 7fae82d9d700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] conn(0x7fae6c0b9590 0x7fae6c0bb9e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:20.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.584+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== osd.7 v2:172.21.15.137:6816/1075953971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae7402de50 con 0x7fae6c0b9590 2024-04-25T13:40:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.600+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 --> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae840761a0 con 0x7fae6c0b9590 2024-04-25T13:40:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.600+0000 7fae737fe700 1 -- 172.21.15.94:0/320725694 <== osd.7 v2:172.21.15.137:6816/1075953971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fae840761a0 con 0x7fae6c0b9590 2024-04-25T13:40:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] conn(0x7fae6c0b9590 msgr2=0x7fae6c0bb9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] conn(0x7fae6c0b9590 0x7fae6c0bb9e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae6c0afd60 msgr2=0x7fae6c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae6c0afd60 0x7fae6c0b2270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fae84101800 tx=0x7fae7800a380 comp rx=0 tx=0).stop 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 msgr2=0x7fae84117520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 --2- 172.21.15.94:0/320725694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae84103140 0x7fae84117520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b580 tx=0x7fae740047b0 comp rx=0 tx=0).stop 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae82d9d700 1 -- 172.21.15.94:0/320725694 reap_dead start 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 shutdown_connections 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 >> 172.21.15.94:0/320725694 conn(0x7fae840fee20 msgr2=0x7fae84105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 shutdown_connections 2024-04-25T13:40:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:20.601+0000 7fae88968700 1 -- 172.21.15.94:0/320725694 wait complete. 2024-04-25T13:40:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:20 smithi137 ceph-mon[32568]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T13:40:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:20 smithi138 ceph-mon[32977]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T13:40:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:22 smithi137 ceph-mon[32568]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T13:40:22.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:22 smithi094 ceph-mon[29195]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T13:40:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:22 smithi138 ceph-mon[32977]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T13:40:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.866+0000 7fbf5759e700 1 -- 172.21.15.94:0/1236815862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 msgr2=0x7fbf58107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.866+0000 7fbf5759e700 1 --2- 172.21.15.94:0/1236815862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf58107ec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0099c0 tx=0x7fbf4c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 -- 172.21.15.94:0/1236815862 shutdown_connections 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 --2- 172.21.15.94:0/1236815862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf58108400 0x7fbf5810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 --2- 172.21.15.94:0/1236815862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58103180 0x7fbf58103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 --2- 172.21.15.94:0/1236815862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf58107ec0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 -- 172.21.15.94:0/1236815862 >> 172.21.15.94:0/1236815862 conn(0x7fbf58078980 msgr2=0x7fbf58078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 -- 172.21.15.94:0/1236815862 shutdown_connections 2024-04-25T13:40:23.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.867+0000 7fbf5759e700 1 -- 172.21.15.94:0/1236815862 wait complete. 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 Processor -- start 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 -- start start 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf58103180 0x7fbf5819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf5819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58108400 0x7fbf58199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58113010 con 0x7fbf58107ab0 2024-04-25T13:40:23.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58112e90 con 0x7fbf58103180 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5759e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58113190 con 0x7fbf58108400 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf55d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf5819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf55d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf5819fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40094/0 (socket says 172.21.15.94:40094) 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf55d9b700 1 -- 172.21.15.94:0/2939488126 learned_addr learned my addr 172.21.15.94:0/2939488126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf5659c700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf58103180 0x7fbf5819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf56d9d700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58108400 0x7fbf58199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.868+0000 7fbf55d9b700 1 -- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58108400 msgr2=0x7fbf58199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf55d9b700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58108400 0x7fbf58199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf55d9b700 1 -- 172.21.15.94:0/2939488126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf58103180 msgr2=0x7fbf5819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf55d9b700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf58103180 0x7fbf5819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf55d9b700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf58199fa0 con 0x7fbf58107ab0 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf55d9b700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf5819fb90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c027070 con 0x7fbf58107ab0 2024-04-25T13:40:23.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf4c004020 con 0x7fbf58107ab0 2024-04-25T13:40:23.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c022be0 con 0x7fbf58107ab0 2024-04-25T13:40:23.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5819a230 con 0x7fbf58107ab0 2024-04-25T13:40:23.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.869+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5806b4e0 con 0x7fbf58107ab0 2024-04-25T13:40:23.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.870+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf5806ba00 con 0x7fbf58107ab0 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.871+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbf4c02a070 con 0x7fbf58107ab0 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf477fe700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbf400afc90 0x7fbf400b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf5659c700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbf400afc90 0x7fbf400b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fbf4c0e06e0 con 0x7fbf58107ab0 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf477fe700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] conn(0x7fbf400b94c0 0x7fbf400bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf4c01de50 con 0x7fbf400b94c0 2024-04-25T13:40:23.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c0e0bc0 con 0x7fbf58107ab0 2024-04-25T13:40:23.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf5659c700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbf400afc90 0x7fbf400b21a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf5819aba0 tx=0x7fbf4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.872+0000 7fbf56d9d700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] conn(0x7fbf400b94c0 0x7fbf400bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.873+0000 7fbf56d9d700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] conn(0x7fbf400b94c0 0x7fbf400bb910 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.874+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== osd.2 v2:172.21.15.138:6800/4136773447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf4c01de50 con 0x7fbf400b94c0 2024-04-25T13:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.890+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 --> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf5806be30 con 0x7fbf400b94c0 2024-04-25T13:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.890+0000 7fbf477fe700 1 -- 172.21.15.94:0/2939488126 <== osd.2 v2:172.21.15.138:6800/4136773447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf5806be30 con 0x7fbf400b94c0 2024-04-25T13:40:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] conn(0x7fbf400b94c0 msgr2=0x7fbf400bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] conn(0x7fbf400b94c0 0x7fbf400bb910 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbf400afc90 msgr2=0x7fbf400b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbf400afc90 0x7fbf400b21a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf5819aba0 tx=0x7fbf4800b410 comp rx=0 tx=0).stop 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 msgr2=0x7fbf5819fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 --2- 172.21.15.94:0/2939488126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf58107ab0 0x7fbf5819fb90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf56d9d700 1 -- 172.21.15.94:0/2939488126 reap_dead start 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 shutdown_connections 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 >> 172.21.15.94:0/2939488126 conn(0x7fbf58078980 msgr2=0x7fbf58100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 shutdown_connections 2024-04-25T13:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.891+0000 7fbf5759e700 1 -- 172.21.15.94:0/2939488126 wait complete. 2024-04-25T13:40:24.400 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:24 smithi094 ceph-mon[29195]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:24.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 -- 172.21.15.94:0/3719062354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58108d90 msgr2=0x7f4a581091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3719062354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58108d90 0x7f4a581091a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a480099c0 tx=0x7f4a48009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 -- 172.21.15.94:0/3719062354 shutdown_connections 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3719062354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581096e0 0x7f4a58111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3719062354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a58104460 0x7f4a58104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3719062354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58108d90 0x7f4a581091a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 -- 172.21.15.94:0/3719062354 >> 172.21.15.94:0/3719062354 conn(0x7f4a580ffd40 msgr2=0x7f4a58102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 -- 172.21.15.94:0/3719062354 shutdown_connections 2024-04-25T13:40:24.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.624+0000 7f4a5f549700 1 -- 172.21.15.94:0/3719062354 wait complete. 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 Processor -- start 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 -- start start 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 0x7f4a581a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a58108d90 0x7f4a581a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581096e0 0x7f4a5819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581142f0 con 0x7f4a58104460 2024-04-25T13:40:24.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58114170 con 0x7f4a581096e0 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5f549700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58114470 con 0x7f4a58108d90 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5d2e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 0x7f4a581a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5d2e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 0x7f4a581a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40156/0 (socket says 172.21.15.94:40156) 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5d2e5700 1 -- 172.21.15.94:0/3605870319 learned_addr learned my addr 172.21.15.94:0/3605870319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5cae4700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a58108d90 0x7f4a581a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.625+0000 7f4a5dae6700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581096e0 0x7f4a5819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 -- 172.21.15.94:0/3605870319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a58108d90 msgr2=0x7f4a581a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a58108d90 0x7f4a581a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 -- 172.21.15.94:0/3605870319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581096e0 msgr2=0x7f4a5819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581096e0 0x7f4a5819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5819b260 con 0x7f4a58104460 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5d2e5700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 0x7f4a581a09a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400ca10 con 0x7f4a58104460 2024-04-25T13:40:24.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a5400cbb0 con 0x7f4a58104460 2024-04-25T13:40:24.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5819b550 con 0x7f4a58104460 2024-04-25T13:40:24.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54018680 con 0x7f4a58104460 2024-04-25T13:40:24.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.626+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a7750 con 0x7f4a58104460 2024-04-25T13:40:24.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.627+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a58063de0 con 0x7f4a58104460 2024-04-25T13:40:24.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.629+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4a54010430 con 0x7f4a58104460 2024-04-25T13:40:24.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a4e7fc700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a440afc90 0x7f4a440b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:24.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a5cae4700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a440afc90 0x7f4a440b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f4a540df2c0 con 0x7f4a58104460 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a4e7fc700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] conn(0x7f4a440b94c0 0x7f4a440bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a440bbfe0 con 0x7f4a440b94c0 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f4a540df7a0 con 0x7f4a58104460 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a5dae6700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] conn(0x7f4a440b94c0 0x7f4a440bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.630+0000 7f4a5cae4700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a440afc90 0x7f4a440b21a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.631+0000 7f4a5dae6700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] conn(0x7f4a440b94c0 0x7f4a440bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:24.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.631+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== osd.0 v2:172.21.15.94:6802/2637923455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a440bbfe0 con 0x7f4a440b94c0 2024-04-25T13:40:24.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:24 smithi137 ceph-mon[32568]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:24.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 --> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a58064210 con 0x7f4a440b94c0 2024-04-25T13:40:24.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a4e7fc700 1 -- 172.21.15.94:0/3605870319 <== osd.0 v2:172.21.15.94:6802/2637923455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a58064210 con 0x7f4a440b94c0 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] conn(0x7f4a440b94c0 msgr2=0x7f4a440bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] conn(0x7f4a440b94c0 0x7f4a440bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a440afc90 msgr2=0x7f4a440b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a440afc90 0x7f4a440b21a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a4801a040 comp rx=0 tx=0).stop 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 msgr2=0x7f4a581a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:24.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 --2- 172.21.15.94:0/3605870319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a58104460 0x7f4a581a09a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:40:24.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5dae6700 1 -- 172.21.15.94:0/3605870319 reap_dead start 2024-04-25T13:40:24.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 shutdown_connections 2024-04-25T13:40:24.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.648+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 >> 172.21.15.94:0/3605870319 conn(0x7f4a580ffd40 msgr2=0x7f4a58102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:24.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.649+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 shutdown_connections 2024-04-25T13:40:24.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.649+0000 7f4a5f549700 1 -- 172.21.15.94:0/3605870319 wait complete. 2024-04-25T13:40:24.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:24 smithi138 ceph-mon[32977]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:25.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 -- 172.21.15.94:0/2119590622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3501085e0 msgr2=0x7fb35010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2119590622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3501085e0 0x7fb35010ff40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb34800b0d0 tx=0x7fb34800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 -- 172.21.15.94:0/2119590622 shutdown_connections 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2119590622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3501085e0 0x7fb35010ff40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2119590622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb350103360 0x7fb350103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2119590622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb350107c90 0x7fb3501080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 -- 172.21.15.94:0/2119590622 >> 172.21.15.94:0/2119590622 conn(0x7fb3500fec90 msgr2=0x7fb3501010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 -- 172.21.15.94:0/2119590622 shutdown_connections 2024-04-25T13:40:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.919+0000 7fb3550ae700 1 -- 172.21.15.94:0/2119590622 wait complete. 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.920+0000 7fb3550ae700 1 Processor -- start 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.920+0000 7fb3550ae700 1 -- start start 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 0x7fb35019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb350107c90 0x7fb35019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3501085e0 0x7fb35019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb350113210 con 0x7fb350103360 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb350113090 con 0x7fb3501085e0 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb3550ae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb350113390 con 0x7fb350107c90 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 0x7fb35019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 0x7fb35019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40188/0 (socket says 172.21.15.94:40188) 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 -- 172.21.15.94:0/2280943224 learned_addr learned my addr 172.21.15.94:0/2280943224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34f59e700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3501085e0 0x7fb35019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34e59c700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb350107c90 0x7fb35019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 -- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb350107c90 msgr2=0x7fb35019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb350107c90 0x7fb35019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 -- 172.21.15.94:0/2280943224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3501085e0 msgr2=0x7fb35019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3501085e0 0x7fb35019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3501a6db0 con 0x7fb350103360 2024-04-25T13:40:25.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34f59e700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3501085e0 0x7fb35019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.921+0000 7fb34ed9d700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 0x7fb35019bad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c9b0 tx=0x7fb34000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340007b00 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3501a70a0 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb340007ca0 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340018730 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3501a7630 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.922+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb350063da0 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.924+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb34000f450 con 0x7fb350103360 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb337fff700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb3380afc10 0x7fb3380b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb34e59c700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb3380afc10 0x7fb3380b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fb3400df340 con 0x7fb350103360 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb337fff700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] conn(0x7fb3380b9440 0x7fb3380bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb340014ec0 con 0x7fb3380b9440 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb3400df820 con 0x7fb350103360 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb34e59c700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb3380afc10 0x7fb3380b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb344005950 tx=0x7fb3440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.925+0000 7fb34f59e700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] conn(0x7fb3380b9440 0x7fb3380bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.926+0000 7fb34f59e700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] conn(0x7fb3380b9440 0x7fb3380bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.927+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== osd.11 v2:172.21.15.138:6824/2554799218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb340014ec0 con 0x7fb3380b9440 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.943+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 --> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3500641d0 con 0x7fb3380b9440 2024-04-25T13:40:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.943+0000 7fb337fff700 1 -- 172.21.15.94:0/2280943224 <== osd.11 v2:172.21.15.138:6824/2554799218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb3500641d0 con 0x7fb3380b9440 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] conn(0x7fb3380b9440 msgr2=0x7fb3380bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] conn(0x7fb3380b9440 0x7fb3380bb890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb3380afc10 msgr2=0x7fb3380b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb3380afc10 0x7fb3380b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb344005950 tx=0x7fb3440058e0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 msgr2=0x7fb35019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 --2- 172.21.15.94:0/2280943224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350103360 0x7fb35019bad0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c9b0 tx=0x7fb34000cd70 comp rx=0 tx=0).stop 2024-04-25T13:40:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb34f59e700 1 -- 172.21.15.94:0/2280943224 reap_dead start 2024-04-25T13:40:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 shutdown_connections 2024-04-25T13:40:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 >> 172.21.15.94:0/2280943224 conn(0x7fb3500fec90 msgr2=0x7fb350107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 shutdown_connections 2024-04-25T13:40:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.944+0000 7fb3550ae700 1 -- 172.21.15.94:0/2280943224 wait complete. 2024-04-25T13:40:25.727 INFO:teuthology.orchestra.run.smithi094.stdout:317827579925 2024-04-25T13:40:25.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 -- 172.21.15.94:0/3282074243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8108df0 msgr2=0x7fa8e8109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 --2- 172.21.15.94:0/3282074243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8108df0 0x7fa8e8109200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0099c0 tx=0x7fa8dc009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 -- 172.21.15.94:0/3282074243 shutdown_connections 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 --2- 172.21.15.94:0/3282074243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8e8109740 0x7fa8e81110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 --2- 172.21.15.94:0/3282074243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e81044c0 0x7fa8e81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 --2- 172.21.15.94:0/3282074243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8108df0 0x7fa8e8109200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 -- 172.21.15.94:0/3282074243 >> 172.21.15.94:0/3282074243 conn(0x7fa8e80fff10 msgr2=0x7fa8e8102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:25.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 -- 172.21.15.94:0/3282074243 shutdown_connections 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 -- 172.21.15.94:0/3282074243 wait complete. 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.412+0000 7fa8edc20700 1 Processor -- start 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 -- start start 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e81044c0 0x7fa8e8075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8e8108df0 0x7fa8e8075ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 0x7fa8e8076410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8114370 con 0x7fa8e8109740 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81141f0 con 0x7fa8e81044c0 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8edc20700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81144f0 con 0x7fa8e8108df0 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 0x7fa8e8076410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 0x7fa8e8076410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40110/0 (socket says 172.21.15.94:40110) 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 -- 172.21.15.94:0/2999328616 learned_addr learned my addr 172.21.15.94:0/2999328616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:25.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e77fe700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e81044c0 0x7fa8e8075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e6ffd700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8e8108df0 0x7fa8e8075ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 -- 172.21.15.94:0/2999328616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8e8108df0 msgr2=0x7fa8e8075ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8e8108df0 0x7fa8e8075ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 -- 172.21.15.94:0/2999328616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e81044c0 msgr2=0x7fa8e8075990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e81044c0 0x7fa8e8075990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e81a9160 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.413+0000 7fa8e7fff700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 0x7fa8e8076410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000ada0 tx=0x7fa8e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017020 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e81a9450 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e000ce30 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0005350 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e81a9960 con 0x7fa8e8109740 2024-04-25T13:40:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.414+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8e8063d60 con 0x7fa8e8109740 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.416+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa8e000c980 con 0x7fa8e8109740 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e4ff9700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8d00afc10 0x7fa8d00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e77fe700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8d00afc10 0x7fa8d00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa8e00df010 con 0x7fa8e8109740 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e4ff9700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] conn(0x7fa8d00b9440 0x7fa8d00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8d00bbf60 con 0x7fa8d00b9440 2024-04-25T13:40:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa8e00df4f0 con 0x7fa8e8109740 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e6ffd700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] conn(0x7fa8d00b9440 0x7fa8d00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.417+0000 7fa8e77fe700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8d00afc10 0x7fa8d00b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007be0 tx=0x7fa8d80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.418+0000 7fa8e6ffd700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] conn(0x7fa8d00b9440 0x7fa8d00bb890 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.419+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== osd.6 v2:172.21.15.94:6818/1177969162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8d00bbf60 con 0x7fa8d00b9440 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.435+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 --> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8e8064160 con 0x7fa8d00b9440 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.435+0000 7fa8e4ff9700 1 -- 172.21.15.94:0/2999328616 <== osd.6 v2:172.21.15.94:6818/1177969162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa8e8064160 con 0x7fa8d00b9440 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] conn(0x7fa8d00b9440 msgr2=0x7fa8d00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] conn(0x7fa8d00b9440 0x7fa8d00bb890 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8d00afc10 msgr2=0x7fa8d00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8d00afc10 0x7fa8d00b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007be0 tx=0x7fa8d80073d0 comp rx=0 tx=0).stop 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 msgr2=0x7fa8e8076410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 --2- 172.21.15.94:0/2999328616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8e8109740 0x7fa8e8076410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000ada0 tx=0x7fa8e000c330 comp rx=0 tx=0).stop 2024-04-25T13:40:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8e7fff700 1 -- 172.21.15.94:0/2999328616 reap_dead start 2024-04-25T13:40:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 shutdown_connections 2024-04-25T13:40:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 >> 172.21.15.94:0/2999328616 conn(0x7fa8e80fff10 msgr2=0x7fa8e810a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 shutdown_connections 2024-04-25T13:40:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.436+0000 7fa8edc20700 1 -- 172.21.15.94:0/2999328616 wait complete. 2024-04-25T13:40:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 -- 172.21.15.94:0/1732322424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c1091b0 msgr2=0x7f2f9c1095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/1732322424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c1091b0 0x7f2f9c1095c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009990 tx=0x7f2f98009ca0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 -- 172.21.15.94:0/1732322424 shutdown_connections 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/1732322424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f9c102f90 0x7f2f9c111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/1732322424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c1091b0 0x7f2f9c1095c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/1732322424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9c102640 0x7f2f9c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.427+0000 7f2fa42b7700 1 -- 172.21.15.94:0/1732322424 >> 172.21.15.94:0/1732322424 conn(0x7f2f9c0fe370 msgr2=0x7f2f9c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- 172.21.15.94:0/1732322424 shutdown_connections 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- 172.21.15.94:0/1732322424 wait complete. 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 Processor -- start 2024-04-25T13:40:26.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- start start 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 0x7f2f9c103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f9c102f90 0x7f2f9c103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9c1091b0 0x7f2f9c1042a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c1144b0 con 0x7f2f9c102640 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c114330 con 0x7f2f9c102f90 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.428+0000 7f2fa42b7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c114630 con 0x7f2f9c1091b0 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 0x7f2f9c103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 0x7f2f9c103820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40214/0 (socket says 172.21.15.94:40214) 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 -- 172.21.15.94:0/2143448810 learned_addr learned my addr 172.21.15.94:0/2143448810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa1852700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f9c102f90 0x7f2f9c103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2854700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9c1091b0 0x7f2f9c1042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 -- 172.21.15.94:0/2143448810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9c1091b0 msgr2=0x7f2f9c1042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9c1091b0 0x7f2f9c1042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 -- 172.21.15.94:0/2143448810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f9c102f90 msgr2=0x7f2f9c103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f9c102f90 0x7f2f9c103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c1a91c0 con 0x7f2f9c102640 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa2053700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 0x7f2f9c103820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f98006010 tx=0x7f2f980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98027070 con 0x7f2f9c102640 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c1a9450 con 0x7f2f9c102640 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f98003fe0 con 0x7f2f9c102640 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98022b20 con 0x7f2f9c102640 2024-04-25T13:40:26.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.429+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c1a9900 con 0x7f2f9c102640 2024-04-25T13:40:26.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.430+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f9c063da0 con 0x7f2f9c102640 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.432+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2f9802a070 con 0x7f2f9c102640 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.432+0000 7f2f937fe700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f880afc10 0x7f2f880b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2fa1852700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f880afc10 0x7f2f880b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f2f980e0350 con 0x7f2f9c102640 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2f937fe700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] conn(0x7f2f880b9310 0x7f2f880bb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f880bbe30 con 0x7f2f880b9310 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f2f980e0830 con 0x7f2f9c102640 2024-04-25T13:40:26.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2fa2854700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] conn(0x7f2f880b9310 0x7f2f880bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2fa1852700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f880afc10 0x7f2f880b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c1008e0 tx=0x7f2f8c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.433+0000 7f2fa2854700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] conn(0x7f2f880b9310 0x7f2f880bb760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.434+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== osd.9 v2:172.21.15.94:6826/1188484868 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f880bbe30 con 0x7f2f880b9310 2024-04-25T13:40:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.450+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 --> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f9c0641a0 con 0x7f2f880b9310 2024-04-25T13:40:26.449 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:26 smithi094 ceph-mon[29195]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.450+0000 7f2f937fe700 1 -- 172.21.15.94:0/2143448810 <== osd.9 v2:172.21.15.94:6826/1188484868 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2f9c0641a0 con 0x7f2f880b9310 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] conn(0x7f2f880b9310 msgr2=0x7f2f880bb760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] conn(0x7f2f880b9310 0x7f2f880bb760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f880afc10 msgr2=0x7f2f880b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f880afc10 0x7f2f880b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c1008e0 tx=0x7f2f8c009190 comp rx=0 tx=0).stop 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 msgr2=0x7f2f9c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 --2- 172.21.15.94:0/2143448810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f9c102640 0x7f2f9c103820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f98006010 tx=0x7f2f980047b0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa2854700 1 -- 172.21.15.94:0/2143448810 reap_dead start 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 shutdown_connections 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 >> 172.21.15.94:0/2143448810 conn(0x7f2f9c0fe370 msgr2=0x7f2f9c0fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 shutdown_connections 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:26.451+0000 7f2fa42b7700 1 -- 172.21.15.94:0/2143448810 wait complete. 2024-04-25T13:40:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:26 smithi137 ceph-mon[32568]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:26 smithi138 ceph-mon[32977]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:26.847 INFO:teuthology.orchestra.run.smithi094.stdout:188978561071 2024-04-25T13:40:26.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 -- 172.21.15.94:0/2519927825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81053d0 msgr2=0x7f17e81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 --2- 172.21.15.94:0/2519927825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81053d0 0x7f17e81057e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009990 tx=0x7f17e4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 -- 172.21.15.94:0/2519927825 shutdown_connections 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 --2- 172.21.15.94:0/2519927825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e810e800 0x7f17e8110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 --2- 172.21.15.94:0/2519927825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81053d0 0x7f17e81057e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 --2- 172.21.15.94:0/2519927825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17e8105d20 0x7f17e810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 -- 172.21.15.94:0/2519927825 >> 172.21.15.94:0/2519927825 conn(0x7f17e80fee00 msgr2=0x7f17e8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 -- 172.21.15.94:0/2519927825 shutdown_connections 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.521+0000 7f17ef653700 1 -- 172.21.15.94:0/2519927825 wait complete. 2024-04-25T13:40:26.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 Processor -- start 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 -- start start 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17e81053d0 0x7f17e81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8105d20 0x7f17e81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 0x7f17e819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8114320 con 0x7f17e810e800 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f17e81141a0 con 0x7f17e8105d20 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ef653700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17e81144a0 con 0x7f17e81053d0 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17edbf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 0x7f17e819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17edbf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 0x7f17e819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40136/0 (socket says 172.21.15.94:40136) 2024-04-25T13:40:26.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17edbf0700 1 -- 172.21.15.94:0/3401017908 learned_addr learned my addr 172.21.15.94:0/3401017908 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.522+0000 7f17ecbee700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8105d20 0x7f17e81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17ed3ef700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17e81053d0 0x7f17e81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 -- 172.21.15.94:0/3401017908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17e81053d0 msgr2=0x7f17e81a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17e81053d0 0x7f17e81a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 -- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8105d20 msgr2=0x7f17e81a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8105d20 0x7f17e81a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e819b1e0 con 0x7f17e810e800 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17edbf0700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 0x7f17e819ab00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17e0007ee0 tx=0x7f17e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0004bb0 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e819b4d0 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e0004d50 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0005920 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.523+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e81a7770 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.524+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17e8063e20 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.526+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f17e001a020 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.526+0000 7f17de7fc700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17d40afc10 0x7f17d40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.526+0000 7f17ed3ef700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17d40afc10 0x7f17d40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.526+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f17e00dec90 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.526+0000 7f17de7fc700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] conn(0x7f17d40b9440 0x7f17d40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.527+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17e0019e70 con 0x7f17d40b9440 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.527+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f17e00df170 con 0x7f17e810e800 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.527+0000 7f17ed3ef700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17d40afc10 0x7f17d40b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b5c0 tx=0x7f17e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.527+0000 7f17ecbee700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] conn(0x7f17d40b9440 0x7f17d40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.528+0000 7f17ecbee700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] conn(0x7f17d40b9440 0x7f17d40bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.528+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== osd.1 v2:172.21.15.137:6800/1683710608 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17e0019e70 con 0x7f17d40b9440 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.545+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 --> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17e80642a0 con 0x7f17d40b9440 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17de7fc700 1 -- 172.21.15.94:0/3401017908 <== osd.1 v2:172.21.15.137:6800/1683710608 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f17e80642a0 con 0x7f17d40b9440 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] conn(0x7f17d40b9440 msgr2=0x7f17d40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] conn(0x7f17d40b9440 0x7f17d40bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17d40afc10 msgr2=0x7f17d40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17d40afc10 0x7f17d40b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b5c0 tx=0x7f17e401a040 comp rx=0 tx=0).stop 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 msgr2=0x7f17e819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 --2- 172.21.15.94:0/3401017908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e810e800 0x7f17e819ab00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17e0007ee0 tx=0x7f17e000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17edbf0700 1 -- 172.21.15.94:0/3401017908 reap_dead start 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 shutdown_connections 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.546+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 >> 172.21.15.94:0/3401017908 conn(0x7f17e80fee00 msgr2=0x7f17e8100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.547+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 shutdown_connections 2024-04-25T13:40:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.547+0000 7f17ef653700 1 -- 172.21.15.94:0/3401017908 wait complete. 2024-04-25T13:40:27.520 INFO:teuthology.orchestra.run.smithi094.stdout:55834574919 2024-04-25T13:40:27.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:40:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 -- 172.21.15.94:0/2617188477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 msgr2=0x7f9be4110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 --2- 172.21.15.94:0/2617188477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be4110cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00b0a0 tx=0x7f9bdc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 -- 172.21.15.94:0/2617188477 shutdown_connections 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 --2- 172.21.15.94:0/2617188477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be4110cd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 --2- 172.21.15.94:0/2617188477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41053c0 0x7f9be41057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 --2- 172.21.15.94:0/2617188477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4105d10 0x7f9be410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 -- 172.21.15.94:0/2617188477 >> 172.21.15.94:0/2617188477 conn(0x7f9be4078cd0 msgr2=0x7f9be4079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 -- 172.21.15.94:0/2617188477 shutdown_connections 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.032+0000 7f9bec59d700 1 -- 172.21.15.94:0/2617188477 wait complete. 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 Processor -- start 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 -- start start 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be41053c0 0x7f9be41a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be4105d10 0x7f9be41a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4114370 con 0x7f9be410e7f0 2024-04-25T13:40:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41141f0 con 0x7f9be41053c0 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bec59d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41144f0 con 0x7f9be4105d10 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40098/0 (socket says 172.21.15.94:40098) 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 -- 172.21.15.94:0/752865277 learned_addr learned my addr 172.21.15.94:0/752865277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9bea339700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be41053c0 0x7f9be41a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9be9b38700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be4105d10 0x7f9be41a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 -- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be4105d10 msgr2=0x7f9be41a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be4105d10 0x7f9be41a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 -- 172.21.15.94:0/752865277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be41053c0 msgr2=0x7f9be41a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be41053c0 0x7f9be41a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9beab3a700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be419b260 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.033+0000 7f9be9b38700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be4105d10 0x7f9be41a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9beab3a700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be419ab30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0062a0 tx=0x7f9bdc004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc021070 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bdc003c00 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be419b4f0 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004830 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.034+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be41a7750 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.035+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9be4063da0 con 0x7f9be410e7f0 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.036+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9bdc007a00 con 0x7f9be410e7f0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bdb7fe700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afc90 0x7f9bd00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bea339700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afc90 0x7f9bd00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f9bdc0e0020 con 0x7f9be410e7f0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bdb7fe700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] conn(0x7f9bd00b94c0 0x7f9bd00bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bdc0075c0 con 0x7f9bd00b94c0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f9bdc0e0500 con 0x7f9be410e7f0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9bea339700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afc90 0x7f9bd00b21a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009bf0 tx=0x7f9bd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.037+0000 7f9be9b38700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] conn(0x7f9bd00b94c0 0x7f9bd00bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.038+0000 7f9be9b38700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] conn(0x7f9bd00b94c0 0x7f9bd00bb910 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.039+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== osd.8 v2:172.21.15.138:6816/3084581661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bdc0075c0 con 0x7f9bd00b94c0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.055+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 --> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9be4064220 con 0x7f9bd00b94c0 2024-04-25T13:40:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.055+0000 7f9bdb7fe700 1 -- 172.21.15.94:0/752865277 <== osd.8 v2:172.21.15.138:6816/3084581661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9be4064220 con 0x7f9bd00b94c0 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] conn(0x7f9bd00b94c0 msgr2=0x7f9bd00bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] conn(0x7f9bd00b94c0 0x7f9bd00bb910 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afc90 msgr2=0x7f9bd00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afc90 0x7f9bd00b21a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009bf0 tx=0x7f9bd4009260 comp rx=0 tx=0).stop 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 msgr2=0x7f9be419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 --2- 172.21.15.94:0/752865277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be410e7f0 0x7f9be419ab30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0062a0 tx=0x7f9bdc004d30 comp rx=0 tx=0).stop 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9beab3a700 1 -- 172.21.15.94:0/752865277 reap_dead start 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 shutdown_connections 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 >> 172.21.15.94:0/752865277 conn(0x7f9be4078cd0 msgr2=0x7f9be40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 shutdown_connections 2024-04-25T13:40:27.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:24.056+0000 7f9bec59d700 1 -- 172.21.15.94:0/752865277 wait complete. 2024-04-25T13:40:28.358 INFO:teuthology.orchestra.run.smithi094.stdout:240518168613 2024-04-25T13:40:28.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 -- 172.21.15.94:0/985398056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 msgr2=0x7f94bc111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 --2- 172.21.15.94:0/985398056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc111040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f94a40099c0 tx=0x7f94a4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 -- 172.21.15.94:0/985398056 shutdown_connections 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 --2- 172.21.15.94:0/985398056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc111040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 --2- 172.21.15.94:0/985398056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94bc104460 0x7f94bc104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 --2- 172.21.15.94:0/985398056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc108d90 0x7f94bc1091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 -- 172.21.15.94:0/985398056 >> 172.21.15.94:0/985398056 conn(0x7f94bc0ffd40 msgr2=0x7f94bc102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 -- 172.21.15.94:0/985398056 shutdown_connections 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 -- 172.21.15.94:0/985398056 wait complete. 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.342+0000 7f94c136d700 1 Processor -- start 2024-04-25T13:40:28.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 -- start start 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94bc104460 0x7f94bc1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc108d90 0x7f94bc1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc114380 con 0x7f94bc1096e0 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc114200 con 0x7f94bc104460 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94c136d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc114500 con 0x7f94bc108d90 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60310/0 (socket says 172.21.15.94:60310) 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 -- 172.21.15.94:0/2277637182 learned_addr learned my addr 172.21.15.94:0/2277637182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94baffd700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94bc104460 0x7f94bc1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94ba7fc700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc108d90 0x7f94bc1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 -- 172.21.15.94:0/2277637182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc108d90 msgr2=0x7f94bc1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc108d90 0x7f94bc1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 -- 172.21.15.94:0/2277637182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94bc104460 msgr2=0x7f94bc1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94bc104460 0x7f94bc1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.343+0000 7f94bb7fe700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc19b1c0 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94bb7fe700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc19aae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005910 tx=0x7f94a4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4027070 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a4003f20 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc19b450 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4022c50 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.344+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc1a7750 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.345+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94bc063e20 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f94a402a070 con 0x7f94bc1096e0 2024-04-25T13:40:28.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94a80afc10 0x7f94a80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94baffd700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94a80afc10 0x7f94a80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f94a40dffb0 con 0x7f94bc1096e0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] conn(0x7f94a80b9440 0x7f94a80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94a400eea0 con 0x7f94a80b9440 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.347+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f94a40e0490 con 0x7f94bc1096e0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.348+0000 7f94ba7fc700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] conn(0x7f94a80b9440 0x7f94a80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.348+0000 7f94baffd700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94a80afc10 0x7f94a80b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009550 tx=0x7f94b40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.348+0000 7f94ba7fc700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] conn(0x7f94a80b9440 0x7f94a80bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.349+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== osd.3 v2:172.21.15.94:6810/3561678692 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94a400eea0 con 0x7f94a80b9440 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.365+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 --> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94bc0642a0 con 0x7f94a80b9440 2024-04-25T13:40:28.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.365+0000 7f94b3fff700 1 -- 172.21.15.94:0/2277637182 <== osd.3 v2:172.21.15.94:6810/3561678692 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94bc0642a0 con 0x7f94a80b9440 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.365+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] conn(0x7f94a80b9440 msgr2=0x7f94a80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.365+0000 7f94c136d700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] conn(0x7f94a80b9440 0x7f94a80bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94a80afc10 msgr2=0x7f94a80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94a80afc10 0x7f94a80b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009550 tx=0x7f94b40091b0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 msgr2=0x7f94bc19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 --2- 172.21.15.94:0/2277637182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94bc1096e0 0x7f94bc19aae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005910 tx=0x7f94a4009580 comp rx=0 tx=0).stop 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94bb7fe700 1 -- 172.21.15.94:0/2277637182 reap_dead start 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 shutdown_connections 2024-04-25T13:40:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 >> 172.21.15.94:0/2277637182 conn(0x7f94bc0ffd40 msgr2=0x7f94bc108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:28.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 shutdown_connections 2024-04-25T13:40:28.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:23.366+0000 7f94c136d700 1 -- 172.21.15.94:0/2277637182 wait complete. 2024-04-25T13:40:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:28 smithi137 ceph-mon[32568]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T13:40:28.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:28 smithi094 ceph-mon[29195]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T13:40:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:28 smithi138 ceph-mon[32977]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T13:40:29.119 INFO:teuthology.orchestra.run.smithi094.stdout:111669149758 2024-04-25T13:40:29.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:40:29.754 INFO:teuthology.orchestra.run.smithi094.stdout:34359738445 2024-04-25T13:40:29.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:40:30.165 INFO:teuthology.orchestra.run.smithi094.stdout:77309411393 2024-04-25T13:40:30.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:40:30.311 INFO:teuthology.orchestra.run.smithi094.stdout:210453397545 2024-04-25T13:40:30.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:40:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:30 smithi094 ceph-mon[29195]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T13:40:30.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:30 smithi137 ceph-mon[32568]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T13:40:30.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:30 smithi138 ceph-mon[32977]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T13:40:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:266287972384 2024-04-25T13:40:31.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:40:31.340 INFO:teuthology.orchestra.run.smithi094.stdout:292057776154 2024-04-25T13:40:31.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:40:31.797 INFO:teuthology.orchestra.run.smithi094.stdout:163208757299 2024-04-25T13:40:31.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:40:32.192 INFO:teuthology.orchestra.run.smithi094.stdout:137438953526 2024-04-25T13:40:32.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:40:32.388 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.389 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.394 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.395 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.399 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.399 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.401 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.404 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:32 smithi094 ceph-mon[29195]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T13:40:32.480 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:32.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:32 smithi137 ceph-mon[32568]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T13:40:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:32 smithi138 ceph-mon[32977]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-25T13:40:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:34 smithi137 ceph-mon[32568]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:34 smithi094 ceph-mon[29195]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:34 smithi138 ceph-mon[32977]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:40:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:36 smithi137 ceph-mon[32568]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:36 smithi094 ceph-mon[29195]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:36 smithi138 ceph-mon[32977]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:40:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:38 smithi137 ceph-mon[32568]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:40:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:38 smithi094 ceph-mon[29195]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:40:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:38 smithi138 ceph-mon[32977]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:40:40.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-25T13:40:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:40:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T13:40:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T13:40:40.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T13:40:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T13:40:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T13:40:41.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-25T13:40:41.582 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T13:40:41.582 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:40:41.582 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:41 smithi094 ceph-mon[29195]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-25T13:40:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T13:40:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:40:41.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:41 smithi137 ceph-mon[32568]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T13:40:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T13:40:41.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:41 smithi138 ceph-mon[32977]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 -- 172.21.15.94:0/157984723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94741044b0 msgr2=0x7f94741048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 --2- 172.21.15.94:0/157984723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94741044b0 0x7f94741048c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f94640099c0 tx=0x7f9464009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 -- 172.21.15.94:0/157984723 shutdown_connections 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 --2- 172.21.15.94:0/157984723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474109730 0x7f9474111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 --2- 172.21.15.94:0/157984723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94741044b0 0x7f94741048c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 --2- 172.21.15.94:0/157984723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9474108de0 0x7f94741091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 -- 172.21.15.94:0/157984723 >> 172.21.15.94:0/157984723 conn(0x7f94740ffd40 msgr2=0x7f9474102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 -- 172.21.15.94:0/157984723 shutdown_connections 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.271+0000 7f9479348700 1 -- 172.21.15.94:0/157984723 wait complete. 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.272+0000 7f9479348700 1 Processor -- start 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.272+0000 7f9479348700 1 -- start start 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94741044b0 0x7f947419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474108de0 0x7f947419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9474114360 con 0x7f9474109730 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94741141e0 con 0x7f94741044b0 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9479348700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94741144e0 con 0x7f9474108de0 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51736/0 (socket says 172.21.15.94:51736) 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 -- 172.21.15.94:0/774172836 learned_addr learned my addr 172.21.15.94:0/774172836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f9472ffd700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94741044b0 0x7f947419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94727fc700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474108de0 0x7f947419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 -- 172.21.15.94:0/774172836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474108de0 msgr2=0x7f947419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474108de0 0x7f947419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 -- 172.21.15.94:0/774172836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94741044b0 msgr2=0x7f947419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94741044b0 0x7f947419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 -- 172.21.15.94:0/774172836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94741a1da0 con 0x7f9474109730 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.273+0000 7f94737fe700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f946c007ee0 tx=0x7f946c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.274+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c004bb0 con 0x7f9474109730 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.274+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94741a2090 con 0x7f9474109730 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.274+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c004d50 con 0x7f9474109730 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.274+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c005920 con 0x7f9474109730 2024-04-25T13:40:42.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.274+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94741a99b0 con 0x7f9474109730 2024-04-25T13:40:42.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.275+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9474051190 con 0x7f9474109730 2024-04-25T13:40:42.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.276+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f946c01a020 con 0x7f9474109730 2024-04-25T13:40:42.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.277+0000 7f945bfff700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 0x7f945c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.277+0000 7f9472ffd700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 0x7f945c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.277+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f946c0dead0 con 0x7f9474109730 2024-04-25T13:40:42.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.277+0000 7f9472ffd700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 0x7f945c0b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f946401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.281+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f946c010070 con 0x7f9474109730 2024-04-25T13:40:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:42 smithi094 ceph-mon[29195]: pgmap v312: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-04-25T13:40:42.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.491+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f94741a9b80 con 0x7f9474109730 2024-04-25T13:40:42.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.491+0000 7f945bfff700 1 -- 172.21.15.94:0/774172836 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f946c0a7530 con 0x7f9474109730 2024-04-25T13:40:42.492 INFO:teuthology.orchestra.run.smithi094.stdout:55834574922 2024-04-25T13:40:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 msgr2=0x7f945c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 0x7f945c0b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f946401a040 comp rx=0 tx=0).stop 2024-04-25T13:40:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 msgr2=0x7f94741a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f946c007ee0 tx=0x7f946c00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 shutdown_connections 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f945c0afc10 0x7f945c0b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9474109730 0x7f94741a1700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94741044b0 0x7f947419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 --2- 172.21.15.94:0/774172836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9474108de0 0x7f947419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 >> 172.21.15.94:0/774172836 conn(0x7f94740ffd40 msgr2=0x7f947410a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 shutdown_connections 2024-04-25T13:40:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:42.494+0000 7f9479348700 1 -- 172.21.15.94:0/774172836 wait complete. 2024-04-25T13:40:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:42 smithi137 ceph-mon[32568]: pgmap v312: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-04-25T13:40:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:42 smithi138 ceph-mon[32977]: pgmap v312: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-04-25T13:40:43.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/774172836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:40:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/774172836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:40:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/774172836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:40:44.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 -- 172.21.15.94:0/3346163497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 msgr2=0x7f78501095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:44.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3346163497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f78501095b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7840009990 tx=0x7f7840009ca0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 -- 172.21.15.94:0/3346163497 shutdown_connections 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3346163497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102f80 0x7f7850111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3346163497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f78501095b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3346163497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102630 0x7f7850102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 -- 172.21.15.94:0/3346163497 >> 172.21.15.94:0/3346163497 conn(0x7f78500fe3f0 msgr2=0x7f7850100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 -- 172.21.15.94:0/3346163497 shutdown_connections 2024-04-25T13:40:44.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 -- 172.21.15.94:0/3346163497 wait complete. 2024-04-25T13:40:44.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.048+0000 7f7857ec0700 1 Processor -- start 2024-04-25T13:40:44.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 -- start start 2024-04-25T13:40:44.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 0x7f785019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:44.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102f80 0x7f785019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7850114440 con 0x7f78501091a0 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78501142c0 con 0x7f7850102630 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7857ec0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78501145c0 con 0x7f7850102f80 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51910/0 (socket says 172.21.15.94:51910) 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 -- 172.21.15.94:0/3795445569 learned_addr learned my addr 172.21.15.94:0/3795445569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785545b700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102f80 0x7f785019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7855c5c700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 0x7f785019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 -- 172.21.15.94:0/3795445569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102f80 msgr2=0x7f785019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102f80 0x7f785019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 -- 172.21.15.94:0/3795445569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 msgr2=0x7f785019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:44.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 0x7f785019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 -- 172.21.15.94:0/3795445569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785019fd00 con 0x7f78501091a0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f7855c5c700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 0x7f785019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.049+0000 7f785645d700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7848007ee0 tx=0x7f784800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.050+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784800ddd0 con 0x7f78501091a0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.050+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7848004020 con 0x7f78501091a0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.050+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785019fff0 con 0x7f78501091a0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.050+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848015660 con 0x7f78501091a0 2024-04-25T13:40:44.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.050+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a7910 con 0x7f78501091a0 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.052+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7850051190 con 0x7f78501091a0 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.052+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7848015800 con 0x7f78501091a0 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.053+0000 7f7846ffd700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 0x7f783c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.053+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f78480de870 con 0x7f78501091a0 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.053+0000 7f7855c5c700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 0x7f783c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:44.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.054+0000 7f7855c5c700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 0x7f783c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7840006010 tx=0x7f784001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:44.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.059+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7848010070 con 0x7f78501091a0 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.263+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f785002e510 con 0x7f78501091a0 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.263+0000 7f7846ffd700 1 -- 172.21.15.94:0/3795445569 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78480a72d0 con 0x7f78501091a0 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi094.stdout:77309411397 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.265+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 msgr2=0x7f783c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.265+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 0x7f783c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7840006010 tx=0x7f784001a040 comp rx=0 tx=0).stop 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.265+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 msgr2=0x7f785019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.265+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7848007ee0 tx=0x7f784800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 shutdown_connections 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f783c0afc10 0x7f783c0b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501091a0 0x7f785019f660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7850102630 0x7f785019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 --2- 172.21.15.94:0/3795445569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7850102f80 0x7f785019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 >> 172.21.15.94:0/3795445569 conn(0x7f78500fe3f0 msgr2=0x7f7850100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 shutdown_connections 2024-04-25T13:40:44.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:44.266+0000 7f7857ec0700 1 -- 172.21.15.94:0/3795445569 wait complete. 2024-04-25T13:40:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3795445569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:40:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:44 smithi137 ceph-mon[32568]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:40:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3795445569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:40:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:44 smithi094 ceph-mon[29195]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:40:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3795445569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:40:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:44 smithi138 ceph-mon[32977]: pgmap v313: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T13:40:45.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2024-04-25T13:40:45.117 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:45.599 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:45 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:40:45.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:45 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:40:45.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:45 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:40:45.947 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411397 for osd.2 2024-04-25T13:40:45.948 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 -- 172.21.15.94:0/3632735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2301085e0 msgr2=0x7fb23010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 --2- 172.21.15.94:0/3632735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2301085e0 0x7fb23010ff40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b0d0 tx=0x7fb22800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 -- 172.21.15.94:0/3632735912 shutdown_connections 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 --2- 172.21.15.94:0/3632735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2301085e0 0x7fb23010ff40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 --2- 172.21.15.94:0/3632735912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230103360 0x7fb230103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 --2- 172.21.15.94:0/3632735912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230107c90 0x7fb2301080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 -- 172.21.15.94:0/3632735912 >> 172.21.15.94:0/3632735912 conn(0x7fb2300fec90 msgr2=0x7fb2301010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 -- 172.21.15.94:0/3632735912 shutdown_connections 2024-04-25T13:40:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.141+0000 7fb234d03700 1 -- 172.21.15.94:0/3632735912 wait complete. 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 Processor -- start 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 -- start start 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 0x7fb2301a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2301085e0 0x7fb23019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb230113210 con 0x7fb230103360 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.142+0000 7fb234d03700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb230113090 con 0x7fb230107c90 2024-04-25T13:40:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb234d03700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb230113390 con 0x7fb2301085e0 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51926/0 (socket says 172.21.15.94:51926) 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 -- 172.21.15.94:0/77110532 learned_addr learned my addr 172.21.15.94:0/77110532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22dd9b700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 0x7fb2301a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22ed9d700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2301085e0 0x7fb23019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 -- 172.21.15.94:0/77110532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2301085e0 msgr2=0x7fb23019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2301085e0 0x7fb23019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 -- 172.21.15.94:0/77110532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 msgr2=0x7fb2301a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 0x7fb2301a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 -- 172.21.15.94:0/77110532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23019b150 con 0x7fb230103360 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22dd9b700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 0x7fb2301a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb22e59c700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9a0 tx=0x7fb22000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000ca10 con 0x7fb230103360 2024-04-25T13:40:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23019b440 con 0x7fb230103360 2024-04-25T13:40:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22000cbb0 con 0x7fb230103360 2024-04-25T13:40:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220018680 con 0x7fb230103360 2024-04-25T13:40:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.143+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301a7710 con 0x7fb230103360 2024-04-25T13:40:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.144+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230051190 con 0x7fb230103360 2024-04-25T13:40:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.146+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb220010430 con 0x7fb230103360 2024-04-25T13:40:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.147+0000 7fb21f7fe700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 0x7fb2180b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.147+0000 7fb22dd9b700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 0x7fb2180b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.147+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb2200df2f0 con 0x7fb230103360 2024-04-25T13:40:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.148+0000 7fb22dd9b700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 0x7fb2180b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb230109740 tx=0x7fb224009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:46.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.151+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2200a7d50 con 0x7fb230103360 2024-04-25T13:40:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.367+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb23019bfd0 con 0x7fb230103360 2024-04-25T13:40:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.368+0000 7fb21f7fe700 1 -- 172.21.15.94:0/77110532 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb2200a7650 con 0x7fb230103360 2024-04-25T13:40:46.368 INFO:teuthology.orchestra.run.smithi094.stdout:111669149762 2024-04-25T13:40:46.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 msgr2=0x7fb2180b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 0x7fb2180b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb230109740 tx=0x7fb224009260 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 msgr2=0x7fb2301a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e9a0 tx=0x7fb22000ecb0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 shutdown_connections 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2301085e0 0x7fb23019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2180afc10 0x7fb2180b2120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb230103360 0x7fb2301a0880 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 --2- 172.21.15.94:0/77110532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb230107c90 0x7fb2301a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 >> 172.21.15.94:0/77110532 conn(0x7fb2300fec90 msgr2=0x7fb23006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 shutdown_connections 2024-04-25T13:40:46.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.370+0000 7fb234d03700 1 -- 172.21.15.94:0/77110532 wait complete. 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/77110532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2248822592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:40:47.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4271139528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/77110532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2248822592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:40:47.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4271139528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/77110532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2248822592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:40:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4271139528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:40:48.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149762 for osd.3 2024-04-25T13:40:48.330 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 -- 172.21.15.94:0/3145053506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 msgr2=0x7f56bc10fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/3145053506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc10fff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56b400b0d0 tx=0x7f56b400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 -- 172.21.15.94:0/3145053506 shutdown_connections 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/3145053506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc10fff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/3145053506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc1039e0 0x7f56bc103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/3145053506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc107d00 0x7f56bc108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 -- 172.21.15.94:0/3145053506 >> 172.21.15.94:0/3145053506 conn(0x7f56bc0fe370 msgr2=0x7f56bc1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 -- 172.21.15.94:0/3145053506 shutdown_connections 2024-04-25T13:40:48.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.532+0000 7f56c1fb4700 1 -- 172.21.15.94:0/3145053506 wait complete. 2024-04-25T13:40:48.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.533+0000 7f56c1fb4700 1 Processor -- start 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.533+0000 7f56c1fb4700 1 -- start start 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.533+0000 7f56c1fb4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc1039e0 0x7f56bc0fe850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.533+0000 7f56c1fb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc107d00 0x7f56bc0fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56c1fb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56c1fb4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1132c0 con 0x7f56bc108650 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56c1fb4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113140 con 0x7f56bc1039e0 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56c1fb4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113440 con 0x7f56bc107d00 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52006/0 (socket says 172.21.15.94:52006) 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc1039e0 0x7f56bc0fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 -- 172.21.15.94:0/976960969 learned_addr learned my addr 172.21.15.94:0/976960969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:48.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56baffd700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc107d00 0x7f56bc0fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 -- 172.21.15.94:0/976960969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc107d00 msgr2=0x7f56bc0fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc107d00 0x7f56bc0fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 -- 172.21.15.94:0/976960969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc1039e0 msgr2=0x7f56bc0fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc1039e0 0x7f56bc0fe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 -- 172.21.15.94:0/976960969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1a8ff0 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56bbfff700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56b40095c0 tx=0x7f56b4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b401f070 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.534+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc1a9280 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.535+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a9730 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.535+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b4003c00 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.535+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b401b6a0 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.535+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a8005330 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.537+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f56b4022070 con 0x7f56bc108650 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.538+0000 7f56b8ff9700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 0x7f56a40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:48.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.538+0000 7f56bb7fe700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 0x7f56a40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:48.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.538+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f56b400f020 con 0x7f56bc108650 2024-04-25T13:40:48.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.538+0000 7f56bb7fe700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 0x7f56a40b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007be0 tx=0x7f56ac0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:48.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.543+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56b4029020 con 0x7f56bc108650 2024-04-25T13:40:48.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:48 smithi094 ceph-mon[29195]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:48 smithi137 ceph-mon[32568]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:48 smithi138 ceph-mon[32977]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-04-25T13:40:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.749+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f56a80058f0 con 0x7f56bc108650 2024-04-25T13:40:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.749+0000 7f56b8ff9700 1 -- 172.21.15.94:0/976960969 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f56b40a8e50 con 0x7f56bc108650 2024-04-25T13:40:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:317827579930 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.751+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 msgr2=0x7f56a40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 0x7f56a40b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007be0 tx=0x7f56ac0073d0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 msgr2=0x7f56bc0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56b40095c0 tx=0x7f56b4007a20 comp rx=0 tx=0).stop 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 shutdown_connections 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56a40afc10 0x7f56a40b2120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56bc108650 0x7f56bc0ff2d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56bc1039e0 0x7f56bc0fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 --2- 172.21.15.94:0/976960969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56bc107d00 0x7f56bc0fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 >> 172.21.15.94:0/976960969 conn(0x7f56bc0fe370 msgr2=0x7f56bc1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 shutdown_connections 2024-04-25T13:40:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:48.752+0000 7f56c1fb4700 1 -- 172.21.15.94:0/976960969 wait complete. 2024-04-25T13:40:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 -- 172.21.15.94:0/2876374169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 msgr2=0x7fe17c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 --2- 172.21.15.94:0/2876374169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c102c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe164009990 tx=0x7fe164009ca0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 -- 172.21.15.94:0/2876374169 shutdown_connections 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 --2- 172.21.15.94:0/2876374169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c10f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 --2- 172.21.15.94:0/2876374169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 0x7fe17c108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 --2- 172.21.15.94:0/2876374169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c102c80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 -- 172.21.15.94:0/2876374169 >> 172.21.15.94:0/2876374169 conn(0x7fe17c0fe3b0 msgr2=0x7fe17c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 -- 172.21.15.94:0/2876374169 shutdown_connections 2024-04-25T13:40:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.359+0000 7fe1818aa700 1 -- 172.21.15.94:0/2876374169 wait complete. 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 Processor -- start 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 -- start start 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 0x7fe17c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c112190 con 0x7fe17c102870 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c112010 con 0x7fe17c1088a0 2024-04-25T13:40:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe1818aa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c112310 con 0x7fe17c1031c0 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52010/0 (socket says 172.21.15.94:52010) 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 -- 172.21.15.94:0/353610536 learned_addr learned my addr 172.21.15.94:0/353610536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe173fff700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 -- 172.21.15.94:0/353610536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 msgr2=0x7fe17c1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 -- 172.21.15.94:0/353610536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 msgr2=0x7fe17c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17b7fe700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 0x7fe17c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 0x7fe17c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.360+0000 7fe17affd700 1 -- 172.21.15.94:0/353610536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c19b1c0 con 0x7fe17c102870 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe173fff700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe17affd700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d980 tx=0x7fe16c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c017070 con 0x7fe17c102870 2024-04-25T13:40:49.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17c19b4b0 con 0x7fe17c102870 2024-04-25T13:40:49.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16c010460 con 0x7fe17c102870 2024-04-25T13:40:49.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00de60 con 0x7fe17c102870 2024-04-25T13:40:49.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.361+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17c1987a0 con 0x7fe17c102870 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.362+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17c051190 con 0x7fe17c102870 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.364+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe16c010610 con 0x7fe17c102870 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.364+0000 7fe1808a8700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 0x7fe1680b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.364+0000 7fe173fff700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 0x7fe1680b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.365+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe16c0de910 con 0x7fe17c102870 2024-04-25T13:40:49.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.365+0000 7fe173fff700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 0x7fe1680b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe164006010 tx=0x7fe16401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:49.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.368+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe16c0a7370 con 0x7fe17c102870 2024-04-25T13:40:49.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/976960969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:40:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.578+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe17c02e510 con 0x7fe17c102870 2024-04-25T13:40:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.579+0000 7fe1808a8700 1 -- 172.21.15.94:0/353610536 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe16c0a6c70 con 0x7fe17c102870 2024-04-25T13:40:49.580 INFO:teuthology.orchestra.run.smithi094.stdout:210453397550 2024-04-25T13:40:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 msgr2=0x7fe1680b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 0x7fe1680b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe164006010 tx=0x7fe16401a040 comp rx=0 tx=0).stop 2024-04-25T13:40:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 msgr2=0x7fe17c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d980 tx=0x7fe16c00dd40 comp rx=0 tx=0).stop 2024-04-25T13:40:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 shutdown_connections 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe17c1088a0 0x7fe17c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe1680afc10 0x7fe1680b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe17c102870 0x7fe17c1a08a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 --2- 172.21.15.94:0/353610536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe17c1031c0 0x7fe17c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 >> 172.21.15.94:0/353610536 conn(0x7fe17c0fe3b0 msgr2=0x7fe17c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 shutdown_connections 2024-04-25T13:40:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:49.581+0000 7fe1818aa700 1 -- 172.21.15.94:0/353610536 wait complete. 2024-04-25T13:40:49.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/976960969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:40:49.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/976960969' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 -- 172.21.15.94:0/2686936493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 msgr2=0x7f054c103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 --2- 172.21.15.94:0/2686936493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c103760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05400099c0 tx=0x7f0540009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 -- 172.21.15.94:0/2686936493 shutdown_connections 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 --2- 172.21.15.94:0/2686936493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c10ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 --2- 172.21.15.94:0/2686936493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c103760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 --2- 172.21.15.94:0/2686936493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 0x7f054c108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 -- 172.21.15.94:0/2686936493 >> 172.21.15.94:0/2686936493 conn(0x7f054c0fec30 msgr2=0x7f054c101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 -- 172.21.15.94:0/2686936493 shutdown_connections 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.182+0000 7f055221a700 1 -- 172.21.15.94:0/2686936493 wait complete. 2024-04-25T13:40:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 Processor -- start 2024-04-25T13:40:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 -- start start 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 0x7f054c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f054c113200 con 0x7f054c103350 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f054c113080 con 0x7f054c1085d0 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f055221a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f054c113380 con 0x7f054c107c80 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f054b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f054b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52014/0 (socket says 172.21.15.94:52014) 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.183+0000 7f054b7fe700 1 -- 172.21.15.94:0/3100094029 learned_addr learned my addr 172.21.15.94:0/3100094029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054affd700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 0x7f054c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054bfff700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 -- 172.21.15.94:0/3100094029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 msgr2=0x7f054c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 0x7f054c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 -- 172.21.15.94:0/3100094029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 msgr2=0x7f054c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 -- 172.21.15.94:0/3100094029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c19b280 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054bfff700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f054b7fe700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f054000b580 tx=0x7f05400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540027070 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0540003f20 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c19b510 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540022d70 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.184+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1a7770 con 0x7f054c103350 2024-04-25T13:40:50.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.185+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c19b9f0 con 0x7f054c103350 2024-04-25T13:40:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.187+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0540022500 con 0x7f054c103350 2024-04-25T13:40:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.188+0000 7f0548ff9700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 0x7f05340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.188+0000 7f054affd700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 0x7f05340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.188+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f05400dff60 con 0x7f054c103350 2024-04-25T13:40:50.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.188+0000 7f054affd700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 0x7f05340b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f054c19c280 tx=0x7f053c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:50.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.192+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0540013070 con 0x7f054c103350 2024-04-25T13:40:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.403+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f054c19bff0 con 0x7f054c103350 2024-04-25T13:40:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.404+0000 7f0548ff9700 1 -- 172.21.15.94:0/3100094029 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f05400a89f0 con 0x7f054c103350 2024-04-25T13:40:50.404 INFO:teuthology.orchestra.run.smithi094.stdout:292057776160 2024-04-25T13:40:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 msgr2=0x7f05340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 0x7f05340b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f054c19c280 tx=0x7f053c00a300 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 msgr2=0x7f054c1a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f054000b580 tx=0x7f05400047b0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 shutdown_connections 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f054c1085d0 0x7f054c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05340afc10 0x7f05340b2120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054c103350 0x7f054c1a09c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 --2- 172.21.15.94:0/3100094029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054c107c80 0x7f054c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 >> 172.21.15.94:0/3100094029 conn(0x7f054c0fec30 msgr2=0x7f054c106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 shutdown_connections 2024-04-25T13:40:50.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:50.406+0000 7f055221a700 1 -- 172.21.15.94:0/3100094029 wait complete. 2024-04-25T13:40:50.595 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:50 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/353610536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:40:50.595 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:50 smithi094 ceph-mon[29195]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T13:40:50.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397545 got 210453397550 for osd.7 2024-04-25T13:40:50.622 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:50 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/353610536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:40:50.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:50 smithi137 ceph-mon[32568]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T13:40:50.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:50 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/353610536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:40:50.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:50 smithi138 ceph-mon[32977]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 -- 172.21.15.94:0/1743608645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4108610 msgr2=0x7f98b410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 --2- 172.21.15.94:0/1743608645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4108610 0x7f98b410ff70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f98a40099c0 tx=0x7f98a4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 -- 172.21.15.94:0/1743608645 shutdown_connections 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 --2- 172.21.15.94:0/1743608645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4108610 0x7f98b410ff70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 --2- 172.21.15.94:0/1743608645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4103390 0x7f98b41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 --2- 172.21.15.94:0/1743608645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4107cc0 0x7f98b41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 -- 172.21.15.94:0/1743608645 >> 172.21.15.94:0/1743608645 conn(0x7f98b40fec90 msgr2=0x7f98b41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 -- 172.21.15.94:0/1743608645 shutdown_connections 2024-04-25T13:40:51.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.034+0000 7f98bb331700 1 -- 172.21.15.94:0/1743608645 wait complete. 2024-04-25T13:40:51.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 Processor -- start 2024-04-25T13:40:51.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 -- start start 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4103390 0x7f98b419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4108610 0x7f98b4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4113240 con 0x7f98b4107cc0 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41130c0 con 0x7f98b4108610 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.035+0000 7f98bb331700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41133c0 con 0x7f98b4103390 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52040/0 (socket says 172.21.15.94:52040) 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 -- 172.21.15.94:0/3598348878 learned_addr learned my addr 172.21.15.94:0/3598348878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b98ce700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4108610 0x7f98b4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b90cd700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4103390 0x7f98b419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 -- 172.21.15.94:0/3598348878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4103390 msgr2=0x7f98b419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4103390 0x7f98b419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 -- 172.21.15.94:0/3598348878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4108610 msgr2=0x7f98b4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4108610 0x7f98b4199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 -- 172.21.15.94:0/3598348878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b419a0d0 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b88cc700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c8f0 tx=0x7f98a800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8007940 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98a800f450 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b419a3c0 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8018660 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.036+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41a6690 con 0x7f98b4107cc0 2024-04-25T13:40:51.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.037+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b4051190 con 0x7f98b4107cc0 2024-04-25T13:40:51.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.039+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f98a800f890 con 0x7f98b4107cc0 2024-04-25T13:40:51.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.039+0000 7f98b27fc700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 0x7f989c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.040+0000 7f98b90cd700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 0x7f989c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.040+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f98a80deca0 con 0x7f98b4107cc0 2024-04-25T13:40:51.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.040+0000 7f98b90cd700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 0x7f989c0b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0060b0 tx=0x7f98ac0077b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.043+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98a8014030 con 0x7f98b4107cc0 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 -- 172.21.15.94:0/1148851048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548107cc0 msgr2=0x7fe5481080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 --2- 172.21.15.94:0/1148851048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548107cc0 0x7fe5481080d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe5440099c0 tx=0x7fe544009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 -- 172.21.15.94:0/1148851048 shutdown_connections 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 --2- 172.21.15.94:0/1148851048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548108610 0x7fe54810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 --2- 172.21.15.94:0/1148851048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 0x7fe5481037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 --2- 172.21.15.94:0/1148851048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548107cc0 0x7fe5481080d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 -- 172.21.15.94:0/1148851048 >> 172.21.15.94:0/1148851048 conn(0x7fe5480fec90 msgr2=0x7fe5481010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 -- 172.21.15.94:0/1148851048 shutdown_connections 2024-04-25T13:40:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.800+0000 7fe5502c1700 1 -- 172.21.15.94:0/1148851048 wait complete. 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 Processor -- start 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 -- start start 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 0x7fe5481a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 0x7fe5481a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe548113220 con 0x7fe548108610 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481130a0 con 0x7fe548107cc0 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe5502c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481133a0 con 0x7fe548103390 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe54e85e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe54e85e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51984/0 (socket says 172.21.15.94:51984) 2024-04-25T13:40:51.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe54e85e700 1 -- 172.21.15.94:0/4271139528 learned_addr learned my addr 172.21.15.94:0/4271139528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stdout:163208757304 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.801+0000 7fe54e05d700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 0x7fe5481a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54d85c700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 0x7fe5481a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 -- 172.21.15.94:0/4271139528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 msgr2=0x7fe5481a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 0x7fe5481a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 -- 172.21.15.94:0/4271139528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 msgr2=0x7fe5481a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 0x7fe5481a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 -- 172.21.15.94:0/4271139528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54819b1d0 con 0x7fe548108610 2024-04-25T13:40:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54d85c700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 0x7fe5481a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe54e85e700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe54000ef30 tx=0x7fe54000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5400092e0 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54819b460 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe540004bb0 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540004280 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.802+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a7760 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.803+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548051190 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.805+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe54001b030 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.805+0000 7fe53f7fe700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 0x7fe5340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.806+0000 7fe54e05d700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 0x7fe5340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.806+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe5400de8b0 con 0x7fe548108610 2024-04-25T13:40:51.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.806+0000 7fe54e05d700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 0x7fe5340b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe54819bc30 tx=0x7fe53800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.811+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe540022020 con 0x7fe548108610 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.017+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe54802e510 con 0x7fe548108610 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.017+0000 7fe53f7fe700 1 -- 172.21.15.94:0/4271139528 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe540010040 con 0x7fe548108610 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 msgr2=0x7fe5340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 0x7fe5340b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe54819bc30 tx=0x7fe53800b410 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 msgr2=0x7fe54819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe54000ef30 tx=0x7fe54000c3a0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 shutdown_connections 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5340afc10 0x7fe5340b2120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe548108610 0x7fe54819aaf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe548103390 0x7fe5481a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 --2- 172.21.15.94:0/4271139528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe548107cc0 0x7fe5481a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.019+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 >> 172.21.15.94:0/4271139528 conn(0x7fe5480fec90 msgr2=0x7fe54806b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.020+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 shutdown_connections 2024-04-25T13:40:51.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:47.020+0000 7fe5502c1700 1 -- 172.21.15.94:0/4271139528 wait complete. 2024-04-25T13:40:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.250+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f98b419af50 con 0x7f98b4107cc0 2024-04-25T13:40:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.251+0000 7f98b27fc700 1 -- 172.21.15.94:0/3598348878 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98a80a7810 con 0x7f98b4107cc0 2024-04-25T13:40:51.251 INFO:teuthology.orchestra.run.smithi094.stdout:240518168618 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 msgr2=0x7f989c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 0x7f989c0b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0060b0 tx=0x7f98ac0077b0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 msgr2=0x7f98b419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c8f0 tx=0x7f98a800cc00 comp rx=0 tx=0).stop 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 shutdown_connections 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98b4108610 0x7f98b4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98b4103390 0x7f98b419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f989c0afc90 0x7f989c0b21a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 --2- 172.21.15.94:0/3598348878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b4107cc0 0x7f98b419fd40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 >> 172.21.15.94:0/3598348878 conn(0x7f98b40fec90 msgr2=0x7f98b4107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 shutdown_connections 2024-04-25T13:40:51.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.253+0000 7f98bb331700 1 -- 172.21.15.94:0/3598348878 wait complete. 2024-04-25T13:40:51.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:51 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3100094029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:40:51.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:51 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3598348878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:40:51.660 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:51 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3100094029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:40:51.660 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:51 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3598348878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:40:51.718 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:51 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3100094029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:40:51.718 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:51 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3598348878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:40:51.750 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757304 for osd.5 2024-04-25T13:40:51.750 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 -- 172.21.15.94:0/3058000042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468107ce0 msgr2=0x7fe4681080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 --2- 172.21.15.94:0/3058000042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468107ce0 0x7fe4681080f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4580099c0 tx=0x7fe458009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 -- 172.21.15.94:0/3058000042 shutdown_connections 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 --2- 172.21.15.94:0/3058000042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468108630 0x7fe46810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 --2- 172.21.15.94:0/3058000042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 0x7fe4681037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 --2- 172.21.15.94:0/3058000042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468107ce0 0x7fe4681080f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 -- 172.21.15.94:0/3058000042 >> 172.21.15.94:0/3058000042 conn(0x7fe4680fee00 msgr2=0x7fe468101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 -- 172.21.15.94:0/3058000042 shutdown_connections 2024-04-25T13:40:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.950+0000 7fe46ff26700 1 -- 172.21.15.94:0/3058000042 wait complete. 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 Processor -- start 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 -- start start 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 0x7fe46819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468107ce0 0x7fe46819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681132b0 con 0x7fe468108630 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe468113130 con 0x7fe4681033b0 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46ff26700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe468113430 con 0x7fe468107ce0 2024-04-25T13:40:51.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46e4c3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46e4c3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52046/0 (socket says 172.21.15.94:52046) 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46e4c3700 1 -- 172.21.15.94:0/1121411327 learned_addr learned my addr 172.21.15.94:0/1121411327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46d4c1700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468107ce0 0x7fe46819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.951+0000 7fe46dcc2700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 0x7fe46819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 -- 172.21.15.94:0/1121411327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468107ce0 msgr2=0x7fe46819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468107ce0 0x7fe46819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 -- 172.21.15.94:0/1121411327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 msgr2=0x7fe46819bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 0x7fe46819bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 -- 172.21.15.94:0/1121411327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46819c980 con 0x7fe468108630 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46e4c3700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d980 tx=0x7fe46000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460017070 con 0x7fe468108630 2024-04-25T13:40:51.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681a7270 con 0x7fe468108630 2024-04-25T13:40:51.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe460005020 con 0x7fe468108630 2024-04-25T13:40:51.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4600099b0 con 0x7fe468108630 2024-04-25T13:40:51.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.952+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681a7730 con 0x7fe468108630 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.953+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe468051190 con 0x7fe468108630 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.954+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe460009b90 con 0x7fe468108630 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.955+0000 7fe45effd700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 0x7fe4540b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.955+0000 7fe46dcc2700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 0x7fe4540b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.955+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe4600dece0 con 0x7fe468108630 2024-04-25T13:40:51.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.955+0000 7fe46dcc2700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 0x7fe4540b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe464007dc0 tx=0x7fe4640072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:51.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:51.960+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe460020020 con 0x7fe468108630 2024-04-25T13:40:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 -- 172.21.15.94:0/3522333753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 msgr2=0x7f0718108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 --2- 172.21.15.94:0/3522333753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f0718108090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f070c0099c0 tx=0x7f070c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 -- 172.21.15.94:0/3522333753 shutdown_connections 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 --2- 172.21.15.94:0/3522333753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07181085d0 0x7f071810ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 --2- 172.21.15.94:0/3522333753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0718103350 0x7f0718103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 --2- 172.21.15.94:0/3522333753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f0718108090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 -- 172.21.15.94:0/3522333753 >> 172.21.15.94:0/3522333753 conn(0x7f07180fec30 msgr2=0x7f0718101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 -- 172.21.15.94:0/3522333753 shutdown_connections 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.499+0000 7f071de9e700 1 -- 172.21.15.94:0/3522333753 wait complete. 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 Processor -- start 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 -- start start 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 0x7f071819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f071819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 0x7f0718199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07181131e0 con 0x7f0718107c80 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0718113060 con 0x7f07181085d0 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f071de9e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0718113360 con 0x7f0718103350 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f07177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 0x7f071819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f0717fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 0x7f0718199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f07177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 0x7f071819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40332/0 (socket says 172.21.15.94:40332) 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f07177fe700 1 -- 172.21.15.94:0/2248822592 learned_addr learned my addr 172.21.15.94:0/2248822592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.500+0000 7f0716ffd700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f071819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:188978561075 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 -- 172.21.15.94:0/2248822592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 msgr2=0x7f071819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 0x7f071819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 -- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 msgr2=0x7f071819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f071819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 -- 172.21.15.94:0/2248822592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071819a0f0 con 0x7f07181085d0 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0716ffd700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f071819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0717fff700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 0x7f0718199a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0710007f10 tx=0x7f071000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710004db0 con 0x7f07181085d0 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071819a3e0 con 0x7f07181085d0 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0710009e20 con 0x7f07181085d0 2024-04-25T13:40:52.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710015400 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.501+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181a66b0 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.502+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0718051190 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.504+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0710005260 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.504+0000 7f0714ff9700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 0x7f07000b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.504+0000 7f07177fe700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 0x7f07000b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.504+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f07100deaf0 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.505+0000 7f07177fe700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 0x7f07000b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f071819ab80 tx=0x7f0708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.508+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0710010070 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.723+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f071802e510 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.724+0000 7f0714ff9700 1 -- 172.21.15.94:0/2248822592 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f07100a7550 con 0x7f07181085d0 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 msgr2=0x7f07000b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 0x7f07000b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f071819ab80 tx=0x7f0708009260 comp rx=0 tx=0).stop 2024-04-25T13:40:52.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 msgr2=0x7f0718199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 0x7f0718199a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0710007f10 tx=0x7f071000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 shutdown_connections 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07181085d0 0x7f0718199a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0718103350 0x7f071819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07000afc90 0x7f07000b21a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 --2- 172.21.15.94:0/2248822592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0718107c80 0x7f071819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 >> 172.21.15.94:0/2248822592 conn(0x7f07180fec30 msgr2=0x7f0718109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 shutdown_connections 2024-04-25T13:40:52.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:46.726+0000 7f071de9e700 1 -- 172.21.15.94:0/2248822592 wait complete. 2024-04-25T13:40:52.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.167+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe46802e510 con 0x7fe468108630 2024-04-25T13:40:52.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.167+0000 7fe45effd700 1 -- 172.21.15.94:0/1121411327 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4600a7740 con 0x7fe468108630 2024-04-25T13:40:52.167 INFO:teuthology.orchestra.run.smithi094.stdout:34359738450 2024-04-25T13:40:52.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 msgr2=0x7fe4540b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 0x7fe4540b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe464007dc0 tx=0x7fe4640072f0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 msgr2=0x7fe4681a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d980 tx=0x7fe46000dc90 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 shutdown_connections 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4540afc10 0x7fe4540b2120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe468108630 0x7fe4681a6c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4681033b0 0x7fe46819bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 --2- 172.21.15.94:0/1121411327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe468107ce0 0x7fe46819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 >> 172.21.15.94:0/1121411327 conn(0x7fe4680fee00 msgr2=0x7fe468107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 shutdown_connections 2024-04-25T13:40:52.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:52.169+0000 7fe46ff26700 1 -- 172.21.15.94:0/1121411327 wait complete. 2024-04-25T13:40:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:52 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1121411327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:40:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:52 smithi137 ceph-mon[32568]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T13:40:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:52 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1121411327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:40:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:52 smithi094 ceph-mon[29195]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T13:40:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:52 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1121411327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:40:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:52 smithi138 ceph-mon[32977]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T13:40:52.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561075 for osd.6 2024-04-25T13:40:52.835 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:53.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738450 for osd.0 2024-04-25T13:40:53.293 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:53.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579925 got 317827579930 for osd.11 2024-04-25T13:40:53.737 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:54.133 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:54.381 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:54 smithi094 ceph-mon[29195]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:54 smithi137 ceph-mon[32568]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:54.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:54 smithi138 ceph-mon[32977]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:54.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 -- 172.21.15.94:0/486615284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664102a20 msgr2=0x7f1664102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 --2- 172.21.15.94:0/486615284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664102a20 0x7f1664102e30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f16540099c0 tx=0x7f1654009cd0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 -- 172.21.15.94:0/486615284 shutdown_connections 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 --2- 172.21.15.94:0/486615284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664109720 0x7f1664111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 --2- 172.21.15.94:0/486615284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664102a20 0x7f1664102e30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 --2- 172.21.15.94:0/486615284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664108dd0 0x7f16641091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 -- 172.21.15.94:0/486615284 >> 172.21.15.94:0/486615284 conn(0x7f16640fe5c0 msgr2=0x7f16641009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 -- 172.21.15.94:0/486615284 shutdown_connections 2024-04-25T13:40:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.874+0000 7f166a3cb700 1 -- 172.21.15.94:0/486615284 wait complete. 2024-04-25T13:40:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.875+0000 7f166a3cb700 1 Processor -- start 2024-04-25T13:40:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.875+0000 7f166a3cb700 1 -- start start 2024-04-25T13:40:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 0x7f166419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:54.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664108dd0 0x7f166419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16641143c0 con 0x7f1664109720 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1664114240 con 0x7f1664102a20 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1664114540 con 0x7f1664108dd0 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50124/0 (socket says 172.21.15.94:50124) 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 -- 172.21.15.94:0/1495168901 learned_addr learned my addr 172.21.15.94:0/1495168901 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1663fff700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 0x7f166419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f16637fe700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664108dd0 0x7f166419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 -- 172.21.15.94:0/1495168901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664108dd0 msgr2=0x7f166419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664108dd0 0x7f166419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 -- 172.21.15.94:0/1495168901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 msgr2=0x7f166419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 0x7f166419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 -- 172.21.15.94:0/1495168901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166419fbb0 con 0x7f1664109720 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1663fff700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 0x7f166419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f1668968700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f165c007d70 tx=0x7f165c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c016020 con 0x7f1664109720 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165c00ad40 con 0x7f1664109720 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166419fea0 con 0x7f1664109720 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.876+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c0115e0 con 0x7f1664109720 2024-04-25T13:40:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.877+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16641a03e0 con 0x7f1664109720 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.877+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1664051190 con 0x7f1664109720 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.879+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f165c019020 con 0x7f1664109720 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.879+0000 7f16617fa700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 0x7f164c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.880+0000 7f1663fff700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 0x7f164c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.880+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f165c0df010 con 0x7f1664109720 2024-04-25T13:40:54.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.880+0000 7f1663fff700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 0x7f164c0b21a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1654003800 tx=0x7f165401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:54.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:54.883+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f165c020020 con 0x7f1664109720 2024-04-25T13:40:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.087+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f166419c0f0 con 0x7f1664109720 2024-04-25T13:40:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.087+0000 7f16617fa700 1 -- 172.21.15.94:0/1495168901 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f165c0a7c20 con 0x7f1664109720 2024-04-25T13:40:55.088 INFO:teuthology.orchestra.run.smithi094.stdout:266287972389 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.089+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 msgr2=0x7f164c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.089+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 0x7f164c0b21a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1654003800 tx=0x7f165401a040 comp rx=0 tx=0).stop 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.089+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 msgr2=0x7f166419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.089+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f165c007d70 tx=0x7f165c00a340 comp rx=0 tx=0).stop 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 shutdown_connections 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f164c0afc90 0x7f164c0b21a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1664109720 0x7f166419f5c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1664102a20 0x7f166419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 --2- 172.21.15.94:0/1495168901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1664108dd0 0x7f166419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 >> 172.21.15.94:0/1495168901 conn(0x7f16640fe5c0 msgr2=0x7f1664108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 shutdown_connections 2024-04-25T13:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:55.090+0000 7f166a3cb700 1 -- 172.21.15.94:0/1495168901 wait complete. 2024-04-25T13:40:55.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776160 for osd.10 2024-04-25T13:40:55.326 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1495168901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:40:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1495168901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:40:55.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1495168901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:40:56.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972384 got 266287972389 for osd.9 2024-04-25T13:40:56.394 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:56 smithi137 ceph-mon[32568]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:56.675 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:56 smithi094 ceph-mon[29195]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:56 smithi138 ceph-mon[32977]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T13:40:57.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168618 for osd.8 2024-04-25T13:40:57.076 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 -- 172.21.15.94:0/3166495025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234108610 msgr2=0x7f323410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 --2- 172.21.15.94:0/3166495025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234108610 0x7f323410ff70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b0d0 tx=0x7f322c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 -- 172.21.15.94:0/3166495025 shutdown_connections 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 --2- 172.21.15.94:0/3166495025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234108610 0x7f323410ff70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 --2- 172.21.15.94:0/3166495025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234103390 0x7f32341037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 --2- 172.21.15.94:0/3166495025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234107cc0 0x7f32341080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 -- 172.21.15.94:0/3166495025 >> 172.21.15.94:0/3166495025 conn(0x7f32340fec90 msgr2=0x7f32341010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 -- 172.21.15.94:0/3166495025 shutdown_connections 2024-04-25T13:40:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.984+0000 7f323b540700 1 -- 172.21.15.94:0/3166495025 wait complete. 2024-04-25T13:40:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 Processor -- start 2024-04-25T13:40:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 -- start start 2024-04-25T13:40:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 0x7f323419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234108610 0x7f3234199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3234113240 con 0x7f3234107cc0 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32341130c0 con 0x7f3234108610 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f323b540700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32341133c0 con 0x7f3234103390 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f3238adb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f3238adb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50140/0 (socket says 172.21.15.94:50140) 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.985+0000 7f3238adb700 1 -- 172.21.15.94:0/1044728926 learned_addr learned my addr 172.21.15.94:0/1044728926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f32392dc700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 0x7f323419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3239add700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234108610 0x7f3234199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 -- 172.21.15.94:0/1044728926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 msgr2=0x7f323419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 0x7f323419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 -- 172.21.15.94:0/1044728926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234108610 msgr2=0x7f3234199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234108610 0x7f3234199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 -- 172.21.15.94:0/1044728926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234199fc0 con 0x7f3234107cc0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f32392dc700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 0x7f323419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f3238adb700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f322400c9e0 tx=0x7f322400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224007aa0 con 0x7f3234107cc0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3224007c40 con 0x7f3234107cc0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323419a2b0 con 0x7f3234107cc0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224018660 con 0x7f3234107cc0 2024-04-25T13:40:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.986+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32341a65f0 con 0x7f3234107cc0 2024-04-25T13:40:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.987+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3234051190 con 0x7f3234107cc0 2024-04-25T13:40:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.989+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3224018800 con 0x7f3234107cc0 2024-04-25T13:40:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.989+0000 7f322a7fc700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 0x7f32200b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.989+0000 7f32392dc700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 0x7f32200b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.989+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f32240def90 con 0x7f3234107cc0 2024-04-25T13:40:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.990+0000 7f32392dc700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 0x7f32200b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3230005950 tx=0x7f3230005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:57.993+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3224053ff0 con 0x7f3234107cc0 2024-04-25T13:40:58.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.194+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f323419aeb0 con 0x7f3234107cc0 2024-04-25T13:40:58.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.194+0000 7f322a7fc700 1 -- 172.21.15.94:0/1044728926 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3224014030 con 0x7f3234107cc0 2024-04-25T13:40:58.195 INFO:teuthology.orchestra.run.smithi094.stdout:137438953535 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 msgr2=0x7f32200b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 0x7f32200b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3230005950 tx=0x7f3230005f50 comp rx=0 tx=0).stop 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 msgr2=0x7f323419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f322400c9e0 tx=0x7f322400cda0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 shutdown_connections 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.196+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3234108610 0x7f3234199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3234103390 0x7f323419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32200afc10 0x7f32200b2120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 --2- 172.21.15.94:0/1044728926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3234107cc0 0x7f323419fca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 >> 172.21.15.94:0/1044728926 conn(0x7f32340fec90 msgr2=0x7f3234107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 shutdown_connections 2024-04-25T13:40:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:40:58.197+0000 7f323b540700 1 -- 172.21.15.94:0/1044728926 wait complete. 2024-04-25T13:40:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:58 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1044728926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:40:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:58 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1044728926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:40:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:58 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1044728926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:40:58.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953526 got 137438953535 for osd.4 2024-04-25T13:40:58.963 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:58.964 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:40:58.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:40:59.351 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:40:59.563 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:40:59 smithi094 ceph-mon[29195]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:40:59.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:40:59 smithi137 ceph-mon[32568]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:40:59.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:40:59 smithi138 ceph-mon[32977]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:00.604 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:00 smithi094 ceph-mon[29195]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:00 smithi137 ceph-mon[32568]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:00.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:00 smithi138 ceph-mon[32977]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 -- 172.21.15.94:0/3165612872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105370 msgr2=0x7fb274105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 --2- 172.21.15.94:0/3165612872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105370 0x7fb274105780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb264009990 tx=0x7fb264009ca0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 -- 172.21.15.94:0/3165612872 shutdown_connections 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 --2- 172.21.15.94:0/3165612872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 0x7fb274110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 --2- 172.21.15.94:0/3165612872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105370 0x7fb274105780 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 --2- 172.21.15.94:0/3165612872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105cc0 0x7fb27410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 -- 172.21.15.94:0/3165612872 >> 172.21.15.94:0/3165612872 conn(0x7fb274078d20 msgr2=0x7fb274079160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 -- 172.21.15.94:0/3165612872 shutdown_connections 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 -- 172.21.15.94:0/3165612872 wait complete. 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.977+0000 7fb2798d0700 1 Processor -- start 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 -- start start 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105370 0x7fb27419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 0x7fb27419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb274114390 con 0x7fb274105cc0 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb274114210 con 0x7fb27410e7e0 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2798d0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb274114510 con 0x7fb274105370 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50162/0 (socket says 172.21.15.94:50162) 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 -- 172.21.15.94:0/2249407318 learned_addr learned my addr 172.21.15.94:0/2249407318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb272ffd700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105370 0x7fb27419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2737fe700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 0x7fb27419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 -- 172.21.15.94:0/2249407318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105370 msgr2=0x7fb27419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105370 0x7fb27419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 -- 172.21.15.94:0/2249407318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 msgr2=0x7fb27419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 0x7fb27419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.978+0000 7fb2727fc700 1 -- 172.21.15.94:0/2249407318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419fc30 con 0x7fb274105cc0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2727fc700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d980 tx=0x7fb26800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb274105cc0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419ff20 con 0x7fb274105cc0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb26800fd40 con 0x7fb274105cc0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800e3b0 con 0x7fb274105cc0 2024-04-25T13:41:00.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.979+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a7840 con 0x7fb274105cc0 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.980+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274051190 con 0x7fb274105cc0 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.981+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb268010610 con 0x7fb274105cc0 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.982+0000 7fb2788ce700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 0x7fb25c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.982+0000 7fb272ffd700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 0x7fb25c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.982+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fb2680de820 con 0x7fb274105cc0 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.983+0000 7fb272ffd700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 0x7fb25c0b2120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb264006010 tx=0x7fb264005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:00.987+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2680a7280 con 0x7fb274105cc0 2024-04-25T13:41:01.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.209+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb27419c0c0 con 0x7fb25c0afc10 2024-04-25T13:41:01.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.216+0000 7fb2788ce700 1 -- 172.21.15.94:0/2249407318 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258755 (secure 0 0 0) 0x7fb2640672e0 con 0x7fb25c0afc10 2024-04-25T13:41:01.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:01.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 msgr2=0x7fb25c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 0x7fb25c0b2120 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb264006010 tx=0x7fb264005960 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 msgr2=0x7fb27419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d980 tx=0x7fb26800dc90 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 shutdown_connections 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb27410e7e0 0x7fb27419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb274105370 0x7fb27419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb25c0afc10 0x7fb25c0b2120 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 --2- 172.21.15.94:0/2249407318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb274105cc0 0x7fb27419b300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 >> 172.21.15.94:0/2249407318 conn(0x7fb274078d20 msgr2=0x7fb274100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 shutdown_connections 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:01.220+0000 7fb2798d0700 1 -- 172.21.15.94:0/2249407318 wait complete. 2024-04-25T13:41:01.223 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:41:01.921 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-25T13:41:00.275915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656295,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":773,"num_read_kb":481,"num_write":360,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403408,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124538416,"statfs":{"total":1151940427776,"available":1151527337984,"internally_reserved":0,"allocated":12890112,"data_stored":7028073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266364+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:40:41.266364+0000","last_peered":"2024-04-25T13:40:41.266364+0000","last_clean":"2024-04-25T13:40:41.266364+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:40:41.266364+0000","last_undegraded":"2024-04-25T13:40:41.266364+0000","last_fullsized":"2024-04-25T13:40:41.266364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264015+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:40:41.264015+0000","last_peered":"2024-04-25T13:40:41.264015+0000","last_clean":"2024-04-25T13:40:41.264015+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:40:41.264015+0000","last_undegraded":"2024-04-25T13:40:41.264015+0000","last_fullsized":"2024-04-25T13:40:41.264015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263998+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:40:41.263998+0000","last_peered":"2024-04-25T13:40:41.263998+0000","last_clean":"2024-04-25T13:40:41.263998+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:40:41.263998+0000","last_undegraded":"2024-04-25T13:40:41.263998+0000","last_fullsized":"2024-04-25T13:40:41.263998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262062+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:40:41.262062+0000","last_peered":"2024-04-25T13:40:41.262062+0000","last_clean":"2024-04-25T13:40:41.262062+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:40:41.262062+0000","last_undegraded":"2024-04-25T13:40:41.262062+0000","last_fullsized":"2024-04-25T13:40:41.262062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267592+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:40:41.267592+0000","last_peered":"2024-04-25T13:40:41.267592+0000","last_clean":"2024-04-25T13:40:41.267592+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:40:41.267592+0000","last_undegraded":"2024-04-25T13:40:41.267592+0000","last_fullsized":"2024-04-25T13:40:41.267592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.265086+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:40:41.265086+0000","last_peered":"2024-04-25T13:40:41.265086+0000","last_clean":"2024-04-25T13:40:41.265086+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:40:41.265086+0000","last_undegraded":"2024-04-25T13:40:41.265086+0000","last_fullsized":"2024-04-25T13:40:41.265086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271762+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:40:41.271762+0000","last_peered":"2024-04-25T13:40:41.271762+0000","last_clean":"2024-04-25T13:40:41.271762+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:40:41.271762+0000","last_undegraded":"2024-04-25T13:40:41.271762+0000","last_fullsized":"2024-04-25T13:40:41.271762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266635+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:40:41.266635+0000","last_peered":"2024-04-25T13:40:41.266635+0000","last_clean":"2024-04-25T13:40:41.266635+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:40:41.266635+0000","last_undegraded":"2024-04-25T13:40:41.266635+0000","last_fullsized":"2024-04-25T13:40:41.266635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273152+0000","last_change":"2024-04-25T13:40:41.273152+0000","last_active":"2024-04-25T13:40:41.273152+0000","last_peered":"2024-04-25T13:40:41.273152+0000","last_clean":"2024-04-25T13:40:41.273152+0000","last_became_active":"2024-04-25T13:40:41.272760+0000","last_became_peered":"2024-04-25T13:40:41.272760+0000","last_unstale":"2024-04-25T13:40:41.273152+0000","last_undegraded":"2024-04-25T13:40:41.273152+0000","last_fullsized":"2024-04-25T13:40:41.273152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:35.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.274781+0000","last_change":"2024-04-25T13:40:41.274781+0000","last_active":"2024-04-25T13:40:41.274781+0000","last_peered":"2024-04-25T13:40:41.274781+0000","last_clean":"2024-04-25T13:40:41.274781+0000","last_became_active":"2024-04-25T13:40:41.274043+0000","last_became_peered":"2024-04-25T13:40:41.274043+0000","last_unstale":"2024-04-25T13:40:41.274781+0000","last_undegraded":"2024-04-25T13:40:41.274781+0000","last_fullsized":"2024-04-25T13:40:41.274781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:01.989580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273905+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:40:41.273905+0000","last_peered":"2024-04-25T13:40:41.273905+0000","last_clean":"2024-04-25T13:40:41.273905+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:40:41.273905+0000","last_undegraded":"2024-04-25T13:40:41.273905+0000","last_fullsized":"2024-04-25T13:40:41.273905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262232+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:40:41.262232+0000","last_peered":"2024-04-25T13:40:41.262232+0000","last_clean":"2024-04-25T13:40:41.262232+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:40:41.262232+0000","last_undegraded":"2024-04-25T13:40:41.262232+0000","last_fullsized":"2024-04-25T13:40:41.262232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260761+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:40:41.260761+0000","last_peered":"2024-04-25T13:40:41.260761+0000","last_clean":"2024-04-25T13:40:41.260761+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:40:41.260761+0000","last_undegraded":"2024-04-25T13:40:41.260761+0000","last_fullsized":"2024-04-25T13:40:41.260761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263572+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:40:41.263572+0000","last_peered":"2024-04-25T13:40:41.263572+0000","last_clean":"2024-04-25T13:40:41.263572+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:40:41.263572+0000","last_undegraded":"2024-04-25T13:40:41.263572+0000","last_fullsized":"2024-04-25T13:40:41.263572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263294+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:40:41.263294+0000","last_peered":"2024-04-25T13:40:41.263294+0000","last_clean":"2024-04-25T13:40:41.263294+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:40:41.263294+0000","last_undegraded":"2024-04-25T13:40:41.263294+0000","last_fullsized":"2024-04-25T13:40:41.263294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266268+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:40:41.266268+0000","last_peered":"2024-04-25T13:40:41.266268+0000","last_clean":"2024-04-25T13:40:41.266268+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:40:41.266268+0000","last_undegraded":"2024-04-25T13:40:41.266268+0000","last_fullsized":"2024-04-25T13:40:41.266268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266361+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:40:41.266361+0000","last_peered":"2024-04-25T13:40:41.266361+0000","last_clean":"2024-04-25T13:40:41.266361+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:40:41.266361+0000","last_undegraded":"2024-04-25T13:40:41.266361+0000","last_fullsized":"2024-04-25T13:40:41.266361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270855+0000","last_change":"2024-04-25T13:39:45.259538+0000","last_active":"2024-04-25T13:40:41.270855+0000","last_peered":"2024-04-25T13:40:41.270855+0000","last_clean":"2024-04-25T13:40:41.270855+0000","last_became_active":"2024-04-25T13:39:45.259097+0000","last_became_peered":"2024-04-25T13:39:45.259097+0000","last_unstale":"2024-04-25T13:40:41.270855+0000","last_undegraded":"2024-04-25T13:40:41.270855+0000","last_fullsized":"2024-04-25T13:40:41.270855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:10.901989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268904+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:40:41.268904+0000","last_peered":"2024-04-25T13:40:41.268904+0000","last_clean":"2024-04-25T13:40:41.268904+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:40:41.268904+0000","last_undegraded":"2024-04-25T13:40:41.268904+0000","last_fullsized":"2024-04-25T13:40:41.268904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":101,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.745283+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:40:57.745283+0000","last_peered":"2024-04-25T13:40:57.745283+0000","last_clean":"2024-04-25T13:40:57.745283+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:40:57.745283+0000","last_undegraded":"2024-04-25T13:40:57.745283+0000","last_fullsized":"2024-04-25T13:40:57.745283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266676+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:40:41.266676+0000","last_peered":"2024-04-25T13:40:41.266676+0000","last_clean":"2024-04-25T13:40:41.266676+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:40:41.266676+0000","last_undegraded":"2024-04-25T13:40:41.266676+0000","last_fullsized":"2024-04-25T13:40:41.266676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268139+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:40:41.268139+0000","last_peered":"2024-04-25T13:40:41.268139+0000","last_clean":"2024-04-25T13:40:41.268139+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:40:41.268139+0000","last_undegraded":"2024-04-25T13:40:41.268139+0000","last_fullsized":"2024-04-25T13:40:41.268139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264016+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:40:41.264016+0000","last_peered":"2024-04-25T13:40:41.264016+0000","last_clean":"2024-04-25T13:40:41.264016+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:40:41.264016+0000","last_undegraded":"2024-04-25T13:40:41.264016+0000","last_fullsized":"2024-04-25T13:40:41.264016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267992+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:40:41.267992+0000","last_peered":"2024-04-25T13:40:41.267992+0000","last_clean":"2024-04-25T13:40:41.267992+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:40:41.267992+0000","last_undegraded":"2024-04-25T13:40:41.267992+0000","last_fullsized":"2024-04-25T13:40:41.267992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266256+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:40:41.266256+0000","last_peered":"2024-04-25T13:40:41.266256+0000","last_clean":"2024-04-25T13:40:41.266256+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:40:41.266256+0000","last_undegraded":"2024-04-25T13:40:41.266256+0000","last_fullsized":"2024-04-25T13:40:41.266256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268053+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:40:41.268053+0000","last_peered":"2024-04-25T13:40:41.268053+0000","last_clean":"2024-04-25T13:40:41.268053+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:40:41.268053+0000","last_undegraded":"2024-04-25T13:40:41.268053+0000","last_fullsized":"2024-04-25T13:40:41.268053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264026+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:40:41.264026+0000","last_peered":"2024-04-25T13:40:41.264026+0000","last_clean":"2024-04-25T13:40:41.264026+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:40:41.264026+0000","last_undegraded":"2024-04-25T13:40:41.264026+0000","last_fullsized":"2024-04-25T13:40:41.264026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:56.745332+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:40:56.745332+0000","last_peered":"2024-04-25T13:40:56.745332+0000","last_clean":"2024-04-25T13:40:56.745332+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:40:56.745332+0000","last_undegraded":"2024-04-25T13:40:56.745332+0000","last_fullsized":"2024-04-25T13:40:56.745332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272700+0000","last_change":"2024-04-25T13:40:41.272700+0000","last_active":"2024-04-25T13:40:41.272700+0000","last_peered":"2024-04-25T13:40:41.272700+0000","last_clean":"2024-04-25T13:40:41.272700+0000","last_became_active":"2024-04-25T13:40:41.272080+0000","last_became_peered":"2024-04-25T13:40:41.272080+0000","last_unstale":"2024-04-25T13:40:41.272700+0000","last_undegraded":"2024-04-25T13:40:41.272700+0000","last_fullsized":"2024-04-25T13:40:41.272700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.149458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.280199+0000","last_change":"2024-04-25T13:40:41.280199+0000","last_active":"2024-04-25T13:40:41.280199+0000","last_peered":"2024-04-25T13:40:41.280199+0000","last_clean":"2024-04-25T13:40:41.280199+0000","last_became_active":"2024-04-25T13:40:41.268773+0000","last_became_peered":"2024-04-25T13:40:41.268773+0000","last_unstale":"2024-04-25T13:40:41.280199+0000","last_undegraded":"2024-04-25T13:40:41.280199+0000","last_fullsized":"2024-04-25T13:40:41.280199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:47.024600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263517+0000","last_change":"2024-04-25T13:39:45.268193+0000","last_active":"2024-04-25T13:40:41.263517+0000","last_peered":"2024-04-25T13:40:41.263517+0000","last_clean":"2024-04-25T13:40:41.263517+0000","last_became_active":"2024-04-25T13:39:45.267842+0000","last_became_peered":"2024-04-25T13:39:45.267842+0000","last_unstale":"2024-04-25T13:40:41.263517+0000","last_undegraded":"2024-04-25T13:40:41.263517+0000","last_fullsized":"2024-04-25T13:40:41.263517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:56.308127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261116+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:40:41.261116+0000","last_peered":"2024-04-25T13:40:41.261116+0000","last_clean":"2024-04-25T13:40:41.261116+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:40:41.261116+0000","last_undegraded":"2024-04-25T13:40:41.261116+0000","last_fullsized":"2024-04-25T13:40:41.261116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266910+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:40:41.266910+0000","last_peered":"2024-04-25T13:40:41.266910+0000","last_clean":"2024-04-25T13:40:41.266910+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:40:41.266910+0000","last_undegraded":"2024-04-25T13:40:41.266910+0000","last_fullsized":"2024-04-25T13:40:41.266910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262954+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:40:41.262954+0000","last_peered":"2024-04-25T13:40:41.262954+0000","last_clean":"2024-04-25T13:40:41.262954+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:40:41.262954+0000","last_undegraded":"2024-04-25T13:40:41.262954+0000","last_fullsized":"2024-04-25T13:40:41.262954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268949+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:40:41.268949+0000","last_peered":"2024-04-25T13:40:41.268949+0000","last_clean":"2024-04-25T13:40:41.268949+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:40:41.268949+0000","last_undegraded":"2024-04-25T13:40:41.268949+0000","last_fullsized":"2024-04-25T13:40:41.268949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":151,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:54.741338+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:40:54.741338+0000","last_peered":"2024-04-25T13:40:54.741338+0000","last_clean":"2024-04-25T13:40:54.741338+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:40:54.741338+0000","last_undegraded":"2024-04-25T13:40:54.741338+0000","last_fullsized":"2024-04-25T13:40:54.741338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267305+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:40:41.267305+0000","last_peered":"2024-04-25T13:40:41.267305+0000","last_clean":"2024-04-25T13:40:41.267305+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:40:41.267305+0000","last_undegraded":"2024-04-25T13:40:41.267305+0000","last_fullsized":"2024-04-25T13:40:41.267305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273894+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:40:41.273894+0000","last_peered":"2024-04-25T13:40:41.273894+0000","last_clean":"2024-04-25T13:40:41.273894+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:40:41.273894+0000","last_undegraded":"2024-04-25T13:40:41.273894+0000","last_fullsized":"2024-04-25T13:40:41.273894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264060+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:40:41.264060+0000","last_peered":"2024-04-25T13:40:41.264060+0000","last_clean":"2024-04-25T13:40:41.264060+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:40:41.264060+0000","last_undegraded":"2024-04-25T13:40:41.264060+0000","last_fullsized":"2024-04-25T13:40:41.264060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264260+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:40:41.264260+0000","last_peered":"2024-04-25T13:40:41.264260+0000","last_clean":"2024-04-25T13:40:41.264260+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:40:41.264260+0000","last_undegraded":"2024-04-25T13:40:41.264260+0000","last_fullsized":"2024-04-25T13:40:41.264260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267041+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:40:41.267041+0000","last_peered":"2024-04-25T13:40:41.267041+0000","last_clean":"2024-04-25T13:40:41.267041+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:40:41.267041+0000","last_undegraded":"2024-04-25T13:40:41.267041+0000","last_fullsized":"2024-04-25T13:40:41.267041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":182,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:54.747130+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:40:54.747130+0000","last_peered":"2024-04-25T13:40:54.747130+0000","last_clean":"2024-04-25T13:40:54.747130+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:40:54.747130+0000","last_undegraded":"2024-04-25T13:40:54.747130+0000","last_fullsized":"2024-04-25T13:40:54.747130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":124,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267623+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:40:41.267623+0000","last_peered":"2024-04-25T13:40:41.267623+0000","last_clean":"2024-04-25T13:40:41.267623+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:40:41.267623+0000","last_undegraded":"2024-04-25T13:40:41.267623+0000","last_fullsized":"2024-04-25T13:40:41.267623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271079+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:40:41.271079+0000","last_peered":"2024-04-25T13:40:41.271079+0000","last_clean":"2024-04-25T13:40:41.271079+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:40:41.271079+0000","last_undegraded":"2024-04-25T13:40:41.271079+0000","last_fullsized":"2024-04-25T13:40:41.271079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260892+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:40:41.260892+0000","last_peered":"2024-04-25T13:40:41.260892+0000","last_clean":"2024-04-25T13:40:41.260892+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:40:41.260892+0000","last_undegraded":"2024-04-25T13:40:41.260892+0000","last_fullsized":"2024-04-25T13:40:41.260892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263559+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:40:41.263559+0000","last_peered":"2024-04-25T13:40:41.263559+0000","last_clean":"2024-04-25T13:40:41.263559+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:40:41.263559+0000","last_undegraded":"2024-04-25T13:40:41.263559+0000","last_fullsized":"2024-04-25T13:40:41.263559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272656+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:40:41.272656+0000","last_peered":"2024-04-25T13:40:41.272656+0000","last_clean":"2024-04-25T13:40:41.272656+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:40:41.272656+0000","last_undegraded":"2024-04-25T13:40:41.272656+0000","last_fullsized":"2024-04-25T13:40:41.272656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.752462+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:40:57.752462+0000","last_peered":"2024-04-25T13:40:57.752462+0000","last_clean":"2024-04-25T13:40:57.752462+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:40:57.752462+0000","last_undegraded":"2024-04-25T13:40:57.752462+0000","last_fullsized":"2024-04-25T13:40:57.752462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271554+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:40:41.271554+0000","last_peered":"2024-04-25T13:40:41.271554+0000","last_clean":"2024-04-25T13:40:41.271554+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:40:41.271554+0000","last_undegraded":"2024-04-25T13:40:41.271554+0000","last_fullsized":"2024-04-25T13:40:41.271554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268389+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:40:41.268389+0000","last_peered":"2024-04-25T13:40:41.268389+0000","last_clean":"2024-04-25T13:40:41.268389+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:40:41.268389+0000","last_undegraded":"2024-04-25T13:40:41.268389+0000","last_fullsized":"2024-04-25T13:40:41.268389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263122+0000","last_change":"2024-04-25T13:39:45.273219+0000","last_active":"2024-04-25T13:40:41.263122+0000","last_peered":"2024-04-25T13:40:41.263122+0000","last_clean":"2024-04-25T13:40:41.263122+0000","last_became_active":"2024-04-25T13:39:45.272891+0000","last_became_peered":"2024-04-25T13:39:45.272891+0000","last_unstale":"2024-04-25T13:40:41.263122+0000","last_undegraded":"2024-04-25T13:40:41.263122+0000","last_fullsized":"2024-04-25T13:40:41.263122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:40.531758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267700+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:40:41.267700+0000","last_peered":"2024-04-25T13:40:41.267700+0000","last_clean":"2024-04-25T13:40:41.267700+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:40:41.267700+0000","last_undegraded":"2024-04-25T13:40:41.267700+0000","last_fullsized":"2024-04-25T13:40:41.267700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271904+0000","last_change":"2024-04-25T13:40:41.271904+0000","last_active":"2024-04-25T13:40:41.271904+0000","last_peered":"2024-04-25T13:40:41.271904+0000","last_clean":"2024-04-25T13:40:41.271904+0000","last_became_active":"2024-04-25T13:40:41.271457+0000","last_became_peered":"2024-04-25T13:40:41.271457+0000","last_unstale":"2024-04-25T13:40:41.271904+0000","last_undegraded":"2024-04-25T13:40:41.271904+0000","last_fullsized":"2024-04-25T13:40:41.271904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:04.766001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270464+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:40:41.270464+0000","last_peered":"2024-04-25T13:40:41.270464+0000","last_clean":"2024-04-25T13:40:41.270464+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:40:41.270464+0000","last_undegraded":"2024-04-25T13:40:41.270464+0000","last_fullsized":"2024-04-25T13:40:41.270464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263373+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:40:41.263373+0000","last_peered":"2024-04-25T13:40:41.263373+0000","last_clean":"2024-04-25T13:40:41.263373+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:40:41.263373+0000","last_undegraded":"2024-04-25T13:40:41.263373+0000","last_fullsized":"2024-04-25T13:40:41.263373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.746133+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:40:57.746133+0000","last_peered":"2024-04-25T13:40:57.746133+0000","last_clean":"2024-04-25T13:40:57.746133+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:40:57.746133+0000","last_undegraded":"2024-04-25T13:40:57.746133+0000","last_fullsized":"2024-04-25T13:40:57.746133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263397+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:40:41.263397+0000","last_peered":"2024-04-25T13:40:41.263397+0000","last_clean":"2024-04-25T13:40:41.263397+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:40:41.263397+0000","last_undegraded":"2024-04-25T13:40:41.263397+0000","last_fullsized":"2024-04-25T13:40:41.263397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263547+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:40:41.263547+0000","last_peered":"2024-04-25T13:40:41.263547+0000","last_clean":"2024-04-25T13:40:41.263547+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:40:41.263547+0000","last_undegraded":"2024-04-25T13:40:41.263547+0000","last_fullsized":"2024-04-25T13:40:41.263547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.751817+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:40:57.751817+0000","last_peered":"2024-04-25T13:40:57.751817+0000","last_clean":"2024-04-25T13:40:57.751817+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:40:57.751817+0000","last_undegraded":"2024-04-25T13:40:57.751817+0000","last_fullsized":"2024-04-25T13:40:57.751817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266655+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:40:41.266655+0000","last_peered":"2024-04-25T13:40:41.266655+0000","last_clean":"2024-04-25T13:40:41.266655+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:40:41.266655+0000","last_undegraded":"2024-04-25T13:40:41.266655+0000","last_fullsized":"2024-04-25T13:40:41.266655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261992+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:40:41.261992+0000","last_peered":"2024-04-25T13:40:41.261992+0000","last_clean":"2024-04-25T13:40:41.261992+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:40:41.261992+0000","last_undegraded":"2024-04-25T13:40:41.261992+0000","last_fullsized":"2024-04-25T13:40:41.261992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272875+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:40:41.272875+0000","last_peered":"2024-04-25T13:40:41.272875+0000","last_clean":"2024-04-25T13:40:41.272875+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:40:41.272875+0000","last_undegraded":"2024-04-25T13:40:41.272875+0000","last_fullsized":"2024-04-25T13:40:41.272875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260431+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:40:41.260431+0000","last_peered":"2024-04-25T13:40:41.260431+0000","last_clean":"2024-04-25T13:40:41.260431+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:40:41.260431+0000","last_undegraded":"2024-04-25T13:40:41.260431+0000","last_fullsized":"2024-04-25T13:40:41.260431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273111+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:40:41.273111+0000","last_peered":"2024-04-25T13:40:41.273111+0000","last_clean":"2024-04-25T13:40:41.273111+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:40:41.273111+0000","last_undegraded":"2024-04-25T13:40:41.273111+0000","last_fullsized":"2024-04-25T13:40:41.273111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.280650+0000","last_change":"2024-04-25T13:40:41.280650+0000","last_active":"2024-04-25T13:40:41.280650+0000","last_peered":"2024-04-25T13:40:41.280650+0000","last_clean":"2024-04-25T13:40:41.280650+0000","last_became_active":"2024-04-25T13:40:41.271193+0000","last_became_peered":"2024-04-25T13:40:41.271193+0000","last_unstale":"2024-04-25T13:40:41.280650+0000","last_undegraded":"2024-04-25T13:40:41.280650+0000","last_fullsized":"2024-04-25T13:40:41.280650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.137595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261394+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:40:41.261394+0000","last_peered":"2024-04-25T13:40:41.261394+0000","last_clean":"2024-04-25T13:40:41.261394+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:40:41.261394+0000","last_undegraded":"2024-04-25T13:40:41.261394+0000","last_fullsized":"2024-04-25T13:40:41.261394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270998+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:40:41.270998+0000","last_peered":"2024-04-25T13:40:41.270998+0000","last_clean":"2024-04-25T13:40:41.270998+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:40:41.270998+0000","last_undegraded":"2024-04-25T13:40:41.270998+0000","last_fullsized":"2024-04-25T13:40:41.270998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268329+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:40:41.268329+0000","last_peered":"2024-04-25T13:40:41.268329+0000","last_clean":"2024-04-25T13:40:41.268329+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:40:41.268329+0000","last_undegraded":"2024-04-25T13:40:41.268329+0000","last_fullsized":"2024-04-25T13:40:41.268329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262137+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:40:41.262137+0000","last_peered":"2024-04-25T13:40:41.262137+0000","last_clean":"2024-04-25T13:40:41.262137+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:40:41.262137+0000","last_undegraded":"2024-04-25T13:40:41.262137+0000","last_fullsized":"2024-04-25T13:40:41.262137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263987+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:40:41.263987+0000","last_peered":"2024-04-25T13:40:41.263987+0000","last_clean":"2024-04-25T13:40:41.263987+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:40:41.263987+0000","last_undegraded":"2024-04-25T13:40:41.263987+0000","last_fullsized":"2024-04-25T13:40:41.263987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266827+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:40:41.266827+0000","last_peered":"2024-04-25T13:40:41.266827+0000","last_clean":"2024-04-25T13:40:41.266827+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:40:41.266827+0000","last_undegraded":"2024-04-25T13:40:41.266827+0000","last_fullsized":"2024-04-25T13:40:41.266827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267466+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:40:41.267466+0000","last_peered":"2024-04-25T13:40:41.267466+0000","last_clean":"2024-04-25T13:40:41.267466+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:40:41.267466+0000","last_undegraded":"2024-04-25T13:40:41.267466+0000","last_fullsized":"2024-04-25T13:40:41.267466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267078+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:40:41.267078+0000","last_peered":"2024-04-25T13:40:41.267078+0000","last_clean":"2024-04-25T13:40:41.267078+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:40:41.267078+0000","last_undegraded":"2024-04-25T13:40:41.267078+0000","last_fullsized":"2024-04-25T13:40:41.267078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264631+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:40:41.264631+0000","last_peered":"2024-04-25T13:40:41.264631+0000","last_clean":"2024-04-25T13:40:41.264631+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:40:41.264631+0000","last_undegraded":"2024-04-25T13:40:41.264631+0000","last_fullsized":"2024-04-25T13:40:41.264631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.284948+0000","last_change":"2024-04-25T13:40:41.284948+0000","last_active":"2024-04-25T13:40:41.284948+0000","last_peered":"2024-04-25T13:40:41.284948+0000","last_clean":"2024-04-25T13:40:41.284948+0000","last_became_active":"2024-04-25T13:40:41.272015+0000","last_became_peered":"2024-04-25T13:40:41.272015+0000","last_unstale":"2024-04-25T13:40:41.284948+0000","last_undegraded":"2024-04-25T13:40:41.284948+0000","last_fullsized":"2024-04-25T13:40:41.284948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:21.620303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266671+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:40:41.266671+0000","last_peered":"2024-04-25T13:40:41.266671+0000","last_clean":"2024-04-25T13:40:41.266671+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:40:41.266671+0000","last_undegraded":"2024-04-25T13:40:41.266671+0000","last_fullsized":"2024-04-25T13:40:41.266671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.282512+0000","last_change":"2024-04-25T13:40:41.282512+0000","last_active":"2024-04-25T13:40:41.282512+0000","last_peered":"2024-04-25T13:40:41.282512+0000","last_clean":"2024-04-25T13:40:41.282512+0000","last_became_active":"2024-04-25T13:40:41.275727+0000","last_became_peered":"2024-04-25T13:40:41.275727+0000","last_unstale":"2024-04-25T13:40:41.282512+0000","last_undegraded":"2024-04-25T13:40:41.282512+0000","last_fullsized":"2024-04-25T13:40:41.282512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.611067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271528+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:40:41.271528+0000","last_peered":"2024-04-25T13:40:41.271528+0000","last_clean":"2024-04-25T13:40:41.271528+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:40:41.271528+0000","last_undegraded":"2024-04-25T13:40:41.271528+0000","last_fullsized":"2024-04-25T13:40:41.271528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.746931+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:40:57.746931+0000","last_peered":"2024-04-25T13:40:57.746931+0000","last_clean":"2024-04-25T13:40:57.746931+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:40:57.746931+0000","last_undegraded":"2024-04-25T13:40:57.746931+0000","last_fullsized":"2024-04-25T13:40:57.746931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273977+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:40:41.273977+0000","last_peered":"2024-04-25T13:40:41.273977+0000","last_clean":"2024-04-25T13:40:41.273977+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:40:41.273977+0000","last_undegraded":"2024-04-25T13:40:41.273977+0000","last_fullsized":"2024-04-25T13:40:41.273977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263982+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:40:41.263982+0000","last_peered":"2024-04-25T13:40:41.263982+0000","last_clean":"2024-04-25T13:40:41.263982+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:40:41.263982+0000","last_undegraded":"2024-04-25T13:40:41.263982+0000","last_fullsized":"2024-04-25T13:40:41.263982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271918+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:40:41.271918+0000","last_peered":"2024-04-25T13:40:41.271918+0000","last_clean":"2024-04-25T13:40:41.271918+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:40:41.271918+0000","last_undegraded":"2024-04-25T13:40:41.271918+0000","last_fullsized":"2024-04-25T13:40:41.271918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261645+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:40:41.261645+0000","last_peered":"2024-04-25T13:40:41.261645+0000","last_clean":"2024-04-25T13:40:41.261645+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:40:41.261645+0000","last_undegraded":"2024-04-25T13:40:41.261645+0000","last_fullsized":"2024-04-25T13:40:41.261645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271682+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:40:41.271682+0000","last_peered":"2024-04-25T13:40:41.271682+0000","last_clean":"2024-04-25T13:40:41.271682+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:40:41.271682+0000","last_undegraded":"2024-04-25T13:40:41.271682+0000","last_fullsized":"2024-04-25T13:40:41.271682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264788+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:40:41.264788+0000","last_peered":"2024-04-25T13:40:41.264788+0000","last_clean":"2024-04-25T13:40:41.264788+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:40:41.264788+0000","last_undegraded":"2024-04-25T13:40:41.264788+0000","last_fullsized":"2024-04-25T13:40:41.264788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270467+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:40:41.270467+0000","last_peered":"2024-04-25T13:40:41.270467+0000","last_clean":"2024-04-25T13:40:41.270467+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:40:41.270467+0000","last_undegraded":"2024-04-25T13:40:41.270467+0000","last_fullsized":"2024-04-25T13:40:41.270467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268611+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:40:41.268611+0000","last_peered":"2024-04-25T13:40:41.268611+0000","last_clean":"2024-04-25T13:40:41.268611+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:40:41.268611+0000","last_undegraded":"2024-04-25T13:40:41.268611+0000","last_fullsized":"2024-04-25T13:40:41.268611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.274874+0000","last_change":"2024-04-25T13:40:41.274874+0000","last_active":"2024-04-25T13:40:41.274874+0000","last_peered":"2024-04-25T13:40:41.274874+0000","last_clean":"2024-04-25T13:40:41.274874+0000","last_became_active":"2024-04-25T13:40:41.274489+0000","last_became_peered":"2024-04-25T13:40:41.274489+0000","last_unstale":"2024-04-25T13:40:41.274874+0000","last_undegraded":"2024-04-25T13:40:41.274874+0000","last_fullsized":"2024-04-25T13:40:41.274874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:34.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261432+0000","last_change":"2024-04-25T13:39:45.261199+0000","last_active":"2024-04-25T13:40:41.261432+0000","last_peered":"2024-04-25T13:40:41.261432+0000","last_clean":"2024-04-25T13:40:41.261432+0000","last_became_active":"2024-04-25T13:39:45.260867+0000","last_became_peered":"2024-04-25T13:39:45.260867+0000","last_unstale":"2024-04-25T13:40:41.261432+0000","last_undegraded":"2024-04-25T13:40:41.261432+0000","last_fullsized":"2024-04-25T13:40:41.261432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:47.043364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266833+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:40:41.266833+0000","last_peered":"2024-04-25T13:40:41.266833+0000","last_clean":"2024-04-25T13:40:41.266833+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:40:41.266833+0000","last_undegraded":"2024-04-25T13:40:41.266833+0000","last_fullsized":"2024-04-25T13:40:41.266833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268409+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:40:41.268409+0000","last_peered":"2024-04-25T13:40:41.268409+0000","last_clean":"2024-04-25T13:40:41.268409+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:40:41.268409+0000","last_undegraded":"2024-04-25T13:40:41.268409+0000","last_fullsized":"2024-04-25T13:40:41.268409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266818+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:40:41.266818+0000","last_peered":"2024-04-25T13:40:41.266818+0000","last_clean":"2024-04-25T13:40:41.266818+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:40:41.266818+0000","last_undegraded":"2024-04-25T13:40:41.266818+0000","last_fullsized":"2024-04-25T13:40:41.266818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263023+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:40:41.263023+0000","last_peered":"2024-04-25T13:40:41.263023+0000","last_clean":"2024-04-25T13:40:41.263023+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:40:41.263023+0000","last_undegraded":"2024-04-25T13:40:41.263023+0000","last_fullsized":"2024-04-25T13:40:41.263023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267100+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:40:41.267100+0000","last_peered":"2024-04-25T13:40:41.267100+0000","last_clean":"2024-04-25T13:40:41.267100+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:40:41.267100+0000","last_undegraded":"2024-04-25T13:40:41.267100+0000","last_fullsized":"2024-04-25T13:40:41.267100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.276377+0000","last_change":"2024-04-25T13:40:41.276377+0000","last_active":"2024-04-25T13:40:41.276377+0000","last_peered":"2024-04-25T13:40:41.276377+0000","last_clean":"2024-04-25T13:40:41.276377+0000","last_became_active":"2024-04-25T13:40:41.275949+0000","last_became_peered":"2024-04-25T13:40:41.275949+0000","last_unstale":"2024-04-25T13:40:41.276377+0000","last_undegraded":"2024-04-25T13:40:41.276377+0000","last_fullsized":"2024-04-25T13:40:41.276377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:49.882272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260501+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:40:41.260501+0000","last_peered":"2024-04-25T13:40:41.260501+0000","last_clean":"2024-04-25T13:40:41.260501+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:40:41.260501+0000","last_undegraded":"2024-04-25T13:40:41.260501+0000","last_fullsized":"2024-04-25T13:40:41.260501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.265864+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:40:41.265864+0000","last_peered":"2024-04-25T13:40:41.265864+0000","last_clean":"2024-04-25T13:40:41.265864+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:40:41.265864+0000","last_undegraded":"2024-04-25T13:40:41.265864+0000","last_fullsized":"2024-04-25T13:40:41.265864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65693,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":697,"num_read_kb":417,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":422903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1589248,"data_stored":1072700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972391,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":913408,"data_stored":422721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":422987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1507328,"data_stored":1015617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":422438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":482231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":423777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":482842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40392,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704760,"statfs":{"total":95995035648,"available":95953674240,"internally_reserved":0,"allocated":913408,"data_stored":422263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":422299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36828,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708324,"statfs":{"total":95995035648,"available":95957323776,"internally_reserved":0,"allocated":1519616,"data_stored":1015295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:41:01.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:41:02.191 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:02.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:02.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:41:02.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:41:02.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:02 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:41:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:02 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:41:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:41:02.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:02 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:41:03.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:03 smithi094 ceph-mon[29195]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:03 smithi138 ceph-mon[32977]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:03.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:03 smithi137 ceph-mon[32568]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.848+0000 7f18f36d1700 1 -- 172.21.15.94:0/3811454575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec104490 msgr2=0x7f18ec1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.848+0000 7f18f36d1700 1 --2- 172.21.15.94:0/3811454575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec104490 0x7f18ec1048a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0099c0 tx=0x7f18dc009cd0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 -- 172.21.15.94:0/3811454575 shutdown_connections 2024-04-25T13:41:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 --2- 172.21.15.94:0/3811454575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec109710 0x7f18ec111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 --2- 172.21.15.94:0/3811454575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec104490 0x7f18ec1048a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 --2- 172.21.15.94:0/3811454575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec108dc0 0x7f18ec1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 -- 172.21.15.94:0/3811454575 >> 172.21.15.94:0/3811454575 conn(0x7f18ec0ffda0 msgr2=0x7f18ec1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 -- 172.21.15.94:0/3811454575 shutdown_connections 2024-04-25T13:41:03.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 -- 172.21.15.94:0/3811454575 wait complete. 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 Processor -- start 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.849+0000 7f18f36d1700 1 -- start start 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec104490 0x7f18ec19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 0x7f18ec19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec1143b0 con 0x7f18ec109710 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec114230 con 0x7f18ec108dc0 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec114530 con 0x7f18ec104490 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37916/0 (socket says 172.21.15.94:37916) 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 -- 172.21.15.94:0/2378004357 learned_addr learned my addr 172.21.15.94:0/2378004357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f0c6c700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 0x7f18ec19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f146d700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec104490 0x7f18ec19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 -- 172.21.15.94:0/2378004357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec104490 msgr2=0x7f18ec19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec104490 0x7f18ec19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 -- 172.21.15.94:0/2378004357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 msgr2=0x7f18ec19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 0x7f18ec19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 -- 172.21.15.94:0/2378004357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec19fb60 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f0c6c700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 0x7f18ec19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f1c6e700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007d70 tx=0x7f18e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4016020 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e400ad40 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40115e0 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.850+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec19fe50 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.851+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a7820 con 0x7f18ec109710 2024-04-25T13:41:03.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.851+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec051190 con 0x7f18ec109710 2024-04-25T13:41:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.853+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f18e4019020 con 0x7f18ec109710 2024-04-25T13:41:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.853+0000 7f18e27fc700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 0x7f18d80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.854+0000 7f18f146d700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 0x7f18d80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.854+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f18e40defe0 con 0x7f18ec109710 2024-04-25T13:41:03.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.854+0000 7f18f146d700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 0x7f18d80b21a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003800 tx=0x7f18dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:03.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:03.857+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18e4020020 con 0x7f18ec109710 2024-04-25T13:41:04.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.053+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18ec19c0a0 con 0x7f18d80afc90 2024-04-25T13:41:04.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.061+0000 7f18e27fc700 1 -- 172.21.15.94:0/2378004357 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258755 (secure 0 0 0) 0x7f18ec19c0a0 con 0x7f18d80afc90 2024-04-25T13:41:04.064 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 msgr2=0x7f18d80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 0x7f18d80b21a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003800 tx=0x7f18dc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 msgr2=0x7f18ec19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f18e4007d70 tx=0x7f18e400a340 comp rx=0 tx=0).stop 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 shutdown_connections 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18d80afc90 0x7f18d80b21a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18ec109710 0x7f18ec19f570 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18ec104490 0x7f18ec19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 --2- 172.21.15.94:0/2378004357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18ec108dc0 0x7f18ec19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 >> 172.21.15.94:0/2378004357 conn(0x7f18ec0ffda0 msgr2=0x7f18ec1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 shutdown_connections 2024-04-25T13:41:04.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:04.064+0000 7f18f36d1700 1 -- 172.21.15.94:0/2378004357 wait complete. 2024-04-25T13:41:04.067 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:41:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:04 smithi137 ceph-mon[32568]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:04 smithi137 ceph-mon[32568]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:04 smithi094 ceph-mon[29195]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:04.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:04 smithi094 ceph-mon[29195]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:04 smithi138 ceph-mon[32977]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:04 smithi138 ceph-mon[32977]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:04.753 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-04-25T13:41:02.276672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656295,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":793,"num_read_kb":496,"num_write":360,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403408,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124538416,"statfs":{"total":1151940427776,"available":1151527337984,"internally_reserved":0,"allocated":12890112,"data_stored":7028073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":400012601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003811"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266364+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:40:41.266364+0000","last_peered":"2024-04-25T13:40:41.266364+0000","last_clean":"2024-04-25T13:40:41.266364+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:40:41.266364+0000","last_undegraded":"2024-04-25T13:40:41.266364+0000","last_fullsized":"2024-04-25T13:40:41.266364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264015+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:40:41.264015+0000","last_peered":"2024-04-25T13:40:41.264015+0000","last_clean":"2024-04-25T13:40:41.264015+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:40:41.264015+0000","last_undegraded":"2024-04-25T13:40:41.264015+0000","last_fullsized":"2024-04-25T13:40:41.264015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263998+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:40:41.263998+0000","last_peered":"2024-04-25T13:40:41.263998+0000","last_clean":"2024-04-25T13:40:41.263998+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:40:41.263998+0000","last_undegraded":"2024-04-25T13:40:41.263998+0000","last_fullsized":"2024-04-25T13:40:41.263998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262062+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:40:41.262062+0000","last_peered":"2024-04-25T13:40:41.262062+0000","last_clean":"2024-04-25T13:40:41.262062+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:40:41.262062+0000","last_undegraded":"2024-04-25T13:40:41.262062+0000","last_fullsized":"2024-04-25T13:40:41.262062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267592+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:40:41.267592+0000","last_peered":"2024-04-25T13:40:41.267592+0000","last_clean":"2024-04-25T13:40:41.267592+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:40:41.267592+0000","last_undegraded":"2024-04-25T13:40:41.267592+0000","last_fullsized":"2024-04-25T13:40:41.267592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.265086+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:40:41.265086+0000","last_peered":"2024-04-25T13:40:41.265086+0000","last_clean":"2024-04-25T13:40:41.265086+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:40:41.265086+0000","last_undegraded":"2024-04-25T13:40:41.265086+0000","last_fullsized":"2024-04-25T13:40:41.265086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271762+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:40:41.271762+0000","last_peered":"2024-04-25T13:40:41.271762+0000","last_clean":"2024-04-25T13:40:41.271762+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:40:41.271762+0000","last_undegraded":"2024-04-25T13:40:41.271762+0000","last_fullsized":"2024-04-25T13:40:41.271762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266635+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:40:41.266635+0000","last_peered":"2024-04-25T13:40:41.266635+0000","last_clean":"2024-04-25T13:40:41.266635+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:40:41.266635+0000","last_undegraded":"2024-04-25T13:40:41.266635+0000","last_fullsized":"2024-04-25T13:40:41.266635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273152+0000","last_change":"2024-04-25T13:40:41.273152+0000","last_active":"2024-04-25T13:40:41.273152+0000","last_peered":"2024-04-25T13:40:41.273152+0000","last_clean":"2024-04-25T13:40:41.273152+0000","last_became_active":"2024-04-25T13:40:41.272760+0000","last_became_peered":"2024-04-25T13:40:41.272760+0000","last_unstale":"2024-04-25T13:40:41.273152+0000","last_undegraded":"2024-04-25T13:40:41.273152+0000","last_fullsized":"2024-04-25T13:40:41.273152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:35.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.274781+0000","last_change":"2024-04-25T13:40:41.274781+0000","last_active":"2024-04-25T13:40:41.274781+0000","last_peered":"2024-04-25T13:40:41.274781+0000","last_clean":"2024-04-25T13:40:41.274781+0000","last_became_active":"2024-04-25T13:40:41.274043+0000","last_became_peered":"2024-04-25T13:40:41.274043+0000","last_unstale":"2024-04-25T13:40:41.274781+0000","last_undegraded":"2024-04-25T13:40:41.274781+0000","last_fullsized":"2024-04-25T13:40:41.274781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:01.989580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273905+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:40:41.273905+0000","last_peered":"2024-04-25T13:40:41.273905+0000","last_clean":"2024-04-25T13:40:41.273905+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:40:41.273905+0000","last_undegraded":"2024-04-25T13:40:41.273905+0000","last_fullsized":"2024-04-25T13:40:41.273905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262232+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:40:41.262232+0000","last_peered":"2024-04-25T13:40:41.262232+0000","last_clean":"2024-04-25T13:40:41.262232+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:40:41.262232+0000","last_undegraded":"2024-04-25T13:40:41.262232+0000","last_fullsized":"2024-04-25T13:40:41.262232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260761+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:40:41.260761+0000","last_peered":"2024-04-25T13:40:41.260761+0000","last_clean":"2024-04-25T13:40:41.260761+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:40:41.260761+0000","last_undegraded":"2024-04-25T13:40:41.260761+0000","last_fullsized":"2024-04-25T13:40:41.260761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263572+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:40:41.263572+0000","last_peered":"2024-04-25T13:40:41.263572+0000","last_clean":"2024-04-25T13:40:41.263572+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:40:41.263572+0000","last_undegraded":"2024-04-25T13:40:41.263572+0000","last_fullsized":"2024-04-25T13:40:41.263572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263294+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:40:41.263294+0000","last_peered":"2024-04-25T13:40:41.263294+0000","last_clean":"2024-04-25T13:40:41.263294+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:40:41.263294+0000","last_undegraded":"2024-04-25T13:40:41.263294+0000","last_fullsized":"2024-04-25T13:40:41.263294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266268+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:40:41.266268+0000","last_peered":"2024-04-25T13:40:41.266268+0000","last_clean":"2024-04-25T13:40:41.266268+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:40:41.266268+0000","last_undegraded":"2024-04-25T13:40:41.266268+0000","last_fullsized":"2024-04-25T13:40:41.266268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266361+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:40:41.266361+0000","last_peered":"2024-04-25T13:40:41.266361+0000","last_clean":"2024-04-25T13:40:41.266361+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:40:41.266361+0000","last_undegraded":"2024-04-25T13:40:41.266361+0000","last_fullsized":"2024-04-25T13:40:41.266361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270855+0000","last_change":"2024-04-25T13:39:45.259538+0000","last_active":"2024-04-25T13:40:41.270855+0000","last_peered":"2024-04-25T13:40:41.270855+0000","last_clean":"2024-04-25T13:40:41.270855+0000","last_became_active":"2024-04-25T13:39:45.259097+0000","last_became_peered":"2024-04-25T13:39:45.259097+0000","last_unstale":"2024-04-25T13:40:41.270855+0000","last_undegraded":"2024-04-25T13:40:41.270855+0000","last_fullsized":"2024-04-25T13:40:41.270855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:10.901989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268904+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:40:41.268904+0000","last_peered":"2024-04-25T13:40:41.268904+0000","last_clean":"2024-04-25T13:40:41.268904+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:40:41.268904+0000","last_undegraded":"2024-04-25T13:40:41.268904+0000","last_fullsized":"2024-04-25T13:40:41.268904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":101,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.745283+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:40:57.745283+0000","last_peered":"2024-04-25T13:40:57.745283+0000","last_clean":"2024-04-25T13:40:57.745283+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:40:57.745283+0000","last_undegraded":"2024-04-25T13:40:57.745283+0000","last_fullsized":"2024-04-25T13:40:57.745283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266676+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:40:41.266676+0000","last_peered":"2024-04-25T13:40:41.266676+0000","last_clean":"2024-04-25T13:40:41.266676+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:40:41.266676+0000","last_undegraded":"2024-04-25T13:40:41.266676+0000","last_fullsized":"2024-04-25T13:40:41.266676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268139+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:40:41.268139+0000","last_peered":"2024-04-25T13:40:41.268139+0000","last_clean":"2024-04-25T13:40:41.268139+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:40:41.268139+0000","last_undegraded":"2024-04-25T13:40:41.268139+0000","last_fullsized":"2024-04-25T13:40:41.268139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264016+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:40:41.264016+0000","last_peered":"2024-04-25T13:40:41.264016+0000","last_clean":"2024-04-25T13:40:41.264016+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:40:41.264016+0000","last_undegraded":"2024-04-25T13:40:41.264016+0000","last_fullsized":"2024-04-25T13:40:41.264016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267992+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:40:41.267992+0000","last_peered":"2024-04-25T13:40:41.267992+0000","last_clean":"2024-04-25T13:40:41.267992+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:40:41.267992+0000","last_undegraded":"2024-04-25T13:40:41.267992+0000","last_fullsized":"2024-04-25T13:40:41.267992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266256+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:40:41.266256+0000","last_peered":"2024-04-25T13:40:41.266256+0000","last_clean":"2024-04-25T13:40:41.266256+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:40:41.266256+0000","last_undegraded":"2024-04-25T13:40:41.266256+0000","last_fullsized":"2024-04-25T13:40:41.266256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268053+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:40:41.268053+0000","last_peered":"2024-04-25T13:40:41.268053+0000","last_clean":"2024-04-25T13:40:41.268053+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:40:41.268053+0000","last_undegraded":"2024-04-25T13:40:41.268053+0000","last_fullsized":"2024-04-25T13:40:41.268053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264026+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:40:41.264026+0000","last_peered":"2024-04-25T13:40:41.264026+0000","last_clean":"2024-04-25T13:40:41.264026+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:40:41.264026+0000","last_undegraded":"2024-04-25T13:40:41.264026+0000","last_fullsized":"2024-04-25T13:40:41.264026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'6","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:56.745332+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:40:56.745332+0000","last_peered":"2024-04-25T13:40:56.745332+0000","last_clean":"2024-04-25T13:40:56.745332+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:40:56.745332+0000","last_undegraded":"2024-04-25T13:40:56.745332+0000","last_fullsized":"2024-04-25T13:40:56.745332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272700+0000","last_change":"2024-04-25T13:40:41.272700+0000","last_active":"2024-04-25T13:40:41.272700+0000","last_peered":"2024-04-25T13:40:41.272700+0000","last_clean":"2024-04-25T13:40:41.272700+0000","last_became_active":"2024-04-25T13:40:41.272080+0000","last_became_peered":"2024-04-25T13:40:41.272080+0000","last_unstale":"2024-04-25T13:40:41.272700+0000","last_undegraded":"2024-04-25T13:40:41.272700+0000","last_fullsized":"2024-04-25T13:40:41.272700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.149458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.280199+0000","last_change":"2024-04-25T13:40:41.280199+0000","last_active":"2024-04-25T13:40:41.280199+0000","last_peered":"2024-04-25T13:40:41.280199+0000","last_clean":"2024-04-25T13:40:41.280199+0000","last_became_active":"2024-04-25T13:40:41.268773+0000","last_became_peered":"2024-04-25T13:40:41.268773+0000","last_unstale":"2024-04-25T13:40:41.280199+0000","last_undegraded":"2024-04-25T13:40:41.280199+0000","last_fullsized":"2024-04-25T13:40:41.280199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:47.024600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263517+0000","last_change":"2024-04-25T13:39:45.268193+0000","last_active":"2024-04-25T13:40:41.263517+0000","last_peered":"2024-04-25T13:40:41.263517+0000","last_clean":"2024-04-25T13:40:41.263517+0000","last_became_active":"2024-04-25T13:39:45.267842+0000","last_became_peered":"2024-04-25T13:39:45.267842+0000","last_unstale":"2024-04-25T13:40:41.263517+0000","last_undegraded":"2024-04-25T13:40:41.263517+0000","last_fullsized":"2024-04-25T13:40:41.263517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:56.308127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261116+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:40:41.261116+0000","last_peered":"2024-04-25T13:40:41.261116+0000","last_clean":"2024-04-25T13:40:41.261116+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:40:41.261116+0000","last_undegraded":"2024-04-25T13:40:41.261116+0000","last_fullsized":"2024-04-25T13:40:41.261116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266910+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:40:41.266910+0000","last_peered":"2024-04-25T13:40:41.266910+0000","last_clean":"2024-04-25T13:40:41.266910+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:40:41.266910+0000","last_undegraded":"2024-04-25T13:40:41.266910+0000","last_fullsized":"2024-04-25T13:40:41.266910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262954+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:40:41.262954+0000","last_peered":"2024-04-25T13:40:41.262954+0000","last_clean":"2024-04-25T13:40:41.262954+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:40:41.262954+0000","last_undegraded":"2024-04-25T13:40:41.262954+0000","last_fullsized":"2024-04-25T13:40:41.262954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268949+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:40:41.268949+0000","last_peered":"2024-04-25T13:40:41.268949+0000","last_clean":"2024-04-25T13:40:41.268949+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:40:41.268949+0000","last_undegraded":"2024-04-25T13:40:41.268949+0000","last_fullsized":"2024-04-25T13:40:41.268949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":161,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:59.749171+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:40:59.749171+0000","last_peered":"2024-04-25T13:40:59.749171+0000","last_clean":"2024-04-25T13:40:59.749171+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:40:59.749171+0000","last_undegraded":"2024-04-25T13:40:59.749171+0000","last_fullsized":"2024-04-25T13:40:59.749171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267305+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:40:41.267305+0000","last_peered":"2024-04-25T13:40:41.267305+0000","last_clean":"2024-04-25T13:40:41.267305+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:40:41.267305+0000","last_undegraded":"2024-04-25T13:40:41.267305+0000","last_fullsized":"2024-04-25T13:40:41.267305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273894+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:40:41.273894+0000","last_peered":"2024-04-25T13:40:41.273894+0000","last_clean":"2024-04-25T13:40:41.273894+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:40:41.273894+0000","last_undegraded":"2024-04-25T13:40:41.273894+0000","last_fullsized":"2024-04-25T13:40:41.273894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264060+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:40:41.264060+0000","last_peered":"2024-04-25T13:40:41.264060+0000","last_clean":"2024-04-25T13:40:41.264060+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:40:41.264060+0000","last_undegraded":"2024-04-25T13:40:41.264060+0000","last_fullsized":"2024-04-25T13:40:41.264060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264260+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:40:41.264260+0000","last_peered":"2024-04-25T13:40:41.264260+0000","last_clean":"2024-04-25T13:40:41.264260+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:40:41.264260+0000","last_undegraded":"2024-04-25T13:40:41.264260+0000","last_fullsized":"2024-04-25T13:40:41.264260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267041+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:40:41.267041+0000","last_peered":"2024-04-25T13:40:41.267041+0000","last_clean":"2024-04-25T13:40:41.267041+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:40:41.267041+0000","last_undegraded":"2024-04-25T13:40:41.267041+0000","last_fullsized":"2024-04-25T13:40:41.267041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'30","reported_seq":192,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:59.754804+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:40:59.754804+0000","last_peered":"2024-04-25T13:40:59.754804+0000","last_clean":"2024-04-25T13:40:59.754804+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:40:59.754804+0000","last_undegraded":"2024-04-25T13:40:59.754804+0000","last_fullsized":"2024-04-25T13:40:59.754804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":129,"num_write":32,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267623+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:40:41.267623+0000","last_peered":"2024-04-25T13:40:41.267623+0000","last_clean":"2024-04-25T13:40:41.267623+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:40:41.267623+0000","last_undegraded":"2024-04-25T13:40:41.267623+0000","last_fullsized":"2024-04-25T13:40:41.267623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271079+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:40:41.271079+0000","last_peered":"2024-04-25T13:40:41.271079+0000","last_clean":"2024-04-25T13:40:41.271079+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:40:41.271079+0000","last_undegraded":"2024-04-25T13:40:41.271079+0000","last_fullsized":"2024-04-25T13:40:41.271079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260892+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:40:41.260892+0000","last_peered":"2024-04-25T13:40:41.260892+0000","last_clean":"2024-04-25T13:40:41.260892+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:40:41.260892+0000","last_undegraded":"2024-04-25T13:40:41.260892+0000","last_fullsized":"2024-04-25T13:40:41.260892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263559+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:40:41.263559+0000","last_peered":"2024-04-25T13:40:41.263559+0000","last_clean":"2024-04-25T13:40:41.263559+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:40:41.263559+0000","last_undegraded":"2024-04-25T13:40:41.263559+0000","last_fullsized":"2024-04-25T13:40:41.263559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272656+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:40:41.272656+0000","last_peered":"2024-04-25T13:40:41.272656+0000","last_clean":"2024-04-25T13:40:41.272656+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:40:41.272656+0000","last_undegraded":"2024-04-25T13:40:41.272656+0000","last_fullsized":"2024-04-25T13:40:41.272656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.752462+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:40:57.752462+0000","last_peered":"2024-04-25T13:40:57.752462+0000","last_clean":"2024-04-25T13:40:57.752462+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:40:57.752462+0000","last_undegraded":"2024-04-25T13:40:57.752462+0000","last_fullsized":"2024-04-25T13:40:57.752462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271554+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:40:41.271554+0000","last_peered":"2024-04-25T13:40:41.271554+0000","last_clean":"2024-04-25T13:40:41.271554+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:40:41.271554+0000","last_undegraded":"2024-04-25T13:40:41.271554+0000","last_fullsized":"2024-04-25T13:40:41.271554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268389+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:40:41.268389+0000","last_peered":"2024-04-25T13:40:41.268389+0000","last_clean":"2024-04-25T13:40:41.268389+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:40:41.268389+0000","last_undegraded":"2024-04-25T13:40:41.268389+0000","last_fullsized":"2024-04-25T13:40:41.268389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263122+0000","last_change":"2024-04-25T13:39:45.273219+0000","last_active":"2024-04-25T13:40:41.263122+0000","last_peered":"2024-04-25T13:40:41.263122+0000","last_clean":"2024-04-25T13:40:41.263122+0000","last_became_active":"2024-04-25T13:39:45.272891+0000","last_became_peered":"2024-04-25T13:39:45.272891+0000","last_unstale":"2024-04-25T13:40:41.263122+0000","last_undegraded":"2024-04-25T13:40:41.263122+0000","last_fullsized":"2024-04-25T13:40:41.263122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:40.531758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267700+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:40:41.267700+0000","last_peered":"2024-04-25T13:40:41.267700+0000","last_clean":"2024-04-25T13:40:41.267700+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:40:41.267700+0000","last_undegraded":"2024-04-25T13:40:41.267700+0000","last_fullsized":"2024-04-25T13:40:41.267700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271904+0000","last_change":"2024-04-25T13:40:41.271904+0000","last_active":"2024-04-25T13:40:41.271904+0000","last_peered":"2024-04-25T13:40:41.271904+0000","last_clean":"2024-04-25T13:40:41.271904+0000","last_became_active":"2024-04-25T13:40:41.271457+0000","last_became_peered":"2024-04-25T13:40:41.271457+0000","last_unstale":"2024-04-25T13:40:41.271904+0000","last_undegraded":"2024-04-25T13:40:41.271904+0000","last_fullsized":"2024-04-25T13:40:41.271904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:04.766001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270464+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:40:41.270464+0000","last_peered":"2024-04-25T13:40:41.270464+0000","last_clean":"2024-04-25T13:40:41.270464+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:40:41.270464+0000","last_undegraded":"2024-04-25T13:40:41.270464+0000","last_fullsized":"2024-04-25T13:40:41.270464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263373+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:40:41.263373+0000","last_peered":"2024-04-25T13:40:41.263373+0000","last_clean":"2024-04-25T13:40:41.263373+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:40:41.263373+0000","last_undegraded":"2024-04-25T13:40:41.263373+0000","last_fullsized":"2024-04-25T13:40:41.263373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"79'5","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.746133+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:40:57.746133+0000","last_peered":"2024-04-25T13:40:57.746133+0000","last_clean":"2024-04-25T13:40:57.746133+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:40:57.746133+0000","last_undegraded":"2024-04-25T13:40:57.746133+0000","last_fullsized":"2024-04-25T13:40:57.746133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263397+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:40:41.263397+0000","last_peered":"2024-04-25T13:40:41.263397+0000","last_clean":"2024-04-25T13:40:41.263397+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:40:41.263397+0000","last_undegraded":"2024-04-25T13:40:41.263397+0000","last_fullsized":"2024-04-25T13:40:41.263397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263547+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:40:41.263547+0000","last_peered":"2024-04-25T13:40:41.263547+0000","last_clean":"2024-04-25T13:40:41.263547+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:40:41.263547+0000","last_undegraded":"2024-04-25T13:40:41.263547+0000","last_fullsized":"2024-04-25T13:40:41.263547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.751817+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:40:57.751817+0000","last_peered":"2024-04-25T13:40:57.751817+0000","last_clean":"2024-04-25T13:40:57.751817+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:40:57.751817+0000","last_undegraded":"2024-04-25T13:40:57.751817+0000","last_fullsized":"2024-04-25T13:40:57.751817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266655+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:40:41.266655+0000","last_peered":"2024-04-25T13:40:41.266655+0000","last_clean":"2024-04-25T13:40:41.266655+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:40:41.266655+0000","last_undegraded":"2024-04-25T13:40:41.266655+0000","last_fullsized":"2024-04-25T13:40:41.266655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261992+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:40:41.261992+0000","last_peered":"2024-04-25T13:40:41.261992+0000","last_clean":"2024-04-25T13:40:41.261992+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:40:41.261992+0000","last_undegraded":"2024-04-25T13:40:41.261992+0000","last_fullsized":"2024-04-25T13:40:41.261992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.272875+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:40:41.272875+0000","last_peered":"2024-04-25T13:40:41.272875+0000","last_clean":"2024-04-25T13:40:41.272875+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:40:41.272875+0000","last_undegraded":"2024-04-25T13:40:41.272875+0000","last_fullsized":"2024-04-25T13:40:41.272875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260431+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:40:41.260431+0000","last_peered":"2024-04-25T13:40:41.260431+0000","last_clean":"2024-04-25T13:40:41.260431+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:40:41.260431+0000","last_undegraded":"2024-04-25T13:40:41.260431+0000","last_fullsized":"2024-04-25T13:40:41.260431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273111+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:40:41.273111+0000","last_peered":"2024-04-25T13:40:41.273111+0000","last_clean":"2024-04-25T13:40:41.273111+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:40:41.273111+0000","last_undegraded":"2024-04-25T13:40:41.273111+0000","last_fullsized":"2024-04-25T13:40:41.273111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.280650+0000","last_change":"2024-04-25T13:40:41.280650+0000","last_active":"2024-04-25T13:40:41.280650+0000","last_peered":"2024-04-25T13:40:41.280650+0000","last_clean":"2024-04-25T13:40:41.280650+0000","last_became_active":"2024-04-25T13:40:41.271193+0000","last_became_peered":"2024-04-25T13:40:41.271193+0000","last_unstale":"2024-04-25T13:40:41.280650+0000","last_undegraded":"2024-04-25T13:40:41.280650+0000","last_fullsized":"2024-04-25T13:40:41.280650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.137595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261394+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:40:41.261394+0000","last_peered":"2024-04-25T13:40:41.261394+0000","last_clean":"2024-04-25T13:40:41.261394+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:40:41.261394+0000","last_undegraded":"2024-04-25T13:40:41.261394+0000","last_fullsized":"2024-04-25T13:40:41.261394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270998+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:40:41.270998+0000","last_peered":"2024-04-25T13:40:41.270998+0000","last_clean":"2024-04-25T13:40:41.270998+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:40:41.270998+0000","last_undegraded":"2024-04-25T13:40:41.270998+0000","last_fullsized":"2024-04-25T13:40:41.270998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268329+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:40:41.268329+0000","last_peered":"2024-04-25T13:40:41.268329+0000","last_clean":"2024-04-25T13:40:41.268329+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:40:41.268329+0000","last_undegraded":"2024-04-25T13:40:41.268329+0000","last_fullsized":"2024-04-25T13:40:41.268329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262137+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:40:41.262137+0000","last_peered":"2024-04-25T13:40:41.262137+0000","last_clean":"2024-04-25T13:40:41.262137+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:40:41.262137+0000","last_undegraded":"2024-04-25T13:40:41.262137+0000","last_fullsized":"2024-04-25T13:40:41.262137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263987+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:40:41.263987+0000","last_peered":"2024-04-25T13:40:41.263987+0000","last_clean":"2024-04-25T13:40:41.263987+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:40:41.263987+0000","last_undegraded":"2024-04-25T13:40:41.263987+0000","last_fullsized":"2024-04-25T13:40:41.263987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266827+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:40:41.266827+0000","last_peered":"2024-04-25T13:40:41.266827+0000","last_clean":"2024-04-25T13:40:41.266827+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:40:41.266827+0000","last_undegraded":"2024-04-25T13:40:41.266827+0000","last_fullsized":"2024-04-25T13:40:41.266827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267466+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:40:41.267466+0000","last_peered":"2024-04-25T13:40:41.267466+0000","last_clean":"2024-04-25T13:40:41.267466+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:40:41.267466+0000","last_undegraded":"2024-04-25T13:40:41.267466+0000","last_fullsized":"2024-04-25T13:40:41.267466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267078+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:40:41.267078+0000","last_peered":"2024-04-25T13:40:41.267078+0000","last_clean":"2024-04-25T13:40:41.267078+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:40:41.267078+0000","last_undegraded":"2024-04-25T13:40:41.267078+0000","last_fullsized":"2024-04-25T13:40:41.267078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264631+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:40:41.264631+0000","last_peered":"2024-04-25T13:40:41.264631+0000","last_clean":"2024-04-25T13:40:41.264631+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:40:41.264631+0000","last_undegraded":"2024-04-25T13:40:41.264631+0000","last_fullsized":"2024-04-25T13:40:41.264631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.284948+0000","last_change":"2024-04-25T13:40:41.284948+0000","last_active":"2024-04-25T13:40:41.284948+0000","last_peered":"2024-04-25T13:40:41.284948+0000","last_clean":"2024-04-25T13:40:41.284948+0000","last_became_active":"2024-04-25T13:40:41.272015+0000","last_became_peered":"2024-04-25T13:40:41.272015+0000","last_unstale":"2024-04-25T13:40:41.284948+0000","last_undegraded":"2024-04-25T13:40:41.284948+0000","last_fullsized":"2024-04-25T13:40:41.284948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:21.620303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266671+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:40:41.266671+0000","last_peered":"2024-04-25T13:40:41.266671+0000","last_clean":"2024-04-25T13:40:41.266671+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:40:41.266671+0000","last_undegraded":"2024-04-25T13:40:41.266671+0000","last_fullsized":"2024-04-25T13:40:41.266671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.282512+0000","last_change":"2024-04-25T13:40:41.282512+0000","last_active":"2024-04-25T13:40:41.282512+0000","last_peered":"2024-04-25T13:40:41.282512+0000","last_clean":"2024-04-25T13:40:41.282512+0000","last_became_active":"2024-04-25T13:40:41.275727+0000","last_became_peered":"2024-04-25T13:40:41.275727+0000","last_unstale":"2024-04-25T13:40:41.282512+0000","last_undegraded":"2024-04-25T13:40:41.282512+0000","last_fullsized":"2024-04-25T13:40:41.282512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.611067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271528+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:40:41.271528+0000","last_peered":"2024-04-25T13:40:41.271528+0000","last_clean":"2024-04-25T13:40:41.271528+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:40:41.271528+0000","last_undegraded":"2024-04-25T13:40:41.271528+0000","last_fullsized":"2024-04-25T13:40:41.271528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":94,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:57.746931+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:40:57.746931+0000","last_peered":"2024-04-25T13:40:57.746931+0000","last_clean":"2024-04-25T13:40:57.746931+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:40:57.746931+0000","last_undegraded":"2024-04-25T13:40:57.746931+0000","last_fullsized":"2024-04-25T13:40:57.746931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.273977+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:40:41.273977+0000","last_peered":"2024-04-25T13:40:41.273977+0000","last_clean":"2024-04-25T13:40:41.273977+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:40:41.273977+0000","last_undegraded":"2024-04-25T13:40:41.273977+0000","last_fullsized":"2024-04-25T13:40:41.273977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263982+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:40:41.263982+0000","last_peered":"2024-04-25T13:40:41.263982+0000","last_clean":"2024-04-25T13:40:41.263982+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:40:41.263982+0000","last_undegraded":"2024-04-25T13:40:41.263982+0000","last_fullsized":"2024-04-25T13:40:41.263982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271918+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:40:41.271918+0000","last_peered":"2024-04-25T13:40:41.271918+0000","last_clean":"2024-04-25T13:40:41.271918+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:40:41.271918+0000","last_undegraded":"2024-04-25T13:40:41.271918+0000","last_fullsized":"2024-04-25T13:40:41.271918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261645+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:40:41.261645+0000","last_peered":"2024-04-25T13:40:41.261645+0000","last_clean":"2024-04-25T13:40:41.261645+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:40:41.261645+0000","last_undegraded":"2024-04-25T13:40:41.261645+0000","last_fullsized":"2024-04-25T13:40:41.261645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271682+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:40:41.271682+0000","last_peered":"2024-04-25T13:40:41.271682+0000","last_clean":"2024-04-25T13:40:41.271682+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:40:41.271682+0000","last_undegraded":"2024-04-25T13:40:41.271682+0000","last_fullsized":"2024-04-25T13:40:41.271682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.264788+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:40:41.264788+0000","last_peered":"2024-04-25T13:40:41.264788+0000","last_clean":"2024-04-25T13:40:41.264788+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:40:41.264788+0000","last_undegraded":"2024-04-25T13:40:41.264788+0000","last_fullsized":"2024-04-25T13:40:41.264788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270467+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:40:41.270467+0000","last_peered":"2024-04-25T13:40:41.270467+0000","last_clean":"2024-04-25T13:40:41.270467+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:40:41.270467+0000","last_undegraded":"2024-04-25T13:40:41.270467+0000","last_fullsized":"2024-04-25T13:40:41.270467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268611+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:40:41.268611+0000","last_peered":"2024-04-25T13:40:41.268611+0000","last_clean":"2024-04-25T13:40:41.268611+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:40:41.268611+0000","last_undegraded":"2024-04-25T13:40:41.268611+0000","last_fullsized":"2024-04-25T13:40:41.268611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.274874+0000","last_change":"2024-04-25T13:40:41.274874+0000","last_active":"2024-04-25T13:40:41.274874+0000","last_peered":"2024-04-25T13:40:41.274874+0000","last_clean":"2024-04-25T13:40:41.274874+0000","last_became_active":"2024-04-25T13:40:41.274489+0000","last_became_peered":"2024-04-25T13:40:41.274489+0000","last_unstale":"2024-04-25T13:40:41.274874+0000","last_undegraded":"2024-04-25T13:40:41.274874+0000","last_fullsized":"2024-04-25T13:40:41.274874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:34.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261432+0000","last_change":"2024-04-25T13:39:45.261199+0000","last_active":"2024-04-25T13:40:41.261432+0000","last_peered":"2024-04-25T13:40:41.261432+0000","last_clean":"2024-04-25T13:40:41.261432+0000","last_became_active":"2024-04-25T13:39:45.260867+0000","last_became_peered":"2024-04-25T13:39:45.260867+0000","last_unstale":"2024-04-25T13:40:41.261432+0000","last_undegraded":"2024-04-25T13:40:41.261432+0000","last_fullsized":"2024-04-25T13:40:41.261432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:47.043364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266833+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:40:41.266833+0000","last_peered":"2024-04-25T13:40:41.266833+0000","last_clean":"2024-04-25T13:40:41.266833+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:40:41.266833+0000","last_undegraded":"2024-04-25T13:40:41.266833+0000","last_fullsized":"2024-04-25T13:40:41.266833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.268409+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:40:41.268409+0000","last_peered":"2024-04-25T13:40:41.268409+0000","last_clean":"2024-04-25T13:40:41.268409+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:40:41.268409+0000","last_undegraded":"2024-04-25T13:40:41.268409+0000","last_fullsized":"2024-04-25T13:40:41.268409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.266818+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:40:41.266818+0000","last_peered":"2024-04-25T13:40:41.266818+0000","last_clean":"2024-04-25T13:40:41.266818+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:40:41.266818+0000","last_undegraded":"2024-04-25T13:40:41.266818+0000","last_fullsized":"2024-04-25T13:40:41.266818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.263023+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:40:41.263023+0000","last_peered":"2024-04-25T13:40:41.263023+0000","last_clean":"2024-04-25T13:40:41.263023+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:40:41.263023+0000","last_undegraded":"2024-04-25T13:40:41.263023+0000","last_fullsized":"2024-04-25T13:40:41.263023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.267100+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:40:41.267100+0000","last_peered":"2024-04-25T13:40:41.267100+0000","last_clean":"2024-04-25T13:40:41.267100+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:40:41.267100+0000","last_undegraded":"2024-04-25T13:40:41.267100+0000","last_fullsized":"2024-04-25T13:40:41.267100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.276377+0000","last_change":"2024-04-25T13:40:41.276377+0000","last_active":"2024-04-25T13:40:41.276377+0000","last_peered":"2024-04-25T13:40:41.276377+0000","last_clean":"2024-04-25T13:40:41.276377+0000","last_became_active":"2024-04-25T13:40:41.275949+0000","last_became_peered":"2024-04-25T13:40:41.275949+0000","last_unstale":"2024-04-25T13:40:41.276377+0000","last_undegraded":"2024-04-25T13:40:41.276377+0000","last_fullsized":"2024-04-25T13:40:41.276377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:49.882272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.260501+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:40:41.260501+0000","last_peered":"2024-04-25T13:40:41.260501+0000","last_clean":"2024-04-25T13:40:41.260501+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:40:41.260501+0000","last_undegraded":"2024-04-25T13:40:41.260501+0000","last_fullsized":"2024-04-25T13:40:41.260501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.265864+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:40:41.265864+0000","last_peered":"2024-04-25T13:40:41.265864+0000","last_clean":"2024-04-25T13:40:41.265864+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:40:41.265864+0000","last_undegraded":"2024-04-25T13:40:41.265864+0000","last_fullsized":"2024-04-25T13:40:41.265864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65693,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":717,"num_read_kb":432,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":422903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1589248,"data_stored":1072700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972391,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":913408,"data_stored":422721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":422987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1507328,"data_stored":1015617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":422438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":482231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":423777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":482842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40392,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704760,"statfs":{"total":95995035648,"available":95953674240,"internally_reserved":0,"allocated":913408,"data_stored":422263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":422299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36828,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708324,"statfs":{"total":95995035648,"available":95957323776,"internally_reserved":0,"allocated":1519616,"data_stored":1015295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:41:04.757 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:41:04.757 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:41:04.757 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:41:04.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:41:04.996 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 -- 172.21.15.94:0/1379823624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2001085e0 msgr2=0x7fc20010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 --2- 172.21.15.94:0/1379823624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2001085e0 0x7fc20010ff40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00099c0 tx=0x7fc1f0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 -- 172.21.15.94:0/1379823624 shutdown_connections 2024-04-25T13:41:06.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 --2- 172.21.15.94:0/1379823624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2001085e0 0x7fc20010ff40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 --2- 172.21.15.94:0/1379823624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc200103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 --2- 172.21.15.94:0/1379823624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc200107c90 0x7fc2001080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 -- 172.21.15.94:0/1379823624 >> 172.21.15.94:0/1379823624 conn(0x7fc2000fec90 msgr2=0x7fc2001010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 -- 172.21.15.94:0/1379823624 shutdown_connections 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.544+0000 7fc205532700 1 -- 172.21.15.94:0/1379823624 wait complete. 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 Processor -- start 2024-04-25T13:41:06.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 -- start start 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc20019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 0x7fc200199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113290 con 0x7fc200107c90 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113110 con 0x7fc2001085e0 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.545+0000 7fc205532700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc200113410 con 0x7fc200103360 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37938/0 (socket says 172.21.15.94:37938) 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 -- 172.21.15.94:0/3641275708 learned_addr learned my addr 172.21.15.94:0/3641275708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1feffd700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc20019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1ff7fe700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 0x7fc200199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 -- 172.21.15.94:0/3641275708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 msgr2=0x7fc20019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc20019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 -- 172.21.15.94:0/3641275708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 msgr2=0x7fc200199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 0x7fc200199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 -- 172.21.15.94:0/3641275708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20019a0c0 con 0x7fc200107c90 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1feffd700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc20019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1ff7fe700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 0x7fc200199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1fe7fc700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d980 tx=0x7fc1f400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4017070 con 0x7fc200107c90 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4010460 con 0x7fc200107c90 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20019a3b0 con 0x7fc200107c90 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400de60 con 0x7fc200107c90 2024-04-25T13:41:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.546+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001a6680 con 0x7fc200107c90 2024-04-25T13:41:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.547+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc200051190 con 0x7fc200107c90 2024-04-25T13:41:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.549+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc1f4010610 con 0x7fc200107c90 2024-04-25T13:41:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.550+0000 7fc1e7fff700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 0x7fc1e80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.550+0000 7fc1feffd700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 0x7fc1e80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.550+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc1f40de980 con 0x7fc200107c90 2024-04-25T13:41:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.551+0000 7fc1feffd700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 0x7fc1e80b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc20019ab50 tx=0x7fc1f80073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.554+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1f40a73e0 con 0x7fc200107c90 2024-04-25T13:41:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:06 smithi137 ceph-mon[32568]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:06 smithi094 ceph-mon[29195]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:06 smithi138 ceph-mon[32977]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.798+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc20002e510 con 0x7fc200107c90 2024-04-25T13:41:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.799+0000 7fc1e7fff700 1 -- 172.21.15.94:0/3641275708 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc1f40a6ce0 con 0x7fc200107c90 2024-04-25T13:41:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:06.799 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:41:06.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 msgr2=0x7fc1e80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 0x7fc1e80b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc20019ab50 tx=0x7fc1f80073e0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 msgr2=0x7fc20019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d980 tx=0x7fc1f400dd40 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 shutdown_connections 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2001085e0 0x7fc200199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc200103360 0x7fc20019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1e80afc10 0x7fc1e80b2120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 --2- 172.21.15.94:0/3641275708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc200107c90 0x7fc20019fd30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 >> 172.21.15.94:0/3641275708 conn(0x7fc2000fec90 msgr2=0x7fc200107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 shutdown_connections 2024-04-25T13:41:06.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:06.801+0000 7fc205532700 1 -- 172.21.15.94:0/3641275708 wait complete. 2024-04-25T13:41:07.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:07 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3641275708' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:41:07.479 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:41:07.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:07.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 -- 172.21.15.94:0/724211384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 msgr2=0x7f249410be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 --2- 172.21.15.94:0/724211384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f249410be30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f248c009ca0 tx=0x7f248c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 -- 172.21.15.94:0/724211384 shutdown_connections 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 --2- 172.21.15.94:0/724211384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f249410be30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 --2- 172.21.15.94:0/724211384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494104d10 0x7f2494105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 --2- 172.21.15.94:0/724211384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494103ad0 0x7f24941047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.593+0000 7f249bd1f640 1 -- 172.21.15.94:0/724211384 >> 172.21.15.94:0/724211384 conn(0x7f24940fece0 msgr2=0x7f2494101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- 172.21.15.94:0/724211384 shutdown_connections 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- 172.21.15.94:0/724211384 wait complete. 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 Processor -- start 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- start start 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494103ad0 0x7f249419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494104d10 0x7f249419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f650 con 0x7f2494109a10 2024-04-25T13:41:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f4d0 con 0x7f2494104d10 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249bd1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f7d0 con 0x7f2494103ad0 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.594+0000 7f249a2bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37952/0 (socket says 172.21.15.94:37952) 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 -- 172.21.15.94:0/2208582298 learned_addr learned my addr 172.21.15.94:0/2208582298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f24992ba640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494104d10 0x7f249419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f2499abb640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494103ad0 0x7f249419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 -- 172.21.15.94:0/2208582298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494103ad0 msgr2=0x7f249419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494103ad0 0x7f249419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 -- 172.21.15.94:0/2208582298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494104d10 msgr2=0x7f249419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494104d10 0x7f249419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 -- 172.21.15.94:0/2208582298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24941a1c50 con 0x7f2494109a10 2024-04-25T13:41:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249a2bc640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f248c0097b0 tx=0x7f248c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c004240 con 0x7f2494109a10 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248c0317c0 con 0x7f2494109a10 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a1ee0 con 0x7f2494109a10 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c02fde0 con 0x7f2494109a10 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.595+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a2420 con 0x7f2494109a10 2024-04-25T13:41:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.596+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494108820 con 0x7f2494109a10 2024-04-25T13:41:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.597+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f248c0043e0 con 0x7f2494109a10 2024-04-25T13:41:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.598+0000 7f248affd640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 0x7f24800b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.598+0000 7f2499abb640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 0x7f24800b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.598+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f248c100940 con 0x7f2494109a10 2024-04-25T13:41:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.598+0000 7f2499abb640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 0x7f24800b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24900058c0 tx=0x7f2490005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.602+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f248c0c94b0 con 0x7f2494109a10 2024-04-25T13:41:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:07 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3641275708' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:41:07.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:07 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3641275708' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:41:07.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.784+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f249402dd20 con 0x7f2494109a10 2024-04-25T13:41:07.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.786+0000 7f248affd640 1 -- 172.21.15.94:0/2208582298 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7f248c0c8db0 con 0x7f2494109a10 2024-04-25T13:41:07.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:07.786 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 msgr2=0x7f24800b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 0x7f24800b2110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24900058c0 tx=0x7f2490005850 comp rx=0 tx=0).stop 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 msgr2=0x7f24941a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f248c0097b0 tx=0x7f248c003c20 comp rx=0 tx=0).stop 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 shutdown_connections 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24800afc00 0x7f24800b2110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2494109a10 0x7f24941a1690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2494104d10 0x7f249419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 --2- 172.21.15.94:0/2208582298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2494103ad0 0x7f249419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 >> 172.21.15.94:0/2208582298 conn(0x7f24940fece0 msgr2=0x7f24941059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 shutdown_connections 2024-04-25T13:41:07.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.788+0000 7f249bd1f640 1 -- 172.21.15.94:0/2208582298 wait complete. 2024-04-25T13:41:07.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:07.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 -- 172.21.15.94:0/361485340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881033b0 msgr2=0x7f4c881037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/361485340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881033b0 0x7f4c881037c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00aa10 tx=0x7f4c7c02b720 comp rx=0 tx=0).stop 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 -- 172.21.15.94:0/361485340 shutdown_connections 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/361485340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c881080b0 0x7f4c8810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/361485340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881033b0 0x7f4c881037c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/361485340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c88102170 0x7f4c88102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.911+0000 7f4c8d73f640 1 -- 172.21.15.94:0/361485340 >> 172.21.15.94:0/361485340 conn(0x7f4c880fd340 msgr2=0x7f4c880ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- 172.21.15.94:0/361485340 shutdown_connections 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- 172.21.15.94:0/361485340 wait complete. 2024-04-25T13:41:07.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 Processor -- start 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- start start 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c88102170 0x7f4c88198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c881033b0 0x7f4c88198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d400 con 0x7f4c881080b0 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d280 con 0x7f4c881033b0 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.912+0000 7f4c8d73f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8810d580 con 0x7f4c88102170 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37966/0 (socket says 172.21.15.94:37966) 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 -- 172.21.15.94:0/2193556892 learned_addr learned my addr 172.21.15.94:0/2193556892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c867fc640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c881033b0 0x7f4c88198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c86ffd640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c88102170 0x7f4c88198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 -- 172.21.15.94:0/2193556892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c88102170 msgr2=0x7f4c88198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c88102170 0x7f4c88198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 -- 172.21.15.94:0/2193556892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c881033b0 msgr2=0x7f4c88198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c881033b0 0x7f4c88198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 -- 172.21.15.94:0/2193556892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8819fb80 con 0x7f4c881080b0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c877fe640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000cad0 tx=0x7f4c8000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000ae20 con 0x7f4c881080b0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8819fe70 con 0x7f4c881080b0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c80005020 con 0x7f4c881080b0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f6b0 con 0x7f4c881080b0 2024-04-25T13:41:07.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.913+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881a02d0 con 0x7f4c881080b0 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.914+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c880692e0 con 0x7f4c881080b0 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.916+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4c800051d0 con 0x7f4c881080b0 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.916+0000 7f4c6ffff640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 0x7f4c700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.916+0000 7f4c86ffd640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 0x7f4c700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.917+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f4c800df270 con 0x7f4c881080b0 2024-04-25T13:41:07.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.917+0000 7f4c86ffd640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 0x7f4c700b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c88102d70 tx=0x7f4c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:07.920+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c80014030 con 0x7f4c881080b0 2024-04-25T13:41:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.096+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c8802db80 con 0x7f4c881080b0 2024-04-25T13:41:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.096+0000 7f4c6ffff640 1 -- 172.21.15.94:0/2193556892 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f4c800a7cd0 con 0x7f4c881080b0 2024-04-25T13:41:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 msgr2=0x7f4c700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 0x7f4c700b2090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c88102d70 tx=0x7f4c78005f90 comp rx=0 tx=0).stop 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 msgr2=0x7f4c8819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000cad0 tx=0x7f4c8000cf90 comp rx=0 tx=0).stop 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 shutdown_connections 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c700afb80 0x7f4c700b2090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c881080b0 0x7f4c8819f510 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c881033b0 0x7f4c88198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.098+0000 7f4c8d73f640 1 --2- 172.21.15.94:0/2193556892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c88102170 0x7f4c88198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.099+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 >> 172.21.15.94:0/2193556892 conn(0x7f4c880fd340 msgr2=0x7f4c88104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.099+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 shutdown_connections 2024-04-25T13:41:08.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.099+0000 7f4c8d73f640 1 -- 172.21.15.94:0/2193556892 wait complete. 2024-04-25T13:41:08.111 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 -- 172.21.15.94:0/1141920062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708102f80 msgr2=0x7f9708103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 --2- 172.21.15.94:0/1141920062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708102f80 0x7f9708103390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009900 tx=0x7f96f802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 -- 172.21.15.94:0/1141920062 shutdown_connections 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 --2- 172.21.15.94:0/1141920062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708104480 0x7f970810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 --2- 172.21.15.94:0/1141920062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708103960 0x7f9708103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 --2- 172.21.15.94:0/1141920062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708102f80 0x7f9708103390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 -- 172.21.15.94:0/1141920062 >> 172.21.15.94:0/1141920062 conn(0x7f97080fd380 msgr2=0x7f97080ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 -- 172.21.15.94:0/1141920062 shutdown_connections 2024-04-25T13:41:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.220+0000 7f970e2f2640 1 -- 172.21.15.94:0/1141920062 wait complete. 2024-04-25T13:41:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 Processor -- start 2024-04-25T13:41:08.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 -- start start 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708102f80 0x7f9708199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103960 0x7f9708199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f970810e520 con 0x7f9708104480 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f970810e3a0 con 0x7f9708103960 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970e2f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f970810e6a0 con 0x7f9708102f80 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970c88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970c88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37972/0 (socket says 172.21.15.94:37972) 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f970c88f640 1 -- 172.21.15.94:0/3319178926 learned_addr learned my addr 172.21.15.94:0/3319178926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f97077fe640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103960 0x7f9708199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.221+0000 7f9707fff640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708102f80 0x7f9708199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 -- 172.21.15.94:0/3319178926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708102f80 msgr2=0x7f9708199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708102f80 0x7f9708199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 -- 172.21.15.94:0/3319178926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103960 msgr2=0x7f9708199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103960 0x7f9708199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 -- 172.21.15.94:0/3319178926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97081a0c60 con 0x7f9708104480 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970c88f640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f970000ebb0 tx=0x7f9700007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700007ca0 con 0x7f9708104480 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9700007e40 con 0x7f9708104480 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97081017b0 con 0x7f9708104480 2024-04-25T13:41:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000057d0 con 0x7f9708104480 2024-04-25T13:41:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.222+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708101cb0 con 0x7f9708104480 2024-04-25T13:41:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.223+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970806a160 con 0x7f9708104480 2024-04-25T13:41:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.225+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9700004a40 con 0x7f9708104480 2024-04-25T13:41:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.225+0000 7f97057fa640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 0x7f96f00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.225+0000 7f9707fff640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 0x7f96f00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.225+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f97000e0770 con 0x7f9708104480 2024-04-25T13:41:08.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.226+0000 7f9707fff640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 0x7f96f00b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009230 tx=0x7f96f8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.229+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97000a8b80 con 0x7f9708104480 2024-04-25T13:41:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.409+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97081075e0 con 0x7f9708104480 2024-04-25T13:41:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.410+0000 7f97057fa640 1 -- 172.21.15.94:0/3319178926 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f97000ace80 con 0x7f9708104480 2024-04-25T13:41:08.410 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.411+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 msgr2=0x7f96f00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.411+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 0x7f96f00b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009230 tx=0x7f96f8031400 comp rx=0 tx=0).stop 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.411+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 msgr2=0x7f97081a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.411+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f970000ebb0 tx=0x7f9700007590 comp rx=0 tx=0).stop 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 shutdown_connections 2024-04-25T13:41:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f00afb80 0x7f96f00b2090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9708104480 0x7f97081a05f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103960 0x7f9708199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 --2- 172.21.15.94:0/3319178926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9708102f80 0x7f9708199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 >> 172.21.15.94:0/3319178926 conn(0x7f97080fd380 msgr2=0x7f9708104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 shutdown_connections 2024-04-25T13:41:08.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.412+0000 7f970e2f2640 1 -- 172.21.15.94:0/3319178926 wait complete. 2024-04-25T13:41:08.422 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2208582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:08.422 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2193556892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:08.422 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:08 smithi094 ceph-mon[29195]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:08.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 -- 172.21.15.94:0/3144678648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc1029e0 msgr2=0x7f2cfc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/3144678648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc1036e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009900 tx=0x7f2cec02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 -- 172.21.15.94:0/3144678648 shutdown_connections 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/3144678648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc108920 0x7f2cfc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/3144678648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 0x7f2cfc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/3144678648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc1036e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 -- 172.21.15.94:0/3144678648 >> 172.21.15.94:0/3144678648 conn(0x7f2cfc0fdbd0 msgr2=0x7f2cfc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.534+0000 7f2d00c5a640 1 -- 172.21.15.94:0/3144678648 shutdown_connections 2024-04-25T13:41:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 -- 172.21.15.94:0/3144678648 wait complete. 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 Processor -- start 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 -- start start 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10e4e0 con 0x7f2cfc108920 2024-04-25T13:41:08.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10e360 con 0x7f2cfc103c20 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.535+0000 7f2d00c5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc10e660 con 0x7f2cfc1029e0 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37984/0 (socket says 172.21.15.94:37984) 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 -- 172.21.15.94:0/1249529317 learned_addr learned my addr 172.21.15.94:0/1249529317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cf3fff640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfa59c640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 -- 172.21.15.94:0/1249529317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc1029e0 msgr2=0x7f2cfc199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 -- 172.21.15.94:0/1249529317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 msgr2=0x7f2cfc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 -- 172.21.15.94:0/1249529317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a0f50 con 0x7f2cfc108920 2024-04-25T13:41:08.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cfad9d640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400eaa0 tx=0x7f2cf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400cdb0 con 0x7f2cfc108920 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf4005020 con 0x7f2cfc108920 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40116d0 con 0x7f2cfc108920 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a1240 con 0x7f2cfc108920 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.536+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a1740 con 0x7f2cfc108920 2024-04-25T13:41:08.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.537+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc107730 con 0x7f2cfc108920 2024-04-25T13:41:08.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.539+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2cf40051d0 con 0x7f2cfc108920 2024-04-25T13:41:08.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.539+0000 7f2cf37fe640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 0x7f2ce80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.539+0000 7f2cfa59c640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 0x7f2ce80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.539+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f2cf40df290 con 0x7f2cfc108920 2024-04-25T13:41:08.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.540+0000 7f2cfa59c640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 0x7f2ce80b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005ab0 tx=0x7f2cec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.543+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cf4016030 con 0x7f2cfc108920 2024-04-25T13:41:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:08 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2208582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:08 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2193556892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:08 smithi137 ceph-mon[32568]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:08.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.722+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2cfc02dd90 con 0x7f2cfc108920 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.722+0000 7f2cf37fe640 1 -- 172.21.15.94:0/1249529317 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f2cf40a76a0 con 0x7f2cfc108920 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 msgr2=0x7f2ce80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 0x7f2ce80b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005ab0 tx=0x7f2cec031400 comp rx=0 tx=0).stop 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 msgr2=0x7f2cfc1a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400eaa0 tx=0x7f2cf400ef60 comp rx=0 tx=0).stop 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 shutdown_connections 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ce80afb80 0x7f2ce80b2090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cfc108920 0x7f2cfc1a08e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cfc103c20 0x7f2cfc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 --2- 172.21.15.94:0/1249529317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc1029e0 0x7f2cfc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 >> 172.21.15.94:0/1249529317 conn(0x7f2cfc0fdbd0 msgr2=0x7f2cfc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 shutdown_connections 2024-04-25T13:41:08.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.724+0000 7f2d00c5a640 1 -- 172.21.15.94:0/1249529317 wait complete. 2024-04-25T13:41:08.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2208582298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2193556892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:08 smithi138 ceph-mon[32977]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.842+0000 7fcaa884a640 1 -- 172.21.15.94:0/62127838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4108940 msgr2=0x7fcaa410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.842+0000 7fcaa884a640 1 --2- 172.21.15.94:0/62127838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4108940 0x7fcaa410ad60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca94009ca0 tx=0x7fca9402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 -- 172.21.15.94:0/62127838 shutdown_connections 2024-04-25T13:41:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 --2- 172.21.15.94:0/62127838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4108940 0x7fcaa410ad60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 --2- 172.21.15.94:0/62127838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4103c40 0x7fcaa4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 --2- 172.21.15.94:0/62127838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4102a00 0x7fcaa4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 -- 172.21.15.94:0/62127838 >> 172.21.15.94:0/62127838 conn(0x7fcaa40fdbf0 msgr2=0x7fcaa41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 -- 172.21.15.94:0/62127838 shutdown_connections 2024-04-25T13:41:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 -- 172.21.15.94:0/62127838 wait complete. 2024-04-25T13:41:08.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 Processor -- start 2024-04-25T13:41:08.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.843+0000 7fcaa884a640 1 -- start start 2024-04-25T13:41:08.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 0x7fcaa419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4108940 0x7fcaa41a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410f630 con 0x7fcaa4103c40 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410f4b0 con 0x7fcaa4102a00 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410f7b0 con 0x7fcaa4108940 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38000/0 (socket says 172.21.15.94:38000) 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 -- 172.21.15.94:0/1011027409 learned_addr learned my addr 172.21.15.94:0/1011027409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa259c640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 0x7fcaa419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa2d9d640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4108940 0x7fcaa41a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 -- 172.21.15.94:0/1011027409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4108940 msgr2=0x7fcaa41a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4108940 0x7fcaa41a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 -- 172.21.15.94:0/1011027409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 msgr2=0x7fcaa419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 0x7fcaa419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 -- 172.21.15.94:0/1011027409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa41a1cc0 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa259c640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 0x7fcaa419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa1d9b640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d950 tx=0x7fca9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98017030 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98004020 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa41a1fb0 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980047a0 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.844+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa41a2420 con 0x7fcaa4103c40 2024-04-25T13:41:08.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.845+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa4107750 con 0x7fcaa4103c40 2024-04-25T13:41:08.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.847+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fca98009a30 con 0x7fcaa4103c40 2024-04-25T13:41:08.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.847+0000 7fca937fe640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 0x7fca8c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.847+0000 7fcaa259c640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 0x7fca8c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.848+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fca980def40 con 0x7fcaa4103c40 2024-04-25T13:41:08.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.848+0000 7fcaa259c640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 0x7fca8c0b2090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4103600 tx=0x7fca9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:08.851+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca980a79a0 con 0x7fcaa4103c40 2024-04-25T13:41:09.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.067+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa402db50 con 0x7fcaa4103c40 2024-04-25T13:41:09.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.068+0000 7fca937fe640 1 -- 172.21.15.94:0/1011027409 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7fca980a72a0 con 0x7fcaa4103c40 2024-04-25T13:41:09.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:09.068 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:41:09.069 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 msgr2=0x7fca8c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 0x7fca8c0b2090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4103600 tx=0x7fca9c005f90 comp rx=0 tx=0).stop 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 msgr2=0x7fcaa419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca9800d950 tx=0x7fca9800de10 comp rx=0 tx=0).stop 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 shutdown_connections 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fca8c0afb80 0x7fca8c0b2090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaa4108940 0x7fcaa41a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaa4103c40 0x7fcaa419a6c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 --2- 172.21.15.94:0/1011027409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4102a00 0x7fcaa419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 >> 172.21.15.94:0/1011027409 conn(0x7fcaa40fdbf0 msgr2=0x7fcaa4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 shutdown_connections 2024-04-25T13:41:09.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.070+0000 7fcaa884a640 1 -- 172.21.15.94:0/1011027409 wait complete. 2024-04-25T13:41:09.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.188+0000 7f3055f40640 1 -- 172.21.15.94:0/3395246643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050109890 msgr2=0x7f305010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.188+0000 7f3055f40640 1 --2- 172.21.15.94:0/3395246643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050109890 0x7f305010bcb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3048009ca0 tx=0x7f304802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 -- 172.21.15.94:0/3395246643 shutdown_connections 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 --2- 172.21.15.94:0/3395246643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050109890 0x7f305010bcb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 --2- 172.21.15.94:0/3395246643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050106f30 0x7f3050109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 --2- 172.21.15.94:0/3395246643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 0x7f30501069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 -- 172.21.15.94:0/3395246643 >> 172.21.15.94:0/3395246643 conn(0x7f30500fdbd0 msgr2=0x7f3050100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 -- 172.21.15.94:0/3395246643 shutdown_connections 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 -- 172.21.15.94:0/3395246643 wait complete. 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 Processor -- start 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.189+0000 7f3055f40640 1 -- start start 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 0x7f305019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050109890 0x7f30501a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f305010f610 con 0x7f3050106f30 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f305010f490 con 0x7f3050109890 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f305010f790 con 0x7f30501045d0 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38018/0 (socket says 172.21.15.94:38018) 2024-04-25T13:41:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 -- 172.21.15.94:0/604602813 learned_addr learned my addr 172.21.15.94:0/604602813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304f7fe640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 0x7f305019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304ffff640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050109890 0x7f30501a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 -- 172.21.15.94:0/604602813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 msgr2=0x7f305019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 0x7f305019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 -- 172.21.15.94:0/604602813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050109890 msgr2=0x7f30501a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050109890 0x7f30501a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 -- 172.21.15.94:0/604602813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30501a1cf0 con 0x7f3050106f30 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304effd640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f304400d950 tx=0x7f304400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044017030 con 0x7f3050106f30 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3044004020 con 0x7f3050106f30 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30501a1fe0 con 0x7f3050106f30 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440047a0 con 0x7f3050106f30 2024-04-25T13:41:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.190+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30501a2450 con 0x7f3050106f30 2024-04-25T13:41:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.191+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30500ff6f0 con 0x7f3050106f30 2024-04-25T13:41:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.193+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3044009a30 con 0x7f3050106f30 2024-04-25T13:41:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.193+0000 7f304cff9640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 0x7f30380b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.193+0000 7f304f7fe640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 0x7f30380b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.194+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f30440dee90 con 0x7f3050106f30 2024-04-25T13:41:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.194+0000 7f304f7fe640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 0x7f30380b2090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3040005fd0 tx=0x7f304000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.197+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30440a78f0 con 0x7f3050106f30 2024-04-25T13:41:09.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.379+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f305002dda0 con 0x7f3050106f30 2024-04-25T13:41:09.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.381+0000 7f304cff9640 1 -- 172.21.15.94:0/604602813 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7f30440a71f0 con 0x7f3050106f30 2024-04-25T13:41:09.381 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:09.381 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:09.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 msgr2=0x7f30380b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 0x7f30380b2090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3040005fd0 tx=0x7f304000b390 comp rx=0 tx=0).stop 2024-04-25T13:41:09.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 msgr2=0x7f305019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f304400d950 tx=0x7f304400de10 comp rx=0 tx=0).stop 2024-04-25T13:41:09.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 shutdown_connections 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afb80 0x7f30380b2090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3050109890 0x7f30501a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3050106f30 0x7f305019a6f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 --2- 172.21.15.94:0/604602813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30501045d0 0x7f305019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 >> 172.21.15.94:0/604602813 conn(0x7f30500fdbd0 msgr2=0x7f3050105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 shutdown_connections 2024-04-25T13:41:09.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.383+0000 7f3055f40640 1 -- 172.21.15.94:0/604602813 wait complete. 2024-04-25T13:41:09.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:09.396 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- 172.21.15.94:0/658922699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824103c20 msgr2=0x7fc824104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- 172.21.15.94:0/658922699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824103c20 0x7fc824104030 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc810009900 tx=0x7fc81002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- 172.21.15.94:0/658922699 shutdown_connections 2024-04-25T13:41:09.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- 172.21.15.94:0/658922699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 0x7fc82410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- 172.21.15.94:0/658922699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824103c20 0x7fc824104030 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- 172.21.15.94:0/658922699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8241029e0 0x7fc8241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- 172.21.15.94:0/658922699 >> 172.21.15.94:0/658922699 conn(0x7fc8240fdbd0 msgr2=0x7fc824100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- 172.21.15.94:0/658922699 shutdown_connections 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- 172.21.15.94:0/658922699 wait complete. 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 Processor -- start 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- start start 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 0x7fc82419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 0x7fc8241a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e500 con 0x7fc8241029e0 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e380 con 0x7fc824108920 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.505+0000 7fc82ac34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e680 con 0x7fc824103c20 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38040/0 (socket says 172.21.15.94:38040) 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 -- 172.21.15.94:0/1601773633 learned_addr learned my addr 172.21.15.94:0/1601773633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8291d1640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 0x7fc8241a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc823fff640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 0x7fc82419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 -- 172.21.15.94:0/1601773633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 msgr2=0x7fc82419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 0x7fc82419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 -- 172.21.15.94:0/1601773633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 msgr2=0x7fc8241a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 0x7fc8241a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 -- 172.21.15.94:0/1601773633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8241a20b0 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc823fff640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 0x7fc82419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc8289d0640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc81800c970 tx=0x7fc81800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818004220 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8180043c0 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818004a00 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8241a23a0 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.506+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8241a28a0 con 0x7fc8241029e0 2024-04-25T13:41:09.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.507+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82406c2a0 con 0x7fc8241029e0 2024-04-25T13:41:09.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.509+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc8180079b0 con 0x7fc8241029e0 2024-04-25T13:41:09.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.509+0000 7fc821ffb640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 0x7fc8140b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.509+0000 7fc823fff640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 0x7fc8140b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.509+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc8180debf0 con 0x7fc8241029e0 2024-04-25T13:41:09.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.510+0000 7fc823fff640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 0x7fc8140b2090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc810006010 tx=0x7fc810000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.514+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc818014030 con 0x7fc8241029e0 2024-04-25T13:41:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:09 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3319178926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:09 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1249529317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:09 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1011027409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:09.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.694+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8241a2b60 con 0x7fc8241029e0 2024-04-25T13:41:09.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.695+0000 7fc821ffb640 1 -- 172.21.15.94:0/1601773633 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fc8180a7650 con 0x7fc8241029e0 2024-04-25T13:41:09.695 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 msgr2=0x7fc8140b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 0x7fc8140b2090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc810006010 tx=0x7fc810000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 msgr2=0x7fc82419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc81800c970 tx=0x7fc81800ce30 comp rx=0 tx=0).stop 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 shutdown_connections 2024-04-25T13:41:09.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afb80 0x7fc8140b2090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824108920 0x7fc8241a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103c20 0x7fc82419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 --2- 172.21.15.94:0/1601773633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8241029e0 0x7fc82419a570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 >> 172.21.15.94:0/1601773633 conn(0x7fc8240fdbd0 msgr2=0x7fc824069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 shutdown_connections 2024-04-25T13:41:09.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.697+0000 7fc82ac34640 1 -- 172.21.15.94:0/1601773633 wait complete. 2024-04-25T13:41:09.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:09 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3319178926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:09 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1249529317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:09 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1011027409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:09.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3319178926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1249529317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1011027409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- 172.21.15.94:0/4134041808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274102a50 msgr2=0x7fc274103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 --2- 172.21.15.94:0/4134041808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274102a50 0x7fc274103750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009900 tx=0x7fc26c02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- 172.21.15.94:0/4134041808 shutdown_connections 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 --2- 172.21.15.94:0/4134041808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274108990 0x7fc27410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 --2- 172.21.15.94:0/4134041808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274103c90 0x7fc2741040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 --2- 172.21.15.94:0/4134041808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274102a50 0x7fc274103750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- 172.21.15.94:0/4134041808 >> 172.21.15.94:0/4134041808 conn(0x7fc2740fdc40 msgr2=0x7fc2741000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- 172.21.15.94:0/4134041808 shutdown_connections 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- 172.21.15.94:0/4134041808 wait complete. 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 Processor -- start 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.820+0000 7fc27c534640 1 -- start start 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274102a50 0x7fc274075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 0x7fc2741a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410e540 con 0x7fc274103c90 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410e3c0 con 0x7fc274102a50 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc27410e6c0 con 0x7fc274108990 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38056/0 (socket says 172.21.15.94:38056) 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 -- 172.21.15.94:0/4225169480 learned_addr learned my addr 172.21.15.94:0/4225169480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27a2d0640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274102a50 0x7fc274075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27aad1640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 0x7fc2741a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 -- 172.21.15.94:0/4225169480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 msgr2=0x7fc2741a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 0x7fc2741a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 -- 172.21.15.94:0/4225169480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274102a50 msgr2=0x7fc274075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274102a50 0x7fc274075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 -- 172.21.15.94:0/4225169480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2741a1d40 con 0x7fc274103c90 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27aad1640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 0x7fc2741a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:09.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc279acf640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc26800b4a0 tx=0x7fc26800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268014070 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc268010ac0 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2741a1fd0 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268017c30 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.821+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2741a24c0 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.822+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27406a160 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.824+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc268017070 con 0x7fc274103c90 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.824+0000 7fc2737fe640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 0x7fc25c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.824+0000 7fc27a2d0640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 0x7fc25c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.824+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc2680de0c0 con 0x7fc274103c90 2024-04-25T13:41:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.825+0000 7fc27a2d0640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 0x7fc25c0b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005ab0 tx=0x7fc26c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:09.829+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2680a6c30 con 0x7fc274103c90 2024-04-25T13:41:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.007+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc27402db50 con 0x7fc274103c90 2024-04-25T13:41:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.008+0000 7fc2737fe640 1 -- 172.21.15.94:0/4225169480 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fc2680a6530 con 0x7fc274103c90 2024-04-25T13:41:10.008 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 msgr2=0x7fc25c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 0x7fc25c0b2090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005ab0 tx=0x7fc26c031400 comp rx=0 tx=0).stop 2024-04-25T13:41:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 msgr2=0x7fc27419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc26800b4a0 tx=0x7fc26800b960 comp rx=0 tx=0).stop 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 shutdown_connections 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc25c0afb80 0x7fc25c0b2090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc274108990 0x7fc2741a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc274103c90 0x7fc27419f060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 --2- 172.21.15.94:0/4225169480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc274102a50 0x7fc274075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 >> 172.21.15.94:0/4225169480 conn(0x7fc2740fdc40 msgr2=0x7fc274104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 shutdown_connections 2024-04-25T13:41:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.010+0000 7fc27c534640 1 -- 172.21.15.94:0/4225169480 wait complete. 2024-04-25T13:41:10.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:10.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 -- 172.21.15.94:0/457937780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020109a40 msgr2=0x7f702010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 --2- 172.21.15.94:0/457937780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020109a40 0x7f702010be60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7010009900 tx=0x7f701002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:10.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 -- 172.21.15.94:0/457937780 shutdown_connections 2024-04-25T13:41:10.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 --2- 172.21.15.94:0/457937780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020109a40 0x7f702010be60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 --2- 172.21.15.94:0/457937780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 0x7f7020105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 --2- 172.21.15.94:0/457937780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020103b00 0x7f7020104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 -- 172.21.15.94:0/457937780 >> 172.21.15.94:0/457937780 conn(0x7f70200fecf0 msgr2=0x7f70201011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 -- 172.21.15.94:0/457937780 shutdown_connections 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.132+0000 7f702564c640 1 -- 172.21.15.94:0/457937780 wait complete. 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 Processor -- start 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 -- start start 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 0x7f702019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020109a40 0x7f70201a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f702010f680 con 0x7f7020103b00 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f702010f500 con 0x7f7020104d40 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f702564c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f702010f800 con 0x7f7020109a40 2024-04-25T13:41:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f701effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f701effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38078/0 (socket says 172.21.15.94:38078) 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f701effd640 1 -- 172.21.15.94:0/480400108 learned_addr learned my addr 172.21.15.94:0/480400108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f701f7fe640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020109a40 0x7f70201a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.133+0000 7f701e7fc640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 0x7f702019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 -- 172.21.15.94:0/480400108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020109a40 msgr2=0x7f70201a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020109a40 0x7f70201a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 -- 172.21.15.94:0/480400108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 msgr2=0x7f702019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 0x7f702019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 -- 172.21.15.94:0/480400108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70201a1d30 con 0x7f7020103b00 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f701effd640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f701800ead0 tx=0x7f701800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701800cde0 con 0x7f7020103b00 2024-04-25T13:41:10.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7018003e20 con 0x7f7020103b00 2024-04-25T13:41:10.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70201a1fc0 con 0x7f7020103b00 2024-04-25T13:41:10.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70180116d0 con 0x7f7020103b00 2024-04-25T13:41:10.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.134+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a2430 con 0x7f7020103b00 2024-04-25T13:41:10.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.135+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7020108850 con 0x7f7020103b00 2024-04-25T13:41:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.136+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7018004270 con 0x7f7020103b00 2024-04-25T13:41:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.137+0000 7f7007fff640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 0x7f70080b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.137+0000 7f701e7fc640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 0x7f70080b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.137+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f70180df4b0 con 0x7f7020103b00 2024-04-25T13:41:10.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.137+0000 7f701e7fc640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 0x7f70080b2090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7014005fd0 tx=0x7f7014005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.141+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7018015030 con 0x7f7020103b00 2024-04-25T13:41:10.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.350+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f702002dce0 con 0x7f7020103b00 2024-04-25T13:41:10.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.352+0000 7f7007fff640 1 -- 172.21.15.94:0/480400108 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f70180a7fa0 con 0x7f7020103b00 2024-04-25T13:41:10.352 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:10.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 msgr2=0x7f70080b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 0x7f70080b2090 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7014005fd0 tx=0x7f7014005850 comp rx=0 tx=0).stop 2024-04-25T13:41:10.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 msgr2=0x7f702019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f701800ead0 tx=0x7f701800ef90 comp rx=0 tx=0).stop 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 shutdown_connections 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70080afb80 0x7f70080b2090 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7020109a40 0x7f70201a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7020104d40 0x7f702019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 --2- 172.21.15.94:0/480400108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7020103b00 0x7f702019a1f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 >> 172.21.15.94:0/480400108 conn(0x7f70200fecf0 msgr2=0x7f7020105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 shutdown_connections 2024-04-25T13:41:10.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.354+0000 7f702564c640 1 -- 172.21.15.94:0/480400108 wait complete. 2024-04-25T13:41:10.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.481+0000 7f43383c5640 1 -- 172.21.15.94:0/251123780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330102230 msgr2=0x7f433010bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.481+0000 7f43383c5640 1 --2- 172.21.15.94:0/251123780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330102230 0x7f433010bc90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4328009ca0 tx=0x7f432802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- 172.21.15.94:0/251123780 shutdown_connections 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- 172.21.15.94:0/251123780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330102230 0x7f433010bc90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- 172.21.15.94:0/251123780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330101730 0x7f4330101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- 172.21.15.94:0/251123780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330105080 0x7f4330105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- 172.21.15.94:0/251123780 >> 172.21.15.94:0/251123780 conn(0x7f43300fd330 msgr2=0x7f43300ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- 172.21.15.94:0/251123780 shutdown_connections 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- 172.21.15.94:0/251123780 wait complete. 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 Processor -- start 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- start start 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330102230 0x7f433019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330105080 0x7f43301a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f433010f5f0 con 0x7f4330101730 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f433010f470 con 0x7f4330102230 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.482+0000 7f43383c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f433010f770 con 0x7f4330105080 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38104/0 (socket says 172.21.15.94:38104) 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 -- 172.21.15.94:0/1975057969 learned_addr learned my addr 172.21.15.94:0/1975057969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4335960640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330102230 0x7f433019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336962640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330105080 0x7f43301a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 -- 172.21.15.94:0/1975057969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330105080 msgr2=0x7f43301a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330105080 0x7f43301a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 -- 172.21.15.94:0/1975057969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330102230 msgr2=0x7f433019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330102230 0x7f433019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 -- 172.21.15.94:0/1975057969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43301a1ce0 con 0x7f4330101730 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f4336161640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f432c00e9f0 tx=0x7f432c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c00ccc0 con 0x7f4330101730 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c0035f0 con 0x7f4330101730 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43301a1fd0 con 0x7f4330101730 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c00ce60 con 0x7f4330101730 2024-04-25T13:41:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.483+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43301a2430 con 0x7f4330101730 2024-04-25T13:41:10.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.484+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4330069340 con 0x7f4330101730 2024-04-25T13:41:10.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.485+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f432c003aa0 con 0x7f4330101730 2024-04-25T13:41:10.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.486+0000 7f43277fe640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 0x7f431c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.486+0000 7f4335960640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 0x7f431c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.486+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f432c0df140 con 0x7f4330101730 2024-04-25T13:41:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.487+0000 7f4335960640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 0x7f431c0b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f433019b240 tx=0x7f4320005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.490+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4330069340 con 0x7f4330101730 2024-04-25T13:41:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:10 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/604602813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:10 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1601773633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:10 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4225169480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:10 smithi137 ceph-mon[32568]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:10.646 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:10 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/604602813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:10.646 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:10 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1601773633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.646 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:10 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4225169480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.646 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:10 smithi094 ceph-mon[29195]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:10.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.716+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f433002dc10 con 0x7f4330101730 2024-04-25T13:41:10.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.717+0000 7f43277fe640 1 -- 172.21.15.94:0/1975057969 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7f432c0a7d00 con 0x7f4330101730 2024-04-25T13:41:10.717 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:10.717 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:41:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 msgr2=0x7f431c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 0x7f431c0b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f433019b240 tx=0x7f4320005f90 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 msgr2=0x7f433019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f432c00e9f0 tx=0x7f432c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 shutdown_connections 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f431c0afaf0 0x7f431c0b2000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4330105080 0x7f43301a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4330102230 0x7f433019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 --2- 172.21.15.94:0/1975057969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4330101730 0x7f433019a1a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.719+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 >> 172.21.15.94:0/1975057969 conn(0x7f43300fd330 msgr2=0x7f4330107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.720+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 shutdown_connections 2024-04-25T13:41:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.720+0000 7f43383c5640 1 -- 172.21.15.94:0/1975057969 wait complete. 2024-04-25T13:41:10.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:10 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/604602813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:10 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1601773633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:10 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4225169480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:10.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:10 smithi138 ceph-mon[32977]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/194129250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18102230 msgr2=0x7f6c1810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/194129250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18102230 0x7f6c1810bc90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009ca0 tx=0x7f6c1002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/194129250 shutdown_connections 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/194129250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18102230 0x7f6c1810bc90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/194129250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18101730 0x7f6c18101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/194129250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c18105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/194129250 >> 172.21.15.94:0/194129250 conn(0x7f6c180fd330 msgr2=0x7f6c180ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/194129250 shutdown_connections 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.842+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/194129250 wait complete. 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 Processor -- start 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 -- start start 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18102230 0x7f6c1819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c181a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810f660 con 0x7f6c18101730 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810f4e0 con 0x7f6c18105080 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1fd1c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810f7e0 con 0x7f6c18102230 2024-04-25T13:41:10.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1dab8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1dab8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38124/0 (socket says 172.21.15.94:38124) 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.843+0000 7f6c1dab8640 1 -- 172.21.15.94:0/2786193113 learned_addr learned my addr 172.21.15.94:0/2786193113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1d2b7640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18102230 0x7f6c1819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1e2b9640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c181a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 -- 172.21.15.94:0/2786193113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18102230 msgr2=0x7f6c1819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18102230 0x7f6c1819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 -- 172.21.15.94:0/2786193113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 msgr2=0x7f6c181a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c181a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 -- 172.21.15.94:0/2786193113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c181a1cf0 con 0x7f6c18101730 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1e2b9640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c181a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1dab8640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c970 tx=0x7f6c1400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14007bb0 con 0x7f6c18101730 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c14007d50 con 0x7f6c18101730 2024-04-25T13:41:10.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14018660 con 0x7f6c18101730 2024-04-25T13:41:10.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c181a1fe0 con 0x7f6c18101730 2024-04-25T13:41:10.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.844+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c181a2520 con 0x7f6c18101730 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.845+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1810aa20 con 0x7f6c18101730 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.847+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6c1400f990 con 0x7f6c18101730 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.848+0000 7f6c0effd640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 0x7f6c040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.848+0000 7f6c1d2b7640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 0x7f6c040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.848+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6c140dec10 con 0x7f6c18101730 2024-04-25T13:41:10.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.849+0000 7f6c1d2b7640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 0x7f6c040b2080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c1819b300 tx=0x7f6c08009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:10.853+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c14014030 con 0x7f6c18101730 2024-04-25T13:41:11.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.038+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c1802dd70 con 0x7f6c18101730 2024-04-25T13:41:11.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.039+0000 7f6c0effd640 1 -- 172.21.15.94:0/2786193113 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7f6c140a77d0 con 0x7f6c18101730 2024-04-25T13:41:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:11.040 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 msgr2=0x7f6c040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 0x7f6c040b2080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c1819b300 tx=0x7f6c08009110 comp rx=0 tx=0).stop 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 msgr2=0x7f6c1819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c970 tx=0x7f6c1400cd70 comp rx=0 tx=0).stop 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 shutdown_connections 2024-04-25T13:41:11.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c040afb70 0x7f6c040b2080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c18105080 0x7f6c181a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18102230 0x7f6c1819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 --2- 172.21.15.94:0/2786193113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c18101730 0x7f6c1819a260 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 >> 172.21.15.94:0/2786193113 conn(0x7f6c180fd330 msgr2=0x7f6c18107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 shutdown_connections 2024-04-25T13:41:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.042+0000 7f6c1fd1c640 1 -- 172.21.15.94:0/2786193113 wait complete. 2024-04-25T13:41:11.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 -- 172.21.15.94:0/2122217330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 msgr2=0x7f0044109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 --2- 172.21.15.94:0/2122217330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f0044109330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0038009900 tx=0x7f003802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 -- 172.21.15.94:0/2122217330 shutdown_connections 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 --2- 172.21.15.94:0/2122217330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044109870 0x7f004410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 --2- 172.21.15.94:0/2122217330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f0044109330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 --2- 172.21.15.94:0/2122217330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00441045b0 0x7f00441069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 -- 172.21.15.94:0/2122217330 >> 172.21.15.94:0/2122217330 conn(0x7f00440fdbd0 msgr2=0x7f0044100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 -- 172.21.15.94:0/2122217330 shutdown_connections 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.163+0000 7f0049172640 1 -- 172.21.15.94:0/2122217330 wait complete. 2024-04-25T13:41:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 Processor -- start 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 -- start start 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 0x7f004419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 0x7f00441a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f004410f660 con 0x7f0044106f10 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f004410f4e0 con 0x7f0044109870 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0049172640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f004410f7e0 con 0x7f00441045b0 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f004259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f004259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38138/0 (socket says 172.21.15.94:38138) 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f004259c640 1 -- 172.21.15.94:0/2884361080 learned_addr learned my addr 172.21.15.94:0/2884361080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f0042d9d640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 0x7f004419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.164+0000 7f004359e640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 0x7f00441a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 -- 172.21.15.94:0/2884361080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 msgr2=0x7f004419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 0x7f004419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 -- 172.21.15.94:0/2884361080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 msgr2=0x7f00441a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 0x7f00441a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 -- 172.21.15.94:0/2884361080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441a1dd0 con 0x7f0044106f10 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004359e640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 0x7f00441a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f0042d9d640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 0x7f004419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f004259c640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f0038004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003803f030 con 0x7f0044106f10 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441a2060 con 0x7f0044106f10 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0038003cb0 con 0x7f0044106f10 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038035070 con 0x7f0044106f10 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.165+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a25d0 con 0x7f0044106f10 2024-04-25T13:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.166+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440ff6f0 con 0x7f0044106f10 2024-04-25T13:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.167+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0038042030 con 0x7f0044106f10 2024-04-25T13:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.168+0000 7f002bfff640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 0x7f002c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.168+0000 7f0042d9d640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 0x7f002c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.168+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f00380ff370 con 0x7f0044106f10 2024-04-25T13:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.169+0000 7f0042d9d640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 0x7f002c0b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00340058c0 tx=0x7f0034005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.173+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0038003e60 con 0x7f0044106f10 2024-04-25T13:41:11.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.416+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00440637a0 con 0x7f0044106f10 2024-04-25T13:41:11.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.416+0000 7f002bfff640 1 -- 172.21.15.94:0/2884361080 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7f00380c7f30 con 0x7f0044106f10 2024-04-25T13:41:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:11.417 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:41:11.417 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 msgr2=0x7f002c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 0x7f002c0b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f00340058c0 tx=0x7f0034005850 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 msgr2=0x7f004419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f0038004340 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 shutdown_connections 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f002c0afaf0 0x7f002c0b2000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044109870 0x7f00441a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0044106f10 0x7f004419a7d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 --2- 172.21.15.94:0/2884361080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00441045b0 0x7f004419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.418+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 >> 172.21.15.94:0/2884361080 conn(0x7f00440fdbd0 msgr2=0x7f0044105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.419+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 shutdown_connections 2024-04-25T13:41:11.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.419+0000 7f0049172640 1 -- 172.21.15.94:0/2884361080 wait complete. 2024-04-25T13:41:11.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:39:44.966108+0000', 'modified': '2024-04-25T13:39:51.640274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24512, 'mds_2': 24502, 'mds_3': 24488, 'mds_4': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/3883623467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 3883623467}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 3883623467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6841/2301018309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6840', 'nonce': 2301018309}, {'type': 'v1', 'addr': '172.21.15.94:6841', 'nonce': 2301018309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/4012763239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 4012763239}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 4012763239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/3830518540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 3830518540}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 3830518540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/1621301649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1621301649}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 1621301649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.138:6837/356914816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 356914816}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 356914816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.137:6835/3929131029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3929131029}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3929131029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.138:6839/2137998525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2137998525}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2137998525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/1274977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 1274977027}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 1274977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/3143839143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3143839143}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3143839143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24488, 24502, 24475, 24557, 24512]} 2024-04-25T13:41:11.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T13:41:11.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:41:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 -- 172.21.15.94:0/1553197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04102fa0 msgr2=0x7fad041033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 --2- 172.21.15.94:0/1553197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04102fa0 0x7fad041033b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7facf4009900 tx=0x7facf402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:11.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 -- 172.21.15.94:0/1553197908 shutdown_connections 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 --2- 172.21.15.94:0/1553197908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad04104460 0x7fad0410aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 --2- 172.21.15.94:0/1553197908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04103980 0x7fad04103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 --2- 172.21.15.94:0/1553197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04102fa0 0x7fad041033b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 -- 172.21.15.94:0/1553197908 >> 172.21.15.94:0/1553197908 conn(0x7fad040fd340 msgr2=0x7fad040ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 -- 172.21.15.94:0/1553197908 shutdown_connections 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.541+0000 7fad0b39d640 1 -- 172.21.15.94:0/1553197908 wait complete. 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 Processor -- start 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 -- start start 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad04102fa0 0x7fad04199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 0x7fad041a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e4f0 con 0x7fad04103980 2024-04-25T13:41:11.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e370 con 0x7fad04104460 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0b39d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e670 con 0x7fad04102fa0 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38158/0 (socket says 172.21.15.94:38158) 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 -- 172.21.15.94:0/2567920007 learned_addr learned my addr 172.21.15.94:0/2567920007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0993a640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 0x7fad041a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 -- 172.21.15.94:0/2567920007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad04102fa0 msgr2=0x7fad04199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad04102fa0 0x7fad04199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 -- 172.21.15.94:0/2567920007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 msgr2=0x7fad041a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 0x7fad041a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 -- 172.21.15.94:0/2567920007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad041a0c70 con 0x7fad04103980 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad0993a640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 0x7fad041a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.542+0000 7fad08938640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d950 tx=0x7fad0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.543+0000 7facfa7fc640 1 -- 172.21.15.94:0/2567920007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00017070 con 0x7fad04103980 2024-04-25T13:41:11.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.543+0000 7facfa7fc640 1 -- 172.21.15.94:0/2567920007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad00009c30 con 0x7fad04103980 2024-04-25T13:41:11.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.543+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad041a0f60 con 0x7fad04103980 2024-04-25T13:41:11.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.543+0000 7facfa7fc640 1 -- 172.21.15.94:0/2567920007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0000f620 con 0x7fad04103980 2024-04-25T13:41:11.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.543+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad041a13d0 con 0x7fad04103980 2024-04-25T13:41:11.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 Processor -- start 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 -- start start 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f58330b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f58337a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f58293740 con 0x562f57f8d270 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f5822f3a0 con 0x562f57f6c590 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.545+0000 7fad12250740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f5828f700 con 0x562f57f643b0 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38166/0 (socket says 172.21.15.94:38166) 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 -- 172.21.15.94:0/1579826933 learned_addr learned my addr 172.21.15.94:0/1579826933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad09139640 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f58330b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 -- 172.21.15.94:0/1579826933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 msgr2=0x562f58330b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f58330b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 -- 172.21.15.94:0/1579826933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 msgr2=0x562f58337a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f58337a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 -- 172.21.15.94:0/1579826933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5824b750 con 0x562f57f8d270 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7fad0993a640 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7facfc009f10 tx=0x7facfc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fb007fb3d063c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7facef7fe640 1 -- 172.21.15.94:0/1579826933 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc016070 con 0x562f57f8d270 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7facfa7fc640 1 -- 172.21.15.94:0/2567920007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fad0000f7c0 con 0x7fad04103980 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7facef7fe640 1 -- 172.21.15.94:0/1579826933 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc004d20 con 0x562f57f8d270 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.546+0000 7facef7fe640 1 -- 172.21.15.94:0/1579826933 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc011810 con 0x562f57f8d270 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7facfa7fc640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 0x7facf00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad09139640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 0x7facf00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 -- 172.21.15.94:0/1579826933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 msgr2=0x562f58330580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7facfc009f10 tx=0x7facfc00e6c0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7facfa7fc640 1 -- 172.21.15.94:0/2567920007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fad000de940 con 0x7fad04103980 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 -- 172.21.15.94:0/1579826933 shutdown_connections 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f58337a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f58330b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 --2- 172.21.15.94:0/1579826933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f8d270 0x562f58330580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 -- 172.21.15.94:0/1579826933 >> 172.21.15.94:0/1579826933 conn(0x562f57f813c0 msgr2=0x562f58329d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 -- 172.21.15.94:0/1579826933 shutdown_connections 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad12250740 1 -- 172.21.15.94:0/1579826933 wait complete. 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.547+0000 7fad09139640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 0x7facf00b2000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x562f58331960 tx=0x7facf4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 -- start start 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 Processor -- start 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f5833a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f8d270 0x562f58341970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f58338450 con 0x562f57f6c590 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f5821afe0 con 0x562f57f8d270 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f583385d0 con 0x562f57f643b0 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad12250740 10 client.0 _ll_register_callbacks cb 0x562f58231150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38172/0 (socket says 172.21.15.94:38172) 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad08938640 1 -- 172.21.15.94:0/356059128 learned_addr learned my addr 172.21.15.94:0/356059128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad09139640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f5833a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad0993a640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f8d270 0x562f58341970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.548+0000 7fad08938640 1 -- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 msgr2=0x562f5833a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad08938640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f5833a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad08938640 1 -- 172.21.15.94:0/356059128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f8d270 msgr2=0x562f58341970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad08938640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f8d270 0x562f58341970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad08938640 1 -- 172.21.15.94:0/356059128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f57fa52d0 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad08938640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad000a4070 tx=0x7fad000a66a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cff8d1c07167a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00020040 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fad000a33f0 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc004d20 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fad00009de0 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 10 client.15063 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562f5806f6b0 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc004d20 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fad000a5b60 con 0x562f57f6c590 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 20 client.15063 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f4be40 0x562f583481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 4 client.15063 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.549+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f582a1350 con 0x562f57f4be40 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.550+0000 7fad09139640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f4be40 0x562f583481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.550+0000 7facecff9640 10 client.15063 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.550+0000 7fad09139640 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f4be40 0x562f583481d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad7fb56c00c37460 server_cookie=d31a81c496147954 in_seq=0 out_seq=0 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7facecff9640 1 -- 172.21.15.94:0/356059128 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f582a1350 con 0x562f57f4be40 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7facecff9640 10 client.15063 handle_command_reply: tid=0 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 client.15063 shutdown 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 msgr2=0x562f5833aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad000a4070 tx=0x7fad000a66a0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 shutdown_connections 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f4be40 0x562f583481d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f8d270 0x562f58341970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7facecff9640 0 client.15063 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:11.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f5833aa10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 --2- 172.21.15.94:0/356059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f5833a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 >> 172.21.15.94:0/356059128 conn(0x562f57f813c0 msgr2=0x562f58328d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 shutdown_connections 2024-04-25T13:41:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.551+0000 7fad12250740 1 -- 172.21.15.94:0/356059128 wait complete. 2024-04-25T13:41:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.552+0000 7fad12250740 20 client.15063 trim_cache size 0 max 16384 2024-04-25T13:41:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 Processor -- start 2024-04-25T13:41:11.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 -- start start 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f4be40 0x562f58334f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f583504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f58346510 con 0x562f57f643b0 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f58346be0 con 0x562f57f6c590 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad12250740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f583472b0 con 0x562f57f4be40 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad08938640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38174/0 (socket says 172.21.15.94:38174) 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.570+0000 7fad08938640 1 -- 172.21.15.94:0/1286966867 learned_addr learned my addr 172.21.15.94:0/1286966867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad09139640 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f4be40 0x562f58334f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 -- 172.21.15.94:0/1286966867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f4be40 msgr2=0x562f58334f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f4be40 0x562f58334f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 -- 172.21.15.94:0/1286966867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 msgr2=0x562f583504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f583504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 -- 172.21.15.94:0/1286966867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f58331780 con 0x562f57f643b0 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7fad08938640 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad000a6d90 tx=0x7fad00010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796bdc3ccae2a56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7faceeffd640 1 -- 172.21.15.94:0/1286966867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00020070 con 0x562f57f643b0 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7faceeffd640 1 -- 172.21.15.94:0/1286966867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad000c5070 con 0x562f57f643b0 2024-04-25T13:41:11.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.571+0000 7faceeffd640 1 -- 172.21.15.94:0/1286966867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000a6270 con 0x562f57f643b0 2024-04-25T13:41:11.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- 172.21.15.94:0/1286966867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 msgr2=0x562f58324a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad000a6d90 tx=0x7fad00010cb0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- 172.21.15.94:0/1286966867 shutdown_connections 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f6c590 0x562f583504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f643b0 0x562f58324a20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 --2- 172.21.15.94:0/1286966867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f4be40 0x562f58334f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- 172.21.15.94:0/1286966867 >> 172.21.15.94:0/1286966867 conn(0x562f57f813c0 msgr2=0x562f5832d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- 172.21.15.94:0/1286966867 shutdown_connections 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- 172.21.15.94:0/1286966867 wait complete. 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 -- start start 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.572+0000 7fad12250740 1 Processor -- start 2024-04-25T13:41:11.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f4be40 0x562f58341480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f583419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f58348050 con 0x562f57f6c590 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f58347980 con 0x562f57f4be40 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f58348720 con 0x562f57f643b0 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 10 client.0 _ll_register_callbacks cb 0x7fad000a67e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38176/0 (socket says 172.21.15.94:38176) 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 -- 172.21.15.94:0/3517179333 learned_addr learned my addr 172.21.15.94:0/3517179333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad08938640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f583419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad09139640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f4be40 0x562f58341480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 -- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 msgr2=0x562f583419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f583419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 -- 172.21.15.94:0/3517179333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f4be40 msgr2=0x562f58341480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f4be40 0x562f58341480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 -- 172.21.15.94:0/3517179333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f583af870 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad08938640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f583419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad0993a640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562f58331750 tx=0x7facfc005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f1992fc74552a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7facecff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc028060 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7facfc005580 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.573+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad000c5070 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7facfc016040 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 10 client.15069 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562f583b0af0 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad000c5070 con 0x562f57f6c590 2024-04-25T13:41:11.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7facfc00f040 con 0x562f57f6c590 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 10 client.15069 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 10 client.15069 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 20 client.15069 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f8d270 0x562f583b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 4 client.15069 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "status"}) -- 0x562f5829b1c0 con 0x562f57f8d270 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.574+0000 7fad09139640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f8d270 0x562f583b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7facecff9640 10 client.15069 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad09139640 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f8d270 0x562f583b4000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1550cae3812d67c7 server_cookie=f738b4157a8ede70 in_seq=0 out_seq=0 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7facecff9640 1 -- 172.21.15.94:0/3517179333 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562f5829b1c0 con 0x562f57f8d270 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7facecff9640 10 client.15069 handle_command_reply: tid=0 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 client.15069 shutdown 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 msgr2=0x562f583af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562f58331750 tx=0x7facfc005280 comp rx=0 tx=0).stop 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 shutdown_connections 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562f57f8d270 0x562f583b4000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f57f6c590 0x562f583af280 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7facecff9640 0 client.15069 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f57f643b0 0x562f583419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 --2- 172.21.15.94:0/3517179333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f57f4be40 0x562f58341480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 >> 172.21.15.94:0/3517179333 conn(0x562f57f813c0 msgr2=0x562f5832d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 shutdown_connections 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 1 -- 172.21.15.94:0/3517179333 wait complete. 2024-04-25T13:41:11.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.575+0000 7fad12250740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 4, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24475, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 80.881657981000004, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 104.80489282800001 2024-04-25T13:41:11.583 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:11.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 msgr2=0x7facf00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 0x7facf00b2000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x562f58331960 tx=0x7facf4031400 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 msgr2=0x7fad04199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d950 tx=0x7fad0000de10 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 shutdown_connections 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7facf00afaf0 0x7facf00b2000 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad04104460 0x7fad041a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad04103980 0x7fad04199670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 --2- 172.21.15.94:0/2567920007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad04102fa0 0x7fad04199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 >> 172.21.15.94:0/2567920007 conn(0x7fad040fd340 msgr2=0x7fad04104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 shutdown_connections 2024-04-25T13:41:11.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.576+0000 7fad0b39d640 1 -- 172.21.15.94:0/2567920007 wait complete. 2024-04-25T13:41:11.587 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T13:41:11.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:11 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/480400108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:11.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:11 smithi094 ceph-mon[29195]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:41:11.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:11 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1975057969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:11 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2786193113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:11 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/480400108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:11 smithi137 ceph-mon[32568]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:41:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:11 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1975057969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:11 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2786193113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 -- 172.21.15.94:0/2321328225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 msgr2=0x7fe2b410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 --2- 172.21.15.94:0/2321328225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b410ad60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009ca0 tx=0x7fe2ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 -- 172.21.15.94:0/2321328225 shutdown_connections 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 --2- 172.21.15.94:0/2321328225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b410ad60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 --2- 172.21.15.94:0/2321328225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4103c40 0x7fe2b4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 --2- 172.21.15.94:0/2321328225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4102a00 0x7fe2b4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 -- 172.21.15.94:0/2321328225 >> 172.21.15.94:0/2321328225 conn(0x7fe2b40fdbf0 msgr2=0x7fe2b41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 -- 172.21.15.94:0/2321328225 shutdown_connections 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.697+0000 7fe2bb155640 1 -- 172.21.15.94:0/2321328225 wait complete. 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 Processor -- start 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 -- start start 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 0x7fe2b4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4103c40 0x7fe2b419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410e580 con 0x7fe2b4108940 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410e400 con 0x7fe2b4103c40 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2bb155640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410e700 con 0x7fe2b4102a00 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2b96f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2b96f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38192/0 (socket says 172.21.15.94:38192) 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.698+0000 7fe2b96f2640 1 -- 172.21.15.94:0/176764396 learned_addr learned my addr 172.21.15.94:0/176764396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4103c40 0x7fe2b419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 0x7fe2b4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 -- 172.21.15.94:0/176764396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 msgr2=0x7fe2b4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 0x7fe2b4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 -- 172.21.15.94:0/176764396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4103c40 msgr2=0x7fe2b419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4103c40 0x7fe2b419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 -- 172.21.15.94:0/176764396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b41a1ce0 con 0x7fe2b4108940 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 0x7fe2b4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b96f2640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b1ffb640 1 -- 172.21.15.94:0/176764396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac003de0 con 0x7fe2b4108940 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b41a1f70 con 0x7fe2b4108940 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b1ffb640 1 -- 172.21.15.94:0/176764396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2ac0044f0 con 0x7fe2b4108940 2024-04-25T13:41:11.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2b1ffb640 1 -- 172.21.15.94:0/176764396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac02fce0 con 0x7fe2b4108940 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.699+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41a24b0 con 0x7fe2b4108940 2024-04-25T13:41:11.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.701+0000 7fe2b1ffb640 1 -- 172.21.15.94:0/176764396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe2ac0046a0 con 0x7fe2b4108940 2024-04-25T13:41:11.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 Processor -- start 2024-04-25T13:41:11.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 -- start start 2024-04-25T13:41:11.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 0x558a952c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a95228740 con 0x558a94ef93b0 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a951c43a0 con 0x558a94f01590 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2c2005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a95224700 con 0x558a94f22270 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38200/0 (socket says 172.21.15.94:38200) 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/3717836545 learned_addr learned my addr 172.21.15.94:0/3717836545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b1ffb640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 0x7fe29c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 0x558a952c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/3717836545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 msgr2=0x558a952c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b96f2640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 0x7fe29c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 0x558a952c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/3717836545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 msgr2=0x558a952ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/3717836545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a951e0750 con 0x558a94ef93b0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 0x558a952c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b1ffb640 1 -- 172.21.15.94:0/176764396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe2ac1004f0 con 0x7fe2b4108940 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.702+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4008c20 tx=0x7fe2a400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4163bb16f086c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe29affd640 1 -- 172.21.15.94:0/3717836545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4014070 con 0x558a94ef93b0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2b96f2640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 0x7fe29c0b2000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c4a30 tx=0x7fe2ac08f2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe29affd640 1 -- 172.21.15.94:0/3717836545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a40054d0 con 0x558a94ef93b0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe29affd640 1 -- 172.21.15.94:0/3717836545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004b40 con 0x558a94ef93b0 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 -- 172.21.15.94:0/3717836545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 msgr2=0x558a952c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4008c20 tx=0x7fe2a400e720 comp rx=0 tx=0).stop 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 -- 172.21.15.94:0/3717836545 shutdown_connections 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952c38f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.703+0000 7fe2c2005740 1 --2- 172.21.15.94:0/3717836545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f22270 0x558a952c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- 172.21.15.94:0/3717836545 >> 172.21.15.94:0/3717836545 conn(0x558a94f163c0 msgr2=0x558a952bed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- 172.21.15.94:0/3717836545 shutdown_connections 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- 172.21.15.94:0/3717836545 wait complete. 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- start start 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 Processor -- start 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f01590 0x558a952cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f22270 0x558a952d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a952cb240 con 0x558a94ef93b0 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a951affe0 con 0x558a94f22270 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a952cb3c0 con 0x558a94f01590 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2c2005740 10 client.0 _ll_register_callbacks cb 0x558a951c6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.704+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38212/0 (socket says 172.21.15.94:38212) 2024-04-25T13:41:11.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 -- 172.21.15.94:0/2499578410 learned_addr learned my addr 172.21.15.94:0/2499578410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f01590 0x558a952cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f22270 0x558a952d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 -- 172.21.15.94:0/2499578410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f01590 msgr2=0x558a952cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f01590 0x558a952cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 -- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f22270 msgr2=0x558a952d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f22270 0x558a952d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 -- 172.21.15.94:0/2499578410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a94f3a2d0 con 0x558a94ef93b0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80167d0 tx=0x7fe2a8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b969ab54ab3ad0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe293fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8011b70 con 0x558a94ef93b0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe2a8011d10 con 0x558a94ef93b0 2024-04-25T13:41:11.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a40054d0 con 0x558a94ef93b0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8016e10 con 0x558a94ef93b0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2c2005740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558a950046b0 con 0x558a94ef93b0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.705+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a40054d0 con 0x558a94ef93b0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a8006510 con 0x558a94ef93b0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 10 client.15081 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 20 client.15081 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94ee0e40 0x558a952daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 4 client.15081 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a95236350 con 0x558a94ee0e40 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94ee0e40 0x558a952daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe293fff640 10 client.15081 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.706+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94ee0e40 0x558a952daf60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11f5c31673df3c14 server_cookie=65093cc4eec60fd4 in_seq=0 out_seq=0 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.707+0000 7fe293fff640 1 -- 172.21.15.94:0/2499578410 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a95236350 con 0x558a94ee0e40 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.707+0000 7fe293fff640 10 client.15081 handle_command_reply: tid=0 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.707+0000 7fe2c2005740 1 client.15081 shutdown 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 msgr2=0x558a952cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80167d0 tx=0x7fe2a8009670 comp rx=0 tx=0).stop 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 shutdown_connections 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94ee0e40 0x558a952daf60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f22270 0x558a952d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe293fff640 0 client.15081 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94f01590 0x558a952cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2499578410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952cd260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 >> 172.21.15.94:0/2499578410 conn(0x558a94f163c0 msgr2=0x558a952bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 shutdown_connections 2024-04-25T13:41:11.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 1 -- 172.21.15.94:0/2499578410 wait complete. 2024-04-25T13:41:11.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.708+0000 7fe2c2005740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 Processor -- start 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 -- start start 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ee0e40 0x558a952b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a952d92a0 con 0x558a94ef93b0 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a952d9970 con 0x558a94f01590 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.726+0000 7fe2c2005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a952da040 con 0x558a94ee0e40 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38218/0 (socket says 172.21.15.94:38218) 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/2708673428 learned_addr learned my addr 172.21.15.94:0/2708673428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/2708673428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ee0e40 msgr2=0x558a952b9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ee0e40 0x558a952b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/2708673428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 msgr2=0x558a952d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 -- 172.21.15.94:0/2708673428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a952c80a0 con 0x558a94ef93b0 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558a952c4750 tx=0x7fe2a4008510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9ab8db7d5270f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe29a7fc640 1 -- 172.21.15.94:0/2708673428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4014040 con 0x558a94ef93b0 2024-04-25T13:41:11.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe29a7fc640 1 -- 172.21.15.94:0/2708673428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a4026e50 con 0x558a94ef93b0 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.727+0000 7fe29a7fc640 1 -- 172.21.15.94:0/2708673428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004c70 con 0x558a94ef93b0 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- 172.21.15.94:0/2708673428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 msgr2=0x558a952d4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558a952c4750 tx=0x7fe2a4008510 comp rx=0 tx=0).stop 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- 172.21.15.94:0/2708673428 shutdown_connections 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ef93b0 0x558a952d4690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 --2- 172.21.15.94:0/2708673428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ee0e40 0x558a952b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- 172.21.15.94:0/2708673428 >> 172.21.15.94:0/2708673428 conn(0x558a94f163c0 msgr2=0x558a952cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- 172.21.15.94:0/2708673428 shutdown_connections 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- 172.21.15.94:0/2708673428 wait complete. 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 -- start start 2024-04-25T13:41:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.728+0000 7fe2c2005740 1 Processor -- start 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ef93b0 0x558a952d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a952dade0 con 0x558a94ee0e40 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a952da710 con 0x558a94f01590 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a952db4b0 con 0x558a94ef93b0 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 10 client.0 _ll_register_callbacks cb 0x7fe2a8009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38228/0 (socket says 172.21.15.94:38228) 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 -- 172.21.15.94:0/1613690529 learned_addr learned my addr 172.21.15.94:0/1613690529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 -- 172.21.15.94:0/1613690529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ef93b0 msgr2=0x558a952d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ef93b0 0x558a952d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 -- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 msgr2=0x558a952d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 -- 172.21.15.94:0/1613690529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a952d4400 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2b3fff640 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009860 tx=0x7fe2a80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26b2df8eb918d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe293fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8018950 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe2a8018af0 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a4026e50 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8018cf0 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.729+0000 7fe2c2005740 10 client.15087 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558a95344130 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a4026e50 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a80239c0 con 0x558a94ee0e40 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 10 client.15087 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 20 client.15087 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94f22270 0x558a95347ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 4 client.15087 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "status"}) -- 0x558a952301c0 con 0x558a94f22270 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.730+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94f22270 0x558a95347ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe293fff640 10 client.15087 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2b8ef1640 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94f22270 0x558a95347ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e65e34c4cc42a09 server_cookie=289ef8551707f489 in_seq=0 out_seq=0 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe293fff640 1 -- 172.21.15.94:0/1613690529 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558a952301c0 con 0x558a94f22270 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe293fff640 10 client.15087 handle_command_reply: tid=0 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 client.15087 shutdown 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 msgr2=0x558a952d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009860 tx=0x7fe2a80094e0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 shutdown_connections 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x558a94f22270 0x558a95347ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a94f01590 0x558a952d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe293fff640 0 client.15087 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a94ef93b0 0x558a952d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 --2- 172.21.15.94:0/1613690529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a94ee0e40 0x558a952d09c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 >> 172.21.15.94:0/1613690529 conn(0x558a94f163c0 msgr2=0x558a952be6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 shutdown_connections 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 1 -- 172.21.15.94:0/1613690529 wait complete. 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.731+0000 7fe2c2005740 20 client.15087 trim_cache size 0 max 16384 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 3, 2024-04-25T13:41:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24488, 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 82.061626892000007, 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 110.347704291 2024-04-25T13:41:11.739 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 msgr2=0x7fe29c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 0x7fe29c0b2000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c4a30 tx=0x7fe2ac08f2c0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 msgr2=0x7fe2b41a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac0039f0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 shutdown_connections 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe29c0afaf0 0x7fe29c0b2000 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2b4108940 0x7fe2b41a1550 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b4103c40 0x7fe2b419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 --2- 172.21.15.94:0/176764396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b4102a00 0x7fe2b4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 >> 172.21.15.94:0/176764396 conn(0x7fe2b40fdbf0 msgr2=0x7fe2b4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 shutdown_connections 2024-04-25T13:41:11.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.732+0000 7fe2bb155640 1 -- 172.21.15.94:0/176764396 wait complete. 2024-04-25T13:41:11.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:41:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:11 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/480400108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:11 smithi138 ceph-mon[32977]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:41:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:11 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1975057969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:11 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2786193113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 -- 172.21.15.94:0/2843826633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108940 msgr2=0x7f15c010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 --2- 172.21.15.94:0/2843826633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108940 0x7f15c010ad60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009ca0 tx=0x7f15b802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 -- 172.21.15.94:0/2843826633 shutdown_connections 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 --2- 172.21.15.94:0/2843826633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108940 0x7f15c010ad60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 --2- 172.21.15.94:0/2843826633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0103c40 0x7f15c0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 --2- 172.21.15.94:0/2843826633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0102a00 0x7f15c0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 -- 172.21.15.94:0/2843826633 >> 172.21.15.94:0/2843826633 conn(0x7f15c00fdbf0 msgr2=0x7f15c01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 -- 172.21.15.94:0/2843826633 shutdown_connections 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.892+0000 7f15c5e27640 1 -- 172.21.15.94:0/2843826633 wait complete. 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 Processor -- start 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 -- start start 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0102a00 0x7f15c0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0108940 0x7f15c01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e530 con 0x7f15c0103c40 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e3b0 con 0x7f15c0108940 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15c5e27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e6b0 con 0x7f15c0102a00 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38232/0 (socket says 172.21.15.94:38232) 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 -- 172.21.15.94:0/4210574808 learned_addr learned my addr 172.21.15.94:0/4210574808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0102a00 0x7f15c0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 -- 172.21.15.94:0/4210574808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0102a00 msgr2=0x7f15c0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15bffff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0108940 0x7f15c01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0102a00 0x7f15c0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 -- 172.21.15.94:0/4210574808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0108940 msgr2=0x7f15c01a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0108940 0x7f15c01a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.893+0000 7f15b7fff640 1 -- 172.21.15.94:0/4210574808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c01a1d20 con 0x7f15c0103c40 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15b7fff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9f0 tx=0x7f15a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15bd7fa640 1 -- 172.21.15.94:0/4210574808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a800cd80 con 0x7f15c0103c40 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15bd7fa640 1 -- 172.21.15.94:0/4210574808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a80054d0 con 0x7f15c0103c40 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15bd7fa640 1 -- 172.21.15.94:0/4210574808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8010690 con 0x7f15c0103c40 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c01a1fb0 con 0x7f15c0103c40 2024-04-25T13:41:11.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.894+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c01a2450 con 0x7f15c0103c40 2024-04-25T13:41:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.896+0000 7f15ccdbb740 1 Processor -- start 2024-04-25T13:41:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.896+0000 7f15bd7fa640 1 -- 172.21.15.94:0/4210574808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f15a8004e90 con 0x7f15c0103c40 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.896+0000 7f15ccdbb740 1 -- start start 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befeb33b0 0x558bf027d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 0x558bf02847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558bf01e2740 con 0x558befedc270 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558bf017e3a0 con 0x558befebb590 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bf01de700 con 0x558befeb33b0 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38246/0 (socket says 172.21.15.94:38246) 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 -- 172.21.15.94:0/245377358 learned_addr learned my addr 172.21.15.94:0/245377358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15b7fff640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 0x558bf02847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befeb33b0 0x558bf027d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bd7fa640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 0x7f15ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 -- 172.21.15.94:0/245377358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befeb33b0 msgr2=0x558bf027d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befeb33b0 0x558bf027d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 -- 172.21.15.94:0/245377358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 msgr2=0x558bf02847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 0x558bf02847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 -- 172.21.15.94:0/245377358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf019a750 con 0x558befedc270 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15b7fff640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 0x558bf02847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bd7fa640 1 -- 172.21.15.94:0/4210574808 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15a80df160 con 0x7f15c0103c40 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006010 tx=0x7f15b802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793f6b5e0047e7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15bffff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 0x7f15ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.897+0000 7f15b5ffb640 1 -- 172.21.15.94:0/245377358 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8049040 con 0x558befedc270 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15b5ffb640 1 -- 172.21.15.94:0/245377358 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b8036070 con 0x558befedc270 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15b5ffb640 1 -- 172.21.15.94:0/245377358 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b802f8c0 con 0x558befedc270 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 -- 172.21.15.94:0/245377358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 msgr2=0x558bf027d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15bffff640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 0x7f15ac0b2000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15b8044000 tx=0x7f15b802ff60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006010 tx=0x7f15b802f3a0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 -- 172.21.15.94:0/245377358 shutdown_connections 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befebb590 0x558bf02847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befeb33b0 0x558bf027d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/245377358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befedc270 0x558bf027d370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 -- 172.21.15.94:0/245377358 >> 172.21.15.94:0/245377358 conn(0x558befed03c0 msgr2=0x558bf0278d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 -- 172.21.15.94:0/245377358 shutdown_connections 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.898+0000 7f15ccdbb740 1 -- 172.21.15.94:0/245377358 wait complete. 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 -- start start 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 Processor -- start 2024-04-25T13:41:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf0287190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 0x558bf028e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558bf02851e0 con 0x558befebb590 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558bf0169fe0 con 0x558befeb33b0 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bf0285360 con 0x558befedc270 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15ccdbb740 10 client.0 _ll_register_callbacks cb 0x558bf0180150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38250/0 (socket says 172.21.15.94:38250) 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.899+0000 7f15b7fff640 1 -- 172.21.15.94:0/2399435194 learned_addr learned my addr 172.21.15.94:0/2399435194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf0287190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15bffff640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 0x558bf028e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 -- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 msgr2=0x558bf028e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 0x558bf028e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 -- 172.21.15.94:0/2399435194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 msgr2=0x558bf0287190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf0287190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 -- 172.21.15.94:0/2399435194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558befef42d0 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15bffff640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 0x558bf028e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf0287190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9c0 tx=0x7f15a8005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd693204150498e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8005680 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15a80046c0 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b8036070 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15a80a7d20 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558beffbe6b0 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b8036070 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f15a80afff0 con 0x558befebb590 2024-04-25T13:41:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.900+0000 7f15ccdbb740 20 client.15099 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befe9ae40 0x558bf0294e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f15ccdbb740 4 client.15099 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bf01f0350 con 0x558befe9ae40 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befe9ae40 0x558bf0294e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f159effd640 10 client.15099 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.901+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befe9ae40 0x558bf0294e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a9273647f64223c server_cookie=2360feacdc5dd34 in_seq=0 out_seq=0 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f159effd640 1 -- 172.21.15.94:0/2399435194 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bf01f0350 con 0x558befe9ae40 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f159effd640 10 client.15099 handle_command_reply: tid=0 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 client.15099 shutdown 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 msgr2=0x558bf02876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9c0 tx=0x7f15a8005180 comp rx=0 tx=0).stop 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 shutdown_connections 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befe9ae40 0x558bf0294e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befedc270 0x558bf028e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f159effd640 0 client.15099 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befebb590 0x558bf02876d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2399435194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf0287190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 >> 172.21.15.94:0/2399435194 conn(0x558befed03c0 msgr2=0x558bf0277d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 shutdown_connections 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2399435194 wait complete. 2024-04-25T13:41:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.902+0000 7f15ccdbb740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 Processor -- start 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 -- start start 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf02927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558bf0293190 con 0x558befed03c0 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558bf0293860 con 0x558befeb33b0 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bf0293f30 con 0x558befebb590 2024-04-25T13:41:11.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38254/0 (socket says 172.21.15.94:38254) 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 -- 172.21.15.94:0/1269974248 learned_addr learned my addr 172.21.15.94:0/1269974248 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 -- 172.21.15.94:0/1269974248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 msgr2=0x558bf028e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 -- 172.21.15.94:0/1269974248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 msgr2=0x558bf02927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf02927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 -- 172.21.15.94:0/1269974248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf0282000 con 0x558befed03c0 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.921+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x558bf027e750 tx=0x7f15b0005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b782e3310632a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15b57fa640 1 -- 172.21.15.94:0/1269974248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001f030 con 0x558befed03c0 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15b57fa640 1 -- 172.21.15.94:0/1269974248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b001b380 con 0x558befed03c0 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15b57fa640 1 -- 172.21.15.94:0/1269974248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0003850 con 0x558befed03c0 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 -- 172.21.15.94:0/1269974248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 msgr2=0x558bf0285ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x558bf027e750 tx=0x7f15b0005f50 comp rx=0 tx=0).stop 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 -- 172.21.15.94:0/1269974248 shutdown_connections 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befeb33b0 0x558bf02927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/1269974248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befed03c0 0x558bf0285ea0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 -- 172.21.15.94:0/1269974248 >> 172.21.15.94:0/1269974248 conn(0x558bf0280e80 msgr2=0x558bf0285790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 -- 172.21.15.94:0/1269974248 shutdown_connections 2024-04-25T13:41:11.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.922+0000 7f15ccdbb740 1 -- 172.21.15.94:0/1269974248 wait complete. 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 -- start start 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 Processor -- start 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 0x558bf028e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558bf0294cd0 con 0x558befeb33b0 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558bf0294600 con 0x558befed03c0 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bf02953a0 con 0x558befebb590 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15ccdbb740 10 client.0 _ll_register_callbacks cb 0x7f15a800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38264/0 (socket says 172.21.15.94:38264) 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 -- 172.21.15.94:0/2707892105 learned_addr learned my addr 172.21.15.94:0/2707892105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:11.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bffff640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 0x558bf028e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 -- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 msgr2=0x558bf028b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 -- 172.21.15.94:0/2707892105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 msgr2=0x558bf028e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 0x558bf028e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.923+0000 7f15bf7fe640 1 -- 172.21.15.94:0/2707892105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf028e730 con 0x558befeb33b0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15bffff640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 0x558bf028e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15bf7fe640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15b0004610 tx=0x7f15b00040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1406b439b82a1cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b001b730 con 0x558befeb33b0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15b001b8d0 con 0x558befeb33b0 2024-04-25T13:41:11.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b001b380 con 0x558befeb33b0 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f15b00035f0 con 0x558befeb33b0 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 10 client.15105 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558bf02fe600 con 0x558befeb33b0 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b001b380 con 0x558befeb33b0 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f15b002e730 con 0x558befeb33b0 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 10 client.15105 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 20 client.15105 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befedc270 0x558bf0301b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 4 client.15105 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.924+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "status"}) -- 0x558bf01ea1c0 con 0x558befedc270 2024-04-25T13:41:11.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befedc270 0x558bf0301b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f159effd640 10 client.15105 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15b7fff640 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befedc270 0x558bf0301b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a51cdf380094e4be server_cookie=752e1a66f6d81737 in_seq=0 out_seq=0 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f159effd640 1 -- 172.21.15.94:0/2707892105 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558bf01ea1c0 con 0x558befedc270 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f159effd640 10 client.15105 handle_command_reply: tid=0 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 client.15105 shutdown 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 msgr2=0x558bf028adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15b0004610 tx=0x7f15b00040b0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 shutdown_connections 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558befedc270 0x558bf0301b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558befed03c0 0x558bf028e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558befebb590 0x558bf028b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f159effd640 0 client.15105 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.925+0000 7f15ccdbb740 1 --2- 172.21.15.94:0/2707892105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558befeb33b0 0x558bf028adb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 >> 172.21.15.94:0/2707892105 conn(0x558bf0280e80 msgr2=0x558bf02786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 shutdown_connections 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15ccdbb740 1 -- 172.21.15.94:0/2707892105 wait complete. 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15ccdbb740 20 client.15105 trim_cache size 0 max 16384 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-04-25T13:41:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24502, 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 83.259970680999999, 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 99.567871633999999 2024-04-25T13:41:11.935 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 msgr2=0x7f15ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 0x7f15ac0b2000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15b8044000 tx=0x7f15b802ff60 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 msgr2=0x7f15c019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9f0 tx=0x7f15a800edf0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 shutdown_connections 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15ac0afaf0 0x7f15ac0b2000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0108940 0x7f15c01a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0103c40 0x7f15c019efb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 --2- 172.21.15.94:0/4210574808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0102a00 0x7f15c0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 >> 172.21.15.94:0/4210574808 conn(0x7f15c00fdbf0 msgr2=0x7f15c0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 shutdown_connections 2024-04-25T13:41:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:11.926+0000 7f15c5e27640 1 -- 172.21.15.94:0/4210574808 wait complete. 2024-04-25T13:41:11.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T13:41:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/3489230225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 msgr2=0x7fd1f810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/3489230225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f810bcb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009ca0 tx=0x7fd1f002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/3489230225 shutdown_connections 2024-04-25T13:41:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/3489230225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f810bcb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/3489230225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8106c30 0x7fd1f8109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/3489230225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8101e30 0x7fd1f8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/3489230225 >> 172.21.15.94:0/3489230225 conn(0x7fd1f80fd380 msgr2=0x7fd1f80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/3489230225 shutdown_connections 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.046+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/3489230225 wait complete. 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 Processor -- start 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 -- start start 2024-04-25T13:41:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 0x7fd1f8104300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 0x7fd1f81029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810f610 con 0x7fd1f81097a0 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810f490 con 0x7fd1f8106c30 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1ff2f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810f790 con 0x7fd1f8101e30 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1fd88d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1fd88d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38270/0 (socket says 172.21.15.94:38270) 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1fd88d640 1 -- 172.21.15.94:0/2232724693 learned_addr learned my addr 172.21.15.94:0/2232724693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.047+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 0x7fd1f8104300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 0x7fd1f81029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 -- 172.21.15.94:0/2232724693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 msgr2=0x7fd1f8104300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 0x7fd1f8104300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 -- 172.21.15.94:0/2232724693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 msgr2=0x7fd1f81029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 0x7fd1f81029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 -- 172.21.15.94:0/2232724693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f8103610 con 0x7fd1f81097a0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 0x7fd1f8104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 0x7fd1f81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1fd88d640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097b0 tx=0x7fd1f0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1ee7fc640 1 -- 172.21.15.94:0/2232724693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004240 con 0x7fd1f81097a0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1ee7fc640 1 -- 172.21.15.94:0/2232724693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f002f380 con 0x7fd1f81097a0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f81a4240 con 0x7fd1f81097a0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1ee7fc640 1 -- 172.21.15.94:0/2232724693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002fc30 con 0x7fd1f81097a0 2024-04-25T13:41:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.048+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a46a0 con 0x7fd1f81097a0 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.050+0000 7fd1ee7fc640 1 -- 172.21.15.94:0/2232724693 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd1f00043e0 con 0x7fd1f81097a0 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.050+0000 7fd20619b740 1 Processor -- start 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 -- start start 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642b52fe740 con 0x5642b4ff8270 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642b529a3a0 con 0x5642b4fd7590 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd20619b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642b52fa700 con 0x5642b4fcf3b0 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38272/0 (socket says 172.21.15.94:38272) 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 -- 172.21.15.94:0/2086273736 learned_addr learned my addr 172.21.15.94:0/2086273736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1ee7fc640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 0x7fd1e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fd88d640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 0x7fd1e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 -- 172.21.15.94:0/2086273736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 msgr2=0x5642b53998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 -- 172.21.15.94:0/2086273736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 msgr2=0x5642b53a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 -- 172.21.15.94:0/2086273736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b52b6750 con 0x5642b4ff8270 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1ee7fc640 1 -- 172.21.15.94:0/2232724693 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd1f0100480 con 0x7fd1f81097a0 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.051+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009fa0 tx=0x7fd1e8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff9fe578b6ba46d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd1fd88d640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 0x7fd1e40b2000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00c88d0 tx=0x7fd1f00f7090 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd1e37fe640 1 -- 172.21.15.94:0/2086273736 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80150f0 con 0x5642b4ff8270 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd1e37fe640 1 -- 172.21.15.94:0/2086273736 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e8005020 con 0x5642b4ff8270 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd1e37fe640 1 -- 172.21.15.94:0/2086273736 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8004a00 con 0x5642b4ff8270 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd20619b740 1 -- 172.21.15.94:0/2086273736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 msgr2=0x5642b5399370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.052+0000 7fd20619b740 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009fa0 tx=0x7fd1e8009650 comp rx=0 tx=0).stop 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- 172.21.15.94:0/2086273736 shutdown_connections 2024-04-25T13:41:12.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- 172.21.15.94:0/2086273736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4ff8270 0x5642b5399370 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- 172.21.15.94:0/2086273736 >> 172.21.15.94:0/2086273736 conn(0x5642b4fec3c0 msgr2=0x5642b5394d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- 172.21.15.94:0/2086273736 shutdown_connections 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- 172.21.15.94:0/2086273736 wait complete. 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- start start 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 Processor -- start 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 0x5642b53a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4ff8270 0x5642b53aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642b53a1240 con 0x5642b4fd7590 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642b5285fe0 con 0x5642b4fcf3b0 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642b53a13c0 con 0x5642b4ff8270 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.053+0000 7fd20619b740 10 client.0 _ll_register_callbacks cb 0x5642b529c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38276/0 (socket says 172.21.15.94:38276) 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 -- 172.21.15.94:0/2551379024 learned_addr learned my addr 172.21.15.94:0/2551379024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 0x5642b53a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4ff8270 0x5642b53aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 -- 172.21.15.94:0/2551379024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4ff8270 msgr2=0x5642b53aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4ff8270 0x5642b53aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 -- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 msgr2=0x5642b53a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 0x5642b53a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 -- 172.21.15.94:0/2551379024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b50102d0 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 0x5642b53a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400f710 tx=0x7fd1f400fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e14706e4798355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4016070 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd1f400bb00 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e8005020 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd1f400bd00 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd20619b740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.054+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5642b50da6b0 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e8005020 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd1f401a6d0 con 0x5642b4fd7590 2024-04-25T13:41:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 20 client.15117 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4fb6e40 0x5642b53b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 4 client.15117 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b530c350 con 0x5642b4fb6e40 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.055+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4fb6e40 0x5642b53b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.056+0000 7fd1e0ff9640 10 client.15117 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.056+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4fb6e40 0x5642b53b1130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0357e032ef5adfc server_cookie=dcce62e67047fd1c in_seq=0 out_seq=0 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.056+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/2551379024 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642b530c350 con 0x5642b4fb6e40 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.056+0000 7fd1e0ff9640 10 client.15117 handle_command_reply: tid=0 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 client.15117 shutdown 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 msgr2=0x5642b53a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400f710 tx=0x7fd1f400fbd0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 shutdown_connections 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4fb6e40 0x5642b53b1130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4ff8270 0x5642b53aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd1e0ff9640 0 client.15117 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fd7590 0x5642b53a3840 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 --2- 172.21.15.94:0/2551379024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fcf3b0 0x5642b53a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 >> 172.21.15.94:0/2551379024 conn(0x5642b4fec3c0 msgr2=0x5642b5393d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 shutdown_connections 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 1 -- 172.21.15.94:0/2551379024 wait complete. 2024-04-25T13:41:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.057+0000 7fd20619b740 20 client.15117 trim_cache size 0 max 16384 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 Processor -- start 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 -- start start 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642b53af470 con 0x5642b4fec3c0 2024-04-25T13:41:12.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642b53afb40 con 0x5642b4fd7590 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.075+0000 7fd20619b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642b53b0210 con 0x5642b4fcf3b0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38280/0 (socket says 172.21.15.94:38280) 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 -- 172.21.15.94:0/746029348 learned_addr learned my addr 172.21.15.94:0/746029348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 -- 172.21.15.94:0/746029348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 msgr2=0x5642b53a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 -- 172.21.15.94:0/746029348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 msgr2=0x5642b53a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 -- 172.21.15.94:0/746029348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b539e120 con 0x5642b4fec3c0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009ee0 tx=0x7fd1e80153f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6868eaefe1f50a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1e2ffd640 1 -- 172.21.15.94:0/746029348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80093e0 con 0x5642b4fec3c0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1e2ffd640 1 -- 172.21.15.94:0/746029348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e802b070 con 0x5642b4fec3c0 2024-04-25T13:41:12.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.076+0000 7fd1e2ffd640 1 -- 172.21.15.94:0/746029348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8003970 con 0x5642b4fec3c0 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- 172.21.15.94:0/746029348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 msgr2=0x5642b538fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009ee0 tx=0x7fd1e80153f0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- 172.21.15.94:0/746029348 shutdown_connections 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 --2- 172.21.15.94:0/746029348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b538fa20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- 172.21.15.94:0/746029348 >> 172.21.15.94:0/746029348 conn(0x5642b53af060 msgr2=0x5642b53a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- 172.21.15.94:0/746029348 shutdown_connections 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- 172.21.15.94:0/746029348 wait complete. 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 -- start start 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.077+0000 7fd20619b740 1 Processor -- start 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5642b53b0fb0 con 0x5642b4fec3c0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642b53b08e0 con 0x5642b4fd7590 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642b53b1680 con 0x5642b4fcf3b0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 10 client.0 _ll_register_callbacks cb 0x7fd1f400d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38288/0 (socket says 172.21.15.94:38288) 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 -- 172.21.15.94:0/609919839 learned_addr learned my addr 172.21.15.94:0/609919839 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 -- 172.21.15.94:0/609919839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 msgr2=0x5642b53a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 -- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 msgr2=0x5642b53a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 -- 172.21.15.94:0/609919839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b53a92d0 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1fc88b640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8017040 tx=0x7fd1e8033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e63a446c5e9394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80042c0 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd1e803a040 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e802b070 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd1e8045070 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 10 client.15123 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.078+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5642b541a1e0 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e802b070 con 0x5642b4fec3c0 2024-04-25T13:41:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd1e8021040 con 0x5642b4fec3c0 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 20 client.15123 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4ff8270 0x5642b541d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 4 client.15123 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "status"}) -- 0x5642b53061c0 con 0x5642b4ff8270 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.079+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4ff8270 0x5642b541d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd1e0ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd1fd08c640 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4ff8270 0x5642b541d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7c6cd2c3a778c01 server_cookie=c2983708242c5868 in_seq=0 out_seq=0 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd1e0ff9640 1 -- 172.21.15.94:0/609919839 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5642b53061c0 con 0x5642b4ff8270 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd1e0ff9640 10 client.15123 handle_command_reply: tid=0 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 client.15123 shutdown 2024-04-25T13:41:12.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 msgr2=0x5642b53a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8017040 tx=0x7fd1e8033040 comp rx=0 tx=0).stop 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 shutdown_connections 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5642b4ff8270 0x5642b541d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5642b4fec3c0 0x5642b53a8d10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642b4fd7590 0x5642b53a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd1e0ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 --2- 172.21.15.94:0/609919839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642b4fcf3b0 0x5642b53a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 >> 172.21.15.94:0/609919839 conn(0x5642b53af060 msgr2=0x5642b53a25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 shutdown_connections 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 1 -- 172.21.15.94:0/609919839 wait complete. 2024-04-25T13:41:12.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.080+0000 7fd20619b740 20 client.15123 trim_cache size 0 max 16384 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24512, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 84.429629238000004, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 102.547629336 2024-04-25T13:41:12.088 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 msgr2=0x7fd1e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 0x7fd1e40b2000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00c88d0 tx=0x7fd1f00f7090 comp rx=0 tx=0).stop 2024-04-25T13:41:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 msgr2=0x7fd1f8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097b0 tx=0x7fd1f0003c20 comp rx=0 tx=0).stop 2024-04-25T13:41:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 shutdown_connections 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1e40afaf0 0x7fd1e40b2000 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1f81097a0 0x7fd1f8102ee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1f8106c30 0x7fd1f81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 --2- 172.21.15.94:0/2232724693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f8101e30 0x7fd1f8104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 >> 172.21.15.94:0/2232724693 conn(0x7fd1f80fd380 msgr2=0x7fd1f8107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 shutdown_connections 2024-04-25T13:41:12.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.081+0000 7fd1ff2f0640 1 -- 172.21.15.94:0/2232724693 wait complete. 2024-04-25T13:41:12.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.200+0000 7f0920586640 1 -- 172.21.15.94:0/255657280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 msgr2=0x7f0918106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.200+0000 7f0920586640 1 --2- 172.21.15.94:0/255657280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f0918106b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0914009900 tx=0x7f091402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- 172.21.15.94:0/255657280 shutdown_connections 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- 172.21.15.94:0/255657280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f091810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- 172.21.15.94:0/255657280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 0x7f09181090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- 172.21.15.94:0/255657280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f0918106b20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- 172.21.15.94:0/255657280 >> 172.21.15.94:0/255657280 conn(0x7f09180fd350 msgr2=0x7f09180ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- 172.21.15.94:0/255657280 shutdown_connections 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- 172.21.15.94:0/255657280 wait complete. 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 Processor -- start 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- start start 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 0x7f091819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f09181a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f091810f5e0 con 0x7f0918101e60 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f091810f460 con 0x7f0918102360 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.201+0000 7f0920586640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f091810f760 con 0x7f0918109780 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38314/0 (socket says 172.21.15.94:38314) 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 -- 172.21.15.94:0/4015477888 learned_addr learned my addr 172.21.15.94:0/4015477888 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091db21640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 0x7f091819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091eb23640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f09181a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 -- 172.21.15.94:0/4015477888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 msgr2=0x7f09181a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f09181a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 -- 172.21.15.94:0/4015477888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 msgr2=0x7f091819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 0x7f091819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 -- 172.21.15.94:0/4015477888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09181a1d20 con 0x7f0918101e60 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091eb23640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f09181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f091e322640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0914006010 tx=0x7f0914031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f090f7fe640 1 -- 172.21.15.94:0/4015477888 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091403f030 con 0x7f0918101e60 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f090f7fe640 1 -- 172.21.15.94:0/4015477888 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0914003aa0 con 0x7f0918101e60 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f090f7fe640 1 -- 172.21.15.94:0/4015477888 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914035070 con 0x7f0918101e60 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09181a1fb0 con 0x7f0918101e60 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.202+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09181a2410 con 0x7f0918101e60 2024-04-25T13:41:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 Processor -- start 2024-04-25T13:41:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f090f7fe640 1 -- 172.21.15.94:0/4015477888 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0914042030 con 0x7f0918101e60 2024-04-25T13:41:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 -- start start 2024-04-25T13:41:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9b7590 0x5592ddd807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592ddcde740 con 0x5592dd9d8270 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592ddc7a3a0 con 0x5592dd9af3b0 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f092743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592ddcda700 con 0x5592dd9b7590 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38320/0 (socket says 172.21.15.94:38320) 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.205+0000 7f091eb23640 1 -- 172.21.15.94:0/3574526697 learned_addr learned my addr 172.21.15.94:0/3574526697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091e322640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091db21640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9b7590 0x5592ddd807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 -- 172.21.15.94:0/3574526697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9b7590 msgr2=0x5592ddd807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9b7590 0x5592ddd807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 -- 172.21.15.94:0/3574526697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 msgr2=0x5592ddd798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 -- 172.21.15.94:0/3574526697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ddc96750 con 0x5592dd9d8270 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091e322640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f090f7fe640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 0x7f09040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091e322640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 0x7f09040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f091eb23640 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0910009f90 tx=0x7f091000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b638b283b27558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f090f7fe640 1 -- 172.21.15.94:0/4015477888 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f09140ff230 con 0x7f0918101e60 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f090cbfe640 1 -- 172.21.15.94:0/3574526697 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910003b70 con 0x5592dd9d8270 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f090cbfe640 1 -- 172.21.15.94:0/3574526697 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0910003d10 con 0x5592dd9d8270 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.206+0000 7f090cbfe640 1 -- 172.21.15.94:0/3574526697 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910011900 con 0x5592dd9d8270 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- 172.21.15.94:0/3574526697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 msgr2=0x5592ddd79370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f091e322640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 0x7f09040b2080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5592ddd7a750 tx=0x7f091403b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0910009f90 tx=0x7f091000f7f0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- 172.21.15.94:0/3574526697 shutdown_connections 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9b7590 0x5592ddd807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 --2- 172.21.15.94:0/3574526697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9d8270 0x5592ddd79370 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- 172.21.15.94:0/3574526697 >> 172.21.15.94:0/3574526697 conn(0x5592dd9cc3c0 msgr2=0x5592ddd74d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- 172.21.15.94:0/3574526697 shutdown_connections 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- 172.21.15.94:0/3574526697 wait complete. 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.207+0000 7f092743e740 1 -- start start 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 Processor -- start 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd8a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd81240 con 0x5592dd9af3b0 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592ddc65fe0 con 0x5592dd9b7590 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd813c0 con 0x5592dd9d8270 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f092743e740 10 client.0 _ll_register_callbacks cb 0x5592ddc7c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38332/0 (socket says 172.21.15.94:38332) 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 -- 172.21.15.94:0/579336052 learned_addr learned my addr 172.21.15.94:0/579336052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091db21640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 -- 172.21.15.94:0/579336052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 msgr2=0x5592ddd8a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd8a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 -- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 msgr2=0x5592ddd83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd83790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 -- 172.21.15.94:0/579336052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dd9f02d0 con 0x5592dd9af3b0 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091db21640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f091eb23640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0910003bf0 tx=0x7f0910003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df9cdf87ec728b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f0901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091002a060 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.208+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f0910004d90 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0910003d10 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0910003ec0 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 10 client.15135 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592ddaba6b0 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0910003d10 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0910011c00 con 0x5592dd9af3b0 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 10 client.15135 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 20 client.15135 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:12.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd996e40 0x5592ddd90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 4 client.15135 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592ddcec350 con 0x5592dd996e40 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.209+0000 7f091db21640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd996e40 0x5592ddd90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.210+0000 7f0901ffb640 10 client.15135 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.210+0000 7f091db21640 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd996e40 0x5592ddd90b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9f1fc31d40a976 server_cookie=4193622bdfd7ecef in_seq=0 out_seq=0 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.210+0000 7f0901ffb640 1 -- 172.21.15.94:0/579336052 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592ddcec350 con 0x5592dd996e40 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.210+0000 7f0901ffb640 10 client.15135 handle_command_reply: tid=0 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 client.15135 shutdown 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 msgr2=0x5592ddd83250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0910003bf0 tx=0x7f0910003c20 comp rx=0 tx=0).stop 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 shutdown_connections 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd996e40 0x5592ddd90b80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd8a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f0901ffb640 0 client.15135 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 --2- 172.21.15.94:0/579336052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd83250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 >> 172.21.15.94:0/579336052 conn(0x5592dd9cc3c0 msgr2=0x5592ddd73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 shutdown_connections 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 1 -- 172.21.15.94:0/579336052 wait complete. 2024-04-25T13:41:12.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.211+0000 7f092743e740 20 client.15135 trim_cache size 0 max 16384 2024-04-25T13:41:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 Processor -- start 2024-04-25T13:41:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 -- start start 2024-04-25T13:41:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd7d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd8eec0 con 0x5592dd9b7590 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd8f590 con 0x5592dd9af3b0 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f092743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd8fc60 con 0x5592dd9d8270 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38346/0 (socket says 172.21.15.94:38346) 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 -- 172.21.15.94:0/1448703867 learned_addr learned my addr 172.21.15.94:0/1448703867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 -- 172.21.15.94:0/1448703867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 msgr2=0x5592ddd746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 -- 172.21.15.94:0/1448703867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 msgr2=0x5592ddd7d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd7d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 -- 172.21.15.94:0/1448703867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ddd78f20 con 0x5592dd9b7590 2024-04-25T13:41:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f091db21640 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09080058c0 tx=0x7f090800a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf2f97bfdea05ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f0903fff640 1 -- 172.21.15.94:0/1448703867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908005020 con 0x5592dd9b7590 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f0903fff640 1 -- 172.21.15.94:0/1448703867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09080047a0 con 0x5592dd9b7590 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.230+0000 7f0903fff640 1 -- 172.21.15.94:0/1448703867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090801b690 con 0x5592dd9b7590 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- 172.21.15.94:0/1448703867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 msgr2=0x5592ddd7db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09080058c0 tx=0x7f090800a310 comp rx=0 tx=0).stop 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- 172.21.15.94:0/1448703867 shutdown_connections 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592ddd746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9b7590 0x5592ddd7db80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 --2- 172.21.15.94:0/1448703867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9af3b0 0x5592ddd7d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- 172.21.15.94:0/1448703867 >> 172.21.15.94:0/1448703867 conn(0x5592dd9cc3c0 msgr2=0x5592ddd6fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- 172.21.15.94:0/1448703867 shutdown_connections 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- 172.21.15.94:0/1448703867 wait complete. 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.231+0000 7f092743e740 1 -- start start 2024-04-25T13:41:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 Processor -- start 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592dddf79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd90a00 con 0x5592dd9af3b0 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd90330 con 0x5592dd9b7590 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592ddd910d0 con 0x5592dd9d8270 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 10 client.0 _ll_register_callbacks cb 0x7f0910015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38356/0 (socket says 172.21.15.94:38356) 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 -- 172.21.15.94:0/2935301726 learned_addr learned my addr 172.21.15.94:0/2935301726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 -- 172.21.15.94:0/2935301726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 msgr2=0x5592dddf79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592dddf79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 -- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 msgr2=0x5592ddd853a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 -- 172.21.15.94:0/2935301726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592dddf7f60 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f091eb23640 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0910009a60 tx=0x7f0910011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d34fdcbaf699bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f0901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910034030 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f0910022070 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.232+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09080047a0 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f091002a040 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 10 client.15141 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592dddf9210 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09080047a0 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0910011980 con 0x5592dd9af3b0 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 10 client.15141 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 20 client.15141 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd96c280 0x5592dddfcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 4 client.15141 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "status"}) -- 0x5592ddce61c0 con 0x5592dd96c280 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.233+0000 7f091db21640 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd96c280 0x5592dddfcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f091db21640 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd96c280 0x5592dddfcb80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ae3442e11acf29 server_cookie=4bf7ed74d42ed758 in_seq=0 out_seq=0 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f0901ffb640 10 client.15141 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f0901ffb640 1 -- 172.21.15.94:0/2935301726 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5592ddce61c0 con 0x5592dd96c280 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f0901ffb640 10 client.15141 handle_command_reply: tid=0 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 client.15141 shutdown 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 msgr2=0x5592ddd84e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0910009a60 tx=0x7f0910011630 comp rx=0 tx=0).stop 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 shutdown_connections 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592dd96c280 0x5592dddfcb80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592dd9d8270 0x5592dddf79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592dd9b7590 0x5592ddd853a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f0901ffb640 0 client.15141 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 --2- 172.21.15.94:0/2935301726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592dd9af3b0 0x5592ddd84e60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 >> 172.21.15.94:0/2935301726 conn(0x5592dd9cc3c0 msgr2=0x5592ddd7d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 shutdown_connections 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 1 -- 172.21.15.94:0/2935301726 wait complete. 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.234+0000 7f092743e740 20 client.15141 trim_cache size 0 max 16384 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T13:41:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24557, 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 86.627496628000003, 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 91.615108387999996 2024-04-25T13:41:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 msgr2=0x7f09040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 0x7f09040b2080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5592ddd7a750 tx=0x7f091403b5d0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 msgr2=0x7f091819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0914006010 tx=0x7f0914031a50 comp rx=0 tx=0).stop 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 shutdown_connections 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09040afb70 0x7f09040b2080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918109780 0x7f09181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0918102360 0x7f091819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 --2- 172.21.15.94:0/4015477888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0918101e60 0x7f091819a1e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 >> 172.21.15.94:0/4015477888 conn(0x7f09180fd350 msgr2=0x7f0918106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 shutdown_connections 2024-04-25T13:41:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:12.235+0000 7f0920586640 1 -- 172.21.15.94:0/4015477888 wait complete. 2024-04-25T13:41:12.246 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:41:12.247 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:41:12.257 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T13:41:12.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:41:12.506 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:12.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:12 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2884361080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:12.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:12 smithi094 ceph-mon[29195]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:41:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:12 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2884361080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:12 smithi137 ceph-mon[32568]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:41:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2884361080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:12 smithi138 ceph-mon[32977]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 -- 172.21.15.94:0/2231446778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104bb0 msgr2=0x7fcb1010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 --2- 172.21.15.94:0/2231446778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104bb0 0x7fcb1010b460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800b0a0 tx=0x7fcb0800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 -- 172.21.15.94:0/2231446778 shutdown_connections 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 --2- 172.21.15.94:0/2231446778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104bb0 0x7fcb1010b460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 --2- 172.21.15.94:0/2231446778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb101030e0 0x7fcb101034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 --2- 172.21.15.94:0/2231446778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb10104210 0x7fcb10104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 -- 172.21.15.94:0/2231446778 >> 172.21.15.94:0/2231446778 conn(0x7fcb100fe740 msgr2=0x7fcb10100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 -- 172.21.15.94:0/2231446778 shutdown_connections 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.167+0000 7fcb17493700 1 -- 172.21.15.94:0/2231446778 wait complete. 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 Processor -- start 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 -- start start 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb101030e0 0x7fcb10106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10104bb0 0x7fcb10105a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010ec40 con 0x7fcb10104210 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010eac0 con 0x7fcb101030e0 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb17493700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010edc0 con 0x7fcb10104bb0 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.168+0000 7fcb14a2e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52016/0 (socket says 172.21.15.94:52016) 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 -- 172.21.15.94:0/1255193276 learned_addr learned my addr 172.21.15.94:0/1255193276 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb1522f700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb101030e0 0x7fcb10106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb15a30700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10104bb0 0x7fcb10105a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 -- 172.21.15.94:0/1255193276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10104bb0 msgr2=0x7fcb10105a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10104bb0 0x7fcb10105a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 -- 172.21.15.94:0/1255193276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb101030e0 msgr2=0x7fcb10106ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb101030e0 0x7fcb10106ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 -- 172.21.15.94:0/1255193276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101061f0 con 0x7fcb10104210 2024-04-25T13:41:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb14a2e700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000b530 tx=0x7fcb0000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00015020 con 0x7fcb10104210 2024-04-25T13:41:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00011ce0 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101a49c0 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.169+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00010dd0 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.170+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a4f20 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.171+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb10051190 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.172+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb00018020 con 0x7fcb10104210 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.172+0000 7fcb067fc700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 0x7fcafc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.173+0000 7fcb1522f700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 0x7fcafc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.173+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcb000de8a0 con 0x7fcb10104210 2024-04-25T13:41:14.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.173+0000 7fcb1522f700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 0x7fcafc0b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb10104110 tx=0x7fcb0c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.177+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb0001f020 con 0x7fcb10104210 2024-04-25T13:41:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.396+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcb101a5150 con 0x7fcafc0afb80 2024-04-25T13:41:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.397+0000 7fcb067fc700 1 -- 172.21.15.94:0/1255193276 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcb101a5150 con 0x7fcafc0afb80 2024-04-25T13:41:14.397 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-04-25T13:41:14.397 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-04-25T13:41:14.397 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-04-25T13:41:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 msgr2=0x7fcafc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 0x7fcafc0b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb10104110 tx=0x7fcb0c009190 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 msgr2=0x7fcb10105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000b530 tx=0x7fcb0000b840 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 shutdown_connections 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10104bb0 0x7fcb10105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb101030e0 0x7fcb10106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcafc0afb80 0x7fcafc0b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 --2- 172.21.15.94:0/1255193276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb10104210 0x7fcb10105550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 >> 172.21.15.94:0/1255193276 conn(0x7fcb100fe740 msgr2=0x7fcb101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 shutdown_connections 2024-04-25T13:41:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:14.399+0000 7fcb17493700 1 -- 172.21.15.94:0/1255193276 wait complete. 2024-04-25T13:41:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:14 smithi094 ceph-mon[29195]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:14 smithi137 ceph-mon[32568]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:14 smithi138 ceph-mon[32977]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:15.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:41:15.284 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:15.615 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:15 smithi094 ceph-mon[29195]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:15 smithi137 ceph-mon[32568]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:15 smithi138 ceph-mon[32977]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:16.631 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:16 smithi094 ceph-mon[29195]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:16 smithi137 ceph-mon[32568]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:16 smithi138 ceph-mon[32977]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 -- 172.21.15.94:0/2298213979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 msgr2=0x7f6d0810b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 --2- 172.21.15.94:0/2298213979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d0810b4a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b0a0 tx=0x7f6d0000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 -- 172.21.15.94:0/2298213979 shutdown_connections 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 --2- 172.21.15.94:0/2298213979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d0810b4a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 --2- 172.21.15.94:0/2298213979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 0x7f6d08103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 --2- 172.21.15.94:0/2298213979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 0x7f6d08104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 -- 172.21.15.94:0/2298213979 >> 172.21.15.94:0/2298213979 conn(0x7f6d080fe760 msgr2=0x7f6d08100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 -- 172.21.15.94:0/2298213979 shutdown_connections 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.867+0000 7f6d0f923700 1 -- 172.21.15.94:0/2298213979 wait complete. 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 Processor -- start 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 -- start start 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 0x7f6d0819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 0x7f6d0819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0810eb90 con 0x7f6d08104bf0 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0810ea10 con 0x7f6d08103120 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0f923700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0810ed10 con 0x7f6d08104250 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52028/0 (socket says 172.21.15.94:52028) 2024-04-25T13:41:16.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 -- 172.21.15.94:0/1163490316 learned_addr learned my addr 172.21.15.94:0/1163490316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0cebe700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 0x7f6d0819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0d6bf700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 0x7f6d0819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 -- 172.21.15.94:0/1163490316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 msgr2=0x7f6d0819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 0x7f6d0819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 -- 172.21.15.94:0/1163490316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 msgr2=0x7f6d0819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 0x7f6d0819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.868+0000 7f6d0dec0700 1 -- 172.21.15.94:0/1163490316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d081a22c0 con 0x7f6d08104bf0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6d0dec0700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b070 tx=0x7f6d00004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00021070 con 0x7f6d08104bf0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d00007c00 con 0x7f6d08104bf0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d081a2550 con 0x7f6d08104bf0 2024-04-25T13:41:16.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d000052a0 con 0x7f6d08104bf0 2024-04-25T13:41:16.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.869+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d081a2ae0 con 0x7f6d08104bf0 2024-04-25T13:41:16.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.870+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d08051190 con 0x7f6d08104bf0 2024-04-25T13:41:16.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.871+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6d00024070 con 0x7f6d08104bf0 2024-04-25T13:41:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.872+0000 7f6cfe7fc700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 0x7f6cf40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.872+0000 7f6d0d6bf700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 0x7f6cf40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.872+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6d0000f020 con 0x7f6d08104bf0 2024-04-25T13:41:16.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.872+0000 7f6d0d6bf700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 0x7f6cf40b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d08104150 tx=0x7f6d04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:16.876+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d0002b020 con 0x7f6d08104bf0 2024-04-25T13:41:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.086+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6d08100750 con 0x7f6cf40afc00 2024-04-25T13:41:17.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.100+0000 7f6cfe7fc700 1 -- 172.21.15.94:0/1163490316 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6d08100750 con 0x7f6cf40afc00 2024-04-25T13:41:17.100 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:41:17.100 INFO:teuthology.orchestra.run.smithi094.stdout:mds.a smithi094 running (99s) 15s ago 99s 84.8M - 19.0.0-3291-g994e2f20 d53c96779a29 d5fdd1d08f3e 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.b smithi137 running (96s) 30s ago 96s 96.0M - 19.0.0-3291-g994e2f20 d53c96779a29 a6e3ee9a6b3a 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.c smithi138 running (110s) 30s ago 110s 83.7M - 19.0.0-3291-g994e2f20 d53c96779a29 0cf929995c85 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.d smithi094 running (118s) 15s ago 118s 83.0M - 19.0.0-3291-g994e2f20 d53c96779a29 29f43754b6a8 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.e smithi137 running (116s) 30s ago 115s 84.9M - 19.0.0-3291-g994e2f20 d53c96779a29 6524bf848e74 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.f smithi138 running (113s) 30s ago 112s 89.1M - 19.0.0-3291-g994e2f20 d53c96779a29 c586f1c3852b 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.g smithi094 running (102s) 15s ago 101s 86.2M - 19.0.0-3291-g994e2f20 d53c96779a29 8a1744e7809c 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.h smithi137 running (107s) 30s ago 107s 85.5M - 19.0.0-3291-g994e2f20 d53c96779a29 64b977006618 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.i smithi138 running (105s) 30s ago 104s 84.7M - 19.0.0-3291-g994e2f20 d53c96779a29 a10d3a91f64b 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.j smithi094 running (2m) 15s ago 2m 83.9M - 19.0.0-3291-g994e2f20 d53c96779a29 abaab5d7e28e 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.k smithi137 running (2m) 30s ago 2m 86.7M - 19.0.0-3291-g994e2f20 d53c96779a29 c5ae7ce5f129 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mds.l smithi138 running (2m) 30s ago 2m 83.2M - 19.0.0-3291-g994e2f20 d53c96779a29 a002a84e2196 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi094 *:9283,8765 running (9m) 15s ago 9m 585M - 19.0.0-3291-g994e2f20 d53c96779a29 fdb23ed7978a 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi137 *:8443,8765 running (7m) 30s ago 7m 523M - 19.0.0-3291-g994e2f20 d53c96779a29 2e733b5a3d35 2024-04-25T13:41:17.101 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi138 *:8443,8765 running (7m) 30s ago 7m 525M - 19.0.0-3291-g994e2f20 d53c96779a29 744714fd3aea 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (9m) 15s ago 9m 119M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 5f7c39663efe 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi137 running (7m) 30s ago 7m 109M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 d7eb3cd3aa0a 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi138 running (7m) 30s ago 7m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 0a48be0527ea 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 15s ago 7m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 200c596276c5 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi137 running (6m) 30s ago 6m 115M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 8207e9eeee04 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi138 running (6m) 30s ago 6m 118M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 cd5d55b4f86d 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi094 running (5m) 15s ago 5m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 da4ab2244597 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi137 running (5m) 30s ago 5m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 200bc586b1d6 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi138 running (5m) 30s ago 5m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 185b6b25d175 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 smithi094 running (4m) 15s ago 4m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 32eb9cb47d9b 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 smithi137 running (4m) 30s ago 4m 116M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 1890b6d09e9a 2024-04-25T13:41:17.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 smithi138 running (3m) 30s ago 3m 122M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 d0a11f6b1a3b 2024-04-25T13:41:17.103 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 smithi094 running (3m) 15s ago 3m 119M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 b8aece4a841b 2024-04-25T13:41:17.103 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 smithi137 running (2m) 30s ago 2m 122M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 5b107141e1fc 2024-04-25T13:41:17.103 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 smithi138 running (2m) 30s ago 2m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 6bda862c6789 2024-04-25T13:41:17.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 msgr2=0x7f6cf40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 0x7f6cf40b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d08104150 tx=0x7f6d04009260 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 msgr2=0x7f6d081a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b070 tx=0x7f6d00004d90 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 shutdown_connections 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf40afc00 0x7f6cf40b2110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d08104bf0 0x7f6d081a1c40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d08103120 0x7f6d0819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 --2- 172.21.15.94:0/1163490316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d08104250 0x7f6d0819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 >> 172.21.15.94:0/1163490316 conn(0x7f6d080fe760 msgr2=0x7f6d08100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 shutdown_connections 2024-04-25T13:41:17.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:17.102+0000 7f6d0f923700 1 -- 172.21.15.94:0/1163490316 wait complete. 2024-04-25T13:41:17.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:17 smithi137 ceph-mon[32568]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:17 smithi094 ceph-mon[29195]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:17 smithi138 ceph-mon[32977]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:17.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:41:18.002 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:18.404 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:18 smithi094 ceph-mon[29195]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T13:41:18.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:18 smithi137 ceph-mon[32568]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T13:41:18.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:18 smithi138 ceph-mon[32977]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 -- 172.21.15.94:0/941187661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c1030e0 msgr2=0x7f603c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 --2- 172.21.15.94:0/941187661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c1030e0 0x7f603c1034f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f602c009990 tx=0x7f602c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 -- 172.21.15.94:0/941187661 shutdown_connections 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 --2- 172.21.15.94:0/941187661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 0x7f603c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 --2- 172.21.15.94:0/941187661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c1030e0 0x7f603c1034f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 --2- 172.21.15.94:0/941187661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c104210 0x7f603c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 -- 172.21.15.94:0/941187661 >> 172.21.15.94:0/941187661 conn(0x7f603c0fe740 msgr2=0x7f603c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 -- 172.21.15.94:0/941187661 shutdown_connections 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.726+0000 7f604439a700 1 -- 172.21.15.94:0/941187661 wait complete. 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 Processor -- start 2024-04-25T13:41:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 -- start start 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 0x7f603c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 0x7f603c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10eba0 con 0x7f603c104210 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10ea20 con 0x7f603c1030e0 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f604439a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10ed20 con 0x7f603c104bb0 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f6041935700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f6041935700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52048/0 (socket says 172.21.15.94:52048) 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f6041935700 1 -- 172.21.15.94:0/2271754452 learned_addr learned my addr 172.21.15.94:0/2271754452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.727+0000 7f6042937700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 0x7f603c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6042136700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 0x7f603c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 -- 172.21.15.94:0/2271754452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 msgr2=0x7f603c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 0x7f603c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 -- 172.21.15.94:0/2271754452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 msgr2=0x7f603c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 0x7f603c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 -- 172.21.15.94:0/2271754452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c1a1240 con 0x7f603c104210 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6042136700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 0x7f603c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f6041935700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f603800b530 tx=0x7f603800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038015020 con 0x7f603c104210 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6038011ce0 con 0x7f603c104210 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038010e40 con 0x7f603c104210 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c1a1530 con 0x7f603c104210 2024-04-25T13:41:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.728+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1a1aa0 con 0x7f603c104210 2024-04-25T13:41:19.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.729+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c051190 con 0x7f603c104210 2024-04-25T13:41:19.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.730+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6038018020 con 0x7f603c104210 2024-04-25T13:41:19.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.731+0000 7f60337fe700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 0x7f60280b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.731+0000 7f6042136700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 0x7f60280b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.731+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f60380de3e0 con 0x7f603c104210 2024-04-25T13:41:19.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.731+0000 7f6042136700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 0x7f60280b2110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f602c00b5c0 tx=0x7f602c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.738+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f603801f020 con 0x7f603c104210 2024-04-25T13:41:19.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.946+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f603c063ad0 con 0x7f60280afc00 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.952+0000 7f60337fe700 1 -- 172.21.15.94:0/2271754452 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f603c063ad0 con 0x7f60280afc00 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:mds.all 12/12 33s ago 2m smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 33s ago 7m smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 33s ago 8m smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T13:41:19.953 INFO:teuthology.orchestra.run.smithi094.stdout:osd 12 33s ago - 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 msgr2=0x7f60280b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 0x7f60280b2110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f602c00b5c0 tx=0x7f602c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 msgr2=0x7f603c199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f603800b530 tx=0x7f603800b840 comp rx=0 tx=0).stop 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 shutdown_connections 2024-04-25T13:41:19.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f603c104bb0 0x7f603c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f603c1030e0 0x7f603c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60280afc00 0x7f60280b2110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 --2- 172.21.15.94:0/2271754452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f603c104210 0x7f603c199cb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 >> 172.21.15.94:0/2271754452 conn(0x7f603c0fe740 msgr2=0x7f603c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 shutdown_connections 2024-04-25T13:41:19.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:19.955+0000 7f604439a700 1 -- 172.21.15.94:0/2271754452 wait complete. 2024-04-25T13:41:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:20 smithi137 ceph-mon[32568]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:20 smithi137 ceph-mon[32568]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:20.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:20 smithi094 ceph-mon[29195]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:20.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:20 smithi094 ceph-mon[29195]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:20.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:41:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:20 smithi138 ceph-mon[32977]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:20 smithi138 ceph-mon[32977]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T13:41:20.978 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:22.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:22 smithi094 ceph-mon[29195]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.628+0000 7f18dbe63700 1 -- 172.21.15.94:0/3531140979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104bf0 msgr2=0x7f18d410b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.628+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3531140979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104bf0 0x7f18d410b4a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b0a0 tx=0x7f18cc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 -- 172.21.15.94:0/3531140979 shutdown_connections 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3531140979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104bf0 0x7f18d410b4a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3531140979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 0x7f18d4103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3531140979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104250 0x7f18d4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 -- 172.21.15.94:0/3531140979 >> 172.21.15.94:0/3531140979 conn(0x7f18d40fe760 msgr2=0x7f18d4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 -- 172.21.15.94:0/3531140979 shutdown_connections 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 -- 172.21.15.94:0/3531140979 wait complete. 2024-04-25T13:41:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 Processor -- start 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.629+0000 7f18dbe63700 1 -- start start 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 0x7f18d41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 0x7f18d41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18d410ec10 con 0x7f18d4104250 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18d410ea90 con 0x7f18d4103120 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18d410ed90 con 0x7f18d4104bf0 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52054/0 (socket says 172.21.15.94:52054) 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 -- 172.21.15.94:0/3906917496 learned_addr learned my addr 172.21.15.94:0/3906917496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d9bff700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 0x7f18d41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18da400700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 0x7f18d41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 -- 172.21.15.94:0/3906917496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 msgr2=0x7f18d41a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 0x7f18d41a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 -- 172.21.15.94:0/3906917496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 msgr2=0x7f18d41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 0x7f18d41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 -- 172.21.15.94:0/3906917496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d41a1220 con 0x7f18d4104250 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18da400700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 0x7f18d41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18d93fe700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f18c400d9e0 tx=0x7f18c400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c401b070 con 0x7f18d4104250 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c400ecd0 con 0x7f18d4104250 2024-04-25T13:41:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d4075920 con 0x7f18d4104250 2024-04-25T13:41:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c40146c0 con 0x7f18d4104250 2024-04-25T13:41:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.630+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d4075ee0 con 0x7f18d4104250 2024-04-25T13:41:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.631+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d4051190 con 0x7f18d4104250 2024-04-25T13:41:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.633+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f18c401e070 con 0x7f18d4104250 2024-04-25T13:41:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.634+0000 7f18caffd700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 0x7f18c00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.634+0000 7f18d9bff700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 0x7f18c00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.634+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f18c40e06c0 con 0x7f18d4104250 2024-04-25T13:41:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.634+0000 7f18d9bff700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 0x7f18c00b2090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f18d0005950 tx=0x7f18d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.638+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18c40a9310 con 0x7f18d4104250 2024-04-25T13:41:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:22 smithi137 ceph-mon[32568]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:22 smithi138 ceph-mon[32977]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:41:22.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.843+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f18d40761a0 con 0x7f18c00afb80 2024-04-25T13:41:22.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.845+0000 7f18caffd700 1 -- 172.21.15.94:0/3906917496 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f18d40761a0 con 0x7f18c00afb80 2024-04-25T13:41:22.846 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:41:22.846 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-04-25T13:41:22.846 INFO:teuthology.orchestra.run.smithi094.stdout:smithi137 172.21.15.137 2024-04-25T13:41:22.846 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 172.21.15.138 2024-04-25T13:41:22.846 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 msgr2=0x7f18c00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 0x7f18c00b2090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f18d0005950 tx=0x7f18d00058e0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 msgr2=0x7f18d4199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f18c400d9e0 tx=0x7f18c400dda0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 shutdown_connections 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d4104bf0 0x7f18d41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18d4103120 0x7f18d41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f18c00afb80 0x7f18c00b2090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 --2- 172.21.15.94:0/3906917496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18d4104250 0x7f18d4199c00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 >> 172.21.15.94:0/3906917496 conn(0x7f18d40fe760 msgr2=0x7f18d4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 shutdown_connections 2024-04-25T13:41:22.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:22.847+0000 7f18dbe63700 1 -- 172.21.15.94:0/3906917496 wait complete. 2024-04-25T13:41:23.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:23 smithi094 ceph-mon[29195]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:23.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:41:23.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:23 smithi137 ceph-mon[32568]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:23 smithi138 ceph-mon[32977]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:23.812 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:24 smithi094 ceph-mon[29195]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:41:24.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:24 smithi137 ceph-mon[32568]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:41:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:24 smithi138 ceph-mon[32977]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 -- 172.21.15.94:0/1684052166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438103100 msgr2=0x7f3438103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 --2- 172.21.15.94:0/1684052166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438103100 0x7f3438103510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3434009a10 tx=0x7f3434009d20 comp rx=0 tx=0).stop 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 -- 172.21.15.94:0/1684052166 shutdown_connections 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 --2- 172.21.15.94:0/1684052166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 0x7f343810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 --2- 172.21.15.94:0/1684052166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438103100 0x7f3438103510 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 --2- 172.21.15.94:0/1684052166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438104230 0x7f3438104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 -- 172.21.15.94:0/1684052166 >> 172.21.15.94:0/1684052166 conn(0x7f34380fe740 msgr2=0x7f3438100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 -- 172.21.15.94:0/1684052166 shutdown_connections 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 -- 172.21.15.94:0/1684052166 wait complete. 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.461+0000 7f343f64b700 1 Processor -- start 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 -- start start 2024-04-25T13:41:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438103100 0x7f343819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 0x7f34381a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f343810daf0 con 0x7f3438104230 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f343810d970 con 0x7f3438104bd0 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343f64b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f343810dc70 con 0x7f3438103100 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53888/0 (socket says 172.21.15.94:53888) 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 -- 172.21.15.94:0/4235831904 learned_addr learned my addr 172.21.15.94:0/4235831904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343d3e7700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438103100 0x7f343819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343dbe8700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 0x7f34381a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 -- 172.21.15.94:0/4235831904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438103100 msgr2=0x7f343819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438103100 0x7f343819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 -- 172.21.15.94:0/4235831904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 msgr2=0x7f34381a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 0x7f34381a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 -- 172.21.15.94:0/4235831904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34381a4560 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.462+0000 7f343cbe6700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f342800d980 tx=0x7f342800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.463+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428016070 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.463+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34280044d0 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.463+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34381a4850 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.463+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428004d00 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.463+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381a4d90 con 0x7f3438104230 2024-04-25T13:41:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.464+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438051190 con 0x7f3438104230 2024-04-25T13:41:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.465+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3428004020 con 0x7f3438104230 2024-04-25T13:41:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.466+0000 7f342e7fc700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 0x7f34240b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.466+0000 7f343d3e7700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 0x7f34240b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.466+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f34280df0f0 con 0x7f3438104230 2024-04-25T13:41:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.466+0000 7f343d3e7700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 0x7f34240b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3434018040 tx=0x7f3434005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.469+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f342801f020 con 0x7f3438104230 2024-04-25T13:41:25.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.679+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3438063ad0 con 0x7f34240afb80 2024-04-25T13:41:25.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.680+0000 7f342e7fc700 1 -- 172.21.15.94:0/4235831904 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3438063ad0 con 0x7f34240afb80 2024-04-25T13:41:25.680 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:41:25.680 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:41:25.680 INFO:teuthology.orchestra.run.smithi094.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:41:25.680 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 msgr2=0x7f34240b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 0x7f34240b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3434018040 tx=0x7f3434005fb0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 msgr2=0x7f343819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f342800d980 tx=0x7f342800dd40 comp rx=0 tx=0).stop 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 shutdown_connections 2024-04-25T13:41:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3438104bd0 0x7f34381a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3438103100 0x7f343819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34240afb80 0x7f34240b2090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 --2- 172.21.15.94:0/4235831904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3438104230 0x7f343819cf40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 >> 172.21.15.94:0/4235831904 conn(0x7f34380fe740 msgr2=0x7f3438107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 shutdown_connections 2024-04-25T13:41:25.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:25.683+0000 7f343f64b700 1 -- 172.21.15.94:0/4235831904 wait complete. 2024-04-25T13:41:26.492 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:41:26.501 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T13:41:26.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:41:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:26 smithi137 ceph-mon[32568]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:26 smithi137 ceph-mon[32568]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:41:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:26 smithi094 ceph-mon[29195]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:26 smithi094 ceph-mon[29195]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:41:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:26 smithi138 ceph-mon[32977]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:26 smithi138 ceph-mon[32977]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:41:28.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 -- 172.21.15.94:0/370969553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0104ce0 msgr2=0x7f42a0107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 --2- 172.21.15.94:0/370969553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0104ce0 0x7f42a0107100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4290009990 tx=0x7f4290009ca0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 -- 172.21.15.94:0/370969553 shutdown_connections 2024-04-25T13:41:28.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 --2- 172.21.15.94:0/370969553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0109fa0 0x7f42a010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 --2- 172.21.15.94:0/370969553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0104ce0 0x7f42a0107100 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 --2- 172.21.15.94:0/370969553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0107640 0x7f42a0109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 -- 172.21.15.94:0/370969553 >> 172.21.15.94:0/370969553 conn(0x7f42a00fe740 msgr2=0x7f42a0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 -- 172.21.15.94:0/370969553 shutdown_connections 2024-04-25T13:41:28.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:28 smithi094 ceph-mon[29195]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 -- 172.21.15.94:0/370969553 wait complete. 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.409+0000 7f42a61df700 1 Processor -- start 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 -- start start 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 0x7f42a019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0107640 0x7f42a019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42a010fc20 con 0x7f42a0109fa0 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42a010faa0 con 0x7f42a0107640 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f42a61df700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42a010fda0 con 0x7f42a0104ce0 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 0x7f42a019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53906/0 (socket says 172.21.15.94:53906) 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 0x7f42a019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60532/0 (socket says 172.21.15.94:60532) 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 -- 172.21.15.94:0/1806218099 learned_addr learned my addr 172.21.15.94:0/1806218099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429effd700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0107640 0x7f42a019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 -- 172.21.15.94:0/1806218099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 msgr2=0x7f42a019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 0x7f42a019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 -- 172.21.15.94:0/1806218099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0107640 msgr2=0x7f42a019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0107640 0x7f42a019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.410+0000 7f429ffff700 1 -- 172.21.15.94:0/1806218099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a01a22f0 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f429ffff700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4298007ee0 tx=0x7f429800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298004bb0 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4298004d50 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298005920 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a01a25e0 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.411+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a01a2a10 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.412+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a0051190 con 0x7f42a0109fa0 2024-04-25T13:41:28.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.414+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f429801a020 con 0x7f42a0109fa0 2024-04-25T13:41:28.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.415+0000 7f429cff9700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 0x7f42880b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.415+0000 7f429f7fe700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 0x7f42880b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.415+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f42980de860 con 0x7f42a0109fa0 2024-04-25T13:41:28.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.415+0000 7f429f7fe700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 0x7f42880b2110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4290006010 tx=0x7f429001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.419+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4298010070 con 0x7f42a0109fa0 2024-04-25T13:41:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:28 smithi137 ceph-mon[32568]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:28.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.675+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f42a002e510 con 0x7f42a0109fa0 2024-04-25T13:41:28.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.675+0000 7f429cff9700 1 -- 172.21.15.94:0/1806218099 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3096 (secure 0 0 0) 0x7f42980a7420 con 0x7f42a0109fa0 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:e24 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 24 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T13:39:44.966108+0000 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-25T13:39:51.640274+0000 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-25T13:41:28.675 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 300 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 5 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:in 0,1,2,3,4 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24557,1=24512,2=24502,3=24488,4=24475} 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-25T13:41:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24557 members: 24488,24502,24475,24557,24512 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.f{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.138:6834/1621301649,v1:172.21.15.138:6835/1621301649] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.h{1:24512} state up:active seq 6 addr [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.k{1:24455} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/3830518540,v1:172.21.15.137:6833/3830518540] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.i{2:24502} state up:active seq 6 addr [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.l{2:24433} state up:standby-replay seq 1 addr [v2:172.21.15.138:6832/4012763239,v1:172.21.15.138:6833/4012763239] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.e{3:24488} state up:active seq 9 addr [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.677 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.94:6840/2301018309,v1:172.21.15.94:6841/2301018309] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c{4:24475} state up:active seq 7 addr [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.g{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.94:6838/3883623467,v1:172.21.15.94:6839/3883623467] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.94:6834/4024796429,v1:172.21.15.94:6835/4024796429] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.678 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.94:6836/2786321466,v1:172.21.15.94:6837/2786321466] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.677+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 msgr2=0x7f42880b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.677+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 0x7f42880b2110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4290006010 tx=0x7f429001a040 comp rx=0 tx=0).stop 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.677+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 msgr2=0x7f42a01a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.677+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4298007ee0 tx=0x7f429800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 shutdown_connections 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42880afc00 0x7f42880b2110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a0109fa0 0x7f42a01a1c70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a0104ce0 0x7f42a019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 --2- 172.21.15.94:0/1806218099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0107640 0x7f42a019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 >> 172.21.15.94:0/1806218099 conn(0x7f42a00fe740 msgr2=0x7f42a01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 shutdown_connections 2024-04-25T13:41:28.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:28.678+0000 7f42a61df700 1 -- 172.21.15.94:0/1806218099 wait complete. 2024-04-25T13:41:28.680 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:28 smithi138 ceph-mon[32977]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:29.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9021454-0307-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:41:29.620 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:29.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:29 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1806218099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:41:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:29 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1806218099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:41:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:29 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1806218099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:41:30.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:30 smithi137 ceph-mon[32568]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:30 smithi094 ceph-mon[29195]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:30 smithi138 ceph-mon[32977]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 -- 172.21.15.94:0/2402797190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 msgr2=0x7f25b8105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/2402797190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b8105c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009990 tx=0x7f25a8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 -- 172.21.15.94:0/2402797190 shutdown_connections 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/2402797190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 0x7f25b810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/2402797190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b8105c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/2402797190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 0x7f25b8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 -- 172.21.15.94:0/2402797190 >> 172.21.15.94:0/2402797190 conn(0x7f25b80ff870 msgr2=0x7f25b8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 -- 172.21.15.94:0/2402797190 shutdown_connections 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 -- 172.21.15.94:0/2402797190 wait complete. 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.264+0000 7f25bd3f8700 1 Processor -- start 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 -- start start 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 0x7f25b819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 0x7f25b819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810fd70 con 0x7f25b8105850 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810fbf0 con 0x7f25b8104880 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25bd3f8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810fef0 con 0x7f25b8103e00 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53924/0 (socket says 172.21.15.94:53924) 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 -- 172.21.15.94:0/4266007449 learned_addr learned my addr 172.21.15.94:0/4266007449 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b6ffd700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 0x7f25b819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b67fc700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 0x7f25b819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 -- 172.21.15.94:0/4266007449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 msgr2=0x7f25b819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 0x7f25b819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 -- 172.21.15.94:0/4266007449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 msgr2=0x7f25b819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 0x7f25b819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.265+0000 7f25b77fe700 1 -- 172.21.15.94:0/4266007449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b81a2410 con 0x7f25b8105850 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f25b77fe700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f25b000ada0 tx=0x7f25b000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0017020 con 0x7f25b8105850 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b81a2700 con 0x7f25b8105850 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b000ce30 con 0x7f25b8105850 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0005350 con 0x7f25b8105850 2024-04-25T13:41:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.266+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a2bf0 con 0x7f25b8105850 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.267+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b8051190 con 0x7f25b8105850 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.268+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f25b000c980 con 0x7f25b8105850 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.269+0000 7f259ffff700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 0x7f25a00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.269+0000 7f25b6ffd700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 0x7f25a00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.269+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f25b00de940 con 0x7f25b8105850 2024-04-25T13:41:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.269+0000 7f25b6ffd700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 0x7f25a00b2090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25a80038a0 tx=0x7f25a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.272+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25b0021020 con 0x7f25b8105850 2024-04-25T13:41:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.471+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f25b80642a0 con 0x7f25b8105850 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.472+0000 7f259ffff700 1 -- 172.21.15.94:0/4266007449 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5343 (secure 0 0 0) 0x7f25b00a7590 con 0x7f25b8105850 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 81 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:fsid f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T13:31:45.992646+0000 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-25T13:40:41.257024+0000 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 26 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-04-25T13:41:31.472 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client luminous 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 12 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6802/2637923455,v1:172.21.15.94:6803/2637923455] [v2:172.21.15.94:6804/2637923455,v1:172.21.15.94:6805/2637923455] exists,up 71e93888-dbdc-44cc-b0c5-bd211fc1fbed 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/1683710608,v1:172.21.15.137:6801/1683710608] [v2:172.21.15.137:6802/1683710608,v1:172.21.15.137:6803/1683710608] exists,up 814d5805-282b-4231-9570-1ff9e7e41fec 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/4136773447,v1:172.21.15.138:6801/4136773447] [v2:172.21.15.138:6802/4136773447,v1:172.21.15.138:6803/4136773447] exists,up 14d09b14-8c4c-48b2-b295-b6581d524d75 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 26 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6810/3561678692,v1:172.21.15.94:6811/3561678692] [v2:172.21.15.94:6812/3561678692,v1:172.21.15.94:6813/3561678692] exists,up 046f0d40-5505-42d2-961f-4cf5250b1eda 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 32 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/3127701916,v1:172.21.15.137:6809/3127701916] [v2:172.21.15.137:6810/3127701916,v1:172.21.15.137:6811/3127701916] exists,up 0d740f16-7559-45f6-bd33-f834183b06f3 2024-04-25T13:41:31.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 38 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/699823626,v1:172.21.15.138:6809/699823626] [v2:172.21.15.138:6810/699823626,v1:172.21.15.138:6811/699823626] exists,up eeda2336-9862-4bd4-b776-454204034ebb 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 44 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6818/1177969162,v1:172.21.15.94:6819/1177969162] [v2:172.21.15.94:6820/1177969162,v1:172.21.15.94:6821/1177969162] exists,up a884764f-45af-4c83-b7d3-65ebd314f012 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 49 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/1075953971,v1:172.21.15.137:6817/1075953971] [v2:172.21.15.137:6818/1075953971,v1:172.21.15.137:6819/1075953971] exists,up 73b89099-7f29-4114-b29f-c0b33124e04e 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 up in weight 1 up_from 56 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3084581661,v1:172.21.15.138:6817/3084581661] [v2:172.21.15.138:6818/3084581661,v1:172.21.15.138:6819/3084581661] exists,up 47e5d4bf-c849-4b0c-bb6b-f4c14675072a 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 up in weight 1 up_from 62 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6826/1188484868,v1:172.21.15.94:6827/1188484868] [v2:172.21.15.94:6828/1188484868,v1:172.21.15.94:6829/1188484868] exists,up d4cec5f6-fc9d-4166-9979-a7dad7ce3585 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 up in weight 1 up_from 68 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/3471315400,v1:172.21.15.137:6825/3471315400] [v2:172.21.15.137:6826/3471315400,v1:172.21.15.137:6827/3471315400] exists,up 110b4c5b-fa58-48a4-9567-c88bd908afa7 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 up in weight 1 up_from 74 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/2554799218,v1:172.21.15.138:6825/2554799218] [v2:172.21.15.138:6826/2554799218,v1:172.21.15.138:6827/2554799218] exists,up 3af033e4-3bf8-4a1d-a355-55edad2c8226 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.1a [8,1] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.e [4,8] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.1a [5,6] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T13:41:31.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.2a [3,8] 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.36 [4,9] 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.37 [5,6] 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/1021554684 expires 2024-04-26T13:32:40.167913+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1911980702 expires 2024-04-26T13:32:40.167913+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/1021554684 expires 2024-04-26T13:32:40.167913+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2063913042 expires 2024-04-26T13:32:40.167913+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2471492892 expires 2024-04-26T13:32:13.498920+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1740247749 expires 2024-04-26T13:32:13.498920+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/2945875633 expires 2024-04-26T13:32:13.498920+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/2945875633 expires 2024-04-26T13:32:13.498920+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/315514889 expires 2024-04-26T13:32:13.498920+0000 2024-04-25T13:41:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2862296884 expires 2024-04-26T13:32:40.167913+0000 2024-04-25T13:41:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 msgr2=0x7f25a00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 0x7f25a00b2090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25a80038a0 tx=0x7f25a801a040 comp rx=0 tx=0).stop 2024-04-25T13:41:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 msgr2=0x7f25b81a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f25b000ada0 tx=0x7f25b000c330 comp rx=0 tx=0).stop 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 shutdown_connections 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a00afb80 0x7f25a00b2090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b8105850 0x7f25b81a1d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103e00 0x7f25b819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 --2- 172.21.15.94:0/4266007449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8104880 0x7f25b819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 >> 172.21.15.94:0/4266007449 conn(0x7f25b80ff870 msgr2=0x7f25b8108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 shutdown_connections 2024-04-25T13:41:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:31.475+0000 7f25bd3f8700 1 -- 172.21.15.94:0/4266007449 wait complete. 2024-04-25T13:41:31.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:31 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4266007449' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:41:31.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:31 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4266007449' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:41:31.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:31 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4266007449' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:41:32.230 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:41:32.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4207539669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918108940 msgr2=0x7fc91810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4207539669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918108940 0x7fc91810ad60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc914009900 tx=0x7fc91402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4207539669 shutdown_connections 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4207539669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918108940 0x7fc91810ad60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4207539669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918103c40 0x7fc918104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4207539669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc918102a00 0x7fc918103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4207539669 >> 172.21.15.94:0/4207539669 conn(0x7fc9180fdbf0 msgr2=0x7fc9181000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4207539669 shutdown_connections 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.355+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4207539669 wait complete. 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 Processor -- start 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 -- start start 2024-04-25T13:41:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918102a00 0x7fc918075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc918108940 0x7fc9181a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc91810e530 con 0x7fc918103c40 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc91810e3b0 con 0x7fc918108940 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91ffc5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc91810e6b0 con 0x7fc918102a00 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53940/0 (socket says 172.21.15.94:53940) 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 -- 172.21.15.94:0/4099650716 learned_addr learned my addr 172.21.15.94:0/4099650716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91dd61640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918102a00 0x7fc918075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 -- 172.21.15.94:0/4099650716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918102a00 msgr2=0x7fc918075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918102a00 0x7fc918075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 -- 172.21.15.94:0/4099650716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc918108940 msgr2=0x7fc9181a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc918108940 0x7fc9181a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.356+0000 7fc91d560640 1 -- 172.21.15.94:0/4099650716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9181a1c90 con 0x7fc918103c40 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc91d560640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c970 tx=0x7fc90800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908007bb0 con 0x7fc918103c40 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc908007d50 con 0x7fc918103c40 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9181a1f80 con 0x7fc918103c40 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800f450 con 0x7fc918103c40 2024-04-25T13:41:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.357+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a23f0 con 0x7fc918103c40 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.358+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc918107750 con 0x7fc918103c40 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.359+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc908004f20 con 0x7fc918103c40 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.360+0000 7fc90effd640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 0x7fc9040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.360+0000 7fc91dd61640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 0x7fc9040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.360+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc9080dea80 con 0x7fc918103c40 2024-04-25T13:41:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.360+0000 7fc91dd61640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 0x7fc9040b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc918103600 tx=0x7fc910008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.364+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc908014030 con 0x7fc918103c40 2024-04-25T13:41:32.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.546+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9180637a0 con 0x7fc918103c40 2024-04-25T13:41:32.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.548+0000 7fc90effd640 1 -- 172.21.15.94:0/4099650716 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7fc9080a7640 con 0x7fc918103c40 2024-04-25T13:41:32.547 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 msgr2=0x7fc9040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 0x7fc9040b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc918103600 tx=0x7fc910008230 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 msgr2=0x7fc91819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c970 tx=0x7fc90800ce30 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 shutdown_connections 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9040afaf0 0x7fc9040b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc918108940 0x7fc9181a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc918103c40 0x7fc91819ef20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 --2- 172.21.15.94:0/4099650716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc918102a00 0x7fc918075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 >> 172.21.15.94:0/4099650716 conn(0x7fc9180fdbf0 msgr2=0x7fc918104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 shutdown_connections 2024-04-25T13:41:32.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.550+0000 7fc91ffc5640 1 -- 172.21.15.94:0/4099650716 wait complete. 2024-04-25T13:41:32.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:32.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 -- 172.21.15.94:0/2767835436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790108920 msgr2=0x7f279010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 --2- 172.21.15.94:0/2767835436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790108920 0x7f279010ad40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ca0 tx=0x7f278802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 -- 172.21.15.94:0/2767835436 shutdown_connections 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 --2- 172.21.15.94:0/2767835436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790108920 0x7f279010ad40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 --2- 172.21.15.94:0/2767835436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2790103c20 0x7f2790104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.673+0000 7f2796ad1640 1 --2- 172.21.15.94:0/2767835436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27901029e0 0x7f27901036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- 172.21.15.94:0/2767835436 >> 172.21.15.94:0/2767835436 conn(0x7f27900fdbd0 msgr2=0x7f2790100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- 172.21.15.94:0/2767835436 shutdown_connections 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- 172.21.15.94:0/2767835436 wait complete. 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 Processor -- start 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- start start 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27901029e0 0x7f2790199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2790108920 0x7f27901a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e510 con 0x7f2790103c20 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e390 con 0x7f27901029e0 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f2796ad1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e690 con 0x7f2790108920 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f278ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f278ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53966/0 (socket says 172.21.15.94:53966) 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.674+0000 7f278ffff640 1 -- 172.21.15.94:0/3989419774 learned_addr learned my addr 172.21.15.94:0/3989419774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f279486d640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27901029e0 0x7f2790199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 -- 172.21.15.94:0/3989419774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2790108920 msgr2=0x7f27901a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f279506e640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2790108920 0x7f27901a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2790108920 0x7f27901a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 -- 172.21.15.94:0/3989419774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27901029e0 msgr2=0x7f2790199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27901029e0 0x7f2790199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 -- 172.21.15.94:0/3989419774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27901a0f50 con 0x7f2790103c20 2024-04-25T13:41:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278ffff640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f278400c890 tx=0x7f278400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784004220 con 0x7f2790103c20 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27840043c0 con 0x7f2790103c20 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784004a00 con 0x7f2790103c20 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901a1240 con 0x7f2790103c20 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.675+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a17e0 con 0x7f2790103c20 2024-04-25T13:41:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.676+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2790107730 con 0x7f2790103c20 2024-04-25T13:41:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.678+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f27840079b0 con 0x7f2790103c20 2024-04-25T13:41:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.678+0000 7f278dffb640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 0x7f27780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.678+0000 7f279486d640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 0x7f27780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.678+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f27840de9e0 con 0x7f2790103c20 2024-04-25T13:41:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.679+0000 7f279486d640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 0x7f27780b2080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2780005fd0 tx=0x7f2780005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.683+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2784014030 con 0x7f2790103c20 2024-04-25T13:41:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.863+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f279002dd30 con 0x7f2790103c20 2024-04-25T13:41:32.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.863+0000 7f278dffb640 1 -- 172.21.15.94:0/3989419774 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f27840a75a0 con 0x7f2790103c20 2024-04-25T13:41:32.863 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 msgr2=0x7f27780b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 0x7f27780b2080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2780005fd0 tx=0x7f2780005850 comp rx=0 tx=0).stop 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 msgr2=0x7f2790199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f278400c890 tx=0x7f278400cd50 comp rx=0 tx=0).stop 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 shutdown_connections 2024-04-25T13:41:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27780afb70 0x7f27780b2080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2790108920 0x7f27901a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2790103c20 0x7f2790199950 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 --2- 172.21.15.94:0/3989419774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27901029e0 0x7f2790199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 >> 172.21.15.94:0/3989419774 conn(0x7f27900fdbd0 msgr2=0x7f2790104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 shutdown_connections 2024-04-25T13:41:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.866+0000 7f2796ad1640 1 -- 172.21.15.94:0/3989419774 wait complete. 2024-04-25T13:41:32.875 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:32 smithi094 ceph-mon[29195]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:32.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:32.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:32 smithi137 ceph-mon[32568]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 -- 172.21.15.94:0/4069407620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 msgr2=0x7f69781054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 --2- 172.21.15.94:0/4069407620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781054c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6968009900 tx=0x7f696802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 -- 172.21.15.94:0/4069407620 shutdown_connections 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 --2- 172.21.15.94:0/4069407620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978102260 0x7f697810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 --2- 172.21.15.94:0/4069407620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978101760 0x7f6978101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 --2- 172.21.15.94:0/4069407620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781054c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 -- 172.21.15.94:0/4069407620 >> 172.21.15.94:0/4069407620 conn(0x7f69780fd380 msgr2=0x7f69780ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 -- 172.21.15.94:0/4069407620 shutdown_connections 2024-04-25T13:41:32.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 -- 172.21.15.94:0/4069407620 wait complete. 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.987+0000 7f697e13f640 1 Processor -- start 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 -- start start 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978101760 0x7f697819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978102260 0x7f697819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f697810f620 con 0x7f69781050b0 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f697810f4a0 con 0x7f6978102260 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f697e13f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f697810f7a0 con 0x7f6978101760 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53980/0 (socket says 172.21.15.94:53980) 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 -- 172.21.15.94:0/1735554988 learned_addr learned my addr 172.21.15.94:0/1735554988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:32.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f69777fe640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978101760 0x7f697819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6976ffd640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978102260 0x7f697819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 -- 172.21.15.94:0/1735554988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978101760 msgr2=0x7f697819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978101760 0x7f697819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 -- 172.21.15.94:0/1735554988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978102260 msgr2=0x7f697819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978102260 0x7f697819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.988+0000 7f6977fff640 1 -- 172.21.15.94:0/1735554988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781a1d60 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f6977fff640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f697000eaa0 tx=0x7f697000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cdb0 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970005020 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69781a1ff0 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700116d0 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.989+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69781a24b0 con 0x7f69781050b0 2024-04-25T13:41:32.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.990+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6978108a40 con 0x7f69781050b0 2024-04-25T13:41:32.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.991+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f69700051d0 con 0x7f69781050b0 2024-04-25T13:41:32.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.992+0000 7f6974ff9640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 0x7f69600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.992+0000 7f69777fe640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 0x7f69600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.992+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f69700df120 con 0x7f69781050b0 2024-04-25T13:41:32.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.992+0000 7f69777fe640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 0x7f69600b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6968005ab0 tx=0x7f6968031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:32.996+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6970016030 con 0x7f69781050b0 2024-04-25T13:41:32.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:32 smithi138 ceph-mon[32977]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T13:41:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.175+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69780637a0 con 0x7f69781050b0 2024-04-25T13:41:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.175+0000 7f6974ff9640 1 -- 172.21.15.94:0/1735554988 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f69700a7ce0 con 0x7f69781050b0 2024-04-25T13:41:33.175 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 msgr2=0x7f69600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 0x7f69600b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6968005ab0 tx=0x7f6968031400 comp rx=0 tx=0).stop 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 msgr2=0x7f69781a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f697000eaa0 tx=0x7f697000ef60 comp rx=0 tx=0).stop 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 shutdown_connections 2024-04-25T13:41:33.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69600afaf0 0x7f69600b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69781050b0 0x7f69781a16f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6978102260 0x7f697819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 --2- 172.21.15.94:0/1735554988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6978101760 0x7f697819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 >> 172.21.15.94:0/1735554988 conn(0x7f69780fd380 msgr2=0x7f6978105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 shutdown_connections 2024-04-25T13:41:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.178+0000 7f697e13f640 1 -- 172.21.15.94:0/1735554988 wait complete. 2024-04-25T13:41:33.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.299+0000 7f69b3bc7640 1 -- 172.21.15.94:0/2926307081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1045e0 msgr2=0x7f69ac106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.299+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/2926307081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1045e0 0x7f69ac106a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009ca0 tx=0x7f69a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- 172.21.15.94:0/2926307081 shutdown_connections 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/2926307081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1098a0 0x7f69ac10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/2926307081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/2926307081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1045e0 0x7f69ac106a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- 172.21.15.94:0/2926307081 >> 172.21.15.94:0/2926307081 conn(0x7f69ac0fdbe0 msgr2=0x7f69ac100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- 172.21.15.94:0/2926307081 shutdown_connections 2024-04-25T13:41:33.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- 172.21.15.94:0/2926307081 wait complete. 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 Processor -- start 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- start start 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 0x7f69ac19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10f640 con 0x7f69ac1098a0 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10f4c0 con 0x7f69ac1045e0 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.300+0000 7f69b3bc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac10f7c0 con 0x7f69ac106f40 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54004/0 (socket says 172.21.15.94:54004) 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 -- 172.21.15.94:0/3631238183 learned_addr learned my addr 172.21.15.94:0/3631238183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b1963640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 0x7f69ac19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b1162640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 -- 172.21.15.94:0/3631238183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 msgr2=0x7f69ac19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 -- 172.21.15.94:0/3631238183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 msgr2=0x7f69ac19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 0x7f69ac19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 -- 172.21.15.94:0/3631238183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac1a1df0 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b1963640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 0x7f69ac19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b1162640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b2164640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d950 tx=0x7f69a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4017070 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a4009c30 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1a20e0 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400f620 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.301+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1a25e0 con 0x7f69ac1098a0 2024-04-25T13:41:33.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.302+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac0ff700 con 0x7f69ac1098a0 2024-04-25T13:41:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.304+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f69a401a070 con 0x7f69ac1098a0 2024-04-25T13:41:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.304+0000 7f69a2ffd640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 0x7f69980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.304+0000 7f69b1963640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 0x7f69980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.304+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f69a40de720 con 0x7f69ac1098a0 2024-04-25T13:41:33.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.305+0000 7f69b1963640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 0x7f69980b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69a8000f80 tx=0x7f69a8031b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.308+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69a40a72e0 con 0x7f69ac1098a0 2024-04-25T13:41:33.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.489+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f69ac0637a0 con 0x7f69ac1098a0 2024-04-25T13:41:33.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.489+0000 7f69a2ffd640 1 -- 172.21.15.94:0/3631238183 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f69a40a6be0 con 0x7f69ac1098a0 2024-04-25T13:41:33.488 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 msgr2=0x7f69980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 0x7f69980b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69a8000f80 tx=0x7f69a8031b10 comp rx=0 tx=0).stop 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 msgr2=0x7f69ac1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d950 tx=0x7f69a400de10 comp rx=0 tx=0).stop 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 shutdown_connections 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69980afaf0 0x7f69980b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69ac1098a0 0x7f69ac1a1780 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69ac106f40 0x7f69ac19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 --2- 172.21.15.94:0/3631238183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69ac1045e0 0x7f69ac19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 >> 172.21.15.94:0/3631238183 conn(0x7f69ac0fdbe0 msgr2=0x7f69ac106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 shutdown_connections 2024-04-25T13:41:33.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.491+0000 7f69b3bc7640 1 -- 172.21.15.94:0/3631238183 wait complete. 2024-04-25T13:41:33.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 -- 172.21.15.94:0/869735465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1098b0 msgr2=0x7f22dc10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/869735465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1098b0 0x7f22dc10bcd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ca0 tx=0x7f22d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 -- 172.21.15.94:0/869735465 shutdown_connections 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/869735465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1098b0 0x7f22dc10bcd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/869735465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/869735465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1045f0 0x7f22dc106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.611+0000 7f22e3d9e640 1 -- 172.21.15.94:0/869735465 >> 172.21.15.94:0/869735465 conn(0x7f22dc0fdbf0 msgr2=0x7f22dc1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- 172.21.15.94:0/869735465 shutdown_connections 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- 172.21.15.94:0/869735465 wait complete. 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 Processor -- start 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- start start 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 0x7f22dc1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10f700 con 0x7f22dc1045f0 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10f580 con 0x7f22dc1098b0 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e3d9e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10f880 con 0x7f22dc106f50 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e1b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e1b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34588/0 (socket says 172.21.15.94:34588) 2024-04-25T13:41:33.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.612+0000 7f22e1b3a640 1 -- 172.21.15.94:0/1979318962 learned_addr learned my addr 172.21.15.94:0/1979318962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e233b640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 0x7f22dc1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 -- 172.21.15.94:0/1979318962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 msgr2=0x7f22dc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1339640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 -- 172.21.15.94:0/1979318962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 msgr2=0x7f22dc1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 0x7f22dc1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 -- 172.21.15.94:0/1979318962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1a1e20 con 0x7f22dc1045f0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e233b640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 0x7f22dc1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1339640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e1b3a640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22d800b450 tx=0x7f22d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8015020 con 0x7f22dc1045f0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a2110 con 0x7f22dc1045f0 2024-04-25T13:41:33.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d8011d40 con 0x7f22dc1045f0 2024-04-25T13:41:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a2730 con 0x7f22dc1045f0 2024-04-25T13:41:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.613+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d800f340 con 0x7f22dc1045f0 2024-04-25T13:41:33.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.614+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0ff710 con 0x7f22dc1045f0 2024-04-25T13:41:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.616+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f22d8018020 con 0x7f22dc1045f0 2024-04-25T13:41:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.616+0000 7f22d2ffd640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 0x7f22c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.616+0000 7f22e1339640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 0x7f22c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.616+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f22d80ded20 con 0x7f22dc1045f0 2024-04-25T13:41:33.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.617+0000 7f22e1339640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 0x7f22c80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.620+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22d801f020 con 0x7f22dc1045f0 2024-04-25T13:41:33.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.837+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22dc02db50 con 0x7f22dc1045f0 2024-04-25T13:41:33.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.838+0000 7f22d2ffd640 1 -- 172.21.15.94:0/1979318962 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7f22d800d040 con 0x7f22dc1045f0 2024-04-25T13:41:33.837 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:33.838 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:41:33.838 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 msgr2=0x7f22c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 0x7f22c80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006fd0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 msgr2=0x7f22dc19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22d800b450 tx=0x7f22d800b910 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 shutdown_connections 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22c80afaf0 0x7f22c80b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22dc1098b0 0x7f22dc1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc106f50 0x7f22dc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 --2- 172.21.15.94:0/1979318962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc1045f0 0x7f22dc19a2e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 >> 172.21.15.94:0/1979318962 conn(0x7f22dc0fdbf0 msgr2=0x7f22dc105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 shutdown_connections 2024-04-25T13:41:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.840+0000 7f22e3d9e640 1 -- 172.21.15.94:0/1979318962 wait complete. 2024-04-25T13:41:33.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4099650716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:33.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3989419774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1735554988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3631238183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:33.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:33 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4099650716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:33.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:33 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3989419774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:33 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1735554988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:33 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3631238183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 -- 172.21.15.94:0/1862787331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c103b30 msgr2=0x7f5e0c104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 --2- 172.21.15.94:0/1862787331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c103b30 0x7f5e0c104000 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009900 tx=0x7f5dfc02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 -- 172.21.15.94:0/1862787331 shutdown_connections 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 --2- 172.21.15.94:0/1862787331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c104690 0x7f5e0c10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 --2- 172.21.15.94:0/1862787331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c103b30 0x7f5e0c104000 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 --2- 172.21.15.94:0/1862787331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 -- 172.21.15.94:0/1862787331 >> 172.21.15.94:0/1862787331 conn(0x7f5e0c0fd370 msgr2=0x7f5e0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 -- 172.21.15.94:0/1862787331 shutdown_connections 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.960+0000 7f5e13834640 1 -- 172.21.15.94:0/1862787331 wait complete. 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 Processor -- start 2024-04-25T13:41:33.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 -- start start 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c103b30 0x7f5e0c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10f860 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10f6e0 con 0x7f5e0c102fd0 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e13834640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10f9e0 con 0x7f5e0c103b30 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34610/0 (socket says 172.21.15.94:34610) 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 -- 172.21.15.94:0/1809483686 learned_addr learned my addr 172.21.15.94:0/1809483686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e115d0640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 -- 172.21.15.94:0/1809483686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c103b30 msgr2=0x7f5e0c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c103b30 0x7f5e0c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 -- 172.21.15.94:0/1809483686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 msgr2=0x7f5e0c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e11dd1640 1 -- 172.21.15.94:0/1809483686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a1f60 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.961+0000 7f5e115d0640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e11dd1640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400eaa0 tx=0x7f5e0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400cdb0 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e04005020 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a2250 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e040116d0 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.962+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a2750 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.963+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c06b3a0 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.964+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5e040051d0 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.965+0000 7f5e027fc640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 0x7f5df80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.965+0000 7f5e115d0640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 0x7f5df80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.965+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f5e040df280 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.965+0000 7f5e115d0640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 0x7f5df80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5e080058c0 tx=0x7f5e08005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:33.969+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e04016030 con 0x7f5e0c19f4d0 2024-04-25T13:41:33.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4099650716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:33.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3989419774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1735554988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:33.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3631238183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:34.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.148+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e0c02dda0 con 0x7f5e0c19f4d0 2024-04-25T13:41:34.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.149+0000 7f5e027fc640 1 -- 172.21.15.94:0/1809483686 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7f5e040a7e40 con 0x7f5e0c19f4d0 2024-04-25T13:41:34.149 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:34.149 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 msgr2=0x7f5df80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 0x7f5df80b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5e080058c0 tx=0x7f5e08005d30 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 msgr2=0x7f5e0c1a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400eaa0 tx=0x7f5e0400ef60 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 shutdown_connections 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5df80afaf0 0x7f5df80b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e0c19f4d0 0x7f5e0c1a18f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c103b30 0x7f5e0c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 --2- 172.21.15.94:0/1809483686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e0c102fd0 0x7f5e0c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 >> 172.21.15.94:0/1809483686 conn(0x7f5e0c0fd370 msgr2=0x7f5e0c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 shutdown_connections 2024-04-25T13:41:34.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.151+0000 7f5e13834640 1 -- 172.21.15.94:0/1809483686 wait complete. 2024-04-25T13:41:34.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:34.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:34.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 -- 172.21.15.94:0/2307494461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818108920 msgr2=0x7f081810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 --2- 172.21.15.94:0/2307494461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818108920 0x7f081810ad40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 -- 172.21.15.94:0/2307494461 shutdown_connections 2024-04-25T13:41:34.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 --2- 172.21.15.94:0/2307494461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818108920 0x7f081810ad40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 --2- 172.21.15.94:0/2307494461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818103c20 0x7f0818104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 --2- 172.21.15.94:0/2307494461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08181029e0 0x7f08181036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 -- 172.21.15.94:0/2307494461 >> 172.21.15.94:0/2307494461 conn(0x7f08180fdbd0 msgr2=0x7f0818100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 -- 172.21.15.94:0/2307494461 shutdown_connections 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.274+0000 7f081d5c7640 1 -- 172.21.15.94:0/2307494461 wait complete. 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 Processor -- start 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 -- start start 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08181029e0 0x7f0818075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 0x7f08181a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e510 con 0x7f0818103c20 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e390 con 0x7f08181029e0 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f081d5c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e690 con 0x7f0818108920 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34638/0 (socket says 172.21.15.94:34638) 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f080ffff640 1 -- 172.21.15.94:0/190449641 learned_addr learned my addr 172.21.15.94:0/190449641 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.275+0000 7f0816ffd640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08181029e0 0x7f0818075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f08177fe640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 0x7f08181a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 -- 172.21.15.94:0/190449641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 msgr2=0x7f08181a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 0x7f08181a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 -- 172.21.15.94:0/190449641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08181029e0 msgr2=0x7f0818075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08181029e0 0x7f0818075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 -- 172.21.15.94:0/190449641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a1d90 con 0x7f0818103c20 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f08177fe640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 0x7f08181a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f080ffff640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f080000d950 tx=0x7f080000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800017070 con 0x7f0818103c20 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0800009c30 con 0x7f0818103c20 2024-04-25T13:41:34.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a2080 con 0x7f0818103c20 2024-04-25T13:41:34.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080000f620 con 0x7f0818103c20 2024-04-25T13:41:34.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.276+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a2620 con 0x7f0818103c20 2024-04-25T13:41:34.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.277+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0818107730 con 0x7f0818103c20 2024-04-25T13:41:34.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.278+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0800010460 con 0x7f0818103c20 2024-04-25T13:41:34.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.279+0000 7f0814ff9640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 0x7f08040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.279+0000 7f0816ffd640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 0x7f08040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.279+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f08000de7a0 con 0x7f0818103c20 2024-04-25T13:41:34.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.279+0000 7f0816ffd640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 0x7f08040b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f080800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.283+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08000a73e0 con 0x7f0818103c20 2024-04-25T13:41:34.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.462+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08180637a0 con 0x7f0818103c20 2024-04-25T13:41:34.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.462+0000 7f0814ff9640 1 -- 172.21.15.94:0/190449641 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f0800009de0 con 0x7f0818103c20 2024-04-25T13:41:34.462 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 msgr2=0x7f08040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 0x7f08040b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f080800b390 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 msgr2=0x7f081819f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f080000d950 tx=0x7f080000de10 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 shutdown_connections 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afb70 0x7f08040b2080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818108920 0x7f08181a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818103c20 0x7f081819f000 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 --2- 172.21.15.94:0/190449641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08181029e0 0x7f0818075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 >> 172.21.15.94:0/190449641 conn(0x7f08180fdbd0 msgr2=0x7f0818104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 shutdown_connections 2024-04-25T13:41:34.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.464+0000 7f081d5c7640 1 -- 172.21.15.94:0/190449641 wait complete. 2024-04-25T13:41:34.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.587+0000 7fc8c897a640 1 -- 172.21.15.94:0/3993227059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c4109890 msgr2=0x7fc8c410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.587+0000 7fc8c897a640 1 --2- 172.21.15.94:0/3993227059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c4109890 0x7fc8c410bcb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009900 tx=0x7fc8bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 -- 172.21.15.94:0/3993227059 shutdown_connections 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 --2- 172.21.15.94:0/3993227059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c4109890 0x7fc8c410bcb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 --2- 172.21.15.94:0/3993227059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4106f30 0x7fc8c4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 --2- 172.21.15.94:0/3993227059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c41045d0 0x7fc8c41069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 -- 172.21.15.94:0/3993227059 >> 172.21.15.94:0/3993227059 conn(0x7fc8c40fdbd0 msgr2=0x7fc8c4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 -- 172.21.15.94:0/3993227059 shutdown_connections 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 -- 172.21.15.94:0/3993227059 wait complete. 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 Processor -- start 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.588+0000 7fc8c897a640 1 -- start start 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c4106f30 0x7fc8c419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 0x7fc8c41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c410f610 con 0x7fc8c41045d0 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c410f490 con 0x7fc8c4109890 2024-04-25T13:41:34.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c410f790 con 0x7fc8c4106f30 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34656/0 (socket says 172.21.15.94:34656) 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 -- 172.21.15.94:0/2301241799 learned_addr learned my addr 172.21.15.94:0/2301241799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c2d9d640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 0x7fc8c41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 -- 172.21.15.94:0/2301241799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c4106f30 msgr2=0x7fc8c419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c1d9b640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c4106f30 0x7fc8c419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c4106f30 0x7fc8c419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 -- 172.21.15.94:0/2301241799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 msgr2=0x7fc8c41a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 0x7fc8c41a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 -- 172.21.15.94:0/2301241799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c41a2080 con 0x7fc8c41045d0 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c2d9d640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 0x7fc8c41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c259c640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400cce0 tx=0x7fc8b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004b20 con 0x7fc8c41045d0 2024-04-25T13:41:34.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b4004cc0 con 0x7fc8c41045d0 2024-04-25T13:41:34.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c41a2370 con 0x7fc8c41045d0 2024-04-25T13:41:34.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.589+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b400f6b0 con 0x7fc8c41045d0 2024-04-25T13:41:34.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.590+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c41a2870 con 0x7fc8c41045d0 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.590+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c406c2a0 con 0x7fc8c41045d0 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.593+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc8b400f870 con 0x7fc8c41045d0 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.593+0000 7fc8b37fe640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 0x7fc8ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.593+0000 7fc8c1d9b640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 0x7fc8ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.593+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc8b40df190 con 0x7fc8c41045d0 2024-04-25T13:41:34.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.594+0000 7fc8c1d9b640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 0x7fc8ac0b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8005fd0 tx=0x7fc8b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.597+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8b4014030 con 0x7fc8c41045d0 2024-04-25T13:41:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1979318962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1809483686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:34 smithi094 ceph-mon[29195]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/190449641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:34.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.776+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8c41a2b30 con 0x7fc8c41045d0 2024-04-25T13:41:34.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.777+0000 7fc8b37fe640 1 -- 172.21.15.94:0/2301241799 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fc8b40a7d50 con 0x7fc8c41045d0 2024-04-25T13:41:34.776 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:34.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 msgr2=0x7fc8ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 0x7fc8ac0b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8005fd0 tx=0x7fc8b8005850 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 msgr2=0x7fc8c419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400cce0 tx=0x7fc8b400a620 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 shutdown_connections 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8ac0afaf0 0x7fc8ac0b2000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c4109890 0x7fc8c41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c4106f30 0x7fc8c419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 --2- 172.21.15.94:0/2301241799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8c41045d0 0x7fc8c419a540 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 >> 172.21.15.94:0/2301241799 conn(0x7fc8c40fdbd0 msgr2=0x7fc8c4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 shutdown_connections 2024-04-25T13:41:34.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.779+0000 7fc8c897a640 1 -- 172.21.15.94:0/2301241799 wait complete. 2024-04-25T13:41:34.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1979318962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1809483686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:34 smithi137 ceph-mon[32568]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:34.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/190449641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 -- 172.21.15.94:0/2269688438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0103af0 msgr2=0x7f14d01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 --2- 172.21.15.94:0/2269688438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0103af0 0x7f14d01047f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009900 tx=0x7f14c802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 -- 172.21.15.94:0/2269688438 shutdown_connections 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 --2- 172.21.15.94:0/2269688438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0109a30 0x7f14d010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 --2- 172.21.15.94:0/2269688438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 --2- 172.21.15.94:0/2269688438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0103af0 0x7f14d01047f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 -- 172.21.15.94:0/2269688438 >> 172.21.15.94:0/2269688438 conn(0x7f14d00fece0 msgr2=0x7f14d0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 -- 172.21.15.94:0/2269688438 shutdown_connections 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.900+0000 7f14d59ee640 1 -- 172.21.15.94:0/2269688438 wait complete. 2024-04-25T13:41:34.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 Processor -- start 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 -- start start 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 0x7f14d019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010f650 con 0x7f14d0109a30 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010f4d0 con 0x7f14d0103af0 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14d59ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010f7d0 con 0x7f14d0104d30 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34676/0 (socket says 172.21.15.94:34676) 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14cf7fe640 1 -- 172.21.15.94:0/4097598434 learned_addr learned my addr 172.21.15.94:0/4097598434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.901+0000 7f14ceffd640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 0x7f14d019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14ce7fc640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 -- 172.21.15.94:0/4097598434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 msgr2=0x7f14d019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 -- 172.21.15.94:0/4097598434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 msgr2=0x7f14d019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 0x7f14d019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 -- 172.21.15.94:0/4097598434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01a1df0 con 0x7f14d0109a30 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14ceffd640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 0x7f14d019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14ce7fc640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14cf7fe640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d750 tx=0x7f14c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0017070 con 0x7f14d0109a30 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c00054d0 con 0x7f14d0109a30 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01a20e0 con 0x7f14d0109a30 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004c60 con 0x7f14d0109a30 2024-04-25T13:41:34.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.902+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01a2660 con 0x7f14d0109a30 2024-04-25T13:41:34.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.903+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d0108840 con 0x7f14d0109a30 2024-04-25T13:41:34.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.904+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f14c0009bd0 con 0x7f14d0109a30 2024-04-25T13:41:34.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.905+0000 7f14d49ec640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 0x7f14b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.905+0000 7f14ceffd640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 0x7f14b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.905+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f14c00dead0 con 0x7f14d0109a30 2024-04-25T13:41:34.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.906+0000 7f14ceffd640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 0x7f14b80b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009230 tx=0x7f14c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:34.909+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14c00a7690 con 0x7f14d0109a30 2024-04-25T13:41:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1979318962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1809483686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:34 smithi138 ceph-mon[32977]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:41:34.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/190449641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.086+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14d00637a0 con 0x7f14d0109a30 2024-04-25T13:41:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.086+0000 7f14d49ec640 1 -- 172.21.15.94:0/4097598434 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f14c00a6f90 con 0x7f14d0109a30 2024-04-25T13:41:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 msgr2=0x7f14b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 0x7f14b80b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009230 tx=0x7f14c8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 msgr2=0x7f14d01a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d750 tx=0x7f14c000dc10 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 shutdown_connections 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14b80afaf0 0x7f14b80b2000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14d0109a30 0x7f14d01a1780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14d0104d30 0x7f14d019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 --2- 172.21.15.94:0/4097598434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0103af0 0x7f14d019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 >> 172.21.15.94:0/4097598434 conn(0x7f14d00fece0 msgr2=0x7f14d0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 shutdown_connections 2024-04-25T13:41:35.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.088+0000 7f14d59ee640 1 -- 172.21.15.94:0/4097598434 wait complete. 2024-04-25T13:41:35.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 -- 172.21.15.94:0/1282241889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c106f30 msgr2=0x7f412c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 --2- 172.21.15.94:0/1282241889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c106f30 0x7f412c109350 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4120009900 tx=0x7f412002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 -- 172.21.15.94:0/1282241889 shutdown_connections 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 --2- 172.21.15.94:0/1282241889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c109890 0x7f412c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 --2- 172.21.15.94:0/1282241889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c106f30 0x7f412c109350 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 --2- 172.21.15.94:0/1282241889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c1045d0 0x7f412c1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 -- 172.21.15.94:0/1282241889 >> 172.21.15.94:0/1282241889 conn(0x7f412c0fdbd0 msgr2=0x7f412c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 -- 172.21.15.94:0/1282241889 shutdown_connections 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.207+0000 7f41313a8640 1 -- 172.21.15.94:0/1282241889 wait complete. 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 Processor -- start 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 -- start start 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c106f30 0x7f412c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c109890 0x7f412c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10f610 con 0x7f412c1045d0 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10f490 con 0x7f412c109890 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f41313a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10f790 con 0x7f412c106f30 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.208+0000 7f412affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34696/0 (socket says 172.21.15.94:34696) 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 -- 172.21.15.94:0/865429620 learned_addr learned my addr 172.21.15.94:0/865429620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412b7fe640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c109890 0x7f412c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412a7fc640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c106f30 0x7f412c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 -- 172.21.15.94:0/865429620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c106f30 msgr2=0x7f412c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c106f30 0x7f412c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 -- 172.21.15.94:0/865429620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c109890 msgr2=0x7f412c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c109890 0x7f412c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 -- 172.21.15.94:0/865429620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c1a1de0 con 0x7f412c1045d0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f412affd640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e9c0 tx=0x7f411c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c00cd80 con 0x7f412c1045d0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c0054d0 con 0x7f412c1045d0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c1a20d0 con 0x7f412c1045d0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c010430 con 0x7f412c1045d0 2024-04-25T13:41:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.209+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1a2620 con 0x7f412c1045d0 2024-04-25T13:41:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.210+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c0ff6f0 con 0x7f412c1045d0 2024-04-25T13:41:35.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.212+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f411c004fb0 con 0x7f412c1045d0 2024-04-25T13:41:35.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.212+0000 7f4113fff640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 0x7f41140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.213+0000 7f412a7fc640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 0x7f41140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.213+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f411c0df380 con 0x7f412c1045d0 2024-04-25T13:41:35.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.213+0000 7f412a7fc640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 0x7f41140b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4120031500 tx=0x7f4120000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.217+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f411c053ff0 con 0x7f412c1045d0 2024-04-25T13:41:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.432+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f412c02db50 con 0x7f412c1045d0 2024-04-25T13:41:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.432+0000 7f4113fff640 1 -- 172.21.15.94:0/865429620 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11941 (secure 0 0 0) 0x7f411c0a7f40 con 0x7f412c1045d0 2024-04-25T13:41:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:39:51.640274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:41:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 msgr2=0x7f41140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 0x7f41140b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4120031500 tx=0x7f4120000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 msgr2=0x7f412c19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e9c0 tx=0x7f411c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 shutdown_connections 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f41140afaf0 0x7f41140b2000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f412c109890 0x7f412c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f412c106f30 0x7f412c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 --2- 172.21.15.94:0/865429620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f412c1045d0 0x7f412c19a2a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.434+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 >> 172.21.15.94:0/865429620 conn(0x7f412c0fdbd0 msgr2=0x7f412c106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.435+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 shutdown_connections 2024-04-25T13:41:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.435+0000 7f41313a8640 1 -- 172.21.15.94:0/865429620 wait complete. 2024-04-25T13:41:35.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 -- 172.21.15.94:0/3904475691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 msgr2=0x7f101c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 --2- 172.21.15.94:0/3904475691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c101bc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f100c009900 tx=0x7f100c02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 -- 172.21.15.94:0/3904475691 shutdown_connections 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 --2- 172.21.15.94:0/3904475691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 --2- 172.21.15.94:0/3904475691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c101bc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 --2- 172.21.15.94:0/3904475691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 0x7f101c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 -- 172.21.15.94:0/3904475691 >> 172.21.15.94:0/3904475691 conn(0x7f101c0fd370 msgr2=0x7f101c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 -- 172.21.15.94:0/3904475691 shutdown_connections 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 -- 172.21.15.94:0/3904475691 wait complete. 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.554+0000 7f10241b2640 1 Processor -- start 2024-04-25T13:41:35.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 -- start start 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 0x7f101c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10f630 con 0x7f101c101750 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10f4b0 con 0x7f101c1050a0 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f101c10f7b0 con 0x7f101c102250 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34720/0 (socket says 172.21.15.94:34720) 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 -- 172.21.15.94:0/502326560 learned_addr learned my addr 172.21.15.94:0/502326560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f102174d640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f102274f640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 0x7f101c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 -- 172.21.15.94:0/502326560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 msgr2=0x7f101c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 -- 172.21.15.94:0/502326560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 msgr2=0x7f101c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 0x7f101c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 -- 172.21.15.94:0/502326560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c1a1c70 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f102174d640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1021f4e640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f101800c970 tx=0x7f101800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018007c70 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1018007e10 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.555+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c1a1f60 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.556+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180187b0 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.556+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c1a23c0 con 0x7f101c101750 2024-04-25T13:41:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.557+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c06a190 con 0x7f101c101750 2024-04-25T13:41:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.558+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f101800fab0 con 0x7f101c101750 2024-04-25T13:41:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.559+0000 7f1012ffd640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 0x7f10080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.559+0000 7f102174d640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 0x7f10080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.559+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f10180deef0 con 0x7f101c101750 2024-04-25T13:41:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.559+0000 7f102174d640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 0x7f10080b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.563+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1018053ff0 con 0x7f101c101750 2024-04-25T13:41:35.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2301241799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4097598434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/865429620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:35.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.745+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f101c02dc10 con 0x7f101c101750 2024-04-25T13:41:35.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.747+0000 7f1012ffd640 1 -- 172.21.15.94:0/502326560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22770 (secure 0 0 0) 0x7f1018014030 con 0x7f101c101750 2024-04-25T13:41:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:35.747 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:40:41.257024+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 msgr2=0x7f10080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 0x7f10080b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c005d50 comp rx=0 tx=0).stop 2024-04-25T13:41:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 msgr2=0x7f101c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f101800c970 tx=0x7f101800ce30 comp rx=0 tx=0).stop 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 shutdown_connections 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f10080afaf0 0x7f10080b2000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f101c1050a0 0x7f101c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c102250 0x7f101c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 --2- 172.21.15.94:0/502326560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f101c101750 0x7f101c19a1e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 >> 172.21.15.94:0/502326560 conn(0x7f101c0fd370 msgr2=0x7f101c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 shutdown_connections 2024-04-25T13:41:35.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.749+0000 7f10241b2640 1 -- 172.21.15.94:0/502326560 wait complete. 2024-04-25T13:41:35.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:39:44.966108+0000', 'modified': '2024-04-25T13:39:51.640274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24512, 'mds_2': 24502, 'mds_3': 24488, 'mds_4': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/3883623467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 3883623467}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 3883623467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6841/2301018309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6840', 'nonce': 2301018309}, {'type': 'v1', 'addr': '172.21.15.94:6841', 'nonce': 2301018309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/4012763239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 4012763239}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 4012763239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/3830518540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 3830518540}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 3830518540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/1621301649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1621301649}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 1621301649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.138:6837/356914816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 356914816}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 356914816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.137:6835/3929131029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3929131029}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3929131029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.138:6839/2137998525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2137998525}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2137998525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/1274977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 1274977027}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 1274977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/3143839143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3143839143}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3143839143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24488, 24502, 24475, 24557, 24512]} 2024-04-25T13:41:35.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T13:41:35.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/642939255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8103c20 msgr2=0x7ff6f8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/642939255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8103c20 0x7ff6f8104030 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009900 tx=0x7ff6ec02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/642939255 shutdown_connections 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/642939255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f8108920 0x7ff6f810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/642939255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8103c20 0x7ff6f8104030 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/642939255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f81029e0 0x7ff6f81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/642939255 >> 172.21.15.94:0/642939255 conn(0x7ff6f80fdbd0 msgr2=0x7ff6f8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/642939255 shutdown_connections 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.870+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/642939255 wait complete. 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 Processor -- start 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 -- start start 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 0x7ff6f819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 0x7ff6f819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f810f610 con 0x7ff6f8108920 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f810f490 con 0x7ff6f8103c20 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fe7a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f810f790 con 0x7ff6f81029e0 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fcd45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fcd45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34732/0 (socket says 172.21.15.94:34732) 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6fcd45640 1 -- 172.21.15.94:0/1384383357 learned_addr learned my addr 172.21.15.94:0/1384383357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.871+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 0x7ff6f819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 0x7ff6f819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 -- 172.21.15.94:0/1384383357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 msgr2=0x7ff6f819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 0x7ff6f819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 -- 172.21.15.94:0/1384383357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 msgr2=0x7ff6f819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 0x7ff6f819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 -- 172.21.15.94:0/1384383357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f81a2090 con 0x7ff6f8108920 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 0x7ff6f819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 0x7ff6f819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fcd45640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000cad0 tx=0x7ff6f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f57fa640 1 -- 172.21.15.94:0/1384383357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000ae20 con 0x7ff6f8108920 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f57fa640 1 -- 172.21.15.94:0/1384383357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f0005020 con 0x7ff6f8108920 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f81a2380 con 0x7ff6f8108920 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6f57fa640 1 -- 172.21.15.94:0/1384383357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000f6b0 con 0x7ff6f8108920 2024-04-25T13:41:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.872+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f81a2880 con 0x7ff6f8108920 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f57fa640 1 -- 172.21.15.94:0/1384383357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff6f00051d0 con 0x7ff6f8108920 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 Processor -- start 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 -- start start 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf37370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fece9c740 con 0x559fecb6d3b0 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fece383a0 con 0x559fecb96270 2024-04-25T13:41:35.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff705657740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fece98700 con 0x559fecb75590 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34744/0 (socket says 172.21.15.94:34744) 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 -- 172.21.15.94:0/2097176382 learned_addr learned my addr 172.21.15.94:0/2097176382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 -- 172.21.15.94:0/2097176382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 msgr2=0x559fecf3e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf37370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 -- 172.21.15.94:0/2097176382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 msgr2=0x559fecf37370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f57fa640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 0x7ff6e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf37370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 -- 172.21.15.94:0/2097176382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fece54750 con 0x559fecb6d3b0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6fcd45640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 0x7ff6e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.875+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800cc40 tx=0x7ff6e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680a37714d548ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff6de7fc640 1 -- 172.21.15.94:0/2097176382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8014070 con 0x559fecb6d3b0 2024-04-25T13:41:35.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff6f57fa640 1 -- 172.21.15.94:0/1384383357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff6f00ded40 con 0x7ff6f8108920 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff6de7fc640 1 -- 172.21.15.94:0/2097176382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e8019b10 con 0x559fecb6d3b0 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff6de7fc640 1 -- 172.21.15.94:0/2097176382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8010be0 con 0x559fecb6d3b0 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 -- 172.21.15.94:0/2097176382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 msgr2=0x559fecf378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff6fcd45640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 0x7ff6e00b2000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00dcb90 tx=0x7ff6f0075d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800cc40 tx=0x7ff6e800e720 comp rx=0 tx=0).stop 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 -- 172.21.15.94:0/2097176382 shutdown_connections 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf3e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf378f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 --2- 172.21.15.94:0/2097176382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf37370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.876+0000 7ff705657740 1 -- 172.21.15.94:0/2097176382 >> 172.21.15.94:0/2097176382 conn(0x559fecb8a3c0 msgr2=0x559fecf32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- 172.21.15.94:0/2097176382 shutdown_connections 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- 172.21.15.94:0/2097176382 wait complete. 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- start start 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 Processor -- start 2024-04-25T13:41:35.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fecf3f240 con 0x559fecb6d3b0 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fece23fe0 con 0x559fecb96270 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.877+0000 7ff705657740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fecf3f3c0 con 0x559fecb75590 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff705657740 10 client.0 _ll_register_callbacks cb 0x559fece3a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34752/0 (socket says 172.21.15.94:34752) 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1189938123 learned_addr learned my addr 172.21.15.94:0/1189938123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 msgr2=0x559fecf417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1189938123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 msgr2=0x559fecf48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1189938123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fecbae2d0 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec032fd0 tx=0x7ff6ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b35e2ad7093b2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec035040 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff6ec005040 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8019b10 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec005240 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff705657740 10 client.15261 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559fecc786b0 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.878+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e8019b10 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff6ec003680 con 0x559fecb6d3b0 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 20 client.15261 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:35.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb54e40 0x559fecf4ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 4 client.15261 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559feceaa350 con 0x559fecb54e40 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb54e40 0x559fecf4ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff6d77fe640 10 client.15261 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.879+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb54e40 0x559fecf4ef90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80b07ba7824cc477 server_cookie=bc6be882af2f0d55 in_seq=0 out_seq=0 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.880+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1189938123 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559feceaa350 con 0x559fecb54e40 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.880+0000 7ff6d77fe640 10 client.15261 handle_command_reply: tid=0 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.880+0000 7ff705657740 1 client.15261 shutdown 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 msgr2=0x559fecf41290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec032fd0 tx=0x7ff6ec000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 shutdown_connections 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb54e40 0x559fecf4ef90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff6d77fe640 0 client.15261 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 --2- 172.21.15.94:0/1189938123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf41290 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 >> 172.21.15.94:0/1189938123 conn(0x559fecb8a3c0 msgr2=0x559fecf31d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 shutdown_connections 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 1 -- 172.21.15.94:0/1189938123 wait complete. 2024-04-25T13:41:35.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.881+0000 7ff705657740 20 client.15261 trim_cache size 0 max 16384 2024-04-25T13:41:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2301241799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4097598434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/865429620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:35.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 Processor -- start 2024-04-25T13:41:35.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 -- start start 2024-04-25T13:41:35.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf3f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4d2d0 con 0x559fecb6d3b0 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4d9a0 con 0x559fecb96270 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff705657740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4e070 con 0x559fecb75590 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34766/0 (socket says 172.21.15.94:34766) 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 -- 172.21.15.94:0/3290655396 learned_addr learned my addr 172.21.15.94:0/3290655396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 -- 172.21.15.94:0/3290655396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 msgr2=0x559fecf486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 -- 172.21.15.94:0/3290655396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 msgr2=0x559fecf3f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.899+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf3f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.900+0000 7ff6f77fe640 1 -- 172.21.15.94:0/3290655396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fecf3c110 con 0x559fecb6d3b0 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.900+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0096c0 tx=0x7ff6ec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b38381de2f852b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.900+0000 7ff6ddffb640 1 -- 172.21.15.94:0/3290655396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec035040 con 0x559fecb6d3b0 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.900+0000 7ff6ddffb640 1 -- 172.21.15.94:0/3290655396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec046070 con 0x559fecb6d3b0 2024-04-25T13:41:35.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.900+0000 7ff6ddffb640 1 -- 172.21.15.94:0/3290655396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec032030 con 0x559fecb6d3b0 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- 172.21.15.94:0/3290655396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 msgr2=0x559fecf2da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0096c0 tx=0x7ff6ec031400 comp rx=0 tx=0).stop 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- 172.21.15.94:0/3290655396 shutdown_connections 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb96270 0x559fecf3f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb75590 0x559fecf486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- 172.21.15.94:0/3290655396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb6d3b0 0x559fecf2da20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- 172.21.15.94:0/3290655396 >> 172.21.15.94:0/3290655396 conn(0x559fecb8a3c0 msgr2=0x559fecf3a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- 172.21.15.94:0/3290655396 shutdown_connections 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- 172.21.15.94:0/3290655396 wait complete. 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- start start 2024-04-25T13:41:35.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 Processor -- start 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb6d3b0 0x559fecf43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 0x559fecf43dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4ee10 con 0x559fecb96270 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4e740 con 0x559fecb75590 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fecf4f4e0 con 0x559fecb6d3b0 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.901+0000 7ff705657740 10 client.0 _ll_register_callbacks cb 0x7ff6ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34770/0 (socket says 172.21.15.94:34770) 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1417641560 learned_addr learned my addr 172.21.15.94:0/1417641560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb6d3b0 0x559fecf43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 0x559fecf43dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb6d3b0 msgr2=0x559fecf43890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb6d3b0 0x559fecf43890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1417641560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 msgr2=0x559fecf43dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 0x559fecf43dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 -- 172.21.15.94:0/1417641560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fecf47280 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 0x559fecf43dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6f77fe640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec003bc0 tx=0x7ff6ec005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aba968e3506165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec035070 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff6ec003dd0 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec046070 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec031a50 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff705657740 10 client.15267 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559fecf48500 con 0x559fecb96270 2024-04-25T13:41:35.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.902+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec046070 con 0x559fecb96270 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7ff6ec032780 con 0x559fecb96270 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 10 client.15267 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 20 client.15267 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb2a280 0x559fecfbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 4 client.15267 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "status"}) -- 0x559fecea41c0 con 0x559fecb2a280 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb2a280 0x559fecfbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff6f7fff640 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb2a280 0x559fecfbac00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1129becb52b39c95 server_cookie=143e1886a08040cb in_seq=0 out_seq=0 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.903+0000 7ff6d77fe640 10 client.15267 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff6d77fe640 1 -- 172.21.15.94:0/1417641560 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559fecea41c0 con 0x559fecb2a280 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff6d77fe640 10 client.15267 handle_command_reply: tid=0 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 client.15267 shutdown 2024-04-25T13:41:35.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 msgr2=0x559fecf46cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec003bc0 tx=0x7ff6ec005c90 comp rx=0 tx=0).stop 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 shutdown_connections 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559fecb2a280 0x559fecfbac00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fecb96270 0x559fecf46cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fecb75590 0x559fecf43dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff6d77fe640 0 client.15267 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 --2- 172.21.15.94:0/1417641560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fecb6d3b0 0x559fecf43890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 >> 172.21.15.94:0/1417641560 conn(0x559fecb8a3c0 msgr2=0x559fecf3a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 shutdown_connections 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 1 -- 172.21.15.94:0/1417641560 wait complete. 2024-04-25T13:41:35.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.904+0000 7ff705657740 20 client.15267 trim_cache size 0 max 16384 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 4, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24475, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 105.209306041, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 129.13253975200001 2024-04-25T13:41:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:35.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 msgr2=0x7ff6e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 0x7ff6e00b2000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00dcb90 tx=0x7ff6f0075d50 comp rx=0 tx=0).stop 2024-04-25T13:41:35.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 msgr2=0x7ff6f81a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000cad0 tx=0x7ff6f000cf90 comp rx=0 tx=0).stop 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 shutdown_connections 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6e00afaf0 0x7ff6e00b2000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6f8108920 0x7ff6f81a1a20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6f8103c20 0x7ff6f819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 --2- 172.21.15.94:0/1384383357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6f81029e0 0x7ff6f819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 >> 172.21.15.94:0/1384383357 conn(0x7ff6f80fdbd0 msgr2=0x7ff6f8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 shutdown_connections 2024-04-25T13:41:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:35.905+0000 7ff6fe7a8640 1 -- 172.21.15.94:0/1384383357 wait complete. 2024-04-25T13:41:35.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T13:41:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2301241799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4097598434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/865429620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 -- 172.21.15.94:0/2183360155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8102370 msgr2=0x7fcce8109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 --2- 172.21.15.94:0/2183360155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8102370 0x7fcce8109100 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 -- 172.21.15.94:0/2183360155 shutdown_connections 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 --2- 172.21.15.94:0/2183360155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcce8109790 0x7fcce810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 --2- 172.21.15.94:0/2183360155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8102370 0x7fcce8109100 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 --2- 172.21.15.94:0/2183360155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8101e70 0x7fcce8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 -- 172.21.15.94:0/2183360155 >> 172.21.15.94:0/2183360155 conn(0x7fcce80fd380 msgr2=0x7fcce80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 -- 172.21.15.94:0/2183360155 shutdown_connections 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.024+0000 7fccef772640 1 -- 172.21.15.94:0/2183360155 wait complete. 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 Processor -- start 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 -- start start 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 0x7fcce819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcce8109790 0x7fcce81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810f600 con 0x7fcce8101e70 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810f480 con 0x7fcce8102370 2024-04-25T13:41:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccef772640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810f780 con 0x7fcce8109790 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34788/0 (socket says 172.21.15.94:34788) 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 -- 172.21.15.94:0/3126492903 learned_addr learned my addr 172.21.15.94:0/3126492903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccecd0d640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 0x7fcce819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 -- 172.21.15.94:0/3126492903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcce8109790 msgr2=0x7fcce81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcce8109790 0x7fcce81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 -- 172.21.15.94:0/3126492903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 msgr2=0x7fcce819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 0x7fcce819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 -- 172.21.15.94:0/3126492903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81a1c90 con 0x7fcce8101e70 2024-04-25T13:41:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fccecd0d640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 0x7fcce819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.025+0000 7fcced50e640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c920 tx=0x7fcce400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.026+0000 7fccde7fc640 1 -- 172.21.15.94:0/3126492903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4007c70 con 0x7fcce8101e70 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.026+0000 7fccde7fc640 1 -- 172.21.15.94:0/3126492903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce4007e10 con 0x7fcce8101e70 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.026+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81a1f80 con 0x7fcce8101e70 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.026+0000 7fccde7fc640 1 -- 172.21.15.94:0/3126492903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4018820 con 0x7fcce8101e70 2024-04-25T13:41:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.026+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a2440 con 0x7fcce8101e70 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.028+0000 7fccde7fc640 1 -- 172.21.15.94:0/3126492903 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcce400f450 con 0x7fcce8101e70 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.028+0000 7fccf661d740 1 Processor -- start 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.028+0000 7fccf661d740 1 -- start start 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864de270 0x55fe8687f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe8687f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe867e4740 con 0x55fe864bd590 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe867803a0 con 0x55fe864b53b0 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccf661d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe867e0700 con 0x55fe864de270 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccde7fc640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 0x7fccd40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34796/0 (socket says 172.21.15.94:34796) 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccedd0f640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 0x7fccd40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 -- 172.21.15.94:0/548308731 learned_addr learned my addr 172.21.15.94:0/548308731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fcced50e640 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe8687f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccedd0f640 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864de270 0x55fe8687f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccde7fc640 1 -- 172.21.15.94:0/3126492903 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcce40dedc0 con 0x7fcce8101e70 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 -- 172.21.15.94:0/548308731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864de270 msgr2=0x55fe8687f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864de270 0x55fe8687f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 -- 172.21.15.94:0/548308731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 msgr2=0x55fe8687f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe8687f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 -- 172.21.15.94:0/548308731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe8679c750 con 0x55fe864bd590 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccedd0f640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 0x7fccd40b2080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcce0013450 tx=0x7fcce0013910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccecd0d640 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fccd80096c0 tx=0x7fccd8031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a25635830a76abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.029+0000 7fccd37fe640 1 -- 172.21.15.94:0/548308731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8035030 con 0x55fe864bd590 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccd37fe640 1 -- 172.21.15.94:0/548308731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd80317a0 con 0x55fe864bd590 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccd37fe640 1 -- 172.21.15.94:0/548308731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8003c90 con 0x55fe864bd590 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 -- 172.21.15.94:0/548308731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 msgr2=0x55fe868867f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fccd80096c0 tx=0x7fccd8031e80 comp rx=0 tx=0).stop 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 -- 172.21.15.94:0/548308731 shutdown_connections 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe868867f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe8687f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 --2- 172.21.15.94:0/548308731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864de270 0x55fe8687f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 -- 172.21.15.94:0/548308731 >> 172.21.15.94:0/548308731 conn(0x55fe864d23c0 msgr2=0x55fe8687ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 -- 172.21.15.94:0/548308731 shutdown_connections 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.030+0000 7fccf661d740 1 -- 172.21.15.94:0/548308731 wait complete. 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 -- start start 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 Processor -- start 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe868892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864bd590 0x55fe868897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe868871e0 con 0x55fe864de270 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe8676bfe0 con 0x55fe864b53b0 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe86887360 con 0x55fe864bd590 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccf661d740 10 client.0 _ll_register_callbacks cb 0x55fe86782150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccedd0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccedd0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34806/0 (socket says 172.21.15.94:34806) 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.031+0000 7fccedd0f640 1 -- 172.21.15.94:0/2014056502 learned_addr learned my addr 172.21.15.94:0/2014056502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fcced50e640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe868892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccecd0d640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864bd590 0x55fe868897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 -- 172.21.15.94:0/2014056502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864bd590 msgr2=0x55fe868897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864bd590 0x55fe868897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 -- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 msgr2=0x55fe868892a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe868892a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 -- 172.21.15.94:0/2014056502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe864f62d0 con 0x55fe864de270 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccedd0f640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcce0010940 tx=0x7fcce0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bec9ebf4985b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0025070 con 0x55fe864de270 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcce0011d70 con 0x55fe864de270 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd80317a0 con 0x55fe864de270 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcce0031050 con 0x55fe864de270 2024-04-25T13:41:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccf661d740 10 client.15279 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fe865c06b0 con 0x55fe864de270 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.032+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd80317a0 con 0x55fe864de270 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcce000f040 con 0x55fe864de270 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 10 client.15279 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 20 client.15279 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe8649ce40 0x55fe86897070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 4 client.15279 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe867f2350 con 0x55fe8649ce40 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fcced50e640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe8649ce40 0x55fe86897070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fccd0ff9640 10 client.15279 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.033+0000 7fcced50e640 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe8649ce40 0x55fe86897070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90d83db806195f4e server_cookie=3a59d0afd3c12207 in_seq=0 out_seq=0 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccd0ff9640 1 -- 172.21.15.94:0/2014056502 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe867f2350 con 0x55fe8649ce40 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccd0ff9640 10 client.15279 handle_command_reply: tid=0 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 client.15279 shutdown 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 msgr2=0x55fe868906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcce0010940 tx=0x7fcce0005e00 comp rx=0 tx=0).stop 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 shutdown_connections 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe8649ce40 0x55fe86897070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864de270 0x55fe868906d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864bd590 0x55fe868897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccd0ff9640 0 client.15279 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 --2- 172.21.15.94:0/2014056502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864b53b0 0x55fe868892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.034+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 >> 172.21.15.94:0/2014056502 conn(0x55fe864d23c0 msgr2=0x55fe86879d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.035+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 shutdown_connections 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.035+0000 7fccf661d740 1 -- 172.21.15.94:0/2014056502 wait complete. 2024-04-25T13:41:36.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.035+0000 7fccf661d740 20 client.15279 trim_cache size 0 max 16384 2024-04-25T13:41:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 Processor -- start 2024-04-25T13:41:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 -- start start 2024-04-25T13:41:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe868905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864bd590 0x55fe8687a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe868953b0 con 0x55fe8649ce40 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe86895a80 con 0x55fe864bd590 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fccf661d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe86896150 con 0x55fe864b53b0 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fcced50e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fcced50e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34822/0 (socket says 172.21.15.94:34822) 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.053+0000 7fcced50e640 1 -- 172.21.15.94:0/1067347915 learned_addr learned my addr 172.21.15.94:0/1067347915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 -- 172.21.15.94:0/1067347915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 msgr2=0x55fe868905f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fccecd0d640 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe868905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe868905f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 -- 172.21.15.94:0/1067347915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864bd590 msgr2=0x55fe8687a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864bd590 0x55fe8687a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 -- 172.21.15.94:0/1067347915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe86884150 con 0x55fe8649ce40 2024-04-25T13:41:36.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fccecd0d640 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe868905f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fcced50e640 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcce40dc8a0 tx=0x7fcce40a2b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c6cf2f857ddf5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fccd2ffd640 1 -- 172.21.15.94:0/1067347915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce400fa90 con 0x55fe8649ce40 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fccd2ffd640 1 -- 172.21.15.94:0/1067347915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce400f740 con 0x55fe8649ce40 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.054+0000 7fccd2ffd640 1 -- 172.21.15.94:0/1067347915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40a7740 con 0x55fe8649ce40 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- 172.21.15.94:0/1067347915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 msgr2=0x55fe86894990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcce40dc8a0 tx=0x7fcce40a2b40 comp rx=0 tx=0).stop 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- 172.21.15.94:0/1067347915 shutdown_connections 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe864bd590 0x55fe8687a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe868905f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 --2- 172.21.15.94:0/1067347915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe8649ce40 0x55fe86894990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- 172.21.15.94:0/1067347915 >> 172.21.15.94:0/1067347915 conn(0x55fe864d23c0 msgr2=0x55fe868878e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- 172.21.15.94:0/1067347915 shutdown_connections 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- 172.21.15.94:0/1067347915 wait complete. 2024-04-25T13:41:36.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 -- start start 2024-04-25T13:41:36.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.055+0000 7fccf661d740 1 Processor -- start 2024-04-25T13:41:36.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe8649ce40 0x55fe8688c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe8688c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe86896ef0 con 0x55fe864bd590 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe86896820 con 0x55fe8649ce40 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe868975c0 con 0x55fe864b53b0 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 10 client.0 _ll_register_callbacks cb 0x7fcce00094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34838/0 (socket says 172.21.15.94:34838) 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 -- 172.21.15.94:0/3498889585 learned_addr learned my addr 172.21.15.94:0/3498889585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fcced50e640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe8649ce40 0x55fe8688c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccecd0d640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe8688c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 -- 172.21.15.94:0/3498889585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 msgr2=0x55fe8688c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe8688c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 -- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe8649ce40 msgr2=0x55fe8688c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe8649ce40 0x55fe8688c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 -- 172.21.15.94:0/3498889585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe8688fc40 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccedd0f640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009da0 tx=0x7fcce0011370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f0291f667850f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0022070 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcce0011d70 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce400f740 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcce0025040 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 10 client.15285 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.056+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fe869001a0 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce400f740 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcce001dae0 con 0x55fe864bd590 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 10 client.15285 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 10 client.15285 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T13:41:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 20 client.15285 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe864de270 0x55fe86903a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 4 client.15285 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "status"}) -- 0x55fe867ec1c0 con 0x55fe864de270 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.057+0000 7fcced50e640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe864de270 0x55fe86903a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccd0ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fcced50e640 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe864de270 0x55fe86903a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9d1d733d7cb40f3 server_cookie=48f6e9bf70fde14c in_seq=0 out_seq=0 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccd0ff9640 1 -- 172.21.15.94:0/3498889585 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55fe867ec1c0 con 0x55fe864de270 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccd0ff9640 10 client.15285 handle_command_reply: tid=0 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 client.15285 shutdown 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 msgr2=0x55fe8688f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009da0 tx=0x7fcce0011370 comp rx=0 tx=0).stop 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 shutdown_connections 2024-04-25T13:41:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe864de270 0x55fe86903a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe864bd590 0x55fe8688f680 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe864b53b0 0x55fe8688c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccd0ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 --2- 172.21.15.94:0/3498889585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe8649ce40 0x55fe8688c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 >> 172.21.15.94:0/3498889585 conn(0x55fe864d23c0 msgr2=0x55fe8687a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 shutdown_connections 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 1 -- 172.21.15.94:0/3498889585 wait complete. 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.058+0000 7fccf661d740 20 client.15285 trim_cache size 0 max 16384 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 3, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24488, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 106.387625929, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 134.673700217 2024-04-25T13:41:36.065 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:36.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 msgr2=0x7fccd40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 0x7fccd40b2080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcce0013450 tx=0x7fcce0013910 comp rx=0 tx=0).stop 2024-04-25T13:41:36.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 msgr2=0x7fcce819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c920 tx=0x7fcce400cde0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 shutdown_connections 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fccd40afb70 0x7fccd40b2080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcce8109790 0x7fcce81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcce8102370 0x7fcce819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 --2- 172.21.15.94:0/3126492903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce8101e70 0x7fcce819a200 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 >> 172.21.15.94:0/3126492903 conn(0x7fcce80fd380 msgr2=0x7fcce810a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 shutdown_connections 2024-04-25T13:41:36.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.059+0000 7fccef772640 1 -- 172.21.15.94:0/3126492903 wait complete. 2024-04-25T13:41:36.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 -- 172.21.15.94:0/2980984479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c101e70 msgr2=0x7f6a6c106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 --2- 172.21.15.94:0/2980984479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c101e70 0x7f6a6c106b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009900 tx=0x7f6a5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 -- 172.21.15.94:0/2980984479 shutdown_connections 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 --2- 172.21.15.94:0/2980984479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c109790 0x7f6a6c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 --2- 172.21.15.94:0/2980984479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c102370 0x7f6a6c109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 --2- 172.21.15.94:0/2980984479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c101e70 0x7f6a6c106b30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 -- 172.21.15.94:0/2980984479 >> 172.21.15.94:0/2980984479 conn(0x7f6a6c0fd380 msgr2=0x7f6a6c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 -- 172.21.15.94:0/2980984479 shutdown_connections 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.178+0000 7f6a72434640 1 -- 172.21.15.94:0/2980984479 wait complete. 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 Processor -- start 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 -- start start 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 0x7f6a6c104340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c109790 0x7f6a6c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c10f5f0 con 0x7f6a6c102370 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c10f470 con 0x7f6a6c101e70 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a72434640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c10f770 con 0x7f6a6c109790 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34862/0 (socket says 172.21.15.94:34862) 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/910427858 learned_addr learned my addr 172.21.15.94:0/910427858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.179+0000 7f6a709d1640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c109790 0x7f6a6c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 0x7f6a6c104340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/910427858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c109790 msgr2=0x7f6a6c102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c109790 0x7f6a6c102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/910427858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 msgr2=0x7f6a6c104340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 0x7f6a6c104340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/910427858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c103650 con 0x7f6a6c102370 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 0x7f6a6c104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000d770 tx=0x7f6a6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a697fa640 1 -- 172.21.15.94:0/910427858 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60017070 con 0x7f6a6c102370 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c1a42e0 con 0x7f6a6c102370 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a697fa640 1 -- 172.21.15.94:0/910427858 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a600054d0 con 0x7f6a6c102370 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a697fa640 1 -- 172.21.15.94:0/910427858 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60004c30 con 0x7f6a6c102370 2024-04-25T13:41:36.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.180+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1a4750 con 0x7f6a6c102370 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.182+0000 7f6a793c0740 1 Processor -- start 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.182+0000 7f6a793c0740 1 -- start start 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.182+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4062e270 0x557e409cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.182+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e406053b0 0x557e409cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a793c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e40934740 con 0x557e4060d590 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a793c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e408d03a0 con 0x557e406053b0 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a793c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e40930700 con 0x557e4062e270 2024-04-25T13:41:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a697fa640 1 -- 172.21.15.94:0/910427858 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6a60010460 con 0x7f6a6c102370 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34876/0 (socket says 172.21.15.94:34876) 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/4090191110 learned_addr learned my addr 172.21.15.94:0/4090191110 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e406053b0 0x557e409cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a709d1640 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4062e270 0x557e409cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/4090191110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4062e270 msgr2=0x557e409cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4062e270 0x557e409cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/4090191110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e406053b0 msgr2=0x557e409cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e406053b0 0x557e409cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 -- 172.21.15.94:0/4090191110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e408ec750 con 0x557e4060d590 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6a600e4c10 tx=0x7f6a600e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa2ccf101eeeda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a697fa640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 0x7f6a540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a527fc640 1 -- 172.21.15.94:0/4090191110 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600ed070 con 0x557e4060d590 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.183+0000 7f6a709d1640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 0x7f6a540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a527fc640 1 -- 172.21.15.94:0/4090191110 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a600a6710 con 0x557e4060d590 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a527fc640 1 -- 172.21.15.94:0/4090191110 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600a69d0 con 0x557e4060d590 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a697fa640 1 -- 172.21.15.94:0/910427858 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a600de740 con 0x7f6a6c102370 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 -- 172.21.15.94:0/4090191110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 msgr2=0x557e409d67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a709d1640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 0x7f6a540b2000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557e409d0540 tx=0x7f6a6400a4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6a600e4c10 tx=0x7f6a600e6690 comp rx=0 tx=0).stop 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 -- 172.21.15.94:0/4090191110 shutdown_connections 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409d67f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e406053b0 0x557e409cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 --2- 172.21.15.94:0/4090191110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4062e270 0x557e409cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 -- 172.21.15.94:0/4090191110 >> 172.21.15.94:0/4090191110 conn(0x557e406223c0 msgr2=0x557e409cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 -- 172.21.15.94:0/4090191110 shutdown_connections 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.184+0000 7f6a793c0740 1 -- 172.21.15.94:0/4090191110 wait complete. 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 -- start start 2024-04-25T13:41:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 Processor -- start 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4060d590 0x557e409d9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 0x557e409e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e409d7240 con 0x557e406053b0 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e408bbfe0 con 0x557e4062e270 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e409d73c0 con 0x557e4060d590 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a793c0740 10 client.0 _ll_register_callbacks cb 0x557e408d2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34884/0 (socket says 172.21.15.94:34884) 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.185+0000 7f6a6bfff640 1 -- 172.21.15.94:0/1048808043 learned_addr learned my addr 172.21.15.94:0/1048808043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 -- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4060d590 msgr2=0x557e409d9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a709d1640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 0x557e409e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4060d590 0x557e409d9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4060d590 0x557e409d9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 -- 172.21.15.94:0/1048808043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 msgr2=0x557e409e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 0x557e409e0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 -- 172.21.15.94:0/1048808043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e406462d0 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a709d1640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 0x557e409e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009900 tx=0x7f6a5c0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7929c94368137c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c040070 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a5c004250 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a600a6710 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a5c04a070 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a793c0740 10 client.15297 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e407106b0 con 0x557e406053b0 2024-04-25T13:41:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a600a6710 con 0x557e406053b0 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.186+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6a5c0049c0 con 0x557e406053b0 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 20 client.15297 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e405ece40 0x557e409e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 4 client.15297 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e40942350 con 0x557e405ece40 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e405ece40 0x557e409e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a4b7fe640 10 client.15297 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.187+0000 7f6a6b7fe640 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e405ece40 0x557e409e6f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a74cd1eeabc91f0 server_cookie=c8ea55b9fd9e5458 in_seq=0 out_seq=0 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.188+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/1048808043 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e40942350 con 0x557e405ece40 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.188+0000 7f6a4b7fe640 10 client.15297 handle_command_reply: tid=0 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.188+0000 7f6a793c0740 1 client.15297 shutdown 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 msgr2=0x557e409d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009900 tx=0x7f6a5c0317d0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 shutdown_connections 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e405ece40 0x557e409e6f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4062e270 0x557e409e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4060d590 0x557e409d9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a4b7fe640 0 client.15297 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1048808043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e406053b0 0x557e409d91d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 >> 172.21.15.94:0/1048808043 conn(0x557e406223c0 msgr2=0x557e409c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 shutdown_connections 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 1 -- 172.21.15.94:0/1048808043 wait complete. 2024-04-25T13:41:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.189+0000 7f6a793c0740 20 client.15297 trim_cache size 0 max 16384 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 Processor -- start 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 -- start start 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e409e5260 con 0x557e4060d590 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e409e5930 con 0x557e405ece40 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a793c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e409e6000 con 0x557e406053b0 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.207+0000 7f6a709d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34898/0 (socket says 172.21.15.94:34898) 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 -- 172.21.15.94:0/1065478186 learned_addr learned my addr 172.21.15.94:0/1065478186 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 -- 172.21.15.94:0/1065478186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 msgr2=0x557e409e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 -- 172.21.15.94:0/1065478186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 msgr2=0x557e409d7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409d7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 -- 172.21.15.94:0/1065478186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e409cef20 con 0x557e4060d590 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a709d1640 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6a6401d600 tx=0x7f6a64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03fb2ac6ada5b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a51ffb640 1 -- 172.21.15.94:0/1065478186 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400f770 con 0x557e4060d590 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a51ffb640 1 -- 172.21.15.94:0/1065478186 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a6400fdc0 con 0x557e4060d590 2024-04-25T13:41:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.208+0000 7f6a51ffb640 1 -- 172.21.15.94:0/1065478186 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64028dd0 con 0x557e4060d590 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- 172.21.15.94:0/1065478186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 msgr2=0x557e409ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6a6401d600 tx=0x7f6a64014000 comp rx=0 tx=0).stop 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- 172.21.15.94:0/1065478186 shutdown_connections 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409ca6b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 --2- 172.21.15.94:0/1065478186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409d7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- 172.21.15.94:0/1065478186 >> 172.21.15.94:0/1065478186 conn(0x557e406223c0 msgr2=0x557e409d77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- 172.21.15.94:0/1065478186 shutdown_connections 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- 172.21.15.94:0/1065478186 wait complete. 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.209+0000 7f6a793c0740 1 -- start start 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 Processor -- start 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409db990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e409e6da0 con 0x557e4060d590 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e409e66d0 con 0x557e405ece40 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e409e7470 con 0x557e406053b0 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a793c0740 10 client.0 _ll_register_callbacks cb 0x7f6a5c032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34900/0 (socket says 172.21.15.94:34900) 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 -- 172.21.15.94:0/2515422588 learned_addr learned my addr 172.21.15.94:0/2515422588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409db990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 -- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 msgr2=0x557e409dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 -- 172.21.15.94:0/2515422588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 msgr2=0x557e409db990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409db990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 -- 172.21.15.94:0/2515422588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e409df360 con 0x557e4060d590 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409db990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a709d1640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400a310 tx=0x7f6a64028d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b5052015f693e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.210+0000 7f6a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64028e20 con 0x557e4060d590 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f6a64004c50 con 0x557e4060d590 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a6400fdc0 con 0x557e4060d590 2024-04-25T13:41:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a64004e50 con 0x557e4060d590 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 10 client.15303 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557e409e05e0 con 0x557e4060d590 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a6400fdc0 con 0x557e4060d590 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6a640054e0 con 0x557e4060d590 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 10 client.15303 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 10 client.15303 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 20 client.15303 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e4062e270 0x557e40a52e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 4 client.15303 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "status"}) -- 0x557e4093c1c0 con 0x557e4062e270 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.211+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e4062e270 0x557e40a52e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a4b7fe640 10 client.15303 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a6bfff640 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e4062e270 0x557e40a52e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45a3cc9bf30e4720 server_cookie=7177008831ad7bf1 in_seq=0 out_seq=0 2024-04-25T13:41:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a4b7fe640 1 -- 172.21.15.94:0/2515422588 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557e4093c1c0 con 0x557e4062e270 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a4b7fe640 10 client.15303 handle_command_reply: tid=0 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 client.15303 shutdown 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 msgr2=0x557e409deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400a310 tx=0x7f6a64028d00 comp rx=0 tx=0).stop 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 shutdown_connections 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557e4062e270 0x557e40a52e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4060d590 0x557e409deda0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e406053b0 0x557e409dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 --2- 172.21.15.94:0/2515422588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e405ece40 0x557e409db990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 >> 172.21.15.94:0/2515422588 conn(0x557e406223c0 msgr2=0x557e409ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a4b7fe640 0 client.15303 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 shutdown_connections 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 1 -- 172.21.15.94:0/2515422588 wait complete. 2024-04-25T13:41:36.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.212+0000 7f6a793c0740 20 client.15303 trim_cache size 0 max 16384 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24502, 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 107.545674468, 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 123.853573099 2024-04-25T13:41:36.220 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 msgr2=0x7f6a540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 0x7f6a540b2000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557e409d0540 tx=0x7f6a6400a4d0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 msgr2=0x7f6a6c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000d770 tx=0x7f6a6000dc30 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 shutdown_connections 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a540afaf0 0x7f6a540b2000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a6c109790 0x7f6a6c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a6c102370 0x7f6a6c1029e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 --2- 172.21.15.94:0/910427858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6c101e70 0x7f6a6c104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 >> 172.21.15.94:0/910427858 conn(0x7f6a6c0fd380 msgr2=0x7f6a6c10a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 shutdown_connections 2024-04-25T13:41:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.213+0000 7f6a72434640 1 -- 172.21.15.94:0/910427858 wait complete. 2024-04-25T13:41:36.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 -- 172.21.15.94:0/744297536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 msgr2=0x7fcd781033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 --2- 172.21.15.94:0/744297536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd781033e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009900 tx=0x7fcd6802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 -- 172.21.15.94:0/744297536 shutdown_connections 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 --2- 172.21.15.94:0/744297536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78104690 0x7fcd7810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 --2- 172.21.15.94:0/744297536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd78103b30 0x7fcd78104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 --2- 172.21.15.94:0/744297536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd781033e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 -- 172.21.15.94:0/744297536 >> 172.21.15.94:0/744297536 conn(0x7fcd780fd350 msgr2=0x7fcd780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 -- 172.21.15.94:0/744297536 shutdown_connections 2024-04-25T13:41:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.333+0000 7fcd7e627640 1 -- 172.21.15.94:0/744297536 wait complete. 2024-04-25T13:41:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 Processor -- start 2024-04-25T13:41:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 -- start start 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78103b30 0x7fcd7819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd7819f360 0x7fcd781a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e710 con 0x7fcd78102fd0 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e590 con 0x7fcd7819f360 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd7e627640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7810e890 con 0x7fcd78103b30 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34910/0 (socket says 172.21.15.94:34910) 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.334+0000 7fcd77fff640 1 -- 172.21.15.94:0/2023350776 learned_addr learned my addr 172.21.15.94:0/2023350776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd7819f360 0x7fcd781a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd777fe640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78103b30 0x7fcd7819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 -- 172.21.15.94:0/2023350776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78103b30 msgr2=0x7fcd7819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78103b30 0x7fcd7819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 -- 172.21.15.94:0/2023350776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd7819f360 msgr2=0x7fcd781a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd7819f360 0x7fcd781a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 -- 172.21.15.94:0/2023350776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781a1f10 con 0x7fcd78102fd0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd77fff640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd68031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd757fa640 1 -- 172.21.15.94:0/2023350776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6803f030 con 0x7fcd78102fd0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781a21a0 con 0x7fcd78102fd0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd757fa640 1 -- 172.21.15.94:0/2023350776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd68003aa0 con 0x7fcd78102fd0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd757fa640 1 -- 172.21.15.94:0/2023350776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68035070 con 0x7fcd78102fd0 2024-04-25T13:41:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.335+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a2690 con 0x7fcd78102fd0 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.337+0000 7fcd757fa640 1 -- 172.21.15.94:0/2023350776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcd68042030 con 0x7fcd78102fd0 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 Processor -- start 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 -- start start 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc41454740 con 0x55bc4114e270 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc413f03a0 con 0x55bc411253b0 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd855bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc41450700 con 0x55bc4112d590 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd757fa640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 0x7fcd600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34920/0 (socket says 172.21.15.94:34920) 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/1004982093 learned_addr learned my addr 172.21.15.94:0/1004982093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 0x7fcd600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd777fe640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd757fa640 1 -- 172.21.15.94:0/2023350776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcd680ff200 con 0x7fcd78102fd0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.338+0000 7fcd77fff640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/1004982093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 msgr2=0x55bc414fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/1004982093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 msgr2=0x55bc414f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/1004982093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4140c750 con 0x55bc4114e270 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd77fff640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcd70014f10 tx=0x7fcd700117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e361332e77cb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 0x7fcd600b2000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005c40 tx=0x7fcd70016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd5e7fc640 1 -- 172.21.15.94:0/1004982093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70024070 con 0x55bc4114e270 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd5e7fc640 1 -- 172.21.15.94:0/1004982093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd70007ce0 con 0x55bc4114e270 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.339+0000 7fcd5e7fc640 1 -- 172.21.15.94:0/1004982093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70005770 con 0x55bc4114e270 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- 172.21.15.94:0/1004982093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 msgr2=0x55bc414f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcd70014f10 tx=0x7fcd700117b0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- 172.21.15.94:0/1004982093 shutdown_connections 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1004982093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc414f3780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- 172.21.15.94:0/1004982093 >> 172.21.15.94:0/1004982093 conn(0x55bc411423c0 msgr2=0x55bc414ead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- 172.21.15.94:0/1004982093 shutdown_connections 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- 172.21.15.94:0/1004982093 wait complete. 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 -- start start 2024-04-25T13:41:36.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.340+0000 7fcd855bb740 1 Processor -- start 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc414fb5f0 con 0x55bc4114e270 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc413dbfe0 con 0x55bc411253b0 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc414fb770 con 0x55bc4112d590 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 10 client.0 _ll_register_callbacks cb 0x55bc413f2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34928/0 (socket says 172.21.15.94:34928) 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/4111760915 learned_addr learned my addr 172.21.15.94:0/4111760915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd777fe640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd77fff640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/4111760915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 msgr2=0x55bc414fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 msgr2=0x55bc414fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/4111760915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc411662d0 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd77fff640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcd70014ac0 tx=0x7fcd70003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d21f013ec67b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd577fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70035060 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcd700054b0 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.341+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc41462350 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd70024050 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 10 client.15315 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc412306b0 con 0x55bc4114e270 2024-04-25T13:41:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bc41462350 con 0x55bc4114e270 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcd70004040 con 0x55bc4114e270 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 20 client.15315 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc4110ce40 0x55bc4150b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 4 client.15315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc4145c1c0 con 0x55bc4110ce40 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.342+0000 7fcd77fff640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc4110ce40 0x55bc4150b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.343+0000 7fcd577fe640 10 client.15315 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.343+0000 7fcd77fff640 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc4110ce40 0x55bc4150b200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=163960a7490bff5e server_cookie=8557cbb0789e86d7 in_seq=0 out_seq=0 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.343+0000 7fcd577fe640 1 -- 172.21.15.94:0/4111760915 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc4145c1c0 con 0x55bc4110ce40 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.343+0000 7fcd577fe640 10 client.15315 handle_command_reply: tid=0 2024-04-25T13:41:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 client.15315 shutdown 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 msgr2=0x55bc41504b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcd70014ac0 tx=0x7fcd70003970 comp rx=0 tx=0).stop 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 shutdown_connections 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc4110ce40 0x55bc4150b200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41504b70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4112d590 0x55bc414fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd577fe640 0 client.15315 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 --2- 172.21.15.94:0/4111760915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 >> 172.21.15.94:0/4111760915 conn(0x55bc411423c0 msgr2=0x55bc414e9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 shutdown_connections 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 1 -- 172.21.15.94:0/4111760915 wait complete. 2024-04-25T13:41:36.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.344+0000 7fcd855bb740 20 client.15315 trim_cache size 0 max 16384 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.362+0000 7fcd855bb740 1 Processor -- start 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.362+0000 7fcd855bb740 1 -- start start 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4114e270 0x55bc414f8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc41509540 con 0x55bc4112d590 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc41509c10 con 0x55bc411253b0 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd855bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc4150a2e0 con 0x55bc4114e270 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34936/0 (socket says 172.21.15.94:34936) 2024-04-25T13:41:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 -- 172.21.15.94:0/1504749666 learned_addr learned my addr 172.21.15.94:0/1504749666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4114e270 0x55bc414f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 -- 172.21.15.94:0/1504749666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4114e270 msgr2=0x55bc414f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4114e270 0x55bc414f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 -- 172.21.15.94:0/1504749666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 msgr2=0x55bc414f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 -- 172.21.15.94:0/1504749666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc414f48e0 con 0x55bc4112d590 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd777fe640 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00bbf0 tx=0x7fcd6c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fe46d1a3f932b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd5dffb640 1 -- 172.21.15.94:0/1504749666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00bda0 con 0x55bc4112d590 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd5dffb640 1 -- 172.21.15.94:0/1504749666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c0035f0 con 0x55bc4112d590 2024-04-25T13:41:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.363+0000 7fcd5dffb640 1 -- 172.21.15.94:0/1504749666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c010630 con 0x55bc4112d590 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- 172.21.15.94:0/1504749666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 msgr2=0x55bc41504a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00bbf0 tx=0x7fcd6c012820 comp rx=0 tx=0).stop 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- 172.21.15.94:0/1504749666 shutdown_connections 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc4114e270 0x55bc414f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4112d590 0x55bc41504a90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 --2- 172.21.15.94:0/1504749666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc411253b0 0x55bc414f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- 172.21.15.94:0/1504749666 >> 172.21.15.94:0/1504749666 conn(0x55bc411423c0 msgr2=0x55bc414f6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- 172.21.15.94:0/1504749666 shutdown_connections 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- 172.21.15.94:0/1504749666 wait complete. 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.364+0000 7fcd855bb740 1 -- start start 2024-04-25T13:41:36.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 Processor -- start 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc411253b0 0x55bc41501e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc4112d590 0x55bc41502340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc4150b080 con 0x55bc4114e270 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc4150a9b0 con 0x55bc4112d590 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc4150b750 con 0x55bc411253b0 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd855bb740 10 client.0 _ll_register_callbacks cb 0x7fcd6c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34944/0 (socket says 172.21.15.94:34944) 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/3580213232 learned_addr learned my addr 172.21.15.94:0/3580213232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd77fff640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc411253b0 0x55bc41501e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd777fe640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc4112d590 0x55bc41502340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/3580213232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc411253b0 msgr2=0x55bc41501e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc411253b0 0x55bc41501e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc4112d590 msgr2=0x55bc41502340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc4112d590 0x55bc41502340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.365+0000 7fcd7cbc4640 1 -- 172.21.15.94:0/3580213232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc41503830 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd7cbc4640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcd700117f0 tx=0x7fcd70003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d5f2f75f2e42bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70035040 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fcd70003d70 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c0035f0 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd7004b050 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 10 client.15321 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc41504b50 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c0035f0 con 0x55bc4114e270 2024-04-25T13:41:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcd70042070 con 0x55bc4114e270 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 10 client.15321 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 20 client.15321 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc410e2280 0x55bc41577480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 4 client.15321 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.366+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "status"}) -- 0x55bc413d79a0 con 0x55bc410e2280 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd77fff640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc410e2280 0x55bc41577480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd77fff640 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc410e2280 0x55bc41577480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6b21ffd95df5414 server_cookie=6b5cbf077e94d8e in_seq=0 out_seq=0 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd577fe640 10 client.15321 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd577fe640 1 -- 172.21.15.94:0/3580213232 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bc413d79a0 con 0x55bc410e2280 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd577fe640 10 client.15321 handle_command_reply: tid=0 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 client.15321 shutdown 2024-04-25T13:41:36.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 msgr2=0x55bc41503270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcd700117f0 tx=0x7fcd70003af0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 shutdown_connections 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55bc410e2280 0x55bc41577480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc4114e270 0x55bc41503270 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc4112d590 0x55bc41502340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd577fe640 0 client.15321 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 --2- 172.21.15.94:0/3580213232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc411253b0 0x55bc41501e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.367+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 >> 172.21.15.94:0/3580213232 conn(0x55bc411423c0 msgr2=0x55bc414f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 shutdown_connections 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd855bb740 1 -- 172.21.15.94:0/3580213232 wait complete. 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd855bb740 20 client.15321 trim_cache size 0 max 16384 2024-04-25T13:41:36.373 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24512, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 108.71605998299999, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 126.834061546 2024-04-25T13:41:36.374 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 msgr2=0x7fcd600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 0x7fcd600b2000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005c40 tx=0x7fcd70016000 comp rx=0 tx=0).stop 2024-04-25T13:41:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 msgr2=0x7fcd78102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006010 tx=0x7fcd68031a50 comp rx=0 tx=0).stop 2024-04-25T13:41:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 shutdown_connections 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd600afaf0 0x7fcd600b2000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd7819f360 0x7fcd781a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd78103b30 0x7fcd7819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 --2- 172.21.15.94:0/2023350776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd78102fd0 0x7fcd78102270 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 >> 172.21.15.94:0/2023350776 conn(0x7fcd780fd350 msgr2=0x7fcd780ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 shutdown_connections 2024-04-25T13:41:36.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.368+0000 7fcd7e627640 1 -- 172.21.15.94:0/2023350776 wait complete. 2024-04-25T13:41:36.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:41:36.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 -- 172.21.15.94:0/75167875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe441050b0 msgr2=0x7efe441054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 --2- 172.21.15.94:0/75167875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe441050b0 0x7efe441054c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe34009900 tx=0x7efe3402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 -- 172.21.15.94:0/75167875 shutdown_connections 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 --2- 172.21.15.94:0/75167875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe44102260 0x7efe4410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 --2- 172.21.15.94:0/75167875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 0x7efe44101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 --2- 172.21.15.94:0/75167875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe441050b0 0x7efe441054c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 -- 172.21.15.94:0/75167875 >> 172.21.15.94:0/75167875 conn(0x7efe440fd380 msgr2=0x7efe440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 -- 172.21.15.94:0/75167875 shutdown_connections 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.488+0000 7efe49030640 1 -- 172.21.15.94:0/75167875 wait complete. 2024-04-25T13:41:36.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 Processor -- start 2024-04-25T13:41:36.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 -- start start 2024-04-25T13:41:36.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 0x7efe4419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe441050b0 0x7efe441a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410f680 con 0x7efe44102260 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410f500 con 0x7efe44101760 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe49030640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410f800 con 0x7efe441050b0 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34960/0 (socket says 172.21.15.94:34960) 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4259c640 1 -- 172.21.15.94:0/3548685936 learned_addr learned my addr 172.21.15.94:0/3548685936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe42d9d640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 0x7efe4419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4359e640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe441050b0 0x7efe441a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4259c640 1 -- 172.21.15.94:0/3548685936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe441050b0 msgr2=0x7efe441a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.489+0000 7efe4259c640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe441050b0 0x7efe441a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe4259c640 1 -- 172.21.15.94:0/3548685936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 msgr2=0x7efe4419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe4259c640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 0x7efe4419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe4259c640 1 -- 172.21.15.94:0/3548685936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441a1dc0 con 0x7efe44102260 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe4259c640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d950 tx=0x7efe3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe2bfff640 1 -- 172.21.15.94:0/3548685936 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38017070 con 0x7efe44102260 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441a20b0 con 0x7efe44102260 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe2bfff640 1 -- 172.21.15.94:0/3548685936 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe38009c30 con 0x7efe44102260 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe2bfff640 1 -- 172.21.15.94:0/3548685936 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800f620 con 0x7efe44102260 2024-04-25T13:41:36.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.490+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe441a26d0 con 0x7efe44102260 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.492+0000 7efe4ffbd740 1 Processor -- start 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.492+0000 7efe2bfff640 1 -- 172.21.15.94:0/3548685936 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efe38010460 con 0x7efe44102260 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.492+0000 7efe4ffbd740 1 -- start start 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.492+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f805270 0x56084fba6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.492+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 0x56084fbad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56084fb0b740 con 0x56084f7dc3b0 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084faa73a0 con 0x56084f805270 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56084fb07700 con 0x56084f7e4590 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34976/0 (socket says 172.21.15.94:34976) 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 -- 172.21.15.94:0/4117034866 learned_addr learned my addr 172.21.15.94:0/4117034866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4359e640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f805270 0x56084fba6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe2bfff640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 0x7efe2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4259c640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 0x56084fbad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 -- 172.21.15.94:0/4117034866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 msgr2=0x56084fbad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 0x56084fbad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 -- 172.21.15.94:0/4117034866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f805270 msgr2=0x56084fba6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f805270 0x56084fba6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 -- 172.21.15.94:0/4117034866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084fac3750 con 0x56084f7dc3b0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4259c640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 0x56084fbad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe4359e640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 0x7efe2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efe34009230 tx=0x7efe34031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd4f46d18a31068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe2bfff640 1 -- 172.21.15.94:0/3548685936 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7efe380de8b0 con 0x7efe44102260 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe297fa640 1 -- 172.21.15.94:0/4117034866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34035030 con 0x56084f7dc3b0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe297fa640 1 -- 172.21.15.94:0/4117034866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe34040070 con 0x56084f7dc3b0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.493+0000 7efe297fa640 1 -- 172.21.15.94:0/4117034866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34003bb0 con 0x56084f7dc3b0 2024-04-25T13:41:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4117034866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 msgr2=0x56084fba68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4359e640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 0x7efe2c0b2000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56084fba7540 tx=0x7efe3c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efe34009230 tx=0x7efe34031c50 comp rx=0 tx=0).stop 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4117034866 shutdown_connections 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7e4590 0x56084fbad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fba68f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4117034866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f805270 0x56084fba6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4117034866 >> 172.21.15.94:0/4117034866 conn(0x56084f7f93c0 msgr2=0x56084fba1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4117034866 shutdown_connections 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.494+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4117034866 wait complete. 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 -- start start 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 Processor -- start 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56084fbae1e0 con 0x56084f7dc3b0 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084fa92fe0 con 0x56084f7e4590 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56084fbae360 con 0x56084f805270 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4ffbd740 10 client.0 _ll_register_callbacks cb 0x56084faa9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34992/0 (socket says 172.21.15.94:34992) 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 -- 172.21.15.94:0/2363213212 learned_addr learned my addr 172.21.15.94:0/2363213212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe4259c640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 -- 172.21.15.94:0/2363213212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 msgr2=0x56084fbb76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 -- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 msgr2=0x56084fbb0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 -- 172.21.15.94:0/2363213212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084f81d2d0 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe42d9d640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efe34005ab0 tx=0x7efe34031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa0d0598f1dce4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.495+0000 7efe26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34035070 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7efe34004450 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe34040070 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe34003930 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 10 client.15333 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56084f8e76b0 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe34040070 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7efe34045bf0 con 0x56084f7dc3b0 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 20 client.15333 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f7c3e40 0x56084fbbda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 4 client.15333 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.496+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084fb19350 con 0x56084f7c3e40 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.497+0000 7efe4259c640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f7c3e40 0x56084fbbda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.497+0000 7efe4259c640 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f7c3e40 0x56084fbbda40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14cbd1a84f702cf server_cookie=d22c32fd5ee1ddda in_seq=0 out_seq=0 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.497+0000 7efe26ff5640 10 client.15333 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe26ff5640 1 -- 172.21.15.94:0/2363213212 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56084fb19350 con 0x56084f7c3e40 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe26ff5640 10 client.15333 handle_command_reply: tid=0 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 client.15333 shutdown 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 msgr2=0x56084fbb0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efe34005ab0 tx=0x7efe34031850 comp rx=0 tx=0).stop 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 shutdown_connections 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f7c3e40 0x56084fbbda40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe26ff5640 0 client.15333 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2363213212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb0250 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 >> 172.21.15.94:0/2363213212 conn(0x56084f7f93c0 msgr2=0x56084fba0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 shutdown_connections 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2363213212 wait complete. 2024-04-25T13:41:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.498+0000 7efe4ffbd740 20 client.15333 trim_cache size 0 max 16384 2024-04-25T13:41:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.516+0000 7efe4ffbd740 1 Processor -- start 2024-04-25T13:41:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.516+0000 7efe4ffbd740 1 -- start start 2024-04-25T13:41:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7c3e40 0x56084fbaeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbbd80 con 0x56084f7dc3b0 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbc450 con 0x56084f7e4590 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbcb20 con 0x56084f7c3e40 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34996/0 (socket says 172.21.15.94:34996) 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 -- 172.21.15.94:0/2526676801 learned_addr learned my addr 172.21.15.94:0/2526676801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 -- 172.21.15.94:0/2526676801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7c3e40 msgr2=0x56084fbaeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7c3e40 0x56084fbaeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 -- 172.21.15.94:0/2526676801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 msgr2=0x56084fbb7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 -- 172.21.15.94:0/2526676801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084fbab0c0 con 0x56084f7dc3b0 2024-04-25T13:41:36.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe4259c640 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe380dc390 tx=0x7efe38010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096b665cae7f2b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe28ff9640 1 -- 172.21.15.94:0/2526676801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38020040 con 0x56084f7dc3b0 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe28ff9640 1 -- 172.21.15.94:0/2526676801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe380afbd0 con 0x56084f7dc3b0 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.517+0000 7efe28ff9640 1 -- 172.21.15.94:0/2526676801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380a26e0 con 0x56084f7dc3b0 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2526676801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 msgr2=0x56084fbb7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efe380dc390 tx=0x7efe38010cb0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2526676801 shutdown_connections 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7dc3b0 0x56084fbb7610 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/2526676801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f7c3e40 0x56084fbaeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2526676801 >> 172.21.15.94:0/2526676801 conn(0x56084f7f93c0 msgr2=0x56084fbae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2526676801 shutdown_connections 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- 172.21.15.94:0/2526676801 wait complete. 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 -- start start 2024-04-25T13:41:36.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.518+0000 7efe4ffbd740 1 Processor -- start 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbd8c0 con 0x56084f7c3e40 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbd1f0 con 0x56084f7e4590 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56084fbbdf90 con 0x56084f805270 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 10 client.0 _ll_register_callbacks cb 0x7efe34031ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34998/0 (socket says 172.21.15.94:34998) 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 -- 172.21.15.94:0/4274914762 learned_addr learned my addr 172.21.15.94:0/4274914762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4259c640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 -- 172.21.15.94:0/4274914762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 msgr2=0x56084fbb6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 -- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 msgr2=0x56084fbb37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 -- 172.21.15.94:0/4274914762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084fbb6c40 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4259c640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe42d9d640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efe34009900 tx=0x7efe34031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c5b2ea6db41b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34035040 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7efe34004250 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe380afbd0 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe34004450 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 10 client.15339 fetch_fsmap learned FSMap version 24 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.519+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56084fbb7f30 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe380afbd0 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7efe34045500 con 0x56084f7c3e40 2024-04-25T13:41:36.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 20 client.15339 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f799280 0x56084fc274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 4 client.15339 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "status"}) -- 0x56084fb131c0 con 0x56084f799280 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.520+0000 7efe4259c640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f799280 0x56084fc274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe26ff5640 10 client.15339 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4259c640 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f799280 0x56084fc274d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7185f3345c65cec server_cookie=1f5528612247f27a in_seq=0 out_seq=0 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe26ff5640 1 -- 172.21.15.94:0/4274914762 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56084fb131c0 con 0x56084f799280 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe26ff5640 10 client.15339 handle_command_reply: tid=0 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 client.15339 shutdown 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 msgr2=0x56084fbb3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efe34009900 tx=0x7efe34031400 comp rx=0 tx=0).stop 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 shutdown_connections 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56084f799280 0x56084fc274d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56084f805270 0x56084fbb6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe26ff5640 0 client.15339 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084f7e4590 0x56084fbb37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 --2- 172.21.15.94:0/4274914762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56084f7c3e40 0x56084fbb3270 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 >> 172.21.15.94:0/4274914762 conn(0x56084f7f93c0 msgr2=0x56084fba16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 shutdown_connections 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 1 -- 172.21.15.94:0/4274914762 wait complete. 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.521+0000 7efe4ffbd740 20 client.15339 trim_cache size 0 max 16384 2024-04-25T13:41:36.527 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f9021454-0307-11ef-bc93-c7b262605968", 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24557, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 110.913347573, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 24, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 115.900959198 2024-04-25T13:41:36.528 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:36.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 msgr2=0x7efe2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 0x7efe2c0b2000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56084fba7540 tx=0x7efe3c00c040 comp rx=0 tx=0).stop 2024-04-25T13:41:36.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 msgr2=0x7efe4419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d950 tx=0x7efe3800de10 comp rx=0 tx=0).stop 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 shutdown_connections 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe2c0afaf0 0x7efe2c0b2000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe441050b0 0x7efe441a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe44102260 0x7efe4419a7c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 --2- 172.21.15.94:0/3548685936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe44101760 0x7efe4419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 >> 172.21.15.94:0/3548685936 conn(0x7efe440fd380 msgr2=0x7efe44105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 shutdown_connections 2024-04-25T13:41:36.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:36.522+0000 7efe49030640 1 -- 172.21.15.94:0/3548685936 wait complete. 2024-04-25T13:41:36.532 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:41:36.543 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:41:36.543 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:41:36.543 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:41:36.543 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:41:36.576 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:41:36.576 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:41:36.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:36.576 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T13:41:36.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:36.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T13:41:36.651 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:36 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/502326560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:36.651 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:36 smithi094 ceph-mon[29195]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:36.656 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-04-25T13:41:36.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:36.658 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T13:41:36.658 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:41:36.658 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:36.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:41:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:36 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/502326560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:36.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:36 smithi137 ceph-mon[32568]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:41:36.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:36 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/502326560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:36.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:36 smithi138 ceph-mon[32977]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T13:41:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:38 smithi137 ceph-mon[32568]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:38.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:38 smithi094 ceph-mon[29195]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:38.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 -- 172.21.15.94:0/3421009679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330108dc0 msgr2=0x7f93301091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 --2- 172.21.15.94:0/3421009679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330108dc0 0x7f93301091d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93240099c0 tx=0x7f9324009cd0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 -- 172.21.15.94:0/3421009679 shutdown_connections 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 --2- 172.21.15.94:0/3421009679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330109710 0x7f9330111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 --2- 172.21.15.94:0/3421009679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330104490 0x7f93301048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 --2- 172.21.15.94:0/3421009679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330108dc0 0x7f93301091d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 -- 172.21.15.94:0/3421009679 >> 172.21.15.94:0/3421009679 conn(0x7f93300ffda0 msgr2=0x7f93301021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 -- 172.21.15.94:0/3421009679 shutdown_connections 2024-04-25T13:41:38.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.667+0000 7f9335061700 1 -- 172.21.15.94:0/3421009679 wait complete. 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 Processor -- start 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 -- start start 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330108dc0 0x7f9330076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330109710 0x7f9330077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9330114390 con 0x7f9330104490 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9330114210 con 0x7f9330109710 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.668+0000 7f9335061700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9330114510 con 0x7f9330108dc0 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35018/0 (socket says 172.21.15.94:35018) 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 -- 172.21.15.94:0/953235661 learned_addr learned my addr 172.21.15.94:0/953235661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932f59e700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330109710 0x7f9330077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932e59c700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330108dc0 0x7f9330076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 -- 172.21.15.94:0/953235661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330108dc0 msgr2=0x7f9330076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330108dc0 0x7f9330076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 -- 172.21.15.94:0/953235661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330109710 msgr2=0x7f9330077080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330109710 0x7f9330077080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 -- 172.21.15.94:0/953235661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300778b0 con 0x7f9330104490 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f932ed9d700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f932000c920 tx=0x7f932000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320007b00 con 0x7f9330104490 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93301ad810 con 0x7f9330104490 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9320004ce0 con 0x7f9330104490 2024-04-25T13:41:38.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320005500 con 0x7f9330104490 2024-04-25T13:41:38.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.669+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93301add70 con 0x7f9330104490 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.670+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330105940 con 0x7f9330104490 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.672+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9320007ca0 con 0x7f9330104490 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.672+0000 7f9317fff700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 0x7f93180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.673+0000 7f932e59c700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 0x7f93180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.673+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f93200deba0 con 0x7f9330104490 2024-04-25T13:41:38.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.673+0000 7f932e59c700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 0x7f93180b2090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f932401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.676+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9320014030 con 0x7f9330104490 2024-04-25T13:41:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:38 smithi138 ceph-mon[32977]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T13:41:38.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.924+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f933002e510 con 0x7f9330104490 2024-04-25T13:41:38.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.925+0000 7f9317fff700 1 -- 172.21.15.94:0/953235661 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f93200a7760 con 0x7f9330104490 2024-04-25T13:41:38.925 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:41:38.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 msgr2=0x7f93180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 0x7f93180b2090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f932401a040 comp rx=0 tx=0).stop 2024-04-25T13:41:38.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 msgr2=0x7f9330078640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f932000c920 tx=0x7f932000cc30 comp rx=0 tx=0).stop 2024-04-25T13:41:38.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 shutdown_connections 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9330109710 0x7f9330077080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93180afb80 0x7f93180b2090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9330104490 0x7f9330078640 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 --2- 172.21.15.94:0/953235661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9330108dc0 0x7f9330076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 >> 172.21.15.94:0/953235661 conn(0x7f93300ffda0 msgr2=0x7f933010a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 shutdown_connections 2024-04-25T13:41:38.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:38.928+0000 7f9335061700 1 -- 172.21.15.94:0/953235661 wait complete. 2024-04-25T13:41:39.627 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:39 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/953235661' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:41:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:39 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/953235661' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:41:39.655 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:41:39.656 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:41:39.656 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:41:39.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:39.656 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T13:41:39.672 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:41:39.672 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:41:39.673 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::9d70:8b0e:721a:ae75/64 scope link noprefixroute 2024-04-25T13:41:39.674 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:39.674 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:41:39.674 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:41:39.675 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T13:41:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/953235661' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:41:39.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:39.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:39.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:39.858 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:41:39.881 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T13:41:39.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:39.883 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T13:41:39.968 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T13:41:39.968 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:41:39.968 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:41:39.968 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:41:39.968 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:41:39.969 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T13:41:40.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:40.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:40.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:40.197 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T13:41:40.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:40.213 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T13:41:40.268 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:40.268 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T13:41:40.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:40.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:41:40.268 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T13:41:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:40.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:40.390 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:41:40.391 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T13:41:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:40.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:41:40.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T13:41:40.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T13:41:40.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:40 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]: dispatch 2024-04-25T13:41:40.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:40 smithi094 ceph-mon[29195]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:40.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:40.522 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:40.522 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T13:41:40.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T13:41:40.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:41:40.522 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T13:41:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:40.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:40.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:40.598 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:40.612 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:41:40.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:40.613 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:41:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T13:41:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T13:41:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:40 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]: dispatch 2024-04-25T13:41:40.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:40 smithi137 ceph-mon[32568]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:40.667 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:41:40.667 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:41:40.667 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/76de3e12-c8d1-4ba0-beb9-97fd0823e7ee 2024-04-25T13:41:40.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:40.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/76de3e12-c8d1-4ba0-beb9-97fd0823e7ee /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-25T13:41:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:41:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T13:41:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T13:41:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:40 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]: dispatch 2024-04-25T13:41:40.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:40 smithi138 ceph-mon[32977]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T13:41:40.763 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-25T13:41:40.764 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-25T13:41:40.764 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.137:3300,172.21.15.138:3300" 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:3300,172.21.15.137:3300,172.21.15.138:3300:/volumes/qa/sv_1/76de3e12-c8d1-4ba0-beb9-97fd0823e7ee 2024-04-25T13:41:40.879 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=f9021454-0307-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T13:41:40.880 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:41:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:41:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:41:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:41:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:41:40.882 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:41:40.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:40.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:40.916 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:41:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:41:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:client15348 2024-04-25T13:41:41.013 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:41:41.013 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:41.029 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:41:41.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:41:41.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:41.125 INFO:teuthology.orchestra.run.smithi094.stdout:client15348 2024-04-25T13:41:41.125 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:41:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:41:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:41.180 INFO:teuthology.orchestra.run.smithi094.stdout:client15348 2024-04-25T13:41:41.181 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f9021454-0307-11ef-bc93-c7b262605968.client15348/status') 2024-04-25T13:41:41.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:41.238 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.15348 (3)192.168.0.1:0/3067204020 2024-04-25T13:41:41.238 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-04-25T13:41:41.239 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-04-25T13:41:41.239 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-04-25T13:41:41.239 INFO:teuthology.orchestra.run.smithi094.stderr:70 bytes copied, 2.2645e-05 s, 3.1 MB/s 2024-04-25T13:41:41.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:41.246 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:41:41.256 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:41:41.256 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T13:41:41.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 -- 172.21.15.94:0/2966999410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c103c60 msgr2=0x7f340c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 --2- 172.21.15.94:0/2966999410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c103c60 0x7f340c104070 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009900 tx=0x7f33f402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 -- 172.21.15.94:0/2966999410 shutdown_connections 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 --2- 172.21.15.94:0/2966999410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 --2- 172.21.15.94:0/2966999410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c103c60 0x7f340c104070 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 --2- 172.21.15.94:0/2966999410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c102a20 0x7f340c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 -- 172.21.15.94:0/2966999410 >> 172.21.15.94:0/2966999410 conn(0x7f340c0fdc10 msgr2=0x7f340c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 -- 172.21.15.94:0/2966999410 shutdown_connections 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.371+0000 7f341196c640 1 -- 172.21.15.94:0/2966999410 wait complete. 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 Processor -- start 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 -- start start 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c103c60 0x7f340c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10e520 con 0x7f340c102a20 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10e3a0 con 0x7f340c103c60 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f341196c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10e6a0 con 0x7f340c108960 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f340affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f340affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35030/0 (socket says 172.21.15.94:35030) 2024-04-25T13:41:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f340affd640 1 -- 172.21.15.94:0/2877338918 learned_addr learned my addr 172.21.15.94:0/2877338918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f3403fff640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c103c60 0x7f340c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.372+0000 7f340b7fe640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 -- 172.21.15.94:0/2877338918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 msgr2=0x7f340c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 -- 172.21.15.94:0/2877338918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c103c60 msgr2=0x7f340c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c103c60 0x7f340c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 -- 172.21.15.94:0/2877338918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c1a1d40 con 0x7f340c102a20 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340b7fe640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f340affd640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d740 tx=0x7f33fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc017070 con 0x7f340c102a20 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33fc00de50 con 0x7f340c102a20 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c1a1fd0 con 0x7f340c102a20 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003c20 con 0x7f340c102a20 2024-04-25T13:41:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.373+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c1a2490 con 0x7f340c102a20 2024-04-25T13:41:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.374+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c107770 con 0x7f340c102a20 2024-04-25T13:41:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.375+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f33fc003dc0 con 0x7f340c102a20 2024-04-25T13:41:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.376+0000 7f341096a640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 0x7f33f80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.376+0000 7f3403fff640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 0x7f33f80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.376+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f33fc0deac0 con 0x7f340c102a20 2024-04-25T13:41:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.377+0000 7f3403fff640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 0x7f33f80b1ff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009230 tx=0x7f33f4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:41.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.380+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33fc0a7600 con 0x7f340c102a20 2024-04-25T13:41:41.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:41.617+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f340c02db50 con 0x7f340c102a20 2024-04-25T13:41:41.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:41:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T13:41:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T13:41:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]': finished 2024-04-25T13:41:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:41 smithi137 ceph-mon[32568]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:41:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:41:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T13:41:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T13:41:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]': finished 2024-04-25T13:41:41.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:41 smithi094 ceph-mon[29195]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:41:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:41:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T13:41:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T13:41:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 7]}]': finished 2024-04-25T13:41:41.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:41 smithi138 ceph-mon[32977]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T13:41:42.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.366+0000 7f341096a640 1 -- 172.21.15.94:0/2877338918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f33fc0c9020 con 0x7f340c102a20 2024-04-25T13:41:42.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 msgr2=0x7f33f80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 0x7f33f80b1ff0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009230 tx=0x7f33f4031400 comp rx=0 tx=0).stop 2024-04-25T13:41:42.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 msgr2=0x7f340c06ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d740 tx=0x7f33fc00dc00 comp rx=0 tx=0).stop 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 shutdown_connections 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33f80afae0 0x7f33f80b1ff0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f340c108960 0x7f340c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f340c103c60 0x7f340c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 --2- 172.21.15.94:0/2877338918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f340c102a20 0x7f340c06ae30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 >> 172.21.15.94:0/2877338918 conn(0x7f340c0fdc10 msgr2=0x7f340c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 shutdown_connections 2024-04-25T13:41:42.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.370+0000 7f341196c640 1 -- 172.21.15.94:0/2877338918 wait complete. 2024-04-25T13:41:42.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T13:41:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 -- 172.21.15.94:0/3276569647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c103c60 msgr2=0x7fbe5c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 --2- 172.21.15.94:0/3276569647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c103c60 0x7fbe5c104070 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009900 tx=0x7fbe4c02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 -- 172.21.15.94:0/3276569647 shutdown_connections 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 --2- 172.21.15.94:0/3276569647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe5c108960 0x7fbe5c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 --2- 172.21.15.94:0/3276569647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c103c60 0x7fbe5c104070 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 --2- 172.21.15.94:0/3276569647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe5c102a20 0x7fbe5c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 -- 172.21.15.94:0/3276569647 >> 172.21.15.94:0/3276569647 conn(0x7fbe5c0fdc10 msgr2=0x7fbe5c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 -- 172.21.15.94:0/3276569647 shutdown_connections 2024-04-25T13:41:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.501+0000 7fbe61357640 1 -- 172.21.15.94:0/3276569647 wait complete. 2024-04-25T13:41:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 Processor -- start 2024-04-25T13:41:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 -- start start 2024-04-25T13:41:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe5c103c60 0x7fbe5c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe5c108960 0x7fbe5c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10f650 con 0x7fbe5c102a20 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10f4d0 con 0x7fbe5c103c60 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe61357640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10f7d0 con 0x7fbe5c108960 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35042/0 (socket says 172.21.15.94:35042) 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 -- 172.21.15.94:0/606168829 learned_addr learned my addr 172.21.15.94:0/606168829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 -- 172.21.15.94:0/606168829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe5c108960 msgr2=0x7fbe5c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe5c108960 0x7fbe5c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 -- 172.21.15.94:0/606168829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe5c103c60 msgr2=0x7fbe5c19a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe5c103c60 0x7fbe5c19a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.502+0000 7fbe5affd640 1 -- 172.21.15.94:0/606168829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c1a1da0 con 0x7fbe5c102a20 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe5affd640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe50005d00 tx=0x7fbe5000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50017070 con 0x7fbe5c102a20 2024-04-25T13:41:42.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe50009c30 con 0x7fbe5c102a20 2024-04-25T13:41:42.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c1a2090 con 0x7fbe5c102a20 2024-04-25T13:41:42.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f620 con 0x7fbe5c102a20 2024-04-25T13:41:42.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.503+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a2590 con 0x7fbe5c102a20 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.504+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c107770 con 0x7fbe5c102a20 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.505+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbe5001a070 con 0x7fbe5c102a20 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.506+0000 7fbe43fff640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 0x7fbe440b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.506+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbe500de7d0 con 0x7fbe5c102a20 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.506+0000 7fbe5a7fc640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 0x7fbe440b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.506+0000 7fbe5a7fc640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 0x7fbe440b1ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02a930 tx=0x7fbe4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.511+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe50009de0 con 0x7fbe5c102a20 2024-04-25T13:41:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:42 smithi137 ceph-mon[32568]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:41:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:42 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:41:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:42 smithi137 ceph-mon[32568]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T13:41:42.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.718+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbe5c063140 con 0x7fbe440afae0 2024-04-25T13:41:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-mon[29195]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:41:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:41:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-mon[29195]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T13:41:42.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.736+0000 7fbe43fff640 1 -- 172.21.15.94:0/606168829 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fbe5c063140 con 0x7fbe440afae0 2024-04-25T13:41:42.737 INFO:teuthology.orchestra.run.smithi094.stdout:{} 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 msgr2=0x7fbe440b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 0x7fbe440b1ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c02a930 tx=0x7fbe4c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 msgr2=0x7fbe5c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe50005d00 tx=0x7fbe5000dde0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 shutdown_connections 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe440afae0 0x7fbe440b1ff0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe5c108960 0x7fbe5c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe5c103c60 0x7fbe5c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 --2- 172.21.15.94:0/606168829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe5c102a20 0x7fbe5c19a260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 >> 172.21.15.94:0/606168829 conn(0x7fbe5c0fdc10 msgr2=0x7fbe5c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 shutdown_connections 2024-04-25T13:41:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.738+0000 7fbe61357640 1 -- 172.21.15.94:0/606168829 wait complete. 2024-04-25T13:41:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:42 smithi138 ceph-mon[32977]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:41:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:42 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T13:41:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:42 smithi138 ceph-mon[32977]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T13:41:42.749 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:41:42.761 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T13:41:42.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.890+0000 7f6b30a34640 1 -- 172.21.15.94:0/1368705424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 msgr2=0x7f6b2c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.890+0000 7f6b30a34640 1 --2- 172.21.15.94:0/1368705424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c102e80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009900 tx=0x7f6b2402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 -- 172.21.15.94:0/1368705424 shutdown_connections 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 --2- 172.21.15.94:0/1368705424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1080c0 0x7f6b2c10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 --2- 172.21.15.94:0/1368705424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 --2- 172.21.15.94:0/1368705424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c102e80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 -- 172.21.15.94:0/1368705424 >> 172.21.15.94:0/1368705424 conn(0x7f6b2c0fd3b0 msgr2=0x7f6b2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 -- 172.21.15.94:0/1368705424 shutdown_connections 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 -- 172.21.15.94:0/1368705424 wait complete. 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 Processor -- start 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.891+0000 7f6b30a34640 1 -- start start 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1080c0 0x7f6b2c1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10dc70 con 0x7f6b2c102180 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10daf0 con 0x7f6b2c1033c0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b30a34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10ddf0 con 0x7f6b2c1080c0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35090/0 (socket says 172.21.15.94:35090) 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 -- 172.21.15.94:0/2916234758 learned_addr learned my addr 172.21.15.94:0/2916234758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b29d9b640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 -- 172.21.15.94:0/2916234758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1080c0 msgr2=0x7f6b2c1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2ad9d640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1080c0 0x7f6b2c1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1080c0 0x7f6b2c1a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 -- 172.21.15.94:0/2916234758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 msgr2=0x7f6b2c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 -- 172.21.15.94:0/2916234758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1a1d20 con 0x7f6b2c102180 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b29d9b640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b2a59c640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6b24032fd0 tx=0x7f6b240042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.892+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403f070 con 0x7f6b2c102180 2024-04-25T13:41:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.893+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b240044d0 con 0x7f6b2c102180 2024-04-25T13:41:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.893+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a1fb0 con 0x7f6b2c102180 2024-04-25T13:41:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.893+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24035070 con 0x7f6b2c102180 2024-04-25T13:41:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.893+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a23b0 con 0x7f6b2c102180 2024-04-25T13:41:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.893+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c1077c0 con 0x7f6b2c102180 2024-04-25T13:41:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.895+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6b2403b5f0 con 0x7f6b2c102180 2024-04-25T13:41:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.895+0000 7f6b1b7fe640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 0x7f6b140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.896+0000 7f6b29d9b640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 0x7f6b140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.896+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b240ff990 con 0x7f6b2c102180 2024-04-25T13:41:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.896+0000 7f6b29d9b640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 0x7f6b140b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c109970 tx=0x7f6b20008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:42.900+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b240c84d0 con 0x7f6b2c102180 2024-04-25T13:41:42.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:41:42.723+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:41:42.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:41:42.723+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:41:42.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:41:42.723+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:41:42.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:41:42.723+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:41:42.975 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 13:41:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T13:41:42.723+0000 7f3e9715e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:41:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.078+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b2c02dd20 con 0x7f6b2c102180 2024-04-25T13:41:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.080+0000 7f6b1b7fe640 1 -- 172.21.15.94:0/2916234758 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22958 (secure 0 0 0) 0x7f6b240c7dd0 con 0x7f6b2c102180 2024-04-25T13:41:43.080 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:43.080 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":83,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:41:41.352560+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:43.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 msgr2=0x7f6b140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 0x7f6b140b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c109970 tx=0x7f6b20008370 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 msgr2=0x7f6b2c109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6b24032fd0 tx=0x7f6b240042e0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 shutdown_connections 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b140afaf0 0x7f6b140b2000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c1080c0 0x7f6b2c1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b2c1033c0 0x7f6b2c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 --2- 172.21.15.94:0/2916234758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2c102180 0x7f6b2c109fd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 >> 172.21.15.94:0/2916234758 conn(0x7f6b2c0fd3b0 msgr2=0x7f6b2c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 shutdown_connections 2024-04-25T13:41:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.082+0000 7f6b30a34640 1 -- 172.21.15.94:0/2916234758 wait complete. 2024-04-25T13:41:43.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:43.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:43.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- 172.21.15.94:0/2349232162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 msgr2=0x7f3fcc1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2349232162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc1054b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009900 tx=0x7f3fbc02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:43.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- 172.21.15.94:0/2349232162 shutdown_connections 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2349232162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc102250 0x7f3fcc10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2349232162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc101750 0x7f3fcc101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2349232162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc1054b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- 172.21.15.94:0/2349232162 >> 172.21.15.94:0/2349232162 conn(0x7f3fcc0fd350 msgr2=0x7f3fcc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- 172.21.15.94:0/2349232162 shutdown_connections 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- 172.21.15.94:0/2349232162 wait complete. 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 Processor -- start 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.206+0000 7f3fd2442640 1 -- start start 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc101750 0x7f3fcc104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc102250 0x7f3fcc1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc10f610 con 0x7f3fcc1050a0 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc10f490 con 0x7f3fcc101750 2024-04-25T13:41:43.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc10f790 con 0x7f3fcc102250 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35114/0 (socket says 172.21.15.94:35114) 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 -- 172.21.15.94:0/2411390658 learned_addr learned my addr 172.21.15.94:0/2411390658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fcbfff640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc101750 0x7f3fcc104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fcb7fe640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc102250 0x7f3fcc1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 -- 172.21.15.94:0/2411390658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc102250 msgr2=0x7f3fcc1028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc102250 0x7f3fcc1028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 -- 172.21.15.94:0/2411390658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc101750 msgr2=0x7f3fcc104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc101750 0x7f3fcc104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 -- 172.21.15.94:0/2411390658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc103560 con 0x7f3fcc1050a0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd09df640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400cad0 tx=0x7f3fc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400ae20 con 0x7f3fcc1050a0 2024-04-25T13:41:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc4005020 con 0x7f3fcc1050a0 2024-04-25T13:41:43.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc1a4330 con 0x7f3fcc1050a0 2024-04-25T13:41:43.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400f6b0 con 0x7f3fcc1050a0 2024-04-25T13:41:43.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.207+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc1a47f0 con 0x7f3fcc1050a0 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.208+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc108a30 con 0x7f3fcc1050a0 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.211+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3fc40051d0 con 0x7f3fcc1050a0 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.211+0000 7f3fc97fa640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 0x7f3fb40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.211+0000 7f3fcbfff640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 0x7f3fb40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.211+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f3fc40df080 con 0x7f3fcc1050a0 2024-04-25T13:41:43.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.212+0000 7f3fcbfff640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 0x7f3fb40b2080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005ab0 tx=0x7f3fbc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.215+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fc4014030 con 0x7f3fcc1050a0 2024-04-25T13:41:43.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.396+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3fcc0637a0 con 0x7f3fcc1050a0 2024-04-25T13:41:43.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.396+0000 7f3fc97fa640 1 -- 172.21.15.94:0/2411390658 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f3fc40a7c40 con 0x7f3fcc1050a0 2024-04-25T13:41:43.397 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 msgr2=0x7f3fb40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 0x7f3fb40b2080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005ab0 tx=0x7f3fbc031400 comp rx=0 tx=0).stop 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 msgr2=0x7f3fcc102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400cad0 tx=0x7f3fc400cf90 comp rx=0 tx=0).stop 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 shutdown_connections 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3fb40afb70 0x7f3fb40b2080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fcc1050a0 0x7f3fcc102e30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fcc102250 0x7f3fcc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 --2- 172.21.15.94:0/2411390658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc101750 0x7f3fcc104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.398+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 >> 172.21.15.94:0/2411390658 conn(0x7f3fcc0fd350 msgr2=0x7f3fcc105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.399+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 shutdown_connections 2024-04-25T13:41:43.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.399+0000 7f3fd2442640 1 -- 172.21.15.94:0/2411390658 wait complete. 2024-04-25T13:41:43.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- 172.21.15.94:0/615706839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090104d30 msgr2=0x7ff090105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 --2- 172.21.15.94:0/615706839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090104d30 0x7ff090105140 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff084009900 tx=0x7ff08402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- 172.21.15.94:0/615706839 shutdown_connections 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 --2- 172.21.15.94:0/615706839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090109a30 0x7ff09010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 --2- 172.21.15.94:0/615706839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090104d30 0x7ff090105140 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 --2- 172.21.15.94:0/615706839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090103af0 0x7ff0901047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- 172.21.15.94:0/615706839 >> 172.21.15.94:0/615706839 conn(0x7ff0900fece0 msgr2=0x7ff090101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- 172.21.15.94:0/615706839 shutdown_connections 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- 172.21.15.94:0/615706839 wait complete. 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 Processor -- start 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.521+0000 7ff0973e3640 1 -- start start 2024-04-25T13:41:43.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 0x7ff09019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090109a30 0x7ff0901a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010f610 con 0x7ff090103af0 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010f490 con 0x7ff090104d30 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010f790 con 0x7ff090109a30 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53982/0 (socket says 172.21.15.94:53982) 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 -- 172.21.15.94:0/1368830262 learned_addr learned my addr 172.21.15.94:0/1368830262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff095980640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090109a30 0x7ff0901a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09497e640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 0x7ff09019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 -- 172.21.15.94:0/1368830262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090109a30 msgr2=0x7ff0901a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090109a30 0x7ff0901a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 -- 172.21.15.94:0/1368830262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 msgr2=0x7ff09019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 0x7ff09019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 -- 172.21.15.94:0/1368830262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a1cd0 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09497e640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 0x7ff09019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff09517f640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ead0 tx=0x7ff08800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08800cde0 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff088003e20 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a1fc0 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.522+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880116d0 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.523+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a2420 con 0x7ff090103af0 2024-04-25T13:41:43.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.523+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff090108840 con 0x7ff090103af0 2024-04-25T13:41:43.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.525+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff088004270 con 0x7ff090103af0 2024-04-25T13:41:43.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.526+0000 7ff08e7fc640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 0x7ff0780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.526+0000 7ff09497e640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 0x7ff0780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.526+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff0880df3a0 con 0x7ff090103af0 2024-04-25T13:41:43.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.526+0000 7ff09497e640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 0x7ff0780b2000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff084006010 tx=0x7ff084000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.530+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff088015030 con 0x7ff090103af0 2024-04-25T13:41:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:41:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:43 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:41:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:43 smithi137 ceph-mon[32568]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:43.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:43 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2916234758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:43.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.709+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff09002dda0 con 0x7ff090103af0 2024-04-25T13:41:43.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.709+0000 7ff08e7fc640 1 -- 172.21.15.94:0/1368830262 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7ff0880a7f70 con 0x7ff090103af0 2024-04-25T13:41:43.709 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 msgr2=0x7ff0780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 0x7ff0780b2000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff084006010 tx=0x7ff084000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 msgr2=0x7ff09019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff08800ead0 tx=0x7ff08800ef90 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 shutdown_connections 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff0780afaf0 0x7ff0780b2000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff090109a30 0x7ff0901a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff090104d30 0x7ff09019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 --2- 172.21.15.94:0/1368830262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff090103af0 0x7ff09019a240 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 >> 172.21.15.94:0/1368830262 conn(0x7ff0900fece0 msgr2=0x7ff090105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 shutdown_connections 2024-04-25T13:41:43.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.711+0000 7ff0973e3640 1 -- 172.21.15.94:0/1368830262 wait complete. 2024-04-25T13:41:43.721 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:41:43.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:43 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:41:43.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:43 smithi094 ceph-mon[29195]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:43.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:43 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2916234758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:43.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2877338918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T13:41:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:43 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:41:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:43 smithi138 ceph-mon[32977]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:41:43.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2916234758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:43.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 -- 172.21.15.94:0/1388849415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044104d30 msgr2=0x7f7044105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 --2- 172.21.15.94:0/1388849415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044104d30 0x7f7044105140 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7034009900 tx=0x7f703402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 -- 172.21.15.94:0/1388849415 shutdown_connections 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 --2- 172.21.15.94:0/1388849415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044109a30 0x7f704410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 --2- 172.21.15.94:0/1388849415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044104d30 0x7f7044105140 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 --2- 172.21.15.94:0/1388849415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044103af0 0x7f70441047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 -- 172.21.15.94:0/1388849415 >> 172.21.15.94:0/1388849415 conn(0x7f70440fece0 msgr2=0x7f7044101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 -- 172.21.15.94:0/1388849415 shutdown_connections 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.833+0000 7f704a95f640 1 -- 172.21.15.94:0/1388849415 wait complete. 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 Processor -- start 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 -- start start 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044104d30 0x7f704419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044109a30 0x7f70441a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f704410f620 con 0x7f7044103af0 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f704410f4a0 con 0x7f7044109a30 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f704a95f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f704410f7a0 con 0x7f7044104d30 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f7043fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f7043fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53994/0 (socket says 172.21.15.94:53994) 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.834+0000 7f7043fff640 1 -- 172.21.15.94:0/3080871206 learned_addr learned my addr 172.21.15.94:0/3080871206 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7048efc640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044109a30 0x7f70441a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f70437fe640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044104d30 0x7f704419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 -- 172.21.15.94:0/3080871206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044104d30 msgr2=0x7f704419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044104d30 0x7f704419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 -- 172.21.15.94:0/3080871206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044109a30 msgr2=0x7f70441a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044109a30 0x7f70441a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 -- 172.21.15.94:0/3080871206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441a1d20 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f7043fff640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f703800c970 tx=0x7f703800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038004220 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70380043c0 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441a1fb0 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800f450 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.835+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441a2470 con 0x7f7044103af0 2024-04-25T13:41:43.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.836+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7044108840 con 0x7f7044103af0 2024-04-25T13:41:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.837+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f70380079b0 con 0x7f7044103af0 2024-04-25T13:41:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.838+0000 7f70417fa640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 0x7f702c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.838+0000 7f70437fe640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 0x7f702c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.838+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f70380dee70 con 0x7f7044103af0 2024-04-25T13:41:43.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.838+0000 7f70437fe640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 0x7f702c0b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f703402a930 tx=0x7f7034000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:43.842+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7038014030 con 0x7f7044103af0 2024-04-25T13:41:44.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.021+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f704402dce0 con 0x7f7044103af0 2024-04-25T13:41:44.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.021+0000 7f70417fa640 1 -- 172.21.15.94:0/3080871206 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f70380a79b0 con 0x7f7044103af0 2024-04-25T13:41:44.022 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 msgr2=0x7f702c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 0x7f702c0b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f703402a930 tx=0x7f7034000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 msgr2=0x7f704419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f703800c970 tx=0x7f703800ce30 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 shutdown_connections 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f702c0afaf0 0x7f702c0b2000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7044109a30 0x7f70441a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7044104d30 0x7f704419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 --2- 172.21.15.94:0/3080871206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7044103af0 0x7f704419a1e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 >> 172.21.15.94:0/3080871206 conn(0x7f70440fece0 msgr2=0x7f7044105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 shutdown_connections 2024-04-25T13:41:44.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.023+0000 7f704a95f640 1 -- 172.21.15.94:0/3080871206 wait complete. 2024-04-25T13:41:44.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 -- 172.21.15.94:0/720585725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 msgr2=0x7f693c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 --2- 172.21.15.94:0/720585725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c10be40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6934009ca0 tx=0x7f693402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 -- 172.21.15.94:0/720585725 shutdown_connections 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 --2- 172.21.15.94:0/720585725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c10be40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 --2- 172.21.15.94:0/720585725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c104d20 0x7f693c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 --2- 172.21.15.94:0/720585725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c103ae0 0x7f693c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 -- 172.21.15.94:0/720585725 >> 172.21.15.94:0/720585725 conn(0x7f693c0fed10 msgr2=0x7f693c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 -- 172.21.15.94:0/720585725 shutdown_connections 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.143+0000 7f6941a34640 1 -- 172.21.15.94:0/720585725 wait complete. 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 Processor -- start 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 -- start start 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 0x7f693c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c104d20 0x7f693c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10f610 con 0x7f693c109a20 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10f490 con 0x7f693c103ae0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f6941a34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10f790 con 0x7f693c104d20 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54020/0 (socket says 172.21.15.94:54020) 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 -- 172.21.15.94:0/3475765336 learned_addr learned my addr 172.21.15.94:0/3475765336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693a7fc640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c104d20 0x7f693c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693affd640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 0x7f693c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 -- 172.21.15.94:0/3475765336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c104d20 msgr2=0x7f693c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c104d20 0x7f693c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 -- 172.21.15.94:0/3475765336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 msgr2=0x7f693c19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 0x7f693c19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693b7fe640 1 -- 172.21.15.94:0/3475765336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1a1cd0 con 0x7f693c109a20 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.144+0000 7f693affd640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 0x7f693c19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f693b7fe640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f69340097b0 tx=0x7f6934003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934004240 con 0x7f693c109a20 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693402f380 con 0x7f693c109a20 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a1f60 con 0x7f693c109a20 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693402fc30 con 0x7f693c109a20 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.145+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a23c0 con 0x7f693c109a20 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.146+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c06a190 con 0x7f693c109a20 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.148+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f69340043e0 con 0x7f693c109a20 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.148+0000 7f6940a32640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 0x7f69240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.149+0000 7f693affd640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 0x7f69240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.149+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6934100570 con 0x7f693c109a20 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.149+0000 7f693affd640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 0x7f69240b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f693c1046e0 tx=0x7f6930005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.153+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69340c90b0 con 0x7f693c109a20 2024-04-25T13:41:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.334+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f693c0637a0 con 0x7f693c109a20 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.336+0000 7f6940a32640 1 -- 172.21.15.94:0/3475765336 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22958 (secure 0 0 0) 0x7f69340098b0 con 0x7f693c109a20 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":83,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:41:41.352560+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 msgr2=0x7f69240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 0x7f69240b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f693c1046e0 tx=0x7f6930005e70 comp rx=0 tx=0).stop 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 msgr2=0x7f693c1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f69340097b0 tx=0x7f6934003c20 comp rx=0 tx=0).stop 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 shutdown_connections 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69240afaf0 0x7f69240b2000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c109a20 0x7f693c1a1660 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c104d20 0x7f693c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 --2- 172.21.15.94:0/3475765336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f693c103ae0 0x7f693c19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 >> 172.21.15.94:0/3475765336 conn(0x7f693c0fed10 msgr2=0x7f693c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 shutdown_connections 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.338+0000 7f6941a34640 1 -- 172.21.15.94:0/3475765336 wait complete. 2024-04-25T13:41:44.350 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:44.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 -- 172.21.15.94:0/3009877288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc103c20 msgr2=0x7fa4fc104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 --2- 172.21.15.94:0/3009877288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc103c20 0x7fa4fc104030 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009900 tx=0x7fa4f002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 -- 172.21.15.94:0/3009877288 shutdown_connections 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 --2- 172.21.15.94:0/3009877288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc108920 0x7fa4fc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 --2- 172.21.15.94:0/3009877288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc103c20 0x7fa4fc104030 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 --2- 172.21.15.94:0/3009877288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.464+0000 7fa50226e640 1 -- 172.21.15.94:0/3009877288 >> 172.21.15.94:0/3009877288 conn(0x7fa4fc0fdbd0 msgr2=0x7fa4fc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 -- 172.21.15.94:0/3009877288 shutdown_connections 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 -- 172.21.15.94:0/3009877288 wait complete. 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 Processor -- start 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 -- start start 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc103c20 0x7fa4fc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc108920 0x7fa4fc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.465+0000 7fa50226e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10e500 con 0x7fa4fc1029e0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa50226e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10e380 con 0x7fa4fc103c20 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa50226e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10e680 con 0x7fa4fc108920 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54042/0 (socket says 172.21.15.94:54042) 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 -- 172.21.15.94:0/891751469 learned_addr learned my addr 172.21.15.94:0/891751469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fb7fe640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc103c20 0x7fa4fc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa50080b640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc108920 0x7fa4fc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 -- 172.21.15.94:0/891751469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc108920 msgr2=0x7fa4fc1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc108920 0x7fa4fc1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 -- 172.21.15.94:0/891751469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc103c20 msgr2=0x7fa4fc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc103c20 0x7fa4fc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 -- 172.21.15.94:0/891751469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc1a1de0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4fbfff640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9f0 tx=0x7fa4ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00ccc0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc1a20d0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ec0035f0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00ce60 con 0x7fa4fc1029e0 2024-04-25T13:41:44.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.466+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1a25d0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.467+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc107730 con 0x7fa4fc1029e0 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.469+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fa4ec003aa0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.470+0000 7fa4f97fa640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 0x7fa4e40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.470+0000 7fa4fb7fe640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 0x7fa4e40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.470+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa4ec0df2b0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.470+0000 7fa4fb7fe640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 0x7fa4e40b2090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009230 tx=0x7fa4f0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.474+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4ec0a7c90 con 0x7fa4fc1029e0 2024-04-25T13:41:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2411390658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1368830262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3080871206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:44 smithi137 ceph-mon[32568]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-25T13:41:44.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3475765336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:44.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.652+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4fc02dda0 con 0x7fa4fc1029e0 2024-04-25T13:41:44.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.653+0000 7fa4f97fa640 1 -- 172.21.15.94:0/891751469 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fa4ec0a7590 con 0x7fa4fc1029e0 2024-04-25T13:41:44.653 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 msgr2=0x7fa4e40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 0x7fa4e40b2090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009230 tx=0x7fa4f0031400 comp rx=0 tx=0).stop 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 msgr2=0x7fa4fc075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9f0 tx=0x7fa4ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 shutdown_connections 2024-04-25T13:41:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4e40afb80 0x7fa4e40b2090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4fc108920 0x7fa4fc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4fc103c20 0x7fa4fc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 --2- 172.21.15.94:0/891751469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4fc1029e0 0x7fa4fc075860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 >> 172.21.15.94:0/891751469 conn(0x7fa4fc0fdbd0 msgr2=0x7fa4fc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 shutdown_connections 2024-04-25T13:41:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.655+0000 7fa50226e640 1 -- 172.21.15.94:0/891751469 wait complete. 2024-04-25T13:41:44.665 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2411390658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.665 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1368830262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.665 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3080871206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:44 smithi094 ceph-mon[29195]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-25T13:41:44.666 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3475765336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:44.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2411390658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1368830262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3080871206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:44 smithi138 ceph-mon[32977]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-25T13:41:44.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3475765336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 -- 172.21.15.94:0/287433030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 msgr2=0x7fe3c8105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 --2- 172.21.15.94:0/287433030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c8105130 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009900 tx=0x7fe3bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 -- 172.21.15.94:0/287433030 shutdown_connections 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 --2- 172.21.15.94:0/287433030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c8109a20 0x7fe3c810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 --2- 172.21.15.94:0/287433030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c8105130 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 --2- 172.21.15.94:0/287433030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 0x7fe3c81047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 -- 172.21.15.94:0/287433030 >> 172.21.15.94:0/287433030 conn(0x7fe3c80fecd0 msgr2=0x7fe3c8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 -- 172.21.15.94:0/287433030 shutdown_connections 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 -- 172.21.15.94:0/287433030 wait complete. 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.778+0000 7fe3cd297640 1 Processor -- start 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 -- start start 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 0x7fe3c819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c8109a20 0x7fe3c81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810f5e0 con 0x7fe3c8104d20 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810f460 con 0x7fe3c8103ae0 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3cd297640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c810f760 con 0x7fe3c8109a20 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54048/0 (socket says 172.21.15.94:54048) 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 -- 172.21.15.94:0/3193392644 learned_addr learned my addr 172.21.15.94:0/3193392644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c6ffd640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 0x7fe3c819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 -- 172.21.15.94:0/3193392644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c8109a20 msgr2=0x7fe3c81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c8109a20 0x7fe3c81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 -- 172.21.15.94:0/3193392644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 msgr2=0x7fe3c819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 0x7fe3c819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 -- 172.21.15.94:0/3193392644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c81a1c90 con 0x7fe3c8104d20 2024-04-25T13:41:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.779+0000 7fe3c67fc640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc006010 tx=0x7fe3bc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.780+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc03f030 con 0x7fe3c8104d20 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.780+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3bc003cb0 con 0x7fe3c8104d20 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.780+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81a1f20 con 0x7fe3c8104d20 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.780+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc035070 con 0x7fe3c8104d20 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.780+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81a2380 con 0x7fe3c8104d20 2024-04-25T13:41:44.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.781+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c8108830 con 0x7fe3c8104d20 2024-04-25T13:41:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.782+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fe3bc042030 con 0x7fe3c8104d20 2024-04-25T13:41:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.783+0000 7fe3affff640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 0x7fe3b00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.783+0000 7fe3c6ffd640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 0x7fe3b00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.783+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe3bc0ffd20 con 0x7fe3c8104d20 2024-04-25T13:41:44.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.784+0000 7fe3c6ffd640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 0x7fe3b00b2090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8005db0 tx=0x7fe3b8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.787+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3bc0c8700 con 0x7fe3c8104d20 2024-04-25T13:41:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.966+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3c8063840 con 0x7fe3c8104d20 2024-04-25T13:41:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.966+0000 7fe3affff640 1 -- 172.21.15.94:0/3193392644 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fe3bc0c8000 con 0x7fe3c8104d20 2024-04-25T13:41:44.967 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 msgr2=0x7fe3b00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 0x7fe3b00b2090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8005db0 tx=0x7fe3b8005d40 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 msgr2=0x7fe3c819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc006010 tx=0x7fe3bc0043b0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 shutdown_connections 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3b00afb80 0x7fe3b00b2090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c8109a20 0x7fe3c81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3c8104d20 0x7fe3c819a7b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 --2- 172.21.15.94:0/3193392644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3c8103ae0 0x7fe3c819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 >> 172.21.15.94:0/3193392644 conn(0x7fe3c80fecd0 msgr2=0x7fe3c8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 shutdown_connections 2024-04-25T13:41:44.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:44.968+0000 7fe3cd297640 1 -- 172.21.15.94:0/3193392644 wait complete. 2024-04-25T13:41:44.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:45.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 -- 172.21.15.94:0/4022531554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c109890 msgr2=0x7f704c10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 --2- 172.21.15.94:0/4022531554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c109890 0x7f704c10bcb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7044009ca0 tx=0x7f704402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 -- 172.21.15.94:0/4022531554 shutdown_connections 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 --2- 172.21.15.94:0/4022531554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c109890 0x7f704c10bcb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 --2- 172.21.15.94:0/4022531554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c106f30 0x7f704c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 --2- 172.21.15.94:0/4022531554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c1045d0 0x7f704c1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 -- 172.21.15.94:0/4022531554 >> 172.21.15.94:0/4022531554 conn(0x7f704c0fdbd0 msgr2=0x7f704c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 -- 172.21.15.94:0/4022531554 shutdown_connections 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.090+0000 7f7052a34640 1 -- 172.21.15.94:0/4022531554 wait complete. 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 Processor -- start 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 -- start start 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c1045d0 0x7f704c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c109890 0x7f704c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10f5f0 con 0x7f704c106f30 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10f470 con 0x7f704c1045d0 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7052a34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10f770 con 0x7f704c109890 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54066/0 (socket says 172.21.15.94:54066) 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 -- 172.21.15.94:0/1214998719 learned_addr learned my addr 172.21.15.94:0/1214998719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f7050fd1640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c109890 0x7f704c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704bfff640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c1045d0 0x7f704c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 -- 172.21.15.94:0/1214998719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c109890 msgr2=0x7f704c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c109890 0x7f704c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 -- 172.21.15.94:0/1214998719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c1045d0 msgr2=0x7f704c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c1045d0 0x7f704c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.091+0000 7f704b7fe640 1 -- 172.21.15.94:0/1214998719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a1cf0 con 0x7f704c106f30 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f704b7fe640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c920 tx=0x7f703c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c007c70 con 0x7f704c106f30 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c007e10 con 0x7f704c106f30 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a1fe0 con 0x7f704c106f30 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c018820 con 0x7f704c106f30 2024-04-25T13:41:45.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.092+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a2450 con 0x7f704c106f30 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.093+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c0ff6f0 con 0x7f704c106f30 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.094+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f703c00f450 con 0x7f704c106f30 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.095+0000 7f70497fa640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 0x7f70380b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.095+0000 7f704bfff640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 0x7f70380b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.095+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f703c0df280 con 0x7f704c106f30 2024-04-25T13:41:45.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.095+0000 7f704bfff640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 0x7f70380b2090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7034005d00 tx=0x7f7034005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.099+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f703c014030 con 0x7f704c106f30 2024-04-25T13:41:45.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.278+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f704c02dce0 con 0x7f704c106f30 2024-04-25T13:41:45.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.279+0000 7f70497fa640 1 -- 172.21.15.94:0/1214998719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f703c0a7cf0 con 0x7f704c106f30 2024-04-25T13:41:45.279 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 msgr2=0x7f70380b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 0x7f70380b2090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7034005d00 tx=0x7f7034005c50 comp rx=0 tx=0).stop 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 msgr2=0x7f704c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c920 tx=0x7f703c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 shutdown_connections 2024-04-25T13:41:45.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70380afb80 0x7f70380b2090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704c109890 0x7f704c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f704c106f30 0x7f704c19a7a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 --2- 172.21.15.94:0/1214998719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f704c1045d0 0x7f704c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 >> 172.21.15.94:0/1214998719 conn(0x7f704c0fdbd0 msgr2=0x7f704c105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 shutdown_connections 2024-04-25T13:41:45.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.281+0000 7f7052a34640 1 -- 172.21.15.94:0/1214998719 wait complete. 2024-04-25T13:41:45.292 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T13:41:45.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 -- 172.21.15.94:0/1752089969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 msgr2=0x7f72a8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 --2- 172.21.15.94:0/1752089969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a8104010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7290009900 tx=0x7f729002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 -- 172.21.15.94:0/1752089969 shutdown_connections 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 --2- 172.21.15.94:0/1752089969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 0x7f72a810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 --2- 172.21.15.94:0/1752089969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a8104010 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 --2- 172.21.15.94:0/1752089969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 -- 172.21.15.94:0/1752089969 >> 172.21.15.94:0/1752089969 conn(0x7f72a80fdbd0 msgr2=0x7f72a8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 -- 172.21.15.94:0/1752089969 shutdown_connections 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.401+0000 7f72ad284640 1 -- 172.21.15.94:0/1752089969 wait complete. 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 Processor -- start 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 -- start start 2024-04-25T13:41:45.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 0x7f72a81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e4f0 con 0x7f72a8103c00 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e370 con 0x7f72a81029c0 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f72ad284640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810e670 con 0x7f72a8108900 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f729ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f729ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54084/0 (socket says 172.21.15.94:54084) 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.402+0000 7f729ffff640 1 -- 172.21.15.94:0/2736140735 learned_addr learned my addr 172.21.15.94:0/2736140735 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a6ffd640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a77fe640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 0x7f72a81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 -- 172.21.15.94:0/2736140735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 msgr2=0x7f72a81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 0x7f72a81a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 -- 172.21.15.94:0/2736140735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 msgr2=0x7f72a819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 -- 172.21.15.94:0/2736140735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a81a1d30 con 0x7f72a8103c00 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a6ffd640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f729ffff640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7290006010 tx=0x7f7290004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729003f030 con 0x7f72a8103c00 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7290003cb0 con 0x7f72a8103c00 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81a1fc0 con 0x7f72a8103c00 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290035070 con 0x7f72a8103c00 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.403+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81a23d0 con 0x7f72a8103c00 2024-04-25T13:41:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.404+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a81a2630 con 0x7f72a8103c00 2024-04-25T13:41:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.405+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f7290042030 con 0x7f72a8103c00 2024-04-25T13:41:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.406+0000 7f72a4ff9640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 0x7f72940b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.406+0000 7f72a6ffd640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 0x7f72940b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.407+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f72900ff5a0 con 0x7f72a8103c00 2024-04-25T13:41:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.407+0000 7f72a6ffd640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 0x7f72940b2090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72980058c0 tx=0x7f7298005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.412+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7290003e60 con 0x7f72a8103c00 2024-04-25T13:41:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.628+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a81a29c0 con 0x7f72a8103c00 2024-04-25T13:41:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.629+0000 7f72a4ff9640 1 -- 172.21.15.94:0/2736140735 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7f72900c7f80 con 0x7f72a8103c00 2024-04-25T13:41:45.630 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:45.630 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:41:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:41:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 msgr2=0x7f72940b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 0x7f72940b2090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72980058c0 tx=0x7f7298005850 comp rx=0 tx=0).stop 2024-04-25T13:41:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 msgr2=0x7f72a819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7290006010 tx=0x7f7290004340 comp rx=0 tx=0).stop 2024-04-25T13:41:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 shutdown_connections 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72940afb80 0x7f72940b2090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72a8108900 0x7f72a81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72a8103c00 0x7f72a819a730 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 --2- 172.21.15.94:0/2736140735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a81029c0 0x7f72a819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 >> 172.21.15.94:0/2736140735 conn(0x7f72a80fdbd0 msgr2=0x7f72a80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 shutdown_connections 2024-04-25T13:41:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.631+0000 7f72ad284640 1 -- 172.21.15.94:0/2736140735 wait complete. 2024-04-25T13:41:45.642 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:45 smithi094 ceph-mon[29195]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:41:45.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/891751469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3193392644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1214998719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.678 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:45 smithi137 ceph-mon[32568]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:41:45.678 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/891751469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.678 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3193392644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.678 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1214998719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.679 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T13:41:45.679 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T13:41:45.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:45 smithi138 ceph-mon[32977]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T13:41:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/891751469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3193392644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1214998719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 -- 172.21.15.94:0/2808186228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8102250 msgr2=0x7f82c810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/2808186228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8102250 0x7f82c810bcb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009ca0 tx=0x7f82c002a8f0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 -- 172.21.15.94:0/2808186228 shutdown_connections 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/2808186228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8102250 0x7f82c810bcb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/2808186228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8101750 0x7f82c8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/2808186228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c81050a0 0x7f82c81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 -- 172.21.15.94:0/2808186228 >> 172.21.15.94:0/2808186228 conn(0x7f82c80fd370 msgr2=0x7f82c80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 -- 172.21.15.94:0/2808186228 shutdown_connections 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.788+0000 7f82ce1b2640 1 -- 172.21.15.94:0/2808186228 wait complete. 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 Processor -- start 2024-04-25T13:41:45.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 -- start start 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c8102250 0x7f82c819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 0x7f82c81a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f720 con 0x7f82c8101750 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f5a0 con 0x7f82c81050a0 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82ce1b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f8a0 con 0x7f82c8102250 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54114/0 (socket says 172.21.15.94:54114) 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.789+0000 7f82c77fe640 1 -- 172.21.15.94:0/76325480 learned_addr learned my addr 172.21.15.94:0/76325480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c6ffd640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c8102250 0x7f82c819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c7fff640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 0x7f82c81a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 -- 172.21.15.94:0/76325480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c8102250 msgr2=0x7f82c819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c8102250 0x7f82c819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 -- 172.21.15.94:0/76325480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 msgr2=0x7f82c81a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 0x7f82c81a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 -- 172.21.15.94:0/76325480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c81a1e20 con 0x7f82c8101750 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c7fff640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 0x7f82c81a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c77fe640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b450 tx=0x7f82b800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8015020 con 0x7f82c8101750 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81a2110 con 0x7f82c8101750 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b8011d40 con 0x7f82c8101750 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b800f340 con 0x7f82c8101750 2024-04-25T13:41:45.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.790+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81a2690 con 0x7f82c8101750 2024-04-25T13:41:45.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.791+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c810aa40 con 0x7f82c8101750 2024-04-25T13:41:45.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.793+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f82b8018020 con 0x7f82c8101750 2024-04-25T13:41:45.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.794+0000 7f82c4ff9640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 0x7f82b00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:45.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.794+0000 7f82c6ffd640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 0x7f82b00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:45.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.794+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f82b80dee30 con 0x7f82c8101750 2024-04-25T13:41:45.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.794+0000 7f82c6ffd640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 0x7f82b00b2090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006fd0 tx=0x7f82bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:45.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.798+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82b801f020 con 0x7f82c8101750 2024-04-25T13:41:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.974+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82c8062fe0 con 0x7f82b00afb80 2024-04-25T13:41:45.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.981+0000 7f82c4ff9640 1 -- 172.21.15.94:0/76325480 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258902 (secure 0 0 0) 0x7f82c8062fe0 con 0x7f82b00afb80 2024-04-25T13:41:45.984 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 msgr2=0x7f82b00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 0x7f82b00b2090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006fd0 tx=0x7f82bc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 msgr2=0x7f82c819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b450 tx=0x7f82b800b910 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 shutdown_connections 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82b00afb80 0x7f82b00b2090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c81050a0 0x7f82c81a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c8102250 0x7f82c819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 --2- 172.21.15.94:0/76325480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c8101750 0x7f82c819a2e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.984+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 >> 172.21.15.94:0/76325480 conn(0x7f82c80fd370 msgr2=0x7f82c8107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:45.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.985+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 shutdown_connections 2024-04-25T13:41:45.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:45.985+0000 7f82ce1b2640 1 -- 172.21.15.94:0/76325480 wait complete. 2024-04-25T13:41:45.996 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T13:41:44.287520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657076,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1216,"num_read_kb":704,"num_write":371,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":807,"ondisk_log_size":807,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":411956,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529868,"statfs":{"total":1151940427776,"available":1151518584832,"internally_reserved":0,"allocated":13160448,"data_stored":7179230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":391,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":31,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001894"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.280697+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:41:10.280697+0000","last_peered":"2024-04-25T13:41:10.280697+0000","last_clean":"2024-04-25T13:41:10.280697+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:41:10.280697+0000","last_undegraded":"2024-04-25T13:41:10.280697+0000","last_fullsized":"2024-04-25T13:41:10.280697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362416+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:41:41.362416+0000","last_peered":"2024-04-25T13:41:41.362416+0000","last_clean":"2024-04-25T13:41:41.362416+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:41:41.362416+0000","last_undegraded":"2024-04-25T13:41:41.362416+0000","last_fullsized":"2024-04-25T13:41:41.362416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362698+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:41:41.362698+0000","last_peered":"2024-04-25T13:41:41.362698+0000","last_clean":"2024-04-25T13:41:41.362698+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:41:41.362698+0000","last_undegraded":"2024-04-25T13:41:41.362698+0000","last_fullsized":"2024-04-25T13:41:41.362698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.283426+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:41:10.283426+0000","last_peered":"2024-04-25T13:41:10.283426+0000","last_clean":"2024-04-25T13:41:10.283426+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:41:10.283426+0000","last_undegraded":"2024-04-25T13:41:10.283426+0000","last_fullsized":"2024-04-25T13:41:10.283426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446495+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:41:41.446495+0000","last_peered":"2024-04-25T13:41:41.446495+0000","last_clean":"2024-04-25T13:41:41.446495+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:41:41.446495+0000","last_undegraded":"2024-04-25T13:41:41.446495+0000","last_fullsized":"2024-04-25T13:41:41.446495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.362739+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:41:40.362739+0000","last_peered":"2024-04-25T13:41:40.362739+0000","last_clean":"2024-04-25T13:41:40.362739+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:41:40.362739+0000","last_undegraded":"2024-04-25T13:41:40.362739+0000","last_fullsized":"2024-04-25T13:41:40.362739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357446+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:41:41.357446+0000","last_peered":"2024-04-25T13:41:41.357446+0000","last_clean":"2024-04-25T13:41:41.357446+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:41:41.357446+0000","last_undegraded":"2024-04-25T13:41:41.357446+0000","last_fullsized":"2024-04-25T13:41:41.357446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444909+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:41:41.444909+0000","last_peered":"2024-04-25T13:41:41.444909+0000","last_clean":"2024-04-25T13:41:41.444909+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:41:41.444909+0000","last_undegraded":"2024-04-25T13:41:41.444909+0000","last_fullsized":"2024-04-25T13:41:41.444909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360334+0000","last_change":"2024-04-25T13:40:41.273152+0000","last_active":"2024-04-25T13:41:41.360334+0000","last_peered":"2024-04-25T13:41:41.360334+0000","last_clean":"2024-04-25T13:41:41.360334+0000","last_became_active":"2024-04-25T13:40:41.272760+0000","last_became_peered":"2024-04-25T13:40:41.272760+0000","last_unstale":"2024-04-25T13:41:41.360334+0000","last_undegraded":"2024-04-25T13:41:41.360334+0000","last_fullsized":"2024-04-25T13:41:41.360334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:35.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.363138+0000","last_change":"2024-04-25T13:40:41.274781+0000","last_active":"2024-04-25T13:41:40.363138+0000","last_peered":"2024-04-25T13:41:40.363138+0000","last_clean":"2024-04-25T13:41:40.363138+0000","last_became_active":"2024-04-25T13:40:41.274043+0000","last_became_peered":"2024-04-25T13:40:41.274043+0000","last_unstale":"2024-04-25T13:41:40.363138+0000","last_undegraded":"2024-04-25T13:41:40.363138+0000","last_fullsized":"2024-04-25T13:41:40.363138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:01.989580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358355+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:41:41.358355+0000","last_peered":"2024-04-25T13:41:41.358355+0000","last_clean":"2024-04-25T13:41:41.358355+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:41:41.358355+0000","last_undegraded":"2024-04-25T13:41:41.358355+0000","last_fullsized":"2024-04-25T13:41:41.358355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.289528+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:41:10.289528+0000","last_peered":"2024-04-25T13:41:10.289528+0000","last_clean":"2024-04-25T13:41:10.289528+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:41:10.289528+0000","last_undegraded":"2024-04-25T13:41:10.289528+0000","last_fullsized":"2024-04-25T13:41:10.289528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359749+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:41:41.359749+0000","last_peered":"2024-04-25T13:41:41.359749+0000","last_clean":"2024-04-25T13:41:41.359749+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:41:41.359749+0000","last_undegraded":"2024-04-25T13:41:41.359749+0000","last_fullsized":"2024-04-25T13:41:41.359749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362114+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:41:41.362114+0000","last_peered":"2024-04-25T13:41:41.362114+0000","last_clean":"2024-04-25T13:41:41.362114+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:41:41.362114+0000","last_undegraded":"2024-04-25T13:41:41.362114+0000","last_fullsized":"2024-04-25T13:41:41.362114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361807+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:41:41.361807+0000","last_peered":"2024-04-25T13:41:41.361807+0000","last_clean":"2024-04-25T13:41:41.361807+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:41:41.361807+0000","last_undegraded":"2024-04-25T13:41:41.361807+0000","last_fullsized":"2024-04-25T13:41:41.361807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358442+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:41:41.358442+0000","last_peered":"2024-04-25T13:41:41.358442+0000","last_clean":"2024-04-25T13:41:41.358442+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:41:41.358442+0000","last_undegraded":"2024-04-25T13:41:41.358442+0000","last_fullsized":"2024-04-25T13:41:41.358442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.280823+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:41:10.280823+0000","last_peered":"2024-04-25T13:41:10.280823+0000","last_clean":"2024-04-25T13:41:10.280823+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:41:10.280823+0000","last_undegraded":"2024-04-25T13:41:10.280823+0000","last_fullsized":"2024-04-25T13:41:10.280823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.287454+0000","last_change":"2024-04-25T13:39:45.259538+0000","last_active":"2024-04-25T13:41:10.287454+0000","last_peered":"2024-04-25T13:41:10.287454+0000","last_clean":"2024-04-25T13:41:10.287454+0000","last_became_active":"2024-04-25T13:39:45.259097+0000","last_became_peered":"2024-04-25T13:39:45.259097+0000","last_unstale":"2024-04-25T13:41:10.287454+0000","last_undegraded":"2024-04-25T13:41:10.287454+0000","last_fullsized":"2024-04-25T13:41:10.287454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:10.901989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359196+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:41:41.359196+0000","last_peered":"2024-04-25T13:41:41.359196+0000","last_clean":"2024-04-25T13:41:41.359196+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:41:41.359196+0000","last_undegraded":"2024-04-25T13:41:41.359196+0000","last_fullsized":"2024-04-25T13:41:41.359196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:37.803221+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:41:37.803221+0000","last_peered":"2024-04-25T13:41:37.803221+0000","last_clean":"2024-04-25T13:41:37.803221+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:41:37.803221+0000","last_undegraded":"2024-04-25T13:41:37.803221+0000","last_fullsized":"2024-04-25T13:41:37.803221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445375+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:41:41.445375+0000","last_peered":"2024-04-25T13:41:41.445375+0000","last_clean":"2024-04-25T13:41:41.445375+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:41:41.445375+0000","last_undegraded":"2024-04-25T13:41:41.445375+0000","last_fullsized":"2024-04-25T13:41:41.445375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445462+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:41:41.445462+0000","last_peered":"2024-04-25T13:41:41.445462+0000","last_clean":"2024-04-25T13:41:41.445462+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:41:41.445462+0000","last_undegraded":"2024-04-25T13:41:41.445462+0000","last_fullsized":"2024-04-25T13:41:41.445462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362671+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:41:41.362671+0000","last_peered":"2024-04-25T13:41:41.362671+0000","last_clean":"2024-04-25T13:41:41.362671+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:41:41.362671+0000","last_undegraded":"2024-04-25T13:41:41.362671+0000","last_fullsized":"2024-04-25T13:41:41.362671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446521+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:41:41.446521+0000","last_peered":"2024-04-25T13:41:41.446521+0000","last_clean":"2024-04-25T13:41:41.446521+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:41:41.446521+0000","last_undegraded":"2024-04-25T13:41:41.446521+0000","last_fullsized":"2024-04-25T13:41:41.446521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359128+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:41:41.359128+0000","last_peered":"2024-04-25T13:41:41.359128+0000","last_clean":"2024-04-25T13:41:41.359128+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:41:41.359128+0000","last_undegraded":"2024-04-25T13:41:41.359128+0000","last_fullsized":"2024-04-25T13:41:41.359128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446334+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:41:41.446334+0000","last_peered":"2024-04-25T13:41:41.446334+0000","last_clean":"2024-04-25T13:41:41.446334+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:41:41.446334+0000","last_undegraded":"2024-04-25T13:41:41.446334+0000","last_fullsized":"2024-04-25T13:41:41.446334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361645+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:41:41.361645+0000","last_peered":"2024-04-25T13:41:41.361645+0000","last_clean":"2024-04-25T13:41:41.361645+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:41:41.361645+0000","last_undegraded":"2024-04-25T13:41:41.361645+0000","last_fullsized":"2024-04-25T13:41:41.361645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.811056+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:41:41.811056+0000","last_peered":"2024-04-25T13:41:41.811056+0000","last_clean":"2024-04-25T13:41:41.811056+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:41:41.811056+0000","last_undegraded":"2024-04-25T13:41:41.811056+0000","last_fullsized":"2024-04-25T13:41:41.811056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446058+0000","last_change":"2024-04-25T13:40:41.272700+0000","last_active":"2024-04-25T13:41:41.446058+0000","last_peered":"2024-04-25T13:41:41.446058+0000","last_clean":"2024-04-25T13:41:41.446058+0000","last_became_active":"2024-04-25T13:40:41.272080+0000","last_became_peered":"2024-04-25T13:40:41.272080+0000","last_unstale":"2024-04-25T13:41:41.446058+0000","last_undegraded":"2024-04-25T13:41:41.446058+0000","last_fullsized":"2024-04-25T13:41:41.446058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.149458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.362596+0000","last_change":"2024-04-25T13:40:41.280199+0000","last_active":"2024-04-25T13:41:40.362596+0000","last_peered":"2024-04-25T13:41:40.362596+0000","last_clean":"2024-04-25T13:41:40.362596+0000","last_became_active":"2024-04-25T13:40:41.268773+0000","last_became_peered":"2024-04-25T13:40:41.268773+0000","last_unstale":"2024-04-25T13:41:40.362596+0000","last_undegraded":"2024-04-25T13:41:40.362596+0000","last_fullsized":"2024-04-25T13:41:40.362596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:47.024600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.447783+0000","last_change":"2024-04-25T13:41:41.447783+0000","last_active":"2024-04-25T13:41:41.447783+0000","last_peered":"2024-04-25T13:41:41.447783+0000","last_clean":"2024-04-25T13:41:41.447783+0000","last_became_active":"2024-04-25T13:41:41.447293+0000","last_became_peered":"2024-04-25T13:41:41.447293+0000","last_unstale":"2024-04-25T13:41:41.447783+0000","last_undegraded":"2024-04-25T13:41:41.447783+0000","last_fullsized":"2024-04-25T13:41:41.447783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:39.807786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359388+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:41:41.359388+0000","last_peered":"2024-04-25T13:41:41.359388+0000","last_clean":"2024-04-25T13:41:41.359388+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:41:41.359388+0000","last_undegraded":"2024-04-25T13:41:41.359388+0000","last_fullsized":"2024-04-25T13:41:41.359388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444835+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:41:41.444835+0000","last_peered":"2024-04-25T13:41:41.444835+0000","last_clean":"2024-04-25T13:41:41.444835+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:41:41.444835+0000","last_undegraded":"2024-04-25T13:41:41.444835+0000","last_fullsized":"2024-04-25T13:41:41.444835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360949+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:41:41.360949+0000","last_peered":"2024-04-25T13:41:41.360949+0000","last_clean":"2024-04-25T13:41:41.360949+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:41:41.360949+0000","last_undegraded":"2024-04-25T13:41:41.360949+0000","last_fullsized":"2024-04-25T13:41:41.360949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.287539+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:41:10.287539+0000","last_peered":"2024-04-25T13:41:10.287539+0000","last_clean":"2024-04-25T13:41:10.287539+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:41:10.287539+0000","last_undegraded":"2024-04-25T13:41:10.287539+0000","last_fullsized":"2024-04-25T13:41:10.287539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":244,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.363110+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:41:40.363110+0000","last_peered":"2024-04-25T13:41:40.363110+0000","last_clean":"2024-04-25T13:41:40.363110+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:41:40.363110+0000","last_undegraded":"2024-04-25T13:41:40.363110+0000","last_fullsized":"2024-04-25T13:41:40.363110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359936+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:41:41.359936+0000","last_peered":"2024-04-25T13:41:41.359936+0000","last_clean":"2024-04-25T13:41:41.359936+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:41:41.359936+0000","last_undegraded":"2024-04-25T13:41:41.359936+0000","last_fullsized":"2024-04-25T13:41:41.359936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358783+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:41:41.358783+0000","last_peered":"2024-04-25T13:41:41.358783+0000","last_clean":"2024-04-25T13:41:41.358783+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:41:41.358783+0000","last_undegraded":"2024-04-25T13:41:41.358783+0000","last_fullsized":"2024-04-25T13:41:41.358783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362772+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:41:41.362772+0000","last_peered":"2024-04-25T13:41:41.362772+0000","last_clean":"2024-04-25T13:41:41.362772+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:41:41.362772+0000","last_undegraded":"2024-04-25T13:41:41.362772+0000","last_fullsized":"2024-04-25T13:41:41.362772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361825+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:41:41.361825+0000","last_peered":"2024-04-25T13:41:41.361825+0000","last_clean":"2024-04-25T13:41:41.361825+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:41:41.361825+0000","last_undegraded":"2024-04-25T13:41:41.361825+0000","last_fullsized":"2024-04-25T13:41:41.361825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359348+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:41:41.359348+0000","last_peered":"2024-04-25T13:41:41.359348+0000","last_clean":"2024-04-25T13:41:41.359348+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:41:41.359348+0000","last_undegraded":"2024-04-25T13:41:41.359348+0000","last_fullsized":"2024-04-25T13:41:41.359348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":278,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.362985+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:41:40.362985+0000","last_peered":"2024-04-25T13:41:40.362985+0000","last_clean":"2024-04-25T13:41:40.362985+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:41:40.362985+0000","last_undegraded":"2024-04-25T13:41:40.362985+0000","last_fullsized":"2024-04-25T13:41:40.362985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59129,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":170,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446277+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:41:41.446277+0000","last_peered":"2024-04-25T13:41:41.446277+0000","last_clean":"2024-04-25T13:41:41.446277+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:41:41.446277+0000","last_undegraded":"2024-04-25T13:41:41.446277+0000","last_fullsized":"2024-04-25T13:41:41.446277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358927+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:41:41.358927+0000","last_peered":"2024-04-25T13:41:41.358927+0000","last_clean":"2024-04-25T13:41:41.358927+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:41:41.358927+0000","last_undegraded":"2024-04-25T13:41:41.358927+0000","last_fullsized":"2024-04-25T13:41:41.358927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359428+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:41:41.359428+0000","last_peered":"2024-04-25T13:41:41.359428+0000","last_clean":"2024-04-25T13:41:41.359428+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:41:41.359428+0000","last_undegraded":"2024-04-25T13:41:41.359428+0000","last_fullsized":"2024-04-25T13:41:41.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361253+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:41:41.361253+0000","last_peered":"2024-04-25T13:41:41.361253+0000","last_clean":"2024-04-25T13:41:41.361253+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:41:41.361253+0000","last_undegraded":"2024-04-25T13:41:41.361253+0000","last_fullsized":"2024-04-25T13:41:41.361253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358264+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:41:41.358264+0000","last_peered":"2024-04-25T13:41:41.358264+0000","last_clean":"2024-04-25T13:41:41.358264+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:41:41.358264+0000","last_undegraded":"2024-04-25T13:41:41.358264+0000","last_fullsized":"2024-04-25T13:41:41.358264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'5","reported_seq":134,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:37.811924+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:41:37.811924+0000","last_peered":"2024-04-25T13:41:37.811924+0000","last_clean":"2024-04-25T13:41:37.811924+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:41:37.811924+0000","last_undegraded":"2024-04-25T13:41:37.811924+0000","last_fullsized":"2024-04-25T13:41:37.811924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357450+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:41:41.357450+0000","last_peered":"2024-04-25T13:41:41.357450+0000","last_clean":"2024-04-25T13:41:41.357450+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:41:41.357450+0000","last_undegraded":"2024-04-25T13:41:41.357450+0000","last_fullsized":"2024-04-25T13:41:41.357450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.276375+0000","last_change":"2024-04-25T13:39:45.273219+0000","last_active":"2024-04-25T13:41:10.276375+0000","last_peered":"2024-04-25T13:41:10.276375+0000","last_clean":"2024-04-25T13:41:10.276375+0000","last_became_active":"2024-04-25T13:39:45.272891+0000","last_became_peered":"2024-04-25T13:39:45.272891+0000","last_unstale":"2024-04-25T13:41:10.276375+0000","last_undegraded":"2024-04-25T13:41:10.276375+0000","last_fullsized":"2024-04-25T13:41:10.276375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:40.531758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446655+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:41:41.446655+0000","last_peered":"2024-04-25T13:41:41.446655+0000","last_clean":"2024-04-25T13:41:41.446655+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:41:41.446655+0000","last_undegraded":"2024-04-25T13:41:41.446655+0000","last_fullsized":"2024-04-25T13:41:41.446655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.279040+0000","last_change":"2024-04-25T13:40:41.271904+0000","last_active":"2024-04-25T13:41:10.279040+0000","last_peered":"2024-04-25T13:41:10.279040+0000","last_clean":"2024-04-25T13:41:10.279040+0000","last_became_active":"2024-04-25T13:40:41.271457+0000","last_became_peered":"2024-04-25T13:40:41.271457+0000","last_unstale":"2024-04-25T13:41:10.279040+0000","last_undegraded":"2024-04-25T13:41:10.279040+0000","last_fullsized":"2024-04-25T13:41:10.279040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:04.766001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.270464+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:40:41.270464+0000","last_peered":"2024-04-25T13:40:41.270464+0000","last_clean":"2024-04-25T13:40:41.270464+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:40:41.270464+0000","last_undegraded":"2024-04-25T13:40:41.270464+0000","last_fullsized":"2024-04-25T13:40:41.270464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.363157+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:41:40.363157+0000","last_peered":"2024-04-25T13:41:40.363157+0000","last_clean":"2024-04-25T13:41:40.363157+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:41:40.363157+0000","last_undegraded":"2024-04-25T13:41:40.363157+0000","last_fullsized":"2024-04-25T13:41:40.363157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.811283+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:41:42.811283+0000","last_peered":"2024-04-25T13:41:42.811283+0000","last_clean":"2024-04-25T13:41:42.811283+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:41:42.811283+0000","last_undegraded":"2024-04-25T13:41:42.811283+0000","last_fullsized":"2024-04-25T13:41:42.811283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:41:40.362747+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:41:40.362747+0000","last_peered":"2024-04-25T13:41:40.362747+0000","last_clean":"2024-04-25T13:41:40.362747+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:41:40.362747+0000","last_undegraded":"2024-04-25T13:41:40.362747+0000","last_fullsized":"2024-04-25T13:41:40.362747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361116+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:41:41.361116+0000","last_peered":"2024-04-25T13:41:41.361116+0000","last_clean":"2024-04-25T13:41:41.361116+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:41:41.361116+0000","last_undegraded":"2024-04-25T13:41:41.361116+0000","last_fullsized":"2024-04-25T13:41:41.361116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.819794+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:41:42.819794+0000","last_peered":"2024-04-25T13:41:42.819794+0000","last_clean":"2024-04-25T13:41:42.819794+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:41:42.819794+0000","last_undegraded":"2024-04-25T13:41:42.819794+0000","last_fullsized":"2024-04-25T13:41:42.819794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445164+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:41:41.445164+0000","last_peered":"2024-04-25T13:41:41.445164+0000","last_clean":"2024-04-25T13:41:41.445164+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:41:41.445164+0000","last_undegraded":"2024-04-25T13:41:41.445164+0000","last_fullsized":"2024-04-25T13:41:41.445164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.261992+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:40:41.261992+0000","last_peered":"2024-04-25T13:40:41.261992+0000","last_clean":"2024-04-25T13:40:41.261992+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:40:41.261992+0000","last_undegraded":"2024-04-25T13:40:41.261992+0000","last_fullsized":"2024-04-25T13:40:41.261992+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357715+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:41:41.357715+0000","last_peered":"2024-04-25T13:41:41.357715+0000","last_clean":"2024-04-25T13:41:41.357715+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:41:41.357715+0000","last_undegraded":"2024-04-25T13:41:41.357715+0000","last_fullsized":"2024-04-25T13:41:41.357715+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359258+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:41:41.359258+0000","last_peered":"2024-04-25T13:41:41.359258+0000","last_clean":"2024-04-25T13:41:41.359258+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:41:41.359258+0000","last_undegraded":"2024-04-25T13:41:41.359258+0000","last_fullsized":"2024-04-25T13:41:41.359258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358003+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:41:41.358003+0000","last_peered":"2024-04-25T13:41:41.358003+0000","last_clean":"2024-04-25T13:41:41.358003+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:41:41.358003+0000","last_undegraded":"2024-04-25T13:41:41.358003+0000","last_fullsized":"2024-04-25T13:41:41.358003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.280650+0000","last_change":"2024-04-25T13:40:41.280650+0000","last_active":"2024-04-25T13:40:41.280650+0000","last_peered":"2024-04-25T13:40:41.280650+0000","last_clean":"2024-04-25T13:40:41.280650+0000","last_became_active":"2024-04-25T13:40:41.271193+0000","last_became_peered":"2024-04-25T13:40:41.271193+0000","last_unstale":"2024-04-25T13:40:41.280650+0000","last_undegraded":"2024-04-25T13:40:41.280650+0000","last_fullsized":"2024-04-25T13:40:41.280650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.137595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359610+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:41:41.359610+0000","last_peered":"2024-04-25T13:41:41.359610+0000","last_clean":"2024-04-25T13:41:41.359610+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:41:41.359610+0000","last_undegraded":"2024-04-25T13:41:41.359610+0000","last_fullsized":"2024-04-25T13:41:41.359610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.271051+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:40:41.271051+0000","last_peered":"2024-04-25T13:40:41.271051+0000","last_clean":"2024-04-25T13:40:41.271051+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:40:41.271051+0000","last_undegraded":"2024-04-25T13:40:41.271051+0000","last_fullsized":"2024-04-25T13:40:41.271051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445401+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:41:41.445401+0000","last_peered":"2024-04-25T13:41:41.445401+0000","last_clean":"2024-04-25T13:41:41.445401+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:41:41.445401+0000","last_undegraded":"2024-04-25T13:41:41.445401+0000","last_fullsized":"2024-04-25T13:41:41.445401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.262137+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:40:41.262137+0000","last_peered":"2024-04-25T13:40:41.262137+0000","last_clean":"2024-04-25T13:40:41.262137+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:40:41.262137+0000","last_undegraded":"2024-04-25T13:40:41.262137+0000","last_fullsized":"2024-04-25T13:40:41.262137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362311+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:41:41.362311+0000","last_peered":"2024-04-25T13:41:41.362311+0000","last_clean":"2024-04-25T13:41:41.362311+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:41:41.362311+0000","last_undegraded":"2024-04-25T13:41:41.362311+0000","last_fullsized":"2024-04-25T13:41:41.362311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358635+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:41:41.358635+0000","last_peered":"2024-04-25T13:41:41.358635+0000","last_clean":"2024-04-25T13:41:41.358635+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:41:41.358635+0000","last_undegraded":"2024-04-25T13:41:41.358635+0000","last_fullsized":"2024-04-25T13:41:41.358635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445009+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:41:41.445009+0000","last_peered":"2024-04-25T13:41:41.445009+0000","last_clean":"2024-04-25T13:41:41.445009+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:41:41.445009+0000","last_undegraded":"2024-04-25T13:41:41.445009+0000","last_fullsized":"2024-04-25T13:41:41.445009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.271821+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:41:10.271821+0000","last_peered":"2024-04-25T13:41:10.271821+0000","last_clean":"2024-04-25T13:41:10.271821+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:41:10.271821+0000","last_undegraded":"2024-04-25T13:41:10.271821+0000","last_fullsized":"2024-04-25T13:41:10.271821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:40:41.284948+0000","last_change":"2024-04-25T13:40:41.284948+0000","last_active":"2024-04-25T13:40:41.284948+0000","last_peered":"2024-04-25T13:40:41.284948+0000","last_clean":"2024-04-25T13:40:41.284948+0000","last_became_active":"2024-04-25T13:40:41.272015+0000","last_became_peered":"2024-04-25T13:40:41.272015+0000","last_unstale":"2024-04-25T13:40:41.284948+0000","last_undegraded":"2024-04-25T13:40:41.284948+0000","last_fullsized":"2024-04-25T13:40:41.284948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:21.620303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445232+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:41:41.445232+0000","last_peered":"2024-04-25T13:41:41.445232+0000","last_clean":"2024-04-25T13:41:41.445232+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:41:41.445232+0000","last_undegraded":"2024-04-25T13:41:41.445232+0000","last_fullsized":"2024-04-25T13:41:41.445232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446492+0000","last_change":"2024-04-25T13:40:41.282512+0000","last_active":"2024-04-25T13:41:41.446492+0000","last_peered":"2024-04-25T13:41:41.446492+0000","last_clean":"2024-04-25T13:41:41.446492+0000","last_became_active":"2024-04-25T13:40:41.275727+0000","last_became_peered":"2024-04-25T13:40:41.275727+0000","last_unstale":"2024-04-25T13:41:41.446492+0000","last_undegraded":"2024-04-25T13:41:41.446492+0000","last_fullsized":"2024-04-25T13:41:41.446492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.611067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357910+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:41:41.357910+0000","last_peered":"2024-04-25T13:41:41.357910+0000","last_clean":"2024-04-25T13:41:41.357910+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:41:41.357910+0000","last_undegraded":"2024-04-25T13:41:41.357910+0000","last_fullsized":"2024-04-25T13:41:41.357910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.812487+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:41:42.812487+0000","last_peered":"2024-04-25T13:41:42.812487+0000","last_clean":"2024-04-25T13:41:42.812487+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:41:42.812487+0000","last_undegraded":"2024-04-25T13:41:42.812487+0000","last_fullsized":"2024-04-25T13:41:42.812487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358485+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:41:41.358485+0000","last_peered":"2024-04-25T13:41:41.358485+0000","last_clean":"2024-04-25T13:41:41.358485+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:41:41.358485+0000","last_undegraded":"2024-04-25T13:41:41.358485+0000","last_fullsized":"2024-04-25T13:41:41.358485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361741+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:41:41.361741+0000","last_peered":"2024-04-25T13:41:41.361741+0000","last_clean":"2024-04-25T13:41:41.361741+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:41:41.361741+0000","last_undegraded":"2024-04-25T13:41:41.361741+0000","last_fullsized":"2024-04-25T13:41:41.361741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358380+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:41:41.358380+0000","last_peered":"2024-04-25T13:41:41.358380+0000","last_clean":"2024-04-25T13:41:41.358380+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:41:41.358380+0000","last_undegraded":"2024-04-25T13:41:41.358380+0000","last_fullsized":"2024-04-25T13:41:41.358380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359596+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:41:41.359596+0000","last_peered":"2024-04-25T13:41:41.359596+0000","last_clean":"2024-04-25T13:41:41.359596+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:41:41.359596+0000","last_undegraded":"2024-04-25T13:41:41.359596+0000","last_fullsized":"2024-04-25T13:41:41.359596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358692+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:41:41.358692+0000","last_peered":"2024-04-25T13:41:41.358692+0000","last_clean":"2024-04-25T13:41:41.358692+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:41:41.358692+0000","last_undegraded":"2024-04-25T13:41:41.358692+0000","last_fullsized":"2024-04-25T13:41:41.358692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.279753+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:41:10.279753+0000","last_peered":"2024-04-25T13:41:10.279753+0000","last_clean":"2024-04-25T13:41:10.279753+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:41:10.279753+0000","last_undegraded":"2024-04-25T13:41:10.279753+0000","last_fullsized":"2024-04-25T13:41:10.279753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358989+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:41:41.358989+0000","last_peered":"2024-04-25T13:41:41.358989+0000","last_clean":"2024-04-25T13:41:41.358989+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:41:41.358989+0000","last_undegraded":"2024-04-25T13:41:41.358989+0000","last_fullsized":"2024-04-25T13:41:41.358989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358931+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:41:41.358931+0000","last_peered":"2024-04-25T13:41:41.358931+0000","last_clean":"2024-04-25T13:41:41.358931+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:41:41.358931+0000","last_undegraded":"2024-04-25T13:41:41.358931+0000","last_fullsized":"2024-04-25T13:41:41.358931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361885+0000","last_change":"2024-04-25T13:40:41.274874+0000","last_active":"2024-04-25T13:41:41.361885+0000","last_peered":"2024-04-25T13:41:41.361885+0000","last_clean":"2024-04-25T13:41:41.361885+0000","last_became_active":"2024-04-25T13:40:41.274489+0000","last_became_peered":"2024-04-25T13:40:41.274489+0000","last_unstale":"2024-04-25T13:41:41.361885+0000","last_undegraded":"2024-04-25T13:41:41.361885+0000","last_fullsized":"2024-04-25T13:41:41.361885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:34.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.364287+0000","last_change":"2024-04-25T13:41:41.364287+0000","last_active":"2024-04-25T13:41:41.364287+0000","last_peered":"2024-04-25T13:41:41.364287+0000","last_clean":"2024-04-25T13:41:41.364287+0000","last_became_active":"2024-04-25T13:41:41.363853+0000","last_became_peered":"2024-04-25T13:41:41.363853+0000","last_unstale":"2024-04-25T13:41:41.364287+0000","last_undegraded":"2024-04-25T13:41:41.364287+0000","last_fullsized":"2024-04-25T13:41:41.364287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:11.271445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.279666+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:41:10.279666+0000","last_peered":"2024-04-25T13:41:10.279666+0000","last_clean":"2024-04-25T13:41:10.279666+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:41:10.279666+0000","last_undegraded":"2024-04-25T13:41:10.279666+0000","last_fullsized":"2024-04-25T13:41:10.279666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359493+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:41:41.359493+0000","last_peered":"2024-04-25T13:41:41.359493+0000","last_clean":"2024-04-25T13:41:41.359493+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:41:41.359493+0000","last_undegraded":"2024-04-25T13:41:41.359493+0000","last_fullsized":"2024-04-25T13:41:41.359493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445348+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:41:41.445348+0000","last_peered":"2024-04-25T13:41:41.445348+0000","last_clean":"2024-04-25T13:41:41.445348+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:41:41.445348+0000","last_undegraded":"2024-04-25T13:41:41.445348+0000","last_fullsized":"2024-04-25T13:41:41.445348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361057+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:41:41.361057+0000","last_peered":"2024-04-25T13:41:41.361057+0000","last_clean":"2024-04-25T13:41:41.361057+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:41:41.361057+0000","last_undegraded":"2024-04-25T13:41:41.361057+0000","last_fullsized":"2024-04-25T13:41:41.361057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445102+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:41:41.445102+0000","last_peered":"2024-04-25T13:41:41.445102+0000","last_clean":"2024-04-25T13:41:41.445102+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:41:41.445102+0000","last_undegraded":"2024-04-25T13:41:41.445102+0000","last_fullsized":"2024-04-25T13:41:41.445102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358923+0000","last_change":"2024-04-25T13:40:41.276377+0000","last_active":"2024-04-25T13:41:41.358923+0000","last_peered":"2024-04-25T13:41:41.358923+0000","last_clean":"2024-04-25T13:41:41.358923+0000","last_became_active":"2024-04-25T13:40:41.275949+0000","last_became_peered":"2024-04-25T13:40:41.275949+0000","last_unstale":"2024-04-25T13:41:41.358923+0000","last_undegraded":"2024-04-25T13:41:41.358923+0000","last_fullsized":"2024-04-25T13:41:41.358923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:49.882272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359800+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:41:41.359800+0000","last_peered":"2024-04-25T13:41:41.359800+0000","last_clean":"2024-04-25T13:41:41.359800+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:41:41.359800+0000","last_undegraded":"2024-04-25T13:41:41.359800+0000","last_fullsized":"2024-04-25T13:41:41.359800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:41:10.289867+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:41:10.289867+0000","last_peered":"2024-04-25T13:41:10.289867+0000","last_clean":"2024-04-25T13:41:10.289867+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:41:10.289867+0000","last_undegraded":"2024-04-25T13:41:10.289867+0000","last_fullsized":"2024-04-25T13:41:10.289867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66474,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1140,"num_read_kb":640,"num_write":246,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":209756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":422903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1072949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40388,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704764,"statfs":{"total":95995035648,"available":95953678336,"internally_reserved":0,"allocated":913408,"data_stored":422721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":440297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1033767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":440139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":505194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40432,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704720,"statfs":{"total":95995035648,"available":95953633280,"internally_reserved":0,"allocated":950272,"data_stored":441087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":505555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":946176,"data_stored":439573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":422299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1552384,"data_stored":1032746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:41:46.001 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:46.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- 172.21.15.94:0/3496327913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0102f70 msgr2=0x7fa7d0103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3496327913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0102f70 0x7fa7d0103380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009900 tx=0x7fa7cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- 172.21.15.94:0/3496327913 shutdown_connections 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3496327913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0104470 0x7fa7d010abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3496327913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0103950 0x7fa7d0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3496327913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0102f70 0x7fa7d0103380 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- 172.21.15.94:0/3496327913 >> 172.21.15.94:0/3496327913 conn(0x7fa7d00fd370 msgr2=0x7fa7d00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- 172.21.15.94:0/3496327913 shutdown_connections 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- 172.21.15.94:0/3496327913 wait complete. 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 Processor -- start 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 -- start start 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 0x7fa7d01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.144+0000 7fa7d835e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0104470 0x7fa7d01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d835e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010e4f0 con 0x7fa7d0103950 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d835e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010e370 con 0x7fa7d0104470 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d835e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010e670 con 0x7fa7d0102f70 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54136/0 (socket says 172.21.15.94:54136) 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 -- 172.21.15.94:0/3630627038 learned_addr learned my addr 172.21.15.94:0/3630627038 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d68fb640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0104470 0x7fa7d01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d60fa640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 0x7fa7d01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 -- 172.21.15.94:0/3630627038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 msgr2=0x7fa7d01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 0x7fa7d01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 -- 172.21.15.94:0/3630627038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0104470 msgr2=0x7fa7d01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0104470 0x7fa7d01a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 -- 172.21.15.94:0/3630627038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d01a0b50 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d60fa640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 0x7fa7d01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d58f9640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c920 tx=0x7fa7c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0007c70 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c0007e10 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d01a0e40 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0018820 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.145+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d01a12b0 con 0x7fa7d0103950 2024-04-25T13:41:46.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.146+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d0107910 con 0x7fa7d0103950 2024-04-25T13:41:46.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.148+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fa7c000f450 con 0x7fa7d0103950 2024-04-25T13:41:46.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.148+0000 7fa7c77fe640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 0x7fa7bc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.148+0000 7fa7d60fa640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 0x7fa7bc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.148+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa7c00df290 con 0x7fa7d0103950 2024-04-25T13:41:46.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.149+0000 7fa7d60fa640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 0x7fa7bc0b2090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc0314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.152+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7c0014030 con 0x7fa7d0103950 2024-04-25T13:41:46.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.328+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7d0062fe0 con 0x7fa7bc0afb80 2024-04-25T13:41:46.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.335+0000 7fa7c77fe640 1 -- 172.21.15.94:0/3630627038 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7fa7d0062fe0 con 0x7fa7bc0afb80 2024-04-25T13:41:46.340 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:41:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 msgr2=0x7fa7bc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 0x7fa7bc0b2090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc0314c0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 msgr2=0x7fa7d0199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c920 tx=0x7fa7c000cde0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 shutdown_connections 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa7bc0afb80 0x7fa7bc0b2090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7d0104470 0x7fa7d01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7d0103950 0x7fa7d0199600 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 --2- 172.21.15.94:0/3630627038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d0102f70 0x7fa7d01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.338+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 >> 172.21.15.94:0/3630627038 conn(0x7fa7d00fd370 msgr2=0x7fa7d0104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.339+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 shutdown_connections 2024-04-25T13:41:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.339+0000 7fa7d835e640 1 -- 172.21.15.94:0/3630627038 wait complete. 2024-04-25T13:41:46.350 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-25T13:41:46.288071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662397,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1249,"num_read_kb":731,"num_write":380,"num_write_kb":2301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412104,"kb_used_data":12984,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529720,"statfs":{"total":1151940427776,"available":1151518433280,"internally_reserved":0,"allocated":13295616,"data_stored":7254099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5712,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":58,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001865"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357193+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:41:41.357193+0000","last_peered":"2024-04-25T13:41:41.357193+0000","last_clean":"2024-04-25T13:41:41.357193+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:41:41.357193+0000","last_undegraded":"2024-04-25T13:41:41.357193+0000","last_fullsized":"2024-04-25T13:41:41.357193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362416+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:41:41.362416+0000","last_peered":"2024-04-25T13:41:41.362416+0000","last_clean":"2024-04-25T13:41:41.362416+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:41:41.362416+0000","last_undegraded":"2024-04-25T13:41:41.362416+0000","last_fullsized":"2024-04-25T13:41:41.362416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362698+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:41:41.362698+0000","last_peered":"2024-04-25T13:41:41.362698+0000","last_clean":"2024-04-25T13:41:41.362698+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:41:41.362698+0000","last_undegraded":"2024-04-25T13:41:41.362698+0000","last_fullsized":"2024-04-25T13:41:41.362698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360147+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:41:41.360147+0000","last_peered":"2024-04-25T13:41:41.360147+0000","last_clean":"2024-04-25T13:41:41.360147+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:41:41.360147+0000","last_undegraded":"2024-04-25T13:41:41.360147+0000","last_fullsized":"2024-04-25T13:41:41.360147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446495+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:41:41.446495+0000","last_peered":"2024-04-25T13:41:41.446495+0000","last_clean":"2024-04-25T13:41:41.446495+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:41:41.446495+0000","last_undegraded":"2024-04-25T13:41:41.446495+0000","last_fullsized":"2024-04-25T13:41:41.446495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360468+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:41:41.360468+0000","last_peered":"2024-04-25T13:41:41.360468+0000","last_clean":"2024-04-25T13:41:41.360468+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:41:41.360468+0000","last_undegraded":"2024-04-25T13:41:41.360468+0000","last_fullsized":"2024-04-25T13:41:41.360468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357446+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:41:41.357446+0000","last_peered":"2024-04-25T13:41:41.357446+0000","last_clean":"2024-04-25T13:41:41.357446+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:41:41.357446+0000","last_undegraded":"2024-04-25T13:41:41.357446+0000","last_fullsized":"2024-04-25T13:41:41.357446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444909+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:41:41.444909+0000","last_peered":"2024-04-25T13:41:41.444909+0000","last_clean":"2024-04-25T13:41:41.444909+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:41:41.444909+0000","last_undegraded":"2024-04-25T13:41:41.444909+0000","last_fullsized":"2024-04-25T13:41:41.444909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360334+0000","last_change":"2024-04-25T13:40:41.273152+0000","last_active":"2024-04-25T13:41:41.360334+0000","last_peered":"2024-04-25T13:41:41.360334+0000","last_clean":"2024-04-25T13:41:41.360334+0000","last_became_active":"2024-04-25T13:40:41.272760+0000","last_became_peered":"2024-04-25T13:40:41.272760+0000","last_unstale":"2024-04-25T13:41:41.360334+0000","last_undegraded":"2024-04-25T13:41:41.360334+0000","last_fullsized":"2024-04-25T13:41:41.360334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:35.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361389+0000","last_change":"2024-04-25T13:40:41.274781+0000","last_active":"2024-04-25T13:41:41.361389+0000","last_peered":"2024-04-25T13:41:41.361389+0000","last_clean":"2024-04-25T13:41:41.361389+0000","last_became_active":"2024-04-25T13:40:41.274043+0000","last_became_peered":"2024-04-25T13:40:41.274043+0000","last_unstale":"2024-04-25T13:41:41.361389+0000","last_undegraded":"2024-04-25T13:41:41.361389+0000","last_fullsized":"2024-04-25T13:41:41.361389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:01.989580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358355+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:41:41.358355+0000","last_peered":"2024-04-25T13:41:41.358355+0000","last_clean":"2024-04-25T13:41:41.358355+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:41:41.358355+0000","last_undegraded":"2024-04-25T13:41:41.358355+0000","last_fullsized":"2024-04-25T13:41:41.358355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360296+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:41:41.360296+0000","last_peered":"2024-04-25T13:41:41.360296+0000","last_clean":"2024-04-25T13:41:41.360296+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:41:41.360296+0000","last_undegraded":"2024-04-25T13:41:41.360296+0000","last_fullsized":"2024-04-25T13:41:41.360296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359749+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:41:41.359749+0000","last_peered":"2024-04-25T13:41:41.359749+0000","last_clean":"2024-04-25T13:41:41.359749+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:41:41.359749+0000","last_undegraded":"2024-04-25T13:41:41.359749+0000","last_fullsized":"2024-04-25T13:41:41.359749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362114+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:41:41.362114+0000","last_peered":"2024-04-25T13:41:41.362114+0000","last_clean":"2024-04-25T13:41:41.362114+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:41:41.362114+0000","last_undegraded":"2024-04-25T13:41:41.362114+0000","last_fullsized":"2024-04-25T13:41:41.362114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361807+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:41:41.361807+0000","last_peered":"2024-04-25T13:41:41.361807+0000","last_clean":"2024-04-25T13:41:41.361807+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:41:41.361807+0000","last_undegraded":"2024-04-25T13:41:41.361807+0000","last_fullsized":"2024-04-25T13:41:41.361807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358442+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:41:41.358442+0000","last_peered":"2024-04-25T13:41:41.358442+0000","last_clean":"2024-04-25T13:41:41.358442+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:41:41.358442+0000","last_undegraded":"2024-04-25T13:41:41.358442+0000","last_fullsized":"2024-04-25T13:41:41.358442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357296+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:41:41.357296+0000","last_peered":"2024-04-25T13:41:41.357296+0000","last_clean":"2024-04-25T13:41:41.357296+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:41:41.357296+0000","last_undegraded":"2024-04-25T13:41:41.357296+0000","last_fullsized":"2024-04-25T13:41:41.357296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362394+0000","last_change":"2024-04-25T13:41:41.362394+0000","last_active":"2024-04-25T13:41:41.362394+0000","last_peered":"2024-04-25T13:41:41.362394+0000","last_clean":"2024-04-25T13:41:41.362394+0000","last_became_active":"2024-04-25T13:41:41.361973+0000","last_became_peered":"2024-04-25T13:41:41.361973+0000","last_unstale":"2024-04-25T13:41:41.362394+0000","last_undegraded":"2024-04-25T13:41:41.362394+0000","last_fullsized":"2024-04-25T13:41:41.362394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:44.656494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359196+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:41:41.359196+0000","last_peered":"2024-04-25T13:41:41.359196+0000","last_clean":"2024-04-25T13:41:41.359196+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:41:41.359196+0000","last_undegraded":"2024-04-25T13:41:41.359196+0000","last_fullsized":"2024-04-25T13:41:41.359196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.809954+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:41:42.809954+0000","last_peered":"2024-04-25T13:41:42.809954+0000","last_clean":"2024-04-25T13:41:42.809954+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:41:42.809954+0000","last_undegraded":"2024-04-25T13:41:42.809954+0000","last_fullsized":"2024-04-25T13:41:42.809954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445375+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:41:41.445375+0000","last_peered":"2024-04-25T13:41:41.445375+0000","last_clean":"2024-04-25T13:41:41.445375+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:41:41.445375+0000","last_undegraded":"2024-04-25T13:41:41.445375+0000","last_fullsized":"2024-04-25T13:41:41.445375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445462+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:41:41.445462+0000","last_peered":"2024-04-25T13:41:41.445462+0000","last_clean":"2024-04-25T13:41:41.445462+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:41:41.445462+0000","last_undegraded":"2024-04-25T13:41:41.445462+0000","last_fullsized":"2024-04-25T13:41:41.445462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362671+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:41:41.362671+0000","last_peered":"2024-04-25T13:41:41.362671+0000","last_clean":"2024-04-25T13:41:41.362671+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:41:41.362671+0000","last_undegraded":"2024-04-25T13:41:41.362671+0000","last_fullsized":"2024-04-25T13:41:41.362671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446521+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:41:41.446521+0000","last_peered":"2024-04-25T13:41:41.446521+0000","last_clean":"2024-04-25T13:41:41.446521+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:41:41.446521+0000","last_undegraded":"2024-04-25T13:41:41.446521+0000","last_fullsized":"2024-04-25T13:41:41.446521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359128+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:41:41.359128+0000","last_peered":"2024-04-25T13:41:41.359128+0000","last_clean":"2024-04-25T13:41:41.359128+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:41:41.359128+0000","last_undegraded":"2024-04-25T13:41:41.359128+0000","last_fullsized":"2024-04-25T13:41:41.359128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446334+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:41:41.446334+0000","last_peered":"2024-04-25T13:41:41.446334+0000","last_clean":"2024-04-25T13:41:41.446334+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:41:41.446334+0000","last_undegraded":"2024-04-25T13:41:41.446334+0000","last_fullsized":"2024-04-25T13:41:41.446334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361645+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:41:41.361645+0000","last_peered":"2024-04-25T13:41:41.361645+0000","last_clean":"2024-04-25T13:41:41.361645+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:41:41.361645+0000","last_undegraded":"2024-04-25T13:41:41.361645+0000","last_fullsized":"2024-04-25T13:41:41.361645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.811056+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:41:41.811056+0000","last_peered":"2024-04-25T13:41:41.811056+0000","last_clean":"2024-04-25T13:41:41.811056+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:41:41.811056+0000","last_undegraded":"2024-04-25T13:41:41.811056+0000","last_fullsized":"2024-04-25T13:41:41.811056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446058+0000","last_change":"2024-04-25T13:40:41.272700+0000","last_active":"2024-04-25T13:41:41.446058+0000","last_peered":"2024-04-25T13:41:41.446058+0000","last_clean":"2024-04-25T13:41:41.446058+0000","last_became_active":"2024-04-25T13:40:41.272080+0000","last_became_peered":"2024-04-25T13:40:41.272080+0000","last_unstale":"2024-04-25T13:41:41.446058+0000","last_undegraded":"2024-04-25T13:41:41.446058+0000","last_fullsized":"2024-04-25T13:41:41.446058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.149458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360468+0000","last_change":"2024-04-25T13:40:41.280199+0000","last_active":"2024-04-25T13:41:41.360468+0000","last_peered":"2024-04-25T13:41:41.360468+0000","last_clean":"2024-04-25T13:41:41.360468+0000","last_became_active":"2024-04-25T13:40:41.268773+0000","last_became_peered":"2024-04-25T13:40:41.268773+0000","last_unstale":"2024-04-25T13:41:41.360468+0000","last_undegraded":"2024-04-25T13:41:41.360468+0000","last_fullsized":"2024-04-25T13:41:41.360468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:47.024600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.447783+0000","last_change":"2024-04-25T13:41:41.447783+0000","last_active":"2024-04-25T13:41:41.447783+0000","last_peered":"2024-04-25T13:41:41.447783+0000","last_clean":"2024-04-25T13:41:41.447783+0000","last_became_active":"2024-04-25T13:41:41.447293+0000","last_became_peered":"2024-04-25T13:41:41.447293+0000","last_unstale":"2024-04-25T13:41:41.447783+0000","last_undegraded":"2024-04-25T13:41:41.447783+0000","last_fullsized":"2024-04-25T13:41:41.447783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:39.807786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359388+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:41:41.359388+0000","last_peered":"2024-04-25T13:41:41.359388+0000","last_clean":"2024-04-25T13:41:41.359388+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:41:41.359388+0000","last_undegraded":"2024-04-25T13:41:41.359388+0000","last_fullsized":"2024-04-25T13:41:41.359388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444835+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:41:41.444835+0000","last_peered":"2024-04-25T13:41:41.444835+0000","last_clean":"2024-04-25T13:41:41.444835+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:41:41.444835+0000","last_undegraded":"2024-04-25T13:41:41.444835+0000","last_fullsized":"2024-04-25T13:41:41.444835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360949+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:41:41.360949+0000","last_peered":"2024-04-25T13:41:41.360949+0000","last_clean":"2024-04-25T13:41:41.360949+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:41:41.360949+0000","last_undegraded":"2024-04-25T13:41:41.360949+0000","last_fullsized":"2024-04-25T13:41:41.360949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.355975+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:41:41.355975+0000","last_peered":"2024-04-25T13:41:41.355975+0000","last_clean":"2024-04-25T13:41:41.355975+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:41:41.355975+0000","last_undegraded":"2024-04-25T13:41:41.355975+0000","last_fullsized":"2024-04-25T13:41:41.355975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":256,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:44.814249+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:41:44.814249+0000","last_peered":"2024-04-25T13:41:44.814249+0000","last_clean":"2024-04-25T13:41:44.814249+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:41:44.814249+0000","last_undegraded":"2024-04-25T13:41:44.814249+0000","last_fullsized":"2024-04-25T13:41:44.814249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359936+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:41:41.359936+0000","last_peered":"2024-04-25T13:41:41.359936+0000","last_clean":"2024-04-25T13:41:41.359936+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:41:41.359936+0000","last_undegraded":"2024-04-25T13:41:41.359936+0000","last_fullsized":"2024-04-25T13:41:41.359936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358783+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:41:41.358783+0000","last_peered":"2024-04-25T13:41:41.358783+0000","last_clean":"2024-04-25T13:41:41.358783+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:41:41.358783+0000","last_undegraded":"2024-04-25T13:41:41.358783+0000","last_fullsized":"2024-04-25T13:41:41.358783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362772+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:41:41.362772+0000","last_peered":"2024-04-25T13:41:41.362772+0000","last_clean":"2024-04-25T13:41:41.362772+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:41:41.362772+0000","last_undegraded":"2024-04-25T13:41:41.362772+0000","last_fullsized":"2024-04-25T13:41:41.362772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361825+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:41:41.361825+0000","last_peered":"2024-04-25T13:41:41.361825+0000","last_clean":"2024-04-25T13:41:41.361825+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:41:41.361825+0000","last_undegraded":"2024-04-25T13:41:41.361825+0000","last_fullsized":"2024-04-25T13:41:41.361825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359348+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:41:41.359348+0000","last_peered":"2024-04-25T13:41:41.359348+0000","last_clean":"2024-04-25T13:41:41.359348+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:41:41.359348+0000","last_undegraded":"2024-04-25T13:41:41.359348+0000","last_fullsized":"2024-04-25T13:41:41.359348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:44.822900+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:41:44.822900+0000","last_peered":"2024-04-25T13:41:44.822900+0000","last_clean":"2024-04-25T13:41:44.822900+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:41:44.822900+0000","last_undegraded":"2024-04-25T13:41:44.822900+0000","last_fullsized":"2024-04-25T13:41:44.822900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64142,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":181,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446277+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:41:41.446277+0000","last_peered":"2024-04-25T13:41:41.446277+0000","last_clean":"2024-04-25T13:41:41.446277+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:41:41.446277+0000","last_undegraded":"2024-04-25T13:41:41.446277+0000","last_fullsized":"2024-04-25T13:41:41.446277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358927+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:41:41.358927+0000","last_peered":"2024-04-25T13:41:41.358927+0000","last_clean":"2024-04-25T13:41:41.358927+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:41:41.358927+0000","last_undegraded":"2024-04-25T13:41:41.358927+0000","last_fullsized":"2024-04-25T13:41:41.358927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359428+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:41:41.359428+0000","last_peered":"2024-04-25T13:41:41.359428+0000","last_clean":"2024-04-25T13:41:41.359428+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:41:41.359428+0000","last_undegraded":"2024-04-25T13:41:41.359428+0000","last_fullsized":"2024-04-25T13:41:41.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361253+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:41:41.361253+0000","last_peered":"2024-04-25T13:41:41.361253+0000","last_clean":"2024-04-25T13:41:41.361253+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:41:41.361253+0000","last_undegraded":"2024-04-25T13:41:41.361253+0000","last_fullsized":"2024-04-25T13:41:41.361253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358264+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:41:41.358264+0000","last_peered":"2024-04-25T13:41:41.358264+0000","last_clean":"2024-04-25T13:41:41.358264+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:41:41.358264+0000","last_undegraded":"2024-04-25T13:41:41.358264+0000","last_fullsized":"2024-04-25T13:41:41.358264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.820375+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:41:42.820375+0000","last_peered":"2024-04-25T13:41:42.820375+0000","last_clean":"2024-04-25T13:41:42.820375+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:41:42.820375+0000","last_undegraded":"2024-04-25T13:41:42.820375+0000","last_fullsized":"2024-04-25T13:41:42.820375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357450+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:41:41.357450+0000","last_peered":"2024-04-25T13:41:41.357450+0000","last_clean":"2024-04-25T13:41:41.357450+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:41:41.357450+0000","last_undegraded":"2024-04-25T13:41:41.357450+0000","last_fullsized":"2024-04-25T13:41:41.357450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362909+0000","last_change":"2024-04-25T13:41:41.362306+0000","last_active":"2024-04-25T13:41:41.362909+0000","last_peered":"2024-04-25T13:41:41.362909+0000","last_clean":"2024-04-25T13:41:41.362909+0000","last_became_active":"2024-04-25T13:41:41.361910+0000","last_became_peered":"2024-04-25T13:41:41.361910+0000","last_unstale":"2024-04-25T13:41:41.362909+0000","last_undegraded":"2024-04-25T13:41:41.362909+0000","last_fullsized":"2024-04-25T13:41:41.362909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:09.616231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446655+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:41:41.446655+0000","last_peered":"2024-04-25T13:41:41.446655+0000","last_clean":"2024-04-25T13:41:41.446655+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:41:41.446655+0000","last_undegraded":"2024-04-25T13:41:41.446655+0000","last_fullsized":"2024-04-25T13:41:41.446655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357273+0000","last_change":"2024-04-25T13:40:41.271904+0000","last_active":"2024-04-25T13:41:41.357273+0000","last_peered":"2024-04-25T13:41:41.357273+0000","last_clean":"2024-04-25T13:41:41.357273+0000","last_became_active":"2024-04-25T13:40:41.271457+0000","last_became_peered":"2024-04-25T13:40:41.271457+0000","last_unstale":"2024-04-25T13:41:41.357273+0000","last_undegraded":"2024-04-25T13:41:41.357273+0000","last_fullsized":"2024-04-25T13:41:41.357273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:04.766001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356059+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:41:41.356059+0000","last_peered":"2024-04-25T13:41:41.356059+0000","last_clean":"2024-04-25T13:41:41.356059+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:41:41.356059+0000","last_undegraded":"2024-04-25T13:41:41.356059+0000","last_fullsized":"2024-04-25T13:41:41.356059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360789+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:41:41.360789+0000","last_peered":"2024-04-25T13:41:41.360789+0000","last_clean":"2024-04-25T13:41:41.360789+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:41:41.360789+0000","last_undegraded":"2024-04-25T13:41:41.360789+0000","last_fullsized":"2024-04-25T13:41:41.360789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.811283+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:41:42.811283+0000","last_peered":"2024-04-25T13:41:42.811283+0000","last_clean":"2024-04-25T13:41:42.811283+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:41:42.811283+0000","last_undegraded":"2024-04-25T13:41:42.811283+0000","last_fullsized":"2024-04-25T13:41:42.811283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360718+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:41:41.360718+0000","last_peered":"2024-04-25T13:41:41.360718+0000","last_clean":"2024-04-25T13:41:41.360718+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:41:41.360718+0000","last_undegraded":"2024-04-25T13:41:41.360718+0000","last_fullsized":"2024-04-25T13:41:41.360718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361116+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:41:41.361116+0000","last_peered":"2024-04-25T13:41:41.361116+0000","last_clean":"2024-04-25T13:41:41.361116+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:41:41.361116+0000","last_undegraded":"2024-04-25T13:41:41.361116+0000","last_fullsized":"2024-04-25T13:41:41.361116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.819794+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:41:42.819794+0000","last_peered":"2024-04-25T13:41:42.819794+0000","last_clean":"2024-04-25T13:41:42.819794+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:41:42.819794+0000","last_undegraded":"2024-04-25T13:41:42.819794+0000","last_fullsized":"2024-04-25T13:41:42.819794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445164+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:41:41.445164+0000","last_peered":"2024-04-25T13:41:41.445164+0000","last_clean":"2024-04-25T13:41:41.445164+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:41:41.445164+0000","last_undegraded":"2024-04-25T13:41:41.445164+0000","last_fullsized":"2024-04-25T13:41:41.445164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360618+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:41:41.360618+0000","last_peered":"2024-04-25T13:41:41.360618+0000","last_clean":"2024-04-25T13:41:41.360618+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:41:41.360618+0000","last_undegraded":"2024-04-25T13:41:41.360618+0000","last_fullsized":"2024-04-25T13:41:41.360618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357715+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:41:41.357715+0000","last_peered":"2024-04-25T13:41:41.357715+0000","last_clean":"2024-04-25T13:41:41.357715+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:41:41.357715+0000","last_undegraded":"2024-04-25T13:41:41.357715+0000","last_fullsized":"2024-04-25T13:41:41.357715+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359258+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:41:41.359258+0000","last_peered":"2024-04-25T13:41:41.359258+0000","last_clean":"2024-04-25T13:41:41.359258+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:41:41.359258+0000","last_undegraded":"2024-04-25T13:41:41.359258+0000","last_fullsized":"2024-04-25T13:41:41.359258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358003+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:41:41.358003+0000","last_peered":"2024-04-25T13:41:41.358003+0000","last_clean":"2024-04-25T13:41:41.358003+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:41:41.358003+0000","last_undegraded":"2024-04-25T13:41:41.358003+0000","last_fullsized":"2024-04-25T13:41:41.358003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356936+0000","last_change":"2024-04-25T13:40:41.280650+0000","last_active":"2024-04-25T13:41:41.356936+0000","last_peered":"2024-04-25T13:41:41.356936+0000","last_clean":"2024-04-25T13:41:41.356936+0000","last_became_active":"2024-04-25T13:40:41.271193+0000","last_became_peered":"2024-04-25T13:40:41.271193+0000","last_unstale":"2024-04-25T13:41:41.356936+0000","last_undegraded":"2024-04-25T13:41:41.356936+0000","last_fullsized":"2024-04-25T13:41:41.356936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.137595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359610+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:41:41.359610+0000","last_peered":"2024-04-25T13:41:41.359610+0000","last_clean":"2024-04-25T13:41:41.359610+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:41:41.359610+0000","last_undegraded":"2024-04-25T13:41:41.359610+0000","last_fullsized":"2024-04-25T13:41:41.359610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.629856+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:41:41.629856+0000","last_peered":"2024-04-25T13:41:41.629856+0000","last_clean":"2024-04-25T13:41:41.629856+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:41:41.629856+0000","last_undegraded":"2024-04-25T13:41:41.629856+0000","last_fullsized":"2024-04-25T13:41:41.629856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445401+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:41:41.445401+0000","last_peered":"2024-04-25T13:41:41.445401+0000","last_clean":"2024-04-25T13:41:41.445401+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:41:41.445401+0000","last_undegraded":"2024-04-25T13:41:41.445401+0000","last_fullsized":"2024-04-25T13:41:41.445401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360057+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:41:41.360057+0000","last_peered":"2024-04-25T13:41:41.360057+0000","last_clean":"2024-04-25T13:41:41.360057+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:41:41.360057+0000","last_undegraded":"2024-04-25T13:41:41.360057+0000","last_fullsized":"2024-04-25T13:41:41.360057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362311+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:41:41.362311+0000","last_peered":"2024-04-25T13:41:41.362311+0000","last_clean":"2024-04-25T13:41:41.362311+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:41:41.362311+0000","last_undegraded":"2024-04-25T13:41:41.362311+0000","last_fullsized":"2024-04-25T13:41:41.362311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358635+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:41:41.358635+0000","last_peered":"2024-04-25T13:41:41.358635+0000","last_clean":"2024-04-25T13:41:41.358635+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:41:41.358635+0000","last_undegraded":"2024-04-25T13:41:41.358635+0000","last_fullsized":"2024-04-25T13:41:41.358635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445009+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:41:41.445009+0000","last_peered":"2024-04-25T13:41:41.445009+0000","last_clean":"2024-04-25T13:41:41.445009+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:41:41.445009+0000","last_undegraded":"2024-04-25T13:41:41.445009+0000","last_fullsized":"2024-04-25T13:41:41.445009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356645+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:41:41.356645+0000","last_peered":"2024-04-25T13:41:41.356645+0000","last_clean":"2024-04-25T13:41:41.356645+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:41:41.356645+0000","last_undegraded":"2024-04-25T13:41:41.356645+0000","last_fullsized":"2024-04-25T13:41:41.356645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356811+0000","last_change":"2024-04-25T13:40:41.284948+0000","last_active":"2024-04-25T13:41:41.356811+0000","last_peered":"2024-04-25T13:41:41.356811+0000","last_clean":"2024-04-25T13:41:41.356811+0000","last_became_active":"2024-04-25T13:40:41.272015+0000","last_became_peered":"2024-04-25T13:40:41.272015+0000","last_unstale":"2024-04-25T13:41:41.356811+0000","last_undegraded":"2024-04-25T13:41:41.356811+0000","last_fullsized":"2024-04-25T13:41:41.356811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:21.620303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445232+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:41:41.445232+0000","last_peered":"2024-04-25T13:41:41.445232+0000","last_clean":"2024-04-25T13:41:41.445232+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:41:41.445232+0000","last_undegraded":"2024-04-25T13:41:41.445232+0000","last_fullsized":"2024-04-25T13:41:41.445232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446492+0000","last_change":"2024-04-25T13:40:41.282512+0000","last_active":"2024-04-25T13:41:41.446492+0000","last_peered":"2024-04-25T13:41:41.446492+0000","last_clean":"2024-04-25T13:41:41.446492+0000","last_became_active":"2024-04-25T13:40:41.275727+0000","last_became_peered":"2024-04-25T13:40:41.275727+0000","last_unstale":"2024-04-25T13:41:41.446492+0000","last_undegraded":"2024-04-25T13:41:41.446492+0000","last_fullsized":"2024-04-25T13:41:41.446492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.611067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357910+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:41:41.357910+0000","last_peered":"2024-04-25T13:41:41.357910+0000","last_clean":"2024-04-25T13:41:41.357910+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:41:41.357910+0000","last_undegraded":"2024-04-25T13:41:41.357910+0000","last_fullsized":"2024-04-25T13:41:41.357910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.812487+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:41:42.812487+0000","last_peered":"2024-04-25T13:41:42.812487+0000","last_clean":"2024-04-25T13:41:42.812487+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:41:42.812487+0000","last_undegraded":"2024-04-25T13:41:42.812487+0000","last_fullsized":"2024-04-25T13:41:42.812487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358485+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:41:41.358485+0000","last_peered":"2024-04-25T13:41:41.358485+0000","last_clean":"2024-04-25T13:41:41.358485+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:41:41.358485+0000","last_undegraded":"2024-04-25T13:41:41.358485+0000","last_fullsized":"2024-04-25T13:41:41.358485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361741+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:41:41.361741+0000","last_peered":"2024-04-25T13:41:41.361741+0000","last_clean":"2024-04-25T13:41:41.361741+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:41:41.361741+0000","last_undegraded":"2024-04-25T13:41:41.361741+0000","last_fullsized":"2024-04-25T13:41:41.361741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358380+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:41:41.358380+0000","last_peered":"2024-04-25T13:41:41.358380+0000","last_clean":"2024-04-25T13:41:41.358380+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:41:41.358380+0000","last_undegraded":"2024-04-25T13:41:41.358380+0000","last_fullsized":"2024-04-25T13:41:41.358380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359596+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:41:41.359596+0000","last_peered":"2024-04-25T13:41:41.359596+0000","last_clean":"2024-04-25T13:41:41.359596+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:41:41.359596+0000","last_undegraded":"2024-04-25T13:41:41.359596+0000","last_fullsized":"2024-04-25T13:41:41.359596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358692+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:41:41.358692+0000","last_peered":"2024-04-25T13:41:41.358692+0000","last_clean":"2024-04-25T13:41:41.358692+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:41:41.358692+0000","last_undegraded":"2024-04-25T13:41:41.358692+0000","last_fullsized":"2024-04-25T13:41:41.358692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357006+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:41:41.357006+0000","last_peered":"2024-04-25T13:41:41.357006+0000","last_clean":"2024-04-25T13:41:41.357006+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:41:41.357006+0000","last_undegraded":"2024-04-25T13:41:41.357006+0000","last_fullsized":"2024-04-25T13:41:41.357006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358989+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:41:41.358989+0000","last_peered":"2024-04-25T13:41:41.358989+0000","last_clean":"2024-04-25T13:41:41.358989+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:41:41.358989+0000","last_undegraded":"2024-04-25T13:41:41.358989+0000","last_fullsized":"2024-04-25T13:41:41.358989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358931+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:41:41.358931+0000","last_peered":"2024-04-25T13:41:41.358931+0000","last_clean":"2024-04-25T13:41:41.358931+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:41:41.358931+0000","last_undegraded":"2024-04-25T13:41:41.358931+0000","last_fullsized":"2024-04-25T13:41:41.358931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361885+0000","last_change":"2024-04-25T13:40:41.274874+0000","last_active":"2024-04-25T13:41:41.361885+0000","last_peered":"2024-04-25T13:41:41.361885+0000","last_clean":"2024-04-25T13:41:41.361885+0000","last_became_active":"2024-04-25T13:40:41.274489+0000","last_became_peered":"2024-04-25T13:40:41.274489+0000","last_unstale":"2024-04-25T13:41:41.361885+0000","last_undegraded":"2024-04-25T13:41:41.361885+0000","last_fullsized":"2024-04-25T13:41:41.361885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:34.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.364287+0000","last_change":"2024-04-25T13:41:41.364287+0000","last_active":"2024-04-25T13:41:41.364287+0000","last_peered":"2024-04-25T13:41:41.364287+0000","last_clean":"2024-04-25T13:41:41.364287+0000","last_became_active":"2024-04-25T13:41:41.363853+0000","last_became_peered":"2024-04-25T13:41:41.363853+0000","last_unstale":"2024-04-25T13:41:41.364287+0000","last_undegraded":"2024-04-25T13:41:41.364287+0000","last_fullsized":"2024-04-25T13:41:41.364287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:11.271445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357161+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:41:41.357161+0000","last_peered":"2024-04-25T13:41:41.357161+0000","last_clean":"2024-04-25T13:41:41.357161+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:41:41.357161+0000","last_undegraded":"2024-04-25T13:41:41.357161+0000","last_fullsized":"2024-04-25T13:41:41.357161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359493+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:41:41.359493+0000","last_peered":"2024-04-25T13:41:41.359493+0000","last_clean":"2024-04-25T13:41:41.359493+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:41:41.359493+0000","last_undegraded":"2024-04-25T13:41:41.359493+0000","last_fullsized":"2024-04-25T13:41:41.359493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445348+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:41:41.445348+0000","last_peered":"2024-04-25T13:41:41.445348+0000","last_clean":"2024-04-25T13:41:41.445348+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:41:41.445348+0000","last_undegraded":"2024-04-25T13:41:41.445348+0000","last_fullsized":"2024-04-25T13:41:41.445348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361057+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:41:41.361057+0000","last_peered":"2024-04-25T13:41:41.361057+0000","last_clean":"2024-04-25T13:41:41.361057+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:41:41.361057+0000","last_undegraded":"2024-04-25T13:41:41.361057+0000","last_fullsized":"2024-04-25T13:41:41.361057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445102+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:41:41.445102+0000","last_peered":"2024-04-25T13:41:41.445102+0000","last_clean":"2024-04-25T13:41:41.445102+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:41:41.445102+0000","last_undegraded":"2024-04-25T13:41:41.445102+0000","last_fullsized":"2024-04-25T13:41:41.445102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358923+0000","last_change":"2024-04-25T13:40:41.276377+0000","last_active":"2024-04-25T13:41:41.358923+0000","last_peered":"2024-04-25T13:41:41.358923+0000","last_clean":"2024-04-25T13:41:41.358923+0000","last_became_active":"2024-04-25T13:40:41.275949+0000","last_became_peered":"2024-04-25T13:40:41.275949+0000","last_unstale":"2024-04-25T13:41:41.358923+0000","last_undegraded":"2024-04-25T13:41:41.358923+0000","last_fullsized":"2024-04-25T13:41:41.358923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:49.882272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359800+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:41:41.359800+0000","last_peered":"2024-04-25T13:41:41.359800+0000","last_clean":"2024-04-25T13:41:41.359800+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:41:41.359800+0000","last_undegraded":"2024-04-25T13:41:41.359800+0000","last_fullsized":"2024-04-25T13:41:41.359800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356863+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:41:41.356863+0000","last_peered":"2024-04-25T13:41:41.356863+0000","last_clean":"2024-04-25T13:41:41.356863+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:41:41.356863+0000","last_undegraded":"2024-04-25T13:41:41.356863+0000","last_fullsized":"2024-04-25T13:41:41.356863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71795,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1173,"num_read_kb":667,"num_write":255,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":440213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1626112,"data_stored":1095580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972400,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40420,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704732,"statfs":{"total":95995035648,"available":95953645568,"internally_reserved":0,"allocated":946176,"data_stored":440031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":440297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1033767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":440139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":505194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40432,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704720,"statfs":{"total":95995035648,"available":95953633280,"internally_reserved":0,"allocated":950272,"data_stored":441087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":505555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":946176,"data_stored":439573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1552384,"data_stored":1032746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:41:46.354 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T13:41:46.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- 172.21.15.94:0/3311086999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984102f70 msgr2=0x7fe984103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- 172.21.15.94:0/3311086999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984102f70 0x7fe984103380 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe974009900 tx=0x7fe97402a410 comp rx=0 tx=0).stop 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- 172.21.15.94:0/3311086999 shutdown_connections 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- 172.21.15.94:0/3311086999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984104470 0x7fe98410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- 172.21.15.94:0/3311086999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984103950 0x7fe984103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- 172.21.15.94:0/3311086999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984102f70 0x7fe984103380 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- 172.21.15.94:0/3311086999 >> 172.21.15.94:0/3311086999 conn(0x7fe9840fd3b0 msgr2=0x7fe9840ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- 172.21.15.94:0/3311086999 shutdown_connections 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- 172.21.15.94:0/3311086999 wait complete. 2024-04-25T13:41:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 Processor -- start 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 -- start start 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984102f70 0x7fe984102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.500+0000 7fe989a96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984104470 0x7fe9841a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe989a96640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410e510 con 0x7fe984103950 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe989a96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410e390 con 0x7fe984102f70 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe989a96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410e690 con 0x7fe984104470 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54150/0 (socket says 172.21.15.94:54150) 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 -- 172.21.15.94:0/919906189 learned_addr learned my addr 172.21.15.94:0/919906189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe9837fe640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984102f70 0x7fe984102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe983fff640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984104470 0x7fe9841a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 -- 172.21.15.94:0/919906189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984104470 msgr2=0x7fe9841a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984104470 0x7fe9841a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 -- 172.21.15.94:0/919906189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984102f70 msgr2=0x7fe984102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984102f70 0x7fe984102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 -- 172.21.15.94:0/919906189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9841a1d00 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe97bfff640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d770 tx=0x7fe96c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017070 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96c0054d0 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9841a1ff0 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004c30 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.501+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9841a2460 con 0x7fe984103950 2024-04-25T13:41:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.502+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe984107910 con 0x7fe984103950 2024-04-25T13:41:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.504+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fe96c004dd0 con 0x7fe984103950 2024-04-25T13:41:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.504+0000 7fe9817fa640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 0x7fe9700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.505+0000 7fe9837fe640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 0x7fe9700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.505+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe96c0ded60 con 0x7fe984103950 2024-04-25T13:41:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.505+0000 7fe9837fe640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 0x7fe9700b2090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe974009230 tx=0x7fe974031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.508+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe96c0a7740 con 0x7fe984103950 2024-04-25T13:41:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2736140735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:46 smithi137 ceph-mon[32568]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:46 smithi137 ceph-mon[32568]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T13:41:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.683+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 --> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe984062fe0 con 0x7fe9700afb80 2024-04-25T13:41:46.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.690+0000 7fe9817fa640 1 -- 172.21.15.94:0/919906189 <== mgr.14152 v2:172.21.15.94:6800/449370799 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7fe984062fe0 con 0x7fe9700afb80 2024-04-25T13:41:46.693 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:46.694 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 msgr2=0x7fe9700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 0x7fe9700b2090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe974009230 tx=0x7fe974031400 comp rx=0 tx=0).stop 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 msgr2=0x7fe98419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d770 tx=0x7fe96c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 shutdown_connections 2024-04-25T13:41:46.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9700afb80 0x7fe9700b2090 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984104470 0x7fe9841a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe984103950 0x7fe98419ef30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 --2- 172.21.15.94:0/919906189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe984102f70 0x7fe984102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 >> 172.21.15.94:0/919906189 conn(0x7fe9840fd3b0 msgr2=0x7fe984104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 shutdown_connections 2024-04-25T13:41:46.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.693+0000 7fe989a96640 1 -- 172.21.15.94:0/919906189 wait complete. 2024-04-25T13:41:46.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2736140735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:46.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:46 smithi094 ceph-mon[29195]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:46.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:46 smithi094 ceph-mon[29195]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T13:41:46.705 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-25T13:41:46.288071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662397,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1249,"num_read_kb":731,"num_write":380,"num_write_kb":2301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412104,"kb_used_data":12984,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529720,"statfs":{"total":1151940427776,"available":1151518433280,"internally_reserved":0,"allocated":13295616,"data_stored":7254099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5712,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":58,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001865"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357193+0000","last_change":"2024-04-25T13:39:45.270218+0000","last_active":"2024-04-25T13:41:41.357193+0000","last_peered":"2024-04-25T13:41:41.357193+0000","last_clean":"2024-04-25T13:41:41.357193+0000","last_became_active":"2024-04-25T13:39:45.269886+0000","last_became_peered":"2024-04-25T13:39:45.269886+0000","last_unstale":"2024-04-25T13:41:41.357193+0000","last_undegraded":"2024-04-25T13:41:41.357193+0000","last_fullsized":"2024-04-25T13:41:41.357193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:52.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362416+0000","last_change":"2024-04-25T13:39:45.267079+0000","last_active":"2024-04-25T13:41:41.362416+0000","last_peered":"2024-04-25T13:41:41.362416+0000","last_clean":"2024-04-25T13:41:41.362416+0000","last_became_active":"2024-04-25T13:39:45.266559+0000","last_became_peered":"2024-04-25T13:39:45.266559+0000","last_unstale":"2024-04-25T13:41:41.362416+0000","last_undegraded":"2024-04-25T13:41:41.362416+0000","last_fullsized":"2024-04-25T13:41:41.362416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:52.840383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362698+0000","last_change":"2024-04-25T13:39:45.267557+0000","last_active":"2024-04-25T13:41:41.362698+0000","last_peered":"2024-04-25T13:41:41.362698+0000","last_clean":"2024-04-25T13:41:41.362698+0000","last_became_active":"2024-04-25T13:39:45.267002+0000","last_became_peered":"2024-04-25T13:39:45.267002+0000","last_unstale":"2024-04-25T13:41:41.362698+0000","last_undegraded":"2024-04-25T13:41:41.362698+0000","last_fullsized":"2024-04-25T13:41:41.362698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360147+0000","last_change":"2024-04-25T13:39:45.270380+0000","last_active":"2024-04-25T13:41:41.360147+0000","last_peered":"2024-04-25T13:41:41.360147+0000","last_clean":"2024-04-25T13:41:41.360147+0000","last_became_active":"2024-04-25T13:39:45.270029+0000","last_became_peered":"2024-04-25T13:39:45.270029+0000","last_unstale":"2024-04-25T13:41:41.360147+0000","last_undegraded":"2024-04-25T13:41:41.360147+0000","last_fullsized":"2024-04-25T13:41:41.360147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:17.262133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446495+0000","last_change":"2024-04-25T13:39:45.267908+0000","last_active":"2024-04-25T13:41:41.446495+0000","last_peered":"2024-04-25T13:41:41.446495+0000","last_clean":"2024-04-25T13:41:41.446495+0000","last_became_active":"2024-04-25T13:39:45.267555+0000","last_became_peered":"2024-04-25T13:39:45.267555+0000","last_unstale":"2024-04-25T13:41:41.446495+0000","last_undegraded":"2024-04-25T13:41:41.446495+0000","last_fullsized":"2024-04-25T13:41:41.446495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:53.256355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360468+0000","last_change":"2024-04-25T13:39:45.269803+0000","last_active":"2024-04-25T13:41:41.360468+0000","last_peered":"2024-04-25T13:41:41.360468+0000","last_clean":"2024-04-25T13:41:41.360468+0000","last_became_active":"2024-04-25T13:39:45.268375+0000","last_became_peered":"2024-04-25T13:39:45.268375+0000","last_unstale":"2024-04-25T13:41:41.360468+0000","last_undegraded":"2024-04-25T13:41:41.360468+0000","last_fullsized":"2024-04-25T13:41:41.360468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:54.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357446+0000","last_change":"2024-04-25T13:39:45.262857+0000","last_active":"2024-04-25T13:41:41.357446+0000","last_peered":"2024-04-25T13:41:41.357446+0000","last_clean":"2024-04-25T13:41:41.357446+0000","last_became_active":"2024-04-25T13:39:45.262451+0000","last_became_peered":"2024-04-25T13:39:45.262451+0000","last_unstale":"2024-04-25T13:41:41.357446+0000","last_undegraded":"2024-04-25T13:41:41.357446+0000","last_fullsized":"2024-04-25T13:41:41.357446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:00.669746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444909+0000","last_change":"2024-04-25T13:39:45.264051+0000","last_active":"2024-04-25T13:41:41.444909+0000","last_peered":"2024-04-25T13:41:41.444909+0000","last_clean":"2024-04-25T13:41:41.444909+0000","last_became_active":"2024-04-25T13:39:45.263663+0000","last_became_peered":"2024-04-25T13:39:45.263663+0000","last_unstale":"2024-04-25T13:41:41.444909+0000","last_undegraded":"2024-04-25T13:41:41.444909+0000","last_fullsized":"2024-04-25T13:41:41.444909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:14.310883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360334+0000","last_change":"2024-04-25T13:40:41.273152+0000","last_active":"2024-04-25T13:41:41.360334+0000","last_peered":"2024-04-25T13:41:41.360334+0000","last_clean":"2024-04-25T13:41:41.360334+0000","last_became_active":"2024-04-25T13:40:41.272760+0000","last_became_peered":"2024-04-25T13:40:41.272760+0000","last_unstale":"2024-04-25T13:41:41.360334+0000","last_undegraded":"2024-04-25T13:41:41.360334+0000","last_fullsized":"2024-04-25T13:41:41.360334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:35.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361389+0000","last_change":"2024-04-25T13:40:41.274781+0000","last_active":"2024-04-25T13:41:41.361389+0000","last_peered":"2024-04-25T13:41:41.361389+0000","last_clean":"2024-04-25T13:41:41.361389+0000","last_became_active":"2024-04-25T13:40:41.274043+0000","last_became_peered":"2024-04-25T13:40:41.274043+0000","last_unstale":"2024-04-25T13:41:41.361389+0000","last_undegraded":"2024-04-25T13:41:41.361389+0000","last_fullsized":"2024-04-25T13:41:41.361389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:01.989580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358355+0000","last_change":"2024-04-25T13:39:45.264776+0000","last_active":"2024-04-25T13:41:41.358355+0000","last_peered":"2024-04-25T13:41:41.358355+0000","last_clean":"2024-04-25T13:41:41.358355+0000","last_became_active":"2024-04-25T13:39:45.264375+0000","last_became_peered":"2024-04-25T13:39:45.264375+0000","last_unstale":"2024-04-25T13:41:41.358355+0000","last_undegraded":"2024-04-25T13:41:41.358355+0000","last_fullsized":"2024-04-25T13:41:41.358355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:21.514234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360296+0000","last_change":"2024-04-25T13:39:45.272077+0000","last_active":"2024-04-25T13:41:41.360296+0000","last_peered":"2024-04-25T13:41:41.360296+0000","last_clean":"2024-04-25T13:41:41.360296+0000","last_became_active":"2024-04-25T13:39:45.271777+0000","last_became_peered":"2024-04-25T13:39:45.271777+0000","last_unstale":"2024-04-25T13:41:41.360296+0000","last_undegraded":"2024-04-25T13:41:41.360296+0000","last_fullsized":"2024-04-25T13:41:41.360296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:50.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359749+0000","last_change":"2024-04-25T13:39:45.267658+0000","last_active":"2024-04-25T13:41:41.359749+0000","last_peered":"2024-04-25T13:41:41.359749+0000","last_clean":"2024-04-25T13:41:41.359749+0000","last_became_active":"2024-04-25T13:39:45.267324+0000","last_became_peered":"2024-04-25T13:39:45.267324+0000","last_unstale":"2024-04-25T13:41:41.359749+0000","last_undegraded":"2024-04-25T13:41:41.359749+0000","last_fullsized":"2024-04-25T13:41:41.359749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:49.533154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362114+0000","last_change":"2024-04-25T13:39:45.265569+0000","last_active":"2024-04-25T13:41:41.362114+0000","last_peered":"2024-04-25T13:41:41.362114+0000","last_clean":"2024-04-25T13:41:41.362114+0000","last_became_active":"2024-04-25T13:39:45.265252+0000","last_became_peered":"2024-04-25T13:39:45.265252+0000","last_unstale":"2024-04-25T13:41:41.362114+0000","last_undegraded":"2024-04-25T13:41:41.362114+0000","last_fullsized":"2024-04-25T13:41:41.362114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:43.538797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361807+0000","last_change":"2024-04-25T13:39:45.269671+0000","last_active":"2024-04-25T13:41:41.361807+0000","last_peered":"2024-04-25T13:41:41.361807+0000","last_clean":"2024-04-25T13:41:41.361807+0000","last_became_active":"2024-04-25T13:39:45.269347+0000","last_became_peered":"2024-04-25T13:39:45.269347+0000","last_unstale":"2024-04-25T13:41:41.361807+0000","last_undegraded":"2024-04-25T13:41:41.361807+0000","last_fullsized":"2024-04-25T13:41:41.361807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:12.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358442+0000","last_change":"2024-04-25T13:39:45.264808+0000","last_active":"2024-04-25T13:41:41.358442+0000","last_peered":"2024-04-25T13:41:41.358442+0000","last_clean":"2024-04-25T13:41:41.358442+0000","last_became_active":"2024-04-25T13:39:45.264440+0000","last_became_peered":"2024-04-25T13:39:45.264440+0000","last_unstale":"2024-04-25T13:41:41.358442+0000","last_undegraded":"2024-04-25T13:41:41.358442+0000","last_fullsized":"2024-04-25T13:41:41.358442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.164196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357296+0000","last_change":"2024-04-25T13:39:45.271844+0000","last_active":"2024-04-25T13:41:41.357296+0000","last_peered":"2024-04-25T13:41:41.357296+0000","last_clean":"2024-04-25T13:41:41.357296+0000","last_became_active":"2024-04-25T13:39:45.271547+0000","last_became_peered":"2024-04-25T13:39:45.271547+0000","last_unstale":"2024-04-25T13:41:41.357296+0000","last_undegraded":"2024-04-25T13:41:41.357296+0000","last_fullsized":"2024-04-25T13:41:41.357296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.885748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362394+0000","last_change":"2024-04-25T13:41:41.362394+0000","last_active":"2024-04-25T13:41:41.362394+0000","last_peered":"2024-04-25T13:41:41.362394+0000","last_clean":"2024-04-25T13:41:41.362394+0000","last_became_active":"2024-04-25T13:41:41.361973+0000","last_became_peered":"2024-04-25T13:41:41.361973+0000","last_unstale":"2024-04-25T13:41:41.362394+0000","last_undegraded":"2024-04-25T13:41:41.362394+0000","last_fullsized":"2024-04-25T13:41:41.362394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:44.656494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359196+0000","last_change":"2024-04-25T13:39:45.256957+0000","last_active":"2024-04-25T13:41:41.359196+0000","last_peered":"2024-04-25T13:41:41.359196+0000","last_clean":"2024-04-25T13:41:41.359196+0000","last_became_active":"2024-04-25T13:39:45.256583+0000","last_became_peered":"2024-04-25T13:39:45.256583+0000","last_unstale":"2024-04-25T13:41:41.359196+0000","last_undegraded":"2024-04-25T13:41:41.359196+0000","last_fullsized":"2024-04-25T13:41:41.359196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.313028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.809954+0000","last_change":"2024-04-25T13:39:44.627251+0000","last_active":"2024-04-25T13:41:42.809954+0000","last_peered":"2024-04-25T13:41:42.809954+0000","last_clean":"2024-04-25T13:41:42.809954+0000","last_became_active":"2024-04-25T13:39:44.625764+0000","last_became_peered":"2024-04-25T13:39:44.625764+0000","last_unstale":"2024-04-25T13:41:42.809954+0000","last_undegraded":"2024-04-25T13:41:42.809954+0000","last_fullsized":"2024-04-25T13:41:42.809954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:53.069311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445375+0000","last_change":"2024-04-25T13:39:45.263479+0000","last_active":"2024-04-25T13:41:41.445375+0000","last_peered":"2024-04-25T13:41:41.445375+0000","last_clean":"2024-04-25T13:41:41.445375+0000","last_became_active":"2024-04-25T13:39:45.263130+0000","last_became_peered":"2024-04-25T13:39:45.263130+0000","last_unstale":"2024-04-25T13:41:41.445375+0000","last_undegraded":"2024-04-25T13:41:41.445375+0000","last_fullsized":"2024-04-25T13:41:41.445375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445462+0000","last_change":"2024-04-25T13:39:44.629995+0000","last_active":"2024-04-25T13:41:41.445462+0000","last_peered":"2024-04-25T13:41:41.445462+0000","last_clean":"2024-04-25T13:41:41.445462+0000","last_became_active":"2024-04-25T13:39:44.629683+0000","last_became_peered":"2024-04-25T13:39:44.629683+0000","last_unstale":"2024-04-25T13:41:41.445462+0000","last_undegraded":"2024-04-25T13:41:41.445462+0000","last_fullsized":"2024-04-25T13:41:41.445462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:48.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362671+0000","last_change":"2024-04-25T13:39:45.259873+0000","last_active":"2024-04-25T13:41:41.362671+0000","last_peered":"2024-04-25T13:41:41.362671+0000","last_clean":"2024-04-25T13:41:41.362671+0000","last_became_active":"2024-04-25T13:39:45.259540+0000","last_became_peered":"2024-04-25T13:39:45.259540+0000","last_unstale":"2024-04-25T13:41:41.362671+0000","last_undegraded":"2024-04-25T13:41:41.362671+0000","last_fullsized":"2024-04-25T13:41:41.362671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:44.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446521+0000","last_change":"2024-04-25T13:39:44.625395+0000","last_active":"2024-04-25T13:41:41.446521+0000","last_peered":"2024-04-25T13:41:41.446521+0000","last_clean":"2024-04-25T13:41:41.446521+0000","last_became_active":"2024-04-25T13:39:44.624879+0000","last_became_peered":"2024-04-25T13:39:44.624879+0000","last_unstale":"2024-04-25T13:41:41.446521+0000","last_undegraded":"2024-04-25T13:41:41.446521+0000","last_fullsized":"2024-04-25T13:41:41.446521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.404246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359128+0000","last_change":"2024-04-25T13:39:45.259399+0000","last_active":"2024-04-25T13:41:41.359128+0000","last_peered":"2024-04-25T13:41:41.359128+0000","last_clean":"2024-04-25T13:41:41.359128+0000","last_became_active":"2024-04-25T13:39:45.259027+0000","last_became_peered":"2024-04-25T13:39:45.259027+0000","last_unstale":"2024-04-25T13:41:41.359128+0000","last_undegraded":"2024-04-25T13:41:41.359128+0000","last_fullsized":"2024-04-25T13:41:41.359128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446334+0000","last_change":"2024-04-25T13:39:44.618812+0000","last_active":"2024-04-25T13:41:41.446334+0000","last_peered":"2024-04-25T13:41:41.446334+0000","last_clean":"2024-04-25T13:41:41.446334+0000","last_became_active":"2024-04-25T13:39:44.618460+0000","last_became_peered":"2024-04-25T13:39:44.618460+0000","last_unstale":"2024-04-25T13:41:41.446334+0000","last_undegraded":"2024-04-25T13:41:41.446334+0000","last_fullsized":"2024-04-25T13:41:41.446334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.305567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361645+0000","last_change":"2024-04-25T13:39:45.272447+0000","last_active":"2024-04-25T13:41:41.361645+0000","last_peered":"2024-04-25T13:41:41.361645+0000","last_clean":"2024-04-25T13:41:41.361645+0000","last_became_active":"2024-04-25T13:39:45.272115+0000","last_became_peered":"2024-04-25T13:39:45.272115+0000","last_unstale":"2024-04-25T13:41:41.361645+0000","last_undegraded":"2024-04-25T13:41:41.361645+0000","last_fullsized":"2024-04-25T13:41:41.361645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:14.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.811056+0000","last_change":"2024-04-25T13:39:44.620131+0000","last_active":"2024-04-25T13:41:41.811056+0000","last_peered":"2024-04-25T13:41:41.811056+0000","last_clean":"2024-04-25T13:41:41.811056+0000","last_became_active":"2024-04-25T13:39:44.619778+0000","last_became_peered":"2024-04-25T13:39:44.619778+0000","last_unstale":"2024-04-25T13:41:41.811056+0000","last_undegraded":"2024-04-25T13:41:41.811056+0000","last_fullsized":"2024-04-25T13:41:41.811056+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446058+0000","last_change":"2024-04-25T13:40:41.272700+0000","last_active":"2024-04-25T13:41:41.446058+0000","last_peered":"2024-04-25T13:41:41.446058+0000","last_clean":"2024-04-25T13:41:41.446058+0000","last_became_active":"2024-04-25T13:40:41.272080+0000","last_became_peered":"2024-04-25T13:40:41.272080+0000","last_unstale":"2024-04-25T13:41:41.446058+0000","last_undegraded":"2024-04-25T13:41:41.446058+0000","last_fullsized":"2024-04-25T13:41:41.446058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.149458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360468+0000","last_change":"2024-04-25T13:40:41.280199+0000","last_active":"2024-04-25T13:41:41.360468+0000","last_peered":"2024-04-25T13:41:41.360468+0000","last_clean":"2024-04-25T13:41:41.360468+0000","last_became_active":"2024-04-25T13:40:41.268773+0000","last_became_peered":"2024-04-25T13:40:41.268773+0000","last_unstale":"2024-04-25T13:41:41.360468+0000","last_undegraded":"2024-04-25T13:41:41.360468+0000","last_fullsized":"2024-04-25T13:41:41.360468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:47.024600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.447783+0000","last_change":"2024-04-25T13:41:41.447783+0000","last_active":"2024-04-25T13:41:41.447783+0000","last_peered":"2024-04-25T13:41:41.447783+0000","last_clean":"2024-04-25T13:41:41.447783+0000","last_became_active":"2024-04-25T13:41:41.447293+0000","last_became_peered":"2024-04-25T13:41:41.447293+0000","last_unstale":"2024-04-25T13:41:41.447783+0000","last_undegraded":"2024-04-25T13:41:41.447783+0000","last_fullsized":"2024-04-25T13:41:41.447783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:39.807786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359388+0000","last_change":"2024-04-25T13:39:44.618973+0000","last_active":"2024-04-25T13:41:41.359388+0000","last_peered":"2024-04-25T13:41:41.359388+0000","last_clean":"2024-04-25T13:41:41.359388+0000","last_became_active":"2024-04-25T13:39:44.618620+0000","last_became_peered":"2024-04-25T13:39:44.618620+0000","last_unstale":"2024-04-25T13:41:41.359388+0000","last_undegraded":"2024-04-25T13:41:41.359388+0000","last_fullsized":"2024-04-25T13:41:41.359388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:03.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.444835+0000","last_change":"2024-04-25T13:39:45.260851+0000","last_active":"2024-04-25T13:41:41.444835+0000","last_peered":"2024-04-25T13:41:41.444835+0000","last_clean":"2024-04-25T13:41:41.444835+0000","last_became_active":"2024-04-25T13:39:45.260517+0000","last_became_peered":"2024-04-25T13:39:45.260517+0000","last_unstale":"2024-04-25T13:41:41.444835+0000","last_undegraded":"2024-04-25T13:41:41.444835+0000","last_fullsized":"2024-04-25T13:41:41.444835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.431965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360949+0000","last_change":"2024-04-25T13:39:44.622407+0000","last_active":"2024-04-25T13:41:41.360949+0000","last_peered":"2024-04-25T13:41:41.360949+0000","last_clean":"2024-04-25T13:41:41.360949+0000","last_became_active":"2024-04-25T13:39:44.622091+0000","last_became_peered":"2024-04-25T13:39:44.622091+0000","last_unstale":"2024-04-25T13:41:41.360949+0000","last_undegraded":"2024-04-25T13:41:41.360949+0000","last_fullsized":"2024-04-25T13:41:41.360949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.011641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.355975+0000","last_change":"2024-04-25T13:39:45.264489+0000","last_active":"2024-04-25T13:41:41.355975+0000","last_peered":"2024-04-25T13:41:41.355975+0000","last_clean":"2024-04-25T13:41:41.355975+0000","last_became_active":"2024-04-25T13:39:45.264092+0000","last_became_peered":"2024-04-25T13:39:45.264092+0000","last_unstale":"2024-04-25T13:41:41.355975+0000","last_undegraded":"2024-04-25T13:41:41.355975+0000","last_fullsized":"2024-04-25T13:41:41.355975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:23.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":256,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:44.814249+0000","last_change":"2024-04-25T13:39:44.621166+0000","last_active":"2024-04-25T13:41:44.814249+0000","last_peered":"2024-04-25T13:41:44.814249+0000","last_clean":"2024-04-25T13:41:44.814249+0000","last_became_active":"2024-04-25T13:39:44.620486+0000","last_became_peered":"2024-04-25T13:39:44.620486+0000","last_unstale":"2024-04-25T13:41:44.814249+0000","last_undegraded":"2024-04-25T13:41:44.814249+0000","last_fullsized":"2024-04-25T13:41:44.814249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.236160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359936+0000","last_change":"2024-04-25T13:39:45.258960+0000","last_active":"2024-04-25T13:41:41.359936+0000","last_peered":"2024-04-25T13:41:41.359936+0000","last_clean":"2024-04-25T13:41:41.359936+0000","last_became_active":"2024-04-25T13:39:45.258603+0000","last_became_peered":"2024-04-25T13:39:45.258603+0000","last_unstale":"2024-04-25T13:41:41.359936+0000","last_undegraded":"2024-04-25T13:41:41.359936+0000","last_fullsized":"2024-04-25T13:41:41.359936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:43.083672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358783+0000","last_change":"2024-04-25T13:39:44.618946+0000","last_active":"2024-04-25T13:41:41.358783+0000","last_peered":"2024-04-25T13:41:41.358783+0000","last_clean":"2024-04-25T13:41:41.358783+0000","last_became_active":"2024-04-25T13:39:44.618614+0000","last_became_peered":"2024-04-25T13:39:44.618614+0000","last_unstale":"2024-04-25T13:41:41.358783+0000","last_undegraded":"2024-04-25T13:41:41.358783+0000","last_fullsized":"2024-04-25T13:41:41.358783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:59.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362772+0000","last_change":"2024-04-25T13:39:45.269917+0000","last_active":"2024-04-25T13:41:41.362772+0000","last_peered":"2024-04-25T13:41:41.362772+0000","last_clean":"2024-04-25T13:41:41.362772+0000","last_became_active":"2024-04-25T13:39:45.269016+0000","last_became_peered":"2024-04-25T13:39:45.269016+0000","last_unstale":"2024-04-25T13:41:41.362772+0000","last_undegraded":"2024-04-25T13:41:41.362772+0000","last_fullsized":"2024-04-25T13:41:41.362772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:23.709243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361825+0000","last_change":"2024-04-25T13:39:44.621607+0000","last_active":"2024-04-25T13:41:41.361825+0000","last_peered":"2024-04-25T13:41:41.361825+0000","last_clean":"2024-04-25T13:41:41.361825+0000","last_became_active":"2024-04-25T13:39:44.621262+0000","last_became_peered":"2024-04-25T13:39:44.621262+0000","last_unstale":"2024-04-25T13:41:41.361825+0000","last_undegraded":"2024-04-25T13:41:41.361825+0000","last_fullsized":"2024-04-25T13:41:41.361825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:07.638190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359348+0000","last_change":"2024-04-25T13:39:45.269617+0000","last_active":"2024-04-25T13:41:41.359348+0000","last_peered":"2024-04-25T13:41:41.359348+0000","last_clean":"2024-04-25T13:41:41.359348+0000","last_became_active":"2024-04-25T13:39:45.269252+0000","last_became_peered":"2024-04-25T13:39:45.269252+0000","last_unstale":"2024-04-25T13:41:41.359348+0000","last_undegraded":"2024-04-25T13:41:41.359348+0000","last_fullsized":"2024-04-25T13:41:41.359348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:46.941860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:44.822900+0000","last_change":"2024-04-25T13:39:44.624521+0000","last_active":"2024-04-25T13:41:44.822900+0000","last_peered":"2024-04-25T13:41:44.822900+0000","last_clean":"2024-04-25T13:41:44.822900+0000","last_became_active":"2024-04-25T13:39:44.624201+0000","last_became_peered":"2024-04-25T13:39:44.624201+0000","last_unstale":"2024-04-25T13:41:44.822900+0000","last_undegraded":"2024-04-25T13:41:44.822900+0000","last_fullsized":"2024-04-25T13:41:44.822900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:38.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64142,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":181,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446277+0000","last_change":"2024-04-25T13:39:45.269771+0000","last_active":"2024-04-25T13:41:41.446277+0000","last_peered":"2024-04-25T13:41:41.446277+0000","last_clean":"2024-04-25T13:41:41.446277+0000","last_became_active":"2024-04-25T13:39:45.269395+0000","last_became_peered":"2024-04-25T13:39:45.269395+0000","last_unstale":"2024-04-25T13:41:41.446277+0000","last_undegraded":"2024-04-25T13:41:41.446277+0000","last_fullsized":"2024-04-25T13:41:41.446277+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.303288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358927+0000","last_change":"2024-04-25T13:39:44.623920+0000","last_active":"2024-04-25T13:41:41.358927+0000","last_peered":"2024-04-25T13:41:41.358927+0000","last_clean":"2024-04-25T13:41:41.358927+0000","last_became_active":"2024-04-25T13:39:44.623471+0000","last_became_peered":"2024-04-25T13:39:44.623471+0000","last_unstale":"2024-04-25T13:41:41.358927+0000","last_undegraded":"2024-04-25T13:41:41.358927+0000","last_fullsized":"2024-04-25T13:41:41.358927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:02.572232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359428+0000","last_change":"2024-04-25T13:39:45.264975+0000","last_active":"2024-04-25T13:41:41.359428+0000","last_peered":"2024-04-25T13:41:41.359428+0000","last_clean":"2024-04-25T13:41:41.359428+0000","last_became_active":"2024-04-25T13:39:45.263458+0000","last_became_peered":"2024-04-25T13:39:45.263458+0000","last_unstale":"2024-04-25T13:41:41.359428+0000","last_undegraded":"2024-04-25T13:41:41.359428+0000","last_fullsized":"2024-04-25T13:41:41.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:34.214783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361253+0000","last_change":"2024-04-25T13:39:44.629000+0000","last_active":"2024-04-25T13:41:41.361253+0000","last_peered":"2024-04-25T13:41:41.361253+0000","last_clean":"2024-04-25T13:41:41.361253+0000","last_became_active":"2024-04-25T13:39:44.628567+0000","last_became_peered":"2024-04-25T13:39:44.628567+0000","last_unstale":"2024-04-25T13:41:41.361253+0000","last_undegraded":"2024-04-25T13:41:41.361253+0000","last_fullsized":"2024-04-25T13:41:41.361253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:16.508354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358264+0000","last_change":"2024-04-25T13:39:45.271342+0000","last_active":"2024-04-25T13:41:41.358264+0000","last_peered":"2024-04-25T13:41:41.358264+0000","last_clean":"2024-04-25T13:41:41.358264+0000","last_became_active":"2024-04-25T13:39:45.270960+0000","last_became_peered":"2024-04-25T13:39:45.270960+0000","last_unstale":"2024-04-25T13:41:41.358264+0000","last_undegraded":"2024-04-25T13:41:41.358264+0000","last_fullsized":"2024-04-25T13:41:41.358264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.680581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.820375+0000","last_change":"2024-04-25T13:39:44.623199+0000","last_active":"2024-04-25T13:41:42.820375+0000","last_peered":"2024-04-25T13:41:42.820375+0000","last_clean":"2024-04-25T13:41:42.820375+0000","last_became_active":"2024-04-25T13:39:44.622759+0000","last_became_peered":"2024-04-25T13:39:44.622759+0000","last_unstale":"2024-04-25T13:41:42.820375+0000","last_undegraded":"2024-04-25T13:41:42.820375+0000","last_fullsized":"2024-04-25T13:41:42.820375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:17.140231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357450+0000","last_change":"2024-04-25T13:39:45.264533+0000","last_active":"2024-04-25T13:41:41.357450+0000","last_peered":"2024-04-25T13:41:41.357450+0000","last_clean":"2024-04-25T13:41:41.357450+0000","last_became_active":"2024-04-25T13:39:45.264033+0000","last_became_peered":"2024-04-25T13:39:45.264033+0000","last_unstale":"2024-04-25T13:41:41.357450+0000","last_undegraded":"2024-04-25T13:41:41.357450+0000","last_fullsized":"2024-04-25T13:41:41.357450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:32.716713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:44.621215+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:44.620888+0000","last_became_peered":"2024-04-25T13:39:44.620888+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.490234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362909+0000","last_change":"2024-04-25T13:41:41.362306+0000","last_active":"2024-04-25T13:41:41.362909+0000","last_peered":"2024-04-25T13:41:41.362909+0000","last_clean":"2024-04-25T13:41:41.362909+0000","last_became_active":"2024-04-25T13:41:41.361910+0000","last_became_peered":"2024-04-25T13:41:41.361910+0000","last_unstale":"2024-04-25T13:41:41.362909+0000","last_undegraded":"2024-04-25T13:41:41.362909+0000","last_fullsized":"2024-04-25T13:41:41.362909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:09.616231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446655+0000","last_change":"2024-04-25T13:39:44.624652+0000","last_active":"2024-04-25T13:41:41.446655+0000","last_peered":"2024-04-25T13:41:41.446655+0000","last_clean":"2024-04-25T13:41:41.446655+0000","last_became_active":"2024-04-25T13:39:44.624262+0000","last_became_peered":"2024-04-25T13:39:44.624262+0000","last_unstale":"2024-04-25T13:41:41.446655+0000","last_undegraded":"2024-04-25T13:41:41.446655+0000","last_fullsized":"2024-04-25T13:41:41.446655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:44.266480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357273+0000","last_change":"2024-04-25T13:40:41.271904+0000","last_active":"2024-04-25T13:41:41.357273+0000","last_peered":"2024-04-25T13:41:41.357273+0000","last_clean":"2024-04-25T13:41:41.357273+0000","last_became_active":"2024-04-25T13:40:41.271457+0000","last_became_peered":"2024-04-25T13:40:41.271457+0000","last_unstale":"2024-04-25T13:41:41.357273+0000","last_undegraded":"2024-04-25T13:41:41.357273+0000","last_fullsized":"2024-04-25T13:41:41.357273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:04.766001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356059+0000","last_change":"2024-04-25T13:39:44.615534+0000","last_active":"2024-04-25T13:41:41.356059+0000","last_peered":"2024-04-25T13:41:41.356059+0000","last_clean":"2024-04-25T13:41:41.356059+0000","last_became_active":"2024-04-25T13:39:44.615148+0000","last_became_peered":"2024-04-25T13:39:44.615148+0000","last_unstale":"2024-04-25T13:41:41.356059+0000","last_undegraded":"2024-04-25T13:41:41.356059+0000","last_fullsized":"2024-04-25T13:41:41.356059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:02.529944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360789+0000","last_change":"2024-04-25T13:39:45.257839+0000","last_active":"2024-04-25T13:41:41.360789+0000","last_peered":"2024-04-25T13:41:41.360789+0000","last_clean":"2024-04-25T13:41:41.360789+0000","last_became_active":"2024-04-25T13:39:45.257468+0000","last_became_peered":"2024-04-25T13:39:45.257468+0000","last_unstale":"2024-04-25T13:41:41.360789+0000","last_undegraded":"2024-04-25T13:41:41.360789+0000","last_fullsized":"2024-04-25T13:41:41.360789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:09.657120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.811283+0000","last_change":"2024-04-25T13:39:44.620255+0000","last_active":"2024-04-25T13:41:42.811283+0000","last_peered":"2024-04-25T13:41:42.811283+0000","last_clean":"2024-04-25T13:41:42.811283+0000","last_became_active":"2024-04-25T13:39:44.619906+0000","last_became_peered":"2024-04-25T13:39:44.619906+0000","last_unstale":"2024-04-25T13:41:42.811283+0000","last_undegraded":"2024-04-25T13:41:42.811283+0000","last_fullsized":"2024-04-25T13:41:42.811283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:30.508097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360718+0000","last_change":"2024-04-25T13:39:45.269715+0000","last_active":"2024-04-25T13:41:41.360718+0000","last_peered":"2024-04-25T13:41:41.360718+0000","last_clean":"2024-04-25T13:41:41.360718+0000","last_became_active":"2024-04-25T13:39:45.269186+0000","last_became_peered":"2024-04-25T13:39:45.269186+0000","last_unstale":"2024-04-25T13:41:41.360718+0000","last_undegraded":"2024-04-25T13:41:41.360718+0000","last_fullsized":"2024-04-25T13:41:41.360718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.305126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361116+0000","last_change":"2024-04-25T13:39:45.271753+0000","last_active":"2024-04-25T13:41:41.361116+0000","last_peered":"2024-04-25T13:41:41.361116+0000","last_clean":"2024-04-25T13:41:41.361116+0000","last_became_active":"2024-04-25T13:39:45.271171+0000","last_became_peered":"2024-04-25T13:39:45.271171+0000","last_unstale":"2024-04-25T13:41:41.361116+0000","last_undegraded":"2024-04-25T13:41:41.361116+0000","last_fullsized":"2024-04-25T13:41:41.361116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:40.521827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.819794+0000","last_change":"2024-04-25T13:39:44.629961+0000","last_active":"2024-04-25T13:41:42.819794+0000","last_peered":"2024-04-25T13:41:42.819794+0000","last_clean":"2024-04-25T13:41:42.819794+0000","last_became_active":"2024-04-25T13:39:44.629602+0000","last_became_peered":"2024-04-25T13:39:44.629602+0000","last_unstale":"2024-04-25T13:41:42.819794+0000","last_undegraded":"2024-04-25T13:41:42.819794+0000","last_fullsized":"2024-04-25T13:41:42.819794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.585840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445164+0000","last_change":"2024-04-25T13:39:45.263469+0000","last_active":"2024-04-25T13:41:41.445164+0000","last_peered":"2024-04-25T13:41:41.445164+0000","last_clean":"2024-04-25T13:41:41.445164+0000","last_became_active":"2024-04-25T13:39:45.263108+0000","last_became_peered":"2024-04-25T13:39:45.263108+0000","last_unstale":"2024-04-25T13:41:41.445164+0000","last_undegraded":"2024-04-25T13:41:41.445164+0000","last_fullsized":"2024-04-25T13:41:41.445164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.388417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360618+0000","last_change":"2024-04-25T13:39:44.621474+0000","last_active":"2024-04-25T13:41:41.360618+0000","last_peered":"2024-04-25T13:41:41.360618+0000","last_clean":"2024-04-25T13:41:41.360618+0000","last_became_active":"2024-04-25T13:39:44.621084+0000","last_became_peered":"2024-04-25T13:39:44.621084+0000","last_unstale":"2024-04-25T13:41:41.360618+0000","last_undegraded":"2024-04-25T13:41:41.360618+0000","last_fullsized":"2024-04-25T13:41:41.360618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:22.323441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357715+0000","last_change":"2024-04-25T13:38:31.975106+0000","last_active":"2024-04-25T13:41:41.357715+0000","last_peered":"2024-04-25T13:41:41.357715+0000","last_clean":"2024-04-25T13:41:41.357715+0000","last_became_active":"2024-04-25T13:38:31.968405+0000","last_became_peered":"2024-04-25T13:38:31.968405+0000","last_unstale":"2024-04-25T13:41:41.357715+0000","last_undegraded":"2024-04-25T13:41:41.357715+0000","last_fullsized":"2024-04-25T13:41:41.357715+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:35:09.147083+0000","last_clean_scrub_stamp":"2024-04-25T13:35:09.147083+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:30.069943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359258+0000","last_change":"2024-04-25T13:39:45.266589+0000","last_active":"2024-04-25T13:41:41.359258+0000","last_peered":"2024-04-25T13:41:41.359258+0000","last_clean":"2024-04-25T13:41:41.359258+0000","last_became_active":"2024-04-25T13:39:45.266220+0000","last_became_peered":"2024-04-25T13:39:45.266220+0000","last_unstale":"2024-04-25T13:41:41.359258+0000","last_undegraded":"2024-04-25T13:41:41.359258+0000","last_fullsized":"2024-04-25T13:41:41.359258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:43.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358003+0000","last_change":"2024-04-25T13:39:44.625901+0000","last_active":"2024-04-25T13:41:41.358003+0000","last_peered":"2024-04-25T13:41:41.358003+0000","last_clean":"2024-04-25T13:41:41.358003+0000","last_became_active":"2024-04-25T13:39:44.625425+0000","last_became_peered":"2024-04-25T13:39:44.625425+0000","last_unstale":"2024-04-25T13:41:41.358003+0000","last_undegraded":"2024-04-25T13:41:41.358003+0000","last_fullsized":"2024-04-25T13:41:41.358003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.698349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356936+0000","last_change":"2024-04-25T13:40:41.280650+0000","last_active":"2024-04-25T13:41:41.356936+0000","last_peered":"2024-04-25T13:41:41.356936+0000","last_clean":"2024-04-25T13:41:41.356936+0000","last_became_active":"2024-04-25T13:40:41.271193+0000","last_became_peered":"2024-04-25T13:40:41.271193+0000","last_unstale":"2024-04-25T13:41:41.356936+0000","last_undegraded":"2024-04-25T13:41:41.356936+0000","last_fullsized":"2024-04-25T13:41:41.356936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:23.137595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359610+0000","last_change":"2024-04-25T13:39:45.257635+0000","last_active":"2024-04-25T13:41:41.359610+0000","last_peered":"2024-04-25T13:41:41.359610+0000","last_clean":"2024-04-25T13:41:41.359610+0000","last_became_active":"2024-04-25T13:39:45.257210+0000","last_became_peered":"2024-04-25T13:39:45.257210+0000","last_unstale":"2024-04-25T13:41:41.359610+0000","last_undegraded":"2024-04-25T13:41:41.359610+0000","last_fullsized":"2024-04-25T13:41:41.359610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.470843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.629856+0000","last_change":"2024-04-25T13:39:44.626253+0000","last_active":"2024-04-25T13:41:41.629856+0000","last_peered":"2024-04-25T13:41:41.629856+0000","last_clean":"2024-04-25T13:41:41.629856+0000","last_became_active":"2024-04-25T13:39:44.625924+0000","last_became_peered":"2024-04-25T13:39:44.625924+0000","last_unstale":"2024-04-25T13:41:41.629856+0000","last_undegraded":"2024-04-25T13:41:41.629856+0000","last_fullsized":"2024-04-25T13:41:41.629856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:28.988535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445401+0000","last_change":"2024-04-25T13:39:45.261765+0000","last_active":"2024-04-25T13:41:41.445401+0000","last_peered":"2024-04-25T13:41:41.445401+0000","last_clean":"2024-04-25T13:41:41.445401+0000","last_became_active":"2024-04-25T13:39:45.261434+0000","last_became_peered":"2024-04-25T13:39:45.261434+0000","last_unstale":"2024-04-25T13:41:41.445401+0000","last_undegraded":"2024-04-25T13:41:41.445401+0000","last_fullsized":"2024-04-25T13:41:41.445401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:55.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.360057+0000","last_change":"2024-04-25T13:39:44.622474+0000","last_active":"2024-04-25T13:41:41.360057+0000","last_peered":"2024-04-25T13:41:41.360057+0000","last_clean":"2024-04-25T13:41:41.360057+0000","last_became_active":"2024-04-25T13:39:44.622120+0000","last_became_peered":"2024-04-25T13:39:44.622120+0000","last_unstale":"2024-04-25T13:41:41.360057+0000","last_undegraded":"2024-04-25T13:41:41.360057+0000","last_fullsized":"2024-04-25T13:41:41.360057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:17.668503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.362311+0000","last_change":"2024-04-25T13:39:45.269508+0000","last_active":"2024-04-25T13:41:41.362311+0000","last_peered":"2024-04-25T13:41:41.362311+0000","last_clean":"2024-04-25T13:41:41.362311+0000","last_became_active":"2024-04-25T13:39:45.269190+0000","last_became_peered":"2024-04-25T13:39:45.269190+0000","last_unstale":"2024-04-25T13:41:41.362311+0000","last_undegraded":"2024-04-25T13:41:41.362311+0000","last_fullsized":"2024-04-25T13:41:41.362311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:40.420801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358635+0000","last_change":"2024-04-25T13:39:44.626835+0000","last_active":"2024-04-25T13:41:41.358635+0000","last_peered":"2024-04-25T13:41:41.358635+0000","last_clean":"2024-04-25T13:41:41.358635+0000","last_became_active":"2024-04-25T13:39:44.626533+0000","last_became_peered":"2024-04-25T13:39:44.626533+0000","last_unstale":"2024-04-25T13:41:41.358635+0000","last_undegraded":"2024-04-25T13:41:41.358635+0000","last_fullsized":"2024-04-25T13:41:41.358635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.323493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445922+0000","last_change":"2024-04-25T13:39:45.268737+0000","last_active":"2024-04-25T13:41:41.445922+0000","last_peered":"2024-04-25T13:41:41.445922+0000","last_clean":"2024-04-25T13:41:41.445922+0000","last_became_active":"2024-04-25T13:39:45.268430+0000","last_became_peered":"2024-04-25T13:39:45.268430+0000","last_unstale":"2024-04-25T13:41:41.445922+0000","last_undegraded":"2024-04-25T13:41:41.445922+0000","last_fullsized":"2024-04-25T13:41:41.445922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.076925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445009+0000","last_change":"2024-04-25T13:39:44.630389+0000","last_active":"2024-04-25T13:41:41.445009+0000","last_peered":"2024-04-25T13:41:41.445009+0000","last_clean":"2024-04-25T13:41:41.445009+0000","last_became_active":"2024-04-25T13:39:44.628999+0000","last_became_peered":"2024-04-25T13:39:44.628999+0000","last_unstale":"2024-04-25T13:41:41.445009+0000","last_undegraded":"2024-04-25T13:41:41.445009+0000","last_fullsized":"2024-04-25T13:41:41.445009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:03.876760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356645+0000","last_change":"2024-04-25T13:39:45.271102+0000","last_active":"2024-04-25T13:41:41.356645+0000","last_peered":"2024-04-25T13:41:41.356645+0000","last_clean":"2024-04-25T13:41:41.356645+0000","last_became_active":"2024-04-25T13:39:45.270782+0000","last_became_peered":"2024-04-25T13:39:45.270782+0000","last_unstale":"2024-04-25T13:41:41.356645+0000","last_undegraded":"2024-04-25T13:41:41.356645+0000","last_fullsized":"2024-04-25T13:41:41.356645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.749459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356811+0000","last_change":"2024-04-25T13:40:41.284948+0000","last_active":"2024-04-25T13:41:41.356811+0000","last_peered":"2024-04-25T13:41:41.356811+0000","last_clean":"2024-04-25T13:41:41.356811+0000","last_became_active":"2024-04-25T13:40:41.272015+0000","last_became_peered":"2024-04-25T13:40:41.272015+0000","last_unstale":"2024-04-25T13:41:41.356811+0000","last_undegraded":"2024-04-25T13:41:41.356811+0000","last_fullsized":"2024-04-25T13:41:41.356811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:21.620303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445232+0000","last_change":"2024-04-25T13:39:45.262732+0000","last_active":"2024-04-25T13:41:41.445232+0000","last_peered":"2024-04-25T13:41:41.445232+0000","last_clean":"2024-04-25T13:41:41.445232+0000","last_became_active":"2024-04-25T13:39:45.261422+0000","last_became_peered":"2024-04-25T13:39:45.261422+0000","last_unstale":"2024-04-25T13:41:41.445232+0000","last_undegraded":"2024-04-25T13:41:41.445232+0000","last_fullsized":"2024-04-25T13:41:41.445232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:13.645107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.446492+0000","last_change":"2024-04-25T13:40:41.282512+0000","last_active":"2024-04-25T13:41:41.446492+0000","last_peered":"2024-04-25T13:41:41.446492+0000","last_clean":"2024-04-25T13:41:41.446492+0000","last_became_active":"2024-04-25T13:40:41.275727+0000","last_became_peered":"2024-04-25T13:40:41.275727+0000","last_unstale":"2024-04-25T13:41:41.446492+0000","last_undegraded":"2024-04-25T13:41:41.446492+0000","last_fullsized":"2024-04-25T13:41:41.446492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.611067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357910+0000","last_change":"2024-04-25T13:39:45.264181+0000","last_active":"2024-04-25T13:41:41.357910+0000","last_peered":"2024-04-25T13:41:41.357910+0000","last_clean":"2024-04-25T13:41:41.357910+0000","last_became_active":"2024-04-25T13:39:45.263776+0000","last_became_peered":"2024-04-25T13:39:45.263776+0000","last_unstale":"2024-04-25T13:41:41.357910+0000","last_undegraded":"2024-04-25T13:41:41.357910+0000","last_fullsized":"2024-04-25T13:41:41.357910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:04.718121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:42.812487+0000","last_change":"2024-04-25T13:39:44.627963+0000","last_active":"2024-04-25T13:41:42.812487+0000","last_peered":"2024-04-25T13:41:42.812487+0000","last_clean":"2024-04-25T13:41:42.812487+0000","last_became_active":"2024-04-25T13:39:44.627581+0000","last_became_peered":"2024-04-25T13:39:44.627581+0000","last_unstale":"2024-04-25T13:41:42.812487+0000","last_undegraded":"2024-04-25T13:41:42.812487+0000","last_fullsized":"2024-04-25T13:41:42.812487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:48.191764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358485+0000","last_change":"2024-04-25T13:39:45.260815+0000","last_active":"2024-04-25T13:41:41.358485+0000","last_peered":"2024-04-25T13:41:41.358485+0000","last_clean":"2024-04-25T13:41:41.358485+0000","last_became_active":"2024-04-25T13:39:45.260399+0000","last_became_peered":"2024-04-25T13:39:45.260399+0000","last_unstale":"2024-04-25T13:41:41.358485+0000","last_undegraded":"2024-04-25T13:41:41.358485+0000","last_fullsized":"2024-04-25T13:41:41.358485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:39.049325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361741+0000","last_change":"2024-04-25T13:39:44.628976+0000","last_active":"2024-04-25T13:41:41.361741+0000","last_peered":"2024-04-25T13:41:41.361741+0000","last_clean":"2024-04-25T13:41:41.361741+0000","last_became_active":"2024-04-25T13:39:44.628539+0000","last_became_peered":"2024-04-25T13:39:44.628539+0000","last_unstale":"2024-04-25T13:41:41.361741+0000","last_undegraded":"2024-04-25T13:41:41.361741+0000","last_fullsized":"2024-04-25T13:41:41.361741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358380+0000","last_change":"2024-04-25T13:39:45.261909+0000","last_active":"2024-04-25T13:41:41.358380+0000","last_peered":"2024-04-25T13:41:41.358380+0000","last_clean":"2024-04-25T13:41:41.358380+0000","last_became_active":"2024-04-25T13:39:45.261532+0000","last_became_peered":"2024-04-25T13:39:45.261532+0000","last_unstale":"2024-04-25T13:41:41.358380+0000","last_undegraded":"2024-04-25T13:41:41.358380+0000","last_fullsized":"2024-04-25T13:41:41.358380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:01.939085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359596+0000","last_change":"2024-04-25T13:39:44.630940+0000","last_active":"2024-04-25T13:41:41.359596+0000","last_peered":"2024-04-25T13:41:41.359596+0000","last_clean":"2024-04-25T13:41:41.359596+0000","last_became_active":"2024-04-25T13:39:44.630527+0000","last_became_peered":"2024-04-25T13:39:44.630527+0000","last_unstale":"2024-04-25T13:41:41.359596+0000","last_undegraded":"2024-04-25T13:41:41.359596+0000","last_fullsized":"2024-04-25T13:41:41.359596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:43.598512+0000","last_clean_scrub_stamp":"2024-04-25T13:39:43.598512+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:49.456481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358692+0000","last_change":"2024-04-25T13:39:45.265642+0000","last_active":"2024-04-25T13:41:41.358692+0000","last_peered":"2024-04-25T13:41:41.358692+0000","last_clean":"2024-04-25T13:41:41.358692+0000","last_became_active":"2024-04-25T13:39:45.265273+0000","last_became_peered":"2024-04-25T13:39:45.265273+0000","last_unstale":"2024-04-25T13:41:41.358692+0000","last_undegraded":"2024-04-25T13:41:41.358692+0000","last_fullsized":"2024-04-25T13:41:41.358692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.928599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357006+0000","last_change":"2024-04-25T13:39:45.272925+0000","last_active":"2024-04-25T13:41:41.357006+0000","last_peered":"2024-04-25T13:41:41.357006+0000","last_clean":"2024-04-25T13:41:41.357006+0000","last_became_active":"2024-04-25T13:39:45.272601+0000","last_became_peered":"2024-04-25T13:39:45.272601+0000","last_unstale":"2024-04-25T13:41:41.357006+0000","last_undegraded":"2024-04-25T13:41:41.357006+0000","last_fullsized":"2024-04-25T13:41:41.357006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:45.728687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358989+0000","last_change":"2024-04-25T13:39:45.268477+0000","last_active":"2024-04-25T13:41:41.358989+0000","last_peered":"2024-04-25T13:41:41.358989+0000","last_clean":"2024-04-25T13:41:41.358989+0000","last_became_active":"2024-04-25T13:39:45.266805+0000","last_became_peered":"2024-04-25T13:39:45.266805+0000","last_unstale":"2024-04-25T13:41:41.358989+0000","last_undegraded":"2024-04-25T13:41:41.358989+0000","last_fullsized":"2024-04-25T13:41:41.358989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:42.380952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358931+0000","last_change":"2024-04-25T13:39:45.263988+0000","last_active":"2024-04-25T13:41:41.358931+0000","last_peered":"2024-04-25T13:41:41.358931+0000","last_clean":"2024-04-25T13:41:41.358931+0000","last_became_active":"2024-04-25T13:39:45.263643+0000","last_became_peered":"2024-04-25T13:39:45.263643+0000","last_unstale":"2024-04-25T13:41:41.358931+0000","last_undegraded":"2024-04-25T13:41:41.358931+0000","last_fullsized":"2024-04-25T13:41:41.358931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:17.231288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361885+0000","last_change":"2024-04-25T13:40:41.274874+0000","last_active":"2024-04-25T13:41:41.361885+0000","last_peered":"2024-04-25T13:41:41.361885+0000","last_clean":"2024-04-25T13:41:41.361885+0000","last_became_active":"2024-04-25T13:40:41.274489+0000","last_became_peered":"2024-04-25T13:40:41.274489+0000","last_unstale":"2024-04-25T13:41:41.361885+0000","last_undegraded":"2024-04-25T13:41:41.361885+0000","last_fullsized":"2024-04-25T13:41:41.361885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:34.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.364287+0000","last_change":"2024-04-25T13:41:41.364287+0000","last_active":"2024-04-25T13:41:41.364287+0000","last_peered":"2024-04-25T13:41:41.364287+0000","last_clean":"2024-04-25T13:41:41.364287+0000","last_became_active":"2024-04-25T13:41:41.363853+0000","last_became_peered":"2024-04-25T13:41:41.363853+0000","last_unstale":"2024-04-25T13:41:41.364287+0000","last_undegraded":"2024-04-25T13:41:41.364287+0000","last_fullsized":"2024-04-25T13:41:41.364287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:11.271445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.357161+0000","last_change":"2024-04-25T13:39:45.264473+0000","last_active":"2024-04-25T13:41:41.357161+0000","last_peered":"2024-04-25T13:41:41.357161+0000","last_clean":"2024-04-25T13:41:41.357161+0000","last_became_active":"2024-04-25T13:39:45.264124+0000","last_became_peered":"2024-04-25T13:39:45.264124+0000","last_unstale":"2024-04-25T13:41:41.357161+0000","last_undegraded":"2024-04-25T13:41:41.357161+0000","last_fullsized":"2024-04-25T13:41:41.357161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:51.332413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359493+0000","last_change":"2024-04-25T13:39:45.266670+0000","last_active":"2024-04-25T13:41:41.359493+0000","last_peered":"2024-04-25T13:41:41.359493+0000","last_clean":"2024-04-25T13:41:41.359493+0000","last_became_active":"2024-04-25T13:39:45.266084+0000","last_became_peered":"2024-04-25T13:39:45.266084+0000","last_unstale":"2024-04-25T13:41:41.359493+0000","last_undegraded":"2024-04-25T13:41:41.359493+0000","last_fullsized":"2024-04-25T13:41:41.359493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:44.743960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445348+0000","last_change":"2024-04-25T13:39:45.264669+0000","last_active":"2024-04-25T13:41:41.445348+0000","last_peered":"2024-04-25T13:41:41.445348+0000","last_clean":"2024-04-25T13:41:41.445348+0000","last_became_active":"2024-04-25T13:39:45.264128+0000","last_became_peered":"2024-04-25T13:39:45.264128+0000","last_unstale":"2024-04-25T13:41:41.445348+0000","last_undegraded":"2024-04-25T13:41:41.445348+0000","last_fullsized":"2024-04-25T13:41:41.445348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:24.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.361057+0000","last_change":"2024-04-25T13:39:45.267296+0000","last_active":"2024-04-25T13:41:41.361057+0000","last_peered":"2024-04-25T13:41:41.361057+0000","last_clean":"2024-04-25T13:41:41.361057+0000","last_became_active":"2024-04-25T13:39:45.266946+0000","last_became_peered":"2024-04-25T13:39:45.266946+0000","last_unstale":"2024-04-25T13:41:41.361057+0000","last_undegraded":"2024-04-25T13:41:41.361057+0000","last_fullsized":"2024-04-25T13:41:41.361057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:56.266544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.445102+0000","last_change":"2024-04-25T13:39:45.261054+0000","last_active":"2024-04-25T13:41:41.445102+0000","last_peered":"2024-04-25T13:41:41.445102+0000","last_clean":"2024-04-25T13:41:41.445102+0000","last_became_active":"2024-04-25T13:39:45.260739+0000","last_became_peered":"2024-04-25T13:39:45.260739+0000","last_unstale":"2024-04-25T13:41:41.445102+0000","last_undegraded":"2024-04-25T13:41:41.445102+0000","last_fullsized":"2024-04-25T13:41:41.445102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:57.967432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.358923+0000","last_change":"2024-04-25T13:40:41.276377+0000","last_active":"2024-04-25T13:41:41.358923+0000","last_peered":"2024-04-25T13:41:41.358923+0000","last_clean":"2024-04-25T13:41:41.358923+0000","last_became_active":"2024-04-25T13:40:41.275949+0000","last_became_peered":"2024-04-25T13:40:41.275949+0000","last_unstale":"2024-04-25T13:41:41.358923+0000","last_undegraded":"2024-04-25T13:41:41.358923+0000","last_fullsized":"2024-04-25T13:41:41.358923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:49.882272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.359800+0000","last_change":"2024-04-25T13:39:45.260528+0000","last_active":"2024-04-25T13:41:41.359800+0000","last_peered":"2024-04-25T13:41:41.359800+0000","last_clean":"2024-04-25T13:41:41.359800+0000","last_became_active":"2024-04-25T13:39:45.258993+0000","last_became_peered":"2024-04-25T13:39:45.258993+0000","last_unstale":"2024-04-25T13:41:41.359800+0000","last_undegraded":"2024-04-25T13:41:41.359800+0000","last_fullsized":"2024-04-25T13:41:41.359800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:21.547519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T13:41:41.356863+0000","last_change":"2024-04-25T13:39:45.274426+0000","last_active":"2024-04-25T13:41:41.356863+0000","last_peered":"2024-04-25T13:41:41.356863+0000","last_clean":"2024-04-25T13:41:41.356863+0000","last_became_active":"2024-04-25T13:39:45.274065+0000","last_became_peered":"2024-04-25T13:39:45.274065+0000","last_unstale":"2024-04-25T13:41:41.356863+0000","last_undegraded":"2024-04-25T13:41:41.356863+0000","last_fullsized":"2024-04-25T13:41:41.356863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:39:44.600487+0000","last_clean_scrub_stamp":"2024-04-25T13:39:44.600487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71795,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1173,"num_read_kb":667,"num_write":255,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":440213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1626112,"data_stored":1095580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972400,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40420,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704732,"statfs":{"total":95995035648,"available":95953645568,"internally_reserved":0,"allocated":946176,"data_stored":440031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":440297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1033767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":440139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":505194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40432,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704720,"statfs":{"total":95995035648,"available":95953633280,"internally_reserved":0,"allocated":950272,"data_stored":441087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":505555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":946176,"data_stored":439573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1552384,"data_stored":1032746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:41:46.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2736140735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:46.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:46 smithi138 ceph-mon[32977]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:46.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:46 smithi138 ceph-mon[32977]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.852+0000 7f2f38d38640 1 -- 172.21.15.94:0/3935067922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 msgr2=0x7f2f34104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.852+0000 7f2f38d38640 1 --2- 172.21.15.94:0/3935067922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f34104050 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009900 tx=0x7f2f2802a410 comp rx=0 tx=0).stop 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 -- 172.21.15.94:0/3935067922 shutdown_connections 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 --2- 172.21.15.94:0/3935067922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34108940 0x7f2f3410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 --2- 172.21.15.94:0/3935067922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f34104050 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 --2- 172.21.15.94:0/3935067922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34102a00 0x7f2f34103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 -- 172.21.15.94:0/3935067922 >> 172.21.15.94:0/3935067922 conn(0x7f2f340fdbf0 msgr2=0x7f2f341000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 -- 172.21.15.94:0/3935067922 shutdown_connections 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 -- 172.21.15.94:0/3935067922 wait complete. 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 Processor -- start 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.853+0000 7f2f38d38640 1 -- start start 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34102a00 0x7f2f3419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34108940 0x7f2f341a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410f630 con 0x7f2f34103c40 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410f4b0 con 0x7f2f34102a00 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3410f7b0 con 0x7f2f34108940 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54178/0 (socket says 172.21.15.94:54178) 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 -- 172.21.15.94:0/37167159 learned_addr learned my addr 172.21.15.94:0/37167159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f3259c640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34102a00 0x7f2f3419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f32d9d640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34108940 0x7f2f341a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 -- 172.21.15.94:0/37167159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34108940 msgr2=0x7f2f341a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34108940 0x7f2f341a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 -- 172.21.15.94:0/37167159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34102a00 msgr2=0x7f2f3419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34102a00 0x7f2f3419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 -- 172.21.15.94:0/37167159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f341a1cc0 con 0x7f2f34103c40 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f31d9b640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2f28006010 tx=0x7f2f28004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2803f030 con 0x7f2f34103c40 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f28003cb0 con 0x7f2f34103c40 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f341a1f50 con 0x7f2f34103c40 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28035070 con 0x7f2f34103c40 2024-04-25T13:41:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.854+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a2360 con 0x7f2f34103c40 2024-04-25T13:41:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.855+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f34107750 con 0x7f2f34103c40 2024-04-25T13:41:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.857+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f2f28042030 con 0x7f2f34103c40 2024-04-25T13:41:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.858+0000 7f2f237fe640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 0x7f2f1c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:46.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.858+0000 7f2f3259c640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 0x7f2f1c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:46.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.858+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2f280ffc00 con 0x7f2f34103c40 2024-04-25T13:41:46.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.858+0000 7f2f3259c640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 0x7f2f1c0b2090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2f24005fd0 tx=0x7f2f24005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:46.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:46.862+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f280c85e0 con 0x7f2f34103c40 2024-04-25T13:41:47.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.042+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f3402dd20 con 0x7f2f34103c40 2024-04-25T13:41:47.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.044+0000 7f2f237fe640 1 -- 172.21.15.94:0/37167159 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22958 (secure 0 0 0) 0x7f2f28003e60 con 0x7f2f34103c40 2024-04-25T13:41:47.044 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:47.044 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":83,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:41:41.352560+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 msgr2=0x7f2f1c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 0x7f2f1c0b2090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2f24005fd0 tx=0x7f2f24005850 comp rx=0 tx=0).stop 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 msgr2=0x7f2f3419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2f28006010 tx=0x7f2f28004340 comp rx=0 tx=0).stop 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 shutdown_connections 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2f1c0afb80 0x7f2f1c0b2090 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f34108940 0x7f2f341a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f34103c40 0x7f2f3419a770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 --2- 172.21.15.94:0/37167159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f34102a00 0x7f2f3419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 >> 172.21.15.94:0/37167159 conn(0x7f2f340fdbf0 msgr2=0x7f2f34104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 shutdown_connections 2024-04-25T13:41:47.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.046+0000 7f2f38d38640 1 -- 172.21.15.94:0/37167159 wait complete. 2024-04-25T13:41:47.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:47.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:47.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 -- 172.21.15.94:0/1655594159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578108940 msgr2=0x7f257810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 --2- 172.21.15.94:0/1655594159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578108940 0x7f257810ad60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2570009ca0 tx=0x7f257002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 -- 172.21.15.94:0/1655594159 shutdown_connections 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 --2- 172.21.15.94:0/1655594159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578108940 0x7f257810ad60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 --2- 172.21.15.94:0/1655594159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578103c40 0x7f2578104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 --2- 172.21.15.94:0/1655594159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2578102a00 0x7f2578103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 -- 172.21.15.94:0/1655594159 >> 172.21.15.94:0/1655594159 conn(0x7f25780fdbf0 msgr2=0x7f25781000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 -- 172.21.15.94:0/1655594159 shutdown_connections 2024-04-25T13:41:47.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.169+0000 7f257c9ba640 1 -- 172.21.15.94:0/1655594159 wait complete. 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 Processor -- start 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 -- start start 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2578102a00 0x7f257819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578108940 0x7f25781a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f257810e530 con 0x7f2578103c40 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f257810e3b0 con 0x7f2578102a00 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f257c9ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f257810e6b0 con 0x7f2578108940 2024-04-25T13:41:47.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54194/0 (socket says 172.21.15.94:54194) 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 -- 172.21.15.94:0/3486979859 learned_addr learned my addr 172.21.15.94:0/3486979859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f2576d9d640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578108940 0x7f25781a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 -- 172.21.15.94:0/3486979859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578108940 msgr2=0x7f25781a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578108940 0x7f25781a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 -- 172.21.15.94:0/3486979859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2578102a00 msgr2=0x7f257819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2578102a00 0x7f257819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.170+0000 7f256ffff640 1 -- 172.21.15.94:0/3486979859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25781a1d50 con 0x7f2578103c40 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f256ffff640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f256000d770 tx=0x7f256000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560017070 con 0x7f2578103c40 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25600054d0 con 0x7f2578103c40 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560004c30 con 0x7f2578103c40 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781a1fe0 con 0x7f2578103c40 2024-04-25T13:41:47.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.171+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781a2480 con 0x7f2578103c40 2024-04-25T13:41:47.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.172+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25781a26e0 con 0x7f2578103c40 2024-04-25T13:41:47.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.173+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f2560004dd0 con 0x7f2578103c40 2024-04-25T13:41:47.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.174+0000 7f256f7fe640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 0x7f25640b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.174+0000 7f257659c640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 0x7f25640b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.174+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f25600dee10 con 0x7f2578103c40 2024-04-25T13:41:47.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.175+0000 7f257659c640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 0x7f25640b2090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2578103600 tx=0x7f2568005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.178+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25600a77f0 con 0x7f2578103c40 2024-04-25T13:41:47.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.356+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25781a28b0 con 0x7f2578103c40 2024-04-25T13:41:47.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.356+0000 7f256f7fe640 1 -- 172.21.15.94:0/3486979859 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f25600a70f0 con 0x7f2578103c40 2024-04-25T13:41:47.357 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T13:41:47.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 msgr2=0x7f25640b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 0x7f25640b2090 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2578103600 tx=0x7f2568005c10 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 msgr2=0x7f257819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f256000d770 tx=0x7f256000dc30 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 shutdown_connections 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25640afb80 0x7f25640b2090 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2578108940 0x7f25781a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2578103c40 0x7f257819a750 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 --2- 172.21.15.94:0/3486979859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2578102a00 0x7f257819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 >> 172.21.15.94:0/3486979859 conn(0x7f25780fdbf0 msgr2=0x7f2578109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 shutdown_connections 2024-04-25T13:41:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.358+0000 7f257c9ba640 1 -- 172.21.15.94:0/3486979859 wait complete. 2024-04-25T13:41:47.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 -- 172.21.15.94:0/1971810340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f0101760 msgr2=0x7fc0f0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/1971810340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f0101760 0x7fc0f0101bd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 -- 172.21.15.94:0/1971810340 shutdown_connections 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/1971810340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 0x7fc0f010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/1971810340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f0101760 0x7fc0f0101bd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/1971810340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f01050b0 0x7fc0f01054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 -- 172.21.15.94:0/1971810340 >> 172.21.15.94:0/1971810340 conn(0x7fc0f00fd380 msgr2=0x7fc0f00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 -- 172.21.15.94:0/1971810340 shutdown_connections 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.480+0000 7fc0f7d34640 1 -- 172.21.15.94:0/1971810340 wait complete. 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 Processor -- start 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 -- start start 2024-04-25T13:41:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 0x7fc0f019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 0x7fc0f019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010f640 con 0x7fc0f01050b0 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010f4c0 con 0x7fc0f0101760 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f7d34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010f7c0 con 0x7fc0f0102260 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f62d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f62d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54198/0 (socket says 172.21.15.94:54198) 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.481+0000 7fc0f62d1640 1 -- 172.21.15.94:0/3762364233 learned_addr learned my addr 172.21.15.94:0/3762364233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f5ad0640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 0x7fc0f019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f52cf640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 0x7fc0f019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 -- 172.21.15.94:0/3762364233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 msgr2=0x7fc0f019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 0x7fc0f019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 -- 172.21.15.94:0/3762364233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 msgr2=0x7fc0f019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 0x7fc0f019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 -- 172.21.15.94:0/3762364233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f01a1d80 con 0x7fc0f01050b0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f5ad0640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 0x7fc0f019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f62d1640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800eaa0 tx=0x7fc0e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800cdb0 con 0x7fc0f01050b0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e8005020 con 0x7fc0f01050b0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80116d0 con 0x7fc0f01050b0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f01a2070 con 0x7fc0f01050b0 2024-04-25T13:41:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.482+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f01a25c0 con 0x7fc0f01050b0 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.483+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f0108a40 con 0x7fc0f01050b0 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.485+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fc0e80051d0 con 0x7fc0f01050b0 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.485+0000 7fc0e6ffd640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 0x7fc0dc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.485+0000 7fc0f5ad0640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 0x7fc0dc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.485+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc0e80df470 con 0x7fc0f01050b0 2024-04-25T13:41:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.486+0000 7fc0f5ad0640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 0x7fc0dc0b2090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0f006b2d0 tx=0x7fc0ec005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.489+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0e8016030 con 0x7fc0f01050b0 2024-04-25T13:41:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:47 smithi137 ceph-mon[32568]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:47 smithi137 ceph-mon[32568]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/37167159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:47 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3486979859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.666+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc0f002dd00 con 0x7fc0f01050b0 2024-04-25T13:41:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.666+0000 7fc0e6ffd640 1 -- 172.21.15.94:0/3762364233 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fc0e80a7e50 con 0x7fc0f01050b0 2024-04-25T13:41:47.667 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 msgr2=0x7fc0dc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 0x7fc0dc0b2090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0f006b2d0 tx=0x7fc0ec005e70 comp rx=0 tx=0).stop 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 msgr2=0x7fc0f01a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800eaa0 tx=0x7fc0e800ef60 comp rx=0 tx=0).stop 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 shutdown_connections 2024-04-25T13:41:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0dc0afb80 0x7fc0dc0b2090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0f01050b0 0x7fc0f01a1710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0f0102260 0x7fc0f019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 --2- 172.21.15.94:0/3762364233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f0101760 0x7fc0f019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 >> 172.21.15.94:0/3762364233 conn(0x7fc0f00fd380 msgr2=0x7fc0f0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 shutdown_connections 2024-04-25T13:41:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.669+0000 7fc0f7d34640 1 -- 172.21.15.94:0/3762364233 wait complete. 2024-04-25T13:41:47.679 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:47 smithi094 ceph-mon[29195]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.679 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:47 smithi094 ceph-mon[29195]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.679 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/37167159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:47.679 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:47 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3486979859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:47.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:47 smithi138 ceph-mon[32977]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:47 smithi138 ceph-mon[32977]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:41:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/37167159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:47 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3486979859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 -- 172.21.15.94:0/152804762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834108930 msgr2=0x7f583410ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 --2- 172.21.15.94:0/152804762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834108930 0x7f583410ad50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f582c009ca0 tx=0x7f582c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 -- 172.21.15.94:0/152804762 shutdown_connections 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 --2- 172.21.15.94:0/152804762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834108930 0x7f583410ad50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 --2- 172.21.15.94:0/152804762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5834103c30 0x7f5834104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 --2- 172.21.15.94:0/152804762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58341029f0 0x7f58341036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 -- 172.21.15.94:0/152804762 >> 172.21.15.94:0/152804762 conn(0x7f58340fdbc0 msgr2=0x7f5834100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 -- 172.21.15.94:0/152804762 shutdown_connections 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.791+0000 7f58395c5640 1 -- 172.21.15.94:0/152804762 wait complete. 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 Processor -- start 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 -- start start 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58341029f0 0x7f583419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5834108930 0x7f58341a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f583410f6a0 con 0x7f5834103c30 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f583410f520 con 0x7f58341029f0 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.792+0000 7f58395c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f583410f820 con 0x7f5834108930 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54204/0 (socket says 172.21.15.94:54204) 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 -- 172.21.15.94:0/3438284919 learned_addr learned my addr 172.21.15.94:0/3438284919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f5832ffd640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58341029f0 0x7f583419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f58337fe640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5834108930 0x7f58341a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 -- 172.21.15.94:0/3438284919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5834108930 msgr2=0x7f58341a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5834108930 0x7f58341a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 -- 172.21.15.94:0/3438284919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58341029f0 msgr2=0x7f583419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58341029f0 0x7f583419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 -- 172.21.15.94:0/3438284919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341a1db0 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f582a7fc640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b4a0 tx=0x7f581c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c014070 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581c010ac0 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341a20a0 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c017c30 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.793+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a2670 con 0x7f5834103c30 2024-04-25T13:41:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.794+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834107740 con 0x7f5834103c30 2024-04-25T13:41:47.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.796+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f581c017070 con 0x7f5834103c30 2024-04-25T13:41:47.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.796+0000 7f5830ff9640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 0x7f58200b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.797+0000 7f5832ffd640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 0x7f58200b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.797+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f581c0de470 con 0x7f5834103c30 2024-04-25T13:41:47.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.797+0000 7f5832ffd640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 0x7f58200b2090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58240058c0 tx=0x7f582400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.801+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f581c0a6f40 con 0x7f5834103c30 2024-04-25T13:41:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.977+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f58340637a0 con 0x7f5834103c30 2024-04-25T13:41:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.977+0000 7f5830ff9640 1 -- 172.21.15.94:0/3438284919 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f581c0a6840 con 0x7f5834103c30 2024-04-25T13:41:47.977 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 msgr2=0x7f58200b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 0x7f58200b2090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58240058c0 tx=0x7f582400a2d0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 msgr2=0x7f583419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b4a0 tx=0x7f581c00b960 comp rx=0 tx=0).stop 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 shutdown_connections 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58200afb80 0x7f58200b2090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5834108930 0x7f58341a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5834103c30 0x7f583419a7b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 --2- 172.21.15.94:0/3438284919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58341029f0 0x7f583419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 >> 172.21.15.94:0/3438284919 conn(0x7f58340fdbc0 msgr2=0x7f5834104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.979+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 shutdown_connections 2024-04-25T13:41:47.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:47.980+0000 7f58395c5640 1 -- 172.21.15.94:0/3438284919 wait complete. 2024-04-25T13:41:47.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:48.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- 172.21.15.94:0/3903299577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 msgr2=0x7fd0901036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 --2- 172.21.15.94:0/3903299577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd0901036c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd080009900 tx=0x7fd08002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:48.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- 172.21.15.94:0/3903299577 shutdown_connections 2024-04-25T13:41:48.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 --2- 172.21.15.94:0/3903299577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd090108900 0x7fd09010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 --2- 172.21.15.94:0/3903299577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 --2- 172.21.15.94:0/3903299577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd0901036c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- 172.21.15.94:0/3903299577 >> 172.21.15.94:0/3903299577 conn(0x7fd0900fdbd0 msgr2=0x7fd090100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- 172.21.15.94:0/3903299577 shutdown_connections 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- 172.21.15.94:0/3903299577 wait complete. 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 Processor -- start 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.099+0000 7fd096338640 1 -- start start 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd090108900 0x7fd0901a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd09010e4c0 con 0x7fd0901029c0 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd09010e340 con 0x7fd090103c00 2024-04-25T13:41:48.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd096338640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd09010e640 con 0x7fd090108900 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54218/0 (socket says 172.21.15.94:54218) 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 -- 172.21.15.94:0/2654302560 learned_addr learned my addr 172.21.15.94:0/2654302560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08f7fe640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 -- 172.21.15.94:0/2654302560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd090108900 msgr2=0x7fd0901a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd090108900 0x7fd0901a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 -- 172.21.15.94:0/2654302560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 msgr2=0x7fd090199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 -- 172.21.15.94:0/2654302560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901a0bb0 con 0x7fd0901029c0 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08f7fe640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08ffff640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd080006010 tx=0x7fd080031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.100+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08003f030 con 0x7fd0901029c0 2024-04-25T13:41:48.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.101+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0901a0e40 con 0x7fd0901029c0 2024-04-25T13:41:48.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.101+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd080003aa0 con 0x7fd0901029c0 2024-04-25T13:41:48.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.101+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080035070 con 0x7fd0901029c0 2024-04-25T13:41:48.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.101+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a12a0 con 0x7fd0901029c0 2024-04-25T13:41:48.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.101+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd090107710 con 0x7fd0901029c0 2024-04-25T13:41:48.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.103+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fd080042030 con 0x7fd0901029c0 2024-04-25T13:41:48.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.104+0000 7fd08d7fa640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 0x7fd0780b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.104+0000 7fd08f7fe640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 0x7fd0780b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.104+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd0800ff8a0 con 0x7fd0901029c0 2024-04-25T13:41:48.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.104+0000 7fd08f7fe640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 0x7fd0780b2090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd09019a1c0 tx=0x7fd084005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.108+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0800c8280 con 0x7fd0901029c0 2024-04-25T13:41:48.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.322+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd09002dc70 con 0x7fd0901029c0 2024-04-25T13:41:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.323+0000 7fd08d7fa640 1 -- 172.21.15.94:0/2654302560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fd0800c7b80 con 0x7fd0901029c0 2024-04-25T13:41:48.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:48.323 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:41:48.323 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 msgr2=0x7fd0780b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 0x7fd0780b2090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd09019a1c0 tx=0x7fd084005db0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 msgr2=0x7fd090199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd080006010 tx=0x7fd080031a50 comp rx=0 tx=0).stop 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 shutdown_connections 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0780afb80 0x7fd0780b2090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd090108900 0x7fd0901a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd090103c00 0x7fd090199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 --2- 172.21.15.94:0/2654302560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0901029c0 0x7fd090199120 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 >> 172.21.15.94:0/2654302560 conn(0x7fd0900fdbd0 msgr2=0x7fd0901048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 shutdown_connections 2024-04-25T13:41:48.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.325+0000 7fd096338640 1 -- 172.21.15.94:0/2654302560 wait complete. 2024-04-25T13:41:48.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:48.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- 172.21.15.94:0/145322567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c103990 msgr2=0x7efe5c103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 --2- 172.21.15.94:0/145322567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c103990 0x7efe5c103e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efe50009900 tx=0x7efe5002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- 172.21.15.94:0/145322567 shutdown_connections 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 --2- 172.21.15.94:0/145322567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c104490 0x7efe5c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 --2- 172.21.15.94:0/145322567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c103990 0x7efe5c103e00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 --2- 172.21.15.94:0/145322567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c102fb0 0x7efe5c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- 172.21.15.94:0/145322567 >> 172.21.15.94:0/145322567 conn(0x7efe5c0fd390 msgr2=0x7efe5c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- 172.21.15.94:0/145322567 shutdown_connections 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- 172.21.15.94:0/145322567 wait complete. 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 Processor -- start 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.446+0000 7efe60e92640 1 -- start start 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c103990 0x7efe5c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c104490 0x7efe5c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c10f650 con 0x7efe5c102fb0 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c10f4d0 con 0x7efe5c103990 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c10f7d0 con 0x7efe5c104490 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54240/0 (socket says 172.21.15.94:54240) 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 -- 172.21.15.94:0/557908538 learned_addr learned my addr 172.21.15.94:0/557908538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5ad9d640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c104490 0x7efe5c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe59d9b640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c103990 0x7efe5c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 -- 172.21.15.94:0/557908538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c104490 msgr2=0x7efe5c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c104490 0x7efe5c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 -- 172.21.15.94:0/557908538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c103990 msgr2=0x7efe5c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c103990 0x7efe5c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 -- 172.21.15.94:0/557908538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c1a1d70 con 0x7efe5c102fb0 2024-04-25T13:41:48.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe5a59c640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005d00 tx=0x7efe4c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c017070 con 0x7efe5c102fb0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4c009c30 con 0x7efe5c102fb0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c1a2060 con 0x7efe5c102fb0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00f620 con 0x7efe5c102fb0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.447+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a2560 con 0x7efe5c102fb0 2024-04-25T13:41:48.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.448+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c107950 con 0x7efe5c102fb0 2024-04-25T13:41:48.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.450+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7efe4c01a070 con 0x7efe5c102fb0 2024-04-25T13:41:48.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.450+0000 7efe4b7fe640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 0x7efe440b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.450+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7efe4c0de940 con 0x7efe5c102fb0 2024-04-25T13:41:48.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.450+0000 7efe59d9b640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 0x7efe440b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.451+0000 7efe59d9b640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 0x7efe440b2090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe5002a930 tx=0x7efe50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.454+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe4c009de0 con 0x7efe5c102fb0 2024-04-25T13:41:48.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.634+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe5c0637a0 con 0x7efe5c102fb0 2024-04-25T13:41:48.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.636+0000 7efe4b7fe640 1 -- 172.21.15.94:0/557908538 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22958 (secure 0 0 0) 0x7efe4c0a7320 con 0x7efe5c102fb0 2024-04-25T13:41:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:41:48.636 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":83,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:41:41.352560+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 msgr2=0x7efe440b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 0x7efe440b2090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe5002a930 tx=0x7efe50000bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 msgr2=0x7efe5c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005d00 tx=0x7efe4c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 shutdown_connections 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efe440afb80 0x7efe440b2090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5c104490 0x7efe5c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe5c103990 0x7efe5c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 --2- 172.21.15.94:0/557908538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe5c102fb0 0x7efe5c19a230 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 >> 172.21.15.94:0/557908538 conn(0x7efe5c0fd390 msgr2=0x7efe5c104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 shutdown_connections 2024-04-25T13:41:48.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.638+0000 7efe60e92640 1 -- 172.21.15.94:0/557908538 wait complete. 2024-04-25T13:41:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:48 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3762364233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:48 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3438284919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:48 smithi137 ceph-mon[32568]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T13:41:48.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:48 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2654302560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:48.648 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:48 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3762364233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.648 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:48 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3438284919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.648 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:48 smithi094 ceph-mon[29195]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T13:41:48.648 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:48 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2654302560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:48.650 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T13:41:48.650 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:41:48.661 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T13:41:48.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:41:48.661 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T13:41:48.661 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:41:48.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:48.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:41:48.664 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T13:41:48.686 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:41:48.687 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T13:41:48.687 INFO:tasks.workunit:timeout=3h 2024-04-25T13:41:48.687 INFO:tasks.workunit:cleanup=True 2024-04-25T13:41:48.687 DEBUG:teuthology.orchestra.run.smithi094:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T13:41:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 13:39:51.112120722 +0000 2024-04-25T13:41:48.703 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 13:39:51.112120722 +0000 2024-04-25T13:41:48.703 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 13:41:40.909966086 +0000 2024-04-25T13:41:48.703 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 13:39:51.112120722 +0000 2024-04-25T13:41:48.704 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:48.704 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T13:41:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3762364233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3438284919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:41:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:48 smithi138 ceph-mon[32977]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T13:41:48.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2654302560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:41:48.777 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:41:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 -- 172.21.15.94:0/423198188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101e30 msgr2=0x7f816c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 --2- 172.21.15.94:0/423198188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101e30 0x7f816c102240 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8160009900 tx=0x7f816002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 -- 172.21.15.94:0/423198188 shutdown_connections 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 --2- 172.21.15.94:0/423198188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c109780 0x7f816c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 --2- 172.21.15.94:0/423198188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c106c10 0x7f816c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 --2- 172.21.15.94:0/423198188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c101e30 0x7f816c102240 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 -- 172.21.15.94:0/423198188 >> 172.21.15.94:0/423198188 conn(0x7f816c0fd340 msgr2=0x7f816c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 -- 172.21.15.94:0/423198188 shutdown_connections 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.776+0000 7f8170d7d640 1 -- 172.21.15.94:0/423198188 wait complete. 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 Processor -- start 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 -- start start 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c101e30 0x7f816c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c106c10 0x7f816c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f816c10f5e0 con 0x7f816c109780 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f816c10f460 con 0x7f816c106c10 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8170d7d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f816c10f760 con 0x7f816c101e30 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54258/0 (socket says 172.21.15.94:54258) 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 -- 172.21.15.94:0/3570110634 learned_addr learned my addr 172.21.15.94:0/3570110634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816a59c640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c101e30 0x7f816c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f8169d9b640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c106c10 0x7f816c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 -- 172.21.15.94:0/3570110634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c101e30 msgr2=0x7f816c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c101e30 0x7f816c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 -- 172.21.15.94:0/3570110634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c106c10 msgr2=0x7f816c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c106c10 0x7f816c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.777+0000 7f816ad9d640 1 -- 172.21.15.94:0/3570110634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1a1d80 con 0x7f816c109780 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f816ad9d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8164009fd0 tx=0x7f816400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f815b7fe640 1 -- 172.21.15.94:0/3570110634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164017070 con 0x7f816c109780 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f815b7fe640 1 -- 172.21.15.94:0/3570110634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8164003e20 con 0x7f816c109780 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1a2070 con 0x7f816c109780 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f815b7fe640 1 -- 172.21.15.94:0/3570110634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010460 con 0x7f816c109780 2024-04-25T13:41:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.778+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c1a2570 con 0x7f816c109780 2024-04-25T13:41:48.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.780+0000 7f8177c2c740 1 Processor -- start 2024-04-25T13:41:48.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f815b7fe640 1 -- 172.21.15.94:0/3570110634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f8164003970 con 0x7f816c109780 2024-04-25T13:41:48.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 -- start start 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1be24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1be9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5575b1b47b00 con 0x5575b1818550 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575b1ae3920 con 0x5575b1841410 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8177c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5575b1b43ac0 con 0x5575b1820730 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54260/0 (socket says 172.21.15.94:54260) 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f816a59c640 1 -- 172.21.15.94:0/2114141841 learned_addr learned my addr 172.21.15.94:0/2114141841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1be9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.781+0000 7f8169d9b640 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1be24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f815b7fe640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 0x7f81540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 -- 172.21.15.94:0/2114141841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 msgr2=0x5575b1be9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1be9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 -- 172.21.15.94:0/2114141841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 msgr2=0x5575b1be24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1be24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 -- 172.21.15.94:0/2114141841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b19217e0 con 0x5575b1818550 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816ad9d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 0x7f81540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f815b7fe640 1 -- 172.21.15.94:0/3570110634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f81640dec30 con 0x7f816c109780 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816a59c640 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8160031500 tx=0x7f8160031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204836d64900ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f8158ff9640 1 -- 172.21.15.94:0/2114141841 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160035030 con 0x5575b1818550 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f816ad9d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 0x7f81540b2090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81640a6ea0 tx=0x7f81640d5660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f8158ff9640 1 -- 172.21.15.94:0/2114141841 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160040070 con 0x5575b1818550 2024-04-25T13:41:48.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.782+0000 7f8158ff9640 1 -- 172.21.15.94:0/2114141841 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600039f0 con 0x5575b1818550 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 -- 172.21.15.94:0/2114141841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 msgr2=0x5575b1be2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8160031500 tx=0x7f8160031bc0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 -- 172.21.15.94:0/2114141841 shutdown_connections 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1be9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1be2a20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 --2- 172.21.15.94:0/2114141841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1be24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 -- 172.21.15.94:0/2114141841 >> 172.21.15.94:0/2114141841 conn(0x5575b1835560 msgr2=0x5575b1bdde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 -- 172.21.15.94:0/2114141841 shutdown_connections 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.783+0000 7f8177c2c740 1 -- 172.21.15.94:0/2114141841 wait complete. 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 -- start start 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 Processor -- start 2024-04-25T13:41:48.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1bec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1bf3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5575b1bea310 con 0x5575b1818550 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575b1ae15d0 con 0x5575b1841410 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5575b1bea490 con 0x5575b1820730 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8177c2c740 10 client.0 _ll_register_callbacks cb 0x5575b1acf080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54270/0 (socket says 172.21.15.94:54270) 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 -- 172.21.15.94:0/3057147127 learned_addr learned my addr 172.21.15.94:0/3057147127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f816a59c640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1bec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1bf3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 -- 172.21.15.94:0/3057147127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 msgr2=0x5575b1bec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1bec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 -- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 msgr2=0x5575b1bf3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1bf3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.784+0000 7f8169d9b640 1 -- 172.21.15.94:0/3057147127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b1981f70 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8169d9b640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f815c004640 tx=0x7f815c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292189970d1ab823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c005020 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f815c0051c0 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160040070 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f815c0053c0 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575b1819170 con 0x5575b1818550 2024-04-25T13:41:48.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160040070 con 0x5575b1818550 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f815c0125e0 con 0x5575b1818550 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 20 client.15486 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b18001f0 0x5575b1bf9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 4 client.15486 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.785+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575b1b55710 con 0x5575b18001f0 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.786+0000 7f816a59c640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b18001f0 0x5575b1bf9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.786+0000 7f816a59c640 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b18001f0 0x5575b1bf9c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d1b4467ddf3c8 server_cookie=60080e45e1a35b56 in_seq=0 out_seq=0 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.786+0000 7f8151ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8151ffb640 1 -- 172.21.15.94:0/3057147127 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575b1b55710 con 0x5575b18001f0 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8151ffb640 10 client.15486 handle_command_reply: tid=0 2024-04-25T13:41:48.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 client.15486 shutdown 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 msgr2=0x5575b1bec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f815c004640 tx=0x7f815c00cb50 comp rx=0 tx=0).stop 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 shutdown_connections 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b18001f0 0x5575b1bf9c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1841410 0x5575b1bf3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8151ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1820730 0x5575b1bec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 --2- 172.21.15.94:0/3057147127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bec270 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 >> 172.21.15.94:0/3057147127 conn(0x5575b1835560 msgr2=0x5575b1bdcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 shutdown_connections 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 1 -- 172.21.15.94:0/3057147127 wait complete. 2024-04-25T13:41:48.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.787+0000 7f8177c2c740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T13:41:48.799 INFO:tasks.workunit.client.0.smithi094.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T13:41:48.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 Processor -- start 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 -- start start 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1818550 0x5575b1be1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1be1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1ab570 con 0x5575b18001f0 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1ad0b0 con 0x5575b1820730 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8177c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1abc40 con 0x5575b1818550 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54272/0 (socket says 172.21.15.94:54272) 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 -- 172.21.15.94:0/1824529146 learned_addr learned my addr 172.21.15.94:0/1824529146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f816a59c640 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1818550 0x5575b1be1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1be1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 -- 172.21.15.94:0/1824529146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1818550 msgr2=0x5575b1be1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1818550 0x5575b1be1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 -- 172.21.15.94:0/1824529146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 msgr2=0x5575b1be1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1be1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.807+0000 7f8169d9b640 1 -- 172.21.15.94:0/1824529146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b1beac90 con 0x5575b18001f0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.808+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f815c011da0 tx=0x7f815c015930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f078b25149a4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.808+0000 7f8153fff640 1 -- 172.21.15.94:0/1824529146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c005020 con 0x5575b18001f0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.808+0000 7f8153fff640 1 -- 172.21.15.94:0/1824529146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f815c02f070 con 0x5575b18001f0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.808+0000 7f8153fff640 1 -- 172.21.15.94:0/1824529146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c004640 con 0x5575b18001f0 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- 172.21.15.94:0/1824529146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 msgr2=0x5575b1be6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f815c011da0 tx=0x7f815c015930 comp rx=0 tx=0).stop 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- 172.21.15.94:0/1824529146 shutdown_connections 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1be1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b1818550 0x5575b1be1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1824529146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b18001f0 0x5575b1be6dc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- 172.21.15.94:0/1824529146 >> 172.21.15.94:0/1824529146 conn(0x5575b1835560 msgr2=0x5575b1be0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- 172.21.15.94:0/1824529146 shutdown_connections 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- 172.21.15.94:0/1824529146 wait complete. 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- start start 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 Processor -- start 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b18001f0 0x5575b1bf1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1bf29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1ac9e0 con 0x5575b1818550 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1ac310 con 0x5575b1820730 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1a9550 con 0x5575b18001f0 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.809+0000 7f8177c2c740 10 client.0 _ll_register_callbacks cb 0x7f815c015ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54288/0 (socket says 172.21.15.94:54288) 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 -- 172.21.15.94:0/1420572464 learned_addr learned my addr 172.21.15.94:0/1420572464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b18001f0 0x5575b1bf1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1bf29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 -- 172.21.15.94:0/1420572464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b18001f0 msgr2=0x5575b1bf1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b18001f0 0x5575b1bf1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 -- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 msgr2=0x5575b1bf29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1bf29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 -- 172.21.15.94:0/1420572464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b1bf2f70 con 0x5575b1818550 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1bf29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f816a59c640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f816002a930 tx=0x7f816002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245c0f1c9e589c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160057020 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f8160045880 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f815c02f070 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8160062050 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8177c2c740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.810+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575b1bf4230 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f815c02f070 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f816003ca50 con 0x5575b1818550 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 20 client.15492 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b1841410 0x5575b1c64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 4 client.15492 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575b1b4f580 con 0x5575b1841410 2024-04-25T13:41:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.811+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b1841410 0x5575b1c64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.812+0000 7f8151ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.812+0000 7f8169d9b640 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b1841410 0x5575b1c64440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eddc87699b0de71e server_cookie=faa488df259dc257 in_seq=0 out_seq=0 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8151ffb640 1 -- 172.21.15.94:0/1420572464 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575b1b4f580 con 0x5575b1841410 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8151ffb640 10 client.15492 handle_command_reply: tid=0 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 client.15492 shutdown 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 msgr2=0x5575b1bf1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f816002a930 tx=0x7f816002a410 comp rx=0 tx=0).stop 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 shutdown_connections 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5575b1841410 0x5575b1c64440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8151ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b1820730 0x5575b1bf29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575b1818550 0x5575b1bf1a80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 --2- 172.21.15.94:0/1420572464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575b18001f0 0x5575b1bf1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 >> 172.21.15.94:0/1420572464 conn(0x5575b1835560 msgr2=0x5575b1be1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 shutdown_connections 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 1 -- 172.21.15.94:0/1420572464 wait complete. 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.813+0000 7f8177c2c740 20 client.15492 trim_cache size 0 max 16384 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ec4be814-3a43-4e66-b755-341c3577f40c", 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:41:48.821 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:48.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 msgr2=0x7f81540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 0x7f81540b2090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81640a6ea0 tx=0x7f81640d5660 comp rx=0 tx=0).stop 2024-04-25T13:41:48.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 msgr2=0x7f816c1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8164009fd0 tx=0x7f816400b260 comp rx=0 tx=0).stop 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 shutdown_connections 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81540afb80 0x7f81540b2090 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f816c109780 0x7f816c1a1710 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f816c106c10 0x7f816c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 --2- 172.21.15.94:0/3570110634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f816c101e30 0x7f816c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 >> 172.21.15.94:0/3570110634 conn(0x7f816c0fd340 msgr2=0x7f816c10a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 shutdown_connections 2024-04-25T13:41:48.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.814+0000 7f8170d7d640 1 -- 172.21.15.94:0/3570110634 wait complete. 2024-04-25T13:41:48.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:41:48.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:41:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.977+0000 7f3308e6e640 1 -- 172.21.15.94:0/1323015773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304108920 msgr2=0x7f330410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.977+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1323015773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304108920 0x7f330410ad40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009ca0 tx=0x7f32fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 -- 172.21.15.94:0/1323015773 shutdown_connections 2024-04-25T13:41:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1323015773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304108920 0x7f330410ad40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1323015773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3304103c20 0x7f3304104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1323015773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33041029e0 0x7f33041036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 -- 172.21.15.94:0/1323015773 >> 172.21.15.94:0/1323015773 conn(0x7f33040fdbd0 msgr2=0x7f3304100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 -- 172.21.15.94:0/1323015773 shutdown_connections 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 -- 172.21.15.94:0/1323015773 wait complete. 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.978+0000 7f3308e6e640 1 Processor -- start 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 -- start start 2024-04-25T13:41:48.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33041029e0 0x7f330406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3304108920 0x7f33041a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f330410e510 con 0x7f3304103c20 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f330410e390 con 0x7f3304108920 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3308e6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f330410e690 con 0x7f33041029e0 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54300/0 (socket says 172.21.15.94:54300) 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 -- 172.21.15.94:0/1821713457 learned_addr learned my addr 172.21.15.94:0/1821713457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f330259c640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33041029e0 0x7f330406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3304108920 0x7f33041a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 -- 172.21.15.94:0/1821713457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33041029e0 msgr2=0x7f330406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33041029e0 0x7f330406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 -- 172.21.15.94:0/1821713457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3304108920 msgr2=0x7f33041a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:48.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3304108920 0x7f33041a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:48.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 -- 172.21.15.94:0/1821713457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33041a2000 con 0x7f3304103c20 2024-04-25T13:41:48.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.979+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00d770 tx=0x7f32ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:48.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.980+0000 7f32fbfff640 1 -- 172.21.15.94:0/1821713457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec017070 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.980+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33041a22f0 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.980+0000 7f32fbfff640 1 -- 172.21.15.94:0/1821713457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec0054d0 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.980+0000 7f32fbfff640 1 -- 172.21.15.94:0/1821713457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004c30 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.980+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33041a2840 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.982+0000 7f32fbfff640 1 -- 172.21.15.94:0/1821713457 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f32ec010460 con 0x7f3304103c20 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.982+0000 7f330fe01740 1 Processor -- start 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.982+0000 7f330fe01740 1 -- start start 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e52d44ed80 con 0x55e52d148390 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e52d4088b0 con 0x55e52d11f4d0 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330fe01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e52d44ad40 con 0x55e52d1276b0 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54314/0 (socket says 172.21.15.94:54314) 2024-04-25T13:41:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 -- 172.21.15.94:0/2536947459 learned_addr learned my addr 172.21.15.94:0/2536947459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330259c640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f32fbfff640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 0x7f32f00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 -- 172.21.15.94:0/2536947459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 msgr2=0x55e52d4eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 -- 172.21.15.94:0/2536947459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 msgr2=0x55e52d4e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 -- 172.21.15.94:0/2536947459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52d3d9a40 con 0x55e52d148390 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f330259c640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f32fbfff640 1 -- 172.21.15.94:0/1821713457 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f32ec0de9e0 con 0x7f3304103c20 2024-04-25T13:41:49.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.983+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 0x7f32f00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.984+0000 7f3302d9d640 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0011c0 tx=0x7f32fc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4e101df4d0c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.984+0000 7f32f8d99640 1 -- 172.21.15.94:0/2536947459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc004040 con 0x55e52d148390 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.984+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 0x7f32f00b2090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f32fc04c000 tx=0x7f32fc049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.984+0000 7f32f8d99640 1 -- 172.21.15.94:0/2536947459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32fc036040 con 0x55e52d148390 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.984+0000 7f32f8d99640 1 -- 172.21.15.94:0/2536947459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc02fa20 con 0x55e52d148390 2024-04-25T13:41:49.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- 172.21.15.94:0/2536947459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 msgr2=0x55e52d4e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0011c0 tx=0x7f32fc009800 comp rx=0 tx=0).stop 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- 172.21.15.94:0/2536947459 shutdown_connections 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- 172.21.15.94:0/2536947459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d148390 0x55e52d4e9a60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- 172.21.15.94:0/2536947459 >> 172.21.15.94:0/2536947459 conn(0x55e52d13c4e0 msgr2=0x55e52d4e5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- 172.21.15.94:0/2536947459 shutdown_connections 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- 172.21.15.94:0/2536947459 wait complete. 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- start start 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 Processor -- start 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d148390 0x55e52d4fac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e52d4eb600 con 0x55e52d11f4d0 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e52d3d5d10 con 0x55e52d148390 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.985+0000 7f330fe01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e52d4eb780 con 0x55e52d1276b0 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330fe01740 10 client.0 _ll_register_callbacks cb 0x55e52d402210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54322/0 (socket says 172.21.15.94:54322) 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 -- 172.21.15.94:0/1322708020 learned_addr learned my addr 172.21.15.94:0/1322708020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 -- 172.21.15.94:0/1322708020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 msgr2=0x55e52d4f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d148390 0x55e52d4fac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 -- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d148390 msgr2=0x55e52d4fac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d148390 0x55e52d4fac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 -- 172.21.15.94:0/1322708020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52d22a7d0 con 0x55e52d11f4d0 2024-04-25T13:41:49.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:49.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330259c640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32f400df40 tx=0x7f32f40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d78898e959abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f401f060 con 0x55e52d11f4d0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f32f4010960 con 0x55e52d11f4d0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32fc036040 con 0x55e52d11f4d0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32f4010b60 con 0x55e52d11f4d0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330fe01740 10 client.15504 fetch_fsmap learned FSMap version 25 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e52d1603f0 con 0x55e52d11f4d0 2024-04-25T13:41:49.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.986+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32fc036040 con 0x55e52d11f4d0 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f32f4019030 con 0x55e52d11f4d0 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 10 client.15504 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 20 client.15504 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d106f60 0x55e52d501240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 4 client.15504 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:49.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e52d45c990 con 0x55e52d106f60 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.987+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d106f60 0x55e52d501240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f32e9ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d106f60 0x55e52d501240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741a978eab3f7a07 server_cookie=ba87f6767e6eae47 in_seq=0 out_seq=0 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1322708020 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e52d45c990 con 0x55e52d106f60 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f32e9ffb640 10 client.15504 handle_command_reply: tid=0 2024-04-25T13:41:49.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f330fe01740 1 client.15504 shutdown 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 msgr2=0x55e52d4f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32f400df40 tx=0x7f32f40128e0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 shutdown_connections 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.988+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d106f60 0x55e52d501240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d148390 0x55e52d4fac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f32e9ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 --2- 172.21.15.94:0/1322708020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d11f4d0 0x55e52d4f37c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 >> 172.21.15.94:0/1322708020 conn(0x55e52d13c4e0 msgr2=0x55e52d4e43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 shutdown_connections 2024-04-25T13:41:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 1 -- 172.21.15.94:0/1322708020 wait complete. 2024-04-25T13:41:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:48.989+0000 7f330fe01740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T13:41:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.006+0000 7f330fe01740 1 Processor -- start 2024-04-25T13:41:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.006+0000 7f330fe01740 1 -- start start 2024-04-25T13:41:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33041ab450 con 0x55e52d106f60 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33041acf90 con 0x55e52d11f4d0 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330fe01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33041abb20 con 0x55e52d1276b0 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54334/0 (socket says 172.21.15.94:54334) 2024-04-25T13:41:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 -- 172.21.15.94:0/141736576 learned_addr learned my addr 172.21.15.94:0/141736576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f3302d9d640 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 -- 172.21.15.94:0/141736576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 msgr2=0x55e52d4e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 -- 172.21.15.94:0/141736576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 msgr2=0x55e52d4e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 -- 172.21.15.94:0/141736576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52d4ee150 con 0x55e52d106f60 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f330259c640 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32f4024ef0 tx=0x7f32f40057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8f4950c9f3161e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f32ebfff640 1 -- 172.21.15.94:0/141736576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4032070 con 0x55e52d106f60 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f32ebfff640 1 -- 172.21.15.94:0/141736576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f401a760 con 0x55e52d106f60 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.007+0000 7f32ebfff640 1 -- 172.21.15.94:0/141736576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4024760 con 0x55e52d106f60 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- 172.21.15.94:0/141736576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 msgr2=0x55e52d4f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32f4024ef0 tx=0x7f32f40057e0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- 172.21.15.94:0/141736576 shutdown_connections 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d1276b0 0x55e52d4e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d11f4d0 0x55e52d4e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 --2- 172.21.15.94:0/141736576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d106f60 0x55e52d4f2410 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- 172.21.15.94:0/141736576 >> 172.21.15.94:0/141736576 conn(0x55e52d13c4e0 msgr2=0x55e52d4e8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- 172.21.15.94:0/141736576 shutdown_connections 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- 172.21.15.94:0/141736576 wait complete. 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.008+0000 7f330fe01740 1 -- start start 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 Processor -- start 2024-04-25T13:41:49.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 0x55e52d4f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d11f4d0 0x55e52d4f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33041ac8c0 con 0x55e52d1276b0 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33041ac1f0 con 0x55e52d106f60 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33041a9430 con 0x55e52d11f4d0 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330fe01740 10 client.0 _ll_register_callbacks cb 0x7f32f400d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54344/0 (socket says 172.21.15.94:54344) 2024-04-25T13:41:49.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 -- 172.21.15.94:0/1329824092 learned_addr learned my addr 172.21.15.94:0/1329824092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330259c640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 0x55e52d4f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f32f9d9b640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d11f4d0 0x55e52d4f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 -- 172.21.15.94:0/1329824092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d11f4d0 msgr2=0x55e52d4f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d11f4d0 0x55e52d4f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 -- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 msgr2=0x55e52d4f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 0x55e52d4f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 -- 172.21.15.94:0/1329824092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52d1200f0 con 0x55e52d1276b0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f330259c640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 0x55e52d4f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f3302d9d640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005cc0 tx=0x7f32fc005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91567e0745ab36f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f32e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.009+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc062060 con 0x55e52d1276b0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f32fc004040 con 0x55e52d1276b0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f401a760 con 0x55e52d1276b0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32fc009360 con 0x55e52d1276b0 2024-04-25T13:41:49.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e52d40f4b0 con 0x55e52d1276b0 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f401a760 con 0x55e52d1276b0 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f32fc05c030 con 0x55e52d1276b0 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 20 client.15510 populate_metadata read hostname 'smithi094' 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d148390 0x55e52d56ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 4 client.15510 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e52d456800 con 0x55e52d148390 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.010+0000 7f330259c640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d148390 0x55e52d56ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330259c640 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d148390 0x55e52d56ae80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b895d499071c89c1 server_cookie=e55d5ef19496c3b7 in_seq=0 out_seq=0 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f32e9ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f32e9ffb640 1 -- 172.21.15.94:0/1329824092 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e52d456800 con 0x55e52d148390 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f32e9ffb640 10 client.15510 handle_command_reply: tid=0 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 client.15510 shutdown 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 msgr2=0x55e52d5675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005cc0 tx=0x7f32fc005df0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 shutdown_connections 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e52d148390 0x55e52d56ae80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e52d1276b0 0x55e52d5675a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e52d11f4d0 0x55e52d4f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f32e9ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 --2- 172.21.15.94:0/1329824092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e52d106f60 0x55e52d4f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 >> 172.21.15.94:0/1329824092 conn(0x55e52d13c4e0 msgr2=0x55e52d4e8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 shutdown_connections 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 1 -- 172.21.15.94:0/1329824092 wait complete. 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.011+0000 7f330fe01740 20 client.15510 trim_cache size 0 max 16384 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:41:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "ec4be814-3a43-4e66-b755-341c3577f40c": { 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ec4be814-3a43-4e66-b755-341c3577f40c", 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:41:49.039 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:41:49.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 msgr2=0x7f32f00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 0x7f32f00b2090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f32fc04c000 tx=0x7f32fc049000 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 msgr2=0x7f330419f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00d770 tx=0x7f32ec00dc30 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 shutdown_connections 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32f00afb80 0x7f32f00b2090 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3304108920 0x7f33041a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3304103c20 0x7f330419f270 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 --2- 172.21.15.94:0/1821713457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33041029e0 0x7f330406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 >> 172.21.15.94:0/1821713457 conn(0x7f33040fdbd0 msgr2=0x7f3304104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 shutdown_connections 2024-04-25T13:41:49.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:41:49.012+0000 7f3308e6e640 1 -- 172.21.15.94:0/1821713457 wait complete. 2024-04-25T13:41:49.042 INFO:tasks.cephfs.filesystem:scrub status for tag:ec4be814-3a43-4e66-b755-341c3577f40c - {'path': '/', 'tag': 'ec4be814-3a43-4e66-b755-341c3577f40c', 'options': 'recursive,force'} 2024-04-25T13:41:49.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:49 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/557908538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:49.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:49 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/557908538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:49.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/557908538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:41:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:50 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:41:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:50 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:50 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:41:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:50 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:50 smithi094 ceph-mon[29195]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:41:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:50 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:41:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:50 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:50 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:41:50.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:50 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:50 smithi138 ceph-mon[32977]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:41:50.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:50 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:41:50.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:50 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:50 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:41:50.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:50 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:41:50.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:50 smithi137 ceph-mon[32568]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:41:51.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:41:51.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:41:51.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:41:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:52 smithi094 ceph-mon[29195]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T13:41:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:52 smithi138 ceph-mon[32977]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T13:41:52.794 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:52 smithi137 ceph-mon[32568]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-25T13:41:54.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:54 smithi137 ceph-mon[32568]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T13:41:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:54 smithi094 ceph-mon[29195]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T13:41:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:54 smithi138 ceph-mon[32977]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T13:41:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:56 smithi137 ceph-mon[32568]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:41:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:56 smithi094 ceph-mon[29195]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:41:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:56 smithi138 ceph-mon[32977]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-25T13:41:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:41:58 smithi137 ceph-mon[32568]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T13:41:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:41:58 smithi094 ceph-mon[29195]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T13:41:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:41:58 smithi138 ceph-mon[32977]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T13:42:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:01 smithi137 ceph-mon[32568]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T13:42:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:01 smithi094 ceph-mon[29195]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T13:42:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:01 smithi138 ceph-mon[32977]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-25T13:42:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 -- 172.21.15.94:0/2714623863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50109a70 msgr2=0x7f7f5010be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 --2- 172.21.15.94:0/2714623863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50109a70 0x7f7f5010be90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 -- 172.21.15.94:0/2714623863 shutdown_connections 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 --2- 172.21.15.94:0/2714623863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50109a70 0x7f7f5010be90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 --2- 172.21.15.94:0/2714623863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50104d70 0x7f7f50105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 --2- 172.21.15.94:0/2714623863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 0x7f7f50104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 -- 172.21.15.94:0/2714623863 >> 172.21.15.94:0/2714623863 conn(0x7f7f500fed40 msgr2=0x7f7f501011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 -- 172.21.15.94:0/2714623863 shutdown_connections 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 -- 172.21.15.94:0/2714623863 wait complete. 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.811+0000 7f7f56e14640 1 Processor -- start 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 -- start start 2024-04-25T13:42:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 0x7f7f5019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 0x7f7f501a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f660 con 0x7f7f50104d70 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f4e0 con 0x7f7f50103b30 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f56e14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010f7e0 con 0x7f7f50109a70 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39772/0 (socket says 172.21.15.94:39772) 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 -- 172.21.15.94:0/802353094 learned_addr learned my addr 172.21.15.94:0/802353094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f553b1640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 0x7f7f501a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 0x7f7f5019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 -- 172.21.15.94:0/802353094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 msgr2=0x7f7f501a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 0x7f7f501a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 -- 172.21.15.94:0/802353094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 msgr2=0x7f7f5019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 0x7f7f5019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 -- 172.21.15.94:0/802353094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f501a1d20 con 0x7f7f50104d70 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f553b1640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 0x7f7f501a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000eb90 tx=0x7f7f4000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.812+0000 7f7f4dffb640 1 -- 172.21.15.94:0/802353094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004d20 con 0x7f7f50104d70 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.813+0000 7f7f4dffb640 1 -- 172.21.15.94:0/802353094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4000ee90 con 0x7f7f50104d70 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.813+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501a1fb0 con 0x7f7f50104d70 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.813+0000 7f7f4dffb640 1 -- 172.21.15.94:0/802353094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40010670 con 0x7f7f50104d70 2024-04-25T13:42:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.813+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501a2450 con 0x7f7f50104d70 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.815+0000 7f7f4dffb640 1 -- 172.21.15.94:0/802353094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f7f40005020 con 0x7f7f50104d70 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.815+0000 7f7f5dd9b740 1 Processor -- start 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.815+0000 7f7f5dd9b740 1 -- start start 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef801b0 0x555a0f320970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef5f4d0 0x555a0f327df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555a0f286460 con 0x555a0ef572f0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a0f240590 con 0x555a0ef5f4d0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a0f282420 con 0x555a0ef801b0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f4dffb640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 0x7f7f3c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39788/0 (socket says 172.21.15.94:39788) 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 -- 172.21.15.94:0/410710592 learned_addr learned my addr 172.21.15.94:0/410710592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef5f4d0 0x555a0f327df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f553b1640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 0x7f7f3c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f553b1640 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef801b0 0x555a0f320970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f4dffb640 1 -- 172.21.15.94:0/802353094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7f400df550 con 0x7f7f50104d70 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 -- 172.21.15.94:0/410710592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef801b0 msgr2=0x555a0f320970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef801b0 0x555a0f320970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 -- 172.21.15.94:0/410710592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef5f4d0 msgr2=0x555a0f327df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef5f4d0 0x555a0f327df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 -- 172.21.15.94:0/410710592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a0f24d0d0 con 0x555a0ef572f0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005bc0 tx=0x7f7f3800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1a0072c1c2d106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.816+0000 7f7f553b1640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 0x7f7f3c0b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555a0f321b40 tx=0x7f7f48031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f46ffd640 1 -- 172.21.15.94:0/410710592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38016070 con 0x555a0ef572f0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f46ffd640 1 -- 172.21.15.94:0/410710592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011920 con 0x555a0ef572f0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f46ffd640 1 -- 172.21.15.94:0/410710592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380106f0 con 0x555a0ef572f0 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/410710592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 msgr2=0x555a0f320ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005bc0 tx=0x7f7f3800e6c0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/410710592 shutdown_connections 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef5f4d0 0x555a0f327df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f320ef0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/410710592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef801b0 0x555a0f320970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/410710592 >> 172.21.15.94:0/410710592 conn(0x555a0ef74300 msgr2=0x555a0f31c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/410710592 shutdown_connections 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.817+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/410710592 wait complete. 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 -- start start 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 Processor -- start 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555a0f328840 con 0x555a0ef801b0 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a0f20db60 con 0x555a0ef572f0 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a0f3289c0 con 0x555a0ef5f4d0 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f5dd9b740 10 client.0 _ll_register_callbacks cb 0x555a0f2322b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f553b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f553b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39792/0 (socket says 172.21.15.94:39792) 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.818+0000 7f7f553b1640 1 -- 172.21.15.94:0/3572212149 learned_addr learned my addr 172.21.15.94:0/3572212149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 -- 172.21.15.94:0/3572212149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 msgr2=0x555a0f32ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 -- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 msgr2=0x555a0f32a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 -- 172.21.15.94:0/3572212149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a0ef98210 con 0x555a0ef801b0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f553b1640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7f48031a90 tx=0x7f7f4802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eff7a5131717182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4804e070 con 0x555a0ef801b0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7f480035f0 con 0x555a0ef801b0 2024-04-25T13:42:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38011920 con 0x555a0ef801b0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f48043040 con 0x555a0ef801b0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f5dd9b740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.819+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a0f064620 con 0x555a0ef801b0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011920 con 0x555a0ef801b0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4803a040 con 0x555a0ef801b0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 20 client.15522 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef3ed80 0x555a0f3383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 4 client.15522 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a0f294070 con 0x555a0ef3ed80 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef3ed80 0x555a0f3383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f2ffff640 10 client.15522 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.820+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef3ed80 0x555a0f3383e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9407cc85c7ec141f server_cookie=17d39bd0abe75782 in_seq=0 out_seq=0 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f2ffff640 1 -- 172.21.15.94:0/3572212149 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a0f294070 con 0x555a0ef3ed80 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f2ffff640 10 client.15522 handle_command_reply: tid=0 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 client.15522 shutdown 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 msgr2=0x555a0f331ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7f48031a90 tx=0x7f7f4802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 shutdown_connections 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef3ed80 0x555a0f3383e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f331ab0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f2ffff640 0 client.15522 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3572212149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.821+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 >> 172.21.15.94:0/3572212149 conn(0x555a0ef74300 msgr2=0x555a0f31f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.822+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 shutdown_connections 2024-04-25T13:42:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.822+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3572212149 wait complete. 2024-04-25T13:42:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.822+0000 7f7f5dd9b740 20 client.15522 trim_cache size 0 max 16384 2024-04-25T13:42:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 Processor -- start 2024-04-25T13:42:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 -- start start 2024-04-25T13:42:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f3319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 0x555a0f3292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ab570 con 0x555a0ef572f0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ad0b0 con 0x555a0ef801b0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501abc40 con 0x555a0ef5f4d0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39804/0 (socket says 172.21.15.94:39804) 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 -- 172.21.15.94:0/3708474869 learned_addr learned my addr 172.21.15.94:0/3708474869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f3319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f553b1640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 0x555a0f3292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 -- 172.21.15.94:0/3708474869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 msgr2=0x555a0f3319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f3319d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 -- 172.21.15.94:0/3708474869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 msgr2=0x555a0f3292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 0x555a0f3292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f54bb0640 1 -- 172.21.15.94:0/3708474869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a0f32b210 con 0x555a0ef572f0 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f3319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.837+0000 7f7f553b1640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 0x555a0f3292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.838+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x555a0f321d50 tx=0x7f7f38009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d9a74be5bfda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.838+0000 7f7f467fc640 1 -- 172.21.15.94:0/3708474869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38016040 con 0x555a0ef572f0 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.838+0000 7f7f467fc640 1 -- 172.21.15.94:0/3708474869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011640 con 0x555a0ef572f0 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.838+0000 7f7f467fc640 1 -- 172.21.15.94:0/3708474869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380048a0 con 0x555a0ef572f0 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3708474869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 msgr2=0x555a0f328d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x555a0f321d50 tx=0x7f7f38009450 comp rx=0 tx=0).stop 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3708474869 shutdown_connections 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef801b0 0x555a0f3292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f3319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/3708474869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef572f0 0x555a0f328d90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3708474869 >> 172.21.15.94:0/3708474869 conn(0x555a0ef74300 msgr2=0x555a0f324a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3708474869 shutdown_connections 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/3708474869 wait complete. 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 -- start start 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.839+0000 7f7f5dd9b740 1 Processor -- start 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ac9e0 con 0x555a0ef801b0 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501ac310 con 0x555a0ef572f0 2024-04-25T13:42:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501a9550 con 0x555a0ef5f4d0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 10 client.0 _ll_register_callbacks cb 0x7f7f400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39816/0 (socket says 172.21.15.94:39816) 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 -- 172.21.15.94:0/4099221612 learned_addr learned my addr 172.21.15.94:0/4099221612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f4ffff640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 -- 172.21.15.94:0/4099221612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 msgr2=0x555a0f32ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 -- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 msgr2=0x555a0f32ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 -- 172.21.15.94:0/4099221612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a0f39ecf0 con 0x555a0ef801b0 2024-04-25T13:42:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f553b1640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7f48033000 tx=0x7f7f48009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92d5d00ddb7c423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4804e040 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7f480037f0 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.840+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38011640 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f48035040 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555a0f39ff50 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38011640 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4803a040 con 0x555a0ef801b0 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 20 client.15528 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef141c0 0x555a0f3a2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 4 client.15528 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:42:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a0f28dee0 con 0x555a0ef141c0 2024-04-25T13:42:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.841+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef141c0 0x555a0f3a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.842+0000 7f7f2ffff640 10 client.15528 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.842+0000 7f7f54bb0640 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef141c0 0x555a0f3a2920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f139347c0426ba server_cookie=23a30ed3bdb92b7d in_seq=0 out_seq=0 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f2ffff640 1 -- 172.21.15.94:0/4099221612 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a0f28dee0 con 0x555a0ef141c0 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f2ffff640 10 client.15528 handle_command_reply: tid=0 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 client.15528 shutdown 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 msgr2=0x555a0f39e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7f48033000 tx=0x7f7f48009520 comp rx=0 tx=0).stop 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 shutdown_connections 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a0ef141c0 0x555a0f3a2920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0ef801b0 0x555a0f39e780 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f2ffff640 0 client.15528 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a0ef5f4d0 0x555a0f32ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 --2- 172.21.15.94:0/4099221612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0ef572f0 0x555a0f32ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 >> 172.21.15.94:0/4099221612 conn(0x555a0ef74300 msgr2=0x555a0f31f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 shutdown_connections 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 1 -- 172.21.15.94:0/4099221612 wait complete. 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.854+0000 7f7f5dd9b740 20 client.15528 trim_cache size 0 max 16384 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:42:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:42:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 msgr2=0x7f7f3c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 0x7f7f3c0b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555a0f321b40 tx=0x7f7f48031400 comp rx=0 tx=0).stop 2024-04-25T13:42:01.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 msgr2=0x7f7f5019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000eb90 tx=0x7f7f4000c6a0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 shutdown_connections 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7f3c0afb70 0x7f7f3c0b2080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f50109a70 0x7f7f501a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f50104d70 0x7f7f5019a720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 --2- 172.21.15.94:0/802353094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f50103b30 0x7f7f5019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 >> 172.21.15.94:0/802353094 conn(0x7f7f500fed40 msgr2=0x7f7f50105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 shutdown_connections 2024-04-25T13:42:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:01.855+0000 7f7f56e14640 1 -- 172.21.15.94:0/802353094 wait complete. 2024-04-25T13:42:03.436 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:03 smithi094 ceph-mon[29195]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:42:03.437 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:03 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:42:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:03 smithi138 ceph-mon[32977]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:42:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:03 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:42:03.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:03 smithi137 ceph-mon[32568]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-04-25T13:42:03.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:03 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:42:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:42:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:42:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:42:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:06 smithi137 ceph-mon[32568]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:06 smithi094 ceph-mon[29195]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:06 smithi138 ceph-mon[32977]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:08.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:08 smithi137 ceph-mon[32568]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-25T13:42:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:08 smithi094 ceph-mon[29195]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-25T13:42:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:08 smithi138 ceph-mon[32977]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-25T13:42:10.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:10 smithi137 ceph-mon[32568]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:42:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:10 smithi094 ceph-mon[29195]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:42:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:10 smithi138 ceph-mon[32977]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-25T13:42:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:12 smithi137 ceph-mon[32568]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-25T13:42:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:12 smithi094 ceph-mon[29195]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-25T13:42:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:12 smithi138 ceph-mon[32977]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-04-25T13:42:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:14 smithi137 ceph-mon[32568]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:14 smithi094 ceph-mon[29195]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:14 smithi138 ceph-mon[32977]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 -- 172.21.15.94:0/4036750223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1033f0 msgr2=0x7f692c103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 --2- 172.21.15.94:0/4036750223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1033f0 0x7f692c103800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f692000aa10 tx=0x7f692002b720 comp rx=0 tx=0).stop 2024-04-25T13:42:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 -- 172.21.15.94:0/4036750223 shutdown_connections 2024-04-25T13:42:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 --2- 172.21.15.94:0/4036750223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1080f0 0x7f692c10a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 --2- 172.21.15.94:0/4036750223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1033f0 0x7f692c103800 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 --2- 172.21.15.94:0/4036750223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1021b0 0x7f692c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 -- 172.21.15.94:0/4036750223 >> 172.21.15.94:0/4036750223 conn(0x7f692c0fd3a0 msgr2=0x7f692c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 -- 172.21.15.94:0/4036750223 shutdown_connections 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.960+0000 7f6931d8f640 1 -- 172.21.15.94:0/4036750223 wait complete. 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 Processor -- start 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 -- start start 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1033f0 0x7f692c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1080f0 0x7f692c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10d440 con 0x7f692c1021b0 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10d2c0 con 0x7f692c1080f0 2024-04-25T13:42:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f6931d8f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10d5c0 con 0x7f692c1033f0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.961+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54068/0 (socket says 172.21.15.94:54068) 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 -- 172.21.15.94:0/3348203363 learned_addr learned my addr 172.21.15.94:0/3348203363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692bfff640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1080f0 0x7f692c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692affd640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1033f0 0x7f692c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 -- 172.21.15.94:0/3348203363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1033f0 msgr2=0x7f692c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1033f0 0x7f692c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 -- 172.21.15.94:0/3348203363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1080f0 msgr2=0x7f692c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1080f0 0x7f692c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 -- 172.21.15.94:0/3348203363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a1d10 con 0x7f692c1021b0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f692b7fe640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f691c005fd0 tx=0x7f691c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f6928ff9640 1 -- 172.21.15.94:0/3348203363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c017070 con 0x7f692c1021b0 2024-04-25T13:42:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f6928ff9640 1 -- 172.21.15.94:0/3348203363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c009c30 con 0x7f692c1021b0 2024-04-25T13:42:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1a1fa0 con 0x7f692c1021b0 2024-04-25T13:42:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f6928ff9640 1 -- 172.21.15.94:0/3348203363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c003d10 con 0x7f692c1021b0 2024-04-25T13:42:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.962+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a24b0 con 0x7f692c1021b0 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 Processor -- start 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6928ff9640 1 -- 172.21.15.94:0/3348203363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f691c003eb0 con 0x7f692c1021b0 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 -- start start 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330ed1b0 0x55d83348d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330cc4d0 0x55d833494df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d8333f3460 con 0x55d8330c42f0 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8333ad590 con 0x55d8330ed1b0 2024-04-25T13:42:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f6938c44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8333ef420 con 0x55d8330cc4d0 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54086/0 (socket says 172.21.15.94:54086) 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f692b7fe640 1 -- 172.21.15.94:0/1464908225 learned_addr learned my addr 172.21.15.94:0/1464908225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f692affd640 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330cc4d0 0x55d833494df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.965+0000 7f692bfff640 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330ed1b0 0x55d83348d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 -- 172.21.15.94:0/1464908225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330cc4d0 msgr2=0x55d833494df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330cc4d0 0x55d833494df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 -- 172.21.15.94:0/1464908225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330ed1b0 msgr2=0x55d83348d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330ed1b0 0x55d83348d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 -- 172.21.15.94:0/1464908225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8333ba0d0 con 0x55d8330c42f0 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f6928ff9640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 0x7f69140b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692b7fe640 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f691c0dc250 tx=0x7f691c0e7190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8790adf4398c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f692bfff640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 0x7f69140b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f6928ff9640 1 -- 172.21.15.94:0/3348203363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f691c0dea00 con 0x7f692c1021b0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f6911ffb640 1 -- 172.21.15.94:0/1464908225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0e5070 con 0x55d8330c42f0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f6911ffb640 1 -- 172.21.15.94:0/1464908225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c08f020 con 0x55d8330c42f0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.966+0000 7f6911ffb640 1 -- 172.21.15.94:0/1464908225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0a6450 con 0x55d8330c42f0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- 172.21.15.94:0/1464908225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 msgr2=0x55d83348def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f692bfff640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 0x7f69140b2100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55d83348eb40 tx=0x7f69240096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f691c0dc250 tx=0x7f691c0e7190 comp rx=0 tx=0).stop 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- 172.21.15.94:0/1464908225 shutdown_connections 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330cc4d0 0x55d833494df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330c42f0 0x55d83348def0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 --2- 172.21.15.94:0/1464908225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330ed1b0 0x55d83348d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- 172.21.15.94:0/1464908225 >> 172.21.15.94:0/1464908225 conn(0x55d8330e1300 msgr2=0x55d833489300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- 172.21.15.94:0/1464908225 shutdown_connections 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- 172.21.15.94:0/1464908225 wait complete. 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.967+0000 7f6938c44740 1 -- start start 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 Processor -- start 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833497670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d833497bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d833495840 con 0x55d8330ed1b0 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d83337ab60 con 0x55d8330cc4d0 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8334959c0 con 0x55d8330c42f0 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f6938c44740 10 client.0 _ll_register_callbacks cb 0x55d83339f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54088/0 (socket says 172.21.15.94:54088) 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 -- 172.21.15.94:0/4078095041 learned_addr learned my addr 172.21.15.94:0/4078095041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692b7fe640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833497670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692affd640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d833497bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 -- 172.21.15.94:0/4078095041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 msgr2=0x55d833497670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833497670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 -- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 msgr2=0x55d833497bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d833497bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.968+0000 7f692bfff640 1 -- 172.21.15.94:0/4078095041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d833105210 con 0x55d8330ed1b0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f692bfff640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6924010790 tx=0x7f6924004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e419d64b2b5869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924004b20 con 0x55d8330ed1b0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6924004cc0 con 0x55d8330ed1b0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c08f020 con 0x55d8330ed1b0 2024-04-25T13:42:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f692401f8b0 con 0x55d8330ed1b0 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 10 client.15540 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d8331d1620 con 0x55d8330ed1b0 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c08f020 con 0x55d8330ed1b0 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6924026070 con 0x55d8330ed1b0 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 10 client.15540 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 20 client.15540 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330abd80 0x55d8334a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 4 client.15540 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.969+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d833401070 con 0x55d8330abd80 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.970+0000 7f692b7fe640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330abd80 0x55d8334a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.970+0000 7f690affd640 10 client.15540 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.970+0000 7f692b7fe640 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330abd80 0x55d8334a53f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691c5b6fe4fe7270 server_cookie=e8034677be05452a in_seq=0 out_seq=0 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f690affd640 1 -- 172.21.15.94:0/4078095041 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d833401070 con 0x55d8330abd80 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f690affd640 10 client.15540 handle_command_reply: tid=0 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 client.15540 shutdown 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 msgr2=0x55d83349eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6924010790 tx=0x7f6924004ab0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 shutdown_connections 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330abd80 0x55d8334a53f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330ed1b0 0x55d83349eb10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d833497bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f690affd640 0 client.15540 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 --2- 172.21.15.94:0/4078095041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833497670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 >> 172.21.15.94:0/4078095041 conn(0x55d8330e1300 msgr2=0x55d833488250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 shutdown_connections 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 1 -- 172.21.15.94:0/4078095041 wait complete. 2024-04-25T13:42:14.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.971+0000 7f6938c44740 20 client.15540 trim_cache size 0 max 16384 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 Processor -- start 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 -- start start 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833488ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d8334890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1ab610 con 0x55d8330abd80 2024-04-25T13:42:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1ad150 con 0x55d8330cc4d0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f6938c44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1abce0 con 0x55d8330c42f0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54092/0 (socket says 172.21.15.94:54092) 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.986+0000 7f692b7fe640 1 -- 172.21.15.94:0/707613852 learned_addr learned my addr 172.21.15.94:0/707613852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692affd640 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833488ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 -- 172.21.15.94:0/707613852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 msgr2=0x55d833488ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833488ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692bfff640 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d8334890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 -- 172.21.15.94:0/707613852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 msgr2=0x55d8334890e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d8334890e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 -- 172.21.15.94:0/707613852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d833492510 con 0x55d8330abd80 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f692b7fe640 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f691c000c00 tx=0x7f691c00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9c771b8c302d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f69117fa640 1 -- 172.21.15.94:0/707613852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0e5040 con 0x55d8330abd80 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f69117fa640 1 -- 172.21.15.94:0/707613852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c08f040 con 0x55d8330abd80 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.987+0000 7f69117fa640 1 -- 172.21.15.94:0/707613852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c009de0 con 0x55d8330abd80 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- 172.21.15.94:0/707613852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 msgr2=0x55d833496470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f691c000c00 tx=0x7f691c00df10 comp rx=0 tx=0).stop 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- 172.21.15.94:0/707613852 shutdown_connections 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330cc4d0 0x55d8334890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833488ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 --2- 172.21.15.94:0/707613852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330abd80 0x55d833496470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- 172.21.15.94:0/707613852 >> 172.21.15.94:0/707613852 conn(0x55d8330e1300 msgr2=0x55d833495d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- 172.21.15.94:0/707613852 shutdown_connections 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- 172.21.15.94:0/707613852 wait complete. 2024-04-25T13:42:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 -- start start 2024-04-25T13:42:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.988+0000 7f6938c44740 1 Processor -- start 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 0x55d833508b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833509070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1aca80 con 0x55d8330cc4d0 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1ac3b0 con 0x55d8330abd80 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1a95f0 con 0x55d8330c42f0 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 10 client.0 _ll_register_callbacks cb 0x7f6924009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54104/0 (socket says 172.21.15.94:54104) 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 -- 172.21.15.94:0/683909147 learned_addr learned my addr 172.21.15.94:0/683909147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692b7fe640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 0x55d833508b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692affd640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833509070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 -- 172.21.15.94:0/683909147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 msgr2=0x55d833509070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833509070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 -- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 msgr2=0x55d833508b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 0x55d833508b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 -- 172.21.15.94:0/683909147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d833510590 con 0x55d8330cc4d0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692b7fe640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 0x55d833508b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f692bfff640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6924006010 tx=0x7f692401f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9614ade676f99d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f690affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924034070 con 0x55d8330cc4d0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6924027790 con 0x55d8330cc4d0 2024-04-25T13:42:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.989+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c08f040 con 0x55d8330cc4d0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f692401fe60 con 0x55d8330cc4d0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 10 client.15546 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d833511820 con 0x55d8330cc4d0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c08f040 con 0x55d8330cc4d0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6924026040 con 0x55d8330cc4d0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 10 client.15546 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 20 client.15546 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330ed1b0 0x55d8335141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 4 client.15546 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8333faee0 con 0x55d8330ed1b0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.990+0000 7f692b7fe640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330ed1b0 0x55d8335141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.991+0000 7f690affd640 10 client.15546 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:14.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.991+0000 7f692b7fe640 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330ed1b0 0x55d8335141c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434c348b72ebde74 server_cookie=10cd1ecfd20fab03 in_seq=0 out_seq=0 2024-04-25T13:42:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f690affd640 1 -- 172.21.15.94:0/683909147 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8333faee0 con 0x55d8330ed1b0 2024-04-25T13:42:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f690affd640 10 client.15546 handle_command_reply: tid=0 2024-04-25T13:42:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f6938c44740 1 client.15546 shutdown 2024-04-25T13:42:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 msgr2=0x55d83350ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6924006010 tx=0x7f692401f2f0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.994+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 shutdown_connections 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d8330ed1b0 0x55d8335141c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8330cc4d0 0x55d83350ffd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f690affd640 0 client.15546 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8330c42f0 0x55d833509070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 --2- 172.21.15.94:0/683909147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8330abd80 0x55d833508b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 >> 172.21.15.94:0/683909147 conn(0x55d8330e1300 msgr2=0x55d833495d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 shutdown_connections 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 1 -- 172.21.15.94:0/683909147 wait complete. 2024-04-25T13:42:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6938c44740 20 client.15546 trim_cache size 0 max 16384 2024-04-25T13:42:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:42:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:42:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:42:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 msgr2=0x7f69140b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 0x7f69140b2100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55d83348eb40 tx=0x7f69240096a0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 msgr2=0x7f692c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f691c005fd0 tx=0x7f691c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 shutdown_connections 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69140afbf0 0x7f69140b2100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f692c1080f0 0x7f692c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f692c1033f0 0x7f692c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 --2- 172.21.15.94:0/3348203363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f692c1021b0 0x7f692c19a1d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 >> 172.21.15.94:0/3348203363 conn(0x7f692c0fd3a0 msgr2=0x7f692c1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 shutdown_connections 2024-04-25T13:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:14.995+0000 7f6931d8f640 1 -- 172.21.15.94:0/3348203363 wait complete. 2024-04-25T13:42:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:16 smithi137 ceph-mon[32568]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:16 smithi094 ceph-mon[29195]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:16 smithi138 ceph-mon[32977]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T13:42:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:18 smithi137 ceph-mon[32568]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T13:42:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:18 smithi094 ceph-mon[29195]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T13:42:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:18 smithi138 ceph-mon[32977]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-04-25T13:42:19.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:42:19.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:42:19.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 -- 172.21.15.94:0/3076194864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960102240 msgr2=0x7fa96010bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 --2- 172.21.15.94:0/3076194864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960102240 0x7fa96010bca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa958009900 tx=0x7fa95802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:19.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 -- 172.21.15.94:0/3076194864 shutdown_connections 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 --2- 172.21.15.94:0/3076194864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960102240 0x7fa96010bca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 --2- 172.21.15.94:0/3076194864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960101740 0x7fa960101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 --2- 172.21.15.94:0/3076194864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960105090 0x7fa9601054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 -- 172.21.15.94:0/3076194864 >> 172.21.15.94:0/3076194864 conn(0x7fa9600fd340 msgr2=0x7fa9600ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 -- 172.21.15.94:0/3076194864 shutdown_connections 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.174+0000 7fa96816c640 1 -- 172.21.15.94:0/3076194864 wait complete. 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 Processor -- start 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 -- start start 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960102240 0x7fa96019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 0x7fa9601a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010f620 con 0x7fa960101740 2024-04-25T13:42:19.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010f4a0 con 0x7fa960102240 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa96816c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010f7a0 con 0x7fa960105090 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54118/0 (socket says 172.21.15.94:54118) 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 -- 172.21.15.94:0/28661591 learned_addr learned my addr 172.21.15.94:0/28661591 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965707640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960102240 0x7fa96019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa966709640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 0x7fa9601a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 -- 172.21.15.94:0/28661591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 msgr2=0x7fa9601a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 0x7fa9601a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 -- 172.21.15.94:0/28661591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960102240 msgr2=0x7fa96019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960102240 0x7fa96019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 -- 172.21.15.94:0/28661591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601a1ce0 con 0x7fa960101740 2024-04-25T13:42:19.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa966709640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 0x7fa9601a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.175+0000 7fa965f08640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00ead0 tx=0x7fa95c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.176+0000 7fa956ffd640 1 -- 172.21.15.94:0/28661591 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00ce40 con 0x7fa960101740 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.176+0000 7fa956ffd640 1 -- 172.21.15.94:0/28661591 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95c011920 con 0x7fa960101740 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.176+0000 7fa956ffd640 1 -- 172.21.15.94:0/28661591 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c01aae0 con 0x7fa960101740 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.176+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9601a1fd0 con 0x7fa960101740 2024-04-25T13:42:19.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.176+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a2430 con 0x7fa960101740 2024-04-25T13:42:19.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.178+0000 7fa96f01b740 1 Processor -- start 2024-04-25T13:42:19.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.178+0000 7fa96f01b740 1 -- start start 2024-04-25T13:42:19.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.178+0000 7fa956ffd640 1 -- 172.21.15.94:0/28661591 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fa95c01ac80 con 0x7fa960101740 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658c0f390 0x556658fb4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658be64d0 0x556658fb53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556658f15d80 con 0x556658bee6b0 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556658ecf8b0 con 0x556658c0f390 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa96f01b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556658f11d40 con 0x556658be64d0 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54130/0 (socket says 172.21.15.94:54130) 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 -- 172.21.15.94:0/3552154705 learned_addr learned my addr 172.21.15.94:0/3552154705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965f08640 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658be64d0 0x556658fb53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa966709640 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658c0f390 0x556658fb4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 -- 172.21.15.94:0/3552154705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658be64d0 msgr2=0x556658fb53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658be64d0 0x556658fb53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 -- 172.21.15.94:0/3552154705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658c0f390 msgr2=0x556658fb4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658c0f390 0x556658fb4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 -- 172.21.15.94:0/3552154705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556658ea0a40 con 0x556658bee6b0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa965707640 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa950004650 tx=0x7fa95000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e72eedbfdb2c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.179+0000 7fa956ffd640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 0x7fa94c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa94bfff640 1 -- 172.21.15.94:0/3552154705 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950003bd0 con 0x556658bee6b0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa94bfff640 1 -- 172.21.15.94:0/3552154705 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950015070 con 0x556658bee6b0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa94bfff640 1 -- 172.21.15.94:0/3552154705 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950004d20 con 0x556658bee6b0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa966709640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 0x7fa94c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa956ffd640 1 -- 172.21.15.94:0/28661591 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa95c0df360 con 0x7fa960101740 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa96f01b740 1 -- 172.21.15.94:0/3552154705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 msgr2=0x556658fb6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa950004650 tx=0x7fa95000bba0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.180+0000 7fa966709640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 0x7fa94c0b2090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa958031440 tx=0x7fa958009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 -- 172.21.15.94:0/3552154705 shutdown_connections 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fb6080 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658be64d0 0x556658fb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3552154705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658c0f390 0x556658fb4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 -- 172.21.15.94:0/3552154705 >> 172.21.15.94:0/3552154705 conn(0x556658c034e0 msgr2=0x556658fac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 -- 172.21.15.94:0/3552154705 shutdown_connections 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 -- 172.21.15.94:0/3552154705 wait complete. 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 -- start start 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.181+0000 7fa96f01b740 1 Processor -- start 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fbed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658c0f390 0x556658fc61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556658fb6b90 con 0x556658bee6b0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556658e9cd10 con 0x556658be64d0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556658fb6d10 con 0x556658c0f390 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 10 client.0 _ll_register_callbacks cb 0x556658ec9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54144/0 (socket says 172.21.15.94:54144) 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 -- 172.21.15.94:0/269588527 learned_addr learned my addr 172.21.15.94:0/269588527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965f08640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fbed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa966709640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658c0f390 0x556658fc61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 -- 172.21.15.94:0/269588527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658c0f390 msgr2=0x556658fc61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658c0f390 0x556658fc61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 -- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 msgr2=0x556658fbed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fbed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 -- 172.21.15.94:0/269588527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556658cf17d0 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa965707640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa950004280 tx=0x7fa9500043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ccefc441efe7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950028060 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa950011640 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa950015070 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa950011840 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.182+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556658c273f0 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950015070 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa950022070 con 0x556658bee6b0 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 20 client.15558 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658bcdf60 0x556658fcc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 4 client.15558 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556658f23990 con 0x556658bcdf60 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.183+0000 7fa965f08640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658bcdf60 0x556658fcc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.184+0000 7fa9497fa640 10 client.15558 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.184+0000 7fa965f08640 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658bcdf60 0x556658fcc380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d670387b69703b8 server_cookie=5250a7c8912d747d in_seq=0 out_seq=0 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.184+0000 7fa9497fa640 1 -- 172.21.15.94:0/269588527 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556658f23990 con 0x556658bcdf60 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa9497fa640 10 client.15558 handle_command_reply: tid=0 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 client.15558 shutdown 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 msgr2=0x556658fbf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa950004280 tx=0x7fa9500043a0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 shutdown_connections 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658bcdf60 0x556658fcc380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658c0f390 0x556658fc61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bee6b0 0x556658fbf250 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa9497fa640 0 client.15558 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 --2- 172.21.15.94:0/269588527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fbed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 >> 172.21.15.94:0/269588527 conn(0x556658c034e0 msgr2=0x556658faf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 shutdown_connections 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 1 -- 172.21.15.94:0/269588527 wait complete. 2024-04-25T13:42:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.185+0000 7fa96f01b740 20 client.15558 trim_cache size 0 max 16384 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 Processor -- start 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 -- start start 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 0x556658fbda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 0x556658fabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601a92b0 con 0x556658be64d0 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601aadf0 con 0x556658bee6b0 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa96f01b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601a9980 con 0x556658bcdf60 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54160/0 (socket says 172.21.15.94:54160) 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 -- 172.21.15.94:0/55604382 learned_addr learned my addr 172.21.15.94:0/55604382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa966709640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 0x556658fabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965f08640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 0x556658fbda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 -- 172.21.15.94:0/55604382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 msgr2=0x556658fbda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 0x556658fbda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 -- 172.21.15.94:0/55604382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 msgr2=0x556658fabc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 0x556658fabc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 -- 172.21.15.94:0/55604382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556658be70f0 con 0x556658be64d0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965f08640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 0x556658fbda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa966709640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 0x556658fabc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.203+0000 7fa965707640 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b620 tx=0x7fa95000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fe7254380facd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa94b7fe640 1 -- 172.21.15.94:0/55604382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950028070 con 0x556658be64d0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa94b7fe640 1 -- 172.21.15.94:0/55604382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950015040 con 0x556658be64d0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa94b7fe640 1 -- 172.21.15.94:0/55604382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95002d780 con 0x556658be64d0 2024-04-25T13:42:19.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 -- 172.21.15.94:0/55604382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 msgr2=0x556658fc60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b620 tx=0x7fa95000b5b0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 -- 172.21.15.94:0/55604382 shutdown_connections 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658bee6b0 0x556658fabc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658be64d0 0x556658fc60d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 --2- 172.21.15.94:0/55604382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bcdf60 0x556658fbda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 -- 172.21.15.94:0/55604382 >> 172.21.15.94:0/55604382 conn(0x556658c034e0 msgr2=0x556658fbd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 -- 172.21.15.94:0/55604382 shutdown_connections 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.204+0000 7fa96f01b740 1 -- 172.21.15.94:0/55604382 wait complete. 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 -- start start 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 Processor -- start 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fc33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bee6b0 0x556659032600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601aa720 con 0x556658bcdf60 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601aa050 con 0x556658be64d0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601a7290 con 0x556658bee6b0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa96f01b740 10 client.0 _ll_register_callbacks cb 0x7fa95c0e2d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa965f08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa965f08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54174/0 (socket says 172.21.15.94:54174) 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.205+0000 7fa965f08640 1 -- 172.21.15.94:0/3050527345 learned_addr learned my addr 172.21.15.94:0/3050527345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa966709640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bee6b0 0x556659032600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965707640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fc33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 -- 172.21.15.94:0/3050527345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bee6b0 msgr2=0x556659032600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bee6b0 0x556659032600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 -- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 msgr2=0x556658fc33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fc33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 -- 172.21.15.94:0/3050527345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556658ed64b0 con 0x556658bcdf60 2024-04-25T13:42:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa965f08640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a75d0 tx=0x7fa95c0ca040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732dd10c3d399181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0a6db0 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa95c0dcec0 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa950015040 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa95c0a7150 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556659033880 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950015040 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa95c022030 con 0x556658bcdf60 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 20 client.15564 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658c0f390 0x556659036220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 4 client.15564 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.206+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556658f1d800 con 0x556658c0f390 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa965707640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658c0f390 0x556659036220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa965707640 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658c0f390 0x556659036220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed56825a5fb9c33f server_cookie=c8844855a7440762 in_seq=0 out_seq=0 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa9497fa640 10 client.15564 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa9497fa640 1 -- 172.21.15.94:0/3050527345 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556658f1d800 con 0x556658c0f390 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa9497fa640 10 client.15564 handle_command_reply: tid=0 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa96f01b740 1 client.15564 shutdown 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 msgr2=0x556658fc2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.207+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a75d0 tx=0x7fa95c0ca040 comp rx=0 tx=0).stop 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 shutdown_connections 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556658c0f390 0x556659036220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556658bee6b0 0x556659032600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556658be64d0 0x556658fc33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 --2- 172.21.15.94:0/3050527345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556658bcdf60 0x556658fc2ea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa9497fa640 0 client.15564 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 >> 172.21.15.94:0/3050527345 conn(0x556658c034e0 msgr2=0x556658fabc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 shutdown_connections 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 1 -- 172.21.15.94:0/3050527345 wait complete. 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96f01b740 20 client.15564 trim_cache size 0 max 16384 2024-04-25T13:42:19.215 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:19.216 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:42:19.216 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:42:19.216 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 msgr2=0x7fa94c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 0x7fa94c0b2090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa958031440 tx=0x7fa958009680 comp rx=0 tx=0).stop 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 msgr2=0x7fa96019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00ead0 tx=0x7fa95c00ef90 comp rx=0 tx=0).stop 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 shutdown_connections 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa94c0afb80 0x7fa94c0b2090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa960105090 0x7fa9601a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa960102240 0x7fa96019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 --2- 172.21.15.94:0/28661591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa960101740 0x7fa96019a1a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 >> 172.21.15.94:0/28661591 conn(0x7fa9600fd340 msgr2=0x7fa960107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 shutdown_connections 2024-04-25T13:42:19.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.208+0000 7fa96816c640 1 -- 172.21.15.94:0/28661591 wait complete. 2024-04-25T13:42:19.219 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:42:19.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 -- 172.21.15.94:0/470932122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 msgr2=0x7f322c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 --2- 172.21.15.94:0/470932122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c10ad60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3224009ca0 tx=0x7f322402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 -- 172.21.15.94:0/470932122 shutdown_connections 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 --2- 172.21.15.94:0/470932122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c10ad60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 --2- 172.21.15.94:0/470932122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 0x7f322c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 --2- 172.21.15.94:0/470932122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 0x7f322c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 -- 172.21.15.94:0/470932122 >> 172.21.15.94:0/470932122 conn(0x7f322c0fdbf0 msgr2=0x7f322c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 -- 172.21.15.94:0/470932122 shutdown_connections 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.327+0000 7f3232068640 1 -- 172.21.15.94:0/470932122 wait complete. 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 Processor -- start 2024-04-25T13:42:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 -- start start 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 0x7f322c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 0x7f322c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e530 con 0x7f322c108940 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e3b0 con 0x7f322c102a00 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f3232068640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f322c10e6b0 con 0x7f322c103c40 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f322bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f322bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54184/0 (socket says 172.21.15.94:54184) 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f322bfff640 1 -- 172.21.15.94:0/2094554186 learned_addr learned my addr 172.21.15.94:0/2094554186 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f322affd640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 0x7f322c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.328+0000 7f322b7fe640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 0x7f322c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 -- 172.21.15.94:0/2094554186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 msgr2=0x7f322c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 0x7f322c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 -- 172.21.15.94:0/2094554186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 msgr2=0x7f322c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 0x7f322c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 -- 172.21.15.94:0/2094554186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c1a1cc0 con 0x7f322c108940 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f322bfff640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32240097b0 tx=0x7f3224031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224003bf0 con 0x7f322c108940 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1a1f50 con 0x7f322c108940 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3224003d90 con 0x7f322c108940 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322402fc30 con 0x7f322c108940 2024-04-25T13:42:19.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.329+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a23b0 con 0x7f322c108940 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.330+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c06a190 con 0x7f322c108940 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.332+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f3224004040 con 0x7f322c108940 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.332+0000 7f3228ff9640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 0x7f32140b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.332+0000 7f322b7fe640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 0x7f32140b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.332+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f32241004d0 con 0x7f322c108940 2024-04-25T13:42:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.333+0000 7f322b7fe640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 0x7f32140b2090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.336+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32240c8fc0 con 0x7f322c108940 2024-04-25T13:42:19.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.552+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c02dc10 con 0x7f322c108940 2024-04-25T13:42:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.553+0000 7f3228ff9640 1 -- 172.21.15.94:0/2094554186 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7f32240c88c0 con 0x7f322c108940 2024-04-25T13:42:19.553 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:42:19.553 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:42:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 msgr2=0x7f32140b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 0x7f32140b2090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b410 comp rx=0 tx=0).stop 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 msgr2=0x7f322c1a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32240097b0 tx=0x7f3224031840 comp rx=0 tx=0).stop 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 shutdown_connections 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32140afb80 0x7f32140b2090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c108940 0x7f322c1a14a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c103c40 0x7f322c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 --2- 172.21.15.94:0/2094554186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c102a00 0x7f322c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 >> 172.21.15.94:0/2094554186 conn(0x7f322c0fdbf0 msgr2=0x7f322c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 shutdown_connections 2024-04-25T13:42:19.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.555+0000 7f3232068640 1 -- 172.21.15.94:0/2094554186 wait complete. 2024-04-25T13:42:19.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 -- 172.21.15.94:0/2080557883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c104680 msgr2=0x7fd41c10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 --2- 172.21.15.94:0/2080557883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c104680 0x7fd41c10add0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd414009860 tx=0x7fd41402a340 comp rx=0 tx=0).stop 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 -- 172.21.15.94:0/2080557883 shutdown_connections 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 --2- 172.21.15.94:0/2080557883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c104680 0x7fd41c10add0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 --2- 172.21.15.94:0/2080557883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 0x7fd41c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 --2- 172.21.15.94:0/2080557883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c102fc0 0x7fd41c1033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 -- 172.21.15.94:0/2080557883 >> 172.21.15.94:0/2080557883 conn(0x7fd41c0fd340 msgr2=0x7fd41c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 -- 172.21.15.94:0/2080557883 shutdown_connections 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 -- 172.21.15.94:0/2080557883 wait complete. 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.677+0000 7fd4243e3640 1 Processor -- start 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 -- start start 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 0x7fd41c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c19f2b0 0x7fd41c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c10e750 con 0x7fd41c102fc0 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c10e5d0 con 0x7fd41c103b20 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c10e8d0 con 0x7fd41c19f2b0 2024-04-25T13:42:19.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54206/0 (socket says 172.21.15.94:54206) 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 -- 172.21.15.94:0/516242174 learned_addr learned my addr 172.21.15.94:0/516242174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd422980640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c19f2b0 0x7fd41c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42197e640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 0x7fd41c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 -- 172.21.15.94:0/516242174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c19f2b0 msgr2=0x7fd41c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c19f2b0 0x7fd41c1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 -- 172.21.15.94:0/516242174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 msgr2=0x7fd41c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 0x7fd41c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 -- 172.21.15.94:0/516242174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1a1f10 con 0x7fd41c102fc0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd42217f640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd41800ea50 tx=0x7fd41800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4137fe640 1 -- 172.21.15.94:0/516242174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41800cdc0 con 0x7fd41c102fc0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4137fe640 1 -- 172.21.15.94:0/516242174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd418005020 con 0x7fd41c102fc0 2024-04-25T13:42:19.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c1a2200 con 0x7fd41c102fc0 2024-04-25T13:42:19.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.678+0000 7fd4137fe640 1 -- 172.21.15.94:0/516242174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180116b0 con 0x7fd41c102fc0 2024-04-25T13:42:19.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.679+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1a2700 con 0x7fd41c102fc0 2024-04-25T13:42:19.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.681+0000 7fd42b36b740 1 Processor -- start 2024-04-25T13:42:19.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 -- start start 2024-04-25T13:42:19.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef094d0 0x55a81f2d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd4137fe640 1 -- 172.21.15.94:0/516242174 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fd4180051d0 con 0x7fd41c102fc0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a81f238d80 con 0x55a81ef32390 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a81f1f2a20 con 0x55a81ef094d0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42b36b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a81f234d40 con 0x55a81ef116b0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54220/0 (socket says 172.21.15.94:54220) 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 -- 172.21.15.94:0/3826113012 learned_addr learned my addr 172.21.15.94:0/3826113012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42197e640 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 -- 172.21.15.94:0/3826113012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 msgr2=0x55a81f2d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd42217f640 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef094d0 0x55a81f2d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 -- 172.21.15.94:0/3826113012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef094d0 msgr2=0x55a81f2d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef094d0 0x55a81f2d4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.682+0000 7fd422980640 1 -- 172.21.15.94:0/3826113012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81f1fdc30 con 0x55a81ef32390 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd422980640 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd414005da0 tx=0x7fd414005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95b88c8ab77d48c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd4137fe640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 0x7fd4080b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd410bfe640 1 -- 172.21.15.94:0/3826113012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414037070 con 0x55a81ef32390 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd410bfe640 1 -- 172.21.15.94:0/3826113012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd414038040 con 0x55a81ef32390 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd410bfe640 1 -- 172.21.15.94:0/3826113012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4140052a0 con 0x55a81ef32390 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd42217f640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 0x7fd4080b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.683+0000 7fd4137fe640 1 -- 172.21.15.94:0/516242174 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd4180df1e0 con 0x7fd41c102fc0 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- 172.21.15.94:0/3826113012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 msgr2=0x55a81f2d4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42217f640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 0x7fd4080b2090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a81f2d4530 tx=0x7fd41800cf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd414005da0 tx=0x7fd414005dd0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- 172.21.15.94:0/3826113012 shutdown_connections 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef094d0 0x55a81f2d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3826113012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef32390 0x55a81f2d4810 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- 172.21.15.94:0/3826113012 >> 172.21.15.94:0/3826113012 conn(0x55a81ef264e0 msgr2=0x55a81f2d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- 172.21.15.94:0/3826113012 shutdown_connections 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- 172.21.15.94:0/3826113012 wait complete. 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 -- start start 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.684+0000 7fd42b36b740 1 Processor -- start 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef32390 0x55a81f2e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a81f2d64d0 con 0x55a81ef094d0 2024-04-25T13:42:19.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a81f1bfdc0 con 0x55a81ef32390 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a81f2d6650 con 0x55a81ef116b0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 10 client.0 _ll_register_callbacks cb 0x55a81f1db600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54226/0 (socket says 172.21.15.94:54226) 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 -- 172.21.15.94:0/2452483152 learned_addr learned my addr 172.21.15.94:0/2452483152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef32390 0x55a81f2e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42197e640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 -- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 msgr2=0x55a81f2dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 -- 172.21.15.94:0/2452483152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef32390 msgr2=0x55a81f2e59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef32390 0x55a81f2e59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 -- 172.21.15.94:0/2452483152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81ef4a3f0 con 0x55a81ef094d0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd422980640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd41402a860 tx=0x7fd414005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fccfbb85611f82b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41404d020 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd4140046c0 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.685+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd414038040 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd414058050 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a81f0147d0 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd414038040 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd414045780 con 0x55a81ef094d0 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 10 client.15582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 20 client.15582 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81eef0f60 0x55a81f2ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 4 client.15582 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a81f246990 con 0x55a81eef0f60 2024-04-25T13:42:19.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.686+0000 7fd42197e640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81eef0f60 0x55a81f2ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.687+0000 7fd405ffb640 10 client.15582 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:19.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.687+0000 7fd42197e640 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81eef0f60 0x55a81f2ebe20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccfceb932dabb43f server_cookie=d17e0a478db4f68b in_seq=0 out_seq=0 2024-04-25T13:42:19.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd405ffb640 1 -- 172.21.15.94:0/2452483152 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a81f246990 con 0x55a81eef0f60 2024-04-25T13:42:19.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd405ffb640 10 client.15582 handle_command_reply: tid=0 2024-04-25T13:42:19.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 client.15582 shutdown 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 msgr2=0x55a81f2de530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd41402a860 tx=0x7fd414005e70 comp rx=0 tx=0).stop 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 shutdown_connections 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81eef0f60 0x55a81f2ebe20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef32390 0x55a81f2e59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd405ffb640 0 client.15582 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef116b0 0x55a81f2dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2452483152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2de530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 >> 172.21.15.94:0/2452483152 conn(0x55a81ef264e0 msgr2=0x55a81f2cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 shutdown_connections 2024-04-25T13:42:19.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 1 -- 172.21.15.94:0/2452483152 wait complete. 2024-04-25T13:42:19.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.688+0000 7fd42b36b740 20 client.15582 trim_cache size 0 max 16384 2024-04-25T13:42:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.701+0000 7fd42b36b740 1 Processor -- start 2024-04-25T13:42:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.701+0000 7fd42b36b740 1 -- start start 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81eef0f60 0x55a81f2d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef094d0 0x55a81f2e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1ab460 con 0x55a81ef116b0 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1acfa0 con 0x55a81eef0f60 2024-04-25T13:42:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42b36b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1abb30 con 0x55a81ef094d0 2024-04-25T13:42:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54236/0 (socket says 172.21.15.94:54236) 2024-04-25T13:42:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 -- 172.21.15.94:0/2480699012 learned_addr learned my addr 172.21.15.94:0/2480699012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd42197e640 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef094d0 0x55a81f2e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 -- 172.21.15.94:0/2480699012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef094d0 msgr2=0x55a81f2e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef094d0 0x55a81f2e5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 -- 172.21.15.94:0/2480699012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81eef0f60 msgr2=0x55a81f2d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81eef0f60 0x55a81f2d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 -- 172.21.15.94:0/2480699012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81f2cb7f0 con 0x55a81ef116b0 2024-04-25T13:42:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd422980640 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4140304b0 tx=0x7fd414000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdf9732c302339b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd407fff640 1 -- 172.21.15.94:0/2480699012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41404d040 con 0x55a81ef116b0 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd407fff640 1 -- 172.21.15.94:0/2480699012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd414030dc0 con 0x55a81ef116b0 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.702+0000 7fd407fff640 1 -- 172.21.15.94:0/2480699012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414037070 con 0x55a81ef116b0 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- 172.21.15.94:0/2480699012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 msgr2=0x55a81f2f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4140304b0 tx=0x7fd414000bc0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- 172.21.15.94:0/2480699012 shutdown_connections 2024-04-25T13:42:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef116b0 0x55a81f2f14b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81ef094d0 0x55a81f2e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 --2- 172.21.15.94:0/2480699012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81eef0f60 0x55a81f2d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- 172.21.15.94:0/2480699012 >> 172.21.15.94:0/2480699012 conn(0x55a81ef264e0 msgr2=0x55a81f2dccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- 172.21.15.94:0/2480699012 shutdown_connections 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- 172.21.15.94:0/2480699012 wait complete. 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.703+0000 7fd42b36b740 1 -- start start 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 Processor -- start 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81eef0f60 0x55a81f2e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 0x55a81f2e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1ac8d0 con 0x55a81ef094d0 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1ac200 con 0x55a81ef116b0 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1a9440 con 0x55a81eef0f60 2024-04-25T13:42:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 10 client.0 _ll_register_callbacks cb 0x7fd40c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54242/0 (socket says 172.21.15.94:54242) 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 -- 172.21.15.94:0/3508736175 learned_addr learned my addr 172.21.15.94:0/3508736175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd422980640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81eef0f60 0x55a81f2e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd422980640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 0x55a81f2e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 -- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81eef0f60 msgr2=0x55a81f2e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81eef0f60 0x55a81f2e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 -- 172.21.15.94:0/3508736175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 msgr2=0x55a81f2e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 0x55a81f2e4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 -- 172.21.15.94:0/3508736175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81f2e4930 con 0x55a81ef094d0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd422980640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 0x55a81f2e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42197e640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd40c004700 tx=0x7fd40c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a9cc8304bc81d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01e800 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd40c01e9a0 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.704+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd414030dc0 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd40c012e20 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a81f2e5cb0 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd414030dc0 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd40c00f040 con 0x55a81ef094d0 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 10 client.15588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 20 client.15588 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81ef32390 0x55a81f3566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 4 client.15588 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a81f240800 con 0x55a81ef32390 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.705+0000 7fd422980640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81ef32390 0x55a81f3566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd405ffb640 10 client.15588 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd422980640 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81ef32390 0x55a81f3566f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=702ebe65d8f2f6cd server_cookie=2dbcc5a8d1b75011 in_seq=0 out_seq=0 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd405ffb640 1 -- 172.21.15.94:0/3508736175 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a81f240800 con 0x55a81ef32390 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd405ffb640 10 client.15588 handle_command_reply: tid=0 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 client.15588 shutdown 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 msgr2=0x55a81f2e3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd40c004700 tx=0x7fd40c012820 comp rx=0 tx=0).stop 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 shutdown_connections 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a81ef32390 0x55a81f3566f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a81ef116b0 0x55a81f2e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd405ffb640 0 client.15588 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a81ef094d0 0x55a81f2e3490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 --2- 172.21.15.94:0/3508736175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a81eef0f60 0x55a81f2e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 >> 172.21.15.94:0/3508736175 conn(0x55a81ef264e0 msgr2=0x55a81f2d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 shutdown_connections 2024-04-25T13:42:19.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 1 -- 172.21.15.94:0/3508736175 wait complete. 2024-04-25T13:42:19.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.706+0000 7fd42b36b740 20 client.15588 trim_cache size 0 max 16384 2024-04-25T13:42:19.724 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 msgr2=0x7fd4080b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 0x7fd4080b2090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a81f2d4530 tx=0x7fd41800cf60 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 msgr2=0x7fd41c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd41800ea50 tx=0x7fd41800ef10 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 shutdown_connections 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4080afb80 0x7fd4080b2090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd41c19f2b0 0x7fd41c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd41c103b20 0x7fd41c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 --2- 172.21.15.94:0/516242174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd41c102fc0 0x7fd41c102280 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 >> 172.21.15.94:0/516242174 conn(0x7fd41c0fd340 msgr2=0x7fd41c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 shutdown_connections 2024-04-25T13:42:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.707+0000 7fd4243e3640 1 -- 172.21.15.94:0/516242174 wait complete. 2024-04-25T13:42:19.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:42:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- 172.21.15.94:0/3532284730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a8101760 msgr2=0x7fa4a8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 --2- 172.21.15.94:0/3532284730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a8101760 0x7fa4a8101bd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009900 tx=0x7fa4a402a410 comp rx=0 tx=0).stop 2024-04-25T13:42:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- 172.21.15.94:0/3532284730 shutdown_connections 2024-04-25T13:42:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 --2- 172.21.15.94:0/3532284730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 0x7fa4a810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 --2- 172.21.15.94:0/3532284730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a8101760 0x7fa4a8101bd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 --2- 172.21.15.94:0/3532284730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a81050b0 0x7fa4a81054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- 172.21.15.94:0/3532284730 >> 172.21.15.94:0/3532284730 conn(0x7fa4a80fd380 msgr2=0x7fa4a80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- 172.21.15.94:0/3532284730 shutdown_connections 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- 172.21.15.94:0/3532284730 wait complete. 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 Processor -- start 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.836+0000 7fa4afe25640 1 -- start start 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 0x7fa4a819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 0x7fa4a819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810f690 con 0x7fa4a81050b0 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810f510 con 0x7fa4a8102260 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a810f810 con 0x7fa4a8101760 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54262/0 (socket says 172.21.15.94:54262) 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 -- 172.21.15.94:0/1017530721 learned_addr learned my addr 172.21.15.94:0/1017530721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 0x7fa4a819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 0x7fa4a819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 -- 172.21.15.94:0/1017530721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 msgr2=0x7fa4a819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 0x7fa4a819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 -- 172.21.15.94:0/1017530721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 msgr2=0x7fa4a819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 0x7fa4a819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 -- 172.21.15.94:0/1017530721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a81a1de0 con 0x7fa4a81050b0 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 0x7fa4a819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4ae3c2640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b970 tx=0x7fa4a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa49effd640 1 -- 172.21.15.94:0/1017530721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0019070 con 0x7fa4a81050b0 2024-04-25T13:42:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa49effd640 1 -- 172.21.15.94:0/1017530721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a0014440 con 0x7fa4a81050b0 2024-04-25T13:42:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a81a20d0 con 0x7fa4a81050b0 2024-04-25T13:42:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa49effd640 1 -- 172.21.15.94:0/1017530721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00075d0 con 0x7fa4a81050b0 2024-04-25T13:42:19.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.837+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a2650 con 0x7fa4a81050b0 2024-04-25T13:42:19.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.840+0000 7fa4b6daa740 1 Processor -- start 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.840+0000 7fa49effd640 1 -- 172.21.15.94:0/1017530721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fa4a001c070 con 0x7fa4a81050b0 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.840+0000 7fa4b6daa740 1 -- start start 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e8810810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e844d6b0 0x5556e8811a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5556e8774d80 con 0x5556e84454d0 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556e872ea20 con 0x5556e846e390 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556e8770d40 con 0x5556e844d6b0 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54274/0 (socket says 172.21.15.94:54274) 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 -- 172.21.15.94:0/3495298175 learned_addr learned my addr 172.21.15.94:0/3495298175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa49effd640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 0x7fa4940b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e844d6b0 0x5556e8811a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e8810810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4ae3c2640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 0x7fa4940b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 -- 172.21.15.94:0/3495298175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e844d6b0 msgr2=0x5556e8811a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e844d6b0 0x5556e8811a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 -- 172.21.15.94:0/3495298175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 msgr2=0x5556e8810810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e8810810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 -- 172.21.15.94:0/3495298175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e8739c30 con 0x5556e84454d0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e8810810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa49effd640 1 -- 172.21.15.94:0/1017530721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa4a00de2d0 con 0x7fa4a81050b0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.841+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa49800bce0 tx=0x7fa49800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba51933119e9ad1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa493fff640 1 -- 172.21.15.94:0/3495298175 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498015070 con 0x5556e84454d0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa4ae3c2640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 0x7fa4940b2090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00a6470 tx=0x7fa4a008f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa493fff640 1 -- 172.21.15.94:0/3495298175 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4980118f0 con 0x5556e84454d0 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa493fff640 1 -- 172.21.15.94:0/3495298175 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498010870 con 0x5556e84454d0 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa4b6daa740 1 -- 172.21.15.94:0/3495298175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 msgr2=0x5556e8810d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.842+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa49800bce0 tx=0x7fa49800f730 comp rx=0 tx=0).stop 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 -- 172.21.15.94:0/3495298175 shutdown_connections 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e844d6b0 0x5556e8811a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e84454d0 0x5556e8810d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/3495298175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e8810810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 -- 172.21.15.94:0/3495298175 >> 172.21.15.94:0/3495298175 conn(0x5556e84624e0 msgr2=0x5556e880c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 -- 172.21.15.94:0/3495298175 shutdown_connections 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 -- 172.21.15.94:0/3495298175 wait complete. 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 -- start start 2024-04-25T13:42:19.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 Processor -- start 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 0x5556e881a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.843+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e846e390 0x5556e8821a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5556e8812530 con 0x5556e844d6b0 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556e86fbdc0 con 0x5556e84454d0 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556e88126b0 con 0x5556e846e390 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 10 client.0 _ll_register_callbacks cb 0x5556e8717600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54284/0 (socket says 172.21.15.94:54284) 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 -- 172.21.15.94:0/905745611 learned_addr learned my addr 172.21.15.94:0/905745611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 0x5556e881a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 -- 172.21.15.94:0/905745611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e846e390 msgr2=0x5556e8821a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e846e390 0x5556e8821a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 -- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 msgr2=0x5556e881a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 0x5556e881a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 -- 172.21.15.94:0/905745611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e84863f0 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 0x5556e881a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa498011670 tx=0x7fa4980116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a174a3f5dba0b844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4917fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498028060 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa498011cb0 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980118f0 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa498015050 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.844+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556e85507d0 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4980118f0 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa498004b00 con 0x5556e844d6b0 2024-04-25T13:42:19.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 10 client.15600 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 20 client.15600 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e842cf60 0x5556e8827f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 4 client.15600 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556e8782990 con 0x5556e842cf60 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.845+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e842cf60 0x5556e8827f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.846+0000 7fa4917fa640 10 client.15600 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.846+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e842cf60 0x5556e8827f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24794ec01e5dac36 server_cookie=dece89d99841dc0d in_seq=0 out_seq=0 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.846+0000 7fa4917fa640 1 -- 172.21.15.94:0/905745611 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556e8782990 con 0x5556e842cf60 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.846+0000 7fa4917fa640 10 client.15600 handle_command_reply: tid=0 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 client.15600 shutdown 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 msgr2=0x5556e881ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa498011670 tx=0x7fa4980116a0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 shutdown_connections 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e842cf60 0x5556e8827f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e846e390 0x5556e8821a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881ab00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/905745611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e84454d0 0x5556e881a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4917fa640 0 client.15600 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 >> 172.21.15.94:0/905745611 conn(0x5556e84624e0 msgr2=0x5556e880ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 shutdown_connections 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 1 -- 172.21.15.94:0/905745611 wait complete. 2024-04-25T13:42:19.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.847+0000 7fa4b6daa740 20 client.15600 trim_cache size 0 max 16384 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 Processor -- start 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 -- start start 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e880ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 0x5556e880bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81ab600 con 0x5556e846e390 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81ad140 con 0x5556e844d6b0 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81abcd0 con 0x5556e84454d0 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54290/0 (socket says 172.21.15.94:54290) 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 -- 172.21.15.94:0/1535840774 learned_addr learned my addr 172.21.15.94:0/1535840774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 -- 172.21.15.94:0/1535840774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 msgr2=0x5556e880ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e880ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 -- 172.21.15.94:0/1535840774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 msgr2=0x5556e880bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 0x5556e880bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 0x5556e880bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 -- 172.21.15.94:0/1535840774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e8814dc0 con 0x5556e846e390 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 0x5556e880bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000c00 tx=0x7fa4a4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9588b142ac3aeb4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.861+0000 7fa4937fe640 1 -- 172.21.15.94:0/1535840774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4035040 con 0x5556e846e390 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4937fe640 1 -- 172.21.15.94:0/1535840774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a404a070 con 0x5556e846e390 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4937fe640 1 -- 172.21.15.94:0/1535840774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4003850 con 0x5556e846e390 2024-04-25T13:42:19.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1535840774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 msgr2=0x5556e8818cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000c00 tx=0x7fa4a4004040 comp rx=0 tx=0).stop 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1535840774 shutdown_connections 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e846e390 0x5556e8818cb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e844d6b0 0x5556e880bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1535840774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e880ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1535840774 >> 172.21.15.94:0/1535840774 conn(0x5556e84624e0 msgr2=0x5556e8815310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1535840774 shutdown_connections 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.862+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1535840774 wait complete. 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 -- start start 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 Processor -- start 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e881e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e881f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81aca70 con 0x5556e844d6b0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81ac3a0 con 0x5556e846e390 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81a95e0 con 0x5556e84454d0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4b6daa740 10 client.0 _ll_register_callbacks cb 0x7fa49800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4adbc1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54298/0 (socket says 172.21.15.94:54298) 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4adbc1640 1 -- 172.21.15.94:0/1217366967 learned_addr learned my addr 172.21.15.94:0/1217366967 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.863+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e881e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 -- 172.21.15.94:0/1217366967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 msgr2=0x5556e881e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e881e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e881f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 -- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 msgr2=0x5556e881f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e881f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 -- 172.21.15.94:0/1217366967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e881fca0 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e881f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4adbc1640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa498001eb0 tx=0x7fa49800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf5bb3393d68706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4917fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498028070 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa49800b2c0 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a404a070 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa49803d050 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4b6daa740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556e8820ec0 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.864+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a404a070 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa498004a30 con 0x5556e844d6b0 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 10 client.15606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:42:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 20 client.15606 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e84023a0 0x5556e8892560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 4 client.15606 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556e877c800 con 0x5556e84023a0 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e84023a0 0x5556e8892560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4917fa640 10 client.15606 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4ad3c0640 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e84023a0 0x5556e8892560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f25c3b0b40e3be90 server_cookie=787851946488563e in_seq=0 out_seq=0 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4917fa640 1 -- 172.21.15.94:0/1217366967 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556e877c800 con 0x5556e84023a0 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.865+0000 7fa4917fa640 10 client.15606 handle_command_reply: tid=0 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 client.15606 shutdown 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 msgr2=0x5556e881e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa498001eb0 tx=0x7fa49800b960 comp rx=0 tx=0).stop 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 shutdown_connections 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5556e84023a0 0x5556e8892560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556e846e390 0x5556e881f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556e844d6b0 0x5556e881e7b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4917fa640 0 client.15606 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 --2- 172.21.15.94:0/1217366967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556e84454d0 0x5556e881e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 >> 172.21.15.94:0/1217366967 conn(0x5556e84624e0 msgr2=0x5556e8818cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 shutdown_connections 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 1 -- 172.21.15.94:0/1217366967 wait complete. 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4b6daa740 20 client.15606 trim_cache size 0 max 16384 2024-04-25T13:42:19.873 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 msgr2=0x7fa4940b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 0x7fa4940b2090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00a6470 tx=0x7fa4a008f040 comp rx=0 tx=0).stop 2024-04-25T13:42:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 msgr2=0x7fa4a81a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b970 tx=0x7fa4a000be30 comp rx=0 tx=0).stop 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 shutdown_connections 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4940afb80 0x7fa4940b2090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4a81050b0 0x7fa4a81a1770 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a8102260 0x7fa4a819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 --2- 172.21.15.94:0/1017530721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a8101760 0x7fa4a819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.866+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 >> 172.21.15.94:0/1017530721 conn(0x7fa4a80fd380 msgr2=0x7fa4a8105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.867+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 shutdown_connections 2024-04-25T13:42:19.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.867+0000 7fa4afe25640 1 -- 172.21.15.94:0/1017530721 wait complete. 2024-04-25T13:42:19.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/1460547187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8103af0 msgr2=0x7f4fd81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/1460547187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8103af0 0x7f4fd81047f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009ca0 tx=0x7f4fd002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/1460547187 shutdown_connections 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/1460547187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8109a30 0x7f4fd810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/1460547187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8104d30 0x7f4fd8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/1460547187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8103af0 0x7f4fd81047f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/1460547187 >> 172.21.15.94:0/1460547187 conn(0x7f4fd80fece0 msgr2=0x7f4fd8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/1460547187 shutdown_connections 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.987+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/1460547187 wait complete. 2024-04-25T13:42:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 Processor -- start 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 -- start start 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8103af0 0x7f4fd819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 0x7f4fd819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810f5d0 con 0x7f4fd8109a30 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810f450 con 0x7f4fd8103af0 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdf2c6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810f750 con 0x7f4fd8104d30 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54324/0 (socket says 172.21.15.94:54324) 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 -- 172.21.15.94:0/506183244 learned_addr learned my addr 172.21.15.94:0/506183244 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd062640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8103af0 0x7f4fd819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdc861640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 0x7f4fd819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 -- 172.21.15.94:0/506183244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 msgr2=0x7f4fd819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 0x7f4fd819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 -- 172.21.15.94:0/506183244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8103af0 msgr2=0x7f4fd819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8103af0 0x7f4fd819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdd863640 1 -- 172.21.15.94:0/506183244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81a1cd0 con 0x7f4fd8109a30 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.988+0000 7f4fdc861640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 0x7f4fd819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fdd863640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c970 tx=0x7f4fd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fce7fc640 1 -- 172.21.15.94:0/506183244 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007bb0 con 0x7f4fd8109a30 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fce7fc640 1 -- 172.21.15.94:0/506183244 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd4007d50 con 0x7f4fd8109a30 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a1fc0 con 0x7f4fd8109a30 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fce7fc640 1 -- 172.21.15.94:0/506183244 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4018770 con 0x7f4fd8109a30 2024-04-25T13:42:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.989+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a23d0 con 0x7f4fd8109a30 2024-04-25T13:42:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fce7fc640 1 -- 172.21.15.94:0/506183244 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f4fd400f990 con 0x7f4fd8109a30 2024-04-25T13:42:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 Processor -- start 2024-04-25T13:42:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 -- start start 2024-04-25T13:42:20.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 0x557612280810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612281a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5576121e4d80 con 0x557611eb54d0 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55761219ea20 con 0x557611ede390 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fe6177740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576121e0d40 con 0x557611ebd6b0 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54336/0 (socket says 172.21.15.94:54336) 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 -- 172.21.15.94:0/2872686142 learned_addr learned my addr 172.21.15.94:0/2872686142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 0x557612280810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612281a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fce7fc640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 0x7f4fc40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 -- 172.21.15.94:0/2872686142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 msgr2=0x557612281a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612281a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 -- 172.21.15.94:0/2872686142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 msgr2=0x557612280810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 0x557612280810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd062640 1 -- 172.21.15.94:0/2872686142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576121a9c30 con 0x557611eb54d0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 0x557612280810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.992+0000 7f4fdd863640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 0x7f4fc40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.993+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81033c0 tx=0x7f4fd00098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63acfd33d59439df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.993+0000 7f4fce7fc640 1 -- 172.21.15.94:0/506183244 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4fd40def00 con 0x7f4fd8109a30 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.993+0000 7f4fc37fe640 1 -- 172.21.15.94:0/2872686142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0031790 con 0x557611eb54d0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.993+0000 7f4fc37fe640 1 -- 172.21.15.94:0/2872686142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0036030 con 0x557611eb54d0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.993+0000 7f4fc37fe640 1 -- 172.21.15.94:0/2872686142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd002f660 con 0x557611eb54d0 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- 172.21.15.94:0/2872686142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 msgr2=0x557612280d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fdd863640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 0x7f4fc40b2090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4094dc0 tx=0x7f4fd4074a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81033c0 tx=0x7f4fd00098b0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- 172.21.15.94:0/2872686142 shutdown_connections 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612281a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612280d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2872686142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611ede390 0x557612280810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- 172.21.15.94:0/2872686142 >> 172.21.15.94:0/2872686142 conn(0x557611ed24e0 msgr2=0x55761227c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- 172.21.15.94:0/2872686142 shutdown_connections 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- 172.21.15.94:0/2872686142 wait complete. 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 -- start start 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.994+0000 7f4fe6177740 1 Processor -- start 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611eb54d0 0x55761228a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x55761228a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557612282350 con 0x557611ede390 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55761216bdc0 con 0x557611eb54d0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576122824d0 con 0x557611ebd6b0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fe6177740 10 client.0 _ll_register_callbacks cb 0x557612187600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54350/0 (socket says 172.21.15.94:54350) 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 -- 172.21.15.94:0/2072585550 learned_addr learned my addr 172.21.15.94:0/2072585550 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611eb54d0 0x55761228a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x55761228a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 -- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 msgr2=0x55761228a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x55761228a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 -- 172.21.15.94:0/2072585550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611eb54d0 msgr2=0x55761228a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611eb54d0 0x55761228a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 -- 172.21.15.94:0/2072585550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557611ef63f0 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fdc861640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8012e00 tx=0x7f4fc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e077474fed8044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.995+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8004020 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4fc80041c0 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd0036030 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4fc8027050 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557611fc07d0 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0036030 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4fc801f3a0 con 0x557611ede390 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 10 client.15618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 20 client.15618 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611e9cf60 0x5576122980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 4 client.15618 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576121f2990 con 0x557611e9cf60 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.996+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611e9cf60 0x5576122980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.997+0000 7f4fc0ff9640 10 client.15618 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.997+0000 7f4fdd062640 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611e9cf60 0x5576122980f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84adb0963b9420ff server_cookie=52032fa6564f545a in_seq=0 out_seq=0 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/2072585550 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576121f2990 con 0x557611e9cf60 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fc0ff9640 10 client.15618 handle_command_reply: tid=0 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 client.15618 shutdown 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 msgr2=0x557612291870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8012e00 tx=0x7f4fc8004730 comp rx=0 tx=0).stop 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 shutdown_connections 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611e9cf60 0x5576122980f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611ede390 0x557612291870 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x55761228a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fc0ff9640 0 client.15618 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 --2- 172.21.15.94:0/2072585550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611eb54d0 0x55761228a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 >> 172.21.15.94:0/2072585550 conn(0x557611ed24e0 msgr2=0x55761227ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 shutdown_connections 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 1 -- 172.21.15.94:0/2072585550 wait complete. 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:19.998+0000 7f4fe6177740 20 client.15618 trim_cache size 0 max 16384 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.011+0000 7f4fe6177740 1 Processor -- start 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.011+0000 7f4fe6177740 1 -- start start 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x5576122889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612288f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ab330 con 0x557611eb54d0 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ace70 con 0x557611e9cf60 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fe6177740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81aba00 con 0x557611ebd6b0 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54364/0 (socket says 172.21.15.94:54364) 2024-04-25T13:42:20.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 -- 172.21.15.94:0/4236688037 learned_addr learned my addr 172.21.15.94:0/4236688037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 -- 172.21.15.94:0/4236688037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 msgr2=0x557612288f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdc861640 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612288f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612288f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 -- 172.21.15.94:0/4236688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 msgr2=0x5576122889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x5576122889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 -- 172.21.15.94:0/4236688037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761228afd0 con 0x557611eb54d0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fdd062640 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4fd002f470 tx=0x7f4fd0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abac8744e93f96f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fc2ffd640 1 -- 172.21.15.94:0/4236688037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd005e070 con 0x557611eb54d0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fc2ffd640 1 -- 172.21.15.94:0/4236688037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0055070 con 0x557611eb54d0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.012+0000 7f4fc2ffd640 1 -- 172.21.15.94:0/4236688037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0056ad0 con 0x557611eb54d0 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- 172.21.15.94:0/4236688037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 msgr2=0x557612291800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4fd002f470 tx=0x7f4fd0000f40 comp rx=0 tx=0).stop 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- 172.21.15.94:0/4236688037 shutdown_connections 2024-04-25T13:42:20.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x557612288f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x557612291800 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 --2- 172.21.15.94:0/4236688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x5576122889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- 172.21.15.94:0/4236688037 >> 172.21.15.94:0/4236688037 conn(0x557611ed24e0 msgr2=0x5576122852b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- 172.21.15.94:0/4236688037 shutdown_connections 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- 172.21.15.94:0/4236688037 wait complete. 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.013+0000 7f4fe6177740 1 -- start start 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 Processor -- start 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x55761228e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x5576122fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ac7a0 con 0x557611eb54d0 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ac0d0 con 0x557611e9cf60 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81a9310 con 0x557611ebd6b0 2024-04-25T13:42:20.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fe6177740 10 client.0 _ll_register_callbacks cb 0x7f4fc800eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54378/0 (socket says 172.21.15.94:54378) 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 -- 172.21.15.94:0/461293600 learned_addr learned my addr 172.21.15.94:0/461293600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdc861640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x55761228e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdc861640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x5576122fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 -- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 msgr2=0x5576122fe440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x5576122fe440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 -- 172.21.15.94:0/461293600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 msgr2=0x55761228e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x55761228e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.014+0000 7f4fdd062640 1 -- 172.21.15.94:0/461293600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557612291a20 con 0x557611eb54d0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fdd062640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005e30 tx=0x7f4fd0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e553f2ccbf8691fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd005e070 con 0x557611eb54d0 2024-04-25T13:42:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4fd0031650 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd0055070 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4fd0056a60 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576122ff890 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0055070 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4fd0043070 con 0x557611eb54d0 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 20 client.15624 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611ede390 0x557612301f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 4 client.15624 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.015+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576121ec800 con 0x557611ede390 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fdc861640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611ede390 0x557612301f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fdc861640 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611ede390 0x557612301f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28f95729efae5d78 server_cookie=68242edd9e729798 in_seq=0 out_seq=0 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fc0ff9640 10 client.15624 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fc0ff9640 1 -- 172.21.15.94:0/461293600 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576121ec800 con 0x557611ede390 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fc0ff9640 10 client.15624 handle_command_reply: tid=0 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 client.15624 shutdown 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 msgr2=0x55761228ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005e30 tx=0x7f4fd0009c60 comp rx=0 tx=0).stop 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 shutdown_connections 2024-04-25T13:42:20.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x557611ede390 0x557612301f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557611ebd6b0 0x5576122fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557611eb54d0 0x55761228ea60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fc0ff9640 0 client.15624 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 --2- 172.21.15.94:0/461293600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557611e9cf60 0x55761228e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.016+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 >> 172.21.15.94:0/461293600 conn(0x557611ed24e0 msgr2=0x557612288880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 shutdown_connections 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fe6177740 1 -- 172.21.15.94:0/461293600 wait complete. 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fe6177740 20 client.15624 trim_cache size 0 max 16384 2024-04-25T13:42:20.024 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 msgr2=0x7f4fc40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 0x7f4fc40b2090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4094dc0 tx=0x7f4fd4074a90 comp rx=0 tx=0).stop 2024-04-25T13:42:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 msgr2=0x7f4fd81a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c970 tx=0x7f4fd400ce30 comp rx=0 tx=0).stop 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 shutdown_connections 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc40afb80 0x7f4fc40b2090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8109a30 0x7f4fd81a1710 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8104d30 0x7f4fd819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 --2- 172.21.15.94:0/506183244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8103af0 0x7f4fd819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 >> 172.21.15.94:0/506183244 conn(0x7f4fd80fece0 msgr2=0x7f4fd8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 shutdown_connections 2024-04-25T13:42:20.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.017+0000 7f4fdf2c6640 1 -- 172.21.15.94:0/506183244 wait complete. 2024-04-25T13:42:20.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:42:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.138+0000 7f1e680af640 1 -- 172.21.15.94:0/2677570691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60103c20 msgr2=0x7f1e60104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.138+0000 7f1e680af640 1 --2- 172.21.15.94:0/2677570691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60103c20 0x7f1e60104030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009900 tx=0x7f1e5002a410 comp rx=0 tx=0).stop 2024-04-25T13:42:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- 172.21.15.94:0/2677570691 shutdown_connections 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- 172.21.15.94:0/2677570691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e60108920 0x7f1e6010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- 172.21.15.94:0/2677570691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60103c20 0x7f1e60104030 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- 172.21.15.94:0/2677570691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e601029e0 0x7f1e601036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- 172.21.15.94:0/2677570691 >> 172.21.15.94:0/2677570691 conn(0x7f1e600fdbd0 msgr2=0x7f1e60100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- 172.21.15.94:0/2677570691 shutdown_connections 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- 172.21.15.94:0/2677570691 wait complete. 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 Processor -- start 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- start start 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 0x7f1e60199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 0x7f1e601999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010e500 con 0x7f1e60108920 2024-04-25T13:42:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010e380 con 0x7f1e601029e0 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.139+0000 7f1e680af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010e680 con 0x7f1e60103c20 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54398/0 (socket says 172.21.15.94:54398) 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 -- 172.21.15.94:0/1064265407 learned_addr learned my addr 172.21.15.94:0/1064265407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 0x7f1e60199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 -- 172.21.15.94:0/1064265407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 msgr2=0x7f1e601999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6564a640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 0x7f1e601999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 0x7f1e601999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 -- 172.21.15.94:0/1064265407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 msgr2=0x7f1e60199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 0x7f1e60199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 -- 172.21.15.94:0/1064265407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e601a0fa0 con 0x7f1e60108920 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 0x7f1e60199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6564a640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 0x7f1e601999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e6664c640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c970 tx=0x7f1e5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e56ffd640 1 -- 172.21.15.94:0/1064265407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58007bb0 con 0x7f1e60108920 2024-04-25T13:42:20.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e601a1290 con 0x7f1e60108920 2024-04-25T13:42:20.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e56ffd640 1 -- 172.21.15.94:0/1064265407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e58007d50 con 0x7f1e60108920 2024-04-25T13:42:20.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e56ffd640 1 -- 172.21.15.94:0/1064265407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58004e60 con 0x7f1e60108920 2024-04-25T13:42:20.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.140+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e601a1790 con 0x7f1e60108920 2024-04-25T13:42:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 Processor -- start 2024-04-25T13:42:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e56ffd640 1 -- 172.21.15.94:0/1064265407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f1e58005020 con 0x7f1e60108920 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 -- start start 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff049390 0x55cbff3eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cbff34fd80 con 0x55cbff0286b0 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbff309a20 con 0x55cbff049390 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbff34bd40 con 0x55cbff0204d0 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54400/0 (socket says 172.21.15.94:54400) 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e6564a640 1 -- 172.21.15.94:0/1392574144 learned_addr learned my addr 172.21.15.94:0/1392574144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.143+0000 7f1e56ffd640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 0x7f1e4c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 -- 172.21.15.94:0/1392574144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 msgr2=0x55cbff3ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 -- 172.21.15.94:0/1392574144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff049390 msgr2=0x55cbff3eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff049390 0x55cbff3eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6664c640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 0x7f1e4c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 -- 172.21.15.94:0/1392574144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbff314c30 con 0x55cbff0286b0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e56ffd640 1 -- 172.21.15.94:0/1064265407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1e580dec30 con 0x7f1e60108920 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e6564a640 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005f60 tx=0x7f1e50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f30810e962bac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e4bfff640 1 -- 172.21.15.94:0/1392574144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50041070 con 0x55cbff0286b0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e4bfff640 1 -- 172.21.15.94:0/1392574144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e50032e10 con 0x55cbff0286b0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.144+0000 7f1e4bfff640 1 -- 172.21.15.94:0/1392574144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50035040 con 0x55cbff0286b0 2024-04-25T13:42:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- 172.21.15.94:0/1392574144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 msgr2=0x55cbff3eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6664c640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 0x7f1e4c0b2090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e580a6c10 tx=0x7f1e58075a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005f60 tx=0x7f1e50000bc0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- 172.21.15.94:0/1392574144 shutdown_connections 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0286b0 0x55cbff3eca20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/1392574144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff049390 0x55cbff3eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- 172.21.15.94:0/1392574144 >> 172.21.15.94:0/1392574144 conn(0x55cbff03d4e0 msgr2=0x55cbff3e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- 172.21.15.94:0/1392574144 shutdown_connections 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- 172.21.15.94:0/1392574144 wait complete. 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 -- start start 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.145+0000 7f1e6ef63740 1 Processor -- start 2024-04-25T13:42:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cbff3ed4d0 con 0x55cbff049390 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbff2d6dc0 con 0x55cbff0286b0 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbff3ed650 con 0x55cbff0204d0 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6ef63740 10 client.0 _ll_register_callbacks cb 0x55cbff2f2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54416/0 (socket says 172.21.15.94:54416) 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 -- 172.21.15.94:0/4154786897 learned_addr learned my addr 172.21.15.94:0/4154786897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 -- 172.21.15.94:0/4154786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 msgr2=0x55cbff3f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 -- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 msgr2=0x55cbff3f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 -- 172.21.15.94:0/4154786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbff0613f0 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e6564a640 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e500056f0 tx=0x7f1e500319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6d7d2ab4b5fce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.146+0000 7f1e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50000e60 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e50032e10 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1e50004460 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e50057050 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbff12b7d0 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e50032e10 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1e5003a050 con 0x55cbff049390 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 10 client.15636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 20 client.15636 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbff007f60 0x55cbff402de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 4 client.15636 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.147+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbff35d990 con 0x55cbff007f60 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.148+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbff007f60 0x55cbff402de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.148+0000 7f1e497fa640 10 client.15636 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.148+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbff007f60 0x55cbff402de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25f105954d888cf server_cookie=4378c534cdd42a7b in_seq=0 out_seq=0 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e497fa640 1 -- 172.21.15.94:0/4154786897 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cbff35d990 con 0x55cbff007f60 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e497fa640 10 client.15636 handle_command_reply: tid=0 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 client.15636 shutdown 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 msgr2=0x55cbff3fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e500056f0 tx=0x7f1e500319a0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 shutdown_connections 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbff007f60 0x55cbff402de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff049390 0x55cbff3fc990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/4154786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff0204d0 0x55cbff3f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e497fa640 0 client.15636 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 >> 172.21.15.94:0/4154786897 conn(0x55cbff03d4e0 msgr2=0x55cbff3e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 shutdown_connections 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 1 -- 172.21.15.94:0/4154786897 wait complete. 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.149+0000 7f1e6ef63740 20 client.15636 trim_cache size 0 max 16384 2024-04-25T13:42:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.162+0000 7f1e6ef63740 1 Processor -- start 2024-04-25T13:42:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.162+0000 7f1e6ef63740 1 -- start start 2024-04-25T13:42:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a93b0 con 0x55cbff0204d0 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aaef0 con 0x55cbff0286b0 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a9a80 con 0x55cbff049390 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54432/0 (socket says 172.21.15.94:54432) 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 -- 172.21.15.94:0/342438407 learned_addr learned my addr 172.21.15.94:0/342438407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 -- 172.21.15.94:0/342438407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 msgr2=0x55cbff3f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 -- 172.21.15.94:0/342438407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 msgr2=0x55cbff3fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 -- 172.21.15.94:0/342438407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbff3ed7a0 con 0x55cbff0204d0 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e6564a640 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1e50033000 tx=0x7f1e50031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2647cf96cffcaf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e4b7fe640 1 -- 172.21.15.94:0/342438407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50054020 con 0x55cbff0204d0 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e4b7fe640 1 -- 172.21.15.94:0/342438407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5004b030 con 0x55cbff0204d0 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.163+0000 7f1e4b7fe640 1 -- 172.21.15.94:0/342438407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50035040 con 0x55cbff0204d0 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- 172.21.15.94:0/342438407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 msgr2=0x55cbff3e7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1e50033000 tx=0x7f1e50031400 comp rx=0 tx=0).stop 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- 172.21.15.94:0/342438407 shutdown_connections 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/342438407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3e7160 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- 172.21.15.94:0/342438407 >> 172.21.15.94:0/342438407 conn(0x55cbff03d4e0 msgr2=0x55cbff3e6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- 172.21.15.94:0/342438407 shutdown_connections 2024-04-25T13:42:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- 172.21.15.94:0/342438407 wait complete. 2024-04-25T13:42:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 -- start start 2024-04-25T13:42:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.164+0000 7f1e6ef63740 1 Processor -- start 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa820 con 0x55cbff0204d0 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa150 con 0x55cbff0286b0 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a7390 con 0x55cbff049390 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6ef63740 10 client.0 _ll_register_callbacks cb 0x7f1e5c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54434/0 (socket says 172.21.15.94:54434) 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 -- 172.21.15.94:0/638094065 learned_addr learned my addr 172.21.15.94:0/638094065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 -- 172.21.15.94:0/638094065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 msgr2=0x55cbff3fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 -- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 msgr2=0x55cbff3fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 -- 172.21.15.94:0/638094065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbff3fcdf0 con 0x55cbff0204d0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e6564a640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e50003b10 tx=0x7f1e50003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cff38ee1bfa1ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.165+0000 7f1e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50068060 con 0x55cbff0204d0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1e500051a0 con 0x55cbff0204d0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e5004b030 con 0x55cbff0204d0 2024-04-25T13:42:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e50035070 con 0x55cbff0204d0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbff3f0cc0 con 0x55cbff0204d0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5004b030 con 0x55cbff0204d0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1e5003a030 con 0x55cbff0204d0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 20 client.15642 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbfefdd3a0 0x55cbff46d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 4 client.15642 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbff3ebe30 con 0x55cbfefdd3a0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.166+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbfefdd3a0 0x55cbff46d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e497fa640 10 client.15642 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e65e4b640 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbfefdd3a0 0x55cbff46d620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa72963c68031f04 server_cookie=f2d0aedaf9d06782 in_seq=0 out_seq=0 2024-04-25T13:42:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e497fa640 1 -- 172.21.15.94:0/638094065 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbff3ebe30 con 0x55cbfefdd3a0 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e497fa640 10 client.15642 handle_command_reply: tid=0 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 client.15642 shutdown 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 msgr2=0x55cbff3fa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1e50003b10 tx=0x7f1e50003d10 comp rx=0 tx=0).stop 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 shutdown_connections 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cbfefdd3a0 0x55cbff46d620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbff049390 0x55cbff3fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbff0286b0 0x55cbff3fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 --2- 172.21.15.94:0/638094065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbff0204d0 0x55cbff3fa140 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e497fa640 0 client.15642 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 >> 172.21.15.94:0/638094065 conn(0x55cbff03d4e0 msgr2=0x55cbff3e9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 shutdown_connections 2024-04-25T13:42:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 1 -- 172.21.15.94:0/638094065 wait complete. 2024-04-25T13:42:20.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.167+0000 7f1e6ef63740 20 client.15642 trim_cache size 0 max 16384 2024-04-25T13:42:20.175 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 msgr2=0x7f1e4c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 0x7f1e4c0b2090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e580a6c10 tx=0x7f1e58075a60 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 msgr2=0x7f1e601a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800c970 tx=0x7f1e5800ce30 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 shutdown_connections 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1e4c0afb80 0x7f1e4c0b2090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e60108920 0x7f1e601a0930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60103c20 0x7f1e601999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 --2- 172.21.15.94:0/1064265407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e601029e0 0x7f1e60199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 >> 172.21.15.94:0/1064265407 conn(0x7f1e600fdbd0 msgr2=0x7f1e60104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 shutdown_connections 2024-04-25T13:42:20.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.168+0000 7f1e680af640 1 -- 172.21.15.94:0/1064265407 wait complete. 2024-04-25T13:42:20.178 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:20 smithi094 ceph-mon[29195]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:20.178 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:20 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2094554186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:20.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:42:20.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 -- 172.21.15.94:0/1848538452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c106f30 msgr2=0x7f571c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 --2- 172.21.15.94:0/1848538452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c106f30 0x7f571c109350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 -- 172.21.15.94:0/1848538452 shutdown_connections 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 --2- 172.21.15.94:0/1848538452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 0x7f571c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 --2- 172.21.15.94:0/1848538452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c106f30 0x7f571c109350 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 --2- 172.21.15.94:0/1848538452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c1045d0 0x7f571c1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 -- 172.21.15.94:0/1848538452 >> 172.21.15.94:0/1848538452 conn(0x7f571c0fdbd0 msgr2=0x7f571c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 -- 172.21.15.94:0/1848538452 shutdown_connections 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.289+0000 7f5721451640 1 -- 172.21.15.94:0/1848538452 wait complete. 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 Processor -- start 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 -- start start 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c106f30 0x7f571c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 0x7f571c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f571c10f5f0 con 0x7f571c1045d0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f571c10f470 con 0x7f571c109890 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5721451640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f571c10f770 con 0x7f571c106f30 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54452/0 (socket says 172.21.15.94:54452) 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 -- 172.21.15.94:0/2795399699 learned_addr learned my addr 172.21.15.94:0/2795399699 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f5713fff640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c106f30 0x7f571c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571b7fe640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 0x7f571c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 -- 172.21.15.94:0/2795399699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c106f30 msgr2=0x7f571c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c106f30 0x7f571c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 -- 172.21.15.94:0/2795399699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 msgr2=0x7f571c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 0x7f571c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.290+0000 7f571affd640 1 -- 172.21.15.94:0/2795399699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c1a2090 con 0x7f571c1045d0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f571affd640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f570c005fd0 tx=0x7f570c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f5718ff9640 1 -- 172.21.15.94:0/2795399699 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c017070 con 0x7f571c1045d0 2024-04-25T13:42:20.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f5718ff9640 1 -- 172.21.15.94:0/2795399699 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c009c30 con 0x7f571c1045d0 2024-04-25T13:42:20.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c1a2380 con 0x7f571c1045d0 2024-04-25T13:42:20.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f5718ff9640 1 -- 172.21.15.94:0/2795399699 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c003d30 con 0x7f571c1045d0 2024-04-25T13:42:20.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.291+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c1a28d0 con 0x7f571c1045d0 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.293+0000 7f5718ff9640 1 -- 172.21.15.94:0/2795399699 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f570c003ed0 con 0x7f571c1045d0 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.293+0000 7f57283e1740 1 Processor -- start 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.293+0000 7f57283e1740 1 -- start start 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7febb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7febca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561d7fe1fd80 con 0x561d7faf04d0 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561d7fdd9a20 con 0x561d7fb19390 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f57283e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d7fe1bd40 con 0x561d7faf86b0 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54458/0 (socket says 172.21.15.94:54458) 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 -- 172.21.15.94:0/3618484259 learned_addr learned my addr 172.21.15.94:0/3618484259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571b7fe640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7febb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f5718ff9640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 0x7f57080b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f5713fff640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7febca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571b7fe640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 0x7f57080b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 -- 172.21.15.94:0/3618484259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 msgr2=0x561d7febca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7febca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 -- 172.21.15.94:0/3618484259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 msgr2=0x561d7febb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7febb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571affd640 1 -- 172.21.15.94:0/3618484259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7fde4c30 con 0x561d7faf04d0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f571b7fe640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7febb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.294+0000 7f5718ff9640 1 -- 172.21.15.94:0/2795399699 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f570c0dec00 con 0x7f571c1045d0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.295+0000 7f571affd640 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f570c0a2f60 tx=0x7f570c0a6960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6146a7674617c4cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.295+0000 7f571b7fe640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 0x7f57080b2090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x561d7febc860 tx=0x7f5714004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.295+0000 7f57117fa640 1 -- 172.21.15.94:0/3618484259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c009de0 con 0x561d7faf04d0 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.295+0000 7f57117fa640 1 -- 172.21.15.94:0/3618484259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c0a75e0 con 0x561d7faf04d0 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.295+0000 7f57117fa640 1 -- 172.21.15.94:0/3618484259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0a3170 con 0x561d7faf04d0 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- 172.21.15.94:0/3618484259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 msgr2=0x561d7febbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f570c0a2f60 tx=0x7f570c0a6960 comp rx=0 tx=0).stop 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- 172.21.15.94:0/3618484259 shutdown_connections 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7febca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7febbd90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- 172.21.15.94:0/3618484259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7febb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- 172.21.15.94:0/3618484259 >> 172.21.15.94:0/3618484259 conn(0x561d7fb0d4e0 msgr2=0x561d7feb7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- 172.21.15.94:0/3618484259 shutdown_connections 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- 172.21.15.94:0/3618484259 wait complete. 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 -- start start 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 Processor -- start 2024-04-25T13:42:20.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf04d0 0x561d7fec5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf86b0 0x561d7fec59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.296+0000 7f57283e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57283e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561d7febd470 con 0x561d7fb19390 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57283e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561d7fda6dc0 con 0x561d7faf86b0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57283e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d7febd5f0 con 0x561d7faf04d0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57283e1740 10 client.0 _ll_register_callbacks cb 0x561d7fdc2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54470/0 (socket says 172.21.15.94:54470) 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 -- 172.21.15.94:0/145368581 learned_addr learned my addr 172.21.15.94:0/145368581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571affd640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf04d0 0x561d7fec5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f5713fff640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf86b0 0x561d7fec59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 -- 172.21.15.94:0/145368581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf04d0 msgr2=0x561d7fec5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf04d0 0x561d7fec5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 -- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf86b0 msgr2=0x561d7fec59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf86b0 0x561d7fec59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 -- 172.21.15.94:0/145368581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7fb313f0 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f571b7fe640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5714014390 tx=0x7f5714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1632e8867b72f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57027fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714025070 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5714004b20 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.297+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570c0a75e0 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5714030050 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d7fbfb7d0 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c0a75e0 con 0x561d7fb19390 2024-04-25T13:42:20.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5714020040 con 0x561d7fb19390 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 20 client.15654 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7fad7f60 0x561d7fed3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 4 client.15654 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.298+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d7fe2d990 con 0x561d7fad7f60 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.299+0000 7f571affd640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7fad7f60 0x561d7fed3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.299+0000 7f571affd640 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7fad7f60 0x561d7fed3180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5884b97b8186fee server_cookie=b10c17fcddc828ee in_seq=0 out_seq=0 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.299+0000 7f57027fc640 10 client.15654 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57027fc640 1 -- 172.21.15.94:0/145368581 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d7fe2d990 con 0x561d7fad7f60 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57027fc640 10 client.15654 handle_command_reply: tid=0 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 client.15654 shutdown 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 msgr2=0x561d7fecc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5714014390 tx=0x7f5714004a40 comp rx=0 tx=0).stop 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 shutdown_connections 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7fad7f60 0x561d7fed3180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7fb19390 0x561d7fecc890 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf86b0 0x561d7fec59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 --2- 172.21.15.94:0/145368581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf04d0 0x561d7fec5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 >> 172.21.15.94:0/145368581 conn(0x561d7fb0d4e0 msgr2=0x561d7feb9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57027fc640 0 client.15654 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 shutdown_connections 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 1 -- 172.21.15.94:0/145368581 wait complete. 2024-04-25T13:42:20.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.300+0000 7f57283e1740 20 client.15654 trim_cache size 0 max 16384 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.313+0000 7f57283e1740 1 Processor -- start 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.313+0000 7f57283e1740 1 -- start start 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7fec0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7fec07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1ab5d0 con 0x561d7faf04d0 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1ad110 con 0x561d7fb19390 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f57283e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1abca0 con 0x561d7faf86b0 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54486/0 (socket says 172.21.15.94:54486) 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 -- 172.21.15.94:0/1466928669 learned_addr learned my addr 172.21.15.94:0/1466928669 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 -- 172.21.15.94:0/1466928669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 msgr2=0x561d7fec0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7fec0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 -- 172.21.15.94:0/1466928669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 msgr2=0x561d7fec07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7fec07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 -- 172.21.15.94:0/1466928669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d7fec6060 con 0x561d7faf04d0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f571affd640 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f570c000c00 tx=0x7f570c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8812898a51bfb702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.314+0000 7f5710ff9640 1 -- 172.21.15.94:0/1466928669 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c06c070 con 0x561d7faf04d0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f5710ff9640 1 -- 172.21.15.94:0/1466928669 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c0ab290 con 0x561d7faf04d0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f5710ff9640 1 -- 172.21.15.94:0/1466928669 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0dc830 con 0x561d7faf04d0 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 -- 172.21.15.94:0/1466928669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 msgr2=0x561d7feb1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f570c000c00 tx=0x7f570c009de0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 -- 172.21.15.94:0/1466928669 shutdown_connections 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7fb19390 0x561d7fec07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7faf86b0 0x561d7fec0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 --2- 172.21.15.94:0/1466928669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf04d0 0x561d7feb1d30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 -- 172.21.15.94:0/1466928669 >> 172.21.15.94:0/1466928669 conn(0x561d7fb0d4e0 msgr2=0x561d7febb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 -- 172.21.15.94:0/1466928669 shutdown_connections 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.315+0000 7f57283e1740 1 -- 172.21.15.94:0/1466928669 wait complete. 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 -- start start 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 Processor -- start 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 0x561d7fec9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7fb19390 0x561d7feca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1aca40 con 0x561d7faf86b0 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1ac370 con 0x561d7faf04d0 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1a95b0 con 0x561d7fb19390 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f57283e1740 10 client.0 _ll_register_callbacks cb 0x7f5714014f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54498/0 (socket says 172.21.15.94:54498) 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 -- 172.21.15.94:0/3161808549 learned_addr learned my addr 172.21.15.94:0/3161808549 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f571affd640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 0x561d7fec9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 -- 172.21.15.94:0/3161808549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7fb19390 msgr2=0x561d7feca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7fb19390 0x561d7feca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 -- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 msgr2=0x561d7fec9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 0x561d7fec9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f5713fff640 1 -- 172.21.15.94:0/3161808549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d7fecab60 con 0x561d7faf86b0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.316+0000 7f571affd640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 0x561d7fec9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f5713fff640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f5704031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efc1c25d748ec9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704035050 con 0x561d7faf86b0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5704004450 con 0x561d7faf86b0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561d7fe27800 con 0x561d7faf86b0 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5704005040 con 0x561d7faf86b0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d7fecbe80 con 0x561d7faf86b0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561d7fe27800 con 0x561d7faf86b0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5704042070 con 0x561d7faf86b0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 20 client.15660 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7faad3a0 0x561d7ff3d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 4 client.15660 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.317+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d7fda2610 con 0x561d7faad3a0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f571affd640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7faad3a0 0x561d7ff3d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f571affd640 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7faad3a0 0x561d7ff3d970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7195456237dcffc server_cookie=81df25353a8d3114 in_seq=0 out_seq=0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57027fc640 10 client.15660 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57027fc640 1 -- 172.21.15.94:0/3161808549 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d7fda2610 con 0x561d7faad3a0 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57027fc640 10 client.15660 handle_command_reply: tid=0 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 client.15660 shutdown 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 msgr2=0x561d7fec96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f5704031840 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 shutdown_connections 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561d7faad3a0 0x561d7ff3d970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d7fb19390 0x561d7feca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d7faf86b0 0x561d7fec96c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57027fc640 0 client.15660 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 --2- 172.21.15.94:0/3161808549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d7faf04d0 0x561d7fec9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 >> 172.21.15.94:0/3161808549 conn(0x561d7fb0d4e0 msgr2=0x561d7febb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 shutdown_connections 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 1 -- 172.21.15.94:0/3161808549 wait complete. 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.318+0000 7f57283e1740 20 client.15660 trim_cache size 0 max 16384 2024-04-25T13:42:20.326 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 msgr2=0x7f57080b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 0x7f57080b2090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x561d7febc860 tx=0x7f5714004ab0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 msgr2=0x7f571c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f570c005fd0 tx=0x7f570c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 shutdown_connections 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57080afb80 0x7f57080b2090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f571c109890 0x7f571c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f571c106f30 0x7f571c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 --2- 172.21.15.94:0/2795399699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f571c1045d0 0x7f571c19a550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 >> 172.21.15.94:0/2795399699 conn(0x7f571c0fdbd0 msgr2=0x7f571c105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 shutdown_connections 2024-04-25T13:42:20.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:20.319+0000 7f5721451640 1 -- 172.21.15.94:0/2795399699 wait complete. 2024-04-25T13:42:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:20 smithi137 ceph-mon[32568]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:20.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:20 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2094554186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:20 smithi138 ceph-mon[32977]: pgmap v362: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:20 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2094554186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:21.330 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:42:21.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:21.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:42:21.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 -- 172.21.15.94:0/2627743600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54103c40 msgr2=0x7f0b54104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 --2- 172.21.15.94:0/2627743600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54103c40 0x7f0b54104050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009900 tx=0x7f0b4402a410 comp rx=0 tx=0).stop 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 -- 172.21.15.94:0/2627743600 shutdown_connections 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 --2- 172.21.15.94:0/2627743600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54108940 0x7f0b5410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 --2- 172.21.15.94:0/2627743600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54103c40 0x7f0b54104050 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 --2- 172.21.15.94:0/2627743600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b54103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 -- 172.21.15.94:0/2627743600 >> 172.21.15.94:0/2627743600 conn(0x7f0b540fdbf0 msgr2=0x7f0b541000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 -- 172.21.15.94:0/2627743600 shutdown_connections 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.455+0000 7f0b59049640 1 -- 172.21.15.94:0/2627743600 wait complete. 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 Processor -- start 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 -- start start 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b5419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54103c40 0x7f0b5419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410e500 con 0x7f0b54108940 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410e380 con 0x7f0b54102a00 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.456+0000 7f0b59049640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410e680 con 0x7f0b54103c40 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54510/0 (socket says 172.21.15.94:54510) 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 -- 172.21.15.94:0/455863475 learned_addr learned my addr 172.21.15.94:0/455863475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5259c640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54103c40 0x7f0b5419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b5419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 -- 172.21.15.94:0/455863475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54103c40 msgr2=0x7f0b5419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54103c40 0x7f0b5419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 -- 172.21.15.94:0/455863475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 msgr2=0x7f0b5419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b5419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 -- 172.21.15.94:0/455863475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b541a1d10 con 0x7f0b54108940 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b5419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b5359e640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00eb10 tx=0x7f0b4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b3bfff640 1 -- 172.21.15.94:0/455863475 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c004b20 con 0x7f0b54108940 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b3bfff640 1 -- 172.21.15.94:0/455863475 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4c004cc0 con 0x7f0b54108940 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b541a1fa0 con 0x7f0b54108940 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b3bfff640 1 -- 172.21.15.94:0/455863475 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c010670 con 0x7f0b54108940 2024-04-25T13:42:21.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.457+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541a2410 con 0x7f0b54108940 2024-04-25T13:42:21.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b3bfff640 1 -- 172.21.15.94:0/455863475 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f0b4c005000 con 0x7f0b54108940 2024-04-25T13:42:21.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 Processor -- start 2024-04-25T13:42:21.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 -- start start 2024-04-25T13:42:21.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56179cd12b00 con 0x56179c9e3550 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56179ccae920 con 0x56179ca0c410 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179cd0eac0 con 0x56179c9eb730 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54512/0 (socket says 172.21.15.94:54512) 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 -- 172.21.15.94:0/4129533082 learned_addr learned my addr 172.21.15.94:0/4129533082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5259c640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5259c640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 -- 172.21.15.94:0/4129533082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 msgr2=0x56179cdb4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b3bfff640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 0x7f0b3c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 -- 172.21.15.94:0/4129533082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 msgr2=0x56179cdad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b52d9d640 1 -- 172.21.15.94:0/4129533082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179caec7e0 con 0x56179c9e3550 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.460+0000 7f0b5259c640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b5359e640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 0x7f0b3c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800bce0 tx=0x7f0b4800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc641089a9673aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b3bfff640 1 -- 172.21.15.94:0/455863475 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0b4c0df4c0 con 0x7f0b54108940 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b397fa640 1 -- 172.21.15.94:0/4129533082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48015070 con 0x56179c9e3550 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b397fa640 1 -- 172.21.15.94:0/4129533082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b480118f0 con 0x56179c9e3550 2024-04-25T13:42:21.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.461+0000 7f0b397fa640 1 -- 172.21.15.94:0/4129533082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48010870 con 0x56179c9e3550 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/4129533082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 msgr2=0x56179cdada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5359e640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 0x7f0b3c0b2090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c01dfd0 tx=0x7f0b4c00efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800bce0 tx=0x7f0b4800f730 comp rx=0 tx=0).stop 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/4129533082 shutdown_connections 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdada20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/4129533082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/4129533082 >> 172.21.15.94:0/4129533082 conn(0x56179ca00560 msgr2=0x56179cda8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/4129533082 shutdown_connections 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/4129533082 wait complete. 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- start start 2024-04-25T13:42:21.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 Processor -- start 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdb7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdbe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56179cdb5370 con 0x56179c9eb730 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56179ccac5d0 con 0x56179ca0c410 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.462+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179cdb54f0 con 0x56179c9e3550 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5ffd2740 10 client.0 _ll_register_callbacks cb 0x56179cc9a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54516/0 (socket says 172.21.15.94:54516) 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 -- 172.21.15.94:0/2793874410 learned_addr learned my addr 172.21.15.94:0/2793874410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5259c640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdb7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5259c640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdbe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 -- 172.21.15.94:0/2793874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 msgr2=0x56179cdb7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdb7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 -- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 msgr2=0x56179cdbe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdbe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 -- 172.21.15.94:0/2793874410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179cb4cf70 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5259c640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b48011670 tx=0x7f0b480116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e091d3b5098089cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48028060 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0b48011cb0 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b480118f0 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b48015050 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5ffd2740 10 client.15672 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.463+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56179c9e4170 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b480118f0 con 0x56179c9eb730 2024-04-25T13:42:21.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0b48004af0 con 0x56179c9eb730 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 10 client.15672 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 20 client.15672 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9cb1f0 0x56179cdc4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 4 client.15672 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179cd20710 con 0x56179c9cb1f0 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.464+0000 7f0b5259c640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9cb1f0 0x56179cdc4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.465+0000 7f0b36ff5640 10 client.15672 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.465+0000 7f0b5259c640 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9cb1f0 0x56179cdc4790 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ab6027e1be9964 server_cookie=30adbb3579053332 in_seq=0 out_seq=0 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.465+0000 7f0b36ff5640 1 -- 172.21.15.94:0/2793874410 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56179cd20710 con 0x56179c9cb1f0 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.465+0000 7f0b36ff5640 10 client.15672 handle_command_reply: tid=0 2024-04-25T13:42:21.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.465+0000 7f0b5ffd2740 1 client.15672 shutdown 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 msgr2=0x56179cdb77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b48011670 tx=0x7f0b480116a0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 shutdown_connections 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9cb1f0 0x56179cdc4790 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9eb730 0x56179cdb77a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b36ff5640 0 client.15672 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/2793874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdb7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 >> 172.21.15.94:0/2793874410 conn(0x56179ca00560 msgr2=0x56179cda7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 shutdown_connections 2024-04-25T13:42:21.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/2793874410 wait complete. 2024-04-25T13:42:21.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.466+0000 7f0b5ffd2740 20 client.15672 trim_cache size 0 max 16384 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.483+0000 7f0b5ffd2740 1 Processor -- start 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.483+0000 7f0b5ffd2740 1 -- start start 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdc8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541ab570 con 0x56179c9e3550 2024-04-25T13:42:21.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541ad0b0 con 0x56179ca0c410 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541abc40 con 0x56179c9eb730 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54524/0 (socket says 172.21.15.94:54524) 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 -- 172.21.15.94:0/3164730331 learned_addr learned my addr 172.21.15.94:0/3164730331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 -- 172.21.15.94:0/3164730331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 msgr2=0x56179cdb0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 -- 172.21.15.94:0/3164730331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 msgr2=0x56179cdc8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdc8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 -- 172.21.15.94:0/3164730331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179cdb83a0 con 0x56179c9e3550 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b5259c640 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b44031500 tx=0x7f0b44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c107274df5315b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b38ff9640 1 -- 172.21.15.94:0/3164730331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44035040 con 0x56179c9e3550 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b38ff9640 1 -- 172.21.15.94:0/3164730331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4404f070 con 0x56179c9e3550 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.484+0000 7f0b38ff9640 1 -- 172.21.15.94:0/3164730331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b440038e0 con 0x56179c9e3550 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/3164730331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 msgr2=0x56179cdb1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b44031500 tx=0x7f0b44000bc0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/3164730331 shutdown_connections 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179ca0c410 0x56179cdc8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9eb730 0x56179cdb0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/3164730331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179c9e3550 0x56179cdb1d50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/3164730331 >> 172.21.15.94:0/3164730331 conn(0x56179ca00560 msgr2=0x56179cdb0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/3164730331 shutdown_connections 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/3164730331 wait complete. 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 -- start start 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.485+0000 7f0b5ffd2740 1 Processor -- start 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdbb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 0x56179cdbb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541ac9e0 con 0x56179ca0c410 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541ac310 con 0x56179c9eb730 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541a9550 con 0x56179c9e3550 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 10 client.0 _ll_register_callbacks cb 0x7f0b4800b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54528/0 (socket says 172.21.15.94:54528) 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 -- 172.21.15.94:0/1845968208 learned_addr learned my addr 172.21.15.94:0/1845968208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdbb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 0x56179cdbb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 -- 172.21.15.94:0/1845968208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 msgr2=0x56179cdbb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdbb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 -- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 msgr2=0x56179cdbb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 0x56179cdbb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 -- 172.21.15.94:0/1845968208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179cdbcc90 con 0x56179ca0c410 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 0x56179cdbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5259c640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b44004630 tx=0x7f0b44005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d136255c2ec847ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44059020 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0b4404a6a0 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.486+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4404f070 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b44067050 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56179cdbdf50 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4404f070 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0b440316c0 con 0x56179ca0c410 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 20 client.15678 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9a0630 0x56179ce2f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 4 client.15678 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:42:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56179cd1a580 con 0x56179c9a0630 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.487+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9a0630 0x56179ce2f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.488+0000 7f0b52d9d640 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9a0630 0x56179ce2f160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d669f2333193c9d0 server_cookie=a5c9f5b4451ed947 in_seq=0 out_seq=0 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.488+0000 7f0b36ff5640 10 client.15678 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b36ff5640 1 -- 172.21.15.94:0/1845968208 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56179cd1a580 con 0x56179c9a0630 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b36ff5640 10 client.15678 handle_command_reply: tid=0 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 client.15678 shutdown 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 msgr2=0x56179cdbc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b44004630 tx=0x7f0b44005e30 comp rx=0 tx=0).stop 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 shutdown_connections 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56179c9a0630 0x56179ce2f160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179ca0c410 0x56179cdbc6d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179c9eb730 0x56179cdbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b36ff5640 0 client.15678 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 --2- 172.21.15.94:0/1845968208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179c9e3550 0x56179cdbb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 >> 172.21.15.94:0/1845968208 conn(0x56179ca00560 msgr2=0x56179cdabc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 shutdown_connections 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 1 -- 172.21.15.94:0/1845968208 wait complete. 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.489+0000 7f0b5ffd2740 20 client.15678 trim_cache size 0 max 16384 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "7e82a34c-4efb-474c-b558-834b382518cb", 2024-04-25T13:42:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:42:21.498 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 msgr2=0x7f0b3c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 0x7f0b3c0b2090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c01dfd0 tx=0x7f0b4c00efd0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 msgr2=0x7f0b541a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00eb10 tx=0x7f0b4c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 shutdown_connections 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b3c0afb80 0x7f0b3c0b2090 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b54108940 0x7f0b541a16a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b54103c40 0x7f0b5419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 --2- 172.21.15.94:0/455863475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b54102a00 0x7f0b5419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 >> 172.21.15.94:0/455863475 conn(0x7f0b540fdbf0 msgr2=0x7f0b54109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 shutdown_connections 2024-04-25T13:42:21.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.490+0000 7f0b59049640 1 -- 172.21.15.94:0/455863475 wait complete. 2024-04-25T13:42:21.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:42:21.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:42:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 -- 172.21.15.94:0/1989393124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c103af0 msgr2=0x7f409c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 --2- 172.21.15.94:0/1989393124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c103af0 0x7f409c1047f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 -- 172.21.15.94:0/1989393124 shutdown_connections 2024-04-25T13:42:21.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 --2- 172.21.15.94:0/1989393124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c109a30 0x7f409c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 --2- 172.21.15.94:0/1989393124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c104d30 0x7f409c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 --2- 172.21.15.94:0/1989393124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c103af0 0x7f409c1047f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 -- 172.21.15.94:0/1989393124 >> 172.21.15.94:0/1989393124 conn(0x7f409c0fece0 msgr2=0x7f409c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 -- 172.21.15.94:0/1989393124 shutdown_connections 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.616+0000 7f40a0b84640 1 -- 172.21.15.94:0/1989393124 wait complete. 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 Processor -- start 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 -- start start 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c103af0 0x7f409c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c109a30 0x7f409c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10f5f0 con 0x7f409c104d30 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10f470 con 0x7f409c103af0 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f40a0b84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10f770 con 0x7f409c109a30 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54548/0 (socket says 172.21.15.94:54548) 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 -- 172.21.15.94:0/2407211372 learned_addr learned my addr 172.21.15.94:0/2407211372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f409ad9d640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c109a30 0x7f409c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f409a59c640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c103af0 0x7f409c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 -- 172.21.15.94:0/2407211372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c109a30 msgr2=0x7f409c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c109a30 0x7f409c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 -- 172.21.15.94:0/2407211372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c103af0 msgr2=0x7f409c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c103af0 0x7f409c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.617+0000 7f4099d9b640 1 -- 172.21.15.94:0/2407211372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1a1da0 con 0x7f409c104d30 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f408400d770 tx=0x7f408400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f40937fe640 1 -- 172.21.15.94:0/2407211372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084017070 con 0x7f409c104d30 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1a2090 con 0x7f409c104d30 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f40937fe640 1 -- 172.21.15.94:0/2407211372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40840054d0 con 0x7f409c104d30 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f40937fe640 1 -- 172.21.15.94:0/2407211372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084004c30 con 0x7f409c104d30 2024-04-25T13:42:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.618+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a25e0 con 0x7f409c104d30 2024-04-25T13:42:21.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.620+0000 7f40a7a3b740 1 Processor -- start 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 -- start start 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40937fe640 1 -- 172.21.15.94:0/2407211372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f4084004e10 con 0x7f409c104d30 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0284d0 0x56022e3f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0306b0 0x56022e3f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56022e357d80 con 0x56022e051390 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56022e3118b0 con 0x56022e0306b0 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56022e353d40 con 0x56022e0284d0 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54552/0 (socket says 172.21.15.94:54552) 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 -- 172.21.15.94:0/3976083720 learned_addr learned my addr 172.21.15.94:0/3976083720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f4099d9b640 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0306b0 0x56022e3f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 -- 172.21.15.94:0/3976083720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0284d0 msgr2=0x56022e3f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0284d0 0x56022e3f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 -- 172.21.15.94:0/3976083720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0306b0 msgr2=0x56022e3f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0306b0 0x56022e3f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 -- 172.21.15.94:0/3976083720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022e2e2a40 con 0x56022e051390 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f409ad9d640 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ab80 tx=0x7f408c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0f982fc164a877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.621+0000 7f40937fe640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 0x7f40880b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.622+0000 7f4090ff9640 1 -- 172.21.15.94:0/3976083720 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c01d070 con 0x56022e051390 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.622+0000 7f4090ff9640 1 -- 172.21.15.94:0/3976083720 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c018af0 con 0x56022e051390 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.622+0000 7f409a59c640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 0x7f40880b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.622+0000 7f4090ff9640 1 -- 172.21.15.94:0/3976083720 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c0109d0 con 0x56022e051390 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.622+0000 7f40937fe640 1 -- 172.21.15.94:0/2407211372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f40840de950 con 0x7f409c104d30 2024-04-25T13:42:21.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- 172.21.15.94:0/3976083720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 msgr2=0x56022e3f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f409a59c640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 0x7f40880b2090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4094000f80 tx=0x7f4094031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ab80 tx=0x7f408c00f790 comp rx=0 tx=0).stop 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- 172.21.15.94:0/3976083720 shutdown_connections 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0306b0 0x56022e3f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0284d0 0x56022e3f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/3976083720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e3f2a60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- 172.21.15.94:0/3976083720 >> 172.21.15.94:0/3976083720 conn(0x56022e0454e0 msgr2=0x56022e3ee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- 172.21.15.94:0/3976083720 shutdown_connections 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- 172.21.15.94:0/3976083720 wait complete. 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- start start 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 Processor -- start 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e3fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e3fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56022e3f4780 con 0x56022e051390 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56022e2ded10 con 0x56022e0284d0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56022e3f4900 con 0x56022e0306b0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f40a7a3b740 10 client.0 _ll_register_callbacks cb 0x56022e30b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f409ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54556/0 (socket says 172.21.15.94:54556) 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.623+0000 7f409ad9d640 1 -- 172.21.15.94:0/1747655774 learned_addr learned my addr 172.21.15.94:0/1747655774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409a59c640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e3fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4099d9b640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e3fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 -- 172.21.15.94:0/1747655774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 msgr2=0x56022e3fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e3fcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 -- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 msgr2=0x56022e3fca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e3fca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 -- 172.21.15.94:0/1747655774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022e1337d0 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409a59c640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e3fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f409ad9d640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f408c000c00 tx=0x7f408c018ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf4cf7baefec81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c027020 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f408c005240 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c018af0 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f408c032050 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f40a7a3b740 10 client.15690 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56022e0693f0 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.624+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c018af0 con 0x56022e051390 2024-04-25T13:42:21.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f408c004750 con 0x56022e051390 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 20 client.15690 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e00ff60 0x56022e40a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 4 client.15690 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56022e365990 con 0x56022e00ff60 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.625+0000 7f409a59c640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e00ff60 0x56022e40a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f4081ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f409a59c640 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e00ff60 0x56022e40a0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7137fe1e616664f server_cookie=5d1647ddcf38c641 in_seq=0 out_seq=0 2024-04-25T13:42:21.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f4081ffb640 1 -- 172.21.15.94:0/1747655774 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56022e365990 con 0x56022e00ff60 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f4081ffb640 10 client.15690 handle_command_reply: tid=0 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 client.15690 shutdown 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 msgr2=0x56022e403f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f408c000c00 tx=0x7f408c018ed0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 shutdown_connections 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e00ff60 0x56022e40a0d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e051390 0x56022e403f00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f4081ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e3fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1747655774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e3fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 >> 172.21.15.94:0/1747655774 conn(0x56022e0454e0 msgr2=0x56022e3ed3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.626+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 shutdown_connections 2024-04-25T13:42:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.627+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1747655774 wait complete. 2024-04-25T13:42:21.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.627+0000 7f40a7a3b740 20 client.15690 trim_cache size 0 max 16384 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 Processor -- start 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 -- start start 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e00ff60 0x56022e3fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e40e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ab5e0 con 0x56022e0284d0 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ad120 con 0x56022e00ff60 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.644+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1abcb0 con 0x56022e0306b0 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54572/0 (socket says 172.21.15.94:54572) 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 -- 172.21.15.94:0/1429468713 learned_addr learned my addr 172.21.15.94:0/1429468713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f409a59c640 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e00ff60 0x56022e3fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f409ad9d640 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e40e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 -- 172.21.15.94:0/1429468713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 msgr2=0x56022e40e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e40e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 -- 172.21.15.94:0/1429468713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e00ff60 msgr2=0x56022e3fca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e00ff60 0x56022e3fca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 -- 172.21.15.94:0/1429468713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022e0290f0 con 0x56022e0284d0 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4099d9b640 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f40840dc290 tx=0x7f40840a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd76d6b69466108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4083fff640 1 -- 172.21.15.94:0/1429468713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084020040 con 0x56022e0284d0 2024-04-25T13:42:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4083fff640 1 -- 172.21.15.94:0/1429468713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40840a6740 con 0x56022e0284d0 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.645+0000 7f4083fff640 1 -- 172.21.15.94:0/1429468713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840afa90 con 0x56022e0284d0 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1429468713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 msgr2=0x56022e3fcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f40840dc290 tx=0x7f40840a6ac0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1429468713 shutdown_connections 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e40e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e0284d0 0x56022e3fcfa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/1429468713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e00ff60 0x56022e3fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1429468713 >> 172.21.15.94:0/1429468713 conn(0x56022e0454e0 msgr2=0x56022e3f77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1429468713 shutdown_connections 2024-04-25T13:42:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- 172.21.15.94:0/1429468713 wait complete. 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 -- start start 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 Processor -- start 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e400af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.646+0000 7f40a7a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e470310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aca50 con 0x56022e00ff60 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ac380 con 0x56022e0284d0 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a95c0 con 0x56022e0306b0 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 10 client.0 _ll_register_callbacks cb 0x7f408c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54586/0 (socket says 172.21.15.94:54586) 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 -- 172.21.15.94:0/2024806151 learned_addr learned my addr 172.21.15.94:0/2024806151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e400af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 -- 172.21.15.94:0/2024806151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 msgr2=0x56022e470310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e470310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 -- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 msgr2=0x56022e400af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e400af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 -- 172.21.15.94:0/2024806151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022e3184b0 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f409a59c640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19fd97aa481504c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f4081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004240 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f40940043e0 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40840a6740 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4094061050 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.647+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56022e471a20 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40840a6740 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4094052250 con 0x56022e00ff60 2024-04-25T13:42:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 20 client.15696 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e051390 0x56022e474440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 4 client.15696 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56022e35f800 con 0x56022e051390 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.648+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e051390 0x56022e474440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f4081ffb640 10 client.15696 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f4099d9b640 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e051390 0x56022e474440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbbc5d61a876563 server_cookie=47a3b6547ba14202 in_seq=0 out_seq=0 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f4081ffb640 1 -- 172.21.15.94:0/2024806151 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56022e35f800 con 0x56022e051390 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f4081ffb640 10 client.15696 handle_command_reply: tid=0 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 client.15696 shutdown 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 msgr2=0x56022e4005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409403e070 comp rx=0 tx=0).stop 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 shutdown_connections 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56022e051390 0x56022e474440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56022e0306b0 0x56022e470310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56022e0284d0 0x56022e400af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f4081ffb640 0 client.15696 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 --2- 172.21.15.94:0/2024806151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56022e00ff60 0x56022e4005b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 >> 172.21.15.94:0/2024806151 conn(0x56022e0454e0 msgr2=0x56022e3fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 shutdown_connections 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 1 -- 172.21.15.94:0/2024806151 wait complete. 2024-04-25T13:42:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a7a3b740 20 client.15696 trim_cache size 0 max 16384 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout: "7e82a34c-4efb-474c-b558-834b382518cb": { 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:42:21.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "7e82a34c-4efb-474c-b558-834b382518cb", 2024-04-25T13:42:21.660 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:42:21.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:42:21.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:42:21.660 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 msgr2=0x7f40880b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 0x7f40880b2090 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4094000f80 tx=0x7f4094031af0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 msgr2=0x7f409c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.649+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f408400d770 tx=0x7f408400dc30 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 shutdown_connections 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40880afb80 0x7f40880b2090 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f409c109a30 0x7f409c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c104d30 0x7f409c19a7a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 --2- 172.21.15.94:0/2407211372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f409c103af0 0x7f409c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 >> 172.21.15.94:0/2407211372 conn(0x7f409c0fece0 msgr2=0x7f409c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 shutdown_connections 2024-04-25T13:42:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:21.650+0000 7f40a0b84640 1 -- 172.21.15.94:0/2407211372 wait complete. 2024-04-25T13:42:21.662 INFO:tasks.cephfs.filesystem:scrub status for tag:7e82a34c-4efb-474c-b558-834b382518cb - {'path': '/', 'tag': '7e82a34c-4efb-474c-b558-834b382518cb', 'options': 'recursive,force'} 2024-04-25T13:42:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:22 smithi137 ceph-mon[32568]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:22 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:42:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:42:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:22 smithi094 ceph-mon[29195]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:22 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:42:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:42:22.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:22 smithi138 ceph-mon[32977]: pgmap v363: 97 pgs: 97 active+clean; 657 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:22 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:42:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:42:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:24 smithi137 ceph-mon[32568]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T13:42:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:24 smithi094 ceph-mon[29195]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T13:42:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:24 smithi138 ceph-mon[32977]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T13:42:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:26 smithi137 ceph-mon[32568]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T13:42:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:42:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:26 smithi094 ceph-mon[29195]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T13:42:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:42:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:26 smithi138 ceph-mon[32977]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T13:42:26.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 -- 172.21.15.94:0/3800315046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a20 msgr2=0x7fdc18103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 --2- 172.21.15.94:0/3800315046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a20 0x7fdc18103720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009900 tx=0x7fdc0802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 -- 172.21.15.94:0/3800315046 shutdown_connections 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 --2- 172.21.15.94:0/3800315046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108960 0x7fdc1810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 --2- 172.21.15.94:0/3800315046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18103c60 0x7fdc18104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 --2- 172.21.15.94:0/3800315046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a20 0x7fdc18103720 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 -- 172.21.15.94:0/3800315046 >> 172.21.15.94:0/3800315046 conn(0x7fdc180fdc10 msgr2=0x7fdc181000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 -- 172.21.15.94:0/3800315046 shutdown_connections 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.104+0000 7fdc1de20640 1 -- 172.21.15.94:0/3800315046 wait complete. 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 Processor -- start 2024-04-25T13:42:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 -- start start 2024-04-25T13:42:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 0x7fdc18199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108960 0x7fdc181a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810e520 con 0x7fdc18103c60 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810e3a0 con 0x7fdc18108960 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc1de20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810e6a0 con 0x7fdc18102a20 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36952/0 (socket says 172.21.15.94:36952) 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 -- 172.21.15.94:0/2624765855 learned_addr learned my addr 172.21.15.94:0/2624765855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc177fe640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 0x7fdc18199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108960 0x7fdc181a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 -- 172.21.15.94:0/2624765855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 msgr2=0x7fdc18199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 0x7fdc18199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 -- 172.21.15.94:0/2624765855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108960 msgr2=0x7fdc181a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108960 0x7fdc181a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.105+0000 7fdc16ffd640 1 -- 172.21.15.94:0/2624765855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc181a0cc0 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc177fe640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 0x7fdc18199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d950 tx=0x7fdc0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc14ff9640 1 -- 172.21.15.94:0/2624765855 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c017030 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc181a0fb0 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc14ff9640 1 -- 172.21.15.94:0/2624765855 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c004020 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc14ff9640 1 -- 172.21.15.94:0/2624765855 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0047a0 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.106+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc181a1470 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.108+0000 7fdc24dab740 1 Processor -- start 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.108+0000 7fdc24dab740 1 -- start start 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc14ff9640 1 -- 172.21.15.94:0/2624765855 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fdc0c004940 con 0x7fdc18103c60 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f6f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557c19ed5460 con 0x557c19bae4d0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557c19e8f590 con 0x557c19bcf1b0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc24dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c19ed1420 con 0x557c19ba62f0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36958/0 (socket says 172.21.15.94:36958) 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 -- 172.21.15.94:0/3296198069 learned_addr learned my addr 172.21.15.94:0/3296198069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc17fff640 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc177fe640 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f6f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 -- 172.21.15.94:0/3296198069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 msgr2=0x557c19f6f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f6f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 -- 172.21.15.94:0/3296198069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 msgr2=0x557c19f70ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f70ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 -- 172.21.15.94:0/3296198069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c19e9c0d0 con 0x557c19bae4d0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0dc250 tx=0x7fdc0c0e2e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3b823c90daf86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.109+0000 7fdc14ff9640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 0x7fdc000b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdbfdffb640 1 -- 172.21.15.94:0/3296198069 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0f1070 con 0x557c19bae4d0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 0x7fdc000b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdbfdffb640 1 -- 172.21.15.94:0/3296198069 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c0a67f0 con 0x557c19bae4d0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdbfdffb640 1 -- 172.21.15.94:0/3296198069 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0e5070 con 0x557c19bae4d0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc14ff9640 1 -- 172.21.15.94:0/2624765855 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fdc0c0dea00 con 0x7fdc18103c60 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc24dab740 1 -- 172.21.15.94:0/3296198069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 msgr2=0x557c19f6fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 0x7fdc000b2080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x557c19f70b00 tx=0x7fdc1000c330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc24dab740 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0dc250 tx=0x7fdc0c0e2e60 comp rx=0 tx=0).stop 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc24dab740 1 -- 172.21.15.94:0/3296198069 shutdown_connections 2024-04-25T13:42:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc24dab740 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6fc60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.110+0000 7fdc24dab740 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f6f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 --2- 172.21.15.94:0/3296198069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f70ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- 172.21.15.94:0/3296198069 >> 172.21.15.94:0/3296198069 conn(0x557c19bc3300 msgr2=0x557c19f6b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- 172.21.15.94:0/3296198069 shutdown_connections 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- 172.21.15.94:0/3296198069 wait complete. 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- start start 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 Processor -- start 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bae4d0 0x557c19f7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bcf1b0 0x557c19f84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557c19f7bcd0 con 0x557c19ba62f0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557c19e5cb60 con 0x557c19bae4d0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.111+0000 7fdc24dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c19f7be50 con 0x557c19bcf1b0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc24dab740 10 client.0 _ll_register_callbacks cb 0x557c19e812b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36974/0 (socket says 172.21.15.94:36974) 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 -- 172.21.15.94:0/4033248094 learned_addr learned my addr 172.21.15.94:0/4033248094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bae4d0 0x557c19f7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc17fff640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bcf1b0 0x557c19f84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 -- 172.21.15.94:0/4033248094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bcf1b0 msgr2=0x557c19f84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bcf1b0 0x557c19f84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 -- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bae4d0 msgr2=0x557c19f7df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bae4d0 0x557c19f7df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 -- 172.21.15.94:0/4033248094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c19be7210 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc177fe640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000c00 tx=0x7fdc080318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644036d2d69f3f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdbf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08035070 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0c0a67f0 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.112+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fdc08004240 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc08004440 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 10 client.15708 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c19cb3620 con 0x557c19ba62f0 2024-04-25T13:42:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c0a67f0 con 0x557c19ba62f0 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc08004cd0 con 0x557c19ba62f0 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 20 client.15708 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b8dd80 0x557c19f8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 4 client.15708 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c19ee3070 con 0x557c19b8dd80 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.113+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b8dd80 0x557c19f8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.114+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b8dd80 0x557c19f8b6c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6df89269fe3534 server_cookie=b63ec2da4bdceaca in_seq=0 out_seq=0 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.114+0000 7fdbf6ffd640 10 client.15708 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.114+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/4033248094 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c19ee3070 con 0x557c19b8dd80 2024-04-25T13:42:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.114+0000 7fdbf6ffd640 10 client.15708 handle_command_reply: tid=0 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.114+0000 7fdc24dab740 1 client.15708 shutdown 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 msgr2=0x557c19f7da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000c00 tx=0x7fdc080318a0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 shutdown_connections 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b8dd80 0x557c19f8b6c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bcf1b0 0x557c19f84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bae4d0 0x557c19f7df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 --2- 172.21.15.94:0/4033248094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f7da30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdbf6ffd640 0 client.15708 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 >> 172.21.15.94:0/4033248094 conn(0x557c19bc3300 msgr2=0x557c19f6e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 shutdown_connections 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 1 -- 172.21.15.94:0/4033248094 wait complete. 2024-04-25T13:42:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.115+0000 7fdc24dab740 20 client.15708 trim_cache size 0 max 16384 2024-04-25T13:42:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 Processor -- start 2024-04-25T13:42:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 -- start start 2024-04-25T13:42:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f78920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f7c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181a9440 con 0x557c19bae4d0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181aaf80 con 0x557c19bcf1b0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc24dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181a9b10 con 0x557c19ba62f0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc16ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36976/0 (socket says 172.21.15.94:36976) 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.130+0000 7fdc16ffd640 1 -- 172.21.15.94:0/1153096447 learned_addr learned my addr 172.21.15.94:0/1153096447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc177fe640 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f78920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 -- 172.21.15.94:0/1153096447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 msgr2=0x557c19f78920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f78920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 -- 172.21.15.94:0/1153096447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 msgr2=0x557c19f7c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f7c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 -- 172.21.15.94:0/1153096447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c19f6ed50 con 0x557c19bae4d0 2024-04-25T13:42:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005c60 tx=0x7fdc0c0c7f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4140ea2914ccc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdbfd7fa640 1 -- 172.21.15.94:0/1153096447 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c014070 con 0x557c19bae4d0 2024-04-25T13:42:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdbfd7fa640 1 -- 172.21.15.94:0/1153096447 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c08f040 con 0x557c19bae4d0 2024-04-25T13:42:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.131+0000 7fdbfd7fa640 1 -- 172.21.15.94:0/1153096447 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0e5040 con 0x557c19bae4d0 2024-04-25T13:42:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- 172.21.15.94:0/1153096447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 msgr2=0x557c19f6e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005c60 tx=0x7fdc0c0c7f90 comp rx=0 tx=0).stop 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- 172.21.15.94:0/1153096447 shutdown_connections 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f7c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19bae4d0 0x557c19f6e810 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- 172.21.15.94:0/1153096447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19ba62f0 0x557c19f78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- 172.21.15.94:0/1153096447 >> 172.21.15.94:0/1153096447 conn(0x557c19bc3300 msgr2=0x557c19f6e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- 172.21.15.94:0/1153096447 shutdown_connections 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- 172.21.15.94:0/1153096447 wait complete. 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 -- start start 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 Processor -- start 2024-04-25T13:42:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.132+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bae4d0 0x557c19f81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181aa8b0 con 0x557c19ba62f0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181aa1e0 con 0x557c19bcf1b0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181a7420 con 0x557c19bae4d0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 10 client.0 _ll_register_callbacks cb 0x7fdc08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36990/0 (socket says 172.21.15.94:36990) 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 -- 172.21.15.94:0/98192655 learned_addr learned my addr 172.21.15.94:0/98192655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bae4d0 0x557c19f81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc17fff640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 -- 172.21.15.94:0/98192655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bae4d0 msgr2=0x557c19f81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bae4d0 0x557c19f81d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 -- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 msgr2=0x557c19f82b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f82b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 -- 172.21.15.94:0/98192655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c19f83150 con 0x557c19ba62f0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc177fe640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdc080037e0 tx=0x7fdc0802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcda2bfe3bb0253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdbf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08035040 con 0x557c19ba62f0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fdc08003ef0 con 0x557c19ba62f0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0c08f040 con 0x557c19ba62f0 2024-04-25T13:42:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc0805d050 con 0x557c19ba62f0 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.133+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c19f844b0 con 0x557c19ba62f0 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c08f040 con 0x557c19ba62f0 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc08044b00 con 0x557c19ba62f0 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 20 client.15714 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b631c0 0x557c19ff5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 4 client.15714 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:42:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c19edcee0 con 0x557c19b631c0 2024-04-25T13:42:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.134+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b631c0 0x557c19ff5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.135+0000 7fdc16ffd640 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b631c0 0x557c19ff5f10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c82d47e308e86f7 server_cookie=ece2c28f031f2f75 in_seq=0 out_seq=0 2024-04-25T13:42:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.135+0000 7fdbf6ffd640 10 client.15714 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.142+0000 7fdbf6ffd640 1 -- 172.21.15.94:0/98192655 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c19edcee0 con 0x557c19b631c0 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.142+0000 7fdbf6ffd640 10 client.15714 handle_command_reply: tid=0 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.142+0000 7fdc24dab740 1 client.15714 shutdown 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.142+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 msgr2=0x557c19f817e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.142+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdc080037e0 tx=0x7fdc0802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 shutdown_connections 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c19b631c0 0x557c19ff5f10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c19bcf1b0 0x557c19f82b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c19bae4d0 0x557c19f81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 --2- 172.21.15.94:0/98192655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c19ba62f0 0x557c19f817e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdbf6ffd640 0 client.15714 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 >> 172.21.15.94:0/98192655 conn(0x557c19bc3300 msgr2=0x557c19f7c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 shutdown_connections 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 1 -- 172.21.15.94:0/98192655 wait complete. 2024-04-25T13:42:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc24dab740 20 client.15714 trim_cache size 0 max 16384 2024-04-25T13:42:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:42:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:42:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:42:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 msgr2=0x7fdc000b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 0x7fdc000b2080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x557c19f70b00 tx=0x7fdc1000c330 comp rx=0 tx=0).stop 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 msgr2=0x7fdc181996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00d950 tx=0x7fdc0c00de10 comp rx=0 tx=0).stop 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 shutdown_connections 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc000afb70 0x7fdc000b2080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108960 0x7fdc181a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c60 0x7fdc181996c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 --2- 172.21.15.94:0/2624765855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18102a20 0x7fdc18199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 >> 172.21.15.94:0/2624765855 conn(0x7fdc180fdc10 msgr2=0x7fdc18104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 shutdown_connections 2024-04-25T13:42:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:28.143+0000 7fdc1de20640 1 -- 172.21.15.94:0/2624765855 wait complete. 2024-04-25T13:42:28.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:28 smithi137 ceph-mon[32568]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T13:42:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:28 smithi094 ceph-mon[29195]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T13:42:28.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:28 smithi138 ceph-mon[32977]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T13:42:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:30 smithi137 ceph-mon[32568]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T13:42:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:30 smithi094 ceph-mon[29195]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T13:42:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:30 smithi138 ceph-mon[32977]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-25T13:42:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:32 smithi137 ceph-mon[32568]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-25T13:42:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:32 smithi094 ceph-mon[29195]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-25T13:42:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:32 smithi138 ceph-mon[32977]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-25T13:42:34.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:34 smithi137 ceph-mon[32568]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T13:42:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:34 smithi094 ceph-mon[29195]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T13:42:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:34 smithi138 ceph-mon[32977]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T13:42:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:36 smithi137 ceph-mon[32568]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:36 smithi094 ceph-mon[29195]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:36 smithi138 ceph-mon[32977]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:38 smithi137 ceph-mon[32568]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:38 smithi094 ceph-mon[29195]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:38.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:38 smithi138 ceph-mon[32977]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-25T13:42:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:40 smithi137 ceph-mon[32568]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T13:42:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:40 smithi094 ceph-mon[29195]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T13:42:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:40 smithi138 ceph-mon[32977]: pgmap v372: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 -- 172.21.15.94:0/71162574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c1098d0 msgr2=0x7fa01c10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 --2- 172.21.15.94:0/71162574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c1098d0 0x7fa01c10bcf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa014009ca0 tx=0x7fa01402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 -- 172.21.15.94:0/71162574 shutdown_connections 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 --2- 172.21.15.94:0/71162574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c1098d0 0x7fa01c10bcf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 --2- 172.21.15.94:0/71162574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c106f70 0x7fa01c109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 --2- 172.21.15.94:0/71162574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c104610 0x7fa01c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.243+0000 7fa022164640 1 -- 172.21.15.94:0/71162574 >> 172.21.15.94:0/71162574 conn(0x7fa01c0fdc10 msgr2=0x7fa01c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.244+0000 7fa022164640 1 -- 172.21.15.94:0/71162574 shutdown_connections 2024-04-25T13:42:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.244+0000 7fa022164640 1 -- 172.21.15.94:0/71162574 wait complete. 2024-04-25T13:42:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.244+0000 7fa022164640 1 Processor -- start 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.244+0000 7fa022164640 1 -- start start 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c106f70 0x7fa01c102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 0x7fa01c102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10f630 con 0x7fa01c104610 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10f4b0 con 0x7fa01c106f70 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c10f7b0 con 0x7fa01c1098d0 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42098/0 (socket says 172.21.15.94:42098) 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 -- 172.21.15.94:0/4062146968 learned_addr learned my addr 172.21.15.94:0/4062146968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01affd640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c106f70 0x7fa01c102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 0x7fa01c102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 -- 172.21.15.94:0/4062146968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 msgr2=0x7fa01c102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 0x7fa01c102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 -- 172.21.15.94:0/4062146968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c106f70 msgr2=0x7fa01c102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c106f70 0x7fa01c102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 -- 172.21.15.94:0/4062146968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c102ce0 con 0x7fa01c104610 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 0x7fa01c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c970 tx=0x7fa00c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa018ff9640 1 -- 172.21.15.94:0/4062146968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c007bb0 con 0x7fa01c104610 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.245+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01c1a42c0 con 0x7fa01c104610 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.246+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01c1a4720 con 0x7fa01c104610 2024-04-25T13:42:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.246+0000 7fa018ff9640 1 -- 172.21.15.94:0/4062146968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00c007d50 con 0x7fa01c104610 2024-04-25T13:42:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.246+0000 7fa018ff9640 1 -- 172.21.15.94:0/4062146968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01d060 con 0x7fa01c104610 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 Processor -- start 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 -- start start 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186aecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556186a52460 con 0x55618672b4d0 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556186a0c590 con 0x55618674c1b0 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa029014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556186a4e420 con 0x5561867232f0 2024-04-25T13:42:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42110/0 (socket says 172.21.15.94:42110) 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 -- 172.21.15.94:0/3237868204 learned_addr learned my addr 172.21.15.94:0/3237868204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa018ff9640 1 -- 172.21.15.94:0/4062146968 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fa00c004cf0 con 0x7fa01c104610 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01bfff640 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 -- 172.21.15.94:0/3237868204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 msgr2=0x556186aecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186aecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 -- 172.21.15.94:0/3237868204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 msgr2=0x556186aec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.249+0000 7fa01affd640 1 -- 172.21.15.94:0/3237868204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556186a190d0 con 0x55618672b4d0 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa01affd640 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ac80 tx=0x7fa01000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920e9db3b2eed1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa018ff9640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 0x7fa0040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa001ffb640 1 -- 172.21.15.94:0/3237868204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01001d070 con 0x55618672b4d0 2024-04-25T13:42:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa001ffb640 1 -- 172.21.15.94:0/3237868204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010018af0 con 0x55618672b4d0 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa018ff9640 1 -- 172.21.15.94:0/4062146968 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa00c0deb20 con 0x7fa01c104610 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 0x7fa0040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.250+0000 7fa001ffb640 1 -- 172.21.15.94:0/3237868204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010010b40 con 0x55618672b4d0 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- 172.21.15.94:0/3237868204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 msgr2=0x556186af3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 0x7fa0040b2080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x556186aedb40 tx=0x7fa014031a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ac80 tx=0x7fa01000f790 comp rx=0 tx=0).stop 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- 172.21.15.94:0/3237868204 shutdown_connections 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af3df0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186aecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 --2- 172.21.15.94:0/3237868204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- 172.21.15.94:0/3237868204 >> 172.21.15.94:0/3237868204 conn(0x556186740300 msgr2=0x556186ae8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- 172.21.15.94:0/3237868204 shutdown_connections 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- 172.21.15.94:0/3237868204 wait complete. 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.251+0000 7fa029014740 1 -- start start 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 Processor -- start 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186af67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186afdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556186af48e0 con 0x55618672b4d0 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5561869d9b60 con 0x55618674c1b0 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556186af4a60 con 0x5561867232f0 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa029014740 10 client.0 _ll_register_callbacks cb 0x5561869fe2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42126/0 (socket says 172.21.15.94:42126) 2024-04-25T13:42:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01affd640 1 -- 172.21.15.94:0/4289089494 learned_addr learned my addr 172.21.15.94:0/4289089494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186af67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186afdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01affd640 1 -- 172.21.15.94:0/4289089494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 msgr2=0x556186af67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.252+0000 7fa01affd640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186af67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01affd640 1 -- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 msgr2=0x556186afdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01affd640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186afdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01affd640 1 -- 172.21.15.94:0/4289089494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556186764210 con 0x55618672b4d0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01bfff640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186afdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186af67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa01affd640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa01000a670 tx=0x7fa010018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca528e103a92a0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7f9ffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010027020 con 0x55618672b4d0 2024-04-25T13:42:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa010018830 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa010018af0 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa010032050 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa029014740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556186830620 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.253+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010018af0 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa0100046c0 con 0x55618672b4d0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 20 client.15726 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55618670ad80 0x556186b04110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 4 client.15726 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556186a60070 con 0x55618670ad80 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.254+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55618670ad80 0x556186b04110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.255+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55618670ad80 0x556186b04110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975ce398495547a server_cookie=e21833debd8d93de in_seq=0 out_seq=0 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.255+0000 7f9ffaffd640 10 client.15726 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.255+0000 7f9ffaffd640 1 -- 172.21.15.94:0/4289089494 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556186a60070 con 0x55618670ad80 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.255+0000 7f9ffaffd640 10 client.15726 handle_command_reply: tid=0 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.255+0000 7fa029014740 1 client.15726 shutdown 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 msgr2=0x556186af6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa01000a670 tx=0x7fa010018440 comp rx=0 tx=0).stop 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 shutdown_connections 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55618670ad80 0x556186b04110 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186afdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186af6ce0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7f9ffaffd640 0 client.15726 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 --2- 172.21.15.94:0/4289089494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186af67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 >> 172.21.15.94:0/4289089494 conn(0x556186740300 msgr2=0x556186ae7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 shutdown_connections 2024-04-25T13:42:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 1 -- 172.21.15.94:0/4289089494 wait complete. 2024-04-25T13:42:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.256+0000 7fa029014740 20 client.15726 trim_cache size 0 max 16384 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.275+0000 7fa029014740 1 Processor -- start 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.275+0000 7fa029014740 1 -- start start 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aeb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800e730 con 0x55618672b4d0 2024-04-25T13:42:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa008010270 con 0x55618674c1b0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa029014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800ee00 con 0x5561867232f0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42136/0 (socket says 172.21.15.94:42136) 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 -- 172.21.15.94:0/2034777009 learned_addr learned my addr 172.21.15.94:0/2034777009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aeb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57620/0 (socket says 172.21.15.94:57620) 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 -- 172.21.15.94:0/2034777009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 msgr2=0x556186ae83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 -- 172.21.15.94:0/2034777009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 msgr2=0x556186aeb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aeb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 -- 172.21.15.94:0/2034777009 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556186af0600 con 0x55618672b4d0 2024-04-25T13:42:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01bfff640 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aeb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.276+0000 7fa01affd640 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ac80 tx=0x7fa010010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550a34c7cdf6f7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.277+0000 7fa0017fa640 1 -- 172.21.15.94:0/2034777009 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010027070 con 0x55618672b4d0 2024-04-25T13:42:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.277+0000 7fa0017fa640 1 -- 172.21.15.94:0/2034777009 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010004860 con 0x55618672b4d0 2024-04-25T13:42:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.277+0000 7fa0017fa640 1 -- 172.21.15.94:0/2034777009 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01001d040 con 0x55618672b4d0 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- 172.21.15.94:0/2034777009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 msgr2=0x556186aeb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ac80 tx=0x7fa010010640 comp rx=0 tx=0).stop 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- 172.21.15.94:0/2034777009 shutdown_connections 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186aeb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186aeb100 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- 172.21.15.94:0/2034777009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186ae83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- 172.21.15.94:0/2034777009 >> 172.21.15.94:0/2034777009 conn(0x556186740300 msgr2=0x556186ae7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- 172.21.15.94:0/2034777009 shutdown_connections 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- 172.21.15.94:0/2034777009 wait complete. 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- start start 2024-04-25T13:42:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 Processor -- start 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186b6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800fba0 con 0x55618672b4d0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800f4d0 con 0x55618674c1b0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.278+0000 7fa029014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800c710 con 0x5561867232f0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa029014740 10 client.0 _ll_register_callbacks cb 0x7fa01001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42150/0 (socket says 172.21.15.94:42150) 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57630/0 (socket says 172.21.15.94:57630) 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01affd640 1 -- 172.21.15.94:0/2558198004 learned_addr learned my addr 172.21.15.94:0/2558198004 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01bfff640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186b6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 -- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 msgr2=0x556186b6a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186b6a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 -- 172.21.15.94:0/2558198004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 msgr2=0x556186afa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 -- 172.21.15.94:0/2558198004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556186b6a9e0 con 0x5561867232f0 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01bfff640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186b6a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01affd640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa01b7fe640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0dc430 tx=0x7fa00c0de040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2cbedf9e9c2d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7f9ffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:42:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c005220 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.279+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa010004860 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa00c0a6160 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa00c005590 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 10 client.25201 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556186b6bd40 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010004860 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa00c0be020 con 0x5561867232f0 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 10 client.25201 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 10 client.25201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 20 client.25201 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5561866e01c0 0x556186b6e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 4 client.25201 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:42:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556186a59ee0 con 0x5561866e01c0 2024-04-25T13:42:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.280+0000 7fa01affd640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5561866e01c0 0x556186b6e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.281+0000 7fa01affd640 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5561866e01c0 0x556186b6e710 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8480b2ac28d57b6 server_cookie=6c33c31743cce0e1 in_seq=0 out_seq=0 2024-04-25T13:42:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.281+0000 7f9ffaffd640 10 client.25201 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7f9ffaffd640 1 -- 172.21.15.94:0/2558198004 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556186a59ee0 con 0x5561866e01c0 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7f9ffaffd640 10 client.25201 handle_command_reply: tid=0 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 client.25201 shutdown 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 msgr2=0x556186afa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0dc430 tx=0x7fa00c0de040 comp rx=0 tx=0).stop 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 shutdown_connections 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5561866e01c0 0x556186b6e710 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618674c1b0 0x556186b6a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55618672b4d0 0x556186afa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7f9ffaffd640 0 client.25201 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 --2- 172.21.15.94:0/2558198004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561867232f0 0x556186afa2a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 >> 172.21.15.94:0/2558198004 conn(0x556186740300 msgr2=0x556186ae6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 shutdown_connections 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 1 -- 172.21.15.94:0/2558198004 wait complete. 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.285+0000 7fa029014740 20 client.25201 trim_cache size 0 max 16384 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:42:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 msgr2=0x7fa0040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 0x7fa0040b2080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x556186aedb40 tx=0x7fa014031a20 comp rx=0 tx=0).stop 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 msgr2=0x7fa01c103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c970 tx=0x7fa00c00cd70 comp rx=0 tx=0).stop 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 shutdown_connections 2024-04-25T13:42:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0040afb70 0x7fa0040b2080 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01c1098d0 0x7fa01c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa01c106f70 0x7fa01c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 --2- 172.21.15.94:0/4062146968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa01c104610 0x7fa01c103a60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 >> 172.21.15.94:0/4062146968 conn(0x7fa01c0fdc10 msgr2=0x7fa01c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 shutdown_connections 2024-04-25T13:42:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:41.286+0000 7fa022164640 1 -- 172.21.15.94:0/4062146968 wait complete. 2024-04-25T13:42:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:42 smithi137 ceph-mon[32568]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:42 smithi094 ceph-mon[29195]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:42 smithi138 ceph-mon[32977]: pgmap v373: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T13:42:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:44 smithi137 ceph-mon[32568]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:42:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:44 smithi094 ceph-mon[29195]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:42:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:44 smithi138 ceph-mon[32977]: pgmap v374: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:42:46.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:46 smithi137 ceph-mon[32568]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:42:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:46 smithi094 ceph-mon[29195]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:42:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:46 smithi138 ceph-mon[32977]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T13:42:48.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:48 smithi137 ceph-mon[32568]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:48 smithi094 ceph-mon[29195]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:48 smithi138 ceph-mon[32977]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:50 smithi137 ceph-mon[32568]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:50 smithi094 ceph-mon[29195]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:50 smithi138 ceph-mon[32977]: pgmap v377: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:42:51.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:42:51.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:42:51.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 -- 172.21.15.94:0/1923466593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 msgr2=0x7f55781054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 --2- 172.21.15.94:0/1923466593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781054b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f556802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 -- 172.21.15.94:0/1923466593 shutdown_connections 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 --2- 172.21.15.94:0/1923466593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578102250 0x7f557810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 --2- 172.21.15.94:0/1923466593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578101750 0x7f5578101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 --2- 172.21.15.94:0/1923466593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781054b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 -- 172.21.15.94:0/1923466593 >> 172.21.15.94:0/1923466593 conn(0x7f55780fd350 msgr2=0x7f55780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 -- 172.21.15.94:0/1923466593 shutdown_connections 2024-04-25T13:42:51.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.783+0000 7f557c984640 1 -- 172.21.15.94:0/1923466593 wait complete. 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 Processor -- start 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 -- start start 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578101750 0x7f557819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578102250 0x7f557819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.784+0000 7f557c984640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f557810f660 con 0x7f55781050a0 2024-04-25T13:42:51.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f557c984640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f557810f4e0 con 0x7f5578101750 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f557c984640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f557810f7e0 con 0x7f5578102250 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58376/0 (socket says 172.21.15.94:58376) 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 -- 172.21.15.94:0/1005735047 learned_addr learned my addr 172.21.15.94:0/1005735047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f557659c640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578101750 0x7f557819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5575d9b640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578102250 0x7f557819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 -- 172.21.15.94:0/1005735047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578102250 msgr2=0x7f557819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578102250 0x7f557819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 -- 172.21.15.94:0/1005735047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578101750 msgr2=0x7f557819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578101750 0x7f557819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 -- 172.21.15.94:0/1005735047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55781a1da0 con 0x7f55781050a0 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.785+0000 7f5576d9d640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f557000eb10 tx=0x7f557000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.786+0000 7f55677fe640 1 -- 172.21.15.94:0/1005735047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570004b20 con 0x7f55781050a0 2024-04-25T13:42:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.786+0000 7f55677fe640 1 -- 172.21.15.94:0/1005735047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5570004cc0 con 0x7f55781050a0 2024-04-25T13:42:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.786+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781a2090 con 0x7f55781050a0 2024-04-25T13:42:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.786+0000 7f55677fe640 1 -- 172.21.15.94:0/1005735047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570010670 con 0x7f55781050a0 2024-04-25T13:42:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.786+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a25e0 con 0x7f55781050a0 2024-04-25T13:42:51.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f55677fe640 1 -- 172.21.15.94:0/1005735047 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f5570005000 con 0x7f55781050a0 2024-04-25T13:42:51.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 Processor -- start 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 -- start start 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aee4733d80 con 0x55aee442d390 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aee46ed8b0 con 0x55aee440c6b0 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f558383a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aee472fd40 con 0x55aee44044d0 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.789+0000 7f55677fe640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 0x7f55600b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f5575d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55968/0 (socket says 172.21.15.94:55968) 2024-04-25T13:42:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 -- 172.21.15.94:0/3431163561 learned_addr learned my addr 172.21.15.94:0/3431163561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f55677fe640 1 -- 172.21.15.94:0/1005735047 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55700df410 con 0x7f55781050a0 2024-04-25T13:42:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f5576d9d640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 0x7f55600b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 -- 172.21.15.94:0/3431163561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 msgr2=0x55aee47cfc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cfc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 -- 172.21.15.94:0/3431163561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 msgr2=0x55aee47cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f5576d9d640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 0x7f55600b2190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f55700a7620 tx=0x7f557000cee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f557659c640 1 -- 172.21.15.94:0/3431163561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee46bea40 con 0x55aee44044d0 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.790+0000 7f5575d9b640 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.791+0000 7f557659c640 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f5568004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93fa3a6152372804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.791+0000 7f5564ff9640 1 -- 172.21.15.94:0/3431163561 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568035030 con 0x55aee44044d0 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.791+0000 7f5564ff9640 1 -- 172.21.15.94:0/3431163561 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556803fc90 con 0x55aee44044d0 2024-04-25T13:42:51.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.791+0000 7f5564ff9640 1 -- 172.21.15.94:0/3431163561 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568003cd0 con 0x55aee44044d0 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 -- 172.21.15.94:0/3431163561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 msgr2=0x55aee47cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f5568004040 comp rx=0 tx=0).stop 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 -- 172.21.15.94:0/3431163561 shutdown_connections 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47cefe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 --2- 172.21.15.94:0/3431163561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 -- 172.21.15.94:0/3431163561 >> 172.21.15.94:0/3431163561 conn(0x55aee44214e0 msgr2=0x55aee47ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 -- 172.21.15.94:0/3431163561 shutdown_connections 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.792+0000 7f558383a740 1 -- 172.21.15.94:0/3431163561 wait complete. 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.793+0000 7f558383a740 1 -- start start 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.793+0000 7f558383a740 1 Processor -- start 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58402/0 (socket says 172.21.15.94:58402) 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 -- 172.21.15.94:0/4249859770 learned_addr learned my addr 172.21.15.94:0/4249859770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee442d390 0x55aee47dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f5575d9b640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aee47d0720 con 0x55aee440c6b0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aee46bad10 con 0x55aee442d390 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aee47d08a0 con 0x55aee44044d0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f558383a740 10 client.0 _ll_register_callbacks cb 0x55aee46e7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f5575d9b640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee442d390 0x55aee47dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 -- 172.21.15.94:0/4249859770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 msgr2=0x55aee47d88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47d88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 -- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee442d390 msgr2=0x55aee47dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee442d390 0x55aee47dfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.794+0000 7f557659c640 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee450f7d0 con 0x55aee440c6b0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f557659c640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f556802aa00 tx=0x7f5568005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2165a522ae9611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f555dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568035030 con 0x55aee440c6b0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f556803fc90 con 0x55aee440c6b0 2024-04-25T13:42:51.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f556803f7e0 con 0x55aee440c6b0 2024-04-25T13:42:51.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f556803f9e0 con 0x55aee440c6b0 2024-04-25T13:42:51.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f558383a740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:51.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.795+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aee44453f0 con 0x55aee440c6b0 2024-04-25T13:42:51.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556803fc90 con 0x55aee440c6b0 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5568046070 con 0x55aee440c6b0 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 20 client.15744 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43ebf60 0x55aee47e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 4 client.15744 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.796+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee4741990 con 0x55aee43ebf60 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.797+0000 7f5575d9b640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43ebf60 0x55aee47e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.797+0000 7f555dffb640 10 client.15744 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.797+0000 7f5575d9b640 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43ebf60 0x55aee47e5df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9928d2da2d37bc server_cookie=57f2190c4ece03ec in_seq=0 out_seq=0 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f555dffb640 1 -- 172.21.15.94:0/4249859770 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aee4741990 con 0x55aee43ebf60 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f555dffb640 10 client.15744 handle_command_reply: tid=0 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 client.15744 shutdown 2024-04-25T13:42:51.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 msgr2=0x55aee47d8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f556802aa00 tx=0x7f5568005db0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 shutdown_connections 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43ebf60 0x55aee47e5df0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee442d390 0x55aee47dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee440c6b0 0x55aee47d8e30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 --2- 172.21.15.94:0/4249859770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 >> 172.21.15.94:0/4249859770 conn(0x55aee44214e0 msgr2=0x55aee47cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f555dffb640 0 client.15744 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 shutdown_connections 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 1 -- 172.21.15.94:0/4249859770 wait complete. 2024-04-25T13:42:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.798+0000 7f558383a740 20 client.15744 trim_cache size 0 max 16384 2024-04-25T13:42:51.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 Processor -- start 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 -- start start 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55781a9340 con 0x55aee442d390 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55781aae80 con 0x55aee440c6b0 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.816+0000 7f558383a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55781a9a10 con 0x55aee44044d0 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f5575d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f5575d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58418/0 (socket says 172.21.15.94:58418) 2024-04-25T13:42:51.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f5575d9b640 1 -- 172.21.15.94:0/1085171916 learned_addr learned my addr 172.21.15.94:0/1085171916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f5575d9b640 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 -- 172.21.15.94:0/1085171916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 msgr2=0x55aee47ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 -- 172.21.15.94:0/1085171916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 msgr2=0x55aee47cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 -- 172.21.15.94:0/1085171916 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee44050f0 con 0x55aee440c6b0 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f5575d9b640 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f557659c640 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cd070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f5568031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e959f5087c6f1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.817+0000 7f555ffff640 1 -- 172.21.15.94:0/1085171916 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568035070 con 0x55aee440c6b0 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f555ffff640 1 -- 172.21.15.94:0/1085171916 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5568055070 con 0x55aee440c6b0 2024-04-25T13:42:51.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f555ffff640 1 -- 172.21.15.94:0/1085171916 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568003a60 con 0x55aee440c6b0 2024-04-25T13:42:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 -- 172.21.15.94:0/1085171916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 msgr2=0x55aee47cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cd070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f5568031400 comp rx=0 tx=0).stop 2024-04-25T13:42:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 -- 172.21.15.94:0/1085171916 shutdown_connections 2024-04-25T13:42:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee442d390 0x55aee47cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47cd070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.818+0000 7f558383a740 1 --2- 172.21.15.94:0/1085171916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee44044d0 0x55aee47ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.819+0000 7f558383a740 1 -- 172.21.15.94:0/1085171916 >> 172.21.15.94:0/1085171916 conn(0x55aee44214e0 msgr2=0x55aee47c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.819+0000 7f558383a740 1 -- 172.21.15.94:0/1085171916 shutdown_connections 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.819+0000 7f558383a740 1 -- 172.21.15.94:0/1085171916 wait complete. 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.819+0000 7f558383a740 1 -- start start 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.819+0000 7f558383a740 1 Processor -- start 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee442d390 0x55aee47dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55781aa7b0 con 0x55aee44044d0 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55781aa0e0 con 0x55aee440c6b0 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55781a7320 con 0x55aee442d390 2024-04-25T13:42:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58432/0 (socket says 172.21.15.94:58432) 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 -- 172.21.15.94:0/3682871941 learned_addr learned my addr 172.21.15.94:0/3682871941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f557659c640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee442d390 0x55aee47dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 -- 172.21.15.94:0/3682871941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee442d390 msgr2=0x55aee47dc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee442d390 0x55aee47dc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 -- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 msgr2=0x55aee47dbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47dbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f5575d9b640 1 -- 172.21.15.94:0/3682871941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee46f44b0 con 0x55aee44044d0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 10 client.0 _ll_register_callbacks cb 0x7f556c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.820+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5568055070 con 0x55aee44044d0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f5575d9b640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f556c0041e0 tx=0x7f556c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d4ca7cab2df79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f555dffb640 10 client.15750 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f555dffb640 1 -- 172.21.15.94:0/3682871941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c003b00 con 0x55aee44044d0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f555dffb640 1 -- 172.21.15.94:0/3682871941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f556c020c70 con 0x55aee44044d0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f555dffb640 1 -- 172.21.15.94:0/3682871941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f556c003ca0 con 0x55aee44044d0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55aee484cf80 con 0x55aee44044d0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f555dffb640 1 -- 172.21.15.94:0/3682871941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f556c010430 con 0x55aee44044d0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 20 client.15750 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43c13a0 0x55aee484f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 4 client.15750 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.821+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aee473b800 con 0x55aee43c13a0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f557659c640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43c13a0 0x55aee484f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f557659c640 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43c13a0 0x55aee484f860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beac18232e1bf589 server_cookie=10de40f79b7dc0ff in_seq=0 out_seq=0 2024-04-25T13:42:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f555dffb640 10 client.15750 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f555dffb640 1 -- 172.21.15.94:0/3682871941 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aee473b800 con 0x55aee43c13a0 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f555dffb640 10 client.15750 handle_command_reply: tid=0 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f558383a740 1 client.15750 shutdown 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 msgr2=0x55aee47df210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.822+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f556c0041e0 tx=0x7f556c005f90 comp rx=0 tx=0).stop 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 shutdown_connections 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aee43c13a0 0x55aee484f860 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aee442d390 0x55aee47dc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aee440c6b0 0x55aee47dbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f555dffb640 0 client.15750 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 --2- 172.21.15.94:0/3682871941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aee44044d0 0x55aee47df210 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 >> 172.21.15.94:0/3682871941 conn(0x55aee44214e0 msgr2=0x55aee47c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 shutdown_connections 2024-04-25T13:42:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 1 -- 172.21.15.94:0/3682871941 wait complete. 2024-04-25T13:42:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f558383a740 20 client.15750 trim_cache size 0 max 16384 2024-04-25T13:42:51.828 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:51.828 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:42:51.828 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:42:51.828 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 msgr2=0x7f55600b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 0x7f55600b2190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f55700a7620 tx=0x7f557000cee0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 msgr2=0x7f55781a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f557000eb10 tx=0x7f557000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 shutdown_connections 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55600afc80 0x7f55600b2190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55781050a0 0x7f55781a1730 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5578102250 0x7f557819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 --2- 172.21.15.94:0/1005735047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5578101750 0x7f557819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.823+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 >> 172.21.15.94:0/1005735047 conn(0x7f55780fd350 msgr2=0x7f5578105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.824+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 shutdown_connections 2024-04-25T13:42:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.824+0000 7f557c984640 1 -- 172.21.15.94:0/1005735047 wait complete. 2024-04-25T13:42:51.835 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:42:51.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 -- 172.21.15.94:0/3848836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc105080 msgr2=0x7f9edc105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/3848836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc105080 0x7f9edc105490 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802a410 comp rx=0 tx=0).stop 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 -- 172.21.15.94:0/3848836104 shutdown_connections 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/3848836104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/3848836104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc101730 0x7f9edc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/3848836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc105080 0x7f9edc105490 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 -- 172.21.15.94:0/3848836104 >> 172.21.15.94:0/3848836104 conn(0x7f9edc0fd330 msgr2=0x7f9edc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 -- 172.21.15.94:0/3848836104 shutdown_connections 2024-04-25T13:42:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.947+0000 7f9ee34bd640 1 -- 172.21.15.94:0/3848836104 wait complete. 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 Processor -- start 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 -- start start 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc19f770 0x7f9edc1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10f5f0 con 0x7f9edc101730 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10f470 con 0x7f9edc19f770 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee34bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10f770 con 0x7f9edc102230 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58462/0 (socket says 172.21.15.94:58462) 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 -- 172.21.15.94:0/2897194382 learned_addr learned my addr 172.21.15.94:0/2897194382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee0a58640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1a5a640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc19f770 0x7f9edc1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 -- 172.21.15.94:0/2897194382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 msgr2=0x7f9edc19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 -- 172.21.15.94:0/2897194382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc19f770 msgr2=0x7f9edc1a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc19f770 0x7f9edc1a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee1259640 1 -- 172.21.15.94:0/2897194382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc102830 con 0x7f9edc101730 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.948+0000 7f9ee0a58640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ee1259640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed803eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8035070 con 0x7f9edc101730 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc102ac0 con 0x7f9edc101730 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed8031910 con 0x7f9edc101730 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8048680 con 0x7f9edc101730 2024-04-25T13:42:51.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.949+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc102fb0 con 0x7f9edc101730 2024-04-25T13:42:51.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.951+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc02db50 con 0x7f9edc101730 2024-04-25T13:42:51.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.963+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f9ed80035f0 con 0x7f9edc101730 2024-04-25T13:42:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.964+0000 7f9ed27fc640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 0x7f9ec80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.964+0000 7f9ee0a58640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 0x7f9ec80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.964+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9ed8106d90 con 0x7f9edc101730 2024-04-25T13:42:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.964+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ed810b3c0 con 0x7f9edc101730 2024-04-25T13:42:51.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:51.965+0000 7f9ee0a58640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 0x7f9ec80b2110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc005d00 tx=0x7f9ecc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:52 smithi137 ceph-mon[32568]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:52.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.411+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9edc108580 con 0x7f9edc101730 2024-04-25T13:42:52.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.412+0000 7f9ed27fc640 1 -- 172.21.15.94:0/2897194382 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7f9ed80043f0 con 0x7f9edc101730 2024-04-25T13:42:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:42:52.412 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:42:52.412 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:42:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 msgr2=0x7f9ec80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 0x7f9ec80b2110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc005d00 tx=0x7f9ecc005ed0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 msgr2=0x7f9edc19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8006010 tx=0x7f9ed803eca0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 shutdown_connections 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9ec80afc00 0x7f9ec80b2110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9edc19f770 0x7f9edc1a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9edc102230 0x7f9edc19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 --2- 172.21.15.94:0/2897194382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc101730 0x7f9edc19a2b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 >> 172.21.15.94:0/2897194382 conn(0x7f9edc0fd330 msgr2=0x7f9edc105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 shutdown_connections 2024-04-25T13:42:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.415+0000 7f9ee34bd640 1 -- 172.21.15.94:0/2897194382 wait complete. 2024-04-25T13:42:52.429 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:52 smithi094 ceph-mon[29195]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:52.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:42:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:52 smithi138 ceph-mon[32977]: pgmap v378: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.600+0000 7f7fd680c640 1 -- 172.21.15.94:0/3076006046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 msgr2=0x7f7fd010bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.600+0000 7f7fd680c640 1 --2- 172.21.15.94:0/3076006046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd010bc80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009ca0 tx=0x7f7fc802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 -- 172.21.15.94:0/3076006046 shutdown_connections 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 --2- 172.21.15.94:0/3076006046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd010bc80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 --2- 172.21.15.94:0/3076006046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 0x7f7fd01090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 --2- 172.21.15.94:0/3076006046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 0x7f7fd0106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 -- 172.21.15.94:0/3076006046 >> 172.21.15.94:0/3076006046 conn(0x7f7fd00fdbd0 msgr2=0x7f7fd0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 -- 172.21.15.94:0/3076006046 shutdown_connections 2024-04-25T13:42:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 -- 172.21.15.94:0/3076006046 wait complete. 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.601+0000 7f7fd680c640 1 Processor -- start 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 -- start start 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 0x7f7fd019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 0x7f7fd019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010f650 con 0x7f7fd0109770 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010f4d0 con 0x7f7fd01041f0 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd680c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010f7d0 con 0x7f7fd0106c00 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd4da9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd4da9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58492/0 (socket says 172.21.15.94:58492) 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fd4da9640 1 -- 172.21.15.94:0/4044196369 learned_addr learned my addr 172.21.15.94:0/4044196369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 0x7f7fd019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.602+0000 7f7fcffff640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 0x7f7fd019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 -- 172.21.15.94:0/4044196369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 msgr2=0x7f7fd019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 0x7f7fd019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 -- 172.21.15.94:0/4044196369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 msgr2=0x7f7fd019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 0x7f7fd019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 -- 172.21.15.94:0/4044196369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01a2020 con 0x7f7fd0109770 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd4da9640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009430 tx=0x7f7fc80316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fcd7fa640 1 -- 172.21.15.94:0/4044196369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003b70 con 0x7f7fd0109770 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fcd7fa640 1 -- 172.21.15.94:0/4044196369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8003d10 con 0x7f7fd0109770 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd01a22b0 con 0x7f7fd0109770 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fcd7fa640 1 -- 172.21.15.94:0/4044196369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc802fb80 con 0x7f7fd0109770 2024-04-25T13:42:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.603+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd01a27b0 con 0x7f7fd0109770 2024-04-25T13:42:52.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 Processor -- start 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fcd7fa640 1 -- 172.21.15.94:0/4044196369 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f7fc802fd20 con 0x7f7fd0109770 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 -- start start 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598015810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598015d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598016a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555597f79d80 con 0x555597c73390 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555597f33a20 con 0x555597c4a4d0 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.607+0000 7f7fdd798740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555597f75d40 con 0x555597c526b0 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598015810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598015810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58500/0 (socket says 172.21.15.94:58500) 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/2720360004 learned_addr learned my addr 172.21.15.94:0/2720360004 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598015d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 -- 172.21.15.94:0/2720360004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 msgr2=0x555598016a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598016a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 -- 172.21.15.94:0/2720360004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 msgr2=0x555598015810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598015810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcd7fa640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 0x7f7fb80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 -- 172.21.15.94:0/2720360004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555597f3ec30 con 0x555597c4a4d0 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fd4da9640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 0x7f7fb80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcd7fa640 1 -- 172.21.15.94:0/4044196369 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7fc8100370 con 0x7f7fd0109770 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.608+0000 7f7fcffff640 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598015d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009bf0 tx=0x7f7fc0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b102df1422fe7a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.609+0000 7f7fb67fc640 1 -- 172.21.15.94:0/2720360004 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0009da0 con 0x555597c4a4d0 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.609+0000 7f7fb67fc640 1 -- 172.21.15.94:0/2720360004 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc0005020 con 0x555597c4a4d0 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fb67fc640 1 -- 172.21.15.94:0/2720360004 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0004b60 con 0x555597c4a4d0 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- 172.21.15.94:0/2720360004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 msgr2=0x555598015d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fd4da9640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 0x7f7fb80b2090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80049d0 tx=0x7f7fc802eb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598015d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009bf0 tx=0x7f7fc0011810 comp rx=0 tx=0).stop 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- 172.21.15.94:0/2720360004 shutdown_connections 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598016a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598015d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 --2- 172.21.15.94:0/2720360004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598015810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- 172.21.15.94:0/2720360004 >> 172.21.15.94:0/2720360004 conn(0x555597c674e0 msgr2=0x555598011020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- 172.21.15.94:0/2720360004 shutdown_connections 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- 172.21.15.94:0/2720360004 wait complete. 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.610+0000 7f7fdd798740 1 -- start start 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 Processor -- start 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x55559801f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598026840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555980175d0 con 0x555597c4a4d0 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555597f00dc0 con 0x555597c73390 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555598017750 con 0x555597c526b0 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fdd798740 10 client.0 _ll_register_callbacks cb 0x555597f1c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58512/0 (socket says 172.21.15.94:58512) 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.611+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/1372019330 learned_addr learned my addr 172.21.15.94:0/1372019330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcffff640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x55559801f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598026840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 msgr2=0x55559801f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x55559801f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/1372019330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 msgr2=0x555598026840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598026840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/1372019330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555597c8b3f0 con 0x555597c4a4d0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcffff640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x55559801f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009ef0 tx=0x7f7fc4009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363162d76fdb93ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40120d0 con 0x555597c4a4d0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7fc4010430 con 0x555597c4a4d0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.612+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc0005020 con 0x555597c4a4d0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4028050 con 0x555597c4a4d0 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555597d557d0 con 0x555597c4a4d0 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc0005020 con 0x555597c4a4d0 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc4020420 con 0x555597c4a4d0 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 10 client.15768 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 20 client.15768 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c31f60 0x55559802d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 4 client.15768 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.613+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555597f87990 con 0x555597c31f60 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.614+0000 7f7fcffff640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c31f60 0x55559802d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.614+0000 7f7faf7fe640 10 client.15768 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.614+0000 7f7fcffff640 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c31f60 0x55559802d070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd8ad637d5c363cd server_cookie=7c21ab8b1c85c066 in_seq=0 out_seq=0 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.615+0000 7f7faf7fe640 1 -- 172.21.15.94:0/1372019330 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555597f87990 con 0x555597c31f60 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.615+0000 7f7faf7fe640 10 client.15768 handle_command_reply: tid=0 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.615+0000 7f7fdd798740 1 client.15768 shutdown 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 msgr2=0x55559801f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009ef0 tx=0x7f7fc4009ad0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 shutdown_connections 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c31f60 0x55559802d070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598026840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x55559801f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7faf7fe640 0 client.15768 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 --2- 172.21.15.94:0/1372019330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801f3a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 >> 172.21.15.94:0/1372019330 conn(0x555597c674e0 msgr2=0x555598013f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 shutdown_connections 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 1 -- 172.21.15.94:0/1372019330 wait complete. 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.616+0000 7f7fdd798740 20 client.15768 trim_cache size 0 max 16384 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 Processor -- start 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 -- start start 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x5555980267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598014530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01ab4c0 con 0x555597c4a4d0 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01ad000 con 0x555597c73390 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fdd798740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01abb90 con 0x555597c526b0 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58520/0 (socket says 172.21.15.94:58520) 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.643+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/117383341 learned_addr learned my addr 172.21.15.94:0/117383341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598014530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcffff640 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x5555980267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/117383341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 msgr2=0x5555980267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x5555980267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/117383341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 msgr2=0x555598014530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598014530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/117383341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555598027430 con 0x555597c4a4d0 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40096a0 tx=0x7f7fc4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb24aaf85c426a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fb5ffb640 1 -- 172.21.15.94:0/117383341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4020e40 con 0x555597c4a4d0 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fb5ffb640 1 -- 172.21.15.94:0/117383341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc401c040 con 0x555597c4a4d0 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.644+0000 7f7fb5ffb640 1 -- 172.21.15.94:0/117383341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4020390 con 0x555597c4a4d0 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 -- 172.21.15.94:0/117383341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 msgr2=0x55559801a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40096a0 tx=0x7f7fc4005970 comp rx=0 tx=0).stop 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 -- 172.21.15.94:0/117383341 shutdown_connections 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c73390 0x555598014530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x5555980267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 --2- 172.21.15.94:0/117383341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c4a4d0 0x55559801a670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 -- 172.21.15.94:0/117383341 >> 172.21.15.94:0/117383341 conn(0x555597c674e0 msgr2=0x555598013e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.645+0000 7f7fdd798740 1 -- 172.21.15.94:0/117383341 shutdown_connections 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.646+0000 7f7fdd798740 1 -- 172.21.15.94:0/117383341 wait complete. 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.646+0000 7f7fdd798740 1 -- start start 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.646+0000 7f7fdd798740 1 Processor -- start 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598025210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598025750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01ac930 con 0x555597c73390 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01ac260 con 0x555597c4a4d0 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01a94a0 con 0x555597c526b0 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58536/0 (socket says 172.21.15.94:58536) 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/4241512661 learned_addr learned my addr 172.21.15.94:0/4241512661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fdd798740 10 client.0 _ll_register_callbacks cb 0x7f7fc4009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598025210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcffff640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598025750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 msgr2=0x555598025750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598025750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/4241512661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 msgr2=0x555598025210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598025210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.647+0000 7f7fcf7fe640 1 -- 172.21.15.94:0/4241512661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555980935e0 con 0x555597c73390 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7fcf7fe640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009b70 tx=0x7f7fc4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41c98fe1291ef8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40205a0 con 0x555597c73390 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7fc40050b0 con 0x555597c73390 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc401c040 con 0x555597c73390 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7fc40052d0 con 0x555597c73390 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7fdd798740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555598094840 con 0x555597c73390 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.648+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc401c040 con 0x555597c73390 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc4010430 con 0x555597c73390 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 10 client.15774 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 20 client.15774 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c073a0 0x555598097080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 4 client.15774 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555597f81800 con 0x555597c073a0 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fcffff640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c073a0 0x555598097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7faf7fe640 10 client.15774 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.649+0000 7f7fcffff640 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c073a0 0x555598097080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66bc05f8e969d69a server_cookie=581b33c9635e75b8 in_seq=0 out_seq=0 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7faf7fe640 1 -- 172.21.15.94:0/4241512661 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555597f81800 con 0x555597c073a0 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7faf7fe640 10 client.15774 handle_command_reply: tid=0 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 client.15774 shutdown 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 msgr2=0x555598025c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009b70 tx=0x7f7fc4005f90 comp rx=0 tx=0).stop 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 shutdown_connections 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x555597c073a0 0x555598097080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555597c73390 0x555598025c90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7faf7fe640 0 client.15774 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555597c526b0 0x555598025750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 --2- 172.21.15.94:0/4241512661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555597c4a4d0 0x555598025210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 >> 172.21.15.94:0/4241512661 conn(0x555597c674e0 msgr2=0x55559801f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 shutdown_connections 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 1 -- 172.21.15.94:0/4241512661 wait complete. 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.650+0000 7f7fdd798740 20 client.15774 trim_cache size 0 max 16384 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 msgr2=0x7f7fb80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 0x7f7fb80b2090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80049d0 tx=0x7f7fc802eb70 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 msgr2=0x7f7fd01a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009430 tx=0x7f7fc80316f0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 shutdown_connections 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.651+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7fb80afb80 0x7f7fb80b2090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fd0109770 0x7f7fd01a19b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd0106c00 0x7f7fd019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 --2- 172.21.15.94:0/4044196369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd01041f0 0x7f7fd019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 >> 172.21.15.94:0/4044196369 conn(0x7f7fd00fdbd0 msgr2=0x7f7fd0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 shutdown_connections 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.652+0000 7f7fd680c640 1 -- 172.21.15.94:0/4044196369 wait complete. 2024-04-25T13:42:52.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 -- 172.21.15.94:0/1011606865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c84108920 msgr2=0x7f2c8410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1011606865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c84108920 0x7f2c8410ad40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009ca0 tx=0x7f2c7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 -- 172.21.15.94:0/1011606865 shutdown_connections 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1011606865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c84108920 0x7f2c8410ad40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1011606865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84103c20 0x7f2c84104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1011606865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c841029e0 0x7f2c841036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 -- 172.21.15.94:0/1011606865 >> 172.21.15.94:0/1011606865 conn(0x7f2c840fdbd0 msgr2=0x7f2c84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 -- 172.21.15.94:0/1011606865 shutdown_connections 2024-04-25T13:42:52.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.789+0000 7f2c89f64640 1 -- 172.21.15.94:0/1011606865 wait complete. 2024-04-25T13:42:52.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.790+0000 7f2c89f64640 1 Processor -- start 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.790+0000 7f2c89f64640 1 -- start start 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 0x7f2c8419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84108920 0x7f2c841a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f610 con 0x7f2c841029e0 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f490 con 0x7f2c84108920 2024-04-25T13:42:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c89f64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f790 con 0x7f2c84103c20 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58566/0 (socket says 172.21.15.94:58566) 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 -- 172.21.15.94:0/1303478044 learned_addr learned my addr 172.21.15.94:0/1303478044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 0x7f2c8419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c83fff640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84108920 0x7f2c841a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 -- 172.21.15.94:0/1303478044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 msgr2=0x7f2c8419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 0x7f2c8419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 -- 172.21.15.94:0/1303478044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84108920 msgr2=0x7f2c841a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84108920 0x7f2c841a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 -- 172.21.15.94:0/1303478044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841a2020 con 0x7f2c841029e0 2024-04-25T13:42:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 0x7f2c8419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.791+0000 7f2c837fe640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005d00 tx=0x7f2c7400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.792+0000 7f2c80ff9640 1 -- 172.21.15.94:0/1303478044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74017070 con 0x7f2c841029e0 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.792+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841a2310 con 0x7f2c841029e0 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.792+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a2860 con 0x7f2c841029e0 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.792+0000 7f2c80ff9640 1 -- 172.21.15.94:0/1303478044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c74009c30 con 0x7f2c841029e0 2024-04-25T13:42:52.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.792+0000 7f2c80ff9640 1 -- 172.21.15.94:0/1303478044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400f620 con 0x7f2c841029e0 2024-04-25T13:42:52.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.794+0000 7f2c80ff9640 1 -- 172.21.15.94:0/1303478044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f2c7401a070 con 0x7f2c841029e0 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.794+0000 7f2c90e14740 1 Processor -- start 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.794+0000 7f2c90e14740 1 -- start start 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d2021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d202e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562d0d161d80 con 0x562d0ce5b390 2024-04-25T13:42:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d0d11ba20 con 0x562d0ce3a6b0 2024-04-25T13:42:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c90e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d0d15dd40 con 0x562d0ce324d0 2024-04-25T13:42:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58582/0 (socket says 172.21.15.94:58582) 2024-04-25T13:42:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 -- 172.21.15.94:0/867810923 learned_addr learned my addr 172.21.15.94:0/867810923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d202e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c80ff9640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 0x7f2c6c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c837fe640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d2021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 -- 172.21.15.94:0/867810923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 msgr2=0x562d0d2021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d2021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 -- 172.21.15.94:0/867810923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 msgr2=0x562d0d202e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d202e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 -- 172.21.15.94:0/867810923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d0d126c30 con 0x562d0ce5b390 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c80ff9640 1 -- 172.21.15.94:0/1303478044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2c740de770 con 0x7f2c841029e0 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.795+0000 7f2c83fff640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 0x7f2c6c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c83fff640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c035000 tx=0x7f2c7c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b12480550e91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d202e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c69ffb640 1 -- 172.21.15.94:0/867810923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c042040 con 0x562d0ce5b390 2024-04-25T13:42:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c83fff640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 0x7f2c6c0b2110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c04d000 tx=0x7f2c7c04e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c69ffb640 1 -- 172.21.15.94:0/867810923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c036070 con 0x562d0ce5b390 2024-04-25T13:42:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.796+0000 7f2c69ffb640 1 -- 172.21.15.94:0/867810923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c02fcc0 con 0x562d0ce5b390 2024-04-25T13:42:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 -- 172.21.15.94:0/867810923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 msgr2=0x562d0d201c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c035000 tx=0x7f2c7c009960 comp rx=0 tx=0).stop 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 -- 172.21.15.94:0/867810923 shutdown_connections 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d202e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d2021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 --2- 172.21.15.94:0/867810923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d201c20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 -- 172.21.15.94:0/867810923 >> 172.21.15.94:0/867810923 conn(0x562d0ce4f4e0 msgr2=0x562d0d1fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 -- 172.21.15.94:0/867810923 shutdown_connections 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.797+0000 7f2c90e14740 1 -- 172.21.15.94:0/867810923 wait complete. 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 -- start start 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 Processor -- start 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d20ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d20bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562d0d203940 con 0x562d0ce5b390 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d0d0e8dc0 con 0x562d0ce3a6b0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d0d203ac0 con 0x562d0ce324d0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c83fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58590/0 (socket says 172.21.15.94:58590) 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c83fff640 1 -- 172.21.15.94:0/3889024607 learned_addr learned my addr 172.21.15.94:0/3889024607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c90e14740 10 client.0 _ll_register_callbacks cb 0x562d0d104600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d20ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.798+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d20bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 -- 172.21.15.94:0/3889024607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 msgr2=0x562d0d20ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d20ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 -- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 msgr2=0x562d0d20bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d20bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 -- 172.21.15.94:0/3889024607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d0ce733f0 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c83fff640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c004af0 tx=0x7f2c7c033030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aefb30071e6f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004400 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2c7c0045a0 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c036070 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c02f660 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c90e14740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d0cf3d7d0 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.799+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c036070 con 0x562d0ce5b390 2024-04-25T13:42:52.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c7c0604d0 con 0x562d0ce5b390 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 10 client.15786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 20 client.15786 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0ce19f60 0x562d0d219260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 4 client.15786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d0d16f990 con 0x562d0ce19f60 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0ce19f60 0x562d0d219260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0ce19f60 0x562d0d219260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ad9be17af39a552 server_cookie=82c6b8defe0ddf26 in_seq=0 out_seq=0 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.800+0000 7f2c62ffd640 10 client.15786 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3889024607 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d0d16f990 con 0x562d0ce19f60 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c62ffd640 10 client.15786 handle_command_reply: tid=0 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 client.15786 shutdown 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 msgr2=0x562d0d212e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c004af0 tx=0x7f2c7c033030 comp rx=0 tx=0).stop 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 shutdown_connections 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0ce19f60 0x562d0d219260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce5b390 0x562d0d212e40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d20bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c62ffd640 0 client.15786 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3889024607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d20ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.801+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 >> 172.21.15.94:0/3889024607 conn(0x562d0ce4f4e0 msgr2=0x562d0d1f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.802+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 shutdown_connections 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.802+0000 7f2c90e14740 1 -- 172.21.15.94:0/3889024607 wait complete. 2024-04-25T13:42:52.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.802+0000 7f2c90e14740 20 client.15786 trim_cache size 0 max 16384 2024-04-25T13:42:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 Processor -- start 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 -- start start 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d206740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce5b390 0x562d0d212d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000e690 con 0x562d0ce3a6b0 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c700101d0 con 0x562d0ce5b390 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c82ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c90e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000ed60 con 0x562d0ce324d0 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c82ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58592/0 (socket says 172.21.15.94:58592) 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.815+0000 7f2c82ffd640 1 -- 172.21.15.94:0/1075965308 learned_addr learned my addr 172.21.15.94:0/1075965308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c837fe640 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d206740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 -- 172.21.15.94:0/1075965308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 msgr2=0x562d0d206740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d206740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 -- 172.21.15.94:0/1075965308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce5b390 msgr2=0x562d0d212d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c83fff640 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce5b390 0x562d0d212d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce5b390 0x562d0d212d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 -- 172.21.15.94:0/1075965308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d0d2132a0 con 0x562d0ce3a6b0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c78005fd0 tx=0x7f2c780129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab61e426a24378e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c697fa640 1 -- 172.21.15.94:0/1075965308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78005020 con 0x562d0ce3a6b0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c697fa640 1 -- 172.21.15.94:0/1075965308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c780047a0 con 0x562d0ce3a6b0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.816+0000 7f2c697fa640 1 -- 172.21.15.94:0/1075965308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780108d0 con 0x562d0ce3a6b0 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 -- 172.21.15.94:0/1075965308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 msgr2=0x562d0d206c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c78005fd0 tx=0x7f2c780129e0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 -- 172.21.15.94:0/1075965308 shutdown_connections 2024-04-25T13:42:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce5b390 0x562d0d212d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce3a6b0 0x562d0d206c80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 --2- 172.21.15.94:0/1075965308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce324d0 0x562d0d206740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 -- 172.21.15.94:0/1075965308 >> 172.21.15.94:0/1075965308 conn(0x562d0ce4f4e0 msgr2=0x562d0d20a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 -- 172.21.15.94:0/1075965308 shutdown_connections 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.817+0000 7f2c90e14740 1 -- 172.21.15.94:0/1075965308 wait complete. 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 -- start start 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 Processor -- start 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d210570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce5b390 0x562d0d2114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000fb00 con 0x562d0ce324d0 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000f430 con 0x562d0ce3a6b0 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000c670 con 0x562d0ce5b390 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c90e14740 10 client.0 _ll_register_callbacks cb 0x7f2c7c009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58600/0 (socket says 172.21.15.94:58600) 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c837fe640 1 -- 172.21.15.94:0/3625781829 learned_addr learned my addr 172.21.15.94:0/3625781829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d210570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.818+0000 7f2c83fff640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce5b390 0x562d0d2114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 -- 172.21.15.94:0/3625781829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce5b390 msgr2=0x562d0d2114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce5b390 0x562d0d2114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 -- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 msgr2=0x562d0d210570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d210570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 -- 172.21.15.94:0/3625781829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d0d211a60 con 0x562d0ce324d0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c837fe640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c740dc0b0 tx=0x7f2c7400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8cdcf029c8b913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74021040 con 0x562d0ce324d0 2024-04-25T13:42:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c780047a0 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2c740af420 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c740af680 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c90e14740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d0d212db0 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.819+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c780047a0 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c740c9220 con 0x562d0ce324d0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 10 client.15792 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 20 client.15792 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0cdef3a0 0x562d0d283a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 4 client.15792 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d0d169800 con 0x562d0cdef3a0 2024-04-25T13:42:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0cdef3a0 0x562d0d283a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c62ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.820+0000 7f2c82ffd640 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0cdef3a0 0x562d0d283a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9731c5aa2c8bc96b server_cookie=db65195727348b54 in_seq=0 out_seq=0 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c62ffd640 1 -- 172.21.15.94:0/3625781829 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d0d169800 con 0x562d0cdef3a0 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c62ffd640 10 client.15792 handle_command_reply: tid=0 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 client.15792 shutdown 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 msgr2=0x562d0d210030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2c740dc0b0 tx=0x7f2c7400bed0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 shutdown_connections 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562d0cdef3a0 0x562d0d283a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c62ffd640 0 client.15792 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d0ce5b390 0x562d0d2114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d0ce3a6b0 0x562d0d210570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 --2- 172.21.15.94:0/3625781829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562d0ce324d0 0x562d0d210030 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 >> 172.21.15.94:0/3625781829 conn(0x562d0ce4f4e0 msgr2=0x562d0d20a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 shutdown_connections 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 1 -- 172.21.15.94:0/3625781829 wait complete. 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.821+0000 7f2c90e14740 20 client.15792 trim_cache size 0 max 16384 2024-04-25T13:42:52.828 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:52.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 msgr2=0x7f2c6c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 0x7f2c6c0b2110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c04d000 tx=0x7f2c7c04e000 comp rx=0 tx=0).stop 2024-04-25T13:42:52.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 msgr2=0x7f2c8419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005d00 tx=0x7f2c7400dde0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 shutdown_connections 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c6c0afc00 0x7f2c6c0b2110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c84108920 0x7f2c841a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c84103c20 0x7f2c8419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 --2- 172.21.15.94:0/1303478044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c841029e0 0x7f2c8419a4e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 >> 172.21.15.94:0/1303478044 conn(0x7f2c840fdbd0 msgr2=0x7f2c84104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 shutdown_connections 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.822+0000 7f2c89f64640 1 -- 172.21.15.94:0/1303478044 wait complete. 2024-04-25T13:42:52.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:42:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 -- 172.21.15.94:0/3477597682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 msgr2=0x7fafbc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/3477597682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc1036e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009900 tx=0x7fafb002a410 comp rx=0 tx=0).stop 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 -- 172.21.15.94:0/3477597682 shutdown_connections 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/3477597682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/3477597682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 0x7fafbc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/3477597682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc1036e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 -- 172.21.15.94:0/3477597682 >> 172.21.15.94:0/3477597682 conn(0x7fafbc0fdbd0 msgr2=0x7fafbc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 -- 172.21.15.94:0/3477597682 shutdown_connections 2024-04-25T13:42:52.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.949+0000 7fafc2d1f640 1 -- 172.21.15.94:0/3477597682 wait complete. 2024-04-25T13:42:52.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 Processor -- start 2024-04-25T13:42:52.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 -- start start 2024-04-25T13:42:52.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 0x7fafbc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10f610 con 0x7fafbc1029e0 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10f490 con 0x7fafbc108920 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc2d1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10f790 con 0x7fafbc103c20 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc0abb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc0abb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58626/0 (socket says 172.21.15.94:58626) 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc0abb640 1 -- 172.21.15.94:0/845756792 learned_addr learned my addr 172.21.15.94:0/845756792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafbbfff640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 0x7fafbc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.950+0000 7fafc12bc640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 -- 172.21.15.94:0/845756792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 msgr2=0x7fafbc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 0x7fafbc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 -- 172.21.15.94:0/845756792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 msgr2=0x7fafbc1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 -- 172.21.15.94:0/845756792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1a20b0 con 0x7fafbc1029e0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc12bc640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc0abb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fafb0006010 tx=0x7fafb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafb9ffb640 1 -- 172.21.15.94:0/845756792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb003f030 con 0x7fafbc1029e0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1a2340 con 0x7fafbc1029e0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafb9ffb640 1 -- 172.21.15.94:0/845756792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb0031910 con 0x7fafbc1029e0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafb9ffb640 1 -- 172.21.15.94:0/845756792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0035070 con 0x7fafbc1029e0 2024-04-25T13:42:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.951+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1a2880 con 0x7fafbc1029e0 2024-04-25T13:42:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.954+0000 7fafc9bce740 1 Processor -- start 2024-04-25T13:42:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.954+0000 7fafc9bce740 1 -- start start 2024-04-25T13:42:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9ea8d80 con 0x55e9d9b816b0 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9e62a20 con 0x55e9d9b794d0 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9ea4d40 con 0x55e9d9ba2390 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafb9ffb640 1 -- 172.21.15.94:0/845756792 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fafb0042030 con 0x7fafbc1029e0 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56158/0 (socket says 172.21.15.94:56158) 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 -- 172.21.15.94:0/1546636265 learned_addr learned my addr 172.21.15.94:0/1546636265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 -- 172.21.15.94:0/1546636265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 msgr2=0x55e9d9f44d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 -- 172.21.15.94:0/1546636265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 msgr2=0x55e9d9f45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafc12bc640 1 -- 172.21.15.94:0/1546636265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d9e6dc30 con 0x55e9d9ba2390 2024-04-25T13:42:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.955+0000 7fafbbfff640 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafc12bc640 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009f90 tx=0x7fafb400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e6ee0a3e9244dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafb9ffb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 0x7fafac0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafa6ffd640 1 -- 172.21.15.94:0/1546636265 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003b70 con 0x55e9d9ba2390 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafc0abb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 0x7fafac0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafa6ffd640 1 -- 172.21.15.94:0/1546636265 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb4014d50 con 0x55e9d9ba2390 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafb9ffb640 1 -- 172.21.15.94:0/845756792 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fafb00fef20 con 0x7fafbc1029e0 2024-04-25T13:42:52.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.956+0000 7fafa6ffd640 1 -- 172.21.15.94:0/1546636265 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004660 con 0x55e9d9ba2390 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- 172.21.15.94:0/1546636265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 msgr2=0x55e9d9f44810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc0abb640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 0x7fafac0b2090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fafb0031440 tx=0x7fafb0094a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009f90 tx=0x7fafb400f7f0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- 172.21.15.94:0/1546636265 shutdown_connections 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 --2- 172.21.15.94:0/1546636265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9f44810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- 172.21.15.94:0/1546636265 >> 172.21.15.94:0/1546636265 conn(0x55e9d9b964e0 msgr2=0x55e9d9f40020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- 172.21.15.94:0/1546636265 shutdown_connections 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- 172.21.15.94:0/1546636265 wait complete. 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.957+0000 7fafc9bce740 1 -- start start 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 Processor -- start 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b816b0 0x55e9d9f4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9ba2390 0x55e9d9f55ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9f465d0 con 0x55e9d9b794d0 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9e2fdc0 con 0x55e9d9ba2390 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9f46750 con 0x55e9d9b816b0 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc9bce740 10 client.0 _ll_register_callbacks cb 0x55e9d9e4b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58646/0 (socket says 172.21.15.94:58646) 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafc12bc640 1 -- 172.21.15.94:0/979585431 learned_addr learned my addr 172.21.15.94:0/979585431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.958+0000 7fafbbfff640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b816b0 0x55e9d9f4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9ba2390 0x55e9d9f55ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 -- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b816b0 msgr2=0x55e9d9f4eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b816b0 0x55e9d9f4eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 -- 172.21.15.94:0/979585431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9ba2390 msgr2=0x55e9d9f55ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9ba2390 0x55e9d9f55ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 -- 172.21.15.94:0/979585431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d9bba3f0 con 0x55e9d9b794d0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc12bc640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fafb4014fd0 tx=0x7fafb4014f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c5271137ff0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7faf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40043e0 con 0x55e9d9b794d0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb4014d50 con 0x55e9d9b794d0 2024-04-25T13:42:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fafb4004580 con 0x55e9d9b794d0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafb4004780 con 0x55e9d9b794d0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc9bce740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9d9c847d0 con 0x55e9d9b794d0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.959+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb4014d50 con 0x55e9d9b794d0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fafb4005620 con 0x55e9d9b794d0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 10 client.15804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 20 client.15804 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b60f60 0x55e9d9f5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 4 client.15804 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9d9eb6990 con 0x55e9d9b60f60 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7fafbbfff640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b60f60 0x55e9d9f5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.960+0000 7faf9bfff640 10 client.15804 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:52.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.961+0000 7fafbbfff640 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b60f60 0x55e9d9f5bf90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e3778a29ec5af5c server_cookie=b03d99c9778352b7 in_seq=0 out_seq=0 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.961+0000 7faf9bfff640 1 -- 172.21.15.94:0/979585431 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9d9eb6990 con 0x55e9d9b60f60 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.961+0000 7faf9bfff640 10 client.15804 handle_command_reply: tid=0 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.961+0000 7fafc9bce740 1 client.15804 shutdown 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 msgr2=0x55e9d9f4e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fafb4014fd0 tx=0x7fafb4014f00 comp rx=0 tx=0).stop 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 shutdown_connections 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b60f60 0x55e9d9f5bf90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9ba2390 0x55e9d9f55ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b816b0 0x55e9d9f4eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 --2- 172.21.15.94:0/979585431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b794d0 0x55e9d9f4e640 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 >> 172.21.15.94:0/979585431 conn(0x55e9d9b964e0 msgr2=0x55e9d9f3f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7faf9bfff640 0 client.15804 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 shutdown_connections 2024-04-25T13:42:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 1 -- 172.21.15.94:0/979585431 wait complete. 2024-04-25T13:42:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.962+0000 7fafc9bce740 20 client.15804 trim_cache size 0 max 16384 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 Processor -- start 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 -- start start 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b794d0 0x55e9d9f5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 0x55e9d9f49290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a9350 con 0x55e9d9ba2390 2024-04-25T13:42:52.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aae90 con 0x55e9d9b816b0 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a9a20 con 0x55e9d9b794d0 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc12bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58650/0 (socket says 172.21.15.94:58650) 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc12bc640 1 -- 172.21.15.94:0/2975223286 learned_addr learned my addr 172.21.15.94:0/2975223286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b794d0 0x55e9d9f5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafbbfff640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 0x55e9d9f49290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.976+0000 7fafc12bc640 1 -- 172.21.15.94:0/2975223286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b794d0 msgr2=0x55e9d9f5bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b794d0 0x55e9d9f5bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafc12bc640 1 -- 172.21.15.94:0/2975223286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 msgr2=0x55e9d9f49290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 0x55e9d9f49290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafc12bc640 1 -- 172.21.15.94:0/2975223286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d9f40bb0 con 0x55e9d9ba2390 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafbbfff640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 0x55e9d9f49290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:52.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009fd0 tx=0x7fafb4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c43cbf71cfa63c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafa67fc640 1 -- 172.21.15.94:0/2975223286 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004920 con 0x55e9d9ba2390 2024-04-25T13:42:52.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafa67fc640 1 -- 172.21.15.94:0/2975223286 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb403d030 con 0x55e9d9ba2390 2024-04-25T13:42:52.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.977+0000 7fafa67fc640 1 -- 172.21.15.94:0/2975223286 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003970 con 0x55e9d9ba2390 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- 172.21.15.94:0/2975223286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 msgr2=0x55e9d9f497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009fd0 tx=0x7fafb4016040 comp rx=0 tx=0).stop 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- 172.21.15.94:0/2975223286 shutdown_connections 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9ba2390 0x55e9d9f497d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b816b0 0x55e9d9f49290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2975223286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9b794d0 0x55e9d9f5bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- 172.21.15.94:0/2975223286 >> 172.21.15.94:0/2975223286 conn(0x55e9d9b964e0 msgr2=0x55e9d9f4cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- 172.21.15.94:0/2975223286 shutdown_connections 2024-04-25T13:42:52.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- 172.21.15.94:0/2975223286 wait complete. 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 -- start start 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.978+0000 7fafc9bce740 1 Processor -- start 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f51d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9fc2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aa7c0 con 0x55e9d9b816b0 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aa0f0 con 0x55e9d9b794d0 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a7330 con 0x55e9d9ba2390 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc9bce740 10 client.0 _ll_register_callbacks cb 0x7fafa8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58654/0 (socket says 172.21.15.94:58654) 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 -- 172.21.15.94:0/2516477458 learned_addr learned my addr 172.21.15.94:0/2516477458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:52.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9fc2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f51d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 -- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 msgr2=0x55e9d9fc2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9fc2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 -- 172.21.15.94:0/2516477458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 msgr2=0x55e9d9f51d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f51d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafbbfff640 1 -- 172.21.15.94:0/2516477458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d9f55290 con 0x55e9d9b816b0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.979+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f51d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7fafbbfff640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fafa8004700 tx=0x7fafa8009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d65c5d753ad538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7faf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8023020 con 0x55e9d9b816b0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fafa801f630 con 0x55e9d9b816b0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb403d030 con 0x55e9d9b816b0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafa802e050 con 0x55e9d9b816b0 2024-04-25T13:42:52.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7fafc9bce740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9d9fcbe40 con 0x55e9d9b816b0 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.980+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb403d030 con 0x55e9d9b816b0 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fafa80273a0 con 0x55e9d9b816b0 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 10 client.15810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 20 client.15810 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b363a0 0x55e9d9fce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 4 client.15810 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9d9eb0800 con 0x55e9d9b363a0 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b363a0 0x55e9d9fce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7faf9bfff640 10 client.15810 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.981+0000 7fafc12bc640 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b363a0 0x55e9d9fce8b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=101f08a10a6333ba server_cookie=8e9f0a979c66f421 in_seq=0 out_seq=0 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7faf9bfff640 1 -- 172.21.15.94:0/2516477458 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9d9eb0800 con 0x55e9d9b363a0 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7faf9bfff640 10 client.15810 handle_command_reply: tid=0 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 client.15810 shutdown 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 msgr2=0x55e9d9f522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fafa8004700 tx=0x7fafa8009de0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 shutdown_connections 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e9d9b363a0 0x55e9d9fce8b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9d9ba2390 0x55e9d9fc2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7faf9bfff640 0 client.15810 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e9d9b816b0 0x55e9d9f522d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 --2- 172.21.15.94:0/2516477458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e9d9b794d0 0x55e9d9f51d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 >> 172.21.15.94:0/2516477458 conn(0x55e9d9b964e0 msgr2=0x55e9d9f4e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 shutdown_connections 2024-04-25T13:42:52.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 1 -- 172.21.15.94:0/2516477458 wait complete. 2024-04-25T13:42:52.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.982+0000 7fafc9bce740 20 client.15810 trim_cache size 0 max 16384 2024-04-25T13:42:52.988 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 msgr2=0x7fafac0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 0x7fafac0b2090 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fafb0031440 tx=0x7fafb0094a90 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 msgr2=0x7fafbc19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fafb0006010 tx=0x7fafb0004270 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 shutdown_connections 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafac0afb80 0x7fafac0b2090 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafbc108920 0x7fafbc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafbc103c20 0x7fafbc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 --2- 172.21.15.94:0/845756792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc1029e0 0x7fafbc19a570 secure :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fafb0006010 tx=0x7fafb0004270 comp rx=0 tx=0).stop 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 >> 172.21.15.94:0/845756792 conn(0x7fafbc0fdbd0 msgr2=0x7fafbc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 shutdown_connections 2024-04-25T13:42:52.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:52.983+0000 7fafc2d1f640 1 -- 172.21.15.94:0/845756792 wait complete. 2024-04-25T13:42:52.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.104+0000 7f761ecee640 1 -- 172.21.15.94:0/729482714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618109a40 msgr2=0x7f761810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.104+0000 7f761ecee640 1 --2- 172.21.15.94:0/729482714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618109a40 0x7f761810be60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7610009ca0 tx=0x7f761002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 -- 172.21.15.94:0/729482714 shutdown_connections 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 --2- 172.21.15.94:0/729482714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618109a40 0x7f761810be60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 --2- 172.21.15.94:0/729482714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 0x7f7618105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 --2- 172.21.15.94:0/729482714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618103b00 0x7f7618104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 -- 172.21.15.94:0/729482714 >> 172.21.15.94:0/729482714 conn(0x7f76180fecf0 msgr2=0x7f76181011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 -- 172.21.15.94:0/729482714 shutdown_connections 2024-04-25T13:42:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 -- 172.21.15.94:0/729482714 wait complete. 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 Processor -- start 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.105+0000 7f761ecee640 1 -- start start 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 0x7f761819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 0x7f76181a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f761810f680 con 0x7f7618103b00 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f761810f500 con 0x7f7618109a40 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ecee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761810f800 con 0x7f7618104d40 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58678/0 (socket says 172.21.15.94:58678) 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 -- 172.21.15.94:0/3417986644 learned_addr learned my addr 172.21.15.94:0/3417986644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f7617fff640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 0x7f761819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 -- 172.21.15.94:0/3417986644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 msgr2=0x7f761819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761d28b640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 0x7f76181a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 0x7f761819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 -- 172.21.15.94:0/3417986644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 msgr2=0x7f76181a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 0x7f76181a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 -- 172.21.15.94:0/3417986644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76181a1d30 con 0x7f7618103b00 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761d28b640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 0x7f76181a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.106+0000 7f761ca8a640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f760c005fd0 tx=0x7f760c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.107+0000 7f7615ffb640 1 -- 172.21.15.94:0/3417986644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c017070 con 0x7f7618103b00 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.107+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76181a1fc0 con 0x7f7618103b00 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.107+0000 7f7615ffb640 1 -- 172.21.15.94:0/3417986644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c009c30 con 0x7f7618103b00 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.107+0000 7f7615ffb640 1 -- 172.21.15.94:0/3417986644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c003d10 con 0x7f7618103b00 2024-04-25T13:42:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.108+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76181a24d0 con 0x7f7618103b00 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 Processor -- start 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 -- start start 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dde810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7615ffb640 1 -- 172.21.15.94:0/3417986644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f760c003eb0 con 0x7f7618103b00 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d59d42d80 con 0x563d59a1b6b0 2024-04-25T13:42:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d59cfca20 con 0x563d59a3c390 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.110+0000 7f7625ba0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d59d3ed40 con 0x563d59a134d0 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58690/0 (socket says 172.21.15.94:58690) 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 -- 172.21.15.94:0/718021917 learned_addr learned my addr 172.21.15.94:0/718021917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f761d28b640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dde810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f761ca8a640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 -- 172.21.15.94:0/718021917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 msgr2=0x563d59dded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 -- 172.21.15.94:0/718021917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 msgr2=0x563d59dde810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dde810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7617fff640 1 -- 172.21.15.94:0/718021917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d59d07c30 con 0x563d59a1b6b0 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f761d28b640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dde810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f761ca8a640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.111+0000 7f7615ffb640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 0x7f76080b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f7617fff640 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7604009b10 tx=0x7f760400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a56e21092d6274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f761d28b640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 0x7f76080b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f7615ffb640 1 -- 172.21.15.94:0/3417986644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f760c0de8b0 con 0x7f7618103b00 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f7602ffd640 1 -- 172.21.15.94:0/718021917 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400ecf0 con 0x563d59a1b6b0 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f7602ffd640 1 -- 172.21.15.94:0/718021917 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760400ee90 con 0x563d59a1b6b0 2024-04-25T13:42:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.112+0000 7f7602ffd640 1 -- 172.21.15.94:0/718021917 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604011470 con 0x563d59a1b6b0 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- 172.21.15.94:0/718021917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 msgr2=0x563d59ddfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7604009b10 tx=0x7f760400e2f0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f761d28b640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 0x7f76080b2090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7610006010 tx=0x7f7610031a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- 172.21.15.94:0/718021917 shutdown_connections 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59ddfa20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 --2- 172.21.15.94:0/718021917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dde810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- 172.21.15.94:0/718021917 >> 172.21.15.94:0/718021917 conn(0x563d59a304e0 msgr2=0x563d59dda020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- 172.21.15.94:0/718021917 shutdown_connections 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- 172.21.15.94:0/718021917 wait complete. 2024-04-25T13:42:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 -- start start 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.113+0000 7f7625ba0740 1 Processor -- start 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59de85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59defa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d59de05d0 con 0x563d59a1b6b0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d59cc9dc0 con 0x563d59a3c390 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d59de0750 con 0x563d59a134d0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7625ba0740 10 client.0 _ll_register_callbacks cb 0x563d59ce5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58706/0 (socket says 172.21.15.94:58706) 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 -- 172.21.15.94:0/368124772 learned_addr learned my addr 172.21.15.94:0/368124772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f761ca8a640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59de85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f761d28b640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59defa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 -- 172.21.15.94:0/368124772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 msgr2=0x563d59de85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59de85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 -- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 msgr2=0x563d59defa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59defa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f7617fff640 1 -- 172.21.15.94:0/368124772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d59a543f0 con 0x563d59a1b6b0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.114+0000 7f761ca8a640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59de85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7617fff640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f760400ed20 tx=0x7f760400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1132be7df18da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604003f20 con 0x563d59a1b6b0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760400ee90 con 0x563d59a1b6b0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f76040040c0 con 0x563d59a1b6b0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76040042c0 con 0x563d59a1b6b0 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7625ba0740 10 client.15822 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d59b1e7d0 con 0x563d59a1b6b0 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760400ee90 con 0x563d59a1b6b0 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7604018070 con 0x563d59a1b6b0 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7625ba0740 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.115+0000 7f7625ba0740 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f7625ba0740 20 client.15822 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599faf60 0x563d59df6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f7625ba0740 4 client.15822 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d59d50990 con 0x563d599faf60 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f761ca8a640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599faf60 0x563d59df6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f75f7fff640 10 client.15822 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.116+0000 7f761ca8a640 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599faf60 0x563d59df6030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeb6d1168ddabb00 server_cookie=5750f6662b7f0d4b in_seq=0 out_seq=0 2024-04-25T13:42:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f75f7fff640 1 -- 172.21.15.94:0/368124772 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d59d50990 con 0x563d599faf60 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f75f7fff640 10 client.15822 handle_command_reply: tid=0 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 client.15822 shutdown 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 msgr2=0x563d59de8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f760400ed20 tx=0x7f760400ed50 comp rx=0 tx=0).stop 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 shutdown_connections 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599faf60 0x563d59df6030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59defa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a1b6b0 0x563d59de8b20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 --2- 172.21.15.94:0/368124772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59de85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 >> 172.21.15.94:0/368124772 conn(0x563d59a304e0 msgr2=0x563d59dd9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f75f7fff640 0 client.15822 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 shutdown_connections 2024-04-25T13:42:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 1 -- 172.21.15.94:0/368124772 wait complete. 2024-04-25T13:42:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.117+0000 7f7625ba0740 20 client.15822 trim_cache size 0 max 16384 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 Processor -- start 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 -- start start 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dde790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 0x563d59dd9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76181ab590 con 0x563d59a3c390 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76181ad0d0 con 0x563d59a1b6b0 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.136+0000 7f7625ba0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76181abc60 con 0x563d59a134d0 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58720/0 (socket says 172.21.15.94:58720) 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 -- 172.21.15.94:0/2564919394 learned_addr learned my addr 172.21.15.94:0/2564919394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f7617fff640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 0x563d59dd9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761ca8a640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dde790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 -- 172.21.15.94:0/2564919394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 msgr2=0x563d59dde790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dde790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 -- 172.21.15.94:0/2564919394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 msgr2=0x563d59dd9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 0x563d59dd9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 -- 172.21.15.94:0/2564919394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d59ddee70 con 0x563d59a3c390 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f7617fff640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 0x563d59dd9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f761d28b640 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7610035000 tx=0x7f7610040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e37df01f34eedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.137+0000 7f76027fc640 1 -- 172.21.15.94:0/2564919394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610003d40 con 0x563d59a3c390 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f76027fc640 1 -- 172.21.15.94:0/2564919394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7610036050 con 0x563d59a3c390 2024-04-25T13:42:53.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f76027fc640 1 -- 172.21.15.94:0/2564919394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761002fdd0 con 0x563d59a3c390 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 -- 172.21.15.94:0/2564919394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 msgr2=0x563d59dd9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7610035000 tx=0x7f7610040000 comp rx=0 tx=0).stop 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 -- 172.21.15.94:0/2564919394 shutdown_connections 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a3c390 0x563d59dd9cb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a1b6b0 0x563d59dd9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.138+0000 7f7625ba0740 1 --2- 172.21.15.94:0/2564919394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a134d0 0x563d59dde790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.139+0000 7f7625ba0740 1 -- 172.21.15.94:0/2564919394 >> 172.21.15.94:0/2564919394 conn(0x563d59a304e0 msgr2=0x563d59de3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.139+0000 7f7625ba0740 1 -- 172.21.15.94:0/2564919394 shutdown_connections 2024-04-25T13:42:53.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.139+0000 7f7625ba0740 1 -- 172.21.15.94:0/2564919394 wait complete. 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.139+0000 7f7625ba0740 1 -- start start 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.139+0000 7f7625ba0740 1 Processor -- start 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a1b6b0 0x563d59debad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76181aca00 con 0x563d59a134d0 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76181ac330 con 0x563d59a3c390 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76181a9570 con 0x563d59a1b6b0 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7625ba0740 10 client.0 _ll_register_callbacks cb 0x7f7610009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58734/0 (socket says 172.21.15.94:58734) 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 -- 172.21.15.94:0/1709398774 learned_addr learned my addr 172.21.15.94:0/1709398774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f7617fff640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a1b6b0 0x563d59debad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761d28b640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 -- 172.21.15.94:0/1709398774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a1b6b0 msgr2=0x563d59debad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a1b6b0 0x563d59debad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 -- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 msgr2=0x563d59dec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 -- 172.21.15.94:0/1709398774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d59e5c3d0 con 0x563d59a134d0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761d28b640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f761ca8a640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f760c0058c0 tx=0x7f760c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f93a043abd7414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.140+0000 7f75f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0af9f0 con 0x563d59a134d0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f760c0a3180 con 0x563d59a134d0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7610036050 con 0x563d59a134d0 2024-04-25T13:42:53.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f760c0a2e20 con 0x563d59a134d0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d59e5d6b0 con 0x563d59a134d0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7610036050 con 0x563d59a134d0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f760c0a2500 con 0x563d59a134d0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 10 client.15828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 20 client.15828 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.141+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599d03a0 0x563d59e60040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.142+0000 7f7625ba0740 4 client.15828 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.142+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d59d4a800 con 0x563d599d03a0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.142+0000 7f7617fff640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599d03a0 0x563d59e60040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.142+0000 7f7617fff640 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599d03a0 0x563d59e60040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9769ecbaf8fcaef server_cookie=bfdc9906d815a304 in_seq=0 out_seq=0 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.142+0000 7f75f7fff640 10 client.15828 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:53.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.143+0000 7f75f7fff640 1 -- 172.21.15.94:0/1709398774 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d59d4a800 con 0x563d599d03a0 2024-04-25T13:42:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.143+0000 7f75f7fff640 10 client.15828 handle_command_reply: tid=0 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.143+0000 7f7625ba0740 1 client.15828 shutdown 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 msgr2=0x563d59deef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f760c0058c0 tx=0x7f760c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 shutdown_connections 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563d599d03a0 0x563d59e60040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d59a3c390 0x563d59dec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f75f7fff640 0 client.15828 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d59a1b6b0 0x563d59debad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 --2- 172.21.15.94:0/1709398774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d59a134d0 0x563d59deef50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 >> 172.21.15.94:0/1709398774 conn(0x563d59a304e0 msgr2=0x563d59ddce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 shutdown_connections 2024-04-25T13:42:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 1 -- 172.21.15.94:0/1709398774 wait complete. 2024-04-25T13:42:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.144+0000 7f7625ba0740 20 client.15828 trim_cache size 0 max 16384 2024-04-25T13:42:53.149 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 msgr2=0x7f76080b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 0x7f76080b2090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7610006010 tx=0x7f7610031a20 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 msgr2=0x7f761819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f760c005fd0 tx=0x7f760c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 shutdown_connections 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76080afb80 0x7f76080b2090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618109a40 0x7f76181a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7618104d40 0x7f761819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 --2- 172.21.15.94:0/3417986644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7618103b00 0x7f761819a1f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 >> 172.21.15.94:0/3417986644 conn(0x7f76180fecf0 msgr2=0x7f7618105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 shutdown_connections 2024-04-25T13:42:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.145+0000 7f761ecee640 1 -- 172.21.15.94:0/3417986644 wait complete. 2024-04-25T13:42:53.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 -- 172.21.15.94:0/722695295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 msgr2=0x7f554010bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 --2- 172.21.15.94:0/722695295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554010bca0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ca0 tx=0x7f553802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 -- 172.21.15.94:0/722695295 shutdown_connections 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 --2- 172.21.15.94:0/722695295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554010bca0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 --2- 172.21.15.94:0/722695295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540101740 0x7f5540101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 --2- 172.21.15.94:0/722695295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540105090 0x7f55401054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 -- 172.21.15.94:0/722695295 >> 172.21.15.94:0/722695295 conn(0x7f55400fd340 msgr2=0x7f55400ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 -- 172.21.15.94:0/722695295 shutdown_connections 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 -- 172.21.15.94:0/722695295 wait complete. 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.271+0000 7f5545ad9640 1 Processor -- start 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 -- start start 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540101740 0x7f554019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 0x7f55401a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f554010f620 con 0x7f5540102240 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f554010f4a0 con 0x7f5540105090 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f5545ad9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f554010f7a0 con 0x7f5540101740 2024-04-25T13:42:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58744/0 (socket says 172.21.15.94:58744) 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 -- 172.21.15.94:0/3016012790 learned_addr learned my addr 172.21.15.94:0/3016012790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553f7fe640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540101740 0x7f554019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553ffff640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 0x7f55401a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 -- 172.21.15.94:0/3016012790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540101740 msgr2=0x7f554019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540101740 0x7f554019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 -- 172.21.15.94:0/3016012790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 msgr2=0x7f55401a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 0x7f55401a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.272+0000 7f553effd640 1 -- 172.21.15.94:0/3016012790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540102840 con 0x7f5540102240 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f553ffff640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 0x7f55401a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f553effd640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5534005fd0 tx=0x7f5534007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f553cff9640 1 -- 172.21.15.94:0/3016012790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534007dc0 con 0x7f5540102240 2024-04-25T13:42:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5540102b30 con 0x7f5540102240 2024-04-25T13:42:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f553cff9640 1 -- 172.21.15.94:0/3016012790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534004030 con 0x7f5540102240 2024-04-25T13:42:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.273+0000 7f553cff9640 1 -- 172.21.15.94:0/3016012790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534003c50 con 0x7f5540102240 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.274+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55401030d0 con 0x7f5540102240 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 Processor -- start 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 -- start start 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 0x5599d2a56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26936b0 0x5599d2a57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5599d29bad80 con 0x5599d268b4d0 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599d2974a20 con 0x5599d26936b0 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.276+0000 7f554c981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5599d29b6d40 con 0x5599d26b4390 2024-04-25T13:42:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56274/0 (socket says 172.21.15.94:56274) 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 -- 172.21.15.94:0/851894583 learned_addr learned my addr 172.21.15.94:0/851894583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 -- 172.21.15.94:0/851894583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26936b0 msgr2=0x5599d2a57a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26936b0 0x5599d2a57a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 -- 172.21.15.94:0/851894583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 msgr2=0x5599d2a56d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553cff9640 1 -- 172.21.15.94:0/3016012790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f5534012a60 con 0x7f5540102240 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553f7fe640 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 0x5599d2a56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 0x5599d2a56d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553ffff640 1 -- 172.21.15.94:0/851894583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d297fc30 con 0x5599d26b4390 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.277+0000 7f553f7fe640 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 0x5599d2a56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.278+0000 7f553ffff640 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f553802f440 tx=0x7f553802f470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2680dcda92445174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.278+0000 7f5525ffb640 1 -- 172.21.15.94:0/851894583 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538031650 con 0x5599d26b4390 2024-04-25T13:42:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.278+0000 7f5525ffb640 1 -- 172.21.15.94:0/851894583 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5538036040 con 0x5599d26b4390 2024-04-25T13:42:53.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.278+0000 7f5525ffb640 1 -- 172.21.15.94:0/851894583 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380046d0 con 0x5599d26b4390 2024-04-25T13:42:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- 172.21.15.94:0/851894583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 msgr2=0x5599d2a56810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f553802f440 tx=0x7f553802f470 comp rx=0 tx=0).stop 2024-04-25T13:42:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- 172.21.15.94:0/851894583 shutdown_connections 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26936b0 0x5599d2a57a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d268b4d0 0x5599d2a56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 --2- 172.21.15.94:0/851894583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d26b4390 0x5599d2a56810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- 172.21.15.94:0/851894583 >> 172.21.15.94:0/851894583 conn(0x5599d26a84e0 msgr2=0x5599d2a52020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f553cff9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 0x7f55280b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- 172.21.15.94:0/851894583 shutdown_connections 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- 172.21.15.94:0/851894583 wait complete. 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f553ffff640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 0x7f55280b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.279+0000 7f554c981740 1 -- start start 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553cff9640 1 -- 172.21.15.94:0/3016012790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55340e6c30 con 0x7f5540102240 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 Processor -- start 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553ffff640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 0x7f55280b2090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f553802ade0 tx=0x7f553802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a60570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26b4390 0x5599d2a679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5599d2a584d0 con 0x5599d26936b0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599d2941dc0 con 0x5599d26b4390 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5599d2a58650 con 0x5599d268b4d0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f554c981740 10 client.0 _ll_register_callbacks cb 0x5599d295d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58758/0 (socket says 172.21.15.94:58758) 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553effd640 1 -- 172.21.15.94:0/3381602185 learned_addr learned my addr 172.21.15.94:0/3381602185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553f7fe640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a60570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.280+0000 7f553ffff640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26b4390 0x5599d2a679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 -- 172.21.15.94:0/3381602185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 msgr2=0x5599d2a60570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a60570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 -- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26b4390 msgr2=0x5599d2a679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26b4390 0x5599d2a679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 -- 172.21.15.94:0/3381602185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d26cc3f0 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f553effd640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55340ac180 tx=0x7f55340ae920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ff9480bce0ad78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f551effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340af610 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55340e4790 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5538036040 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55340aef10 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f554c981740 10 client.15837 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599d27967d0 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.281+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55340b32c0 con 0x5599d26936b0 2024-04-25T13:42:53.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5534027030 con 0x5599d26936b0 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 10 client.15837 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 20 client.15837 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d2672f60 0x5599d2a6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 4 client.15837 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5538036040 con 0x5599d2672f60 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.282+0000 7f553f7fe640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d2672f60 0x5599d2a6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.283+0000 7f553f7fe640 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d2672f60 0x5599d2a6e150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266c9fe7313a187c server_cookie=5e12977734e35623 in_seq=0 out_seq=0 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.283+0000 7f551effd640 10 client.15837 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.283+0000 7f551effd640 1 -- 172.21.15.94:0/3381602185 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5538036040 con 0x5599d2672f60 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.283+0000 7f551effd640 10 client.15837 handle_command_reply: tid=0 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.283+0000 7f554c981740 1 client.15837 shutdown 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 msgr2=0x5599d2a60ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55340ac180 tx=0x7f55340ae920 comp rx=0 tx=0).stop 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 shutdown_connections 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d2672f60 0x5599d2a6e150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f551effd640 0 client.15837 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d26b4390 0x5599d2a679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60ab0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 --2- 172.21.15.94:0/3381602185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a60570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 >> 172.21.15.94:0/3381602185 conn(0x5599d26a84e0 msgr2=0x5599d2a51040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 shutdown_connections 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 1 -- 172.21.15.94:0/3381602185 wait complete. 2024-04-25T13:42:53.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.284+0000 7f554c981740 20 client.15837 trim_cache size 0 max 16384 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 Processor -- start 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 -- start start 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d2672f60 0x5599d2a60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d268b4d0 0x5599d2a678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a9320 con 0x5599d26936b0 2024-04-25T13:42:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55401aae60 con 0x5599d268b4d0 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f554c981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a99f0 con 0x5599d2672f60 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58760/0 (socket says 172.21.15.94:58760) 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 -- 172.21.15.94:0/425789161 learned_addr learned my addr 172.21.15.94:0/425789161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553f7fe640 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d2672f60 0x5599d2a60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553effd640 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d268b4d0 0x5599d2a678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 -- 172.21.15.94:0/425789161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d2672f60 msgr2=0x5599d2a60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d2672f60 0x5599d2a60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.301+0000 7f553ffff640 1 -- 172.21.15.94:0/425789161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d268b4d0 msgr2=0x5599d2a678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f553ffff640 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d268b4d0 0x5599d2a678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f553ffff640 1 -- 172.21.15.94:0/425789161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d2a61610 con 0x5599d26936b0 2024-04-25T13:42:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f553ffff640 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5599d2a57860 tx=0x7f553804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4852b6a3bb5fc68b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f55257fa640 1 -- 172.21.15.94:0/425789161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538060070 con 0x5599d26936b0 2024-04-25T13:42:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f55257fa640 1 -- 172.21.15.94:0/425789161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5538036050 con 0x5599d26936b0 2024-04-25T13:42:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.302+0000 7f55257fa640 1 -- 172.21.15.94:0/425789161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538056070 con 0x5599d26936b0 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- 172.21.15.94:0/425789161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 msgr2=0x5599d2a60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5599d2a57860 tx=0x7f553804d000 comp rx=0 tx=0).stop 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- 172.21.15.94:0/425789161 shutdown_connections 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a60680 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d268b4d0 0x5599d2a678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 --2- 172.21.15.94:0/425789161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d2672f60 0x5599d2a60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- 172.21.15.94:0/425789161 >> 172.21.15.94:0/425789161 conn(0x5599d26a84e0 msgr2=0x5599d2a5b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- 172.21.15.94:0/425789161 shutdown_connections 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- 172.21.15.94:0/425789161 wait complete. 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.303+0000 7f554c981740 1 -- start start 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 Processor -- start 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55401aa790 con 0x5599d26936b0 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55401aa0c0 con 0x5599d2672f60 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f554c981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a7300 con 0x5599d268b4d0 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f553f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58910/0 (socket says 172.21.15.94:58910) 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f553f7fe640 1 -- 172.21.15.94:0/1499680122 learned_addr learned my addr 172.21.15.94:0/1499680122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f553ffff640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.304+0000 7f553effd640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f554c981740 10 client.0 _ll_register_callbacks cb 0x7f55340aaf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 -- 172.21.15.94:0/1499680122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 msgr2=0x5599d2a654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 -- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 msgr2=0x5599d2a64f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 -- 172.21.15.94:0/1499680122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d2a669a0 con 0x5599d26936b0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553f7fe640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5538036050 con 0x5599d26936b0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f553ffff640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a663e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f553805e040 tx=0x7f55380490c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1de97b7b1fe7b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f551effd640 10 client.15843 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f551effd640 1 -- 172.21.15.94:0/1499680122 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538068020 con 0x5599d26936b0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f551effd640 1 -- 172.21.15.94:0/1499680122 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f553802fc10 con 0x5599d26936b0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f551effd640 1 -- 172.21.15.94:0/1499680122 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5538056070 con 0x5599d26936b0 2024-04-25T13:42:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f554c981740 10 client.15843 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.305+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5599d2a67b20 con 0x5599d26936b0 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f551effd640 1 -- 172.21.15.94:0/1499680122 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5538063030 con 0x5599d26936b0 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 10 client.15843 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 20 client.15843 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d26b4390 0x5599d2ad84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 4 client.15843 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599d29c8990 con 0x5599d26b4390 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.306+0000 7f553f7fe640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d26b4390 0x5599d2ad84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f551effd640 10 client.15843 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f553f7fe640 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d26b4390 0x5599d2ad84d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddba7c6a85acbcf server_cookie=1a777735488932c4 in_seq=0 out_seq=0 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f551effd640 1 -- 172.21.15.94:0/1499680122 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599d29c8990 con 0x5599d26b4390 2024-04-25T13:42:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f551effd640 10 client.15843 handle_command_reply: tid=0 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 client.15843 shutdown 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 msgr2=0x5599d2a663e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a663e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f553805e040 tx=0x7f55380490c0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 shutdown_connections 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5599d26b4390 0x5599d2ad84d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599d26936b0 0x5599d2a663e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599d268b4d0 0x5599d2a654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 --2- 172.21.15.94:0/1499680122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d2672f60 0x5599d2a64f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f551effd640 0 client.15843 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 >> 172.21.15.94:0/1499680122 conn(0x5599d26a84e0 msgr2=0x5599d2a60120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 shutdown_connections 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 1 -- 172.21.15.94:0/1499680122 wait complete. 2024-04-25T13:42:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.307+0000 7f554c981740 20 client.15843 trim_cache size 0 max 16384 2024-04-25T13:42:53.313 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:42:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 msgr2=0x7f55280b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 0x7f55280b2090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f553802ade0 tx=0x7f553802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 msgr2=0x7f554019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5534005fd0 tx=0x7f5534007590 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 shutdown_connections 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55280afb80 0x7f55280b2090 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5540105090 0x7f55401a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5540102240 0x7f554019a6e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 --2- 172.21.15.94:0/3016012790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5540101740 0x7f554019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 >> 172.21.15.94:0/3016012790 conn(0x7f55400fd340 msgr2=0x7f5540107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 shutdown_connections 2024-04-25T13:42:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:53.308+0000 7f5545ad9640 1 -- 172.21.15.94:0/3016012790 wait complete. 2024-04-25T13:42:53.318 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:53 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2897194382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:53.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:53 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2897194382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:53.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:53 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2897194382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:42:54.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:42:54.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:54.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:42:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:54 smithi137 ceph-mon[32568]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 -- 172.21.15.94:0/1066213602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000104440 msgr2=0x7f3000102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 --2- 172.21.15.94:0/1066213602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000104440 0x7f3000102e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402a410 comp rx=0 tx=0).stop 2024-04-25T13:42:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 -- 172.21.15.94:0/1066213602 shutdown_connections 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 --2- 172.21.15.94:0/1066213602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000108ae0 0x7f300010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 --2- 172.21.15.94:0/1066213602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000104440 0x7f3000102e30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 --2- 172.21.15.94:0/1066213602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000103fa0 0x7f3000102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 -- 172.21.15.94:0/1066213602 >> 172.21.15.94:0/1066213602 conn(0x7f30000fd390 msgr2=0x7f30000ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 -- 172.21.15.94:0/1066213602 shutdown_connections 2024-04-25T13:42:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.437+0000 7f3005c92640 1 -- 172.21.15.94:0/1066213602 wait complete. 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 Processor -- start 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 -- start start 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 0x7f300019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000104440 0x7f300019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f640 con 0x7f3000108ae0 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f4c0 con 0x7f3000104440 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.438+0000 7f3005c92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f300010f7c0 con 0x7f3000103fa0 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43090/0 (socket says 172.21.15.94:43090) 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2ffffff640 1 -- 172.21.15.94:0/1576907961 learned_addr learned my addr 172.21.15.94:0/1576907961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 0x7f300019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000104440 0x7f300019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 -- 172.21.15.94:0/1576907961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000104440 msgr2=0x7f300019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000104440 0x7f300019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 -- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 msgr2=0x7f30001a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2fff7fe640 1 -- 172.21.15.94:0/1576907961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001a1d60 con 0x7f3000103fa0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.439+0000 7f2ffffff640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.440+0000 7f2fff7fe640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 0x7f300019a220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e9c0 tx=0x7f2ff000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.440+0000 7f2ffcff9640 1 -- 172.21.15.94:0/1576907961 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000ccc0 con 0x7f3000103fa0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.440+0000 7f2ffcff9640 1 -- 172.21.15.94:0/1576907961 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff00108e0 con 0x7f3000103fa0 2024-04-25T13:42:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.440+0000 7f2ffcff9640 1 -- 172.21.15.94:0/1576907961 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00187c0 con 0x7f3000103fa0 2024-04-25T13:42:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.441+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001a1ff0 con 0x7f3000103fa0 2024-04-25T13:42:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.441+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001a2460 con 0x7f3000103fa0 2024-04-25T13:42:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 Processor -- start 2024-04-25T13:42:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 -- start start 2024-04-25T13:42:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 0x558a88c5d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888832f0 0x558a88c5df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a88bb2460 con 0x558a8888b4d0 2024-04-25T13:42:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a88b6c590 con 0x558a888ac1b0 2024-04-25T13:42:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.444+0000 7f300cc1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a88bae420 con 0x558a888832f0 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.445+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.445+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43096/0 (socket says 172.21.15.94:43096) 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.445+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2066191310 learned_addr learned my addr 172.21.15.94:0/2066191310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.445+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 0x558a88c5d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 -- 172.21.15.94:0/2066191310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888832f0 msgr2=0x558a88c5df00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888832f0 0x558a88c5df00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 -- 172.21.15.94:0/2066191310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 msgr2=0x558a88c64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 -- 172.21.15.94:0/2066191310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a88b790d0 con 0x558a888ac1b0 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffcff9640 1 -- 172.21.15.94:0/1576907961 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f2ff0010430 con 0x7f3000103fa0 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 0x558a88c5d980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011780 tx=0x7f2ff8011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed2c35ac2e46a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2fe5ffb640 1 -- 172.21.15.94:0/2066191310 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800fae0 con 0x558a888ac1b0 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2fe5ffb640 1 -- 172.21.15.94:0/2066191310 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff800fc80 con 0x558a888ac1b0 2024-04-25T13:42:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.446+0000 7f2fe5ffb640 1 -- 172.21.15.94:0/2066191310 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff801a650 con 0x558a888ac1b0 2024-04-25T13:42:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.447+0000 7f2ffcff9640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 0x7f2fe80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.447+0000 7f2fff7fe640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 0x7f2fe80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.447+0000 7f300cc1b740 1 -- 172.21.15.94:0/2066191310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 msgr2=0x558a88c5d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.447+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 0x558a88c5d980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011780 tx=0x7f2ff8011c40 comp rx=0 tx=0).stop 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 -- 172.21.15.94:0/2066191310 shutdown_connections 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888832f0 0x558a88c5df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2066191310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888ac1b0 0x558a88c5d980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 -- 172.21.15.94:0/2066191310 >> 172.21.15.94:0/2066191310 conn(0x558a888a0300 msgr2=0x558a88c48300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f2ffcff9640 1 -- 172.21.15.94:0/1576907961 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2ff00df180 con 0x7f3000103fa0 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f2fff7fe640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 0x7f2fe80b2100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000eff0 tx=0x7f2ff00138f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 -- 172.21.15.94:0/2066191310 shutdown_connections 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 -- 172.21.15.94:0/2066191310 wait complete. 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 -- start start 2024-04-25T13:42:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.448+0000 7f300cc1b740 1 Processor -- start 2024-04-25T13:42:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888ac1b0 0x558a88c62e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a88c65640 con 0x558a8888b4d0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a88b39b60 con 0x558a888832f0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a88c657c0 con 0x558a888ac1b0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f300cc1b740 10 client.0 _ll_register_callbacks cb 0x558a88b5e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43100/0 (socket says 172.21.15.94:43100) 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2667844309 learned_addr learned my addr 172.21.15.94:0/2667844309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888ac1b0 0x558a88c62e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2667844309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888ac1b0 msgr2=0x558a88c62e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888ac1b0 0x558a88c62e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 msgr2=0x558a88c647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2667844309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a888c4210 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009230 tx=0x7f2ff4031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b58e949a7290e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.449+0000 7f2fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4035040 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2ff4004240 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff800fc80 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ff40318f0 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 10 client.15855 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a88990620 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4004440 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff4044020 con 0x558a8888b4d0 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 10 client.15855 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 20 client.15855 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a8886ad80 0x558a88c7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 4 client.15855 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.450+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ff800fc80 con 0x558a8886ad80 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.451+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a8886ad80 0x558a88c7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.451+0000 7f2fdeffd640 10 client.15855 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.452+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a8886ad80 0x558a88c7d980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499be0048ab260e7 server_cookie=3ef5e8b6892badc8 in_seq=0 out_seq=0 2024-04-25T13:42:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.452+0000 7f2fdeffd640 1 -- 172.21.15.94:0/2667844309 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2ff800fc80 con 0x558a8886ad80 2024-04-25T13:42:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.452+0000 7f2fdeffd640 10 client.15855 handle_command_reply: tid=0 2024-04-25T13:42:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.452+0000 7f300cc1b740 1 client.15855 shutdown 2024-04-25T13:42:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 msgr2=0x558a88c64ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009230 tx=0x7f2ff4031e80 comp rx=0 tx=0).stop 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 shutdown_connections 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a8886ad80 0x558a88c7d980 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a888ac1b0 0x558a88c62e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c64ce0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f2fdeffd640 0 client.15855 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2667844309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 >> 172.21.15.94:0/2667844309 conn(0x558a888a0300 msgr2=0x558a88c47250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 shutdown_connections 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 1 -- 172.21.15.94:0/2667844309 wait complete. 2024-04-25T13:42:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.453+0000 7f300cc1b740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T13:42:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.468+0000 7f300cc1b740 1 Processor -- start 2024-04-25T13:42:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.468+0000 7f300cc1b740 1 -- start start 2024-04-25T13:42:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 0x558a88c68990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c68ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3000073240 con 0x558a8888b4d0 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30001a2690 con 0x558a888832f0 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f300cc1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3000073910 con 0x558a8886ad80 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43110/0 (socket says 172.21.15.94:43110) 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffffff640 1 -- 172.21.15.94:0/2961151313 learned_addr learned my addr 172.21.15.94:0/2961151313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 0x558a88c68990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c68ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.469+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2961151313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 msgr2=0x558a88c68990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 0x558a88c68990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2961151313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 msgr2=0x558a88c616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffeffd640 1 -- 172.21.15.94:0/2961151313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a88c695d0 con 0x558a888832f0 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffffff640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 0x558a88c68990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c68ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49b08f42a68ccb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2fe57fa640 1 -- 172.21.15.94:0/2961151313 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4035070 con 0x558a888832f0 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2fe57fa640 1 -- 172.21.15.94:0/2961151313 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff40044d0 con 0x558a888832f0 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.470+0000 7f2fe57fa640 1 -- 172.21.15.94:0/2961151313 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4003ce0 con 0x558a888832f0 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 -- 172.21.15.94:0/2961151313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 msgr2=0x558a88c68ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c68ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402a410 comp rx=0 tx=0).stop 2024-04-25T13:42:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f48de7fc640 1 -- 172.21.15.94:0/4265113701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d00318a0 con 0x7f48e01029e0 2024-04-25T13:42:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 -- 172.21.15.94:0/2961151313 shutdown_connections 2024-04-25T13:42:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8888b4d0 0x558a88c616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c68ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 --2- 172.21.15.94:0/2961151313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8886ad80 0x558a88c68990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.471+0000 7f300cc1b740 1 -- 172.21.15.94:0/2961151313 >> 172.21.15.94:0/2961151313 conn(0x558a888a0300 msgr2=0x558a88c67c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f300cc1b740 1 -- 172.21.15.94:0/2961151313 shutdown_connections 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f300cc1b740 1 -- 172.21.15.94:0/2961151313 wait complete. 2024-04-25T13:42:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f300cc1b740 1 -- start start 2024-04-25T13:42:54.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 -- 172.21.15.94:0/4265113701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 msgr2=0x7f48e01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 --2- 172.21.15.94:0/4265113701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e01036e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009900 tx=0x7f48d002a3f0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 -- 172.21.15.94:0/4265113701 shutdown_connections 2024-04-25T13:42:54.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 --2- 172.21.15.94:0/4265113701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0108920 0x7f48e010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 --2- 172.21.15.94:0/4265113701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0103c20 0x7f48e0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 --2- 172.21.15.94:0/4265113701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e01036e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f300cc1b740 1 Processor -- start 2024-04-25T13:42:54.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 -- 172.21.15.94:0/4265113701 >> 172.21.15.94:0/4265113701 conn(0x7f48e00fdbd0 msgr2=0x7f48e0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 -- 172.21.15.94:0/4265113701 shutdown_connections 2024-04-25T13:42:54.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.472+0000 7f48e5a61640 1 -- 172.21.15.94:0/4265113701 wait complete. 2024-04-25T13:42:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c75740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43120/0 (socket says 172.21.15.94:43120) 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffffff640 1 -- 172.21.15.94:0/1322747160 learned_addr learned my addr 172.21.15.94:0/1322747160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8888b4d0 0x558a88c76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30000746b0 con 0x558a8886ad80 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3000073fe0 con 0x558a888832f0 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3000071220 con 0x558a8888b4d0 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c75740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f300cc1b740 10 client.0 _ll_register_callbacks cb 0x7f2ff4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffeffd640 1 -- 172.21.15.94:0/1322747160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8888b4d0 msgr2=0x558a88c76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.473+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8888b4d0 0x558a88c76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2ffeffd640 1 -- 172.21.15.94:0/1322747160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 msgr2=0x558a88c75200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2ffeffd640 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a88c76c30 con 0x558a888832f0 2024-04-25T13:42:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2ffffff640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 Processor -- start 2024-04-25T13:42:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 -- start start 2024-04-25T13:42:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2ffeffd640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c75740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4046d10 tx=0x7f2ff4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba57dc6ed3704c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:42:54.481 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:54 smithi094 ceph-mon[29195]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff405c060 con 0x558a888832f0 2024-04-25T13:42:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2ff4003a20 con 0x558a888832f0 2024-04-25T13:42:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff40044d0 con 0x558a888832f0 2024-04-25T13:42:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0103c20 0x7f48e010b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0108920 0x7f48e010ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48e010f670 con 0x7f48e01029e0 2024-04-25T13:42:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48e010f4f0 con 0x7f48e0103c20 2024-04-25T13:42:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.474+0000 7f48e5a61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48e010f7f0 con 0x7f48e0108920 2024-04-25T13:42:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4054070 con 0x558a888832f0 2024-04-25T13:42:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f300cc1b740 10 client.25355 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558a88c77ea0 con 0x558a888832f0 2024-04-25T13:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43124/0 (socket says 172.21.15.94:43124) 2024-04-25T13:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 -- 172.21.15.94:0/72064574 learned_addr learned my addr 172.21.15.94:0/72064574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48d67fc640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0103c20 0x7f48e010b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48df7fe640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0108920 0x7f48e010ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff40044d0 con 0x558a888832f0 2024-04-25T13:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 -- 172.21.15.94:0/72064574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0108920 msgr2=0x7f48e010ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0108920 0x7f48e010ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 -- 172.21.15.94:0/72064574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0103c20 msgr2=0x7f48e010b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0103c20 0x7f48e010b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.475+0000 7f48deffd640 1 -- 172.21.15.94:0/72064574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e010c1c0 con 0x7f48e01029e0 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff403c8f0 con 0x558a888832f0 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 10 client.25355 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 20 client.25355 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a888ac1b0 0x558a88ce8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 4 client.25355 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:42:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a88bc0070 con 0x558a888ac1b0 2024-04-25T13:42:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48deffd640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f48d0031500 tx=0x7f48d0003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f2ffffff640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a888ac1b0 0x558a88ce8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48e4a5f640 1 -- 172.21.15.94:0/72064574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d003f070 con 0x7f48e01029e0 2024-04-25T13:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e01a45d0 con 0x7f48e01029e0 2024-04-25T13:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e01a49d0 con 0x7f48e01029e0 2024-04-25T13:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48e4a5f640 1 -- 172.21.15.94:0/72064574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d0031c50 con 0x7f48e01029e0 2024-04-25T13:42:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.476+0000 7f48e4a5f640 1 -- 172.21.15.94:0/72064574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0035030 con 0x7f48e01029e0 2024-04-25T13:42:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.477+0000 7f2fdeffd640 10 client.25355 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.477+0000 7f2ffffff640 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a888ac1b0 0x558a88ce8350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d198322578e1a4 server_cookie=61e8ab175ccfe652 in_seq=0 out_seq=0 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.479+0000 7f48ec9ee740 1 Processor -- start 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 -- start start 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ba753fb00 con 0x563ba7218730 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ba74db920 con 0x563ba7239410 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ba753bac0 con 0x563ba7210550 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48d67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38886/0 (socket says 172.21.15.94:38886) 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48df7fe640 1 -- 172.21.15.94:0/761548405 learned_addr learned my addr 172.21.15.94:0/761548405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.480+0000 7f48e4a5f640 1 -- 172.21.15.94:0/72064574 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f48d0042070 con 0x7f48e01029e0 2024-04-25T13:42:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f2fdeffd640 1 -- 172.21.15.94:0/1322747160 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a88bc0070 con 0x558a888ac1b0 2024-04-25T13:42:54.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f2fdeffd640 10 client.25355 handle_command_reply: tid=0 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48deffd640 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 -- 172.21.15.94:0/761548405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 msgr2=0x563ba75d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75d6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 -- 172.21.15.94:0/761548405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 msgr2=0x563ba75d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 -- 172.21.15.94:0/761548405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba73197e0 con 0x563ba7218730 2024-04-25T13:42:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 client.25355 shutdown 2024-04-25T13:42:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 msgr2=0x558a88c75740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c75740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4046d10 tx=0x7f2ff4003820 comp rx=0 tx=0).stop 2024-04-25T13:42:54.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d67fc640 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75dcf50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005d00 tx=0x7f48c80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad052fb72aed2e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 shutdown_connections 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a888ac1b0 0x558a88ce8350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8888b4d0 0x558a88c76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a888832f0 0x558a88c75740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 --2- 172.21.15.94:0/1322747160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a8886ad80 0x558a88c75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 >> 172.21.15.94:0/1322747160 conn(0x558a888a0300 msgr2=0x558a88c64760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f2fdeffd640 0 client.25355 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 shutdown_connections 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f300cc1b740 1 -- 172.21.15.94:0/1322747160 wait complete. 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f300cc1b740 20 client.25355 trim_cache size 0 max 16384 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:42:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:42:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 msgr2=0x7f2fe80b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 0x7f2fe80b2100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000eff0 tx=0x7f2ff00138f0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 msgr2=0x7f300019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 0x7f300019a220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000e9c0 tx=0x7f2ff000ee80 comp rx=0 tx=0).stop 2024-04-25T13:42:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 shutdown_connections 2024-04-25T13:42:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2fe80afbf0 0x7f2fe80b2100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3000108ae0 0x7f30001a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3000104440 0x7f300019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 --2- 172.21.15.94:0/1576907961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3000103fa0 0x7f300019a220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 >> 172.21.15.94:0/1576907961 conn(0x7f30000fd390 msgr2=0x7f3000104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 shutdown_connections 2024-04-25T13:42:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f3005c92640 1 -- 172.21.15.94:0/1576907961 wait complete. 2024-04-25T13:42:54.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:54 smithi138 ceph-mon[32977]: pgmap v379: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d5ffb640 1 -- 172.21.15.94:0/761548405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c801c680 con 0x563ba7218730 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.481+0000 7f48d5ffb640 1 -- 172.21.15.94:0/761548405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c800be20 con 0x563ba7218730 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f48d5ffb640 1 -- 172.21.15.94:0/761548405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011ba0 con 0x563ba7218730 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f48e4a5f640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 0x7f48cc0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f48e4a5f640 1 -- 172.21.15.94:0/72064574 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f48d00ff610 con 0x7f48e01029e0 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.482+0000 7f48df7fe640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 0x7f48cc0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 -- 172.21.15.94:0/761548405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 msgr2=0x563ba75dcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75dcf50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005d00 tx=0x7f48c80107a0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48df7fe640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 0x7f48cc0b2290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x563ba75d6ca0 tx=0x7f48d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 -- 172.21.15.94:0/761548405 shutdown_connections 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75dcf50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/761548405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 -- 172.21.15.94:0/761548405 >> 172.21.15.94:0/761548405 conn(0x563ba722d560 msgr2=0x563ba75de640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 -- 172.21.15.94:0/761548405 shutdown_connections 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.483+0000 7f48ec9ee740 1 -- 172.21.15.94:0/761548405 wait complete. 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.484+0000 7f48ec9ee740 1 -- start start 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.484+0000 7f48ec9ee740 1 Processor -- start 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ba75dd9a0 con 0x563ba7239410 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ba74d95d0 con 0x563ba7218730 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ba75ddb20 con 0x563ba7210550 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48d67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48d67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38890/0 (socket says 172.21.15.94:38890) 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43150/0 (socket says 172.21.15.94:43150) 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48d67fc640 1 -- 172.21.15.94:0/3930752009 learned_addr learned my addr 172.21.15.94:0/3930752009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.485+0000 7f48ec9ee740 10 client.0 _ll_register_callbacks cb 0x563ba74c7080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 -- 172.21.15.94:0/3930752009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 msgr2=0x563ba75f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 -- 172.21.15.94:0/3930752009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 msgr2=0x563ba75fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 -- 172.21.15.94:0/3930752009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba7379f70 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48deffd640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48df7fe640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48d67fc640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48c80116c0 tx=0x7f48c80118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d2ce136664949b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48beffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c802b060 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f48c8005040 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.486+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c800be20 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.487+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48c8005240 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.487+0000 7f48ec9ee740 10 client.25364 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.487+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ba7211170 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.487+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c80053e0 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f48c8025070 con 0x563ba7218730 2024-04-25T13:42:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 10 client.25364 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 20 client.25364 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71f81f0 0x563ba76028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 4 client.25364 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.488+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48c800be20 con 0x563ba71f81f0 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.489+0000 7f48deffd640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71f81f0 0x563ba76028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.489+0000 7f48beffd640 10 client.25364 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.489+0000 7f48deffd640 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71f81f0 0x563ba76028e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e3069393fdbb4a server_cookie=7a63a9c8e0b2b4e3 in_seq=0 out_seq=0 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48beffd640 1 -- 172.21.15.94:0/3930752009 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f48c800be20 con 0x563ba71f81f0 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48beffd640 10 client.25364 handle_command_reply: tid=0 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 client.25364 shutdown 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 msgr2=0x563ba75f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48c80116c0 tx=0x7f48c80118e0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 shutdown_connections 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71f81f0 0x563ba76028e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7239410 0x563ba75fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7218730 0x563ba75f5880 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/3930752009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 >> 172.21.15.94:0/3930752009 conn(0x563ba722d560 msgr2=0x563ba75d4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.490+0000 7f48beffd640 0 client.25364 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.491+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 shutdown_connections 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.491+0000 7f48ec9ee740 1 -- 172.21.15.94:0/3930752009 wait complete. 2024-04-25T13:42:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.491+0000 7f48ec9ee740 20 client.25364 trim_cache size 0 max 16384 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.526+0000 7f48ec9ee740 1 Processor -- start 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 -- start start 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7218730 0x563ba75f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01a9160 con 0x563ba7210550 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01aaca0 con 0x563ba7239410 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48ec9ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01a9830 con 0x563ba7218730 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43160/0 (socket says 172.21.15.94:43160) 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 -- 172.21.15.94:0/4117930054 learned_addr learned my addr 172.21.15.94:0/4117930054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48df7fe640 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48d67fc640 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7218730 0x563ba75f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 -- 172.21.15.94:0/4117930054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7218730 msgr2=0x563ba75f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7218730 0x563ba75f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 -- 172.21.15.94:0/4117930054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 msgr2=0x563ba75fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.527+0000 7f48deffd640 1 -- 172.21.15.94:0/4117930054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba75da7b0 con 0x563ba7210550 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.528+0000 7f48deffd640 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c78d0 tx=0x7f48d00d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aa1c87f7c6334a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.528+0000 7f48d57fa640 1 -- 172.21.15.94:0/4117930054 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0049040 con 0x563ba7210550 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.528+0000 7f48d57fa640 1 -- 172.21.15.94:0/4117930054 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d0031e00 con 0x563ba7210550 2024-04-25T13:42:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.528+0000 7f48d57fa640 1 -- 172.21.15.94:0/4117930054 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d00d06f0 con 0x563ba7210550 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 -- 172.21.15.94:0/4117930054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 msgr2=0x563ba75f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c78d0 tx=0x7f48d00d0040 comp rx=0 tx=0).stop 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 -- 172.21.15.94:0/4117930054 shutdown_connections 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7218730 0x563ba75f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/4117930054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7210550 0x563ba75f3940 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 -- 172.21.15.94:0/4117930054 >> 172.21.15.94:0/4117930054 conn(0x563ba722d560 msgr2=0x563ba75de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 -- 172.21.15.94:0/4117930054 shutdown_connections 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.529+0000 7f48ec9ee740 1 -- 172.21.15.94:0/4117930054 wait complete. 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48ec9ee740 1 -- start start 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48ec9ee740 1 Processor -- start 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48ec9ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48d67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.530+0000 7f48d67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43162/0 (socket says 172.21.15.94:43162) 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48d67fc640 1 -- 172.21.15.94:0/2447262534 learned_addr learned my addr 172.21.15.94:0/2447262534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01aa5d0 con 0x563ba7218730 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01a9f00 con 0x563ba7239410 2024-04-25T13:42:54.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48e01a71f0 con 0x563ba7210550 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48df7fe640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 -- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 msgr2=0x563ba75f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 -- 172.21.15.94:0/2447262534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 msgr2=0x563ba75f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba7668b70 con 0x563ba7210550 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48ec9ee740 10 client.0 _ll_register_callbacks cb 0x7f48c800b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ba754d710 con 0x563ba7210550 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48d67fc640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48deffd640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75fcee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c7640 tx=0x7f48d00cffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8041db26c664ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.531+0000 7f48beffd640 10 client.25324 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48beffd640 1 -- 172.21.15.94:0/2447262534 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0049070 con 0x563ba7210550 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48beffd640 1 -- 172.21.15.94:0/2447262534 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f48d00c3b40 con 0x563ba7210550 2024-04-25T13:42:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48beffd640 1 -- 172.21.15.94:0/2447262534 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48d00d06f0 con 0x563ba7210550 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 10 client.25324 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=25}) -- 0x563ba7669be0 con 0x563ba7210550 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48beffd640 1 -- 172.21.15.94:0/2447262534 <== mon.1 v2:172.21.15.138:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f48d00dcce0 con 0x563ba7210550 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 10 client.25324 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 20 client.25324 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71cd630 0x563ba766c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 4 client.25324 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.532+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ba7547580 con 0x563ba71cd630 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.533+0000 7f48d67fc640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71cd630 0x563ba766c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.533+0000 7f48beffd640 10 client.25324 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.533+0000 7f48d67fc640 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71cd630 0x563ba766c510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1685df830472a5 server_cookie=182869aba9c3b5c in_seq=0 out_seq=0 2024-04-25T13:42:54.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.534+0000 7f48beffd640 1 -- 172.21.15.94:0/2447262534 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ba7547580 con 0x563ba71cd630 2024-04-25T13:42:54.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.534+0000 7f48beffd640 10 client.25324 handle_command_reply: tid=0 2024-04-25T13:42:54.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.534+0000 7f48ec9ee740 1 client.25324 shutdown 2024-04-25T13:42:54.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.534+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 msgr2=0x563ba75fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.534+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75fcee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48d00c7640 tx=0x7f48d00cffd0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 shutdown_connections 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ba71cd630 0x563ba766c510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ba7239410 0x563ba75f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48beffd640 0 client.25324 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ba7218730 0x563ba75f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 --2- 172.21.15.94:0/2447262534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ba7210550 0x563ba75fcee0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 >> 172.21.15.94:0/2447262534 conn(0x563ba722d560 msgr2=0x563ba75f3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 shutdown_connections 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 1 -- 172.21.15.94:0/2447262534 wait complete. 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48ec9ee740 20 client.25324 trim_cache size 0 max 16384 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "880a8df0-45cd-4383-87f8-b9f12f63b5a2", 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 msgr2=0x7f48cc0b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 0x7f48cc0b2290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x563ba75d6ca0 tx=0x7f48d8009500 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 msgr2=0x7f48e010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.535+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f48d0031500 tx=0x7f48d0003bb0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 shutdown_connections 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48cc0afd80 0x7f48cc0b2290 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e0108920 0x7f48e010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48e0103c20 0x7f48e010b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 --2- 172.21.15.94:0/72064574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48e01029e0 0x7f48e010ceb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 >> 172.21.15.94:0/72064574 conn(0x7f48e00fdbd0 msgr2=0x7f48e0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 shutdown_connections 2024-04-25T13:42:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.536+0000 7f48e5a61640 1 -- 172.21.15.94:0/72064574 wait complete. 2024-04-25T13:42:54.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:42:54.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:42:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 -- 172.21.15.94:0/3265014089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 msgr2=0x7fc07010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 --2- 172.21.15.94:0/3265014089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc07010ad40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc068009ca0 tx=0x7fc06802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 -- 172.21.15.94:0/3265014089 shutdown_connections 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 --2- 172.21.15.94:0/3265014089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc07010ad40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 --2- 172.21.15.94:0/3265014089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc070103c20 0x7fc070104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 --2- 172.21.15.94:0/3265014089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0701029e0 0x7fc0701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 -- 172.21.15.94:0/3265014089 >> 172.21.15.94:0/3265014089 conn(0x7fc0700fdbd0 msgr2=0x7fc070100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 -- 172.21.15.94:0/3265014089 shutdown_connections 2024-04-25T13:42:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.699+0000 7fc074e05640 1 -- 172.21.15.94:0/3265014089 wait complete. 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 Processor -- start 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 -- start start 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0701029e0 0x7fc07019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc070103c20 0x7fc07019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f610 con 0x7fc070108920 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f490 con 0x7fc0701029e0 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc074e05640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f790 con 0x7fc070103c20 2024-04-25T13:42:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43180/0 (socket says 172.21.15.94:43180) 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc06ed9d640 1 -- 172.21.15.94:0/239953345 learned_addr learned my addr 172.21.15.94:0/239953345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc06e59c640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0701029e0 0x7fc07019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.700+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc070103c20 0x7fc07019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 -- 172.21.15.94:0/239953345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc070103c20 msgr2=0x7fc07019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc070103c20 0x7fc07019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 -- 172.21.15.94:0/239953345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0701029e0 msgr2=0x7fc07019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0701029e0 0x7fc07019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 -- 172.21.15.94:0/239953345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0701a2040 con 0x7fc070108920 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc06ed9d640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc0680097b0 tx=0x7fc068003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc05f7fe640 1 -- 172.21.15.94:0/239953345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068004240 con 0x7fc070108920 2024-04-25T13:42:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0701a22d0 con 0x7fc070108920 2024-04-25T13:42:54.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0701a2870 con 0x7fc070108920 2024-04-25T13:42:54.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc05f7fe640 1 -- 172.21.15.94:0/239953345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06802f380 con 0x7fc070108920 2024-04-25T13:42:54.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.701+0000 7fc05f7fe640 1 -- 172.21.15.94:0/239953345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06802fc30 con 0x7fc070108920 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.704+0000 7fc07bd94740 1 Processor -- start 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.704+0000 7fc05f7fe640 1 -- 172.21.15.94:0/239953345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7fc0680043e0 con 0x7fc070108920 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.704+0000 7fc07bd94740 1 -- start start 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642857fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642858c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5566427bcd80 con 0x5566424b6390 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5566427768b0 con 0x55664248d4d0 2024-04-25T13:42:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc07bd94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566427b8d40 con 0x5566424956b0 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43186/0 (socket says 172.21.15.94:43186) 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 -- 172.21.15.94:0/213877522 learned_addr learned my addr 172.21.15.94:0/213877522 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06e59c640 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642857fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc05f7fe640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 0x7fc0580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 -- 172.21.15.94:0/213877522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 msgr2=0x556642858c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642858c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 -- 172.21.15.94:0/213877522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 msgr2=0x556642857fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642857fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06dd9b640 1 -- 172.21.15.94:0/213877522 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556642747a40 con 0x5566424b6390 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06e59c640 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.705+0000 7fc06ed9d640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 0x7fc0580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.706+0000 7fc05f7fe640 1 -- 172.21.15.94:0/239953345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc068100700 con 0x7fc070108920 2024-04-25T13:42:54.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.706+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc06400bd90 tx=0x7fc06400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714aa73a732902dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.706+0000 7fc05cff9640 1 -- 172.21.15.94:0/213877522 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064015070 con 0x5566424b6390 2024-04-25T13:42:54.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.706+0000 7fc05cff9640 1 -- 172.21.15.94:0/213877522 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06400fe40 con 0x5566424b6390 2024-04-25T13:42:54.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.706+0000 7fc05cff9640 1 -- 172.21.15.94:0/213877522 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064011890 con 0x5566424b6390 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 -- 172.21.15.94:0/213877522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 msgr2=0x556642857a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc06ed9d640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 0x7fc0580b2090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0680c8a80 tx=0x7fc068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc06400bd90 tx=0x7fc06400f730 comp rx=0 tx=0).stop 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 -- 172.21.15.94:0/213877522 shutdown_connections 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642858c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 --2- 172.21.15.94:0/213877522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424b6390 0x556642857a60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 -- 172.21.15.94:0/213877522 >> 172.21.15.94:0/213877522 conn(0x5566424aa4e0 msgr2=0x556642853270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 -- 172.21.15.94:0/213877522 shutdown_connections 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.707+0000 7fc07bd94740 1 -- 172.21.15.94:0/213877522 wait complete. 2024-04-25T13:42:54.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc07bd94740 1 -- start start 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc07bd94740 1 Processor -- start 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642861e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642868da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.708+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc07bd94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556642859780 con 0x55664248d4d0 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43192/0 (socket says 172.21.15.94:43192) 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 -- 172.21.15.94:0/832049799 learned_addr learned my addr 172.21.15.94:0/832049799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556642743d10 con 0x5566424b6390 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556642859900 con 0x5566424956b0 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc07bd94740 10 client.0 _ll_register_callbacks cb 0x556642770210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06e59c640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642861e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 -- 172.21.15.94:0/832049799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 msgr2=0x556642861e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642861e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 -- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 msgr2=0x556642868da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642868da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566425987d0 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b460 tx=0x7fc064005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a609d4440678df29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064027020 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566427ca990 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.709+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc064004220 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc064015040 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 10 client.15891 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566424ce3f0 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5566427ca990 con 0x55664248d4d0 2024-04-25T13:42:54.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc064022070 con 0x55664248d4d0 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 10 client.15891 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 20 client.15891 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556642474f60 0x55664286eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 4 client.15891 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566427c4800 con 0x556642474f60 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.710+0000 7fc06e59c640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556642474f60 0x55664286eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.711+0000 7fc055ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.711+0000 7fc06e59c640 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556642474f60 0x55664286eec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff0fcb08cfb145d server_cookie=c2fa1553d5373d1f in_seq=0 out_seq=0 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc055ffb640 1 -- 172.21.15.94:0/832049799 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566427c4800 con 0x556642474f60 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc055ffb640 10 client.15891 handle_command_reply: tid=0 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 client.15891 shutdown 2024-04-25T13:42:54.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 msgr2=0x556642861900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b460 tx=0x7fc064005270 comp rx=0 tx=0).stop 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 shutdown_connections 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556642474f60 0x55664286eec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc055ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642868da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424956b0 0x556642861e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 --2- 172.21.15.94:0/832049799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55664248d4d0 0x556642861900 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 >> 172.21.15.94:0/832049799 conn(0x5566424aa4e0 msgr2=0x556642856180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 shutdown_connections 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 1 -- 172.21.15.94:0/832049799 wait complete. 2024-04-25T13:42:54.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.712+0000 7fc07bd94740 20 client.15891 trim_cache size 0 max 16384 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.730+0000 7fc07bd94740 1 Processor -- start 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.730+0000 7fc07bd94740 1 -- start start 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.730+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55664248d4d0 0x55664285ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.730+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642860440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc07bd94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ad780 con 0x5566424956b0 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc07bd94740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701af2c0 con 0x5566424b6390 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc07bd94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ade50 con 0x55664248d4d0 2024-04-25T13:42:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43198/0 (socket says 172.21.15.94:43198) 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 -- 172.21.15.94:0/1883843793 learned_addr learned my addr 172.21.15.94:0/1883843793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55664248d4d0 0x55664285ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642860440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 -- 172.21.15.94:0/1883843793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55664248d4d0 msgr2=0x55664285ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55664248d4d0 0x55664285ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 -- 172.21.15.94:0/1883843793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 msgr2=0x556642860440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642860440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 -- 172.21.15.94:0/1883843793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664248e0f0 con 0x5566424956b0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc06e59c640 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc06000be60 tx=0x7fc06000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c51f80f81de765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc057fff640 1 -- 172.21.15.94:0/1883843793 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060005220 con 0x5566424956b0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.731+0000 7fc057fff640 1 -- 172.21.15.94:0/1883843793 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060024030 con 0x5566424956b0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc057fff640 1 -- 172.21.15.94:0/1883843793 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0600205b0 con 0x5566424956b0 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 -- 172.21.15.94:0/1883843793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 msgr2=0x556642868d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc06000be60 tx=0x7fc06000b390 comp rx=0 tx=0).stop 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 -- 172.21.15.94:0/1883843793 shutdown_connections 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566424b6390 0x556642860440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x556642868d30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 --2- 172.21.15.94:0/1883843793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55664248d4d0 0x55664285ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 -- 172.21.15.94:0/1883843793 >> 172.21.15.94:0/1883843793 conn(0x5566424aa4e0 msgr2=0x5566428579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.732+0000 7fc07bd94740 1 -- 172.21.15.94:0/1883843793 shutdown_connections 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 -- 172.21.15.94:0/1883843793 wait complete. 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 -- start start 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 Processor -- start 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642868d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 0x556642865f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701aebf0 con 0x5566424956b0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ae520 con 0x55664248d4d0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701ab760 con 0x5566424b6390 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc07bd94740 10 client.0 _ll_register_callbacks cb 0x7fc06000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43210/0 (socket says 172.21.15.94:43210) 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc06e59c640 1 -- 172.21.15.94:0/4097533062 learned_addr learned my addr 172.21.15.94:0/4097533062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642868d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.733+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 0x556642865f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 -- 172.21.15.94:0/4097533062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 msgr2=0x556642865f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 0x556642865f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 -- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 msgr2=0x556642868d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642868d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 -- 172.21.15.94:0/4097533062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664277d4b0 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 0x556642865f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc06e59c640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc06000bfd0 tx=0x7fc060005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5774c696096189aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06002d070 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc060004730 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc060024030 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc060039050 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc07bd94740 10 client.15897 fetch_fsmap learned FSMap version 25 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566428d62c0 con 0x5566424956b0 2024-04-25T13:42:54.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.734+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060024030 con 0x5566424956b0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc060029480 con 0x5566424956b0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 10 client.15897 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 20 client.15897 populate_metadata read hostname 'smithi094' 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55664244a3a0 0x5566428d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 4 client.15897 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55664273f4a0 con 0x55664244a3a0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.735+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55664244a3a0 0x5566428d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc06dd9b640 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55664244a3a0 0x5566428d8d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3add56bc47868a8 server_cookie=d1ea66d6a064e550 in_seq=0 out_seq=0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc055ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc055ffb640 1 -- 172.21.15.94:0/4097533062 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55664273f4a0 con 0x55664244a3a0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc055ffb640 10 client.15897 handle_command_reply: tid=0 2024-04-25T13:42:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 client.15897 shutdown 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 msgr2=0x5566428658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc06000bfd0 tx=0x7fc060005130 comp rx=0 tx=0).stop 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 shutdown_connections 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55664244a3a0 0x5566428d8d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566424b6390 0x556642865f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566424956b0 0x5566428658f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 --2- 172.21.15.94:0/4097533062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55664248d4d0 0x556642868d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 >> 172.21.15.94:0/4097533062 conn(0x5566424aa4e0 msgr2=0x55664285ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc055ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 shutdown_connections 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 1 -- 172.21.15.94:0/4097533062 wait complete. 2024-04-25T13:42:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.736+0000 7fc07bd94740 20 client.15897 trim_cache size 0 max 16384 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "880a8df0-45cd-4383-87f8-b9f12f63b5a2": { 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "880a8df0-45cd-4383-87f8-b9f12f63b5a2", 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 msgr2=0x7fc0580b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 0x7fc0580b2090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0680c8a80 tx=0x7fc068005fd0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 msgr2=0x7fc0701a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc0680097b0 tx=0x7fc068003c20 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 shutdown_connections 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0580afb80 0x7fc0580b2090 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc070108920 0x7fc0701a19d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc070103c20 0x7fc07019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 --2- 172.21.15.94:0/239953345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0701029e0 0x7fc07019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 >> 172.21.15.94:0/239953345 conn(0x7fc0700fdbd0 msgr2=0x7fc070104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 shutdown_connections 2024-04-25T13:42:54.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:42:54.737+0000 7fc074e05640 1 -- 172.21.15.94:0/239953345 wait complete. 2024-04-25T13:42:54.748 INFO:tasks.cephfs.filesystem:scrub status for tag:880a8df0-45cd-4383-87f8-b9f12f63b5a2 - {'path': '/', 'tag': '880a8df0-45cd-4383-87f8-b9f12f63b5a2', 'options': 'recursive,force'} 2024-04-25T13:42:55.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:55 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:42:55.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:55.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:42:55.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:55.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:55 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:42:55.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:55.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:42:55.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:55.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:55 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:42:55.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:55.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:42:55.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:42:56.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:56 smithi137 ceph-mon[32568]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:56.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:42:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:56 smithi094 ceph-mon[29195]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:56.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:42:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:56 smithi138 ceph-mon[32977]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T13:42:56.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:42:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:42:58 smithi137 ceph-mon[32568]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:42:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:42:58 smithi094 ceph-mon[29195]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:42:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:42:58 smithi138 ceph-mon[32977]: pgmap v381: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T13:43:00.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:00 smithi094 ceph-mon[29195]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:43:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:00 smithi138 ceph-mon[32977]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:43:00.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:00 smithi137 ceph-mon[32568]: pgmap v382: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T13:43:02.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:02 smithi094 ceph-mon[29195]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:02 smithi138 ceph-mon[32977]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:02 smithi137 ceph-mon[32568]: pgmap v383: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:04 smithi094 ceph-mon[29195]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:04 smithi138 ceph-mon[32977]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:04.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:04 smithi137 ceph-mon[32568]: pgmap v384: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:05.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:05 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:43:05.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:05 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:43:05.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:05 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:43:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:06 smithi094 ceph-mon[29195]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:06 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:06 smithi138 ceph-mon[32977]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:06 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:06 smithi137 ceph-mon[32568]: pgmap v385: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T13:43:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:06 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:07.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:43:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:43:07.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:43:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:43:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 -- 172.21.15.94:0/176715166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 msgr2=0x7f378410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 --2- 172.21.15.94:0/176715166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378410a510 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 -- 172.21.15.94:0/176715166 shutdown_connections 2024-04-25T13:43:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 --2- 172.21.15.94:0/176715166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378410a510 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 --2- 172.21.15.94:0/176715166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841033f0 0x7f3784103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 --2- 172.21.15.94:0/176715166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841021b0 0x7f3784102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 -- 172.21.15.94:0/176715166 >> 172.21.15.94:0/176715166 conn(0x7f37840fd3a0 msgr2=0x7f37840ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.632+0000 7f378a642640 1 -- 172.21.15.94:0/176715166 shutdown_connections 2024-04-25T13:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.633+0000 7f378a642640 1 -- 172.21.15.94:0/176715166 wait complete. 2024-04-25T13:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.633+0000 7f378a642640 1 Processor -- start 2024-04-25T13:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.633+0000 7f378a642640 1 -- start start 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 0x7f3784197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841033f0 0x7f3784198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f3788bdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f3788bdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58572/0 (socket says 172.21.15.94:58572) 2024-04-25T13:43:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f3788bdf640 1 -- 172.21.15.94:0/2499997763 learned_addr learned my addr 172.21.15.94:0/2499997763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d440 con 0x7f37841080f0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d2c0 con 0x7f37841021b0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f3783fff640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 0x7f3784197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f378410d5c0 con 0x7f37841033f0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.634+0000 7f37837fe640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841033f0 0x7f3784198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 -- 172.21.15.94:0/2499997763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841033f0 msgr2=0x7f3784198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841033f0 0x7f3784198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 -- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 msgr2=0x7f378419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419fb20 con 0x7f37841021b0 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3788bdf640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f3783fff640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 0x7f3784197fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3784101a80 tx=0x7f377400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f37817fa640 1 -- 172.21.15.94:0/2499997763 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377400cc40 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f37817fa640 1 -- 172.21.15.94:0/2499997763 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377400cde0 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f37817fa640 1 -- 172.21.15.94:0/2499997763 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774010430 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419fe10 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.635+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a0220 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 Processor -- start 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 -- start start 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37817fa640 1 -- 172.21.15.94:0/2499997763 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 147571+0+0 (secure 0 0 0) 0x7f37740105d0 con 0x7f37841021b0 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 0x556277aad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556277a0a460 con 0x5562777041b0 2024-04-25T13:43:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5562779c4590 con 0x5562776db2f0 2024-04-25T13:43:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37915cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556277a06420 con 0x5562776e34d0 2024-04-25T13:43:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f3788bdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f3788bdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58574/0 (socket says 172.21.15.94:58574) 2024-04-25T13:43:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f3788bdf640 1 -- 172.21.15.94:0/4290001562 learned_addr learned my addr 172.21.15.94:0/4290001562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f37837fe640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37817fa640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 0x7f376c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 -- 172.21.15.94:0/4290001562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 msgr2=0x556277aad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.639+0000 7f3783fff640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 0x556277aad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 0x556277aad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 -- 172.21.15.94:0/4290001562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 msgr2=0x556277aad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 -- 172.21.15.94:0/4290001562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562779d10d0 con 0x5562776e34d0 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f3788bdf640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37817fa640 1 -- 172.21.15.94:0/2499997763 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f37740df250 con 0x7f37841021b0 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f3788bdf640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 0x7f376c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f3783fff640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 0x556277aad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f37837fe640 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab4600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f377800ac60 tx=0x7f3778005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835fb65ff433b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.640+0000 7f376a7fc640 1 -- 172.21.15.94:0/4290001562 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778004230 con 0x5562776e34d0 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.641+0000 7f376a7fc640 1 -- 172.21.15.94:0/4290001562 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377800ae10 con 0x5562776e34d0 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.641+0000 7f376a7fc640 1 -- 172.21.15.94:0/4290001562 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778018720 con 0x5562776e34d0 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f3788bdf640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 0x7f376c0b2090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f377c033040 tx=0x7f377c009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 -- 172.21.15.94:0/4290001562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 msgr2=0x556277ab4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab4600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f377800ac60 tx=0x7f3778005f90 comp rx=0 tx=0).stop 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 -- 172.21.15.94:0/4290001562 shutdown_connections 2024-04-25T13:43:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab4600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776db2f0 0x556277aad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 --2- 172.21.15.94:0/4290001562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562777041b0 0x556277aad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 -- 172.21.15.94:0/4290001562 >> 172.21.15.94:0/4290001562 conn(0x5562776f8300 msgr2=0x556277aa0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.642+0000 7f37915cb740 1 -- 172.21.15.94:0/4290001562 shutdown_connections 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.643+0000 7f37915cb740 1 -- 172.21.15.94:0/4290001562 wait complete. 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.643+0000 7f37915cb740 1 -- start start 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.644+0000 7f37915cb740 1 Processor -- start 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.644+0000 7f37915cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.644+0000 7f37915cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.644+0000 7f3783fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.644+0000 7f3783fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58588/0 (socket says 172.21.15.94:58588) 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277abe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41320/0 (socket says 172.21.15.94:41320) 2024-04-25T13:43:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556277ab5050 con 0x5562776db2f0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f3783fff640 1 -- 172.21.15.94:0/176800358 learned_addr learned my addr 172.21.15.94:0/176800358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556277991b60 con 0x5562777041b0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556277ab51d0 con 0x5562776e34d0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f3788bdf640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277abe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 -- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 msgr2=0x556277abe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277abe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 10 client.0 _ll_register_callbacks cb 0x5562779b62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 -- 172.21.15.94:0/176800358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 msgr2=0x556277ab6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37837fe640 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55627771c210 con 0x5562776e34d0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377800ae10 con 0x5562776e34d0 2024-04-25T13:43:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.645+0000 7f3783fff640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37837fe640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f377801a040 tx=0x7f3778003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c07bc48af973d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37637fe640 10 client.25351 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37637fe640 1 -- 172.21.15.94:0/176800358 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778032050 con 0x5562776e34d0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37637fe640 1 -- 172.21.15.94:0/176800358 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f37780041e0 con 0x5562776e34d0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37637fe640 1 -- 172.21.15.94:0/176800358 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3778028070 con 0x5562776e34d0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37915cb740 10 client.25351 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.646+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5562777e8620 con 0x5562776e34d0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37637fe640 1 -- 172.21.15.94:0/176800358 <== mon.1 v2:172.21.15.138:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f377800b040 con 0x5562776e34d0 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 10 client.25351 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 10 client.25351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 20 client.25351 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5562776c2d80 0x556277ac4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 4 client.25351 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556277a18070 con 0x5562776c2d80 2024-04-25T13:43:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.647+0000 7f3783fff640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5562776c2d80 0x556277ac4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:43:07.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:43:07.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:43:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.648+0000 7f37637fe640 10 client.25351 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.648+0000 7f3783fff640 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5562776c2d80 0x556277ac4670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d89fbb805cce63 server_cookie=22522fc9fa36d4e5 in_seq=0 out_seq=0 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.648+0000 7f37637fe640 1 -- 172.21.15.94:0/176800358 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556277a18070 con 0x5562776c2d80 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.648+0000 7f37637fe640 10 client.25351 handle_command_reply: tid=0 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.648+0000 7f37915cb740 1 client.25351 shutdown 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 msgr2=0x556277ab7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f377801a040 tx=0x7f3778003820 comp rx=0 tx=0).stop 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 shutdown_connections 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5562776c2d80 0x556277ac4670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277abe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37637fe640 0 client.25351 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776e34d0 0x556277ab7390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 --2- 172.21.15.94:0/176800358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277ab6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 >> 172.21.15.94:0/176800358 conn(0x5562776f8300 msgr2=0x556277a9f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 shutdown_connections 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 1 -- 172.21.15.94:0/176800358 wait complete. 2024-04-25T13:43:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.649+0000 7f37915cb740 20 client.25351 trim_cache size 0 max 16384 2024-04-25T13:43:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.679+0000 7f37915cb740 1 Processor -- start 2024-04-25T13:43:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.679+0000 7f37915cb740 1 -- start start 2024-04-25T13:43:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.679+0000 7f37915cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 0x556277aa02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.679+0000 7f37915cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f3783fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 0x556277aa02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58594/0 (socket says 172.21.15.94:58594) 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.679+0000 7f37915cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277a9e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f3783fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 0x556277aa02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41330/0 (socket says 172.21.15.94:41330) 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 -- 172.21.15.94:0/3806146921 learned_addr learned my addr 172.21.15.94:0/3806146921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37841a9300 con 0x5562776e34d0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37841aae40 con 0x5562777041b0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37841a99d0 con 0x5562776db2f0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f3788bdf640 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277a9e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 -- 172.21.15.94:0/3806146921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 msgr2=0x556277aa02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 0x556277aa02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 -- 172.21.15.94:0/3806146921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 msgr2=0x556277a9e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277a9e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f37837fe640 1 -- 172.21.15.94:0/3806146921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556277ab1cf0 con 0x5562776e34d0 2024-04-25T13:43:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.680+0000 7f3788bdf640 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277a9e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.681+0000 7f37837fe640 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f377800c010 tx=0x7f3778025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cc72c803d742c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.681+0000 7f3769ffb640 1 -- 172.21.15.94:0/3806146921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778032040 con 0x5562776e34d0 2024-04-25T13:43:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.681+0000 7f3769ffb640 1 -- 172.21.15.94:0/3806146921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377803b020 con 0x5562776e34d0 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.681+0000 7f3769ffb640 1 -- 172.21.15.94:0/3806146921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780042a0 con 0x5562776e34d0 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 msgr2=0x556277abe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f377800c010 tx=0x7f3778025000 comp rx=0 tx=0).stop 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 shutdown_connections 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562777041b0 0x556277a9e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776e34d0 0x556277abe210 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 --2- 172.21.15.94:0/3806146921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562776db2f0 0x556277aa02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.682+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 >> 172.21.15.94:0/3806146921 conn(0x5562776f8300 msgr2=0x556277a9fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.683+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 shutdown_connections 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.683+0000 7f37915cb740 1 -- 172.21.15.94:0/3806146921 wait complete. 2024-04-25T13:43:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.683+0000 7f37915cb740 1 -- start start 2024-04-25T13:43:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.683+0000 7f37915cb740 1 Processor -- start 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562777041b0 0x556277abbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776981c0 0x556277abcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37841aa770 con 0x5562776db2f0 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37841aa0a0 con 0x5562776981c0 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37841a72e0 con 0x5562777041b0 2024-04-25T13:43:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37915cb740 10 client.0 _ll_register_callbacks cb 0x7f3774005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58596/0 (socket says 172.21.15.94:58596) 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 -- 172.21.15.94:0/2785335075 learned_addr learned my addr 172.21.15.94:0/2785335075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37837fe640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562777041b0 0x556277abbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3788bdf640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776981c0 0x556277abcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 -- 172.21.15.94:0/2785335075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562777041b0 msgr2=0x556277abbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562777041b0 0x556277abbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 -- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776981c0 msgr2=0x556277abcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776981c0 0x556277abcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 -- 172.21.15.94:0/2785335075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556277abd220 con 0x5562776db2f0 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f3783fff640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f37740057a0 tx=0x7f37740dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6985702fc8fc70b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.684+0000 7f37637fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740a74d0 con 0x5562776db2f0 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f37740afea0 con 0x5562776db2f0 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377803b020 con 0x5562776db2f0 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37740ab880 con 0x5562776db2f0 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37915cb740 10 client.15915 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556277abe580 con 0x5562776db2f0 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377803b020 con 0x5562776db2f0 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f37740a2e40 con 0x5562776db2f0 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37915cb740 10 client.15915 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.685+0000 7f37915cb740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37915cb740 20 client.15915 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556277b2cfc0 0x556277b2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37915cb740 4 client.15915 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:43:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556277a11ee0 con 0x556277b2cfc0 2024-04-25T13:43:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37837fe640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556277b2cfc0 0x556277b2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37637fe640 10 client.15915 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.686+0000 7f37837fe640 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556277b2cfc0 0x556277b2f3e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f123f0609868a6e1 server_cookie=73d2cac0d8b11dd7 in_seq=0 out_seq=0 2024-04-25T13:43:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37637fe640 1 -- 172.21.15.94:0/2785335075 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556277a11ee0 con 0x556277b2cfc0 2024-04-25T13:43:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37637fe640 10 client.15915 handle_command_reply: tid=0 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 client.15915 shutdown 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 msgr2=0x556277abb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f37740057a0 tx=0x7f37740dd040 comp rx=0 tx=0).stop 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 shutdown_connections 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556277b2cfc0 0x556277b2f3e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562776981c0 0x556277abcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562777041b0 0x556277abbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 --2- 172.21.15.94:0/2785335075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562776db2f0 0x556277abb7f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 >> 172.21.15.94:0/2785335075 conn(0x5562776f8300 msgr2=0x556277aa31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37637fe640 0 client.15915 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 shutdown_connections 2024-04-25T13:43:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.691+0000 7f37915cb740 1 -- 172.21.15.94:0/2785335075 wait complete. 2024-04-25T13:43:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f37915cb740 20 client.15915 trim_cache size 0 max 16384 2024-04-25T13:43:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:43:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:43:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:43:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:43:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 msgr2=0x7f376c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 0x7f376c0b2090 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f377c033040 tx=0x7f377c009880 comp rx=0 tx=0).stop 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 msgr2=0x7f3784197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 0x7f3784197fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3784101a80 tx=0x7f377400ee50 comp rx=0 tx=0).stop 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 shutdown_connections 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f376c0afb80 0x7f376c0b2090 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37841080f0 0x7f378419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37841033f0 0x7f3784198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 --2- 172.21.15.94:0/2499997763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37841021b0 0x7f3784197fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.692+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 >> 172.21.15.94:0/2499997763 conn(0x7f37840fd3a0 msgr2=0x7f37841040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.693+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 shutdown_connections 2024-04-25T13:43:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:07.693+0000 7f378a642640 1 -- 172.21.15.94:0/2499997763 wait complete. 2024-04-25T13:43:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:08 smithi094 ceph-mon[29195]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:08.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:08 smithi138 ceph-mon[32977]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:08 smithi137 ceph-mon[32568]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T13:43:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:10 smithi094 ceph-mon[29195]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T13:43:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:10 smithi138 ceph-mon[32977]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T13:43:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:10 smithi137 ceph-mon[32568]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T13:43:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:12 smithi094 ceph-mon[29195]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:43:12.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:12 smithi094 ceph-mon[29195]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T13:43:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:12 smithi138 ceph-mon[32977]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:43:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:12 smithi138 ceph-mon[32977]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T13:43:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:12 smithi137 ceph-mon[32568]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-25T13:43:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:12 smithi137 ceph-mon[32568]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T13:43:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:14 smithi094 ceph-mon[29195]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:43:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:14 smithi138 ceph-mon[32977]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:43:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:14 smithi137 ceph-mon[32568]: pgmap v389: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T13:43:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:16 smithi094 ceph-mon[29195]: pgmap v390: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:43:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:16 smithi138 ceph-mon[32977]: pgmap v390: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:43:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:16 smithi137 ceph-mon[32568]: pgmap v390: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T13:43:16.658 INFO:tasks.workunit.client.0.smithi094.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: git switch -c 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr:Or undo this operation with: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: git switch - 2024-04-25T13:43:16.659 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.660 INFO:tasks.workunit.client.0.smithi094.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T13:43:16.660 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-25T13:43:16.660 INFO:tasks.workunit.client.0.smithi094.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:43:16.665 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T13:43:16.707 INFO:tasks.workunit.client.0.smithi094.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T13:43:16.709 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T13:43:16.709 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T13:43:17.425 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T13:43:17.481 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T13:43:17.521 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T13:43:17.523 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T13:43:17.523 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T13:43:17.562 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T13:43:17.566 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T13:43:17.566 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T13:43:17.621 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-25T13:43:17.622 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-25T13:43:17.623 DEBUG:teuthology.orchestra.run.smithi094:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-25T13:43:17.682 INFO:tasks.workunit.client.0.smithi094.stderr:+ dbench 1 2024-04-25T13:43:17.687 INFO:tasks.workunit.client.0.smithi094.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T13:43:17.688 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T13:43:17.688 INFO:tasks.workunit.client.0.smithi094.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T13:43:17.688 INFO:tasks.workunit.client.0.smithi094.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-25T13:43:17.789 INFO:tasks.workunit.client.0.smithi094.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-25T13:43:17.789 INFO:tasks.workunit.client.0.smithi094.stdout:releasing clients 2024-04-25T13:43:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:18 smithi094 ceph-mon[29195]: pgmap v391: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T13:43:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:18 smithi138 ceph-mon[32977]: pgmap v391: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T13:43:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:18 smithi137 ceph-mon[32568]: pgmap v391: 97 pgs: 97 active+clean; 670 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T13:43:18.788 INFO:tasks.workunit.client.0.smithi094.stdout: 1 3780 39.76 MB/sec warmup 1 sec latency 6.442 ms 2024-04-25T13:43:19.788 INFO:tasks.workunit.client.0.smithi094.stdout: 1 7173 28.86 MB/sec warmup 2 sec latency 16.731 ms 2024-04-25T13:43:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:20 smithi094 ceph-mon[29195]: pgmap v392: 97 pgs: 97 active+clean; 827 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-25T13:43:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:20 smithi138 ceph-mon[32977]: pgmap v392: 97 pgs: 97 active+clean; 827 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-25T13:43:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:20 smithi137 ceph-mon[32568]: pgmap v392: 97 pgs: 97 active+clean; 827 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-25T13:43:20.788 INFO:tasks.workunit.client.0.smithi094.stdout: 1 10111 23.92 MB/sec warmup 3 sec latency 20.194 ms 2024-04-25T13:43:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 -- 172.21.15.94:0/3949119629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138106f70 msgr2=0x7ff138109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/3949119629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138106f70 0x7ff138109390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff128009900 tx=0x7ff12802a410 comp rx=0 tx=0).stop 2024-04-25T13:43:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 -- 172.21.15.94:0/3949119629 shutdown_connections 2024-04-25T13:43:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/3949119629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1381098d0 0x7ff13810bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/3949119629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138106f70 0x7ff138109390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/3949119629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff138104610 0x7ff138106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 -- 172.21.15.94:0/3949119629 >> 172.21.15.94:0/3949119629 conn(0x7ff1380fdc10 msgr2=0x7ff1381000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 -- 172.21.15.94:0/3949119629 shutdown_connections 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.810+0000 7ff13f6b6640 1 -- 172.21.15.94:0/3949119629 wait complete. 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 Processor -- start 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 -- start start 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138106f70 0x7ff13819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1381098d0 0x7ff1381a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810f650 con 0x7ff138104610 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810f4d0 con 0x7ff138106f70 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13f6b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810f7d0 con 0x7ff1381098d0 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57344/0 (socket says 172.21.15.94:57344) 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 -- 172.21.15.94:0/2985988230 learned_addr learned my addr 172.21.15.94:0/2985988230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13cc51640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138106f70 0x7ff13819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13dc53640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1381098d0 0x7ff1381a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 -- 172.21.15.94:0/2985988230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1381098d0 msgr2=0x7ff1381a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1381098d0 0x7ff1381a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 -- 172.21.15.94:0/2985988230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138106f70 msgr2=0x7ff13819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138106f70 0x7ff13819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 -- 172.21.15.94:0/2985988230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a1dc0 con 0x7ff138104610 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.811+0000 7ff13d452640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.812+0000 7ff12e7fc640 1 -- 172.21.15.94:0/2985988230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13400ccc0 con 0x7ff138104610 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.812+0000 7ff12e7fc640 1 -- 172.21.15.94:0/2985988230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340054d0 con 0x7ff138104610 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.812+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a20b0 con 0x7ff138104610 2024-04-25T13:43:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.812+0000 7ff12e7fc640 1 -- 172.21.15.94:0/2985988230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134010430 con 0x7ff138104610 2024-04-25T13:43:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.812+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a2600 con 0x7ff138104610 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.814+0000 7ff146569740 1 Processor -- start 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.814+0000 7ff12e7fc640 1 -- 172.21.15.94:0/2985988230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff1340105d0 con 0x7ff138104610 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.814+0000 7ff146569740 1 -- start start 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788adcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e788a3e460 con 0x55e78870f2f0 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e7889f8590 con 0x55e7887381b0 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff146569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e788a3a420 con 0x55e7887174d0 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13dc53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788adcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57346/0 (socket says 172.21.15.94:57346) 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13cc51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 -- 172.21.15.94:0/2906617878 learned_addr learned my addr 172.21.15.94:0/2906617878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff12e7fc640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 0x7ff1240b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 -- 172.21.15.94:0/2906617878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 msgr2=0x55e788ae4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 -- 172.21.15.94:0/2906617878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 msgr2=0x55e788adcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788adcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 -- 172.21.15.94:0/2906617878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e788a050d0 con 0x55e78870f2f0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13cc51640 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13dc53640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 0x7ff1240b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff12e7fc640 1 -- 172.21.15.94:0/2985988230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff1340df040 con 0x7ff138104610 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.815+0000 7ff13d452640 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff13408c6c0 tx=0x7ff13406d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a81da329caf31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.816+0000 7ff1237fe640 1 -- 172.21.15.94:0/2906617878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340a7250 con 0x55e78870f2f0 2024-04-25T13:43:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.816+0000 7ff13dc53640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 0x7ff1240b2070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e788addf50 tx=0x7ff130004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.816+0000 7ff1237fe640 1 -- 172.21.15.94:0/2906617878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.816+0000 7ff1237fe640 1 -- 172.21.15.94:0/2906617878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340a6bf0 con 0x55e78870f2f0 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- 172.21.15.94:0/2906617878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 msgr2=0x55e788add300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff13408c6c0 tx=0x7ff13406d8f0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- 172.21.15.94:0/2906617878 shutdown_connections 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788add300 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- 172.21.15.94:0/2906617878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788adcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- 172.21.15.94:0/2906617878 >> 172.21.15.94:0/2906617878 conn(0x55e78872c300 msgr2=0x55e788ad4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- 172.21.15.94:0/2906617878 shutdown_connections 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- 172.21.15.94:0/2906617878 wait complete. 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- start start 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 Processor -- start 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788aede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e788ae4bf0 con 0x55e78870f2f0 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e7889c5b60 con 0x55e7887381b0 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.817+0000 7ff146569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e788ae4d70 con 0x55e7887174d0 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff146569740 10 client.0 _ll_register_callbacks cb 0x55e7889ea2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57352/0 (socket says 172.21.15.94:57352) 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 -- 172.21.15.94:0/2393045462 learned_addr learned my addr 172.21.15.94:0/2393045462 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13dc53640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788aede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13cc51640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 -- 172.21.15.94:0/2393045462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 msgr2=0x55e788ae6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 -- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 msgr2=0x55e788aede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788aede30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 -- 172.21.15.94:0/2393045462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e788750210 con 0x55e78870f2f0 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff13d452640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1340dcaf0 tx=0x7ff1340096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce0d48225bf0428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340a6dd0 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff134076070 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1340a7250 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff146569740 10 client.15927 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.818+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e78881c620 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff13406d4d0 con 0x55e78870f2f0 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 10 client.15927 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 20 client.15927 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886f6d80 0x55e788af4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 4 client.15927 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e788a4c070 con 0x55e7886f6d80 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.819+0000 7ff13cc51640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886f6d80 0x55e788af4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff120ff9640 10 client.15927 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff13cc51640 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886f6d80 0x55e788af4280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49786a63db7eca8 server_cookie=4c872346461e5954 in_seq=0 out_seq=0 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff120ff9640 1 -- 172.21.15.94:0/2393045462 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e788a4c070 con 0x55e7886f6d80 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff120ff9640 10 client.15927 handle_command_reply: tid=0 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff146569740 1 client.15927 shutdown 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 msgr2=0x55e788ae6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.820+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1340dcaf0 tx=0x7ff1340096c0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 shutdown_connections 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886f6d80 0x55e788af4280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788aede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff120ff9640 0 client.15927 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788ae6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 --2- 172.21.15.94:0/2393045462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ae6990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 >> 172.21.15.94:0/2393045462 conn(0x55e78872c300 msgr2=0x55e788ad7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 shutdown_connections 2024-04-25T13:43:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 1 -- 172.21.15.94:0/2393045462 wait complete. 2024-04-25T13:43:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.821+0000 7ff146569740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 Processor -- start 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 -- start start 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 0x55e788aeddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 0x55e788ad3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381a9390 con 0x55e78870f2f0 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381aaed0 con 0x55e7887174d0 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff146569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381a9a60 con 0x55e7887381b0 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57364/0 (socket says 172.21.15.94:57364) 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 -- 172.21.15.94:0/64883212 learned_addr learned my addr 172.21.15.94:0/64883212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13cc51640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 0x55e788aeddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13dc53640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 0x55e788ad3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 -- 172.21.15.94:0/64883212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 msgr2=0x55e788ad3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 0x55e788ad3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 -- 172.21.15.94:0/64883212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 msgr2=0x55e788aeddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 0x55e788aeddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13d452640 1 -- 172.21.15.94:0/64883212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e788ad7d50 con 0x55e78870f2f0 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13dc53640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 0x55e788ad3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.836+0000 7ff13cc51640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 0x55e788aeddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff13d452640 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a3500 tx=0x7ff134005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df22ddead7a6dc4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff122ffd640 1 -- 172.21.15.94:0/64883212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340a6dd0 con 0x55e78870f2f0 2024-04-25T13:43:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff122ffd640 1 -- 172.21.15.94:0/64883212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff122ffd640 1 -- 172.21.15.94:0/64883212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340b02e0 con 0x55e78870f2f0 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff146569740 1 -- 172.21.15.94:0/64883212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 msgr2=0x55e788ad7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.837+0000 7ff146569740 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a3500 tx=0x7ff134005e30 comp rx=0 tx=0).stop 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- 172.21.15.94:0/64883212 shutdown_connections 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887381b0 0x55e788ad3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887174d0 0x55e788aeddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- 172.21.15.94:0/64883212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788ad7810 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- 172.21.15.94:0/64883212 >> 172.21.15.94:0/64883212 conn(0x55e78872c300 msgr2=0x55e788ad7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- 172.21.15.94:0/64883212 shutdown_connections 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- 172.21.15.94:0/64883212 wait complete. 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- start start 2024-04-25T13:43:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 Processor -- start 2024-04-25T13:43:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788aeb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788b5a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381aa800 con 0x55e78870f2f0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381aa130 con 0x55e7887381b0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381a7370 con 0x55e7887174d0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.838+0000 7ff146569740 10 client.0 _ll_register_callbacks cb 0x7ff1300099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57380/0 (socket says 172.21.15.94:57380) 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 -- 172.21.15.94:0/3859238586 learned_addr learned my addr 172.21.15.94:0/3859238586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13dc53640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788b5a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13cc51640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788aeb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 -- 172.21.15.94:0/3859238586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 msgr2=0x55e788aeb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788aeb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 -- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 msgr2=0x55e788b5a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788b5a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 -- 172.21.15.94:0/3859238586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e788aee130 con 0x55e78870f2f0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13cc51640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788aeb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff13d452640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a7640 tx=0x7ff1340a3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b7570c168e0e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340a6dd0 con 0x55e78870f2f0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff1340afe50 con 0x55e78870f2f0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1340b02e0 con 0x55e78870f2f0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff146569740 10 client.15933 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.839+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e788aee130 con 0x55e78870f2f0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1340a6f90 con 0x55e78870f2f0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff134005000 con 0x55e78870f2f0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 10 client.15933 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 20 client.15933 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886cc1c0 0x55e788b5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 4 client.15933 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e788a45ee0 con 0x55e7886cc1c0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.840+0000 7ff13cc51640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886cc1c0 0x55e788b5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.841+0000 7ff120ff9640 10 client.15933 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.841+0000 7ff13cc51640 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886cc1c0 0x55e788b5e140 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b157ba33282e99 server_cookie=dc8c73031f777c3b in_seq=0 out_seq=0 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff120ff9640 1 -- 172.21.15.94:0/3859238586 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e788a45ee0 con 0x55e7886cc1c0 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff120ff9640 10 client.15933 handle_command_reply: tid=0 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 client.15933 shutdown 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 msgr2=0x55e788aeacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff1340a7640 tx=0x7ff1340a3100 comp rx=0 tx=0).stop 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 shutdown_connections 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e7886cc1c0 0x55e788b5e140 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7887381b0 0x55e788b5a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff120ff9640 0 client.15933 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7887174d0 0x55e788aeb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 --2- 172.21.15.94:0/3859238586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e78870f2f0 0x55e788aeacf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.915+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 >> 172.21.15.94:0/3859238586 conn(0x55e78872c300 msgr2=0x55e788ae6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 shutdown_connections 2024-04-25T13:43:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff146569740 1 -- 172.21.15.94:0/3859238586 wait complete. 2024-04-25T13:43:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff146569740 20 client.15933 trim_cache size 0 max 16384 2024-04-25T13:43:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:43:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:43:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:43:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 msgr2=0x7ff1240b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 0x7ff1240b2070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e788addf50 tx=0x7ff130004a40 comp rx=0 tx=0).stop 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 msgr2=0x7ff13819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 shutdown_connections 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1240afb60 0x7ff1240b2070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1381098d0 0x7ff1381a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138106f70 0x7ff13819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 --2- 172.21.15.94:0/2985988230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff138104610 0x7ff13819a280 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 >> 172.21.15.94:0/2985988230 conn(0x7ff1380fdc10 msgr2=0x7ff1381051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 shutdown_connections 2024-04-25T13:43:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:20.916+0000 7ff13f6b6640 1 -- 172.21.15.94:0/2985988230 wait complete. 2024-04-25T13:43:21.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 12856 21.03 MB/sec warmup 4 sec latency 39.453 ms 2024-04-25T13:43:22.444 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:22 smithi137 ceph-mon[32568]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-04-25T13:43:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:22 smithi094 ceph-mon[29195]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-04-25T13:43:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:22 smithi138 ceph-mon[32977]: pgmap v393: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-04-25T13:43:22.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 15688 20.51 MB/sec warmup 5 sec latency 15.281 ms 2024-04-25T13:43:23.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 18687 19.56 MB/sec warmup 6 sec latency 19.279 ms 2024-04-25T13:43:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:24 smithi094 ceph-mon[29195]: pgmap v394: 97 pgs: 97 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 369 KiB/s wr, 59 op/s 2024-04-25T13:43:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:24 smithi138 ceph-mon[32977]: pgmap v394: 97 pgs: 97 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 369 KiB/s wr, 59 op/s 2024-04-25T13:43:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:24 smithi137 ceph-mon[32568]: pgmap v394: 97 pgs: 97 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 369 KiB/s wr, 59 op/s 2024-04-25T13:43:24.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:43:24.749 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:43:24.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 21989 19.41 MB/sec warmup 7 sec latency 10.737 ms 2024-04-25T13:43:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 -- 172.21.15.94:0/4264684319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 msgr2=0x7f9b2010bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 --2- 172.21.15.94:0/4264684319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2010bca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009ca0 tx=0x7f9b1802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 -- 172.21.15.94:0/4264684319 shutdown_connections 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 --2- 172.21.15.94:0/4264684319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2010bca0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 --2- 172.21.15.94:0/4264684319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20101740 0x7f9b20101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 --2- 172.21.15.94:0/4264684319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20105090 0x7f9b201054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 -- 172.21.15.94:0/4264684319 >> 172.21.15.94:0/4264684319 conn(0x7f9b200fd340 msgr2=0x7f9b200ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 -- 172.21.15.94:0/4264684319 shutdown_connections 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 -- 172.21.15.94:0/4264684319 wait complete. 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.865+0000 7f9b2577a640 1 Processor -- start 2024-04-25T13:43:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 -- start start 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20101740 0x7f9b2019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20105090 0x7f9b201a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2010f620 con 0x7f9b20102240 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2010f4a0 con 0x7f9b20105090 2024-04-25T13:43:24.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b2577a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2010f7a0 con 0x7f9b20101740 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57098/0 (socket says 172.21.15.94:57098) 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3497957968 learned_addr learned my addr 172.21.15.94:0/3497957968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1effd640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20101740 0x7f9b2019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20105090 0x7f9b201a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3497957968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20101740 msgr2=0x7f9b2019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20101740 0x7f9b2019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3497957968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20105090 msgr2=0x7f9b201a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20105090 0x7f9b201a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.866+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3497957968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b201a1d00 con 0x7f9b20102240 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400bf20 tx=0x7f9b1400dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b07fff640 1 -- 172.21.15.94:0/3497957968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14018070 con 0x7f9b20102240 2024-04-25T13:43:24.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b20102820 con 0x7f9b20102240 2024-04-25T13:43:24.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b07fff640 1 -- 172.21.15.94:0/3497957968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b14009c30 con 0x7f9b20102240 2024-04-25T13:43:24.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b07fff640 1 -- 172.21.15.94:0/3497957968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1401b910 con 0x7f9b20102240 2024-04-25T13:43:24.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.867+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b20102dc0 con 0x7f9b20102240 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b07fff640 1 -- 172.21.15.94:0/3497957968 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9b1401bab0 con 0x7f9b20102240 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 Processor -- start 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 -- start start 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 0x55b943ddea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 0x55b943ddefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b943d43d80 con 0x55b943a1c6b0 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b943cfd8b0 con 0x55b943a144d0 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b2c621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b943d3fd40 con 0x55b943a3d390 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b1f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 0x55b943ddea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 0x55b943ddefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57104/0 (socket says 172.21.15.94:57104) 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/2042710245 learned_addr learned my addr 172.21.15.94:0/2042710245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.870+0000 7f9b07fff640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 0x7f9b080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/2042710245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 msgr2=0x55b943ddea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 0x55b943ddea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/2042710245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 msgr2=0x55b943ddefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 0x55b943ddefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/2042710245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b943ccea40 con 0x55b943a1c6b0 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b07fff640 1 -- 172.21.15.94:0/3497957968 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9b140e2a80 con 0x7f9b20102240 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 0x55b943ddea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9b1408d6b0 tx=0x7f9b140713e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8a03b8d5f165b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 0x7f9b080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b1effd640 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 0x55b943ddefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b057fa640 1 -- 172.21.15.94:0/2042710245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140aac90 con 0x55b943a1c6b0 2024-04-25T13:43:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b057fa640 1 -- 172.21.15.94:0/2042710245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b140ec040 con 0x55b943a1c6b0 2024-04-25T13:43:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.871+0000 7f9b057fa640 1 -- 172.21.15.94:0/2042710245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140754f0 con 0x55b943a1c6b0 2024-04-25T13:43:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 -- 172.21.15.94:0/2042710245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 msgr2=0x55b943ddfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 0x7f9b080b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b943ddfab0 tx=0x7f9b180095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9b1408d6b0 tx=0x7f9b140713e0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 -- 172.21.15.94:0/2042710245 shutdown_connections 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a1c6b0 0x55b943ddfc70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a144d0 0x55b943ddefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2042710245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a3d390 0x55b943ddea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.872+0000 7f9b2c621740 1 -- 172.21.15.94:0/2042710245 >> 172.21.15.94:0/2042710245 conn(0x55b943a314e0 msgr2=0x55b943dda270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- 172.21.15.94:0/2042710245 shutdown_connections 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- 172.21.15.94:0/2042710245 wait complete. 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- start start 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 Processor -- start 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943de8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a3d390 0x55b943defda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b943de0720 con 0x55b943a144d0 2024-04-25T13:43:24.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b943ccad10 con 0x55b943a3d390 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b943de08a0 con 0x55b943a1c6b0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b2c621740 10 client.0 _ll_register_callbacks cb 0x55b943cf7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57118/0 (socket says 172.21.15.94:57118) 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b1effd640 1 -- 172.21.15.94:0/2771732084 learned_addr learned my addr 172.21.15.94:0/2771732084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943de8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.873+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a3d390 0x55b943defda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 -- 172.21.15.94:0/2771732084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 msgr2=0x55b943de8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943de8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 -- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a3d390 msgr2=0x55b943defda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a3d390 0x55b943defda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 -- 172.21.15.94:0/2771732084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b943b1f7d0 con 0x55b943a144d0 2024-04-25T13:43:24.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b1effd640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000bf20 tx=0x7f9b100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6de6bb68da08ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10005020 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9b100051c0 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b140ec040 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b100053c0 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b2c621740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b943a553f0 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.874+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b140ec040 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b100108e0 con 0x55b943a144d0 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 20 client.15945 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:24.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9439fbf60 0x55b943df6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 4 client.15945 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b943d51990 con 0x55b9439fbf60 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9439fbf60 0x55b943df6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b02ff5640 10 client.15945 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.875+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9439fbf60 0x55b943df6300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23044810d01ded7c server_cookie=2210891a4ef095c1 in_seq=0 out_seq=0 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b02ff5640 1 -- 172.21.15.94:0/2771732084 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b943d51990 con 0x55b9439fbf60 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b02ff5640 10 client.15945 handle_command_reply: tid=0 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 client.15945 shutdown 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 msgr2=0x55b943de8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000bf20 tx=0x7f9b100118d0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 shutdown_connections 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9439fbf60 0x55b943df6300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a3d390 0x55b943defda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943de8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b02ff5640 0 client.15945 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 --2- 172.21.15.94:0/2771732084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943de8900 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 >> 172.21.15.94:0/2771732084 conn(0x55b943a314e0 msgr2=0x55b943dd93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 shutdown_connections 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.877+0000 7f9b2c621740 1 -- 172.21.15.94:0/2771732084 wait complete. 2024-04-25T13:43:24.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.878+0000 7f9b2c621740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 Processor -- start 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 -- start start 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9439fbf60 0x55b943de8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943dfaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201a93e0 con 0x55b943a144d0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201aaf20 con 0x55b9439fbf60 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.895+0000 7f9b2c621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201a9ab0 con 0x55b943a1c6b0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57126/0 (socket says 172.21.15.94:57126) 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3941704028 learned_addr learned my addr 172.21.15.94:0/3941704028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943dfaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1effd640 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9439fbf60 0x55b943de8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3941704028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 msgr2=0x55b943dfaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943dfaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3941704028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9439fbf60 msgr2=0x55b943de8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9439fbf60 0x55b943de8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/3941704028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b943a150f0 con 0x55b943a144d0 2024-04-25T13:43:24.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9b140aabc0 tx=0x7f9b14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cb047871fb66dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b04ff9640 1 -- 172.21.15.94:0/3941704028 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140cd040 con 0x55b943a144d0 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b04ff9640 1 -- 172.21.15.94:0/3941704028 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b140e9070 con 0x55b943a144d0 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.896+0000 7f9b04ff9640 1 -- 172.21.15.94:0/3941704028 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14076020 con 0x55b943a144d0 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- 172.21.15.94:0/3941704028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 msgr2=0x55b943defcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9b140aabc0 tx=0x7f9b14000bc0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- 172.21.15.94:0/3941704028 shutdown_connections 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b943a1c6b0 0x55b943dfaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943defcc0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 --2- 172.21.15.94:0/3941704028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9439fbf60 0x55b943de8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- 172.21.15.94:0/3941704028 >> 172.21.15.94:0/3941704028 conn(0x55b943a314e0 msgr2=0x55b943de84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- 172.21.15.94:0/3941704028 shutdown_connections 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- 172.21.15.94:0/3941704028 wait complete. 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 -- start start 2024-04-25T13:43:24.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.897+0000 7f9b2c621740 1 Processor -- start 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9439fbf60 0x55b943decc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a1c6b0 0x55b943e5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201aa850 con 0x55b943a144d0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201aa180 con 0x55b943a1c6b0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b201a73c0 con 0x55b9439fbf60 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 10 client.0 _ll_register_callbacks cb 0x7f9b1000b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57132/0 (socket says 172.21.15.94:57132) 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/1113141818 learned_addr learned my addr 172.21.15.94:0/1113141818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1effd640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9439fbf60 0x55b943decc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1f7fe640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a1c6b0 0x55b943e5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/1113141818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9439fbf60 msgr2=0x55b943decc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9439fbf60 0x55b943decc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a1c6b0 msgr2=0x55b943e5c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a1c6b0 0x55b943e5c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 -- 172.21.15.94:0/1113141818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b943d044b0 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b1e7fc640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b140a8130 tx=0x7f9b140ab030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defa3998ffd1666f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140cd070 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9b140a7150 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.898+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b140e9070 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b14076680 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 10 client.15951 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b943e5db60 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b140e9070 con 0x55b943a144d0 2024-04-25T13:43:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b140720b0 con 0x55b943a144d0 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 10 client.15951 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 20 client.15951 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b943a3d390 0x55b943e60190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 4 client.15951 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b943ddf3e0 con 0x55b943a3d390 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b1effd640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b943a3d390 0x55b943e60190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b1effd640 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b943a3d390 0x55b943e60190 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741443c4ec8b2542 server_cookie=b82ca45615024cf7 in_seq=0 out_seq=0 2024-04-25T13:43:24.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.899+0000 7f9b02ff5640 10 client.15951 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b02ff5640 1 -- 172.21.15.94:0/1113141818 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b943ddf3e0 con 0x55b943a3d390 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b02ff5640 10 client.15951 handle_command_reply: tid=0 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 client.15951 shutdown 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 msgr2=0x55b943ded1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b140a8130 tx=0x7f9b140ab030 comp rx=0 tx=0).stop 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 shutdown_connections 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b943a3d390 0x55b943e60190 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b943a1c6b0 0x55b943e5c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b02ff5640 0 client.15951 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b943a144d0 0x55b943ded1d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 --2- 172.21.15.94:0/1113141818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9439fbf60 0x55b943decc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 >> 172.21.15.94:0/1113141818 conn(0x55b943a314e0 msgr2=0x55b943de8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 shutdown_connections 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 1 -- 172.21.15.94:0/1113141818 wait complete. 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2c621740 20 client.15951 trim_cache size 0 max 16384 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:43:24.909 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 msgr2=0x7f9b080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 0x7f9b080b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b943ddfab0 tx=0x7f9b180095c0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 msgr2=0x7f9b2019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400bf20 tx=0x7f9b1400dd60 comp rx=0 tx=0).stop 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 shutdown_connections 2024-04-25T13:43:24.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b080afaf0 0x7f9b080b2000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b20105090 0x7f9b201a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b20102240 0x7f9b2019a700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 --2- 172.21.15.94:0/3497957968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b20101740 0x7f9b2019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.902+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 >> 172.21.15.94:0/3497957968 conn(0x7f9b200fd340 msgr2=0x7f9b20107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.903+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 shutdown_connections 2024-04-25T13:43:24.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:24.903+0000 7f9b2577a640 1 -- 172.21.15.94:0/3497957968 wait complete. 2024-04-25T13:43:24.913 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:43:24.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:25.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 -- 172.21.15.94:0/4267730480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070103c00 msgr2=0x7fa070104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 --2- 172.21.15.94:0/4267730480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070103c00 0x7fa070104010 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa060009900 tx=0x7fa06002a410 comp rx=0 tx=0).stop 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 -- 172.21.15.94:0/4267730480 shutdown_connections 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 --2- 172.21.15.94:0/4267730480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa070108900 0x7fa07010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 --2- 172.21.15.94:0/4267730480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070103c00 0x7fa070104010 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 --2- 172.21.15.94:0/4267730480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0701029c0 0x7fa0701036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 -- 172.21.15.94:0/4267730480 >> 172.21.15.94:0/4267730480 conn(0x7fa0700fdbd0 msgr2=0x7fa070100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 -- 172.21.15.94:0/4267730480 shutdown_connections 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.064+0000 7fa077c96640 1 -- 172.21.15.94:0/4267730480 wait complete. 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 Processor -- start 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 -- start start 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0701029c0 0x7fa07019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa070103c00 0x7fa07019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa07010e4e0 con 0x7fa070108900 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa07010e360 con 0x7fa0701029c0 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa077c96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa07010e660 con 0x7fa070103c00 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57150/0 (socket says 172.21.15.94:57150) 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 -- 172.21.15.94:0/2250313715 learned_addr learned my addr 172.21.15.94:0/2250313715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa075a32640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0701029c0 0x7fa07019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa075231640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa070103c00 0x7fa07019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 -- 172.21.15.94:0/2250313715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa070103c00 msgr2=0x7fa07019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa070103c00 0x7fa07019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 -- 172.21.15.94:0/2250313715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0701029c0 msgr2=0x7fa07019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0701029c0 0x7fa07019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 -- 172.21.15.94:0/2250313715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0701a1da0 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa076233640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efd0 tx=0x7fa06800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.065+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068003b70 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.066+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0701a2090 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.066+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa068003d10 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.066+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680108a0 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.066+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701a2590 con 0x7fa070108900 2024-04-25T13:43:25.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.067+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0701a2850 con 0x7fa070108900 2024-04-25T13:43:25.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.068+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa068004b40 con 0x7fa070108900 2024-04-25T13:43:25.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.069+0000 7fa066ffd640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 0x7fa05c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.069+0000 7fa075a32640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 0x7fa05c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.069+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa0680df4e0 con 0x7fa070108900 2024-04-25T13:43:25.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.069+0000 7fa075a32640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 0x7fa05c0b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0701035c0 tx=0x7fa06c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.073+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0680a8000 con 0x7fa070108900 2024-04-25T13:43:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.308+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0701a2a20 con 0x7fa070108900 2024-04-25T13:43:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.308+0000 7fa066ffd640 1 -- 172.21.15.94:0/2250313715 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fa0680a7900 con 0x7fa070108900 2024-04-25T13:43:25.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:43:25.309 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:43:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 msgr2=0x7fa05c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 0x7fa05c0b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0701035c0 tx=0x7fa06c005f90 comp rx=0 tx=0).stop 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 msgr2=0x7fa0701a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efd0 tx=0x7fa06800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 shutdown_connections 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.310+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa05c0afaf0 0x7fa05c0b2000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa070108900 0x7fa0701a1730 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa070103c00 0x7fa07019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 --2- 172.21.15.94:0/2250313715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0701029c0 0x7fa07019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 >> 172.21.15.94:0/2250313715 conn(0x7fa0700fdbd0 msgr2=0x7fa070109e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 shutdown_connections 2024-04-25T13:43:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.311+0000 7fa077c96640 1 -- 172.21.15.94:0/2250313715 wait complete. 2024-04-25T13:43:25.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:43:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 -- 172.21.15.94:0/274622978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2980760a0 msgr2=0x7fb2980764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 --2- 172.21.15.94:0/274622978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2980760a0 0x7fb2980764b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb288009900 tx=0x7fb28802a410 comp rx=0 tx=0).stop 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 -- 172.21.15.94:0/274622978 shutdown_connections 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 --2- 172.21.15.94:0/274622978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29810ba70 0x7fb29810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 --2- 172.21.15.94:0/274622978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 0x7fb29810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 --2- 172.21.15.94:0/274622978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2980760a0 0x7fb2980764b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 -- 172.21.15.94:0/274622978 >> 172.21.15.94:0/274622978 conn(0x7fb2980fed10 msgr2=0x7fb2981011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 -- 172.21.15.94:0/274622978 shutdown_connections 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.435+0000 7fb29dc13640 1 -- 172.21.15.94:0/274622978 wait complete. 2024-04-25T13:43:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 Processor -- start 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 -- start start 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 0x7fb29819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 0x7fb29819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111830 con 0x7fb29810ba70 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981116b0 con 0x7fb2980769f0 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb29dc13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981119b0 con 0x7fb2980760a0 2024-04-25T13:43:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb297fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb297fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57168/0 (socket says 172.21.15.94:57168) 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb297fff640 1 -- 172.21.15.94:0/2663347005 learned_addr learned my addr 172.21.15.94:0/2663347005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb296ffd640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 0x7fb29819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.436+0000 7fb2977fe640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 0x7fb29819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 -- 172.21.15.94:0/2663347005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 msgr2=0x7fb29819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 0x7fb29819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 -- 172.21.15.94:0/2663347005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 msgr2=0x7fb29819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 0x7fb29819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 -- 172.21.15.94:0/2663347005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981a3fe0 con 0x7fb29810ba70 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb2977fe640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 0x7fb29819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb297fff640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb40 tx=0x7fb29000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb294ff9640 1 -- 172.21.15.94:0/2663347005 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290004b20 con 0x7fb29810ba70 2024-04-25T13:43:25.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2981a42d0 con 0x7fb29810ba70 2024-04-25T13:43:25.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb294ff9640 1 -- 172.21.15.94:0/2663347005 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb290004cc0 con 0x7fb29810ba70 2024-04-25T13:43:25.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb294ff9640 1 -- 172.21.15.94:0/2663347005 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290010720 con 0x7fb29810ba70 2024-04-25T13:43:25.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.437+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a4870 con 0x7fb29810ba70 2024-04-25T13:43:25.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.439+0000 7fb2a4b9d740 1 Processor -- start 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.439+0000 7fb294ff9640 1 -- 172.21.15.94:0/2663347005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb290005560 con 0x7fb29810ba70 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.439+0000 7fb2a4b9d740 1 -- start start 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0e0b390 0x55e0e11ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e0e1111d80 con 0x55e0e0dea6b0 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0e10cba20 con 0x55e0e0de24d0 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0e110dd40 con 0x55e0e0e0b390 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb2977fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57182/0 (socket says 172.21.15.94:57182) 2024-04-25T13:43:25.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 -- 172.21.15.94:0/3391632789 learned_addr learned my addr 172.21.15.94:0/3391632789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0e0b390 0x55e0e11ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 -- 172.21.15.94:0/3391632789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0e0b390 msgr2=0x55e0e11ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0e0b390 0x55e0e11ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb294ff9640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 0x7fb2800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 -- 172.21.15.94:0/3391632789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 msgr2=0x55e0e11add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 -- 172.21.15.94:0/3391632789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e10d6c30 con 0x55e0e0dea6b0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb297fff640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 0x7fb2800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009fd0 tx=0x7fb28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a16104bf4b94f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb294ff9640 1 -- 172.21.15.94:0/2663347005 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb2900df4b0 con 0x7fb29810ba70 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb27dffb640 1 -- 172.21.15.94:0/3391632789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004030 con 0x55e0e0dea6b0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb27dffb640 1 -- 172.21.15.94:0/3391632789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c005020 con 0x55e0e0dea6b0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.440+0000 7fb27dffb640 1 -- 172.21.15.94:0/3391632789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004bc0 con 0x55e0e0dea6b0 2024-04-25T13:43:25.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3391632789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 msgr2=0x55e0e11aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb297fff640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 0x7fb2800b2000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb10 tx=0x7fb2900ca040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009fd0 tx=0x7fb28c004730 comp rx=0 tx=0).stop 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3391632789 shutdown_connections 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dea6b0 0x55e0e11aea20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3391632789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0e0b390 0x55e0e11ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3391632789 >> 172.21.15.94:0/3391632789 conn(0x55e0e0dff4e0 msgr2=0x55e0e11a9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3391632789 shutdown_connections 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.441+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3391632789 wait complete. 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 -- start start 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 Processor -- start 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0e0b390 0x55e0e11be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e0e11af530 con 0x55e0e0de24d0 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0e1098dc0 con 0x55e0e0e0b390 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0e11af6b0 con 0x55e0e0dea6b0 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2a4b9d740 10 client.0 _ll_register_callbacks cb 0x55e0e10b4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57184/0 (socket says 172.21.15.94:57184) 2024-04-25T13:43:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb2977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb296ffd640 1 -- 172.21.15.94:0/2598238313 learned_addr learned my addr 172.21.15.94:0/2598238313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.442+0000 7fb296ffd640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0e0b390 0x55e0e11be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 -- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 msgr2=0x55e0e11b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 -- 172.21.15.94:0/2598238313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0e0b390 msgr2=0x55e0e11be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0e0b390 0x55e0e11be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 -- 172.21.15.94:0/2598238313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e0e233f0 con 0x55e0e0de24d0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb296ffd640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006010 tx=0x7fb28c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b417134de0bfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004d90 con 0x55e0e0de24d0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c005020 con 0x55e0e0de24d0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb28c004050 con 0x55e0e0de24d0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb28c004250 con 0x55e0e0de24d0 2024-04-25T13:43:25.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb2a4b9d740 10 client.15969 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0e0eed7d0 con 0x55e0e0de24d0 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c005020 con 0x55e0e0de24d0 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb28c003710 con 0x55e0e0de24d0 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.443+0000 7fb2a4b9d740 10 client.15969 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2a4b9d740 10 client.15969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2a4b9d740 20 client.15969 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0dc9f60 0x55e0e11c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2a4b9d740 4 client.15969 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e111f990 con 0x55e0e0dc9f60 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2977fe640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0dc9f60 0x55e0e11c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb276ffd640 10 client.15969 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.444+0000 7fb2977fe640 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0dc9f60 0x55e0e11c4f00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f9ef5e39bc73268 server_cookie=cfeca57a33192c6 in_seq=0 out_seq=0 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb276ffd640 1 -- 172.21.15.94:0/2598238313 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0e111f990 con 0x55e0e0dc9f60 2024-04-25T13:43:25.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb276ffd640 10 client.15969 handle_command_reply: tid=0 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 client.15969 shutdown 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 msgr2=0x55e0e11b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006010 tx=0x7fb28c009cb0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 shutdown_connections 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0dc9f60 0x55e0e11c4f00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0e0b390 0x55e0e11be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.445+0000 7fb276ffd640 0 client.15969 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/2598238313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0de24d0 0x55e0e11b74d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 >> 172.21.15.94:0/2598238313 conn(0x55e0e0dff4e0 msgr2=0x55e0e11abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 shutdown_connections 2024-04-25T13:43:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/2598238313 wait complete. 2024-04-25T13:43:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.446+0000 7fb2a4b9d740 20 client.15969 trim_cache size 0 max 16384 2024-04-25T13:43:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.459+0000 7fb2a4b9d740 1 Processor -- start 2024-04-25T13:43:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.459+0000 7fb2a4b9d740 1 -- start start 2024-04-25T13:43:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981ad800 con 0x55e0e0dc9f60 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981af340 con 0x55e0e0de24d0 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981aded0 con 0x55e0e0dea6b0 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57194/0 (socket says 172.21.15.94:57194) 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 -- 172.21.15.94:0/643739917 learned_addr learned my addr 172.21.15.94:0/643739917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb2977fe640 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 -- 172.21.15.94:0/643739917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 msgr2=0x55e0e11b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 -- 172.21.15.94:0/643739917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 msgr2=0x55e0e11b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 -- 172.21.15.94:0/643739917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e11ae3f0 con 0x55e0e0dc9f60 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb296ffd640 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009d90 tx=0x7fb28c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66367626ea35cd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb27d7fa640 1 -- 172.21.15.94:0/643739917 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c005010 con 0x55e0e0dc9f60 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb27d7fa640 1 -- 172.21.15.94:0/643739917 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c0035f0 con 0x55e0e0dc9f60 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.460+0000 7fb27d7fa640 1 -- 172.21.15.94:0/643739917 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00f790 con 0x55e0e0dc9f60 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/643739917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 msgr2=0x55e0e11a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009d90 tx=0x7fb28c009d20 comp rx=0 tx=0).stop 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/643739917 shutdown_connections 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0dea6b0 0x55e0e11b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0de24d0 0x55e0e11b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/643739917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11a9160 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/643739917 >> 172.21.15.94:0/643739917 conn(0x55e0e0dff4e0 msgr2=0x55e0e11a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/643739917 shutdown_connections 2024-04-25T13:43:25.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/643739917 wait complete. 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.461+0000 7fb2a4b9d740 1 -- start start 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 Processor -- start 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0de24d0 0x55e0e11bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0dea6b0 0x55e0e11bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981aec70 con 0x55e0e0dc9f60 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981ae5a0 con 0x55e0e0dea6b0 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981ab7e0 con 0x55e0e0de24d0 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2a4b9d740 10 client.0 _ll_register_callbacks cb 0x7fb28c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57196/0 (socket says 172.21.15.94:57196) 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 -- 172.21.15.94:0/3471541197 learned_addr learned my addr 172.21.15.94:0/3471541197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb2977fe640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0de24d0 0x55e0e11bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0dea6b0 0x55e0e11bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 -- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0de24d0 msgr2=0x55e0e11bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0de24d0 0x55e0e11bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 -- 172.21.15.94:0/3471541197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0dea6b0 msgr2=0x55e0e11bc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0dea6b0 0x55e0e11bc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.462+0000 7fb296ffd640 1 -- 172.21.15.94:0/3471541197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e11b2c20 con 0x55e0e0dc9f60 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb296ffd640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0115e0 tx=0x7fb28c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d9cb87ba1e3403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0039b0 con 0x55e0e0dc9f60 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb28c004e10 con 0x55e0e0dc9f60 2024-04-25T13:43:25.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c0035f0 con 0x55e0e0dc9f60 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb28c005010 con 0x55e0e0dc9f60 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 10 client.15975 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0e11bdc90 con 0x55e0e0dc9f60 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c0035f0 con 0x55e0e0dc9f60 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb28c01f490 con 0x55e0e0dc9f60 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 10 client.15975 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 10 client.15975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:43:25.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 20 client.15975 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0e0b390 0x55e0e122f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 4 client.15975 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0e1119800 con 0x55e0e0e0b390 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.463+0000 7fb2977fe640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0e0b390 0x55e0e122f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2977fe640 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0e0b390 0x55e0e122f300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0cd4df5ea40d1c7 server_cookie=a2a66226a854c6a8 in_seq=0 out_seq=0 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb276ffd640 10 client.15975 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb276ffd640 1 -- 172.21.15.94:0/3471541197 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0e1119800 con 0x55e0e0e0b390 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb276ffd640 10 client.15975 handle_command_reply: tid=0 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 client.15975 shutdown 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 msgr2=0x55e0e11bb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0115e0 tx=0x7fb28c0097a0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 shutdown_connections 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e0e0e0b390 0x55e0e122f300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0e0dea6b0 0x55e0e11bc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0e0de24d0 0x55e0e11bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb276ffd640 0 client.15975 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 --2- 172.21.15.94:0/3471541197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0e0dc9f60 0x55e0e11bb220 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 >> 172.21.15.94:0/3471541197 conn(0x55e0e0dff4e0 msgr2=0x55e0e11abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 shutdown_connections 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 1 -- 172.21.15.94:0/3471541197 wait complete. 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.464+0000 7fb2a4b9d740 20 client.15975 trim_cache size 0 max 16384 2024-04-25T13:43:25.473 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:25.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 msgr2=0x7fb2800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 0x7fb2800b2000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb10 tx=0x7fb2900ca040 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 msgr2=0x7fb2981a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb29000eb40 tx=0x7fb29000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 shutdown_connections 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2800afaf0 0x7fb2800b2000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb29810ba70 0x7fb2981a3970 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2980769f0 0x7fb29819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 --2- 172.21.15.94:0/2663347005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2980760a0 0x7fb29819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 >> 172.21.15.94:0/2663347005 conn(0x7fb2980fed10 msgr2=0x7fb298107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 shutdown_connections 2024-04-25T13:43:25.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.465+0000 7fb29dc13640 1 -- 172.21.15.94:0/2663347005 wait complete. 2024-04-25T13:43:25.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:43:25.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 -- 172.21.15.94:0/1795234422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24103c20 msgr2=0x7efc24104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 --2- 172.21.15.94:0/1795234422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24103c20 0x7efc24104030 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc10009900 tx=0x7efc1002a410 comp rx=0 tx=0).stop 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 -- 172.21.15.94:0/1795234422 shutdown_connections 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 --2- 172.21.15.94:0/1795234422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc24108920 0x7efc2410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 --2- 172.21.15.94:0/1795234422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24103c20 0x7efc24104030 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 --2- 172.21.15.94:0/1795234422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc241029e0 0x7efc241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 -- 172.21.15.94:0/1795234422 >> 172.21.15.94:0/1795234422 conn(0x7efc240fdbd0 msgr2=0x7efc24100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 -- 172.21.15.94:0/1795234422 shutdown_connections 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.593+0000 7efc2b100640 1 -- 172.21.15.94:0/1795234422 wait complete. 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 Processor -- start 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 -- start start 2024-04-25T13:43:25.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 0x7efc2419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc24103c20 0x7efc2419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e4e0 con 0x7efc24108920 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e360 con 0x7efc24103c20 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2b100640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc2410e660 con 0x7efc241029e0 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2969d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc28e9c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 0x7efc2419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2969d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57214/0 (socket says 172.21.15.94:57214) 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc23fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc24103c20 0x7efc2419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.594+0000 7efc2969d640 1 -- 172.21.15.94:0/1126706988 learned_addr learned my addr 172.21.15.94:0/1126706988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 -- 172.21.15.94:0/1126706988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 msgr2=0x7efc2419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 0x7efc2419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 -- 172.21.15.94:0/1126706988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc24103c20 msgr2=0x7efc2419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc24103c20 0x7efc2419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 -- 172.21.15.94:0/1126706988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc241a2070 con 0x7efc24108920 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc28e9c640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 0x7efc2419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2969d640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009fd0 tx=0x7efc1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc21ffb640 1 -- 172.21.15.94:0/1126706988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c017070 con 0x7efc24108920 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc21ffb640 1 -- 172.21.15.94:0/1126706988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c003e20 con 0x7efc24108920 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241a2360 con 0x7efc24108920 2024-04-25T13:43:25.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc21ffb640 1 -- 172.21.15.94:0/1126706988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c010460 con 0x7efc24108920 2024-04-25T13:43:25.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.595+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a2860 con 0x7efc24108920 2024-04-25T13:43:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.597+0000 7efc21ffb640 1 -- 172.21.15.94:0/1126706988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efc1c003970 con 0x7efc24108920 2024-04-25T13:43:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 Processor -- start 2024-04-25T13:43:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 -- start start 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559467143d80 con 0x559466e144d0 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5594670fda20 con 0x559466e1c6b0 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc31fb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55946713fd40 con 0x559466e3d390 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc21ffb640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 0x7efc140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57226/0 (socket says 172.21.15.94:57226) 2024-04-25T13:43:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 -- 172.21.15.94:0/1736369150 learned_addr learned my addr 172.21.15.94:0/1736369150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc23fff640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc2969d640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 0x7efc140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc23fff640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc21ffb640 1 -- 172.21.15.94:0/1126706988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7efc1c0de9b0 con 0x7efc24108920 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 -- 172.21.15.94:0/1736369150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 msgr2=0x5594671df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 -- 172.21.15.94:0/1736369150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 msgr2=0x5594671e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc28e9c640 1 -- 172.21.15.94:0/1736369150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559467108c30 con 0x559466e144d0 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.598+0000 7efc23fff640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.599+0000 7efc2969d640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 0x7efc140b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0dc780 tx=0x7efc1c0725f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.599+0000 7efc28e9c640 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efc180114f0 tx=0x7efc180119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240c59d33d899844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.599+0000 7efc0effd640 1 -- 172.21.15.94:0/1736369150 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18011310 con 0x559466e144d0 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.599+0000 7efc0effd640 1 -- 172.21.15.94:0/1736369150 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc180054d0 con 0x559466e144d0 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.599+0000 7efc0effd640 1 -- 172.21.15.94:0/1736369150 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1800f560 con 0x559466e144d0 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- 172.21.15.94:0/1736369150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 msgr2=0x5594671dfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efc180114f0 tx=0x7efc180119b0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- 172.21.15.94:0/1736369150 shutdown_connections 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671dfd90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 --2- 172.21.15.94:0/1736369150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- 172.21.15.94:0/1736369150 >> 172.21.15.94:0/1736369150 conn(0x559466e314e0 msgr2=0x5594671db020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- 172.21.15.94:0/1736369150 shutdown_connections 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- 172.21.15.94:0/1736369150 wait complete. 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 -- start start 2024-04-25T13:43:25.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.600+0000 7efc31fb5740 1 Processor -- start 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e144d0 0x5594671e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5594671e1530 con 0x559466e1c6b0 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5594670cadc0 con 0x559466e3d390 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594671e16b0 con 0x559466e144d0 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 10 client.0 _ll_register_callbacks cb 0x5594670e6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57240/0 (socket says 172.21.15.94:57240) 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 -- 172.21.15.94:0/3224734571 learned_addr learned my addr 172.21.15.94:0/3224734571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e144d0 0x5594671e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc23fff640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 -- 172.21.15.94:0/3224734571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e144d0 msgr2=0x5594671e9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e144d0 0x5594671e9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 -- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 msgr2=0x5594671f0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671f0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 -- 172.21.15.94:0/3224734571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559466e553f0 con 0x559466e1c6b0 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efc1801b040 tx=0x7efc180048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0015022ec9578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc03fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18029070 con 0x559466e1c6b0 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7efc18005160 con 0x559466e1c6b0 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.601+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc180054d0 con 0x559466e1c6b0 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc18035050 con 0x559466e1c6b0 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559466f1f7d0 con 0x559466e1c6b0 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc180054d0 con 0x559466e1c6b0 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efc18019430 con 0x559466e1c6b0 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 10 client.15987 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 20 client.15987 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dfbf60 0x5594671f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 4 client.15987 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559467151990 con 0x559466dfbf60 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.602+0000 7efc23fff640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dfbf60 0x5594671f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc23fff640 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dfbf60 0x5594671f6f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8353508d98767af server_cookie=8848eef2375d3f4a in_seq=0 out_seq=0 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc03fff640 10 client.15987 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc03fff640 1 -- 172.21.15.94:0/3224734571 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559467151990 con 0x559466dfbf60 2024-04-25T13:43:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc03fff640 10 client.15987 handle_command_reply: tid=0 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 client.15987 shutdown 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 msgr2=0x5594671e9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efc1801b040 tx=0x7efc180048d0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 shutdown_connections 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dfbf60 0x5594671f6f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e1c6b0 0x5594671e9a70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc03fff640 0 client.15987 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3224734571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e144d0 0x5594671e9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.603+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 >> 172.21.15.94:0/3224734571 conn(0x559466e314e0 msgr2=0x5594671ddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.604+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 shutdown_connections 2024-04-25T13:43:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.604+0000 7efc31fb5740 1 -- 172.21.15.94:0/3224734571 wait complete. 2024-04-25T13:43:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.604+0000 7efc31fb5740 20 client.15987 trim_cache size 0 max 16384 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.617+0000 7efc31fb5740 1 Processor -- start 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.617+0000 7efc31fb5740 1 -- start start 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ab5a0 con 0x559466e144d0 2024-04-25T13:43:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ad0e0 con 0x559466e1c6b0 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc31fb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc241abc70 con 0x559466e3d390 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57242/0 (socket says 172.21.15.94:57242) 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 -- 172.21.15.94:0/4156331652 learned_addr learned my addr 172.21.15.94:0/4156331652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc28e9c640 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 -- 172.21.15.94:0/4156331652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 msgr2=0x5594671e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 -- 172.21.15.94:0/4156331652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 msgr2=0x5594671f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 -- 172.21.15.94:0/4156331652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594671e4cc0 con 0x559466e144d0 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc23fff640 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efc10032fd0 tx=0x7efc10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ddbc5d317d0a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc0e7fc640 1 -- 172.21.15.94:0/4156331652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10035040 con 0x559466e144d0 2024-04-25T13:43:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc0e7fc640 1 -- 172.21.15.94:0/4156331652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100318c0 con 0x559466e144d0 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.618+0000 7efc0e7fc640 1 -- 172.21.15.94:0/4156331652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1004f820 con 0x559466e144d0 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 -- 172.21.15.94:0/4156331652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 msgr2=0x5594671db160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efc10032fd0 tx=0x7efc10000bc0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 -- 172.21.15.94:0/4156331652 shutdown_connections 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e3d390 0x5594671e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e1c6b0 0x5594671f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 --2- 172.21.15.94:0/4156331652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671db160 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 -- 172.21.15.94:0/4156331652 >> 172.21.15.94:0/4156331652 conn(0x559466e314e0 msgr2=0x5594671daa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 -- 172.21.15.94:0/4156331652 shutdown_connections 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.619+0000 7efc31fb5740 1 -- 172.21.15.94:0/4156331652 wait complete. 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 -- start start 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 Processor -- start 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e1c6b0 0x5594671edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc241aca10 con 0x559466e144d0 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc241ac340 con 0x559466e3d390 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc241a9580 con 0x559466e1c6b0 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc31fb5740 10 client.0 _ll_register_callbacks cb 0x7efc10032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57256/0 (socket says 172.21.15.94:57256) 2024-04-25T13:43:25.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 -- 172.21.15.94:0/3703968829 learned_addr learned my addr 172.21.15.94:0/3703968829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e1c6b0 0x5594671edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 -- 172.21.15.94:0/3703968829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e1c6b0 msgr2=0x5594671edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e1c6b0 0x5594671edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 -- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 msgr2=0x5594671eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.620+0000 7efc23fff640 1 -- 172.21.15.94:0/3703968829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594671f0dc0 con 0x559466e144d0 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc23fff640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc10004380 tx=0x7efc10031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4863d4f20309052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10035070 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7efc10031a70 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc100318c0 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc10031c70 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 10 client.15993 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5594671f04c0 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100318c0 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efc10003990 con 0x559466e144d0 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 10 client.15993 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 10 client.15993 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 20 client.15993 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dd13a0 0x5594672613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 4 client.15993 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.621+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55946714b800 con 0x559466dd13a0 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dd13a0 0x5594672613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc03fff640 10 client.15993 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc28e9c640 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dd13a0 0x5594672613a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56354aeb30bf899a server_cookie=537405697e4fe932 in_seq=0 out_seq=0 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc03fff640 1 -- 172.21.15.94:0/3703968829 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55946714b800 con 0x559466dd13a0 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc03fff640 10 client.15993 handle_command_reply: tid=0 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 client.15993 shutdown 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 msgr2=0x5594671eda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc10004380 tx=0x7efc10031620 comp rx=0 tx=0).stop 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 shutdown_connections 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559466dd13a0 0x5594672613a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559466e3d390 0x5594671eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559466e1c6b0 0x5594671edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 --2- 172.21.15.94:0/3703968829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559466e144d0 0x5594671eda50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc03fff640 0 client.15993 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 >> 172.21.15.94:0/3703968829 conn(0x559466e314e0 msgr2=0x5594671e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 shutdown_connections 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 1 -- 172.21.15.94:0/3703968829 wait complete. 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.622+0000 7efc31fb5740 20 client.15993 trim_cache size 0 max 16384 2024-04-25T13:43:25.642 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 msgr2=0x7efc140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 0x7efc140b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0dc780 tx=0x7efc1c0725f0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 msgr2=0x7efc241a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009fd0 tx=0x7efc1c00b260 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 shutdown_connections 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc140afaf0 0x7efc140b2000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc24108920 0x7efc241a1a00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc24103c20 0x7efc2419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 --2- 172.21.15.94:0/1126706988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc241029e0 0x7efc2419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 >> 172.21.15.94:0/1126706988 conn(0x7efc240fdbd0 msgr2=0x7efc24109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 shutdown_connections 2024-04-25T13:43:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.623+0000 7efc2b100640 1 -- 172.21.15.94:0/1126706988 wait complete. 2024-04-25T13:43:25.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- 172.21.15.94:0/450832753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344103c30 msgr2=0x7f3344104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- 172.21.15.94:0/450832753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344103c30 0x7f3344104040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3334009900 tx=0x7f333402a410 comp rx=0 tx=0).stop 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- 172.21.15.94:0/450832753 shutdown_connections 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- 172.21.15.94:0/450832753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344108930 0x7f334410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- 172.21.15.94:0/450832753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344103c30 0x7f3344104040 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- 172.21.15.94:0/450832753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f33441036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- 172.21.15.94:0/450832753 >> 172.21.15.94:0/450832753 conn(0x7f33440fdc00 msgr2=0x7f3344100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- 172.21.15.94:0/450832753 shutdown_connections 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- 172.21.15.94:0/450832753 wait complete. 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 Processor -- start 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- start start 2024-04-25T13:43:25.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f334419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 0x7f334419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f334410e510 con 0x7f3344108930 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f334410e390 con 0x7f3344103c30 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.757+0000 7f334af96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f334410e690 con 0x7f33441029f0 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3348d32640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f334419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f333bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 0x7f334419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57274/0 (socket says 172.21.15.94:57274) 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3348d32640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f334419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50304/0 (socket says 172.21.15.94:50304) 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f333bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 0x7f334419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41876/0 (socket says 172.21.15.94:41876) 2024-04-25T13:43:25.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 -- 172.21.15.94:0/2759942498 learned_addr learned my addr 172.21.15.94:0/2759942498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 -- 172.21.15.94:0/2759942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 msgr2=0x7f334419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f334419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 -- 172.21.15.94:0/2759942498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 msgr2=0x7f334419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 0x7f334419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 -- 172.21.15.94:0/2759942498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33441a1d80 con 0x7f3344108930 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3349533640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f333c00eb10 tx=0x7f333c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3339ffb640 1 -- 172.21.15.94:0/2759942498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c004b20 con 0x7f3344108930 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3339ffb640 1 -- 172.21.15.94:0/2759942498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c004cc0 con 0x7f3344108930 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33441a2070 con 0x7f3344108930 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f3339ffb640 1 -- 172.21.15.94:0/2759942498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c010670 con 0x7f3344108930 2024-04-25T13:43:25.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.758+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33441a25c0 con 0x7f3344108930 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3339ffb640 1 -- 172.21.15.94:0/2759942498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f333c005020 con 0x7f3344108930 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 Processor -- start 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 -- start start 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e196b0 0x561e431dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e43140d80 con 0x561e42e3a390 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e430faa20 con 0x561e42e114d0 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3351e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e4313cd40 con 0x561e42e196b0 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f333bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f333bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57278/0 (socket says 172.21.15.94:57278) 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f333bfff640 1 -- 172.21.15.94:0/1901099803 learned_addr learned my addr 172.21.15.94:0/1901099803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3348d32640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f333bfff640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e196b0 0x561e431dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3339ffb640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 0x7f332c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.761+0000 7f3349533640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 0x7f332c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 -- 172.21.15.94:0/1901099803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e196b0 msgr2=0x561e431dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e196b0 0x561e431dda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 -- 172.21.15.94:0/1901099803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 msgr2=0x561e431dc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 -- 172.21.15.94:0/1901099803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e43105c30 con 0x561e42e114d0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f333bfff640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3339ffb640 1 -- 172.21.15.94:0/2759942498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f333c0df200 con 0x7f3344108930 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3348d32640 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431dcd90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33441022c0 tx=0x7f3340011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abbeb6a63c9a7b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f3349533640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 0x7f332c0b2080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f333c0a7340 tx=0x7f333c06e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f332affd640 1 -- 172.21.15.94:0/1901099803 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340004230 con 0x561e42e114d0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f332affd640 1 -- 172.21.15.94:0/1901099803 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3340009e10 con 0x561e42e114d0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.762+0000 7f332affd640 1 -- 172.21.15.94:0/1901099803 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340019680 con 0x561e42e114d0 2024-04-25T13:43:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- 172.21.15.94:0/1901099803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 msgr2=0x561e431dcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431dcd90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33441022c0 tx=0x7f3340011810 comp rx=0 tx=0).stop 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- 172.21.15.94:0/1901099803 shutdown_connections 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e196b0 0x561e431dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431dcd90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1901099803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e3a390 0x561e431dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- 172.21.15.94:0/1901099803 >> 172.21.15.94:0/1901099803 conn(0x561e42e2e4e0 msgr2=0x561e431d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- 172.21.15.94:0/1901099803 shutdown_connections 2024-04-25T13:43:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- 172.21.15.94:0/1901099803 wait complete. 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 -- start start 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.763+0000 7f3351e4a740 1 Processor -- start 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e431de530 con 0x561e42e196b0 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e430c7dc0 con 0x561e42e114d0 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e431de6b0 con 0x561e42e3a390 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3351e4a740 10 client.0 _ll_register_callbacks cb 0x561e430e3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57282/0 (socket says 172.21.15.94:57282) 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 -- 172.21.15.94:0/4088622532 learned_addr learned my addr 172.21.15.94:0/4088622532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f333bfff640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f333bfff640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 -- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 msgr2=0x561e431eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 -- 172.21.15.94:0/4088622532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 msgr2=0x561e431e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 -- 172.21.15.94:0/4088622532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e42e523f0 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3348d32640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f334001b040 tx=0x7f3340009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a22634226bc152f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.764+0000 7f3323fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340028070 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f3340003800 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3340009e10 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3340034050 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e42f1c7d0 con 0x561e42e196b0 2024-04-25T13:43:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3340009e10 con 0x561e42e196b0 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3340004910 con 0x561e42e196b0 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 10 client.16005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 20 client.16005 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42df8f60 0x561e431f4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 4 client.16005 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e4314e990 con 0x561e42df8f60 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.765+0000 7f333bfff640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42df8f60 0x561e431f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.766+0000 7f3323fff640 10 client.16005 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.766+0000 7f333bfff640 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42df8f60 0x561e431f4010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c70837b4bce9306 server_cookie=e0811149f12b542a in_seq=0 out_seq=0 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.766+0000 7f3323fff640 1 -- 172.21.15.94:0/4088622532 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e4314e990 con 0x561e42df8f60 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.766+0000 7f3323fff640 10 client.16005 handle_command_reply: tid=0 2024-04-25T13:43:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 client.16005 shutdown 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 msgr2=0x561e431e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f334001b040 tx=0x7f3340009830 comp rx=0 tx=0).stop 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 shutdown_connections 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42df8f60 0x561e431f4010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431e6aa0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3323fff640 0 client.16005 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:25.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 --2- 172.21.15.94:0/4088622532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 >> 172.21.15.94:0/4088622532 conn(0x561e42e2e4e0 msgr2=0x561e431daf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 shutdown_connections 2024-04-25T13:43:25.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 1 -- 172.21.15.94:0/4088622532 wait complete. 2024-04-25T13:43:25.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.767+0000 7f3351e4a740 20 client.16005 trim_cache size 0 max 16384 2024-04-25T13:43:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 Processor -- start 2024-04-25T13:43:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 -- start start 2024-04-25T13:43:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431db530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ab4f0 con 0x561e42e196b0 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ad030 con 0x561e42e114d0 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3351e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33441abbc0 con 0x561e42e3a390 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57288/0 (socket says 172.21.15.94:57288) 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 -- 172.21.15.94:0/1147890595 learned_addr learned my addr 172.21.15.94:0/1147890595 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f333bfff640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f333bfff640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431db530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 -- 172.21.15.94:0/1147890595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 msgr2=0x561e431f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 -- 172.21.15.94:0/1147890595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 msgr2=0x561e431db530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431db530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f3348d32640 1 -- 172.21.15.94:0/1147890595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e431de410 con 0x561e42e196b0 2024-04-25T13:43:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.781+0000 7f333bfff640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f3348d32640 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3340009250 tx=0x7f33400059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8248b5b6de033b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f332a7fc640 1 -- 172.21.15.94:0/1147890595 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334003d070 con 0x561e42e196b0 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f332a7fc640 1 -- 172.21.15.94:0/1147890595 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3340004b00 con 0x561e42e196b0 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f332a7fc640 1 -- 172.21.15.94:0/1147890595 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340028040 con 0x561e42e196b0 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f3351e4a740 1 -- 172.21.15.94:0/1147890595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 msgr2=0x561e431ed920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.782+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3340009250 tx=0x7f33400059b0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- 172.21.15.94:0/1147890595 shutdown_connections 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e3a390 0x561e431f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ed920 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- 172.21.15.94:0/1147890595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e114d0 0x561e431db530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- 172.21.15.94:0/1147890595 >> 172.21.15.94:0/1147890595 conn(0x561e42e2e4e0 msgr2=0x561e431dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- 172.21.15.94:0/1147890595 shutdown_connections 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- 172.21.15.94:0/1147890595 wait complete. 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- start start 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 Processor -- start 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 0x561e431eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e3a390 0x561e4325a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ac960 con 0x561e42e196b0 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33441ac290 con 0x561e42e3a390 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33441a94d0 con 0x561e42e114d0 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3351e4a740 10 client.0 _ll_register_callbacks cb 0x7f3340005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.783+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57296/0 (socket says 172.21.15.94:57296) 2024-04-25T13:43:25.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 -- 172.21.15.94:0/3631008010 learned_addr learned my addr 172.21.15.94:0/3631008010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f333bfff640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e3a390 0x561e4325a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f333bfff640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 0x561e431eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 -- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 msgr2=0x561e431eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 0x561e431eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 -- 172.21.15.94:0/3631008010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e3a390 msgr2=0x561e4325a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e3a390 0x561e4325a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 -- 172.21.15.94:0/3631008010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e4325a8b0 con 0x561e42e196b0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f333bfff640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 0x561e431eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3348d32640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3340012fd0 tx=0x7f3340024660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65bd2deb376e04d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3323fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334003d070 con 0x561e42e196b0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f3340004850 con 0x561e42e196b0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e43148800 con 0x561e42e196b0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3340028040 con 0x561e42e196b0 2024-04-25T13:43:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3351e4a740 10 client.16011 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e4325ba10 con 0x561e42e196b0 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.784+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561e43148800 con 0x561e42e196b0 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3340004f00 con 0x561e42e196b0 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 10 client.16011 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 10 client.16011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 20 client.16011 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42dce3a0 0x561e4325e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 4 client.16011 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e430c3610 con 0x561e42dce3a0 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f333bfff640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42dce3a0 0x561e4325e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f3323fff640 10 client.16011 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.785+0000 7f333bfff640 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42dce3a0 0x561e4325e380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33007383f93ea7c5 server_cookie=ae24b49f1f0c52c7 in_seq=0 out_seq=0 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3323fff640 1 -- 172.21.15.94:0/3631008010 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e430c3610 con 0x561e42dce3a0 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3323fff640 10 client.16011 handle_command_reply: tid=0 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 client.16011 shutdown 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 msgr2=0x561e431ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3340012fd0 tx=0x7f3340024660 comp rx=0 tx=0).stop 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 shutdown_connections 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561e42dce3a0 0x561e4325e380 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e42e3a390 0x561e4325a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e42e196b0 0x561e431ecf60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3323fff640 0 client.16011 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 --2- 172.21.15.94:0/3631008010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e42e114d0 0x561e431eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 >> 172.21.15.94:0/3631008010 conn(0x561e42e2e4e0 msgr2=0x561e431dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 shutdown_connections 2024-04-25T13:43:25.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 1 -- 172.21.15.94:0/3631008010 wait complete. 2024-04-25T13:43:25.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f3351e4a740 20 client.16011 trim_cache size 0 max 16384 2024-04-25T13:43:25.800 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 msgr2=0x7f332c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 0x7f332c0b2080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f333c0a7340 tx=0x7f333c06e000 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 msgr2=0x7f33441a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f333c00eb10 tx=0x7f333c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 shutdown_connections 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f332c0afb70 0x7f332c0b2080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3344108930 0x7f33441a1710 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3344103c30 0x7f334419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.786+0000 7f334af96640 1 --2- 172.21.15.94:0/2759942498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33441029f0 0x7f334419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.787+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 >> 172.21.15.94:0/2759942498 conn(0x7f33440fdc00 msgr2=0x7f3344109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.787+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 shutdown_connections 2024-04-25T13:43:25.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.787+0000 7f334af96640 1 -- 172.21.15.94:0/2759942498 wait complete. 2024-04-25T13:43:25.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 25259 19.36 MB/sec warmup 8 sec latency 16.028 ms 2024-04-25T13:43:25.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:43:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 -- 172.21.15.94:0/2452879884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830108940 msgr2=0x7fd83010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 --2- 172.21.15.94:0/2452879884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830108940 0x7fd83010ad60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd828009ca0 tx=0x7fd82802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 -- 172.21.15.94:0/2452879884 shutdown_connections 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 --2- 172.21.15.94:0/2452879884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830108940 0x7fd83010ad60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 --2- 172.21.15.94:0/2452879884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830103c40 0x7fd830104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 --2- 172.21.15.94:0/2452879884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd830103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 -- 172.21.15.94:0/2452879884 >> 172.21.15.94:0/2452879884 conn(0x7fd8300fdbf0 msgr2=0x7fd8301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 -- 172.21.15.94:0/2452879884 shutdown_connections 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.920+0000 7fd834a51640 1 -- 172.21.15.94:0/2452879884 wait complete. 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 Processor -- start 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 -- start start 2024-04-25T13:43:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd83019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830108940 0x7fd8301a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e580 con 0x7fd830103c40 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e400 con 0x7fd830108940 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd834a51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e700 con 0x7fd830102a00 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57312/0 (socket says 172.21.15.94:57312) 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd82e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd83019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd82dd9b640 1 -- 172.21.15.94:0/1549314739 learned_addr learned my addr 172.21.15.94:0/1549314739 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.921+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830108940 0x7fd8301a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 -- 172.21.15.94:0/1549314739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 msgr2=0x7fd83019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd83019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 -- 172.21.15.94:0/1549314739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830108940 msgr2=0x7fd8301a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830108940 0x7fd8301a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 -- 172.21.15.94:0/1549314739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a1d50 con 0x7fd830103c40 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82e59c640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd83019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9f0 tx=0x7fd82400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd81f7fe640 1 -- 172.21.15.94:0/1549314739 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400cd80 con 0x7fd830103c40 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd81f7fe640 1 -- 172.21.15.94:0/1549314739 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8240108e0 con 0x7fd830103c40 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a1fe0 con 0x7fd830103c40 2024-04-25T13:43:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd81f7fe640 1 -- 172.21.15.94:0/1549314739 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824018980 con 0x7fd830103c40 2024-04-25T13:43:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.922+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a24d0 con 0x7fd830103c40 2024-04-25T13:43:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.924+0000 7fd83b9e4740 1 Processor -- start 2024-04-25T13:43:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.924+0000 7fd81f7fe640 1 -- 172.21.15.94:0/1549314739 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd824010430 con 0x7fd830103c40 2024-04-25T13:43:25.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 -- start start 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697bfed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562697b62d80 con 0x56269783b6b0 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562697b1ca20 con 0x56269785c390 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562697b5ed40 con 0x5626978334d0 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57324/0 (socket says 172.21.15.94:57324) 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41938/0 (socket says 172.21.15.94:41938) 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82dd9b640 1 -- 172.21.15.94:0/3726149700 learned_addr learned my addr 172.21.15.94:0/3726149700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82e59c640 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697bfed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 -- 172.21.15.94:0/3726149700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 msgr2=0x562697bfed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697bfed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd81f7fe640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 0x7fd8180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 -- 172.21.15.94:0/3726149700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 msgr2=0x562697bffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82ed9d640 1 -- 172.21.15.94:0/3726149700 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562697b27c30 con 0x56269785c390 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.925+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd81f7fe640 1 -- 172.21.15.94:0/1549314739 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd8240def90 con 0x7fd830103c40 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 0x7fd8180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fafe975b550f668b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd81cff9640 1 -- 172.21.15.94:0/3726149700 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828031730 con 0x56269785c390 2024-04-25T13:43:25.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd81cff9640 1 -- 172.21.15.94:0/3726149700 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd828036030 con 0x56269785c390 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd81cff9640 1 -- 172.21.15.94:0/3726149700 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82802f870 con 0x56269785c390 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.926+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 0x7fd8180b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd828005e30 tx=0x7fd82804b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- 172.21.15.94:0/3726149700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 msgr2=0x562697bfe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd828006010 tx=0x7fd82802f3a0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- 172.21.15.94:0/3726149700 shutdown_connections 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697bffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697bfed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/3726149700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697bfe810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- 172.21.15.94:0/3726149700 >> 172.21.15.94:0/3726149700 conn(0x5626978504e0 msgr2=0x562697bfa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- 172.21.15.94:0/3726149700 shutdown_connections 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- 172.21.15.94:0/3726149700 wait complete. 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 -- start start 2024-04-25T13:43:25.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.927+0000 7fd83b9e4740 1 Processor -- start 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697c083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697c0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562697c003b0 con 0x56269783b6b0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562697ae9dc0 con 0x56269785c390 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562697c00530 con 0x5626978334d0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 10 client.0 _ll_register_callbacks cb 0x562697b05600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57330/0 (socket says 172.21.15.94:57330) 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2226837242 learned_addr learned my addr 172.21.15.94:0/2226837242 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697c083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697c0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2226837242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 msgr2=0x562697c083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697c083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 msgr2=0x562697c0f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697c0f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2226837242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626978743f0 con 0x56269783b6b0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd8240a3500 tx=0x7fd824009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2af1f0d238f552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240a6dd0 con 0x56269783b6b0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd8240e10d0 con 0x56269783b6b0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd828036030 con 0x56269783b6b0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8240a6bf0 con 0x56269783b6b0 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 10 client.16023 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.928+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56269793e7d0 con 0x56269783b6b0 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd828036030 con 0x56269783b6b0 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd8240afda0 con 0x56269783b6b0 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 10 client.16023 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 20 client.16023 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269781af60 0x562697c161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 4 client.16023 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562697b70990 con 0x56269781af60 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.929+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269781af60 0x562697c161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.930+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269781af60 0x562697c161a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b84dde1c9f2dd5a0 server_cookie=c1f3b63a46b5b233 in_seq=0 out_seq=0 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.930+0000 7fd815ffb640 10 client.16023 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd815ffb640 1 -- 172.21.15.94:0/2226837242 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562697b70990 con 0x56269781af60 2024-04-25T13:43:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd815ffb640 10 client.16023 handle_command_reply: tid=0 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 client.16023 shutdown 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 msgr2=0x562697c08910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd8240a3500 tx=0x7fd824009310 comp rx=0 tx=0).stop 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 shutdown_connections 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269781af60 0x562697c161a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269785c390 0x562697c0f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56269783b6b0 0x562697c08910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd815ffb640 0 client.16023 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:25.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2226837242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626978334d0 0x562697c083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 >> 172.21.15.94:0/2226837242 conn(0x5626978504e0 msgr2=0x562697bfcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 shutdown_connections 2024-04-25T13:43:25.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2226837242 wait complete. 2024-04-25T13:43:25.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.931+0000 7fd83b9e4740 20 client.16023 trim_cache size 0 max 16384 2024-04-25T13:43:25.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 Processor -- start 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 -- start start 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269781af60 0x562697bfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 0x562697bfce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ab550 con 0x5626978334d0 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ad090 con 0x56269781af60 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301abc20 con 0x56269783b6b0 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57348/0 (socket says 172.21.15.94:57348) 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.945+0000 7fd82dd9b640 1 -- 172.21.15.94:0/822978756 learned_addr learned my addr 172.21.15.94:0/822978756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82e59c640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269781af60 0x562697bfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 0x562697bfce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 -- 172.21.15.94:0/822978756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 msgr2=0x562697bfce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 0x562697bfce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 -- 172.21.15.94:0/822978756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269781af60 msgr2=0x562697bfa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269781af60 0x562697bfa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 -- 172.21.15.94:0/822978756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562697c03ca0 con 0x5626978334d0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 0x562697bfce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd8240dca40 tx=0x7fd8240b0370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515cf6e71738edd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd817fff640 1 -- 172.21.15.94:0/822978756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824010720 con 0x5626978334d0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd817fff640 1 -- 172.21.15.94:0/822978756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8240a6f90 con 0x5626978334d0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.946+0000 7fd817fff640 1 -- 172.21.15.94:0/822978756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240aed60 con 0x5626978334d0 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- 172.21.15.94:0/822978756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 msgr2=0x562697c0f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd8240dca40 tx=0x7fd8240b0370 comp rx=0 tx=0).stop 2024-04-25T13:43:25.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- 172.21.15.94:0/822978756 shutdown_connections 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269783b6b0 0x562697bfce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0f790 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/822978756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269781af60 0x562697bfa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- 172.21.15.94:0/822978756 >> 172.21.15.94:0/822978756 conn(0x5626978504e0 msgr2=0x562697bf9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- 172.21.15.94:0/822978756 shutdown_connections 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- 172.21.15.94:0/822978756 wait complete. 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- start start 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 Processor -- start 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 0x562697c0c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269783b6b0 0x562697c0dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ac9c0 con 0x5626978334d0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ac2f0 con 0x56269783b6b0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a9530 con 0x56269781af60 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.947+0000 7fd83b9e4740 10 client.0 _ll_register_callbacks cb 0x7fd824009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57350/0 (socket says 172.21.15.94:57350) 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2811101245 learned_addr learned my addr 172.21.15.94:0/2811101245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82ed9d640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269783b6b0 0x562697c0dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 0x562697c0c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2811101245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 msgr2=0x562697c0c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 0x562697c0c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269783b6b0 msgr2=0x562697c0dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269783b6b0 0x562697c0dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 -- 172.21.15.94:0/2811101245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562697c0fcd0 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 0x562697c0c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd82dd9b640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd8240a7180 tx=0x7fd8240aee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522a01753ee4e758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240affd0 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd8240a3000 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8240a6f90 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd824010720 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd83b9e4740 10 client.16029 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562697c0f480 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.948+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8240a6f90 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd8240ac910 con 0x5626978334d0 2024-04-25T13:43:25.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 10 client.16029 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 10 client.16029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 20 client.16029 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269785c390 0x562697c80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 4 client.16029 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562697bff190 con 0x56269785c390 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269785c390 0x562697c80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd82e59c640 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269785c390 0x562697c80560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ba185c9c4287f64 server_cookie=df9de86771959a48 in_seq=0 out_seq=0 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd815ffb640 10 client.16029 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd815ffb640 1 -- 172.21.15.94:0/2811101245 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562697bff190 con 0x56269785c390 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd815ffb640 10 client.16029 handle_command_reply: tid=0 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.949+0000 7fd83b9e4740 1 client.16029 shutdown 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 msgr2=0x562697c0ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd8240a7180 tx=0x7fd8240aee30 comp rx=0 tx=0).stop 2024-04-25T13:43:25.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 shutdown_connections 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56269785c390 0x562697c80560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56269783b6b0 0x562697c0dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626978334d0 0x562697c0ce40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 --2- 172.21.15.94:0/2811101245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56269781af60 0x562697c0c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd815ffb640 0 client.16029 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 >> 172.21.15.94:0/2811101245 conn(0x5626978504e0 msgr2=0x562697bf8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 shutdown_connections 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 1 -- 172.21.15.94:0/2811101245 wait complete. 2024-04-25T13:43:25.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd83b9e4740 20 client.16029 trim_cache size 0 max 16384 2024-04-25T13:43:25.959 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:25.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 msgr2=0x7fd8180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 0x7fd8180b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd828005e30 tx=0x7fd82804b000 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 msgr2=0x7fd83019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e9f0 tx=0x7fd82400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 shutdown_connections 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd8180afaf0 0x7fd8180b2000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd830108940 0x7fd8301a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd830103c40 0x7fd83019a750 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 --2- 172.21.15.94:0/1549314739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd830102a00 0x7fd83019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 >> 172.21.15.94:0/1549314739 conn(0x7fd8300fdbf0 msgr2=0x7fd8301010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 shutdown_connections 2024-04-25T13:43:25.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:25.950+0000 7fd834a51640 1 -- 172.21.15.94:0/1549314739 wait complete. 2024-04-25T13:43:25.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:43:26.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 -- 172.21.15.94:0/1966013459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 msgr2=0x7f65b01054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 --2- 172.21.15.94:0/1966013459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01054a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009900 tx=0x7f65ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 -- 172.21.15.94:0/1966013459 shutdown_connections 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 --2- 172.21.15.94:0/1966013459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0102240 0x7f65b010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 --2- 172.21.15.94:0/1966013459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0101740 0x7f65b0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 --2- 172.21.15.94:0/1966013459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01054a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 -- 172.21.15.94:0/1966013459 >> 172.21.15.94:0/1966013459 conn(0x7f65b00fd340 msgr2=0x7f65b00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 -- 172.21.15.94:0/1966013459 shutdown_connections 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.076+0000 7f65b7584640 1 -- 172.21.15.94:0/1966013459 wait complete. 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 Processor -- start 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 -- start start 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0101740 0x7f65b019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102240 0x7f65b019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010f600 con 0x7f65b0105090 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010f480 con 0x7f65b0102240 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b7584640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010f780 con 0x7f65b0101740 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b5b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b5b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57376/0 (socket says 172.21.15.94:57376) 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b5b21640 1 -- 172.21.15.94:0/3947567784 learned_addr learned my addr 172.21.15.94:0/3947567784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b5320640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0101740 0x7f65b019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.077+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102240 0x7f65b019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 -- 172.21.15.94:0/3947567784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0101740 msgr2=0x7f65b019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0101740 0x7f65b019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 -- 172.21.15.94:0/3947567784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102240 msgr2=0x7f65b019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102240 0x7f65b019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 -- 172.21.15.94:0/3947567784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b01a1da0 con 0x7f65b0105090 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b5b21640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65a800eb10 tx=0x7f65a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65a67fc640 1 -- 172.21.15.94:0/3947567784 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004b20 con 0x7f65b0105090 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65a67fc640 1 -- 172.21.15.94:0/3947567784 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a8004cc0 con 0x7f65b0105090 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01a2090 con 0x7f65b0105090 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65a67fc640 1 -- 172.21.15.94:0/3947567784 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8010670 con 0x7f65b0105090 2024-04-25T13:43:26.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.078+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01a25e0 con 0x7f65b0105090 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.080+0000 7f65be439740 1 Processor -- start 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65a67fc640 1 -- 172.21.15.94:0/3947567784 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f65a8005020 con 0x7f65b0105090 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 -- start start 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 0x55db985acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db98510d80 con 0x55db981e96b0 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db984caa20 con 0x55db981e14d0 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65be439740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db9850cd40 con 0x55db9820a390 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b5320640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 0x55db985acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57390/0 (socket says 172.21.15.94:57390) 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 -- 172.21.15.94:0/2307268645 learned_addr learned my addr 172.21.15.94:0/2307268645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65a67fc640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 0x7f659c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 -- 172.21.15.94:0/2307268645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 msgr2=0x55db985ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 -- 172.21.15.94:0/2307268645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 msgr2=0x55db985acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 0x55db985acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b4b1f640 1 -- 172.21.15.94:0/2307268645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db984d5c30 con 0x55db981e96b0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.081+0000 7f65b5320640 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 0x55db985acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f65b5b21640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 0x7f659c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009fd0 tx=0x7f65a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e60654f1bb65ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f65a67fc640 1 -- 172.21.15.94:0/3947567784 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f65a80df210 con 0x7f65b0105090 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f659b7fe640 1 -- 172.21.15.94:0/2307268645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0004030 con 0x55db981e96b0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f659b7fe640 1 -- 172.21.15.94:0/2307268645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a0005020 con 0x55db981e96b0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.082+0000 7f659b7fe640 1 -- 172.21.15.94:0/2307268645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0004bc0 con 0x55db981e96b0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- 172.21.15.94:0/2307268645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 msgr2=0x55db985ada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65b5b21640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 0x7f659c0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65a80a75e0 tx=0x7f65a806db70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009fd0 tx=0x7f65a0004730 comp rx=0 tx=0).stop 2024-04-25T13:43:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- 172.21.15.94:0/2307268645 shutdown_connections 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985ada20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e14d0 0x55db985acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 --2- 172.21.15.94:0/2307268645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- 172.21.15.94:0/2307268645 >> 172.21.15.94:0/2307268645 conn(0x55db981fe4e0 msgr2=0x55db985a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- 172.21.15.94:0/2307268645 shutdown_connections 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- 172.21.15.94:0/2307268645 wait complete. 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 -- start start 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.083+0000 7f65be439740 1 Processor -- start 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db985b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db985bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db985ae530 con 0x55db981e96b0 2024-04-25T13:43:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db98497dc0 con 0x55db9820a390 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db985ae6b0 con 0x55db981e14d0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65be439740 10 client.0 _ll_register_callbacks cb 0x55db984b3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57402/0 (socket says 172.21.15.94:57402) 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 -- 172.21.15.94:0/1302417674 learned_addr learned my addr 172.21.15.94:0/1302417674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db985bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db985b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 -- 172.21.15.94:0/1302417674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 msgr2=0x55db985b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db985b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 -- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 msgr2=0x55db985bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db985bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 -- 172.21.15.94:0/1302417674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db982223f0 con 0x55db981e96b0 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db985b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f65b5320640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f65ac031500 tx=0x7f65ac031dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e42f277016e62af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.084+0000 7f6598ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac035040 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f65ac004040 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a0005020 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65ac004240 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 10 client.16041 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db982ec7d0 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a0005020 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65ac044020 con 0x55db981e96b0 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 10 client.16041 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 20 client.16041 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db981c8f60 0x55db985c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 4 client.16041 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.085+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9851e990 con 0x55db981c8f60 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.086+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db981c8f60 0x55db985c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.086+0000 7f6598ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.086+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db981c8f60 0x55db985c41b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7029b2a6c27ad492 server_cookie=a3c38c10e735a370 in_seq=0 out_seq=0 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f6598ff9640 1 -- 172.21.15.94:0/1302417674 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db9851e990 con 0x55db981c8f60 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f6598ff9640 10 client.16041 handle_command_reply: tid=0 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 client.16041 shutdown 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 msgr2=0x55db985b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f65ac031500 tx=0x7f65ac031dd0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 shutdown_connections 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db981c8f60 0x55db985c41b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db985bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f6598ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db985b6aa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 --2- 172.21.15.94:0/1302417674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db985b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 >> 172.21.15.94:0/1302417674 conn(0x55db981fe4e0 msgr2=0x55db985aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 shutdown_connections 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 1 -- 172.21.15.94:0/1302417674 wait complete. 2024-04-25T13:43:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.087+0000 7f65be439740 20 client.16041 trim_cache size 0 max 16384 2024-04-25T13:43:26.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.101+0000 7f65be439740 1 Processor -- start 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.101+0000 7f65be439740 1 -- start start 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e96b0 0x55db985ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01ab5c0 con 0x55db981e14d0 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01ad100 con 0x55db981e96b0 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65be439740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01abc90 con 0x55db9820a390 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57412/0 (socket says 172.21.15.94:57412) 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 -- 172.21.15.94:0/1324570577 learned_addr learned my addr 172.21.15.94:0/1324570577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b5320640 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e96b0 0x55db985ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 -- 172.21.15.94:0/1324570577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 msgr2=0x55db985b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 -- 172.21.15.94:0/1324570577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e96b0 msgr2=0x55db985ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e96b0 0x55db985ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 -- 172.21.15.94:0/1324570577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db985b1ce0 con 0x55db981e14d0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f65a00057c0 tx=0x7f65a0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faeef916d28e992e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f659affd640 1 -- 172.21.15.94:0/1324570577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0011040 con 0x55db981e14d0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f659affd640 1 -- 172.21.15.94:0/1324570577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a00111e0 con 0x55db981e14d0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.102+0000 7f659affd640 1 -- 172.21.15.94:0/1324570577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a002e400 con 0x55db981e14d0 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 -- 172.21.15.94:0/1324570577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 msgr2=0x55db985aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f65a00057c0 tx=0x7f65a0009b80 comp rx=0 tx=0).stop 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 -- 172.21.15.94:0/1324570577 shutdown_connections 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db9820a390 0x55db985b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db981e96b0 0x55db985ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 --2- 172.21.15.94:0/1324570577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e14d0 0x55db985aae20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 -- 172.21.15.94:0/1324570577 >> 172.21.15.94:0/1324570577 conn(0x55db981fe4e0 msgr2=0x55db985a7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 -- 172.21.15.94:0/1324570577 shutdown_connections 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.103+0000 7f65be439740 1 -- 172.21.15.94:0/1324570577 wait complete. 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 -- start start 2024-04-25T13:43:26.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 Processor -- start 2024-04-25T13:43:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db98627250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db9862e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01aca30 con 0x55db981e96b0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01ac360 con 0x55db9820a390 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65b01a95a0 con 0x55db981e14d0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65be439740 10 client.0 _ll_register_callbacks cb 0x7f65ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65b5320640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65b4b1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db98627250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65b5320640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57420/0 (socket says 172.21.15.94:57420) 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65b5320640 1 -- 172.21.15.94:0/199577318 learned_addr learned my addr 172.21.15.94:0/199577318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.104+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db9862e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 -- 172.21.15.94:0/199577318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 msgr2=0x55db98627250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db98627250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 -- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 msgr2=0x55db9862e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db9862e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 -- 172.21.15.94:0/199577318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9862ec40 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65b5320640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f65ac02a930 tx=0x7f65ac0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b441488b8316ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f6598ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac035070 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f65ac046700 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a00111e0 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65ac000e60 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65be439740 10 client.16047 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.105+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db9862ff40 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a00111e0 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65ac044070 con 0x55db981e96b0 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 10 client.16047 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 20 client.16047 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db9819e3a0 0x55db986329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:26.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 4 client.16047 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db98518800 con 0x55db9819e3a0 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db9819e3a0 0x55db986329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f6598ff9640 10 client.16047 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.106+0000 7f65b4b1f640 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db9819e3a0 0x55db986329b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651318dcfd3ff6ce server_cookie=c144ab21016eb146 in_seq=0 out_seq=0 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f6598ff9640 1 -- 172.21.15.94:0/199577318 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db98518800 con 0x55db9819e3a0 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f6598ff9640 10 client.16047 handle_command_reply: tid=0 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 client.16047 shutdown 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 msgr2=0x55db98627790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f65ac02a930 tx=0x7f65ac0045f0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 shutdown_connections 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db9819e3a0 0x55db986329b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db9820a390 0x55db9862e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f6598ff9640 0 client.16047 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db981e96b0 0x55db98627790 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 --2- 172.21.15.94:0/199577318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db981e14d0 0x55db98627250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 >> 172.21.15.94:0/199577318 conn(0x55db981fe4e0 msgr2=0x55db985aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 shutdown_connections 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 1 -- 172.21.15.94:0/199577318 wait complete. 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65be439740 20 client.16047 trim_cache size 0 max 16384 2024-04-25T13:43:26.114 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:26.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.107+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 msgr2=0x7f659c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 0x7f659c0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65a80a75e0 tx=0x7f65a806db70 comp rx=0 tx=0).stop 2024-04-25T13:43:26.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 msgr2=0x7f65b01a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:26.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65a800eb10 tx=0x7f65a800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 shutdown_connections 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f659c0afaf0 0x7f659c0b2000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65b0105090 0x7f65b01a1730 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102240 0x7f65b019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 --2- 172.21.15.94:0/3947567784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b0101740 0x7f65b019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 >> 172.21.15.94:0/3947567784 conn(0x7f65b00fd340 msgr2=0x7f65b0105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 shutdown_connections 2024-04-25T13:43:26.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:26.108+0000 7f65b7584640 1 -- 172.21.15.94:0/3947567784 wait complete. 2024-04-25T13:43:26.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:26 smithi094 ceph-mon[29195]: pgmap v395: 97 pgs: 97 active+clean; 7.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 704 KiB/s wr, 120 op/s 2024-04-25T13:43:26.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:26 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2250313715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:26 smithi138 ceph-mon[32977]: pgmap v395: 97 pgs: 97 active+clean; 7.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 704 KiB/s wr, 120 op/s 2024-04-25T13:43:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2250313715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:26 smithi137 ceph-mon[32568]: pgmap v395: 97 pgs: 97 active+clean; 7.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 704 KiB/s wr, 120 op/s 2024-04-25T13:43:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:26 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2250313715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:26.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 28242 18.82 MB/sec warmup 9 sec latency 15.680 ms 2024-04-25T13:43:27.120 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:43:27.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:27.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 -- 172.21.15.94:0/2015179303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c108920 msgr2=0x7f553c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 --2- 172.21.15.94:0/2015179303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c108920 0x7f553c10ad40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5534009ca0 tx=0x7f553402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 -- 172.21.15.94:0/2015179303 shutdown_connections 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 --2- 172.21.15.94:0/2015179303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c108920 0x7f553c10ad40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 --2- 172.21.15.94:0/2015179303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c103c20 0x7f553c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 --2- 172.21.15.94:0/2015179303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c1029e0 0x7f553c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 -- 172.21.15.94:0/2015179303 >> 172.21.15.94:0/2015179303 conn(0x7f553c0fdbd0 msgr2=0x7f553c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 -- 172.21.15.94:0/2015179303 shutdown_connections 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.234+0000 7f5542546640 1 -- 172.21.15.94:0/2015179303 wait complete. 2024-04-25T13:43:27.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 Processor -- start 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 -- start start 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 0x7f553c10d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 0x7f553c10be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10f610 con 0x7f553c103c20 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10f490 con 0x7f553c108920 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5542546640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10f790 con 0x7f553c1029e0 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57446/0 (socket says 172.21.15.94:57446) 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f553b7fe640 1 -- 172.21.15.94:0/4226299942 learned_addr learned my addr 172.21.15.94:0/4226299942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f553bfff640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 0x7f553c10d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.235+0000 7f5540ae3640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 0x7f553c10be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 -- 172.21.15.94:0/4226299942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 msgr2=0x7f553c10d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 0x7f553c10d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 -- 172.21.15.94:0/4226299942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 msgr2=0x7f553c10be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 0x7f553c10be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 -- 172.21.15.94:0/4226299942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c10c5c0 con 0x7f553c103c20 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553bfff640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 0x7f553c10d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f5540ae3640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 0x7f553c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f553b7fe640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f553000e9f0 tx=0x7f553000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f55397fa640 1 -- 172.21.15.94:0/4226299942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553000cd80 con 0x7f553c103c20 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f55397fa640 1 -- 172.21.15.94:0/4226299942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55300054d0 con 0x7f553c103c20 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a45c0 con 0x7f553c103c20 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f55397fa640 1 -- 172.21.15.94:0/4226299942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530010690 con 0x7f553c103c20 2024-04-25T13:43:27.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.236+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a4a30 con 0x7f553c103c20 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55397fa640 1 -- 172.21.15.94:0/4226299942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5530004e90 con 0x7f553c103c20 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 Processor -- start 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 -- start start 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 0x55cb204a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 0x55cb204a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb2040bb00 con 0x55cb20105410 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb203a7920 con 0x55cb200e4730 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f55493f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb20407ac0 con 0x55cb200dc550 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 0x55cb204a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 0x55cb204a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57456/0 (socket says 172.21.15.94:57456) 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f5540ae3640 1 -- 172.21.15.94:0/1778307046 learned_addr learned my addr 172.21.15.94:0/1778307046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f553bfff640 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 0x55cb204a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.239+0000 7f553b7fe640 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f55397fa640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 0x7f55240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 -- 172.21.15.94:0/1778307046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 msgr2=0x55cb204ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 -- 172.21.15.94:0/1778307046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 msgr2=0x55cb204a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 0x55cb204a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 -- 172.21.15.94:0/1778307046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb201e57e0 con 0x55cb200dc550 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f5540ae3640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 0x7f55240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f55397fa640 1 -- 172.21.15.94:0/4226299942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55300df130 con 0x7f553c103c20 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f553bfff640 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 0x55cb204a6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f553c1022b0 tx=0x7f552c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495ca3871c76abdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f55227fc640 1 -- 172.21.15.94:0/1778307046 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c004310 con 0x55cb200dc550 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f5540ae3640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 0x7f55240b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55cb204a7670 tx=0x7f553402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f55227fc640 1 -- 172.21.15.94:0/1778307046 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c0044b0 con 0x55cb200dc550 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.240+0000 7f55227fc640 1 -- 172.21.15.94:0/1778307046 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0035f0 con 0x55cb200dc550 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- 172.21.15.94:0/1778307046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 msgr2=0x55cb204a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 0x55cb204a6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f553c1022b0 tx=0x7f552c0118d0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- 172.21.15.94:0/1778307046 shutdown_connections 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200dc550 0x55cb204a6a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 --2- 172.21.15.94:0/1778307046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb20105410 0x55cb204a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- 172.21.15.94:0/1778307046 >> 172.21.15.94:0/1778307046 conn(0x55cb200f9560 msgr2=0x55cb204a1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- 172.21.15.94:0/1778307046 shutdown_connections 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- 172.21.15.94:0/1778307046 wait complete. 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.241+0000 7f55493f6740 1 -- start start 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 Processor -- start 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 0x55cb204b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb204ae370 con 0x55cb200dc550 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb203a55d0 con 0x55cb20105410 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb204ae4f0 con 0x55cb200e4730 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f55493f6740 10 client.0 _ll_register_callbacks cb 0x55cb20393080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 0x55cb204b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50478/0 (socket says 172.21.15.94:50478) 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 0x55cb204b76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42038/0 (socket says 172.21.15.94:42038) 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 -- 172.21.15.94:0/3648408198 learned_addr learned my addr 172.21.15.94:0/3648408198 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 -- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 msgr2=0x55cb204b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 0x55cb204b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 -- 172.21.15.94:0/3648408198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 msgr2=0x55cb204b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 -- 172.21.15.94:0/3648408198 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb20245f70 con 0x55cb200e4730 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f553b7fe640 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55300dcb30 tx=0x7f55300048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526ffbdea24304f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.242+0000 7f551b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300a7020 con 0x55cb200e4730 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55300b00f0 con 0x55cb200e4730 2024-04-25T13:43:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552c0044b0 con 0x55cb200e4730 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5530005680 con 0x55cb200e4730 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 10 client.25498 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb200dd170 con 0x55cb200e4730 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c0044b0 con 0x55cb200e4730 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f55300a2e80 con 0x55cb200e4730 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 10 client.25498 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 20 client.25498 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb200c41f0 0x55cb204bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 4 client.25498 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb20419710 con 0x55cb200c41f0 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.243+0000 7f553bfff640 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb200c41f0 0x55cb204bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.244+0000 7f551b7fe640 10 client.25498 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.244+0000 7f553bfff640 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb200c41f0 0x55cb204bdbe0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e89e87945e875c server_cookie=206570bb4602eb95 in_seq=0 out_seq=0 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f551b7fe640 1 -- 172.21.15.94:0/3648408198 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb20419710 con 0x55cb200c41f0 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f551b7fe640 10 client.25498 handle_command_reply: tid=0 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 client.25498 shutdown 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 msgr2=0x55cb204b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55300dcb30 tx=0x7f55300048f0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 shutdown_connections 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb200c41f0 0x55cb204bdbe0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb20105410 0x55cb204b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f551b7fe640 0 client.25498 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200e4730 0x55cb204b0780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 --2- 172.21.15.94:0/3648408198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 >> 172.21.15.94:0/3648408198 conn(0x55cb200f9560 msgr2=0x55cb204a0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 shutdown_connections 2024-04-25T13:43:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 1 -- 172.21.15.94:0/3648408198 wait complete. 2024-04-25T13:43:27.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.245+0000 7f55493f6740 20 client.25498 trim_cache size 0 max 16384 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 Processor -- start 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 -- start start 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 0x55cb204a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1ad6d0 con 0x55cb200e4730 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1af210 con 0x55cb200dc550 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f55493f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1adda0 con 0x55cb200c41f0 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f5540ae3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57474/0 (socket says 172.21.15.94:57474) 2024-04-25T13:43:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f5540ae3640 1 -- 172.21.15.94:0/1320081292 learned_addr learned my addr 172.21.15.94:0/1320081292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f553b7fe640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 0x55cb204a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f553bfff640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.264+0000 7f5540ae3640 1 -- 172.21.15.94:0/1320081292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 msgr2=0x55cb204aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5540ae3640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5540ae3640 1 -- 172.21.15.94:0/1320081292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 msgr2=0x55cb204a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5540ae3640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 0x55cb204a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5540ae3640 1 -- 172.21.15.94:0/1320081292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb204bb920 con 0x55cb200e4730 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f553b7fe640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 0x55cb204a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5540ae3640 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5534035000 tx=0x7f55340093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f008cf156487a511 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5521ffb640 1 -- 172.21.15.94:0/1320081292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534046070 con 0x55cb200e4730 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5521ffb640 1 -- 172.21.15.94:0/1320081292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534036040 con 0x55cb200e4730 2024-04-25T13:43:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.265+0000 7f5521ffb640 1 -- 172.21.15.94:0/1320081292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553402fe80 con 0x55cb200e4730 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- 172.21.15.94:0/1320081292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 msgr2=0x55cb204b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5534035000 tx=0x7f55340093d0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- 172.21.15.94:0/1320081292 shutdown_connections 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200e4730 0x55cb204b7600 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200dc550 0x55cb204a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- 172.21.15.94:0/1320081292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- 172.21.15.94:0/1320081292 >> 172.21.15.94:0/1320081292 conn(0x55cb200f9560 msgr2=0x55cb204c2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- 172.21.15.94:0/1320081292 shutdown_connections 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- 172.21.15.94:0/1320081292 wait complete. 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- start start 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 Processor -- start 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1aeb40 con 0x55cb200dc550 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1ae470 con 0x55cb200e4730 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.266+0000 7f55493f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1ab6b0 con 0x55cb200c41f0 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f55493f6740 10 client.0 _ll_register_callbacks cb 0x7f5530009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57478/0 (socket says 172.21.15.94:57478) 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 -- 172.21.15.94:0/2453342348 learned_addr learned my addr 172.21.15.94:0/2453342348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553bfff640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f5540ae3640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 -- 172.21.15.94:0/2453342348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 msgr2=0x55cb204b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 -- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 msgr2=0x55cb204b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 -- 172.21.15.94:0/2453342348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb204b7700 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f553b7fe640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f55300e2910 tx=0x7f5530090070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba67f7e316207dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f551b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300a6970 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f55300a61e0 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534036040 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55300c8050 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f55493f6740 10 client.16065 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.267+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb20526540 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534036040 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f55300a63e0 con 0x55cb200dc550 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 10 client.16065 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 20 client.16065 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:27.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb20105410 0x55cb20528da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 4 client.16065 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:43:27.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb20413580 con 0x55cb20105410 2024-04-25T13:43:27.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f553bfff640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb20105410 0x55cb20528da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f551b7fe640 10 client.16065 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.268+0000 7f553bfff640 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb20105410 0x55cb20528da0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08d611c0192bc18 server_cookie=cc581c9ca81fbcd5 in_seq=0 out_seq=0 2024-04-25T13:43:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f551b7fe640 1 -- 172.21.15.94:0/2453342348 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb20413580 con 0x55cb20105410 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f551b7fe640 10 client.16065 handle_command_reply: tid=0 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 client.16065 shutdown 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 msgr2=0x55cb204b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f55300e2910 tx=0x7f5530090070 comp rx=0 tx=0).stop 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 shutdown_connections 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cb20105410 0x55cb20528da0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb200e4730 0x55cb204b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb200dc550 0x55cb204b6210 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f551b7fe640 0 client.16065 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 --2- 172.21.15.94:0/2453342348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb200c41f0 0x55cb204b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 >> 172.21.15.94:0/2453342348 conn(0x55cb200f9560 msgr2=0x55cb204a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 shutdown_connections 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 1 -- 172.21.15.94:0/2453342348 wait complete. 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.280+0000 7f55493f6740 20 client.16065 trim_cache size 0 max 16384 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ba6ae560-4b37-421e-96b6-be5fcb0bb244", 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:43:27.282 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:43:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 msgr2=0x7f55240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 0x7f55240b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55cb204a7670 tx=0x7f553402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 msgr2=0x7f553c10b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f553000e9f0 tx=0x7f553000edf0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 shutdown_connections 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55240afaf0 0x7f55240b2000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f553c108920 0x7f553c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553c103c20 0x7f553c10b950 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 --2- 172.21.15.94:0/4226299942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f553c1029e0 0x7f553c10d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 >> 172.21.15.94:0/4226299942 conn(0x7f553c0fdbd0 msgr2=0x7f553c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 shutdown_connections 2024-04-25T13:43:27.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.281+0000 7f5542546640 1 -- 172.21.15.94:0/4226299942 wait complete. 2024-04-25T13:43:27.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:43:27.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:43:27.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/486402458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 msgr2=0x7fd9e01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/486402458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e01069f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009900 tx=0x7fd9d002a410 comp rx=0 tx=0).stop 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/486402458 shutdown_connections 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/486402458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0109890 0x7fd9e010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/486402458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0106f30 0x7fd9e0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/486402458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e01069f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/486402458 >> 172.21.15.94:0/486402458 conn(0x7fd9e00fdbd0 msgr2=0x7fd9e0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/486402458 shutdown_connections 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.408+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/486402458 wait complete. 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 Processor -- start 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 -- start start 2024-04-25T13:43:27.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0106f30 0x7fd9e01020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0109890 0x7fd9e0102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010f690 con 0x7fd9e01045d0 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010f510 con 0x7fd9e0106f30 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9e4f0d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010f810 con 0x7fd9e0109890 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57496/0 (socket says 172.21.15.94:57496) 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9de59c640 1 -- 172.21.15.94:0/3183291194 learned_addr learned my addr 172.21.15.94:0/3183291194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0106f30 0x7fd9e01020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.409+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0109890 0x7fd9e0102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 -- 172.21.15.94:0/3183291194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0109890 msgr2=0x7fd9e0102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0109890 0x7fd9e0102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 -- 172.21.15.94:0/3183291194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0106f30 msgr2=0x7fd9e01020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0106f30 0x7fd9e01020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 -- 172.21.15.94:0/3183291194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e0102d30 con 0x7fd9e01045d0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0031440 tx=0x7fd9d0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/3183291194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d003f070 con 0x7fd9e01045d0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/3183291194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d003b460 con 0x7fd9e01045d0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/3183291194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0035030 con 0x7fd9e01045d0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e01a46d0 con 0x7fd9e01045d0 2024-04-25T13:43:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.410+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a4bb0 con 0x7fd9e01045d0 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/3183291194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd9d0042070 con 0x7fd9e01045d0 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 Processor -- start 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 -- start start 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab24bd80 con 0x55c8aaf45390 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab2058b0 con 0x55c8aaf246b0 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab247d40 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ded9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ddd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ded9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57508/0 (socket says 172.21.15.94:57508) 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9ded9d640 1 -- 172.21.15.94:0/3459802072 learned_addr learned my addr 172.21.15.94:0/3459802072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.413+0000 7fd9cf7fe640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 0x7fd9c80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 -- 172.21.15.94:0/3459802072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf1c4d0 msgr2=0x55c8ab2e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 -- 172.21.15.94:0/3459802072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 msgr2=0x55c8ab2e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 -- 172.21.15.94:0/3459802072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ab1d6a40 con 0x55c8aaf45390 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9cf7fe640 1 -- 172.21.15.94:0/3183291194 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd9d00ff4f0 con 0x7fd9e01045d0 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 0x7fd9c80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8010540 tx=0x7fd9d8010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0adafc1f328dd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ccff9640 1 -- 172.21.15.94:0/3459802072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8022070 con 0x55c8aaf45390 2024-04-25T13:43:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ccff9640 1 -- 172.21.15.94:0/3459802072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d801c390 con 0x55c8aaf45390 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.414+0000 7fd9ccff9640 1 -- 172.21.15.94:0/3459802072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800e630 con 0x55c8aaf45390 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3459802072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 msgr2=0x55c8ab2e6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 0x7fd9c80b2080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd9d801f000 tx=0x7fd9d801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8010540 tx=0x7fd9d8010a00 comp rx=0 tx=0).stop 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3459802072 shutdown_connections 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3459802072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf45390 0x55c8ab2e6a60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3459802072 >> 172.21.15.94:0/3459802072 conn(0x55c8aaf394e0 msgr2=0x55c8ab2e2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3459802072 shutdown_connections 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.415+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3459802072 wait complete. 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 -- start start 2024-04-25T13:43:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 Processor -- start 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf45390 0x55c8ab2f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab2e8780 con 0x55c8aaf246b0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab1d2d10 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ab2e8900 con 0x55c8aaf45390 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ebdc4740 10 client.0 _ll_register_callbacks cb 0x55c8ab1ff210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57518/0 (socket says 172.21.15.94:57518) 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/3889031930 learned_addr learned my addr 172.21.15.94:0/3889031930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf45390 0x55c8ab2f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/3889031930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf45390 msgr2=0x55c8ab2f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf45390 0x55c8ab2f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 msgr2=0x55c8ab2f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/3889031930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ab0277d0 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.416+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4011570 tx=0x7fd9d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45858dbb704b7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4013670 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd9d4013810 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d801c390 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9d4013a10 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9ebdc4740 10 client.16077 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c8aaf5d3f0 con 0x55c8aaf246b0 2024-04-25T13:43:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d801c390 con 0x55c8aaf246b0 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.417+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd9d4027450 con 0x55c8aaf246b0 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 20 client.16077 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf03f60 0x55c8ab2fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 4 client.16077 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8ab259990 con 0x55c8aaf03f60 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf03f60 0x55c8ab2fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9c5ffb640 10 client.16077 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.418+0000 7fd9de59c640 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf03f60 0x55c8ab2fe280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ca77b087c9a85d server_cookie=8efdaa65a387900e in_seq=0 out_seq=0 2024-04-25T13:43:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/3889031930 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8ab259990 con 0x55c8aaf03f60 2024-04-25T13:43:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9c5ffb640 10 client.16077 handle_command_reply: tid=0 2024-04-25T13:43:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 client.16077 shutdown 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 msgr2=0x55c8ab2f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4011570 tx=0x7fd9d4009f90 comp rx=0 tx=0).stop 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 shutdown_connections 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf03f60 0x55c8ab2fe280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf45390 0x55c8ab2f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f0e10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9c5ffb640 0 client.16077 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/3889031930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.419+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 >> 172.21.15.94:0/3889031930 conn(0x55c8aaf394e0 msgr2=0x55c8ab2e5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.420+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 shutdown_connections 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.420+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/3889031930 wait complete. 2024-04-25T13:43:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.420+0000 7fd9ebdc4740 20 client.16077 trim_cache size 0 max 16384 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 Processor -- start 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 -- start start 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf03f60 0x55c8ab2e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2eef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01ad7c0 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01af300 con 0x55c8aaf246b0 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01ade90 con 0x55c8aaf03f60 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57520/0 (socket says 172.21.15.94:57520) 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/2636263698 learned_addr learned my addr 172.21.15.94:0/2636263698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9de59c640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf03f60 0x55c8ab2e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/2636263698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf03f60 msgr2=0x55c8ab2e1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf03f60 0x55c8ab2e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/2636263698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 msgr2=0x55c8ab2eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 -- 172.21.15.94:0/2636263698 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8aaf1d0f0 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.438+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4000c00 tx=0x7fd9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bece2162a2bf0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.439+0000 7fd9c7fff640 1 -- 172.21.15.94:0/2636263698 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4027450 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.439+0000 7fd9c7fff640 1 -- 172.21.15.94:0/2636263698 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d40275f0 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.439+0000 7fd9c7fff640 1 -- 172.21.15.94:0/2636263698 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d401d6e0 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.439+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/2636263698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 msgr2=0x55c8ab2e21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.439+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4000c00 tx=0x7fd9d40046c0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/2636263698 shutdown_connections 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf246b0 0x55c8ab2eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2e21e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/2636263698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf03f60 0x55c8ab2e1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/2636263698 >> 172.21.15.94:0/2636263698 conn(0x55c8aaf394e0 msgr2=0x55c8ab2eb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/2636263698 shutdown_connections 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/2636263698 wait complete. 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- start start 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 Processor -- start 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01aec30 con 0x55c8aaf03f60 2024-04-25T13:43:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01ae560 con 0x55c8aaf1c4d0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01ab7a0 con 0x55c8aaf246b0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ebdc4740 10 client.0 _ll_register_callbacks cb 0x7fd9d4009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57532/0 (socket says 172.21.15.94:57532) 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9de59c640 1 -- 172.21.15.94:0/729898048 learned_addr learned my addr 172.21.15.94:0/729898048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ded9d640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.440+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 -- 172.21.15.94:0/729898048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf246b0 msgr2=0x55c8ab2f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 -- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 msgr2=0x55c8ab2f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 -- 172.21.15.94:0/729898048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ab20c4b0 con 0x55c8aaf03f60 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9de59c640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00fcf50 tx=0x7fd9d00ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b31a9aca667093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:27.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0049040 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd9d00c3bc0 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d40275f0 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9d00dee60 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 10 client.16083 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c8ab3657f0 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d40275f0 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd9d00ea0b0 con 0x55c8aaf03f60 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 10 client.16083 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 20 client.16083 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf45390 0x55c8ab3681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 4 client.16083 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:43:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.441+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8ab253800 con 0x55c8aaf45390 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.442+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf45390 0x55c8ab3681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.442+0000 7fd9c5ffb640 10 client.16083 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.442+0000 7fd9ddd9b640 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf45390 0x55c8ab3681c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c8f50fcb81215f server_cookie=c09e6e8b40606b3f in_seq=0 out_seq=0 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9c5ffb640 1 -- 172.21.15.94:0/729898048 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8ab253800 con 0x55c8aaf45390 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9c5ffb640 10 client.16083 handle_command_reply: tid=0 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 client.16083 shutdown 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 msgr2=0x55c8ab2f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00fcf50 tx=0x7fd9d00ea040 comp rx=0 tx=0).stop 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 shutdown_connections 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c8aaf45390 0x55c8ab3681c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8aaf246b0 0x55c8ab2f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8aaf1c4d0 0x55c8ab2f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9c5ffb640 0 client.16083 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 --2- 172.21.15.94:0/729898048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8aaf03f60 0x55c8ab2f73b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 >> 172.21.15.94:0/729898048 conn(0x55c8aaf394e0 msgr2=0x55c8ab2e1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 shutdown_connections 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 1 -- 172.21.15.94:0/729898048 wait complete. 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.443+0000 7fd9ebdc4740 20 client.16083 trim_cache size 0 max 16384 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "ba6ae560-4b37-421e-96b6-be5fcb0bb244": { 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:43:27.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ba6ae560-4b37-421e-96b6-be5fcb0bb244", 2024-04-25T13:43:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:43:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:43:27.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:43:27.451 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:43:27.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 msgr2=0x7fd9c80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 0x7fd9c80b2080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd9d801f000 tx=0x7fd9d801d000 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 msgr2=0x7fd9e0103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0031440 tx=0x7fd9d0003d50 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 shutdown_connections 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9c80afb70 0x7fd9c80b2080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e0109890 0x7fd9e0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9e0106f30 0x7fd9e01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 --2- 172.21.15.94:0/3183291194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9e01045d0 0x7fd9e0103a20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 >> 172.21.15.94:0/3183291194 conn(0x7fd9e00fdbd0 msgr2=0x7fd9e0105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 shutdown_connections 2024-04-25T13:43:27.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:27.444+0000 7fd9e4f0d640 1 -- 172.21.15.94:0/3183291194 wait complete. 2024-04-25T13:43:27.455 INFO:tasks.cephfs.filesystem:scrub status for tag:ba6ae560-4b37-421e-96b6-be5fcb0bb244 - {'path': '/', 'tag': 'ba6ae560-4b37-421e-96b6-be5fcb0bb244', 'options': 'recursive,force'} 2024-04-25T13:43:27.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 30858 18.11 MB/sec warmup 10 sec latency 16.420 ms 2024-04-25T13:43:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:28 smithi094 ceph-mon[29195]: pgmap v396: 97 pgs: 97 active+clean; 9.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 950 KiB/s wr, 161 op/s 2024-04-25T13:43:28.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:28 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:43:28.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:28 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:28 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:43:28.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:28 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:28 smithi138 ceph-mon[32977]: pgmap v396: 97 pgs: 97 active+clean; 9.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 950 KiB/s wr, 161 op/s 2024-04-25T13:43:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:28 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:43:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:28 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:28 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:43:28.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:28 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:28 smithi137 ceph-mon[32568]: pgmap v396: 97 pgs: 97 active+clean; 9.7 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 950 KiB/s wr, 161 op/s 2024-04-25T13:43:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:28 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:43:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:28 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:28 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:43:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:28 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:43:28.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 33623 18.03 MB/sec warmup 11 sec latency 15.415 ms 2024-04-25T13:43:29.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36382 17.74 MB/sec warmup 12 sec latency 21.050 ms 2024-04-25T13:43:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:30 smithi094 ceph-mon[29195]: pgmap v397: 97 pgs: 97 active+clean; 10 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 992 KiB/s wr, 162 op/s 2024-04-25T13:43:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:30 smithi138 ceph-mon[32977]: pgmap v397: 97 pgs: 97 active+clean; 10 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 992 KiB/s wr, 162 op/s 2024-04-25T13:43:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:30 smithi137 ceph-mon[32568]: pgmap v397: 97 pgs: 97 active+clean; 10 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 992 KiB/s wr, 162 op/s 2024-04-25T13:43:30.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 39471 17.79 MB/sec warmup 13 sec latency 39.307 ms 2024-04-25T13:43:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:43:31.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:43:31.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:43:31.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 42694 17.79 MB/sec warmup 14 sec latency 16.322 ms 2024-04-25T13:43:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:32 smithi094 ceph-mon[29195]: pgmap v398: 97 pgs: 97 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-25T13:43:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:32 smithi138 ceph-mon[32977]: pgmap v398: 97 pgs: 97 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-25T13:43:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:32 smithi137 ceph-mon[32568]: pgmap v398: 97 pgs: 97 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-25T13:43:32.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 45570 17.53 MB/sec warmup 15 sec latency 16.239 ms 2024-04-25T13:43:33.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 48126 17.19 MB/sec warmup 16 sec latency 17.106 ms 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 -- 172.21.15.94:0/3213539363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c103c40 msgr2=0x7fb83c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 --2- 172.21.15.94:0/3213539363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c103c40 0x7fb83c104050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009900 tx=0x7fb82c02a410 comp rx=0 tx=0).stop 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 -- 172.21.15.94:0/3213539363 shutdown_connections 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 --2- 172.21.15.94:0/3213539363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c108940 0x7fb83c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 --2- 172.21.15.94:0/3213539363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c103c40 0x7fb83c104050 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 --2- 172.21.15.94:0/3213539363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c102a00 0x7fb83c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 -- 172.21.15.94:0/3213539363 >> 172.21.15.94:0/3213539363 conn(0x7fb83c0fdbd0 msgr2=0x7fb83c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 -- 172.21.15.94:0/3213539363 shutdown_connections 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.017+0000 7fb84375e640 1 -- 172.21.15.94:0/3213539363 wait complete. 2024-04-25T13:43:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 Processor -- start 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 -- start start 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c103c40 0x7fb83c0fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c108940 0x7fb83c0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c10e500 con 0x7fb83c102a00 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c10e380 con 0x7fb83c103c40 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb84375e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c10e680 con 0x7fb83c108940 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb8414fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb8414fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52700/0 (socket says 172.21.15.94:52700) 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb8414fa640 1 -- 172.21.15.94:0/499085743 learned_addr learned my addr 172.21.15.94:0/499085743 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb840cf9640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c103c40 0x7fb83c0fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.018+0000 7fb841cfb640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c108940 0x7fb83c0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 -- 172.21.15.94:0/499085743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c108940 msgr2=0x7fb83c0fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c108940 0x7fb83c0fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 -- 172.21.15.94:0/499085743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c103c40 msgr2=0x7fb83c0fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c103c40 0x7fb83c0fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 -- 172.21.15.94:0/499085743 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c0fed70 con 0x7fb83c102a00 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8414fa640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb838005fd0 tx=0x7fb83800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8327fc640 1 -- 172.21.15.94:0/499085743 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb83c102a00 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c1a42f0 con 0x7fb83c102a00 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8327fc640 1 -- 172.21.15.94:0/499085743 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb838009c30 con 0x7fb83c102a00 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb8327fc640 1 -- 172.21.15.94:0/499085743 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838003d10 con 0x7fb83c102a00 2024-04-25T13:43:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.019+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a47e0 con 0x7fb83c102a00 2024-04-25T13:43:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.021+0000 7fb8327fc640 1 -- 172.21.15.94:0/499085743 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb838003eb0 con 0x7fb83c102a00 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 Processor -- start 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 -- start start 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05be1b0 0x55d4f095e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb8327fc640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 0x7fb8280b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 0x55d4f0965df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.022+0000 7fb84a60e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d4f08c4460 con 0x55d4f05952f0 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb84a60e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d4f087e590 con 0x55d4f059d4d0 2024-04-25T13:43:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb84a60e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d4f08c0420 con 0x55d4f05be1b0 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52714/0 (socket says 172.21.15.94:52714) 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 -- 172.21.15.94:0/882436690 learned_addr learned my addr 172.21.15.94:0/882436690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb841cfb640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 0x55d4f0965df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb841cfb640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05be1b0 0x55d4f095e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb8414fa640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 0x7fb8280b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb8327fc640 1 -- 172.21.15.94:0/499085743 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb8380de980 con 0x7fb83c102a00 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 -- 172.21.15.94:0/882436690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05be1b0 msgr2=0x55d4f095e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05be1b0 0x55d4f095e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 -- 172.21.15.94:0/882436690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 msgr2=0x55d4f0965df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 0x55d4f0965df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 -- 172.21.15.94:0/882436690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f088b0d0 con 0x55d4f05952f0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb841cfb640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 0x55d4f0965df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb8414fa640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 0x7fb8280b1ff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8380dc380 tx=0x7fb83800df10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.023+0000 7fb840cf9640 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb82c031500 tx=0x7fb82c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af03d0af8b00c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.024+0000 7fb8277fe640 1 -- 172.21.15.94:0/882436690 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035030 con 0x55d4f05952f0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.024+0000 7fb8277fe640 1 -- 172.21.15.94:0/882436690 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c040070 con 0x55d4f05952f0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.024+0000 7fb8277fe640 1 -- 172.21.15.94:0/882436690 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0038e0 con 0x55d4f05952f0 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- 172.21.15.94:0/882436690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 msgr2=0x55d4f095eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb82c031500 tx=0x7fb82c031b10 comp rx=0 tx=0).stop 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- 172.21.15.94:0/882436690 shutdown_connections 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f059d4d0 0x55d4f0965df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05952f0 0x55d4f095eef0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 --2- 172.21.15.94:0/882436690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05be1b0 0x55d4f095e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- 172.21.15.94:0/882436690 >> 172.21.15.94:0/882436690 conn(0x55d4f05b2300 msgr2=0x55d4f095a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- 172.21.15.94:0/882436690 shutdown_connections 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- 172.21.15.94:0/882436690 wait complete. 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 -- start start 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.025+0000 7fb84a60e740 1 Processor -- start 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f09685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f096fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d4f09667e0 con 0x55d4f059d4d0 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d4f084bb60 con 0x55d4f05be1b0 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d4f0966960 con 0x55d4f05952f0 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 10 client.0 _ll_register_callbacks cb 0x55d4f08702b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52722/0 (socket says 172.21.15.94:52722) 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 -- 172.21.15.94:0/3524663771 learned_addr learned my addr 172.21.15.94:0/3524663771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb841cfb640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f096fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb841cfb640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f09685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 -- 172.21.15.94:0/3524663771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 msgr2=0x55d4f09685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f09685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 -- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 msgr2=0x55d4f096fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f096fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 -- 172.21.15.94:0/3524663771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f05d6210 con 0x55d4f059d4d0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb841cfb640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f09685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb840cf9640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005ab0 tx=0x7fb82c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e95443fd94e28c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c053060 con 0x55d4f059d4d0 2024-04-25T13:43:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb82c003660 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c040070 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.026+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb82c035070 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 10 client.16095 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4f06a2620 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb82c040070 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb82c04d030 con 0x55d4f059d4d0 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 20 client.16095 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f057cd80 0x55d4f0975ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 4 client.16095 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4f08d2070 con 0x55d4f057cd80 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.027+0000 7fb841cfb640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f057cd80 0x55d4f0975ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.028+0000 7fb824ff9640 10 client.16095 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.028+0000 7fb841cfb640 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f057cd80 0x55d4f0975ed0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d4b8735fe3d6b0 server_cookie=e1ac10a2c843b017 in_seq=0 out_seq=0 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.028+0000 7fb824ff9640 1 -- 172.21.15.94:0/3524663771 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d4f08d2070 con 0x55d4f057cd80 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.028+0000 7fb824ff9640 10 client.16095 handle_command_reply: tid=0 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.028+0000 7fb84a60e740 1 client.16095 shutdown 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 msgr2=0x55d4f0968b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005ab0 tx=0x7fb82c004250 comp rx=0 tx=0).stop 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 shutdown_connections 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f057cd80 0x55d4f0975ed0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f096fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f0968b20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb824ff9640 0 client.16095 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 --2- 172.21.15.94:0/3524663771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f09685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 >> 172.21.15.94:0/3524663771 conn(0x55d4f05b2300 msgr2=0x55d4f095d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 shutdown_connections 2024-04-25T13:43:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 1 -- 172.21.15.94:0/3524663771 wait complete. 2024-04-25T13:43:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.029+0000 7fb84a60e740 20 client.16095 trim_cache size 0 max 16384 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 Processor -- start 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 -- start start 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05952f0 0x55d4f095d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 0x55d4f096f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.046+0000 7fb84a60e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ad740 con 0x55d4f05be1b0 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb84a60e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1af280 con 0x55d4f05952f0 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb84a60e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ade10 con 0x55d4f059d4d0 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52728/0 (socket says 172.21.15.94:52728) 2024-04-25T13:43:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 -- 172.21.15.94:0/1902295189 learned_addr learned my addr 172.21.15.94:0/1902295189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb840cf9640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 0x55d4f096f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05952f0 0x55d4f095d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 -- 172.21.15.94:0/1902295189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 msgr2=0x55d4f096f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 0x55d4f096f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 -- 172.21.15.94:0/1902295189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05952f0 msgr2=0x55d4f095d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05952f0 0x55d4f095d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 -- 172.21.15.94:0/1902295189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f09634b0 con 0x55d4f05be1b0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb840cf9640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 0x55d4f096f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb841cfb640 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb834009270 tx=0x7fb834009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684f14709e532d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb826ffd640 1 -- 172.21.15.94:0/1902295189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834004b20 con 0x55d4f05be1b0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb826ffd640 1 -- 172.21.15.94:0/1902295189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834003e20 con 0x55d4f05be1b0 2024-04-25T13:43:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.047+0000 7fb826ffd640 1 -- 172.21.15.94:0/1902295189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834023700 con 0x55d4f05be1b0 2024-04-25T13:43:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 -- 172.21.15.94:0/1902295189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 msgr2=0x55d4f0959c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb834009270 tx=0x7fb834009500 comp rx=0 tx=0).stop 2024-04-25T13:43:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 -- 172.21.15.94:0/1902295189 shutdown_connections 2024-04-25T13:43:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f05be1b0 0x55d4f0959c10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f059d4d0 0x55d4f096f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 --2- 172.21.15.94:0/1902295189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05952f0 0x55d4f095d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 -- 172.21.15.94:0/1902295189 >> 172.21.15.94:0/1902295189 conn(0x55d4f05b2300 msgr2=0x55d4f095d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 -- 172.21.15.94:0/1902295189 shutdown_connections 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.048+0000 7fb84a60e740 1 -- 172.21.15.94:0/1902295189 wait complete. 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 -- start start 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 Processor -- start 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f096ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f09dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1aebb0 con 0x55d4f059d4d0 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ae4e0 con 0x55d4f05be1b0 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ab720 con 0x55d4f05952f0 2024-04-25T13:43:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb84a60e740 10 client.0 _ll_register_callbacks cb 0x7fb834009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb840cf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52730/0 (socket says 172.21.15.94:52730) 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb840cf9640 1 -- 172.21.15.94:0/4043170736 learned_addr learned my addr 172.21.15.94:0/4043170736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb841cfb640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f096ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.049+0000 7fb841cfb640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f09dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 -- 172.21.15.94:0/4043170736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 msgr2=0x55d4f096ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f096ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 -- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 msgr2=0x55d4f09dc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f09dc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 -- 172.21.15.94:0/4043170736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f096fee0 con 0x55d4f059d4d0 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb841cfb640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f096ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb840cf9640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005a80 tx=0x7fb82c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672ed1d2c8ea9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c053070 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb82c0042c0 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb834003e20 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb82c000e60 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb84a60e740 10 client.16101 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4f09dd7a0 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.050+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834003e20 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb82c04d040 con 0x55d4f059d4d0 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 10 client.16101 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 20 client.16101 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f05521c0 0x55d4f09e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 4 client.16101 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:43:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4f08cbee0 con 0x55d4f05521c0 2024-04-25T13:43:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb841cfb640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f05521c0 0x55d4f09e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb824ff9640 10 client.16101 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.051+0000 7fb841cfb640 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f05521c0 0x55d4f09e0130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb1c0d1df114cd server_cookie=61ab9d8e8d3d231f in_seq=0 out_seq=0 2024-04-25T13:43:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.195+0000 7fb824ff9640 1 -- 172.21.15.94:0/4043170736 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4f08cbee0 con 0x55d4f05521c0 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb824ff9640 10 client.16101 handle_command_reply: tid=0 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 client.16101 shutdown 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 msgr2=0x55d4f096cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005a80 tx=0x7fb82c005850 comp rx=0 tx=0).stop 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 shutdown_connections 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4f05521c0 0x55d4f09e0130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4f05be1b0 0x55d4f09dc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4f059d4d0 0x55d4f096cf70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb824ff9640 0 client.16101 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 --2- 172.21.15.94:0/4043170736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4f05952f0 0x55d4f096ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 >> 172.21.15.94:0/4043170736 conn(0x55d4f05b2300 msgr2=0x55d4f0966d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 shutdown_connections 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 1 -- 172.21.15.94:0/4043170736 wait complete. 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.196+0000 7fb84a60e740 20 client.16101 trim_cache size 0 max 16384 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:43:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 msgr2=0x7fb8280b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 0x7fb8280b1ff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8380dc380 tx=0x7fb83800df10 comp rx=0 tx=0).stop 2024-04-25T13:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 msgr2=0x7fb83c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb838005fd0 tx=0x7fb83800dde0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 shutdown_connections 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8280afae0 0x7fb8280b1ff0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb83c108940 0x7fb83c0fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb83c103c40 0x7fb83c0fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 --2- 172.21.15.94:0/499085743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb83c102a00 0x7fb83c0ffa60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 >> 172.21.15.94:0/499085743 conn(0x7fb83c0fdbd0 msgr2=0x7fb83c109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 shutdown_connections 2024-04-25T13:43:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:34.197+0000 7fb84375e640 1 -- 172.21.15.94:0/499085743 wait complete. 2024-04-25T13:43:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:34 smithi094 ceph-mon[29195]: pgmap v399: 97 pgs: 97 active+clean; 16 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T13:43:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:34 smithi138 ceph-mon[32977]: pgmap v399: 97 pgs: 97 active+clean; 16 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T13:43:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:34 smithi137 ceph-mon[32568]: pgmap v399: 97 pgs: 97 active+clean; 16 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-25T13:43:34.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50479 17.01 MB/sec warmup 17 sec latency 68.365 ms 2024-04-25T13:43:35.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 53622 17.10 MB/sec warmup 18 sec latency 16.724 ms 2024-04-25T13:43:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:36 smithi094 ceph-mon[29195]: pgmap v400: 97 pgs: 97 active+clean; 21 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-25T13:43:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:36 smithi138 ceph-mon[32977]: pgmap v400: 97 pgs: 97 active+clean; 21 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-25T13:43:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:36 smithi137 ceph-mon[32568]: pgmap v400: 97 pgs: 97 active+clean; 21 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-25T13:43:36.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 56764 17.12 MB/sec warmup 19 sec latency 15.143 ms 2024-04-25T13:43:37.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 59362 16.75 MB/sec warmup 20 sec latency 14.539 ms 2024-04-25T13:43:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:38 smithi094 ceph-mon[29195]: pgmap v401: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T13:43:38.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:38 smithi138 ceph-mon[32977]: pgmap v401: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T13:43:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:38 smithi137 ceph-mon[32568]: pgmap v401: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T13:43:38.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 62309 16.74 MB/sec warmup 21 sec latency 15.624 ms 2024-04-25T13:43:39.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 65279 16.83 MB/sec warmup 22 sec latency 15.735 ms 2024-04-25T13:43:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:40 smithi094 ceph-mon[29195]: pgmap v402: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 269 op/s 2024-04-25T13:43:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:40 smithi138 ceph-mon[32977]: pgmap v402: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 269 op/s 2024-04-25T13:43:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:40 smithi137 ceph-mon[32568]: pgmap v402: 97 pgs: 97 active+clean; 19 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.5 MiB/s wr, 269 op/s 2024-04-25T13:43:40.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 68142 16.72 MB/sec warmup 23 sec latency 17.667 ms 2024-04-25T13:43:41.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 71349 16.81 MB/sec warmup 24 sec latency 7.291 ms 2024-04-25T13:43:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:42 smithi094 ceph-mon[29195]: pgmap v403: 97 pgs: 97 active+clean; 15 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-04-25T13:43:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:42 smithi138 ceph-mon[32977]: pgmap v403: 97 pgs: 97 active+clean; 15 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-04-25T13:43:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:42 smithi137 ceph-mon[32568]: pgmap v403: 97 pgs: 97 active+clean; 15 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.9 MiB/s wr, 394 op/s 2024-04-25T13:43:42.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 74365 16.72 MB/sec warmup 25 sec latency 15.084 ms 2024-04-25T13:43:43.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75295 16.27 MB/sec warmup 26 sec latency 247.713 ms 2024-04-25T13:43:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:44 smithi094 ceph-mon[29195]: pgmap v404: 97 pgs: 97 active+clean; 18 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-25T13:43:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:44 smithi138 ceph-mon[32977]: pgmap v404: 97 pgs: 97 active+clean; 18 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-25T13:43:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:44 smithi137 ceph-mon[32568]: pgmap v404: 97 pgs: 97 active+clean; 18 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-25T13:43:44.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 77882 16.19 MB/sec warmup 27 sec latency 316.858 ms 2024-04-25T13:43:45.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 79344 15.85 MB/sec warmup 28 sec latency 33.232 ms 2024-04-25T13:43:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:46 smithi094 ceph-mon[29195]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-25T13:43:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:46 smithi138 ceph-mon[32977]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-25T13:43:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:46 smithi137 ceph-mon[32568]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-25T13:43:46.789 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82191 15.89 MB/sec warmup 29 sec latency 15.549 ms 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 -- 172.21.15.94:0/3077618923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81701021c0 msgr2=0x7f8170102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 --2- 172.21.15.94:0/3077618923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81701021c0 0x7f8170102ec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f816000adb0 tx=0x7f816002b7a0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 -- 172.21.15.94:0/3077618923 shutdown_connections 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 --2- 172.21.15.94:0/3077618923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 0x7f817010a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 --2- 172.21.15.94:0/3077618923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8170103400 0x7f8170103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 --2- 172.21.15.94:0/3077618923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81701021c0 0x7f8170102ec0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.294+0000 7f8175185640 1 -- 172.21.15.94:0/3077618923 >> 172.21.15.94:0/3077618923 conn(0x7f81700fd390 msgr2=0x7f81700ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.295+0000 7f8175185640 1 -- 172.21.15.94:0/3077618923 shutdown_connections 2024-04-25T13:43:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.295+0000 7f8175185640 1 -- 172.21.15.94:0/3077618923 wait complete. 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.295+0000 7f8175185640 1 Processor -- start 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 -- start start 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81701021c0 0x7f817019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 0x7f81701a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d450 con 0x7f8170103400 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d2d0 con 0x7f81701021c0 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8175185640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d5d0 con 0x7f8170108100 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47098/0 (socket says 172.21.15.94:47098) 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 -- 172.21.15.94:0/3425854068 learned_addr learned my addr 172.21.15.94:0/3425854068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816f59e640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 0x7f81701a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816ed9d640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81701021c0 0x7f817019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 -- 172.21.15.94:0/3425854068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 msgr2=0x7f81701a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 0x7f81701a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 -- 172.21.15.94:0/3425854068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81701021c0 msgr2=0x7f817019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81701021c0 0x7f817019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 -- 172.21.15.94:0/3425854068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701a1d80 con 0x7f8170103400 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f816e59c640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.296+0000 7f8157fff640 1 -- 172.21.15.94:0/3425854068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400cd80 con 0x7f8170103400 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.297+0000 7f8157fff640 1 -- 172.21.15.94:0/3425854068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81640108e0 con 0x7f8170103400 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.297+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701a2070 con 0x7f8170103400 2024-04-25T13:43:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.297+0000 7f8157fff640 1 -- 172.21.15.94:0/3425854068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164018980 con 0x7f8170103400 2024-04-25T13:43:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.297+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a25c0 con 0x7f8170103400 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 Processor -- start 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f8157fff640 1 -- 172.21.15.94:0/3425854068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8164010430 con 0x7f8170103400 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 -- start start 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575872d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575872def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x555758734df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555758693460 con 0x5557583642f0 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55575864d590 con 0x55575836c4d0 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.299+0000 7f817c031740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55575868f420 con 0x55575838d1b0 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575872def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575872d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575872def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47104/0 (socket says 172.21.15.94:47104) 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816ed9d640 1 -- 172.21.15.94:0/4100321208 learned_addr learned my addr 172.21.15.94:0/4100321208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816e59c640 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x555758734df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 -- 172.21.15.94:0/4100321208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 msgr2=0x555758734df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x555758734df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 -- 172.21.15.94:0/4100321208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 msgr2=0x55575872def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575872def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 -- 172.21.15.94:0/4100321208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575865a0d0 con 0x55575838d1b0 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f8157fff640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 0x7f81580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816ed9d640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 0x7f81580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f816f59e640 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575872d970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8168009b80 tx=0x7f8168011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53fabd8a957a0646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f8157fff640 1 -- 172.21.15.94:0/3425854068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f81640def90 con 0x7f8170103400 2024-04-25T13:43:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f81557fa640 1 -- 172.21.15.94:0/4100321208 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168004030 con 0x55575838d1b0 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f81557fa640 1 -- 172.21.15.94:0/4100321208 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168005020 con 0x55575838d1b0 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.300+0000 7f81557fa640 1 -- 172.21.15.94:0/4100321208 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168004c00 con 0x55575838d1b0 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 -- 172.21.15.94:0/4100321208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 msgr2=0x55575872d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575872d970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8168009b80 tx=0x7f8168011810 comp rx=0 tx=0).stop 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f816ed9d640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 0x7f81580b1ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8160004af0 tx=0x7f8160005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 -- 172.21.15.94:0/4100321208 shutdown_connections 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x555758734df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575872def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 --2- 172.21.15.94:0/4100321208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575872d970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 -- 172.21.15.94:0/4100321208 >> 172.21.15.94:0/4100321208 conn(0x555758381300 msgr2=0x555758729300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 -- 172.21.15.94:0/4100321208 shutdown_connections 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.301+0000 7f817c031740 1 -- 172.21.15.94:0/4100321208 wait complete. 2024-04-25T13:43:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 -- start start 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 Processor -- start 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557583642f0 0x5557587376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575873eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555758735840 con 0x55575836c4d0 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55575861ab60 con 0x5557583642f0 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557587359c0 con 0x55575838d1b0 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f817c031740 10 client.0 _ll_register_callbacks cb 0x55575863f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47120/0 (socket says 172.21.15.94:47120) 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 -- 172.21.15.94:0/348556601 learned_addr learned my addr 172.21.15.94:0/348556601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816ed9d640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557583642f0 0x5557587376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816f59e640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575873eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 -- 172.21.15.94:0/348556601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 msgr2=0x55575873eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575873eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 -- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557583642f0 msgr2=0x5557587376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557583642f0 0x5557587376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.302+0000 7f816e59c640 1 -- 172.21.15.94:0/348556601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557583a5210 con 0x55575836c4d0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f816e59c640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f81640a3500 tx=0x7f81640e2660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627a00436568099f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010720 con 0x55575836c4d0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f81640e10d0 con 0x55575836c4d0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8168005020 con 0x55575836c4d0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f81640a6f90 con 0x55575836c4d0 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 10 client.16113 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555758471620 con 0x55575836c4d0 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8168005020 con 0x55575836c4d0 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f81640afda0 con 0x55575836c4d0 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 10 client.16113 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 20 client.16113 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55575834bd80 0x555758745380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 4 client.16113 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557586a1070 con 0x55575834bd80 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.303+0000 7f816ed9d640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55575834bd80 0x555758745380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.304+0000 7f816ed9d640 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55575834bd80 0x555758745380 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b814bf8cd4074fc5 server_cookie=e4c9d94eec8fb901 in_seq=0 out_seq=0 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.304+0000 7f8152ff5640 10 client.16113 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f8152ff5640 1 -- 172.21.15.94:0/348556601 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557586a1070 con 0x55575834bd80 2024-04-25T13:43:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f8152ff5640 10 client.16113 handle_command_reply: tid=0 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 client.16113 shutdown 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 msgr2=0x555758737c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f81640a3500 tx=0x7f81640e2660 comp rx=0 tx=0).stop 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 shutdown_connections 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55575834bd80 0x555758745380 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x55575873eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55575836c4d0 0x555758737c10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f8152ff5640 0 client.16113 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 --2- 172.21.15.94:0/348556601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557583642f0 0x5557587376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 >> 172.21.15.94:0/348556601 conn(0x555758381300 msgr2=0x555758728250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 shutdown_connections 2024-04-25T13:43:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 1 -- 172.21.15.94:0/348556601 wait complete. 2024-04-25T13:43:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.306+0000 7f817c031740 20 client.16113 trim_cache size 0 max 16384 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 Processor -- start 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 -- start start 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x55575873ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x555758729120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ab4f0 con 0x5557583642f0 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ad030 con 0x55575836c4d0 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f817c031740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81701abbc0 con 0x55575838d1b0 2024-04-25T13:43:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47136/0 (socket says 172.21.15.94:47136) 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 -- 172.21.15.94:0/4104895156 learned_addr learned my addr 172.21.15.94:0/4104895156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816f59e640 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x555758729120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816e59c640 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x55575873ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 -- 172.21.15.94:0/4104895156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 msgr2=0x555758729120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x555758729120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 -- 172.21.15.94:0/4104895156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 msgr2=0x55575873ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x55575873ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.322+0000 7f816ed9d640 1 -- 172.21.15.94:0/4104895156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555758734f10 con 0x5557583642f0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f816ed9d640 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8160006010 tx=0x7f8160045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b88c60c55ebfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f8154ff9640 1 -- 172.21.15.94:0/4104895156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160003af0 con 0x5557583642f0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f8154ff9640 1 -- 172.21.15.94:0/4104895156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160003c90 con 0x5557583642f0 2024-04-25T13:43:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f8154ff9640 1 -- 172.21.15.94:0/4104895156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600044d0 con 0x5557583642f0 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f817c031740 1 -- 172.21.15.94:0/4104895156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 msgr2=0x555758728be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.323+0000 7f817c031740 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8160006010 tx=0x7f8160045000 comp rx=0 tx=0).stop 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- 172.21.15.94:0/4104895156 shutdown_connections 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575838d1b0 0x555758729120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575836c4d0 0x55575873ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- 172.21.15.94:0/4104895156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x555758728be0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- 172.21.15.94:0/4104895156 >> 172.21.15.94:0/4104895156 conn(0x555758381300 msgr2=0x555758730c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- 172.21.15.94:0/4104895156 shutdown_connections 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- 172.21.15.94:0/4104895156 wait complete. 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- start start 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 Processor -- start 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575836c4d0 0x55575873c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 0x55575873d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ac960 con 0x5557583642f0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81701ac290 con 0x55575838d1b0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81701a94d0 con 0x55575836c4d0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f817c031740 10 client.0 _ll_register_callbacks cb 0x7f81640e2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f816ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47144/0 (socket says 172.21.15.94:47144) 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f816ed9d640 1 -- 172.21.15.94:0/1367891093 learned_addr learned my addr 172.21.15.94:0/1367891093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f816e59c640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575836c4d0 0x55575873c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.324+0000 7f816f59e640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 0x55575873d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 -- 172.21.15.94:0/1367891093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575836c4d0 msgr2=0x55575873c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575836c4d0 0x55575873c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 -- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 msgr2=0x55575873d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 0x55575873d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 -- 172.21.15.94:0/1367891093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575873da70 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816f59e640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 0x55575873d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f816ed9d640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8160000f80 tx=0x7f8160003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a2f4c5985e2843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160004250 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f8160043040 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160003c90 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f81600043f0 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 10 client.16119 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55575873ed00 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160003c90 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f816005c070 con 0x5557583642f0 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 10 client.16119 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 20 client.16119 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5557583211c0 0x5557587afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 4 client.16119 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.325+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55575869aee0 con 0x5557583211c0 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.326+0000 7f816e59c640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5557583211c0 0x5557587afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.326+0000 7f816e59c640 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5557583211c0 0x5557587afd70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc4c6c74c99ec5b server_cookie=d50958b55d1beffa in_seq=0 out_seq=0 2024-04-25T13:43:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.326+0000 7f8152ff5640 10 client.16119 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f8152ff5640 1 -- 172.21.15.94:0/1367891093 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55575869aee0 con 0x5557583211c0 2024-04-25T13:43:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f8152ff5640 10 client.16119 handle_command_reply: tid=0 2024-04-25T13:43:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 client.16119 shutdown 2024-04-25T13:43:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 msgr2=0x55575873c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8160000f80 tx=0x7f8160003bc0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 shutdown_connections 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5557583211c0 0x5557587afd70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55575838d1b0 0x55575873d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55575836c4d0 0x55575873c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 --2- 172.21.15.94:0/1367891093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557583642f0 0x55575873c090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.399+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 >> 172.21.15.94:0/1367891093 conn(0x555758381300 msgr2=0x555758728be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8152ff5640 0 client.16119 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 shutdown_connections 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f817c031740 1 -- 172.21.15.94:0/1367891093 wait complete. 2024-04-25T13:43:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f817c031740 20 client.16119 trim_cache size 0 max 16384 2024-04-25T13:43:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:43:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:43:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:43:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 msgr2=0x7f81580b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 0x7f81580b1ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8160004af0 tx=0x7f8160005d80 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 msgr2=0x7f817019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9f0 tx=0x7f816400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 shutdown_connections 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81580afae0 0x7f81580b1ff0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8170108100 0x7f81701a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8170103400 0x7f817019a780 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.400+0000 7f8175185640 1 --2- 172.21.15.94:0/3425854068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81701021c0 0x7f817019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.401+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 >> 172.21.15.94:0/3425854068 conn(0x7f81700fd390 msgr2=0x7f81701040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.401+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 shutdown_connections 2024-04-25T13:43:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:43:47.401+0000 7f8175185640 1 -- 172.21.15.94:0/3425854068 wait complete. 2024-04-25T13:43:47.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85250 15.95 MB/sec warmup 30 sec latency 22.178 ms 2024-04-25T13:43:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:48 smithi094 ceph-mon[29195]: pgmap v406: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 6.6 MiB/s wr, 399 op/s 2024-04-25T13:43:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:48 smithi138 ceph-mon[32977]: pgmap v406: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 6.6 MiB/s wr, 399 op/s 2024-04-25T13:43:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:48 smithi137 ceph-mon[32568]: pgmap v406: 97 pgs: 97 active+clean; 66 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 6.6 MiB/s wr, 399 op/s 2024-04-25T13:43:48.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 87632 15.71 MB/sec warmup 31 sec latency 15.257 ms 2024-04-25T13:43:49.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 89977 15.66 MB/sec warmup 32 sec latency 16.307 ms 2024-04-25T13:43:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:50 smithi094 ceph-mon[29195]: pgmap v407: 97 pgs: 97 active+clean; 67 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 7.4 MiB/s wr, 388 op/s 2024-04-25T13:43:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:50 smithi138 ceph-mon[32977]: pgmap v407: 97 pgs: 97 active+clean; 67 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 7.4 MiB/s wr, 388 op/s 2024-04-25T13:43:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:50 smithi137 ceph-mon[32568]: pgmap v407: 97 pgs: 97 active+clean; 67 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 7.4 MiB/s wr, 388 op/s 2024-04-25T13:43:50.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92829 15.70 MB/sec warmup 33 sec latency 7.850 ms 2024-04-25T13:43:51.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 95973 15.77 MB/sec warmup 34 sec latency 19.667 ms 2024-04-25T13:43:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:52 smithi094 ceph-mon[29195]: pgmap v408: 97 pgs: 97 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 527 op/s 2024-04-25T13:43:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:52 smithi138 ceph-mon[32977]: pgmap v408: 97 pgs: 97 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 527 op/s 2024-04-25T13:43:52.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:52 smithi137 ceph-mon[32568]: pgmap v408: 97 pgs: 97 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 527 op/s 2024-04-25T13:43:52.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 98574 15.67 MB/sec warmup 35 sec latency 15.290 ms 2024-04-25T13:43:53.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 100620 15.52 MB/sec warmup 36 sec latency 14.940 ms 2024-04-25T13:43:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:54 smithi094 ceph-mon[29195]: pgmap v409: 97 pgs: 97 active+clean; 26 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 412 op/s 2024-04-25T13:43:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:54 smithi138 ceph-mon[32977]: pgmap v409: 97 pgs: 97 active+clean; 26 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 412 op/s 2024-04-25T13:43:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:54 smithi137 ceph-mon[32568]: pgmap v409: 97 pgs: 97 active+clean; 26 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 412 op/s 2024-04-25T13:43:54.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103376 15.56 MB/sec warmup 37 sec latency 14.746 ms 2024-04-25T13:43:55.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 106460 15.61 MB/sec warmup 38 sec latency 44.235 ms 2024-04-25T13:43:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:56 smithi138 ceph-mon[32977]: pgmap v410: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 450 op/s 2024-04-25T13:43:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:56 smithi137 ceph-mon[32568]: pgmap v410: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 450 op/s 2024-04-25T13:43:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:56 smithi094 ceph-mon[29195]: pgmap v410: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 450 op/s 2024-04-25T13:43:56.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 108891 15.43 MB/sec warmup 39 sec latency 14.956 ms 2024-04-25T13:43:57.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:43:57.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:43:57.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.571+0000 7f33f41a8640 1 -- 172.21.15.94:0/1898497130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec1029e0 msgr2=0x7f33ec1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.571+0000 7f33f41a8640 1 --2- 172.21.15.94:0/1898497130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec1029e0 0x7f33ec1036e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009900 tx=0x7f33e802a410 comp rx=0 tx=0).stop 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.571+0000 7f33f41a8640 1 -- 172.21.15.94:0/1898497130 shutdown_connections 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.571+0000 7f33f41a8640 1 --2- 172.21.15.94:0/1898497130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 0x7f33ec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 --2- 172.21.15.94:0/1898497130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec103c20 0x7f33ec104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 --2- 172.21.15.94:0/1898497130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec1029e0 0x7f33ec1036e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- 172.21.15.94:0/1898497130 >> 172.21.15.94:0/1898497130 conn(0x7f33ec0fdbd0 msgr2=0x7f33ec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- 172.21.15.94:0/1898497130 shutdown_connections 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- 172.21.15.94:0/1898497130 wait complete. 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 Processor -- start 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- start start 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec1029e0 0x7f33ec19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 0x7f33ec1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10f610 con 0x7f33ec103c20 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10f490 con 0x7f33ec108920 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.572+0000 7f33f41a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10f790 con 0x7f33ec1029e0 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51782/0 (socket says 172.21.15.94:51782) 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 -- 172.21.15.94:0/2241849806 learned_addr learned my addr 172.21.15.94:0/2241849806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f2745640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 0x7f33ec1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1f44640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec1029e0 0x7f33ec19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 -- 172.21.15.94:0/2241849806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec1029e0 msgr2=0x7f33ec19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec1029e0 0x7f33ec19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 -- 172.21.15.94:0/2241849806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 msgr2=0x7f33ec1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 0x7f33ec1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 -- 172.21.15.94:0/2241849806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec1a1dd0 con 0x7f33ec103c20 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f1743640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e9f0 tx=0x7f33dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33e2ffd640 1 -- 172.21.15.94:0/2241849806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00cd80 con 0x7f33ec103c20 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33e2ffd640 1 -- 172.21.15.94:0/2241849806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc0054d0 con 0x7f33ec103c20 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec1a20c0 con 0x7f33ec103c20 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33e2ffd640 1 -- 172.21.15.94:0/2241849806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc010690 con 0x7f33ec103c20 2024-04-25T13:43:57.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.573+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec1a25c0 con 0x7f33ec103c20 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.575+0000 7f33e2ffd640 1 -- 172.21.15.94:0/2241849806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f33dc004e90 con 0x7f33ec103c20 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33fb054740 1 Processor -- start 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33fb054740 1 -- start start 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33e2ffd640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 0x7f33d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33fb054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e3c390 0x55f8351dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33f2745640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 0x7f33d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33fb054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.576+0000 7f33fb054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 0x55f8351dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33fb054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f835142d80 con 0x55f834e1b6b0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33fb054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8350fc8b0 con 0x55f834e134d0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33fb054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f83513ed40 con 0x55f834e3c390 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33e2ffd640 1 -- 172.21.15.94:0/2241849806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f33dc0df180 con 0x7f33ec103c20 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f2745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 0x55f8351dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f2745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 0x55f8351dec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51788/0 (socket says 172.21.15.94:51788) 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f2745640 1 -- 172.21.15.94:0/3891201185 learned_addr learned my addr 172.21.15.94:0/3891201185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1f44640 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e3c390 0x55f8351dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 -- 172.21.15.94:0/3891201185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e3c390 msgr2=0x55f8351dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e3c390 0x55f8351dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f2745640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 0x7f33d80b2000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f33e4013830 tx=0x7f33e4013c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 -- 172.21.15.94:0/3891201185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 msgr2=0x55f8351dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 0x55f8351dec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 -- 172.21.15.94:0/3891201185 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8350cda40 con 0x55f834e134d0 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33f1743640 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351ddfe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0dcf50 tx=0x7f33dc0a6d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae23c604a4da248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33d7fff640 1 -- 172.21.15.94:0/3891201185 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0c1070 con 0x55f834e134d0 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33d7fff640 1 -- 172.21.15.94:0/3891201185 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc0a7cc0 con 0x55f834e134d0 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.577+0000 7f33d7fff640 1 -- 172.21.15.94:0/3891201185 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0a3850 con 0x55f834e134d0 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 -- 172.21.15.94:0/3891201185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 msgr2=0x55f8351ddfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351ddfe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0dcf50 tx=0x7f33dc0a6d30 comp rx=0 tx=0).stop 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 -- 172.21.15.94:0/3891201185 shutdown_connections 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e1b6b0 0x55f8351dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351ddfe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 --2- 172.21.15.94:0/3891201185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e3c390 0x55f8351dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.578+0000 7f33fb054740 1 -- 172.21.15.94:0/3891201185 >> 172.21.15.94:0/3891201185 conn(0x55f834e304e0 msgr2=0x55f8351d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- 172.21.15.94:0/3891201185 shutdown_connections 2024-04-25T13:43:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- 172.21.15.94:0/3891201185 wait complete. 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- start start 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 Processor -- start 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 0x55f8351eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8351df780 con 0x55f834e134d0 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8350c9d10 con 0x55f834e3c390 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.579+0000 7f33fb054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8351df900 con 0x55f834e1b6b0 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33fb054740 10 client.0 _ll_register_callbacks cb 0x55f8350f6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51796/0 (socket says 172.21.15.94:51796) 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 -- 172.21.15.94:0/2351921532 learned_addr learned my addr 172.21.15.94:0/2351921532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f2745640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 0x55f8351eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1743640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 -- 172.21.15.94:0/2351921532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 msgr2=0x55f8351e7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351e7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 -- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 msgr2=0x55f8351eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 0x55f8351eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 -- 172.21.15.94:0/2351921532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f834f1e7d0 con 0x55f834e134d0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f2745640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 0x55f8351eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33f1f44640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005940 tx=0x7f33e80317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb46acf1cd294d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035040 con 0x55f834e134d0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f33e8004240 con 0x55f834e134d0 2024-04-25T13:43:57.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33dc0a7cc0 con 0x55f834e134d0 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33e8004440 con 0x55f834e134d0 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33fb054740 10 client.16131 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.580+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f834e543f0 con 0x55f834e134d0 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc0a7cc0 con 0x55f834e134d0 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33e80049f0 con 0x55f834e134d0 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 20 client.16131 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834dfaf60 0x55f8351f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 4 client.16131 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f835150990 con 0x55f834dfaf60 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33f1743640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834dfaf60 0x55f8351f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33d57fa640 10 client.16131 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.581+0000 7f33f1743640 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834dfaf60 0x55f8351f5220 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397342c1167acd04 server_cookie=9ad3b9a91937d1db in_seq=0 out_seq=0 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33d57fa640 1 -- 172.21.15.94:0/2351921532 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f835150990 con 0x55f834dfaf60 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33d57fa640 10 client.16131 handle_command_reply: tid=0 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 client.16131 shutdown 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 msgr2=0x55f8351e7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005940 tx=0x7f33e80317d0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 shutdown_connections 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834dfaf60 0x55f8351f5220 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e3c390 0x55f8351eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351e7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33d57fa640 0 client.16131 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 --2- 172.21.15.94:0/2351921532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834e134d0 0x55f8351e7960 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 >> 172.21.15.94:0/2351921532 conn(0x55f834e304e0 msgr2=0x55f8351d83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 shutdown_connections 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 1 -- 172.21.15.94:0/2351921532 wait complete. 2024-04-25T13:43:57.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.583+0000 7f33fb054740 20 client.16131 trim_cache size 0 max 16384 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.601+0000 7f33fb054740 1 Processor -- start 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.601+0000 7f33fb054740 1 -- start start 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1ab5f0 con 0x55f834dfaf60 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1ad130 con 0x55f834e134d0 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33fb054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1abcc0 con 0x55f834e1b6b0 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51804/0 (socket says 172.21.15.94:51804) 2024-04-25T13:43:57.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 -- 172.21.15.94:0/924844378 learned_addr learned my addr 172.21.15.94:0/924844378 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1743640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f2745640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 -- 172.21.15.94:0/924844378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 msgr2=0x55f8351ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 -- 172.21.15.94:0/924844378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 msgr2=0x55f8351eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 -- 172.21.15.94:0/924844378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f834e140f0 con 0x55f834dfaf60 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f2745640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33f1f44640 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f33e8031440 tx=0x7f33e8005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2941a96086dc970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33d77fe640 1 -- 172.21.15.94:0/924844378 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8000e60 con 0x55f834dfaf60 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33d77fe640 1 -- 172.21.15.94:0/924844378 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e8004b70 con 0x55f834dfaf60 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.602+0000 7f33d77fe640 1 -- 172.21.15.94:0/924844378 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035070 con 0x55f834dfaf60 2024-04-25T13:43:57.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- 172.21.15.94:0/924844378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 msgr2=0x55f8351e6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f33e8031440 tx=0x7f33e8005990 comp rx=0 tx=0).stop 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- 172.21.15.94:0/924844378 shutdown_connections 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e1b6b0 0x55f8351ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e134d0 0x55f8351eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 --2- 172.21.15.94:0/924844378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351e6610 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- 172.21.15.94:0/924844378 >> 172.21.15.94:0/924844378 conn(0x55f834e304e0 msgr2=0x55f8351e5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- 172.21.15.94:0/924844378 shutdown_connections 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- 172.21.15.94:0/924844378 wait complete. 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 -- start start 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.603+0000 7f33fb054740 1 Processor -- start 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e134d0 0x55f8351ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 0x55f8351ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1aca60 con 0x55f834dfaf60 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1ac390 con 0x55f834e1b6b0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1a95d0 con 0x55f834e134d0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33fb054740 10 client.0 _ll_register_callbacks cb 0x7f33dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51812/0 (socket says 172.21.15.94:51812) 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 -- 172.21.15.94:0/1813538451 learned_addr learned my addr 172.21.15.94:0/1813538451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1743640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e134d0 0x55f8351ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f2745640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 0x55f8351ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 -- 172.21.15.94:0/1813538451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e134d0 msgr2=0x55f8351ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e134d0 0x55f8351ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 -- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 msgr2=0x55f8351ec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:57.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 0x55f8351ec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 -- 172.21.15.94:0/1813538451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8351034b0 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f2745640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 0x55f8351ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33f1f44640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005940 tx=0x7f33e8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6099986c85674032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.604+0000 7f33d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8035050 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f33e8005550 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e8004b70 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33e8068050 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 10 client.16137 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f83525c4f0 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e8004b70 con 0x55f834dfaf60 2024-04-25T13:43:57.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33e8003b60 con 0x55f834dfaf60 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 10 client.16137 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 20 client.16137 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834e3c390 0x55f83525ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 4 client.16137 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f83514a800 con 0x55f834e3c390 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.605+0000 7f33f1743640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834e3c390 0x55f83525ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.606+0000 7f33d57fa640 10 client.16137 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:57.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.606+0000 7f33f1743640 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834e3c390 0x55f83525ee80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f15f6fe39d38d2 server_cookie=4de096ab7006a68e in_seq=0 out_seq=0 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.608+0000 7f33d57fa640 1 -- 172.21.15.94:0/1813538451 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f83514a800 con 0x55f834e3c390 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.608+0000 7f33d57fa640 10 client.16137 handle_command_reply: tid=0 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.608+0000 7f33fb054740 1 client.16137 shutdown 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 msgr2=0x55f8351ef560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33e8005940 tx=0x7f33e8005e40 comp rx=0 tx=0).stop 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 shutdown_connections 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f834e3c390 0x55f83525ee80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f834e1b6b0 0x55f8351ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f834e134d0 0x55f8351ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33d57fa640 0 client.16137 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:57.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 --2- 172.21.15.94:0/1813538451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f834dfaf60 0x55f8351ef560 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 >> 172.21.15.94:0/1813538451 conn(0x55f834e304e0 msgr2=0x55f8351dc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 shutdown_connections 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 1 -- 172.21.15.94:0/1813538451 wait complete. 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33fb054740 20 client.16137 trim_cache size 0 max 16384 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:43:57.615 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:43:57.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 msgr2=0x7f33d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.609+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 0x7f33d80b2000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f33e4013830 tx=0x7f33e4013c30 comp rx=0 tx=0).stop 2024-04-25T13:43:57.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 msgr2=0x7f33ec19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e9f0 tx=0x7f33dc00edf0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 shutdown_connections 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33d80afaf0 0x7f33d80b2000 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33ec108920 0x7f33ec1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33ec103c20 0x7f33ec19a7d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 --2- 172.21.15.94:0/2241849806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec1029e0 0x7f33ec19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 >> 172.21.15.94:0/2241849806 conn(0x7f33ec0fdbd0 msgr2=0x7f33ec104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 shutdown_connections 2024-04-25T13:43:57.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.610+0000 7f33f41a8640 1 -- 172.21.15.94:0/2241849806 wait complete. 2024-04-25T13:43:57.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:43:57.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:57.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 -- 172.21.15.94:0/2853095630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 msgr2=0x7fd9501036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 --2- 172.21.15.94:0/2853095630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd9501036e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd948009900 tx=0x7fd94802a410 comp rx=0 tx=0).stop 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 -- 172.21.15.94:0/2853095630 shutdown_connections 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 --2- 172.21.15.94:0/2853095630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950108920 0x7fd95010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 --2- 172.21.15.94:0/2853095630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950103c20 0x7fd950104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 --2- 172.21.15.94:0/2853095630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd9501036e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 -- 172.21.15.94:0/2853095630 >> 172.21.15.94:0/2853095630 conn(0x7fd9500fdbd0 msgr2=0x7fd950100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 -- 172.21.15.94:0/2853095630 shutdown_connections 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.735+0000 7fd95594d640 1 -- 172.21.15.94:0/2853095630 wait complete. 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 Processor -- start 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 -- start start 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950103c20 0x7fd95019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950108920 0x7fd9501a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd95010e4e0 con 0x7fd9501029e0 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd95010e360 con 0x7fd950103c20 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd95594d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd95010e660 con 0x7fd950108920 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51838/0 (socket says 172.21.15.94:51838) 2024-04-25T13:43:57.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 -- 172.21.15.94:0/3157131801 learned_addr learned my addr 172.21.15.94:0/3157131801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:57.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94e7fc640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950103c20 0x7fd95019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94f7fe640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950108920 0x7fd9501a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 -- 172.21.15.94:0/3157131801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950108920 msgr2=0x7fd9501a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950108920 0x7fd9501a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 -- 172.21.15.94:0/3157131801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950103c20 msgr2=0x7fd95019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950103c20 0x7fd95019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.736+0000 7fd94effd640 1 -- 172.21.15.94:0/3157131801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a1dc0 con 0x7fd9501029e0 2024-04-25T13:43:57.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd94effd640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd948006010 tx=0x7fd948004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94803f030 con 0x7fd9501029e0 2024-04-25T13:43:57.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd948031910 con 0x7fd9501029e0 2024-04-25T13:43:57.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a2050 con 0x7fd9501029e0 2024-04-25T13:43:57.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948035070 con 0x7fd9501029e0 2024-04-25T13:43:57.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.737+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a2540 con 0x7fd9501029e0 2024-04-25T13:43:57.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.738+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950107730 con 0x7fd9501029e0 2024-04-25T13:43:57.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.739+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd948042030 con 0x7fd9501029e0 2024-04-25T13:43:57.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.740+0000 7fd95494b640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 0x7fd9380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:57.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.740+0000 7fd94e7fc640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 0x7fd9380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:57.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.740+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd9480ff400 con 0x7fd9501029e0 2024-04-25T13:43:57.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.740+0000 7fd94e7fc640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 0x7fd9380b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9500751e0 tx=0x7fd944009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:57.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.745+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9480c78f0 con 0x7fd9501029e0 2024-04-25T13:43:57.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110990 15.37 MB/sec warmup 40 sec latency 17.227 ms 2024-04-25T13:43:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.988+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd95002db50 con 0x7fd9501029e0 2024-04-25T13:43:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.989+0000 7fd95494b640 1 -- 172.21.15.94:0/3157131801 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fd9480cbbf0 con 0x7fd9501029e0 2024-04-25T13:43:57.991 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:43:57.991 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:43:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:43:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 msgr2=0x7fd9380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 0x7fd9380b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9500751e0 tx=0x7fd944009500 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 msgr2=0x7fd950075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd948006010 tx=0x7fd948004270 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 shutdown_connections 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd9380afaf0 0x7fd9380b2000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd950108920 0x7fd9501a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd950103c20 0x7fd95019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 --2- 172.21.15.94:0/3157131801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9501029e0 0x7fd950075840 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 >> 172.21.15.94:0/3157131801 conn(0x7fd9500fdbd0 msgr2=0x7fd950104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 shutdown_connections 2024-04-25T13:43:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:57.991+0000 7fd95594d640 1 -- 172.21.15.94:0/3157131801 wait complete. 2024-04-25T13:43:58.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:43:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 -- 172.21.15.94:0/583269549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 msgr2=0x7f5390101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 --2- 172.21.15.94:0/583269549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f5390101bb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402a410 comp rx=0 tx=0).stop 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 -- 172.21.15.94:0/583269549 shutdown_connections 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 --2- 172.21.15.94:0/583269549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 0x7f539010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 --2- 172.21.15.94:0/583269549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f5390101bb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 --2- 172.21.15.94:0/583269549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 0x7f53901054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 -- 172.21.15.94:0/583269549 >> 172.21.15.94:0/583269549 conn(0x7f53900fd340 msgr2=0x7f53900ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 -- 172.21.15.94:0/583269549 shutdown_connections 2024-04-25T13:43:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.120+0000 7f5398451640 1 -- 172.21.15.94:0/583269549 wait complete. 2024-04-25T13:43:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 Processor -- start 2024-04-25T13:43:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 -- start start 2024-04-25T13:43:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 0x7f539019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 0x7f53901a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f539010f600 con 0x7f5390101740 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f539010f480 con 0x7f5390105090 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f5398451640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f539010f780 con 0x7f5390102240 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51864/0 (socket says 172.21.15.94:51864) 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 -- 172.21.15.94:0/1451328588 learned_addr learned my addr 172.21.15.94:0/1451328588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53959ec640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 0x7f539019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53969ee640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 0x7f53901a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 -- 172.21.15.94:0/1451328588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 msgr2=0x7f539019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 0x7f539019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 -- 172.21.15.94:0/1451328588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 msgr2=0x7f53901a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 0x7f53901a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.121+0000 7f53961ed640 1 -- 172.21.15.94:0/1451328588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a1d80 con 0x7f5390101740 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f53961ed640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f538800e9f0 tx=0x7f538800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f538f7fe640 1 -- 172.21.15.94:0/1451328588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538800ccc0 con 0x7f5390101740 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f538f7fe640 1 -- 172.21.15.94:0/1451328588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53880035f0 con 0x7f5390101740 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a2070 con 0x7f5390101740 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f538f7fe640 1 -- 172.21.15.94:0/1451328588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538800ce60 con 0x7f5390101740 2024-04-25T13:43:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.122+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a25c0 con 0x7f5390101740 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 Processor -- start 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f538f7fe640 1 -- 172.21.15.94:0/1451328588 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5388003aa0 con 0x7f5390101740 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 -- start start 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 0x560f4e6da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f4e639d80 con 0x560f4e3126b0 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f4e5f3a20 con 0x560f4e30a4d0 2024-04-25T13:43:58.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f539f3d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f4e635d40 con 0x560f4e333390 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f53959ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f53969ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f53961ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 0x560f4e6da1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f53969ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39120/0 (socket says 172.21.15.94:39120) 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f53969ee640 1 -- 172.21.15.94:0/1241593584 learned_addr learned my addr 172.21.15.94:0/1241593584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.125+0000 7f538f7fe640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 0x7f53780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 -- 172.21.15.94:0/1241593584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 msgr2=0x560f4e6d9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 -- 172.21.15.94:0/1241593584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 msgr2=0x560f4e6dae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6dae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 -- 172.21.15.94:0/1241593584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4e5fec30 con 0x560f4e30a4d0 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53959ec640 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6dae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53969ee640 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53969ee640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 0x7f53780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f538f7fe640 1 -- 172.21.15.94:0/1451328588 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53880def20 con 0x7f5390101740 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53961ed640 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 0x560f4e6da1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53880dc950 tx=0x7f5388021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d6f7235e9e0178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f53969ee640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 0x7f53780b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x560f4e6dac70 tx=0x7f5380004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f538cbfe640 1 -- 172.21.15.94:0/1241593584 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388091030 con 0x560f4e30a4d0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f538cbfe640 1 -- 172.21.15.94:0/1241593584 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53880a6e70 con 0x560f4e30a4d0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.126+0000 7f538cbfe640 1 -- 172.21.15.94:0/1241593584 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53880a6ad0 con 0x560f4e30a4d0 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 -- 172.21.15.94:0/1241593584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 msgr2=0x560f4e6da1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 0x560f4e6da1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53880dc950 tx=0x7f5388021000 comp rx=0 tx=0).stop 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 -- 172.21.15.94:0/1241593584 shutdown_connections 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6dae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e30a4d0 0x560f4e6da1a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1241593584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 -- 172.21.15.94:0/1241593584 >> 172.21.15.94:0/1241593584 conn(0x560f4e3274e0 msgr2=0x560f4e6d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 -- 172.21.15.94:0/1241593584 shutdown_connections 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.127+0000 7f539f3d6740 1 -- 172.21.15.94:0/1241593584 wait complete. 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 -- start start 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 Processor -- start 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e3126b0 0x560f4e6e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f4e6db8e0 con 0x560f4e30a4d0 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f4e5c0dc0 con 0x560f4e3126b0 2024-04-25T13:43:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f4e6dba60 con 0x560f4e333390 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f539f3d6740 10 client.0 _ll_register_callbacks cb 0x560f4e5dc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51868/0 (socket says 172.21.15.94:51868) 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 -- 172.21.15.94:0/3306589338 learned_addr learned my addr 172.21.15.94:0/3306589338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53959ec640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e3126b0 0x560f4e6e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53969ee640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 -- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 msgr2=0x560f4e6ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6ead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 -- 172.21.15.94:0/3306589338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e3126b0 msgr2=0x560f4e6e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e3126b0 0x560f4e6e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 -- 172.21.15.94:0/3306589338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4e34b3f0 con 0x560f4e30a4d0 2024-04-25T13:43:58.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53969ee640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f53961ed640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f53880dc9d0 tx=0x7f53880df220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa45a2b7fa8e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f5375ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.128+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388091050 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53880a6e70 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53880a75c0 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5388013190 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 10 client.16155 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f4e4157d0 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53880a6e70 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f53880afd30 con 0x560f4e30a4d0 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 10 client.16155 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 10 client.16155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 20 client.16155 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2f1f60 0x560f4e6f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 4 client.16155 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4e647990 con 0x560f4e2f1f60 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.129+0000 7f53959ec640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2f1f60 0x560f4e6f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.130+0000 7f53959ec640 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2f1f60 0x560f4e6f1190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8b7555a15af7adc server_cookie=804573269e024bbf in_seq=0 out_seq=0 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.130+0000 7f5375ffb640 10 client.16155 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.130+0000 7f5375ffb640 1 -- 172.21.15.94:0/3306589338 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f4e647990 con 0x560f4e2f1f60 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.130+0000 7f5375ffb640 10 client.16155 handle_command_reply: tid=0 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 client.16155 shutdown 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 msgr2=0x560f4e6e38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f53880dc9d0 tx=0x7f53880df220 comp rx=0 tx=0).stop 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 shutdown_connections 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2f1f60 0x560f4e6f1190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e333390 0x560f4e6ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e3126b0 0x560f4e6e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 --2- 172.21.15.94:0/3306589338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e30a4d0 0x560f4e6e38a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f5375ffb640 0 client.16155 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 >> 172.21.15.94:0/3306589338 conn(0x560f4e3274e0 msgr2=0x560f4e6d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 shutdown_connections 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 1 -- 172.21.15.94:0/3306589338 wait complete. 2024-04-25T13:43:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.131+0000 7f539f3d6740 20 client.16155 trim_cache size 0 max 16384 2024-04-25T13:43:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.144+0000 7f539f3d6740 1 Processor -- start 2024-04-25T13:43:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.144+0000 7f539f3d6740 1 -- start start 2024-04-25T13:43:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e6f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53901a93d0 con 0x560f4e3126b0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53901aaf10 con 0x560f4e333390 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f539f3d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53901a9aa0 con 0x560f4e30a4d0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51874/0 (socket says 172.21.15.94:51874) 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 -- 172.21.15.94:0/877896316 learned_addr learned my addr 172.21.15.94:0/877896316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53961ed640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53969ee640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e6f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 -- 172.21.15.94:0/877896316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 msgr2=0x560f4e6e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 -- 172.21.15.94:0/877896316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 msgr2=0x560f4e6f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e6f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 -- 172.21.15.94:0/877896316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4e6df070 con 0x560f4e3126b0 2024-04-25T13:43:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53969ee640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e6f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f53959ec640 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5384033000 tx=0x7f5384031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f969eb1380e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f5377fff640 1 -- 172.21.15.94:0/877896316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384035040 con 0x560f4e3126b0 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f5377fff640 1 -- 172.21.15.94:0/877896316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538404b070 con 0x560f4e3126b0 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.145+0000 7f5377fff640 1 -- 172.21.15.94:0/877896316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53840044e0 con 0x560f4e3126b0 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 -- 172.21.15.94:0/877896316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 msgr2=0x560f4e6f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5384033000 tx=0x7f5384031870 comp rx=0 tx=0).stop 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 -- 172.21.15.94:0/877896316 shutdown_connections 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e6f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6f58c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 --2- 172.21.15.94:0/877896316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 -- 172.21.15.94:0/877896316 >> 172.21.15.94:0/877896316 conn(0x560f4e3274e0 msgr2=0x560f4e6e3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 -- 172.21.15.94:0/877896316 shutdown_connections 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.146+0000 7f539f3d6740 1 -- 172.21.15.94:0/877896316 wait complete. 2024-04-25T13:43:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 -- start start 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 Processor -- start 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e757640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53901aa840 con 0x560f4e3126b0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53901aa170 con 0x560f4e333390 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53901a73b0 con 0x560f4e30a4d0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f539f3d6740 10 client.0 _ll_register_callbacks cb 0x7f5384009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51882/0 (socket says 172.21.15.94:51882) 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 -- 172.21.15.94:0/1464515912 learned_addr learned my addr 172.21.15.94:0/1464515912 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53969ee640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e757640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53961ed640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 -- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 msgr2=0x560f4e6e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 -- 172.21.15.94:0/1464515912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 msgr2=0x560f4e757640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e757640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 -- 172.21.15.94:0/1464515912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4e757c30 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f53959ec640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5384005ab0 tx=0x7f538402abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ec2cbbfb2b88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.147+0000 7f5375ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384054070 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53840467e0 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538404b070 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5384060050 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 10 client.16161 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f4e758eb0 con 0x560f4e3126b0 2024-04-25T13:43:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538404b070 con 0x560f4e3126b0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5384050400 con 0x560f4e3126b0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 10 client.16161 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 10 client.16161 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 20 client.16161 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2c73a0 0x560f4e75b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 4 client.16161 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f4e641800 con 0x560f4e2c73a0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.148+0000 7f53961ed640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2c73a0 0x560f4e75b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f5375ffb640 10 client.16161 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f53961ed640 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2c73a0 0x560f4e75b8d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bb5dc07922b16fc server_cookie=2554ab1d1e0afda in_seq=0 out_seq=0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f5375ffb640 1 -- 172.21.15.94:0/1464515912 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f4e641800 con 0x560f4e2c73a0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f5375ffb640 10 client.16161 handle_command_reply: tid=0 2024-04-25T13:43:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 client.16161 shutdown 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 msgr2=0x560f4e6e9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5384005ab0 tx=0x7f538402abc0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 shutdown_connections 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560f4e2c73a0 0x560f4e75b8d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f4e333390 0x560f4e757640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f4e3126b0 0x560f4e6e9c20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 --2- 172.21.15.94:0/1464515912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f4e30a4d0 0x560f4e6e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 >> 172.21.15.94:0/1464515912 conn(0x560f4e3274e0 msgr2=0x560f4e6d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f5375ffb640 0 client.16161 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 shutdown_connections 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 1 -- 172.21.15.94:0/1464515912 wait complete. 2024-04-25T13:43:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.149+0000 7f539f3d6740 20 client.16161 trim_cache size 0 max 16384 2024-04-25T13:43:58.157 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 msgr2=0x7f53780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 0x7f53780b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x560f4e6dac70 tx=0x7f5380004a40 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 msgr2=0x7f539019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f538800e9f0 tx=0x7f538800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 shutdown_connections 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53780afaf0 0x7f53780b2000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5390105090 0x7f53901a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5390102240 0x7f539019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 --2- 172.21.15.94:0/1451328588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390101740 0x7f539019a240 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 >> 172.21.15.94:0/1451328588 conn(0x7f53900fd340 msgr2=0x7f5390105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 shutdown_connections 2024-04-25T13:43:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.150+0000 7f5398451640 1 -- 172.21.15.94:0/1451328588 wait complete. 2024-04-25T13:43:58.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:43:58.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 -- 172.21.15.94:0/1031759869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e01050b0 msgr2=0x7ff2e01054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1031759869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e01050b0 0x7ff2e01054c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009900 tx=0x7ff2dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 -- 172.21.15.94:0/1031759869 shutdown_connections 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1031759869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1031759869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e0101760 0x7ff2e0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1031759869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e01050b0 0x7ff2e01054c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 -- 172.21.15.94:0/1031759869 >> 172.21.15.94:0/1031759869 conn(0x7ff2e00fd380 msgr2=0x7ff2e00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 -- 172.21.15.94:0/1031759869 shutdown_connections 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.274+0000 7ff2e8138640 1 -- 172.21.15.94:0/1031759869 wait complete. 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 Processor -- start 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 -- start start 2024-04-25T13:43:58.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e01050b0 0x7ff2e01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010f620 con 0x7ff2e0101760 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010f4a0 con 0x7ff2e01050b0 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e8138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010f7a0 con 0x7ff2e0102260 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51896/0 (socket says 172.21.15.94:51896) 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/1574752272 learned_addr learned my addr 172.21.15.94:0/1574752272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.275+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e01050b0 0x7ff2e01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/1574752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 msgr2=0x7ff2e019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/1574752272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e01050b0 msgr2=0x7ff2e01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e01050b0 0x7ff2e01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/1574752272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01a1d60 con 0x7ff2e0101760 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc006010 tx=0x7ff2dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2d6ffd640 1 -- 172.21.15.94:0/1574752272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc03f030 con 0x7ff2e0101760 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2d6ffd640 1 -- 172.21.15.94:0/1574752272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc031910 con 0x7ff2e0101760 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01a1ff0 con 0x7ff2e0101760 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2d6ffd640 1 -- 172.21.15.94:0/1574752272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc035070 con 0x7ff2e0101760 2024-04-25T13:43:58.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.276+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a2450 con 0x7ff2e0101760 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 Processor -- start 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 -- start start 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2d6ffd640 1 -- 172.21.15.94:0/1574752272 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff2dc042030 con 0x7ff2e0101760 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054361c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054362e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5600542c1d80 con 0x560053f924d0 2024-04-25T13:43:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56005427ba20 con 0x560053fbb390 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2eefec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5600542bdd40 con 0x560053f9a6b0 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51908/0 (socket says 172.21.15.94:51908) 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2254433493 learned_addr learned my addr 172.21.15.94:0/2254433493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054361c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054362e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2254433493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 msgr2=0x560054362e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054362e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2254433493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 msgr2=0x560054361c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054361c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.279+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2254433493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560054286c30 con 0x560053f924d0 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2d6ffd640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 0x7ff2cc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc107160 tx=0x7ff2dc107250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db7cec145ef38fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 0x7ff2cc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2d6ffd640 1 -- 172.21.15.94:0/1574752272 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff2dc0fef70 con 0x7ff2e0101760 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2cbfff640 1 -- 172.21.15.94:0/2254433493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0c3320 con 0x560053f924d0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2cbfff640 1 -- 172.21.15.94:0/2254433493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0c6f70 con 0x560053f924d0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.280+0000 7ff2cbfff640 1 -- 172.21.15.94:0/2254433493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0c6bd0 con 0x560053f924d0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- 172.21.15.94:0/2254433493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 msgr2=0x5600543621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 0x7ff2cc0b2080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560054362c70 tx=0x7ff2d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc107160 tx=0x7ff2dc107250 comp rx=0 tx=0).stop 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- 172.21.15.94:0/2254433493 shutdown_connections 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054362e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x5600543621a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2254433493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054361c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- 172.21.15.94:0/2254433493 >> 172.21.15.94:0/2254433493 conn(0x560053faf4e0 msgr2=0x560054359020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- 172.21.15.94:0/2254433493 shutdown_connections 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- 172.21.15.94:0/2254433493 wait complete. 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 -- start start 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.281+0000 7ff2eefec740 1 Processor -- start 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053f9a6b0 0x56005436bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053fbb390 0x560054372b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560054363700 con 0x560053f924d0 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560054248dc0 con 0x560053f9a6b0 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560054363880 con 0x560053fbb390 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2eefec740 10 client.0 _ll_register_callbacks cb 0x560054264600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51918/0 (socket says 172.21.15.94:51918) 2024-04-25T13:43:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2865448737 learned_addr learned my addr 172.21.15.94:0/2865448737 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053f9a6b0 0x56005436bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053fbb390 0x560054372b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2865448737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053fbb390 msgr2=0x560054372b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053fbb390 0x560054372b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053f9a6b0 msgr2=0x56005436bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053f9a6b0 0x56005436bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2865448737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560053fd33f0 con 0x560053f924d0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.282+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0fca20 tx=0x7ff2dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58396389de87c4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc049030 con 0x560053f924d0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff2dc003d30 con 0x560053f924d0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc0c6f70 con 0x560053f924d0 2024-04-25T13:43:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc0c6db0 con 0x560053f924d0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 10 client.16173 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56005409d7d0 con 0x560053f924d0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0c6f70 con 0x560053f924d0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff2dc102140 con 0x560053f924d0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 10 client.16173 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 10 client.16173 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 20 client.16173 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f79f60 0x560054378f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 4 client.16173 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600542cf990 con 0x560053f79f60 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.283+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f79f60 0x560054378f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.284+0000 7ff2c97fa640 10 client.16173 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.284+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f79f60 0x560054378f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43086ec7b1216786 server_cookie=9fe416bcef93b92f in_seq=0 out_seq=0 2024-04-25T13:43:58.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.284+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2865448737 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600542cf990 con 0x560053f79f60 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.284+0000 7ff2c97fa640 10 client.16173 handle_command_reply: tid=0 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 client.16173 shutdown 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 msgr2=0x56005436b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0fca20 tx=0x7ff2dc009260 comp rx=0 tx=0).stop 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 shutdown_connections 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f79f60 0x560054378f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053fbb390 0x560054372b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053f9a6b0 0x56005436bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2865448737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x56005436b680 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 >> 172.21.15.94:0/2865448737 conn(0x560053faf4e0 msgr2=0x56005435bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2c97fa640 0 client.16173 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 shutdown_connections 2024-04-25T13:43:58.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 1 -- 172.21.15.94:0/2865448737 wait complete. 2024-04-25T13:43:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.285+0000 7ff2eefec740 20 client.16173 trim_cache size 0 max 16384 2024-04-25T13:43:58.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.299+0000 7ff2eefec740 1 Processor -- start 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.299+0000 7ff2eefec740 1 -- start start 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f924d0 0x560054366bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054358ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01a9290 con 0x560053f9a6b0 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01aadd0 con 0x560053fbb390 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2eefec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01a9960 con 0x560053f924d0 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51920/0 (socket says 172.21.15.94:51920) 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 -- 172.21.15.94:0/1761272032 learned_addr learned my addr 172.21.15.94:0/1761272032 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f924d0 0x560054366bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054358ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 -- 172.21.15.94:0/1761272032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f924d0 msgr2=0x560054366bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f924d0 0x560054366bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 -- 172.21.15.94:0/1761272032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 msgr2=0x560054358ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054358ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 -- 172.21.15.94:0/1761272032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600543616d0 con 0x560053f9a6b0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054358ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff2e019b2c0 tx=0x7ff2d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe549398cbfdcc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.300+0000 7ff2cb7fe640 1 -- 172.21.15.94:0/1761272032 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004250 con 0x560053f9a6b0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2cb7fe640 1 -- 172.21.15.94:0/1761272032 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00043f0 con 0x560053f9a6b0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2cb7fe640 1 -- 172.21.15.94:0/1761272032 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00035f0 con 0x560053f9a6b0 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 -- 172.21.15.94:0/1761272032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 msgr2=0x5600543589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff2e019b2c0 tx=0x7ff2d0005f90 comp rx=0 tx=0).stop 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 -- 172.21.15.94:0/1761272032 shutdown_connections 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x560054358ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f9a6b0 0x5600543589a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 --2- 172.21.15.94:0/1761272032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f924d0 0x560054366bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 -- 172.21.15.94:0/1761272032 >> 172.21.15.94:0/1761272032 conn(0x560053faf4e0 msgr2=0x560054369ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 -- 172.21.15.94:0/1761272032 shutdown_connections 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.301+0000 7ff2eefec740 1 -- 172.21.15.94:0/1761272032 wait complete. 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 -- start start 2024-04-25T13:43:58.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 Processor -- start 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054372170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x56005436e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01aa700 con 0x560053f924d0 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01aa030 con 0x560053fbb390 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01a7270 con 0x560053f9a6b0 2024-04-25T13:43:58.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2eefec740 10 client.0 _ll_register_callbacks cb 0x7ff2d8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2e5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51932/0 (socket says 172.21.15.94:51932) 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2923189921 learned_addr learned my addr 172.21.15.94:0/2923189921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054372170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.302+0000 7ff2e66d5640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x56005436e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2923189921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 msgr2=0x560054372170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054372170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 msgr2=0x56005436e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x56005436e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 -- 172.21.15.94:0/2923189921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005436f000 con 0x560053f924d0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2e5ed4640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0fcd40 tx=0x7ff2dc107250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21dee07ad775826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc049040 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff2dc102560 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00043f0 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc0c7230 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2eefec740 10 client.16179 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56005436f000 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00043f0 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff2dc102140 con 0x560053f924d0 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.303+0000 7ff2eefec740 10 client.16179 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 10 client.16179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 20 client.16179 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f4f3a0 0x5600543e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 4 client.16179 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:58.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600542c9800 con 0x560053f4f3a0 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f4f3a0 0x5600543e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2c97fa640 10 client.16179 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2e56d3640 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f4f3a0 0x5600543e2670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=898b74191ebab2db server_cookie=740ca03c81a70645 in_seq=0 out_seq=0 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2c97fa640 1 -- 172.21.15.94:0/2923189921 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600542c9800 con 0x560053f4f3a0 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2c97fa640 10 client.16179 handle_command_reply: tid=0 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 client.16179 shutdown 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 msgr2=0x560054371c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0fcd40 tx=0x7ff2dc107250 comp rx=0 tx=0).stop 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 shutdown_connections 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.304+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560053f4f3a0 0x5600543e2670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560053fbb390 0x56005436e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560053f9a6b0 0x560054372170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 --2- 172.21.15.94:0/2923189921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560053f924d0 0x560054371c30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2c97fa640 0 client.16179 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:43:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 >> 172.21.15.94:0/2923189921 conn(0x560053faf4e0 msgr2=0x56005435be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 shutdown_connections 2024-04-25T13:43:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 1 -- 172.21.15.94:0/2923189921 wait complete. 2024-04-25T13:43:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2eefec740 20 client.16179 trim_cache size 0 max 16384 2024-04-25T13:43:58.312 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 msgr2=0x7ff2cc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 0x7ff2cc0b2080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560054362c70 tx=0x7ff2d8009500 comp rx=0 tx=0).stop 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 msgr2=0x7ff2e019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc006010 tx=0x7ff2dc004270 comp rx=0 tx=0).stop 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 shutdown_connections 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2cc0afb70 0x7ff2cc0b2080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2e01050b0 0x7ff2e01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102260 0x7ff2e019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 --2- 172.21.15.94:0/1574752272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2e0101760 0x7ff2e019a220 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 >> 172.21.15.94:0/1574752272 conn(0x7ff2e00fd380 msgr2=0x7ff2e0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 shutdown_connections 2024-04-25T13:43:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.305+0000 7ff2e8138640 1 -- 172.21.15.94:0/1574752272 wait complete. 2024-04-25T13:43:58.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:58 smithi094 ceph-mon[29195]: pgmap v411: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-04-25T13:43:58.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:43:58 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3157131801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:58.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 -- 172.21.15.94:0/88529315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60109780 msgr2=0x7f6b6010bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 --2- 172.21.15.94:0/88529315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60109780 0x7f6b6010bc90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009ca0 tx=0x7f6b5802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 -- 172.21.15.94:0/88529315 shutdown_connections 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 --2- 172.21.15.94:0/88529315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60109780 0x7f6b6010bc90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 --2- 172.21.15.94:0/88529315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60106c10 0x7f6b601090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 --2- 172.21.15.94:0/88529315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60101e30 0x7f6b60102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 -- 172.21.15.94:0/88529315 >> 172.21.15.94:0/88529315 conn(0x7f6b600fd380 msgr2=0x7f6b600ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 -- 172.21.15.94:0/88529315 shutdown_connections 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.431+0000 7f6b65557640 1 -- 172.21.15.94:0/88529315 wait complete. 2024-04-25T13:43:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 Processor -- start 2024-04-25T13:43:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 -- start start 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60101e30 0x7f6b6019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60109780 0x7f6b601a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6010f610 con 0x7f6b60106c10 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6010f490 con 0x7f6b60109780 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.432+0000 7f6b65557640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6010f790 con 0x7f6b60101e30 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51946/0 (socket says 172.21.15.94:51946) 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 -- 172.21.15.94:0/2225493872 learned_addr learned my addr 172.21.15.94:0/2225493872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60109780 0x7f6b601a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60101e30 0x7f6b6019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 -- 172.21.15.94:0/2225493872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60101e30 msgr2=0x7f6b6019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60101e30 0x7f6b6019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 -- 172.21.15.94:0/2225493872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60109780 msgr2=0x7f6b601a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60109780 0x7f6b601a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 -- 172.21.15.94:0/2225493872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b601a1cc0 con 0x7f6b60106c10 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d770 tx=0x7f6b5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b47fff640 1 -- 172.21.15.94:0/2225493872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54017070 con 0x7f6b60106c10 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601a1fb0 con 0x7f6b60106c10 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b47fff640 1 -- 172.21.15.94:0/2225493872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b540054d0 con 0x7f6b60106c10 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b47fff640 1 -- 172.21.15.94:0/2225493872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004c30 con 0x7f6b60106c10 2024-04-25T13:43:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.433+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a2410 con 0x7f6b60106c10 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 Processor -- start 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 -- start start 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a827031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a82703e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a82662d80 con 0x556a8235c390 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556a8261ca20 con 0x556a8233b6b0 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b6c408740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556a8265ed40 con 0x556a823334d0 2024-04-25T13:43:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b47fff640 1 -- 172.21.15.94:0/2225493872 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6b54010460 con 0x7f6b60106c10 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51962/0 (socket says 172.21.15.94:51962) 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/4154922697 learned_addr learned my addr 172.21.15.94:0/4154922697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a82703e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5effd640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a827031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/4154922697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 msgr2=0x556a827031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a827031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/4154922697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 msgr2=0x556a82703e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a82703e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/4154922697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a82627c30 con 0x556a8235c390 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5effd640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a827031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.436+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6b58035000 tx=0x7f6b580098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb42f434a48665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b457fa640 1 -- 172.21.15.94:0/4154922697 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58031710 con 0x556a8235c390 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b457fa640 1 -- 172.21.15.94:0/4154922697 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b58036030 con 0x556a8235c390 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b457fa640 1 -- 172.21.15.94:0/4154922697 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5802f660 con 0x556a8235c390 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b47fff640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 0x7f6b480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 0x7f6b480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.437+0000 7f6b47fff640 1 -- 172.21.15.94:0/2225493872 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b540de680 con 0x7f6b60106c10 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- 172.21.15.94:0/4154922697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 msgr2=0x556a82702c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 0x7f6b480b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x556a82702940 tx=0x7f6b5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6b58035000 tx=0x7f6b580098b0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- 172.21.15.94:0/4154922697 shutdown_connections 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a82703e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a827031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- 172.21.15.94:0/4154922697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82702c20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- 172.21.15.94:0/4154922697 >> 172.21.15.94:0/4154922697 conn(0x556a823504e0 msgr2=0x556a826fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- 172.21.15.94:0/4154922697 shutdown_connections 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- 172.21.15.94:0/4154922697 wait complete. 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- start start 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 Processor -- start 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a8270c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a8270cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a82704760 con 0x556a8235c390 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556a825e9dc0 con 0x556a8233b6b0 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556a827048e0 con 0x556a823334d0 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b6c408740 10 client.0 _ll_register_callbacks cb 0x556a82605600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.438+0000 7f6b5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51966/0 (socket says 172.21.15.94:51966) 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/1922696772 learned_addr learned my addr 172.21.15.94:0/1922696772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a8270c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a8270cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 msgr2=0x556a8270c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a8270c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/1922696772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 msgr2=0x556a8270cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a8270cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 -- 172.21.15.94:0/1922696772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a823743f0 con 0x556a8235c390 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6b580094e0 tx=0x7f6b5802f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8605a369a0428e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5804a040 con 0x556a8235c390 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b58003b70 con 0x556a8235c390 2024-04-25T13:43:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b58036030 con 0x556a8235c390 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b58003d90 con 0x556a8235c390 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b6c408740 10 client.16191 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a8243e7d0 con 0x556a8235c390 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.439+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b58036030 con 0x556a8235c390 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b5804d030 con 0x556a8235c390 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 10 client.16191 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 20 client.16191 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a8231af60 0x556a8271a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 4 client.16191 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a82670990 con 0x556a8231af60 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b5effd640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a8231af60 0x556a8271a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b42ff5640 10 client.16191 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.440+0000 7f6b5effd640 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a8231af60 0x556a8271a0e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cde37c0eec5363c server_cookie=a950d48d4e23cf6 in_seq=0 out_seq=0 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b42ff5640 1 -- 172.21.15.94:0/1922696772 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a82670990 con 0x556a8231af60 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b42ff5640 10 client.16191 handle_command_reply: tid=0 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 client.16191 shutdown 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 msgr2=0x556a82713d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6b580094e0 tx=0x7f6b5802f430 comp rx=0 tx=0).stop 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 shutdown_connections 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a8231af60 0x556a8271a0e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a8235c390 0x556a82713d00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8233b6b0 0x556a8270cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b42ff5640 0 client.16191 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 --2- 172.21.15.94:0/1922696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a823334d0 0x556a8270c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 >> 172.21.15.94:0/1922696772 conn(0x556a823504e0 msgr2=0x556a826f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 shutdown_connections 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 1 -- 172.21.15.94:0/1922696772 wait complete. 2024-04-25T13:43:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.441+0000 7f6b6c408740 20 client.16191 trim_cache size 0 max 16384 2024-04-25T13:43:58.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 Processor -- start 2024-04-25T13:43:58.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 -- start start 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82713c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a8270b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a90d0 con 0x556a823334d0 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601aac10 con 0x556a8235c390 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b6c408740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a97a0 con 0x556a8233b6b0 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51976/0 (socket says 172.21.15.94:51976) 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b5effd640 1 -- 172.21.15.94:0/2019261974 learned_addr learned my addr 172.21.15.94:0/2019261974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b5e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82713c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.455+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a8270b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 -- 172.21.15.94:0/2019261974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 msgr2=0x556a82713c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82713c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 -- 172.21.15.94:0/2019261974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 msgr2=0x556a8270b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a8270b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 -- 172.21.15.94:0/2019261974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a826f5500 con 0x556a823334d0 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b5effd640 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6b50005fd0 tx=0x7f6b50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7e585623ba8ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b44ff9640 1 -- 172.21.15.94:0/2019261974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50004020 con 0x556a823334d0 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b44ff9640 1 -- 172.21.15.94:0/2019261974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5001fca0 con 0x556a823334d0 2024-04-25T13:43:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.456+0000 7f6b44ff9640 1 -- 172.21.15.94:0/2019261974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500286c0 con 0x556a823334d0 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- 172.21.15.94:0/2019261974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 msgr2=0x556a8270ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6b50005fd0 tx=0x7f6b50014000 comp rx=0 tx=0).stop 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- 172.21.15.94:0/2019261974 shutdown_connections 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a8270b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82713c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 --2- 172.21.15.94:0/2019261974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a8270ae80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- 172.21.15.94:0/2019261974 >> 172.21.15.94:0/2019261974 conn(0x556a823504e0 msgr2=0x556a82707740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- 172.21.15.94:0/2019261974 shutdown_connections 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- 172.21.15.94:0/2019261974 wait complete. 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 -- start start 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.457+0000 7f6b6c408740 1 Processor -- start 2024-04-25T13:43:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82710f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a82780480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601aa540 con 0x556a823334d0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a9e70 con 0x556a8235c390 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a70b0 con 0x556a8233b6b0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 10 client.0 _ll_register_callbacks cb 0x7f6b58009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51990/0 (socket says 172.21.15.94:51990) 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 -- 172.21.15.94:0/3661123516 learned_addr learned my addr 172.21.15.94:0/3661123516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82710f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a82780480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 -- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 msgr2=0x556a82710f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82710f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 -- 172.21.15.94:0/3661123516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 msgr2=0x556a82780480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a82780480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 -- 172.21.15.94:0/3661123516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a82711580 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5f7fe640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a82780480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b5effd640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b690 tx=0x7f6b5001fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3b67fea5649ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50036020 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b5001fa00 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.458+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5001fca0 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b50041050 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 10 client.16197 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a82781a10 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b5001fca0 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b50028ce0 con 0x556a823334d0 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 10 client.16197 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 10 client.16197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:43:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 20 client.16197 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a822f03a0 0x556a827844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 4 client.16197 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a8266a800 con 0x556a822f03a0 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.459+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a822f03a0 0x556a827844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b5e7fc640 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a822f03a0 0x556a827844b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f1a5fa97c93c50d server_cookie=c4734834c74916ac in_seq=0 out_seq=0 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b42ff5640 10 client.16197 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b42ff5640 1 -- 172.21.15.94:0/3661123516 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a8266a800 con 0x556a822f03a0 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b42ff5640 10 client.16197 handle_command_reply: tid=0 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 client.16197 shutdown 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 msgr2=0x556a82710a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b690 tx=0x7f6b5001fed0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 shutdown_connections 2024-04-25T13:43:58.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a822f03a0 0x556a827844b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a8235c390 0x556a82780480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a8233b6b0 0x556a82710f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 --2- 172.21.15.94:0/3661123516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a823334d0 0x556a82710a30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b42ff5640 0 client.16197 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 >> 172.21.15.94:0/3661123516 conn(0x556a823504e0 msgr2=0x556a8270c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 shutdown_connections 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 1 -- 172.21.15.94:0/3661123516 wait complete. 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.460+0000 7f6b6c408740 20 client.16197 trim_cache size 0 max 16384 2024-04-25T13:43:58.467 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 msgr2=0x7f6b480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 0x7f6b480b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x556a82702940 tx=0x7f6b5000b410 comp rx=0 tx=0).stop 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 msgr2=0x7f6b6019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d770 tx=0x7f6b5400dc30 comp rx=0 tx=0).stop 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 shutdown_connections 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b480afaf0 0x7f6b480b2000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b60109780 0x7f6b601a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b60106c10 0x7f6b6019a6c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 --2- 172.21.15.94:0/2225493872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b60101e30 0x7f6b6019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 >> 172.21.15.94:0/2225493872 conn(0x7f6b600fd380 msgr2=0x7f6b60107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 shutdown_connections 2024-04-25T13:43:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.461+0000 7f6b65557640 1 -- 172.21.15.94:0/2225493872 wait complete. 2024-04-25T13:43:58.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:43:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:58 smithi138 ceph-mon[32977]: pgmap v411: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-04-25T13:43:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:43:58 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3157131801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 -- 172.21.15.94:0/598762347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 msgr2=0x7f5f60103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 --2- 172.21.15.94:0/598762347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60103530 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009900 tx=0x7f5f5002a410 comp rx=0 tx=0).stop 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 -- 172.21.15.94:0/598762347 shutdown_connections 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 --2- 172.21.15.94:0/598762347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 0x7f5f6010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 --2- 172.21.15.94:0/598762347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 --2- 172.21.15.94:0/598762347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60103530 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 -- 172.21.15.94:0/598762347 >> 172.21.15.94:0/598762347 conn(0x7f5f600fda40 msgr2=0x7f5f600ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 -- 172.21.15.94:0/598762347 shutdown_connections 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.584+0000 7f5f656d6640 1 -- 172.21.15.94:0/598762347 wait complete. 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 Processor -- start 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 -- start start 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 0x7f5f601a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6010e330 con 0x7f5f60102830 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6010e1b0 con 0x7f5f60103a70 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f656d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6010e4b0 con 0x7f5f60108770 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52012/0 (socket says 172.21.15.94:52012) 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f5ffff640 1 -- 172.21.15.94:0/2792416157 learned_addr learned my addr 172.21.15.94:0/2792416157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.585+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 0x7f5f601a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 -- 172.21.15.94:0/2792416157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 msgr2=0x7f5f601a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 0x7f5f601a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 -- 172.21.15.94:0/2792416157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 msgr2=0x7f5f60199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 -- 172.21.15.94:0/2792416157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f601a0a90 con 0x7f5f60102830 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5f50006010 tx=0x7f5f50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5d7fa640 1 -- 172.21.15.94:0/2792416157 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5003f030 con 0x7f5f60102830 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5d7fa640 1 -- 172.21.15.94:0/2792416157 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f50003aa0 con 0x7f5f60102830 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f601a0d20 con 0x7f5f60102830 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f5d7fa640 1 -- 172.21.15.94:0/2792416157 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50035070 con 0x7f5f60102830 2024-04-25T13:43:58.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.586+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f601a11c0 con 0x7f5f60102830 2024-04-25T13:43:58.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5d7fa640 1 -- 172.21.15.94:0/2792416157 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5f50042030 con 0x7f5f60102830 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 Processor -- start 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 -- start start 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x565461626bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461627e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565461588d80 con 0x565461282390 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565461542a20 con 0x5654612594d0 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565461584d40 con 0x5654612616b0 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f64ed5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x565461626bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:48362/0 (socket says 172.21.15.94:48362) 2024-04-25T13:43:58.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f64ed5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x565461626bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52014/0 (socket says 172.21.15.94:52014) 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5ffff640 1 -- 172.21.15.94:0/3652113850 learned_addr learned my addr 172.21.15.94:0/3652113850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461627e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.589+0000 7f5f5d7fa640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 0x7f5f480b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 0x7f5f480b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3652113850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 msgr2=0x565461627170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3652113850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 msgr2=0x565461626bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x565461626bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3652113850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546154dc30 con 0x5654612616b0 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5d7fa640 1 -- 172.21.15.94:0/2792416157 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5f500ff240 con 0x7f5f60102830 2024-04-25T13:43:58.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461627e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fa0 tx=0x7f5f540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a862269ae62333b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 0x7f5f480b1ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x565461627c40 tx=0x7f5f58004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f467fc640 1 -- 172.21.15.94:0/3652113850 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004030 con 0x5654612616b0 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f467fc640 1 -- 172.21.15.94:0/3652113850 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f540054d0 con 0x5654612616b0 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.590+0000 7f5f467fc640 1 -- 172.21.15.94:0/3652113850 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004c80 con 0x5654612616b0 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3652113850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 msgr2=0x565461627e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461627e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fa0 tx=0x7f5f540118d0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3652113850 shutdown_connections 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461627e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461627170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3652113850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x565461626bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3652113850 >> 172.21.15.94:0/3652113850 conn(0x5654612764e0 msgr2=0x565461622400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3652113850 shutdown_connections 2024-04-25T13:43:58.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3652113850 wait complete. 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.591+0000 7f5f6d7ef740 1 -- start start 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 Processor -- start 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x565461630970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461637e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565461628910 con 0x5654612616b0 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56546150fdc0 con 0x565461282390 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565461628a90 con 0x5654612594d0 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f6d7ef740 10 client.0 _ll_register_callbacks cb 0x56546152b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52026/0 (socket says 172.21.15.94:52026) 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/2218256109 learned_addr learned my addr 172.21.15.94:0/2218256109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x565461630970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461637e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/2218256109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 msgr2=0x565461630970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x565461630970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 msgr2=0x565461637e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461637e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/2218256109 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56546129a3f0 con 0x5654612616b0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461637e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.592+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x565461630970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f5401b040 tx=0x7f5f5400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922a0f0e45df272a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54028020 con 0x5654612616b0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5f540049c0 con 0x5654612616b0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f540054d0 con 0x5654612616b0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f54033050 con 0x5654612616b0 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654613647d0 con 0x5654612616b0 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f540054d0 con 0x5654612616b0 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5f54023470 con 0x5654612616b0 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 20 client.16209 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x565461240f60 0x56546163e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 4 client.16209 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.593+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565461596990 con 0x565461240f60 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.594+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x565461240f60 0x56546163e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.594+0000 7f5f3f7fe640 10 client.16209 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.594+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x565461240f60 0x56546163e4f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87c1bc7f617aa2b server_cookie=f53cd51604a3a80c in_seq=0 out_seq=0 2024-04-25T13:43:58.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.594+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/2218256109 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565461596990 con 0x565461240f60 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f3f7fe640 10 client.16209 handle_command_reply: tid=0 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 client.16209 shutdown 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 msgr2=0x565461630eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f5401b040 tx=0x7f5f5400fc80 comp rx=0 tx=0).stop 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 shutdown_connections 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x565461240f60 0x56546163e4f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461637e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461630eb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/2218256109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x565461630970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f3f7fe640 0 client.16209 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 >> 172.21.15.94:0/2218256109 conn(0x5654612764e0 msgr2=0x565461625310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 shutdown_connections 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/2218256109 wait complete. 2024-04-25T13:43:58.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.595+0000 7f5f6d7ef740 20 client.16209 trim_cache size 0 max 16384 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.608+0000 7f5f6d7ef740 1 Processor -- start 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.608+0000 7f5f6d7ef740 1 -- start start 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x56546162f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461622150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601a9170 con 0x5654612616b0 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601aacb0 con 0x565461282390 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601a9840 con 0x5654612594d0 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52036/0 (socket says 172.21.15.94:52036) 2024-04-25T13:43:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3094169832 learned_addr learned my addr 172.21.15.94:0/3094169832 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461622150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x56546162f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3094169832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 msgr2=0x56546162f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x56546162f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3094169832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 msgr2=0x565461622150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461622150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 -- 172.21.15.94:0/3094169832 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56546162fd50 con 0x5654612616b0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x56546162f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f540238c0 tx=0x7f5f54009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e21277bad84ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f45ffb640 1 -- 172.21.15.94:0/3094169832 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54028040 con 0x5654612616b0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f45ffb640 1 -- 172.21.15.94:0/3094169832 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f540055a0 con 0x5654612616b0 2024-04-25T13:43:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.609+0000 7f5f45ffb640 1 -- 172.21.15.94:0/3094169832 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5401d040 con 0x5654612616b0 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3094169832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 msgr2=0x565461621c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f540238c0 tx=0x7f5f54009ce0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3094169832 shutdown_connections 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565461282390 0x565461622150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654612616b0 0x565461621c10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/3094169832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612594d0 0x56546162f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3094169832 >> 172.21.15.94:0/3094169832 conn(0x5654612764e0 msgr2=0x56546162f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3094169832 shutdown_connections 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/3094169832 wait complete. 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.610+0000 7f5f6d7ef740 1 -- start start 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 Processor -- start 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461634950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x5654616a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601aa5e0 con 0x565461282390 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601a9f10 con 0x5654612594d0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601a7150 con 0x5654612616b0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f6d7ef740 10 client.0 _ll_register_callbacks cb 0x7f5f58009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f64ed5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x5654616a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:48388/0 (socket says 172.21.15.94:48388) 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 -- 172.21.15.94:0/549835970 learned_addr learned my addr 172.21.15.94:0/549835970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461634950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 -- 172.21.15.94:0/549835970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 msgr2=0x565461634950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461634950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 -- 172.21.15.94:0/549835970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 msgr2=0x5654616a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x5654616a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f5ffff640 1 -- 172.21.15.94:0/549835970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654616a4d90 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.611+0000 7f5f64ed5640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x5654616a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f5ffff640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f500c76c0 tx=0x7f5f500b0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72d0c04e02e7f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500e1060 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f540055a0 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5f5003bd90 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f50049040 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 10 client.25700 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5654616a6070 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f540055a0 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5f50003c50 con 0x5654612594d0 2024-04-25T13:43:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 10 client.25700 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 10 client.25700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 20 client.25700 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5654612163a0 0x5654616a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 4 client.25700 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.612+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565461590800 con 0x5654612163a0 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5654612163a0 0x5654616a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f3f7fe640 10 client.25700 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f5f7fe640 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5654612163a0 0x5654616a8930 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cab52ee14f673ea server_cookie=9fb52642dcf3c7bb in_seq=0 out_seq=0 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f3f7fe640 1 -- 172.21.15.94:0/549835970 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565461590800 con 0x5654612163a0 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f3f7fe640 10 client.25700 handle_command_reply: tid=0 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 client.25700 shutdown 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 msgr2=0x565461634410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f500c76c0 tx=0x7f5f500b0040 comp rx=0 tx=0).stop 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 shutdown_connections 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5654612163a0 0x5654616a8930 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565461282390 0x5654616a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654612616b0 0x565461634950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f3f7fe640 0 client.25700 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 --2- 172.21.15.94:0/549835970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654612594d0 0x565461634410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 >> 172.21.15.94:0/549835970 conn(0x5654612764e0 msgr2=0x565461621c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 shutdown_connections 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.613+0000 7f5f6d7ef740 1 -- 172.21.15.94:0/549835970 wait complete. 2024-04-25T13:43:58.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f6d7ef740 20 client.25700 trim_cache size 0 max 16384 2024-04-25T13:43:58.622 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 msgr2=0x7f5f480b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 0x7f5f480b1ff0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x565461627c40 tx=0x7f5f58004ab0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 msgr2=0x7f5f60198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5f50006010 tx=0x7f5f50004270 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 shutdown_connections 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f480afae0 0x7f5f480b1ff0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f60108770 0x7f5f601a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f60103a70 0x7f5f60199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 --2- 172.21.15.94:0/2792416157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f60102830 0x7f5f60198f50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 >> 172.21.15.94:0/2792416157 conn(0x7f5f600fda40 msgr2=0x7f5f60104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 shutdown_connections 2024-04-25T13:43:58.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.614+0000 7f5f656d6640 1 -- 172.21.15.94:0/2792416157 wait complete. 2024-04-25T13:43:58.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:43:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:58 smithi137 ceph-mon[32568]: pgmap v411: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-04-25T13:43:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:43:58 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3157131801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:43:58.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 -- 172.21.15.94:0/2709654974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed81029e0 msgr2=0x7f0ed81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2709654974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed81029e0 0x7f0ed81036e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009900 tx=0x7f0ed402a410 comp rx=0 tx=0).stop 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 -- 172.21.15.94:0/2709654974 shutdown_connections 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2709654974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed8108920 0x7f0ed810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2709654974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8103c20 0x7f0ed8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2709654974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed81029e0 0x7f0ed81036e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 -- 172.21.15.94:0/2709654974 >> 172.21.15.94:0/2709654974 conn(0x7f0ed80fdbd0 msgr2=0x7f0ed8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 -- 172.21.15.94:0/2709654974 shutdown_connections 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.738+0000 7f0edff0d640 1 -- 172.21.15.94:0/2709654974 wait complete. 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 Processor -- start 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 -- start start 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed81029e0 0x7f0ed8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8108920 0x7f0ed81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed810e4e0 con 0x7f0ed8103c20 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed810e360 con 0x7f0ed8108920 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edff0d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed810e660 con 0x7f0ed81029e0 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edd4a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0eddca9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed81029e0 0x7f0ed8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edd4a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52066/0 (socket says 172.21.15.94:52066) 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0edd4a8640 1 -- 172.21.15.94:0/2433125713 learned_addr learned my addr 172.21.15.94:0/2433125713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.739+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8108920 0x7f0ed81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 -- 172.21.15.94:0/2433125713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed81029e0 msgr2=0x7f0ed8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed81029e0 0x7f0ed8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 -- 172.21.15.94:0/2433125713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8108920 msgr2=0x7f0ed81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8108920 0x7f0ed81a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 -- 172.21.15.94:0/2433125713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed81a0f50 con 0x7f0ed8103c20 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800d950 tx=0x7f0ec800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0eceffd640 1 -- 172.21.15.94:0/2433125713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8017030 con 0x7f0ed8103c20 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed81a1240 con 0x7f0ed8103c20 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0eceffd640 1 -- 172.21.15.94:0/2433125713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec8004020 con 0x7f0ed8103c20 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0eceffd640 1 -- 172.21.15.94:0/2433125713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80047a0 con 0x7f0ed8103c20 2024-04-25T13:43:58.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.740+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed81a1790 con 0x7f0ed8103c20 2024-04-25T13:43:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 Processor -- start 2024-04-25T13:43:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 -- start start 2024-04-25T13:43:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0eceffd640 1 -- 172.21.15.94:0/2433125713 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0ec8009a30 con 0x7f0ed8103c20 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d396a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56306d2f9d80 con 0x56306cff3390 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56306d2b3a20 con 0x56306cfca4d0 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56306d2f5d40 con 0x56306cfd26b0 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0eddca9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0eddca9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:48420/0 (socket says 172.21.15.94:48420) 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.743+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52076/0 (socket says 172.21.15.94:52076) 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0eddca9640 1 -- 172.21.15.94:0/2659839033 learned_addr learned my addr 172.21.15.94:0/2659839033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d396a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 -- 172.21.15.94:0/2659839033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 msgr2=0x56306d396a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d396a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 -- 172.21.15.94:0/2659839033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 msgr2=0x56306d395d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 -- 172.21.15.94:0/2659839033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306d2bec30 con 0x56306cff3390 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0eddca9640 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0eceffd640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 0x7f0ec40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009eb0 tx=0x7f0ed0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d816d216a41dedcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0eddca9640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 0x7f0ec40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0eceffd640 1 -- 172.21.15.94:0/2433125713 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ec80de6d0 con 0x7f0ed8103c20 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ec3fff640 1 -- 172.21.15.94:0/2659839033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0016070 con 0x56306cff3390 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ec3fff640 1 -- 172.21.15.94:0/2659839033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0004d20 con 0x56306cff3390 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.744+0000 7f0ec3fff640 1 -- 172.21.15.94:0/2659839033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0012780 con 0x56306cff3390 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/2659839033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 msgr2=0x56306d395810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0eddca9640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 0x7f0ec40b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x56306d395530 tx=0x7f0ed4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009eb0 tx=0x7f0ed0010740 comp rx=0 tx=0).stop 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/2659839033 shutdown_connections 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d396a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d395d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/2659839033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d395810 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/2659839033 >> 172.21.15.94:0/2659839033 conn(0x56306cfe74e0 msgr2=0x56306d391020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/2659839033 shutdown_connections 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.745+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/2659839033 wait complete. 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 -- start start 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 Processor -- start 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d39fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cff3390 0x56306d3a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56306d397530 con 0x56306cfca4d0 2024-04-25T13:43:58.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56306d280dc0 con 0x56306cff3390 2024-04-25T13:43:58.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56306d3976b0 con 0x56306cfd26b0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ee6dc4740 10 client.0 _ll_register_callbacks cb 0x56306d29c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0eddca9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0eddca9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52080/0 (socket says 172.21.15.94:52080) 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0eddca9640 1 -- 172.21.15.94:0/4162716899 learned_addr learned my addr 172.21.15.94:0/4162716899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d39fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.746+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cff3390 0x56306d3a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 -- 172.21.15.94:0/4162716899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 msgr2=0x56306d39fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d39fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 -- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cff3390 msgr2=0x56306d3a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cff3390 0x56306d3a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 -- 172.21.15.94:0/4162716899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306d00b3f0 con 0x56306cfca4d0 2024-04-25T13:43:58.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0eddca9640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009900 tx=0x7f0ed4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cbe977136cb7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4035040 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed40043c0 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed0004d20 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ed40035f0 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ee6dc4740 10 client.16227 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56306d0d57d0 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.747+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0004d20 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed4044070 con 0x56306cfca4d0 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 10 client.16227 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 20 client.16227 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cfb1f60 0x56306d3ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 4 client.16227 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:58.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56306d307990 con 0x56306cfb1f60 2024-04-25T13:43:58.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cfb1f60 0x56306d3ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0ec17fa640 10 client.16227 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:58.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.748+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cfb1f60 0x56306d3ad3b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9571f6838418f159 server_cookie=297afc53ecac9a21 in_seq=0 out_seq=0 2024-04-25T13:43:58.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.750+0000 7f0ec17fa640 1 -- 172.21.15.94:0/4162716899 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56306d307990 con 0x56306cfb1f60 2024-04-25T13:43:58.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.750+0000 7f0ec17fa640 10 client.16227 handle_command_reply: tid=0 2024-04-25T13:43:58.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.750+0000 7f0ee6dc4740 1 client.16227 shutdown 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 msgr2=0x56306d39f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009900 tx=0x7f0ed4009260 comp rx=0 tx=0).stop 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 shutdown_connections 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cfb1f60 0x56306d3ad3b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cff3390 0x56306d3a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d39fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ec17fa640 0 client.16227 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/4162716899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cfca4d0 0x56306d39f660 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 >> 172.21.15.94:0/4162716899 conn(0x56306cfe74e0 msgr2=0x56306d390040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 shutdown_connections 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/4162716899 wait complete. 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.751+0000 7f0ee6dc4740 20 client.16227 trim_cache size 0 max 16384 2024-04-25T13:43:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 Processor -- start 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 -- start start 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d39aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81aa3a0 con 0x56306cff3390 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81abee0 con 0x56306cfca4d0 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81aaa70 con 0x56306cfd26b0 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52088/0 (socket says 172.21.15.94:52088) 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 -- 172.21.15.94:0/781568188 learned_addr learned my addr 172.21.15.94:0/781568188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0eddca9640 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d39aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0edd4a8640 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 -- 172.21.15.94:0/781568188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 msgr2=0x56306d3a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 -- 172.21.15.94:0/781568188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 msgr2=0x56306d39aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d39aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.765+0000 7f0ede4aa640 1 -- 172.21.15.94:0/781568188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306d39df50 con 0x56306cff3390 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x56306d396860 tx=0x7f0ed0004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ea0196dc777621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ec37fe640 1 -- 172.21.15.94:0/781568188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002a070 con 0x56306cff3390 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ec37fe640 1 -- 172.21.15.94:0/781568188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0012900 con 0x56306cff3390 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ec37fe640 1 -- 172.21.15.94:0/781568188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0016040 con 0x56306cff3390 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/781568188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 msgr2=0x56306d394530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x56306d396860 tx=0x7f0ed0004b10 comp rx=0 tx=0).stop 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/781568188 shutdown_connections 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d394530 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/781568188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d39aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.766+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/781568188 >> 172.21.15.94:0/781568188 conn(0x56306cfe74e0 msgr2=0x56306d393e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/781568188 shutdown_connections 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/781568188 wait complete. 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- start start 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 Processor -- start 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d3a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ab810 con 0x56306cff3390 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ab140 con 0x56306cfca4d0 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81a8380 con 0x56306cfd26b0 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ee6dc4740 10 client.0 _ll_register_callbacks cb 0x7f0ed4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0edd4a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ede4aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52102/0 (socket says 172.21.15.94:52102) 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0ede4aa640 1 -- 172.21.15.94:0/1526174557 learned_addr learned my addr 172.21.15.94:0/1526174557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.767+0000 7f0eddca9640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d3a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 -- 172.21.15.94:0/1526174557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 msgr2=0x56306d3a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 -- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 msgr2=0x56306d3a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d3a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 -- 172.21.15.94:0/1526174557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306d3a5ec0 con 0x56306cff3390 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ede4aa640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0011010 tx=0x7f0ed0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70588d11c9f37ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0016070 con 0x56306cff3390 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed00125d0 con 0x56306cff3390 2024-04-25T13:43:58.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed0012900 con 0x56306cff3390 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ed002a040 con 0x56306cff3390 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 10 client.16233 fetch_fsmap learned FSMap version 25 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56306d4157e0 con 0x56306cff3390 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0012900 con 0x56306cff3390 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed002d070 con 0x56306cff3390 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.768+0000 7f0ee6dc4740 20 client.16233 populate_metadata read hostname 'smithi094' 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cf873a0 0x56306d417ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0ee6dc4740 4 client.16233 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56306d301800 con 0x56306cf873a0 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0eddca9640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cf873a0 0x56306d417ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0ec17fa640 10 client.16233 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:58.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.769+0000 7f0eddca9640 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cf873a0 0x56306d417ef0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf13c6c05a6ec655 server_cookie=eae3c31df2b283bf in_seq=0 out_seq=0 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ec17fa640 1 -- 172.21.15.94:0/1526174557 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56306d301800 con 0x56306cf873a0 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ec17fa640 10 client.16233 handle_command_reply: tid=0 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 client.16233 shutdown 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 msgr2=0x56306d3a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0011010 tx=0x7f0ed0009c00 comp rx=0 tx=0).stop 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 shutdown_connections 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56306cf873a0 0x56306d417ef0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56306cff3390 0x56306d3a5900 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56306cfd26b0 0x56306d3a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 --2- 172.21.15.94:0/1526174557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56306cfca4d0 0x56306d3a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ec17fa640 0 client.16233 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:43:58.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 >> 172.21.15.94:0/1526174557 conn(0x56306cfe74e0 msgr2=0x56306d394850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 shutdown_connections 2024-04-25T13:43:58.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 1 -- 172.21.15.94:0/1526174557 wait complete. 2024-04-25T13:43:58.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.770+0000 7f0ee6dc4740 20 client.16233 trim_cache size 0 max 16384 2024-04-25T13:43:58.867 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:43:58.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 msgr2=0x7f0ec40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 0x7f0ec40b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x56306d395530 tx=0x7f0ed4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 msgr2=0x7f0ed8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800d950 tx=0x7f0ec800de10 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 shutdown_connections 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ec40afaf0 0x7f0ec40b2000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed8108920 0x7f0ed81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed8103c20 0x7f0ed8199950 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 --2- 172.21.15.94:0/2433125713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed81029e0 0x7f0ed8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 >> 172.21.15.94:0/2433125713 conn(0x7f0ed80fdbd0 msgr2=0x7f0ed8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 shutdown_connections 2024-04-25T13:43:58.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:58.771+0000 7f0edff0d640 1 -- 172.21.15.94:0/2433125713 wait complete. 2024-04-25T13:43:58.870 INFO:tasks.workunit.client.0.smithi094.stdout: 1 113947 15.44 MB/sec warmup 41 sec latency 12.201 ms 2024-04-25T13:43:59.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 117071 15.48 MB/sec warmup 42 sec latency 16.593 ms 2024-04-25T13:43:59.870 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:43:59.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:59.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:43:59.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 -- 172.21.15.94:0/2831316784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c103c30 msgr2=0x7fc04c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 --2- 172.21.15.94:0/2831316784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c103c30 0x7fc04c104040 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009900 tx=0x7fc03c02a410 comp rx=0 tx=0).stop 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 -- 172.21.15.94:0/2831316784 shutdown_connections 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 --2- 172.21.15.94:0/2831316784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c108930 0x7fc04c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 --2- 172.21.15.94:0/2831316784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c103c30 0x7fc04c104040 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 --2- 172.21.15.94:0/2831316784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 0x7fc04c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 -- 172.21.15.94:0/2831316784 >> 172.21.15.94:0/2831316784 conn(0x7fc04c0fdbc0 msgr2=0x7fc04c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 -- 172.21.15.94:0/2831316784 shutdown_connections 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.986+0000 7fc050ed1640 1 -- 172.21.15.94:0/2831316784 wait complete. 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 Processor -- start 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 -- start start 2024-04-25T13:43:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 0x7fc04c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 0x7fc04c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e4f0 con 0x7fc04c108930 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e370 con 0x7fc04c103c30 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc050ed1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c10e670 con 0x7fc04c1029f0 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc04ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc04ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52116/0 (socket says 172.21.15.94:52116) 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc04ad9d640 1 -- 172.21.15.94:0/1167606900 learned_addr learned my addr 172.21.15.94:0/1167606900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc04a59c640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 0x7fc04c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.987+0000 7fc049d9b640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 0x7fc04c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 -- 172.21.15.94:0/1167606900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 msgr2=0x7fc04c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 0x7fc04c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 -- 172.21.15.94:0/1167606900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 msgr2=0x7fc04c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 0x7fc04c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 -- 172.21.15.94:0/1167606900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c1a1d20 con 0x7fc04c108930 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc049d9b640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 0x7fc04c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc04ad9d640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc04400eaa0 tx=0x7fc04400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc03b7fe640 1 -- 172.21.15.94:0/1167606900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04400cdb0 con 0x7fc04c108930 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc03b7fe640 1 -- 172.21.15.94:0/1167606900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc044005020 con 0x7fc04c108930 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c1a1fb0 con 0x7fc04c108930 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc03b7fe640 1 -- 172.21.15.94:0/1167606900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440116d0 con 0x7fc04c108930 2024-04-25T13:43:59.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.988+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1a2420 con 0x7fc04c108930 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 Processor -- start 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc03b7fe640 1 -- 172.21.15.94:0/1167606900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc0440051d0 con 0x7fc04c108930 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 -- start start 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c50544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c5054a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5603c4fb9b00 con 0x5603c4c92730 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5603c4f55920 con 0x5603c4cb3410 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.991+0000 7fc057d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603c4fb5ac0 con 0x5603c4c8a550 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52122/0 (socket says 172.21.15.94:52122) 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 -- 172.21.15.94:0/2523997801 learned_addr learned my addr 172.21.15.94:0/2523997801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc04a59c640 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c5054a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc03b7fe640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 0x7fc0340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c50544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 -- 172.21.15.94:0/2523997801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 msgr2=0x5603c5054a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c5054a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 -- 172.21.15.94:0/2523997801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 msgr2=0x5603c50544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c50544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc04ad9d640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 0x7fc0340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 -- 172.21.15.94:0/2523997801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c4d937e0 con 0x5603c4c92730 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc03b7fe640 1 -- 172.21.15.94:0/1167606900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc0440deeb0 con 0x7fc04c108930 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005710 tx=0x7fc03c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a1c13754a1f5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc038ff9640 1 -- 172.21.15.94:0/2523997801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c035040 con 0x5603c4c92730 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc038ff9640 1 -- 172.21.15.94:0/2523997801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03c031620 con 0x5603c4c92730 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.992+0000 7fc038ff9640 1 -- 172.21.15.94:0/2523997801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0046c0 con 0x5603c4c92730 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 -- 172.21.15.94:0/2523997801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 msgr2=0x5603c505b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005710 tx=0x7fc03c031400 comp rx=0 tx=0).stop 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc04ad9d640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 0x7fc0340b2000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc0440a6ff0 tx=0x7fc04406dc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 -- 172.21.15.94:0/2523997801 shutdown_connections 2024-04-25T13:43:59.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c505b920 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c5054a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 --2- 172.21.15.94:0/2523997801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c50544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 -- 172.21.15.94:0/2523997801 >> 172.21.15.94:0/2523997801 conn(0x5603c4ca7560 msgr2=0x5603c504fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 -- 172.21.15.94:0/2523997801 shutdown_connections 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.993+0000 7fc057d80740 1 -- 172.21.15.94:0/2523997801 wait complete. 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 -- start start 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 Processor -- start 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c505e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5603c505c310 con 0x5603c4cb3410 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5603c4f535d0 con 0x5603c4c92730 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603c505c490 con 0x5603c4c8a550 2024-04-25T13:44:00.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc057d80740 10 client.0 _ll_register_callbacks cb 0x5603c4f41080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52126/0 (socket says 172.21.15.94:52126) 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 -- 172.21.15.94:0/3030146984 learned_addr learned my addr 172.21.15.94:0/3030146984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc04a59c640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c505e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 -- 172.21.15.94:0/3030146984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 msgr2=0x5603c505e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 -- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 msgr2=0x5603c505e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c505e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.994+0000 7fc049d9b640 1 -- 172.21.15.94:0/3030146984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c4df3f70 con 0x5603c4cb3410 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc049d9b640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001eb0 tx=0x7fc03c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce20e154690e83f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c052060 con 0x5603c4cb3410 2024-04-25T13:44:00.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc03c004e00 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc03c031620 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc03c02aa90 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 10 client.16245 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603c4c8b170 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03c031620 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc03c03a050 con 0x5603c4cb3410 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 10 client.16245 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 20 client.16245 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c721f0 0x5603c506b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 4 client.16245 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:00.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:43:59.995+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c4fc7710 con 0x5603c4c721f0 2024-04-25T13:44:00.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.031+0000 7fc04a59c640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c721f0 0x5603c506b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.031+0000 7fc04a59c640 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c721f0 0x5603c506b7f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297d430d9785f842 server_cookie=e208a43dc28eac44 in_seq=0 out_seq=0 2024-04-25T13:44:00.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.031+0000 7fc031ffb640 10 client.16245 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc031ffb640 1 -- 172.21.15.94:0/3030146984 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603c4fc7710 con 0x5603c4c721f0 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc031ffb640 10 client.16245 handle_command_reply: tid=0 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 client.16245 shutdown 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 msgr2=0x5603c5065730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc03c001eb0 tx=0x7fc03c041040 comp rx=0 tx=0).stop 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 shutdown_connections 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c721f0 0x5603c506b7f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4cb3410 0x5603c5065730 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c505e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc031ffb640 0 client.16245 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 --2- 172.21.15.94:0/3030146984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.032+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 >> 172.21.15.94:0/3030146984 conn(0x5603c4ca7560 msgr2=0x5603c504eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.033+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 shutdown_connections 2024-04-25T13:44:00.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.033+0000 7fc057d80740 1 -- 172.21.15.94:0/3030146984 wait complete. 2024-04-25T13:44:00.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.033+0000 7fc057d80740 20 client.16245 trim_cache size 0 max 16384 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 Processor -- start 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 -- start start 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c5065b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1ab450 con 0x5603c4c92730 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1acf90 con 0x5603c4cb3410 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc057d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1abb20 con 0x5603c4c8a550 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52130/0 (socket says 172.21.15.94:52130) 2024-04-25T13:44:00.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 -- 172.21.15.94:0/2407773502 learned_addr learned my addr 172.21.15.94:0/2407773502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c5065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 -- 172.21.15.94:0/2407773502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 msgr2=0x5603c505cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 -- 172.21.15.94:0/2407773502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 msgr2=0x5603c5065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c5065b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 -- 172.21.15.94:0/2407773502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c5059170 con 0x5603c4c92730 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc049d9b640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.051+0000 7fc04a59c640 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc04000d410 tx=0x7fc0400139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86635bc430b0280c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.052+0000 7fc033fff640 1 -- 172.21.15.94:0/2407773502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040005020 con 0x5603c4c92730 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.052+0000 7fc033fff640 1 -- 172.21.15.94:0/2407773502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04001d430 con 0x5603c4c92730 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.052+0000 7fc033fff640 1 -- 172.21.15.94:0/2407773502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040004a00 con 0x5603c4c92730 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.052+0000 7fc057d80740 1 -- 172.21.15.94:0/2407773502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 msgr2=0x5603c5065650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.052+0000 7fc057d80740 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc04000d410 tx=0x7fc0400139c0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- 172.21.15.94:0/2407773502 shutdown_connections 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4cb3410 0x5603c5065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c92730 0x5603c5065650 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- 172.21.15.94:0/2407773502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4c8a550 0x5603c505cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- 172.21.15.94:0/2407773502 >> 172.21.15.94:0/2407773502 conn(0x5603c4ca7560 msgr2=0x5603c505c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- 172.21.15.94:0/2407773502 shutdown_connections 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- 172.21.15.94:0/2407773502 wait complete. 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- start start 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 Processor -- start 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c5063a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4cb3410 0x5603c5064960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1ac8c0 con 0x5603c4c8a550 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1ac1f0 con 0x5603c4c92730 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1a9430 con 0x5603c4cb3410 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc057d80740 10 client.0 _ll_register_callbacks cb 0x7fc03c004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc049d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52138/0 (socket says 172.21.15.94:52138) 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc049d9b640 1 -- 172.21.15.94:0/419447071 learned_addr learned my addr 172.21.15.94:0/419447071 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc04a59c640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c5063a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.053+0000 7fc049d9b640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4cb3410 0x5603c5064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 -- 172.21.15.94:0/419447071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4cb3410 msgr2=0x5603c5064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4cb3410 0x5603c5064960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 -- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 msgr2=0x5603c5063a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c5063a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 -- 172.21.15.94:0/419447071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c5064f20 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc049d9b640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005710 tx=0x7fc03c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09f00fcab7ae4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c035070 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc03c0319e0 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04001d430 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc03c052040 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 10 client.16251 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603c50661e0 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04001d430 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc03c03a070 con 0x5603c4c8a550 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 20 client.16251 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c47630 0x5603c50d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 4 client.16251 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.054+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603c4fc1580 con 0x5603c4c47630 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.055+0000 7fc04a59c640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c47630 0x5603c50d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.055+0000 7fc04a59c640 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c47630 0x5603c50d6140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe185463e6619d5 server_cookie=9e60158f72d117fa in_seq=0 out_seq=0 2024-04-25T13:44:00.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.055+0000 7fc031ffb640 10 client.16251 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc031ffb640 1 -- 172.21.15.94:0/419447071 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603c4fc1580 con 0x5603c4c47630 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc031ffb640 10 client.16251 handle_command_reply: tid=0 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 client.16251 shutdown 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 msgr2=0x5603c50634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005710 tx=0x7fc03c02a410 comp rx=0 tx=0).stop 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 shutdown_connections 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5603c4c47630 0x5603c50d6140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c4cb3410 0x5603c5064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c4c92730 0x5603c5063a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc031ffb640 0 client.16251 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 --2- 172.21.15.94:0/419447071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5603c4c8a550 0x5603c50634f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 >> 172.21.15.94:0/419447071 conn(0x5603c4ca7560 msgr2=0x5603c5052c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 shutdown_connections 2024-04-25T13:44:00.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 1 -- 172.21.15.94:0/419447071 wait complete. 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.058+0000 7fc057d80740 20 client.16251 trim_cache size 0 max 16384 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "8e1426e5-d5fc-41ec-bc4f-b6c961d68fe9", 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:44:00.064 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 msgr2=0x7fc0340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 0x7fc0340b2000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc0440a6ff0 tx=0x7fc04406dc00 comp rx=0 tx=0).stop 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 msgr2=0x7fc04c1a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc04400eaa0 tx=0x7fc04400ef60 comp rx=0 tx=0).stop 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 shutdown_connections 2024-04-25T13:44:00.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0340afaf0 0x7fc0340b2000 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc04c108930 0x7fc04c1a1500 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c103c30 0x7fc04c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 --2- 172.21.15.94:0/1167606900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc04c1029f0 0x7fc04c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 >> 172.21.15.94:0/1167606900 conn(0x7fc04c0fdbc0 msgr2=0x7fc04c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 shutdown_connections 2024-04-25T13:44:00.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.059+0000 7fc050ed1640 1 -- 172.21.15.94:0/1167606900 wait complete. 2024-04-25T13:44:00.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:44:00.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:44:00.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/766160774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc108940 msgr2=0x7ff7dc10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/766160774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc108940 0x7ff7dc10ad60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009ca0 tx=0x7ff7d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/766160774 shutdown_connections 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/766160774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc108940 0x7ff7dc10ad60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/766160774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 0x7ff7dc104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/766160774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc102a00 0x7ff7dc103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/766160774 >> 172.21.15.94:0/766160774 conn(0x7ff7dc0fdbf0 msgr2=0x7ff7dc1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/766160774 shutdown_connections 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.183+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/766160774 wait complete. 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 Processor -- start 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 -- start start 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 0x7ff7dc0fe120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc108940 0x7ff7dc0fe660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc10e530 con 0x7ff7dc102a00 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc10e3b0 con 0x7ff7dc103c40 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e3ce2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc10e6b0 con 0x7ff7dc108940 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52156/0 (socket says 172.21.15.94:52156) 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/482432357 learned_addr learned my addr 172.21.15.94:0/482432357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e127d640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 0x7ff7dc0fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e227f640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc108940 0x7ff7dc0fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/482432357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc108940 msgr2=0x7ff7dc0fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc108940 0x7ff7dc0fe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/482432357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 msgr2=0x7ff7dc0fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 0x7ff7dc0fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.184+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/482432357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc0fed90 con 0x7ff7dc102a00 2024-04-25T13:44:00.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9f0 tx=0x7ff7d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7d2ffd640 1 -- 172.21.15.94:0/482432357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800ccc0 con 0x7ff7dc102a00 2024-04-25T13:44:00.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc1a42b0 con 0x7ff7dc102a00 2024-04-25T13:44:00.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7d2ffd640 1 -- 172.21.15.94:0/482432357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80035f0 con 0x7ff7dc102a00 2024-04-25T13:44:00.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7d2ffd640 1 -- 172.21.15.94:0/482432357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800ce60 con 0x7ff7dc102a00 2024-04-25T13:44:00.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.185+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc1a4720 con 0x7ff7dc102a00 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 Processor -- start 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 -- start start 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7d2ffd640 1 -- 172.21.15.94:0/482432357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff7d8003aa0 con 0x7ff7dc102a00 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cc0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cc1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a94c25d80 con 0x559a9491f390 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a94bdf8b0 con 0x559a948fe6b0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a94c21d40 con 0x559a948f64d0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e227f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cc0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e127d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cc1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e227f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cc0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52168/0 (socket says 172.21.15.94:52168) 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e227f640 1 -- 172.21.15.94:0/631183100 learned_addr learned my addr 172.21.15.94:0/631183100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.188+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/631183100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 msgr2=0x559a94cc1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cc1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/631183100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 msgr2=0x559a94cc0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cc0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/631183100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a94bb0a40 con 0x559a948f64d0 2024-04-25T13:44:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7d2ffd640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 0x7ff7c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e227f640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 0x7ff7c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cc0fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80dcad0 tx=0x7ff7d80e7220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a629eed6fd59c6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7d2ffd640 1 -- 172.21.15.94:0/482432357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff7d80df0a0 con 0x7ff7dc102a00 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7c7fff640 1 -- 172.21.15.94:0/631183100 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a33a0 con 0x559a948f64d0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7c7fff640 1 -- 172.21.15.94:0/631183100 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80a6ff0 con 0x559a948f64d0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.189+0000 7ff7c7fff640 1 -- 172.21.15.94:0/631183100 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a6c50 con 0x559a948f64d0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- 172.21.15.94:0/631183100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 msgr2=0x559a94cc0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7e227f640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 0x7ff7c80b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x559a94cc1ab0 tx=0x7ff7d4031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cc0fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80dcad0 tx=0x7ff7d80e7220 comp rx=0 tx=0).stop 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- 172.21.15.94:0/631183100 shutdown_connections 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cc1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cc0fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/631183100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cc0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- 172.21.15.94:0/631183100 >> 172.21.15.94:0/631183100 conn(0x559a949134e0 msgr2=0x559a94cbc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- 172.21.15.94:0/631183100 shutdown_connections 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- 172.21.15.94:0/631183100 wait complete. 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.190+0000 7ff7eab8e740 1 -- start start 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 Processor -- start 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94ccae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cd1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a94cc2780 con 0x559a9491f390 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a94bacd10 con 0x559a948fe6b0 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a94cc2900 con 0x559a948f64d0 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7eab8e740 10 client.0 _ll_register_callbacks cb 0x559a94bd9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e227f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cd1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e127d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94ccae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39470/0 (socket says 172.21.15.94:39470) 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/275627880 learned_addr learned my addr 172.21.15.94:0/275627880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e227f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cd1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52174/0 (socket says 172.21.15.94:52174) 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 msgr2=0x559a94ccae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94ccae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/275627880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 msgr2=0x559a94cd1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cd1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/275627880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a94a017d0 con 0x559a948f64d0 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.191+0000 7ff7e127d640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94ccae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80dcad0 tx=0x7ff7d80e2d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ad2b4eb539b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a6e30 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff7d80afeb0 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d80a6ff0 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7d80a72b0 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 10 client.25684 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a949373f0 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80a6ff0 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff7d80e2120 con 0x559a948f64d0 2024-04-25T13:44:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 10 client.25684 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 20 client.25684 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948ddf60 0x559a94cd7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 4 client.25684 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.192+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a94c33990 con 0x559a948ddf60 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.193+0000 7ff7e127d640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948ddf60 0x559a94cd7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.193+0000 7ff7c57fa640 10 client.25684 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.193+0000 7ff7e127d640 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948ddf60 0x559a94cd7e90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780d2f79731eb64 server_cookie=d1b160ffe1ec7195 in_seq=0 out_seq=0 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.194+0000 7ff7c57fa640 1 -- 172.21.15.94:0/275627880 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a94c33990 con 0x559a948ddf60 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.194+0000 7ff7c57fa640 10 client.25684 handle_command_reply: tid=0 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.194+0000 7ff7eab8e740 1 client.25684 shutdown 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 msgr2=0x559a94cca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80dcad0 tx=0x7ff7d80e2d30 comp rx=0 tx=0).stop 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 shutdown_connections 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948ddf60 0x559a94cd7e90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a9491f390 0x559a94cd1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94ccae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/275627880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948f64d0 0x559a94cca930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7c57fa640 0 client.25684 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 >> 172.21.15.94:0/275627880 conn(0x559a949134e0 msgr2=0x559a94cbb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 shutdown_connections 2024-04-25T13:44:00.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 1 -- 172.21.15.94:0/275627880 wait complete. 2024-04-25T13:44:00.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.195+0000 7ff7eab8e740 20 client.25684 trim_cache size 0 max 16384 2024-04-25T13:44:00.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 Processor -- start 2024-04-25T13:44:00.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 -- start start 2024-04-25T13:44:00.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948fe6b0 0x559a94cd1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 0x559a94cd22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ad760 con 0x559a948f64d0 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1af2a0 con 0x559a9491f390 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ade30 con 0x559a948fe6b0 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52184/0 (socket says 172.21.15.94:52184) 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/248761101 learned_addr learned my addr 172.21.15.94:0/248761101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e127d640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948fe6b0 0x559a94cd1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e227f640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 0x559a94cd22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/248761101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948fe6b0 msgr2=0x559a94cd1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948fe6b0 0x559a94cd1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/248761101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 msgr2=0x559a94cd22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 0x559a94cd22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/248761101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a948f70f0 con 0x559a948f64d0 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e227f640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 0x559a94cd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80e4040 tx=0x7ff7d80e2da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4a88d0258f00f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7c77fe640 1 -- 172.21.15.94:0/248761101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a6e30 con 0x559a948f64d0 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.213+0000 7ff7c77fe640 1 -- 172.21.15.94:0/248761101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80b0130 con 0x559a948f64d0 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7c77fe640 1 -- 172.21.15.94:0/248761101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a33a0 con 0x559a948f64d0 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 -- 172.21.15.94:0/248761101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 msgr2=0x559a94ccac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80e4040 tx=0x7ff7d80e2da0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 -- 172.21.15.94:0/248761101 shutdown_connections 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a9491f390 0x559a94cd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a948fe6b0 0x559a94cd1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/248761101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94ccac10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 -- 172.21.15.94:0/248761101 >> 172.21.15.94:0/248761101 conn(0x559a949134e0 msgr2=0x559a94cca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 -- 172.21.15.94:0/248761101 shutdown_connections 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.214+0000 7ff7eab8e740 1 -- 172.21.15.94:0/248761101 wait complete. 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 -- start start 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 Processor -- start 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 0x559a94ccf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1aebd0 con 0x559a948f64d0 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ae500 con 0x559a948fe6b0 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1ab740 con 0x559a9491f390 2024-04-25T13:44:00.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7eab8e740 10 client.0 _ll_register_callbacks cb 0x7ff7cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52198/0 (socket says 172.21.15.94:52198) 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/771989367 learned_addr learned my addr 172.21.15.94:0/771989367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e227f640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 0x559a94ccf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e127d640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/771989367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 msgr2=0x559a94ccf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 0x559a94ccf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 msgr2=0x559a94cced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e1a7e640 1 -- 172.21.15.94:0/771989367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a94be64b0 con 0x559a948f64d0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.215+0000 7ff7e227f640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 0x559a94ccf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7e1a7e640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80e4010 tx=0x7ff7d80e2b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f49ee8c4072f6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80a6c50 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff7d80a3ad0 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d80b0130 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7d80a33a0 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 10 client.16269 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a94be64b0 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80b0130 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff7d8003fb0 con 0x559a948f64d0 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 10 client.16269 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 20 client.16269 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948b33a0 0x559a94d41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 4 client.16269 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:44:00.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.216+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a94c2d800 con 0x559a948b33a0 2024-04-25T13:44:00.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.217+0000 7ff7e127d640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948b33a0 0x559a94d41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.217+0000 7ff7c57fa640 10 client.16269 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.217+0000 7ff7e127d640 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948b33a0 0x559a94d41350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f38dfb260f3424 server_cookie=858f88c00e7a1c0b in_seq=0 out_seq=0 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7c57fa640 1 -- 172.21.15.94:0/771989367 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559a94c2d800 con 0x559a948b33a0 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7c57fa640 10 client.16269 handle_command_reply: tid=0 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 client.16269 shutdown 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 msgr2=0x559a94cd2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80e4010 tx=0x7ff7d80e2b60 comp rx=0 tx=0).stop 2024-04-25T13:44:00.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 shutdown_connections 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a948b33a0 0x559a94d41350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a9491f390 0x559a94ccf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a948fe6b0 0x559a94cced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7c57fa640 0 client.16269 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 --2- 172.21.15.94:0/771989367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a948f64d0 0x559a94cd2140 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 >> 172.21.15.94:0/771989367 conn(0x559a949134e0 msgr2=0x559a94ccaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 shutdown_connections 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.219+0000 7ff7eab8e740 1 -- 172.21.15.94:0/771989367 wait complete. 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7eab8e740 20 client.16269 trim_cache size 0 max 16384 2024-04-25T13:44:00.226 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "8e1426e5-d5fc-41ec-bc4f-b6c961d68fe9": { 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "8e1426e5-d5fc-41ec-bc4f-b6c961d68fe9", 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:44:00.227 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 msgr2=0x7ff7c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 0x7ff7c80b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x559a94cc1ab0 tx=0x7ff7d4031af0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 msgr2=0x7ff7dc0ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9f0 tx=0x7ff7d800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 shutdown_connections 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff7c80afaf0 0x7ff7c80b2000 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7dc108940 0x7ff7dc0fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7dc103c40 0x7ff7dc0fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 --2- 172.21.15.94:0/482432357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7dc102a00 0x7ff7dc0ffa80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 >> 172.21.15.94:0/482432357 conn(0x7ff7dc0fdbf0 msgr2=0x7ff7dc109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 shutdown_connections 2024-04-25T13:44:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:00.220+0000 7ff7e3ce2640 1 -- 172.21.15.94:0/482432357 wait complete. 2024-04-25T13:44:00.232 INFO:tasks.cephfs.filesystem:scrub status for tag:8e1426e5-d5fc-41ec-bc4f-b6c961d68fe9 - {'path': '/', 'tag': '8e1426e5-d5fc-41ec-bc4f-b6c961d68fe9', 'options': 'recursive,force'} 2024-04-25T13:44:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:00 smithi138 ceph-mon[32977]: pgmap v412: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-04-25T13:44:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 -- 172.21.15.94:0/752870082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd670103970 msgr2=0x7fd670103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 --2- 172.21.15.94:0/752870082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd670103970 0x7fd670103e20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd664009900 tx=0x7fd66402a410 comp rx=0 tx=0).stop 2024-04-25T13:44:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 -- 172.21.15.94:0/752870082 shutdown_connections 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 --2- 172.21.15.94:0/752870082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6701044b0 0x7fd67010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 --2- 172.21.15.94:0/752870082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd670103970 0x7fd670103e20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 --2- 172.21.15.94:0/752870082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670102f90 0x7fd6701033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 -- 172.21.15.94:0/752870082 >> 172.21.15.94:0/752870082 conn(0x7fd6700fd390 msgr2=0x7fd6700ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 -- 172.21.15.94:0/752870082 shutdown_connections 2024-04-25T13:44:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.502+0000 7fd6767d7640 1 -- 172.21.15.94:0/752870082 wait complete. 2024-04-25T13:44:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 Processor -- start 2024-04-25T13:44:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 -- start start 2024-04-25T13:44:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd670102f90 0x7fd67019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670103970 0x7fd67019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd67010e580 con 0x7fd6701044b0 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd67010e400 con 0x7fd670103970 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd6767d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd67010e700 con 0x7fd670102f90 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd674d74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670103970 0x7fd67019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd674d74640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52204/0 (socket says 172.21.15.94:52204) 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd674d74640 1 -- 172.21.15.94:0/3417449283 learned_addr learned my addr 172.21.15.94:0/3417449283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.503+0000 7fd66ffff640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd670102f90 0x7fd67019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 -- 172.21.15.94:0/3417449283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd670102f90 msgr2=0x7fd67019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd670102f90 0x7fd67019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 -- 172.21.15.94:0/3417449283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670103970 msgr2=0x7fd67019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670103970 0x7fd67019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 -- 172.21.15.94:0/3417449283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6701a1d50 con 0x7fd6701044b0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd674d74640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd66800eb10 tx=0x7fd66800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd66d7fa640 1 -- 172.21.15.94:0/3417449283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004b20 con 0x7fd6701044b0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6701a1fe0 con 0x7fd6701044b0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd66d7fa640 1 -- 172.21.15.94:0/3417449283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd668004cc0 con 0x7fd6701044b0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd66d7fa640 1 -- 172.21.15.94:0/3417449283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668010690 con 0x7fd6701044b0 2024-04-25T13:44:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.504+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701a24f0 con 0x7fd6701044b0 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.506+0000 7fd66d7fa640 1 -- 172.21.15.94:0/3417449283 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd668005020 con 0x7fd6701044b0 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 Processor -- start 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 -- start start 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 0x561cec8afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561cec815460 con 0x561cec4e62f0 2024-04-25T13:44:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561cec7cf590 con 0x561cec50f1b0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd67d767740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561cec811420 con 0x561cec4ee4d0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 0x561cec8afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:48594/0 (socket says 172.21.15.94:48594) 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 0x561cec8afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52218/0 (socket says 172.21.15.94:52218) 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 -- 172.21.15.94:0/387330503 learned_addr learned my addr 172.21.15.94:0/387330503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66d7fa640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 0x7fd6580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd674d74640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 0x7fd6580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 -- 172.21.15.94:0/387330503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 msgr2=0x561cec8b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 -- 172.21.15.94:0/387330503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 msgr2=0x561cec8afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 0x561cec8afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66f7fe640 1 -- 172.21.15.94:0/387330503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cec7dc0d0 con 0x561cec50f1b0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.507+0000 7fd66d7fa640 1 -- 172.21.15.94:0/3417449283 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6680df200 con 0x7fd6701044b0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.508+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a930 tx=0x7fd664000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f535bcd5634fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.508+0000 7fd674d74640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 0x7fd6580b1ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd6680049d0 tx=0x7fd66801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.508+0000 7fd6567fc640 1 -- 172.21.15.94:0/387330503 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664035040 con 0x561cec50f1b0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.508+0000 7fd6567fc640 1 -- 172.21.15.94:0/387330503 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd664004040 con 0x561cec50f1b0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.508+0000 7fd6567fc640 1 -- 172.21.15.94:0/387330503 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664003850 con 0x561cec50f1b0 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- 172.21.15.94:0/387330503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 msgr2=0x561cec8af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a930 tx=0x7fd664000bc0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- 172.21.15.94:0/387330503 shutdown_connections 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4e62f0 0x561cec8afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 --2- 172.21.15.94:0/387330503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8af970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- 172.21.15.94:0/387330503 >> 172.21.15.94:0/387330503 conn(0x561cec503300 msgr2=0x561cec8ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- 172.21.15.94:0/387330503 shutdown_connections 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- 172.21.15.94:0/387330503 wait complete. 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 -- start start 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.509+0000 7fd67d767740 1 Processor -- start 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561cec8b77e0 con 0x561cec4ee4d0 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561cec79cb60 con 0x561cec50f1b0 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561cec8b7960 con 0x561cec4e62f0 2024-04-25T13:44:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 10 client.0 _ll_register_callbacks cb 0x561cec7c12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39528/0 (socket says 172.21.15.94:39528) 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 -- 172.21.15.94:0/2974135436 learned_addr learned my addr 172.21.15.94:0/2974135436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 -- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 msgr2=0x561cec8c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 -- 172.21.15.94:0/2974135436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 msgr2=0x561cec8b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 -- 172.21.15.94:0/2974135436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cec527210 con 0x561cec4e62f0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66ffff640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096c0 tx=0x7fd664046040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3ae6d259272c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd64f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664035070 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6640315a0 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.510+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd664004040 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6640317a0 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 10 client.25702 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cec5f3620 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd664004040 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd6640587a0 con 0x561cec4e62f0 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 10 client.25702 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 20 client.25702 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec4cdd80 0x561cec8c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 4 client.25702 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cec823070 con 0x561cec4cdd80 2024-04-25T13:44:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.511+0000 7fd66ffff640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec4cdd80 0x561cec8c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.512+0000 7fd66ffff640 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec4cdd80 0x561cec8c6e90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7910297e43e97e7e server_cookie=9f3a15d6bff9c405 in_seq=0 out_seq=0 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.512+0000 7fd64f7fe640 10 client.25702 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd64f7fe640 1 -- 172.21.15.94:0/2974135436 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561cec823070 con 0x561cec4cdd80 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd64f7fe640 10 client.25702 handle_command_reply: tid=0 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 client.25702 shutdown 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 msgr2=0x561cec8b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096c0 tx=0x7fd664046040 comp rx=0 tx=0).stop 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 shutdown_connections 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec4cdd80 0x561cec8c6e90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec50f1b0 0x561cec8c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd64f7fe640 0 client.25702 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 --2- 172.21.15.94:0/2974135436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4e62f0 0x561cec8b95c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 >> 172.21.15.94:0/2974135436 conn(0x561cec503300 msgr2=0x561cec8ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 shutdown_connections 2024-04-25T13:44:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 1 -- 172.21.15.94:0/2974135436 wait complete. 2024-04-25T13:44:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.513+0000 7fd67d767740 20 client.25702 trim_cache size 0 max 16384 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.528+0000 7fd67d767740 1 Processor -- start 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.528+0000 7fd67d767740 1 -- start start 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.528+0000 7fd67d767740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd67d767740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd67d767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd67d767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ab5a0 con 0x561cec4ee4d0 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd67d767740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ad0e0 con 0x561cec4e62f0 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd67d767740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701abc70 con 0x561cec4cdd80 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39542/0 (socket says 172.21.15.94:39542) 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3992676729 learned_addr learned my addr 172.21.15.94:0/3992676729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3992676729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 msgr2=0x561cec8c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3992676729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 msgr2=0x561cec8aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3992676729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cec8b71b0 con 0x561cec4cdd80 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a930 tx=0x7fd664032e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724004c6c8f4f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd655ffb640 1 -- 172.21.15.94:0/3992676729 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664035070 con 0x561cec4cdd80 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd655ffb640 1 -- 172.21.15.94:0/3992676729 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6640315a0 con 0x561cec4cdd80 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.529+0000 7fd655ffb640 1 -- 172.21.15.94:0/3992676729 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66403d590 con 0x561cec4cdd80 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 -- 172.21.15.94:0/3992676729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 msgr2=0x561cec8ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a930 tx=0x7fd664032e40 comp rx=0 tx=0).stop 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 -- 172.21.15.94:0/3992676729 shutdown_connections 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4ee4d0 0x561cec8aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 --2- 172.21.15.94:0/3992676729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4cdd80 0x561cec8ae810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 -- 172.21.15.94:0/3992676729 >> 172.21.15.94:0/3992676729 conn(0x561cec503300 msgr2=0x561cec8ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 -- 172.21.15.94:0/3992676729 shutdown_connections 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.530+0000 7fd67d767740 1 -- 172.21.15.94:0/3992676729 wait complete. 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 -- start start 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 Processor -- start 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8be990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701aca10 con 0x561cec4cdd80 2024-04-25T13:44:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701ac340 con 0x561cec4e62f0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701a9580 con 0x561cec4ee4d0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd67d767740 10 client.0 _ll_register_callbacks cb 0x7fd6600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd66f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52240/0 (socket says 172.21.15.94:52240) 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3705891489 learned_addr learned my addr 172.21.15.94:0/3705891489 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd66ffff640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.531+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3705891489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 msgr2=0x561cec8bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8bf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 msgr2=0x561cec8be990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8be990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 -- 172.21.15.94:0/3705891489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cec8b44a0 con 0x561cec4cdd80 2024-04-25T13:44:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd66f7fe640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096c0 tx=0x7fd66403d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67c6034853208c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664035070 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd664031b50 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6640315a0 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd664031d50 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd67d767740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cec92f110 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6640315a0 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.532+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd66403b450 con 0x561cec4cdd80 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 20 client.16287 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec50f1b0 0x561cec931af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 4 client.16287 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cec81cee0 con 0x561cec50f1b0 2024-04-25T13:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd66ffff640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec50f1b0 0x561cec931af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd64f7fe640 10 client.16287 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.533+0000 7fd66ffff640 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec50f1b0 0x561cec931af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd9d26f8da6e618 server_cookie=466ce0a812703766 in_seq=0 out_seq=0 2024-04-25T13:44:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:00 smithi137 ceph-mon[32568]: pgmap v412: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-04-25T13:44:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd64f7fe640 1 -- 172.21.15.94:0/3705891489 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cec81cee0 con 0x561cec50f1b0 2024-04-25T13:44:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd64f7fe640 10 client.16287 handle_command_reply: tid=0 2024-04-25T13:44:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 client.16287 shutdown 2024-04-25T13:44:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 msgr2=0x561cec8be450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd6640096c0 tx=0x7fd66403d4a0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 shutdown_connections 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561cec50f1b0 0x561cec931af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561cec4ee4d0 0x561cec8bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd64f7fe640 0 client.16287 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cec4e62f0 0x561cec8be990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 --2- 172.21.15.94:0/3705891489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cec4cdd80 0x561cec8be450 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.679+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 >> 172.21.15.94:0/3705891489 conn(0x561cec503300 msgr2=0x561cec8b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 shutdown_connections 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd67d767740 1 -- 172.21.15.94:0/3705891489 wait complete. 2024-04-25T13:44:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd67d767740 20 client.16287 trim_cache size 0 max 16384 2024-04-25T13:44:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:44:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:44:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:44:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 msgr2=0x7fd6580b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 0x7fd6580b1ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd6680049d0 tx=0x7fd66801d000 comp rx=0 tx=0).stop 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 msgr2=0x7fd6701a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd66800eb10 tx=0x7fd66800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 shutdown_connections 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6580afae0 0x7fd6580b1ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6701044b0 0x7fd6701a16e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd670103970 0x7fd67019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 --2- 172.21.15.94:0/3417449283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd670102f90 0x7fd67019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 >> 172.21.15.94:0/3417449283 conn(0x7fd6700fd390 msgr2=0x7fd670109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 shutdown_connections 2024-04-25T13:44:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:00.680+0000 7fd6767d7640 1 -- 172.21.15.94:0/3417449283 wait complete. 2024-04-25T13:44:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:00 smithi094 ceph-mon[29195]: pgmap v412: 97 pgs: 97 active+clean; 32 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-04-25T13:44:00.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 118525 15.28 MB/sec warmup 43 sec latency 84.004 ms 2024-04-25T13:44:01.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:01 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:44:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:01 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:01 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:44:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:01 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:44:01.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:01 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:44:01.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:01 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:01 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:44:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:01 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:44:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:01 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:44:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:01 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:01 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:44:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:01 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:44:01.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 121174 15.25 MB/sec warmup 44 sec latency 12.935 ms 2024-04-25T13:44:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:02 smithi138 ceph-mon[32977]: pgmap v413: 97 pgs: 97 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-04-25T13:44:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:02 smithi137 ceph-mon[32568]: pgmap v413: 97 pgs: 97 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-04-25T13:44:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:02 smithi094 ceph-mon[29195]: pgmap v413: 97 pgs: 97 active+clean; 38 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-04-25T13:44:02.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124249 15.31 MB/sec warmup 45 sec latency 36.326 ms 2024-04-25T13:44:03.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 126611 15.16 MB/sec warmup 46 sec latency 17.708 ms 2024-04-25T13:44:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:04 smithi138 ceph-mon[32977]: pgmap v414: 97 pgs: 97 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-04-25T13:44:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:04 smithi137 ceph-mon[32568]: pgmap v414: 97 pgs: 97 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-04-25T13:44:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:04 smithi094 ceph-mon[29195]: pgmap v414: 97 pgs: 97 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-04-25T13:44:04.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128891 15.14 MB/sec warmup 47 sec latency 14.919 ms 2024-04-25T13:44:05.790 INFO:tasks.workunit.client.0.smithi094.stdout: 1 131704 15.18 MB/sec warmup 48 sec latency 14.203 ms 2024-04-25T13:44:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:06 smithi138 ceph-mon[32977]: pgmap v415: 97 pgs: 97 active+clean; 35 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-04-25T13:44:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:06 smithi137 ceph-mon[32568]: pgmap v415: 97 pgs: 97 active+clean; 35 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-04-25T13:44:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:06 smithi094 ceph-mon[29195]: pgmap v415: 97 pgs: 97 active+clean; 35 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-04-25T13:44:06.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 134298 15.13 MB/sec warmup 49 sec latency 99.553 ms 2024-04-25T13:44:07.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:44:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:07 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:44:07.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:07 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:44:07.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:07 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:07.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 136469 15.06 MB/sec warmup 50 sec latency 16.211 ms 2024-04-25T13:44:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:08 smithi137 ceph-mon[32568]: pgmap v416: 97 pgs: 97 active+clean; 36 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-04-25T13:44:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:44:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:44:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:08 smithi094 ceph-mon[29195]: pgmap v416: 97 pgs: 97 active+clean; 36 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-04-25T13:44:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:44:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:44:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:08 smithi138 ceph-mon[32977]: pgmap v416: 97 pgs: 97 active+clean; 36 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-04-25T13:44:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:44:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:44:08.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:44:08.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139065 15.04 MB/sec warmup 51 sec latency 17.063 ms 2024-04-25T13:44:09.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142181 15.11 MB/sec warmup 52 sec latency 10.910 ms 2024-04-25T13:44:10.526 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:10 smithi094 ceph-mon[29195]: pgmap v417: 97 pgs: 97 active+clean; 36 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-04-25T13:44:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:10 smithi137 ceph-mon[32568]: pgmap v417: 97 pgs: 97 active+clean; 36 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-04-25T13:44:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:10 smithi138 ceph-mon[32977]: pgmap v417: 97 pgs: 97 active+clean; 36 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-04-25T13:44:10.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 144900 15.07 MB/sec warmup 53 sec latency 16.130 ms 2024-04-25T13:44:11.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 147038 15.00 MB/sec warmup 54 sec latency 12.866 ms 2024-04-25T13:44:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:12 smithi137 ceph-mon[32568]: pgmap v418: 97 pgs: 97 active+clean; 42 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-04-25T13:44:12.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:12 smithi094 ceph-mon[29195]: pgmap v418: 97 pgs: 97 active+clean; 42 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-04-25T13:44:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:12 smithi138 ceph-mon[32977]: pgmap v418: 97 pgs: 97 active+clean; 42 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-04-25T13:44:12.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 149586 14.98 MB/sec warmup 55 sec latency 16.331 ms 2024-04-25T13:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 -- 172.21.15.94:0/1946136729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc103400 msgr2=0x7fdedc103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 --2- 172.21.15.94:0/1946136729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc103400 0x7fdedc103810 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009900 tx=0x7fdec802a410 comp rx=0 tx=0).stop 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 -- 172.21.15.94:0/1946136729 shutdown_connections 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 --2- 172.21.15.94:0/1946136729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc108100 0x7fdedc10a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 --2- 172.21.15.94:0/1946136729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc103400 0x7fdedc103810 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 --2- 172.21.15.94:0/1946136729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc1021c0 0x7fdedc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 -- 172.21.15.94:0/1946136729 >> 172.21.15.94:0/1946136729 conn(0x7fdedc0fd390 msgr2=0x7fdedc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 -- 172.21.15.94:0/1946136729 shutdown_connections 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.780+0000 7fdee319e640 1 -- 172.21.15.94:0/1946136729 wait complete. 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 Processor -- start 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 -- start start 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc103400 0x7fdedc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc108100 0x7fdedc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e570 con 0x7fdedc1021c0 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e3f0 con 0x7fdedc108100 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee319e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc10e6f0 con 0x7fdedc103400 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47090/0 (socket says 172.21.15.94:47090) 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3426481631 learned_addr learned my addr 172.21.15.94:0/3426481631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdedbfff640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc103400 0x7fdedc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.781+0000 7fdee173b640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc108100 0x7fdedc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3426481631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc103400 msgr2=0x7fdedc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc103400 0x7fdedc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3426481631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc108100 msgr2=0x7fdedc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc108100 0x7fdedc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3426481631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc1a0c50 con 0x7fdedc1021c0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fded000eb00 tx=0x7fded0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fded9ffb640 1 -- 172.21.15.94:0/3426481631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded000bbd0 con 0x7fdedc1021c0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fded9ffb640 1 -- 172.21.15.94:0/3426481631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded000bd70 con 0x7fdedc1021c0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc10ac70 con 0x7fdedc1021c0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fded9ffb640 1 -- 172.21.15.94:0/3426481631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0017410 con 0x7fdedc1021c0 2024-04-25T13:44:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.782+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc10b210 con 0x7fdedc1021c0 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.784+0000 7fded9ffb640 1 -- 172.21.15.94:0/3426481631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fded001e020 con 0x7fdedc1021c0 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 Processor -- start 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 -- start start 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fded9ffb640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 0x7fdecc0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5ef24d0 0x55cbc62badf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cbc6219460 con 0x55cbc5f131b0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbc61d3590 con 0x55cbc5eea2f0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdeea04b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbc6215420 con 0x55cbc5ef24d0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdee173b640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 0x7fdecc0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47092/0 (socket says 172.21.15.94:47092) 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3571442120 learned_addr learned my addr 172.21.15.94:0/3571442120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdedbfff640 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fded9ffb640 1 -- 172.21.15.94:0/3426481631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fded00e0170 con 0x7fdedc1021c0 2024-04-25T13:44:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.785+0000 7fdee173b640 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5ef24d0 0x55cbc62badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3571442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5ef24d0 msgr2=0x55cbc62badf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5ef24d0 0x55cbc62badf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3571442120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 msgr2=0x55cbc62b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3571442120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc61e00d0 con 0x55cbc5f131b0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee173b640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 0x7fdecc0b2070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fded4005fd0 tx=0x7fded4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fded0004770 tx=0x7fded00a3e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb54577ccf5e23c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdec6ffd640 1 -- 172.21.15.94:0/3571442120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00a7d20 con 0x55cbc5f131b0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdec6ffd640 1 -- 172.21.15.94:0/3571442120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded00a80c0 con 0x55cbc5f131b0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.786+0000 7fdec6ffd640 1 -- 172.21.15.94:0/3571442120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00adaf0 con 0x55cbc5f131b0 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- 172.21.15.94:0/3571442120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 msgr2=0x55cbc62b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fded0004770 tx=0x7fded00a3e70 comp rx=0 tx=0).stop 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- 172.21.15.94:0/3571442120 shutdown_connections 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5ef24d0 0x55cbc62badf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3571442120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5f131b0 0x55cbc62b3970 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- 172.21.15.94:0/3571442120 >> 172.21.15.94:0/3571442120 conn(0x55cbc5f07300 msgr2=0x55cbc62af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- 172.21.15.94:0/3571442120 shutdown_connections 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- 172.21.15.94:0/3571442120 wait complete. 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 -- start start 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.787+0000 7fdeea04b740 1 Processor -- start 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cbc62bb7e0 con 0x55cbc5ef24d0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cbc61a0b60 con 0x55cbc5eea2f0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbc62bb960 con 0x55cbc5f131b0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdeea04b740 10 client.0 _ll_register_callbacks cb 0x55cbc61c52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62bd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58810/0 (socket says 172.21.15.94:58810) 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47100/0 (socket says 172.21.15.94:47100) 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee0f3a640 1 -- 172.21.15.94:0/3781914632 learned_addr learned my addr 172.21.15.94:0/3781914632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 -- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 msgr2=0x55cbc62bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 -- 172.21.15.94:0/3781914632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 msgr2=0x55cbc62bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 -- 172.21.15.94:0/3781914632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc5f2b210 con 0x55cbc5f131b0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdedbfff640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdee173b640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c4a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fded401e540 tx=0x7fded401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b379c35eb02e9a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.788+0000 7fdebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4024070 con 0x55cbc5f131b0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded00a80c0 con 0x55cbc5f131b0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fded4003970 con 0x55cbc5f131b0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fded4003b70 con 0x55cbc5f131b0 2024-04-25T13:44:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 10 client.25720 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbc5ff7620 con 0x55cbc5f131b0 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded00a80c0 con 0x55cbc5f131b0 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fded4020600 con 0x55cbc5f131b0 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 10 client.25720 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 20 client.25720 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ed1d80 0x55cbc62cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 4 client.25720 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc6227070 con 0x55cbc5ed1d80 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.789+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ed1d80 0x55cbc62cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.790+0000 7fdebbfff640 10 client.25720 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.790+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ed1d80 0x55cbc62cb2b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91ecc372569e790 server_cookie=e7f95f55a9e20f4c in_seq=0 out_seq=0 2024-04-25T13:44:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdebbfff640 1 -- 172.21.15.94:0/3781914632 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cbc6227070 con 0x55cbc5ed1d80 2024-04-25T13:44:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdebbfff640 10 client.25720 handle_command_reply: tid=0 2024-04-25T13:44:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 client.25720 shutdown 2024-04-25T13:44:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 msgr2=0x55cbc62c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c4a60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fded401e540 tx=0x7fded401d000 comp rx=0 tx=0).stop 2024-04-25T13:44:13.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 152711 15.04 MB/sec warmup 56 sec latency 13.165 ms 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 shutdown_connections 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ed1d80 0x55cbc62cb2b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c4a60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 --2- 172.21.15.94:0/3781914632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdebbfff640 0 client.25720 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 >> 172.21.15.94:0/3781914632 conn(0x55cbc5f07300 msgr2=0x55cbc62b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 shutdown_connections 2024-04-25T13:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 1 -- 172.21.15.94:0/3781914632 wait complete. 2024-04-25T13:44:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.791+0000 7fdeea04b740 20 client.25720 trim_cache size 0 max 16384 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 Processor -- start 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 -- start start 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a93f0 con 0x55cbc5ef24d0 2024-04-25T13:44:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1aaf30 con 0x55cbc5eea2f0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdeea04b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a9ac0 con 0x55cbc5f131b0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47114/0 (socket says 172.21.15.94:47114) 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 -- 172.21.15.94:0/1911109971 learned_addr learned my addr 172.21.15.94:0/1911109971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdee173b640 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 -- 172.21.15.94:0/1911109971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 msgr2=0x55cbc62b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 -- 172.21.15.94:0/1911109971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 msgr2=0x55cbc62b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.806+0000 7fdedbfff640 1 -- 172.21.15.94:0/1911109971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc62b84d0 con 0x55cbc5ef24d0 2024-04-25T13:44:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdedbfff640 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdec802a930 tx=0x7fdec8031d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f571b0ee206abcb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdec67fc640 1 -- 172.21.15.94:0/1911109971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8035040 con 0x55cbc5ef24d0 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdec67fc640 1 -- 172.21.15.94:0/1911109971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec8031e40 con 0x55cbc5ef24d0 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdec67fc640 1 -- 172.21.15.94:0/1911109971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80038e0 con 0x55cbc5ef24d0 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdeea04b740 1 -- 172.21.15.94:0/1911109971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 msgr2=0x55cbc62c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.807+0000 7fdeea04b740 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdec802a930 tx=0x7fdec8031d50 comp rx=0 tx=0).stop 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- 172.21.15.94:0/1911109971 shutdown_connections 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c4980 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- 172.21.15.94:0/1911109971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5eea2f0 0x55cbc62b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- 172.21.15.94:0/1911109971 >> 172.21.15.94:0/1911109971 conn(0x55cbc5f07300 msgr2=0x55cbc62b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- 172.21.15.94:0/1911109971 shutdown_connections 2024-04-25T13:44:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- 172.21.15.94:0/1911109971 wait complete. 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- start start 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 Processor -- start 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1aa860 con 0x55cbc5eea2f0 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1aa190 con 0x55cbc5ef24d0 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc1a73d0 con 0x55cbc5f131b0 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.808+0000 7fdeea04b740 10 client.0 _ll_register_callbacks cb 0x7fded4009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee173b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdedbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47124/0 (socket says 172.21.15.94:47124) 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 -- 172.21.15.94:0/2406855952 learned_addr learned my addr 172.21.15.94:0/2406855952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 -- 172.21.15.94:0/2406855952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 msgr2=0x55cbc62c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 -- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5ef24d0 msgr2=0x55cbc62c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 -- 172.21.15.94:0/2406855952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc62c4ec0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdee0f3a640 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b714890c6e23f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdebbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00ae0f0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fded00ae290 con 0x55cbc5eea2f0 2024-04-25T13:44:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbc6220ee0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fded00a80c0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdeea04b740 10 client.16305 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.809+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cbc62c4ae0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbc6220ee0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fded00afce0 con 0x55cbc5eea2f0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 10 client.16305 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 20 client.16305 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ea71c0 0x55cbc63357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 4 client.16305 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbc619c5f0 con 0x55cbc5ea71c0 2024-04-25T13:44:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdedbfff640 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ea71c0 0x55cbc63357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdebbfff640 10 client.16305 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.810+0000 7fdedbfff640 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ea71c0 0x55cbc63357f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8377348900599cff server_cookie=a7c7df6a471c8bee in_seq=0 out_seq=0 2024-04-25T13:44:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdebbfff640 1 -- 172.21.15.94:0/2406855952 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbc619c5f0 con 0x55cbc5ea71c0 2024-04-25T13:44:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdebbfff640 10 client.16305 handle_command_reply: tid=0 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 client.16305 shutdown 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 msgr2=0x55cbc62c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded00096c0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 shutdown_connections 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cbc5ea71c0 0x55cbc63357f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbc5f131b0 0x55cbc62c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cbc5ef24d0 0x55cbc62c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 --2- 172.21.15.94:0/2406855952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbc5eea2f0 0x55cbc62c1f20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 >> 172.21.15.94:0/2406855952 conn(0x55cbc5f07300 msgr2=0x55cbc62b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdebbfff640 0 client.16305 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 shutdown_connections 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 1 -- 172.21.15.94:0/2406855952 wait complete. 2024-04-25T13:44:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.951+0000 7fdeea04b740 20 client.16305 trim_cache size 0 max 16384 2024-04-25T13:44:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:44:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:44:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:44:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 msgr2=0x7fdecc0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 0x7fdecc0b2070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fded4005fd0 tx=0x7fded4009f10 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 msgr2=0x7fdedc199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fded000eb00 tx=0x7fded0009ec0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 shutdown_connections 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdecc0afb60 0x7fdecc0b2070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdedc108100 0x7fdedc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdedc103400 0x7fdedc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 --2- 172.21.15.94:0/3426481631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdedc1021c0 0x7fdedc199110 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 >> 172.21.15.94:0/3426481631 conn(0x7fdedc0fd390 msgr2=0x7fdedc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 shutdown_connections 2024-04-25T13:44:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:13.952+0000 7fdee319e640 1 -- 172.21.15.94:0/3426481631 wait complete. 2024-04-25T13:44:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:14 smithi137 ceph-mon[32568]: pgmap v419: 97 pgs: 97 active+clean; 42 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 309 op/s 2024-04-25T13:44:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:14 smithi094 ceph-mon[29195]: pgmap v419: 97 pgs: 97 active+clean; 42 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 309 op/s 2024-04-25T13:44:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:14 smithi138 ceph-mon[32977]: pgmap v419: 97 pgs: 97 active+clean; 42 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 309 op/s 2024-04-25T13:44:14.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 154750 14.90 MB/sec warmup 57 sec latency 80.349 ms 2024-04-25T13:44:15.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156877 14.88 MB/sec warmup 58 sec latency 13.318 ms 2024-04-25T13:44:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:16 smithi137 ceph-mon[32568]: pgmap v420: 97 pgs: 97 active+clean; 35 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-04-25T13:44:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:16 smithi094 ceph-mon[29195]: pgmap v420: 97 pgs: 97 active+clean; 35 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-04-25T13:44:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:16 smithi138 ceph-mon[32977]: pgmap v420: 97 pgs: 97 active+clean; 35 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 417 op/s 2024-04-25T13:44:16.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 159894 14.94 MB/sec warmup 59 sec latency 16.148 ms 2024-04-25T13:44:17.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 162621 14.90 MB/sec warmup 60 sec latency 14.557 ms 2024-04-25T13:44:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:18 smithi137 ceph-mon[32568]: pgmap v421: 97 pgs: 97 active+clean; 35 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-25T13:44:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:18 smithi094 ceph-mon[29195]: pgmap v421: 97 pgs: 97 active+clean; 35 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-25T13:44:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:18 smithi138 ceph-mon[32977]: pgmap v421: 97 pgs: 97 active+clean; 35 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-04-25T13:44:18.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 164702 14.85 MB/sec warmup 61 sec latency 23.115 ms 2024-04-25T13:44:19.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167115 14.83 MB/sec warmup 62 sec latency 90.196 ms 2024-04-25T13:44:20.549 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:20 smithi138 ceph-mon[32977]: pgmap v422: 97 pgs: 97 active+clean; 36 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 263 op/s 2024-04-25T13:44:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:20 smithi137 ceph-mon[32568]: pgmap v422: 97 pgs: 97 active+clean; 36 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 263 op/s 2024-04-25T13:44:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:20 smithi094 ceph-mon[29195]: pgmap v422: 97 pgs: 97 active+clean; 36 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 263 op/s 2024-04-25T13:44:20.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 169674 14.80 MB/sec warmup 63 sec latency 39.468 ms 2024-04-25T13:44:21.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 171459 14.70 MB/sec warmup 64 sec latency 36.399 ms 2024-04-25T13:44:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:22 smithi137 ceph-mon[32568]: pgmap v423: 97 pgs: 97 active+clean; 43 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-04-25T13:44:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:22 smithi094 ceph-mon[29195]: pgmap v423: 97 pgs: 97 active+clean; 43 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-04-25T13:44:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:22 smithi138 ceph-mon[32977]: pgmap v423: 97 pgs: 97 active+clean; 43 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-04-25T13:44:22.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174306 14.74 MB/sec warmup 65 sec latency 9.425 ms 2024-04-25T13:44:23.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 177486 14.78 MB/sec warmup 66 sec latency 24.631 ms 2024-04-25T13:44:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:24 smithi137 ceph-mon[32568]: pgmap v424: 97 pgs: 97 active+clean; 43 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-04-25T13:44:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:24 smithi094 ceph-mon[29195]: pgmap v424: 97 pgs: 97 active+clean; 43 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-04-25T13:44:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:24 smithi138 ceph-mon[32977]: pgmap v424: 97 pgs: 97 active+clean; 43 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 307 op/s 2024-04-25T13:44:24.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 179360 14.67 MB/sec warmup 67 sec latency 69.429 ms 2024-04-25T13:44:25.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 181598 14.65 MB/sec warmup 68 sec latency 14.012 ms 2024-04-25T13:44:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:26 smithi137 ceph-mon[32568]: pgmap v425: 97 pgs: 97 active+clean; 45 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-25T13:44:26.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:26 smithi094 ceph-mon[29195]: pgmap v425: 97 pgs: 97 active+clean; 45 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-25T13:44:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:26 smithi138 ceph-mon[32977]: pgmap v425: 97 pgs: 97 active+clean; 45 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-25T13:44:26.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 184699 14.71 MB/sec warmup 69 sec latency 12.571 ms 2024-04-25T13:44:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 -- 172.21.15.94:0/3975185382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d94103990 msgr2=0x7f0d94103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/3975185382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d94103990 0x7f0d94103e20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009900 tx=0x7f0d8802a410 comp rx=0 tx=0).stop 2024-04-25T13:44:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 -- 172.21.15.94:0/3975185382 shutdown_connections 2024-04-25T13:44:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/3975185382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d941044b0 0x7f0d9410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/3975185382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d94103990 0x7f0d94103e20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/3975185382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94102fb0 0x7f0d941033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 -- 172.21.15.94:0/3975185382 >> 172.21.15.94:0/3975185382 conn(0x7f0d940fd390 msgr2=0x7f0d940ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 -- 172.21.15.94:0/3975185382 shutdown_connections 2024-04-25T13:44:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.056+0000 7f0d98b6c640 1 -- 172.21.15.94:0/3975185382 wait complete. 2024-04-25T13:44:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 Processor -- start 2024-04-25T13:44:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 -- start start 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94102fb0 0x7f0d94199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 0x7f0d941996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9410e580 con 0x7f0d941044b0 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9410e400 con 0x7f0d94102fb0 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d98b6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9410e700 con 0x7f0d94103990 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d9259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94102fb0 0x7f0d94199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59374/0 (socket says 172.21.15.94:59374) 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 -- 172.21.15.94:0/4234743668 learned_addr learned my addr 172.21.15.94:0/4234743668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 0x7f0d941996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 -- 172.21.15.94:0/4234743668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 msgr2=0x7f0d941996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 0x7f0d941996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 -- 172.21.15.94:0/4234743668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94102fb0 msgr2=0x7f0d94199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94102fb0 0x7f0d94199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 -- 172.21.15.94:0/4234743668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d941a0ca0 con 0x7f0d941044b0 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 0x7f0d941996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.057+0000 7f0d92d9d640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00eb10 tx=0x7f0d8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.058+0000 7f0d837fe640 1 -- 172.21.15.94:0/4234743668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c004b20 con 0x7f0d941044b0 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.058+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d941a0f90 con 0x7f0d941044b0 2024-04-25T13:44:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.058+0000 7f0d837fe640 1 -- 172.21.15.94:0/4234743668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c004cc0 con 0x7f0d941044b0 2024-04-25T13:44:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.058+0000 7f0d837fe640 1 -- 172.21.15.94:0/4234743668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c010670 con 0x7f0d941044b0 2024-04-25T13:44:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.058+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d941a13f0 con 0x7f0d941044b0 2024-04-25T13:44:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.060+0000 7f0d837fe640 1 -- 172.21.15.94:0/4234743668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0d8c005020 con 0x7f0d941044b0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.060+0000 7f0d9fa17740 1 Processor -- start 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.060+0000 7f0d9fa17740 1 -- start start 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d36a1b0 0x55647d70a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55647d670460 con 0x55647d3494d0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55647d62a590 con 0x55647d36a1b0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55647d66c420 con 0x55647d3412f0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59390/0 (socket says 172.21.15.94:59390) 2024-04-25T13:44:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 -- 172.21.15.94:0/1746199556 learned_addr learned my addr 172.21.15.94:0/1746199556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9259c640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33962/0 (socket says 172.21.15.94:33962) 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d36a1b0 0x55647d70a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d837fe640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 0x7f0d7c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d92d9d640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 0x7f0d7c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 -- 172.21.15.94:0/1746199556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 msgr2=0x55647d70aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 -- 172.21.15.94:0/1746199556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d36a1b0 msgr2=0x55647d70a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d36a1b0 0x55647d70a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d91d9b640 1 -- 172.21.15.94:0/1746199556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647d6370d0 con 0x55647d3494d0 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d9259c640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.061+0000 7f0d837fe640 1 -- 172.21.15.94:0/4234743668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0d8c0df2f0 con 0x7f0d941044b0 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.062+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005940 tx=0x7f0d88005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773996f16ee4f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.062+0000 7f0d92d9d640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 0x7f0d7c0b1ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0a49a0 tx=0x7f0d8c076280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.062+0000 7f0d80ff9640 1 -- 172.21.15.94:0/1746199556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88035040 con 0x55647d3494d0 2024-04-25T13:44:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.062+0000 7f0d80ff9640 1 -- 172.21.15.94:0/1746199556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d88005020 con 0x55647d3494d0 2024-04-25T13:44:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.062+0000 7f0d80ff9640 1 -- 172.21.15.94:0/1746199556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880044e0 con 0x55647d3494d0 2024-04-25T13:44:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1746199556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 msgr2=0x55647d711df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005940 tx=0x7f0d88005a40 comp rx=0 tx=0).stop 2024-04-25T13:44:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1746199556 shutdown_connections 2024-04-25T13:44:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d711df0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3412f0 0x55647d70aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1746199556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d36a1b0 0x55647d70a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1746199556 >> 172.21.15.94:0/1746199556 conn(0x55647d35e300 msgr2=0x55647d706300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1746199556 shutdown_connections 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1746199556 wait complete. 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 -- start start 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 Processor -- start 2024-04-25T13:44:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3494d0 0x55647d714b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.063+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d71ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55647d7127e0 con 0x55647d3412f0 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55647d5f7b60 con 0x55647d3494d0 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55647d712960 con 0x55647d36a1b0 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 10 client.0 _ll_register_callbacks cb 0x55647d61c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59394/0 (socket says 172.21.15.94:59394) 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 -- 172.21.15.94:0/3424317370 learned_addr learned my addr 172.21.15.94:0/3424317370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9259c640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3494d0 0x55647d714b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d71ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 -- 172.21.15.94:0/3424317370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 msgr2=0x55647d71ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d71ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 -- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3494d0 msgr2=0x55647d714b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3494d0 0x55647d714b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 -- 172.21.15.94:0/3424317370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647d382210 con 0x55647d3412f0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005e00 tx=0x7f0d88032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35cb21ae7f2640a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88035070 con 0x55647d3412f0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0d88004270 con 0x55647d3412f0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d88005020 con 0x55647d3412f0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d8804e040 con 0x55647d3412f0 2024-04-25T13:44:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 10 client.16317 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.064+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55647d44e620 con 0x55647d3412f0 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d88005020 con 0x55647d3412f0 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d8805a070 con 0x55647d3412f0 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 10 client.16317 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 20 client.16317 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d328d80 0x55647d721ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 4 client.16317 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647d67e070 con 0x55647d328d80 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9259c640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d328d80 0x55647d721ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d9259c640 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d328d80 0x55647d721ee0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e772ed03b6850d0e server_cookie=69a87199bcd4177e in_seq=0 out_seq=0 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.065+0000 7f0d79ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.067+0000 7f0d79ffb640 1 -- 172.21.15.94:0/3424317370 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55647d67e070 con 0x55647d328d80 2024-04-25T13:44:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.067+0000 7f0d79ffb640 10 client.16317 handle_command_reply: tid=0 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 client.16317 shutdown 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 msgr2=0x55647d7145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005e00 tx=0x7f0d88032f40 comp rx=0 tx=0).stop 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 shutdown_connections 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d328d80 0x55647d721ee0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d71ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d79ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3494d0 0x55647d714b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/3424317370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3412f0 0x55647d7145f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 >> 172.21.15.94:0/3424317370 conn(0x55647d35e300 msgr2=0x55647d709210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 shutdown_connections 2024-04-25T13:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 1 -- 172.21.15.94:0/3424317370 wait complete. 2024-04-25T13:44:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.068+0000 7f0d9fa17740 20 client.16317 trim_cache size 0 max 16384 2024-04-25T13:44:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 Processor -- start 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 -- start start 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d705d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d709100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941a91f0 con 0x55647d3494d0 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941aad30 con 0x55647d3412f0 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941a98c0 con 0x55647d36a1b0 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59402/0 (socket says 172.21.15.94:59402) 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d9259c640 1 -- 172.21.15.94:0/1362530105 learned_addr learned my addr 172.21.15.94:0/1362530105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.083+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d705d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d709100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 -- 172.21.15.94:0/1362530105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 msgr2=0x55647d709100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d709100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 -- 172.21.15.94:0/1362530105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 msgr2=0x55647d705d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d705d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 -- 172.21.15.94:0/1362530105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647d712210 con 0x55647d3494d0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d709100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d9259c640 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55647d707090 tx=0x7f0d84005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8661d7a2ed538b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d7bfff640 1 -- 172.21.15.94:0/1362530105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84010430 con 0x55647d3494d0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d7bfff640 1 -- 172.21.15.94:0/1362530105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84010a80 con 0x55647d3494d0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.084+0000 7f0d7bfff640 1 -- 172.21.15.94:0/1362530105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8401ea20 con 0x55647d3494d0 2024-04-25T13:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1362530105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 msgr2=0x55647d706270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55647d707090 tx=0x7f0d84005f90 comp rx=0 tx=0).stop 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1362530105 shutdown_connections 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d36a1b0 0x55647d709100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d3494d0 0x55647d706270 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/1362530105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d705d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1362530105 >> 172.21.15.94:0/1362530105 conn(0x55647d35e300 msgr2=0x55647d70ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1362530105 shutdown_connections 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- 172.21.15.94:0/1362530105 wait complete. 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- start start 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 Processor -- start 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d7186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 0x55647d718bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941aa660 con 0x55647d36a1b0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941a9f90 con 0x55647d3412f0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941a71d0 con 0x55647d3494d0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9fa17740 10 client.0 _ll_register_callbacks cb 0x7f0d84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.085+0000 7f0d9259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 0x55647d718bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59404/0 (socket says 172.21.15.94:59404) 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d91d9b640 1 -- 172.21.15.94:0/2552304841 learned_addr learned my addr 172.21.15.94:0/2552304841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d7186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 -- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 msgr2=0x55647d7186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d7186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 -- 172.21.15.94:0/2552304841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 msgr2=0x55647d719ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 -- 172.21.15.94:0/2552304841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647d71a090 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9259c640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 0x55647d718bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009e50 tx=0x7f0d84005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634ab6a0ef0e99fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84010840 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d84010a80 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0d84005050 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d84004490 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9fa17740 10 client.25738 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.086+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55647d71b2e0 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84010a80 con 0x55647d3494d0 2024-04-25T13:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d8402b430 con 0x55647d3494d0 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 10 client.25738 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 20 client.25738 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d2fe1c0 0x55647d78c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 4 client.25738 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55647d677ee0 con 0x55647d2fe1c0 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d2fe1c0 0x55647d78c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d79ffb640 10 client.25738 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.087+0000 7f0d91d9b640 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d2fe1c0 0x55647d78c790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e341f0296ef82218 server_cookie=c2770feb239332e4 in_seq=0 out_seq=0 2024-04-25T13:44:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d79ffb640 1 -- 172.21.15.94:0/2552304841 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55647d677ee0 con 0x55647d2fe1c0 2024-04-25T13:44:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d79ffb640 10 client.25738 handle_command_reply: tid=0 2024-04-25T13:44:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 client.25738 shutdown 2024-04-25T13:44:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 msgr2=0x55647d718bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 0x55647d718bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009e50 tx=0x7f0d84005b60 comp rx=0 tx=0).stop 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 shutdown_connections 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55647d2fe1c0 0x55647d78c790 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55647d36a1b0 0x55647d719ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55647d3494d0 0x55647d718bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d79ffb640 0 client.25738 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 --2- 172.21.15.94:0/2552304841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55647d3412f0 0x55647d7186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.277+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 >> 172.21.15.94:0/2552304841 conn(0x55647d35e300 msgr2=0x55647d705c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 shutdown_connections 2024-04-25T13:44:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d9fa17740 1 -- 172.21.15.94:0/2552304841 wait complete. 2024-04-25T13:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d9fa17740 20 client.25738 trim_cache size 0 max 16384 2024-04-25T13:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:44:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 msgr2=0x7f0d7c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 0x7f0d7c0b1ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0a49a0 tx=0x7f0d8c076280 comp rx=0 tx=0).stop 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 msgr2=0x7f0d941a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00eb10 tx=0x7f0d8c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 shutdown_connections 2024-04-25T13:44:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d7c0afae0 0x7f0d7c0b1ff0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d941044b0 0x7f0d941a0630 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d94103990 0x7f0d941996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 --2- 172.21.15.94:0/4234743668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d94102fb0 0x7f0d94199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 >> 172.21.15.94:0/4234743668 conn(0x7f0d940fd390 msgr2=0x7f0d94104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 shutdown_connections 2024-04-25T13:44:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:27.278+0000 7f0d98b6c640 1 -- 172.21.15.94:0/4234743668 wait complete. 2024-04-25T13:44:27.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 186885 14.61 MB/sec warmup 70 sec latency 41.220 ms 2024-04-25T13:44:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:28 smithi137 ceph-mon[32568]: pgmap v426: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-04-25T13:44:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:28 smithi094 ceph-mon[29195]: pgmap v426: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-04-25T13:44:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:28 smithi138 ceph-mon[32977]: pgmap v426: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-04-25T13:44:28.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 189064 14.60 MB/sec warmup 71 sec latency 17.896 ms 2024-04-25T13:44:29.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191875 14.63 MB/sec warmup 72 sec latency 90.476 ms 2024-04-25T13:44:30.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:44:30.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 -- 172.21.15.94:0/613422777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc108920 msgr2=0x7f1ecc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/613422777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc108920 0x7f1ecc10ad40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009be0 tx=0x7f1ec402a6c0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 -- 172.21.15.94:0/613422777 shutdown_connections 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/613422777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc108920 0x7f1ecc10ad40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/613422777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 0x7f1ecc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/613422777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 -- 172.21.15.94:0/613422777 >> 172.21.15.94:0/613422777 conn(0x7f1ecc0fdbd0 msgr2=0x7f1ecc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 -- 172.21.15.94:0/613422777 shutdown_connections 2024-04-25T13:44:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.350+0000 7f1ed22aa640 1 -- 172.21.15.94:0/613422777 wait complete. 2024-04-25T13:44:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.351+0000 7f1ed22aa640 1 Processor -- start 2024-04-25T13:44:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.351+0000 7f1ed22aa640 1 -- start start 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 0x7f1ecc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc108920 0x7f1ecc1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10e510 con 0x7f1ecc1029e0 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10e390 con 0x7f1ecc108920 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10e690 con 0x7f1ecc103c20 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59424/0 (socket says 172.21.15.94:59424) 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1123617326 learned_addr learned my addr 172.21.15.94:0/1123617326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc108920 0x7f1ecc1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 0x7f1ecc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1123617326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 msgr2=0x7f1ecc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 0x7f1ecc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1123617326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc108920 msgr2=0x7f1ecc1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc108920 0x7f1ecc1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1123617326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1a0f20 con 0x7f1ecc1029e0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9c0 tx=0x7f1ebc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ec97fa640 1 -- 172.21.15.94:0/1123617326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00ccc0 con 0x7f1ecc1029e0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ec97fa640 1 -- 172.21.15.94:0/1123617326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0035f0 con 0x7f1ecc1029e0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1a1210 con 0x7f1ecc1029e0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ec97fa640 1 -- 172.21.15.94:0/1123617326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00ce60 con 0x7f1ecc1029e0 2024-04-25T13:44:30.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.352+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a1760 con 0x7f1ecc1029e0 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 Processor -- start 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ec97fa640 1 -- 172.21.15.94:0/1123617326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1ebc010750 con 0x7f1ecc1029e0 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 -- start start 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x562463616a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x562463616fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56246357bd80 con 0x56246324c4d0 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624635358b0 con 0x562463275390 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed9233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562463577d40 con 0x5624632546b0 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x562463616fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed0847640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x562463616a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x562463616fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59430/0 (socket says 172.21.15.94:59430) 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ed0847640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x562463616a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54786/0 (socket says 172.21.15.94:54786) 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34044/0 (socket says 172.21.15.94:34044) 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.355+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1489779604 learned_addr learned my addr 172.21.15.94:0/1489779604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/1489779604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 msgr2=0x562463616a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x562463616a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/1489779604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 msgr2=0x562463616fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x562463616fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ec97fa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 0x7f1eb40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/1489779604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562463506a40 con 0x5624632546b0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 0x7f1eb40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ec97fa640 1 -- 172.21.15.94:0/1123617326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1ebc0deec0 con 0x7f1ecc1029e0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009bd0 tx=0x7f1ec0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d1c9b65ede0fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1eb27fc640 1 -- 172.21.15.94:0/1489779604 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004230 con 0x5624632546b0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1eb27fc640 1 -- 172.21.15.94:0/1489779604 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec0009e40 con 0x5624632546b0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.356+0000 7f1eb27fc640 1 -- 172.21.15.94:0/1489779604 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004c60 con 0x5624632546b0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- 172.21.15.94:0/1489779604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 msgr2=0x562463617c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 0x7f1eb40b2000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec4030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009bd0 tx=0x7f1ec0011810 comp rx=0 tx=0).stop 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- 172.21.15.94:0/1489779604 shutdown_connections 2024-04-25T13:44:30.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x562463617c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x562463616fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1489779604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x562463616a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- 172.21.15.94:0/1489779604 >> 172.21.15.94:0/1489779604 conn(0x5624632694e0 msgr2=0x562463612270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- 172.21.15.94:0/1489779604 shutdown_connections 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- 172.21.15.94:0/1489779604 wait complete. 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.357+0000 7f1ed9233740 1 -- start start 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 Processor -- start 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x5624636208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463627d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562463618780 con 0x5624632546b0 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562463502d10 con 0x56246324c4d0 2024-04-25T13:44:30.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562463618900 con 0x562463275390 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed9233740 10 client.0 _ll_register_callbacks cb 0x56246352f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59436/0 (socket says 172.21.15.94:59436) 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/2413676690 learned_addr learned my addr 172.21.15.94:0/2413676690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x5624636208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed0847640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463627d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/2413676690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 msgr2=0x562463627d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463627d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 msgr2=0x5624636208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x5624636208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecb7fe640 1 -- 172.21.15.94:0/2413676690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624633577d0 con 0x5624632546b0 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x5624636208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.358+0000 7f1ed0847640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463627d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00058c0 tx=0x7f1ec0005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64922410dab48dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0027020 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1ec00037f0 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562463589990 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ec0032050 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 10 client.16335 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56246328d3f0 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562463589990 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ec0018070 con 0x5624632546b0 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 20 client.16335 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:30.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562463233f60 0x56246362e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 4 client.16335 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.359+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562463583800 con 0x562463233f60 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.360+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562463233f60 0x56246362e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.360+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562463233f60 0x56246362e100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6ca429bb6b5c16 server_cookie=982298e52e400a70 in_seq=0 out_seq=0 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.360+0000 7f1eab7fe640 10 client.16335 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.361+0000 7f1eab7fe640 1 -- 172.21.15.94:0/2413676690 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562463583800 con 0x562463233f60 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.361+0000 7f1eab7fe640 10 client.16335 handle_command_reply: tid=0 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.361+0000 7f1ed9233740 1 client.16335 shutdown 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 msgr2=0x562463620e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00058c0 tx=0x7f1ec0005670 comp rx=0 tx=0).stop 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 shutdown_connections 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562463233f60 0x56246362e100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463627d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1eab7fe640 0 client.16335 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463620e20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 --2- 172.21.15.94:0/2413676690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x5624636208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 >> 172.21.15.94:0/2413676690 conn(0x5624632694e0 msgr2=0x5624636113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 shutdown_connections 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 1 -- 172.21.15.94:0/2413676690 wait complete. 2024-04-25T13:44:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.362+0000 7f1ed9233740 20 client.16335 trim_cache size 0 max 16384 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 Processor -- start 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 -- start start 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x56246361bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x5624636169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1aa3e0 con 0x56246324c4d0 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1abf20 con 0x562463275390 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed9233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1aaab0 con 0x5624632546b0 2024-04-25T13:44:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59438/0 (socket says 172.21.15.94:59438) 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 -- 172.21.15.94:0/3673126453 learned_addr learned my addr 172.21.15.94:0/3673126453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ed0847640 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x5624636169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x56246361bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 -- 172.21.15.94:0/3673126453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 msgr2=0x56246361bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x56246361bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 -- 172.21.15.94:0/3673126453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 msgr2=0x5624636169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x5624636169e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.381+0000 7f1ecbfff640 1 -- 172.21.15.94:0/3673126453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246324d0f0 con 0x56246324c4d0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.382+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009720 tx=0x7f1ebc0e2570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd900b794185f161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.382+0000 7f1eb1ffb640 1 -- 172.21.15.94:0/3673126453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a6a70 con 0x56246324c4d0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.382+0000 7f1eb1ffb640 1 -- 172.21.15.94:0/3673126453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0a6e10 con 0x56246324c4d0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.382+0000 7f1eb1ffb640 1 -- 172.21.15.94:0/3673126453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003aa0 con 0x56246324c4d0 2024-04-25T13:44:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- 172.21.15.94:0/3673126453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 msgr2=0x56246361b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009720 tx=0x7f1ebc0e2570 comp rx=0 tx=0).stop 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- 172.21.15.94:0/3673126453 shutdown_connections 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562463275390 0x5624636169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624632546b0 0x56246361bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- 172.21.15.94:0/3673126453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56246324c4d0 0x56246361b640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- 172.21.15.94:0/3673126453 >> 172.21.15.94:0/3673126453 conn(0x5624632694e0 msgr2=0x562463611ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- 172.21.15.94:0/3673126453 shutdown_connections 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- 172.21.15.94:0/3673126453 wait complete. 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- start start 2024-04-25T13:44:30.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 Processor -- start 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x562463626ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463624020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1ab850 con 0x5624632546b0 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1ab180 con 0x56246324c4d0 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.383+0000 7f1ed9233740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1a83c0 con 0x562463275390 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed9233740 10 client.0 _ll_register_callbacks cb 0x7f1ebc0e2870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x562463626ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463624020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x562463626ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54804/0 (socket says 172.21.15.94:54804) 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecbfff640 1 -- 172.21.15.94:0/1488050096 learned_addr learned my addr 172.21.15.94:0/1488050096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59450/0 (socket says 172.21.15.94:59450) 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 -- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 msgr2=0x562463626ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x562463626ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 -- 172.21.15.94:0/1488050096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 msgr2=0x562463623ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 -- 172.21.15.94:0/1488050096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246353c4b0 con 0x562463275390 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ecb7fe640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed0847640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463624020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ec402abe0 tx=0x7f1ec4009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689768c836645557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4036040 con 0x562463275390 2024-04-25T13:44:30.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1ec4004510 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ebc0a6e10 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.384+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ec4004710 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 10 client.25753 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562463695120 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0a6e10 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ec4054020 con 0x562463275390 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 10 client.25753 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 20 client.25753 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5624632093a0 0x562463697af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 4 client.25753 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624634fe4a0 con 0x5624632093a0 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5624632093a0 0x562463697af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1eab7fe640 10 client.25753 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.385+0000 7f1ecbfff640 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5624632093a0 0x562463697af0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e496bb4f368e44 server_cookie=15936b16ac6a116f in_seq=0 out_seq=0 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1eab7fe640 1 -- 172.21.15.94:0/1488050096 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5624634fe4a0 con 0x5624632093a0 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1eab7fe640 10 client.25753 handle_command_reply: tid=0 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 client.25753 shutdown 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 msgr2=0x562463624020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463624020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ec402abe0 tx=0x7f1ec4009500 comp rx=0 tx=0).stop 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 shutdown_connections 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5624632093a0 0x562463697af0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562463275390 0x562463624020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624632546b0 0x562463623ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1eab7fe640 0 client.25753 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:30.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 --2- 172.21.15.94:0/1488050096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56246324c4d0 0x562463626ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 >> 172.21.15.94:0/1488050096 conn(0x5624632694e0 msgr2=0x5624636169e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 shutdown_connections 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 1 -- 172.21.15.94:0/1488050096 wait complete. 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.386+0000 7f1ed9233740 20 client.25753 trim_cache size 0 max 16384 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:44:30.393 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 msgr2=0x7f1eb40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 0x7f1eb40b2000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec4030970 comp rx=0 tx=0).stop 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 msgr2=0x7f1ecc1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9c0 tx=0x7f1ebc00ee80 comp rx=0 tx=0).stop 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 shutdown_connections 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1eb40afaf0 0x7f1eb40b2000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ecc108920 0x7f1ecc1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc103c20 0x7f1ecc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 --2- 172.21.15.94:0/1123617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1029e0 0x7f1ecc1993e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 >> 172.21.15.94:0/1123617326 conn(0x7f1ecc0fdbd0 msgr2=0x7f1ecc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 shutdown_connections 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.387+0000 7f1ed22aa640 1 -- 172.21.15.94:0/1123617326 wait complete. 2024-04-25T13:44:30.398 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:44:30.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 -- 172.21.15.94:0/375800716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078108900 msgr2=0x7f507810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 --2- 172.21.15.94:0/375800716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078108900 0x7f507810ad20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5070009ca0 tx=0x7f507002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 -- 172.21.15.94:0/375800716 shutdown_connections 2024-04-25T13:44:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 --2- 172.21.15.94:0/375800716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078108900 0x7f507810ad20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 --2- 172.21.15.94:0/375800716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5078103c00 0x7f5078104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 --2- 172.21.15.94:0/375800716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50781029c0 0x7f50781036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.511+0000 7f507f615640 1 -- 172.21.15.94:0/375800716 >> 172.21.15.94:0/375800716 conn(0x7f50780fdbd0 msgr2=0x7f5078100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- 172.21.15.94:0/375800716 shutdown_connections 2024-04-25T13:44:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- 172.21.15.94:0/375800716 wait complete. 2024-04-25T13:44:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 Processor -- start 2024-04-25T13:44:30.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- start start 2024-04-25T13:44:30.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50781029c0 0x7f507819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5078108900 0x7f50781a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f507810f5f0 con 0x7f5078103c00 2024-04-25T13:44:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f507810f470 con 0x7f50781029c0 2024-04-25T13:44:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507f615640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f507810f770 con 0x7f5078108900 2024-04-25T13:44:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507cbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507cbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59470/0 (socket says 172.21.15.94:59470) 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507cbb0640 1 -- 172.21.15.94:0/1155885140 learned_addr learned my addr 172.21.15.94:0/1155885140 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.512+0000 7f507d3b1640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50781029c0 0x7f507819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507dbb2640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5078108900 0x7f50781a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 -- 172.21.15.94:0/1155885140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5078108900 msgr2=0x7f50781a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5078108900 0x7f50781a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 -- 172.21.15.94:0/1155885140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50781029c0 msgr2=0x7f507819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50781029c0 0x7f507819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 -- 172.21.15.94:0/1155885140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781a1d00 con 0x7f5078103c00 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507cbb0640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f506800e9f0 tx=0x7f506800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506800cd80 con 0x7f5078103c00 2024-04-25T13:44:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50680108e0 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781a1ff0 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068018980 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.513+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a2460 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.514+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5078107710 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.516+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5068010430 con 0x7f5078103c00 2024-04-25T13:44:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.517+0000 7f506e7fc640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 0x7f50640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.517+0000 7f507d3b1640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 0x7f50640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.517+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f50680df470 con 0x7f5078103c00 2024-04-25T13:44:30.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.517+0000 7f507d3b1640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 0x7f50640b2000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50781035c0 tx=0x7f5074005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.521+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50680a7960 con 0x7f5078103c00 2024-04-25T13:44:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:30 smithi137 ceph-mon[32568]: pgmap v427: 97 pgs: 97 active+clean; 45 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-25T13:44:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:30 smithi094 ceph-mon[29195]: pgmap v427: 97 pgs: 97 active+clean; 45 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-25T13:44:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:30 smithi138 ceph-mon[32977]: pgmap v427: 97 pgs: 97 active+clean; 45 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-04-25T13:44:30.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.756+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f507802dd10 con 0x7f5078103c00 2024-04-25T13:44:30.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.756+0000 7f506e7fc640 1 -- 172.21.15.94:0/1155885140 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7f50680abc60 con 0x7f5078103c00 2024-04-25T13:44:30.757 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:44:30.757 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:44:30.757 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:44:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 msgr2=0x7f50640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 0x7f50640b2000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50781035c0 tx=0x7f5074005f90 comp rx=0 tx=0).stop 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 msgr2=0x7f507819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f506800e9f0 tx=0x7f506800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 shutdown_connections 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50640afaf0 0x7f50640b2000 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5078108900 0x7f50781a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5078103c00 0x7f507819a700 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 --2- 172.21.15.94:0/1155885140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50781029c0 0x7f507819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.758+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 >> 172.21.15.94:0/1155885140 conn(0x7f50780fdbd0 msgr2=0x7f50781048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.760+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 shutdown_connections 2024-04-25T13:44:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.760+0000 7f507f615640 1 -- 172.21.15.94:0/1155885140 wait complete. 2024-04-25T13:44:30.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:44:30.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 194476 14.60 MB/sec warmup 73 sec latency 38.986 ms 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 -- 172.21.15.94:0/1669420262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c108920 msgr2=0x7fbb5c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 --2- 172.21.15.94:0/1669420262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c108920 0x7fbb5c10ad40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009ca0 tx=0x7fbb5402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 -- 172.21.15.94:0/1669420262 shutdown_connections 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 --2- 172.21.15.94:0/1669420262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c108920 0x7fbb5c10ad40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 --2- 172.21.15.94:0/1669420262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c103c20 0x7fbb5c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 --2- 172.21.15.94:0/1669420262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 -- 172.21.15.94:0/1669420262 >> 172.21.15.94:0/1669420262 conn(0x7fbb5c0fdbd0 msgr2=0x7fbb5c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 -- 172.21.15.94:0/1669420262 shutdown_connections 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 -- 172.21.15.94:0/1669420262 wait complete. 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.888+0000 7fbb63b08640 1 Processor -- start 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 -- start start 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c108920 0x7fbb5c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10d400 con 0x7fbb5c103c20 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10d280 con 0x7fbb5c108920 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb63b08640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10d580 con 0x7fbb5c1029e0 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59484/0 (socket says 172.21.15.94:59484) 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb610a3640 1 -- 172.21.15.94:0/3036101327 learned_addr learned my addr 172.21.15.94:0/3036101327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb620a5640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c108920 0x7fbb5c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 -- 172.21.15.94:0/3036101327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c108920 msgr2=0x7fbb5c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c108920 0x7fbb5c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 -- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 msgr2=0x7fbb5c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 -- 172.21.15.94:0/3036101327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c19fe30 con 0x7fbb5c1029e0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb610a3640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.889+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1982f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c1022b0 tx=0x7fbb58010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.890+0000 7fbb52ffd640 1 -- 172.21.15.94:0/3036101327 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58004230 con 0x7fbb5c1029e0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.890+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c1a0120 con 0x7fbb5c1029e0 2024-04-25T13:44:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.890+0000 7fbb52ffd640 1 -- 172.21.15.94:0/3036101327 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5800edc0 con 0x7fbb5c1029e0 2024-04-25T13:44:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.890+0000 7fbb52ffd640 1 -- 172.21.15.94:0/3036101327 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58016480 con 0x7fbb5c1029e0 2024-04-25T13:44:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.890+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c1a0670 con 0x7fbb5c1029e0 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.893+0000 7fbb6a9b7740 1 Processor -- start 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.893+0000 7fbb6a9b7740 1 -- start start 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.893+0000 7fbb52ffd640 1 -- 172.21.15.94:0/3036101327 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbb58014050 con 0x7fbb5c1029e0 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b1c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 0x55daa4b1da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55daa4a80d80 con 0x55daa47514d0 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daa4a3aa20 con 0x55daa477a390 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daa4a7cd40 con 0x55daa47596b0 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59486/0 (socket says 172.21.15.94:59486) 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb618a4640 1 -- 172.21.15.94:0/1648875450 learned_addr learned my addr 172.21.15.94:0/1648875450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 0x55daa4b1da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb620a5640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b1c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 -- 172.21.15.94:0/1648875450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 msgr2=0x55daa4b1c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b1c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 -- 172.21.15.94:0/1648875450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 msgr2=0x55daa4b1cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 -- 172.21.15.94:0/1648875450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa4a45c30 con 0x55daa47596b0 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb618a4640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb52ffd640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 0x7fbb480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb620a5640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 0x7fbb480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb610a3640 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 0x55daa4b1da20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c199390 tx=0x7fbb4c0119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86f3ada3b479d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb52ffd640 1 -- 172.21.15.94:0/3036101327 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbb580e1090 con 0x7fbb5c1029e0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.894+0000 7fbb47fff640 1 -- 172.21.15.94:0/1648875450 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004030 con 0x55daa47596b0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb47fff640 1 -- 172.21.15.94:0/1648875450 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c005020 con 0x55daa47596b0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb47fff640 1 -- 172.21.15.94:0/1648875450 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004d30 con 0x55daa47596b0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/1648875450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 msgr2=0x55daa4b1da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb620a5640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 0x7fbb480b2080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55daa4b1d860 tx=0x7fbb54009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 0x55daa4b1da20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c199390 tx=0x7fbb4c0119a0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/1648875450 shutdown_connections 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47596b0 0x55daa4b1da20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47514d0 0x55daa4b1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/1648875450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b1c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.895+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/1648875450 >> 172.21.15.94:0/1648875450 conn(0x55daa476e4e0 msgr2=0x55daa4b18020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/1648875450 shutdown_connections 2024-04-25T13:44:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/1648875450 wait complete. 2024-04-25T13:44:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- start start 2024-04-25T13:44:30.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 Processor -- start 2024-04-25T13:44:30.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55daa4b1e530 con 0x55daa47596b0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daa4a07dc0 con 0x55daa47514d0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daa4b1e6b0 con 0x55daa477a390 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.896+0000 7fbb6a9b7740 10 client.0 _ll_register_callbacks cb 0x55daa4a23600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59500/0 (socket says 172.21.15.94:59500) 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb610a3640 1 -- 172.21.15.94:0/168053256 learned_addr learned my addr 172.21.15.94:0/168053256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb620a5640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 -- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 msgr2=0x55daa4b2d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b2d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 -- 172.21.15.94:0/168053256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 msgr2=0x55daa4b26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 -- 172.21.15.94:0/168053256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa47923f0 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb610a3640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb618a4640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b26530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb580deb40 tx=0x7fbb58003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f792bbefa6aba54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb457fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a92a0 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbb580e4d20 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4c005020 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.897+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb580a8c40 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 10 client.25841 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55daa485c7d0 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c005020 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb58027060 con 0x55daa47514d0 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 10 client.25841 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 10 client.25841 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:44:30.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 20 client.25841 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa4738f60 0x55daa4b34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 4 client.25841 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daa4a8e990 con 0x55daa4738f60 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.898+0000 7fbb610a3640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa4738f60 0x55daa4b34230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb457fa640 10 client.25841 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb610a3640 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa4738f60 0x55daa4b34230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eea0ac22328aeebc server_cookie=6f7d39fb4ee02fce in_seq=0 out_seq=0 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb457fa640 1 -- 172.21.15.94:0/168053256 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55daa4a8e990 con 0x55daa4738f60 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb457fa640 10 client.25841 handle_command_reply: tid=0 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 client.25841 shutdown 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 msgr2=0x55daa4b26530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b26530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb580deb40 tx=0x7fbb58003ed0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 shutdown_connections 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa4738f60 0x55daa4b34230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/168053256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b26530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb457fa640 0 client.25841 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.899+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 >> 172.21.15.94:0/168053256 conn(0x55daa476e4e0 msgr2=0x55daa4b17040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.900+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 shutdown_connections 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.900+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/168053256 wait complete. 2024-04-25T13:44:30.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.900+0000 7fbb6a9b7740 20 client.25841 trim_cache size 0 max 16384 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 Processor -- start 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 -- start start 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 0x55daa4b13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1a92e0 con 0x55daa47596b0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1aae20 con 0x55daa477a390 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1a99b0 con 0x55daa47514d0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 0x55daa4b13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59504/0 (socket says 172.21.15.94:59504) 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb610a3640 1 -- 172.21.15.94:0/3914723565 learned_addr learned my addr 172.21.15.94:0/3914723565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.913+0000 7fbb620a5640 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 -- 172.21.15.94:0/3914723565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 msgr2=0x55daa4b26640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b26640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 -- 172.21.15.94:0/3914723565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 msgr2=0x55daa4b26100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 -- 172.21.15.94:0/3914723565 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa4b12d30 con 0x55daa47514d0 2024-04-25T13:44:30.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb610a3640 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:30.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb618a4640 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 0x55daa4b13190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb580ada40 tx=0x7fbb58092150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d94bdd6aa3052a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb477fe640 1 -- 172.21.15.94:0/3914723565 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a8e20 con 0x55daa47514d0 2024-04-25T13:44:30.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb477fe640 1 -- 172.21.15.94:0/3914723565 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb580a8fe0 con 0x55daa47514d0 2024-04-25T13:44:30.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.914+0000 7fbb477fe640 1 -- 172.21.15.94:0/3914723565 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a8730 con 0x55daa47514d0 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/3914723565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 msgr2=0x55daa4b13190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 0x55daa4b13190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb580ada40 tx=0x7fbb58092150 comp rx=0 tx=0).stop 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/3914723565 shutdown_connections 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa477a390 0x55daa4b26640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b26100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/3914723565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa47514d0 0x55daa4b13190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/3914723565 >> 172.21.15.94:0/3914723565 conn(0x55daa476e4e0 msgr2=0x55daa4b21290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/3914723565 shutdown_connections 2024-04-25T13:44:30.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/3914723565 wait complete. 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 -- start start 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.915+0000 7fbb6a9b7740 1 Processor -- start 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b29dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b9a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1aa750 con 0x55daa47596b0 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1aa080 con 0x55daa47514d0 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1a72c0 con 0x55daa477a390 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb6a9b7740 10 client.0 _ll_register_callbacks cb 0x7fbb58003850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59508/0 (socket says 172.21.15.94:59508) 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb618a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b29dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 -- 172.21.15.94:0/2892825498 learned_addr learned my addr 172.21.15.94:0/2892825498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:30.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb620a5640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b9a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 -- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 msgr2=0x55daa4b9a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b9a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 -- 172.21.15.94:0/2892825498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 msgr2=0x55daa4b29dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b29dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 -- 172.21.15.94:0/2892825498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa4b2d2c0 con 0x55daa47596b0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb610a3640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01b040 tx=0x7fbb4c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c064d7133748a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.916+0000 7fbb457fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011240 con 0x55daa47596b0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbb4c016040 con 0x55daa47596b0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb580a8fe0 con 0x55daa47596b0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c005370 con 0x55daa47596b0 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 10 client.16365 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:30.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55daa4b9bae0 con 0x55daa47596b0 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb580a8fe0 con 0x55daa47596b0 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb4c004050 con 0x55daa47596b0 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 10 client.16365 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 10 client.16365 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 20 client.16365 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa470e3a0 0x55daa4b9e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 4 client.16365 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55daa4a88800 con 0x55daa470e3a0 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.917+0000 7fbb618a4640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa470e3a0 0x55daa4b9e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb457fa640 10 client.16365 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:44:30.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb618a4640 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa470e3a0 0x55daa4b9e3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d921b82304fc926 server_cookie=ef0fd98ab6ea88e3 in_seq=0 out_seq=0 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb457fa640 1 -- 172.21.15.94:0/2892825498 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55daa4a88800 con 0x55daa470e3a0 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb457fa640 10 client.16365 handle_command_reply: tid=0 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 client.16365 shutdown 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 msgr2=0x55daa4b2a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01b040 tx=0x7fbb4c004ee0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 shutdown_connections 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55daa470e3a0 0x55daa4b9e3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa477a390 0x55daa4b9a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb457fa640 0 client.16365 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55daa47596b0 0x55daa4b2a300 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 --2- 172.21.15.94:0/2892825498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daa47514d0 0x55daa4b29dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 >> 172.21.15.94:0/2892825498 conn(0x55daa476e4e0 msgr2=0x55daa4b26100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 shutdown_connections 2024-04-25T13:44:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 1 -- 172.21.15.94:0/2892825498 wait complete. 2024-04-25T13:44:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.918+0000 7fbb6a9b7740 20 client.16365 trim_cache size 0 max 16384 2024-04-25T13:44:30.927 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 msgr2=0x7fbb480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 0x7fbb480b2080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55daa4b1d860 tx=0x7fbb54009380 comp rx=0 tx=0).stop 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 msgr2=0x7fbb5c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1982f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c1022b0 tx=0x7fbb58010f30 comp rx=0 tx=0).stop 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 shutdown_connections 2024-04-25T13:44:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbb480afb70 0x7fbb480b2080 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb5c108920 0x7fbb5c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb5c103c20 0x7fbb5c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 --2- 172.21.15.94:0/3036101327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c1029e0 0x7fbb5c1982f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 >> 172.21.15.94:0/3036101327 conn(0x7fbb5c0fdbd0 msgr2=0x7fbb5c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 shutdown_connections 2024-04-25T13:44:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:30.919+0000 7fbb63b08640 1 -- 172.21.15.94:0/3036101327 wait complete. 2024-04-25T13:44:30.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- 172.21.15.94:0/366941950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 msgr2=0x7fa8a410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/366941950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a410ad60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009ca0 tx=0x7fa89c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- 172.21.15.94:0/366941950 shutdown_connections 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/366941950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a410ad60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/366941950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4103c40 0x7fa8a4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/366941950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4102a00 0x7fa8a4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- 172.21.15.94:0/366941950 >> 172.21.15.94:0/366941950 conn(0x7fa8a40fdbf0 msgr2=0x7fa8a41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- 172.21.15.94:0/366941950 shutdown_connections 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- 172.21.15.94:0/366941950 wait complete. 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 Processor -- start 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.087+0000 7fa8aa97d640 1 -- start start 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4102a00 0x7fa8a41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4103c40 0x7fa8a4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410e530 con 0x7fa8a4108940 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410e3b0 con 0x7fa8a4103c40 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8aa97d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a410e6b0 con 0x7fa8a4102a00 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59518/0 (socket says 172.21.15.94:59518) 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/3438160947 learned_addr learned my addr 172.21.15.94:0/3438160947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4102a00 0x7fa8a41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4103c40 0x7fa8a4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/3438160947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4102a00 msgr2=0x7fa8a41990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4102a00 0x7fa8a41990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/3438160947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4103c40 msgr2=0x7fa8a4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4103c40 0x7fa8a4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/3438160947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a41a0bd0 con 0x7fa8a4108940 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.088+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa89c004af0 tx=0x7fa89c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.089+0000 7fa8a17fa640 1 -- 172.21.15.94:0/3438160947 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c003b70 con 0x7fa8a4108940 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.089+0000 7fa8a17fa640 1 -- 172.21.15.94:0/3438160947 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89c003d10 con 0x7fa8a4108940 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.089+0000 7fa8a17fa640 1 -- 172.21.15.94:0/3438160947 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c02fb20 con 0x7fa8a4108940 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.089+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a41a0e60 con 0x7fa8a4108940 2024-04-25T13:44:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.089+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a41a1320 con 0x7fa8a4108940 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8a17fa640 1 -- 172.21.15.94:0/3438160947 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa89c004580 con 0x7fa8a4108940 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 Processor -- start 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 -- start start 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec520810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec521a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5639ec484d80 con 0x5639ec1554d0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5639ec43ea20 con 0x5639ec17e390 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8b182f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639ec480d40 con 0x5639ec15d6b0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.091+0000 7fa8a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec520810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59522/0 (socket says 172.21.15.94:59522) 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 -- 172.21.15.94:0/3814727545 learned_addr learned my addr 172.21.15.94:0/3814727545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec521a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 -- 172.21.15.94:0/3814727545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 msgr2=0x5639ec521a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec521a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 -- 172.21.15.94:0/3814727545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 msgr2=0x5639ec520810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec520810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 -- 172.21.15.94:0/3814727545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ec449c30 con 0x5639ec1554d0 2024-04-25T13:44:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a17fa640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 0x7fa88c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa894009c60 tx=0x7fa894011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5afa44545fd347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 0x7fa88c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa8a17fa640 1 -- 172.21.15.94:0/3438160947 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa89c100210 con 0x7fa8a4108940 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa88a7fc640 1 -- 172.21.15.94:0/3814727545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894004220 con 0x5639ec1554d0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa88a7fc640 1 -- 172.21.15.94:0/3814727545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8940043c0 con 0x5639ec1554d0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.092+0000 7fa88a7fc640 1 -- 172.21.15.94:0/3814727545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894004d30 con 0x5639ec1554d0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 -- 172.21.15.94:0/3814727545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 msgr2=0x5639ec520d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 0x7fa88c0b2080 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5639ec521860 tx=0x7fa898005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa894009c60 tx=0x7fa894011810 comp rx=0 tx=0).stop 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 -- 172.21.15.94:0/3814727545 shutdown_connections 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec521a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec520d90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 --2- 172.21.15.94:0/3814727545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec520810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 -- 172.21.15.94:0/3814727545 >> 172.21.15.94:0/3814727545 conn(0x5639ec1724e0 msgr2=0x5639ec51c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 -- 172.21.15.94:0/3814727545 shutdown_connections 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.093+0000 7fa8b182f740 1 -- 172.21.15.94:0/3814727545 wait complete. 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 -- start start 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 Processor -- start 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec1554d0 0x5639ec52a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec15d6b0 0x5639ec52aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5639ec522530 con 0x5639ec17e390 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5639ec40bdc0 con 0x5639ec15d6b0 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639ec5226b0 con 0x5639ec1554d0 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8b182f740 10 client.0 _ll_register_callbacks cb 0x5639ec427600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8a8f1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8a8f1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59534/0 (socket says 172.21.15.94:59534) 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec1554d0 0x5639ec52a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/841627372 learned_addr learned my addr 172.21.15.94:0/841627372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.094+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec15d6b0 0x5639ec52aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/841627372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec1554d0 msgr2=0x5639ec52a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec1554d0 0x5639ec52a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec15d6b0 msgr2=0x5639ec52aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec15d6b0 0x5639ec52aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 -- 172.21.15.94:0/841627372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ec1963f0 con 0x5639ec17e390 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0c8350 tx=0x7fa89c0c8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508477800f726430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0c8d50 con 0x5639ec17e390 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa89c0b10b0 con 0x5639ec17e390 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8940043c0 con 0x5639ec17e390 2024-04-25T13:44:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa89c0c8650 con 0x5639ec17e390 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8b182f740 10 client.16377 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639ec2607d0 con 0x5639ec17e390 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8940043c0 con 0x5639ec17e390 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa89c0c8940 con 0x5639ec17e390 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8b182f740 10 client.16377 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.095+0000 7fa8b182f740 10 client.16377 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8b182f740 20 client.16377 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec13cf60 0x5639ec5382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8b182f740 4 client.16377 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639ec492990 con 0x5639ec13cf60 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec13cf60 0x5639ec5382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec13cf60 0x5639ec5382a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cef956b2eb740d89 server_cookie=5f172856d6579649 in_seq=0 out_seq=0 2024-04-25T13:44:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.096+0000 7fa8837fe640 10 client.16377 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8837fe640 1 -- 172.21.15.94:0/841627372 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639ec492990 con 0x5639ec13cf60 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8837fe640 10 client.16377 handle_command_reply: tid=0 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 client.16377 shutdown 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 msgr2=0x5639ec5319d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0c8350 tx=0x7fa89c0c8060 comp rx=0 tx=0).stop 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 shutdown_connections 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec13cf60 0x5639ec5382a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec17e390 0x5639ec5319d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec15d6b0 0x5639ec52aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 --2- 172.21.15.94:0/841627372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec1554d0 0x5639ec52a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.097+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 >> 172.21.15.94:0/841627372 conn(0x5639ec1724e0 msgr2=0x5639ec51ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.098+0000 7fa8837fe640 0 client.16377 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:44:31.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.098+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 shutdown_connections 2024-04-25T13:44:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.098+0000 7fa8b182f740 1 -- 172.21.15.94:0/841627372 wait complete. 2024-04-25T13:44:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.098+0000 7fa8b182f740 20 client.16377 trim_cache size 0 max 16384 2024-04-25T13:44:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 Processor -- start 2024-04-25T13:44:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 -- start start 2024-04-25T13:44:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec525790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec51b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41aa3c0 con 0x5639ec1554d0 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41abf00 con 0x5639ec17e390 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.112+0000 7fa8b182f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41aaa90 con 0x5639ec15d6b0 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59548/0 (socket says 172.21.15.94:59548) 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 -- 172.21.15.94:0/69075682 learned_addr learned my addr 172.21.15.94:0/69075682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec51b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec525790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 -- 172.21.15.94:0/69075682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 msgr2=0x5639ec525790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec525790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 -- 172.21.15.94:0/69075682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 msgr2=0x5639ec51b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec51b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 -- 172.21.15.94:0/69075682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ec5227a0 con 0x5639ec1554d0 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec525790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa894009730 tx=0x7fa8940096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaefa5e5eb58a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa889ffb640 1 -- 172.21.15.94:0/69075682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89401d040 con 0x5639ec1554d0 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa889ffb640 1 -- 172.21.15.94:0/69075682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8940055d0 con 0x5639ec1554d0 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.113+0000 7fa889ffb640 1 -- 172.21.15.94:0/69075682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894003920 con 0x5639ec1554d0 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- 172.21.15.94:0/69075682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 msgr2=0x5639ec525250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa894009730 tx=0x7fa8940096a0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- 172.21.15.94:0/69075682 shutdown_connections 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec51b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec525790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 --2- 172.21.15.94:0/69075682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec525250 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- 172.21.15.94:0/69075682 >> 172.21.15.94:0/69075682 conn(0x5639ec1724e0 msgr2=0x5639ec520790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- 172.21.15.94:0/69075682 shutdown_connections 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- 172.21.15.94:0/69075682 wait complete. 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.114+0000 7fa8b182f740 1 -- start start 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 Processor -- start 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec531450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec5a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41ab830 con 0x5639ec1554d0 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41ab160 con 0x5639ec17e390 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41a83a0 con 0x5639ec15d6b0 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8b182f740 10 client.0 _ll_register_callbacks cb 0x7fa89c009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59552/0 (socket says 172.21.15.94:59552) 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 -- 172.21.15.94:0/2952740948 learned_addr learned my addr 172.21.15.94:0/2952740948 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec5a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec531450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 -- 172.21.15.94:0/2952740948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 msgr2=0x5639ec531450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec531450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 -- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 msgr2=0x5639ec5a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec5a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a3fff640 1 -- 172.21.15.94:0/2952740948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ec5a0f90 con 0x5639ec1554d0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.115+0000 7fa8a8f1a640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec5a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8a3fff640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa894009250 tx=0x7fa894005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc2ed1e2db1f835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89401d040 con 0x5639ec1554d0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa89402c4f0 con 0x5639ec1554d0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8940055d0 con 0x5639ec1554d0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa89402c710 con 0x5639ec1554d0 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 10 client.16383 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639ec5a2210 con 0x5639ec1554d0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8940055d0 con 0x5639ec1554d0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa89402cce0 con 0x5639ec1554d0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 10 client.16383 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 10 client.16383 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 20 client.16383 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec1123a0 0x5639ec5a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 4 client.16383 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.116+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639ec48c800 con 0x5639ec1123a0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec1123a0 0x5639ec5a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8837fe640 10 client.16383 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8a37fe640 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec1123a0 0x5639ec5a4990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1209b0dc2c4e07bf server_cookie=a935d132f39e4c80 in_seq=0 out_seq=0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8837fe640 1 -- 172.21.15.94:0/2952740948 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639ec48c800 con 0x5639ec1123a0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8837fe640 10 client.16383 handle_command_reply: tid=0 2024-04-25T13:44:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 client.16383 shutdown 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 msgr2=0x5639ec530f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa894009250 tx=0x7fa894005150 comp rx=0 tx=0).stop 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 shutdown_connections 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5639ec1123a0 0x5639ec5a4990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639ec17e390 0x5639ec5a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.117+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639ec15d6b0 0x5639ec531450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8837fe640 0 client.16383 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8b182f740 1 --2- 172.21.15.94:0/2952740948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5639ec1554d0 0x5639ec530f10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 >> 172.21.15.94:0/2952740948 conn(0x5639ec1724e0 msgr2=0x5639ec52a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 shutdown_connections 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8b182f740 1 -- 172.21.15.94:0/2952740948 wait complete. 2024-04-25T13:44:31.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8b182f740 20 client.16383 trim_cache size 0 max 16384 2024-04-25T13:44:31.124 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:44:31.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 msgr2=0x7fa88c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 0x7fa88c0b2080 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5639ec521860 tx=0x7fa898005850 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 msgr2=0x7fa8a41a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa89c004af0 tx=0x7fa89c031700 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 shutdown_connections 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa88c0afb70 0x7fa88c0b2080 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a4108940 0x7fa8a41a0560 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a4103c40 0x7fa8a4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 --2- 172.21.15.94:0/3438160947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a4102a00 0x7fa8a41990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 >> 172.21.15.94:0/3438160947 conn(0x7fa8a40fdbf0 msgr2=0x7fa8a4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 shutdown_connections 2024-04-25T13:44:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.118+0000 7fa8aa97d640 1 -- 172.21.15.94:0/3438160947 wait complete. 2024-04-25T13:44:31.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:44:31.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 -- 172.21.15.94:0/3661856879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 msgr2=0x7fd6f01054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 --2- 172.21.15.94:0/3661856879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01054a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e002a410 comp rx=0 tx=0).stop 2024-04-25T13:44:31.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 -- 172.21.15.94:0/3661856879 shutdown_connections 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 --2- 172.21.15.94:0/3661856879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0102240 0x7fd6f010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 --2- 172.21.15.94:0/3661856879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0101740 0x7fd6f0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 --2- 172.21.15.94:0/3661856879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01054a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 -- 172.21.15.94:0/3661856879 >> 172.21.15.94:0/3661856879 conn(0x7fd6f00fd340 msgr2=0x7fd6f00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 -- 172.21.15.94:0/3661856879 shutdown_connections 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 -- 172.21.15.94:0/3661856879 wait complete. 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.245+0000 7fd6f501a640 1 Processor -- start 2024-04-25T13:44:31.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 -- start start 2024-04-25T13:44:31.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0101740 0x7fd6f019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0102240 0x7fd6f019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010f600 con 0x7fd6f0105090 2024-04-25T13:44:31.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010f480 con 0x7fd6f0101740 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6f501a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010f780 con 0x7fd6f0102240 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59568/0 (socket says 172.21.15.94:59568) 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0101740 0x7fd6f019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 -- 172.21.15.94:0/2814104016 learned_addr learned my addr 172.21.15.94:0/2814104016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0102240 0x7fd6f019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 -- 172.21.15.94:0/2814104016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0102240 msgr2=0x7fd6f019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0102240 0x7fd6f019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 -- 172.21.15.94:0/2814104016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0101740 msgr2=0x7fd6f019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0101740 0x7fd6f019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.246+0000 7fd6ef59e640 1 -- 172.21.15.94:0/2814104016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f01a1d90 con 0x7fd6f0105090 2024-04-25T13:44:31.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6ef59e640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fd0 tx=0x7fd6e800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6d7fff640 1 -- 172.21.15.94:0/2814104016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8017070 con 0x7fd6f0105090 2024-04-25T13:44:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6d7fff640 1 -- 172.21.15.94:0/2814104016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e8003e20 con 0x7fd6f0105090 2024-04-25T13:44:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01a2080 con 0x7fd6f0105090 2024-04-25T13:44:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6d7fff640 1 -- 172.21.15.94:0/2814104016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8010460 con 0x7fd6f0105090 2024-04-25T13:44:31.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.247+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a2580 con 0x7fd6f0105090 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.249+0000 7fd6fbfa1740 1 Processor -- start 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6d7fff640 1 -- 172.21.15.94:0/2814104016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd6e8010640 con 0x7fd6f0105090 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 -- start start 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607d29c6d80 con 0x5607d269f6b0 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5607d2980a20 con 0x5607d26974d0 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607d29c2d40 con 0x5607d26c0390 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34190/0 (socket says 172.21.15.94:34190) 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a62d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54940/0 (socket says 172.21.15.94:54940) 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.250+0000 7fd6ee59c640 1 -- 172.21.15.94:0/2481759657 learned_addr learned my addr 172.21.15.94:0/2481759657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6d7fff640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 0x7fd6d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 -- 172.21.15.94:0/2481759657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 msgr2=0x5607d2a62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 -- 172.21.15.94:0/2481759657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 msgr2=0x5607d2a63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a63a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 -- 172.21.15.94:0/2481759657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d298bc30 con 0x5607d26c0390 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ef59e640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 0x7fd6d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6d7fff640 1 -- 172.21.15.94:0/2814104016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6e80de990 con 0x7fd6f0105090 2024-04-25T13:44:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4010970 tx=0x7fd6e4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5620e391ba31550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6d57fa640 1 -- 172.21.15.94:0/2481759657 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4016070 con 0x5607d26c0390 2024-04-25T13:44:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6ef59e640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 0x7fd6d80b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80a2f00 tx=0x7fd6e806d3a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6d57fa640 1 -- 172.21.15.94:0/2481759657 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e400f7c0 con 0x5607d26c0390 2024-04-25T13:44:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.251+0000 7fd6d57fa640 1 -- 172.21.15.94:0/2481759657 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4019040 con 0x5607d26c0390 2024-04-25T13:44:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/2481759657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 msgr2=0x5607d2a62810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4010970 tx=0x7fd6e4005f90 comp rx=0 tx=0).stop 2024-04-25T13:44:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/2481759657 shutdown_connections 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/2481759657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a62810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/2481759657 >> 172.21.15.94:0/2481759657 conn(0x5607d26b44e0 msgr2=0x5607d2a5e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/2481759657 shutdown_connections 2024-04-25T13:44:31.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.252+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/2481759657 wait complete. 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 -- start start 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 Processor -- start 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607d2a64530 con 0x5607d269f6b0 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5607d294ddc0 con 0x5607d26974d0 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607d2a646b0 con 0x5607d26c0390 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6fbfa1740 10 client.0 _ll_register_callbacks cb 0x5607d2969600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59584/0 (socket says 172.21.15.94:59584) 2024-04-25T13:44:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 -- 172.21.15.94:0/1680536833 learned_addr learned my addr 172.21.15.94:0/1680536833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 -- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 msgr2=0x5607d2a739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a739d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 -- 172.21.15.94:0/1680536833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 msgr2=0x5607d2a6c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a6c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 -- 172.21.15.94:0/1680536833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d26d83f0 con 0x5607d269f6b0 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a6c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009230 tx=0x7fd6e0031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ecf20d8c5ca095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.253+0000 7fd6d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035040 con 0x5607d269f6b0 2024-04-25T13:44:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6e0004240 con 0x5607d269f6b0 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e400f7c0 con 0x5607d269f6b0 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6e00318f0 con 0x5607d269f6b0 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 10 client.16395 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607d27a27d0 con 0x5607d269f6b0 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e400f7c0 con 0x5607d269f6b0 2024-04-25T13:44:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e0044020 con 0x5607d269f6b0 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 10 client.16395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 20 client.16395 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d267ef60 0x5607d2a7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 4 client.16395 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607d29d4990 con 0x5607d267ef60 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.254+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d267ef60 0x5607d2a7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.255+0000 7fd6d2ff5640 10 client.16395 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.255+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d267ef60 0x5607d2a7a1a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9c7392f25dda894 server_cookie=bfc9b6fd28f56fb2 in_seq=0 out_seq=0 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/1680536833 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607d29d4990 con 0x5607d267ef60 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6d2ff5640 10 client.16395 handle_command_reply: tid=0 2024-04-25T13:44:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 client.16395 shutdown 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 msgr2=0x5607d2a6ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009230 tx=0x7fd6e0031e80 comp rx=0 tx=0).stop 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 shutdown_connections 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d267ef60 0x5607d2a7a1a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26c0390 0x5607d2a739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a6ca70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/1680536833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26974d0 0x5607d2a6c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 >> 172.21.15.94:0/1680536833 conn(0x5607d26b44e0 msgr2=0x5607d2a60f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6d2ff5640 0 client.16395 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 shutdown_connections 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/1680536833 wait complete. 2024-04-25T13:44:31.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.256+0000 7fd6fbfa1740 20 client.16395 trim_cache size 0 max 16384 2024-04-25T13:44:31.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 Processor -- start 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 -- start start 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d269f6b0 0x5607d2a738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a5da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ab4f0 con 0x5607d26974d0 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ad030 con 0x5607d26c0390 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01abbc0 con 0x5607d269f6b0 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59594/0 (socket says 172.21.15.94:59594) 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6ee59c640 1 -- 172.21.15.94:0/3968393458 learned_addr learned my addr 172.21.15.94:0/3968393458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d269f6b0 0x5607d2a738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.270+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a5da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 -- 172.21.15.94:0/3968393458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d269f6b0 msgr2=0x5607d2a738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d269f6b0 0x5607d2a738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 -- 172.21.15.94:0/3968393458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 msgr2=0x5607d2a5da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a5da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 -- 172.21.15.94:0/3968393458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d2a73e30 con 0x5607d26974d0 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.271+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009a10 tx=0x7fd6e4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644ac82d420e9320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.274+0000 7fd6d4ff9640 1 -- 172.21.15.94:0/3968393458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4016070 con 0x5607d26974d0 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.274+0000 7fd6d4ff9640 1 -- 172.21.15.94:0/3968393458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e400f7c0 con 0x5607d26974d0 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.274+0000 7fd6d4ff9640 1 -- 172.21.15.94:0/3968393458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4028040 con 0x5607d26974d0 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/3968393458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 msgr2=0x5607d2a6acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009a10 tx=0x7fd6e4009620 comp rx=0 tx=0).stop 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/3968393458 shutdown_connections 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a5da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d269f6b0 0x5607d2a738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/3968393458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d26974d0 0x5607d2a6acb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/3968393458 >> 172.21.15.94:0/3968393458 conn(0x5607d26b44e0 msgr2=0x5607d2a67290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/3968393458 shutdown_connections 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/3968393458 wait complete. 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 -- start start 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.275+0000 7fd6fbfa1740 1 Processor -- start 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26974d0 0x5607d2a70e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ac960 con 0x5607d269f6b0 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ac290 con 0x5607d26c0390 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01a94d0 con 0x5607d26974d0 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6fbfa1740 10 client.0 _ll_register_callbacks cb 0x7fd6e0009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59596/0 (socket says 172.21.15.94:59596) 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 -- 172.21.15.94:0/299605765 learned_addr learned my addr 172.21.15.94:0/299605765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26974d0 0x5607d2a70e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 -- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26974d0 msgr2=0x5607d2a70e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26974d0 0x5607d2a70e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 -- 172.21.15.94:0/299605765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 msgr2=0x5607d2a72240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a72240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 -- 172.21.15.94:0/299605765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d2a72800 con 0x5607d269f6b0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6eed9d640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000c00 tx=0x7fd6e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f11a1d060e7cf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.276+0000 7fd6d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.278+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0054030 con 0x5607d269f6b0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.278+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6e0004250 con 0x5607d269f6b0 2024-04-25T13:44:31.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.278+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e400f7c0 con 0x5607d269f6b0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6e005e050 con 0x5607d269f6b0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 10 client.16401 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607d2a73b20 con 0x5607d269f6b0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e400f7c0 con 0x5607d269f6b0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e0044040 con 0x5607d269f6b0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 10 client.16401 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 10 client.16401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 20 client.16401 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d26543a0 0x5607d2ae4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 4 client.16401 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607d29ce800 con 0x5607d26543a0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.279+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d26543a0 0x5607d2ae4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6d2ff5640 10 client.16401 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:44:31.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6ee59c640 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d26543a0 0x5607d2ae4ce0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=254e653fe6796c6b server_cookie=cbee20e14dac05b5 in_seq=0 out_seq=0 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6d2ff5640 1 -- 172.21.15.94:0/299605765 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607d29ce800 con 0x5607d26543a0 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6d2ff5640 10 client.16401 handle_command_reply: tid=0 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 client.16401 shutdown 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 msgr2=0x5607d2a71380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000c00 tx=0x7fd6e0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 shutdown_connections 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5607d26543a0 0x5607d2ae4ce0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607d26c0390 0x5607d2a72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607d269f6b0 0x5607d2a71380 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6d2ff5640 0 client.16401 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 --2- 172.21.15.94:0/299605765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d26974d0 0x5607d2a70e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 >> 172.21.15.94:0/299605765 conn(0x5607d26b44e0 msgr2=0x5607d2a5d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 shutdown_connections 2024-04-25T13:44:31.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 1 -- 172.21.15.94:0/299605765 wait complete. 2024-04-25T13:44:31.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.280+0000 7fd6fbfa1740 20 client.16401 trim_cache size 0 max 16384 2024-04-25T13:44:31.294 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:44:31.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 msgr2=0x7fd6d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 0x7fd6d80b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80a2f00 tx=0x7fd6e806d3a0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 msgr2=0x7fd6f01a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fd0 tx=0x7fd6e800b260 comp rx=0 tx=0).stop 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 shutdown_connections 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6d80afaf0 0x7fd6d80b2000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f0105090 0x7fd6f01a1720 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0102240 0x7fd6f019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 --2- 172.21.15.94:0/2814104016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f0101740 0x7fd6f019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 >> 172.21.15.94:0/2814104016 conn(0x7fd6f00fd340 msgr2=0x7fd6f0105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 shutdown_connections 2024-04-25T13:44:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.281+0000 7fd6f501a640 1 -- 172.21.15.94:0/2814104016 wait complete. 2024-04-25T13:44:31.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1121213692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70103960 msgr2=0x7f9d70103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1121213692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70103960 0x7f9d70103e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009900 tx=0x7f9d5802a410 comp rx=0 tx=0).stop 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1121213692 shutdown_connections 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1121213692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d701044c0 0x7f9d7010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1121213692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70103960 0x7f9d70103e30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1121213692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d70102e00 0x7f9d70103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1121213692 >> 172.21.15.94:0/1121213692 conn(0x7f9d700fd1a0 msgr2=0x7f9d700ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1121213692 shutdown_connections 2024-04-25T13:44:31.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.411+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1121213692 wait complete. 2024-04-25T13:44:31.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.412+0000 7f9d6f59e640 1 Processor -- start 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.412+0000 7f9d6f59e640 1 -- start start 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d70103960 0x7f9d70199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d7019e1c0 0x7f9d701a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010e570 con 0x7f9d70102e00 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010e3f0 con 0x7f9d70103960 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7010e6f0 con 0x7f9d7019e1c0 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59628/0 (socket says 172.21.15.94:59628) 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 -- 172.21.15.94:0/1415326836 learned_addr learned my addr 172.21.15.94:0/1415326836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d70103960 0x7f9d70199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d7019e1c0 0x7f9d701a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 -- 172.21.15.94:0/1415326836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d7019e1c0 msgr2=0x7f9d701a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d7019e1c0 0x7f9d701a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 -- 172.21.15.94:0/1415326836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d70103960 msgr2=0x7f9d70199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d70103960 0x7f9d70199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 -- 172.21.15.94:0/1415326836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701a0c50 con 0x7f9d70102e00 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d770 tx=0x7f9d6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d67fff640 1 -- 172.21.15.94:0/1415326836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60017070 con 0x7f9d70102e00 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a0f40 con 0x7f9d70102e00 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.413+0000 7f9d67fff640 1 -- 172.21.15.94:0/1415326836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d60009c30 con 0x7f9d70102e00 2024-04-25T13:44:31.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.414+0000 7f9d67fff640 1 -- 172.21.15.94:0/1415326836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000f620 con 0x7f9d70102e00 2024-04-25T13:44:31.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.414+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a13b0 con 0x7f9d70102e00 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 Processor -- start 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 -- start start 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d67fff640 1 -- 172.21.15.94:0/1415326836 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9d6001a070 con 0x7f9d70102e00 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519786b0 0x560e51d3ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e51c9fd80 con 0x560e519704d0 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e51c59a20 con 0x560e51999390 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e51c9bd40 con 0x560e519786b0 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.416+0000 7f9d6e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59636/0 (socket says 172.21.15.94:59636) 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55004/0 (socket says 172.21.15.94:55004) 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6e59c640 1 -- 172.21.15.94:0/746729160 learned_addr learned my addr 172.21.15.94:0/746729160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519786b0 0x560e51d3ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/746729160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519786b0 msgr2=0x560e51d3ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519786b0 0x560e51d3ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/746729160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 msgr2=0x560e51d3e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/746729160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e51c64c30 con 0x560e51999390 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d67fff640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 0x7f9d5c0b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009ed0 tx=0x7f9d680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991447612065ef40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 0x7f9d5c0b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d64d99640 1 -- 172.21.15.94:0/746729160 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003b70 con 0x560e51999390 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d64d99640 1 -- 172.21.15.94:0/746729160 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d68015070 con 0x560e51999390 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d64d99640 1 -- 172.21.15.94:0/746729160 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d680117b0 con 0x560e51999390 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.417+0000 7f9d67fff640 1 -- 172.21.15.94:0/1415326836 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9d600de7a0 con 0x7f9d70102e00 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/746729160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 msgr2=0x560e51d3dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009ed0 tx=0x7f9d680107a0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d6e59c640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 0x7f9d5c0b2020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d60016ef0 tx=0x7f9d600e4550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/746729160 shutdown_connections 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519786b0 0x560e51d3ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/746729160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d3dbf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/746729160 >> 172.21.15.94:0/746729160 conn(0x560e5198d4e0 msgr2=0x560e51d371f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/746729160 shutdown_connections 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.418+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/746729160 wait complete. 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 -- start start 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 Processor -- start 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d4ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e51d3f910 con 0x560e519786b0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e51c26dc0 con 0x560e51999390 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e51d3fa90 con 0x560e519704d0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d7bb8f740 10 client.0 _ll_register_callbacks cb 0x560e51c42600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59650/0 (socket says 172.21.15.94:59650) 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 -- 172.21.15.94:0/3819559935 learned_addr learned my addr 172.21.15.94:0/3819559935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d4ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 -- 172.21.15.94:0/3819559935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 msgr2=0x560e51d47940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d47940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 -- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 msgr2=0x560e51d4ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d4ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 -- 172.21.15.94:0/3819559935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e519b13f0 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d47940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9d5802a930 tx=0x7f9d58031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eb6d4202fa1b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.419+0000 7f9d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58035040 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9d58004240 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d68015070 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d580318f0 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 10 client.16413 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e51a7b7d0 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d68015070 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d5803c4d0 con 0x560e519786b0 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 10 client.16413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:44:31.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 20 client.16413 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e51957f60 0x560e51d55620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 4 client.16413 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e51cad990 con 0x560e51957f60 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.420+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e51957f60 0x560e51d55620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.421+0000 7f9d55ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.421+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e51957f60 0x560e51d55620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f063fdcea7d50d4 server_cookie=9b57591a19dabc0c in_seq=0 out_seq=0 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.421+0000 7f9d55ffb640 1 -- 172.21.15.94:0/3819559935 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e51cad990 con 0x560e51957f60 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d55ffb640 10 client.16413 handle_command_reply: tid=0 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 client.16413 shutdown 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 msgr2=0x560e51d47e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9d5802a930 tx=0x7f9d58031ea0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 shutdown_connections 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e51957f60 0x560e51d55620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d4ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d47e80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/3819559935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d47940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d55ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 >> 172.21.15.94:0/3819559935 conn(0x560e5198d4e0 msgr2=0x560e51d363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 shutdown_connections 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/3819559935 wait complete. 2024-04-25T13:44:31.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.422+0000 7f9d7bb8f740 20 client.16413 trim_cache size 0 max 16384 2024-04-25T13:44:31.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 Processor -- start 2024-04-25T13:44:31.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 -- start start 2024-04-25T13:44:31.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701a91a0 con 0x560e519786b0 2024-04-25T13:44:31.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701aace0 con 0x560e51999390 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701a9870 con 0x560e519704d0 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59660/0 (socket says 172.21.15.94:59660) 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 -- 172.21.15.94:0/2280840682 learned_addr learned my addr 172.21.15.94:0/2280840682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 -- 172.21.15.94:0/2280840682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 msgr2=0x560e51d59dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d59dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 -- 172.21.15.94:0/2280840682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 msgr2=0x560e51d5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d5a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 -- 172.21.15.94:0/2280840682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e51d28ba0 con 0x560e519786b0 2024-04-25T13:44:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.436+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9d5803c920 tx=0x7f9d5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d49b25fadb9fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d57fff640 1 -- 172.21.15.94:0/2280840682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58054030 con 0x560e519786b0 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d57fff640 1 -- 172.21.15.94:0/2280840682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58005850 con 0x560e519786b0 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d57fff640 1 -- 172.21.15.94:0/2280840682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58035040 con 0x560e519786b0 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/2280840682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 msgr2=0x560e51d5a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9d5803c920 tx=0x7f9d5802a410 comp rx=0 tx=0).stop 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/2280840682 shutdown_connections 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e51999390 0x560e51d5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519786b0 0x560e51d5a310 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/2280840682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e519704d0 0x560e51d59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/2280840682 >> 172.21.15.94:0/2280840682 conn(0x560e5198d4e0 msgr2=0x560e51d426d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/2280840682 shutdown_connections 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.437+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/2280840682 wait complete. 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 -- start start 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 Processor -- start 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 0x560e51d4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e51999390 0x560e51d4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701aa610 con 0x560e519704d0 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701a9f40 con 0x560e519786b0 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701a7180 con 0x560e51999390 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d7bb8f740 10 client.0 _ll_register_callbacks cb 0x7f9d58009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59664/0 (socket says 172.21.15.94:59664) 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/1715314569 learned_addr learned my addr 172.21.15.94:0/1715314569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 0x560e51d4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e51999390 0x560e51d4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/1715314569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e51999390 msgr2=0x560e51d4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e51999390 0x560e51d4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 msgr2=0x560e51d4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 0x560e51d4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 -- 172.21.15.94:0/1715314569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e51d4d230 con 0x560e519704d0 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 0x560e51d4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d6ed9d640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009a40 tx=0x7f9d68009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2445af6ba4e96014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.438+0000 7f9d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68031020 con 0x560e519704d0 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9d6802a7c0 con 0x560e519704d0 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d58005850 con 0x560e519704d0 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d6802a9c0 con 0x560e519704d0 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 10 client.16419 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e51d4e510 con 0x560e519704d0 2024-04-25T13:44:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58005850 con 0x560e519704d0 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d68037480 con 0x560e519704d0 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 10 client.16419 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 10 client.16419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 20 client.16419 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e5192d3a0 0x560e51dbfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 4 client.16419 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e51ca7800 con 0x560e5192d3a0 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.439+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e5192d3a0 0x560e51dbfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d65d9b640 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e5192d3a0 0x560e51dbfde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2941673a64bb7407 server_cookie=ec9070b24b6f6e3e in_seq=0 out_seq=0 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d55ffb640 10 client.16419 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d55ffb640 1 -- 172.21.15.94:0/1715314569 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e51ca7800 con 0x560e5192d3a0 2024-04-25T13:44:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d55ffb640 10 client.16419 handle_command_reply: tid=0 2024-04-25T13:44:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 client.16419 shutdown 2024-04-25T13:44:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 msgr2=0x560e51d4b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009a40 tx=0x7f9d68009500 comp rx=0 tx=0).stop 2024-04-25T13:44:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 shutdown_connections 2024-04-25T13:44:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560e5192d3a0 0x560e51dbfde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e51999390 0x560e51d4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d55ffb640 0 client.16419 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e519786b0 0x560e51d4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 --2- 172.21.15.94:0/1715314569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e519704d0 0x560e51d4b800 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 >> 172.21.15.94:0/1715314569 conn(0x560e5198d4e0 msgr2=0x560e51d39ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 shutdown_connections 2024-04-25T13:44:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 1 -- 172.21.15.94:0/1715314569 wait complete. 2024-04-25T13:44:31.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.440+0000 7f9d7bb8f740 20 client.16419 trim_cache size 0 max 16384 2024-04-25T13:44:31.462 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:44:31.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 msgr2=0x7f9d5c0b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 0x7f9d5c0b2020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d60016ef0 tx=0x7f9d600e4550 comp rx=0 tx=0).stop 2024-04-25T13:44:31.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 msgr2=0x7f9d70198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000d770 tx=0x7f9d6000dc30 comp rx=0 tx=0).stop 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 shutdown_connections 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9d5c0afb10 0x7f9d5c0b2020 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d7019e1c0 0x7f9d701a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d70103960 0x7f9d70199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 --2- 172.21.15.94:0/1415326836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d70102e00 0x7f9d70198d40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 >> 172.21.15.94:0/1415326836 conn(0x7f9d700fd1a0 msgr2=0x7f9d700ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 shutdown_connections 2024-04-25T13:44:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.441+0000 7f9d6f59e640 1 -- 172.21.15.94:0/1415326836 wait complete. 2024-04-25T13:44:31.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:31 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1155885140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:44:31.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:44:31.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 -- 172.21.15.94:0/804652115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b0108920 msgr2=0x7f82b010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 --2- 172.21.15.94:0/804652115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b0108920 0x7f82b010ad40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 -- 172.21.15.94:0/804652115 shutdown_connections 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 --2- 172.21.15.94:0/804652115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b0108920 0x7f82b010ad40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 --2- 172.21.15.94:0/804652115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 0x7f82b0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 --2- 172.21.15.94:0/804652115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b01029e0 0x7f82b01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 -- 172.21.15.94:0/804652115 >> 172.21.15.94:0/804652115 conn(0x7f82b00fdbd0 msgr2=0x7f82b0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 -- 172.21.15.94:0/804652115 shutdown_connections 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.585+0000 7f82b4964640 1 -- 172.21.15.94:0/804652115 wait complete. 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 Processor -- start 2024-04-25T13:44:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 -- start start 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 0x7f82b019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b0108920 0x7f82b01a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010e560 con 0x7f82b01029e0 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010e3e0 con 0x7f82b0103c20 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82b4964640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010e6e0 con 0x7f82b0108920 2024-04-25T13:44:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59688/0 (socket says 172.21.15.94:59688) 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82ae59c640 1 -- 172.21.15.94:0/2310387455 learned_addr learned my addr 172.21.15.94:0/2310387455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 0x7f82b019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.586+0000 7f82aed9d640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b0108920 0x7f82b01a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 -- 172.21.15.94:0/2310387455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b0108920 msgr2=0x7f82b01a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b0108920 0x7f82b01a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 -- 172.21.15.94:0/2310387455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 msgr2=0x7f82b019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 0x7f82b019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 -- 172.21.15.94:0/2310387455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01a2070 con 0x7f82b01029e0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82ae59c640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f82a000e9f0 tx=0x7f82a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82a7fff640 1 -- 172.21.15.94:0/2310387455 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000ccc0 con 0x7f82b01029e0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01a2360 con 0x7f82b01029e0 2024-04-25T13:44:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82a7fff640 1 -- 172.21.15.94:0/2310387455 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a00035f0 con 0x7f82b01029e0 2024-04-25T13:44:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82a7fff640 1 -- 172.21.15.94:0/2310387455 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000ce60 con 0x7f82b01029e0 2024-04-25T13:44:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.587+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01a2860 con 0x7f82b01029e0 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.590+0000 7f82bb811740 1 Processor -- start 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.590+0000 7f82bb811740 1 -- start start 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.590+0000 7f82bb811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82bb811740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e69fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82bb811740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82a7fff640 1 -- 172.21.15.94:0/2310387455 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f82a0010690 con 0x7f82b01029e0 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82bb811740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e603d80 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82bb811740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e5bda20 con 0x55fa5e2d44d0 2024-04-25T13:44:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82bb811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e5ffd40 con 0x55fa5e2fd390 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34302/0 (socket says 172.21.15.94:34302) 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 -- 172.21.15.94:0/1381900785 learned_addr learned my addr 172.21.15.94:0/1381900785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82ae59c640 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e69fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 -- 172.21.15.94:0/1381900785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 msgr2=0x55fa5e69fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e69fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 -- 172.21.15.94:0/1381900785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 msgr2=0x55fa5e6a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 -- 172.21.15.94:0/1381900785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5e5c8c30 con 0x55fa5e2fd390 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82a8035000 tx=0x7f82a802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166e35b0e158c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.591+0000 7f82a7fff640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 0x7f829c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.592+0000 7f82ae59c640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 0x7f829c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.592+0000 7f82a4d99640 1 -- 172.21.15.94:0/1381900785 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8031650 con 0x55fa5e2fd390 2024-04-25T13:44:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.592+0000 7f82a4d99640 1 -- 172.21.15.94:0/1381900785 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8036030 con 0x55fa5e2fd390 2024-04-25T13:44:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.592+0000 7f82a4d99640 1 -- 172.21.15.94:0/1381900785 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004680 con 0x55fa5e2fd390 2024-04-25T13:44:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.592+0000 7f82a7fff640 1 -- 172.21.15.94:0/2310387455 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f82a00df020 con 0x7f82b01029e0 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- 172.21.15.94:0/1381900785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 msgr2=0x55fa5e69f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82ae59c640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 0x7f829c0b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fa5e69f530 tx=0x7f82a00e2b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f82a8035000 tx=0x7f82a802f3a0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- 172.21.15.94:0/1381900785 shutdown_connections 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e69fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 --2- 172.21.15.94:0/1381900785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2fd390 0x55fa5e69f810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- 172.21.15.94:0/1381900785 >> 172.21.15.94:0/1381900785 conn(0x55fa5e2f14e0 msgr2=0x55fa5e69b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- 172.21.15.94:0/1381900785 shutdown_connections 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- 172.21.15.94:0/1381900785 wait complete. 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 -- start start 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.593+0000 7f82bb811740 1 Processor -- start 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2d44d0 0x55fa5e6a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e6a1350 con 0x55fa5e2fd390 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e58adc0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa5e6a14d0 con 0x55fa5e2d44d0 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 10 client.0 _ll_register_callbacks cb 0x55fa5e5a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59702/0 (socket says 172.21.15.94:59702) 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 -- 172.21.15.94:0/2294855051 learned_addr learned my addr 172.21.15.94:0/2294855051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2d44d0 0x55fa5e6a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 -- 172.21.15.94:0/2294855051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2d44d0 msgr2=0x55fa5e6a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2d44d0 0x55fa5e6a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 -- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2dc6b0 msgr2=0x55fa5e6a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 -- 172.21.15.94:0/2294855051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5e3153f0 con 0x55fa5e2fd390 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82aed9d640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f799807b1f42761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8059060 con 0x55fa5e2fd390 2024-04-25T13:44:31.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f82a8003d80 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.594+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a8036030 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82a8004b40 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 10 client.16431 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa5e3df7d0 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8036030 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f82a8043070 con 0x55fa5e2fd390 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 20 client.16431 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2bbf60 0x55fa5e6b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 4 client.16431 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5e611990 con 0x55fa5e2bbf60 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.595+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2bbf60 0x55fa5e6b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.596+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2bbf60 0x55fa5e6b6c20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c41f2544a125d19 server_cookie=1f6860b07da6a663 in_seq=0 out_seq=0 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.596+0000 7f8295ffb640 10 client.16431 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f8295ffb640 1 -- 172.21.15.94:0/2294855051 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa5e611990 con 0x55fa5e2bbf60 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f8295ffb640 10 client.16431 handle_command_reply: tid=0 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 client.16431 shutdown 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 msgr2=0x55fa5e6b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8031af0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 shutdown_connections 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2bbf60 0x55fa5e6b6c20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2fd390 0x55fa5e6b07f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f8295ffb640 0 client.16431 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e6a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.597+0000 7f82bb811740 1 --2- 172.21.15.94:0/2294855051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2d44d0 0x55fa5e6a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.598+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 >> 172.21.15.94:0/2294855051 conn(0x55fa5e2f14e0 msgr2=0x55fa5e69df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.598+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 shutdown_connections 2024-04-25T13:44:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.598+0000 7f82bb811740 1 -- 172.21.15.94:0/2294855051 wait complete. 2024-04-25T13:44:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.598+0000 7f82bb811740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 Processor -- start 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 -- start start 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e699b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e69a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ab4e0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ad020 con 0x55fa5e2d44d0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82bb811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01abbb0 con 0x55fa5e2f14e0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82a5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e699b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34326/0 (socket says 172.21.15.94:34326) 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 -- 172.21.15.94:0/3485967046 learned_addr learned my addr 172.21.15.94:0/3485967046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e69a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 -- 172.21.15.94:0/3485967046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 msgr2=0x55fa5e699b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e699b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.612+0000 7f82aed9d640 1 -- 172.21.15.94:0/3485967046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 msgr2=0x55fa5e69a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f82aed9d640 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e69a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f82aed9d640 1 -- 172.21.15.94:0/3485967046 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5e6a4c60 con 0x55fa5e2f14e0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f82aed9d640 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82a8031580 tx=0x7f82a8004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c467f109f66724d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f8297fff640 1 -- 172.21.15.94:0/3485967046 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a802f3a0 con 0x55fa5e2f14e0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f8297fff640 1 -- 172.21.15.94:0/3485967046 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8036070 con 0x55fa5e2f14e0 2024-04-25T13:44:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.613+0000 7f8297fff640 1 -- 172.21.15.94:0/3485967046 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8003b80 con 0x55fa5e2f14e0 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- 172.21.15.94:0/3485967046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 msgr2=0x55fa5e699610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82a8031580 tx=0x7f82a8004ce0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- 172.21.15.94:0/3485967046 shutdown_connections 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e69a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2d44d0 0x55fa5e699b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 --2- 172.21.15.94:0/3485967046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2f14e0 0x55fa5e699610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- 172.21.15.94:0/3485967046 >> 172.21.15.94:0/3485967046 conn(0x55fa5e69ffa0 msgr2=0x55fa5e6a7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- 172.21.15.94:0/3485967046 shutdown_connections 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- 172.21.15.94:0/3485967046 wait complete. 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 -- start start 2024-04-25T13:44:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.614+0000 7f82bb811740 1 Processor -- start 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e71ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2f14e0 0x55fa5e721cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ac950 con 0x55fa5e2d44d0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01ac280 con 0x55fa5e2f14e0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a94c0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 10 client.0 _ll_register_callbacks cb 0x7f82980097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59726/0 (socket says 172.21.15.94:59726) 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82aed9d640 1 -- 172.21.15.94:0/1379628696 learned_addr learned my addr 172.21.15.94:0/1379628696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e71ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2f14e0 0x55fa5e721cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 -- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2f14e0 msgr2=0x55fa5e721cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2f14e0 0x55fa5e721cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 -- 172.21.15.94:0/1379628696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 msgr2=0x55fa5e71a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 -- 172.21.15.94:0/1379628696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5e722280 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82a5d9b640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e71ad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55fa5e6b7310 tx=0x7f8298012c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1956f720d5929c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f8295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298004250 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f82980043f0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.615+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a8036070 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f829801f8d0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 10 client.25846 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa5e7235d0 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8036070 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8298027030 con 0x55fa5e2dc6b0 2024-04-25T13:44:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 10 client.25846 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 20 client.25846 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2fd390 0x55fa5e725f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 4 client.25846 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa5e60b800 con 0x55fa5e2fd390 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.616+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2fd390 0x55fa5e725f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.617+0000 7f82aed9d640 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2fd390 0x55fa5e725f90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91380b622808d0d4 server_cookie=96b33c2cc79cd222 in_seq=0 out_seq=0 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.617+0000 7f8295ffb640 10 client.25846 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f8295ffb640 1 -- 172.21.15.94:0/1379628696 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa5e60b800 con 0x55fa5e2fd390 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f8295ffb640 10 client.25846 handle_command_reply: tid=0 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 client.25846 shutdown 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 msgr2=0x55fa5e71ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e71ad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55fa5e6b7310 tx=0x7f8298012c50 comp rx=0 tx=0).stop 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 shutdown_connections 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5e2fd390 0x55fa5e725f90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5e2f14e0 0x55fa5e721cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5e2dc6b0 0x55fa5e71ad60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 --2- 172.21.15.94:0/1379628696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5e2d44d0 0x55fa5e71a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f8295ffb640 0 client.25846 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 >> 172.21.15.94:0/1379628696 conn(0x55fa5e69ffa0 msgr2=0x55fa5e6a78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 shutdown_connections 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 1 -- 172.21.15.94:0/1379628696 wait complete. 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.618+0000 7f82bb811740 20 client.25846 trim_cache size 0 max 16384 2024-04-25T13:44:31.623 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:44:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 msgr2=0x7f829c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 0x7f829c0b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55fa5e69f530 tx=0x7f82a00e2b00 comp rx=0 tx=0).stop 2024-04-25T13:44:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 msgr2=0x7f82b019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:31.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f82a000e9f0 tx=0x7f82a000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 shutdown_connections 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f829c0afaf0 0x7f829c0b2000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b0108920 0x7f82b01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82b0103c20 0x7f82b019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 --2- 172.21.15.94:0/2310387455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82b01029e0 0x7f82b019a530 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 >> 172.21.15.94:0/2310387455 conn(0x7f82b00fdbd0 msgr2=0x7f82b0109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 shutdown_connections 2024-04-25T13:44:31.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:31.619+0000 7f82b4964640 1 -- 172.21.15.94:0/2310387455 wait complete. 2024-04-25T13:44:31.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:31 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1155885140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:44:31.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:31 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1155885140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:44:31.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 196626 14.57 MB/sec warmup 74 sec latency 12.509 ms 2024-04-25T13:44:32.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:44:32.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:32.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:44:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:32 smithi137 ceph-mon[32568]: pgmap v428: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:44:32.651 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:32 smithi094 ceph-mon[29195]: pgmap v428: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:44:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:32 smithi138 ceph-mon[32977]: pgmap v428: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:44:32.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.752+0000 7f00b17ce640 1 -- 172.21.15.94:0/1037947643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 msgr2=0x7f00ac1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.752+0000 7f00b17ce640 1 --2- 172.21.15.94:0/1037947643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac1047e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009900 tx=0x7f00a402a410 comp rx=0 tx=0).stop 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 -- 172.21.15.94:0/1037947643 shutdown_connections 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 --2- 172.21.15.94:0/1037947643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 0x7f00ac10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 --2- 172.21.15.94:0/1037947643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 --2- 172.21.15.94:0/1037947643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac1047e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 -- 172.21.15.94:0/1037947643 >> 172.21.15.94:0/1037947643 conn(0x7f00ac0fecd0 msgr2=0x7f00ac101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 -- 172.21.15.94:0/1037947643 shutdown_connections 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 -- 172.21.15.94:0/1037947643 wait complete. 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.753+0000 7f00b17ce640 1 Processor -- start 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 -- start start 2024-04-25T13:44:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 0x7f00ac1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac10f5e0 con 0x7f00ac103ae0 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac10f460 con 0x7f00ac104d20 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00b17ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac10f760 con 0x7f00ac109a20 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59732/0 (socket says 172.21.15.94:59732) 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 -- 172.21.15.94:0/832624813 learned_addr learned my addr 172.21.15.94:0/832624813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aa7fc640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 0x7f00ac1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 -- 172.21.15.94:0/832624813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 msgr2=0x7f00ac1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 0x7f00ac1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 -- 172.21.15.94:0/832624813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 msgr2=0x7f00ac19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 -- 172.21.15.94:0/832624813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1a1d80 con 0x7f00ac103ae0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aa7fc640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.754+0000 7f00aaffd640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00a4006010 tx=0x7f00a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.755+0000 7f0093fff640 1 -- 172.21.15.94:0/832624813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a403f030 con 0x7f00ac103ae0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.755+0000 7f0093fff640 1 -- 172.21.15.94:0/832624813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a4003aa0 con 0x7f00ac103ae0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.755+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1a2010 con 0x7f00ac103ae0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.755+0000 7f0093fff640 1 -- 172.21.15.94:0/832624813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4035070 con 0x7f00ac103ae0 2024-04-25T13:44:32.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.755+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1a2500 con 0x7f00ac103ae0 2024-04-25T13:44:32.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f0093fff640 1 -- 172.21.15.94:0/832624813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f00a4042030 con 0x7f00ac103ae0 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 Processor -- start 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 -- start start 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644ea70db00 con 0x5644ea3de550 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5644ea6a9920 con 0x5644ea3e6730 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00b867d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644ea709ac0 con 0x5644ea407410 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59740/0 (socket says 172.21.15.94:59740) 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00aaffd640 1 -- 172.21.15.94:0/2351966804 learned_addr learned my addr 172.21.15.94:0/2351966804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00aa7fc640 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.757+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f0093fff640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 -- 172.21.15.94:0/2351966804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 msgr2=0x5644ea7a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 -- 172.21.15.94:0/2351966804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 msgr2=0x5644ea7af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 -- 172.21.15.94:0/2351966804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ea4e77e0 con 0x5644ea3de550 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f0093fff640 1 -- 172.21.15.94:0/832624813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f00a40ff2f0 con 0x7f00ac103ae0 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00aaffd640 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b03b0 tx=0x7f00a408dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b509a07841fb085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00917fa640 1 -- 172.21.15.94:0/2351966804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40cee60 con 0x5644ea3de550 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5644ea7a9670 tx=0x7f009c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00917fa640 1 -- 172.21.15.94:0/2351966804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a40b1ad0 con 0x5644ea3de550 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.758+0000 7f00917fa640 1 -- 172.21.15.94:0/2351966804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40c7080 con 0x5644ea3de550 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- 172.21.15.94:0/2351966804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 msgr2=0x5644ea7a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b03b0 tx=0x7f00a408dc50 comp rx=0 tx=0).stop 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- 172.21.15.94:0/2351966804 shutdown_connections 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a8a20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 --2- 172.21.15.94:0/2351966804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- 172.21.15.94:0/2351966804 >> 172.21.15.94:0/2351966804 conn(0x5644ea3fb560 msgr2=0x5644ea7a3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- 172.21.15.94:0/2351966804 shutdown_connections 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- 172.21.15.94:0/2351966804 wait complete. 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.759+0000 7f00b867d740 1 -- start start 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 Processor -- start 2024-04-25T13:44:32.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 0x5644ea7b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644ea70db00 con 0x5644ea407410 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5644ea6a75d0 con 0x5644ea3de550 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644ea7b0130 con 0x5644ea3e6730 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00b867d740 10 client.0 _ll_register_callbacks cb 0x5644ea695080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59746/0 (socket says 172.21.15.94:59746) 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 -- 172.21.15.94:0/3125888904 learned_addr learned my addr 172.21.15.94:0/3125888904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 0x5644ea7b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00aa7fc640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 -- 172.21.15.94:0/3125888904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 msgr2=0x5644ea7b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 -- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 msgr2=0x5644ea7b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 0x5644ea7b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 -- 172.21.15.94:0/3125888904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ea547f70 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 0x5644ea7b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f009c0116f0 tx=0x7f009c005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8879aac9b5cbc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.760+0000 7f008eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c01c9c0 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f009c01cb60 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644ea71b710 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f009c01cd60 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 10 client.16449 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644ea3df170 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5644ea71b710 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f009c023070 con 0x5644ea407410 2024-04-25T13:44:32.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 20 client.16449 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea3c61f0 0x5644ea7bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 4 client.16449 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ea715580 con 0x5644ea3c61f0 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.761+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea3c61f0 0x5644ea7bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.762+0000 7f008eff5640 10 client.16449 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.762+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea3c61f0 0x5644ea7bf930 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30db189ebcef3060 server_cookie=d2e8c7fb95a2ea07 in_seq=0 out_seq=0 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f008eff5640 1 -- 172.21.15.94:0/3125888904 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644ea715580 con 0x5644ea3c61f0 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f008eff5640 10 client.16449 handle_command_reply: tid=0 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 client.16449 shutdown 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 msgr2=0x5644ea7b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f009c0116f0 tx=0x7f009c005e10 comp rx=0 tx=0).stop 2024-04-25T13:44:32.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 shutdown_connections 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea3c61f0 0x5644ea7bf930 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea407410 0x5644ea7b9370 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f008eff5640 0 client.16449 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 --2- 172.21.15.94:0/3125888904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3de550 0x5644ea7b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 >> 172.21.15.94:0/3125888904 conn(0x5644ea3fb560 msgr2=0x5644ea7a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 shutdown_connections 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 1 -- 172.21.15.94:0/3125888904 wait complete. 2024-04-25T13:44:32.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.763+0000 7f00b867d740 20 client.16449 trim_cache size 0 max 16384 2024-04-25T13:44:32.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.781+0000 7f00b867d740 1 Processor -- start 2024-04-25T13:44:32.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.781+0000 7f00b867d740 1 -- start start 2024-04-25T13:44:32.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7acc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1ab550 con 0x5644ea3de550 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1ad090 con 0x5644ea3e6730 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00b867d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1abc20 con 0x5644ea407410 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59750/0 (socket says 172.21.15.94:59750) 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55122/0 (socket says 172.21.15.94:55122) 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 -- 172.21.15.94:0/3023564016 learned_addr learned my addr 172.21.15.94:0/3023564016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 -- 172.21.15.94:0/3023564016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 msgr2=0x5644ea7b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 -- 172.21.15.94:0/3023564016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 msgr2=0x5644ea7acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 -- 172.21.15.94:0/3023564016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ea7b2b40 con 0x5644ea3de550 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f00aaffd640 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00a40c76c0 tx=0x7f00a402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3273335f88976f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f0090ff9640 1 -- 172.21.15.94:0/3023564016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4049040 con 0x5644ea3de550 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f0090ff9640 1 -- 172.21.15.94:0/3023564016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a40c7240 con 0x5644ea3de550 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.782+0000 7f0090ff9640 1 -- 172.21.15.94:0/3023564016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40cee60 con 0x5644ea3de550 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- 172.21.15.94:0/3023564016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 msgr2=0x5644ea7a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00a40c76c0 tx=0x7f00a402a410 comp rx=0 tx=0).stop 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- 172.21.15.94:0/3023564016 shutdown_connections 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea407410 0x5644ea7b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea3e6730 0x5644ea7acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 --2- 172.21.15.94:0/3023564016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea3de550 0x5644ea7a4050 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- 172.21.15.94:0/3023564016 >> 172.21.15.94:0/3023564016 conn(0x5644ea3fb560 msgr2=0x5644ea7b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- 172.21.15.94:0/3023564016 shutdown_connections 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- 172.21.15.94:0/3023564016 wait complete. 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 -- start start 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.783+0000 7f00b867d740 1 Processor -- start 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea407410 0x5644ea7b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1ac9c0 con 0x5644ea39b630 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1ac2f0 con 0x5644ea407410 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1a9530 con 0x5644ea3e6730 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 10 client.0 _ll_register_callbacks cb 0x7f009c00fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59756/0 (socket says 172.21.15.94:59756) 2024-04-25T13:44:32.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 -- 172.21.15.94:0/1270272102 learned_addr learned my addr 172.21.15.94:0/1270272102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00aa7fc640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea407410 0x5644ea7b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 -- 172.21.15.94:0/1270272102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 msgr2=0x5644ea7b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 -- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea407410 msgr2=0x5644ea7b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea407410 0x5644ea7b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 -- 172.21.15.94:0/1270272102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ea7b9df0 con 0x5644ea39b630 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00ab7fe640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f009c025690 tx=0x7f009c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf0dd179ec7691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f008eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c032070 con 0x5644ea39b630 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f009c025a90 con 0x5644ea39b630 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.784+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a40c7240 con 0x5644ea39b630 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f009c025c90 con 0x5644ea39b630 2024-04-25T13:44:32.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 10 client.16455 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644ea8272e0 con 0x5644ea39b630 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a40c7240 con 0x5644ea39b630 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f009c023040 con 0x5644ea39b630 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 20 client.16455 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea827d10 0x5644ea82a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 4 client.16455 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5644ea690ac0 con 0x5644ea827d10 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00aaffd640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea827d10 0x5644ea82a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f008eff5640 10 client.16455 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.785+0000 7f00aaffd640 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea827d10 0x5644ea82a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110f374237ad123a server_cookie=b56a9a04fda3dbb9 in_seq=0 out_seq=0 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f008eff5640 1 -- 172.21.15.94:0/1270272102 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5644ea690ac0 con 0x5644ea827d10 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f008eff5640 10 client.16455 handle_command_reply: tid=0 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 client.16455 shutdown 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 msgr2=0x5644ea825cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f009c025690 tx=0x7f009c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 shutdown_connections 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644ea827d10 0x5644ea82a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ea39b630 0x5644ea825cd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644ea407410 0x5644ea7b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 --2- 172.21.15.94:0/1270272102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644ea3e6730 0x5644ea7b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 >> 172.21.15.94:0/1270272102 conn(0x5644ea3fb560 msgr2=0x5644ea7a24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f008eff5640 0 client.16455 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 shutdown_connections 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 1 -- 172.21.15.94:0/1270272102 wait complete. 2024-04-25T13:44:32.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.788+0000 7f00b867d740 20 client.16455 trim_cache size 0 max 16384 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "60462351-2997-4392-84d4-d345946447d9", 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:44:32.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 msgr2=0x7f00940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5644ea7a9670 tx=0x7f009c0046c0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 msgr2=0x7f00ac19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00a4006010 tx=0x7f00a4004270 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 shutdown_connections 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ac109a20 0x7f00ac1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac104d20 0x7f00ac19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 --2- 172.21.15.94:0/832624813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac103ae0 0x7f00ac19a240 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 >> 172.21.15.94:0/832624813 conn(0x7f00ac0fecd0 msgr2=0x7f00ac105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 shutdown_connections 2024-04-25T13:44:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.789+0000 7f00b17ce640 1 -- 172.21.15.94:0/832624813 wait complete. 2024-04-25T13:44:32.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 199199 14.56 MB/sec warmup 75 sec latency 17.758 ms 2024-04-25T13:44:32.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:44:32.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:44:32.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.959+0000 7f66763ce640 1 -- 172.21.15.94:0/1563610288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6670108a20 msgr2=0x7f667010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.959+0000 7f66763ce640 1 --2- 172.21.15.94:0/1563610288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6670108a20 0x7f667010ae40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6668009ca0 tx=0x7f666802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 -- 172.21.15.94:0/1563610288 shutdown_connections 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 --2- 172.21.15.94:0/1563610288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6670108a20 0x7f667010ae40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 --2- 172.21.15.94:0/1563610288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66701043d0 0x7f6670102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 --2- 172.21.15.94:0/1563610288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f66701027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 -- 172.21.15.94:0/1563610288 >> 172.21.15.94:0/1563610288 conn(0x7f66700fd350 msgr2=0x7f66700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 -- 172.21.15.94:0/1563610288 shutdown_connections 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 -- 172.21.15.94:0/1563610288 wait complete. 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 Processor -- start 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.960+0000 7f66763ce640 1 -- start start 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f6670102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 0x7f667019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 0x7f66701a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f667010e530 con 0x7f667019f0e0 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f667010e3b0 con 0x7f6670103ef0 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f66763ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f667010e6b0 con 0x7f6670108a20 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f667496b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 0x7f66701a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f6670102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f667496b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 0x7f66701a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59768/0 (socket says 172.21.15.94:59768) 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 0x7f667019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f667496b640 1 -- 172.21.15.94:0/4167187841 learned_addr learned my addr 172.21.15.94:0/4167187841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 -- 172.21.15.94:0/4167187841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 msgr2=0x7f6670102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f6670102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 -- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 msgr2=0x7f66701a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 0x7f66701a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666f7fe640 1 -- 172.21.15.94:0/4167187841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701a1d20 con 0x7f6670108a20 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.961+0000 7f666ffff640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f6670102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f666f7fe640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 0x7f667019eba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f666400e9f0 tx=0x7f666400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f666d7fa640 1 -- 172.21.15.94:0/4167187841 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666400ccc0 con 0x7f6670108a20 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66701a1fb0 con 0x7f6670108a20 2024-04-25T13:44:32.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f666d7fa640 1 -- 172.21.15.94:0/4167187841 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6664005020 con 0x7f6670108a20 2024-04-25T13:44:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f666d7fa640 1 -- 172.21.15.94:0/4167187841 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664004a00 con 0x7f6670108a20 2024-04-25T13:44:32.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.962+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66701a2470 con 0x7f6670108a20 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.965+0000 7f667d281740 1 Processor -- start 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.965+0000 7f667d281740 1 -- start start 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 0x55f2b2d9bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2d9cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2d00d80 con 0x55f2b29d96b0 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2cba8b0 con 0x55f2b29d14d0 2024-04-25T13:44:32.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667d281740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2cfcd40 con 0x55f2b29fa390 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f666ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 0x55f2b2d9bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34420/0 (socket says 172.21.15.94:34420) 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f666ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 0x55f2b2d9bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55166/0 (socket says 172.21.15.94:55166) 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 -- 172.21.15.94:0/3609275899 learned_addr learned my addr 172.21.15.94:0/3609275899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 -- 172.21.15.94:0/3609275899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 msgr2=0x55f2b2d9bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 0x55f2b2d9bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 -- 172.21.15.94:0/3609275899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 msgr2=0x55f2b2d9cbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2d9cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 -- 172.21.15.94:0/3609275899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b2c8ba40 con 0x55f2b29fa390 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.966+0000 7f667496b640 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6668009800 tx=0x7f6668009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21d8fda7d286aeef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.967+0000 7f66567fc640 1 -- 172.21.15.94:0/3609275899 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668031650 con 0x55f2b29fa390 2024-04-25T13:44:32.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.967+0000 7f66567fc640 1 -- 172.21.15.94:0/3609275899 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668036030 con 0x55f2b29fa390 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.967+0000 7f666d7fa640 1 -- 172.21.15.94:0/4167187841 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6664004ba0 con 0x7f6670108a20 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.967+0000 7f66567fc640 1 -- 172.21.15.94:0/3609275899 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66680047c0 con 0x55f2b29fa390 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f666d7fa640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 0x7f66580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f666ffff640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 0x7f66580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f666d7fa640 1 -- 172.21.15.94:0/4167187841 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f66640dec70 con 0x7f6670108a20 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 -- 172.21.15.94:0/3609275899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 msgr2=0x55f2b2d9b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6668009800 tx=0x7f6668009830 comp rx=0 tx=0).stop 2024-04-25T13:44:32.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 -- 172.21.15.94:0/3609275899 shutdown_connections 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2d9cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d14d0 0x55f2b2d9bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 --2- 172.21.15.94:0/3609275899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29fa390 0x55f2b2d9b990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 -- 172.21.15.94:0/3609275899 >> 172.21.15.94:0/3609275899 conn(0x55f2b29ee4e0 msgr2=0x55f2b2d97270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f666ffff640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 0x7f66580b2000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f2b2d9b6b0 tx=0x7f6660005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 -- 172.21.15.94:0/3609275899 shutdown_connections 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.968+0000 7f667d281740 1 -- 172.21.15.94:0/3609275899 wait complete. 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.969+0000 7f667d281740 1 -- start start 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.969+0000 7f667d281740 1 Processor -- start 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2da5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29fa390 0x55f2b2daccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2d9d4f0 con 0x55f2b29d96b0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2c87d10 con 0x55f2b29fa390 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2d9d670 con 0x55f2b29d14d0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667d281740 10 client.0 _ll_register_callbacks cb 0x55f2b2cb4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34422/0 (socket says 172.21.15.94:34422) 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f667496b640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29fa390 0x55f2b2daccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666ffff640 1 -- 172.21.15.94:0/1142980817 learned_addr learned my addr 172.21.15.94:0/1142980817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666f7fe640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2da5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666ffff640 1 -- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29fa390 msgr2=0x55f2b2daccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.970+0000 7f666ffff640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29fa390 0x55f2b2daccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f666ffff640 1 -- 172.21.15.94:0/1142980817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 msgr2=0x55f2b2da5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f666ffff640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2da5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f666ffff640 1 -- 172.21.15.94:0/1142980817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b2adc7d0 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f666ffff640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6660004620 tx=0x7f6660004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=965da2dfe16c1a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f664f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666000f830 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6668036030 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f666000f9d0 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.971+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f666000fc10 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.972+0000 7f667d281740 10 client.25870 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.972+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f2b2a123f0 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.972+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668036030 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6660027070 con 0x55f2b29d14d0 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 10 client.25870 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:32.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 20 client.25870 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:32.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29b8f60 0x55f2b2dc41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 4 client.25870 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:32.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b2d0e990 con 0x55f2b29b8f60 2024-04-25T13:44:32.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.973+0000 7f666f7fe640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29b8f60 0x55f2b2dc41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.974+0000 7f664f7fe640 10 client.25870 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.974+0000 7f666f7fe640 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29b8f60 0x55f2b2dc41d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe5b35cc32ed241 server_cookie=ac5de4865f17a060 in_seq=0 out_seq=0 2024-04-25T13:44:32.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.976+0000 7f664f7fe640 1 -- 172.21.15.94:0/1142980817 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2b2d0e990 con 0x55f2b29b8f60 2024-04-25T13:44:32.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.976+0000 7f664f7fe640 10 client.25870 handle_command_reply: tid=0 2024-04-25T13:44:32.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.976+0000 7f667d281740 1 client.25870 shutdown 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 msgr2=0x55f2b2da5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6660004620 tx=0x7f6660004730 comp rx=0 tx=0).stop 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 shutdown_connections 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29b8f60 0x55f2b2dc41d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29fa390 0x55f2b2daccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2da5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f664f7fe640 0 client.25870 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 --2- 172.21.15.94:0/1142980817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2da5880 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 >> 172.21.15.94:0/1142980817 conn(0x55f2b29ee4e0 msgr2=0x55f2b2d963f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 shutdown_connections 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 1 -- 172.21.15.94:0/1142980817 wait complete. 2024-04-25T13:44:32.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.977+0000 7f667d281740 20 client.25870 trim_cache size 0 max 16384 2024-04-25T13:44:32.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 Processor -- start 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 -- start start 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29b8f60 0x55f2b2dc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2dacc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66701a92a0 con 0x55f2b29d96b0 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66701aade0 con 0x55f2b29b8f60 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.997+0000 7f667d281740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66701a9970 con 0x55f2b29d14d0 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f666f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2dacc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f666ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29b8f60 0x55f2b2dc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59784/0 (socket says 172.21.15.94:59784) 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 -- 172.21.15.94:0/435488339 learned_addr learned my addr 172.21.15.94:0/435488339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 -- 172.21.15.94:0/435488339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 msgr2=0x55f2b2dacc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2dacc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 -- 172.21.15.94:0/435488339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29b8f60 msgr2=0x55f2b2dc8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29b8f60 0x55f2b2dc8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 -- 172.21.15.94:0/435488339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b29d20f0 con 0x55f2b29d96b0 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f667496b640 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f66680098a0 tx=0x7f666802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4beb745d70637b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f6655ffb640 1 -- 172.21.15.94:0/435488339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666804a040 con 0x55f2b29d96b0 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f6655ffb640 1 -- 172.21.15.94:0/435488339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668036050 con 0x55f2b29d96b0 2024-04-25T13:44:32.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.998+0000 7f6655ffb640 1 -- 172.21.15.94:0/435488339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668004590 con 0x55f2b29d96b0 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 -- 172.21.15.94:0/435488339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 msgr2=0x55f2b2dad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f66680098a0 tx=0x7f666802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 -- 172.21.15.94:0/435488339 shutdown_connections 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29d96b0 0x55f2b2dad180 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2dacc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 --2- 172.21.15.94:0/435488339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29b8f60 0x55f2b2dc8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 -- 172.21.15.94:0/435488339 >> 172.21.15.94:0/435488339 conn(0x55f2b29ee4e0 msgr2=0x55f2b2da3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:32.999+0000 7f667d281740 1 -- 172.21.15.94:0/435488339 shutdown_connections 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 -- 172.21.15.94:0/435488339 wait complete. 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 -- start start 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 Processor -- start 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2daab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d96b0 0x55f2b2e2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66701aa710 con 0x55f2b29b8f60 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66701aa040 con 0x55f2b29d96b0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66701a7280 con 0x55f2b29d14d0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f667d281740 10 client.0 _ll_register_callbacks cb 0x7f6660009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f666ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f666ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59796/0 (socket says 172.21.15.94:59796) 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f666ffff640 1 -- 172.21.15.94:0/2461492222 learned_addr learned my addr 172.21.15.94:0/2461492222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.000+0000 7f666f7fe640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2daab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f667496b640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d96b0 0x55f2b2e2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 -- 172.21.15.94:0/2461492222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 msgr2=0x55f2b2daab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2daab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 -- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d96b0 msgr2=0x55f2b2e2a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d96b0 0x55f2b2e2a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 -- 172.21.15.94:0/2461492222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b2cc14b0 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666f7fe640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2daab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f666ffff640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6660009f70 tx=0x7f666000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65072ec66104871b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f664f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660032a80 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6660032c20 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6668036050 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6660042050 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f667d281740 10 client.16470 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.001+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f2b2e2bbf0 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6668036050 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6660027040 con 0x55f2b29b8f60 2024-04-25T13:44:33.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 10 client.16470 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 20 client.16470 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29fa390 0x55f2b2e2e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 4 client.16470 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2b2d08800 con 0x55f2b29fa390 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f666f7fe640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29fa390 0x55f2b2e2e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f664f7fe640 10 client.16470 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.002+0000 7f666f7fe640 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29fa390 0x55f2b2e2e610 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5778a4ec057813b server_cookie=426309cc5ea583a9 in_seq=0 out_seq=0 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f664f7fe640 1 -- 172.21.15.94:0/2461492222 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f2b2d08800 con 0x55f2b29fa390 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f664f7fe640 10 client.16470 handle_command_reply: tid=0 2024-04-25T13:44:33.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 client.16470 shutdown 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 msgr2=0x55f2b2daa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6660009f70 tx=0x7f666000f830 comp rx=0 tx=0).stop 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 shutdown_connections 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2b29fa390 0x55f2b2e2e610 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b29d96b0 0x55f2b2e2a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b29d14d0 0x55f2b2daab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 --2- 172.21.15.94:0/2461492222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2b29b8f60 0x55f2b2daa5e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f664f7fe640 0 client.16470 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.004+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 >> 172.21.15.94:0/2461492222 conn(0x55f2b29ee4e0 msgr2=0x55f2b2d9a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 shutdown_connections 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f667d281740 1 -- 172.21.15.94:0/2461492222 wait complete. 2024-04-25T13:44:33.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f667d281740 20 client.16470 trim_cache size 0 max 16384 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "60462351-2997-4392-84d4-d345946447d9": { 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "60462351-2997-4392-84d4-d345946447d9", 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:44:33.010 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:44:33.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 msgr2=0x7f66580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 0x7f66580b2000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f2b2d9b6b0 tx=0x7f6660005f90 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 msgr2=0x7f667019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 0x7f667019eba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f666400e9f0 tx=0x7f666400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 shutdown_connections 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f66580afaf0 0x7f66580b2000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f667019f0e0 0x7f66701a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6670108a20 0x7f667019eba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 --2- 172.21.15.94:0/4167187841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6670103ef0 0x7f6670102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 >> 172.21.15.94:0/4167187841 conn(0x7f66700fd350 msgr2=0x7f66700ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 shutdown_connections 2024-04-25T13:44:33.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:44:33.005+0000 7f66763ce640 1 -- 172.21.15.94:0/4167187841 wait complete. 2024-04-25T13:44:33.016 INFO:tasks.cephfs.filesystem:scrub status for tag:60462351-2997-4392-84d4-d345946447d9 - {'path': '/', 'tag': '60462351-2997-4392-84d4-d345946447d9', 'options': 'recursive,force'} 2024-04-25T13:44:33.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202272 14.60 MB/sec warmup 76 sec latency 13.009 ms 2024-04-25T13:44:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:34 smithi137 ceph-mon[32568]: pgmap v429: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T13:44:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:34 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:44:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:34 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:34 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:44:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:34 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:34 smithi094 ceph-mon[29195]: pgmap v429: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T13:44:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:34 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:44:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:34 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:34 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:44:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:34 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:34 smithi138 ceph-mon[32977]: pgmap v429: 97 pgs: 97 active+clean; 37 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T13:44:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:34 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:44:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:34 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:34 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:44:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:34 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:44:34.791 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203824 14.50 MB/sec warmup 77 sec latency 82.544 ms 2024-04-25T13:44:35.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 206277 14.49 MB/sec warmup 78 sec latency 51.637 ms 2024-04-25T13:44:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:36 smithi137 ceph-mon[32568]: pgmap v430: 97 pgs: 97 active+clean; 41 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-04-25T13:44:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:44:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:36 smithi094 ceph-mon[29195]: pgmap v430: 97 pgs: 97 active+clean; 41 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-04-25T13:44:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:44:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:36 smithi138 ceph-mon[32977]: pgmap v430: 97 pgs: 97 active+clean; 41 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 437 op/s 2024-04-25T13:44:36.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:44:36.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 209257 14.53 MB/sec warmup 79 sec latency 44.039 ms 2024-04-25T13:44:37.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 211647 14.45 MB/sec warmup 80 sec latency 13.138 ms 2024-04-25T13:44:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:38 smithi137 ceph-mon[32568]: pgmap v431: 97 pgs: 97 active+clean; 41 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:44:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:38 smithi094 ceph-mon[29195]: pgmap v431: 97 pgs: 97 active+clean; 41 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:44:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:38 smithi138 ceph-mon[32977]: pgmap v431: 97 pgs: 97 active+clean; 41 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:44:38.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213732 14.43 MB/sec warmup 81 sec latency 14.764 ms 2024-04-25T13:44:39.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 216648 14.48 MB/sec warmup 82 sec latency 98.628 ms 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 -- 172.21.15.94:0/4135949872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc1098d0 msgr2=0x7f53cc10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 --2- 172.21.15.94:0/4135949872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc1098d0 0x7f53cc10bcf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009ca0 tx=0x7f53c402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 -- 172.21.15.94:0/4135949872 shutdown_connections 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 --2- 172.21.15.94:0/4135949872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc1098d0 0x7f53cc10bcf0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 --2- 172.21.15.94:0/4135949872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 0x7f53cc109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 --2- 172.21.15.94:0/4135949872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc104610 0x7f53cc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 -- 172.21.15.94:0/4135949872 >> 172.21.15.94:0/4135949872 conn(0x7f53cc0fdc10 msgr2=0x7f53cc1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 -- 172.21.15.94:0/4135949872 shutdown_connections 2024-04-25T13:44:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 -- 172.21.15.94:0/4135949872 wait complete. 2024-04-25T13:44:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.385+0000 7f53d2fba640 1 Processor -- start 2024-04-25T13:44:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 -- start start 2024-04-25T13:44:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 0x7f53cc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc1098d0 0x7f53cc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10f650 con 0x7f53cc104610 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10f4d0 con 0x7f53cc106f70 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d2fba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10f7d0 con 0x7f53cc1098d0 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50300/0 (socket says 172.21.15.94:50300) 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 -- 172.21.15.94:0/1224546423 learned_addr learned my addr 172.21.15.94:0/1224546423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d1557640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc1098d0 0x7f53cc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53cbfff640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 0x7f53cc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 -- 172.21.15.94:0/1224546423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc1098d0 msgr2=0x7f53cc1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc1098d0 0x7f53cc1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 -- 172.21.15.94:0/1224546423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 msgr2=0x7f53cc19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 0x7f53cc19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 -- 172.21.15.94:0/1224546423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc1a1d60 con 0x7f53cc104610 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53d0d56640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f53c000e9f0 tx=0x7f53c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.386+0000 7f53c9ffb640 1 -- 172.21.15.94:0/1224546423 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c000ccc0 con 0x7f53cc104610 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.387+0000 7f53c9ffb640 1 -- 172.21.15.94:0/1224546423 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c00054d0 con 0x7f53cc104610 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.387+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc1a1ff0 con 0x7f53cc104610 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.387+0000 7f53c9ffb640 1 -- 172.21.15.94:0/1224546423 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0010430 con 0x7f53cc104610 2024-04-25T13:44:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.387+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1a2460 con 0x7f53cc104610 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 Processor -- start 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 -- start start 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53c9ffb640 1 -- 172.21.15.94:0/1224546423 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f53c00046c0 con 0x7f53cc104610 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c901ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c908df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55596c867460 con 0x55596c5611b0 2024-04-25T13:44:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55596c821590 con 0x55596c5382f0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55596c863420 con 0x55596c5404d0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50306/0 (socket says 172.21.15.94:50306) 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d1557640 1 -- 172.21.15.94:0/2862537744 learned_addr learned my addr 172.21.15.94:0/2862537744 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53cbfff640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c908df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.390+0000 7f53d0d56640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c901ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 -- 172.21.15.94:0/2862537744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 msgr2=0x55596c908df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c908df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 -- 172.21.15.94:0/2862537744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 msgr2=0x55596c901ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c901ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 -- 172.21.15.94:0/2862537744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596c82e0d0 con 0x55596c5611b0 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d0d56640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c901ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53c9ffb640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 0x7f53bc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d1557640 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f53cc06b4a0 tx=0x7f53c402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6f3806ffd53cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53d0d56640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 0x7f53bc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53b6ffd640 1 -- 172.21.15.94:0/2862537744 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4031650 con 0x55596c5611b0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53b6ffd640 1 -- 172.21.15.94:0/2862537744 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c4036030 con 0x55596c5611b0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53b6ffd640 1 -- 172.21.15.94:0/2862537744 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4004730 con 0x55596c5611b0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.391+0000 7f53c9ffb640 1 -- 172.21.15.94:0/1224546423 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53c00df010 con 0x7f53cc104610 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d0d56640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 0x7f53bc0b1ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55596c902d50 tx=0x7f53c000ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 -- 172.21.15.94:0/2862537744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 msgr2=0x55596c901970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f53cc06b4a0 tx=0x7f53c402f410 comp rx=0 tx=0).stop 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 -- 172.21.15.94:0/2862537744 shutdown_connections 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c908df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c901ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/2862537744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c901970 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 -- 172.21.15.94:0/2862537744 >> 172.21.15.94:0/2862537744 conn(0x55596c555300 msgr2=0x55596c8fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 -- 172.21.15.94:0/2862537744 shutdown_connections 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.392+0000 7f53d9e5f740 1 -- 172.21.15.94:0/2862537744 wait complete. 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 -- start start 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 Processor -- start 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c90b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 0x55596c912900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55596c909660 con 0x55596c5404d0 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55596c7eeb60 con 0x55596c5382f0 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55596c9097e0 con 0x55596c5611b0 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d9e5f740 10 client.0 _ll_register_callbacks cb 0x55596c8132b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50310/0 (socket says 172.21.15.94:50310) 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53cbfff640 1 -- 172.21.15.94:0/608381431 learned_addr learned my addr 172.21.15.94:0/608381431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d1557640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c90b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.393+0000 7f53d1557640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 0x55596c912900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 -- 172.21.15.94:0/608381431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 msgr2=0x55596c912900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 0x55596c912900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 -- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 msgr2=0x55596c90b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c90b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 -- 172.21.15.94:0/608381431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596c579210 con 0x55596c5404d0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53d1557640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 0x55596c912900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53cbfff640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f53b8012700 tx=0x7f53b8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a4900e1f4a723e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8004050 con 0x55596c5404d0 2024-04-25T13:44:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53b80041f0 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c4036030 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53b80043f0 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53d9e5f740 10 client.16482 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55596c645620 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.394+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c4036030 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f53b8003b70 con 0x55596c5404d0 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 10 client.16482 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 20 client.16482 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c51fd80 0x55596c919160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 4 client.16482 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55596c875070 con 0x55596c51fd80 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d1557640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c51fd80 0x55596c919160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53d1557640 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c51fd80 0x55596c919160 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d24cbd6c1085e8 server_cookie=4f54235efc70c59 in_seq=0 out_seq=0 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.395+0000 7f53abfff640 10 client.16482 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.396+0000 7f53abfff640 1 -- 172.21.15.94:0/608381431 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55596c875070 con 0x55596c51fd80 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.396+0000 7f53abfff640 10 client.16482 handle_command_reply: tid=0 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.396+0000 7f53d9e5f740 1 client.16482 shutdown 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 msgr2=0x55596c90b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f53b8012700 tx=0x7f53b8012bc0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 shutdown_connections 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c51fd80 0x55596c919160 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5611b0 0x55596c912900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5404d0 0x55596c90b9a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/608381431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5382f0 0x55596c90b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53abfff640 0 client.16482 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 >> 172.21.15.94:0/608381431 conn(0x55596c555300 msgr2=0x55596c8fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 shutdown_connections 2024-04-25T13:44:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 1 -- 172.21.15.94:0/608381431 wait complete. 2024-04-25T13:44:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.397+0000 7f53d9e5f740 20 client.16482 trim_cache size 0 max 16384 2024-04-25T13:44:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 Processor -- start 2024-04-25T13:44:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 -- start start 2024-04-25T13:44:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c912820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5611b0 0x55596c900100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ab470 con 0x55596c5382f0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1acfb0 con 0x55596c5611b0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1abb40 con 0x55596c5404d0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50320/0 (socket says 172.21.15.94:50320) 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 -- 172.21.15.94:0/4207049044 learned_addr learned my addr 172.21.15.94:0/4207049044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53cbfff640 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c912820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5611b0 0x55596c900100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 -- 172.21.15.94:0/4207049044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 msgr2=0x55596c912820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c912820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 -- 172.21.15.94:0/4207049044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5611b0 msgr2=0x55596c900100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5611b0 0x55596c900100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.414+0000 7f53d1557640 1 -- 172.21.15.94:0/4207049044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596c912d60 con 0x55596c5382f0 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.415+0000 7f53d1557640 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f53c4000f80 tx=0x7f53c402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eea1ec55d99898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.415+0000 7f53b67fc640 1 -- 172.21.15.94:0/4207049044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40046d0 con 0x55596c5382f0 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.415+0000 7f53b67fc640 1 -- 172.21.15.94:0/4207049044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c405d030 con 0x55596c5382f0 2024-04-25T13:44:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.415+0000 7f53b67fc640 1 -- 172.21.15.94:0/4207049044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c402f8c0 con 0x55596c5382f0 2024-04-25T13:44:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- 172.21.15.94:0/4207049044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 msgr2=0x55596c8fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f53c4000f80 tx=0x7f53c402f3a0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- 172.21.15.94:0/4207049044 shutdown_connections 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5611b0 0x55596c900100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5404d0 0x55596c912820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/4207049044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5382f0 0x55596c8fd2b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- 172.21.15.94:0/4207049044 >> 172.21.15.94:0/4207049044 conn(0x55596c555300 msgr2=0x55596c8fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- 172.21.15.94:0/4207049044 shutdown_connections 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- 172.21.15.94:0/4207049044 wait complete. 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 -- start start 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.416+0000 7f53d9e5f740 1 Processor -- start 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5382f0 0x55596c90fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5404d0 0x55596c910390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ac8e0 con 0x55596c5611b0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1ac210 con 0x55596c5404d0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc1a9450 con 0x55596c5382f0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d9e5f740 10 client.0 _ll_register_callbacks cb 0x7f53b80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50334/0 (socket says 172.21.15.94:50334) 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 -- 172.21.15.94:0/247198046 learned_addr learned my addr 172.21.15.94:0/247198046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53cbfff640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5404d0 0x55596c910390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5382f0 0x55596c90fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 -- 172.21.15.94:0/247198046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5382f0 msgr2=0x55596c90fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5382f0 0x55596c90fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 -- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5404d0 msgr2=0x55596c910390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5404d0 0x55596c910390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 -- 172.21.15.94:0/247198046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596c911810 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53d1557640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f53c4031b50 tx=0x7f53c4057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff1b1ec69ebe378 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.417+0000 7f53abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003a50 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f53c4031450 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c405d030 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53c4031650 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 10 client.16488 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55596c912aa0 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c405d030 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f53c4043070 con 0x55596c5611b0 2024-04-25T13:44:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 10 client.16488 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 20 client.16488 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c4f51c0 0x55596c983860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 4 client.16488 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55596c86eee0 con 0x55596c4f51c0 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.418+0000 7f53cbfff640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c4f51c0 0x55596c983860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.419+0000 7f53abfff640 10 client.16488 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.419+0000 7f53cbfff640 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c4f51c0 0x55596c983860 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4131c1a54ea7510 server_cookie=7f7209acb1ec1dfb in_seq=0 out_seq=0 2024-04-25T13:44:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53abfff640 1 -- 172.21.15.94:0/247198046 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55596c86eee0 con 0x55596c4f51c0 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53abfff640 10 client.16488 handle_command_reply: tid=0 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 client.16488 shutdown 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 msgr2=0x55596c911250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f53c4031b50 tx=0x7f53c4057040 comp rx=0 tx=0).stop 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 shutdown_connections 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55596c4f51c0 0x55596c983860 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53abfff640 0 client.16488 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55596c5611b0 0x55596c911250 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596c5404d0 0x55596c910390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 --2- 172.21.15.94:0/247198046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55596c5382f0 0x55596c90fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 >> 172.21.15.94:0/247198046 conn(0x55596c555300 msgr2=0x55596c8fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 shutdown_connections 2024-04-25T13:44:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 1 -- 172.21.15.94:0/247198046 wait complete. 2024-04-25T13:44:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.552+0000 7f53d9e5f740 20 client.16488 trim_cache size 0 max 16384 2024-04-25T13:44:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:44:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:44:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:44:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 msgr2=0x7f53bc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 0x7f53bc0b1ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55596c902d50 tx=0x7f53c000ce60 comp rx=0 tx=0).stop 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 msgr2=0x7f53cc19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f53c000e9f0 tx=0x7f53c000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 shutdown_connections 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53bc0afae0 0x7f53bc0b1ff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc1098d0 0x7f53cc1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc106f70 0x7f53cc19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 --2- 172.21.15.94:0/1224546423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc104610 0x7f53cc19a220 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 >> 172.21.15.94:0/1224546423 conn(0x7f53cc0fdc10 msgr2=0x7f53cc105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 shutdown_connections 2024-04-25T13:44:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:40.553+0000 7f53d2fba640 1 -- 172.21.15.94:0/1224546423 wait complete. 2024-04-25T13:44:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:40 smithi137 ceph-mon[32568]: pgmap v432: 97 pgs: 97 active+clean; 40 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-04-25T13:44:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:40 smithi094 ceph-mon[29195]: pgmap v432: 97 pgs: 97 active+clean; 40 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-04-25T13:44:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:40 smithi138 ceph-mon[32977]: pgmap v432: 97 pgs: 97 active+clean; 40 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 347 op/s 2024-04-25T13:44:40.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 218198 14.37 MB/sec warmup 83 sec latency 116.312 ms 2024-04-25T13:44:41.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220683 14.37 MB/sec warmup 84 sec latency 12.823 ms 2024-04-25T13:44:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:42 smithi137 ceph-mon[32568]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2024-04-25T13:44:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:42 smithi094 ceph-mon[29195]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2024-04-25T13:44:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:42 smithi138 ceph-mon[32977]: pgmap v433: 97 pgs: 97 active+clean; 45 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 436 op/s 2024-04-25T13:44:42.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 223739 14.42 MB/sec warmup 85 sec latency 12.165 ms 2024-04-25T13:44:43.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 226072 14.35 MB/sec warmup 86 sec latency 35.038 ms 2024-04-25T13:44:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:44 smithi137 ceph-mon[32568]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-25T13:44:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:44 smithi094 ceph-mon[29195]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-25T13:44:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:44 smithi138 ceph-mon[32977]: pgmap v434: 97 pgs: 97 active+clean; 45 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2024-04-25T13:44:44.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 228225 14.33 MB/sec warmup 87 sec latency 18.130 ms 2024-04-25T13:44:45.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 230824 14.36 MB/sec warmup 88 sec latency 74.364 ms 2024-04-25T13:44:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:46 smithi137 ceph-mon[32568]: pgmap v435: 97 pgs: 97 active+clean; 40 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 426 op/s 2024-04-25T13:44:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:46 smithi094 ceph-mon[29195]: pgmap v435: 97 pgs: 97 active+clean; 40 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 426 op/s 2024-04-25T13:44:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:46 smithi138 ceph-mon[32977]: pgmap v435: 97 pgs: 97 active+clean; 40 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 426 op/s 2024-04-25T13:44:46.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 233382 14.33 MB/sec warmup 89 sec latency 14.475 ms 2024-04-25T13:44:47.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235409 14.28 MB/sec warmup 90 sec latency 14.902 ms 2024-04-25T13:44:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:48 smithi137 ceph-mon[32568]: pgmap v436: 97 pgs: 97 active+clean; 40 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-04-25T13:44:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:48 smithi094 ceph-mon[29195]: pgmap v436: 97 pgs: 97 active+clean; 40 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-04-25T13:44:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:48 smithi138 ceph-mon[32977]: pgmap v436: 97 pgs: 97 active+clean; 40 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-04-25T13:44:48.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 238163 14.31 MB/sec warmup 91 sec latency 13.137 ms 2024-04-25T13:44:49.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 240882 14.30 MB/sec warmup 92 sec latency 64.292 ms 2024-04-25T13:44:50.548 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:50 smithi138 ceph-mon[32977]: pgmap v437: 97 pgs: 97 active+clean; 40 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-04-25T13:44:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:50 smithi137 ceph-mon[32568]: pgmap v437: 97 pgs: 97 active+clean; 40 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-04-25T13:44:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:50 smithi094 ceph-mon[29195]: pgmap v437: 97 pgs: 97 active+clean; 40 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 346 op/s 2024-04-25T13:44:50.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 242422 14.23 MB/sec warmup 93 sec latency 100.017 ms 2024-04-25T13:44:51.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 245202 14.26 MB/sec warmup 94 sec latency 13.757 ms 2024-04-25T13:44:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:52 smithi137 ceph-mon[32568]: pgmap v438: 97 pgs: 97 active+clean; 45 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-04-25T13:44:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:52 smithi094 ceph-mon[29195]: pgmap v438: 97 pgs: 97 active+clean; 45 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-04-25T13:44:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:52 smithi138 ceph-mon[32977]: pgmap v438: 97 pgs: 97 active+clean; 45 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-04-25T13:44:52.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 248137 14.26 MB/sec warmup 95 sec latency 14.192 ms 2024-04-25T13:44:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- 172.21.15.94:0/733234487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 msgr2=0x7fd684104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/733234487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684104070 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd674009900 tx=0x7fd67402a410 comp rx=0 tx=0).stop 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- 172.21.15.94:0/733234487 shutdown_connections 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/733234487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684108960 0x7fd68410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/733234487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684104070 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/733234487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684102a20 0x7fd684103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- 172.21.15.94:0/733234487 >> 172.21.15.94:0/733234487 conn(0x7fd6840fdc10 msgr2=0x7fd6841000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- 172.21.15.94:0/733234487 shutdown_connections 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- 172.21.15.94:0/733234487 wait complete. 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 Processor -- start 2024-04-25T13:44:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.653+0000 7fd68a8f2640 1 -- start start 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684102a20 0x7fd684199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684108960 0x7fd6841a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e520 con 0x7fd684103c60 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e3a0 con 0x7fd684108960 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd68a8f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e6a0 con 0x7fd684102a20 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41274/0 (socket says 172.21.15.94:41274) 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 -- 172.21.15.94:0/2706930995 learned_addr learned my addr 172.21.15.94:0/2706930995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd688e8f640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684108960 0x7fd6841a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd683fff640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684102a20 0x7fd684199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 -- 172.21.15.94:0/2706930995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684102a20 msgr2=0x7fd684199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684102a20 0x7fd684199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 -- 172.21.15.94:0/2706930995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684108960 msgr2=0x7fd6841a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684108960 0x7fd6841a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 -- 172.21.15.94:0/2706930995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68406a440 con 0x7fd684103c60 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.654+0000 7fd6837fe640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd674006010 tx=0x7fd674031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.655+0000 7fd6817fa640 1 -- 172.21.15.94:0/2706930995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67403f030 con 0x7fd684103c60 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.655+0000 7fd6817fa640 1 -- 172.21.15.94:0/2706930995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd674003aa0 con 0x7fd684103c60 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.655+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68406a6d0 con 0x7fd684103c60 2024-04-25T13:44:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.655+0000 7fd6817fa640 1 -- 172.21.15.94:0/2706930995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674035070 con 0x7fd684103c60 2024-04-25T13:44:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.655+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68406abc0 con 0x7fd684103c60 2024-04-25T13:44:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.657+0000 7fd6917a7740 1 Processor -- start 2024-04-25T13:44:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.657+0000 7fd6917a7740 1 -- start start 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.657+0000 7fd6817fa640 1 -- 172.21.15.94:0/2706930995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd674042030 con 0x7fd684103c60 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efae4d0 0x562e4f376df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562e4f2d5460 con 0x562e4efcf1b0 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562e4f28f590 con 0x562e4efa62f0 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6917a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e4f2d1420 con 0x562e4efae4d0 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd688e8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efae4d0 0x562e4f376df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33892/0 (socket says 172.21.15.94:33892) 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd688e8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41278/0 (socket says 172.21.15.94:41278) 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 -- 172.21.15.94:0/3859665105 learned_addr learned my addr 172.21.15.94:0/3859665105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 -- 172.21.15.94:0/3859665105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efae4d0 msgr2=0x562e4f376df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efae4d0 0x562e4f376df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 -- 172.21.15.94:0/3859665105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 msgr2=0x562e4f36f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 -- 172.21.15.94:0/3859665105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4f29c0d0 con 0x562e4efa62f0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd688e8f640 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6817fa640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 0x7fd6700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd688e8f640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 0x7fd6700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd683fff640 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009c50 tx=0x7fd66c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fada0853154cf616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd6817fa640 1 -- 172.21.15.94:0/2706930995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6740ff230 con 0x7fd684103c60 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.658+0000 7fd67a7fc640 1 -- 172.21.15.94:0/3859665105 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004030 con 0x562e4efa62f0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.659+0000 7fd67a7fc640 1 -- 172.21.15.94:0/3859665105 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c0035f0 con 0x562e4efa62f0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.659+0000 7fd67a7fc640 1 -- 172.21.15.94:0/3859665105 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0196d0 con 0x562e4efa62f0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.659+0000 7fd6917a7740 1 -- 172.21.15.94:0/3859665105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 msgr2=0x562e4f36fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.659+0000 7fd688e8f640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 0x7fd6700b1ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x562e4f370b40 tx=0x7fd67c007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.659+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009c50 tx=0x7fd66c011810 comp rx=0 tx=0).stop 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- 172.21.15.94:0/3859665105 shutdown_connections 2024-04-25T13:44:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efae4d0 0x562e4f376df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f36fef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3859665105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efcf1b0 0x562e4f36f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- 172.21.15.94:0/3859665105 >> 172.21.15.94:0/3859665105 conn(0x562e4efc3300 msgr2=0x562e4f36b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- 172.21.15.94:0/3859665105 shutdown_connections 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- 172.21.15.94:0/3859665105 wait complete. 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- start start 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 Processor -- start 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f379650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f380a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562e4f377840 con 0x562e4efae4d0 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562e4f25cb60 con 0x562e4efa62f0 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.660+0000 7fd6917a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e4f3779c0 con 0x562e4efcf1b0 2024-04-25T13:44:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6917a7740 10 client.0 _ll_register_callbacks cb 0x562e4f2812b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41280/0 (socket says 172.21.15.94:41280) 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 -- 172.21.15.94:0/1492114211 learned_addr learned my addr 172.21.15.94:0/1492114211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd683fff640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f379650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd688e8f640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f380a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 -- 172.21.15.94:0/1492114211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 msgr2=0x562e4f380a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f380a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 -- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 msgr2=0x562e4f379650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f379650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 -- 172.21.15.94:0/1492114211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4efe7210 con 0x562e4efae4d0 2024-04-25T13:44:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6837fe640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6740c4990 tx=0x7fd674003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798b16a54ecd0142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6637fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674049070 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd6740b00b0 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66c0035f0 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6740c35e0 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6917a7740 10 client.16500 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.661+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e4f0b3620 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c0035f0 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd6740c7980 con 0x562e4efae4d0 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 10 client.16500 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 20 client.16500 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef8dd80 0x562e4f3872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 4 client.16500 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e4f2e3070 con 0x562e4ef8dd80 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd683fff640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef8dd80 0x562e4f3872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd683fff640 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef8dd80 0x562e4f3872d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9687c48e7cd622a4 server_cookie=c2ed0824a3b2607c in_seq=0 out_seq=0 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.662+0000 7fd6637fe640 10 client.16500 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6637fe640 1 -- 172.21.15.94:0/1492114211 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e4f2e3070 con 0x562e4ef8dd80 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6637fe640 10 client.16500 handle_command_reply: tid=0 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 client.16500 shutdown 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 msgr2=0x562e4f379b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6740c4990 tx=0x7fd674003e80 comp rx=0 tx=0).stop 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 shutdown_connections 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef8dd80 0x562e4f3872d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f380a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f379b90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6637fe640 0 client.16500 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 --2- 172.21.15.94:0/1492114211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efa62f0 0x562e4f379650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 >> 172.21.15.94:0/1492114211 conn(0x562e4efc3300 msgr2=0x562e4f36e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.663+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 shutdown_connections 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.664+0000 7fd6917a7740 1 -- 172.21.15.94:0/1492114211 wait complete. 2024-04-25T13:44:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.664+0000 7fd6917a7740 20 client.16500 trim_cache size 0 max 16384 2024-04-25T13:44:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 Processor -- start 2024-04-25T13:44:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 -- start start 2024-04-25T13:44:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efa62f0 0x562e4f36ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efcf1b0 0x562e4f36b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841ad630 con 0x562e4efae4d0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841af170 con 0x562e4efcf1b0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6917a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841add00 con 0x562e4efa62f0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41292/0 (socket says 172.21.15.94:41292) 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd6837fe640 1 -- 172.21.15.94:0/3384366127 learned_addr learned my addr 172.21.15.94:0/3384366127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd688e8f640 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efcf1b0 0x562e4f36b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.679+0000 7fd683fff640 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efa62f0 0x562e4f36ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 -- 172.21.15.94:0/3384366127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efa62f0 msgr2=0x562e4f36ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efa62f0 0x562e4f36ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 -- 172.21.15.94:0/3384366127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efcf1b0 msgr2=0x562e4f36b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efcf1b0 0x562e4f36b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 -- 172.21.15.94:0/3384366127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4f377a80 con 0x562e4efae4d0 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd6837fe640 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd674031440 tx=0x7fd67403b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a881b27393d169ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd679ffb640 1 -- 172.21.15.94:0/3384366127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674049070 con 0x562e4efae4d0 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd679ffb640 1 -- 172.21.15.94:0/3384366127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6740c7230 con 0x562e4efae4d0 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.680+0000 7fd679ffb640 1 -- 172.21.15.94:0/3384366127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6740c6e90 con 0x562e4efae4d0 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- 172.21.15.94:0/3384366127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 msgr2=0x562e4f3809a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd674031440 tx=0x7fd67403b640 comp rx=0 tx=0).stop 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- 172.21.15.94:0/3384366127 shutdown_connections 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efcf1b0 0x562e4f36b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efae4d0 0x562e4f3809a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 --2- 172.21.15.94:0/3384366127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efa62f0 0x562e4f36ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- 172.21.15.94:0/3384366127 >> 172.21.15.94:0/3384366127 conn(0x562e4efc3300 msgr2=0x562e4f373ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- 172.21.15.94:0/3384366127 shutdown_connections 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- 172.21.15.94:0/3384366127 wait complete. 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 -- start start 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.681+0000 7fd6917a7740 1 Processor -- start 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efae4d0 0x562e4f37ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f37fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841aeaa0 con 0x562e4efa62f0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841ae3d0 con 0x562e4efae4d0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841ab610 con 0x562e4efcf1b0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 10 client.0 _ll_register_callbacks cb 0x7fd674005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41306/0 (socket says 172.21.15.94:41306) 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 -- 172.21.15.94:0/4040714018 learned_addr learned my addr 172.21.15.94:0/4040714018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6837fe640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efae4d0 0x562e4f37ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd688e8f640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f37fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 -- 172.21.15.94:0/4040714018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 msgr2=0x562e4f37fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f37fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 -- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efae4d0 msgr2=0x562e4f37ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efae4d0 0x562e4f37ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 -- 172.21.15.94:0/4040714018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4f3800a0 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd683fff640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd66c014040 tx=0x7fd66c009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a61897ac98c6dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6637fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c01f040 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd66c0050c0 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.682+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6740c7230 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd66c0035f0 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 10 client.16506 fetch_fsmap learned FSMap version 25 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e4f3efb40 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6740c7230 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd66c034070 con 0x562e4efa62f0 2024-04-25T13:44:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 10 client.16506 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 20 client.16506 populate_metadata read hostname 'smithi094' 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef631c0 0x562e4f3f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 4 client.16506 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e4f2dcee0 con 0x562e4ef631c0 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.683+0000 7fd6837fe640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef631c0 0x562e4f3f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.684+0000 7fd6637fe640 10 client.16506 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.684+0000 7fd6837fe640 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef631c0 0x562e4f3f2390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cff957d6b6130a5 server_cookie=aafef21663963a98 in_seq=0 out_seq=0 2024-04-25T13:44:53.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249848 14.22 MB/sec warmup 96 sec latency 58.222 ms 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6637fe640 1 -- 172.21.15.94:0/4040714018 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e4f2dcee0 con 0x562e4ef631c0 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6637fe640 10 client.16506 handle_command_reply: tid=0 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 client.16506 shutdown 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 msgr2=0x562e4f37e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd66c014040 tx=0x7fd66c009fb0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 shutdown_connections 2024-04-25T13:44:53.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562e4ef631c0 0x562e4f3f2390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e4efcf1b0 0x562e4f37fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6637fe640 0 client.16506 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e4efae4d0 0x562e4f37ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 --2- 172.21.15.94:0/4040714018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562e4efa62f0 0x562e4f37e670 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 >> 172.21.15.94:0/4040714018 conn(0x562e4efc3300 msgr2=0x562e4f373ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 shutdown_connections 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.823+0000 7fd6917a7740 1 -- 172.21.15.94:0/4040714018 wait complete. 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd6917a7740 20 client.16506 trim_cache size 0 max 16384 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:44:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:44:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:44:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 msgr2=0x7fd6700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 0x7fd6700b1ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x562e4f370b40 tx=0x7fd67c007520 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 msgr2=0x7fd684199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd674006010 tx=0x7fd674031a50 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 shutdown_connections 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6700afae0 0x7fd6700b1ff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd684108960 0x7fd6841a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd684103c60 0x7fd684199660 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 --2- 172.21.15.94:0/2706930995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd684102a20 0x7fd684199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 >> 172.21.15.94:0/2706930995 conn(0x7fd6840fdc10 msgr2=0x7fd684104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 shutdown_connections 2024-04-25T13:44:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:44:53.824+0000 7fd68a8f2640 1 -- 172.21.15.94:0/2706930995 wait complete. 2024-04-25T13:44:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:54 smithi137 ceph-mon[32568]: pgmap v439: 97 pgs: 97 active+clean; 48 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-04-25T13:44:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:54 smithi094 ceph-mon[29195]: pgmap v439: 97 pgs: 97 active+clean; 48 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-04-25T13:44:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:54 smithi138 ceph-mon[32977]: pgmap v439: 97 pgs: 97 active+clean; 48 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-04-25T13:44:54.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252361 14.21 MB/sec warmup 97 sec latency 72.584 ms 2024-04-25T13:44:55.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 255075 14.20 MB/sec warmup 98 sec latency 94.558 ms 2024-04-25T13:44:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:56 smithi137 ceph-mon[32568]: pgmap v440: 97 pgs: 97 active+clean; 44 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-25T13:44:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:56 smithi094 ceph-mon[29195]: pgmap v440: 97 pgs: 97 active+clean; 44 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-25T13:44:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:56 smithi138 ceph-mon[32977]: pgmap v440: 97 pgs: 97 active+clean; 44 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 471 op/s 2024-04-25T13:44:56.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257059 14.17 MB/sec warmup 99 sec latency 13.753 ms 2024-04-25T13:44:57.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 259578 14.17 MB/sec warmup 100 sec latency 12.196 ms 2024-04-25T13:44:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:44:58 smithi137 ceph-mon[32568]: pgmap v441: 97 pgs: 97 active+clean; 43 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 345 op/s 2024-04-25T13:44:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:44:58 smithi094 ceph-mon[29195]: pgmap v441: 97 pgs: 97 active+clean; 43 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 345 op/s 2024-04-25T13:44:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:44:58 smithi138 ceph-mon[32977]: pgmap v441: 97 pgs: 97 active+clean; 43 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 345 op/s 2024-04-25T13:44:58.792 INFO:tasks.workunit.client.0.smithi094.stdout: 1 262623 14.20 MB/sec warmup 101 sec latency 16.575 ms 2024-04-25T13:44:59.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 264890 14.15 MB/sec warmup 102 sec latency 14.224 ms 2024-04-25T13:45:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:00 smithi137 ceph-mon[32568]: pgmap v442: 97 pgs: 97 active+clean; 43 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-25T13:45:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:00 smithi094 ceph-mon[29195]: pgmap v442: 97 pgs: 97 active+clean; 43 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-25T13:45:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:00 smithi138 ceph-mon[32977]: pgmap v442: 97 pgs: 97 active+clean; 43 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-25T13:45:00.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 266721 14.13 MB/sec warmup 103 sec latency 86.801 ms 2024-04-25T13:45:01.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 269725 14.16 MB/sec warmup 104 sec latency 22.734 ms 2024-04-25T13:45:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:02 smithi137 ceph-mon[32568]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-04-25T13:45:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:02 smithi094 ceph-mon[29195]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-04-25T13:45:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:02 smithi138 ceph-mon[32977]: pgmap v443: 97 pgs: 97 active+clean; 43 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-04-25T13:45:02.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 272164 14.12 MB/sec warmup 105 sec latency 16.312 ms 2024-04-25T13:45:03.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:45:03.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:45:03.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 -- 172.21.15.94:0/2591442073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8108920 msgr2=0x7f2af810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 --2- 172.21.15.94:0/2591442073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8108920 0x7f2af810ad40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009ca0 tx=0x7f2af002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 -- 172.21.15.94:0/2591442073 shutdown_connections 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 --2- 172.21.15.94:0/2591442073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af8108920 0x7f2af810ad40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 --2- 172.21.15.94:0/2591442073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8103c20 0x7f2af8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 --2- 172.21.15.94:0/2591442073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af81029e0 0x7f2af81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 -- 172.21.15.94:0/2591442073 >> 172.21.15.94:0/2591442073 conn(0x7f2af80fdbd0 msgr2=0x7f2af8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 -- 172.21.15.94:0/2591442073 shutdown_connections 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 -- 172.21.15.94:0/2591442073 wait complete. 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.135+0000 7f2afd613640 1 Processor -- start 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 -- start start 2024-04-25T13:45:03.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 0x7f2af819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8108920 0x7f2af81a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2af810f670 con 0x7f2af81029e0 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2af810f4f0 con 0x7f2af8108920 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2afd613640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2af810f7f0 con 0x7f2af8103c20 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41330/0 (socket says 172.21.15.94:41330) 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 0x7f2af819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2019612396 learned_addr learned my addr 172.21.15.94:0/2019612396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af77fe640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8108920 0x7f2af81a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2019612396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 msgr2=0x7f2af819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 0x7f2af819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2019612396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8108920 msgr2=0x7f2af81a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8108920 0x7f2af81a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2019612396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81a2000 con 0x7f2af81029e0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.136+0000 7f2af67fc640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 0x7f2af819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9f0 tx=0x7f2aec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2adffff640 1 -- 172.21.15.94:0/2019612396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00ccc0 con 0x7f2af81029e0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2adffff640 1 -- 172.21.15.94:0/2019612396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0035f0 con 0x7f2af81029e0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2adffff640 1 -- 172.21.15.94:0/2019612396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00ce60 con 0x7f2af81029e0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af81a22f0 con 0x7f2af81029e0 2024-04-25T13:45:03.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.137+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81a2840 con 0x7f2af81029e0 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.139+0000 7f2adffff640 1 -- 172.21.15.94:0/2019612396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2aec003aa0 con 0x7f2af81029e0 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.139+0000 7f2b0459e740 1 Processor -- start 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.139+0000 7f2b0459e740 1 -- start start 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f4f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f456d80 con 0x55dc5f150390 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f4108b0 con 0x55dc5f1274d0 2024-04-25T13:45:03.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2b0459e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f452d40 con 0x55dc5f12f6b0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f4f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f4f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41342/0 (socket says 172.21.15.94:41342) 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af77fe640 1 -- 172.21.15.94:0/2758102540 learned_addr learned my addr 172.21.15.94:0/2758102540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af67fc640 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2adffff640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 0x7f2ae00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af77fe640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 0x7f2ae00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2adffff640 1 -- 172.21.15.94:0/2019612396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2aec0def90 con 0x7f2af81029e0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2758102540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 msgr2=0x55dc5f4f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2758102540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 msgr2=0x55dc5f4f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f4f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2758102540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5f3e1a40 con 0x55dc5f1274d0 2024-04-25T13:45:03.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af67fc640 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4f1fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0dca40 tx=0x7f2aec003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba0833e1f905027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2af77fe640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 0x7f2ae00b2080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55dc5f4f2ab0 tx=0x7f2af002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.140+0000 7f2add7fa640 1 -- 172.21.15.94:0/2758102540 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0a6b40 con 0x55dc5f1274d0 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2add7fa640 1 -- 172.21.15.94:0/2758102540 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0a6ee0 con 0x55dc5f1274d0 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2add7fa640 1 -- 172.21.15.94:0/2758102540 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0afda0 con 0x55dc5f1274d0 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 -- 172.21.15.94:0/2758102540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 msgr2=0x55dc5f4f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4f1fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0dca40 tx=0x7f2aec003a10 comp rx=0 tx=0).stop 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 -- 172.21.15.94:0/2758102540 shutdown_connections 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4f1fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2758102540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f4f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 -- 172.21.15.94:0/2758102540 >> 172.21.15.94:0/2758102540 conn(0x55dc5f1444e0 msgr2=0x55dc5f4ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 -- 172.21.15.94:0/2758102540 shutdown_connections 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.141+0000 7f2b0459e740 1 -- 172.21.15.94:0/2758102540 wait complete. 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 -- start start 2024-04-25T13:45:03.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 Processor -- start 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f4f3720 con 0x55dc5f150390 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f3ddd10 con 0x55dc5f1274d0 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc5f4f38a0 con 0x55dc5f12f6b0 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2b0459e740 10 client.0 _ll_register_callbacks cb 0x55dc5f40a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41350/0 (socket says 172.21.15.94:41350) 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2af77fe640 1 -- 172.21.15.94:0/154577981 learned_addr learned my addr 172.21.15.94:0/154577981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.142+0000 7f2af67fc640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 -- 172.21.15.94:0/154577981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 msgr2=0x55dc5f4fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 -- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 msgr2=0x55dc5f4fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 -- 172.21.15.94:0/154577981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5f2327d0 con 0x55dc5f150390 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2af77fe640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005e30 tx=0x7f2af00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b167ac17d50aca1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0004710 con 0x55dc5f150390 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2af002f610 con 0x55dc5f150390 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec0a6ee0 con 0x55dc5f150390 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2af0004b40 con 0x55dc5f150390 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2b0459e740 10 client.16518 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc5f1683f0 con 0x55dc5f150390 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0a6ee0 con 0x55dc5f150390 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.143+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2af004a040 con 0x55dc5f150390 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 10 client.16518 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 20 client.16518 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f10ef60 0x55dc5f509360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 4 client.16518 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc5f464990 con 0x55dc5f10ef60 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f10ef60 0x55dc5f509360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f10ef60 0x55dc5f509360 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc5caf9e60e104a server_cookie=beeb1b6c7f0143e3 in_seq=0 out_seq=0 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.144+0000 7f2adaff5640 10 client.16518 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2adaff5640 1 -- 172.21.15.94:0/154577981 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc5f464990 con 0x55dc5f10ef60 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2adaff5640 10 client.16518 handle_command_reply: tid=0 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 client.16518 shutdown 2024-04-25T13:45:03.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 msgr2=0x55dc5f502e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005e30 tx=0x7f2af00095c0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 shutdown_connections 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f10ef60 0x55dc5f509360 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f502e40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 --2- 172.21.15.94:0/154577981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2adaff5640 0 client.16518 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 >> 172.21.15.94:0/154577981 conn(0x55dc5f1444e0 msgr2=0x55dc5f4f0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 shutdown_connections 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 1 -- 172.21.15.94:0/154577981 wait complete. 2024-04-25T13:45:03.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.145+0000 7f2b0459e740 20 client.16518 trim_cache size 0 max 16384 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 Processor -- start 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 -- start start 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f150390 0x55dc5f4f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ab560 con 0x55dc5f1274d0 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ad0a0 con 0x55dc5f12f6b0 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2b0459e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81abc30 con 0x55dc5f150390 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41360/0 (socket says 172.21.15.94:41360) 2024-04-25T13:45:03.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2432520234 learned_addr learned my addr 172.21.15.94:0/2432520234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af67fc640 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2432520234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f150390 msgr2=0x55dc5f4f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f150390 0x55dc5f4f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2432520234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 msgr2=0x55dc5f4ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af6ffd640 1 -- 172.21.15.94:0/2432520234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5f1280f0 con 0x55dc5f1274d0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.163+0000 7f2af67fc640 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0dca40 tx=0x7f2aec0df290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1266adbadc953d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2adcff9640 1 -- 172.21.15.94:0/2432520234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0a6b40 con 0x55dc5f1274d0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2adcff9640 1 -- 172.21.15.94:0/2432520234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0a6ee0 con 0x55dc5f1274d0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2adcff9640 1 -- 172.21.15.94:0/2432520234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0afda0 con 0x55dc5f1274d0 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2b0459e740 1 -- 172.21.15.94:0/2432520234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 msgr2=0x55dc5f4ecc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.164+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0dca40 tx=0x7f2aec0df290 comp rx=0 tx=0).stop 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- 172.21.15.94:0/2432520234 shutdown_connections 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f150390 0x55dc5f4f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- 172.21.15.94:0/2432520234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ecc30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- 172.21.15.94:0/2432520234 >> 172.21.15.94:0/2432520234 conn(0x55dc5f1444e0 msgr2=0x55dc5f4f9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- 172.21.15.94:0/2432520234 shutdown_connections 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- 172.21.15.94:0/2432520234 wait complete. 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- start start 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 Processor -- start 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ac9d0 con 0x55dc5f150390 2024-04-25T13:45:03.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ac300 con 0x55dc5f1274d0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81a9540 con 0x55dc5f12f6b0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2b0459e740 10 client.0 _ll_register_callbacks cb 0x7f2af0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2af67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2af77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41364/0 (socket says 172.21.15.94:41364) 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2af77fe640 1 -- 172.21.15.94:0/3487246489 learned_addr learned my addr 172.21.15.94:0/3487246489 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.165+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 -- 172.21.15.94:0/3487246489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 msgr2=0x55dc5f4ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 -- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 msgr2=0x55dc5f4ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 -- 172.21.15.94:0/3487246489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5f4174b0 con 0x55dc5f150390 2024-04-25T13:45:03.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2af77fe640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2af0031550 tx=0x7f2af0030c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65785a14eae47966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0063060 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2af002f610 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec0a6ee0 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2af0004820 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 10 client.16524 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc5f502300 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0a6ee0 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2af004a040 con 0x55dc5f150390 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 10 client.16524 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 20 client.16524 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.166+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f0e43a0 0x55dc5f573a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.167+0000 7f2b0459e740 4 client.16524 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:45:03.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.167+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc5f45e800 con 0x55dc5f0e43a0 2024-04-25T13:45:03.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.167+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f0e43a0 0x55dc5f573a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.167+0000 7f2adaff5640 10 client.16524 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:03.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.167+0000 7f2af6ffd640 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f0e43a0 0x55dc5f573a20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6895d7bd68b808cd server_cookie=6967a2df804f2778 in_seq=0 out_seq=0 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.168+0000 7f2adaff5640 1 -- 172.21.15.94:0/3487246489 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc5f45e800 con 0x55dc5f0e43a0 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.168+0000 7f2adaff5640 10 client.16524 handle_command_reply: tid=0 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 client.16524 shutdown 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 msgr2=0x55dc5f500c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2af0031550 tx=0x7f2af0030c00 comp rx=0 tx=0).stop 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 shutdown_connections 2024-04-25T13:45:03.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc5f0e43a0 0x55dc5f573a20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc5f150390 0x55dc5f500c50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc5f12f6b0 0x55dc5f4ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2adaff5640 0 client.16524 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 --2- 172.21.15.94:0/3487246489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc5f1274d0 0x55dc5f4ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 >> 172.21.15.94:0/3487246489 conn(0x55dc5f1444e0 msgr2=0x55dc5f4ecc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 shutdown_connections 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 1 -- 172.21.15.94:0/3487246489 wait complete. 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2b0459e740 20 client.16524 trim_cache size 0 max 16384 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:03.175 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:45:03.176 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:45:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:03.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 msgr2=0x7f2ae00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 0x7f2ae00b2080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55dc5f4f2ab0 tx=0x7f2af002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 msgr2=0x7f2af819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e9f0 tx=0x7f2aec00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 shutdown_connections 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ae00afb70 0x7f2ae00b2080 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af8108920 0x7f2af81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af8103c20 0x7f2af819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 --2- 172.21.15.94:0/2019612396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2af81029e0 0x7f2af819a4c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 >> 172.21.15.94:0/2019612396 conn(0x7f2af80fdbd0 msgr2=0x7f2af8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 shutdown_connections 2024-04-25T13:45:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.169+0000 7f2afd613640 1 -- 172.21.15.94:0/2019612396 wait complete. 2024-04-25T13:45:03.181 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:45:03.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:03.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 -- 172.21.15.94:0/2249731338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00801050a0 msgr2=0x7f00801054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 --2- 172.21.15.94:0/2249731338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00801050a0 0x7f00801054b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0074009900 tx=0x7f007402a410 comp rx=0 tx=0).stop 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 -- 172.21.15.94:0/2249731338 shutdown_connections 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 --2- 172.21.15.94:0/2249731338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080102250 0x7f008010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 --2- 172.21.15.94:0/2249731338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0080101750 0x7f0080101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 --2- 172.21.15.94:0/2249731338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00801050a0 0x7f00801054b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 -- 172.21.15.94:0/2249731338 >> 172.21.15.94:0/2249731338 conn(0x7f00800fd370 msgr2=0x7f00800ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 -- 172.21.15.94:0/2249731338 shutdown_connections 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.294+0000 7f0084c13640 1 -- 172.21.15.94:0/2249731338 wait complete. 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 Processor -- start 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 -- start start 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101750 0x7f008019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00801050a0 0x7f00801a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f008010f660 con 0x7f0080102250 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f008010f4e0 con 0x7f0080101750 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f0084c13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f008010f7e0 con 0x7f00801050a0 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f007dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f007dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41368/0 (socket says 172.21.15.94:41368) 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f007dd9b640 1 -- 172.21.15.94:0/2079024594 learned_addr learned my addr 172.21.15.94:0/2079024594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f007e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101750 0x7f008019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.295+0000 7f007ed9d640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00801050a0 0x7f00801a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 -- 172.21.15.94:0/2079024594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00801050a0 msgr2=0x7f00801a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00801050a0 0x7f00801a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 -- 172.21.15.94:0/2079024594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101750 msgr2=0x7f008019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101750 0x7f008019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 -- 172.21.15.94:0/2079024594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00801a1d70 con 0x7f0080102250 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f007dd9b640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f007000e9f0 tx=0x7f007000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007000cd80 con 0x7f0080102250 2024-04-25T13:45:03.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00700054d0 con 0x7f0080102250 2024-04-25T13:45:03.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00801a2060 con 0x7f0080102250 2024-04-25T13:45:03.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070010690 con 0x7f0080102250 2024-04-25T13:45:03.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.296+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00801a25b0 con 0x7f0080102250 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.297+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0080108a30 con 0x7f0080102250 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.299+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0070004e30 con 0x7f0080102250 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.300+0000 7f006f7fe640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 0x7f00680b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.300+0000 7f007e59c640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 0x7f00680b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.300+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f00700df2a0 con 0x7f0080102250 2024-04-25T13:45:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.300+0000 7f007e59c640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 0x7f00680b2080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f007402aa00 tx=0x7f00740314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.304+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0070053ff0 con 0x7f0080102250 2024-04-25T13:45:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.539+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008002db50 con 0x7f0080102250 2024-04-25T13:45:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.540+0000 7f006f7fe640 1 -- 172.21.15.94:0/2079024594 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7f00700a7d80 con 0x7f0080102250 2024-04-25T13:45:03.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:45:03.540 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:45:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 msgr2=0x7f00680b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 0x7f00680b2080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f007402aa00 tx=0x7f00740314a0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 msgr2=0x7f008019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f007000e9f0 tx=0x7f007000edf0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 shutdown_connections 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00680afb70 0x7f00680b2080 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00801050a0 0x7f00801a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0080102250 0x7f008019a770 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 --2- 172.21.15.94:0/2079024594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0080101750 0x7f008019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.542+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 >> 172.21.15.94:0/2079024594 conn(0x7f00800fd370 msgr2=0x7f0080105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.543+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 shutdown_connections 2024-04-25T13:45:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.543+0000 7f0084c13640 1 -- 172.21.15.94:0/2079024594 wait complete. 2024-04-25T13:45:03.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:45:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- 172.21.15.94:0/536442778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a81029e0 msgr2=0x7fa3a81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/536442778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a81029e0 0x7fa3a81036e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- 172.21.15.94:0/536442778 shutdown_connections 2024-04-25T13:45:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/536442778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a8108920 0x7fa3a810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/536442778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8103c20 0x7fa3a8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/536442778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a81029e0 0x7fa3a81036e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- 172.21.15.94:0/536442778 >> 172.21.15.94:0/536442778 conn(0x7fa3a80fdbd0 msgr2=0x7fa3a8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- 172.21.15.94:0/536442778 shutdown_connections 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- 172.21.15.94:0/536442778 wait complete. 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 Processor -- start 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.669+0000 7fa3b04f0640 1 -- start start 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a81029e0 0x7fa3a806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8108920 0x7fa3a81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810e4e0 con 0x7fa3a8103c20 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810e360 con 0x7fa3a81029e0 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810e660 con 0x7fa3a8108920 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44730/0 (socket says 172.21.15.94:44730) 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 -- 172.21.15.94:0/1762335534 learned_addr learned my addr 172.21.15.94:0/1762335534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a81029e0 0x7fa3a806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8108920 0x7fa3a81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 -- 172.21.15.94:0/1762335534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8108920 msgr2=0x7fa3a81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8108920 0x7fa3a81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 -- 172.21.15.94:0/1762335534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a81029e0 msgr2=0x7fa3a806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a81029e0 0x7fa3a806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 -- 172.21.15.94:0/1762335534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a81a2090 con 0x7fa3a8103c20 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e9f0 tx=0x7fa3a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa39b7fe640 1 -- 172.21.15.94:0/1762335534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400cd80 con 0x7fa3a8103c20 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa39b7fe640 1 -- 172.21.15.94:0/1762335534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a40054d0 con 0x7fa3a8103c20 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a81a2380 con 0x7fa3a8103c20 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa39b7fe640 1 -- 172.21.15.94:0/1762335534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4010690 con 0x7fa3a8103c20 2024-04-25T13:45:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.670+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a81a2880 con 0x7fa3a8103c20 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.673+0000 7fa39b7fe640 1 -- 172.21.15.94:0/1762335534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa3a4004e90 con 0x7fa3a8103c20 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.673+0000 7fa3b739b740 1 Processor -- start 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 -- start start 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa39b7fe640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 0x7fa3940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd2011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 0x55a2cd201e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd160d80 con 0x55a2cce396b0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd11aa20 con 0x55a2cce314d0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3b739b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd15cd40 con 0x55a2cce5a390 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 0x7fa3940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:54780/0 (socket says 172.21.15.94:54780) 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 -- 172.21.15.94:0/673973429 learned_addr learned my addr 172.21.15.94:0/673973429 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3aea8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 0x55a2cd201e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd2011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa39b7fe640 1 -- 172.21.15.94:0/1762335534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa3a40df0d0 con 0x7fa3a8103c20 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 -- 172.21.15.94:0/673973429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 msgr2=0x55a2cd2011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd2011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 -- 172.21.15.94:0/673973429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 msgr2=0x55a2cd201e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 0x55a2cd201e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ae28c640 1 -- 172.21.15.94:0/673973429 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2cd125c30 con 0x55a2cce5a390 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd2011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.674+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 0x55a2cd201e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.675+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009230 tx=0x7fa39c031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429b6a59485993a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.675+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 0x7fa3940b2000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005fd0 tx=0x7fa3a0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.675+0000 7fa398bfe640 1 -- 172.21.15.94:0/673973429 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c035030 con 0x55a2cce5a390 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.675+0000 7fa398bfe640 1 -- 172.21.15.94:0/673973429 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c040070 con 0x55a2cce5a390 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.675+0000 7fa398bfe640 1 -- 172.21.15.94:0/673973429 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003970 con 0x55a2cce5a390 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- 172.21.15.94:0/673973429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 msgr2=0x55a2cd200c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009230 tx=0x7fa39c031b10 comp rx=0 tx=0).stop 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- 172.21.15.94:0/673973429 shutdown_connections 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce396b0 0x55a2cd201e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd2011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- 172.21.15.94:0/673973429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce5a390 0x55a2cd200c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- 172.21.15.94:0/673973429 >> 172.21.15.94:0/673973429 conn(0x55a2cce4e4e0 msgr2=0x55a2cd1f8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- 172.21.15.94:0/673973429 shutdown_connections 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- 172.21.15.94:0/673973429 wait complete. 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 -- start start 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 Processor -- start 2024-04-25T13:45:03.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.676+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce5a390 0x55a2cd211e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd2028e0 con 0x55a2cce314d0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd0e7dc0 con 0x55a2cce5a390 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2cd202a60 con 0x55a2cce396b0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 10 client.0 _ll_register_callbacks cb 0x55a2cd103600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44746/0 (socket says 172.21.15.94:44746) 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3242592404 learned_addr learned my addr 172.21.15.94:0/3242592404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce5a390 0x55a2cd211e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 msgr2=0x55a2cd20aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3242592404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce5a390 msgr2=0x55a2cd211e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce5a390 0x55a2cd211e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3242592404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2cce723f0 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa39c02a930 tx=0x7fa39c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b4257ab7423266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c035070 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa39c0036f0 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c040070 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa39c0038f0 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 10 client.16542 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.677+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2ccf3c7d0 con 0x55a2cce314d0 2024-04-25T13:45:03.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c040070 con 0x55a2cce314d0 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa39c045ac0 con 0x55a2cce314d0 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 10 client.16542 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 10 client.16542 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 20 client.16542 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce18f60 0x55a2cd218280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 4 client.16542 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2cd16e990 con 0x55a2cce18f60 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce18f60 0x55a2cd218280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa391ffb640 10 client.16542 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.678+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce18f60 0x55a2cd218280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb28a63306cf849f server_cookie=852f19b499c71fd5 in_seq=0 out_seq=0 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa391ffb640 1 -- 172.21.15.94:0/3242592404 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2cd16e990 con 0x55a2cce18f60 2024-04-25T13:45:03.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa391ffb640 10 client.16542 handle_command_reply: tid=0 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 client.16542 shutdown 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 msgr2=0x55a2cd20a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa39c02a930 tx=0x7fa39c02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 shutdown_connections 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce18f60 0x55a2cd218280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce5a390 0x55a2cd211e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa391ffb640 0 client.16542 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3242592404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce314d0 0x55a2cd20a990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 >> 172.21.15.94:0/3242592404 conn(0x55a2cce4e4e0 msgr2=0x55a2cd1f7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 shutdown_connections 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.679+0000 7fa3b739b740 1 -- 172.21.15.94:0/3242592404 wait complete. 2024-04-25T13:45:03.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.680+0000 7fa3b739b740 20 client.16542 trim_cache size 0 max 16384 2024-04-25T13:45:03.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 Processor -- start 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 -- start start 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd21ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd21d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81a92e0 con 0x55a2cce18f60 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81aae20 con 0x55a2cce314d0 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.694+0000 7fa3b739b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81a99b0 con 0x55a2cce396b0 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44758/0 (socket says 172.21.15.94:44758) 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3517482472 learned_addr learned my addr 172.21.15.94:0/3517482472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd21d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd21ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3517482472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 msgr2=0x55a2cd21d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd21d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3517482472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 msgr2=0x55a2cd21ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd21ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 -- 172.21.15.94:0/3517482472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2cd212270 con 0x55a2cce18f60 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa39c03c4e0 tx=0x7fa39c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de13e31c5f1abe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa393fff640 1 -- 172.21.15.94:0/3517482472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c05b020 con 0x55a2cce18f60 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa393fff640 1 -- 172.21.15.94:0/3517482472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c040040 con 0x55a2cce18f60 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.695+0000 7fa393fff640 1 -- 172.21.15.94:0/3517482472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c035040 con 0x55a2cce18f60 2024-04-25T13:45:03.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- 172.21.15.94:0/3517482472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 msgr2=0x55a2cd21c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa39c03c4e0 tx=0x7fa39c005fd0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- 172.21.15.94:0/3517482472 shutdown_connections 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd21d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd21ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 --2- 172.21.15.94:0/3517482472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd21c910 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- 172.21.15.94:0/3517482472 >> 172.21.15.94:0/3517482472 conn(0x55a2cce4e4e0 msgr2=0x55a2cd1fb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- 172.21.15.94:0/3517482472 shutdown_connections 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- 172.21.15.94:0/3517482472 wait complete. 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 -- start start 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.696+0000 7fa3b739b740 1 Processor -- start 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd20edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81aa750 con 0x55a2cce18f60 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81aa080 con 0x55a2cce314d0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81a72c0 con 0x55a2cce396b0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3b739b740 10 client.0 _ll_register_callbacks cb 0x7fa3a0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44772/0 (socket says 172.21.15.94:44772) 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 -- 172.21.15.94:0/2550423308 learned_addr learned my addr 172.21.15.94:0/2550423308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd20edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3aea8d640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 -- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 msgr2=0x55a2cd20fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 -- 172.21.15.94:0/2550423308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 msgr2=0x55a2cd20edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd20edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 -- 172.21.15.94:0/2550423308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2cd2102a0 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa3ae28c640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c0314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8465cb6adc869fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.697+0000 7fa391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c05b030 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa39c045470 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c040040 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa39c06f050 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 10 client.16548 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2cd211520 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c040040 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa39c03c810 con 0x55a2cce18f60 2024-04-25T13:45:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 10 client.16548 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 10 client.16548 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 20 client.16548 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce5a390 0x55a2cd282a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 4 client.16548 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2cd168800 con 0x55a2cce5a390 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.698+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce5a390 0x55a2cd282a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3ada8b640 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce5a390 0x55a2cd282a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=351128571e9ae7f server_cookie=915edd6e10bdac74 in_seq=0 out_seq=0 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa391ffb640 10 client.16548 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa391ffb640 1 -- 172.21.15.94:0/2550423308 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2cd168800 con 0x55a2cce5a390 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa391ffb640 10 client.16548 handle_command_reply: tid=0 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 client.16548 shutdown 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 msgr2=0x55a2cd20e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c0314c0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 shutdown_connections 2024-04-25T13:45:03.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a2cce5a390 0x55a2cd282a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2cce396b0 0x55a2cd20fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2cce314d0 0x55a2cd20edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 --2- 172.21.15.94:0/2550423308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2cce18f60 0x55a2cd20e870 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa391ffb640 0 client.16548 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 >> 172.21.15.94:0/2550423308 conn(0x55a2cce4e4e0 msgr2=0x55a2cd1fb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 shutdown_connections 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 1 -- 172.21.15.94:0/2550423308 wait complete. 2024-04-25T13:45:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.699+0000 7fa3b739b740 20 client.16548 trim_cache size 0 max 16384 2024-04-25T13:45:03.711 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 msgr2=0x7fa3940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 0x7fa3940b2000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005fd0 tx=0x7fa3a0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 msgr2=0x7fa3a819f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e9f0 tx=0x7fa3a400edf0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 shutdown_connections 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3940afaf0 0x7fa3940b2000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a8108920 0x7fa3a81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a8103c20 0x7fa3a819f320 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 --2- 172.21.15.94:0/1762335534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a81029e0 0x7fa3a806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 >> 172.21.15.94:0/1762335534 conn(0x7fa3a80fdbd0 msgr2=0x7fa3a8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 shutdown_connections 2024-04-25T13:45:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.700+0000 7fa3b04f0640 1 -- 172.21.15.94:0/1762335534 wait complete. 2024-04-25T13:45:03.714 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:45:03.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 274372 14.11 MB/sec warmup 106 sec latency 12.914 ms 2024-04-25T13:45:03.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 -- 172.21.15.94:0/2765375273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 msgr2=0x7f2e1c1073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/2765375273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c1073a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009900 tx=0x7f2e1002a410 comp rx=0 tx=0).stop 2024-04-25T13:45:03.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 -- 172.21.15.94:0/2765375273 shutdown_connections 2024-04-25T13:45:03.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/2765375273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/2765375273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c1073a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/2765375273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 -- 172.21.15.94:0/2765375273 >> 172.21.15.94:0/2765375273 conn(0x7f2e1c100f40 msgr2=0x7f2e1c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 -- 172.21.15.94:0/2765375273 shutdown_connections 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 -- 172.21.15.94:0/2765375273 wait complete. 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.827+0000 7f2e21fd7640 1 Processor -- start 2024-04-25T13:45:03.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 -- start start 2024-04-25T13:45:03.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c1a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1117e0 con 0x7f2e1c106f90 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111660 con 0x7f2e1c10bc90 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e21fd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c111960 con 0x7f2e1c105d50 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44800/0 (socket says 172.21.15.94:44800) 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 -- 172.21.15.94:0/4034106024 learned_addr learned my addr 172.21.15.94:0/4034106024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c1a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 -- 172.21.15.94:0/4034106024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 msgr2=0x7f2e1c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 -- 172.21.15.94:0/4034106024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 msgr2=0x7f2e1c1a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c1a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 -- 172.21.15.94:0/4034106024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c1a3f30 con 0x7f2e1c106f90 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c1a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.828+0000 7f2e1affd640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2e10006010 tx=0x7f2e10031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.829+0000 7f2e18ff9640 1 -- 172.21.15.94:0/4034106024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1003f030 con 0x7f2e1c106f90 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.829+0000 7f2e18ff9640 1 -- 172.21.15.94:0/4034106024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e10003aa0 con 0x7f2e1c106f90 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.829+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c1a41c0 con 0x7f2e1c106f90 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.829+0000 7f2e18ff9640 1 -- 172.21.15.94:0/4034106024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10035070 con 0x7f2e1c106f90 2024-04-25T13:45:03.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.829+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1a46b0 con 0x7f2e1c106f90 2024-04-25T13:45:03.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e18ff9640 1 -- 172.21.15.94:0/4034106024 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2e10042030 con 0x7f2e1c106f90 2024-04-25T13:45:03.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 Processor -- start 2024-04-25T13:45:03.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 -- start start 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f43aad3d80 con 0x55f43a7ac6b0 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f43aa8da20 con 0x55f43a7cd390 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.831+0000 7f2e28f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f43aacfd40 con 0x55f43a7a44d0 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44814/0 (socket says 172.21.15.94:44814) 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 -- 172.21.15.94:0/1697820073 learned_addr learned my addr 172.21.15.94:0/1697820073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e18ff9640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 0x7f2e040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 0x7f2e040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 -- 172.21.15.94:0/1697820073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 msgr2=0x55f43ab6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 -- 172.21.15.94:0/1697820073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 msgr2=0x55f43ab6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 -- 172.21.15.94:0/1697820073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43aa98c30 con 0x55f43a7ac6b0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e18ff9640 1 -- 172.21.15.94:0/4034106024 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2e100ff230 con 0x7f2e1c106f90 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1affd640 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c37a0 tx=0x7f2e10082540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a25f4895b1c5ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 0x7f2e040b2000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f43ab70860 tx=0x7f2e14004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e01ffb640 1 -- 172.21.15.94:0/1697820073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10049070 con 0x55f43a7ac6b0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e01ffb640 1 -- 172.21.15.94:0/1697820073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e10038e80 con 0x55f43a7ac6b0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.832+0000 7f2e01ffb640 1 -- 172.21.15.94:0/1697820073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100d04d0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 -- 172.21.15.94:0/1697820073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 msgr2=0x55f43ab70a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c37a0 tx=0x7f2e10082540 comp rx=0 tx=0).stop 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 -- 172.21.15.94:0/1697820073 shutdown_connections 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab70a20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 --2- 172.21.15.94:0/1697820073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 -- 172.21.15.94:0/1697820073 >> 172.21.15.94:0/1697820073 conn(0x55f43a7c14e0 msgr2=0x55f43ab6b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 -- 172.21.15.94:0/1697820073 shutdown_connections 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.833+0000 7f2e28f65740 1 -- 172.21.15.94:0/1697820073 wait complete. 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 -- start start 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 Processor -- start 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab80670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f43ab712f0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f43aa5adc0 con 0x55f43a7a44d0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f43ab71470 con 0x55f43a7cd390 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e28f65740 10 client.0 _ll_register_callbacks cb 0x55f43aa76600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44818/0 (socket says 172.21.15.94:44818) 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e1affd640 1 -- 172.21.15.94:0/3816398835 learned_addr learned my addr 172.21.15.94:0/3816398835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.834+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab80670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 -- 172.21.15.94:0/3816398835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 msgr2=0x55f43ab80670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab80670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 -- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 msgr2=0x55f43ab791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 -- 172.21.15.94:0/3816398835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43a7e53f0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e1affd640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2e100fcce0 tx=0x7f2e1002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d671dfc2427f4d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10049040 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2e100b00c0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f43aae1990 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e10038ba0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 10 client.16560 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f43a8af7d0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f43aae1990 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2e100c7980 con 0x55f43a7ac6b0 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 10 client.16560 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 10 client.16560 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:03.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.835+0000 7f2e28f65740 20 client.16560 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a78bf60 0x55f43ab86c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2e28f65740 4 client.16560 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43aadb800 con 0x55f43a78bf60 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a78bf60 0x55f43ab86c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2dfaffd640 10 client.16560 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.836+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a78bf60 0x55f43ab86c60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ee7788e01d42c server_cookie=c55b77f35ad5602b in_seq=0 out_seq=0 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2dfaffd640 1 -- 172.21.15.94:0/3816398835 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f43aadb800 con 0x55f43a78bf60 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2dfaffd640 10 client.16560 handle_command_reply: tid=0 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 client.16560 shutdown 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 msgr2=0x55f43ab79710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2e100fcce0 tx=0x7f2e1002a410 comp rx=0 tx=0).stop 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 shutdown_connections 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a78bf60 0x55f43ab86c60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab80670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2dfaffd640 0 client.16560 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab79710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 --2- 172.21.15.94:0/3816398835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 >> 172.21.15.94:0/3816398835 conn(0x55f43a7c14e0 msgr2=0x55f43ab6df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 shutdown_connections 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 1 -- 172.21.15.94:0/3816398835 wait complete. 2024-04-25T13:45:03.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.837+0000 7f2e28f65740 20 client.16560 trim_cache size 0 max 16384 2024-04-25T13:45:03.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 Processor -- start 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 -- start start 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1ad660 con 0x55f43a7ac6b0 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1af1a0 con 0x55f43a7cd390 2024-04-25T13:45:03.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e28f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1add30 con 0x55f43a7a44d0 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44834/0 (socket says 172.21.15.94:44834) 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 -- 172.21.15.94:0/2743755219 learned_addr learned my addr 172.21.15.94:0/2743755219 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 -- 172.21.15.94:0/2743755219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 msgr2=0x55f43ab6aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 -- 172.21.15.94:0/2743755219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 msgr2=0x55f43ab6af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 -- 172.21.15.94:0/2743755219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43ab74ae0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.851+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c37a0 tx=0x7f2e1003b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491f78af3ddbc130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e017fa640 1 -- 172.21.15.94:0/2743755219 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10049040 con 0x55f43a7ac6b0 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e017fa640 1 -- 172.21.15.94:0/2743755219 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e100c7230 con 0x55f43a7ac6b0 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e017fa640 1 -- 172.21.15.94:0/2743755219 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100c7980 con 0x55f43a7ac6b0 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 -- 172.21.15.94:0/2743755219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 msgr2=0x55f43ab80590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2e100c37a0 tx=0x7f2e1003b5d0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 -- 172.21.15.94:0/2743755219 shutdown_connections 2024-04-25T13:45:03.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7cd390 0x55f43ab6af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab80590 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2743755219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7a44d0 0x55f43ab6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 -- 172.21.15.94:0/2743755219 >> 172.21.15.94:0/2743755219 conn(0x55f43a7c14e0 msgr2=0x55f43ab777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 -- 172.21.15.94:0/2743755219 shutdown_connections 2024-04-25T13:45:03.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.852+0000 7f2e28f65740 1 -- 172.21.15.94:0/2743755219 wait complete. 2024-04-25T13:45:03.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 -- start start 2024-04-25T13:45:03.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 Processor -- start 2024-04-25T13:45:03.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1aead0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1ae400 con 0x55f43a7a44d0 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1ab640 con 0x55f43a7cd390 2024-04-25T13:45:03.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e28f65740 10 client.0 _ll_register_callbacks cb 0x7f2e14009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44848/0 (socket says 172.21.15.94:44848) 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e1affd640 1 -- 172.21.15.94:0/2120772598 learned_addr learned my addr 172.21.15.94:0/2120772598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.853+0000 7f2e1bfff640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 -- 172.21.15.94:0/2120772598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 msgr2=0x55f43ab7ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab7ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 -- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 msgr2=0x55f43ab7d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab7d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 -- 172.21.15.94:0/2120772598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43ab7f3c0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab7d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e1affd640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2e10038020 tx=0x7f2e100c7d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c670f5b22a64777e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2dfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10049070 con 0x55f43a7ac6b0 2024-04-25T13:45:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f2e100aebc0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e100c7230 con 0x55f43a7ac6b0 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e100d04d0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e28f65740 10 client.16566 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f43ab7f3c0 con 0x55f43a7ac6b0 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.854+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e100c7230 con 0x55f43a7ac6b0 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2e100c6980 con 0x55f43a7ac6b0 2024-04-25T13:45:03.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 10 client.16566 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 10 client.16566 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 20 client.16566 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a7613a0 0x55f43abf1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 4 client.16566 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f43aa56610 con 0x55f43a7613a0 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a7613a0 0x55f43abf1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2dfaffd640 10 client.16566 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e1b7fe640 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a7613a0 0x55f43abf1030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c0a84c89d01431 server_cookie=d68b1e7334cf5472 in_seq=0 out_seq=0 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2dfaffd640 1 -- 172.21.15.94:0/2120772598 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f43aa56610 con 0x55f43a7613a0 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2dfaffd640 10 client.16566 handle_command_reply: tid=0 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.855+0000 7f2e28f65740 1 client.16566 shutdown 2024-04-25T13:45:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 msgr2=0x55f43ab7ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2e10038020 tx=0x7f2e100c7d20 comp rx=0 tx=0).stop 2024-04-25T13:45:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 shutdown_connections 2024-04-25T13:45:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f43a7613a0 0x55f43abf1030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f43a7cd390 0x55f43ab7ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f43a7ac6b0 0x55f43ab7ded0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 --2- 172.21.15.94:0/2120772598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f43a7a44d0 0x55f43ab7d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 >> 172.21.15.94:0/2120772598 conn(0x55f43a7c14e0 msgr2=0x55f43ab6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2dfaffd640 0 client.16566 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 shutdown_connections 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 1 -- 172.21.15.94:0/2120772598 wait complete. 2024-04-25T13:45:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e28f65740 20 client.16566 trim_cache size 0 max 16384 2024-04-25T13:45:03.879 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 msgr2=0x7f2e040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 0x7f2e040b2000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f43ab70860 tx=0x7f2e14004ab0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 msgr2=0x7f2e1c19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2e10006010 tx=0x7f2e10031a50 comp rx=0 tx=0).stop 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 shutdown_connections 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e040afaf0 0x7f2e040b2000 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e1c10bc90 0x7f2e1c1a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e1c106f90 0x7f2e1c19c930 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.856+0000 7f2e21fd7640 1 --2- 172.21.15.94:0/4034106024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e1c105d50 0x7f2e1c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.857+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 >> 172.21.15.94:0/4034106024 conn(0x7f2e1c100f40 msgr2=0x7f2e1c107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.857+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 shutdown_connections 2024-04-25T13:45:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.857+0000 7f2e21fd7640 1 -- 172.21.15.94:0/4034106024 wait complete. 2024-04-25T13:45:03.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:45:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1445082037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 msgr2=0x7fb5ec101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1445082037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec101bc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009900 tx=0x7fb5e002a410 comp rx=0 tx=0).stop 2024-04-25T13:45:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1445082037 shutdown_connections 2024-04-25T13:45:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1445082037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 0x7fb5ec10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1445082037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec101bc0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1445082037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 0x7fb5ec1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1445082037 >> 172.21.15.94:0/1445082037 conn(0x7fb5ec0fd350 msgr2=0x7fb5ec0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1445082037 shutdown_connections 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.998+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1445082037 wait complete. 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 Processor -- start 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 -- start start 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 0x7fb5ec19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 0x7fb5ec1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10f630 con 0x7fb5ec101750 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10f4b0 con 0x7fb5ec1050a0 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5f1a67640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10f7b0 con 0x7fb5ec102250 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44866/0 (socket says 172.21.15.94:44866) 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5eaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 0x7fb5ec19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5eb7fe640 1 -- 172.21.15.94:0/1785761718 learned_addr learned my addr 172.21.15.94:0/1785761718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:03.999+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 0x7fb5ec1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 -- 172.21.15.94:0/1785761718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 msgr2=0x7fb5ec19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 0x7fb5ec19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 -- 172.21.15.94:0/1785761718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 msgr2=0x7fb5ec1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 0x7fb5ec1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 -- 172.21.15.94:0/1785761718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1a1d40 con 0x7fb5ec101750 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5e8ff9640 1 -- 172.21.15.94:0/1785761718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00ccc0 con 0x7fb5ec101750 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5e8ff9640 1 -- 172.21.15.94:0/1785761718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc0035f0 con 0x7fb5ec101750 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1a1fd0 con 0x7fb5ec101750 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5e8ff9640 1 -- 172.21.15.94:0/1785761718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00ce60 con 0x7fb5ec101750 2024-04-25T13:45:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.000+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1a2490 con 0x7fb5ec101750 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.002+0000 7fb5e8ff9640 1 -- 172.21.15.94:0/1785761718 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb5dc003aa0 con 0x7fb5ec101750 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.002+0000 7fb5f89fc740 1 Processor -- start 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 -- start start 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac92d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac92ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac891d80 con 0x55e1ac58b390 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac84ba20 con 0x55e1ac5624d0 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac88dd40 con 0x55e1ac56a6b0 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac92d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac92d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44882/0 (socket says 172.21.15.94:44882) 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5ebfff640 1 -- 172.21.15.94:0/3271115957 learned_addr learned my addr 172.21.15.94:0/3271115957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac92ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5e8ff9640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 0x7fb5d40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3271115957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 msgr2=0x55e1ac92dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3271115957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 msgr2=0x55e1ac92d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac92d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3271115957 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ac856c30 con 0x55e1ac56a6b0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 0x7fb5d40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5e8ff9640 1 -- 172.21.15.94:0/1785761718 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb5dc0def70 con 0x7fb5ec101750 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.003+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac92ea20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005ab0 tx=0x7fb5e0009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb353fee5d48206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.004+0000 7fb5d1ffb640 1 -- 172.21.15.94:0/3271115957 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0035030 con 0x55e1ac56a6b0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.004+0000 7fb5d1ffb640 1 -- 172.21.15.94:0/3271115957 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0040070 con 0x55e1ac56a6b0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.004+0000 7fb5d1ffb640 1 -- 172.21.15.94:0/3271115957 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00037f0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3271115957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 msgr2=0x55e1ac92ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 0x7fb5d40b2000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009fd0 tx=0x7fb5e4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac92ea20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005ab0 tx=0x7fb5e0009600 comp rx=0 tx=0).stop 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3271115957 shutdown_connections 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac92ea20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3271115957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac92d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3271115957 >> 172.21.15.94:0/3271115957 conn(0x55e1ac57f4e0 msgr2=0x55e1ac929020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3271115957 shutdown_connections 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3271115957 wait complete. 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- start start 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 Processor -- start 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac937560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac93ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac92f4d0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac818dc0 con 0x55e1ac5624d0 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.005+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1ac92f650 con 0x55e1ac58b390 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5f89fc740 10 client.0 _ll_register_callbacks cb 0x55e1ac834600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44894/0 (socket says 172.21.15.94:44894) 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3299854089 learned_addr learned my addr 172.21.15.94:0/3299854089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac937560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac93ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 msgr2=0x55e1ac93ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac93ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3299854089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 msgr2=0x55e1ac937560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac937560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 -- 172.21.15.94:0/3299854089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ac5a33f0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac937560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0031f00 tx=0x7fb5e0031f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e9c95fedc5fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0035040 con 0x55e1ac56a6b0 2024-04-25T13:45:04.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb5e00315a0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0040070 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.006+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5e00317a0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 10 client.16578 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1ac66d7d0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0040070 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb5e0045470 con 0x55e1ac56a6b0 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 10 client.16578 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 10 client.16578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 20 client.16578 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac549f60 0x55e1ac944e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 4 client.16578 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:04.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1ac89f990 con 0x55e1ac549f60 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.007+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac549f60 0x55e1ac944e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac549f60 0x55e1ac944e50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a96f46c26b62b6b6 server_cookie=f975a570cfae77af in_seq=0 out_seq=0 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5caffd640 10 client.16578 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5caffd640 1 -- 172.21.15.94:0/3299854089 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e1ac89f990 con 0x55e1ac549f60 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5caffd640 10 client.16578 handle_command_reply: tid=0 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 client.16578 shutdown 2024-04-25T13:45:04.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 msgr2=0x55e1ac937aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0031f00 tx=0x7fb5e0031f30 comp rx=0 tx=0).stop 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 shutdown_connections 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac549f60 0x55e1ac944e50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac93ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac937aa0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/3299854089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac937560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5caffd640 0 client.16578 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.008+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 >> 172.21.15.94:0/3299854089 conn(0x55e1ac57f4e0 msgr2=0x55e1ac92bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.009+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 shutdown_connections 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.009+0000 7fb5f89fc740 1 -- 172.21.15.94:0/3299854089 wait complete. 2024-04-25T13:45:04.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.009+0000 7fb5f89fc740 20 client.16578 trim_cache size 0 max 16384 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.022+0000 7fb5f89fc740 1 Processor -- start 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.022+0000 7fb5f89fc740 1 -- start start 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ab540 con 0x55e1ac58b390 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ad080 con 0x55e1ac5624d0 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1abc10 con 0x55e1ac56a6b0 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44908/0 (socket says 172.21.15.94:44908) 2024-04-25T13:45:04.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1774263011 learned_addr learned my addr 172.21.15.94:0/1774263011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1774263011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 msgr2=0x55e1ac93e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1774263011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 msgr2=0x55e1ac92c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1774263011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ac932ce0 con 0x55e1ac58b390 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4020710 tx=0x7fb5e4013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d7e0d508a6fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.023+0000 7fb5d17fa640 1 -- 172.21.15.94:0/1774263011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4004d20 con 0x55e1ac58b390 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5d17fa640 1 -- 172.21.15.94:0/1774263011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e401eae0 con 0x55e1ac58b390 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5d17fa640 1 -- 172.21.15.94:0/1774263011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40056c0 con 0x55e1ac58b390 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1774263011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 msgr2=0x55e1ac94a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4020710 tx=0x7fb5e4013030 comp rx=0 tx=0).stop 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1774263011 shutdown_connections 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac58b390 0x55e1ac94a4e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1774263011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac92c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1774263011 >> 172.21.15.94:0/1774263011 conn(0x55e1ac57f4e0 msgr2=0x55e1ac92be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1774263011 shutdown_connections 2024-04-25T13:45:04.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.024+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1774263011 wait complete. 2024-04-25T13:45:04.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 -- start start 2024-04-25T13:45:04.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 Processor -- start 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac93dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac9ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ac9b0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ac2e0 con 0x55e1ac5624d0 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1a9520 con 0x55e1ac58b390 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5f89fc740 10 client.0 _ll_register_callbacks cb 0x7fb5e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44916/0 (socket says 172.21.15.94:44916) 2024-04-25T13:45:04.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.025+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1709713803 learned_addr learned my addr 172.21.15.94:0/1709713803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac93dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac9ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 msgr2=0x55e1ac9ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac9ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1709713803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 msgr2=0x55e1ac93dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac93dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1709713803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ac93ee60 con 0x55e1ac56a6b0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5eaffd640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0045a30 tx=0x7fb5e0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40625c86f1c8db04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5caffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e005c030 con 0x55e1ac56a6b0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb5e004b070 con 0x55e1ac56a6b0 2024-04-25T13:45:04.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e401eae0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5e0066050 con 0x55e1ac56a6b0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5f89fc740 10 client.16584 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1ac9ac950 con 0x55e1ac56a6b0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.026+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e401eae0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb5e0003ba0 con 0x55e1ac56a6b0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 10 client.16584 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 10 client.16584 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 20 client.16584 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac51f3a0 0x55e1ac9af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 4 client.16584 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1ac899800 con 0x55e1ac51f3a0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac51f3a0 0x55e1ac9af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5caffd640 10 client.16584 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5eb7fe640 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac51f3a0 0x55e1ac9af390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afbc838c43d1ad91 server_cookie=7418929922206a8e in_seq=0 out_seq=0 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5caffd640 1 -- 172.21.15.94:0/1709713803 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1ac899800 con 0x55e1ac51f3a0 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5caffd640 10 client.16584 handle_command_reply: tid=0 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.027+0000 7fb5f89fc740 1 client.16584 shutdown 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 msgr2=0x55e1ac93e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0045a30 tx=0x7fb5e0009590 comp rx=0 tx=0).stop 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 shutdown_connections 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1ac51f3a0 0x55e1ac9af390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1ac58b390 0x55e1ac9ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1ac56a6b0 0x55e1ac93e190 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5caffd640 0 client.16584 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 --2- 172.21.15.94:0/1709713803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1ac5624d0 0x55e1ac93dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 >> 172.21.15.94:0/1709713803 conn(0x55e1ac57f4e0 msgr2=0x55e1ac92be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 shutdown_connections 2024-04-25T13:45:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 1 -- 172.21.15.94:0/1709713803 wait complete. 2024-04-25T13:45:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f89fc740 20 client.16584 trim_cache size 0 max 16384 2024-04-25T13:45:04.036 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 msgr2=0x7fb5d40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 0x7fb5d40b2000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009fd0 tx=0x7fb5e4004a40 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 msgr2=0x7fb5ec19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 shutdown_connections 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d40afaf0 0x7fb5d40b2000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec1050a0 0x7fb5ec1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec102250 0x7fb5ec19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 --2- 172.21.15.94:0/1785761718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec101750 0x7fb5ec19a200 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 >> 172.21.15.94:0/1785761718 conn(0x7fb5ec0fd350 msgr2=0x7fb5ec105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 shutdown_connections 2024-04-25T13:45:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.028+0000 7fb5f1a67640 1 -- 172.21.15.94:0/1785761718 wait complete. 2024-04-25T13:45:04.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 -- 172.21.15.94:0/51452574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac801029e0 msgr2=0x7fac801036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 --2- 172.21.15.94:0/51452574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac801029e0 0x7fac801036e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fac74009900 tx=0x7fac7402a410 comp rx=0 tx=0).stop 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 -- 172.21.15.94:0/51452574 shutdown_connections 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 --2- 172.21.15.94:0/51452574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 0x7fac8010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 --2- 172.21.15.94:0/51452574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac80103c20 0x7fac80104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 --2- 172.21.15.94:0/51452574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac801029e0 0x7fac801036e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 -- 172.21.15.94:0/51452574 >> 172.21.15.94:0/51452574 conn(0x7fac800fdbd0 msgr2=0x7fac80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 -- 172.21.15.94:0/51452574 shutdown_connections 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.158+0000 7fac86b64640 1 -- 172.21.15.94:0/51452574 wait complete. 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 Processor -- start 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 -- start start 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 0x7fac8019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 0x7fac801a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac8010e4e0 con 0x7fac80103c20 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac8010e360 con 0x7fac80108920 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac86b64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac8010e660 con 0x7fac801029e0 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac84900640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 0x7fac8019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44940/0 (socket says 172.21.15.94:44940) 2024-04-25T13:45:04.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac7ffff640 1 -- 172.21.15.94:0/4293739941 learned_addr learned my addr 172.21.15.94:0/4293739941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.159+0000 7fac85101640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 0x7fac801a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 -- 172.21.15.94:0/4293739941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 msgr2=0x7fac8019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 0x7fac8019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 -- 172.21.15.94:0/4293739941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 msgr2=0x7fac801a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 0x7fac801a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 -- 172.21.15.94:0/4293739941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801a20e0 con 0x7fac80103c20 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac84900640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 0x7fac8019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7ffff640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00e9f0 tx=0x7fac6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7dffb640 1 -- 172.21.15.94:0/4293739941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00cd80 con 0x7fac80103c20 2024-04-25T13:45:04.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801a23d0 con 0x7fac80103c20 2024-04-25T13:45:04.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7dffb640 1 -- 172.21.15.94:0/4293739941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6c0054d0 con 0x7fac80103c20 2024-04-25T13:45:04.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac7dffb640 1 -- 172.21.15.94:0/4293739941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c010690 con 0x7fac80103c20 2024-04-25T13:45:04.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.160+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a2920 con 0x7fac80103c20 2024-04-25T13:45:04.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac7dffb640 1 -- 172.21.15.94:0/4293739941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fac6c004e90 con 0x7fac80103c20 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 Processor -- start 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 -- start start 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 0x557b3f487810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0c46b0 0x557b3f488a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b3f3ebd80 con 0x557b3f0bc4d0 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b3f3a5a20 con 0x557b3f0e5390 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac8da12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b3f3e7d40 con 0x557b3f0c46b0 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac84900640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac85101640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 0x557b3f487810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac84900640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44944/0 (socket says 172.21.15.94:44944) 2024-04-25T13:45:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac84900640 1 -- 172.21.15.94:0/1101703171 learned_addr learned my addr 172.21.15.94:0/1101703171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0c46b0 0x557b3f488a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac7dffb640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 0x7fac700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac85101640 1 -- 172.21.15.94:0/1101703171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0c46b0 msgr2=0x557b3f488a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.163+0000 7fac85101640 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0c46b0 0x557b3f488a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 -- 172.21.15.94:0/1101703171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 msgr2=0x557b3f487d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 -- 172.21.15.94:0/1101703171 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3f3b0c30 con 0x557b3f0e5390 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac84900640 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 0x7fac700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac7dffb640 1 -- 172.21.15.94:0/4293739941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fac6c0df0d0 con 0x7fac80103c20 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 0x557b3f487810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fd0 tx=0x7fac78011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee0bec66e971a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac85101640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 0x7fac700b2000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac78022000 tx=0x7fac78023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac6affd640 1 -- 172.21.15.94:0/1101703171 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78017070 con 0x557b3f0e5390 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac6affd640 1 -- 172.21.15.94:0/1101703171 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7800fcb0 con 0x557b3f0e5390 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.164+0000 7fac6affd640 1 -- 172.21.15.94:0/1101703171 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800eab0 con 0x557b3f0e5390 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- 172.21.15.94:0/1101703171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 msgr2=0x557b3f487810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 0x557b3f487810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fd0 tx=0x7fac78011920 comp rx=0 tx=0).stop 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- 172.21.15.94:0/1101703171 shutdown_connections 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0c46b0 0x557b3f488a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f487d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 --2- 172.21.15.94:0/1101703171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0e5390 0x557b3f487810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- 172.21.15.94:0/1101703171 >> 172.21.15.94:0/1101703171 conn(0x557b3f0d94e0 msgr2=0x557b3f483020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- 172.21.15.94:0/1101703171 shutdown_connections 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- 172.21.15.94:0/1101703171 wait complete. 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 -- start start 2024-04-25T13:45:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.165+0000 7fac8da12740 1 Processor -- start 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f491aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0e5390 0x557b3f498a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b3f489530 con 0x557b3f0bc4d0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b3f372dc0 con 0x557b3f0c46b0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b3f4896b0 con 0x557b3f0e5390 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac8da12740 10 client.0 _ll_register_callbacks cb 0x557b3f38e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44954/0 (socket says 172.21.15.94:44954) 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 -- 172.21.15.94:0/1950495568 learned_addr learned my addr 172.21.15.94:0/1950495568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac85101640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0e5390 0x557b3f498a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f491aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 -- 172.21.15.94:0/1950495568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0e5390 msgr2=0x557b3f498a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0e5390 0x557b3f498a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 -- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 msgr2=0x557b3f491aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f491aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 -- 172.21.15.94:0/1950495568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3f0fd3f0 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac84900640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fac7402a930 tx=0x7fac740317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bc90d0129e9ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.166+0000 7fac5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74035040 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fac74004450 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7800fcb0 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac74040850 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 10 client.16596 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b3f1c77d0 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7800fcb0 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fac74048070 con 0x557b3f0bc4d0 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 10 client.16596 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 10 client.16596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:04.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 20 client.16596 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0a3f60 0x557b3f49f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 4 client.16596 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3f3f9990 con 0x557b3f0a3f60 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.167+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0a3f60 0x557b3f49f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.168+0000 7fac5ffff640 10 client.16596 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.168+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0a3f60 0x557b3f49f1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=396b0d9260e4a20b server_cookie=3114880231890c38 in_seq=0 out_seq=0 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.168+0000 7fac5ffff640 1 -- 172.21.15.94:0/1950495568 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b3f3f9990 con 0x557b3f0a3f60 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.168+0000 7fac5ffff640 10 client.16596 handle_command_reply: tid=0 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 client.16596 shutdown 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 msgr2=0x557b3f491560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fac7402a930 tx=0x7fac740317d0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 shutdown_connections 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0a3f60 0x557b3f49f1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac5ffff640 0 client.16596 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0e5390 0x557b3f498a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f491aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 --2- 172.21.15.94:0/1950495568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f491560 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 >> 172.21.15.94:0/1950495568 conn(0x557b3f0d94e0 msgr2=0x557b3f482040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 shutdown_connections 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 1 -- 172.21.15.94:0/1950495568 wait complete. 2024-04-25T13:45:04.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.169+0000 7fac8da12740 20 client.16596 trim_cache size 0 max 16384 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 Processor -- start 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 -- start start 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f491800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0bc4d0 0x557b3f498920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac801a92e0 con 0x557b3f0c46b0 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac801aae20 con 0x557b3f0bc4d0 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac8da12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac801a99b0 con 0x557b3f0a3f60 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44962/0 (socket says 172.21.15.94:44962) 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 -- 172.21.15.94:0/3933778535 learned_addr learned my addr 172.21.15.94:0/3933778535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac7ffff640 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0bc4d0 0x557b3f498920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac84900640 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f491800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 -- 172.21.15.94:0/3933778535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 msgr2=0x557b3f491800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f491800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 -- 172.21.15.94:0/3933778535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0bc4d0 msgr2=0x557b3f498920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0bc4d0 0x557b3f498920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.183+0000 7fac85101640 1 -- 172.21.15.94:0/3933778535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3f498e60 con 0x557b3f0c46b0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.184+0000 7fac85101640 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac78009730 tx=0x7fac780096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db7214eccfd7b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.184+0000 7fac6a7fc640 1 -- 172.21.15.94:0/3933778535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78037070 con 0x557b3f0c46b0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.184+0000 7fac6a7fc640 1 -- 172.21.15.94:0/3933778535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7800f540 con 0x557b3f0c46b0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.184+0000 7fac6a7fc640 1 -- 172.21.15.94:0/3933778535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78017040 con 0x557b3f0c46b0 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- 172.21.15.94:0/3933778535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 msgr2=0x557b3f4a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac78009730 tx=0x7fac780096c0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- 172.21.15.94:0/3933778535 shutdown_connections 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0c46b0 0x557b3f4a3980 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0bc4d0 0x557b3f498920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- 172.21.15.94:0/3933778535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f491800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- 172.21.15.94:0/3933778535 >> 172.21.15.94:0/3933778535 conn(0x557b3f0d94e0 msgr2=0x557b3f486850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- 172.21.15.94:0/3933778535 shutdown_connections 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- 172.21.15.94:0/3933778535 wait complete. 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 -- start start 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 Processor -- start 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f495f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.185+0000 7fac8da12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f497360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac801aa750 con 0x557b3f0bc4d0 2024-04-25T13:45:04.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac801aa080 con 0x557b3f0c46b0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac801a72c0 con 0x557b3f0a3f60 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 10 client.0 _ll_register_callbacks cb 0x7fac74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44978/0 (socket says 172.21.15.94:44978) 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 -- 172.21.15.94:0/1823103796 learned_addr learned my addr 172.21.15.94:0/1823103796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac85101640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f497360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac84900640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f495f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 -- 172.21.15.94:0/1823103796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 msgr2=0x557b3f495f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f495f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 -- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 msgr2=0x557b3f497360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f497360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 -- 172.21.15.94:0/1823103796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3f497920 con 0x557b3f0bc4d0 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac85101640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f497360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac7ffff640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0a3590 tx=0x7fac6c0a3c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458625a6be0c638a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c005180 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fac6c0afee0 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7800f540 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac6c0a7020 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 10 client.16602 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.186+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557b3f498c70 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7800f540 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fac6c0c8300 con 0x557b3f0bc4d0 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 10 client.16602 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 10 client.16602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 20 client.16602 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0e5390 0x557b3f509db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 4 client.16602 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b3f3f3800 con 0x557b3f0e5390 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac84900640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0e5390 0x557b3f509db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.187+0000 7fac5ffff640 10 client.16602 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac84900640 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0e5390 0x557b3f509db0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2728de713bced98 server_cookie=2504222dd0afd2eb in_seq=0 out_seq=0 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac5ffff640 1 -- 172.21.15.94:0/1823103796 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b3f3f3800 con 0x557b3f0e5390 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac5ffff640 10 client.16602 handle_command_reply: tid=0 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 client.16602 shutdown 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 msgr2=0x557b3f4964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0a3590 tx=0x7fac6c0a3c90 comp rx=0 tx=0).stop 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 shutdown_connections 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x557b3f0e5390 0x557b3f509db0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b3f0c46b0 0x557b3f497360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b3f0bc4d0 0x557b3f4964a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac5ffff640 0 client.16602 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 --2- 172.21.15.94:0/1823103796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b3f0a3f60 0x557b3f495f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 >> 172.21.15.94:0/1823103796 conn(0x557b3f0d94e0 msgr2=0x557b3f4865b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 shutdown_connections 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 1 -- 172.21.15.94:0/1823103796 wait complete. 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.188+0000 7fac8da12740 20 client.16602 trim_cache size 0 max 16384 2024-04-25T13:45:04.195 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 msgr2=0x7fac700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 0x7fac700b2000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac78022000 tx=0x7fac78023000 comp rx=0 tx=0).stop 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 msgr2=0x7fac8019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00e9f0 tx=0x7fac6c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 shutdown_connections 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fac700afaf0 0x7fac700b2000 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac80108920 0x7fac801a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac80103c20 0x7fac8019ab50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 --2- 172.21.15.94:0/4293739941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac801029e0 0x7fac8019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 >> 172.21.15.94:0/4293739941 conn(0x7fac800fdbd0 msgr2=0x7fac80109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 shutdown_connections 2024-04-25T13:45:04.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.189+0000 7fac86b64640 1 -- 172.21.15.94:0/4293739941 wait complete. 2024-04-25T13:45:04.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:45:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.316+0000 7fdde8f34640 1 -- 172.21.15.94:0/2125752337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 msgr2=0x7fdde4103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.316+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2125752337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4103750 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a410 comp rx=0 tx=0).stop 2024-04-25T13:45:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 -- 172.21.15.94:0/2125752337 shutdown_connections 2024-04-25T13:45:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2125752337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4108990 0x7fdde410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2125752337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4103c90 0x7fdde41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2125752337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4103750 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 -- 172.21.15.94:0/2125752337 >> 172.21.15.94:0/2125752337 conn(0x7fdde40fdc40 msgr2=0x7fdde41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 -- 172.21.15.94:0/2125752337 shutdown_connections 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 -- 172.21.15.94:0/2125752337 wait complete. 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 Processor -- start 2024-04-25T13:45:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.317+0000 7fdde8f34640 1 -- start start 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4103c90 0x7fdde4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4108990 0x7fdde41a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410e530 con 0x7fdde4102a50 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410e3b0 con 0x7fdde4108990 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410e6b0 con 0x7fdde4103c90 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45000/0 (socket says 172.21.15.94:45000) 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 -- 172.21.15.94:0/2135713739 learned_addr learned my addr 172.21.15.94:0/2135713739 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdddbfff640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4103c90 0x7fdde4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4108990 0x7fdde41a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 -- 172.21.15.94:0/2135713739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4103c90 msgr2=0x7fdde4199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4103c90 0x7fdde4199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 -- 172.21.15.94:0/2135713739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4108990 msgr2=0x7fdde41a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4108990 0x7fdde41a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 -- 172.21.15.94:0/2135713739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde41a0c70 con 0x7fdde4102a50 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde259c640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031440 tx=0x7fddd4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdddb7fe640 1 -- 172.21.15.94:0/2135713739 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd403f070 con 0x7fdde4102a50 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde41a0f00 con 0x7fdde4102a50 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdddb7fe640 1 -- 172.21.15.94:0/2135713739 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd4031c50 con 0x7fdde4102a50 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdddb7fe640 1 -- 172.21.15.94:0/2135713739 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035030 con 0x7fdde4102a50 2024-04-25T13:45:04.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.318+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a1300 con 0x7fdde4102a50 2024-04-25T13:45:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.321+0000 7fdddb7fe640 1 -- 172.21.15.94:0/2135713739 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fddd4042070 con 0x7fdde4102a50 2024-04-25T13:45:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.321+0000 7fddefde4740 1 Processor -- start 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 -- start start 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4d56b0 0x560d5e899a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560d5e7fcd80 con 0x560d5e4cd4d0 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d5e7b6a20 con 0x560d5e4f6390 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fddefde4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d5e7f8d40 con 0x560d5e4d56b0 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45010/0 (socket says 172.21.15.94:45010) 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdddb7fe640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 0x7fddd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde259c640 1 -- 172.21.15.94:0/2459890747 learned_addr learned my addr 172.21.15.94:0/2459890747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59028/0 (socket says 172.21.15.94:59028) 2024-04-25T13:45:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdddbfff640 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4d56b0 0x560d5e899a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 0x7fddd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdddb7fe640 1 -- 172.21.15.94:0/2135713739 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fddd40ff3d0 con 0x7fdde4102a50 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2459890747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4d56b0 msgr2=0x560d5e899a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4d56b0 0x560d5e899a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2459890747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 msgr2=0x560d5e898d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2459890747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d5e7c1c30 con 0x560d5e4f6390 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.322+0000 7fdde259c640 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.323+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005fd0 tx=0x7fdddc013ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9bc1372ee5443f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.323+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 0x7fddd00b2000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdddc01e000 tx=0x7fdddc020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.323+0000 7fddd8ff9640 1 -- 172.21.15.94:0/2459890747 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc01fce0 con 0x560d5e4f6390 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.323+0000 7fddd8ff9640 1 -- 172.21.15.94:0/2459890747 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc023020 con 0x560d5e4f6390 2024-04-25T13:45:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.323+0000 7fddd8ff9640 1 -- 172.21.15.94:0/2459890747 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc004030 con 0x560d5e4f6390 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- 172.21.15.94:0/2459890747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 msgr2=0x560d5e898810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005fd0 tx=0x7fdddc013ef0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- 172.21.15.94:0/2459890747 shutdown_connections 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4d56b0 0x560d5e899a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e898d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- 172.21.15.94:0/2459890747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4f6390 0x560d5e898810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- 172.21.15.94:0/2459890747 >> 172.21.15.94:0/2459890747 conn(0x560d5e4ea4e0 msgr2=0x560d5e894020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- 172.21.15.94:0/2459890747 shutdown_connections 2024-04-25T13:45:04.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- 172.21.15.94:0/2459890747 wait complete. 2024-04-25T13:45:04.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- start start 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 Processor -- start 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 0x560d5e8a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560d5e89a4d0 con 0x560d5e4d56b0 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d5e783dc0 con 0x560d5e4cd4d0 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.324+0000 7fddefde4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d5e89a650 con 0x560d5e4f6390 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddefde4740 10 client.0 _ll_register_callbacks cb 0x560d5e79f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45012/0 (socket says 172.21.15.94:45012) 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 -- 172.21.15.94:0/4022325896 learned_addr learned my addr 172.21.15.94:0/4022325896 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 0x560d5e8a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdde259c640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 -- 172.21.15.94:0/4022325896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 msgr2=0x560d5e8a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 0x560d5e8a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 -- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 msgr2=0x560d5e8a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 -- 172.21.15.94:0/4022325896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d5e50e3f0 con 0x560d5e4d56b0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 0x560d5e8a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdde259c640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fdddbfff640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fddcc012700 tx=0x7fddcc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da563e7b3eeab2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc004050 con 0x560d5e4d56b0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fddcc0041f0 con 0x560d5e4d56b0 2024-04-25T13:45:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdddc023020 con 0x560d5e4d56b0 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddcc0043f0 con 0x560d5e4d56b0 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddefde4740 10 client.16614 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.325+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d5e5d87d0 con 0x560d5e4d56b0 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc023020 con 0x560d5e4d56b0 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddcc003b50 con 0x560d5e4d56b0 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 10 client.16614 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 20 client.16614 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4b4f60 0x560d5e8b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 4 client.16614 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d5e80a990 con 0x560d5e4b4f60 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fdde259c640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4b4f60 0x560d5e8b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fddc9ffb640 10 client.16614 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.326+0000 7fdde259c640 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4b4f60 0x560d5e8b0270 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478a76a888c2a86 server_cookie=397be43b839b29f6 in_seq=0 out_seq=0 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddc9ffb640 1 -- 172.21.15.94:0/4022325896 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d5e80a990 con 0x560d5e4b4f60 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddc9ffb640 10 client.16614 handle_command_reply: tid=0 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 client.16614 shutdown 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 msgr2=0x560d5e8a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fddcc012700 tx=0x7fddcc012bc0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 shutdown_connections 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4b4f60 0x560d5e8b0270 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4f6390 0x560d5e8a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddc9ffb640 0 client.16614 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4d56b0 0x560d5e8a2a70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 --2- 172.21.15.94:0/4022325896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.327+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 >> 172.21.15.94:0/4022325896 conn(0x560d5e4ea4e0 msgr2=0x560d5e893040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.328+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 shutdown_connections 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.328+0000 7fddefde4740 1 -- 172.21.15.94:0/4022325896 wait complete. 2024-04-25T13:45:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.328+0000 7fddefde4740 20 client.16614 trim_cache size 0 max 16384 2024-04-25T13:45:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.343+0000 7fddefde4740 1 Processor -- start 2024-04-25T13:45:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.343+0000 7fddefde4740 1 -- start start 2024-04-25T13:45:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41aa470 con 0x560d5e4cd4d0 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41abfb0 con 0x560d5e4ea4e0 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddefde4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41aab40 con 0x560d5e4b4f60 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45018/0 (socket says 172.21.15.94:45018) 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2907026607 learned_addr learned my addr 172.21.15.94:0/2907026607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdddbfff640 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde259c640 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2907026607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 msgr2=0x560d5e8a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2907026607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 msgr2=0x560d5e8a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 -- 172.21.15.94:0/2907026607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d5e8a9e30 con 0x560d5e4cd4d0 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x560d5e899860 tx=0x7fdddc013970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bb1b57b314a7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.344+0000 7fddcbfff640 1 -- 172.21.15.94:0/2907026607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc01fe60 con 0x560d5e4cd4d0 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddcbfff640 1 -- 172.21.15.94:0/2907026607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc023070 con 0x560d5e4cd4d0 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddcbfff640 1 -- 172.21.15.94:0/2907026607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc005370 con 0x560d5e4cd4d0 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 -- 172.21.15.94:0/2907026607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 msgr2=0x560d5e8b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x560d5e899860 tx=0x7fdddc013970 comp rx=0 tx=0).stop 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 -- 172.21.15.94:0/2907026607 shutdown_connections 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8b4a20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 --2- 172.21.15.94:0/2907026607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 -- 172.21.15.94:0/2907026607 >> 172.21.15.94:0/2907026607 conn(0x560d5e8a1390 msgr2=0x560d5e89d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 -- 172.21.15.94:0/2907026607 shutdown_connections 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.345+0000 7fddefde4740 1 -- 172.21.15.94:0/2907026607 wait complete. 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 -- start start 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 Processor -- start 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41ab8e0 con 0x560d5e4cd4d0 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41ab210 con 0x560d5e4ea4e0 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41a8450 con 0x560d5e4b4f60 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fddefde4740 10 client.0 _ll_register_callbacks cb 0x7fddcc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45026/0 (socket says 172.21.15.94:45026) 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 -- 172.21.15.94:0/3228855897 learned_addr learned my addr 172.21.15.94:0/3228855897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:04.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdde2d9d640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdde259c640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 -- 172.21.15.94:0/3228855897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 msgr2=0x560d5e8a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 -- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 msgr2=0x560d5e8a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.346+0000 7fdddbfff640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fdddbfff640 1 -- 172.21.15.94:0/3228855897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d5e8a8a90 con 0x560d5e4cd4d0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fdddbfff640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fddcc012700 tx=0x7fddcc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6397f7530d7ea565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc034060 con 0x560d5e4cd4d0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fddcc003fa0 con 0x560d5e4cd4d0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdddc023070 con 0x560d5e4cd4d0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddcc02b040 con 0x560d5e4cd4d0 2024-04-25T13:45:04.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 10 client.16620 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d5e9185e0 con 0x560d5e4cd4d0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc023070 con 0x560d5e4cd4d0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddcc00fbf0 con 0x560d5e4cd4d0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 10 client.16620 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 20 client.16620 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4d56b0 0x560d5e91afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 4 client.16620 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.347+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d5e804800 con 0x560d5e4d56b0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fdde259c640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4d56b0 0x560d5e91afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddc9ffb640 10 client.16620 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fdde259c640 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4d56b0 0x560d5e91afd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e9d723b2b10d98 server_cookie=3a0c0534ee245f29 in_seq=0 out_seq=0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddc9ffb640 1 -- 172.21.15.94:0/3228855897 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d5e804800 con 0x560d5e4d56b0 2024-04-25T13:45:04.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddc9ffb640 10 client.16620 handle_command_reply: tid=0 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 client.16620 shutdown 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 msgr2=0x560d5e8a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fddcc012700 tx=0x7fddcc0107a0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 shutdown_connections 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560d5e4d56b0 0x560d5e91afd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d5e4ea4e0 0x560d5e8a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddc9ffb640 0 client.16620 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d5e4cd4d0 0x560d5e8a7610 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 --2- 172.21.15.94:0/3228855897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d5e4b4f60 0x560d5e8a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 >> 172.21.15.94:0/3228855897 conn(0x560d5e8a1390 msgr2=0x560d5e896e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 shutdown_connections 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.348+0000 7fddefde4740 1 -- 172.21.15.94:0/3228855897 wait complete. 2024-04-25T13:45:04.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fddefde4740 20 client.16620 trim_cache size 0 max 16384 2024-04-25T13:45:04.356 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:04.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 msgr2=0x7fddd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 0x7fddd00b2000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdddc01e000 tx=0x7fdddc020000 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 msgr2=0x7fdde4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031440 tx=0x7fddd4003c10 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 shutdown_connections 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fddd00afaf0 0x7fddd00b2000 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde4108990 0x7fdde41a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde4103c90 0x7fdde4199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 --2- 172.21.15.94:0/2135713739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde4102a50 0x7fdde4199130 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 >> 172.21.15.94:0/2135713739 conn(0x7fdde40fdc40 msgr2=0x7fdde4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 shutdown_connections 2024-04-25T13:45:04.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:04.349+0000 7fdde8f34640 1 -- 172.21.15.94:0/2135713739 wait complete. 2024-04-25T13:45:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:04 smithi094 ceph-mon[29195]: pgmap v444: 97 pgs: 97 active+clean; 41 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:45:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:04 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2079024594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:04 smithi137 ceph-mon[32568]: pgmap v444: 97 pgs: 97 active+clean; 41 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:45:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:04 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2079024594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:04 smithi138 ceph-mon[32977]: pgmap v444: 97 pgs: 97 active+clean; 41 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:45:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:04 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2079024594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:04.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 277155 14.14 MB/sec warmup 107 sec latency 16.307 ms 2024-04-25T13:45:05.361 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:45:05.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:05.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:45:05.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3890425845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4102260 msgr2=0x7ff4b410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3890425845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4102260 0x7ff4b410bcc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009900 tx=0x7ff4ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3890425845 shutdown_connections 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3890425845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4102260 0x7ff4b410bcc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3890425845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4101760 0x7ff4b4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3890425845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3890425845 >> 172.21.15.94:0/3890425845 conn(0x7ff4b40fd380 msgr2=0x7ff4b40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3890425845 shutdown_connections 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.477+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3890425845 wait complete. 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 Processor -- start 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 -- start start 2024-04-25T13:45:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 0x7ff4b419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410f640 con 0x7ff4b4101760 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410f4c0 con 0x7ff4b41050b0 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b9bce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410f7c0 con 0x7ff4b4102260 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45040/0 (socket says 172.21.15.94:45040) 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 0x7ff4b419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b37fe640 1 -- 172.21.15.94:0/3578749524 learned_addr learned my addr 172.21.15.94:0/3578749524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.478+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 -- 172.21.15.94:0/3578749524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 msgr2=0x7ff4b419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 0x7ff4b419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 -- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 msgr2=0x7ff4b419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 -- 172.21.15.94:0/3578749524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41a1d30 con 0x7ff4b41050b0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 0x7ff4b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41a16c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009230 tx=0x7ff4ac0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b0ff9640 1 -- 172.21.15.94:0/3578749524 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac03f030 con 0x7ff4b41050b0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a1fc0 con 0x7ff4b41050b0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b0ff9640 1 -- 172.21.15.94:0/3578749524 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4ac031910 con 0x7ff4b41050b0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b0ff9640 1 -- 172.21.15.94:0/3578749524 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac035070 con 0x7ff4b41050b0 2024-04-25T13:45:05.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.479+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a2420 con 0x7ff4b41050b0 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b0ff9640 1 -- 172.21.15.94:0/3578749524 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff4ac042030 con 0x7ff4b41050b0 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 Processor -- start 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 -- start start 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b596f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55a5550 0x5614b596fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614b58d4b00 con 0x5614b55ad730 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614b5870920 con 0x5614b55a5550 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614b58d0ac0 con 0x5614b55ce410 2024-04-25T13:45:05.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45050/0 (socket says 172.21.15.94:45050) 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/109873579 learned_addr learned my addr 172.21.15.94:0/109873579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b596f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.482+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55a5550 0x5614b596fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b0ff9640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 0x7ff49c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/109873579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 msgr2=0x5614b596f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b596f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/109873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55a5550 msgr2=0x5614b596fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55a5550 0x5614b596fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/109873579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b56ae7e0 con 0x5614b55ad730 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 0x7ff49c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8004630 tx=0x7ff4a8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ef61a05427c589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff4b0ff9640 1 -- 172.21.15.94:0/3578749524 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff4ac0ff190 con 0x7ff4b41050b0 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff499ffb640 1 -- 172.21.15.94:0/109873579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800f880 con 0x5614b55ad730 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff499ffb640 1 -- 172.21.15.94:0/109873579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a801f440 con 0x5614b55ad730 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.483+0000 7ff499ffb640 1 -- 172.21.15.94:0/109873579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e5f0 con 0x5614b55ad730 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- 172.21.15.94:0/109873579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 msgr2=0x5614b5976920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4b3fff640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 0x7ff49c0b2000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0fcf60 tx=0x7ff4ac092020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8004630 tx=0x7ff4a8009b80 comp rx=0 tx=0).stop 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- 172.21.15.94:0/109873579 shutdown_connections 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5976920 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55a5550 0x5614b596fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/109873579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b596f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- 172.21.15.94:0/109873579 >> 172.21.15.94:0/109873579 conn(0x5614b55c2560 msgr2=0x5614b596ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- 172.21.15.94:0/109873579 shutdown_connections 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- 172.21.15.94:0/109873579 wait complete. 2024-04-25T13:45:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.484+0000 7ff4c0a82740 1 -- start start 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 Processor -- start 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 0x5614b5979830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b5980790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614b5977370 con 0x5614b55a5550 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614b586e5d0 con 0x5614b55ad730 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614b59774f0 con 0x5614b55ce410 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4c0a82740 10 client.0 _ll_register_callbacks cb 0x5614b585c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45062/0 (socket says 172.21.15.94:45062) 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/607716037 learned_addr learned my addr 172.21.15.94:0/607716037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 0x5614b5979830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b5980790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/607716037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 msgr2=0x5614b5980790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b5980790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 msgr2=0x5614b5979830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 0x5614b5979830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b2ffd640 1 -- 172.21.15.94:0/607716037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b570ef70 con 0x5614b55a5550 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.485+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 0x5614b5979830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009710 tx=0x7ff4a80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec09bcc5366ff66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800fb90 con 0x5614b55a5550 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff4a800fd30 con 0x5614b55a5550 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a801f440 con 0x5614b55a5550 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff4a8005020 con 0x5614b55a5550 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 10 client.16632 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:05.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614b55a6170 con 0x5614b55a5550 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a801f440 con 0x5614b55a5550 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a801d020 con 0x5614b55a5550 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 10 client.16632 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 20 client.16632 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b558d1f0 0x5614b5986930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 4 client.16632 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.486+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b58e2710 con 0x5614b558d1f0 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.487+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b558d1f0 0x5614b5986930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.487+0000 7ff492ffd640 10 client.16632 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.487+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b558d1f0 0x5614b5986930 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7eef032f0367aa server_cookie=ad27023b836dd680 in_seq=0 out_seq=0 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.488+0000 7ff492ffd640 1 -- 172.21.15.94:0/607716037 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614b58e2710 con 0x5614b558d1f0 2024-04-25T13:45:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.488+0000 7ff492ffd640 10 client.16632 handle_command_reply: tid=0 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.488+0000 7ff4c0a82740 1 client.16632 shutdown 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 msgr2=0x5614b59792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009710 tx=0x7ff4a80096a0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 shutdown_connections 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b558d1f0 0x5614b5986930 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55ce410 0x5614b5980790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff492ffd640 0 client.16632 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ad730 0x5614b5979830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/607716037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55a5550 0x5614b59792f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 >> 172.21.15.94:0/607716037 conn(0x5614b55c2560 msgr2=0x5614b5969ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 shutdown_connections 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 1 -- 172.21.15.94:0/607716037 wait complete. 2024-04-25T13:45:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.489+0000 7ff4c0a82740 20 client.16632 trim_cache size 0 max 16384 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 Processor -- start 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 -- start start 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b5973db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b5977fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41ab510 con 0x5614b55ad730 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41ad050 con 0x5614b55ce410 2024-04-25T13:45:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41abbe0 con 0x5614b55a5550 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45068/0 (socket says 172.21.15.94:45068) 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 -- 172.21.15.94:0/777387417 learned_addr learned my addr 172.21.15.94:0/777387417 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b5973db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b5977fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 -- 172.21.15.94:0/777387417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 msgr2=0x5614b5973db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b5973db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 -- 172.21.15.94:0/777387417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 msgr2=0x5614b5977fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b5977fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 -- 172.21.15.94:0/777387417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b5970620 con 0x5614b55ad730 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b5977fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.508+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5614b5970880 tx=0x7ff4a4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5b93368cd03251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4997fa640 1 -- 172.21.15.94:0/777387417 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004b20 con 0x5614b55ad730 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4997fa640 1 -- 172.21.15.94:0/777387417 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4004cc0 con 0x5614b55ad730 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4997fa640 1 -- 172.21.15.94:0/777387417 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40053d0 con 0x5614b55ad730 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 -- 172.21.15.94:0/777387417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 msgr2=0x5614b5980720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5614b5970880 tx=0x7ff4a4009500 comp rx=0 tx=0).stop 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 -- 172.21.15.94:0/777387417 shutdown_connections 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b5977fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b5980720 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/777387417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b5973db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 -- 172.21.15.94:0/777387417 >> 172.21.15.94:0/777387417 conn(0x5614b55c2560 msgr2=0x5614b59778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 -- 172.21.15.94:0/777387417 shutdown_connections 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.509+0000 7ff4c0a82740 1 -- 172.21.15.94:0/777387417 wait complete. 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 -- start start 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 Processor -- start 2024-04-25T13:45:05.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b597da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b597eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41ac980 con 0x5614b55ad730 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41ac2b0 con 0x5614b55ce410 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b41a94f0 con 0x5614b55a5550 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4c0a82740 10 client.0 _ll_register_callbacks cb 0x7ff4a801fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b597da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45076/0 (socket says 172.21.15.94:45076) 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4b37fe640 1 -- 172.21.15.94:0/3108509661 learned_addr learned my addr 172.21.15.94:0/3108509661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.510+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b597eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 -- 172.21.15.94:0/3108509661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 msgr2=0x5614b597da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b597da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 -- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 msgr2=0x5614b597eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b597eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 -- 172.21.15.94:0/3108509661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b597f460 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b597da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4b37fe640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4024000 tx=0x7ff4a400ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaf42ea25e91d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a402e070 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7ff4a40203e0 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a4004cc0 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff4a403a050 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 10 client.16638 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614b5980750 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4004cc0 con 0x5614b55ad730 2024-04-25T13:45:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a40057f0 con 0x5614b55ad730 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 10 client.16638 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 20 client.16638 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b5562630 0x5614b59f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 4 client.16638 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.511+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5614b58dc580 con 0x5614b5562630 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.512+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b5562630 0x5614b59f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.512+0000 7ff492ffd640 10 client.16638 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.512+0000 7ff4b2ffd640 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b5562630 0x5614b59f11c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba507aae0a0d4c99 server_cookie=29cdcb19e46bb1b2 in_seq=0 out_seq=0 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.513+0000 7ff492ffd640 1 -- 172.21.15.94:0/3108509661 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614b58dc580 con 0x5614b5562630 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.513+0000 7ff492ffd640 10 client.16638 handle_command_reply: tid=0 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.513+0000 7ff4c0a82740 1 client.16638 shutdown 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 msgr2=0x5614b597df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4024000 tx=0x7ff4a400ff10 comp rx=0 tx=0).stop 2024-04-25T13:45:05.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 shutdown_connections 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5614b5562630 0x5614b59f11c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614b55ce410 0x5614b597eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614b55ad730 0x5614b597df70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 --2- 172.21.15.94:0/3108509661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614b55a5550 0x5614b597da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 >> 172.21.15.94:0/3108509661 conn(0x5614b55c2560 msgr2=0x5614b5979650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff492ffd640 0 client.16638 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 shutdown_connections 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 1 -- 172.21.15.94:0/3108509661 wait complete. 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4c0a82740 20 client.16638 trim_cache size 0 max 16384 2024-04-25T13:45:05.520 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:45:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "5d1c1ad2-c276-479f-bc97-ada5a4ec1720", 2024-04-25T13:45:05.521 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:45:05.521 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 msgr2=0x7ff49c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 0x7ff49c0b2000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0fcf60 tx=0x7ff4ac092020 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 msgr2=0x7ff4b41a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41a16c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009230 tx=0x7ff4ac0042f0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 shutdown_connections 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff49c0afaf0 0x7ff49c0b2000 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4b41050b0 0x7ff4b41a16c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4b4102260 0x7ff4b419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 --2- 172.21.15.94:0/3578749524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b4101760 0x7ff4b419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.514+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 >> 172.21.15.94:0/3578749524 conn(0x7ff4b40fd380 msgr2=0x7ff4b4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.515+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 shutdown_connections 2024-04-25T13:45:05.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.515+0000 7ff4b9bce640 1 -- 172.21.15.94:0/3578749524 wait complete. 2024-04-25T13:45:05.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:45:05.525 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 -- 172.21.15.94:0/375242729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 msgr2=0x7f6ba81054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 --2- 172.21.15.94:0/375242729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba81054b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009900 tx=0x7f6b9802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 -- 172.21.15.94:0/375242729 shutdown_connections 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 --2- 172.21.15.94:0/375242729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8102250 0x7f6ba810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 --2- 172.21.15.94:0/375242729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8101750 0x7f6ba8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 --2- 172.21.15.94:0/375242729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba81054b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 -- 172.21.15.94:0/375242729 >> 172.21.15.94:0/375242729 conn(0x7f6ba80fd350 msgr2=0x7f6ba80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 -- 172.21.15.94:0/375242729 shutdown_connections 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.640+0000 7f6badb4d640 1 -- 172.21.15.94:0/375242729 wait complete. 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 Processor -- start 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 -- start start 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 0x7f6ba8104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8102250 0x7f6ba81028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810f610 con 0x7f6ba81050a0 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810f490 con 0x7f6ba8101750 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6badb4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba810f790 con 0x7f6ba8102250 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6ba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 0x7f6ba8104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6ba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45096/0 (socket says 172.21.15.94:45096) 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6ba7fff640 1 -- 172.21.15.94:0/841348052 learned_addr learned my addr 172.21.15.94:0/841348052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.641+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8102250 0x7f6ba81028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 -- 172.21.15.94:0/841348052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8102250 msgr2=0x7f6ba81028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8102250 0x7f6ba81028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 -- 172.21.15.94:0/841348052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 msgr2=0x7f6ba8104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 0x7f6ba8104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 -- 172.21.15.94:0/841348052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba8103560 con 0x7f6ba81050a0 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 0x7f6ba8104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba7fff640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000eb10 tx=0x7f6ba000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba4ff9640 1 -- 172.21.15.94:0/841348052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0004b20 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba4ff9640 1 -- 172.21.15.94:0/841348052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba0004cc0 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6ba4ff9640 1 -- 172.21.15.94:0/841348052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0010670 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81a4340 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.642+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a47b0 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 Processor -- start 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba4ff9640 1 -- 172.21.15.94:0/841348052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6ba0005020 con 0x7f6ba81050a0 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 -- start start 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122e80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e81c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562122de5d80 con 0x562122ab64d0 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562122d9f8b0 con 0x562122abe6b0 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562122de1d40 con 0x562122adf390 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45110/0 (socket says 172.21.15.94:45110) 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 -- 172.21.15.94:0/3442913370 learned_addr learned my addr 172.21.15.94:0/3442913370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e81c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122e80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba4ff9640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 0x7f6b900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 -- 172.21.15.94:0/3442913370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 msgr2=0x562122e80a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122e80a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 -- 172.21.15.94:0/3442913370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 msgr2=0x562122e81c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e81c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba77fe640 1 -- 172.21.15.94:0/3442913370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562122d70a40 con 0x562122ab64d0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e81c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba7fff640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 0x7f6b900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.645+0000 7f6ba4ff9640 1 -- 172.21.15.94:0/841348052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6ba00df280 con 0x7f6ba81050a0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.646+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009230 tx=0x7f6b98032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde6ca44fe3a96ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.646+0000 7f6b8dffb640 1 -- 172.21.15.94:0/3442913370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98035070 con 0x562122ab64d0 2024-04-25T13:45:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.646+0000 7f6b8dffb640 1 -- 172.21.15.94:0/3442913370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b980317a0 con 0x562122ab64d0 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.646+0000 7f6b8dffb640 1 -- 172.21.15.94:0/3442913370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98003e30 con 0x562122ab64d0 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.646+0000 7f6ba7fff640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 0x7f6b900b2000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00a3740 tx=0x7f6ba0013330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- 172.21.15.94:0/3442913370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 msgr2=0x562122e80fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009230 tx=0x7f6b98032f90 comp rx=0 tx=0).stop 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- 172.21.15.94:0/3442913370 shutdown_connections 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e81c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e80fe0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/3442913370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122e80a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- 172.21.15.94:0/3442913370 >> 172.21.15.94:0/3442913370 conn(0x562122ad34e0 msgr2=0x562122e7c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- 172.21.15.94:0/3442913370 shutdown_connections 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- 172.21.15.94:0/3442913370 wait complete. 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 -- start start 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.647+0000 7f6bb4a02740 1 Processor -- start 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122ab64d0 0x562122e8a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562122e82720 con 0x562122adf390 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562122d6cd10 con 0x562122abe6b0 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562122e828a0 con 0x562122ab64d0 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 10 client.0 _ll_register_callbacks cb 0x562122d99210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45126/0 (socket says 172.21.15.94:45126) 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1968768139 learned_addr learned my addr 172.21.15.94:0/1968768139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122ab64d0 0x562122e8a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1968768139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122ab64d0 msgr2=0x562122e8a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122ab64d0 0x562122e8a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 msgr2=0x562122e8aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1968768139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562122bc17d0 con 0x562122adf390 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c011e00 tx=0x7f6b9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9607feca75b3ab79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c004220 con 0x562122adf390 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b9c0043c0 con 0x562122adf390 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.648+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b980317a0 con 0x562122adf390 2024-04-25T13:45:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b9c02b050 con 0x562122adf390 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 10 client.16650 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562122af73f0 con 0x562122adf390 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b980317a0 con 0x562122adf390 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b9c00e5c0 con 0x562122adf390 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 10 client.16650 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 20 client.16650 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a9df60 0x562122e982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 4 client.16650 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562122df3990 con 0x562122a9df60 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.649+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a9df60 0x562122e982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.650+0000 7f6b86ffd640 10 client.16650 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.650+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a9df60 0x562122e982e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dadaa9432b0d57 server_cookie=392201516a81d3f6 in_seq=0 out_seq=0 2024-04-25T13:45:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6b86ffd640 1 -- 172.21.15.94:0/1968768139 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562122df3990 con 0x562122a9df60 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6b86ffd640 10 client.16650 handle_command_reply: tid=0 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 client.16650 shutdown 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 msgr2=0x562122e91e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c011e00 tx=0x7f6b9c004730 comp rx=0 tx=0).stop 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 shutdown_connections 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a9df60 0x562122e982e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122adf390 0x562122e91e20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1968768139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122ab64d0 0x562122e8a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6b86ffd640 0 client.16650 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 >> 172.21.15.94:0/1968768139 conn(0x562122ad34e0 msgr2=0x562122e7f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 shutdown_connections 2024-04-25T13:45:05.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1968768139 wait complete. 2024-04-25T13:45:05.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.652+0000 7f6bb4a02740 20 client.16650 trim_cache size 0 max 16384 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.671+0000 7f6bb4a02740 1 Processor -- start 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.671+0000 7f6bb4a02740 1 -- start start 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122abe6b0 0x562122e7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122adf390 0x562122e91db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ad740 con 0x562122ab64d0 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81af280 con 0x562122adf390 2024-04-25T13:45:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ade10 con 0x562122abe6b0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45130/0 (socket says 172.21.15.94:45130) 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1470494626 learned_addr learned my addr 172.21.15.94:0/1470494626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122abe6b0 0x562122e7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122adf390 0x562122e91db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1470494626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122abe6b0 msgr2=0x562122e7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122abe6b0 0x562122e7c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1470494626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122adf390 msgr2=0x562122e91db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122adf390 0x562122e91db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/1470494626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562122ab70f0 con 0x562122ab64d0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dd0 tx=0x7f6b9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac03e396369848fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6b8d7fa640 1 -- 172.21.15.94:0/1470494626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005d30 con 0x562122ab64d0 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6b8d7fa640 1 -- 172.21.15.94:0/1470494626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c0044f0 con 0x562122ab64d0 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.672+0000 7f6b8d7fa640 1 -- 172.21.15.94:0/1470494626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c003970 con 0x562122ab64d0 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1470494626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 msgr2=0x562122e7bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009dd0 tx=0x7f6b9c005f90 comp rx=0 tx=0).stop 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1470494626 shutdown_connections 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122adf390 0x562122e91db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122abe6b0 0x562122e7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/1470494626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e7bc10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1470494626 >> 172.21.15.94:0/1470494626 conn(0x562122ad34e0 msgr2=0x562122e809e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1470494626 shutdown_connections 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.673+0000 7f6bb4a02740 1 -- 172.21.15.94:0/1470494626 wait complete. 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 -- start start 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 Processor -- start 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122efe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81aebb0 con 0x562122ab64d0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ae4e0 con 0x562122abe6b0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ab720 con 0x562122adf390 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6bb4a02740 10 client.0 _ll_register_callbacks cb 0x7f6b9c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45138/0 (socket says 172.21.15.94:45138) 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/2139585208 learned_addr learned my addr 172.21.15.94:0/2139585208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.674+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122efe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/2139585208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 msgr2=0x562122efe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122efe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 msgr2=0x562122e8fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 -- 172.21.15.94:0/2139585208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562122da64b0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6ba6ffd640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009710 tx=0x7f6b9c00f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9864a6168e17eddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c024650 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6b9c0247f0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c0044f0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b9c0249f0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 10 client.16656 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562122effa20 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c0044f0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6b9c03f9a0 con 0x562122ab64d0 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 10 client.16656 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 20 client.16656 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a733a0 0x562122f02410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 4 client.16656 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.675+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562122e77730 con 0x562122a733a0 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.676+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a733a0 0x562122f02410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.676+0000 7f6b86ffd640 10 client.16656 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.676+0000 7f6ba77fe640 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a733a0 0x562122f02410 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b59ebbbcba06a96 server_cookie=13db89b094eb748a in_seq=0 out_seq=0 2024-04-25T13:45:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6b86ffd640 1 -- 172.21.15.94:0/2139585208 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562122e77730 con 0x562122a733a0 2024-04-25T13:45:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6b86ffd640 10 client.16656 handle_command_reply: tid=0 2024-04-25T13:45:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 client.16656 shutdown 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 msgr2=0x562122e8f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009710 tx=0x7f6b9c00f5d0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 shutdown_connections 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562122a733a0 0x562122f02410 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562122adf390 0x562122efe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562122abe6b0 0x562122e8fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6b86ffd640 0 client.16656 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 --2- 172.21.15.94:0/2139585208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562122ab64d0 0x562122e8f4e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 >> 172.21.15.94:0/2139585208 conn(0x562122ad34e0 msgr2=0x562122e809e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 shutdown_connections 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 1 -- 172.21.15.94:0/2139585208 wait complete. 2024-04-25T13:45:05.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.678+0000 7f6bb4a02740 20 client.16656 trim_cache size 0 max 16384 2024-04-25T13:45:05.685 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:05.685 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:45:05.685 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:45:05.685 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 msgr2=0x7f6b900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 0x7f6b900b2000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00a3740 tx=0x7f6ba0013330 comp rx=0 tx=0).stop 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 msgr2=0x7f6ba8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000eb10 tx=0x7f6ba000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 shutdown_connections 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6b900afaf0 0x7f6b900b2000 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba81050a0 0x7f6ba8102e30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba8102250 0x7f6ba81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 --2- 172.21.15.94:0/841348052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ba8101750 0x7f6ba8104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 >> 172.21.15.94:0/841348052 conn(0x7f6ba80fd350 msgr2=0x7f6ba8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 shutdown_connections 2024-04-25T13:45:05.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.679+0000 7f6badb4d640 1 -- 172.21.15.94:0/841348052 wait complete. 2024-04-25T13:45:05.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:45:05.691 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:05.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 279608 14.12 MB/sec warmup 108 sec latency 109.554 ms 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 -- 172.21.15.94:0/2324381894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 msgr2=0x7fd15810b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 --2- 172.21.15.94:0/2324381894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15810b330 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd148009900 tx=0x7fd14802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 -- 172.21.15.94:0/2324381894 shutdown_connections 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 --2- 172.21.15.94:0/2324381894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 0x7fd15810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 --2- 172.21.15.94:0/2324381894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15810b330 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 --2- 172.21.15.94:0/2324381894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 0x7fd1581043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 -- 172.21.15.94:0/2324381894 >> 172.21.15.94:0/2324381894 conn(0x7fd1580fd330 msgr2=0x7fd1580ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 -- 172.21.15.94:0/2324381894 shutdown_connections 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.806+0000 7fd15f615640 1 -- 172.21.15.94:0/2324381894 wait complete. 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 Processor -- start 2024-04-25T13:45:05.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 -- start start 2024-04-25T13:45:05.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 0x7fd15819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 0x7fd1581a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd158111830 con 0x7fd158108e90 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1581116b0 con 0x7fd15810b9c0 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15f615640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1581119b0 con 0x7fd158103fb0 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15cbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15cbb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45152/0 (socket says 172.21.15.94:45152) 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15d3b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 0x7fd15819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15cbb0640 1 -- 172.21.15.94:0/3454255423 learned_addr learned my addr 172.21.15.94:0/3454255423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.807+0000 7fd15dbb2640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 0x7fd1581a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 -- 172.21.15.94:0/3454255423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 msgr2=0x7fd15819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 0x7fd15819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 -- 172.21.15.94:0/3454255423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 msgr2=0x7fd1581a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 0x7fd1581a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 -- 172.21.15.94:0/3454255423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1581a3fa0 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15cbb0640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd148031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14803f030 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1581a4230 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd148003aa0 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148035070 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.808+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1581a4770 con 0x7fd158108e90 2024-04-25T13:45:05.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.809+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1580fee50 con 0x7fd158108e90 2024-04-25T13:45:05.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.811+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd148042030 con 0x7fd158108e90 2024-04-25T13:45:05.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.811+0000 7fd14e7fc640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 0x7fd1440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:05.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.812+0000 7fd15d3b1640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 0x7fd1440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:05.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.812+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd1480ff880 con 0x7fd158108e90 2024-04-25T13:45:05.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.812+0000 7fd15d3b1640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 0x7fd1440b2000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd154005fd0 tx=0x7fd15400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:05.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:05.815+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1480c7d70 con 0x7fd158108e90 2024-04-25T13:45:06.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.050+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd15802db50 con 0x7fd158108e90 2024-04-25T13:45:06.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.051+0000 7fd14e7fc640 1 -- 172.21.15.94:0/3454255423 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fd1480cc070 con 0x7fd158108e90 2024-04-25T13:45:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:45:06.051 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:45:06.052 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 msgr2=0x7fd1440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 0x7fd1440b2000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd154005fd0 tx=0x7fd15400b390 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 msgr2=0x7fd15819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd148031a50 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 shutdown_connections 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1440afaf0 0x7fd1440b2000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd15810b9c0 0x7fd1581a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd158108e90 0x7fd15819c9a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 --2- 172.21.15.94:0/3454255423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd158103fb0 0x7fd15819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 >> 172.21.15.94:0/3454255423 conn(0x7fd1580fd330 msgr2=0x7fd158109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 shutdown_connections 2024-04-25T13:45:06.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.053+0000 7fd15f615640 1 -- 172.21.15.94:0/3454255423 wait complete. 2024-04-25T13:45:06.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:45:06.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 -- 172.21.15.94:0/676280450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 msgr2=0x7f575c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 --2- 172.21.15.94:0/676280450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c10be40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5754009ca0 tx=0x7f575402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 -- 172.21.15.94:0/676280450 shutdown_connections 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 --2- 172.21.15.94:0/676280450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c10be40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 --2- 172.21.15.94:0/676280450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c104d20 0x7f575c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 --2- 172.21.15.94:0/676280450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c103ae0 0x7f575c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 -- 172.21.15.94:0/676280450 >> 172.21.15.94:0/676280450 conn(0x7f575c0fecd0 msgr2=0x7f575c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 -- 172.21.15.94:0/676280450 shutdown_connections 2024-04-25T13:45:06.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.198+0000 7f5762138640 1 -- 172.21.15.94:0/676280450 wait complete. 2024-04-25T13:45:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.199+0000 7f5762138640 1 Processor -- start 2024-04-25T13:45:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 -- start start 2024-04-25T13:45:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 0x7f575c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c104d20 0x7f575c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10f610 con 0x7f575c109a20 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10f490 con 0x7f575c104d20 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f5762138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10f790 con 0x7f575c103ae0 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 0x7f575c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c104d20 0x7f575c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45158/0 (socket says 172.21.15.94:45158) 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575bfff640 1 -- 172.21.15.94:0/3876726420 learned_addr learned my addr 172.21.15.94:0/3876726420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 -- 172.21.15.94:0/3876726420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c104d20 msgr2=0x7f575c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c104d20 0x7f575c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 -- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 msgr2=0x7f575c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.200+0000 7f575b7fe640 1 -- 172.21.15.94:0/3876726420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c074ed0 con 0x7f575c103ae0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f575bfff640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f575b7fe640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 0x7f575c19a1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f575c1033b0 tx=0x7f574c00db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f5758ff9640 1 -- 172.21.15.94:0/3876726420 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c017070 con 0x7f575c103ae0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f5758ff9640 1 -- 172.21.15.94:0/3876726420 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f574c009cd0 con 0x7f575c103ae0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c0751c0 con 0x7f575c103ae0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f5758ff9640 1 -- 172.21.15.94:0/3876726420 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c003bd0 con 0x7f575c103ae0 2024-04-25T13:45:06.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.201+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c075760 con 0x7f575c103ae0 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5758ff9640 1 -- 172.21.15.94:0/3876726420 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f574c010460 con 0x7f575c103ae0 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 Processor -- start 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 -- start start 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f65d80 con 0x55c8d5c364d0 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f1fa20 con 0x55c8d5c5f390 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5768fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5f61d40 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55166/0 (socket says 172.21.15.94:55166) 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f5758ff9640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 0x7f57440b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45164/0 (socket says 172.21.15.94:45164) 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575b7fe640 1 -- 172.21.15.94:0/1976147008 learned_addr learned my addr 172.21.15.94:0/1976147008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.204+0000 7f575bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59220/0 (socket says 172.21.15.94:59220) 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575bfff640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 0x7f57440b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f5758ff9640 1 -- 172.21.15.94:0/3876726420 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f574c0de990 con 0x7f575c103ae0 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 -- 172.21.15.94:0/1976147008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 msgr2=0x55c8d6001810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 -- 172.21.15.94:0/1976147008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 msgr2=0x55c8d6001d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 -- 172.21.15.94:0/1976147008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d5f2ac30 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575b7fe640 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575bfff640 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575affd640 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5750009bd0 tx=0x7f5750011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbaeb9912986b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f575bfff640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 0x7f57440b2100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5754009a60 tx=0x7f575402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f5741ffb640 1 -- 172.21.15.94:0/1976147008 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750004230 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f5741ffb640 1 -- 172.21.15.94:0/1976147008 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5750009e40 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.205+0000 7f5741ffb640 1 -- 172.21.15.94:0/1976147008 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750004c70 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- 172.21.15.94:0/1976147008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 msgr2=0x55c8d6002a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5750009bd0 tx=0x7f5750011810 comp rx=0 tx=0).stop 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- 172.21.15.94:0/1976147008 shutdown_connections 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6002a20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c364d0 0x55c8d6001d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 --2- 172.21.15.94:0/1976147008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6001810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- 172.21.15.94:0/1976147008 >> 172.21.15.94:0/1976147008 conn(0x55c8d5c534e0 msgr2=0x55c8d5ffd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- 172.21.15.94:0/1976147008 shutdown_connections 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- 172.21.15.94:0/1976147008 wait complete. 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.206+0000 7f5768fed740 1 -- start start 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 Processor -- start 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6003530 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8d5eecdc0 con 0x55c8d5c5f390 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8d60036b0 con 0x55c8d5c364d0 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f5768fed740 10 client.0 _ll_register_callbacks cb 0x55c8d5f08600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59236/0 (socket says 172.21.15.94:59236) 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55168/0 (socket says 172.21.15.94:55168) 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.207+0000 7f575bfff640 1 -- 172.21.15.94:0/4243170957 learned_addr learned my addr 172.21.15.94:0/4243170957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 -- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 msgr2=0x55c8d600b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 -- 172.21.15.94:0/4243170957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 msgr2=0x55c8d600ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 -- 172.21.15.94:0/4243170957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d5c773f0 con 0x55c8d5c5f390 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f575bfff640 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57540097b0 tx=0x7f57540095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ef85a974a307a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f573affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754050030 con 0x55c8d5c5f390 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8d5f73990 con 0x55c8d5c5f390 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5754004750 con 0x55c8d5c5f390 2024-04-25T13:45:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5754003e20 con 0x55c8d5c5f390 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f5768fed740 10 client.26156 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c8d5d417d0 con 0x55c8d5c5f390 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.208+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c8d5f73990 con 0x55c8d5c5f390 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f575404b040 con 0x55c8d5c5f390 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 10 client.26156 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 10 client.26156 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 20 client.26156 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5c1df60 0x55c8d6019220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 4 client.26156 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d5f6d800 con 0x55c8d5c1df60 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f575b7fe640 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5c1df60 0x55c8d6019220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f573affd640 10 client.26156 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.209+0000 7f575b7fe640 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5c1df60 0x55c8d6019220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e7b6777159f86f8 server_cookie=802c3735ad0aa4fb in_seq=0 out_seq=0 2024-04-25T13:45:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f573affd640 1 -- 172.21.15.94:0/4243170957 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8d5f6d800 con 0x55c8d5c1df60 2024-04-25T13:45:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f573affd640 10 client.26156 handle_command_reply: tid=0 2024-04-25T13:45:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 client.26156 shutdown 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 msgr2=0x55c8d6012980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57540097b0 tx=0x7f57540095c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 shutdown_connections 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5c1df60 0x55c8d6019220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c5f390 0x55c8d6012980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 --2- 172.21.15.94:0/4243170957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f573affd640 0 client.26156 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.210+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 >> 172.21.15.94:0/4243170957 conn(0x55c8d5c534e0 msgr2=0x55c8d5ffff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.211+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 shutdown_connections 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.211+0000 7f5768fed740 1 -- 172.21.15.94:0/4243170957 wait complete. 2024-04-25T13:45:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.211+0000 7f5768fed740 20 client.26156 trim_cache size 0 max 16384 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 Processor -- start 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 -- start start 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c5f390 0x55c8d6006270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1ad6b0 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1af1f0 con 0x55c8d5c364d0 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f5768fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1add80 con 0x55c8d5c5f390 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45170/0 (socket says 172.21.15.94:45170) 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 -- 172.21.15.94:0/2153727084 learned_addr learned my addr 172.21.15.94:0/2153727084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575bfff640 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c5f390 0x55c8d6006270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 -- 172.21.15.94:0/2153727084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c5f390 msgr2=0x55c8d6006270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c5f390 0x55c8d6006270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 -- 172.21.15.94:0/2153727084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 msgr2=0x55c8d600b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 -- 172.21.15.94:0/2153727084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d6001390 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575b7fe640 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.225+0000 7f575affd640 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f57500058c0 tx=0x7f57500051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d176e03a613a1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.226+0000 7f57417fa640 1 -- 172.21.15.94:0/2153727084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750028070 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.226+0000 7f57417fa640 1 -- 172.21.15.94:0/2153727084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5750004680 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.226+0000 7f57417fa640 1 -- 172.21.15.94:0/2153727084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575001e040 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- 172.21.15.94:0/2153727084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 msgr2=0x55c8d600ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f57500058c0 tx=0x7f57500051d0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- 172.21.15.94:0/2153727084 shutdown_connections 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c5f390 0x55c8d6006270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c3e6b0 0x55c8d600ba20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- 172.21.15.94:0/2153727084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c364d0 0x55c8d600b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- 172.21.15.94:0/2153727084 >> 172.21.15.94:0/2153727084 conn(0x55c8d5c534e0 msgr2=0x55c8d6009cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- 172.21.15.94:0/2153727084 shutdown_connections 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- 172.21.15.94:0/2153727084 wait complete. 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- start start 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 Processor -- start 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d60109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6010ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1aeb20 con 0x55c8d5c5f390 2024-04-25T13:45:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1ae450 con 0x55c8d5c3e6b0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1ab690 con 0x55c8d5c364d0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.227+0000 7f5768fed740 10 client.0 _ll_register_callbacks cb 0x7f5754003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45184/0 (socket says 172.21.15.94:45184) 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d60109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575bfff640 1 -- 172.21.15.94:0/1043340980 learned_addr learned my addr 172.21.15.94:0/1043340980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575affd640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6010ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 -- 172.21.15.94:0/1043340980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c3e6b0 msgr2=0x55c8d6010ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6010ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 -- 172.21.15.94:0/1043340980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 msgr2=0x55c8d6011430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 -- 172.21.15.94:0/1043340980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d607f740 con 0x55c8d5c364d0 2024-04-25T13:45:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575bfff640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f575b7fe640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d60109b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f574c0a4090 tx=0x7f574c0a2e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6acb3df0df9d39e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f573affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c020040 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f574c0a30b0 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5750004680 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f574c0c1430 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f5768fed740 10 client.26080 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.228+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c8d6080a70 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5750004680 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f574c0a2790 con 0x55c8d5c364d0 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 10 client.26080 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 10 client.26080 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 20 client.26080 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5bf33a0 0x55c8d6083460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 4 client.26080 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8d5ee8610 con 0x55c8d5bf33a0 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f575affd640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5bf33a0 0x55c8d6083460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f573affd640 10 client.26080 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.229+0000 7f575affd640 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5bf33a0 0x55c8d6083460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=538a510b6d75ac1d server_cookie=2c200ff629702fab in_seq=0 out_seq=0 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f573affd640 1 -- 172.21.15.94:0/1043340980 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8d5ee8610 con 0x55c8d5bf33a0 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f573affd640 10 client.26080 handle_command_reply: tid=0 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 client.26080 shutdown 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 msgr2=0x55c8d60109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d60109b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f574c0a4090 tx=0x7f574c0a2e10 comp rx=0 tx=0).stop 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 shutdown_connections 2024-04-25T13:45:06.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c8d5bf33a0 0x55c8d6083460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c8d5c5f390 0x55c8d6011430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c8d5c3e6b0 0x55c8d6010ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 --2- 172.21.15.94:0/1043340980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8d5c364d0 0x55c8d60109b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f573affd640 0 client.26080 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 >> 172.21.15.94:0/1043340980 conn(0x55c8d5c534e0 msgr2=0x55c8d6001790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 shutdown_connections 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 1 -- 172.21.15.94:0/1043340980 wait complete. 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5768fed740 20 client.26080 trim_cache size 0 max 16384 2024-04-25T13:45:06.237 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:06.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 msgr2=0x7f57440b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 0x7f57440b2100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5754009a60 tx=0x7f575402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 msgr2=0x7f575c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.230+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 0x7f575c19a1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f575c1033b0 tx=0x7f574c00db30 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 shutdown_connections 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57440afbf0 0x7f57440b2100 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f575c109a20 0x7f575c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c104d20 0x7f575c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 --2- 172.21.15.94:0/3876726420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c103ae0 0x7f575c19a1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 >> 172.21.15.94:0/3876726420 conn(0x7f575c0fecd0 msgr2=0x7f575c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 shutdown_connections 2024-04-25T13:45:06.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.231+0000 7f5762138640 1 -- 172.21.15.94:0/3876726420 wait complete. 2024-04-25T13:45:06.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:45:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 -- 172.21.15.94:0/3321450637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc108920 msgr2=0x7f61dc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 --2- 172.21.15.94:0/3321450637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc108920 0x7f61dc10ad40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009ca0 tx=0x7f61d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 -- 172.21.15.94:0/3321450637 shutdown_connections 2024-04-25T13:45:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 --2- 172.21.15.94:0/3321450637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc108920 0x7f61dc10ad40 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 --2- 172.21.15.94:0/3321450637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 0x7f61dc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 --2- 172.21.15.94:0/3321450637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1029e0 0x7f61dc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.357+0000 7f61e357a640 1 -- 172.21.15.94:0/3321450637 >> 172.21.15.94:0/3321450637 conn(0x7f61dc0fdbd0 msgr2=0x7f61dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- 172.21.15.94:0/3321450637 shutdown_connections 2024-04-25T13:45:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- 172.21.15.94:0/3321450637 wait complete. 2024-04-25T13:45:06.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 Processor -- start 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- start start 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 0x7f61dc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 0x7f61dc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10f610 con 0x7f61dc1029e0 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10f490 con 0x7f61dc103c20 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.358+0000 7f61e357a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10f790 con 0x7f61dc108920 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45212/0 (socket says 172.21.15.94:45212) 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 -- 172.21.15.94:0/1509384798 learned_addr learned my addr 172.21.15.94:0/1509384798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e0b15640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 0x7f61dc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1b17640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 0x7f61dc1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 -- 172.21.15.94:0/1509384798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 msgr2=0x7f61dc1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 0x7f61dc1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 -- 172.21.15.94:0/1509384798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 msgr2=0x7f61dc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 0x7f61dc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 -- 172.21.15.94:0/1509384798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a2040 con 0x7f61dc1029e0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1b17640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 0x7f61dc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e1316640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e9f0 tx=0x7f61d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61d27fc640 1 -- 172.21.15.94:0/1509384798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d800ccc0 con 0x7f61dc1029e0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61d27fc640 1 -- 172.21.15.94:0/1509384798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80035f0 con 0x7f61dc1029e0 2024-04-25T13:45:06.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a2330 con 0x7f61dc1029e0 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61d27fc640 1 -- 172.21.15.94:0/1509384798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d800ce60 con 0x7f61dc1029e0 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.359+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a2880 con 0x7f61dc1029e0 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61d27fc640 1 -- 172.21.15.94:0/1509384798 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f61d8003aa0 con 0x7f61dc1029e0 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 Processor -- start 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 -- start start 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b8ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556393aeed80 con 0x5563937e8390 2024-04-25T13:45:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556393aa8a20 con 0x5563937c76b0 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61ea41f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556393aead40 con 0x5563937bf4d0 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45218/0 (socket says 172.21.15.94:45218) 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61e1b17640 1 -- 172.21.15.94:0/2272738305 learned_addr learned my addr 172.21.15.94:0/2272738305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61e1316640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.362+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b8ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 -- 172.21.15.94:0/2272738305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 msgr2=0x556393b8ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b8ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 -- 172.21.15.94:0/2272738305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 msgr2=0x556393b8a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61d27fc640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 0x7f61c80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 -- 172.21.15.94:0/2272738305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556393ab3c30 con 0x5563937c76b0 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e1b17640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e1316640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e1b17640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 0x7f61c80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b8ba20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009bf0 tx=0x7f61cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8df85b7320d0ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61d27fc640 1 -- 172.21.15.94:0/1509384798 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f61d80df040 con 0x7f61dc1029e0 2024-04-25T13:45:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61c77fe640 1 -- 172.21.15.94:0/2272738305 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004050 con 0x5563937c76b0 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61c77fe640 1 -- 172.21.15.94:0/2272738305 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc0054d0 con 0x5563937c76b0 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.363+0000 7f61c77fe640 1 -- 172.21.15.94:0/2272738305 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004cf0 con 0x5563937c76b0 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 -- 172.21.15.94:0/2272738305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 msgr2=0x556393b8ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b8ba20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009bf0 tx=0x7f61cc011810 comp rx=0 tx=0).stop 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61e1b17640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 0x7f61c80b2080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556393b8b860 tx=0x7f61d4031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 -- 172.21.15.94:0/2272738305 shutdown_connections 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b8ba20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2272738305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b8a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 -- 172.21.15.94:0/2272738305 >> 172.21.15.94:0/2272738305 conn(0x5563937dc4e0 msgr2=0x556393b86020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 -- 172.21.15.94:0/2272738305 shutdown_connections 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.364+0000 7f61ea41f740 1 -- 172.21.15.94:0/2272738305 wait complete. 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 -- start start 2024-04-25T13:45:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 Processor -- start 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b94530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556393b8c530 con 0x5563937e8390 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556393a75dc0 con 0x5563937c76b0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556393b8c6b0 con 0x5563937bf4d0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61ea41f740 10 client.0 _ll_register_callbacks cb 0x556393a91600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45228/0 (socket says 172.21.15.94:45228) 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61e1b17640 1 -- 172.21.15.94:0/2977046707 learned_addr learned my addr 172.21.15.94:0/2977046707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61e1316640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b94530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.365+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 -- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 msgr2=0x556393b94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 -- 172.21.15.94:0/2977046707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 msgr2=0x556393b9b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 -- 172.21.15.94:0/2977046707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563938003f0 con 0x5563937bf4d0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1b17640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61e1316640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b94530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61d80dcaf0 tx=0x7f61d8003de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3300b7261c2b22f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80a6dd0 con 0x5563937bf4d0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f61d80a3710 con 0x5563937bf4d0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc0054d0 con 0x5563937bf4d0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d80a6bf0 con 0x5563937bf4d0 2024-04-25T13:45:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61ea41f740 10 client.26092 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563938ca7d0 con 0x5563937bf4d0 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc0054d0 con 0x5563937bf4d0 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61d80afe50 con 0x5563937bf4d0 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61ea41f740 10 client.26092 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.366+0000 7f61ea41f740 10 client.26092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61ea41f740 20 client.26092 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5563937a6f60 0x556393ba2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61ea41f740 4 client.26092 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556393afc990 con 0x5563937a6f60 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5563937a6f60 0x556393ba2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61c4ff9640 10 client.26092 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.367+0000 7f61e0b15640 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5563937a6f60 0x556393ba2260 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9eb5e00af2781c8 server_cookie=aea57d972b1d52fb in_seq=0 out_seq=0 2024-04-25T13:45:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61c4ff9640 1 -- 172.21.15.94:0/2977046707 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556393afc990 con 0x5563937a6f60 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61c4ff9640 10 client.26092 handle_command_reply: tid=0 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 client.26092 shutdown 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 msgr2=0x556393b94530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b94530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61d80dcaf0 tx=0x7f61d8003de0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 shutdown_connections 2024-04-25T13:45:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5563937a6f60 0x556393ba2260 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b9b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937c76b0 0x556393b94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61c4ff9640 0 client.26092 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 --2- 172.21.15.94:0/2977046707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937bf4d0 0x556393b94530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 >> 172.21.15.94:0/2977046707 conn(0x5563937dc4e0 msgr2=0x556393b85040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 shutdown_connections 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 1 -- 172.21.15.94:0/2977046707 wait complete. 2024-04-25T13:45:06.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.368+0000 7f61ea41f740 20 client.26092 trim_cache size 0 max 16384 2024-04-25T13:45:06.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.382+0000 7f61ea41f740 1 Processor -- start 2024-04-25T13:45:06.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.382+0000 7f61ea41f740 1 -- start start 2024-04-25T13:45:06.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.382+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.382+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61ea41f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ab580 con 0x5563937e8390 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61ea41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ad0c0 con 0x5563937bf4d0 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61ea41f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1abc50 con 0x5563937c76b0 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59298/0 (socket says 172.21.15.94:59298) 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e0b15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45234/0 (socket says 172.21.15.94:45234) 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 -- 172.21.15.94:0/1016291669 learned_addr learned my addr 172.21.15.94:0/1016291669 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 -- 172.21.15.94:0/1016291669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 msgr2=0x556393b94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 -- 172.21.15.94:0/1016291669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 msgr2=0x556393b94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 -- 172.21.15.94:0/1016291669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556393b93900 con 0x5563937bf4d0 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1b17640 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61e1316640 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61d800c6a0 tx=0x7f61d80e2b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15cae26d1eb6175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61c6ffd640 1 -- 172.21.15.94:0/1016291669 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80a6dd0 con 0x5563937bf4d0 2024-04-25T13:45:06.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61c6ffd640 1 -- 172.21.15.94:0/1016291669 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80a6f90 con 0x5563937bf4d0 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.383+0000 7f61c6ffd640 1 -- 172.21.15.94:0/1016291669 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80a2df0 con 0x5563937bf4d0 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 -- 172.21.15.94:0/1016291669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 msgr2=0x556393b933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f61d800c6a0 tx=0x7f61d80e2b90 comp rx=0 tx=0).stop 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 -- 172.21.15.94:0/1016291669 shutdown_connections 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 --2- 172.21.15.94:0/1016291669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b933c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 -- 172.21.15.94:0/1016291669 >> 172.21.15.94:0/1016291669 conn(0x5563937dc4e0 msgr2=0x556393b92cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 -- 172.21.15.94:0/1016291669 shutdown_connections 2024-04-25T13:45:06.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.384+0000 7f61ea41f740 1 -- 172.21.15.94:0/1016291669 wait complete. 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 -- start start 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 Processor -- start 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b98640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b98b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ac9f0 con 0x5563937e8390 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ac320 con 0x5563937bf4d0 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1a9560 con 0x5563937c76b0 2024-04-25T13:45:06.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61ea41f740 10 client.0 _ll_register_callbacks cb 0x7f61d80045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61e0b15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b98b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61e1b17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45244/0 (socket says 172.21.15.94:45244) 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61e1316640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b98640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.385+0000 7f61e1b17640 1 -- 172.21.15.94:0/943585801 learned_addr learned my addr 172.21.15.94:0/943585801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 -- 172.21.15.94:0/943585801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 msgr2=0x556393b98b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b98b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 -- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 msgr2=0x556393b98640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b98640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 -- 172.21.15.94:0/943585801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556393b9a070 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61e1b17640 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61d4003f30 tx=0x7f61d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b577782c970a49b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4058060 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f61d4004710 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d80a6f90 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61d402f660 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61ea41f740 10 client.16698 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556393b9b360 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d80a6f90 con 0x5563937e8390 2024-04-25T13:45:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.386+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f61d40308f0 con 0x5563937e8390 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 10 client.16698 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 10 client.16698 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 20 client.16698 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55639377c3a0 0x556393c11200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 4 client.16698 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556393af6800 con 0x55639377c3a0 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61e1316640 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55639377c3a0 0x556393c11200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61c4ff9640 10 client.16698 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61e1316640 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55639377c3a0 0x556393c11200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9421a9680b6fe27 server_cookie=e86f570129b869d8 in_seq=0 out_seq=0 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61c4ff9640 1 -- 172.21.15.94:0/943585801 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556393af6800 con 0x55639377c3a0 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61c4ff9640 10 client.16698 handle_command_reply: tid=0 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.387+0000 7f61ea41f740 1 client.16698 shutdown 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 msgr2=0x556393b99ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61d4003f30 tx=0x7f61d40095c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 shutdown_connections 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55639377c3a0 0x556393c11200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563937e8390 0x556393b99ab0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61c4ff9640 0 client.16698 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563937c76b0 0x556393b98b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 --2- 172.21.15.94:0/943585801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563937bf4d0 0x556393b98640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 >> 172.21.15.94:0/943585801 conn(0x5563937dc4e0 msgr2=0x556393b94100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 shutdown_connections 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 1 -- 172.21.15.94:0/943585801 wait complete. 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61ea41f740 20 client.16698 trim_cache size 0 max 16384 2024-04-25T13:45:06.400 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:06.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 msgr2=0x7f61c80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 0x7f61c80b2080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556393b8b860 tx=0x7f61d4031af0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 msgr2=0x7f61dc19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.388+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e9f0 tx=0x7f61d800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 shutdown_connections 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61c80afb70 0x7f61c80b2080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc108920 0x7f61dc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc103c20 0x7f61dc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 --2- 172.21.15.94:0/1509384798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc1029e0 0x7f61dc19a500 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 >> 172.21.15.94:0/1509384798 conn(0x7f61dc0fdbd0 msgr2=0x7f61dc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 shutdown_connections 2024-04-25T13:45:06.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.389+0000 7f61e357a640 1 -- 172.21.15.94:0/1509384798 wait complete. 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: pgmap v445: 97 pgs: 97 active+clean; 46 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 446 op/s 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.403 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:45:06.404 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:06 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3454255423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:06.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 -- 172.21.15.94:0/3187555254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 msgr2=0x7f4c781094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 --2- 172.21.15.94:0/3187555254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781094e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009900 tx=0x7f4c6002a410 comp rx=0 tx=0).stop 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 -- 172.21.15.94:0/3187555254 shutdown_connections 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 --2- 172.21.15.94:0/3187555254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78109a20 0x7f4c7810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 --2- 172.21.15.94:0/3187555254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781094e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 --2- 172.21.15.94:0/3187555254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78101f10 0x7f4c78106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 -- 172.21.15.94:0/3187555254 >> 172.21.15.94:0/3187555254 conn(0x7f4c780fd350 msgr2=0x7f4c780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 -- 172.21.15.94:0/3187555254 shutdown_connections 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 -- 172.21.15.94:0/3187555254 wait complete. 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.518+0000 7f4c7d849640 1 Processor -- start 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 -- start start 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78101f10 0x7f4c78104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78109a20 0x7f4c78102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7810f6a0 con 0x7f4c781070c0 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7810f520 con 0x7f4c78101f10 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c7d849640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7810f820 con 0x7f4c78109a20 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45254/0 (socket says 172.21.15.94:45254) 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 -- 172.21.15.94:0/2961419715 learned_addr learned my addr 172.21.15.94:0/2961419715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78101f10 0x7f4c78104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c777fe640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78109a20 0x7f4c78102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 -- 172.21.15.94:0/2961419715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78109a20 msgr2=0x7f4c78102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78109a20 0x7f4c78102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 -- 172.21.15.94:0/2961419715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78101f10 msgr2=0x7f4c78104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78101f10 0x7f4c78104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 -- 172.21.15.94:0/2961419715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c78103660 con 0x7f4c781070c0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.519+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006010 tx=0x7f4c60004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.520+0000 7f4c7c847640 1 -- 172.21.15.94:0/2961419715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6003f030 con 0x7f4c781070c0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.520+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c781a4390 con 0x7f4c781070c0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.520+0000 7f4c7c847640 1 -- 172.21.15.94:0/2961419715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c60003cb0 con 0x7f4c781070c0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.520+0000 7f4c7c847640 1 -- 172.21.15.94:0/2961419715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60035070 con 0x7f4c781070c0 2024-04-25T13:45:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.520+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c781a4790 con 0x7f4c781070c0 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.522+0000 7f4c7c847640 1 -- 172.21.15.94:0/2961419715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4c60042030 con 0x7f4c781070c0 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.522+0000 7f4c847d2740 1 Processor -- start 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.522+0000 7f4c847d2740 1 -- start start 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 0x56509605b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c986b0 0x56509605ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565095fbfd80 con 0x565095c904d0 2024-04-25T13:45:06.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565095f79a20 con 0x565095cb9390 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c847d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565095fbbd40 con 0x565095c986b0 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 0x56509605b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45260/0 (socket says 172.21.15.94:45260) 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c76ffd640 1 -- 172.21.15.94:0/890642204 learned_addr learned my addr 172.21.15.94:0/890642204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c767fc640 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c986b0 0x56509605ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 -- 172.21.15.94:0/890642204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c986b0 msgr2=0x56509605ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c986b0 0x56509605ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 -- 172.21.15.94:0/890642204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 msgr2=0x56509605bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 -- 172.21.15.94:0/890642204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565095f84c30 con 0x565095cb9390 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c7c847640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 0x7f4c640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c777fe640 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 0x56509605b810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c680045b0 tx=0x7f4c6800bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aeb992b577be657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 0x7f4c640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c6dffb640 1 -- 172.21.15.94:0/890642204 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68003bd0 con 0x565095cb9390 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.523+0000 7f4c7c847640 1 -- 172.21.15.94:0/2961419715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4c600ff2a0 con 0x7f4c781070c0 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c6dffb640 1 -- 172.21.15.94:0/890642204 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68015070 con 0x565095cb9390 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c6dffb640 1 -- 172.21.15.94:0/890642204 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680116a0 con 0x565095cb9390 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 -- 172.21.15.94:0/890642204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 msgr2=0x56509605b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 0x7f4c640b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x56509605b530 tx=0x7f4c70009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 0x56509605b810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c680045b0 tx=0x7f4c6800bb10 comp rx=0 tx=0).stop 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 -- 172.21.15.94:0/890642204 shutdown_connections 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c986b0 0x56509605ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x56509605bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 --2- 172.21.15.94:0/890642204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095cb9390 0x56509605b810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 -- 172.21.15.94:0/890642204 >> 172.21.15.94:0/890642204 conn(0x565095cad4e0 msgr2=0x565096057020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.524+0000 7f4c847d2740 1 -- 172.21.15.94:0/890642204 shutdown_connections 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 -- 172.21.15.94:0/890642204 wait complete. 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 -- start start 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 Processor -- start 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x56509606caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56509605d530 con 0x565095c904d0 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565095f46dc0 con 0x565095c986b0 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56509605d6b0 con 0x565095cb9390 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c847d2740 10 client.0 _ll_register_callbacks cb 0x565095f62600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45274/0 (socket says 172.21.15.94:45274) 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c767fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.525+0000 7f4c76ffd640 1 -- 172.21.15.94:0/2533675692 learned_addr learned my addr 172.21.15.94:0/2533675692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c777fe640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x56509606caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 -- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 msgr2=0x56509606caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x56509606caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 -- 172.21.15.94:0/2533675692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 msgr2=0x565096065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 -- 172.21.15.94:0/2533675692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565095cd13f0 con 0x565095c904d0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005f40 tx=0x7f4c70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d03b4c728956c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7001e980 con 0x565095c904d0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4c7001eb20 con 0x565095c904d0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c68015070 con 0x565095c904d0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c7001ed60 con 0x565095c904d0 2024-04-25T13:45:06.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c847d2740 10 client.16710 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565095d9b7d0 con 0x565095c904d0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.526+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68015070 con 0x565095c904d0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7001a040 con 0x565095c904d0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 10 client.16710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 20 client.16710 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c77f60 0x5650960733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 4 client.16710 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565095fcd990 con 0x565095c77f60 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c77f60 0x5650960733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c56ffd640 10 client.16710 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.527+0000 7f4c767fc640 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c77f60 0x5650960733f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be391798b396ed8a server_cookie=e03645a5155748d1 in_seq=0 out_seq=0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c56ffd640 1 -- 172.21.15.94:0/2533675692 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565095fcd990 con 0x565095c77f60 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c56ffd640 10 client.16710 handle_command_reply: tid=0 2024-04-25T13:45:06.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 client.16710 shutdown 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 msgr2=0x565096065650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005f40 tx=0x7f4c70004a40 comp rx=0 tx=0).stop 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 shutdown_connections 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c77f60 0x5650960733f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x56509606caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c56ffd640 0 client.16710 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 --2- 172.21.15.94:0/2533675692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096065650 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 >> 172.21.15.94:0/2533675692 conn(0x565095cad4e0 msgr2=0x565096056040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 shutdown_connections 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.528+0000 7f4c847d2740 1 -- 172.21.15.94:0/2533675692 wait complete. 2024-04-25T13:45:06.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.529+0000 7f4c847d2740 20 client.16710 trim_cache size 0 max 16384 2024-04-25T13:45:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.543+0000 7f4c847d2740 1 Processor -- start 2024-04-25T13:45:06.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.543+0000 7f4c847d2740 1 -- start start 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.543+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 0x565096065220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c847d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781ad730 con 0x565095cb9390 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c847d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781af270 con 0x565095c986b0 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c847d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781ade00 con 0x565095c904d0 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45282/0 (socket says 172.21.15.94:45282) 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 -- 172.21.15.94:0/3149209225 learned_addr learned my addr 172.21.15.94:0/3149209225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c767fc640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 0x565096065220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 -- 172.21.15.94:0/3149209225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 msgr2=0x565096065220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 0x565096065220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 -- 172.21.15.94:0/3149209225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 msgr2=0x565096065760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 -- 172.21.15.94:0/3149209225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565096063770 con 0x565095cb9390 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 0x565096065220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c777fe640 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x56509605c860 tx=0x7f4c6800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd59a9eb9e0284b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c6d7fa640 1 -- 172.21.15.94:0/3149209225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68003bd0 con 0x565095cb9390 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c6d7fa640 1 -- 172.21.15.94:0/3149209225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68015050 con 0x565095cb9390 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.544+0000 7f4c6d7fa640 1 -- 172.21.15.94:0/3149209225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68011440 con 0x565095cb9390 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 -- 172.21.15.94:0/3149209225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 msgr2=0x565096077ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x56509605c860 tx=0x7f4c6800bf50 comp rx=0 tx=0).stop 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 -- 172.21.15.94:0/3149209225 shutdown_connections 2024-04-25T13:45:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095cb9390 0x565096077ba0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096065760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 --2- 172.21.15.94:0/3149209225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095c904d0 0x565096065220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 -- 172.21.15.94:0/3149209225 >> 172.21.15.94:0/3149209225 conn(0x565095cad4e0 msgr2=0x5650960603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 -- 172.21.15.94:0/3149209225 shutdown_connections 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.545+0000 7f4c847d2740 1 -- 172.21.15.94:0/3149209225 wait complete. 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 -- start start 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 Processor -- start 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096069430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x5650960d9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781aeba0 con 0x565095c904d0 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781ae4d0 con 0x565095c986b0 2024-04-25T13:45:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c781ab710 con 0x565095cb9390 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c847d2740 10 client.0 _ll_register_callbacks cb 0x7f4c700039f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45292/0 (socket says 172.21.15.94:45292) 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 -- 172.21.15.94:0/1813893276 learned_addr learned my addr 172.21.15.94:0/1813893276 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c767fc640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c777fe640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x5650960d9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 -- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 msgr2=0x5650960d9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x5650960d9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 -- 172.21.15.94:0/1813893276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 msgr2=0x565096069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096069430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.546+0000 7f4c76ffd640 1 -- 172.21.15.94:0/1813893276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509606c3a0 con 0x565095c904d0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c76ffd640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000f80 tx=0x7f4c70030b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6b565c243bbed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7001e7d0 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4c7001e970 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c68015050 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c7001eb70 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c847d2740 10 client.16716 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650960dac30 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68015050 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.547+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4c70040070 con 0x565095c904d0 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 10 client.16716 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 20 client.16716 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c4d3a0 0x5650960dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 4 client.16716 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565095fc7800 con 0x565095c4d3a0 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c767fc640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c4d3a0 0x5650960dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c56ffd640 10 client.16716 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c767fc640 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c4d3a0 0x5650960dd6a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad85c38f9bbfb529 server_cookie=ab1b8c1f0ead1aaa in_seq=0 out_seq=0 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c56ffd640 1 -- 172.21.15.94:0/1813893276 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565095fc7800 con 0x565095c4d3a0 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c56ffd640 10 client.16716 handle_command_reply: tid=0 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.548+0000 7f4c847d2740 1 client.16716 shutdown 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 msgr2=0x565096068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000f80 tx=0x7f4c70030b10 comp rx=0 tx=0).stop 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 shutdown_connections 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x565095c4d3a0 0x5650960dd6a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565095cb9390 0x5650960d9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c56ffd640 0 client.16716 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565095c986b0 0x565096069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 --2- 172.21.15.94:0/1813893276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565095c904d0 0x565096068ef0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 >> 172.21.15.94:0/1813893276 conn(0x565095cad4e0 msgr2=0x565096059e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 shutdown_connections 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 1 -- 172.21.15.94:0/1813893276 wait complete. 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c847d2740 20 client.16716 trim_cache size 0 max 16384 2024-04-25T13:45:06.556 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 msgr2=0x7f4c640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 0x7f4c640b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x56509605b530 tx=0x7f4c70009680 comp rx=0 tx=0).stop 2024-04-25T13:45:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 msgr2=0x7f4c781029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006010 tx=0x7f4c60004340 comp rx=0 tx=0).stop 2024-04-25T13:45:06.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 shutdown_connections 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4c640afaf0 0x7f4c640b2000 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c78109a20 0x7f4c78102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c781070c0 0x7f4c781029f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 --2- 172.21.15.94:0/2961419715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c78101f10 0x7f4c78104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.549+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 >> 172.21.15.94:0/2961419715 conn(0x7f4c780fd350 msgr2=0x7f4c78105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.550+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 shutdown_connections 2024-04-25T13:45:06.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.550+0000 7f4c7d849640 1 -- 172.21.15.94:0/2961419715 wait complete. 2024-04-25T13:45:06.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:45:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: pgmap v445: 97 pgs: 97 active+clean; 46 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 446 op/s 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:45:06.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:06 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3454255423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 -- 172.21.15.94:0/1334029784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 msgr2=0x7fc00c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 --2- 172.21.15.94:0/1334029784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c1069f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009900 tx=0x7fbffc02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 -- 172.21.15.94:0/1334029784 shutdown_connections 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 --2- 172.21.15.94:0/1334029784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 --2- 172.21.15.94:0/1334029784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 --2- 172.21.15.94:0/1334029784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c1069f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 -- 172.21.15.94:0/1334029784 >> 172.21.15.94:0/1334029784 conn(0x7fc00c0fdbd0 msgr2=0x7fc00c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 -- 172.21.15.94:0/1334029784 shutdown_connections 2024-04-25T13:45:06.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.678+0000 7fc01278f640 1 -- 172.21.15.94:0/1334029784 wait complete. 2024-04-25T13:45:06.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 Processor -- start 2024-04-25T13:45:06.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 -- start start 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f5e0 con 0x7fc00c1045d0 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f460 con 0x7fc00c109890 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.679+0000 7fc01278f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10f760 con 0x7fc00c106f30 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45324/0 (socket says 172.21.15.94:45324) 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00bfff640 1 -- 172.21.15.94:0/431724446 learned_addr learned my addr 172.21.15.94:0/431724446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc010d2c640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 -- 172.21.15.94:0/431724446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 msgr2=0x7fc00c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 -- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 msgr2=0x7fc00c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 -- 172.21.15.94:0/431724446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a20d0 con 0x7fc00c106f30 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc010d2c640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00bfff640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c19aad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc00000e9f0 tx=0x7fc00000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc0097fa640 1 -- 172.21.15.94:0/431724446 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00000ccc0 con 0x7fc00c106f30 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a23c0 con 0x7fc00c106f30 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc0097fa640 1 -- 172.21.15.94:0/431724446 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc000005020 con 0x7fc00c106f30 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.680+0000 7fc0097fa640 1 -- 172.21.15.94:0/431724446 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000004a00 con 0x7fc00c106f30 2024-04-25T13:45:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.681+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a2910 con 0x7fc00c106f30 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.683+0000 7fc019642740 1 Processor -- start 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.683+0000 7fc0097fa640 1 -- 172.21.15.94:0/431724446 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc000004ba0 con 0x7fc00c106f30 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.683+0000 7fc019642740 1 -- start start 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.683+0000 7fc019642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 0x55ed1eb44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.683+0000 7fc019642740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc019642740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc019642740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ed1eaa8d80 con 0x55ed1e7794d0 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc019642740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ed1ea62a20 con 0x55ed1e7816b0 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc019642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed1eaa4d40 con 0x55ed1e7a2390 2024-04-25T13:45:06.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc010d2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 0x55ed1eb44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc010d2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 0x55ed1eb44810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55340/0 (socket says 172.21.15.94:55340) 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45336/0 (socket says 172.21.15.94:45336) 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc010d2c640 1 -- 172.21.15.94:0/547514309 learned_addr learned my addr 172.21.15.94:0/547514309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc0097fa640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 -- 172.21.15.94:0/547514309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 msgr2=0x55ed1eb44810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 0x55ed1eb44810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 -- 172.21.15.94:0/547514309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 msgr2=0x55ed1eb45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 -- 172.21.15.94:0/547514309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1ea6dc30 con 0x55ed1e7794d0 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc010d2c640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc0097fa640 1 -- 172.21.15.94:0/431724446 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fc0000dec80 con 0x7fc00c106f30 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fc00bfff640 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbffc031440 tx=0x7fbffc0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a69be957f55e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.684+0000 7fbff27fc640 1 -- 172.21.15.94:0/547514309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035070 con 0x55ed1e7794d0 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc010d2c640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 0x7fbff40b2000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ed1eb45860 tx=0x7fc004004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fbff27fc640 1 -- 172.21.15.94:0/547514309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc0318a0 con 0x55ed1e7794d0 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fbff27fc640 1 -- 172.21.15.94:0/547514309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc03c440 con 0x55ed1e7794d0 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 -- 172.21.15.94:0/547514309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 msgr2=0x55ed1eb44d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbffc031440 tx=0x7fbffc0095a0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 -- 172.21.15.94:0/547514309 shutdown_connections 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb44d90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.685+0000 7fc019642740 1 --2- 172.21.15.94:0/547514309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7a2390 0x55ed1eb44810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- 172.21.15.94:0/547514309 >> 172.21.15.94:0/547514309 conn(0x55ed1e7964e0 msgr2=0x55ed1eb40020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- 172.21.15.94:0/547514309 shutdown_connections 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- 172.21.15.94:0/547514309 wait complete. 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- start start 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 Processor -- start 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 0x55ed1eb55a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ed1eb464d0 con 0x55ed1e7a2390 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ed1ea2fdc0 con 0x55ed1e7794d0 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed1eb46650 con 0x55ed1e7816b0 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.686+0000 7fc019642740 10 client.0 _ll_register_callbacks cb 0x55ed1ea4b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc010d2c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 0x55ed1eb55a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59394/0 (socket says 172.21.15.94:59394) 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 -- 172.21.15.94:0/3549703193 learned_addr learned my addr 172.21.15.94:0/3549703193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 -- 172.21.15.94:0/3549703193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 msgr2=0x55ed1eb4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 -- 172.21.15.94:0/3549703193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 msgr2=0x55ed1eb55a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 0x55ed1eb55a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 -- 172.21.15.94:0/3549703193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1e7ba3f0 con 0x55ed1e7794d0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc010d2c640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 0x55ed1eb55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02a450 tx=0x7fbffc005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b4324db3d57ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fbfeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035030 con 0x55ed1e7794d0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbffc0037f0 con 0x55ed1e7794d0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc0318a0 con 0x55ed1e7794d0 2024-04-25T13:45:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbffc04a040 con 0x55ed1e7794d0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc019642740 10 client.26210 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.687+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed1e8847d0 con 0x55ed1e7794d0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc0318a0 con 0x55ed1e7794d0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbffc04d070 con 0x55ed1e7794d0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 10 client.26210 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 10 client.26210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 20 client.26210 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e760f60 0x55ed1eb5be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 4 client.26210 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed1eab6990 con 0x55ed1e760f60 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.688+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e760f60 0x55ed1eb5be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.689+0000 7fbfeb7fe640 10 client.26210 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.689+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e760f60 0x55ed1eb5be50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4409ee9ec0317d9 server_cookie=22adeef1878d4d2e in_seq=0 out_seq=0 2024-04-25T13:45:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3549703193 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed1eab6990 con 0x55ed1e760f60 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fbfeb7fe640 10 client.26210 handle_command_reply: tid=0 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 client.26210 shutdown 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 msgr2=0x55ed1eb4e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02a450 tx=0x7fbffc005e60 comp rx=0 tx=0).stop 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 shutdown_connections 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e760f60 0x55ed1eb5be50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7a2390 0x55ed1eb55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fbfeb7fe640 0 client.26210 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 --2- 172.21.15.94:0/3549703193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 >> 172.21.15.94:0/3549703193 conn(0x55ed1e7964e0 msgr2=0x55ed1eb42f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 shutdown_connections 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 1 -- 172.21.15.94:0/3549703193 wait complete. 2024-04-25T13:45:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.690+0000 7fc019642740 20 client.26210 trim_cache size 0 max 16384 2024-04-25T13:45:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 Processor -- start 2024-04-25T13:45:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 -- start start 2024-04-25T13:45:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e760f60 0x55ed1eb3b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ab510 con 0x55ed1e7794d0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ad050 con 0x55ed1e7816b0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc019642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1abbe0 con 0x55ed1e760f60 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.704+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45354/0 (socket says 172.21.15.94:45354) 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 -- 172.21.15.94:0/617294162 learned_addr learned my addr 172.21.15.94:0/617294162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e760f60 0x55ed1eb3b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc010d2c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 -- 172.21.15.94:0/617294162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e760f60 msgr2=0x55ed1eb3b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e760f60 0x55ed1eb3b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 -- 172.21.15.94:0/617294162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 msgr2=0x55ed1eb4e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 -- 172.21.15.94:0/617294162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1eb4ebb0 con 0x55ed1e7794d0 2024-04-25T13:45:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.705+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc0000a7050 tx=0x7fc000005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e7314ac30f0fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.706+0000 7fbff1ffb640 1 -- 172.21.15.94:0/617294162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00001c040 con 0x55ed1e7794d0 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.706+0000 7fbff1ffb640 1 -- 172.21.15.94:0/617294162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0000a6bd0 con 0x55ed1e7794d0 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.706+0000 7fbff1ffb640 1 -- 172.21.15.94:0/617294162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0000aff20 con 0x55ed1e7794d0 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- 172.21.15.94:0/617294162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 msgr2=0x55ed1eb4e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc0000a7050 tx=0x7fc000005460 comp rx=0 tx=0).stop 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- 172.21.15.94:0/617294162 shutdown_connections 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7816b0 0x55ed1eb4e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e7794d0 0x55ed1eb4e130 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 --2- 172.21.15.94:0/617294162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e760f60 0x55ed1eb3b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- 172.21.15.94:0/617294162 >> 172.21.15.94:0/617294162 conn(0x55ed1e7964e0 msgr2=0x55ed1eb4ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- 172.21.15.94:0/617294162 shutdown_connections 2024-04-25T13:45:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- 172.21.15.94:0/617294162 wait complete. 2024-04-25T13:45:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 -- start start 2024-04-25T13:45:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.707+0000 7fc019642740 1 Processor -- start 2024-04-25T13:45:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb53620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ac980 con 0x55ed1e760f60 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ac2b0 con 0x55ed1e7794d0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1a94f0 con 0x55ed1e7816b0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 10 client.0 _ll_register_callbacks cb 0x7fc0000e2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45370/0 (socket says 172.21.15.94:45370) 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 -- 172.21.15.94:0/3556044794 learned_addr learned my addr 172.21.15.94:0/3556044794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc010d2c640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb53620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 -- 172.21.15.94:0/3556044794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 msgr2=0x55ed1eb53620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb53620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 -- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 msgr2=0x55ed1eb526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 -- 172.21.15.94:0/3556044794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1eb53be0 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc00bfff640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbffc03c860 tx=0x7fbffc03c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f9a95b99d73fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fbfeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc067060 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fbffc0035f0 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.708+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0000a6bd0 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbffc035040 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 10 client.16734 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed1eb54ec0 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0000a6bd0 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbffc04d040 con 0x55ed1e760f60 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 10 client.16734 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 20 client.16734 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e7a2390 0x55ed1ebc6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 4 client.16734 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed1eab0800 con 0x55ed1e7a2390 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.709+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e7a2390 0x55ed1ebc6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc00b7fe640 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e7a2390 0x55ed1ebc6360 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6b81deee88cac4a server_cookie=2b070fe20d2becc3 in_seq=0 out_seq=0 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fbfeb7fe640 10 client.16734 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fbfeb7fe640 1 -- 172.21.15.94:0/3556044794 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed1eab0800 con 0x55ed1e7a2390 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fbfeb7fe640 10 client.16734 handle_command_reply: tid=0 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 client.16734 shutdown 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 msgr2=0x55ed1eb521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbffc03c860 tx=0x7fbffc03c670 comp rx=0 tx=0).stop 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 shutdown_connections 2024-04-25T13:45:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ed1e7a2390 0x55ed1ebc6360 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed1e7816b0 0x55ed1eb53620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1e7794d0 0x55ed1eb526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fbfeb7fe640 0 client.16734 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 --2- 172.21.15.94:0/3556044794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ed1e760f60 0x55ed1eb521b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 >> 172.21.15.94:0/3556044794 conn(0x55ed1e7964e0 msgr2=0x55ed1eb4ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 shutdown_connections 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 1 -- 172.21.15.94:0/3556044794 wait complete. 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.710+0000 7fc019642740 20 client.16734 trim_cache size 0 max 16384 2024-04-25T13:45:06.718 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 msgr2=0x7fbff40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 0x7fbff40b2000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ed1eb45860 tx=0x7fc004004a40 comp rx=0 tx=0).stop 2024-04-25T13:45:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 msgr2=0x7fc00c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c19aad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc00000e9f0 tx=0x7fc00000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 shutdown_connections 2024-04-25T13:45:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc00c109890 0x7fc00c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc00c106f30 0x7fc00c19aad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 --2- 172.21.15.94:0/431724446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc00c1045d0 0x7fc00c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 >> 172.21.15.94:0/431724446 conn(0x7fc00c0fdbd0 msgr2=0x7fc00c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 shutdown_connections 2024-04-25T13:45:06.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.711+0000 7fc01278f640 1 -- 172.21.15.94:0/431724446 wait complete. 2024-04-25T13:45:06.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:45:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: pgmap v445: 97 pgs: 97 active+clean; 46 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 446 op/s 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:45:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:06 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3454255423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:06.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 281654 14.10 MB/sec warmup 109 sec latency 12.832 ms 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 -- 172.21.15.94:0/61707072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a01029e0 msgr2=0x7f36a01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 --2- 172.21.15.94:0/61707072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a01029e0 0x7f36a01036e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3690009900 tx=0x7f369002a410 comp rx=0 tx=0).stop 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 -- 172.21.15.94:0/61707072 shutdown_connections 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 --2- 172.21.15.94:0/61707072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a0108920 0x7f36a010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 --2- 172.21.15.94:0/61707072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 --2- 172.21.15.94:0/61707072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a01029e0 0x7f36a01036e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 -- 172.21.15.94:0/61707072 >> 172.21.15.94:0/61707072 conn(0x7f36a00fdbd0 msgr2=0x7f36a0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 -- 172.21.15.94:0/61707072 shutdown_connections 2024-04-25T13:45:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.838+0000 7f36a5734640 1 -- 172.21.15.94:0/61707072 wait complete. 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 Processor -- start 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 -- start start 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01029e0 0x7f36a019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 0x7f36a01a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010e4e0 con 0x7f36a0103c20 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010e360 con 0x7f36a0108920 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f36a5734640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010e660 con 0x7f36a01029e0 2024-04-25T13:45:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45394/0 (socket says 172.21.15.94:45394) 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f369f7fe640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 0x7f36a01a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 -- 172.21.15.94:0/520218261 learned_addr learned my addr 172.21.15.94:0/520218261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f369effd640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01029e0 0x7f36a019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 -- 172.21.15.94:0/520218261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01029e0 msgr2=0x7f36a019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01029e0 0x7f36a019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 -- 172.21.15.94:0/520218261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 msgr2=0x7f36a01a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 0x7f36a01a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f3697fff640 1 -- 172.21.15.94:0/520218261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a01a42e0 con 0x7f36a0103c20 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.839+0000 7f369f7fe640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 0x7f36a01a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f3697fff640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9f0 tx=0x7f368800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f369cff9640 1 -- 172.21.15.94:0/520218261 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368800cd80 con 0x7f36a0103c20 2024-04-25T13:45:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a01a45d0 con 0x7f36a0103c20 2024-04-25T13:45:06.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f369cff9640 1 -- 172.21.15.94:0/520218261 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36880108e0 con 0x7f36a0103c20 2024-04-25T13:45:06.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f369cff9640 1 -- 172.21.15.94:0/520218261 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688018980 con 0x7f36a0103c20 2024-04-25T13:45:06.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.840+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01a4b70 con 0x7f36a0103c20 2024-04-25T13:45:06.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.842+0000 7f36ac5e9740 1 Processor -- start 2024-04-25T13:45:06.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.842+0000 7f36ac5e9740 1 -- start start 2024-04-25T13:45:06.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc27d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 0x56323fc28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56323fb8bd80 con 0x56323f885390 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56323fb45a20 con 0x56323f85c4d0 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56323fb87d40 con 0x56323f8646b0 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45402/0 (socket says 172.21.15.94:45402) 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369f7fe640 1 -- 172.21.15.94:0/1221754087 learned_addr learned my addr 172.21.15.94:0/1221754087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc27d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369cff9640 1 -- 172.21.15.94:0/520218261 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3688010430 con 0x7f36a0103c20 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f3697fff640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 0x56323fc28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 -- 172.21.15.94:0/1221754087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 msgr2=0x56323fc28a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 0x56323fc28a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 -- 172.21.15.94:0/1221754087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 msgr2=0x56323fc27810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369effd640 1 -- 172.21.15.94:0/1221754087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323fb50c30 con 0x56323f85c4d0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f3697fff640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 0x56323fc28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.843+0000 7f369f7fe640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f369effd640 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc27d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f369002a930 tx=0x7f3690000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5ae66f22351b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f36957fa640 1 -- 172.21.15.94:0/1221754087 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690035030 con 0x56323f85c4d0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f36957fa640 1 -- 172.21.15.94:0/1221754087 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690040070 con 0x56323f85c4d0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f36957fa640 1 -- 172.21.15.94:0/1221754087 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690003870 con 0x56323f85c4d0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f369cff9640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 0x7f368c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f369f7fe640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 0x7f368c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.844+0000 7f369cff9640 1 -- 172.21.15.94:0/520218261 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f36880def50 con 0x7f36a0103c20 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f369f7fe640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 0x7f368c0b2080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56323fc28860 tx=0x7f3698009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1221754087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 msgr2=0x56323fc27d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc27d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f369002a930 tx=0x7f3690000bc0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1221754087 shutdown_connections 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f8646b0 0x56323fc28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc27d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1221754087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f885390 0x56323fc27810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1221754087 >> 172.21.15.94:0/1221754087 conn(0x56323f8794e0 msgr2=0x56323fc23020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1221754087 shutdown_connections 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.845+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1221754087 wait complete. 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 -- start start 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 Processor -- start 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f885390 0x56323fc38a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56323fc294d0 con 0x56323f8646b0 2024-04-25T13:45:06.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56323fb12dc0 con 0x56323f85c4d0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56323fc29650 con 0x56323f885390 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f36ac5e9740 10 client.0 _ll_register_callbacks cb 0x56323fb2e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f369effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45406/0 (socket says 172.21.15.94:45406) 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f3697fff640 1 -- 172.21.15.94:0/1611226407 learned_addr learned my addr 172.21.15.94:0/1611226407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.846+0000 7f369f7fe640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f885390 0x56323fc38a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 -- 172.21.15.94:0/1611226407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f885390 msgr2=0x56323fc38a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f885390 0x56323fc38a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 -- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 msgr2=0x56323fc31590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc31590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 -- 172.21.15.94:0/1611226407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323f89d3f0 con 0x56323f8646b0 2024-04-25T13:45:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36867fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f3697fff640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f36880a7140 tx=0x7f36880a6e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e644504c04d9d84d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880a6f50 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f36880a36d0 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3690040070 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36880a7440 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36ac5e9740 10 client.16746 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56323f9677d0 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.847+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690040070 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36880a2db0 con 0x56323f8646b0 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 10 client.16746 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 20 client.16746 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f843f60 0x56323fc3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 4 client.16746 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56323fb99990 con 0x56323f843f60 2024-04-25T13:45:06.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f369effd640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f843f60 0x56323fc3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f36867fc640 10 client.16746 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.848+0000 7f369effd640 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f843f60 0x56323fc3f1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93cdd3e40cef4f0f server_cookie=b222c8f7cae7c39a in_seq=0 out_seq=0 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36867fc640 1 -- 172.21.15.94:0/1611226407 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56323fb99990 con 0x56323f843f60 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36867fc640 10 client.16746 handle_command_reply: tid=0 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 client.16746 shutdown 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 msgr2=0x56323fc31ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f36880a7140 tx=0x7f36880a6e50 comp rx=0 tx=0).stop 2024-04-25T13:45:06.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 shutdown_connections 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f843f60 0x56323fc3f1b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f885390 0x56323fc38a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36867fc640 0 client.16746 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f8646b0 0x56323fc31ad0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/1611226407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f85c4d0 0x56323fc31590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 >> 172.21.15.94:0/1611226407 conn(0x56323f8794e0 msgr2=0x56323fc25f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 shutdown_connections 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 1 -- 172.21.15.94:0/1611226407 wait complete. 2024-04-25T13:45:06.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.851+0000 7f36ac5e9740 20 client.16746 trim_cache size 0 max 16384 2024-04-25T13:45:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 Processor -- start 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 -- start start 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ad760 con 0x56323f85c4d0 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01af2a0 con 0x56323f8646b0 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ade30 con 0x56323f843f60 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f369effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f369f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f369effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55404/0 (socket says 172.21.15.94:55404) 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f3697fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45412/0 (socket says 172.21.15.94:45412) 2024-04-25T13:45:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.865+0000 7f369effd640 1 -- 172.21.15.94:0/3891039000 learned_addr learned my addr 172.21.15.94:0/3891039000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 -- 172.21.15.94:0/3891039000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 msgr2=0x56323fc25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 -- 172.21.15.94:0/3891039000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 msgr2=0x56323fc38950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 -- 172.21.15.94:0/3891039000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323fc2f6a0 con 0x56323f843f60 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f3697fff640 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f369effd640 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3690005ab0 tx=0x7f3690031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132d1a521f81e706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f3694ff9640 1 -- 172.21.15.94:0/3891039000 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690035040 con 0x56323f843f60 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f3694ff9640 1 -- 172.21.15.94:0/3891039000 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690040030 con 0x56323f843f60 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.866+0000 7f3694ff9640 1 -- 172.21.15.94:0/3891039000 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900456d0 con 0x56323f843f60 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3891039000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 msgr2=0x56323fc230f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3690005ab0 tx=0x7f3690031400 comp rx=0 tx=0).stop 2024-04-25T13:45:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3891039000 shutdown_connections 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f85c4d0 0x56323fc38950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3891039000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f843f60 0x56323fc230f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3891039000 >> 172.21.15.94:0/3891039000 conn(0x56323f8794e0 msgr2=0x56323fc229e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3891039000 shutdown_connections 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3891039000 wait complete. 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.867+0000 7f36ac5e9740 1 -- start start 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 Processor -- start 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 0x56323fc36040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc36f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01aebd0 con 0x56323f843f60 2024-04-25T13:45:06.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ae500 con 0x56323f8646b0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ab740 con 0x56323f85c4d0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f36ac5e9740 10 client.0 _ll_register_callbacks cb 0x7f3688009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45416/0 (socket says 172.21.15.94:45416) 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 -- 172.21.15.94:0/3487365 learned_addr learned my addr 172.21.15.94:0/3487365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f3697fff640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 0x56323fc36040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369f7fe640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc36f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 -- 172.21.15.94:0/3487365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 msgr2=0x56323fc36040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 0x56323fc36040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 -- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 msgr2=0x56323fc36f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc36f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f369effd640 1 -- 172.21.15.94:0/3487365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323fc37530 con 0x56323f843f60 2024-04-25T13:45:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.868+0000 7f3697fff640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 0x56323fc36040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f369effd640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f369002a450 tx=0x7f3690031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7e223567fa0217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36867fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369005f060 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3690040030 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f3690051630 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3690035040 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36ac5e9740 10 client.16752 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56323fc387b0 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.869+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690040030 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3690045a40 con 0x56323f843f60 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 10 client.16752 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 20 client.16752 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f885390 0x56323fca9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 4 client.16752 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56323fb93800 con 0x56323f885390 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f3697fff640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f885390 0x56323fca9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.870+0000 7f36867fc640 10 client.16752 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.871+0000 7f3697fff640 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f885390 0x56323fca9530 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210fe1632eed74d4 server_cookie=d143629b78f37a97 in_seq=0 out_seq=0 2024-04-25T13:45:06.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36867fc640 1 -- 172.21.15.94:0/3487365 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56323fb93800 con 0x56323f885390 2024-04-25T13:45:06.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36867fc640 10 client.16752 handle_command_reply: tid=0 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 client.16752 shutdown 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 msgr2=0x56323fc35b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f369002a450 tx=0x7f3690031b10 comp rx=0 tx=0).stop 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 shutdown_connections 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56323f885390 0x56323fca9530 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56323f8646b0 0x56323fc36f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56323f85c4d0 0x56323fc36040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36867fc640 0 client.16752 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 --2- 172.21.15.94:0/3487365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56323f843f60 0x56323fc35b00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 >> 172.21.15.94:0/3487365 conn(0x56323f8794e0 msgr2=0x56323fc2fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 shutdown_connections 2024-04-25T13:45:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.872+0000 7f36ac5e9740 1 -- 172.21.15.94:0/3487365 wait complete. 2024-04-25T13:45:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36ac5e9740 20 client.16752 trim_cache size 0 max 16384 2024-04-25T13:45:06.879 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 msgr2=0x7f368c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 0x7f368c0b2080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56323fc28860 tx=0x7f3698009500 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 msgr2=0x7f36a019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9f0 tx=0x7f368800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 shutdown_connections 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f368c0afb70 0x7f368c0b2080 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a0108920 0x7f36a01a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a0103c20 0x7f36a019cce0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 --2- 172.21.15.94:0/520218261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01029e0 0x7f36a019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 >> 172.21.15.94:0/520218261 conn(0x7f36a00fdbd0 msgr2=0x7f36a00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 shutdown_connections 2024-04-25T13:45:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:06.873+0000 7f36a5734640 1 -- 172.21.15.94:0/520218261 wait complete. 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 -- 172.21.15.94:0/4164533211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56601050e0 msgr2=0x7f56601054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 --2- 172.21.15.94:0/4164533211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56601050e0 0x7f56601054f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f565c009860 tx=0x7f565c02a340 comp rx=0 tx=0).stop 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 -- 172.21.15.94:0/4164533211 shutdown_connections 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 --2- 172.21.15.94:0/4164533211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 0x7f566010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 --2- 172.21.15.94:0/4164533211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5660101790 0x7f5660101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 --2- 172.21.15.94:0/4164533211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56601050e0 0x7f56601054f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 -- 172.21.15.94:0/4164533211 >> 172.21.15.94:0/4164533211 conn(0x7f56600fd390 msgr2=0x7f56600ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 -- 172.21.15.94:0/4164533211 shutdown_connections 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 -- 172.21.15.94:0/4164533211 wait complete. 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.930+0000 7f5667300640 1 Processor -- start 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 -- start start 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 0x7f566019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56601050e0 0x7f56601a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f566010f650 con 0x7f5660101790 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f566010f4d0 con 0x7f5660102290 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f5667300640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f566010f7d0 con 0x7f56601050e0 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45428/0 (socket says 172.21.15.94:45428) 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 -- 172.21.15.94:0/531759137 learned_addr learned my addr 172.21.15.94:0/531759137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566489b640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 0x7f566019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566589d640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56601050e0 0x7f56601a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 -- 172.21.15.94:0/531759137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56601050e0 msgr2=0x7f56601a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56601050e0 0x7f56601a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 -- 172.21.15.94:0/531759137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 msgr2=0x7f566019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 0x7f566019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 -- 172.21.15.94:0/531759137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601a1df0 con 0x7f5660101790 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.931+0000 7f566509c640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f565c02a860 tx=0x7f565c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.932+0000 7f56567fc640 1 -- 172.21.15.94:0/531759137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c037040 con 0x7f5660101790 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.932+0000 7f56567fc640 1 -- 172.21.15.94:0/531759137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c030c10 con 0x7f5660101790 2024-04-25T13:45:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.932+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601a2080 con 0x7f5660101790 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.932+0000 7f56567fc640 1 -- 172.21.15.94:0/531759137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c0052b0 con 0x7f5660101790 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.932+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a2560 con 0x7f5660101790 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f56567fc640 1 -- 172.21.15.94:0/531759137 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f565c043020 con 0x7f5660101790 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 Processor -- start 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 -- start start 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564953456460 con 0x5649531501b0 2024-04-25T13:45:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564953410590 con 0x56495312f4d0 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566e1b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564953452420 con 0x5649531272f0 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566589d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566489b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566589d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45438/0 (socket says 172.21.15.94:45438) 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566489b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59486/0 (socket says 172.21.15.94:59486) 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566589d640 1 -- 172.21.15.94:0/3044059212 learned_addr learned my addr 172.21.15.94:0/3044059212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.935+0000 7f566509c640 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 -- 172.21.15.94:0/3044059212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 msgr2=0x5649534f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 -- 172.21.15.94:0/3044059212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 msgr2=0x5649534f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 -- 172.21.15.94:0/3044059212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495341d0d0 con 0x5649531501b0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566489b640 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f56567fc640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 0x7f564c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566509c640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 0x7f564c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f566589d640 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f565800e580 tx=0x7f56580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef235fd23a82b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f56567fc640 1 -- 172.21.15.94:0/531759137 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f565c0fdf70 con 0x7f5660101790 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f564b7fe640 1 -- 172.21.15.94:0/3044059212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658003970 con 0x5649531501b0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f564b7fe640 1 -- 172.21.15.94:0/3044059212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565800d830 con 0x5649531501b0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.936+0000 7f564b7fe640 1 -- 172.21.15.94:0/3044059212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580044c0 con 0x5649531501b0 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 -- 172.21.15.94:0/3044059212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 msgr2=0x5649534f0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566509c640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 0x7f564c0b2070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f565c03d000 tx=0x7f565c1019b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f565800e580 tx=0x7f56580095c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 -- 172.21.15.94:0/3044059212 shutdown_connections 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56495312f4d0 0x5649534f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 --2- 172.21.15.94:0/3044059212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531501b0 0x5649534f0970 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 -- 172.21.15.94:0/3044059212 >> 172.21.15.94:0/3044059212 conn(0x564953144300 msgr2=0x5649534ec300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 -- 172.21.15.94:0/3044059212 shutdown_connections 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.937+0000 7f566e1b2740 1 -- 172.21.15.94:0/3044059212 wait complete. 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 -- start start 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 Processor -- start 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x564953501b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5649534f8840 con 0x56495312f4d0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5649533ddb60 con 0x5649531501b0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5649534f89c0 con 0x5649531272f0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566e1b2740 10 client.0 _ll_register_callbacks cb 0x5649534022b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566489b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566589d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x564953501b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566489b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45448/0 (socket says 172.21.15.94:45448) 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566489b640 1 -- 172.21.15.94:0/1943035036 learned_addr learned my addr 172.21.15.94:0/1943035036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.938+0000 7f566589d640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 -- 172.21.15.94:0/1943035036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 msgr2=0x5649534fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 -- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 msgr2=0x564953501b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x564953501b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 -- 172.21.15.94:0/1943035036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564953168210 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566589d640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x564953501b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566489b640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f56500126e0 tx=0x7f5650012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13aa1a3aba981be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f5648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650004050 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f56500041f0 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565800d830 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f56500043f0 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566e1b2740 10 client.16764 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564953234620 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.939+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565800d830 con 0x56495312f4d0 2024-04-25T13:45:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5650003b40 con 0x56495312f4d0 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 10 client.16764 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 20 client.16764 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56495310ed80 0x564953508390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 4 client.16764 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564953464070 con 0x56495310ed80 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566589d640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56495310ed80 0x564953508390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f5648ff9640 10 client.16764 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.940+0000 7f566589d640 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56495310ed80 0x564953508390 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e994011a43a345 server_cookie=b73cc2ed0f5268a7 in_seq=0 out_seq=0 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f5648ff9640 1 -- 172.21.15.94:0/1943035036 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564953464070 con 0x56495310ed80 2024-04-25T13:45:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f5648ff9640 10 client.16764 handle_command_reply: tid=0 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 client.16764 shutdown 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 msgr2=0x5649534fabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f56500126e0 tx=0x7f5650012ba0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 shutdown_connections 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56495310ed80 0x564953508390 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x564953501b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x5649534fabe0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f5648ff9640 0 client.16764 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1943035036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x5649534fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.941+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 >> 172.21.15.94:0/1943035036 conn(0x564953144300 msgr2=0x5649534ef210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.942+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 shutdown_connections 2024-04-25T13:45:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.942+0000 7f566e1b2740 1 -- 172.21.15.94:0/1943035036 wait complete. 2024-04-25T13:45:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.942+0000 7f566e1b2740 20 client.16764 trim_cache size 0 max 16384 2024-04-25T13:45:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 Processor -- start 2024-04-25T13:45:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 -- start start 2024-04-25T13:45:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56495312f4d0 0x5649534ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649534ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56601ab650 con 0x5649531272f0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56601ad190 con 0x5649531501b0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566e1b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56601abd20 con 0x56495312f4d0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45460/0 (socket says 172.21.15.94:45460) 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 -- 172.21.15.94:0/1936616532 learned_addr learned my addr 172.21.15.94:0/1936616532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566489b640 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56495312f4d0 0x5649534ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649534ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 -- 172.21.15.94:0/1936616532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56495312f4d0 msgr2=0x5649534ec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56495312f4d0 0x5649534ec200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 -- 172.21.15.94:0/1936616532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 msgr2=0x5649534ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649534ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.957+0000 7f566589d640 1 -- 172.21.15.94:0/1936616532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649534f7ee0 con 0x5649531272f0 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.958+0000 7f566589d640 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5658009c90 tx=0x7f56580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57951e1b1f83e53d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.958+0000 7f564affd640 1 -- 172.21.15.94:0/1936616532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800de80 con 0x5649531272f0 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.958+0000 7f564affd640 1 -- 172.21.15.94:0/1936616532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5658003e20 con 0x5649531272f0 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.958+0000 7f564affd640 1 -- 172.21.15.94:0/1936616532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658031a20 con 0x5649531272f0 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- 172.21.15.94:0/1936616532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 msgr2=0x5649534ebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5658009c90 tx=0x7f56580095c0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- 172.21.15.94:0/1936616532 shutdown_connections 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649534ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56495312f4d0 0x5649534ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- 172.21.15.94:0/1936616532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649531272f0 0x5649534ebcc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- 172.21.15.94:0/1936616532 >> 172.21.15.94:0/1936616532 conn(0x564953144300 msgr2=0x5649534f4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- 172.21.15.94:0/1936616532 shutdown_connections 2024-04-25T13:45:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- 172.21.15.94:0/1936616532 wait complete. 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- start start 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 Processor -- start 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x56495356bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649535730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56601acac0 con 0x56495312f4d0 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56601ac3f0 con 0x5649531501b0 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56601a9630 con 0x5649531272f0 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566e1b2740 10 client.0 _ll_register_callbacks cb 0x7f5650009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.959+0000 7f566489b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45462/0 (socket says 172.21.15.94:45462) 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 -- 172.21.15.94:0/2623440086 learned_addr learned my addr 172.21.15.94:0/2623440086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566589d640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649535730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566589d640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x56495356bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 -- 172.21.15.94:0/2623440086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 msgr2=0x56495356bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x56495356bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 -- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 msgr2=0x5649535730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649535730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 -- 172.21.15.94:0/2623440086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649535736a0 con 0x56495312f4d0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566489b640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f56500126e0 tx=0x7f5650009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba7e081468ee84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f5648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650035060 con 0x56495312f4d0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5658003e20 con 0x56495312f4d0 2024-04-25T13:45:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5650004150 con 0x56495312f4d0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f565002c040 con 0x56495312f4d0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566e1b2740 10 client.16770 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.960+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564953574930 con 0x56495312f4d0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5658003e20 con 0x56495312f4d0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f565002f070 con 0x56495312f4d0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 10 client.16770 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 20 client.16770 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5649530e41c0 0x564953577350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 4 client.16770 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56495345dee0 con 0x5649530e41c0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f566589d640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5649530e41c0 0x564953577350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.961+0000 7f5648ff9640 10 client.16770 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:06.962+0000 7f566589d640 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5649530e41c0 0x564953577350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d68cd9b81845e0b server_cookie=aa36acc8ae3b473e in_seq=0 out_seq=0 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f5648ff9640 1 -- 172.21.15.94:0/2623440086 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56495345dee0 con 0x5649530e41c0 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f5648ff9640 10 client.16770 handle_command_reply: tid=0 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 client.16770 shutdown 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 msgr2=0x56495356c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f56500126e0 tx=0x7f5650009f20 comp rx=0 tx=0).stop 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 shutdown_connections 2024-04-25T13:45:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5649530e41c0 0x564953577350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5649531501b0 0x5649535730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56495312f4d0 0x56495356c180 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f5648ff9640 0 client.16770 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 --2- 172.21.15.94:0/2623440086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5649531272f0 0x56495356bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 >> 172.21.15.94:0/2623440086 conn(0x564953144300 msgr2=0x5649534f4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 shutdown_connections 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 1 -- 172.21.15.94:0/2623440086 wait complete. 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.123+0000 7f566e1b2740 20 client.16770 trim_cache size 0 max 16384 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:45:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:45:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 msgr2=0x7f564c0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 0x7f564c0b2070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f565c03d000 tx=0x7f565c1019b0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 msgr2=0x7f566019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f565c02a860 tx=0x7f565c032d50 comp rx=0 tx=0).stop 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 shutdown_connections 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f564c0afb60 0x7f564c0b2070 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56601050e0 0x7f56601a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5660102290 0x7f566019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 --2- 172.21.15.94:0/531759137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5660101790 0x7f566019a2b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 >> 172.21.15.94:0/531759137 conn(0x7f56600fd390 msgr2=0x7f5660105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 shutdown_connections 2024-04-25T13:45:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:07.124+0000 7f5667300640 1 -- 172.21.15.94:0/531759137 wait complete. 2024-04-25T13:45:07.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 284110 14.09 MB/sec warmup 110 sec latency 87.872 ms 2024-04-25T13:45:07.886 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:45:07.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:07.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:45:08.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 -- 172.21.15.94:0/2008840432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938102f40 msgr2=0x7f9938103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 --2- 172.21.15.94:0/2008840432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938102f40 0x7f9938103350 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9928009900 tx=0x7f992802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 -- 172.21.15.94:0/2008840432 shutdown_connections 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 --2- 172.21.15.94:0/2008840432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938104460 0x7f993810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 --2- 172.21.15.94:0/2008840432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938103920 0x7f9938103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 --2- 172.21.15.94:0/2008840432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938102f40 0x7f9938103350 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 -- 172.21.15.94:0/2008840432 >> 172.21.15.94:0/2008840432 conn(0x7f99380fd340 msgr2=0x7f99380ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 -- 172.21.15.94:0/2008840432 shutdown_connections 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.003+0000 7f993d914640 1 -- 172.21.15.94:0/2008840432 wait complete. 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 Processor -- start 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 -- start start 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938102f40 0x7f9938199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938104460 0x7f99381a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f993810e4f0 con 0x7f9938103920 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f993810e370 con 0x7f9938104460 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.004+0000 7f993d914640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f993810e670 con 0x7f9938102f40 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45490/0 (socket says 172.21.15.94:45490) 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 -- 172.21.15.94:0/3217078625 learned_addr learned my addr 172.21.15.94:0/3217078625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f9936ffd640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938102f40 0x7f9938199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99377fe640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938104460 0x7f99381a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 -- 172.21.15.94:0/3217078625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938102f40 msgr2=0x7f9938199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938102f40 0x7f9938199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 -- 172.21.15.94:0/3217078625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938104460 msgr2=0x7f99381a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938104460 0x7f99381a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 -- 172.21.15.94:0/3217078625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99381a0c60 con 0x7f9938103920 2024-04-25T13:45:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f99367fc640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e9f0 tx=0x7f992c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f993c912640 1 -- 172.21.15.94:0/3217078625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c00cd80 con 0x7f9938103920 2024-04-25T13:45:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381a0f50 con 0x7f9938103920 2024-04-25T13:45:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f993c912640 1 -- 172.21.15.94:0/3217078625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c0108e0 con 0x7f9938103920 2024-04-25T13:45:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f993c912640 1 -- 172.21.15.94:0/3217078625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c018980 con 0x7f9938103920 2024-04-25T13:45:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.005+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381a1410 con 0x7f9938103920 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 Processor -- start 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 -- start start 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f993c912640 1 -- 172.21.15.94:0/3217078625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f992c018b20 con 0x7f9938103920 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46850550 0x557f46c1cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f46b7fb00 con 0x557f46858730 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f46b1b920 con 0x557f46850550 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99447c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f46b7bac0 con 0x557f46879410 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99377fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46850550 0x557f46c1cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c23b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.008+0000 7f99377fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55480/0 (socket says 172.21.15.94:55480) 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 -- 172.21.15.94:0/3451145855 learned_addr learned my addr 172.21.15.94:0/3451145855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 -- 172.21.15.94:0/3451145855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46850550 msgr2=0x557f46c1cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46850550 0x557f46c1cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 -- 172.21.15.94:0/3451145855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 msgr2=0x557f46c23b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f993c912640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 0x7f99200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c23b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 -- 172.21.15.94:0/3451145855 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f469597e0 con 0x557f46879410 2024-04-25T13:45:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f993c912640 1 -- 172.21.15.94:0/3217078625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f992c0df020 con 0x7f9938103920 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 0x7f99200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.009+0000 7f99377fe640 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9930009fd0 tx=0x7f99300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f16b1241957e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f991dffb640 1 -- 172.21.15.94:0/3451145855 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000f6e0 con 0x557f46879410 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f991dffb640 1 -- 172.21.15.94:0/3451145855 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993000fd30 con 0x557f46879410 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f991dffb640 1 -- 172.21.15.94:0/3451145855 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993001bb50 con 0x557f46879410 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 -- 172.21.15.94:0/3451145855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 msgr2=0x557f46c1c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9930009fd0 tx=0x7f99300118d0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99377fe640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 0x7f99200b2000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9930020000 tx=0x7f9930021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 -- 172.21.15.94:0/3451145855 shutdown_connections 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46850550 0x557f46c1cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 --2- 172.21.15.94:0/3451145855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46879410 0x557f46c1c6b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 -- 172.21.15.94:0/3451145855 >> 172.21.15.94:0/3451145855 conn(0x557f4686d560 msgr2=0x557f46c15e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.010+0000 7f99447c6740 1 -- 172.21.15.94:0/3451145855 shutdown_connections 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 -- 172.21.15.94:0/3451145855 wait complete. 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 -- start start 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 Processor -- start 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c26a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46879410 0x557f46c2d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f46c24580 con 0x557f46858730 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f46b195d0 con 0x557f46879410 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f46c24700 con 0x557f46850550 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.011+0000 7f99447c6740 10 client.0 _ll_register_callbacks cb 0x557f46b07080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c26a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:55482/0 (socket says 172.21.15.94:55482) 2024-04-25T13:45:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c26a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45504/0 (socket says 172.21.15.94:45504) 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99377fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46879410 0x557f46c2d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 -- 172.21.15.94:0/902138928 learned_addr learned my addr 172.21.15.94:0/902138928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 -- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46879410 msgr2=0x557f46c2d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46879410 0x557f46c2d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 -- 172.21.15.94:0/902138928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 msgr2=0x557f46c26a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c26a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 -- 172.21.15.94:0/902138928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f469b9f70 con 0x557f46850550 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9936ffd640 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9928009230 tx=0x7f9928031dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7585aabd5f04142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928035040 con 0x557f46850550 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993000fd30 con 0x557f46850550 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9928004040 con 0x557f46850550 2024-04-25T13:45:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9928004240 con 0x557f46850550 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99447c6740 10 client.26179 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.012+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f46851170 con 0x557f46850550 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993000fd30 con 0x557f46850550 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9928044020 con 0x557f46850550 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 10 client.26179 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 20 client.26179 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f468381f0 0x557f46c33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 4 client.26179 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f46b8d710 con 0x557f468381f0 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.013+0000 7f99367fc640 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f468381f0 0x557f46c33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.014+0000 7f9916ffd640 10 client.26179 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.014+0000 7f99367fc640 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f468381f0 0x557f46c33d90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79728fe0baf30a4 server_cookie=fb3ea0f02c796adf in_seq=0 out_seq=0 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.016+0000 7f9916ffd640 1 -- 172.21.15.94:0/902138928 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f46b8d710 con 0x557f468381f0 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.016+0000 7f9916ffd640 10 client.26179 handle_command_reply: tid=0 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.016+0000 7f99447c6740 1 client.26179 shutdown 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.016+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 msgr2=0x557f46c264e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.016+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9928009230 tx=0x7f9928031dd0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 shutdown_connections 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f468381f0 0x557f46c33d90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46879410 0x557f46c2d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46858730 0x557f46c26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f9916ffd640 0 client.26179 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 --2- 172.21.15.94:0/902138928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46850550 0x557f46c264e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 >> 172.21.15.94:0/902138928 conn(0x557f4686d560 msgr2=0x557f46c14ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 shutdown_connections 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 1 -- 172.21.15.94:0/902138928 wait complete. 2024-04-25T13:45:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.017+0000 7f99447c6740 20 client.26179 trim_cache size 0 max 16384 2024-04-25T13:45:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 Processor -- start 2024-04-25T13:45:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 -- start start 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 0x557f46c254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46858730 0x557f46c39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99381a9340 con 0x557f46850550 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99381aae80 con 0x557f46858730 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.035+0000 7f99447c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99381a9a10 con 0x557f4686d560 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45520/0 (socket says 172.21.15.94:45520) 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 -- 172.21.15.94:0/3817468664 learned_addr learned my addr 172.21.15.94:0/3817468664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f9936ffd640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 0x557f46c254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99377fe640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46858730 0x557f46c39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 -- 172.21.15.94:0/3817468664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 msgr2=0x557f46c254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 0x557f46c254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 -- 172.21.15.94:0/3817468664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46858730 msgr2=0x557f46c39550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46858730 0x557f46c39550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 -- 172.21.15.94:0/3817468664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f46c27580 con 0x557f46850550 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f9936ffd640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 0x557f46c254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f99367fc640 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c6a0 tx=0x7f992c0dc950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1997f86a2a417dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f991d7fa640 1 -- 172.21.15.94:0/3817468664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c0a6c80 con 0x557f46850550 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f991d7fa640 1 -- 172.21.15.94:0/3817468664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c0a7020 con 0x557f46850550 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.036+0000 7f991d7fa640 1 -- 172.21.15.94:0/3817468664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c0a37a0 con 0x557f46850550 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 -- 172.21.15.94:0/3817468664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 msgr2=0x557f46c19340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c6a0 tx=0x7f992c0dc950 comp rx=0 tx=0).stop 2024-04-25T13:45:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 -- 172.21.15.94:0/3817468664 shutdown_connections 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46858730 0x557f46c39550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c19340 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 --2- 172.21.15.94:0/3817468664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f4686d560 0x557f46c254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 -- 172.21.15.94:0/3817468664 >> 172.21.15.94:0/3817468664 conn(0x557f46c26bf0 msgr2=0x557f46c18c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 -- 172.21.15.94:0/3817468664 shutdown_connections 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.037+0000 7f99447c6740 1 -- 172.21.15.94:0/3817468664 wait complete. 2024-04-25T13:45:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 -- start start 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 Processor -- start 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 0x557f46c2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f4686d560 0x557f46c9a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99381aa7b0 con 0x557f46850550 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99381aa0e0 con 0x557f4686d560 2024-04-25T13:45:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99381a7320 con 0x557f46858730 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99447c6740 10 client.0 _ll_register_callbacks cb 0x7f9928005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99367fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 0x557f46c2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45530/0 (socket says 172.21.15.94:45530) 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 -- 172.21.15.94:0/1211926115 learned_addr learned my addr 172.21.15.94:0/1211926115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f99377fe640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f4686d560 0x557f46c9a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 -- 172.21.15.94:0/1211926115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 msgr2=0x557f46c2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 0x557f46c2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 -- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f4686d560 msgr2=0x557f46c9a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.038+0000 7f9936ffd640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f4686d560 0x557f46c9a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9936ffd640 1 -- 172.21.15.94:0/1211926115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f46c9a770 con 0x557f46850550 2024-04-25T13:45:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f99367fc640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 0x557f46c2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9936ffd640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9928046660 tx=0x7f9928005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe514c0a9d9e2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928035040 con 0x557f46850550 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992c0a7020 con 0x557f46850550 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f9928046b80 con 0x557f46850550 2024-04-25T13:45:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9928060050 con 0x557f46850550 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f99447c6740 10 client.16788 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f46c9bad0 con 0x557f46850550 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c0a7020 con 0x557f46850550 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.039+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9928044050 con 0x557f46850550 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 10 client.16788 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 20 client.16788 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46879410 0x557f46c9e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 4 client.16788 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f46b87580 con 0x557f46879410 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99367fc640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46879410 0x557f46c9e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f9916ffd640 10 client.16788 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.040+0000 7f99367fc640 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46879410 0x557f46c9e4f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b23d5286e89fdb server_cookie=19ed380f62427f3a in_seq=0 out_seq=0 2024-04-25T13:45:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.042+0000 7f9916ffd640 1 -- 172.21.15.94:0/1211926115 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f46b87580 con 0x557f46879410 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.042+0000 7f9916ffd640 10 client.16788 handle_command_reply: tid=0 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.042+0000 7f99447c6740 1 client.16788 shutdown 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 msgr2=0x557f46c2d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9928046660 tx=0x7f9928005fd0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 shutdown_connections 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46879410 0x557f46c9e4f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f4686d560 0x557f46c9a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f9916ffd640 0 client.16788 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46858730 0x557f46c2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 --2- 172.21.15.94:0/1211926115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46850550 0x557f46c2d350 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 >> 172.21.15.94:0/1211926115 conn(0x557f46c26bf0 msgr2=0x557f46c19660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 shutdown_connections 2024-04-25T13:45:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 1 -- 172.21.15.94:0/1211926115 wait complete. 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f99447c6740 20 client.16788 trim_cache size 0 max 16384 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "a0e0bbfe-8d0b-49a9-b462-aa713cbd4ddd", 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:45:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 msgr2=0x7f99200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 0x7f99200b2000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9930020000 tx=0x7f9930021000 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 msgr2=0x7f9938199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.043+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e9f0 tx=0x7f992c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 shutdown_connections 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99200afaf0 0x7f99200b2000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9938104460 0x7f99381a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938103920 0x7f9938199660 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 --2- 172.21.15.94:0/3217078625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938102f40 0x7f9938199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 >> 172.21.15.94:0/3217078625 conn(0x7f99380fd340 msgr2=0x7f9938104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 shutdown_connections 2024-04-25T13:45:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.044+0000 7f993d914640 1 -- 172.21.15.94:0/3217078625 wait complete. 2024-04-25T13:45:08.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:45:08.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:45:08.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 -- 172.21.15.94:0/3570590047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c101e10 msgr2=0x7f5d6c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 --2- 172.21.15.94:0/3570590047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c101e10 0x7f5d6c102220 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009900 tx=0x7f5d5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 -- 172.21.15.94:0/3570590047 shutdown_connections 2024-04-25T13:45:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 --2- 172.21.15.94:0/3570590047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c109780 0x7f5d6c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 --2- 172.21.15.94:0/3570590047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c106c10 0x7f5d6c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 --2- 172.21.15.94:0/3570590047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c101e10 0x7f5d6c102220 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 -- 172.21.15.94:0/3570590047 >> 172.21.15.94:0/3570590047 conn(0x7f5d6c0fd340 msgr2=0x7f5d6c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 -- 172.21.15.94:0/3570590047 shutdown_connections 2024-04-25T13:45:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.191+0000 7f5d70d46640 1 -- 172.21.15.94:0/3570590047 wait complete. 2024-04-25T13:45:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.192+0000 7f5d70d46640 1 Processor -- start 2024-04-25T13:45:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.192+0000 7f5d70d46640 1 -- start start 2024-04-25T13:45:08.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.192+0000 7f5d70d46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.192+0000 7f5d70d46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d70d46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 0x7f5d6c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d70d46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10f5e0 con 0x7f5d6c106c10 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d70d46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10f460 con 0x7f5d6c101e10 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d70d46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10f760 con 0x7f5d6c109780 2024-04-25T13:45:08.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59602/0 (socket says 172.21.15.94:59602) 2024-04-25T13:45:08.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45542/0 (socket says 172.21.15.94:45542) 2024-04-25T13:45:08.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6a59c640 1 -- 172.21.15.94:0/1445848660 learned_addr learned my addr 172.21.15.94:0/1445848660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 0x7f5d6c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 -- 172.21.15.94:0/1445848660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 msgr2=0x7f5d6c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 -- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 msgr2=0x7f5d6c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 -- 172.21.15.94:0/1445848660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c1a1d90 con 0x7f5d6c109780 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.193+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 0x7f5d6c1a1720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009fd0 tx=0x7f5d6400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.194+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/1445848660 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400ce20 con 0x7f5d6c109780 2024-04-25T13:45:08.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.194+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/1445848660 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d64005020 con 0x7f5d6c109780 2024-04-25T13:45:08.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.194+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/1445848660 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64010690 con 0x7f5d6c109780 2024-04-25T13:45:08.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.194+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c1a2080 con 0x7f5d6c109780 2024-04-25T13:45:08.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.194+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c1a2580 con 0x7f5d6c109780 2024-04-25T13:45:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.197+0000 7f5d77bfb740 1 Processor -- start 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.197+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/1445848660 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5d64004b20 con 0x7f5d6c109780 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 -- start start 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f593af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f593bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0f589fd80 con 0x55c0f55704d0 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0f58598b0 con 0x55c0f5599390 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0f589bd40 con 0x55c0f55786b0 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f593af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59604/0 (socket says 172.21.15.94:59604) 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2174469700 learned_addr learned my addr 172.21.15.94:0/2174469700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f593af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45554/0 (socket says 172.21.15.94:45554) 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f593bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.198+0000 7f5d5b7fe640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 0x7f5d540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2174469700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 msgr2=0x55c0f593bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f593bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2174469700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 msgr2=0x55c0f593af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f593af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2174469700 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f582aa40 con 0x55c0f5599390 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/1445848660 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5d640df320 con 0x7f5d6c109780 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 0x7f5d540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d60011bf0 tx=0x7f5d60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e58a0a7ffd9d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d58ff9640 1 -- 172.21.15.94:0/2174469700 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000fc50 con 0x55c0f5599390 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d58ff9640 1 -- 172.21.15.94:0/2174469700 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d6000fdf0 con 0x55c0f5599390 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.199+0000 7f5d58ff9640 1 -- 172.21.15.94:0/2174469700 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000e5b0 con 0x55c0f5599390 2024-04-25T13:45:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2174469700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 msgr2=0x55c0f593a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d60011bf0 tx=0x7f5d60009b80 comp rx=0 tx=0).stop 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 0x7f5d540b2080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d6401a040 tx=0x7f5d64005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2174469700 shutdown_connections 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f593bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f593af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2174469700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5599390 0x55c0f593a990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.200+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2174469700 >> 172.21.15.94:0/2174469700 conn(0x55c0f558d4e0 msgr2=0x55c0f5936270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2174469700 shutdown_connections 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2174469700 wait complete. 2024-04-25T13:45:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- start start 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 Processor -- start 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f594d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f5599390 0x55c0f5954580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0f593c6d0 con 0x55c0f55786b0 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5826d10 con 0x55c0f55704d0 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.201+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0f593c850 con 0x55c0f5599390 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d77bfb740 10 client.0 _ll_register_callbacks cb 0x55c0f5853210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f594d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45556/0 (socket says 172.21.15.94:45556) 2024-04-25T13:45:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 -- 172.21.15.94:0/2013818709 learned_addr learned my addr 172.21.15.94:0/2013818709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f5599390 0x55c0f5954580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 -- 172.21.15.94:0/2013818709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f5599390 msgr2=0x55c0f5954580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f5599390 0x55c0f5954580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 -- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 msgr2=0x55c0f594d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f594d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 -- 172.21.15.94:0/2013818709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f567b7d0 con 0x55c0f55786b0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005ab0 tx=0x7f5d5c0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0705f9661691bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c035040 con 0x55c0f55786b0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d6000fdf0 con 0x55c0f55786b0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5d5c004450 con 0x55c0f55786b0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c031900 con 0x55c0f55786b0 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.202+0000 7f5d77bfb740 10 client.16800 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:08.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0f55b13f0 con 0x55c0f55786b0 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d6000fdf0 con 0x55c0f55786b0 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d5c044070 con 0x55c0f55786b0 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 10 client.16800 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 20 client.16800 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5557f60 0x55c0f595aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 4 client.16800 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f58ad990 con 0x55c0f5557f60 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.203+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5557f60 0x55c0f595aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.204+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5557f60 0x55c0f595aea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250e121ae9557477 server_cookie=2c663c4d266a8cc in_seq=0 out_seq=0 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.204+0000 7f5d51ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d51ffb640 1 -- 172.21.15.94:0/2013818709 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0f58ad990 con 0x55c0f5557f60 2024-04-25T13:45:08.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d51ffb640 10 client.16800 handle_command_reply: tid=0 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 client.16800 shutdown 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 msgr2=0x55c0f594d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005ab0 tx=0x7f5d5c0317d0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 shutdown_connections 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5557f60 0x55c0f595aea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f5599390 0x55c0f5954580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d51ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594d620 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2013818709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f594d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 >> 172.21.15.94:0/2013818709 conn(0x55c0f558d4e0 msgr2=0x55c0f5939180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 shutdown_connections 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2013818709 wait complete. 2024-04-25T13:45:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.205+0000 7f5d77bfb740 20 client.16800 trim_cache size 0 max 16384 2024-04-25T13:45:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 Processor -- start 2024-04-25T13:45:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 -- start start 2024-04-25T13:45:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f558d4e0 0x55c0f5947fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f5948500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a9270 con 0x55c0f55786b0 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aadb0 con 0x55c0f55704d0 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a9940 con 0x55c0f558d4e0 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45566/0 (socket says 172.21.15.94:45566) 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2246842771 learned_addr learned my addr 172.21.15.94:0/2246842771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f5948500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.225+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f558d4e0 0x55c0f5947fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2246842771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f558d4e0 msgr2=0x55c0f5947fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f558d4e0 0x55c0f5947fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2246842771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 msgr2=0x55c0f5948500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f5948500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 -- 172.21.15.94:0/2246842771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f55710f0 con 0x55c0f55786b0 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009b50 tx=0x7f5d600117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2e01d4a63f53c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d53fff640 1 -- 172.21.15.94:0/2246842771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000fd80 con 0x55c0f55786b0 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d53fff640 1 -- 172.21.15.94:0/2246842771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d600118b0 con 0x55c0f55786b0 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.226+0000 7f5d53fff640 1 -- 172.21.15.94:0/2246842771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003aa0 con 0x55c0f55786b0 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2246842771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 msgr2=0x55c0f594b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009b50 tx=0x7f5d600117a0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2246842771 shutdown_connections 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55786b0 0x55c0f594b700 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f55704d0 0x55c0f5948500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/2246842771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f558d4e0 0x55c0f5947fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2246842771 >> 172.21.15.94:0/2246842771 conn(0x55c0f5936380 msgr2=0x55c0f594d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2246842771 shutdown_connections 2024-04-25T13:45:08.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- 172.21.15.94:0/2246842771 wait complete. 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 -- start start 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.227+0000 7f5d77bfb740 1 Processor -- start 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f5954c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f558d4e0 0x55c0f594fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aa6e0 con 0x55c0f55704d0 2024-04-25T13:45:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aa010 con 0x55c0f558d4e0 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a7250 con 0x55c0f55786b0 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d77bfb740 10 client.0 _ll_register_callbacks cb 0x7f5d5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45582/0 (socket says 172.21.15.94:45582) 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 -- 172.21.15.94:0/3513756322 learned_addr learned my addr 172.21.15.94:0/3513756322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f5954c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f558d4e0 0x55c0f594fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 -- 172.21.15.94:0/3513756322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 msgr2=0x55c0f5954c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f5954c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 -- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f558d4e0 msgr2=0x55c0f594fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f558d4e0 0x55c0f594fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 -- 172.21.15.94:0/3513756322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f58604b0 con 0x55c0f55704d0 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000a010 tx=0x7f5d60005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9582da792da1b887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.228+0000 7f5d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6003f060 con 0x55c0f55704d0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5d600035f0 con 0x55c0f55704d0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d600118b0 con 0x55c0f55704d0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d600037f0 con 0x55c0f55704d0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 10 client.16806 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c0f59c2150 con 0x55c0f55704d0 2024-04-25T13:45:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d600118b0 con 0x55c0f55704d0 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d6001d020 con 0x55c0f55704d0 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 10 client.16806 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 20 client.16806 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5599390 0x55c0f59c4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 4 client.16806 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:45:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.229+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0f58a7800 con 0x55c0f5599390 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5599390 0x55c0f59c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d51ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5599390 0x55c0f59c4870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6cbbca3b8901f9 server_cookie=191eb431f91607df in_seq=0 out_seq=0 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3513756322 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c0f58a7800 con 0x55c0f5599390 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d51ffb640 10 client.16806 handle_command_reply: tid=0 2024-04-25T13:45:08.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 client.16806 shutdown 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 msgr2=0x55c0f594f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000a010 tx=0x7f5d60005850 comp rx=0 tx=0).stop 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 shutdown_connections 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0f5599390 0x55c0f59c4870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f558d4e0 0x55c0f594fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d51ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0f55786b0 0x55c0f5954c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 --2- 172.21.15.94:0/3513756322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0f55704d0 0x55c0f594f7a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 >> 172.21.15.94:0/3513756322 conn(0x55c0f5936380 msgr2=0x55c0f594d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 shutdown_connections 2024-04-25T13:45:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.230+0000 7f5d77bfb740 1 -- 172.21.15.94:0/3513756322 wait complete. 2024-04-25T13:45:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d77bfb740 20 client.16806 trim_cache size 0 max 16384 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "a0e0bbfe-8d0b-49a9-b462-aa713cbd4ddd": { 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "a0e0bbfe-8d0b-49a9-b462-aa713cbd4ddd", 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:45:08.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:45:08.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:45:08.248 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 msgr2=0x7f5d540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 0x7f5d540b2080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d6401a040 tx=0x7f5d64005f70 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 msgr2=0x7f5d6c1a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 0x7f5d6c1a1720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009fd0 tx=0x7f5d6400ef10 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 shutdown_connections 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb70 0x7f5d540b2080 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c109780 0x7f5d6c1a1720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c106c10 0x7f5d6c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 --2- 172.21.15.94:0/1445848660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c101e10 0x7f5d6c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 >> 172.21.15.94:0/1445848660 conn(0x7f5d6c0fd340 msgr2=0x7f5d6c108d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:08.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 shutdown_connections 2024-04-25T13:45:08.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:08.231+0000 7f5d70d46640 1 -- 172.21.15.94:0/1445848660 wait complete. 2024-04-25T13:45:08.251 INFO:tasks.cephfs.filesystem:scrub status for tag:a0e0bbfe-8d0b-49a9-b462-aa713cbd4ddd - {'path': '/', 'tag': 'a0e0bbfe-8d0b-49a9-b462-aa713cbd4ddd', 'options': 'recursive,force'} 2024-04-25T13:45:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:08 smithi137 ceph-mon[32568]: pgmap v446: 97 pgs: 97 active+clean; 48 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-04-25T13:45:08.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:08 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:45:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:08 smithi094 ceph-mon[29195]: pgmap v446: 97 pgs: 97 active+clean; 48 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-04-25T13:45:08.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:08 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:45:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:08 smithi138 ceph-mon[32977]: pgmap v446: 97 pgs: 97 active+clean; 48 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 326 op/s 2024-04-25T13:45:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:08 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:45:08.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 286562 14.07 MB/sec warmup 111 sec latency 37.121 ms 2024-04-25T13:45:09.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:45:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:45:09.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:45:09.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:45:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:45:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:45:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:45:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:45:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:45:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:45:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:45:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:45:09.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 288780 14.06 MB/sec warmup 112 sec latency 15.297 ms 2024-04-25T13:45:10.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:10 smithi094 ceph-mon[29195]: pgmap v447: 97 pgs: 97 active+clean; 48 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:45:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:10 smithi137 ceph-mon[32568]: pgmap v447: 97 pgs: 97 active+clean; 48 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:45:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:10 smithi138 ceph-mon[32977]: pgmap v447: 97 pgs: 97 active+clean; 48 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:45:10.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 291270 14.06 MB/sec warmup 113 sec latency 62.746 ms 2024-04-25T13:45:11.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:11 smithi137 ceph-mon[32568]: pgmap v448: 97 pgs: 97 active+clean; 38 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-04-25T13:45:11.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:45:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:11 smithi094 ceph-mon[29195]: pgmap v448: 97 pgs: 97 active+clean; 38 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-04-25T13:45:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:45:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:11 smithi138 ceph-mon[32977]: pgmap v448: 97 pgs: 97 active+clean; 38 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-04-25T13:45:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:45:11.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 293896 14.05 MB/sec warmup 114 sec latency 18.802 ms 2024-04-25T13:45:12.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 296111 14.03 MB/sec warmup 115 sec latency 15.755 ms 2024-04-25T13:45:13.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:13 smithi094 ceph-mon[29195]: pgmap v449: 97 pgs: 97 active+clean; 38 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-04-25T13:45:13.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:13 smithi138 ceph-mon[32977]: pgmap v449: 97 pgs: 97 active+clean; 38 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-04-25T13:45:13.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 298696 14.03 MB/sec warmup 116 sec latency 16.189 ms 2024-04-25T13:45:13.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:13 smithi137 ceph-mon[32568]: pgmap v449: 97 pgs: 97 active+clean; 38 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-04-25T13:45:14.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 301764 14.07 MB/sec warmup 117 sec latency 13.457 ms 2024-04-25T13:45:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:15 smithi094 ceph-mon[29195]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:45:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:15 smithi138 ceph-mon[32977]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:45:15.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 303868 14.01 MB/sec warmup 118 sec latency 88.685 ms 2024-04-25T13:45:15.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:15 smithi137 ceph-mon[32568]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:45:16.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 305856 14.00 MB/sec warmup 119 sec latency 14.950 ms 2024-04-25T13:45:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:17 smithi094 ceph-mon[29195]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-04-25T13:45:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:17 smithi138 ceph-mon[32977]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-04-25T13:45:17.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:17 smithi137 ceph-mon[32568]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-04-25T13:45:18.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 311373 11.31 MB/sec execute 1 sec latency 18.159 ms 2024-04-25T13:45:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:19 smithi094 ceph-mon[29195]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-25T13:45:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:19 smithi138 ceph-mon[32977]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-25T13:45:19.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 313414 10.78 MB/sec execute 2 sec latency 18.785 ms 2024-04-25T13:45:19.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:19 smithi137 ceph-mon[32568]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-25T13:45:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- 172.21.15.94:0/1783625344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c108960 msgr2=0x7fa10c10ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 --2- 172.21.15.94:0/1783625344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c108960 0x7fa10c10ad80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa104009ca0 tx=0x7fa10402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- 172.21.15.94:0/1783625344 shutdown_connections 2024-04-25T13:45:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 --2- 172.21.15.94:0/1783625344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c108960 0x7fa10c10ad80 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 --2- 172.21.15.94:0/1783625344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 0x7fa10c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 --2- 172.21.15.94:0/1783625344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c102a20 0x7fa10c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- 172.21.15.94:0/1783625344 >> 172.21.15.94:0/1783625344 conn(0x7fa10c0fdc10 msgr2=0x7fa10c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- 172.21.15.94:0/1783625344 shutdown_connections 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- 172.21.15.94:0/1783625344 wait complete. 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 Processor -- start 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.222+0000 7fa112bd7640 1 -- start start 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 0x7fa10c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c108960 0x7fa10c1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10e550 con 0x7fa10c102a20 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10e3d0 con 0x7fa10c108960 2024-04-25T13:45:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10e6d0 con 0x7fa10c103c60 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45592/0 (socket says 172.21.15.94:45592) 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 -- 172.21.15.94:0/76612607 learned_addr learned my addr 172.21.15.94:0/76612607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa10bfff640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 0x7fa10c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa111174640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c108960 0x7fa10c1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 -- 172.21.15.94:0/76612607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 msgr2=0x7fa10c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 0x7fa10c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 -- 172.21.15.94:0/76612607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c108960 msgr2=0x7fa10c1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c108960 0x7fa10c1a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 -- 172.21.15.94:0/76612607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c1a3f80 con 0x7fa10c102a20 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa110973640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa109ffb640 1 -- 172.21.15.94:0/76612607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000ccc0 con 0x7fa10c102a20 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa109ffb640 1 -- 172.21.15.94:0/76612607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1000035f0 con 0x7fa10c102a20 2024-04-25T13:45:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.223+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c1a4270 con 0x7fa10c102a20 2024-04-25T13:45:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.224+0000 7fa109ffb640 1 -- 172.21.15.94:0/76612607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000ce60 con 0x7fa10c102a20 2024-04-25T13:45:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.224+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a4810 con 0x7fa10c102a20 2024-04-25T13:45:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa109ffb640 1 -- 172.21.15.94:0/76612607 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa100003aa0 con 0x7fa10c102a20 2024-04-25T13:45:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 Processor -- start 2024-04-25T13:45:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 -- start start 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75db1b0 0x55bae797b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75b22f0 0x55bae797bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bae78e1460 con 0x55bae75ba4d0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bae789b590 con 0x55bae75b22f0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.226+0000 7fa119b65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bae78dd420 con 0x55bae75db1b0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa109ffb640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 0x7fa0fc0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa111174640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75db1b0 0x55bae797b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45598/0 (socket says 172.21.15.94:45598) 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa111174640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 0x7fa0fc0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 -- 172.21.15.94:0/2027044031 learned_addr learned my addr 172.21.15.94:0/2027044031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa110973640 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75b22f0 0x55bae797bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa109ffb640 1 -- 172.21.15.94:0/76612607 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa1000def40 con 0x7fa10c102a20 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 -- 172.21.15.94:0/2027044031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75db1b0 msgr2=0x55bae797b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75db1b0 0x55bae797b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 -- 172.21.15.94:0/2027044031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75b22f0 msgr2=0x55bae797bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75b22f0 0x55bae797bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 -- 172.21.15.94:0/2027044031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae78a80d0 con 0x55bae75ba4d0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa111174640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 0x7fa0fc0b2070 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa10402f500 tx=0x7fa104004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa10bfff640 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009ca0 tx=0x7fa0f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe8a971c7f9c71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa0f6ffd640 1 -- 172.21.15.94:0/2027044031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8005220 con 0x55bae75ba4d0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa0f6ffd640 1 -- 172.21.15.94:0/2027044031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f8004c50 con 0x55bae75ba4d0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.227+0000 7fa0f6ffd640 1 -- 172.21.15.94:0/2027044031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80186c0 con 0x55bae75ba4d0 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 -- 172.21.15.94:0/2027044031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 msgr2=0x55bae7982df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009ca0 tx=0x7fa0f8011810 comp rx=0 tx=0).stop 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 -- 172.21.15.94:0/2027044031 shutdown_connections 2024-04-25T13:45:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7982df0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75b22f0 0x55bae797bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 --2- 172.21.15.94:0/2027044031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75db1b0 0x55bae797b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 -- 172.21.15.94:0/2027044031 >> 172.21.15.94:0/2027044031 conn(0x55bae75cf300 msgr2=0x55bae7977300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 -- 172.21.15.94:0/2027044031 shutdown_connections 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.228+0000 7fa119b65740 1 -- 172.21.15.94:0/2027044031 wait complete. 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 -- start start 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 Processor -- start 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae79855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75db1b0 0x55bae798ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bae7983840 con 0x55bae75ba4d0 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bae7868b60 con 0x55bae75db1b0 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bae79839c0 con 0x55bae75b22f0 2024-04-25T13:45:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa119b65740 10 client.0 _ll_register_callbacks cb 0x55bae788d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45602/0 (socket says 172.21.15.94:45602) 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa10bfff640 1 -- 172.21.15.94:0/1365935492 learned_addr learned my addr 172.21.15.94:0/1365935492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa110973640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae79855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.229+0000 7fa111174640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75db1b0 0x55bae798ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 -- 172.21.15.94:0/1365935492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 msgr2=0x55bae79855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae79855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 -- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75db1b0 msgr2=0x55bae798ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75db1b0 0x55bae798ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 -- 172.21.15.94:0/1365935492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae75f3210 con 0x55bae75ba4d0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa10bfff640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80050b0 tx=0x7fa0f80050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80414ca4fd1aa426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa0ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8028020 con 0x55bae75ba4d0 2024-04-25T13:45:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa0f8024470 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f8004c50 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0f8033050 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa119b65740 10 client.16818 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bae76bf620 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.230+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f8004c50 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa0f8018560 con 0x55bae75ba4d0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 10 client.16818 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 20 client.16818 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae7599d80 0x55bae7993110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 4 client.16818 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bae78ef070 con 0x55bae7599d80 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa110973640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae7599d80 0x55bae7993110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa110973640 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae7599d80 0x55bae7993110 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4959226a50618279 server_cookie=ff2d3fbe6ab6c073 in_seq=0 out_seq=0 2024-04-25T13:45:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.231+0000 7fa0ebfff640 10 client.16818 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa0ebfff640 1 -- 172.21.15.94:0/1365935492 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bae78ef070 con 0x55bae7599d80 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa0ebfff640 10 client.16818 handle_command_reply: tid=0 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 client.16818 shutdown 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 msgr2=0x55bae7985b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80050b0 tx=0x7fa0f80050e0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 shutdown_connections 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae7599d80 0x55bae7993110 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae75db1b0 0x55bae798ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7985b20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 --2- 172.21.15.94:0/1365935492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae79855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa0ebfff640 0 client.16818 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 >> 172.21.15.94:0/1365935492 conn(0x55bae75cf300 msgr2=0x55bae797a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 shutdown_connections 2024-04-25T13:45:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.232+0000 7fa119b65740 1 -- 172.21.15.94:0/1365935492 wait complete. 2024-04-25T13:45:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.233+0000 7fa119b65740 20 client.16818 trim_cache size 0 max 16384 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 Processor -- start 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 -- start start 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae7984470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ad7c0 con 0x55bae75ba4d0 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1af300 con 0x55bae7599d80 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa119b65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ade90 con 0x55bae75b22f0 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa111174640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa111174640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45612/0 (socket says 172.21.15.94:45612) 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53558/0 (socket says 172.21.15.94:53558) 2024-04-25T13:45:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa110973640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae7984470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.248+0000 7fa111174640 1 -- 172.21.15.94:0/517659485 learned_addr learned my addr 172.21.15.94:0/517659485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 -- 172.21.15.94:0/517659485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 msgr2=0x55bae7984470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae7984470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 -- 172.21.15.94:0/517659485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 msgr2=0x55bae7998910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 -- 172.21.15.94:0/517659485 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae798cee0 con 0x55bae75b22f0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa111174640 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa10bfff640 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80189d0 tx=0x7fa0f8011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96501a47709c94e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa0f67fc640 1 -- 172.21.15.94:0/517659485 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8028040 con 0x55bae75b22f0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa0f67fc640 1 -- 172.21.15.94:0/517659485 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f8004a80 con 0x55bae75b22f0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.249+0000 7fa0f67fc640 1 -- 172.21.15.94:0/517659485 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f801e040 con 0x55bae75b22f0 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- 172.21.15.94:0/517659485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 msgr2=0x55bae798c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80189d0 tx=0x7fa0f8011d50 comp rx=0 tx=0).stop 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- 172.21.15.94:0/517659485 shutdown_connections 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae7998910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 --2- 172.21.15.94:0/517659485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae7984470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- 172.21.15.94:0/517659485 >> 172.21.15.94:0/517659485 conn(0x55bae75cf300 msgr2=0x55bae7983d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- 172.21.15.94:0/517659485 shutdown_connections 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- 172.21.15.94:0/517659485 wait complete. 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 -- start start 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.250+0000 7fa119b65740 1 Processor -- start 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae798bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae79f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1aec30 con 0x55bae75ba4d0 2024-04-25T13:45:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ae560 con 0x55bae7599d80 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c1ab7a0 con 0x55bae75b22f0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa119b65740 10 client.0 _ll_register_callbacks cb 0x7fa1000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae79f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa110973640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae798bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53562/0 (socket says 172.21.15.94:53562) 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa10bfff640 1 -- 172.21.15.94:0/2946917640 learned_addr learned my addr 172.21.15.94:0/2946917640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 -- 172.21.15.94:0/2946917640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 msgr2=0x55bae798c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 -- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 msgr2=0x55bae798bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae798bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 -- 172.21.15.94:0/2946917640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae79f9c70 con 0x55bae75ba4d0 2024-04-25T13:45:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa111174640 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae79f9680 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa10402ade0 tx=0x7fa104049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae377694bdb90451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa0ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.251+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10405b060 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa104004450 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f8004a80 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa104042050 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 10 client.16824 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bae79faf50 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f8004a80 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa1040305c0 con 0x55bae75ba4d0 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 10 client.16824 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 20 client.16824 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae75db1b0 0x55bae79fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 4 client.16824 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:45:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bae78e8ee0 con 0x55bae75db1b0 2024-04-25T13:45:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.252+0000 7fa110973640 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae75db1b0 0x55bae79fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.253+0000 7fa0ebfff640 10 client.16824 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.253+0000 7fa110973640 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae75db1b0 0x55bae79fd970 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5fd13aa1c3b7b99 server_cookie=ca1ed838b592c9a5 in_seq=0 out_seq=0 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa0ebfff640 1 -- 172.21.15.94:0/2946917640 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bae78e8ee0 con 0x55bae75db1b0 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa0ebfff640 10 client.16824 handle_command_reply: tid=0 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 client.16824 shutdown 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 msgr2=0x55bae79f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae79f9680 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa10402ade0 tx=0x7fa104049000 comp rx=0 tx=0).stop 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 shutdown_connections 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bae75db1b0 0x55bae79fd970 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bae75ba4d0 0x55bae79f9680 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa0ebfff640 0 client.16824 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bae75b22f0 0x55bae798c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 --2- 172.21.15.94:0/2946917640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bae7599d80 0x55bae798bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 >> 172.21.15.94:0/2946917640 conn(0x55bae75cf300 msgr2=0x55bae797a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 shutdown_connections 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 1 -- 172.21.15.94:0/2946917640 wait complete. 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.398+0000 7fa119b65740 20 client.16824 trim_cache size 0 max 16384 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:45:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 msgr2=0x7fa0fc0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 0x7fa0fc0b2070 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa10402f500 tx=0x7fa104004ab0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 msgr2=0x7fa10c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 shutdown_connections 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0fc0afb60 0x7fa0fc0b2070 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa10c108960 0x7fa10c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10c103c60 0x7fa10c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 --2- 172.21.15.94:0/76612607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa10c102a20 0x7fa10c19c440 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 >> 172.21.15.94:0/76612607 conn(0x7fa10c0fdc10 msgr2=0x7fa10c100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 shutdown_connections 2024-04-25T13:45:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:20.399+0000 7fa112bd7640 1 -- 172.21.15.94:0/76612607 wait complete. 2024-04-25T13:45:20.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 315846 12.42 MB/sec execute 3 sec latency 103.395 ms 2024-04-25T13:45:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:21 smithi094 ceph-mon[29195]: pgmap v453: 97 pgs: 97 active+clean; 46 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-04-25T13:45:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:21 smithi138 ceph-mon[32977]: pgmap v453: 97 pgs: 97 active+clean; 46 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-04-25T13:45:21.793 INFO:tasks.workunit.client.0.smithi094.stdout: 1 318105 11.39 MB/sec execute 4 sec latency 28.571 ms 2024-04-25T13:45:21.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:21 smithi137 ceph-mon[32568]: pgmap v453: 97 pgs: 97 active+clean; 46 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-04-25T13:45:22.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 320130 11.67 MB/sec execute 5 sec latency 13.601 ms 2024-04-25T13:45:23.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:23 smithi137 ceph-mon[32568]: pgmap v454: 97 pgs: 97 active+clean; 37 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-25T13:45:23.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:23 smithi094 ceph-mon[29195]: pgmap v454: 97 pgs: 97 active+clean; 37 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-25T13:45:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:23 smithi138 ceph-mon[32977]: pgmap v454: 97 pgs: 97 active+clean; 37 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-25T13:45:23.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 323091 12.75 MB/sec execute 6 sec latency 25.163 ms 2024-04-25T13:45:24.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 325770 12.76 MB/sec execute 7 sec latency 11.539 ms 2024-04-25T13:45:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:25 smithi094 ceph-mon[29195]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:45:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:25 smithi138 ceph-mon[32977]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:45:25.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 327762 12.62 MB/sec execute 8 sec latency 16.170 ms 2024-04-25T13:45:25.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:25 smithi137 ceph-mon[32568]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:45:26.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 330204 12.75 MB/sec execute 9 sec latency 66.475 ms 2024-04-25T13:45:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:27 smithi094 ceph-mon[29195]: pgmap v456: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-04-25T13:45:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:27 smithi138 ceph-mon[32977]: pgmap v456: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-04-25T13:45:27.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 333095 12.84 MB/sec execute 10 sec latency 11.342 ms 2024-04-25T13:45:27.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:27 smithi137 ceph-mon[32568]: pgmap v456: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 422 op/s 2024-04-25T13:45:28.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 335336 12.69 MB/sec execute 11 sec latency 13.615 ms 2024-04-25T13:45:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:29 smithi094 ceph-mon[29195]: pgmap v457: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:45:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:29 smithi138 ceph-mon[32977]: pgmap v457: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:45:29.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 337848 12.82 MB/sec execute 12 sec latency 12.533 ms 2024-04-25T13:45:29.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:29 smithi137 ceph-mon[32568]: pgmap v457: 97 pgs: 97 active+clean; 43 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:45:30.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 340833 13.24 MB/sec execute 13 sec latency 13.362 ms 2024-04-25T13:45:31.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:31 smithi094 ceph-mon[29195]: pgmap v458: 97 pgs: 97 active+clean; 46 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-04-25T13:45:31.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:31 smithi138 ceph-mon[32977]: pgmap v458: 97 pgs: 97 active+clean; 46 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-04-25T13:45:31.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 343003 12.87 MB/sec execute 14 sec latency 63.056 ms 2024-04-25T13:45:31.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:31 smithi137 ceph-mon[32568]: pgmap v458: 97 pgs: 97 active+clean; 46 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-04-25T13:45:32.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 345246 12.90 MB/sec execute 15 sec latency 11.575 ms 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 -- 172.21.15.94:0/32457684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428103c40 msgr2=0x7fe428104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 --2- 172.21.15.94:0/32457684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428103c40 0x7fe428104050 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe418009900 tx=0x7fe41802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 -- 172.21.15.94:0/32457684 shutdown_connections 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 --2- 172.21.15.94:0/32457684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 0x7fe42810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 --2- 172.21.15.94:0/32457684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428103c40 0x7fe428104050 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 --2- 172.21.15.94:0/32457684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428102a00 0x7fe428103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 -- 172.21.15.94:0/32457684 >> 172.21.15.94:0/32457684 conn(0x7fe4280fdbd0 msgr2=0x7fe4281000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 -- 172.21.15.94:0/32457684 shutdown_connections 2024-04-25T13:45:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.504+0000 7fe42f449640 1 -- 172.21.15.94:0/32457684 wait complete. 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 Processor -- start 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 -- start start 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428103c40 0x7fe428199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 0x7fe4281a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810e500 con 0x7fe428102a00 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810e380 con 0x7fe428108940 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42f449640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810e680 con 0x7fe428103c40 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42d1e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42d1e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44584/0 (socket says 172.21.15.94:44584) 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.505+0000 7fe42d1e5640 1 -- 172.21.15.94:0/4243592360 learned_addr learned my addr 172.21.15.94:0/4243592360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 0x7fe4281a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428103c40 0x7fe428199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 -- 172.21.15.94:0/4243592360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428103c40 msgr2=0x7fe428199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428103c40 0x7fe428199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 -- 172.21.15.94:0/4243592360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 msgr2=0x7fe4281a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 0x7fe4281a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 -- 172.21.15.94:0/4243592360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4281a0c70 con 0x7fe428102a00 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe41e7fc640 1 -- 172.21.15.94:0/4243592360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42400ccc0 con 0x7fe428102a00 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe41e7fc640 1 -- 172.21.15.94:0/4243592360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4240035f0 con 0x7fe428102a00 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4281a0f60 con 0x7fe428102a00 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe41e7fc640 1 -- 172.21.15.94:0/4243592360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42400ce60 con 0x7fe428102a00 2024-04-25T13:45:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.506+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4281a13c0 con 0x7fe428102a00 2024-04-25T13:45:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe41e7fc640 1 -- 172.21.15.94:0/4243592360 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe424003aa0 con 0x7fe428102a00 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 Processor -- start 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 -- start start 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 0x563490687100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x56349068e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5634905ea460 con 0x5634902e41b0 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5634905a4590 con 0x5634902c34d0 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe4363d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634905e6420 con 0x5634902bb2f0 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44600/0 (socket says 172.21.15.94:44600) 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42c9e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x56349068e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 -- 172.21.15.94:0/2988710835 learned_addr learned my addr 172.21.15.94:0/2988710835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe41e7fc640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 0x7fe4140b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 0x563490687100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 -- 172.21.15.94:0/2988710835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 msgr2=0x563490687100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 0x563490687100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 -- 172.21.15.94:0/2988710835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 msgr2=0x56349068e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x56349068e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.509+0000 7fe42d9e6640 1 -- 172.21.15.94:0/2988710835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634905b10d0 con 0x5634902e41b0 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x56349068e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 0x563490687100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 0x7fe4140b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe41e7fc640 1 -- 172.21.15.94:0/4243592360 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe4240df040 con 0x7fe428102a00 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe420011590 tx=0x7fe420011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afca628a60f01d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe4137fe640 1 -- 172.21.15.94:0/2988710835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420017050 con 0x5634902e41b0 2024-04-25T13:45:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe4137fe640 1 -- 172.21.15.94:0/2988710835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42000fcf0 con 0x5634902e41b0 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.510+0000 7fe4137fe640 1 -- 172.21.15.94:0/2988710835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42001b940 con 0x5634902e41b0 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- 172.21.15.94:0/2988710835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 msgr2=0x563490686b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 0x7fe4140b1ff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe420022000 tx=0x7fe420023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe420011590 tx=0x7fe420011a50 comp rx=0 tx=0).stop 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- 172.21.15.94:0/2988710835 shutdown_connections 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x56349068e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902bb2f0 0x563490687100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2988710835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490686b80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- 172.21.15.94:0/2988710835 >> 172.21.15.94:0/2988710835 conn(0x5634902d8300 msgr2=0x563490680300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- 172.21.15.94:0/2988710835 shutdown_connections 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- 172.21.15.94:0/2988710835 wait complete. 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 -- start start 2024-04-25T13:45:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.511+0000 7fe4363d1740 1 Processor -- start 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x563490690890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902c34d0 0x563490690dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56349068ea50 con 0x5634902e41b0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563490571b60 con 0x5634902bb2f0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56349068ebd0 con 0x5634902c34d0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe4363d1740 10 client.0 _ll_register_callbacks cb 0x5634905962b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44606/0 (socket says 172.21.15.94:44606) 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 -- 172.21.15.94:0/1968708099 learned_addr learned my addr 172.21.15.94:0/1968708099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x563490690890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902c34d0 0x563490690dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 -- 172.21.15.94:0/1968708099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902c34d0 msgr2=0x563490690dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902c34d0 0x563490690dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 -- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 msgr2=0x563490690890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x563490690890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 -- 172.21.15.94:0/1968708099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634902fc210 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe420011f50 tx=0x7fe42001bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8813d3e47b2b84ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe410ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420004bf0 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.512+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe420004d90 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe42000fcf0 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe42000f640 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 10 client.16836 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634903c8620 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42000fcf0 con 0x5634902e41b0 2024-04-25T13:45:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe42001a070 con 0x5634902e41b0 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 10 client.16836 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 20 client.16836 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902a2d80 0x56349069e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 4 client.16836 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634905f8070 con 0x5634902a2d80 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.513+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902a2d80 0x56349069e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.514+0000 7fe410ff9640 10 client.16836 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.514+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902a2d80 0x56349069e300 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e207c5f1508d44a6 server_cookie=15b22a2ff11719c8 in_seq=0 out_seq=0 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe410ff9640 1 -- 172.21.15.94:0/1968708099 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634905f8070 con 0x5634902a2d80 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe410ff9640 10 client.16836 handle_command_reply: tid=0 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 client.16836 shutdown 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 msgr2=0x563490697d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe420011f50 tx=0x7fe42001bf50 comp rx=0 tx=0).stop 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 shutdown_connections 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902a2d80 0x56349069e300 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902e41b0 0x563490697d30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902c34d0 0x563490690dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe410ff9640 0 client.16836 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 --2- 172.21.15.94:0/1968708099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x563490690890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 >> 172.21.15.94:0/1968708099 conn(0x5634902d8300 msgr2=0x563490683210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 shutdown_connections 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 1 -- 172.21.15.94:0/1968708099 wait complete. 2024-04-25T13:45:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.515+0000 7fe4363d1740 20 client.16836 trim_cache size 0 max 16384 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 Processor -- start 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 -- start start 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x5634906a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281a9460 con 0x5634902c34d0 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281aafa0 con 0x5634902bb2f0 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe4363d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281a9b30 con 0x5634902a2d80 2024-04-25T13:45:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d9e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44612/0 (socket says 172.21.15.94:44612) 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d9e6640 1 -- 172.21.15.94:0/2498874514 learned_addr learned my addr 172.21.15.94:0/2498874514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x5634906a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2498874514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 msgr2=0x5634906a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x5634906a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2498874514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 msgr2=0x563490697c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2498874514 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563490698190 con 0x5634902a2d80 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.531+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906a3ae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4240dca70 tx=0x7fe4240ab040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffeb5f8e6bf36356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe412ffd640 1 -- 172.21.15.94:0/2498874514 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240a7b80 con 0x5634902a2d80 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe412ffd640 1 -- 172.21.15.94:0/2498874514 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4240a7480 con 0x5634902a2d80 2024-04-25T13:45:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe412ffd640 1 -- 172.21.15.94:0/2498874514 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240e10f0 con 0x5634902a2d80 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 -- 172.21.15.94:0/2498874514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 msgr2=0x5634906a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906a3ae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4240dca70 tx=0x7fe4240ab040 comp rx=0 tx=0).stop 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 -- 172.21.15.94:0/2498874514 shutdown_connections 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902c34d0 0x563490697c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902bb2f0 0x5634906a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2498874514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906a3ae0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.532+0000 7fe4363d1740 1 -- 172.21.15.94:0/2498874514 >> 172.21.15.94:0/2498874514 conn(0x5634902d8300 msgr2=0x56349068ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- 172.21.15.94:0/2498874514 shutdown_connections 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- 172.21.15.94:0/2498874514 wait complete. 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- start start 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 Processor -- start 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x563490696530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281aa8d0 con 0x5634902bb2f0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281aa200 con 0x5634902c34d0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281a7440 con 0x5634902a2d80 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe4363d1740 10 client.0 _ll_register_callbacks cb 0x7fe4200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42c9e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42d1e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42d1e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52000/0 (socket says 172.21.15.94:52000) 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2022601543 learned_addr learned my addr 172.21.15.94:0/2022601543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44616/0 (socket says 172.21.15.94:44616) 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.533+0000 7fe42d9e6640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x563490696530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 msgr2=0x563490696530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x563490696530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2022601543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 msgr2=0x563490695600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 -- 172.21.15.94:0/2022601543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563490696af0 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe42d1e5640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4240dcaa0 tx=0x7fe424003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c7e4f9abf6305d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe410ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240bb020 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe4240e12f0 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4240a7480 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe4240ab830 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe4363d1740 10 client.26239 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563490697d80 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.534+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4240a7480 con 0x5634902a2d80 2024-04-25T13:45:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe424003fd0 con 0x5634902a2d80 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 10 client.26239 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 20 client.26239 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902e41b0 0x563490708c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 4 client.26239 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634905f1ee0 con 0x5634902e41b0 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902e41b0 0x563490708c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe410ff9640 10 client.26239 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.535+0000 7fe42c9e4640 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902e41b0 0x563490708c90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e320fc158ad5d3 server_cookie=816e9ac1032bcf19 in_seq=0 out_seq=0 2024-04-25T13:45:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe410ff9640 1 -- 172.21.15.94:0/2022601543 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634905f1ee0 con 0x5634902e41b0 2024-04-25T13:45:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe410ff9640 10 client.26239 handle_command_reply: tid=0 2024-04-25T13:45:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe4363d1740 1 client.26239 shutdown 2024-04-25T13:45:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 msgr2=0x5634906950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4240dcaa0 tx=0x7fe424003e70 comp rx=0 tx=0).stop 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 shutdown_connections 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.677+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634902e41b0 0x563490708c90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634902c34d0 0x563490696530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe410ff9640 0 client.26239 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634902bb2f0 0x563490695600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 --2- 172.21.15.94:0/2022601543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634902a2d80 0x5634906950c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 >> 172.21.15.94:0/2022601543 conn(0x5634902d8300 msgr2=0x563490683100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 shutdown_connections 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 1 -- 172.21.15.94:0/2022601543 wait complete. 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe4363d1740 20 client.26239 trim_cache size 0 max 16384 2024-04-25T13:45:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:45:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:45:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:45:33.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 msgr2=0x7fe4140b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 0x7fe4140b1ff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe420022000 tx=0x7fe420023000 comp rx=0 tx=0).stop 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 msgr2=0x7fe428199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 shutdown_connections 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe4140afae0 0x7fe4140b1ff0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe428108940 0x7fe4281a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.678+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428103c40 0x7fe428199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.679+0000 7fe42f449640 1 --2- 172.21.15.94:0/4243592360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe428102a00 0x7fe428199130 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.679+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 >> 172.21.15.94:0/4243592360 conn(0x7fe4280fdbd0 msgr2=0x7fe428104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.679+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 shutdown_connections 2024-04-25T13:45:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:33.679+0000 7fe42f449640 1 -- 172.21.15.94:0/4243592360 wait complete. 2024-04-25T13:45:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:33 smithi094 ceph-mon[29195]: pgmap v459: 97 pgs: 97 active+clean; 46 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-04-25T13:45:33.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:33 smithi138 ceph-mon[32977]: pgmap v459: 97 pgs: 97 active+clean; 46 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-04-25T13:45:33.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 347904 13.14 MB/sec execute 16 sec latency 93.422 ms 2024-04-25T13:45:33.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:33 smithi137 ceph-mon[32568]: pgmap v459: 97 pgs: 97 active+clean; 46 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 364 op/s 2024-04-25T13:45:34.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 350525 13.13 MB/sec execute 17 sec latency 26.906 ms 2024-04-25T13:45:35.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:35 smithi094 ceph-mon[29195]: pgmap v460: 97 pgs: 97 active+clean; 46 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-04-25T13:45:35.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:35 smithi138 ceph-mon[32977]: pgmap v460: 97 pgs: 97 active+clean; 46 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-04-25T13:45:35.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 352611 13.04 MB/sec execute 18 sec latency 15.956 ms 2024-04-25T13:45:35.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:35 smithi137 ceph-mon[32568]: pgmap v460: 97 pgs: 97 active+clean; 46 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-04-25T13:45:36.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 355012 13.08 MB/sec execute 19 sec latency 82.116 ms 2024-04-25T13:45:37.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:37 smithi094 ceph-mon[29195]: pgmap v461: 97 pgs: 97 active+clean; 43 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-04-25T13:45:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:37 smithi138 ceph-mon[32977]: pgmap v461: 97 pgs: 97 active+clean; 43 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-04-25T13:45:37.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 357698 13.07 MB/sec execute 20 sec latency 16.850 ms 2024-04-25T13:45:37.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:37 smithi137 ceph-mon[32568]: pgmap v461: 97 pgs: 97 active+clean; 43 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-04-25T13:45:38.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:45:38.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 -- 172.21.15.94:0/1463499544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20109a40 msgr2=0x7f4a2010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1463499544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20109a40 0x7f4a2010be60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009900 tx=0x7f4a1802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 -- 172.21.15.94:0/1463499544 shutdown_connections 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1463499544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20109a40 0x7f4a2010be60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1463499544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20104d40 0x7f4a20105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1463499544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20103b00 0x7f4a20104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 -- 172.21.15.94:0/1463499544 >> 172.21.15.94:0/1463499544 conn(0x7f4a200fecf0 msgr2=0x7f4a201011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 -- 172.21.15.94:0/1463499544 shutdown_connections 2024-04-25T13:45:38.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.368+0000 7f4a2526e640 1 -- 172.21.15.94:0/1463499544 wait complete. 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 Processor -- start 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 -- start start 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 0x7f4a2019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20109a40 0x7f4a201a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f630 con 0x7f4a20103b00 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f4b0 con 0x7f4a20109a40 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.369+0000 7f4a2526e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2010f7b0 con 0x7f4a20104d40 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44634/0 (socket says 172.21.15.94:44634) 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 -- 172.21.15.94:0/1928289353 learned_addr learned my addr 172.21.15.94:0/1928289353 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20109a40 0x7f4a201a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 0x7f4a2019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 -- 172.21.15.94:0/1928289353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 msgr2=0x7f4a2019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 0x7f4a2019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 -- 172.21.15.94:0/1928289353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20109a40 msgr2=0x7f4a201a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20109a40 0x7f4a201a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 -- 172.21.15.94:0/1928289353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a201a1d10 con 0x7f4a20103b00 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 0x7f4a2019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000eb50 tx=0x7f4a1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a07fff640 1 -- 172.21.15.94:0/1928289353 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003b70 con 0x7f4a20103b00 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a07fff640 1 -- 172.21.15.94:0/1928289353 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1000ee50 con 0x7f4a20103b00 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a201a1fa0 con 0x7f4a20103b00 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a07fff640 1 -- 172.21.15.94:0/1928289353 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10010790 con 0x7f4a20103b00 2024-04-25T13:45:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.370+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201a2410 con 0x7f4a20103b00 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a07fff640 1 -- 172.21.15.94:0/1928289353 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4a10004020 con 0x7f4a20103b00 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 Processor -- start 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 -- start start 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35da2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 0x561c35da3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561c35d07d80 con 0x561c359e06b0 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c35cc18b0 con 0x561c35a01390 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c35d03d40 con 0x561c359d84d0 2024-04-25T13:45:38.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a1e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 0x561c35da3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a1effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35da2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57478/0 (socket says 172.21.15.94:57478) 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/3202172687 learned_addr learned my addr 172.21.15.94:0/3202172687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.373+0000 7f4a07fff640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 0x7f4a080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/3202172687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 msgr2=0x561c35da2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35da2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/3202172687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 msgr2=0x561c35da3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 0x561c35da3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/3202172687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c35c92a40 con 0x561c35a01390 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 0x561c35da3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 0x7f4a080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a07fff640 1 -- 172.21.15.94:0/1928289353 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4a100df580 con 0x7f4a20103b00 2024-04-25T13:45:38.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009900 tx=0x7f4a18031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28866e7881679ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a057fa640 1 -- 172.21.15.94:0/3202172687 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1804f070 con 0x561c35a01390 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a057fa640 1 -- 172.21.15.94:0/3202172687 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a18004240 con 0x561c35a01390 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.374+0000 7f4a057fa640 1 -- 172.21.15.94:0/3202172687 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18035040 con 0x561c35a01390 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/3202172687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 msgr2=0x561c35da2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009900 tx=0x7f4a18031b10 comp rx=0 tx=0).stop 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 0x7f4a080b2000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a1804c000 tx=0x7f4a18049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/3202172687 shutdown_connections 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c359e06b0 0x561c35da3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35da2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/3202172687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c35a01390 0x561c35da2a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/3202172687 >> 172.21.15.94:0/3202172687 conn(0x561c359f54e0 msgr2=0x561c35d9e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/3202172687 shutdown_connections 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.375+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/3202172687 wait complete. 2024-04-25T13:45:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 -- start start 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 Processor -- start 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35dac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35dace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561c35da4780 con 0x561c35a01390 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c35c8ed10 con 0x561c359d84d0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c35da4900 con 0x561c359e06b0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a2c1fe740 10 client.0 _ll_register_callbacks cb 0x561c35cbb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35dac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35dace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44646/0 (socket says 172.21.15.94:44646) 2024-04-25T13:45:38.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/4096443052 learned_addr learned my addr 172.21.15.94:0/4096443052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 -- 172.21.15.94:0/4096443052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 msgr2=0x561c35dace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35dace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 -- 172.21.15.94:0/4096443052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 msgr2=0x561c35db3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1effd640 1 -- 172.21.15.94:0/4096443052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c35ae37d0 con 0x561c359d84d0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35dace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.376+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a1effd640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35dac900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a100dd030 tx=0x7f4a100e2d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cadbd35b20cb946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1008ee60 con 0x561c359d84d0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4a100ca0e0 con 0x561c359d84d0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a18004240 con 0x561c359d84d0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a100a7130 con 0x561c359d84d0 2024-04-25T13:45:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 10 client.26336 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c35a193f0 con 0x561c359d84d0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a18004240 con 0x561c359d84d0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4a10004310 con 0x561c359d84d0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 10 client.26336 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 10 client.26336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 20 client.26336 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.377+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359bff60 0x561c35dba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.378+0000 7f4a2c1fe740 4 client.26336 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.378+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c35d15990 con 0x561c359bff60 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.378+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359bff60 0x561c35dba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.378+0000 7f4a02ff5640 10 client.26336 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.378+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359bff60 0x561c35dba240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed0a660ed75855d server_cookie=bb016588455efff8 in_seq=0 out_seq=0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a02ff5640 1 -- 172.21.15.94:0/4096443052 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c35d15990 con 0x561c359bff60 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a02ff5640 10 client.26336 handle_command_reply: tid=0 2024-04-25T13:45:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 client.26336 shutdown 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 msgr2=0x561c35dac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35dac900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a100dd030 tx=0x7f4a100e2d50 comp rx=0 tx=0).stop 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 shutdown_connections 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359bff60 0x561c35dba240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35db3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35dace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/4096443052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35dac900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a02ff5640 0 client.26336 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 >> 172.21.15.94:0/4096443052 conn(0x561c359f54e0 msgr2=0x561c35da1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 shutdown_connections 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.379+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/4096443052 wait complete. 2024-04-25T13:45:38.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.380+0000 7f4a2c1fe740 20 client.26336 trim_cache size 0 max 16384 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 Processor -- start 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 -- start start 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35d9dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35d9e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ab450 con 0x561c35a01390 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201acf90 con 0x561c359d84d0 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.397+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201abb20 con 0x561c359e06b0 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44648/0 (socket says 172.21.15.94:44648) 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/2346773117 learned_addr learned my addr 172.21.15.94:0/2346773117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1effd640 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35d9dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35d9e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/2346773117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 msgr2=0x561c35d9e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35d9e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/2346773117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 msgr2=0x561c35d9dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35d9dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/2346773117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c359d90f0 con 0x561c35a01390 2024-04-25T13:45:38.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802a930 tx=0x7f4a1803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5487fbbc052155a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a04ff9640 1 -- 172.21.15.94:0/2346773117 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18035070 con 0x561c35a01390 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a04ff9640 1 -- 172.21.15.94:0/2346773117 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a18005d50 con 0x561c35a01390 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.398+0000 7f4a04ff9640 1 -- 172.21.15.94:0/2346773117 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1804f070 con 0x561c35a01390 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/2346773117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 msgr2=0x561c35dac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802a930 tx=0x7f4a1803e000 comp rx=0 tx=0).stop 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/2346773117 shutdown_connections 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35dac4d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359e06b0 0x561c35d9e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/2346773117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359d84d0 0x561c35d9dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/2346773117 >> 172.21.15.94:0/2346773117 conn(0x561c359f54e0 msgr2=0x561c35da76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/2346773117 shutdown_connections 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/2346773117 wait complete. 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 -- start start 2024-04-25T13:45:38.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.399+0000 7f4a2c1fe740 1 Processor -- start 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35db2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359e06b0 0x561c35db3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ac8c0 con 0x561c35a01390 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ac1f0 con 0x561c359e06b0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201a9430 con 0x561c359d84d0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 10 client.0 _ll_register_callbacks cb 0x7f4a1000ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44650/0 (socket says 172.21.15.94:44650) 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/1884994288 learned_addr learned my addr 172.21.15.94:0/1884994288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1e7fc640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359e06b0 0x561c35db3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35db2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/1884994288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 msgr2=0x561c35db2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35db2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359e06b0 msgr2=0x561c35db3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359e06b0 0x561c35db3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 -- 172.21.15.94:0/1884994288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c35cc84b0 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35db2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a1f7fe640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4a180314e0 tx=0x7f4a18003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51e13849eb1285c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18061020 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4a18049480 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.400+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a18005d50 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a1806c050 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 10 client.16860 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c35e21050 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a18005d50 con 0x561c35a01390 2024-04-25T13:45:38.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4a18053640 con 0x561c35a01390 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 10 client.16860 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 20 client.16860 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359953a0 0x561c35e235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 4 client.16860 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c35d0f800 con 0x561c359953a0 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.401+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359953a0 0x561c35e235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.402+0000 7f4a02ff5640 10 client.16860 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:38.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.402+0000 7f4a1effd640 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359953a0 0x561c35e235b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e787bb0361fd5e6 server_cookie=89265ec4c7984a86 in_seq=0 out_seq=0 2024-04-25T13:45:38.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a02ff5640 1 -- 172.21.15.94:0/1884994288 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c35d0f800 con 0x561c359953a0 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a02ff5640 10 client.16860 handle_command_reply: tid=0 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 client.16860 shutdown 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 msgr2=0x561c35daf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4a180314e0 tx=0x7f4a18003c90 comp rx=0 tx=0).stop 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 shutdown_connections 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c359953a0 0x561c35e235b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c35a01390 0x561c35daf940 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c359e06b0 0x561c35db3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a02ff5640 0 client.16860 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 --2- 172.21.15.94:0/1884994288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c359d84d0 0x561c35db2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 >> 172.21.15.94:0/1884994288 conn(0x561c359f54e0 msgr2=0x561c35d9dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 shutdown_connections 2024-04-25T13:45:38.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 1 -- 172.21.15.94:0/1884994288 wait complete. 2024-04-25T13:45:38.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.403+0000 7f4a2c1fe740 20 client.16860 trim_cache size 0 max 16384 2024-04-25T13:45:38.409 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:45:38.409 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:45:38.409 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 msgr2=0x7f4a080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 0x7f4a080b2000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a1804c000 tx=0x7f4a18049000 comp rx=0 tx=0).stop 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 msgr2=0x7f4a2019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000eb50 tx=0x7f4a1000c6a0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 shutdown_connections 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4a080afaf0 0x7f4a080b2000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a20109a40 0x7f4a201a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a20104d40 0x7f4a2019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 --2- 172.21.15.94:0/1928289353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a20103b00 0x7f4a2019a1d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 >> 172.21.15.94:0/1928289353 conn(0x7f4a200fecf0 msgr2=0x7f4a20105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 shutdown_connections 2024-04-25T13:45:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.404+0000 7f4a2526e640 1 -- 172.21.15.94:0/1928289353 wait complete. 2024-04-25T13:45:38.414 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:45:38.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:38.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2052283377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4041029c0 msgr2=0x7fa4041036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2052283377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4041029c0 0x7fa4041036c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009900 tx=0x7fa3f402a410 comp rx=0 tx=0).stop 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2052283377 shutdown_connections 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2052283377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 0x7fa40410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2052283377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa404103c00 0x7fa404104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2052283377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4041029c0 0x7fa4041036c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2052283377 >> 172.21.15.94:0/2052283377 conn(0x7fa4040fdbd0 msgr2=0x7fa404100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2052283377 shutdown_connections 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.531+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2052283377 wait complete. 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 Processor -- start 2024-04-25T13:45:38.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 -- start start 2024-04-25T13:45:38.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 0x7fa40406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 0x7fa4041a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410e4c0 con 0x7fa404103c00 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410e340 con 0x7fa4041029c0 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa40a7ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410e640 con 0x7fa404108900 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa4037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa4037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44668/0 (socket says 172.21.15.94:44668) 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa4037fe640 1 -- 172.21.15.94:0/2199793328 learned_addr learned my addr 172.21.15.94:0/2199793328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa403fff640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 0x7fa40406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.532+0000 7fa408d6b640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 0x7fa4041a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 -- 172.21.15.94:0/2199793328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 msgr2=0x7fa4041a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 0x7fa4041a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 -- 172.21.15.94:0/2199793328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 msgr2=0x7fa40406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 0x7fa40406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 -- 172.21.15.94:0/2199793328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4041a1d40 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa403fff640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 0x7fa40406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4037fe640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e9f0 tx=0x7fa3f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800cd80 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa404198e00 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f80108e0 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8018980 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.533+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa404199350 con 0x7fa404103c00 2024-04-25T13:45:38.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.534+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404107710 con 0x7fa404103c00 2024-04-25T13:45:38.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.536+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa3f8010430 con 0x7fa404103c00 2024-04-25T13:45:38.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.537+0000 7fa4017fa640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 0x7fa3ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.537+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa3f80defc0 con 0x7fa404103c00 2024-04-25T13:45:38.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.537+0000 7fa403fff640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 0x7fa3ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.538+0000 7fa403fff640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 0x7fa3ec0b2000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005ab0 tx=0x7fa3f4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.543+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3f80a74b0 con 0x7fa404103c00 2024-04-25T13:45:38.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.783+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40402db50 con 0x7fa404103c00 2024-04-25T13:45:38.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.783+0000 7fa4017fa640 1 -- 172.21.15.94:0/2199793328 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fa3f80ab7b0 con 0x7fa404103c00 2024-04-25T13:45:38.782 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:45:38.783 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:45:38.783 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.785+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 msgr2=0x7fa3ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.785+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 0x7fa3ec0b2000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005ab0 tx=0x7fa3f4031400 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.785+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 msgr2=0x7fa40419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.785+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e9f0 tx=0x7fa3f800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 shutdown_connections 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3ec0afaf0 0x7fa3ec0b2000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa404108900 0x7fa4041a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa404103c00 0x7fa40419efd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 --2- 172.21.15.94:0/2199793328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4041029c0 0x7fa40406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 >> 172.21.15.94:0/2199793328 conn(0x7fa4040fdbd0 msgr2=0x7fa4041048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 shutdown_connections 2024-04-25T13:45:38.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.786+0000 7fa40a7ce640 1 -- 172.21.15.94:0/2199793328 wait complete. 2024-04-25T13:45:38.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 359834 13.01 MB/sec execute 21 sec latency 15.672 ms 2024-04-25T13:45:38.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:45:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 -- 172.21.15.94:0/3164392837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 msgr2=0x7f42801036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 --2- 172.21.15.94:0/3164392837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f42801036e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 -- 172.21.15.94:0/3164392837 shutdown_connections 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 --2- 172.21.15.94:0/3164392837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280108920 0x7f428010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 --2- 172.21.15.94:0/3164392837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280103c20 0x7f4280104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 --2- 172.21.15.94:0/3164392837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f42801036e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 -- 172.21.15.94:0/3164392837 >> 172.21.15.94:0/3164392837 conn(0x7f42800fdbd0 msgr2=0x7f4280100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 -- 172.21.15.94:0/3164392837 shutdown_connections 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.919+0000 7f428812e640 1 -- 172.21.15.94:0/3164392837 wait complete. 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 Processor -- start 2024-04-25T13:45:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 -- start start 2024-04-25T13:45:38.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280103c20 0x7f428019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280108920 0x7f42801a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f428010e4e0 con 0x7f42801029e0 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f428010e360 con 0x7f4280108920 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f428812e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f428010e660 con 0x7f4280103c20 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f4285eca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f4285eca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44682/0 (socket says 172.21.15.94:44682) 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f4285eca640 1 -- 172.21.15.94:0/1806853488 learned_addr learned my addr 172.21.15.94:0/1806853488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f42856c9640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280103c20 0x7f428019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.920+0000 7f42866cb640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280108920 0x7f42801a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 -- 172.21.15.94:0/1806853488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280103c20 msgr2=0x7f428019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280103c20 0x7f428019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 -- 172.21.15.94:0/1806853488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280108920 msgr2=0x7f42801a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280108920 0x7f42801a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 -- 172.21.15.94:0/1806853488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42801a20f0 con 0x7f42801029e0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4285eca640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4276ffd640 1 -- 172.21.15.94:0/1806853488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c03f030 con 0x7f42801029e0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4276ffd640 1 -- 172.21.15.94:0/1806853488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c031910 con 0x7f42801029e0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f4276ffd640 1 -- 172.21.15.94:0/1806853488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c035070 con 0x7f42801029e0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42801a2380 con 0x7f42801029e0 2024-04-25T13:45:38.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.921+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801a28c0 con 0x7f42801029e0 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 Processor -- start 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 -- start start 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1485a4d0 0x55ff14c25d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c26a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ff14b89d80 con 0x55ff1485a4d0 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff14b43a20 con 0x55ff14883390 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f428efda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ff14b85d40 con 0x55ff148626b0 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f42866cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f4276ffd640 1 -- 172.21.15.94:0/1806853488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f427c042030 con 0x7f42801029e0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f42866cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57554/0 (socket says 172.21.15.94:57554) 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.924+0000 7f42866cb640 1 -- 172.21.15.94:0/2404403126 learned_addr learned my addr 172.21.15.94:0/2404403126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42856c9640 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c26a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f4285eca640 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1485a4d0 0x55ff14c25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 -- 172.21.15.94:0/2404403126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 msgr2=0x55ff14c26a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c26a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 -- 172.21.15.94:0/2404403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1485a4d0 msgr2=0x55ff14c25d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1485a4d0 0x55ff14c25d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 -- 172.21.15.94:0/2404403126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff14b4ec30 con 0x55ff14883390 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f42866cb640 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4278009f50 tx=0x7f42780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c28ed64157d93d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f4276ffd640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 0x7f426c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f426bfff640 1 -- 172.21.15.94:0/2404403126 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278004d20 con 0x55ff14883390 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f426bfff640 1 -- 172.21.15.94:0/2404403126 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004050 con 0x55ff14883390 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f426bfff640 1 -- 172.21.15.94:0/2404403126 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278005280 con 0x55ff14883390 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f4285eca640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 0x7f426c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.925+0000 7f4276ffd640 1 -- 172.21.15.94:0/1806853488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f427c0fef20 con 0x7f42801029e0 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 -- 172.21.15.94:0/2404403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 msgr2=0x55ff14c25810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f4285eca640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 0x7f426c0b2080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55ff14c25530 tx=0x7f427c0b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4278009f50 tx=0x7f42780118d0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 -- 172.21.15.94:0/2404403126 shutdown_connections 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1485a4d0 0x55ff14c25d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 --2- 172.21.15.94:0/2404403126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff14883390 0x55ff14c25810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 -- 172.21.15.94:0/2404403126 >> 172.21.15.94:0/2404403126 conn(0x55ff148774e0 msgr2=0x55ff14c21020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 -- 172.21.15.94:0/2404403126 shutdown_connections 2024-04-25T13:45:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.926+0000 7f428efda740 1 -- 172.21.15.94:0/2404403126 wait complete. 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 -- start start 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 Processor -- start 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c2f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff14883390 0x55ff14c369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ff14c27530 con 0x55ff148626b0 2024-04-25T13:45:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff14b10dc0 con 0x55ff1485a4d0 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ff14c276b0 con 0x55ff14883390 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f428efda740 10 client.0 _ll_register_callbacks cb 0x55ff14b2c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f42856c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f42856c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44702/0 (socket says 172.21.15.94:44702) 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f42856c9640 1 -- 172.21.15.94:0/802933923 learned_addr learned my addr 172.21.15.94:0/802933923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.927+0000 7f42866cb640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c2f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42866cb640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff14883390 0x55ff14c369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 -- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff14883390 msgr2=0x55ff14c369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff14883390 0x55ff14c369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 -- 172.21.15.94:0/802933923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 msgr2=0x55ff14c2f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c2f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 -- 172.21.15.94:0/802933923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1489b3f0 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42856c9640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4270012700 tx=0x7f4270012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12d1847ea81eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42697fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270004030 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f42700041d0 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278004050 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f42700043d0 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f428efda740 10 client.16878 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff149657d0 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.928+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004050 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f42700039f0 con 0x55ff148626b0 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 10 client.16878 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 10 client.16878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 20 client.16878 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff14841f60 0x55ff14c3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 4 client.16878 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff14b97990 con 0x55ff14841f60 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f42866cb640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff14841f60 0x55ff14c3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f42697fa640 10 client.16878 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.929+0000 7f42866cb640 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff14841f60 0x55ff14c3d240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5712d8cdd0a1fa8 server_cookie=9ccfd9c03084df8a in_seq=0 out_seq=0 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f42697fa640 1 -- 172.21.15.94:0/802933923 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff14b97990 con 0x55ff14841f60 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f42697fa640 10 client.16878 handle_command_reply: tid=0 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 client.16878 shutdown 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 msgr2=0x55ff14c2fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4270012700 tx=0x7f4270012bc0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 shutdown_connections 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff14841f60 0x55ff14c3d240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff14883390 0x55ff14c369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff148626b0 0x55ff14c2fa60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f42697fa640 0 client.16878 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 --2- 172.21.15.94:0/802933923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c2f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 >> 172.21.15.94:0/802933923 conn(0x55ff148774e0 msgr2=0x55ff14c20040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 shutdown_connections 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.930+0000 7f428efda740 1 -- 172.21.15.94:0/802933923 wait complete. 2024-04-25T13:45:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.931+0000 7f428efda740 20 client.16878 trim_cache size 0 max 16384 2024-04-25T13:45:38.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 Processor -- start 2024-04-25T13:45:38.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 -- start start 2024-04-25T13:45:38.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ab570 con 0x55ff14883390 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ad0b0 con 0x55ff1485a4d0 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f428efda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42801abc40 con 0x55ff148626b0 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.945+0000 7f42866cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44710/0 (socket says 172.21.15.94:44710) 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 -- 172.21.15.94:0/3842469093 learned_addr learned my addr 172.21.15.94:0/3842469093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42856c9640 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 -- 172.21.15.94:0/3842469093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 msgr2=0x55ff14c368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 -- 172.21.15.94:0/3842469093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 msgr2=0x55ff14c419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 -- 172.21.15.94:0/3842469093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff14c36e20 con 0x55ff14883390 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f42866cb640 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c6acee81eebda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f426b7fe640 1 -- 172.21.15.94:0/3842469093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278004280 con 0x55ff14883390 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f426b7fe640 1 -- 172.21.15.94:0/3842469093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004ce0 con 0x55ff14883390 2024-04-25T13:45:38.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.946+0000 7f426b7fe640 1 -- 172.21.15.94:0/3842469093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278005040 con 0x55ff14883390 2024-04-25T13:45:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 -- 172.21.15.94:0/3842469093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 msgr2=0x55ff14c41f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f4278009500 comp rx=0 tx=0).stop 2024-04-25T13:45:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 -- 172.21.15.94:0/3842469093 shutdown_connections 2024-04-25T13:45:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c41f30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff148626b0 0x55ff14c368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 --2- 172.21.15.94:0/3842469093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1485a4d0 0x55ff14c419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 -- 172.21.15.94:0/3842469093 >> 172.21.15.94:0/3842469093 conn(0x55ff148774e0 msgr2=0x55ff14c2a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 -- 172.21.15.94:0/3842469093 shutdown_connections 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.947+0000 7f428efda740 1 -- 172.21.15.94:0/3842469093 wait complete. 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 -- start start 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 Processor -- start 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff1485a4d0 0x55ff14c340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff148626b0 0x55ff14c34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ac9e0 con 0x55ff14883390 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ac310 con 0x55ff148626b0 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42801a9550 con 0x55ff1485a4d0 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f428efda740 10 client.0 _ll_register_callbacks cb 0x7f4270009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44724/0 (socket says 172.21.15.94:44724) 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 -- 172.21.15.94:0/1040818486 learned_addr learned my addr 172.21.15.94:0/1040818486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff1485a4d0 0x55ff14c340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42856c9640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff148626b0 0x55ff14c34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 -- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff1485a4d0 msgr2=0x55ff14c340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff1485a4d0 0x55ff14c340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 -- 172.21.15.94:0/1040818486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff148626b0 msgr2=0x55ff14c34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff148626b0 0x55ff14c34610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.948+0000 7f42866cb640 1 -- 172.21.15.94:0/1040818486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff14c35b00 con 0x55ff14883390 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42866cb640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f42780048f0 tx=0x7f4278031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672feccd5a090c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278004270 con 0x55ff14883390 2024-04-25T13:45:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f42780051a0 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278004ce0 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4278004410 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 10 client.16884 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ff14ca55b0 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4278004ce0 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4278040020 con 0x55ff14883390 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 10 client.16884 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 10 client.16884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 20 client.16884 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff148173a0 0x55ff14ca7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 4 client.16884 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.949+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff14b91800 con 0x55ff148173a0 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42856c9640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff148173a0 0x55ff14ca7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:38.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42697fa640 10 client.16884 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42856c9640 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff148173a0 0x55ff14ca7c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8567b4aca170d3bf server_cookie=74182f5eef3d0dca in_seq=0 out_seq=0 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42697fa640 1 -- 172.21.15.94:0/1040818486 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff14b91800 con 0x55ff148173a0 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42697fa640 10 client.16884 handle_command_reply: tid=0 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 client.16884 shutdown 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 msgr2=0x55ff14c35540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f42780048f0 tx=0x7f4278031040 comp rx=0 tx=0).stop 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 shutdown_connections 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ff148173a0 0x55ff14ca7c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff14883390 0x55ff14c35540 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff148626b0 0x55ff14c34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f42697fa640 0 client.16884 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.950+0000 7f428efda740 1 --2- 172.21.15.94:0/1040818486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff1485a4d0 0x55ff14c340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 >> 172.21.15.94:0/1040818486 conn(0x55ff148774e0 msgr2=0x55ff14c23e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 shutdown_connections 2024-04-25T13:45:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428efda740 1 -- 172.21.15.94:0/1040818486 wait complete. 2024-04-25T13:45:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428efda740 20 client.16884 trim_cache size 0 max 16384 2024-04-25T13:45:38.957 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 msgr2=0x7f426c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 0x7f426c0b2080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55ff14c25530 tx=0x7f427c0b0040 comp rx=0 tx=0).stop 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 msgr2=0x7f428019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c004270 comp rx=0 tx=0).stop 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 shutdown_connections 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f426c0afb70 0x7f426c0b2080 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4280108920 0x7f42801a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4280103c20 0x7f428019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 --2- 172.21.15.94:0/1806853488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42801029e0 0x7f428019a5b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:38.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 >> 172.21.15.94:0/1806853488 conn(0x7f42800fdbd0 msgr2=0x7f4280109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:38.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 shutdown_connections 2024-04-25T13:45:38.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:38.951+0000 7f428812e640 1 -- 172.21.15.94:0/1806853488 wait complete. 2024-04-25T13:45:38.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 -- 172.21.15.94:0/2559059598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc1029e0 msgr2=0x7fd0fc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 --2- 172.21.15.94:0/2559059598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc1036e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009900 tx=0x7fd0f802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 -- 172.21.15.94:0/2559059598 shutdown_connections 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 --2- 172.21.15.94:0/2559059598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 0x7fd0fc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 --2- 172.21.15.94:0/2559059598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc103c20 0x7fd0fc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 --2- 172.21.15.94:0/2559059598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc1036e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 -- 172.21.15.94:0/2559059598 >> 172.21.15.94:0/2559059598 conn(0x7fd0fc0fdbd0 msgr2=0x7fd0fc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 -- 172.21.15.94:0/2559059598 shutdown_connections 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.082+0000 7fd104772640 1 -- 172.21.15.94:0/2559059598 wait complete. 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 Processor -- start 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 -- start start 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 0x7fd0fc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10e4c0 con 0x7fd0fc103c20 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10e340 con 0x7fd0fc1029e0 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd104772640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10e640 con 0x7fd0fc108920 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd101d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.083+0000 7fd10250e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd101d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44748/0 (socket says 172.21.15.94:44748) 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57602/0 (socket says 172.21.15.94:57602) 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd101d0d640 1 -- 172.21.15.94:0/3868164834 learned_addr learned my addr 172.21.15.94:0/3868164834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 0x7fd0fc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 -- 172.21.15.94:0/3868164834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 msgr2=0x7fd0fc1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 0x7fd0fc1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 -- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 msgr2=0x7fd0fc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 -- 172.21.15.94:0/3868164834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc1a1da0 con 0x7fd0fc1029e0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd10250e640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802a930 tx=0x7fd0f8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd0f37fe640 1 -- 172.21.15.94:0/3868164834 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f803f070 con 0x7fd0fc1029e0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc1a2030 con 0x7fd0fc1029e0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd0f37fe640 1 -- 172.21.15.94:0/3868164834 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f8003ca0 con 0x7fd0fc1029e0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd0f37fe640 1 -- 172.21.15.94:0/3868164834 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8035070 con 0x7fd0fc1029e0 2024-04-25T13:45:39.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.084+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1a2540 con 0x7fd0fc1029e0 2024-04-25T13:45:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 Processor -- start 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 -- start start 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd0f37fe640 1 -- 172.21.15.94:0/3868164834 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd0f803b460 con 0x7fd0fc1029e0 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef47a390 0x556cef81c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556cef780d80 con 0x556cef4596b0 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556cef73aa20 con 0x556cef4514d0 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10b61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cef77cd40 con 0x556cef47a390 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd101d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd10250e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd101d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44760/0 (socket says 172.21.15.94:44760) 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.087+0000 7fd101d0d640 1 -- 172.21.15.94:0/2032565085 learned_addr learned my addr 172.21.15.94:0/2032565085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd102d0f640 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef47a390 0x556cef81c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 -- 172.21.15.94:0/2032565085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef47a390 msgr2=0x556cef81c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef47a390 0x556cef81c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 -- 172.21.15.94:0/2032565085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 msgr2=0x556cef81cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 -- 172.21.15.94:0/2032565085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cef745c30 con 0x556cef4596b0 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd0f37fe640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 0x7fd0e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd101d0d640 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0115f0 tx=0x7fd0ec011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1429282463709e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 0x7fd0e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd0f0ff9640 1 -- 172.21.15.94:0/2032565085 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec01e070 con 0x556cef4596b0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd0f37fe640 1 -- 172.21.15.94:0/3868164834 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd0f80fee20 con 0x7fd0fc1029e0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd0f0ff9640 1 -- 172.21.15.94:0/2032565085 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec00f660 con 0x556cef4596b0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.088+0000 7fd0f0ff9640 1 -- 172.21.15.94:0/2032565085 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec003830 con 0x556cef4596b0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 -- 172.21.15.94:0/2032565085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 msgr2=0x556cef81da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 0x7fd0e80b2000 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556cef81d860 tx=0x7fd0f4009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0115f0 tx=0x7fd0ec011ab0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 -- 172.21.15.94:0/2032565085 shutdown_connections 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef81da20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 --2- 172.21.15.94:0/2032565085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef47a390 0x556cef81c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 -- 172.21.15.94:0/2032565085 >> 172.21.15.94:0/2032565085 conn(0x556cef46e4e0 msgr2=0x556cef818020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 -- 172.21.15.94:0/2032565085 shutdown_connections 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.089+0000 7fd10b61d740 1 -- 172.21.15.94:0/2032565085 wait complete. 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 -- start start 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 Processor -- start 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef826590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef826ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556cef81e530 con 0x556cef47a390 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556cef707dc0 con 0x556cef4514d0 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cef81e6b0 con 0x556cef4596b0 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10b61d740 10 client.0 _ll_register_callbacks cb 0x556cef723600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44764/0 (socket says 172.21.15.94:44764) 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 -- 172.21.15.94:0/1336038822 learned_addr learned my addr 172.21.15.94:0/1336038822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10250e640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef826590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd101d0d640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef826ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 -- 172.21.15.94:0/1336038822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 msgr2=0x556cef826ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef826ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 -- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 msgr2=0x556cef826590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef826590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd102d0f640 1 -- 172.21.15.94:0/1336038822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cef4923f0 con 0x556cef47a390 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd10250e640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef826590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.090+0000 7fd101d0d640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef826ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd102d0f640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40042b0 tx=0x7fd0f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67e87457fcf7991 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4025070 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd0f4011d60 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ec00f660 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0f4031050 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 10 client.16896 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cef55c7d0 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec00f660 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f400f040 con 0x556cef47a390 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 10 client.16896 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 10 client.16896 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 20 client.16896 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.091+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef438f60 0x556cef834420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.092+0000 7fd10b61d740 4 client.16896 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.092+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cef78e990 con 0x556cef438f60 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.092+0000 7fd10250e640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef438f60 0x556cef834420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.092+0000 7fd0e5ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.092+0000 7fd10250e640 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef438f60 0x556cef834420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73feb638dee196b4 server_cookie=6a1facd36299e4bd in_seq=0 out_seq=0 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/1336038822 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cef78e990 con 0x556cef438f60 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd0e5ffb640 10 client.16896 handle_command_reply: tid=0 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 client.16896 shutdown 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 msgr2=0x556cef82da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40042b0 tx=0x7fd0f4004a40 comp rx=0 tx=0).stop 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 shutdown_connections 2024-04-25T13:45:39.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef438f60 0x556cef834420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef47a390 0x556cef82da30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd0e5ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef826ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 --2- 172.21.15.94:0/1336038822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef826590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 >> 172.21.15.94:0/1336038822 conn(0x556cef46e4e0 msgr2=0x556cef817040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 shutdown_connections 2024-04-25T13:45:39.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.093+0000 7fd10b61d740 1 -- 172.21.15.94:0/1336038822 wait complete. 2024-04-25T13:45:39.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.094+0000 7fd10b61d740 20 client.16896 trim_cache size 0 max 16384 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.107+0000 7fd10b61d740 1 Processor -- start 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.107+0000 7fd10b61d740 1 -- start start 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 0x556cef81c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1ab570 con 0x556cef4596b0 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1ad0b0 con 0x556cef4514d0 2024-04-25T13:45:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10b61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1abc40 con 0x556cef438f60 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44778/0 (socket says 172.21.15.94:44778) 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 -- 172.21.15.94:0/3783726060 learned_addr learned my addr 172.21.15.94:0/3783726060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10250e640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 0x556cef81c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 -- 172.21.15.94:0/3783726060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 msgr2=0x556cef81c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 0x556cef81c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 -- 172.21.15.94:0/3783726060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 msgr2=0x556cef81ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 -- 172.21.15.94:0/3783726060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cef818bb0 con 0x556cef4596b0 2024-04-25T13:45:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd10250e640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 0x556cef81c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4029550 tx=0x7fd0f4029490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b8f3e878492fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd0e7fff640 1 -- 172.21.15.94:0/3783726060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4025070 con 0x556cef4596b0 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd0e7fff640 1 -- 172.21.15.94:0/3783726060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f4011e60 con 0x556cef4596b0 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.108+0000 7fd0e7fff640 1 -- 172.21.15.94:0/3783726060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f402a440 con 0x556cef4596b0 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 -- 172.21.15.94:0/3783726060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 msgr2=0x556cef8212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4029550 tx=0x7fd0f4029490 comp rx=0 tx=0).stop 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 -- 172.21.15.94:0/3783726060 shutdown_connections 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef4596b0 0x556cef8212f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef81ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3783726060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef438f60 0x556cef81c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 -- 172.21.15.94:0/3783726060 >> 172.21.15.94:0/3783726060 conn(0x556cef46e4e0 msgr2=0x556cef824cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 -- 172.21.15.94:0/3783726060 shutdown_connections 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.109+0000 7fd10b61d740 1 -- 172.21.15.94:0/3783726060 wait complete. 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 -- start start 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 Processor -- start 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef8298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef82c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1ac9e0 con 0x556cef438f60 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1ac310 con 0x556cef4514d0 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc1a9550 con 0x556cef4596b0 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10b61d740 10 client.0 _ll_register_callbacks cb 0x7fd0f4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10250e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10250e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44788/0 (socket says 172.21.15.94:44788) 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10250e640 1 -- 172.21.15.94:0/3000523562 learned_addr learned my addr 172.21.15.94:0/3000523562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef8298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd102d0f640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef82c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10250e640 1 -- 172.21.15.94:0/3000523562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 msgr2=0x556cef82c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.110+0000 7fd10250e640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef82c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10250e640 1 -- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 msgr2=0x556cef8298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10250e640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef8298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10250e640 1 -- 172.21.15.94:0/3000523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cef82cde0 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef8298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10250e640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80fc880 tx=0x7fd0f8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7f785a75a9c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8048040 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd0f80c7570 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4011e60 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0f80c6a80 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10b61d740 10 client.16902 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cef89b2f0 con 0x556cef438f60 2024-04-25T13:45:39.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.111+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f4011e60 con 0x556cef438f60 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd0f80cfc30 con 0x556cef438f60 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 10 client.16902 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 10 client.16902 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 20 client.16902 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef47a390 0x556cef89dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 4 client.16902 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cef788800 con 0x556cef47a390 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef47a390 0x556cef89dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd0e5ffb640 10 client.16902 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:39.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd101d0d640 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef47a390 0x556cef89dc40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e58f2e52a01b590 server_cookie=2e22cac223d094b3 in_seq=0 out_seq=0 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.112+0000 7fd0e5ffb640 1 -- 172.21.15.94:0/3000523562 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cef788800 con 0x556cef47a390 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd0e5ffb640 10 client.16902 handle_command_reply: tid=0 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 client.16902 shutdown 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 msgr2=0x556cef8293a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80fc880 tx=0x7fd0f8009620 comp rx=0 tx=0).stop 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 shutdown_connections 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x556cef47a390 0x556cef89dc40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cef4596b0 0x556cef82c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd0e5ffb640 0 client.16902 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cef4514d0 0x556cef8298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 --2- 172.21.15.94:0/3000523562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cef438f60 0x556cef8293a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 >> 172.21.15.94:0/3000523562 conn(0x556cef46e4e0 msgr2=0x556cef824cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 shutdown_connections 2024-04-25T13:45:39.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 1 -- 172.21.15.94:0/3000523562 wait complete. 2024-04-25T13:45:39.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd10b61d740 20 client.16902 trim_cache size 0 max 16384 2024-04-25T13:45:39.122 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:39.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 msgr2=0x7fd0e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 0x7fd0e80b2000 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556cef81d860 tx=0x7fd0f4009600 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 msgr2=0x7fd0fc075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802a930 tx=0x7fd0f8004270 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 shutdown_connections 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.113+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0e80afaf0 0x7fd0e80b2000 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0fc108920 0x7fd0fc1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0fc103c20 0x7fd0fc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 --2- 172.21.15.94:0/3868164834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0fc1029e0 0x7fd0fc075860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 >> 172.21.15.94:0/3868164834 conn(0x7fd0fc0fdbd0 msgr2=0x7fd0fc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 shutdown_connections 2024-04-25T13:45:39.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.114+0000 7fd104772640 1 -- 172.21.15.94:0/3868164834 wait complete. 2024-04-25T13:45:39.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 -- 172.21.15.94:0/2982441417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 msgr2=0x7f0ef0104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 --2- 172.21.15.94:0/2982441417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef0104050 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009900 tx=0x7f0ed802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 -- 172.21.15.94:0/2982441417 shutdown_connections 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 --2- 172.21.15.94:0/2982441417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0108940 0x7f0ef010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 --2- 172.21.15.94:0/2982441417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef0104050 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 --2- 172.21.15.94:0/2982441417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0102a00 0x7f0ef0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.248+0000 7f0ef5851640 1 -- 172.21.15.94:0/2982441417 >> 172.21.15.94:0/2982441417 conn(0x7f0ef00fdbf0 msgr2=0x7f0ef01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- 172.21.15.94:0/2982441417 shutdown_connections 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- 172.21.15.94:0/2982441417 wait complete. 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 Processor -- start 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- start start 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 0x7f0ef019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 0x7f0ef01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010e530 con 0x7f0ef0103c40 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010e3b0 con 0x7f0ef0108940 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ef5851640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010e6b0 con 0x7f0ef0102a00 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.249+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44808/0 (socket says 172.21.15.94:44808) 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1140011451 learned_addr learned my addr 172.21.15.94:0/1140011451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 0x7f0ef01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 0x7f0ef019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1140011451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 msgr2=0x7f0ef019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 0x7f0ef019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1140011451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 msgr2=0x7f0ef01a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 0x7f0ef01a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1140011451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01a1dc0 con 0x7f0ef0103c40 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 0x7f0ef019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 0x7f0ef01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ef484f640 1 -- 172.21.15.94:0/1140011451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed803f030 con 0x7f0ef0103c40 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ef484f640 1 -- 172.21.15.94:0/1140011451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8031910 con 0x7f0ef0103c40 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a2050 con 0x7f0ef0103c40 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ef484f640 1 -- 172.21.15.94:0/1140011451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8035070 con 0x7f0ef0103c40 2024-04-25T13:45:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.250+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a2540 con 0x7f0ef0103c40 2024-04-25T13:45:39.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 Processor -- start 2024-04-25T13:45:39.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 -- start start 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0ef484f640 1 -- 172.21.15.94:0/1140011451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0ed8042030 con 0x7f0ef0103c40 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556dd7520d80 con 0x556dd71f96b0 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556dd74daa20 con 0x556dd721a390 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.253+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dd751cd40 con 0x556dd71f14d0 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57680/0 (socket says 172.21.15.94:57680) 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52222/0 (socket says 172.21.15.94:52222) 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eef7fe640 1 -- 172.21.15.94:0/1076487392 learned_addr learned my addr 172.21.15.94:0/1076487392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1076487392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 msgr2=0x556dd75c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1076487392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 msgr2=0x556dd75cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ef484f640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 0x7f0edc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1076487392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd74e5c30 con 0x556dd71f14d0 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 0x7f0edc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ef484f640 1 -- 172.21.15.94:0/1140011451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed80ff070 con 0x7f0ef0103c40 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ef01022d0 tx=0x7f0ee00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0d2063c96155f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ee5ffb640 1 -- 172.21.15.94:0/1076487392 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0004310 con 0x556dd71f14d0 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ee5ffb640 1 -- 172.21.15.94:0/1076487392 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee00044b0 con 0x556dd71f14d0 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.254+0000 7f0ee5ffb640 1 -- 172.21.15.94:0/1076487392 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00035f0 con 0x556dd71f14d0 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1076487392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 msgr2=0x556dd75c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 0x7f0edc0b2000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009fd0 tx=0x7f0ee8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ef01022d0 tx=0x7f0ee00118d0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1076487392 shutdown_connections 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75c9cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1076487392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1076487392 >> 172.21.15.94:0/1076487392 conn(0x556dd720e4e0 msgr2=0x556dd75b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1076487392 shutdown_connections 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.255+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1076487392 wait complete. 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 -- start start 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 Processor -- start 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f96b0 0x556dd75cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556dd75ccf60 con 0x556dd71f14d0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556dd74a7dc0 con 0x556dd721a390 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dd75cd0e0 con 0x556dd71f96b0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0efc7d7740 10 client.0 _ll_register_callbacks cb 0x556dd74c3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0eeeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0eeeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44826/0 (socket says 172.21.15.94:44826) 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0eeeffd640 1 -- 172.21.15.94:0/4017089645 learned_addr learned my addr 172.21.15.94:0/4017089645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f96b0 0x556dd75cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.256+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 -- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f96b0 msgr2=0x556dd75cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f96b0 0x556dd75cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 -- 172.21.15.94:0/4017089645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 msgr2=0x556dd75d6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75d6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 -- 172.21.15.94:0/4017089645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd72323f0 con 0x556dd71f14d0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009580 tx=0x7f0ee0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3ec4a87ec87666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0029070 con 0x556dd71f14d0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ee0003c40 con 0x556dd71f14d0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee00044b0 con 0x556dd71f14d0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ee0035050 con 0x556dd71f14d0 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0efc7d7740 10 client.16914 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dd72fc7d0 con 0x556dd71f14d0 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.257+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee00044b0 con 0x556dd71f14d0 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0ee000f480 con 0x556dd71f14d0 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 10 client.16914 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 20 client.16914 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd71d8f60 0x556dd75dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 4 client.16914 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dd752e990 con 0x556dd71d8f60 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd71d8f60 0x556dd75dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0eceffd640 10 client.16914 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.258+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd71d8f60 0x556dd75dc6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff8f13411639edb6 server_cookie=cf06470d8e2fd73f in_seq=0 out_seq=0 2024-04-25T13:45:39.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0eceffd640 1 -- 172.21.15.94:0/4017089645 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556dd752e990 con 0x556dd71d8f60 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0eceffd640 10 client.16914 handle_command_reply: tid=0 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 client.16914 shutdown 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 msgr2=0x556dd75cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009580 tx=0x7f0ee0004430 comp rx=0 tx=0).stop 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 shutdown_connections 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd71d8f60 0x556dd75dc6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd721a390 0x556dd75d6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f96b0 0x556dd75cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/4017089645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75cecb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0eceffd640 0 client.16914 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 >> 172.21.15.94:0/4017089645 conn(0x556dd720e4e0 msgr2=0x556dd75b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 shutdown_connections 2024-04-25T13:45:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.259+0000 7f0efc7d7740 1 -- 172.21.15.94:0/4017089645 wait complete. 2024-04-25T13:45:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.260+0000 7f0efc7d7740 20 client.16914 trim_cache size 0 max 16384 2024-04-25T13:45:39.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 Processor -- start 2024-04-25T13:45:39.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 -- start start 2024-04-25T13:45:39.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 0x556dd75c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd71f96b0 0x556dd75cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01a9360 con 0x556dd71f14d0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aaea0 con 0x556dd71f96b0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01a9a30 con 0x556dd720e4e0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44840/0 (socket says 172.21.15.94:44840) 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1018197936 learned_addr learned my addr 172.21.15.94:0/1018197936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 0x556dd75c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.274+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd71f96b0 0x556dd75cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1018197936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd71f96b0 msgr2=0x556dd75cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd71f96b0 0x556dd75cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1018197936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 msgr2=0x556dd75d6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 -- 172.21.15.94:0/1018197936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd75d65b0 con 0x556dd720e4e0 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 0x556dd75c4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009380 tx=0x7f0ee0011dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea71131fa1b5563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0ee57fa640 1 -- 172.21.15.94:0/1018197936 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0029070 con 0x556dd720e4e0 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0ee57fa640 1 -- 172.21.15.94:0/1018197936 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee000fac0 con 0x556dd720e4e0 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.275+0000 7f0ee57fa640 1 -- 172.21.15.94:0/1018197936 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee002e6a0 con 0x556dd720e4e0 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1018197936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 msgr2=0x556dd75c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 0x556dd75c4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009380 tx=0x7f0ee0011dd0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1018197936 shutdown_connections 2024-04-25T13:45:39.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd71f96b0 0x556dd75cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f14d0 0x556dd75d6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/1018197936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd720e4e0 0x556dd75c4f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1018197936 >> 172.21.15.94:0/1018197936 conn(0x556dd75cf3c0 msgr2=0x556dd75ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1018197936 shutdown_connections 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- 172.21.15.94:0/1018197936 wait complete. 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 -- start start 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.276+0000 7f0efc7d7740 1 Processor -- start 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd720e4e0 0x556dd75d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aa7d0 con 0x556dd71f96b0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01aa100 con 0x556dd720e4e0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01a7340 con 0x556dd71f14d0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 10 client.0 _ll_register_callbacks cb 0x7f0ed8102ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44856/0 (socket says 172.21.15.94:44856) 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 -- 172.21.15.94:0/430271041 learned_addr learned my addr 172.21.15.94:0/430271041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0eef7fe640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd720e4e0 0x556dd75d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 -- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 msgr2=0x556dd75d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 -- 172.21.15.94:0/430271041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd720e4e0 msgr2=0x556dd75d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd720e4e0 0x556dd75d48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 -- 172.21.15.94:0/430271041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd75d4e80 con 0x556dd71f96b0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80c3680 tx=0x7f0ed80fca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafabd7f5b75d257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8049040 con 0x556dd71f96b0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed80b00b0 con 0x556dd71f96b0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.277+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee000fac0 con 0x556dd71f96b0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ed80c6f50 con 0x556dd71f96b0 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 10 client.16920 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556dd75d6200 con 0x556dd71f96b0 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee000fac0 con 0x556dd71f96b0 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed80c3890 con 0x556dd71f96b0 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 10 client.16920 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 20 client.16920 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd721a390 0x556dd7647150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 4 client.16920 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dd7528800 con 0x556dd721a390 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.278+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd721a390 0x556dd7647150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0eceffd640 10 client.16920 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:39.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0eeeffd640 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd721a390 0x556dd7647150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4bd09d7dab1db8e server_cookie=50d48fd9a30f3ae1 in_seq=0 out_seq=0 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0eceffd640 1 -- 172.21.15.94:0/430271041 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dd7528800 con 0x556dd721a390 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0eceffd640 10 client.16920 handle_command_reply: tid=0 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 client.16920 shutdown 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 msgr2=0x556dd75d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80c3680 tx=0x7f0ed80fca40 comp rx=0 tx=0).stop 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 shutdown_connections 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556dd721a390 0x556dd7647150 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556dd720e4e0 0x556dd75d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556dd71f96b0 0x556dd75d3990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0eceffd640 0 client.16920 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 --2- 172.21.15.94:0/430271041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dd71f14d0 0x556dd75d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 >> 172.21.15.94:0/430271041 conn(0x556dd75cf3c0 msgr2=0x556dd75cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 shutdown_connections 2024-04-25T13:45:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 1 -- 172.21.15.94:0/430271041 wait complete. 2024-04-25T13:45:39.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.279+0000 7f0efc7d7740 20 client.16920 trim_cache size 0 max 16384 2024-04-25T13:45:39.287 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 msgr2=0x7f0edc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 0x7f0edc0b2000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009fd0 tx=0x7f0ee8009500 comp rx=0 tx=0).stop 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 msgr2=0x7f0ef019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed80042f0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 shutdown_connections 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0edc0afaf0 0x7f0edc0b2000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef0108940 0x7f0ef01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0103c40 0x7f0ef019a7c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 --2- 172.21.15.94:0/1140011451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ef0102a00 0x7f0ef019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 >> 172.21.15.94:0/1140011451 conn(0x7f0ef00fdbf0 msgr2=0x7f0ef0109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 shutdown_connections 2024-04-25T13:45:39.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.280+0000 7f0ef5851640 1 -- 172.21.15.94:0/1140011451 wait complete. 2024-04-25T13:45:39.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 -- 172.21.15.94:0/192925147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584102a50 msgr2=0x7f1584103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 --2- 172.21.15.94:0/192925147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584102a50 0x7f1584103750 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1574009900 tx=0x7f157402a410 comp rx=0 tx=0).stop 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 -- 172.21.15.94:0/192925147 shutdown_connections 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 --2- 172.21.15.94:0/192925147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584108990 0x7f158410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 --2- 172.21.15.94:0/192925147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584103c90 0x7f15841040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 --2- 172.21.15.94:0/192925147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584102a50 0x7f1584103750 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 -- 172.21.15.94:0/192925147 >> 172.21.15.94:0/192925147 conn(0x7f15840fdc40 msgr2=0x7f15841000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 -- 172.21.15.94:0/192925147 shutdown_connections 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.412+0000 7f1588efa640 1 -- 172.21.15.94:0/192925147 wait complete. 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 Processor -- start 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 -- start start 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 0x7f15840ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584103c90 0x7f15840fe170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f158410e4c0 con 0x7f1584108990 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f158410e340 con 0x7f1584103c90 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.413+0000 7f1588efa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f158410e640 con 0x7f1584102a50 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f158259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 0x7f15840ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584103c90 0x7f15840fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44874/0 (socket says 172.21.15.94:44874) 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 -- 172.21.15.94:0/2845044785 learned_addr learned my addr 172.21.15.94:0/2845044785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 -- 172.21.15.94:0/2845044785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 msgr2=0x7f15840ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 0x7f15840ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 -- 172.21.15.94:0/2845044785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584103c90 msgr2=0x7f15840fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584103c90 0x7f15840fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 -- 172.21.15.94:0/2845044785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15840fede0 con 0x7f1584108990 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f158259c640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 0x7f15840ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1582d9d640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f157c00eb10 tx=0x7f157c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f15737fe640 1 -- 172.21.15.94:0/2845044785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c004b20 con 0x7f1584108990 2024-04-25T13:45:39.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15841a4300 con 0x7f1584108990 2024-04-25T13:45:39.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f15737fe640 1 -- 172.21.15.94:0/2845044785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c004cc0 con 0x7f1584108990 2024-04-25T13:45:39.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f15737fe640 1 -- 172.21.15.94:0/2845044785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c010670 con 0x7f1584108990 2024-04-25T13:45:39.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.414+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15841a4770 con 0x7f1584108990 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 Processor -- start 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f15737fe640 1 -- 172.21.15.94:0/2845044785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f157c010830 con 0x7f1584108990 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 -- start start 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e63394810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e63395a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e632f8d80 con 0x559e62fc94d0 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e632b2a20 con 0x559e62ff2390 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158fda8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e632f4d40 con 0x559e62fd16b0 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e63394810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44878/0 (socket says 172.21.15.94:44878) 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f158259c640 1 -- 172.21.15.94:0/2425985202 learned_addr learned my addr 172.21.15.94:0/2425985202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.417+0000 7f1581d9b640 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e63395a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f15737fe640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 0x7f156c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 -- 172.21.15.94:0/2425985202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 msgr2=0x559e63395a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e63395a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 -- 172.21.15.94:0/2425985202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 msgr2=0x559e63394d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1582d9d640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 0x7f156c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 -- 172.21.15.94:0/2425985202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e632bdc30 con 0x559e62ff2390 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f158259c640 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f15737fe640 1 -- 172.21.15.94:0/2845044785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f157c0df250 con 0x7f1584108990 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1581d9b640 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e63394810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15840ff470 tx=0x7f1578004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f32cd9c840ea5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1570ff9640 1 -- 172.21.15.94:0/2425985202 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578004050 con 0x559e62ff2390 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1570ff9640 1 -- 172.21.15.94:0/2425985202 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157800fc70 con 0x559e62ff2390 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1570ff9640 1 -- 172.21.15.94:0/2425985202 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578003900 con 0x559e62ff2390 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.418+0000 7f1582d9d640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 0x7f156c0b2000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f157c0dd020 tx=0x7f157c09bfd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 -- 172.21.15.94:0/2425985202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 msgr2=0x559e63394810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e63394810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15840ff470 tx=0x7f1578004730 comp rx=0 tx=0).stop 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 -- 172.21.15.94:0/2425985202 shutdown_connections 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e63395a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e63394d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 --2- 172.21.15.94:0/2425985202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e63394810 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 -- 172.21.15.94:0/2425985202 >> 172.21.15.94:0/2425985202 conn(0x559e62fe64e0 msgr2=0x559e63390020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 -- 172.21.15.94:0/2425985202 shutdown_connections 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.419+0000 7f158fda8740 1 -- 172.21.15.94:0/2425985202 wait complete. 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 -- start start 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 Processor -- start 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6339e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e633a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e63396530 con 0x559e62fd16b0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e6327fdc0 con 0x559e62fc94d0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e633966b0 con 0x559e62ff2390 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158fda8740 10 client.0 _ll_register_callbacks cb 0x559e6329b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44886/0 (socket says 172.21.15.94:44886) 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 -- 172.21.15.94:0/3525179337 learned_addr learned my addr 172.21.15.94:0/3525179337 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6339e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e633a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 -- 172.21.15.94:0/3525179337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 msgr2=0x559e633a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e633a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 -- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 msgr2=0x559e6339e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6339e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 -- 172.21.15.94:0/3525179337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6300a3f0 con 0x559e62fd16b0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f158259c640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f157402a930 tx=0x7f15740317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe594169326b126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.420+0000 7f1569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574035040 con 0x559e62fd16b0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1574004450 con 0x559e62fd16b0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157800fc70 con 0x559e62fd16b0 2024-04-25T13:45:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1574031920 con 0x559e62fd16b0 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 10 client.16932 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e630d47d0 con 0x559e62fd16b0 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157800fc70 con 0x559e62fd16b0 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1574044070 con 0x559e62fd16b0 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 10 client.16932 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 20 client.16932 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62fb0f60 0x559e633ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 4 client.16932 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e63306990 con 0x559e62fb0f60 2024-04-25T13:45:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.421+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62fb0f60 0x559e633ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.422+0000 7f1569ffb640 10 client.16932 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.422+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62fb0f60 0x559e633ac220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e510b738c49a9cee server_cookie=53710585f48086ec in_seq=0 out_seq=0 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.422+0000 7f1569ffb640 1 -- 172.21.15.94:0/3525179337 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e63306990 con 0x559e62fb0f60 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.422+0000 7f1569ffb640 10 client.16932 handle_command_reply: tid=0 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.422+0000 7f158fda8740 1 client.16932 shutdown 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 msgr2=0x559e6339eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f157402a930 tx=0x7f15740317b0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 shutdown_connections 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62fb0f60 0x559e633ac220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e633a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e6339eb30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 --2- 172.21.15.94:0/3525179337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6339e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f1569ffb640 0 client.16932 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 >> 172.21.15.94:0/3525179337 conn(0x559e62fe64e0 msgr2=0x559e63392f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 shutdown_connections 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 1 -- 172.21.15.94:0/3525179337 wait complete. 2024-04-25T13:45:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.423+0000 7f158fda8740 20 client.16932 trim_cache size 0 max 16384 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 Processor -- start 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 -- start start 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6338fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e6338ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ad6b0 con 0x559e62fd16b0 2024-04-25T13:45:39.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15841af1f0 con 0x559e62fc94d0 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158fda8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15841add80 con 0x559e62ff2390 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44898/0 (socket says 172.21.15.94:44898) 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 -- 172.21.15.94:0/3617041157 learned_addr learned my addr 172.21.15.94:0/3617041157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6338fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e6338ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 -- 172.21.15.94:0/3617041157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 msgr2=0x559e6338ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e6338ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 -- 172.21.15.94:0/3617041157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 msgr2=0x559e6338fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6338fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 -- 172.21.15.94:0/3617041157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e633a5e80 con 0x559e62fd16b0 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.438+0000 7f158259c640 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1574031440 tx=0x7f1574005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b85c67de93e384e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.439+0000 7f156bfff640 1 -- 172.21.15.94:0/3617041157 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574053020 con 0x559e62fd16b0 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.439+0000 7f156bfff640 1 -- 172.21.15.94:0/3617041157 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157403cc40 con 0x559e62fd16b0 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.439+0000 7f156bfff640 1 -- 172.21.15.94:0/3617041157 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574035040 con 0x559e62fd16b0 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.439+0000 7f158fda8740 1 -- 172.21.15.94:0/3617041157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 msgr2=0x559e633a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.439+0000 7f158fda8740 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1574031440 tx=0x7f1574005fd0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- 172.21.15.94:0/3617041157 shutdown_connections 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62ff2390 0x559e6338ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fd16b0 0x559e633a5940 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- 172.21.15.94:0/3617041157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62fc94d0 0x559e6338fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- 172.21.15.94:0/3617041157 >> 172.21.15.94:0/3617041157 conn(0x559e62fe64e0 msgr2=0x559e63399350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- 172.21.15.94:0/3617041157 shutdown_connections 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- 172.21.15.94:0/3617041157 wait complete. 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- start start 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 Processor -- start 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e633a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e633a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15841aeb20 con 0x559e62fc94d0 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ae450 con 0x559e62ff2390 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ab690 con 0x559e62fd16b0 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158fda8740 10 client.0 _ll_register_callbacks cb 0x7f1574009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44902/0 (socket says 172.21.15.94:44902) 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f1581d9b640 1 -- 172.21.15.94:0/3303782280 learned_addr learned my addr 172.21.15.94:0/3303782280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f158259c640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e633a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.440+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e633a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 -- 172.21.15.94:0/3303782280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 msgr2=0x559e633a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e633a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 -- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 msgr2=0x559e633a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e633a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 -- 172.21.15.94:0/3303782280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e633a45e0 con 0x559e62fc94d0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f158259c640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e633a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1581d9b640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f157800f790 tx=0x7f1578004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefd23c85e7f5644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578034070 con 0x559e62fc94d0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1578011050 con 0x559e62fc94d0 2024-04-25T13:45:39.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157403cc40 con 0x559e62fc94d0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1578011250 con 0x559e62fc94d0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f158fda8740 10 client.16938 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e633a5860 con 0x559e62fc94d0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.441+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157403cc40 con 0x559e62fc94d0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f157802f040 con 0x559e62fc94d0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 10 client.16938 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 20 client.16938 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62f863a0 0x559e63416820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 4 client.16938 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e63300800 con 0x559e62f863a0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.442+0000 7f158259c640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62f863a0 0x559e63416820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158259c640 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62f863a0 0x559e63416820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e88888e515917b9 server_cookie=5dc990aec14ec663 in_seq=0 out_seq=0 2024-04-25T13:45:39.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f1569ffb640 10 client.16938 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f1569ffb640 1 -- 172.21.15.94:0/3303782280 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e63300800 con 0x559e62f863a0 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f1569ffb640 10 client.16938 handle_command_reply: tid=0 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 client.16938 shutdown 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 msgr2=0x559e633a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f157800f790 tx=0x7f1578004730 comp rx=0 tx=0).stop 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 shutdown_connections 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559e62f863a0 0x559e63416820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e62ff2390 0x559e633a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e62fd16b0 0x559e633a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 --2- 172.21.15.94:0/3303782280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e62fc94d0 0x559e633a2c20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f1569ffb640 0 client.16938 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 >> 172.21.15.94:0/3303782280 conn(0x559e62fe64e0 msgr2=0x559e63392e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 shutdown_connections 2024-04-25T13:45:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 1 -- 172.21.15.94:0/3303782280 wait complete. 2024-04-25T13:45:39.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.443+0000 7f158fda8740 20 client.16938 trim_cache size 0 max 16384 2024-04-25T13:45:39.450 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 msgr2=0x7f156c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 0x7f156c0b2000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f157c0dd020 tx=0x7f157c09bfd0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 msgr2=0x7f15840fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f157c00eb10 tx=0x7f157c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 shutdown_connections 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f156c0afaf0 0x7f156c0b2000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1584108990 0x7f15840fe6b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1584103c90 0x7f15840fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 --2- 172.21.15.94:0/2845044785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1584102a50 0x7f15840ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 >> 172.21.15.94:0/2845044785 conn(0x7f15840fdc40 msgr2=0x7f1584109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 shutdown_connections 2024-04-25T13:45:39.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.444+0000 7f1588efa640 1 -- 172.21.15.94:0/2845044785 wait complete. 2024-04-25T13:45:39.454 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 -- 172.21.15.94:0/3128498473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824102260 msgr2=0x7fb82410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 --2- 172.21.15.94:0/3128498473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824102260 0x7fb82410bcc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ca0 tx=0x7fb81c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 -- 172.21.15.94:0/3128498473 shutdown_connections 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 --2- 172.21.15.94:0/3128498473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824102260 0x7fb82410bcc0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 --2- 172.21.15.94:0/3128498473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824101760 0x7fb824101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 --2- 172.21.15.94:0/3128498473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 0x7fb8241054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 -- 172.21.15.94:0/3128498473 >> 172.21.15.94:0/3128498473 conn(0x7fb8240fd380 msgr2=0x7fb8240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 -- 172.21.15.94:0/3128498473 shutdown_connections 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.569+0000 7fb828b1c640 1 -- 172.21.15.94:0/3128498473 wait complete. 2024-04-25T13:45:39.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 Processor -- start 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 -- start start 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 0x7fb82419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 0x7fb8241a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410f640 con 0x7fb824101760 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410f4c0 con 0x7fb824102260 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb828b1c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410f7c0 con 0x7fb8241050b0 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb82259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb82259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44910/0 (socket says 172.21.15.94:44910) 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb82259c640 1 -- 172.21.15.94:0/1311190375 learned_addr learned my addr 172.21.15.94:0/1311190375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb821d9b640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 0x7fb82419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.570+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 0x7fb8241a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 -- 172.21.15.94:0/1311190375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 msgr2=0x7fb8241a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 0x7fb8241a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 -- 172.21.15.94:0/1311190375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 msgr2=0x7fb82419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 0x7fb82419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 -- 172.21.15.94:0/1311190375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8241a1cc0 con 0x7fb824101760 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb821d9b640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 0x7fb82419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb82259c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb814005d00 tx=0x7fb81400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb8137fe640 1 -- 172.21.15.94:0/1311190375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814017070 con 0x7fb824101760 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb8137fe640 1 -- 172.21.15.94:0/1311190375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb814009c30 con 0x7fb824101760 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8241a1fb0 con 0x7fb824101760 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb8137fe640 1 -- 172.21.15.94:0/1311190375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81400f620 con 0x7fb824101760 2024-04-25T13:45:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.571+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a2410 con 0x7fb824101760 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 Processor -- start 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 -- start start 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb8137fe640 1 -- 172.21.15.94:0/1311190375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb81401a070 con 0x7fb824101760 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a104a4d0 0x5635a141a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a10526b0 0x5635a141ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635a1379d80 con 0x5635a1073390 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635a1333a20 con 0x5635a104a4d0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635a1375d40 con 0x5635a10526b0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44912/0 (socket says 172.21.15.94:44912) 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb822d9d640 1 -- 172.21.15.94:0/1844308113 learned_addr learned my addr 172.21.15.94:0/1844308113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb821d9b640 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a10526b0 0x5635a141ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.574+0000 7fb82259c640 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a104a4d0 0x5635a141a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 -- 172.21.15.94:0/1844308113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a10526b0 msgr2=0x5635a141ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a10526b0 0x5635a141ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 -- 172.21.15.94:0/1844308113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a104a4d0 msgr2=0x5635a141a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a104a4d0 0x5635a141a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 -- 172.21.15.94:0/1844308113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635a133ec30 con 0x5635a1073390 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb8137fe640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 0x7fb80c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb82259c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 0x7fb80c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035000 tx=0x7fb81c02f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441a179a099fecf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb8137fe640 1 -- 172.21.15.94:0/1311190375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb8140de670 con 0x7fb824101760 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb810ff9640 1 -- 172.21.15.94:0/1844308113 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c031650 con 0x5635a1073390 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb810ff9640 1 -- 172.21.15.94:0/1844308113 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c036030 con 0x5635a1073390 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.575+0000 7fb810ff9640 1 -- 172.21.15.94:0/1844308113 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0047c0 con 0x5635a1073390 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1844308113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 msgr2=0x5635a1419c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb81c035000 tx=0x7fb81c02f4c0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82259c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 0x7fb80c0b2000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5635a1419940 tx=0x7fb81408f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1844308113 shutdown_connections 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a10526b0 0x5635a141ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a104a4d0 0x5635a141a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1844308113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1419c20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1844308113 >> 172.21.15.94:0/1844308113 conn(0x5635a10674e0 msgr2=0x5635a1411020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.576+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1844308113 shutdown_connections 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1844308113 wait complete. 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 -- start start 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 Processor -- start 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a1073390 0x5635a142ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635a141b760 con 0x5635a10526b0 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635a1300dc0 con 0x5635a1073390 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635a141b8e0 con 0x5635a104a4d0 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb82f9cc740 10 client.0 _ll_register_callbacks cb 0x5635a131c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb821d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb821d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44924/0 (socket says 172.21.15.94:44924) 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb821d9b640 1 -- 172.21.15.94:0/3953774014 learned_addr learned my addr 172.21.15.94:0/3953774014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52350/0 (socket says 172.21.15.94:52350) 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.577+0000 7fb822d9d640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a1073390 0x5635a142ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 -- 172.21.15.94:0/3953774014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 msgr2=0x5635a1423760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 -- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a1073390 msgr2=0x5635a142ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a1073390 0x5635a142ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 -- 172.21.15.94:0/3953774014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635a108b3f0 con 0x5635a10526b0 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb822d9d640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb821d9b640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb818012d00 tx=0x7fb81800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4561ac6309a170d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818009d90 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb818012110 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c036030 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb818004050 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb82f9cc740 10 client.16950 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635a11557d0 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.578+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c036030 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb818010430 con 0x5635a10526b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 10 client.16950 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 20 client.16950 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a1031f60 0x5635a14314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 4 client.16950 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635a1387990 con 0x5635a1031f60 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb822d9d640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a1031f60 0x5635a14314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb822d9d640 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a1031f60 0x5635a14314d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc80bc581d9b2dc1 server_cookie=b1ce4ca8f6df676d in_seq=0 out_seq=0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.579+0000 7fb809ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.580+0000 7fb809ffb640 1 -- 172.21.15.94:0/3953774014 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635a1387990 con 0x5635a1031f60 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.580+0000 7fb809ffb640 10 client.16950 handle_command_reply: tid=0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.580+0000 7fb82f9cc740 1 client.16950 shutdown 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.580+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 msgr2=0x5635a1423ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.580+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb818012d00 tx=0x7fb81800f790 comp rx=0 tx=0).stop 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 shutdown_connections 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a1031f60 0x5635a14314d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a1073390 0x5635a142ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a10526b0 0x5635a1423ca0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb809ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/3953774014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a1423760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 >> 172.21.15.94:0/3953774014 conn(0x5635a10674e0 msgr2=0x5635a1410040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 shutdown_connections 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 1 -- 172.21.15.94:0/3953774014 wait complete. 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.581+0000 7fb82f9cc740 20 client.16950 trim_cache size 0 max 16384 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 Processor -- start 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 -- start start 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a140f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a142ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241a91b0 con 0x5635a1073390 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241aacf0 con 0x5635a10526b0 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241a9880 con 0x5635a104a4d0 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44926/0 (socket says 172.21.15.94:44926) 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 -- 172.21.15.94:0/2279409174 learned_addr learned my addr 172.21.15.94:0/2279409174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a140f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb821d9b640 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a142ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 -- 172.21.15.94:0/2279409174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 msgr2=0x5635a140f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a140f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 -- 172.21.15.94:0/2279409174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 msgr2=0x5635a142ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a142ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 -- 172.21.15.94:0/2279409174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635a141f070 con 0x5635a1073390 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.596+0000 7fb822d9d640 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000f80 tx=0x7fb81c02f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d063577d1325d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb80bfff640 1 -- 172.21.15.94:0/2279409174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0040c0 con 0x5635a1073390 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb80bfff640 1 -- 172.21.15.94:0/2279409174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c036040 con 0x5635a1073390 2024-04-25T13:45:39.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb80bfff640 1 -- 172.21.15.94:0/2279409174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c004840 con 0x5635a1073390 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb82f9cc740 1 -- 172.21.15.94:0/2279409174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 msgr2=0x5635a140fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000f80 tx=0x7fb81c02f4c0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.597+0000 7fb82f9cc740 1 -- 172.21.15.94:0/2279409174 shutdown_connections 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a140fb50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a142ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/2279409174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a140f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- 172.21.15.94:0/2279409174 >> 172.21.15.94:0/2279409174 conn(0x5635a10674e0 msgr2=0x5635a1421d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- 172.21.15.94:0/2279409174 shutdown_connections 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- 172.21.15.94:0/2279409174 wait complete. 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- start start 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 Processor -- start 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a14280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a1428620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241aa620 con 0x5635a1073390 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241a9f50 con 0x5635a10526b0 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241a7190 con 0x5635a104a4d0 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb82f9cc740 10 client.0 _ll_register_callbacks cb 0x7fb818009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb822d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44934/0 (socket says 172.21.15.94:44934) 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb821d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a1428620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb822d9d640 1 -- 172.21.15.94:0/1070306005 learned_addr learned my addr 172.21.15.94:0/1070306005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.598+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a14280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 -- 172.21.15.94:0/1070306005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 msgr2=0x5635a14280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a14280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 -- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 msgr2=0x5635a1428620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a1428620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 -- 172.21.15.94:0/1070306005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635a1429ac0 con 0x5635a1073390 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb822d9d640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb81c031b50 tx=0x7fb81c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ea110e1a95c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c058070 con 0x5635a1073390 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb81c05f070 con 0x5635a1073390 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c036040 con 0x5635a1073390 2024-04-25T13:45:39.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb81c004460 con 0x5635a1073390 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb82f9cc740 10 client.16956 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635a142ad40 con 0x5635a1073390 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c036040 con 0x5635a1073390 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.599+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb81c065040 con 0x5635a1073390 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 10 client.16956 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 20 client.16956 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a10073a0 0x5635a149bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 4 client.16956 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635a1381800 con 0x5635a10073a0 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb821d9b640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a10073a0 0x5635a149bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb809ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.600+0000 7fb821d9b640 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a10073a0 0x5635a149bc40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12c2885cf841241 server_cookie=8968192f7cfceeb8 in_seq=0 out_seq=0 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.601+0000 7fb809ffb640 1 -- 172.21.15.94:0/1070306005 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635a1381800 con 0x5635a10073a0 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.601+0000 7fb809ffb640 10 client.16956 handle_command_reply: tid=0 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.601+0000 7fb82f9cc740 1 client.16956 shutdown 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 msgr2=0x5635a1429500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb81c031b50 tx=0x7fb81c009360 comp rx=0 tx=0).stop 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 shutdown_connections 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635a10073a0 0x5635a149bc40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a1073390 0x5635a1429500 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb809ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635a10526b0 0x5635a1428620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 --2- 172.21.15.94:0/1070306005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635a104a4d0 0x5635a14280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 >> 172.21.15.94:0/1070306005 conn(0x5635a10674e0 msgr2=0x5635a1413e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 shutdown_connections 2024-04-25T13:45:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 1 -- 172.21.15.94:0/1070306005 wait complete. 2024-04-25T13:45:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb82f9cc740 20 client.16956 trim_cache size 0 max 16384 2024-04-25T13:45:39.607 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 msgr2=0x7fb80c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 0x7fb80c0b2000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5635a1419940 tx=0x7fb81408f040 comp rx=0 tx=0).stop 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 msgr2=0x7fb82419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb814005d00 tx=0x7fb81400dde0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 shutdown_connections 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb80c0afaf0 0x7fb80c0b2000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8241050b0 0x7fb8241a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824102260 0x7fb82419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 --2- 172.21.15.94:0/1311190375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb824101760 0x7fb82419a180 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 >> 172.21.15.94:0/1311190375 conn(0x7fb8240fd380 msgr2=0x7fb824107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 shutdown_connections 2024-04-25T13:45:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:39.602+0000 7fb828b1c640 1 -- 172.21.15.94:0/1311190375 wait complete. 2024-04-25T13:45:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:39 smithi094 ceph-mon[29195]: pgmap v462: 97 pgs: 97 active+clean; 42 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-04-25T13:45:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:39 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2199793328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:39 smithi138 ceph-mon[32977]: pgmap v462: 97 pgs: 97 active+clean; 42 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-04-25T13:45:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2199793328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:39.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 362361 13.05 MB/sec execute 22 sec latency 8.461 ms 2024-04-25T13:45:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:39 smithi137 ceph-mon[32568]: pgmap v462: 97 pgs: 97 active+clean; 42 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-04-25T13:45:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:39 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2199793328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:45:40.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:45:40.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:40.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:45:40.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 -- 172.21.15.94:0/2821819705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77541070d0 msgr2=0x7f77541094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 --2- 172.21.15.94:0/2821819705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77541070d0 0x7f77541094f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7748009900 tx=0x7f774802a410 comp rx=0 tx=0).stop 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 -- 172.21.15.94:0/2821819705 shutdown_connections 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 --2- 172.21.15.94:0/2821819705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754109a30 0x7f775410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 --2- 172.21.15.94:0/2821819705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77541070d0 0x7f77541094f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 --2- 172.21.15.94:0/2821819705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7754101f20 0x7f7754106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 -- 172.21.15.94:0/2821819705 >> 172.21.15.94:0/2821819705 conn(0x7f77540fd380 msgr2=0x7f77540ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.735+0000 7f7759267640 1 -- 172.21.15.94:0/2821819705 shutdown_connections 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 -- 172.21.15.94:0/2821819705 wait complete. 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 Processor -- start 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 -- start start 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754101f20 0x7f775419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77541070d0 0x7f775419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f775410f6d0 con 0x7f7754109a30 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f775410f550 con 0x7f77541070d0 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.736+0000 7f7759267640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f775410f850 con 0x7f7754101f20 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754101f20 0x7f775419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77527fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77541070d0 0x7f775419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44950/0 (socket says 172.21.15.94:44950) 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 -- 172.21.15.94:0/3901473977 learned_addr learned my addr 172.21.15.94:0/3901473977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 -- 172.21.15.94:0/3901473977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754101f20 msgr2=0x7f775419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754101f20 0x7f775419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 -- 172.21.15.94:0/3901473977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77541070d0 msgr2=0x7f775419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77541070d0 0x7f775419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 -- 172.21.15.94:0/3901473977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77541a1dc0 con 0x7f7754109a30 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f77537fe640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e970 tx=0x7f774c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f773bfff640 1 -- 172.21.15.94:0/3901473977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00ccc0 con 0x7f7754109a30 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f773bfff640 1 -- 172.21.15.94:0/3901473977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774c00ce60 con 0x7f7754109a30 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77541a20b0 con 0x7f7754109a30 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f773bfff640 1 -- 172.21.15.94:0/3901473977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c010430 con 0x7f7754109a30 2024-04-25T13:45:40.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.737+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77541a2600 con 0x7f7754109a30 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f773bfff640 1 -- 172.21.15.94:0/3901473977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f774c0105d0 con 0x7f7754109a30 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 Processor -- start 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 -- start start 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1db1410 0x5590a21524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a2159920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5590a20b7b00 con 0x5590a1d88550 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590a2053920 con 0x5590a1d90730 2024-04-25T13:45:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77601f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590a20b3ac0 con 0x5590a1db1410 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f773bfff640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 0x7f773c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44956/0 (socket says 172.21.15.94:44956) 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77527fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1db1410 0x5590a21524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f7752ffd640 1 -- 172.21.15.94:0/46282521 learned_addr learned my addr 172.21.15.94:0/46282521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77537fe640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 0x7f773c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.740+0000 7f77527fc640 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a2159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f773bfff640 1 -- 172.21.15.94:0/3901473977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f774c0df090 con 0x7f7754109a30 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 -- 172.21.15.94:0/46282521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1db1410 msgr2=0x5590a21524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1db1410 0x5590a21524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 -- 172.21.15.94:0/46282521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 msgr2=0x5590a2159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a2159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 -- 172.21.15.94:0/46282521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a1e917e0 con 0x5590a1d88550 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f7752ffd640 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7744011470 tx=0x7f7744011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c74407736992e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f77537fe640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 0x7f773c0b2000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f774c0dce60 tx=0x7f774c075c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f77397fa640 1 -- 172.21.15.94:0/46282521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774401e070 con 0x5590a1d88550 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f77397fa640 1 -- 172.21.15.94:0/46282521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7744003c30 con 0x5590a1d88550 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.741+0000 7f77397fa640 1 -- 172.21.15.94:0/46282521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774400f850 con 0x5590a1d88550 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 -- 172.21.15.94:0/46282521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 msgr2=0x5590a2152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7744011470 tx=0x7f7744011930 comp rx=0 tx=0).stop 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 -- 172.21.15.94:0/46282521 shutdown_connections 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a2159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a2152a20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 --2- 172.21.15.94:0/46282521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1db1410 0x5590a21524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 -- 172.21.15.94:0/46282521 >> 172.21.15.94:0/46282521 conn(0x5590a1da5560 msgr2=0x5590a214de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 -- 172.21.15.94:0/46282521 shutdown_connections 2024-04-25T13:45:40.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.742+0000 7f77601f9740 1 -- 172.21.15.94:0/46282521 wait complete. 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 -- start start 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 Processor -- start 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d88550 0x5590a215c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a2163740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5590a215a370 con 0x5590a1db1410 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590a20515d0 con 0x5590a1d90730 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590a215a4f0 con 0x5590a1d88550 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77601f9740 10 client.0 _ll_register_callbacks cb 0x5590a203f080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a2163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57864/0 (socket says 172.21.15.94:57864) 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 -- 172.21.15.94:0/3561755088 learned_addr learned my addr 172.21.15.94:0/3561755088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f77527fc640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d88550 0x5590a215c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 -- 172.21.15.94:0/3561755088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d88550 msgr2=0x5590a215c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d88550 0x5590a215c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 -- 172.21.15.94:0/3561755088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 msgr2=0x5590a2163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a2163740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.743+0000 7f7752ffd640 1 -- 172.21.15.94:0/3561755088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a1ef1f70 con 0x5590a1d90730 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7752ffd640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7744009a00 tx=0x7f7744004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad1377b4fe0a68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774401e040 con 0x5590a1d90730 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7744003c30 con 0x5590a1d90730 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7744003a30 con 0x5590a1d90730 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7744003de0 con 0x5590a1d90730 2024-04-25T13:45:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 10 client.26450 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590a1d89170 con 0x5590a1d90730 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7744003c30 con 0x5590a1d90730 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7744021030 con 0x5590a1d90730 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 10 client.26450 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 10 client.26450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 20 client.26450 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.744+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d701f0 0x5590a2169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.745+0000 7f77601f9740 4 client.26450 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.745+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a20c5710 con 0x5590a1d701f0 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.745+0000 7f77527fc640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d701f0 0x5590a2169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.745+0000 7f7736ff5640 10 client.26450 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.745+0000 7f77527fc640 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d701f0 0x5590a2169960 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39747b40b6bea206 server_cookie=3573df813ede9a10 in_seq=0 out_seq=0 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f7736ff5640 1 -- 172.21.15.94:0/3561755088 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590a20c5710 con 0x5590a1d701f0 2024-04-25T13:45:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f7736ff5640 10 client.26450 handle_command_reply: tid=0 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 client.26450 shutdown 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 msgr2=0x5590a215c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7744009a00 tx=0x7f7744004300 comp rx=0 tx=0).stop 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 shutdown_connections 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d701f0 0x5590a2169960 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a2163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f7736ff5640 0 client.26450 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d90730 0x5590a215c7e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 --2- 172.21.15.94:0/3561755088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d88550 0x5590a215c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 >> 172.21.15.94:0/3561755088 conn(0x5590a1da5560 msgr2=0x5590a214cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 shutdown_connections 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 1 -- 172.21.15.94:0/3561755088 wait complete. 2024-04-25T13:45:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.746+0000 7f77601f9740 20 client.26450 trim_cache size 0 max 16384 2024-04-25T13:45:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.764+0000 7f77601f9740 1 Processor -- start 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.764+0000 7f77601f9740 1 -- start start 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d88550 0x5590a214dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a215a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ab5d0 con 0x5590a1db1410 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ad110 con 0x5590a1d88550 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77601f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77541abca0 con 0x5590a1d90730 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44972/0 (socket says 172.21.15.94:44972) 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 -- 172.21.15.94:0/3920484035 learned_addr learned my addr 172.21.15.94:0/3920484035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d88550 0x5590a214dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f7752ffd640 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a215a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 -- 172.21.15.94:0/3920484035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 msgr2=0x5590a215a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a215a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 -- 172.21.15.94:0/3920484035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d88550 msgr2=0x5590a214dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d88550 0x5590a214dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 -- 172.21.15.94:0/3920484035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a2157150 con 0x5590a1db1410 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.765+0000 7f77527fc640 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f774802a450 tx=0x7f7748031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecacb67ecb8db8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f7738ff9640 1 -- 172.21.15.94:0/3920484035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748035040 con 0x5590a1db1410 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f7738ff9640 1 -- 172.21.15.94:0/3920484035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774804e3e0 con 0x5590a1db1410 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f7738ff9640 1 -- 172.21.15.94:0/3920484035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003850 con 0x5590a1db1410 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 -- 172.21.15.94:0/3920484035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 msgr2=0x5590a215add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f774802a450 tx=0x7f7748031400 comp rx=0 tx=0).stop 2024-04-25T13:45:40.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 -- 172.21.15.94:0/3920484035 shutdown_connections 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1db1410 0x5590a215add0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a215a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 --2- 172.21.15.94:0/3920484035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1d88550 0x5590a214dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 -- 172.21.15.94:0/3920484035 >> 172.21.15.94:0/3920484035 conn(0x5590a1da5560 msgr2=0x5590a214d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 -- 172.21.15.94:0/3920484035 shutdown_connections 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.766+0000 7f77601f9740 1 -- 172.21.15.94:0/3920484035 wait complete. 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 -- start start 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 Processor -- start 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a21606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1db1410 0x5590a2161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77541aca40 con 0x5590a1d88550 2024-04-25T13:45:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ac370 con 0x5590a1db1410 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77541a95b0 con 0x5590a1d90730 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77601f9740 10 client.0 _ll_register_callbacks cb 0x7f774c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77527fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44976/0 (socket says 172.21.15.94:44976) 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77527fc640 1 -- 172.21.15.94:0/198812088 learned_addr learned my addr 172.21.15.94:0/198812088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f7752ffd640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a21606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.767+0000 7f77527fc640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1db1410 0x5590a2161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 -- 172.21.15.94:0/198812088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 msgr2=0x5590a21606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a21606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 -- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1db1410 msgr2=0x5590a2161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1db1410 0x5590a2161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 -- 172.21.15.94:0/198812088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a2161bd0 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77527fc640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f774802a930 tx=0x7f7748000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6052e05c4102e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748035040 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7748004450 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774804e3e0 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f774804e890 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77601f9740 10 client.16974 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590a2162e90 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774804e3e0 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.768+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f774803d6f0 con 0x5590a1d88550 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 20 client.16974 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d45630 0x5590a21d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 4 client.16974 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:45:40.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5590a20bf580 con 0x5590a1d45630 2024-04-25T13:45:40.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f7752ffd640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d45630 0x5590a21d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f7736ff5640 10 client.16974 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.769+0000 7f7752ffd640 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d45630 0x5590a21d41f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea98ad93f12f4ab server_cookie=48113f162845cee1 in_seq=0 out_seq=0 2024-04-25T13:45:40.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f7736ff5640 1 -- 172.21.15.94:0/198812088 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5590a20bf580 con 0x5590a1d45630 2024-04-25T13:45:40.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f7736ff5640 10 client.16974 handle_command_reply: tid=0 2024-04-25T13:45:40.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 client.16974 shutdown 2024-04-25T13:45:40.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 msgr2=0x5590a21601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f774802a930 tx=0x7f7748000bc0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 shutdown_connections 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5590a1d45630 0x5590a21d41f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590a1db1410 0x5590a2161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f7736ff5640 0 client.16974 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a1d90730 0x5590a21606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 --2- 172.21.15.94:0/198812088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590a1d88550 0x5590a21601a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 >> 172.21.15.94:0/198812088 conn(0x5590a1da5560 msgr2=0x5590a214c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 shutdown_connections 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 1 -- 172.21.15.94:0/198812088 wait complete. 2024-04-25T13:45:40.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.771+0000 7f77601f9740 20 client.16974 trim_cache size 0 max 16384 2024-04-25T13:45:40.777 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:40.777 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:45:40.777 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "5f9e1691-eadc-41ae-bef1-264994d4470b", 2024-04-25T13:45:40.777 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:45:40.777 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 msgr2=0x7f773c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 0x7f773c0b2000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f774c0dce60 tx=0x7f774c075c40 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 msgr2=0x7f77541a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e970 tx=0x7f774c00ed70 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 shutdown_connections 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f773c0afaf0 0x7f773c0b2000 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7754109a30 0x7f77541a1750 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77541070d0 0x7f775419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 --2- 172.21.15.94:0/3901473977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7754101f20 0x7f775419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 >> 172.21.15.94:0/3901473977 conn(0x7f77540fd380 msgr2=0x7f7754105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 shutdown_connections 2024-04-25T13:45:40.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.772+0000 7f7759267640 1 -- 172.21.15.94:0/3901473977 wait complete. 2024-04-25T13:45:40.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:45:40.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:45:40.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 365446 13.25 MB/sec execute 23 sec latency 13.420 ms 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 -- 172.21.15.94:0/2879362828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104108920 msgr2=0x7f310410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 --2- 172.21.15.94:0/2879362828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104108920 0x7f310410ad40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009ca0 tx=0x7f30fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 -- 172.21.15.94:0/2879362828 shutdown_connections 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 --2- 172.21.15.94:0/2879362828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104108920 0x7f310410ad40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 --2- 172.21.15.94:0/2879362828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104103c20 0x7f3104104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 --2- 172.21.15.94:0/2879362828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31041029e0 0x7f31041036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 -- 172.21.15.94:0/2879362828 >> 172.21.15.94:0/2879362828 conn(0x7f31040fdbd0 msgr2=0x7f3104100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 -- 172.21.15.94:0/2879362828 shutdown_connections 2024-04-25T13:45:40.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.901+0000 7f3108eb6640 1 -- 172.21.15.94:0/2879362828 wait complete. 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 Processor -- start 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 -- start start 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104103c20 0x7f310419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104108920 0x7f31041a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f310410e510 con 0x7f31041029e0 2024-04-25T13:45:40.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f310410e390 con 0x7f3104108920 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3108eb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f310410e690 con 0x7f3104103c20 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44992/0 (socket says 172.21.15.94:44992) 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 -- 172.21.15.94:0/71430049 learned_addr learned my addr 172.21.15.94:0/71430049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f30fbfff640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104103c20 0x7f310419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f3102d9d640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104108920 0x7f31041a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 -- 172.21.15.94:0/71430049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104103c20 msgr2=0x7f310419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104103c20 0x7f310419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 -- 172.21.15.94:0/71430049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104108920 msgr2=0x7f31041a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104108920 0x7f31041a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.902+0000 7f310259c640 1 -- 172.21.15.94:0/71430049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041a2030 con 0x7f31041029e0 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f310259c640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9f0 tx=0x7f30f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f30fb7fe640 1 -- 172.21.15.94:0/71430049 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400ccc0 con 0x7f31041029e0 2024-04-25T13:45:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f30fb7fe640 1 -- 172.21.15.94:0/71430049 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40035f0 con 0x7f31041029e0 2024-04-25T13:45:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041a2320 con 0x7f31041029e0 2024-04-25T13:45:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f30fb7fe640 1 -- 172.21.15.94:0/71430049 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400ce60 con 0x7f31041029e0 2024-04-25T13:45:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.903+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a2820 con 0x7f31041029e0 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 Processor -- start 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 -- start start 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f30fb7fe640 1 -- 172.21.15.94:0/71430049 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f30f4003aa0 con 0x7f31041029e0 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e42ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e42cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e390d80 con 0x55fc1e0614d0 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e34a8b0 con 0x55fc1e08a390 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310fd69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e38cd40 con 0x55fc1e0696b0 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f3102d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e42ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45004/0 (socket says 172.21.15.94:45004) 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f310259c640 1 -- 172.21.15.94:0/1775972505 learned_addr learned my addr 172.21.15.94:0/1775972505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.906+0000 7f30fbfff640 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e42cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 -- 172.21.15.94:0/1775972505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 msgr2=0x55fc1e42cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e42cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 -- 172.21.15.94:0/1775972505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 msgr2=0x55fc1e42ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e42ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 -- 172.21.15.94:0/1775972505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1e31ba40 con 0x55fc1e0614d0 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f3102d9d640 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e42ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 0x7f30f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f310259c640 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f30f40e2f70 tx=0x7f30f40e71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2042327933a9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f3102d9d640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 0x7f30f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f30f8ff9640 1 -- 172.21.15.94:0/1775972505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a32f0 con 0x55fc1e0614d0 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f30fb7fe640 1 -- 172.21.15.94:0/71430049 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f30f40deff0 con 0x7f31041029e0 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f30f8ff9640 1 -- 172.21.15.94:0/1775972505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40a6f40 con 0x55fc1e0614d0 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.907+0000 7f30f8ff9640 1 -- 172.21.15.94:0/1775972505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a6ba0 con 0x55fc1e0614d0 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 -- 172.21.15.94:0/1775972505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 msgr2=0x55fc1e42bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f3102d9d640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 0x7f30f00b2000 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55fc1e42cab0 tx=0x7f30fc02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f30f40e2f70 tx=0x7f30f40e71a0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 -- 172.21.15.94:0/1775972505 shutdown_connections 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e42cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e42bfe0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 --2- 172.21.15.94:0/1775972505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e42ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 -- 172.21.15.94:0/1775972505 >> 172.21.15.94:0/1775972505 conn(0x55fc1e07e4e0 msgr2=0x55fc1e427270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 -- 172.21.15.94:0/1775972505 shutdown_connections 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.908+0000 7f310fd69740 1 -- 172.21.15.94:0/1775972505 wait complete. 2024-04-25T13:45:40.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 -- start start 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 Processor -- start 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e435e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e43cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e42d720 con 0x55fc1e0614d0 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e317d10 con 0x55fc1e08a390 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc1e42d8a0 con 0x55fc1e0696b0 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310fd69740 10 client.0 _ll_register_callbacks cb 0x55fc1e344210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45014/0 (socket says 172.21.15.94:45014) 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 -- 172.21.15.94:0/2778826746 learned_addr learned my addr 172.21.15.94:0/2778826746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f30fbfff640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e435e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f3102d9d640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e43cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 -- 172.21.15.94:0/2778826746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 msgr2=0x55fc1e435e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e435e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 -- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 msgr2=0x55fc1e43cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e43cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.909+0000 7f310259c640 1 -- 172.21.15.94:0/2778826746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1e16c7d0 con 0x55fc1e0614d0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310259c640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f30f40a34b0 tx=0x7f30f401cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadd627b2d66ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a6d80 con 0x55fc1e0614d0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f30f40e2170 con 0x55fc1e0614d0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f40a6f40 con 0x55fc1e0614d0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30f40a7200 con 0x55fc1e0614d0 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 10 client.16986 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc1e0a23f0 con 0x55fc1e0614d0 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40a6f40 con 0x55fc1e0614d0 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30f40e2370 con 0x55fc1e0614d0 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 10 client.16986 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 20 client.16986 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e048f60 0x55fc1e442dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 4 client.16986 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1e39e990 con 0x55fc1e048f60 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.910+0000 7f30fbfff640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e048f60 0x55fc1e442dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.911+0000 7f30e9ffb640 10 client.16986 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.911+0000 7f30fbfff640 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e048f60 0x55fc1e442dd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8239908d1d685d server_cookie=3d9ef4a22722a0fb in_seq=0 out_seq=0 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f30e9ffb640 1 -- 172.21.15.94:0/2778826746 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc1e39e990 con 0x55fc1e048f60 2024-04-25T13:45:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f30e9ffb640 10 client.16986 handle_command_reply: tid=0 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 client.16986 shutdown 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 msgr2=0x55fc1e4358e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f30f40a34b0 tx=0x7f30f401cf90 comp rx=0 tx=0).stop 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 shutdown_connections 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e048f60 0x55fc1e442dd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f30e9ffb640 0 client.16986 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e08a390 0x55fc1e43cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e435e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 --2- 172.21.15.94:0/2778826746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e0614d0 0x55fc1e4358e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 >> 172.21.15.94:0/2778826746 conn(0x55fc1e07e4e0 msgr2=0x55fc1e42a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 shutdown_connections 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 1 -- 172.21.15.94:0/2778826746 wait complete. 2024-04-25T13:45:40.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.912+0000 7f310fd69740 20 client.16986 trim_cache size 0 max 16384 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.930+0000 7f310fd69740 1 Processor -- start 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.930+0000 7f310fd69740 1 -- start start 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0614d0 0x55fc1e4260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e434ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31041ab590 con 0x55fc1e048f60 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31041ad0d0 con 0x55fc1e0614d0 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310fd69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31041abc60 con 0x55fc1e0696b0 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45030/0 (socket says 172.21.15.94:45030) 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 -- 172.21.15.94:0/3105837978 learned_addr learned my addr 172.21.15.94:0/3105837978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f30fbfff640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0614d0 0x55fc1e4260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f3102d9d640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e434ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 -- 172.21.15.94:0/3105837978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 msgr2=0x55fc1e434ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e434ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 -- 172.21.15.94:0/3105837978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0614d0 msgr2=0x55fc1e4260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0614d0 0x55fc1e4260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 -- 172.21.15.94:0/3105837978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1e0620f0 con 0x55fc1e048f60 2024-04-25T13:45:40.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f3102d9d640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e434ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f310259c640 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f30f40e76a0 tx=0x7f30f401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c0f3b1a47e58c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f30ebfff640 1 -- 172.21.15.94:0/3105837978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a6d80 con 0x55fc1e048f60 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f30ebfff640 1 -- 172.21.15.94:0/3105837978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40a6f40 con 0x55fc1e048f60 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.931+0000 7f30ebfff640 1 -- 172.21.15.94:0/3105837978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a6ba0 con 0x55fc1e048f60 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- 172.21.15.94:0/3105837978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 msgr2=0x55fc1e434570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f30f40e76a0 tx=0x7f30f401d000 comp rx=0 tx=0).stop 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- 172.21.15.94:0/3105837978 shutdown_connections 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0696b0 0x55fc1e434ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0614d0 0x55fc1e4260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 --2- 172.21.15.94:0/3105837978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e434570 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- 172.21.15.94:0/3105837978 >> 172.21.15.94:0/3105837978 conn(0x55fc1e07e4e0 msgr2=0x55fc1e4259c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- 172.21.15.94:0/3105837978 shutdown_connections 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- 172.21.15.94:0/3105837978 wait complete. 2024-04-25T13:45:40.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.932+0000 7f310fd69740 1 -- start start 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 Processor -- start 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0614d0 0x55fc1e43a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0696b0 0x55fc1e43b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31041aca00 con 0x55fc1e048f60 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31041ac330 con 0x55fc1e0696b0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31041a9570 con 0x55fc1e0614d0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310fd69740 10 client.0 _ll_register_callbacks cb 0x7f30f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45034/0 (socket says 172.21.15.94:45034) 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 -- 172.21.15.94:0/3000784223 learned_addr learned my addr 172.21.15.94:0/3000784223 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f3102d9d640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0696b0 0x55fc1e43b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f30fbfff640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0614d0 0x55fc1e43a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 -- 172.21.15.94:0/3000784223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0614d0 msgr2=0x55fc1e43a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0614d0 0x55fc1e43a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 -- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0696b0 msgr2=0x55fc1e43b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0696b0 0x55fc1e43b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 -- 172.21.15.94:0/3000784223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1e3514b0 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f310259c640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f30f40a7130 tx=0x7f30f40a79a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290783efb4b05f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.933+0000 7f30e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40a6ba0 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f30f40a6690 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f40a6f40 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30f40a6d80 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 10 client.16992 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc1e3514b0 con 0x55fc1e048f60 2024-04-25T13:45:40.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40a6f40 con 0x55fc1e048f60 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30f4003fb0 con 0x55fc1e048f60 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 10 client.16992 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 20 client.16992 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e08a390 0x55fc1e4ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 4 client.16992 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc1e398800 con 0x55fc1e08a390 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.934+0000 7f30fbfff640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e08a390 0x55fc1e4ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.935+0000 7f30e9ffb640 10 client.16992 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.935+0000 7f30fbfff640 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e08a390 0x55fc1e4ad4e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d879541b291fd0fd server_cookie=278b5c893ea38676 in_seq=0 out_seq=0 2024-04-25T13:45:40.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f30e9ffb640 1 -- 172.21.15.94:0/3000784223 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fc1e398800 con 0x55fc1e08a390 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f30e9ffb640 10 client.16992 handle_command_reply: tid=0 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 client.16992 shutdown 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 msgr2=0x55fc1e439d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f30f40a7130 tx=0x7f30f40a79a0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 shutdown_connections 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc1e08a390 0x55fc1e4ad4e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc1e0696b0 0x55fc1e43b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc1e0614d0 0x55fc1e43a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f30e9ffb640 0 client.16992 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 --2- 172.21.15.94:0/3000784223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc1e048f60 0x55fc1e439d80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 >> 172.21.15.94:0/3000784223 conn(0x55fc1e07e4e0 msgr2=0x55fc1e42a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 shutdown_connections 2024-04-25T13:45:40.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 1 -- 172.21.15.94:0/3000784223 wait complete. 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.937+0000 7f310fd69740 20 client.16992 trim_cache size 0 max 16384 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "5f9e1691-eadc-41ae-bef1-264994d4470b": { 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "5f9e1691-eadc-41ae-bef1-264994d4470b", 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:45:40.943 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:45:40.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 msgr2=0x7f30f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 0x7f30f00b2000 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55fc1e42cab0 tx=0x7f30fc02f310 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 msgr2=0x7f310406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9f0 tx=0x7f30f400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 shutdown_connections 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30f00afaf0 0x7f30f00b2000 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3104108920 0x7f31041a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104103c20 0x7f310419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 --2- 172.21.15.94:0/71430049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31041029e0 0x7f310406a960 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 >> 172.21.15.94:0/71430049 conn(0x7f31040fdbd0 msgr2=0x7f3104104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 shutdown_connections 2024-04-25T13:45:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:45:40.938+0000 7f3108eb6640 1 -- 172.21.15.94:0/71430049 wait complete. 2024-04-25T13:45:40.948 INFO:tasks.cephfs.filesystem:scrub status for tag:5f9e1691-eadc-41ae-bef1-264994d4470b - {'path': '/', 'tag': '5f9e1691-eadc-41ae-bef1-264994d4470b', 'options': 'recursive,force'} 2024-04-25T13:45:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:41 smithi094 ceph-mon[29195]: pgmap v463: 97 pgs: 97 active+clean; 40 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-04-25T13:45:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:41 smithi138 ceph-mon[32977]: pgmap v463: 97 pgs: 97 active+clean; 40 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-04-25T13:45:41.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 366978 12.98 MB/sec execute 24 sec latency 88.802 ms 2024-04-25T13:45:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:41 smithi137 ceph-mon[32568]: pgmap v463: 97 pgs: 97 active+clean; 40 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-04-25T13:45:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:42 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:45:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:45:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:42 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:45:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:45:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:42.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 369467 13.01 MB/sec execute 25 sec latency 10.339 ms 2024-04-25T13:45:42.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:42 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:45:42.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:45:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:45:43.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:43 smithi094 ceph-mon[29195]: pgmap v464: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-04-25T13:45:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:43 smithi138 ceph-mon[32977]: pgmap v464: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-04-25T13:45:43.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 372517 13.19 MB/sec execute 26 sec latency 13.703 ms 2024-04-25T13:45:43.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:43 smithi137 ceph-mon[32568]: pgmap v464: 97 pgs: 97 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-04-25T13:45:44.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 374801 13.02 MB/sec execute 27 sec latency 17.570 ms 2024-04-25T13:45:45.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:45 smithi094 ceph-mon[29195]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-25T13:45:45.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:45 smithi138 ceph-mon[32977]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-25T13:45:45.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 376794 13.00 MB/sec execute 28 sec latency 18.025 ms 2024-04-25T13:45:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:45 smithi137 ceph-mon[32568]: pgmap v465: 97 pgs: 97 active+clean; 44 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-25T13:45:46.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:46 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:45:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:46 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:45:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.776+0000 7f47ec99d640 1 -- 172.21.15.94:0/2250864827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8108cd0 msgr2=0x7f47e810b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.776+0000 7f47ec99d640 1 --2- 172.21.15.94:0/2250864827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8108cd0 0x7f47e810b170 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009900 tx=0x7f47dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:45:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 -- 172.21.15.94:0/2250864827 shutdown_connections 2024-04-25T13:45:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 --2- 172.21.15.94:0/2250864827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e810b800 0x7f47e810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 --2- 172.21.15.94:0/2250864827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8108cd0 0x7f47e810b170 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 --2- 172.21.15.94:0/2250864827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8103e20 0x7f47e8104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 -- 172.21.15.94:0/2250864827 >> 172.21.15.94:0/2250864827 conn(0x7f47e80fd1c0 msgr2=0x7f47e80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 -- 172.21.15.94:0/2250864827 shutdown_connections 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 -- 172.21.15.94:0/2250864827 wait complete. 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 Processor -- start 2024-04-25T13:45:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.777+0000 7f47ec99d640 1 -- start start 2024-04-25T13:45:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e8108cd0 0x7f47e819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e810b800 0x7f47e81a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111680 con 0x7f47e8103e20 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111500 con 0x7f47e8108cd0 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111800 con 0x7f47e810b800 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38450/0 (socket says 172.21.15.94:38450) 2024-04-25T13:45:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 -- 172.21.15.94:0/3197008070 learned_addr learned my addr 172.21.15.94:0/3197008070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e8108cd0 0x7f47e819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e7fff640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e810b800 0x7f47e81a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 -- 172.21.15.94:0/3197008070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e810b800 msgr2=0x7f47e81a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e810b800 0x7f47e81a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 -- 172.21.15.94:0/3197008070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e8108cd0 msgr2=0x7f47e819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e8108cd0 0x7f47e819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 -- 172.21.15.94:0/3197008070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e81a3d90 con 0x7f47e8103e20 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e77fe640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f47d800e970 tx=0x7f47d800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e4ff9640 1 -- 172.21.15.94:0/3197008070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d800cd80 con 0x7f47e8103e20 2024-04-25T13:45:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e81a4080 con 0x7f47e8103e20 2024-04-25T13:45:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e4ff9640 1 -- 172.21.15.94:0/3197008070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d80035f0 con 0x7f47e8103e20 2024-04-25T13:45:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.778+0000 7f47e4ff9640 1 -- 172.21.15.94:0/3197008070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8010690 con 0x7f47e8103e20 2024-04-25T13:45:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.779+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a4620 con 0x7f47e8103e20 2024-04-25T13:45:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47e4ff9640 1 -- 172.21.15.94:0/3197008070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f47d80108f0 con 0x7f47e8103e20 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47f4b8c740 1 Processor -- start 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47f4b8c740 1 -- start start 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47e4ff9640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 0x7f47d00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.781+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b78f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559ae4ad7460 con 0x559ae47d11b0 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559ae4a91590 con 0x559ae47b04d0 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ae4ad3420 con 0x559ae47a82f0 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e7fff640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 0x7f47d00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38456/0 (socket says 172.21.15.94:38456) 2024-04-25T13:45:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 -- 172.21.15.94:0/514642666 learned_addr learned my addr 172.21.15.94:0/514642666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e7fff640 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b78f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e4ff9640 1 -- 172.21.15.94:0/3197008070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f47d80defe0 con 0x7f47e8103e20 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 -- 172.21.15.94:0/514642666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 msgr2=0x559ae4b72090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b72090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 -- 172.21.15.94:0/514642666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 msgr2=0x559ae4b78f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b78f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 -- 172.21.15.94:0/514642666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ae4a9e0d0 con 0x559ae47d11b0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e7fff640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 0x7f47d00b1ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005c40 tx=0x7f47e0009df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47e77fe640 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f47d80a7100 tx=0x7f47d80a71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a5f883fe28248b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47cdffb640 1 -- 172.21.15.94:0/514642666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80c1ed0 con 0x559ae47d11b0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47cdffb640 1 -- 172.21.15.94:0/514642666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d80a6f10 con 0x559ae47d11b0 2024-04-25T13:45:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.782+0000 7f47cdffb640 1 -- 172.21.15.94:0/514642666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8014230 con 0x559ae47d11b0 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 -- 172.21.15.94:0/514642666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 msgr2=0x559ae4b71b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f47d80a7100 tx=0x7f47d80a71d0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 -- 172.21.15.94:0/514642666 shutdown_connections 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b78f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/514642666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47d11b0 0x559ae4b71b10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 -- 172.21.15.94:0/514642666 >> 172.21.15.94:0/514642666 conn(0x559ae47c5300 msgr2=0x559ae4b6d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 -- 172.21.15.94:0/514642666 shutdown_connections 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.783+0000 7f47f4b8c740 1 -- 172.21.15.94:0/514642666 wait complete. 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 -- start start 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 Processor -- start 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47d11b0 0x559ae4b82bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559ae4b79920 con 0x559ae47a82f0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559ae4a5eb60 con 0x559ae47b04d0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ae4b79aa0 con 0x559ae47d11b0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47f4b8c740 10 client.0 _ll_register_callbacks cb 0x559ae4a832b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38460/0 (socket says 172.21.15.94:38460) 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47e77fe640 1 -- 172.21.15.94:0/1985582583 learned_addr learned my addr 172.21.15.94:0/1985582583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47e7fff640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47d11b0 0x559ae4b82bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.784+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 -- 172.21.15.94:0/1985582583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47d11b0 msgr2=0x559ae4b82bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47d11b0 0x559ae4b82bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 -- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 msgr2=0x559ae4b7bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b7bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 -- 172.21.15.94:0/1985582583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ae47e9210 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ab970 tx=0x7f47d80a7490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fcf89e3a5c6341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80c4e00 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f47d8003aa0 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d80a6f10 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47d80c1cf0 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 10 client.17004 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ae48b5620 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d80a6f10 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f47d80afdd0 con 0x559ae47a82f0 2024-04-25T13:45:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 10 client.17004 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 20 client.17004 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae478fd80 0x559ae4b89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 4 client.17004 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.785+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ae4ae5070 con 0x559ae478fd80 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.786+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae478fd80 0x559ae4b89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.786+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae478fd80 0x559ae4b89010 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2714e17c7b6c8ee0 server_cookie=6bd56d02e27bc2b9 in_seq=0 out_seq=0 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.786+0000 7f47c6ffd640 10 client.17004 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47c6ffd640 1 -- 172.21.15.94:0/1985582583 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ae4ae5070 con 0x559ae478fd80 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47c6ffd640 10 client.17004 handle_command_reply: tid=0 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 client.17004 shutdown 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 msgr2=0x559ae4b7b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ab970 tx=0x7f47d80a7490 comp rx=0 tx=0).stop 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 shutdown_connections 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae478fd80 0x559ae4b89010 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47d11b0 0x559ae4b82bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47c6ffd640 0 client.17004 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b7bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1985582583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47a82f0 0x559ae4b7b720 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 >> 172.21.15.94:0/1985582583 conn(0x559ae47c5300 msgr2=0x559ae4b703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 shutdown_connections 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1985582583 wait complete. 2024-04-25T13:45:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.787+0000 7f47f4b8c740 20 client.17004 trim_cache size 0 max 16384 2024-04-25T13:45:46.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 379667 13.15 MB/sec execute 29 sec latency 81.319 ms 2024-04-25T13:45:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 Processor -- start 2024-04-25T13:45:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 -- start start 2024-04-25T13:45:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ad650 con 0x559ae478fd80 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81af190 con 0x559ae47b04d0 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81add20 con 0x559ae47a82f0 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38476/0 (socket says 172.21.15.94:38476) 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 -- 172.21.15.94:0/1437221764 learned_addr learned my addr 172.21.15.94:0/1437221764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e7fff640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 -- 172.21.15.94:0/1437221764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 msgr2=0x559ae4b82b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 -- 172.21.15.94:0/1437221764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 msgr2=0x559ae4b8ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b8ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 -- 172.21.15.94:0/1437221764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ae4b7c790 con 0x559ae478fd80 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e7fff640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.804+0000 7f47e77fe640 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f47d80a7100 tx=0x7f47d80a71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9d2b4a9c9968e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47cd7fa640 1 -- 172.21.15.94:0/1437221764 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80ab7b0 con 0x559ae478fd80 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47cd7fa640 1 -- 172.21.15.94:0/1437221764 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d80a6f10 con 0x559ae478fd80 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47cd7fa640 1 -- 172.21.15.94:0/1437221764 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80aff90 con 0x559ae478fd80 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1437221764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 msgr2=0x559ae4b8e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f47d80a7100 tx=0x7f47d80a71d0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1437221764 shutdown_connections 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae47b04d0 0x559ae4b8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/1437221764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae478fd80 0x559ae4b8e6f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.805+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1437221764 >> 172.21.15.94:0/1437221764 conn(0x559ae47c5300 msgr2=0x559ae4b7a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1437221764 shutdown_connections 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- 172.21.15.94:0/1437221764 wait complete. 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- start start 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 Processor -- start 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae478fd80 0x559ae4b81bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81aeac0 con 0x559ae47b04d0 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ae3f0 con 0x559ae478fd80 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ab630 con 0x559ae47a82f0 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47f4b8c740 10 client.0 _ll_register_callbacks cb 0x7f47e0012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.806+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38478/0 (socket says 172.21.15.94:38478) 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae478fd80 0x559ae4b81bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 -- 172.21.15.94:0/195610982 learned_addr learned my addr 172.21.15.94:0/195610982 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 -- 172.21.15.94:0/195610982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 msgr2=0x559ae4b82100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 -- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae478fd80 msgr2=0x559ae4b81bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae478fd80 0x559ae4b81bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 -- 172.21.15.94:0/195610982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ae4bef800 con 0x559ae47b04d0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e6ffd640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47e7fff640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f47e0010db0 tx=0x7f47e001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5f9d3eff1e26d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e002b470 con 0x559ae47b04d0 2024-04-25T13:45:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f47e002b610 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d80a6f10 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47e002b810 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47f4b8c740 10 client.17010 fetch_fsmap learned FSMap version 25 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.807+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ae4bf0b10 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d80a6f10 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f47e001f050 con 0x559ae47b04d0 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 20 client.17010 populate_metadata read hostname 'smithi094' 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae47d11b0 0x559ae4bf35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 4 client.17010 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:45:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ae4adeee0 con 0x559ae47d11b0 2024-04-25T13:45:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47e77fe640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae47d11b0 0x559ae4bf35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47c6ffd640 10 client.17010 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.808+0000 7f47e77fe640 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae47d11b0 0x559ae4bf35b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319311799674efc5 server_cookie=dd926a26f8a8180a in_seq=0 out_seq=0 2024-04-25T13:45:46.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:46 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47c6ffd640 1 -- 172.21.15.94:0/195610982 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ae4adeee0 con 0x559ae47d11b0 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47c6ffd640 10 client.17010 handle_command_reply: tid=0 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 client.17010 shutdown 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 msgr2=0x559ae4bef290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f47e0010db0 tx=0x7f47e001c000 comp rx=0 tx=0).stop 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 shutdown_connections 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559ae47d11b0 0x559ae4bf35b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559ae47b04d0 0x559ae4bef290 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ae47a82f0 0x559ae4b82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 --2- 172.21.15.94:0/195610982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae478fd80 0x559ae4b81bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 >> 172.21.15.94:0/195610982 conn(0x559ae47c5300 msgr2=0x559ae4b702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47c6ffd640 0 client.17010 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 shutdown_connections 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 1 -- 172.21.15.94:0/195610982 wait complete. 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.958+0000 7f47f4b8c740 20 client.17010 trim_cache size 0 max 16384 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:45:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 msgr2=0x7f47d00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 0x7f47d00b1ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005c40 tx=0x7f47e0009df0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 msgr2=0x7f47e819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f47d800e970 tx=0x7f47d800ee30 comp rx=0 tx=0).stop 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 shutdown_connections 2024-04-25T13:45:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47d00afae0 0x7f47d00b1ff0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e810b800 0x7f47e81a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e8108cd0 0x7f47e819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 --2- 172.21.15.94:0/3197008070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e8103e20 0x7f47e819c250 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 >> 172.21.15.94:0/3197008070 conn(0x7f47e80fd1c0 msgr2=0x7f47e8109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 shutdown_connections 2024-04-25T13:45:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:45:46.959+0000 7f47ec99d640 1 -- 172.21.15.94:0/3197008070 wait complete. 2024-04-25T13:45:47.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:47 smithi094 ceph-mon[29195]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:45:47.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:47 smithi138 ceph-mon[32977]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:45:47.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 380973 12.93 MB/sec execute 30 sec latency 71.391 ms 2024-04-25T13:45:47.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:47 smithi137 ceph-mon[32568]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:45:48.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 383576 12.96 MB/sec execute 31 sec latency 10.552 ms 2024-04-25T13:45:49.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:49 smithi094 ceph-mon[29195]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-04-25T13:45:49.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:49 smithi138 ceph-mon[32977]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-04-25T13:45:49.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 386498 13.01 MB/sec execute 32 sec latency 41.806 ms 2024-04-25T13:45:49.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:49 smithi137 ceph-mon[32568]: pgmap v467: 97 pgs: 97 active+clean; 48 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 311 op/s 2024-04-25T13:45:50.794 INFO:tasks.workunit.client.0.smithi094.stdout: 1 388584 12.92 MB/sec execute 33 sec latency 48.515 ms 2024-04-25T13:45:51.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:51 smithi094 ceph-mon[29195]: pgmap v468: 97 pgs: 97 active+clean; 44 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:45:51.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:51 smithi138 ceph-mon[32977]: pgmap v468: 97 pgs: 97 active+clean; 44 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:45:51.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 390799 12.95 MB/sec execute 34 sec latency 110.419 ms 2024-04-25T13:45:51.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:51 smithi137 ceph-mon[32568]: pgmap v468: 97 pgs: 97 active+clean; 44 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:45:52.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 393847 13.08 MB/sec execute 35 sec latency 18.304 ms 2024-04-25T13:45:53.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:53 smithi094 ceph-mon[29195]: pgmap v469: 97 pgs: 97 active+clean; 39 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-04-25T13:45:53.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:53 smithi138 ceph-mon[32977]: pgmap v469: 97 pgs: 97 active+clean; 39 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-04-25T13:45:53.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 396106 12.95 MB/sec execute 36 sec latency 21.744 ms 2024-04-25T13:45:53.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:53 smithi137 ceph-mon[32568]: pgmap v469: 97 pgs: 97 active+clean; 39 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 418 op/s 2024-04-25T13:45:54.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 398193 12.95 MB/sec execute 37 sec latency 12.676 ms 2024-04-25T13:45:55.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:55 smithi094 ceph-mon[29195]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-04-25T13:45:55.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:55 smithi138 ceph-mon[32977]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-04-25T13:45:55.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 401195 13.09 MB/sec execute 38 sec latency 11.426 ms 2024-04-25T13:45:55.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:55 smithi137 ceph-mon[32568]: pgmap v470: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-04-25T13:45:56.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 403261 12.94 MB/sec execute 39 sec latency 89.291 ms 2024-04-25T13:45:57.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:57 smithi094 ceph-mon[29195]: pgmap v471: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-25T13:45:57.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:57 smithi138 ceph-mon[32977]: pgmap v471: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-25T13:45:57.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 405275 12.93 MB/sec execute 40 sec latency 17.517 ms 2024-04-25T13:45:57.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:57 smithi137 ceph-mon[32568]: pgmap v471: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-04-25T13:45:58.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 408249 13.06 MB/sec execute 41 sec latency 12.266 ms 2024-04-25T13:45:59.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:45:59 smithi094 ceph-mon[29195]: pgmap v472: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-04-25T13:45:59.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:45:59 smithi138 ceph-mon[32977]: pgmap v472: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-04-25T13:45:59.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 410946 13.06 MB/sec execute 42 sec latency 22.957 ms 2024-04-25T13:45:59.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:45:59 smithi137 ceph-mon[32568]: pgmap v472: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- 172.21.15.94:0/589308917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654103990 msgr2=0x7fa654103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 --2- 172.21.15.94:0/589308917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654103990 0x7fa654103e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa64802a410 comp rx=0 tx=0).stop 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- 172.21.15.94:0/589308917 shutdown_connections 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 --2- 172.21.15.94:0/589308917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 0x7fa65410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 --2- 172.21.15.94:0/589308917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654103990 0x7fa654103e20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 --2- 172.21.15.94:0/589308917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654102fb0 0x7fa6541033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- 172.21.15.94:0/589308917 >> 172.21.15.94:0/589308917 conn(0x7fa6540fd390 msgr2=0x7fa6540ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- 172.21.15.94:0/589308917 shutdown_connections 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- 172.21.15.94:0/589308917 wait complete. 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 Processor -- start 2024-04-25T13:46:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.061+0000 7fa65a651640 1 -- start start 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654103990 0x7fa6541a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 0x7fa6541a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410e580 con 0x7fa654102fb0 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410e400 con 0x7fa6541044b0 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410e700 con 0x7fa654103990 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51344/0 (socket says 172.21.15.94:51344) 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 -- 172.21.15.94:0/3793472409 learned_addr learned my addr 172.21.15.94:0/3793472409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa658bee640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 0x7fa6541a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654103990 0x7fa6541a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 -- 172.21.15.94:0/3793472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654103990 msgr2=0x7fa6541a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654103990 0x7fa6541a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 -- 172.21.15.94:0/3793472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 msgr2=0x7fa6541a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 0x7fa6541a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 -- 172.21.15.94:0/3793472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6541a3fa0 con 0x7fa654102fb0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa653fff640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e9f0 tx=0x7fa64400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa6517fa640 1 -- 172.21.15.94:0/3793472409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64400ccc0 con 0x7fa654102fb0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6541a4290 con 0x7fa654102fb0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa6517fa640 1 -- 172.21.15.94:0/3793472409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6440054d0 con 0x7fa654102fb0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa6517fa640 1 -- 172.21.15.94:0/3793472409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644010430 con 0x7fa654102fb0 2024-04-25T13:46:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.062+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6541a47e0 con 0x7fa654102fb0 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 Processor -- start 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 -- start start 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6517fa640 1 -- 172.21.15.94:0/3793472409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa6440046c0 con 0x7fa654102fb0 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc82d460 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc7e7590 con 0x55e3fc5271b0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6615e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc829420 con 0x55e3fc5064d0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa6537fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51354/0 (socket says 172.21.15.94:51354) 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa653fff640 1 -- 172.21.15.94:0/443988168 learned_addr learned my addr 172.21.15.94:0/443988168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.066+0000 7fa658bee640 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 -- 172.21.15.94:0/443988168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 msgr2=0x55e3fc8c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 -- 172.21.15.94:0/443988168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 msgr2=0x55e3fc8c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 -- 172.21.15.94:0/443988168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fc7f40d0 con 0x55e3fc5064d0 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6517fa640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 0x7fa63c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa658bee640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 0x7fa63c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6537fe640 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8cedf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa648005ab0 tx=0x7fa648009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac41bd2b6ee781b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa6517fa640 1 -- 172.21.15.94:0/3793472409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa6440deff0 con 0x7fa654102fb0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa63a7fc640 1 -- 172.21.15.94:0/443988168 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648035030 con 0x55e3fc5064d0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa63a7fc640 1 -- 172.21.15.94:0/443988168 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648040070 con 0x55e3fc5064d0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.067+0000 7fa63a7fc640 1 -- 172.21.15.94:0/443988168 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648003850 con 0x55e3fc5064d0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- 172.21.15.94:0/443988168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 msgr2=0x55e3fc8cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa658bee640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 0x7fa63c0b1ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e3fc8c8b40 tx=0x7fa64c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8cedf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa648005ab0 tx=0x7fa648009600 comp rx=0 tx=0).stop 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- 172.21.15.94:0/443988168 shutdown_connections 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8cedf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 --2- 172.21.15.94:0/443988168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- 172.21.15.94:0/443988168 >> 172.21.15.94:0/443988168 conn(0x55e3fc51b300 msgr2=0x55e3fc8c3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- 172.21.15.94:0/443988168 shutdown_connections 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- 172.21.15.94:0/443988168 wait complete. 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.068+0000 7fa6615e9740 1 -- start start 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 Processor -- start 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8d9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 0x55e3fc8e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc8cf7e0 con 0x55e3fc5271b0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc7b4b60 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e3fc8cf960 con 0x55e3fc5064d0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6615e9740 10 client.0 _ll_register_callbacks cb 0x55e3fc7d92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa658bee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 0x55e3fc8e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:54600/0 (socket says 172.21.15.94:54600) 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 -- 172.21.15.94:0/2574383234 learned_addr learned my addr 172.21.15.94:0/2574383234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa653fff640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8d9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 -- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc4fe2f0 msgr2=0x55e3fc8d9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8d9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 -- 172.21.15.94:0/2574383234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 msgr2=0x55e3fc8e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 0x55e3fc8e12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa6537fe640 1 -- 172.21.15.94:0/2574383234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fc53f210 con 0x55e3fc5064d0 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.069+0000 7fa658bee640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 0x55e3fc8e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6537fe640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa648031f00 tx=0x7fa648031f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3347be2b890df2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648035040 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa648040070 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa6480315a0 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6480317a0 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 10 client.26416 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3fc60b620 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648040070 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa648045590 con 0x55e3fc5064d0 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 10 client.26416 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 10 client.26416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 20 client.26416 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4e5d80 0x55e3fc8e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 4 client.26416 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.070+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3fc83b070 con 0x55e3fc4e5d80 2024-04-25T13:46:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.071+0000 7fa653fff640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4e5d80 0x55e3fc8e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.071+0000 7fa6337fe640 10 client.26416 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.071+0000 7fa653fff640 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4e5d80 0x55e3fc8e76f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6e9803adc46c3d server_cookie=bd515170d6312d0c in_seq=0 out_seq=0 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6337fe640 1 -- 172.21.15.94:0/2574383234 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3fc83b070 con 0x55e3fc4e5d80 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6337fe640 10 client.26416 handle_command_reply: tid=0 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 client.26416 shutdown 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 msgr2=0x55e3fc8da340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa648031f00 tx=0x7fa648031f30 comp rx=0 tx=0).stop 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 shutdown_connections 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4e5d80 0x55e3fc8e76f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc5271b0 0x55e3fc8e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8da340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 --2- 172.21.15.94:0/2574383234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8d9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6337fe640 0 client.26416 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 >> 172.21.15.94:0/2574383234 conn(0x55e3fc51b300 msgr2=0x55e3fc8c2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 shutdown_connections 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 1 -- 172.21.15.94:0/2574383234 wait complete. 2024-04-25T13:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.073+0000 7fa6615e9740 20 client.26416 trim_cache size 0 max 16384 2024-04-25T13:46:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 Processor -- start 2024-04-25T13:46:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 -- start start 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8e11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541a9300 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541aae40 con 0x55e3fc5271b0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6615e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541a99d0 con 0x55e3fc5064d0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51376/0 (socket says 172.21.15.94:51376) 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa653fff640 1 -- 172.21.15.94:0/3631017680 learned_addr learned my addr 172.21.15.94:0/3631017680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8e11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.089+0000 7fa658bee640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 -- 172.21.15.94:0/3631017680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 msgr2=0x55e3fc8c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 -- 172.21.15.94:0/3631017680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 msgr2=0x55e3fc8c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 -- 172.21.15.94:0/3631017680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fc8c8470 con 0x55e3fc5064d0 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa653fff640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8e11c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6480092a0 tx=0x7fa648031730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c643e85a8f8ad585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa639ffb640 1 -- 172.21.15.94:0/3631017680 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64805c070 con 0x55e3fc5064d0 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa639ffb640 1 -- 172.21.15.94:0/3631017680 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648040070 con 0x55e3fc5064d0 2024-04-25T13:46:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.090+0000 7fa639ffb640 1 -- 172.21.15.94:0/3631017680 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648035040 con 0x55e3fc5064d0 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- 172.21.15.94:0/3631017680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 msgr2=0x55e3fc8e11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8e11c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6480092a0 tx=0x7fa648031730 comp rx=0 tx=0).stop 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- 172.21.15.94:0/3631017680 shutdown_connections 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5271b0 0x55e3fc8c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5064d0 0x55e3fc8e11c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3631017680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- 172.21.15.94:0/3631017680 >> 172.21.15.94:0/3631017680 conn(0x55e3fc51b300 msgr2=0x55e3fc8cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- 172.21.15.94:0/3631017680 shutdown_connections 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- 172.21.15.94:0/3631017680 wait complete. 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.091+0000 7fa6615e9740 1 -- start start 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 Processor -- start 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5064d0 0x55e3fc8de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5271b0 0x55e3fc94da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541aa770 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541aa0a0 con 0x55e3fc5064d0 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541a7300 con 0x55e3fc5271b0 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6615e9740 10 client.0 _ll_register_callbacks cb 0x7fa6440e29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51382/0 (socket says 172.21.15.94:51382) 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 -- 172.21.15.94:0/3609068571 learned_addr learned my addr 172.21.15.94:0/3609068571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5064d0 0x55e3fc8de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa658bee640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5271b0 0x55e3fc94da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 -- 172.21.15.94:0/3609068571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5271b0 msgr2=0x55e3fc94da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5271b0 0x55e3fc94da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 -- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5064d0 msgr2=0x55e3fc8de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5064d0 0x55e3fc8de730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 -- 172.21.15.94:0/3609068571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fc8e1700 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa653fff640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e9c0 tx=0x7fa6440a32f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846df6ea8d26b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.092+0000 7fa6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a6ba0 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fa6440e10e0 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa648040070 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa644005680 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 10 client.17028 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3fc94f0c0 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648040070 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa6440a7690 con 0x55e3fc4fe2f0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 10 client.17028 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 20 client.17028 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4bb1c0 0x55e3fc951ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 4 client.17028 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3fc834ee0 con 0x55e3fc4bb1c0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.093+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4bb1c0 0x55e3fc951ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.094+0000 7fa6337fe640 10 client.17028 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.094+0000 7fa6537fe640 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4bb1c0 0x55e3fc951ae0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1391cf5945411219 server_cookie=c6584f28060bb581 in_seq=0 out_seq=0 2024-04-25T13:46:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.244+0000 7fa6337fe640 1 -- 172.21.15.94:0/3609068571 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3fc834ee0 con 0x55e3fc4bb1c0 2024-04-25T13:46:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.244+0000 7fa6337fe640 10 client.17028 handle_command_reply: tid=0 2024-04-25T13:46:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.244+0000 7fa6615e9740 1 client.17028 shutdown 2024-04-25T13:46:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.244+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 msgr2=0x55e3fc8de1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.244+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e9c0 tx=0x7fa6440a32f0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 shutdown_connections 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e3fc4bb1c0 0x55e3fc951ae0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3fc5271b0 0x55e3fc94da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3fc5064d0 0x55e3fc8de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 --2- 172.21.15.94:0/3609068571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3fc4fe2f0 0x55e3fc8de1f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 >> 172.21.15.94:0/3609068571 conn(0x55e3fc51b300 msgr2=0x55e3fc8c2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6337fe640 0 client.17028 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 shutdown_connections 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 1 -- 172.21.15.94:0/3609068571 wait complete. 2024-04-25T13:46:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa6615e9740 20 client.17028 trim_cache size 0 max 16384 2024-04-25T13:46:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:46:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:46:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:46:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:46:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 msgr2=0x7fa63c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 0x7fa63c0b1ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e3fc8c8b40 tx=0x7fa64c009500 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 msgr2=0x7fa654102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa64400e9f0 tx=0x7fa64400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 shutdown_connections 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa63c0afae0 0x7fa63c0b1ff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6541044b0 0x7fa6541a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654103990 0x7fa6541a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 --2- 172.21.15.94:0/3793472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa654102fb0 0x7fa654102250 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.245+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 >> 172.21.15.94:0/3793472409 conn(0x7fa6540fd390 msgr2=0x7fa654104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.246+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 shutdown_connections 2024-04-25T13:46:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:00.246+0000 7fa65a651640 1 -- 172.21.15.94:0/3793472409 wait complete. 2024-04-25T13:46:00.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 412435 12.92 MB/sec execute 43 sec latency 77.918 ms 2024-04-25T13:46:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:01 smithi094 ceph-mon[29195]: pgmap v473: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-04-25T13:46:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:01 smithi138 ceph-mon[32977]: pgmap v473: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-04-25T13:46:01.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 415249 13.04 MB/sec execute 44 sec latency 115.588 ms 2024-04-25T13:46:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:01 smithi137 ceph-mon[32568]: pgmap v473: 97 pgs: 97 active+clean; 46 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-04-25T13:46:02.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 417519 12.93 MB/sec execute 45 sec latency 23.160 ms 2024-04-25T13:46:03.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:03 smithi094 ceph-mon[29195]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2024-04-25T13:46:03.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:03 smithi138 ceph-mon[32977]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2024-04-25T13:46:03.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 419507 12.91 MB/sec execute 46 sec latency 26.269 ms 2024-04-25T13:46:03.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:03 smithi137 ceph-mon[32568]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2024-04-25T13:46:04.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 422455 13.02 MB/sec execute 47 sec latency 12.474 ms 2024-04-25T13:46:05.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:05 smithi094 ceph-mon[29195]: pgmap v475: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:46:05.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:05 smithi138 ceph-mon[32977]: pgmap v475: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:46:05.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 425305 13.03 MB/sec execute 48 sec latency 13.115 ms 2024-04-25T13:46:05.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:05 smithi137 ceph-mon[32568]: pgmap v475: 97 pgs: 97 active+clean; 38 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:46:06.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 427166 12.99 MB/sec execute 49 sec latency 37.305 ms 2024-04-25T13:46:07.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:07 smithi094 ceph-mon[29195]: pgmap v476: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-04-25T13:46:07.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:07 smithi138 ceph-mon[32977]: pgmap v476: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-04-25T13:46:07.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 429615 13.01 MB/sec execute 50 sec latency 77.509 ms 2024-04-25T13:46:07.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:07 smithi137 ceph-mon[32568]: pgmap v476: 97 pgs: 97 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 374 op/s 2024-04-25T13:46:08.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 432611 13.04 MB/sec execute 51 sec latency 11.758 ms 2024-04-25T13:46:09.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:09 smithi094 ceph-mon[29195]: pgmap v477: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 406 op/s 2024-04-25T13:46:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:46:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:46:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:46:09.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:46:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:09 smithi138 ceph-mon[32977]: pgmap v477: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 406 op/s 2024-04-25T13:46:09.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:46:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:46:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:46:09.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:46:09.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 434461 12.98 MB/sec execute 52 sec latency 23.734 ms 2024-04-25T13:46:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:09 smithi137 ceph-mon[32568]: pgmap v477: 97 pgs: 97 active+clean; 41 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 406 op/s 2024-04-25T13:46:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:46:09.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:46:09.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:46:09.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:46:10.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 437029 13.00 MB/sec execute 53 sec latency 19.409 ms 2024-04-25T13:46:10.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:46:10.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:46:11.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 -- 172.21.15.94:0/3289943568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a20102170 msgr2=0x7f5a20102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 --2- 172.21.15.94:0/3289943568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a20102170 0x7f5a20102e70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009ca0 tx=0x7f5a1802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 -- 172.21.15.94:0/3289943568 shutdown_connections 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 --2- 172.21.15.94:0/3289943568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 0x7f5a2010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 --2- 172.21.15.94:0/3289943568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a201033b0 0x7f5a201037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 --2- 172.21.15.94:0/3289943568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a20102170 0x7f5a20102e70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 -- 172.21.15.94:0/3289943568 >> 172.21.15.94:0/3289943568 conn(0x7f5a200fd340 msgr2=0x7f5a200ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 -- 172.21.15.94:0/3289943568 shutdown_connections 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.067+0000 7f5a27972640 1 -- 172.21.15.94:0/3289943568 wait complete. 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 Processor -- start 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 -- start start 2024-04-25T13:46:11.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a20102170 0x7f5a2010b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 0x7f5a201a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2010e500 con 0x7f5a201033b0 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2010e380 con 0x7f5a20102170 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a27972640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2010e680 con 0x7f5a201080b0 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a24f0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a24f0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56322/0 (socket says 172.21.15.94:56322) 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a24f0d640 1 -- 172.21.15.94:0/790354257 learned_addr learned my addr 172.21.15.94:0/790354257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.068+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 0x7f5a201a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 -- 172.21.15.94:0/790354257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 msgr2=0x7f5a201a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 0x7f5a201a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 -- 172.21.15.94:0/790354257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a20102170 msgr2=0x7f5a2010b630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:46:11.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a20102170 0x7f5a2010b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 -- 172.21.15.94:0/790354257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201a1d90 con 0x7f5a201033b0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a24f0d640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9f0 tx=0x7f5a1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a167fc640 1 -- 172.21.15.94:0/790354257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1000cd80 con 0x7f5a201033b0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a167fc640 1 -- 172.21.15.94:0/790354257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a100108e0 con 0x7f5a201033b0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201a2080 con 0x7f5a201033b0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a167fc640 1 -- 172.21.15.94:0/790354257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10018980 con 0x7f5a201033b0 2024-04-25T13:46:11.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.069+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201a25d0 con 0x7f5a201033b0 2024-04-25T13:46:11.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a167fc640 1 -- 172.21.15.94:0/790354257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5a10010430 con 0x7f5a201033b0 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 Processor -- start 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 -- start start 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a167fc640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 0x7f5a0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2046b0 0x557daa5c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557daa52bd80 con 0x557daa225390 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557daa4e58b0 con 0x557daa2046b0 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557daa527d40 con 0x557daa1fc4d0 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a2570e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.072+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a167fc640 1 -- 172.21.15.94:0/790354257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5a100df040 con 0x7f5a201033b0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41370/0 (socket says 172.21.15.94:41370) 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56336/0 (socket says 172.21.15.94:56336) 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 -- 172.21.15.94:0/2555956080 learned_addr learned my addr 172.21.15.94:0/2555956080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2046b0 0x557daa5c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a24f0d640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 0x7f5a0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 -- 172.21.15.94:0/2555956080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2046b0 msgr2=0x557daa5c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2046b0 0x557daa5c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 -- 172.21.15.94:0/2555956080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 msgr2=0x557daa5c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 -- 172.21.15.94:0/2555956080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557daa4b6a40 con 0x557daa1fc4d0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a24f0d640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 0x7f5a0c0b2000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9c0 tx=0x7f5a10013d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004ab0 tx=0x7f5a1802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeccac30434a1cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a0b7fe640 1 -- 172.21.15.94:0/2555956080 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18031650 con 0x557daa1fc4d0 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a0b7fe640 1 -- 172.21.15.94:0/2555956080 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1803f070 con 0x557daa1fc4d0 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.073+0000 7f5a0b7fe640 1 -- 172.21.15.94:0/2555956080 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004540 con 0x557daa1fc4d0 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2555956080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 msgr2=0x557daa5c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004ab0 tx=0x7f5a1802f310 comp rx=0 tx=0).stop 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2555956080 shutdown_connections 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2046b0 0x557daa5c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5c6fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2555956080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2555956080 >> 172.21.15.94:0/2555956080 conn(0x557daa2194e0 msgr2=0x557daa5c2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2555956080 shutdown_connections 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.074+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2555956080 wait complete. 2024-04-25T13:46:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 -- start start 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 Processor -- start 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 0x557daa5d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557daa5c8720 con 0x557daa225390 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557daa4b2d10 con 0x557daa1fc4d0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557daa5c88a0 con 0x557daa2046b0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2e81e740 10 client.0 _ll_register_callbacks cb 0x557daa4df210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2570e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 0x557daa5d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2570e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41380/0 (socket says 172.21.15.94:41380) 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.075+0000 7f5a2570e640 1 -- 172.21.15.94:0/2314854565 learned_addr learned my addr 172.21.15.94:0/2314854565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 -- 172.21.15.94:0/2314854565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 msgr2=0x557daa5d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 -- 172.21.15.94:0/2314854565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 msgr2=0x557daa5d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 -- 172.21.15.94:0/2314854565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557daa3077d0 con 0x557daa1fc4d0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 0x557daa5d0890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x557daa5c7ab0 tx=0x7f5a1c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abff2825c94f88f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00fce0 con 0x557daa1fc4d0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1803f070 con 0x557daa1fc4d0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5a1c0130d0 con 0x557daa1fc4d0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c00fe80 con 0x557daa1fc4d0 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a2e81e740 10 client.26519 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557daa23d3f0 con 0x557daa1fc4d0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.076+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1803f070 con 0x557daa1fc4d0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5a1c011900 con 0x557daa1fc4d0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 10 client.26519 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 10 client.26519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 20 client.26519 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa1e3f60 0x557daa5de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 4 client.26519 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557daa539990 con 0x557daa1e3f60 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa1e3f60 0x557daa5de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a08ff9640 10 client.26519 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.077+0000 7f5a2570e640 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa1e3f60 0x557daa5de1d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8190f1429f8f585 server_cookie=4517329dd79c877e in_seq=0 out_seq=0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a08ff9640 1 -- 172.21.15.94:0/2314854565 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557daa539990 con 0x557daa1e3f60 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a08ff9640 10 client.26519 handle_command_reply: tid=0 2024-04-25T13:46:11.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 client.26519 shutdown 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 msgr2=0x557daa5d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 0x557daa5d0890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x557daa5c7ab0 tx=0x7f5a1c005850 comp rx=0 tx=0).stop 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 shutdown_connections 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa1e3f60 0x557daa5de1d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa225390 0x557daa5d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/2314854565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa1fc4d0 0x557daa5d0890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a08ff9640 0 client.26519 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 >> 172.21.15.94:0/2314854565 conn(0x557daa2194e0 msgr2=0x557daa5c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 shutdown_connections 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 1 -- 172.21.15.94:0/2314854565 wait complete. 2024-04-25T13:46:11.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.078+0000 7f5a2e81e740 20 client.26519 trim_cache size 0 max 16384 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 Processor -- start 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 -- start start 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.097+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201ab5d0 con 0x557daa2046b0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201ad110 con 0x557daa2194e0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201abca0 con 0x557daa1fc4d0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56342/0 (socket says 172.21.15.94:56342) 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 -- 172.21.15.94:0/4102247485 learned_addr learned my addr 172.21.15.94:0/4102247485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a2570e640 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 -- 172.21.15.94:0/4102247485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 msgr2=0x557daa5d7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5d7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 -- 172.21.15.94:0/4102247485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 msgr2=0x557daa5d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 -- 172.21.15.94:0/4102247485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557daa1fd0f0 con 0x557daa2046b0 2024-04-25T13:46:11.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c013b20 tx=0x7f5a1c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865f86e639a705dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a0affd640 1 -- 172.21.15.94:0/4102247485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0130d0 con 0x557daa2046b0 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a0affd640 1 -- 172.21.15.94:0/4102247485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c013270 con 0x557daa2046b0 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.098+0000 7f5a0affd640 1 -- 172.21.15.94:0/4102247485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c022430 con 0x557daa2046b0 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 -- 172.21.15.94:0/4102247485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 msgr2=0x557daa5d8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c013b20 tx=0x7f5a1c0098a0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 -- 172.21.15.94:0/4102247485 shutdown_connections 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa2046b0 0x557daa5d8200 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa1fc4d0 0x557daa5d7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/4102247485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 -- 172.21.15.94:0/4102247485 >> 172.21.15.94:0/4102247485 conn(0x557daa5cf6d0 msgr2=0x557daa5d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 -- 172.21.15.94:0/4102247485 shutdown_connections 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.099+0000 7f5a2e81e740 1 -- 172.21.15.94:0/4102247485 wait complete. 2024-04-25T13:46:11.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 -- start start 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 Processor -- start 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201aca40 con 0x557daa1fc4d0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201ac370 con 0x557daa2194e0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201a95b0 con 0x557daa2046b0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2e81e740 10 client.0 _ll_register_callbacks cb 0x7f5a1c013550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56358/0 (socket says 172.21.15.94:56358) 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 -- 172.21.15.94:0/3398064660 learned_addr learned my addr 172.21.15.94:0/3398064660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a2570e640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 -- 172.21.15.94:0/3398064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 msgr2=0x557daa5d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 -- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 msgr2=0x557daa5d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.100+0000 7f5a25f0f640 1 -- 172.21.15.94:0/3398064660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557daa4ec4b0 con 0x557daa1fc4d0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a25f0f640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000c00 tx=0x7f5a1c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46955116cfcffbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c02b650 con 0x557daa1fc4d0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f5a1c02b7f0 con 0x557daa1fc4d0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c013270 con 0x557daa1fc4d0 2024-04-25T13:46:11.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c03a450 con 0x557daa1fc4d0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 10 client.17046 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557daa5d7dc0 con 0x557daa1fc4d0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c013270 con 0x557daa1fc4d0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5a1c044070 con 0x557daa1fc4d0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 10 client.17046 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 20 client.17046 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa225390 0x557daa648550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 4 client.17046 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.101+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557daa533800 con 0x557daa225390 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.102+0000 7f5a2570e640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa225390 0x557daa648550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.102+0000 7f5a08ff9640 10 client.17046 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.102+0000 7f5a2570e640 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa225390 0x557daa648550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bab975ace31d7d server_cookie=9ae61d9d87ae9275 in_seq=0 out_seq=0 2024-04-25T13:46:11.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a08ff9640 1 -- 172.21.15.94:0/3398064660 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557daa533800 con 0x557daa225390 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a08ff9640 10 client.17046 handle_command_reply: tid=0 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 client.17046 shutdown 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 msgr2=0x557daa5d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000c00 tx=0x7f5a1c009ef0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 shutdown_connections 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557daa225390 0x557daa648550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557daa2194e0 0x557daa5d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557daa2046b0 0x557daa5d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a08ff9640 0 client.17046 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 --2- 172.21.15.94:0/3398064660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557daa1fc4d0 0x557daa5d5320 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 >> 172.21.15.94:0/3398064660 conn(0x557daa5cf6d0 msgr2=0x557daa5c09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 shutdown_connections 2024-04-25T13:46:11.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.105+0000 7f5a2e81e740 1 -- 172.21.15.94:0/3398064660 wait complete. 2024-04-25T13:46:11.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a2e81e740 20 client.17046 trim_cache size 0 max 16384 2024-04-25T13:46:11.113 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:46:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:46:11.113 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 msgr2=0x7f5a0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 0x7f5a0c0b2000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9c0 tx=0x7f5a10013d50 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 msgr2=0x7f5a2019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9f0 tx=0x7f5a1000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 shutdown_connections 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5a0c0afaf0 0x7f5a0c0b2000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a201080b0 0x7f5a201a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a201033b0 0x7f5a2019f020 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 --2- 172.21.15.94:0/790354257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a20102170 0x7f5a2010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 >> 172.21.15.94:0/790354257 conn(0x7f5a200fd340 msgr2=0x7f5a20104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 shutdown_connections 2024-04-25T13:46:11.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.106+0000 7f5a27972640 1 -- 172.21.15.94:0/790354257 wait complete. 2024-04-25T13:46:11.117 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:46:11.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 -- 172.21.15.94:0/2093364887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 msgr2=0x7fb5a81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 --2- 172.21.15.94:0/2093364887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a81036e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009900 tx=0x7fb59c02a410 comp rx=0 tx=0).stop 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 -- 172.21.15.94:0/2093364887 shutdown_connections 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 --2- 172.21.15.94:0/2093364887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8108920 0x7fb5a810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 --2- 172.21.15.94:0/2093364887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8103c20 0x7fb5a8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 --2- 172.21.15.94:0/2093364887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a81036e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 -- 172.21.15.94:0/2093364887 >> 172.21.15.94:0/2093364887 conn(0x7fb5a80fdbf0 msgr2=0x7fb5a8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 -- 172.21.15.94:0/2093364887 shutdown_connections 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.231+0000 7fb5af127640 1 -- 172.21.15.94:0/2093364887 wait complete. 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 Processor -- start 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 -- start start 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8103c20 0x7fb5a819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8108920 0x7fb5a81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a810e540 con 0x7fb5a81029e0 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a810e3c0 con 0x7fb5a8108920 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5af127640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a810e6c0 con 0x7fb5a8103c20 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56368/0 (socket says 172.21.15.94:56368) 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 -- 172.21.15.94:0/3038593630 learned_addr learned my addr 172.21.15.94:0/3038593630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5a7fff640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8103c20 0x7fb5a819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5ad6c4640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8108920 0x7fb5a81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 -- 172.21.15.94:0/3038593630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8103c20 msgr2=0x7fb5a819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8103c20 0x7fb5a819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 -- 172.21.15.94:0/3038593630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8108920 msgr2=0x7fb5a81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8108920 0x7fb5a81a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.232+0000 7fb5acec3640 1 -- 172.21.15.94:0/3038593630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a81a1e20 con 0x7fb5a81029e0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5acec3640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb59c006010 tx=0x7fb59c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c03f030 con 0x7fb5a81029e0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a81a20b0 con 0x7fb5a81029e0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb59c003aa0 con 0x7fb5a81029e0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c035070 con 0x7fb5a81029e0 2024-04-25T13:46:11.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.233+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a81a2620 con 0x7fb5a81029e0 2024-04-25T13:46:11.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.234+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a8107730 con 0x7fb5a81029e0 2024-04-25T13:46:11.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.236+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb59c042030 con 0x7fb5a81029e0 2024-04-25T13:46:11.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.236+0000 7fb5a5ffb640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 0x7fb5980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.236+0000 7fb5a7fff640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 0x7fb5980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.236+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fb59c0ff660 con 0x7fb5a81029e0 2024-04-25T13:46:11.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.237+0000 7fb5a7fff640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 0x7fb5980b2000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80751e0 tx=0x7fb594005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.241+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb59c0c7b50 con 0x7fb5a81029e0 2024-04-25T13:46:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.472+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5a802db50 con 0x7fb5a81029e0 2024-04-25T13:46:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.472+0000 7fb5a5ffb640 1 -- 172.21.15.94:0/3038593630 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11941 (secure 0 0 0) 0x7fb59c0d0220 con 0x7fb5a81029e0 2024-04-25T13:46:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:46:11.473 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:41:41.618990+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1621301649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1621301649},{"type":"v1","addr":"172.21.15.138:6835","nonce":1621301649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:46:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 msgr2=0x7fb5980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 0x7fb5980b2000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80751e0 tx=0x7fb594005f90 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 msgr2=0x7fb5a8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb59c006010 tx=0x7fb59c004270 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 shutdown_connections 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5980afaf0 0x7fb5980b2000 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8108920 0x7fb5a81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8103c20 0x7fb5a819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 --2- 172.21.15.94:0/3038593630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5a81029e0 0x7fb5a8075840 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 >> 172.21.15.94:0/3038593630 conn(0x7fb5a80fdbf0 msgr2=0x7fb5a8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 shutdown_connections 2024-04-25T13:46:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.475+0000 7fb5af127640 1 -- 172.21.15.94:0/3038593630 wait complete. 2024-04-25T13:46:11.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:11 smithi094 ceph-mon[29195]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-04-25T13:46:11.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- 172.21.15.94:0/3761092516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61441045d0 msgr2=0x7f61441069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 --2- 172.21.15.94:0/3761092516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61441045d0 0x7f61441069f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6138009860 tx=0x7f613802a340 comp rx=0 tx=0).stop 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- 172.21.15.94:0/3761092516 shutdown_connections 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 --2- 172.21.15.94:0/3761092516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144109890 0x7f614410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 --2- 172.21.15.94:0/3761092516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6144106f30 0x7f6144109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 --2- 172.21.15.94:0/3761092516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61441045d0 0x7f61441069f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- 172.21.15.94:0/3761092516 >> 172.21.15.94:0/3761092516 conn(0x7f61440fdbd0 msgr2=0x7f6144100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- 172.21.15.94:0/3761092516 shutdown_connections 2024-04-25T13:46:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- 172.21.15.94:0/3761092516 wait complete. 2024-04-25T13:46:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 Processor -- start 2024-04-25T13:46:11.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.603+0000 7f614a484640 1 -- start start 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 0x7f614419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6144109890 0x7f61441a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f614410f5c0 con 0x7f6144106f30 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f614410f440 con 0x7f6144109890 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f614a484640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f614410f740 con 0x7f61441045d0 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56390/0 (socket says 172.21.15.94:56390) 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 -- 172.21.15.94:0/16531360 learned_addr learned my addr 172.21.15.94:0/16531360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f6143fff640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 0x7f614419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f6148a21640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6144109890 0x7f61441a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 -- 172.21.15.94:0/16531360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 msgr2=0x7f614419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 0x7f614419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 -- 172.21.15.94:0/16531360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6144109890 msgr2=0x7f61441a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6144109890 0x7f61441a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 -- 172.21.15.94:0/16531360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441a20e0 con 0x7f6144106f30 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f6143fff640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 0x7f614419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.604+0000 7f61437fe640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f613400e9f0 tx=0x7f613400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.605+0000 7f61417fa640 1 -- 172.21.15.94:0/16531360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613400cd80 con 0x7f6144106f30 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.605+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441a23d0 con 0x7f6144106f30 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.605+0000 7f61417fa640 1 -- 172.21.15.94:0/16531360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61340108e0 con 0x7f6144106f30 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.605+0000 7f61417fa640 1 -- 172.21.15.94:0/16531360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134018980 con 0x7f6144106f30 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.605+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441a2970 con 0x7f6144106f30 2024-04-25T13:46:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f61417fa640 1 -- 172.21.15.94:0/16531360 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6134018b20 con 0x7f6144106f30 2024-04-25T13:46:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 Processor -- start 2024-04-25T13:46:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 -- start start 2024-04-25T13:46:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f157d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f157dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f157ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6f14e1d80 con 0x55e6f11b24d0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6f149ba20 con 0x55e6f11ba6b0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6151409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6f14ddd40 con 0x55e6f11db390 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6148a21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f157d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f157dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6148a21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f157d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41446/0 (socket says 172.21.15.94:41446) 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6148a21640 1 -- 172.21.15.94:0/2037876612 learned_addr learned my addr 172.21.15.94:0/2037876612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f61437fe640 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f157ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f61417fa640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 0x7f612c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6148a21640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 0x7f612c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 -- 172.21.15.94:0/2037876612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 msgr2=0x55e6f157d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f157d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 -- 172.21.15.94:0/2037876612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 msgr2=0x55e6f157ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f157ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.608+0000 7f6143fff640 1 -- 172.21.15.94:0/2037876612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f14a6c30 con 0x55e6f11b24d0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f61417fa640 1 -- 172.21.15.94:0/16531360 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f61340df020 con 0x7f6144106f30 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f6143fff640 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f157dd90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6138035040 tx=0x7f6138005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a9d22ab1a5df21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f6148a21640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 0x7f612c0b2000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e6f157e860 tx=0x7f613c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f612a7fc640 1 -- 172.21.15.94:0/2037876612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138041070 con 0x55e6f11b24d0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f612a7fc640 1 -- 172.21.15.94:0/2037876612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6138030e10 con 0x55e6f11b24d0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.609+0000 7f612a7fc640 1 -- 172.21.15.94:0/2037876612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138037040 con 0x55e6f11b24d0 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- 172.21.15.94:0/2037876612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 msgr2=0x55e6f157dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f157dd90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6138035040 tx=0x7f6138005cd0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- 172.21.15.94:0/2037876612 shutdown_connections 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f157ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f157dd90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 --2- 172.21.15.94:0/2037876612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f157d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- 172.21.15.94:0/2037876612 >> 172.21.15.94:0/2037876612 conn(0x55e6f11cf4e0 msgr2=0x55e6f1579020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- 172.21.15.94:0/2037876612 shutdown_connections 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- 172.21.15.94:0/2037876612 wait complete. 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 -- start start 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.610+0000 7f6151409740 1 Processor -- start 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11b24d0 0x55e6f1587590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f1587ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6f157f4d0 con 0x55e6f11db390 2024-04-25T13:46:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6f1468dc0 con 0x55e6f11ba6b0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6f157f650 con 0x55e6f11b24d0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6151409740 10 client.0 _ll_register_callbacks cb 0x55e6f1484600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56406/0 (socket says 172.21.15.94:56406) 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 -- 172.21.15.94:0/3730537201 learned_addr learned my addr 172.21.15.94:0/3730537201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f61437fe640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f1587ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6143fff640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11b24d0 0x55e6f1587590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 -- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11b24d0 msgr2=0x55e6f1587590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11b24d0 0x55e6f1587590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 -- 172.21.15.94:0/3730537201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 msgr2=0x55e6f1587ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f1587ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 -- 172.21.15.94:0/3730537201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f11f33f0 con 0x55e6f11db390 2024-04-25T13:46:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f61437fe640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f1587ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f6148a21640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e450 tx=0x7f613c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f266f85ac66596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f61237fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c01e9f0 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.611+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f613c01eb90 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6138030e10 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f613c01ed90 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 10 client.17064 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e6f12bd7d0 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6138030e10 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f613c00b040 con 0x55e6f11db390 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 10 client.17064 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 10 client.17064 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 20 client.17064 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f1199f60 0x55e6f15953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 4 client.17064 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f14ef990 con 0x55e6f1199f60 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.612+0000 7f6143fff640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f1199f60 0x55e6f15953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.613+0000 7f61237fe640 10 client.17064 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.613+0000 7f6143fff640 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f1199f60 0x55e6f15953f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74ed9ab962dd664e server_cookie=2ec0e68739ba6b47 in_seq=0 out_seq=0 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.613+0000 7f61237fe640 1 -- 172.21.15.94:0/3730537201 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6f14ef990 con 0x55e6f1199f60 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.613+0000 7f61237fe640 10 client.17064 handle_command_reply: tid=0 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.613+0000 7f6151409740 1 client.17064 shutdown 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 msgr2=0x55e6f158ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e450 tx=0x7f613c004a40 comp rx=0 tx=0).stop 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 shutdown_connections 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f1199f60 0x55e6f15953f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11db390 0x55e6f158ea30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f1587ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 --2- 172.21.15.94:0/3730537201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11b24d0 0x55e6f1587590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f61237fe640 0 client.17064 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 >> 172.21.15.94:0/3730537201 conn(0x55e6f11cf4e0 msgr2=0x55e6f157bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 shutdown_connections 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 1 -- 172.21.15.94:0/3730537201 wait complete. 2024-04-25T13:46:11.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.614+0000 7f6151409740 20 client.17064 trim_cache size 0 max 16384 2024-04-25T13:46:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 Processor -- start 2024-04-25T13:46:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 -- start start 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11ba6b0 0x55e6f1582850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11db390 0x55e6f158e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61441ab5f0 con 0x55e6f11b24d0 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61441ad130 con 0x55e6f11db390 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6151409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61441abcc0 con 0x55e6f11ba6b0 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56414/0 (socket says 172.21.15.94:56414) 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 -- 172.21.15.94:0/2456051117 learned_addr learned my addr 172.21.15.94:0/2456051117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f61437fe640 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11ba6b0 0x55e6f1582850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6148a21640 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11db390 0x55e6f158e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 -- 172.21.15.94:0/2456051117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11ba6b0 msgr2=0x55e6f1582850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11ba6b0 0x55e6f1582850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 -- 172.21.15.94:0/2456051117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11db390 msgr2=0x55e6f158e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11db390 0x55e6f158e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 -- 172.21.15.94:0/2456051117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f158ee90 con 0x55e6f11b24d0 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6143fff640 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f613802a860 tx=0x7f6138034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396737b669593c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.628+0000 7f6129ffb640 1 -- 172.21.15.94:0/2456051117 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138052070 con 0x55e6f11b24d0 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6129ffb640 1 -- 172.21.15.94:0/2456051117 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6138030d30 con 0x55e6f11b24d0 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6129ffb640 1 -- 172.21.15.94:0/2456051117 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138037030 con 0x55e6f11b24d0 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 -- 172.21.15.94:0/2456051117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 msgr2=0x55e6f1582310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f613802a860 tx=0x7f6138034040 comp rx=0 tx=0).stop 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 -- 172.21.15.94:0/2456051117 shutdown_connections 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11db390 0x55e6f158e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11ba6b0 0x55e6f1582850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 --2- 172.21.15.94:0/2456051117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f1582310 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 -- 172.21.15.94:0/2456051117 >> 172.21.15.94:0/2456051117 conn(0x55e6f11cf4e0 msgr2=0x55e6f157d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 -- 172.21.15.94:0/2456051117 shutdown_connections 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.629+0000 7f6151409740 1 -- 172.21.15.94:0/2456051117 wait complete. 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 -- start start 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 Processor -- start 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f158d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f158de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61441aca60 con 0x55e6f11b24d0 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61441ac390 con 0x55e6f11ba6b0 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61441a95d0 con 0x55e6f11db390 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6151409740 10 client.0 _ll_register_callbacks cb 0x7f613c00fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6143fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6143fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56422/0 (socket says 172.21.15.94:56422) 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6143fff640 1 -- 172.21.15.94:0/2216639129 learned_addr learned my addr 172.21.15.94:0/2216639129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f6148a21640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f158de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.630+0000 7f61437fe640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f158d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 -- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 msgr2=0x55e6f158de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f158de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 -- 172.21.15.94:0/2216639129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 msgr2=0x55e6f158d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f158d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 -- 172.21.15.94:0/2216639129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f15fb900 con 0x55e6f11b24d0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6143fff640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f61380306d0 tx=0x7f6138030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787c4c21eee52ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138052070 con 0x55e6f11b24d0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f6138004240 con 0x55e6f11b24d0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6138030d30 con 0x55e6f11b24d0 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6138037040 con 0x55e6f11b24d0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6151409740 10 client.17070 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e6f15fcb80 con 0x55e6f11b24d0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6138030d30 con 0x55e6f11b24d0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.631+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f613804dce0 con 0x55e6f11b24d0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 10 client.17070 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 10 client.17070 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 20 client.17070 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f116f3a0 0x55e6f15ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 4 client.17070 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6f14e9800 con 0x55e6f116f3a0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f61437fe640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f116f3a0 0x55e6f15ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f61237fe640 10 client.17070 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.632+0000 7f61437fe640 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f116f3a0 0x55e6f15ff0f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=989bcd8957392e54 server_cookie=bbb0ec624a1d89b0 in_seq=0 out_seq=0 2024-04-25T13:46:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f61237fe640 1 -- 172.21.15.94:0/2216639129 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6f14e9800 con 0x55e6f116f3a0 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f61237fe640 10 client.17070 handle_command_reply: tid=0 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 client.17070 shutdown 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 msgr2=0x55e6f158d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f61380306d0 tx=0x7f6138030700 comp rx=0 tx=0).stop 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 shutdown_connections 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e6f116f3a0 0x55e6f15ff0f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6f11db390 0x55e6f158de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6f11ba6b0 0x55e6f158d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f61237fe640 0 client.17070 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 --2- 172.21.15.94:0/2216639129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6f11b24d0 0x55e6f158d3c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 >> 172.21.15.94:0/2216639129 conn(0x55e6f11cf4e0 msgr2=0x55e6f157d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 shutdown_connections 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 1 -- 172.21.15.94:0/2216639129 wait complete. 2024-04-25T13:46:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f6151409740 20 client.17070 trim_cache size 0 max 16384 2024-04-25T13:46:11.640 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 msgr2=0x7f612c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 0x7f612c0b2000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e6f157e860 tx=0x7f613c004ab0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 msgr2=0x7f614419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.633+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f613400e9f0 tx=0x7f613400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 shutdown_connections 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f612c0afaf0 0x7f612c0b2000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6144109890 0x7f61441a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6144106f30 0x7f614419aae0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 --2- 172.21.15.94:0/16531360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61441045d0 0x7f614419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 >> 172.21.15.94:0/16531360 conn(0x7f61440fdbd0 msgr2=0x7f6144104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 shutdown_connections 2024-04-25T13:46:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.634+0000 7f614a484640 1 -- 172.21.15.94:0/16531360 wait complete. 2024-04-25T13:46:11.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:46:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:11 smithi138 ceph-mon[32977]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- 172.21.15.94:0/3464181726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 msgr2=0x7fe570104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 --2- 172.21.15.94:0/3464181726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe570104040 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe564009900 tx=0x7fe56402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- 172.21.15.94:0/3464181726 shutdown_connections 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 --2- 172.21.15.94:0/3464181726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108930 0x7fe57010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 --2- 172.21.15.94:0/3464181726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe570104040 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 --2- 172.21.15.94:0/3464181726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5701029f0 0x7fe5701036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- 172.21.15.94:0/3464181726 >> 172.21.15.94:0/3464181726 conn(0x7fe5700fdbc0 msgr2=0x7fe570100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- 172.21.15.94:0/3464181726 shutdown_connections 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- 172.21.15.94:0/3464181726 wait complete. 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 Processor -- start 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.764+0000 7fe576fd6640 1 -- start start 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5701029f0 0x7fe5700ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570108930 0x7fe5700fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e4f0 con 0x7fe570103c30 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e370 con 0x7fe5701029f0 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e670 con 0x7fe570108930 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56442/0 (socket says 172.21.15.94:56442) 2024-04-25T13:46:11.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 -- 172.21.15.94:0/2467700204 learned_addr learned my addr 172.21.15.94:0/2467700204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe574d72640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5701029f0 0x7fe5700ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe575573640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570108930 0x7fe5700fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 -- 172.21.15.94:0/2467700204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570108930 msgr2=0x7fe5700fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570108930 0x7fe5700fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 -- 172.21.15.94:0/2467700204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5701029f0 msgr2=0x7fe5700ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5701029f0 0x7fe5700ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 -- 172.21.15.94:0/2467700204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5700fed60 con 0x7fe570103c30 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe564006010 tx=0x7fe564031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56dffb640 1 -- 172.21.15.94:0/2467700204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56403f030 con 0x7fe570103c30 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56dffb640 1 -- 172.21.15.94:0/2467700204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe564003aa0 con 0x7fe570103c30 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5701a4360 con 0x7fe570103c30 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.765+0000 7fe56dffb640 1 -- 172.21.15.94:0/2467700204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564035070 con 0x7fe570103c30 2024-04-25T13:46:11.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.766+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5701a4760 con 0x7fe570103c30 2024-04-25T13:46:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe56dffb640 1 -- 172.21.15.94:0/2467700204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe564042030 con 0x7fe570103c30 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 Processor -- start 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 -- start start 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5187810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 0x5645e5187d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5645e50ebd80 con 0x5645e4dc46b0 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5645e50a5a20 con 0x5645e4dbc4d0 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.768+0000 7fe57de85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645e50e7d40 con 0x5645e4de5390 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56450/0 (socket says 172.21.15.94:56450) 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe575573640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5187810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56ffff640 1 -- 172.21.15.94:0/1105025330 learned_addr learned my addr 172.21.15.94:0/1105025330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 0x5645e5187d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56dffb640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 0x7fe5580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe575573640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 0x7fe5580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 -- 172.21.15.94:0/1105025330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 msgr2=0x5645e5187810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5187810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 -- 172.21.15.94:0/1105025330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 msgr2=0x5645e5188a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56dffb640 1 -- 172.21.15.94:0/2467700204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe5640ff300 con 0x7fe570103c30 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 -- 172.21.15.94:0/1105025330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e50b0c30 con 0x5645e4dbc4d0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe56ffff640 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe574d72640 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 0x5645e5187d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5701022c0 tx=0x7fe568003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31df76f34477b358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.769+0000 7fe575573640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 0x7fe5580b2000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5645e5188860 tx=0x7fe560004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe556ffd640 1 -- 172.21.15.94:0/1105025330 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568004b20 con 0x5645e4dbc4d0 2024-04-25T13:46:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe556ffd640 1 -- 172.21.15.94:0/1105025330 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe568004cc0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe556ffd640 1 -- 172.21.15.94:0/1105025330 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680126b0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 -- 172.21.15.94:0/1105025330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 msgr2=0x5645e5187d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 0x5645e5187d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5701022c0 tx=0x7fe568003e00 comp rx=0 tx=0).stop 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 -- 172.21.15.94:0/1105025330 shutdown_connections 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5188a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dbc4d0 0x5645e5187d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 --2- 172.21.15.94:0/1105025330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5187810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.770+0000 7fe57de85740 1 -- 172.21.15.94:0/1105025330 >> 172.21.15.94:0/1105025330 conn(0x5645e4dd94e0 msgr2=0x5645e5183020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- 172.21.15.94:0/1105025330 shutdown_connections 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- 172.21.15.94:0/1105025330 wait complete. 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- start start 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 Processor -- start 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 0x5645e5191aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5198a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5645e5189530 con 0x5645e4dbc4d0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5645e5072dc0 con 0x5645e4dc46b0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645e51896b0 con 0x5645e4de5390 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe57de85740 10 client.0 _ll_register_callbacks cb 0x5645e508e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe574d72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe574d72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56456/0 (socket says 172.21.15.94:56456) 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.771+0000 7fe574d72640 1 -- 172.21.15.94:0/4192467254 learned_addr learned my addr 172.21.15.94:0/4192467254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe575573640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe56ffff640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 0x5645e5191aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 -- 172.21.15.94:0/4192467254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 msgr2=0x5645e5198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 -- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 msgr2=0x5645e5191aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 0x5645e5191aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 -- 172.21.15.94:0/4192467254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e4dfd3f0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe575573640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe56ffff640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 0x5645e5191aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe574d72640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe568004010 tx=0x7fe568009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9487b1bcdf2f7b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe54ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568005280 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe568005420 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe568004cc0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe568005620 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe57de85740 10 client.17082 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.772+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5645e4ec77d0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe568004cc0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe56800f040 con 0x5645e4dbc4d0 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 10 client.17082 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 10 client.17082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 20 client.17082 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4da3f60 0x5645e519f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 4 client.17082 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645e50f9990 con 0x5645e4da3f60 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe56ffff640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4da3f60 0x5645e519f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe54ffff640 10 client.17082 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.773+0000 7fe56ffff640 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4da3f60 0x5645e519f060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef55b3470986ef28 server_cookie=2ffa2f66e4941d49 in_seq=0 out_seq=0 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe54ffff640 1 -- 172.21.15.94:0/4192467254 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645e50f9990 con 0x5645e4da3f60 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe54ffff640 10 client.17082 handle_command_reply: tid=0 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 client.17082 shutdown 2024-04-25T13:46:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 msgr2=0x5645e5191560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe568004010 tx=0x7fe568009d50 comp rx=0 tx=0).stop 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 shutdown_connections 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4da3f60 0x5645e519f060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4de5390 0x5645e5198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4dc46b0 0x5645e5191aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 --2- 172.21.15.94:0/4192467254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dbc4d0 0x5645e5191560 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe54ffff640 0 client.17082 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 >> 172.21.15.94:0/4192467254 conn(0x5645e4dd94e0 msgr2=0x5645e5185f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.774+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 shutdown_connections 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.775+0000 7fe57de85740 1 -- 172.21.15.94:0/4192467254 wait complete. 2024-04-25T13:46:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.775+0000 7fe57de85740 20 client.17082 trim_cache size 0 max 16384 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 Processor -- start 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 -- start start 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5182e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5185e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701ad810 con 0x5645e4dc46b0 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701af350 con 0x5645e4de5390 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe57de85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701adee0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56468/0 (socket says 172.21.15.94:56468) 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 -- 172.21.15.94:0/2016782786 learned_addr learned my addr 172.21.15.94:0/2016782786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe575573640 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5185e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe574d72640 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5182e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 -- 172.21.15.94:0/2016782786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 msgr2=0x5645e5182e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5182e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 -- 172.21.15.94:0/2016782786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 msgr2=0x5645e5185e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5185e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.789+0000 7fe56ffff640 1 -- 172.21.15.94:0/2016782786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e5189410 con 0x5645e4dc46b0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.790+0000 7fe56ffff640 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe5640c7670 tx=0x7fe564101040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd5c90db1b21c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.790+0000 7fe5567fc640 1 -- 172.21.15.94:0/2016782786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564049040 con 0x5645e4dc46b0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.790+0000 7fe5567fc640 1 -- 172.21.15.94:0/2016782786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56403b460 con 0x5645e4dc46b0 2024-04-25T13:46:11.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.790+0000 7fe5567fc640 1 -- 172.21.15.94:0/2016782786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640c76e0 con 0x5645e4dc46b0 2024-04-25T13:46:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- 172.21.15.94:0/2016782786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 msgr2=0x5645e5198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe5640c7670 tx=0x7fe564101040 comp rx=0 tx=0).stop 2024-04-25T13:46:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- 172.21.15.94:0/2016782786 shutdown_connections 2024-04-25T13:46:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5185e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5198920 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- 172.21.15.94:0/2016782786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5182e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- 172.21.15.94:0/2016782786 >> 172.21.15.94:0/2016782786 conn(0x5645e4dd94e0 msgr2=0x5645e5182720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- 172.21.15.94:0/2016782786 shutdown_connections 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- 172.21.15.94:0/2016782786 wait complete. 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 -- start start 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 Processor -- start 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5198890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.791+0000 7fe57de85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5195a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701aec80 con 0x5645e4dc46b0 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701ae5b0 con 0x5645e4de5390 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701ab7f0 con 0x5645e4dbc4d0 2024-04-25T13:46:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 10 client.0 _ll_register_callbacks cb 0x7fe5640c3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56476/0 (socket says 172.21.15.94:56476) 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 -- 172.21.15.94:0/503482345 learned_addr learned my addr 172.21.15.94:0/503482345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe575573640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe574d72640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 -- 172.21.15.94:0/503482345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 msgr2=0x5645e5198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 -- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 msgr2=0x5645e5195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 -- 172.21.15.94:0/503482345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e5205430 con 0x5645e4dc46b0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe56ffff640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe564105040 tx=0x7fe564005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3896f4149a89c8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe54ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564049070 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fe5640c3970 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645e50f3800 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5640cba90 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 10 client.17088 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.792+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5645e52066a0 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5645e50f3800 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe5640d00b0 con 0x5645e4dc46b0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 10 client.17088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 20 client.17088 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4d793a0 0x5645e52090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 4 client.17088 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645e506e610 con 0x5645e4d793a0 2024-04-25T13:46:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe574d72640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4d793a0 0x5645e52090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe54ffff640 10 client.17088 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.793+0000 7fe574d72640 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4d793a0 0x5645e52090c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0696e7c720667a2 server_cookie=a848b2edd899507 in_seq=0 out_seq=0 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe54ffff640 1 -- 172.21.15.94:0/503482345 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645e506e610 con 0x5645e4d793a0 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe54ffff640 10 client.17088 handle_command_reply: tid=0 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 client.17088 shutdown 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 msgr2=0x5645e5195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe564105040 tx=0x7fe564005e50 comp rx=0 tx=0).stop 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 shutdown_connections 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5645e4d793a0 0x5645e52090c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645e4de5390 0x5645e5195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645e4dc46b0 0x5645e5195410 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe54ffff640 0 client.17088 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 --2- 172.21.15.94:0/503482345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645e4dbc4d0 0x5645e5198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 >> 172.21.15.94:0/503482345 conn(0x5645e4dd94e0 msgr2=0x5645e51865b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 shutdown_connections 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 1 -- 172.21.15.94:0/503482345 wait complete. 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.794+0000 7fe57de85740 20 client.17088 trim_cache size 0 max 16384 2024-04-25T13:46:11.801 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:11.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 msgr2=0x7fe5580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 0x7fe5580b2000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5645e5188860 tx=0x7fe560004730 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 msgr2=0x7fe5700fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe564006010 tx=0x7fe564031a50 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 shutdown_connections 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe5580afaf0 0x7fe5580b2000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570108930 0x7fe5700fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570103c30 0x7fe5700fe0f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 --2- 172.21.15.94:0/2467700204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5701029f0 0x7fe5700ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 >> 172.21.15.94:0/2467700204 conn(0x7fe5700fdbc0 msgr2=0x7fe570109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 shutdown_connections 2024-04-25T13:46:11.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.795+0000 7fe576fd6640 1 -- 172.21.15.94:0/2467700204 wait complete. 2024-04-25T13:46:11.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 440143 13.10 MB/sec execute 54 sec latency 9.804 ms 2024-04-25T13:46:11.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:46:11.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:11 smithi137 ceph-mon[32568]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 -- 172.21.15.94:0/56206624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28201029e0 msgr2=0x7f28201036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 --2- 172.21.15.94:0/56206624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28201029e0 0x7f28201036e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2814009900 tx=0x7f281402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 -- 172.21.15.94:0/56206624 shutdown_connections 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 --2- 172.21.15.94:0/56206624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f282010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 --2- 172.21.15.94:0/56206624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820103c20 0x7f2820104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 --2- 172.21.15.94:0/56206624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28201029e0 0x7f28201036e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.937+0000 7f2826b6c640 1 -- 172.21.15.94:0/56206624 >> 172.21.15.94:0/56206624 conn(0x7f28200fdbd0 msgr2=0x7f2820100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- 172.21.15.94:0/56206624 shutdown_connections 2024-04-25T13:46:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- 172.21.15.94:0/56206624 wait complete. 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 Processor -- start 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- start start 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28201029e0 0x7f2820199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f28201a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f282010e4e0 con 0x7f2820103c20 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f282010e360 con 0x7f2820108920 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.938+0000 7f2826b6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f282010e660 con 0x7f28201029e0 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f2825109640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f28201a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f2824908640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28201029e0 0x7f2820199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56498/0 (socket says 172.21.15.94:56498) 2024-04-25T13:46:11.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f2825109640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f28201a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55596/0 (socket says 172.21.15.94:55596) 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 -- 172.21.15.94:0/3561688782 learned_addr learned my addr 172.21.15.94:0/3561688782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 -- 172.21.15.94:0/3561688782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28201029e0 msgr2=0x7f2820199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28201029e0 0x7f2820199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 -- 172.21.15.94:0/3561688782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 msgr2=0x7f28201a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f28201a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 -- 172.21.15.94:0/3561688782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28201a0fb0 con 0x7f2820103c20 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281ffff640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9f0 tx=0x7f280c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281dffb640 1 -- 172.21.15.94:0/3561688782 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c00cd80 con 0x7f2820103c20 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281dffb640 1 -- 172.21.15.94:0/3561688782 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c0054d0 con 0x7f2820103c20 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28201a12a0 con 0x7f2820103c20 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f281dffb640 1 -- 172.21.15.94:0/3561688782 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c010690 con 0x7f2820103c20 2024-04-25T13:46:11.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.939+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201a17a0 con 0x7f2820103c20 2024-04-25T13:46:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f281dffb640 1 -- 172.21.15.94:0/3561688782 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f280c004e70 con 0x7f2820103c20 2024-04-25T13:46:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 Processor -- start 2024-04-25T13:46:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 -- start start 2024-04-25T13:46:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 0x55e6b0b53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0ab7d80 con 0x55e6b07b1390 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0a71a20 con 0x55e6b07906b0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f282da19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0ab3d40 con 0x55e6b07884d0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.942+0000 7f281dffb640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 0x7f28100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 0x55e6b0b53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56506/0 (socket says 172.21.15.94:56506) 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 -- 172.21.15.94:0/623108750 learned_addr learned my addr 172.21.15.94:0/623108750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f281ffff640 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 0x7f28100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f281dffb640 1 -- 172.21.15.94:0/3561688782 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f280c0df0a0 con 0x7f2820103c20 2024-04-25T13:46:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 -- 172.21.15.94:0/623108750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 msgr2=0x55e6b0b54a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b54a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 -- 172.21.15.94:0/623108750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 msgr2=0x55e6b0b53810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 -- 172.21.15.94:0/623108750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b0a7cc30 con 0x55e6b07884d0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2824908640 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 0x55e6b0b53d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f281402a930 tx=0x7f2814000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdef17073fe84eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f2825109640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 0x7f28100b2000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e6b0b54860 tx=0x7f2818009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f280affd640 1 -- 172.21.15.94:0/623108750 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814035030 con 0x55e6b07884d0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f280affd640 1 -- 172.21.15.94:0/623108750 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2814040070 con 0x55e6b07884d0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.943+0000 7f280affd640 1 -- 172.21.15.94:0/623108750 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814003870 con 0x55e6b07884d0 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 -- 172.21.15.94:0/623108750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 msgr2=0x55e6b0b53d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 0x55e6b0b53d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f281402a930 tx=0x7f2814000bc0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 -- 172.21.15.94:0/623108750 shutdown_connections 2024-04-25T13:46:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07884d0 0x55e6b0b53d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 --2- 172.21.15.94:0/623108750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07b1390 0x55e6b0b53810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 -- 172.21.15.94:0/623108750 >> 172.21.15.94:0/623108750 conn(0x55e6b07a54e0 msgr2=0x55e6b0b4f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 -- 172.21.15.94:0/623108750 shutdown_connections 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.944+0000 7f282da19740 1 -- 172.21.15.94:0/623108750 wait complete. 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 -- start start 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 Processor -- start 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b5dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b64a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0b554d0 con 0x55e6b07884d0 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0a3edc0 con 0x55e6b07906b0 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6b0b55650 con 0x55e6b07b1390 2024-04-25T13:46:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f282da19740 10 client.0 _ll_register_callbacks cb 0x55e6b0a5a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f2824908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f2824908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56512/0 (socket says 172.21.15.94:56512) 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f2824908640 1 -- 172.21.15.94:0/2684872510 learned_addr learned my addr 172.21.15.94:0/2684872510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.945+0000 7f2825109640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b64a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f281ffff640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b5dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 -- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 msgr2=0x55e6b0b64a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b64a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 -- 172.21.15.94:0/2684872510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 msgr2=0x55e6b0b5dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b5dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 -- 172.21.15.94:0/2684872510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b07c93f0 con 0x55e6b07884d0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f2824908640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2814005ab0 tx=0x7f2814004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964244f3d8733034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814035040 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f28140315a0 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2814040070 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2814031e20 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f282da19740 10 client.17100 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e6b08937d0 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2814040070 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.946+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f281404f590 con 0x55e6b07884d0 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 10 client.17100 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 10 client.17100 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 20 client.17100 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b076ff60 0x55e6b0b6ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 4 client.17100 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6b0ac5990 con 0x55e6b076ff60 2024-04-25T13:46:11.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f281ffff640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b076ff60 0x55e6b0b6ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f27fffff640 10 client.17100 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.947+0000 7f281ffff640 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b076ff60 0x55e6b0b6ae80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f4ddd1904a6680c server_cookie=37d153ff407ec2e9 in_seq=0 out_seq=0 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f27fffff640 1 -- 172.21.15.94:0/2684872510 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6b0ac5990 con 0x55e6b076ff60 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f27fffff640 10 client.17100 handle_command_reply: tid=0 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 client.17100 shutdown 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 msgr2=0x55e6b0b5d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2814005ab0 tx=0x7f2814004270 comp rx=0 tx=0).stop 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 shutdown_connections 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b076ff60 0x55e6b0b6ae80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b64a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07906b0 0x55e6b0b5dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f27fffff640 0 client.17100 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 --2- 172.21.15.94:0/2684872510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d590 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 >> 172.21.15.94:0/2684872510 conn(0x55e6b07a54e0 msgr2=0x55e6b0b51f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 shutdown_connections 2024-04-25T13:46:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 1 -- 172.21.15.94:0/2684872510 wait complete. 2024-04-25T13:46:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.948+0000 7f282da19740 20 client.17100 trim_cache size 0 max 16384 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 Processor -- start 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 -- start start 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28201aa3f0 con 0x55e6b07906b0 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28201abf30 con 0x55e6b07884d0 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f282da19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28201aaac0 con 0x55e6b07b1390 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56524/0 (socket says 172.21.15.94:56524) 2024-04-25T13:46:11.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 -- 172.21.15.94:0/3660002784 learned_addr learned my addr 172.21.15.94:0/3660002784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f2824908640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f2825109640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 -- 172.21.15.94:0/3660002784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 msgr2=0x55e6b0b6fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b6fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 -- 172.21.15.94:0/3660002784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 msgr2=0x55e6b0b5d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f281ffff640 1 -- 172.21.15.94:0/3660002784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b0b5b6a0 con 0x55e6b07906b0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.963+0000 7f2824908640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.964+0000 7f281ffff640 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9c0 tx=0x7f280c0760f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1414712dc3d3978c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.964+0000 7f280a7fc640 1 -- 172.21.15.94:0/3660002784 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0a6ff0 con 0x55e6b07906b0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.964+0000 7f280a7fc640 1 -- 172.21.15.94:0/3660002784 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c0a7be0 con 0x55e6b07906b0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.964+0000 7f280a7fc640 1 -- 172.21.15.94:0/3660002784 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0a3770 con 0x55e6b07906b0 2024-04-25T13:46:11.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- 172.21.15.94:0/3660002784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 msgr2=0x55e6b0b6f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9c0 tx=0x7f280c0760f0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- 172.21.15.94:0/3660002784 shutdown_connections 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07b1390 0x55e6b0b6fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07906b0 0x55e6b0b6f500 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- 172.21.15.94:0/3660002784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07884d0 0x55e6b0b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- 172.21.15.94:0/3660002784 >> 172.21.15.94:0/3660002784 conn(0x55e6b07a54e0 msgr2=0x55e6b0b5d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- 172.21.15.94:0/3660002784 shutdown_connections 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- 172.21.15.94:0/3660002784 wait complete. 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- start start 2024-04-25T13:46:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 Processor -- start 2024-04-25T13:46:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07906b0 0x55e6b0b637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07b1390 0x55e6b0bd1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28201ab860 con 0x55e6b07884d0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28201ab190 con 0x55e6b07b1390 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.965+0000 7f282da19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28201a83d0 con 0x55e6b07906b0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f282da19740 10 client.0 _ll_register_callbacks cb 0x7f280c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56530/0 (socket says 172.21.15.94:56530) 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 -- 172.21.15.94:0/1132540066 learned_addr learned my addr 172.21.15.94:0/1132540066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f281ffff640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07906b0 0x55e6b0b637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2825109640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07b1390 0x55e6b0bd1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 -- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07906b0 msgr2=0x55e6b0b637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07906b0 0x55e6b0b637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 -- 172.21.15.94:0/1132540066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07b1390 msgr2=0x55e6b0bd1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07b1390 0x55e6b0bd1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 -- 172.21.15.94:0/1132540066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b0bd17d0 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f2824908640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2814009900 tx=0x7f281402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c001fce104fd917b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f27fffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281404fd40 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f281404f470 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c0a7be0 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2814068050 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f282da19740 10 client.17106 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.966+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e6b0bd2ab0 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c0a7be0 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2814003b60 con 0x55e6b07884d0 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 10 client.17106 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 10 client.17106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 20 client.17106 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b07453a0 0x55e6b0bd54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 4 client.17106 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6b0abf800 con 0x55e6b07453a0 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f281ffff640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b07453a0 0x55e6b0bd54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f27fffff640 10 client.17106 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.967+0000 7f281ffff640 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b07453a0 0x55e6b0bd54d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99d69158c468f812 server_cookie=1b69221238d3ae9e in_seq=0 out_seq=0 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f27fffff640 1 -- 172.21.15.94:0/1132540066 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6b0abf800 con 0x55e6b07453a0 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f27fffff640 10 client.17106 handle_command_reply: tid=0 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 client.17106 shutdown 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 msgr2=0x55e6b0b632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2814009900 tx=0x7f281402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 shutdown_connections 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e6b07453a0 0x55e6b0bd54d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6b07b1390 0x55e6b0bd1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6b07906b0 0x55e6b0b637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f27fffff640 0 client.17106 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 --2- 172.21.15.94:0/1132540066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6b07884d0 0x55e6b0b632b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 >> 172.21.15.94:0/1132540066 conn(0x55e6b07a54e0 msgr2=0x55e6b0b525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 shutdown_connections 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 1 -- 172.21.15.94:0/1132540066 wait complete. 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f282da19740 20 client.17106 trim_cache size 0 max 16384 2024-04-25T13:46:11.975 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 msgr2=0x7f28100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 0x7f28100b2000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e6b0b54860 tx=0x7f2818009500 comp rx=0 tx=0).stop 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 msgr2=0x7f28201999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9f0 tx=0x7f280c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 shutdown_connections 2024-04-25T13:46:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28100afaf0 0x7f28100b2000 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2820108920 0x7f28201a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2820103c20 0x7f28201999b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.968+0000 7f2826b6c640 1 --2- 172.21.15.94:0/3561688782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28201029e0 0x7f2820199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.969+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 >> 172.21.15.94:0/3561688782 conn(0x7f28200fdbd0 msgr2=0x7f2820104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.969+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 shutdown_connections 2024-04-25T13:46:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:11.969+0000 7f2826b6c640 1 -- 172.21.15.94:0/3561688782 wait complete. 2024-04-25T13:46:11.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:46:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 -- 172.21.15.94:0/3112446736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 msgr2=0x7f260410ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 --2- 172.21.15.94:0/3112446736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f260410ad50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ca0 tx=0x7f25fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 -- 172.21.15.94:0/3112446736 shutdown_connections 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 --2- 172.21.15.94:0/3112446736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f260410ad50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 --2- 172.21.15.94:0/3112446736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103c30 0x7f2604104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 --2- 172.21.15.94:0/3112446736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26041029f0 0x7f26041036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 -- 172.21.15.94:0/3112446736 >> 172.21.15.94:0/3112446736 conn(0x7f26040fdc00 msgr2=0x7f2604100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 -- 172.21.15.94:0/3112446736 shutdown_connections 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.094+0000 7f260b834640 1 -- 172.21.15.94:0/3112446736 wait complete. 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 Processor -- start 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 -- start start 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 0x7f260406bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 0x7f26041a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e500 con 0x7f2604108930 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e380 con 0x7f2604103c30 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f260b834640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e680 con 0x7f26041029f0 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56542/0 (socket says 172.21.15.94:56542) 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f2609dd1640 1 -- 172.21.15.94:0/1605755642 learned_addr learned my addr 172.21.15.94:0/1605755642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f2608dcf640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 0x7f26041a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f26095d0640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 0x7f260406bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.095+0000 7f2608dcf640 1 -- 172.21.15.94:0/1605755642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 msgr2=0x7f260406bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2608dcf640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 0x7f260406bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2608dcf640 1 -- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 msgr2=0x7f26041a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2608dcf640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2608dcf640 1 -- 172.21.15.94:0/1605755642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041a3ee0 con 0x7f2604103c30 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2609dd1640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f26095d0640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 0x7f260406bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f2608dcf640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 0x7f26041a1350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9f0 tx=0x7f25f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f25fa7fc640 1 -- 172.21.15.94:0/1605755642 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400ccc0 con 0x7f2604103c30 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a41d0 con 0x7f2604103c30 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f25fa7fc640 1 -- 172.21.15.94:0/1605755642 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f40108e0 con 0x7f2604103c30 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f25fa7fc640 1 -- 172.21.15.94:0/1605755642 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4018880 con 0x7f2604103c30 2024-04-25T13:46:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.096+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a4720 con 0x7f2604103c30 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f25fa7fc640 1 -- 172.21.15.94:0/1605755642 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f25f4010430 con 0x7f2604103c30 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 Processor -- start 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 -- start start 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e458638d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458639a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e45859cd80 con 0x55e45826d4d0 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e458556a20 con 0x55e458296390 2024-04-25T13:46:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26127be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e458598d40 con 0x55e4582756b0 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f26095d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e458638d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55674/0 (socket says 172.21.15.94:55674) 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f2609dd1640 1 -- 172.21.15.94:0/1884989040 learned_addr learned my addr 172.21.15.94:0/1884989040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.099+0000 7f2608dcf640 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458639a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 -- 172.21.15.94:0/1884989040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 msgr2=0x55e458639a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458639a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 -- 172.21.15.94:0/1884989040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 msgr2=0x55e458638d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e458638d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 -- 172.21.15.94:0/1884989040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e458561c30 con 0x55e458296390 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f25fa7fc640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 0x7f25f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f26095d0640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 0x7f25f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f2609dd1640 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02f420 tx=0x7f25fc02f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596e56d4d0557ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f25fa7fc640 1 -- 172.21.15.94:0/1605755642 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f25f40df010 con 0x7f2604103c30 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f25ef7fe640 1 -- 172.21.15.94:0/1884989040 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc031650 con 0x55e458296390 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f25ef7fe640 1 -- 172.21.15.94:0/1884989040 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc036040 con 0x55e458296390 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.100+0000 7f25ef7fe640 1 -- 172.21.15.94:0/1884989040 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004610 con 0x55e458296390 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26095d0640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 0x7f25f00b2000 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e458638530 tx=0x7f2600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- 172.21.15.94:0/1884989040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 msgr2=0x55e458638810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02f420 tx=0x7f25fc02f450 comp rx=0 tx=0).stop 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- 172.21.15.94:0/1884989040 shutdown_connections 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458639a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e458638d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 --2- 172.21.15.94:0/1884989040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458296390 0x55e458638810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- 172.21.15.94:0/1884989040 >> 172.21.15.94:0/1884989040 conn(0x55e45828a4e0 msgr2=0x55e458634020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- 172.21.15.94:0/1884989040 shutdown_connections 2024-04-25T13:46:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- 172.21.15.94:0/1884989040 wait complete. 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 -- start start 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.101+0000 7f26127be740 1 Processor -- start 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e458642410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e458642950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e45863a350 con 0x55e458296390 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e458523dc0 con 0x55e4582756b0 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e45863a4d0 con 0x55e45826d4d0 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26127be740 10 client.0 _ll_register_callbacks cb 0x55e45853f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56568/0 (socket says 172.21.15.94:56568) 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 -- 172.21.15.94:0/317185607 learned_addr learned my addr 172.21.15.94:0/317185607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2608dcf640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e458642950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26095d0640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e458642410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 -- 172.21.15.94:0/317185607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 msgr2=0x55e458642410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e458642410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 -- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 msgr2=0x55e458642950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e458642950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f2609dd1640 1 -- 172.21.15.94:0/317185607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4582ae3f0 con 0x55e458296390 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.102+0000 7f26095d0640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e458642410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f2609dd1640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f25fc033040 tx=0x7f25fc004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c190e107170af810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009360 con 0x55e458296390 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f25fc003b80 con 0x55e458296390 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc036040 con 0x55e458296390 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25fc004850 con 0x55e458296390 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 10 client.17118 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4583787d0 con 0x55e458296390 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc036040 con 0x55e458296390 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f25fc04f070 con 0x55e458296390 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 10 client.17118 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 10 client.17118 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 20 client.17118 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458254f60 0x55e45864fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 4 client.17118 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.103+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4585aa990 con 0x55e458254f60 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.104+0000 7f26095d0640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458254f60 0x55e45864fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.104+0000 7f25ecff9640 10 client.17118 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.104+0000 7f26095d0640 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458254f60 0x55e45864fd00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef0f9a069657a0bb server_cookie=2775360c628bf93 in_seq=0 out_seq=0 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f25ecff9640 1 -- 172.21.15.94:0/317185607 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4585aa990 con 0x55e458254f60 2024-04-25T13:46:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f25ecff9640 10 client.17118 handle_command_reply: tid=0 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 client.17118 shutdown 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 msgr2=0x55e4586498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f25fc033040 tx=0x7f25fc004660 comp rx=0 tx=0).stop 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 shutdown_connections 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458254f60 0x55e45864fd00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458296390 0x55e4586498b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f25ecff9640 0 client.17118 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e458642950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 --2- 172.21.15.94:0/317185607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e458642410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 >> 172.21.15.94:0/317185607 conn(0x55e45828a4e0 msgr2=0x55e458633040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 shutdown_connections 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 1 -- 172.21.15.94:0/317185607 wait complete. 2024-04-25T13:46:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.105+0000 7f26127be740 20 client.17118 trim_cache size 0 max 16384 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 Processor -- start 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 -- start start 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e4586497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e4586544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26041ad820 con 0x55e458254f60 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26041af360 con 0x55e4582756b0 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26127be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26041adef0 con 0x55e45826d4d0 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56572/0 (socket says 172.21.15.94:56572) 2024-04-25T13:46:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 -- 172.21.15.94:0/465593707 learned_addr learned my addr 172.21.15.94:0/465593707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f2609dd1640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e4586544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 -- 172.21.15.94:0/465593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 msgr2=0x55e4586497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f2608dcf640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e4586497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e4586497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 -- 172.21.15.94:0/465593707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 msgr2=0x55e4586544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e4586544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 -- 172.21.15.94:0/465593707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e45863dd20 con 0x55e458254f60 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f2608dcf640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e4586497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.119+0000 7f26095d0640 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f26041022c0 tx=0x7f2600017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eb83f4d09e512f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f25eeffd640 1 -- 172.21.15.94:0/465593707 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600003c40 con 0x55e458254f60 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f25eeffd640 1 -- 172.21.15.94:0/465593707 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2600003de0 con 0x55e458254f60 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f25eeffd640 1 -- 172.21.15.94:0/465593707 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260002b9d0 con 0x55e458254f60 2024-04-25T13:46:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 -- 172.21.15.94:0/465593707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 msgr2=0x55e458642af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f26041022c0 tx=0x7f2600017000 comp rx=0 tx=0).stop 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 -- 172.21.15.94:0/465593707 shutdown_connections 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4582756b0 0x55e4586544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e45826d4d0 0x55e4586497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 --2- 172.21.15.94:0/465593707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e458254f60 0x55e458642af0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 -- 172.21.15.94:0/465593707 >> 172.21.15.94:0/465593707 conn(0x55e45828a4e0 msgr2=0x55e4586423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 -- 172.21.15.94:0/465593707 shutdown_connections 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.120+0000 7f26127be740 1 -- 172.21.15.94:0/465593707 wait complete. 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 -- start start 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 Processor -- start 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458254f60 0x55e458646f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458648360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26041aec90 con 0x55e45826d4d0 2024-04-25T13:46:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26041ae5c0 con 0x55e458254f60 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26041ab800 con 0x55e4582756b0 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26127be740 10 client.0 _ll_register_callbacks cb 0x7f2600009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f2608dcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f2608dcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56586/0 (socket says 172.21.15.94:56586) 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f2608dcf640 1 -- 172.21.15.94:0/3225764650 learned_addr learned my addr 172.21.15.94:0/3225764650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f26095d0640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458254f60 0x55e458646f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.121+0000 7f2609dd1640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458648360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 -- 172.21.15.94:0/3225764650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 msgr2=0x55e458648360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458648360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 -- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458254f60 msgr2=0x55e458646f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458254f60 0x55e458646f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 -- 172.21.15.94:0/3225764650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e458648920 con 0x55e45826d4d0 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f2608dcf640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f25f40a4770 tx=0x7f25f400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cf2bf0136030f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f40a72d0 con 0x55e45826d4d0 2024-04-25T13:46:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f25f4010a90 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2600003de0 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25f40a6c70 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 10 client.17124 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e458649be0 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2600003de0 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f25f40e2ca0 con 0x55e45826d4d0 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 10 client.17124 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 10 client.17124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 20 client.17124 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.122+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458296390 0x55e4586ba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 4 client.17124 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4585a4800 con 0x55e458296390 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26095d0640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458296390 0x55e4586ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f25ecff9640 10 client.17124 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26095d0640 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458296390 0x55e4586ba880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19e20f3c2924b931 server_cookie=bf5ee1a9deea0aa1 in_seq=0 out_seq=0 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f25ecff9640 1 -- 172.21.15.94:0/3225764650 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4585a4800 con 0x55e458296390 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f25ecff9640 10 client.17124 handle_command_reply: tid=0 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 client.17124 shutdown 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 msgr2=0x55e4586474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f25f40a4770 tx=0x7f25f400cef0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 shutdown_connections 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e458296390 0x55e4586ba880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4582756b0 0x55e458648360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e45826d4d0 0x55e4586474a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f25ecff9640 0 client.17124 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 --2- 172.21.15.94:0/3225764650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e458254f60 0x55e458646f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.123+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 >> 172.21.15.94:0/3225764650 conn(0x55e45828a4e0 msgr2=0x55e458636e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 shutdown_connections 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f26127be740 1 -- 172.21.15.94:0/3225764650 wait complete. 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f26127be740 20 client.17124 trim_cache size 0 max 16384 2024-04-25T13:46:12.132 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 msgr2=0x7f25f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 0x7f25f00b2000 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e458638530 tx=0x7f2600005f90 comp rx=0 tx=0).stop 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 msgr2=0x7f26041a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 0x7f26041a1350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e9f0 tx=0x7f25f400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 shutdown_connections 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25f00afaf0 0x7f25f00b2000 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108930 0x7f26041a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604103c30 0x7f26041a1350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 --2- 172.21.15.94:0/1605755642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26041029f0 0x7f260406bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 >> 172.21.15.94:0/1605755642 conn(0x7f26040fdc00 msgr2=0x7f2604104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 shutdown_connections 2024-04-25T13:46:12.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.124+0000 7f260b834640 1 -- 172.21.15.94:0/1605755642 wait complete. 2024-04-25T13:46:12.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 -- 172.21.15.94:0/285741139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c781029e0 msgr2=0x7f1c781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/285741139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c781029e0 0x7f1c781036e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009900 tx=0x7f1c7402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 -- 172.21.15.94:0/285741139 shutdown_connections 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/285741139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 0x7f1c7810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/285741139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c78103c20 0x7f1c78104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/285741139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c781029e0 0x7f1c781036e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 -- 172.21.15.94:0/285741139 >> 172.21.15.94:0/285741139 conn(0x7f1c780fdbd0 msgr2=0x7f1c78100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 -- 172.21.15.94:0/285741139 shutdown_connections 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.249+0000 7f1c7fce9640 1 -- 172.21.15.94:0/285741139 wait complete. 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 Processor -- start 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 -- start start 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c781029e0 0x7f1c7819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 0x7f1c781a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810e4e0 con 0x7f1c78103c20 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810e360 con 0x7f1c78108920 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7fce9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810e660 con 0x7f1c781029e0 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56598/0 (socket says 172.21.15.94:56598) 2024-04-25T13:46:12.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 -- 172.21.15.94:0/272420976 learned_addr learned my addr 172.21.15.94:0/272420976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7da85640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c781029e0 0x7f1c7819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7e286640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 0x7f1c781a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 -- 172.21.15.94:0/272420976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c781029e0 msgr2=0x7f1c7819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c781029e0 0x7f1c7819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 -- 172.21.15.94:0/272420976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 msgr2=0x7f1c781a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 0x7f1c781a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.250+0000 7f1c7d284640 1 -- 172.21.15.94:0/272420976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c781a20e0 con 0x7f1c78103c20 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c7d284640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800d770 tx=0x7f1c6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c6effd640 1 -- 172.21.15.94:0/272420976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68017070 con 0x7f1c78103c20 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781a23d0 con 0x7f1c78103c20 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c6effd640 1 -- 172.21.15.94:0/272420976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c680054d0 con 0x7f1c78103c20 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c6effd640 1 -- 172.21.15.94:0/272420976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004c30 con 0x7f1c78103c20 2024-04-25T13:46:12.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.251+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a2920 con 0x7f1c78103c20 2024-04-25T13:46:12.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 Processor -- start 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 -- start start 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f751390 0x55d04faf3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04faf3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c6effd640 1 -- 172.21.15.94:0/272420976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c68004e10 con 0x7f1c78103c20 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d04fa57d80 con 0x55d04f7284d0 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d04fa11a20 con 0x55d04f751390 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c86b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d04fa53d40 con 0x55d04f7306b0 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04faf3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7d284640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7e286640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f751390 0x55d04faf3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7d284640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41648/0 (socket says 172.21.15.94:41648) 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04faf3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56604/0 (socket says 172.21.15.94:56604) 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.254+0000 7f1c7d284640 1 -- 172.21.15.94:0/4173480826 learned_addr learned my addr 172.21.15.94:0/4173480826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 -- 172.21.15.94:0/4173480826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f751390 msgr2=0x55d04faf3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f751390 0x55d04faf3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 -- 172.21.15.94:0/4173480826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 msgr2=0x55d04faf3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04faf3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 -- 172.21.15.94:0/4173480826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04fa1cc30 con 0x55d04f7306b0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7d284640 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c680e4c10 tx=0x7f1c680e6690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f43256d4591320e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c6effd640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 0x7f1c640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c63fff640 1 -- 172.21.15.94:0/4173480826 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c680ed070 con 0x55d04f7306b0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c7e286640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 0x7f1c640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c63fff640 1 -- 172.21.15.94:0/4173480826 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c68090f30 con 0x55d04f7306b0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c63fff640 1 -- 172.21.15.94:0/4173480826 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c680a6510 con 0x55d04f7306b0 2024-04-25T13:46:12.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.255+0000 7f1c6effd640 1 -- 172.21.15.94:0/272420976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1c680de6d0 con 0x7f1c78103c20 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c7e286640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 0x7f1c640b2000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55d04faf4860 tx=0x7f1c70009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 -- 172.21.15.94:0/4173480826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 msgr2=0x55d04faf4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c680e4c10 tx=0x7f1c680e6690 comp rx=0 tx=0).stop 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 -- 172.21.15.94:0/4173480826 shutdown_connections 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04faf4a20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04faf3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 --2- 172.21.15.94:0/4173480826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f751390 0x55d04faf3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 -- 172.21.15.94:0/4173480826 >> 172.21.15.94:0/4173480826 conn(0x55d04f7454e0 msgr2=0x55d04faef020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 -- 172.21.15.94:0/4173480826 shutdown_connections 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.256+0000 7f1c86b95740 1 -- 172.21.15.94:0/4173480826 wait complete. 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 -- start start 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 Processor -- start 2024-04-25T13:46:12.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb01de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb08d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d04faf5530 con 0x55d04f7284d0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d04f9dedc0 con 0x55d04f7306b0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d04faf56b0 con 0x55d04f751390 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c86b95740 10 client.0 _ll_register_callbacks cb 0x55d04f9fa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56614/0 (socket says 172.21.15.94:56614) 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c7da85640 1 -- 172.21.15.94:0/2858960970 learned_addr learned my addr 172.21.15.94:0/2858960970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c7e286640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb08d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.257+0000 7f1c7d284640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb01de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 -- 172.21.15.94:0/2858960970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 msgr2=0x55d04fb08d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb08d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 -- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 msgr2=0x55d04fb01de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb01de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 -- 172.21.15.94:0/2858960970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04f7693f0 con 0x55d04f7284d0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c7da85640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009230 tx=0x7f1c74031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54f175541eb231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74035040 con 0x55d04f7284d0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1c74004240 con 0x55d04f7284d0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c68090f30 con 0x55d04f7284d0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c740318f0 con 0x55d04f7284d0 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c86b95740 10 client.17136 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:12.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d04f8337d0 con 0x55d04f7284d0 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.258+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c68090f30 con 0x55d04f7284d0 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c74044020 con 0x55d04f7284d0 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 10 client.17136 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 20 client.17136 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f70ff60 0x55d04fb0f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 4 client.17136 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04fa65990 con 0x55d04f70ff60 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c7d284640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f70ff60 0x55d04fb0f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c617fa640 10 client.17136 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:12.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.259+0000 7f1c7d284640 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f70ff60 0x55d04fb0f550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4e20e4fd6e882f server_cookie=ac1925d79da4f483 in_seq=0 out_seq=0 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c617fa640 1 -- 172.21.15.94:0/2858960970 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d04fa65990 con 0x55d04f70ff60 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c617fa640 10 client.17136 handle_command_reply: tid=0 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 client.17136 shutdown 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 msgr2=0x55d04fb018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009230 tx=0x7f1c74031e80 comp rx=0 tx=0).stop 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 shutdown_connections 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f70ff60 0x55d04fb0f550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb08d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb01de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c617fa640 0 client.17136 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 --2- 172.21.15.94:0/2858960970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb018a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 >> 172.21.15.94:0/2858960970 conn(0x55d04f7454e0 msgr2=0x55d04faee040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 shutdown_connections 2024-04-25T13:46:12.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 1 -- 172.21.15.94:0/2858960970 wait complete. 2024-04-25T13:46:12.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.261+0000 7f1c86b95740 20 client.17136 trim_cache size 0 max 16384 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 Processor -- start 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 -- start start 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7284d0 0x55d04fb00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04fb08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781a92f0 con 0x55d04f751390 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781aae30 con 0x55d04f7284d0 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c86b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781a99c0 con 0x55d04f7306b0 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56630/0 (socket says 172.21.15.94:56630) 2024-04-25T13:46:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 -- 172.21.15.94:0/1856729494 learned_addr learned my addr 172.21.15.94:0/1856729494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7d284640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04fb08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7da85640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7284d0 0x55d04fb00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 -- 172.21.15.94:0/1856729494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 msgr2=0x55d04fb08c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04fb08c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 -- 172.21.15.94:0/1856729494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7284d0 msgr2=0x55d04fb00840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7284d0 0x55d04fb00840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 -- 172.21.15.94:0/1856729494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04faf8c20 con 0x55d04f751390 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7d284640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04fb08c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.277+0000 7f1c7e286640 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1c70003f70 tx=0x7f1c7001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768e5037183116c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.278+0000 7f1c637fe640 1 -- 172.21.15.94:0/1856729494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000f6e0 con 0x55d04f751390 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.278+0000 7f1c637fe640 1 -- 172.21.15.94:0/1856729494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7000fd30 con 0x55d04f751390 2024-04-25T13:46:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.278+0000 7f1c637fe640 1 -- 172.21.15.94:0/1856729494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70029e10 con 0x55d04f751390 2024-04-25T13:46:12.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- 172.21.15.94:0/1856729494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 msgr2=0x55d04fb01470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1c70003f70 tx=0x7f1c7001a000 comp rx=0 tx=0).stop 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- 172.21.15.94:0/1856729494 shutdown_connections 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f751390 0x55d04fb01470 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f7306b0 0x55d04fb08c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 --2- 172.21.15.94:0/1856729494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7284d0 0x55d04fb00840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- 172.21.15.94:0/1856729494 >> 172.21.15.94:0/1856729494 conn(0x55d04f7454e0 msgr2=0x55d04fb00130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- 172.21.15.94:0/1856729494 shutdown_connections 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- 172.21.15.94:0/1856729494 wait complete. 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 -- start start 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.279+0000 7f1c86b95740 1 Processor -- start 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781aa760 con 0x55d04f7284d0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781aa090 con 0x55d04f7306b0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781a72d0 con 0x55d04f751390 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c86b95740 10 client.0 _ll_register_callbacks cb 0x7f1c74009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56646/0 (socket says 172.21.15.94:56646) 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 -- 172.21.15.94:0/544717298 learned_addr learned my addr 172.21.15.94:0/544717298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7d284640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7e286640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 -- 172.21.15.94:0/544717298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 msgr2=0x55d04fb07790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb07790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 -- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 msgr2=0x55d04fb06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 -- 172.21.15.94:0/544717298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04fb091a0 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7d284640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c7da85640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1c74046610 tx=0x7f1c74005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3491b71142cd98c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.280+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74035040 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f1c74004450 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7000fd30 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c74046ab0 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 10 client.17142 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d04fb08e70 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7000fd30 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c74044040 con 0x55d04f7284d0 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 10 client.17142 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:12.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 20 client.17142 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f6e53a0 0x55d04fb7a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 4 client.17142 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d04fa5f800 con 0x55d04f6e53a0 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.281+0000 7f1c7d284640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f6e53a0 0x55d04fb7a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.282+0000 7f1c617fa640 10 client.17142 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:12.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.282+0000 7f1c7d284640 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f6e53a0 0x55d04fb7a040 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28c9a178f825c39 server_cookie=e8b2e1c7cd421afe in_seq=0 out_seq=0 2024-04-25T13:46:12.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c617fa640 1 -- 172.21.15.94:0/544717298 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d04fa5f800 con 0x55d04f6e53a0 2024-04-25T13:46:12.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c617fa640 10 client.17142 handle_command_reply: tid=0 2024-04-25T13:46:12.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 client.17142 shutdown 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 msgr2=0x55d04fb06320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1c74046610 tx=0x7f1c74005fd0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 shutdown_connections 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d04f6e53a0 0x55d04fb7a040 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d04f751390 0x55d04fb07790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c617fa640 0 client.17142 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d04f7306b0 0x55d04fb06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 --2- 172.21.15.94:0/544717298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d04f7284d0 0x55d04fb06320 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 >> 172.21.15.94:0/544717298 conn(0x55d04f7454e0 msgr2=0x55d04faf1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 shutdown_connections 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.284+0000 7f1c86b95740 1 -- 172.21.15.94:0/544717298 wait complete. 2024-04-25T13:46:12.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c86b95740 20 client.17142 trim_cache size 0 max 16384 2024-04-25T13:46:12.290 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 msgr2=0x7f1c640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 0x7f1c640b2000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55d04faf4860 tx=0x7f1c70009500 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 msgr2=0x7f1c7819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800d770 tx=0x7f1c6800dc30 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 shutdown_connections 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c640afaf0 0x7f1c640b2000 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c78108920 0x7f1c781a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c78103c20 0x7f1c7819ab50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 --2- 172.21.15.94:0/272420976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c781029e0 0x7f1c7819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 >> 172.21.15.94:0/272420976 conn(0x7f1c780fdbd0 msgr2=0x7f1c78109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 shutdown_connections 2024-04-25T13:46:12.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:12.285+0000 7f1c7fce9640 1 -- 172.21.15.94:0/272420976 wait complete. 2024-04-25T13:46:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:12 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3038593630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3038593630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:12.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 442253 13.00 MB/sec execute 55 sec latency 66.194 ms 2024-04-25T13:46:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:12 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3038593630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:13.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:46:13.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:13.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:46:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 -- 172.21.15.94:0/3718971196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c106f70 msgr2=0x7fd33c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 --2- 172.21.15.94:0/3718971196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c106f70 0x7fd33c109390 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd330009900 tx=0x7fd33002a400 comp rx=0 tx=0).stop 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 -- 172.21.15.94:0/3718971196 shutdown_connections 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 --2- 172.21.15.94:0/3718971196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c10bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 --2- 172.21.15.94:0/3718971196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c106f70 0x7fd33c109390 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 --2- 172.21.15.94:0/3718971196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c104610 0x7fd33c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 -- 172.21.15.94:0/3718971196 >> 172.21.15.94:0/3718971196 conn(0x7fd33c0fdc10 msgr2=0x7fd33c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 -- 172.21.15.94:0/3718971196 shutdown_connections 2024-04-25T13:46:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.349+0000 7fd341582640 1 -- 172.21.15.94:0/3718971196 wait complete. 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.350+0000 7fd341582640 1 Processor -- start 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 -- start start 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c104610 0x7fd33c103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 0x7fd33c102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c10f6f0 con 0x7fd33c104610 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c10f570 con 0x7fd33c106f70 2024-04-25T13:46:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd341582640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c10f870 con 0x7fd33c1098d0 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 0x7fd33c102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 0x7fd33c102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55766/0 (socket says 172.21.15.94:55766) 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd33a7fc640 1 -- 172.21.15.94:0/3683243837 learned_addr learned my addr 172.21.15.94:0/3683243837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.351+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33affd640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c104610 0x7fd33c103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 -- 172.21.15.94:0/3683243837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 msgr2=0x7fd33c102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 0x7fd33c102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 -- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c104610 msgr2=0x7fd33c103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c104610 0x7fd33c103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 -- 172.21.15.94:0/3683243837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c19df60 con 0x7fd33c1098d0 2024-04-25T13:46:13.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.352+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c102640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd334005fd0 tx=0x7fd33400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.353+0000 7fd323fff640 1 -- 172.21.15.94:0/3683243837 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33400cd30 con 0x7fd33c1098d0 2024-04-25T13:46:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.353+0000 7fd323fff640 1 -- 172.21.15.94:0/3683243837 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd334005020 con 0x7fd33c1098d0 2024-04-25T13:46:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.353+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c102ca0 con 0x7fd33c1098d0 2024-04-25T13:46:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.353+0000 7fd323fff640 1 -- 172.21.15.94:0/3683243837 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334010670 con 0x7fd33c1098d0 2024-04-25T13:46:13.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.353+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c1a8b60 con 0x7fd33c1098d0 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.356+0000 7fd348437740 1 Processor -- start 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.356+0000 7fd323fff640 1 -- 172.21.15.94:0/3683243837 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd334010810 con 0x7fd33c1098d0 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.356+0000 7fd348437740 1 -- start start 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.356+0000 7fd348437740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 0x5593fa60c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd348437740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd348437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa613df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd348437740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5593fa572460 con 0x5593fa26c1b0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd348437740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593fa52c590 con 0x5593fa2432f0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd348437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593fa56e420 con 0x5593fa24b4d0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 0x5593fa60c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55782/0 (socket says 172.21.15.94:55782) 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 -- 172.21.15.94:0/1211086503 learned_addr learned my addr 172.21.15.94:0/1211086503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa613df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 -- 172.21.15.94:0/1211086503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 msgr2=0x5593fa613df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa613df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 -- 172.21.15.94:0/1211086503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 msgr2=0x5593fa60c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 0x5593fa60c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33affd640 1 -- 172.21.15.94:0/1211086503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fa5390d0 con 0x5593fa2432f0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 0x5593fa60c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.357+0000 7fd323fff640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 0x7fd3240b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd33affd640 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009a40 tx=0x7fd32c009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec39cd395cbd8391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 0x7fd3240b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd323fff640 1 -- 172.21.15.94:0/3683243837 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd3340df070 con 0x7fd33c1098d0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd3217fa640 1 -- 172.21.15.94:0/1211086503 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c017070 con 0x5593fa2432f0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd3217fa640 1 -- 172.21.15.94:0/1211086503 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32c013470 con 0x5593fa2432f0 2024-04-25T13:46:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.358+0000 7fd3217fa640 1 -- 172.21.15.94:0/1211086503 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c012630 con 0x5593fa2432f0 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 -- 172.21.15.94:0/1211086503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 msgr2=0x5593fa60cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009a40 tx=0x7fd32c009f00 comp rx=0 tx=0).stop 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 0x7fd3240b2070 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5593fa60db40 tx=0x7fd330005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 -- 172.21.15.94:0/1211086503 shutdown_connections 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa613df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa60cef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 --2- 172.21.15.94:0/1211086503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa26c1b0 0x5593fa60c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 -- 172.21.15.94:0/1211086503 >> 172.21.15.94:0/1211086503 conn(0x5593fa260300 msgr2=0x5593fa608300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 -- 172.21.15.94:0/1211086503 shutdown_connections 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.359+0000 7fd348437740 1 -- 172.21.15.94:0/1211086503 wait complete. 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 -- start start 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 Processor -- start 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa6165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa26c1b0 0x5593fa61da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5593fa614840 con 0x5593fa24b4d0 2024-04-25T13:46:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593fa4f9b60 con 0x5593fa2432f0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593fa6149c0 con 0x5593fa26c1b0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd348437740 10 client.0 _ll_register_callbacks cb 0x5593fa51e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56682/0 (socket says 172.21.15.94:56682) 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd33affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa6165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd33a7fc640 1 -- 172.21.15.94:0/2013711643 learned_addr learned my addr 172.21.15.94:0/2013711643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.360+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa26c1b0 0x5593fa61da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 -- 172.21.15.94:0/2013711643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa26c1b0 msgr2=0x5593fa61da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa26c1b0 0x5593fa61da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 -- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 msgr2=0x5593fa6165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa6165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 -- 172.21.15.94:0/2013711643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fa284210 con 0x5593fa24b4d0 2024-04-25T13:46:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd330004ce0 tx=0x7fd330000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7446c7afd8a1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd31eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330035040 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd32c013470 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd330003b80 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd330003d80 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd348437740 10 client.17154 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593fa350620 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.361+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32c013470 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd330048070 con 0x5593fa24b4d0 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 10 client.17154 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 20 client.17154 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa22ad80 0x5593fa624320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 4 client.17154 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593fa580070 con 0x5593fa22ad80 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd33affd640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa22ad80 0x5593fa624320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd33affd640 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa22ad80 0x5593fa624320 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d234a961b5f02f5 server_cookie=28b63618825ad9a4 in_seq=0 out_seq=0 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.362+0000 7fd31eff5640 10 client.17154 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd31eff5640 1 -- 172.21.15.94:0/2013711643 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593fa580070 con 0x5593fa22ad80 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd31eff5640 10 client.17154 handle_command_reply: tid=0 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 client.17154 shutdown 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 msgr2=0x5593fa616af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd330004ce0 tx=0x7fd330000bc0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 shutdown_connections 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa22ad80 0x5593fa624320 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa26c1b0 0x5593fa61da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa24b4d0 0x5593fa616af0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd31eff5640 0 client.17154 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.363+0000 7fd348437740 1 --2- 172.21.15.94:0/2013711643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa6165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.364+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 >> 172.21.15.94:0/2013711643 conn(0x5593fa260300 msgr2=0x5593fa60b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.364+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 shutdown_connections 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.364+0000 7fd348437740 1 -- 172.21.15.94:0/2013711643 wait complete. 2024-04-25T13:46:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.364+0000 7fd348437740 20 client.17154 trim_cache size 0 max 16384 2024-04-25T13:46:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 Processor -- start 2024-04-25T13:46:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 -- start start 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa6083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1b1960 con 0x5593fa22ad80 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1b34a0 con 0x5593fa2432f0 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd348437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1b2030 con 0x5593fa24b4d0 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd33affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa6083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.380+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41716/0 (socket says 172.21.15.94:41716) 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 -- 172.21.15.94:0/2755807344 learned_addr learned my addr 172.21.15.94:0/2755807344 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 -- 172.21.15.94:0/2755807344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 msgr2=0x5593fa61d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 -- 172.21.15.94:0/2755807344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 msgr2=0x5593fa6083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa6083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 -- 172.21.15.94:0/2755807344 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fa611460 con 0x5593fa24b4d0 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3340a47c0 tx=0x7fd3340dca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691947a3d0f913cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd320ff9640 1 -- 172.21.15.94:0/2755807344 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340ab900 con 0x5593fa24b4d0 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd320ff9640 1 -- 172.21.15.94:0/2755807344 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3340c1070 con 0x5593fa24b4d0 2024-04-25T13:46:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.381+0000 7fd320ff9640 1 -- 172.21.15.94:0/2755807344 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340aff20 con 0x5593fa24b4d0 2024-04-25T13:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 -- 172.21.15.94:0/2755807344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 msgr2=0x5593fa60b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3340a47c0 tx=0x7fd3340dca40 comp rx=0 tx=0).stop 2024-04-25T13:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 -- 172.21.15.94:0/2755807344 shutdown_connections 2024-04-25T13:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa60b100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 --2- 172.21.15.94:0/2755807344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa6083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 -- 172.21.15.94:0/2755807344 >> 172.21.15.94:0/2755807344 conn(0x5593fa260300 msgr2=0x5593fa607cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 -- 172.21.15.94:0/2755807344 shutdown_connections 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.382+0000 7fd348437740 1 -- 172.21.15.94:0/2755807344 wait complete. 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 -- start start 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 Processor -- start 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa61b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa61c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1b2dd0 con 0x5593fa22ad80 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1b2700 con 0x5593fa2432f0 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1af940 con 0x5593fa24b4d0 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd348437740 10 client.0 _ll_register_callbacks cb 0x7fd3300045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd33a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55796/0 (socket says 172.21.15.94:55796) 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd33a7fc640 1 -- 172.21.15.94:0/1153041610 learned_addr learned my addr 172.21.15.94:0/1153041610 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa61c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.383+0000 7fd33affd640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa61b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 -- 172.21.15.94:0/1153041610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 msgr2=0x5593fa61c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa61c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 -- 172.21.15.94:0/1153041610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 msgr2=0x5593fa61b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa61b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 -- 172.21.15.94:0/1153041610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fa61cde0 con 0x5593fa2432f0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33b7fe640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa61c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd33a7fc640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd330004ce0 tx=0x7fd33005e450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd630a992d111073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd31eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330035040 con 0x5593fa2432f0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7fd330003980 con 0x5593fa2432f0 2024-04-25T13:46:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3340c1070 con 0x5593fa2432f0 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd33005e940 con 0x5593fa2432f0 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd348437740 10 client.26636 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.384+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593fa68c8a0 con 0x5593fa2432f0 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3340c1070 con 0x5593fa2432f0 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd330048040 con 0x5593fa2432f0 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 10 client.26636 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 10 client.26636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 20 client.26636 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa26c1b0 0x5593fa68f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 4 client.26636 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:46:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593fa579ee0 con 0x5593fa26c1b0 2024-04-25T13:46:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd33affd640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa26c1b0 0x5593fa68f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.385+0000 7fd31eff5640 10 client.26636 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.386+0000 7fd33affd640 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa26c1b0 0x5593fa68f210 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ecb64945b609d3 server_cookie=c9ca91a57d1929d1 in_seq=0 out_seq=0 2024-04-25T13:46:13.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.432+0000 7f7a78b61640 1 -- 172.21.15.94:0/3426805505 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64031810 con 0x7f7a7410a100 2024-04-25T13:46:13.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.432+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3426805505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 msgr2=0x7f7a7410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.432+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3426805505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a7410a510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009900 tx=0x7f7a6402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.437+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3426805505 shutdown_connections 2024-04-25T13:46:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.437+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3426805505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410bb70 0x7f7a7410c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.437+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3426805505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410b220 0x7f7a7410b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.437+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3426805505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a7410a510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.437+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3426805505 >> 172.21.15.94:0/3426805505 conn(0x7f7a74070940 msgr2=0x7f7a74072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.438+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3426805505 shutdown_connections 2024-04-25T13:46:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.440+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3426805505 wait complete. 2024-04-25T13:46:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 Processor -- start 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 -- start start 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a741a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 0x7f7a741a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 0x7f7a741b9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74113190 con 0x7f7a7410bb70 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74113010 con 0x7f7a7410b220 2024-04-25T13:46:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.441+0000 7f7a7bdc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74113310 con 0x7f7a7410a100 2024-04-25T13:46:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.442+0000 7f7a79362640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 0x7f7a741a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.442+0000 7f7a79362640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 0x7f7a741a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55804/0 (socket says 172.21.15.94:55804) 2024-04-25T13:46:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.442+0000 7f7a79362640 1 -- 172.21.15.94:0/3613822552 learned_addr learned my addr 172.21.15.94:0/3613822552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.445+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 0x7f7a741b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a7a364640 1 -- 172.21.15.94:0/3613822552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 msgr2=0x7f7a741a0960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a79b63640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a741a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a741a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a7a364640 1 -- 172.21.15.94:0/3613822552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 msgr2=0x7f7a741a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 0x7f7a741a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.446+0000 7f7a7a364640 1 -- 172.21.15.94:0/3613822552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741b9580 con 0x7f7a7410bb70 2024-04-25T13:46:13.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.447+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 0x7f7a741b9040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00be20 tx=0x7f7a6c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.451+0000 7f7a6affd640 1 -- 172.21.15.94:0/3613822552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c019070 con 0x7f7a7410bb70 2024-04-25T13:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.451+0000 7f7a6affd640 1 -- 172.21.15.94:0/3613822552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c0148f0 con 0x7f7a7410bb70 2024-04-25T13:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.451+0000 7f7a6affd640 1 -- 172.21.15.94:0/3613822552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c012820 con 0x7f7a7410bb70 2024-04-25T13:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.452+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741b9750 con 0x7f7a7410bb70 2024-04-25T13:46:13.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.452+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741b9cd0 con 0x7f7a7410bb70 2024-04-25T13:46:13.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.457+0000 7f7a82c73740 1 Processor -- start 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.461+0000 7f7a82c73740 1 -- start start 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af77c410 0x55c7afb1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb24920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c7afa82b00 con 0x55c7af753550 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7afa1e920 con 0x55c7af75b730 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a82c73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7afa7eac0 con 0x55c7af77c410 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a79b63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a79b63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39998/0 (socket says 172.21.15.94:39998) 2024-04-25T13:46:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.462+0000 7f7a79b63640 1 -- 172.21.15.94:0/1778475745 learned_addr learned my addr 172.21.15.94:0/1778475745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a6affd640 1 -- 172.21.15.94:0/3613822552 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7a6c01c070 con 0x7f7a7410bb70 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 -- 172.21.15.94:0/1778475745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af77c410 msgr2=0x55c7afb1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af77c410 0x55c7afb1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 -- 172.21.15.94:0/1778475745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 msgr2=0x55c7afb24920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb24920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 -- 172.21.15.94:0/1778475745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7af85c7e0 con 0x55c7af753550 2024-04-25T13:46:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.463+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7a64000c00 tx=0x7f7a64009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06ef248a60fd436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.464+0000 7f7a6affd640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 0x7f7a600b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.464+0000 7f7a6affd640 1 -- 172.21.15.94:0/3613822552 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7a6c0de3d0 con 0x7f7a7410bb70 2024-04-25T13:46:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.465+0000 7f7a5ffff640 1 -- 172.21.15.94:0/1778475745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6403f070 con 0x55c7af753550 2024-04-25T13:46:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.465+0000 7f7a5ffff640 1 -- 172.21.15.94:0/1778475745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64004490 con 0x55c7af753550 2024-04-25T13:46:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.465+0000 7f7a5ffff640 1 -- 172.21.15.94:0/1778475745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64035030 con 0x55c7af753550 2024-04-25T13:46:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.467+0000 7f7a82c73740 1 -- 172.21.15.94:0/1778475745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 msgr2=0x55c7afb1da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.467+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7a64000c00 tx=0x7f7a64009460 comp rx=0 tx=0).stop 2024-04-25T13:46:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.467+0000 7f7a79362640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 0x7f7a600b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.469+0000 7f7a79362640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 0x7f7a600b2210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006aa0 tx=0x7f7a70006a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.470+0000 7f7a82c73740 1 -- 172.21.15.94:0/1778475745 shutdown_connections 2024-04-25T13:46:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.470+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb24920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.470+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af753550 0x55c7afb1da20 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.470+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1778475745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af77c410 0x55c7afb1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.470+0000 7f7a82c73740 1 -- 172.21.15.94:0/1778475745 >> 172.21.15.94:0/1778475745 conn(0x55c7af770560 msgr2=0x55c7afb18e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.473+0000 7f7a82c73740 1 -- 172.21.15.94:0/1778475745 shutdown_connections 2024-04-25T13:46:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.473+0000 7f7a82c73740 1 -- 172.21.15.94:0/1778475745 wait complete. 2024-04-25T13:46:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.473+0000 7f7a82c73740 1 -- start start 2024-04-25T13:46:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.474+0000 7f7a82c73740 1 Processor -- start 2024-04-25T13:46:13.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.474+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 0x55c7afb27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a79b63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a79362640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 0x55c7afb27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a79362640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 0x55c7afb27820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40012/0 (socket says 172.21.15.94:40012) 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a79b63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb272e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47922/0 (socket says 172.21.15.94:47922) 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a79362640 1 -- 172.21.15.94:0/3781124995 learned_addr learned my addr 172.21.15.94:0/3781124995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 0x55c7afb2e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.475+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c7afb25310 con 0x55c7af75b730 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.476+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7afa1c5d0 con 0x55c7af77c410 2024-04-25T13:46:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.476+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 0x55c7afb2e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.476+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7afb25490 con 0x55c7af753550 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.476+0000 7f7a82c73740 10 client.0 _ll_register_callbacks cb 0x55c7afa0a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 -- 172.21.15.94:0/3781124995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 msgr2=0x55c7afb272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 -- 172.21.15.94:0/3781124995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 msgr2=0x55c7afb27820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 0x55c7afb27820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7af8bcf70 con 0x55c7af77c410 2024-04-25T13:46:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a7a364640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 0x55c7afb2e780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0dbdd0 tx=0x7f7a6c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f491c5354a112ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.477+0000 7f7a5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:46:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.479+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c0c0060 con 0x55c7af77c410 2024-04-25T13:46:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.479+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7a6c0c90b0 con 0x55c7af77c410 2024-04-25T13:46:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.479+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a64004490 con 0x55c7af77c410 2024-04-25T13:46:13.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.481+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a6c0aab60 con 0x55c7af77c410 2024-04-25T13:46:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.481+0000 7f7a82c73740 10 client.26642 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.481+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7af754170 con 0x55c7af77c410 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c0af610 con 0x55c7af77c410 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7a6c0de030 con 0x55c7af77c410 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 10 client.26642 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 20 client.26642 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af73b1f0 0x55c7afb34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 4 client.26642 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.483+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7a64004490 con 0x55c7af73b1f0 2024-04-25T13:46:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.487+0000 7f7a79b63640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af73b1f0 0x55c7afb34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.487+0000 7f7a79b63640 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af73b1f0 0x55c7afb34c70 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fee4d2c6c61dfd5 server_cookie=672e058d602673d6 in_seq=0 out_seq=0 2024-04-25T13:46:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.488+0000 7f7a5d7fa640 10 client.26642 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd31eff5640 1 -- 172.21.15.94:0/1153041610 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593fa579ee0 con 0x5593fa26c1b0 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd31eff5640 10 client.26636 handle_command_reply: tid=0 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 client.26636 shutdown 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 msgr2=0x5593fa61b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd330004ce0 tx=0x7fd33005e450 comp rx=0 tx=0).stop 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 shutdown_connections 2024-04-25T13:46:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5593fa26c1b0 0x5593fa68f210 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593fa24b4d0 0x5593fa61c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593fa2432f0 0x5593fa61b8f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 --2- 172.21.15.94:0/1153041610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593fa22ad80 0x5593fa61b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 >> 172.21.15.94:0/1153041610 conn(0x5593fa260300 msgr2=0x5593fa60b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd31eff5640 0 client.26636 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.527+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 shutdown_connections 2024-04-25T13:46:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd348437740 1 -- 172.21.15.94:0/1153041610 wait complete. 2024-04-25T13:46:13.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/3781124995 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f7a64004490 con 0x55c7af73b1f0 2024-04-25T13:46:13.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a5d7fa640 10 client.26642 handle_command_reply: tid=0 2024-04-25T13:46:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd348437740 20 client.26636 trim_cache size 0 max 16384 2024-04-25T13:46:13.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 client.26642 shutdown 2024-04-25T13:46:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:46:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:46:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:46:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 msgr2=0x55c7afb2e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 0x55c7afb2e780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0dbdd0 tx=0x7f7a6c009f90 comp rx=0 tx=0).stop 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 shutdown_connections 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af73b1f0 0x55c7afb34c70 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af77c410 0x55c7afb2e780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a5d7fa640 0 client.26642 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af75b730 0x55c7afb27820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 --2- 172.21.15.94:0/3781124995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 >> 172.21.15.94:0/3781124995 conn(0x55c7af770560 msgr2=0x55c7afb17ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 msgr2=0x7fd3240b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 0x7fd3240b2070 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5593fa60db40 tx=0x7fd330005a20 comp rx=0 tx=0).stop 2024-04-25T13:46:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 msgr2=0x7fd33c102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.528+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c102640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd334005fd0 tx=0x7fd33400ee50 comp rx=0 tx=0).stop 2024-04-25T13:46:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 shutdown_connections 2024-04-25T13:46:13.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.528+0000 7f7a82c73740 1 -- 172.21.15.94:0/3781124995 wait complete. 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 shutdown_connections 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3240afb60 0x7fd3240b2070 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33c1098d0 0x7fd33c102640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106f70 0x7fd33c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 --2- 172.21.15.94:0/3683243837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33c104610 0x7fd33c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 >> 172.21.15.94:0/3683243837 conn(0x7fd33c0fdc10 msgr2=0x7fd33c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.529+0000 7f7a82c73740 20 client.26642 trim_cache size 0 max 16384 2024-04-25T13:46:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 shutdown_connections 2024-04-25T13:46:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:13.529+0000 7fd341582640 1 -- 172.21.15.94:0/3683243837 wait complete. 2024-04-25T13:46:13.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.549+0000 7f7a82c73740 1 Processor -- start 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.549+0000 7f7a82c73740 1 -- start start 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c2bb0 con 0x55c7af77c410 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c46f0 con 0x55c7af75b730 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a82c73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c3280 con 0x55c7af753550 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a79b63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40020/0 (socket says 172.21.15.94:40020) 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 -- 172.21.15.94:0/1803569369 learned_addr learned my addr 172.21.15.94:0/1803569369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a79362640 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 -- 172.21.15.94:0/1803569369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 msgr2=0x55c7afb39460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb39460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 -- 172.21.15.94:0/1803569369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 msgr2=0x55c7afb2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 -- 172.21.15.94:0/1803569369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7afb24b50 con 0x55c7af77c410 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a7a364640 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0a6560 tx=0x7f7a6c0a6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3cdac34d8bb2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.550+0000 7f7a5f7fe640 1 -- 172.21.15.94:0/1803569369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c0a2720 con 0x55c7af77c410 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a5f7fe640 1 -- 172.21.15.94:0/1803569369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c0c90e0 con 0x55c7af77c410 2024-04-25T13:46:13.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a5f7fe640 1 -- 172.21.15.94:0/1803569369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c0af180 con 0x55c7af77c410 2024-04-25T13:46:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 -- 172.21.15.94:0/1803569369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 msgr2=0x55c7afb2ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0a6560 tx=0x7f7a6c0a6720 comp rx=0 tx=0).stop 2024-04-25T13:46:13.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 -- 172.21.15.94:0/1803569369 shutdown_connections 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2ebe0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af75b730 0x55c7afb2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1803569369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af753550 0x55c7afb39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 -- 172.21.15.94:0/1803569369 >> 172.21.15.94:0/1803569369 conn(0x55c7af770560 msgr2=0x55c7afb25920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 -- 172.21.15.94:0/1803569369 shutdown_connections 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.551+0000 7f7a82c73740 1 -- 172.21.15.94:0/1803569369 wait complete. 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 -- start start 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 Processor -- start 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 0x55c7afb2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af75b730 0x55c7afb2cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c4020 con 0x55c7af77c410 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c3950 con 0x55c7af753550 2024-04-25T13:46:13.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741c0b90 con 0x55c7af75b730 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a82c73740 10 client.0 _ll_register_callbacks cb 0x7f7a6c0a61b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a7a364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a7a364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40036/0 (socket says 172.21.15.94:40036) 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a7a364640 1 -- 172.21.15.94:0/1402579829 learned_addr learned my addr 172.21.15.94:0/1402579829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 0x55c7afb2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.552+0000 7f7a79362640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af75b730 0x55c7afb2cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 -- 172.21.15.94:0/1402579829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af75b730 msgr2=0x55c7afb2cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af75b730 0x55c7afb2cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 -- 172.21.15.94:0/1402579829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 msgr2=0x55c7afb2da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 -- 172.21.15.94:0/1402579829 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7afb2dfe0 con 0x55c7af753550 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a7a364640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a79b63640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 0x55c7afb2c620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009900 tx=0x7f7a64009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6220e14a30cbaa4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6403f040 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6c0c90e0 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f7a64050830 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a6405d050 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a82c73740 10 client.26648 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.553+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7afb9d020 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c0c90e0 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7a6404d070 con 0x55c7af753550 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 10 client.26648 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 20 client.26648 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af710630 0x55c7afb9f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 4 client.26648 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:46:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7afa90710 con 0x55c7af710630 2024-04-25T13:46:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a79362640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af710630 0x55c7afb9f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a5d7fa640 10 client.26648 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.554+0000 7f7a79362640 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af710630 0x55c7afb9f490 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30df25d4d0dda400 server_cookie=8fef04dd6a971fb6 in_seq=0 out_seq=0 2024-04-25T13:46:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a5d7fa640 1 -- 172.21.15.94:0/1402579829 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7afa90710 con 0x55c7af710630 2024-04-25T13:46:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a5d7fa640 10 client.26648 handle_command_reply: tid=0 2024-04-25T13:46:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 client.26648 shutdown 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 msgr2=0x55c7afb2c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 0x55c7afb2c620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009900 tx=0x7f7a64009250 comp rx=0 tx=0).stop 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 shutdown_connections 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c7af710630 0x55c7afb9f490 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7af77c410 0x55c7afb2da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7af75b730 0x55c7afb2cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a5d7fa640 0 client.26648 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.557+0000 7f7a82c73740 1 --2- 172.21.15.94:0/1402579829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7af753550 0x55c7afb2c620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 >> 172.21.15.94:0/1402579829 conn(0x55c7af770560 msgr2=0x55c7afb1bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 shutdown_connections 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a82c73740 1 -- 172.21.15.94:0/1402579829 wait complete. 2024-04-25T13:46:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a82c73740 20 client.26648 trim_cache size 0 max 16384 2024-04-25T13:46:13.563 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:13.563 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:46:13.563 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "392b484b-2250-44a4-9573-ead426fc9010", 2024-04-25T13:46:13.563 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:46:13.563 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 msgr2=0x7f7a600b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 0x7f7a600b2210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006aa0 tx=0x7f7a70006a30 comp rx=0 tx=0).stop 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 msgr2=0x7f7a741b9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 0x7f7a741b9040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00be20 tx=0x7f7a6c00bf10 comp rx=0 tx=0).stop 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 shutdown_connections 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a600afd00 0x7f7a600b2210 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a7410bb70 0x7f7a741b9040 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a7410b220 0x7f7a741a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 --2- 172.21.15.94:0/3613822552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7410a100 0x7f7a741a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 >> 172.21.15.94:0/3613822552 conn(0x7f7a74070940 msgr2=0x7f7a7410d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 shutdown_connections 2024-04-25T13:46:13.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.558+0000 7f7a7bdc7640 1 -- 172.21.15.94:0/3613822552 wait complete. 2024-04-25T13:46:13.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:46:13.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:46:13.632 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:13 smithi094 ceph-mon[29195]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 371 op/s 2024-04-25T13:46:13.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 -- 172.21.15.94:0/2781615232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 msgr2=0x7f764c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 --2- 172.21.15.94:0/2781615232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c1047f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f763c009900 tx=0x7f763c02a410 comp rx=0 tx=0).stop 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 -- 172.21.15.94:0/2781615232 shutdown_connections 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 --2- 172.21.15.94:0/2781615232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109a30 0x7f764c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 --2- 172.21.15.94:0/2781615232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c104d30 0x7f764c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 --2- 172.21.15.94:0/2781615232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c1047f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 -- 172.21.15.94:0/2781615232 >> 172.21.15.94:0/2781615232 conn(0x7f764c0fece0 msgr2=0x7f764c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 -- 172.21.15.94:0/2781615232 shutdown_connections 2024-04-25T13:46:13.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.727+0000 7f7650caf640 1 -- 172.21.15.94:0/2781615232 wait complete. 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 Processor -- start 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 -- start start 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c104d30 0x7f764c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c109a30 0x7f764c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10f5f0 con 0x7f764c103af0 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10f470 con 0x7f764c104d30 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7650caf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10f770 con 0x7f764c109a30 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f764a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f764a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40062/0 (socket says 172.21.15.94:40062) 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f764a59c640 1 -- 172.21.15.94:0/1139687258 learned_addr learned my addr 172.21.15.94:0/1139687258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f7643fff640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c104d30 0x7f764c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.728+0000 7f764ad9d640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c109a30 0x7f764c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 -- 172.21.15.94:0/1139687258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c109a30 msgr2=0x7f764c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c109a30 0x7f764c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 -- 172.21.15.94:0/1139687258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c104d30 msgr2=0x7f764c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c104d30 0x7f764c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 -- 172.21.15.94:0/1139687258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1a1da0 con 0x7f764c103af0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f764a59c640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f763c006010 tx=0x7f763c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f76437fe640 1 -- 172.21.15.94:0/1139687258 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c03f030 con 0x7f764c103af0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1a2030 con 0x7f764c103af0 2024-04-25T13:46:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f76437fe640 1 -- 172.21.15.94:0/1139687258 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c031910 con 0x7f764c103af0 2024-04-25T13:46:13.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c1a2570 con 0x7f764c103af0 2024-04-25T13:46:13.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.729+0000 7f76437fe640 1 -- 172.21.15.94:0/1139687258 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035070 con 0x7f764c103af0 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 Processor -- start 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 -- start start 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f76437fe640 1 -- 172.21.15.94:0/1139687258 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f763c042030 con 0x7f764c103af0 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc4893fd80 con 0x55dc48639390 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc488f98b0 con 0x55dc486186b0 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7657b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc4893bd40 con 0x55dc486104d0 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40070/0 (socket says 172.21.15.94:40070) 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f764ad9d640 1 -- 172.21.15.94:0/4150592170 learned_addr learned my addr 172.21.15.94:0/4150592170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.732+0000 7f7643fff640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764a59c640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 -- 172.21.15.94:0/4150592170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 msgr2=0x55dc489dafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489dafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 -- 172.21.15.94:0/4150592170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 msgr2=0x55dc489dbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489dbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 -- 172.21.15.94:0/4150592170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc488caa40 con 0x55dc48639390 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f7643fff640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489dbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f76437fe640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 0x7f76380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7644011440 tx=0x7f7644011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6cdb3397a5c573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f764a59c640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 0x7f76380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f76437fe640 1 -- 172.21.15.94:0/1139687258 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f763c0fefc0 con 0x7f764c103af0 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f7640ff9640 1 -- 172.21.15.94:0/4150592170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644003c30 con 0x55dc48639390 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f7640ff9640 1 -- 172.21.15.94:0/4150592170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644003dd0 con 0x55dc48639390 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.733+0000 7f7640ff9640 1 -- 172.21.15.94:0/4150592170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400fa20 con 0x55dc48639390 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 -- 172.21.15.94:0/4150592170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 msgr2=0x55dc489daa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7644011440 tx=0x7f7644011900 comp rx=0 tx=0).stop 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f764a59c640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 0x7f76380b2000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55dc489da780 tx=0x7f763c0b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 -- 172.21.15.94:0/4150592170 shutdown_connections 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489dbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489dafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 --2- 172.21.15.94:0/4150592170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc48639390 0x55dc489daa60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 -- 172.21.15.94:0/4150592170 >> 172.21.15.94:0/4150592170 conn(0x55dc4862d4e0 msgr2=0x55dc489d6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 -- 172.21.15.94:0/4150592170 shutdown_connections 2024-04-25T13:46:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.734+0000 7f7657b67740 1 -- 172.21.15.94:0/4150592170 wait complete. 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 -- start start 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 Processor -- start 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 0x55dc489ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc489dc780 con 0x55dc486186b0 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc488c6d10 con 0x55dc48639390 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc489dc900 con 0x55dc486104d0 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7657b67740 10 client.0 _ll_register_callbacks cb 0x55dc488f3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40076/0 (socket says 172.21.15.94:40076) 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 -- 172.21.15.94:0/4011906515 learned_addr learned my addr 172.21.15.94:0/4011906515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 0x55dc489ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 -- 172.21.15.94:0/4011906515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 msgr2=0x55dc489e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 -- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 msgr2=0x55dc489ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 0x55dc489ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f7643fff640 1 -- 172.21.15.94:0/4011906515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4871b7d0 con 0x55dc486186b0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.735+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 0x55dc489ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7643fff640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76340126e0 tx=0x7f7634012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f04cff18dcd63d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004050 con 0x55dc486186b0 2024-04-25T13:46:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f76340041f0 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7644003dd0 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f76340043f0 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc486513f0 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644003dd0 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7634003b40 con 0x55dc486186b0 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 20 client.17187 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc485f7f60 0x55dc489f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 4 client.17187 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.736+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4894d990 con 0x55dc485f7f60 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.737+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc485f7f60 0x55dc489f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.737+0000 7f7631ffb640 10 client.17187 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.737+0000 7f764ad9d640 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc485f7f60 0x55dc489f22b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e919fe192e8dff77 server_cookie=a27b1c73772807be in_seq=0 out_seq=0 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7631ffb640 1 -- 172.21.15.94:0/4011906515 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc4894d990 con 0x55dc485f7f60 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7631ffb640 10 client.17187 handle_command_reply: tid=0 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 client.17187 shutdown 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 msgr2=0x55dc489e4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76340126e0 tx=0x7f7634012ba0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 shutdown_connections 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc485f7f60 0x55dc489f22b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc48639390 0x55dc489ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7631ffb640 0 client.17187 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486186b0 0x55dc489e4f10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 --2- 172.21.15.94:0/4011906515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 >> 172.21.15.94:0/4011906515 conn(0x55dc4862d4e0 msgr2=0x55dc489d9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 shutdown_connections 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 1 -- 172.21.15.94:0/4011906515 wait complete. 2024-04-25T13:46:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.738+0000 7f7657b67740 20 client.17187 trim_cache size 0 max 16384 2024-04-25T13:46:13.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:13 smithi138 ceph-mon[32977]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 371 op/s 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 Processor -- start 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 -- start start 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc485f7f60 0x55dc489e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ab5c0 con 0x55dc486104d0 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ad100 con 0x55dc486186b0 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7657b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1abc90 con 0x55dc485f7f60 2024-04-25T13:46:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40092/0 (socket says 172.21.15.94:40092) 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 -- 172.21.15.94:0/3374275272 learned_addr learned my addr 172.21.15.94:0/3374275272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f764ad9d640 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 -- 172.21.15.94:0/3374275272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc485f7f60 msgr2=0x55dc489e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc485f7f60 0x55dc489e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 -- 172.21.15.94:0/3374275272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 msgr2=0x55dc489e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 -- 172.21.15.94:0/3374275272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc486110f0 con 0x55dc486104d0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f764ad9d640 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7643fff640 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7634026590 tx=0x7f7634026820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d188841d6f55f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.757+0000 7f7633fff640 1 -- 172.21.15.94:0/3374275272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634003f90 con 0x55dc486104d0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7633fff640 1 -- 172.21.15.94:0/3374275272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634026e40 con 0x55dc486104d0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7633fff640 1 -- 172.21.15.94:0/3374275272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634005040 con 0x55dc486104d0 2024-04-25T13:46:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 -- 172.21.15.94:0/3374275272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 msgr2=0x55dc489ebd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7634026590 tx=0x7f7634026820 comp rx=0 tx=0).stop 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 -- 172.21.15.94:0/3374275272 shutdown_connections 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc486104d0 0x55dc489ebd20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 --2- 172.21.15.94:0/3374275272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc485f7f60 0x55dc489e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 -- 172.21.15.94:0/3374275272 >> 172.21.15.94:0/3374275272 conn(0x55dc4862d4e0 msgr2=0x55dc489df760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.758+0000 7f7657b67740 1 -- 172.21.15.94:0/3374275272 shutdown_connections 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 -- 172.21.15.94:0/3374275272 wait complete. 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 -- start start 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 Processor -- start 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1aca30 con 0x55dc485f7f60 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ac360 con 0x55dc486186b0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1a95a0 con 0x55dc486104d0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7657b67740 10 client.0 _ll_register_callbacks cb 0x7f7634009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40098/0 (socket says 172.21.15.94:40098) 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 -- 172.21.15.94:0/1589596498 learned_addr learned my addr 172.21.15.94:0/1589596498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f7643fff640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 -- 172.21.15.94:0/1589596498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 msgr2=0x55dc489e87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 -- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 msgr2=0x55dc489e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.759+0000 7f764ad9d640 1 -- 172.21.15.94:0/1589596498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc489004b0 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f764ad9d640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7644011940 tx=0x7f7644000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc75ac07b10fa769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400fb80 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f764400fd20 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc48947800 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7644004050 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 10 client.17193 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc48a59700 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dc48947800 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7644025040 con 0x55dc485f7f60 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 20 client.17193 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc48639390 0x55dc48a5bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 4 client.17193 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.760+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc488c24a0 con 0x55dc48639390 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.761+0000 7f7643fff640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc48639390 0x55dc48a5bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.761+0000 7f7631ffb640 10 client.17193 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.761+0000 7f7643fff640 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc48639390 0x55dc48a5bee0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9002267ebe155e server_cookie=cb4de45aa061b2b0 in_seq=0 out_seq=0 2024-04-25T13:46:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7631ffb640 1 -- 172.21.15.94:0/1589596498 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc488c24a0 con 0x55dc48639390 2024-04-25T13:46:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7631ffb640 10 client.17193 handle_command_reply: tid=0 2024-04-25T13:46:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 client.17193 shutdown 2024-04-25T13:46:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 msgr2=0x55dc489ebc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7644011940 tx=0x7f7644000f40 comp rx=0 tx=0).stop 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 shutdown_connections 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc48639390 0x55dc48a5bee0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc486186b0 0x55dc489e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7631ffb640 0 client.17193 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc486104d0 0x55dc489e87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 --2- 172.21.15.94:0/1589596498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc485f7f60 0x55dc489ebc70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 >> 172.21.15.94:0/1589596498 conn(0x55dc4862d4e0 msgr2=0x55dc489e2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 shutdown_connections 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 1 -- 172.21.15.94:0/1589596498 wait complete. 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.765+0000 7f7657b67740 20 client.17193 trim_cache size 0 max 16384 2024-04-25T13:46:13.769 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "392b484b-2250-44a4-9573-ead426fc9010": { 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "392b484b-2250-44a4-9573-ead426fc9010", 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:46:13.770 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 msgr2=0x7f76380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 0x7f76380b2000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55dc489da780 tx=0x7f763c0b0040 comp rx=0 tx=0).stop 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 msgr2=0x7f764c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f763c006010 tx=0x7f763c004270 comp rx=0 tx=0).stop 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 shutdown_connections 2024-04-25T13:46:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76380afaf0 0x7f76380b2000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c109a30 0x7f764c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c104d30 0x7f764c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 --2- 172.21.15.94:0/1139687258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f764c103af0 0x7f764c19a260 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 >> 172.21.15.94:0/1139687258 conn(0x7f764c0fece0 msgr2=0x7f764c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 shutdown_connections 2024-04-25T13:46:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:13.766+0000 7f7650caf640 1 -- 172.21.15.94:0/1139687258 wait complete. 2024-04-25T13:46:13.777 INFO:tasks.cephfs.filesystem:scrub status for tag:392b484b-2250-44a4-9573-ead426fc9010 - {'path': '/', 'tag': '392b484b-2250-44a4-9573-ead426fc9010', 'options': 'recursive,force'} 2024-04-25T13:46:13.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 444115 12.99 MB/sec execute 56 sec latency 68.086 ms 2024-04-25T13:46:13.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:13 smithi137 ceph-mon[32568]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 371 op/s 2024-04-25T13:46:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:14 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:46:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:14 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:14 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:46:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:14 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:14.795 INFO:tasks.workunit.client.0.smithi094.stdout: 1 447158 13.08 MB/sec execute 57 sec latency 23.234 ms 2024-04-25T13:46:14.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:14 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:46:14.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:14 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:14.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:14 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:46:14.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:14 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:14.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:14 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:46:14.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:14 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:14.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:14 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:46:14.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:14 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:15 smithi138 ceph-mon[32977]: pgmap v480: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-04-25T13:46:15.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 449435 13.00 MB/sec execute 58 sec latency 18.522 ms 2024-04-25T13:46:15.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:15 smithi137 ceph-mon[32568]: pgmap v480: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-04-25T13:46:15.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:15 smithi094 ceph-mon[29195]: pgmap v480: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-04-25T13:46:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:46:16.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 451475 12.98 MB/sec execute 59 sec latency 15.058 ms 2024-04-25T13:46:16.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:46:16.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:46:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:17 smithi138 ceph-mon[32977]: pgmap v481: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:46:17.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 454156 13.05 MB/sec execute 60 sec latency 65.441 ms 2024-04-25T13:46:17.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:17 smithi137 ceph-mon[32568]: pgmap v481: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:46:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:17 smithi094 ceph-mon[29195]: pgmap v481: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:46:18.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 456385 12.97 MB/sec execute 61 sec latency 50.392 ms 2024-04-25T13:46:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:19 smithi138 ceph-mon[32977]: pgmap v482: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-04-25T13:46:19.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 458344 12.96 MB/sec execute 62 sec latency 49.708 ms 2024-04-25T13:46:19.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:19 smithi137 ceph-mon[32568]: pgmap v482: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-04-25T13:46:19.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:19 smithi094 ceph-mon[29195]: pgmap v482: 97 pgs: 97 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-04-25T13:46:20.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 460843 13.21 MB/sec execute 63 sec latency 74.877 ms 2024-04-25T13:46:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:21 smithi138 ceph-mon[32977]: pgmap v483: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:46:21.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 463110 13.22 MB/sec execute 64 sec latency 18.108 ms 2024-04-25T13:46:21.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:21 smithi137 ceph-mon[32568]: pgmap v483: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:46:21.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:21 smithi094 ceph-mon[29195]: pgmap v483: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:46:22.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 465709 13.27 MB/sec execute 65 sec latency 65.882 ms 2024-04-25T13:46:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:23 smithi138 ceph-mon[32977]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-04-25T13:46:23.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 468418 13.26 MB/sec execute 66 sec latency 15.536 ms 2024-04-25T13:46:23.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:23 smithi137 ceph-mon[32568]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-04-25T13:46:23.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:23 smithi094 ceph-mon[29195]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-04-25T13:46:24.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 470768 13.24 MB/sec execute 67 sec latency 14.865 ms 2024-04-25T13:46:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:25 smithi138 ceph-mon[32977]: pgmap v485: 97 pgs: 97 active+clean; 32 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:46:25.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 473483 13.26 MB/sec execute 68 sec latency 12.061 ms 2024-04-25T13:46:25.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:25 smithi137 ceph-mon[32568]: pgmap v485: 97 pgs: 97 active+clean; 32 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:46:25.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:25 smithi094 ceph-mon[29195]: pgmap v485: 97 pgs: 97 active+clean; 32 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:46:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 -- 172.21.15.94:0/2252974716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 msgr2=0x7f8ef0109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/2252974716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef0109370 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009900 tx=0x7f8edc02a410 comp rx=0 tx=0).stop 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 -- 172.21.15.94:0/2252974716 shutdown_connections 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/2252974716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01098b0 0x7f8ef010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/2252974716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef0109370 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/2252974716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01045f0 0x7f8ef0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 -- 172.21.15.94:0/2252974716 >> 172.21.15.94:0/2252974716 conn(0x7f8ef00fdbd0 msgr2=0x7f8ef01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 -- 172.21.15.94:0/2252974716 shutdown_connections 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.630+0000 7f8ef6eca640 1 -- 172.21.15.94:0/2252974716 wait complete. 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 Processor -- start 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 -- start start 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 0x7f8ef019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01098b0 0x7f8ef01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010f610 con 0x7f8ef0106f50 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010f490 con 0x7f8ef01098b0 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef6eca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010f790 con 0x7f8ef01045f0 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49152/0 (socket says 172.21.15.94:49152) 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 -- 172.21.15.94:0/3965035757 learned_addr learned my addr 172.21.15.94:0/3965035757 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef5467640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01098b0 0x7f8ef01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 0x7f8ef019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 -- 172.21.15.94:0/3965035757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 msgr2=0x7f8ef019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 0x7f8ef019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 -- 172.21.15.94:0/3965035757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01098b0 msgr2=0x7f8ef01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01098b0 0x7f8ef01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 -- 172.21.15.94:0/3965035757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a1db0 con 0x7f8ef0106f50 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 0x7f8ef019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.631+0000 7f8eeffff640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8edc006010 tx=0x7f8edc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.632+0000 7f8eedffb640 1 -- 172.21.15.94:0/3965035757 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc03f030 con 0x7f8ef0106f50 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.632+0000 7f8eedffb640 1 -- 172.21.15.94:0/3965035757 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc003aa0 con 0x7f8ef0106f50 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.632+0000 7f8eedffb640 1 -- 172.21.15.94:0/3965035757 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc035070 con 0x7f8ef0106f50 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.632+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a2040 con 0x7f8ef0106f50 2024-04-25T13:46:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.632+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a2580 con 0x7f8ef0106f50 2024-04-25T13:46:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.634+0000 7f8eedffb640 1 -- 172.21.15.94:0/3965035757 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8edc042030 con 0x7f8ef0106f50 2024-04-25T13:46:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 Processor -- start 2024-04-25T13:46:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 -- start start 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed64d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed654df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed5b3460 con 0x55f8ed2842f0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed56d590 con 0x55f8ed28c4d0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed5af420 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49160/0 (socket says 172.21.15.94:49160) 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 -- 172.21.15.94:0/1360673811 learned_addr learned my addr 172.21.15.94:0/1360673811 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8eeffff640 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed654df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef5467640 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed64d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8eedffb640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 0x7f8ee00b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 -- 172.21.15.94:0/1360673811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 msgr2=0x55f8ed64d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed64d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 -- 172.21.15.94:0/1360673811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 msgr2=0x55f8ed654df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed654df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef4c66640 1 -- 172.21.15.94:0/1360673811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ed57a0d0 con 0x55f8ed2842f0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.635+0000 7f8ef5467640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 0x7f8ee00b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8eedffb640 1 -- 172.21.15.94:0/3965035757 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f8edc0ff230 con 0x7f8ef0106f50 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009cf0 tx=0x7f8ee40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907ac530968a118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8edaffd640 1 -- 172.21.15.94:0/1360673811 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4012450 con 0x55f8ed2842f0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8ef5467640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 0x7f8ee00b2070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55f8ed64eb40 tx=0x7f8ee8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8edaffd640 1 -- 172.21.15.94:0/1360673811 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee4012aa0 con 0x55f8ed2842f0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.636+0000 7f8edaffd640 1 -- 172.21.15.94:0/1360673811 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400fb20 con 0x55f8ed2842f0 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- 172.21.15.94:0/1360673811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 msgr2=0x55f8ed64def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009cf0 tx=0x7f8ee40117b0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- 172.21.15.94:0/1360673811 shutdown_connections 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed654df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed64def0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/1360673811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed64d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- 172.21.15.94:0/1360673811 >> 172.21.15.94:0/1360673811 conn(0x55f8ed2a1300 msgr2=0x55f8ed649300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- 172.21.15.94:0/1360673811 shutdown_connections 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- 172.21.15.94:0/1360673811 wait complete. 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 -- start start 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.637+0000 7f8efdd7a740 1 Processor -- start 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed2842f0 0x55f8ed657680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed655840 con 0x55f8ed28c4d0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed53ab60 con 0x55f8ed2842f0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8ed6559c0 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8efdd7a740 10 client.0 _ll_register_callbacks cb 0x55f8ed55f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49172/0 (socket says 172.21.15.94:49172) 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 -- 172.21.15.94:0/2257735894 learned_addr learned my addr 172.21.15.94:0/2257735894 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed2842f0 0x55f8ed657680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 -- 172.21.15.94:0/2257735894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 msgr2=0x55f8ed65eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 -- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed2842f0 msgr2=0x55f8ed657680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed2842f0 0x55f8ed657680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 -- 172.21.15.94:0/2257735894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ed2c5210 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0c48e0 tx=0x7f8edc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea14cec44ff5e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.638+0000 7f8ecffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc049070 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f8edc0c3900 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee4012aa0 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8edc0c3530 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 10 client.17205 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8ed391620 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee4012aa0 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8edc0c78d0 con 0x55f8ed28c4d0 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 20 client.17205 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed26bd80 0x55f8ed665350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 4 client.17205 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ed5c1070 con 0x55f8ed26bd80 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.639+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed26bd80 0x55f8ed665350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.640+0000 7f8ecffff640 10 client.17205 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.640+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed26bd80 0x55f8ed665350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c850d6c347232f server_cookie=14596621a11b00b5 in_seq=0 out_seq=0 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.640+0000 7f8ecffff640 1 -- 172.21.15.94:0/2257735894 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8ed5c1070 con 0x55f8ed26bd80 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.640+0000 7f8ecffff640 10 client.17205 handle_command_reply: tid=0 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.640+0000 7f8efdd7a740 1 client.17205 shutdown 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 msgr2=0x55f8ed657bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0c48e0 tx=0x7f8edc003e80 comp rx=0 tx=0).stop 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 shutdown_connections 2024-04-25T13:46:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed26bd80 0x55f8ed665350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8ecffff640 0 client.17205 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed28c4d0 0x55f8ed657bc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2257735894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed2842f0 0x55f8ed657680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 >> 172.21.15.94:0/2257735894 conn(0x55f8ed2a1300 msgr2=0x55f8ed64c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 shutdown_connections 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2257735894 wait complete. 2024-04-25T13:46:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.641+0000 7f8efdd7a740 20 client.17205 trim_cache size 0 max 16384 2024-04-25T13:46:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.656+0000 7f8efdd7a740 1 Processor -- start 2024-04-25T13:46:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.656+0000 7f8efdd7a740 1 -- start start 2024-04-25T13:46:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 0x55f8ed648c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ab540 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ad080 con 0x55f8ed28c4d0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01abc10 con 0x55f8ed2842f0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49188/0 (socket says 172.21.15.94:49188) 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 -- 172.21.15.94:0/2119866264 learned_addr learned my addr 172.21.15.94:0/2119866264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 0x55f8ed648c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 -- 172.21.15.94:0/2119866264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 msgr2=0x55f8ed648c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 0x55f8ed648c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 -- 172.21.15.94:0/2119866264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 msgr2=0x55f8ed65e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 -- 172.21.15.94:0/2119866264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ed658720 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 0x55f8ed648c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8010670 tx=0x7f8ee8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb570c30b2696685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8eda7fc640 1 -- 172.21.15.94:0/2119866264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8011440 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8eda7fc640 1 -- 172.21.15.94:0/2119866264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8011a90 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.657+0000 7f8eda7fc640 1 -- 172.21.15.94:0/2119866264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8021c70 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2119866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 msgr2=0x55f8ed649150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8010670 tx=0x7f8ee8016000 comp rx=0 tx=0).stop 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2119866264 shutdown_connections 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed649150 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2119866264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2842f0 0x55f8ed648c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2119866264 >> 172.21.15.94:0/2119866264 conn(0x55f8ed2a1300 msgr2=0x55f8ed651ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2119866264 shutdown_connections 2024-04-25T13:46:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.658+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2119866264 wait complete. 2024-04-25T13:46:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 -- start start 2024-04-25T13:46:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 Processor -- start 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ac9b0 con 0x55f8ed2842f0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ac2e0 con 0x55f8ed28c4d0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01a9520 con 0x55f8ed2ad1b0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8efdd7a740 10 client.0 _ll_register_callbacks cb 0x7f8edc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49204/0 (socket says 172.21.15.94:49204) 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 -- 172.21.15.94:0/2332213498 learned_addr learned my addr 172.21.15.94:0/2332213498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 -- 172.21.15.94:0/2332213498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 msgr2=0x55f8ed65db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 -- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 msgr2=0x55f8ed65cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef4c66640 1 -- 172.21.15.94:0/2332213498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ed65e150 con 0x55f8ed2842f0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.659+0000 7f8ef5467640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ef4c66640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x55f8ed64ed50 tx=0x7f8ee4011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6eeb1b7a606ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40052c0 con 0x55f8ed2842f0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f8ee4005460 con 0x55f8ed2842f0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee8011a90 con 0x55f8ed2842f0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ee40035f0 con 0x55f8ed2842f0 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 10 client.17211 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8ed6cdb80 con 0x55f8ed2842f0 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8011a90 con 0x55f8ed2842f0 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8ee402a3d0 con 0x55f8ed2842f0 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 10 client.17211 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 20 client.17211 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed2411c0 0x55f8ed6d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.660+0000 7f8efdd7a740 4 client.17211 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.661+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8ed5baee0 con 0x55f8ed2411c0 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.661+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed2411c0 0x55f8ed6d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.661+0000 7f8ecffff640 10 client.17211 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.661+0000 7f8eeffff640 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed2411c0 0x55f8ed6d0260 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e90f5bf63a95f0b server_cookie=e214aaaf6d549d9f in_seq=0 out_seq=0 2024-04-25T13:46:26.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 476387 13.33 MB/sec execute 69 sec latency 73.186 ms 2024-04-25T13:46:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.828+0000 7f8ecffff640 1 -- 172.21.15.94:0/2332213498 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8ed5baee0 con 0x55f8ed2411c0 2024-04-25T13:46:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.828+0000 7f8ecffff640 10 client.17211 handle_command_reply: tid=0 2024-04-25T13:46:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.828+0000 7f8efdd7a740 1 client.17211 shutdown 2024-04-25T13:46:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 msgr2=0x55f8ed65c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x55f8ed64ed50 tx=0x7f8ee4011df0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 shutdown_connections 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f8ed2411c0 0x55f8ed6d0260 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8ed2ad1b0 0x55f8ed65db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8ed28c4d0 0x55f8ed65cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 --2- 172.21.15.94:0/2332213498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8ed2842f0 0x55f8ed65c720 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 >> 172.21.15.94:0/2332213498 conn(0x55f8ed2a1300 msgr2=0x55f8ed64c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8ecffff640 0 client.17211 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 shutdown_connections 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 1 -- 172.21.15.94:0/2332213498 wait complete. 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.829+0000 7f8efdd7a740 20 client.17211 trim_cache size 0 max 16384 2024-04-25T13:46:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:46:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:46:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:46:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:46:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 msgr2=0x7f8ee00b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 0x7f8ee00b2070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55f8ed64eb40 tx=0x7f8ee8004a40 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 msgr2=0x7f8ef019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8edc006010 tx=0x7f8edc031a50 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 shutdown_connections 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ee00afb60 0x7f8ee00b2070 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef01098b0 0x7f8ef01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0106f50 0x7f8ef019a7b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 --2- 172.21.15.94:0/3965035757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef01045f0 0x7f8ef019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 >> 172.21.15.94:0/3965035757 conn(0x7f8ef00fdbd0 msgr2=0x7f8ef01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 shutdown_connections 2024-04-25T13:46:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:26.830+0000 7f8ef6eca640 1 -- 172.21.15.94:0/3965035757 wait complete. 2024-04-25T13:46:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:27 smithi138 ceph-mon[32977]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 353 op/s 2024-04-25T13:46:27.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 478505 13.25 MB/sec execute 70 sec latency 114.437 ms 2024-04-25T13:46:27.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:27 smithi137 ceph-mon[32568]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 353 op/s 2024-04-25T13:46:27.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:27 smithi094 ceph-mon[29195]: pgmap v486: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 353 op/s 2024-04-25T13:46:28.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 480552 13.24 MB/sec execute 71 sec latency 74.531 ms 2024-04-25T13:46:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:29 smithi138 ceph-mon[32977]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-04-25T13:46:29.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 483614 13.31 MB/sec execute 72 sec latency 8.867 ms 2024-04-25T13:46:29.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:29 smithi137 ceph-mon[32568]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-04-25T13:46:29.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:29 smithi094 ceph-mon[29195]: pgmap v487: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-04-25T13:46:30.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 486735 13.37 MB/sec execute 73 sec latency 18.393 ms 2024-04-25T13:46:31.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 489203 13.31 MB/sec execute 74 sec latency 18.590 ms 2024-04-25T13:46:31.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:31 smithi137 ceph-mon[32568]: pgmap v488: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-04-25T13:46:31.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:31 smithi094 ceph-mon[29195]: pgmap v488: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-04-25T13:46:31.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:31 smithi138 ceph-mon[32977]: pgmap v488: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-04-25T13:46:32.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 491111 13.30 MB/sec execute 75 sec latency 67.162 ms 2024-04-25T13:46:33.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 494227 13.37 MB/sec execute 76 sec latency 11.618 ms 2024-04-25T13:46:33.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:33 smithi137 ceph-mon[32568]: pgmap v489: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-04-25T13:46:33.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:33 smithi094 ceph-mon[29195]: pgmap v489: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-04-25T13:46:33.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:33 smithi138 ceph-mon[32977]: pgmap v489: 97 pgs: 97 active+clean; 20 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-04-25T13:46:34.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 497145 13.38 MB/sec execute 77 sec latency 13.580 ms 2024-04-25T13:46:35.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 499769 13.36 MB/sec execute 78 sec latency 14.210 ms 2024-04-25T13:46:35.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:35 smithi137 ceph-mon[32568]: pgmap v490: 97 pgs: 97 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-04-25T13:46:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:35 smithi094 ceph-mon[29195]: pgmap v490: 97 pgs: 97 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-04-25T13:46:35.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:35 smithi138 ceph-mon[32977]: pgmap v490: 97 pgs: 97 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-04-25T13:46:36.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 502014 13.37 MB/sec execute 79 sec latency 12.517 ms 2024-04-25T13:46:37.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 504737 13.41 MB/sec execute 80 sec latency 87.043 ms 2024-04-25T13:46:37.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:37 smithi137 ceph-mon[32568]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:46:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:37 smithi094 ceph-mon[29195]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:46:37.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:37 smithi138 ceph-mon[32977]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-04-25T13:46:38.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 506926 13.34 MB/sec execute 81 sec latency 49.379 ms 2024-04-25T13:46:39.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 509140 13.35 MB/sec execute 82 sec latency 14.042 ms 2024-04-25T13:46:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:39 smithi137 ceph-mon[32568]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.3 MiB/s wr, 363 op/s 2024-04-25T13:46:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 -- 172.21.15.94:0/2320475857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 msgr2=0x7f4f3c10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 --2- 172.21.15.94:0/2320475857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c10bcd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009ca0 tx=0x7f4f3402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 -- 172.21.15.94:0/2320475857 shutdown_connections 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 --2- 172.21.15.94:0/2320475857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c10bcd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 --2- 172.21.15.94:0/2320475857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 --2- 172.21.15.94:0/2320475857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 -- 172.21.15.94:0/2320475857 >> 172.21.15.94:0/2320475857 conn(0x7f4f3c0fdbd0 msgr2=0x7f4f3c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 -- 172.21.15.94:0/2320475857 shutdown_connections 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.929+0000 7f4f411f3640 1 -- 172.21.15.94:0/2320475857 wait complete. 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.930+0000 7f4f411f3640 1 Processor -- start 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.930+0000 7f4f411f3640 1 -- start start 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10f660 con 0x7f4f3c1098b0 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10f4e0 con 0x7f4f3c106f50 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f411f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10f7e0 con 0x7f4f3c1045f0 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41656/0 (socket says 172.21.15.94:41656) 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/1769563630 learned_addr learned my addr 172.21.15.94:0/1769563630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/1769563630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 msgr2=0x7f4f3c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 msgr2=0x7f4f3c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/1769563630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c1a1d00 con 0x7f4f3c1045f0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000e9a0 tx=0x7f4f3000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.931+0000 7f4f23fff640 1 -- 172.21.15.94:0/1769563630 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000cd80 con 0x7f4f3c1045f0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.932+0000 7f4f23fff640 1 -- 172.21.15.94:0/1769563630 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f300035f0 con 0x7f4f3c1045f0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.932+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c1a1ff0 con 0x7f4f3c1045f0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.932+0000 7f4f23fff640 1 -- 172.21.15.94:0/1769563630 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30010690 con 0x7f4f3c1045f0 2024-04-25T13:46:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.932+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c1a2450 con 0x7f4f3c1045f0 2024-04-25T13:46:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.934+0000 7f4f4817b740 1 Processor -- start 2024-04-25T13:46:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.934+0000 7f4f4817b740 1 -- start start 2024-04-25T13:46:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e39aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 0x55e58e3a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e58e300460 con 0x55e58dfd12f0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e58e2ba590 con 0x55e58dffa1b0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f4817b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e58e2fc420 con 0x55e58dfd94d0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f23fff640 1 -- 172.21.15.94:0/1769563630 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4f30004000 con 0x7f4f3c1045f0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35380/0 (socket says 172.21.15.94:35380) 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 0x55e58e3a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3b59e640 1 -- 172.21.15.94:0/3314334480 learned_addr learned my addr 172.21.15.94:0/3314334480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e39aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 -- 172.21.15.94:0/3314334480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 msgr2=0x55e58e39a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 -- 172.21.15.94:0/3314334480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 msgr2=0x55e58e39aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e39aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 -- 172.21.15.94:0/3314334480 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58e2c70d0 con 0x55e58dfd94d0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.935+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 0x55e58e3a1df0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009fa0 tx=0x7f4f2c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2545ff359858f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f23fff640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 0x7f4f240b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f217fa640 1 -- 172.21.15.94:0/3314334480 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c004030 con 0x55e58dfd94d0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f217fa640 1 -- 172.21.15.94:0/3314334480 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f2c0054d0 con 0x55e58dfd94d0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f217fa640 1 -- 172.21.15.94:0/3314334480 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c004c80 con 0x55e58dfd94d0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 0x7f4f240b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f23fff640 1 -- 172.21.15.94:0/1769563630 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4f300def60 con 0x7f4f3c1045f0 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f4817b740 1 -- 172.21.15.94:0/3314334480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 msgr2=0x55e58e3a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 0x7f4f240b1ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55e58e39bb40 tx=0x7f4f34031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.936+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 0x55e58e3a1df0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009fa0 tx=0x7f4f2c0118d0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- 172.21.15.94:0/3314334480 shutdown_connections 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd94d0 0x55e58e3a1df0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e39aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3314334480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dffa1b0 0x55e58e39a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- 172.21.15.94:0/3314334480 >> 172.21.15.94:0/3314334480 conn(0x55e58dfee300 msgr2=0x55e58e396300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- 172.21.15.94:0/3314334480 shutdown_connections 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- 172.21.15.94:0/3314334480 wait complete. 2024-04-25T13:46:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- start start 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 Processor -- start 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 0x55e58e3abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e58e3a2840 con 0x55e58dffa1b0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e58e287b60 con 0x55e58dfd94d0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e58e3a29c0 con 0x55e58dfd12f0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f4817b740 10 client.0 _ll_register_callbacks cb 0x55e58e2ac2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 0x55e58e3abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41674/0 (socket says 172.21.15.94:41674) 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/3272656016 learned_addr learned my addr 172.21.15.94:0/3272656016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.937+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 msgr2=0x55e58e3a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/3272656016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 msgr2=0x55e58e3abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 0x55e58e3abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 -- 172.21.15.94:0/3272656016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58e012210 con 0x55e58dfd12f0 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 0x55e58e3abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c6a0 tx=0x7f4f300e2630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6c8e3a2e2c9070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f300a7170 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4f300e1110 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2c0054d0 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f300a6b10 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f4817b740 10 client.26599 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e58e0de620 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.938+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f2c0054d0 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f300afd70 con 0x55e58dfd12f0 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 10 client.26599 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 20 client.26599 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dfb8d80 0x55e58e3b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 4 client.26599 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e58e30e070 con 0x55e58dfb8d80 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dfb8d80 0x55e58e3b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f1eff5640 10 client.26599 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.939+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dfb8d80 0x55e58e3b22d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced8cdb83bf403d9 server_cookie=a81163dafb22a7e4 in_seq=0 out_seq=0 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f1eff5640 1 -- 172.21.15.94:0/3272656016 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e58e30e070 con 0x55e58dfb8d80 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f1eff5640 10 client.26599 handle_command_reply: tid=0 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 client.26599 shutdown 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 msgr2=0x55e58e3a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c6a0 tx=0x7f4f300e2630 comp rx=0 tx=0).stop 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 shutdown_connections 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dfb8d80 0x55e58e3b22d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dffa1b0 0x55e58e3abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f1eff5640 0 client.26599 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 --2- 172.21.15.94:0/3272656016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfd12f0 0x55e58e3a4670 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 >> 172.21.15.94:0/3272656016 conn(0x55e58dfee300 msgr2=0x55e58e399210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 shutdown_connections 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 1 -- 172.21.15.94:0/3272656016 wait complete. 2024-04-25T13:46:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.941+0000 7f4f4817b740 20 client.26599 trim_cache size 0 max 16384 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.956+0000 7f4f4817b740 1 Processor -- start 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.956+0000 7f4f4817b740 1 -- start start 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e395cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd12f0 0x55e58e3abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ab570 con 0x55e58dfd94d0 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ad0b0 con 0x55e58dfd12f0 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f4817b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1abc40 con 0x55e58dfb8d80 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50582/0 (socket says 172.21.15.94:50582) 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e395cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 -- 172.21.15.94:0/2968891799 learned_addr learned my addr 172.21.15.94:0/2968891799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd12f0 0x55e58e3abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 -- 172.21.15.94:0/2968891799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 msgr2=0x55e58e395cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e395cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 -- 172.21.15.94:0/2968891799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd12f0 msgr2=0x55e58e3abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd12f0 0x55e58e3abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 -- 172.21.15.94:0/2968891799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58e3a1eb0 con 0x55e58dfd94d0 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e395cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c075eb0 tx=0x7f4f34041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579838a836fffaa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.957+0000 7f4f20ff9640 1 -- 172.21.15.94:0/2968891799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34003cb0 con 0x55e58dfd94d0 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f20ff9640 1 -- 172.21.15.94:0/2968891799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34036050 con 0x55e58dfd94d0 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f20ff9640 1 -- 172.21.15.94:0/2968891799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3402f950 con 0x55e58dfd94d0 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 -- 172.21.15.94:0/2968891799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 msgr2=0x55e58e396200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c075eb0 tx=0x7f4f34041000 comp rx=0 tx=0).stop 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 -- 172.21.15.94:0/2968891799 shutdown_connections 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd94d0 0x55e58e396200 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd12f0 0x55e58e3abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 --2- 172.21.15.94:0/2968891799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e395cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 -- 172.21.15.94:0/2968891799 >> 172.21.15.94:0/2968891799 conn(0x55e58dfee300 msgr2=0x55e58e39eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 -- 172.21.15.94:0/2968891799 shutdown_connections 2024-04-25T13:46:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.958+0000 7f4f4817b740 1 -- 172.21.15.94:0/2968891799 wait complete. 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 -- start start 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 Processor -- start 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e3a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ac9e0 con 0x55e58dfd12f0 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1ac310 con 0x55e58dfd94d0 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c1a9550 con 0x55e58dfb8d80 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f4817b740 10 client.0 _ll_register_callbacks cb 0x7f4f3000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50586/0 (socket says 172.21.15.94:50586) 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 -- 172.21.15.94:0/1150338998 learned_addr learned my addr 172.21.15.94:0/1150338998 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e3a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3b59e640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 -- 172.21.15.94:0/1150338998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 msgr2=0x55e58e3a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e3a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 -- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 msgr2=0x55e58e3aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 -- 172.21.15.94:0/1150338998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58e3aaea0 con 0x55e58dfd12f0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f3a59c640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009730 tx=0x7f4f2c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573e2d3c092c769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.959+0000 7f4f1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c035060 con 0x55e58dfd12f0 2024-04-25T13:46:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8385+0+0 (secure 0 0 0) 0x7f4f2c016040 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f34036050 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f2c01d040 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 10 client.17229 fetch_fsmap learned FSMap version 25 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e58e41a720 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34036050 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f2c004660 con 0x55e58dfd12f0 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 10 client.17229 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 20 client.17229 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dffa1b0 0x55e58e41cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 4 client.17229 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:46:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e58e307ee0 con 0x55e58dffa1b0 2024-04-25T13:46:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.960+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dffa1b0 0x55e58e41cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.961+0000 7f4f3ad9d640 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dffa1b0 0x55e58e41cf60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e156670059c4311 server_cookie=e93eb2b9019815dd in_seq=0 out_seq=0 2024-04-25T13:46:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:39.961+0000 7f4f1eff5640 10 client.17229 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:39.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:39 smithi094 ceph-mon[29195]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.3 MiB/s wr, 363 op/s 2024-04-25T13:46:39.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:39 smithi138 ceph-mon[32977]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.3 MiB/s wr, 363 op/s 2024-04-25T13:46:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.133+0000 7f4f1eff5640 1 -- 172.21.15.94:0/1150338998 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e58e307ee0 con 0x55e58dffa1b0 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.133+0000 7f4f1eff5640 10 client.17229 handle_command_reply: tid=0 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 client.17229 shutdown 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 msgr2=0x55e58e3a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009730 tx=0x7f4f2c005d40 comp rx=0 tx=0).stop 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 shutdown_connections 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e58dffa1b0 0x55e58e41cf60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58dfd94d0 0x55e58e3aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58dfd12f0 0x55e58e3a99b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 --2- 172.21.15.94:0/1150338998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58dfb8d80 0x55e58e3a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 >> 172.21.15.94:0/1150338998 conn(0x55e58dfee300 msgr2=0x55e58e399100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f1eff5640 0 client.17229 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 shutdown_connections 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 1 -- 172.21.15.94:0/1150338998 wait complete. 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.134+0000 7f4f4817b740 20 client.17229 trim_cache size 0 max 16384 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 msgr2=0x7f4f240b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 0x7f4f240b1ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55e58e39bb40 tx=0x7f4f34031af0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 msgr2=0x7f4f3c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000e9a0 tx=0x7f4f3000ee60 comp rx=0 tx=0).stop 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 shutdown_connections 2024-04-25T13:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4f240afae0 0x7f4f240b1ff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f3c1098b0 0x7f4f3c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f3c106f50 0x7f4f3c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 --2- 172.21.15.94:0/1769563630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f3c1045f0 0x7f4f3c19a1c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 >> 172.21.15.94:0/1769563630 conn(0x7f4f3c0fdbd0 msgr2=0x7f4f3c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.135+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 shutdown_connections 2024-04-25T13:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:40.136+0000 7f4f411f3640 1 -- 172.21.15.94:0/1769563630 wait complete. 2024-04-25T13:46:40.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 511819 13.39 MB/sec execute 83 sec latency 109.275 ms 2024-04-25T13:46:40.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:46:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T13:46:40.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:46:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T13:46:40.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:46:40 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/3447132981 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56173b5e9a40 con 0x56173b631600 2024-04-25T13:46:40.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:46:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T13:46:41.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:41 smithi094 ceph-mon[29195]: pgmap v493: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-04-25T13:46:41.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:41 smithi094 ceph-mon[29195]: daemon mds.f restarted 2024-04-25T13:46:41.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 514344 13.36 MB/sec execute 84 sec latency 18.856 ms 2024-04-25T13:46:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:41 smithi137 ceph-mon[32568]: pgmap v493: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-04-25T13:46:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:41 smithi137 ceph-mon[32568]: daemon mds.f restarted 2024-04-25T13:46:41.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:41 smithi138 ceph-mon[32977]: pgmap v493: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.9 MiB/s wr, 384 op/s 2024-04-25T13:46:41.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:41 smithi138 ceph-mon[32977]: daemon mds.f restarted 2024-04-25T13:46:42.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 516048 13.31 MB/sec execute 85 sec latency 69.256 ms 2024-04-25T13:46:42.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:42 smithi137 ceph-mon[32568]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:46:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:42 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/1482550752,v1:172.21.15.138:6835/1482550752] up:boot 2024-04-25T13:46:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:42 smithi137 ceph-mon[32568]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:46:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:42 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:46:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:42 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:46:42.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:42 smithi094 ceph-mon[29195]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:46:42.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:42 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/1482550752,v1:172.21.15.138:6835/1482550752] up:boot 2024-04-25T13:46:42.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:42 smithi094 ceph-mon[29195]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:46:42.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:42 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:46:42.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:42 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:46:42.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:42 smithi138 ceph-mon[32977]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:46:42.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:42 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/1482550752,v1:172.21.15.138:6835/1482550752] up:boot 2024-04-25T13:46:42.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:42 smithi138 ceph-mon[32977]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:46:42.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:42 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:46:42.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:42 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:46:43.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:46:43.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:46:43.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 518996 13.37 MB/sec execute 86 sec latency 22.797 ms 2024-04-25T13:46:43.798 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:43 smithi094 ceph-mon[29195]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-25T13:46:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 -- 172.21.15.94:0/583956223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 msgr2=0x7f208410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 --2- 172.21.15.94:0/583956223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208410bcc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f207c009ca0 tx=0x7f207c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 -- 172.21.15.94:0/583956223 shutdown_connections 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 --2- 172.21.15.94:0/583956223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208410bcc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 --2- 172.21.15.94:0/583956223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 0x7f2084101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 --2- 172.21.15.94:0/583956223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 0x7f20841054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 -- 172.21.15.94:0/583956223 >> 172.21.15.94:0/583956223 conn(0x7f20840fd380 msgr2=0x7f20840ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 -- 172.21.15.94:0/583956223 shutdown_connections 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.892+0000 7f208a892640 1 -- 172.21.15.94:0/583956223 wait complete. 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 Processor -- start 2024-04-25T13:46:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 -- start start 2024-04-25T13:46:43.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:43 smithi137 ceph-mon[32568]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 0x7f208419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 0x7f20841a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f208410f640 con 0x7f2084102260 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f208410f4c0 con 0x7f2084101760 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f208a892640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f208410f7c0 con 0x7f20841050b0 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f20837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f20837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56200/0 (socket says 172.21.15.94:56200) 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f20837fe640 1 -- 172.21.15.94:0/2683673949 learned_addr learned my addr 172.21.15.94:0/2683673949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f2083fff640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 0x7f208419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.893+0000 7f2088e2f640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 0x7f20841a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 -- 172.21.15.94:0/2683673949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 msgr2=0x7f20841a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 0x7f20841a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 -- 172.21.15.94:0/2683673949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 msgr2=0x7f208419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 0x7f208419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 -- 172.21.15.94:0/2683673949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20841a1d00 con 0x7f2084102260 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20837fe640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f207800e9f0 tx=0x7f207800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20817fa640 1 -- 172.21.15.94:0/2683673949 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207800cd80 con 0x7f2084102260 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20817fa640 1 -- 172.21.15.94:0/2683673949 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20780108e0 con 0x7f2084102260 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20841a1ff0 con 0x7f2084102260 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f20817fa640 1 -- 172.21.15.94:0/2683673949 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078018980 con 0x7f2084102260 2024-04-25T13:46:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.894+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20841a2460 con 0x7f2084102260 2024-04-25T13:46:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.896+0000 7f20817fa640 1 -- 172.21.15.94:0/2683673949 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2078010430 con 0x7f2084102260 2024-04-25T13:46:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 Processor -- start 2024-04-25T13:46:43.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 -- start start 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef63495a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef63496c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef633fad80 con 0x55ef630cb4d0 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef633b48b0 con 0x55ef630f4390 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f209181a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef633f6d40 con 0x55ef630d36b0 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f20817fa640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 0x7f206c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56204/0 (socket says 172.21.15.94:56204) 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 -- 172.21.15.94:0/87292843 learned_addr learned my addr 172.21.15.94:0/87292843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2088e2f640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef63495a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2088e2f640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 0x7f206c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f20837fe640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef63496c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 -- 172.21.15.94:0/87292843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 msgr2=0x55ef63496c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef63496c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 -- 172.21.15.94:0/87292843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 msgr2=0x55ef63495a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef63495a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f2083fff640 1 -- 172.21.15.94:0/87292843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef63385a40 con 0x55ef630cb4d0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f20837fe640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef63496c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.897+0000 7f20817fa640 1 -- 172.21.15.94:0/2683673949 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f20780def90 con 0x7f2084102260 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.898+0000 7f2083fff640 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2074009c60 tx=0x7f2074011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f4aeae761f8f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.898+0000 7f2088e2f640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 0x7f206c0b2000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f207c0094e0 tx=0x7f207c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.898+0000 7f206a7fc640 1 -- 172.21.15.94:0/87292843 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074004220 con 0x55ef630cb4d0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.898+0000 7f206a7fc640 1 -- 172.21.15.94:0/87292843 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20740043c0 con 0x55ef630cb4d0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.898+0000 7f206a7fc640 1 -- 172.21.15.94:0/87292843 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074019780 con 0x55ef630cb4d0 2024-04-25T13:46:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- 172.21.15.94:0/87292843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 msgr2=0x55ef63495fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2074009c60 tx=0x7f2074011810 comp rx=0 tx=0).stop 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- 172.21.15.94:0/87292843 shutdown_connections 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef63496c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef63495fe0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 --2- 172.21.15.94:0/87292843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef63495a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- 172.21.15.94:0/87292843 >> 172.21.15.94:0/87292843 conn(0x55ef630e84e0 msgr2=0x55ef63491270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- 172.21.15.94:0/87292843 shutdown_connections 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- 172.21.15.94:0/87292843 wait complete. 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 -- start start 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.899+0000 7f209181a740 1 Processor -- start 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630d36b0 0x55ef6349fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef63497780 con 0x55ef630cb4d0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef63381d10 con 0x55ef630d36b0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef63497900 con 0x55ef630f4390 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 10 client.0 _ll_register_callbacks cb 0x55ef633ae210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56218/0 (socket says 172.21.15.94:56218) 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 -- 172.21.15.94:0/1319329756 learned_addr learned my addr 172.21.15.94:0/1319329756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f20837fe640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630d36b0 0x55ef6349fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2088e2f640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 -- 172.21.15.94:0/1319329756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 msgr2=0x55ef634a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634a6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 -- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630d36b0 msgr2=0x55ef6349fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630d36b0 0x55ef6349fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 -- 172.21.15.94:0/1319329756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef631d67d0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f2083fff640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2074009c30 tx=0x7f20740052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c542691e01a4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f20637fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074028070 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f20740035f0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20740043c0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2074034050 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 10 client.17247 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.900+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef6310c3f0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20740043c0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2074004bb0 con 0x55ef630cb4d0 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 10 client.17247 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 20 client.17247 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:43.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630b2f60 0x55ef634acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 4 client.17247 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef63408990 con 0x55ef630b2f60 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f20837fe640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630b2f60 0x55ef634acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f20637fe640 10 client.17247 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.901+0000 7f20837fe640 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630b2f60 0x55ef634acfb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862a6811fb85a05e server_cookie=d05759916c18bec1 in_seq=0 out_seq=0 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.903+0000 7f20637fe640 1 -- 172.21.15.94:0/1319329756 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef63408990 con 0x55ef630b2f60 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.903+0000 7f20637fe640 10 client.17247 handle_command_reply: tid=0 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.903+0000 7f209181a740 1 client.17247 shutdown 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 msgr2=0x55ef6349f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2074009c30 tx=0x7f20740052f0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 shutdown_connections 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630b2f60 0x55ef634acfb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630d36b0 0x55ef6349fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 --2- 172.21.15.94:0/1319329756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef6349f8e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f20637fe640 0 client.17247 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 >> 172.21.15.94:0/1319329756 conn(0x55ef630e84e0 msgr2=0x55ef63494180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 shutdown_connections 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 1 -- 172.21.15.94:0/1319329756 wait complete. 2024-04-25T13:46:43.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.904+0000 7f209181a740 20 client.17247 trim_cache size 0 max 16384 2024-04-25T13:46:43.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.922+0000 7f209181a740 1 Processor -- start 2024-04-25T13:46:43.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.922+0000 7f209181a740 1 -- start start 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630cb4d0 0x55ef6349fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20841ab4e0 con 0x55ef630d36b0 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20841ad020 con 0x55ef630cb4d0 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f209181a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20841abbb0 con 0x55ef630f4390 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56226/0 (socket says 172.21.15.94:56226) 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2088e2f640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 -- 172.21.15.94:0/2307766079 learned_addr learned my addr 172.21.15.94:0/2307766079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2083fff640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630cb4d0 0x55ef6349fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 -- 172.21.15.94:0/2307766079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 msgr2=0x55ef634900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 -- 172.21.15.94:0/2307766079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630cb4d0 msgr2=0x55ef6349fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630cb4d0 0x55ef6349fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 -- 172.21.15.94:0/2307766079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef630cc0f0 con 0x55ef630d36b0 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2088e2f640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f20837fe640 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f20780a6f40 tx=0x7f2078072c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a515dabbe9c759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2069ffb640 1 -- 172.21.15.94:0/2307766079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078074e60 con 0x55ef630d36b0 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2069ffb640 1 -- 172.21.15.94:0/2307766079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20780ab780 con 0x55ef630d36b0 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.923+0000 7f2069ffb640 1 -- 172.21.15.94:0/2307766079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780afda0 con 0x55ef630d36b0 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- 172.21.15.94:0/2307766079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 msgr2=0x55ef63491310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f20780a6f40 tx=0x7f2078072c90 comp rx=0 tx=0).stop 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- 172.21.15.94:0/2307766079 shutdown_connections 2024-04-25T13:46:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630f4390 0x55ef634900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630d36b0 0x55ef63491310 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 --2- 172.21.15.94:0/2307766079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630cb4d0 0x55ef6349fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- 172.21.15.94:0/2307766079 >> 172.21.15.94:0/2307766079 conn(0x55ef630e84e0 msgr2=0x55ef6348f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- 172.21.15.94:0/2307766079 shutdown_connections 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- 172.21.15.94:0/2307766079 wait complete. 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.924+0000 7f209181a740 1 -- start start 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 Processor -- start 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef634a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef634a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20841ac950 con 0x55ef630cb4d0 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20841ac280 con 0x55ef630f4390 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a94c0 con 0x55ef630d36b0 2024-04-25T13:46:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f209181a740 10 client.0 _ll_register_callbacks cb 0x7f2074005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56232/0 (socket says 172.21.15.94:56232) 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 -- 172.21.15.94:0/1097010473 learned_addr learned my addr 172.21.15.94:0/1097010473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2088e2f640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef634a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f20837fe640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef634a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 -- 172.21.15.94:0/1097010473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 msgr2=0x55ef634a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef634a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 -- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 msgr2=0x55ef634a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef634a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 -- 172.21.15.94:0/1097010473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef633bb4b0 con 0x55ef630cb4d0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f2083fff640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2074019590 tx=0x7f20740099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722d8c6050261ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.925+0000 7f20637fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207403c060 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20780ab780 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2074019cb0 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2074028040 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 10 client.17253 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef63514170 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20780ab780 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2074003ca0 con 0x55ef630cb4d0 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 10 client.17253 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 20 client.17253 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630883a0 0x55ef63516b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 4 client.17253 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:46:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.926+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef63402800 con 0x55ef630883a0 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20837fe640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630883a0 0x55ef63516b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20637fe640 10 client.17253 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20837fe640 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630883a0 0x55ef63516b80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b793af1b234e79f2 server_cookie=623e07fa2404e948 in_seq=0 out_seq=0 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20637fe640 1 -- 172.21.15.94:0/1097010473 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef63402800 con 0x55ef630883a0 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20637fe640 10 client.17253 handle_command_reply: tid=0 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 client.17253 shutdown 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 msgr2=0x55ef634a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2074019590 tx=0x7f20740099b0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 shutdown_connections 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef630883a0 0x55ef63516b80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef630f4390 0x55ef634a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef630d36b0 0x55ef634a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f20637fe640 0 client.17253 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 --2- 172.21.15.94:0/1097010473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef630cb4d0 0x55ef634a72d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 >> 172.21.15.94:0/1097010473 conn(0x55ef630e84e0 msgr2=0x55ef63490150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 shutdown_connections 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.927+0000 7f209181a740 1 -- 172.21.15.94:0/1097010473 wait complete. 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f209181a740 20 client.17253 trim_cache size 0 max 16384 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:46:43.934 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 msgr2=0x7f206c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 0x7f206c0b2000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f207c0094e0 tx=0x7f207c0095c0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 msgr2=0x7f208419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f207800e9f0 tx=0x7f207800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 shutdown_connections 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f206c0afaf0 0x7f206c0b2000 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20841050b0 0x7f20841a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2084102260 0x7f208419a700 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 --2- 172.21.15.94:0/2683673949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2084101760 0x7f208419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 >> 172.21.15.94:0/2683673949 conn(0x7f20840fd380 msgr2=0x7f2084107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 shutdown_connections 2024-04-25T13:46:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:43.928+0000 7f208a892640 1 -- 172.21.15.94:0/2683673949 wait complete. 2024-04-25T13:46:43.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:46:43.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:43.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:43 smithi138 ceph-mon[32977]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-04-25T13:46:44.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 -- 172.21.15.94:0/2356093924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 msgr2=0x7f333c10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 --2- 172.21.15.94:0/2356093924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c10bca0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ca0 tx=0x7f333402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 -- 172.21.15.94:0/2356093924 shutdown_connections 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 --2- 172.21.15.94:0/2356093924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c10bca0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 --2- 172.21.15.94:0/2356093924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c106f20 0x7f333c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 --2- 172.21.15.94:0/2356093924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c1045c0 0x7f333c1069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 -- 172.21.15.94:0/2356093924 >> 172.21.15.94:0/2356093924 conn(0x7f333c0fdba0 msgr2=0x7f333c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 -- 172.21.15.94:0/2356093924 shutdown_connections 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 -- 172.21.15.94:0/2356093924 wait complete. 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.095+0000 7f3341a4c640 1 Processor -- start 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 -- start start 2024-04-25T13:46:44.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 0x7f333c1a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c106f20 0x7f333c1a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10d420 con 0x7f333c109880 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10d2a0 con 0x7f333c1045c0 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3341a4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10d5a0 con 0x7f333c106f20 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56246/0 (socket says 172.21.15.94:56246) 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 -- 172.21.15.94:0/117187078 learned_addr learned my addr 172.21.15.94:0/117187078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333affd640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 0x7f333c1a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f3333fff640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c106f20 0x7f333c1a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 -- 172.21.15.94:0/117187078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c106f20 msgr2=0x7f333c1a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c106f20 0x7f333c1a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 -- 172.21.15.94:0/117187078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 msgr2=0x7f333c1a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 0x7f333c1a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333b7fe640 1 -- 172.21.15.94:0/117187078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c1b0ae0 con 0x7f333c109880 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.096+0000 7f333affd640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 0x7f333c1a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f333b7fe640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3334033040 tx=0x7f3334031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334003b70 con 0x7f333c109880 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3334003d10 con 0x7f333c109880 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1b0d70 con 0x7f333c109880 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333402fab0 con 0x7f333c109880 2024-04-25T13:46:44.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.097+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1b1230 con 0x7f333c109880 2024-04-25T13:46:44.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.098+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c0ff6c0 con 0x7f333c109880 2024-04-25T13:46:44.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.100+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f333402fc50 con 0x7f333c109880 2024-04-25T13:46:44.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.101+0000 7f3340a4a640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 0x7f33280b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.101+0000 7f333affd640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 0x7f33280b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.101+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f33341003d0 con 0x7f333c109880 2024-04-25T13:46:44.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.101+0000 7f333affd640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 0x7f33280b2080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f333c06bcf0 tx=0x7f332c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.105+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33340c8f10 con 0x7f333c109880 2024-04-25T13:46:44.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.336+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333c02dc70 con 0x7f333c109880 2024-04-25T13:46:44.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.336+0000 7f3340a4a640 1 -- 172.21.15.94:0/117187078 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f33340c8810 con 0x7f333c109880 2024-04-25T13:46:44.335 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:46:44.336 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:46:44.336 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 msgr2=0x7f33280b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 0x7f33280b2080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f333c06bcf0 tx=0x7f332c005f90 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 msgr2=0x7f333c1b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3334033040 tx=0x7f3334031680 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 shutdown_connections 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33280afb70 0x7f33280b2080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c109880 0x7f333c1b0470 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f333c106f20 0x7f333c1a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 --2- 172.21.15.94:0/117187078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f333c1045c0 0x7f333c1a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 >> 172.21.15.94:0/117187078 conn(0x7f333c0fdba0 msgr2=0x7f333c1050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 shutdown_connections 2024-04-25T13:46:44.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.339+0000 7f3341a4c640 1 -- 172.21.15.94:0/117187078 wait complete. 2024-04-25T13:46:44.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:46:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.465+0000 7f16c93ee640 1 -- 172.21.15.94:0/455267769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4108920 msgr2=0x7f16c410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.465+0000 7f16c93ee640 1 --2- 172.21.15.94:0/455267769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4108920 0x7f16c410ad40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009ca0 tx=0x7f16bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- 172.21.15.94:0/455267769 shutdown_connections 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- 172.21.15.94:0/455267769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4108920 0x7f16c410ad40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- 172.21.15.94:0/455267769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103c20 0x7f16c4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- 172.21.15.94:0/455267769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- 172.21.15.94:0/455267769 >> 172.21.15.94:0/455267769 conn(0x7f16c40fdbd0 msgr2=0x7f16c4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- 172.21.15.94:0/455267769 shutdown_connections 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- 172.21.15.94:0/455267769 wait complete. 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 Processor -- start 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- start start 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4108920 0x7f16c41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410e510 con 0x7f16c4103c20 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410e390 con 0x7f16c41029e0 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.466+0000 7f16c93ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410e690 con 0x7f16c4108920 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56262/0 (socket says 172.21.15.94:56262) 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 -- 172.21.15.94:0/526690221 learned_addr learned my addr 172.21.15.94:0/526690221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c37fe640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4108920 0x7f16c41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 -- 172.21.15.94:0/526690221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4108920 msgr2=0x7f16c41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4108920 0x7f16c41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 -- 172.21.15.94:0/526690221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 msgr2=0x7f16c41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 -- 172.21.15.94:0/526690221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c41a0f20 con 0x7f16c4103c20 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16bbfff640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00eb00 tx=0x7f16ac00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c0ff9640 1 -- 172.21.15.94:0/526690221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00bc30 con 0x7f16c4103c20 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c0ff9640 1 -- 172.21.15.94:0/526690221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ac00bdd0 con 0x7f16c4103c20 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c4069ef0 con 0x7f16c4103c20 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c0ff9640 1 -- 172.21.15.94:0/526690221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac017410 con 0x7f16c4103c20 2024-04-25T13:46:44.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.467+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c406a3f0 con 0x7f16c4103c20 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.470+0000 7f16d0375740 1 Processor -- start 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.470+0000 7f16d0375740 1 -- start start 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.470+0000 7f16c0ff9640 1 -- 172.21.15.94:0/526690221 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f16ac004f70 con 0x7f16c4103c20 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 0x55658d3b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55658d314d80 con 0x55658cfe54d0 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55658d2cea20 con 0x55658cfed6b0 2024-04-25T13:46:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16d0375740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55658d310d40 con 0x55658d00e390 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56270/0 (socket says 172.21.15.94:56270) 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 -- 172.21.15.94:0/379910145 learned_addr learned my addr 172.21.15.94:0/379910145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c37fe640 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 0x55658d3b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 -- 172.21.15.94:0/379910145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 msgr2=0x55658d3b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 0x55658d3b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 -- 172.21.15.94:0/379910145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 msgr2=0x55658d3b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 -- 172.21.15.94:0/379910145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658d2d9c30 con 0x55658cfe54d0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c37fe640 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 0x55658d3b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c0ff9640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 0x7f16b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.471+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f16b40114f0 tx=0x7f16b40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9058bceda25684c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.472+0000 7f16c37fe640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 0x7f16b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.472+0000 7f16c0ff9640 1 -- 172.21.15.94:0/526690221 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f16ac0e0570 con 0x7f16c4103c20 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.472+0000 7f16b97fa640 1 -- 172.21.15.94:0/379910145 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4005020 con 0x55658cfe54d0 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.472+0000 7f16b97fa640 1 -- 172.21.15.94:0/379910145 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b40047a0 con 0x55658cfe54d0 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.472+0000 7f16b97fa640 1 -- 172.21.15.94:0/379910145 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400f880 con 0x55658cfe54d0 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- 172.21.15.94:0/379910145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 msgr2=0x55658d3b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16c37fe640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 0x7f16b00b2000 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55658d3b1860 tx=0x7f16bc02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f16b40114f0 tx=0x7f16b40119b0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- 172.21.15.94:0/379910145 shutdown_connections 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3b0d90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 --2- 172.21.15.94:0/379910145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658d00e390 0x55658d3b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- 172.21.15.94:0/379910145 >> 172.21.15.94:0/379910145 conn(0x55658d0024e0 msgr2=0x55658d3ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- 172.21.15.94:0/379910145 shutdown_connections 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- 172.21.15.94:0/379910145 wait complete. 2024-04-25T13:46:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 -- start start 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.473+0000 7f16d0375740 1 Processor -- start 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d3baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658d00e390 0x55658d3c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55658d3b2530 con 0x55658cfe54d0 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55658d29bdc0 con 0x55658d00e390 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55658d3b26b0 con 0x55658cfed6b0 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 10 client.0 _ll_register_callbacks cb 0x55658d2b7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56282/0 (socket says 172.21.15.94:56282) 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 -- 172.21.15.94:0/4213005524 learned_addr learned my addr 172.21.15.94:0/4213005524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c37fe640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658d00e390 0x55658d3c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16bbfff640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d3baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 -- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 msgr2=0x55658d3baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d3baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 -- 172.21.15.94:0/4213005524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658d00e390 msgr2=0x55658d3c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658d00e390 0x55658d3c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 -- 172.21.15.94:0/4213005524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658d0263f0 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f16b401b040 tx=0x7f16b40051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b242d911bc69b93f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4028070 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f16b400f620 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b40047a0 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16b4034050 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 10 client.17271 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.474+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55658d0f07d0 con 0x55658cfe54d0 2024-04-25T13:46:44.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b40047a0 con 0x55658cfe54d0 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16b4024770 con 0x55658cfe54d0 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 10 client.17271 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 10 client.17271 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 20 client.17271 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658cfccf60 0x55658d3c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 4 client.17271 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658d322990 con 0x55658cfccf60 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.475+0000 7f16bbfff640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658cfccf60 0x55658d3c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.476+0000 7f16bbfff640 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658cfccf60 0x55658d3c8010 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=991cd854050bece1 server_cookie=b2e4c690c7c05d9b in_seq=0 out_seq=0 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.476+0000 7f16aa7fc640 10 client.17271 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.476+0000 7f16aa7fc640 1 -- 172.21.15.94:0/4213005524 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55658d322990 con 0x55658cfccf60 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.476+0000 7f16aa7fc640 10 client.17271 handle_command_reply: tid=0 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.476+0000 7f16d0375740 1 client.17271 shutdown 2024-04-25T13:46:44.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 msgr2=0x55658d3ba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f16b401b040 tx=0x7f16b40051b0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 shutdown_connections 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658cfccf60 0x55658d3c8010 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658d00e390 0x55658d3c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d3baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16aa7fc640 0 client.17271 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 --2- 172.21.15.94:0/4213005524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d3ba520 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 >> 172.21.15.94:0/4213005524 conn(0x55658d0024e0 msgr2=0x55658d3aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 shutdown_connections 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 1 -- 172.21.15.94:0/4213005524 wait complete. 2024-04-25T13:46:44.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.477+0000 7f16d0375740 20 client.17271 trim_cache size 0 max 16384 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.490+0000 7f16d0375740 1 Processor -- start 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.490+0000 7f16d0375740 1 -- start start 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 0x55658d3cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9300 con 0x55658cfccf60 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aae40 con 0x55658cfed6b0 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16d0375740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a99d0 con 0x55658cfe54d0 2024-04-25T13:46:44.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 0x55658d3cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56296/0 (socket says 172.21.15.94:56296) 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 -- 172.21.15.94:0/2646086046 learned_addr learned my addr 172.21.15.94:0/2646086046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c37fe640 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 -- 172.21.15.94:0/2646086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 msgr2=0x55658d3cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 0x55658d3cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 -- 172.21.15.94:0/2646086046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 msgr2=0x55658d3ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 -- 172.21.15.94:0/2646086046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658d3b2410 con 0x55658cfccf60 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16bbfff640 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 0x55658d3cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005e00 tx=0x7f16b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a2807cb2ba7a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.491+0000 7f16b8ff9640 1 -- 172.21.15.94:0/2646086046 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4028040 con 0x55658cfccf60 2024-04-25T13:46:44.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16b8ff9640 1 -- 172.21.15.94:0/2646086046 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b4004780 con 0x55658cfccf60 2024-04-25T13:46:44.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16b8ff9640 1 -- 172.21.15.94:0/2646086046 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400fad0 con 0x55658cfccf60 2024-04-25T13:46:44.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 -- 172.21.15.94:0/2646086046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 msgr2=0x55658d3ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005e00 tx=0x7f16b4009b50 comp rx=0 tx=0).stop 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 -- 172.21.15.94:0/2646086046 shutdown_connections 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfed6b0 0x55658d3ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfe54d0 0x55658d3cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.492+0000 7f16d0375740 1 --2- 172.21.15.94:0/2646086046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfccf60 0x55658d3ba800 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- 172.21.15.94:0/2646086046 >> 172.21.15.94:0/2646086046 conn(0x55658d0024e0 msgr2=0x55658d3ba0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- 172.21.15.94:0/2646086046 shutdown_connections 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- 172.21.15.94:0/2646086046 wait complete. 2024-04-25T13:46:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- start start 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 Processor -- start 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfccf60 0x55658d42a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d431e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aa770 con 0x55658cfe54d0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aa0a0 con 0x55658cfccf60 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a72e0 con 0x55658cfed6b0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16d0375740 10 client.0 _ll_register_callbacks cb 0x7f16ac0e3f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56302/0 (socket says 172.21.15.94:56302) 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16bbfff640 1 -- 172.21.15.94:0/2216102132 learned_addr learned my addr 172.21.15.94:0/2216102132 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfccf60 0x55658d42a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.493+0000 7f16c37fe640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d431e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 -- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 msgr2=0x55658d431e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d431e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 -- 172.21.15.94:0/2216102132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfccf60 msgr2=0x55658d42a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfccf60 0x55658d42a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 -- 172.21.15.94:0/2216102132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658d4323e0 con 0x55658cfe54d0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16bbfff640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0a8760 tx=0x7f16ac0e60e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655760d101d0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0b1150 con 0x55658cfe54d0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f16ac0046c0 con 0x55658cfe54d0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b4004780 con 0x55658cfe54d0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16ac0a8570 con 0x55658cfe54d0 2024-04-25T13:46:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 10 client.17277 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55658d433730 con 0x55658cfe54d0 2024-04-25T13:46:44.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b4004780 con 0x55658cfe54d0 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f16ac0a9ce0 con 0x55658cfe54d0 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 10 client.17277 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 10 client.17277 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 20 client.17277 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658d00e390 0x55658d4360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 4 client.17277 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.494+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55658d31c800 con 0x55658d00e390 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658d00e390 0x55658d4360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16aa7fc640 10 client.17277 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16c2ffd640 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658d00e390 0x55658d4360b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca52b8ffd0979860 server_cookie=f507b9dcc85b9e5c in_seq=0 out_seq=0 2024-04-25T13:46:44.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16aa7fc640 1 -- 172.21.15.94:0/2216102132 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55658d31c800 con 0x55658d00e390 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16aa7fc640 10 client.17277 handle_command_reply: tid=0 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 client.17277 shutdown 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 msgr2=0x55658d42aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0a8760 tx=0x7f16ac0e60e0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 shutdown_connections 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55658d00e390 0x55658d4360b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55658cfed6b0 0x55658d431e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55658cfe54d0 0x55658d42aec0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16aa7fc640 0 client.17277 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 --2- 172.21.15.94:0/2216102132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55658cfccf60 0x55658d42a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 >> 172.21.15.94:0/2216102132 conn(0x55658d0024e0 msgr2=0x55658d3aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 shutdown_connections 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 1 -- 172.21.15.94:0/2216102132 wait complete. 2024-04-25T13:46:44.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.495+0000 7f16d0375740 20 client.17277 trim_cache size 0 max 16384 2024-04-25T13:46:44.505 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:44.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 msgr2=0x7f16b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 0x7f16b00b2000 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55658d3b1860 tx=0x7f16bc02f310 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 msgr2=0x7f16c4199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00eb00 tx=0x7f16ac00ef00 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 shutdown_connections 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b00afaf0 0x7f16b00b2000 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4108920 0x7f16c41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c4103c20 0x7f16c4199920 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 --2- 172.21.15.94:0/526690221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c41029e0 0x7f16c41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 >> 172.21.15.94:0/526690221 conn(0x7f16c40fdbd0 msgr2=0x7f16c4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 shutdown_connections 2024-04-25T13:46:44.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.496+0000 7f16c93ee640 1 -- 172.21.15.94:0/526690221 wait complete. 2024-04-25T13:46:44.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/211276585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48108940 msgr2=0x7f0f4810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/211276585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48108940 0x7f0f4810ad60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009ca0 tx=0x7f0f4002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/211276585 shutdown_connections 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/211276585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48108940 0x7f0f4810ad60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/211276585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48103c40 0x7f0f48104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/211276585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 0x7f0f48103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/211276585 >> 172.21.15.94:0/211276585 conn(0x7f0f480fdbf0 msgr2=0x7f0f481000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/211276585 shutdown_connections 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.624+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/211276585 wait complete. 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 Processor -- start 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 -- start start 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 0x7f0f4819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48108940 0x7f0f481a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4810d400 con 0x7f0f48103c40 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4810d280 con 0x7f0f48108940 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4ff7a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4810d580 con 0x7f0f48102a00 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56316/0 (socket says 172.21.15.94:56316) 2024-04-25T13:46:44.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 -- 172.21.15.94:0/699190620 learned_addr learned my addr 172.21.15.94:0/699190620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 0x7f0f4819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4e517640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48108940 0x7f0f481a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 -- 172.21.15.94:0/699190620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 msgr2=0x7f0f4819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 0x7f0f4819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 -- 172.21.15.94:0/699190620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48108940 msgr2=0x7f0f481a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48108940 0x7f0f481a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.625+0000 7f0f4d515640 1 -- 172.21.15.94:0/699190620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f481a3f10 con 0x7f0f48103c40 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f4d515640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9f0 tx=0x7f0f3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f3effd640 1 -- 172.21.15.94:0/699190620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800cd80 con 0x7f0f48103c40 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f3effd640 1 -- 172.21.15.94:0/699190620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f380108e0 con 0x7f0f48103c40 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481a4200 con 0x7f0f48103c40 2024-04-25T13:46:44.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f3effd640 1 -- 172.21.15.94:0/699190620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38018980 con 0x7f0f48103c40 2024-04-25T13:46:44.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.626+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481a4750 con 0x7f0f48103c40 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.628+0000 7f0f3effd640 1 -- 172.21.15.94:0/699190620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0f38010430 con 0x7f0f48103c40 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.628+0000 7f0f56e21740 1 Processor -- start 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 -- start start 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559723843d80 con 0x55972353d390 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5597237fda20 con 0x5597235144d0 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f56e21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55972383fd40 con 0x55972351c6b0 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f3effd640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 0x7f0f340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4e517640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4e517640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238df810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56330/0 (socket says 172.21.15.94:56330) 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4e517640 1 -- 172.21.15.94:0/2737370397 learned_addr learned my addr 172.21.15.94:0/2737370397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4e517640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 0x7f0f340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f3effd640 1 -- 172.21.15.94:0/699190620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0f380deef0 con 0x7f0f48103c40 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4d515640 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4dd16640 1 -- 172.21.15.94:0/2737370397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 msgr2=0x5597238e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4dd16640 1 -- 172.21.15.94:0/2737370397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 msgr2=0x5597238df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.629+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f4dd16640 1 -- 172.21.15.94:0/2737370397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559723808c30 con 0x5597235144d0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f4e517640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 0x7f0f340b2000 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5597238e0860 tx=0x7f0f40009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238dfd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f48103600 tx=0x7f0f4400bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5ca61ac7c893f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f33fff640 1 -- 172.21.15.94:0/2737370397 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44003da0 con 0x5597235144d0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f33fff640 1 -- 172.21.15.94:0/2737370397 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44015070 con 0x5597235144d0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.630+0000 7f0f33fff640 1 -- 172.21.15.94:0/2737370397 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440116a0 con 0x5597235144d0 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- 172.21.15.94:0/2737370397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 msgr2=0x5597238dfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238dfd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f48103600 tx=0x7f0f4400bbe0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- 172.21.15.94:0/2737370397 shutdown_connections 2024-04-25T13:46:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238dfd90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 --2- 172.21.15.94:0/2737370397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- 172.21.15.94:0/2737370397 >> 172.21.15.94:0/2737370397 conn(0x5597235314e0 msgr2=0x5597238db020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- 172.21.15.94:0/2737370397 shutdown_connections 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- 172.21.15.94:0/2737370397 wait complete. 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 -- start start 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.631+0000 7f0f56e21740 1 Processor -- start 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597238e1580 con 0x55972353d390 2024-04-25T13:46:44.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5597237cadc0 con 0x5597235144d0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5597238e1700 con 0x55972351c6b0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f56e21740 10 client.0 _ll_register_callbacks cb 0x5597237e6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56332/0 (socket says 172.21.15.94:56332) 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 -- 172.21.15.94:0/459253354 learned_addr learned my addr 172.21.15.94:0/459253354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4d515640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 -- 172.21.15.94:0/459253354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 msgr2=0x5597238e9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 -- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 msgr2=0x5597238e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 -- 172.21.15.94:0/459253354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597235553f0 con 0x55972353d390 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.632+0000 7f0f4e517640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f4002ade0 tx=0x7f0f4002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28838359023b2f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40052070 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0f40004250 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f44015070 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f4005e050 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 10 client.17289 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55972361f7d0 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44015070 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f4004c040 con 0x55972353d390 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 10 client.17289 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 10 client.17289 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 20 client.17289 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234fbf60 0x5597238f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 4 client.17289 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.633+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559723851990 con 0x5597234fbf60 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.634+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234fbf60 0x5597238f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.634+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234fbf60 0x5597238f7250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=106e79b1c83595f6 server_cookie=ec4e5598647692c in_seq=0 out_seq=0 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.634+0000 7f0f317fa640 10 client.17289 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f317fa640 1 -- 172.21.15.94:0/459253354 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559723851990 con 0x5597234fbf60 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f317fa640 10 client.17289 handle_command_reply: tid=0 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 client.17289 shutdown 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 msgr2=0x5597238f0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f4002ade0 tx=0x7f0f4002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 shutdown_connections 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234fbf60 0x5597238f7250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55972353d390 0x5597238f0a20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f317fa640 0 client.17289 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 --2- 172.21.15.94:0/459253354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597235144d0 0x5597238e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 >> 172.21.15.94:0/459253354 conn(0x5597235314e0 msgr2=0x5597238ddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 shutdown_connections 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 1 -- 172.21.15.94:0/459253354 wait complete. 2024-04-25T13:46:44.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.635+0000 7f0f56e21740 20 client.17289 trim_cache size 0 max 16384 2024-04-25T13:46:44.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 Processor -- start 2024-04-25T13:46:44.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 -- start start 2024-04-25T13:46:44.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972351c6b0 0x5597238f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972353d390 0x5597238fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ad720 con 0x5597235144d0 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481af260 con 0x55972351c6b0 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f56e21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481addf0 con 0x55972353d390 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56340/0 (socket says 172.21.15.94:56340) 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1589767204 learned_addr learned my addr 172.21.15.94:0/1589767204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4d515640 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972351c6b0 0x5597238f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4e517640 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972353d390 0x5597238fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1589767204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972353d390 msgr2=0x5597238fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972353d390 0x5597238fba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1589767204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972351c6b0 msgr2=0x5597238f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972351c6b0 0x5597238f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1589767204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597238df1a0 con 0x5597235144d0 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0f44004540 tx=0x7f0f44013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db92054330faa9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.649+0000 7f0f337fe640 1 -- 172.21.15.94:0/1589767204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44005040 con 0x5597235144d0 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f337fe640 1 -- 172.21.15.94:0/1589767204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44015070 con 0x5597235144d0 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f337fe640 1 -- 172.21.15.94:0/1589767204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004dd0 con 0x5597235144d0 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 -- 172.21.15.94:0/1589767204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 msgr2=0x5597238e8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0f44004540 tx=0x7f0f44013040 comp rx=0 tx=0).stop 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 -- 172.21.15.94:0/1589767204 shutdown_connections 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972353d390 0x5597238fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972351c6b0 0x5597238f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1589767204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238e8470 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 -- 172.21.15.94:0/1589767204 >> 172.21.15.94:0/1589767204 conn(0x5597235314e0 msgr2=0x5597238e7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 -- 172.21.15.94:0/1589767204 shutdown_connections 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.650+0000 7f0f56e21740 1 -- 172.21.15.94:0/1589767204 wait complete. 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 -- start start 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 Processor -- start 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972353d390 0x55972395d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aeb90 con 0x5597235144d0 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ae4c0 con 0x55972353d390 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ab700 con 0x55972351c6b0 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f56e21740 10 client.0 _ll_register_callbacks cb 0x7f0f4002f3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56352/0 (socket says 172.21.15.94:56352) 2024-04-25T13:46:44.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1321755797 learned_addr learned my addr 172.21.15.94:0/1321755797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4d515640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4e517640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972353d390 0x55972395d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1321755797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 msgr2=0x5597238f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972353d390 msgr2=0x55972395d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972353d390 0x55972395d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.651+0000 7f0f4dd16640 1 -- 172.21.15.94:0/1321755797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597238f0e80 con 0x5597235144d0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f4dd16640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000c00 tx=0x7f0f4400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e582904c27985786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004b50 con 0x5597235144d0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0f44004cf0 con 0x5597235144d0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f44015070 con 0x5597235144d0 2024-04-25T13:46:44.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f44005040 con 0x5597235144d0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597238e17a0 con 0x5597235144d0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44015070 con 0x5597235144d0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f4402c530 con 0x5597235144d0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 10 client.17295 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 20 client.17295 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234d13a0 0x559723961610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 4 client.17295 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.652+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55972384b800 con 0x5597234d13a0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f4d515640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234d13a0 0x559723961610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f317fa640 10 client.17295 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f4d515640 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234d13a0 0x559723961610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4db1d0cf7341e0c6 server_cookie=8d7dbebabef4ee16 in_seq=0 out_seq=0 2024-04-25T13:46:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f317fa640 1 -- 172.21.15.94:0/1321755797 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55972384b800 con 0x5597234d13a0 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f317fa640 10 client.17295 handle_command_reply: tid=0 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 client.17295 shutdown 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 msgr2=0x5597238efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000c00 tx=0x7f0f4400be90 comp rx=0 tx=0).stop 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 shutdown_connections 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5597234d13a0 0x559723961610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55972353d390 0x55972395d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55972351c6b0 0x5597238f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 --2- 172.21.15.94:0/1321755797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597235144d0 0x5597238efd40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f317fa640 0 client.17295 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 >> 172.21.15.94:0/1321755797 conn(0x5597235314e0 msgr2=0x5597238dde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 shutdown_connections 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.653+0000 7f0f56e21740 1 -- 172.21.15.94:0/1321755797 wait complete. 2024-04-25T13:46:44.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f56e21740 20 client.17295 trim_cache size 0 max 16384 2024-04-25T13:46:44.660 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 msgr2=0x7f0f340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 0x7f0f340b2000 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5597238e0860 tx=0x7f0f40009360 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 msgr2=0x7f0f4819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9f0 tx=0x7f0f3800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 shutdown_connections 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f340afaf0 0x7f0f340b2000 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f48108940 0x7f0f481a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f48103c40 0x7f0f4819c910 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 --2- 172.21.15.94:0/699190620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102a00 0x7f0f4819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 >> 172.21.15.94:0/699190620 conn(0x7f0f480fdbf0 msgr2=0x7f0f48100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 shutdown_connections 2024-04-25T13:46:44.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.654+0000 7f0f4ff7a640 1 -- 172.21.15.94:0/699190620 wait complete. 2024-04-25T13:46:44.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/117187078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:44.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:46:44.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- 172.21.15.94:0/4012111576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5501080b0 msgr2=0x7fa55010a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 --2- 172.21.15.94:0/4012111576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5501080b0 0x7fa55010a4d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa548009ca0 tx=0x7fa54802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- 172.21.15.94:0/4012111576 shutdown_connections 2024-04-25T13:46:44.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 --2- 172.21.15.94:0/4012111576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5501080b0 0x7fa55010a4d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 --2- 172.21.15.94:0/4012111576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 0x7fa5501037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 --2- 172.21.15.94:0/4012111576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa550102170 0x7fa550102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- 172.21.15.94:0/4012111576 >> 172.21.15.94:0/4012111576 conn(0x7fa5500fd340 msgr2=0x7fa5500ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- 172.21.15.94:0/4012111576 shutdown_connections 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- 172.21.15.94:0/4012111576 wait complete. 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 Processor -- start 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.778+0000 7fa558300640 1 -- start start 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 0x7fa55019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5501080b0 0x7fa5501a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010f610 con 0x7fa550102170 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010f490 con 0x7fa5501080b0 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010f790 con 0x7fa5501033b0 2024-04-25T13:46:44.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56370/0 (socket says 172.21.15.94:56370) 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 -- 172.21.15.94:0/2144560233 learned_addr learned my addr 172.21.15.94:0/2144560233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55589b640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 0x7fa55019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55689d640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5501080b0 0x7fa5501a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 -- 172.21.15.94:0/2144560233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 msgr2=0x7fa55019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 0x7fa55019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 -- 172.21.15.94:0/2144560233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5501080b0 msgr2=0x7fa5501a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5501080b0 0x7fa5501a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 -- 172.21.15.94:0/2144560233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5501a1d20 con 0x7fa550102170 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa55609c640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9f0 tx=0x7fa54c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa5477fe640 1 -- 172.21.15.94:0/2144560233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00ccc0 con 0x7fa550102170 2024-04-25T13:46:44.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa5477fe640 1 -- 172.21.15.94:0/2144560233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa54c0035f0 con 0x7fa550102170 2024-04-25T13:46:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5501a1fb0 con 0x7fa550102170 2024-04-25T13:46:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa5477fe640 1 -- 172.21.15.94:0/2144560233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00ce60 con 0x7fa550102170 2024-04-25T13:46:44.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.779+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5501a2470 con 0x7fa550102170 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 Processor -- start 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 -- start start 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 0x55a2d77a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 0x55a2d77a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a2d7709d80 con 0x55a2d73da4d0 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2d76c3a20 con 0x55a2d73e26b0 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2d7705d40 con 0x55a2d7403390 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55689d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 0x55a2d77a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.782+0000 7fa55689d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 0x55a2d77a5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36550/0 (socket says 172.21.15.94:36550) 2024-04-25T13:46:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55689d640 1 -- 172.21.15.94:0/4293843285 learned_addr learned my addr 172.21.15.94:0/4293843285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 0x55a2d77a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 -- 172.21.15.94:0/4293843285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 msgr2=0x55a2d77a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 0x55a2d77a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 -- 172.21.15.94:0/4293843285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 msgr2=0x55a2d77a5d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77a5d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 -- 172.21.15.94:0/4293843285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d76cec30 con 0x55a2d73e26b0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa5477fe640 1 -- 172.21.15.94:0/2144560233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa54c003aa0 con 0x7fa550102170 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa55589b640 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 0x55a2d77a6a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa55019b280 tx=0x7fa54000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f160d85056dd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa544bfe640 1 -- 172.21.15.94:0/4293843285 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540005020 con 0x55a2d73e26b0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa544bfe640 1 -- 172.21.15.94:0/4293843285 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa540005670 con 0x55a2d73e26b0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa544bfe640 1 -- 172.21.15.94:0/4293843285 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5400186a0 con 0x55a2d73e26b0 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.783+0000 7fa5477fe640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 0x7fa53c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55689d640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 0x7fa53c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa5477fe640 1 -- 172.21.15.94:0/2144560233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa54c0df030 con 0x7fa550102170 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 -- 172.21.15.94:0/4293843285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 msgr2=0x55a2d77a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55689d640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 0x7fa53c0b2000 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5480094e0 tx=0x7fa548009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 0x55a2d77a6a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa55019b280 tx=0x7fa54000f3c0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 -- 172.21.15.94:0/4293843285 shutdown_connections 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73e26b0 0x55a2d77a6a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/4293843285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d7403390 0x55a2d77a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 -- 172.21.15.94:0/4293843285 >> 172.21.15.94:0/4293843285 conn(0x55a2d73f74e0 msgr2=0x55a2d77a1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 -- 172.21.15.94:0/4293843285 shutdown_connections 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.784+0000 7fa55f1b1740 1 -- 172.21.15.94:0/4293843285 wait complete. 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 -- start start 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 Processor -- start 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d77bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a2d77a7530 con 0x55a2d73da4d0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2d7690dc0 con 0x55a2d7403390 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2d77a76b0 con 0x55a2d73e26b0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55f1b1740 10 client.0 _ll_register_callbacks cb 0x55a2d76ac600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56372/0 (socket says 172.21.15.94:56372) 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 -- 172.21.15.94:0/3185416296 learned_addr learned my addr 172.21.15.94:0/3185416296 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55689d640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d77bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55589b640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 -- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 msgr2=0x55a2d77b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 -- 172.21.15.94:0/3185416296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 msgr2=0x55a2d77bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d77bae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.785+0000 7fa55609c640 1 -- 172.21.15.94:0/3185416296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d741b3f0 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55609c640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa54c012020 tx=0x7fa54c0a6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936318d7505dd785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0a6f80 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa54c004230 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa540005670 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa54c0a7b70 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 10 client.17304 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2d74e57d0 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa540005670 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa54c01f070 con 0x55a2d73da4d0 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 10 client.17304 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 10 client.17304 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 20 client.17304 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73c1f60 0x55a2d77c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 4 client.17304 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.786+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d7717990 con 0x55a2d73c1f60 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.787+0000 7fa55589b640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73c1f60 0x55a2d77c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.787+0000 7fa539ffb640 10 client.17304 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.787+0000 7fa55589b640 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73c1f60 0x55a2d77c15f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db88beca6f2c37b6 server_cookie=6f4dd4f8807c73bf in_seq=0 out_seq=0 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa539ffb640 1 -- 172.21.15.94:0/3185416296 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2d7717990 con 0x55a2d73c1f60 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa539ffb640 10 client.17304 handle_command_reply: tid=0 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 client.17304 shutdown 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 msgr2=0x55a2d77b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa54c012020 tx=0x7fa54c0a6ed0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 shutdown_connections 2024-04-25T13:46:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73c1f60 0x55a2d77c15f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d77bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa539ffb640 0 client.17304 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3185416296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3980 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 >> 172.21.15.94:0/3185416296 conn(0x55a2d73f74e0 msgr2=0x55a2d77a0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 shutdown_connections 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3185416296 wait complete. 2024-04-25T13:46:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.788+0000 7fa55f1b1740 20 client.17304 trim_cache size 0 max 16384 2024-04-25T13:46:44.796 INFO:tasks.workunit.client.0.smithi094.stdout: 1 521859 13.37 MB/sec execute 87 sec latency 12.945 ms 2024-04-25T13:46:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.801+0000 7fa55f1b1740 1 Processor -- start 2024-04-25T13:46:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.801+0000 7fa55f1b1740 1 -- start start 2024-04-25T13:46:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501a9360 con 0x55a2d7403390 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501aaea0 con 0x55a2d73da4d0 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501a9a30 con 0x55a2d73e26b0 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55589b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55609c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56384/0 (socket says 172.21.15.94:56384) 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 -- 172.21.15.94:0/3477950178 learned_addr learned my addr 172.21.15.94:0/3477950178 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55589b640 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36568/0 (socket says 172.21.15.94:36568) 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 -- 172.21.15.94:0/3477950178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 msgr2=0x55a2d77b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 -- 172.21.15.94:0/3477950178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 msgr2=0x55a2d77b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 -- 172.21.15.94:0/3477950178 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d779c7f0 con 0x55a2d7403390 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55609c640 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.802+0000 7fa55689d640 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55a2d77a6860 tx=0x7fa548044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddae94d4eaa37873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.803+0000 7fa53bfff640 1 -- 172.21.15.94:0/3477950178 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548004040 con 0x55a2d7403390 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.803+0000 7fa53bfff640 1 -- 172.21.15.94:0/3477950178 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548036050 con 0x55a2d7403390 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.803+0000 7fa53bfff640 1 -- 172.21.15.94:0/3477950178 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54802feb0 con 0x55a2d7403390 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3477950178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 msgr2=0x55a2d77c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55a2d77a6860 tx=0x7fa548044000 comp rx=0 tx=0).stop 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3477950178 shutdown_connections 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d7403390 0x55a2d77c5df0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73e26b0 0x55a2d77b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/3477950178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d73da4d0 0x55a2d77b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3477950178 >> 172.21.15.94:0/3477950178 conn(0x55a2d73f74e0 msgr2=0x55a2d77aa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3477950178 shutdown_connections 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- 172.21.15.94:0/3477950178 wait complete. 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- start start 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 Processor -- start 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73da4d0 0x55a2d77b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d78279a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501aa7d0 con 0x55a2d73e26b0 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501aa100 con 0x55a2d7403390 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501a7340 con 0x55a2d73da4d0 2024-04-25T13:46:44.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55f1b1740 10 client.0 _ll_register_callbacks cb 0x7fa54c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55589b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55589b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56388/0 (socket says 172.21.15.94:56388) 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55589b640 1 -- 172.21.15.94:0/1152610116 learned_addr learned my addr 172.21.15.94:0/1152610116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55609c640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73da4d0 0x55a2d77b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.804+0000 7fa55689d640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d78279a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 -- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73da4d0 msgr2=0x55a2d77b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73da4d0 0x55a2d77b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 -- 172.21.15.94:0/1152610116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 msgr2=0x55a2d78279a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d78279a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 -- 172.21.15.94:0/1152610116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d7827ee0 con 0x55a2d73e26b0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55589b640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55a2d77a1db0 tx=0x7fa540005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a8709d02523db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54002b430 con 0x55a2d73e26b0 2024-04-25T13:46:44.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa54002b5d0 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa548036050 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa54002b7f0 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55f1b1740 10 client.17310 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2d7829120 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.805+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548036050 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa54002bce0 con 0x55a2d73e26b0 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 10 client.17310 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 10 client.17310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 20 client.17310 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73973a0 0x55a2d782b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 4 client.17310 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:44.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2d7711800 con 0x55a2d73973a0 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55609c640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73973a0 0x55a2d782b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa539ffb640 10 client.17310 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55609c640 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73973a0 0x55a2d782b890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68ade7a63fd09b9a server_cookie=d799440cc023c066 in_seq=0 out_seq=0 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa539ffb640 1 -- 172.21.15.94:0/1152610116 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2d7711800 con 0x55a2d73973a0 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa539ffb640 10 client.17310 handle_command_reply: tid=0 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 client.17310 shutdown 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 msgr2=0x55a2d77b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55a2d77a1db0 tx=0x7fa540005d00 comp rx=0 tx=0).stop 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 shutdown_connections 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2d73973a0 0x55a2d782b890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2d7403390 0x55a2d78279a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2d73e26b0 0x55a2d77b79f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 --2- 172.21.15.94:0/1152610116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d73da4d0 0x55a2d77b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa539ffb640 0 client.17310 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:46:44.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 >> 172.21.15.94:0/1152610116 conn(0x55a2d73f74e0 msgr2=0x55a2d77b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.806+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 shutdown_connections 2024-04-25T13:46:44.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa55f1b1740 1 -- 172.21.15.94:0/1152610116 wait complete. 2024-04-25T13:46:44.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa55f1b1740 20 client.17310 trim_cache size 0 max 16384 2024-04-25T13:46:44.813 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 msgr2=0x7fa53c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 0x7fa53c0b2000 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5480094e0 tx=0x7fa548009450 comp rx=0 tx=0).stop 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 msgr2=0x7fa55019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9f0 tx=0x7fa54c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 shutdown_connections 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa53c0afaf0 0x7fa53c0b2000 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5501080b0 0x7fa5501a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5501033b0 0x7fa55019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 --2- 172.21.15.94:0/2144560233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa550102170 0x7fa55019a1e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 >> 172.21.15.94:0/2144560233 conn(0x7fa5500fd340 msgr2=0x7fa550104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 shutdown_connections 2024-04-25T13:46:44.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.807+0000 7fa558300640 1 -- 172.21.15.94:0/2144560233 wait complete. 2024-04-25T13:46:44.817 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:46:44.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/117187078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.930+0000 7f88dc784640 1 -- 172.21.15.94:0/645991231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 msgr2=0x7f88d4101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.930+0000 7f88dc784640 1 --2- 172.21.15.94:0/645991231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d4101bd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009900 tx=0x7f88c402a410 comp rx=0 tx=0).stop 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- 172.21.15.94:0/645991231 shutdown_connections 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- 172.21.15.94:0/645991231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d4102260 0x7f88d410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- 172.21.15.94:0/645991231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d4101bd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- 172.21.15.94:0/645991231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d41050b0 0x7f88d41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- 172.21.15.94:0/645991231 >> 172.21.15.94:0/645991231 conn(0x7f88d40fd380 msgr2=0x7f88d40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- 172.21.15.94:0/645991231 shutdown_connections 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- 172.21.15.94:0/645991231 wait complete. 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 Processor -- start 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- start start 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d4102260 0x7f88d419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 0x7f88d41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d410f640 con 0x7f88d4101760 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d410f4c0 con 0x7f88d4102260 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88dc784640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d410f7c0 con 0x7f88d41050b0 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88da520640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.931+0000 7f88da520640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56408/0 (socket says 172.21.15.94:56408) 2024-04-25T13:46:44.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 -- 172.21.15.94:0/3653490755 learned_addr learned my addr 172.21.15.94:0/3653490755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d4102260 0x7f88d419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88dad21640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 0x7f88d41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 -- 172.21.15.94:0/3653490755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 msgr2=0x7f88d41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 0x7f88d41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 -- 172.21.15.94:0/3653490755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d4102260 msgr2=0x7f88d419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d4102260 0x7f88d419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 -- 172.21.15.94:0/3653490755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d41a1d80 con 0x7f88d4101760 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88dad21640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 0x7f88d41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88da520640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f88d000e9f0 tx=0x7f88d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88cb7fe640 1 -- 172.21.15.94:0/3653490755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d000ccc0 con 0x7f88d4101760 2024-04-25T13:46:44.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88cb7fe640 1 -- 172.21.15.94:0/3653490755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d00035f0 con 0x7f88d4101760 2024-04-25T13:46:44.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88cb7fe640 1 -- 172.21.15.94:0/3653490755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d000ce60 con 0x7f88d4101760 2024-04-25T13:46:44.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d41a2070 con 0x7f88d4101760 2024-04-25T13:46:44.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.932+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41a2610 con 0x7f88d4101760 2024-04-25T13:46:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 Processor -- start 2024-04-25T13:46:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88cb7fe640 1 -- 172.21.15.94:0/3653490755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f88d0003aa0 con 0x7f88d4101760 2024-04-25T13:46:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 -- start start 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 0x564b93647810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b93647d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932846b0 0x564b93648a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b935abd80 con 0x564b9327c4d0 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b93565a20 con 0x564b932a5390 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88e3638740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b935a7d40 con 0x564b932846b0 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88da520640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b93647d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.935+0000 7f88dad21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 0x564b93647810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88dad21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 0x564b93647810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53538/0 (socket says 172.21.15.94:53538) 2024-04-25T13:46:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88dad21640 1 -- 172.21.15.94:0/1579717985 learned_addr learned my addr 172.21.15.94:0/1579717985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932846b0 0x564b93648a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88cb7fe640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 0x7f88c00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 -- 172.21.15.94:0/1579717985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932846b0 msgr2=0x564b93648a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932846b0 0x564b93648a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 -- 172.21.15.94:0/1579717985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 msgr2=0x564b93647810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 0x564b93647810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 -- 172.21.15.94:0/1579717985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b93570c30 con 0x564b9327c4d0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88dad21640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 0x7f88c00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88cb7fe640 1 -- 172.21.15.94:0/3653490755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f88d00defc0 con 0x7f88d4101760 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88da520640 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b93647d90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f88d00dc9f0 tx=0x7f88d00d5b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ff3372611b8b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88c8ff9640 1 -- 172.21.15.94:0/1579717985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00a71d0 con 0x564b9327c4d0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88dad21640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 0x7f88c00b2080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564b93648860 tx=0x7f88cc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88c8ff9640 1 -- 172.21.15.94:0/1579717985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d00a6f10 con 0x564b9327c4d0 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.936+0000 7f88c8ff9640 1 -- 172.21.15.94:0/1579717985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00a6b70 con 0x564b9327c4d0 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 -- 172.21.15.94:0/1579717985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 msgr2=0x564b93647d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b93647d90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f88d00dc9f0 tx=0x7f88d00d5b50 comp rx=0 tx=0).stop 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 -- 172.21.15.94:0/1579717985 shutdown_connections 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932846b0 0x564b93648a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b93647d90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 --2- 172.21.15.94:0/1579717985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932a5390 0x564b93647810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 -- 172.21.15.94:0/1579717985 >> 172.21.15.94:0/1579717985 conn(0x564b932994e0 msgr2=0x564b93643020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 -- 172.21.15.94:0/1579717985 shutdown_connections 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.937+0000 7f88e3638740 1 -- 172.21.15.94:0/1579717985 wait complete. 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 -- start start 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 Processor -- start 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b9327c4d0 0x564b93651440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b936588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b936494d0 con 0x564b932846b0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b93532dc0 con 0x564b9327c4d0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b93649650 con 0x564b932a5390 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88e3638740 10 client.0 _ll_register_callbacks cb 0x564b9354e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56412/0 (socket says 172.21.15.94:56412) 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 -- 172.21.15.94:0/3745160838 learned_addr learned my addr 172.21.15.94:0/3745160838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88da520640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b9327c4d0 0x564b93651440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88dad21640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b936588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 -- 172.21.15.94:0/3745160838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 msgr2=0x564b936588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b936588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 -- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b9327c4d0 msgr2=0x564b93651440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b9327c4d0 0x564b93651440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.938+0000 7f88d9d1f640 1 -- 172.21.15.94:0/3745160838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b932bd3f0 con 0x564b932846b0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f88c402a930 tx=0x7f88c4031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b7f8904b62e4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4035040 con 0x564b932846b0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f88c4004040 con 0x564b932846b0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d00a6f10 con 0x564b932846b0 2024-04-25T13:46:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88c4004240 con 0x564b932846b0 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 10 client.17322 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b933877d0 con 0x564b932846b0 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d00a6f10 con 0x564b932846b0 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f88c4044020 con 0x564b932846b0 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 10 client.17322 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 10 client.17322 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 20 client.17322 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b93263f60 0x564b9365f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 4 client.17322 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.939+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b935b9990 con 0x564b93263f60 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.940+0000 7f88da520640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b93263f60 0x564b9365f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.940+0000 7f88bdffb640 10 client.17322 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.940+0000 7f88da520640 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b93263f60 0x564b9365f120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c3fb40f838ebc14 server_cookie=1b09f9d36bb9cf06 in_seq=0 out_seq=0 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88bdffb640 1 -- 172.21.15.94:0/3745160838 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b935b9990 con 0x564b93263f60 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88bdffb640 10 client.17322 handle_command_reply: tid=0 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 client.17322 shutdown 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 msgr2=0x564b93651980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f88c402a930 tx=0x7f88c4031e80 comp rx=0 tx=0).stop 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 shutdown_connections 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b93263f60 0x564b9365f120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b936588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932846b0 0x564b93651980 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 --2- 172.21.15.94:0/3745160838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b9327c4d0 0x564b93651440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88bdffb640 0 client.17322 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 >> 172.21.15.94:0/3745160838 conn(0x564b932994e0 msgr2=0x564b93645f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 shutdown_connections 2024-04-25T13:46:44.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 1 -- 172.21.15.94:0/3745160838 wait complete. 2024-04-25T13:46:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.941+0000 7f88e3638740 20 client.17322 trim_cache size 0 max 16384 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.954+0000 7f88e3638740 1 Processor -- start 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.954+0000 7f88e3638740 1 -- start start 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93658800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b93642f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41ab5b0 con 0x564b9327c4d0 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41ad0f0 con 0x564b932846b0 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88e3638740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41abc80 con 0x564b932a5390 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56426/0 (socket says 172.21.15.94:56426) 2024-04-25T13:46:44.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 -- 172.21.15.94:0/2342338012 learned_addr learned my addr 172.21.15.94:0/2342338012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88d9d1f640 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93658800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88dad21640 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b93642f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 -- 172.21.15.94:0/2342338012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 msgr2=0x564b93642f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b93642f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 -- 172.21.15.94:0/2342338012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 msgr2=0x564b93658800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93658800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 -- 172.21.15.94:0/2342338012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b93658d40 con 0x564b9327c4d0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88da520640 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f88d00dcd90 tx=0x7f88d0003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b325883ffda75e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88bffff640 1 -- 172.21.15.94:0/2342338012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00a6d50 con 0x564b9327c4d0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88bffff640 1 -- 172.21.15.94:0/2342338012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d0076450 con 0x564b9327c4d0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.955+0000 7f88bffff640 1 -- 172.21.15.94:0/2342338012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00ab7b0 con 0x564b9327c4d0 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- 172.21.15.94:0/2342338012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 msgr2=0x564b936429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f88d00dcd90 tx=0x7f88d0003a10 comp rx=0 tx=0).stop 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- 172.21.15.94:0/2342338012 shutdown_connections 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b932a5390 0x564b93642f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93658800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 --2- 172.21.15.94:0/2342338012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9327c4d0 0x564b936429e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- 172.21.15.94:0/2342338012 >> 172.21.15.94:0/2342338012 conn(0x564b932994e0 msgr2=0x564b9364c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- 172.21.15.94:0/2342338012 shutdown_connections 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- 172.21.15.94:0/2342338012 wait complete. 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 -- start start 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.956+0000 7f88e3638740 1 Processor -- start 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b9327c4d0 0x564b936566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93656bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41aca20 con 0x564b932a5390 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41ac350 con 0x564b932846b0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41a9590 con 0x564b9327c4d0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 10 client.0 _ll_register_callbacks cb 0x7f88c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88d9d1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93656bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56440/0 (socket says 172.21.15.94:56440) 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 -- 172.21.15.94:0/28156306 learned_addr learned my addr 172.21.15.94:0/28156306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88da520640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b9327c4d0 0x564b936566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 -- 172.21.15.94:0/28156306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b9327c4d0 msgr2=0x564b936566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b9327c4d0 0x564b936566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 -- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 msgr2=0x564b93656bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:44.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93656bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 -- 172.21.15.94:0/28156306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b936580e0 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88dad21640 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00fb00 tx=0x7f88cc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55588b0dfac1ba30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc02c060 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f88cc027440 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.957+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b935b3800 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88cc027640 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 10 client.17328 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b936c7b80 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564b935b3800 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f88cc0108d0 con 0x564b932a5390 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 10 client.17328 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:44.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 10 client.17328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 20 client.17328 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b932393a0 0x564b936ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 4 client.17328 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b9352e610 con 0x564b932393a0 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88da520640 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b932393a0 0x564b936ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88bdffb640 10 client.17328 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.958+0000 7f88da520640 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b932393a0 0x564b936ca5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa524eeb5a300e79 server_cookie=4a497cd714216df2 in_seq=0 out_seq=0 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88bdffb640 1 -- 172.21.15.94:0/28156306 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b9352e610 con 0x564b932393a0 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88bdffb640 10 client.17328 handle_command_reply: tid=0 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 client.17328 shutdown 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 msgr2=0x564b93657b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00fb00 tx=0x7f88cc014040 comp rx=0 tx=0).stop 2024-04-25T13:46:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 shutdown_connections 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564b932393a0 0x564b936ca5d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b932a5390 0x564b93657b20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b932846b0 0x564b93656bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 --2- 172.21.15.94:0/28156306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b9327c4d0 0x564b936566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88bdffb640 0 client.17328 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 >> 172.21.15.94:0/28156306 conn(0x564b932994e0 msgr2=0x564b936429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 shutdown_connections 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 1 -- 172.21.15.94:0/28156306 wait complete. 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88e3638740 20 client.17328 trim_cache size 0 max 16384 2024-04-25T13:46:44.966 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 msgr2=0x7f88c00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 0x7f88c00b2080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x564b93648860 tx=0x7f88cc004a40 comp rx=0 tx=0).stop 2024-04-25T13:46:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 msgr2=0x7f88d419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.959+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f88d000e9f0 tx=0x7f88d000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 shutdown_connections 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afb70 0x7f88c00b2080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d41050b0 0x7f88d41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d4102260 0x7f88d419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 --2- 172.21.15.94:0/3653490755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101760 0x7f88d419a2b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 >> 172.21.15.94:0/3653490755 conn(0x7f88d40fd380 msgr2=0x7f88d4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:44.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 shutdown_connections 2024-04-25T13:46:44.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:44.960+0000 7f88dc784640 1 -- 172.21.15.94:0/3653490755 wait complete. 2024-04-25T13:46:44.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:46:44.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/117187078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:46:45.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 -- 172.21.15.94:0/2554129889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840104460 msgr2=0x7f384010aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 --2- 172.21.15.94:0/2554129889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840104460 0x7f384010aba0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3838009ca0 tx=0x7f383802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 -- 172.21.15.94:0/2554129889 shutdown_connections 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 --2- 172.21.15.94:0/2554129889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840104460 0x7f384010aba0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 --2- 172.21.15.94:0/2554129889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840103920 0x7f3840103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 --2- 172.21.15.94:0/2554129889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840102f40 0x7f3840103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 -- 172.21.15.94:0/2554129889 >> 172.21.15.94:0/2554129889 conn(0x7f38400fd340 msgr2=0x7f38400ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 -- 172.21.15.94:0/2554129889 shutdown_connections 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.086+0000 7f3847a15640 1 -- 172.21.15.94:0/2554129889 wait complete. 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 Processor -- start 2024-04-25T13:46:45.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 -- start start 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840103920 0x7f3840199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840104460 0x7f38401a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e530 con 0x7f3840102f40 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e3b0 con 0x7f3840104460 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3847a15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f384010e6b0 con 0x7f3840103920 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f38457b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f38457b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56458/0 (socket says 172.21.15.94:56458) 2024-04-25T13:46:45.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f38457b1640 1 -- 172.21.15.94:0/273449716 learned_addr learned my addr 172.21.15.94:0/273449716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3844fb0640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840103920 0x7f3840199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.087+0000 7f3845fb2640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840104460 0x7f38401a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 -- 172.21.15.94:0/273449716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840103920 msgr2=0x7f3840199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840103920 0x7f3840199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 -- 172.21.15.94:0/273449716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840104460 msgr2=0x7f38401a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840104460 0x7f38401a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 -- 172.21.15.94:0/273449716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38401a0c20 con 0x7f3840102f40 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38457b1640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9c0 tx=0x7f383c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38367fc640 1 -- 172.21.15.94:0/273449716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c00ccc0 con 0x7f3840102f40 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38367fc640 1 -- 172.21.15.94:0/273449716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c0054d0 con 0x7f3840102f40 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38401a0f10 con 0x7f3840102f40 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f38367fc640 1 -- 172.21.15.94:0/273449716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c010430 con 0x7f3840102f40 2024-04-25T13:46:45.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.088+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a1370 con 0x7f3840102f40 2024-04-25T13:46:45.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 Processor -- start 2024-04-25T13:46:45.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f38367fc640 1 -- 172.21.15.94:0/273449716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f383c005020 con 0x7f3840102f40 2024-04-25T13:46:45.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 -- start start 2024-04-25T13:46:45.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741c5390 0x557074567810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741a46b0 0x557074568a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570744cbd80 con 0x55707419c4d0 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557074485a20 con 0x5570741a46b0 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f384e9a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570744c7d40 con 0x5570741c5390 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f38457b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f38457b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56464/0 (socket says 172.21.15.94:56464) 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f38457b1640 1 -- 172.21.15.94:0/60520630 learned_addr learned my addr 172.21.15.94:0/60520630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f3844fb0640 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741a46b0 0x557074568a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.091+0000 7f3845fb2640 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741c5390 0x557074567810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 -- 172.21.15.94:0/60520630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741c5390 msgr2=0x557074567810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741c5390 0x557074567810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38367fc640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 0x7f382c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 -- 172.21.15.94:0/60520630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741a46b0 msgr2=0x557074568a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741a46b0 0x557074568a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 -- 172.21.15.94:0/60520630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557074490c30 con 0x55707419c4d0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f3845fb2640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 0x7f382c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38367fc640 1 -- 172.21.15.94:0/273449716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f383c0def70 con 0x7f3840102f40 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f38457b1640 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f383c0c6970 tx=0x7f383c0a3a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc0a1868c59e7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f382b7fe640 1 -- 172.21.15.94:0/60520630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0a7180 con 0x55707419c4d0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f382b7fe640 1 -- 172.21.15.94:0/60520630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383c005310 con 0x55707419c4d0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.092+0000 7f382b7fe640 1 -- 172.21.15.94:0/60520630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c00ce60 con 0x55707419c4d0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 -- 172.21.15.94:0/60520630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 msgr2=0x557074567d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f3845fb2640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 0x7f382c0b2080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557074568860 tx=0x7f3838009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f383c0c6970 tx=0x7f383c0a3a00 comp rx=0 tx=0).stop 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 -- 172.21.15.94:0/60520630 shutdown_connections 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741a46b0 0x557074568a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55707419c4d0 0x557074567d90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 --2- 172.21.15.94:0/60520630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741c5390 0x557074567810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 -- 172.21.15.94:0/60520630 >> 172.21.15.94:0/60520630 conn(0x5570741b94e0 msgr2=0x557074563020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 -- 172.21.15.94:0/60520630 shutdown_connections 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.093+0000 7f384e9a6740 1 -- 172.21.15.94:0/60520630 wait complete. 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 -- start start 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 Processor -- start 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x557074571560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074571aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557074569530 con 0x5570741c5390 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557074452dc0 con 0x55707419c4d0 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570745696b0 con 0x5570741a46b0 2024-04-25T13:46:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f384e9a6740 10 client.0 _ll_register_callbacks cb 0x55707446e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f3845fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f3844fb0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074571aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f38457b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x557074571560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f3845fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56466/0 (socket says 172.21.15.94:56466) 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.094+0000 7f3845fb2640 1 -- 172.21.15.94:0/660839142 learned_addr learned my addr 172.21.15.94:0/660839142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 -- 172.21.15.94:0/660839142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 msgr2=0x557074571aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074571aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 -- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 msgr2=0x557074571560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x557074571560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 -- 172.21.15.94:0/660839142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570741dd3f0 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f38457b1640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x557074571560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3845fb2640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3838035000 tx=0x7f383802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c0682dbeef05db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3828ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004250 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f38380043f0 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570744d9990 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38380045f0 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f384e9a6740 10 client.17340 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570742a77d0 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.095+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5570744d9990 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3838051020 con 0x5570741c5390 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 10 client.17340 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 20 client.17340 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557074183f60 0x55707457f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 4 client.17340 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570744d3800 con 0x557074183f60 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f38457b1640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557074183f60 0x55707457f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f3828ff9640 10 client.17340 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.096+0000 7f38457b1640 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557074183f60 0x55707457f1f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8435c2efba820e8 server_cookie=23ad6a5e8d8fc2b6 in_seq=0 out_seq=0 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f3828ff9640 1 -- 172.21.15.94:0/660839142 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570744d3800 con 0x557074183f60 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f3828ff9640 10 client.17340 handle_command_reply: tid=0 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 client.17340 shutdown 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 msgr2=0x557074578a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3838035000 tx=0x7f383802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 shutdown_connections 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557074183f60 0x55707457f1f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x557074578a00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074571aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 --2- 172.21.15.94:0/660839142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x557074571560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f3828ff9640 0 client.17340 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.097+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 >> 172.21.15.94:0/660839142 conn(0x5570741b94e0 msgr2=0x557074565f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.098+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 shutdown_connections 2024-04-25T13:46:45.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.098+0000 7f384e9a6740 1 -- 172.21.15.94:0/660839142 wait complete. 2024-04-25T13:46:45.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.098+0000 7f384e9a6740 20 client.17340 trim_cache size 0 max 16384 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 Processor -- start 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 -- start start 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x5570745703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074578920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aa420 con 0x5570741c5390 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f38401abf60 con 0x55707419c4d0 2024-04-25T13:46:45.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.111+0000 7f384e9a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aaaf0 con 0x5570741a46b0 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56470/0 (socket says 172.21.15.94:56470) 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 -- 172.21.15.94:0/2264293859 learned_addr learned my addr 172.21.15.94:0/2264293859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f38457b1640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x5570745703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3844fb0640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074578920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 -- 172.21.15.94:0/2264293859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 msgr2=0x557074578920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074578920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 -- 172.21.15.94:0/2264293859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 msgr2=0x5570745703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x5570745703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 -- 172.21.15.94:0/2264293859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557074578e60 con 0x5570741c5390 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3844fb0640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074578920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f38457b1640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x5570745703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f3845fb2640 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3838006010 tx=0x7f3838030bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b7c68b71b970d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f382affd640 1 -- 172.21.15.94:0/2264293859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838061070 con 0x5570741c5390 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f382affd640 1 -- 172.21.15.94:0/2264293859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838036050 con 0x5570741c5390 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.112+0000 7f382affd640 1 -- 172.21.15.94:0/2264293859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383802fb10 con 0x5570741c5390 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- 172.21.15.94:0/2264293859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 msgr2=0x5570745627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3838006010 tx=0x7f3838030bb0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- 172.21.15.94:0/2264293859 shutdown_connections 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741c5390 0x5570745627c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570741a46b0 0x557074578920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 --2- 172.21.15.94:0/2264293859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55707419c4d0 0x5570745703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- 172.21.15.94:0/2264293859 >> 172.21.15.94:0/2264293859 conn(0x5570741b94e0 msgr2=0x55707456fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- 172.21.15.94:0/2264293859 shutdown_connections 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- 172.21.15.94:0/2264293859 wait complete. 2024-04-25T13:46:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.113+0000 7f384e9a6740 1 -- start start 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 Processor -- start 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55707419c4d0 0x557074575bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741c5390 0x557074577020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ab890 con 0x5570741a46b0 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ab1c0 con 0x5570741c5390 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38401a8400 con 0x55707419c4d0 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f384e9a6740 10 client.0 _ll_register_callbacks cb 0x7f3838009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56474/0 (socket says 172.21.15.94:56474) 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 -- 172.21.15.94:0/3451251402 learned_addr learned my addr 172.21.15.94:0/3451251402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f38457b1640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55707419c4d0 0x557074575bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3845fb2640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741c5390 0x557074577020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 -- 172.21.15.94:0/3451251402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55707419c4d0 msgr2=0x557074575bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55707419c4d0 0x557074575bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 -- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741c5390 msgr2=0x557074577020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741c5390 0x557074577020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.114+0000 7f3844fb0640 1 -- 172.21.15.94:0/3451251402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570745775e0 con 0x5570741a46b0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3844fb0640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f38300107f0 tx=0x7f3830010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294c486ff9238bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830023060 con 0x5570741a46b0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3830004460 con 0x5570741a46b0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3838036050 con 0x5570741a46b0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f383000f920 con 0x5570741a46b0 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 10 client.17346 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570745788d0 con 0x5570741a46b0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3838036050 con 0x5570741a46b0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f383001c370 con 0x5570741a46b0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 10 client.17346 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 20 client.17346 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570741593a0 0x5570745eba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 4 client.17346 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55707444e610 con 0x5570741593a0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.115+0000 7f38457b1640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570741593a0 0x5570745eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.116+0000 7f3828ff9640 10 client.17346 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.116+0000 7f38457b1640 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570741593a0 0x5570745eba90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a25958c4a6689c6 server_cookie=36b116d56de0959d in_seq=0 out_seq=0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f3828ff9640 1 -- 172.21.15.94:0/3451251402 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55707444e610 con 0x5570741593a0 2024-04-25T13:46:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f3828ff9640 10 client.17346 handle_command_reply: tid=0 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 client.17346 shutdown 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 msgr2=0x5570745760f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f38300107f0 tx=0x7f3830010cb0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 shutdown_connections 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570741593a0 0x5570745eba90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570741c5390 0x557074577020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570741a46b0 0x5570745760f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f3828ff9640 0 client.17346 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 --2- 172.21.15.94:0/3451251402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55707419c4d0 0x557074575bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.117+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 >> 172.21.15.94:0/3451251402 conn(0x5570741b94e0 msgr2=0x557074561610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 shutdown_connections 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f384e9a6740 1 -- 172.21.15.94:0/3451251402 wait complete. 2024-04-25T13:46:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f384e9a6740 20 client.17346 trim_cache size 0 max 16384 2024-04-25T13:46:45.127 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:46:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 msgr2=0x7f382c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 0x7f382c0b2080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x557074568860 tx=0x7f3838009380 comp rx=0 tx=0).stop 2024-04-25T13:46:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 msgr2=0x7f38401990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e9c0 tx=0x7f383c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 shutdown_connections 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f382c0afb70 0x7f382c0b2080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3840104460 0x7f38401a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3840103920 0x7f3840199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 --2- 172.21.15.94:0/273449716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3840102f40 0x7f38401990e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 >> 172.21.15.94:0/273449716 conn(0x7f38400fd340 msgr2=0x7f3840106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 shutdown_connections 2024-04-25T13:46:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:45.118+0000 7f3847a15640 1 -- 172.21.15.94:0/273449716 wait complete. 2024-04-25T13:46:45.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 524355 13.35 MB/sec execute 88 sec latency 12.597 ms 2024-04-25T13:46:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:45 smithi137 ceph-mon[32568]: pgmap v496: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-04-25T13:46:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:45 smithi094 ceph-mon[29195]: pgmap v496: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-04-25T13:46:45.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:45 smithi138 ceph-mon[32977]: pgmap v496: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-04-25T13:46:46.130 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:46:46.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:46.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:46:46.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 -- 172.21.15.94:0/1412520432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c108920 msgr2=0x7f4d7c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 --2- 172.21.15.94:0/1412520432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c108920 0x7f4d7c10ad40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009ca0 tx=0x7f4d7402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 -- 172.21.15.94:0/1412520432 shutdown_connections 2024-04-25T13:46:46.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 --2- 172.21.15.94:0/1412520432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c108920 0x7f4d7c10ad40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 --2- 172.21.15.94:0/1412520432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c103c20 0x7f4d7c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 --2- 172.21.15.94:0/1412520432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 -- 172.21.15.94:0/1412520432 >> 172.21.15.94:0/1412520432 conn(0x7f4d7c0fdbd0 msgr2=0x7f4d7c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 -- 172.21.15.94:0/1412520432 shutdown_connections 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.249+0000 7f4d836ee640 1 -- 172.21.15.94:0/1412520432 wait complete. 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 Processor -- start 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 -- start start 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c103c20 0x7f4d7c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c108920 0x7f4d7c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10e510 con 0x7f4d7c1029e0 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10e390 con 0x7f4d7c108920 2024-04-25T13:46:46.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d836ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c10e690 con 0x7f4d7c103c20 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d8148a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d8148a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56480/0 (socket says 172.21.15.94:56480) 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d8148a640 1 -- 172.21.15.94:0/752480240 learned_addr learned my addr 172.21.15.94:0/752480240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d80c89640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c103c20 0x7f4d7c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.250+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c108920 0x7f4d7c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 -- 172.21.15.94:0/752480240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c103c20 msgr2=0x7f4d7c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c103c20 0x7f4d7c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 -- 172.21.15.94:0/752480240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c108920 msgr2=0x7f4d7c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c108920 0x7f4d7c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 -- 172.21.15.94:0/752480240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c1a2030 con 0x7f4d7c1029e0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d8148a640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d727fc640 1 -- 172.21.15.94:0/752480240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800ccc0 con 0x7f4d7c1029e0 2024-04-25T13:46:46.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d727fc640 1 -- 172.21.15.94:0/752480240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d780035f0 con 0x7f4d7c1029e0 2024-04-25T13:46:46.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c1a2320 con 0x7f4d7c1029e0 2024-04-25T13:46:46.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d727fc640 1 -- 172.21.15.94:0/752480240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800ce60 con 0x7f4d7c1029e0 2024-04-25T13:46:46.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.251+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c1a2820 con 0x7f4d7c1029e0 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 Processor -- start 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 -- start start 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d727fc640 1 -- 172.21.15.94:0/752480240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4d78003aa0 con 0x7f4d7c1029e0 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562638574550 0x56263893ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263857c730 0x562638945920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5626388a3b00 con 0x56263857c730 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56263883f920 con 0x56263859d410 2024-04-25T13:46:46.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.254+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56263889fac0 con 0x562638574550 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d80c89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263857c730 0x562638945920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53638/0 (socket says 172.21.15.94:53638) 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2933429185 learned_addr learned my addr 172.21.15.94:0/2933429185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d8148a640 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562638574550 0x56263893ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2933429185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562638574550 msgr2=0x56263893ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562638574550 0x56263893ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2933429185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263857c730 msgr2=0x562638945920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263857c730 0x562638945920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2933429185 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56263867d7e0 con 0x56263859d410 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d727fc640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 0x7f4d680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d74035000 tx=0x7f4d7402f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793a6383ebcf14c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d8148a640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 0x7f4d680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d677fe640 1 -- 172.21.15.94:0/2933429185 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74031650 con 0x56263859d410 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d727fc640 1 -- 172.21.15.94:0/752480240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d780deff0 con 0x7f4d7c1029e0 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d677fe640 1 -- 172.21.15.94:0/2933429185 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d74036030 con 0x56263859d410 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.255+0000 7f4d677fe640 1 -- 172.21.15.94:0/2933429185 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004690 con 0x56263859d410 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2933429185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 msgr2=0x56263893e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8148a640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 0x7f4d680b2000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56263893f880 tx=0x7f4d78003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d74035000 tx=0x7f4d7402f410 comp rx=0 tx=0).stop 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2933429185 shutdown_connections 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263857c730 0x562638945920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562638574550 0x56263893ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2933429185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56263859d410 0x56263893e4a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2933429185 >> 172.21.15.94:0/2933429185 conn(0x562638591560 msgr2=0x562638939e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2933429185 shutdown_connections 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.256+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2933429185 wait complete. 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 -- start start 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 Processor -- start 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x5626389480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x5626389485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562638946190 con 0x56263859d410 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56263883d5d0 con 0x562638574550 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562638946310 con 0x56263857c730 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d8a5a0740 10 client.0 _ll_register_callbacks cb 0x56263882b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:46.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56494/0 (socket says 172.21.15.94:56494) 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.257+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2435587697 learned_addr learned my addr 172.21.15.94:0/2435587697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x5626389485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x5626389480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 -- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 msgr2=0x5626389480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x5626389480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 -- 172.21.15.94:0/2435587697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 msgr2=0x56263894f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 -- 172.21.15.94:0/2435587697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626386ddf70 con 0x56263857c730 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d80c89640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x5626389485f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06a2c0 tx=0x7f4d6c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e382595d793e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:46:46.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00f8c0 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d6c00fa60 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d74036030 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d6c00fc60 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d8a5a0740 10 client.26725 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.258+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562638575170 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d74036030 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d6c010430 con 0x56263857c730 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 10 client.26725 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 20 client.26725 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56263855c1f0 0x562638955a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 4 client.26725 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:46.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626388b1710 con 0x56263855c1f0 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56263855c1f0 0x562638955a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d64ff9640 10 client.26725 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.259+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56263855c1f0 0x562638955a40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac92ca28ffe4290f server_cookie=841d45f8ab46b664 in_seq=0 out_seq=0 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d64ff9640 1 -- 172.21.15.94:0/2435587697 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626388b1710 con 0x56263855c1f0 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d64ff9640 10 client.26725 handle_command_reply: tid=0 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 client.26725 shutdown 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 msgr2=0x5626389485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x5626389485f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06a2c0 tx=0x7f4d6c012b60 comp rx=0 tx=0).stop 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 shutdown_connections 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56263855c1f0 0x562638955a40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263894f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x5626389485f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2435587697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x5626389480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d64ff9640 0 client.26725 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 >> 172.21.15.94:0/2435587697 conn(0x562638591560 msgr2=0x562638938ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 shutdown_connections 2024-04-25T13:46:46.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2435587697 wait complete. 2024-04-25T13:46:46.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.261+0000 7f4d8a5a0740 20 client.26725 trim_cache size 0 max 16384 2024-04-25T13:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.280+0000 7f4d8a5a0740 1 Processor -- start 2024-04-25T13:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.280+0000 7f4d8a5a0740 1 -- start start 2024-04-25T13:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x562638942d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263893d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1ab4d0 con 0x56263859d410 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1ad010 con 0x562638574550 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1abba0 con 0x56263857c730 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56504/0 (socket says 172.21.15.94:56504) 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2607658086 learned_addr learned my addr 172.21.15.94:0/2607658086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d80c89640 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263893d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x562638942d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2607658086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 msgr2=0x56263893d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263893d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2607658086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 msgr2=0x562638942d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x562638942d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 -- 172.21.15.94:0/2607658086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562638945a70 con 0x56263859d410 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4d74006010 tx=0x7f4d7402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597aaaff22f78ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d66ffd640 1 -- 172.21.15.94:0/2607658086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740317e0 con 0x56263859d410 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d66ffd640 1 -- 172.21.15.94:0/2607658086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d74036070 con 0x56263859d410 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.281+0000 7f4d66ffd640 1 -- 172.21.15.94:0/2607658086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7402f950 con 0x56263859d410 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2607658086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 msgr2=0x56263893d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4d74006010 tx=0x7f4d7402f3a0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2607658086 shutdown_connections 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x56263893d880 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263893d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/2607658086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x562638942d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2607658086 >> 172.21.15.94:0/2607658086 conn(0x562638591560 msgr2=0x56263893cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2607658086 shutdown_connections 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.282+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/2607658086 wait complete. 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 -- start start 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 Processor -- start 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x56263894cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263894d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1ac940 con 0x56263859d410 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1ac270 con 0x562638574550 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c1a94b0 con 0x56263857c730 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d8a5a0740 10 client.0 _ll_register_callbacks cb 0x7f4d6c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d81c8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56514/0 (socket says 172.21.15.94:56514) 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d81c8b640 1 -- 172.21.15.94:0/4203219084 learned_addr learned my addr 172.21.15.94:0/4203219084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x56263894cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.283+0000 7f4d80c89640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263894d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 -- 172.21.15.94:0/4203219084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 msgr2=0x56263894d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263894d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 -- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 msgr2=0x56263894cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x56263894cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 -- 172.21.15.94:0/4203219084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626389bc320 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d81c8b640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4d74002230 tx=0x7f4d7402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b4f2b24c80866c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74003b70 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d74003d10 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d74036070 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d7402f6d0 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 10 client.17364 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626389bd420 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d74036070 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d74043070 con 0x56263859d410 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 10 client.17364 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 20 client.17364 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562638531630 0x5626389bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 4 client.17364 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.284+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5626388ab580 con 0x562638531630 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.285+0000 7f4d80c89640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562638531630 0x5626389bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.285+0000 7f4d64ff9640 10 client.17364 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.285+0000 7f4d80c89640 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562638531630 0x5626389bfaf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fec8a35c519f802 server_cookie=52cd2b586c7e2c9d in_seq=0 out_seq=0 2024-04-25T13:46:46.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d64ff9640 1 -- 172.21.15.94:0/4203219084 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5626388ab580 con 0x562638531630 2024-04-25T13:46:46.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d64ff9640 10 client.17364 handle_command_reply: tid=0 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 client.17364 shutdown 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 msgr2=0x5626389bbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4d74002230 tx=0x7f4d7402b040 comp rx=0 tx=0).stop 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 shutdown_connections 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562638531630 0x5626389bfaf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56263859d410 0x5626389bbde0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56263857c730 0x56263894d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 --2- 172.21.15.94:0/4203219084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562638574550 0x56263894cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 >> 172.21.15.94:0/4203219084 conn(0x562638591560 msgr2=0x56263893d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d64ff9640 0 client.17364 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 shutdown_connections 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 1 -- 172.21.15.94:0/4203219084 wait complete. 2024-04-25T13:46:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.287+0000 7f4d8a5a0740 20 client.17364 trim_cache size 0 max 16384 2024-04-25T13:46:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:46:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b14a1012-0008-4e65-9cf9-2d85f2c6a2e8", 2024-04-25T13:46:46.293 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:46:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 msgr2=0x7f4d680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 0x7f4d680b2000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56263893f880 tx=0x7f4d78003a10 comp rx=0 tx=0).stop 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 msgr2=0x7f4d7c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 shutdown_connections 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d680afaf0 0x7f4d680b2000 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d7c108920 0x7f4d7c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7c103c20 0x7f4d7c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 --2- 172.21.15.94:0/752480240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d7c1029e0 0x7f4d7c06a920 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 >> 172.21.15.94:0/752480240 conn(0x7f4d7c0fdbd0 msgr2=0x7f4d7c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 shutdown_connections 2024-04-25T13:46:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.288+0000 7f4d836ee640 1 -- 172.21.15.94:0/752480240 wait complete. 2024-04-25T13:46:46.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:46:46.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:46:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 -- 172.21.15.94:0/847687961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 msgr2=0x7f52b8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 --2- 172.21.15.94:0/847687961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8102e70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f52b400aa10 tx=0x7f52b402b720 comp rx=0 tx=0).stop 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 -- 172.21.15.94:0/847687961 shutdown_connections 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 --2- 172.21.15.94:0/847687961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 0x7f52b810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 --2- 172.21.15.94:0/847687961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 0x7f52b81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 --2- 172.21.15.94:0/847687961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8102e70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 -- 172.21.15.94:0/847687961 >> 172.21.15.94:0/847687961 conn(0x7f52b80fd340 msgr2=0x7f52b80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 -- 172.21.15.94:0/847687961 shutdown_connections 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.412+0000 7f52bff5f640 1 -- 172.21.15.94:0/847687961 wait complete. 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 Processor -- start 2024-04-25T13:46:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 -- start start 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 0x7f52b8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 0x7f52b819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52b810d400 con 0x7f52b8102170 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52b810d280 con 0x7f52b81080b0 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bff5f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52b810d580 con 0x7f52b81033b0 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56540/0 (socket says 172.21.15.94:56540) 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bdcfb640 1 -- 172.21.15.94:0/673840105 learned_addr learned my addr 172.21.15.94:0/673840105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52be4fc640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 0x7f52b819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.413+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 0x7f52b8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 -- 172.21.15.94:0/673840105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 msgr2=0x7f52b8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 0x7f52b8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 -- 172.21.15.94:0/673840105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 msgr2=0x7f52b819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 0x7f52b819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 -- 172.21.15.94:0/673840105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819fb90 con 0x7f52b8102170 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006010 tx=0x7f52b4030b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52aeffd640 1 -- 172.21.15.94:0/673840105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4040070 con 0x7f52b8102170 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52aeffd640 1 -- 172.21.15.94:0/673840105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b4003ca0 con 0x7f52b8102170 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819fe20 con 0x7f52b8102170 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52aeffd640 1 -- 172.21.15.94:0/673840105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4036030 con 0x7f52b8102170 2024-04-25T13:46:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.414+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a0220 con 0x7f52b8102170 2024-04-25T13:46:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.416+0000 7f52aeffd640 1 -- 172.21.15.94:0/673840105 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f52b4043070 con 0x7f52b8102170 2024-04-25T13:46:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 Processor -- start 2024-04-25T13:46:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 -- start start 2024-04-25T13:46:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a571390 0x55869a912a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a913c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55869a877d80 con 0x55869a5484d0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55869a8318b0 con 0x55869a5506b0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55869a873d40 con 0x55869a571390 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56554/0 (socket says 172.21.15.94:56554) 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52bdcfb640 1 -- 172.21.15.94:0/1637134191 learned_addr learned my addr 172.21.15.94:0/1637134191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52be4fc640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a571390 0x55869a912a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52aeffd640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 0x7f52a40b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a913c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.417+0000 7f52be4fc640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 0x7f52a40b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52aeffd640 1 -- 172.21.15.94:0/673840105 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f52b4100720 con 0x7f52b8102170 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 -- 172.21.15.94:0/1637134191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a571390 msgr2=0x55869a912a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a571390 0x55869a912a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 -- 172.21.15.94:0/1637134191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 msgr2=0x55869a912fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 -- 172.21.15.94:0/1637134191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a802a40 con 0x55869a5506b0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a913c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009bd0 tx=0x7f52a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71f085e50449670 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52be4fc640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 0x7f52a40b4210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009fd0 tx=0x7f52b0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52a3fff640 1 -- 172.21.15.94:0/1637134191 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8004230 con 0x55869a5506b0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52a3fff640 1 -- 172.21.15.94:0/1637134191 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a8009e40 con 0x55869a5506b0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.418+0000 7f52a3fff640 1 -- 172.21.15.94:0/1637134191 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8004b90 con 0x55869a5506b0 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- 172.21.15.94:0/1637134191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 msgr2=0x55869a913c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a913c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009bd0 tx=0x7f52a8011810 comp rx=0 tx=0).stop 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- 172.21.15.94:0/1637134191 shutdown_connections 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a913c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5484d0 0x55869a912fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/1637134191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a571390 0x55869a912a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- 172.21.15.94:0/1637134191 >> 172.21.15.94:0/1637134191 conn(0x55869a5654e0 msgr2=0x55869a90e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- 172.21.15.94:0/1637134191 shutdown_connections 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- 172.21.15.94:0/1637134191 wait complete. 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 -- start start 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.419+0000 7f52c6e0b740 1 Processor -- start 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a91c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a571390 0x55869a923d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55869a914780 con 0x55869a5506b0 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55869a7fed10 con 0x55869a571390 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55869a914900 con 0x55869a5484d0 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52c6e0b740 10 client.0 _ll_register_callbacks cb 0x55869a82b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:46.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56570/0 (socket says 172.21.15.94:56570) 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 -- 172.21.15.94:0/619732772 learned_addr learned my addr 172.21.15.94:0/619732772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52be4fc640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a571390 0x55869a923d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a91c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 -- 172.21.15.94:0/619732772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 msgr2=0x55869a91c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a91c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 -- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a571390 msgr2=0x55869a923d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a571390 0x55869a923d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 -- 172.21.15.94:0/619732772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869a6537d0 con 0x55869a5506b0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f52a8019af0 tx=0x7f52a8019b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d7bbc8b5768859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.420+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8028020 con 0x55869a5506b0 2024-04-25T13:46:46.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f52a8024470 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55869a885990 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52a8033050 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 10 client.17376 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55869a5893f0 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55869a885990 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f52a8018070 con 0x55869a5506b0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 20 client.17376 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a52ff60 0x55869a92a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 4 client.17376 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.421+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869a87f800 con 0x55869a52ff60 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.422+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a52ff60 0x55869a92a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.422+0000 7f52a17fa640 10 client.17376 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.422+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a52ff60 0x55869a92a0b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c624643b65d285c9 server_cookie=b54cbf592e1ce388 in_seq=0 out_seq=0 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52a17fa640 1 -- 172.21.15.94:0/619732772 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55869a87f800 con 0x55869a52ff60 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52a17fa640 10 client.17376 handle_command_reply: tid=0 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 client.17376 shutdown 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 msgr2=0x55869a91ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f52a8019af0 tx=0x7f52a8019b20 comp rx=0 tx=0).stop 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 shutdown_connections 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a52ff60 0x55869a92a0b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a571390 0x55869a923d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a5506b0 0x55869a91ce20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52a17fa640 0 client.17376 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/619732772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a91c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 >> 172.21.15.94:0/619732772 conn(0x55869a5654e0 msgr2=0x55869a90d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 shutdown_connections 2024-04-25T13:46:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 1 -- 172.21.15.94:0/619732772 wait complete. 2024-04-25T13:46:46.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.423+0000 7f52c6e0b740 20 client.17376 trim_cache size 0 max 16384 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 Processor -- start 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 -- start start 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5484d0 0x55869a911780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5506b0 0x55869a92f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81a92c0 con 0x55869a52ff60 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aae00 con 0x55869a5484d0 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81a9990 con 0x55869a5506b0 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56580/0 (socket says 172.21.15.94:56580) 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52bdcfb640 1 -- 172.21.15.94:0/656760300 learned_addr learned my addr 172.21.15.94:0/656760300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5484d0 0x55869a911780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.441+0000 7f52be4fc640 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5506b0 0x55869a92f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 -- 172.21.15.94:0/656760300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5506b0 msgr2=0x55869a92f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5506b0 0x55869a92f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 -- 172.21.15.94:0/656760300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5484d0 msgr2=0x55869a911780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5484d0 0x55869a911780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 -- 172.21.15.94:0/656760300 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a5490f0 con 0x55869a52ff60 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b40fe050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364b76aa4fae704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52a37fe640 1 -- 172.21.15.94:0/656760300 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b404a040 con 0x55869a52ff60 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52a37fe640 1 -- 172.21.15.94:0/656760300 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b40d1300 con 0x55869a52ff60 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.442+0000 7f52a37fe640 1 -- 172.21.15.94:0/656760300 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40d19c0 con 0x55869a52ff60 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- 172.21.15.94:0/656760300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 msgr2=0x55869a91b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b40fe050 comp rx=0 tx=0).stop 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- 172.21.15.94:0/656760300 shutdown_connections 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5506b0 0x55869a92f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5484d0 0x55869a911780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/656760300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a91b930 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- 172.21.15.94:0/656760300 >> 172.21.15.94:0/656760300 conn(0x55869a5654e0 msgr2=0x55869a911070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- 172.21.15.94:0/656760300 shutdown_connections 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- 172.21.15.94:0/656760300 wait complete. 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 -- start start 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.443+0000 7f52c6e0b740 1 Processor -- start 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a9243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a920af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aa730 con 0x55869a52ff60 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aa060 con 0x55869a5506b0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81a72a0 con 0x55869a5484d0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52c6e0b740 10 client.0 _ll_register_callbacks cb 0x7f52b40c52e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56594/0 (socket says 172.21.15.94:56594) 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a9243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 -- 172.21.15.94:0/2809724807 learned_addr learned my addr 172.21.15.94:0/2809724807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52be4fc640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a920af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 -- 172.21.15.94:0/2809724807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 msgr2=0x55869a9243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a9243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 -- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 msgr2=0x55869a920af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a920af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 -- 172.21.15.94:0/2809724807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869a8384b0 con 0x55869a52ff60 2024-04-25T13:46:46.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a9243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52bdcfb640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005f70 tx=0x7f52b4030510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d6460cb5ccb510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.444+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b404a070 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f52b403c560 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55869a7fa4a0 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f52b40c8380 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 10 client.17382 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55869a990f30 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55869a7fa4a0 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f52b400b040 con 0x55869a52ff60 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 10 client.17382 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 20 client.17382 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:46.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a571390 0x55869a993740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 4 client.17382 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:46:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55869a848240 con 0x55869a571390 2024-04-25T13:46:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.445+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a571390 0x55869a993740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.446+0000 7f52a17fa640 10 client.17382 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.446+0000 7f52bd4fa640 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a571390 0x55869a993740 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed7d3127b6734ca server_cookie=c541b4ec24966b0b in_seq=0 out_seq=0 2024-04-25T13:46:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52a17fa640 1 -- 172.21.15.94:0/2809724807 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55869a848240 con 0x55869a571390 2024-04-25T13:46:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52a17fa640 10 client.17382 handle_command_reply: tid=0 2024-04-25T13:46:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 client.17382 shutdown 2024-04-25T13:46:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 msgr2=0x55869a923eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005f70 tx=0x7f52b4030510 comp rx=0 tx=0).stop 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 shutdown_connections 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55869a571390 0x55869a993740 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55869a5506b0 0x55869a920af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55869a5484d0 0x55869a9243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 --2- 172.21.15.94:0/2809724807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55869a52ff60 0x55869a923eb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52a17fa640 0 client.17382 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 >> 172.21.15.94:0/2809724807 conn(0x55869a5654e0 msgr2=0x55869a911aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 shutdown_connections 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 1 -- 172.21.15.94:0/2809724807 wait complete. 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52c6e0b740 20 client.17382 trim_cache size 0 max 16384 2024-04-25T13:46:46.452 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "b14a1012-0008-4e65-9cf9-2d85f2c6a2e8": { 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b14a1012-0008-4e65-9cf9-2d85f2c6a2e8", 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:46:46.453 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 msgr2=0x7f52a40b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 0x7f52a40b4210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009fd0 tx=0x7f52b0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 msgr2=0x7f52b8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.448+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006010 tx=0x7f52b4030b60 comp rx=0 tx=0).stop 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 shutdown_connections 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52a40b1d00 0x7f52a40b4210 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b81080b0 0x7f52b819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b81033b0 0x7f52b8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 --2- 172.21.15.94:0/673840105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52b8102170 0x7f52b8198050 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 >> 172.21.15.94:0/673840105 conn(0x7f52b80fd340 msgr2=0x7f52b8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 shutdown_connections 2024-04-25T13:46:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:46:46.449+0000 7f52bff5f640 1 -- 172.21.15.94:0/673840105 wait complete. 2024-04-25T13:46:46.459 INFO:tasks.cephfs.filesystem:scrub status for tag:b14a1012-0008-4e65-9cf9-2d85f2c6a2e8 - {'path': '/', 'tag': 'b14a1012-0008-4e65-9cf9-2d85f2c6a2e8', 'options': 'recursive,force'} 2024-04-25T13:46:46.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 526708 13.36 MB/sec execute 89 sec latency 12.677 ms 2024-04-25T13:46:47.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 529632 13.41 MB/sec execute 90 sec latency 65.585 ms 2024-04-25T13:46:47.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:47 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:46:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:46:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:47 smithi137 ceph-mon[32568]: pgmap v497: 97 pgs: 97 active+clean; 22 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.3 MiB/s wr, 389 op/s 2024-04-25T13:46:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:47 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:46:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:46:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:47 smithi094 ceph-mon[29195]: pgmap v497: 97 pgs: 97 active+clean; 22 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.3 MiB/s wr, 389 op/s 2024-04-25T13:46:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:47 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:46:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:46:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:46:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:47 smithi138 ceph-mon[32977]: pgmap v497: 97 pgs: 97 active+clean; 22 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 3.3 MiB/s wr, 389 op/s 2024-04-25T13:46:48.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 532240 13.40 MB/sec execute 91 sec latency 30.868 ms 2024-04-25T13:46:49.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 534418 13.38 MB/sec execute 92 sec latency 13.439 ms 2024-04-25T13:46:49.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:49 smithi137 ceph-mon[32568]: pgmap v498: 97 pgs: 97 active+clean; 23 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-04-25T13:46:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:49 smithi094 ceph-mon[29195]: pgmap v498: 97 pgs: 97 active+clean; 23 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-04-25T13:46:49.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:49 smithi138 ceph-mon[32977]: pgmap v498: 97 pgs: 97 active+clean; 23 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.3 MiB/s wr, 393 op/s 2024-04-25T13:46:50.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 537097 13.39 MB/sec execute 93 sec latency 13.673 ms 2024-04-25T13:46:51.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 540238 13.45 MB/sec execute 94 sec latency 17.024 ms 2024-04-25T13:46:51.797 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:51 smithi138 ceph-mon[32977]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:46:51.797 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:46:51.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:51 smithi137 ceph-mon[32568]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:46:51.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:46:51.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:51 smithi094 ceph-mon[29195]: pgmap v499: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:46:51.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:46:52.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 542738 13.43 MB/sec execute 95 sec latency 15.736 ms 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 -- 172.21.15.94:0/2124891317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc104d80 msgr2=0x7fa2cc105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/2124891317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc104d80 0x7fa2cc105190 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009900 tx=0x7fa2c002a410 comp rx=0 tx=0).stop 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 -- 172.21.15.94:0/2124891317 shutdown_connections 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/2124891317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc109a80 0x7fa2cc10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/2124891317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc104d80 0x7fa2cc105190 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/2124891317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 0x7fa2cc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 -- 172.21.15.94:0/2124891317 >> 172.21.15.94:0/2124891317 conn(0x7fa2cc0fed30 msgr2=0x7fa2cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 -- 172.21.15.94:0/2124891317 shutdown_connections 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 -- 172.21.15.94:0/2124891317 wait complete. 2024-04-25T13:46:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.233+0000 7fa2d20bd640 1 Processor -- start 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 -- start start 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 0x7fa2cc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc104d80 0x7fa2cc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10f660 con 0x7fa2cc109a80 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10f4e0 con 0x7fa2cc104d80 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2d20bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10f7e0 con 0x7fa2cc103b40 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56608/0 (socket says 172.21.15.94:56608) 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 -- 172.21.15.94:0/1358796614 learned_addr learned my addr 172.21.15.94:0/1358796614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2caffd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc104d80 0x7fa2cc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 0x7fa2cc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 -- 172.21.15.94:0/1358796614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 msgr2=0x7fa2cc19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 0x7fa2cc19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 -- 172.21.15.94:0/1358796614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc104d80 msgr2=0x7fa2cc19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc104d80 0x7fa2cc19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 -- 172.21.15.94:0/1358796614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc1a1d70 con 0x7fa2cc109a80 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2cbfff640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400eb10 tx=0x7fa2c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.234+0000 7fa2c8ff9640 1 -- 172.21.15.94:0/1358796614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4004b20 con 0x7fa2cc109a80 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.235+0000 7fa2c8ff9640 1 -- 172.21.15.94:0/1358796614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c4004cc0 con 0x7fa2cc109a80 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.235+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc1a2060 con 0x7fa2cc109a80 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.235+0000 7fa2c8ff9640 1 -- 172.21.15.94:0/1358796614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4010670 con 0x7fa2cc109a80 2024-04-25T13:46:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.235+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1a2560 con 0x7fa2cc109a80 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 Processor -- start 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2c8ff9640 1 -- 172.21.15.94:0/1358796614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa2c4005020 con 0x7fa2cc109a80 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 -- start start 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271e92f0 0x5569275b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556927518460 con 0x5569272121b0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5569274d2590 con 0x5569271e92f0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556927514420 con 0x5569271f14d0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56616/0 (socket says 172.21.15.94:56616) 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2caffd640 1 -- 172.21.15.94:0/1994384392 learned_addr learned my addr 172.21.15.94:0/1994384392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.237+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271e92f0 0x5569275b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 -- 172.21.15.94:0/1994384392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 msgr2=0x5569275b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2c8ff9640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 0x7fa2b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 -- 172.21.15.94:0/1994384392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271e92f0 msgr2=0x5569275b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271e92f0 0x5569275b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 -- 172.21.15.94:0/1994384392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569274df0d0 con 0x5569272121b0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2cbfff640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 0x7fa2b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2caffd640 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031500 tx=0x7fa2c0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e150e214851590ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2c8ff9640 1 -- 172.21.15.94:0/1358796614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa2c40df280 con 0x7fa2cc109a80 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2b1ffb640 1 -- 172.21.15.94:0/1994384392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0035070 con 0x5569272121b0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2b1ffb640 1 -- 172.21.15.94:0/1994384392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00317a0 con 0x5569272121b0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.238+0000 7fa2b1ffb640 1 -- 172.21.15.94:0/1994384392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003dc0 con 0x5569272121b0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- 172.21.15.94:0/1994384392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 msgr2=0x5569275b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2cbfff640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 0x7fa2b40b1ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40a73c0 tx=0x7fa2c406dbe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031500 tx=0x7fa2c0009560 comp rx=0 tx=0).stop 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- 172.21.15.94:0/1994384392 shutdown_connections 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271e92f0 0x5569275b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/1994384392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275b2970 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- 172.21.15.94:0/1994384392 >> 172.21.15.94:0/1994384392 conn(0x556927206300 msgr2=0x5569275ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- 172.21.15.94:0/1994384392 shutdown_connections 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- 172.21.15.94:0/1994384392 wait complete. 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.239+0000 7fa2d8f73740 1 -- start start 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 Processor -- start 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x5569275c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569275ba7e0 con 0x5569271e92f0 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55692749fb60 con 0x5569272121b0 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569275ba960 con 0x5569271f14d0 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2d8f73740 10 client.0 _ll_register_callbacks cb 0x5569274c42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56632/0 (socket says 172.21.15.94:56632) 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 -- 172.21.15.94:0/3884085382 learned_addr learned my addr 172.21.15.94:0/3884085382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x5569275c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 -- 172.21.15.94:0/3884085382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 msgr2=0x5569275bcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275bcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 -- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 msgr2=0x5569275c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x5569275c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 -- 172.21.15.94:0/3884085382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692722a210 con 0x5569271e92f0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031440 tx=0x7fa2c0009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea6c80b1496d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.240+0000 7fa2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0035030 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa2c0003f50 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c00317a0 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0004150 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 10 client.17394 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569272f6620 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00317a0 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2c003bbe0 con 0x5569271e92f0 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 10 client.17394 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 20 client.17394 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271d0d80 0x5569275c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 4 client.17394 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556927526070 con 0x5569271d0d80 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.241+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271d0d80 0x5569275c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.242+0000 7fa2aaffd640 10 client.17394 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.242+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271d0d80 0x5569275c9ee0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80643540646cff2 server_cookie=2797a24d09e2fd2b in_seq=0 out_seq=0 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3884085382 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556927526070 con 0x5569271d0d80 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2aaffd640 10 client.17394 handle_command_reply: tid=0 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 client.17394 shutdown 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 msgr2=0x5569275bc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031440 tx=0x7fa2c0009630 comp rx=0 tx=0).stop 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 shutdown_connections 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271d0d80 0x5569275c9ee0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x5569275c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2aaffd640 0 client.17394 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x5569275bcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3884085382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x5569275bc5f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 >> 172.21.15.94:0/3884085382 conn(0x556927206300 msgr2=0x5569275b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 shutdown_connections 2024-04-25T13:46:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3884085382 wait complete. 2024-04-25T13:46:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.244+0000 7fa2d8f73740 20 client.17394 trim_cache size 0 max 16384 2024-04-25T13:46:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.259+0000 7fa2d8f73740 1 Processor -- start 2024-04-25T13:46:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.259+0000 7fa2d8f73740 1 -- start start 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271e92f0 0x5569275adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271f14d0 0x5569275c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1ab450 con 0x5569272121b0 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1acf90 con 0x5569271f14d0 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1abb20 con 0x5569271e92f0 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56648/0 (socket says 172.21.15.94:56648) 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 -- 172.21.15.94:0/354439780 learned_addr learned my addr 172.21.15.94:0/354439780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271e92f0 0x5569275adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271f14d0 0x5569275c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 -- 172.21.15.94:0/354439780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271e92f0 msgr2=0x5569275adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271e92f0 0x5569275adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 -- 172.21.15.94:0/354439780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271f14d0 msgr2=0x5569275c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271f14d0 0x5569275c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 -- 172.21.15.94:0/354439780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569275b7500 con 0x5569272121b0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2caffd640 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031b60 tx=0x7fa2c0005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a709ce924f30ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2b17fa640 1 -- 172.21.15.94:0/354439780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0035070 con 0x5569272121b0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2b17fa640 1 -- 172.21.15.94:0/354439780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c0051070 con 0x5569272121b0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.260+0000 7fa2b17fa640 1 -- 172.21.15.94:0/354439780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0061070 con 0x5569272121b0 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 -- 172.21.15.94:0/354439780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 msgr2=0x5569275ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0031b60 tx=0x7fa2c0005950 comp rx=0 tx=0).stop 2024-04-25T13:46:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 -- 172.21.15.94:0/354439780 shutdown_connections 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569272121b0 0x5569275ae190 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569271f14d0 0x5569275c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/354439780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271e92f0 0x5569275adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 -- 172.21.15.94:0/354439780 >> 172.21.15.94:0/354439780 conn(0x556927206300 msgr2=0x5569275c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 -- 172.21.15.94:0/354439780 shutdown_connections 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.261+0000 7fa2d8f73740 1 -- 172.21.15.94:0/354439780 wait complete. 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 -- start start 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 Processor -- start 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x55692762d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x556927634720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1ac8c0 con 0x5569271e92f0 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1ac1f0 con 0x5569272121b0 2024-04-25T13:46:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1a9430 con 0x5569271f14d0 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2d8f73740 10 client.0 _ll_register_callbacks cb 0x7fa2bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56652/0 (socket says 172.21.15.94:56652) 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 -- 172.21.15.94:0/3408404837 learned_addr learned my addr 172.21.15.94:0/3408404837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x55692762d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x556927634720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 -- 172.21.15.94:0/3408404837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 msgr2=0x55692762d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x55692762d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 -- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 msgr2=0x556927634720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.262+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x556927634720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2caffd640 1 -- 172.21.15.94:0/3408404837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556927634ce0 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2caffd640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005f40 tx=0x7fa2c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb9347a9388628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0061070 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa2c004ca40 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c0051070 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0035040 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 10 client.17400 fetch_fsmap learned FSMap version 27 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556927635f70 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c0051070 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2c003b910 con 0x5569271e92f0 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 10 client.17400 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 20 client.17400 populate_metadata read hostname 'smithi094' 2024-04-25T13:46:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271a61c0 0x556927638830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 4 client.17400 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:46:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.263+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55692751fee0 con 0x5569271a61c0 2024-04-25T13:46:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.264+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271a61c0 0x556927638830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.264+0000 7fa2aaffd640 10 client.17400 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.264+0000 7fa2cb7fe640 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271a61c0 0x556927638830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b108a28497b069f server_cookie=d4418612e28fe4b4 in_seq=0 out_seq=0 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2aaffd640 1 -- 172.21.15.94:0/3408404837 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55692751fee0 con 0x5569271a61c0 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2aaffd640 10 client.17400 handle_command_reply: tid=0 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 client.17400 shutdown 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 msgr2=0x55692762d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005f40 tx=0x7fa2c0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 shutdown_connections 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569271a61c0 0x556927638830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569272121b0 0x556927634720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569271f14d0 0x55692762d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 --2- 172.21.15.94:0/3408404837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569271e92f0 0x55692762d280 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2aaffd640 0 client.17400 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 >> 172.21.15.94:0/3408404837 conn(0x556927206300 msgr2=0x5569275b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 shutdown_connections 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 1 -- 172.21.15.94:0/3408404837 wait complete. 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.440+0000 7fa2d8f73740 20 client.17400 trim_cache size 0 max 16384 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:46:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:46:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 msgr2=0x7fa2b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 0x7fa2b40b1ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40a73c0 tx=0x7fa2c406dbe0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 msgr2=0x7fa2cc1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400eb10 tx=0x7fa2c400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 shutdown_connections 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa2b40afae0 0x7fa2b40b1ff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2cc109a80 0x7fa2cc1a1700 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2cc104d80 0x7fa2cc19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 --2- 172.21.15.94:0/1358796614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2cc103b40 0x7fa2cc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 >> 172.21.15.94:0/1358796614 conn(0x7fa2cc0fed30 msgr2=0x7fa2cc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 shutdown_connections 2024-04-25T13:46:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:46:53.441+0000 7fa2d20bd640 1 -- 172.21.15.94:0/1358796614 wait complete. 2024-04-25T13:46:53.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 543913 13.37 MB/sec execute 96 sec latency 90.054 ms 2024-04-25T13:46:53.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:53 smithi137 ceph-mon[32568]: pgmap v500: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-04-25T13:46:53.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:53 smithi094 ceph-mon[29195]: pgmap v500: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-04-25T13:46:53.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:53 smithi138 ceph-mon[32977]: pgmap v500: 97 pgs: 97 active+clean; 27 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-04-25T13:46:54.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 546492 13.36 MB/sec execute 97 sec latency 13.884 ms 2024-04-25T13:46:55.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 548495 13.34 MB/sec execute 98 sec latency 17.635 ms 2024-04-25T13:46:55.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:55 smithi137 ceph-mon[32568]: pgmap v501: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-04-25T13:46:55.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:55 smithi094 ceph-mon[29195]: pgmap v501: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-04-25T13:46:55.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:55 smithi138 ceph-mon[32977]: pgmap v501: 97 pgs: 97 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 500 op/s 2024-04-25T13:46:56.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 551070 13.35 MB/sec execute 99 sec latency 10.452 ms 2024-04-25T13:46:57.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 554141 13.39 MB/sec execute 100 sec latency 21.481 ms 2024-04-25T13:46:57.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:57 smithi137 ceph-mon[32568]: pgmap v502: 97 pgs: 97 active+clean; 40 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-04-25T13:46:57.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:57 smithi094 ceph-mon[29195]: pgmap v502: 97 pgs: 97 active+clean; 40 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-04-25T13:46:57.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:57 smithi138 ceph-mon[32977]: pgmap v502: 97 pgs: 97 active+clean; 40 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-04-25T13:46:58.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 555444 13.30 MB/sec execute 101 sec latency 108.592 ms 2024-04-25T13:46:59.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 558162 13.33 MB/sec execute 102 sec latency 17.266 ms 2024-04-25T13:46:59.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:46:59 smithi137 ceph-mon[32568]: pgmap v503: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-25T13:46:59.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:46:59 smithi094 ceph-mon[29195]: pgmap v503: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-25T13:46:59.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:46:59 smithi138 ceph-mon[32977]: pgmap v503: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-25T13:47:00.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 561080 13.35 MB/sec execute 103 sec latency 17.055 ms 2024-04-25T13:47:01.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 563234 13.31 MB/sec execute 104 sec latency 19.283 ms 2024-04-25T13:47:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:01 smithi137 ceph-mon[32568]: pgmap v504: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 431 op/s 2024-04-25T13:47:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:01 smithi094 ceph-mon[29195]: pgmap v504: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 431 op/s 2024-04-25T13:47:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:01 smithi138 ceph-mon[32977]: pgmap v504: 97 pgs: 97 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 431 op/s 2024-04-25T13:47:02.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 565536 13.32 MB/sec execute 105 sec latency 22.180 ms 2024-04-25T13:47:03.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 568146 13.33 MB/sec execute 106 sec latency 108.190 ms 2024-04-25T13:47:03.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:03 smithi137 ceph-mon[32568]: pgmap v505: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 389 op/s 2024-04-25T13:47:03.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:03 smithi094 ceph-mon[29195]: pgmap v505: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 389 op/s 2024-04-25T13:47:03.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:03 smithi138 ceph-mon[32977]: pgmap v505: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 389 op/s 2024-04-25T13:47:04.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 570234 13.30 MB/sec execute 107 sec latency 13.857 ms 2024-04-25T13:47:05.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 572506 13.30 MB/sec execute 108 sec latency 18.839 ms 2024-04-25T13:47:05.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:05 smithi137 ceph-mon[32568]: pgmap v506: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 458 op/s 2024-04-25T13:47:05.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:05 smithi094 ceph-mon[29195]: pgmap v506: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 458 op/s 2024-04-25T13:47:05.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:05 smithi138 ceph-mon[32977]: pgmap v506: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 458 op/s 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 -- 172.21.15.94:0/114495529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0103c60 msgr2=0x7f2dc0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/114495529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0103c60 0x7f2dc0104070 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009900 tx=0x7f2db002a410 comp rx=0 tx=0).stop 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 -- 172.21.15.94:0/114495529 shutdown_connections 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/114495529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0108960 0x7f2dc010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/114495529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0103c60 0x7f2dc0104070 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/114495529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 0x7f2dc0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 -- 172.21.15.94:0/114495529 >> 172.21.15.94:0/114495529 conn(0x7f2dc00fdc10 msgr2=0x7f2dc01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 -- 172.21.15.94:0/114495529 shutdown_connections 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 -- 172.21.15.94:0/114495529 wait complete. 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.544+0000 7f2dc5de3640 1 Processor -- start 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 -- start start 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 0x7f2dc0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0103c60 0x7f2dc0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010e550 con 0x7f2dc0108960 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010e3d0 con 0x7f2dc0102a20 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dc5de3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010e6d0 con 0x7f2dc0103c60 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56620/0 (socket says 172.21.15.94:56620) 2024-04-25T13:47:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 -- 172.21.15.94:0/2418045832 learned_addr learned my addr 172.21.15.94:0/2418045832 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 0x7f2dc0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0103c60 0x7f2dc0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 -- 172.21.15.94:0/2418045832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0103c60 msgr2=0x7f2dc0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0103c60 0x7f2dc0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 -- 172.21.15.94:0/2418045832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 msgr2=0x7f2dc0199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 0x7f2dc0199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 -- 172.21.15.94:0/2418045832 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc01a0c40 con 0x7f2dc0108960 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.545+0000 7f2dbffff640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2db800eb10 tx=0x7f2db800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.546+0000 7f2dbcff9640 1 -- 172.21.15.94:0/2418045832 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8004b20 con 0x7f2dc0108960 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.546+0000 7f2dbcff9640 1 -- 172.21.15.94:0/2418045832 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db8004cc0 con 0x7f2dc0108960 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.546+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc01a0f30 con 0x7f2dc0108960 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.546+0000 7f2dbcff9640 1 -- 172.21.15.94:0/2418045832 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8010670 con 0x7f2dc0108960 2024-04-25T13:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.546+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc01a13f0 con 0x7f2dc0108960 2024-04-25T13:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.548+0000 7f2dccd6d740 1 Processor -- start 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.548+0000 7f2dbcff9640 1 -- 172.21.15.94:0/2418045832 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2db8010810 con 0x7f2dc0108960 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 -- start start 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa949df460 con 0x55aa946b02f0 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa94999590 con 0x55aa946d91b0 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa949db420 con 0x55aa946b84d0 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a7c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56624/0 (socket says 172.21.15.94:56624) 2024-04-25T13:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbf7fe640 1 -- 172.21.15.94:0/930963968 learned_addr learned my addr 172.21.15.94:0/930963968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbcff9640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 0x7f2da80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 -- 172.21.15.94:0/930963968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 msgr2=0x55aa94a83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 -- 172.21.15.94:0/930963968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 msgr2=0x55aa94a7c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a7c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbeffd640 1 -- 172.21.15.94:0/930963968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa949a60d0 con 0x55aa946d91b0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.549+0000 7f2dbffff640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 0x7f2da80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.550+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a7bb80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005710 tx=0x7f2db0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16cdd13be9908863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.550+0000 7f2dbcff9640 1 -- 172.21.15.94:0/2418045832 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2db80df250 con 0x7f2dc0108960 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.550+0000 7f2da5ffb640 1 -- 172.21.15.94:0/930963968 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0035040 con 0x55aa946d91b0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.550+0000 7f2da5ffb640 1 -- 172.21.15.94:0/930963968 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db0031830 con 0x55aa946d91b0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.550+0000 7f2da5ffb640 1 -- 172.21.15.94:0/930963968 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db00048c0 con 0x55aa946d91b0 2024-04-25T13:47:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 -- 172.21.15.94:0/930963968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 msgr2=0x55aa94a7bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dbffff640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 0x7f2da80b2070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2db80a7620 tx=0x7f2db8072de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a7bb80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005710 tx=0x7f2db0031400 comp rx=0 tx=0).stop 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 -- 172.21.15.94:0/930963968 shutdown_connections 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/930963968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a7bb80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 -- 172.21.15.94:0/930963968 >> 172.21.15.94:0/930963968 conn(0x55aa946cd300 msgr2=0x55aa94a75300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 -- 172.21.15.94:0/930963968 shutdown_connections 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.551+0000 7f2dccd6d740 1 -- 172.21.15.94:0/930963968 wait complete. 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 -- start start 2024-04-25T13:47:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 Processor -- start 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 0x55aa94a85850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 0x55aa94a85d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa94a839f0 con 0x55aa946b84d0 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa94966b60 con 0x55aa946d91b0 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa94a83b70 con 0x55aa946b02f0 2024-04-25T13:47:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dccd6d740 10 client.0 _ll_register_callbacks cb 0x55aa9498b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 0x55aa94a85d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dbf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 0x55aa94a85d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56638/0 (socket says 172.21.15.94:56638) 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 0x55aa94a85850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.552+0000 7f2dbf7fe640 1 -- 172.21.15.94:0/3036825553 learned_addr learned my addr 172.21.15.94:0/3036825553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 -- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 msgr2=0x55aa94a8ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a8ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 -- 172.21.15.94:0/3036825553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 msgr2=0x55aa94a85d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 0x55aa94a85d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 -- 172.21.15.94:0/3036825553 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa946f1210 con 0x55aa946b02f0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 0x55aa94a85850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005710 tx=0x7f2db0005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c032fa0aa4b2fc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2d9effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0035070 con 0x55aa946b02f0 2024-04-25T13:47:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2db0004450 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.553+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0031830 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2db00035f0 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 10 client.26776 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa947bd620 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db0031830 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2db003a040 con 0x55aa946b02f0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 10 client.26776 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 10 client.26776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 20 client.26776 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa94697d80 0x55aa94a93120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 4 client.26776 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.554+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa949ed070 con 0x55aa94697d80 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.555+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa94697d80 0x55aa94a93120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.555+0000 7f2d9effd640 10 client.26776 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.555+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa94697d80 0x55aa94a93120 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bebc153b6143c93 server_cookie=ad2c833356040636 in_seq=0 out_seq=0 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2d9effd640 1 -- 172.21.15.94:0/3036825553 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa949ed070 con 0x55aa94697d80 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2d9effd640 10 client.26776 handle_command_reply: tid=0 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 client.26776 shutdown 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 msgr2=0x55aa94a85850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 0x55aa94a85850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2db0005710 tx=0x7f2db0005f00 comp rx=0 tx=0).stop 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 shutdown_connections 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa94697d80 0x55aa94a93120 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946d91b0 0x55aa94a8ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b84d0 0x55aa94a85d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/3036825553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b02f0 0x55aa94a85850 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2d9effd640 0 client.26776 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 >> 172.21.15.94:0/3036825553 conn(0x55aa946cd300 msgr2=0x55aa94a78210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 shutdown_connections 2024-04-25T13:47:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 1 -- 172.21.15.94:0/3036825553 wait complete. 2024-04-25T13:47:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.557+0000 7f2dccd6d740 20 client.26776 trim_cache size 0 max 16384 2024-04-25T13:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.572+0000 7f2dccd6d740 1 Processor -- start 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.572+0000 7f2dccd6d740 1 -- start start 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b02f0 0x55aa94a78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01a92e0 con 0x55aa946d91b0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01aae20 con 0x55aa946b02f0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01a99b0 con 0x55aa946b84d0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56654/0 (socket says 172.21.15.94:56654) 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1865350971 learned_addr learned my addr 172.21.15.94:0/1865350971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b02f0 0x55aa94a78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1865350971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 msgr2=0x55aa94a8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1865350971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b02f0 msgr2=0x55aa94a78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b02f0 0x55aa94a78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1865350971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa94a806f0 con 0x55aa946d91b0 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2db00096c0 tx=0x7f2db0032ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd86324a748978a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2da57fa640 1 -- 172.21.15.94:0/1865350971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db005a070 con 0x55aa946d91b0 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2da57fa640 1 -- 172.21.15.94:0/1865350971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db005b040 con 0x55aa946d91b0 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.573+0000 7f2da57fa640 1 -- 172.21.15.94:0/1865350971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0035040 con 0x55aa946d91b0 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1865350971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 msgr2=0x55aa94a74c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2db00096c0 tx=0x7f2db0032ab0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1865350971 shutdown_connections 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946d91b0 0x55aa94a74c10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946b84d0 0x55aa94a8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1865350971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b02f0 0x55aa94a78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1865350971 >> 172.21.15.94:0/1865350971 conn(0x55aa946cd300 msgr2=0x55aa94a78100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1865350971 shutdown_connections 2024-04-25T13:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.574+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1865350971 wait complete. 2024-04-25T13:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 -- start start 2024-04-25T13:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 Processor -- start 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b84d0 0x55aa94a8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946d91b0 0x55aa94a8b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01aa750 con 0x55aa946b02f0 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01aa080 con 0x55aa946b84d0 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc01a72c0 con 0x55aa946d91b0 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dccd6d740 10 client.0 _ll_register_callbacks cb 0x7f2db40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56668/0 (socket says 172.21.15.94:56668) 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1966324407 learned_addr learned my addr 172.21.15.94:0/1966324407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b84d0 0x55aa94a8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.575+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946d91b0 0x55aa94a8b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1966324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946d91b0 msgr2=0x55aa94a8b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946d91b0 0x55aa94a8b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b84d0 msgr2=0x55aa94a8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b84d0 0x55aa94a8a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 -- 172.21.15.94:0/1966324407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa94a8bd90 con 0x55aa946b02f0 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dbeffd640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2db002a930 tx=0x7f2db0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504a7905517666c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2d9effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0035030 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2db0003790 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db005b040 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2db005a070 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dccd6d740 10 client.17418 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa94afb360 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.576+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db005b040 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2db003a070 con 0x55aa946b02f0 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 10 client.17418 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 20 client.17418 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9466d1c0 0x55aa94afdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 4 client.17418 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa949e6ee0 con 0x55aa9466d1c0 2024-04-25T13:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9466d1c0 0x55aa94afdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2d9effd640 10 client.17418 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.577+0000 7f2dbf7fe640 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9466d1c0 0x55aa94afdb20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633d23f97916c0a9 server_cookie=926ae5ee8691e42c in_seq=0 out_seq=0 2024-04-25T13:47:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2d9effd640 1 -- 172.21.15.94:0/1966324407 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa949e6ee0 con 0x55aa9466d1c0 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2d9effd640 10 client.17418 handle_command_reply: tid=0 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 client.17418 shutdown 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 msgr2=0x55aa94a8a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2db002a930 tx=0x7f2db0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 shutdown_connections 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9466d1c0 0x55aa94afdb20 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa946d91b0 0x55aa94a8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa946b84d0 0x55aa94a8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2d9effd640 0 client.17418 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 --2- 172.21.15.94:0/1966324407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa946b02f0 0x55aa94a8a360 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 >> 172.21.15.94:0/1966324407 conn(0x55aa946cd300 msgr2=0x55aa94a85420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 shutdown_connections 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 1 -- 172.21.15.94:0/1966324407 wait complete. 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.759+0000 7f2dccd6d740 20 client.17418 trim_cache size 0 max 16384 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 msgr2=0x7f2da80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 0x7f2da80b2070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2db80a7620 tx=0x7f2db8072de0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 msgr2=0x7f2dc01a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2db800eb10 tx=0x7f2db800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 shutdown_connections 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2da80afb60 0x7f2da80b2070 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc0108960 0x7f2dc01a05d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dc0103c60 0x7f2dc0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 --2- 172.21.15.94:0/2418045832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dc0102a20 0x7f2dc0199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 >> 172.21.15.94:0/2418045832 conn(0x7f2dc00fdc10 msgr2=0x7f2dc0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 shutdown_connections 2024-04-25T13:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:06.760+0000 7f2dc5de3640 1 -- 172.21.15.94:0/2418045832 wait complete. 2024-04-25T13:47:06.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 575093 13.30 MB/sec execute 109 sec latency 121.923 ms 2024-04-25T13:47:07.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 577025 13.28 MB/sec execute 110 sec latency 18.477 ms 2024-04-25T13:47:07.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:07 smithi137 ceph-mon[32568]: pgmap v507: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-04-25T13:47:07.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:07 smithi094 ceph-mon[29195]: pgmap v507: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-04-25T13:47:07.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:07 smithi138 ceph-mon[32977]: pgmap v507: 97 pgs: 97 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 309 op/s 2024-04-25T13:47:08.797 INFO:tasks.workunit.client.0.smithi094.stdout: 1 579289 13.28 MB/sec execute 111 sec latency 81.648 ms 2024-04-25T13:47:09.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 581751 13.27 MB/sec execute 112 sec latency 18.438 ms 2024-04-25T13:47:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:09 smithi137 ceph-mon[32568]: pgmap v508: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-04-25T13:47:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:09 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:47:09.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:09 smithi094 ceph-mon[29195]: pgmap v508: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-04-25T13:47:09.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:09 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:47:09.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:09 smithi138 ceph-mon[32977]: pgmap v508: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-04-25T13:47:09.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:09 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:47:10.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 583501 13.23 MB/sec execute 113 sec latency 21.445 ms 2024-04-25T13:47:10.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:47:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:47:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:10 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:47:10.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:47:10.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:47:10.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:10 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:47:10.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:47:10.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:47:10.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:10 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:47:11.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 586399 13.27 MB/sec execute 114 sec latency 12.994 ms 2024-04-25T13:47:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:12 smithi137 ceph-mon[32568]: pgmap v509: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-04-25T13:47:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:12 smithi094 ceph-mon[29195]: pgmap v509: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-04-25T13:47:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:12 smithi138 ceph-mon[32977]: pgmap v509: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 387 op/s 2024-04-25T13:47:12.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 589082 13.27 MB/sec execute 115 sec latency 14.489 ms 2024-04-25T13:47:13.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 590568 13.22 MB/sec execute 116 sec latency 71.075 ms 2024-04-25T13:47:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:14 smithi137 ceph-mon[32568]: pgmap v510: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-04-25T13:47:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:14 smithi094 ceph-mon[29195]: pgmap v510: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-04-25T13:47:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:14 smithi138 ceph-mon[32977]: pgmap v510: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-04-25T13:47:14.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 593533 13.26 MB/sec execute 117 sec latency 8.051 ms 2024-04-25T13:47:15.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 596185 13.25 MB/sec execute 118 sec latency 13.974 ms 2024-04-25T13:47:16.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:47:16.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 -- 172.21.15.94:0/2745224807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874103b00 msgr2=0x7f7874104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 --2- 172.21.15.94:0/2745224807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874103b00 0x7f7874104800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7864009900 tx=0x7f786402a410 comp rx=0 tx=0).stop 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 -- 172.21.15.94:0/2745224807 shutdown_connections 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 --2- 172.21.15.94:0/2745224807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874109a40 0x7f787410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 --2- 172.21.15.94:0/2745224807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 0x7f7874105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 --2- 172.21.15.94:0/2745224807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874103b00 0x7f7874104800 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 -- 172.21.15.94:0/2745224807 >> 172.21.15.94:0/2745224807 conn(0x7f78740fed10 msgr2=0x7f78741011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 -- 172.21.15.94:0/2745224807 shutdown_connections 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.584+0000 7f7878d86640 1 -- 172.21.15.94:0/2745224807 wait complete. 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.585+0000 7f7878d86640 1 Processor -- start 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.585+0000 7f7878d86640 1 -- start start 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.585+0000 7f7878d86640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 0x7f787419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.585+0000 7f7878d86640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 0x7f787419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f787410f5e0 con 0x7f7874109a40 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f787410f460 con 0x7f7874104d40 2024-04-25T13:47:16.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f787410f760 con 0x7f7874103b00 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37438/0 (socket says 172.21.15.94:37438) 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f787259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 0x7f787419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 -- 172.21.15.94:0/1077597966 learned_addr learned my addr 172.21.15.94:0/1077597966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f786bfff640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 0x7f787419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 -- 172.21.15.94:0/1077597966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 msgr2=0x7f787419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 0x7f787419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 -- 172.21.15.94:0/1077597966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 msgr2=0x7f787419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 0x7f787419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 -- 172.21.15.94:0/1077597966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741a1de0 con 0x7f7874109a40 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f787259c640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 0x7f787419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:16.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7872d9d640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f786c00eb10 tx=0x7f786c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f786b7fe640 1 -- 172.21.15.94:0/1077597966 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c004b20 con 0x7f7874109a40 2024-04-25T13:47:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741a20d0 con 0x7f7874109a40 2024-04-25T13:47:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f786b7fe640 1 -- 172.21.15.94:0/1077597966 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c004cc0 con 0x7f7874109a40 2024-04-25T13:47:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f786b7fe640 1 -- 172.21.15.94:0/1077597966 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c010670 con 0x7f7874109a40 2024-04-25T13:47:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.586+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741a25d0 con 0x7f7874109a40 2024-04-25T13:47:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f786b7fe640 1 -- 172.21.15.94:0/1077597966 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f786c005020 con 0x7f7874109a40 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 Processor -- start 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 -- start start 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 0x558f13626fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132646b0 0x558f13627c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558f1358bd80 con 0x558f13285390 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558f135458b0 con 0x558f132646b0 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787fc3f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f13587d40 con 0x558f1325c4d0 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f786bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f786bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37444/0 (socket says 172.21.15.94:37444) 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f786bfff640 1 -- 172.21.15.94:0/3277720608 learned_addr learned my addr 172.21.15.94:0/3277720608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.589+0000 7f787259c640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 0x558f13626fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f786bfff640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132646b0 0x558f13627c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f786b7fe640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 0x7f78600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f7872d9d640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 0x7f78600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 -- 172.21.15.94:0/3277720608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132646b0 msgr2=0x558f13627c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132646b0 0x558f13627c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 -- 172.21.15.94:0/3277720608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 msgr2=0x558f13626a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 -- 172.21.15.94:0/3277720608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f13516a40 con 0x558f1325c4d0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f786bfff640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f786b7fe640 1 -- 172.21.15.94:0/1077597966 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f786c0df330 con 0x7f7874109a40 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f787259c640 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 0x558f13626fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f786402a930 tx=0x7f7864005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7d243c3bb82f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f7868ff9640 1 -- 172.21.15.94:0/3277720608 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864040070 con 0x558f1325c4d0 2024-04-25T13:47:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f7868ff9640 1 -- 172.21.15.94:0/3277720608 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7864031860 con 0x558f1325c4d0 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f7872d9d640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 0x7f78600b2000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f786c0a7470 tx=0x7f786c0b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.590+0000 7f7868ff9640 1 -- 172.21.15.94:0/3277720608 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864035040 con 0x558f1325c4d0 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 -- 172.21.15.94:0/3277720608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 msgr2=0x558f13626fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 0x558f13626fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f786402a930 tx=0x7f7864005d70 comp rx=0 tx=0).stop 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 -- 172.21.15.94:0/3277720608 shutdown_connections 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132646b0 0x558f13627c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f1325c4d0 0x558f13626fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3277720608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f13285390 0x558f13626a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 -- 172.21.15.94:0/3277720608 >> 172.21.15.94:0/3277720608 conn(0x558f132794e0 msgr2=0x558f13622270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 -- 172.21.15.94:0/3277720608 shutdown_connections 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.591+0000 7f787fc3f740 1 -- 172.21.15.94:0/3277720608 wait complete. 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 -- start start 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 Processor -- start 2024-04-25T13:47:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f1325c4d0 0x558f13630960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f13637e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558f13628720 con 0x558f132646b0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558f13512d10 con 0x558f1325c4d0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f136288a0 con 0x558f13285390 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787fc3f740 10 client.0 _ll_register_callbacks cb 0x558f1353f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f786bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f13637e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37452/0 (socket says 172.21.15.94:37452) 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 -- 172.21.15.94:0/3725441615 learned_addr learned my addr 172.21.15.94:0/3725441615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f786bfff640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f1325c4d0 0x558f13630960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 -- 172.21.15.94:0/3725441615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 msgr2=0x558f13637e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f13637e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 -- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f1325c4d0 msgr2=0x558f13630960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f1325c4d0 0x558f13630960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.592+0000 7f787259c640 1 -- 172.21.15.94:0/3725441615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f133677d0 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787259c640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7864005ab0 tx=0x7f7864031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b183e81dd004dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786404c020 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7864004240 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7864031860 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7864057050 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 10 client.17430 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f1329d3f0 con 0x558f132646b0 2024-04-25T13:47:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7864031860 con 0x558f132646b0 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7864004810 con 0x558f132646b0 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 10 client.17430 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 20 client.17430 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f13243f60 0x558f1363dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 4 client.17430 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.593+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f13599990 con 0x558f13243f60 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.594+0000 7f786bfff640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f13243f60 0x558f1363dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.594+0000 7f7859ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.594+0000 7f786bfff640 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f13243f60 0x558f1363dec0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6460d519110b136f server_cookie=c65b9e83b89c7877 in_seq=0 out_seq=0 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f7859ffb640 1 -- 172.21.15.94:0/3725441615 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f13599990 con 0x558f13243f60 2024-04-25T13:47:16.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f7859ffb640 10 client.17430 handle_command_reply: tid=0 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 client.17430 shutdown 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 msgr2=0x558f13630ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7864005ab0 tx=0x7f7864031400 comp rx=0 tx=0).stop 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 shutdown_connections 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f13243f60 0x558f1363dec0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f13637e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f132646b0 0x558f13630ea0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3725441615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f1325c4d0 0x558f13630960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f7859ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 >> 172.21.15.94:0/3725441615 conn(0x558f132794e0 msgr2=0x558f13625180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 shutdown_connections 2024-04-25T13:47:16.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 1 -- 172.21.15.94:0/3725441615 wait complete. 2024-04-25T13:47:16.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.596+0000 7f787fc3f740 20 client.17430 trim_cache size 0 max 16384 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 Processor -- start 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 -- start start 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f13285390 0x558f13622320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f132193a0 0x558f13637dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78741ab530 con 0x558f1325c4d0 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78741ad070 con 0x558f13285390 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787fc3f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78741abc00 con 0x558f132193a0 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37460/0 (socket says 172.21.15.94:37460) 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 -- 172.21.15.94:0/1045352306 learned_addr learned my addr 172.21.15.94:0/1045352306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f787259c640 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f13285390 0x558f13622320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f132193a0 0x558f13637dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 -- 172.21.15.94:0/1045352306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f132193a0 msgr2=0x558f13637dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f132193a0 0x558f13637dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 -- 172.21.15.94:0/1045352306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f13285390 msgr2=0x558f13622320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f13285390 0x558f13622320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 -- 172.21.15.94:0/1045352306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1325d0f0 con 0x558f1325c4d0 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.615+0000 7f786bfff640 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f785c013730 tx=0x7f785c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60652cd32330991d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f785bfff640 1 -- 172.21.15.94:0/1045352306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c027e60 con 0x558f1325c4d0 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f785bfff640 1 -- 172.21.15.94:0/1045352306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c005510 con 0x558f1325c4d0 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f785bfff640 1 -- 172.21.15.94:0/1045352306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c004b20 con 0x558f1325c4d0 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 -- 172.21.15.94:0/1045352306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 msgr2=0x558f1362bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f785c013730 tx=0x7f785c0096c0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 -- 172.21.15.94:0/1045352306 shutdown_connections 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f132193a0 0x558f13637dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f13285390 0x558f13622320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 --2- 172.21.15.94:0/1045352306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f1362bd30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.616+0000 7f787fc3f740 1 -- 172.21.15.94:0/1045352306 >> 172.21.15.94:0/1045352306 conn(0x558f132794e0 msgr2=0x558f13621c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- 172.21.15.94:0/1045352306 shutdown_connections 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- 172.21.15.94:0/1045352306 wait complete. 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- start start 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 Processor -- start 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132193a0 0x558f136355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f136a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78741ac9a0 con 0x558f1325c4d0 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78741ac2d0 con 0x558f132193a0 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78741a9510 con 0x558f13285390 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787fc3f740 10 client.0 _ll_register_callbacks cb 0x7f785c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:16.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f786bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132193a0 0x558f136355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37470/0 (socket says 172.21.15.94:37470) 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f787259c640 1 -- 172.21.15.94:0/3111315196 learned_addr learned my addr 172.21.15.94:0/3111315196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.617+0000 7f786bfff640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f136a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 -- 172.21.15.94:0/3111315196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 msgr2=0x558f136a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f136a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 -- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132193a0 msgr2=0x558f136355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132193a0 0x558f136355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 -- 172.21.15.94:0/3111315196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1354c4b0 con 0x558f1325c4d0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787259c640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7864009900 tx=0x7f78640044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e57f1bdc621824b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864060060 con 0x558f1325c4d0 2024-04-25T13:47:16.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7864004ea0 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c005510 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f786404c040 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 10 client.17436 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f136a5660 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c005510 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7864031b10 con 0x558f1325c4d0 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 10 client.17436 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 20 client.17436 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f136a5f80 0x558f136a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 4 client.17436 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.618+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f13593800 con 0x558f136a5f80 2024-04-25T13:47:16.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.619+0000 7f786bfff640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f136a5f80 0x558f136a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.619+0000 7f7859ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:16.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.619+0000 7f786bfff640 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f136a5f80 0x558f136a83a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d98c2a6d9c9ce0 server_cookie=8ba1f16fbb5e3380 in_seq=0 out_seq=0 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f7859ffb640 1 -- 172.21.15.94:0/3111315196 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558f13593800 con 0x558f136a5f80 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f7859ffb640 10 client.17436 handle_command_reply: tid=0 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f787fc3f740 1 client.17436 shutdown 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 msgr2=0x558f13635af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7864009900 tx=0x7f78640044f0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 shutdown_connections 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.620+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558f136a5f80 0x558f136a83a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f13285390 0x558f136a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7859ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f1325c4d0 0x558f13635af0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 --2- 172.21.15.94:0/3111315196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f132193a0 0x558f136355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 >> 172.21.15.94:0/3111315196 conn(0x558f132794e0 msgr2=0x558f136209c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 shutdown_connections 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 1 -- 172.21.15.94:0/3111315196 wait complete. 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f787fc3f740 20 client.17436 trim_cache size 0 max 16384 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:47:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:47:16.628 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:16.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 msgr2=0x7f78600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 0x7f78600b2000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f786c0a7470 tx=0x7f786c0b0040 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 msgr2=0x7f78741a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f786c00eb10 tx=0x7f786c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 shutdown_connections 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f78600afaf0 0x7f78600b2000 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7874109a40 0x7f78741a1770 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7874104d40 0x7f787419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 --2- 172.21.15.94:0/1077597966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7874103b00 0x7f787419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 >> 172.21.15.94:0/1077597966 conn(0x7f78740fed10 msgr2=0x7f7874105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 shutdown_connections 2024-04-25T13:47:16.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.621+0000 7f7878d86640 1 -- 172.21.15.94:0/1077597966 wait complete. 2024-04-25T13:47:16.631 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:16 smithi094 ceph-mon[29195]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T13:47:16.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:47:16.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:16 smithi137 ceph-mon[32568]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T13:47:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- 172.21.15.94:0/661860099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd90109870 msgr2=0x7efd9010bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 --2- 172.21.15.94:0/661860099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd90109870 0x7efd9010bc90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efd88009be0 tx=0x7efd8802a6c0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- 172.21.15.94:0/661860099 shutdown_connections 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 --2- 172.21.15.94:0/661860099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd90109870 0x7efd9010bc90 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 --2- 172.21.15.94:0/661860099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 0x7efd90109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 --2- 172.21.15.94:0/661860099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd901045b0 0x7efd901069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- 172.21.15.94:0/661860099 >> 172.21.15.94:0/661860099 conn(0x7efd900fdbd0 msgr2=0x7efd90100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- 172.21.15.94:0/661860099 shutdown_connections 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- 172.21.15.94:0/661860099 wait complete. 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 Processor -- start 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.745+0000 7efd95ef9640 1 -- start start 2024-04-25T13:47:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 0x7efd9019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd90109870 0x7efd901a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f5f0 con 0x7efd901045b0 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f470 con 0x7efd90109870 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f770 con 0x7efd90106f10 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37488/0 (socket says 172.21.15.94:37488) 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 -- 172.21.15.94:0/136782542 learned_addr learned my addr 172.21.15.94:0/136782542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd87fff640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 0x7efd9019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8ffff640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd90109870 0x7efd901a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 -- 172.21.15.94:0/136782542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 msgr2=0x7efd9019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 0x7efd9019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 -- 172.21.15.94:0/136782542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd90109870 msgr2=0x7efd901a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd90109870 0x7efd901a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 -- 172.21.15.94:0/136782542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd901a1d00 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8f7fe640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efd8000e9c0 tx=0x7efd8000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000ccc0 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901a1ff0 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd800054d0 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80010430 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.746+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901a2450 con 0x7efd901045b0 2024-04-25T13:47:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.748+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9006afa0 con 0x7efd901045b0 2024-04-25T13:47:16.751 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:16 smithi138 ceph-mon[32977]: pgmap v511: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T13:47:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.749+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efd80004f20 con 0x7efd901045b0 2024-04-25T13:47:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.750+0000 7efd8d7fa640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 0x7efd7c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.750+0000 7efd87fff640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 0x7efd7c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.750+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efd800df550 con 0x7efd901045b0 2024-04-25T13:47:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.750+0000 7efd87fff640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 0x7efd7c0b2000 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd9019b260 tx=0x7efd78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.754+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd800a8090 con 0x7efd901045b0 2024-04-25T13:47:16.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 598217 13.24 MB/sec execute 119 sec latency 24.925 ms 2024-04-25T13:47:16.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.991+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd9002dc10 con 0x7efd901045b0 2024-04-25T13:47:16.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.992+0000 7efd8d7fa640 1 -- 172.21.15.94:0/136782542 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7efd800a7990 con 0x7efd901045b0 2024-04-25T13:47:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:47:16.992 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:47:16.993 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 msgr2=0x7efd7c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 0x7efd7c0b2000 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd9019b260 tx=0x7efd78009260 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 msgr2=0x7efd9019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efd8000e9c0 tx=0x7efd8000edc0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 shutdown_connections 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd7c0afaf0 0x7efd7c0b2000 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd90109870 0x7efd901a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd90106f10 0x7efd9019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 --2- 172.21.15.94:0/136782542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd901045b0 0x7efd9019a1c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.994+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 >> 172.21.15.94:0/136782542 conn(0x7efd900fdbd0 msgr2=0x7efd90105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.995+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 shutdown_connections 2024-04-25T13:47:16.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:16.995+0000 7efd95ef9640 1 -- 172.21.15.94:0/136782542 wait complete. 2024-04-25T13:47:17.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:47:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 -- 172.21.15.94:0/2626890515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c108930 msgr2=0x7f1c7c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 --2- 172.21.15.94:0/2626890515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c108930 0x7f1c7c10ad50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009ca0 tx=0x7f1c7402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 -- 172.21.15.94:0/2626890515 shutdown_connections 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 --2- 172.21.15.94:0/2626890515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c108930 0x7f1c7c10ad50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 --2- 172.21.15.94:0/2626890515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c103c30 0x7f1c7c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 --2- 172.21.15.94:0/2626890515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 -- 172.21.15.94:0/2626890515 >> 172.21.15.94:0/2626890515 conn(0x7f1c7c0fdbc0 msgr2=0x7f1c7c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 -- 172.21.15.94:0/2626890515 shutdown_connections 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.119+0000 7f1c80e67640 1 -- 172.21.15.94:0/2626890515 wait complete. 2024-04-25T13:47:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 Processor -- start 2024-04-25T13:47:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 -- start start 2024-04-25T13:47:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 0x7f1c7c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c108930 0x7f1c7c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10e520 con 0x7f1c7c103c30 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10e3a0 con 0x7f1c7c1029f0 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c80e67640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10e6a0 con 0x7f1c7c108930 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 0x7f1c7c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38878/0 (socket says 172.21.15.94:38878) 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.120+0000 7f1c7a59c640 1 -- 172.21.15.94:0/1912093461 learned_addr learned my addr 172.21.15.94:0/1912093461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c108930 0x7f1c7c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 -- 172.21.15.94:0/1912093461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c108930 msgr2=0x7f1c7c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c108930 0x7f1c7c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 -- 172.21.15.94:0/1912093461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 msgr2=0x7f1c7c19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 -- 172.21.15.94:0/1912093461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c1a1d20 con 0x7f1c7c103c30 2024-04-25T13:47:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 0x7f1c7c19a720 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400d720 tx=0x7f1c6400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c73fff640 1 -- 172.21.15.94:0/1912093461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64017070 con 0x7f1c7c103c30 2024-04-25T13:47:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c73fff640 1 -- 172.21.15.94:0/1912093461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6400de30 con 0x7f1c7c103c30 2024-04-25T13:47:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1a1fb0 con 0x7f1c7c103c30 2024-04-25T13:47:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c73fff640 1 -- 172.21.15.94:0/1912093461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640047a0 con 0x7f1c7c103c30 2024-04-25T13:47:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.121+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c1a2450 con 0x7f1c7c103c30 2024-04-25T13:47:17.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 Processor -- start 2024-04-25T13:47:17.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 -- start start 2024-04-25T13:47:17.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c73fff640 1 -- 172.21.15.94:0/1912093461 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c640099d0 con 0x7f1c7c103c30 2024-04-25T13:47:17.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c1da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a40b80d80 con 0x559a408596b0 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a40b3aa20 con 0x559a4087a390 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.124+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a40b7cd40 con 0x559a408514d0 2024-04-25T13:47:17.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c1da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38890/0 (socket says 172.21.15.94:38890) 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7ad9d640 1 -- 172.21.15.94:0/4270638858 learned_addr learned my addr 172.21.15.94:0/4270638858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 -- 172.21.15.94:0/4270638858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 msgr2=0x559a40c1c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 -- 172.21.15.94:0/4270638858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 msgr2=0x559a40c1da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c1da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 -- 172.21.15.94:0/4270638858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a40b45c30 con 0x559a408514d0 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c1da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c73fff640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 0x7f1c680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 0x7f1c680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c1cd90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00bc60 tx=0x7f1c6c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b395c9658f5ded3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c73fff640 1 -- 172.21.15.94:0/1912093461 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1c640de8a0 con 0x7f1c7c103c30 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c70d99640 1 -- 172.21.15.94:0/4270638858 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004220 con 0x559a408514d0 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c70d99640 1 -- 172.21.15.94:0/4270638858 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c00be10 con 0x559a408514d0 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.125+0000 7f1c70d99640 1 -- 172.21.15.94:0/4270638858 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0116a0 con 0x559a408514d0 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 -- 172.21.15.94:0/4270638858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 msgr2=0x559a40c1cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 0x7f1c680b2000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x559a40c1d860 tx=0x7f1c7402f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c1cd90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00bc60 tx=0x7f1c6c0107a0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 -- 172.21.15.94:0/4270638858 shutdown_connections 2024-04-25T13:47:17.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c1da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c1cd90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/4270638858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c1c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 -- 172.21.15.94:0/4270638858 >> 172.21.15.94:0/4270638858 conn(0x559a4086e4e0 msgr2=0x559a40c18020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 -- 172.21.15.94:0/4270638858 shutdown_connections 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.126+0000 7f1c87dfa740 1 -- 172.21.15.94:0/4270638858 wait complete. 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 -- start start 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 Processor -- start 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a40c1e530 con 0x559a408596b0 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a40b07dc0 con 0x559a4087a390 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a40c1e6b0 con 0x559a408514d0 2024-04-25T13:47:17.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c87dfa740 10 client.0 _ll_register_callbacks cb 0x559a40b23600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33410/0 (socket says 172.21.15.94:33410) 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c7a59c640 1 -- 172.21.15.94:0/451948444 learned_addr learned my addr 172.21.15.94:0/451948444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37510/0 (socket says 172.21.15.94:37510) 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 -- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 msgr2=0x559a40c26560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 -- 172.21.15.94:0/451948444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 msgr2=0x559a40c2da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.127+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c2da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c71d9b640 1 -- 172.21.15.94:0/451948444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a408923f0 con 0x559a408596b0 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1c640dc350 tx=0x7f1c64004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1df661a3bd6dd66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640a6630 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1c640a2f70 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6c00be10 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c640a6450 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 10 client.17454 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a4095c7d0 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c00be10 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c640c1030 con 0x559a408596b0 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 10 client.17454 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 10 client.17454 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 20 client.17454 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a40838f60 0x559a40c34320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 4 client.17454 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.128+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a40b8e990 con 0x559a40838f60 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.129+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a40838f60 0x559a40c34320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.129+0000 7f1c61ffb640 10 client.17454 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.129+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a40838f60 0x559a40c34320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fc077f90b0b439 server_cookie=6e7fa6544304c9de in_seq=0 out_seq=0 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c61ffb640 1 -- 172.21.15.94:0/451948444 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a40b8e990 con 0x559a40838f60 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c61ffb640 10 client.17454 handle_command_reply: tid=0 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 client.17454 shutdown 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 msgr2=0x559a40c26aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1c640dc350 tx=0x7f1c64004220 comp rx=0 tx=0).stop 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 shutdown_connections 2024-04-25T13:47:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a40838f60 0x559a40c34320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4087a390 0x559a40c2da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c26aa0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c61ffb640 0 client.17454 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/451948444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a408514d0 0x559a40c26560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 >> 172.21.15.94:0/451948444 conn(0x559a4086e4e0 msgr2=0x559a40c1af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 shutdown_connections 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 1 -- 172.21.15.94:0/451948444 wait complete. 2024-04-25T13:47:17.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.130+0000 7f1c87dfa740 20 client.17454 trim_cache size 0 max 16384 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 Processor -- start 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 -- start start 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408596b0 0x559a40c24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1ab5c0 con 0x559a408514d0 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1ad100 con 0x559a408596b0 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.144+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1abc90 con 0x559a40838f60 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37522/0 (socket says 172.21.15.94:37522) 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 -- 172.21.15.94:0/78671745 learned_addr learned my addr 172.21.15.94:0/78671745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408596b0 0x559a40c24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 -- 172.21.15.94:0/78671745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 msgr2=0x559a40c21290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c21290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 -- 172.21.15.94:0/78671745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408596b0 msgr2=0x559a40c24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408596b0 0x559a40c24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 -- 172.21.15.94:0/78671745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a40c276a0 con 0x559a408514d0 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1c64004770 tx=0x7f1c640c7f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d34db37dd765bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c63fff640 1 -- 172.21.15.94:0/78671745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640c5030 con 0x559a408514d0 2024-04-25T13:47:17.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c63fff640 1 -- 172.21.15.94:0/78671745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6408f040 con 0x559a408514d0 2024-04-25T13:47:17.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.145+0000 7f1c63fff640 1 -- 172.21.15.94:0/78671745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640a73e0 con 0x559a408514d0 2024-04-25T13:47:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- 172.21.15.94:0/78671745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 msgr2=0x559a40c217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1c64004770 tx=0x7f1c640c7f90 comp rx=0 tx=0).stop 2024-04-25T13:47:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- 172.21.15.94:0/78671745 shutdown_connections 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408596b0 0x559a40c24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408514d0 0x559a40c217d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/78671745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c21290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- 172.21.15.94:0/78671745 >> 172.21.15.94:0/78671745 conn(0x559a4086e4e0 msgr2=0x559a40c12d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- 172.21.15.94:0/78671745 shutdown_connections 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- 172.21.15.94:0/78671745 wait complete. 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 -- start start 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.146+0000 7f1c87dfa740 1 Processor -- start 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408514d0 0x559a40c29810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c29d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1aca30 con 0x559a408596b0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1ac360 con 0x559a408514d0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c1a95a0 con 0x559a40838f60 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c87dfa740 10 client.0 _ll_register_callbacks cb 0x7f1c640046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33426/0 (socket says 172.21.15.94:33426) 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 -- 172.21.15.94:0/1395640999 learned_addr learned my addr 172.21.15.94:0/1395640999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408514d0 0x559a40c29810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 -- 172.21.15.94:0/1395640999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408514d0 msgr2=0x559a40c29810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408514d0 0x559a40c29810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 -- 172.21.15.94:0/1395640999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 msgr2=0x559a40c29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 -- 172.21.15.94:0/1395640999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a40c2a290 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7ad9d640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c7a59c640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c011010 tx=0x7f1c6c00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0dd5164e0b21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.147+0000 7f1c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c005240 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1c6c0053e0 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6408f040 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c0055e0 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 10 client.26824 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a40c9b5e0 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6408f040 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6c03c650 con 0x559a40838f60 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 10 client.26824 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 10 client.26824 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:47:17.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 20 client.26824 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a4087a390 0x559a40c9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 4 client.26824 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a40b88800 con 0x559a4087a390 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.148+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a4087a390 0x559a40c9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c61ffb640 10 client.26824 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c71d9b640 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a4087a390 0x559a40c9dd50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=203e5bb4baf4840c server_cookie=28278912d692c355 in_seq=0 out_seq=0 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c61ffb640 1 -- 172.21.15.94:0/1395640999 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a40b88800 con 0x559a4087a390 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c61ffb640 10 client.26824 handle_command_reply: tid=0 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 client.26824 shutdown 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 msgr2=0x559a40c2cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c011010 tx=0x7f1c6c00bef0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 shutdown_connections 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559a4087a390 0x559a40c9dd50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a408596b0 0x559a40c29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a408514d0 0x559a40c29810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 --2- 172.21.15.94:0/1395640999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a40838f60 0x559a40c2cc40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c61ffb640 0 client.26824 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 >> 172.21.15.94:0/1395640999 conn(0x559a4086e4e0 msgr2=0x559a40c21290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 shutdown_connections 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 1 -- 172.21.15.94:0/1395640999 wait complete. 2024-04-25T13:47:17.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.149+0000 7f1c87dfa740 20 client.26824 trim_cache size 0 max 16384 2024-04-25T13:47:17.159 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 msgr2=0x7f1c680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 0x7f1c680b2000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x559a40c1d860 tx=0x7f1c7402f4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 msgr2=0x7f1c7c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 0x7f1c7c19a720 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400d720 tx=0x7f1c6400dbe0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 shutdown_connections 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c680afaf0 0x7f1c680b2000 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7c108930 0x7f1c7c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c7c103c30 0x7f1c7c19a720 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 --2- 172.21.15.94:0/1912093461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c7c1029f0 0x7f1c7c19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 >> 172.21.15.94:0/1912093461 conn(0x7f1c7c0fdbc0 msgr2=0x7f1c7c109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 shutdown_connections 2024-04-25T13:47:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.150+0000 7f1c80e67640 1 -- 172.21.15.94:0/1912093461 wait complete. 2024-04-25T13:47:17.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:47:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 -- 172.21.15.94:0/1145158109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8108920 msgr2=0x7ffac810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 --2- 172.21.15.94:0/1145158109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8108920 0x7ffac810ad40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009ca0 tx=0x7ffac002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 -- 172.21.15.94:0/1145158109 shutdown_connections 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 --2- 172.21.15.94:0/1145158109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8108920 0x7ffac810ad40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 --2- 172.21.15.94:0/1145158109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac8103c20 0x7ffac8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 --2- 172.21.15.94:0/1145158109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac81029e0 0x7ffac81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 -- 172.21.15.94:0/1145158109 >> 172.21.15.94:0/1145158109 conn(0x7ffac80fdbd0 msgr2=0x7ffac8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 -- 172.21.15.94:0/1145158109 shutdown_connections 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 -- 172.21.15.94:0/1145158109 wait complete. 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.277+0000 7ffacd26e640 1 Processor -- start 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 -- start start 2024-04-25T13:47:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac81029e0 0x7ffac8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac8108920 0x7ffac81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffac810e510 con 0x7ffac8103c20 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffac810e390 con 0x7ffac81029e0 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffacd26e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffac810e690 con 0x7ffac8108920 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37544/0 (socket says 172.21.15.94:37544) 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2966894798 learned_addr learned my addr 172.21.15.94:0/2966894798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac81029e0 0x7ffac8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffac77fe640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac8108920 0x7ffac81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2966894798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac8108920 msgr2=0x7ffac81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac8108920 0x7ffac81a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2966894798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac81029e0 msgr2=0x7ffac8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac81029e0 0x7ffac8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2966894798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac81a0f50 con 0x7ffac8103c20 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.278+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e9f0 tx=0x7ffab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.279+0000 7ffac4ff9640 1 -- 172.21.15.94:0/2966894798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab000cd80 con 0x7ffac8103c20 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.279+0000 7ffac4ff9640 1 -- 172.21.15.94:0/2966894798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab00108e0 con 0x7ffac8103c20 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.279+0000 7ffac4ff9640 1 -- 172.21.15.94:0/2966894798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0018980 con 0x7ffac8103c20 2024-04-25T13:47:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.279+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac81a1240 con 0x7ffac8103c20 2024-04-25T13:47:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.279+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac81a17e0 con 0x7ffac8103c20 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.281+0000 7ffad4202740 1 Processor -- start 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 -- start start 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 0x55e4cf348810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 0x55e4cf349a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf2acd80 con 0x55e4cef7d4d0 2024-04-25T13:47:17.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac4ff9640 1 -- 172.21.15.94:0/2966894798 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ffab0010430 con 0x7ffac8103c20 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf266a20 con 0x55e4cefa6390 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffad4202740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf2a8d40 con 0x55e4cef856b0 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37560/0 (socket says 172.21.15.94:37560) 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac6ffd640 1 -- 172.21.15.94:0/1412469513 learned_addr learned my addr 172.21.15.94:0/1412469513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 0x55e4cf348810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 0x55e4cf349a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.282+0000 7ffac77fe640 1 -- 172.21.15.94:0/1412469513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 msgr2=0x55e4cf349a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac77fe640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 0x55e4cf349a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac77fe640 1 -- 172.21.15.94:0/1412469513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 msgr2=0x55e4cf348d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac77fe640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac77fe640 1 -- 172.21.15.94:0/1412469513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cf271c30 con 0x55e4cefa6390 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 0x55e4cf349a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:17.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac77fe640 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 0x55e4cf348810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffac0031b50 tx=0x7ffac002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ffcc21ed6a343f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac4ff9640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 0x7ffab40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffabd7fa640 1 -- 172.21.15.94:0/1412469513 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0031650 con 0x55e4cefa6390 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffabd7fa640 1 -- 172.21.15.94:0/1412469513 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0036030 con 0x55e4cefa6390 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffabd7fa640 1 -- 172.21.15.94:0/1412469513 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004660 con 0x55e4cefa6390 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 0x7ffab40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.283+0000 7ffac4ff9640 1 -- 172.21.15.94:0/2966894798 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffab00def10 con 0x7ffac8103c20 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- 172.21.15.94:0/1412469513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 msgr2=0x55e4cf348810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 0x55e4cf348810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffac0031b50 tx=0x7ffac002f410 comp rx=0 tx=0).stop 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 0x7ffab40b2080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55e4cf348530 tx=0x7ffab8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- 172.21.15.94:0/1412469513 shutdown_connections 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef856b0 0x55e4cf349a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf348d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 --2- 172.21.15.94:0/1412469513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cefa6390 0x55e4cf348810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- 172.21.15.94:0/1412469513 >> 172.21.15.94:0/1412469513 conn(0x55e4cef9a4e0 msgr2=0x55e4cf344020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- 172.21.15.94:0/1412469513 shutdown_connections 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- 172.21.15.94:0/1412469513 wait complete. 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.284+0000 7ffad4202740 1 -- start start 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 Processor -- start 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef7d4d0 0x55e4cf3591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf35b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf34a350 con 0x55e4cefa6390 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf233dc0 con 0x55e4cef856b0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e4cf34a4d0 con 0x55e4cef7d4d0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffad4202740 10 client.0 _ll_register_callbacks cb 0x55e4cf24f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37566/0 (socket says 172.21.15.94:37566) 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 -- 172.21.15.94:0/2420099914 learned_addr learned my addr 172.21.15.94:0/2420099914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf35b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef7d4d0 0x55e4cf3591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 -- 172.21.15.94:0/2420099914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef7d4d0 msgr2=0x55e4cf3591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef7d4d0 0x55e4cf3591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 -- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 msgr2=0x55e4cf35b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf35b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 -- 172.21.15.94:0/2420099914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cefbe3f0 con 0x55e4cefa6390 2024-04-25T13:47:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf35b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffac77fe640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004680 tx=0x7ffac00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440b278d6077e226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.285+0000 7ffaae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac004a040 con 0x55e4cefa6390 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffac0003b70 con 0x55e4cefa6390 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac0036030 con 0x55e4cefa6390 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffac0009360 con 0x55e4cefa6390 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 10 client.17472 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e4cf0887d0 con 0x55e4cefa6390 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0036030 con 0x55e4cefa6390 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffac004d030 con 0x55e4cefa6390 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 10 client.17472 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 10 client.17472 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 20 client.17472 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cef64f60 0x55e4cf364170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 4 client.17472 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4cf2ba990 con 0x55e4cef64f60 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.286+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cef64f60 0x55e4cf364170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.287+0000 7ffaae7fc640 10 client.17472 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.287+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cef64f60 0x55e4cf364170 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46228244eea4457c server_cookie=7079aef147b3e1ed in_seq=0 out_seq=0 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2420099914 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4cf2ba990 con 0x55e4cef64f60 2024-04-25T13:47:17.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffaae7fc640 10 client.17472 handle_command_reply: tid=0 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 client.17472 shutdown 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 msgr2=0x55e4cf35dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004680 tx=0x7ffac00046b0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 shutdown_connections 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cef64f60 0x55e4cf364170 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cefa6390 0x55e4cf35dc80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf35b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffaae7fc640 0 client.17472 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 --2- 172.21.15.94:0/2420099914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef7d4d0 0x55e4cf3591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 >> 172.21.15.94:0/2420099914 conn(0x55e4cef9a4e0 msgr2=0x55e4cf343040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 shutdown_connections 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 1 -- 172.21.15.94:0/2420099914 wait complete. 2024-04-25T13:47:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.288+0000 7ffad4202740 20 client.17472 trim_cache size 0 max 16384 2024-04-25T13:47:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 Processor -- start 2024-04-25T13:47:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 -- start start 2024-04-25T13:47:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf369930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef7d4d0 0x55e4cf369e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81a9210 con 0x55e4cef856b0 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81aad50 con 0x55e4cef7d4d0 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffad4202740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81a98e0 con 0x55e4cef64f60 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffac77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37568/0 (socket says 172.21.15.94:37568) 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffac77fe640 1 -- 172.21.15.94:0/4284930474 learned_addr learned my addr 172.21.15.94:0/4284930474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.302+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef7d4d0 0x55e4cf369e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf369930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 -- 172.21.15.94:0/4284930474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 msgr2=0x55e4cf369930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf369930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 -- 172.21.15.94:0/4284930474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef7d4d0 msgr2=0x55e4cf369e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef7d4d0 0x55e4cf369e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 -- 172.21.15.94:0/4284930474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cf34fe00 con 0x55e4cef856b0 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffac77fe640 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ffac002ade0 tx=0x7ffac00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42eb158fa4294bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffabcff9640 1 -- 172.21.15.94:0/4284930474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0009360 con 0x55e4cef856b0 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffabcff9640 1 -- 172.21.15.94:0/4284930474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0036070 con 0x55e4cef856b0 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.303+0000 7ffabcff9640 1 -- 172.21.15.94:0/4284930474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac002f660 con 0x55e4cef856b0 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- 172.21.15.94:0/4284930474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 msgr2=0x55e4cf34e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ffac002ade0 tx=0x7ffac00317e0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- 172.21.15.94:0/4284930474 shutdown_connections 2024-04-25T13:47:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef856b0 0x55e4cf34e850 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef7d4d0 0x55e4cf369e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 --2- 172.21.15.94:0/4284930474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf369930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- 172.21.15.94:0/4284930474 >> 172.21.15.94:0/4284930474 conn(0x55e4cef9a4e0 msgr2=0x55e4cf346e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- 172.21.15.94:0/4284930474 shutdown_connections 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- 172.21.15.94:0/4284930474 wait complete. 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 -- start start 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.304+0000 7ffad4202740 1 Processor -- start 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf35cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf3ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81aa680 con 0x55e4cef7d4d0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81a9fb0 con 0x55e4cef856b0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffac81a71f0 con 0x55e4cef64f60 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 10 client.0 _ll_register_callbacks cb 0x7ffab8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37570/0 (socket says 172.21.15.94:37570) 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2888889225 learned_addr learned my addr 172.21.15.94:0/2888889225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf35cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffac77fe640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf3ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2888889225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 msgr2=0x55e4cf35cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf35cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 msgr2=0x55e4cf3ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf3ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 -- 172.21.15.94:0/2888889225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cf3cab50 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffabe7fc640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ffab00dc970 tx=0x7ffab000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67e2bb570d9df0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffaae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a6f10 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffab00a6870 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.305+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac0036070 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffab00a7400 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 10 client.17478 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e4cf3cbed0 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac0036070 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffab00a60d0 con 0x55e4cef7d4d0 2024-04-25T13:47:17.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 10 client.17478 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 10 client.17478 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 20 client.17478 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cefa6390 0x55e4cf3ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 4 client.17478 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4cf2b4800 con 0x55e4cefa6390 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.306+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cefa6390 0x55e4cf3ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffaae7fc640 10 client.17478 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffac6ffd640 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cefa6390 0x55e4cf3ce870 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85d7198dc243203b server_cookie=3e19e5e43421248d in_seq=0 out_seq=0 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffaae7fc640 1 -- 172.21.15.94:0/2888889225 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4cf2b4800 con 0x55e4cefa6390 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffaae7fc640 10 client.17478 handle_command_reply: tid=0 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 client.17478 shutdown 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 msgr2=0x55e4cf35d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ffab00dc970 tx=0x7ffab000cf20 comp rx=0 tx=0).stop 2024-04-25T13:47:17.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 shutdown_connections 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e4cefa6390 0x55e4cf3ce870 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e4cef856b0 0x55e4cf3ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4cef7d4d0 0x55e4cf35d480 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 --2- 172.21.15.94:0/2888889225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4cef64f60 0x55e4cf35cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 >> 172.21.15.94:0/2888889225 conn(0x55e4cef9a4e0 msgr2=0x55e4cf34e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffaae7fc640 0 client.17478 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 shutdown_connections 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 1 -- 172.21.15.94:0/2888889225 wait complete. 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.307+0000 7ffad4202740 20 client.17478 trim_cache size 0 max 16384 2024-04-25T13:47:17.317 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:17.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 msgr2=0x7ffab40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 0x7ffab40b2080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55e4cf348530 tx=0x7ffab8005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 msgr2=0x7ffac8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e9f0 tx=0x7ffab000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 shutdown_connections 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffab40afb70 0x7ffab40b2080 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffac8108920 0x7ffac81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffac8103c20 0x7ffac8199950 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 --2- 172.21.15.94:0/2966894798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffac81029e0 0x7ffac8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 >> 172.21.15.94:0/2966894798 conn(0x7ffac80fdbd0 msgr2=0x7ffac8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 shutdown_connections 2024-04-25T13:47:17.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.308+0000 7ffacd26e640 1 -- 172.21.15.94:0/2966894798 wait complete. 2024-04-25T13:47:17.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:47:17.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 -- 172.21.15.94:0/2682251879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 msgr2=0x7f61981033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2682251879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f61981033c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6188009900 tx=0x7f618802a410 comp rx=0 tx=0).stop 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 -- 172.21.15.94:0/2682251879 shutdown_connections 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2682251879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198104470 0x7f619810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2682251879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198103990 0x7f6198103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2682251879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f61981033c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 -- 172.21.15.94:0/2682251879 >> 172.21.15.94:0/2682251879 conn(0x7f61980fd350 msgr2=0x7f61980ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 -- 172.21.15.94:0/2682251879 shutdown_connections 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.434+0000 7f619d4d1640 1 -- 172.21.15.94:0/2682251879 wait complete. 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 Processor -- start 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 -- start start 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198103990 0x7f619819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198104470 0x7f61981a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f619810e570 con 0x7f6198102fb0 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f619810e3f0 con 0x7f6198104470 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.435+0000 7f619d4d1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f619810e6f0 con 0x7f6198103990 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37592/0 (socket says 172.21.15.94:37592) 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 -- 172.21.15.94:0/2470343053 learned_addr learned my addr 172.21.15.94:0/2470343053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f61977fe640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198104470 0x7f61981a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f61967fc640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198103990 0x7f619819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 -- 172.21.15.94:0/2470343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198103990 msgr2=0x7f619819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198103990 0x7f619819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 -- 172.21.15.94:0/2470343053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198104470 msgr2=0x7f61981a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198104470 0x7f61981a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 -- 172.21.15.94:0/2470343053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61981a1d20 con 0x7f6198102fb0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6188031440 tx=0x7f6188003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f617ffff640 1 -- 172.21.15.94:0/2470343053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618803f070 con 0x7f6198102fb0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f617ffff640 1 -- 172.21.15.94:0/2470343053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188031c50 con 0x7f6198102fb0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61981a1fb0 con 0x7f6198102fb0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f617ffff640 1 -- 172.21.15.94:0/2470343053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188035030 con 0x7f6198102fb0 2024-04-25T13:47:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.436+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61981a23b0 con 0x7f6198102fb0 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.439+0000 7f61a437f740 1 Processor -- start 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.439+0000 7f61a437f740 1 -- start start 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715af7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715af7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556715a5bd80 con 0x5567157346b0 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556715a15a20 con 0x556715755390 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61a437f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556715a57d40 con 0x55671572c4d0 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61977fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715af7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37594/0 (socket says 172.21.15.94:37594) 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 -- 172.21.15.94:0/4130892556 learned_addr learned my addr 172.21.15.94:0/4130892556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f617ffff640 1 -- 172.21.15.94:0/2470343053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6188042070 con 0x7f6198102fb0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 -- 172.21.15.94:0/4130892556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 msgr2=0x556715af7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715af7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 -- 172.21.15.94:0/4130892556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 msgr2=0x556715af7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715af7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 -- 172.21.15.94:0/4130892556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556715a20c30 con 0x5567157346b0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61977fe640 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715af7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f61967fc640 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f618c009c90 tx=0x7f618c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c8fc62b93ff7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f617d7fa640 1 -- 172.21.15.94:0/4130892556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c005220 con 0x5567157346b0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f617ffff640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 0x7f61800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f617d7fa640 1 -- 172.21.15.94:0/4130892556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c004c50 con 0x5567157346b0 2024-04-25T13:47:17.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.440+0000 7f617d7fa640 1 -- 172.21.15.94:0/4130892556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c019760 con 0x5567157346b0 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61977fe640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 0x7f61800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f617ffff640 1 -- 172.21.15.94:0/2470343053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f61880ff390 con 0x7f6198102fb0 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 -- 172.21.15.94:0/4130892556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 msgr2=0x556715af8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61977fe640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 0x7f61800b2000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556715af8860 tx=0x7f6190009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f618c009c90 tx=0x7f618c011810 comp rx=0 tx=0).stop 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 -- 172.21.15.94:0/4130892556 shutdown_connections 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715af8a20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715af7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 --2- 172.21.15.94:0/4130892556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715af7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 -- 172.21.15.94:0/4130892556 >> 172.21.15.94:0/4130892556 conn(0x5567157494e0 msgr2=0x556715af3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 -- 172.21.15.94:0/4130892556 shutdown_connections 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.441+0000 7f61a437f740 1 -- 172.21.15.94:0/4130892556 wait complete. 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 -- start start 2024-04-25T13:47:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 Processor -- start 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b01ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556715af9530 con 0x55671572c4d0 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5567159e2dc0 con 0x5567157346b0 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556715af96b0 con 0x556715755390 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61a437f740 10 client.0 _ll_register_callbacks cb 0x5567159fe600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37606/0 (socket says 172.21.15.94:37606) 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 -- 172.21.15.94:0/3140970521 learned_addr learned my addr 172.21.15.94:0/3140970521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61977fe640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61967fc640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b01ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 -- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 msgr2=0x556715b08a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b08a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 -- 172.21.15.94:0/3140970521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 msgr2=0x556715b01ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b01ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f6196ffd640 1 -- 172.21.15.94:0/3140970521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671576d3f0 con 0x55671572c4d0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.442+0000 7f61967fc640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b01ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f6196ffd640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6188000c00 tx=0x7f61880c7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c85a8ddc40de016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188049070 con 0x55671572c4d0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f618803b460 con 0x55671572c4d0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c004c50 con 0x55671572c4d0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61880c7390 con 0x55671572c4d0 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 10 client.17490 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567158377d0 con 0x55671572c4d0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c004c50 con 0x55671572c4d0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f61880c7ae0 con 0x55671572c4d0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 10 client.17490 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 10 client.17490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 20 client.17490 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556715713f60 0x556715b0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 4 client.17490 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556715a69990 con 0x556715713f60 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.443+0000 7f61967fc640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556715713f60 0x556715b0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.444+0000 7f617aff5640 10 client.17490 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.444+0000 7f61967fc640 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556715713f60 0x556715b0f1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=800a291535f7edf3 server_cookie=466d1e91323057c0 in_seq=0 out_seq=0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f617aff5640 1 -- 172.21.15.94:0/3140970521 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556715a69990 con 0x556715713f60 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f617aff5640 10 client.17490 handle_command_reply: tid=0 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 client.17490 shutdown 2024-04-25T13:47:17.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 msgr2=0x556715b01590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6188000c00 tx=0x7f61880c7780 comp rx=0 tx=0).stop 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 shutdown_connections 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556715713f60 0x556715b0f1b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b08a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b01ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f617aff5640 0 client.17490 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 --2- 172.21.15.94:0/3140970521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715b01590 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 >> 172.21.15.94:0/3140970521 conn(0x5567157494e0 msgr2=0x556715af2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 shutdown_connections 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 1 -- 172.21.15.94:0/3140970521 wait complete. 2024-04-25T13:47:17.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.445+0000 7f61a437f740 20 client.17490 trim_cache size 0 max 16384 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 Processor -- start 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 -- start start 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61981ab4a0 con 0x55671572c4d0 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61981acfe0 con 0x5567157346b0 2024-04-25T13:47:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.459+0000 7f61a437f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61981abb70 con 0x556715755390 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37614/0 (socket says 172.21.15.94:37614) 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 -- 172.21.15.94:0/2606288882 learned_addr learned my addr 172.21.15.94:0/2606288882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f61967fc640 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f61977fe640 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 -- 172.21.15.94:0/2606288882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 msgr2=0x556715b139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 -- 172.21.15.94:0/2606288882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 msgr2=0x556715b089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 -- 172.21.15.94:0/2606288882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556715af8100 con 0x55671572c4d0 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f6196ffd640 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61880ff360 tx=0x7f618803b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39996b4e2842ee93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f617cff9640 1 -- 172.21.15.94:0/2606288882 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188049070 con 0x55671572c4d0 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f617cff9640 1 -- 172.21.15.94:0/2606288882 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880c7880 con 0x55671572c4d0 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.460+0000 7f617cff9640 1 -- 172.21.15.94:0/2606288882 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618803b580 con 0x55671572c4d0 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- 172.21.15.94:0/2606288882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 msgr2=0x556715aee190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61880ff360 tx=0x7f618803b790 comp rx=0 tx=0).stop 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- 172.21.15.94:0/2606288882 shutdown_connections 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556715755390 0x556715b139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567157346b0 0x556715b089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 --2- 172.21.15.94:0/2606288882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55671572c4d0 0x556715aee190 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- 172.21.15.94:0/2606288882 >> 172.21.15.94:0/2606288882 conn(0x5567157494e0 msgr2=0x556715b01160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- 172.21.15.94:0/2606288882 shutdown_connections 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- 172.21.15.94:0/2606288882 wait complete. 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 -- start start 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.461+0000 7f61a437f740 1 Processor -- start 2024-04-25T13:47:17.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715b061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715b07650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61981ac910 con 0x5567157346b0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61981ac240 con 0x556715755390 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61981a9480 con 0x55671572c4d0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 10 client.0 _ll_register_callbacks cb 0x7f6188005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37620/0 (socket says 172.21.15.94:37620) 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715b061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 -- 172.21.15.94:0/322395225 learned_addr learned my addr 172.21.15.94:0/322395225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61977fe640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715b07650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 -- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 msgr2=0x556715b061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715b061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 -- 172.21.15.94:0/322395225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 msgr2=0x556715b07650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715b07650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 -- 172.21.15.94:0/322395225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556715b07c10 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61967fc640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f618c009be0 tx=0x7f618c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82470a560300a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f617aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c031070 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f618c004460 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.462+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61880c7880 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f618c03d050 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 10 client.17496 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556715b77560 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880c7880 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f618c003860 con 0x5567157346b0 2024-04-25T13:47:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 10 client.17496 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 10 client.17496 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 20 client.17496 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5567156e93a0 0x556715b79c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 4 client.17496 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556715a63800 con 0x5567156e93a0 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.463+0000 7f6196ffd640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5567156e93a0 0x556715b79c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f617aff5640 10 client.17496 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f6196ffd640 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5567156e93a0 0x556715b79c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7b72e32b79af1ea server_cookie=735594a221cd9b60 in_seq=0 out_seq=0 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f617aff5640 1 -- 172.21.15.94:0/322395225 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556715a63800 con 0x5567156e93a0 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f617aff5640 10 client.17496 handle_command_reply: tid=0 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 client.17496 shutdown 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 msgr2=0x556715b06720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f618c009be0 tx=0x7f618c009f00 comp rx=0 tx=0).stop 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 shutdown_connections 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5567156e93a0 0x556715b79c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556715755390 0x556715b07650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567157346b0 0x556715b06720 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f617aff5640 0 client.17496 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 --2- 172.21.15.94:0/322395225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55671572c4d0 0x556715b061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 >> 172.21.15.94:0/322395225 conn(0x5567157494e0 msgr2=0x556715af5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 shutdown_connections 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 1 -- 172.21.15.94:0/322395225 wait complete. 2024-04-25T13:47:17.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.464+0000 7f61a437f740 20 client.17496 trim_cache size 0 max 16384 2024-04-25T13:47:17.472 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:17.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 msgr2=0x7f61800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 0x7f61800b2000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556715af8860 tx=0x7f6190009500 comp rx=0 tx=0).stop 2024-04-25T13:47:17.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 msgr2=0x7f6198102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6188031440 tx=0x7f6188003c10 comp rx=0 tx=0).stop 2024-04-25T13:47:17.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 shutdown_connections 2024-04-25T13:47:17.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61800afaf0 0x7f61800b2000 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6198104470 0x7f61981a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6198103990 0x7f619819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 --2- 172.21.15.94:0/2470343053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6198102fb0 0x7f6198102220 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 >> 172.21.15.94:0/2470343053 conn(0x7f61980fd350 msgr2=0x7f6198104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 shutdown_connections 2024-04-25T13:47:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.465+0000 7f619d4d1640 1 -- 172.21.15.94:0/2470343053 wait complete. 2024-04-25T13:47:17.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 -- 172.21.15.94:0/1398875657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 msgr2=0x7f9570106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 --2- 172.21.15.94:0/1398875657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f9570106a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f956c009860 tx=0x7f956c02a340 comp rx=0 tx=0).stop 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 -- 172.21.15.94:0/1398875657 shutdown_connections 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 --2- 172.21.15.94:0/1398875657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95701098a0 0x7f957010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 --2- 172.21.15.94:0/1398875657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9570106f40 0x7f9570109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 --2- 172.21.15.94:0/1398875657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f9570106a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 -- 172.21.15.94:0/1398875657 >> 172.21.15.94:0/1398875657 conn(0x7f95700fdbc0 msgr2=0x7f9570100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 -- 172.21.15.94:0/1398875657 shutdown_connections 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.598+0000 7f9578267640 1 -- 172.21.15.94:0/1398875657 wait complete. 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 Processor -- start 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 -- start start 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701098a0 0x7f95701a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f957010f5f0 con 0x7f9570106f40 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f957010f470 con 0x7f95701045e0 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9578267640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f957010f770 con 0x7f95701098a0 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.599+0000 7f9575802640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39048/0 (socket says 172.21.15.94:39048) 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9575802640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37632/0 (socket says 172.21.15.94:37632) 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 -- 172.21.15.94:0/2871291107 learned_addr learned my addr 172.21.15.94:0/2871291107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576804640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701098a0 0x7f95701a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 -- 172.21.15.94:0/2871291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701098a0 msgr2=0x7f95701a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701098a0 0x7f95701a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 -- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 msgr2=0x7f957019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 -- 172.21.15.94:0/2871291107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95701a1d90 con 0x7f95701045e0 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9575802640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9576003640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f956c009780 tx=0x7f956c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f95677fe640 1 -- 172.21.15.94:0/2871291107 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037040 con 0x7f95701045e0 2024-04-25T13:47:17.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95701a2020 con 0x7f95701045e0 2024-04-25T13:47:17.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f95677fe640 1 -- 172.21.15.94:0/2871291107 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956c030c10 con 0x7f95701045e0 2024-04-25T13:47:17.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f95677fe640 1 -- 172.21.15.94:0/2871291107 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c003850 con 0x7f95701045e0 2024-04-25T13:47:17.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.600+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701a24b0 con 0x7f95701045e0 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 Processor -- start 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 -- start start 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe94fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe950a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe8b3d80 con 0x55ffbe5ad390 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe86da20 con 0x55ffbe5844d0 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f957f1f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe8afd40 con 0x55ffbe58c6b0 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37644/0 (socket says 172.21.15.94:37644) 2024-04-25T13:47:17.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.603+0000 7f9576804640 1 -- 172.21.15.94:0/146012376 learned_addr learned my addr 172.21.15.94:0/146012376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9575802640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe950a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f95677fe640 1 -- 172.21.15.94:0/2871291107 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f956c043020 con 0x7f95701045e0 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 -- 172.21.15.94:0/146012376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 msgr2=0x55ffbe950a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe950a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 -- 172.21.15.94:0/146012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 msgr2=0x55ffbe94fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576003640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe94fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe94fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 -- 172.21.15.94:0/146012376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffbe878c30 con 0x55ffbe5ad390 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576003640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe94fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9576804640 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9568009a40 tx=0x7f95680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8636e12f611a79cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9564bfe640 1 -- 172.21.15.94:0/146012376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003970 con 0x55ffbe5ad390 2024-04-25T13:47:17.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9564bfe640 1 -- 172.21.15.94:0/146012376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956800d6b0 con 0x55ffbe5ad390 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f9564bfe640 1 -- 172.21.15.94:0/146012376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568004410 con 0x55ffbe5ad390 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.604+0000 7f95677fe640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 0x7f955c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f9576003640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 0x7f955c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f95677fe640 1 -- 172.21.15.94:0/2871291107 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f956c0fe250 con 0x7f95701045e0 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 -- 172.21.15.94:0/146012376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 msgr2=0x55ffbe94f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f9576003640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 0x7f955c0b2000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ffbe94f530 tx=0x7f956c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9568009a40 tx=0x7f95680095c0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 -- 172.21.15.94:0/146012376 shutdown_connections 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe950a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe94fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 --2- 172.21.15.94:0/146012376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe94f810 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 -- 172.21.15.94:0/146012376 >> 172.21.15.94:0/146012376 conn(0x55ffbe5a14e0 msgr2=0x55ffbe94b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 -- 172.21.15.94:0/146012376 shutdown_connections 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.605+0000 7f957f1f9740 1 -- 172.21.15.94:0/146012376 wait complete. 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 -- start start 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 Processor -- start 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe9595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe959b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe951530 con 0x55ffbe5ad390 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe83adc0 con 0x55ffbe5844d0 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffbe9516b0 con 0x55ffbe58c6b0 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f957f1f9740 10 client.0 _ll_register_callbacks cb 0x55ffbe856600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37660/0 (socket says 172.21.15.94:37660) 2024-04-25T13:47:17.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f9575802640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe959b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f9576804640 1 -- 172.21.15.94:0/3985491540 learned_addr learned my addr 172.21.15.94:0/3985491540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.606+0000 7f9576804640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe9595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 -- 172.21.15.94:0/3985491540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 msgr2=0x55ffbe959b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe959b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 -- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 msgr2=0x55ffbe9595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe9595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 -- 172.21.15.94:0/3985491540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffbe5c53f0 con 0x55ffbe5ad390 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9575802640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe959b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9576804640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f956800dde0 tx=0x7f9568005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d092bfc764c1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003de0 con 0x55ffbe5ad390 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9568004190 con 0x55ffbe5ad390 2024-04-25T13:47:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956800d6b0 con 0x55ffbe5ad390 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9568004390 con 0x55ffbe5ad390 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f957f1f9740 10 client.17508 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffbe68f7d0 con 0x55ffbe5ad390 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956800d6b0 con 0x55ffbe5ad390 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.607+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f956800b040 con 0x55ffbe5ad390 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 10 client.17508 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 10 client.17508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 20 client.17508 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe56bf60 0x55ffbe966f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 4 client.17508 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffbe8c1990 con 0x55ffbe56bf60 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f9575802640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe56bf60 0x55ffbe966f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f9559ffb640 10 client.17508 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.608+0000 7f9575802640 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe56bf60 0x55ffbe966f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbc42abaa894d8d9 server_cookie=2f6d56f4291ab27a in_seq=0 out_seq=0 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f9559ffb640 1 -- 172.21.15.94:0/3985491540 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ffbe8c1990 con 0x55ffbe56bf60 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f9559ffb640 10 client.17508 handle_command_reply: tid=0 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 client.17508 shutdown 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 msgr2=0x55ffbe960a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f956800dde0 tx=0x7f9568005a10 comp rx=0 tx=0).stop 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 shutdown_connections 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe56bf60 0x55ffbe966f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe5ad390 0x55ffbe960a60 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe58c6b0 0x55ffbe959b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f9559ffb640 0 client.17508 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3985491540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe5844d0 0x55ffbe9595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 >> 172.21.15.94:0/3985491540 conn(0x55ffbe5a14e0 msgr2=0x55ffbe94a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 shutdown_connections 2024-04-25T13:47:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.609+0000 7f957f1f9740 1 -- 172.21.15.94:0/3985491540 wait complete. 2024-04-25T13:47:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.610+0000 7f957f1f9740 20 client.17508 trim_cache size 0 max 16384 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 Processor -- start 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 -- start start 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe56bf60 0x55ffbe96b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe960980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95701ab4e0 con 0x55ffbe58c6b0 2024-04-25T13:47:17.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95701ad020 con 0x55ffbe56bf60 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f957f1f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95701abbb0 con 0x55ffbe5844d0 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f9575802640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe960980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37666/0 (socket says 172.21.15.94:37666) 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.623+0000 7f9576804640 1 -- 172.21.15.94:0/3229521986 learned_addr learned my addr 172.21.15.94:0/3229521986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9576804640 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe56bf60 0x55ffbe96b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 -- 172.21.15.94:0/3229521986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe56bf60 msgr2=0x55ffbe96b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe56bf60 0x55ffbe96b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 -- 172.21.15.94:0/3229521986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 msgr2=0x55ffbe96bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 -- 172.21.15.94:0/3229521986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffbe960ec0 con 0x55ffbe5844d0 2024-04-25T13:47:17.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9576804640 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f9575802640 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe960980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ffbe967570 tx=0x7f9560005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5875e8d5683bef8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f955bfff640 1 -- 172.21.15.94:0/3229521986 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560026070 con 0x55ffbe5844d0 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f955bfff640 1 -- 172.21.15.94:0/3229521986 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600047a0 con 0x55ffbe5844d0 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.624+0000 7f955bfff640 1 -- 172.21.15.94:0/3229521986 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560003850 con 0x55ffbe5844d0 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- 172.21.15.94:0/3229521986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 msgr2=0x55ffbe960980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe960980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ffbe967570 tx=0x7f9560005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- 172.21.15.94:0/3229521986 shutdown_connections 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe58c6b0 0x55ffbe96bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe960980 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 --2- 172.21.15.94:0/3229521986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe56bf60 0x55ffbe96b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- 172.21.15.94:0/3229521986 >> 172.21.15.94:0/3229521986 conn(0x55ffbe5a14e0 msgr2=0x55ffbe954370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- 172.21.15.94:0/3229521986 shutdown_connections 2024-04-25T13:47:17.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- 172.21.15.94:0/3229521986 wait complete. 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 -- start start 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 Processor -- start 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.625+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 0x55ffbe95df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe58c6b0 0x55ffbe95f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95701ac950 con 0x55ffbe56bf60 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95701ac280 con 0x55ffbe58c6b0 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95701a94c0 con 0x55ffbe5844d0 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 10 client.0 _ll_register_callbacks cb 0x7f9560009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9576804640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 0x55ffbe95df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33602/0 (socket says 172.21.15.94:33602) 2024-04-25T13:47:17.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 -- 172.21.15.94:0/1503688935 learned_addr learned my addr 172.21.15.94:0/1503688935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9576804640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe58c6b0 0x55ffbe95f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 -- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe58c6b0 msgr2=0x55ffbe95f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe58c6b0 0x55ffbe95f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 -- 172.21.15.94:0/1503688935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 msgr2=0x55ffbe95df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 0x55ffbe95df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 -- 172.21.15.94:0/1503688935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffbe95f9a0 con 0x55ffbe5844d0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9576804640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 0x55ffbe95df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9575802640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f95600049d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3ff9b961484254e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560030020 con 0x55ffbe5844d0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9560005220 con 0x55ffbe5844d0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.626+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95600047a0 con 0x55ffbe5844d0 2024-04-25T13:47:17.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f956003b050 con 0x55ffbe5844d0 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 10 client.26875 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffbe960c50 con 0x55ffbe5844d0 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95600047a0 con 0x55ffbe5844d0 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9560022110 con 0x55ffbe5844d0 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 10 client.26875 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 10 client.26875 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 20 client.26875 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe5ad390 0x55ffbe9d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 4 client.26875 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffbe8bb800 con 0x55ffbe5ad390 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9576804640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe5ad390 0x55ffbe9d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9559ffb640 10 client.26875 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:17.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.627+0000 7f9576804640 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe5ad390 0x55ffbe9d1970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e61840686bfbd5dd server_cookie=d714613c19a93c73 in_seq=0 out_seq=0 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f9559ffb640 1 -- 172.21.15.94:0/1503688935 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffbe8bb800 con 0x55ffbe5ad390 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f9559ffb640 10 client.26875 handle_command_reply: tid=0 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 client.26875 shutdown 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 msgr2=0x55ffbe95e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f95600049d0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 shutdown_connections 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55ffbe5ad390 0x55ffbe9d1970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffbe58c6b0 0x55ffbe95f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffbe5844d0 0x55ffbe95e4b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 --2- 172.21.15.94:0/1503688935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffbe56bf60 0x55ffbe95df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f9559ffb640 0 client.26875 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 >> 172.21.15.94:0/1503688935 conn(0x55ffbe5a14e0 msgr2=0x55ffbe954370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 shutdown_connections 2024-04-25T13:47:17.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 1 -- 172.21.15.94:0/1503688935 wait complete. 2024-04-25T13:47:17.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.628+0000 7f957f1f9740 20 client.26875 trim_cache size 0 max 16384 2024-04-25T13:47:17.636 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 msgr2=0x7f955c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 0x7f955c0b2000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ffbe94f530 tx=0x7f956c005bd0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 msgr2=0x7f957019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f956c009780 tx=0x7f956c032d50 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 shutdown_connections 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f955c0afaf0 0x7f955c0b2000 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701098a0 0x7f95701a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9570106f40 0x7f957019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 --2- 172.21.15.94:0/2871291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95701045e0 0x7f957019a250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 >> 172.21.15.94:0/2871291107 conn(0x7f95700fdbc0 msgr2=0x7f9570105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 shutdown_connections 2024-04-25T13:47:17.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.629+0000 7f9578267640 1 -- 172.21.15.94:0/2871291107 wait complete. 2024-04-25T13:47:17.639 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/136782542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:17.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:47:17.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:17 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/136782542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/136782542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:17.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 -- 172.21.15.94:0/3902870260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc103c20 msgr2=0x7f3efc104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/3902870260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc103c20 0x7f3efc104030 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009900 tx=0x7f3ef002a410 comp rx=0 tx=0).stop 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 -- 172.21.15.94:0/3902870260 shutdown_connections 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/3902870260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc108920 0x7f3efc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/3902870260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc103c20 0x7f3efc104030 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/3902870260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 -- 172.21.15.94:0/3902870260 >> 172.21.15.94:0/3902870260 conn(0x7f3efc0fdbd0 msgr2=0x7f3efc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 -- 172.21.15.94:0/3902870260 shutdown_connections 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 -- 172.21.15.94:0/3902870260 wait complete. 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.752+0000 7f3f01cd9640 1 Processor -- start 2024-04-25T13:47:17.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 -- start start 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc103c20 0x7f3efc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10e500 con 0x7f3efc108920 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10e380 con 0x7f3efc103c20 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3f01cd9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10e680 con 0x7f3efc1029e0 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37694/0 (socket says 172.21.15.94:37694) 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 -- 172.21.15.94:0/873609345 learned_addr learned my addr 172.21.15.94:0/873609345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efaffd640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc103c20 0x7f3efc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 -- 172.21.15.94:0/873609345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 msgr2=0x7f3efc075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 -- 172.21.15.94:0/873609345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc103c20 msgr2=0x7f3efc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc103c20 0x7f3efc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 -- 172.21.15.94:0/873609345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a1de0 con 0x7f3efc108920 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.753+0000 7f3efbfff640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.754+0000 7f3ef8ff9640 1 -- 172.21.15.94:0/873609345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4004b20 con 0x7f3efc108920 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.754+0000 7f3ef8ff9640 1 -- 172.21.15.94:0/873609345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4004cc0 con 0x7f3efc108920 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.754+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a20d0 con 0x7f3efc108920 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.754+0000 7f3ef8ff9640 1 -- 172.21.15.94:0/873609345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4010670 con 0x7f3efc108920 2024-04-25T13:47:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.754+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a25d0 con 0x7f3efc108920 2024-04-25T13:47:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.756+0000 7f3f08b82740 1 Processor -- start 2024-04-25T13:47:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.756+0000 7f3ef8ff9640 1 -- 172.21.15.94:0/873609345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3ef4005020 con 0x7f3efc108920 2024-04-25T13:47:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.756+0000 7f3f08b82740 1 -- start start 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c846810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c847a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56380c7aad80 con 0x56380c47b4d0 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56380c764a20 con 0x56380c4836b0 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3f08b82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56380c7a6d40 con 0x56380c4a4390 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c846810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37698/0 (socket says 172.21.15.94:37698) 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efb7fe640 1 -- 172.21.15.94:0/1959866937 learned_addr learned my addr 172.21.15.94:0/1959866937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c847a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3ef8ff9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 0x7f3ee40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 -- 172.21.15.94:0/1959866937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 msgr2=0x56380c847a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c847a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 -- 172.21.15.94:0/1959866937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 msgr2=0x56380c846d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 -- 172.21.15.94:0/1959866937 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380c76fc30 con 0x56380c4a4390 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efbfff640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 0x7f3ee40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c846810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005ab0 tx=0x7f3ef0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d55d6c6a7b8acbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.757+0000 7f3ef8ff9640 1 -- 172.21.15.94:0/873609345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3ef40df260 con 0x7f3efc108920 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3ee1ffb640 1 -- 172.21.15.94:0/1959866937 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0035070 con 0x56380c4a4390 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3ee1ffb640 1 -- 172.21.15.94:0/1959866937 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef00318e0 con 0x56380c4a4390 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3ee1ffb640 1 -- 172.21.15.94:0/1959866937 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004760 con 0x56380c4a4390 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 -- 172.21.15.94:0/1959866937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 msgr2=0x56380c846810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c846810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005ab0 tx=0x7f3ef0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3efbfff640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 0x7f3ee40b2000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40a3720 tx=0x7f3ef401bd20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 -- 172.21.15.94:0/1959866937 shutdown_connections 2024-04-25T13:47:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c847a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c846d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1959866937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c846810 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.758+0000 7f3f08b82740 1 -- 172.21.15.94:0/1959866937 >> 172.21.15.94:0/1959866937 conn(0x56380c4984e0 msgr2=0x56380c842020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- 172.21.15.94:0/1959866937 shutdown_connections 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- 172.21.15.94:0/1959866937 wait complete. 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- start start 2024-04-25T13:47:17.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 Processor -- start 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c850b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c857af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56380c8484d0 con 0x56380c47b4d0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56380c731dc0 con 0x56380c4836b0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56380c848650 con 0x56380c4a4390 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.759+0000 7f3f08b82740 10 client.0 _ll_register_callbacks cb 0x56380c74d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37714/0 (socket says 172.21.15.94:37714) 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 -- 172.21.15.94:0/122831893 learned_addr learned my addr 172.21.15.94:0/122831893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c850b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c857af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 -- 172.21.15.94:0/122831893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 msgr2=0x56380c857af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c857af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 -- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 msgr2=0x56380c850b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c850b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 -- 172.21.15.94:0/122831893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380c4bc3f0 con 0x56380c47b4d0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3efaffd640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0032f60 tx=0x7f3ef0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8968223d81a1b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0035070 con 0x56380c47b4d0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3ef0031d10 con 0x56380c47b4d0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef00318e0 con 0x56380c47b4d0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ef0004030 con 0x56380c47b4d0 2024-04-25T13:47:17.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3f08b82740 10 client.17526 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.760+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56380c5867d0 con 0x56380c47b4d0 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef00318e0 con 0x56380c47b4d0 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef0003a80 con 0x56380c47b4d0 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 10 client.17526 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 20 client.17526 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c462f60 0x56380c85df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 4 client.17526 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380c7b8990 con 0x56380c462f60 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c462f60 0x56380c85df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3edaffd640 10 client.17526 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:17.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.761+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c462f60 0x56380c85df40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a18f2060f0fc6a0 server_cookie=393d15d7cdd69673 in_seq=0 out_seq=0 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.762+0000 7f3edaffd640 1 -- 172.21.15.94:0/122831893 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56380c7b8990 con 0x56380c462f60 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.762+0000 7f3edaffd640 10 client.17526 handle_command_reply: tid=0 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.762+0000 7f3f08b82740 1 client.17526 shutdown 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 msgr2=0x56380c850650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0032f60 tx=0x7f3ef0005d50 comp rx=0 tx=0).stop 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 shutdown_connections 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c462f60 0x56380c85df40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4a4390 0x56380c857af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3edaffd640 0 client.17526 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4836b0 0x56380c850b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 --2- 172.21.15.94:0/122831893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c850650 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 >> 172.21.15.94:0/122831893 conn(0x56380c4984e0 msgr2=0x56380c841040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 shutdown_connections 2024-04-25T13:47:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 1 -- 172.21.15.94:0/122831893 wait complete. 2024-04-25T13:47:17.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.763+0000 7f3f08b82740 20 client.17526 trim_cache size 0 max 16384 2024-04-25T13:47:17.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 Processor -- start 2024-04-25T13:47:17.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 -- start start 2024-04-25T13:47:17.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 0x56380c850220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1ab5d0 con 0x56380c4836b0 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1ad110 con 0x56380c4a4390 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3f08b82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1abca0 con 0x56380c47b4d0 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37718/0 (socket says 172.21.15.94:37718) 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 -- 172.21.15.94:0/1186396492 learned_addr learned my addr 172.21.15.94:0/1186396492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 0x56380c850220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 -- 172.21.15.94:0/1186396492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 msgr2=0x56380c850220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 0x56380c850220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 -- 172.21.15.94:0/1186396492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 msgr2=0x56380c8624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 -- 172.21.15.94:0/1186396492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380c84e770 con 0x56380c4836b0 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efaffd640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 0x56380c850220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x56380c8516c0 tx=0x7f3eec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e799cdd0a6f1ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.777+0000 7f3ee17fa640 1 -- 172.21.15.94:0/1186396492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec025070 con 0x56380c4836b0 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3ee17fa640 1 -- 172.21.15.94:0/1186396492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec010d90 con 0x56380c4836b0 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3ee17fa640 1 -- 172.21.15.94:0/1186396492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec003be0 con 0x56380c4836b0 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 -- 172.21.15.94:0/1186396492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 msgr2=0x56380c850760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x56380c8516c0 tx=0x7f3eec005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 -- 172.21.15.94:0/1186396492 shutdown_connections 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c4836b0 0x56380c850760 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 --2- 172.21.15.94:0/1186396492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c47b4d0 0x56380c850220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 -- 172.21.15.94:0/1186396492 >> 172.21.15.94:0/1186396492 conn(0x56380c4984e0 msgr2=0x56380c84b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 -- 172.21.15.94:0/1186396492 shutdown_connections 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.778+0000 7f3f08b82740 1 -- 172.21.15.94:0/1186396492 wait complete. 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 -- start start 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 Processor -- start 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 0x56380c854500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1aca40 con 0x56380c47b4d0 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1ac370 con 0x56380c4a4390 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1a95b0 con 0x56380c4836b0 2024-04-25T13:47:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3f08b82740 10 client.0 _ll_register_callbacks cb 0x7f3eec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3efaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37730/0 (socket says 172.21.15.94:37730) 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3efaffd640 1 -- 172.21.15.94:0/70145275 learned_addr learned my addr 172.21.15.94:0/70145275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 0x56380c854500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.779+0000 7f3efaffd640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 -- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 msgr2=0x56380c8c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 -- 172.21.15.94:0/70145275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 msgr2=0x56380c853fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 -- 172.21.15.94:0/70145275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380c854b10 con 0x56380c4836b0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efaffd640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3efb7fe640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 0x56380c854500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3eec014010 tx=0x7f3eec003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9340aae8a87eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec025040 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3eec004050 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eec010d90 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3eec004270 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3f08b82740 10 client.26893 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56380c8c5610 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec010d90 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.780+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3eec0053b0 con 0x56380c4836b0 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 10 client.26893 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 10 client.26893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 20 client.26893 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c4383a0 0x56380c8c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:17.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 4 client.26893 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56380c7b2800 con 0x56380c4383a0 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3efaffd640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c4383a0 0x56380c8c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3efaffd640 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c4383a0 0x56380c8c7f80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81730d65cb1f202a server_cookie=78d443f10c21bad9 in_seq=0 out_seq=0 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3edaffd640 10 client.26893 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3edaffd640 1 -- 172.21.15.94:0/70145275 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56380c7b2800 con 0x56380c4383a0 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3edaffd640 10 client.26893 handle_command_reply: tid=0 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.781+0000 7f3f08b82740 1 client.26893 shutdown 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 msgr2=0x56380c854500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 0x56380c854500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3eec014010 tx=0x7f3eec003680 comp rx=0 tx=0).stop 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 shutdown_connections 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56380c4383a0 0x56380c8c7f80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56380c4a4390 0x56380c8c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56380c4836b0 0x56380c854500 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 --2- 172.21.15.94:0/70145275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56380c47b4d0 0x56380c853fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3edaffd640 0 client.26893 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 >> 172.21.15.94:0/70145275 conn(0x56380c4984e0 msgr2=0x56380c850220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 shutdown_connections 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 1 -- 172.21.15.94:0/70145275 wait complete. 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f08b82740 20 client.26893 trim_cache size 0 max 16384 2024-04-25T13:47:17.789 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:17.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 msgr2=0x7f3ee40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 0x7f3ee40b2000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40a3720 tx=0x7f3ef401bd20 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 msgr2=0x7f3efc1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 shutdown_connections 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ee40afaf0 0x7f3ee40b2000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3efc108920 0x7f3efc1a15c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3efc103c20 0x7f3efc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 --2- 172.21.15.94:0/873609345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3efc1029e0 0x7f3efc075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 >> 172.21.15.94:0/873609345 conn(0x7f3efc0fdbd0 msgr2=0x7f3efc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.782+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 shutdown_connections 2024-04-25T13:47:17.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:17.783+0000 7f3f01cd9640 1 -- 172.21.15.94:0/873609345 wait complete. 2024-04-25T13:47:17.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 600786 13.25 MB/sec execute 120 sec latency 13.986 ms 2024-04-25T13:47:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:18 smithi137 ceph-mon[32568]: pgmap v512: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:47:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:18 smithi094 ceph-mon[29195]: pgmap v512: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:47:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:18 smithi138 ceph-mon[32977]: pgmap v512: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-04-25T13:47:18.794 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:47:18.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:18.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:47:18.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 603153 13.24 MB/sec execute 121 sec latency 68.813 ms 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/1038706418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68108920 msgr2=0x7ffb6810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/1038706418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68108920 0x7ffb6810ad40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009ca0 tx=0x7ffb6002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/1038706418 shutdown_connections 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/1038706418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68108920 0x7ffb6810ad40 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/1038706418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb68103c20 0x7ffb68104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/1038706418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681029e0 0x7ffb681036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/1038706418 >> 172.21.15.94:0/1038706418 conn(0x7ffb680fdbd0 msgr2=0x7ffb68100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/1038706418 shutdown_connections 2024-04-25T13:47:18.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.911+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/1038706418 wait complete. 2024-04-25T13:47:18.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 Processor -- start 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 -- start start 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68103c20 0x7ffb6819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68108920 0x7ffb681a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f610 con 0x7ffb68103c20 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f490 con 0x7ffb68108920 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6d1a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810f790 con 0x7ffb681029e0 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68103c20 0x7ffb6819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.912+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33670/0 (socket says 172.21.15.94:33670) 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 -- 172.21.15.94:0/2373628554 learned_addr learned my addr 172.21.15.94:0/2373628554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:18.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb6759e640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68108920 0x7ffb681a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 -- 172.21.15.94:0/2373628554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68108920 msgr2=0x7ffb681a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68108920 0x7ffb681a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 -- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68103c20 msgr2=0x7ffb6819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68103c20 0x7ffb6819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 -- 172.21.15.94:0/2373628554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb681a4280 con 0x7ffb681029e0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb4ffff640 1 -- 172.21.15.94:0/2373628554 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00ccc0 con 0x7ffb681029e0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb4ffff640 1 -- 172.21.15.94:0/2373628554 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb5c0035f0 con 0x7ffb681029e0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb681a4570 con 0x7ffb681029e0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.913+0000 7ffb4ffff640 1 -- 172.21.15.94:0/2373628554 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00ce60 con 0x7ffb681029e0 2024-04-25T13:47:18.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.914+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a4b10 con 0x7ffb681029e0 2024-04-25T13:47:18.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb4ffff640 1 -- 172.21.15.94:0/2373628554 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ffb5c003aa0 con 0x7ffb681029e0 2024-04-25T13:47:18.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 Processor -- start 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 -- start start 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec16ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec173920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dec0d1b00 con 0x561debdaa730 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dec06d920 con 0x561debdcb410 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.916+0000 7ffb74051740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561dec0cdac0 con 0x561debda2550 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec173920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec16ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39172/0 (socket says 172.21.15.94:39172) 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6759e640 1 -- 172.21.15.94:0/2097942710 learned_addr learned my addr 172.21.15.94:0/2097942710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb4ffff640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 0x7ffb500b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 -- 172.21.15.94:0/2097942710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 msgr2=0x561dec16ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec16ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 -- 172.21.15.94:0/2097942710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 msgr2=0x561dec16c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 -- 172.21.15.94:0/2097942710 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561debeab7e0 con 0x561debdaa730 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6759e640 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6759e640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 0x7ffb500b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb4ffff640 1 -- 172.21.15.94:0/2373628554 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffb5c0def40 con 0x7ffb681029e0 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb6659c640 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec173920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009bd0 tx=0x7ffb58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96959235110fc853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb4d7fa640 1 -- 172.21.15.94:0/2097942710 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004230 con 0x561debdaa730 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb4d7fa640 1 -- 172.21.15.94:0/2097942710 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb58009e40 con 0x561debdaa730 2024-04-25T13:47:18.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.917+0000 7ffb4d7fa640 1 -- 172.21.15.94:0/2097942710 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004d00 con 0x561debdaa730 2024-04-25T13:47:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 -- 172.21.15.94:0/2097942710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 msgr2=0x561dec173920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb6759e640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 0x7ffb500b2080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009a60 tx=0x7ffb600095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec173920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009bd0 tx=0x7ffb58011810 comp rx=0 tx=0).stop 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 -- 172.21.15.94:0/2097942710 shutdown_connections 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec173920 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec16ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 --2- 172.21.15.94:0/2097942710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec16c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 -- 172.21.15.94:0/2097942710 >> 172.21.15.94:0/2097942710 conn(0x561debdbf560 msgr2=0x561dec167e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 -- 172.21.15.94:0/2097942710 shutdown_connections 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.918+0000 7ffb74051740 1 -- 172.21.15.94:0/2097942710 wait complete. 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 -- start start 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 Processor -- start 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec176270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec17d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dec174370 con 0x561debdaa730 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dec06b5d0 con 0x561debdcb410 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561dec1744f0 con 0x561debda2550 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb74051740 10 client.0 _ll_register_callbacks cb 0x561dec059080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37764/0 (socket says 172.21.15.94:37764) 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb6659c640 1 -- 172.21.15.94:0/52855185 learned_addr learned my addr 172.21.15.94:0/52855185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec176270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.919+0000 7ffb6759e640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec17d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 -- 172.21.15.94:0/52855185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 msgr2=0x561dec176270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec176270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 -- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 msgr2=0x561dec17d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec17d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 -- 172.21.15.94:0/52855185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561debf0bf70 con 0x561debdaa730 2024-04-25T13:47:18.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb6659c640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098b0 tx=0x7ffb580047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7dfa1ae96eaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58027020 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffb580038f0 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561dec0df710 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb58032050 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb74051740 10 client.17544 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561debda3170 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.920+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561dec0df710 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffb58018070 con 0x561debdaa730 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 10 client.17544 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 20 client.17544 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd8a1f0 0x561dec183980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 4 client.17544 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.921+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dec0d9580 con 0x561debd8a1f0 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.922+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd8a1f0 0x561dec183980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.922+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd8a1f0 0x561dec183980 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1571ff6788d75ea server_cookie=c7f248fbc76528c8 in_seq=0 out_seq=0 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.922+0000 7ffb4aff5640 10 client.17544 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb4aff5640 1 -- 172.21.15.94:0/52855185 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dec0d9580 con 0x561debd8a1f0 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb4aff5640 10 client.17544 handle_command_reply: tid=0 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 client.17544 shutdown 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 msgr2=0x561dec1767b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098b0 tx=0x7ffb580047a0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 shutdown_connections 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd8a1f0 0x561dec183980 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec17d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec1767b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb4aff5640 0 client.17544 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 --2- 172.21.15.94:0/52855185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec176270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 >> 172.21.15.94:0/52855185 conn(0x561debdbf560 msgr2=0x561dec166ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 shutdown_connections 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 1 -- 172.21.15.94:0/52855185 wait complete. 2024-04-25T13:47:18.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.923+0000 7ffb74051740 20 client.17544 trim_cache size 0 max 16384 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.941+0000 7ffb74051740 1 Processor -- start 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 -- start start 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec168050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec174fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ad7b0 con 0x561debdaa730 2024-04-25T13:47:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681af2f0 con 0x561debdcb410 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb74051740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ade80 con 0x561debda2550 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37766/0 (socket says 172.21.15.94:37766) 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 -- 172.21.15.94:0/721549712 learned_addr learned my addr 172.21.15.94:0/721549712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec168050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6759e640 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec174fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 -- 172.21.15.94:0/721549712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 msgr2=0x561dec168050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec168050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 -- 172.21.15.94:0/721549712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 msgr2=0x561dec174fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec174fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 -- 172.21.15.94:0/721549712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dec162160 con 0x561debdaa730 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb6659c640 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ffb58011850 tx=0x7ffb58011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5667e84725e6d8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb4cff9640 1 -- 172.21.15.94:0/721549712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58027070 con 0x561debdaa730 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.942+0000 7ffb4cff9640 1 -- 172.21.15.94:0/721549712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb58023ac0 con 0x561debdaa730 2024-04-25T13:47:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb4cff9640 1 -- 172.21.15.94:0/721549712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5801e040 con 0x561debdaa730 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 -- 172.21.15.94:0/721549712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 msgr2=0x561dec170d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ffb58011850 tx=0x7ffb58011d10 comp rx=0 tx=0).stop 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 -- 172.21.15.94:0/721549712 shutdown_connections 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec174fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debdaa730 0x561dec170d60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 --2- 172.21.15.94:0/721549712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debda2550 0x561dec168050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 -- 172.21.15.94:0/721549712 >> 172.21.15.94:0/721549712 conn(0x561debdbf560 msgr2=0x561dec174890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 -- 172.21.15.94:0/721549712 shutdown_connections 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.943+0000 7ffb74051740 1 -- 172.21.15.94:0/721549712 wait complete. 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 -- start start 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 Processor -- start 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debdaa730 0x561dec17b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec1e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681aec20 con 0x561debda2550 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ae550 con 0x561debdcb410 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681ab790 con 0x561debdaa730 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb74051740 10 client.0 _ll_register_callbacks cb 0x7ffb58009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37782/0 (socket says 172.21.15.94:37782) 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 -- 172.21.15.94:0/175796241 learned_addr learned my addr 172.21.15.94:0/175796241 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb6659c640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debdaa730 0x561dec17b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb6759e640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec1e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 -- 172.21.15.94:0/175796241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debdaa730 msgr2=0x561dec17b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debdaa730 0x561dec17b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 -- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 msgr2=0x561dec1e9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec1e9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.944+0000 7ffb66d9d640 1 -- 172.21.15.94:0/175796241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dec17e0f0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb66d9d640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0dc9a0 tx=0x7ffb5c0a7210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49434ff61f6150db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0dce60 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ffb5c0e10b0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb58023ac0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb5c0a6e90 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 10 client.17550 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561dec1eb1a0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb58023ac0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffb5c0a75e0 con 0x561debda2550 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 10 client.17550 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:18.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 20 client.17550 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd5f630 0x561dec1edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 4 client.17550 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.945+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561dec054ac0 con 0x561debd5f630 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.946+0000 7ffb6659c640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd5f630 0x561dec1edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.946+0000 7ffb4aff5640 10 client.17550 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.946+0000 7ffb6659c640 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd5f630 0x561dec1edb60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9cfe5f25903c63 server_cookie=3bdc58918f04cf42 in_seq=0 out_seq=0 2024-04-25T13:47:18.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.947+0000 7ffb4aff5640 1 -- 172.21.15.94:0/175796241 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561dec054ac0 con 0x561debd5f630 2024-04-25T13:47:18.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.947+0000 7ffb4aff5640 10 client.17550 handle_command_reply: tid=0 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.947+0000 7ffb74051740 1 client.17550 shutdown 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.947+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 msgr2=0x561dec17abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.947+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0dc9a0 tx=0x7ffb5c0a7210 comp rx=0 tx=0).stop 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 shutdown_connections 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561debd5f630 0x561dec1edb60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561debdcb410 0x561dec1e9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb4aff5640 0 client.17550 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561debdaa730 0x561dec17b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 --2- 172.21.15.94:0/175796241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561debda2550 0x561dec17abf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 >> 172.21.15.94:0/175796241 conn(0x561debdbf560 msgr2=0x561dec175020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 shutdown_connections 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 1 -- 172.21.15.94:0/175796241 wait complete. 2024-04-25T13:47:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb74051740 20 client.17550 trim_cache size 0 max 16384 2024-04-25T13:47:18.954 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:47:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "c7693b23-b7de-491a-916d-a26feb06e5b9", 2024-04-25T13:47:18.954 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:47:18.954 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 msgr2=0x7ffb500b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 0x7ffb500b2080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009a60 tx=0x7ffb600095c0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 msgr2=0x7ffb6819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 shutdown_connections 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ffb500afb70 0x7ffb500b2080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb68108920 0x7ffb681a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb68103c20 0x7ffb6819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 --2- 172.21.15.94:0/2373628554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb681029e0 0x7ffb6819c740 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.948+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 >> 172.21.15.94:0/2373628554 conn(0x7ffb680fdbd0 msgr2=0x7ffb680ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.949+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 shutdown_connections 2024-04-25T13:47:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:18.949+0000 7ffb6d1a8640 1 -- 172.21.15.94:0/2373628554 wait complete. 2024-04-25T13:47:18.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:47:18.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:47:19.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 -- 172.21.15.94:0/292265251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 msgr2=0x7faf881036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 --2- 172.21.15.94:0/292265251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf881036e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7faf78009900 tx=0x7faf7802a410 comp rx=0 tx=0).stop 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 -- 172.21.15.94:0/292265251 shutdown_connections 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 --2- 172.21.15.94:0/292265251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 0x7faf8810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 --2- 172.21.15.94:0/292265251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 0x7faf88104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 --2- 172.21.15.94:0/292265251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf881036e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 -- 172.21.15.94:0/292265251 >> 172.21.15.94:0/292265251 conn(0x7faf880fdbd0 msgr2=0x7faf88100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 -- 172.21.15.94:0/292265251 shutdown_connections 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.075+0000 7faf8e706640 1 -- 172.21.15.94:0/292265251 wait complete. 2024-04-25T13:47:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 Processor -- start 2024-04-25T13:47:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 -- start start 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 0x7faf8819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 0x7faf881a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810e4e0 con 0x7faf881029e0 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810e360 con 0x7faf88103c20 2024-04-25T13:47:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8e706640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810e660 con 0x7faf88108920 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37794/0 (socket says 172.21.15.94:37794) 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 -- 172.21.15.94:0/365589355 learned_addr learned my addr 172.21.15.94:0/365589355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf8cca3640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 0x7faf881a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf877fe640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 0x7faf8819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 -- 172.21.15.94:0/365589355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 msgr2=0x7faf881a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 0x7faf881a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 -- 172.21.15.94:0/365589355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 msgr2=0x7faf8819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 0x7faf8819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.076+0000 7faf87fff640 1 -- 172.21.15.94:0/365589355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf881a2090 con 0x7faf881029e0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf87fff640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7faf78006010 tx=0x7faf78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf857fa640 1 -- 172.21.15.94:0/365589355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7803f030 con 0x7faf881029e0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf857fa640 1 -- 172.21.15.94:0/365589355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf78031910 con 0x7faf881029e0 2024-04-25T13:47:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf881a2320 con 0x7faf881029e0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf857fa640 1 -- 172.21.15.94:0/365589355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78035070 con 0x7faf881029e0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.077+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf881a2810 con 0x7faf881029e0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 Processor -- start 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 -- start start 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 0x5570a7404a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7404fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570a7369d80 con 0x5570a70426b0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5570a73238b0 con 0x5570a7063390 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf955ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570a7365d40 con 0x5570a703a4d0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf857fa640 1 -- 172.21.15.94:0/365589355 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7faf78042030 con 0x7faf881029e0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37798/0 (socket says 172.21.15.94:37798) 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf8cca3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 0x5570a7404a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf877fe640 1 -- 172.21.15.94:0/243008985 learned_addr learned my addr 172.21.15.94:0/243008985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf877fe640 1 -- 172.21.15.94:0/243008985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 msgr2=0x5570a7404fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.080+0000 7faf877fe640 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7404fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf877fe640 1 -- 172.21.15.94:0/243008985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 msgr2=0x5570a7404a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf877fe640 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 0x5570a7404a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf877fe640 1 -- 172.21.15.94:0/243008985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a72f4a40 con 0x5570a70426b0 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf8cca3640 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 0x5570a7404a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf877fe640 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009c90 tx=0x7faf7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e489868470e5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf857fa640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 0x7faf700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf6e7fc640 1 -- 172.21.15.94:0/243008985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c005220 con 0x5570a70426b0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf6e7fc640 1 -- 172.21.15.94:0/243008985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c004c50 con 0x5570a70426b0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf8cca3640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 0x7faf700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf6e7fc640 1 -- 172.21.15.94:0/243008985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0196b0 con 0x5570a70426b0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.081+0000 7faf857fa640 1 -- 172.21.15.94:0/365589355 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faf780feff0 con 0x7faf881029e0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- 172.21.15.94:0/243008985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 msgr2=0x5570a7405c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009c90 tx=0x7faf7c011810 comp rx=0 tx=0).stop 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf8cca3640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 0x7faf700b2000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5570a7405ab0 tx=0x7faf80009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- 172.21.15.94:0/243008985 shutdown_connections 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a7405c70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7404fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 --2- 172.21.15.94:0/243008985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a7063390 0x5570a7404a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- 172.21.15.94:0/243008985 >> 172.21.15.94:0/243008985 conn(0x5570a70574e0 msgr2=0x5570a7400270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- 172.21.15.94:0/243008985 shutdown_connections 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- 172.21.15.94:0/243008985 wait complete. 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.082+0000 7faf955ba740 1 -- start start 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 Processor -- start 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 0x5570a740e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a7063390 0x5570a7415e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570a7406780 con 0x5570a70426b0 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5570a72f0d10 con 0x5570a703a4d0 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570a7406900 con 0x5570a7063390 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf955ba740 10 client.0 _ll_register_callbacks cb 0x5570a731d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37802/0 (socket says 172.21.15.94:37802) 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 -- 172.21.15.94:0/1924150409 learned_addr learned my addr 172.21.15.94:0/1924150409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf87fff640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 0x5570a740e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf8cca3640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a7063390 0x5570a7415e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 -- 172.21.15.94:0/1924150409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a7063390 msgr2=0x5570a7415e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a7063390 0x5570a7415e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 -- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 msgr2=0x5570a740e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 0x5570a740e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf877fe640 1 -- 172.21.15.94:0/1924150409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a71457d0 con 0x5570a70426b0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.083+0000 7faf87fff640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 0x5570a740e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf877fe640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7faf7c01b040 tx=0x7faf7c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d295033b21f1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c029070 con 0x5570a70426b0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faf7c023840 con 0x5570a70426b0 2024-04-25T13:47:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c004c50 con 0x5570a70426b0 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf7c035050 con 0x5570a70426b0 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 10 client.17562 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570a707b3f0 con 0x5570a70426b0 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7c004c50 con 0x5570a70426b0 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faf7c0196d0 con 0x5570a70426b0 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 10 client.17562 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 20 client.17562 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7021f60 0x5570a741c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 4 client.17562 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570a7377990 con 0x5570a7021f60 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.084+0000 7faf87fff640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7021f60 0x5570a741c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.085+0000 7faf677fe640 10 client.17562 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.085+0000 7faf87fff640 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7021f60 0x5570a741c090 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ce6ced036c315c server_cookie=373e2a7747b42902 in_seq=0 out_seq=0 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf677fe640 1 -- 172.21.15.94:0/1924150409 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570a7377990 con 0x5570a7021f60 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf677fe640 10 client.17562 handle_command_reply: tid=0 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 client.17562 shutdown 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 msgr2=0x5570a740eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7faf7c01b040 tx=0x7faf7c004970 comp rx=0 tx=0).stop 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 shutdown_connections 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7021f60 0x5570a741c090 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a7063390 0x5570a7415e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf677fe640 0 client.17562 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a70426b0 0x5570a740eee0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 --2- 172.21.15.94:0/1924150409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a703a4d0 0x5570a740e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 >> 172.21.15.94:0/1924150409 conn(0x5570a70574e0 msgr2=0x5570a73ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.086+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 shutdown_connections 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.087+0000 7faf955ba740 1 -- 172.21.15.94:0/1924150409 wait complete. 2024-04-25T13:47:19.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.087+0000 7faf955ba740 20 client.17562 trim_cache size 0 max 16384 2024-04-25T13:47:19.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.104+0000 7faf955ba740 1 Processor -- start 2024-04-25T13:47:19.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.104+0000 7faf955ba740 1 -- start start 2024-04-25T13:47:19.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7403780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7403cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ab530 con 0x5570a7021f60 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ad070 con 0x5570a70426b0 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf955ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf881abc00 con 0x5570a703a4d0 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37816/0 (socket says 172.21.15.94:37816) 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 -- 172.21.15.94:0/2551418331 learned_addr learned my addr 172.21.15.94:0/2551418331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf8cca3640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7403cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf877fe640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7403780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 -- 172.21.15.94:0/2551418331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 msgr2=0x5570a7403780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7403780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 -- 172.21.15.94:0/2551418331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 msgr2=0x5570a7403cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7403cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 -- 172.21.15.94:0/2551418331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a7409dc0 con 0x5570a7021f60 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf8cca3640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7403cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf87fff640 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7faf780c3560 tx=0x7faf7803b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39f7370042d7a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf6dffb640 1 -- 172.21.15.94:0/2551418331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78049040 con 0x5570a7021f60 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf6dffb640 1 -- 172.21.15.94:0/2551418331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf780d0290 con 0x5570a7021f60 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.105+0000 7faf6dffb640 1 -- 172.21.15.94:0/2551418331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780c3900 con 0x5570a7021f60 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 -- 172.21.15.94:0/2551418331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 msgr2=0x5570a741c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7faf780c3560 tx=0x7faf7803b4f0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 -- 172.21.15.94:0/2551418331 shutdown_connections 2024-04-25T13:47:19.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7403cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7403780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 --2- 172.21.15.94:0/2551418331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a741c000 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 -- 172.21.15.94:0/2551418331 >> 172.21.15.94:0/2551418331 conn(0x5570a70574e0 msgr2=0x5570a7403070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 -- 172.21.15.94:0/2551418331 shutdown_connections 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.106+0000 7faf955ba740 1 -- 172.21.15.94:0/2551418331 wait complete. 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 -- start start 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 Processor -- start 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7413f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7414ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ac9a0 con 0x5570a7021f60 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf881ac2d0 con 0x5570a70426b0 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf881a9510 con 0x5570a703a4d0 2024-04-25T13:47:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf955ba740 10 client.0 _ll_register_callbacks cb 0x7faf80009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37830/0 (socket says 172.21.15.94:37830) 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 -- 172.21.15.94:0/3732134151 learned_addr learned my addr 172.21.15.94:0/3732134151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf877fe640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7413f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf8cca3640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7414ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 -- 172.21.15.94:0/3732134151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 msgr2=0x5570a7413f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7413f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 -- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 msgr2=0x5570a7414ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7414ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.107+0000 7faf87fff640 1 -- 172.21.15.94:0/3732134151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a703b0f0 con 0x5570a7021f60 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf87fff640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7faf780fcaa0 tx=0x7faf780c73d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c23fef18df8d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780e0070 con 0x5570a7021f60 2024-04-25T13:47:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faf781010c0 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf780d0290 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf78049070 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 10 client.17568 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570a732a4b0 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf780d0290 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faf780cfe00 con 0x5570a7021f60 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 10 client.17568 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 20 client.17568 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7063390 0x5570a7486510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 4 client.17568 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570a7371800 con 0x5570a7063390 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.108+0000 7faf877fe640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7063390 0x5570a7486510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.109+0000 7faf677fe640 10 client.17568 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.109+0000 7faf877fe640 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7063390 0x5570a7486510 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323a53e1ddc99c26 server_cookie=ce1c5b7297016a81 in_seq=0 out_seq=0 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.110+0000 7faf677fe640 1 -- 172.21.15.94:0/3732134151 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5570a7371800 con 0x5570a7063390 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.110+0000 7faf677fe640 10 client.17568 handle_command_reply: tid=0 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 client.17568 shutdown 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 msgr2=0x5570a7413a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7faf780fcaa0 tx=0x7faf780c73d0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 shutdown_connections 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5570a7063390 0x5570a7486510 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570a70426b0 0x5570a7414ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570a703a4d0 0x5570a7413f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf677fe640 0 client.17568 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 --2- 172.21.15.94:0/3732134151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a7021f60 0x5570a7413a30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 >> 172.21.15.94:0/3732134151 conn(0x5570a70574e0 msgr2=0x5570a7403aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 shutdown_connections 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 1 -- 172.21.15.94:0/3732134151 wait complete. 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf955ba740 20 client.17568 trim_cache size 0 max 16384 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: "c7693b23-b7de-491a-916d-a26feb06e5b9": { 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:47:19.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "c7693b23-b7de-491a-916d-a26feb06e5b9", 2024-04-25T13:47:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:47:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:47:19.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:47:19.120 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 msgr2=0x7faf700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 0x7faf700b2000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5570a7405ab0 tx=0x7faf80009500 comp rx=0 tx=0).stop 2024-04-25T13:47:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 msgr2=0x7faf8819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.111+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7faf78006010 tx=0x7faf78004270 comp rx=0 tx=0).stop 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 shutdown_connections 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faf700afaf0 0x7faf700b2000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf88108920 0x7faf881a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf88103c20 0x7faf8819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 --2- 172.21.15.94:0/365589355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf881029e0 0x7faf8819a550 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 >> 172.21.15.94:0/365589355 conn(0x7faf880fdbd0 msgr2=0x7faf88109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 shutdown_connections 2024-04-25T13:47:19.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:19.112+0000 7faf8e706640 1 -- 172.21.15.94:0/365589355 wait complete. 2024-04-25T13:47:19.124 INFO:tasks.cephfs.filesystem:scrub status for tag:c7693b23-b7de-491a-916d-a26feb06e5b9 - {'path': '/', 'tag': 'c7693b23-b7de-491a-916d-a26feb06e5b9', 'options': 'recursive,force'} 2024-04-25T13:47:19.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 604987 13.20 MB/sec execute 122 sec latency 15.148 ms 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 -- 172.21.15.94:0/3983788796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 msgr2=0x7fe934103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 --2- 172.21.15.94:0/3983788796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe934103720 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93002a410 comp rx=0 tx=0).stop 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 -- 172.21.15.94:0/3983788796 shutdown_connections 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 --2- 172.21.15.94:0/3983788796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe93410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 --2- 172.21.15.94:0/3983788796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 0x7fe934104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 --2- 172.21.15.94:0/3983788796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe934103720 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 -- 172.21.15.94:0/3983788796 >> 172.21.15.94:0/3983788796 conn(0x7fe9340fdbf0 msgr2=0x7fe9341000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 -- 172.21.15.94:0/3983788796 shutdown_connections 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.860+0000 7fe93b496640 1 -- 172.21.15.94:0/3983788796 wait complete. 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 Processor -- start 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 -- start start 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 0x7fe93419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe9341a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410f650 con 0x7fe934102a20 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410f4d0 con 0x7fe934103c60 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe93b496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410f7d0 con 0x7fe934108960 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37842/0 (socket says 172.21.15.94:37842) 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 -- 172.21.15.94:0/1532661057 learned_addr learned my addr 172.21.15.94:0/1532661057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe938a31640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 0x7fe93419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939a33640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe9341a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 -- 172.21.15.94:0/1532661057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 msgr2=0x7fe9341a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe9341a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 -- 172.21.15.94:0/1532661057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 msgr2=0x7fe93419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 0x7fe93419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 -- 172.21.15.94:0/1532661057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9341a1d70 con 0x7fe934102a20 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939a33640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe9341a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe939232640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe930006010 tx=0x7fe930004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.861+0000 7fe92a7fc640 1 -- 172.21.15.94:0/1532661057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93003f030 con 0x7fe934102a20 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.862+0000 7fe92a7fc640 1 -- 172.21.15.94:0/1532661057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930031910 con 0x7fe934102a20 2024-04-25T13:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.862+0000 7fe92a7fc640 1 -- 172.21.15.94:0/1532661057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930035070 con 0x7fe934102a20 2024-04-25T13:47:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.862+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9341a2000 con 0x7fe934102a20 2024-04-25T13:47:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.862+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a24f0 con 0x7fe934102a20 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe92a7fc640 1 -- 172.21.15.94:0/1532661057 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe930042030 con 0x7fe934102a20 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 Processor -- start 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 -- start start 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 0x55d9246e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243274d0 0x55d9246efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d92464e460 con 0x55d92431f2f0 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d924608590 con 0x55d9243481b0 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.864+0000 7fe942429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d92464a420 con 0x55d9243274d0 2024-04-25T13:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 0x55d9246e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939a33640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939a33640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39280/0 (socket says 172.21.15.94:39280) 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939a33640 1 -- 172.21.15.94:0/2135732083 learned_addr learned my addr 172.21.15.94:0/2135732083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe938a31640 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243274d0 0x55d9246efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe92a7fc640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 0x7fe9200b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 -- 172.21.15.94:0/2135732083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243274d0 msgr2=0x55d9246efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243274d0 0x55d9246efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 -- 172.21.15.94:0/2135732083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 msgr2=0x55d9246e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 -- 172.21.15.94:0/2135732083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9246150d0 con 0x55d92431f2f0 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939a33640 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939a33640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 0x7fe9200b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe92a7fc640 1 -- 172.21.15.94:0/1532661057 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe9300ff090 con 0x7fe934102a20 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe939232640 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 0x55d9246e8ef0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe9300fcac0 tx=0x7fe9300f5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670fb8f756338e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe91f7fe640 1 -- 172.21.15.94:0/2135732083 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930049070 con 0x55d92431f2f0 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe91f7fe640 1 -- 172.21.15.94:0/2135732083 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9300c6fe0 con 0x55d92431f2f0 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.865+0000 7fe91f7fe640 1 -- 172.21.15.94:0/2135732083 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9300c72a0 con 0x55d92431f2f0 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 -- 172.21.15.94:0/2135732083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 msgr2=0x55d9246e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe939a33640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 0x7fe9200b1ff0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009fd0 tx=0x7fe92c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 0x55d9246e8ef0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe9300fcac0 tx=0x7fe9300f5c20 comp rx=0 tx=0).stop 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 -- 172.21.15.94:0/2135732083 shutdown_connections 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243274d0 0x55d9246efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d92431f2f0 0x55d9246e8ef0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 --2- 172.21.15.94:0/2135732083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 -- 172.21.15.94:0/2135732083 >> 172.21.15.94:0/2135732083 conn(0x55d92433c300 msgr2=0x55d9246e4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 -- 172.21.15.94:0/2135732083 shutdown_connections 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.866+0000 7fe942429740 1 -- 172.21.15.94:0/2135732083 wait complete. 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 -- start start 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 Processor -- start 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 0x55d9246f6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d92464e460 con 0x55d9243274d0 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d9245d5b60 con 0x55d9243481b0 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9246f0600 con 0x55d92431f2f0 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe942429740 10 client.0 _ll_register_callbacks cb 0x55d9245fa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37872/0 (socket says 172.21.15.94:37872) 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe938a31640 1 -- 172.21.15.94:0/303721248 learned_addr learned my addr 172.21.15.94:0/303721248 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe939232640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 0x55d9246f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.867+0000 7fe939a33640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 -- 172.21.15.94:0/303721248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 msgr2=0x55d9246f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 0x55d9246f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 -- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 msgr2=0x55d9246fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246fdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 -- 172.21.15.94:0/303721248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d924360210 con 0x55d9243274d0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe939232640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 0x55d9246f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe938a31640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe924012700 tx=0x7fe924012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6916baeb0277ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe91cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924004050 con 0x55d9243274d0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9300c6fe0 con 0x55d9243274d0 2024-04-25T13:47:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe9240041f0 con 0x55d9243274d0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9240043f0 con 0x55d9243274d0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe942429740 10 client.17580 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d92442c620 con 0x55d9243274d0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.868+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9300c6fe0 con 0x55d9243274d0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe924003b20 con 0x55d9243274d0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 10 client.17580 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 10 client.17580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 20 client.17580 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d924306d80 0x55d924704480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 4 client.17580 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d92465c070 con 0x55d924306d80 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe939232640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d924306d80 0x55d924704480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe91cff9640 10 client.17580 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.869+0000 7fe939232640 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d924306d80 0x55d924704480 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd40c77a9ad10dff server_cookie=1d777e1f2a14f37c in_seq=0 out_seq=0 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe91cff9640 1 -- 172.21.15.94:0/303721248 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d92465c070 con 0x55d924306d80 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe91cff9640 10 client.17580 handle_command_reply: tid=0 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 client.17580 shutdown 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 msgr2=0x55d9246f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe924012700 tx=0x7fe924012bc0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 shutdown_connections 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d924306d80 0x55d924704480 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9243481b0 0x55d9246fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f6bb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 --2- 172.21.15.94:0/303721248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d92431f2f0 0x55d9246f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe91cff9640 0 client.17580 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 >> 172.21.15.94:0/303721248 conn(0x55d92433c300 msgr2=0x55d9246e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 shutdown_connections 2024-04-25T13:47:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 1 -- 172.21.15.94:0/303721248 wait complete. 2024-04-25T13:47:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.871+0000 7fe942429740 20 client.17580 trim_cache size 0 max 16384 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 Processor -- start 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 -- start start 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a92f0 con 0x55d9243274d0 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341aae30 con 0x55d92431f2f0 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe942429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a99c0 con 0x55d9243481b0 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37882/0 (socket says 172.21.15.94:37882) 2024-04-25T13:47:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe938a31640 1 -- 172.21.15.94:0/1973291926 learned_addr learned my addr 172.21.15.94:0/1973291926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe939232640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.887+0000 7fe939a33640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 -- 172.21.15.94:0/1973291926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 msgr2=0x55d9246fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 -- 172.21.15.94:0/1973291926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 msgr2=0x55d9246f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 -- 172.21.15.94:0/1973291926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9246fdf70 con 0x55d9243274d0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe939232640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe938a31640 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe924003ef0 tx=0x7fe9240107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63eb9d5c73a380fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe91effd640 1 -- 172.21.15.94:0/1973291926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924004000 con 0x55d9243274d0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe91effd640 1 -- 172.21.15.94:0/1973291926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240041a0 con 0x55d9243274d0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.888+0000 7fe91effd640 1 -- 172.21.15.94:0/1973291926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924005040 con 0x55d9243274d0 2024-04-25T13:47:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- 172.21.15.94:0/1973291926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 msgr2=0x55d9246f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe924003ef0 tx=0x7fe9240107b0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- 172.21.15.94:0/1973291926 shutdown_connections 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246f51d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 --2- 172.21.15.94:0/1973291926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- 172.21.15.94:0/1973291926 >> 172.21.15.94:0/1973291926 conn(0x55d92433c300 msgr2=0x55d9246f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- 172.21.15.94:0/1973291926 shutdown_connections 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- 172.21.15.94:0/1973291926 wait complete. 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 -- start start 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.889+0000 7fe942429740 1 Processor -- start 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246fc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341aa760 con 0x55d9243274d0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341aa090 con 0x55d92431f2f0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341a72d0 con 0x55d9243481b0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 10 client.0 _ll_register_callbacks cb 0x7fe9240097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe939232640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37892/0 (socket says 172.21.15.94:37892) 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 -- 172.21.15.94:0/2188021749 learned_addr learned my addr 172.21.15.94:0/2188021749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe939a33640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 -- 172.21.15.94:0/2188021749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 msgr2=0x55d9246fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 -- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 msgr2=0x55d9246fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 -- 172.21.15.94:0/2188021749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92476a9f0 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe939232640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe938a31640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe9240043c0 tx=0x7fe92400f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601e944feebd3851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe91cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92403b060 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9240041a0 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.890+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe92400fe40 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe924010950 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 10 client.17586 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d92476bc80 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240041a0 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe9240035f0 con 0x55d9243274d0 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 10 client.17586 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 20 client.17586 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d9242dc1c0 0x55d92476e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 4 client.17586 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d924655ee0 con 0x55d9242dc1c0 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.891+0000 7fe939232640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d9242dc1c0 0x55d92476e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.892+0000 7fe91cff9640 10 client.17586 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:19.892+0000 7fe939232640 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d9242dc1c0 0x55d92476e690 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41609735534d1dfa server_cookie=22b5058caa2c6f5 in_seq=0 out_seq=0 2024-04-25T13:47:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.033+0000 7fe91cff9640 1 -- 172.21.15.94:0/2188021749 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d924655ee0 con 0x55d9242dc1c0 2024-04-25T13:47:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.033+0000 7fe91cff9640 10 client.17586 handle_command_reply: tid=0 2024-04-25T13:47:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.033+0000 7fe942429740 1 client.17586 shutdown 2024-04-25T13:47:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 msgr2=0x55d9246fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe9240043c0 tx=0x7fe92400f800 comp rx=0 tx=0).stop 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 shutdown_connections 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d9242dc1c0 0x55d92476e690 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9243481b0 0x55d9246fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9243274d0 0x55d9246fca70 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 --2- 172.21.15.94:0/2188021749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92431f2f0 0x55d9246fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe91cff9640 0 client.17586 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 >> 172.21.15.94:0/2188021749 conn(0x55d92433c300 msgr2=0x55d9246f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 shutdown_connections 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 1 -- 172.21.15.94:0/2188021749 wait complete. 2024-04-25T13:47:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.034+0000 7fe942429740 20 client.17586 trim_cache size 0 max 16384 2024-04-25T13:47:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:47:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:47:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:47:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 msgr2=0x7fe9200b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 0x7fe9200b1ff0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009fd0 tx=0x7fe92c004a40 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 msgr2=0x7fe93419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe930006010 tx=0x7fe930004270 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 shutdown_connections 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe9200afae0 0x7fe9200b1ff0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108960 0x7fe9341a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe934103c60 0x7fe93419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 --2- 172.21.15.94:0/1532661057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe934102a20 0x7fe93419a230 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 >> 172.21.15.94:0/1532661057 conn(0x7fe9340fdbf0 msgr2=0x7fe934104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 shutdown_connections 2024-04-25T13:47:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:20.035+0000 7fe93b496640 1 -- 172.21.15.94:0/1532661057 wait complete. 2024-04-25T13:47:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:20 smithi137 ceph-mon[32568]: pgmap v513: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-04-25T13:47:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:20 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:47:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:47:20.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:20 smithi094 ceph-mon[29195]: pgmap v513: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-04-25T13:47:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:20 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:47:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:47:20.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:20 smithi138 ceph-mon[32977]: pgmap v513: 97 pgs: 97 active+clean; 44 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 342 op/s 2024-04-25T13:47:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:20 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:47:20.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:47:20.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:20.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 607625 13.23 MB/sec execute 123 sec latency 67.808 ms 2024-04-25T13:47:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:21 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:47:21.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:21 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:47:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:21 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:47:21.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 610265 13.23 MB/sec execute 124 sec latency 11.906 ms 2024-04-25T13:47:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:22 smithi137 ceph-mon[32568]: pgmap v514: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T13:47:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:22 smithi094 ceph-mon[29195]: pgmap v514: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T13:47:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:22 smithi138 ceph-mon[32977]: pgmap v514: 97 pgs: 97 active+clean; 49 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T13:47:22.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 612342 13.22 MB/sec execute 125 sec latency 13.075 ms 2024-04-25T13:47:23.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 614730 13.22 MB/sec execute 126 sec latency 135.019 ms 2024-04-25T13:47:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:24 smithi137 ceph-mon[32568]: pgmap v515: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-04-25T13:47:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:24 smithi094 ceph-mon[29195]: pgmap v515: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-04-25T13:47:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:24 smithi138 ceph-mon[32977]: pgmap v515: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-04-25T13:47:24.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 617196 13.21 MB/sec execute 127 sec latency 21.832 ms 2024-04-25T13:47:25.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 619162 13.18 MB/sec execute 128 sec latency 17.865 ms 2024-04-25T13:47:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:26 smithi137 ceph-mon[32568]: pgmap v516: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-04-25T13:47:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:26 smithi094 ceph-mon[29195]: pgmap v516: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-04-25T13:47:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:26 smithi138 ceph-mon[32977]: pgmap v516: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-04-25T13:47:26.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 621946 13.21 MB/sec execute 129 sec latency 17.180 ms 2024-04-25T13:47:27.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 624876 13.22 MB/sec execute 130 sec latency 14.936 ms 2024-04-25T13:47:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:28 smithi137 ceph-mon[32568]: pgmap v517: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:28 smithi094 ceph-mon[29195]: pgmap v517: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:28 smithi138 ceph-mon[32977]: pgmap v517: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:28.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 626486 13.19 MB/sec execute 131 sec latency 76.118 ms 2024-04-25T13:47:29.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 629032 13.20 MB/sec execute 132 sec latency 26.383 ms 2024-04-25T13:47:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:30 smithi137 ceph-mon[32568]: pgmap v518: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-04-25T13:47:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:30 smithi094 ceph-mon[29195]: pgmap v518: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-04-25T13:47:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:30 smithi138 ceph-mon[32977]: pgmap v518: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-04-25T13:47:30.798 INFO:tasks.workunit.client.0.smithi094.stdout: 1 631913 13.21 MB/sec execute 133 sec latency 13.995 ms 2024-04-25T13:47:31.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 633825 13.19 MB/sec execute 134 sec latency 16.178 ms 2024-04-25T13:47:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:32 smithi137 ceph-mon[32568]: pgmap v519: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:47:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:32 smithi094 ceph-mon[29195]: pgmap v519: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:47:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:32 smithi138 ceph-mon[32977]: pgmap v519: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-25T13:47:32.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 636311 13.19 MB/sec execute 135 sec latency 13.585 ms 2024-04-25T13:47:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 -- 172.21.15.94:0/489113530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 msgr2=0x7f6fa0104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 --2- 172.21.15.94:0/489113530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa0104060 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009900 tx=0x7f6f9002a410 comp rx=0 tx=0).stop 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 -- 172.21.15.94:0/489113530 shutdown_connections 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 --2- 172.21.15.94:0/489113530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 0x7f6fa010ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 --2- 172.21.15.94:0/489113530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa0104060 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 --2- 172.21.15.94:0/489113530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 -- 172.21.15.94:0/489113530 >> 172.21.15.94:0/489113530 conn(0x7f6fa00fdc00 msgr2=0x7f6fa01000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 -- 172.21.15.94:0/489113530 shutdown_connections 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.132+0000 7f6fa78af640 1 -- 172.21.15.94:0/489113530 wait complete. 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 Processor -- start 2024-04-25T13:47:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 -- start start 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 0x7f6fa01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010e530 con 0x7f6fa0103c50 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010e3b0 con 0x7f6fa0108950 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa78af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010e6b0 con 0x7f6fa0102a10 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa4e4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa564b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa4e4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52026/0 (socket says 172.21.15.94:52026) 2024-04-25T13:47:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa4e4a640 1 -- 172.21.15.94:0/340986228 learned_addr learned my addr 172.21.15.94:0/340986228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.133+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 0x7f6fa01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 -- 172.21.15.94:0/340986228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 msgr2=0x7f6fa01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 0x7f6fa01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 -- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 msgr2=0x7f6fa01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 -- 172.21.15.94:0/340986228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa01a0ca0 con 0x7f6fa0102a10 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa564b640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0199160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6fa01022e0 tx=0x7f6f9c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6f967fc640 1 -- 172.21.15.94:0/340986228 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00cc40 con 0x7f6fa0102a10 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01a0f90 con 0x7f6fa0102a10 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6f967fc640 1 -- 172.21.15.94:0/340986228 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c00cde0 con 0x7f6fa0102a10 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6f967fc640 1 -- 172.21.15.94:0/340986228 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c010430 con 0x7f6fa0102a10 2024-04-25T13:47:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.134+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01a13f0 con 0x7f6fa0102a10 2024-04-25T13:47:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 Processor -- start 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6f967fc640 1 -- 172.21.15.94:0/340986228 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6f9c003680 con 0x7f6fa0102a10 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 -- start start 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6fed2f0 0x5646a73b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6ff54d0 0x5646a73bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646a731c460 con 0x5646a70161b0 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646a72d6590 con 0x5646a6fed2f0 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fae765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646a7318420 con 0x5646a6ff54d0 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52040/0 (socket says 172.21.15.94:52040) 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/2205854003 learned_addr learned my addr 172.21.15.94:0/2205854003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa564b640 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6fed2f0 0x5646a73b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6ff54d0 0x5646a73bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/2205854003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6ff54d0 msgr2=0x5646a73bddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6ff54d0 0x5646a73bddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/2205854003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6fed2f0 msgr2=0x5646a73b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6fed2f0 0x5646a73b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.137+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/2205854003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a72e30d0 con 0x5646a70161b0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009fd0 tx=0x7f6f980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ed076b9952dadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6f967fc640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 0x7f6f8c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6f8b7fe640 1 -- 172.21.15.94:0/2205854003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004d20 con 0x5646a70161b0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6fa564b640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 0x7f6f8c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6f8b7fe640 1 -- 172.21.15.94:0/2205854003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98004050 con 0x5646a70161b0 2024-04-25T13:47:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6f8b7fe640 1 -- 172.21.15.94:0/2205854003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98005260 con 0x5646a70161b0 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.138+0000 7f6f967fc640 1 -- 172.21.15.94:0/340986228 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6f9c0defc0 con 0x7f6fa0102a10 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- 172.21.15.94:0/2205854003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 msgr2=0x5646a73b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fa564b640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 0x7f6f8c0b1ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5646a73b7d50 tx=0x7f6f9c090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009fd0 tx=0x7f6f980118d0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- 172.21.15.94:0/2205854003 shutdown_connections 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6ff54d0 0x5646a73bddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6fed2f0 0x5646a73b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 --2- 172.21.15.94:0/2205854003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73b6970 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- 172.21.15.94:0/2205854003 >> 172.21.15.94:0/2205854003 conn(0x5646a700a300 msgr2=0x5646a73b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- 172.21.15.94:0/2205854003 shutdown_connections 2024-04-25T13:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- 172.21.15.94:0/2205854003 wait complete. 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 -- start start 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.139+0000 7f6fae765740 1 Processor -- start 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646a73be840 con 0x5646a70161b0 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646a72a3b60 con 0x5646a6ff54d0 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646a73be9c0 con 0x5646a6fed2f0 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 10 client.0 _ll_register_callbacks cb 0x5646a72c82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52052/0 (socket says 172.21.15.94:52052) 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/157935152 learned_addr learned my addr 172.21.15.94:0/157935152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/157935152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 msgr2=0x5646a73c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 msgr2=0x5646a73c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/157935152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a702e210 con 0x5646a70161b0 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009de0 tx=0x7f6f98009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b94479cd9826853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9802c060 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6f9800f6e0 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.140+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f98004050 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f98004690 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 10 client.17598 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646a70fa620 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98004050 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6f980055b0 con 0x5646a70161b0 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 10 client.17598 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 20 client.17598 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6fd4d80 0x5646a73ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 4 client.17598 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a732a070 con 0x5646a6fd4d80 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.141+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6fd4d80 0x5646a73ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.142+0000 7f6f88ff9640 10 client.17598 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.142+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6fd4d80 0x5646a73ce610 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c938f0de4dba9910 server_cookie=41f5808c31c016b9 in_seq=0 out_seq=0 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.142+0000 7f6f88ff9640 1 -- 172.21.15.94:0/157935152 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646a732a070 con 0x5646a6fd4d80 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.142+0000 7f6f88ff9640 10 client.17598 handle_command_reply: tid=0 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.142+0000 7f6fae765740 1 client.17598 shutdown 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 msgr2=0x5646a73c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009de0 tx=0x7f6f98009c70 comp rx=0 tx=0).stop 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 shutdown_connections 2024-04-25T13:47:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6fd4d80 0x5646a73ce610 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a70161b0 0x5646a73c7aa0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6f88ff9640 0 client.17598 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 --2- 172.21.15.94:0/157935152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 >> 172.21.15.94:0/157935152 conn(0x5646a700a300 msgr2=0x5646a73b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 shutdown_connections 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 1 -- 172.21.15.94:0/157935152 wait complete. 2024-04-25T13:47:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.143+0000 7f6fae765740 20 client.17598 trim_cache size 0 max 16384 2024-04-25T13:47:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 Processor -- start 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 -- start start 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a70161b0 0x5646a73b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01aa4b0 con 0x5646a6fed2f0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01abff0 con 0x5646a6ff54d0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fae765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01aab80 con 0x5646a70161b0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52060/0 (socket says 172.21.15.94:52060) 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/3414525552 learned_addr learned my addr 172.21.15.94:0/3414525552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a70161b0 0x5646a73b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/3414525552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a70161b0 msgr2=0x5646a73b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a70161b0 0x5646a73b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/3414525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 msgr2=0x5646a73b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa5e4c640 1 -- 172.21.15.94:0/3414525552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a73bb520 con 0x5646a6fed2f0 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.158+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005d70 tx=0x7f6f980096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8efbc7c02ef078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6f8affd640 1 -- 172.21.15.94:0/3414525552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98004690 con 0x5646a6fed2f0 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6f8affd640 1 -- 172.21.15.94:0/3414525552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98031430 con 0x5646a6fed2f0 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6f8affd640 1 -- 172.21.15.94:0/3414525552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98005000 con 0x5646a6fed2f0 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6fae765740 1 -- 172.21.15.94:0/3414525552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 msgr2=0x5646a73b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.159+0000 7f6fae765740 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005d70 tx=0x7f6f980096c0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- 172.21.15.94:0/3414525552 shutdown_connections 2024-04-25T13:47:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a70161b0 0x5646a73b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a6ff54d0 0x5646a73b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- 172.21.15.94:0/3414525552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6fed2f0 0x5646a73b22f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- 172.21.15.94:0/3414525552 >> 172.21.15.94:0/3414525552 conn(0x5646a700a300 msgr2=0x5646a73b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- 172.21.15.94:0/3414525552 shutdown_connections 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- 172.21.15.94:0/3414525552 wait complete. 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- start start 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 Processor -- start 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a70161b0 0x5646a7436c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01ab920 con 0x5646a6ff54d0 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01ab250 con 0x5646a70161b0 2024-04-25T13:47:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01a8490 con 0x5646a6fed2f0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fae765740 10 client.0 _ll_register_callbacks cb 0x7f6f9c0e2970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fa4e4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.160+0000 7f6fa5e4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52066/0 (socket says 172.21.15.94:52066) 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 -- 172.21.15.94:0/3281470668 learned_addr learned my addr 172.21.15.94:0/3281470668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a70161b0 0x5646a7436c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 -- 172.21.15.94:0/3281470668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 msgr2=0x5646a73c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 -- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a70161b0 msgr2=0x5646a7436c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a70161b0 0x5646a7436c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 -- 172.21.15.94:0/3281470668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a7437240 con 0x5646a6ff54d0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fa4e4a640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006010 tx=0x7f6f900043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca04da06e1356c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90035040 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6f900475b0 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f98031430 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f90056050 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fae765740 10 client.17604 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.161+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646a74385a0 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98031430 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6f900035f0 con 0x5646a6ff54d0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 10 client.17604 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 20 client.17604 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6faa1c0 0x5646a743afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 4 client.17604 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646a7323ee0 con 0x5646a6faa1c0 2024-04-25T13:47:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6faa1c0 0x5646a743afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6f88ff9640 10 client.17604 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.162+0000 7f6fa5e4c640 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6faa1c0 0x5646a743afc0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbc755b148a3aa7 server_cookie=72d226d4ba9ff698 in_seq=0 out_seq=0 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6f88ff9640 1 -- 172.21.15.94:0/3281470668 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646a7323ee0 con 0x5646a6faa1c0 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6f88ff9640 10 client.17604 handle_command_reply: tid=0 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 client.17604 shutdown 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 msgr2=0x5646a73c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006010 tx=0x7f6f900043a0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 shutdown_connections 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646a6faa1c0 0x5646a743afc0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646a70161b0 0x5646a7436c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646a6ff54d0 0x5646a73c7500 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6f88ff9640 0 client.17604 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 --2- 172.21.15.94:0/3281470668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646a6fed2f0 0x5646a73c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 >> 172.21.15.94:0/3281470668 conn(0x5646a700a300 msgr2=0x5646a73b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 shutdown_connections 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 1 -- 172.21.15.94:0/3281470668 wait complete. 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.321+0000 7f6fae765740 20 client.17604 trim_cache size 0 max 16384 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:47:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:47:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 msgr2=0x7f6f8c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 0x7f6f8c0b1ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5646a73b7d50 tx=0x7f6f9c090040 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 msgr2=0x7f6fa0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0199160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6fa01022e0 tx=0x7f6f9c00ee50 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 shutdown_connections 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f8c0afae0 0x7f6f8c0b1ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fa0108950 0x7f6fa01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa0103c50 0x7f6fa01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 --2- 172.21.15.94:0/340986228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa0102a10 0x7f6fa0199160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 >> 172.21.15.94:0/340986228 conn(0x7f6fa00fdc00 msgr2=0x7f6fa0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 shutdown_connections 2024-04-25T13:47:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:33.322+0000 7f6fa78af640 1 -- 172.21.15.94:0/340986228 wait complete. 2024-04-25T13:47:33.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 638699 13.19 MB/sec execute 136 sec latency 105.522 ms 2024-04-25T13:47:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:34 smithi137 ceph-mon[32568]: pgmap v520: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-04-25T13:47:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:34 smithi094 ceph-mon[29195]: pgmap v520: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-04-25T13:47:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:34 smithi138 ceph-mon[32977]: pgmap v520: 97 pgs: 97 active+clean; 52 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 341 op/s 2024-04-25T13:47:34.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 640083 13.14 MB/sec execute 137 sec latency 72.799 ms 2024-04-25T13:47:35.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 643048 13.18 MB/sec execute 138 sec latency 8.160 ms 2024-04-25T13:47:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:36 smithi137 ceph-mon[32568]: pgmap v521: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-04-25T13:47:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:36 smithi094 ceph-mon[29195]: pgmap v521: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-04-25T13:47:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:36 smithi138 ceph-mon[32977]: pgmap v521: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-04-25T13:47:36.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 645403 13.14 MB/sec execute 139 sec latency 20.899 ms 2024-04-25T13:47:37.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 647491 13.14 MB/sec execute 140 sec latency 17.053 ms 2024-04-25T13:47:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:38 smithi137 ceph-mon[32568]: pgmap v522: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-04-25T13:47:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:38 smithi094 ceph-mon[29195]: pgmap v522: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-04-25T13:47:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:38 smithi138 ceph-mon[32977]: pgmap v522: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-04-25T13:47:38.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 650290 13.17 MB/sec execute 141 sec latency 13.703 ms 2024-04-25T13:47:39.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 652487 13.14 MB/sec execute 142 sec latency 45.712 ms 2024-04-25T13:47:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:40 smithi137 ceph-mon[32568]: pgmap v523: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:47:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:40 smithi094 ceph-mon[29195]: pgmap v523: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:47:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:40 smithi138 ceph-mon[32977]: pgmap v523: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:47:40.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 654419 13.13 MB/sec execute 143 sec latency 13.584 ms 2024-04-25T13:47:41.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 657281 13.16 MB/sec execute 144 sec latency 12.413 ms 2024-04-25T13:47:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:42 smithi137 ceph-mon[32568]: pgmap v524: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-04-25T13:47:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:42 smithi094 ceph-mon[29195]: pgmap v524: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-04-25T13:47:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:42 smithi138 ceph-mon[32977]: pgmap v524: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-04-25T13:47:42.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 659898 13.16 MB/sec execute 145 sec latency 12.273 ms 2024-04-25T13:47:43.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 661532 13.12 MB/sec execute 146 sec latency 54.974 ms 2024-04-25T13:47:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:44 smithi137 ceph-mon[32568]: pgmap v525: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:44 smithi094 ceph-mon[29195]: pgmap v525: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:44 smithi138 ceph-mon[32977]: pgmap v525: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-04-25T13:47:44.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 664184 13.15 MB/sec execute 147 sec latency 70.299 ms 2024-04-25T13:47:45.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 666040 13.10 MB/sec execute 148 sec latency 34.901 ms 2024-04-25T13:47:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 -- 172.21.15.94:0/110643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c301098d0 msgr2=0x7f2c3010bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 --2- 172.21.15.94:0/110643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c301098d0 0x7f2c3010bcf0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009ca0 tx=0x7f2c2802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 -- 172.21.15.94:0/110643289 shutdown_connections 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 --2- 172.21.15.94:0/110643289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c301098d0 0x7f2c3010bcf0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 --2- 172.21.15.94:0/110643289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c30106f70 0x7f2c30109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 --2- 172.21.15.94:0/110643289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 -- 172.21.15.94:0/110643289 >> 172.21.15.94:0/110643289 conn(0x7f2c300fdc10 msgr2=0x7f2c301000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 -- 172.21.15.94:0/110643289 shutdown_connections 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 -- 172.21.15.94:0/110643289 wait complete. 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.425+0000 7f2c35dba640 1 Processor -- start 2024-04-25T13:47:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 -- start start 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c301098d0 0x7f2c30102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010f630 con 0x7f2c30106f70 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010f4b0 con 0x7f2c301098d0 2024-04-25T13:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c35dba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3010f7b0 con 0x7f2c30104610 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33920/0 (socket says 172.21.15.94:33920) 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48982/0 (socket says 172.21.15.94:48982) 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2744393595 learned_addr learned my addr 172.21.15.94:0/2744393595 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c301098d0 0x7f2c30102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 -- 172.21.15.94:0/2744393595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 msgr2=0x7f2c30103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 -- 172.21.15.94:0/2744393595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c301098d0 msgr2=0x7f2c30102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c301098d0 0x7f2c30102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2effd640 1 -- 172.21.15.94:0/2744393595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c30102d70 con 0x7f2c30106f70 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.426+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e9f0 tx=0x7f2c2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c2cff9640 1 -- 172.21.15.94:0/2744393595 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000ccc0 con 0x7f2c30106f70 2024-04-25T13:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c2cff9640 1 -- 172.21.15.94:0/2744393595 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c200108e0 con 0x7f2c30106f70 2024-04-25T13:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c301a4270 con 0x7f2c30106f70 2024-04-25T13:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c2cff9640 1 -- 172.21.15.94:0/2744393595 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20018880 con 0x7f2c30106f70 2024-04-25T13:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.427+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c301a4730 con 0x7f2c30106f70 2024-04-25T13:47:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.429+0000 7f2c2cff9640 1 -- 172.21.15.94:0/2744393595 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2c20010430 con 0x7f2c30106f70 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 Processor -- start 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 -- start start 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 0x55d661a15970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a1cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d66197b460 con 0x55d6616751b0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d661935590 con 0x55d6616544d0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d661977420 con 0x55d66164c2f0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 0x55d661a15970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a1cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33930/0 (socket says 172.21.15.94:33930) 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 0x55d661a15970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48988/0 (socket says 172.21.15.94:48988) 2024-04-25T13:47:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3046130449 learned_addr learned my addr 172.21.15.94:0/3046130449 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2cff9640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 0x7f2c180b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 0x7f2c180b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3046130449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 msgr2=0x55d661a1cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a1cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3046130449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 msgr2=0x55d661a15970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 0x55d661a15970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3046130449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6619420d0 con 0x55d66164c2f0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.430+0000 7f2c2cff9640 1 -- 172.21.15.94:0/2744393595 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2c200df010 con 0x7f2c30106f70 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.431+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c240115f0 tx=0x7f2c24011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9865f77e3e8579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.431+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 0x7f2c180b1ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2c28004af0 tx=0x7f2c2802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.431+0000 7f2c15ffb640 1 -- 172.21.15.94:0/3046130449 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2401e070 con 0x55d66164c2f0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.431+0000 7f2c15ffb640 1 -- 172.21.15.94:0/3046130449 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2400f660 con 0x55d66164c2f0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.431+0000 7f2c15ffb640 1 -- 172.21.15.94:0/3046130449 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24003850 con 0x55d66164c2f0 2024-04-25T13:47:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3046130449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 msgr2=0x55d661a15ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c240115f0 tx=0x7f2c24011ab0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3046130449 shutdown_connections 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a1cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a15ef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3046130449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d6616751b0 0x55d661a15970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3046130449 >> 172.21.15.94:0/3046130449 conn(0x55d661669300 msgr2=0x55d661a11300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3046130449 shutdown_connections 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3046130449 wait complete. 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 -- start start 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.432+0000 7f2c3cc6a740 1 Processor -- start 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6616544d0 0x55d661a1fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616751b0 0x55d661a26ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d661a1d840 con 0x55d66164c2f0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d661902b60 con 0x55d6616751b0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d661a1d9c0 con 0x55d6616544d0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 10 client.0 _ll_register_callbacks cb 0x55d6619272b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48996/0 (socket says 172.21.15.94:48996) 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2786322165 learned_addr learned my addr 172.21.15.94:0/2786322165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6616544d0 0x55d661a1fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616751b0 0x55d661a26ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2786322165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6616544d0 msgr2=0x55d661a1fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6616544d0 0x55d661a1fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616751b0 msgr2=0x55d661a26ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616751b0 0x55d661a26ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2786322165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66168d210 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2c24004650 tx=0x7f2c240052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54826873294eec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c0effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24026020 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2c240054e0 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.433+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2400f660 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c24031050 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 10 client.17616 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d661759620 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2400f660 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2c24021030 con 0x55d66164c2f0 2024-04-25T13:47:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 10 client.17616 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 20 client.17616 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d661633d80 0x55d661a2d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 4 client.17616 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d661989070 con 0x55d661633d80 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.434+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d661633d80 0x55d661a2d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.435+0000 7f2c0effd640 10 client.17616 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.435+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d661633d80 0x55d661a2d0e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade4b4514c64939f server_cookie=f2e7839aaf47e574 in_seq=0 out_seq=0 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.435+0000 7f2c0effd640 1 -- 172.21.15.94:0/2786322165 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d661989070 con 0x55d661633d80 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.435+0000 7f2c0effd640 10 client.17616 handle_command_reply: tid=0 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.435+0000 7f2c3cc6a740 1 client.17616 shutdown 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 msgr2=0x55d661a1f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2c24004650 tx=0x7f2c240052f0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 shutdown_connections 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d661633d80 0x55d661a2d0e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616751b0 0x55d661a26ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c0effd640 0 client.17616 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6616544d0 0x55d661a1fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2786322165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d66164c2f0 0x55d661a1f610 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 >> 172.21.15.94:0/2786322165 conn(0x55d661669300 msgr2=0x55d661a14210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 shutdown_connections 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2786322165 wait complete. 2024-04-25T13:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.436+0000 7f2c3cc6a740 20 client.17616 trim_cache size 0 max 16384 2024-04-25T13:47:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.451+0000 7f2c3cc6a740 1 Processor -- start 2024-04-25T13:47:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.451+0000 7f2c3cc6a740 1 -- start start 2024-04-25T13:47:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ad7b0 con 0x55d661633d80 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301af2f0 con 0x55d6616544d0 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ade80 con 0x55d66164c2f0 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49012/0 (socket says 172.21.15.94:49012) 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2007360943 learned_addr learned my addr 172.21.15.94:0/2007360943 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2effd640 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2007360943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 msgr2=0x55d661a10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a10c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2007360943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 msgr2=0x55d661a11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/2007360943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d661a1a4b0 con 0x55d661633d80 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009c90 tx=0x7f2c24011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a31ce2c6f1e91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c157fa640 1 -- 172.21.15.94:0/2007360943 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24026040 con 0x55d661633d80 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c157fa640 1 -- 172.21.15.94:0/2007360943 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2400f660 con 0x55d661633d80 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.452+0000 7f2c157fa640 1 -- 172.21.15.94:0/2007360943 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2401e040 con 0x55d661633d80 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2007360943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 msgr2=0x55d661a14810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009c90 tx=0x7f2c24011ab0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2007360943 shutdown_connections 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/2007360943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a14810 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2007360943 >> 172.21.15.94:0/2007360943 conn(0x55d661669300 msgr2=0x55d661a14100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2007360943 shutdown_connections 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/2007360943 wait complete. 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.453+0000 7f2c3cc6a740 1 -- start start 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 Processor -- start 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301aec20 con 0x55d661633d80 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ae550 con 0x55d6616544d0 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ab790 con 0x55d66164c2f0 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c3cc6a740 10 client.0 _ll_register_callbacks cb 0x7f2c2401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49022/0 (socket says 172.21.15.94:49022) 2024-04-25T13:47:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3357824326 learned_addr learned my addr 172.21.15.94:0/3357824326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2ffff640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2effd640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3357824326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 msgr2=0x55d661a234f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a234f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 msgr2=0x55d661a93480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a93480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.454+0000 7f2c2f7fe640 1 -- 172.21.15.94:0/3357824326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d661a23b60 con 0x55d661633d80 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c2f7fe640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2c240194d0 tx=0x7f2c24004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12affcf17579158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24039070 con 0x55d661633d80 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2c240198c0 con 0x55d661633d80 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2400f660 con 0x55d661633d80 2024-04-25T13:47:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c24044050 con 0x55d661633d80 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 10 client.17622 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d661a94a80 con 0x55d661633d80 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2400f660 con 0x55d661633d80 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2c24021070 con 0x55d661633d80 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 10 client.17622 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 20 client.17622 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d6616751b0 0x55d661a97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 4 client.17622 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:47:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.455+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d661982ee0 con 0x55d6616751b0 2024-04-25T13:47:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.456+0000 7f2c2effd640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d6616751b0 0x55d661a97360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.456+0000 7f2c2effd640 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d6616751b0 0x55d661a97360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6716690244e425 server_cookie=198445a60fe134d in_seq=0 out_seq=0 2024-04-25T13:47:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.456+0000 7f2c0effd640 10 client.17622 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.611+0000 7f2c0effd640 1 -- 172.21.15.94:0/3357824326 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d661982ee0 con 0x55d6616751b0 2024-04-25T13:47:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.611+0000 7f2c0effd640 10 client.17622 handle_command_reply: tid=0 2024-04-25T13:47:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.611+0000 7f2c3cc6a740 1 client.17622 shutdown 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 msgr2=0x55d661a22fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2c240194d0 tx=0x7f2c24004f00 comp rx=0 tx=0).stop 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 shutdown_connections 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d6616751b0 0x55d661a97360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6616544d0 0x55d661a93480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c0effd640 0 client.17622 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d66164c2f0 0x55d661a234f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 --2- 172.21.15.94:0/3357824326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d661633d80 0x55d661a22fb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 >> 172.21.15.94:0/3357824326 conn(0x55d661669300 msgr2=0x55d661a1dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 shutdown_connections 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 1 -- 172.21.15.94:0/3357824326 wait complete. 2024-04-25T13:47:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.612+0000 7f2c3cc6a740 20 client.17622 trim_cache size 0 max 16384 2024-04-25T13:47:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:47:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:47:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:47:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 msgr2=0x7f2c180b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 0x7f2c180b1ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2c28004af0 tx=0x7f2c2802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 msgr2=0x7f2c30102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e9f0 tx=0x7f2c2000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 shutdown_connections 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c180afae0 0x7f2c180b1ff0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c301098d0 0x7f2c30102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c30106f70 0x7f2c30102100 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 --2- 172.21.15.94:0/2744393595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c30104610 0x7f2c30103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 >> 172.21.15.94:0/2744393595 conn(0x7f2c300fdc10 msgr2=0x7f2c30105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 shutdown_connections 2024-04-25T13:47:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:46.613+0000 7f2c35dba640 1 -- 172.21.15.94:0/2744393595 wait complete. 2024-04-25T13:47:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:46 smithi137 ceph-mon[32568]: pgmap v526: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-04-25T13:47:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:46 smithi094 ceph-mon[29195]: pgmap v526: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-04-25T13:47:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:46 smithi138 ceph-mon[32977]: pgmap v526: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-04-25T13:47:46.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 668100 13.11 MB/sec execute 149 sec latency 81.685 ms 2024-04-25T13:47:47.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 671165 13.14 MB/sec execute 150 sec latency 14.560 ms 2024-04-25T13:47:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:48 smithi137 ceph-mon[32568]: pgmap v527: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 262 op/s 2024-04-25T13:47:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:48 smithi094 ceph-mon[29195]: pgmap v527: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 262 op/s 2024-04-25T13:47:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:48 smithi138 ceph-mon[32977]: pgmap v527: 97 pgs: 97 active+clean; 50 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 262 op/s 2024-04-25T13:47:48.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 673402 13.10 MB/sec execute 151 sec latency 14.078 ms 2024-04-25T13:47:49.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:47:49.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/437938158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 msgr2=0x7f4d5810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/437938158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d5810abb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009ca0 tx=0x7f4d5002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/437938158 shutdown_connections 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/437938158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d5810abb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/437938158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58103930 0x7f4d58103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/437938158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58102f50 0x7f4d58103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/437938158 >> 172.21.15.94:0/437938158 conn(0x7f4d580fd350 msgr2=0x7f4d580ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/437938158 shutdown_connections 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.264+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/437938158 wait complete. 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 Processor -- start 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 -- start start 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 0x7f4d58102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58103930 0x7f4d5819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5810e540 con 0x7f4d58104470 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5810e3c0 con 0x7f4d58102f50 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5ceaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5810e6c0 con 0x7f4d58103930 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 0x7f4d58102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49032/0 (socket says 172.21.15.94:49032) 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 0x7f4d58102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34130/0 (socket says 172.21.15.94:34130) 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 -- 172.21.15.94:0/2478668963 learned_addr learned my addr 172.21.15.94:0/2478668963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58103930 0x7f4d5819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 -- 172.21.15.94:0/2478668963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58103930 msgr2=0x7f4d5819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58103930 0x7f4d5819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 -- 172.21.15.94:0/2478668963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 msgr2=0x7f4d58102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 0x7f4d58102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 -- 172.21.15.94:0/2478668963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d581a1ce0 con 0x7f4d58104470 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.265+0000 7f4d56d9d640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4d50004af0 tx=0x7f4d50031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.266+0000 7f4d477fe640 1 -- 172.21.15.94:0/2478668963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50003b70 con 0x7f4d58104470 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.266+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d581a1f70 con 0x7f4d58104470 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.266+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d581a2430 con 0x7f4d58104470 2024-04-25T13:47:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.266+0000 7f4d477fe640 1 -- 172.21.15.94:0/2478668963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d50003d10 con 0x7f4d58104470 2024-04-25T13:47:49.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.266+0000 7f4d477fe640 1 -- 172.21.15.94:0/2478668963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5002fb20 con 0x7f4d58104470 2024-04-25T13:47:49.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.268+0000 7f4d477fe640 1 -- 172.21.15.94:0/2478668963 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4d5002fcc0 con 0x7f4d58104470 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.268+0000 7f4d63d66740 1 Processor -- start 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.268+0000 7f4d63d66740 1 -- start start 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94296b0 0x55b1d97ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b1d9750d80 con 0x55b1d94214d0 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1d970a8b0 con 0x55b1d944a390 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d63d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b1d974cd40 con 0x55b1d94296b0 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d477fe640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 0x7f4d400b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49046/0 (socket says 172.21.15.94:49046) 2024-04-25T13:47:49.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d5659c640 1 -- 172.21.15.94:0/3264363167 learned_addr learned my addr 172.21.15.94:0/3264363167 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94296b0 0x55b1d97ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d56d9d640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 0x7f4d400b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d477fe640 1 -- 172.21.15.94:0/2478668963 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d500fff90 con 0x7f4d58104470 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3264363167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94296b0 msgr2=0x55b1d97ecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94296b0 0x55b1d97ecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3264363167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 msgr2=0x55b1d97ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3264363167 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d96dba40 con 0x55b1d944a390 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d5659c640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d56d9d640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 0x7f4d400b2080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4d500fdd60 tx=0x7f4d50092b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.269+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97eba60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d58101bb0 tx=0x7f4d4c009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ec1bcd527c4632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.270+0000 7f4d44ff9640 1 -- 172.21.15.94:0/3264363167 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c004020 con 0x55b1d944a390 2024-04-25T13:47:49.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.270+0000 7f4d44ff9640 1 -- 172.21.15.94:0/3264363167 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c0041c0 con 0x55b1d944a390 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.270+0000 7f4d44ff9640 1 -- 172.21.15.94:0/3264363167 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01f6d0 con 0x55b1d944a390 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.270+0000 7f4d63d66740 1 -- 172.21.15.94:0/3264363167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 msgr2=0x55b1d97eba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.270+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97eba60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d58101bb0 tx=0x7f4d4c009660 comp rx=0 tx=0).stop 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- 172.21.15.94:0/3264363167 shutdown_connections 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94296b0 0x55b1d97ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3264363167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97eba60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- 172.21.15.94:0/3264363167 >> 172.21.15.94:0/3264363167 conn(0x55b1d943e4e0 msgr2=0x55b1d97e7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- 172.21.15.94:0/3264363167 shutdown_connections 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- 172.21.15.94:0/3264363167 wait complete. 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- start start 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 Processor -- start 2024-04-25T13:47:49.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 0x55b1d97f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b1d97ed780 con 0x55b1d94296b0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1d96d7d10 con 0x55b1d944a390 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b1d97ed900 con 0x55b1d94214d0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d63d66740 10 client.0 _ll_register_callbacks cb 0x55b1d9704210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.271+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49050/0 (socket says 172.21.15.94:49050) 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 -- 172.21.15.94:0/2161615898 learned_addr learned my addr 172.21.15.94:0/2161615898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 0x55b1d97f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 -- 172.21.15.94:0/2161615898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 msgr2=0x55b1d97f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 0x55b1d97f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 -- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 msgr2=0x55b1d97fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 -- 172.21.15.94:0/2161615898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d952c7d0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 0x55b1d97f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d5659c640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4d48012720 tx=0x7f4d48012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113cecb928bf6bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48004050 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d480041f0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4c0041c0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d480043f0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d63d66740 10 client.17634 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1d94623f0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.272+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c0041c0 con 0x55b1d94296b0 2024-04-25T13:47:49.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d48003c70 con 0x55b1d94296b0 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 10 client.17634 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 20 client.17634 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d9408f60 0x55b1d9803350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 4 client.17634 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1d975e990 con 0x55b1d9408f60 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.273+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d9408f60 0x55b1d9803350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d9408f60 0x55b1d9803350 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9327a75bd4796b server_cookie=35e4bbd2203b7b48 in_seq=0 out_seq=0 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d3dffb640 10 client.17634 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d3dffb640 1 -- 172.21.15.94:0/2161615898 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1d975e990 con 0x55b1d9408f60 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d3dffb640 10 client.17634 handle_command_reply: tid=0 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 client.17634 shutdown 2024-04-25T13:47:49.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 msgr2=0x55b1d97f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4d48012720 tx=0x7f4d48012be0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 shutdown_connections 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d9408f60 0x55b1d9803350 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d944a390 0x55b1d97fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f5ef0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.274+0000 7f4d63d66740 1 --2- 172.21.15.94:0/2161615898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d94214d0 0x55b1d97f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.275+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 >> 172.21.15.94:0/2161615898 conn(0x55b1d943e4e0 msgr2=0x55b1d97ea180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.275+0000 7f4d3dffb640 0 client.17634 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.275+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 shutdown_connections 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.275+0000 7f4d63d66740 1 -- 172.21.15.94:0/2161615898 wait complete. 2024-04-25T13:47:49.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.275+0000 7f4d63d66740 20 client.17634 trim_cache size 0 max 16384 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 Processor -- start 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 -- start start 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94296b0 0x55b1d97eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d97ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800e730 con 0x55b1d94214d0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38010270 con 0x55b1d94296b0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d63d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800ee00 con 0x55b1d944a390 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49052/0 (socket says 172.21.15.94:49052) 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3728281254 learned_addr learned my addr 172.21.15.94:0/3728281254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d5659c640 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94296b0 0x55b1d97eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3728281254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 msgr2=0x55b1d97ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d97ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3728281254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94296b0 msgr2=0x55b1d97eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94296b0 0x55b1d97eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 -- 172.21.15.94:0/3728281254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d94220f0 con 0x55b1d94214d0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.293+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009630 tx=0x7f4d4c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adc0b6c5d743011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d3ffff640 1 -- 172.21.15.94:0/3728281254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c033070 con 0x55b1d94214d0 2024-04-25T13:47:49.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d3ffff640 1 -- 172.21.15.94:0/3728281254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c003c00 con 0x55b1d94214d0 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d3ffff640 1 -- 172.21.15.94:0/3728281254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01f560 con 0x55b1d94214d0 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 -- 172.21.15.94:0/3728281254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 msgr2=0x55b1d97f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009630 tx=0x7f4d4c017040 comp rx=0 tx=0).stop 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 -- 172.21.15.94:0/3728281254 shutdown_connections 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d97ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94296b0 0x55b1d97eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 --2- 172.21.15.94:0/3728281254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94214d0 0x55b1d97f0d10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 -- 172.21.15.94:0/3728281254 >> 172.21.15.94:0/3728281254 conn(0x55b1d943e4e0 msgr2=0x55b1d97ea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.294+0000 7f4d63d66740 1 -- 172.21.15.94:0/3728281254 shutdown_connections 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 -- 172.21.15.94:0/3728281254 wait complete. 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 -- start start 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 Processor -- start 2024-04-25T13:47:49.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94214d0 0x55b1d97f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d98696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800fba0 con 0x55b1d94296b0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800f4d0 con 0x55b1d94214d0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3800c710 con 0x55b1d944a390 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d63d66740 10 client.0 _ll_register_callbacks cb 0x7f4d48009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d5659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49060/0 (socket says 172.21.15.94:49060) 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d5659c640 1 -- 172.21.15.94:0/4070676692 learned_addr learned my addr 172.21.15.94:0/4070676692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94214d0 0x55b1d97f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.295+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d98696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 -- 172.21.15.94:0/4070676692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 msgr2=0x55b1d98696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d98696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 -- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94214d0 msgr2=0x55b1d97f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94214d0 0x55b1d97f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 -- 172.21.15.94:0/4070676692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d97114b0 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d5659c640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4d48012720 tx=0x7f4d4800f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8332321b9c28e091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48005040 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f4d480051e0 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4c003c00 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d480053e0 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 10 client.17640 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1d986ad50 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d4c003c00 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d48004590 con 0x55b1d94296b0 2024-04-25T13:47:49.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 10 client.17640 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 20 client.17640 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d93de3a0 0x55b1d986d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 4 client.17640 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.296+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1d9758800 con 0x55b1d93de3a0 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d93de3a0 0x55b1d986d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d3dffb640 10 client.17640 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d55d9b640 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d93de3a0 0x55b1d986d7c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3665dfdb5b00c30c server_cookie=29a2dc835febb685 in_seq=0 out_seq=0 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d3dffb640 1 -- 172.21.15.94:0/4070676692 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b1d9758800 con 0x55b1d93de3a0 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d3dffb640 10 client.17640 handle_command_reply: tid=0 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 client.17640 shutdown 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 msgr2=0x55b1d97f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4d48012720 tx=0x7f4d4800f9a0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 shutdown_connections 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b1d93de3a0 0x55b1d986d7c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1d944a390 0x55b1d98696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1d94296b0 0x55b1d97f9af0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d3dffb640 0 client.17640 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 --2- 172.21.15.94:0/4070676692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1d94214d0 0x55b1d97f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 >> 172.21.15.94:0/4070676692 conn(0x55b1d943e4e0 msgr2=0x55b1d97ea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 shutdown_connections 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 1 -- 172.21.15.94:0/4070676692 wait complete. 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.297+0000 7f4d63d66740 20 client.17640 trim_cache size 0 max 16384 2024-04-25T13:47:49.303 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:49.304 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:47:49.304 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:47:49.304 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 msgr2=0x7f4d400b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 0x7f4d400b2080 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4d500fdd60 tx=0x7f4d50092b60 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 msgr2=0x7f4d581a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4d50004af0 tx=0x7f4d50031700 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 shutdown_connections 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d400afb70 0x7f4d400b2080 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d58104470 0x7f4d581a14c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d58103930 0x7f4d5819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 --2- 172.21.15.94:0/2478668963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d58102f50 0x7f4d58102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 >> 172.21.15.94:0/2478668963 conn(0x7f4d580fd350 msgr2=0x7f4d58106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 shutdown_connections 2024-04-25T13:47:49.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.298+0000 7f4d5ceaf640 1 -- 172.21.15.94:0/2478668963 wait complete. 2024-04-25T13:47:49.309 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:47:49.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/248841195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4103c00 msgr2=0x7f2dd4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/248841195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4103c00 0x7f2dd4104010 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009900 tx=0x7f2dc402a410 comp rx=0 tx=0).stop 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/248841195 shutdown_connections 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/248841195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd4108900 0x7f2dd410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/248841195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4103c00 0x7f2dd4104010 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/248841195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd41029c0 0x7f2dd41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/248841195 >> 172.21.15.94:0/248841195 conn(0x7f2dd40fdbd0 msgr2=0x7f2dd4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/248841195 shutdown_connections 2024-04-25T13:47:49.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.421+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/248841195 wait complete. 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 Processor -- start 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 -- start start 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd41029c0 0x7f2dd419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd4103c00 0x7f2dd419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e4f0 con 0x7f2dd4108900 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e370 con 0x7f2dd4103c00 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2ddb9f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e670 con 0x7f2dd41029c0 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2dd9f95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2dd9794640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd41029c0 0x7f2dd419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2dd9f95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49082/0 (socket says 172.21.15.94:49082) 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.422+0000 7f2dd9f95640 1 -- 172.21.15.94:0/1695954086 learned_addr learned my addr 172.21.15.94:0/1695954086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd8f93640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd4103c00 0x7f2dd419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 -- 172.21.15.94:0/1695954086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd41029c0 msgr2=0x7f2dd419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd41029c0 0x7f2dd419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 -- 172.21.15.94:0/1695954086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd4103c00 msgr2=0x7f2dd419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd4103c00 0x7f2dd419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 -- 172.21.15.94:0/1695954086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a1d30 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dd9f95640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00eaa0 tx=0x7f2dcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00cdb0 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dcc005020 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a1fc0 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0116d0 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.423+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a2430 con 0x7f2dd4108900 2024-04-25T13:47:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.424+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd41a26f0 con 0x7f2dd4108900 2024-04-25T13:47:49.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.426+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2dcc0051d0 con 0x7f2dd4108900 2024-04-25T13:47:49.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.427+0000 7f2dca7fc640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 0x7f2dc00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.427+0000 7f2dd9794640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 0x7f2dc00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.427+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2dcc0df160 con 0x7f2dd4108900 2024-04-25T13:47:49.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.427+0000 7f2dd9794640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 0x7f2dc00b2000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2dd41035c0 tx=0x7f2dd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.431+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dcc016030 con 0x7f2dd4108900 2024-04-25T13:47:49.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.660+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dd41a28c0 con 0x7f2dd4108900 2024-04-25T13:47:49.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.660+0000 7f2dca7fc640 1 -- 172.21.15.94:0/1695954086 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f2dcc0a7650 con 0x7f2dd4108900 2024-04-25T13:47:49.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:47:49.659 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:47:49.660 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:47:49.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 msgr2=0x7f2dc00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 0x7f2dc00b2000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2dd41035c0 tx=0x7f2dd0005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 msgr2=0x7f2dd41a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00eaa0 tx=0x7f2dcc00ef60 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 shutdown_connections 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2dc00afaf0 0x7f2dc00b2000 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd4108900 0x7f2dd41a16c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dd4103c00 0x7f2dd419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 --2- 172.21.15.94:0/1695954086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dd41029c0 0x7f2dd419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 >> 172.21.15.94:0/1695954086 conn(0x7f2dd40fdbd0 msgr2=0x7f2dd4109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 shutdown_connections 2024-04-25T13:47:49.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.662+0000 7f2ddb9f8640 1 -- 172.21.15.94:0/1695954086 wait complete. 2024-04-25T13:47:49.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:47:49.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 675229 13.10 MB/sec execute 152 sec latency 77.275 ms 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.813+0000 7fb792314640 1 -- 172.21.15.94:0/3852890171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 msgr2=0x7fb78c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.813+0000 7fb792314640 1 --2- 172.21.15.94:0/3852890171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c10ad50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb784009ca0 tx=0x7fb78402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 -- 172.21.15.94:0/3852890171 shutdown_connections 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 --2- 172.21.15.94:0/3852890171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c10ad50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 --2- 172.21.15.94:0/3852890171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 0x7fb78c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 --2- 172.21.15.94:0/3852890171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 -- 172.21.15.94:0/3852890171 >> 172.21.15.94:0/3852890171 conn(0x7fb78c0fdc00 msgr2=0x7fb78c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 -- 172.21.15.94:0/3852890171 shutdown_connections 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.814+0000 7fb792314640 1 -- 172.21.15.94:0/3852890171 wait complete. 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 Processor -- start 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 -- start start 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 0x7fb78c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c10e500 con 0x7fb78c103c30 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c10e380 con 0x7fb78c1029f0 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.815+0000 7fb792314640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c10e680 con 0x7fb78c108930 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb7908b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34194/0 (socket says 172.21.15.94:34194) 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb7908b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34030/0 (socket says 172.21.15.94:34030) 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 -- 172.21.15.94:0/3712726763 learned_addr learned my addr 172.21.15.94:0/3712726763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 0x7fb78c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 -- 172.21.15.94:0/3712726763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 msgr2=0x7fb78c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 -- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 msgr2=0x7fb78c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 0x7fb78c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 -- 172.21.15.94:0/3712726763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c19fac0 con 0x7fb78c1029f0 2024-04-25T13:47:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.816+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005fd0 tx=0x7fb77c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.817+0000 7fb7897fa640 1 -- 172.21.15.94:0/3712726763 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c017070 con 0x7fb78c1029f0 2024-04-25T13:47:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.817+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c19fdb0 con 0x7fb78c1029f0 2024-04-25T13:47:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.817+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c1a0350 con 0x7fb78c1029f0 2024-04-25T13:47:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.817+0000 7fb7897fa640 1 -- 172.21.15.94:0/3712726763 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77c009c30 con 0x7fb78c1029f0 2024-04-25T13:47:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.817+0000 7fb7897fa640 1 -- 172.21.15.94:0/3712726763 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003d10 con 0x7fb78c1029f0 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.819+0000 7fb7991c9740 1 Processor -- start 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 -- start start 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7897fa640 1 -- 172.21.15.94:0/3712726763 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb77c010460 con 0x7fb78c1029f0 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959cf4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cf4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556959c58d80 con 0x556959952390 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556959c12a20 con 0x5569599294d0 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7991c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556959c54d40 con 0x5569599316b0 2024-04-25T13:47:49.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34034/0 (socket says 172.21.15.94:34034) 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb78b7fe640 1 -- 172.21.15.94:0/477577521 learned_addr learned my addr 172.21.15.94:0/477577521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb7908b1640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959cf4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.820+0000 7fb78b7fe640 1 -- 172.21.15.94:0/477577521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 msgr2=0x556959cf4d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78bfff640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cf4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7897fa640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 0x7fb7740b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cf4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78b7fe640 1 -- 172.21.15.94:0/477577521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 msgr2=0x556959cf4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959cf4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78b7fe640 1 -- 172.21.15.94:0/477577521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556959c1dc30 con 0x5569599316b0 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7908b1640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959cf4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7897fa640 1 -- 172.21.15.94:0/3712726763 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb77c0de8b0 con 0x7fb78c1029f0 2024-04-25T13:47:49.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb78000bc30 tx=0x7fb7800107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9a52ed090d9939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7908b1640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 0x7fb7740b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7727fc640 1 -- 172.21.15.94:0/477577521 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78001c480 con 0x5569599316b0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7727fc640 1 -- 172.21.15.94:0/477577521 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78001cad0 con 0x5569599316b0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.821+0000 7fb7727fc640 1 -- 172.21.15.94:0/477577521 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780011b40 con 0x5569599316b0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- 172.21.15.94:0/477577521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 msgr2=0x556959cf5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7908b1640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 0x7fb7740b2080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb784005e30 tx=0x7fb7840095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb78000bc30 tx=0x7fb7800107a0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- 172.21.15.94:0/477577521 shutdown_connections 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599316b0 0x556959cf5a20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cf4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 --2- 172.21.15.94:0/477577521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959cf4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- 172.21.15.94:0/477577521 >> 172.21.15.94:0/477577521 conn(0x5569599464e0 msgr2=0x556959cf0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- 172.21.15.94:0/477577521 shutdown_connections 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- 172.21.15.94:0/477577521 wait complete. 2024-04-25T13:47:49.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.822+0000 7fb7991c9740 1 -- start start 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 Processor -- start 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cfe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556959952390 0x556959d059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556959cf6530 con 0x5569599316b0 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556959bdfdc0 con 0x5569599294d0 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556959cf66b0 con 0x556959952390 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7991c9740 10 client.0 _ll_register_callbacks cb 0x556959bfb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cfe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49102/0 (socket says 172.21.15.94:49102) 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 -- 172.21.15.94:0/1874119566 learned_addr learned my addr 172.21.15.94:0/1874119566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb7908b1640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556959952390 0x556959d059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 -- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556959952390 msgr2=0x556959d059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556959952390 0x556959d059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 -- 172.21.15.94:0/1874119566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 msgr2=0x556959cfe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cfe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78b7fe640 1 -- 172.21.15.94:0/1874119566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55695996a3f0 con 0x5569599316b0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.823+0000 7fb78bfff640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cfe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb780011010 tx=0x7fb78001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c42f1c14d10f99c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002b060 con 0x5569599316b0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb780004040 con 0x5569599316b0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78001cad0 con 0x5569599316b0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb780004240 con 0x5569599316b0 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 10 client.17658 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:49.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556959a347d0 con 0x5569599316b0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78001cad0 con 0x5569599316b0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb780011bb0 con 0x5569599316b0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 10 client.17658 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 10 client.17658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 20 client.17658 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.824+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556959910f60 0x556959d0bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.825+0000 7fb7991c9740 4 client.17658 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.825+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556959c66990 con 0x556959910f60 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.825+0000 7fb78bfff640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556959910f60 0x556959d0bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.825+0000 7fb76b7fe640 10 client.17658 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.825+0000 7fb78bfff640 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556959910f60 0x556959d0bf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dee697d1ee5bcdd5 server_cookie=44e411131eaaf5c1 in_seq=0 out_seq=0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb76b7fe640 1 -- 172.21.15.94:0/1874119566 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556959c66990 con 0x556959910f60 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb76b7fe640 10 client.17658 handle_command_reply: tid=0 2024-04-25T13:47:49.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 client.17658 shutdown 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 msgr2=0x556959cfea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb780011010 tx=0x7fb78001ceb0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 shutdown_connections 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556959910f60 0x556959d0bf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb76b7fe640 0 client.17658 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556959952390 0x556959d059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959cfea80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 --2- 172.21.15.94:0/1874119566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599294d0 0x556959cfe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.826+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 >> 172.21.15.94:0/1874119566 conn(0x5569599464e0 msgr2=0x556959cef040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.827+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 shutdown_connections 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.827+0000 7fb7991c9740 1 -- 172.21.15.94:0/1874119566 wait complete. 2024-04-25T13:47:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.827+0000 7fb7991c9740 20 client.17658 trim_cache size 0 max 16384 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.841+0000 7fb7991c9740 1 Processor -- start 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.841+0000 7fb7991c9740 1 -- start start 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.841+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959ceb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556959952390 0x556959cf9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7991c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb77800e730 con 0x5569599316b0 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7991c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb778010270 con 0x556959952390 2024-04-25T13:47:49.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7991c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb77800ee00 con 0x5569599294d0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49112/0 (socket says 172.21.15.94:49112) 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 -- 172.21.15.94:0/403270154 learned_addr learned my addr 172.21.15.94:0/403270154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb7908b1640 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556959952390 0x556959cf9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78bfff640 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959ceb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 -- 172.21.15.94:0/403270154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 msgr2=0x556959ceb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959ceb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 -- 172.21.15.94:0/403270154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556959952390 msgr2=0x556959cf9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556959952390 0x556959cf9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 -- 172.21.15.94:0/403270154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556959cff140 con 0x5569599316b0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b620 tx=0x7fb7800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6571a8b3fc309055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb771ffb640 1 -- 172.21.15.94:0/403270154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002b040 con 0x5569599316b0 2024-04-25T13:47:49.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb771ffb640 1 -- 172.21.15.94:0/403270154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb780005850 con 0x5569599316b0 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.842+0000 7fb771ffb640 1 -- 172.21.15.94:0/403270154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78001c8a0 con 0x5569599316b0 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- 172.21.15.94:0/403270154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 msgr2=0x556959d05900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b620 tx=0x7fb7800107a0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- 172.21.15.94:0/403270154 shutdown_connections 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556959952390 0x556959cf9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569599316b0 0x556959d05900 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 --2- 172.21.15.94:0/403270154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959ceb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- 172.21.15.94:0/403270154 >> 172.21.15.94:0/403270154 conn(0x5569599464e0 msgr2=0x556959cf4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- 172.21.15.94:0/403270154 shutdown_connections 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- 172.21.15.94:0/403270154 wait complete. 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.843+0000 7fb7991c9740 1 -- start start 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 Processor -- start 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959d02ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 0x556959d030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb77800fba0 con 0x556959952390 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb77800f4d0 con 0x5569599316b0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb77800c710 con 0x5569599294d0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7991c9740 10 client.0 _ll_register_callbacks cb 0x7fb784005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 0x556959d030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49122/0 (socket says 172.21.15.94:49122) 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 -- 172.21.15.94:0/3960738885 learned_addr learned my addr 172.21.15.94:0/3960738885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959d02ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 -- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 msgr2=0x556959d02ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959d02ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 -- 172.21.15.94:0/3960738885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 msgr2=0x556959d030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 0x556959d030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb7908b1640 1 -- 172.21.15.94:0/3960738885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556959d045d0 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb78b7fe640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 0x556959d030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.844+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959d02ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7908b1640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb784009ca0 tx=0x7fb78402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f98d9b8b9a2ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840305c0 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb784030760 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb780005850 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb784030960 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 10 client.17664 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556959d058f0 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb780005850 con 0x556959952390 2024-04-25T13:47:49.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb784003e40 con 0x556959952390 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 10 client.17664 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 10 client.17664 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 20 client.17664 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5569598e63a0 0x556959d76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 4 client.17664 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.845+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556959c60800 con 0x5569598e63a0 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5569598e63a0 0x556959d76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb76b7fe640 10 client.17664 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb78bfff640 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5569598e63a0 0x556959d76710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=deb9a0555eb78e2a server_cookie=eadd7eadbbdd29b3 in_seq=0 out_seq=0 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb76b7fe640 1 -- 172.21.15.94:0/3960738885 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556959c60800 con 0x5569598e63a0 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb76b7fe640 10 client.17664 handle_command_reply: tid=0 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 client.17664 shutdown 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 msgr2=0x556959d04010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb784009ca0 tx=0x7fb78402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 shutdown_connections 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5569598e63a0 0x556959d76710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556959952390 0x556959d04010 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569599316b0 0x556959d030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 --2- 172.21.15.94:0/3960738885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569599294d0 0x556959d02ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 >> 172.21.15.94:0/3960738885 conn(0x5569599464e0 msgr2=0x556959cf9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.846+0000 7fb76b7fe640 0 client.17664 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 shutdown_connections 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb7991c9740 1 -- 172.21.15.94:0/3960738885 wait complete. 2024-04-25T13:47:49.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb7991c9740 20 client.17664 trim_cache size 0 max 16384 2024-04-25T13:47:49.853 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:49.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 msgr2=0x7fb7740b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 0x7fb7740b2080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb784005e30 tx=0x7fb7840095c0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 msgr2=0x7fb78c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005fd0 tx=0x7fb77c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 shutdown_connections 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7740afb70 0x7fb7740b2080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb78c108930 0x7fb78c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb78c103c30 0x7fb78c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 --2- 172.21.15.94:0/3712726763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb78c1029f0 0x7fb78c197f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 >> 172.21.15.94:0/3712726763 conn(0x7fb78c0fdc00 msgr2=0x7fb78c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 shutdown_connections 2024-04-25T13:47:49.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.847+0000 7fb792314640 1 -- 172.21.15.94:0/3712726763 wait complete. 2024-04-25T13:47:49.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 -- 172.21.15.94:0/2952388427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558102fb0 msgr2=0x7f75581033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 --2- 172.21.15.94:0/2952388427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558102fb0 0x7f75581033c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f755402a410 comp rx=0 tx=0).stop 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 -- 172.21.15.94:0/2952388427 shutdown_connections 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 --2- 172.21.15.94:0/2952388427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104670 0x7f755810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 --2- 172.21.15.94:0/2952388427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558103b10 0x7f7558103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 --2- 172.21.15.94:0/2952388427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558102fb0 0x7f75581033c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 -- 172.21.15.94:0/2952388427 >> 172.21.15.94:0/2952388427 conn(0x7f75580fd350 msgr2=0x7f75580ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 -- 172.21.15.94:0/2952388427 shutdown_connections 2024-04-25T13:47:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.974+0000 7f755f384640 1 -- 172.21.15.94:0/2952388427 wait complete. 2024-04-25T13:47:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.975+0000 7f755f384640 1 Processor -- start 2024-04-25T13:47:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.975+0000 7f755f384640 1 -- start start 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.975+0000 7f755f384640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 0x7f75581022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.975+0000 7f755f384640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.975+0000 7f755f384640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 0x7f75581a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755f384640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f755810e710 con 0x7f7558103b10 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755f384640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f755810e590 con 0x7f7558102fb0 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755f384640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f755810e890 con 0x7f755819f300 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49142/0 (socket says 172.21.15.94:49142) 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755d120640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 0x7f75581022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 -- 172.21.15.94:0/631666230 learned_addr learned my addr 172.21.15.94:0/631666230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755d921640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 0x7f75581a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 -- 172.21.15.94:0/631666230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 msgr2=0x7f75581a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 0x7f75581a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 -- 172.21.15.94:0/631666230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 msgr2=0x7f75581022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 0x7f75581022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 -- 172.21.15.94:0/631666230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75581a1fa0 con 0x7f7558103b10 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755d921640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 0x7f75581a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755d120640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 0x7f75581022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755c91f640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f754e7fc640 1 -- 172.21.15.94:0/631666230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800ccc0 con 0x7f7558103b10 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75581a2290 con 0x7f7558103b10 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f754e7fc640 1 -- 172.21.15.94:0/631666230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754800ce60 con 0x7f7558103b10 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f754e7fc640 1 -- 172.21.15.94:0/631666230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548010430 con 0x7f7558103b10 2024-04-25T13:47:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.976+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75581a27e0 con 0x7f7558103b10 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 Processor -- start 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 -- start start 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f754e7fc640 1 -- 172.21.15.94:0/631666230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f75480105f0 con 0x7f7558103b10 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4bb1390 0x5582b4f53810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b906b0 0x5582b4f54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5582b4eb7d80 con 0x5582b4b906b0 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5582b4e71a20 con 0x5582b4b884d0 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.979+0000 7f7566234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582b4eb3d40 con 0x5582b4bb1390 2024-04-25T13:47:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d921640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4bb1390 0x5582b4f53810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755c91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b906b0 0x5582b4f54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34236/0 (socket says 172.21.15.94:34236) 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 -- 172.21.15.94:0/2226377065 learned_addr learned my addr 172.21.15.94:0/2226377065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 -- 172.21.15.94:0/2226377065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4bb1390 msgr2=0x5582b4f53810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4bb1390 0x5582b4f53810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 -- 172.21.15.94:0/2226377065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b906b0 msgr2=0x5582b4f54a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b906b0 0x5582b4f54a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 -- 172.21.15.94:0/2226377065 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b4e7cc30 con 0x5582b4b884d0 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d120640 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f7554032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d6b8e15ab0f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f754e7fc640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 0x7f75440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f755d921640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 0x7f75440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f75437fe640 1 -- 172.21.15.94:0/2226377065 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554035070 con 0x5582b4b884d0 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f75437fe640 1 -- 172.21.15.94:0/2226377065 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540317a0 con 0x5582b4b884d0 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f75437fe640 1 -- 172.21.15.94:0/2226377065 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554003d70 con 0x5582b4b884d0 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.980+0000 7f754e7fc640 1 -- 172.21.15.94:0/631666230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f75480df060 con 0x7f7558103b10 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 -- 172.21.15.94:0/2226377065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 msgr2=0x5582b4f53d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f7554032f90 comp rx=0 tx=0).stop 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f755d921640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 0x7f75440b2000 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5582b4f54860 tx=0x7f7550009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 -- 172.21.15.94:0/2226377065 shutdown_connections 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b906b0 0x5582b4f54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f53d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 --2- 172.21.15.94:0/2226377065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4bb1390 0x5582b4f53810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 -- 172.21.15.94:0/2226377065 >> 172.21.15.94:0/2226377065 conn(0x5582b4ba54e0 msgr2=0x5582b4f4f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 -- 172.21.15.94:0/2226377065 shutdown_connections 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.981+0000 7f7566234740 1 -- 172.21.15.94:0/2226377065 wait complete. 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 -- start start 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 Processor -- start 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f5d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f5db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5582b4f554d0 con 0x5582b4bb1390 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5582b4e3edc0 con 0x5582b4b884d0 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582b4f55650 con 0x5582b4b906b0 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f7566234740 10 client.0 _ll_register_callbacks cb 0x5582b4e5a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:49.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755d921640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755d120640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f5d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755c91f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f5db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755d120640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f5d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34242/0 (socket says 172.21.15.94:34242) 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755d921640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f64a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49158/0 (socket says 172.21.15.94:49158) 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.982+0000 7f755d120640 1 -- 172.21.15.94:0/2706133209 learned_addr learned my addr 172.21.15.94:0/2706133209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 -- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 msgr2=0x5582b4f5d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f5d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 -- 172.21.15.94:0/2706133209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 msgr2=0x5582b4f64a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f64a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 -- 172.21.15.94:0/2706133209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b4bc93f0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f755c91f640 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f5db00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75480dcb10 tx=0x7f7548005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7bd7960f20710e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75480a6c10 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f75480046c0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75540317a0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75480a6fb0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7566234740 10 client.27031 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582b4c937d0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.983+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540317a0 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f75480e1110 con 0x5582b4b906b0 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 10 client.27031 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 10 client.27031 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 20 client.27031 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b6ff60 0x5582b4f6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 4 client.27031 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b4ec5990 con 0x5582b4b6ff60 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f755d120640 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b6ff60 0x5582b4f6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f7540ff9640 10 client.27031 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.984+0000 7f755d120640 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b6ff60 0x5582b4f6b350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b40fadd0129f544 server_cookie=a0651ad399f6ad5a in_seq=0 out_seq=0 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7540ff9640 1 -- 172.21.15.94:0/2706133209 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582b4ec5990 con 0x5582b4b6ff60 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7540ff9640 10 client.27031 handle_command_reply: tid=0 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 client.27031 shutdown 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 msgr2=0x5582b4f5db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f5db00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75480dcb10 tx=0x7f7548005740 comp rx=0 tx=0).stop 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 shutdown_connections 2024-04-25T13:47:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b6ff60 0x5582b4f6b350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f64a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f5db00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7540ff9640 0 client.27031 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 --2- 172.21.15.94:0/2706133209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b884d0 0x5582b4f5d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 >> 172.21.15.94:0/2706133209 conn(0x5582b4ba54e0 msgr2=0x5582b4f51f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 shutdown_connections 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 1 -- 172.21.15.94:0/2706133209 wait complete. 2024-04-25T13:47:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:49.985+0000 7f7566234740 20 client.27031 trim_cache size 0 max 16384 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 Processor -- start 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 -- start start 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b884d0 0x5582b4f5c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b906b0 0x5582b4f64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75581ab550 con 0x5582b4bb1390 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75581ad090 con 0x5582b4b906b0 2024-04-25T13:47:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f7566234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75581abc20 con 0x5582b4b884d0 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49160/0 (socket says 172.21.15.94:49160) 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 -- 172.21.15.94:0/3944390143 learned_addr learned my addr 172.21.15.94:0/3944390143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d120640 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b884d0 0x5582b4f5c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755c91f640 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b906b0 0x5582b4f64980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 -- 172.21.15.94:0/3944390143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b884d0 msgr2=0x5582b4f5c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b884d0 0x5582b4f5c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 -- 172.21.15.94:0/3944390143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b906b0 msgr2=0x5582b4f64980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b906b0 0x5582b4f64980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.002+0000 7f755d921640 1 -- 172.21.15.94:0/3944390143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b4f5ba50 con 0x5582b4bb1390 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.003+0000 7f755d921640 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f755000fa40 tx=0x7f755001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfc536c75a26a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.003+0000 7f7542ffd640 1 -- 172.21.15.94:0/3944390143 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500119c0 con 0x5582b4bb1390 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.003+0000 7f7542ffd640 1 -- 172.21.15.94:0/3944390143 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550011b60 con 0x5582b4bb1390 2024-04-25T13:47:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.003+0000 7f7542ffd640 1 -- 172.21.15.94:0/3944390143 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755001f660 con 0x5582b4bb1390 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- 172.21.15.94:0/3944390143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 msgr2=0x5582b4f4d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f755000fa40 tx=0x7f755001a000 comp rx=0 tx=0).stop 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- 172.21.15.94:0/3944390143 shutdown_connections 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4bb1390 0x5582b4f4d610 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4b906b0 0x5582b4f64980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- 172.21.15.94:0/3944390143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b884d0 0x5582b4f5c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- 172.21.15.94:0/3944390143 >> 172.21.15.94:0/3944390143 conn(0x5582b4ba54e0 msgr2=0x5582b4f5bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- 172.21.15.94:0/3944390143 shutdown_connections 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- 172.21.15.94:0/3944390143 wait complete. 2024-04-25T13:47:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 -- start start 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 Processor -- start 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.004+0000 7f7566234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4bb1390 0x5582b4f636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7566234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75581ac9c0 con 0x5582b4b884d0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7566234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75581ac2f0 con 0x5582b4bb1390 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7566234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75581a9530 con 0x5582b4b906b0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7566234740 10 client.0 _ll_register_callbacks cb 0x7f754800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755c91f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49162/0 (socket says 172.21.15.94:49162) 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d921640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4bb1390 0x5582b4f636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 -- 172.21.15.94:0/1187063767 learned_addr learned my addr 172.21.15.94:0/1187063767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755c91f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34116/0 (socket says 172.21.15.94:34116) 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 -- 172.21.15.94:0/1187063767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 msgr2=0x5582b4f627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 -- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4bb1390 msgr2=0x5582b4f636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4bb1390 0x5582b4f636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 -- 172.21.15.94:0/1187063767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b4f63cb0 con 0x5582b4b884d0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f755d120640 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7554031500 tx=0x7f75540317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc29991c4d93b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554035040 con 0x5582b4b884d0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7554031b60 con 0x5582b4b884d0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.005+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550011b60 con 0x5582b4b884d0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f755404a040 con 0x5582b4b884d0 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 10 client.17682 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582b4fd3800 con 0x5582b4b884d0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550011b60 con 0x5582b4b884d0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7554065450 con 0x5582b4b884d0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 10 client.17682 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 10 client.17682 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 20 client.17682 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b453a0 0x5582b4fd5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 4 client.17682 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582b4ebf800 con 0x5582b4b453a0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f755c91f640 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b453a0 0x5582b4fd5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.006+0000 7f7540ff9640 10 client.17682 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f755c91f640 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b453a0 0x5582b4fd5cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b90c5c1850b5f396 server_cookie=9cfb07ed55a13fb8 in_seq=0 out_seq=0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7540ff9640 1 -- 172.21.15.94:0/1187063767 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582b4ebf800 con 0x5582b4b453a0 2024-04-25T13:47:50.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7540ff9640 10 client.17682 handle_command_reply: tid=0 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 client.17682 shutdown 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 msgr2=0x5582b4f62280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7554031500 tx=0x7f75540317f0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 shutdown_connections 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5582b4b453a0 0x5582b4fd5cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582b4bb1390 0x5582b4f636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7540ff9640 0 client.17682 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b4b906b0 0x5582b4f627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 --2- 172.21.15.94:0/1187063767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5582b4b884d0 0x5582b4f62280 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 >> 172.21.15.94:0/1187063767 conn(0x5582b4ba54e0 msgr2=0x5582b4f51e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 shutdown_connections 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 1 -- 172.21.15.94:0/1187063767 wait complete. 2024-04-25T13:47:50.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.007+0000 7f7566234740 20 client.17682 trim_cache size 0 max 16384 2024-04-25T13:47:50.013 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:50.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 msgr2=0x7f75440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 0x7f75440b2000 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5582b4f54860 tx=0x7f7550009500 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 msgr2=0x7f755819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f754800e9f0 tx=0x7f754800edf0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 shutdown_connections 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75440afaf0 0x7f75440b2000 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755819f300 0x7f75581a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7558103b10 0x7f755819edc0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 --2- 172.21.15.94:0/631666230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558102fb0 0x7f75581022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 >> 172.21.15.94:0/631666230 conn(0x7f75580fd350 msgr2=0x7f75580ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 shutdown_connections 2024-04-25T13:47:50.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.008+0000 7f755f384640 1 -- 172.21.15.94:0/631666230 wait complete. 2024-04-25T13:47:50.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:47:50.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 -- 172.21.15.94:0/4217223909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1033d0 msgr2=0x7f312c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4217223909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1033d0 0x7f312c1037e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f311c009900 tx=0x7f311c02a410 comp rx=0 tx=0).stop 2024-04-25T13:47:50.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 -- 172.21.15.94:0/4217223909 shutdown_connections 2024-04-25T13:47:50.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4217223909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1080d0 0x7f312c10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4217223909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1033d0 0x7f312c1037e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4217223909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 -- 172.21.15.94:0/4217223909 >> 172.21.15.94:0/4217223909 conn(0x7f312c0fd380 msgr2=0x7f312c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 -- 172.21.15.94:0/4217223909 shutdown_connections 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.134+0000 7f3133ae2640 1 -- 172.21.15.94:0/4217223909 wait complete. 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 Processor -- start 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 -- start start 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1033d0 0x7f312c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10dd00 con 0x7f312c1080d0 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10db80 con 0x7f312c1033d0 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f3133ae2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10de80 con 0x7f312c102190 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313207f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313207f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49176/0 (socket says 172.21.15.94:49176) 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313187e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313207f640 1 -- 172.21.15.94:0/4187739100 learned_addr learned my addr 172.21.15.94:0/4187739100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313107d640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1033d0 0x7f312c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313207f640 1 -- 172.21.15.94:0/4187739100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 msgr2=0x7f312c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.135+0000 7f313207f640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f313207f640 1 -- 172.21.15.94:0/4187739100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1033d0 msgr2=0x7f312c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f313207f640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1033d0 0x7f312c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f313207f640 1 -- 172.21.15.94:0/4187739100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1a0c50 con 0x7f312c1080d0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f313187e640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f313207f640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f312400eb10 tx=0x7f312400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f3122ffd640 1 -- 172.21.15.94:0/4187739100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124004b20 con 0x7f312c1080d0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f3122ffd640 1 -- 172.21.15.94:0/4187739100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3124004cc0 con 0x7f312c1080d0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1a0f40 con 0x7f312c1080d0 2024-04-25T13:47:50.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f3122ffd640 1 -- 172.21.15.94:0/4187739100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124010670 con 0x7f312c1080d0 2024-04-25T13:47:50.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.136+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1a1350 con 0x7f312c1080d0 2024-04-25T13:47:50.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.138+0000 7f3122ffd640 1 -- 172.21.15.94:0/4187739100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3124005020 con 0x7f312c1080d0 2024-04-25T13:47:50.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 Processor -- start 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 -- start start 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5dc6d80 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5d80a20 con 0x55bfb5ac0390 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313a98e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5dc2d40 con 0x55bfb5a974d0 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313107d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313107d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34286/0 (socket says 172.21.15.94:34286) 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313107d640 1 -- 172.21.15.94:0/2245122550 learned_addr learned my addr 172.21.15.94:0/2245122550 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313107d640 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f313187e640 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.139+0000 7f3122ffd640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 0x7f31180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313207f640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 0x7f31180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 -- 172.21.15.94:0/2245122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 msgr2=0x55bfb5e62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 -- 172.21.15.94:0/2245122550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a9f6b0 msgr2=0x55bfb5e63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e63a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 -- 172.21.15.94:0/2245122550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb5d8bc30 con 0x55bfb5ac0390 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f3122ffd640 1 -- 172.21.15.94:0/4187739100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f31240df240 con 0x7f312c1080d0 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313107d640 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f311c005de0 tx=0x7f311c005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb0b46e956a5dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f3117fff640 1 -- 172.21.15.94:0/2245122550 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c035040 con 0x55bfb5ac0390 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f313207f640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 0x7f31180b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f31240a48f0 tx=0x7f3124071ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f3117fff640 1 -- 172.21.15.94:0/2245122550 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c0315a0 con 0x55bfb5ac0390 2024-04-25T13:47:50.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.140+0000 7f3117fff640 1 -- 172.21.15.94:0/2245122550 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03d690 con 0x55bfb5ac0390 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 -- 172.21.15.94:0/2245122550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 msgr2=0x55bfb5e62810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f311c005de0 tx=0x7f311c005d70 comp rx=0 tx=0).stop 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 -- 172.21.15.94:0/2245122550 shutdown_connections 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 --2- 172.21.15.94:0/2245122550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e62810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 -- 172.21.15.94:0/2245122550 >> 172.21.15.94:0/2245122550 conn(0x55bfb5ab44e0 msgr2=0x55bfb5e5e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 -- 172.21.15.94:0/2245122550 shutdown_connections 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.141+0000 7f313a98e740 1 -- 172.21.15.94:0/2245122550 wait complete. 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 -- start start 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 Processor -- start 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e6c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5ac0390 0x55bfb5e73950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5e64470 con 0x55bfb5ac0390 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5d4ddc0 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfb5e645f0 con 0x55bfb5a974d0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313a98e740 10 client.0 _ll_register_callbacks cb 0x55bfb5d69600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313107d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5ac0390 0x55bfb5e73950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313107d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313187e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e6c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313107d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34146/0 (socket says 172.21.15.94:34146) 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313187e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e6c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34290/0 (socket says 172.21.15.94:34290) 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.142+0000 7f313107d640 1 -- 172.21.15.94:0/3683454189 learned_addr learned my addr 172.21.15.94:0/3683454189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 -- 172.21.15.94:0/3683454189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 msgr2=0x55bfb5e6c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e6c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 -- 172.21.15.94:0/3683454189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5ac0390 msgr2=0x55bfb5e73950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5ac0390 0x55bfb5e73950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 -- 172.21.15.94:0/3683454189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb5ad83f0 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313107d640 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f311c031500 tx=0x7f311c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa37e991d0be452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c035030 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f311c003ac0 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311c0315a0 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f311c031b90 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313a98e740 10 client.27049 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfb5ba27d0 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c0315a0 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.143+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f311c045040 con 0x55bfb5a974d0 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 10 client.27049 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 10 client.27049 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:47:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 20 client.27049 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a7ef60 0x55bfb5e79da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 4 client.27049 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb5dd4990 con 0x55bfb5a7ef60 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313187e640 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a7ef60 0x55bfb5e79da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f31157fa640 10 client.27049 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.144+0000 7f313187e640 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a7ef60 0x55bfb5e79da0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bb44846c31ba4c8 server_cookie=4c26b502d1d23094 in_seq=0 out_seq=0 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f31157fa640 1 -- 172.21.15.94:0/3683454189 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfb5dd4990 con 0x55bfb5a7ef60 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f31157fa640 10 client.27049 handle_command_reply: tid=0 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 client.27049 shutdown 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 msgr2=0x55bfb5e6c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f311c031500 tx=0x7f311c02a410 comp rx=0 tx=0).stop 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 shutdown_connections 2024-04-25T13:47:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a7ef60 0x55bfb5e79da0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5ac0390 0x55bfb5e73950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f31157fa640 0 client.27049 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e6c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 --2- 172.21.15.94:0/3683454189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6c4b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 >> 172.21.15.94:0/3683454189 conn(0x55bfb5ab44e0 msgr2=0x55bfb5e60f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 shutdown_connections 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 1 -- 172.21.15.94:0/3683454189 wait complete. 2024-04-25T13:47:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.145+0000 7f313a98e740 20 client.27049 trim_cache size 0 max 16384 2024-04-25T13:47:50.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 Processor -- start 2024-04-25T13:47:50.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 -- start start 2024-04-25T13:47:50.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1a9310 con 0x55bfb5a974d0 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1aae50 con 0x55bfb5ac0390 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313a98e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1a99e0 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313107d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313187e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313187e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34152/0 (socket says 172.21.15.94:34152) 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.160+0000 7f313107d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49198/0 (socket says 172.21.15.94:49198) 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 -- 172.21.15.94:0/3885894268 learned_addr learned my addr 172.21.15.94:0/3885894268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313107d640 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e6c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 -- 172.21.15.94:0/3885894268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 msgr2=0x55bfb5e6c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e6c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 -- 172.21.15.94:0/3885894268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 msgr2=0x55bfb5e6b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 -- 172.21.15.94:0/3885894268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb5e6b890 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f313187e640 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f312c102d90 tx=0x7f3128005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43eb1950f0149bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f31177fe640 1 -- 172.21.15.94:0/3885894268 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128004490 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f31177fe640 1 -- 172.21.15.94:0/3885894268 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3128010430 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.161+0000 7f31177fe640 1 -- 172.21.15.94:0/3885894268 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312801e680 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 -- 172.21.15.94:0/3885894268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 msgr2=0x55bfb5e738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f312c102d90 tx=0x7f3128005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 -- 172.21.15.94:0/3885894268 shutdown_connections 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5ac0390 0x55bfb5e6c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e738e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 --2- 172.21.15.94:0/3885894268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e6b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 -- 172.21.15.94:0/3885894268 >> 172.21.15.94:0/3885894268 conn(0x55bfb5ab44e0 msgr2=0x55bfb5e6ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 -- 172.21.15.94:0/3885894268 shutdown_connections 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.162+0000 7f313a98e740 1 -- 172.21.15.94:0/3885894268 wait complete. 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 -- start start 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 Processor -- start 2024-04-25T13:47:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5ac0390 0x55bfb5edff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1aa780 con 0x55bfb5a974d0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1aa0b0 con 0x55bfb5a9f6b0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1a72f0 con 0x55bfb5ac0390 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313a98e740 10 client.0 _ll_register_callbacks cb 0x7f311c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313107d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313187e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313107d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49210/0 (socket says 172.21.15.94:49210) 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313107d640 1 -- 172.21.15.94:0/2311367265 learned_addr learned my addr 172.21.15.94:0/2311367265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.163+0000 7f313107d640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5ac0390 0x55bfb5edff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 -- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5ac0390 msgr2=0x55bfb5edff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5ac0390 0x55bfb5edff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 -- 172.21.15.94:0/2311367265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 msgr2=0x55bfb5e70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 -- 172.21.15.94:0/2311367265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb5ee04b0 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313107d640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f311c057f00 tx=0x7f311c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e82d83166758c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c035070 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f311c057860 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3128010430 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f311c057a60 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313a98e740 10 client.17700 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfb5ee1630 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3128010430 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f311c03d8b0 con 0x55bfb5a974d0 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313a98e740 10 client.17700 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.164+0000 7f313a98e740 10 client.17700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:47:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f313a98e740 20 client.17700 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a543a0 0x55bfb5ee4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f313a98e740 4 client.17700 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfb5dce800 con 0x55bfb5a543a0 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f313187e640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a543a0 0x55bfb5ee4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.165+0000 7f31157fa640 10 client.17700 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313187e640 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a543a0 0x55bfb5ee4050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdc38c05eb31622b server_cookie=8e8c1b286667f0f8 in_seq=0 out_seq=0 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f31157fa640 1 -- 172.21.15.94:0/2311367265 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfb5dce800 con 0x55bfb5a543a0 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f31157fa640 10 client.17700 handle_command_reply: tid=0 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 client.17700 shutdown 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 msgr2=0x55bfb5e70860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f311c057f00 tx=0x7f311c0041d0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 shutdown_connections 2024-04-25T13:47:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfb5a543a0 0x55bfb5ee4050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb5ac0390 0x55bfb5edff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfb5a9f6b0 0x55bfb5e70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 --2- 172.21.15.94:0/2311367265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfb5a974d0 0x55bfb5e70860 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f31157fa640 0 client.17700 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 >> 172.21.15.94:0/2311367265 conn(0x55bfb5ab44e0 msgr2=0x55bfb5e6c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 shutdown_connections 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 1 -- 172.21.15.94:0/2311367265 wait complete. 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.166+0000 7f313a98e740 20 client.17700 trim_cache size 0 max 16384 2024-04-25T13:47:50.171 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 msgr2=0x7f31180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 0x7f31180b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f31240a48f0 tx=0x7f3124071ea0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 msgr2=0x7f312c1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f312400eb10 tx=0x7f312400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 shutdown_connections 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31180afaf0 0x7f31180b2000 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f312c1080d0 0x7f312c1a05e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f312c1033d0 0x7f312c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 --2- 172.21.15.94:0/4187739100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f312c102190 0x7f312c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 >> 172.21.15.94:0/4187739100 conn(0x7f312c0fd380 msgr2=0x7f312c10a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 shutdown_connections 2024-04-25T13:47:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.167+0000 7f3133ae2640 1 -- 172.21.15.94:0/4187739100 wait complete. 2024-04-25T13:47:50.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:47:50.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/1672697503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 msgr2=0x7ff8c81054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/1672697503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81054a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009900 tx=0x7ff8b802a410 comp rx=0 tx=0).stop 2024-04-25T13:47:50.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/1672697503 shutdown_connections 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/1672697503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8102240 0x7ff8c810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/1672697503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8101740 0x7ff8c8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/1672697503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81054a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/1672697503 >> 172.21.15.94:0/1672697503 conn(0x7ff8c80fd340 msgr2=0x7ff8c80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/1672697503 shutdown_connections 2024-04-25T13:47:50.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.292+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/1672697503 wait complete. 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 Processor -- start 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 -- start start 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 0x7ff8c819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 0x7ff8c819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810f5e0 con 0x7ff8c8105090 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810f460 con 0x7ff8c8101740 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.293+0000 7ff8cd0b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810f760 con 0x7ff8c8102240 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 0x7ff8c819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 0x7ff8c819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49228/0 (socket says 172.21.15.94:49228) 2024-04-25T13:47:50.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 -- 172.21.15.94:0/4090180044 learned_addr learned my addr 172.21.15.94:0/4090180044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 -- 172.21.15.94:0/4090180044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 msgr2=0x7ff8c819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 0x7ff8c819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 -- 172.21.15.94:0/4090180044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 msgr2=0x7ff8c819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 0x7ff8c819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 -- 172.21.15.94:0/4090180044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81a1d80 con 0x7ff8c8105090 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c659c640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 0x7ff8c819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 0x7ff8c819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8c759e640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eb10 tx=0x7ff8c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8affff640 1 -- 172.21.15.94:0/4090180044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004b20 con 0x7ff8c8105090 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81a2070 con 0x7ff8c8105090 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8affff640 1 -- 172.21.15.94:0/4090180044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0004cc0 con 0x7ff8c8105090 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8affff640 1 -- 172.21.15.94:0/4090180044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010670 con 0x7ff8c8105090 2024-04-25T13:47:50.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.294+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a2570 con 0x7ff8c8105090 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.297+0000 7ff8affff640 1 -- 172.21.15.94:0/4090180044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff8c0005020 con 0x7ff8c8105090 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.297+0000 7ff8d3f6b740 1 Processor -- start 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.297+0000 7ff8d3f6b740 1 -- start start 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a013d80 con 0x55cb19ce44d0 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb19fcda20 con 0x55cb19cec6b0 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a00fd40 con 0x55cb19d0d390 2024-04-25T13:47:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49230/0 (socket says 172.21.15.94:49230) 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 -- 172.21.15.94:0/504010605 learned_addr learned my addr 172.21.15.94:0/504010605 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8affff640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 0x7ff8b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c659c640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c659c640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c759e640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 0x7ff8b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 -- 172.21.15.94:0/504010605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 msgr2=0x55cb1a0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 -- 172.21.15.94:0/504010605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19cec6b0 msgr2=0x55cb1a0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 -- 172.21.15.94:0/504010605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb19fd8c30 con 0x55cb19ce44d0 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c659c640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8affff640 1 -- 172.21.15.94:0/4090180044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8c00df280 con 0x7ff8c8105090 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.298+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009230 tx=0x7ff8b8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcda1a8c32ca136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8c759e640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 0x7ff8b00b2000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a3740 tx=0x7ff8c00ca040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8ad7fa640 1 -- 172.21.15.94:0/504010605 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8035070 con 0x55cb19ce44d0 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8ad7fa640 1 -- 172.21.15.94:0/504010605 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b80317a0 con 0x55cb19ce44d0 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8ad7fa640 1 -- 172.21.15.94:0/504010605 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8003e30 con 0x55cb19ce44d0 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/504010605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 msgr2=0x55cb1a0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.299+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009230 tx=0x7ff8b8032f90 comp rx=0 tx=0).stop 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/504010605 shutdown_connections 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ce44d0 0x55cb1a0afd90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/504010605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/504010605 >> 172.21.15.94:0/504010605 conn(0x55cb19d014e0 msgr2=0x55cb1a0ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/504010605 shutdown_connections 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/504010605 wait complete. 2024-04-25T13:47:50.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- start start 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 Processor -- start 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a0b14d0 con 0x55cb19cec6b0 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb19f9adc0 con 0x55cb19ce44d0 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a0b1650 con 0x55cb19d0d390 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.300+0000 7ff8d3f6b740 10 client.0 _ll_register_callbacks cb 0x55cb19fb6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34352/0 (socket says 172.21.15.94:34352) 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49234/0 (socket says 172.21.15.94:49234) 2024-04-25T13:47:50.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 -- 172.21.15.94:0/2200495152 learned_addr learned my addr 172.21.15.94:0/2200495152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 -- 172.21.15.94:0/2200495152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 msgr2=0x55cb1a0c09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0c09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 -- 172.21.15.94:0/2200495152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 msgr2=0x55cb1a0b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 -- 172.21.15.94:0/2200495152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb19d253f0 con 0x55cb19ce44d0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8c659c640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc011e10 tx=0x7ff8bc009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f494b872cd63f742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc004050 con 0x55cb19ce44d0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b80317a0 con 0x55cb19ce44d0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8bc0041f0 con 0x55cb19ce44d0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc0043f0 con 0x55cb19ce44d0 2024-04-25T13:47:50.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8d3f6b740 10 client.27182 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.301+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb19def7d0 con 0x55cb19ce44d0 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b80317a0 con 0x55cb19ce44d0 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8bc02b450 con 0x55cb19ce44d0 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 10 client.27182 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 10 client.27182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 20 client.27182 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19ccbf60 0x55cb1a0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 4 client.27182 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb1a021990 con 0x55cb19ccbf60 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.302+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19ccbf60 0x55cb1a0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8aaff5640 10 client.27182 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19ccbf60 0x55cb1a0c71a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a70323e06246b740 server_cookie=4999069b949ecd8a in_seq=0 out_seq=0 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8aaff5640 1 -- 172.21.15.94:0/2200495152 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb1a021990 con 0x55cb19ccbf60 2024-04-25T13:47:50.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8aaff5640 10 client.27182 handle_command_reply: tid=0 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8d3f6b740 1 client.27182 shutdown 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 msgr2=0x55cb1a0b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.303+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc011e10 tx=0x7ff8bc009eb0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 shutdown_connections 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19ccbf60 0x55cb1a0c71a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19d0d390 0x55cb1a0c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8aaff5640 0 client.27182 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/2200495152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0b9530 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 >> 172.21.15.94:0/2200495152 conn(0x55cb19d014e0 msgr2=0x55cb1a0adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 shutdown_connections 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/2200495152 wait complete. 2024-04-25T13:47:50.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.304+0000 7ff8d3f6b740 20 client.27182 trim_cache size 0 max 16384 2024-04-25T13:47:50.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 Processor -- start 2024-04-25T13:47:50.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 -- start start 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19cec6b0 0x55cb1a0a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ab540 con 0x55cb19ccbf60 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ad080 con 0x55cb19ce44d0 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81abc10 con 0x55cb19cec6b0 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49240/0 (socket says 172.21.15.94:49240) 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 -- 172.21.15.94:0/1679196252 learned_addr learned my addr 172.21.15.94:0/1679196252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19cec6b0 0x55cb1a0a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 -- 172.21.15.94:0/1679196252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19cec6b0 msgr2=0x55cb1a0a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19cec6b0 0x55cb1a0a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 -- 172.21.15.94:0/1679196252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 msgr2=0x55cb1a0c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.320+0000 7ff8c659c640 1 -- 172.21.15.94:0/1679196252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1a0b0840 con 0x55cb19ccbf60 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.321+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc001eb0 tx=0x7ff8bc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa75c7f6ab7b621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.321+0000 7ff8acff9640 1 -- 172.21.15.94:0/1679196252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc009250 con 0x55cb19ccbf60 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.321+0000 7ff8acff9640 1 -- 172.21.15.94:0/1679196252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc032030 con 0x55cb19ccbf60 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.321+0000 7ff8acff9640 1 -- 172.21.15.94:0/1679196252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc004590 con 0x55cb19ccbf60 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1679196252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 msgr2=0x55cb1a0b9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc001eb0 tx=0x7ff8bc011440 comp rx=0 tx=0).stop 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1679196252 shutdown_connections 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19cec6b0 0x55cb1a0a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ce44d0 0x55cb1a0c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1679196252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19ccbf60 0x55cb1a0b9810 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1679196252 >> 172.21.15.94:0/1679196252 conn(0x55cb19d014e0 msgr2=0x55cb1a0b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1679196252 shutdown_connections 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1679196252 wait complete. 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- start start 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 Processor -- start 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ccbf60 0x55cb1a0c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19ce44d0 0x55cb1a0bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ac9b0 con 0x55cb19cec6b0 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81ac2e0 con 0x55cb19ccbf60 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81a9520 con 0x55cb19ce44d0 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8d3f6b740 10 client.0 _ll_register_callbacks cb 0x7ff8bc011c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ccbf60 0x55cb1a0c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49248/0 (socket says 172.21.15.94:49248) 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8c659c640 1 -- 172.21.15.94:0/1475476692 learned_addr learned my addr 172.21.15.94:0/1475476692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.322+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19ce44d0 0x55cb1a0bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 -- 172.21.15.94:0/1475476692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19ce44d0 msgr2=0x55cb1a0bd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19ce44d0 0x55cb1a0bd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 -- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ccbf60 msgr2=0x55cb1a0c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ccbf60 0x55cb1a0c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 -- 172.21.15.94:0/1475476692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1a12d500 con 0x55cb19cec6b0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8c659c640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba07a71bb0bc6e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc004590 con 0x55cb19cec6b0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8bc032030 con 0x55cb19cec6b0 2024-04-25T13:47:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8bc02ba10 con 0x55cb19cec6b0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc02bc10 con 0x55cb19cec6b0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8d3f6b740 10 client.17718 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb1a12e630 con 0x55cb19cec6b0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.323+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc032030 con 0x55cb19cec6b0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8bc038400 con 0x55cb19cec6b0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 10 client.17718 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 20 client.17718 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19d0d390 0x55cb1a130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 4 client.17718 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb1a01b800 con 0x55cb19d0d390 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19d0d390 0x55cb1a130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8c6d9d640 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19d0d390 0x55cb1a130ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80d15ebde002aaf4 server_cookie=b55b4e9ca5a195f4 in_seq=0 out_seq=0 2024-04-25T13:47:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8aaff5640 10 client.17718 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8aaff5640 1 -- 172.21.15.94:0/1475476692 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb1a01b800 con 0x55cb19d0d390 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8aaff5640 10 client.17718 handle_command_reply: tid=0 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.324+0000 7ff8d3f6b740 1 client.17718 shutdown 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 msgr2=0x55cb1a0bd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc011840 comp rx=0 tx=0).stop 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 shutdown_connections 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cb19d0d390 0x55cb1a130ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb19cec6b0 0x55cb1a0bd910 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb19ce44d0 0x55cb1a0bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8aaff5640 0 client.17718 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 --2- 172.21.15.94:0/1475476692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb19ccbf60 0x55cb1a0c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 >> 172.21.15.94:0/1475476692 conn(0x55cb19d014e0 msgr2=0x55cb1a0a9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 shutdown_connections 2024-04-25T13:47:50.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 1 -- 172.21.15.94:0/1475476692 wait complete. 2024-04-25T13:47:50.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8d3f6b740 20 client.17718 trim_cache size 0 max 16384 2024-04-25T13:47:50.331 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 msgr2=0x7ff8b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 0x7ff8b00b2000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a3740 tx=0x7ff8c00ca040 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 msgr2=0x7ff8c81a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eb10 tx=0x7ff8c000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 shutdown_connections 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8b00afaf0 0x7ff8b00b2000 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c8105090 0x7ff8c81a1710 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102240 0x7ff8c819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 --2- 172.21.15.94:0/4090180044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8c8101740 0x7ff8c819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.325+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 >> 172.21.15.94:0/4090180044 conn(0x7ff8c80fd340 msgr2=0x7ff8c8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.326+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 shutdown_connections 2024-04-25T13:47:50.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.326+0000 7ff8cd0b6640 1 -- 172.21.15.94:0/4090180044 wait complete. 2024-04-25T13:47:50.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 -- 172.21.15.94:0/4070560889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1033d0 msgr2=0x7ff81c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 --2- 172.21.15.94:0/4070560889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1033d0 0x7ff81c1037e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009900 tx=0x7ff80c02a410 comp rx=0 tx=0).stop 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 -- 172.21.15.94:0/4070560889 shutdown_connections 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 --2- 172.21.15.94:0/4070560889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1080d0 0x7ff81c10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 --2- 172.21.15.94:0/4070560889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1033d0 0x7ff81c1037e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 --2- 172.21.15.94:0/4070560889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 -- 172.21.15.94:0/4070560889 >> 172.21.15.94:0/4070560889 conn(0x7ff81c0fd380 msgr2=0x7ff81c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 -- 172.21.15.94:0/4070560889 shutdown_connections 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 -- 172.21.15.94:0/4070560889 wait complete. 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.454+0000 7ff823870640 1 Processor -- start 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 -- start start 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1033d0 0x7ff81c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10f630 con 0x7ff81c1080d0 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10f4b0 con 0x7ff81c102190 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff823870640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c10f7b0 con 0x7ff81c1033d0 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff82160c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff82160c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34372/0 (socket says 172.21.15.94:34372) 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49260/0 (socket says 172.21.15.94:49260) 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff82160c640 1 -- 172.21.15.94:0/2104710114 learned_addr learned my addr 172.21.15.94:0/2104710114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff820e0b640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1033d0 0x7ff81c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 -- 172.21.15.94:0/2104710114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1033d0 msgr2=0x7ff81c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1033d0 0x7ff81c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 -- 172.21.15.94:0/2104710114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 msgr2=0x7ff81c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 -- 172.21.15.94:0/2104710114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1a1d70 con 0x7ff81c1080d0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff82160c640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.455+0000 7ff821e0d640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff81400d680 tx=0x7ff81400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.456+0000 7ff8127fc640 1 -- 172.21.15.94:0/2104710114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814017070 con 0x7ff81c1080d0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.456+0000 7ff8127fc640 1 -- 172.21.15.94:0/2104710114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814009820 con 0x7ff81c1080d0 2024-04-25T13:47:50.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.456+0000 7ff8127fc640 1 -- 172.21.15.94:0/2104710114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814004ce0 con 0x7ff81c1080d0 2024-04-25T13:47:50.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.456+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1a2060 con 0x7ff81c1080d0 2024-04-25T13:47:50.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.456+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a25b0 con 0x7ff81c1080d0 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.458+0000 7ff82a7fd740 1 Processor -- start 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff8127fc640 1 -- 172.21.15.94:0/2104710114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff8140099d0 con 0x7ff81c1080d0 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 -- start start 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1a9390 0x55ab0a54b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a54ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a4afd80 con 0x55ab0a1804d0 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a469a20 con 0x55ab0a1886b0 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a4abd40 con 0x55ab0a1a9390 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49268/0 (socket says 172.21.15.94:49268) 2024-04-25T13:47:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 -- 172.21.15.94:0/3415183477 learned_addr learned my addr 172.21.15.94:0/3415183477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff820e0b640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a54ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff820e0b640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1a9390 0x55ab0a54b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 -- 172.21.15.94:0/3415183477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1a9390 msgr2=0x55ab0a54b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1a9390 0x55ab0a54b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 -- 172.21.15.94:0/3415183477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 msgr2=0x55ab0a54ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a54ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 -- 172.21.15.94:0/3415183477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0a474c30 con 0x55ab0a1804d0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff820e0b640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a54ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff8127fc640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 0x7ff8080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff820e0b640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 0x7ff8080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff82160c640 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff8180118f0 tx=0x7ff818011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c36bc7aca80951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.459+0000 7ff8127fc640 1 -- 172.21.15.94:0/2104710114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8140de460 con 0x7ff81c1080d0 2024-04-25T13:47:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.460+0000 7ff8077fe640 1 -- 172.21.15.94:0/3415183477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81801e070 con 0x55ab0a1804d0 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.460+0000 7ff8077fe640 1 -- 172.21.15.94:0/3415183477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81800fe50 con 0x55ab0a1804d0 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.460+0000 7ff8077fe640 1 -- 172.21.15.94:0/3415183477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818019c90 con 0x55ab0a1804d0 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3415183477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 msgr2=0x55ab0a54bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff8180118f0 tx=0x7ff818011db0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff820e0b640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 0x7ff8080b2000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009230 tx=0x7ff80c03f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3415183477 shutdown_connections 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a54ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a54bd90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3415183477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1a9390 0x55ab0a54b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3415183477 >> 172.21.15.94:0/3415183477 conn(0x55ab0a19d4e0 msgr2=0x55ab0a547020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3415183477 shutdown_connections 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3415183477 wait complete. 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 -- start start 2024-04-25T13:47:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 Processor -- start 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 0x55ab0a5554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a555a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.461+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a54d530 con 0x55ab0a1a9390 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a436dc0 con 0x55ab0a1886b0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab0a54d6b0 con 0x55ab0a1804d0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 10 client.0 _ll_register_callbacks cb 0x55ab0a452600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49274/0 (socket says 172.21.15.94:49274) 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 -- 172.21.15.94:0/3947580123 learned_addr learned my addr 172.21.15.94:0/3947580123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff820e0b640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a555a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82160c640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 0x55ab0a5554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 -- 172.21.15.94:0/3947580123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 msgr2=0x55ab0a5554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 0x55ab0a5554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 -- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 msgr2=0x55ab0a555a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a555a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 -- 172.21.15.94:0/3947580123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0a1c13f0 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82160c640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 0x55ab0a5554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff821e0d640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a29d0 tx=0x7ff814004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763b8a1224dee0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814020070 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81800fe50 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8140c90b0 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8140aac50 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 10 client.17730 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.462+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab0a28b7d0 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81800fe50 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8140a5bb0 con 0x55ab0a1a9390 2024-04-25T13:47:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 10 client.17730 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 20 client.17730 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a167f60 0x55ab0a5630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 4 client.17730 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0a4bd990 con 0x55ab0a167f60 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82160c640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a167f60 0x55ab0a5630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff804ff9640 10 client.17730 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.463+0000 7ff82160c640 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a167f60 0x55ab0a5630f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0e08d0c0944c7a server_cookie=e3477feec925c1b5 in_seq=0 out_seq=0 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff804ff9640 1 -- 172.21.15.94:0/3947580123 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab0a4bd990 con 0x55ab0a167f60 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff804ff9640 10 client.17730 handle_command_reply: tid=0 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 client.17730 shutdown 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 msgr2=0x55ab0a55c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a29d0 tx=0x7ff814004f70 comp rx=0 tx=0).stop 2024-04-25T13:47:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 shutdown_connections 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a167f60 0x55ab0a5630f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a55c970 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1886b0 0x55ab0a555a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/3947580123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1804d0 0x55ab0a5554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 >> 172.21.15.94:0/3947580123 conn(0x55ab0a19d4e0 msgr2=0x55ab0a549f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.464+0000 7ff804ff9640 0 client.17730 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.465+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 shutdown_connections 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.465+0000 7ff82a7fd740 1 -- 172.21.15.94:0/3947580123 wait complete. 2024-04-25T13:47:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.465+0000 7ff82a7fd740 20 client.17730 trim_cache size 0 max 16384 2024-04-25T13:47:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 Processor -- start 2024-04-25T13:47:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 -- start start 2024-04-25T13:47:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a546f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 0x55ab0a553cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ab600 con 0x55ab0a1804d0 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ad140 con 0x55ab0a1a9390 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.478+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1abcd0 con 0x55ab0a1886b0 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49282/0 (socket says 172.21.15.94:49282) 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 -- 172.21.15.94:0/1423481090 learned_addr learned my addr 172.21.15.94:0/1423481090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff821e0d640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 0x55ab0a553cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff820e0b640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a546f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 -- 172.21.15.94:0/1423481090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 msgr2=0x55ab0a546f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a546f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 -- 172.21.15.94:0/1423481090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 msgr2=0x55ab0a553cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 0x55ab0a553cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 -- 172.21.15.94:0/1423481090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0a547bb0 con 0x55ab0a1804d0 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff820e0b640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a546f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff821e0d640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 0x55ab0a553cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff82160c640 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff818005c40 tx=0x7ff818009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06f5eb11da9480b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff806ffd640 1 -- 172.21.15.94:0/1423481090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81801e040 con 0x55ab0a1804d0 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff806ffd640 1 -- 172.21.15.94:0/1423481090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81801f040 con 0x55ab0a1804d0 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.479+0000 7ff806ffd640 1 -- 172.21.15.94:0/1423481090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818019910 con 0x55ab0a1804d0 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 -- 172.21.15.94:0/1423481090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 msgr2=0x55ab0a546a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff818005c40 tx=0x7ff818009f60 comp rx=0 tx=0).stop 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 -- 172.21.15.94:0/1423481090 shutdown_connections 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1a9390 0x55ab0a553cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a546f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/1423481090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1804d0 0x55ab0a546a50 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 -- 172.21.15.94:0/1423481090 >> 172.21.15.94:0/1423481090 conn(0x55ab0a19d4e0 msgr2=0x55ab0a550250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 -- 172.21.15.94:0/1423481090 shutdown_connections 2024-04-25T13:47:50.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.480+0000 7ff82a7fd740 1 -- 172.21.15.94:0/1423481090 wait complete. 2024-04-25T13:47:50.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 -- start start 2024-04-25T13:47:50.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 Processor -- start 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1804d0 0x55ab0a558dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a559310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1aca70 con 0x55ab0a1a9390 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1ac3a0 con 0x55ab0a1804d0 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1a95e0 con 0x55ab0a1886b0 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82a7fd740 10 client.0 _ll_register_callbacks cb 0x7ff8140a6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49294/0 (socket says 172.21.15.94:49294) 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 -- 172.21.15.94:0/149277138 learned_addr learned my addr 172.21.15.94:0/149277138 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff82160c640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1804d0 0x55ab0a558dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff820e0b640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a559310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 -- 172.21.15.94:0/149277138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 msgr2=0x55ab0a559310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a559310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 -- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1804d0 msgr2=0x55ab0a558dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.481+0000 7ff821e0d640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1804d0 0x55ab0a558dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff821e0d640 1 -- 172.21.15.94:0/149277138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0a5598d0 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff821e0d640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a3bc0 tx=0x7ff81400d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6baeec8fcc6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814020040 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff8140a5bb0 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81801f040 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8140a6460 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 10 client.17736 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab0a5ca9a0 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff81801f040 con 0x55ab0a1a9390 2024-04-25T13:47:50.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8140a2390 con 0x55ab0a1a9390 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 10 client.17736 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 20 client.17736 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a13d3a0 0x55ab0a5cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 4 client.17736 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.482+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab0a4b7800 con 0x55ab0a13d3a0 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82160c640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a13d3a0 0x55ab0a5cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff804ff9640 10 client.17736 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82160c640 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a13d3a0 0x55ab0a5cd3e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3845887ad8f12193 server_cookie=d90a917f2587881 in_seq=0 out_seq=0 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff804ff9640 1 -- 172.21.15.94:0/149277138 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab0a4b7800 con 0x55ab0a13d3a0 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff804ff9640 10 client.17736 handle_command_reply: tid=0 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 client.17736 shutdown 2024-04-25T13:47:50.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 msgr2=0x55ab0a5c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a3bc0 tx=0x7ff81400d5d0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 shutdown_connections 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ab0a13d3a0 0x55ab0a5cd3e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab0a1a9390 0x55ab0a5c9450 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0a1886b0 0x55ab0a559310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff804ff9640 0 client.17736 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.483+0000 7ff82a7fd740 1 --2- 172.21.15.94:0/149277138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0a1804d0 0x55ab0a558dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 >> 172.21.15.94:0/149277138 conn(0x55ab0a19d4e0 msgr2=0x55ab0a553cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 shutdown_connections 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff82a7fd740 1 -- 172.21.15.94:0/149277138 wait complete. 2024-04-25T13:47:50.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff82a7fd740 20 client.17736 trim_cache size 0 max 16384 2024-04-25T13:47:50.490 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:47:50.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 msgr2=0x7ff8080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 0x7ff8080b2000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009230 tx=0x7ff80c03f040 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 msgr2=0x7ff81c1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff81400d680 tx=0x7ff81400db40 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 shutdown_connections 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff8080afaf0 0x7ff8080b2000 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c1080d0 0x7ff81c1a1700 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff81c1033d0 0x7ff81c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 --2- 172.21.15.94:0/2104710114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff81c102190 0x7ff81c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.484+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 >> 172.21.15.94:0/2104710114 conn(0x7ff81c0fd380 msgr2=0x7ff81c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.485+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 shutdown_connections 2024-04-25T13:47:50.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:50.485+0000 7ff823870640 1 -- 172.21.15.94:0/2104710114 wait complete. 2024-04-25T13:47:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:50 smithi137 ceph-mon[32568]: pgmap v528: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-04-25T13:47:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:50 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1695954086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:50 smithi094 ceph-mon[29195]: pgmap v528: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-04-25T13:47:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:50 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1695954086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:50 smithi138 ceph-mon[32977]: pgmap v528: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 234 op/s 2024-04-25T13:47:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:50 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1695954086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:47:50.799 INFO:tasks.workunit.client.0.smithi094.stdout: 1 678032 13.11 MB/sec execute 153 sec latency 21.298 ms 2024-04-25T13:47:51.495 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:47:51.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:51.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:47:51.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 -- 172.21.15.94:0/1456952942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 msgr2=0x7fb2f4105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 --2- 172.21.15.94:0/1456952942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f4105140 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009900 tx=0x7fb2e402a410 comp rx=0 tx=0).stop 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 -- 172.21.15.94:0/1456952942 shutdown_connections 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 --2- 172.21.15.94:0/1456952942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4109a30 0x7fb2f410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 --2- 172.21.15.94:0/1456952942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f4105140 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 --2- 172.21.15.94:0/1456952942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4103af0 0x7fb2f41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 -- 172.21.15.94:0/1456952942 >> 172.21.15.94:0/1456952942 conn(0x7fb2f40fece0 msgr2=0x7fb2f4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 -- 172.21.15.94:0/1456952942 shutdown_connections 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.614+0000 7fb2fc586640 1 -- 172.21.15.94:0/1456952942 wait complete. 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 Processor -- start 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 -- start start 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 0x7fb2f419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4109a30 0x7fb2f41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f640 con 0x7fb2f4104d30 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f4c0 con 0x7fb2f4103af0 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2fc586640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f410f7c0 con 0x7fb2f4109a30 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.615+0000 7fb2f9b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49326/0 (socket says 172.21.15.94:49326) 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2fa322640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 0x7fb2f419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 -- 172.21.15.94:0/2877719086 learned_addr learned my addr 172.21.15.94:0/2877719086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2fab23640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4109a30 0x7fb2f41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 -- 172.21.15.94:0/2877719086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4109a30 msgr2=0x7fb2f41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4109a30 0x7fb2f41a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 -- 172.21.15.94:0/2877719086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 msgr2=0x7fb2f419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 0x7fb2f419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 -- 172.21.15.94:0/2877719086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f41a1d60 con 0x7fb2f4104d30 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 0x7fb2f419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4031440 tx=0x7fb2e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2eb7fe640 1 -- 172.21.15.94:0/2877719086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e403f070 con 0x7fb2f4104d30 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41a1ff0 con 0x7fb2f4104d30 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2eb7fe640 1 -- 172.21.15.94:0/2877719086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e4031c50 con 0x7fb2f4104d30 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2eb7fe640 1 -- 172.21.15.94:0/2877719086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4035030 con 0x7fb2f4104d30 2024-04-25T13:47:51.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.616+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a2400 con 0x7fb2f4104d30 2024-04-25T13:47:51.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 Processor -- start 2024-04-25T13:47:51.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 -- start start 2024-04-25T13:47:51.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb2eb7fe640 1 -- 172.21.15.94:0/2877719086 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb2e4042070 con 0x7fb2f4104d30 2024-04-25T13:47:51.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d5765c550 0x555d57a2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 0x555d57a31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d5798bb00 con 0x555d57685410 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d57927920 con 0x555d57664730 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb30343f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d57987ac0 con 0x555d5765c550 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.619+0000 7fb2fab23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49342/0 (socket says 172.21.15.94:49342) 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2f9b21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 0x555d57a31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 -- 172.21.15.94:0/238950179 learned_addr learned my addr 172.21.15.94:0/238950179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fa322640 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d5765c550 0x555d57a2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 -- 172.21.15.94:0/238950179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d5765c550 msgr2=0x555d57a2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d5765c550 0x555d57a2ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 -- 172.21.15.94:0/238950179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 msgr2=0x555d57a31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 0x555d57a31d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 -- 172.21.15.94:0/238950179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d577657e0 con 0x555d57685410 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 0x555d57a31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2eb7fe640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 0x7fb2e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fab23640 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009fd0 tx=0x7fb2ec0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70af9cf1f014efc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 0x7fb2e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2e8bfe640 1 -- 172.21.15.94:0/238950179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004d20 con 0x555d57685410 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2eb7fe640 1 -- 172.21.15.94:0/2877719086 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb2e40ff4c0 con 0x7fb2f4104d30 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2e8bfe640 1 -- 172.21.15.94:0/238950179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec004050 con 0x555d57685410 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.620+0000 7fb2e8bfe640 1 -- 172.21.15.94:0/238950179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec005260 con 0x555d57685410 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 -- 172.21.15.94:0/238950179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 msgr2=0x555d57a2a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 0x7fb2e00b2000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555d57a2bc90 tx=0x7fb2f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009fd0 tx=0x7fb2ec0118d0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 -- 172.21.15.94:0/238950179 shutdown_connections 2024-04-25T13:47:51.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d57664730 0x555d57a31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d5765c550 0x555d57a2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 --2- 172.21.15.94:0/238950179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57685410 0x555d57a2a8b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 -- 172.21.15.94:0/238950179 >> 172.21.15.94:0/238950179 conn(0x555d57679560 msgr2=0x555d57a21e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 -- 172.21.15.94:0/238950179 shutdown_connections 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.621+0000 7fb30343f740 1 -- 172.21.15.94:0/238950179 wait complete. 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 -- start start 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 Processor -- start 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57685410 0x555d57a3bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d57a32780 con 0x555d57664730 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d579255d0 con 0x555d5765c550 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d57a32900 con 0x555d57685410 2024-04-25T13:47:51.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb30343f740 10 client.0 _ll_register_callbacks cb 0x555d57913080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2f9b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2f9b21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49352/0 (socket says 172.21.15.94:49352) 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2f9b21640 1 -- 172.21.15.94:0/455709123 learned_addr learned my addr 172.21.15.94:0/455709123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fab23640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57685410 0x555d57a3bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 -- 172.21.15.94:0/455709123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57685410 msgr2=0x555d57a3bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57685410 0x555d57a3bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 -- 172.21.15.94:0/455709123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 msgr2=0x555d57a34d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.622+0000 7fb2fa322640 1 -- 172.21.15.94:0/455709123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d577c5f70 con 0x555d5765c550 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:51.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2fa322640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a347d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x555d57a22bc0 tx=0x7fb2f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb28c0f25487a0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0003e40 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2ec004050 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb2f00104f0 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2f00106f0 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb30343f740 10 client.27218 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d5765d170 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.623+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ec004050 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2f00279d0 con 0x555d5765c550 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 10 client.27218 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 10 client.27218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 20 client.27218 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d576441f0 0x555d57a42110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 4 client.27218 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d57999710 con 0x555d576441f0 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d576441f0 0x555d57a42110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d576441f0 0x555d57a42110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c50cc1607f7c13 server_cookie=cbff41b3966b3a20 in_seq=0 out_seq=0 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.624+0000 7fb2ddffb640 10 client.27218 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb2ddffb640 1 -- 172.21.15.94:0/455709123 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d57999710 con 0x555d576441f0 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb2ddffb640 10 client.27218 handle_command_reply: tid=0 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 client.27218 shutdown 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 msgr2=0x555d57a347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a347d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x555d57a22bc0 tx=0x7fb2f00046c0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 shutdown_connections 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d576441f0 0x555d57a42110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57685410 0x555d57a3bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d57664730 0x555d57a34d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb2ddffb640 0 client.27218 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 --2- 172.21.15.94:0/455709123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a347d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 >> 172.21.15.94:0/455709123 conn(0x555d57679560 msgr2=0x555d57a20ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 shutdown_connections 2024-04-25T13:47:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 1 -- 172.21.15.94:0/455709123 wait complete. 2024-04-25T13:47:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.625+0000 7fb30343f740 20 client.27218 trim_cache size 0 max 16384 2024-04-25T13:47:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 Processor -- start 2024-04-25T13:47:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 -- start start 2024-04-25T13:47:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57a478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41a93b0 con 0x555d576441f0 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41aaef0 con 0x555d5765c550 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb30343f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41a9a80 con 0x555d57664730 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.644+0000 7fb2fa322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49368/0 (socket says 172.21.15.94:49368) 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 -- 172.21.15.94:0/2004900226 learned_addr learned my addr 172.21.15.94:0/2004900226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fab23640 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57a478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 -- 172.21.15.94:0/2004900226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 msgr2=0x555d57a478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57a478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 -- 172.21.15.94:0/2004900226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 msgr2=0x555d57a3bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 -- 172.21.15.94:0/2004900226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d57a2dcd0 con 0x555d576441f0 2024-04-25T13:47:51.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2fa322640 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0028ff0 tx=0x7fb2f000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457773ce34cb3166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2dffff640 1 -- 172.21.15.94:0/2004900226 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0034070 con 0x555d576441f0 2024-04-25T13:47:51.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2dffff640 1 -- 172.21.15.94:0/2004900226 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f0003e40 con 0x555d576441f0 2024-04-25T13:47:51.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.645+0000 7fb2dffff640 1 -- 172.21.15.94:0/2004900226 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0027a30 con 0x555d576441f0 2024-04-25T13:47:51.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- 172.21.15.94:0/2004900226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 msgr2=0x555d57a253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0028ff0 tx=0x7fb2f000b4b0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- 172.21.15.94:0/2004900226 shutdown_connections 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57a478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 --2- 172.21.15.94:0/2004900226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a253c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- 172.21.15.94:0/2004900226 >> 172.21.15.94:0/2004900226 conn(0x555d57679560 msgr2=0x555d57a24cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- 172.21.15.94:0/2004900226 shutdown_connections 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- 172.21.15.94:0/2004900226 wait complete. 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 -- start start 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.646+0000 7fb30343f740 1 Processor -- start 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57aa8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41aa820 con 0x555d576441f0 2024-04-25T13:47:51.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41aa150 con 0x555d5765c550 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f41a7390 con 0x555d57664730 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb30343f740 10 client.0 _ll_register_callbacks cb 0x7fb2f0004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49382/0 (socket says 172.21.15.94:49382) 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 -- 172.21.15.94:0/3606852796 learned_addr learned my addr 172.21.15.94:0/3606852796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fab23640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57aa8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 -- 172.21.15.94:0/3606852796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 msgr2=0x555d57aa8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57aa8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 -- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 msgr2=0x555d57a3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 -- 172.21.15.94:0/3606852796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d57aa8fe0 con 0x555d576441f0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2fa322640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0004770 tx=0x7fb2f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfc751150e5a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.647+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0049060 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb2f0003c40 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f0003e40 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2f0034040 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 10 client.17754 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d57aaa2a0 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f0003e40 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb2f001d750 con 0x555d576441f0 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 10 client.17754 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 20 client.17754 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d57685410 0x555d57aacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 4 client.17754 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:47:51.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d57993580 con 0x555d57685410 2024-04-25T13:47:51.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.648+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d57685410 0x555d57aacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.649+0000 7fb2ddffb640 10 client.17754 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.649+0000 7fb2f9b21640 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d57685410 0x555d57aacc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dcd5f87f54bbd4 server_cookie=52999a467a5a0734 in_seq=0 out_seq=0 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb2ddffb640 1 -- 172.21.15.94:0/3606852796 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d57993580 con 0x555d57685410 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb2ddffb640 10 client.17754 handle_command_reply: tid=0 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 client.17754 shutdown 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 msgr2=0x555d57a3b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0004770 tx=0x7fb2f0015000 comp rx=0 tx=0).stop 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 shutdown_connections 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555d57685410 0x555d57aacc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d57664730 0x555d57aa8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d5765c550 0x555d57a3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 --2- 172.21.15.94:0/3606852796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d576441f0 0x555d57a3b260 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb2ddffb640 0 client.17754 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 >> 172.21.15.94:0/3606852796 conn(0x555d57679560 msgr2=0x555d57a25440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 shutdown_connections 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 1 -- 172.21.15.94:0/3606852796 wait complete. 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.652+0000 7fb30343f740 20 client.17754 trim_cache size 0 max 16384 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:47:51.656 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "37196add-07ae-4be1-a192-1534dd323e93", 2024-04-25T13:47:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:47:51.657 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:51.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 msgr2=0x7fb2e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 0x7fb2e00b2000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x555d57a2bc90 tx=0x7fb2f000b390 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 msgr2=0x7fb2f419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4031440 tx=0x7fb2e4003c10 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 shutdown_connections 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2e00afaf0 0x7fb2e00b2000 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f4109a30 0x7fb2f41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2f4104d30 0x7fb2f419a760 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 --2- 172.21.15.94:0/2877719086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2f4103af0 0x7fb2f419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 >> 172.21.15.94:0/2877719086 conn(0x7fb2f40fece0 msgr2=0x7fb2f4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 shutdown_connections 2024-04-25T13:47:51.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.653+0000 7fb2fc586640 1 -- 172.21.15.94:0/2877719086 wait complete. 2024-04-25T13:47:51.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:47:51.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 -- 172.21.15.94:0/4252507459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc103ae0 msgr2=0x7fbffc1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 --2- 172.21.15.94:0/4252507459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc103ae0 0x7fbffc1047e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02a410 comp rx=0 tx=0).stop 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 -- 172.21.15.94:0/4252507459 shutdown_connections 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 --2- 172.21.15.94:0/4252507459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc109a20 0x7fbffc10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 --2- 172.21.15.94:0/4252507459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc104d20 0x7fbffc105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 --2- 172.21.15.94:0/4252507459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc103ae0 0x7fbffc1047e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 -- 172.21.15.94:0/4252507459 >> 172.21.15.94:0/4252507459 conn(0x7fbffc0fecd0 msgr2=0x7fbffc101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 -- 172.21.15.94:0/4252507459 shutdown_connections 2024-04-25T13:47:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.789+0000 7fc002138640 1 -- 172.21.15.94:0/4252507459 wait complete. 2024-04-25T13:47:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 Processor -- start 2024-04-25T13:47:51.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 -- start start 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc103ae0 0x7fbffc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 0x7fbffc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10f5e0 con 0x7fbffc109a20 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10f460 con 0x7fbffc103ae0 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fc002138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc10f760 con 0x7fbffc104d20 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 0x7fbffc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.790+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc103ae0 0x7fbffc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49408/0 (socket says 172.21.15.94:49408) 2024-04-25T13:47:51.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 -- 172.21.15.94:0/3017729497 learned_addr learned my addr 172.21.15.94:0/3017729497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 -- 172.21.15.94:0/3017729497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 msgr2=0x7fbffc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 0x7fbffc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 -- 172.21.15.94:0/3017729497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc103ae0 msgr2=0x7fbffc19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc103ae0 0x7fbffc19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 -- 172.21.15.94:0/3017729497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1a1d50 con 0x7fbffc109a20 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffaffd640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 0x7fbffc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbffbfff640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbff400eb10 tx=0x7fbff400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbff8ff9640 1 -- 172.21.15.94:0/3017729497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4004b20 con 0x7fbffc109a20 2024-04-25T13:47:51.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbff8ff9640 1 -- 172.21.15.94:0/3017729497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff4004cc0 con 0x7fbffc109a20 2024-04-25T13:47:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1a1fe0 con 0x7fbffc109a20 2024-04-25T13:47:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fbff8ff9640 1 -- 172.21.15.94:0/3017729497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4010670 con 0x7fbffc109a20 2024-04-25T13:47:51.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.791+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a24a0 con 0x7fbffc109a20 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.794+0000 7fc008fed740 1 Processor -- start 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.794+0000 7fbff8ff9640 1 -- 172.21.15.94:0/3017729497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbff4010810 con 0x7fbffc109a20 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 -- start start 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 0x55c5c0e1cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e1dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0d81d80 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0d3b8b0 con 0x55c5c0a7b390 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fc008fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0d7dd40 con 0x55c5c0a524d0 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e1dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 0x55c5c0e1cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34500/0 (socket says 172.21.15.94:34500) 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 0x55c5c0e1cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34376/0 (socket says 172.21.15.94:34376) 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 -- 172.21.15.94:0/3438047260 learned_addr learned my addr 172.21.15.94:0/3438047260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbff8ff9640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 0x7fbfe40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 -- 172.21.15.94:0/3438047260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 msgr2=0x55c5c0e1cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 0x55c5c0e1cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 -- 172.21.15.94:0/3438047260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 msgr2=0x55c5c0e1dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e1dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 -- 172.21.15.94:0/3438047260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c0d0ca40 con 0x55c5c0a7b390 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffbfff640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 0x7fbfe40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbff8ff9640 1 -- 172.21.15.94:0/3017729497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fbff40df1f0 con 0x7fbffc109a20 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.795+0000 7fbffaffd640 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbff0010d60 tx=0x7fbff0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49649cd9b359f540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.796+0000 7fbfe1ffb640 1 -- 172.21.15.94:0/3438047260 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0016070 con 0x55c5c0a7b390 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.796+0000 7fbfe1ffb640 1 -- 172.21.15.94:0/3438047260 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff000fe50 con 0x55c5c0a7b390 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.796+0000 7fbfe1ffb640 1 -- 172.21.15.94:0/3438047260 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0019040 con 0x55c5c0a7b390 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- 172.21.15.94:0/3438047260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 msgr2=0x55c5c0e1ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fbffbfff640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 0x7fbfe40b2000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbff40a7330 tx=0x7fbff406db50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbff0010d60 tx=0x7fbff0005f90 comp rx=0 tx=0).stop 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- 172.21.15.94:0/3438047260 shutdown_connections 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e1dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a524d0 0x55c5c0e1cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 --2- 172.21.15.94:0/3438047260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a7b390 0x55c5c0e1ca60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- 172.21.15.94:0/3438047260 >> 172.21.15.94:0/3438047260 conn(0x55c5c0a6f4e0 msgr2=0x55c5c0e18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- 172.21.15.94:0/3438047260 shutdown_connections 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- 172.21.15.94:0/3438047260 wait complete. 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 -- start start 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.797+0000 7fc008fed740 1 Processor -- start 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e26e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 0x55c5c0e2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0e1e720 con 0x55c5c0a7b390 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0d08d10 con 0x55c5c0a524d0 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0e1e8a0 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fc008fed740 10 client.0 _ll_register_callbacks cb 0x55c5c0d35210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 0x55c5c0e2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 0x55c5c0e2ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49436/0 (socket says 172.21.15.94:49436) 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffaffd640 1 -- 172.21.15.94:0/1442269204 learned_addr learned my addr 172.21.15.94:0/1442269204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e26e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffaffd640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e26910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 -- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 msgr2=0x55c5c0e26910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e26910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 -- 172.21.15.94:0/1442269204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 msgr2=0x55c5c0e2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 0x55c5c0e2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 -- 172.21.15.94:0/1442269204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c0b5d7d0 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e26e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfec02a930 tx=0x7fbfec031e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c3bd05d65073b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbfdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.798+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec035040 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff000fe50 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fbfec004040 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfec004240 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 10 client.27115 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5c0a933f0 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff000fe50 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbfec0484f0 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 10 client.27115 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 20 client.27115 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a39f60 0x55c5c0e34240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 4 client.27115 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c0d8f990 con 0x55c5c0a39f60 2024-04-25T13:47:51.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.799+0000 7fbffaffd640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a39f60 0x55c5c0e34240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 679922 13.09 MB/sec execute 154 sec latency 16.018 ms 2024-04-25T13:47:51.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.800+0000 7fbffaffd640 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a39f60 0x55c5c0e34240 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8156e36d5980d4b8 server_cookie=b019b84b10c06881 in_seq=0 out_seq=0 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.800+0000 7fbfdaffd640 10 client.27115 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.800+0000 7fbfdaffd640 1 -- 172.21.15.94:0/1442269204 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5c0d8f990 con 0x55c5c0a39f60 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.800+0000 7fbfdaffd640 10 client.27115 handle_command_reply: tid=0 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 client.27115 shutdown 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 msgr2=0x55c5c0e26e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e26e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfec02a930 tx=0x7fbfec031e80 comp rx=0 tx=0).stop 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 shutdown_connections 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a39f60 0x55c5c0e34240 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a7b390 0x55c5c0e2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e26e50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fbfdaffd640 0 client.27115 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 --2- 172.21.15.94:0/1442269204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e26910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 >> 172.21.15.94:0/1442269204 conn(0x55c5c0a6f4e0 msgr2=0x55c5c0e1b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 shutdown_connections 2024-04-25T13:47:51.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 1 -- 172.21.15.94:0/1442269204 wait complete. 2024-04-25T13:47:51.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.801+0000 7fc008fed740 20 client.27115 trim_cache size 0 max 16384 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 Processor -- start 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 -- start start 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a39f60 0x55c5c0e25570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ab520 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ad060 con 0x55c5c0a524d0 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fc008fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1abbf0 con 0x55c5c0a39f60 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34514/0 (socket says 172.21.15.94:34514) 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffb7fe640 1 -- 172.21.15.94:0/52271662 learned_addr learned my addr 172.21.15.94:0/52271662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e170d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49446/0 (socket says 172.21.15.94:49446) 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.820+0000 7fbffaffd640 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a39f60 0x55c5c0e25570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 -- 172.21.15.94:0/52271662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a39f60 msgr2=0x55c5c0e25570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a39f60 0x55c5c0e25570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 -- 172.21.15.94:0/52271662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 msgr2=0x55c5c0e170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 -- 172.21.15.94:0/52271662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c0a530f0 con 0x55c5c0a524d0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbfec048a60 tx=0x7fbfec0314c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415e39149c734713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbfe17fa640 1 -- 172.21.15.94:0/52271662 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec054030 con 0x55c5c0a524d0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbfe17fa640 1 -- 172.21.15.94:0/52271662 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec0317b0 con 0x55c5c0a524d0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.821+0000 7fbfe17fa640 1 -- 172.21.15.94:0/52271662 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec035040 con 0x55c5c0a524d0 2024-04-25T13:47:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- 172.21.15.94:0/52271662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 msgr2=0x55c5c0e216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbfec048a60 tx=0x7fbfec0314c0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- 172.21.15.94:0/52271662 shutdown_connections 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e216a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 --2- 172.21.15.94:0/52271662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a39f60 0x55c5c0e25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- 172.21.15.94:0/52271662 >> 172.21.15.94:0/52271662 conn(0x55c5c0a6f4e0 msgr2=0x55c5c0e169c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- 172.21.15.94:0/52271662 shutdown_connections 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- 172.21.15.94:0/52271662 wait complete. 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.822+0000 7fc008fed740 1 -- start start 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 Processor -- start 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e2c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e2d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ac990 con 0x55c5c0a39f60 2024-04-25T13:47:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ac2c0 con 0x55c5c0a524d0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1a9500 con 0x55c5c0a5a6b0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fc008fed740 10 client.0 _ll_register_callbacks cb 0x7fbfec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e2c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49454/0 (socket says 172.21.15.94:49454) 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffaffd640 1 -- 172.21.15.94:0/2543287834 learned_addr learned my addr 172.21.15.94:0/2543287834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffaffd640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e2d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 -- 172.21.15.94:0/2543287834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 msgr2=0x55c5c0e2d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e2d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 -- 172.21.15.94:0/2543287834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 msgr2=0x55c5c0e2be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffb7fe640 1 -- 172.21.15.94:0/2543287834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c0d424b0 con 0x55c5c0a524d0 2024-04-25T13:47:51.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.823+0000 7fbffaffd640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbffb7fe640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e2c3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbfec031ae0 tx=0x7fbfec048d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6910df33011951c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec05d060 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec0317b0 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fbfec03c440 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfec035040 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 10 client.27242 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5c0e9c600 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec0317b0 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbfec062430 con 0x55c5c0a524d0 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 10 client.27242 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 10 client.27242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 20 client.27242 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:51.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a7b390 0x55c5c0e9ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 4 client.27242 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.824+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5c0d89800 con 0x55c5c0a7b390 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbffaffd640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a7b390 0x55c5c0e9ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbfdaffd640 10 client.27242 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbffaffd640 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a7b390 0x55c5c0e9ed10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb8eff9264475e6 server_cookie=b0c121536be41f22 in_seq=0 out_seq=0 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbfdaffd640 1 -- 172.21.15.94:0/2543287834 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c5c0d89800 con 0x55c5c0a7b390 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbfdaffd640 10 client.27242 handle_command_reply: tid=0 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 client.27242 shutdown 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 msgr2=0x55c5c0e2c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e2c3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbfec031ae0 tx=0x7fbfec048d80 comp rx=0 tx=0).stop 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 shutdown_connections 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c5c0a7b390 0x55c5c0e9ed10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5c0a5a6b0 0x55c5c0e2d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fbfdaffd640 0 client.27242 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5c0a524d0 0x55c5c0e2c3d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 --2- 172.21.15.94:0/2543287834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c5c0a39f60 0x55c5c0e2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 >> 172.21.15.94:0/2543287834 conn(0x55c5c0a6f4e0 msgr2=0x55c5c0e1b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 shutdown_connections 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.825+0000 7fc008fed740 1 -- 172.21.15.94:0/2543287834 wait complete. 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc008fed740 20 client.27242 trim_cache size 0 max 16384 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:47:51.833 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: "37196add-07ae-4be1-a192-1534dd323e93": { 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "37196add-07ae-4be1-a192-1534dd323e93", 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:47:51.834 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 msgr2=0x7fbfe40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 0x7fbfe40b2000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbff40a7330 tx=0x7fbff406db50 comp rx=0 tx=0).stop 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 msgr2=0x7fbffc1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbff400eb10 tx=0x7fbff400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 shutdown_connections 2024-04-25T13:47:51.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfe40afaf0 0x7fbfe40b2000 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbffc109a20 0x7fbffc1a16e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbffc104d20 0x7fbffc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 --2- 172.21.15.94:0/3017729497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbffc103ae0 0x7fbffc19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 >> 172.21.15.94:0/3017729497 conn(0x7fbffc0fecd0 msgr2=0x7fbffc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 shutdown_connections 2024-04-25T13:47:51.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:47:51.826+0000 7fc002138640 1 -- 172.21.15.94:0/3017729497 wait complete. 2024-04-25T13:47:51.837 INFO:tasks.cephfs.filesystem:scrub status for tag:37196add-07ae-4be1-a192-1534dd323e93 - {'path': '/', 'tag': '37196add-07ae-4be1-a192-1534dd323e93', 'options': 'recursive,force'} 2024-04-25T13:47:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:52 smithi137 ceph-mon[32568]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 324 op/s 2024-04-25T13:47:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:52 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:47:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:52 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:47:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:52 smithi094 ceph-mon[29195]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 324 op/s 2024-04-25T13:47:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:52 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:47:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:52 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:47:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:52 smithi138 ceph-mon[32977]: pgmap v529: 97 pgs: 97 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 324 op/s 2024-04-25T13:47:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:52 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:47:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:52 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:47:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:47:52.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 682418 13.09 MB/sec execute 155 sec latency 12.644 ms 2024-04-25T13:47:53.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 685472 13.12 MB/sec execute 156 sec latency 12.825 ms 2024-04-25T13:47:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:54 smithi137 ceph-mon[32568]: pgmap v530: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-04-25T13:47:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:54 smithi094 ceph-mon[29195]: pgmap v530: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-04-25T13:47:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:54 smithi138 ceph-mon[32977]: pgmap v530: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-04-25T13:47:54.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 686906 13.08 MB/sec execute 157 sec latency 41.191 ms 2024-04-25T13:47:55.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 689394 13.09 MB/sec execute 158 sec latency 14.202 ms 2024-04-25T13:47:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:56 smithi137 ceph-mon[32568]: pgmap v531: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:47:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:47:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:56 smithi094 ceph-mon[29195]: pgmap v531: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:47:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:47:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:56 smithi138 ceph-mon[32977]: pgmap v531: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-04-25T13:47:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:47:56.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 692311 13.10 MB/sec execute 159 sec latency 13.133 ms 2024-04-25T13:47:57.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 694183 13.08 MB/sec execute 160 sec latency 17.914 ms 2024-04-25T13:47:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:47:58 smithi137 ceph-mon[32568]: pgmap v532: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:47:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:47:58 smithi094 ceph-mon[29195]: pgmap v532: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:47:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:47:58 smithi138 ceph-mon[32977]: pgmap v532: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-04-25T13:47:58.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 696575 13.08 MB/sec execute 161 sec latency 18.125 ms 2024-04-25T13:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 -- 172.21.15.94:0/4037326214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c108110 msgr2=0x7f095c10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 --2- 172.21.15.94:0/4037326214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c108110 0x7f095c10a530 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0954009ca0 tx=0x7f095402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 -- 172.21.15.94:0/4037326214 shutdown_connections 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 --2- 172.21.15.94:0/4037326214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c108110 0x7f095c10a530 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 --2- 172.21.15.94:0/4037326214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 0x7f095c103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 --2- 172.21.15.94:0/4037326214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c1021d0 0x7f095c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 -- 172.21.15.94:0/4037326214 >> 172.21.15.94:0/4037326214 conn(0x7f095c0fd3c0 msgr2=0x7f095c0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 -- 172.21.15.94:0/4037326214 shutdown_connections 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.713+0000 7f09645f8640 1 -- 172.21.15.94:0/4037326214 wait complete. 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 Processor -- start 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 -- start start 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 0x7f095c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c108110 0x7f095c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10f670 con 0x7f095c1021d0 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10f4f0 con 0x7f095c108110 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f09645f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10f7f0 con 0x7f095c103410 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f0962394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f0962394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56458/0 (socket says 172.21.15.94:56458) 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.714+0000 7f0962394640 1 -- 172.21.15.94:0/3583419649 learned_addr learned my addr 172.21.15.94:0/3583419649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0961b93640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 0x7f095c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962b95640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c108110 0x7f095c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 -- 172.21.15.94:0/3583419649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 msgr2=0x7f095c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 0x7f095c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 -- 172.21.15.94:0/3583419649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c108110 msgr2=0x7f095c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c108110 0x7f095c1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 -- 172.21.15.94:0/3583419649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a1d30 con 0x7f095c1021d0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f0962394640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f095800e9f0 tx=0x7f095800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f09537fe640 1 -- 172.21.15.94:0/3583419649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800ccc0 con 0x7f095c1021d0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f09537fe640 1 -- 172.21.15.94:0/3583419649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09580035f0 con 0x7f095c1021d0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f09537fe640 1 -- 172.21.15.94:0/3583419649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800ce60 con 0x7f095c1021d0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a1fc0 con 0x7f095c1021d0 2024-04-25T13:47:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.715+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a2480 con 0x7f095c1021d0 2024-04-25T13:47:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.718+0000 7f096b589740 1 Processor -- start 2024-04-25T13:47:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 -- start start 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e38aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e391df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56160e2f0460 con 0x56160dfea1b0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56160e2aa590 con 0x56160dfc12f0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f09537fe640 1 -- 172.21.15.94:0/3583419649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0958010690 con 0x7f095c1021d0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f096b589740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56160e2ec420 con 0x56160dfc94d0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56466/0 (socket says 172.21.15.94:56466) 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 -- 172.21.15.94:0/4217134609 learned_addr learned my addr 172.21.15.94:0/4217134609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0961b93640 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e391df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962394640 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e38aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 -- 172.21.15.94:0/4217134609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 msgr2=0x56160e391df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e391df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 -- 172.21.15.94:0/4217134609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 msgr2=0x56160e38aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e38aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.719+0000 7f0962b95640 1 -- 172.21.15.94:0/4217134609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56160e2b70d0 con 0x56160dfea1b0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f0962b95640 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0954035000 tx=0x7f095402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72ae451e53fc55f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f09537fe640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 0x7f09480b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f0950bfe640 1 -- 172.21.15.94:0/4217134609 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954031650 con 0x56160dfea1b0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f0950bfe640 1 -- 172.21.15.94:0/4217134609 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0954036030 con 0x56160dfea1b0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f0962394640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 0x7f09480b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f0950bfe640 1 -- 172.21.15.94:0/4217134609 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004690 con 0x56160dfea1b0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.720+0000 7f09537fe640 1 -- 172.21.15.94:0/3583419649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f09580defb0 con 0x7f095c1021d0 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- 172.21.15.94:0/4217134609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 msgr2=0x56160e38a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0954035000 tx=0x7f095402f410 comp rx=0 tx=0).stop 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f0962394640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 0x7f09480b1ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56160e38bd50 tx=0x7f0958090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- 172.21.15.94:0/4217134609 shutdown_connections 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e391df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e38aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 --2- 172.21.15.94:0/4217134609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e38a970 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- 172.21.15.94:0/4217134609 >> 172.21.15.94:0/4217134609 conn(0x56160dfde300 msgr2=0x56160e386300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- 172.21.15.94:0/4217134609 shutdown_connections 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- 172.21.15.94:0/4217134609 wait complete. 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 -- start start 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.721+0000 7f096b589740 1 Processor -- start 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 0x56160e394470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56160e392660 con 0x56160dfc94d0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56160e277b60 con 0x56160dfea1b0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56160e3927e0 con 0x56160dfc12f0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 10 client.0 _ll_register_callbacks cb 0x56160e29c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56482/0 (socket says 172.21.15.94:56482) 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 -- 172.21.15.94:0/3520135095 learned_addr learned my addr 172.21.15.94:0/3520135095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0962b95640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 0x56160e394470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0962b95640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 -- 172.21.15.94:0/3520135095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 msgr2=0x56160e394470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 0x56160e394470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 -- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 msgr2=0x56160e39b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 -- 172.21.15.94:0/3520135095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56160e002210 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0962b95640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 0x56160e394470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0961b93640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f094c012720 tx=0x7f094c012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac98468ed0225091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c004030 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f094c0041d0 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0954036030 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.722+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f094c0043d0 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 10 client.17784 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56160e0ce620 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0954036030 con 0x56160dfc94d0 2024-04-25T13:47:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f094c003b80 con 0x56160dfc94d0 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 10 client.17784 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 10 client.17784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 20 client.17784 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160dfa8d80 0x56160e3a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 4 client.17784 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56160e2fe070 con 0x56160dfa8d80 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.723+0000 7f0962b95640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160dfa8d80 0x56160e3a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.724+0000 7f0945ffb640 10 client.17784 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.724+0000 7f0962b95640 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160dfa8d80 0x56160e3a21a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a15b080a39971e3 server_cookie=413708f883810ad3 in_seq=0 out_seq=0 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.724+0000 7f0945ffb640 1 -- 172.21.15.94:0/3520135095 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56160e2fe070 con 0x56160dfa8d80 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.724+0000 7f0945ffb640 10 client.17784 handle_command_reply: tid=0 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.724+0000 7f096b589740 1 client.17784 shutdown 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 msgr2=0x56160e3949b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f094c012720 tx=0x7f094c012be0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 shutdown_connections 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160dfa8d80 0x56160e3a21a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc94d0 0x56160e3949b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 --2- 172.21.15.94:0/3520135095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc12f0 0x56160e394470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f0945ffb640 0 client.17784 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 >> 172.21.15.94:0/3520135095 conn(0x56160dfde300 msgr2=0x56160e385250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 shutdown_connections 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 1 -- 172.21.15.94:0/3520135095 wait complete. 2024-04-25T13:47:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.725+0000 7f096b589740 20 client.17784 trim_cache size 0 max 16384 2024-04-25T13:47:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 Processor -- start 2024-04-25T13:47:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 -- start start 2024-04-25T13:47:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e3862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e39b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ab560 con 0x56160dfea1b0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ad0a0 con 0x56160dfc12f0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f096b589740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1abc30 con 0x56160dfc94d0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f0961b93640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e39b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56498/0 (socket says 172.21.15.94:56498) 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f0962b95640 1 -- 172.21.15.94:0/2873937952 learned_addr learned my addr 172.21.15.94:0/2873937952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.743+0000 7f0962b95640 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e3862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 -- 172.21.15.94:0/2873937952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 msgr2=0x56160e39b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e39b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 -- 172.21.15.94:0/2873937952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 msgr2=0x56160e3862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e3862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 -- 172.21.15.94:0/2873937952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56160e39bd70 con 0x56160dfea1b0 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0962b95640 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x56160e386f00 tx=0x7f095402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1abb804dae59b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0947fff640 1 -- 172.21.15.94:0/2873937952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004160 con 0x56160dfea1b0 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0947fff640 1 -- 172.21.15.94:0/2873937952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095405e070 con 0x56160dfea1b0 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.744+0000 7f0947fff640 1 -- 172.21.15.94:0/2873937952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954003c60 con 0x56160dfea1b0 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.745+0000 7f096b589740 1 -- 172.21.15.94:0/2873937952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 msgr2=0x56160e384820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.745+0000 7f096b589740 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x56160e386f00 tx=0x7f095402f410 comp rx=0 tx=0).stop 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 -- 172.21.15.94:0/2873937952 shutdown_connections 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfea1b0 0x56160e384820 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e39b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 --2- 172.21.15.94:0/2873937952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfc12f0 0x56160e3862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 -- 172.21.15.94:0/2873937952 >> 172.21.15.94:0/2873937952 conn(0x56160dfde300 msgr2=0x56160e385ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 -- 172.21.15.94:0/2873937952 shutdown_connections 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 -- 172.21.15.94:0/2873937952 wait complete. 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 -- start start 2024-04-25T13:47:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.746+0000 7f096b589740 1 Processor -- start 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e399ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ac9d0 con 0x56160dfc12f0 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1ac300 con 0x56160dfea1b0 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095c1a9540 con 0x56160dfc94d0 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f096b589740 10 client.0 _ll_register_callbacks cb 0x7f094c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f0962b95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56506/0 (socket says 172.21.15.94:56506) 2024-04-25T13:47:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f0962b95640 1 -- 172.21.15.94:0/3507787233 learned_addr learned my addr 172.21.15.94:0/3507787233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f0961b93640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e399ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.747+0000 7f0962b95640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 -- 172.21.15.94:0/3507787233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 msgr2=0x56160e399ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e399ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 -- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 msgr2=0x56160e39aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 -- 172.21.15.94:0/3507787233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56160e39afd0 con 0x56160dfc12f0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0962b95640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0954006010 tx=0x7f095402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2460e6fa1a355ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095404a040 con 0x56160dfc12f0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0954031650 con 0x56160dfc12f0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.748+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095405e070 con 0x56160dfc12f0 2024-04-25T13:47:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09540039e0 con 0x56160dfc12f0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 10 client.17790 fetch_fsmap learned FSMap version 27 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56160e40aa40 con 0x56160dfc12f0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0954036040 con 0x56160dfc12f0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f095406e070 con 0x56160dfc12f0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 10 client.17790 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 10 client.17790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 20 client.17790 populate_metadata read hostname 'smithi094' 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160df7e1c0 0x56160e40cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 4 client.17790 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.749+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f095405e070 con 0x56160df7e1c0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.750+0000 7f0961b93640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160df7e1c0 0x56160e40cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.750+0000 7f0961b93640 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160df7e1c0 0x56160e40cf70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44cde65d409f627 server_cookie=4e0745cfc6d4b31d in_seq=0 out_seq=0 2024-04-25T13:47:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.750+0000 7f0945ffb640 10 client.17790 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:59.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 698646 13.05 MB/sec execute 162 sec latency 60.555 ms 2024-04-25T13:47:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.941+0000 7f0945ffb640 1 -- 172.21.15.94:0/3507787233 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f095405e070 con 0x56160df7e1c0 2024-04-25T13:47:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.941+0000 7f0945ffb640 10 client.17790 handle_command_reply: tid=0 2024-04-25T13:47:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.941+0000 7f096b589740 1 client.17790 shutdown 2024-04-25T13:47:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 msgr2=0x56160e3995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0954006010 tx=0x7f095402f3a0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 shutdown_connections 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56160df7e1c0 0x56160e40cf70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56160dfea1b0 0x56160e39aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f0945ffb640 0 client.17790 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56160dfc94d0 0x56160e399ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 --2- 172.21.15.94:0/3507787233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56160dfc12f0 0x56160e3995a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 >> 172.21.15.94:0/3507787233 conn(0x56160dfde300 msgr2=0x56160e384820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 shutdown_connections 2024-04-25T13:47:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 1 -- 172.21.15.94:0/3507787233 wait complete. 2024-04-25T13:47:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f096b589740 20 client.17790 trim_cache size 0 max 16384 2024-04-25T13:47:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:47:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:47:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:47:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.942+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 msgr2=0x7f09480b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 0x7f09480b1ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56160e38bd50 tx=0x7f0958090040 comp rx=0 tx=0).stop 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 msgr2=0x7f095c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f095800e9f0 tx=0x7f095800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 shutdown_connections 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f09480afae0 0x7f09480b1ff0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f095c108110 0x7f095c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095c103410 0x7f095c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 --2- 172.21.15.94:0/3583419649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f095c1021d0 0x7f095c19a1f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 >> 172.21.15.94:0/3583419649 conn(0x7f095c0fd3c0 msgr2=0x7f095c1040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 shutdown_connections 2024-04-25T13:47:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:47:59.943+0000 7f09645f8640 1 -- 172.21.15.94:0/3583419649 wait complete. 2024-04-25T13:48:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:00 smithi137 ceph-mon[32568]: pgmap v533: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-04-25T13:48:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:00 smithi094 ceph-mon[29195]: pgmap v533: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-04-25T13:48:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:00 smithi138 ceph-mon[32977]: pgmap v533: 97 pgs: 97 active+clean; 45 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-04-25T13:48:00.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 700315 13.04 MB/sec execute 163 sec latency 98.490 ms 2024-04-25T13:48:01.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 703339 13.07 MB/sec execute 164 sec latency 13.525 ms 2024-04-25T13:48:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:02 smithi137 ceph-mon[32568]: pgmap v534: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-04-25T13:48:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:02 smithi094 ceph-mon[29195]: pgmap v534: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-04-25T13:48:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:02 smithi138 ceph-mon[32977]: pgmap v534: 97 pgs: 97 active+clean; 47 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.7 MiB/s wr, 387 op/s 2024-04-25T13:48:02.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 705581 13.04 MB/sec execute 165 sec latency 16.264 ms 2024-04-25T13:48:03.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 707597 13.04 MB/sec execute 166 sec latency 15.003 ms 2024-04-25T13:48:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:04 smithi137 ceph-mon[32568]: pgmap v535: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-04-25T13:48:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:04 smithi094 ceph-mon[29195]: pgmap v535: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-04-25T13:48:04.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:04 smithi138 ceph-mon[32977]: pgmap v535: 97 pgs: 97 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-04-25T13:48:04.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 710313 13.06 MB/sec execute 167 sec latency 101.279 ms 2024-04-25T13:48:05.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 712203 13.02 MB/sec execute 168 sec latency 12.426 ms 2024-04-25T13:48:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:06 smithi137 ceph-mon[32568]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-04-25T13:48:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:06 smithi094 ceph-mon[29195]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-04-25T13:48:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:06 smithi138 ceph-mon[32977]: pgmap v536: 97 pgs: 97 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 360 op/s 2024-04-25T13:48:06.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 714475 13.03 MB/sec execute 169 sec latency 27.161 ms 2024-04-25T13:48:07.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 717527 13.06 MB/sec execute 170 sec latency 13.711 ms 2024-04-25T13:48:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:08 smithi137 ceph-mon[32568]: pgmap v537: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 362 op/s 2024-04-25T13:48:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:08 smithi094 ceph-mon[29195]: pgmap v537: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 362 op/s 2024-04-25T13:48:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:08 smithi138 ceph-mon[32977]: pgmap v537: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 362 op/s 2024-04-25T13:48:08.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 719908 13.03 MB/sec execute 171 sec latency 14.263 ms 2024-04-25T13:48:09.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 721637 13.02 MB/sec execute 172 sec latency 97.656 ms 2024-04-25T13:48:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:10 smithi137 ceph-mon[32568]: pgmap v538: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-25T13:48:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:10 smithi094 ceph-mon[29195]: pgmap v538: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-25T13:48:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:10 smithi138 ceph-mon[32977]: pgmap v538: 97 pgs: 97 active+clean; 44 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-25T13:48:10.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 724564 13.05 MB/sec execute 173 sec latency 16.811 ms 2024-04-25T13:48:11.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:48:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:48:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:48:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:48:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:48:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:48:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:48:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:48:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:48:11.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 726859 13.02 MB/sec execute 174 sec latency 17.686 ms 2024-04-25T13:48:12.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:12 smithi094 ceph-mon[29195]: pgmap v539: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-04-25T13:48:12.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:12 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:48:12.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:12 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:48:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:12 smithi137 ceph-mon[32568]: pgmap v539: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-04-25T13:48:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:12 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:48:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:12 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:48:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:12 smithi138 ceph-mon[32977]: pgmap v539: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-04-25T13:48:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:12 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:48:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:12 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:48:12.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 728925 13.02 MB/sec execute 175 sec latency 14.327 ms 2024-04-25T13:48:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- 172.21.15.94:0/117178614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac1089f0 msgr2=0x7ff6ac10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 --2- 172.21.15.94:0/117178614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac10ae10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009ca0 tx=0x7ff6a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- 172.21.15.94:0/117178614 shutdown_connections 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 --2- 172.21.15.94:0/117178614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac10ae10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 --2- 172.21.15.94:0/117178614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 --2- 172.21.15.94:0/117178614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac102ab0 0x7ff6ac1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- 172.21.15.94:0/117178614 >> 172.21.15.94:0/117178614 conn(0x7ff6ac0fdca0 msgr2=0x7ff6ac100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- 172.21.15.94:0/117178614 shutdown_connections 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- 172.21.15.94:0/117178614 wait complete. 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 Processor -- start 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.043+0000 7ff6b279d640 1 -- start start 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac102ab0 0x7ff6ac075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10e570 con 0x7ff6ac103cf0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10e3f0 con 0x7ff6ac1089f0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b279d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10e6f0 con 0x7ff6ac102ab0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47426/0 (socket says 172.21.15.94:47426) 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/3500896806 learned_addr learned my addr 172.21.15.94:0/3500896806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6abfff640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac102ab0 0x7ff6ac075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/3500896806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac102ab0 msgr2=0x7ff6ac075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac102ab0 0x7ff6ac075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/3500896806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 msgr2=0x7ff6ac1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/3500896806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1a1d00 con 0x7ff6ac103cf0 2024-04-25T13:48:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.044+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e9f0 tx=0x7ff6a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6a97fa640 1 -- 172.21.15.94:0/3500896806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000cd80 con 0x7ff6ac103cf0 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac1a1ff0 con 0x7ff6ac103cf0 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6a97fa640 1 -- 172.21.15.94:0/3500896806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a00054d0 con 0x7ff6ac103cf0 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6a97fa640 1 -- 172.21.15.94:0/3500896806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0010690 con 0x7ff6ac103cf0 2024-04-25T13:48:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.045+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1a2460 con 0x7ff6ac103cf0 2024-04-25T13:48:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.047+0000 7ff6a97fa640 1 -- 172.21.15.94:0/3500896806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff6a0004e90 con 0x7ff6ac103cf0 2024-04-25T13:48:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 Processor -- start 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 -- start start 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6a97fa640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 0x7ff6940b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c7312300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7319200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654c7273460 con 0x5654c6f6d1b0 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654c722d590 con 0x5654c6f4c4d0 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b964d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c726f420 con 0x5654c6f442f0 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b0d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6abfff640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 0x7ff6940b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b0d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47430/0 (socket says 172.21.15.94:47430) 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b0d3a640 1 -- 172.21.15.94:0/1727032836 learned_addr learned my addr 172.21.15.94:0/1727032836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c7312300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7319200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6a97fa640 1 -- 172.21.15.94:0/3500896806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff6a00df0d0 con 0x7ff6ac103cf0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/1727032836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 msgr2=0x5654c7319200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7319200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/1727032836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 msgr2=0x5654c7311d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/1727032836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c723a0d0 con 0x5654c6f442f0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.048+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.049+0000 7ff6abfff640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 0x7ff6940b1ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac1036b0 tx=0x7ff69c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.049+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c7312300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00dcb80 tx=0x7ff6a0007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc435b0ea31471c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.049+0000 7ff6927fc640 1 -- 172.21.15.94:0/1727032836 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00a6e60 con 0x5654c6f442f0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.049+0000 7ff6927fc640 1 -- 172.21.15.94:0/1727032836 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a00a7020 con 0x5654c6f442f0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.049+0000 7ff6927fc640 1 -- 172.21.15.94:0/1727032836 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0005180 con 0x5654c6f442f0 2024-04-25T13:48:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- 172.21.15.94:0/1727032836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 msgr2=0x5654c7312300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c7312300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00dcb80 tx=0x7ff6a0007ea0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- 172.21.15.94:0/1727032836 shutdown_connections 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7319200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c7312300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 --2- 172.21.15.94:0/1727032836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c7311d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- 172.21.15.94:0/1727032836 >> 172.21.15.94:0/1727032836 conn(0x5654c6f61300 msgr2=0x5654c7309300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- 172.21.15.94:0/1727032836 shutdown_connections 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- 172.21.15.94:0/1727032836 wait complete. 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 -- start start 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.050+0000 7ff6b964d740 1 Processor -- start 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f442f0 0x5654c731ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f6d1b0 0x5654c7322ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5654c7319c50 con 0x5654c6f4c4d0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654c71fab60 con 0x5654c6f442f0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c7319dd0 con 0x5654c6f6d1b0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 10 client.0 _ll_register_callbacks cb 0x5654c721f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f442f0 0x5654c731ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47436/0 (socket says 172.21.15.94:47436) 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/4147067442 learned_addr learned my addr 172.21.15.94:0/4147067442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f6d1b0 0x5654c7322ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/4147067442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f6d1b0 msgr2=0x5654c7322ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f6d1b0 0x5654c7322ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f442f0 msgr2=0x5654c731ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f442f0 0x5654c731ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/4147067442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c6f85210 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e9c0 tx=0x7ff6a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f91650306f83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff68b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b05c0 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff6a00a3b00 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.051+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a00a7020 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6a00a7c10 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 10 client.17802 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654c7051620 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a00a7020 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6a0013190 con 0x5654c6f4c4d0 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 10 client.17802 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 20 client.17802 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f2bd80 0x5654c7329310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 4 client.17802 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c7281070 con 0x5654c6f2bd80 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.052+0000 7ff6abfff640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f2bd80 0x5654c7329310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.053+0000 7ff68b7fe640 10 client.17802 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.053+0000 7ff6abfff640 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f2bd80 0x5654c7329310 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379e23c1800316c8 server_cookie=bacadcb14d956c9a in_seq=0 out_seq=0 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.053+0000 7ff68b7fe640 1 -- 172.21.15.94:0/4147067442 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654c7281070 con 0x5654c6f2bd80 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.053+0000 7ff68b7fe640 10 client.17802 handle_command_reply: tid=0 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.053+0000 7ff6b964d740 1 client.17802 shutdown 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 msgr2=0x5654c731bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e9c0 tx=0x7ff6a00048f0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 shutdown_connections 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f2bd80 0x5654c7329310 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f6d1b0 0x5654c7322ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f4c4d0 0x5654c731bf60 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff68b7fe640 0 client.17802 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 --2- 172.21.15.94:0/4147067442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f442f0 0x5654c731ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 >> 172.21.15.94:0/4147067442 conn(0x5654c6f61300 msgr2=0x5654c730c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 shutdown_connections 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 1 -- 172.21.15.94:0/4147067442 wait complete. 2024-04-25T13:48:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.054+0000 7ff6b964d740 20 client.17802 trim_cache size 0 max 16384 2024-04-25T13:48:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 Processor -- start 2024-04-25T13:48:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 -- start start 2024-04-25T13:48:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 0x5654c73163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f6d1b0 0x5654c7322de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1a9290 con 0x5654c6f442f0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1aadd0 con 0x5654c6f6d1b0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6b964d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1a9960 con 0x5654c6f4c4d0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47438/0 (socket says 172.21.15.94:47438) 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6abfff640 1 -- 172.21.15.94:0/2883775128 learned_addr learned my addr 172.21.15.94:0/2883775128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.069+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 0x5654c73163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6b0d3a640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f6d1b0 0x5654c7322de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/2883775128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f6d1b0 msgr2=0x5654c7322de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f6d1b0 0x5654c7322de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/2883775128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 msgr2=0x5654c7315ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 -- 172.21.15.94:0/2883775128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c7323320 con 0x5654c6f4c4d0 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6abfff640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 0x5654c73163e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00afee0 tx=0x7ff6a00a34d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eeff336dad22658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff691ffb640 1 -- 172.21.15.94:0/2883775128 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00a8640 con 0x5654c6f4c4d0 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff691ffb640 1 -- 172.21.15.94:0/2883775128 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a00a7020 con 0x5654c6f4c4d0 2024-04-25T13:48:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.070+0000 7ff691ffb640 1 -- 172.21.15.94:0/2883775128 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00a6c80 con 0x5654c6f4c4d0 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- 172.21.15.94:0/2883775128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 msgr2=0x5654c73163e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 0x5654c73163e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00afee0 tx=0x7ff6a00a34d0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- 172.21.15.94:0/2883775128 shutdown_connections 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f6d1b0 0x5654c7322de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f4c4d0 0x5654c73163e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 --2- 172.21.15.94:0/2883775128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f442f0 0x5654c7315ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- 172.21.15.94:0/2883775128 >> 172.21.15.94:0/2883775128 conn(0x5654c6f61300 msgr2=0x5654c731ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- 172.21.15.94:0/2883775128 shutdown_connections 2024-04-25T13:48:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- 172.21.15.94:0/2883775128 wait complete. 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.071+0000 7ff6b964d740 1 -- start start 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 Processor -- start 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c731fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7320310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c738f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1aa700 con 0x5654c6f6d1b0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1aa030 con 0x5654c6f4c4d0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1a7270 con 0x5654c6f442f0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b964d740 10 client.0 _ll_register_callbacks cb 0x7ff6a0005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b0d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c738f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b0d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c738f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47454/0 (socket says 172.21.15.94:47454) 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7320310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6b0d3a640 1 -- 172.21.15.94:0/995674826 learned_addr learned my addr 172.21.15.94:0/995674826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c731fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 -- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 msgr2=0x5654c7320310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7320310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 -- 172.21.15.94:0/995674826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 msgr2=0x5654c738f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c738f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.072+0000 7ff6abfff640 1 -- 172.21.15.94:0/995674826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c738fa80 con 0x5654c6f442f0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6abfff640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c731fdd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5654c7323da0 tx=0x7ff69c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bddf21ea90dddee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004020 con 0x5654c6f442f0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff69c0041c0 con 0x5654c6f442f0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a00a7020 con 0x5654c6f442f0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff69c0043c0 con 0x5654c6f442f0 2024-04-25T13:48:13.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 10 client.27157 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654c7390d90 con 0x5654c6f442f0 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a00a7020 con 0x5654c6f442f0 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff69c030070 con 0x5654c6f442f0 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 10 client.27157 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 10 client.27157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 20 client.27157 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f011c0 0x5654c73937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 4 client.27157 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.073+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654c727aee0 con 0x5654c6f011c0 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.074+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f011c0 0x5654c73937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.074+0000 7ff68b7fe640 10 client.27157 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.074+0000 7ff6ab7fe640 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f011c0 0x5654c73937b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2ab46892ac4f23 server_cookie=f70ba558defe2645 in_seq=0 out_seq=0 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.238+0000 7ff68b7fe640 1 -- 172.21.15.94:0/995674826 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654c727aee0 con 0x5654c6f011c0 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.238+0000 7ff68b7fe640 10 client.27157 handle_command_reply: tid=0 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.238+0000 7ff6b964d740 1 client.27157 shutdown 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.238+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 msgr2=0x5654c731fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.238+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c731fdd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5654c7323da0 tx=0x7ff69c004730 comp rx=0 tx=0).stop 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 shutdown_connections 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5654c6f011c0 0x5654c73937b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5654c6f6d1b0 0x5654c738f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654c6f4c4d0 0x5654c7320310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 --2- 172.21.15.94:0/995674826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c6f442f0 0x5654c731fdd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 >> 172.21.15.94:0/995674826 conn(0x5654c6f61300 msgr2=0x5654c731ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff68b7fe640 0 client.27157 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 shutdown_connections 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 1 -- 172.21.15.94:0/995674826 wait complete. 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b964d740 20 client.27157 trim_cache size 0 max 16384 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:48:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:48:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:48:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 msgr2=0x7ff6940b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 0x7ff6940b1ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac1036b0 tx=0x7ff69c005f90 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 msgr2=0x7ff6ac19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.239+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000e9f0 tx=0x7ff6a000edf0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 shutdown_connections 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6940afae0 0x7ff6940b1ff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6ac1089f0 0x7ff6ac1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6ac103cf0 0x7ff6ac19ef70 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 --2- 172.21.15.94:0/3500896806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6ac102ab0 0x7ff6ac075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 >> 172.21.15.94:0/3500896806 conn(0x7ff6ac0fdca0 msgr2=0x7ff6ac1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 shutdown_connections 2024-04-25T13:48:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:13.240+0000 7ff6b279d640 1 -- 172.21.15.94:0/3500896806 wait complete. 2024-04-25T13:48:13.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:13 smithi137 ceph-mon[32568]: pgmap v540: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:48:13.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:13 smithi094 ceph-mon[29195]: pgmap v540: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:48:13.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:13 smithi138 ceph-mon[32977]: pgmap v540: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:48:13.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 731637 13.04 MB/sec execute 176 sec latency 65.726 ms 2024-04-25T13:48:14.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 733792 13.02 MB/sec execute 177 sec latency 32.453 ms 2024-04-25T13:48:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:15 smithi094 ceph-mon[29195]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-04-25T13:48:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:15 smithi138 ceph-mon[32977]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-04-25T13:48:15.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 735705 13.01 MB/sec execute 178 sec latency 90.066 ms 2024-04-25T13:48:15.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:15 smithi137 ceph-mon[32568]: pgmap v541: 97 pgs: 97 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-04-25T13:48:16.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 738722 13.04 MB/sec execute 179 sec latency 13.617 ms 2024-04-25T13:48:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:17 smithi094 ceph-mon[29195]: pgmap v542: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-04-25T13:48:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:17 smithi138 ceph-mon[32977]: pgmap v542: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-04-25T13:48:17.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 740940 13.01 MB/sec execute 180 sec latency 12.849 ms 2024-04-25T13:48:17.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:17 smithi137 ceph-mon[32568]: pgmap v542: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-04-25T13:48:18.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 742870 13.01 MB/sec execute 181 sec latency 12.695 ms 2024-04-25T13:48:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:19 smithi094 ceph-mon[29195]: pgmap v543: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 302 op/s 2024-04-25T13:48:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:19 smithi138 ceph-mon[32977]: pgmap v543: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 302 op/s 2024-04-25T13:48:19.800 INFO:tasks.workunit.client.0.smithi094.stdout: 1 745884 13.04 MB/sec execute 182 sec latency 16.165 ms 2024-04-25T13:48:19.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:19 smithi137 ceph-mon[32568]: pgmap v543: 97 pgs: 97 active+clean; 41 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 302 op/s 2024-04-25T13:48:20.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 747452 13.00 MB/sec execute 183 sec latency 58.670 ms 2024-04-25T13:48:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:21 smithi094 ceph-mon[29195]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:48:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:21 smithi138 ceph-mon[32977]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:48:21.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 749685 13.00 MB/sec execute 184 sec latency 19.432 ms 2024-04-25T13:48:21.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:48:21.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:48:21.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:21 smithi137 ceph-mon[32568]: pgmap v544: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T13:48:21.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 -- 172.21.15.94:0/1654796200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 msgr2=0x7efeb410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 --2- 172.21.15.94:0/1654796200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb410ad40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7efeac009ca0 tx=0x7efeac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 -- 172.21.15.94:0/1654796200 shutdown_connections 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 --2- 172.21.15.94:0/1654796200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb410ad40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 --2- 172.21.15.94:0/1654796200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 0x7efeb4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 --2- 172.21.15.94:0/1654796200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 -- 172.21.15.94:0/1654796200 >> 172.21.15.94:0/1654796200 conn(0x7efeb40fdbd0 msgr2=0x7efeb4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.951+0000 7efebc2d9640 1 -- 172.21.15.94:0/1654796200 shutdown_connections 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 -- 172.21.15.94:0/1654796200 wait complete. 2024-04-25T13:48:21.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 Processor -- start 2024-04-25T13:48:21.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 -- start start 2024-04-25T13:48:21.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 0x7efeb4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb410e510 con 0x7efeb4108920 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb410e390 con 0x7efeb4103c20 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efebc2d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb410e690 con 0x7efeb41029e0 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.952+0000 7efeba876640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba075640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44006/0 (socket says 172.21.15.94:44006) 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 -- 172.21.15.94:0/2389099883 learned_addr learned my addr 172.21.15.94:0/2389099883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeb9874640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 0x7efeb4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 -- 172.21.15.94:0/2389099883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 msgr2=0x7efeb4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 -- 172.21.15.94:0/2389099883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 msgr2=0x7efeb4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 0x7efeb4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 -- 172.21.15.94:0/2389099883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb41a0f50 con 0x7efeb4108920 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba075640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeba876640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7efeac004af0 tx=0x7efeac031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeab7fe640 1 -- 172.21.15.94:0/2389099883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac04a070 con 0x7efeb4108920 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb4069e90 con 0x7efeb4108920 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeab7fe640 1 -- 172.21.15.94:0/2389099883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeac003b80 con 0x7efeb4108920 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efeab7fe640 1 -- 172.21.15.94:0/2389099883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac02f8c0 con 0x7efeb4108920 2024-04-25T13:48:21.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.953+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb406a430 con 0x7efeb4108920 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 Processor -- start 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeab7fe640 1 -- 172.21.15.94:0/2389099883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efeac04d070 con 0x7efeb4108920 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 -- start start 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 0x563da903aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c786b0 0x563da903bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563da8f9fd80 con 0x563da8c704d0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563da8f598b0 con 0x563da8c786b0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efec3182740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563da8f9bd40 con 0x563da8c99390 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeba075640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeba075640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44012/0 (socket says 172.21.15.94:44012) 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeba075640 1 -- 172.21.15.94:0/868247427 learned_addr learned my addr 172.21.15.94:0/868247427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeb9874640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c786b0 0x563da903bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.956+0000 7efeb9874640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 0x563da903aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 -- 172.21.15.94:0/868247427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 msgr2=0x563da903aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 0x563da903aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 -- 172.21.15.94:0/868247427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c786b0 msgr2=0x563da903bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c786b0 0x563da903bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 -- 172.21.15.94:0/868247427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da8f2aa40 con 0x563da8c704d0 2024-04-25T13:48:21.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeb9874640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 0x563da903aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeab7fe640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 0x7efea00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeba075640 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efeb00114f0 tx=0x7efeb00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83b40cd734da440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeb9874640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 0x7efea00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efeab7fe640 1 -- 172.21.15.94:0/2389099883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efeac101420 con 0x7efeb4108920 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efea8bfe640 1 -- 172.21.15.94:0/868247427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb001e070 con 0x563da8c704d0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efea8bfe640 1 -- 172.21.15.94:0/868247427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb00035f0 con 0x563da8c704d0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.957+0000 7efea8bfe640 1 -- 172.21.15.94:0/868247427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb000f880 con 0x563da8c704d0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- 172.21.15.94:0/868247427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 msgr2=0x563da903afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efeb9874640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 0x7efea00b2000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x563da903bab0 tx=0x7efea40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efeb00114f0 tx=0x7efeb00119b0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- 172.21.15.94:0/868247427 shutdown_connections 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c786b0 0x563da903bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da903afe0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 --2- 172.21.15.94:0/868247427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c99390 0x563da903aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- 172.21.15.94:0/868247427 >> 172.21.15.94:0/868247427 conn(0x563da8c8d4e0 msgr2=0x563da9036270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- 172.21.15.94:0/868247427 shutdown_connections 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- 172.21.15.94:0/868247427 wait complete. 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 -- start start 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.958+0000 7efec3182740 1 Processor -- start 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da9044e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c99390 0x563da904bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563da903c780 con 0x563da8c704d0 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563da8f26d10 con 0x563da8c99390 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563da903c900 con 0x563da8c786b0 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efec3182740 10 client.0 _ll_register_callbacks cb 0x563da8f53210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44020/0 (socket says 172.21.15.94:44020) 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 -- 172.21.15.94:0/525852348 learned_addr learned my addr 172.21.15.94:0/525852348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba876640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c99390 0x563da904bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeb9874640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da9044e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 -- 172.21.15.94:0/525852348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 msgr2=0x563da9044e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da9044e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 -- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c99390 msgr2=0x563da904bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c99390 0x563da904bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 -- 172.21.15.94:0/525852348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da8d7b7d0 con 0x563da8c704d0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efeba075640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efeb00114c0 tx=0x7efeb00042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7818086f0513e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.959+0000 7efe9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0026020 con 0x563da8c704d0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efeb0009be0 con 0x563da8c704d0 2024-04-25T13:48:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb00035f0 con 0x563da8c704d0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efeb0031050 con 0x563da8c704d0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 10 client.17820 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563da8cb13f0 con 0x563da8c704d0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb00035f0 con 0x563da8c704d0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efeb0021030 con 0x563da8c704d0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 10 client.17820 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 20 client.17820 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c57f60 0x563da9052050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 4 client.17820 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da8fad990 con 0x563da8c57f60 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.960+0000 7efeb9874640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c57f60 0x563da9052050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.961+0000 7efe9dffb640 10 client.17820 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.961+0000 7efeb9874640 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c57f60 0x563da9052050 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a151e8f58be5aa21 server_cookie=8bd52bf6812033bb in_seq=0 out_seq=0 2024-04-25T13:48:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efe9dffb640 1 -- 172.21.15.94:0/525852348 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563da8fad990 con 0x563da8c57f60 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efe9dffb640 10 client.17820 handle_command_reply: tid=0 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 client.17820 shutdown 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 msgr2=0x563da9044910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efeb00114c0 tx=0x7efeb00042b0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 shutdown_connections 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c57f60 0x563da9052050 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c99390 0x563da904bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da9044e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efe9dffb640 0 client.17820 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 --2- 172.21.15.94:0/525852348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9044910 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 >> 172.21.15.94:0/525852348 conn(0x563da8c8d4e0 msgr2=0x563da9039180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 shutdown_connections 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 1 -- 172.21.15.94:0/525852348 wait complete. 2024-04-25T13:48:21.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.962+0000 7efec3182740 20 client.17820 trim_cache size 0 max 16384 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 Processor -- start 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 -- start start 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 0x563da9044bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c704d0 0x563da9039070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41a92b0 con 0x563da8c786b0 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41aadf0 con 0x563da8c704d0 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efec3182740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41a9980 con 0x563da8c57f60 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba876640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 0x563da9044bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba876640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44024/0 (socket says 172.21.15.94:44024) 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba876640 1 -- 172.21.15.94:0/175985037 learned_addr learned my addr 172.21.15.94:0/175985037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeb9874640 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c704d0 0x563da9039070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 -- 172.21.15.94:0/175985037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c704d0 msgr2=0x563da9039070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c704d0 0x563da9039070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 -- 172.21.15.94:0/175985037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 msgr2=0x563da90395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba075640 1 -- 172.21.15.94:0/175985037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da8c710f0 con 0x563da8c57f60 2024-04-25T13:48:21.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.984+0000 7efeba876640 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.985+0000 7efeba075640 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 0x563da9044bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009250 tx=0x7efeb00035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8056a6a07052596f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.985+0000 7efe9ffff640 1 -- 172.21.15.94:0/175985037 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0026070 con 0x563da8c57f60 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.985+0000 7efe9ffff640 1 -- 172.21.15.94:0/175985037 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb0003e10 con 0x563da8c57f60 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.985+0000 7efe9ffff640 1 -- 172.21.15.94:0/175985037 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb001e040 con 0x563da8c57f60 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- 172.21.15.94:0/175985037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 msgr2=0x563da9044bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 0x563da9044bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009250 tx=0x7efeb00035f0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- 172.21.15.94:0/175985037 shutdown_connections 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c786b0 0x563da90395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c704d0 0x563da9039070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- 172.21.15.94:0/175985037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c57f60 0x563da9044bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- 172.21.15.94:0/175985037 >> 172.21.15.94:0/175985037 conn(0x563da8c8d4e0 msgr2=0x563da90444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- 172.21.15.94:0/175985037 shutdown_connections 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- 172.21.15.94:0/175985037 wait complete. 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- start start 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 Processor -- start 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da90b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41aa720 con 0x563da8c704d0 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41aa050 con 0x563da8c57f60 2024-04-25T13:48:21.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb41a7290 con 0x563da8c786b0 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efec3182740 10 client.0 _ll_register_callbacks cb 0x7efeb001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.986+0000 7efeb9874640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43852/0 (socket says 172.21.15.94:43852) 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeb9874640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44040/0 (socket says 172.21.15.94:44040) 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 -- 172.21.15.94:0/59778696 learned_addr learned my addr 172.21.15.94:0/59778696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba876640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da90b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 -- 172.21.15.94:0/59778696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 msgr2=0x563da90b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da90b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 -- 172.21.15.94:0/59778696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 msgr2=0x563da9048710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 -- 172.21.15.94:0/59778696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da8f604b0 con 0x563da8c57f60 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeb9874640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efeba075640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edddaf68b2daf3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efe9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0026070 con 0x563da8c57f60 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efeb00042b0 con 0x563da8c57f60 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb0003e10 con 0x563da8c57f60 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efeb0046050 con 0x563da8c57f60 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efec3182740 10 client.27296 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.987+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563da90b9740 con 0x563da8c57f60 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb0003e10 con 0x563da8c57f60 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efeb0021040 con 0x563da8c57f60 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 10 client.27296 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 20 client.27296 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c99390 0x563da90bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 4 client.27296 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:48:21.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563da8fa7800 con 0x563da8c99390 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efeb9874640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c99390 0x563da90bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efe9dffb640 10 client.27296 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.988+0000 7efeb9874640 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c99390 0x563da90bc110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659fcd62a64c7675 server_cookie=9a9cc779eb678167 in_seq=0 out_seq=0 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efe9dffb640 1 -- 172.21.15.94:0/59778696 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563da8fa7800 con 0x563da8c99390 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efe9dffb640 10 client.27296 handle_command_reply: tid=0 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 client.27296 shutdown 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 msgr2=0x563da90481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0009d60 comp rx=0 tx=0).stop 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 shutdown_connections 2024-04-25T13:48:21.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563da8c99390 0x563da90bc110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563da8c786b0 0x563da90b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563da8c704d0 0x563da9048710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efe9dffb640 0 client.27296 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 --2- 172.21.15.94:0/59778696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563da8c57f60 0x563da90481d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 >> 172.21.15.94:0/59778696 conn(0x563da8c8d4e0 msgr2=0x563da9039070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 shutdown_connections 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 1 -- 172.21.15.94:0/59778696 wait complete. 2024-04-25T13:48:21.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efec3182740 20 client.27296 trim_cache size 0 max 16384 2024-04-25T13:48:22.000 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:48:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:48:22.000 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:22.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 msgr2=0x7efea00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 0x7efea00b2000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x563da903bab0 tx=0x7efea40046c0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 msgr2=0x7efeb41a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7efeac004af0 tx=0x7efeac031680 comp rx=0 tx=0).stop 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 shutdown_connections 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea00afaf0 0x7efea00b2000 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb4108920 0x7efeb41a08e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb4103c20 0x7efeb4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 --2- 172.21.15.94:0/2389099883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb41029e0 0x7efeb4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.989+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 >> 172.21.15.94:0/2389099883 conn(0x7efeb40fdbd0 msgr2=0x7efeb4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.990+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 shutdown_connections 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:21.990+0000 7efebc2d9640 1 -- 172.21.15.94:0/2389099883 wait complete. 2024-04-25T13:48:22.003 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:48:22.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 -- 172.21.15.94:0/3265050205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c102250 msgr2=0x7fa41c10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 --2- 172.21.15.94:0/3265050205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c102250 0x7fa41c10bcb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa414009ca0 tx=0x7fa41402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 -- 172.21.15.94:0/3265050205 shutdown_connections 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 --2- 172.21.15.94:0/3265050205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c102250 0x7fa41c10bcb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 --2- 172.21.15.94:0/3265050205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c101750 0x7fa41c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 --2- 172.21.15.94:0/3265050205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 0x7fa41c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 -- 172.21.15.94:0/3265050205 >> 172.21.15.94:0/3265050205 conn(0x7fa41c0fd370 msgr2=0x7fa41c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 -- 172.21.15.94:0/3265050205 shutdown_connections 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.121+0000 7fa4226b1640 1 -- 172.21.15.94:0/3265050205 wait complete. 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 Processor -- start 2024-04-25T13:48:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 -- start start 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c102250 0x7fa41c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 0x7fa41c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c10f610 con 0x7fa41c101750 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c10f490 con 0x7fa41c1050a0 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa4226b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c10f790 con 0x7fa41c102250 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44056/0 (socket says 172.21.15.94:44056) 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa41bfff640 1 -- 172.21.15.94:0/1857934826 learned_addr learned my addr 172.21.15.94:0/1857934826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa41b7fe640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c102250 0x7fa41c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.122+0000 7fa420c4e640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 0x7fa41c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 -- 172.21.15.94:0/1857934826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c102250 msgr2=0x7fa41c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c102250 0x7fa41c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 -- 172.21.15.94:0/1857934826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 msgr2=0x7fa41c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 0x7fa41c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 -- 172.21.15.94:0/1857934826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c1a1c90 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa41bfff640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e9f0 tx=0x7fa40c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ccc0 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c0035f0 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c1a1f80 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ce60 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.123+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1a23e0 con 0x7fa41c101750 2024-04-25T13:48:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.124+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c06a1c0 con 0x7fa41c101750 2024-04-25T13:48:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.126+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa40c003aa0 con 0x7fa41c101750 2024-04-25T13:48:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.126+0000 7fa4197fa640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 0x7fa4040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.126+0000 7fa41b7fe640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 0x7fa4040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.126+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa40c0deff0 con 0x7fa41c101750 2024-04-25T13:48:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.127+0000 7fa41b7fe640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 0x7fa4040b2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa41c19b1f0 tx=0x7fa41000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.130+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa41c06a1c0 con 0x7fa41c101750 2024-04-25T13:48:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.358+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa41c02dc10 con 0x7fa41c101750 2024-04-25T13:48:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.359+0000 7fa4197fa640 1 -- 172.21.15.94:0/1857934826 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fa40c0a7b30 con 0x7fa41c101750 2024-04-25T13:48:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:48:22.359 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:48:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:48:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 msgr2=0x7fa4040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 0x7fa4040b2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa41c19b1f0 tx=0x7fa41000b410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 msgr2=0x7fa41c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e9f0 tx=0x7fa40c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 shutdown_connections 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4040afaf0 0x7fa4040b2000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa41c1050a0 0x7fa41c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa41c102250 0x7fa41c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 --2- 172.21.15.94:0/1857934826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa41c101750 0x7fa41c19a150 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 >> 172.21.15.94:0/1857934826 conn(0x7fa41c0fd370 msgr2=0x7fa41c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 shutdown_connections 2024-04-25T13:48:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.362+0000 7fa4226b1640 1 -- 172.21.15.94:0/1857934826 wait complete. 2024-04-25T13:48:22.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:48:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/2112066136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241043e0 msgr2=0x7f2e24102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/2112066136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241043e0 0x7f2e24102de0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009900 tx=0x7f2e1802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/2112066136 shutdown_connections 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/2112066136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 0x7f2e2410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/2112066136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241043e0 0x7f2e24102de0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/2112066136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e241027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.487+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/2112066136 >> 172.21.15.94:0/2112066136 conn(0x7f2e240fd380 msgr2=0x7f2e240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.488+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/2112066136 shutdown_connections 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.488+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/2112066136 wait complete. 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.488+0000 7f2e2a7c5640 1 Processor -- start 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.488+0000 7f2e2a7c5640 1 -- start start 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e2419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 0x7f2e2419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410f640 con 0x7f2e241a14a0 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410f4c0 con 0x7f2e24108a30 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2410f7c0 con 0x7f2e24103f00 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e2419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 0x7f2e2419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44078/0 (socket says 172.21.15.94:44078) 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 -- 172.21.15.94:0/3098776848 learned_addr learned my addr 172.21.15.94:0/3098776848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 -- 172.21.15.94:0/3098776848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 msgr2=0x7f2e2419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e2419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 -- 172.21.15.94:0/3098776848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 msgr2=0x7f2e2419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 0x7f2e2419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 -- 172.21.15.94:0/3098776848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e241a3f30 con 0x7f2e241a14a0 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e23fff640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e2419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e28d62640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00eb10 tx=0x7f2e1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e217fa640 1 -- 172.21.15.94:0/3098776848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004b20 con 0x7f2e241a14a0 2024-04-25T13:48:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e217fa640 1 -- 172.21.15.94:0/3098776848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c004cc0 con 0x7f2e241a14a0 2024-04-25T13:48:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e241a4220 con 0x7f2e241a14a0 2024-04-25T13:48:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e217fa640 1 -- 172.21.15.94:0/3098776848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c010670 con 0x7f2e241a14a0 2024-04-25T13:48:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.489+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a4720 con 0x7f2e241a14a0 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.492+0000 7f2e31679740 1 Processor -- start 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.492+0000 7f2e217fa640 1 -- 172.21.15.94:0/3098776848 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2e1c005020 con 0x7f2e241a14a0 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 -- start start 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e96b0 0x5628bc5ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628bc510d80 con 0x5628bc20a390 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5628bc4caa20 con 0x5628bc1e96b0 2024-04-25T13:48:22.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e31679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5628bc50cd40 con 0x5628bc1e14d0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44080/0 (socket says 172.21.15.94:44080) 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e237fe640 1 -- 172.21.15.94:0/3343055878 learned_addr learned my addr 172.21.15.94:0/3343055878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e237fe640 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e96b0 0x5628bc5ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e217fa640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 0x7f2e0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 -- 172.21.15.94:0/3343055878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e96b0 msgr2=0x5628bc5ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e96b0 0x5628bc5ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 -- 172.21.15.94:0/3343055878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 msgr2=0x5628bc5ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e23fff640 1 -- 172.21.15.94:0/3343055878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628bc4d5c30 con 0x5628bc1e14d0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e217fa640 1 -- 172.21.15.94:0/3098776848 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2e1c0df280 con 0x7f2e241a14a0 2024-04-25T13:48:22.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.493+0000 7f2e28d62640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 0x7f2e0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.494+0000 7f2e23fff640 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5acd90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009fa0 tx=0x7f2e140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3bd060faec3ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.494+0000 7f2e0a7fc640 1 -- 172.21.15.94:0/3343055878 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14004030 con 0x5628bc1e14d0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.494+0000 7f2e0a7fc640 1 -- 172.21.15.94:0/3343055878 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e14005020 con 0x5628bc1e14d0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.494+0000 7f2e0a7fc640 1 -- 172.21.15.94:0/3343055878 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14004c00 con 0x5628bc1e14d0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- 172.21.15.94:0/3343055878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 msgr2=0x5628bc5acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e28d62640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 0x7f2e0c0b2000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0a4930 tx=0x7f2e1c00cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5acd90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009fa0 tx=0x7f2e140118d0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- 172.21.15.94:0/3343055878 shutdown_connections 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e96b0 0x5628bc5ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5acd90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 --2- 172.21.15.94:0/3343055878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- 172.21.15.94:0/3343055878 >> 172.21.15.94:0/3343055878 conn(0x5628bc1fe4e0 msgr2=0x5628bc5a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- 172.21.15.94:0/3343055878 shutdown_connections 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- 172.21.15.94:0/3343055878 wait complete. 2024-04-25T13:48:22.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 -- start start 2024-04-25T13:48:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.495+0000 7f2e31679740 1 Processor -- start 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc5bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628bc5ae530 con 0x5628bc1e96b0 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5628bc497dc0 con 0x5628bc20a390 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5628bc5ae6b0 con 0x5628bc1e14d0 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e31679740 10 client.0 _ll_register_callbacks cb 0x5628bc4b3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44086/0 (socket says 172.21.15.94:44086) 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 -- 172.21.15.94:0/944829235 learned_addr learned my addr 172.21.15.94:0/944829235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e237fe640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e237fe640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc5bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 -- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 msgr2=0x5628bc5b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 -- 172.21.15.94:0/944829235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 msgr2=0x5628bc5bda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc5bda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 -- 172.21.15.94:0/944829235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628bc2223f0 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e23fff640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400a010 tx=0x7f2e140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30125098448a0a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.496+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1401d050 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2e14004940 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e14005020 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e14005540 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 10 client.17844 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628bc2ec7d0 con 0x5628bc1e96b0 2024-04-25T13:48:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e14005020 con 0x5628bc1e96b0 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1400f540 con 0x5628bc1e96b0 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 10 client.17844 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 10 client.17844 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 20 client.17844 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc1c8f60 0x5628bc5c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 4 client.17844 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628bc51e990 con 0x5628bc1c8f60 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.497+0000 7f2e237fe640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc1c8f60 0x5628bc5c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.498+0000 7f2e037fe640 10 client.17844 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.498+0000 7f2e237fe640 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc1c8f60 0x5628bc5c4030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=521087d72e4ba3ec server_cookie=a4cce9547ea4249 in_seq=0 out_seq=0 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.498+0000 7f2e037fe640 1 -- 172.21.15.94:0/944829235 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628bc51e990 con 0x5628bc1c8f60 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.498+0000 7f2e037fe640 10 client.17844 handle_command_reply: tid=0 2024-04-25T13:48:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.498+0000 7f2e31679740 1 client.17844 shutdown 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 msgr2=0x5628bc5b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400a010 tx=0x7f2e140042f0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 shutdown_connections 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc1c8f60 0x5628bc5c4030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc5bda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5b6ad0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e037fe640 0 client.17844 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 --2- 172.21.15.94:0/944829235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 >> 172.21.15.94:0/944829235 conn(0x5628bc1fe4e0 msgr2=0x5628bc5aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 shutdown_connections 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 1 -- 172.21.15.94:0/944829235 wait complete. 2024-04-25T13:48:22.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.499+0000 7f2e31679740 20 client.17844 trim_cache size 0 max 16384 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.512+0000 7f2e31679740 1 Processor -- start 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 -- start start 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e14d0 0x5628bc5b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 0x5628bc5b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ad7c0 con 0x5628bc20a390 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241af300 con 0x5628bc1e14d0 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e31679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ade90 con 0x5628bc1e96b0 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44100/0 (socket says 172.21.15.94:44100) 2024-04-25T13:48:22.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 -- 172.21.15.94:0/43488959 learned_addr learned my addr 172.21.15.94:0/43488959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e14d0 0x5628bc5b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e23fff640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 0x5628bc5b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 -- 172.21.15.94:0/43488959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 msgr2=0x5628bc5b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 0x5628bc5b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 -- 172.21.15.94:0/43488959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e14d0 msgr2=0x5628bc5b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e14d0 0x5628bc5b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 -- 172.21.15.94:0/43488959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628bc5b46a0 con 0x5628bc20a390 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e23fff640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 0x5628bc5b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e237fe640 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005940 tx=0x7f2e1802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fe69bb77a7e60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.513+0000 7f2e09ffb640 1 -- 172.21.15.94:0/43488959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18035040 con 0x5628bc20a390 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e09ffb640 1 -- 172.21.15.94:0/43488959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1804f480 con 0x5628bc20a390 2024-04-25T13:48:22.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e09ffb640 1 -- 172.21.15.94:0/43488959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18003850 con 0x5628bc20a390 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 -- 172.21.15.94:0/43488959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 msgr2=0x5628bc5c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005940 tx=0x7f2e1802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 -- 172.21.15.94:0/43488959 shutdown_connections 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc20a390 0x5628bc5c97a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e96b0 0x5628bc5b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 --2- 172.21.15.94:0/43488959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc1e14d0 0x5628bc5b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.514+0000 7f2e31679740 1 -- 172.21.15.94:0/43488959 >> 172.21.15.94:0/43488959 conn(0x5628bc1fe4e0 msgr2=0x5628bc5ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- 172.21.15.94:0/43488959 shutdown_connections 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- 172.21.15.94:0/43488959 wait complete. 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- start start 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 Processor -- start 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc62a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241aec30 con 0x5628bc1e96b0 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ae560 con 0x5628bc20a390 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ab7a0 con 0x5628bc1e14d0 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.515+0000 7f2e31679740 10 client.0 _ll_register_callbacks cb 0x7f2e14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44104/0 (socket says 172.21.15.94:44104) 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5bbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47824/0 (socket says 172.21.15.94:47824) 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 -- 172.21.15.94:0/2312139154 learned_addr learned my addr 172.21.15.94:0/2312139154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e237fe640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc62a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 -- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 msgr2=0x5628bc5bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 -- 172.21.15.94:0/2312139154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 msgr2=0x5628bc62a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc62a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 -- 172.21.15.94:0/2312139154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628bc5bca40 con 0x5628bc1e96b0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e23fff640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009730 tx=0x7f2e14009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f114390d0922307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1403c020 con 0x5628bc1e96b0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2e14004d50 con 0x5628bc1e96b0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1804f480 con 0x5628bc1e96b0 2024-04-25T13:48:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e1401d040 con 0x5628bc1e96b0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e31679740 10 client.17850 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.516+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628bc62b890 con 0x5628bc1e96b0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1804f480 con 0x5628bc1e96b0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e14003b30 con 0x5628bc1e96b0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 10 client.17850 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 10 client.17850 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 20 client.17850 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc19e3a0 0x5628bc62e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 4 client.17850 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628bc518800 con 0x5628bc19e3a0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e237fe640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc19e3a0 0x5628bc62e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e037fe640 10 client.17850 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.517+0000 7f2e237fe640 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc19e3a0 0x5628bc62e2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfd4d8a49e6684c8 server_cookie=9a8e715d1131c5f6 in_seq=0 out_seq=0 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e037fe640 1 -- 172.21.15.94:0/2312139154 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628bc518800 con 0x5628bc19e3a0 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e037fe640 10 client.17850 handle_command_reply: tid=0 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 client.17850 shutdown 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 msgr2=0x5628bc5bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009730 tx=0x7f2e14009e20 comp rx=0 tx=0).stop 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 shutdown_connections 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5628bc19e3a0 0x5628bc62e2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628bc20a390 0x5628bc62a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e037fe640 0 client.17850 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628bc1e96b0 0x5628bc5bc1f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 --2- 172.21.15.94:0/2312139154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628bc1e14d0 0x5628bc5bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 >> 172.21.15.94:0/2312139154 conn(0x5628bc1fe4e0 msgr2=0x5628bc5ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 shutdown_connections 2024-04-25T13:48:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 1 -- 172.21.15.94:0/2312139154 wait complete. 2024-04-25T13:48:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.518+0000 7f2e31679740 20 client.17850 trim_cache size 0 max 16384 2024-04-25T13:48:22.525 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 msgr2=0x7f2e0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 0x7f2e0c0b2000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0a4930 tx=0x7f2e1c00cf70 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 msgr2=0x7f2e241a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00eb10 tx=0x7f2e1c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 shutdown_connections 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2e0c0afaf0 0x7f2e0c0b2000 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e241a14a0 0x7f2e241a38c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e24108a30 0x7f2e2419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 --2- 172.21.15.94:0/3098776848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e24103f00 0x7f2e2419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 >> 172.21.15.94:0/3098776848 conn(0x7f2e240fd380 msgr2=0x7f2e240fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 shutdown_connections 2024-04-25T13:48:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.519+0000 7f2e2a7c5640 1 -- 172.21.15.94:0/3098776848 wait complete. 2024-04-25T13:48:22.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:48:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 -- 172.21.15.94:0/2048968537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5681029e0 msgr2=0x7fa5681036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 --2- 172.21.15.94:0/2048968537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5681029e0 0x7fa5681036e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02a410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 -- 172.21.15.94:0/2048968537 shutdown_connections 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 --2- 172.21.15.94:0/2048968537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa568108920 0x7fa56810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 --2- 172.21.15.94:0/2048968537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568103c20 0x7fa568104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 --2- 172.21.15.94:0/2048968537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5681029e0 0x7fa5681036e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 -- 172.21.15.94:0/2048968537 >> 172.21.15.94:0/2048968537 conn(0x7fa5680fdbd0 msgr2=0x7fa568100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 -- 172.21.15.94:0/2048968537 shutdown_connections 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.643+0000 7fa56e051640 1 -- 172.21.15.94:0/2048968537 wait complete. 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 Processor -- start 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 -- start start 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5681029e0 0x7fa568199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568108920 0x7fa5681a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810e4e0 con 0x7fa568103c20 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810e360 con 0x7fa568108920 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa56e051640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810e660 con 0x7fa5681029e0 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44128/0 (socket says 172.21.15.94:44128) 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa566ffd640 1 -- 172.21.15.94:0/2648472316 learned_addr learned my addr 172.21.15.94:0/2648472316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5681029e0 0x7fa568199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.644+0000 7fa567fff640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568108920 0x7fa5681a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 -- 172.21.15.94:0/2648472316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5681029e0 msgr2=0x7fa568199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5681029e0 0x7fa568199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 -- 172.21.15.94:0/2648472316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568108920 msgr2=0x7fa5681a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568108920 0x7fa5681a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 -- 172.21.15.94:0/2648472316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5681a0f50 con 0x7fa568103c20 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e9f0 tx=0x7fa55800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa564ff9640 1 -- 172.21.15.94:0/2648472316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55800cd80 con 0x7fa568103c20 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa564ff9640 1 -- 172.21.15.94:0/2648472316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580108e0 con 0x7fa568103c20 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5681a1240 con 0x7fa568103c20 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa564ff9640 1 -- 172.21.15.94:0/2648472316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558018980 con 0x7fa568103c20 2024-04-25T13:48:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.645+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a1790 con 0x7fa568103c20 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.647+0000 7fa564ff9640 1 -- 172.21.15.94:0/2648472316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa558010430 con 0x7fa568103c20 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 Processor -- start 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa564ff9640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 0x7fa5500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 -- start start 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa567fff640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 0x7fa5500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 0x560ba392a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa564ff9640 1 -- 172.21.15.94:0/2648472316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa5580defe0 con 0x7fa568103c20 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba392ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ba388ed80 con 0x560ba355f4d0 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ba3848a20 con 0x560ba3588390 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa574fd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ba388ad40 con 0x560ba35676b0 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44130/0 (socket says 172.21.15.94:44130) 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa566ffd640 1 -- 172.21.15.94:0/2279060376 learned_addr learned my addr 172.21.15.94:0/2279060376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 0x560ba392a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.648+0000 7fa567fff640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba392ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 -- 172.21.15.94:0/2279060376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 msgr2=0x560ba392ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba392ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 -- 172.21.15.94:0/2279060376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 msgr2=0x560ba392a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 0x560ba392a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 -- 172.21.15.94:0/2279060376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba3853c30 con 0x560ba355f4d0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa567fff640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba392ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 0x560ba392a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa567fff640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 0x7fa5500b2000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a3530 tx=0x7fa5580107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730cffc868a747b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa54dffb640 1 -- 172.21.15.94:0/2279060376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ab7b0 con 0x560ba355f4d0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa54dffb640 1 -- 172.21.15.94:0/2279060376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580b0260 con 0x560ba355f4d0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.649+0000 7fa54dffb640 1 -- 172.21.15.94:0/2279060376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580a6970 con 0x560ba355f4d0 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- 172.21.15.94:0/2279060376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 msgr2=0x560ba392ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a3530 tx=0x7fa5580107e0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- 172.21.15.94:0/2279060376 shutdown_connections 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba392ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba355f4d0 0x560ba392ad90 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2279060376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba3588390 0x560ba392a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- 172.21.15.94:0/2279060376 >> 172.21.15.94:0/2279060376 conn(0x560ba357c4e0 msgr2=0x560ba3926020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- 172.21.15.94:0/2279060376 shutdown_connections 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- 172.21.15.94:0/2279060376 wait complete. 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 -- start start 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.650+0000 7fa574fd7740 1 Processor -- start 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3934560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba3588390 0x560ba393ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ba392c530 con 0x560ba35676b0 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ba3815dc0 con 0x560ba355f4d0 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ba392c6b0 con 0x560ba3588390 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 10 client.0 _ll_register_callbacks cb 0x560ba3831600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44142/0 (socket says 172.21.15.94:44142) 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3934560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 -- 172.21.15.94:0/2428673154 learned_addr learned my addr 172.21.15.94:0/2428673154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa567fff640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba3588390 0x560ba393ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 -- 172.21.15.94:0/2428673154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba3588390 msgr2=0x560ba393ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba3588390 0x560ba393ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 -- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 msgr2=0x560ba3934560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3934560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 -- 172.21.15.94:0/2428673154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba35a03f0 con 0x560ba35676b0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3934560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa566ffd640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a7360 tx=0x7fa5580a6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fedb5ca9ebf3926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ab7b0 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa5580aed40 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.651+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5580b0260 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5580c82e0 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 10 client.17862 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ba366a7d0 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580b0260 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa5580a6210 con 0x560ba35676b0 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 10 client.17862 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 10 client.17862 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 20 client.17862 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba3546f60 0x560ba3941e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 4 client.17862 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ba389c990 con 0x560ba3546f60 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.652+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba3546f60 0x560ba3941e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.653+0000 7fa546ffd640 10 client.17862 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.653+0000 7fa5677fe640 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba3546f60 0x560ba3941e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e232bef75a80b5c server_cookie=fe04301791d605a2 in_seq=0 out_seq=0 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.653+0000 7fa546ffd640 1 -- 172.21.15.94:0/2428673154 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ba389c990 con 0x560ba3546f60 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.653+0000 7fa546ffd640 10 client.17862 handle_command_reply: tid=0 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.653+0000 7fa574fd7740 1 client.17862 shutdown 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 msgr2=0x560ba3934aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a7360 tx=0x7fa5580a6af0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 shutdown_connections 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba3546f60 0x560ba3941e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba3588390 0x560ba393ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba35676b0 0x560ba3934aa0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa546ffd640 0 client.17862 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 --2- 172.21.15.94:0/2428673154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3934560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 >> 172.21.15.94:0/2428673154 conn(0x560ba357c4e0 msgr2=0x560ba3928f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 shutdown_connections 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 1 -- 172.21.15.94:0/2428673154 wait complete. 2024-04-25T13:48:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.654+0000 7fa574fd7740 20 client.17862 trim_cache size 0 max 16384 2024-04-25T13:48:22.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.667+0000 7fa574fd7740 1 Processor -- start 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.667+0000 7fa574fd7740 1 -- start start 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3947590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3947ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681a9460 con 0x560ba3588390 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681aafa0 con 0x560ba355f4d0 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa574fd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681a9b30 con 0x560ba35676b0 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44144/0 (socket says 172.21.15.94:44144) 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 -- 172.21.15.94:0/3127000431 learned_addr learned my addr 172.21.15.94:0/3127000431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa566ffd640 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3947ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa5677fe640 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3947590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 -- 172.21.15.94:0/3127000431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 msgr2=0x560ba3947ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3947ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 -- 172.21.15.94:0/3127000431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 msgr2=0x560ba3947590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3947590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 -- 172.21.15.94:0/3127000431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba393be60 con 0x560ba3588390 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa567fff640 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa56000fd90 tx=0x7fa56001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd67c5d0da55127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa54d7fa640 1 -- 172.21.15.94:0/3127000431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56002e070 con 0x560ba3588390 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa54d7fa640 1 -- 172.21.15.94:0/3127000431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa560018040 con 0x560ba3588390 2024-04-25T13:48:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.668+0000 7fa54d7fa640 1 -- 172.21.15.94:0/3127000431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56001a040 con 0x560ba3588390 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 -- 172.21.15.94:0/3127000431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 msgr2=0x560ba393b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa56000fd90 tx=0x7fa56001f000 comp rx=0 tx=0).stop 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 -- 172.21.15.94:0/3127000431 shutdown_connections 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba393b920 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3947ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3127000431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3947590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 -- 172.21.15.94:0/3127000431 >> 172.21.15.94:0/3127000431 conn(0x560ba357c4e0 msgr2=0x560ba392f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 -- 172.21.15.94:0/3127000431 shutdown_connections 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.669+0000 7fa574fd7740 1 -- 172.21.15.94:0/3127000431 wait complete. 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 -- start start 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 Processor -- start 2024-04-25T13:48:22.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3938690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3938bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681aa8d0 con 0x560ba3588390 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681aa200 con 0x560ba355f4d0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5681a7440 con 0x560ba35676b0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa574fd7740 10 client.0 _ll_register_callbacks cb 0x7fa5580ab070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44154/0 (socket says 172.21.15.94:44154) 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 -- 172.21.15.94:0/3417265525 learned_addr learned my addr 172.21.15.94:0/3417265525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa5677fe640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3938690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa566ffd640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3938bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 -- 172.21.15.94:0/3417265525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 msgr2=0x560ba3938bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3938bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 -- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 msgr2=0x560ba3938690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3938690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa567fff640 1 -- 172.21.15.94:0/3417265525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba393bb40 con 0x560ba3588390 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.670+0000 7fa5677fe640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3938690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa567fff640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa5600097d0 tx=0x7fa56000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd50a84d9f985dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56002e030 con 0x560ba3588390 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa560018040 con 0x560ba3588390 2024-04-25T13:48:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa560009040 con 0x560ba3588390 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa56001a040 con 0x560ba3588390 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 10 client.17868 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ba39ad9e0 con 0x560ba3588390 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa560018040 con 0x560ba3588390 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa560028030 con 0x560ba3588390 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 10 client.17868 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 10 client.17868 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 20 client.17868 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba351c3a0 0x560ba39b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 4 client.17868 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.671+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ba3896800 con 0x560ba351c3a0 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa5677fe640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba351c3a0 0x560ba39b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa5677fe640 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba351c3a0 0x560ba39b03b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50ab3532edb7d316 server_cookie=16e6c0193300ed04 in_seq=0 out_seq=0 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa546ffd640 10 client.17868 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa546ffd640 1 -- 172.21.15.94:0/3417265525 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ba3896800 con 0x560ba351c3a0 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa546ffd640 10 client.17868 handle_command_reply: tid=0 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 client.17868 shutdown 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 msgr2=0x560ba39ac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa5600097d0 tx=0x7fa56000fb20 comp rx=0 tx=0).stop 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 shutdown_connections 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ba351c3a0 0x560ba39b03b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa546ffd640 0 client.17868 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.672+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ba3588390 0x560ba39ac540 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ba35676b0 0x560ba3938bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 1 --2- 172.21.15.94:0/3417265525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ba355f4d0 0x560ba3938690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 >> 172.21.15.94:0/3417265525 conn(0x560ba357c4e0 msgr2=0x560ba3934560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 shutdown_connections 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 1 -- 172.21.15.94:0/3417265525 wait complete. 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa574fd7740 20 client.17868 trim_cache size 0 max 16384 2024-04-25T13:48:22.681 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:22.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 msgr2=0x7fa5500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 0x7fa5500b2000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560011bc0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 msgr2=0x7fa568199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e9f0 tx=0x7fa55800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 shutdown_connections 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5500afaf0 0x7fa5500b2000 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa568108920 0x7fa5681a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa568103c20 0x7fa568199950 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 --2- 172.21.15.94:0/2648472316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5681029e0 0x7fa568199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 >> 172.21.15.94:0/2648472316 conn(0x7fa5680fdbd0 msgr2=0x7fa568104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 shutdown_connections 2024-04-25T13:48:22.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.673+0000 7fa56e051640 1 -- 172.21.15.94:0/2648472316 wait complete. 2024-04-25T13:48:22.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:22 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1857934826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:22.685 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:48:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:22 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1857934826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 -- 172.21.15.94:0/1606422347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344103c20 msgr2=0x7f0344104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 --2- 172.21.15.94:0/1606422347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344103c20 0x7f0344104030 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402a410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 -- 172.21.15.94:0/1606422347 shutdown_connections 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 --2- 172.21.15.94:0/1606422347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344108920 0x7f034410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 --2- 172.21.15.94:0/1606422347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344103c20 0x7f0344104030 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 --2- 172.21.15.94:0/1606422347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f03441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 -- 172.21.15.94:0/1606422347 >> 172.21.15.94:0/1606422347 conn(0x7f03440fdbd0 msgr2=0x7f0344100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 -- 172.21.15.94:0/1606422347 shutdown_connections 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.798+0000 7f034b706640 1 -- 172.21.15.94:0/1606422347 wait complete. 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 Processor -- start 2024-04-25T13:48:22.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 -- start start 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c20 0x7f0344199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e510 con 0x7f0344108920 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e390 con 0x7f03441029e0 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f034b706640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e690 con 0x7f0344103c20 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f0349ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f03494a2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f0349ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44170/0 (socket says 172.21.15.94:44170) 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f03494a2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43978/0 (socket says 172.21.15.94:43978) 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f0349ca3640 1 -- 172.21.15.94:0/1186709218 learned_addr learned my addr 172.21.15.94:0/1186709218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.799+0000 7f0348ca1640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c20 0x7f0344199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 -- 172.21.15.94:0/1186709218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c20 msgr2=0x7f0344199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c20 0x7f0344199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 -- 172.21.15.94:0/1186709218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 msgr2=0x7f0344199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 -- 172.21.15.94:0/1186709218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03441a0f80 con 0x7f0344108920 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f03494a2640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f0349ca3640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f033c00eb10 tx=0x7f033c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f033a7fc640 1 -- 172.21.15.94:0/1186709218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c004b20 con 0x7f0344108920 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f033a7fc640 1 -- 172.21.15.94:0/1186709218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f033c004cc0 con 0x7f0344108920 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a1270 con 0x7f0344108920 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f033a7fc640 1 -- 172.21.15.94:0/1186709218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c010670 con 0x7f0344108920 2024-04-25T13:48:22.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.800+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0344069e90 con 0x7f0344108920 2024-04-25T13:48:22.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 752405 13.01 MB/sec execute 185 sec latency 17.306 ms 2024-04-25T13:48:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f033a7fc640 1 -- 172.21.15.94:0/1186709218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f033c005020 con 0x7f0344108920 2024-04-25T13:48:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 Processor -- start 2024-04-25T13:48:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 -- start start 2024-04-25T13:48:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db16810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d7536b0 0x55819db17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55819da7ad80 con 0x55819d774390 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55819da34a20 con 0x55819d7536b0 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03525bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55819da76d40 con 0x55819d74b4d0 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f0348ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db16810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47910/0 (socket says 172.21.15.94:47910) 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f0348ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db16810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44184/0 (socket says 172.21.15.94:44184) 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 -- 172.21.15.94:0/3550499708 learned_addr learned my addr 172.21.15.94:0/3550499708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f0348ca1640 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d7536b0 0x55819db17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f033a7fc640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 0x7f03300b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 -- 172.21.15.94:0/3550499708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d7536b0 msgr2=0x55819db17a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d7536b0 0x55819db17a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 -- 172.21.15.94:0/3550499708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 msgr2=0x55819db16810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db16810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f03494a2640 1 -- 172.21.15.94:0/3550499708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819da3fc30 con 0x55819d74b4d0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.803+0000 7f0349ca3640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 0x7f03300b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f033a7fc640 1 -- 172.21.15.94:0/1186709218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f033c0df230 con 0x7f0344108920 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f03494a2640 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f03400114f0 tx=0x7f03400119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee4272dae024bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f032f7fe640 1 -- 172.21.15.94:0/3550499708 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340011310 con 0x55819d74b4d0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f032f7fe640 1 -- 172.21.15.94:0/3550499708 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400054d0 con 0x55819d74b4d0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f032f7fe640 1 -- 172.21.15.94:0/3550499708 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000f7c0 con 0x55819d74b4d0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.804+0000 7f0349ca3640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 0x7f03300b4210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f033c0a7370 tx=0x7f033c0761c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- 172.21.15.94:0/3550499708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 msgr2=0x55819db16d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f03400114f0 tx=0x7f03400119b0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- 172.21.15.94:0/3550499708 shutdown_connections 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d7536b0 0x55819db17a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db16d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 --2- 172.21.15.94:0/3550499708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db16810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- 172.21.15.94:0/3550499708 >> 172.21.15.94:0/3550499708 conn(0x55819d7684e0 msgr2=0x55819db12020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- 172.21.15.94:0/3550499708 shutdown_connections 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- 172.21.15.94:0/3550499708 wait complete. 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 -- start start 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.805+0000 7f03525bc740 1 Processor -- start 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d774390 0x55819db27940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55819db18530 con 0x55819d7536b0 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55819da01dc0 con 0x55819d774390 2024-04-25T13:48:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55819db186b0 con 0x55819d74b4d0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03525bc740 10 client.0 _ll_register_callbacks cb 0x55819da1d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44194/0 (socket says 172.21.15.94:44194) 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 -- 172.21.15.94:0/729280505 learned_addr learned my addr 172.21.15.94:0/729280505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f0348ca1640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f0348ca1640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d774390 0x55819db27940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 -- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 msgr2=0x55819db204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 -- 172.21.15.94:0/729280505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d774390 msgr2=0x55819db27940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d774390 0x55819db27940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 -- 172.21.15.94:0/729280505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819d78c3f0 con 0x55819d7536b0 2024-04-25T13:48:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f03494a2640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0340009350 tx=0x7f03400048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c72798bad4be06e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.806+0000 7f032cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340029070 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0340005190 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400054d0 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0340035050 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 10 client.17880 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55819d8567d0 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400054d0 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03400194c0 con 0x55819d7536b0 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 10 client.17880 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 10 client.17880 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 20 client.17880 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d732f60 0x55819db2df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 4 client.17880 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55819da88990 con 0x55819d732f60 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.807+0000 7f0348ca1640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d732f60 0x55819db2df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.808+0000 7f0348ca1640 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d732f60 0x55819db2df60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0bddf684038d6c9 server_cookie=b9f00cc215edc1fe in_seq=0 out_seq=0 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.808+0000 7f032cff9640 10 client.17880 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.808+0000 7f032cff9640 1 -- 172.21.15.94:0/729280505 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55819da88990 con 0x55819d732f60 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.808+0000 7f032cff9640 10 client.17880 handle_command_reply: tid=0 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 client.17880 shutdown 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 msgr2=0x55819db209e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0340009350 tx=0x7f03400048d0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 shutdown_connections 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d732f60 0x55819db2df60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d774390 0x55819db27940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db209e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f032cff9640 0 client.17880 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 --2- 172.21.15.94:0/729280505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d74b4d0 0x55819db204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 >> 172.21.15.94:0/729280505 conn(0x55819d7684e0 msgr2=0x55819db14f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 shutdown_connections 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 1 -- 172.21.15.94:0/729280505 wait complete. 2024-04-25T13:48:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.809+0000 7f03525bc740 20 client.17880 trim_cache size 0 max 16384 2024-04-25T13:48:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 Processor -- start 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 -- start start 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d7536b0 0x55819db27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03441a93e0 con 0x55819d774390 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03441aaf20 con 0x55819d74b4d0 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03525bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03441a9ab0 con 0x55819d7536b0 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44206/0 (socket says 172.21.15.94:44206) 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 -- 172.21.15.94:0/2470966937 learned_addr learned my addr 172.21.15.94:0/2470966937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f03494a2640 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d7536b0 0x55819db27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 -- 172.21.15.94:0/2470966937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d7536b0 msgr2=0x55819db27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d7536b0 0x55819db27860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.823+0000 7f0348ca1640 1 -- 172.21.15.94:0/2470966937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 msgr2=0x55819db1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f0348ca1640 1 -- 172.21.15.94:0/2470966937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819db27da0 con 0x55819d774390 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0334032d30 tx=0x7f0334031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d2879d6c46e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f032effd640 1 -- 172.21.15.94:0/2470966937 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334035040 con 0x55819d774390 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f032effd640 1 -- 172.21.15.94:0/2470966937 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03340035f0 con 0x55819d774390 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.824+0000 7f032effd640 1 -- 172.21.15.94:0/2470966937 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033403d6b0 con 0x55819d774390 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- 172.21.15.94:0/2470966937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 msgr2=0x55819db11a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0334032d30 tx=0x7f0334031400 comp rx=0 tx=0).stop 2024-04-25T13:48:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- 172.21.15.94:0/2470966937 shutdown_connections 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d774390 0x55819db11a50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d7536b0 0x55819db27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 --2- 172.21.15.94:0/2470966937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- 172.21.15.94:0/2470966937 >> 172.21.15.94:0/2470966937 conn(0x55819d7684e0 msgr2=0x55819db1ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- 172.21.15.94:0/2470966937 shutdown_connections 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- 172.21.15.94:0/2470966937 wait complete. 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 -- start start 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.825+0000 7f03525bc740 1 Processor -- start 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db24b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d774390 0x55819db26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03441aa850 con 0x55819d7536b0 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03441aa180 con 0x55819d74b4d0 2024-04-25T13:48:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03441a73c0 con 0x55819d774390 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 10 client.0 _ll_register_callbacks cb 0x7f033c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44210/0 (socket says 172.21.15.94:44210) 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 -- 172.21.15.94:0/2167038954 learned_addr learned my addr 172.21.15.94:0/2167038954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db24b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d774390 0x55819db26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 -- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d774390 msgr2=0x55819db26000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d774390 0x55819db26000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 -- 172.21.15.94:0/2167038954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 msgr2=0x55819db24b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db24b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 -- 172.21.15.94:0/2167038954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819db265c0 con 0x55819d7536b0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03494a2640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f0340005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10d4db4bc9f7c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f032cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340029040 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0340004b40 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03340035f0 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0340019cc0 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 10 client.17886 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.826+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55819db27840 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03340035f0 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03400051e0 con 0x55819d7536b0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 10 client.17886 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 10 client.17886 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 20 client.17886 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d7083a0 0x55819db98750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 4 client.17886 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55819da82800 con 0x55819d7083a0 2024-04-25T13:48:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.827+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d7083a0 0x55819db98750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f032cff9640 10 client.17886 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f0348ca1640 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d7083a0 0x55819db98750 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=521f289aaeca26fe server_cookie=5318b5b12a1f73ff in_seq=0 out_seq=0 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f032cff9640 1 -- 172.21.15.94:0/2167038954 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55819da82800 con 0x55819d7083a0 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f032cff9640 10 client.17886 handle_command_reply: tid=0 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 client.17886 shutdown 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 msgr2=0x55819db250d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f0340005dc0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 shutdown_connections 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55819d7083a0 0x55819db98750 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55819d774390 0x55819db26000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f032cff9640 0 client.17886 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55819d7536b0 0x55819db250d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 --2- 172.21.15.94:0/2167038954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55819d74b4d0 0x55819db24b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 >> 172.21.15.94:0/2167038954 conn(0x55819d7684e0 msgr2=0x55819db118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 shutdown_connections 2024-04-25T13:48:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 1 -- 172.21.15.94:0/2167038954 wait complete. 2024-04-25T13:48:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.828+0000 7f03525bc740 20 client.17886 trim_cache size 0 max 16384 2024-04-25T13:48:22.835 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 msgr2=0x7f03300b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 0x7f03300b4210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f033c0a7370 tx=0x7f033c0761c0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 msgr2=0x7f03441a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f033c00eb10 tx=0x7f033c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 shutdown_connections 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f03300b1d00 0x7f03300b4210 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0344108920 0x7f03441a0910 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c20 0x7f0344199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 --2- 172.21.15.94:0/1186709218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03441029e0 0x7f0344199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 >> 172.21.15.94:0/1186709218 conn(0x7f03440fdbd0 msgr2=0x7f0344104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 shutdown_connections 2024-04-25T13:48:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.829+0000 7f034b706640 1 -- 172.21.15.94:0/1186709218 wait complete. 2024-04-25T13:48:22.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:48:22.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:22 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1857934826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 -- 172.21.15.94:0/1739143414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 msgr2=0x7f482c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 --2- 172.21.15.94:0/1739143414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1054b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02a410 comp rx=0 tx=0).stop 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 -- 172.21.15.94:0/1739143414 shutdown_connections 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 --2- 172.21.15.94:0/1739143414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 0x7f482c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 --2- 172.21.15.94:0/1739143414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 0x7f482c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 --2- 172.21.15.94:0/1739143414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1054b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 -- 172.21.15.94:0/1739143414 >> 172.21.15.94:0/1739143414 conn(0x7f482c0fd350 msgr2=0x7f482c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 -- 172.21.15.94:0/1739143414 shutdown_connections 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.952+0000 7f4833861640 1 -- 172.21.15.94:0/1739143414 wait complete. 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 Processor -- start 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 -- start start 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 0x7f482c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 0x7f482c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f610 con 0x7f482c1050a0 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f490 con 0x7f482c101750 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4833861640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f790 con 0x7f482c102250 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4831dfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4831dfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44232/0 (socket says 172.21.15.94:44232) 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4831dfe640 1 -- 172.21.15.94:0/2411166386 learned_addr learned my addr 172.21.15.94:0/2411166386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f48315fd640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 0x7f482c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.953+0000 7f4830dfc640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 0x7f482c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 -- 172.21.15.94:0/2411166386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 msgr2=0x7f482c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 0x7f482c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 -- 172.21.15.94:0/2411166386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 msgr2=0x7f482c19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 0x7f482c19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 -- 172.21.15.94:0/2411166386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c1a3f60 con 0x7f482c1050a0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4831dfe640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb10 tx=0x7f482400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f48227fc640 1 -- 172.21.15.94:0/2411166386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824004b20 con 0x7f482c1050a0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c1a4250 con 0x7f482c1050a0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f48227fc640 1 -- 172.21.15.94:0/2411166386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4824004cc0 con 0x7f482c1050a0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f48227fc640 1 -- 172.21.15.94:0/2411166386 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010690 con 0x7f482c1050a0 2024-04-25T13:48:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.954+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a47f0 con 0x7f482c1050a0 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.957+0000 7f483a7ec740 1 Processor -- start 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.957+0000 7f48227fc640 1 -- 172.21.15.94:0/2411166386 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4824005020 con 0x7f482c1050a0 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.957+0000 7f483a7ec740 1 -- start start 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 0x55b9d1596810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11d36b0 0x55b9d1597a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9d14fad80 con 0x55b9d11cb4d0 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9d14b4a20 con 0x55b9d11f4390 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f483a7ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9d14f6d40 con 0x55b9d11d36b0 2024-04-25T13:48:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44236/0 (socket says 172.21.15.94:44236) 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f4830dfc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 0x55b9d1596810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 -- 172.21.15.94:0/2367359749 learned_addr learned my addr 172.21.15.94:0/2367359749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f4830dfc640 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11d36b0 0x55b9d1597a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48227fc640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 0x7f48180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 -- 172.21.15.94:0/2367359749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11d36b0 msgr2=0x55b9d1597a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11d36b0 0x55b9d1597a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 -- 172.21.15.94:0/2367359749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 msgr2=0x55b9d1596810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 0x55b9d1596810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 -- 172.21.15.94:0/2367359749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d14bfc30 con 0x55b9d11cb4d0 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f4830dfc640 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 0x55b9d1596810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f4831dfe640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 0x7f48180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48315fd640 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f481c009230 tx=0x7f481c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cec6b0db721367f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48227fc640 1 -- 172.21.15.94:0/2411166386 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f48240df2c0 con 0x7f482c1050a0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.958+0000 7f48177fe640 1 -- 172.21.15.94:0/2367359749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c035030 con 0x55b9d11cb4d0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f48177fe640 1 -- 172.21.15.94:0/2367359749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c040070 con 0x55b9d11cb4d0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f48177fe640 1 -- 172.21.15.94:0/2367359749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0038e0 con 0x55b9d11cb4d0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 -- 172.21.15.94:0/2367359749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 msgr2=0x55b9d1596d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f4831dfe640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 0x7f48180b2000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48240a7690 tx=0x7f48240d5e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f481c009230 tx=0x7f481c009620 comp rx=0 tx=0).stop 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 -- 172.21.15.94:0/2367359749 shutdown_connections 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11d36b0 0x55b9d1597a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d1596d90 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 --2- 172.21.15.94:0/2367359749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11f4390 0x55b9d1596810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.959+0000 7f483a7ec740 1 -- 172.21.15.94:0/2367359749 >> 172.21.15.94:0/2367359749 conn(0x55b9d11e84e0 msgr2=0x55b9d1592020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- 172.21.15.94:0/2367359749 shutdown_connections 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- 172.21.15.94:0/2367359749 wait complete. 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- start start 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 Processor -- start 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9d15984d0 con 0x55b9d11f4390 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1481dc0 con 0x55b9d11d36b0 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1598650 con 0x55b9d11cb4d0 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f483a7ec740 10 client.0 _ll_register_callbacks cb 0x55b9d149d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.960+0000 7f4830dfc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44244/0 (socket says 172.21.15.94:44244) 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 -- 172.21.15.94:0/1797861766 learned_addr learned my addr 172.21.15.94:0/1797861766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f48315fd640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 -- 172.21.15.94:0/1797861766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 msgr2=0x55b9d15a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 -- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 msgr2=0x55b9d15a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 -- 172.21.15.94:0/1797861766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d120c3f0 con 0x55b9d11f4390 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4828012fb0 tx=0x7f4828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c838f017037769c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004030 con 0x55b9d11f4390 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c040070 con 0x55b9d11f4390 2024-04-25T13:48:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f48280041d0 con 0x55b9d11f4390 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f482802c050 con 0x55b9d11f4390 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f483a7ec740 10 client.17898 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9d12d67d0 con 0x55b9d11f4390 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.961+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c040070 con 0x55b9d11f4390 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f482800f4e0 con 0x55b9d11f4390 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 10 client.17898 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 10 client.17898 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 20 client.17898 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11b2f60 0x55b9d15ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 4 client.17898 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d1508990 con 0x55b9d11b2f60 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f48315fd640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11b2f60 0x55b9d15ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f4814ff9640 10 client.17898 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.962+0000 7f48315fd640 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11b2f60 0x55b9d15ae280 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a92162a279226526 server_cookie=fdb4761caecec7 in_seq=0 out_seq=0 2024-04-25T13:48:22.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f4814ff9640 1 -- 172.21.15.94:0/1797861766 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9d1508990 con 0x55b9d11b2f60 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f4814ff9640 10 client.17898 handle_command_reply: tid=0 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 client.17898 shutdown 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 msgr2=0x55b9d15a7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4828012fb0 tx=0x7f4828004730 comp rx=0 tx=0).stop 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 shutdown_connections 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11b2f60 0x55b9d15ae280 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11f4390 0x55b9d15a7a30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f4814ff9640 0 client.17898 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1797861766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 >> 172.21.15.94:0/1797861766 conn(0x55b9d11e84e0 msgr2=0x55b9d1594f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 shutdown_connections 2024-04-25T13:48:22.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 1 -- 172.21.15.94:0/1797861766 wait complete. 2024-04-25T13:48:22.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.963+0000 7f483a7ec740 20 client.17898 trim_cache size 0 max 16384 2024-04-25T13:48:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 Processor -- start 2024-04-25T13:48:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 -- start start 2024-04-25T13:48:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 0x55b9d1595530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1ad7a0 con 0x55b9d11cb4d0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1af2e0 con 0x55b9d11d36b0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f483a7ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1ade70 con 0x55b9d11b2f60 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f48315fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f48315fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44246/0 (socket says 172.21.15.94:44246) 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f48315fd640 1 -- 172.21.15.94:0/1551518536 learned_addr learned my addr 172.21.15.94:0/1551518536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 0x55b9d1595530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.978+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 -- 172.21.15.94:0/1551518536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 msgr2=0x55b9d1595530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 0x55b9d1595530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 -- 172.21.15.94:0/1551518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 msgr2=0x55b9d15b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 -- 172.21.15.94:0/1551518536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d15a1190 con 0x55b9d11cb4d0 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 0x55b9d1595530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f48315fd640 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x55b9d1596530 tx=0x7f481c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c594ce7642c16be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f4816ffd640 1 -- 172.21.15.94:0/1551518536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c035040 con 0x55b9d11cb4d0 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f4816ffd640 1 -- 172.21.15.94:0/1551518536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c040050 con 0x55b9d11cb4d0 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.979+0000 7f4816ffd640 1 -- 172.21.15.94:0/1551518536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c003e20 con 0x55b9d11cb4d0 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- 172.21.15.94:0/1551518536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 msgr2=0x55b9d15a79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x55b9d1596530 tx=0x7f481c005fd0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- 172.21.15.94:0/1551518536 shutdown_connections 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d15b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a79c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1551518536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11b2f60 0x55b9d1595530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- 172.21.15.94:0/1551518536 >> 172.21.15.94:0/1551518536 conn(0x55b9d11e84e0 msgr2=0x55b9d1594e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- 172.21.15.94:0/1551518536 shutdown_connections 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- 172.21.15.94:0/1551518536 wait complete. 2024-04-25T13:48:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 -- start start 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.980+0000 7f483a7ec740 1 Processor -- start 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d16149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1aec10 con 0x55b9d11b2f60 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1ae540 con 0x55b9d11d36b0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1ab780 con 0x55b9d11cb4d0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f483a7ec740 10 client.0 _ll_register_callbacks cb 0x7f48280124b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44248/0 (socket says 172.21.15.94:44248) 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 -- 172.21.15.94:0/1519069273 learned_addr learned my addr 172.21.15.94:0/1519069273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f48315fd640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d16149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 -- 172.21.15.94:0/1519069273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 msgr2=0x55b9d15a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 -- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 msgr2=0x55b9d16149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d16149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 -- 172.21.15.94:0/1519069273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d1614fe0 con 0x55b9d11b2f60 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f48315fd640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.981+0000 7f4830dfc640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b00 tx=0x7f4828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169be014ccb9ddea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828038070 con 0x55b9d11b2f60 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c040050 con 0x55b9d11b2f60 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f482800fc80 con 0x55b9d11b2f60 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4828004040 con 0x55b9d11b2f60 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 10 client.17904 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:22.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9d1616260 con 0x55b9d11b2f60 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c040050 con 0x55b9d11b2f60 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f482803b030 con 0x55b9d11b2f60 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 10 client.17904 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 10 client.17904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 20 client.17904 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11f4390 0x55b9d1618c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 4 client.17904 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9d1502800 con 0x55b9d11f4390 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.982+0000 7f48315fd640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11f4390 0x55b9d1618c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f4814ff9640 10 client.17904 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f48315fd640 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11f4390 0x55b9d1618c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcf4684070029747 server_cookie=fdc2c9d28798e4c7 in_seq=0 out_seq=0 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f4814ff9640 1 -- 172.21.15.94:0/1519069273 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9d1502800 con 0x55b9d11f4390 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f4814ff9640 10 client.17904 handle_command_reply: tid=0 2024-04-25T13:48:22.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 client.17904 shutdown 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 msgr2=0x55b9d15a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b00 tx=0x7f4828004730 comp rx=0 tx=0).stop 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 shutdown_connections 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9d11f4390 0x55b9d1618c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d11d36b0 0x55b9d16149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9d11cb4d0 0x55b9d15a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f4814ff9640 0 client.17904 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 --2- 172.21.15.94:0/1519069273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9d11b2f60 0x55b9d15a6fa0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 >> 172.21.15.94:0/1519069273 conn(0x55b9d11e84e0 msgr2=0x55b9d15b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 shutdown_connections 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 1 -- 172.21.15.94:0/1519069273 wait complete. 2024-04-25T13:48:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.983+0000 7f483a7ec740 20 client.17904 trim_cache size 0 max 16384 2024-04-25T13:48:22.995 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 msgr2=0x7f48180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 0x7f48180b2000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48240a7690 tx=0x7f48240d5e50 comp rx=0 tx=0).stop 2024-04-25T13:48:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 msgr2=0x7f482c1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb10 tx=0x7f482400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 shutdown_connections 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f48180afaf0 0x7f48180b2000 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c1050a0 0x7f482c1a38f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f482c102250 0x7f482c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 --2- 172.21.15.94:0/2411166386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f482c101750 0x7f482c19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 >> 172.21.15.94:0/2411166386 conn(0x7f482c0fd350 msgr2=0x7f482c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 shutdown_connections 2024-04-25T13:48:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:22.984+0000 7f4833861640 1 -- 172.21.15.94:0/2411166386 wait complete. 2024-04-25T13:48:22.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 -- 172.21.15.94:0/685757380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 msgr2=0x7ff4f0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/685757380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f0104030 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009900 tx=0x7ff4e002a410 comp rx=0 tx=0).stop 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 -- 172.21.15.94:0/685757380 shutdown_connections 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/685757380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f0108920 0x7ff4f010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/685757380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f0104030 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/685757380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f01029e0 0x7ff4f01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 -- 172.21.15.94:0/685757380 >> 172.21.15.94:0/685757380 conn(0x7ff4f00fdbd0 msgr2=0x7ff4f0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 -- 172.21.15.94:0/685757380 shutdown_connections 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.110+0000 7ff4f6b14640 1 -- 172.21.15.94:0/685757380 wait complete. 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 Processor -- start 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 -- start start 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f01029e0 0x7ff4f019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f0108920 0x7ff4f01a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010e510 con 0x7ff4f0103c20 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010e390 con 0x7ff4f0108920 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f6b14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010e690 con 0x7ff4f01029e0 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44262/0 (socket says 172.21.15.94:44262) 2024-04-25T13:48:23.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f48b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f01029e0 0x7ff4f019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 -- 172.21.15.94:0/177601212 learned_addr learned my addr 172.21.15.94:0/177601212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f0108920 0x7ff4f01a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 -- 172.21.15.94:0/177601212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f01029e0 msgr2=0x7ff4f019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f01029e0 0x7ff4f019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 -- 172.21.15.94:0/177601212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f0108920 msgr2=0x7ff4f01a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f0108920 0x7ff4f01a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 -- 172.21.15.94:0/177601212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01a20c0 con 0x7ff4f0103c20 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.111+0000 7ff4effff640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0006010 tx=0x7ff4e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.112+0000 7ff4edffb640 1 -- 172.21.15.94:0/177601212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e003f030 con 0x7ff4f0103c20 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.112+0000 7ff4edffb640 1 -- 172.21.15.94:0/177601212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e0003aa0 con 0x7ff4f0103c20 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.112+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f01a2350 con 0x7ff4f0103c20 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.112+0000 7ff4edffb640 1 -- 172.21.15.94:0/177601212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0035070 con 0x7ff4f0103c20 2024-04-25T13:48:23.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.112+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01a2840 con 0x7ff4f0103c20 2024-04-25T13:48:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 Processor -- start 2024-04-25T13:48:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 -- start start 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4edffb640 1 -- 172.21.15.94:0/177601212 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff4e0042030 con 0x7ff4f0103c20 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09dd9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09ddaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc09d3dd80 con 0x55fc09a0e4d0 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc09cf7a20 con 0x55fc09a166b0 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.114+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc09d39d40 con 0x55fc09a37390 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f48b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f48b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44272/0 (socket says 172.21.15.94:44272) 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f48b0640 1 -- 172.21.15.94:0/708109427 learned_addr learned my addr 172.21.15.94:0/708109427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09ddaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09dd9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 -- 172.21.15.94:0/708109427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 msgr2=0x55fc09dd9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09dd9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 -- 172.21.15.94:0/708109427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 msgr2=0x55fc09dd9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 -- 172.21.15.94:0/708109427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc09d02c30 con 0x55fc09a166b0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4edffb640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 0x7ff4dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 0x7ff4dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4effff640 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09ddaa20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00fcc60 tx=0x7ff4e0107540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa195f9d47470eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4edffb640 1 -- 172.21.15.94:0/177601212 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff4e00ff230 con 0x7ff4f0103c20 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.115+0000 7ff4e6ffd640 1 -- 172.21.15.94:0/708109427 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0049020 con 0x55fc09a166b0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4e6ffd640 1 -- 172.21.15.94:0/708109427 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e003b460 con 0x55fc09a166b0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4e6ffd640 1 -- 172.21.15.94:0/708109427 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00c35e0 con 0x55fc09a166b0 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/708109427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 msgr2=0x55fc09ddaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 0x7ff4dc0b2000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fc09dda860 tx=0x7ff4d8005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09ddaa20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00fcc60 tx=0x7ff4e0107540 comp rx=0 tx=0).stop 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/708109427 shutdown_connections 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09ddaa20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/708109427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09dd9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/708109427 >> 172.21.15.94:0/708109427 conn(0x55fc09a2b4e0 msgr2=0x55fc09dd5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/708109427 shutdown_connections 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.116+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/708109427 wait complete. 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 -- start start 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 Processor -- start 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 0x55fc09de3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a37390 0x55fc09dea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fc09ddb2f0 con 0x55fc09a166b0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc09cc4dc0 con 0x55fc09a37390 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc09ddb470 con 0x55fc09a0e4d0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4fd9ca740 10 client.0 _ll_register_callbacks cb 0x55fc09ce0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44278/0 (socket says 172.21.15.94:44278) 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4effff640 1 -- 172.21.15.94:0/2585663573 learned_addr learned my addr 172.21.15.94:0/2585663573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 0x55fc09de3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.117+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a37390 0x55fc09dea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 -- 172.21.15.94:0/2585663573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 msgr2=0x55fc09de3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 0x55fc09de3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 -- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a37390 msgr2=0x55fc09dea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a37390 0x55fc09dea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 -- 172.21.15.94:0/2585663573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc09a4f3f0 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 0x55fc09de3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4effff640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00fd000 tx=0x7ff4e0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52505fd96c495c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0049070 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff4e00b00d0 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc09d4b990 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4e00c6e90 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4fd9ca740 10 client.17916 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc09b197d0 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.118+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fc09d4b990 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4e00c7980 con 0x55fc09a166b0 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 10 client.17916 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:23.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 10 client.17916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 20 client.17916 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099f5f60 0x55fc09df0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 4 client.17916 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc09d45800 con 0x55fc099f5f60 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099f5f60 0x55fc09df0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4cffff640 10 client.17916 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.119+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099f5f60 0x55fc09df0d00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6a5c51c3fd14ea server_cookie=4775dee5f923707 in_seq=0 out_seq=0 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4cffff640 1 -- 172.21.15.94:0/2585663573 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc09d45800 con 0x55fc099f5f60 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4cffff640 10 client.17916 handle_command_reply: tid=0 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 client.17916 shutdown 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 msgr2=0x55fc09de3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00fd000 tx=0x7ff4e0005ef0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 shutdown_connections 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099f5f60 0x55fc09df0d00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a37390 0x55fc09dea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a166b0 0x55fc09de3750 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/2585663573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a0e4d0 0x55fc09de3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4cffff640 0 client.17916 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 >> 172.21.15.94:0/2585663573 conn(0x55fc09a2b4e0 msgr2=0x55fc09dd4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 shutdown_connections 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/2585663573 wait complete. 2024-04-25T13:48:23.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.120+0000 7ff4fd9ca740 20 client.17916 trim_cache size 0 max 16384 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.134+0000 7ff4fd9ca740 1 Processor -- start 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.134+0000 7ff4fd9ca740 1 -- start start 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09dea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09de1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ab540 con 0x55fc09a0e4d0 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ad080 con 0x55fc09a166b0 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01abc10 con 0x55fc09a37390 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44286/0 (socket says 172.21.15.94:44286) 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 -- 172.21.15.94:0/1403198404 learned_addr learned my addr 172.21.15.94:0/1403198404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4effff640 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09dea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09de1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 -- 172.21.15.94:0/1403198404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 msgr2=0x55fc09de1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09de1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 -- 172.21.15.94:0/1403198404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 msgr2=0x55fc09dea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09dea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 -- 172.21.15.94:0/1403198404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc09deab10 con 0x55fc09a0e4d0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x55fc09dd5db0 tx=0x7ff4e8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5f068b71d20b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4e67fc640 1 -- 172.21.15.94:0/1403198404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004510 con 0x55fc09a0e4d0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4e67fc640 1 -- 172.21.15.94:0/1403198404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e8024020 con 0x55fc09a0e4d0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.135+0000 7ff4e67fc640 1 -- 172.21.15.94:0/1403198404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e801d650 con 0x55fc09a0e4d0 2024-04-25T13:48:23.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1403198404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 msgr2=0x55fc09dd0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x55fc09dd5db0 tx=0x7ff4e8009500 comp rx=0 tx=0).stop 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1403198404 shutdown_connections 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a37390 0x55fc09de1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a166b0 0x55fc09dea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1403198404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a0e4d0 0x55fc09dd0190 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1403198404 >> 172.21.15.94:0/1403198404 conn(0x55fc09a2b4e0 msgr2=0x55fc09dde0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1403198404 shutdown_connections 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1403198404 wait complete. 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 -- start start 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.136+0000 7ff4fd9ca740 1 Processor -- start 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a0e4d0 0x55fc09dea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a166b0 0x55fc09de6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ac9b0 con 0x55fc09a37390 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ac2e0 con 0x55fc09a0e4d0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01a9520 con 0x55fc09a166b0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4fd9ca740 10 client.0 _ll_register_callbacks cb 0x7ff4e8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44294/0 (socket says 172.21.15.94:44294) 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 -- 172.21.15.94:0/1340769538 learned_addr learned my addr 172.21.15.94:0/1340769538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4effff640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a166b0 0x55fc09de6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a0e4d0 0x55fc09dea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 -- 172.21.15.94:0/1340769538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a166b0 msgr2=0x55fc09de6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a166b0 0x55fc09de6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 -- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a0e4d0 msgr2=0x55fc09dea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:23.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a0e4d0 0x55fc09dea2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 -- 172.21.15.94:0/1340769538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc09de7930 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4f50b1640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d801bc00 tx=0x7ff4d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f24834332f8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.137+0000 7ff4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800f920 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff4d800fac0 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e8024020 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4d8037050 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 10 client.17922 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc09e580f0 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e8024020 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d80303e0 con 0x55fc09a37390 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 10 client.17922 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:23.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 20 client.17922 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099cb3a0 0x55fc09e5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 4 client.17922 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.138+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc09cc0610 con 0x55fc099cb3a0 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.139+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099cb3a0 0x55fc09e5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.139+0000 7ff4cffff640 10 client.17922 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:23.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.139+0000 7ff4f48b0640 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099cb3a0 0x55fc09e5ab90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1372d3c761bcbf5 server_cookie=d5284c46bcb1cca9 in_seq=0 out_seq=0 2024-04-25T13:48:23.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4cffff640 1 -- 172.21.15.94:0/1340769538 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc09cc0610 con 0x55fc099cb3a0 2024-04-25T13:48:23.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4cffff640 10 client.17922 handle_command_reply: tid=0 2024-04-25T13:48:23.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 client.17922 shutdown 2024-04-25T13:48:23.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 msgr2=0x55fc09de7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d801bc00 tx=0x7ff4d8004730 comp rx=0 tx=0).stop 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 shutdown_connections 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fc099cb3a0 0x55fc09e5ab90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fc09a37390 0x55fc09de7370 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09a166b0 0x55fc09de6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 --2- 172.21.15.94:0/1340769538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc09a0e4d0 0x55fc09dea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4cffff640 0 client.17922 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 >> 172.21.15.94:0/1340769538 conn(0x55fc09a2b4e0 msgr2=0x55fc09de17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 shutdown_connections 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 1 -- 172.21.15.94:0/1340769538 wait complete. 2024-04-25T13:48:23.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.141+0000 7ff4fd9ca740 20 client.17922 trim_cache size 0 max 16384 2024-04-25T13:48:23.147 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:23.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 msgr2=0x7ff4dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 0x7ff4dc0b2000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55fc09dda860 tx=0x7ff4d8005f60 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 msgr2=0x7ff4f019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0006010 tx=0x7ff4e0031a50 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 shutdown_connections 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff4dc0afaf0 0x7ff4dc0b2000 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4f0108920 0x7ff4f01a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4f0103c20 0x7ff4f019aac0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 --2- 172.21.15.94:0/177601212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f01029e0 0x7ff4f019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 >> 172.21.15.94:0/177601212 conn(0x7ff4f00fdbd0 msgr2=0x7ff4f0109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 shutdown_connections 2024-04-25T13:48:23.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:23.142+0000 7ff4f6b14640 1 -- 172.21.15.94:0/177601212 wait complete. 2024-04-25T13:48:23.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:23 smithi094 ceph-mon[29195]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-04-25T13:48:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:23 smithi138 ceph-mon[32977]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-04-25T13:48:23.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 754221 12.99 MB/sec execute 186 sec latency 14.375 ms 2024-04-25T13:48:23.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:23 smithi137 ceph-mon[32568]: pgmap v545: 97 pgs: 97 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-04-25T13:48:24.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:48:24.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:24.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 -- 172.21.15.94:0/4054169854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d70103c20 msgr2=0x7f8d70104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 --2- 172.21.15.94:0/4054169854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d70103c20 0x7f8d70104030 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009900 tx=0x7f8d6002a410 comp rx=0 tx=0).stop 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 -- 172.21.15.94:0/4054169854 shutdown_connections 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 --2- 172.21.15.94:0/4054169854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70108920 0x7f8d7010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 --2- 172.21.15.94:0/4054169854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d70103c20 0x7f8d70104030 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 --2- 172.21.15.94:0/4054169854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d701029e0 0x7f8d701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.272+0000 7f8d77e67640 1 -- 172.21.15.94:0/4054169854 >> 172.21.15.94:0/4054169854 conn(0x7f8d700fdbd0 msgr2=0x7f8d70100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.273+0000 7f8d77e67640 1 -- 172.21.15.94:0/4054169854 shutdown_connections 2024-04-25T13:48:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.273+0000 7f8d77e67640 1 -- 172.21.15.94:0/4054169854 wait complete. 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.273+0000 7f8d77e67640 1 Processor -- start 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.273+0000 7f8d77e67640 1 -- start start 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 0x7f8d7019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7010e560 con 0x7f8d701029e0 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7010e3e0 con 0x7f8d70108920 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d77e67640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7010e6e0 con 0x7f8d70103c20 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d76404640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36848/0 (socket says 172.21.15.94:36848) 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d76404640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35042/0 (socket says 172.21.15.94:35042) 2024-04-25T13:48:24.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 -- 172.21.15.94:0/3164128874 learned_addr learned my addr 172.21.15.94:0/3164128874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75402640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 0x7f8d7019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 -- 172.21.15.94:0/3164128874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 msgr2=0x7f8d7019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 0x7f8d7019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 -- 172.21.15.94:0/3164128874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 msgr2=0x7f8d701a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 -- 172.21.15.94:0/3164128874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d701a2070 con 0x7f8d701029e0 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d76404640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75402640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 0x7f8d7019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.274+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9f0 tx=0x7f8d6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.275+0000 7f8d66ffd640 1 -- 172.21.15.94:0/3164128874 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00ccc0 con 0x7f8d701029e0 2024-04-25T13:48:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.275+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d701a2360 con 0x7f8d701029e0 2024-04-25T13:48:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.275+0000 7f8d66ffd640 1 -- 172.21.15.94:0/3164128874 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d6c0035f0 con 0x7f8d701029e0 2024-04-25T13:48:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.275+0000 7f8d66ffd640 1 -- 172.21.15.94:0/3164128874 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00ce60 con 0x7f8d701029e0 2024-04-25T13:48:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.275+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d701a2900 con 0x7f8d701029e0 2024-04-25T13:48:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.277+0000 7f8d66ffd640 1 -- 172.21.15.94:0/3164128874 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8d6c003aa0 con 0x7f8d701029e0 2024-04-25T13:48:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 Processor -- start 2024-04-25T13:48:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 -- start start 2024-04-25T13:48:24.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12129730 0x55fe124f2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe12450b00 con 0x55fe12121550 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe123ec920 con 0x55fe12129730 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe1244cac0 con 0x55fe1214a410 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d76404640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d75c03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d76404640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35664/0 (socket says 172.21.15.94:35664) 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d76404640 1 -- 172.21.15.94:0/2310645070 learned_addr learned my addr 172.21.15.94:0/2310645070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.278+0000 7f8d75402640 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12129730 0x55fe124f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 -- 172.21.15.94:0/2310645070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12129730 msgr2=0x55fe124f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12129730 0x55fe124f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 -- 172.21.15.94:0/2310645070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 msgr2=0x55fe124eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d66ffd640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 0x7f8d5c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 -- 172.21.15.94:0/2310645070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe1222a7e0 con 0x55fe1214a410 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d66ffd640 1 -- 172.21.15.94:0/3164128874 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8d6c0df070 con 0x7f8d701029e0 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 0x7f8d5c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d76404640 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d680117e0 tx=0x7f8d68011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6889ca093a38b513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d5bfff640 1 -- 172.21.15.94:0/2310645070 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68010570 con 0x55fe1214a410 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d5bfff640 1 -- 172.21.15.94:0/2310645070 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68010bc0 con 0x55fe1214a410 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.279+0000 7f8d5bfff640 1 -- 172.21.15.94:0/2310645070 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801ca80 con 0x55fe1214a410 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2310645070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 msgr2=0x55fe124eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d76404640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 0x7f8d5c0b2000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d68011490 tx=0x7f8d68021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8d680117e0 tx=0x7f8d68011ca0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2310645070 shutdown_connections 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12129730 0x55fe124f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2310645070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe1214a410 0x55fe124eb4a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2310645070 >> 172.21.15.94:0/2310645070 conn(0x55fe1213e560 msgr2=0x55fe124e6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2310645070 shutdown_connections 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.280+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2310645070 wait complete. 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 -- start start 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 Processor -- start 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 0x55fe124f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe124f3370 con 0x55fe1214a410 2024-04-25T13:48:24.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe123ea5d0 con 0x55fe12121550 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe124f34f0 con 0x55fe12129730 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d7edf9740 10 client.0 _ll_register_callbacks cb 0x55fe123d8080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36864/0 (socket says 172.21.15.94:36864) 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d75c03640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 0x55fe124f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 -- 172.21.15.94:0/2416363732 learned_addr learned my addr 172.21.15.94:0/2416363732 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d75402640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 -- 172.21.15.94:0/2416363732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 msgr2=0x55fe124f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 -- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 msgr2=0x55fe124f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.281+0000 7f8d76404640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 0x55fe124f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d76404640 1 -- 172.21.15.94:0/2416363732 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1228af70 con 0x55fe1214a410 2024-04-25T13:48:24.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d75c03640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 0x55fe124f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d76404640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009fd0 tx=0x7f8d6801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dc615161812f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004a00 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8d6802e070 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d68010bc0 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d68004ba0 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 10 client.17934 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe12122170 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68010bc0 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d68005200 con 0x55fe1214a410 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 10 client.17934 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 20 client.17934 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:24.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe121091f0 0x55fe12502820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 4 client.17934 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:24.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.282+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe1245e710 con 0x55fe121091f0 2024-04-25T13:48:24.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.283+0000 7f8d75c03640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe121091f0 0x55fe12502820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.283+0000 7f8d597fa640 10 client.17934 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.283+0000 7f8d75c03640 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe121091f0 0x55fe12502820 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d67f4fdd26c83e4 server_cookie=edcfd51398205835 in_seq=0 out_seq=0 2024-04-25T13:48:24.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d597fa640 1 -- 172.21.15.94:0/2416363732 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe1245e710 con 0x55fe121091f0 2024-04-25T13:48:24.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d597fa640 10 client.17934 handle_command_reply: tid=0 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 client.17934 shutdown 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 msgr2=0x55fe124fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009fd0 tx=0x7f8d6801cbe0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 shutdown_connections 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe121091f0 0x55fe12502820 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe1214a410 0x55fe124fc550 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d597fa640 0 client.17934 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/2416363732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe12121550 0x55fe124f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 >> 172.21.15.94:0/2416363732 conn(0x55fe1213e560 msgr2=0x55fe124e5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 shutdown_connections 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 1 -- 172.21.15.94:0/2416363732 wait complete. 2024-04-25T13:48:24.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.285+0000 7f8d7edf9740 20 client.17934 trim_cache size 0 max 16384 2024-04-25T13:48:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 Processor -- start 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 -- start start 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe12506fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12121550 0x55fe124fc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ab580 con 0x55fe12129730 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ad0c0 con 0x55fe121091f0 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701abc50 con 0x55fe12121550 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d76404640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d76404640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36870/0 (socket says 172.21.15.94:36870) 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.305+0000 7f8d76404640 1 -- 172.21.15.94:0/266880057 learned_addr learned my addr 172.21.15.94:0/266880057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d75402640 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12121550 0x55fe124fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d75c03640 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe12506fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 -- 172.21.15.94:0/266880057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12121550 msgr2=0x55fe124fc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12121550 0x55fe124fc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 -- 172.21.15.94:0/266880057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 msgr2=0x55fe12506fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe12506fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 -- 172.21.15.94:0/266880057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe124ec290 con 0x55fe12129730 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d76404640 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8d68002230 tx=0x7f8d68009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf94abd0750acc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d5b7fe640 1 -- 172.21.15.94:0/266880057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004a00 con 0x55fe12129730 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d5b7fe640 1 -- 172.21.15.94:0/266880057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68036070 con 0x55fe12129730 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.306+0000 7f8d5b7fe640 1 -- 172.21.15.94:0/266880057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68010a20 con 0x55fe12129730 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- 172.21.15.94:0/266880057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 msgr2=0x55fe124fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8d68002230 tx=0x7f8d68009ee0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- 172.21.15.94:0/266880057 shutdown_connections 2024-04-25T13:48:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12129730 0x55fe124fc9b0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12121550 0x55fe124fc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/266880057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe12506fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- 172.21.15.94:0/266880057 >> 172.21.15.94:0/266880057 conn(0x55fe1213e560 msgr2=0x55fe124ef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- 172.21.15.94:0/266880057 shutdown_connections 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- 172.21.15.94:0/266880057 wait complete. 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 -- start start 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.307+0000 7f8d7edf9740 1 Processor -- start 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe124f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ac9f0 con 0x55fe12121550 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ac320 con 0x55fe121091f0 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701a9560 con 0x55fe12129730 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d7edf9740 10 client.0 _ll_register_callbacks cb 0x7f8d6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:24.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36880/0 (socket says 172.21.15.94:36880) 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 -- 172.21.15.94:0/3653532220 learned_addr learned my addr 172.21.15.94:0/3653532220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe124f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d76404640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 -- 172.21.15.94:0/3653532220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 msgr2=0x55fe124fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 -- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 msgr2=0x55fe124f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe124f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 -- 172.21.15.94:0/3653532220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe124fb900 con 0x55fe12121550 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d75402640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d60031500 tx=0x7f8d60005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a11009a3c52ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.308+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60035040 con 0x55fe12121550 2024-04-25T13:48:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8d600048d0 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d68036070 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d60004ad0 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 10 client.17940 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe124fcc60 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68036070 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d600315a0 con 0x55fe12121550 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 10 client.17940 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 20 client.17940 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe1214a410 0x55fe1256d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 4 client.17940 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:48:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe12458580 con 0x55fe1214a410 2024-04-25T13:48:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.309+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe1214a410 0x55fe1256d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.310+0000 7f8d597fa640 10 client.17940 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.310+0000 7f8d75c03640 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe1214a410 0x55fe1256d060 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfb6afb486d67a5 server_cookie=20e88e4904092128 in_seq=0 out_seq=0 2024-04-25T13:48:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.311+0000 7f8d597fa640 1 -- 172.21.15.94:0/3653532220 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe12458580 con 0x55fe1214a410 2024-04-25T13:48:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.311+0000 7f8d597fa640 10 client.17940 handle_command_reply: tid=0 2024-04-25T13:48:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.311+0000 7f8d7edf9740 1 client.17940 shutdown 2024-04-25T13:48:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.311+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 msgr2=0x55fe124fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.311+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d60031500 tx=0x7f8d60005440 comp rx=0 tx=0).stop 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 shutdown_connections 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fe1214a410 0x55fe1256d060 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe12129730 0x55fe124fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe12121550 0x55fe124fa480 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d597fa640 0 client.17940 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 --2- 172.21.15.94:0/3653532220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe121091f0 0x55fe124f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 >> 172.21.15.94:0/3653532220 conn(0x55fe1213e560 msgr2=0x55fe124e9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 shutdown_connections 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 1 -- 172.21.15.94:0/3653532220 wait complete. 2024-04-25T13:48:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d7edf9740 20 client.17940 trim_cache size 0 max 16384 2024-04-25T13:48:24.319 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:48:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ca6a8d4a-ab95-4494-805e-352169abbcd3", 2024-04-25T13:48:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:48:24.319 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 msgr2=0x7f8d5c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 0x7f8d5c0b2000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d68011490 tx=0x7f8d68021000 comp rx=0 tx=0).stop 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 msgr2=0x7f8d7019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9f0 tx=0x7f8d6c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 shutdown_connections 2024-04-25T13:48:24.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d5c0afaf0 0x7f8d5c0b2000 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d70108920 0x7f8d701a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d70103c20 0x7f8d7019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 --2- 172.21.15.94:0/3164128874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d701029e0 0x7f8d7019a5a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 >> 172.21.15.94:0/3164128874 conn(0x7f8d700fdbd0 msgr2=0x7f8d70109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 shutdown_connections 2024-04-25T13:48:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.312+0000 7f8d77e67640 1 -- 172.21.15.94:0/3164128874 wait complete. 2024-04-25T13:48:24.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:48:24.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:48:24.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 -- 172.21.15.94:0/874037116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8103c20 msgr2=0x7fb1c8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 --2- 172.21.15.94:0/874037116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8103c20 0x7fb1c8104030 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 -- 172.21.15.94:0/874037116 shutdown_connections 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 --2- 172.21.15.94:0/874037116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8108920 0x7fb1c810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 --2- 172.21.15.94:0/874037116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8103c20 0x7fb1c8104030 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 --2- 172.21.15.94:0/874037116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 -- 172.21.15.94:0/874037116 >> 172.21.15.94:0/874037116 conn(0x7fb1c80fdbd0 msgr2=0x7fb1c8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 -- 172.21.15.94:0/874037116 shutdown_connections 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.438+0000 7fb1cf696640 1 -- 172.21.15.94:0/874037116 wait complete. 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 Processor -- start 2024-04-25T13:48:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 -- start start 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 0x7fb1c81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810e500 con 0x7fb1c8108920 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810e380 con 0x7fb1c8103c20 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cf696640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810e680 con 0x7fb1c81029e0 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cd432640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36900/0 (socket says 172.21.15.94:36900) 2024-04-25T13:48:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cd432640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35700/0 (socket says 172.21.15.94:35700) 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 -- 172.21.15.94:0/2436077518 learned_addr learned my addr 172.21.15.94:0/2436077518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 0x7fb1c81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 -- 172.21.15.94:0/2436077518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 msgr2=0x7fb1c8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 -- 172.21.15.94:0/2436077518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 msgr2=0x7fb1c81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 0x7fb1c81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 -- 172.21.15.94:0/2436077518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81a0fa0 con 0x7fb1c8108920 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cd432640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 0x7fb1c81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.439+0000 7fb1cdc33640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000eb10 tx=0x7fb1c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.440+0000 7fb1be7fc640 1 -- 172.21.15.94:0/2436077518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004b20 con 0x7fb1c8108920 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.440+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c81a1290 con 0x7fb1c8108920 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.440+0000 7fb1be7fc640 1 -- 172.21.15.94:0/2436077518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c0004cc0 con 0x7fb1c8108920 2024-04-25T13:48:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.440+0000 7fb1be7fc640 1 -- 172.21.15.94:0/2436077518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0010670 con 0x7fb1c8108920 2024-04-25T13:48:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.440+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81a17e0 con 0x7fb1c8108920 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.442+0000 7fb1be7fc640 1 -- 172.21.15.94:0/2436077518 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb1c0005020 con 0x7fb1c8108920 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.442+0000 7fb1d6624740 1 Processor -- start 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.442+0000 7fb1d6624740 1 -- start start 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 0x564cbbb73a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564cbbad8d80 con 0x564cbb7d2390 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564cbba928b0 con 0x564cbb7a94d0 2024-04-25T13:48:24.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1d6624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564cbbad4d40 con 0x564cbb7b16b0 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 0x564cbbb73a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 0x564cbbb73a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36902/0 (socket says 172.21.15.94:36902) 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1ccc31640 1 -- 172.21.15.94:0/3363814761 learned_addr learned my addr 172.21.15.94:0/3363814761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 -- 172.21.15.94:0/3363814761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 msgr2=0x564cbbb74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1be7fc640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 0x7fb1b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 -- 172.21.15.94:0/3363814761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 msgr2=0x564cbbb73a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 0x564cbbb73a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 -- 172.21.15.94:0/3363814761 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbba63a40 con 0x564cbb7a94d0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cdc33640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 0x7fb1b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1cd432640 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb73fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1c81035e0 tx=0x7fb1c4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=981c7b7d8b6f5259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.443+0000 7fb1be7fc640 1 -- 172.21.15.94:0/2436077518 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb1c00df280 con 0x7fb1c8108920 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.444+0000 7fb1b37fe640 1 -- 172.21.15.94:0/3363814761 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c401c680 con 0x564cbb7a94d0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.444+0000 7fb1b37fe640 1 -- 172.21.15.94:0/3363814761 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c401ccd0 con 0x564cbb7a94d0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.444+0000 7fb1b37fe640 1 -- 172.21.15.94:0/3363814761 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4011dd0 con 0x564cbb7a94d0 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- 172.21.15.94:0/3363814761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 msgr2=0x564cbbb73fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1cdc33640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 0x7fb1b40b2000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00dd050 tx=0x7fb1c0076040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb73fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1c81035e0 tx=0x7fb1c4010860 comp rx=0 tx=0).stop 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- 172.21.15.94:0/3363814761 shutdown_connections 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb73fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- 172.21.15.94:0/3363814761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7d2390 0x564cbbb73a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- 172.21.15.94:0/3363814761 >> 172.21.15.94:0/3363814761 conn(0x564cbb7c64e0 msgr2=0x564cbbb6f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- 172.21.15.94:0/3363814761 shutdown_connections 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- 172.21.15.94:0/3363814761 wait complete. 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- start start 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 Processor -- start 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 0x564cbbb7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb84cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.445+0000 7fb1d6624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564cbbb75780 con 0x564cbb7b16b0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1d6624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564cbba5fd10 con 0x564cbb7a94d0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1d6624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564cbbb75900 con 0x564cbb7d2390 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1d6624740 10 client.0 _ll_register_callbacks cb 0x564cbba8c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 0x564cbbb7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35086/0 (socket says 172.21.15.94:35086) 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1ccc31640 1 -- 172.21.15.94:0/1237891 learned_addr learned my addr 172.21.15.94:0/1237891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb84cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 -- 172.21.15.94:0/1237891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 msgr2=0x564cbbb84cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb84cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 -- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 msgr2=0x564cbbb7d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 -- 172.21.15.94:0/1237891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbb8b47d0 con 0x564cbb7b16b0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1cd432640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 0x564cbbb7dd50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4006010 tx=0x7fb1c400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c0b71ccf6a8a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c402b060 con 0x564cbb7b16b0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb1c4011d60 con 0x564cbb7b16b0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.446+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c401ccd0 con 0x564cbb7b16b0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1c4005040 con 0x564cbb7b16b0 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 10 client.17952 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:24.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cbb7ea3f0 con 0x564cbb7b16b0 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c401ccd0 con 0x564cbb7b16b0 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb1c40046c0 con 0x564cbb7b16b0 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 10 client.17952 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 20 client.17952 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb790f60 0x564cbbb8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 4 client.17952 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cbbae6990 con 0x564cbb790f60 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.447+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb790f60 0x564cbbb8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.448+0000 7fb1b0ff9640 10 client.17952 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.448+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb790f60 0x564cbbb8ae80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48af71d2ca301f2 server_cookie=347df85c228197bc in_seq=0 out_seq=0 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/1237891 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564cbbae6990 con 0x564cbb790f60 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1b0ff9640 10 client.17952 handle_command_reply: tid=0 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 client.17952 shutdown 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 msgr2=0x564cbbb7dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 0x564cbbb7dd50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4006010 tx=0x7fb1c400bd80 comp rx=0 tx=0).stop 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 shutdown_connections 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb790f60 0x564cbbb8ae80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb84cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7b16b0 0x564cbbb7dd50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1b0ff9640 0 client.17952 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1237891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7a94d0 0x564cbbb7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.449+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 >> 172.21.15.94:0/1237891 conn(0x564cbb7c64e0 msgr2=0x564cbbb72180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.450+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 shutdown_connections 2024-04-25T13:48:24.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.450+0000 7fb1d6624740 1 -- 172.21.15.94:0/1237891 wait complete. 2024-04-25T13:48:24.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.450+0000 7fb1d6624740 20 client.17952 trim_cache size 0 max 16384 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 Processor -- start 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 -- start start 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7b16b0 0x564cbbb6f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81aa450 con 0x564cbb7a94d0 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81abf90 con 0x564cbb7b16b0 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1d6624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81aab20 con 0x564cbb7d2390 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36930/0 (socket says 172.21.15.94:36930) 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 -- 172.21.15.94:0/1061426867 learned_addr learned my addr 172.21.15.94:0/1061426867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1cd432640 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7b16b0 0x564cbbb6f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 -- 172.21.15.94:0/1061426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 msgr2=0x564cbbb720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 -- 172.21.15.94:0/1061426867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7b16b0 msgr2=0x564cbbb6f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7b16b0 0x564cbbb6f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 -- 172.21.15.94:0/1061426867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbb7aa0f0 con 0x564cbb7a94d0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.469+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000c00 tx=0x7fb1b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae0ead9dd33db6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.470+0000 7fb1b2ffd640 1 -- 172.21.15.94:0/1061426867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8058070 con 0x564cbb7a94d0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.470+0000 7fb1b2ffd640 1 -- 172.21.15.94:0/1061426867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b80035f0 con 0x564cbb7a94d0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.470+0000 7fb1b2ffd640 1 -- 172.21.15.94:0/1061426867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8035040 con 0x564cbb7a94d0 2024-04-25T13:48:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.470+0000 7fb1d6624740 1 -- 172.21.15.94:0/1061426867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 msgr2=0x564cbbb6eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.470+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000c00 tx=0x7fb1b8009260 comp rx=0 tx=0).stop 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 -- 172.21.15.94:0/1061426867 shutdown_connections 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7d2390 0x564cbbb720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7b16b0 0x564cbbb6f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 --2- 172.21.15.94:0/1061426867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb6eca0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 -- 172.21.15.94:0/1061426867 >> 172.21.15.94:0/1061426867 conn(0x564cbb7c64e0 msgr2=0x564cbbb78560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 -- 172.21.15.94:0/1061426867 shutdown_connections 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 -- 172.21.15.94:0/1061426867 wait complete. 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 -- start start 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.471+0000 7fb1d6624740 1 Processor -- start 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7d2390 0x564cbbbf1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ab8c0 con 0x564cbb7a94d0 2024-04-25T13:48:24.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ab1f0 con 0x564cbb7d2390 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81a8430 con 0x564cbb7b16b0 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 10 client.0 _ll_register_callbacks cb 0x7fb1c400be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36932/0 (socket says 172.21.15.94:36932) 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 -- 172.21.15.94:0/627854973 learned_addr learned my addr 172.21.15.94:0/627854973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7d2390 0x564cbbbf1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1cd432640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 -- 172.21.15.94:0/627854973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 msgr2=0x564cbbb816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 -- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7d2390 msgr2=0x564cbbbf1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7d2390 0x564cbbbf1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 -- 172.21.15.94:0/627854973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbba994b0 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1ccc31640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009230 tx=0x7fb1b80314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ed68b5e136c038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8058070 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb1b80319e0 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.472+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b80035f0 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1b8035070 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 10 client.17958 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cbbbf2540 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b80035f0 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb1b804f450 con 0x564cbb7a94d0 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 10 client.17958 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 20 client.17958 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:24.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb7663a0 0x564cbbbf4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:24.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 4 client.17958 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:48:24.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564cbbae0800 con 0x564cbb7663a0 2024-04-25T13:48:24.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.473+0000 7fb1cd432640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb7663a0 0x564cbbbf4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:24.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.474+0000 7fb1b0ff9640 10 client.17958 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.474+0000 7fb1cd432640 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb7663a0 0x564cbbbf4f10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e590cda57557e9 server_cookie=f3cad00ec11b9d8b in_seq=0 out_seq=0 2024-04-25T13:48:24.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.474+0000 7fb1b0ff9640 1 -- 172.21.15.94:0/627854973 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564cbbae0800 con 0x564cbb7663a0 2024-04-25T13:48:24.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.474+0000 7fb1b0ff9640 10 client.17958 handle_command_reply: tid=0 2024-04-25T13:48:24.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.474+0000 7fb1d6624740 1 client.17958 shutdown 2024-04-25T13:48:24.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 msgr2=0x564cbbb81180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009230 tx=0x7fb1b80314c0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 shutdown_connections 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564cbb7663a0 0x564cbbbf4f10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cbb7d2390 0x564cbbbf1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cbb7b16b0 0x564cbbb816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1b0ff9640 0 client.17958 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 --2- 172.21.15.94:0/627854973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cbb7a94d0 0x564cbbb81180 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 >> 172.21.15.94:0/627854973 conn(0x564cbb7c64e0 msgr2=0x564cbbb6ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 shutdown_connections 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 1 -- 172.21.15.94:0/627854973 wait complete. 2024-04-25T13:48:24.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1d6624740 20 client.17958 trim_cache size 0 max 16384 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "ca6a8d4a-ab95-4494-805e-352169abbcd3": { 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ca6a8d4a-ab95-4494-805e-352169abbcd3", 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:48:24.482 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 msgr2=0x7fb1b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 0x7fb1b40b2000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00dd050 tx=0x7fb1c0076040 comp rx=0 tx=0).stop 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 msgr2=0x7fb1c81a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000eb10 tx=0x7fb1c000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 shutdown_connections 2024-04-25T13:48:24.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1b40afaf0 0x7fb1b40b2000 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108920 0x7fb1c81a0930 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1c8103c20 0x7fb1c81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 --2- 172.21.15.94:0/2436077518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c81029e0 0x7fb1c8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.475+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 >> 172.21.15.94:0/2436077518 conn(0x7fb1c80fdbd0 msgr2=0x7fb1c8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.476+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 shutdown_connections 2024-04-25T13:48:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:24.476+0000 7fb1cf696640 1 -- 172.21.15.94:0/2436077518 wait complete. 2024-04-25T13:48:24.486 INFO:tasks.cephfs.filesystem:scrub status for tag:ca6a8d4a-ab95-4494-805e-352169abbcd3 - {'path': '/', 'tag': 'ca6a8d4a-ab95-4494-805e-352169abbcd3', 'options': 'recursive,force'} 2024-04-25T13:48:24.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 756703 13.00 MB/sec execute 187 sec latency 16.101 ms 2024-04-25T13:48:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:25 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:48:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:48:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:25 smithi094 ceph-mon[29195]: pgmap v546: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-04-25T13:48:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:25 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:48:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:48:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:25 smithi138 ceph-mon[32977]: pgmap v546: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-04-25T13:48:25.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 758736 12.97 MB/sec execute 188 sec latency 91.801 ms 2024-04-25T13:48:25.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:25 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:48:25.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:48:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:25.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:25 smithi137 ceph-mon[32568]: pgmap v546: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-04-25T13:48:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 -- 172.21.15.94:0/880260268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 msgr2=0x7f2be8103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 --2- 172.21.15.94:0/880260268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be8103740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009860 tx=0x7f2bd802a340 comp rx=0 tx=0).stop 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 -- 172.21.15.94:0/880260268 shutdown_connections 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 --2- 172.21.15.94:0/880260268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 --2- 172.21.15.94:0/880260268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 --2- 172.21.15.94:0/880260268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be8103740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 -- 172.21.15.94:0/880260268 >> 172.21.15.94:0/880260268 conn(0x7f2be80fdc30 msgr2=0x7f2be81000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 -- 172.21.15.94:0/880260268 shutdown_connections 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.340+0000 7f2bef042640 1 -- 172.21.15.94:0/880260268 wait complete. 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 Processor -- start 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 -- start start 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be81a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e540 con 0x7f2be8103c80 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e3c0 con 0x7f2be8108980 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bef042640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810e6c0 con 0x7f2be8102a40 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36958/0 (socket says 172.21.15.94:36958) 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 -- 172.21.15.94:0/3997697742 learned_addr learned my addr 172.21.15.94:0/3997697742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be81a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2becdde640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 -- 172.21.15.94:0/3997697742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 msgr2=0x7f2be806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 -- 172.21.15.94:0/3997697742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 msgr2=0x7f2be81a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be81a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 -- 172.21.15.94:0/3997697742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81a1d40 con 0x7f2be8103c80 2024-04-25T13:48:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be81a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.341+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9c0 tx=0x7f2bdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.342+0000 7f2be5ffb640 1 -- 172.21.15.94:0/3997697742 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00cde0 con 0x7f2be8103c80 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.342+0000 7f2be5ffb640 1 -- 172.21.15.94:0/3997697742 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bdc0035f0 con 0x7f2be8103c80 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.342+0000 7f2be5ffb640 1 -- 172.21.15.94:0/3997697742 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc010690 con 0x7f2be8103c80 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.342+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be81a1fd0 con 0x7f2be8103c80 2024-04-25T13:48:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.342+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a24e0 con 0x7f2be8103c80 2024-04-25T13:48:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.344+0000 7f2be5ffb640 1 -- 172.21.15.94:0/3997697742 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2bdc004000 con 0x7f2be8103c80 2024-04-25T13:48:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 Processor -- start 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 -- start start 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 0x5643e4e4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5643e4db4460 con 0x5643e4a8d4d0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5643e4d6e590 con 0x5643e4aae1b0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5643e4db0420 con 0x5643e4a852f0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2becdde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 0x5643e4e4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36964/0 (socket says 172.21.15.94:36964) 2024-04-25T13:48:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2be5ffb640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 0x7f2bd00b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2be7fff640 1 -- 172.21.15.94:0/3762287544 learned_addr learned my addr 172.21.15.94:0/3762287544 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2becdde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35780/0 (socket says 172.21.15.94:35780) 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.345+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 0x7f2bd00b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2be5ffb640 1 -- 172.21.15.94:0/3997697742 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2bdc0defb0 con 0x7f2be8103c80 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 -- 172.21.15.94:0/3762287544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 msgr2=0x5643e4e4e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 0x5643e4e4e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 -- 172.21.15.94:0/3762287544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 msgr2=0x5643e4e55df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 -- 172.21.15.94:0/3762287544 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e4d7b0d0 con 0x5643e4a852f0 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 0x5643e4e4e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2becdde640 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bd802a860 tx=0x7f2bd8005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa46bb0d6a31125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 0x7f2bd00b2070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5643e4e4fb40 tx=0x7f2be0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2bceffd640 1 -- 172.21.15.94:0/3762287544 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8037040 con 0x5643e4a852f0 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2bceffd640 1 -- 172.21.15.94:0/3762287544 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd8030e10 con 0x5643e4a852f0 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.346+0000 7f2bceffd640 1 -- 172.21.15.94:0/3762287544 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8003930 con 0x5643e4a852f0 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3762287544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 msgr2=0x5643e4e4eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bd802a860 tx=0x7f2bd8005cf0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3762287544 shutdown_connections 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e55df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e4eef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3762287544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4aae1b0 0x5643e4e4e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3762287544 >> 172.21.15.94:0/3762287544 conn(0x5643e4aa2300 msgr2=0x5643e4e4a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3762287544 shutdown_connections 2024-04-25T13:48:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3762287544 wait complete. 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 -- start start 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.347+0000 7f2bf5fc8740 1 Processor -- start 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5643e4e567e0 con 0x5643e4aae1b0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5643e4d3bb60 con 0x5643e4a8d4d0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5643e4e56960 con 0x5643e4a852f0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bf5fc8740 10 client.0 _ll_register_callbacks cb 0x5643e4d602b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35784/0 (socket says 172.21.15.94:35784) 2024-04-25T13:48:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36970/0 (socket says 172.21.15.94:36970) 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 -- 172.21.15.94:0/3929116984 learned_addr learned my addr 172.21.15.94:0/3929116984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 -- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 msgr2=0x5643e4e58b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e58b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 -- 172.21.15.94:0/3929116984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 msgr2=0x5643e4e5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 -- 172.21.15.94:0/3929116984 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e4ac6210 con 0x5643e4a852f0 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bed5df640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2becdde640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000c00 tx=0x7f2bd8004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e402a0eb4134d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.348+0000 7f2bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd804c070 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2bd8030760 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd8030e10 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8058050 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 10 client.27313 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643e4b92620 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bd8030e10 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd8003b60 con 0x5643e4a852f0 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 10 client.27313 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 20 client.27313 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4a6cd80 0x5643e4e65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 4 client.27313 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643e4dc2070 con 0x5643e4a6cd80 2024-04-25T13:48:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.349+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4a6cd80 0x5643e4e65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.350+0000 7f2be7fff640 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4a6cd80 0x5643e4e65eb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ae56bae2ca954d server_cookie=5195989a225644cc in_seq=0 out_seq=0 2024-04-25T13:48:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.350+0000 7f2bc7fff640 10 client.27313 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bc7fff640 1 -- 172.21.15.94:0/3929116984 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643e4dc2070 con 0x5643e4a6cd80 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bc7fff640 10 client.27313 handle_command_reply: tid=0 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 client.27313 shutdown 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 msgr2=0x5643e4e585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000c00 tx=0x7f2bd8004420 comp rx=0 tx=0).stop 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 shutdown_connections 2024-04-25T13:48:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4a6cd80 0x5643e4e65eb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aae1b0 0x5643e4e5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/3929116984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e585c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bc7fff640 0 client.27313 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.352+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 >> 172.21.15.94:0/3929116984 conn(0x5643e4aa2300 msgr2=0x5643e4e4d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.353+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 shutdown_connections 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.353+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/3929116984 wait complete. 2024-04-25T13:48:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.353+0000 7f2bf5fc8740 20 client.27313 trim_cache size 0 max 16384 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 Processor -- start 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 -- start start 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 0x5643e4e4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a852f0 0x5643e4e5f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ab590 con 0x5643e4a8d4d0 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ad0d0 con 0x5643e4a852f0 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81abc60 con 0x5643e4aa2300 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36980/0 (socket says 172.21.15.94:36980) 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 -- 172.21.15.94:0/1940993101 learned_addr learned my addr 172.21.15.94:0/1940993101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2be7fff640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a852f0 0x5643e4e5f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2becdde640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 0x5643e4e4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 -- 172.21.15.94:0/1940993101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 msgr2=0x5643e4e4d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 0x5643e4e4d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 -- 172.21.15.94:0/1940993101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a852f0 msgr2=0x5643e4e5f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a852f0 0x5643e4e5f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 -- 172.21.15.94:0/1940993101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e4e534d0 con 0x5643e4a8d4d0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2becdde640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 0x5643e4e4d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bed5df640 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2be000fe60 tx=0x7f2be0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c11d85b005478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.368+0000 7f2bce7fc640 1 -- 172.21.15.94:0/1940993101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004d20 con 0x5643e4a8d4d0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bce7fc640 1 -- 172.21.15.94:0/1940993101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00054d0 con 0x5643e4a8d4d0 2024-04-25T13:48:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bce7fc640 1 -- 172.21.15.94:0/1940993101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0022740 con 0x5643e4a8d4d0 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/1940993101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 msgr2=0x5643e4e49c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2be000fe60 tx=0x7f2be0009f90 comp rx=0 tx=0).stop 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/1940993101 shutdown_connections 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4a8d4d0 0x5643e4e49c10 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a852f0 0x5643e4e5f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/1940993101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4aa2300 0x5643e4e4d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/1940993101 >> 172.21.15.94:0/1940993101 conn(0x5643e4e52350 msgr2=0x5643e4e4d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/1940993101 shutdown_connections 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.369+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/1940993101 wait complete. 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 -- start start 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 Processor -- start 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e5ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81aca00 con 0x5643e4aa2300 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ac330 con 0x5643e4a8d4d0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81a9570 con 0x5643e4a852f0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bf5fc8740 10 client.0 _ll_register_callbacks cb 0x7f2bdc0a32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bed5df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36996/0 (socket says 172.21.15.94:36996) 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2bed5df640 1 -- 172.21.15.94:0/365511207 learned_addr learned my addr 172.21.15.94:0/365511207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2be7fff640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e5ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.370+0000 7f2becdde640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 -- 172.21.15.94:0/365511207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 msgr2=0x5643e4e5c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e5c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 -- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 msgr2=0x5643e4e5ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e5ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 -- 172.21.15.94:0/365511207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e4e5fec0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bed5df640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2be00049d0 tx=0x7f2be0004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ee603c0a27cf92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0039060 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be00054d0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2be00051e0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2be00224c0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bf5fc8740 10 client.17976 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643e4ed1af0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00054d0 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2be002a030 con 0x5643e4aa2300 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bf5fc8740 10 client.17976 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.371+0000 7f2bf5fc8740 10 client.17976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2bf5fc8740 20 client.17976 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4aae1b0 0x5643e4ed44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2bf5fc8740 4 client.17976 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643e4dbbee0 con 0x5643e4aae1b0 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2becdde640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4aae1b0 0x5643e4ed44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2bc7fff640 10 client.17976 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.372+0000 7f2becdde640 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4aae1b0 0x5643e4ed44c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67d47e7dc065009 server_cookie=7f8070b914d14d27 in_seq=0 out_seq=0 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bc7fff640 1 -- 172.21.15.94:0/365511207 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643e4dbbee0 con 0x5643e4aae1b0 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bc7fff640 10 client.17976 handle_command_reply: tid=0 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 client.17976 shutdown 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 msgr2=0x5643e4ed0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2be00049d0 tx=0x7f2be0004e50 comp rx=0 tx=0).stop 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 shutdown_connections 2024-04-25T13:48:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5643e4aae1b0 0x5643e4ed44c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643e4aa2300 0x5643e4ed0560 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5643e4a8d4d0 0x5643e4e5ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bc7fff640 0 client.17976 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 --2- 172.21.15.94:0/365511207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643e4a852f0 0x5643e4e5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.532+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 >> 172.21.15.94:0/365511207 conn(0x5643e4e52350 msgr2=0x5643e4e57dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.533+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 shutdown_connections 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.533+0000 7f2bf5fc8740 1 -- 172.21.15.94:0/365511207 wait complete. 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.533+0000 7f2bf5fc8740 20 client.17976 trim_cache size 0 max 16384 2024-04-25T13:48:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:48:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:48:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:48:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.533+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 msgr2=0x7f2bd00b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.533+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 0x7f2bd00b2070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x5643e4e4fb40 tx=0x7f2be0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 msgr2=0x7f2be819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9c0 tx=0x7f2bdc00ee80 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 shutdown_connections 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2bd00afb60 0x7f2bd00b2070 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be8108980 0x7f2be81a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be8103c80 0x7f2be819efd0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 --2- 172.21.15.94:0/3997697742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be8102a40 0x7f2be806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 >> 172.21.15.94:0/3997697742 conn(0x7f2be80fdc30 msgr2=0x7f2be8104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 shutdown_connections 2024-04-25T13:48:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:26.534+0000 7f2bef042640 1 -- 172.21.15.94:0/3997697742 wait complete. 2024-04-25T13:48:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:48:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:48:26.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 760371 12.96 MB/sec execute 189 sec latency 110.342 ms 2024-04-25T13:48:26.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:48:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:27 smithi094 ceph-mon[29195]: pgmap v547: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-25T13:48:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:27 smithi138 ceph-mon[32977]: pgmap v547: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-25T13:48:27.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 763483 12.99 MB/sec execute 190 sec latency 15.918 ms 2024-04-25T13:48:27.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:27 smithi137 ceph-mon[32568]: pgmap v547: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-25T13:48:28.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 765827 12.96 MB/sec execute 191 sec latency 13.457 ms 2024-04-25T13:48:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:29 smithi094 ceph-mon[29195]: pgmap v548: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-04-25T13:48:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:29 smithi138 ceph-mon[32977]: pgmap v548: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-04-25T13:48:29.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 767859 12.96 MB/sec execute 192 sec latency 13.645 ms 2024-04-25T13:48:29.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:29 smithi137 ceph-mon[32568]: pgmap v548: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-04-25T13:48:30.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 770512 12.98 MB/sec execute 193 sec latency 76.078 ms 2024-04-25T13:48:31.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:31 smithi138 ceph-mon[32977]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 416 op/s 2024-04-25T13:48:31.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 772493 12.95 MB/sec execute 194 sec latency 30.031 ms 2024-04-25T13:48:31.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:31 smithi137 ceph-mon[32568]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 416 op/s 2024-04-25T13:48:31.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:31 smithi094 ceph-mon[29195]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.8 MiB/s wr, 416 op/s 2024-04-25T13:48:32.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 774766 12.96 MB/sec execute 195 sec latency 15.488 ms 2024-04-25T13:48:33.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:33 smithi138 ceph-mon[32977]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-04-25T13:48:33.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 777709 12.98 MB/sec execute 196 sec latency 15.024 ms 2024-04-25T13:48:33.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:33 smithi137 ceph-mon[32568]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-04-25T13:48:33.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:33 smithi094 ceph-mon[29195]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-04-25T13:48:34.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 780026 12.96 MB/sec execute 197 sec latency 17.276 ms 2024-04-25T13:48:35.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:35 smithi138 ceph-mon[32977]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-04-25T13:48:35.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 781723 12.95 MB/sec execute 198 sec latency 89.480 ms 2024-04-25T13:48:35.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:35 smithi137 ceph-mon[32568]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-04-25T13:48:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:35 smithi094 ceph-mon[29195]: pgmap v551: 97 pgs: 97 active+clean; 48 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-04-25T13:48:36.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 784626 12.98 MB/sec execute 199 sec latency 19.069 ms 2024-04-25T13:48:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:37 smithi138 ceph-mon[32977]: pgmap v552: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:48:37.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 786638 12.95 MB/sec execute 200 sec latency 17.017 ms 2024-04-25T13:48:37.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:37 smithi137 ceph-mon[32568]: pgmap v552: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:48:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:37 smithi094 ceph-mon[29195]: pgmap v552: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:48:38.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 788951 12.95 MB/sec execute 201 sec latency 13.685 ms 2024-04-25T13:48:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- 172.21.15.94:0/3389244510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed01021d0 msgr2=0x7f0ed0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/3389244510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed01021d0 0x7f0ed0102ed0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000aa10 tx=0x7f0ec002b720 comp rx=0 tx=0).stop 2024-04-25T13:48:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- 172.21.15.94:0/3389244510 shutdown_connections 2024-04-25T13:48:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/3389244510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0108110 0x7f0ed010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/3389244510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed0103410 0x7f0ed0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/3389244510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed01021d0 0x7f0ed0102ed0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- 172.21.15.94:0/3389244510 >> 172.21.15.94:0/3389244510 conn(0x7f0ed00fd3c0 msgr2=0x7f0ed00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- 172.21.15.94:0/3389244510 shutdown_connections 2024-04-25T13:48:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- 172.21.15.94:0/3389244510 wait complete. 2024-04-25T13:48:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 Processor -- start 2024-04-25T13:48:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- start start 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 0x7f0ed019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 0x7f0ed019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed010d4b0 con 0x7f0ed0108110 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed010d330 con 0x7f0ed01021d0 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ed4c27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed010d630 con 0x7f0ed0103410 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ece59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 0x7f0ed019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 0x7f0ed019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57162/0 (socket says 172.21.15.94:57162) 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ece59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 0x7f0ed019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37602/0 (socket says 172.21.15.94:37602) 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 -- 172.21.15.94:0/1475450673 learned_addr learned my addr 172.21.15.94:0/1475450673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 -- 172.21.15.94:0/1475450673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 msgr2=0x7f0ed019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 0x7f0ed019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 -- 172.21.15.94:0/1475450673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 msgr2=0x7f0ed019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 0x7f0ed019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0eced9d640 1 -- 172.21.15.94:0/1475450673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed01a1e60 con 0x7f0ed0108110 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.636+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 0x7f0ed019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0eced9d640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009fd0 tx=0x7f0ec800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0ebf7fe640 1 -- 172.21.15.94:0/1475450673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8017070 con 0x7f0ed0108110 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0ebf7fe640 1 -- 172.21.15.94:0/1475450673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec8003e20 con 0x7f0ed0108110 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0ebf7fe640 1 -- 172.21.15.94:0/1475450673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8010460 con 0x7f0ed0108110 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed01a2150 con 0x7f0ed0108110 2024-04-25T13:48:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.638+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed01a2650 con 0x7f0ed0108110 2024-04-25T13:48:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ebf7fe640 1 -- 172.21.15.94:0/1475450673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0ec8010640 con 0x7f0ed0108110 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 Processor -- start 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 -- start start 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ebf7fe640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 0x7f0eb80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8321b0 0x56454fbd2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56454fb38460 con 0x56454f8092f0 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56454faf2590 con 0x56454f8114d0 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56454fb34420 con 0x56454f8321b0 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 0x7f0eb80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37608/0 (socket says 172.21.15.94:37608) 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 -- 172.21.15.94:0/3626399936 learned_addr learned my addr 172.21.15.94:0/3626399936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ebf7fe640 1 -- 172.21.15.94:0/1475450673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0ec80de920 con 0x7f0ed0108110 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0ece59c640 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8321b0 0x56454fbd2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 -- 172.21.15.94:0/3626399936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8321b0 msgr2=0x56454fbd2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8321b0 0x56454fbd2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 -- 172.21.15.94:0/3626399936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 msgr2=0x56454fbd2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbd2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.640+0000 7f0eced9d640 1 -- 172.21.15.94:0/3626399936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454faff0d0 con 0x56454f8114d0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.642+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 0x7f0eb80b1ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0ed019b3c0 tx=0x7f0ec4009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.642+0000 7f0eced9d640 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80dc380 tx=0x7f0ec8018f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93dd2b7e6b32b148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.642+0000 7f0ebcff9640 1 -- 172.21.15.94:0/3626399936 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80a6920 con 0x56454f8114d0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.642+0000 7f0ebcff9640 1 -- 172.21.15.94:0/3626399936 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec80a6e10 con 0x56454f8114d0 2024-04-25T13:48:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.642+0000 7f0ebcff9640 1 -- 172.21.15.94:0/3626399936 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80a30a0 con 0x56454f8114d0 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- 172.21.15.94:0/3626399936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 msgr2=0x56454fbd9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80dc380 tx=0x7f0ec8018f20 comp rx=0 tx=0).stop 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- 172.21.15.94:0/3626399936 shutdown_connections 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbd9df0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbd2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/3626399936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8321b0 0x56454fbd2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- 172.21.15.94:0/3626399936 >> 172.21.15.94:0/3626399936 conn(0x56454f826300 msgr2=0x56454fbce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- 172.21.15.94:0/3626399936 shutdown_connections 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- 172.21.15.94:0/3626399936 wait complete. 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.643+0000 7f0edbbb9740 1 -- start start 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 Processor -- start 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbdc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbe3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56454fbda840 con 0x56454f8114d0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56454fabfb60 con 0x56454f8321b0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56454fbda9c0 con 0x56454f8092f0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0edbbb9740 10 client.0 _ll_register_callbacks cb 0x56454fae42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbe3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57182/0 (socket says 172.21.15.94:57182) 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0ecdd9b640 1 -- 172.21.15.94:0/2157115759 learned_addr learned my addr 172.21.15.94:0/2157115759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0ece59c640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbdc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 -- 172.21.15.94:0/2157115759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 msgr2=0x56454fbdc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbdc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 -- 172.21.15.94:0/2157115759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 msgr2=0x56454fbdcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0eced9d640 1 -- 172.21.15.94:0/2157115759 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454f84a210 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.644+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eced9d640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbe3a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x56454fbda490 tx=0x7f0ec80042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b942fe28d56f3e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80afbc0 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec80a6e10 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0ec8003970 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ec801f170 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 10 client.27458 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56454f916620 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec80a6e10 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ec80a2780 con 0x56454f8321b0 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 10 client.27458 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 10 client.27458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.645+0000 7f0edbbb9740 20 client.27458 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7f0d80 0x56454fbe9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0edbbb9740 4 client.27458 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454fb46070 con 0x56454f7f0d80 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0ece59c640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7f0d80 0x56454fbe9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0eb5ffb640 10 client.27458 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.646+0000 7f0ece59c640 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7f0d80 0x56454fbe9ed0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41808b123653b2fd server_cookie=d65929b1161f0c85 in_seq=0 out_seq=0 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.647+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/2157115759 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56454fb46070 con 0x56454f7f0d80 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.647+0000 7f0eb5ffb640 10 client.27458 handle_command_reply: tid=0 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.647+0000 7f0edbbb9740 1 client.27458 shutdown 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 msgr2=0x56454fbe3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbe3a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x56454fbda490 tx=0x7f0ec80042c0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 shutdown_connections 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7f0d80 0x56454fbe9ed0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbe3a80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8114d0 0x56454fbdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/2157115759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbdc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0eb5ffb640 0 client.27458 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 >> 172.21.15.94:0/2157115759 conn(0x56454f826300 msgr2=0x56454fbcd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 shutdown_connections 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 1 -- 172.21.15.94:0/2157115759 wait complete. 2024-04-25T13:48:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.648+0000 7f0edbbb9740 20 client.27458 trim_cache size 0 max 16384 2024-04-25T13:48:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 Processor -- start 2024-04-25T13:48:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 -- start start 2024-04-25T13:48:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 0x56454fbcdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01ab630 con 0x56454f8092f0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01ad170 con 0x56454f8321b0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01abd00 con 0x56454f8114d0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0ece59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 0x56454fbcdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.665+0000 7f0ece59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57196/0 (socket says 172.21.15.94:57196) 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 0x56454fbcdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59824/0 (socket says 172.21.15.94:59824) 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 -- 172.21.15.94:0/1149173104 learned_addr learned my addr 172.21.15.94:0/1149173104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 -- 172.21.15.94:0/1149173104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 msgr2=0x56454fbcdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 0x56454fbcdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 -- 172.21.15.94:0/1149173104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 msgr2=0x56454fbce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 -- 172.21.15.94:0/1149173104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454fbd7490 con 0x56454f8092f0 2024-04-25T13:48:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0ece59c640 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c00 tx=0x7f0ec00305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9445f0e2ab24e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0eb7fff640 1 -- 172.21.15.94:0/1149173104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0036040 con 0x56454f8092f0 2024-04-25T13:48:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0eb7fff640 1 -- 172.21.15.94:0/1149173104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec004c070 con 0x56454f8092f0 2024-04-25T13:48:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.666+0000 7f0eb7fff640 1 -- 172.21.15.94:0/1149173104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0030cf0 con 0x56454f8092f0 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- 172.21.15.94:0/1149173104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 msgr2=0x56454fbdb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c00 tx=0x7f0ec00305b0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- 172.21.15.94:0/1149173104 shutdown_connections 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8321b0 0x56454fbce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8114d0 0x56454fbcdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/1149173104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8092f0 0x56454fbdb470 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- 172.21.15.94:0/1149173104 >> 172.21.15.94:0/1149173104 conn(0x56454f826300 msgr2=0x56454fbdad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- 172.21.15.94:0/1149173104 shutdown_connections 2024-04-25T13:48:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- 172.21.15.94:0/1149173104 wait complete. 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 -- start start 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.667+0000 7f0edbbb9740 1 Processor -- start 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbe0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbe0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01acaa0 con 0x56454f8321b0 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01ac3d0 con 0x56454f8114d0 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01a9610 con 0x56454f8092f0 2024-04-25T13:48:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 10 client.0 _ll_register_callbacks cb 0x7f0ec4011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57210/0 (socket says 172.21.15.94:57210) 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0ecdd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbe0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 -- 172.21.15.94:0/465035807 learned_addr learned my addr 172.21.15.94:0/465035807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0ece59c640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbe0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 -- 172.21.15.94:0/465035807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 msgr2=0x56454fbe0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbe0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 -- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 msgr2=0x56454fbe0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbe0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 -- 172.21.15.94:0/465035807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454fbe1f70 con 0x56454f8321b0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0ecdd9b640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbe0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eced9d640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80a2e90 tx=0x7f0ec80a2c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29bfc5e933ea6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8014040 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0ec8003b80 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.668+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ec004c070 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ec80a6920 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 10 client.17994 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56454fbe3200 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec004c070 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ec80a5ae0 con 0x56454f8321b0 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 10 client.17994 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 20 client.17994 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7c61c0 0x56454fc54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 4 client.17994 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:48:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56454fb3fee0 con 0x56454f7c61c0 2024-04-25T13:48:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.669+0000 7f0ece59c640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7c61c0 0x56454fc54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.670+0000 7f0eb5ffb640 10 client.17994 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.670+0000 7f0ece59c640 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7c61c0 0x56454fc54590 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bd394931048c58 server_cookie=4cab177a36798c81 in_seq=0 out_seq=0 2024-04-25T13:48:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:39 smithi138 ceph-mon[32977]: pgmap v553: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-04-25T13:48:39.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 791531 12.96 MB/sec execute 202 sec latency 56.653 ms 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0eb5ffb640 1 -- 172.21.15.94:0/465035807 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56454fb3fee0 con 0x56454f7c61c0 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0eb5ffb640 10 client.17994 handle_command_reply: tid=0 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0edbbb9740 1 client.17994 shutdown 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 msgr2=0x56454fbe19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80a2e90 tx=0x7f0ec80a2c30 comp rx=0 tx=0).stop 2024-04-25T13:48:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 shutdown_connections 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.831+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56454f7c61c0 0x56454fc54590 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0eb5ffb640 0 client.17994 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56454f8321b0 0x56454fbe19b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454f8114d0 0x56454fbe0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 --2- 172.21.15.94:0/465035807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56454f8092f0 0x56454fbe0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 >> 172.21.15.94:0/465035807 conn(0x56454f826300 msgr2=0x56454fbcdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 shutdown_connections 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 1 -- 172.21.15.94:0/465035807 wait complete. 2024-04-25T13:48:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.832+0000 7f0edbbb9740 20 client.17994 trim_cache size 0 max 16384 2024-04-25T13:48:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:48:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:48:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:48:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 msgr2=0x7f0eb80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 0x7f0eb80b1ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0ed019b3c0 tx=0x7f0ec4009f50 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 msgr2=0x7f0ed01a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009fd0 tx=0x7f0ec800b260 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 shutdown_connections 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0eb80afae0 0x7f0eb80b1ff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed0108110 0x7f0ed01a17f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed0103410 0x7f0ed019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 --2- 172.21.15.94:0/1475450673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ed01021d0 0x7f0ed019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 >> 172.21.15.94:0/1475450673 conn(0x7f0ed00fd3c0 msgr2=0x7f0ed0106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 shutdown_connections 2024-04-25T13:48:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:39.833+0000 7f0ed4c27640 1 -- 172.21.15.94:0/1475450673 wait complete. 2024-04-25T13:48:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:39 smithi137 ceph-mon[32568]: pgmap v553: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-04-25T13:48:39.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:39 smithi094 ceph-mon[29195]: pgmap v553: 97 pgs: 97 active+clean; 47 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-04-25T13:48:40.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 792880 12.93 MB/sec execute 203 sec latency 90.367 ms 2024-04-25T13:48:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:41 smithi138 ceph-mon[32977]: pgmap v554: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:48:41.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 795628 12.95 MB/sec execute 204 sec latency 31.721 ms 2024-04-25T13:48:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:41 smithi137 ceph-mon[32568]: pgmap v554: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:48:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:41 smithi094 ceph-mon[29195]: pgmap v554: 97 pgs: 97 active+clean; 50 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:48:42.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 798320 12.95 MB/sec execute 205 sec latency 14.961 ms 2024-04-25T13:48:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:43 smithi138 ceph-mon[32977]: pgmap v555: 97 pgs: 97 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-04-25T13:48:43.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:43 smithi094 ceph-mon[29195]: pgmap v555: 97 pgs: 97 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-04-25T13:48:43.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 800456 12.94 MB/sec execute 206 sec latency 25.090 ms 2024-04-25T13:48:43.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:43 smithi137 ceph-mon[32568]: pgmap v555: 97 pgs: 97 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-04-25T13:48:44.801 INFO:tasks.workunit.client.0.smithi094.stdout: 1 803072 12.94 MB/sec execute 207 sec latency 13.948 ms 2024-04-25T13:48:45.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:45 smithi138 ceph-mon[32977]: pgmap v556: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-04-25T13:48:45.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 805768 12.95 MB/sec execute 208 sec latency 61.305 ms 2024-04-25T13:48:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:45 smithi137 ceph-mon[32568]: pgmap v556: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-04-25T13:48:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:45 smithi094 ceph-mon[29195]: pgmap v556: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.8 MiB/s wr, 405 op/s 2024-04-25T13:48:46.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 807646 12.94 MB/sec execute 209 sec latency 20.195 ms 2024-04-25T13:48:47.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:47 smithi138 ceph-mon[32977]: pgmap v557: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-04-25T13:48:47.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 810072 12.94 MB/sec execute 210 sec latency 13.138 ms 2024-04-25T13:48:47.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:47 smithi137 ceph-mon[32568]: pgmap v557: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-04-25T13:48:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:47 smithi094 ceph-mon[29195]: pgmap v557: 97 pgs: 97 active+clean; 41 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 356 op/s 2024-04-25T13:48:48.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 813108 12.96 MB/sec execute 211 sec latency 17.809 ms 2024-04-25T13:48:49.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:49 smithi138 ceph-mon[32977]: pgmap v558: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2024-04-25T13:48:49.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 815000 12.94 MB/sec execute 212 sec latency 27.108 ms 2024-04-25T13:48:49.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:49 smithi137 ceph-mon[32568]: pgmap v558: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2024-04-25T13:48:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:49 smithi094 ceph-mon[29195]: pgmap v558: 97 pgs: 97 active+clean; 42 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2024-04-25T13:48:50.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 817155 12.94 MB/sec execute 213 sec latency 62.170 ms 2024-04-25T13:48:51.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:51 smithi138 ceph-mon[32977]: pgmap v559: 97 pgs: 97 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 439 op/s 2024-04-25T13:48:51.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 819894 12.95 MB/sec execute 214 sec latency 18.047 ms 2024-04-25T13:48:51.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:51 smithi137 ceph-mon[32568]: pgmap v559: 97 pgs: 97 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 439 op/s 2024-04-25T13:48:51.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:51 smithi094 ceph-mon[29195]: pgmap v559: 97 pgs: 97 active+clean; 45 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 439 op/s 2024-04-25T13:48:52.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 821604 12.93 MB/sec execute 215 sec latency 13.470 ms 2024-04-25T13:48:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 -- 172.21.15.94:0/756459967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc103b40 msgr2=0x7fb4cc104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 --2- 172.21.15.94:0/756459967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc103b40 0x7fb4cc104840 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009900 tx=0x7fb4bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 -- 172.21.15.94:0/756459967 shutdown_connections 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 --2- 172.21.15.94:0/756459967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 0x7fb4cc10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 --2- 172.21.15.94:0/756459967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc104d80 0x7fb4cc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 --2- 172.21.15.94:0/756459967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc103b40 0x7fb4cc104840 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 -- 172.21.15.94:0/756459967 >> 172.21.15.94:0/756459967 conn(0x7fb4cc0fed30 msgr2=0x7fb4cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 -- 172.21.15.94:0/756459967 shutdown_connections 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.940+0000 7fb4d244a640 1 -- 172.21.15.94:0/756459967 wait complete. 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 Processor -- start 2024-04-25T13:48:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 -- start start 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 0x7fb4cc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 0x7fb4cc1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc10f640 con 0x7fb4cc104d80 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc10f4c0 con 0x7fb4cc109a80 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d244a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc10f7c0 con 0x7fb4cc103b40 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36992/0 (socket says 172.21.15.94:36992) 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/2507518969 learned_addr learned my addr 172.21.15.94:0/2507518969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 0x7fb4cc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.941+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 0x7fb4cc1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/2507518969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 msgr2=0x7fb4cc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 0x7fb4cc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/2507518969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 msgr2=0x7fb4cc1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 0x7fb4cc1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/2507518969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc1a1db0 con 0x7fb4cc104d80 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 0x7fb4cc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000e9f0 tx=0x7fb4c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4c97fa640 1 -- 172.21.15.94:0/2507518969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000cd80 con 0x7fb4cc104d80 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc1a20a0 con 0x7fb4cc104d80 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4c97fa640 1 -- 172.21.15.94:0/2507518969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c00054d0 con 0x7fb4cc104d80 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc1a25f0 con 0x7fb4cc104d80 2024-04-25T13:48:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.942+0000 7fb4c97fa640 1 -- 172.21.15.94:0/2507518969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0010690 con 0x7fb4cc104d80 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.944+0000 7fb4c97fa640 1 -- 172.21.15.94:0/2507518969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb4c0010830 con 0x7fb4cc104d80 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 Processor -- start 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 -- start start 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cd2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd490c4d0 0x55abd4cd9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55abd4c33460 con 0x55abd490c4d0 2024-04-25T13:48:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55abd4bed590 con 0x55abd492d1b0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55abd4c2f420 con 0x55abd49042f0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4c97fa640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 0x7fb4b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd490c4d0 0x55abd4cd9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:48126/0 (socket says 172.21.15.94:48126) 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 -- 172.21.15.94:0/3392284577 learned_addr learned my addr 172.21.15.94:0/3392284577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 0x7fb4b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4c97fa640 1 -- 172.21.15.94:0/2507518969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb4c00df060 con 0x7fb4cc104d80 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cd2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 -- 172.21.15.94:0/3392284577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 msgr2=0x55abd4cd2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.945+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cd2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4d09e7640 1 -- 172.21.15.94:0/3392284577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd490c4d0 msgr2=0x55abd4cd9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd490c4d0 0x55abd4cd9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4d09e7640 1 -- 172.21.15.94:0/3392284577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd4bfa0d0 con 0x55abd492d1b0 2024-04-25T13:48:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 0x7fb4b40b1ff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400fc90 tx=0x7fb4c4018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4011e00 tx=0x7fb4c400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9630c06a5c9083e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4b27fc640 1 -- 172.21.15.94:0/3392284577 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4022070 con 0x55abd492d1b0 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4b27fc640 1 -- 172.21.15.94:0/3392284577 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c400ed10 con 0x55abd492d1b0 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.946+0000 7fb4b27fc640 1 -- 172.21.15.94:0/3392284577 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40057b0 con 0x55abd492d1b0 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3392284577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 msgr2=0x55abd4cd1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4011e00 tx=0x7fb4c400f790 comp rx=0 tx=0).stop 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3392284577 shutdown_connections 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd490c4d0 0x55abd4cd9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cd2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3392284577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd492d1b0 0x55abd4cd1d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3392284577 >> 172.21.15.94:0/3392284577 conn(0x55abd4921300 msgr2=0x55abd4cc9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3392284577 shutdown_connections 2024-04-25T13:48:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3392284577 wait complete. 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 -- start start 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.947+0000 7fb4d93d8740 1 Processor -- start 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cdba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4cdbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55abd4cd9bf0 con 0x55abd492d1b0 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55abd4bbab60 con 0x55abd490c4d0 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55abd4cd9d70 con 0x55abd49042f0 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d93d8740 10 client.0 _ll_register_callbacks cb 0x55abd4bdf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4cdbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cdba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37018/0 (socket says 172.21.15.94:37018) 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 -- 172.21.15.94:0/1056754692 learned_addr learned my addr 172.21.15.94:0/1056754692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 -- 172.21.15.94:0/1056754692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 msgr2=0x55abd4cdba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cdba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 -- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 msgr2=0x55abd4cdbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4cdbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 -- 172.21.15.94:0/1056754692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd4945210 con 0x55abd492d1b0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cdba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005c00 tx=0x7fb4c4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77e4b5aab1422b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.948+0000 7fb4ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c402e060 con 0x55abd492d1b0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb4c4003970 con 0x55abd492d1b0 2024-04-25T13:48:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c400ed10 con 0x55abd492d1b0 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4c4022040 con 0x55abd492d1b0 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 10 client.18006 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abd4a11620 con 0x55abd492d1b0 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c400ed10 con 0x55abd492d1b0 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb4c401ebe0 con 0x55abd492d1b0 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 10 client.18006 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 10 client.18006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 20 client.18006 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd48ebd80 0x55abd4ce94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 4 client.18006 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd4c41070 con 0x55abd48ebd80 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.949+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd48ebd80 0x55abd4ce94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.950+0000 7fb4ab7fe640 10 client.18006 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.950+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd48ebd80 0x55abd4ce94c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac69c537f6aa700 server_cookie=b4608dd78c810d01 in_seq=0 out_seq=0 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/1056754692 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abd4c41070 con 0x55abd48ebd80 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4ab7fe640 10 client.18006 handle_command_reply: tid=0 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 client.18006 shutdown 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 msgr2=0x55abd4ce2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005c00 tx=0x7fb4c4009500 comp rx=0 tx=0).stop 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 shutdown_connections 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd48ebd80 0x55abd4ce94c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4ab7fe640 0 client.18006 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd492d1b0 0x55abd4ce2ea0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4cdbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/1056754692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd49042f0 0x55abd4cdba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 >> 172.21.15.94:0/1056754692 conn(0x55abd4921300 msgr2=0x55abd4ccc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 shutdown_connections 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.951+0000 7fb4d93d8740 1 -- 172.21.15.94:0/1056754692 wait complete. 2024-04-25T13:48:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.952+0000 7fb4d93d8740 20 client.18006 trim_cache size 0 max 16384 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 Processor -- start 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 -- start start 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd48ebd80 0x55abd4cd5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4ce2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1a93d0 con 0x55abd49042f0 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1aaf10 con 0x55abd490c4d0 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1a9aa0 con 0x55abd48ebd80 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37030/0 (socket says 172.21.15.94:37030) 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3411675443 learned_addr learned my addr 172.21.15.94:0/3411675443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd48ebd80 0x55abd4cd5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4ce2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3411675443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd48ebd80 msgr2=0x55abd4cd5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd48ebd80 0x55abd4cd5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3411675443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 msgr2=0x55abd4ce2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4ce2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3411675443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd4ce3300 con 0x55abd49042f0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.967+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4ce2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c6a0 tx=0x7fb4c0090040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9454761e1b29e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4b1ffb640 1 -- 172.21.15.94:0/3411675443 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0005680 con 0x55abd49042f0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4b1ffb640 1 -- 172.21.15.94:0/3411675443 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c00a7c40 con 0x55abd49042f0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4b1ffb640 1 -- 172.21.15.94:0/3411675443 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00aff10 con 0x55abd49042f0 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3411675443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 msgr2=0x55abd4cd6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.968+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c6a0 tx=0x7fb4c0090040 comp rx=0 tx=0).stop 2024-04-25T13:48:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3411675443 shutdown_connections 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd490c4d0 0x55abd4ce2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4cd6420 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3411675443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd48ebd80 0x55abd4cd5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3411675443 >> 172.21.15.94:0/3411675443 conn(0x55abd4921300 msgr2=0x55abd4cda110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3411675443 shutdown_connections 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3411675443 wait complete. 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- start start 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 Processor -- start 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd48ebd80 0x55abd4ce1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 0x55abd4ce2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1aa840 con 0x55abd49042f0 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1aa170 con 0x55abd48ebd80 2024-04-25T13:48:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1a73b0 con 0x55abd490c4d0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d93d8740 10 client.0 _ll_register_callbacks cb 0x7fb4c0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37040/0 (socket says 172.21.15.94:37040) 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3912043208 learned_addr learned my addr 172.21.15.94:0/3912043208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd48ebd80 0x55abd4ce1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.969+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 0x55abd4ce2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3912043208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 msgr2=0x55abd4ce2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 0x55abd4ce2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd48ebd80 msgr2=0x55abd4ce1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd48ebd80 0x55abd4ce1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 -- 172.21.15.94:0/3912043208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd4d4f860 con 0x55abd49042f0 2024-04-25T13:48:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4d09e7640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 0x55abd4ce2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4cb7fe640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0018000 tx=0x7fb4c00b0140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e28b0754a4749a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00b03a0 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb4c00a3b30 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c00a7c40 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4c00a7540 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4d93d8740 10 client.18012 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abd4d50ac0 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.970+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c00a7c40 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb4c00a6210 con 0x55abd49042f0 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 10 client.18012 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 20 client.18012 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd492d1b0 0x55abd4d53350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 4 client.18012 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:48:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd4c3aee0 con 0x55abd492d1b0 2024-04-25T13:48:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd492d1b0 0x55abd4d53350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4ab7fe640 10 client.18012 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:52.971+0000 7fb4cbfff640 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd492d1b0 0x55abd4d53350 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c343f58138495312 server_cookie=455b79115b65ffa7 in_seq=0 out_seq=0 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4ab7fe640 1 -- 172.21.15.94:0/3912043208 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abd4c3aee0 con 0x55abd492d1b0 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4ab7fe640 10 client.18012 handle_command_reply: tid=0 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 client.18012 shutdown 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 msgr2=0x55abd4ce1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0018000 tx=0x7fb4c00b0140 comp rx=0 tx=0).stop 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 shutdown_connections 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55abd492d1b0 0x55abd4d53350 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55abd490c4d0 0x55abd4ce2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abd49042f0 0x55abd4ce1b40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 --2- 172.21.15.94:0/3912043208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55abd48ebd80 0x55abd4ce1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4ab7fe640 0 client.18012 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 >> 172.21.15.94:0/3912043208 conn(0x55abd4921300 msgr2=0x55abd4cda110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 shutdown_connections 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 1 -- 172.21.15.94:0/3912043208 wait complete. 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.159+0000 7fb4d93d8740 20 client.18012 trim_cache size 0 max 16384 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:48:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 msgr2=0x7fb4b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 0x7fb4b40b1ff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400fc90 tx=0x7fb4c4018000 comp rx=0 tx=0).stop 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 msgr2=0x7fb4cc19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000e9f0 tx=0x7fb4c000edf0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 shutdown_connections 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b40afae0 0x7fb4b40b1ff0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4cc109a80 0x7fb4cc1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4cc104d80 0x7fb4cc19a7b0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 --2- 172.21.15.94:0/2507518969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4cc103b40 0x7fb4cc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 >> 172.21.15.94:0/2507518969 conn(0x7fb4cc0fed30 msgr2=0x7fb4cc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 shutdown_connections 2024-04-25T13:48:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:48:53.160+0000 7fb4d244a640 1 -- 172.21.15.94:0/2507518969 wait complete. 2024-04-25T13:48:53.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 823936 12.93 MB/sec execute 216 sec latency 65.298 ms 2024-04-25T13:48:53.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:53 smithi137 ceph-mon[32568]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 370 op/s 2024-04-25T13:48:53.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:53 smithi094 ceph-mon[29195]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 370 op/s 2024-04-25T13:48:53.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:53 smithi138 ceph-mon[32977]: pgmap v560: 97 pgs: 97 active+clean; 46 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 370 op/s 2024-04-25T13:48:54.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:48:54.487 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 -- 172.21.15.94:0/2674173470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 msgr2=0x7fb15410ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 --2- 172.21.15.94:0/2674173470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb15410ad50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009ca0 tx=0x7fb14c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 -- 172.21.15.94:0/2674173470 shutdown_connections 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 --2- 172.21.15.94:0/2674173470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb15410ad50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 --2- 172.21.15.94:0/2674173470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 0x7fb154104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 --2- 172.21.15.94:0/2674173470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb1541036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 -- 172.21.15.94:0/2674173470 >> 172.21.15.94:0/2674173470 conn(0x7fb1540fdc00 msgr2=0x7fb154100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 -- 172.21.15.94:0/2674173470 shutdown_connections 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 -- 172.21.15.94:0/2674173470 wait complete. 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.604+0000 7fb15a496640 1 Processor -- start 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 -- start start 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb15406bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 0x7fb15419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb15410e520 con 0x7fb154108930 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb15410e3a0 con 0x7fb1541029f0 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb15a496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb15410e6a0 con 0x7fb154103c30 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60614/0 (socket says 172.21.15.94:60614) 2024-04-25T13:48:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 -- 172.21.15.94:0/821066188 learned_addr learned my addr 172.21.15.94:0/821066188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb153fff640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb15406bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb1537fe640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 0x7fb15419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 -- 172.21.15.94:0/821066188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 msgr2=0x7fb15419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 0x7fb15419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 -- 172.21.15.94:0/821066188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 msgr2=0x7fb15406bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb15406bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb158a33640 1 -- 172.21.15.94:0/821066188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1541a1ce0 con 0x7fb154108930 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.605+0000 7fb153fff640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb15406bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb158a33640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb14c004af0 tx=0x7fb14c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb1517fa640 1 -- 172.21.15.94:0/821066188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c003b70 con 0x7fb154108930 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb1517fa640 1 -- 172.21.15.94:0/821066188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb14c003d10 con 0x7fb154108930 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1541a1f70 con 0x7fb154108930 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb1517fa640 1 -- 172.21.15.94:0/821066188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c02fb20 con 0x7fb154108930 2024-04-25T13:48:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.606+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1541a23d0 con 0x7fb154108930 2024-04-25T13:48:54.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.608+0000 7fb1517fa640 1 -- 172.21.15.94:0/821066188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb14c004580 con 0x7fb154108930 2024-04-25T13:48:54.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.608+0000 7fb161426740 1 Processor -- start 2024-04-25T13:48:54.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 -- start start 2024-04-25T13:48:54.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1454a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1455c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bbf13b9d80 con 0x55bbf108a4d0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbf13738b0 con 0x55bbf10b3390 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb161426740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbf13b5d40 con 0x55bbf10926b0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb1517fa640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 0x7fb13c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60616/0 (socket says 172.21.15.94:60616) 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 -- 172.21.15.94:0/2687607702 learned_addr learned my addr 172.21.15.94:0/2687607702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1454a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb158a33640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 0x7fb13c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1455c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb1517fa640 1 -- 172.21.15.94:0/821066188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb14c100290 con 0x7fb154108930 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 -- 172.21.15.94:0/2687607702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 msgr2=0x55bbf1455c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1455c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 -- 172.21.15.94:0/2687607702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 msgr2=0x55bbf1454a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1454a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 -- 172.21.15.94:0/2687607702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf1344a40 con 0x55bbf108a4d0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1454a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb153fff640 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb1440114f0 tx=0x7fb1440119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5271ea5e20094fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.609+0000 7fb158a33640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 0x7fb13c0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0c58b0 tx=0x7fb14c097190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb13a7fc640 1 -- 172.21.15.94:0/2687607702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144011310 con 0x55bbf108a4d0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb13a7fc640 1 -- 172.21.15.94:0/2687607702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1440054d0 con 0x55bbf108a4d0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb13a7fc640 1 -- 172.21.15.94:0/2687607702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400f560 con 0x55bbf108a4d0 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 -- 172.21.15.94:0/2687607702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 msgr2=0x55bbf1454fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb1440114f0 tx=0x7fb1440119b0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 -- 172.21.15.94:0/2687607702 shutdown_connections 2024-04-25T13:48:54.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1455c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf1454fe0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 --2- 172.21.15.94:0/2687607702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1454a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 -- 172.21.15.94:0/2687607702 >> 172.21.15.94:0/2687607702 conn(0x55bbf10a74e0 msgr2=0x55bbf1450270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 -- 172.21.15.94:0/2687607702 shutdown_connections 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.610+0000 7fb161426740 1 -- 172.21.15.94:0/2687607702 wait complete. 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 -- start start 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 Processor -- start 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf108a4d0 0x55bbf145e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf145edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bbf1456780 con 0x55bbf10b3390 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbf1340d10 con 0x55bbf108a4d0 2024-04-25T13:48:54.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbf1456900 con 0x55bbf10926b0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb161426740 10 client.0 _ll_register_callbacks cb 0x55bbf136d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60632/0 (socket says 172.21.15.94:60632) 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb1537fe640 1 -- 172.21.15.94:0/2774343367 learned_addr learned my addr 172.21.15.94:0/2774343367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb153fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf145edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.611+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf108a4d0 0x55bbf145e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 -- 172.21.15.94:0/2774343367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 msgr2=0x55bbf145edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf145edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 -- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf108a4d0 msgr2=0x55bbf145e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf108a4d0 0x55bbf145e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 -- 172.21.15.94:0/2774343367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf11957d0 con 0x55bbf10b3390 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1537fe640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb14800f440 tx=0x7fb148004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c860ddff80f690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1337fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14800fe80 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb1480130d0 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1440054d0 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1480132d0 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb161426740 10 client.18024 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbf10cb3f0 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.612+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1440054d0 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb148003b10 con 0x55bbf10b3390 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 10 client.18024 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 20 client.18024 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf1071f60 0x55bbf146c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 4 client.18024 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbf13c7990 con 0x55bbf1071f60 2024-04-25T13:48:54.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb153fff640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf1071f60 0x55bbf146c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb1337fe640 10 client.18024 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.613+0000 7fb153fff640 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf1071f60 0x55bbf146c2c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e52faf81a6bd138 server_cookie=f9969964360a96b4 in_seq=0 out_seq=0 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.614+0000 7fb1337fe640 1 -- 172.21.15.94:0/2774343367 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbf13c7990 con 0x55bbf1071f60 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.614+0000 7fb1337fe640 10 client.18024 handle_command_reply: tid=0 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.614+0000 7fb161426740 1 client.18024 shutdown 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 msgr2=0x55bbf1465d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb14800f440 tx=0x7fb148004730 comp rx=0 tx=0).stop 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 shutdown_connections 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf1071f60 0x55bbf146c2c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10b3390 0x55bbf1465d50 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf145edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 --2- 172.21.15.94:0/2774343367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf108a4d0 0x55bbf145e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb1337fe640 0 client.18024 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:54.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 >> 172.21.15.94:0/2774343367 conn(0x55bbf10a74e0 msgr2=0x55bbf1453180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 shutdown_connections 2024-04-25T13:48:54.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 1 -- 172.21.15.94:0/2774343367 wait complete. 2024-04-25T13:48:54.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.615+0000 7fb161426740 20 client.18024 trim_cache size 0 max 16384 2024-04-25T13:48:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.633+0000 7fb161426740 1 Processor -- start 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.633+0000 7fb161426740 1 -- start start 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1465ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf14501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ab4a0 con 0x55bbf108a4d0 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541acfe0 con 0x55bbf10b3390 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb161426740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541abb70 con 0x55bbf10926b0 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60636/0 (socket says 172.21.15.94:60636) 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 -- 172.21.15.94:0/409047683 learned_addr learned my addr 172.21.15.94:0/409047683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb153fff640 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1465ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf14501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 -- 172.21.15.94:0/409047683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 msgr2=0x55bbf1465ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1465ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 -- 172.21.15.94:0/409047683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 msgr2=0x55bbf14501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf14501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 -- 172.21.15.94:0/409047683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf108b0f0 con 0x55bbf108a4d0 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb1537fe640 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb148013dd0 tx=0x7fb148005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e579aaed9b78be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb139ffb640 1 -- 172.21.15.94:0/409047683 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14802e040 con 0x55bbf108a4d0 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb139ffb640 1 -- 172.21.15.94:0/409047683 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb14800fe80 con 0x55bbf108a4d0 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.634+0000 7fb139ffb640 1 -- 172.21.15.94:0/409047683 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148004260 con 0x55bbf108a4d0 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- 172.21.15.94:0/409047683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 msgr2=0x55bbf144fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb148013dd0 tx=0x7fb148005850 comp rx=0 tx=0).stop 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- 172.21.15.94:0/409047683 shutdown_connections 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf14501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf10926b0 0x55bbf1465ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 --2- 172.21.15.94:0/409047683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf108a4d0 0x55bbf144fca0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- 172.21.15.94:0/409047683 >> 172.21.15.94:0/409047683 conn(0x55bbf10a74e0 msgr2=0x55bbf14595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- 172.21.15.94:0/409047683 shutdown_connections 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- 172.21.15.94:0/409047683 wait complete. 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.635+0000 7fb161426740 1 -- start start 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 Processor -- start 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 0x55bbf1465c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1462d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ac910 con 0x55bbf10926b0 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ac240 con 0x55bbf10b3390 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541a9480 con 0x55bbf108a4d0 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb161426740 10 client.0 _ll_register_callbacks cb 0x7fb148013550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60650/0 (socket says 172.21.15.94:60650) 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 -- 172.21.15.94:0/1707541927 learned_addr learned my addr 172.21.15.94:0/1707541927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb1537fe640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1462d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb1537fe640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 0x55bbf1465c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 -- 172.21.15.94:0/1707541927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 msgr2=0x55bbf1465c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 0x55bbf1465c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 -- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 msgr2=0x55bbf1462d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1462d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb153fff640 1 -- 172.21.15.94:0/1707541927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf137a4b0 con 0x55bbf10926b0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.636+0000 7fb1537fe640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 0x55bbf1465c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb153fff640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1440114c0 tx=0x7fb144009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fc3d54064d7372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440054c0 con 0x55bbf10926b0 2024-04-25T13:48:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fb14402b2c0 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbf13c1800 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb144039050 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 10 client.18030 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbf14d3690 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bbf13c1800 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb14402bce0 con 0x55bbf10926b0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 10 client.18030 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 20 client.18030 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf10473a0 0x55bbf14d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 4 client.18030 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.637+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbf133c4a0 con 0x55bbf10473a0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1537fe640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf10473a0 0x55bbf14d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1337fe640 10 client.18030 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1537fe640 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf10473a0 0x55bbf14d5f10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67639697d910178 server_cookie=67185c40c4415e23 in_seq=0 out_seq=0 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1337fe640 1 -- 172.21.15.94:0/1707541927 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbf133c4a0 con 0x55bbf10473a0 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1337fe640 10 client.18030 handle_command_reply: tid=0 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 client.18030 shutdown 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 msgr2=0x55bbf14627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1440114c0 tx=0x7fb144009310 comp rx=0 tx=0).stop 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 shutdown_connections 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbf10473a0 0x55bbf14d5f10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf10b3390 0x55bbf1462d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbf10926b0 0x55bbf14627e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb1337fe640 0 client.18030 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 --2- 172.21.15.94:0/1707541927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbf108a4d0 0x55bbf1465c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 >> 172.21.15.94:0/1707541927 conn(0x55bbf10a74e0 msgr2=0x55bbf144fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 shutdown_connections 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.638+0000 7fb161426740 1 -- 172.21.15.94:0/1707541927 wait complete. 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb161426740 20 client.18030 trim_cache size 0 max 16384 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:48:54.646 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 msgr2=0x7fb13c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 0x7fb13c0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0c58b0 tx=0x7fb14c097190 comp rx=0 tx=0).stop 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 msgr2=0x7fb1541a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb14c004af0 tx=0x7fb14c031700 comp rx=0 tx=0).stop 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 shutdown_connections 2024-04-25T13:48:54.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb13c0afaf0 0x7fb13c0b2000 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154108930 0x7fb1541a14c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb154103c30 0x7fb15419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 --2- 172.21.15.94:0/821066188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541029f0 0x7fb15406bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 >> 172.21.15.94:0/821066188 conn(0x7fb1540fdc00 msgr2=0x7fb154104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 shutdown_connections 2024-04-25T13:48:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.639+0000 7fb15a496640 1 -- 172.21.15.94:0/821066188 wait complete. 2024-04-25T13:48:54.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:48:54.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:54.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 -- 172.21.15.94:0/3996635766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 msgr2=0x7f9ea8103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 --2- 172.21.15.94:0/3996635766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8103700 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:54.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 -- 172.21.15.94:0/3996635766 shutdown_connections 2024-04-25T13:48:54.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 --2- 172.21.15.94:0/3996635766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8108940 0x7f9ea810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 --2- 172.21.15.94:0/3996635766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8103c40 0x7f9ea8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 --2- 172.21.15.94:0/3996635766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8103700 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 -- 172.21.15.94:0/3996635766 >> 172.21.15.94:0/3996635766 conn(0x7f9ea80fdbf0 msgr2=0x7f9ea81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 -- 172.21.15.94:0/3996635766 shutdown_connections 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.762+0000 7f9eae296640 1 -- 172.21.15.94:0/3996635766 wait complete. 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 Processor -- start 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 -- start start 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8103c40 0x7f9ea8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8108940 0x7f9ea81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810e4e0 con 0x7f9ea8102a00 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810e360 con 0x7f9ea8103c40 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eae296640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810e660 con 0x7f9ea8108940 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60662/0 (socket says 172.21.15.94:60662) 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 -- 172.21.15.94:0/1379118588 learned_addr learned my addr 172.21.15.94:0/1379118588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea77fe640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8103c40 0x7f9ea8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9eac833640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8108940 0x7f9ea81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 -- 172.21.15.94:0/1379118588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8108940 msgr2=0x7f9ea81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8108940 0x7f9ea81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 -- 172.21.15.94:0/1379118588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8103c40 msgr2=0x7f9ea8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.763+0000 7f9ea7fff640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8103c40 0x7f9ea8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9ea7fff640 1 -- 172.21.15.94:0/1379118588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a0c50 con 0x7f9ea8102a00 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9ea7fff640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9803f030 con 0x7f9ea8102a00 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a0ee0 con 0x7f9ea8102a00 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a1340 con 0x7f9ea8102a00 2024-04-25T13:48:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e98003aa0 con 0x7f9ea8102a00 2024-04-25T13:48:54.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.764+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98049060 con 0x7f9ea8102a00 2024-04-25T13:48:54.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.765+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e940052c0 con 0x7f9ea8102a00 2024-04-25T13:48:54.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.766+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9e98042030 con 0x7f9ea8102a00 2024-04-25T13:48:54.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.767+0000 7f9ea57fa640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 0x7f9e900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:54.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.767+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9e980ff7d0 con 0x7f9ea8102a00 2024-04-25T13:48:54.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.767+0000 7f9ea77fe640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 0x7f9e900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:54.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.767+0000 7f9ea77fe640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 0x7f9e900b2000 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ea819a1b0 tx=0x7f9e9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.771+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e980c7cc0 con 0x7f9ea8102a00 2024-04-25T13:48:54.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 826126 12.91 MB/sec execute 217 sec latency 13.014 ms 2024-04-25T13:48:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.997+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e940055b0 con 0x7f9ea8102a00 2024-04-25T13:48:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:54.998+0000 7f9ea57fa640 1 -- 172.21.15.94:0/1379118588 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f9e980cbfc0 con 0x7f9ea8102a00 2024-04-25T13:48:54.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:48:54.999 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:48:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:48:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 msgr2=0x7f9e900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 0x7f9e900b2000 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ea819a1b0 tx=0x7f9e9c00b410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 msgr2=0x7f9ea8199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e98004270 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 shutdown_connections 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9e900afaf0 0x7f9e900b2000 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ea8108940 0x7f9ea81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8103c40 0x7f9ea8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 --2- 172.21.15.94:0/1379118588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ea8102a00 0x7f9ea8199110 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.000+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 >> 172.21.15.94:0/1379118588 conn(0x7f9ea80fdbf0 msgr2=0x7f9ea8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.001+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 shutdown_connections 2024-04-25T13:48:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.001+0000 7f9eae296640 1 -- 172.21.15.94:0/1379118588 wait complete. 2024-04-25T13:48:55.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 -- 172.21.15.94:0/740356103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c106c10 msgr2=0x7f064c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 --2- 172.21.15.94:0/740356103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c106c10 0x7f064c1090f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0634009900 tx=0x7f063402a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 -- 172.21.15.94:0/740356103 shutdown_connections 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 --2- 172.21.15.94:0/740356103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 0x7f064c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 --2- 172.21.15.94:0/740356103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c106c10 0x7f064c1090f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 --2- 172.21.15.94:0/740356103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c101e30 0x7f064c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 -- 172.21.15.94:0/740356103 >> 172.21.15.94:0/740356103 conn(0x7f064c0fd340 msgr2=0x7f064c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 -- 172.21.15.94:0/740356103 shutdown_connections 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.125+0000 7f065120c640 1 -- 172.21.15.94:0/740356103 wait complete. 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 Processor -- start 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 -- start start 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c106c10 0x7f064c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 0x7f064c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f064c10f660 con 0x7f064c101e30 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f064c10f4e0 con 0x7f064c106c10 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f065120c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f064c10f7e0 con 0x7f064c109780 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60678/0 (socket says 172.21.15.94:60678) 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 -- 172.21.15.94:0/3081028436 learned_addr learned my addr 172.21.15.94:0/3081028436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064b59e640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 0x7f064c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f0643fff640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c106c10 0x7f064c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 -- 172.21.15.94:0/3081028436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 msgr2=0x7f064c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 0x7f064c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 -- 172.21.15.94:0/3081028436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c106c10 msgr2=0x7f064c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c106c10 0x7f064c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.126+0000 7f064ad9d640 1 -- 172.21.15.94:0/3081028436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c1a1d40 con 0x7f064c101e30 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f064ad9d640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f063c00e9c0 tx=0x7f063c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f0648d99640 1 -- 172.21.15.94:0/3081028436 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c00cd80 con 0x7f064c101e30 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f0648d99640 1 -- 172.21.15.94:0/3081028436 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f063c0054d0 con 0x7f064c101e30 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c1a1fd0 con 0x7f064c101e30 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f0648d99640 1 -- 172.21.15.94:0/3081028436 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c010430 con 0x7f064c101e30 2024-04-25T13:48:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.127+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c1a2490 con 0x7f064c101e30 2024-04-25T13:48:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.129+0000 7f0648d99640 1 -- 172.21.15.94:0/3081028436 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f063c004fd0 con 0x7f064c101e30 2024-04-25T13:48:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.129+0000 7f0658198740 1 Processor -- start 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.129+0000 7f0658198740 1 -- start start 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b509234d80 con 0x55b508f0d6b0 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5091eea20 con 0x55b508f054d0 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0658198740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b509230d40 con 0x55b508f2e390 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60688/0 (socket says 172.21.15.94:60688) 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0643fff640 1 -- 172.21.15.94:0/1700181630 learned_addr learned my addr 172.21.15.94:0/1700181630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0648d99640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 0x7f06380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 -- 172.21.15.94:0/1700181630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 msgr2=0x55b5092d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 -- 172.21.15.94:0/1700181630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 msgr2=0x55b5092d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 -- 172.21.15.94:0/1700181630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5091f9c30 con 0x55b508f2e390 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0643fff640 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f0648d99640 1 -- 172.21.15.94:0/3081028436 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f063c0df040 con 0x7f064c101e30 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 0x7f06380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.130+0000 7f064b59e640 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092d0810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0644009f00 tx=0x7f0644011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f2c3ba305500485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.131+0000 7f06417fa640 1 -- 172.21.15.94:0/1700181630 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644020070 con 0x55b508f2e390 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.131+0000 7f06417fa640 1 -- 172.21.15.94:0/1700181630 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0644003df0 con 0x55b508f2e390 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.131+0000 7f06417fa640 1 -- 172.21.15.94:0/1700181630 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064401b420 con 0x55b508f2e390 2024-04-25T13:48:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- 172.21.15.94:0/1700181630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 msgr2=0x55b5092d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f064b59e640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 0x7f06380b2000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f064401d000 tx=0x7f064401bba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092d0810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0644009f00 tx=0x7f0644011810 comp rx=0 tx=0).stop 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- 172.21.15.94:0/1700181630 shutdown_connections 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- 172.21.15.94:0/1700181630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092d0810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- 172.21.15.94:0/1700181630 >> 172.21.15.94:0/1700181630 conn(0x55b508f224e0 msgr2=0x55b5092cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- 172.21.15.94:0/1700181630 shutdown_connections 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- 172.21.15.94:0/1700181630 wait complete. 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- start start 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 Processor -- start 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b5092d24d0 con 0x55b508f2e390 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5091bbdc0 con 0x55b508f0d6b0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.132+0000 7f0658198740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5092d2650 con 0x55b508f054d0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f0658198740 10 client.0 _ll_register_callbacks cb 0x55b5091d7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60690/0 (socket says 172.21.15.94:60690) 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f0643fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 -- 172.21.15.94:0/1688666207 learned_addr learned my addr 172.21.15.94:0/1688666207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064ad9d640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 -- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 msgr2=0x55b5092da520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092da520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 -- 172.21.15.94:0/1688666207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 msgr2=0x55b5092daa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092daa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 -- 172.21.15.94:0/1688666207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b508f463f0 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f064b59e640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0644006010 tx=0x7f0644016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abaea5a6007e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f06327fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400ed60 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f064401b040 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0644003df0 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f064401b240 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f0658198740 10 client.18048 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.133+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5090107d0 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0644003df0 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f064402e070 con 0x55b508f2e390 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 10 client.18048 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 10 client.18048 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 20 client.18048 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508eecf60 0x55b5092e8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 4 client.18048 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b509242990 con 0x55b508eecf60 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f064ad9d640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508eecf60 0x55b5092e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f064ad9d640 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508eecf60 0x55b5092e8080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b228384c94722f8c server_cookie=e7af0e2a0cdd55ed in_seq=0 out_seq=0 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.134+0000 7f06327fc640 10 client.18048 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f06327fc640 1 -- 172.21.15.94:0/1688666207 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b509242990 con 0x55b508eecf60 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f06327fc640 10 client.18048 handle_command_reply: tid=0 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 client.18048 shutdown 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 msgr2=0x55b5092e19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0644006010 tx=0x7f0644016000 comp rx=0 tx=0).stop 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 shutdown_connections 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508eecf60 0x55b5092e8080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b5092e19c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f06327fc640 0 client.18048 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 --2- 172.21.15.94:0/1688666207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092da520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 >> 172.21.15.94:0/1688666207 conn(0x55b508f224e0 msgr2=0x55b5092cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 shutdown_connections 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.135+0000 7f0658198740 1 -- 172.21.15.94:0/1688666207 wait complete. 2024-04-25T13:48:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.136+0000 7f0658198740 20 client.18048 trim_cache size 0 max 16384 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 Processor -- start 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 -- start start 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1ab580 con 0x55b508f0d6b0 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1ad0c0 con 0x55b508f054d0 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.149+0000 7f0658198740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1abc50 con 0x55b508f2e390 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60698/0 (socket says 172.21.15.94:60698) 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 -- 172.21.15.94:0/2332209861 learned_addr learned my addr 172.21.15.94:0/2332209861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f064ad9d640 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f064b59e640 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 -- 172.21.15.94:0/2332209861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 msgr2=0x55b5092cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 -- 172.21.15.94:0/2332209861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 msgr2=0x55b5092cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 -- 172.21.15.94:0/2332209861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5092e1e20 con 0x55b508f0d6b0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0643fff640 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0634005f70 tx=0x7f0634004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f849c95389d412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0640ff9640 1 -- 172.21.15.94:0/2332209861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634035040 con 0x55b508f0d6b0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0640ff9640 1 -- 172.21.15.94:0/2332209861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0634004400 con 0x55b508f0d6b0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.150+0000 7f0640ff9640 1 -- 172.21.15.94:0/2332209861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634031a70 con 0x55b508f0d6b0 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 -- 172.21.15.94:0/2332209861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 msgr2=0x55b5092e18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0634005f70 tx=0x7f0634004310 comp rx=0 tx=0).stop 2024-04-25T13:48:55.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 -- 172.21.15.94:0/2332209861 shutdown_connections 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f2e390 0x55b5092cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f0d6b0 0x55b5092e18e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 --2- 172.21.15.94:0/2332209861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f054d0 0x55b5092cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 -- 172.21.15.94:0/2332209861 >> 172.21.15.94:0/2332209861 conn(0x55b508f224e0 msgr2=0x55b5092cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 -- 172.21.15.94:0/2332209861 shutdown_connections 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.151+0000 7f0658198740 1 -- 172.21.15.94:0/2332209861 wait complete. 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 -- start start 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 Processor -- start 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1ac9f0 con 0x55b508f2e390 2024-04-25T13:48:55.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1ac320 con 0x55b508f0d6b0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1a9560 con 0x55b508f054d0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0658198740 10 client.0 _ll_register_callbacks cb 0x7f064401e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60704/0 (socket says 172.21.15.94:60704) 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 -- 172.21.15.94:0/2963688061 learned_addr learned my addr 172.21.15.94:0/2963688061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f0643fff640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064ad9d640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 -- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 msgr2=0x55b5092de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 -- 172.21.15.94:0/2963688061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 msgr2=0x55b5092dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 -- 172.21.15.94:0/2963688061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5092df260 con 0x55b508f2e390 2024-04-25T13:48:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f064b59e640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0644011850 tx=0x7f064400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c08292747711ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.152+0000 7f06327fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644020040 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f06440050f0 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0634004400 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0644011d10 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 10 client.18054 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b50934f9a0 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0634004400 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f064402e070 con 0x55b508f2e390 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 10 client.18054 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 10 client.18054 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 20 client.18054 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508ec23a0 0x55b509352310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 4 client.18054 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b50923c800 con 0x55b508ec23a0 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.153+0000 7f064ad9d640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508ec23a0 0x55b509352310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f06327fc640 10 client.18054 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f064ad9d640 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508ec23a0 0x55b509352310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3645e299f329097c server_cookie=da8fa54d2f1fe195 in_seq=0 out_seq=0 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f06327fc640 1 -- 172.21.15.94:0/2963688061 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b50923c800 con 0x55b508ec23a0 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f06327fc640 10 client.18054 handle_command_reply: tid=0 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 client.18054 shutdown 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 msgr2=0x55b50934e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0644011850 tx=0x7f064400ee10 comp rx=0 tx=0).stop 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 shutdown_connections 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b508ec23a0 0x55b509352310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b508f2e390 0x55b50934e420 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b508f0d6b0 0x55b5092dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f06327fc640 0 client.18054 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 --2- 172.21.15.94:0/2963688061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b508f054d0 0x55b5092de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 >> 172.21.15.94:0/2963688061 conn(0x55b508f224e0 msgr2=0x55b5092ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 shutdown_connections 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 1 -- 172.21.15.94:0/2963688061 wait complete. 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.154+0000 7f0658198740 20 client.18054 trim_cache size 0 max 16384 2024-04-25T13:48:55.162 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 msgr2=0x7f06380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 0x7f06380b2000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f064401d000 tx=0x7f064401bba0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 msgr2=0x7f064c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f063c00e9c0 tx=0x7f063c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 shutdown_connections 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06380afaf0 0x7f06380b2000 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f064c109780 0x7f064c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f064c106c10 0x7f064c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 --2- 172.21.15.94:0/3081028436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f064c101e30 0x7f064c19a200 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 >> 172.21.15.94:0/3081028436 conn(0x7f064c0fd340 msgr2=0x7f064c1078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 shutdown_connections 2024-04-25T13:48:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.155+0000 7f065120c640 1 -- 172.21.15.94:0/3081028436 wait complete. 2024-04-25T13:48:55.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:48:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/2158420035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c1029e0 msgr2=0x7f8e9c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/2158420035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c1036e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009900 tx=0x7f8e8c02a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/2158420035 shutdown_connections 2024-04-25T13:48:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/2158420035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c108920 0x7f8e9c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/2158420035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/2158420035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c1036e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/2158420035 >> 172.21.15.94:0/2158420035 conn(0x7f8e9c0fdbd0 msgr2=0x7f8e9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/2158420035 shutdown_connections 2024-04-25T13:48:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.280+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/2158420035 wait complete. 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 Processor -- start 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 -- start start 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c10e4c0 con 0x7f8e9c108920 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c10e340 con 0x7f8e9c1029e0 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8ea0ce0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c10e640 con 0x7f8e9c103c20 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8e9a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.281+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52676/0 (socket says 172.21.15.94:52676) 2024-04-25T13:48:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47478/0 (socket says 172.21.15.94:47478) 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9a59c640 1 -- 172.21.15.94:0/4068096869 learned_addr learned my addr 172.21.15.94:0/4068096869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60726/0 (socket says 172.21.15.94:60726) 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 -- 172.21.15.94:0/4068096869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 msgr2=0x7f8e9c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 -- 172.21.15.94:0/4068096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 msgr2=0x7f8e9c06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c06a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 -- 172.21.15.94:0/4068096869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1a2040 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e9ad9d640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e940 tx=0x7f8e9400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e937fe640 1 -- 172.21.15.94:0/4068096869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94004d20 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e937fe640 1 -- 172.21.15.94:0/4068096869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e94005020 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8e937fe640 1 -- 172.21.15.94:0/4068096869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940116d0 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c1a2330 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.282+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a2830 con 0x7f8e9c108920 2024-04-25T13:48:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.285+0000 7f8ea7b8a740 1 Processor -- start 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.285+0000 7f8e937fe640 1 -- 172.21.15.94:0/4068096869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8e94005460 con 0x7f8e9c108920 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 -- start start 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758683d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x563758684a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5637585e7d80 con 0x5637582b84d0 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5637585a1a20 con 0x5637582e1390 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5637585e3d40 con 0x5637582c06b0 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52682/0 (socket says 172.21.15.94:52682) 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e93fff640 1 -- 172.21.15.94:0/1757071420 learned_addr learned my addr 172.21.15.94:0/1757071420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758683d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x563758684a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e937fe640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 0x7f8e880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 -- 172.21.15.94:0/1757071420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 msgr2=0x563758684a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x563758684a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 -- 172.21.15.94:0/1757071420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 msgr2=0x563758683810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9a59c640 1 -- 172.21.15.94:0/1757071420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637585acc30 con 0x5637582b84d0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.286+0000 7f8e9ad9d640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 0x7f8e880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e937fe640 1 -- 172.21.15.94:0/4068096869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e940dec80 con 0x7f8e9c108920 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758683d90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9198df5752fd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e90ff9640 1 -- 172.21.15.94:0/1757071420 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c035030 con 0x5637582b84d0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e90ff9640 1 -- 172.21.15.94:0/1757071420 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8c040070 con 0x5637582b84d0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e9ad9d640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 0x7f8e880b2000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a7100 tx=0x7f8e94005800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.287+0000 7f8e90ff9640 1 -- 172.21.15.94:0/1757071420 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0039c0 con 0x5637582b84d0 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1757071420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 msgr2=0x563758683d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758683d90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c031c30 comp rx=0 tx=0).stop 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1757071420 shutdown_connections 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x563758684a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758683d90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1757071420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582e1390 0x563758683810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1757071420 >> 172.21.15.94:0/1757071420 conn(0x5637582d54e0 msgr2=0x56375867f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1757071420 shutdown_connections 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1757071420 wait complete. 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 -- start start 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.288+0000 7f8ea7b8a740 1 Processor -- start 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582b84d0 0x56375868d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x56375868dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5637586854d0 con 0x5637582e1390 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56375856edc0 con 0x5637582b84d0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563758685650 con 0x5637582c06b0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 10 client.0 _ll_register_callbacks cb 0x56375858a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e9a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x56375868dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60744/0 (socket says 172.21.15.94:60744) 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 -- 172.21.15.94:0/2399009834 learned_addr learned my addr 172.21.15.94:0/2399009834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582b84d0 0x56375868d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 -- 172.21.15.94:0/2399009834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 msgr2=0x56375868dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x56375868dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 -- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582b84d0 msgr2=0x56375868d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582b84d0 0x56375868d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 -- 172.21.15.94:0/2399009834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637582f93f0 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x56375868dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e93fff640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8e84012dc0 tx=0x7f8e84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0fec1262696bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400ebe0 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e8400ed80 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.289+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8c040070 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e84004020 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 10 client.18066 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637583c37d0 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8c040070 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e84023030 con 0x5637582e1390 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 10 client.18066 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 10 client.18066 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 20 client.18066 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56375829ff60 0x56375869b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 4 client.18066 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637585f5990 con 0x56375829ff60 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.290+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56375829ff60 0x56375869b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.291+0000 7f8e81ffb640 10 client.18066 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.291+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56375829ff60 0x56375869b330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a38a3bca82909cfa server_cookie=568e3aee4d4673d3 in_seq=0 out_seq=0 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.291+0000 7f8e81ffb640 1 -- 172.21.15.94:0/2399009834 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637585f5990 con 0x56375829ff60 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.291+0000 7f8e81ffb640 10 client.18066 handle_command_reply: tid=0 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.291+0000 7f8ea7b8a740 1 client.18066 shutdown 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 msgr2=0x563758694a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8e84012dc0 tx=0x7f8e84004730 comp rx=0 tx=0).stop 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 shutdown_connections 2024-04-25T13:48:55.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56375829ff60 0x56375869b330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582e1390 0x563758694a30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x56375868dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8e81ffb640 0 client.18066 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/2399009834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582b84d0 0x56375868d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 >> 172.21.15.94:0/2399009834 conn(0x5637582d54e0 msgr2=0x563758681f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 shutdown_connections 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/2399009834 wait complete. 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.292+0000 7f8ea7b8a740 20 client.18066 trim_cache size 0 max 16384 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.305+0000 7f8ea7b8a740 1 Processor -- start 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 -- start start 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56375829ff60 0x563758688310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x5637586a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1ab4e0 con 0x5637582b84d0 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1ad020 con 0x56375829ff60 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1abbb0 con 0x5637582c06b0 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60748/0 (socket says 172.21.15.94:60748) 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 -- 172.21.15.94:0/3549877179 learned_addr learned my addr 172.21.15.94:0/3549877179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e93fff640 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x5637586a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e93fff640 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56375829ff60 0x563758688310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 -- 172.21.15.94:0/3549877179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 msgr2=0x5637586a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x5637586a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 -- 172.21.15.94:0/3549877179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56375829ff60 msgr2=0x563758688310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56375829ff60 0x563758688310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 -- 172.21.15.94:0/3549877179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375868b6a0 con 0x5637582b84d0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005ab0 tx=0x7f8e8c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b92916a78c7e638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e83fff640 1 -- 172.21.15.94:0/3549877179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c054070 con 0x5637582b84d0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e83fff640 1 -- 172.21.15.94:0/3549877179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8c040050 con 0x5637582b84d0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.306+0000 7f8e83fff640 1 -- 172.21.15.94:0/3549877179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c035070 con 0x5637582b84d0 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/3549877179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 msgr2=0x563758688850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005ab0 tx=0x7f8e8c02a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/3549877179 shutdown_connections 2024-04-25T13:48:55.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582c06b0 0x5637586a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637582b84d0 0x563758688850 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/3549877179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56375829ff60 0x563758688310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/3549877179 >> 172.21.15.94:0/3549877179 conn(0x5637582d54e0 msgr2=0x563758683790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/3549877179 shutdown_connections 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/3549877179 wait complete. 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.307+0000 7f8ea7b8a740 1 -- start start 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 Processor -- start 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582b84d0 0x563758692e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582c06b0 0x563758693360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1ac950 con 0x56375829ff60 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1ac280 con 0x5637582c06b0 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1a94c0 con 0x5637582b84d0 2024-04-25T13:48:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8ea7b8a740 10 client.0 _ll_register_callbacks cb 0x7f8e840129d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60750/0 (socket says 172.21.15.94:60750) 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 -- 172.21.15.94:0/1124909441 learned_addr learned my addr 172.21.15.94:0/1124909441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582c06b0 0x563758693360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582b84d0 0x563758692e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 -- 172.21.15.94:0/1124909441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582b84d0 msgr2=0x563758692e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582b84d0 0x563758692e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 -- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582c06b0 msgr2=0x563758693360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582c06b0 0x563758693360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 -- 172.21.15.94:0/1124909441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637587016d0 con 0x56375829ff60 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e93fff640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8e84001eb0 tx=0x7f8e84014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dfd0635a578c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e840035f0 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.308+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e84003790 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8c040050 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e84003990 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 10 client.18072 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563758702910 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8c040050 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e8403c070 con 0x56375829ff60 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 10 client.18072 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 10 client.18072 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 20 client.18072 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5637582e1390 0x563758705330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 4 client.18072 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637585ef800 con 0x5637582e1390 2024-04-25T13:48:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.309+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5637582e1390 0x563758705330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8e81ffb640 10 client.18072 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8e9a59c640 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5637582e1390 0x563758705330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c143a5847b43fc88 server_cookie=417bec4957bcf1de in_seq=0 out_seq=0 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8e81ffb640 1 -- 172.21.15.94:0/1124909441 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637585ef800 con 0x5637582e1390 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8e81ffb640 10 client.18072 handle_command_reply: tid=0 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 client.18072 shutdown 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 msgr2=0x5637586928e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8e84001eb0 tx=0x7f8e84014040 comp rx=0 tx=0).stop 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 shutdown_connections 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5637582e1390 0x563758705330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637582c06b0 0x563758693360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637582b84d0 0x563758692e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8e81ffb640 0 client.18072 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 --2- 172.21.15.94:0/1124909441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56375829ff60 0x5637586928e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 >> 172.21.15.94:0/1124909441 conn(0x5637582d54e0 msgr2=0x563758683790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 shutdown_connections 2024-04-25T13:48:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 1 -- 172.21.15.94:0/1124909441 wait complete. 2024-04-25T13:48:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.310+0000 7f8ea7b8a740 20 client.18072 trim_cache size 0 max 16384 2024-04-25T13:48:55.320 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 msgr2=0x7f8e880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 0x7f8e880b2000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e940a7100 tx=0x7f8e94005800 comp rx=0 tx=0).stop 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 msgr2=0x7f8e9c1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e940 tx=0x7f8e9400ee00 comp rx=0 tx=0).stop 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 shutdown_connections 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e880afaf0 0x7f8e880b2000 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e9c108920 0x7f8e9c1a1820 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e9c103c20 0x7f8e9c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 --2- 172.21.15.94:0/4068096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e9c1029e0 0x7f8e9c06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 >> 172.21.15.94:0/4068096869 conn(0x7f8e9c0fdbd0 msgr2=0x7f8e9c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 shutdown_connections 2024-04-25T13:48:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.311+0000 7f8ea0ce0640 1 -- 172.21.15.94:0/4068096869 wait complete. 2024-04-25T13:48:55.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 -- 172.21.15.94:0/857957054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 msgr2=0x7f51441047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 --2- 172.21.15.94:0/857957054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f51441047f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f513402a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 -- 172.21.15.94:0/857957054 shutdown_connections 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 --2- 172.21.15.94:0/857957054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 0x7f514410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 --2- 172.21.15.94:0/857957054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 0x7f5144105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 --2- 172.21.15.94:0/857957054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f51441047f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 -- 172.21.15.94:0/857957054 >> 172.21.15.94:0/857957054 conn(0x7f51440fece0 msgr2=0x7f5144101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.436+0000 7f5148b5f640 1 -- 172.21.15.94:0/857957054 shutdown_connections 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.437+0000 7f5148b5f640 1 -- 172.21.15.94:0/857957054 wait complete. 2024-04-25T13:48:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.437+0000 7f5148b5f640 1 Processor -- start 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.437+0000 7f5148b5f640 1 -- start start 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 0x7f514419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 0x7f51441a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f514410f5f0 con 0x7f5144103af0 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f514410f470 con 0x7f5144109a30 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f514410f770 con 0x7f5144104d30 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60754/0 (socket says 172.21.15.94:60754) 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 -- 172.21.15.94:0/4140207311 learned_addr learned my addr 172.21.15.94:0/4140207311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5142d9d640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 0x7f51441a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f513bfff640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 0x7f514419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 -- 172.21.15.94:0/4140207311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 msgr2=0x7f514419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 0x7f514419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 -- 172.21.15.94:0/4140207311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 msgr2=0x7f51441a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 0x7f51441a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 -- 172.21.15.94:0/4140207311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5144075ee0 con 0x7f5144103af0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f514259c640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f5134004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f513b7fe640 1 -- 172.21.15.94:0/4140207311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513403f030 con 0x7f5144103af0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f513b7fe640 1 -- 172.21.15.94:0/4140207311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5134031910 con 0x7f5144103af0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5144076170 con 0x7f5144103af0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f513b7fe640 1 -- 172.21.15.94:0/4140207311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035070 con 0x7f5144103af0 2024-04-25T13:48:55.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.438+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5144076660 con 0x7f5144103af0 2024-04-25T13:48:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.441+0000 7f514fa0b740 1 Processor -- start 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.441+0000 7f514fa0b740 1 -- start start 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f513b7fe640 1 -- 172.21.15.94:0/4140207311 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5134042030 con 0x7f5144103af0 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559008c71d80 con 0x55900894a6b0 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559008c2ba20 con 0x5590089424d0 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514fa0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559008c6dd40 con 0x55900896b390 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f513bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47528/0 (socket says 172.21.15.94:47528) 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 -- 172.21.15.94:0/1406176991 learned_addr learned my addr 172.21.15.94:0/1406176991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f514259c640 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 -- 172.21.15.94:0/1406176991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 msgr2=0x559008d0dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d0dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 -- 172.21.15.94:0/1406176991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 msgr2=0x559008d0ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d0ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 -- 172.21.15.94:0/1406176991 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559008c36c30 con 0x55900896b390 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9ecc3372692743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.442+0000 7f513b7fe640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 0x7f51300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f5138ff9640 1 -- 172.21.15.94:0/1406176991 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c004d20 con 0x55900896b390 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514259c640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 0x7f51300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f5138ff9640 1 -- 172.21.15.94:0/1406176991 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c004050 con 0x55900896b390 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f5138ff9640 1 -- 172.21.15.94:0/1406176991 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c005260 con 0x55900896b390 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f513b7fe640 1 -- 172.21.15.94:0/4140207311 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f51340ff030 con 0x7f5144103af0 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 -- 172.21.15.94:0/1406176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 msgr2=0x559008d0d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514259c640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 0x7f51300b2000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x559008d0d530 tx=0x7f51340b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c0118d0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 -- 172.21.15.94:0/1406176991 shutdown_connections 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d0ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d0dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.443+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1406176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d0d810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- 172.21.15.94:0/1406176991 >> 172.21.15.94:0/1406176991 conn(0x55900895f4e0 msgr2=0x559008d09020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- 172.21.15.94:0/1406176991 shutdown_connections 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- 172.21.15.94:0/1406176991 wait complete. 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- start start 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 Processor -- start 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d17a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559008d0f530 con 0x5590089424d0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559008bf8dc0 con 0x55900894a6b0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559008d0f6b0 con 0x55900896b390 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.444+0000 7f514fa0b740 10 client.0 _ll_register_callbacks cb 0x559008c14600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60768/0 (socket says 172.21.15.94:60768) 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 -- 172.21.15.94:0/77607480 learned_addr learned my addr 172.21.15.94:0/77607480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f513bfff640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d17a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 -- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 msgr2=0x559008d1e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 -- 172.21.15.94:0/77607480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 msgr2=0x559008d17a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d17a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 -- 172.21.15.94:0/77607480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590089833f0 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5142d9d640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f513c01b040 tx=0x7f513c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79930e5364d8a396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c00f6e0 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f513c00f880 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f513c004050 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f513c004690 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f514fa0b740 10 client.18084 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559008a4d7d0 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.445+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c004050 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f513c0056d0 con 0x5590089424d0 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 10 client.18084 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 10 client.18084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:48:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 20 client.18084 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x559008929f60 0x559008d24ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 4 client.18084 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559008c7f990 con 0x559008929f60 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f513bfff640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x559008929f60 0x559008d24ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f5129ffb640 10 client.18084 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.446+0000 7f513bfff640 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x559008929f60 0x559008d24ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8b726803da7ff5f server_cookie=cbc50e853d67a2f3 in_seq=0 out_seq=0 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f5129ffb640 1 -- 172.21.15.94:0/77607480 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559008c7f990 con 0x559008929f60 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f5129ffb640 10 client.18084 handle_command_reply: tid=0 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 client.18084 shutdown 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 msgr2=0x559008d17510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f513c01b040 tx=0x7f513c009d40 comp rx=0 tx=0).stop 2024-04-25T13:48:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 shutdown_connections 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x559008929f60 0x559008d24ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d17a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f5129ffb640 0 client.18084 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 --2- 172.21.15.94:0/77607480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d17510 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.447+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 >> 172.21.15.94:0/77607480 conn(0x55900895f4e0 msgr2=0x559008d0bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.448+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 shutdown_connections 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.448+0000 7f514fa0b740 1 -- 172.21.15.94:0/77607480 wait complete. 2024-04-25T13:48:55.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.448+0000 7f514fa0b740 20 client.18084 trim_cache size 0 max 16384 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 Processor -- start 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 -- start start 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d08760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ad770 con 0x55900894a6b0 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51441af2b0 con 0x5590089424d0 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f514fa0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ade40 con 0x55900896b390 2024-04-25T13:48:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f513bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f513bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60782/0 (socket says 172.21.15.94:60782) 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f513bfff640 1 -- 172.21.15.94:0/2950838765 learned_addr learned my addr 172.21.15.94:0/2950838765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.461+0000 7f5142d9d640 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d08760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f5142d9d640 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 -- 172.21.15.94:0/2950838765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 msgr2=0x559008d08760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d08760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 -- 172.21.15.94:0/2950838765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 msgr2=0x559008d170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d170e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 -- 172.21.15.94:0/2950838765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559008d09bb0 con 0x55900894a6b0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f513bfff640 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f512c009c40 tx=0x7f512c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814e1a2a57f95e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f512bfff640 1 -- 172.21.15.94:0/2950838765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c003b20 con 0x55900894a6b0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f512bfff640 1 -- 172.21.15.94:0/2950838765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c0054d0 con 0x55900894a6b0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.462+0000 7f512bfff640 1 -- 172.21.15.94:0/2950838765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c01e900 con 0x55900894a6b0 2024-04-25T13:48:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- 172.21.15.94:0/2950838765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 msgr2=0x559008d17620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f512c009c40 tx=0x7f512c004410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- 172.21.15.94:0/2950838765 shutdown_connections 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d08760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55900894a6b0 0x559008d17620 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- 172.21.15.94:0/2950838765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590089424d0 0x559008d170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- 172.21.15.94:0/2950838765 >> 172.21.15.94:0/2950838765 conn(0x55900895f4e0 msgr2=0x559008d12270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- 172.21.15.94:0/2950838765 shutdown_connections 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- 172.21.15.94:0/2950838765 wait complete. 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- start start 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 Processor -- start 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d1bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aebe0 con 0x5590089424d0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae510 con 0x55900894a6b0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ab750 con 0x55900896b390 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f514fa0b740 10 client.0 _ll_register_callbacks cb 0x7f512c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.463+0000 7f5142d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60792/0 (socket says 172.21.15.94:60792) 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 -- 172.21.15.94:0/1039772251 learned_addr learned my addr 172.21.15.94:0/1039772251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f513bfff640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d1bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 -- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 msgr2=0x559008d1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 -- 172.21.15.94:0/1039772251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 msgr2=0x559008d1bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d1bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 -- 172.21.15.94:0/1039772251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559008d1d110 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5142d9d640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbb1cfdb7ef7d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c044060 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f513c00f4e0 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512c0054d0 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f513c009ea0 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f514fa0b740 10 client.18090 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559008d1e3c0 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.464+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c0054d0 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f513c03c040 con 0x5590089424d0 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 10 client.18090 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 10 client.18090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:48:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 20 client.18090 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5590088ff3a0 0x559008d8fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 4 client.18090 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559008c79800 con 0x5590088ff3a0 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f513bfff640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5590088ff3a0 0x559008d8fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f5129ffb640 10 client.18090 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f513bfff640 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5590088ff3a0 0x559008d8fae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15e4ec0e78f7ffcd server_cookie=9e6c3cc2ae63c76a in_seq=0 out_seq=0 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f5129ffb640 1 -- 172.21.15.94:0/1039772251 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559008c79800 con 0x5590088ff3a0 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f5129ffb640 10 client.18090 handle_command_reply: tid=0 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.465+0000 7f514fa0b740 1 client.18090 shutdown 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 msgr2=0x559008d1b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c004e80 comp rx=0 tx=0).stop 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 shutdown_connections 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5590088ff3a0 0x559008d8fae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55900896b390 0x559008d1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5129ffb640 0 client.18090 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55900894a6b0 0x559008d1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 --2- 172.21.15.94:0/1039772251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5590089424d0 0x559008d1b6e0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 >> 172.21.15.94:0/1039772251 conn(0x55900895f4e0 msgr2=0x559008d170e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 shutdown_connections 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 1 -- 172.21.15.94:0/1039772251 wait complete. 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f514fa0b740 20 client.18090 trim_cache size 0 max 16384 2024-04-25T13:48:55.473 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 msgr2=0x7f51300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 0x7f51300b2000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x559008d0d530 tx=0x7f51340b0040 comp rx=0 tx=0).stop 2024-04-25T13:48:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 msgr2=0x7f514419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f5134004270 comp rx=0 tx=0).stop 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 shutdown_connections 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51300afaf0 0x7f51300b2000 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5144109a30 0x7f51441a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5144104d30 0x7f514419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 --2- 172.21.15.94:0/4140207311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5144103af0 0x7f514419a250 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 >> 172.21.15.94:0/4140207311 conn(0x7f51440fece0 msgr2=0x7f5144105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 shutdown_connections 2024-04-25T13:48:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.466+0000 7f5148b5f640 1 -- 172.21.15.94:0/4140207311 wait complete. 2024-04-25T13:48:55.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:48:55.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 -- 172.21.15.94:0/3944546591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee81029e0 msgr2=0x7f2ee81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 --2- 172.21.15.94:0/3944546591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee81029e0 0x7f2ee81036e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 -- 172.21.15.94:0/3944546591 shutdown_connections 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 --2- 172.21.15.94:0/3944546591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee8108920 0x7f2ee810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 --2- 172.21.15.94:0/3944546591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 --2- 172.21.15.94:0/3944546591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee81029e0 0x7f2ee81036e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 -- 172.21.15.94:0/3944546591 >> 172.21.15.94:0/3944546591 conn(0x7f2ee80fdbd0 msgr2=0x7f2ee8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.588+0000 7f2eee73f640 1 -- 172.21.15.94:0/3944546591 shutdown_connections 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 -- 172.21.15.94:0/3944546591 wait complete. 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 Processor -- start 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 -- start start 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee81029e0 0x7f2ee8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810e4e0 con 0x7f2ee8108920 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810e360 con 0x7f2ee8103c20 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.589+0000 7f2eee73f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810e660 con 0x7f2ee81029e0 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee81029e0 0x7f2ee8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60802/0 (socket says 172.21.15.94:60802) 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eeccdc640 1 -- 172.21.15.94:0/1045802667 learned_addr learned my addr 172.21.15.94:0/1045802667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 -- 172.21.15.94:0/1045802667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee81029e0 msgr2=0x7f2ee8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee81029e0 0x7f2ee8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 -- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 msgr2=0x7f2ee81a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 -- 172.21.15.94:0/1045802667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee81a0fa0 con 0x7f2ee8103c20 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee81999a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9f0 tx=0x7f2edc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee57fa640 1 -- 172.21.15.94:0/1045802667 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00ccc0 con 0x7f2ee8103c20 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a1290 con 0x7f2ee8103c20 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee57fa640 1 -- 172.21.15.94:0/1045802667 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc0108e0 con 0x7f2ee8103c20 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2ee57fa640 1 -- 172.21.15.94:0/1045802667 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc018880 con 0x7f2ee8103c20 2024-04-25T13:48:55.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.590+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a1830 con 0x7f2ee8103c20 2024-04-25T13:48:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 Processor -- start 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 -- start start 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ee57fa640 1 -- 172.21.15.94:0/1045802667 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2edc010430 con 0x7f2ee8103c20 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975494d0 0x564e97914d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564e97878d80 con 0x564e975516b0 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564e97832a20 con 0x564e975494d0 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e97874d40 con 0x564e97572390 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47602/0 (socket says 172.21.15.94:47602) 2024-04-25T13:48:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60814/0 (socket says 172.21.15.94:60814) 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 -- 172.21.15.94:0/836458309 learned_addr learned my addr 172.21.15.94:0/836458309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975494d0 0x564e97914d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 -- 172.21.15.94:0/836458309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975494d0 msgr2=0x564e97914d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975494d0 0x564e97914d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 -- 172.21.15.94:0/836458309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 msgr2=0x564e97915a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2eeccdc640 1 -- 172.21.15.94:0/836458309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9783dc30 con 0x564e97572390 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.593+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ee57fa640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 0x7f2ed00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000f710 tx=0x7f2ee000fbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840a87130e2c177a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 0x7f2ed00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ee57fa640 1 -- 172.21.15.94:0/1045802667 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2edc0df010 con 0x7f2ee8103c20 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ece7fc640 1 -- 172.21.15.94:0/836458309 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0003b70 con 0x564e97572390 2024-04-25T13:48:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ece7fc640 1 -- 172.21.15.94:0/836458309 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ee000ddd0 con 0x564e97572390 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.594+0000 7f2ece7fc640 1 -- 172.21.15.94:0/836458309 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0004780 con 0x564e97572390 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- 172.21.15.94:0/836458309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 msgr2=0x564e97914810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000f710 tx=0x7f2ee000fbd0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 0x7f2ed00b2000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005ab0 tx=0x7f2ed8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- 172.21.15.94:0/836458309 shutdown_connections 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975516b0 0x564e97915a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975494d0 0x564e97914d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/836458309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e97572390 0x564e97914810 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- 172.21.15.94:0/836458309 >> 172.21.15.94:0/836458309 conn(0x564e975664e0 msgr2=0x564e97910020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- 172.21.15.94:0/836458309 shutdown_connections 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- 172.21.15.94:0/836458309 wait complete. 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 -- start start 2024-04-25T13:48:55.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.595+0000 7f2ef55f5740 1 Processor -- start 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e9791eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97925a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564e97916530 con 0x564e975494d0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564e977ffdc0 con 0x564e97572390 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e979166b0 con 0x564e975516b0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 10 client.0 _ll_register_callbacks cb 0x564e9781b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60818/0 (socket says 172.21.15.94:60818) 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 -- 172.21.15.94:0/685159687 learned_addr learned my addr 172.21.15.94:0/685159687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97925a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e9791eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 -- 172.21.15.94:0/685159687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 msgr2=0x564e9791eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e9791eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 -- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 msgr2=0x564e97925a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97925a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 -- 172.21.15.94:0/685159687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9758a3f0 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8772f4ecfacead6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8035040 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2ed8004440 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.596+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee000ddd0 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ed80035f0 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 10 client.18102 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e976547d0 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ee000ddd0 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ed8044070 con 0x564e975494d0 2024-04-25T13:48:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 10 client.18102 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 10 client.18102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 20 client.18102 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e97530f60 0x564e9792c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 4 client.18102 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e97886990 con 0x564e97530f60 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.597+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e97530f60 0x564e9792c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.598+0000 7f2ec77fe640 10 client.18102 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.598+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e97530f60 0x564e9792c340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7602d2cf8ee836b server_cookie=82e0729faf8b1265 in_seq=0 out_seq=0 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ec77fe640 1 -- 172.21.15.94:0/685159687 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e97886990 con 0x564e97530f60 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ec77fe640 10 client.18102 handle_command_reply: tid=0 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 client.18102 shutdown 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 msgr2=0x564e9791e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 shutdown_connections 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e97530f60 0x564e9792c340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97925a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ec77fe640 0 client.18102 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e9791eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/685159687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e5f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 >> 172.21.15.94:0/685159687 conn(0x564e975664e0 msgr2=0x564e97912f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 shutdown_connections 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 1 -- 172.21.15.94:0/685159687 wait complete. 2024-04-25T13:48:55.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.599+0000 7f2ef55f5740 20 client.18102 trim_cache size 0 max 16384 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.612+0000 7f2ef55f5740 1 Processor -- start 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.612+0000 7f2ef55f5740 1 -- start start 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e97930af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97931030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81aa470 con 0x564e975494d0 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81abfb0 con 0x564e97572390 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81aab40 con 0x564e975516b0 2024-04-25T13:48:55.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60826/0 (socket says 172.21.15.94:60826) 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 -- 172.21.15.94:0/511712588 learned_addr learned my addr 172.21.15.94:0/511712588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e97930af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97931030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 -- 172.21.15.94:0/511712588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 msgr2=0x564e97930af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e97930af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 -- 172.21.15.94:0/511712588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 msgr2=0x564e97931030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97931030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 -- 172.21.15.94:0/511712588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e97919d80 con 0x564e975494d0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97931030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005f70 tx=0x7f2ed8004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcd4f9f391a1e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ecdffb640 1 -- 172.21.15.94:0/511712588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8035040 con 0x564e975494d0 2024-04-25T13:48:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ecdffb640 1 -- 172.21.15.94:0/511712588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed805d020 con 0x564e975494d0 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.613+0000 7f2ecdffb640 1 -- 172.21.15.94:0/511712588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8003ed0 con 0x564e975494d0 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- 172.21.15.94:0/511712588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 msgr2=0x564e9791e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005f70 tx=0x7f2ed8004480 comp rx=0 tx=0).stop 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- 172.21.15.94:0/511712588 shutdown_connections 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e97572390 0x564e97931030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975516b0 0x564e97930af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/511712588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e975494d0 0x564e9791e8d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- 172.21.15.94:0/511712588 >> 172.21.15.94:0/511712588 conn(0x564e975664e0 msgr2=0x564e9791e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- 172.21.15.94:0/511712588 shutdown_connections 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- 172.21.15.94:0/511712588 wait complete. 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.614+0000 7f2ef55f5740 1 -- start start 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 Processor -- start 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975494d0 0x564e9798dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975516b0 0x564e9798e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ab8e0 con 0x564e97572390 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ab210 con 0x564e975516b0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81a8450 con 0x564e975494d0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ef55f5740 10 client.0 _ll_register_callbacks cb 0x7f2ed8032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60828/0 (socket says 172.21.15.94:60828) 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975494d0 0x564e9798dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 -- 172.21.15.94:0/3040372240 learned_addr learned my addr 172.21.15.94:0/3040372240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ee77fe640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975516b0 0x564e9798e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 -- 172.21.15.94:0/3040372240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975494d0 msgr2=0x564e9798dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975494d0 0x564e9798dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 -- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975516b0 msgr2=0x564e9798e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975516b0 0x564e9798e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 -- 172.21.15.94:0/3040372240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e97995790 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2eeccdc640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564e97915860 tx=0x7f2ee000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09bebdfb6c83f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.615+0000 7f2ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0028020 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f2ee0004a00 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed805d020 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ee0004c00 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 10 client.18108 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e97996a80 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed805d020 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ee0004350 con 0x564e97572390 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 10 client.18108 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 10 client.18108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 20 client.18108 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e975063a0 0x564e97999270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 4 client.18108 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e97880800 con 0x564e975063a0 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.616+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e975063a0 0x564e97999270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ec77fe640 10 client.18108 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ee7fff640 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e975063a0 0x564e97999270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5a72202e8bad9ec server_cookie=b2afd3c0203139fa in_seq=0 out_seq=0 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ec77fe640 1 -- 172.21.15.94:0/3040372240 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e97880800 con 0x564e975063a0 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ec77fe640 10 client.18108 handle_command_reply: tid=0 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 client.18108 shutdown 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 msgr2=0x564e979951d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564e97915860 tx=0x7f2ee000db50 comp rx=0 tx=0).stop 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 shutdown_connections 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x564e975063a0 0x564e97999270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e97572390 0x564e979951d0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e975516b0 0x564e9798e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 --2- 172.21.15.94:0/3040372240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e975494d0 0x564e9798dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ec77fe640 0 client.18108 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 >> 172.21.15.94:0/3040372240 conn(0x564e975664e0 msgr2=0x564e97912e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 shutdown_connections 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 1 -- 172.21.15.94:0/3040372240 wait complete. 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.617+0000 7f2ef55f5740 20 client.18108 trim_cache size 0 max 16384 2024-04-25T13:48:55.624 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 msgr2=0x7f2ed00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 0x7f2ed00b2000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005ab0 tx=0x7f2ed8031400 comp rx=0 tx=0).stop 2024-04-25T13:48:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 msgr2=0x7f2ee81999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee81999a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e9f0 tx=0x7f2edc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 shutdown_connections 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2ed00afaf0 0x7f2ed00b2000 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ee8108920 0x7f2ee81a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ee8103c20 0x7f2ee81999a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 --2- 172.21.15.94:0/1045802667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee81029e0 0x7f2ee8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 >> 172.21.15.94:0/1045802667 conn(0x7f2ee80fdbd0 msgr2=0x7f2ee8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 shutdown_connections 2024-04-25T13:48:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.618+0000 7f2eee73f640 1 -- 172.21.15.94:0/1045802667 wait complete. 2024-04-25T13:48:55.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:55 smithi094 ceph-mon[29195]: pgmap v561: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:48:55.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1379118588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:55.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.760+0000 7f0c0a934640 1 -- 172.21.15.94:0/3559819606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 msgr2=0x7f0c041036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.760+0000 7f0c0a934640 1 --2- 172.21.15.94:0/3559819606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c041036e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402a410 comp rx=0 tx=0).stop 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 -- 172.21.15.94:0/3559819606 shutdown_connections 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 --2- 172.21.15.94:0/3559819606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c0410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 --2- 172.21.15.94:0/3559819606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 0x7f0c04104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 --2- 172.21.15.94:0/3559819606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c041036e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 -- 172.21.15.94:0/3559819606 >> 172.21.15.94:0/3559819606 conn(0x7f0c040fdbd0 msgr2=0x7f0c04100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 -- 172.21.15.94:0/3559819606 shutdown_connections 2024-04-25T13:48:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 -- 172.21.15.94:0/3559819606 wait complete. 2024-04-25T13:48:55.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 Processor -- start 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.761+0000 7f0c0a934640 1 -- start start 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 0x7f0c041999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c041a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410e530 con 0x7f0c041029e0 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410e3b0 con 0x7f0c04103c20 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410e6b0 con 0x7f0c04108920 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60840/0 (socket says 172.21.15.94:60840) 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 -- 172.21.15.94:0/2798136118 learned_addr learned my addr 172.21.15.94:0/2798136118 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c037fe640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 0x7f0c041999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c041a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 -- 172.21.15.94:0/2798136118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 msgr2=0x7f0c041a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c041a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 -- 172.21.15.94:0/2798136118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 msgr2=0x7f0c041999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 0x7f0c041999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 -- 172.21.15.94:0/2798136118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041a0f50 con 0x7f0c041029e0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c041a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4031440 tx=0x7f0bf4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c017fa640 1 -- 172.21.15.94:0/2798136118 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf403f070 con 0x7f0c041029e0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c017fa640 1 -- 172.21.15.94:0/2798136118 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf4031c50 con 0x7f0c041029e0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c017fa640 1 -- 172.21.15.94:0/2798136118 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4035030 con 0x7f0c041029e0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.762+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041a11e0 con 0x7f0c041029e0 2024-04-25T13:48:55.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.763+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a1720 con 0x7f0c041029e0 2024-04-25T13:48:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 Processor -- start 2024-04-25T13:48:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 -- start start 2024-04-25T13:48:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c017fa640 1 -- 172.21.15.94:0/2798136118 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0bf4042070 con 0x7f0c041029e0 2024-04-25T13:48:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 0x55de0ec14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec15a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55de0eb78d80 con 0x55de0e872390 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de0eb32a20 con 0x55de0e8494d0 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c117e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de0eb74d40 con 0x55de0e8516b0 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60848/0 (socket says 172.21.15.94:60848) 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c08ed1640 1 -- 172.21.15.94:0/2817972798 learned_addr learned my addr 172.21.15.94:0/2817972798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 0x55de0ec14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.765+0000 7f0c037fe640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec15a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 -- 172.21.15.94:0/2817972798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 msgr2=0x55de0ec15a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec15a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 -- 172.21.15.94:0/2817972798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 msgr2=0x55de0ec14d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 0x55de0ec14d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 -- 172.21.15.94:0/2817972798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de0eb3dc30 con 0x55de0e872390 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 0x55de0ec14d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c017fa640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 0x7f0bec0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009fb0 tx=0x7f0bfc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff7f0ed1df58f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 0x7f0bec0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0c017fa640 1 -- 172.21.15.94:0/2798136118 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0bf40ff4d0 con 0x7f0c041029e0 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0bea7fc640 1 -- 172.21.15.94:0/2817972798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc016070 con 0x55de0e872390 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0bea7fc640 1 -- 172.21.15.94:0/2817972798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc012bc0 con 0x55de0e872390 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.766+0000 7f0bea7fc640 1 -- 172.21.15.94:0/2817972798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc011c10 con 0x55de0e872390 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- 172.21.15.94:0/2817972798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 msgr2=0x55de0ec14810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c03fff640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 0x7f0bec0b2080 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4102fd0 tx=0x7f0bf40d0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009fb0 tx=0x7f0bfc010800 comp rx=0 tx=0).stop 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- 172.21.15.94:0/2817972798 shutdown_connections 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec15a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8494d0 0x55de0ec14d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 --2- 172.21.15.94:0/2817972798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec14810 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- 172.21.15.94:0/2817972798 >> 172.21.15.94:0/2817972798 conn(0x55de0e8664e0 msgr2=0x55de0ec10020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- 172.21.15.94:0/2817972798 shutdown_connections 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- 172.21.15.94:0/2817972798 wait complete. 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.767+0000 7f0c117e6740 1 -- start start 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 Processor -- start 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec25a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55de0ec16530 con 0x55de0e8494d0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de0eaffdc0 con 0x55de0e872390 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de0ec166b0 con 0x55de0e8516b0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c117e6740 10 client.0 _ll_register_callbacks cb 0x55de0eb1b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60852/0 (socket says 172.21.15.94:60852) 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 -- 172.21.15.94:0/3495172024 learned_addr learned my addr 172.21.15.94:0/3495172024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec25a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c037fe640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 -- 172.21.15.94:0/3495172024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 msgr2=0x55de0ec1ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec1ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 -- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 msgr2=0x55de0ec25a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec25a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 -- 172.21.15.94:0/3495172024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0e88a3f0 con 0x55de0e8494d0 2024-04-25T13:48:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc011010 tx=0x7f0bfc004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40faec42af6f9bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.768+0000 7f0be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc016070 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0bfc011b90 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc012bc0 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc011d90 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 10 client.18120 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de0e9547d0 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc012bc0 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bfc00f040 con 0x55de0e8494d0 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 10 client.18120 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 20 client.18120 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e830f60 0x55de0ec2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 4 client.18120 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.769+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0eb86990 con 0x55de0e830f60 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.770+0000 7f0c037fe640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e830f60 0x55de0ec2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.770+0000 7f0c037fe640 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e830f60 0x55de0ec2bff0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add5f02db9c17527 server_cookie=e36e6e02403a8e00 in_seq=0 out_seq=0 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.770+0000 7f0be37fe640 10 client.18120 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.771+0000 7f0be37fe640 1 -- 172.21.15.94:0/3495172024 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de0eb86990 con 0x55de0e830f60 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.771+0000 7f0be37fe640 10 client.18120 handle_command_reply: tid=0 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.771+0000 7f0c117e6740 1 client.18120 shutdown 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 msgr2=0x55de0ec1e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc011010 tx=0x7f0bfc004cc0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 shutdown_connections 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e830f60 0x55de0ec2bff0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec25a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8516b0 0x55de0ec1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 --2- 172.21.15.94:0/3495172024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8494d0 0x55de0ec1e590 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0be37fe640 0 client.18120 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 >> 172.21.15.94:0/3495172024 conn(0x55de0e8664e0 msgr2=0x55de0ec0f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 shutdown_connections 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 1 -- 172.21.15.94:0/3495172024 wait complete. 2024-04-25T13:48:55.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.772+0000 7f0c117e6740 20 client.18120 trim_cache size 0 max 16384 2024-04-25T13:48:55.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 Processor -- start 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 -- start start 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8516b0 0x55de0ec1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a9300 con 0x55de0e872390 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041aae40 con 0x55de0e8516b0 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c117e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a99d0 con 0x55de0e8494d0 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60860/0 (socket says 172.21.15.94:60860) 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 -- 172.21.15.94:0/642452205 learned_addr learned my addr 172.21.15.94:0/642452205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c037fe640 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8516b0 0x55de0ec1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 -- 172.21.15.94:0/642452205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 msgr2=0x55de0ec1e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec1e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 -- 172.21.15.94:0/642452205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8516b0 msgr2=0x55de0ec1e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8516b0 0x55de0ec1e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 -- 172.21.15.94:0/642452205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de0ec143d0 con 0x55de0e872390 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.786+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000f80 tx=0x7f0bfc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7104900e013b852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0be9ffb640 1 -- 172.21.15.94:0/642452205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc03a070 con 0x55de0e872390 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0be9ffb640 1 -- 172.21.15.94:0/642452205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc01f030 con 0x55de0e872390 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0be9ffb640 1 -- 172.21.15.94:0/642452205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc016070 con 0x55de0e872390 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 -- 172.21.15.94:0/642452205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 msgr2=0x55de0ec30610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000f80 tx=0x7f0bfc0021f0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 -- 172.21.15.94:0/642452205 shutdown_connections 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e872390 0x55de0ec30610 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e8516b0 0x55de0ec1e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 --2- 172.21.15.94:0/642452205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 -- 172.21.15.94:0/642452205 >> 172.21.15.94:0/642452205 conn(0x55de0e8664e0 msgr2=0x55de0ec192f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 -- 172.21.15.94:0/642452205 shutdown_connections 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.787+0000 7f0c117e6740 1 -- 172.21.15.94:0/642452205 wait complete. 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 -- start start 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 Processor -- start 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041aa770 con 0x55de0e8516b0 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041aa0a0 con 0x55de0e872390 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a72e0 con 0x55de0e8494d0 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c117e6740 10 client.0 _ll_register_callbacks cb 0x7f0bf4102e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:55.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60862/0 (socket says 172.21.15.94:60862) 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.788+0000 7f0c037fe640 1 -- 172.21.15.94:0/1236752860 learned_addr learned my addr 172.21.15.94:0/1236752860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 -- 172.21.15.94:0/1236752860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 msgr2=0x55de0ec22150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec22150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 -- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 msgr2=0x55de0ec921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 -- 172.21.15.94:0/1236752860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0ec92730 con 0x55de0e8516b0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c037fe640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x55de0ec10db0 tx=0x7f0bf8012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2975255af17fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:55.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8005020 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0bf80051c0 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc01f030 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bf80053c0 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c117e6740 10 client.18126 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de0ec93a60 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.789+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc01f030 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bf8003850 con 0x55de0e8516b0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 10 client.18126 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 10 client.18126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 20 client.18126 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e8063a0 0x55de0ec96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 4 client.18126 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de0eb80800 con 0x55de0e8063a0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e8063a0 0x55de0ec96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:55.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0c08ed1640 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e8063a0 0x55de0ec96450 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9ef444b0f3529a server_cookie=ddbdfbdad0cf523f in_seq=0 out_seq=0 2024-04-25T13:48:55.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.790+0000 7f0be37fe640 10 client.18126 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.791+0000 7f0be37fe640 1 -- 172.21.15.94:0/1236752860 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de0eb80800 con 0x55de0e8063a0 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.791+0000 7f0be37fe640 10 client.18126 handle_command_reply: tid=0 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.791+0000 7f0c117e6740 1 client.18126 shutdown 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 msgr2=0x55de0ec22690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x55de0ec10db0 tx=0x7f0bf8012a10 comp rx=0 tx=0).stop 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 shutdown_connections 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55de0e8063a0 0x55de0ec96450 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0e872390 0x55de0ec921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de0e8516b0 0x55de0ec22690 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0be37fe640 0 client.18126 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 --2- 172.21.15.94:0/1236752860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de0e8494d0 0x55de0ec22150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 >> 172.21.15.94:0/1236752860 conn(0x55de0e8664e0 msgr2=0x55de0ec1e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 shutdown_connections 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 1 -- 172.21.15.94:0/1236752860 wait complete. 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c117e6740 20 client.18126 trim_cache size 0 max 16384 2024-04-25T13:48:55.798 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 msgr2=0x7f0bec0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 0x7f0bec0b2080 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4102fd0 tx=0x7f0bf40d0040 comp rx=0 tx=0).stop 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 msgr2=0x7f0c04199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4031440 tx=0x7f0bf4003c10 comp rx=0 tx=0).stop 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 shutdown_connections 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0bec0afb70 0x7f0bec0b2080 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c04108920 0x7f0c041a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c04103c20 0x7f0c041999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 --2- 172.21.15.94:0/2798136118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c041029e0 0x7f0c04199480 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 >> 172.21.15.94:0/2798136118 conn(0x7f0c040fdbd0 msgr2=0x7f0c04104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 shutdown_connections 2024-04-25T13:48:55.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:55.792+0000 7f0c0a934640 1 -- 172.21.15.94:0/2798136118 wait complete. 2024-04-25T13:48:55.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 827736 12.91 MB/sec execute 218 sec latency 60.954 ms 2024-04-25T13:48:55.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:55 smithi137 ceph-mon[32568]: pgmap v561: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:48:55.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1379118588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:55.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:55 smithi138 ceph-mon[32977]: pgmap v561: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-04-25T13:48:55.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1379118588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:48:56.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 830200 12.90 MB/sec execute 219 sec latency 51.721 ms 2024-04-25T13:48:56.804 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:48:56.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:56.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 -- 172.21.15.94:0/3541303680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c109790 msgr2=0x7f994c10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 --2- 172.21.15.94:0/3541303680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c109790 0x7f994c10bca0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9944009ca0 tx=0x7f994402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 -- 172.21.15.94:0/3541303680 shutdown_connections 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 --2- 172.21.15.94:0/3541303680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c109790 0x7f994c10bca0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 --2- 172.21.15.94:0/3541303680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c102370 0x7f994c109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 --2- 172.21.15.94:0/3541303680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c101e70 0x7f994c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.920+0000 7f99530af640 1 -- 172.21.15.94:0/3541303680 >> 172.21.15.94:0/3541303680 conn(0x7f994c0fd380 msgr2=0x7f994c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.921+0000 7f99530af640 1 -- 172.21.15.94:0/3541303680 shutdown_connections 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.921+0000 7f99530af640 1 -- 172.21.15.94:0/3541303680 wait complete. 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.921+0000 7f99530af640 1 Processor -- start 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.921+0000 7f99530af640 1 -- start start 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 0x7f994c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c109790 0x7f994c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10f650 con 0x7f994c102370 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10f4d0 con 0x7f994c101e70 2024-04-25T13:48:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f99530af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10f7d0 con 0x7f994c109790 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60890/0 (socket says 172.21.15.94:60890) 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f9950e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 0x7f994c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 -- 172.21.15.94:0/3972932545 learned_addr learned my addr 172.21.15.94:0/3972932545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f995164c640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c109790 0x7f994c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 -- 172.21.15.94:0/3972932545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c109790 msgr2=0x7f994c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c109790 0x7f994c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 -- 172.21.15.94:0/3972932545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 msgr2=0x7f994c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 0x7f994c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f994bfff640 1 -- 172.21.15.94:0/3972932545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a1ce0 con 0x7f994c102370 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.922+0000 7f9950e4b640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 0x7f994c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f994bfff640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f993800d950 tx=0x7f993800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f9949ffb640 1 -- 172.21.15.94:0/3972932545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938017030 con 0x7f994c102370 2024-04-25T13:48:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f9949ffb640 1 -- 172.21.15.94:0/3972932545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9938004020 con 0x7f994c102370 2024-04-25T13:48:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a1fd0 con 0x7f994c102370 2024-04-25T13:48:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f9949ffb640 1 -- 172.21.15.94:0/3972932545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380047a0 con 0x7f994c102370 2024-04-25T13:48:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.923+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1a2440 con 0x7f994c102370 2024-04-25T13:48:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.926+0000 7f9949ffb640 1 -- 172.21.15.94:0/3972932545 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9938009a30 con 0x7f994c102370 2024-04-25T13:48:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.926+0000 7f9959f67740 1 Processor -- start 2024-04-25T13:48:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.926+0000 7f9959f67740 1 -- start start 2024-04-25T13:48:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd4a2410 0x55e6bd8434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd843a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd84a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd7a8b00 con 0x55e6bd481730 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd744920 con 0x55e6bd479550 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9959f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd7a4ac0 con 0x55e6bd4a2410 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9950e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd843a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9950e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd843a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52922/0 (socket says 172.21.15.94:52922) 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9950e4b640 1 -- 172.21.15.94:0/64192580 learned_addr learned my addr 172.21.15.94:0/64192580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f994bfff640 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd84a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f995164c640 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd4a2410 0x55e6bd8434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.927+0000 7f9949ffb640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 0x7f993c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f995164c640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 0x7f993c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f9949ffb640 1 -- 172.21.15.94:0/3972932545 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f99380de6d0 con 0x7f994c102370 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 -- 172.21.15.94:0/64192580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd4a2410 msgr2=0x55e6bd8434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd4a2410 0x55e6bd8434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 -- 172.21.15.94:0/64192580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 msgr2=0x55e6bd843a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd843a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 -- 172.21.15.94:0/64192580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bd5827e0 con 0x55e6bd481730 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f994bfff640 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd84a920 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f99380a3d20 tx=0x7f99380044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2c32f760dd2cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f9936ffd640 1 -- 172.21.15.94:0/64192580 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380a6400 con 0x55e6bd481730 2024-04-25T13:48:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f995164c640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 0x7f993c0b2080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f994402f540 tx=0x7f994402f450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.928+0000 7f9936ffd640 1 -- 172.21.15.94:0/64192580 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99380af250 con 0x55e6bd481730 2024-04-25T13:48:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.929+0000 7f9936ffd640 1 -- 172.21.15.94:0/64192580 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380a6220 con 0x55e6bd481730 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- 172.21.15.94:0/64192580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 msgr2=0x55e6bd84a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd84a920 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f99380a3d20 tx=0x7f99380044d0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- 172.21.15.94:0/64192580 shutdown_connections 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd84a920 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd843a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 --2- 172.21.15.94:0/64192580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd4a2410 0x55e6bd8434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- 172.21.15.94:0/64192580 >> 172.21.15.94:0/64192580 conn(0x55e6bd496560 msgr2=0x55e6bd83ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- 172.21.15.94:0/64192580 shutdown_connections 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- 172.21.15.94:0/64192580 wait complete. 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 -- start start 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.930+0000 7f9959f67740 1 Processor -- start 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd84d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd8546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd8546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f994bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9950e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd84d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd84b2b0 con 0x55e6bd4a2410 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd7425d0 con 0x55e6bd479550 2024-04-25T13:48:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6bd84b430 con 0x55e6bd481730 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f994bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47724/0 (socket says 172.21.15.94:47724) 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f994bfff640 1 -- 172.21.15.94:0/629175240 learned_addr learned my addr 172.21.15.94:0/629175240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9950e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd84d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52932/0 (socket says 172.21.15.94:52932) 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f9959f67740 10 client.0 _ll_register_callbacks cb 0x55e6bd730080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 -- 172.21.15.94:0/629175240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 msgr2=0x55e6bd84d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 -- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 msgr2=0x55e6bd84d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd84d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f995164c640 1 -- 172.21.15.94:0/629175240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bd5e2f70 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.931+0000 7f994bfff640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f995164c640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd8546e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f994402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2f7e6f096fb366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f992bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994402fe90 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f99440307d0 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e6bd7b6710 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99440309d0 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f9959f67740 10 client.18138 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e6bd47a170 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.932+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e6bd7b6710 con 0x55e6bd4a2410 2024-04-25T13:48:56.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f994404ac40 con 0x55e6bd4a2410 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 10 client.18138 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 20 client.18138 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd4611f0 0x55e6bd85ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 4 client.18138 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6bd7b0580 con 0x55e6bd4611f0 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9950e4b640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd4611f0 0x55e6bd85ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f992bfff640 10 client.18138 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.933+0000 7f9950e4b640 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd4611f0 0x55e6bd85ab60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413fb5da767a688a server_cookie=158a163063459344 in_seq=0 out_seq=0 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.935+0000 7f992bfff640 1 -- 172.21.15.94:0/629175240 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6bd7b0580 con 0x55e6bd4611f0 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.935+0000 7f992bfff640 10 client.18138 handle_command_reply: tid=0 2024-04-25T13:48:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.935+0000 7f9959f67740 1 client.18138 shutdown 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.935+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 msgr2=0x55e6bd8546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd8546e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9944006010 tx=0x7f994402f310 comp rx=0 tx=0).stop 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 shutdown_connections 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd4611f0 0x55e6bd85ab60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd8546e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd481730 0x55e6bd84d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 --2- 172.21.15.94:0/629175240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd479550 0x55e6bd84d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 >> 172.21.15.94:0/629175240 conn(0x55e6bd496560 msgr2=0x55e6bd83def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f992bfff640 0 client.18138 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 shutdown_connections 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 1 -- 172.21.15.94:0/629175240 wait complete. 2024-04-25T13:48:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.936+0000 7f9959f67740 20 client.18138 trim_cache size 0 max 16384 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 Processor -- start 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 -- start start 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd85f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd481730 0x55e6bd854600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1a92e0 con 0x55e6bd4a2410 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aae20 con 0x55e6bd481730 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9959f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1a99b0 con 0x55e6bd479550 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60918/0 (socket says 172.21.15.94:60918) 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 -- 172.21.15.94:0/1153122402 learned_addr learned my addr 172.21.15.94:0/1153122402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f994bfff640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd481730 0x55e6bd854600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f9950e4b640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd85f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 -- 172.21.15.94:0/1153122402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 msgr2=0x55e6bd85f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd85f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 -- 172.21.15.94:0/1153122402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd481730 msgr2=0x55e6bd854600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd481730 0x55e6bd854600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.955+0000 7f995164c640 1 -- 172.21.15.94:0/1153122402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bd84c2c0 con 0x55e6bd4a2410 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.956+0000 7f9950e4b640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd85f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:56.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.956+0000 7f995164c640 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9944005c30 tx=0x7f99440099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18355386492852dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.956+0000 7f99367fc640 1 -- 172.21.15.94:0/1153122402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994402fe90 con 0x55e6bd4a2410 2024-04-25T13:48:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.956+0000 7f99367fc640 1 -- 172.21.15.94:0/1153122402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9944036050 con 0x55e6bd4a2410 2024-04-25T13:48:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.956+0000 7f99367fc640 1 -- 172.21.15.94:0/1153122402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944055a90 con 0x55e6bd4a2410 2024-04-25T13:48:56.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- 172.21.15.94:0/1153122402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 msgr2=0x55e6bd85f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9944005c30 tx=0x7f99440099f0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- 172.21.15.94:0/1153122402 shutdown_connections 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd4a2410 0x55e6bd85f850 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd481730 0x55e6bd854600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 --2- 172.21.15.94:0/1153122402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd85f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- 172.21.15.94:0/1153122402 >> 172.21.15.94:0/1153122402 conn(0x55e6bd496560 msgr2=0x55e6bd847780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- 172.21.15.94:0/1153122402 shutdown_connections 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- 172.21.15.94:0/1153122402 wait complete. 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 -- start start 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.957+0000 7f9959f67740 1 Processor -- start 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd8534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd4a2410 0x55e6bd8c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aa750 con 0x55e6bd481730 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aa080 con 0x55e6bd4a2410 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1a72c0 con 0x55e6bd479550 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9959f67740 10 client.0 _ll_register_callbacks cb 0x7f99440095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60926/0 (socket says 172.21.15.94:60926) 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 -- 172.21.15.94:0/2873151249 learned_addr learned my addr 172.21.15.94:0/2873151249 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f995164c640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd4a2410 0x55e6bd8c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9950e4b640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd8534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 -- 172.21.15.94:0/2873151249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 msgr2=0x55e6bd8534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd8534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 -- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd4a2410 msgr2=0x55e6bd8c0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:56.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd4a2410 0x55e6bd8c0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 -- 172.21.15.94:0/2873151249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bd8c1400 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f9950e4b640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd8534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f994bfff640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f99380a6a40 tx=0x7f99380a6220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5be06416a16cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f992bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.958+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380a71b0 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f99380a5d10 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9944036050 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99380a65c0 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 10 client.18144 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e6bd8c25e0 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9944036050 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f99380a2420 con 0x55e6bd481730 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 10 client.18144 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:56.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 20 client.18144 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd436630 0x55e6bd8c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 4 client.18144 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6bd72bac0 con 0x55e6bd436630 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.959+0000 7f9950e4b640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd436630 0x55e6bd8c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.960+0000 7f992bfff640 10 client.18144 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.960+0000 7f9950e4b640 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd436630 0x55e6bd8c4ba0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec2610d3bdd1954 server_cookie=d64334a8d363201f in_seq=0 out_seq=0 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f992bfff640 1 -- 172.21.15.94:0/2873151249 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6bd72bac0 con 0x55e6bd436630 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f992bfff640 10 client.18144 handle_command_reply: tid=0 2024-04-25T13:48:56.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 client.18144 shutdown 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 msgr2=0x55e6bd853a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f99380a6a40 tx=0x7f99380a6220 comp rx=0 tx=0).stop 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 shutdown_connections 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55e6bd436630 0x55e6bd8c4ba0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6bd4a2410 0x55e6bd8c0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e6bd481730 0x55e6bd853a30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f992bfff640 0 client.18144 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 --2- 172.21.15.94:0/2873151249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6bd479550 0x55e6bd8534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 >> 172.21.15.94:0/2873151249 conn(0x55e6bd496560 msgr2=0x55e6bd841c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 shutdown_connections 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 1 -- 172.21.15.94:0/2873151249 wait complete. 2024-04-25T13:48:56.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f9959f67740 20 client.18144 trim_cache size 0 max 16384 2024-04-25T13:48:56.971 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:56.971 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:48:56.971 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "5fc9d815-887a-4f78-ab8b-c12ca2c2ad17", 2024-04-25T13:48:56.971 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:48:56.971 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 msgr2=0x7f993c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 0x7f993c0b2080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f994402f540 tx=0x7f994402f450 comp rx=0 tx=0).stop 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.962+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 msgr2=0x7f994c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f993800d950 tx=0x7f993800de10 comp rx=0 tx=0).stop 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 shutdown_connections 2024-04-25T13:48:56.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f993c0afb70 0x7f993c0b2080 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f994c109790 0x7f994c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102370 0x7f994c19a6e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 --2- 172.21.15.94:0/3972932545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f994c101e70 0x7f994c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 >> 172.21.15.94:0/3972932545 conn(0x7f994c0fd380 msgr2=0x7f994c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 shutdown_connections 2024-04-25T13:48:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:56.963+0000 7f99530af640 1 -- 172.21.15.94:0/3972932545 wait complete. 2024-04-25T13:48:56.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:48:56.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:48:57.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/3944513481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04103af0 msgr2=0x7f1e041047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/3944513481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04103af0 0x7f1e041047f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009900 tx=0x7f1e0002a410 comp rx=0 tx=0).stop 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/3944513481 shutdown_connections 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/3944513481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04109a30 0x7f1e0410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/3944513481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04104d30 0x7f1e04105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/3944513481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04103af0 0x7f1e041047f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.090+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/3944513481 >> 172.21.15.94:0/3944513481 conn(0x7f1e040fece0 msgr2=0x7f1e04101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/3944513481 shutdown_connections 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/3944513481 wait complete. 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 Processor -- start 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- start start 2024-04-25T13:48:57.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 0x7f1e041a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410f5f0 con 0x7f1e04104d30 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410f470 con 0x7f1e04109a30 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e0c1ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0410f770 con 0x7f1e04103af0 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.091+0000 7f1e09769640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60956/0 (socket says 172.21.15.94:60956) 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47760/0 (socket says 172.21.15.94:47760) 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 -- 172.21.15.94:0/2611169356 learned_addr learned my addr 172.21.15.94:0/2611169356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 0x7f1e041a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 -- 172.21.15.94:0/2611169356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 msgr2=0x7f1e0419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 -- 172.21.15.94:0/2611169356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 msgr2=0x7f1e041a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 0x7f1e041a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 -- 172.21.15.94:0/2611169356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e041a1da0 con 0x7f1e04104d30 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 0x7f1e041a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e09769640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9f0 tx=0x7f1df400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1dfaffd640 1 -- 172.21.15.94:0/2611169356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400cd80 con 0x7f1e04104d30 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e041a2090 con 0x7f1e04104d30 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1dfaffd640 1 -- 172.21.15.94:0/2611169356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df40108e0 con 0x7f1e04104d30 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1dfaffd640 1 -- 172.21.15.94:0/2611169356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4018980 con 0x7f1e04104d30 2024-04-25T13:48:57.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.092+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a2630 con 0x7f1e04104d30 2024-04-25T13:48:57.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 Processor -- start 2024-04-25T13:48:57.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1dfaffd640 1 -- 172.21.15.94:0/2611169356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1df4010430 con 0x7f1e04104d30 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 -- start start 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589ddff5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc2b4d0 0x5589ddff5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5589ddf5ad80 con 0x5589ddc336b0 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589ddf148b0 con 0x5589ddc54390 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e1307f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589ddf56d40 con 0x5589ddc2b4d0 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e09769640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc2b4d0 0x5589ddff5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e09769640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60958/0 (socket says 172.21.15.94:60958) 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e0a76b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589ddff5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.095+0000 7f1e09769640 1 -- 172.21.15.94:0/3169612502 learned_addr learned my addr 172.21.15.94:0/3169612502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 -- 172.21.15.94:0/3169612502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc2b4d0 msgr2=0x5589ddff5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc2b4d0 0x5589ddff5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 -- 172.21.15.94:0/3169612502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 msgr2=0x5589ddff5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589ddff5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 -- 172.21.15.94:0/3169612502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ddee5a40 con 0x5589ddc336b0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589ddff5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1dfaffd640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 0x7f1df00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 0x7f1df00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1e09769640 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1df40e71b0 tx=0x7f1df40e7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86b07b413e4207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1dfaffd640 1 -- 172.21.15.94:0/2611169356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1df40defc0 con 0x7f1e04104d30 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1deffff640 1 -- 172.21.15.94:0/3169612502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4026020 con 0x5589ddc336b0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1deffff640 1 -- 172.21.15.94:0/3169612502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4010720 con 0x5589ddc336b0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.096+0000 7f1deffff640 1 -- 172.21.15.94:0/3169612502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4026020 con 0x5589ddc336b0 2024-04-25T13:48:57.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 -- 172.21.15.94:0/3169612502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 msgr2=0x5589ddff6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 0x7f1df00b2000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5589ddff6ab0 tx=0x7f1dfc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1df40e71b0 tx=0x7f1df40e7670 comp rx=0 tx=0).stop 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 -- 172.21.15.94:0/3169612502 shutdown_connections 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc336b0 0x5589ddff6c70 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc2b4d0 0x5589ddff5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 --2- 172.21.15.94:0/3169612502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589ddff5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 -- 172.21.15.94:0/3169612502 >> 172.21.15.94:0/3169612502 conn(0x5589ddc484e0 msgr2=0x5589ddff1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 -- 172.21.15.94:0/3169612502 shutdown_connections 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.097+0000 7f1e1307f740 1 -- 172.21.15.94:0/3169612502 wait complete. 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 -- start start 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 Processor -- start 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589ddfffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de006d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5589ddff7780 con 0x5589ddc2b4d0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589ddee1d10 con 0x5589ddc336b0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589ddff7900 con 0x5589ddc54390 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e1307f740 10 client.0 _ll_register_callbacks cb 0x5589ddf0e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60972/0 (socket says 172.21.15.94:60972) 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.098+0000 7f1e09f6a640 1 -- 172.21.15.94:0/539151925 learned_addr learned my addr 172.21.15.94:0/539151925 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09769640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589ddfffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de006d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 -- 172.21.15.94:0/539151925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 msgr2=0x5589de006d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de006d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 -- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 msgr2=0x5589ddfffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589ddfffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 -- 172.21.15.94:0/539151925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ddd367d0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09769640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589ddfffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009230 tx=0x7f1e000317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33298c298cbc7846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1ded7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00035040 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1e00004450 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589ddf68990 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e000035f0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e1307f740 10 client.18156 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.099+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589ddc6c3f0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5589ddf68990 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e00044070 con 0x5589ddc2b4d0 2024-04-25T13:48:57.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 10 client.18156 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 20 client.18156 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddc12f60 0x5589de00d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 4 client.18156 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ddf62800 con 0x5589ddc12f60 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1e09769640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddc12f60 0x5589de00d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.100+0000 7f1ded7fa640 10 client.18156 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.101+0000 7f1e09769640 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddc12f60 0x5589de00d1c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24107954db40ce server_cookie=983e0f3c43bec4b2 in_seq=0 out_seq=0 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.101+0000 7f1ded7fa640 1 -- 172.21.15.94:0/539151925 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589ddf62800 con 0x5589ddc12f60 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.101+0000 7f1ded7fa640 10 client.18156 handle_command_reply: tid=0 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.101+0000 7f1e1307f740 1 client.18156 shutdown 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 msgr2=0x5589ddfff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009230 tx=0x7f1e000317d0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 shutdown_connections 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddc12f60 0x5589de00d1c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de006d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589ddfffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 --2- 172.21.15.94:0/539151925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddfff870 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1ded7fa640 0 client.18156 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 >> 172.21.15.94:0/539151925 conn(0x5589ddc484e0 msgr2=0x5589ddff4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 shutdown_connections 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 1 -- 172.21.15.94:0/539151925 wait complete. 2024-04-25T13:48:57.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.102+0000 7f1e1307f740 20 client.18156 trim_cache size 0 max 16384 2024-04-25T13:48:57.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 Processor -- start 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 -- start start 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc336b0 0x5589ddff4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589de006c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041ab5a0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041ad0e0 con 0x5589ddc54390 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e1307f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041abc70 con 0x5589ddc336b0 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60988/0 (socket says 172.21.15.94:60988) 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.120+0000 7f1e09f6a640 1 -- 172.21.15.94:0/480329341 learned_addr learned my addr 172.21.15.94:0/480329341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09769640 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc336b0 0x5589ddff4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589de006c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 -- 172.21.15.94:0/480329341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc336b0 msgr2=0x5589ddff4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc336b0 0x5589ddff4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 -- 172.21.15.94:0/480329341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 msgr2=0x5589de006c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589de006c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 -- 172.21.15.94:0/480329341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ddc2c0f0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006010 tx=0x7f1e0002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d404863c1a02eb4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1def7fe640 1 -- 172.21.15.94:0/480329341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00053070 con 0x5589ddc2b4d0 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1def7fe640 1 -- 172.21.15.94:0/480329341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00046cd0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.121+0000 7f1def7fe640 1 -- 172.21.15.94:0/480329341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00035040 con 0x5589ddc2b4d0 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- 172.21.15.94:0/480329341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 msgr2=0x5589ddffac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006010 tx=0x7f1e0002a410 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- 172.21.15.94:0/480329341 shutdown_connections 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc54390 0x5589de006c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc336b0 0x5589ddff4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 --2- 172.21.15.94:0/480329341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589ddffac30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- 172.21.15.94:0/480329341 >> 172.21.15.94:0/480329341 conn(0x5589ddc484e0 msgr2=0x5589ddff4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- 172.21.15.94:0/480329341 shutdown_connections 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- 172.21.15.94:0/480329341 wait complete. 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 -- start start 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.122+0000 7f1e1307f740 1 Processor -- start 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589de004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de073520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041aca10 con 0x5589ddc2b4d0 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041ac340 con 0x5589ddc336b0 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041a9580 con 0x5589ddc54390 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e1307f740 10 client.0 _ll_register_callbacks cb 0x7f1e00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60990/0 (socket says 172.21.15.94:60990) 2024-04-25T13:48:57.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 -- 172.21.15.94:0/1116284662 learned_addr learned my addr 172.21.15.94:0/1116284662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09769640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589de004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e0a76b640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 -- 172.21.15.94:0/1116284662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 msgr2=0x5589de073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de073520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 -- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 msgr2=0x5589de004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589de004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 -- 172.21.15.94:0/1116284662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ddf1b4b0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1e09f6a640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e0002a930 tx=0x7f1e000317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dce94bc73d52cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.123+0000 7f1ded7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0005c070 con 0x5589ddc2b4d0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f1e000039a0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e00046cd0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e00035040 con 0x5589ddc2b4d0 2024-04-25T13:48:57.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 10 client.18162 fetch_fsmap learned FSMap version 27 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589de074960 con 0x5589ddc2b4d0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00046cd0 con 0x5589ddc2b4d0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1e00044030 con 0x5589ddc2b4d0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 10 client.18162 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 20 client.18162 populate_metadata read hostname 'smithi094' 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddbe83a0 0x5589de077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 4 client.18162 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589ddedd4a0 con 0x5589ddbe83a0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.124+0000 7f1e09769640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddbe83a0 0x5589de077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1ded7fa640 10 client.18162 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e09769640 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddbe83a0 0x5589de077250 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8db50ae29cc143 server_cookie=e8c698986981e67d in_seq=0 out_seq=0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1ded7fa640 1 -- 172.21.15.94:0/1116284662 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5589ddedd4a0 con 0x5589ddbe83a0 2024-04-25T13:48:57.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1ded7fa640 10 client.18162 handle_command_reply: tid=0 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 client.18162 shutdown 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 msgr2=0x5589de004420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e0002a930 tx=0x7f1e000317d0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 shutdown_connections 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5589ddbe83a0 0x5589de077250 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589ddc54390 0x5589de073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589ddc336b0 0x5589de004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1ded7fa640 0 client.18162 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 --2- 172.21.15.94:0/1116284662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589ddc2b4d0 0x5589de004420 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 >> 172.21.15.94:0/1116284662 conn(0x5589ddc484e0 msgr2=0x5589ddfef9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 shutdown_connections 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 1 -- 172.21.15.94:0/1116284662 wait complete. 2024-04-25T13:48:57.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.125+0000 7f1e1307f740 20 client.18162 trim_cache size 0 max 16384 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "5fc9d815-887a-4f78-ab8b-c12ca2c2ad17": { 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "5fc9d815-887a-4f78-ab8b-c12ca2c2ad17", 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:48:57.132 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:48:57.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 msgr2=0x7f1df00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 0x7f1df00b2000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5589ddff6ab0 tx=0x7f1dfc009500 comp rx=0 tx=0).stop 2024-04-25T13:48:57.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 msgr2=0x7f1e0419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:57.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9f0 tx=0x7f1df400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 shutdown_connections 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1df00afaf0 0x7f1df00b2000 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e04109a30 0x7f1e041a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e04104d30 0x7f1e0419a7f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 --2- 172.21.15.94:0/2611169356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103af0 0x7f1e0419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 >> 172.21.15.94:0/2611169356 conn(0x7f1e040fece0 msgr2=0x7f1e04105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 shutdown_connections 2024-04-25T13:48:57.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:48:57.126+0000 7f1e0c1ce640 1 -- 172.21.15.94:0/2611169356 wait complete. 2024-04-25T13:48:57.137 INFO:tasks.cephfs.filesystem:scrub status for tag:5fc9d815-887a-4f78-ab8b-c12ca2c2ad17 - {'path': '/', 'tag': '5fc9d815-887a-4f78-ab8b-c12ca2c2ad17', 'options': 'recursive,force'} 2024-04-25T13:48:57.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 832007 12.88 MB/sec execute 220 sec latency 14.519 ms 2024-04-25T13:48:57.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:57 smithi137 ceph-mon[32568]: pgmap v562: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 370 op/s 2024-04-25T13:48:57.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:57 smithi094 ceph-mon[29195]: pgmap v562: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 370 op/s 2024-04-25T13:48:57.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:57 smithi138 ceph-mon[32977]: pgmap v562: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 370 op/s 2024-04-25T13:48:58.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 834754 12.90 MB/sec execute 221 sec latency 9.892 ms 2024-04-25T13:48:58.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:58 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:48:58.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:58 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:58.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:58 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:48:58.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:58 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:58 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:48:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:58 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:58 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:48:58.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:58 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:58 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:48:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:58 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:58 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:48:58.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:58 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:48:59.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 837626 12.91 MB/sec execute 222 sec latency 18.214 ms 2024-04-25T13:48:59.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:48:59 smithi137 ceph-mon[32568]: pgmap v563: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 329 op/s 2024-04-25T13:48:59.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:48:59 smithi094 ceph-mon[29195]: pgmap v563: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 329 op/s 2024-04-25T13:48:59.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:48:59 smithi138 ceph-mon[32977]: pgmap v563: 97 pgs: 97 active+clean; 37 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 329 op/s 2024-04-25T13:49:00.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 839516 12.90 MB/sec execute 223 sec latency 13.969 ms 2024-04-25T13:49:01.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 841760 12.90 MB/sec execute 224 sec latency 63.989 ms 2024-04-25T13:49:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:01 smithi137 ceph-mon[32568]: pgmap v564: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 412 op/s 2024-04-25T13:49:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:49:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:01 smithi094 ceph-mon[29195]: pgmap v564: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 412 op/s 2024-04-25T13:49:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:49:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:01 smithi138 ceph-mon[32977]: pgmap v564: 97 pgs: 97 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 412 op/s 2024-04-25T13:49:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:49:02.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 844054 12.88 MB/sec execute 225 sec latency 21.609 ms 2024-04-25T13:49:03.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 846080 12.88 MB/sec execute 226 sec latency 13.197 ms 2024-04-25T13:49:03.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:03 smithi137 ceph-mon[32568]: pgmap v565: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:49:03.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:03 smithi094 ceph-mon[29195]: pgmap v565: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:49:03.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:03 smithi138 ceph-mon[32977]: pgmap v565: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2024-04-25T13:49:04.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 848932 12.90 MB/sec execute 227 sec latency 10.470 ms 2024-04-25T13:49:05.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 851556 12.90 MB/sec execute 228 sec latency 23.050 ms 2024-04-25T13:49:05.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:05 smithi137 ceph-mon[32568]: pgmap v566: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-04-25T13:49:05.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:05 smithi094 ceph-mon[29195]: pgmap v566: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-04-25T13:49:05.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:05 smithi138 ceph-mon[32977]: pgmap v566: 97 pgs: 97 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 385 op/s 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.264+0000 7f95b0ab6640 1 -- 172.21.15.94:0/2894890134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac108970 msgr2=0x7f95ac10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.264+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/2894890134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac108970 0x7f95ac10ad90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009ca0 tx=0x7f95a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 -- 172.21.15.94:0/2894890134 shutdown_connections 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/2894890134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac108970 0x7f95ac10ad90 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/2894890134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac103c70 0x7f95ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/2894890134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 -- 172.21.15.94:0/2894890134 >> 172.21.15.94:0/2894890134 conn(0x7f95ac0fdc00 msgr2=0x7f95ac1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 -- 172.21.15.94:0/2894890134 shutdown_connections 2024-04-25T13:49:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 -- 172.21.15.94:0/2894890134 wait complete. 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 Processor -- start 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.265+0000 7f95b0ab6640 1 -- start start 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 0x7f95ac1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10e560 con 0x7f95ac103c70 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10e3e0 con 0x7f95ac102a30 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95b0ab6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10e6e0 con 0x7f95ac108970 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 0x7f95ac1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36020/0 (socket says 172.21.15.94:36020) 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 -- 172.21.15.94:0/4003424263 learned_addr learned my addr 172.21.15.94:0/4003424263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95a3fff640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58968/0 (socket says 172.21.15.94:58968) 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 -- 172.21.15.94:0/4003424263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 msgr2=0x7f95ac1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 0x7f95ac1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 -- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 msgr2=0x7f95ac199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aa59c640 1 -- 172.21.15.94:0/4003424263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac1a0c50 con 0x7f95ac102a30 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95a3fff640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.266+0000 7f95aad9d640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 0x7f95ac1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95aa59c640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9f0 tx=0x7f959c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95a37fe640 1 -- 172.21.15.94:0/4003424263 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ccc0 con 0x7f95ac102a30 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac1a0f40 con 0x7f95ac102a30 2024-04-25T13:49:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95a37fe640 1 -- 172.21.15.94:0/4003424263 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c0035f0 con 0x7f95ac102a30 2024-04-25T13:49:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95a37fe640 1 -- 172.21.15.94:0/4003424263 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ce60 con 0x7f95ac102a30 2024-04-25T13:49:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.267+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac1a13a0 con 0x7f95ac102a30 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.269+0000 7f95b796c740 1 Processor -- start 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.269+0000 7f95b796c740 1 -- start start 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.269+0000 7f95a37fe640 1 -- 172.21.15.94:0/4003424263 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f959c010690 con 0x7f95ac102a30 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5122970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ef5088460 con 0x564ef4d592f0 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564ef5042590 con 0x564ef4d614d0 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95b796c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ef5084420 con 0x564ef4d821b0 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5122ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58978/0 (socket says 172.21.15.94:58978) 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95aa59c640 1 -- 172.21.15.94:0/1002661934 learned_addr learned my addr 172.21.15.94:0/1002661934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36034/0 (socket says 172.21.15.94:36034) 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95aad9d640 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 -- 172.21.15.94:0/1002661934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 msgr2=0x564ef5122970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5122970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 -- 172.21.15.94:0/1002661934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 msgr2=0x564ef5122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 -- 172.21.15.94:0/1002661934 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef504f0d0 con 0x564ef4d614d0 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a37fe640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 0x7f95980b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9594009fa0 tx=0x7f95940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed263e939f175687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.270+0000 7f95aad9d640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 0x7f95980b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95a37fe640 1 -- 172.21.15.94:0/4003424263 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f959c0def90 con 0x7f95ac102a30 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95a0ff9640 1 -- 172.21.15.94:0/1002661934 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940040f0 con 0x564ef4d614d0 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95a0ff9640 1 -- 172.21.15.94:0/1002661934 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940054d0 con 0x564ef4d614d0 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95a0ff9640 1 -- 172.21.15.94:0/1002661934 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594004d10 con 0x564ef4d614d0 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 -- 172.21.15.94:0/1002661934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 msgr2=0x564ef5129df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95aad9d640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 0x7f95980b1ff0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x564ef5123b40 tx=0x7f95a4009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9594009fa0 tx=0x7f95940118d0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 -- 172.21.15.94:0/1002661934 shutdown_connections 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5129df0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.271+0000 7f95b796c740 1 --2- 172.21.15.94:0/1002661934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- 172.21.15.94:0/1002661934 >> 172.21.15.94:0/1002661934 conn(0x564ef4d76300 msgr2=0x564ef511e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- 172.21.15.94:0/1002661934 shutdown_connections 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- 172.21.15.94:0/1002661934 wait complete. 2024-04-25T13:49:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- start start 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 Processor -- start 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d592f0 0x564ef512c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef512cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ef512a840 con 0x564ef4d821b0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564ef500fb60 con 0x564ef4d614d0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ef512a9c0 con 0x564ef4d592f0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95b796c740 10 client.0 _ll_register_callbacks cb 0x564ef50342b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.272+0000 7f95aad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d592f0 0x564ef512c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58986/0 (socket says 172.21.15.94:58986) 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 -- 172.21.15.94:0/293586620 learned_addr learned my addr 172.21.15.94:0/293586620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95a3fff640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef512cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 -- 172.21.15.94:0/293586620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d592f0 msgr2=0x564ef512c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d592f0 0x564ef512c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 -- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 msgr2=0x564ef512cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef512cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 -- 172.21.15.94:0/293586620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef4d9a210 con 0x564ef4d821b0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95a3fff640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef512cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95aad9d640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f95a4035000 tx=0x7f95a402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fe3bf9d430579b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f9591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004250 con 0x564ef4d821b0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f95a40043f0 con 0x564ef4d821b0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.273+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95940054d0 con 0x564ef4d821b0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95a40045f0 con 0x564ef4d821b0 2024-04-25T13:49:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 10 client.18174 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ef4e66620 con 0x564ef4d821b0 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940054d0 con 0x564ef4d821b0 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f95a4052070 con 0x564ef4d821b0 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 10 client.18174 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 10 client.18174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 20 client.18174 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d40d80 0x564ef513a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 4 client.18174 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef5096070 con 0x564ef4d40d80 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.274+0000 7f95aa59c640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d40d80 0x564ef513a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.275+0000 7f95aa59c640 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d40d80 0x564ef513a790 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd97d0803d977a6f server_cookie=96345d0ab6c1ebcc in_seq=0 out_seq=0 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.275+0000 7f9591ffb640 10 client.18174 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.276+0000 7f9591ffb640 1 -- 172.21.15.94:0/293586620 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ef5096070 con 0x564ef4d40d80 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.276+0000 7f9591ffb640 10 client.18174 handle_command_reply: tid=0 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 client.18174 shutdown 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 msgr2=0x564ef5133b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f95a4035000 tx=0x7f95a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 shutdown_connections 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d40d80 0x564ef513a790 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d821b0 0x564ef5133b10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef512cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 --2- 172.21.15.94:0/293586620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d592f0 0x564ef512c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f9591ffb640 0 client.18174 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 >> 172.21.15.94:0/293586620 conn(0x564ef4d76300 msgr2=0x564ef511d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 shutdown_connections 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 1 -- 172.21.15.94:0/293586620 wait complete. 2024-04-25T13:49:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.277+0000 7f95b796c740 20 client.18174 trim_cache size 0 max 16384 2024-04-25T13:49:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.293+0000 7f95b796c740 1 Processor -- start 2024-04-25T13:49:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.293+0000 7f95b796c740 1 -- start start 2024-04-25T13:49:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 0x564ef511dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5121100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1aa450 con 0x564ef4d614d0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1abf90 con 0x564ef4d592f0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95b796c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1aab20 con 0x564ef4d821b0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58998/0 (socket says 172.21.15.94:58998) 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 -- 172.21.15.94:0/2471547388 learned_addr learned my addr 172.21.15.94:0/2471547388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95aa59c640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 0x564ef511dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95aad9d640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 -- 172.21.15.94:0/2471547388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 msgr2=0x564ef5121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 -- 172.21.15.94:0/2471547388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 msgr2=0x564ef511dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 0x564ef511dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 -- 172.21.15.94:0/2471547388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef5123b70 con 0x564ef4d614d0 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95aad9d640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95aa59c640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 0x564ef511dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f95a3fff640 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f95940058c0 tx=0x7f9594009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d3b93f65026958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f9593fff640 1 -- 172.21.15.94:0/2471547388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594029070 con 0x564ef4d614d0 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f9593fff640 1 -- 172.21.15.94:0/2471547388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940054f0 con 0x564ef4d614d0 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.294+0000 7f9593fff640 1 -- 172.21.15.94:0/2471547388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959401d040 con 0x564ef4d614d0 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 -- 172.21.15.94:0/2471547388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 msgr2=0x564ef511e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f95940058c0 tx=0x7f9594009d20 comp rx=0 tx=0).stop 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 -- 172.21.15.94:0/2471547388 shutdown_connections 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef5121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d614d0 0x564ef511e0e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 --2- 172.21.15.94:0/2471547388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d592f0 0x564ef511dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 -- 172.21.15.94:0/2471547388 >> 172.21.15.94:0/2471547388 conn(0x564ef4d76300 msgr2=0x564ef5126ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 -- 172.21.15.94:0/2471547388 shutdown_connections 2024-04-25T13:49:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.295+0000 7f95b796c740 1 -- 172.21.15.94:0/2471547388 wait complete. 2024-04-25T13:49:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 -- start start 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 Processor -- start 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef51319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ab8c0 con 0x564ef4d592f0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ab1f0 con 0x564ef4d614d0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a8430 con 0x564ef4d821b0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95b796c740 10 client.0 _ll_register_callbacks cb 0x7f95a4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59006/0 (socket says 172.21.15.94:59006) 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 -- 172.21.15.94:0/1876856853 learned_addr learned my addr 172.21.15.94:0/1876856853 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 -- 172.21.15.94:0/1876856853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 msgr2=0x564ef51319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef51319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 -- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 msgr2=0x564ef5130ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5130ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.296+0000 7f95aa59c640 1 -- 172.21.15.94:0/1876856853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef5131fa0 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95aa59c640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f959c0df340 tx=0x7f959c0a7260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc43958b811371b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c0a6ee0 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f959c0a3660 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95940054f0 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f959c0a7ad0 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 10 client.18180 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ef5133300 con 0x564ef4d592f0 2024-04-25T13:49:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940054f0 con 0x564ef4d592f0 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f959c003aa0 con 0x564ef4d592f0 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 10 client.18180 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 10 client.18180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 20 client.18180 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d161c0 0x564ef51a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 4 client.18180 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.297+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ef508fee0 con 0x564ef4d161c0 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.298+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d161c0 0x564ef51a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.298+0000 7f95a3fff640 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d161c0 0x564ef51a4fb0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c12433015e5ae2 server_cookie=27c4eec7451d869d in_seq=0 out_seq=0 2024-04-25T13:49:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.298+0000 7f9591ffb640 10 client.18180 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f9591ffb640 1 -- 172.21.15.94:0/1876856853 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ef508fee0 con 0x564ef4d161c0 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f9591ffb640 10 client.18180 handle_command_reply: tid=0 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 client.18180 shutdown 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 msgr2=0x564ef5130570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f959c0df340 tx=0x7f959c0a7260 comp rx=0 tx=0).stop 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 shutdown_connections 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564ef4d161c0 0x564ef51a4fb0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ef4d821b0 0x564ef51319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f9591ffb640 0 client.18180 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ef4d614d0 0x564ef5130ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 --2- 172.21.15.94:0/1876856853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ef4d592f0 0x564ef5130570 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 >> 172.21.15.94:0/1876856853 conn(0x564ef4d76300 msgr2=0x564ef511dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 shutdown_connections 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 1 -- 172.21.15.94:0/1876856853 wait complete. 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.464+0000 7f95b796c740 20 client.18180 trim_cache size 0 max 16384 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 msgr2=0x7f95980b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 0x7f95980b1ff0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x564ef5123b40 tx=0x7f95a4009360 comp rx=0 tx=0).stop 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 msgr2=0x7f95ac199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9f0 tx=0x7f959c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 shutdown_connections 2024-04-25T13:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95980afae0 0x7f95980b1ff0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95ac108970 0x7f95ac1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac103c70 0x7f95ac199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 --2- 172.21.15.94:0/4003424263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95ac102a30 0x7f95ac199110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 >> 172.21.15.94:0/4003424263 conn(0x7f95ac0fdc00 msgr2=0x7f95ac104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 shutdown_connections 2024-04-25T13:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:06.465+0000 7f95b0ab6640 1 -- 172.21.15.94:0/4003424263 wait complete. 2024-04-25T13:49:06.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 852787 12.87 MB/sec execute 229 sec latency 102.885 ms 2024-04-25T13:49:07.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 855750 12.89 MB/sec execute 230 sec latency 18.355 ms 2024-04-25T13:49:07.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:07 smithi137 ceph-mon[32568]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-25T13:49:07.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:07 smithi094 ceph-mon[29195]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-25T13:49:07.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:07 smithi138 ceph-mon[32977]: pgmap v567: 97 pgs: 97 active+clean; 44 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 262 op/s 2024-04-25T13:49:08.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 858094 12.87 MB/sec execute 231 sec latency 15.934 ms 2024-04-25T13:49:09.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 860181 12.87 MB/sec execute 232 sec latency 15.171 ms 2024-04-25T13:49:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:09 smithi137 ceph-mon[32568]: pgmap v568: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2024-04-25T13:49:09.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:09 smithi094 ceph-mon[29195]: pgmap v568: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2024-04-25T13:49:09.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:09 smithi138 ceph-mon[32977]: pgmap v568: 97 pgs: 97 active+clean; 48 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2024-04-25T13:49:10.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 863048 12.89 MB/sec execute 233 sec latency 50.119 ms 2024-04-25T13:49:11.802 INFO:tasks.workunit.client.0.smithi094.stdout: 1 865174 12.87 MB/sec execute 234 sec latency 43.356 ms 2024-04-25T13:49:11.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:11 smithi137 ceph-mon[32568]: pgmap v569: 97 pgs: 97 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-04-25T13:49:11.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:11 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:11.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:11 smithi094 ceph-mon[29195]: pgmap v569: 97 pgs: 97 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-04-25T13:49:11.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:11 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:11.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:11 smithi138 ceph-mon[32977]: pgmap v569: 97 pgs: 97 active+clean; 44 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-04-25T13:49:11.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:11 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:12.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 867166 12.87 MB/sec execute 235 sec latency 16.486 ms 2024-04-25T13:49:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:49:12.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:49:12.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:49:12.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:49:12.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:12 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:49:12.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:49:12.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:49:12.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:12 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:49:12.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:49:12.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:49:12.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:49:12.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:49:12.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:49:12.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:12 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:49:13.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 870146 12.89 MB/sec execute 236 sec latency 13.568 ms 2024-04-25T13:49:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:14 smithi137 ceph-mon[32568]: pgmap v570: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-04-25T13:49:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:14 smithi094 ceph-mon[29195]: pgmap v570: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-04-25T13:49:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:14 smithi138 ceph-mon[32977]: pgmap v570: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-04-25T13:49:14.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 872714 12.89 MB/sec execute 237 sec latency 17.780 ms 2024-04-25T13:49:15.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 874643 12.87 MB/sec execute 238 sec latency 12.383 ms 2024-04-25T13:49:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:16 smithi137 ceph-mon[32568]: pgmap v571: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 417 op/s 2024-04-25T13:49:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:16 smithi094 ceph-mon[29195]: pgmap v571: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 417 op/s 2024-04-25T13:49:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:16 smithi138 ceph-mon[32977]: pgmap v571: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 417 op/s 2024-04-25T13:49:16.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 877223 12.89 MB/sec execute 239 sec latency 62.751 ms 2024-04-25T13:49:17.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 879484 12.87 MB/sec execute 240 sec latency 38.493 ms 2024-04-25T13:49:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:18 smithi137 ceph-mon[32568]: pgmap v572: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-04-25T13:49:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:18 smithi094 ceph-mon[29195]: pgmap v572: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-04-25T13:49:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:18 smithi138 ceph-mon[32977]: pgmap v572: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-04-25T13:49:18.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 881465 12.87 MB/sec execute 241 sec latency 17.234 ms 2024-04-25T13:49:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 -- 172.21.15.94:0/2086230853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48104630 msgr2=0x7faa48106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2086230853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48104630 0x7faa48106a50 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7faa38009900 tx=0x7faa3802a410 comp rx=0 tx=0).stop 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 -- 172.21.15.94:0/2086230853 shutdown_connections 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2086230853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 0x7faa4810bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2086230853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48106f90 0x7faa481093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2086230853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48104630 0x7faa48106a50 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 -- 172.21.15.94:0/2086230853 >> 172.21.15.94:0/2086230853 conn(0x7faa480fdc30 msgr2=0x7faa481000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 -- 172.21.15.94:0/2086230853 shutdown_connections 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 -- 172.21.15.94:0/2086230853 wait complete. 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.565+0000 7faa4deaa640 1 Processor -- start 2024-04-25T13:49:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 -- start start 2024-04-25T13:49:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 0x7faa4819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 0x7faa481a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810f640 con 0x7faa48106f90 2024-04-25T13:49:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810f4c0 con 0x7faa48104630 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa4deaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810f7c0 con 0x7faa481098f0 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60716/0 (socket says 172.21.15.94:60716) 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 0x7faa4819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa46ffd640 1 -- 172.21.15.94:0/2787929373 learned_addr learned my addr 172.21.15.94:0/2787929373 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.566+0000 7faa47fff640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 0x7faa481a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 -- 172.21.15.94:0/2787929373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 msgr2=0x7faa481a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 0x7faa481a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 -- 172.21.15.94:0/2787929373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 msgr2=0x7faa4819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 0x7faa4819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 -- 172.21.15.94:0/2787929373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa481a1d50 con 0x7faa48106f90 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa477fe640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 0x7faa4819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa46ffd640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa44ff9640 1 -- 172.21.15.94:0/2787929373 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00cd80 con 0x7faa48106f90 2024-04-25T13:49:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa44ff9640 1 -- 172.21.15.94:0/2787929373 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c0108e0 con 0x7faa48106f90 2024-04-25T13:49:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa481a1fe0 con 0x7faa48106f90 2024-04-25T13:49:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa44ff9640 1 -- 172.21.15.94:0/2787929373 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c018980 con 0x7faa48106f90 2024-04-25T13:49:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.567+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a24d0 con 0x7faa48106f90 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa44ff9640 1 -- 172.21.15.94:0/2787929373 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7faa3c010430 con 0x7faa48106f90 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 Processor -- start 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 -- start start 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc8c2f0 0x55c4ad055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad05cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c4acfbb460 con 0x55c4accb51b0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c4acf75590 con 0x55c4acc944d0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa54e35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c4acfb7420 con 0x55c4acc8c2f0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa44ff9640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 0x7faa300b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60722/0 (socket says 172.21.15.94:60722) 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa47fff640 1 -- 172.21.15.94:0/1535002114 learned_addr learned my addr 172.21.15.94:0/1535002114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.570+0000 7faa46ffd640 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad05cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa477fe640 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc8c2f0 0x55c4ad055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 0x7faa300b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa44ff9640 1 -- 172.21.15.94:0/2787929373 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faa3c0df040 con 0x7faa48106f90 2024-04-25T13:49:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 -- 172.21.15.94:0/1535002114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc8c2f0 msgr2=0x55c4ad055ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc8c2f0 0x55c4ad055ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 -- 172.21.15.94:0/1535002114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 msgr2=0x55c4ad05cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad05cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 -- 172.21.15.94:0/1535002114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4acf820d0 con 0x55c4accb51b0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7faa40012d70 tx=0x7faa4000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886e6572edd583bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa47fff640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 0x7faa300b1ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ec20 tx=0x7faa40020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa2dffb640 1 -- 172.21.15.94:0/1535002114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40024070 con 0x55c4accb51b0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa2dffb640 1 -- 172.21.15.94:0/1535002114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400128c0 con 0x55c4accb51b0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.571+0000 7faa2dffb640 1 -- 172.21.15.94:0/1535002114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4001fe60 con 0x55c4accb51b0 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 -- 172.21.15.94:0/1535002114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 msgr2=0x55c4ad055970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7faa40012d70 tx=0x7faa4000e720 comp rx=0 tx=0).stop 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 -- 172.21.15.94:0/1535002114 shutdown_connections 2024-04-25T13:49:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad05cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc8c2f0 0x55c4ad055ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 --2- 172.21.15.94:0/1535002114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4accb51b0 0x55c4ad055970 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 -- 172.21.15.94:0/1535002114 >> 172.21.15.94:0/1535002114 conn(0x55c4acca9300 msgr2=0x55c4ad051300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 -- 172.21.15.94:0/1535002114 shutdown_connections 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.572+0000 7faa54e35740 1 -- 172.21.15.94:0/1535002114 wait complete. 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 -- start start 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 Processor -- start 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc8c2f0 0x55c4ad05f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c4ad05d7e0 con 0x55c4acc944d0 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c4acf42b60 con 0x55c4acc8c2f0 2024-04-25T13:49:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c4ad05d960 con 0x55c4accb51b0 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa54e35740 10 client.0 _ll_register_callbacks cb 0x55c4acf672b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60728/0 (socket says 172.21.15.94:60728) 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 -- 172.21.15.94:0/731331182 learned_addr learned my addr 172.21.15.94:0/731331182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa477fe640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc8c2f0 0x55c4ad05f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa47fff640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 -- 172.21.15.94:0/731331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 msgr2=0x55c4ad066ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad066ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 -- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc8c2f0 msgr2=0x55c4ad05f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc8c2f0 0x55c4ad05f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.573+0000 7faa46ffd640 1 -- 172.21.15.94:0/731331182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4acccd210 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa46ffd640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0a3500 tx=0x7faa3c0a3440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8f0df9ff94b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0aedb0 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faa3c0a66e0 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa400128c0 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa3c0e1320 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 10 client.18192 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4acd99620 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400128c0 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa3c0c8310 con 0x55c4acc944d0 2024-04-25T13:49:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 10 client.18192 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 20 client.18192 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc73d80 0x55c4ad06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 4 client.18192 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.574+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4acfc9070 con 0x55c4acc73d80 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.575+0000 7faa477fe640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc73d80 0x55c4ad06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.575+0000 7faa26ffd640 10 client.18192 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.575+0000 7faa477fe640 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc73d80 0x55c4ad06d2d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5976c15e2148a32f server_cookie=d5618bd918c5e317 in_seq=0 out_seq=0 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa26ffd640 1 -- 172.21.15.94:0/731331182 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4acfc9070 con 0x55c4acc73d80 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa26ffd640 10 client.18192 handle_command_reply: tid=0 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 client.18192 shutdown 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 msgr2=0x55c4ad05fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0a3500 tx=0x7faa3c0a3440 comp rx=0 tx=0).stop 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 shutdown_connections 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc73d80 0x55c4ad06d2d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad066ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc944d0 0x55c4ad05fb50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 --2- 172.21.15.94:0/731331182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc8c2f0 0x55c4ad05f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 >> 172.21.15.94:0/731331182 conn(0x55c4acca9300 msgr2=0x55c4ad054210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa26ffd640 0 client.18192 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.576+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 shutdown_connections 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.577+0000 7faa54e35740 1 -- 172.21.15.94:0/731331182 wait complete. 2024-04-25T13:49:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.577+0000 7faa54e35740 20 client.18192 trim_cache size 0 max 16384 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 Processor -- start 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 -- start start 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad0669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad0590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ab5a0 con 0x55c4acc8c2f0 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ad0e0 con 0x55c4acc944d0 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa54e35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa481abc70 con 0x55c4accb51b0 2024-04-25T13:49:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60742/0 (socket says 172.21.15.94:60742) 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 -- 172.21.15.94:0/1777923132 learned_addr learned my addr 172.21.15.94:0/1777923132 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa47fff640 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad0590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa46ffd640 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad0669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 -- 172.21.15.94:0/1777923132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 msgr2=0x55c4ad0590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad0590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 -- 172.21.15.94:0/1777923132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 msgr2=0x55c4ad0669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad0669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 -- 172.21.15.94:0/1777923132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ad056530 con 0x55c4acc8c2f0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.593+0000 7faa477fe640 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7faa38032fd0 tx=0x7faa38004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4d052515e9334b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.594+0000 7faa2d7fa640 1 -- 172.21.15.94:0/1777923132 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38035040 con 0x55c4acc8c2f0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.594+0000 7faa2d7fa640 1 -- 172.21.15.94:0/1777923132 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38031b70 con 0x55c4acc8c2f0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.594+0000 7faa2d7fa640 1 -- 172.21.15.94:0/1777923132 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38003b80 con 0x55c4acc8c2f0 2024-04-25T13:49:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- 172.21.15.94:0/1777923132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 msgr2=0x55c4ad058b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7faa38032fd0 tx=0x7faa38004140 comp rx=0 tx=0).stop 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- 172.21.15.94:0/1777923132 shutdown_connections 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4accb51b0 0x55c4ad0590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4acc944d0 0x55c4ad0669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 --2- 172.21.15.94:0/1777923132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad058b80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- 172.21.15.94:0/1777923132 >> 172.21.15.94:0/1777923132 conn(0x55c4acca9300 msgr2=0x55c4ad059af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- 172.21.15.94:0/1777923132 shutdown_connections 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- 172.21.15.94:0/1777923132 wait complete. 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 -- start start 2024-04-25T13:49:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.595+0000 7faa54e35740 1 Processor -- start 2024-04-25T13:49:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc944d0 0x55c4ad064050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4accb51b0 0x55c4ad064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa481aca10 con 0x55c4acc8c2f0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa481ac340 con 0x55c4accb51b0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a9580 con 0x55c4acc944d0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 10 client.0 _ll_register_callbacks cb 0x7faa3c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60756/0 (socket says 172.21.15.94:60756) 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 -- 172.21.15.94:0/3181823339 learned_addr learned my addr 172.21.15.94:0/3181823339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa47fff640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4accb51b0 0x55c4ad064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa46ffd640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc944d0 0x55c4ad064050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 -- 172.21.15.94:0/3181823339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc944d0 msgr2=0x55c4ad064050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc944d0 0x55c4ad064050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 -- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4accb51b0 msgr2=0x55c4ad064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4accb51b0 0x55c4ad064f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 -- 172.21.15.94:0/3181823339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ad065540 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa477fe640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa38005ab0 tx=0x7faa38031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f6e3f27c054e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38035070 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faa380316c0 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.596+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa38031b70 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa380318c0 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 10 client.18198 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4ad0667d0 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38031b70 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa380325e0 con 0x55c4acc8c2f0 2024-04-25T13:49:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 10 client.18198 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 20 client.18198 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc491c0 0x55c4ad0d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 4 client.18198 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4acfc2ee0 con 0x55c4acc491c0 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.597+0000 7faa46ffd640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc491c0 0x55c4ad0d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.598+0000 7faa26ffd640 10 client.18198 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.598+0000 7faa46ffd640 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc491c0 0x55c4ad0d79d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc04df93780a98c server_cookie=869611ac326e7550 in_seq=0 out_seq=0 2024-04-25T13:49:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.761+0000 7faa26ffd640 1 -- 172.21.15.94:0/3181823339 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4acfc2ee0 con 0x55c4acc491c0 2024-04-25T13:49:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.761+0000 7faa26ffd640 10 client.18198 handle_command_reply: tid=0 2024-04-25T13:49:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.761+0000 7faa54e35740 1 client.18198 shutdown 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.761+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 msgr2=0x55c4ad063b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.761+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa38005ab0 tx=0x7faa38031400 comp rx=0 tx=0).stop 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 shutdown_connections 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c4acc491c0 0x55c4ad0d79d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4accb51b0 0x55c4ad064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c4acc944d0 0x55c4ad064050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa26ffd640 0 client.18198 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 --2- 172.21.15.94:0/3181823339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c4acc8c2f0 0x55c4ad063b10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 >> 172.21.15.94:0/3181823339 conn(0x55c4acca9300 msgr2=0x55c4ad058b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 shutdown_connections 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 1 -- 172.21.15.94:0/3181823339 wait complete. 2024-04-25T13:49:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.762+0000 7faa54e35740 20 client.18198 trim_cache size 0 max 16384 2024-04-25T13:49:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:49:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:49:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:49:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 msgr2=0x7faa300b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 0x7faa300b1ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ec20 tx=0x7faa40020000 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 msgr2=0x7faa4819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00e9f0 tx=0x7faa3c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 shutdown_connections 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faa300afae0 0x7faa300b1ff0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa481098f0 0x7faa481a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48106f90 0x7faa4819a750 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 --2- 172.21.15.94:0/2787929373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48104630 0x7faa4819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 >> 172.21.15.94:0/2787929373 conn(0x7faa480fdc30 msgr2=0x7faa48105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 shutdown_connections 2024-04-25T13:49:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:19.763+0000 7faa4deaa640 1 -- 172.21.15.94:0/2787929373 wait complete. 2024-04-25T13:49:19.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 884245 12.89 MB/sec execute 242 sec latency 100.426 ms 2024-04-25T13:49:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:20 smithi137 ceph-mon[32568]: pgmap v573: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-04-25T13:49:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:20 smithi094 ceph-mon[29195]: pgmap v573: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-04-25T13:49:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:20 smithi138 ceph-mon[32977]: pgmap v573: 97 pgs: 97 active+clean; 45 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-04-25T13:49:20.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 886518 12.87 MB/sec execute 243 sec latency 14.903 ms 2024-04-25T13:49:21.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 888367 12.86 MB/sec execute 244 sec latency 60.502 ms 2024-04-25T13:49:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:22 smithi137 ceph-mon[32568]: pgmap v574: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-04-25T13:49:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:22 smithi094 ceph-mon[29195]: pgmap v574: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-04-25T13:49:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:22 smithi138 ceph-mon[32977]: pgmap v574: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-04-25T13:49:22.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 891312 12.89 MB/sec execute 245 sec latency 24.206 ms 2024-04-25T13:49:23.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 893600 12.87 MB/sec execute 246 sec latency 13.651 ms 2024-04-25T13:49:24.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:24 smithi138 ceph-mon[32977]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-04-25T13:49:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:24 smithi137 ceph-mon[32568]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-04-25T13:49:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:24 smithi094 ceph-mon[29195]: pgmap v575: 97 pgs: 97 active+clean; 49 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-04-25T13:49:24.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 895812 12.87 MB/sec execute 247 sec latency 13.354 ms 2024-04-25T13:49:25.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 898521 12.88 MB/sec execute 248 sec latency 50.484 ms 2024-04-25T13:49:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:26 smithi137 ceph-mon[32568]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-04-25T13:49:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:26 smithi094 ceph-mon[29195]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-04-25T13:49:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:26 smithi138 ceph-mon[32977]: pgmap v576: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-04-25T13:49:26.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 900801 12.87 MB/sec execute 249 sec latency 68.343 ms 2024-04-25T13:49:27.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:49:27.138 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 -- 172.21.15.94:0/3391671272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 msgr2=0x7f3dac104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 --2- 172.21.15.94:0/3391671272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac104030 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009900 tx=0x7f3d9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 -- 172.21.15.94:0/3391671272 shutdown_connections 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 --2- 172.21.15.94:0/3391671272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 0x7f3dac10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 --2- 172.21.15.94:0/3391671272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac104030 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 --2- 172.21.15.94:0/3391671272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 0x7f3dac1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 -- 172.21.15.94:0/3391671272 >> 172.21.15.94:0/3391671272 conn(0x7f3dac0fdbd0 msgr2=0x7f3dac100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 -- 172.21.15.94:0/3391671272 shutdown_connections 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 -- 172.21.15.94:0/3391671272 wait complete. 2024-04-25T13:49:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.252+0000 7f3db34af640 1 Processor -- start 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 -- start start 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 0x7f3dac06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 0x7f3dac1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10e500 con 0x7f3dac103c20 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10e380 con 0x7f3dac108920 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db34af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10e680 con 0x7f3dac1029e0 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38854/0 (socket says 172.21.15.94:38854) 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 -- 172.21.15.94:0/2082975229 learned_addr learned my addr 172.21.15.94:0/2082975229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db124b640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 0x7f3dac06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 0x7f3dac1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 -- 172.21.15.94:0/2082975229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 msgr2=0x7f3dac06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 0x7f3dac06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 -- 172.21.15.94:0/2082975229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 msgr2=0x7f3dac1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 0x7f3dac1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 -- 172.21.15.94:0/2082975229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac1a2080 con 0x7f3dac103c20 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006010 tx=0x7f3d9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3da27fc640 1 -- 172.21.15.94:0/2082975229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c03f030 con 0x7f3dac103c20 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.253+0000 7f3da27fc640 1 -- 172.21.15.94:0/2082975229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d9c031910 con 0x7f3dac103c20 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.254+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac1a2310 con 0x7f3dac103c20 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.254+0000 7f3da27fc640 1 -- 172.21.15.94:0/2082975229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c035070 con 0x7f3dac103c20 2024-04-25T13:49:27.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.254+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac1a2800 con 0x7f3dac103c20 2024-04-25T13:49:27.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 Processor -- start 2024-04-25T13:49:27.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 -- start start 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3da27fc640 1 -- 172.21.15.94:0/2082975229 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3d9c042030 con 0x7f3dac103c20 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee95615fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95616c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ee9557ad80 con 0x55ee95274390 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee955348b0 con 0x55ee952536b0 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3dba367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee95576d40 con 0x55ee9524b4d0 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db1a4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee95615fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db1a4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38864/0 (socket says 172.21.15.94:38864) 2024-04-25T13:49:27.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db0a4a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95616c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db1a4c640 1 -- 172.21.15.94:0/1155006837 learned_addr learned my addr 172.21.15.94:0/1155006837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 -- 172.21.15.94:0/1155006837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 msgr2=0x55ee95616c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95616c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 -- 172.21.15.94:0/1155006837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 msgr2=0x55ee95615a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db124b640 1 -- 172.21.15.94:0/1155006837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee95505a40 con 0x55ee9524b4d0 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95616c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.257+0000 7f3da27fc640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 0x7f3d980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 0x7f3d980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3da27fc640 1 -- 172.21.15.94:0/2082975229 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3d9c0ff180 con 0x7f3dac103c20 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3db124b640 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee95615fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3dac1022b0 tx=0x7f3da8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de8d364e97a7c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3d977fe640 1 -- 172.21.15.94:0/1155006837 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8004230 con 0x55ee9524b4d0 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3d977fe640 1 -- 172.21.15.94:0/1155006837 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da8009e10 con 0x55ee9524b4d0 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.258+0000 7f3d977fe640 1 -- 172.21.15.94:0/1155006837 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80196f0 con 0x55ee9524b4d0 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- 172.21.15.94:0/1155006837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 msgr2=0x55ee95615fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 0x7f3d980b2000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55ee95616ab0 tx=0x7f3da4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee95615fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3dac1022b0 tx=0x7f3da8011810 comp rx=0 tx=0).stop 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- 172.21.15.94:0/1155006837 shutdown_connections 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95616c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee95615fe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 --2- 172.21.15.94:0/1155006837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee95615a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- 172.21.15.94:0/1155006837 >> 172.21.15.94:0/1155006837 conn(0x55ee952684e0 msgr2=0x55ee95611270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- 172.21.15.94:0/1155006837 shutdown_connections 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- 172.21.15.94:0/1155006837 wait complete. 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 -- start start 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.259+0000 7f3dba367740 1 Processor -- start 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee952536b0 0x55ee9561fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95626cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ee95617780 con 0x55ee9524b4d0 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee95501d10 con 0x55ee95274390 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee95617900 con 0x55ee952536b0 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3dba367740 10 client.0 _ll_register_callbacks cb 0x55ee9552e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db124b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db124b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38872/0 (socket says 172.21.15.94:38872) 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db124b640 1 -- 172.21.15.94:0/3072672462 learned_addr learned my addr 172.21.15.94:0/3072672462 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee952536b0 0x55ee9561fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95626cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 -- 172.21.15.94:0/3072672462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee952536b0 msgr2=0x55ee9561fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee952536b0 0x55ee9561fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 -- 172.21.15.94:0/3072672462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 msgr2=0x55ee9561f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 -- 172.21.15.94:0/3072672462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee953567d0 con 0x55ee95274390 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db124b640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95626cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009fd0 tx=0x7f3da4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588bce51a2fefb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.260+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da40043d0 con 0x55ee95274390 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da8009e10 con 0x55ee95274390 2024-04-25T13:49:27.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3da4004570 con 0x55ee95274390 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3da4004770 con 0x55ee95274390 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 10 client.27680 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee9528c3f0 con 0x55ee95274390 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da8009e10 con 0x55ee95274390 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3da4026070 con 0x55ee95274390 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 10 client.27680 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 20 client.27680 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee95232f60 0x55ee9562d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 4 client.27680 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee95588990 con 0x55ee95232f60 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.261+0000 7f3db124b640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee95232f60 0x55ee9562d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.262+0000 7f3d94ff9640 10 client.27680 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:27.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.262+0000 7f3db124b640 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee95232f60 0x55ee9562d290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885b674c304a1761 server_cookie=33d1552b191a652 in_seq=0 out_seq=0 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3072672462 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee95588990 con 0x55ee95232f60 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3d94ff9640 10 client.27680 handle_command_reply: tid=0 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 client.27680 shutdown 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 msgr2=0x55ee95626cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95626cf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009fd0 tx=0x7f3da4004ab0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 shutdown_connections 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee95232f60 0x55ee9562d290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95626cf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee952536b0 0x55ee9561fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3d94ff9640 0 client.27680 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 --2- 172.21.15.94:0/3072672462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee9524b4d0 0x55ee9561f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.264+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 >> 172.21.15.94:0/3072672462 conn(0x55ee952684e0 msgr2=0x55ee95614180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.265+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 shutdown_connections 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.265+0000 7f3dba367740 1 -- 172.21.15.94:0/3072672462 wait complete. 2024-04-25T13:49:27.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.265+0000 7f3dba367740 20 client.27680 trim_cache size 0 max 16384 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 Processor -- start 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 -- start start 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95610ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ab520 con 0x55ee95274390 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ad060 con 0x55ee952536b0 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.283+0000 7f3dba367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1abbf0 con 0x55ee9524b4d0 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38874/0 (socket says 172.21.15.94:38874) 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 -- 172.21.15.94:0/2444062629 learned_addr learned my addr 172.21.15.94:0/2444062629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95610ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db124b640 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 -- 172.21.15.94:0/2444062629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 msgr2=0x55ee956159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 -- 172.21.15.94:0/2444062629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 msgr2=0x55ee95610ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95610ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 -- 172.21.15.94:0/2444062629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee9561e650 con 0x55ee95274390 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3da401e4d0 tx=0x7f3da4005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94480ca3bb4df70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3d96ffd640 1 -- 172.21.15.94:0/2444062629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4035030 con 0x55ee95274390 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3d96ffd640 1 -- 172.21.15.94:0/2444062629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da400fdd0 con 0x55ee95274390 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.284+0000 7f3d96ffd640 1 -- 172.21.15.94:0/2444062629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da40286a0 con 0x55ee95274390 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- 172.21.15.94:0/2444062629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 msgr2=0x55ee956111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3da401e4d0 tx=0x7f3da4005f30 comp rx=0 tx=0).stop 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- 172.21.15.94:0/2444062629 shutdown_connections 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee95274390 0x55ee956111e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee952536b0 0x55ee95610ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 --2- 172.21.15.94:0/2444062629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- 172.21.15.94:0/2444062629 >> 172.21.15.94:0/2444062629 conn(0x55ee952684e0 msgr2=0x55ee9561a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- 172.21.15.94:0/2444062629 shutdown_connections 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- 172.21.15.94:0/2444062629 wait complete. 2024-04-25T13:49:27.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 -- start start 2024-04-25T13:49:27.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.285+0000 7f3dba367740 1 Processor -- start 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95697df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ac990 con 0x55ee952536b0 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ac2c0 con 0x55ee95274390 2024-04-25T13:49:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a9500 con 0x55ee9524b4d0 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3dba367740 10 client.0 _ll_register_callbacks cb 0x7f3da4011e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38878/0 (socket says 172.21.15.94:38878) 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db124b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 -- 172.21.15.94:0/3638549715 learned_addr learned my addr 172.21.15.94:0/3638549715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95697df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 -- 172.21.15.94:0/3638549715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 msgr2=0x55ee956909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 -- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 msgr2=0x55ee95697df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95697df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 -- 172.21.15.94:0/3638549715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9524c0f0 con 0x55ee952536b0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db1a4c640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95697df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db124b640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3db0a4a640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0c7370 tx=0x7f3d9c102be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e723bae0191143e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.286+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c049040 con 0x55ee952536b0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3d9c0c3c60 con 0x55ee952536b0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da400fdd0 con 0x55ee952536b0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d9c0c6fc0 con 0x55ee952536b0 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 10 client.18216 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee9553b4b0 con 0x55ee952536b0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da400fdd0 con 0x55ee952536b0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d9c0c3900 con 0x55ee952536b0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 10 client.18216 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 20 client.18216 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee952083a0 0x55ee9569bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 4 client.18216 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee95582800 con 0x55ee952083a0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.287+0000 7f3db124b640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee952083a0 0x55ee9569bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3d94ff9640 10 client.18216 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3db124b640 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee952083a0 0x55ee9569bd90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23706e32c6e459bf server_cookie=e9e24d05e8d817c7 in_seq=0 out_seq=0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3d94ff9640 1 -- 172.21.15.94:0/3638549715 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee95582800 con 0x55ee952083a0 2024-04-25T13:49:27.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3d94ff9640 10 client.18216 handle_command_reply: tid=0 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 client.18216 shutdown 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 msgr2=0x55ee95690f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0c7370 tx=0x7f3d9c102be0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 shutdown_connections 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ee952083a0 0x55ee9569bd90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee95274390 0x55ee95697df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee952536b0 0x55ee95690f00 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 --2- 172.21.15.94:0/3638549715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee9524b4d0 0x55ee956909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3d94ff9640 0 client.18216 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.288+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 >> 172.21.15.94:0/3638549715 conn(0x55ee952684e0 msgr2=0x55ee95610c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 shutdown_connections 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3dba367740 1 -- 172.21.15.94:0/3638549715 wait complete. 2024-04-25T13:49:27.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3dba367740 20 client.18216 trim_cache size 0 max 16384 2024-04-25T13:49:27.297 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:49:27.297 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:49:27.297 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:49:27.297 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 msgr2=0x7f3d980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 0x7f3d980b2000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55ee95616ab0 tx=0x7f3da4009500 comp rx=0 tx=0).stop 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 msgr2=0x7f3dac19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006010 tx=0x7f3d9c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 shutdown_connections 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d980afaf0 0x7f3d980b2000 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3dac108920 0x7f3dac1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dac103c20 0x7f3dac19f310 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 --2- 172.21.15.94:0/2082975229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dac1029e0 0x7f3dac06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 >> 172.21.15.94:0/2082975229 conn(0x7f3dac0fdbd0 msgr2=0x7f3dac104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 shutdown_connections 2024-04-25T13:49:27.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.289+0000 7f3db34af640 1 -- 172.21.15.94:0/2082975229 wait complete. 2024-04-25T13:49:27.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:49:27.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:49:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 -- 172.21.15.94:0/1798798312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8118106f10 msgr2=0x7f8118109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 --2- 172.21.15.94:0/1798798312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8118106f10 0x7f8118109330 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f810c009900 tx=0x7f810c02a410 comp rx=0 tx=0).stop 2024-04-25T13:49:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 -- 172.21.15.94:0/1798798312 shutdown_connections 2024-04-25T13:49:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 --2- 172.21.15.94:0/1798798312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118109870 0x7f811810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 --2- 172.21.15.94:0/1798798312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8118106f10 0x7f8118109330 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 --2- 172.21.15.94:0/1798798312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81181045b0 0x7f81181069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 -- 172.21.15.94:0/1798798312 >> 172.21.15.94:0/1798798312 conn(0x7f81180fdbd0 msgr2=0x7f8118100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 -- 172.21.15.94:0/1798798312 shutdown_connections 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 -- 172.21.15.94:0/1798798312 wait complete. 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.412+0000 7f811eb84640 1 Processor -- start 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 -- start start 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118106f10 0x7f811819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8118109870 0x7f81181a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f811810f5f0 con 0x7f81181045b0 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f811810f470 con 0x7f8118106f10 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f811810f770 con 0x7f8118109870 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38902/0 (socket says 172.21.15.94:38902) 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 -- 172.21.15.94:0/1650474494 learned_addr learned my addr 172.21.15.94:0/1650474494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f8117fff640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118106f10 0x7f811819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811d121640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8118109870 0x7f81181a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 -- 172.21.15.94:0/1650474494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8118109870 msgr2=0x7f81181a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8118109870 0x7f81181a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 -- 172.21.15.94:0/1650474494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118106f10 msgr2=0x7f811819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118106f10 0x7f811819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 -- 172.21.15.94:0/1650474494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81181a1d70 con 0x7f81181045b0 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811c920640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9f0 tx=0x7f810800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810800ccc0 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81080054d0 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.413+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181a2060 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.414+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108010430 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.414+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181a2560 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.415+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81180ff6f0 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.416+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f81080046c0 con 0x7f81181045b0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.416+0000 7f8115ffb640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 0x7f81000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.417+0000 7f8117fff640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 0x7f81000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.417+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f81080df020 con 0x7f81181045b0 2024-04-25T13:49:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.417+0000 7f8117fff640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 0x7f81000b2000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f810c009230 tx=0x7f810c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.422+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81080a7b60 con 0x7f81181045b0 2024-04-25T13:49:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.655+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f811802db50 con 0x7f81181045b0 2024-04-25T13:49:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.655+0000 7f8115ffb640 1 -- 172.21.15.94:0/1650474494 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f81080a7460 con 0x7f81181045b0 2024-04-25T13:49:27.656 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:49:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:49:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 msgr2=0x7f81000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 0x7f81000b2000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f810c009230 tx=0x7f810c031400 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 msgr2=0x7f811819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9f0 tx=0x7f810800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 shutdown_connections 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81000afaf0 0x7f81000b2000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8118109870 0x7f81181a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8118106f10 0x7f811819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 --2- 172.21.15.94:0/1650474494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81181045b0 0x7f811819a230 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 >> 172.21.15.94:0/1650474494 conn(0x7f81180fdbd0 msgr2=0x7f81181051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 shutdown_connections 2024-04-25T13:49:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.658+0000 7f811eb84640 1 -- 172.21.15.94:0/1650474494 wait complete. 2024-04-25T13:49:27.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:49:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 -- 172.21.15.94:0/188260969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0103c20 msgr2=0x7fc6f0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/188260969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0103c20 0x7fc6f0104030 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009900 tx=0x7fc6e402a410 comp rx=0 tx=0).stop 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 -- 172.21.15.94:0/188260969 shutdown_connections 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/188260969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f0108920 0x7fc6f010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/188260969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0103c20 0x7fc6f0104030 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/188260969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f01029e0 0x7fc6f01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 -- 172.21.15.94:0/188260969 >> 172.21.15.94:0/188260969 conn(0x7fc6f00fdbd0 msgr2=0x7fc6f0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 -- 172.21.15.94:0/188260969 shutdown_connections 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.783+0000 7fc6f4f72640 1 -- 172.21.15.94:0/188260969 wait complete. 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 Processor -- start 2024-04-25T13:49:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 -- start start 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f01029e0 0x7fc6f0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f0103c20 0x7fc6f0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e510 con 0x7fc6f0108920 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e390 con 0x7fc6f0103c20 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6f4f72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e690 con 0x7fc6f01029e0 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38910/0 (socket says 172.21.15.94:38910) 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6eed9d640 1 -- 172.21.15.94:0/2920508084 learned_addr learned my addr 172.21.15.94:0/2920508084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f0103c20 0x7fc6f0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.784+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f01029e0 0x7fc6f0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 -- 172.21.15.94:0/2920508084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f01029e0 msgr2=0x7fc6f0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f01029e0 0x7fc6f0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 -- 172.21.15.94:0/2920508084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f0103c20 msgr2=0x7fc6f0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f0103c20 0x7fc6f0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 -- 172.21.15.94:0/2920508084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f01a0f80 con 0x7fc6f0108920 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6eed9d640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800eb10 tx=0x7fc6e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6df7fe640 1 -- 172.21.15.94:0/2920508084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004b20 con 0x7fc6f0108920 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6df7fe640 1 -- 172.21.15.94:0/2920508084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e8004cc0 con 0x7fc6f0108920 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f01a1270 con 0x7fc6f0108920 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6df7fe640 1 -- 172.21.15.94:0/2920508084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8010670 con 0x7fc6f0108920 2024-04-25T13:49:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.785+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f01a1810 con 0x7fc6f0108920 2024-04-25T13:49:27.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 Processor -- start 2024-04-25T13:49:27.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6df7fe640 1 -- 172.21.15.94:0/2920508084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc6e8005020 con 0x7fc6f0108920 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 -- start start 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccaf8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccaf9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dbcca5cd80 con 0x55dbcc756390 2024-04-25T13:49:27.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dbcca16a20 con 0x55dbcc7356b0 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dbcca58d40 con 0x55dbcc72d4d0 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38920/0 (socket says 172.21.15.94:38920) 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6edd9b640 1 -- 172.21.15.94:0/3637397740 learned_addr learned my addr 172.21.15.94:0/3637397740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccaf8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.788+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccaf9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6df7fe640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 0x7fc6d80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 -- 172.21.15.94:0/3637397740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 msgr2=0x55dbccaf9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccaf9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 -- 172.21.15.94:0/3637397740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 msgr2=0x55dbccaf8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 -- 172.21.15.94:0/3637397740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbcca21c30 con 0x55dbcc72d4d0 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6eed9d640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 0x7fc6d80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccaf8d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6f01022b0 tx=0x7fc6e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faefeb484ec3effe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6df7fe640 1 -- 172.21.15.94:0/2920508084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fc6e80df1b0 con 0x7fc6f0108920 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6dcff9640 1 -- 172.21.15.94:0/3637397740 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0009d70 con 0x55dbcc72d4d0 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6dcff9640 1 -- 172.21.15.94:0/3637397740 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e0005020 con 0x55dbcc72d4d0 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.789+0000 7fc6dcff9640 1 -- 172.21.15.94:0/3637397740 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00049a0 con 0x55dbcc72d4d0 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/3637397740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 msgr2=0x55dbccaf8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccaf8d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6f01022b0 tx=0x7fc6e0011810 comp rx=0 tx=0).stop 2024-04-25T13:49:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6eed9d640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 0x7fc6d80b2080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6e801c000 tx=0x7fc6e8071e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/3637397740 shutdown_connections 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccaf9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccaf8d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/3637397740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc756390 0x55dbccaf8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/3637397740 >> 172.21.15.94:0/3637397740 conn(0x55dbcc74a4e0 msgr2=0x55dbccaf4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/3637397740 shutdown_connections 2024-04-25T13:49:27.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.790+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/3637397740 wait complete. 2024-04-25T13:49:27.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 -- start start 2024-04-25T13:49:27.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 Processor -- start 2024-04-25T13:49:27.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb02520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 0x55dbccb099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dbccafa530 con 0x55dbcc7356b0 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc9e3dc0 con 0x55dbcc756390 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dbccafa6b0 con 0x55dbcc72d4d0 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6fbe1e740 10 client.0 _ll_register_callbacks cb 0x55dbcc9ff600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38930/0 (socket says 172.21.15.94:38930) 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 -- 172.21.15.94:0/4235969817 learned_addr learned my addr 172.21.15.94:0/4235969817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb02520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 0x55dbccb099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 -- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 msgr2=0x55dbccb02520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb02520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 -- 172.21.15.94:0/4235969817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 msgr2=0x55dbccb099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 0x55dbccb099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 -- 172.21.15.94:0/4235969817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbcc76e3f0 con 0x55dbcc7356b0 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 0x55dbccb099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000a010 tx=0x7fc6e0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4716c4f5336020b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.791+0000 7fc6d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0029020 con 0x55dbcc7356b0 2024-04-25T13:49:27.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fc6e0004d70 con 0x55dbcc7356b0 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e0005020 con 0x55dbcc7356b0 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc6e0034050 con 0x55dbcc7356b0 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 10 client.18234 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbcc8387d0 con 0x55dbcc7356b0 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e0005020 con 0x55dbcc7356b0 2024-04-25T13:49:27.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc6e0019450 con 0x55dbcc7356b0 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 10 client.18234 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 10 client.18234 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 20 client.18234 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc714f60 0x55dbccb100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 4 client.18234 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbcca6a990 con 0x55dbcc714f60 2024-04-25T13:49:27.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.792+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc714f60 0x55dbccb100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.793+0000 7fc6d5ffb640 10 client.18234 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.793+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc714f60 0x55dbccb100d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=305129267f854d7d server_cookie=9f56d6ef7f1baf4 in_seq=0 out_seq=0 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/4235969817 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbcca6a990 con 0x55dbcc714f60 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6d5ffb640 10 client.18234 handle_command_reply: tid=0 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 client.18234 shutdown 2024-04-25T13:49:27.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 msgr2=0x55dbccb02a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000a010 tx=0x7fc6e0004970 comp rx=0 tx=0).stop 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 shutdown_connections 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc714f60 0x55dbccb100d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc756390 0x55dbccb099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6d5ffb640 0 client.18234 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccb02a60 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/4235969817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb02520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 >> 172.21.15.94:0/4235969817 conn(0x55dbcc74a4e0 msgr2=0x55dbccaf6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 shutdown_connections 2024-04-25T13:49:27.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/4235969817 wait complete. 2024-04-25T13:49:27.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.794+0000 7fc6fbe1e740 20 client.18234 trim_cache size 0 max 16384 2024-04-25T13:49:27.815 INFO:tasks.workunit.client.0.smithi094.stdout: 1 902741 12.86 MB/sec execute 250 sec latency 24.669 ms 2024-04-25T13:49:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 Processor -- start 2024-04-25T13:49:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 -- start start 2024-04-25T13:49:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc714f60 0x55dbccafd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccafd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ab550 con 0x55dbcc7356b0 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ad090 con 0x55dbcc714f60 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01abc20 con 0x55dbcc72d4d0 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38932/0 (socket says 172.21.15.94:38932) 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6edd9b640 1 -- 172.21.15.94:0/2348238329 learned_addr learned my addr 172.21.15.94:0/2348238329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccafd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.808+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc714f60 0x55dbccafd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 -- 172.21.15.94:0/2348238329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 msgr2=0x55dbccafd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccafd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 -- 172.21.15.94:0/2348238329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc714f60 msgr2=0x55dbccafd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc714f60 0x55dbccafd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 -- 172.21.15.94:0/2348238329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbccafa410 con 0x55dbcc7356b0 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005ab0 tx=0x7fc6e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cd9685f9753e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6d7fff640 1 -- 172.21.15.94:0/2348238329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4053070 con 0x55dbcc7356b0 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6d7fff640 1 -- 172.21.15.94:0/2348238329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4004240 con 0x55dbcc7356b0 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.809+0000 7fc6d7fff640 1 -- 172.21.15.94:0/2348238329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4035040 con 0x55dbcc7356b0 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/2348238329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 msgr2=0x55dbccaf3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005ab0 tx=0x7fc6e402a410 comp rx=0 tx=0).stop 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/2348238329 shutdown_connections 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc7356b0 0x55dbccaf3a50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccafd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/2348238329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc714f60 0x55dbccafd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/2348238329 >> 172.21.15.94:0/2348238329 conn(0x55dbcc74a4e0 msgr2=0x55dbccb02520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/2348238329 shutdown_connections 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/2348238329 wait complete. 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 -- start start 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.810+0000 7fc6fbe1e740 1 Processor -- start 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb05900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccb05e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ac9c0 con 0x55dbcc714f60 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ac2f0 con 0x55dbcc7356b0 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a9530 con 0x55dbcc72d4d0 2024-04-25T13:49:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 10 client.0 _ll_register_callbacks cb 0x7fc6e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38942/0 (socket says 172.21.15.94:38942) 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 -- 172.21.15.94:0/1949819536 learned_addr learned my addr 172.21.15.94:0/1949819536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccb05e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb05900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 -- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 msgr2=0x55dbccb05900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb05900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 -- 172.21.15.94:0/1949819536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 msgr2=0x55dbccb05e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccb05e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 -- 172.21.15.94:0/1949819536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbccb06480 con 0x55dbcc714f60 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6edd9b640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4051040 tx=0x7fc6e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e964325fbcda513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4053040 con 0x55dbcc714f60 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fc6e4003a00 con 0x55dbcc714f60 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.811+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e4004240 con 0x55dbcc714f60 2024-04-25T13:49:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4035040 con 0x55dbcc714f60 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 10 client.18240 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dbccb774b0 con 0x55dbcc714f60 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4004240 con 0x55dbcc714f60 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc6e405d4e0 con 0x55dbcc714f60 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 10 client.18240 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 20 client.18240 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc756390 0x55dbccb79e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 4 client.18240 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbcca64800 con 0x55dbcc756390 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.812+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc756390 0x55dbccb79e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6ee59c640 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc756390 0x55dbccb79e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbae5db01a5b864a server_cookie=e8cb89e4f269cc1f in_seq=0 out_seq=0 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6d5ffb640 10 client.18240 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:49:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6d5ffb640 1 -- 172.21.15.94:0/1949819536 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbcca64800 con 0x55dbcc756390 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6d5ffb640 10 client.18240 handle_command_reply: tid=0 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 client.18240 shutdown 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 msgr2=0x55dbccb08d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4051040 tx=0x7fc6e4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 shutdown_connections 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55dbcc756390 0x55dbccb79e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dbcc7356b0 0x55dbccb05e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbcc72d4d0 0x55dbccb05900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6d5ffb640 0 client.18240 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 --2- 172.21.15.94:0/1949819536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dbcc714f60 0x55dbccb08d80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 >> 172.21.15.94:0/1949819536 conn(0x55dbcc74a4e0 msgr2=0x55dbccb02420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 shutdown_connections 2024-04-25T13:49:27.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 1 -- 172.21.15.94:0/1949819536 wait complete. 2024-04-25T13:49:27.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.813+0000 7fc6fbe1e740 20 client.18240 trim_cache size 0 max 16384 2024-04-25T13:49:27.829 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:49:27.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 msgr2=0x7fc6d80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 0x7fc6d80b2080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6e801c000 tx=0x7fc6e8071e10 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 msgr2=0x7fc6f01a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800eb10 tx=0x7fc6e800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 shutdown_connections 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6d80afb70 0x7fc6d80b2080 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0108920 0x7fc6f01a0910 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f0103c20 0x7fc6f0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 --2- 172.21.15.94:0/2920508084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f01029e0 0x7fc6f0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 >> 172.21.15.94:0/2920508084 conn(0x7fc6f00fdbd0 msgr2=0x7fc6f0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 shutdown_connections 2024-04-25T13:49:27.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.814+0000 7fc6f4f72640 1 -- 172.21.15.94:0/2920508084 wait complete. 2024-04-25T13:49:27.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 -- 172.21.15.94:0/1428674987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 msgr2=0x7f231010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 --2- 172.21.15.94:0/1428674987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231010bb10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f230800b0a0 tx=0x7f230802a710 comp rx=0 tx=0).stop 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 -- 172.21.15.94:0/1428674987 shutdown_connections 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 --2- 172.21.15.94:0/1428674987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231010bb10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 --2- 172.21.15.94:0/1428674987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 0x7f2310101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 --2- 172.21.15.94:0/1428674987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 0x7f2310105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 -- 172.21.15.94:0/1428674987 >> 172.21.15.94:0/1428674987 conn(0x7f23100fd1f0 msgr2=0x7f23100ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 -- 172.21.15.94:0/1428674987 shutdown_connections 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.944+0000 7f23155ff640 1 -- 172.21.15.94:0/1428674987 wait complete. 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 Processor -- start 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 -- start start 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 0x7f231019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 0x7f23101a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f231010f470 con 0x7f23101020b0 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f231010f2f0 con 0x7f23101015b0 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f23155ff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f231010f5f0 con 0x7f2310104f00 2024-04-25T13:49:27.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38954/0 (socket says 172.21.15.94:38954) 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230e7fc640 1 -- 172.21.15.94:0/2857201169 learned_addr learned my addr 172.21.15.94:0/2857201169 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230effd640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 0x7f231019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230f7fe640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 0x7f23101a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.945+0000 7f230e7fc640 1 -- 172.21.15.94:0/2857201169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 msgr2=0x7f23101a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f230e7fc640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 0x7f23101a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f230e7fc640 1 -- 172.21.15.94:0/2857201169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 msgr2=0x7f231019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f230e7fc640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 0x7f231019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f230e7fc640 1 -- 172.21.15.94:0/2857201169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101a1b40 con 0x7f23101020b0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f230e7fc640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f22f7fff640 1 -- 172.21.15.94:0/2857201169 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230400cd80 con 0x7f23101020b0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f22f7fff640 1 -- 172.21.15.94:0/2857201169 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23040108e0 con 0x7f23101020b0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101a1e30 con 0x7f23101020b0 2024-04-25T13:49:27.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f22f7fff640 1 -- 172.21.15.94:0/2857201169 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304018980 con 0x7f23101020b0 2024-04-25T13:49:27.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.946+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a22a0 con 0x7f23101020b0 2024-04-25T13:49:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 Processor -- start 2024-04-25T13:49:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f22f7fff640 1 -- 172.21.15.94:0/2857201169 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2304010430 con 0x7f23101020b0 2024-04-25T13:49:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 -- start start 2024-04-25T13:49:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565283df0d80 con 0x565283aea390 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565283daaa20 con 0x565283ac14d0 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f231c584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565283decd40 con 0x565283ac96b0 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38968/0 (socket says 172.21.15.94:38968) 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 -- 172.21.15.94:0/1891311792 learned_addr learned my addr 172.21.15.94:0/1891311792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230effd640 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230e7fc640 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 -- 172.21.15.94:0/1891311792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 msgr2=0x565283e8dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e8dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 -- 172.21.15.94:0/1891311792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 msgr2=0x565283e8cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e8cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f230f7fe640 1 -- 172.21.15.94:0/1891311792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565283db5c30 con 0x565283aea390 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.949+0000 7f22f7fff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 0x7f22f80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f230effd640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 0x7f22f80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f230802ac30 tx=0x7f2308032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee9a72fc032cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f22f7fff640 1 -- 172.21.15.94:0/2857201169 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f23040df040 con 0x7f23101020b0 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f22f57fa640 1 -- 172.21.15.94:0/1891311792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308041030 con 0x565283aea390 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f22f57fa640 1 -- 172.21.15.94:0/1891311792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23080096e0 con 0x565283aea390 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.950+0000 7f22f57fa640 1 -- 172.21.15.94:0/1891311792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308036040 con 0x565283aea390 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- 172.21.15.94:0/1891311792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 msgr2=0x565283e8c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f230effd640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 0x7f22f80b1ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x565283e8c700 tx=0x7f2300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f230802ac30 tx=0x7f2308032f60 comp rx=0 tx=0).stop 2024-04-25T13:49:27.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- 172.21.15.94:0/1891311792 shutdown_connections 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- 172.21.15.94:0/1891311792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e8c9e0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- 172.21.15.94:0/1891311792 >> 172.21.15.94:0/1891311792 conn(0x565283ade4e0 msgr2=0x565283e881f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- 172.21.15.94:0/1891311792 shutdown_connections 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- 172.21.15.94:0/1891311792 wait complete. 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- start start 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 Processor -- start 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565283e8e6a0 con 0x565283aea390 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565283d77dc0 con 0x565283ac14d0 2024-04-25T13:49:27.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.951+0000 7f231c584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565283e8e820 con 0x565283ac96b0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f231c584740 10 client.0 _ll_register_callbacks cb 0x565283d93600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38984/0 (socket says 172.21.15.94:38984) 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 -- 172.21.15.94:0/1403954214 learned_addr learned my addr 172.21.15.94:0/1403954214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230e7fc640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 -- 172.21.15.94:0/1403954214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 msgr2=0x565283e96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 -- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 msgr2=0x565283e96830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e96830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 -- 172.21.15.94:0/1403954214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565283b023f0 con 0x565283aea390 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2308006130 tx=0x7f2308031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1093469327aff225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f22f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:27.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230804a030 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f23080092e0 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23080096e0 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2308041070 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f231c584740 10 client.18252 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.952+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565283bcc7d0 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23080096e0 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2308046840 con 0x565283aea390 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 10 client.18252 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 10 client.18252 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 20 client.18252 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aa8f60 0x565283ea4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 4 client.18252 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565283dfe990 con 0x565283aa8f60 2024-04-25T13:49:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f230effd640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aa8f60 0x565283ea4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f22f2ff5640 10 client.18252 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.953+0000 7f230effd640 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aa8f60 0x565283ea4120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a8232fda9bccf65 server_cookie=4d7d48d1827b9724 in_seq=0 out_seq=0 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.954+0000 7f22f2ff5640 1 -- 172.21.15.94:0/1403954214 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565283dfe990 con 0x565283aa8f60 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.954+0000 7f22f2ff5640 10 client.18252 handle_command_reply: tid=0 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.954+0000 7f231c584740 1 client.18252 shutdown 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 msgr2=0x565283e9dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2308006130 tx=0x7f2308031ca0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 shutdown_connections 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aa8f60 0x565283ea4120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283aea390 0x565283e9dcd0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 --2- 172.21.15.94:0/1403954214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac14d0 0x565283e96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f22f2ff5640 0 client.18252 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 >> 172.21.15.94:0/1403954214 conn(0x565283ade4e0 msgr2=0x565283e873f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 shutdown_connections 2024-04-25T13:49:27.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 1 -- 172.21.15.94:0/1403954214 wait complete. 2024-04-25T13:49:27.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.955+0000 7f231c584740 20 client.18252 trim_cache size 0 max 16384 2024-04-25T13:49:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.969+0000 7f231c584740 1 Processor -- start 2024-04-25T13:49:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.969+0000 7f231c584740 1 -- start start 2024-04-25T13:49:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283aa8f60 0x565283e96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac96b0 0x565283ea8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23101ab3c0 con 0x565283ac14d0 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23101acf00 con 0x565283ac96b0 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f231c584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23101aba90 con 0x565283aa8f60 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38994/0 (socket says 172.21.15.94:38994) 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 -- 172.21.15.94:0/1542496833 learned_addr learned my addr 172.21.15.94:0/1542496833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230effd640 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283aa8f60 0x565283e96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230f7fe640 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac96b0 0x565283ea8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 -- 172.21.15.94:0/1542496833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283aa8f60 msgr2=0x565283e96830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283aa8f60 0x565283e96830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 -- 172.21.15.94:0/1542496833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac96b0 msgr2=0x565283ea8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac96b0 0x565283ea8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 -- 172.21.15.94:0/1542496833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565283e79ba0 con 0x565283ac14d0 2024-04-25T13:49:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f230e7fc640 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9c0 tx=0x7f2304009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77bb9d7b9e0be5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.970+0000 7f22f4ff9640 1 -- 172.21.15.94:0/1542496833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040a6dd0 con 0x565283ac14d0 2024-04-25T13:49:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f22f4ff9640 1 -- 172.21.15.94:0/1542496833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2304010720 con 0x565283ac14d0 2024-04-25T13:49:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f22f4ff9640 1 -- 172.21.15.94:0/1542496833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040a6bf0 con 0x565283ac14d0 2024-04-25T13:49:27.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 -- 172.21.15.94:0/1542496833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 msgr2=0x565283e96d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9c0 tx=0x7f2304009310 comp rx=0 tx=0).stop 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 -- 172.21.15.94:0/1542496833 shutdown_connections 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283ac96b0 0x565283ea8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e96d70 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 --2- 172.21.15.94:0/1542496833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283aa8f60 0x565283e96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.971+0000 7f231c584740 1 -- 172.21.15.94:0/1542496833 >> 172.21.15.94:0/1542496833 conn(0x565283ade4e0 msgr2=0x565283e94e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- 172.21.15.94:0/1542496833 shutdown_connections 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- 172.21.15.94:0/1542496833 wait complete. 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- start start 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 Processor -- start 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283aa8f60 0x565283e9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e9bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23101ac830 con 0x565283ac14d0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23101ac160 con 0x565283aa8f60 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23101a93a0 con 0x565283ac96b0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f231c584740 10 client.0 _ll_register_callbacks cb 0x7f23000097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f230e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39010/0 (socket says 172.21.15.94:39010) 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f230e7fc640 1 -- 172.21.15.94:0/417784586 learned_addr learned my addr 172.21.15.94:0/417784586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f230effd640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283aa8f60 0x565283e9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.972+0000 7f230f7fe640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e9bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 -- 172.21.15.94:0/417784586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 msgr2=0x565283e9bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e9bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 -- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283aa8f60 msgr2=0x565283e9a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283aa8f60 0x565283e9a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 -- 172.21.15.94:0/417784586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565283e9c220 con 0x565283ac14d0 2024-04-25T13:49:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f230e7fc640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f23040dcaf0 tx=0x7f2304005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eccdcc45c31119b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f22f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040a7480 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f23040aedc0 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565283df8800 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23040a6f90 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f231c584740 10 client.18258 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565283e9d510 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.973+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565283df8800 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2304018c20 con 0x565283ac14d0 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 10 client.18258 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 10 client.18258 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 20 client.18258 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aea390 0x565283f0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 4 client.18258 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565283d73610 con 0x565283aea390 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f230effd640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aea390 0x565283f0e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f230effd640 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aea390 0x565283f0e910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53cb2b9c300ccba4 server_cookie=b07253a001d9ce6e in_seq=0 out_seq=0 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.974+0000 7f22f2ff5640 10 client.18258 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f22f2ff5640 1 -- 172.21.15.94:0/417784586 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565283d73610 con 0x565283aea390 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f22f2ff5640 10 client.18258 handle_command_reply: tid=0 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 client.18258 shutdown 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 msgr2=0x565283e9ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f23040dcaf0 tx=0x7f2304005740 comp rx=0 tx=0).stop 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 shutdown_connections 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x565283aea390 0x565283f0e910 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565283ac96b0 0x565283e9bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565283ac14d0 0x565283e9ad30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 --2- 172.21.15.94:0/417784586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565283aa8f60 0x565283e9a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 >> 172.21.15.94:0/417784586 conn(0x565283ade4e0 msgr2=0x565283e96830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f22f2ff5640 0 client.18258 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 shutdown_connections 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 1 -- 172.21.15.94:0/417784586 wait complete. 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.975+0000 7f231c584740 20 client.18258 trim_cache size 0 max 16384 2024-04-25T13:49:27.981 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 msgr2=0x7f22f80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 0x7f22f80b1ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x565283e8c700 tx=0x7f2300005f90 comp rx=0 tx=0).stop 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 msgr2=0x7f231019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 shutdown_connections 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f22f80afae0 0x7f22f80b1ff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2310104f00 0x7f23101a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23101020b0 0x7f231019a590 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 --2- 172.21.15.94:0/2857201169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23101015b0 0x7f231019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 >> 172.21.15.94:0/2857201169 conn(0x7f23100fd1f0 msgr2=0x7f2310107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 shutdown_connections 2024-04-25T13:49:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:27.976+0000 7f23155ff640 1 -- 172.21.15.94:0/2857201169 wait complete. 2024-04-25T13:49:27.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:49:28.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 -- 172.21.15.94:0/1960426887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a81045d0 msgr2=0x7fa8a81069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 --2- 172.21.15.94:0/1960426887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a81045d0 0x7fa8a81069f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a410 comp rx=0 tx=0).stop 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 -- 172.21.15.94:0/1960426887 shutdown_connections 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 --2- 172.21.15.94:0/1960426887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 0x7fa8a810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 --2- 172.21.15.94:0/1960426887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a8106f30 0x7fa8a8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 --2- 172.21.15.94:0/1960426887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a81045d0 0x7fa8a81069f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 -- 172.21.15.94:0/1960426887 >> 172.21.15.94:0/1960426887 conn(0x7fa8a80fdbd0 msgr2=0x7fa8a8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 -- 172.21.15.94:0/1960426887 shutdown_connections 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.105+0000 7fa8ada15640 1 -- 172.21.15.94:0/1960426887 wait complete. 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 Processor -- start 2024-04-25T13:49:28.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 -- start start 2024-04-25T13:49:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a81045d0 0x7fa8a819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 0x7fa8a81a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810f610 con 0x7fa8a8106f30 2024-04-25T13:49:28.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810f490 con 0x7fa8a8109890 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8ada15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810f790 con 0x7fa8a81045d0 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39030/0 (socket says 172.21.15.94:39030) 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa89e7fc640 1 -- 172.21.15.94:0/2223478316 learned_addr learned my addr 172.21.15.94:0/2223478316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a81045d0 0x7fa8a819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.106+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 0x7fa8a81a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 -- 172.21.15.94:0/2223478316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a81045d0 msgr2=0x7fa8a819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a81045d0 0x7fa8a819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 -- 172.21.15.94:0/2223478316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 msgr2=0x7fa8a81a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 0x7fa8a81a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 -- 172.21.15.94:0/2223478316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a81a20b0 con 0x7fa8a8106f30 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa89000d950 tx=0x7fa89000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa8aca13640 1 -- 172.21.15.94:0/2223478316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890017030 con 0x7fa8a8106f30 2024-04-25T13:49:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa8aca13640 1 -- 172.21.15.94:0/2223478316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa890004020 con 0x7fa8a8106f30 2024-04-25T13:49:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a81a23a0 con 0x7fa8a8106f30 2024-04-25T13:49:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa8aca13640 1 -- 172.21.15.94:0/2223478316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900047a0 con 0x7fa8a8106f30 2024-04-25T13:49:28.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.107+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a28f0 con 0x7fa8a8106f30 2024-04-25T13:49:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 Processor -- start 2024-04-25T13:49:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 -- start start 2024-04-25T13:49:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8aca13640 1 -- 172.21.15.94:0/2223478316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa890009a30 con 0x7fa8a8106f30 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a69624d80 con 0x556a692f54d0 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556a695dea20 con 0x556a6931e390 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556a69620d40 con 0x556a692fd6b0 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36876/0 (socket says 172.21.15.94:36876) 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39036/0 (socket says 172.21.15.94:39036) 2024-04-25T13:49:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 -- 172.21.15.94:0/3001448809 learned_addr learned my addr 172.21.15.94:0/3001448809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 -- 172.21.15.94:0/3001448809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 msgr2=0x556a696c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 -- 172.21.15.94:0/3001448809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 msgr2=0x556a696c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a77fe640 1 -- 172.21.15.94:0/3001448809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a695e9c30 con 0x556a6931e390 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.110+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009f30 tx=0x7fa8a00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce5ddbe9aeb45eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa8aca13640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 0x7fa8940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 0x7fa8940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa89dffb640 1 -- 172.21.15.94:0/3001448809 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004d20 con 0x556a6931e390 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa89dffb640 1 -- 172.21.15.94:0/3001448809 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a0004050 con 0x556a6931e390 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa8aca13640 1 -- 172.21.15.94:0/2223478316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa8900de6d0 con 0x7fa8a8106f30 2024-04-25T13:49:28.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.111+0000 7fa89dffb640 1 -- 172.21.15.94:0/3001448809 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0005280 con 0x556a6931e390 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3001448809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 msgr2=0x556a696c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009f30 tx=0x7fa8a00118d0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 0x7fa8940b2000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa898031440 tx=0x7fa8980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3001448809 shutdown_connections 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3001448809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696c0810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3001448809 >> 172.21.15.94:0/3001448809 conn(0x556a693124e0 msgr2=0x556a696bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3001448809 shutdown_connections 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3001448809 wait complete. 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 -- start start 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.112+0000 7fa8b49a5740 1 Processor -- start 2024-04-25T13:49:28.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 0x556a696ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a6931e390 0x556a696d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a696c2530 con 0x556a692fd6b0 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556a695abdc0 con 0x556a692f54d0 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556a696c26b0 con 0x556a6931e390 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 10 client.0 _ll_register_callbacks cb 0x556a695c7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39048/0 (socket says 172.21.15.94:39048) 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 -- 172.21.15.94:0/3860894788 learned_addr learned my addr 172.21.15.94:0/3860894788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 0x556a696ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a6931e390 0x556a696d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 -- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a6931e390 msgr2=0x556a696d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a6931e390 0x556a696d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 -- 172.21.15.94:0/3860894788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 msgr2=0x556a696ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 0x556a696ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 -- 172.21.15.94:0/3860894788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a693363f0 con 0x556a692fd6b0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 0x556a696ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa8900a3d80 tx=0x7fa8900a2310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6380866507ee3681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900a6460 con 0x556a692fd6b0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a0004050 con 0x556a692fd6b0 2024-04-25T13:49:28.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.113+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa8900041d0 con 0x556a692fd6b0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8900a6280 con 0x556a692fd6b0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 10 client.18270 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a694007d0 con 0x556a692fd6b0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a0004050 con 0x556a692fd6b0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8900a2da0 con 0x556a692fd6b0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 10 client.18270 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 10 client.18270 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 20 client.18270 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692dcf60 0x556a696d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 4 client.18270 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a69632990 con 0x556a692dcf60 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.114+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692dcf60 0x556a696d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa886ffd640 10 client.18270 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692dcf60 0x556a696d8250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1739f3075e1d7025 server_cookie=2ce4b25e0162dde in_seq=0 out_seq=0 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa886ffd640 1 -- 172.21.15.94:0/3860894788 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a69632990 con 0x556a692dcf60 2024-04-25T13:49:28.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa886ffd640 10 client.18270 handle_command_reply: tid=0 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa8b49a5740 1 client.18270 shutdown 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 msgr2=0x556a696cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.115+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa8900a3d80 tx=0x7fa8900a2310 comp rx=0 tx=0).stop 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 shutdown_connections 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692dcf60 0x556a696d8250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a6931e390 0x556a696d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa886ffd640 0 client.18270 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692fd6b0 0x556a696cab30 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/3860894788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a692f54d0 0x556a696ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 >> 172.21.15.94:0/3860894788 conn(0x556a693124e0 msgr2=0x556a696bb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 shutdown_connections 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 1 -- 172.21.15.94:0/3860894788 wait complete. 2024-04-25T13:49:28.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.116+0000 7fa8b49a5740 20 client.18270 trim_cache size 0 max 16384 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 Processor -- start 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 -- start start 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ab4f0 con 0x556a692f54d0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ad030 con 0x556a6931e390 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.129+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81abbc0 con 0x556a692fd6b0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39058/0 (socket says 172.21.15.94:39058) 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/2058636359 learned_addr learned my addr 172.21.15.94:0/2058636359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/2058636359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 msgr2=0x556a696c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/2058636359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 msgr2=0x556a696dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/2058636359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a696c8710 con 0x556a692f54d0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa898004650 tx=0x7fa89803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58099cd85e40964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa89d7fa640 1 -- 172.21.15.94:0/2058636359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898059070 con 0x556a692f54d0 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa89d7fa640 1 -- 172.21.15.94:0/2058636359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8980548d0 con 0x556a692f54d0 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.130+0000 7fa89d7fa640 1 -- 172.21.15.94:0/2058636359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035040 con 0x556a692f54d0 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- 172.21.15.94:0/2058636359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 msgr2=0x556a696c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa898004650 tx=0x7fa89803f040 comp rx=0 tx=0).stop 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- 172.21.15.94:0/2058636359 shutdown_connections 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/2058636359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696c5350 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- 172.21.15.94:0/2058636359 >> 172.21.15.94:0/2058636359 conn(0x556a693124e0 msgr2=0x556a696c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- 172.21.15.94:0/2058636359 shutdown_connections 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- 172.21.15.94:0/2058636359 wait complete. 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.131+0000 7fa8b49a5740 1 -- start start 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 Processor -- start 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ac960 con 0x556a692f54d0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ac290 con 0x556a6931e390 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81a94d0 con 0x556a692fd6b0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8b49a5740 10 client.0 _ll_register_callbacks cb 0x7fa8900e4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39068/0 (socket says 172.21.15.94:39068) 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/1602303224 learned_addr learned my addr 172.21.15.94:0/1602303224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 msgr2=0x556a696cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/1602303224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 msgr2=0x556a696cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a6ffd640 1 -- 172.21.15.94:0/1602303224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a696d0450 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa8a77fe640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.132+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8a6ffd640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a930 tx=0x7fa898003640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25feb1445055b3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035070 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa898003a50 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8980548d0 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa898059070 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 10 client.18276 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a69740650 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8980548d0 con 0x556a692f54d0 2024-04-25T13:49:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa898044030 con 0x556a692f54d0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 10 client.18276 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 10 client.18276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 20 client.18276 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692b23a0 0x556a69743020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 4 client.18276 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.133+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a6962c800 con 0x556a692b23a0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692b23a0 0x556a69743020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa886ffd640 10 client.18276 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa89e7fc640 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692b23a0 0x556a69743020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d139557ad16fbfb server_cookie=75a3fa486a7b335f in_seq=0 out_seq=0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa886ffd640 1 -- 172.21.15.94:0/1602303224 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a6962c800 con 0x556a692b23a0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa886ffd640 10 client.18276 handle_command_reply: tid=0 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 client.18276 shutdown 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 msgr2=0x556a696cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa89802a930 tx=0x7fa898003640 comp rx=0 tx=0).stop 2024-04-25T13:49:28.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 shutdown_connections 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556a692b23a0 0x556a69743020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a6931e390 0x556a696cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556a692fd6b0 0x556a696cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa886ffd640 0 client.18276 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 --2- 172.21.15.94:0/1602303224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a692f54d0 0x556a696cea20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 >> 172.21.15.94:0/1602303224 conn(0x556a693124e0 msgr2=0x556a696c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 shutdown_connections 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 1 -- 172.21.15.94:0/1602303224 wait complete. 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.134+0000 7fa8b49a5740 20 client.18276 trim_cache size 0 max 16384 2024-04-25T13:49:28.141 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:49:28.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 msgr2=0x7fa8940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 0x7fa8940b2000 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa898031440 tx=0x7fa8980314a0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 msgr2=0x7fa8a819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa89000d950 tx=0x7fa89000de10 comp rx=0 tx=0).stop 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 shutdown_connections 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa8940afaf0 0x7fa8940b2000 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8109890 0x7fa8a81a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8a8106f30 0x7fa8a819aab0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 --2- 172.21.15.94:0/2223478316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a81045d0 0x7fa8a819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 >> 172.21.15.94:0/2223478316 conn(0x7fa8a80fdbd0 msgr2=0x7fa8a8105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 shutdown_connections 2024-04-25T13:49:28.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.135+0000 7fa8ada15640 1 -- 172.21.15.94:0/2223478316 wait complete. 2024-04-25T13:49:28.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:49:28.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 -- 172.21.15.94:0/1613459975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b8108930 msgr2=0x7f61b810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 --2- 172.21.15.94:0/1613459975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b8108930 0x7f61b810ad50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ca0 tx=0x7f61b002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 -- 172.21.15.94:0/1613459975 shutdown_connections 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 --2- 172.21.15.94:0/1613459975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b8108930 0x7f61b810ad50 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 --2- 172.21.15.94:0/1613459975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 --2- 172.21.15.94:0/1613459975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b81029f0 0x7f61b81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 -- 172.21.15.94:0/1613459975 >> 172.21.15.94:0/1613459975 conn(0x7f61b80fdbc0 msgr2=0x7f61b8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 -- 172.21.15.94:0/1613459975 shutdown_connections 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.272+0000 7f61bf12e640 1 -- 172.21.15.94:0/1613459975 wait complete. 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 Processor -- start 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 -- start start 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b8108930 0x7f61b81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61b810e520 con 0x7f61b81029f0 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61b810e3a0 con 0x7f61b8103c30 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bf12e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61b810e6a0 con 0x7f61b8108930 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bceca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bceca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39100/0 (socket says 172.21.15.94:39100) 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61bceca640 1 -- 172.21.15.94:0/4277463784 learned_addr learned my addr 172.21.15.94:0/4277463784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.273+0000 7f61b7fff640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b8108930 0x7f61b81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 -- 172.21.15.94:0/4277463784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b8108930 msgr2=0x7f61b81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b8108930 0x7f61b81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 -- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 msgr2=0x7f61b8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 -- 172.21.15.94:0/4277463784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b8074ed0 con 0x7f61b8103c30 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61bceca640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b7fff640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b81996c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e9f0 tx=0x7f61ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b5ffb640 1 -- 172.21.15.94:0/4277463784 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00ccc0 con 0x7f61b8103c30 2024-04-25T13:49:28.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b80751c0 con 0x7f61b8103c30 2024-04-25T13:49:28.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b5ffb640 1 -- 172.21.15.94:0/4277463784 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61ac0108e0 con 0x7f61b8103c30 2024-04-25T13:49:28.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.274+0000 7f61b5ffb640 1 -- 172.21.15.94:0/4277463784 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac018880 con 0x7f61b8103c30 2024-04-25T13:49:28.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.275+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b8075760 con 0x7f61b8103c30 2024-04-25T13:49:28.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.277+0000 7f61c5fdb740 1 Processor -- start 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.277+0000 7f61c5fdb740 1 -- start start 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5f8390 0x55b9ed99a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5d76b0 0x55b9ed99ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed8fed80 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed8b8a20 con 0x55b9ed5d76b0 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed8fad40 con 0x55b9ed5f8390 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61b5ffb640 1 -- 172.21.15.94:0/4277463784 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f61ac018a20 con 0x7f61b8103c30 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39116/0 (socket says 172.21.15.94:39116) 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 -- 172.21.15.94:0/2006932813 learned_addr learned my addr 172.21.15.94:0/2006932813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5f8390 0x55b9ed99a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61b7fff640 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5d76b0 0x55b9ed99ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 -- 172.21.15.94:0/2006932813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5f8390 msgr2=0x55b9ed99a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5f8390 0x55b9ed99a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 -- 172.21.15.94:0/2006932813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5d76b0 msgr2=0x55b9ed99ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5d76b0 0x55b9ed99ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 -- 172.21.15.94:0/2006932813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ed8c3c30 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.278+0000 7f61bceca640 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f61a80115f0 tx=0x7f61a8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251230db0642d9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61b5ffb640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 0x7f61a00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f619effd640 1 -- 172.21.15.94:0/2006932813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a801e070 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f619effd640 1 -- 172.21.15.94:0/2006932813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a800fb10 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f619effd640 1 -- 172.21.15.94:0/2006932813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8019c30 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 0x7f61a00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61b5ffb640 1 -- 172.21.15.94:0/4277463784 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f61ac0df020 con 0x7f61b8103c30 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2006932813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 msgr2=0x55b9ed99ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 0x7f61a00b2080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f61b00094e0 tx=0x7f61b0009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.279+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f61a80115f0 tx=0x7f61a8011ab0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2006932813 shutdown_connections 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5d76b0 0x55b9ed99ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed99ad90 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2006932813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5f8390 0x55b9ed99a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2006932813 >> 172.21.15.94:0/2006932813 conn(0x55b9ed5ec4e0 msgr2=0x55b9ed996020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2006932813 shutdown_connections 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2006932813 wait complete. 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 -- start start 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.280+0000 7f61c5fdb740 1 Processor -- start 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5f8390 0x55b9ed9aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed99c530 con 0x55b9ed5d76b0 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed885dc0 con 0x55b9ed5f8390 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9ed99c6b0 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61c5fdb740 10 client.0 _ll_register_callbacks cb 0x55b9ed8a1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bd6cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5f8390 0x55b9ed9aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39118/0 (socket says 172.21.15.94:39118) 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61b7fff640 1 -- 172.21.15.94:0/1028624977 learned_addr learned my addr 172.21.15.94:0/1028624977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 -- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5f8390 msgr2=0x55b9ed9aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5f8390 0x55b9ed9aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 -- 172.21.15.94:0/1028624977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 msgr2=0x55b9ed9a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61bceca640 1 -- 172.21.15.94:0/1028624977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ed6103f0 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.281+0000 7f61b7fff640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f61bceca640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9a45a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61a801c040 tx=0x7f61a8019600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1177ac0349d84ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f6197fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a801e040 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a800fb10 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f61a800f760 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61a800f960 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f61c5fdb740 10 client.27613 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9ed6da7d0 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.282+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a800fb10 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f61a8021070 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 10 client.27613 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 10 client.27613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 20 client.27613 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5b6f60 0x55b9ed9b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 4 client.27613 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9ed90c990 con 0x55b9ed5b6f60 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61b7fff640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5b6f60 0x55b9ed9b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f6197fff640 10 client.27613 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.283+0000 7f61b7fff640 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5b6f60 0x55b9ed9b2010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6b4684da1899e18 server_cookie=c172789d7ed2ccf2 in_seq=0 out_seq=0 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f6197fff640 1 -- 172.21.15.94:0/1028624977 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9ed90c990 con 0x55b9ed5b6f60 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f6197fff640 10 client.27613 handle_command_reply: tid=0 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 client.27613 shutdown 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 msgr2=0x55b9ed9a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9a45a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61a801c040 tx=0x7f61a8019600 comp rx=0 tx=0).stop 2024-04-25T13:49:28.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 shutdown_connections 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5b6f60 0x55b9ed9b2010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5f8390 0x55b9ed9aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f6197fff640 0 client.27613 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/1028624977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9a45a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.284+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 >> 172.21.15.94:0/1028624977 conn(0x55b9ed5ec4e0 msgr2=0x55b9ed998f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.285+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 shutdown_connections 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.285+0000 7f61c5fdb740 1 -- 172.21.15.94:0/1028624977 wait complete. 2024-04-25T13:49:28.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.285+0000 7f61c5fdb740 20 client.27613 trim_cache size 0 max 16384 2024-04-25T13:49:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 Processor -- start 2024-04-25T13:49:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 -- start start 2024-04-25T13:49:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81ad7b0 con 0x55b9ed5d76b0 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81af2f0 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81ade80 con 0x55b9ed5b6f60 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61bd6cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61bd6cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39126/0 (socket says 172.21.15.94:39126) 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61bd6cb640 1 -- 172.21.15.94:0/732451828 learned_addr learned my addr 172.21.15.94:0/732451828 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.300+0000 7f61bceca640 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 -- 172.21.15.94:0/732451828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 msgr2=0x55b9ed9b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 -- 172.21.15.94:0/732451828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 msgr2=0x55b9ed9b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 -- 172.21.15.94:0/732451828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ed9abea0 con 0x55b9ed5d76b0 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61b7fff640 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f61b0035000 tx=0x7f61b003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f2a09982545154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f619e7fc640 1 -- 172.21.15.94:0/732451828 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0056070 con 0x55b9ed5d76b0 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f619e7fc640 1 -- 172.21.15.94:0/732451828 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0036040 con 0x55b9ed5d76b0 2024-04-25T13:49:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.301+0000 7f619e7fc640 1 -- 172.21.15.94:0/732451828 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b002fa90 con 0x55b9ed5d76b0 2024-04-25T13:49:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- 172.21.15.94:0/732451828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 msgr2=0x55b9ed9ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f61b0035000 tx=0x7f61b003b000 comp rx=0 tx=0).stop 2024-04-25T13:49:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- 172.21.15.94:0/732451828 shutdown_connections 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9ed9ab960 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/732451828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- 172.21.15.94:0/732451828 >> 172.21.15.94:0/732451828 conn(0x55b9ed5ec4e0 msgr2=0x55b9ed9a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- 172.21.15.94:0/732451828 shutdown_connections 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- 172.21.15.94:0/732451828 wait complete. 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 -- start start 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.302+0000 7f61c5fdb740 1 Processor -- start 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81aec20 con 0x55b9ed5d76b0 2024-04-25T13:49:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81ae550 con 0x55b9ed5cf4d0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61b81ab790 con 0x55b9ed5b6f60 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61c5fdb740 10 client.0 _ll_register_callbacks cb 0x7f61a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39142/0 (socket says 172.21.15.94:39142) 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 -- 172.21.15.94:0/2039131888 learned_addr learned my addr 172.21.15.94:0/2039131888 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61b7fff640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bceca640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 -- 172.21.15.94:0/2039131888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 msgr2=0x55b9ed9aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 -- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 msgr2=0x55b9ed9aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 -- 172.21.15.94:0/2039131888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9eda188b0 con 0x55b9ed5d76b0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f61bd6cb640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ca0 tx=0x7f61b00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebaf8852f3bac75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.303+0000 7f6197fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0056040 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f61b002fa50 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b0036040 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61b002fc70 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 10 client.18294 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9eda199e0 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0036040 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f61b0044040 con 0x55b9ed5d76b0 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 10 client.18294 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 10 client.18294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 20 client.18294 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5f8390 0x55b9eda1c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 4 client.18294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:49:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9ed906800 con 0x55b9ed5f8390 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.304+0000 7f61bceca640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5f8390 0x55b9eda1c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f6197fff640 10 client.18294 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61bceca640 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5f8390 0x55b9eda1c400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6eedb64334fcb265 server_cookie=a16a6116987146c4 in_seq=0 out_seq=0 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f6197fff640 1 -- 172.21.15.94:0/2039131888 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9ed906800 con 0x55b9ed5f8390 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f6197fff640 10 client.18294 handle_command_reply: tid=0 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 client.18294 shutdown 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 msgr2=0x55b9eda18370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ca0 tx=0x7f61b00048a0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 shutdown_connections 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b9ed5f8390 0x55b9eda1c400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b9ed5d76b0 0x55b9eda18370 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f6197fff640 0 client.18294 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9ed5cf4d0 0x55b9ed9aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 --2- 172.21.15.94:0/2039131888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9ed5b6f60 0x55b9ed9aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.305+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 >> 172.21.15.94:0/2039131888 conn(0x55b9ed5ec4e0 msgr2=0x55b9ed9a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.306+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 shutdown_connections 2024-04-25T13:49:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.306+0000 7f61c5fdb740 1 -- 172.21.15.94:0/2039131888 wait complete. 2024-04-25T13:49:28.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.306+0000 7f61c5fdb740 20 client.18294 trim_cache size 0 max 16384 2024-04-25T13:49:28.312 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 msgr2=0x7f61a00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 0x7f61a00b2080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f61b00094e0 tx=0x7f61b0009800 comp rx=0 tx=0).stop 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 msgr2=0x7f61b81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b81996c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e9f0 tx=0x7f61ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 shutdown_connections 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61a00afb70 0x7f61a00b2080 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61b8108930 0x7f61b81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61b8103c30 0x7f61b81996c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 --2- 172.21.15.94:0/4277463784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61b81029f0 0x7f61b8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 >> 172.21.15.94:0/4277463784 conn(0x7f61b80fdbc0 msgr2=0x7f61b8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 shutdown_connections 2024-04-25T13:49:28.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.307+0000 7f61bf12e640 1 -- 172.21.15.94:0/4277463784 wait complete. 2024-04-25T13:49:28.318 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:28 smithi094 ceph-mon[29195]: pgmap v577: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-04-25T13:49:28.318 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:28 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1650474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:49:28.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:49:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:28 smithi137 ceph-mon[32568]: pgmap v577: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-04-25T13:49:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:28 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1650474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 -- 172.21.15.94:0/1452287961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450109870 msgr2=0x7f045010bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 --2- 172.21.15.94:0/1452287961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450109870 0x7f045010bc90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0448009ca0 tx=0x7f044802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 -- 172.21.15.94:0/1452287961 shutdown_connections 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 --2- 172.21.15.94:0/1452287961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450109870 0x7f045010bc90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 --2- 172.21.15.94:0/1452287961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450104b70 0x7f0450104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 --2- 172.21.15.94:0/1452287961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450103930 0x7f0450104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 -- 172.21.15.94:0/1452287961 >> 172.21.15.94:0/1452287961 conn(0x7f04500feb20 msgr2=0x7f0450100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 -- 172.21.15.94:0/1452287961 shutdown_connections 2024-04-25T13:49:28.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 -- 172.21.15.94:0/1452287961 wait complete. 2024-04-25T13:49:28.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.441+0000 7f04549fe640 1 Processor -- start 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 -- start start 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450104b70 0x7f045019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109870 0x7f04501a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f045010f460 con 0x7f0450103930 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f045010f2e0 con 0x7f0450104b70 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f04549fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f045010f5e0 con 0x7f0450109870 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39158/0 (socket says 172.21.15.94:39158) 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 -- 172.21.15.94:0/338746041 learned_addr learned my addr 172.21.15.94:0/338746041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044659c640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450104b70 0x7f045019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044f59e640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109870 0x7f04501a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 -- 172.21.15.94:0/338746041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109870 msgr2=0x7f04501a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109870 0x7f04501a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 -- 172.21.15.94:0/338746041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450104b70 msgr2=0x7f045019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450104b70 0x7f045019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.442+0000 7f044ed9d640 1 -- 172.21.15.94:0/338746041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04501a1b40 con 0x7f0450103930 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f044ed9d640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f044cd99640 1 -- 172.21.15.94:0/338746041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000ccc0 con 0x7f0450103930 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f044cd99640 1 -- 172.21.15.94:0/338746041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04400035f0 con 0x7f0450103930 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04501a1e30 con 0x7f0450103930 2024-04-25T13:49:28.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f044cd99640 1 -- 172.21.15.94:0/338746041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000ce60 con 0x7f0450103930 2024-04-25T13:49:28.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.443+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a2290 con 0x7f0450103930 2024-04-25T13:49:28.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 Processor -- start 2024-04-25T13:49:28.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f044cd99640 1 -- 172.21.15.94:0/338746041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0440003aa0 con 0x7f0450103930 2024-04-25T13:49:28.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 -- start start 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c66f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 0x55620c670bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55620c5d3d80 con 0x55620c2a44d0 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55620c58da20 con 0x55620c2cd390 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f045c18d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55620c5cfd40 con 0x55620c2ac6b0 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.446+0000 7f044ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39160/0 (socket says 172.21.15.94:39160) 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c66f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 -- 172.21.15.94:0/1774935511 learned_addr learned my addr 172.21.15.94:0/1774935511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044659c640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 0x55620c670bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044cd99640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 0x7f043c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 -- 172.21.15.94:0/1774935511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 msgr2=0x55620c670bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 0x55620c670bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 -- 172.21.15.94:0/1774935511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 msgr2=0x55620c66f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c66f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 -- 172.21.15.94:0/1774935511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620c598c30 con 0x55620c2a44d0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044659c640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 0x55620c670bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044f59e640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c66f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044f59e640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 0x7f043c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044cd99640 1 -- 172.21.15.94:0/338746041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f04400defb0 con 0x7f0450103930 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044ed9d640 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f04400154a0 tx=0x7f0440021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e27e32f00ca0f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044559a640 1 -- 172.21.15.94:0/1774935511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400a71c0 con 0x55620c2a44d0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044f59e640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 0x7f043c0b1ff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55620c670a30 tx=0x7f04480095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044559a640 1 -- 172.21.15.94:0/1774935511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04400a6f00 con 0x55620c2a44d0 2024-04-25T13:49:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.447+0000 7f044559a640 1 -- 172.21.15.94:0/1774935511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400152d0 con 0x55620c2a44d0 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 -- 172.21.15.94:0/1774935511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 msgr2=0x55620c66ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f04400154a0 tx=0x7f0440021000 comp rx=0 tx=0).stop 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 -- 172.21.15.94:0/1774935511 shutdown_connections 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2ac6b0 0x55620c670bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2a44d0 0x55620c66ff60 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 --2- 172.21.15.94:0/1774935511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c66f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 -- 172.21.15.94:0/1774935511 >> 172.21.15.94:0/1774935511 conn(0x55620c2c14e0 msgr2=0x55620c66b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 -- 172.21.15.94:0/1774935511 shutdown_connections 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.448+0000 7f045c18d740 1 -- 172.21.15.94:0/1774935511 wait complete. 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 -- start start 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 Processor -- start 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c679710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c680bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55620c6716a0 con 0x55620c2ac6b0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55620c55adc0 con 0x55620c2cd390 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55620c671820 con 0x55620c2a44d0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f045c18d740 10 client.0 _ll_register_callbacks cb 0x55620c576600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39166/0 (socket says 172.21.15.94:39166) 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f044659c640 1 -- 172.21.15.94:0/4051301634 learned_addr learned my addr 172.21.15.94:0/4051301634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f044f59e640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c680bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.449+0000 7f044ed9d640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c679710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 -- 172.21.15.94:0/4051301634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 msgr2=0x55620c679710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c679710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 -- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 msgr2=0x55620c680bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c680bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 -- 172.21.15.94:0/4051301634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620c2e53f0 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044ed9d640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c679710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044f59e640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c680bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f044659c640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0438012720 tx=0x7f0438012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309580af0c1174b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f04367fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004050 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f04380041f0 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04400a6f00 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04380043f0 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f045c18d740 10 client.18306 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55620c3af7d0 con 0x55620c2ac6b0 2024-04-25T13:49:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.450+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043800fdd0 con 0x55620c2ac6b0 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0438026450 con 0x55620c2ac6b0 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 10 client.18306 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 20 client.18306 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c28bf60 0x55620c687410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 4 client.18306 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04400a6f00 con 0x55620c28bf60 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f044ed9d640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c28bf60 0x55620c687410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.451+0000 7f04367fc640 10 client.18306 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:28.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f044ed9d640 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c28bf60 0x55620c687410 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f89917a63381bb0 server_cookie=e088dfe17224927d in_seq=0 out_seq=0 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f04367fc640 1 -- 172.21.15.94:0/4051301634 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f04400a6f00 con 0x55620c28bf60 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f04367fc640 10 client.18306 handle_command_reply: tid=0 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f045c18d740 1 client.18306 shutdown 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 msgr2=0x55620c679c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.452+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0438012720 tx=0x7f0438012be0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 shutdown_connections 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c28bf60 0x55620c687410 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c680bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c679c50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 --2- 172.21.15.94:0/4051301634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c679710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 >> 172.21.15.94:0/4051301634 conn(0x55620c2c14e0 msgr2=0x55620c66e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f04367fc640 0 client.18306 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 shutdown_connections 2024-04-25T13:49:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 1 -- 172.21.15.94:0/4051301634 wait complete. 2024-04-25T13:49:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.453+0000 7f045c18d740 20 client.18306 trim_cache size 0 max 16384 2024-04-25T13:49:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.467+0000 7f045c18d740 1 Processor -- start 2024-04-25T13:49:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.467+0000 7f045c18d740 1 -- start start 2024-04-25T13:49:28.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.467+0000 7f045c18d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c68cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.467+0000 7f045c18d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f045c18d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c677e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f045c18d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04501a91b0 con 0x55620c2ac6b0 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f045c18d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04501aacf0 con 0x55620c2cd390 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f045c18d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04501a9880 con 0x55620c2a44d0 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39178/0 (socket says 172.21.15.94:39178) 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 -- 172.21.15.94:0/2531018483 learned_addr learned my addr 172.21.15.94:0/2531018483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044f59e640 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c677e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044ed9d640 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c68cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 -- 172.21.15.94:0/2531018483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 msgr2=0x55620c68cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c68cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 -- 172.21.15.94:0/2531018483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 msgr2=0x55620c677e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c677e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 -- 172.21.15.94:0/2531018483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620c65cba0 con 0x55620c2ac6b0 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f044659c640 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0438005bc0 tx=0x7f043800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54474056d2867b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f0444d99640 1 -- 172.21.15.94:0/2531018483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438026550 con 0x55620c2ac6b0 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f0444d99640 1 -- 172.21.15.94:0/2531018483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043800fdd0 con 0x55620c2ac6b0 2024-04-25T13:49:28.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.468+0000 7f0444d99640 1 -- 172.21.15.94:0/2531018483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004590 con 0x55620c2ac6b0 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 -- 172.21.15.94:0/2531018483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 msgr2=0x55620c68d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0438005bc0 tx=0x7f043800f790 comp rx=0 tx=0).stop 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 -- 172.21.15.94:0/2531018483 shutdown_connections 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2cd390 0x55620c677e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c68d150 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.469+0000 7f045c18d740 1 --2- 172.21.15.94:0/2531018483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2a44d0 0x55620c68cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- 172.21.15.94:0/2531018483 >> 172.21.15.94:0/2531018483 conn(0x55620c2c14e0 msgr2=0x55620c66f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- 172.21.15.94:0/2531018483 shutdown_connections 2024-04-25T13:49:28.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- 172.21.15.94:0/2531018483 wait complete. 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- start start 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 Processor -- start 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2a44d0 0x55620c67dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 0x55620c67f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04501aa620 con 0x55620c2ac6b0 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04501a9f50 con 0x55620c2a44d0 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04501a7190 con 0x55620c2cd390 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f045c18d740 10 client.0 _ll_register_callbacks cb 0x7f0438009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f044659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39186/0 (socket says 172.21.15.94:39186) 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f044659c640 1 -- 172.21.15.94:0/3903241777 learned_addr learned my addr 172.21.15.94:0/3903241777 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:28.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.470+0000 7f044ed9d640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2a44d0 0x55620c67dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044f59e640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 0x55620c67f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 -- 172.21.15.94:0/3903241777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 msgr2=0x55620c67f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 0x55620c67f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 -- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2a44d0 msgr2=0x55620c67dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2a44d0 0x55620c67dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 -- 172.21.15.94:0/3903241777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620c67f670 con 0x55620c2ac6b0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044f59e640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 0x55620c67f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f044659c640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0438026600 tx=0x7f0438026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950028fb2706adef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438026a10 con 0x55620c2ac6b0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0438026bb0 con 0x55620c2ac6b0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043800fdd0 con 0x55620c2ac6b0 2024-04-25T13:49:28.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f043800f920 con 0x55620c2ac6b0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f045c18d740 10 client.18312 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55620c6ef980 con 0x55620c2ac6b0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043800fdd0 con 0x55620c2ac6b0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0438037060 con 0x55620c2ac6b0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f045c18d740 10 client.18312 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.471+0000 7f045c18d740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 20 client.18312 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c2613a0 0x55620c6f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 4 client.18312 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55620c5e1990 con 0x55620c2613a0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f044ed9d640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c2613a0 0x55620c6f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f04367fc640 10 client.18312 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:28.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f044ed9d640 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c2613a0 0x55620c6f23e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d257e033c7baf4 server_cookie=71e3033216433f84 in_seq=0 out_seq=0 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f04367fc640 1 -- 172.21.15.94:0/3903241777 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55620c5e1990 con 0x55620c2613a0 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f04367fc640 10 client.18312 handle_command_reply: tid=0 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 client.18312 shutdown 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 msgr2=0x55620c67e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0438026600 tx=0x7f0438026630 comp rx=0 tx=0).stop 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.472+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 shutdown_connections 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55620c2613a0 0x55620c6f23e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04367fc640 0 client.18312 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55620c2cd390 0x55620c67f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55620c2ac6b0 0x55620c67e1d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 --2- 172.21.15.94:0/3903241777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55620c2a44d0 0x55620c67dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 >> 172.21.15.94:0/3903241777 conn(0x55620c2c14e0 msgr2=0x55620c66f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 shutdown_connections 2024-04-25T13:49:28.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 1 -- 172.21.15.94:0/3903241777 wait complete. 2024-04-25T13:49:28.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f045c18d740 20 client.18312 trim_cache size 0 max 16384 2024-04-25T13:49:28.480 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 msgr2=0x7f043c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 0x7f043c0b1ff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55620c670a30 tx=0x7f04480095c0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 msgr2=0x7f045019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f044000e9f0 tx=0x7f044000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 shutdown_connections 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f043c0afae0 0x7f043c0b1ff0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0450109870 0x7f04501a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0450104b70 0x7f045019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 --2- 172.21.15.94:0/338746041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0450103930 0x7f045019a000 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:28.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.473+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 >> 172.21.15.94:0/338746041 conn(0x7f04500feb20 msgr2=0x7f0450105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:28.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.474+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 shutdown_connections 2024-04-25T13:49:28.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:28.474+0000 7f04549fe640 1 -- 172.21.15.94:0/338746041 wait complete. 2024-04-25T13:49:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:28 smithi138 ceph-mon[32977]: pgmap v577: 97 pgs: 97 active+clean; 40 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 346 op/s 2024-04-25T13:49:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:28 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1650474494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:49:28.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 905586 12.88 MB/sec execute 251 sec latency 9.408 ms 2024-04-25T13:49:29.485 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:49:29.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:49:29.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:49:29.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 -- 172.21.15.94:0/2508898201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab48108920 msgr2=0x7fab4810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/2508898201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab48108920 0x7fab4810ad40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fab40009ca0 tx=0x7fab4002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 -- 172.21.15.94:0/2508898201 shutdown_connections 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/2508898201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab48108920 0x7fab4810ad40 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/2508898201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48103c20 0x7fab48104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/2508898201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab481029e0 0x7fab481036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 -- 172.21.15.94:0/2508898201 >> 172.21.15.94:0/2508898201 conn(0x7fab480fdbd0 msgr2=0x7fab48100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 -- 172.21.15.94:0/2508898201 shutdown_connections 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.610+0000 7fab4f5e3640 1 -- 172.21.15.94:0/2508898201 wait complete. 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 Processor -- start 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 -- start start 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 0x7fab48199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48108920 0x7fab481a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810e510 con 0x7fab481029e0 2024-04-25T13:49:29.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810e390 con 0x7fab48103c20 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4f5e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810e690 con 0x7fab48108920 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39212/0 (socket says 172.21.15.94:39212) 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4d37f640 1 -- 172.21.15.94:0/4268150170 learned_addr learned my addr 172.21.15.94:0/4268150170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 0x7fab48199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.611+0000 7fab4db80640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48108920 0x7fab481a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 -- 172.21.15.94:0/4268150170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48108920 msgr2=0x7fab481a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48108920 0x7fab481a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 -- 172.21.15.94:0/4268150170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 msgr2=0x7fab48199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 0x7fab48199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 -- 172.21.15.94:0/4268150170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab481a0f20 con 0x7fab481029e0 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 0x7fab48199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4d37f640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fab4400e9f0 tx=0x7fab4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab3e7fc640 1 -- 172.21.15.94:0/4268150170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4400ccc0 con 0x7fab481029e0 2024-04-25T13:49:29.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab3e7fc640 1 -- 172.21.15.94:0/4268150170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440035f0 con 0x7fab481029e0 2024-04-25T13:49:29.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab481a1210 con 0x7fab481029e0 2024-04-25T13:49:29.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab3e7fc640 1 -- 172.21.15.94:0/4268150170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4400ce60 con 0x7fab481029e0 2024-04-25T13:49:29.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.612+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab48069e90 con 0x7fab481029e0 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab3e7fc640 1 -- 172.21.15.94:0/4268150170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fab44003aa0 con 0x7fab481029e0 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 Processor -- start 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 -- start start 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c5484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c54f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563a3c4adb00 con 0x563a3c186730 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a3c449920 con 0x563a3c17e550 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab56569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a3c4a9ac0 con 0x563a3c1a7410 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37016/0 (socket says 172.21.15.94:37016) 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 -- 172.21.15.94:0/1013802032 learned_addr learned my addr 172.21.15.94:0/1013802032 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c54f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4db80640 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c5484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 -- 172.21.15.94:0/1013802032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 msgr2=0x563a3c5484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c5484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 -- 172.21.15.94:0/1013802032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 msgr2=0x563a3c54f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c54f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.615+0000 7fab4d37f640 1 -- 172.21.15.94:0/1013802032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3c2877e0 con 0x563a3c17e550 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab3e7fc640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 0x7fab340b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab4d37f640 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab440e2f30 tx=0x7fab440e71a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2dfcecb1b865bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab4db80640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 0x7fab340b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab3e7fc640 1 -- 172.21.15.94:0/4268150170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fab440defb0 con 0x7fab481029e0 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab337fe640 1 -- 172.21.15.94:0/1013802032 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440a32b0 con 0x563a3c17e550 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab337fe640 1 -- 172.21.15.94:0/1013802032 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440a6f00 con 0x563a3c17e550 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.616+0000 7fab337fe640 1 -- 172.21.15.94:0/1013802032 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440a6b60 con 0x563a3c17e550 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- 172.21.15.94:0/1013802032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 msgr2=0x563a3c548a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab4db80640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 0x7fab340b1ff0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x563a3c549670 tx=0x7fab40009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab440e2f30 tx=0x7fab440e71a0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- 172.21.15.94:0/1013802032 shutdown_connections 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c54f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c548a20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 --2- 172.21.15.94:0/1013802032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c5484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- 172.21.15.94:0/1013802032 >> 172.21.15.94:0/1013802032 conn(0x563a3c19b560 msgr2=0x563a3c543e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- 172.21.15.94:0/1013802032 shutdown_connections 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- 172.21.15.94:0/1013802032 wait complete. 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 -- start start 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.617+0000 7fab56569740 1 Processor -- start 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c552800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c559760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563a3c550370 con 0x563a3c17e550 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a3c4475d0 con 0x563a3c186730 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a3c5504f0 con 0x563a3c1a7410 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab56569740 10 client.0 _ll_register_callbacks cb 0x563a3c435080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c552800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4d37f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39224/0 (socket says 172.21.15.94:39224) 2024-04-25T13:49:29.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4d37f640 1 -- 172.21.15.94:0/3216302271 learned_addr learned my addr 172.21.15.94:0/3216302271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4db80640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c559760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3216302271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 msgr2=0x563a3c559760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c559760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3216302271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 msgr2=0x563a3c5522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3216302271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3c2e7f70 con 0x563a3c186730 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c552800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab4819a480 tx=0x7fab3800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb440d6fd4b2112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.618+0000 7fab30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38012110 con 0x563a3c186730 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fab38004050 con 0x563a3c186730 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab440a6f00 con 0x563a3c186730 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab380122b0 con 0x563a3c186730 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 10 client.27794 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a3c17f170 con 0x563a3c186730 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab440a6f00 con 0x563a3c186730 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab38010430 con 0x563a3c186730 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 10 client.27794 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 10 client.27794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 20 client.27794 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c1661f0 0x563a3c55fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 4 client.27794 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.619+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a3c4bb710 con 0x563a3c1661f0 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.620+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c1661f0 0x563a3c55fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.620+0000 7fab30ff9640 10 client.27794 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.620+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c1661f0 0x563a3c55fca0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57985d1a49daa65 server_cookie=e02a2c3a60d62ed6 in_seq=0 out_seq=0 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab30ff9640 1 -- 172.21.15.94:0/3216302271 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a3c4bb710 con 0x563a3c1661f0 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab30ff9640 10 client.27794 handle_command_reply: tid=0 2024-04-25T13:49:29.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 client.27794 shutdown 2024-04-25T13:49:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 msgr2=0x563a3c552800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c552800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab4819a480 tx=0x7fab3800f790 comp rx=0 tx=0).stop 2024-04-25T13:49:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 shutdown_connections 2024-04-25T13:49:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c1661f0 0x563a3c55fca0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c559760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c552800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 --2- 172.21.15.94:0/3216302271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c17e550 0x563a3c5522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab30ff9640 0 client.27794 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 >> 172.21.15.94:0/3216302271 conn(0x563a3c19b560 msgr2=0x563a3c542ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 shutdown_connections 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 1 -- 172.21.15.94:0/3216302271 wait complete. 2024-04-25T13:49:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.621+0000 7fab56569740 20 client.27794 trim_cache size 0 max 16384 2024-04-25T13:49:29.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 Processor -- start 2024-04-25T13:49:29.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 -- start start 2024-04-25T13:49:29.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c17e550 0x563a3c551030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 0x563a3c5424c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ad6e0 con 0x563a3c1a7410 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab481af220 con 0x563a3c186730 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab56569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab481addb0 con 0x563a3c17e550 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37032/0 (socket says 172.21.15.94:37032) 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3673763624 learned_addr learned my addr 172.21.15.94:0/3673763624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c17e550 0x563a3c551030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4db80640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 0x563a3c5424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3673763624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c17e550 msgr2=0x563a3c551030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c17e550 0x563a3c551030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3673763624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 msgr2=0x563a3c5424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 0x563a3c5424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3673763624 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3c54d0d0 con 0x563a3c186730 2024-04-25T13:49:29.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.641+0000 7fab4db80640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 0x563a3c5424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:29.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.642+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab380109a0 tx=0x7fab380121f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466ace3e826ae865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.642+0000 7fab32ffd640 1 -- 172.21.15.94:0/3673763624 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010430 con 0x563a3c186730 2024-04-25T13:49:29.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.642+0000 7fab32ffd640 1 -- 172.21.15.94:0/3673763624 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab380105d0 con 0x563a3c186730 2024-04-25T13:49:29.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.642+0000 7fab32ffd640 1 -- 172.21.15.94:0/3673763624 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3802edf0 con 0x563a3c186730 2024-04-25T13:49:29.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- 172.21.15.94:0/3673763624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 msgr2=0x563a3c559680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab380109a0 tx=0x7fab380121f0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- 172.21.15.94:0/3673763624 shutdown_connections 2024-04-25T13:49:29.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c1a7410 0x563a3c5424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c186730 0x563a3c559680 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 --2- 172.21.15.94:0/3673763624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c17e550 0x563a3c551030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- 172.21.15.94:0/3673763624 >> 172.21.15.94:0/3673763624 conn(0x563a3c19b560 msgr2=0x563a3c550920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- 172.21.15.94:0/3673763624 shutdown_connections 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- 172.21.15.94:0/3673763624 wait complete. 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 -- start start 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.643+0000 7fab56569740 1 Processor -- start 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c5570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c558540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab481aeb50 con 0x563a3c186730 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ae480 con 0x563a3c17e550 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ab6c0 con 0x563a3c1a7410 2024-04-25T13:49:29.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab56569740 10 client.0 _ll_register_callbacks cb 0x7fab38012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39238/0 (socket says 172.21.15.94:39238) 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3203349077 learned_addr learned my addr 172.21.15.94:0/3203349077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c5570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4db80640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c558540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3203349077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 msgr2=0x563a3c558540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c558540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 msgr2=0x563a3c5570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c5570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 -- 172.21.15.94:0/3203349077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3c558b00 con 0x563a3c186730 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab4cb7e640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab3800fd90 tx=0x7fab3802eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fc92556de7ddb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.644+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010430 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fab3802e640 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab380105d0 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab38010a20 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 10 client.18330 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a3c5c8250 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab380105d0 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab38036030 con 0x563a3c186730 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 10 client.18330 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 20 client.18330 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c13b630 0x563a3c5cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 4 client.18330 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a3c4b5580 con 0x563a3c13b630 2024-04-25T13:49:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.645+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c13b630 0x563a3c5cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.646+0000 7fab30ff9640 10 client.18330 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.646+0000 7fab4d37f640 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c13b630 0x563a3c5cabc0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0fc936c6896770 server_cookie=f43ff7fcd667f2cf in_seq=0 out_seq=0 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab30ff9640 1 -- 172.21.15.94:0/3203349077 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a3c4b5580 con 0x563a3c13b630 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab30ff9640 10 client.18330 handle_command_reply: tid=0 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 client.18330 shutdown 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 msgr2=0x563a3c557610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab3800fd90 tx=0x7fab3802eda0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 shutdown_connections 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a3c13b630 0x563a3c5cabc0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a3c1a7410 0x563a3c558540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a3c186730 0x563a3c557610 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 --2- 172.21.15.94:0/3203349077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a3c17e550 0x563a3c5570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab30ff9640 0 client.18330 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 >> 172.21.15.94:0/3203349077 conn(0x563a3c19b560 msgr2=0x563a3c546c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 shutdown_connections 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 1 -- 172.21.15.94:0/3203349077 wait complete. 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.647+0000 7fab56569740 20 client.18330 trim_cache size 0 max 16384 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b3c04c66-994e-4a87-9fac-ad06b8bec4a4", 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:49:29.653 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:49:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 msgr2=0x7fab340b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 0x7fab340b1ff0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x563a3c549670 tx=0x7fab40009450 comp rx=0 tx=0).stop 2024-04-25T13:49:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 msgr2=0x7fab481993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fab4400e9f0 tx=0x7fab4400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 shutdown_connections 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab340afae0 0x7fab340b1ff0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab48108920 0x7fab481a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab48103c20 0x7fab48199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 --2- 172.21.15.94:0/4268150170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab481029e0 0x7fab481993e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 >> 172.21.15.94:0/4268150170 conn(0x7fab480fdbd0 msgr2=0x7fab48104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 shutdown_connections 2024-04-25T13:49:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.648+0000 7fab4f5e3640 1 -- 172.21.15.94:0/4268150170 wait complete. 2024-04-25T13:49:29.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:49:29.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 -- 172.21.15.94:0/2627134801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60103920 msgr2=0x7f6c60103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 --2- 172.21.15.94:0/2627134801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60103920 0x7f6c60103dd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009900 tx=0x7f6c5402a410 comp rx=0 tx=0).stop 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 -- 172.21.15.94:0/2627134801 shutdown_connections 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 --2- 172.21.15.94:0/2627134801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60104460 0x7f6c6010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 --2- 172.21.15.94:0/2627134801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60103920 0x7f6c60103dd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 --2- 172.21.15.94:0/2627134801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 0x7f6c60103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 -- 172.21.15.94:0/2627134801 >> 172.21.15.94:0/2627134801 conn(0x7f6c600fd340 msgr2=0x7f6c600ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 -- 172.21.15.94:0/2627134801 shutdown_connections 2024-04-25T13:49:29.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.780+0000 7f6c64a97640 1 -- 172.21.15.94:0/2627134801 wait complete. 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 Processor -- start 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 -- start start 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 0x7f6c60199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 0x7f6c601a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6010e510 con 0x7f6c60104460 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6010e390 con 0x7f6c60103920 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c64a97640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6010e690 con 0x7f6c60102f40 2024-04-25T13:49:29.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 0x7f6c601a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37066/0 (socket says 172.21.15.94:37066) 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 0x7f6c601a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39248/0 (socket says 172.21.15.94:39248) 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3205277260 learned_addr learned my addr 172.21.15.94:0/3205277260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.781+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 0x7f6c60199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3205277260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 msgr2=0x7f6c60199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 0x7f6c60199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 msgr2=0x7f6c601a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 0x7f6c601a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3205277260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c601a0c50 con 0x7f6c60103920 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c54006010 tx=0x7f6c540042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c4f7fe640 1 -- 172.21.15.94:0/3205277260 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5403f030 con 0x7f6c60103920 2024-04-25T13:49:29.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c4f7fe640 1 -- 172.21.15.94:0/3205277260 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c54031910 con 0x7f6c60103920 2024-04-25T13:49:29.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c601a0ee0 con 0x7f6c60103920 2024-04-25T13:49:29.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c4f7fe640 1 -- 172.21.15.94:0/3205277260 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54035070 con 0x7f6c60103920 2024-04-25T13:49:29.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.782+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c601a12e0 con 0x7f6c60103920 2024-04-25T13:49:29.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 Processor -- start 2024-04-25T13:49:29.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 -- start start 2024-04-25T13:49:29.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a654d0 0x562994e2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 0x562994e30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562994d94d80 con 0x562994a6d6b0 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562994d4e8b0 con 0x562994a8e390 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562994d90d40 con 0x562994a654d0 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c4f7fe640 1 -- 172.21.15.94:0/3205277260 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6c54042030 con 0x7f6c60103920 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c5ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c5dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 0x562994e30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c5ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37082/0 (socket says 172.21.15.94:37082) 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3293718883 learned_addr learned my addr 172.21.15.94:0/3293718883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.785+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a654d0 0x562994e2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3293718883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a654d0 msgr2=0x562994e2ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a654d0 0x562994e2ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3293718883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 msgr2=0x562994e30c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 0x562994e30c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3293718883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562994d1fa40 con 0x562994a8e390 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 0x562994e30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009fd0 tx=0x7f6c580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2fbb0a68832022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c4f7fe640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 0x7f6c480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c4cff9640 1 -- 172.21.15.94:0/3293718883 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004d20 con 0x562994a8e390 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 0x7f6c480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c4f7fe640 1 -- 172.21.15.94:0/3205277260 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6c540ff1e0 con 0x7f6c60103920 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c4cff9640 1 -- 172.21.15.94:0/3293718883 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c58004050 con 0x562994a8e390 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.786+0000 7f6c4cff9640 1 -- 172.21.15.94:0/3293718883 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58005280 con 0x562994a8e390 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.787+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3293718883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 msgr2=0x562994e2fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.787+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 0x7f6c480b2000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562994e2f780 tx=0x7f6c50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.787+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009fd0 tx=0x7f6c580118d0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3293718883 shutdown_connections 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a6d6b0 0x562994e30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a654d0 0x562994e2ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3293718883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e2fa60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3293718883 >> 172.21.15.94:0/3293718883 conn(0x562994a824e0 msgr2=0x562994e2b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3293718883 shutdown_connections 2024-04-25T13:49:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3293718883 wait complete. 2024-04-25T13:49:29.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 -- start start 2024-04-25T13:49:29.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.788+0000 7f6c6ba22740 1 Processor -- start 2024-04-25T13:49:29.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e40e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562994e31780 con 0x562994a654d0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562994d1bd10 con 0x562994a8e390 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562994e31900 con 0x562994a6d6b0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c6ba22740 10 client.0 _ll_register_callbacks cb 0x562994d48210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39260/0 (socket says 172.21.15.94:39260) 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5e59c640 1 -- 172.21.15.94:0/3215687990 learned_addr learned my addr 172.21.15.94:0/3215687990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e40e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3215687990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 msgr2=0x562994e39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3215687990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 msgr2=0x562994e399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 -- 172.21.15.94:0/3215687990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562994b707d0 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e40e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c5801b040 tx=0x7f6c580044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5289d555e404081f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.789+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58028020 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6c58004c50 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c58004050 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c58004e50 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 10 client.27812 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562994aa63f0 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c58004050 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c5800f4e0 con 0x562994a8e390 2024-04-25T13:49:29.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 10 client.27812 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 10 client.27812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 20 client.27812 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a4cf60 0x562994e470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 4 client.27812 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562994da2990 con 0x562994a4cf60 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.790+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a4cf60 0x562994e470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.791+0000 7f6c45ffb640 10 client.27812 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.791+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a4cf60 0x562994e470b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb0f54578ab2e53 server_cookie=3732df68fd080b45 in_seq=0 out_seq=0 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c45ffb640 1 -- 172.21.15.94:0/3215687990 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562994da2990 con 0x562994a4cf60 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c45ffb640 10 client.27812 handle_command_reply: tid=0 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 client.27812 shutdown 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 msgr2=0x562994e40e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e40e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c5801b040 tx=0x7f6c580044b0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 shutdown_connections 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a4cf60 0x562994e470b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c45ffb640 0 client.27812 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a8e390 0x562994e40e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3215687990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 >> 172.21.15.94:0/3215687990 conn(0x562994a824e0 msgr2=0x562994e2e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 shutdown_connections 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3215687990 wait complete. 2024-04-25T13:49:29.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.793+0000 7f6c6ba22740 20 client.27812 trim_cache size 0 max 16384 2024-04-25T13:49:29.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 907849 12.86 MB/sec execute 252 sec latency 39.155 ms 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 Processor -- start 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 -- start start 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a4cf60 0x562994e3a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 0x562994e40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601aa430 con 0x562994a654d0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601abf70 con 0x562994a6d6b0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601aab00 con 0x562994a4cf60 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39276/0 (socket says 172.21.15.94:39276) 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3187121008 learned_addr learned my addr 172.21.15.94:0/3187121008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 0x562994e40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a4cf60 0x562994e3a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3187121008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a4cf60 msgr2=0x562994e3a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a4cf60 0x562994e3a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3187121008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 msgr2=0x562994e40da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.813+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 0x562994e40da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c5dd9b640 1 -- 172.21.15.94:0/3187121008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562994a660f0 con 0x562994a654d0 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 0x562994e40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c4940 tx=0x7f6c5403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989c7d91318028a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c47fff640 1 -- 172.21.15.94:0/3187121008 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54049040 con 0x562994a654d0 2024-04-25T13:49:29.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c47fff640 1 -- 172.21.15.94:0/3187121008 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c540c71e0 con 0x562994a654d0 2024-04-25T13:49:29.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.814+0000 7f6c47fff640 1 -- 172.21.15.94:0/3187121008 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540c1e60 con 0x562994a654d0 2024-04-25T13:49:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3187121008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 msgr2=0x562994e34db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c4940 tx=0x7f6c5403b460 comp rx=0 tx=0).stop 2024-04-25T13:49:29.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3187121008 shutdown_connections 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a6d6b0 0x562994e40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a654d0 0x562994e34db0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/3187121008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a4cf60 0x562994e3a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3187121008 >> 172.21.15.94:0/3187121008 conn(0x562994a824e0 msgr2=0x562994e399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3187121008 shutdown_connections 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.815+0000 7f6c6ba22740 1 -- 172.21.15.94:0/3187121008 wait complete. 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 -- start start 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 Processor -- start 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 0x562994e40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601ab8a0 con 0x562994a4cf60 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601ab1d0 con 0x562994a654d0 2024-04-25T13:49:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c601a8410 con 0x562994a6d6b0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c6ba22740 10 client.0 _ll_register_callbacks cb 0x7f6c54102b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39278/0 (socket says 172.21.15.94:39278) 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 -- 172.21.15.94:0/1005688464 learned_addr learned my addr 172.21.15.94:0/1005688464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 0x562994e40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5ed9d640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e3bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 -- 172.21.15.94:0/1005688464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 msgr2=0x562994e3bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e3bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 -- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 msgr2=0x562994e40b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 0x562994e40b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5e59c640 1 -- 172.21.15.94:0/1005688464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562994d554b0 con 0x562994a4cf60 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.816+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 0x562994e40b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c5e59c640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6c50013010 tx=0x7f6c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77ec6cb4a631745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500035f0 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6c50003790 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c540c71e0 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c50003990 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 10 client.18348 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562994eb0630 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c540c71e0 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c5001fca0 con 0x562994a4cf60 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 10 client.18348 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 20 client.18348 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a8e390 0x562994eb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 4 client.18348 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:49:29.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.817+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562994d9c800 con 0x562994a8e390 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a8e390 0x562994eb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c45ffb640 10 client.18348 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c5dd9b640 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a8e390 0x562994eb3050 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37920f17cdf2613d server_cookie=d6b5f5a82d51a8f0 in_seq=0 out_seq=0 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c45ffb640 1 -- 172.21.15.94:0/1005688464 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562994d9c800 con 0x562994a8e390 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c45ffb640 10 client.18348 handle_command_reply: tid=0 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 client.18348 shutdown 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 msgr2=0x562994e3b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6c50013010 tx=0x7f6c500046c0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 shutdown_connections 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562994a8e390 0x562994eb3050 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562994a6d6b0 0x562994e3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c45ffb640 0 client.18348 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562994a654d0 0x562994e40b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 --2- 172.21.15.94:0/1005688464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562994a4cf60 0x562994e3b6d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 >> 172.21.15.94:0/1005688464 conn(0x562994a824e0 msgr2=0x562994e2ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 shutdown_connections 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 1 -- 172.21.15.94:0/1005688464 wait complete. 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.818+0000 7f6c6ba22740 20 client.18348 trim_cache size 0 max 16384 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "b3c04c66-994e-4a87-9fac-ad06b8bec4a4": { 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:49:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b3c04c66-994e-4a87-9fac-ad06b8bec4a4", 2024-04-25T13:49:29.826 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:49:29.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:49:29.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:49:29.826 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 msgr2=0x7f6c480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 0x7f6c480b2000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562994e2f780 tx=0x7f6c50005f90 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 msgr2=0x7f6c60199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c54006010 tx=0x7f6c540042f0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 shutdown_connections 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c480afaf0 0x7f6c480b2000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c60104460 0x7f6c601a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c60103920 0x7f6c60199650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 --2- 172.21.15.94:0/3205277260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c60102f40 0x7f6c60199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 >> 172.21.15.94:0/3205277260 conn(0x7f6c600fd340 msgr2=0x7f6c60104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 shutdown_connections 2024-04-25T13:49:29.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:29.819+0000 7f6c64a97640 1 -- 172.21.15.94:0/3205277260 wait complete. 2024-04-25T13:49:29.832 INFO:tasks.cephfs.filesystem:scrub status for tag:b3c04c66-994e-4a87-9fac-ad06b8bec4a4 - {'path': '/', 'tag': 'b3c04c66-994e-4a87-9fac-ad06b8bec4a4', 'options': 'recursive,force'} 2024-04-25T13:49:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:30 smithi094 ceph-mon[29195]: pgmap v578: 97 pgs: 97 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-04-25T13:49:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:30 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:49:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:30 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:49:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:30 smithi137 ceph-mon[32568]: pgmap v578: 97 pgs: 97 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-04-25T13:49:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:30 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:49:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:30 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:49:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:30 smithi138 ceph-mon[32977]: pgmap v578: 97 pgs: 97 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-04-25T13:49:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:30 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:49:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:30 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:49:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:49:30.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 909809 12.86 MB/sec execute 253 sec latency 15.104 ms 2024-04-25T13:49:31.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:49:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:49:31.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:49:31.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 912371 12.88 MB/sec execute 254 sec latency 39.810 ms 2024-04-25T13:49:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:32 smithi137 ceph-mon[32568]: pgmap v579: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-04-25T13:49:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:32 smithi094 ceph-mon[29195]: pgmap v579: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-04-25T13:49:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:32 smithi138 ceph-mon[32977]: pgmap v579: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-04-25T13:49:32.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 913990 12.85 MB/sec execute 255 sec latency 25.095 ms 2024-04-25T13:49:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 -- 172.21.15.94:0/3287141075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 msgr2=0x7f313410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 --2- 172.21.15.94:0/3287141075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f313410bea0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f312c009ca0 tx=0x7f312c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 -- 172.21.15.94:0/3287141075 shutdown_connections 2024-04-25T13:49:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 --2- 172.21.15.94:0/3287141075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f313410bea0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 --2- 172.21.15.94:0/3287141075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134104d80 0x7f3134105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 --2- 172.21.15.94:0/3287141075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134103b40 0x7f3134104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 -- 172.21.15.94:0/3287141075 >> 172.21.15.94:0/3287141075 conn(0x7f31340fed10 msgr2=0x7f31341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 -- 172.21.15.94:0/3287141075 shutdown_connections 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.875+0000 7f31394b6640 1 -- 172.21.15.94:0/3287141075 wait complete. 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 Processor -- start 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 -- start start 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 0x7f313419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134104d80 0x7f313419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f313410f6a0 con 0x7f3134109a80 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f313410f520 con 0x7f3134103b40 2024-04-25T13:49:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31394b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f313410f820 con 0x7f3134104d80 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39286/0 (socket says 172.21.15.94:39286) 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f31337fe640 1 -- 172.21.15.94:0/3215702946 learned_addr learned my addr 172.21.15.94:0/3215702946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f3132ffd640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 0x7f313419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.876+0000 7f312bfff640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134104d80 0x7f313419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 -- 172.21.15.94:0/3215702946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134104d80 msgr2=0x7f313419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134104d80 0x7f313419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 -- 172.21.15.94:0/3215702946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 msgr2=0x7f313419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 0x7f313419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 -- 172.21.15.94:0/3215702946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341a3f50 con 0x7f3134109a80 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f3132ffd640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 0x7f313419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31337fe640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f312c009430 tx=0x7f312c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f3130ff9640 1 -- 172.21.15.94:0/3215702946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c003bf0 con 0x7f3134109a80 2024-04-25T13:49:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341a41e0 con 0x7f3134109a80 2024-04-25T13:49:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f3130ff9640 1 -- 172.21.15.94:0/3215702946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312c003d90 con 0x7f3134109a80 2024-04-25T13:49:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f3130ff9640 1 -- 172.21.15.94:0/3215702946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c02fc80 con 0x7f3134109a80 2024-04-25T13:49:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.877+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341a4780 con 0x7f3134109a80 2024-04-25T13:49:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3130ff9640 1 -- 172.21.15.94:0/3215702946 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f312c004710 con 0x7f3134109a80 2024-04-25T13:49:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 Processor -- start 2024-04-25T13:49:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 -- start start 2024-04-25T13:49:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3130ff9640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 0x7f31200b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e552f0 0x55d59921eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 0x55d599225df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d599184460 con 0x55d598e5d4d0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d59913e590 con 0x55d598e7e1b0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f3140369740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d599180420 con 0x55d598e552f0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.880+0000 7f312bfff640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 0x7f31200b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f31337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 0x55d599225df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37152/0 (socket says 172.21.15.94:37152) 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 -- 172.21.15.94:0/2594038061 learned_addr learned my addr 172.21.15.94:0/2594038061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f31337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 0x55d599225df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39302/0 (socket says 172.21.15.94:39302) 2024-04-25T13:49:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f312bfff640 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e552f0 0x55d59921eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3130ff9640 1 -- 172.21.15.94:0/3215702946 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f312c100300 con 0x7f3134109a80 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 -- 172.21.15.94:0/2594038061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e552f0 msgr2=0x55d59921eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e552f0 0x55d59921eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 -- 172.21.15.94:0/2594038061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 msgr2=0x55d599225df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 0x55d599225df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 -- 172.21.15.94:0/2594038061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59914b0d0 con 0x55d598e7e1b0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f312bfff640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 0x7f31200b1ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f313419d4b0 tx=0x7f311c009a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f3132ffd640 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31240104d0 tx=0x7f3124010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc76f56c9a27398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f31297fa640 1 -- 172.21.15.94:0/2594038061 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124011820 con 0x55d598e7e1b0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f31297fa640 1 -- 172.21.15.94:0/2594038061 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312401c480 con 0x55d598e7e1b0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.881+0000 7f31297fa640 1 -- 172.21.15.94:0/2594038061 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400e670 con 0x55d598e7e1b0 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 -- 172.21.15.94:0/2594038061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 msgr2=0x55d59921e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31240104d0 tx=0x7f3124010990 comp rx=0 tx=0).stop 2024-04-25T13:49:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 -- 172.21.15.94:0/2594038061 shutdown_connections 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e5d4d0 0x55d599225df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e552f0 0x55d59921eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 --2- 172.21.15.94:0/2594038061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59921e970 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 -- 172.21.15.94:0/2594038061 >> 172.21.15.94:0/2594038061 conn(0x55d598e72300 msgr2=0x55d59921a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 -- 172.21.15.94:0/2594038061 shutdown_connections 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.882+0000 7f3140369740 1 -- 172.21.15.94:0/2594038061 wait complete. 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 -- start start 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 Processor -- start 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d599228c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59922fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d599226840 con 0x55d598e552f0 2024-04-25T13:49:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d59910bb60 con 0x55d598e7e1b0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5992269c0 con 0x55d598e5d4d0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3140369740 10 client.0 _ll_register_callbacks cb 0x55d5991302b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39310/0 (socket says 172.21.15.94:39310) 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 -- 172.21.15.94:0/3390548131 learned_addr learned my addr 172.21.15.94:0/3390548131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f31337fe640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59922fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f312bfff640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d599228c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 -- 172.21.15.94:0/3390548131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 msgr2=0x55d599228c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d599228c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 -- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 msgr2=0x55d59922fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59922fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.883+0000 7f3132ffd640 1 -- 172.21.15.94:0/3390548131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d598e96210 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3132ffd640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f31240058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced16bcdf2b6f82e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312402a060 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f312401c980 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312401c480 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3124004040 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 10 client.18360 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d598f62620 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312401c480 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31240035f0 con 0x55d598e552f0 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 10 client.18360 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 20 client.18360 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e3cd80 0x55d5992360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 4 client.18360 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.884+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d599192070 con 0x55d598e3cd80 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.885+0000 7f312bfff640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e3cd80 0x55d5992360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.885+0000 7f311a7fc640 10 client.18360 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.885+0000 7f312bfff640 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e3cd80 0x55d5992360d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82d985d12297296 server_cookie=d290e46402c49962 in_seq=0 out_seq=0 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f311a7fc640 1 -- 172.21.15.94:0/3390548131 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d599192070 con 0x55d598e3cd80 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f311a7fc640 10 client.18360 handle_command_reply: tid=0 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 client.18360 shutdown 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 msgr2=0x55d5992286d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f31240058e0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 shutdown_connections 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e3cd80 0x55d5992360d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59922fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d599228c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f311a7fc640 0 client.18360 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 --2- 172.21.15.94:0/3390548131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d5992286d0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 >> 172.21.15.94:0/3390548131 conn(0x55d598e72300 msgr2=0x55d599219250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 shutdown_connections 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 1 -- 172.21.15.94:0/3390548131 wait complete. 2024-04-25T13:49:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.886+0000 7f3140369740 20 client.18360 trim_cache size 0 max 16384 2024-04-25T13:49:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 Processor -- start 2024-04-25T13:49:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 -- start start 2024-04-25T13:49:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59921a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 0x55d5992282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31341ad740 con 0x55d598e7e1b0 2024-04-25T13:49:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31341af280 con 0x55d598e552f0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.903+0000 7f3140369740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31341ade10 con 0x55d598e5d4d0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 0x55d5992282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f312bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59921a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37168/0 (socket says 172.21.15.94:37168) 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f312bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59921a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43552/0 (socket says 172.21.15.94:43552) 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f3132ffd640 1 -- 172.21.15.94:0/1305547424 learned_addr learned my addr 172.21.15.94:0/1305547424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 -- 172.21.15.94:0/1305547424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 msgr2=0x55d59921a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59921a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 -- 172.21.15.94:0/1305547424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 msgr2=0x55d599219ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 -- 172.21.15.94:0/1305547424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d599226b00 con 0x55d598e7e1b0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f3132ffd640 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f31337fe640 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 0x55d5992282a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f312c0fddb0 tx=0x7f312c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55c17230c7966ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.904+0000 7f3128ff9640 1 -- 172.21.15.94:0/1305547424 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c0c8250 con 0x55d598e7e1b0 2024-04-25T13:49:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.905+0000 7f3128ff9640 1 -- 172.21.15.94:0/1305547424 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312c036070 con 0x55d598e7e1b0 2024-04-25T13:49:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.905+0000 7f3128ff9640 1 -- 172.21.15.94:0/1305547424 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c02fe20 con 0x55d598e7e1b0 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.905+0000 7f3140369740 1 -- 172.21.15.94:0/1305547424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 msgr2=0x55d5992282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.905+0000 7f3140369740 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 0x55d5992282a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f312c0fddb0 tx=0x7f312c003f40 comp rx=0 tx=0).stop 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 -- 172.21.15.94:0/1305547424 shutdown_connections 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e7e1b0 0x55d5992282a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59921a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- 172.21.15.94:0/1305547424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e552f0 0x55d599219ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 -- 172.21.15.94:0/1305547424 >> 172.21.15.94:0/1305547424 conn(0x55d598e72300 msgr2=0x55d599222b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 -- 172.21.15.94:0/1305547424 shutdown_connections 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 -- 172.21.15.94:0/1305547424 wait complete. 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 -- start start 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 Processor -- start 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59922c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.906+0000 7f3140369740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59929c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3140369740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31341aebb0 con 0x55d598e552f0 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3140369740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31341ae4e0 con 0x55d598e7e1b0 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3140369740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31341ab720 con 0x55d598e5d4d0 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3140369740 10 client.0 _ll_register_callbacks cb 0x7f312c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39324/0 (socket says 172.21.15.94:39324) 2024-04-25T13:49:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 -- 172.21.15.94:0/1472314130 learned_addr learned my addr 172.21.15.94:0/1472314130 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f312bfff640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59922c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f31337fe640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59929c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 -- 172.21.15.94:0/1472314130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 msgr2=0x55d59922c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59922c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 -- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 msgr2=0x55d59929c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59929c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 -- 172.21.15.94:0/1472314130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59922cb80 con 0x55d598e552f0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f3132ffd640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f31240039f0 tx=0x7f3124010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db619794daca8197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f311a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.907+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312402a040 con 0x55d598e552f0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3124003d60 con 0x55d598e552f0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312c036070 con 0x55d598e552f0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3124040050 con 0x55d598e552f0 2024-04-25T13:49:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f3140369740 10 client.18366 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d59929d990 con 0x55d598e552f0 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f312c036070 con 0x55d598e552f0 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3124037070 con 0x55d598e552f0 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f3140369740 10 client.18366 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.908+0000 7f3140369740 10 client.18366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f3140369740 20 client.18366 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e121c0 0x55d5992a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f3140369740 4 client.18366 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d59918bee0 con 0x55d598e121c0 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f312bfff640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e121c0 0x55d5992a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f311a7fc640 10 client.18366 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:32.909+0000 7f312bfff640 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e121c0 0x55d5992a03b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb13a7d9a2d3ad6 server_cookie=dc71fbc88ccdd411 in_seq=0 out_seq=0 2024-04-25T13:49:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f311a7fc640 1 -- 172.21.15.94:0/1472314130 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d59918bee0 con 0x55d598e121c0 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f311a7fc640 10 client.18366 handle_command_reply: tid=0 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 client.18366 shutdown 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 msgr2=0x55d59922c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f31240039f0 tx=0x7f3124010e90 comp rx=0 tx=0).stop 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 shutdown_connections 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d598e121c0 0x55d5992a03b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f311a7fc640 0 client.18366 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d598e7e1b0 0x55d59929c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d598e5d4d0 0x55d59922c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 --2- 172.21.15.94:0/1472314130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d598e552f0 0x55d59922c000 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 >> 172.21.15.94:0/1472314130 conn(0x55d598e72300 msgr2=0x55d599219ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 shutdown_connections 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 1 -- 172.21.15.94:0/1472314130 wait complete. 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.081+0000 7f3140369740 20 client.18366 trim_cache size 0 max 16384 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:49:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:49:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 msgr2=0x7f31200b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 0x7f31200b1ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f313419d4b0 tx=0x7f311c009a50 comp rx=0 tx=0).stop 2024-04-25T13:49:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 msgr2=0x7f31341a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f312c009430 tx=0x7f312c0317c0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 shutdown_connections 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31200afae0 0x7f31200b1ff0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3134109a80 0x7f31341a38e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3134104d80 0x7f313419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 --2- 172.21.15.94:0/3215702946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3134103b40 0x7f313419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 >> 172.21.15.94:0/3215702946 conn(0x7f31340fed10 msgr2=0x7f31340ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 shutdown_connections 2024-04-25T13:49:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:33.082+0000 7f31394b6640 1 -- 172.21.15.94:0/3215702946 wait complete. 2024-04-25T13:49:33.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 916230 12.86 MB/sec execute 256 sec latency 99.210 ms 2024-04-25T13:49:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:34 smithi137 ceph-mon[32568]: pgmap v580: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 343 op/s 2024-04-25T13:49:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:34 smithi094 ceph-mon[29195]: pgmap v580: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 343 op/s 2024-04-25T13:49:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:34 smithi138 ceph-mon[32977]: pgmap v580: 97 pgs: 97 active+clean; 45 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 343 op/s 2024-04-25T13:49:34.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 917621 12.89 MB/sec execute 257 sec latency 51.172 ms 2024-04-25T13:49:35.803 INFO:tasks.workunit.client.0.smithi094.stdout: 1 920571 12.91 MB/sec execute 258 sec latency 8.274 ms 2024-04-25T13:49:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:36 smithi137 ceph-mon[32568]: pgmap v581: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-04-25T13:49:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:36 smithi094 ceph-mon[29195]: pgmap v581: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-04-25T13:49:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:36 smithi138 ceph-mon[32977]: pgmap v581: 97 pgs: 97 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-04-25T13:49:36.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 922697 12.89 MB/sec execute 259 sec latency 68.821 ms 2024-04-25T13:49:37.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 924602 12.89 MB/sec execute 260 sec latency 19.465 ms 2024-04-25T13:49:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:38 smithi137 ceph-mon[32568]: pgmap v582: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-04-25T13:49:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:38 smithi094 ceph-mon[29195]: pgmap v582: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-04-25T13:49:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:38 smithi138 ceph-mon[32977]: pgmap v582: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-04-25T13:49:38.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 927586 12.91 MB/sec execute 261 sec latency 9.768 ms 2024-04-25T13:49:39.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 930260 12.91 MB/sec execute 262 sec latency 12.381 ms 2024-04-25T13:49:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:40 smithi137 ceph-mon[32568]: pgmap v583: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 364 op/s 2024-04-25T13:49:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:40 smithi094 ceph-mon[29195]: pgmap v583: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 364 op/s 2024-04-25T13:49:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:40 smithi138 ceph-mon[32977]: pgmap v583: 97 pgs: 97 active+clean; 41 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 364 op/s 2024-04-25T13:49:40.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 932308 12.91 MB/sec execute 263 sec latency 18.248 ms 2024-04-25T13:49:41.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 934971 12.91 MB/sec execute 264 sec latency 16.729 ms 2024-04-25T13:49:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:42 smithi137 ceph-mon[32568]: pgmap v584: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-25T13:49:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:42 smithi094 ceph-mon[29195]: pgmap v584: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-25T13:49:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:42 smithi138 ceph-mon[32977]: pgmap v584: 97 pgs: 97 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 399 op/s 2024-04-25T13:49:42.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 937475 12.91 MB/sec execute 265 sec latency 50.772 ms 2024-04-25T13:49:43.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 939838 12.91 MB/sec execute 266 sec latency 12.417 ms 2024-04-25T13:49:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:44 smithi137 ceph-mon[32568]: pgmap v585: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:49:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:44 smithi094 ceph-mon[29195]: pgmap v585: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:49:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:44 smithi138 ceph-mon[32977]: pgmap v585: 97 pgs: 97 active+clean; 30 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-04-25T13:49:44.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 942430 12.91 MB/sec execute 267 sec latency 11.985 ms 2024-04-25T13:49:45.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 945479 12.93 MB/sec execute 268 sec latency 9.869 ms 2024-04-25T13:49:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 -- 172.21.15.94:0/3514752612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c106c60 msgr2=0x7f3d7c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 --2- 172.21.15.94:0/3514752612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c106c60 0x7f3d7c109140 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009900 tx=0x7f3d7002a410 comp rx=0 tx=0).stop 2024-04-25T13:49:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 -- 172.21.15.94:0/3514752612 shutdown_connections 2024-04-25T13:49:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 --2- 172.21.15.94:0/3514752612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c10bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 --2- 172.21.15.94:0/3514752612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c106c60 0x7f3d7c109140 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 --2- 172.21.15.94:0/3514752612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c101ed0 0x7f3d7c1022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 -- 172.21.15.94:0/3514752612 >> 172.21.15.94:0/3514752612 conn(0x7f3d7c0fd380 msgr2=0x7f3d7c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 -- 172.21.15.94:0/3514752612 shutdown_connections 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 -- 172.21.15.94:0/3514752612 wait complete. 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.208+0000 7f3d825c5640 1 Processor -- start 2024-04-25T13:49:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 -- start start 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c101ed0 0x7f3d7c1043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 0x7f3d7c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10f660 con 0x7f3d7c1097d0 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10f4e0 con 0x7f3d7c106c60 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d825c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10f7e0 con 0x7f3d7c101ed0 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c101ed0 0x7f3d7c1043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52350/0 (socket says 172.21.15.94:52350) 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 -- 172.21.15.94:0/1056593695 learned_addr learned my addr 172.21.15.94:0/1056593695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 0x7f3d7c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 -- 172.21.15.94:0/1056593695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c101ed0 msgr2=0x7f3d7c1043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c101ed0 0x7f3d7c1043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 -- 172.21.15.94:0/1056593695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 msgr2=0x7f3d7c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 0x7f3d7c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d80b62640 1 -- 172.21.15.94:0/1056593695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c1036b0 con 0x7f3d7c1097d0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.209+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 0x7f3d7c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d80b62640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400eaa0 tx=0x7f3d7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d797fa640 1 -- 172.21.15.94:0/1056593695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400cdb0 con 0x7f3d7c1097d0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d797fa640 1 -- 172.21.15.94:0/1056593695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74005020 con 0x7f3d7c1097d0 2024-04-25T13:49:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c1a42c0 con 0x7f3d7c1097d0 2024-04-25T13:49:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d797fa640 1 -- 172.21.15.94:0/1056593695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740116d0 con 0x7f3d7c1097d0 2024-04-25T13:49:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.210+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c1a4780 con 0x7f3d7c1097d0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.212+0000 7f3d8947a740 1 Processor -- start 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d797fa640 1 -- 172.21.15.94:0/1056593695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3d74011870 con 0x7f3d7c1097d0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 -- start start 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df272c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df272cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560df2692460 con 0x560df23632f0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df264c590 con 0x560df236b4d0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.213+0000 7f3d8947a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560df268e420 con 0x560df238c1b0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37056/0 (socket says 172.21.15.94:37056) 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/17703308 learned_addr learned my addr 172.21.15.94:0/17703308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df272cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df272c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d797fa640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 0x7f3d640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/17703308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 msgr2=0x560df272c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df272c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/17703308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 msgr2=0x560df272cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df272cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/17703308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df26590d0 con 0x560df236b4d0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df272cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d80b62640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 0x7f3d640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d797fa640 1 -- 172.21.15.94:0/1056593695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3d740deda0 con 0x7f3d7c1097d0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.214+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x560df272db40 tx=0x7f3d70032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90599b7f94c7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d627fc640 1 -- 172.21.15.94:0/17703308 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70035040 con 0x560df236b4d0 2024-04-25T13:49:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d627fc640 1 -- 172.21.15.94:0/17703308 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d700035f0 con 0x560df236b4d0 2024-04-25T13:49:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d627fc640 1 -- 172.21.15.94:0/17703308 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700044e0 con 0x560df236b4d0 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 -- 172.21.15.94:0/17703308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 msgr2=0x560df2733df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d80b62640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 0x7f3d640b1ff0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d740a6f90 tx=0x7f3d740142b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x560df272db40 tx=0x7f3d70032ed0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 -- 172.21.15.94:0/17703308 shutdown_connections 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2733df0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df272cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 --2- 172.21.15.94:0/17703308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df272c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 -- 172.21.15.94:0/17703308 >> 172.21.15.94:0/17703308 conn(0x560df2380300 msgr2=0x560df2728300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.215+0000 7f3d8947a740 1 -- 172.21.15.94:0/17703308 shutdown_connections 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 -- 172.21.15.94:0/17703308 wait complete. 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 -- start start 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 Processor -- start 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df23632f0 0x560df2736670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560df27347e0 con 0x560df236b4d0 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df2619b60 con 0x560df23632f0 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560df2734960 con 0x560df238c1b0 2024-04-25T13:49:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d8947a740 10 client.0 _ll_register_callbacks cb 0x560df263e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df23632f0 0x560df2736670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.216+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34960/0 (socket says 172.21.15.94:34960) 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52380/0 (socket says 172.21.15.94:52380) 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/80477881 learned_addr learned my addr 172.21.15.94:0/80477881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 -- 172.21.15.94:0/80477881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 msgr2=0x560df273db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 -- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df23632f0 msgr2=0x560df2736670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df23632f0 0x560df2736670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 -- 172.21.15.94:0/80477881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df23a4210 con 0x560df236b4d0 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x560df272dd50 tx=0x7f3d6c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77caf11efa0e5315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004030 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3d6c0041d0 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d700035f0 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.217+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c0043d0 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d8947a740 10 client.18378 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df2470620 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d700035f0 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d6c003ab0 con 0x560df236b4d0 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d8947a740 10 client.18378 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d8947a740 10 client.18378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.218+0000 7f3d8947a740 20 client.18378 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df234ad80 0x560df2744360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d8947a740 4 client.18378 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df26a0070 con 0x560df234ad80 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df234ad80 0x560df2744360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d5b7fe640 10 client.18378 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.219+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df234ad80 0x560df2744360 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d7f2b1d99f9664 server_cookie=6844d4aedcbc7923 in_seq=0 out_seq=0 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/80477881 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560df26a0070 con 0x560df234ad80 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d5b7fe640 10 client.18378 handle_command_reply: tid=0 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 client.18378 shutdown 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 msgr2=0x560df2736bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x560df272dd50 tx=0x7f3d6c012b60 comp rx=0 tx=0).stop 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 shutdown_connections 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df234ad80 0x560df2744360 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df238c1b0 0x560df273db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df236b4d0 0x560df2736bb0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 --2- 172.21.15.94:0/80477881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df23632f0 0x560df2736670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d5b7fe640 0 client.18378 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 >> 172.21.15.94:0/80477881 conn(0x560df2380300 msgr2=0x560df272b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 shutdown_connections 2024-04-25T13:49:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.220+0000 7f3d8947a740 1 -- 172.21.15.94:0/80477881 wait complete. 2024-04-25T13:49:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.221+0000 7f3d8947a740 20 client.18378 trim_cache size 0 max 16384 2024-04-25T13:49:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 Processor -- start 2024-04-25T13:49:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 -- start start 2024-04-25T13:49:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 0x560df2727cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2728200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ad790 con 0x560df234ad80 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1af2d0 con 0x560df236b4d0 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.236+0000 7f3d8947a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ade60 con 0x560df23632f0 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 0x560df2727cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 0x560df2727cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52396/0 (socket says 172.21.15.94:52396) 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34966/0 (socket says 172.21.15.94:34966) 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7b7fe640 1 -- 172.21.15.94:0/1436551910 learned_addr learned my addr 172.21.15.94:0/1436551910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2728200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1436551910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 msgr2=0x560df2728200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2728200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1436551910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 msgr2=0x560df2727cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 0x560df2727cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1436551910 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df273df70 con 0x560df23632f0 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.237+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001eb0 tx=0x7f3d6c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbd3949a104e69b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d61ffb640 1 -- 172.21.15.94:0/1436551910 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c003fd0 con 0x560df23632f0 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d61ffb640 1 -- 172.21.15.94:0/1436551910 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c004170 con 0x560df23632f0 2024-04-25T13:49:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d61ffb640 1 -- 172.21.15.94:0/1436551910 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00fe80 con 0x560df23632f0 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 -- 172.21.15.94:0/1436551910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 msgr2=0x560df273da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c001eb0 tx=0x7f3d6c005970 comp rx=0 tx=0).stop 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 -- 172.21.15.94:0/1436551910 shutdown_connections 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df236b4d0 0x560df2728200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df23632f0 0x560df273da30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1436551910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df234ad80 0x560df2727cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.238+0000 7f3d8947a740 1 -- 172.21.15.94:0/1436551910 >> 172.21.15.94:0/1436551910 conn(0x560df2380300 msgr2=0x560df2730b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- 172.21.15.94:0/1436551910 shutdown_connections 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- 172.21.15.94:0/1436551910 wait complete. 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- start start 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 Processor -- start 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df234ad80 0x560df273a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 0x560df27aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1aec00 con 0x560df23632f0 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ae530 con 0x560df234ad80 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c1ab770 con 0x560df236b4d0 2024-04-25T13:49:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d8947a740 10 client.0 _ll_register_callbacks cb 0x7f3d6c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52408/0 (socket says 172.21.15.94:52408) 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1773905182 learned_addr learned my addr 172.21.15.94:0/1773905182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.239+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df234ad80 0x560df273a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 0x560df27aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1773905182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 msgr2=0x560df27aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 0x560df27aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df234ad80 msgr2=0x560df273a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df234ad80 0x560df273a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 -- 172.21.15.94:0/1773905182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df273b440 con 0x560df23632f0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 0x560df27aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d7bfff640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c014010 tx=0x7f3d6c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aededbbf16e102c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00fc00 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3d6c00fda0 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d6c004170 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c040050 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d8947a740 10 client.18384 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.240+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560df27abd00 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c004170 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d6c01c910 con 0x560df23632f0 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 10 client.18384 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 20 client.18384 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df238c1b0 0x560df27ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 4 client.18384 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:49:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560df2699ee0 con 0x560df238c1b0 2024-04-25T13:49:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.241+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df238c1b0 0x560df27ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.242+0000 7f3d5b7fe640 10 client.18384 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.242+0000 7f3d7b7fe640 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df238c1b0 0x560df27ae6c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b0dac3f91234ab server_cookie=da8f74bc9637e650 in_seq=0 out_seq=0 2024-04-25T13:49:46.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:46 smithi137 ceph-mon[32568]: pgmap v586: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d5b7fe640 1 -- 172.21.15.94:0/1773905182 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560df2699ee0 con 0x560df238c1b0 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d5b7fe640 10 client.18384 handle_command_reply: tid=0 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 client.18384 shutdown 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 msgr2=0x560df273add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c014010 tx=0x7f3d6c009ea0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 shutdown_connections 2024-04-25T13:49:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df238c1b0 0x560df27ae6c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df236b4d0 0x560df27aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df23632f0 0x560df273add0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.419+0000 7f3d8947a740 1 --2- 172.21.15.94:0/1773905182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df234ad80 0x560df273a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.420+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 >> 172.21.15.94:0/1773905182 conn(0x560df2380300 msgr2=0x560df272b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.420+0000 7f3d5b7fe640 0 client.18384 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.420+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 shutdown_connections 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.420+0000 7f3d8947a740 1 -- 172.21.15.94:0/1773905182 wait complete. 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.420+0000 7f3d8947a740 20 client.18384 trim_cache size 0 max 16384 2024-04-25T13:49:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 msgr2=0x7f3d640b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 0x7f3d640b1ff0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d740a6f90 tx=0x7f3d740142b0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 msgr2=0x7f3d7c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400eaa0 tx=0x7f3d7400ef60 comp rx=0 tx=0).stop 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 shutdown_connections 2024-04-25T13:49:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d640afae0 0x7f3d640b1ff0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d7c1097d0 0x7f3d7c102f80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d7c106c60 0x7f3d7c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 --2- 172.21.15.94:0/1056593695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7c101ed0 0x7f3d7c1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 >> 172.21.15.94:0/1056593695 conn(0x7f3d7c0fd380 msgr2=0x7f3d7c1078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 shutdown_connections 2024-04-25T13:49:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:46.421+0000 7f3d825c5640 1 -- 172.21.15.94:0/1056593695 wait complete. 2024-04-25T13:49:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:46 smithi094 ceph-mon[29195]: pgmap v586: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-04-25T13:49:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:46 smithi138 ceph-mon[32977]: pgmap v586: 97 pgs: 97 active+clean; 34 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-04-25T13:49:46.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 947560 12.91 MB/sec execute 269 sec latency 63.873 ms 2024-04-25T13:49:47.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 949433 12.91 MB/sec execute 270 sec latency 63.658 ms 2024-04-25T13:49:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:48 smithi137 ceph-mon[32568]: pgmap v587: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-25T13:49:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:48 smithi094 ceph-mon[29195]: pgmap v587: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-25T13:49:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:48 smithi138 ceph-mon[32977]: pgmap v587: 97 pgs: 97 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-04-25T13:49:48.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 952510 12.93 MB/sec execute 271 sec latency 7.698 ms 2024-04-25T13:49:49.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 955404 12.93 MB/sec execute 272 sec latency 25.296 ms 2024-04-25T13:49:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:50 smithi137 ceph-mon[32568]: pgmap v588: 97 pgs: 97 active+clean; 32 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-04-25T13:49:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:50 smithi094 ceph-mon[29195]: pgmap v588: 97 pgs: 97 active+clean; 32 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-04-25T13:49:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:50 smithi138 ceph-mon[32977]: pgmap v588: 97 pgs: 97 active+clean; 32 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-04-25T13:49:50.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 957887 12.93 MB/sec execute 273 sec latency 23.955 ms 2024-04-25T13:49:51.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 960323 12.93 MB/sec execute 274 sec latency 12.838 ms 2024-04-25T13:49:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:52 smithi137 ceph-mon[32568]: pgmap v589: 97 pgs: 97 active+clean; 28 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-04-25T13:49:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:52 smithi094 ceph-mon[29195]: pgmap v589: 97 pgs: 97 active+clean; 28 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-04-25T13:49:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:52 smithi138 ceph-mon[32977]: pgmap v589: 97 pgs: 97 active+clean; 28 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-04-25T13:49:52.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 963143 12.95 MB/sec execute 275 sec latency 79.974 ms 2024-04-25T13:49:53.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 965806 12.95 MB/sec execute 276 sec latency 17.986 ms 2024-04-25T13:49:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:54 smithi137 ceph-mon[32568]: pgmap v590: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-04-25T13:49:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:54 smithi094 ceph-mon[29195]: pgmap v590: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-04-25T13:49:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:54 smithi138 ceph-mon[32977]: pgmap v590: 97 pgs: 97 active+clean; 24 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 3.3 MiB/s wr, 381 op/s 2024-04-25T13:49:54.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 968083 12.94 MB/sec execute 277 sec latency 15.802 ms 2024-04-25T13:49:55.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 970544 12.95 MB/sec execute 278 sec latency 57.242 ms 2024-04-25T13:49:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:56 smithi137 ceph-mon[32568]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:49:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:56 smithi094 ceph-mon[29195]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:49:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:56 smithi138 ceph-mon[32977]: pgmap v591: 97 pgs: 97 active+clean; 25 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:49:56.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 973548 12.96 MB/sec execute 279 sec latency 27.740 ms 2024-04-25T13:49:57.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 975163 12.94 MB/sec execute 280 sec latency 44.235 ms 2024-04-25T13:49:58.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:49:58 smithi137 ceph-mon[32568]: pgmap v592: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.2 MiB/s wr, 355 op/s 2024-04-25T13:49:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:49:58 smithi094 ceph-mon[29195]: pgmap v592: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.2 MiB/s wr, 355 op/s 2024-04-25T13:49:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:49:58 smithi138 ceph-mon[32977]: pgmap v592: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.2 MiB/s wr, 355 op/s 2024-04-25T13:49:58.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 977609 12.95 MB/sec execute 281 sec latency 30.520 ms 2024-04-25T13:49:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- 172.21.15.94:0/1663449922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56741050d0 msgr2=0x7f56741054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 --2- 172.21.15.94:0/1663449922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56741050d0 0x7f56741054e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- 172.21.15.94:0/1663449922 shutdown_connections 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 --2- 172.21.15.94:0/1663449922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674102280 0x7f567410bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 --2- 172.21.15.94:0/1663449922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674101780 0x7f5674101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 --2- 172.21.15.94:0/1663449922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56741050d0 0x7f56741054e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- 172.21.15.94:0/1663449922 >> 172.21.15.94:0/1663449922 conn(0x7f56740fd380 msgr2=0x7f56740ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- 172.21.15.94:0/1663449922 shutdown_connections 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- 172.21.15.94:0/1663449922 wait complete. 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 Processor -- start 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.521+0000 7f567a227640 1 -- start start 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674102280 0x7f567419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56741050d0 0x7f56741a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f567410f640 con 0x7f5674101780 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f567410f4c0 con 0x7f5674102280 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f567410f7c0 con 0x7f56741050d0 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37366/0 (socket says 172.21.15.94:37366) 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 -- 172.21.15.94:0/3151786573 learned_addr learned my addr 172.21.15.94:0/3151786573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f566bfff640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674102280 0x7f567419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f5673fff640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56741050d0 0x7f56741a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 -- 172.21.15.94:0/3151786573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56741050d0 msgr2=0x7f56741a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56741050d0 0x7f56741a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 -- 172.21.15.94:0/3151786573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674102280 msgr2=0x7f567419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674102280 0x7f567419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 -- 172.21.15.94:0/3151786573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a1d80 con 0x7f5674101780 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56737fe640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56717fa640 1 -- 172.21.15.94:0/3151786573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566403f030 con 0x7f5674101780 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56717fa640 1 -- 172.21.15.94:0/3151786573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5664003aa0 con 0x7f5674101780 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a2010 con 0x7f5674101780 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f56717fa640 1 -- 172.21.15.94:0/3151786573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664035070 con 0x7f5674101780 2024-04-25T13:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.522+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a2550 con 0x7f5674101780 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 Processor -- start 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 -- start start 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 0x55a9b124aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1251df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9b11b0460 con 0x55a9b0e812f0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9b116a590 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.525+0000 7f56811bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9b11ac420 con 0x55a9b0e894d0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56717fa640 1 -- 172.21.15.94:0/3151786573 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5664042030 con 0x7f5674101780 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60838/0 (socket says 172.21.15.94:60838) 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 -- 172.21.15.94:0/1604000684 learned_addr learned my addr 172.21.15.94:0/1604000684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f566bfff640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1251df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56737fe640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 0x55a9b124aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 -- 172.21.15.94:0/1604000684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 msgr2=0x55a9b1251df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1251df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 -- 172.21.15.94:0/1604000684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 msgr2=0x55a9b124aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 0x55a9b124aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 -- 172.21.15.94:0/1604000684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b11770d0 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56737fe640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 0x55a9b124aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5673fff640 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f566c010370 tx=0x7f566c010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24b7a3a6960ac9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56717fa640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 0x7f56600b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5669ffb640 1 -- 172.21.15.94:0/1604000684 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c016070 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5669ffb640 1 -- 172.21.15.94:0/1604000684 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c004d20 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f5669ffb640 1 -- 172.21.15.94:0/1604000684 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0053c0 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56737fe640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 0x7f56600b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.526+0000 7f56717fa640 1 -- 172.21.15.94:0/3151786573 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f56640ff220 con 0x7f5674101780 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 -- 172.21.15.94:0/1604000684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 msgr2=0x55a9b124a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56737fe640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 0x7f56600b2070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55a9b124bd50 tx=0x7f56640b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f566c010370 tx=0x7f566c010830 comp rx=0 tx=0).stop 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 -- 172.21.15.94:0/1604000684 shutdown_connections 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1251df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e812f0 0x55a9b124aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 --2- 172.21.15.94:0/1604000684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0eaa1b0 0x55a9b124a970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 -- 172.21.15.94:0/1604000684 >> 172.21.15.94:0/1604000684 conn(0x55a9b0e9e300 msgr2=0x55a9b1246300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 -- 172.21.15.94:0/1604000684 shutdown_connections 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.527+0000 7f56811bc740 1 -- 172.21.15.94:0/1604000684 wait complete. 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 -- start start 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 Processor -- start 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1254670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1254bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1252840 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a9b1137b60 con 0x55a9b0e812f0 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a9b12529c0 con 0x55a9b0e894d0 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f56811bc740 10 client.0 _ll_register_callbacks cb 0x55a9b115c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f5673fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f5673fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37378/0 (socket says 172.21.15.94:37378) 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f5673fff640 1 -- 172.21.15.94:0/2316749301 learned_addr learned my addr 172.21.15.94:0/2316749301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f566bfff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1254bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.528+0000 7f5673fff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1254670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 -- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 msgr2=0x55a9b1254670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1254670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 -- 172.21.15.94:0/2316749301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 msgr2=0x55a9b125bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 -- 172.21.15.94:0/2316749301 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b0ec2210 con 0x55a9b0e894d0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f566bfff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1254bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f567419b2e0 tx=0x7f565c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559878d9c3ebdb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f5652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00f920 con 0x55a9b0e894d0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c004d20 con 0x55a9b0e894d0 2024-04-25T13:49:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f565c00fac0 con 0x55a9b0e894d0 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f565c00fcc0 con 0x55a9b0e894d0 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f56811bc740 10 client.27715 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9b0f8e620 con 0x55a9b0e894d0 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.529+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c004d20 con 0x55a9b0e894d0 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f565c026450 con 0x55a9b0e894d0 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 10 client.27715 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 10 client.27715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 20 client.27715 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e68d80 0x55a9b1262380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 4 client.27715 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b11be070 con 0x55a9b0e68d80 2024-04-25T13:49:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f5673fff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e68d80 0x55a9b1262380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f5652ffd640 10 client.27715 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.530+0000 7f5673fff640 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e68d80 0x55a9b1262380 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1353663143673449 server_cookie=41df883eed56a7bb in_seq=0 out_seq=0 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.532+0000 7f5652ffd640 1 -- 172.21.15.94:0/2316749301 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a9b11be070 con 0x55a9b0e68d80 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.532+0000 7f5652ffd640 10 client.27715 handle_command_reply: tid=0 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.532+0000 7f56811bc740 1 client.27715 shutdown 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.532+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 msgr2=0x55a9b1254bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.532+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1254bb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f567419b2e0 tx=0x7f565c012b60 comp rx=0 tx=0).stop 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 shutdown_connections 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e68d80 0x55a9b1262380 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f5652ffd640 0 client.27715 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b1254bb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 --2- 172.21.15.94:0/2316749301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1254670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 >> 172.21.15.94:0/2316749301 conn(0x55a9b0e9e300 msgr2=0x55a9b1245250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 shutdown_connections 2024-04-25T13:49:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 1 -- 172.21.15.94:0/2316749301 wait complete. 2024-04-25T13:49:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.533+0000 7f56811bc740 20 client.27715 trim_cache size 0 max 16384 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 Processor -- start 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 -- start start 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1245be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b125ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ab550 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ad090 con 0x55a9b0e812f0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f56811bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56741abc20 con 0x55a9b0e894d0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f5673fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f5673fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37380/0 (socket says 172.21.15.94:37380) 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f5673fff640 1 -- 172.21.15.94:0/883910217 learned_addr learned my addr 172.21.15.94:0/883910217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f566bfff640 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b125ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.548+0000 7f5673fff640 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1245be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 -- 172.21.15.94:0/883910217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 msgr2=0x55a9b125ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b125ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 -- 172.21.15.94:0/883910217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 msgr2=0x55a9b1245be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1245be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 -- 172.21.15.94:0/883910217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b125bf70 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f5673fff640 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f566c010d30 tx=0x7f566c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92dad30e6d6ee4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f56697fa640 1 -- 172.21.15.94:0/883910217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c016040 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f56697fa640 1 -- 172.21.15.94:0/883910217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c02d8a0 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.549+0000 7f56697fa640 1 -- 172.21.15.94:0/883910217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c023040 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- 172.21.15.94:0/883910217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 msgr2=0x55a9b1246120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f566c010d30 tx=0x7f566c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- 172.21.15.94:0/883910217 shutdown_connections 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0eaa1b0 0x55a9b1246120 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0e894d0 0x55a9b125ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 --2- 172.21.15.94:0/883910217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1245be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- 172.21.15.94:0/883910217 >> 172.21.15.94:0/883910217 conn(0x55a9b0e9e300 msgr2=0x55a9b124eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- 172.21.15.94:0/883910217 shutdown_connections 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- 172.21.15.94:0/883910217 wait complete. 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 -- start start 2024-04-25T13:49:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.550+0000 7f56811bc740 1 Processor -- start 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1258d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ac9c0 con 0x55a9b0e894d0 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ac2f0 con 0x55a9b0e812f0 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56741a9530 con 0x55a9b0eaa1b0 2024-04-25T13:49:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 10 client.0 _ll_register_callbacks cb 0x7f565c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37396/0 (socket says 172.21.15.94:37396) 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 -- 172.21.15.94:0/2651879437 learned_addr learned my addr 172.21.15.94:0/2651879437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5673fff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1258d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5673fff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 -- 172.21.15.94:0/2651879437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 msgr2=0x55a9b125a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 -- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 msgr2=0x55a9b1258d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1258d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 -- 172.21.15.94:0/2651879437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b125a790 con 0x55a9b0e894d0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5673fff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f566bfff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55a9b12405f0 tx=0x7f565c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba05dd1e45db5252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00f920 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f565c00fac0 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.551+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c02d8a0 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f565c037050 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 10 client.18402 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9b125ba50 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c02d8a0 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f565c0035f0 con 0x55a9b0e894d0 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 10 client.18402 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 20 client.18402 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e3e1c0 0x55a9b12ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 4 client.18402 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9b11b7ee0 con 0x55a9b0e3e1c0 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5673fff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e3e1c0 0x55a9b12ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5652ffd640 10 client.18402 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.552+0000 7f5673fff640 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e3e1c0 0x55a9b12ccc50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f85bf0a6d802c6 server_cookie=8de781617c22a7c9 in_seq=0 out_seq=0 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f5652ffd640 1 -- 172.21.15.94:0/2651879437 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9b11b7ee0 con 0x55a9b0e3e1c0 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f5652ffd640 10 client.18402 handle_command_reply: tid=0 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 client.18402 shutdown 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 msgr2=0x55a9b12592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55a9b12405f0 tx=0x7f565c00f810 comp rx=0 tx=0).stop 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 shutdown_connections 2024-04-25T13:49:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a9b0e3e1c0 0x55a9b12ccc50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a9b0eaa1b0 0x55a9b125a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9b0e894d0 0x55a9b12592a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f5652ffd640 0 client.18402 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 --2- 172.21.15.94:0/2651879437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9b0e812f0 0x55a9b1258d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.742+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 >> 172.21.15.94:0/2651879437 conn(0x55a9b0e9e300 msgr2=0x55a9b1245be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 shutdown_connections 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f56811bc740 1 -- 172.21.15.94:0/2651879437 wait complete. 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f56811bc740 20 client.18402 trim_cache size 0 max 16384 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:49:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:49:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:49:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 msgr2=0x7f56600b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 0x7f56600b2070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55a9b124bd50 tx=0x7f56640b0040 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 msgr2=0x7f567419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664004270 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 shutdown_connections 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f56600afb60 0x7f56600b2070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56741050d0 0x7f56741a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5674102280 0x7f567419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 --2- 172.21.15.94:0/3151786573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5674101780 0x7f567419a240 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.743+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 >> 172.21.15.94:0/3151786573 conn(0x7f56740fd380 msgr2=0x7f5674105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.744+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 shutdown_connections 2024-04-25T13:49:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:49:59.744+0000 7f567a227640 1 -- 172.21.15.94:0/3151786573 wait complete. 2024-04-25T13:49:59.804 INFO:tasks.workunit.client.0.smithi094.stdout: 1 980038 12.94 MB/sec execute 282 sec latency 124.034 ms 2024-04-25T13:49:59.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:49:59.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:49:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 -- 172.21.15.94:0/405855014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288103c20 msgr2=0x7f8288104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 --2- 172.21.15.94:0/405855014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288103c20 0x7f8288104030 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f827802a410 comp rx=0 tx=0).stop 2024-04-25T13:49:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 -- 172.21.15.94:0/405855014 shutdown_connections 2024-04-25T13:49:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 --2- 172.21.15.94:0/405855014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 0x7f828810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 --2- 172.21.15.94:0/405855014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288103c20 0x7f8288104030 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 --2- 172.21.15.94:0/405855014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82881029e0 0x7f82881036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 -- 172.21.15.94:0/405855014 >> 172.21.15.94:0/405855014 conn(0x7f82880fdbd0 msgr2=0x7f8288100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 -- 172.21.15.94:0/405855014 shutdown_connections 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 -- 172.21.15.94:0/405855014 wait complete. 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.947+0000 7f828e18f640 1 Processor -- start 2024-04-25T13:49:59.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 -- start start 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103c20 0x7f828819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 0x7f82881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f828810e4e0 con 0x7f82881029e0 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f828810e360 con 0x7f8288108920 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f828e18f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f828810e660 con 0x7f8288103c20 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37426/0 (socket says 172.21.15.94:37426) 2024-04-25T13:49:59.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 -- 172.21.15.94:0/141497766 learned_addr learned my addr 172.21.15.94:0/141497766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f8286ffd640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103c20 0x7f828819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f8287fff640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 0x7f82881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 -- 172.21.15.94:0/141497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103c20 msgr2=0x7f828819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103c20 0x7f828819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 -- 172.21.15.94:0/141497766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 msgr2=0x7f82881a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 0x7f82881a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.948+0000 7f82877fe640 1 -- 172.21.15.94:0/141497766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82881a2070 con 0x7f82881029e0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f82877fe640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d740 tx=0x7f827c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f8284ff9640 1 -- 172.21.15.94:0/141497766 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c017070 con 0x7f82881029e0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a2360 con 0x7f82881029e0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f8284ff9640 1 -- 172.21.15.94:0/141497766 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c00de50 con 0x7f82881029e0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f8284ff9640 1 -- 172.21.15.94:0/141497766 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c003c30 con 0x7f82881029e0 2024-04-25T13:49:59.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.949+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a28b0 con 0x7f82881029e0 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.951+0000 7f8295042740 1 Processor -- start 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.951+0000 7f8295042740 1 -- start start 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.951+0000 7f8284ff9640 1 -- 172.21.15.94:0/141497766 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f827c010460 con 0x7f82881029e0 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555dd8598d80 con 0x555dd82694d0 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555dd85528b0 con 0x555dd82716b0 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8295042740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dd8594d40 con 0x555dd8292390 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37430/0 (socket says 172.21.15.94:37430) 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8287fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60904/0 (socket says 172.21.15.94:60904) 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8287fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34472/0 (socket says 172.21.15.94:34472) 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8286ffd640 1 -- 172.21.15.94:0/2533949642 learned_addr learned my addr 172.21.15.94:0/2533949642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 -- 172.21.15.94:0/2533949642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 msgr2=0x555dd8633a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 -- 172.21.15.94:0/2533949642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 msgr2=0x555dd8634c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f82877fe640 1 -- 172.21.15.94:0/2533949642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd8523a40 con 0x555dd82694d0 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8287fff640 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8286ffd640 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:49:59.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8284ff9640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 0x7f82700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.952+0000 7f8287fff640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 0x7f82700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.953+0000 7f82877fe640 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f827c0dc0c0 tx=0x7f827c0e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7b959f774c9fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.953+0000 7f8284ff9640 1 -- 172.21.15.94:0/141497766 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f827c0de690 con 0x7f82881029e0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.953+0000 7f826dffb640 1 -- 172.21.15.94:0/2533949642 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c020070 con 0x555dd82694d0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.953+0000 7f826dffb640 1 -- 172.21.15.94:0/2533949642 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c07de50 con 0x555dd82694d0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.953+0000 7f826dffb640 1 -- 172.21.15.94:0/2533949642 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0a2a40 con 0x555dd82694d0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- 172.21.15.94:0/2533949642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 msgr2=0x555dd8633fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8287fff640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 0x7f82700b2000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x555dd8634ab0 tx=0x7f8280009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f827c0dc0c0 tx=0x7f827c0e6690 comp rx=0 tx=0).stop 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- 172.21.15.94:0/2533949642 shutdown_connections 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8634c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8633fe0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 --2- 172.21.15.94:0/2533949642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8292390 0x555dd8633a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- 172.21.15.94:0/2533949642 >> 172.21.15.94:0/2533949642 conn(0x555dd82864e0 msgr2=0x555dd862f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- 172.21.15.94:0/2533949642 shutdown_connections 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- 172.21.15.94:0/2533949642 wait complete. 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 -- start start 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.954+0000 7f8295042740 1 Processor -- start 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd863d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8292390 0x555dd8644de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555dd8635780 con 0x555dd82716b0 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555dd851fd10 con 0x555dd8292390 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dd8635900 con 0x555dd82694d0 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 10 client.0 _ll_register_callbacks cb 0x555dd854c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37434/0 (socket says 172.21.15.94:37434) 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 -- 172.21.15.94:0/768241995 learned_addr learned my addr 172.21.15.94:0/768241995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f82877fe640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd863d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8287fff640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8292390 0x555dd8644de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 -- 172.21.15.94:0/768241995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 msgr2=0x555dd863d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd863d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 -- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8292390 msgr2=0x555dd8644de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8292390 0x555dd8644de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 -- 172.21.15.94:0/768241995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd83747d0 con 0x555dd82716b0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8286ffd640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8278005ab0 tx=0x7f8278031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da453dda06addf65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035040 con 0x555dd82716b0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8278004240 con 0x555dd82716b0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.955+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827c07de50 con 0x555dd82716b0 2024-04-25T13:49:59.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82780318f0 con 0x555dd82716b0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 10 client.18414 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555dd82aa3f0 con 0x555dd82716b0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c07de50 con 0x555dd82716b0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8278044020 con 0x555dd82716b0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 10 client.18414 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 20 client.18414 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8250f60 0x555dd864b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 4 client.18414 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dd85a6990 con 0x555dd8250f60 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.956+0000 7f82877fe640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8250f60 0x555dd864b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.957+0000 7f8266ffd640 10 client.18414 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.957+0000 7f82877fe640 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8250f60 0x555dd864b220 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03b912ecc85747d server_cookie=5e98cc4108b1f711 in_seq=0 out_seq=0 2024-04-25T13:49:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8266ffd640 1 -- 172.21.15.94:0/768241995 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dd85a6990 con 0x555dd8250f60 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8266ffd640 10 client.18414 handle_command_reply: tid=0 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 client.18414 shutdown 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 msgr2=0x555dd863de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8278005ab0 tx=0x7f8278031e80 comp rx=0 tx=0).stop 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 shutdown_connections 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8250f60 0x555dd864b220 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8292390 0x555dd8644de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd863de80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8266ffd640 0 client.18414 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 --2- 172.21.15.94:0/768241995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd863d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 >> 172.21.15.94:0/768241995 conn(0x555dd82864e0 msgr2=0x555dd8632180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 shutdown_connections 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 1 -- 172.21.15.94:0/768241995 wait complete. 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.958+0000 7f8295042740 20 client.18414 trim_cache size 0 max 16384 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 Processor -- start 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 -- start start 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8250f60 0x555dd863c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd86339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82881ab4d0 con 0x555dd82716b0 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82881ad010 con 0x555dd8250f60 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8295042740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82881abba0 con 0x555dd82694d0 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37450/0 (socket says 172.21.15.94:37450) 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 -- 172.21.15.94:0/859261434 learned_addr learned my addr 172.21.15.94:0/859261434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f82877fe640 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8250f60 0x555dd863c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8286ffd640 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd86339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 -- 172.21.15.94:0/859261434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 msgr2=0x555dd86339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd86339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 -- 172.21.15.94:0/859261434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8250f60 msgr2=0x555dd863c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8250f60 0x555dd863c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.976+0000 7f8287fff640 1 -- 172.21.15.94:0/859261434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd826a0f0 con 0x555dd82716b0 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8287fff640 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8280003c90 tx=0x7f828001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787da1f89e3500d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f826d7fa640 1 -- 172.21.15.94:0/859261434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828000fe60 con 0x555dd82716b0 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f826d7fa640 1 -- 172.21.15.94:0/859261434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280003cc0 con 0x555dd82716b0 2024-04-25T13:49:59.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f826d7fa640 1 -- 172.21.15.94:0/859261434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280029db0 con 0x555dd82716b0 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 -- 172.21.15.94:0/859261434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 msgr2=0x555dd8632830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8280003c90 tx=0x7f828001a000 comp rx=0 tx=0).stop 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 -- 172.21.15.94:0/859261434 shutdown_connections 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82716b0 0x555dd8632830 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd82694d0 0x555dd86339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 --2- 172.21.15.94:0/859261434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd8250f60 0x555dd863c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.977+0000 7f8295042740 1 -- 172.21.15.94:0/859261434 >> 172.21.15.94:0/859261434 conn(0x555dd82864e0 msgr2=0x555dd8632120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- 172.21.15.94:0/859261434 shutdown_connections 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- 172.21.15.94:0/859261434 wait complete. 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- start start 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 Processor -- start 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8250f60 0x555dd8642bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8644040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82881ac940 con 0x555dd82694d0 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82881ac270 con 0x555dd82716b0 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82881a94b0 con 0x555dd8250f60 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8295042740 10 client.0 _ll_register_callbacks cb 0x7f8278009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8286ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37464/0 (socket says 172.21.15.94:37464) 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8286ffd640 1 -- 172.21.15.94:0/1952727655 learned_addr learned my addr 172.21.15.94:0/1952727655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f82877fe640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8250f60 0x555dd8642bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.978+0000 7f8287fff640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8644040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 -- 172.21.15.94:0/1952727655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8250f60 msgr2=0x555dd8642bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8250f60 0x555dd8642bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 -- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 msgr2=0x555dd8644040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8644040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 -- 172.21.15.94:0/1952727655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd85594b0 con 0x555dd82694d0 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8287fff640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8644040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8286ffd640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8278009850 tx=0x7f8278031be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6d57f869a0ef67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035040 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f827803c640 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8280003cc0 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f827803c840 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8295042740 10 client.18420 fetch_fsmap learned FSMap version 27 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555dd8645760 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8280003cc0 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.979+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8278044040 con 0x555dd82694d0 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 10 client.18420 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 20 client.18420 populate_metadata read hostname 'smithi094' 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8292390 0x555dd86b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 4 client.18420 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dd85a0800 con 0x555dd8292390 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f82877fe640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8292390 0x555dd86b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f8266ffd640 10 client.18420 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.980+0000 7f82877fe640 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8292390 0x555dd86b5c70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7c8be12b964b9d server_cookie=f47d5694cf8d12fe in_seq=0 out_seq=0 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.982+0000 7f8266ffd640 1 -- 172.21.15.94:0/1952727655 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555dd85a0800 con 0x555dd8292390 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.982+0000 7f8266ffd640 10 client.18420 handle_command_reply: tid=0 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.982+0000 7f8295042740 1 client.18420 shutdown 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 msgr2=0x555dd8643110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8278009850 tx=0x7f8278031be0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 shutdown_connections 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555dd8292390 0x555dd86b5c70 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dd82716b0 0x555dd8644040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555dd82694d0 0x555dd8643110 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 --2- 172.21.15.94:0/1952727655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dd8250f60 0x555dd8642bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8266ffd640 0 client.18420 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 >> 172.21.15.94:0/1952727655 conn(0x555dd82864e0 msgr2=0x555dd862d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 shutdown_connections 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 1 -- 172.21.15.94:0/1952727655 wait complete. 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f8295042740 20 client.18420 trim_cache size 0 max 16384 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:49:59.988 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:49:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 msgr2=0x7f82700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 0x7f82700b2000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x555dd8634ab0 tx=0x7f8280009500 comp rx=0 tx=0).stop 2024-04-25T13:49:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 msgr2=0x7f828806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d740 tx=0x7f827c00dc00 comp rx=0 tx=0).stop 2024-04-25T13:49:59.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 shutdown_connections 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82700afaf0 0x7f82700b2000 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288108920 0x7f82881a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103c20 0x7f828819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 --2- 172.21.15.94:0/141497766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82881029e0 0x7f828806a890 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 >> 172.21.15.94:0/141497766 conn(0x7f82880fdbd0 msgr2=0x7f8288104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.983+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 shutdown_connections 2024-04-25T13:49:59.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:49:59.984+0000 7f828e18f640 1 -- 172.21.15.94:0/141497766 wait complete. 2024-04-25T13:49:59.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:49:59.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- 172.21.15.94:0/162804588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 msgr2=0x7fa4481036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/162804588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa4481036c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa438009900 tx=0x7fa43802a410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- 172.21.15.94:0/162804588 shutdown_connections 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/162804588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448108900 0x7fa44810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/162804588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448103c00 0x7fa448104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/162804588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa4481036c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- 172.21.15.94:0/162804588 >> 172.21.15.94:0/162804588 conn(0x7fa4480fdbd0 msgr2=0x7fa448100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- 172.21.15.94:0/162804588 shutdown_connections 2024-04-25T13:50:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- 172.21.15.94:0/162804588 wait complete. 2024-04-25T13:50:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 Processor -- start 2024-04-25T13:50:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.109+0000 7fa44e4ee640 1 -- start start 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448103c00 0x7fa44819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448108900 0x7fa4481a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa44810f5f0 con 0x7fa4481029c0 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa44810f470 con 0x7fa448108900 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44e4ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa44810f770 con 0x7fa448103c00 2024-04-25T13:50:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37492/0 (socket says 172.21.15.94:37492) 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 -- 172.21.15.94:0/1271514621 learned_addr learned my addr 172.21.15.94:0/1271514621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa44ca8b640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448108900 0x7fa4481a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa4477fe640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448103c00 0x7fa44819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 -- 172.21.15.94:0/1271514621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448103c00 msgr2=0x7fa44819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448103c00 0x7fa44819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 -- 172.21.15.94:0/1271514621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448108900 msgr2=0x7fa4481a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448108900 0x7fa4481a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 -- 172.21.15.94:0/1271514621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481a1d70 con 0x7fa4481029c0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.110+0000 7fa447fff640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa438006010 tx=0x7fa438004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.111+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43803f030 con 0x7fa4481029c0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.111+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481a2000 con 0x7fa4481029c0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.111+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa438003aa0 con 0x7fa4481029c0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.111+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438035070 con 0x7fa4481029c0 2024-04-25T13:50:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.111+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4481a2540 con 0x7fa4481029c0 2024-04-25T13:50:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.112+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa448107710 con 0x7fa4481029c0 2024-04-25T13:50:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.114+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa438042030 con 0x7fa4481029c0 2024-04-25T13:50:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.115+0000 7fa4457fa640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 0x7fa4300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.115+0000 7fa4477fe640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 0x7fa4300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.115+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa4380ff6c0 con 0x7fa4481029c0 2024-04-25T13:50:00.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.115+0000 7fa4477fe640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 0x7fa4300b2000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa44819b2d0 tx=0x7fa43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.119+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4380c7bb0 con 0x7fa4481029c0 2024-04-25T13:50:00.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:00 smithi094 ceph-mon[29195]: pgmap v593: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-04-25T13:50:00.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:00 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T13:50:00.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.351+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa44802db50 con 0x7fa4481029c0 2024-04-25T13:50:00.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.352+0000 7fa4457fa640 1 -- 172.21.15.94:0/1271514621 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fa4380cbeb0 con 0x7fa4481029c0 2024-04-25T13:50:00.352 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:50:00.352 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:50:00.353 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:50:00.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 msgr2=0x7fa4300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 0x7fa4300b2000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa44819b2d0 tx=0x7fa43c005f90 comp rx=0 tx=0).stop 2024-04-25T13:50:00.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 msgr2=0x7fa44819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa438006010 tx=0x7fa438004270 comp rx=0 tx=0).stop 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 shutdown_connections 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa4300afaf0 0x7fa4300b2000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa448108900 0x7fa4481a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa448103c00 0x7fa44819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 --2- 172.21.15.94:0/1271514621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4481029c0 0x7fa44819a230 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 >> 172.21.15.94:0/1271514621 conn(0x7fa4480fdbd0 msgr2=0x7fa4481048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 shutdown_connections 2024-04-25T13:50:00.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.354+0000 7fa44e4ee640 1 -- 172.21.15.94:0/1271514621 wait complete. 2024-04-25T13:50:00.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:50:00.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:00 smithi137 ceph-mon[32568]: pgmap v593: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-04-25T13:50:00.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:00 smithi137 ceph-mon[32568]: overall HEALTH_OK 2024-04-25T13:50:00.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 -- 172.21.15.94:0/243620218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c1029e0 msgr2=0x7f714c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 --2- 172.21.15.94:0/243620218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c1029e0 0x7f714c1036e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f713c009900 tx=0x7f713c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 -- 172.21.15.94:0/243620218 shutdown_connections 2024-04-25T13:50:00.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 --2- 172.21.15.94:0/243620218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c108920 0x7f714c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 --2- 172.21.15.94:0/243620218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 0x7f714c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 --2- 172.21.15.94:0/243620218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c1029e0 0x7f714c1036e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 -- 172.21.15.94:0/243620218 >> 172.21.15.94:0/243620218 conn(0x7f714c0fdbd0 msgr2=0x7f714c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 -- 172.21.15.94:0/243620218 shutdown_connections 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.482+0000 7f71540e0640 1 -- 172.21.15.94:0/243620218 wait complete. 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 Processor -- start 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 -- start start 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c1029e0 0x7f714c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 0x7f714c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10f610 con 0x7f714c108920 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10f490 con 0x7f714c103c20 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f71540e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10f790 con 0x7f714c1029e0 2024-04-25T13:50:00.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37506/0 (socket says 172.21.15.94:37506) 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 -- 172.21.15.94:0/3873853462 learned_addr learned my addr 172.21.15.94:0/3873853462 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715167b640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 0x7f714c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f7151e7c640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c1029e0 0x7f714c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 -- 172.21.15.94:0/3873853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c1029e0 msgr2=0x7f714c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c1029e0 0x7f714c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 -- 172.21.15.94:0/3873853462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 msgr2=0x7f714c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 0x7f714c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 -- 172.21.15.94:0/3873853462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c1a20c0 con 0x7f714c108920 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.483+0000 7f715267d640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f714400eb10 tx=0x7f714400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.484+0000 7f7142ffd640 1 -- 172.21.15.94:0/3873853462 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144004b20 con 0x7f714c108920 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.484+0000 7f7142ffd640 1 -- 172.21.15.94:0/3873853462 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144004cc0 con 0x7f714c108920 2024-04-25T13:50:00.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.484+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c1a23b0 con 0x7f714c108920 2024-04-25T13:50:00.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.484+0000 7f7142ffd640 1 -- 172.21.15.94:0/3873853462 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144010670 con 0x7f714c108920 2024-04-25T13:50:00.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.484+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a28b0 con 0x7f714c108920 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.486+0000 7f7142ffd640 1 -- 172.21.15.94:0/3873853462 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7144005020 con 0x7f714c108920 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.486+0000 7f715af91740 1 Processor -- start 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.486+0000 7f715af91740 1 -- start start 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 0x55db898a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894dd6b0 0x55db898a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db89804d80 con 0x55db894fe390 2024-04-25T13:50:00.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db897bea20 con 0x55db894dd6b0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715af91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db89800d40 con 0x55db894d54d0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37514/0 (socket says 172.21.15.94:37514) 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 -- 172.21.15.94:0/879232494 learned_addr learned my addr 172.21.15.94:0/879232494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f7151e7c640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 0x55db898a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f7142ffd640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 0x7f71380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894dd6b0 0x55db898a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715267d640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 0x7f71380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 -- 172.21.15.94:0/879232494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 msgr2=0x55db898a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 0x55db898a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 -- 172.21.15.94:0/879232494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894dd6b0 msgr2=0x55db898a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894dd6b0 0x55db898a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 -- 172.21.15.94:0/879232494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db897c9c30 con 0x55db894fe390 2024-04-25T13:50:00.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f7151e7c640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 0x55db898a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f7142ffd640 1 -- 172.21.15.94:0/3873853462 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f71440df280 con 0x7f714c108920 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f715167b640 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7148013620 tx=0x7f7148009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef1ef893f84023c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.487+0000 7f7137fff640 1 -- 172.21.15.94:0/879232494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714800f580 con 0x55db894fe390 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f7137fff640 1 -- 172.21.15.94:0/879232494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f714800fbd0 con 0x55db894fe390 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f7137fff640 1 -- 172.21.15.94:0/879232494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148010b00 con 0x55db894fe390 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715267d640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 0x7f71380b2000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71440dd050 tx=0x7f71440d5e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 -- 172.21.15.94:0/879232494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 msgr2=0x55db898a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7148013620 tx=0x7f7148009850 comp rx=0 tx=0).stop 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 -- 172.21.15.94:0/879232494 shutdown_connections 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894dd6b0 0x55db898a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894d54d0 0x55db898a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.488+0000 7f715af91740 1 --2- 172.21.15.94:0/879232494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894fe390 0x55db898a0810 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- 172.21.15.94:0/879232494 >> 172.21.15.94:0/879232494 conn(0x55db894f24e0 msgr2=0x55db8989c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- 172.21.15.94:0/879232494 shutdown_connections 2024-04-25T13:50:00.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- 172.21.15.94:0/879232494 wait complete. 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- start start 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 Processor -- start 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894fe390 0x55db898b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db898a24d0 con 0x55db894dd6b0 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db8978bdc0 con 0x55db894d54d0 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db898a2650 con 0x55db894fe390 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.489+0000 7f715af91740 10 client.0 _ll_register_callbacks cb 0x55db897a7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37524/0 (socket says 172.21.15.94:37524) 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 -- 172.21.15.94:0/2485942754 learned_addr learned my addr 172.21.15.94:0/2485942754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f715167b640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894fe390 0x55db898b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f715167b640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 -- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894fe390 msgr2=0x55db898b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894fe390 0x55db898b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 -- 172.21.15.94:0/2485942754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 msgr2=0x55db898aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 -- 172.21.15.94:0/2485942754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db895163f0 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f7151e7c640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f713c009230 tx=0x7f713c031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09fcb31532ed9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f71357fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c035040 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f713c004040 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f714800fbd0 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f713c004240 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f715af91740 10 client.18438 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.490+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db895e07d0 con 0x55db894dd6b0 2024-04-25T13:50:00.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f714800fbd0 con 0x55db894dd6b0 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f713c03c440 con 0x55db894dd6b0 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 10 client.18438 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 10 client.18438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 20 client.18438 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894bcf60 0x55db898b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 4 client.18438 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db89812990 con 0x55db894bcf60 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715167b640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894bcf60 0x55db898b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f71357fa640 10 client.18438 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.491+0000 7f715167b640 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894bcf60 0x55db898b8210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ab5851a1b956636 server_cookie=4bb9aa9065f4938b in_seq=0 out_seq=0 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f71357fa640 1 -- 172.21.15.94:0/2485942754 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db89812990 con 0x55db894bcf60 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f71357fa640 10 client.18438 handle_command_reply: tid=0 2024-04-25T13:50:00.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 client.18438 shutdown 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 msgr2=0x55db898aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f713c009230 tx=0x7f713c031d20 comp rx=0 tx=0).stop 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 shutdown_connections 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894bcf60 0x55db898b8210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894fe390 0x55db898b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f71357fa640 0 client.18438 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894dd6b0 0x55db898aaab0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 --2- 172.21.15.94:0/2485942754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 >> 172.21.15.94:0/2485942754 conn(0x55db894f24e0 msgr2=0x55db8989ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.492+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 shutdown_connections 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.493+0000 7f715af91740 1 -- 172.21.15.94:0/2485942754 wait complete. 2024-04-25T13:50:00.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.493+0000 7f715af91740 20 client.18438 trim_cache size 0 max 16384 2024-04-25T13:50:00.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:00 smithi138 ceph-mon[32977]: pgmap v593: 97 pgs: 97 active+clean; 29 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 349 op/s 2024-04-25T13:50:00.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:00 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.506+0000 7f715af91740 1 Processor -- start 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.506+0000 7f715af91740 1 -- start start 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db8989bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ab4c0 con 0x55db894bcf60 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ad000 con 0x55db894d54d0 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715af91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1abb90 con 0x55db894dd6b0 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37534/0 (socket says 172.21.15.94:37534) 2024-04-25T13:50:00.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 -- 172.21.15.94:0/905961569 learned_addr learned my addr 172.21.15.94:0/905961569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f7151e7c640 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db8989bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 -- 172.21.15.94:0/905961569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 msgr2=0x55db8989bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db8989bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 -- 172.21.15.94:0/905961569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 msgr2=0x55db898b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 -- 172.21.15.94:0/905961569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db898a4dc0 con 0x55db894bcf60 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f715167b640 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7148013fd0 tx=0x7f7148009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f07053e8fcc6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f71377fe640 1 -- 172.21.15.94:0/905961569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148028040 con 0x55db894bcf60 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f71377fe640 1 -- 172.21.15.94:0/905961569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71480052c0 con 0x55db894bcf60 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.507+0000 7f71377fe640 1 -- 172.21.15.94:0/905961569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148003850 con 0x55db894bcf60 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- 172.21.15.94:0/905961569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 msgr2=0x55db8989ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7148013fd0 tx=0x7f7148009f90 comp rx=0 tx=0).stop 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- 172.21.15.94:0/905961569 shutdown_connections 2024-04-25T13:50:00.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db8989bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 --2- 172.21.15.94:0/905961569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db8989ba50 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- 172.21.15.94:0/905961569 >> 172.21.15.94:0/905961569 conn(0x55db894f24e0 msgr2=0x55db898a52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- 172.21.15.94:0/905961569 shutdown_connections 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- 172.21.15.94:0/905961569 wait complete. 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.508+0000 7f715af91740 1 -- start start 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 Processor -- start 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db898affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ac930 con 0x55db894bcf60 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ac260 con 0x55db894d54d0 2024-04-25T13:50:00.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1a94a0 con 0x55db894dd6b0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715af91740 10 client.0 _ll_register_callbacks cb 0x7f713c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37544/0 (socket says 172.21.15.94:37544) 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 -- 172.21.15.94:0/3992135606 learned_addr learned my addr 172.21.15.94:0/3992135606 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f7151e7c640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db898affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 -- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 msgr2=0x55db898affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db898affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 -- 172.21.15.94:0/3992135606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 msgr2=0x55db898af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.509+0000 7f715167b640 1 -- 172.21.15.94:0/3992135606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db898b1e70 con 0x55db894bcf60 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715167b640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7148006010 tx=0x7f71480057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66935b910e9fac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148030410 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f71480305b0 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71480052c0 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71480307b0 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 10 client.18444 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db898b1620 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71480052c0 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f714800f610 con 0x55db894bcf60 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 10 client.18444 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 10 client.18444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 20 client.18444 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894fe390 0x55db89922590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 4 client.18444 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db8980c800 con 0x55db894fe390 2024-04-25T13:50:00.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.510+0000 7f7151e7c640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894fe390 0x55db89922590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f71357fa640 10 client.18444 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f7151e7c640 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894fe390 0x55db89922590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12cacd22b5988a0c server_cookie=86ed6dcbffd376ea in_seq=0 out_seq=0 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f71357fa640 1 -- 172.21.15.94:0/3992135606 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db8980c800 con 0x55db894fe390 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f71357fa640 10 client.18444 handle_command_reply: tid=0 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 client.18444 shutdown 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 msgr2=0x55db898aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7148006010 tx=0x7f71480057c0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 shutdown_connections 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55db894fe390 0x55db89922590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db894dd6b0 0x55db898affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db894d54d0 0x55db898af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 --2- 172.21.15.94:0/3992135606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db894bcf60 0x55db898aeb90 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f71357fa640 0 client.18444 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:00.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 >> 172.21.15.94:0/3992135606 conn(0x55db894f24e0 msgr2=0x55db8989ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 shutdown_connections 2024-04-25T13:50:00.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 1 -- 172.21.15.94:0/3992135606 wait complete. 2024-04-25T13:50:00.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.511+0000 7f715af91740 20 client.18444 trim_cache size 0 max 16384 2024-04-25T13:50:00.518 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 msgr2=0x7f71380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 0x7f71380b2000 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71440dd050 tx=0x7f71440d5e10 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 msgr2=0x7f714c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f714400eb10 tx=0x7f714400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 shutdown_connections 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71380afaf0 0x7f71380b2000 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714c108920 0x7f714c1a1a50 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f714c103c20 0x7f714c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 --2- 172.21.15.94:0/3873853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f714c1029e0 0x7f714c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 >> 172.21.15.94:0/3873853462 conn(0x7f714c0fdbd0 msgr2=0x7f714c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 shutdown_connections 2024-04-25T13:50:00.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.512+0000 7f71540e0640 1 -- 172.21.15.94:0/3873853462 wait complete. 2024-04-25T13:50:00.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 -- 172.21.15.94:0/1656241082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 msgr2=0x7faea81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 --2- 172.21.15.94:0/1656241082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea81047f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7faea4009900 tx=0x7faea402a410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 -- 172.21.15.94:0/1656241082 shutdown_connections 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 --2- 172.21.15.94:0/1656241082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8109a30 0x7faea810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 --2- 172.21.15.94:0/1656241082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8104d30 0x7faea8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 --2- 172.21.15.94:0/1656241082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea81047f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 -- 172.21.15.94:0/1656241082 >> 172.21.15.94:0/1656241082 conn(0x7faea80fece0 msgr2=0x7faea8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 -- 172.21.15.94:0/1656241082 shutdown_connections 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.638+0000 7faeaed5e640 1 -- 172.21.15.94:0/1656241082 wait complete. 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 Processor -- start 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 -- start start 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 0x7faea819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8109a30 0x7faea81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea810f5f0 con 0x7faea8103af0 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faea810f470 con 0x7faea8109a30 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeaed5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea810f770 con 0x7faea8104d30 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37560/0 (socket says 172.21.15.94:37560) 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 -- 172.21.15.94:0/135679350 learned_addr learned my addr 172.21.15.94:0/135679350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7fae9ffff640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 0x7faea819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faead2fb640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8109a30 0x7faea81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 -- 172.21.15.94:0/135679350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 msgr2=0x7faea819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 0x7faea819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 -- 172.21.15.94:0/135679350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8109a30 msgr2=0x7faea81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8109a30 0x7faea81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7faeacafa640 1 -- 172.21.15.94:0/135679350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea81a1d60 con 0x7faea8103af0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.639+0000 7fae9ffff640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 0x7faea819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7faeacafa640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7faea4006010 tx=0x7faea4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7fae9dffb640 1 -- 172.21.15.94:0/135679350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea403f030 con 0x7faea8103af0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7fae9dffb640 1 -- 172.21.15.94:0/135679350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4031910 con 0x7faea8103af0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea8075ee0 con 0x7faea8103af0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7fae9dffb640 1 -- 172.21.15.94:0/135679350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4035070 con 0x7faea8103af0 2024-04-25T13:50:00.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.640+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea8076420 con 0x7faea8103af0 2024-04-25T13:50:00.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7fae9dffb640 1 -- 172.21.15.94:0/135679350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7faea4042030 con 0x7faea8103af0 2024-04-25T13:50:00.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 Processor -- start 2024-04-25T13:50:00.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 -- start start 2024-04-25T13:50:00.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e96ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e96fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d25e8d2d80 con 0x55d25e5ab6b0 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d25e88ca20 con 0x55d25e5a34d0 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeb5c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d25e8ced40 con 0x55d25e5cc390 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faead2fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faeacafa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e96ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e96fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faead2fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34580/0 (socket says 172.21.15.94:34580) 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.643+0000 7faead2fb640 1 -- 172.21.15.94:0/196494618 learned_addr learned my addr 172.21.15.94:0/196494618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 -- 172.21.15.94:0/196494618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 msgr2=0x55d25e96ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e96ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 -- 172.21.15.94:0/196494618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 msgr2=0x55d25e96fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e96fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae9dffb640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 0x7fae900b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 -- 172.21.15.94:0/196494618 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25e897c30 con 0x55d25e5cc390 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae9ffff640 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e96fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faeacafa640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 0x7fae900b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faead2fb640 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faea0009fd0 tx=0x7faea00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a77eef966103bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae9dffb640 1 -- 172.21.15.94:0/135679350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7faea40fef20 con 0x7faea8103af0 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae8effd640 1 -- 172.21.15.94:0/196494618 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0004d20 con 0x55d25e5cc390 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7faeacafa640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 0x7fae900b2080 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d25e96e530 tx=0x7faea4102a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae8effd640 1 -- 172.21.15.94:0/196494618 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0004050 con 0x55d25e5cc390 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.644+0000 7fae8effd640 1 -- 172.21.15.94:0/196494618 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0005280 con 0x55d25e5cc390 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- 172.21.15.94:0/196494618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 msgr2=0x55d25e96e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faea0009fd0 tx=0x7faea00118d0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- 172.21.15.94:0/196494618 shutdown_connections 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e96fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e96ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 --2- 172.21.15.94:0/196494618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e96e810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- 172.21.15.94:0/196494618 >> 172.21.15.94:0/196494618 conn(0x55d25e5c04e0 msgr2=0x55d25e96a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- 172.21.15.94:0/196494618 shutdown_connections 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- 172.21.15.94:0/196494618 wait complete. 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.645+0000 7faeb5c10740 1 -- start start 2024-04-25T13:50:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 Processor -- start 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5a34d0 0x55d25e978530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e97f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d25e970530 con 0x55d25e5ab6b0 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d25e859dc0 con 0x55d25e5cc390 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d25e9706b0 con 0x55d25e5a34d0 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faeb5c10740 10 client.0 _ll_register_callbacks cb 0x55d25e875600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37576/0 (socket says 172.21.15.94:37576) 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 -- 172.21.15.94:0/2835990080 learned_addr learned my addr 172.21.15.94:0/2835990080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faead2fb640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5a34d0 0x55d25e978530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faead2fb640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e97f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 -- 172.21.15.94:0/2835990080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5a34d0 msgr2=0x55d25e978530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5a34d0 0x55d25e978530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 -- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 msgr2=0x55d25e97f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e97f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7fae9ffff640 1 -- 172.21.15.94:0/2835990080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25e5e43f0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.646+0000 7faead2fb640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e97f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fae98009930 tx=0x7fae98009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3fd678216f315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003e20 con 0x55d25e5ab6b0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fae98005020 con 0x55d25e5ab6b0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0004050 con 0x55d25e5ab6b0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae98005220 con 0x55d25e5ab6b0 2024-04-25T13:50:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 10 client.18456 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d25e6ae7d0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0004050 con 0x55d25e5ab6b0 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae9801e4e0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 10 client.18456 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 10 client.18456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 20 client.18456 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e58af60 0x55d25e986300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 4 client.18456 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.647+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25e8e0990 con 0x55d25e58af60 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.648+0000 7faead2fb640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e58af60 0x55d25e986300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.648+0000 7fae87fff640 10 client.18456 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.648+0000 7faead2fb640 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e58af60 0x55d25e986300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=710c5ae58017730b server_cookie=30aa06771275789d in_seq=0 out_seq=0 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7fae87fff640 1 -- 172.21.15.94:0/2835990080 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d25e8e0990 con 0x55d25e58af60 2024-04-25T13:50:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7fae87fff640 10 client.18456 handle_command_reply: tid=0 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 client.18456 shutdown 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 msgr2=0x55d25e978a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fae98009930 tx=0x7fae98009df0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 shutdown_connections 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e58af60 0x55d25e986300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e97f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7fae87fff640 0 client.18456 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e978a70 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2835990080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5a34d0 0x55d25e978530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 >> 172.21.15.94:0/2835990080 conn(0x55d25e5c04e0 msgr2=0x55d25e96cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 shutdown_connections 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 1 -- 172.21.15.94:0/2835990080 wait complete. 2024-04-25T13:50:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.649+0000 7faeb5c10740 20 client.18456 trim_cache size 0 max 16384 2024-04-25T13:50:00.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 Processor -- start 2024-04-25T13:50:00.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 -- start start 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 0x55d25e97f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e9737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a9320 con 0x55d25e5a34d0 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aae60 con 0x55d25e5cc390 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faeb5c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a99f0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faead2fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faead2fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37580/0 (socket says 172.21.15.94:37580) 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faead2fb640 1 -- 172.21.15.94:0/2983879506 learned_addr learned my addr 172.21.15.94:0/2983879506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 0x55d25e97f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faead2fb640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e9737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 -- 172.21.15.94:0/2983879506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 msgr2=0x55d25e9737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e9737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 -- 172.21.15.94:0/2983879506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 msgr2=0x55d25e973290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7fae9ffff640 1 -- 172.21.15.94:0/2983879506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25e982ba0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.663+0000 7faead2fb640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 0x55d25e97f8f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e950 tx=0x7fae98003fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148aed14077a1a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7fae8e7fc640 1 -- 172.21.15.94:0/2983879506 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98035020 con 0x55d25e5ab6b0 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7fae8e7fc640 1 -- 172.21.15.94:0/2983879506 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9801eb40 con 0x55d25e5ab6b0 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7fae8e7fc640 1 -- 172.21.15.94:0/2983879506 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98031900 con 0x55d25e5ab6b0 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7faeb5c10740 1 -- 172.21.15.94:0/2983879506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 msgr2=0x55d25e97f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 0x55d25e97f8f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e950 tx=0x7fae98003fd0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.664+0000 7faeb5c10740 1 -- 172.21.15.94:0/2983879506 shutdown_connections 2024-04-25T13:50:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5cc390 0x55d25e9737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5ab6b0 0x55d25e97f8f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2983879506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5a34d0 0x55d25e973290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- 172.21.15.94:0/2983879506 >> 172.21.15.94:0/2983879506 conn(0x55d25e5c04e0 msgr2=0x55d25e978530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- 172.21.15.94:0/2983879506 shutdown_connections 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- 172.21.15.94:0/2983879506 wait complete. 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- start start 2024-04-25T13:50:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 Processor -- start 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e97dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e97ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aa790 con 0x55d25e5ab6b0 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aa0c0 con 0x55d25e5a34d0 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faea81a7300 con 0x55d25e5cc390 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7faeb5c10740 10 client.0 _ll_register_callbacks cb 0x7fae98004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.665+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37584/0 (socket says 172.21.15.94:37584) 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 -- 172.21.15.94:0/2647953279 learned_addr learned my addr 172.21.15.94:0/2647953279 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7faead2fb640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e97ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7faead2fb640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e97dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 -- 172.21.15.94:0/2647953279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 msgr2=0x55d25e97ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e97ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 -- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 msgr2=0x55d25e97dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e97dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 -- 172.21.15.94:0/2647953279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25e97fc60 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae9ffff640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e8d0 tx=0x7fae9801e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab672a7ad691f590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae87fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003e20 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fae98031600 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9801eb40 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae98031820 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7faeb5c10740 10 client.18462 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.666+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d25e9ed6c0 con 0x55d25e5ab6b0 2024-04-25T13:50:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9801eb40 con 0x55d25e5ab6b0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae9804eb30 con 0x55d25e5ab6b0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 10 client.18462 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 10 client.18462 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 20 client.18462 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e5603a0 0x55d25e9eff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 4 client.18462 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d25e8da800 con 0x55d25e5603a0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faead2fb640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e5603a0 0x55d25e9eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7fae87fff640 10 client.18462 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.667+0000 7faead2fb640 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e5603a0 0x55d25e9eff80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b24e11cb7dd5411 server_cookie=105755e16a349df in_seq=0 out_seq=0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7fae87fff640 1 -- 172.21.15.94:0/2647953279 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d25e8da800 con 0x55d25e5603a0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7fae87fff640 10 client.18462 handle_command_reply: tid=0 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 client.18462 shutdown 2024-04-25T13:50:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 msgr2=0x55d25e97e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e8d0 tx=0x7fae9801e720 comp rx=0 tx=0).stop 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 shutdown_connections 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55d25e5603a0 0x55d25e9eff80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25e5cc390 0x55d25e97ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d25e5ab6b0 0x55d25e97e4e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7fae87fff640 0 client.18462 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 --2- 172.21.15.94:0/2647953279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d25e5a34d0 0x55d25e97dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 >> 172.21.15.94:0/2647953279 conn(0x55d25e5c04e0 msgr2=0x55d25e978420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 shutdown_connections 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 1 -- 172.21.15.94:0/2647953279 wait complete. 2024-04-25T13:50:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.668+0000 7faeb5c10740 20 client.18462 trim_cache size 0 max 16384 2024-04-25T13:50:00.675 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 msgr2=0x7fae900b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 0x7fae900b2080 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d25e96e530 tx=0x7faea4102a00 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 msgr2=0x7faea819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7faea4006010 tx=0x7faea4004270 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 shutdown_connections 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae900afb70 0x7fae900b2080 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea8109a30 0x7faea81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faea8104d30 0x7faea819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 --2- 172.21.15.94:0/135679350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea8103af0 0x7faea819a220 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 >> 172.21.15.94:0/135679350 conn(0x7faea80fece0 msgr2=0x7faea8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 shutdown_connections 2024-04-25T13:50:00.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.669+0000 7faeaed5e640 1 -- 172.21.15.94:0/135679350 wait complete. 2024-04-25T13:50:00.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 -- 172.21.15.94:0/3901665140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50104d40 msgr2=0x7fdb50105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 --2- 172.21.15.94:0/3901665140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50104d40 0x7fdb50105150 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009900 tx=0x7fdb4c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 -- 172.21.15.94:0/3901665140 shutdown_connections 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 --2- 172.21.15.94:0/3901665140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50109a40 0x7fdb5010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 --2- 172.21.15.94:0/3901665140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50104d40 0x7fdb50105150 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 --2- 172.21.15.94:0/3901665140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50103b00 0x7fdb50104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 -- 172.21.15.94:0/3901665140 >> 172.21.15.94:0/3901665140 conn(0x7fdb500fecf0 msgr2=0x7fdb501011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 -- 172.21.15.94:0/3901665140 shutdown_connections 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 -- 172.21.15.94:0/3901665140 wait complete. 2024-04-25T13:50:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.794+0000 7fdb57772640 1 Processor -- start 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 -- start start 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50104d40 0x7fdb5019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50109a40 0x7fdb501a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010f600 con 0x7fdb50103b00 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010f480 con 0x7fdb50104d40 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb57772640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010f780 con 0x7fdb50109a40 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37608/0 (socket says 172.21.15.94:37608) 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 -- 172.21.15.94:0/2018663537 learned_addr learned my addr 172.21.15.94:0/2018663537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50109a40 0x7fdb501a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50104d40 0x7fdb5019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 -- 172.21.15.94:0/2018663537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50109a40 msgr2=0x7fdb501a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50109a40 0x7fdb501a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 -- 172.21.15.94:0/2018663537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50104d40 msgr2=0x7fdb5019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50104d40 0x7fdb5019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 -- 172.21.15.94:0/2018663537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb501a1e00 con 0x7fdb50103b00 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb5550e640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ed00 tx=0x7fdb4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.795+0000 7fdb467fc640 1 -- 172.21.15.94:0/2018663537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48010630 con 0x7fdb50103b00 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.796+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb501a20f0 con 0x7fdb50103b00 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.796+0000 7fdb467fc640 1 -- 172.21.15.94:0/2018663537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48010c80 con 0x7fdb50103b00 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.796+0000 7fdb467fc640 1 -- 172.21.15.94:0/2018663537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48018d50 con 0x7fdb50103b00 2024-04-25T13:50:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.796+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb501a2640 con 0x7fdb50103b00 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.798+0000 7fdb5e623740 1 Processor -- start 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.798+0000 7fdb5e623740 1 -- start start 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e089d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb467fc640 1 -- 172.21.15.94:0/2018663537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdb4801f020 con 0x7fdb50103b00 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556f8dfedd80 con 0x556f8dcc66b0 2024-04-25T13:50:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8dfa7a20 con 0x556f8dce7390 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5e623740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f8dfe9d40 con 0x556f8dcbe4d0 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb55d0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37612/0 (socket says 172.21.15.94:37612) 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb55d0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:32830/0 (socket says 172.21.15.94:32830) 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 -- 172.21.15.94:0/717598333 learned_addr learned my addr 172.21.15.94:0/717598333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb5550e640 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e089d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 -- 172.21.15.94:0/717598333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 msgr2=0x556f8e089d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e089d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 -- 172.21.15.94:0/717598333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 msgr2=0x556f8e089810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb54d0d640 1 -- 172.21.15.94:0/717598333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8dfb2c30 con 0x556f8dcc66b0 2024-04-25T13:50:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.799+0000 7fdb467fc640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 0x7fdb3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c031500 tx=0x7fdb4c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d862919cfd0e183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 0x7fdb3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb467fc640 1 -- 172.21.15.94:0/2018663537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdb480df050 con 0x7fdb50103b00 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb3b7fe640 1 -- 172.21.15.94:0/717598333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c035030 con 0x556f8dcc66b0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb3b7fe640 1 -- 172.21.15.94:0/717598333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c040070 con 0x556f8dcc66b0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.800+0000 7fdb3b7fe640 1 -- 172.21.15.94:0/717598333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c035030 con 0x556f8dcc66b0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- 172.21.15.94:0/717598333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 msgr2=0x556f8e08aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 0x7fdb3c0b2000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x556f8e08a860 tx=0x7fdb40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c031500 tx=0x7fdb4c009640 comp rx=0 tx=0).stop 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- 172.21.15.94:0/717598333 shutdown_connections 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e08aa20 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e089d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 --2- 172.21.15.94:0/717598333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e089810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- 172.21.15.94:0/717598333 >> 172.21.15.94:0/717598333 conn(0x556f8dcdb4e0 msgr2=0x556f8e085020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- 172.21.15.94:0/717598333 shutdown_connections 2024-04-25T13:50:00.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- 172.21.15.94:0/717598333 wait complete. 2024-04-25T13:50:00.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 -- start start 2024-04-25T13:50:00.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.801+0000 7fdb5e623740 1 Processor -- start 2024-04-25T13:50:00.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 0x556f8e0935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dce7390 0x556f8e09aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556f8e08b4d0 con 0x556f8dcc66b0 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8df74dc0 con 0x556f8dcbe4d0 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f8e08b650 con 0x556f8dce7390 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5e623740 10 client.0 _ll_register_callbacks cb 0x556f8df90600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb5550e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 0x556f8e0935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37614/0 (socket says 172.21.15.94:37614) 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 -- 172.21.15.94:0/122472116 learned_addr learned my addr 172.21.15.94:0/122472116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dce7390 0x556f8e09aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 -- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dce7390 msgr2=0x556f8e09aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dce7390 0x556f8e09aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 -- 172.21.15.94:0/122472116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 msgr2=0x556f8e0935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.802+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 0x556f8e0935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb54d0d640 1 -- 172.21.15.94:0/122472116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8dcff3f0 con 0x556f8dcc66b0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb5550e640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 0x556f8e0935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c004320 tx=0x7fdb4c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46ae428f9aa48d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c035070 con 0x556f8dcc66b0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c040070 con 0x556f8dcc66b0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdb4c0036d0 con 0x556f8dcc66b0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c0038d0 con 0x556f8dcc66b0 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb5e623740 10 client.18474 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f8ddc97d0 con 0x556f8dcc66b0 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.803+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c040070 con 0x556f8dcc66b0 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb4c045a10 con 0x556f8dcc66b0 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 10 client.18474 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 10 client.18474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 20 client.18474 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dca5f60 0x556f8e0a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 4 client.18474 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f8dffb990 con 0x556f8dca5f60 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5550e640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dca5f60 0x556f8e0a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb38ff9640 10 client.18474 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.804+0000 7fdb5550e640 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dca5f60 0x556f8e0a0eb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a620bbeac5a208e1 server_cookie=88aa58932c932b38 in_seq=0 out_seq=0 2024-04-25T13:50:00.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 982198 12.94 MB/sec execute 283 sec latency 13.178 ms 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb38ff9640 1 -- 172.21.15.94:0/122472116 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f8dffb990 con 0x556f8dca5f60 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb38ff9640 10 client.18474 handle_command_reply: tid=0 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 client.18474 shutdown 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 msgr2=0x556f8e093b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c004320 tx=0x7fdb4c004350 comp rx=0 tx=0).stop 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 shutdown_connections 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dca5f60 0x556f8e0a0eb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dce7390 0x556f8e09aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e093b00 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb38ff9640 0 client.18474 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 --2- 172.21.15.94:0/122472116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dcbe4d0 0x556f8e0935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 >> 172.21.15.94:0/122472116 conn(0x556f8dcdb4e0 msgr2=0x556f8e087f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 shutdown_connections 2024-04-25T13:50:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 1 -- 172.21.15.94:0/122472116 wait complete. 2024-04-25T13:50:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.805+0000 7fdb5e623740 20 client.18474 trim_cache size 0 max 16384 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 Processor -- start 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 -- start start 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e0850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e091cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a93f0 con 0x556f8dcc66b0 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501aaf30 con 0x556f8dce7390 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5e623740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a9ac0 con 0x556f8dcbe4d0 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37616/0 (socket says 172.21.15.94:37616) 2024-04-25T13:50:00.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 -- 172.21.15.94:0/1963835490 learned_addr learned my addr 172.21.15.94:0/1963835490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e091cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb5550e640 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e0850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 -- 172.21.15.94:0/1963835490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 msgr2=0x556f8e0850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e0850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 -- 172.21.15.94:0/1963835490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 msgr2=0x556f8e091cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e091cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.820+0000 7fdb54d0d640 1 -- 172.21.15.94:0/1963835490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8e091a50 con 0x556f8dcc66b0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c047010 tx=0x7fdb4c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a914b81c7a51f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb3affd640 1 -- 172.21.15.94:0/1963835490 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c035070 con 0x556f8dcc66b0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb3affd640 1 -- 172.21.15.94:0/1963835490 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c05d070 con 0x556f8dcc66b0 2024-04-25T13:50:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb3affd640 1 -- 172.21.15.94:0/1963835490 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0459b0 con 0x556f8dcc66b0 2024-04-25T13:50:00.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb5e623740 1 -- 172.21.15.94:0/1963835490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 msgr2=0x556f8e09a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.821+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c047010 tx=0x7fdb4c009620 comp rx=0 tx=0).stop 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 -- 172.21.15.94:0/1963835490 shutdown_connections 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e091cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcc66b0 0x556f8e09a980 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1963835490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcbe4d0 0x556f8e0850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 -- 172.21.15.94:0/1963835490 >> 172.21.15.94:0/1963835490 conn(0x556f8dcdb4e0 msgr2=0x556f8e0849e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 -- 172.21.15.94:0/1963835490 shutdown_connections 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 -- 172.21.15.94:0/1963835490 wait complete. 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 -- start start 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.822+0000 7fdb5e623740 1 Processor -- start 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcc66b0 0x556f8e098450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e099380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501aa860 con 0x556f8dcbe4d0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501aa190 con 0x556f8dce7390 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501a73d0 con 0x556f8dcc66b0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 10 client.0 _ll_register_callbacks cb 0x7fdb40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37618/0 (socket says 172.21.15.94:37618) 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 -- 172.21.15.94:0/1446843914 learned_addr learned my addr 172.21.15.94:0/1446843914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb55d0f640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcc66b0 0x556f8e098450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 -- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcc66b0 msgr2=0x556f8e098450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcc66b0 0x556f8e098450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 -- 172.21.15.94:0/1446843914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 msgr2=0x556f8e099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e099380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 -- 172.21.15.94:0/1446843914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8e08ed40 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5550e640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800cba0 tx=0x7fdb480a3410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04f16a4ef76c354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a6c00 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdb48018420 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c05d070 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb480a6fa0 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 10 client.18480 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.823+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f8e09aa60 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c05d070 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb480afe60 con 0x556f8dcbe4d0 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 10 client.18480 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 10 client.18480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 20 client.18480 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dc7b3a0 0x556f8e10b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 4 client.18480 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f8dff5800 con 0x556f8dc7b3a0 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dc7b3a0 0x556f8e10b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb38ff9640 10 client.18480 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.824+0000 7fdb54d0d640 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dc7b3a0 0x556f8e10b580 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3a6330e9a30d5c server_cookie=36c39419e952d8fd in_seq=0 out_seq=0 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb38ff9640 1 -- 172.21.15.94:0/1446843914 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f8dff5800 con 0x556f8dc7b3a0 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb38ff9640 10 client.18480 handle_command_reply: tid=0 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 client.18480 shutdown 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 msgr2=0x556f8e097f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800cba0 tx=0x7fdb480a3410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 shutdown_connections 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556f8dc7b3a0 0x556f8e10b580 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8dce7390 0x556f8e099380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb38ff9640 0 client.18480 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f8dcc66b0 0x556f8e098450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 --2- 172.21.15.94:0/1446843914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556f8dcbe4d0 0x556f8e097f10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 >> 172.21.15.94:0/1446843914 conn(0x556f8dcdb4e0 msgr2=0x556f8e087e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 shutdown_connections 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 1 -- 172.21.15.94:0/1446843914 wait complete. 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb5e623740 20 client.18480 trim_cache size 0 max 16384 2024-04-25T13:50:00.834 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 msgr2=0x7fdb3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 0x7fdb3c0b2000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x556f8e08a860 tx=0x7fdb40005f90 comp rx=0 tx=0).stop 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 msgr2=0x7fdb5019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800ed00 tx=0x7fdb4800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 shutdown_connections 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb3c0afaf0 0x7fdb3c0b2000 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50109a40 0x7fdb501a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb50104d40 0x7fdb5019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 --2- 172.21.15.94:0/2018663537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb50103b00 0x7fdb5019a2c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.825+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 >> 172.21.15.94:0/2018663537 conn(0x7fdb500fecf0 msgr2=0x7fdb50105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.826+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 shutdown_connections 2024-04-25T13:50:00.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.826+0000 7fdb57772640 1 -- 172.21.15.94:0/2018663537 wait complete. 2024-04-25T13:50:00.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- 172.21.15.94:0/2429249589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c109770 msgr2=0x7ff95c10bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 --2- 172.21.15.94:0/2429249589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c109770 0x7ff95c10bc80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff954009ca0 tx=0x7ff95402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- 172.21.15.94:0/2429249589 shutdown_connections 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 --2- 172.21.15.94:0/2429249589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c109770 0x7ff95c10bc80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 --2- 172.21.15.94:0/2429249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c106c00 0x7ff95c1090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 --2- 172.21.15.94:0/2429249589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 0x7ff95c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- 172.21.15.94:0/2429249589 >> 172.21.15.94:0/2429249589 conn(0x7ff95c0fd350 msgr2=0x7ff95c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- 172.21.15.94:0/2429249589 shutdown_connections 2024-04-25T13:50:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- 172.21.15.94:0/2429249589 wait complete. 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 Processor -- start 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.952+0000 7ff962764640 1 -- start start 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 0x7ff95c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c109770 0x7ff95c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c10f600 con 0x7ff95c106c00 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c10f480 con 0x7ff95c101e20 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c10f780 con 0x7ff95c109770 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37636/0 (socket says 172.21.15.94:37636) 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 -- 172.21.15.94:0/1971123264 learned_addr learned my addr 172.21.15.94:0/1971123264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95bfff640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 0x7ff95c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff960d01640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c109770 0x7ff95c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 -- 172.21.15.94:0/1971123264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c109770 msgr2=0x7ff95c1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c109770 0x7ff95c1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 -- 172.21.15.94:0/1971123264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 msgr2=0x7ff95c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 0x7ff95c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 -- 172.21.15.94:0/1971123264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c1a1cc0 con 0x7ff95c106c00 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9f0 tx=0x7ff95000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff9597fa640 1 -- 172.21.15.94:0/1971123264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000cd80 con 0x7ff95c106c00 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff9597fa640 1 -- 172.21.15.94:0/1971123264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9500054d0 con 0x7ff95c106c00 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.953+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c1a1fb0 con 0x7ff95c106c00 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.954+0000 7ff9597fa640 1 -- 172.21.15.94:0/1971123264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950010690 con 0x7ff95c106c00 2024-04-25T13:50:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.954+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1a2420 con 0x7ff95c106c00 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 Processor -- start 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff9597fa640 1 -- 172.21.15.94:0/1971123264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff950004e90 con 0x7ff95c106c00 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 -- start start 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cb5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559930c19d80 con 0x5599308f26b0 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559930bd3a20 con 0x559930913390 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff969612740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559930c15d40 con 0x5599308ea4d0 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff960d01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37638/0 (socket says 172.21.15.94:37638) 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff95b7fe640 1 -- 172.21.15.94:0/4201505969 learned_addr learned my addr 172.21.15.94:0/4201505969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.956+0000 7ff960d01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:32900/0 (socket says 172.21.15.94:32900) 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95bfff640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cb5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 -- 172.21.15.94:0/4201505969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 msgr2=0x559930cb5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cb5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 -- 172.21.15.94:0/4201505969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 msgr2=0x559930cb5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 -- 172.21.15.94:0/4201505969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559930bdec30 con 0x5599308f26b0 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95bfff640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cb5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff960d01640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff9597fa640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 0x7ff9440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff960d01640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 0x7ff9440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff9500e2fd0 tx=0x7ff9500e7220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69176c63dd3e34f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff9597fa640 1 -- 172.21.15.94:0/1971123264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff9500df0d0 con 0x7ff95c106c00 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff9427fc640 1 -- 172.21.15.94:0/4201505969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500a33d0 con 0x5599308f26b0 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff9427fc640 1 -- 172.21.15.94:0/4201505969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9500a7020 con 0x5599308f26b0 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.957+0000 7ff9427fc640 1 -- 172.21.15.94:0/4201505969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950005180 con 0x5599308f26b0 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- 172.21.15.94:0/4201505969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 msgr2=0x559930cb6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff9500e2fd0 tx=0x7ff9500e7220 comp rx=0 tx=0).stop 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff960d01640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 0x7ff9440b2000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559930cb6860 tx=0x7ff95402f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- 172.21.15.94:0/4201505969 shutdown_connections 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308f26b0 0x559930cb6a20 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cb5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 --2- 172.21.15.94:0/4201505969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559930913390 0x559930cb5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- 172.21.15.94:0/4201505969 >> 172.21.15.94:0/4201505969 conn(0x5599309074e0 msgr2=0x559930cb1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- 172.21.15.94:0/4201505969 shutdown_connections 2024-04-25T13:50:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- 172.21.15.94:0/4201505969 wait complete. 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.958+0000 7ff969612740 1 -- start start 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 Processor -- start 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cbf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cbfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559930cb7530 con 0x559930913390 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559930ba0dc0 con 0x5599308f26b0 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559930cb76b0 con 0x5599308ea4d0 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff969612740 10 client.0 _ll_register_callbacks cb 0x559930bbc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37652/0 (socket says 172.21.15.94:37652) 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cbfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 -- 172.21.15.94:0/2932942347 learned_addr learned my addr 172.21.15.94:0/2932942347 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff95bfff640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cbf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 -- 172.21.15.94:0/2932942347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 msgr2=0x559930cbf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cbf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 -- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 msgr2=0x559930cbfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cbfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff960d01640 1 -- 172.21.15.94:0/2932942347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993092b3f0 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.959+0000 7ff95bfff640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cbf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff960d01640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff954035000 tx=0x7ff95402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc9b162d21bc64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9540043d0 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff954004570 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9500a7020 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff954004770 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 10 client.18492 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599309f57d0 con 0x559930913390 2024-04-25T13:50:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9500a7020 con 0x559930913390 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff954051070 con 0x559930913390 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 10 client.18492 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 10 client.18492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 20 client.18492 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5599308d1f60 0x559930ccd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 4 client.18492 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.960+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559930c27990 con 0x5599308d1f60 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.961+0000 7ff95bfff640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5599308d1f60 0x559930ccd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.961+0000 7ff93b7fe640 10 client.18492 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.961+0000 7ff95bfff640 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5599308d1f60 0x559930ccd330 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=beca74c1c047d6f9 server_cookie=f980f6a71d82d4fc in_seq=0 out_seq=0 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff93b7fe640 1 -- 172.21.15.94:0/2932942347 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559930c27990 con 0x5599308d1f60 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff93b7fe640 10 client.18492 handle_command_reply: tid=0 2024-04-25T13:50:00.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 client.18492 shutdown 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 msgr2=0x559930cc6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff954035000 tx=0x7ff95402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 shutdown_connections 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5599308d1f60 0x559930ccd330 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559930913390 0x559930cc6a50 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cbfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff93b7fe640 0 client.18492 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 --2- 172.21.15.94:0/2932942347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cbf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 >> 172.21.15.94:0/2932942347 conn(0x5599309074e0 msgr2=0x559930cb3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 shutdown_connections 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 1 -- 172.21.15.94:0/2932942347 wait complete. 2024-04-25T13:50:00.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.962+0000 7ff969612740 20 client.18492 trim_cache size 0 max 16384 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 Processor -- start 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 -- start start 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ab460 con 0x5599308d1f60 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1acfa0 con 0x5599308f26b0 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff969612740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1abb30 con 0x5599308ea4d0 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37668/0 (socket says 172.21.15.94:37668) 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff95bfff640 1 -- 172.21.15.94:0/3378381129 learned_addr learned my addr 172.21.15.94:0/3378381129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff960d01640 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.976+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 -- 172.21.15.94:0/3378381129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 msgr2=0x559930cba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 -- 172.21.15.94:0/3378381129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 msgr2=0x559930cc6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 -- 172.21.15.94:0/3378381129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559930cb9dc0 con 0x5599308d1f60 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff95bfff640 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000c00 tx=0x7ff94c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c10f01967d6601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff941ffb640 1 -- 172.21.15.94:0/3378381129 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c003d20 con 0x5599308d1f60 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff941ffb640 1 -- 172.21.15.94:0/3378381129 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94c01f5c0 con 0x5599308d1f60 2024-04-25T13:50:00.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.977+0000 7ff941ffb640 1 -- 172.21.15.94:0/3378381129 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c01d900 con 0x5599308d1f60 2024-04-25T13:50:00.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- 172.21.15.94:0/3378381129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 msgr2=0x559930cba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000c00 tx=0x7ff94c004410 comp rx=0 tx=0).stop 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- 172.21.15.94:0/3378381129 shutdown_connections 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- 172.21.15.94:0/3378381129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cba2d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- 172.21.15.94:0/3378381129 >> 172.21.15.94:0/3378381129 conn(0x5599309074e0 msgr2=0x559930cb5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- 172.21.15.94:0/3378381129 shutdown_connections 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- 172.21.15.94:0/3378381129 wait complete. 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 -- start start 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 Processor -- start 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cc4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.978+0000 7ff969612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ac8d0 con 0x5599308d1f60 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ac200 con 0x5599308f26b0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1a9440 con 0x5599308ea4d0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 10 client.0 _ll_register_callbacks cb 0x7ff954009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37684/0 (socket says 172.21.15.94:37684) 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 -- 172.21.15.94:0/877034998 learned_addr learned my addr 172.21.15.94:0/877034998 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cc4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff960d01640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 -- 172.21.15.94:0/877034998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 msgr2=0x559930cc4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cc4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 -- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 msgr2=0x559930cc4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 -- 172.21.15.94:0/877034998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559930cc6eb0 con 0x5599308d1f60 2024-04-25T13:50:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff95bfff640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff94c01d4e0 tx=0x7ff94c009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584f8427b79e0cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff93b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c02c070 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7ff94c003d20 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94c01f5c0 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff94c038050 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 10 client.18498 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.979+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559930cc6610 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94c01f5c0 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff94c004730 con 0x5599308d1f60 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 10 client.18498 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 10 client.18498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 20 client.18498 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559930913390 0x559930d37770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 4 client.18498 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559930c21800 con 0x559930913390 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559930913390 0x559930d37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff93b7fe640 10 client.18498 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.980+0000 7ff95b7fe640 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559930913390 0x559930d37770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=876b1822f2513991 server_cookie=13dd5af88389cb5c in_seq=0 out_seq=0 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff93b7fe640 1 -- 172.21.15.94:0/877034998 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559930c21800 con 0x559930913390 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff93b7fe640 10 client.18498 handle_command_reply: tid=0 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 client.18498 shutdown 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 msgr2=0x559930cc3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff94c01d4e0 tx=0x7ff94c009db0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 shutdown_connections 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559930913390 0x559930d37770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599308f26b0 0x559930cc4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599308ea4d0 0x559930cc4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 --2- 172.21.15.94:0/877034998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5599308d1f60 0x559930cc3b30 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff93b7fe640 0 client.18498 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 >> 172.21.15.94:0/877034998 conn(0x5599309074e0 msgr2=0x559930cb5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 shutdown_connections 2024-04-25T13:50:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 1 -- 172.21.15.94:0/877034998 wait complete. 2024-04-25T13:50:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.981+0000 7ff969612740 20 client.18498 trim_cache size 0 max 16384 2024-04-25T13:50:00.987 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 msgr2=0x7ff9440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 0x7ff9440b2000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559930cb6860 tx=0x7ff95402f310 comp rx=0 tx=0).stop 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 msgr2=0x7ff95c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e9f0 tx=0x7ff95000edf0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 shutdown_connections 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9440afaf0 0x7ff9440b2000 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff95c109770 0x7ff95c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff95c106c00 0x7ff95c19a6c0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 --2- 172.21.15.94:0/1971123264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff95c101e20 0x7ff95c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 >> 172.21.15.94:0/1971123264 conn(0x7ff95c0fd350 msgr2=0x7ff95c107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 shutdown_connections 2024-04-25T13:50:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:00.982+0000 7ff962764640 1 -- 172.21.15.94:0/1971123264 wait complete. 2024-04-25T13:50:00.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:50:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 -- 172.21.15.94:0/1195652262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 msgr2=0x7fd01c1090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 --2- 172.21.15.94:0/1195652262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c1090e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009900 tx=0x7fd00c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 -- 172.21.15.94:0/1195652262 shutdown_connections 2024-04-25T13:50:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 --2- 172.21.15.94:0/1195652262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 0x7fd01c10bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 --2- 172.21.15.94:0/1195652262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c1090e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 --2- 172.21.15.94:0/1195652262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 0x7fd01c106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 -- 172.21.15.94:0/1195652262 >> 172.21.15.94:0/1195652262 conn(0x7fd01c0fdbd0 msgr2=0x7fd01c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 -- 172.21.15.94:0/1195652262 shutdown_connections 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.109+0000 7fd023d5f640 1 -- 172.21.15.94:0/1195652262 wait complete. 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 Processor -- start 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 -- start start 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 0x7fd01c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 0x7fd01c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10f600 con 0x7fd01c106c60 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10f480 con 0x7fd01c109770 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd023d5f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10f780 con 0x7fd01c104270 2024-04-25T13:50:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37692/0 (socket says 172.21.15.94:37692) 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 -- 172.21.15.94:0/2422572769 learned_addr learned my addr 172.21.15.94:0/2422572769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0222fc640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 0x7fd01c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd021afb640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 0x7fd01c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 -- 172.21.15.94:0/2422572769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 msgr2=0x7fd01c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 0x7fd01c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 -- 172.21.15.94:0/2422572769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 msgr2=0x7fd01c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 0x7fd01c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.110+0000 7fd0212fa640 1 -- 172.21.15.94:0/2422572769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c1a2090 con 0x7fd01c106c60 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd0212fa640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd012ffd640 1 -- 172.21.15.94:0/2422572769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c03f030 con 0x7fd01c106c60 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c1a2320 con 0x7fd01c106c60 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd012ffd640 1 -- 172.21.15.94:0/2422572769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c031910 con 0x7fd01c106c60 2024-04-25T13:50:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd012ffd640 1 -- 172.21.15.94:0/2422572769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c035070 con 0x7fd01c106c60 2024-04-25T13:50:01.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.111+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a2860 con 0x7fd01c106c60 2024-04-25T13:50:01.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.113+0000 7fd02ac0b740 1 Processor -- start 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 -- start start 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd012ffd640 1 -- 172.21.15.94:0/2422572769 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd00c042030 con 0x7fd01c106c60 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a688810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a689a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a5ecd80 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a5a6a20 con 0x55fa5a2e6390 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a5e8d40 con 0x55fa5a2c56b0 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd0222fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a688810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37706/0 (socket says 172.21.15.94:37706) 2024-04-25T13:50:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd0212fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a689a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 -- 172.21.15.94:0/2248243368 learned_addr learned my addr 172.21.15.94:0/2248243368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 -- 172.21.15.94:0/2248243368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 msgr2=0x55fa5a689a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a689a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 -- 172.21.15.94:0/2248243368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 msgr2=0x55fa5a688810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a688810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 -- 172.21.15.94:0/2248243368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5a5b1c30 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd0222fc640 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a688810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd0212fa640 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a689a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd012ffd640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 0x7fd0080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd021afb640 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd018009ca0 tx=0x7fd018011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b486a19713b1c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.114+0000 7fd0222fc640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 0x7fd0080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.115+0000 7fd012ffd640 1 -- 172.21.15.94:0/2422572769 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd00c0ff180 con 0x7fd01c106c60 2024-04-25T13:50:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.115+0000 7fd007fff640 1 -- 172.21.15.94:0/2248243368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01801e070 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.115+0000 7fd007fff640 1 -- 172.21.15.94:0/2248243368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018019ae0 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.115+0000 7fd007fff640 1 -- 172.21.15.94:0/2248243368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800e950 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- 172.21.15.94:0/2248243368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 msgr2=0x55fa5a688d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd0222fc640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 0x7fd0080b2000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55fa5a689860 tx=0x7fd014009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd018009ca0 tx=0x7fd018011810 comp rx=0 tx=0).stop 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- 172.21.15.94:0/2248243368 shutdown_connections 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a689a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a688d90 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/2248243368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a688810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- 172.21.15.94:0/2248243368 >> 172.21.15.94:0/2248243368 conn(0x55fa5a2da4e0 msgr2=0x55fa5a684020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- 172.21.15.94:0/2248243368 shutdown_connections 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- 172.21.15.94:0/2248243368 wait complete. 2024-04-25T13:50:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 -- start start 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.116+0000 7fd02ac0b740 1 Processor -- start 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a692510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a692a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a68a530 con 0x55fa5a2e6390 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a573dc0 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa5a68a6b0 con 0x55fa5a2c56b0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd02ac0b740 10 client.0 _ll_register_callbacks cb 0x55fa5a58f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd021afb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a692510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37710/0 (socket says 172.21.15.94:37710) 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 -- 172.21.15.94:0/4068264826 learned_addr learned my addr 172.21.15.94:0/4068264826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0212fa640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a692a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 -- 172.21.15.94:0/4068264826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 msgr2=0x55fa5a692a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a692a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 -- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 msgr2=0x55fa5a692510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a692510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 -- 172.21.15.94:0/4068264826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5a2fe3f0 con 0x55fa5a2e6390 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0212fa640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a692a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd021afb640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a692510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0222fc640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd014003c40 tx=0x7fd014004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f278b809dd161bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.117+0000 7fd0057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01400fe60 con 0x55fa5a2e6390 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd0140045c0 con 0x55fa5a2e6390 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd018019ae0 con 0x55fa5a2e6390 2024-04-25T13:50:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0140047c0 con 0x55fa5a2e6390 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 10 client.18510 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa5a3c87d0 con 0x55fa5a2e6390 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd018019ae0 con 0x55fa5a2e6390 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd014026070 con 0x55fa5a2e6390 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 10 client.18510 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 10 client.18510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 20 client.18510 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a2a4f60 0x55fa5a6a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 4 client.18510 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5a5fa990 con 0x55fa5a2a4f60 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.118+0000 7fd021afb640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a2a4f60 0x55fa5a6a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.119+0000 7fd021afb640 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a2a4f60 0x55fa5a6a04c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd900b16a1531b59 server_cookie=e062786fc01b9707 in_seq=0 out_seq=0 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.119+0000 7fd0057fa640 10 client.18510 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.120+0000 7fd0057fa640 1 -- 172.21.15.94:0/4068264826 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa5a5fa990 con 0x55fa5a2a4f60 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.120+0000 7fd0057fa640 10 client.18510 handle_command_reply: tid=0 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.120+0000 7fd02ac0b740 1 client.18510 shutdown 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 msgr2=0x55fa5a6999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd014003c40 tx=0x7fd014004ab0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 shutdown_connections 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a2a4f60 0x55fa5a6a04c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a6999b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2c56b0 0x55fa5a692a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd0057fa640 0 client.18510 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/4068264826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a692510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 >> 172.21.15.94:0/4068264826 conn(0x55fa5a2da4e0 msgr2=0x55fa5a686f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 shutdown_connections 2024-04-25T13:50:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 1 -- 172.21.15.94:0/4068264826 wait complete. 2024-04-25T13:50:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.121+0000 7fd02ac0b740 20 client.18510 trim_cache size 0 max 16384 2024-04-25T13:50:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.135+0000 7fd02ac0b740 1 Processor -- start 2024-04-25T13:50:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 -- start start 2024-04-25T13:50:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a6839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2c56b0 0x55fa5a6998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ab500 con 0x55fa5a2e6390 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ad040 con 0x55fa5a2c56b0 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1abbd0 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37722/0 (socket says 172.21.15.94:37722) 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 -- 172.21.15.94:0/1459749103 learned_addr learned my addr 172.21.15.94:0/1459749103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd021afb640 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a6839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0212fa640 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2c56b0 0x55fa5a6998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 -- 172.21.15.94:0/1459749103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 msgr2=0x55fa5a6839e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a6839e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 -- 172.21.15.94:0/1459749103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2c56b0 msgr2=0x55fa5a6998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2c56b0 0x55fa5a6998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 -- 172.21.15.94:0/1459749103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5a699e10 con 0x55fa5a2e6390 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0222fc640 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd01401e5b0 tx=0x7fd0140114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328163774f4d9e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0077fe640 1 -- 172.21.15.94:0/1459749103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014035020 con 0x55fa5a2e6390 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.136+0000 7fd0077fe640 1 -- 172.21.15.94:0/1459749103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd014004610 con 0x55fa5a2e6390 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd0077fe640 1 -- 172.21.15.94:0/1459749103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014028840 con 0x55fa5a2e6390 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1459749103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 msgr2=0x55fa5a683f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd01401e5b0 tx=0x7fd0140114a0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1459749103 shutdown_connections 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2e6390 0x55fa5a683f20 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2c56b0 0x55fa5a6998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1459749103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a6839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1459749103 >> 172.21.15.94:0/1459749103 conn(0x55fa5a2da4e0 msgr2=0x55fa5a68d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.137+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1459749103 shutdown_connections 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1459749103 wait complete. 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 -- start start 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 Processor -- start 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a696670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a706820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ac970 con 0x55fa5a2c56b0 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ac2a0 con 0x55fa5a2e6390 2024-04-25T13:50:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a94e0 con 0x55fa5a2bd4d0 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd0212fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd02ac0b740 10 client.0 _ll_register_callbacks cb 0x7fd014009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd0212fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37730/0 (socket says 172.21.15.94:37730) 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.138+0000 7fd0212fa640 1 -- 172.21.15.94:0/1064674815 learned_addr learned my addr 172.21.15.94:0/1064674815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0222fc640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a706820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd021afb640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a696670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 -- 172.21.15.94:0/1064674815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 msgr2=0x55fa5a696670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a696670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 -- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 msgr2=0x55fa5a706820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a706820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 -- 172.21.15.94:0/1064674815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5a699b70 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0212fa640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0c48e0 tx=0x7fd00c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ce6aeed064f418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c049040 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd00c0c3c60 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd014004610 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd00c0c6fc0 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd02ac0b740 10 client.18516 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa5a707e10 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.139+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd014004610 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd00c0c3900 con 0x55fa5a2c56b0 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 10 client.18516 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 10 client.18516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 20 client.18516 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a27a3a0 0x55fa5a70a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 4 client.18516 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa5a5f4800 con 0x55fa5a27a3a0 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd021afb640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a27a3a0 0x55fa5a70a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd0057fa640 10 client.18516 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.140+0000 7fd021afb640 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a27a3a0 0x55fa5a70a830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25c43a8254e40e3 server_cookie=c1958d487c8a4784 in_seq=0 out_seq=0 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd0057fa640 1 -- 172.21.15.94:0/1064674815 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa5a5f4800 con 0x55fa5a27a3a0 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd0057fa640 10 client.18516 handle_command_reply: tid=0 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 client.18516 shutdown 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 msgr2=0x55fa5a696bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0c48e0 tx=0x7fd00c009620 comp rx=0 tx=0).stop 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 shutdown_connections 2024-04-25T13:50:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fa5a27a3a0 0x55fa5a70a830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa5a2e6390 0x55fa5a706820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd0057fa640 0 client.18516 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa5a2c56b0 0x55fa5a696bb0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 --2- 172.21.15.94:0/1064674815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa5a2bd4d0 0x55fa5a696670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 >> 172.21.15.94:0/1064674815 conn(0x55fa5a2da4e0 msgr2=0x55fa5a686ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 shutdown_connections 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 1 -- 172.21.15.94:0/1064674815 wait complete. 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd02ac0b740 20 client.18516 trim_cache size 0 max 16384 2024-04-25T13:50:01.149 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:01.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 msgr2=0x7fd0080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 0x7fd0080b2000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55fa5a689860 tx=0x7fd014009500 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 msgr2=0x7fd01c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 shutdown_connections 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0080afaf0 0x7fd0080b2000 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c109770 0x7fd01c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd01c106c60 0x7fd01c19aa90 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.141+0000 7fd023d5f640 1 --2- 172.21.15.94:0/2422572769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd01c104270 0x7fd01c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.142+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 >> 172.21.15.94:0/2422572769 conn(0x7fd01c0fdbd0 msgr2=0x7fd01c1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.142+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 shutdown_connections 2024-04-25T13:50:01.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:01.142+0000 7fd023d5f640 1 -- 172.21.15.94:0/2422572769 wait complete. 2024-04-25T13:50:01.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:01 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1271514621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:01.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:01 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1271514621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:01.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:01 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1271514621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:01.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 984648 12.94 MB/sec execute 284 sec latency 17.549 ms 2024-04-25T13:50:02.154 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:50:02.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:02.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:50:02.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 -- 172.21.15.94:0/726433506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 msgr2=0x7fe8601090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 --2- 172.21.15.94:0/726433506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe8601090e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe84802a410 comp rx=0 tx=0).stop 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 -- 172.21.15.94:0/726433506 shutdown_connections 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 --2- 172.21.15.94:0/726433506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 0x7fe86010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 --2- 172.21.15.94:0/726433506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe8601090e0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 --2- 172.21.15.94:0/726433506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 -- 172.21.15.94:0/726433506 >> 172.21.15.94:0/726433506 conn(0x7fe8600fdbd0 msgr2=0x7fe860100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 -- 172.21.15.94:0/726433506 shutdown_connections 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.307+0000 7fe865213640 1 -- 172.21.15.94:0/726433506 wait complete. 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 Processor -- start 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 -- start start 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 0x7fe8601026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe86010f5e0 con 0x7fe860106c60 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe86010f460 con 0x7fe860109770 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe865213640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe86010f760 con 0x7fe860104270 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe857fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe857fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37752/0 (socket says 172.21.15.94:37752) 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe857fff640 1 -- 172.21.15.94:0/3813765802 learned_addr learned my addr 172.21.15.94:0/3813765802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.308+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 0x7fe8601026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 -- 172.21.15.94:0/3813765802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 msgr2=0x7fe860103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 -- 172.21.15.94:0/3813765802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 msgr2=0x7fe8601026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 0x7fe8601026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 -- 172.21.15.94:0/3813765802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe860102df0 con 0x7fe860106c60 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe857fff640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe848006010 tx=0x7fe848004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe85cd99640 1 -- 172.21.15.94:0/3813765802 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84803f030 con 0x7fe860106c60 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8601a4630 con 0x7fe860106c60 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe85cd99640 1 -- 172.21.15.94:0/3813765802 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe848003cb0 con 0x7fe860106c60 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe85cd99640 1 -- 172.21.15.94:0/3813765802 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848035070 con 0x7fe860106c60 2024-04-25T13:50:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.309+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8601a4a30 con 0x7fe860106c60 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.311+0000 7fe85cd99640 1 -- 172.21.15.94:0/3813765802 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe848042030 con 0x7fe860106c60 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 Processor -- start 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 -- start start 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd6e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85cd99640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 0x7fe84c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 0x55b13fd6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b13fcd3b00 con 0x55b13f9ac730 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b13fc6f920 con 0x55b13f9a4550 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe86c19f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b13fccfac0 con 0x55b13f9cd410 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 0x7fe84c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe857fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 0x55b13fd6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37766/0 (socket says 172.21.15.94:37766) 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85f59e640 1 -- 172.21.15.94:0/1935802626 learned_addr learned my addr 172.21.15.94:0/1935802626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85cd99640 1 -- 172.21.15.94:0/3813765802 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe8480ff2d0 con 0x7fe860106c60 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.312+0000 7fe85f59e640 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd6e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 -- 172.21.15.94:0/1935802626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 msgr2=0x55b13fd6e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd6e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 -- 172.21.15.94:0/1935802626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 msgr2=0x55b13fd75920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 -- 172.21.15.94:0/1935802626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13faad7e0 con 0x55b13f9a4550 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe85f59e640 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 0x7fe84c0b2000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe850005d00 tx=0x7fe85000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe857fff640 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 0x55b13fd6ea20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c74c0 tx=0x7fe848076920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fae67d147b4d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe8557fa640 1 -- 172.21.15.94:0/1935802626 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848049070 con 0x55b13f9a4550 2024-04-25T13:50:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe8557fa640 1 -- 172.21.15.94:0/1935802626 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe848003e60 con 0x55b13f9a4550 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.313+0000 7fe8557fa640 1 -- 172.21.15.94:0/1935802626 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8480eb3a0 con 0x55b13f9a4550 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 -- 172.21.15.94:0/1935802626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 msgr2=0x55b13fd6ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 0x55b13fd6ea20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c74c0 tx=0x7fe848076920 comp rx=0 tx=0).stop 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 -- 172.21.15.94:0/1935802626 shutdown_connections 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9ac730 0x55b13fd75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9a4550 0x55b13fd6ea20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 --2- 172.21.15.94:0/1935802626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd6e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 -- 172.21.15.94:0/1935802626 >> 172.21.15.94:0/1935802626 conn(0x55b13f9c1560 msgr2=0x55b13fd69e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 -- 172.21.15.94:0/1935802626 shutdown_connections 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.314+0000 7fe86c19f740 1 -- 172.21.15.94:0/1935802626 wait complete. 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 -- start start 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 Processor -- start 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd77fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b13fc6f920 con 0x55b13f9cd410 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b13fc6d5d0 con 0x55b13f9ac730 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b13fd76130 con 0x55b13f9a4550 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe86c19f740 10 client.0 _ll_register_callbacks cb 0x55b13fc5b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37768/0 (socket says 172.21.15.94:37768) 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 -- 172.21.15.94:0/2939192557 learned_addr learned my addr 172.21.15.94:0/2939192557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe857fff640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd77fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 -- 172.21.15.94:0/2939192557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 msgr2=0x55b13fd77fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd77fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.315+0000 7fe85f59e640 1 -- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 msgr2=0x55b13fd784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe85f59e640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd784e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe85f59e640 1 -- 172.21.15.94:0/2939192557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13fb0df70 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd77fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe85f59e640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe858012ca0 tx=0x7fe8580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18de282edf04cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858003970 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe858003b10 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b13fce1710 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe858028050 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe86c19f740 10 client.18528 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b13f9a5170 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b13fce1710 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.316+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe858010430 con 0x55b13f9cd410 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 10 client.18528 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 20 client.18528 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f98c1f0 0x55b13fd85a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 4 client.18528 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13fcdb580 con 0x55b13f98c1f0 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f98c1f0 0x55b13fd85a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f98c1f0 0x55b13fd85a00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ee93e26aa66172 server_cookie=f7614bde4bf699b4 in_seq=0 out_seq=0 2024-04-25T13:50:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.317+0000 7fe8467fc640 10 client.18528 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe8467fc640 1 -- 172.21.15.94:0/2939192557 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b13fcdb580 con 0x55b13f98c1f0 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe8467fc640 10 client.18528 handle_command_reply: tid=0 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 client.18528 shutdown 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 msgr2=0x55b13fd7f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe858012ca0 tx=0x7fe8580117b0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 shutdown_connections 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f98c1f0 0x55b13fd85a00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd7f440 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe8467fc640 0 client.18528 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 --2- 172.21.15.94:0/2939192557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd77fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 >> 172.21.15.94:0/2939192557 conn(0x55b13f9c1560 msgr2=0x55b13fd68ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 shutdown_connections 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 1 -- 172.21.15.94:0/2939192557 wait complete. 2024-04-25T13:50:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.319+0000 7fe86c19f740 20 client.18528 trim_cache size 0 max 16384 2024-04-25T13:50:02.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 Processor -- start 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 -- start start 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd69d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ad6f0 con 0x55b13f9cd410 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601af230 con 0x55b13f9ac730 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe86c19f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601addc0 con 0x55b13f9a4550 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37780/0 (socket says 172.21.15.94:37780) 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 -- 172.21.15.94:0/3631714256 learned_addr learned my addr 172.21.15.94:0/3631714256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe857fff640 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd69d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 -- 172.21.15.94:0/3631714256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 msgr2=0x55b13fd69d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd69d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 -- 172.21.15.94:0/3631714256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 msgr2=0x55b13fd7f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.338+0000 7fe85f59e640 1 -- 172.21.15.94:0/3631714256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13fd731b0 con 0x55b13f9cd410 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.339+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe858006010 tx=0x7fe8580096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad39ecfeb73b8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.339+0000 7fe854ff9640 1 -- 172.21.15.94:0/3631714256 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858010430 con 0x55b13f9cd410 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.339+0000 7fe854ff9640 1 -- 172.21.15.94:0/3631714256 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe858016040 con 0x55b13f9cd410 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.339+0000 7fe854ff9640 1 -- 172.21.15.94:0/3631714256 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858003e40 con 0x55b13f9cd410 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3631714256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 msgr2=0x55b13fd6cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe858006010 tx=0x7fe8580096a0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3631714256 shutdown_connections 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9cd410 0x55b13fd6cc30 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3631714256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9a4550 0x55b13fd69d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3631714256 >> 172.21.15.94:0/3631714256 conn(0x55b13f9c1560 msgr2=0x55b13fd69620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3631714256 shutdown_connections 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3631714256 wait complete. 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- start start 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 Processor -- start 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd7e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe85ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601aeb60 con 0x55b13f9a4550 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe85ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37792/0 (socket says 172.21.15.94:37792) 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe85ed9d640 1 -- 172.21.15.94:0/3964909075 learned_addr learned my addr 172.21.15.94:0/3964909075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ae490 con 0x55b13f9ac730 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.340+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ab6d0 con 0x55b13f9cd410 2024-04-25T13:50:02.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe86c19f740 10 client.0 _ll_register_callbacks cb 0x7fe858012970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe857fff640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd7e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 -- 172.21.15.94:0/3964909075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 msgr2=0x55b13fd7e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd7e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 -- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 msgr2=0x55b13fd7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 -- 172.21.15.94:0/3964909075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13fd7e7c0 con 0x55b13f9a4550 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85f59e640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd7e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe85ed9d640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8500117b0 tx=0x7fe850004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e9d48d915c3395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe8467fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850004020 con 0x55b13f9a4550 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe858016040 con 0x55b13f9a4550 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe8500041c0 con 0x55b13f9a4550 2024-04-25T13:50:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8500043c0 con 0x55b13f9a4550 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe86c19f740 10 client.18534 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b13fdee020 con 0x55b13f9a4550 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.341+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe858016040 con 0x55b13f9a4550 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe85002f070 con 0x55b13f9a4550 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 10 client.18534 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 20 client.18534 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f961630 0x55b13fdf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 4 client.18534 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b13fc56ac0 con 0x55b13f961630 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe857fff640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f961630 0x55b13fdf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe8467fc640 10 client.18534 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.342+0000 7fe857fff640 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f961630 0x55b13fdf0a40 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50cfff2e4bb5e86f server_cookie=c7184bfafbeac444 in_seq=0 out_seq=0 2024-04-25T13:50:02.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.344+0000 7fe8467fc640 1 -- 172.21.15.94:0/3964909075 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b13fc56ac0 con 0x55b13f961630 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.344+0000 7fe8467fc640 10 client.18534 handle_command_reply: tid=0 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.344+0000 7fe86c19f740 1 client.18534 shutdown 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 msgr2=0x55b13fd7ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8500117b0 tx=0x7fe850004730 comp rx=0 tx=0).stop 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 shutdown_connections 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b13f961630 0x55b13fdf0a40 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b13f9cd410 0x55b13fd7e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe8467fc640 0 client.18534 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b13f9ac730 0x55b13fd7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 --2- 172.21.15.94:0/3964909075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b13f9a4550 0x55b13fd7ce00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 >> 172.21.15.94:0/3964909075 conn(0x55b13f9c1560 msgr2=0x55b13fd6cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 shutdown_connections 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 1 -- 172.21.15.94:0/3964909075 wait complete. 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe86c19f740 20 client.18534 trim_cache size 0 max 16384 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "465c4196-4b0f-4bdc-88b8-7ffd10309e32", 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:50:02.350 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 msgr2=0x7fe84c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 0x7fe84c0b2000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe850005d00 tx=0x7fe85000b410 comp rx=0 tx=0).stop 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 msgr2=0x7fe860102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe848006010 tx=0x7fe848004340 comp rx=0 tx=0).stop 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 shutdown_connections 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe84c0afaf0 0x7fe84c0b2000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe860109770 0x7fe8601026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe860106c60 0x7fe860102180 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 --2- 172.21.15.94:0/3813765802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860104270 0x7fe860103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 >> 172.21.15.94:0/3813765802 conn(0x7fe8600fdbd0 msgr2=0x7fe8601054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.345+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 shutdown_connections 2024-04-25T13:50:02.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.346+0000 7fe865213640 1 -- 172.21.15.94:0/3813765802 wait complete. 2024-04-25T13:50:02.355 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:02 smithi094 ceph-mon[29195]: pgmap v594: 97 pgs: 97 active+clean; 32 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-04-25T13:50:02.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:50:02.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:50:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:02 smithi137 ceph-mon[32568]: pgmap v594: 97 pgs: 97 active+clean; 32 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 -- 172.21.15.94:0/2448997477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c81095c0 msgr2=0x7f47c810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/2448997477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c81095c0 0x7f47c810bad0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ca0 tx=0x7f47c002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 -- 172.21.15.94:0/2448997477 shutdown_connections 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/2448997477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c81095c0 0x7f47c810bad0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/2448997477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c8106a50 0x7f47c8108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/2448997477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 0x7f47c8106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 -- 172.21.15.94:0/2448997477 >> 172.21.15.94:0/2448997477 conn(0x7f47c80fda40 msgr2=0x7f47c80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 -- 172.21.15.94:0/2448997477 shutdown_connections 2024-04-25T13:50:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.472+0000 7f47cc9fe640 1 -- 172.21.15.94:0/2448997477 wait complete. 2024-04-25T13:50:02.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.473+0000 7f47cc9fe640 1 Processor -- start 2024-04-25T13:50:02.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.473+0000 7f47cc9fe640 1 -- start start 2024-04-25T13:50:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.473+0000 7f47cc9fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 0x7f47c8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c81095c0 0x7f47c81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47c810f430 con 0x7f47c8106a50 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47c810f2b0 con 0x7f47c81095c0 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c810f5b0 con 0x7f47c8104060 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37808/0 (socket says 172.21.15.94:37808) 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 -- 172.21.15.94:0/278693016 learned_addr learned my addr 172.21.15.94:0/278693016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c759e640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c81095c0 0x7f47c81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 0x7f47c8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 -- 172.21.15.94:0/278693016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 msgr2=0x7f47c8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 0x7f47c8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 -- 172.21.15.94:0/278693016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c81095c0 msgr2=0x7f47c81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c81095c0 0x7f47c81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 -- 172.21.15.94:0/278693016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81a1af0 con 0x7f47c8106a50 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47c659c640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9f0 tx=0x7f47bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47affff640 1 -- 172.21.15.94:0/278693016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cd80 con 0x7f47c8106a50 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47affff640 1 -- 172.21.15.94:0/278693016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0108e0 con 0x7f47c8106a50 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81a1de0 con 0x7f47c8106a50 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.474+0000 7f47affff640 1 -- 172.21.15.94:0/278693016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc018980 con 0x7f47c8106a50 2024-04-25T13:50:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.475+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a2250 con 0x7f47c8106a50 2024-04-25T13:50:02.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47affff640 1 -- 172.21.15.94:0/278693016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f47bc018b20 con 0x7f47c8106a50 2024-04-25T13:50:02.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 Processor -- start 2024-04-25T13:50:02.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 -- start start 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2d0390 0x556d5c671c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 0x556d5c672e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556d5c5d6d80 con 0x556d5c2a74d0 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d5c5908b0 con 0x556d5c2af6b0 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47d418c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556d5c5d2d40 con 0x556d5c2d0390 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37810/0 (socket says 172.21.15.94:37810) 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47c759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2d0390 0x556d5c671c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47c6d9d640 1 -- 172.21.15.94:0/191404359 learned_addr learned my addr 172.21.15.94:0/191404359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47c659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 0x556d5c672e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.477+0000 7f47affff640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 0x7f47b00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c759e640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 0x7f47b00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 -- 172.21.15.94:0/191404359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2d0390 msgr2=0x556d5c671c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2d0390 0x556d5c671c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 -- 172.21.15.94:0/191404359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 msgr2=0x556d5c672e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 0x556d5c672e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 -- 172.21.15.94:0/191404359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5c561a40 con 0x556d5c2a74d0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c659c640 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 0x556d5c672e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47affff640 1 -- 172.21.15.94:0/278693016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f47bc0defc0 con 0x7f47c8106a50 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009ca0 tx=0x7f47b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91de7015fdaa0987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47c759e640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 0x7f47b00b1ff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f47c002f500 tx=0x7f47c002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47ad7fa640 1 -- 172.21.15.94:0/191404359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004220 con 0x556d5c2a74d0 2024-04-25T13:50:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47ad7fa640 1 -- 172.21.15.94:0/191404359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b80043c0 con 0x556d5c2a74d0 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.478+0000 7f47ad7fa640 1 -- 172.21.15.94:0/191404359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b80197f0 con 0x556d5c2a74d0 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- 172.21.15.94:0/191404359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 msgr2=0x556d5c6721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009ca0 tx=0x7f47b8011810 comp rx=0 tx=0).stop 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- 172.21.15.94:0/191404359 shutdown_connections 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2af6b0 0x556d5c672e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2a74d0 0x556d5c6721d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 --2- 172.21.15.94:0/191404359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2d0390 0x556d5c671c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- 172.21.15.94:0/191404359 >> 172.21.15.94:0/191404359 conn(0x556d5c2c44e0 msgr2=0x556d5c66d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- 172.21.15.94:0/191404359 shutdown_connections 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- 172.21.15.94:0/191404359 wait complete. 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 -- start start 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.479+0000 7f47d418c740 1 Processor -- start 2024-04-25T13:50:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c67ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c67bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556d5c673970 con 0x556d5c2d0390 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d5c55dd10 con 0x556d5c2a74d0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556d5c673af0 con 0x556d5c2af6b0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47d418c740 10 client.0 _ll_register_callbacks cb 0x556d5c58a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37814/0 (socket says 172.21.15.94:37814) 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c67ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 -- 172.21.15.94:0/3283201042 learned_addr learned my addr 172.21.15.94:0/3283201042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c67bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 -- 172.21.15.94:0/3283201042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 msgr2=0x556d5c67bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c67bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 -- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 msgr2=0x556d5c67ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c67ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 -- 172.21.15.94:0/3283201042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5c3b27d0 con 0x556d5c2d0390 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c67ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c659c640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c67bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47c759e640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f47c0035000 tx=0x7f47c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec46274c32670476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.480+0000 7f47aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c002f660 con 0x556d5c2d0390 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f47c002f800 con 0x556d5c2d0390 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b80043c0 con 0x556d5c2d0390 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47c005c050 con 0x556d5c2d0390 2024-04-25T13:50:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 10 client.18546 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d5c2e83f0 con 0x556d5c2d0390 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b80043c0 con 0x556d5c2d0390 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f47c004a050 con 0x556d5c2d0390 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 10 client.18546 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 10 client.18546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 20 client.18546 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c28ef60 0x556d5c689480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 4 client.18546 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d5c5e4990 con 0x556d5c28ef60 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.481+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c28ef60 0x556d5c689480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.482+0000 7f47aaff5640 10 client.18546 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.482+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c28ef60 0x556d5c689480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc6c32c5d56edc1 server_cookie=41185695c558d115 in_seq=0 out_seq=0 2024-04-25T13:50:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.482+0000 7f47aaff5640 1 -- 172.21.15.94:0/3283201042 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d5c5e4990 con 0x556d5c28ef60 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.482+0000 7f47aaff5640 10 client.18546 handle_command_reply: tid=0 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.482+0000 7f47d418c740 1 client.18546 shutdown 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 msgr2=0x556d5c682f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f47c0035000 tx=0x7f47c00095c0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 shutdown_connections 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c28ef60 0x556d5c689480 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c682f10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c67bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 --2- 172.21.15.94:0/3283201042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c67ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 >> 172.21.15.94:0/3283201042 conn(0x556d5c2c44e0 msgr2=0x556d5c670370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47aaff5640 0 client.18546 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 shutdown_connections 2024-04-25T13:50:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 1 -- 172.21.15.94:0/3283201042 wait complete. 2024-04-25T13:50:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.483+0000 7f47d418c740 20 client.18546 trim_cache size 0 max 16384 2024-04-25T13:50:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:02 smithi138 ceph-mon[32977]: pgmap v594: 97 pgs: 97 active+clean; 32 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 Processor -- start 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 -- start start 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2a74d0 0x556d5c66ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 0x556d5c66d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ab330 con 0x556d5c2af6b0 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ace70 con 0x556d5c2d0390 2024-04-25T13:50:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47d418c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81aba00 con 0x556d5c2a74d0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37830/0 (socket says 172.21.15.94:37830) 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 -- 172.21.15.94:0/1846449990 learned_addr learned my addr 172.21.15.94:0/1846449990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2a74d0 0x556d5c66ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c759e640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 0x556d5c66d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 -- 172.21.15.94:0/1846449990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2a74d0 msgr2=0x556d5c66ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2a74d0 0x556d5c66ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 -- 172.21.15.94:0/1846449990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 msgr2=0x556d5c66d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 0x556d5c66d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 -- 172.21.15.94:0/1846449990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5c2a80f0 con 0x556d5c2af6b0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c759e640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 0x556d5c66d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.502+0000 7f47c659c640 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0a3680 tx=0x7f47bc0a3d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715b664509c29b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47acff9640 1 -- 172.21.15.94:0/1846449990 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0a7110 con 0x556d5c2af6b0 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47acff9640 1 -- 172.21.15.94:0/1846449990 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0b0260 con 0x556d5c2af6b0 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47acff9640 1 -- 172.21.15.94:0/1846449990 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0a6860 con 0x556d5c2af6b0 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 -- 172.21.15.94:0/1846449990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 msgr2=0x556d5c682e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0a3680 tx=0x7f47bc0a3d80 comp rx=0 tx=0).stop 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 -- 172.21.15.94:0/1846449990 shutdown_connections 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2d0390 0x556d5c66d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2af6b0 0x556d5c682e30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 --2- 172.21.15.94:0/1846449990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2a74d0 0x556d5c66ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 -- 172.21.15.94:0/1846449990 >> 172.21.15.94:0/1846449990 conn(0x556d5c2c44e0 msgr2=0x556d5c6767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.503+0000 7f47d418c740 1 -- 172.21.15.94:0/1846449990 shutdown_connections 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 -- 172.21.15.94:0/1846449990 wait complete. 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 -- start start 2024-04-25T13:50:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 Processor -- start 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c6827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c682d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ac7a0 con 0x556d5c2d0390 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ac0d0 con 0x556d5c2a74d0 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81a9310 con 0x556d5c2af6b0 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47d418c740 10 client.0 _ll_register_callbacks cb 0x7f47c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37832/0 (socket says 172.21.15.94:37832) 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 -- 172.21.15.94:0/642757611 learned_addr learned my addr 172.21.15.94:0/642757611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c6827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c659c640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c682d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 -- 172.21.15.94:0/642757611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 msgr2=0x556d5c682d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c682d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 -- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 msgr2=0x556d5c6827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c6827d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.504+0000 7f47c759e640 1 -- 172.21.15.94:0/642757611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5c5974b0 con 0x556d5c2d0390 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47c759e640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ca0 tx=0x7f47c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1def1e6803ae626b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0065060 con 0x556d5c2d0390 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f47c0004d40 con 0x556d5c2d0390 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0b0260 con 0x556d5c2d0390 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47c00305c0 con 0x556d5c2d0390 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 10 client.18552 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d5c6f02c0 con 0x556d5c2d0390 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0b0260 con 0x556d5c2d0390 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f47c004a040 con 0x556d5c2d0390 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 10 client.18552 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 20 client.18552 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c2643a0 0x556d5c6f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 4 client.18552 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.505+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d5c5de800 con 0x556d5c2643a0 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.506+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c2643a0 0x556d5c6f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.506+0000 7f47c6d9d640 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c2643a0 0x556d5c6f2ce0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39946319139b2e96 server_cookie=ef4b5a093ade6931 in_seq=0 out_seq=0 2024-04-25T13:50:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.506+0000 7f47aaff5640 10 client.18552 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47aaff5640 1 -- 172.21.15.94:0/642757611 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556d5c5de800 con 0x556d5c2643a0 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47aaff5640 10 client.18552 handle_command_reply: tid=0 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 client.18552 shutdown 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 msgr2=0x556d5c67f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ca0 tx=0x7f47c0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 shutdown_connections 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556d5c2643a0 0x556d5c6f2ce0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d5c2d0390 0x556d5c67f350 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47aaff5640 0 client.18552 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d5c2af6b0 0x556d5c682d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 --2- 172.21.15.94:0/642757611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d5c2a74d0 0x556d5c6827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 >> 172.21.15.94:0/642757611 conn(0x556d5c2c44e0 msgr2=0x556d5c67a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 shutdown_connections 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 1 -- 172.21.15.94:0/642757611 wait complete. 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.508+0000 7f47d418c740 20 client.18552 trim_cache size 0 max 16384 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:50:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "465c4196-4b0f-4bdc-88b8-7ffd10309e32": { 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "465c4196-4b0f-4bdc-88b8-7ffd10309e32", 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:50:02.516 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 msgr2=0x7f47b00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 0x7f47b00b1ff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f47c002f500 tx=0x7f47c002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 msgr2=0x7f47c819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9f0 tx=0x7f47bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 shutdown_connections 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b00afae0 0x7f47b00b1ff0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c81095c0 0x7f47c81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47c8106a50 0x7f47c819a4f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 --2- 172.21.15.94:0/278693016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c8104060 0x7f47c8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 >> 172.21.15.94:0/278693016 conn(0x7f47c80fda40 msgr2=0x7f47c8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 shutdown_connections 2024-04-25T13:50:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:02.509+0000 7f47cc9fe640 1 -- 172.21.15.94:0/278693016 wait complete. 2024-04-25T13:50:02.520 INFO:tasks.cephfs.filesystem:scrub status for tag:465c4196-4b0f-4bdc-88b8-7ffd10309e32 - {'path': '/', 'tag': '465c4196-4b0f-4bdc-88b8-7ffd10309e32', 'options': 'recursive,force'} 2024-04-25T13:50:02.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 987007 12.94 MB/sec execute 285 sec latency 77.408 ms 2024-04-25T13:50:03.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:03 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:50:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:03 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:50:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:03 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:50:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:03 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:50:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:03 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:50:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:03 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:50:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:03.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 988829 12.93 MB/sec execute 286 sec latency 18.017 ms 2024-04-25T13:50:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:04 smithi137 ceph-mon[32568]: pgmap v595: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-04-25T13:50:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:04 smithi094 ceph-mon[29195]: pgmap v595: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-04-25T13:50:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:04 smithi138 ceph-mon[32977]: pgmap v595: 97 pgs: 97 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 3.1 MiB/s wr, 274 op/s 2024-04-25T13:50:04.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 991638 12.94 MB/sec execute 287 sec latency 8.105 ms 2024-04-25T13:50:05.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 994242 12.94 MB/sec execute 288 sec latency 51.926 ms 2024-04-25T13:50:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:06 smithi137 ceph-mon[32568]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-04-25T13:50:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:06 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:50:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:06 smithi094 ceph-mon[29195]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-04-25T13:50:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:06 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:50:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:06 smithi138 ceph-mon[32977]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-04-25T13:50:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:06 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:50:06.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 996369 12.94 MB/sec execute 289 sec latency 15.004 ms 2024-04-25T13:50:07.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 998706 12.94 MB/sec execute 290 sec latency 106.052 ms 2024-04-25T13:50:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:08 smithi137 ceph-mon[32568]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-04-25T13:50:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:08 smithi094 ceph-mon[29195]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-04-25T13:50:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:08 smithi138 ceph-mon[32977]: pgmap v597: 97 pgs: 97 active+clean; 25 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 3.6 MiB/s wr, 418 op/s 2024-04-25T13:50:08.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1001216 12.94 MB/sec execute 291 sec latency 20.159 ms 2024-04-25T13:50:09.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1003019 12.92 MB/sec execute 292 sec latency 18.296 ms 2024-04-25T13:50:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:10 smithi137 ceph-mon[32568]: pgmap v598: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.5 MiB/s wr, 357 op/s 2024-04-25T13:50:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:10 smithi094 ceph-mon[29195]: pgmap v598: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.5 MiB/s wr, 357 op/s 2024-04-25T13:50:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:10 smithi138 ceph-mon[32977]: pgmap v598: 97 pgs: 97 active+clean; 31 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.5 MiB/s wr, 357 op/s 2024-04-25T13:50:10.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1005805 12.94 MB/sec execute 293 sec latency 13.540 ms 2024-04-25T13:50:11.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1008426 12.94 MB/sec execute 294 sec latency 12.316 ms 2024-04-25T13:50:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:12 smithi137 ceph-mon[32568]: pgmap v599: 97 pgs: 97 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 421 op/s 2024-04-25T13:50:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:12 smithi094 ceph-mon[29195]: pgmap v599: 97 pgs: 97 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 421 op/s 2024-04-25T13:50:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:12 smithi138 ceph-mon[32977]: pgmap v599: 97 pgs: 97 active+clean; 45 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 421 op/s 2024-04-25T13:50:12.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1009841 12.92 MB/sec execute 295 sec latency 89.933 ms 2024-04-25T13:50:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 -- 172.21.15.94:0/2333395752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83581021d0 msgr2=0x7f8358102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 --2- 172.21.15.94:0/2333395752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83581021d0 0x7f8358102ed0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8354009900 tx=0x7f835402a410 comp rx=0 tx=0).stop 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 -- 172.21.15.94:0/2333395752 shutdown_connections 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 --2- 172.21.15.94:0/2333395752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358108110 0x7f835810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 --2- 172.21.15.94:0/2333395752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8358103410 0x7f8358103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 --2- 172.21.15.94:0/2333395752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83581021d0 0x7f8358102ed0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 -- 172.21.15.94:0/2333395752 >> 172.21.15.94:0/2333395752 conn(0x7f83580fd3c0 msgr2=0x7f83580ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 -- 172.21.15.94:0/2333395752 shutdown_connections 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.852+0000 7f836059d640 1 -- 172.21.15.94:0/2333395752 wait complete. 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 Processor -- start 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 -- start start 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83581021d0 0x7f835819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358103410 0x7f835819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f835810e560 con 0x7f8358108110 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f835810e3e0 con 0x7f83581021d0 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f836059d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f835810e6e0 con 0x7f8358103410 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45928/0 (socket says 172.21.15.94:45928) 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f835db38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358103410 0x7f835819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.853+0000 7f835eb3a640 1 -- 172.21.15.94:0/3979202320 learned_addr learned my addr 172.21.15.94:0/3979202320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835e339640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83581021d0 0x7f835819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 -- 172.21.15.94:0/3979202320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358103410 msgr2=0x7f835819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358103410 0x7f835819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 -- 172.21.15.94:0/3979202320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83581021d0 msgr2=0x7f835819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83581021d0 0x7f835819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 -- 172.21.15.94:0/3979202320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83581a1da0 con 0x7f8358108110 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f835000eaa0 tx=0x7f835000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f834f7fe640 1 -- 172.21.15.94:0/3979202320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000cdb0 con 0x7f8358108110 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581a2090 con 0x7f8358108110 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f834f7fe640 1 -- 172.21.15.94:0/3979202320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8350005020 con 0x7f8358108110 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f834f7fe640 1 -- 172.21.15.94:0/3979202320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500116d0 con 0x7f8358108110 2024-04-25T13:50:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.854+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a25e0 con 0x7f8358108110 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f834f7fe640 1 -- 172.21.15.94:0/3979202320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8350011870 con 0x7f8358108110 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 Processor -- start 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 -- start start 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 0x557e4f5e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f21b4d0 0x557e4f5e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e4f542460 con 0x557e4f2132f0 2024-04-25T13:50:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e4f4fc590 con 0x557e4f21b4d0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.857+0000 7f836744e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e4f53e420 con 0x557e4f23c1b0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835e339640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 0x557e4f5e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835e339640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45932/0 (socket says 172.21.15.94:45932) 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835e339640 1 -- 172.21.15.94:0/2195713495 learned_addr learned my addr 172.21.15.94:0/2195713495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f21b4d0 0x557e4f5e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 -- 172.21.15.94:0/2195713495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f21b4d0 msgr2=0x557e4f5e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f21b4d0 0x557e4f5e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 -- 172.21.15.94:0/2195713495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 msgr2=0x557e4f5e1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 -- 172.21.15.94:0/2195713495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4f5090d0 con 0x557e4f23c1b0 2024-04-25T13:50:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f834f7fe640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 0x7f83440b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835e339640 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835e339640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 0x7f83440b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f834f7fe640 1 -- 172.21.15.94:0/3979202320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f83500dedb0 con 0x7f8358108110 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f835db38640 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 0x557e4f5e0d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f835819b300 tx=0x7f8348004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df57f953940783de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.858+0000 7f834cbfe640 1 -- 172.21.15.94:0/2195713495 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348004050 con 0x557e4f23c1b0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.859+0000 7f834cbfe640 1 -- 172.21.15.94:0/2195713495 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f834800fc70 con 0x557e4f23c1b0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.859+0000 7f834cbfe640 1 -- 172.21.15.94:0/2195713495 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348003920 con 0x557e4f23c1b0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- 172.21.15.94:0/2195713495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 msgr2=0x557e4f5e0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f835e339640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 0x7f83440b1ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x557e4f5e2160 tx=0x7f8354000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 0x557e4f5e0d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f835819b300 tx=0x7f8348004730 comp rx=0 tx=0).stop 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- 172.21.15.94:0/2195713495 shutdown_connections 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f21b4d0 0x557e4f5e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f2132f0 0x557e4f5e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 --2- 172.21.15.94:0/2195713495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f23c1b0 0x557e4f5e0d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- 172.21.15.94:0/2195713495 >> 172.21.15.94:0/2195713495 conn(0x557e4f230300 msgr2=0x557e4f5d8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- 172.21.15.94:0/2195713495 shutdown_connections 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- 172.21.15.94:0/2195713495 wait complete. 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 -- start start 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.860+0000 7f836744e740 1 Processor -- start 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557e4f5e8c50 con 0x557e4f23c1b0 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557e4f4c9b60 con 0x557e4f2132f0 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e4f5e8dd0 con 0x557e4f21b4d0 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f836744e740 10 client.0 _ll_register_callbacks cb 0x557e4f4ee2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835e339640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45944/0 (socket says 172.21.15.94:45944) 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835e339640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53838/0 (socket says 172.21.15.94:53838) 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 -- 172.21.15.94:0/3228478921 learned_addr learned my addr 172.21.15.94:0/3228478921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835db38640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 -- 172.21.15.94:0/3228478921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 msgr2=0x557e4f5eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 -- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 msgr2=0x557e4f5eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 -- 172.21.15.94:0/3228478921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4f254210 con 0x557e4f23c1b0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835e339640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f835eb3a640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f83500a7230 tx=0x7f83500a6ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa5965befd1fb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.861+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835001f070 con 0x557e4f23c1b0 2024-04-25T13:50:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f83500054a0 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834800fc70 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83500ab160 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 10 client.18564 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e4f320620 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f834800fc70 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f835000f040 con 0x557e4f23c1b0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 10 client.18564 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 10 client.18564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 20 client.18564 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1fad80 0x557e4f5f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 4 client.18564 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e4f550070 con 0x557e4f1fad80 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.862+0000 7f835e339640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1fad80 0x557e4f5f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.863+0000 7f835e339640 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1fad80 0x557e4f5f8830 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f24e659e2a4ed server_cookie=fc8e78ace8f33e34 in_seq=0 out_seq=0 2024-04-25T13:50:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.863+0000 7f8341ffb640 10 client.18564 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f8341ffb640 1 -- 172.21.15.94:0/3228478921 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e4f550070 con 0x557e4f1fad80 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f8341ffb640 10 client.18564 handle_command_reply: tid=0 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 client.18564 shutdown 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 msgr2=0x557e4f5f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f83500a7230 tx=0x7f83500a6ef0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 shutdown_connections 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1fad80 0x557e4f5f8830 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5f1f80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f8341ffb640 0 client.18564 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 --2- 172.21.15.94:0/3228478921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 >> 172.21.15.94:0/3228478921 conn(0x557e4f230300 msgr2=0x557e4f5db210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 shutdown_connections 2024-04-25T13:50:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 1 -- 172.21.15.94:0/3228478921 wait complete. 2024-04-25T13:50:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.864+0000 7f836744e740 20 client.18564 trim_cache size 0 max 16384 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.894+0000 7f836744e740 1 Processor -- start 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.894+0000 7f836744e740 1 -- start start 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83581a9380 con 0x557e4f23c1b0 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83581aaec0 con 0x557e4f2132f0 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f836744e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83581a9a50 con 0x557e4f21b4d0 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835db38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45960/0 (socket says 172.21.15.94:45960) 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835db38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5f1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49776/0 (socket says 172.21.15.94:49776) 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 -- 172.21.15.94:0/4132083479 learned_addr learned my addr 172.21.15.94:0/4132083479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835e339640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 -- 172.21.15.94:0/4132083479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 msgr2=0x557e4f5f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 -- 172.21.15.94:0/4132083479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 msgr2=0x557e4f5e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 -- 172.21.15.94:0/4132083479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4f5e1f80 con 0x557e4f23c1b0 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f835eb3a640 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f835000cfd0 tx=0x7f83500a3290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c81b9549f736ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.895+0000 7f8343fff640 1 -- 172.21.15.94:0/4132083479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835001f030 con 0x557e4f23c1b0 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f8343fff640 1 -- 172.21.15.94:0/4132083479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83500ab300 con 0x557e4f23c1b0 2024-04-25T13:50:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f8343fff640 1 -- 172.21.15.94:0/4132083479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500a2e70 con 0x557e4f23c1b0 2024-04-25T13:50:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 -- 172.21.15.94:0/4132083479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 msgr2=0x557e4f5e96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f835000cfd0 tx=0x7f83500a3290 comp rx=0 tx=0).stop 2024-04-25T13:50:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 -- 172.21.15.94:0/4132083479 shutdown_connections 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f23c1b0 0x557e4f5e96b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f21b4d0 0x557e4f5f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.896+0000 7f836744e740 1 --2- 172.21.15.94:0/4132083479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f2132f0 0x557e4f5e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- 172.21.15.94:0/4132083479 >> 172.21.15.94:0/4132083479 conn(0x557e4f230300 msgr2=0x557e4f5ead10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- 172.21.15.94:0/4132083479 shutdown_connections 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- 172.21.15.94:0/4132083479 wait complete. 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- start start 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 Processor -- start 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f2132f0 0x557e4f5f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f23c1b0 0x557e4f65ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83581aa7f0 con 0x557e4f21b4d0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83581aa120 con 0x557e4f23c1b0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83581a7360 con 0x557e4f2132f0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f836744e740 10 client.0 _ll_register_callbacks cb 0x7f83500a3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f835db38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f835db38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45962/0 (socket says 172.21.15.94:45962) 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f835db38640 1 -- 172.21.15.94:0/327467603 learned_addr learned my addr 172.21.15.94:0/327467603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.897+0000 7f835eb3a640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f23c1b0 0x557e4f65ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835e339640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f2132f0 0x557e4f5f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 -- 172.21.15.94:0/327467603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f2132f0 msgr2=0x557e4f5f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f2132f0 0x557e4f5f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 -- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f23c1b0 msgr2=0x557e4f65ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f23c1b0 0x557e4f65ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 -- 172.21.15.94:0/327467603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4f65f2f0 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f835db38640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f83480058c0 tx=0x7f8348004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207b4ea37ce2c8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f8341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834800f9a0 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f834800fb40 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83500ab300 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f834800fd40 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f836744e740 10 client.18570 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e4f660560 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.898+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83500ab300 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f834801f4d0 con 0x557e4f21b4d0 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 10 client.18570 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 10 client.18570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 20 client.18570 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1d01c0 0x557e4f662d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 4 client.18570 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:50:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e4f549ee0 con 0x557e4f1d01c0 2024-04-25T13:50:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f835e339640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1d01c0 0x557e4f662d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f8341ffb640 10 client.18570 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:12.899+0000 7f835e339640 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1d01c0 0x557e4f662d80 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69b47e75eb159fb server_cookie=63592ce059bf46e6 in_seq=0 out_seq=0 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f8341ffb640 1 -- 172.21.15.94:0/327467603 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e4f549ee0 con 0x557e4f1d01c0 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f8341ffb640 10 client.18570 handle_command_reply: tid=0 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 client.18570 shutdown 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 msgr2=0x557e4f5f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f83480058c0 tx=0x7f8348004580 comp rx=0 tx=0).stop 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 shutdown_connections 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557e4f1d01c0 0x557e4f662d80 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e4f23c1b0 0x557e4f65ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e4f21b4d0 0x557e4f5f15d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 --2- 172.21.15.94:0/327467603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e4f2132f0 0x557e4f5f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 >> 172.21.15.94:0/327467603 conn(0x557e4f230300 msgr2=0x557e4f5e9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f8341ffb640 0 client.18570 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 shutdown_connections 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 1 -- 172.21.15.94:0/327467603 wait complete. 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.094+0000 7f836744e740 20 client.18570 trim_cache size 0 max 16384 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:50:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:50:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 msgr2=0x7f83440b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 0x7f83440b1ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x557e4f5e2160 tx=0x7f8354000bc0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 msgr2=0x7f83581a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f835000eaa0 tx=0x7f835000ef60 comp rx=0 tx=0).stop 2024-04-25T13:50:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 shutdown_connections 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83440afae0 0x7f83440b1ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8358108110 0x7f83581a1730 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8358103410 0x7f835819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 --2- 172.21.15.94:0/3979202320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83581021d0 0x7f835819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 >> 172.21.15.94:0/3979202320 conn(0x7f83580fd3c0 msgr2=0x7f8358109620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 shutdown_connections 2024-04-25T13:50:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:13.095+0000 7f836059d640 1 -- 172.21.15.94:0/3979202320 wait complete. 2024-04-25T13:50:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:13.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:13 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:50:13.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:13.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:13.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:13.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:13 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:50:13.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:13.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:13.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:13.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:13 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:50:13.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1012455 12.93 MB/sec execute 296 sec latency 102.801 ms 2024-04-25T13:50:14.374 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:14 smithi094 ceph-mon[29195]: pgmap v600: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-25T13:50:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:14 smithi137 ceph-mon[32568]: pgmap v600: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-25T13:50:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:14 smithi138 ceph-mon[32977]: pgmap v600: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-04-25T13:50:14.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1014163 12.91 MB/sec execute 297 sec latency 33.745 ms 2024-04-25T13:50:15.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1016596 12.91 MB/sec execute 298 sec latency 10.977 ms 2024-04-25T13:50:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:16 smithi137 ceph-mon[32568]: pgmap v601: 97 pgs: 97 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-04-25T13:50:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:16 smithi094 ceph-mon[29195]: pgmap v601: 97 pgs: 97 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-04-25T13:50:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:16 smithi138 ceph-mon[32977]: pgmap v601: 97 pgs: 97 active+clean; 43 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 474 op/s 2024-04-25T13:50:16.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1019567 12.93 MB/sec execute 299 sec latency 13.893 ms 2024-04-25T13:50:17.805 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1020855 12.90 MB/sec execute 300 sec latency 112.854 ms 2024-04-25T13:50:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:18 smithi137 ceph-mon[32568]: pgmap v602: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 379 op/s 2024-04-25T13:50:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:18 smithi094 ceph-mon[29195]: pgmap v602: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 379 op/s 2024-04-25T13:50:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:18 smithi138 ceph-mon[32977]: pgmap v602: 97 pgs: 97 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 379 op/s 2024-04-25T13:50:18.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1023515 12.91 MB/sec execute 301 sec latency 20.850 ms 2024-04-25T13:50:19.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1026131 12.91 MB/sec execute 302 sec latency 16.941 ms 2024-04-25T13:50:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:20 smithi137 ceph-mon[32568]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-04-25T13:50:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:20 smithi094 ceph-mon[29195]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-04-25T13:50:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:20 smithi138 ceph-mon[32977]: pgmap v603: 97 pgs: 97 active+clean; 41 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-04-25T13:50:20.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1027972 12.90 MB/sec execute 303 sec latency 17.061 ms 2024-04-25T13:50:21.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1030635 12.91 MB/sec execute 304 sec latency 9.865 ms 2024-04-25T13:50:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:22 smithi137 ceph-mon[32568]: pgmap v604: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-04-25T13:50:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:22 smithi094 ceph-mon[29195]: pgmap v604: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-04-25T13:50:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:22 smithi138 ceph-mon[32977]: pgmap v604: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-04-25T13:50:22.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1033217 12.91 MB/sec execute 305 sec latency 39.283 ms 2024-04-25T13:50:23.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1034714 12.89 MB/sec execute 306 sec latency 64.761 ms 2024-04-25T13:50:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:24 smithi137 ceph-mon[32568]: pgmap v605: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-04-25T13:50:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:24 smithi094 ceph-mon[29195]: pgmap v605: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-04-25T13:50:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:24 smithi138 ceph-mon[32977]: pgmap v605: 97 pgs: 97 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2024-04-25T13:50:24.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1037609 12.91 MB/sec execute 307 sec latency 12.466 ms 2024-04-25T13:50:25.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1040038 12.90 MB/sec execute 308 sec latency 15.169 ms 2024-04-25T13:50:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 -- 172.21.15.94:0/2037450681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06781044b0 msgr2=0x7f067810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 --2- 172.21.15.94:0/2037450681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06781044b0 0x7f067810abf0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f0670009900 tx=0x7f067002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 -- 172.21.15.94:0/2037450681 shutdown_connections 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 --2- 172.21.15.94:0/2037450681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06781044b0 0x7f067810abf0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 --2- 172.21.15.94:0/2037450681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781039d0 0x7f0678103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 --2- 172.21.15.94:0/2037450681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0678102ff0 0x7f0678103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 -- 172.21.15.94:0/2037450681 >> 172.21.15.94:0/2037450681 conn(0x7f06780fd390 msgr2=0x7f06780ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 -- 172.21.15.94:0/2037450681 shutdown_connections 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 -- 172.21.15.94:0/2037450681 wait complete. 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.197+0000 7f06805f3640 1 Processor -- start 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- start start 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06781039d0 0x7f067819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781044b0 0x7f06781a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f067810e580 con 0x7f0678102ff0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f067810e400 con 0x7f06781039d0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f067810e700 con 0x7f06781044b0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50498/0 (socket says 172.21.15.94:50498) 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 -- 172.21.15.94:0/1319367538 learned_addr learned my addr 172.21.15.94:0/1319367538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067db8e640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06781039d0 0x7f067819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067eb90640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781044b0 0x7f06781a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 -- 172.21.15.94:0/1319367538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781044b0 msgr2=0x7f06781a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781044b0 0x7f06781a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 -- 172.21.15.94:0/1319367538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06781039d0 msgr2=0x7f067819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06781039d0 0x7f067819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 -- 172.21.15.94:0/1319367538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06781a1ce0 con 0x7f0678102ff0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f067e38f640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f066800ed00 tx=0x7f066800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06777fe640 1 -- 172.21.15.94:0/1319367538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668010630 con 0x7f0678102ff0 2024-04-25T13:50:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06777fe640 1 -- 172.21.15.94:0/1319367538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0668010c80 con 0x7f0678102ff0 2024-04-25T13:50:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06781a1fd0 con 0x7f0678102ff0 2024-04-25T13:50:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06777fe640 1 -- 172.21.15.94:0/1319367538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668018d50 con 0x7f0678102ff0 2024-04-25T13:50:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.198+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06781a2430 con 0x7f0678102ff0 2024-04-25T13:50:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.201+0000 7f06777fe640 1 -- 172.21.15.94:0/1319367538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f066801f020 con 0x7f0678102ff0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.201+0000 7f068757d740 1 Processor -- start 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 -- start start 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682167e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 0x55682167eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5568215e4460 con 0x5568212bd4d0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f06777fe640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 0x7f06600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55682159e590 con 0x5568212de1b0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f068757d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568215e0420 con 0x5568212b52f0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067eb90640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 0x7f06600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50510/0 (socket says 172.21.15.94:50510) 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067db8e640 1 -- 172.21.15.94:0/2064157330 learned_addr learned my addr 172.21.15.94:0/2064157330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 0x55682167eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067eb90640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682167e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f06777fe640 1 -- 172.21.15.94:0/1319367538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f06680df020 con 0x7f0678102ff0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 -- 172.21.15.94:0/2064157330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 msgr2=0x55682167e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682167e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 -- 172.21.15.94:0/2064157330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 msgr2=0x556821685df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 -- 172.21.15.94:0/2064157330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568215ab0d0 con 0x5568212b52f0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067db8e640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067eb90640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 0x7f06600b1ff0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0670031440 tx=0x7f0670005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.202+0000 7f067e38f640 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 0x55682167eef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f06680a46d0 tx=0x7f066801ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a555dfe1bc65d277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.203+0000 7f0674bfe640 1 -- 172.21.15.94:0/2064157330 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a6db0 con 0x5568212b52f0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.203+0000 7f0674bfe640 1 -- 172.21.15.94:0/2064157330 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06680a6f70 con 0x5568212b52f0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.203+0000 7f0674bfe640 1 -- 172.21.15.94:0/2064157330 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a6bd0 con 0x5568212b52f0 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- 172.21.15.94:0/2064157330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 msgr2=0x55682167eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 0x55682167eef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f06680a46d0 tx=0x7f066801ac90 comp rx=0 tx=0).stop 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- 172.21.15.94:0/2064157330 shutdown_connections 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821685df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212b52f0 0x55682167eef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- 172.21.15.94:0/2064157330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682167e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- 172.21.15.94:0/2064157330 >> 172.21.15.94:0/2064157330 conn(0x5568212d2300 msgr2=0x55682167a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- 172.21.15.94:0/2064157330 shutdown_connections 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- 172.21.15.94:0/2064157330 wait complete. 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 -- start start 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 Processor -- start 2024-04-25T13:50:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.204+0000 7f068757d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x55682168fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556821686840 con 0x5568212bd4d0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55682156bb60 con 0x5568212b52f0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568216869c0 con 0x5568212de1b0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 10 client.0 _ll_register_callbacks cb 0x5568215902b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067e38f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50514/0 (socket says 172.21.15.94:50514) 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 -- 172.21.15.94:0/180884656 learned_addr learned my addr 172.21.15.94:0/180884656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067eb90640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x55682168fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 -- 172.21.15.94:0/180884656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 msgr2=0x55682168fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x55682168fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 -- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 msgr2=0x5568216886a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216886a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 -- 172.21.15.94:0/180884656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568212f6210 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067e38f640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216886a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f067db8e640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f066c00ba00 tx=0x7f066c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a88555a4f1813a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f065dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c003e80 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f066c011440 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06680a6f70 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f066c005020 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 10 client.18582 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.205+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568213c2620 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06680a6f70 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f066c011640 con 0x5568212bd4d0 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 10 client.18582 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 20 client.18582 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55682129cd80 0x556821696430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 4 client.18582 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568215f2070 con 0x55682129cd80 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f067e38f640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55682129cd80 0x556821696430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f065dffb640 10 client.18582 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.206+0000 7f067e38f640 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55682129cd80 0x556821696430 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954fde5307dde6b server_cookie=cba9cf68443b8d75 in_seq=0 out_seq=0 2024-04-25T13:50:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.208+0000 7f065dffb640 1 -- 172.21.15.94:0/180884656 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568215f2070 con 0x55682129cd80 2024-04-25T13:50:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f065dffb640 10 client.18582 handle_command_reply: tid=0 2024-04-25T13:50:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 client.18582 shutdown 2024-04-25T13:50:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 msgr2=0x556821688be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f066c00ba00 tx=0x7f066c00bec0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 shutdown_connections 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55682129cd80 0x556821696430 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x55682168fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x556821688be0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 --2- 172.21.15.94:0/180884656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216886a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f065dffb640 0 client.18582 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 >> 172.21.15.94:0/180884656 conn(0x5568212d2300 msgr2=0x55682167d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 shutdown_connections 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 1 -- 172.21.15.94:0/180884656 wait complete. 2024-04-25T13:50:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.209+0000 7f068757d740 20 client.18582 trim_cache size 0 max 16384 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 Processor -- start 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 -- start start 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212bd4d0 0x55682169c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682168fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06781ab590 con 0x5568212b52f0 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06781ad0d0 con 0x5568212de1b0 2024-04-25T13:50:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f068757d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06781abc60 con 0x5568212bd4d0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.224+0000 7f067e38f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50526/0 (socket says 172.21.15.94:50526) 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 -- 172.21.15.94:0/45982848 learned_addr learned my addr 172.21.15.94:0/45982848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067eb90640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682168fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067db8e640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212bd4d0 0x55682169c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 -- 172.21.15.94:0/45982848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212bd4d0 msgr2=0x55682169c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212bd4d0 0x55682169c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 -- 172.21.15.94:0/45982848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 msgr2=0x55682168fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682168fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 -- 172.21.15.94:0/45982848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682168ffa0 con 0x5568212b52f0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067eb90640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682168fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f067e38f640 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f06680a3a00 tx=0x7f06680187f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a07e1c1cecf4d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f065ffff640 1 -- 172.21.15.94:0/45982848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a6bd0 con 0x5568212b52f0 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f065ffff640 1 -- 172.21.15.94:0/45982848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06680a6f70 con 0x5568212b52f0 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.225+0000 7f065ffff640 1 -- 172.21.15.94:0/45982848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a6db0 con 0x5568212b52f0 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- 172.21.15.94:0/45982848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 msgr2=0x55682169bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f06680a3a00 tx=0x7f06680187f0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- 172.21.15.94:0/45982848 shutdown_connections 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212de1b0 0x55682168fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212bd4d0 0x55682169c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 --2- 172.21.15.94:0/45982848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212b52f0 0x55682169bc30 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- 172.21.15.94:0/45982848 >> 172.21.15.94:0/45982848 conn(0x5568212d2300 msgr2=0x55682167e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- 172.21.15.94:0/45982848 shutdown_connections 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- 172.21.15.94:0/45982848 wait complete. 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 -- start start 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 Processor -- start 2024-04-25T13:50:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.226+0000 7f068757d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x5568216fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06781aca00 con 0x5568212bd4d0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06781ac330 con 0x5568212b52f0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06781a9570 con 0x5568212de1b0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 10 client.0 _ll_register_callbacks cb 0x7f066c004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50538/0 (socket says 172.21.15.94:50538) 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 -- 172.21.15.94:0/2367382874 learned_addr learned my addr 172.21.15.94:0/2367382874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067e38f640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067eb90640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x5568216fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 -- 172.21.15.94:0/2367382874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 msgr2=0x5568216fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x5568216fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 -- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 msgr2=0x5568216f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 -- 172.21.15.94:0/2367382874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568216fd790 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067e38f640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f067db8e640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f066c00ba00 tx=0x7f066c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9835cd48079b967c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f065dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c011650 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06680a6f70 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.227+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f066c0117f0 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f066c0051c0 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 10 client.18588 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568216fe9e0 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06680a6f70 con 0x5568212bd4d0 2024-04-25T13:50:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f066c024440 con 0x5568212bd4d0 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 10 client.18588 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 20 client.18588 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5568212721c0 0x556821701450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 4 client.18588 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568215ebee0 con 0x5568212721c0 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.228+0000 7f067e38f640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5568212721c0 0x556821701450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.229+0000 7f067e38f640 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5568212721c0 0x556821701450 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56010c2f65b5831 server_cookie=3b1caac241aba3a8 in_seq=0 out_seq=0 2024-04-25T13:50:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.229+0000 7f065dffb640 10 client.18588 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:26 smithi137 ceph-mon[32568]: pgmap v606: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-04-25T13:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f065dffb640 1 -- 172.21.15.94:0/2367382874 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568215ebee0 con 0x5568212721c0 2024-04-25T13:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f065dffb640 10 client.18588 handle_command_reply: tid=0 2024-04-25T13:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f068757d740 1 client.18588 shutdown 2024-04-25T13:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 msgr2=0x5568216fad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f066c00ba00 tx=0x7f066c00b670 comp rx=0 tx=0).stop 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 shutdown_connections 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.411+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5568212721c0 0x556821701450 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f065dffb640 0 client.18588 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568212de1b0 0x5568216fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5568212bd4d0 0x5568216fad00 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 --2- 172.21.15.94:0/2367382874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568212b52f0 0x5568216f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 >> 172.21.15.94:0/2367382874 conn(0x5568212d2300 msgr2=0x55682167e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 shutdown_connections 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 1 -- 172.21.15.94:0/2367382874 wait complete. 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f068757d740 20 client.18588 trim_cache size 0 max 16384 2024-04-25T13:50:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:50:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:50:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:50:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 msgr2=0x7f06600b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 0x7f06600b1ff0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0670031440 tx=0x7f0670005ef0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 msgr2=0x7f0678102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f066800ed00 tx=0x7f066800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.412+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 shutdown_connections 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06600afae0 0x7f06600b1ff0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06781044b0 0x7f06781a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06781039d0 0x7f067819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 --2- 172.21.15.94:0/1319367538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0678102ff0 0x7f0678102290 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 >> 172.21.15.94:0/1319367538 conn(0x7f06780fd390 msgr2=0x7f0678106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 shutdown_connections 2024-04-25T13:50:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:26.413+0000 7f06805f3640 1 -- 172.21.15.94:0/1319367538 wait complete. 2024-04-25T13:50:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:26 smithi094 ceph-mon[29195]: pgmap v606: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-04-25T13:50:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:26 smithi138 ceph-mon[32977]: pgmap v606: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-04-25T13:50:26.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1041581 12.89 MB/sec execute 309 sec latency 101.394 ms 2024-04-25T13:50:27.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1044500 12.90 MB/sec execute 310 sec latency 13.224 ms 2024-04-25T13:50:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:28 smithi137 ceph-mon[32568]: pgmap v607: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-04-25T13:50:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:28 smithi094 ceph-mon[29195]: pgmap v607: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-04-25T13:50:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:28 smithi138 ceph-mon[32977]: pgmap v607: 97 pgs: 97 active+clean; 49 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-04-25T13:50:28.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1045903 12.88 MB/sec execute 311 sec latency 85.919 ms 2024-04-25T13:50:29.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1048427 12.89 MB/sec execute 312 sec latency 10.572 ms 2024-04-25T13:50:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:30 smithi137 ceph-mon[32568]: pgmap v608: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-25T13:50:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:30 smithi094 ceph-mon[29195]: pgmap v608: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-25T13:50:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:30 smithi138 ceph-mon[32977]: pgmap v608: 97 pgs: 97 active+clean; 46 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-25T13:50:30.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1051258 12.89 MB/sec execute 313 sec latency 14.556 ms 2024-04-25T13:50:31.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1053087 12.88 MB/sec execute 314 sec latency 16.618 ms 2024-04-25T13:50:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:32 smithi137 ceph-mon[32568]: pgmap v609: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-04-25T13:50:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:32 smithi094 ceph-mon[29195]: pgmap v609: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-04-25T13:50:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:32 smithi138 ceph-mon[32977]: pgmap v609: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 375 op/s 2024-04-25T13:50:32.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:50:32.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:50:32.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 -- 172.21.15.94:0/1274077734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 msgr2=0x7efcc410bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 --2- 172.21.15.94:0/1274077734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc410bca0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009900 tx=0x7efcc002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 -- 172.21.15.94:0/1274077734 shutdown_connections 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 --2- 172.21.15.94:0/1274077734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc410bca0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 --2- 172.21.15.94:0/1274077734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 0x7efcc4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 --2- 172.21.15.94:0/1274077734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 0x7efcc41054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 -- 172.21.15.94:0/1274077734 >> 172.21.15.94:0/1274077734 conn(0x7efcc40fd340 msgr2=0x7efcc40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 -- 172.21.15.94:0/1274077734 shutdown_connections 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.635+0000 7efccb94d640 1 -- 172.21.15.94:0/1274077734 wait complete. 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 Processor -- start 2024-04-25T13:50:32.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 -- start start 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 0x7efcc419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 0x7efcc41a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcc410f620 con 0x7efcc4102240 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc410f4a0 con 0x7efcc4101740 2024-04-25T13:50:32.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.636+0000 7efccb94d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcc410f7a0 con 0x7efcc4105090 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50560/0 (socket says 172.21.15.94:50560) 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 -- 172.21.15.94:0/4032370106 learned_addr learned my addr 172.21.15.94:0/4032370106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc96e9640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 0x7efcc419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc9eea640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 0x7efcc41a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 -- 172.21.15.94:0/4032370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 msgr2=0x7efcc41a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 0x7efcc41a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 -- 172.21.15.94:0/4032370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 msgr2=0x7efcc419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 0x7efcc419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 -- 172.21.15.94:0/4032370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc41a1ce0 con 0x7efcc4102240 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcba7fc640 1 -- 172.21.15.94:0/4032370106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb400cd80 con 0x7efcc4102240 2024-04-25T13:50:32.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc41a1fd0 con 0x7efcc4102240 2024-04-25T13:50:32.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcba7fc640 1 -- 172.21.15.94:0/4032370106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb40108e0 con 0x7efcc4102240 2024-04-25T13:50:32.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efcba7fc640 1 -- 172.21.15.94:0/4032370106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4018980 con 0x7efcc4102240 2024-04-25T13:50:32.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.637+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc41a2490 con 0x7efcc4102240 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.640+0000 7efcba7fc640 1 -- 172.21.15.94:0/4032370106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efcb4010430 con 0x7efcc4102240 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.640+0000 7efcd2801740 1 Processor -- start 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.640+0000 7efcd2801740 1 -- start start 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.640+0000 7efcd2801740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 0x559e997e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcd2801740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcd2801740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcd2801740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e99747d80 con 0x559e994206b0 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcd2801740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e997018b0 con 0x559e994184d0 2024-04-25T13:50:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcd2801740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e99743d40 con 0x559e99441390 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc96e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc9eea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 0x559e997e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50568/0 (socket says 172.21.15.94:50568) 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 -- 172.21.15.94:0/2186149672 learned_addr learned my addr 172.21.15.94:0/2186149672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 -- 172.21.15.94:0/2186149672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 msgr2=0x559e997e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 0x559e997e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 -- 172.21.15.94:0/2186149672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 msgr2=0x559e997e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc8ee8640 1 -- 172.21.15.94:0/2186149672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e996d2a40 con 0x559e994206b0 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc9eea640 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 0x559e997e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcba7fc640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 0x7efcb00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcc9eea640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 0x7efcb00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.641+0000 7efcba7fc640 1 -- 172.21.15.94:0/4032370106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efcb40def10 con 0x7efcc4102240 2024-04-25T13:50:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.642+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7efcb40e7160 tx=0x7efcb40e7620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72d673f6ef8c410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.642+0000 7efcaf7fe640 1 -- 172.21.15.94:0/2186149672 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a32c0 con 0x559e994206b0 2024-04-25T13:50:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.642+0000 7efcaf7fe640 1 -- 172.21.15.94:0/2186149672 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb4010720 con 0x559e994206b0 2024-04-25T13:50:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.642+0000 7efcaf7fe640 1 -- 172.21.15.94:0/2186149672 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a6b70 con 0x559e994206b0 2024-04-25T13:50:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- 172.21.15.94:0/2186149672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 msgr2=0x559e997e3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcc9eea640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 0x7efcb00b2080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efcc0031440 tx=0x7efcc00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7efcb40e7160 tx=0x7efcb40e7620 comp rx=0 tx=0).stop 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- 172.21.15.94:0/2186149672 shutdown_connections 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e3c70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 --2- 172.21.15.94:0/2186149672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e99441390 0x559e997e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- 172.21.15.94:0/2186149672 >> 172.21.15.94:0/2186149672 conn(0x559e994354e0 msgr2=0x559e997de270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- 172.21.15.94:0/2186149672 shutdown_connections 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- 172.21.15.94:0/2186149672 wait complete. 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 -- start start 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.643+0000 7efcd2801740 1 Processor -- start 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997ecdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 0x559e997f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e997e47d0 con 0x559e99441390 2024-04-25T13:50:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e996ced10 con 0x559e994184d0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e997e4950 con 0x559e994206b0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcd2801740 10 client.0 _ll_register_callbacks cb 0x559e996fb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc9eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 0x559e997f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997ecdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54332/0 (socket says 172.21.15.94:54332) 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc9eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 0x559e997f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50578/0 (socket says 172.21.15.94:50578) 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 -- 172.21.15.94:0/3698123587 learned_addr learned my addr 172.21.15.94:0/3698123587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 -- 172.21.15.94:0/3698123587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 msgr2=0x559e997ecdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997ecdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 -- 172.21.15.94:0/3698123587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 msgr2=0x559e997f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 0x559e997f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.644+0000 7efcc96e9640 1 -- 172.21.15.94:0/3698123587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e995237d0 con 0x559e994184d0 2024-04-25T13:50:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcc96e9640 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcbc011ce0 tx=0x7efcbc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a090713e5491ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc003970 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e99755990 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efcbc003b10 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcbc003d10 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 10 client.28070 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e994593f0 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e99755990 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efcbc01b690 con 0x559e994184d0 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 10 client.28070 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 10 client.28070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 20 client.28070 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e993fff60 0x559e997fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 4 client.28070 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.645+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9974f800 con 0x559e993fff60 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.646+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e993fff60 0x559e997fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.646+0000 7efcacff9640 10 client.28070 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.646+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e993fff60 0x559e997fa2b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e217700d5945709 server_cookie=ac72ec94284adcdc in_seq=0 out_seq=0 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcacff9640 1 -- 172.21.15.94:0/3698123587 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e9974f800 con 0x559e993fff60 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcacff9640 10 client.28070 handle_command_reply: tid=0 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 client.28070 shutdown 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 msgr2=0x559e997ec8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcbc011ce0 tx=0x7efcbc00f790 comp rx=0 tx=0).stop 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 shutdown_connections 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e993fff60 0x559e997fa2b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e99441390 0x559e997f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcacff9640 0 client.28070 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997ecdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 --2- 172.21.15.94:0/3698123587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997ec8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.647+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 >> 172.21.15.94:0/3698123587 conn(0x559e994354e0 msgr2=0x559e997e1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.648+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 shutdown_connections 2024-04-25T13:50:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.648+0000 7efcd2801740 1 -- 172.21.15.94:0/3698123587 wait complete. 2024-04-25T13:50:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.648+0000 7efcd2801740 20 client.28070 trim_cache size 0 max 16384 2024-04-25T13:50:32.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 Processor -- start 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 -- start start 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e993fff60 0x559e997de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41ab600 con 0x559e994206b0 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41ad140 con 0x559e994184d0 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcd2801740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41abcd0 con 0x559e993fff60 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50586/0 (socket says 172.21.15.94:50586) 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 -- 172.21.15.94:0/392881577 learned_addr learned my addr 172.21.15.94:0/392881577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc96e9640 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e993fff60 0x559e997de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 -- 172.21.15.94:0/392881577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e993fff60 msgr2=0x559e997de3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e993fff60 0x559e997de3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 -- 172.21.15.94:0/392881577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 msgr2=0x559e997e10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 -- 172.21.15.94:0/392881577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e994190f0 con 0x559e994206b0 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.666+0000 7efcc9eea640 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7efcc002a930 tx=0x7efcc0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acd2458b8e669df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcaeffd640 1 -- 172.21.15.94:0/392881577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0035040 con 0x559e994206b0 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcaeffd640 1 -- 172.21.15.94:0/392881577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc00035f0 con 0x559e994206b0 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcaeffd640 1 -- 172.21.15.94:0/392881577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc003d670 con 0x559e994206b0 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 -- 172.21.15.94:0/392881577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 msgr2=0x559e997e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7efcc002a930 tx=0x7efcc0040000 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 -- 172.21.15.94:0/392881577 shutdown_connections 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994206b0 0x559e997e1630 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e994184d0 0x559e997e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 --2- 172.21.15.94:0/392881577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e993fff60 0x559e997de3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 -- 172.21.15.94:0/392881577 >> 172.21.15.94:0/392881577 conn(0x559e994354e0 msgr2=0x559e997ddca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 -- 172.21.15.94:0/392881577 shutdown_connections 2024-04-25T13:50:32.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.667+0000 7efcd2801740 1 -- 172.21.15.94:0/392881577 wait complete. 2024-04-25T13:50:32.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 -- start start 2024-04-25T13:50:32.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 Processor -- start 2024-04-25T13:50:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e993fff60 0x559e997f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41aca70 con 0x559e994184d0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41ac3a0 con 0x559e993fff60 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41a95e0 con 0x559e994206b0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcd2801740 10 client.0 _ll_register_callbacks cb 0x7efcbc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcc8ee8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50588/0 (socket says 172.21.15.94:50588) 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcc8ee8640 1 -- 172.21.15.94:0/37633664 learned_addr learned my addr 172.21.15.94:0/37633664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcc96e9640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e993fff60 0x559e997f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.668+0000 7efcc9eea640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 -- 172.21.15.94:0/37633664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 msgr2=0x559e997f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 -- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e993fff60 msgr2=0x559e997f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:32.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e993fff60 0x559e997f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 -- 172.21.15.94:0/37633664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e997084b0 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcc8ee8640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7efcb40dc9c0 tx=0x7efcb4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1611107f9c605ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a6d50 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efcb4010a90 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc00035f0 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcb40a71d0 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcd2801740 10 client.18606 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e997f31e0 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc00035f0 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.669+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efcb40a3690 con 0x559e994184d0 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 10 client.18606 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:32.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 20 client.18606 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e99441390 0x559e99864630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 4 client.18606 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e996ca4a0 con 0x559e99441390 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcc96e9640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e99441390 0x559e99864630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcacff9640 10 client.18606 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcc96e9640 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e99441390 0x559e99864630 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13004848a3482dd server_cookie=f748d03f3da080a5 in_seq=0 out_seq=0 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcacff9640 1 -- 172.21.15.94:0/37633664 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e996ca4a0 con 0x559e99441390 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcacff9640 10 client.18606 handle_command_reply: tid=0 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 client.18606 shutdown 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 msgr2=0x559e997f0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7efcb40dc9c0 tx=0x7efcb4018420 comp rx=0 tx=0).stop 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 shutdown_connections 2024-04-25T13:50:32.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.670+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e99441390 0x559e99864630 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e994206b0 0x559e997f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcacff9640 0 client.18606 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e994184d0 0x559e997f0bc0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 --2- 172.21.15.94:0/37633664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e993fff60 0x559e997f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 >> 172.21.15.94:0/37633664 conn(0x559e994354e0 msgr2=0x559e997e10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 shutdown_connections 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 1 -- 172.21.15.94:0/37633664 wait complete. 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efcd2801740 20 client.18606 trim_cache size 0 max 16384 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:50:32.677 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:50:32.678 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:50:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:50:32.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 msgr2=0x7efcb00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 0x7efcb00b2080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efcc0031440 tx=0x7efcc00314a0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 msgr2=0x7efcc419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 shutdown_connections 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcb00afb70 0x7efcb00b2080 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc4105090 0x7efcc41a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcc4102240 0x7efcc419a6e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 --2- 172.21.15.94:0/4032370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc4101740 0x7efcc419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 >> 172.21.15.94:0/4032370106 conn(0x7efcc40fd340 msgr2=0x7efcc4107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 shutdown_connections 2024-04-25T13:50:32.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.671+0000 7efccb94d640 1 -- 172.21.15.94:0/4032370106 wait complete. 2024-04-25T13:50:32.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:50:32.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 -- 172.21.15.94:0/1258373524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338103c50 msgr2=0x7f3338104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 --2- 172.21.15.94:0/1258373524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338103c50 0x7f3338104060 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3320009900 tx=0x7f332002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 -- 172.21.15.94:0/1258373524 shutdown_connections 2024-04-25T13:50:32.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 --2- 172.21.15.94:0/1258373524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338108950 0x7f333810ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 --2- 172.21.15.94:0/1258373524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338103c50 0x7f3338104060 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 --2- 172.21.15.94:0/1258373524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338102a10 0x7f3338103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 -- 172.21.15.94:0/1258373524 >> 172.21.15.94:0/1258373524 conn(0x7f33380fdc00 msgr2=0x7f33381000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 -- 172.21.15.94:0/1258373524 shutdown_connections 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.794+0000 7f333d6d8640 1 -- 172.21.15.94:0/1258373524 wait complete. 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 Processor -- start 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 -- start start 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 0x7f33381991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338103c50 0x7f33381996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f333810e530 con 0x7f3338108950 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f333810e3b0 con 0x7f3338102a10 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f333d6d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f333810e6b0 con 0x7f3338103c50 2024-04-25T13:50:32.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f3336ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 0x7f33381991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50600/0 (socket says 172.21.15.94:50600) 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 -- 172.21.15.94:0/3194738748 learned_addr learned my addr 172.21.15.94:0/3194738748 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f332ffff640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338103c50 0x7f33381996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 -- 172.21.15.94:0/3194738748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338103c50 msgr2=0x7f33381996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338103c50 0x7f33381996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 -- 172.21.15.94:0/3194738748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 msgr2=0x7f33381991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 0x7f33381991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.795+0000 7f33377fe640 1 -- 172.21.15.94:0/3194738748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a0ce0 con 0x7f3338108950 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f3336ffd640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 0x7f33381991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f33377fe640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f333000eb10 tx=0x7f333000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004b20 con 0x7f3338108950 2024-04-25T13:50:32.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3330004cc0 con 0x7f3338108950 2024-04-25T13:50:32.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a0fd0 con 0x7f3338108950 2024-04-25T13:50:32.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330010670 con 0x7f3338108950 2024-04-25T13:50:32.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.796+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a13e0 con 0x7f3338108950 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.797+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3338069340 con 0x7f3338108950 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.799+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3330005000 con 0x7f3338108950 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.800+0000 7f3334ff9640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 0x7f33240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.800+0000 7f3336ffd640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 0x7f33240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.800+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f33300df4f0 con 0x7f3338108950 2024-04-25T13:50:32.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.800+0000 7f3336ffd640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 0x7f33240b2000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3328005fd0 tx=0x7f332800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:32.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:32.804+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3330053ff0 con 0x7f3338108950 2024-04-25T13:50:32.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1055596 12.88 MB/sec execute 315 sec latency 20.347 ms 2024-04-25T13:50:33.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.054+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333802dc10 con 0x7f3338108950 2024-04-25T13:50:33.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.055+0000 7f3334ff9640 1 -- 172.21.15.94:0/3194738748 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f3338069340 con 0x7f3338108950 2024-04-25T13:50:33.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:50:33.055 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:50:33.055 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 msgr2=0x7f33240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 0x7f33240b2000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3328005fd0 tx=0x7f332800b390 comp rx=0 tx=0).stop 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 msgr2=0x7f33381a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f333000eb10 tx=0x7f333000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 shutdown_connections 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33240afaf0 0x7f33240b2000 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3338108950 0x7f33381a0670 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3338103c50 0x7f33381996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 --2- 172.21.15.94:0/3194738748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3338102a10 0x7f33381991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 >> 172.21.15.94:0/3194738748 conn(0x7f33380fdc00 msgr2=0x7f3338104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 shutdown_connections 2024-04-25T13:50:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.057+0000 7f333d6d8640 1 -- 172.21.15.94:0/3194738748 wait complete. 2024-04-25T13:50:33.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:50:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 -- 172.21.15.94:0/1960793560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c103ae0 msgr2=0x7f0e2c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 --2- 172.21.15.94:0/1960793560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c1047e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009900 tx=0x7f0e1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 -- 172.21.15.94:0/1960793560 shutdown_connections 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 --2- 172.21.15.94:0/1960793560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c109a20 0x7f0e2c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 --2- 172.21.15.94:0/1960793560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c104d20 0x7f0e2c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 --2- 172.21.15.94:0/1960793560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c1047e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 -- 172.21.15.94:0/1960793560 >> 172.21.15.94:0/1960793560 conn(0x7f0e2c0fecd0 msgr2=0x7f0e2c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 -- 172.21.15.94:0/1960793560 shutdown_connections 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.183+0000 7f0e3479d640 1 -- 172.21.15.94:0/1960793560 wait complete. 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 Processor -- start 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 -- start start 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c109a20 0x7f0e2c075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10f5c0 con 0x7f0e2c104d20 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10f440 con 0x7f0e2c103ae0 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e3479d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10f740 con 0x7f0e2c109a20 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50618/0 (socket says 172.21.15.94:50618) 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 -- 172.21.15.94:0/178824631 learned_addr learned my addr 172.21.15.94:0/178824631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c109a20 0x7f0e2c075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 -- 172.21.15.94:0/178824631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c109a20 msgr2=0x7f0e2c075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c109a20 0x7f0e2c075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 -- 172.21.15.94:0/178824631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c103ae0 msgr2=0x7f0e2c076850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c076850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.184+0000 7f0e31d38640 1 -- 172.21.15.94:0/178824631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c19e0b0 con 0x7f0e2c104d20 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e31d38640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d950 tx=0x7f0e2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e237fe640 1 -- 172.21.15.94:0/178824631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017030 con 0x7f0e2c104d20 2024-04-25T13:50:33.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e237fe640 1 -- 172.21.15.94:0/178824631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e28004020 con 0x7f0e2c104d20 2024-04-25T13:50:33.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c075a90 con 0x7f0e2c104d20 2024-04-25T13:50:33.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e237fe640 1 -- 172.21.15.94:0/178824631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e280047a0 con 0x7f0e2c104d20 2024-04-25T13:50:33.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.185+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c1a8cb0 con 0x7f0e2c104d20 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.187+0000 7f0e237fe640 1 -- 172.21.15.94:0/178824631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0e28009a30 con 0x7f0e2c104d20 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.187+0000 7f0e3b650740 1 Processor -- start 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 -- start start 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505ccb0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56505cc14d80 con 0x56505c90e390 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56505cbcea20 con 0x56505c8e54d0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e3b650740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56505cc10d40 con 0x56505c8ed6b0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505ccb0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e237fe640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 0x7f0e180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54378/0 (socket says 172.21.15.94:54378) 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccb1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59494/0 (socket says 172.21.15.94:59494) 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 -- 172.21.15.94:0/787581365 learned_addr learned my addr 172.21.15.94:0/787581365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 0x7f0e180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 -- 172.21.15.94:0/787581365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 msgr2=0x56505ccb1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccb1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 -- 172.21.15.94:0/787581365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 msgr2=0x56505ccb0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505ccb0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e32539640 1 -- 172.21.15.94:0/787581365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56505cbd9c30 con 0x56505c8e54d0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.188+0000 7f0e237fe640 1 -- 172.21.15.94:0/178824631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0e280de6d0 con 0x7f0e2c104d20 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.189+0000 7f0e32539640 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009230 tx=0x7f0e1c031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2bac07dda7f3502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.189+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 0x7f0e180b2000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009fd0 tx=0x7f0e24004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.189+0000 7f0e20ff9640 1 -- 172.21.15.94:0/787581365 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c035030 con 0x56505c8e54d0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.189+0000 7f0e20ff9640 1 -- 172.21.15.94:0/787581365 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c040070 con 0x56505c8e54d0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.189+0000 7f0e20ff9640 1 -- 172.21.15.94:0/787581365 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c003930 con 0x56505c8e54d0 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- 172.21.15.94:0/787581365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 msgr2=0x56505ccb0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009230 tx=0x7f0e1c031b10 comp rx=0 tx=0).stop 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- 172.21.15.94:0/787581365 shutdown_connections 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccb1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccb0d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- 172.21.15.94:0/787581365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505ccb0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- 172.21.15.94:0/787581365 >> 172.21.15.94:0/787581365 conn(0x56505c9024e0 msgr2=0x56505ccac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- 172.21.15.94:0/787581365 shutdown_connections 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- 172.21.15.94:0/787581365 wait complete. 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- start start 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 Processor -- start 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccc1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56505ccb24d0 con 0x56505c8ed6b0 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56505cb9bdc0 con 0x56505c8e54d0 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56505ccb2650 con 0x56505c90e390 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.190+0000 7f0e3b650740 10 client.0 _ll_register_callbacks cb 0x56505cbb7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50632/0 (socket says 172.21.15.94:50632) 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 -- 172.21.15.94:0/2743630380 learned_addr learned my addr 172.21.15.94:0/2743630380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e32539640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccc1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 -- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 msgr2=0x56505ccc1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccc1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 -- 172.21.15.94:0/2743630380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 msgr2=0x56505ccba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 -- 172.21.15.94:0/2743630380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56505c9263f0 con 0x56505c8ed6b0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e31d38640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e280dc130 tx=0x7f0e28005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3897fb5fa191871a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e280aaec0 con 0x56505c8ed6b0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0e280a2da0 con 0x56505c8ed6b0 2024-04-25T13:50:33.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e1c040070 con 0x56505c8ed6b0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e280a6620 con 0x56505c8ed6b0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e3b650740 10 client.18624 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56505c9f07d0 con 0x56505c8ed6b0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.191+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c040070 con 0x56505c8ed6b0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e280deb80 con 0x56505c8ed6b0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 10 client.18624 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 10 client.18624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 20 client.18624 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8ccf60 0x56505ccc8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 4 client.18624 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56505cc22990 con 0x56505c8ccf60 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e32539640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8ccf60 0x56505ccc8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e15ffb640 10 client.18624 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:33.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.192+0000 7f0e32539640 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8ccf60 0x56505ccc8320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d87cea58801d2e0 server_cookie=9a51aee1e81c9d2a in_seq=0 out_seq=0 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e15ffb640 1 -- 172.21.15.94:0/2743630380 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56505cc22990 con 0x56505c8ccf60 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e15ffb640 10 client.18624 handle_command_reply: tid=0 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 client.18624 shutdown 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 msgr2=0x56505ccbaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e280dc130 tx=0x7f0e28005bf0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 shutdown_connections 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8ccf60 0x56505ccc8320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccc1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccbaaa0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e15ffb640 0 client.18624 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 --2- 172.21.15.94:0/2743630380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 >> 172.21.15.94:0/2743630380 conn(0x56505c9024e0 msgr2=0x56505ccaef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 shutdown_connections 2024-04-25T13:50:33.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 1 -- 172.21.15.94:0/2743630380 wait complete. 2024-04-25T13:50:33.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.193+0000 7f0e3b650740 20 client.18624 trim_cache size 0 max 16384 2024-04-25T13:50:33.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.207+0000 7f0e3b650740 1 Processor -- start 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.207+0000 7f0e3b650740 1 -- start start 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccc1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1b1a40 con 0x56505c8ed6b0 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1b3580 con 0x56505c8e54d0 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e3b650740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1b2110 con 0x56505c90e390 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccc1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32d3a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54392/0 (socket says 172.21.15.94:54392) 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32d3a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccabf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59502/0 (socket says 172.21.15.94:59502) 2024-04-25T13:50:33.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 -- 172.21.15.94:0/3764046652 learned_addr learned my addr 172.21.15.94:0/3764046652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccc1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50644/0 (socket says 172.21.15.94:50644) 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 -- 172.21.15.94:0/3764046652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 msgr2=0x56505ccabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 -- 172.21.15.94:0/3764046652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 msgr2=0x56505ccc1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccc1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 -- 172.21.15.94:0/3764046652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56505ccb4dc0 con 0x56505c8e54d0 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.208+0000 7f0e32539640 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009900 tx=0x7f0e1c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767bcd4899d7323c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.209+0000 7f0e17fff640 1 -- 172.21.15.94:0/3764046652 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c057070 con 0x56505c8e54d0 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.209+0000 7f0e17fff640 1 -- 172.21.15.94:0/3764046652 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c040050 con 0x56505c8e54d0 2024-04-25T13:50:33.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.209+0000 7f0e17fff640 1 -- 172.21.15.94:0/3764046652 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c035040 con 0x56505c8e54d0 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- 172.21.15.94:0/3764046652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 msgr2=0x56505ccaba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009900 tx=0x7f0e1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- 172.21.15.94:0/3764046652 shutdown_connections 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c90e390 0x56505ccabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c8ed6b0 0x56505ccc1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3764046652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccaba50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- 172.21.15.94:0/3764046652 >> 172.21.15.94:0/3764046652 conn(0x56505c9024e0 msgr2=0x56505ccb52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- 172.21.15.94:0/3764046652 shutdown_connections 2024-04-25T13:50:33.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- 172.21.15.94:0/3764046652 wait complete. 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- start start 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 Processor -- start 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccbe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1b2eb0 con 0x56505c90e390 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1b27e0 con 0x56505c8e54d0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c1afa20 con 0x56505c8ed6b0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.210+0000 7f0e3b650740 10 client.0 _ll_register_callbacks cb 0x7f0e280046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccbe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e31d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59514/0 (socket says 172.21.15.94:59514) 2024-04-25T13:50:33.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50646/0 (socket says 172.21.15.94:50646) 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e31d38640 1 -- 172.21.15.94:0/3763011769 learned_addr learned my addr 172.21.15.94:0/3763011769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccbe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54408/0 (socket says 172.21.15.94:54408) 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 -- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 msgr2=0x56505ccbee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 -- 172.21.15.94:0/3763011769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 msgr2=0x56505ccbe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccbe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 -- 172.21.15.94:0/3763011769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56505cd2ebc0 con 0x56505c90e390 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e31d38640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e32d3a640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005fd0 tx=0x7f0e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c62b361d7154e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004b20 con 0x56505c90e390 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0e2401c070 con 0x56505c90e390 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e1c040050 con 0x56505c90e390 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e24004cc0 con 0x56505c90e390 2024-04-25T13:50:33.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e3b650740 10 client.18630 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.211+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56505cd2fdd0 con 0x56505c90e390 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c040050 con 0x56505c90e390 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0e24005140 con 0x56505c90e390 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 10 client.18630 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 10 client.18630 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 20 client.18630 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8a23a0 0x56505cd32870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 4 client.18630 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56505cc1c800 con 0x56505c8a23a0 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e32539640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8a23a0 0x56505cd32870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.212+0000 7f0e15ffb640 10 client.18630 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:33.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e32539640 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8a23a0 0x56505cd32870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf93840f7d3b8064 server_cookie=fb68a6da39a5deae in_seq=0 out_seq=0 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e15ffb640 1 -- 172.21.15.94:0/3763011769 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56505cc1c800 con 0x56505c8a23a0 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e15ffb640 10 client.18630 handle_command_reply: tid=0 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 client.18630 shutdown 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 msgr2=0x56505cd2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005fd0 tx=0x7f0e24004a40 comp rx=0 tx=0).stop 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 shutdown_connections 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56505c8a23a0 0x56505cd32870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56505c90e390 0x56505cd2e680 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e15ffb640 0 client.18630 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56505c8ed6b0 0x56505ccbee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 --2- 172.21.15.94:0/3763011769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56505c8e54d0 0x56505ccbe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 >> 172.21.15.94:0/3763011769 conn(0x56505c9024e0 msgr2=0x56505ccab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 shutdown_connections 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 1 -- 172.21.15.94:0/3763011769 wait complete. 2024-04-25T13:50:33.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.213+0000 7f0e3b650740 20 client.18630 trim_cache size 0 max 16384 2024-04-25T13:50:33.220 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:33.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 msgr2=0x7f0e180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 0x7f0e180b2000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009fd0 tx=0x7f0e24004ab0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 msgr2=0x7f0e2c074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d950 tx=0x7f0e2800de10 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 shutdown_connections 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e180afaf0 0x7f0e180b2000 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2c109a20 0x7f0e2c075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e2c104d20 0x7f0e2c074ef0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 --2- 172.21.15.94:0/178824631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c103ae0 0x7f0e2c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 >> 172.21.15.94:0/178824631 conn(0x7f0e2c0fecd0 msgr2=0x7f0e2c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 shutdown_connections 2024-04-25T13:50:33.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.214+0000 7f0e3479d640 1 -- 172.21.15.94:0/178824631 wait complete. 2024-04-25T13:50:33.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 -- 172.21.15.94:0/1704043115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0106f40 msgr2=0x7fcaf0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1704043115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0106f40 0x7fcaf0109360 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 -- 172.21.15.94:0/1704043115 shutdown_connections 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1704043115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf01098a0 0x7fcaf010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1704043115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0106f40 0x7fcaf0109360 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1704043115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 0x7fcaf0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 -- 172.21.15.94:0/1704043115 >> 172.21.15.94:0/1704043115 conn(0x7fcaf00fdc00 msgr2=0x7fcaf0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 -- 172.21.15.94:0/1704043115 shutdown_connections 2024-04-25T13:50:33.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 -- 172.21.15.94:0/1704043115 wait complete. 2024-04-25T13:50:33.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.345+0000 7fcaf5496640 1 Processor -- start 2024-04-25T13:50:33.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 -- start start 2024-04-25T13:50:33.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 0x7fcaf0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 0x7fcaf0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010d440 con 0x7fcaf01098a0 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010d2c0 con 0x7fcaf01045e0 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaf5496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf010d5c0 con 0x7fcaf0106f40 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaeeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 0x7fcaf0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50662/0 (socket says 172.21.15.94:50662) 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaef7fe640 1 -- 172.21.15.94:0/1357520007 learned_addr learned my addr 172.21.15.94:0/1357520007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.346+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 0x7fcaf0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 -- 172.21.15.94:0/1357520007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 msgr2=0x7fcaf0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 0x7fcaf0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 -- 172.21.15.94:0/1357520007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 msgr2=0x7fcaf0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 0x7fcaf0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 -- 172.21.15.94:0/1357520007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf019fb70 con 0x7fcaf01098a0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcae800eb40 tx=0x7fcae800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 0x7fcaf0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcae7fff640 1 -- 172.21.15.94:0/1357520007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8004b20 con 0x7fcaf01098a0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf019fe60 con 0x7fcaf01098a0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcae7fff640 1 -- 172.21.15.94:0/1357520007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8004cc0 con 0x7fcaf01098a0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.347+0000 7fcae7fff640 1 -- 172.21.15.94:0/1357520007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8018760 con 0x7fcaf01098a0 2024-04-25T13:50:33.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.348+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01a0270 con 0x7fcaf01098a0 2024-04-25T13:50:33.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.351+0000 7fcae7fff640 1 -- 172.21.15.94:0/1357520007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcae8005020 con 0x7fcaf01098a0 2024-04-25T13:50:33.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.351+0000 7fcae7fff640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 0x7fcadc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcae7fff640 1 -- 172.21.15.94:0/1357520007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fcae80df270 con 0x7fcaf01098a0 2024-04-25T13:50:33.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 Processor -- start 2024-04-25T13:50:33.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 -- start start 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c7c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b96b0 0x562b39c7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b39be0d80 con 0x562b398da390 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b39b9aa20 con 0x562b398b96b0 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.352+0000 7fcafc429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b39bdcd40 con 0x562b398b14d0 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59530/0 (socket says 172.21.15.94:59530) 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1899398282 learned_addr learned my addr 172.21.15.94:0/1899398282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b96b0 0x562b39c7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 0x7fcadc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c7c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1899398282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b96b0 msgr2=0x562b39c7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.353+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b96b0 0x562b39c7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1899398282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 msgr2=0x562b39c7c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c7c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1899398282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b39ba5c30 con 0x562b398b14d0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c7c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 0x7fcadc0b2080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x562b39c7d860 tx=0x7fcad800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcae0032fd0 tx=0x7fcae0005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90bd297622e7be1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcae57fa640 1 -- 172.21.15.94:0/1899398282 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0035070 con 0x562b398b14d0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcae57fa640 1 -- 172.21.15.94:0/1899398282 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae00318a0 con 0x562b398b14d0 2024-04-25T13:50:33.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.354+0000 7fcae57fa640 1 -- 172.21.15.94:0/1899398282 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae003c440 con 0x562b398b14d0 2024-04-25T13:50:33.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.355+0000 7fcafc429740 1 -- 172.21.15.94:0/1899398282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 msgr2=0x562b39c7cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.355+0000 7fcafc429740 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcae0032fd0 tx=0x7fcae0005eb0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 -- 172.21.15.94:0/1899398282 shutdown_connections 2024-04-25T13:50:33.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b96b0 0x562b39c7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c7cd90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 --2- 172.21.15.94:0/1899398282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c7c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 -- 172.21.15.94:0/1899398282 >> 172.21.15.94:0/1899398282 conn(0x562b398ce4e0 msgr2=0x562b39c78020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 -- 172.21.15.94:0/1899398282 shutdown_connections 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 -- 172.21.15.94:0/1899398282 wait complete. 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 -- start start 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.356+0000 7fcafc429740 1 Processor -- start 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c86560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 0x562b39c86aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b39c7e4d0 con 0x562b398da390 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b39b67dc0 con 0x562b398b14d0 2024-04-25T13:50:33.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b39c7e650 con 0x562b398b96b0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcafc429740 10 client.0 _ll_register_callbacks cb 0x562b39b83600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcaef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50680/0 (socket says 172.21.15.94:50680) 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcaef7fe640 1 -- 172.21.15.94:0/2821086235 learned_addr learned my addr 172.21.15.94:0/2821086235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.357+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 0x562b39c86aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 -- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 msgr2=0x562b39c86560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c86560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c86560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 -- 172.21.15.94:0/2821086235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 msgr2=0x562b39c8da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 -- 172.21.15.94:0/2821086235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b398f23f0 con 0x562b398b96b0 2024-04-25T13:50:33.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 0x562b39c86aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005ab0 tx=0x7fcae0031720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12ab34ed01ca8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0058070 con 0x562b398b96b0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fcae00037f0 con 0x562b398b96b0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae00318a0 con 0x562b398b96b0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.358+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcae0064050 con 0x562b398b96b0 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 10 client.27958 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b399bc7d0 con 0x562b398b96b0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae00318a0 con 0x562b398b96b0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcae003c440 con 0x562b398b96b0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 10 client.27958 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 10 client.27958 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 20 client.27958 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b39898f60 0x562b39c93e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 4 client.27958 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b39bee990 con 0x562b39898f60 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.359+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b39898f60 0x562b39c93e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.360+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b39898f60 0x562b39c93e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51b38ba37e5c622a server_cookie=7d4df3611c150776 in_seq=0 out_seq=0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcad67fc640 10 client.27958 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcad67fc640 1 -- 172.21.15.94:0/2821086235 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b39bee990 con 0x562b39898f60 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcad67fc640 10 client.27958 handle_command_reply: tid=0 2024-04-25T13:50:33.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 client.27958 shutdown 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 msgr2=0x562b39c86aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 0x562b39c86aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005ab0 tx=0x7fcae0031720 comp rx=0 tx=0).stop 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 shutdown_connections 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b39898f60 0x562b39c93e50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398da390 0x562b39c8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b96b0 0x562b39c86aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 --2- 172.21.15.94:0/2821086235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c86560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 >> 172.21.15.94:0/2821086235 conn(0x562b398ce4e0 msgr2=0x562b39c7af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.361+0000 7fcad67fc640 0 client.27958 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.362+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 shutdown_connections 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.362+0000 7fcafc429740 1 -- 172.21.15.94:0/2821086235 wait complete. 2024-04-25T13:50:33.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.362+0000 7fcafc429740 20 client.27958 trim_cache size 0 max 16384 2024-04-25T13:50:33.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.383+0000 7fcafc429740 1 Processor -- start 2024-04-25T13:50:33.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.383+0000 7fcafc429740 1 -- start start 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398da390 0x562b39c8d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01a92e0 con 0x562b398b96b0 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01aae20 con 0x562b398b14d0 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcafc429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01a99b0 con 0x562b398da390 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50696/0 (socket says 172.21.15.94:50696) 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 -- 172.21.15.94:0/55700534 learned_addr learned my addr 172.21.15.94:0/55700534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398da390 0x562b39c8d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 -- 172.21.15.94:0/55700534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398da390 msgr2=0x562b39c8d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398da390 0x562b39c8d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 -- 172.21.15.94:0/55700534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 msgr2=0x562b39c99590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c99590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.384+0000 7fcaee7fc640 1 -- 172.21.15.94:0/55700534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b39c8de60 con 0x562b398b96b0 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.385+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcae0031500 tx=0x7fcae0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d62c5bfc9d474a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.385+0000 7fcae4ff9640 1 -- 172.21.15.94:0/55700534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0058070 con 0x562b398b96b0 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.385+0000 7fcae4ff9640 1 -- 172.21.15.94:0/55700534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae003c440 con 0x562b398b96b0 2024-04-25T13:50:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.385+0000 7fcae4ff9640 1 -- 172.21.15.94:0/55700534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0035030 con 0x562b398b96b0 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 -- 172.21.15.94:0/55700534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 msgr2=0x562b39c99ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcae0031500 tx=0x7fcae0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 -- 172.21.15.94:0/55700534 shutdown_connections 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398da390 0x562b39c8d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c99ad0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 --2- 172.21.15.94:0/55700534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398b14d0 0x562b39c99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 -- 172.21.15.94:0/55700534 >> 172.21.15.94:0/55700534 conn(0x562b398ce4e0 msgr2=0x562b39c812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 -- 172.21.15.94:0/55700534 shutdown_connections 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.386+0000 7fcafc429740 1 -- 172.21.15.94:0/55700534 wait complete. 2024-04-25T13:50:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 -- start start 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 Processor -- start 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c8d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398da390 0x562b39c8c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01aa750 con 0x562b398b96b0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01aa080 con 0x562b398da390 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf01a72c0 con 0x562b398b14d0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcafc429740 10 client.0 _ll_register_callbacks cb 0x7fcae8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcaee7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50710/0 (socket says 172.21.15.94:50710) 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1775233406 learned_addr learned my addr 172.21.15.94:0/1775233406 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcaeeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c8d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.387+0000 7fcaef7fe640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398da390 0x562b39c8c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1775233406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 msgr2=0x562b39c8d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c8d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398da390 msgr2=0x562b39c8c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398da390 0x562b39c8c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 -- 172.21.15.94:0/1775233406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b39c8cb10 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcaee7fc640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae0031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa72f8481a6919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae006d060 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fcae0051980 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae003c440 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcae0058070 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcafc429740 10 client.18648 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b39cfb060 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.388+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae003c440 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcae0051b80 con 0x562b398b96b0 2024-04-25T13:50:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 10 client.18648 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 10 client.18648 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 20 client.18648 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b3986e3a0 0x562b39cfda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 4 client.18648 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b39be8800 con 0x562b3986e3a0 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b3986e3a0 0x562b39cfda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcad67fc640 10 client.18648 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.389+0000 7fcaeeffd640 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b3986e3a0 0x562b39cfda70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b8fb31c8033c6d9 server_cookie=b1c4c4f193ab45d8 in_seq=0 out_seq=0 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcad67fc640 1 -- 172.21.15.94:0/1775233406 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b39be8800 con 0x562b3986e3a0 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcad67fc640 10 client.18648 handle_command_reply: tid=0 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 client.18648 shutdown 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 msgr2=0x562b39c8be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009900 tx=0x7fcae0031950 comp rx=0 tx=0).stop 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 shutdown_connections 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562b3986e3a0 0x562b39cfda70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b398da390 0x562b39c8c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcad67fc640 0 client.18648 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b398b96b0 0x562b39c8be50 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 --2- 172.21.15.94:0/1775233406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b398b14d0 0x562b39c8d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 >> 172.21.15.94:0/1775233406 conn(0x562b398ce4e0 msgr2=0x562b39c84cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 shutdown_connections 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 1 -- 172.21.15.94:0/1775233406 wait complete. 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.390+0000 7fcafc429740 20 client.18648 trim_cache size 0 max 16384 2024-04-25T13:50:33.395 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:33.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 msgr2=0x7fcadc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 0x7fcadc0b2080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x562b39c7d860 tx=0x7fcad800d040 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 msgr2=0x7fcaf019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcae800eb40 tx=0x7fcae800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 shutdown_connections 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcadc0afb70 0x7fcadc0b2080 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf01098a0 0x7fcaf019f500 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf0106f40 0x7fcaf0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 --2- 172.21.15.94:0/1357520007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf01045e0 0x7fcaf0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 >> 172.21.15.94:0/1357520007 conn(0x7fcaf00fdc00 msgr2=0x7fcaf0105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 shutdown_connections 2024-04-25T13:50:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.391+0000 7fcaf5496640 1 -- 172.21.15.94:0/1357520007 wait complete. 2024-04-25T13:50:33.398 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:33 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3194738748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:33.402 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:33 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3194738748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:33.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:50:33.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3194738748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 -- 172.21.15.94:0/3358028268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64108990 msgr2=0x7efd6410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 --2- 172.21.15.94:0/3358028268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64108990 0x7efd6410adb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009900 tx=0x7efd5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 -- 172.21.15.94:0/3358028268 shutdown_connections 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 --2- 172.21.15.94:0/3358028268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64108990 0x7efd6410adb0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 --2- 172.21.15.94:0/3358028268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 0x7efd641040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 --2- 172.21.15.94:0/3358028268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64102a50 0x7efd64103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 -- 172.21.15.94:0/3358028268 >> 172.21.15.94:0/3358028268 conn(0x7efd640fdc40 msgr2=0x7efd641000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 -- 172.21.15.94:0/3358028268 shutdown_connections 2024-04-25T13:50:33.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.531+0000 7efd68ab1640 1 -- 172.21.15.94:0/3358028268 wait complete. 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 Processor -- start 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 -- start start 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 0x7efd6419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 0x7efd641a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410f600 con 0x7efd64102a50 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410f480 con 0x7efd64103c90 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd68ab1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd6410f780 con 0x7efd64108990 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51408/0 (socket says 172.21.15.94:51408) 2024-04-25T13:50:33.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 -- 172.21.15.94:0/1642740711 learned_addr learned my addr 172.21.15.94:0/1642740711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd61d9b640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 0x7efd6419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd62d9d640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 0x7efd641a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 -- 172.21.15.94:0/1642740711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 msgr2=0x7efd641a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 0x7efd641a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 -- 172.21.15.94:0/1642740711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 msgr2=0x7efd6419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 0x7efd6419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 -- 172.21.15.94:0/1642740711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641a1cf0 con 0x7efd64102a50 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd62d9d640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 0x7efd641a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.532+0000 7efd6259c640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efd5400eb50 tx=0x7efd5400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.533+0000 7efd537fe640 1 -- 172.21.15.94:0/1642740711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54003b70 con 0x7efd64102a50 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.533+0000 7efd537fe640 1 -- 172.21.15.94:0/1642740711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5400ee50 con 0x7efd64102a50 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.533+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641a1fe0 con 0x7efd64102a50 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.533+0000 7efd537fe640 1 -- 172.21.15.94:0/1642740711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54010790 con 0x7efd64102a50 2024-04-25T13:50:33.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.533+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a23f0 con 0x7efd64102a50 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.535+0000 7efd6f5fd740 1 Processor -- start 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.535+0000 7efd6f5fd740 1 -- start start 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd537fe640 1 -- 172.21.15.94:0/1642740711 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efd54004020 con 0x7efd64102a50 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dd87314d80 con 0x55dd86fed6b0 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd872cea20 con 0x55dd86fe54d0 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd87310d40 con 0x55dd8700e390 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd61d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd6259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd61d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51410/0 (socket says 172.21.15.94:51410) 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd61d9b640 1 -- 172.21.15.94:0/2159105698 learned_addr learned my addr 172.21.15.94:0/2159105698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 -- 172.21.15.94:0/2159105698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 msgr2=0x55dd873b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 -- 172.21.15.94:0/2159105698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 msgr2=0x55dd873b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 -- 172.21.15.94:0/2159105698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd872d9c30 con 0x55dd8700e390 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd61d9b640 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd537fe640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 0x7efd4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.536+0000 7efd62d9d640 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b0810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd5c005940 tx=0x7efd5c032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2e5acfc7df201f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.537+0000 7efd537fe640 1 -- 172.21.15.94:0/1642740711 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efd540df570 con 0x7efd64102a50 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.537+0000 7efd6259c640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 0x7efd4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.537+0000 7efd50ff9640 1 -- 172.21.15.94:0/2159105698 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c03f030 con 0x55dd8700e390 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.537+0000 7efd50ff9640 1 -- 172.21.15.94:0/2159105698 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c032d10 con 0x55dd8700e390 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.537+0000 7efd50ff9640 1 -- 172.21.15.94:0/2159105698 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c035070 con 0x55dd8700e390 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- 172.21.15.94:0/2159105698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 msgr2=0x55dd873b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6259c640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 0x7efd4c0b2000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd540dd340 tx=0x7efd540e50e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b0810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd5c005940 tx=0x7efd5c032b20 comp rx=0 tx=0).stop 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- 172.21.15.94:0/2159105698 shutdown_connections 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/2159105698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b0810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- 172.21.15.94:0/2159105698 >> 172.21.15.94:0/2159105698 conn(0x55dd870024e0 msgr2=0x55dd873ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- 172.21.15.94:0/2159105698 shutdown_connections 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- 172.21.15.94:0/2159105698 wait complete. 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- start start 2024-04-25T13:50:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 Processor -- start 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dd873b24d0 con 0x55dd86fed6b0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd8729bdc0 con 0x55dd86fe54d0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.538+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd873b2650 con 0x55dd8700e390 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd6f5fd740 10 client.0 _ll_register_callbacks cb 0x55dd872b7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51414/0 (socket says 172.21.15.94:51414) 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 -- 172.21.15.94:0/550490845 learned_addr learned my addr 172.21.15.94:0/550490845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd62d9d640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 -- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 msgr2=0x55dd873c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 -- 172.21.15.94:0/550490845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 msgr2=0x55dd873ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 -- 172.21.15.94:0/550490845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd870263f0 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd62d9d640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd61d9b640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7efd58012740 tx=0x7efd58012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6a5461d3eb3d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5800f920 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efd5800fac0 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.539+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c032d10 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd5800fd00 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 10 client.18660 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd870f07d0 con 0x55dd86fed6b0 2024-04-25T13:50:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c032d10 con 0x55dd86fed6b0 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efd5801c420 con 0x55dd86fed6b0 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 10 client.18660 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 10 client.18660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 20 client.18660 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fccf60 0x55dd873c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 4 client.18660 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd87322990 con 0x55dd86fccf60 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.540+0000 7efd62d9d640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fccf60 0x55dd873c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.541+0000 7efd49ffb640 10 client.18660 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.541+0000 7efd62d9d640 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fccf60 0x55dd873c8380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b36887bf59b10113 server_cookie=bbf8f3833a957c84 in_seq=0 out_seq=0 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.541+0000 7efd49ffb640 1 -- 172.21.15.94:0/550490845 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd87322990 con 0x55dd86fccf60 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.541+0000 7efd49ffb640 10 client.18660 handle_command_reply: tid=0 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.541+0000 7efd6f5fd740 1 client.18660 shutdown 2024-04-25T13:50:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 msgr2=0x55dd873bab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7efd58012740 tx=0x7efd58012c00 comp rx=0 tx=0).stop 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 shutdown_connections 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fccf60 0x55dd873c8380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fed6b0 0x55dd873bab30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/550490845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fe54d0 0x55dd873ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 >> 172.21.15.94:0/550490845 conn(0x55dd870024e0 msgr2=0x55dd873aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd49ffb640 0 client.18660 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 shutdown_connections 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 1 -- 172.21.15.94:0/550490845 wait complete. 2024-04-25T13:50:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.542+0000 7efd6f5fd740 20 client.18660 trim_cache size 0 max 16384 2024-04-25T13:50:33.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.556+0000 7efd6f5fd740 1 Processor -- start 2024-04-25T13:50:33.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.556+0000 7efd6f5fd740 1 -- start start 2024-04-25T13:50:33.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ab320 con 0x55dd86fe54d0 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ace60 con 0x55dd86fed6b0 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ab9f0 con 0x55dd8700e390 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51428/0 (socket says 172.21.15.94:51428) 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 -- 172.21.15.94:0/3320086554 learned_addr learned my addr 172.21.15.94:0/3320086554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd61d9b640 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 -- 172.21.15.94:0/3320086554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 msgr2=0x55dd873b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 -- 172.21.15.94:0/3320086554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 msgr2=0x55dd873c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 -- 172.21.15.94:0/3320086554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd86fcd190 con 0x55dd86fe54d0 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd62d9d640 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7efd5c005f70 tx=0x7efd5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52404f6aaf64956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd4bfff640 1 -- 172.21.15.94:0/3320086554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c059070 con 0x55dd86fe54d0 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd4bfff640 1 -- 172.21.15.94:0/3320086554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c032c10 con 0x55dd86fe54d0 2024-04-25T13:50:33.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.557+0000 7efd4bfff640 1 -- 172.21.15.94:0/3320086554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c035040 con 0x55dd86fe54d0 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 -- 172.21.15.94:0/3320086554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 msgr2=0x55dd873b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7efd5c005f70 tx=0x7efd5c009260 comp rx=0 tx=0).stop 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 -- 172.21.15.94:0/3320086554 shutdown_connections 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd8700e390 0x55dd873b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/3320086554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd86fe54d0 0x55dd873b5350 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 -- 172.21.15.94:0/3320086554 >> 172.21.15.94:0/3320086554 conn(0x55dd870024e0 msgr2=0x55dd873b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 -- 172.21.15.94:0/3320086554 shutdown_connections 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.558+0000 7efd6f5fd740 1 -- 172.21.15.94:0/3320086554 wait complete. 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 -- start start 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 Processor -- start 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd86fe54d0 0x55dd873c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ac790 con 0x55dd8700e390 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ac0c0 con 0x55dd86fed6b0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd641a9300 con 0x55dd86fe54d0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd6f5fd740 10 client.0 _ll_register_callbacks cb 0x7efd58009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51438/0 (socket says 172.21.15.94:51438) 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd61d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 -- 172.21.15.94:0/608390105 learned_addr learned my addr 172.21.15.94:0/608390105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd86fe54d0 0x55dd873c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 -- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd86fe54d0 msgr2=0x55dd873c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd86fe54d0 0x55dd873c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 -- 172.21.15.94:0/608390105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 msgr2=0x55dd873c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 -- 172.21.15.94:0/608390105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd873c12b0 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd62d9d640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64cbdbcd74a0c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.559+0000 7efd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c064060 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7efd5c031c50 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c032c10 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd5c035050 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 10 client.18666 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd8742fc70 con 0x55dd8700e390 2024-04-25T13:50:33.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c032c10 con 0x55dd8700e390 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efd5c053040 con 0x55dd8700e390 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 10 client.18666 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 10 client.18666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 20 client.18666 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fa23a0 0x55dd87432170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 4 client.18666 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd8731c800 con 0x55dd86fa23a0 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.560+0000 7efd61d9b640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fa23a0 0x55dd87432170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd49ffb640 10 client.18666 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd61d9b640 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fa23a0 0x55dd87432170 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6d2e29422eaf9af server_cookie=d8a6a433e634fe8 in_seq=0 out_seq=0 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd49ffb640 1 -- 172.21.15.94:0/608390105 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd8731c800 con 0x55dd86fa23a0 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd49ffb640 10 client.18666 handle_command_reply: tid=0 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 client.18666 shutdown 2024-04-25T13:50:33.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 msgr2=0x55dd8742e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c009620 comp rx=0 tx=0).stop 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 shutdown_connections 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dd86fa23a0 0x55dd87432170 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd8700e390 0x55dd8742e6e0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd49ffb640 0 client.18666 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd86fed6b0 0x55dd873c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 --2- 172.21.15.94:0/608390105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd86fe54d0 0x55dd873c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 >> 172.21.15.94:0/608390105 conn(0x55dd870024e0 msgr2=0x55dd873b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 shutdown_connections 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 1 -- 172.21.15.94:0/608390105 wait complete. 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.561+0000 7efd6f5fd740 20 client.18666 trim_cache size 0 max 16384 2024-04-25T13:50:33.568 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:33.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 msgr2=0x7efd4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 0x7efd4c0b2000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd540dd340 tx=0x7efd540e50e0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 msgr2=0x7efd6419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efd5400eb50 tx=0x7efd5400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 shutdown_connections 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efd4c0afaf0 0x7efd4c0b2000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd64108990 0x7efd641a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd64103c90 0x7efd6419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 --2- 172.21.15.94:0/1642740711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd64102a50 0x7efd6419a1b0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 >> 172.21.15.94:0/1642740711 conn(0x7efd640fdc40 msgr2=0x7efd64104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 shutdown_connections 2024-04-25T13:50:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.562+0000 7efd68ab1640 1 -- 172.21.15.94:0/1642740711 wait complete. 2024-04-25T13:50:33.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/2099363874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8106f40 msgr2=0x7fe7b8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/2099363874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8106f40 0x7fe7b8109360 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009900 tx=0x7fe7a802a410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/2099363874 shutdown_connections 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/2099363874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b81098a0 0x7fe7b810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/2099363874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b8106f40 0x7fe7b8109360 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/2099363874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81045e0 0x7fe7b8106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/2099363874 >> 172.21.15.94:0/2099363874 conn(0x7fe7b80fdc00 msgr2=0x7fe7b8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/2099363874 shutdown_connections 2024-04-25T13:50:33.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.686+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/2099363874 wait complete. 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 Processor -- start 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 -- start start 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8106f40 0x7fe7b819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 0x7fe7b81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b810f620 con 0x7fe7b81045e0 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b810f4a0 con 0x7fe7b81098a0 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bf5a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b810f7a0 con 0x7fe7b8106f40 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51460/0 (socket says 172.21.15.94:51460) 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 -- 172.21.15.94:0/4047472409 learned_addr learned my addr 172.21.15.94:0/4047472409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 0x7fe7b81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8106f40 0x7fe7b819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 -- 172.21.15.94:0/4047472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8106f40 msgr2=0x7fe7b819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8106f40 0x7fe7b819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 -- 172.21.15.94:0/4047472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 msgr2=0x7fe7b81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 0x7fe7b81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bd344640 1 -- 172.21.15.94:0/4047472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81a1d60 con 0x7fe7b81045e0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.687+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 0x7fe7b81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7bd344640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400eb90 tx=0x7fe7b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7ae7fc640 1 -- 172.21.15.94:0/4047472409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4010630 con 0x7fe7b81045e0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7ae7fc640 1 -- 172.21.15.94:0/4047472409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b400ee90 con 0x7fe7b81045e0 2024-04-25T13:50:33.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7ae7fc640 1 -- 172.21.15.94:0/4047472409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4018ac0 con 0x7fe7b81045e0 2024-04-25T13:50:33.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81a1ff0 con 0x7fe7b81045e0 2024-04-25T13:50:33.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.688+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a2460 con 0x7fe7b81045e0 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7ae7fc640 1 -- 172.21.15.94:0/4047472409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe7b4018c60 con 0x7fe7b81045e0 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 Processor -- start 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 -- start start 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e030624d0 0x563e0342dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 0x563e0342ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e03391d80 con 0x563e0308b390 2024-04-25T13:50:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563e0334ba20 con 0x563e030624d0 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7c645c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563e0338dd40 con 0x563e0306a6b0 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51474/0 (socket says 172.21.15.94:51474) 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7bcb43640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 0x563e0342ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e030624d0 0x563e0342dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.691+0000 7fe7bdb45640 1 -- 172.21.15.94:0/181771001 learned_addr learned my addr 172.21.15.94:0/181771001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7ae7fc640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 0x7fe7a40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 -- 172.21.15.94:0/181771001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 msgr2=0x563e0342ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 0x563e0342ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 -- 172.21.15.94:0/181771001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e030624d0 msgr2=0x563e0342dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e030624d0 0x563e0342dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 -- 172.21.15.94:0/181771001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e03356c30 con 0x563e0308b390 2024-04-25T13:50:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 0x563e0342ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7ae7fc640 1 -- 172.21.15.94:0/4047472409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b40df190 con 0x7fe7b81045e0 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0011470 tx=0x7fe7b0011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc7ebd91d7cf939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 0x7fe7a40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7a37fe640 1 -- 172.21.15.94:0/181771001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000f6c0 con 0x563e0308b390 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7a37fe640 1 -- 172.21.15.94:0/181771001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0019490 con 0x563e0308b390 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.692+0000 7fe7a37fe640 1 -- 172.21.15.94:0/181771001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0018740 con 0x563e0308b390 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- 172.21.15.94:0/181771001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 msgr2=0x563e0342d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 0x7fe7a40b2080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe7b001c000 tx=0x7fe7b0018a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0011470 tx=0x7fe7b0011930 comp rx=0 tx=0).stop 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- 172.21.15.94:0/181771001 shutdown_connections 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0306a6b0 0x563e0342ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e030624d0 0x563e0342dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 --2- 172.21.15.94:0/181771001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0342d810 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- 172.21.15.94:0/181771001 >> 172.21.15.94:0/181771001 conn(0x563e0307f4e0 msgr2=0x563e03429020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- 172.21.15.94:0/181771001 shutdown_connections 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- 172.21.15.94:0/181771001 wait complete. 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.693+0000 7fe7c645c740 1 -- start start 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 Processor -- start 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03437840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0308b390 0x563e0343e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e0342f530 con 0x563e030624d0 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563e03318dc0 con 0x563e0306a6b0 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563e0342f6b0 con 0x563e0308b390 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7c645c740 10 client.0 _ll_register_callbacks cb 0x563e03334600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51478/0 (socket says 172.21.15.94:51478) 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 -- 172.21.15.94:0/3246238099 learned_addr learned my addr 172.21.15.94:0/3246238099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03437840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0308b390 0x563e0343e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 -- 172.21.15.94:0/3246238099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0308b390 msgr2=0x563e0343e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0308b390 0x563e0343e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 -- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 msgr2=0x563e03437840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03437840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 -- 172.21.15.94:0/3246238099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e030a33f0 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.694+0000 7fe7bd344640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f80 tx=0x7fe7b40184e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b871f62121e71492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40a6df0 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b40900f0 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0019490 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7b40a7190 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 10 client.18678 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e0316d7d0 con 0x563e030624d0 2024-04-25T13:50:33.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0019490 con 0x563e030624d0 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b408eb20 con 0x563e030624d0 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 10 client.18678 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 10 client.18678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 20 client.18678 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e03049f60 0x563e03444f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 4 client.18678 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e0339f990 con 0x563e03049f60 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.695+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e03049f60 0x563e03444f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.696+0000 7fe7a0ff9640 10 client.18678 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.696+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e03049f60 0x563e03444f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15d98d6b2440f9fe server_cookie=eaae80f6a2732d9d in_seq=0 out_seq=0 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/3246238099 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e0339f990 con 0x563e03049f60 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7a0ff9640 10 client.18678 handle_command_reply: tid=0 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 client.18678 shutdown 2024-04-25T13:50:33.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 msgr2=0x563e03437300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f80 tx=0x7fe7b40184e0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 shutdown_connections 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e03049f60 0x563e03444f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e0308b390 0x563e0343e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7a0ff9640 0 client.18678 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03437840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 --2- 172.21.15.94:0/3246238099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e030624d0 0x563e03437300 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 >> 172.21.15.94:0/3246238099 conn(0x563e0307f4e0 msgr2=0x563e0342bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 shutdown_connections 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 1 -- 172.21.15.94:0/3246238099 wait complete. 2024-04-25T13:50:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.697+0000 7fe7c645c740 20 client.18678 trim_cache size 0 max 16384 2024-04-25T13:50:33.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 Processor -- start 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 -- start start 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e03428a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03428f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81ab5f0 con 0x563e0308b390 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81ad130 con 0x563e0306a6b0 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7c645c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81abcc0 con 0x563e030624d0 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51488/0 (socket says 172.21.15.94:51488) 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 -- 172.21.15.94:0/2980836536 learned_addr learned my addr 172.21.15.94:0/2980836536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03428f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bd344640 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e03428a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 -- 172.21.15.94:0/2980836536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 msgr2=0x563e03428a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e03428a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 -- 172.21.15.94:0/2980836536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 msgr2=0x563e03428f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03428f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 -- 172.21.15.94:0/2980836536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e034383a0 con 0x563e0308b390 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0011970 tx=0x7fe7b0029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb59c0d5b5cc2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.711+0000 7fe7a2ffd640 1 -- 172.21.15.94:0/2980836536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0029070 con 0x563e0308b390 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7a2ffd640 1 -- 172.21.15.94:0/2980836536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b00183b0 con 0x563e0308b390 2024-04-25T13:50:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7a2ffd640 1 -- 172.21.15.94:0/2980836536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0005610 con 0x563e0308b390 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 -- 172.21.15.94:0/2980836536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 msgr2=0x563e03436ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0011970 tx=0x7fe7b0029000 comp rx=0 tx=0).stop 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 -- 172.21.15.94:0/2980836536 shutdown_connections 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e03436ed0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e03428f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 --2- 172.21.15.94:0/2980836536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e03428a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 -- 172.21.15.94:0/2980836536 >> 172.21.15.94:0/2980836536 conn(0x563e0307f4e0 msgr2=0x563e03432080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 -- 172.21.15.94:0/2980836536 shutdown_connections 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.712+0000 7fe7c645c740 1 -- 172.21.15.94:0/2980836536 wait complete. 2024-04-25T13:50:33.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 -- start start 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 Processor -- start 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e0343b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e0343b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81aca60 con 0x563e0308b390 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81ac390 con 0x563e0306a6b0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81a95d0 con 0x563e030624d0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7c645c740 10 client.0 _ll_register_callbacks cb 0x7fe7b40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51492/0 (socket says 172.21.15.94:51492) 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 -- 172.21.15.94:0/1993274560 learned_addr learned my addr 172.21.15.94:0/1993274560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bd344640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e0343b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bcb43640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e0343b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 -- 172.21.15.94:0/1993274560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 msgr2=0x563e0343b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e0343b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 -- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 msgr2=0x563e0343b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e0343b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.713+0000 7fe7bdb45640 1 -- 172.21.15.94:0/1993274560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0343cd90 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7bdb45640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005170 tx=0x7fe7b000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd02ce80327e93c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0045060 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b0011040 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b00183b0 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7b0011280 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 10 client.18684 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e0343e010 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b00183b0 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b00037b0 con 0x563e0308b390 2024-04-25T13:50:33.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 10 client.18684 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 10 client.18684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 20 client.18684 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e0301f3a0 0x563e034af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 4 client.18684 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.714+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e03399800 con 0x563e0301f3a0 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7bd344640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e0301f3a0 0x563e034af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7a0ff9640 10 client.18684 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7bd344640 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e0301f3a0 0x563e034af7d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2de7cc24e278a415 server_cookie=d20c48b880c6d6c7 in_seq=0 out_seq=0 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7a0ff9640 1 -- 172.21.15.94:0/1993274560 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e03399800 con 0x563e0301f3a0 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7a0ff9640 10 client.18684 handle_command_reply: tid=0 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 client.18684 shutdown 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 msgr2=0x563e0343c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005170 tx=0x7fe7b000f7d0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 shutdown_connections 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x563e0301f3a0 0x563e034af7d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e0308b390 0x563e0343c7d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e0306a6b0 0x563e0343b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 --2- 172.21.15.94:0/1993274560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e030624d0 0x563e0343b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 >> 172.21.15.94:0/1993274560 conn(0x563e0307f4e0 msgr2=0x563e03432080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7a0ff9640 0 client.18684 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 shutdown_connections 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.715+0000 7fe7c645c740 1 -- 172.21.15.94:0/1993274560 wait complete. 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7c645c740 20 client.18684 trim_cache size 0 max 16384 2024-04-25T13:50:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:33.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 msgr2=0x7fe7a40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 0x7fe7a40b2080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe7b001c000 tx=0x7fe7b0018a90 comp rx=0 tx=0).stop 2024-04-25T13:50:33.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 msgr2=0x7fe7b819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400eb90 tx=0x7fe7b400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 shutdown_connections 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7a40afb70 0x7fe7a40b2080 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7b81098a0 0x7fe7b81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8106f40 0x7fe7b819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 --2- 172.21.15.94:0/4047472409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7b81045e0 0x7fe7b819a220 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 >> 172.21.15.94:0/4047472409 conn(0x7fe7b80fdc00 msgr2=0x7fe7b8105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 shutdown_connections 2024-04-25T13:50:33.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.716+0000 7fe7bf5a8640 1 -- 172.21.15.94:0/4047472409 wait complete. 2024-04-25T13:50:33.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:50:33.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1057976 12.88 MB/sec execute 316 sec latency 85.715 ms 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 -- 172.21.15.94:0/84642145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda901045e0 msgr2=0x7fda90106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 --2- 172.21.15.94:0/84642145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda901045e0 0x7fda90106a00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fda80009ca0 tx=0x7fda8002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 -- 172.21.15.94:0/84642145 shutdown_connections 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 --2- 172.21.15.94:0/84642145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901098a0 0x7fda9010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 --2- 172.21.15.94:0/84642145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda90106f40 0x7fda90109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 --2- 172.21.15.94:0/84642145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda901045e0 0x7fda90106a00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 -- 172.21.15.94:0/84642145 >> 172.21.15.94:0/84642145 conn(0x7fda900fdc00 msgr2=0x7fda90100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 -- 172.21.15.94:0/84642145 shutdown_connections 2024-04-25T13:50:33.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.841+0000 7fda94ab1640 1 -- 172.21.15.94:0/84642145 wait complete. 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 Processor -- start 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 -- start start 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901045e0 0x7fda9019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda901098a0 0x7fda901a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda9010f5d0 con 0x7fda90106f40 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda9010f450 con 0x7fda901045e0 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda94ab1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda9010f750 con 0x7fda901098a0 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51512/0 (socket says 172.21.15.94:51512) 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 -- 172.21.15.94:0/2354617185 learned_addr learned my addr 172.21.15.94:0/2354617185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8e59c640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901045e0 0x7fda9019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda901098a0 0x7fda901a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 -- 172.21.15.94:0/2354617185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda901098a0 msgr2=0x7fda901a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda901098a0 0x7fda901a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 -- 172.21.15.94:0/2354617185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901045e0 msgr2=0x7fda9019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901045e0 0x7fda9019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.842+0000 7fda8dd9b640 1 -- 172.21.15.94:0/2354617185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda901a1d70 con 0x7fda90106f40 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fda8400e9f0 tx=0x7fda8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda7f7fe640 1 -- 172.21.15.94:0/2354617185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8400cd80 con 0x7fda90106f40 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda7f7fe640 1 -- 172.21.15.94:0/2354617185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda840108e0 con 0x7fda90106f40 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda901a2060 con 0x7fda90106f40 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda7f7fe640 1 -- 172.21.15.94:0/2354617185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84018980 con 0x7fda90106f40 2024-04-25T13:50:33.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.843+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda901a25b0 con 0x7fda90106f40 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.845+0000 7fda9b5fd740 1 Processor -- start 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.845+0000 7fda9b5fd740 1 -- start start 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.845+0000 7fda7f7fe640 1 -- 172.21.15.94:0/2354617185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fda84010430 con 0x7fda90106f40 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 0x55a41be57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a41bdb6d80 con 0x55a41ba8f6b0 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a41bd70a20 con 0x55a41bab0390 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a41bdb2d40 con 0x55a41ba874d0 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 0x55a41be57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39792/0 (socket says 172.21.15.94:39792) 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 -- 172.21.15.94:0/68307261 learned_addr learned my addr 172.21.15.94:0/68307261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 0x55a41be57e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51528/0 (socket says 172.21.15.94:51528) 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8e59c640 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 -- 172.21.15.94:0/68307261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 msgr2=0x55a41be571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 -- 172.21.15.94:0/68307261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 msgr2=0x55a41be57e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 0x55a41be57e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 -- 172.21.15.94:0/68307261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41bd7bc30 con 0x55a41bab0390 2024-04-25T13:50:33.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8e59c640 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda7f7fe640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 0x7fda780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8e59c640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 0x7fda780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda88009c20 tx=0x7fda88011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5f077db450c4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.846+0000 7fda7f7fe640 1 -- 172.21.15.94:0/2354617185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fda840def90 con 0x7fda90106f40 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda7cff9640 1 -- 172.21.15.94:0/68307261 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8801e070 con 0x55a41bab0390 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda7cff9640 1 -- 172.21.15.94:0/68307261 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda88019ae0 con 0x55a41bab0390 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda7cff9640 1 -- 172.21.15.94:0/68307261 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8800e870 con 0x55a41bab0390 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda9b5fd740 1 -- 172.21.15.94:0/68307261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 msgr2=0x55a41be56c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda88009c20 tx=0x7fda88011810 comp rx=0 tx=0).stop 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.847+0000 7fda8e59c640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 0x7fda780b2000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a41be56940 tx=0x7fda80009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- 172.21.15.94:0/68307261 shutdown_connections 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba8f6b0 0x55a41be57e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/68307261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be56c20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- 172.21.15.94:0/68307261 >> 172.21.15.94:0/68307261 conn(0x55a41baa44e0 msgr2=0x55a41be4e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- 172.21.15.94:0/68307261 shutdown_connections 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- 172.21.15.94:0/68307261 wait complete. 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- start start 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 Processor -- start 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be67f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a41be58940 con 0x55a41bab0390 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a41bd3ddc0 con 0x55a41ba8f6b0 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a41be58ac0 con 0x55a41ba874d0 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.848+0000 7fda9b5fd740 10 client.0 _ll_register_callbacks cb 0x55a41bd59600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be67f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:44594/0 (socket says 172.21.15.94:44594) 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8e59c640 1 -- 172.21.15.94:0/3054866995 learned_addr learned my addr 172.21.15.94:0/3054866995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 -- 172.21.15.94:0/3054866995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 msgr2=0x55a41be60a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 -- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 msgr2=0x55a41be60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be60fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 -- 172.21.15.94:0/3054866995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41bac83f0 con 0x55a41bab0390 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8e59c640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be67f00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fda88009bf0 tx=0x7fda88009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3700c2d34a192e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88029020 con 0x55a41bab0390 2024-04-25T13:50:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fda880054d0 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda88019ae0 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda8801e050 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda9b5fd740 10 client.18696 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.849+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a41bb927d0 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda88019ae0 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda8800ea00 con 0x55a41bab0390 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 10 client.18696 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 10 client.18696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 20 client.18696 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba6ef60 0x55a41be6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 4 client.18696 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a41bdc4990 con 0x55a41ba6ef60 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda8e59c640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba6ef60 0x55a41be6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda75ffb640 10 client.18696 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.850+0000 7fda8e59c640 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba6ef60 0x55a41be6e520 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e402b1c5c9c32f9b server_cookie=44d35ab68c84e973 in_seq=0 out_seq=0 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda75ffb640 1 -- 172.21.15.94:0/3054866995 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a41bdc4990 con 0x55a41ba6ef60 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda75ffb640 10 client.18696 handle_command_reply: tid=0 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 client.18696 shutdown 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 msgr2=0x55a41be67f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be67f00 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fda88009bf0 tx=0x7fda88009e90 comp rx=0 tx=0).stop 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 shutdown_connections 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba6ef60 0x55a41be6e520 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be67f00 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/3054866995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be60a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda75ffb640 0 client.18696 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 >> 172.21.15.94:0/3054866995 conn(0x55a41baa44e0 msgr2=0x55a41be4d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.851+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 shutdown_connections 2024-04-25T13:50:33.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.852+0000 7fda9b5fd740 1 -- 172.21.15.94:0/3054866995 wait complete. 2024-04-25T13:50:33.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.852+0000 7fda9b5fd740 20 client.18696 trim_cache size 0 max 16384 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.865+0000 7fda9b5fd740 1 Processor -- start 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.865+0000 7fda9b5fd740 1 -- start start 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be61170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda901a9420 con 0x55a41bab0390 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda901aaf60 con 0x55a41ba8f6b0 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda901a9af0 con 0x55a41ba874d0 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51556/0 (socket says 172.21.15.94:51556) 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 -- 172.21.15.94:0/699278800 learned_addr learned my addr 172.21.15.94:0/699278800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8e59c640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be61170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 -- 172.21.15.94:0/699278800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 msgr2=0x55a41be49190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be49190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 -- 172.21.15.94:0/699278800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 msgr2=0x55a41be61170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be61170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 -- 172.21.15.94:0/699278800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a41be5c230 con 0x55a41bab0390 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be61170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda8800edd0 tx=0x7fda88009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745ecc47a5f162c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda77fff640 1 -- 172.21.15.94:0/699278800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88029040 con 0x55a41bab0390 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda77fff640 1 -- 172.21.15.94:0/699278800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8801f040 con 0x55a41bab0390 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.866+0000 7fda77fff640 1 -- 172.21.15.94:0/699278800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8800e3b0 con 0x55a41bab0390 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 -- 172.21.15.94:0/699278800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 msgr2=0x55a41be5f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda8800edd0 tx=0x7fda88009af0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 -- 172.21.15.94:0/699278800 shutdown_connections 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41bab0390 0x55a41be5f0c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41ba8f6b0 0x55a41be61170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/699278800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba874d0 0x55a41be49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 -- 172.21.15.94:0/699278800 >> 172.21.15.94:0/699278800 conn(0x55a41baa44e0 msgr2=0x55a41be60a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 -- 172.21.15.94:0/699278800 shutdown_connections 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.867+0000 7fda9b5fd740 1 -- 172.21.15.94:0/699278800 wait complete. 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 -- start start 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 Processor -- start 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba8f6b0 0x55a41be67290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be63940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda901aa890 con 0x55a41ba874d0 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda901aa1c0 con 0x55a41bab0390 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda901a7400 con 0x55a41ba8f6b0 2024-04-25T13:50:33.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda9b5fd740 10 client.0 _ll_register_callbacks cb 0x7fda8801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51558/0 (socket says 172.21.15.94:51558) 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 -- 172.21.15.94:0/1703084696 learned_addr learned my addr 172.21.15.94:0/1703084696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8ed9d640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be63940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba8f6b0 0x55a41be67290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 -- 172.21.15.94:0/1703084696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba8f6b0 msgr2=0x55a41be67290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba8f6b0 0x55a41be67290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 -- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 msgr2=0x55a41be63940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be63940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 -- 172.21.15.94:0/1703084696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41be63f80 con 0x55a41ba874d0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda8e59c640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fda8002ade0 tx=0x7fda800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeccddcffcf6d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.868+0000 7fda75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80059070 con 0x55a41ba874d0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fda80003af0 con 0x55a41ba874d0 2024-04-25T13:50:33.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda8801f040 con 0x55a41ba874d0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda80065050 con 0x55a41ba874d0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 10 client.18702 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a41bed5420 con 0x55a41ba874d0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8801f040 con 0x55a41ba874d0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fda800041d0 con 0x55a41ba874d0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 10 client.18702 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 10 client.18702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 20 client.18702 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba443a0 0x55a41bed7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 4 client.18702 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a41bdbe800 con 0x55a41ba443a0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.869+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba443a0 0x55a41bed7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.870+0000 7fda8dd9b640 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba443a0 0x55a41bed7df0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e187d65733f6339f server_cookie=5600a388afdfbf6b in_seq=0 out_seq=0 2024-04-25T13:50:33.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.870+0000 7fda75ffb640 10 client.18702 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda75ffb640 1 -- 172.21.15.94:0/1703084696 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a41bdbe800 con 0x55a41ba443a0 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda75ffb640 10 client.18702 handle_command_reply: tid=0 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 client.18702 shutdown 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 msgr2=0x55a41be66d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fda8002ade0 tx=0x7fda800095c0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 shutdown_connections 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a41ba443a0 0x55a41bed7df0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a41bab0390 0x55a41be63940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a41ba8f6b0 0x55a41be67290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda75ffb640 0 client.18702 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 --2- 172.21.15.94:0/1703084696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a41ba874d0 0x55a41be66d50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 >> 172.21.15.94:0/1703084696 conn(0x55a41baa44e0 msgr2=0x55a41be50e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 shutdown_connections 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 1 -- 172.21.15.94:0/1703084696 wait complete. 2024-04-25T13:50:33.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda9b5fd740 20 client.18702 trim_cache size 0 max 16384 2024-04-25T13:50:33.877 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:50:33.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 msgr2=0x7fda780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 0x7fda780b2000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a41be56940 tx=0x7fda80009360 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 msgr2=0x7fda9019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fda8400e9f0 tx=0x7fda8400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 shutdown_connections 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.871+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fda780afaf0 0x7fda780b2000 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda901098a0 0x7fda901a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda90106f40 0x7fda9019a770 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 --2- 172.21.15.94:0/2354617185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda901045e0 0x7fda9019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 >> 172.21.15.94:0/2354617185 conn(0x7fda900fdc00 msgr2=0x7fda901051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 shutdown_connections 2024-04-25T13:50:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:33.872+0000 7fda94ab1640 1 -- 172.21.15.94:0/2354617185 wait complete. 2024-04-25T13:50:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:34 smithi137 ceph-mon[32568]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 341 op/s 2024-04-25T13:50:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:34 smithi094 ceph-mon[29195]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 341 op/s 2024-04-25T13:50:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:34 smithi138 ceph-mon[32977]: pgmap v610: 97 pgs: 97 active+clean; 44 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 341 op/s 2024-04-25T13:50:34.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1059820 12.87 MB/sec execute 317 sec latency 18.562 ms 2024-04-25T13:50:34.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:50:34.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:34.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 -- 172.21.15.94:0/481498899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 msgr2=0x7f3bc41054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 --2- 172.21.15.94:0/481498899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41054c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009900 tx=0x7f3bc002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 -- 172.21.15.94:0/481498899 shutdown_connections 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 --2- 172.21.15.94:0/481498899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 --2- 172.21.15.94:0/481498899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 --2- 172.21.15.94:0/481498899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41054c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 -- 172.21.15.94:0/481498899 >> 172.21.15.94:0/481498899 conn(0x7f3bc40fd380 msgr2=0x7f3bc40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 -- 172.21.15.94:0/481498899 shutdown_connections 2024-04-25T13:50:35.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.000+0000 7f3bcc340640 1 -- 172.21.15.94:0/481498899 wait complete. 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 Processor -- start 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 -- start start 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410f620 con 0x7f3bc41050b0 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410f4a0 con 0x7f3bc4101760 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bcc340640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410f7a0 con 0x7f3bc4102260 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bca8dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bca8dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51576/0 (socket says 172.21.15.94:51576) 2024-04-25T13:50:35.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bca8dd640 1 -- 172.21.15.94:0/1231356182 learned_addr learned my addr 172.21.15.94:0/1231356182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.001+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 -- 172.21.15.94:0/1231356182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 msgr2=0x7f3bc419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 -- 172.21.15.94:0/1231356182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 msgr2=0x7f3bc419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 -- 172.21.15.94:0/1231356182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41a1d60 con 0x7f3bc41050b0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bca8dd640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00eb10 tx=0x7f3bbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bbb7fe640 1 -- 172.21.15.94:0/1231356182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004b20 con 0x7f3bc41050b0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bbb7fe640 1 -- 172.21.15.94:0/1231356182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc004cc0 con 0x7f3bc41050b0 2024-04-25T13:50:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41a1ff0 con 0x7f3bc41050b0 2024-04-25T13:50:35.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bbb7fe640 1 -- 172.21.15.94:0/1231356182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc010690 con 0x7f3bc41050b0 2024-04-25T13:50:35.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.002+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a24b0 con 0x7f3bc41050b0 2024-04-25T13:50:35.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 Processor -- start 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bbb7fe640 1 -- 172.21.15.94:0/1231356182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3bbc0108f0 con 0x7f3bc41050b0 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 -- start start 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14cbe410 0x563f1505f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f15066920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f14fc4b00 con 0x563f14c9d730 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f14f60920 con 0x563f14c95550 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.005+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f14fc0ac0 con 0x563f14cbe410 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f15066920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39862/0 (socket says 172.21.15.94:39862) 2024-04-25T13:50:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca0dc640 1 -- 172.21.15.94:0/2006979165 learned_addr learned my addr 172.21.15.94:0/2006979165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14cbe410 0x563f1505f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bbb7fe640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 0x7f3bb00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca8dd640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 0x7f3bb00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 -- 172.21.15.94:0/2006979165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14cbe410 msgr2=0x563f1505f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14cbe410 0x563f1505f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 -- 172.21.15.94:0/2006979165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 msgr2=0x563f1505fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bbb7fe640 1 -- 172.21.15.94:0/1231356182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3bbc0df230 con 0x7f3bc41050b0 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 -- 172.21.15.94:0/2006979165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f14d9e7e0 con 0x563f14c9d730 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f15066920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009730 tx=0x7f3bb40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c09b04c10068737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bb8bfe640 1 -- 172.21.15.94:0/2006979165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4011050 con 0x563f14c9d730 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bca8dd640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 0x7f3bb00b2000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0a7600 tx=0x7f3bbc06db90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bb8bfe640 1 -- 172.21.15.94:0/2006979165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bb40054d0 con 0x563f14c9d730 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.006+0000 7f3bb8bfe640 1 -- 172.21.15.94:0/2006979165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400fe80 con 0x563f14c9d730 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2006979165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 msgr2=0x563f15066920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f15066920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009730 tx=0x7f3bb40096c0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2006979165 shutdown_connections 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f15066920 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c95550 0x563f1505fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2006979165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14cbe410 0x563f1505f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2006979165 >> 172.21.15.94:0/2006979165 conn(0x563f14cb2560 msgr2=0x563f1505ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2006979165 shutdown_connections 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.007+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2006979165 wait complete. 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 -- start start 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 Processor -- start 2024-04-25T13:50:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c95550 0x563f150692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 0x563f15070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f15067370 con 0x563f14c9d730 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f14f5e5d0 con 0x563f14cbe410 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f150674f0 con 0x563f14c95550 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bd31f0740 10 client.0 _ll_register_callbacks cb 0x563f14f4c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51592/0 (socket says 172.21.15.94:51592) 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 -- 172.21.15.94:0/2833189137 learned_addr learned my addr 172.21.15.94:0/2833189137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c95550 0x563f150692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 0x563f15070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 -- 172.21.15.94:0/2833189137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c95550 msgr2=0x563f150692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c95550 0x563f150692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 -- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 msgr2=0x563f15070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 0x563f15070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bca0dc640 1 -- 172.21.15.94:0/2833189137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f14dfef70 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.008+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 0x563f15070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3bc002a930 tx=0x7f3bc00317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905a87438d6503a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0035040 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3bc0004450 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb40054d0 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3bc00035f0 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 10 client.18714 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f14c96170 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bb40054d0 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc0044070 con 0x563f14c9d730 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 10 client.18714 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 10 client.18714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 20 client.18714 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14c7d1f0 0x563f15076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 4 client.18714 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.009+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f14fd2710 con 0x563f14c7d1f0 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.010+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14c7d1f0 0x563f15076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.010+0000 7f3bc98db640 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14c7d1f0 0x563f15076bd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4d7754de77b973 server_cookie=5dc4e8855de8eddf in_seq=0 out_seq=0 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.010+0000 7f3badffb640 10 client.18714 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.011+0000 7f3badffb640 1 -- 172.21.15.94:0/2833189137 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f14fd2710 con 0x563f14c7d1f0 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.011+0000 7f3badffb640 10 client.18714 handle_command_reply: tid=0 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.011+0000 7f3bd31f0740 1 client.18714 shutdown 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 msgr2=0x563f150697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3bc002a930 tx=0x7f3bc00317d0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 shutdown_connections 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14c7d1f0 0x563f15076bd0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14cbe410 0x563f15070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3badffb640 0 client.18714 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c9d730 0x563f150697e0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/2833189137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c95550 0x563f150692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 >> 172.21.15.94:0/2833189137 conn(0x563f14cb2560 msgr2=0x563f15059ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 shutdown_connections 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 1 -- 172.21.15.94:0/2833189137 wait complete. 2024-04-25T13:50:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.012+0000 7f3bd31f0740 20 client.18714 trim_cache size 0 max 16384 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 Processor -- start 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 -- start start 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 0x563f150594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c9d730 0x563f15070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41ab4a0 con 0x563f14c95550 2024-04-25T13:50:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41acfe0 con 0x563f14c9d730 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41abb70 con 0x563f14c7d1f0 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.030+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bc98db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 0x563f150594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51600/0 (socket says 172.21.15.94:51600) 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 -- 172.21.15.94:0/887159602 learned_addr learned my addr 172.21.15.94:0/887159602 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bc98db640 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c9d730 0x563f15070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 -- 172.21.15.94:0/887159602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 msgr2=0x563f150594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 0x563f150594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 -- 172.21.15.94:0/887159602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c9d730 msgr2=0x563f15070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c9d730 0x563f15070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 -- 172.21.15.94:0/887159602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f150640f0 con 0x563f14c95550 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bc98db640 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 0x563f150594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0046870 tx=0x7f3bc0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d31ae067c6b59a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3baffff640 1 -- 172.21.15.94:0/887159602 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0053020 con 0x563f14c95550 2024-04-25T13:50:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3baffff640 1 -- 172.21.15.94:0/887159602 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc0046a60 con 0x563f14c95550 2024-04-25T13:50:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.031+0000 7f3baffff640 1 -- 172.21.15.94:0/887159602 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0035040 con 0x563f14c95550 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- 172.21.15.94:0/887159602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 msgr2=0x563f15070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0046870 tx=0x7f3bc0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- 172.21.15.94:0/887159602 shutdown_connections 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c9d730 0x563f15070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f15070660 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/887159602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c7d1f0 0x563f150594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- 172.21.15.94:0/887159602 >> 172.21.15.94:0/887159602 conn(0x563f14cb2560 msgr2=0x563f150678f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- 172.21.15.94:0/887159602 shutdown_connections 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- 172.21.15.94:0/887159602 wait complete. 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 -- start start 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.032+0000 7f3bd31f0740 1 Processor -- start 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 0x563f150da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c9d730 0x563f150e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41ac910 con 0x563f14c95550 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41ac240 con 0x563f14c7d1f0 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41a9480 con 0x563f14c9d730 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 10 client.0 _ll_register_callbacks cb 0x7f3bc0005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51608/0 (socket says 172.21.15.94:51608) 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bc98db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 0x563f150da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 -- 172.21.15.94:0/1327982821 learned_addr learned my addr 172.21.15.94:0/1327982821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c9d730 0x563f150e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 -- 172.21.15.94:0/1327982821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c9d730 msgr2=0x563f150e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c9d730 0x563f150e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 -- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 msgr2=0x563f150da540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 0x563f150da540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 -- 172.21.15.94:0/1327982821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f150e1fa0 con 0x563f14c95550 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 0x563f150da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bca0dc640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00469e0 tx=0x7f3bc0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8522580d0efe6edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3badffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0053030 con 0x563f14c95550 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f3bc0003990 con 0x563f14c95550 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc0046a60 con 0x563f14c95550 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3bc0067050 con 0x563f14c95550 2024-04-25T13:50:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.033+0000 7f3bd31f0740 10 client.18720 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f150e3260 con 0x563f14c95550 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc0046a60 con 0x563f14c95550 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc0044070 con 0x563f14c95550 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 10 client.18720 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 10 client.18720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 20 client.18720 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14cbe410 0x563f150e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 4 client.18720 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f14fcc580 con 0x563f14cbe410 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.034+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14cbe410 0x563f150e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.035+0000 7f3badffb640 10 client.18720 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.035+0000 7f3bc98db640 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14cbe410 0x563f150e5c70 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d9f1eb57cdd5d4 server_cookie=17dae164e69c3051 in_seq=0 out_seq=0 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3badffb640 1 -- 172.21.15.94:0/1327982821 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f14fcc580 con 0x563f14cbe410 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3badffb640 10 client.18720 handle_command_reply: tid=0 2024-04-25T13:50:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 client.18720 shutdown 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 msgr2=0x563f150daa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00469e0 tx=0x7f3bc0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 shutdown_connections 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563f14cbe410 0x563f150e5c70 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f14c9d730 0x563f150e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f14c95550 0x563f150daa80 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3badffb640 0 client.18720 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 --2- 172.21.15.94:0/1327982821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f14c7d1f0 0x563f150da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 >> 172.21.15.94:0/1327982821 conn(0x563f14cb2560 msgr2=0x563f1505dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 shutdown_connections 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 1 -- 172.21.15.94:0/1327982821 wait complete. 2024-04-25T13:50:35.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bd31f0740 20 client.18720 trim_cache size 0 max 16384 2024-04-25T13:50:35.042 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:50:35.042 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:50:35.042 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "a3158806-b3b2-4486-b99b-99bd149b3b40", 2024-04-25T13:50:35.042 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:50:35.042 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 msgr2=0x7f3bb00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 0x7f3bb00b2000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0a7600 tx=0x7f3bbc06db90 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 msgr2=0x7f3bc41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.036+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00eb10 tx=0x7f3bbc00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 shutdown_connections 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bb00afaf0 0x7f3bb00b2000 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc41050b0 0x7f3bc41a16f0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4102260 0x7f3bc419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 --2- 172.21.15.94:0/1231356182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4101760 0x7f3bc419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 >> 172.21.15.94:0/1231356182 conn(0x7f3bc40fd380 msgr2=0x7f3bc4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 shutdown_connections 2024-04-25T13:50:35.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.037+0000 7f3bcc340640 1 -- 172.21.15.94:0/1231356182 wait complete. 2024-04-25T13:50:35.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:50:35.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:50:35.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 -- 172.21.15.94:0/1345822201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c1080b0 msgr2=0x7f933c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 --2- 172.21.15.94:0/1345822201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c1080b0 0x7f933c10a4d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f933400adb0 tx=0x7f933402b7a0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 -- 172.21.15.94:0/1345822201 shutdown_connections 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 --2- 172.21.15.94:0/1345822201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c1080b0 0x7f933c10a4d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 --2- 172.21.15.94:0/1345822201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1033b0 0x7f933c1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 --2- 172.21.15.94:0/1345822201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c102170 0x7f933c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 -- 172.21.15.94:0/1345822201 >> 172.21.15.94:0/1345822201 conn(0x7f933c0fd340 msgr2=0x7f933c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 -- 172.21.15.94:0/1345822201 shutdown_connections 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.162+0000 7f93416a8640 1 -- 172.21.15.94:0/1345822201 wait complete. 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 Processor -- start 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 -- start start 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 0x7f933c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1080b0 0x7f933c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10d400 con 0x7f933c102170 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10d280 con 0x7f933c1033b0 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f93416a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f933c10d580 con 0x7f933c1080b0 2024-04-25T13:50:35.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f933affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f933affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51634/0 (socket says 172.21.15.94:51634) 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f933affd640 1 -- 172.21.15.94:0/1153236963 learned_addr learned my addr 172.21.15.94:0/1153236963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f933a7fc640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 0x7f933c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.163+0000 7f933b7fe640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1080b0 0x7f933c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 -- 172.21.15.94:0/1153236963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1080b0 msgr2=0x7f933c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1080b0 0x7f933c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 -- 172.21.15.94:0/1153236963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 msgr2=0x7f933c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 0x7f933c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 -- 172.21.15.94:0/1153236963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c19fb10 con 0x7f933c102170 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933a7fc640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 0x7f933c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f933affd640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e9f0 tx=0x7f932c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f9323fff640 1 -- 172.21.15.94:0/1153236963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c00ccc0 con 0x7f933c102170 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c19fe00 con 0x7f933c102170 2024-04-25T13:50:35.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f9323fff640 1 -- 172.21.15.94:0/1153236963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c0035f0 con 0x7f933c102170 2024-04-25T13:50:35.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f9323fff640 1 -- 172.21.15.94:0/1153236963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c00ce60 con 0x7f933c102170 2024-04-25T13:50:35.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.164+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1a0260 con 0x7f933c102170 2024-04-25T13:50:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 Processor -- start 2024-04-25T13:50:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 -- start start 2024-04-25T13:50:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9323fff640 1 -- 172.21.15.94:0/1153236963 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f932c003aa0 con 0x7f933c102170 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a58c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a59e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b769bbd80 con 0x562b7668c4d0 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b769758b0 con 0x562b766946b0 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9348636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b769b7d40 con 0x562b766b5390 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51646/0 (socket says 172.21.15.94:51646) 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933affd640 1 -- 172.21.15.94:0/2163956575 learned_addr learned my addr 172.21.15.94:0/2163956575 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a58c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933a7fc640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a59e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 -- 172.21.15.94:0/2163956575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 msgr2=0x562b76a59e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a59e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 -- 172.21.15.94:0/2163956575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 msgr2=0x562b76a591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933b7fe640 1 -- 172.21.15.94:0/2163956575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b76946a40 con 0x562b766b5390 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933affd640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f9323fff640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 0x7f93240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.167+0000 7f933affd640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 0x7f93240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.168+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a58c70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9334031040 tx=0x7f9334004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b2cd84cedb32da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.168+0000 7f9323fff640 1 -- 172.21.15.94:0/1153236963 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f932c0defc0 con 0x7f933c102170 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.168+0000 7f93217fa640 1 -- 172.21.15.94:0/2163956575 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334032400 con 0x562b766b5390 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.168+0000 7f93217fa640 1 -- 172.21.15.94:0/2163956575 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334042070 con 0x562b766b5390 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.168+0000 7f93217fa640 1 -- 172.21.15.94:0/2163956575 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340043e0 con 0x562b766b5390 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- 172.21.15.94:0/2163956575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 msgr2=0x562b76a58c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f933affd640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 0x7f93240b2000 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f932c0a3480 tx=0x7f932c0e2aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a58c70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9334031040 tx=0x7f9334004a40 comp rx=0 tx=0).stop 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- 172.21.15.94:0/2163956575 shutdown_connections 2024-04-25T13:50:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a59e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 --2- 172.21.15.94:0/2163956575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a58c70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- 172.21.15.94:0/2163956575 >> 172.21.15.94:0/2163956575 conn(0x562b766a94e0 msgr2=0x562b76a52270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- 172.21.15.94:0/2163956575 shutdown_connections 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- 172.21.15.94:0/2163956575 wait complete. 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 -- start start 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 Processor -- start 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.169+0000 7f9348636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a6a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b76a5a930 con 0x562b7668c4d0 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b76942d10 con 0x562b766946b0 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b76a5aab0 con 0x562b766b5390 2024-04-25T13:50:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f9348636740 10 client.0 _ll_register_callbacks cb 0x562b7696f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51650/0 (socket says 172.21.15.94:51650) 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933b7fe640 1 -- 172.21.15.94:0/3821873069 learned_addr learned my addr 172.21.15.94:0/3821873069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933b7fe640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a6a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 -- 172.21.15.94:0/3821873069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 msgr2=0x562b76a6a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a6a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 -- 172.21.15.94:0/3821873069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 msgr2=0x562b76a62b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 -- 172.21.15.94:0/3821873069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b767977d0 con 0x562b766946b0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933b7fe640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f933a7fc640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a630b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933c199070 tx=0x7f933000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24f5053810bace9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.170+0000 7f931eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:50:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330012110 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9330004050 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9334042070 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93300122b0 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 10 client.28196 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b766cd3f0 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334042070 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9330010430 con 0x562b766946b0 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 10 client.28196 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 10 client.28196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 20 client.28196 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b76673f60 0x562b76a70500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 4 client.28196 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:35.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.171+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b769c9990 con 0x562b76673f60 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.172+0000 7f933b7fe640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b76673f60 0x562b76a70500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.172+0000 7f933b7fe640 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b76673f60 0x562b76a70500 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5da950deb7b5d4 server_cookie=498e25380023bc3c in_seq=0 out_seq=0 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.172+0000 7f931eff5640 10 client.28196 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f931eff5640 1 -- 172.21.15.94:0/3821873069 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b769c9990 con 0x562b76673f60 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f931eff5640 10 client.28196 handle_command_reply: tid=0 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 client.28196 shutdown 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 msgr2=0x562b76a630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a630b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933c199070 tx=0x7f933000f790 comp rx=0 tx=0).stop 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 shutdown_connections 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b76673f60 0x562b76a70500 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766b5390 0x562b76a6a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b766946b0 0x562b76a630b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 --2- 172.21.15.94:0/3821873069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7668c4d0 0x562b76a62b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f931eff5640 0 client.28196 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 >> 172.21.15.94:0/3821873069 conn(0x562b766a94e0 msgr2=0x562b76a513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 shutdown_connections 2024-04-25T13:50:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 1 -- 172.21.15.94:0/3821873069 wait complete. 2024-04-25T13:50:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.173+0000 7f9348636740 20 client.28196 trim_cache size 0 max 16384 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.192+0000 7f9348636740 1 Processor -- start 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.192+0000 7f9348636740 1 -- start start 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.192+0000 7f9348636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a61820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.192+0000 7f9348636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.192+0000 7f9348636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9348636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9380 con 0x562b766b5390 2024-04-25T13:50:35.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9348636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aaec0 con 0x562b7668c4d0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9348636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a9a50 con 0x562b766946b0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51654/0 (socket says 172.21.15.94:51654) 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 -- 172.21.15.94:0/2960535804 learned_addr learned my addr 172.21.15.94:0/2960535804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a61820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933a7fc640 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 -- 172.21.15.94:0/2960535804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 msgr2=0x562b76a69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 -- 172.21.15.94:0/2960535804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 msgr2=0x562b76a61820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a61820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 -- 172.21.15.94:0/2960535804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b7668d0f0 con 0x562b766b5390 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f933b7fe640 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9334031040 tx=0x7f933402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db65e60c903056a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9320ff9640 1 -- 172.21.15.94:0/2960535804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334051040 con 0x562b766b5390 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9320ff9640 1 -- 172.21.15.94:0/2960535804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334042040 con 0x562b766b5390 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.193+0000 7f9320ff9640 1 -- 172.21.15.94:0/2960535804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334030510 con 0x562b766b5390 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- 172.21.15.94:0/2960535804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 msgr2=0x562b76a74cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9334031040 tx=0x7f933402b7a0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- 172.21.15.94:0/2960535804 shutdown_connections 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76a74cb0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 --2- 172.21.15.94:0/2960535804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a61820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- 172.21.15.94:0/2960535804 >> 172.21.15.94:0/2960535804 conn(0x562b766a94e0 msgr2=0x562b76a61110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- 172.21.15.94:0/2960535804 shutdown_connections 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- 172.21.15.94:0/2960535804 wait complete. 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 -- start start 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.194+0000 7f9348636740 1 Processor -- start 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aa7f0 con 0x562b766b5390 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1aa120 con 0x562b7668c4d0 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1a7360 con 0x562b766946b0 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 10 client.0 _ll_register_callbacks cb 0x7f9330009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51670/0 (socket says 172.21.15.94:51670) 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 -- 172.21.15.94:0/914455097 learned_addr learned my addr 172.21.15.94:0/914455097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 -- 172.21.15.94:0/914455097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 msgr2=0x562b76a672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 -- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 msgr2=0x562b76a66d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a66d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 -- 172.21.15.94:0/914455097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b7697c4b0 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f933b7fe640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f933402bcc0 tx=0x7f9334040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398e381ce9d3cdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f931eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933406b060 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9334031810 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.195+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9334042040 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f933405e040 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 10 client.18738 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b76ad7c40 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9334042040 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9334064040 con 0x562b766b5390 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 10 client.18738 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:35.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 20 client.18738 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b766493a0 0x562b76ada170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 4 client.18738 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b769c3800 con 0x562b766493a0 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f933a7fc640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b766493a0 0x562b76ada170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f931eff5640 10 client.18738 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.196+0000 7f933a7fc640 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b766493a0 0x562b76ada170 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a806079f27a74a80 server_cookie=66157315fc193751 in_seq=0 out_seq=0 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f931eff5640 1 -- 172.21.15.94:0/914455097 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b769c3800 con 0x562b766493a0 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f931eff5640 10 client.18738 handle_command_reply: tid=0 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 client.18738 shutdown 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 msgr2=0x562b76ad6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f933402bcc0 tx=0x7f9334040040 comp rx=0 tx=0).stop 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 shutdown_connections 2024-04-25T13:50:35.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b766493a0 0x562b76ada170 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b766b5390 0x562b76ad6860 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b766946b0 0x562b76a672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 --2- 172.21.15.94:0/914455097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b7668c4d0 0x562b76a66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f931eff5640 0 client.18738 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 >> 172.21.15.94:0/914455097 conn(0x562b766a94e0 msgr2=0x562b76a55120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 shutdown_connections 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 1 -- 172.21.15.94:0/914455097 wait complete. 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.197+0000 7f9348636740 20 client.18738 trim_cache size 0 max 16384 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:50:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "a3158806-b3b2-4486-b99b-99bd149b3b40": { 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "a3158806-b3b2-4486-b99b-99bd149b3b40", 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:50:35.205 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 msgr2=0x7f93240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 0x7f93240b2000 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f932c0a3480 tx=0x7f932c0e2aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 msgr2=0x7f933c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e9f0 tx=0x7f932c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 shutdown_connections 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f93240afaf0 0x7f93240b2000 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c1080b0 0x7f933c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c1033b0 0x7f933c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 --2- 172.21.15.94:0/1153236963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f933c102170 0x7f933c197fd0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 >> 172.21.15.94:0/1153236963 conn(0x7f933c0fd340 msgr2=0x7f933c104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 shutdown_connections 2024-04-25T13:50:35.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:50:35.198+0000 7f93416a8640 1 -- 172.21.15.94:0/1153236963 wait complete. 2024-04-25T13:50:35.209 INFO:tasks.cephfs.filesystem:scrub status for tag:a3158806-b3b2-4486-b99b-99bd149b3b40 - {'path': '/', 'tag': 'a3158806-b3b2-4486-b99b-99bd149b3b40', 'options': 'recursive,force'} 2024-04-25T13:50:35.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1062530 12.88 MB/sec execute 318 sec latency 13.725 ms 2024-04-25T13:50:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: pgmap v611: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2024-04-25T13:50:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:50:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:50:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: pgmap v611: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:50:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: pgmap v611: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2024-04-25T13:50:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:50:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:50:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:50:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:50:36.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1065067 12.88 MB/sec execute 319 sec latency 16.527 ms 2024-04-25T13:50:37.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1066872 12.87 MB/sec execute 320 sec latency 16.734 ms 2024-04-25T13:50:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:38 smithi137 ceph-mon[32568]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-04-25T13:50:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:38 smithi094 ceph-mon[29195]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-04-25T13:50:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:38 smithi138 ceph-mon[32977]: pgmap v612: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 356 op/s 2024-04-25T13:50:38.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1069351 12.88 MB/sec execute 321 sec latency 70.069 ms 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 -- 172.21.15.94:0/3774963977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 msgr2=0x7f8e2c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3774963977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c1066e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002a410 comp rx=0 tx=0).stop 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 -- 172.21.15.94:0/3774963977 shutdown_connections 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3774963977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 0x7f8e2c10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3774963977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 0x7f8e2c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3774963977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c1066e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 -- 172.21.15.94:0/3774963977 >> 172.21.15.94:0/3774963977 conn(0x7f8e2c0fdc20 msgr2=0x7f8e2c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.513+0000 7f8e32b08640 1 -- 172.21.15.94:0/3774963977 shutdown_connections 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 -- 172.21.15.94:0/3774963977 wait complete. 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 Processor -- start 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 -- start start 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 0x7f8e2c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 0x7f8e2c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c10f610 con 0x7f8e2c1042c0 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c10f490 con 0x7f8e2c106cb0 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e32b08640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c10f790 con 0x7f8e2c1097c0 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51700/0 (socket says 172.21.15.94:51700) 2024-04-25T13:50:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e308a4640 1 -- 172.21.15.94:0/3826062468 learned_addr learned my addr 172.21.15.94:0/3826062468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 0x7f8e2c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.514+0000 7f8e310a5640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 0x7f8e2c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 -- 172.21.15.94:0/3826062468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 msgr2=0x7f8e2c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 0x7f8e2c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 -- 172.21.15.94:0/3826062468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 msgr2=0x7f8e2c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 0x7f8e2c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 -- 172.21.15.94:0/3826062468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c1a1d80 con 0x7f8e2c1042c0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006010 tx=0x7f8e20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e29ffb640 1 -- 172.21.15.94:0/3826062468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2003f030 con 0x7f8e2c1042c0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e29ffb640 1 -- 172.21.15.94:0/3826062468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e20003aa0 con 0x7f8e2c1042c0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c1a2010 con 0x7f8e2c1042c0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e29ffb640 1 -- 172.21.15.94:0/3826062468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20035070 con 0x7f8e2c1042c0 2024-04-25T13:50:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.515+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c1a2500 con 0x7f8e2c1042c0 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 Processor -- start 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e29ffb640 1 -- 172.21.15.94:0/3826062468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8e20042030 con 0x7f8e2c1042c0 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 -- start start 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68504970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba6850bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ba6846a460 con 0x55ba6813b2f0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ba68424590 con 0x55ba681641b0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e399b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba68466420 con 0x55ba681434d0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51714/0 (socket says 172.21.15.94:51714) 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e308a4640 1 -- 172.21.15.94:0/3552903048 learned_addr learned my addr 172.21.15.94:0/3552903048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e310a5640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68504970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba6850bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.518+0000 7f8e29ffb640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 0x7f8e1c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 -- 172.21.15.94:0/3552903048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 msgr2=0x55ba6850bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba6850bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 -- 172.21.15.94:0/3552903048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 msgr2=0x55ba68504970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68504970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 -- 172.21.15.94:0/3552903048 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba684310d0 con 0x55ba6813b2f0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba6850bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e310a5640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 0x7f8e1c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e29ffb640 1 -- 172.21.15.94:0/3826062468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e200ff050 con 0x7f8e2c1042c0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e308a4640 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8e20089f00 tx=0x7f8e200f5c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395f57a9df6c134a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e16ffd640 1 -- 172.21.15.94:0/3552903048 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20049070 con 0x55ba6813b2f0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e310a5640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 0x7f8e1c0b1ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ba68505b40 tx=0x7f8e24004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e16ffd640 1 -- 172.21.15.94:0/3552903048 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200c70f0 con 0x55ba6813b2f0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.519+0000 7f8e16ffd640 1 -- 172.21.15.94:0/3552903048 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200c6d50 con 0x55ba6813b2f0 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- 172.21.15.94:0/3552903048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 msgr2=0x55ba68504ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8e20089f00 tx=0x7f8e200f5c80 comp rx=0 tx=0).stop 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- 172.21.15.94:0/3552903048 shutdown_connections 2024-04-25T13:50:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba6850bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68504ef0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 --2- 172.21.15.94:0/3552903048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68504970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- 172.21.15.94:0/3552903048 >> 172.21.15.94:0/3552903048 conn(0x55ba68158300 msgr2=0x55ba68500300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- 172.21.15.94:0/3552903048 shutdown_connections 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- 172.21.15.94:0/3552903048 wait complete. 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 -- start start 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.520+0000 7f8e399b6740 1 Processor -- start 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6813b2f0 0x55ba6850e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 0x55ba6850e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ba6850c600 con 0x55ba681641b0 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ba683f1b60 con 0x55ba681434d0 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba6850c780 con 0x55ba6813b2f0 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e399b6740 10 client.0 _ll_register_callbacks cb 0x55ba684162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e310a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 0x55ba6850e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e310a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51718/0 (socket says 172.21.15.94:51718) 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e310a5640 1 -- 172.21.15.94:0/1311470814 learned_addr learned my addr 172.21.15.94:0/1311470814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e308a4640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6813b2f0 0x55ba6850e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 -- 172.21.15.94:0/1311470814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6813b2f0 msgr2=0x55ba6850e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6813b2f0 0x55ba6850e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 -- 172.21.15.94:0/1311470814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 msgr2=0x55ba685157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 -- 172.21.15.94:0/1311470814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6817c210 con 0x55ba681434d0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e310a5640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 0x55ba6850e890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c19b2e0 tx=0x7f8e18012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288ef699618481c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.521+0000 7f8e0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18004030 con 0x55ba681434d0 2024-04-25T13:50:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e200c70f0 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e180041d0 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e180043d0 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 10 client.28214 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba68248620 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200c70f0 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e18003ac0 con 0x55ba681434d0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 10 client.28214 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 10 client.28214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 20 client.28214 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba68122d80 0x55ba6851c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 4 client.28214 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.522+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba68478070 con 0x55ba68122d80 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.523+0000 7f8e308a4640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba68122d80 0x55ba6851c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.523+0000 7f8e0bfff640 10 client.28214 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.523+0000 7f8e308a4640 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba68122d80 0x55ba6851c080 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74f09774eb35a34 server_cookie=3529189a6578e5fd in_seq=0 out_seq=0 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e0bfff640 1 -- 172.21.15.94:0/1311470814 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba68478070 con 0x55ba68122d80 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e0bfff640 10 client.28214 handle_command_reply: tid=0 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 client.28214 shutdown 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 msgr2=0x55ba6850e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 0x55ba6850e890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c19b2e0 tx=0x7f8e18012b60 comp rx=0 tx=0).stop 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 shutdown_connections 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba68122d80 0x55ba6851c080 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681434d0 0x55ba6850e890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e0bfff640 0 client.28214 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 --2- 172.21.15.94:0/1311470814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6813b2f0 0x55ba6850e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 >> 172.21.15.94:0/1311470814 conn(0x55ba68158300 msgr2=0x55ba684ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 shutdown_connections 2024-04-25T13:50:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 1 -- 172.21.15.94:0/1311470814 wait complete. 2024-04-25T13:50:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.524+0000 7f8e399b6740 20 client.28214 trim_cache size 0 max 16384 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 Processor -- start 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 -- start start 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba6813b2f0 0x55ba684ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba68515710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ab5a0 con 0x55ba681641b0 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ad0e0 con 0x55ba6813b2f0 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e399b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1abc70 con 0x55ba681434d0 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51728/0 (socket says 172.21.15.94:51728) 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 -- 172.21.15.94:0/823297957 learned_addr learned my addr 172.21.15.94:0/823297957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e308a4640 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba6813b2f0 0x55ba684ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba68515710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 -- 172.21.15.94:0/823297957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 msgr2=0x55ba68515710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba68515710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 -- 172.21.15.94:0/823297957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba6813b2f0 msgr2=0x55ba684ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba6813b2f0 0x55ba684ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.539+0000 7f8e310a5640 1 -- 172.21.15.94:0/823297957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba68515c50 con 0x55ba681641b0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.540+0000 7f8e310a5640 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005bf0 tx=0x7f8e24016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3fb1c722659a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.540+0000 7f8e167fc640 1 -- 172.21.15.94:0/823297957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24010520 con 0x55ba681641b0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.540+0000 7f8e167fc640 1 -- 172.21.15.94:0/823297957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24010b70 con 0x55ba681641b0 2024-04-25T13:50:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.540+0000 7f8e167fc640 1 -- 172.21.15.94:0/823297957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2401fda0 con 0x55ba681641b0 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- 172.21.15.94:0/823297957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 msgr2=0x55ba685000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005bf0 tx=0x7f8e24016000 comp rx=0 tx=0).stop 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- 172.21.15.94:0/823297957 shutdown_connections 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba681641b0 0x55ba685000e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba68515710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- 172.21.15.94:0/823297957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba6813b2f0 0x55ba684ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- 172.21.15.94:0/823297957 >> 172.21.15.94:0/823297957 conn(0x55ba68158300 msgr2=0x55ba68508b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- 172.21.15.94:0/823297957 shutdown_connections 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- 172.21.15.94:0/823297957 wait complete. 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- start start 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 Processor -- start 2024-04-25T13:50:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba685130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68513f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1aca10 con 0x55ba6813b2f0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ac340 con 0x55ba681641b0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1a9580 con 0x55ba681434d0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e399b6740 10 client.0 _ll_register_callbacks cb 0x7f8e18009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e308a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51736/0 (socket says 172.21.15.94:51736) 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.541+0000 7f8e308a4640 1 -- 172.21.15.94:0/560600001 learned_addr learned my addr 172.21.15.94:0/560600001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba685130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e310a5640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68513f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 -- 172.21.15.94:0/560600001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 msgr2=0x55ba685130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba685130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 -- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 msgr2=0x55ba68513f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68513f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 -- 172.21.15.94:0/560600001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba68514550 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e308a4640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8e200c7570 tx=0x7f8e20003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5086296df25dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20049040 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f8e200b00d0 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24010b70 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e200a7050 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e399b6740 10 client.18756 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba685157e0 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.542+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24010b70 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8e200fbc20 con 0x55ba6813b2f0 2024-04-25T13:50:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 10 client.18756 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 10 client.18756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 20 client.18756 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba680f81c0 0x55ba68586780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 4 client.18756 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba68471ee0 con 0x55ba680f81c0 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba680f81c0 0x55ba68586780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e0bfff640 10 client.18756 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.543+0000 7f8e2bfff640 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba680f81c0 0x55ba68586780 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e583fbf2e4e090 server_cookie=fbb1f0c74fa9847b in_seq=0 out_seq=0 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e0bfff640 1 -- 172.21.15.94:0/560600001 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba68471ee0 con 0x55ba680f81c0 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e0bfff640 10 client.18756 handle_command_reply: tid=0 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 client.18756 shutdown 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 msgr2=0x55ba68512b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8e200c7570 tx=0x7f8e20003f90 comp rx=0 tx=0).stop 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 shutdown_connections 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ba680f81c0 0x55ba68586780 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba681641b0 0x55ba68513f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba681434d0 0x55ba685130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 --2- 172.21.15.94:0/560600001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ba6813b2f0 0x55ba68512b90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 >> 172.21.15.94:0/560600001 conn(0x55ba68158300 msgr2=0x55ba684ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e0bfff640 0 client.18756 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.721+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 shutdown_connections 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e399b6740 1 -- 172.21.15.94:0/560600001 wait complete. 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e399b6740 20 client.18756 trim_cache size 0 max 16384 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:50:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:50:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 msgr2=0x7f8e1c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 0x7f8e1c0b1ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ba68505b40 tx=0x7f8e24004a40 comp rx=0 tx=0).stop 2024-04-25T13:50:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 msgr2=0x7f8e2c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006010 tx=0x7f8e20004270 comp rx=0 tx=0).stop 2024-04-25T13:50:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 shutdown_connections 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8e1c0afae0 0x7f8e1c0b1ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e2c1097c0 0x7f8e2c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e2c106cb0 0x7f8e2c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 --2- 172.21.15.94:0/3826062468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e2c1042c0 0x7f8e2c19a240 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.722+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 >> 172.21.15.94:0/3826062468 conn(0x7f8e2c0fdc20 msgr2=0x7f8e2c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.723+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 shutdown_connections 2024-04-25T13:50:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:39.723+0000 7f8e32b08640 1 -- 172.21.15.94:0/3826062468 wait complete. 2024-04-25T13:50:39.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1070757 12.86 MB/sec execute 322 sec latency 74.991 ms 2024-04-25T13:50:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:40 smithi137 ceph-mon[32568]: pgmap v613: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:50:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:40 smithi094 ceph-mon[29195]: pgmap v613: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:50:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:40 smithi138 ceph-mon[32977]: pgmap v613: 97 pgs: 97 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 352 op/s 2024-04-25T13:50:40.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1073236 12.86 MB/sec execute 323 sec latency 11.382 ms 2024-04-25T13:50:41.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1075797 12.86 MB/sec execute 324 sec latency 18.131 ms 2024-04-25T13:50:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:42 smithi137 ceph-mon[32568]: pgmap v614: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-04-25T13:50:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:42 smithi094 ceph-mon[29195]: pgmap v614: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-04-25T13:50:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:42 smithi138 ceph-mon[32977]: pgmap v614: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.7 MiB/s wr, 392 op/s 2024-04-25T13:50:42.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1077585 12.85 MB/sec execute 325 sec latency 17.901 ms 2024-04-25T13:50:43.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1080144 12.86 MB/sec execute 326 sec latency 70.589 ms 2024-04-25T13:50:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:44 smithi137 ceph-mon[32568]: pgmap v615: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 278 op/s 2024-04-25T13:50:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:44 smithi094 ceph-mon[29195]: pgmap v615: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 278 op/s 2024-04-25T13:50:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:44 smithi138 ceph-mon[32977]: pgmap v615: 97 pgs: 97 active+clean; 52 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 278 op/s 2024-04-25T13:50:44.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1082453 12.84 MB/sec execute 327 sec latency 15.912 ms 2024-04-25T13:50:45.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1084489 12.85 MB/sec execute 328 sec latency 12.558 ms 2024-04-25T13:50:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:46 smithi094 ceph-mon[29195]: pgmap v616: 97 pgs: 97 active+clean; 48 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-04-25T13:50:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:46 smithi138 ceph-mon[32977]: pgmap v616: 97 pgs: 97 active+clean; 48 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-04-25T13:50:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:46 smithi137 ceph-mon[32568]: pgmap v616: 97 pgs: 97 active+clean; 48 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2024-04-25T13:50:46.806 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1087225 12.86 MB/sec execute 329 sec latency 87.656 ms 2024-04-25T13:50:47.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1089410 12.84 MB/sec execute 330 sec latency 55.577 ms 2024-04-25T13:50:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:48 smithi094 ceph-mon[29195]: pgmap v617: 97 pgs: 97 active+clean; 50 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-04-25T13:50:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:48 smithi138 ceph-mon[32977]: pgmap v617: 97 pgs: 97 active+clean; 50 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-04-25T13:50:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:48 smithi137 ceph-mon[32568]: pgmap v617: 97 pgs: 97 active+clean; 50 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-04-25T13:50:48.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1091172 12.84 MB/sec execute 331 sec latency 86.977 ms 2024-04-25T13:50:49.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1094051 12.85 MB/sec execute 332 sec latency 16.660 ms 2024-04-25T13:50:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:50 smithi094 ceph-mon[29195]: pgmap v618: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-04-25T13:50:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:50 smithi138 ceph-mon[32977]: pgmap v618: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-04-25T13:50:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:50 smithi137 ceph-mon[32568]: pgmap v618: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-04-25T13:50:50.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1095944 12.84 MB/sec execute 333 sec latency 13.638 ms 2024-04-25T13:50:51.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1098169 12.84 MB/sec execute 334 sec latency 9.939 ms 2024-04-25T13:50:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:52 smithi094 ceph-mon[29195]: pgmap v619: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 379 op/s 2024-04-25T13:50:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:52 smithi138 ceph-mon[32977]: pgmap v619: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 379 op/s 2024-04-25T13:50:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:52 smithi137 ceph-mon[32568]: pgmap v619: 97 pgs: 97 active+clean; 51 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 379 op/s 2024-04-25T13:50:52.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1101142 12.85 MB/sec execute 335 sec latency 15.313 ms 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 -- 172.21.15.94:0/2459284487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 msgr2=0x7f0468104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 --2- 172.21.15.94:0/2459284487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468104070 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f0458009900 tx=0x7f045802a410 comp rx=0 tx=0).stop 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 -- 172.21.15.94:0/2459284487 shutdown_connections 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 --2- 172.21.15.94:0/2459284487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468108960 0x7f046810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 --2- 172.21.15.94:0/2459284487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468104070 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 --2- 172.21.15.94:0/2459284487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468102a20 0x7f0468103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 -- 172.21.15.94:0/2459284487 >> 172.21.15.94:0/2459284487 conn(0x7f04680fdc10 msgr2=0x7f04681000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 -- 172.21.15.94:0/2459284487 shutdown_connections 2024-04-25T13:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.823+0000 7f046ff6c640 1 -- 172.21.15.94:0/2459284487 wait complete. 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 Processor -- start 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 -- start start 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 0x7f0468199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468108960 0x7f04681a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e540 con 0x7f0468103c60 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e3c0 con 0x7f0468102a20 2024-04-25T13:50:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046ff6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f046810e6c0 con 0x7f0468108960 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046d507640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046d507640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43968/0 (socket says 172.21.15.94:43968) 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046d507640 1 -- 172.21.15.94:0/1840873364 learned_addr learned my addr 172.21.15.94:0/1840873364 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 0x7f0468199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046e509640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468108960 0x7f04681a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 -- 172.21.15.94:0/1840873364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468108960 msgr2=0x7f04681a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468108960 0x7f04681a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 -- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 msgr2=0x7f0468199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046dd08640 1 -- 172.21.15.94:0/1840873364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04681a0c90 con 0x7f0468102a20 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.824+0000 7f046d507640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f046dd08640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 0x7f0468199150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04681022f0 tx=0x7f046400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f045effd640 1 -- 172.21.15.94:0/1840873364 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464003970 con 0x7f0468102a20 2024-04-25T13:50:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04681a0f80 con 0x7f0468102a20 2024-04-25T13:50:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f045effd640 1 -- 172.21.15.94:0/1840873364 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0464004020 con 0x7f0468102a20 2024-04-25T13:50:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f045effd640 1 -- 172.21.15.94:0/1840873364 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464010690 con 0x7f0468102a20 2024-04-25T13:50:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.825+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a13e0 con 0x7f0468102a20 2024-04-25T13:50:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.827+0000 7f0476e1a740 1 Processor -- start 2024-04-25T13:50:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.827+0000 7f0476e1a740 1 -- start start 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 0x55a700ce9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a700c4d460 con 0x55a70091e2f0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a700c07590 con 0x55a7009264d0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f0476e1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a700c49420 con 0x55a7009471b0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f045effd640 1 -- 172.21.15.94:0/1840873364 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f04640041d0 con 0x7f0468102a20 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046d507640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046d507640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:44096/0 (socket says 172.21.15.94:44096) 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046d507640 1 -- 172.21.15.94:0/3583158056 learned_addr learned my addr 172.21.15.94:0/3583158056 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 0x55a700ce9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046dd08640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 -- 172.21.15.94:0/3583158056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 msgr2=0x55a700cf1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 -- 172.21.15.94:0/3583158056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 msgr2=0x55a700cea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046e509640 1 -- 172.21.15.94:0/3583158056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a700c140d0 con 0x55a7009471b0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.828+0000 7f046d507640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f046e509640 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 0x55a700ce9b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0460009b80 tx=0x7f0460011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5c2960d1d5320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f045effd640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 0x7f04540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f0453fff640 1 -- 172.21.15.94:0/3583158056 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460004030 con 0x55a7009471b0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f0453fff640 1 -- 172.21.15.94:0/3583158056 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0460005020 con 0x55a7009471b0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f0453fff640 1 -- 172.21.15.94:0/3583158056 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460004c60 con 0x55a7009471b0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f046dd08640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 0x7f04540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.829+0000 7f045effd640 1 -- 172.21.15.94:0/1840873364 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f04640df4f0 con 0x7f0468102a20 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f046dd08640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 0x7f04540b1ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f04640a4b40 tx=0x7f04640e2f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- 172.21.15.94:0/3583158056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 msgr2=0x55a700ce9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 0x55a700ce9b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0460009b80 tx=0x7f0460011810 comp rx=0 tx=0).stop 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- 172.21.15.94:0/3583158056 shutdown_connections 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 --2- 172.21.15.94:0/3583158056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7009471b0 0x55a700ce9b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- 172.21.15.94:0/3583158056 >> 172.21.15.94:0/3583158056 conn(0x55a70093b300 msgr2=0x55a700ce3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- 172.21.15.94:0/3583158056 shutdown_connections 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- 172.21.15.94:0/3583158056 wait complete. 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 -- start start 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.830+0000 7f0476e1a740 1 Processor -- start 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cf3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 0x55a700cf3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009471b0 0x55a700cfad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a700cf1a50 con 0x55a7009264d0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a700bd4b60 con 0x55a7009471b0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a700cf1bd0 con 0x55a70091e2f0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f0476e1a740 10 client.0 _ll_register_callbacks cb 0x55a700bf92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046e509640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cf3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 0x55a700cf3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046e509640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cf3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38728/0 (socket says 172.21.15.94:38728) 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046e509640 1 -- 172.21.15.94:0/2293405963 learned_addr learned my addr 172.21.15.94:0/2293405963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046e509640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009471b0 0x55a700cfad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 -- 172.21.15.94:0/2293405963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 msgr2=0x55a700cf3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cf3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 -- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009471b0 msgr2=0x55a700cfad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009471b0 0x55a700cfad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 -- 172.21.15.94:0/2293405963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70095f210 con 0x55a7009264d0 2024-04-25T13:50:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f046d507640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 0x55a700cf3dd0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f04580317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b87a2719d59a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f04517fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.831+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458035040 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0460005020 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0458004450 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04580035f0 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 10 client.18768 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a700a2b620 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0460005020 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0458044070 con 0x55a7009264d0 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 10 client.18768 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 10 client.18768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 20 client.18768 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a700905d80 0x55a700d01520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 4 client.18768 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a700c5b070 con 0x55a700905d80 2024-04-25T13:50:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.832+0000 7f046e509640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a700905d80 0x55a700d01520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.833+0000 7f04517fa640 10 client.18768 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.833+0000 7f046e509640 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a700905d80 0x55a700d01520 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d30678d351e13f server_cookie=14d41dfd90a9bacb in_seq=0 out_seq=0 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.834+0000 7f04517fa640 1 -- 172.21.15.94:0/2293405963 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a700c5b070 con 0x55a700905d80 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.834+0000 7f04517fa640 10 client.18768 handle_command_reply: tid=0 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.834+0000 7f0476e1a740 1 client.18768 shutdown 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 msgr2=0x55a700cf3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 0x55a700cf3dd0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f04580317d0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 shutdown_connections 2024-04-25T13:50:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a700905d80 0x55a700d01520 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009471b0 0x55a700cfad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7009264d0 0x55a700cf3dd0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f04517fa640 0 client.18768 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2293405963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cf3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 >> 172.21.15.94:0/2293405963 conn(0x55a70093b300 msgr2=0x55a700ce6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 shutdown_connections 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 1 -- 172.21.15.94:0/2293405963 wait complete. 2024-04-25T13:50:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.835+0000 7f0476e1a740 20 client.18768 trim_cache size 0 max 16384 2024-04-25T13:50:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 Processor -- start 2024-04-25T13:50:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 -- start start 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cfac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700ce3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04681a92d0 con 0x55a700905d80 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04681aae10 con 0x55a7009264d0 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f0476e1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04681a99a0 con 0x55a70091e2f0 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44006/0 (socket says 172.21.15.94:44006) 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 -- 172.21.15.94:0/2186462504 learned_addr learned my addr 172.21.15.94:0/2186462504 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046d507640 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cfac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700ce3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 -- 172.21.15.94:0/2186462504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 msgr2=0x55a700cfac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cfac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 -- 172.21.15.94:0/2186462504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 msgr2=0x55a700ce3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700ce3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 -- 172.21.15.94:0/2186462504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a700ce3e90 con 0x55a700905d80 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f046e509640 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f0460009990 tx=0x7f0460005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757833bef73b72f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.851+0000 7f04537fe640 1 -- 172.21.15.94:0/2186462504 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046002b030 con 0x55a700905d80 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f04537fe640 1 -- 172.21.15.94:0/2186462504 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04600046c0 con 0x55a700905d80 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f04537fe640 1 -- 172.21.15.94:0/2186462504 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600054e0 con 0x55a700905d80 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 -- 172.21.15.94:0/2186462504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 msgr2=0x55a700ce2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f0460009990 tx=0x7f0460005850 comp rx=0 tx=0).stop 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 -- 172.21.15.94:0/2186462504 shutdown_connections 2024-04-25T13:50:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700ce3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70091e2f0 0x55a700cfac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 --2- 172.21.15.94:0/2186462504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a700905d80 0x55a700ce2c50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 -- 172.21.15.94:0/2186462504 >> 172.21.15.94:0/2186462504 conn(0x55a70093b300 msgr2=0x55a700cedd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.852+0000 7f0476e1a740 1 -- 172.21.15.94:0/2186462504 shutdown_connections 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 -- 172.21.15.94:0/2186462504 wait complete. 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 -- start start 2024-04-25T13:50:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 Processor -- start 2024-04-25T13:50:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a700905d80 0x55a700cf88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04681aa740 con 0x55a70091e2f0 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04681aa070 con 0x55a7009264d0 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04681a72b0 con 0x55a700905d80 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f0476e1a740 10 client.0 _ll_register_callbacks cb 0x7f0458009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f046d507640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f046d507640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44018/0 (socket says 172.21.15.94:44018) 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f046d507640 1 -- 172.21.15.94:0/1101749034 learned_addr learned my addr 172.21.15.94:0/1101749034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.853+0000 7f046e509640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a700905d80 0x55a700cf88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046e509640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 -- 172.21.15.94:0/1101749034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a700905d80 msgr2=0x55a700cf88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a700905d80 0x55a700cf88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 -- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 msgr2=0x55a700cf9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 -- 172.21.15.94:0/1101749034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a700cfb190 con 0x55a70091e2f0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f046d507640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f0458031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946085f5c875feda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f04517fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458053070 con 0x55a70091e2f0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f0458005040 con 0x55a70091e2f0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04600046c0 con 0x55a70091e2f0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0458035070 con 0x55a70091e2f0 2024-04-25T13:50:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f0476e1a740 10 client.18774 fetch_fsmap learned FSMap version 27 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a700d69bd0 con 0x55a70091e2f0 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.854+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04600046c0 con 0x55a70091e2f0 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f045805f070 con 0x55a70091e2f0 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 10 client.18774 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 10 client.18774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 20 client.18774 populate_metadata read hostname 'smithi094' 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7009471b0 0x55a700d6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 4 client.18774 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a700c54ee0 con 0x55a7009471b0 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f046e509640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7009471b0 0x55a700d6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f04517fa640 10 client.18774 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:52.855+0000 7f046e509640 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7009471b0 0x55a700d6c5e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf485e04530a4c33 server_cookie=ac430e2a9ba495ac in_seq=0 out_seq=0 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f04517fa640 1 -- 172.21.15.94:0/1101749034 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a700c54ee0 con 0x55a7009471b0 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f04517fa640 10 client.18774 handle_command_reply: tid=0 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f0476e1a740 1 client.18774 shutdown 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 msgr2=0x55a700cf8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f0458031400 comp rx=0 tx=0).stop 2024-04-25T13:50:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.045+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 shutdown_connections 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7009471b0 0x55a700d6c5e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7009264d0 0x55a700cf9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a70091e2f0 0x55a700cf8df0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 --2- 172.21.15.94:0/1101749034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a700905d80 0x55a700cf88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 >> 172.21.15.94:0/1101749034 conn(0x55a70093b300 msgr2=0x55a700ce6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f04517fa640 0 client.18774 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 shutdown_connections 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 1 -- 172.21.15.94:0/1101749034 wait complete. 2024-04-25T13:50:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f0476e1a740 20 client.18774 trim_cache size 0 max 16384 2024-04-25T13:50:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:50:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:50:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:50:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 msgr2=0x7f04540b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 0x7f04540b1ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f04640a4b40 tx=0x7f04640e2f30 comp rx=0 tx=0).stop 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 msgr2=0x7f0468199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 0x7f0468199150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04681022f0 tx=0x7f046400ef90 comp rx=0 tx=0).stop 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.046+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 shutdown_connections 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f04540afae0 0x7f04540b1ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0468108960 0x7f04681a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0468103c60 0x7f0468199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 --2- 172.21.15.94:0/1840873364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0468102a20 0x7f0468199150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 >> 172.21.15.94:0/1840873364 conn(0x7f04680fdc10 msgr2=0x7f0468104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 shutdown_connections 2024-04-25T13:50:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:50:53.047+0000 7f046ff6c640 1 -- 172.21.15.94:0/1840873364 wait complete. 2024-04-25T13:50:53.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1102056 12.82 MB/sec execute 336 sec latency 99.950 ms 2024-04-25T13:50:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:54 smithi094 ceph-mon[29195]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:50:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:54 smithi138 ceph-mon[32977]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:50:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:54 smithi137 ceph-mon[32568]: pgmap v620: 97 pgs: 97 active+clean; 53 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:50:54.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1104893 12.84 MB/sec execute 337 sec latency 21.736 ms 2024-04-25T13:50:55.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1107112 12.82 MB/sec execute 338 sec latency 20.898 ms 2024-04-25T13:50:56.444 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:56 smithi137 ceph-mon[32568]: pgmap v621: 97 pgs: 97 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-04-25T13:50:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:56 smithi094 ceph-mon[29195]: pgmap v621: 97 pgs: 97 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-04-25T13:50:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:56 smithi138 ceph-mon[32977]: pgmap v621: 97 pgs: 97 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 461 op/s 2024-04-25T13:50:56.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1109025 12.82 MB/sec execute 339 sec latency 15.133 ms 2024-04-25T13:50:57.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1111889 12.83 MB/sec execute 340 sec latency 21.024 ms 2024-04-25T13:50:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:50:58 smithi094 ceph-mon[29195]: pgmap v622: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-04-25T13:50:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:50:58 smithi138 ceph-mon[32977]: pgmap v622: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-04-25T13:50:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:50:58 smithi137 ceph-mon[32568]: pgmap v622: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 364 op/s 2024-04-25T13:50:58.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1113589 12.81 MB/sec execute 341 sec latency 48.949 ms 2024-04-25T13:50:59.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1115993 12.82 MB/sec execute 342 sec latency 17.730 ms 2024-04-25T13:51:00.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:00 smithi094 ceph-mon[29195]: pgmap v623: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-04-25T13:51:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:00 smithi138 ceph-mon[32977]: pgmap v623: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-04-25T13:51:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:00 smithi137 ceph-mon[32568]: pgmap v623: 97 pgs: 97 active+clean; 42 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 315 op/s 2024-04-25T13:51:00.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1119011 12.83 MB/sec execute 343 sec latency 13.629 ms 2024-04-25T13:51:01.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1121343 12.82 MB/sec execute 344 sec latency 12.652 ms 2024-04-25T13:51:02.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:02 smithi094 ceph-mon[29195]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-04-25T13:51:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:02 smithi138 ceph-mon[32977]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-04-25T13:51:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:02 smithi137 ceph-mon[32568]: pgmap v624: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-04-25T13:51:02.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1123332 12.82 MB/sec execute 345 sec latency 12.591 ms 2024-04-25T13:51:03.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1126198 12.83 MB/sec execute 346 sec latency 87.892 ms 2024-04-25T13:51:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:04 smithi094 ceph-mon[29195]: pgmap v625: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:51:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:04 smithi138 ceph-mon[32977]: pgmap v625: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:51:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:04 smithi137 ceph-mon[32568]: pgmap v625: 97 pgs: 97 active+clean; 47 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 366 op/s 2024-04-25T13:51:04.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1128421 12.82 MB/sec execute 347 sec latency 111.874 ms 2024-04-25T13:51:05.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:51:05.211 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- 172.21.15.94:0/2832514053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3341029e0 msgr2=0x7fe3341036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 --2- 172.21.15.94:0/2832514053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3341029e0 0x7fe3341036e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe324009900 tx=0x7fe32402a410 comp rx=0 tx=0).stop 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- 172.21.15.94:0/2832514053 shutdown_connections 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 --2- 172.21.15.94:0/2832514053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe33410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 --2- 172.21.15.94:0/2832514053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe334103c20 0x7fe334104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 --2- 172.21.15.94:0/2832514053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3341029e0 0x7fe3341036e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- 172.21.15.94:0/2832514053 >> 172.21.15.94:0/2832514053 conn(0x7fe3340fdbd0 msgr2=0x7fe334100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- 172.21.15.94:0/2832514053 shutdown_connections 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- 172.21.15.94:0/2832514053 wait complete. 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 Processor -- start 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 -- start start 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.330+0000 7fe339851640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3341029e0 0x7fe334199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe3341a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe33410e4e0 con 0x7fe334103c20 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe33410e360 con 0x7fe334108920 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe33410e660 con 0x7fe3341029e0 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45612/0 (socket says 172.21.15.94:45612) 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 -- 172.21.15.94:0/703012884 learned_addr learned my addr 172.21.15.94:0/703012884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe3337fe640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe3341a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe332ffd640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3341029e0 0x7fe334199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 -- 172.21.15.94:0/703012884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3341029e0 msgr2=0x7fe334199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3341029e0 0x7fe334199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 -- 172.21.15.94:0/703012884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 msgr2=0x7fe3341a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe3341a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 -- 172.21.15.94:0/703012884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3341a0fb0 con 0x7fe334103c20 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe3337fe640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe3341a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe32bfff640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d770 tx=0x7fe31c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe33884f640 1 -- 172.21.15.94:0/703012884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c017070 con 0x7fe334103c20 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe33884f640 1 -- 172.21.15.94:0/703012884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c0054d0 con 0x7fe334103c20 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe334069ef0 con 0x7fe334103c20 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe33884f640 1 -- 172.21.15.94:0/703012884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004c30 con 0x7fe334103c20 2024-04-25T13:51:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.331+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33406a3f0 con 0x7fe334103c20 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe33884f640 1 -- 172.21.15.94:0/703012884 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe31c004e10 con 0x7fe334103c20 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 Processor -- start 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 -- start start 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 0x56514f513a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f514c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56514f478d80 con 0x56514f1494d0 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56514f4328b0 con 0x56514f1516b0 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3407e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56514f474d40 con 0x56514f172390 2024-04-25T13:51:05.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 0x56514f513a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45614/0 (socket says 172.21.15.94:45614) 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe332ffd640 1 -- 172.21.15.94:0/1644209029 learned_addr learned my addr 172.21.15.94:0/1644209029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.334+0000 7fe32bfff640 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f514c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe33884f640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 0x7fe3200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 -- 172.21.15.94:0/1644209029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 msgr2=0x56514f513a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 0x56514f513a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 -- 172.21.15.94:0/1644209029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 msgr2=0x56514f514c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f514c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 -- 172.21.15.94:0/1644209029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514f403a40 con 0x56514f1494d0 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe3337fe640 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 0x56514f513a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe3337fe640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 0x7fe3200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe332ffd640 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe324009230 tx=0x7fe324031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e826427c2f186c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe33884f640 1 -- 172.21.15.94:0/703012884 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe31c0de6d0 con 0x7fe334103c20 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe329ffb640 1 -- 172.21.15.94:0/1644209029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324035030 con 0x56514f1494d0 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe329ffb640 1 -- 172.21.15.94:0/1644209029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324040070 con 0x56514f1494d0 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.335+0000 7fe329ffb640 1 -- 172.21.15.94:0/1644209029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324003910 con 0x56514f1494d0 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- 172.21.15.94:0/1644209029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 msgr2=0x56514f513fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3337fe640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 0x7fe3200b2000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56514f514ab0 tx=0x7fe32c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe324009230 tx=0x7fe324031b10 comp rx=0 tx=0).stop 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- 172.21.15.94:0/1644209029 shutdown_connections 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f514c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f1494d0 0x56514f513fe0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 --2- 172.21.15.94:0/1644209029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f172390 0x56514f513a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- 172.21.15.94:0/1644209029 >> 172.21.15.94:0/1644209029 conn(0x56514f1664e0 msgr2=0x56514f50f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- 172.21.15.94:0/1644209029 shutdown_connections 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- 172.21.15.94:0/1644209029 wait complete. 2024-04-25T13:51:05.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.336+0000 7fe3407e7740 1 -- start start 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 Processor -- start 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f51de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f524dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56514f515720 con 0x56514f172390 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56514f3ffd10 con 0x56514f1494d0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56514f5158a0 con 0x56514f1516b0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3407e7740 10 client.0 _ll_register_callbacks cb 0x56514f42c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43482/0 (socket says 172.21.15.94:43482) 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f524dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 -- 172.21.15.94:0/3344651050 learned_addr learned my addr 172.21.15.94:0/3344651050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe32bfff640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f51de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 -- 172.21.15.94:0/3344651050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 msgr2=0x56514f51de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f51de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 -- 172.21.15.94:0/3344651050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 msgr2=0x56514f524dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f524dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe332ffd640 1 -- 172.21.15.94:0/3344651050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514f2547d0 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.337+0000 7fe3337fe640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f524dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe332ffd640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe324009a20 tx=0x7fe324031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e64094cf9ae7ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324035070 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe324040070 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe3240036d0 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3240038d0 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 10 client.28250 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514f18a3f0 con 0x56514f1494d0 2024-04-25T13:51:05.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324040070 con 0x56514f1494d0 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe324045a40 con 0x56514f1494d0 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 10 client.28250 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 10 client.28250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 20 client.28250 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f130f60 0x56514f52ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 4 client.28250 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.338+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514f486990 con 0x56514f130f60 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.339+0000 7fe32bfff640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f130f60 0x56514f52ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.339+0000 7fe312ffd640 10 client.28250 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.339+0000 7fe32bfff640 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f130f60 0x56514f52ae80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d0169c67b95c50 server_cookie=15dd794827153ee4 in_seq=0 out_seq=0 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe312ffd640 1 -- 172.21.15.94:0/3344651050 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56514f486990 con 0x56514f130f60 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe312ffd640 10 client.28250 handle_command_reply: tid=0 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 client.28250 shutdown 2024-04-25T13:51:05.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 msgr2=0x56514f51d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe324009a20 tx=0x7fe324031850 comp rx=0 tx=0).stop 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 shutdown_connections 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f130f60 0x56514f52ae80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f524dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f51de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe312ffd640 0 client.28250 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 --2- 172.21.15.94:0/3344651050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f51d920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 >> 172.21.15.94:0/3344651050 conn(0x56514f1664e0 msgr2=0x56514f512180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 shutdown_connections 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 1 -- 172.21.15.94:0/3344651050 wait complete. 2024-04-25T13:51:05.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.341+0000 7fe3407e7740 20 client.28250 trim_cache size 0 max 16384 2024-04-25T13:51:05.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.360+0000 7fe3407e7740 1 Processor -- start 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.360+0000 7fe3407e7740 1 -- start start 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1494d0 0x56514f51c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f524ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341a9270 con 0x56514f172390 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341aadb0 con 0x56514f1516b0 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3407e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341a9940 con 0x56514f1494d0 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45640/0 (socket says 172.21.15.94:45640) 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 -- 172.21.15.94:0/2047727983 learned_addr learned my addr 172.21.15.94:0/2047727983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe32bfff640 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f524ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe332ffd640 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1494d0 0x56514f51c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 -- 172.21.15.94:0/2047727983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1494d0 msgr2=0x56514f51c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1494d0 0x56514f51c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 -- 172.21.15.94:0/2047727983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 msgr2=0x56514f524ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f524ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 -- 172.21.15.94:0/2047727983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514f14a0f0 con 0x56514f172390 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004690 tx=0x7fe32c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb9a076c8c7023f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3297fa640 1 -- 172.21.15.94:0/2047727983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c02b070 con 0x56514f172390 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3297fa640 1 -- 172.21.15.94:0/2047727983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32c022040 con 0x56514f172390 2024-04-25T13:51:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.361+0000 7fe3297fa640 1 -- 172.21.15.94:0/2047727983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c021040 con 0x56514f172390 2024-04-25T13:51:05.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 -- 172.21.15.94:0/2047727983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 msgr2=0x56514f525220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004690 tx=0x7fe32c019000 comp rx=0 tx=0).stop 2024-04-25T13:51:05.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 -- 172.21.15.94:0/2047727983 shutdown_connections 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f525220 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1516b0 0x56514f524ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2047727983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1494d0 0x56514f51c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 -- 172.21.15.94:0/2047727983 >> 172.21.15.94:0/2047727983 conn(0x56514f1664e0 msgr2=0x56514f51bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 -- 172.21.15.94:0/2047727983 shutdown_connections 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.362+0000 7fe3407e7740 1 -- 172.21.15.94:0/2047727983 wait complete. 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 -- start start 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 Processor -- start 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f522860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341aa6e0 con 0x56514f172390 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341aa010 con 0x56514f1494d0 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341a7250 con 0x56514f1516b0 2024-04-25T13:51:05.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3407e7740 10 client.0 _ll_register_callbacks cb 0x7fe31c0e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43502/0 (socket says 172.21.15.94:43502) 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45656/0 (socket says 172.21.15.94:45656) 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe332ffd640 1 -- 172.21.15.94:0/2256665980 learned_addr learned my addr 172.21.15.94:0/2256665980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.363+0000 7fe32bfff640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f522860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 -- 172.21.15.94:0/2256665980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 msgr2=0x56514f522860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f522860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 -- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 msgr2=0x56514f522320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 -- 172.21.15.94:0/2256665980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514f4394b0 con 0x56514f172390 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe332ffd640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3337fe640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004010 tx=0x7fe32c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fc6caaf495ff3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:05.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c02b050 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fe32c027680 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32c022040 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe32c021040 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 10 client.18792 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56514f592360 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32c022040 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe32c0454e0 con 0x56514f172390 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 10 client.18792 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 10 client.18792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.364+0000 7fe3407e7740 20 client.18792 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f1063a0 0x56514f594c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe3407e7740 4 client.18792 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56514f480800 con 0x56514f1063a0 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe332ffd640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f1063a0 0x56514f594c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe312ffd640 10 client.18792 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe332ffd640 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f1063a0 0x56514f594c70 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ff7bb130260abf server_cookie=c9917f8eb85829f0 in_seq=0 out_seq=0 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe312ffd640 1 -- 172.21.15.94:0/2256665980 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56514f480800 con 0x56514f1063a0 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe312ffd640 10 client.18792 handle_command_reply: tid=0 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.365+0000 7fe3407e7740 1 client.18792 shutdown 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 msgr2=0x56514f5910f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe32c004010 tx=0x7fe32c0048f0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 shutdown_connections 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56514f1063a0 0x56514f594c70 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514f172390 0x56514f5910f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514f1516b0 0x56514f522860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe312ffd640 0 client.18792 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 --2- 172.21.15.94:0/2256665980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514f1494d0 0x56514f522320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 >> 172.21.15.94:0/2256665980 conn(0x56514f1664e0 msgr2=0x56514f51dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 shutdown_connections 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 1 -- 172.21.15.94:0/2256665980 wait complete. 2024-04-25T13:51:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe3407e7740 20 client.18792 trim_cache size 0 max 16384 2024-04-25T13:51:05.372 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:05.372 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:51:05.372 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:51:05.372 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 msgr2=0x7fe3200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 0x7fe3200b2000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56514f514ab0 tx=0x7fe32c004a40 comp rx=0 tx=0).stop 2024-04-25T13:51:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 msgr2=0x7fe3341999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d770 tx=0x7fe31c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:51:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 shutdown_connections 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe3200afaf0 0x7fe3200b2000 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe334108920 0x7fe3341a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe334103c20 0x7fe3341999b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 --2- 172.21.15.94:0/703012884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3341029e0 0x7fe334199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 >> 172.21.15.94:0/703012884 conn(0x7fe3340fdbd0 msgr2=0x7fe334104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 shutdown_connections 2024-04-25T13:51:05.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.366+0000 7fe339851640 1 -- 172.21.15.94:0/703012884 wait complete. 2024-04-25T13:51:05.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:51:05.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:05.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.499+0000 7fc0af93f640 1 -- 172.21.15.94:0/3695583470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 msgr2=0x7fc0a81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.499+0000 7fc0af93f640 1 --2- 172.21.15.94:0/3695583470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a81036c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ca0 tx=0x7fc0a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 -- 172.21.15.94:0/3695583470 shutdown_connections 2024-04-25T13:51:05.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 --2- 172.21.15.94:0/3695583470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8108900 0x7fc0a810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 --2- 172.21.15.94:0/3695583470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8103c00 0x7fc0a8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 --2- 172.21.15.94:0/3695583470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a81036c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 -- 172.21.15.94:0/3695583470 >> 172.21.15.94:0/3695583470 conn(0x7fc0a80fdbd0 msgr2=0x7fc0a8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 -- 172.21.15.94:0/3695583470 shutdown_connections 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 -- 172.21.15.94:0/3695583470 wait complete. 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 Processor -- start 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.500+0000 7fc0af93f640 1 -- start start 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8103c00 0x7fc0a8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8108900 0x7fc0a81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810e4c0 con 0x7fc0a81029c0 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810e340 con 0x7fc0a8108900 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810e640 con 0x7fc0a8103c00 2024-04-25T13:51:05.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45674/0 (socket says 172.21.15.94:45674) 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 -- 172.21.15.94:0/2595464827 learned_addr learned my addr 172.21.15.94:0/2595464827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0aceda640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8103c00 0x7fc0a8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0adedc640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8108900 0x7fc0a81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 -- 172.21.15.94:0/2595464827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8103c00 msgr2=0x7fc0a8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8103c00 0x7fc0a8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 -- 172.21.15.94:0/2595464827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8108900 msgr2=0x7fc0a81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8108900 0x7fc0a81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 -- 172.21.15.94:0/2595464827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81a0c60 con 0x7fc0a81029c0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0ad6db640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40098d0 tx=0x7fc0a4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4003b70 con 0x7fc0a81029c0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a81a0ef0 con 0x7fc0a81029c0 2024-04-25T13:51:05.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a4003d10 con 0x7fc0a81029c0 2024-04-25T13:51:05.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a402fa50 con 0x7fc0a81029c0 2024-04-25T13:51:05.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.501+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a1360 con 0x7fc0a81029c0 2024-04-25T13:51:05.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.502+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8107710 con 0x7fc0a81029c0 2024-04-25T13:51:05.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.504+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc0a4004580 con 0x7fc0a81029c0 2024-04-25T13:51:05.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.504+0000 7fc09e7fc640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 0x7fc0940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.505+0000 7fc0aceda640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 0x7fc0940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.505+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fc0a41001a0 con 0x7fc0a81029c0 2024-04-25T13:51:05.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.505+0000 7fc0aceda640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 0x7fc0940b2000 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc0a819a1c0 tx=0x7fc098005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.509+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0a40c8ce0 con 0x7fc0a81029c0 2024-04-25T13:51:05.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.750+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0a802dc70 con 0x7fc0a81029c0 2024-04-25T13:51:05.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.750+0000 7fc09e7fc640 1 -- 172.21.15.94:0/2595464827 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fc0a40c85e0 con 0x7fc0a81029c0 2024-04-25T13:51:05.750 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:51:05.751 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:46:41.517486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26611":{"gid":26611,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1482550752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1482550752},{"type":"v1","addr":"172.21.15.138:6835","nonce":1482550752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:51:05.751 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 msgr2=0x7fc0940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 0x7fc0940b2000 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc0a819a1c0 tx=0x7fc098005f90 comp rx=0 tx=0).stop 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 msgr2=0x7fc0a8199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40098d0 tx=0x7fc0a4031700 comp rx=0 tx=0).stop 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 shutdown_connections 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0940afaf0 0x7fc0940b2000 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0a8108900 0x7fc0a81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0a8103c00 0x7fc0a8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 --2- 172.21.15.94:0/2595464827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0a81029c0 0x7fc0a8199120 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 >> 172.21.15.94:0/2595464827 conn(0x7fc0a80fdbd0 msgr2=0x7fc0a81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 shutdown_connections 2024-04-25T13:51:05.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.753+0000 7fc0af93f640 1 -- 172.21.15.94:0/2595464827 wait complete. 2024-04-25T13:51:05.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:51:05.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1130451 12.82 MB/sec execute 348 sec latency 17.666 ms 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 -- 172.21.15.94:0/2147442246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 msgr2=0x7fa31c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 --2- 172.21.15.94:0/2147442246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c104030 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009900 tx=0x7fa30c02a410 comp rx=0 tx=0).stop 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 -- 172.21.15.94:0/2147442246 shutdown_connections 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 --2- 172.21.15.94:0/2147442246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 0x7fa31c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 --2- 172.21.15.94:0/2147442246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c104030 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 --2- 172.21.15.94:0/2147442246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 0x7fa31c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 -- 172.21.15.94:0/2147442246 >> 172.21.15.94:0/2147442246 conn(0x7fa31c0fdbd0 msgr2=0x7fa31c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 -- 172.21.15.94:0/2147442246 shutdown_connections 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 -- 172.21.15.94:0/2147442246 wait complete. 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.882+0000 7fa324338640 1 Processor -- start 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 -- start start 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 0x7fa31c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 0x7fa31c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c10d400 con 0x7fa31c103c20 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c10d280 con 0x7fa31c1029e0 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa324338640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c10d580 con 0x7fa31c108920 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45702/0 (socket says 172.21.15.94:45702) 2024-04-25T13:51:05.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 -- 172.21.15.94:0/1969757553 learned_addr learned my addr 172.21.15.94:0/1969757553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3220d4640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 0x7fa31c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 0x7fa31c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 -- 172.21.15.94:0/1969757553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 msgr2=0x7fa31c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 0x7fa31c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 -- 172.21.15.94:0/1969757553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 msgr2=0x7fa31c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 0x7fa31c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 -- 172.21.15.94:0/1969757553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c19fe70 con 0x7fa31c103c20 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.883+0000 7fa3218d3640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.884+0000 7fa3137fe640 1 -- 172.21.15.94:0/1969757553 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c03f030 con 0x7fa31c103c20 2024-04-25T13:51:05.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.884+0000 7fa3137fe640 1 -- 172.21.15.94:0/1969757553 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30c031910 con 0x7fa31c103c20 2024-04-25T13:51:05.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.884+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1a0100 con 0x7fa31c103c20 2024-04-25T13:51:05.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.884+0000 7fa3137fe640 1 -- 172.21.15.94:0/1969757553 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c035070 con 0x7fa31c103c20 2024-04-25T13:51:05.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.884+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a0640 con 0x7fa31c103c20 2024-04-25T13:51:05.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.886+0000 7fa3137fe640 1 -- 172.21.15.94:0/1969757553 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa30c042030 con 0x7fa31c103c20 2024-04-25T13:51:05.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.886+0000 7fa32b1ef740 1 Processor -- start 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 -- start start 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56452fea8d80 con 0x56452fb816b0 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56452fe62a20 con 0x56452fb794d0 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56452fea4d40 con 0x56452fba2390 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43554/0 (socket says 172.21.15.94:43554) 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 -- 172.21.15.94:0/1140897265 learned_addr learned my addr 172.21.15.94:0/1140897265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3137fe640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 0x7fa3080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 -- 172.21.15.94:0/1140897265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 msgr2=0x56452ff44810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff44810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 -- 172.21.15.94:0/1140897265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 msgr2=0x56452ff45a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff45a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3220d4640 1 -- 172.21.15.94:0/1140897265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452fe6dc30 con 0x56452fb794d0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3218d3640 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 0x7fa3080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.887+0000 7fa3137fe640 1 -- 172.21.15.94:0/1969757553 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa30c0ff080 con 0x7fa31c103c20 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.888+0000 7fa3220d4640 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c60 tx=0x7fa318011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535bb65186cfc429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.888+0000 7fa310bfe640 1 -- 172.21.15.94:0/1140897265 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318004220 con 0x56452fb794d0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.888+0000 7fa310bfe640 1 -- 172.21.15.94:0/1140897265 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3180043c0 con 0x56452fb794d0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.888+0000 7fa310bfe640 1 -- 172.21.15.94:0/1140897265 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318004d20 con 0x56452fb794d0 2024-04-25T13:51:05.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1140897265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 msgr2=0x56452ff44d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 0x7fa3080b2080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56452ff45860 tx=0x7fa314004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c60 tx=0x7fa318011810 comp rx=0 tx=0).stop 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1140897265 shutdown_connections 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff45a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff44d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1140897265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff44810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1140897265 >> 172.21.15.94:0/1140897265 conn(0x56452fb964e0 msgr2=0x56452ff40020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1140897265 shutdown_connections 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1140897265 wait complete. 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 -- start start 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.889+0000 7fa32b1ef740 1 Processor -- start 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 0x56452ff4e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb816b0 0x56452ff4ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56452ff46530 con 0x56452fba2390 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56452fe2fdc0 con 0x56452fb816b0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56452ff466b0 con 0x56452fb794d0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 10 client.0 _ll_register_callbacks cb 0x56452fe4b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb816b0 0x56452ff4ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45726/0 (socket says 172.21.15.94:45726) 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3220d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 0x56452ff4e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 -- 172.21.15.94:0/2100508041 learned_addr learned my addr 172.21.15.94:0/2100508041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 -- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 msgr2=0x56452ff4e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 0x56452ff4e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 -- 172.21.15.94:0/2100508041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb816b0 msgr2=0x56452ff4ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb816b0 0x56452ff4ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 -- 172.21.15.94:0/2100508041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452fbba3f0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3220d4640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 0x56452ff4e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa3228d5640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa31401acc0 tx=0x7fa31400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec766eec7502da8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314025440 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa3140255e0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.890+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3180043c0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3140257e0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 10 client.18810 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56452fc847d0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3180043c0 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa31402e960 con 0x56452fba2390 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 10 client.18810 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:05.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 10 client.18810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 20 client.18810 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb60f60 0x56452ff5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 4 client.18810 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56452feb6990 con 0x56452fb60f60 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.891+0000 7fa3220d4640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb60f60 0x56452ff5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa3220d4640 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb60f60 0x56452ff5c6d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b31f12e02da6d9ac server_cookie=33eb34255dacfbe5 in_seq=0 out_seq=0 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa305ffb640 10 client.18810 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa305ffb640 1 -- 172.21.15.94:0/2100508041 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56452feb6990 con 0x56452fb60f60 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa305ffb640 10 client.18810 handle_command_reply: tid=0 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa32b1ef740 1 client.18810 shutdown 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.892+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 msgr2=0x56452ff559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa31401acc0 tx=0x7fa31400f040 comp rx=0 tx=0).stop 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 shutdown_connections 2024-04-25T13:51:05.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb60f60 0x56452ff5c6d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff559d0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb816b0 0x56452ff4ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa305ffb640 0 client.18810 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/2100508041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb794d0 0x56452ff4e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 >> 172.21.15.94:0/2100508041 conn(0x56452fb964e0 msgr2=0x56452ff42f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 shutdown_connections 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 1 -- 172.21.15.94:0/2100508041 wait complete. 2024-04-25T13:51:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.893+0000 7fa32b1ef740 20 client.18810 trim_cache size 0 max 16384 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 Processor -- start 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 -- start start 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff5c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb816b0 0x56452ff44790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1a9350 con 0x56452fba2390 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1aae90 con 0x56452fb794d0 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1a9a20 con 0x56452fb816b0 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45742/0 (socket says 172.21.15.94:45742) 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 -- 172.21.15.94:0/1105070517 learned_addr learned my addr 172.21.15.94:0/1105070517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3220d4640 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff5c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 -- 172.21.15.94:0/1105070517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb816b0 msgr2=0x56452ff44790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb816b0 0x56452ff44790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 -- 172.21.15.94:0/1105070517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 msgr2=0x56452ff5c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff5c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 -- 172.21.15.94:0/1105070517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452ff4f130 con 0x56452fba2390 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.907+0000 7fa3228d5640 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fa314031040 tx=0x7fa314018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726946fc71c37817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.908+0000 7fa307fff640 1 -- 172.21.15.94:0/1105070517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314025640 con 0x56452fba2390 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.908+0000 7fa307fff640 1 -- 172.21.15.94:0/1105070517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31402a070 con 0x56452fba2390 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.908+0000 7fa307fff640 1 -- 172.21.15.94:0/1105070517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314039440 con 0x56452fba2390 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.908+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1105070517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 msgr2=0x56452ff44cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.908+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fa314031040 tx=0x7fa314018f90 comp rx=0 tx=0).stop 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1105070517 shutdown_connections 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fba2390 0x56452ff44cd0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fb816b0 0x56452ff44790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/1105070517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff5c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1105070517 >> 172.21.15.94:0/1105070517 conn(0x56452fb964e0 msgr2=0x56452ff4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1105070517 shutdown_connections 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- 172.21.15.94:0/1105070517 wait complete. 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- start start 2024-04-25T13:51:05.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 Processor -- start 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff55020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1aa7c0 con 0x56452fb816b0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1aa0f0 con 0x56452fb794d0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1a7330 con 0x56452fba2390 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa32b1ef740 10 client.0 _ll_register_callbacks cb 0x7fa314009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa3218d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45750/0 (socket says 172.21.15.94:45750) 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa3220d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff55020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa3218d3640 1 -- 172.21.15.94:0/3151523379 learned_addr learned my addr 172.21.15.94:0/3151523379 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.909+0000 7fa3228d5640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 -- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 msgr2=0x56452ff53d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff53d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 -- 172.21.15.94:0/3151523379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 msgr2=0x56452ff55020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff55020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 -- 172.21.15.94:0/3151523379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452ff54490 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3220d4640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa3218d3640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0c7310 tx=0x7fa30c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f4f03211b9c935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c049040 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fa30c0c38a0 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31402a070 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa30c0c73e0 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa32b1ef740 10 client.18816 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56452ffc3b60 con 0x56452fb816b0 2024-04-25T13:51:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.910+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31402a070 con 0x56452fb816b0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa30c0c7870 con 0x56452fb816b0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 10 client.18816 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 10 client.18816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 20 client.18816 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb363a0 0x56452ffc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 4 client.18816 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56452feb0800 con 0x56452fb363a0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa3220d4640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb363a0 0x56452ffc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa305ffb640 10 client.18816 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa3220d4640 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb363a0 0x56452ffc6530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95f41b77c2a0be0c server_cookie=e8297a19557bd680 in_seq=0 out_seq=0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa305ffb640 1 -- 172.21.15.94:0/3151523379 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56452feb0800 con 0x56452fb363a0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa305ffb640 10 client.18816 handle_command_reply: tid=0 2024-04-25T13:51:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 client.18816 shutdown 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 msgr2=0x56452ff53690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0c7310 tx=0x7fa30c009880 comp rx=0 tx=0).stop 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 shutdown_connections 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56452fb363a0 0x56452ffc6530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.911+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56452fba2390 0x56452ff53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56452fb816b0 0x56452ff53690 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa305ffb640 0 client.18816 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 1 --2- 172.21.15.94:0/3151523379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56452fb794d0 0x56452ff55020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 >> 172.21.15.94:0/3151523379 conn(0x56452fb964e0 msgr2=0x56452ff44790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 shutdown_connections 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 1 -- 172.21.15.94:0/3151523379 wait complete. 2024-04-25T13:51:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa32b1ef740 20 client.18816 trim_cache size 0 max 16384 2024-04-25T13:51:05.919 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:05.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 msgr2=0x7fa3080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 0x7fa3080b2080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56452ff45860 tx=0x7fa314004a40 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 msgr2=0x7fa31c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 shutdown_connections 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3080afb70 0x7fa3080b2080 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31c108920 0x7fa31c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa31c103c20 0x7fa31c198870 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 --2- 172.21.15.94:0/1969757553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa31c1029e0 0x7fa31c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 >> 172.21.15.94:0/1969757553 conn(0x7fa31c0fdbd0 msgr2=0x7fa31c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 shutdown_connections 2024-04-25T13:51:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:05.912+0000 7fa324338640 1 -- 172.21.15.94:0/1969757553 wait complete. 2024-04-25T13:51:05.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:51:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 -- 172.21.15.94:0/1359870016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 msgr2=0x7f7c0c105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 --2- 172.21.15.94:0/1359870016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c105130 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009900 tx=0x7f7c0002a3f0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 -- 172.21.15.94:0/1359870016 shutdown_connections 2024-04-25T13:51:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 --2- 172.21.15.94:0/1359870016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 0x7f7c0c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 --2- 172.21.15.94:0/1359870016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c105130 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 --2- 172.21.15.94:0/1359870016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 0x7f7c0c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.081+0000 7f7c1200c640 1 -- 172.21.15.94:0/1359870016 >> 172.21.15.94:0/1359870016 conn(0x7f7c0c0fecd0 msgr2=0x7f7c0c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.082+0000 7f7c1200c640 1 -- 172.21.15.94:0/1359870016 shutdown_connections 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.082+0000 7f7c1200c640 1 -- 172.21.15.94:0/1359870016 wait complete. 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.082+0000 7f7c1200c640 1 Processor -- start 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.082+0000 7f7c1200c640 1 -- start start 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 0x7f7c0c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 0x7f7c0c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10f5e0 con 0x7f7c0c104d20 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10f460 con 0x7f7c0c103ae0 2024-04-25T13:51:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c1200c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10f760 con 0x7f7c0c109a20 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 0x7f7c0c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45774/0 (socket says 172.21.15.94:45774) 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 0x7f7c0c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 -- 172.21.15.94:0/106113745 learned_addr learned my addr 172.21.15.94:0/106113745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 -- 172.21.15.94:0/106113745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 msgr2=0x7f7c0c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 0x7f7c0c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 -- 172.21.15.94:0/106113745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 msgr2=0x7f7c0c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 0x7f7c0c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 -- 172.21.15.94:0/106113745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1a1c90 con 0x7f7c0c104d20 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c0affd640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7c00006010 tx=0x7f7c00031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.083+0000 7f7c08ff9640 1 -- 172.21.15.94:0/106113745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0003f030 con 0x7f7c0c104d20 2024-04-25T13:51:06.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.084+0000 7f7c08ff9640 1 -- 172.21.15.94:0/106113745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c00003aa0 con 0x7f7c0c104d20 2024-04-25T13:51:06.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.084+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1a1f20 con 0x7f7c0c104d20 2024-04-25T13:51:06.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.084+0000 7f7c08ff9640 1 -- 172.21.15.94:0/106113745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00035070 con 0x7f7c0c104d20 2024-04-25T13:51:06.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.084+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a2380 con 0x7f7c0c104d20 2024-04-25T13:51:06.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 Processor -- start 2024-04-25T13:51:06.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c08ff9640 1 -- 172.21.15.94:0/106113745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7c00042030 con 0x7f7c0c104d20 2024-04-25T13:51:06.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 -- start start 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f85d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b84ee9d80 con 0x564b84be3390 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b84ea3a20 con 0x564b84bc26b0 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.086+0000 7f7c18f97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b84ee5d40 con 0x564b84bba4d0 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45788/0 (socket says 172.21.15.94:45788) 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0bfff640 1 -- 172.21.15.94:0/3883825208 learned_addr learned my addr 172.21.15.94:0/3883825208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f85d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3883825208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 msgr2=0x564b84f86a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f86a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c08ff9640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 0x7f7bf40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3883825208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 msgr2=0x564b84f85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3883825208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b84eaec30 con 0x564b84bba4d0 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0bfff640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0affd640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f86a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0bfff640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 0x7f7bf40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c08ff9640 1 -- 172.21.15.94:0/106113745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7c000ff1c0 con 0x7f7c0c104d20 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.087+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f85d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c1033b0 tx=0x7f7bfc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c4e08493ccc70d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.088+0000 7f7bf1ffb640 1 -- 172.21.15.94:0/3883825208 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004050 con 0x564b84bba4d0 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.088+0000 7f7c0bfff640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 0x7f7bf40b2080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x564b84f86860 tx=0x7f7c04004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.088+0000 7f7bf1ffb640 1 -- 172.21.15.94:0/3883825208 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc0054d0 con 0x564b84bba4d0 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.088+0000 7f7bf1ffb640 1 -- 172.21.15.94:0/3883825208 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004c20 con 0x564b84bba4d0 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- 172.21.15.94:0/3883825208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 msgr2=0x564b84f85d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f85d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c1033b0 tx=0x7f7bfc011810 comp rx=0 tx=0).stop 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- 172.21.15.94:0/3883825208 shutdown_connections 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f86a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f85d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3883825208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84f85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- 172.21.15.94:0/3883825208 >> 172.21.15.94:0/3883825208 conn(0x564b84bd74e0 msgr2=0x564b84f81020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- 172.21.15.94:0/3883825208 shutdown_connections 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- 172.21.15.94:0/3883825208 wait complete. 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 -- start start 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.089+0000 7f7c18f97740 1 Processor -- start 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84be3390 0x564b84f96a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b84f87530 con 0x564b84bc26b0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b84e70dc0 con 0x564b84be3390 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b84f876b0 con 0x564b84bba4d0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c18f97740 10 client.0 _ll_register_callbacks cb 0x564b84e8c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84be3390 0x564b84f96a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45800/0 (socket says 172.21.15.94:45800) 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0affd640 1 -- 172.21.15.94:0/753325859 learned_addr learned my addr 172.21.15.94:0/753325859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84be3390 msgr2=0x564b84f96a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84be3390 0x564b84f96a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/753325859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 msgr2=0x564b84f8fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/753325859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b84bfb3f0 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0affd640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8f590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc000c00 tx=0x7f7bfc009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fc880b289e96ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7beaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.090+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc029070 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7bfc003a00 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bfc0054d0 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7bfc035050 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 10 client.28132 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b84cc57d0 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc0054d0 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7bfc005120 con 0x564b84bba4d0 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 10 client.28132 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 10 client.28132 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 20 client.28132 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84ba1f60 0x564b84f9cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 4 client.28132 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.091+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b84ef7990 con 0x564b84ba1f60 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.092+0000 7f7c0affd640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84ba1f60 0x564b84f9cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.092+0000 7f7beaffd640 10 client.28132 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.092+0000 7f7c0affd640 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84ba1f60 0x564b84f9cfb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90de22a28bf18a9d server_cookie=5f11131937b948e in_seq=0 out_seq=0 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7beaffd640 1 -- 172.21.15.94:0/753325859 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b84ef7990 con 0x564b84ba1f60 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7beaffd640 10 client.28132 handle_command_reply: tid=0 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 client.28132 shutdown 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 msgr2=0x564b84f8f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8f590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc000c00 tx=0x7f7bfc009fa0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 shutdown_connections 2024-04-25T13:51:06.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84ba1f60 0x564b84f9cfb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84be3390 0x564b84f96a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f8fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 --2- 172.21.15.94:0/753325859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8f590 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7beaffd640 0 client.28132 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 >> 172.21.15.94:0/753325859 conn(0x564b84bd74e0 msgr2=0x564b84f83f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 shutdown_connections 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 1 -- 172.21.15.94:0/753325859 wait complete. 2024-04-25T13:51:06.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.093+0000 7f7c18f97740 20 client.28132 trim_cache size 0 max 16384 2024-04-25T13:51:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 Processor -- start 2024-04-25T13:51:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 -- start start 2024-04-25T13:51:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f96950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84fa2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1ab4f0 con 0x564b84be3390 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1ad030 con 0x564b84bc26b0 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.109+0000 7f7c18f97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1abbc0 con 0x564b84bba4d0 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84fa2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84fa2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45816/0 (socket says 172.21.15.94:45816) 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0bfff640 1 -- 172.21.15.94:0/3976048687 learned_addr learned my addr 172.21.15.94:0/3976048687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0affd640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f96950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3976048687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 msgr2=0x564b84f96950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f96950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3976048687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 msgr2=0x564b84fa2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84fa2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 -- 172.21.15.94:0/3976048687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b84f87410 con 0x564b84bba4d0 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0affd640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f96950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8e3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0095f0 tx=0x7f7bfc0238a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6babd8880f7eb83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7bf17fa640 1 -- 172.21.15.94:0/3976048687 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc029070 con 0x564b84bba4d0 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7bf17fa640 1 -- 172.21.15.94:0/3976048687 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc023c60 con 0x564b84bba4d0 2024-04-25T13:51:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.110+0000 7f7bf17fa640 1 -- 172.21.15.94:0/3976048687 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc02e440 con 0x564b84bba4d0 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 -- 172.21.15.94:0/3976048687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 msgr2=0x564b84f8e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8e3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0095f0 tx=0x7f7bfc0238a0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 -- 172.21.15.94:0/3976048687 shutdown_connections 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84be3390 0x564b84fa2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bc26b0 0x564b84f96950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 --2- 172.21.15.94:0/3976048687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84bba4d0 0x564b84f8e3c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 -- 172.21.15.94:0/3976048687 >> 172.21.15.94:0/3976048687 conn(0x564b84bd74e0 msgr2=0x564b84f8dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.111+0000 7f7c18f97740 1 -- 172.21.15.94:0/3976048687 shutdown_connections 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 -- 172.21.15.94:0/3976048687 wait complete. 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 -- start start 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 Processor -- start 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 0x564b84f95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84be3390 0x564b84f95be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1ac960 con 0x564b84bc26b0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1ac290 con 0x564b84bba4d0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.112+0000 7f7c18f97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1a94d0 con 0x564b84be3390 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c18f97740 10 client.0 _ll_register_callbacks cb 0x7f7c04007960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 0x564b84f95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45832/0 (socket says 172.21.15.94:45832) 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 -- 172.21.15.94:0/4159585333 learned_addr learned my addr 172.21.15.94:0/4159585333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0bfff640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84be3390 0x564b84f95be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 -- 172.21.15.94:0/4159585333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84be3390 msgr2=0x564b84f95be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84be3390 0x564b84f95be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 -- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 msgr2=0x564b84f95160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 0x564b84f95160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 -- 172.21.15.94:0/4159585333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b850032b0 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 0x564b84f95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c0affd640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7c000fcf90 tx=0x7f7c00102b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7a05e5ec4e3b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7beaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00049040 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f7c00003c50 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.113+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bfc023c60 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c0003b460 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 10 client.18834 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b85004530 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc023c60 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7c000c3940 con 0x564b84bc26b0 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 10 client.18834 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 10 client.18834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:51:06.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 20 client.18834 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84b773a0 0x564b85006df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 4 client.18834 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b84ef1800 con 0x564b84b773a0 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.114+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84b773a0 0x564b85006df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7beaffd640 10 client.18834 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c0b7fe640 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84b773a0 0x564b85006df0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75f9f2bc01781f9a server_cookie=2d71d952651ebb43 in_seq=0 out_seq=0 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7beaffd640 1 -- 172.21.15.94:0/4159585333 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b84ef1800 con 0x564b84b773a0 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7beaffd640 10 client.18834 handle_command_reply: tid=0 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c18f97740 1 client.18834 shutdown 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 msgr2=0x564b84f956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7c000fcf90 tx=0x7f7c00102b00 comp rx=0 tx=0).stop 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 shutdown_connections 2024-04-25T13:51:06.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.115+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564b84b773a0 0x564b85006df0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7beaffd640 0 client.18834 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b84be3390 0x564b84f95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b84bc26b0 0x564b84f956a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 --2- 172.21.15.94:0/4159585333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b84bba4d0 0x564b84f95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 >> 172.21.15.94:0/4159585333 conn(0x564b84bd74e0 msgr2=0x564b84f845b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 shutdown_connections 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 1 -- 172.21.15.94:0/4159585333 wait complete. 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c18f97740 20 client.18834 trim_cache size 0 max 16384 2024-04-25T13:51:06.121 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:06.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 msgr2=0x7f7bf40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 0x7f7bf40b2080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x564b84f86860 tx=0x7f7c04004ab0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 msgr2=0x7f7c0c19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7c00006010 tx=0x7f7c00031a50 comp rx=0 tx=0).stop 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 shutdown_connections 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7bf40afb70 0x7f7bf40b2080 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c0c109a20 0x7f7c0c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c0c104d20 0x7f7c0c19a7b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 --2- 172.21.15.94:0/106113745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c103ae0 0x7f7c0c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.116+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 >> 172.21.15.94:0/106113745 conn(0x7f7c0c0fecd0 msgr2=0x7f7c0c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.117+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 shutdown_connections 2024-04-25T13:51:06.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.117+0000 7f7c1200c640 1 -- 172.21.15.94:0/106113745 wait complete. 2024-04-25T13:51:06.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 -- 172.21.15.94:0/2712282081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 msgr2=0x7f611410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 --2- 172.21.15.94:0/2712282081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f611410bcf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f610c009ca0 tx=0x7f610c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 -- 172.21.15.94:0/2712282081 shutdown_connections 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 --2- 172.21.15.94:0/2712282081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f611410bcf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 --2- 172.21.15.94:0/2712282081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 0x7f6114109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 --2- 172.21.15.94:0/2712282081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 0x7f61141066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 -- 172.21.15.94:0/2712282081 >> 172.21.15.94:0/2712282081 conn(0x7f61140fdc60 msgr2=0x7f61141000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 -- 172.21.15.94:0/2712282081 shutdown_connections 2024-04-25T13:51:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.152+0000 7f611bd4d640 1 -- 172.21.15.94:0/2712282081 wait complete. 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 Processor -- start 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 -- start start 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 0x7f611419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 0x7f611419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f611410f670 con 0x7f61141042c0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f611410f4f0 con 0x7f6114106cb0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.153+0000 7f611bd4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f611410f7f0 con 0x7f61141097e0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f6119ae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 0x7f611419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57830/0 (socket says 172.21.15.94:57830) 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 -- 172.21.15.94:0/781126520 learned_addr learned my addr 172.21.15.94:0/781126520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f61192e8640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 0x7f611419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 -- 172.21.15.94:0/781126520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 msgr2=0x7f611419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 0x7f611419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 -- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 msgr2=0x7f611419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 0x7f611419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 -- 172.21.15.94:0/781126520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61141a1d20 con 0x7f61141097e0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f611a2ea640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f610c004af0 tx=0x7f610c031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.154+0000 7f610affd640 1 -- 172.21.15.94:0/781126520 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c003b70 con 0x7f61141097e0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.155+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61141a1fb0 con 0x7f61141097e0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.155+0000 7f610affd640 1 -- 172.21.15.94:0/781126520 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c003d10 con 0x7f61141097e0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.155+0000 7f610affd640 1 -- 172.21.15.94:0/781126520 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c02fb20 con 0x7f61141097e0 2024-04-25T13:51:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.155+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61141a2410 con 0x7f61141097e0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.157+0000 7f610affd640 1 -- 172.21.15.94:0/781126520 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f610c02fcc0 con 0x7f61141097e0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.157+0000 7f6122c00740 1 Processor -- start 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 -- start start 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364274970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5583641da460 con 0x558363eab2f0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558364194590 con 0x558363ed41b0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6122c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583641d6420 con 0x558363eb34d0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45864/0 (socket says 172.21.15.94:45864) 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 -- 172.21.15.94:0/316152335 learned_addr learned my addr 172.21.15.94:0/316152335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f61192e8640 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f61192e8640 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364274970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f610affd640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 0x7f61000b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 -- 172.21.15.94:0/316152335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 msgr2=0x55836427bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 -- 172.21.15.94:0/316152335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 msgr2=0x558364274970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364274970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f6119ae9640 1 -- 172.21.15.94:0/316152335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583641a10d0 con 0x558363eab2f0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.158+0000 7f611a2ea640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 0x7f61000b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.159+0000 7f6119ae9640 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c90 tx=0x7f6110011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162e299f88a4ff02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.159+0000 7f610affd640 1 -- 172.21.15.94:0/781126520 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f610c100270 con 0x7f61141097e0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.159+0000 7f60fffff640 1 -- 172.21.15.94:0/316152335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611001e070 con 0x558363eab2f0 2024-04-25T13:51:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.159+0000 7f60fffff640 1 -- 172.21.15.94:0/316152335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6110009e40 con 0x558363eab2f0 2024-04-25T13:51:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.159+0000 7f60fffff640 1 -- 172.21.15.94:0/316152335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100186e0 con 0x558363eab2f0 2024-04-25T13:51:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- 172.21.15.94:0/316152335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 msgr2=0x558364274ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f611a2ea640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 0x7f61000b1ff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f610c039e40 tx=0x7f610c0a5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c90 tx=0x7f6110011810 comp rx=0 tx=0).stop 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- 172.21.15.94:0/316152335 shutdown_connections 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x558364274ef0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 --2- 172.21.15.94:0/316152335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364274970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- 172.21.15.94:0/316152335 >> 172.21.15.94:0/316152335 conn(0x558363ec8300 msgr2=0x558364270300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- 172.21.15.94:0/316152335 shutdown_connections 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- 172.21.15.94:0/316152335 wait complete. 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 -- start start 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.160+0000 7f6122c00740 1 Processor -- start 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364285ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55836427c840 con 0x558363eab2f0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558364161b60 con 0x558363ed41b0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55836427c9c0 con 0x558363eb34d0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6122c00740 10 client.0 _ll_register_callbacks cb 0x5583641862b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45868/0 (socket says 172.21.15.94:45868) 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 -- 172.21.15.94:0/2517624951 learned_addr learned my addr 172.21.15.94:0/2517624951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364285ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f6119ae9640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 -- 172.21.15.94:0/2517624951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 msgr2=0x55836427eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 -- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 msgr2=0x558364285ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364285ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 -- 172.21.15.94:0/2517624951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558363eec210 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f61192e8640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6104004620 tx=0x7f6104004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a29ab42a2b08fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.161+0000 7f60fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104011900 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6104011aa0 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583641e8070 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6104011ca0 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 10 client.18846 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558363fb8620 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583641e8070 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f610402c450 con 0x558363eab2f0 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 10 client.18846 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 10 client.18846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 20 client.18846 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e92d80 0x55836428c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 4 client.18846 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.162+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583641e1ee0 con 0x558363e92d80 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.163+0000 7f6119ae9640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e92d80 0x55836428c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.163+0000 7f60fd7fa640 10 client.18846 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.163+0000 7f6119ae9640 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e92d80 0x55836428c420 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221e0e2d3df1c64d server_cookie=f8cb1be55a00522d in_seq=0 out_seq=0 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f60fd7fa640 1 -- 172.21.15.94:0/2517624951 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583641e1ee0 con 0x558363e92d80 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f60fd7fa640 10 client.18846 handle_command_reply: tid=0 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 client.18846 shutdown 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 msgr2=0x55836427e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6104004620 tx=0x7f6104004730 comp rx=0 tx=0).stop 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 shutdown_connections 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e92d80 0x55836428c420 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363ed41b0 0x558364285ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x55836427eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f60fd7fa640 0 client.18846 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 --2- 172.21.15.94:0/2517624951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eab2f0 0x55836427e640 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 >> 172.21.15.94:0/2517624951 conn(0x558363ec8300 msgr2=0x558364273210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 shutdown_connections 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 1 -- 172.21.15.94:0/2517624951 wait complete. 2024-04-25T13:51:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.165+0000 7f6122c00740 20 client.18846 trim_cache size 0 max 16384 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 Processor -- start 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 -- start start 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x55836427e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363ed41b0 0x55836427cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ab440 con 0x558363eb34d0 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61141acf80 con 0x558363eab2f0 2024-04-25T13:51:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6122c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61141abb10 con 0x558363ed41b0 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6119ae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x55836427e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6119ae9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45872/0 (socket says 172.21.15.94:45872) 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6119ae9640 1 -- 172.21.15.94:0/4150076391 learned_addr learned my addr 172.21.15.94:0/4150076391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363ed41b0 0x55836427cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 -- 172.21.15.94:0/4150076391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363ed41b0 msgr2=0x55836427cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363ed41b0 0x55836427cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 -- 172.21.15.94:0/4150076391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 msgr2=0x558364285a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f61192e8640 1 -- 172.21.15.94:0/4150076391 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836427be10 con 0x558363eab2f0 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.182+0000 7f6119ae9640 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.183+0000 7f61192e8640 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x55836427e920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6104004560 tx=0x7f6104005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4e2cf7439d0d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.183+0000 7f60ff7fe640 1 -- 172.21.15.94:0/4150076391 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104032070 con 0x558363eab2f0 2024-04-25T13:51:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.183+0000 7f60ff7fe640 1 -- 172.21.15.94:0/4150076391 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6104011900 con 0x558363eab2f0 2024-04-25T13:51:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.183+0000 7f60ff7fe640 1 -- 172.21.15.94:0/4150076391 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104005070 con 0x558363eab2f0 2024-04-25T13:51:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- 172.21.15.94:0/4150076391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 msgr2=0x55836427e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x55836427e920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6104004560 tx=0x7f6104005f90 comp rx=0 tx=0).stop 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- 172.21.15.94:0/4150076391 shutdown_connections 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363ed41b0 0x55836427cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363eb34d0 0x558364285a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 --2- 172.21.15.94:0/4150076391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x55836427e920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- 172.21.15.94:0/4150076391 >> 172.21.15.94:0/4150076391 conn(0x558363ec8300 msgr2=0x55836427e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- 172.21.15.94:0/4150076391 shutdown_connections 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- 172.21.15.94:0/4150076391 wait complete. 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 -- start start 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.184+0000 7f6122c00740 1 Processor -- start 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x558364282f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x558364283440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 0x558364284320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ac8b0 con 0x558363ed41b0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ac1e0 con 0x558363eab2f0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61141a9420 con 0x558363eb34d0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6122c00740 10 client.0 _ll_register_callbacks cb 0x7f6104009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x558364282f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 0x558364284320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 0x558364284320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45878/0 (socket says 172.21.15.94:45878) 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 -- 172.21.15.94:0/3147933025 learned_addr learned my addr 172.21.15.94:0/3147933025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f6119ae9640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x558364283440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 -- 172.21.15.94:0/3147933025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 msgr2=0x558364283440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x558364283440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 -- 172.21.15.94:0/3147933025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 msgr2=0x558364284320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 0x558364284320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.185+0000 7f61192e8640 1 -- 172.21.15.94:0/3147933025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583642848e0 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f61192e8640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x558364282f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61040097a0 tx=0x7f6104000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5eaf3217057c7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104032050 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6104011900 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f6104022670 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f610402c660 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 10 client.28316 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558364285b70 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6104011900 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6104010430 con 0x558363eab2f0 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 10 client.28316 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 10 client.28316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 20 client.28316 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e681c0 0x5583642f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 4 client.28316 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:51:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.186+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55836415d5f0 con 0x558363e681c0 2024-04-25T13:51:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.187+0000 7f6119ae9640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e681c0 0x5583642f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.187+0000 7f60fd7fa640 10 client.28316 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.187+0000 7f6119ae9640 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e681c0 0x5583642f6c20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34f2b3358009c81 server_cookie=abe6e577d82d7a35 in_seq=0 out_seq=0 2024-04-25T13:51:06.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.282+0000 7fdc3ffff640 1 -- 172.21.15.94:0/762391122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010a350 msgr2=0x7fdc4010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.282+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/762391122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010a350 0x7fdc4010ac90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009900 tx=0x7fdc2802a410 comp rx=0 tx=0).stop 2024-04-25T13:51:06.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.289+0000 7fdc3ffff640 1 -- 172.21.15.94:0/762391122 shutdown_connections 2024-04-25T13:51:06.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.289+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/762391122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc40110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.289+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/762391122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010b1d0 0x7fdc4010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.289+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/762391122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010a350 0x7fdc4010ac90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.289+0000 7fdc3ffff640 1 -- 172.21.15.94:0/762391122 >> 172.21.15.94:0/762391122 conn(0x7fdc40070400 msgr2=0x7fdc40072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.297+0000 7fdc3ffff640 1 -- 172.21.15.94:0/762391122 shutdown_connections 2024-04-25T13:51:06.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.301+0000 7fdc3ffff640 1 -- 172.21.15.94:0/762391122 wait complete. 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.302+0000 7fdc3ffff640 1 Processor -- start 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.302+0000 7fdc3ffff640 1 -- start start 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 0x7fdc40116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010b1d0 0x7fdc40117530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc401184d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4011e2b0 con 0x7fdc4010a350 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4011e130 con 0x7fdc4010b1d0 2024-04-25T13:51:06.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.303+0000 7fdc3ffff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4011e430 con 0x7fdc4010fef0 2024-04-25T13:51:06.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.306+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc401184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.307+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc401184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57876/0 (socket says 172.21.15.94:57876) 2024-04-25T13:51:06.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.307+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3338421594 learned_addr learned my addr 172.21.15.94:0/3338421594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.307+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 0x7fdc40116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 -- 172.21.15.94:0/3338421594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 msgr2=0x7fdc401184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc401184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 -- 172.21.15.94:0/3338421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010b1d0 msgr2=0x7fdc40117530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010b1d0 0x7fdc40117530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 -- 172.21.15.94:0/3338421594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc40118b40 con 0x7fdc4010a350 2024-04-25T13:51:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.309+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 0x7fdc40116fb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdc2802aa00 tx=0x7fdc28003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.310+0000 7fdc37fff640 1 -- 172.21.15.94:0/3338421594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2803f070 con 0x7fdc4010a350 2024-04-25T13:51:06.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.310+0000 7fdc37fff640 1 -- 172.21.15.94:0/3338421594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc28031c50 con 0x7fdc4010a350 2024-04-25T13:51:06.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.310+0000 7fdc37fff640 1 -- 172.21.15.94:0/3338421594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28035030 con 0x7fdc4010a350 2024-04-25T13:51:06.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.312+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc40118dd0 con 0x7fdc4010a350 2024-04-25T13:51:06.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.312+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401192c0 con 0x7fdc4010a350 2024-04-25T13:51:06.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.317+0000 7fdc4c57d740 1 Processor -- start 2024-04-25T13:51:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.320+0000 7fdc4c57d740 1 -- start start 2024-04-25T13:51:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.320+0000 7fdc37fff640 1 -- 172.21.15.94:0/3338421594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdc28042070 con 0x7fdc4010a350 2024-04-25T13:51:06.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 0x56169f70cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f70dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56169f670d80 con 0x56169f36a390 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56169f62aa20 con 0x56169f3414d0 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56169f66cd40 con 0x56169f3496b0 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45906/0 (socket says 172.21.15.94:45906) 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.321+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/2647800870 learned_addr learned my addr 172.21.15.94:0/2647800870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.322+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f70dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.324+0000 7fdc3effd640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 0x56169f70cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/2647800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 msgr2=0x56169f70dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f70dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/2647800870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 msgr2=0x56169f70cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 0x56169f70cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/2647800870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169f635c30 con 0x56169f36a390 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3effd640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 0x56169f70cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc37fff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 0x7fdc2c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009a50 tx=0x7fdc30009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b437970d8a605873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc37fff640 1 -- 172.21.15.94:0/3338421594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdc280ffa30 con 0x7fdc4010a350 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 0x7fdc2c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc357fa640 1 -- 172.21.15.94:0/2647800870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30017070 con 0x56169f36a390 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc357fa640 1 -- 172.21.15.94:0/2647800870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc30013470 con 0x56169f36a390 2024-04-25T13:51:06.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.325+0000 7fdc357fa640 1 -- 172.21.15.94:0/2647800870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30012610 con 0x56169f36a390 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- 172.21.15.94:0/2647800870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 msgr2=0x56169f70ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009a50 tx=0x7fdc30009f10 comp rx=0 tx=0).stop 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc3effd640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 0x7fdc2c0b2230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009850 tx=0x7fdc2802a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- 172.21.15.94:0/2647800870 shutdown_connections 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f70dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3414d0 0x56169f70cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/2647800870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f70ca00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- 172.21.15.94:0/2647800870 >> 172.21.15.94:0/2647800870 conn(0x56169f35e4e0 msgr2=0x56169f708210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- 172.21.15.94:0/2647800870 shutdown_connections 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- 172.21.15.94:0/2647800870 wait complete. 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 -- start start 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.327+0000 7fdc4c57d740 1 Processor -- start 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f71dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56169f70e720 con 0x56169f3496b0 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56169f5f7dc0 con 0x56169f36a390 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56169f70e8a0 con 0x56169f3414d0 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc4c57d740 10 client.0 _ll_register_callbacks cb 0x56169f613600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45914/0 (socket says 172.21.15.94:45914) 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.328+0000 7fdc3e7fc640 1 -- 172.21.15.94:0/1580396895 learned_addr learned my addr 172.21.15.94:0/1580396895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 -- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 msgr2=0x56169f7167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f71dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 -- 172.21.15.94:0/1580396895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 msgr2=0x56169f71dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f71dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 -- 172.21.15.94:0/1580396895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169f3823f0 con 0x56169f3496b0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800f360 tx=0x7fdc3800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b695339f243a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc267fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38015070 con 0x56169f3496b0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc30013470 con 0x56169f3496b0 2024-04-25T13:51:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.329+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdc38003d80 con 0x56169f3496b0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc38004050 con 0x56169f3496b0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 10 client.18864 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56169f44c7d0 con 0x56169f3496b0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc30013470 con 0x56169f3496b0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc3801aa70 con 0x56169f3496b0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 10 client.18864 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 10 client.18864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 20 client.18864 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f328f60 0x56169f724420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 4 client.18864 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.330+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169f67e990 con 0x56169f328f60 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.331+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f328f60 0x56169f724420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.331+0000 7fdc267fc640 10 client.18864 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.331+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f328f60 0x56169f724420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3195414a358e738c server_cookie=ca352b10a86b442c in_seq=0 out_seq=0 2024-04-25T13:51:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.332+0000 7fdc267fc640 1 -- 172.21.15.94:0/1580396895 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56169f67e990 con 0x56169f328f60 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.332+0000 7fdc267fc640 10 client.18864 handle_command_reply: tid=0 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 client.18864 shutdown 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 msgr2=0x56169f716cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800f360 tx=0x7fdc3800f820 comp rx=0 tx=0).stop 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 shutdown_connections 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f328f60 0x56169f724420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f71dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3496b0 0x56169f716cf0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/1580396895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc267fc640 0 client.18864 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 >> 172.21.15.94:0/1580396895 conn(0x56169f35e4e0 msgr2=0x56169f70b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 shutdown_connections 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 1 -- 172.21.15.94:0/1580396895 wait complete. 2024-04-25T13:51:06.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.333+0000 7fdc4c57d740 20 client.18864 trim_cache size 0 max 16384 2024-04-25T13:51:06.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.347+0000 7fdc4c57d740 1 Processor -- start 2024-04-25T13:51:06.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.347+0000 7fdc4c57d740 1 -- start start 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3496b0 0x56169f71db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f7079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b1930 con 0x56169f36a390 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b3470 con 0x56169f3496b0 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b2000 con 0x56169f3414d0 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f7079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7155b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57914/0 (socket says 172.21.15.94:57914) 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3765114025 learned_addr learned my addr 172.21.15.94:0/3765114025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3496b0 0x56169f71db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3765114025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 msgr2=0x56169f7155b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7155b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3765114025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3496b0 msgr2=0x56169f71db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3496b0 0x56169f71db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.348+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3765114025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169f711f30 con 0x56169f36a390 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.349+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f7079f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009970 tx=0x7fdc30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeec572100ead9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.349+0000 7fdc34ff9640 1 -- 172.21.15.94:0/3765114025 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30017040 con 0x56169f36a390 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.349+0000 7fdc34ff9640 1 -- 172.21.15.94:0/3765114025 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc300252f0 con 0x56169f36a390 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.349+0000 7fdc34ff9640 1 -- 172.21.15.94:0/3765114025 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3001c6b0 con 0x56169f36a390 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3765114025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 msgr2=0x56169f7079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f7079f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009970 tx=0x7fdc30008040 comp rx=0 tx=0).stop 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3765114025 shutdown_connections 2024-04-25T13:51:06.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f36a390 0x56169f7079f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f3496b0 0x56169f71db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3765114025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3414d0 0x56169f7155b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3765114025 >> 172.21.15.94:0/3765114025 conn(0x56169f35e4e0 msgr2=0x56169f714ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3765114025 shutdown_connections 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3765114025 wait complete. 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- start start 2024-04-25T13:51:06.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 Processor -- start 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f71b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f78a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b2da0 con 0x56169f3414d0 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401b26d0 con 0x56169f36a390 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.350+0000 7fdc4c57d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401af910 con 0x56169f3496b0 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc4c57d740 10 client.0 _ll_register_callbacks cb 0x7fdc380098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f71b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45936/0 (socket says 172.21.15.94:45936) 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f71b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57920/0 (socket says 172.21.15.94:57920) 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3590434046 learned_addr learned my addr 172.21.15.94:0/3590434046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f78a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 msgr2=0x56169f71b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f71b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3590434046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 msgr2=0x56169f78a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f78a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 -- 172.21.15.94:0/3590434046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169f78acc0 con 0x56169f3414d0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc3f7fe640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fdc30008010 tx=0x7fdc30009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0125d91ab343889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc267fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30017040 con 0x56169f3414d0 2024-04-25T13:51:06.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdc30025770 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc300252f0 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc30004030 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc4c57d740 10 client.18870 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.351+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56169f78bea0 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc300252f0 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc300036b0 con 0x56169f3414d0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 10 client.18870 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 10 client.18870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 20 client.18870 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f2fe3a0 0x56169f78e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 4 client.18870 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56169f678800 con 0x56169f2fe3a0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f2fe3a0 0x56169f78e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc3e7fc640 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f2fe3a0 0x56169f78e580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51c392d4b6ac8964 server_cookie=952c8097e4ad587f in_seq=0 out_seq=0 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.352+0000 7fdc267fc640 10 client.18870 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc267fc640 1 -- 172.21.15.94:0/3590434046 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56169f678800 con 0x56169f2fe3a0 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc267fc640 10 client.18870 handle_command_reply: tid=0 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 client.18870 shutdown 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 msgr2=0x56169f71ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fdc30008010 tx=0x7fdc30009f10 comp rx=0 tx=0).stop 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 shutdown_connections 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56169f2fe3a0 0x56169f78e580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc267fc640 0 client.18870 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56169f36a390 0x56169f78a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56169f3496b0 0x56169f71b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 --2- 172.21.15.94:0/3590434046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56169f3414d0 0x56169f71ac30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 >> 172.21.15.94:0/3590434046 conn(0x56169f35e4e0 msgr2=0x56169f70b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 shutdown_connections 2024-04-25T13:51:06.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 1 -- 172.21.15.94:0/3590434046 wait complete. 2024-04-25T13:51:06.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.353+0000 7fdc4c57d740 20 client.18870 trim_cache size 0 max 16384 2024-04-25T13:51:06.360 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 msgr2=0x7fdc2c0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 0x7fdc2c0b2230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009850 tx=0x7fdc2802a410 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 msgr2=0x7fdc40116fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 0x7fdc40116fb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdc2802aa00 tx=0x7fdc28003bb0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 shutdown_connections 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc2c0afd20 0x7fdc2c0b2230 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc4010fef0 0x7fdc401184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc4010b1d0 0x7fdc40117530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 --2- 172.21.15.94:0/3338421594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc4010a350 0x7fdc40116fb0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 >> 172.21.15.94:0/3338421594 conn(0x7fdc40070400 msgr2=0x7fdc4010bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 shutdown_connections 2024-04-25T13:51:06.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.354+0000 7fdc3ffff640 1 -- 172.21.15.94:0/3338421594 wait complete. 2024-04-25T13:51:06.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:51:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.365+0000 7f60fd7fa640 1 -- 172.21.15.94:0/3147933025 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55836415d5f0 con 0x558363e681c0 2024-04-25T13:51:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.365+0000 7f60fd7fa640 10 client.28316 handle_command_reply: tid=0 2024-04-25T13:51:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.365+0000 7f6122c00740 1 client.28316 shutdown 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 msgr2=0x558364282f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x558364282f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61040097a0 tx=0x7f6104000bc0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 shutdown_connections 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558363e681c0 0x5583642f6c20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558363ed41b0 0x558364284320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558363eb34d0 0x558364283440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f60fd7fa640 0 client.28316 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 --2- 172.21.15.94:0/3147933025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558363eab2f0 0x558364282f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 >> 172.21.15.94:0/3147933025 conn(0x558363ec8300 msgr2=0x55836426e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 shutdown_connections 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 1 -- 172.21.15.94:0/3147933025 wait complete. 2024-04-25T13:51:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.366+0000 7f6122c00740 20 client.28316 trim_cache size 0 max 16384 2024-04-25T13:51:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:51:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:51:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:51:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 msgr2=0x7f61000b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 0x7f61000b1ff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f610c039e40 tx=0x7f610c0a5040 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 msgr2=0x7f61141a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f610c004af0 tx=0x7f610c031700 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 shutdown_connections 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61000afae0 0x7f61000b1ff0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61141097e0 0x7f61141a16b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6114106cb0 0x7f611419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 --2- 172.21.15.94:0/781126520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61141042c0 0x7f611419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 >> 172.21.15.94:0/781126520 conn(0x7f61140fdc60 msgr2=0x7f6114104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 shutdown_connections 2024-04-25T13:51:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:06.367+0000 7f611bd4d640 1 -- 172.21.15.94:0/781126520 wait complete. 2024-04-25T13:51:06.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:06 smithi094 ceph-mon[29195]: pgmap v626: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-25T13:51:06.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:06 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2595464827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:06.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 -- 172.21.15.94:0/1201754108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 msgr2=0x7f9460102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 --2- 172.21.15.94:0/1201754108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f9460102240 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f945c009900 tx=0x7f945c02a410 comp rx=0 tx=0).stop 2024-04-25T13:51:06.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 -- 172.21.15.94:0/1201754108 shutdown_connections 2024-04-25T13:51:06.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 --2- 172.21.15.94:0/1201754108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 0x7f946010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 --2- 172.21.15.94:0/1201754108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f94601090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 --2- 172.21.15.94:0/1201754108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f9460102240 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 -- 172.21.15.94:0/1201754108 >> 172.21.15.94:0/1201754108 conn(0x7f94600fd340 msgr2=0x7f94600ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 -- 172.21.15.94:0/1201754108 shutdown_connections 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.482+0000 7f9467c4c640 1 -- 172.21.15.94:0/1201754108 wait complete. 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 Processor -- start 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 -- start start 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f946019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 0x7f94601a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f946010f5e0 con 0x7f9460101e30 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f946010f460 con 0x7f9460106c10 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f9467c4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f946010f760 con 0x7f9460109780 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45952/0 (socket says 172.21.15.94:45952) 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 -- 172.21.15.94:0/660235354 learned_addr learned my addr 172.21.15.94:0/660235354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94651e7640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f946019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94661e9640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 0x7f94601a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 -- 172.21.15.94:0/660235354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 msgr2=0x7f94601a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 0x7f94601a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 -- 172.21.15.94:0/660235354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 msgr2=0x7f946019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f946019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94659e8640 1 -- 172.21.15.94:0/660235354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a1d70 con 0x7f9460101e30 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.483+0000 7f94651e7640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f946019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f94659e8640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f945c006010 tx=0x7f945c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f9456ffd640 1 -- 172.21.15.94:0/660235354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c03f030 con 0x7f9460101e30 2024-04-25T13:51:06.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f9456ffd640 1 -- 172.21.15.94:0/660235354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c003aa0 con 0x7f9460101e30 2024-04-25T13:51:06.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a2000 con 0x7f9460101e30 2024-04-25T13:51:06.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f9456ffd640 1 -- 172.21.15.94:0/660235354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c035070 con 0x7f9460101e30 2024-04-25T13:51:06.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.484+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a2540 con 0x7f9460101e30 2024-04-25T13:51:06.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.486+0000 7f946ebda740 1 Processor -- start 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f9456ffd640 1 -- 172.21.15.94:0/660235354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f945c042030 con 0x7f9460101e30 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 -- start start 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c30a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558469b93d80 con 0x5584698644d0 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558469b4da20 con 0x55846988d390 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f946ebda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558469b8fd40 con 0x55846986c6b0 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94661e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45958/0 (socket says 172.21.15.94:45958) 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94661e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43742/0 (socket says 172.21.15.94:43742) 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 -- 172.21.15.94:0/3308122835 learned_addr learned my addr 172.21.15.94:0/3308122835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94651e7640 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c30a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 -- 172.21.15.94:0/3308122835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 msgr2=0x558469c30a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c30a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 -- 172.21.15.94:0/3308122835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 msgr2=0x558469c2f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94659e8640 1 -- 172.21.15.94:0/3308122835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558469b58c30 con 0x5584698644d0 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f9456ffd640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 0x7f944c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.487+0000 7f94661e9640 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f94661e9640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 0x7f944c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f9456ffd640 1 -- 172.21.15.94:0/660235354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f945c0ff220 con 0x7f9460101e30 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f94659e8640 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f945c107480 tx=0x7f945c102f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653ba5c0c9111b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f944bfff640 1 -- 172.21.15.94:0/3308122835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c049070 con 0x5584698644d0 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f944bfff640 1 -- 172.21.15.94:0/3308122835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c0cf860 con 0x5584698644d0 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.488+0000 7f944bfff640 1 -- 172.21.15.94:0/3308122835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0c74e0 con 0x5584698644d0 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- 172.21.15.94:0/3308122835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 msgr2=0x558469c2fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f94661e9640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 0x7f944c0b2080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x558469c30860 tx=0x7f9458009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f945c107480 tx=0x7f945c102f90 comp rx=0 tx=0).stop 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- 172.21.15.94:0/3308122835 shutdown_connections 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c30a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584698644d0 0x558469c2fd90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 --2- 172.21.15.94:0/3308122835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846988d390 0x558469c2f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- 172.21.15.94:0/3308122835 >> 172.21.15.94:0/3308122835 conn(0x5584698814e0 msgr2=0x558469c2b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- 172.21.15.94:0/3308122835 shutdown_connections 2024-04-25T13:51:06.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- 172.21.15.94:0/3308122835 wait complete. 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.489+0000 7f946ebda740 1 -- start start 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 Processor -- start 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c39230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469c406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558469c312f0 con 0x55846986c6b0 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558469b1adc0 con 0x5584698644d0 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558469c31470 con 0x55846988d390 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f946ebda740 10 client.0 _ll_register_callbacks cb 0x558469b36600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45970/0 (socket says 172.21.15.94:45970) 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 -- 172.21.15.94:0/48205073 learned_addr learned my addr 172.21.15.94:0/48205073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94659e8640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c39230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94661e9640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469c406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 -- 172.21.15.94:0/48205073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 msgr2=0x558469c406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469c406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 -- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 msgr2=0x558469c39230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c39230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.490+0000 7f94651e7640 1 -- 172.21.15.94:0/48205073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584698a53f0 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94651e7640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9450012d00 tx=0x7f945000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8189b906ef708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450009d90 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9450012110 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558469ba1990 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9450004050 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 10 client.18882 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55846996f7d0 con 0x55846986c6b0 2024-04-25T13:51:06.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558469ba1990 con 0x55846986c6b0 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9450010430 con 0x55846986c6b0 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 10 client.18882 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 10 client.18882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 20 client.18882 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846984bf60 0x558469c46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 4 client.18882 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.491+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558469b9b800 con 0x55846984bf60 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.492+0000 7f94659e8640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846984bf60 0x558469c46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.492+0000 7f94497fa640 10 client.18882 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.492+0000 7f94659e8640 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846984bf60 0x558469c46f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e7e066cedb8557f server_cookie=3b761a639fa58d33 in_seq=0 out_seq=0 2024-04-25T13:51:06.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f94497fa640 1 -- 172.21.15.94:0/48205073 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558469b9b800 con 0x55846984bf60 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f94497fa640 10 client.18882 handle_command_reply: tid=0 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 client.18882 shutdown 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 msgr2=0x558469c39770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9450012d00 tx=0x7f945000f790 comp rx=0 tx=0).stop 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 shutdown_connections 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846984bf60 0x558469c46f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469c406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c39770 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 --2- 172.21.15.94:0/48205073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c39230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f94497fa640 0 client.18882 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 >> 172.21.15.94:0/48205073 conn(0x5584698814e0 msgr2=0x558469c2df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 shutdown_connections 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 1 -- 172.21.15.94:0/48205073 wait complete. 2024-04-25T13:51:06.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.493+0000 7f946ebda740 20 client.18882 trim_cache size 0 max 16384 2024-04-25T13:51:06.501 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:06 smithi138 ceph-mon[32977]: pgmap v626: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-25T13:51:06.501 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:06 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2595464827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 Processor -- start 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 -- start start 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c37850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c37d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ab590 con 0x55846988d390 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ad0d0 con 0x5584698644d0 2024-04-25T13:51:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f946ebda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94601abc60 con 0x55846986c6b0 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.507+0000 7f94661e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45980/0 (socket says 172.21.15.94:45980) 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 -- 172.21.15.94:0/2671017786 learned_addr learned my addr 172.21.15.94:0/2671017786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94651e7640 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c37d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94659e8640 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c37850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 -- 172.21.15.94:0/2671017786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 msgr2=0x558469c37d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c37d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 -- 172.21.15.94:0/2671017786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 msgr2=0x558469c37850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c37850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 -- 172.21.15.94:0/2671017786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558469c34820 con 0x55846988d390 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f94661e9640 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9458011810 tx=0x7f9458016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba4b08f43f9f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f944b7fe640 1 -- 172.21.15.94:0/2671017786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004d20 con 0x55846988d390 2024-04-25T13:51:06.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f944b7fe640 1 -- 172.21.15.94:0/2671017786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458003e20 con 0x55846988d390 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.508+0000 7f944b7fe640 1 -- 172.21.15.94:0/2671017786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458005020 con 0x55846988d390 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- 172.21.15.94:0/2671017786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 msgr2=0x558469c4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9458011810 tx=0x7f9458016000 comp rx=0 tx=0).stop 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- 172.21.15.94:0/2671017786 shutdown_connections 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846988d390 0x558469c4b700 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846986c6b0 0x558469c37d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 --2- 172.21.15.94:0/2671017786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c37850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- 172.21.15.94:0/2671017786 >> 172.21.15.94:0/2671017786 conn(0x5584698814e0 msgr2=0x558469c34110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- 172.21.15.94:0/2671017786 shutdown_connections 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- 172.21.15.94:0/2671017786 wait complete. 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 -- start start 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.509+0000 7f946ebda740 1 Processor -- start 2024-04-25T13:51:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c3f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469cad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aca00 con 0x55846986c6b0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ac330 con 0x5584698644d0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94601a9570 con 0x55846988d390 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f946ebda740 10 client.0 _ll_register_callbacks cb 0x7f9450009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94659e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c3f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45988/0 (socket says 172.21.15.94:45988) 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 -- 172.21.15.94:0/3103359723 learned_addr learned my addr 172.21.15.94:0/3103359723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94661e9640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469cad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 -- 172.21.15.94:0/3103359723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 msgr2=0x558469cad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469cad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 -- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 msgr2=0x558469c3f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c3f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 -- 172.21.15.94:0/3103359723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558469cadd40 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94659e8640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c3f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94661e9640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469cad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94651e7640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9450009730 tx=0x7f9450012110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90adb4d0b5f9cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.510+0000 7f94497fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945002b040 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f9450010430 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9458003e20 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9450010a20 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 10 client.18888 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558469caf060 con 0x55846986c6b0 2024-04-25T13:51:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458003e20 con 0x55846986c6b0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9450037070 con 0x55846986c6b0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 10 client.18888 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 10 client.18888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 20 client.18888 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584698213a0 0x558469cb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 4 client.18888 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.511+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558469b16610 con 0x5584698213a0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94659e8640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584698213a0 0x558469cb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94659e8640 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584698213a0 0x558469cb1a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bed94d725d9b848 server_cookie=abf9228932b2d431 in_seq=0 out_seq=0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94497fa640 10 client.18888 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94497fa640 1 -- 172.21.15.94:0/3103359723 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558469b16610 con 0x5584698213a0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94497fa640 10 client.18888 handle_command_reply: tid=0 2024-04-25T13:51:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 client.18888 shutdown 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 msgr2=0x558469c3f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9450009730 tx=0x7f9450012110 comp rx=0 tx=0).stop 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 shutdown_connections 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584698213a0 0x558469cb1a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846988d390 0x558469cad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846986c6b0 0x558469c3f800 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 --2- 172.21.15.94:0/3103359723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584698644d0 0x558469c3f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f94497fa640 0 client.18888 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 >> 172.21.15.94:0/3103359723 conn(0x5584698814e0 msgr2=0x558469c34110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 shutdown_connections 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.512+0000 7f946ebda740 1 -- 172.21.15.94:0/3103359723 wait complete. 2024-04-25T13:51:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f946ebda740 20 client.18888 trim_cache size 0 max 16384 2024-04-25T13:51:06.520 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 msgr2=0x7f944c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 0x7f944c0b2080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x558469c30860 tx=0x7f9458009600 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 msgr2=0x7f946019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f945c006010 tx=0x7f945c004270 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 shutdown_connections 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f944c0afb70 0x7f944c0b2080 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460109780 0x7f94601a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460106c10 0x7f946019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 --2- 172.21.15.94:0/660235354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460101e30 0x7f946019a230 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 >> 172.21.15.94:0/660235354 conn(0x7f94600fd340 msgr2=0x7f9460108d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 shutdown_connections 2024-04-25T13:51:06.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.513+0000 7f9467c4c640 1 -- 172.21.15.94:0/660235354 wait complete. 2024-04-25T13:51:06.525 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:51:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:06 smithi137 ceph-mon[32568]: pgmap v626: 97 pgs: 97 active+clean; 49 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.5 MiB/s wr, 415 op/s 2024-04-25T13:51:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:06 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2595464827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.682+0000 7fdcce440640 1 -- 172.21.15.94:0/2161976138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8109a20 msgr2=0x7fdcc810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.682+0000 7fdcce440640 1 --2- 172.21.15.94:0/2161976138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8109a20 0x7fdcc810be40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009ca0 tx=0x7fdcc002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 -- 172.21.15.94:0/2161976138 shutdown_connections 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 --2- 172.21.15.94:0/2161976138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8109a20 0x7fdcc810be40 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 --2- 172.21.15.94:0/2161976138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8104d20 0x7fdcc8105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 --2- 172.21.15.94:0/2161976138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8103ae0 0x7fdcc81047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 -- 172.21.15.94:0/2161976138 >> 172.21.15.94:0/2161976138 conn(0x7fdcc80fecd0 msgr2=0x7fdcc8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 -- 172.21.15.94:0/2161976138 shutdown_connections 2024-04-25T13:51:06.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 -- 172.21.15.94:0/2161976138 wait complete. 2024-04-25T13:51:06.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 Processor -- start 2024-04-25T13:51:06.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.683+0000 7fdcce440640 1 -- start start 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8103ae0 0x7fdcc819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8109a20 0x7fdcc81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810f5f0 con 0x7fdcc8104d20 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810f470 con 0x7fdcc8103ae0 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc810f770 con 0x7fdcc8109a20 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46002/0 (socket says 172.21.15.94:46002) 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 -- 172.21.15.94:0/2591241243 learned_addr learned my addr 172.21.15.94:0/2591241243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8103ae0 0x7fdcc819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8109a20 0x7fdcc81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 -- 172.21.15.94:0/2591241243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8109a20 msgr2=0x7fdcc81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8109a20 0x7fdcc81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 -- 172.21.15.94:0/2591241243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8103ae0 msgr2=0x7fdcc819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8103ae0 0x7fdcc819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 -- 172.21.15.94:0/2591241243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc81a1d00 con 0x7fdcc8104d20 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00e9f0 tx=0x7fdcbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc57fa640 1 -- 172.21.15.94:0/2591241243 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00cd80 con 0x7fdcc8104d20 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc57fa640 1 -- 172.21.15.94:0/2591241243 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcbc0108e0 con 0x7fdcc8104d20 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81a1ff0 con 0x7fdcc8104d20 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.684+0000 7fdcc57fa640 1 -- 172.21.15.94:0/2591241243 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc018980 con 0x7fdcc8104d20 2024-04-25T13:51:06.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.685+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a2460 con 0x7fdcc8104d20 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 Processor -- start 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 -- start start 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcc57fa640 1 -- 172.21.15.94:0/2591241243 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdcbc010430 con 0x7fdcc8104d20 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 0x55aa90bff810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa90b63d80 con 0x55aa908344d0 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa90b1da20 con 0x55aa9083c6b0 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.687+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa90b5fd40 con 0x55aa9085d390 2024-04-25T13:51:06.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdccc9dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 0x55aa90bff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46012/0 (socket says 172.21.15.94:46012) 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 -- 172.21.15.94:0/1798162728 learned_addr learned my addr 172.21.15.94:0/1798162728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 -- 172.21.15.94:0/1798162728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 msgr2=0x55aa90bff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 0x55aa90bff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 -- 172.21.15.94:0/1798162728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 msgr2=0x55aa90c00a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 -- 172.21.15.94:0/1798162728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa90b28c30 con 0x55aa908344d0 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 0x55aa90bff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc57fa640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 0x7fdcb00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c60 tx=0x7fdcb8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f193400c2e59ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 0x7fdcb00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcc57fa640 1 -- 172.21.15.94:0/2591241243 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdcbc0defd0 con 0x7fdcc8104d20 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcae7fc640 1 -- 172.21.15.94:0/1798162728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8004220 con 0x55aa908344d0 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcae7fc640 1 -- 172.21.15.94:0/1798162728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb80043c0 con 0x55aa908344d0 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.688+0000 7fdcae7fc640 1 -- 172.21.15.94:0/1798162728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8004d50 con 0x55aa908344d0 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 0x7fdcb00b2000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55aa90c00860 tx=0x7fdcc002f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1798162728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 msgr2=0x55aa90bffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c60 tx=0x7fdcb8011810 comp rx=0 tx=0).stop 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1798162728 shutdown_connections 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c00a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90bffd90 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1798162728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9085d390 0x55aa90bff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1798162728 >> 172.21.15.94:0/1798162728 conn(0x55aa908514e0 msgr2=0x55aa90bfb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.689+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1798162728 shutdown_connections 2024-04-25T13:51:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1798162728 wait complete. 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 -- start start 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 Processor -- start 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9083c6b0 0x55aa90c09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9085d390 0x55aa90c10ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa90c01530 con 0x55aa908344d0 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aa90aeadc0 con 0x55aa9085d390 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa90c016b0 con 0x55aa9083c6b0 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcd53cd740 10 client.0 _ll_register_callbacks cb 0x55aa90b06600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.690+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46020/0 (socket says 172.21.15.94:46020) 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 -- 172.21.15.94:0/247948611 learned_addr learned my addr 172.21.15.94:0/247948611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9085d390 0x55aa90c10ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9083c6b0 0x55aa90c09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 -- 172.21.15.94:0/247948611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9083c6b0 msgr2=0x55aa90c09b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9083c6b0 0x55aa90c09b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 -- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9085d390 msgr2=0x55aa90c10ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9085d390 0x55aa90c10ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 -- 172.21.15.94:0/247948611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa908753f0 con 0x55aa908344d0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c00 tx=0x7fdcb8019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa953316c87e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8027030 con 0x55aa908344d0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb80043c0 con 0x55aa908344d0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdcb8019d30 con 0x55aa908344d0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcb801d040 con 0x55aa908344d0 2024-04-25T13:51:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcd53cd740 10 client.18900 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.691+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa9093f7d0 con 0x55aa908344d0 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb80043c0 con 0x55aa908344d0 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcb8035c20 con 0x55aa908344d0 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 10 client.18900 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 10 client.18900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 20 client.18900 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9081bf60 0x55aa90c16f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 4 client.18900 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa90b71990 con 0x55aa9081bf60 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9081bf60 0x55aa90c16f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.692+0000 7fdca77fe640 10 client.18900 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.693+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9081bf60 0x55aa90c16f70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a4fe6965e1ddd3 server_cookie=666fae74c28fbc7e in_seq=0 out_seq=0 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.693+0000 7fdca77fe640 1 -- 172.21.15.94:0/247948611 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa90b71990 con 0x55aa9081bf60 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdca77fe640 10 client.18900 handle_command_reply: tid=0 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 client.18900 shutdown 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 msgr2=0x55aa90c09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c00 tx=0x7fdcb8019b40 comp rx=0 tx=0).stop 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 shutdown_connections 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9081bf60 0x55aa90c16f70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9085d390 0x55aa90c10ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdca77fe640 0 client.18900 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9083c6b0 0x55aa90c09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/247948611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c09620 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 >> 172.21.15.94:0/247948611 conn(0x55aa908514e0 msgr2=0x55aa90bfdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 shutdown_connections 2024-04-25T13:51:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 1 -- 172.21.15.94:0/247948611 wait complete. 2024-04-25T13:51:06.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.694+0000 7fdcd53cd740 20 client.18900 trim_cache size 0 max 16384 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 Processor -- start 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 -- start start 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9081bf60 0x55aa90c09900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ab5b0 con 0x55aa908344d0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ad0f0 con 0x55aa9083c6b0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81abc80 con 0x55aa9081bf60 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46024/0 (socket says 172.21.15.94:46024) 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 -- 172.21.15.94:0/1125263082 learned_addr learned my addr 172.21.15.94:0/1125263082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9081bf60 0x55aa90c09900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 -- 172.21.15.94:0/1125263082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9081bf60 msgr2=0x55aa90c09900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9081bf60 0x55aa90c09900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 -- 172.21.15.94:0/1125263082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 msgr2=0x55aa90c1c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.709+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c1c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.710+0000 7fdcc77fe640 1 -- 172.21.15.94:0/1125263082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa90c04da0 con 0x55aa908344d0 2024-04-25T13:51:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.710+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0dca30 tx=0x7fdcbc090040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748e3978f88747c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.710+0000 7fdcadffb640 1 -- 172.21.15.94:0/1125263082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0ab7c0 con 0x55aa908344d0 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.710+0000 7fdcadffb640 1 -- 172.21.15.94:0/1125263082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcbc0b0270 con 0x55aa908344d0 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.710+0000 7fdcadffb640 1 -- 172.21.15.94:0/1125263082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc018cb0 con 0x55aa908344d0 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1125263082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 msgr2=0x55aa90c109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0dca30 tx=0x7fdcbc090040 comp rx=0 tx=0).stop 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1125263082 shutdown_connections 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9083c6b0 0x55aa90c1c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa908344d0 0x55aa90c109e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/1125263082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa9081bf60 0x55aa90c09900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1125263082 >> 172.21.15.94:0/1125263082 conn(0x55aa908514e0 msgr2=0x55aa90c091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1125263082 shutdown_connections 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- 172.21.15.94:0/1125263082 wait complete. 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 -- start start 2024-04-25T13:51:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.711+0000 7fdcd53cd740 1 Processor -- start 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa908344d0 0x55aa90c0fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81aca20 con 0x55aa9083c6b0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81ac350 con 0x55aa9081bf60 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc81a9590 con 0x55aa908344d0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 10 client.0 _ll_register_callbacks cb 0x7fdcb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcc7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43828/0 (socket says 172.21.15.94:43828) 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcc7fff640 1 -- 172.21.15.94:0/108043304 learned_addr learned my addr 172.21.15.94:0/108043304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46030/0 (socket says 172.21.15.94:46030) 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcc77fe640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa908344d0 0x55aa90c0fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 -- 172.21.15.94:0/108043304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa908344d0 msgr2=0x55aa90c0fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa908344d0 0x55aa90c0fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 -- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 msgr2=0x55aa90c0f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 -- 172.21.15.94:0/108043304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa90c10f20 con 0x55aa9083c6b0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdccc9dd640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031b50 tx=0x7fdcc00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50c395996e56f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0058060 con 0x55aa9083c6b0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc0b0270 con 0x55aa9083c6b0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.712+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fdcc00047a0 con 0x55aa9083c6b0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcc0003e50 con 0x55aa9083c6b0 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 10 client.18906 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa90c7e870 con 0x55aa9083c6b0 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcbc0b0270 con 0x55aa9083c6b0 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcc0030860 con 0x55aa9083c6b0 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 10 client.18906 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 10 client.18906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 20 client.18906 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9085d390 0x55aa90c81290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 4 client.18906 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa90b6b800 con 0x55aa9085d390 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.713+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9085d390 0x55aa90c81290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.714+0000 7fdca77fe640 10 client.18906 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.714+0000 7fdcc7fff640 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9085d390 0x55aa90c81290 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ae64b0dff21652 server_cookie=425a97b589d305a0 in_seq=0 out_seq=0 2024-04-25T13:51:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.716+0000 7fdca77fe640 1 -- 172.21.15.94:0/108043304 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa90b6b800 con 0x55aa9085d390 2024-04-25T13:51:06.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.716+0000 7fdca77fe640 10 client.18906 handle_command_reply: tid=0 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.716+0000 7fdcd53cd740 1 client.18906 shutdown 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 msgr2=0x55aa90c7d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031b50 tx=0x7fdcc00095c0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 shutdown_connections 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aa9085d390 0x55aa90c81290 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa9083c6b0 0x55aa90c7d310 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa908344d0 0x55aa90c0fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdca77fe640 0 client.18906 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 --2- 172.21.15.94:0/108043304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aa9081bf60 0x55aa90c0f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 >> 172.21.15.94:0/108043304 conn(0x55aa908514e0 msgr2=0x55aa90bfde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 shutdown_connections 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 1 -- 172.21.15.94:0/108043304 wait complete. 2024-04-25T13:51:06.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcd53cd740 20 client.18906 trim_cache size 0 max 16384 2024-04-25T13:51:06.722 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 msgr2=0x7fdcb00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 0x7fdcb00b2000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55aa90c00860 tx=0x7fdcc002f310 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 msgr2=0x7fdcc819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00e9f0 tx=0x7fdcbc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.717+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 shutdown_connections 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcb00afaf0 0x7fdcb00b2000 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc8109a20 0x7fdcc81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc8104d20 0x7fdcc819a700 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 --2- 172.21.15.94:0/2591241243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcc8103ae0 0x7fdcc819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 >> 172.21.15.94:0/2591241243 conn(0x7fdcc80fecd0 msgr2=0x7fdcc8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 shutdown_connections 2024-04-25T13:51:06.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:06.718+0000 7fdcce440640 1 -- 172.21.15.94:0/2591241243 wait complete. 2024-04-25T13:51:06.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1133191 12.83 MB/sec execute 349 sec latency 111.859 ms 2024-04-25T13:51:07.729 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:51:07.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:07.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:51:07.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1135495 12.82 MB/sec execute 350 sec latency 19.599 ms 2024-04-25T13:51:07.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 -- 172.21.15.94:0/1344532540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc108920 msgr2=0x7f51fc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 --2- 172.21.15.94:0/1344532540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc108920 0x7f51fc10ad40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009ca0 tx=0x7f51f402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 -- 172.21.15.94:0/1344532540 shutdown_connections 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 --2- 172.21.15.94:0/1344532540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc108920 0x7f51fc10ad40 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 --2- 172.21.15.94:0/1344532540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc103c20 0x7f51fc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 --2- 172.21.15.94:0/1344532540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 0x7f51fc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 -- 172.21.15.94:0/1344532540 >> 172.21.15.94:0/1344532540 conn(0x7f51fc0fdbd0 msgr2=0x7f51fc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 -- 172.21.15.94:0/1344532540 shutdown_connections 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.851+0000 7f5203c51640 1 -- 172.21.15.94:0/1344532540 wait complete. 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 Processor -- start 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 -- start start 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 0x7f51fc06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc108920 0x7f51fc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10e510 con 0x7f51fc103c20 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10e390 con 0x7f51fc108920 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f5203c51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10e690 con 0x7f51fc1029e0 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46062/0 (socket says 172.21.15.94:46062) 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 -- 172.21.15.94:0/3290703399 learned_addr learned my addr 172.21.15.94:0/3290703399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52019ed640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 0x7f51fc06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52021ee640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc108920 0x7f51fc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 -- 172.21.15.94:0/3290703399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 msgr2=0x7f51fc06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 0x7f51fc06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 -- 172.21.15.94:0/3290703399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc108920 msgr2=0x7f51fc1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.852+0000 7f52011ec640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc108920 0x7f51fc1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f52011ec640 1 -- 172.21.15.94:0/3290703399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc1a2030 con 0x7f51fc103c20 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f52011ec640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d950 tx=0x7f51f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f51f2ffd640 1 -- 172.21.15.94:0/3290703399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8017030 con 0x7f51fc103c20 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f51f2ffd640 1 -- 172.21.15.94:0/3290703399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f8004020 con 0x7f51fc103c20 2024-04-25T13:51:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc1a2320 con 0x7f51fc103c20 2024-04-25T13:51:07.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f51f2ffd640 1 -- 172.21.15.94:0/3290703399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80047a0 con 0x7f51fc103c20 2024-04-25T13:51:07.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.853+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a2870 con 0x7f51fc103c20 2024-04-25T13:51:07.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.855+0000 7f520abe1740 1 Processor -- start 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 -- start start 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f51f2ffd640 1 -- 172.21.15.94:0/3290703399 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f51f8004940 con 0x7f51fc103c20 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11940a730 0x55f1197d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f119731b00 con 0x55f11940a730 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1196cd920 con 0x55f11942b410 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f520abe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f11972dac0 con 0x55f119402550 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52011ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11940a730 0x55f1197d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43842/0 (socket says 172.21.15.94:43842) 2024-04-25T13:51:07.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 -- 172.21.15.94:0/1663921464 learned_addr learned my addr 172.21.15.94:0/1663921464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52019ed640 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 -- 172.21.15.94:0/1663921464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 msgr2=0x55f1197cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 -- 172.21.15.94:0/1663921464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11940a730 msgr2=0x55f1197d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11940a730 0x55f1197d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52021ee640 1 -- 172.21.15.94:0/1663921464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11950b7e0 con 0x55f11942b410 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f51f2ffd640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 0x7f51e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.856+0000 7f52019ed640 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f52019ed640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 0x7f51e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f51f2ffd640 1 -- 172.21.15.94:0/3290703399 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f51f80de660 con 0x7f51fc103c20 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f52021ee640 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51f4035000 tx=0x7f51f402f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b964fbcf440ec3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f51e7fff640 1 -- 172.21.15.94:0/1663921464 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4031650 con 0x55f11942b410 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f51e7fff640 1 -- 172.21.15.94:0/1663921464 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f4036030 con 0x55f11942b410 2024-04-25T13:51:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.857+0000 7f51e7fff640 1 -- 172.21.15.94:0/1663921464 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40047a0 con 0x55f11942b410 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- 172.21.15.94:0/1663921464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 msgr2=0x55f1197cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51f4035000 tx=0x7f51f402f4c0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f52019ed640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 0x7f51e80b2000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55f1197cd880 tx=0x7f51ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- 172.21.15.94:0/1663921464 shutdown_connections 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11940a730 0x55f1197d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 --2- 172.21.15.94:0/1663921464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197cc4a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- 172.21.15.94:0/1663921464 >> 172.21.15.94:0/1663921464 conn(0x55f11941f560 msgr2=0x55f1197c7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- 172.21.15.94:0/1663921464 shutdown_connections 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- 172.21.15.94:0/1663921464 wait complete. 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 -- start start 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.858+0000 7f520abe1740 1 Processor -- start 2024-04-25T13:51:07.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 0x55f1197d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f1197dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f1197d4190 con 0x55f11942b410 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1196cb5d0 con 0x55f11940a730 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1197d4310 con 0x55f119402550 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f520abe1740 10 client.0 _ll_register_callbacks cb 0x55f1196b9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52011ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 0x55f1197d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52021ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f1197dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58044/0 (socket says 172.21.15.94:58044) 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52011ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 0x55f1197d6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43858/0 (socket says 172.21.15.94:43858) 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 -- 172.21.15.94:0/3834659704 learned_addr learned my addr 172.21.15.94:0/3834659704 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:07.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 -- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 msgr2=0x55f1197d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 0x55f1197d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 -- 172.21.15.94:0/3834659704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 msgr2=0x55f1197dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f1197dd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.859+0000 7f52019ed640 1 -- 172.21.15.94:0/3834659704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11956bf70 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f52019ed640 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011bb0 tx=0x7f51ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1e478c9d5de12a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec010ce0 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f51ec005020 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f4036030 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51ec005260 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 10 client.28207 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f119403170 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f4036030 con 0x55f119402550 2024-04-25T13:51:07.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f51ec031400 con 0x55f119402550 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 10 client.28207 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 10 client.28207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 20 client.28207 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193ea1f0 0x55f1197e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 4 client.28207 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11973f710 con 0x55f1193ea1f0 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.860+0000 7f52011ec640 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193ea1f0 0x55f1197e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.861+0000 7f51e57fa640 10 client.28207 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.861+0000 7f52011ec640 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193ea1f0 0x55f1197e3ac0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1959460182a15c96 server_cookie=53098d71e42594ba in_seq=0 out_seq=0 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f51e57fa640 1 -- 172.21.15.94:0/3834659704 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f11973f710 con 0x55f1193ea1f0 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f51e57fa640 10 client.28207 handle_command_reply: tid=0 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 client.28207 shutdown 2024-04-25T13:51:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 msgr2=0x55f1197d61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011bb0 tx=0x7f51ec0046c0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 shutdown_connections 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193ea1f0 0x55f1197e3ac0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f1197dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f51e57fa640 0 client.28207 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11940a730 0x55f1197d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 --2- 172.21.15.94:0/3834659704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f119402550 0x55f1197d61f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 >> 172.21.15.94:0/3834659704 conn(0x55f11941f560 msgr2=0x55f1197c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 shutdown_connections 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 1 -- 172.21.15.94:0/3834659704 wait complete. 2024-04-25T13:51:07.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.863+0000 7f520abe1740 20 client.28207 trim_cache size 0 max 16384 2024-04-25T13:51:07.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.882+0000 7f520abe1740 1 Processor -- start 2024-04-25T13:51:07.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.882+0000 7f520abe1740 1 -- start start 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.882+0000 7f520abe1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f520abe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f520abe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f520abe1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ab510 con 0x55f119402550 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f520abe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ad050 con 0x55f11942b410 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f520abe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1abbe0 con 0x55f11940a730 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46094/0 (socket says 172.21.15.94:46094) 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 -- 172.21.15.94:0/176022545 learned_addr learned my addr 172.21.15.94:0/176022545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52021ee640 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52011ec640 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 -- 172.21.15.94:0/176022545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 msgr2=0x55f1197d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 -- 172.21.15.94:0/176022545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 msgr2=0x55f1197dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 -- 172.21.15.94:0/176022545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1197cf8c0 con 0x55f119402550 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f52019ed640 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011800 tx=0x7f51ec009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53303ac1ba36e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f51e77fe640 1 -- 172.21.15.94:0/176022545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec037040 con 0x55f119402550 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f51e77fe640 1 -- 172.21.15.94:0/176022545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec010ae0 con 0x55f119402550 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.883+0000 7f51e77fe640 1 -- 172.21.15.94:0/176022545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec031dc0 con 0x55f119402550 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 -- 172.21.15.94:0/176022545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 msgr2=0x55f1197d5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011800 tx=0x7f51ec009f60 comp rx=0 tx=0).stop 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 -- 172.21.15.94:0/176022545 shutdown_connections 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f11942b410 0x55f1197dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 --2- 172.21.15.94:0/176022545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f119402550 0x55f1197d5260 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 -- 172.21.15.94:0/176022545 >> 172.21.15.94:0/176022545 conn(0x55f11941f560 msgr2=0x55f1197cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 -- 172.21.15.94:0/176022545 shutdown_connections 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.884+0000 7f520abe1740 1 -- 172.21.15.94:0/176022545 wait complete. 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 -- start start 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 Processor -- start 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 0x55f1197dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ac980 con 0x55f11942b410 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ac2b0 con 0x55f119402550 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1a94f0 con 0x55f11940a730 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f520abe1740 10 client.0 _ll_register_callbacks cb 0x7f51ec011830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:07.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f52021ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f52021ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46096/0 (socket says 172.21.15.94:46096) 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f52021ee640 1 -- 172.21.15.94:0/2159177880 learned_addr learned my addr 172.21.15.94:0/2159177880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f52019ed640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 0x55f1197dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.885+0000 7f52011ec640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 -- 172.21.15.94:0/2159177880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 msgr2=0x55f1197dd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197dd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 -- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 msgr2=0x55f1197dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 0x55f1197dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 -- 172.21.15.94:0/2159177880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11984a620 con 0x55f11942b410 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52019ed640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 0x55f1197dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f52021ee640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009800 tx=0x7f51f4031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03712bf7e85d9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f404a040 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f51f402f860 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec010ae0 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51f402fa60 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f520abe1740 10 client.18924 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f11984b930 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec010ae0 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.886+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f51f4043070 con 0x55f11942b410 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 10 client.18924 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 20 client.18924 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193bf630 0x55f11984e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 4 client.18924 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f119739580 con 0x55f1193bf630 2024-04-25T13:51:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f52019ed640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193bf630 0x55f11984e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:07.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f51e57fa640 10 client.18924 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:07.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.887+0000 7f52019ed640 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193bf630 0x55f11984e0c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf305bbfa32b1f1 server_cookie=aacd292fdb643444 in_seq=0 out_seq=0 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f51e57fa640 1 -- 172.21.15.94:0/2159177880 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f119739580 con 0x55f1193bf630 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f51e57fa640 10 client.18924 handle_command_reply: tid=0 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 client.18924 shutdown 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 msgr2=0x55f11984a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009800 tx=0x7f51f4031940 comp rx=0 tx=0).stop 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 shutdown_connections 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1193bf630 0x55f11984e0c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f11942b410 0x55f11984a030 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f11940a730 0x55f1197dd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 --2- 172.21.15.94:0/2159177880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f119402550 0x55f1197dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f51e57fa640 0 client.18924 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 >> 172.21.15.94:0/2159177880 conn(0x55f11941f560 msgr2=0x55f1197cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 shutdown_connections 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 1 -- 172.21.15.94:0/2159177880 wait complete. 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.889+0000 7f520abe1740 20 client.18924 trim_cache size 0 max 16384 2024-04-25T13:51:07.895 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:51:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "8880b6d8-a7ca-4614-bc03-ecc9950b5dfb", 2024-04-25T13:51:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:51:07.896 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 msgr2=0x7f51e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 0x7f51e80b2000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55f1197cd880 tx=0x7f51ec005f90 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 msgr2=0x7f51fc19f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d950 tx=0x7f51f800de10 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 shutdown_connections 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f51e80afaf0 0x7f51e80b2000 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51fc108920 0x7f51fc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51fc103c20 0x7f51fc19f260 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 --2- 172.21.15.94:0/3290703399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51fc1029e0 0x7f51fc06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 >> 172.21.15.94:0/3290703399 conn(0x7f51fc0fdbd0 msgr2=0x7f51fc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 shutdown_connections 2024-04-25T13:51:07.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:07.890+0000 7f5203c51640 1 -- 172.21.15.94:0/3290703399 wait complete. 2024-04-25T13:51:07.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:51:07.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:51:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.015+0000 7f684b6bd640 1 -- 172.21.15.94:0/4002169426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 msgr2=0x7f68441036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.015+0000 7f684b6bd640 1 --2- 172.21.15.94:0/4002169426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f68441036e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6840009900 tx=0x7f684002a410 comp rx=0 tx=0).stop 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 -- 172.21.15.94:0/4002169426 shutdown_connections 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 --2- 172.21.15.94:0/4002169426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 0x7f684410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 --2- 172.21.15.94:0/4002169426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 0x7f6844104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 --2- 172.21.15.94:0/4002169426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f68441036e0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 -- 172.21.15.94:0/4002169426 >> 172.21.15.94:0/4002169426 conn(0x7f68440fdbd0 msgr2=0x7f6844100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 -- 172.21.15.94:0/4002169426 shutdown_connections 2024-04-25T13:51:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 -- 172.21.15.94:0/4002169426 wait complete. 2024-04-25T13:51:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 Processor -- start 2024-04-25T13:51:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 -- start start 2024-04-25T13:51:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.016+0000 7f684b6bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 0x7f68441999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 0x7f68441a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f684410e4e0 con 0x7f68441029e0 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f684410e360 con 0x7f6844108920 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f684410e660 con 0x7f6844103c20 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46112/0 (socket says 172.21.15.94:46112) 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 -- 172.21.15.94:0/34566664 learned_addr learned my addr 172.21.15.94:0/34566664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849c5a640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 0x7f68441a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6848c58640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 0x7f68441999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 -- 172.21.15.94:0/34566664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 msgr2=0x7f68441999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 0x7f68441999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 -- 172.21.15.94:0/34566664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 msgr2=0x7f68441a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 0x7f68441a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 -- 172.21.15.94:0/34566664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68441a0fb0 con 0x7f68441029e0 2024-04-25T13:51:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f6849459640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6840006010 tx=0x7f6840004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f683a7fc640 1 -- 172.21.15.94:0/34566664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684003f030 con 0x7f68441029e0 2024-04-25T13:51:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f683a7fc640 1 -- 172.21.15.94:0/34566664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6840031910 con 0x7f68441029e0 2024-04-25T13:51:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68441a1240 con 0x7f68441029e0 2024-04-25T13:51:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f683a7fc640 1 -- 172.21.15.94:0/34566664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840035070 con 0x7f68441029e0 2024-04-25T13:51:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.017+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68441a1780 con 0x7f68441029e0 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f683a7fc640 1 -- 172.21.15.94:0/34566664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6840042030 con 0x7f68441029e0 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 Processor -- start 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 -- start start 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da28129d80 con 0x55da27dfa4d0 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da280e38b0 con 0x55da27e23390 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6852573740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da28125d40 con 0x55da27e026b0 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6849c5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46128/0 (socket says 172.21.15.94:46128) 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.020+0000 7f6849459640 1 -- 172.21.15.94:0/2957267952 learned_addr learned my addr 172.21.15.94:0/2957267952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6848c58640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f683a7fc640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 0x7f68300b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 -- 172.21.15.94:0/2957267952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 msgr2=0x55da281c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 -- 172.21.15.94:0/2957267952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 msgr2=0x55da281c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 -- 172.21.15.94:0/2957267952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da280b4a40 con 0x55da27e23390 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6848c58640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849459640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 0x7f68300b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f683a7fc640 1 -- 172.21.15.94:0/34566664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f68400fefb0 con 0x7f68441029e0 2024-04-25T13:51:08.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281c4a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f683c009fd0 tx=0x7f683c011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8241e72060548067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f6849c5a640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 0x7f68300b2080 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f683c022000 tx=0x7f683c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f682f7fe640 1 -- 172.21.15.94:0/2957267952 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c017070 con 0x55da27e23390 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.021+0000 7f682f7fe640 1 -- 172.21.15.94:0/2957267952 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c00fcb0 con 0x55da27e23390 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f682f7fe640 1 -- 172.21.15.94:0/2957267952 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c01bb90 con 0x55da27e23390 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 -- 172.21.15.94:0/2957267952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 msgr2=0x55da281c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281c4a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f683c009fd0 tx=0x7f683c011920 comp rx=0 tx=0).stop 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 -- 172.21.15.94:0/2957267952 shutdown_connections 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 --2- 172.21.15.94:0/2957267952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281c4a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.022+0000 7f6852573740 1 -- 172.21.15.94:0/2957267952 >> 172.21.15.94:0/2957267952 conn(0x55da27e174e0 msgr2=0x55da281c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 -- 172.21.15.94:0/2957267952 shutdown_connections 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 -- 172.21.15.94:0/2957267952 wait complete. 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 -- start start 2024-04-25T13:51:08.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 Processor -- start 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.023+0000 7f6852573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da281c6780 con 0x55da27dfa4d0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da280b0d10 con 0x55da27e23390 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da281c6900 con 0x55da27e026b0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 10 client.0 _ll_register_callbacks cb 0x55da280dd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46134/0 (socket says 172.21.15.94:46134) 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 -- 172.21.15.94:0/2955683782 learned_addr learned my addr 172.21.15.94:0/2955683782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6848c58640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849c5a640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 -- 172.21.15.94:0/2955683782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 msgr2=0x55da281ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 -- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 msgr2=0x55da281d5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281d5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 -- 172.21.15.94:0/2955683782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da27f057d0 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6849459640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f68400c7380 tx=0x7f684003b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1842b1a7246cf08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f682cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840049070 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f68400c7650 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f683c00fcb0 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68400c6d40 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 10 client.18936 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.024+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da27e3b3f0 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c00fcb0 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f68400cfdc0 con 0x55da27dfa4d0 2024-04-25T13:51:08.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 10 client.18936 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 20 client.18936 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27de1f60 0x55da281dc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 4 client.18936 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da28137990 con 0x55da27de1f60 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f6848c58640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27de1f60 0x55da281dc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.025+0000 7f682cff9640 10 client.18936 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:08.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.026+0000 7f6848c58640 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27de1f60 0x55da281dc280 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d85f72fd64c089 server_cookie=54d1d3ef641c2f55 in_seq=0 out_seq=0 2024-04-25T13:51:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f682cff9640 1 -- 172.21.15.94:0/2955683782 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da28137990 con 0x55da27de1f60 2024-04-25T13:51:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f682cff9640 10 client.18936 handle_command_reply: tid=0 2024-04-25T13:51:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 client.18936 shutdown 2024-04-25T13:51:08.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 msgr2=0x55da281ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f68400c7380 tx=0x7f684003b4f0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 shutdown_connections 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27de1f60 0x55da281dc280 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281d5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 --2- 172.21.15.94:0/2955683782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce9a0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 >> 172.21.15.94:0/2955683782 conn(0x55da27e174e0 msgr2=0x55da281bf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f682cff9640 0 client.18936 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 shutdown_connections 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 1 -- 172.21.15.94:0/2955683782 wait complete. 2024-04-25T13:51:08.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.027+0000 7f6852573740 20 client.18936 trim_cache size 0 max 16384 2024-04-25T13:51:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 Processor -- start 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 -- start start 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68441a9320 con 0x55da27dfa4d0 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aae60 con 0x55da27e23390 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6852573740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68441a99f0 con 0x55da27e026b0 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46144/0 (socket says 172.21.15.94:46144) 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 -- 172.21.15.94:0/2395533255 learned_addr learned my addr 172.21.15.94:0/2395533255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6848c58640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849c5a640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 -- 172.21.15.94:0/2395533255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 msgr2=0x55da281ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 -- 172.21.15.94:0/2395533255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 msgr2=0x55da281e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 -- 172.21.15.94:0/2395533255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da27dfb0f0 con 0x55da27dfa4d0 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6848c58640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.046+0000 7f6849459640 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f68400c70f0 tx=0x7f6840009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416bf9fdaa24457b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.047+0000 7f682effd640 1 -- 172.21.15.94:0/2395533255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840049050 con 0x55da27dfa4d0 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.047+0000 7f682effd640 1 -- 172.21.15.94:0/2395533255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68400cffc0 con 0x55da27dfa4d0 2024-04-25T13:51:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.047+0000 7f682effd640 1 -- 172.21.15.94:0/2395533255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68400c6b60 con 0x55da27dfa4d0 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.047+0000 7f6852573740 1 -- 172.21.15.94:0/2395533255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 msgr2=0x55da281ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.047+0000 7f6852573740 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f68400c70f0 tx=0x7f6840009260 comp rx=0 tx=0).stop 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- 172.21.15.94:0/2395533255 shutdown_connections 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27e23390 0x55da281e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- 172.21.15.94:0/2395533255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27dfa4d0 0x55da281ce570 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- 172.21.15.94:0/2395533255 >> 172.21.15.94:0/2395533255 conn(0x55da27e174e0 msgr2=0x55da281ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- 172.21.15.94:0/2395533255 shutdown_connections 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- 172.21.15.94:0/2395533255 wait complete. 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- start start 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 Processor -- start 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 0x55da281d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aa790 con 0x55da27e23390 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68441aa0c0 con 0x55da27dfa4d0 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68441a7300 con 0x55da27e026b0 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.048+0000 7f6852573740 10 client.0 _ll_register_callbacks cb 0x7f6840005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46148/0 (socket says 172.21.15.94:46148) 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6848c58640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 -- 172.21.15.94:0/3043281692 learned_addr learned my addr 172.21.15.94:0/3043281692 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849459640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 0x55da281d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 -- 172.21.15.94:0/3043281692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 msgr2=0x55da281d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 -- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 msgr2=0x55da281d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 0x55da281d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 -- 172.21.15.94:0/3043281692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da280ea4b0 con 0x55da27e23390 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6848c58640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849459640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 0x55da281d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6849c5a640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55da281c5ab0 tx=0x7f683c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e8793323397c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f682cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c038060 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7f683c00fba0 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68400cffc0 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f683c017040 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6852573740 10 client.18942 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.049+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da28243950 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68400cffc0 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f683c026440 con 0x55da27e23390 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 10 client.18942 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 20 client.18942 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27db73a0 0x55da282461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 4 client.18942 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:51:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da28131800 con 0x55da27db73a0 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6849459640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27db73a0 0x55da282461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f682cff9640 10 client.18942 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.050+0000 7f6849459640 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27db73a0 0x55da282461d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8f022cc5741657 server_cookie=f1fc7f2aa8134c23 in_seq=0 out_seq=0 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f682cff9640 1 -- 172.21.15.94:0/3043281692 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da28131800 con 0x55da27db73a0 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f682cff9640 10 client.18942 handle_command_reply: tid=0 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 client.18942 shutdown 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 msgr2=0x55da28242620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55da281c5ab0 tx=0x7f683c00fa30 comp rx=0 tx=0).stop 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 shutdown_connections 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da27db73a0 0x55da282461d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da27e23390 0x55da28242620 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da27e026b0 0x55da281d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f682cff9640 0 client.18942 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 --2- 172.21.15.94:0/3043281692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da27dfa4d0 0x55da281d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 >> 172.21.15.94:0/3043281692 conn(0x55da27e174e0 msgr2=0x55da281ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.051+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 shutdown_connections 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f6852573740 1 -- 172.21.15.94:0/3043281692 wait complete. 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f6852573740 20 client.18942 trim_cache size 0 max 16384 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stdout: "8880b6d8-a7ca-4614-bc03-ecc9950b5dfb": { 2024-04-25T13:51:08.058 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:51:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "8880b6d8-a7ca-4614-bc03-ecc9950b5dfb", 2024-04-25T13:51:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:51:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:51:08.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:51:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 msgr2=0x7f68300b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 0x7f68300b2080 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f683c022000 tx=0x7f683c023000 comp rx=0 tx=0).stop 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 msgr2=0x7f6844199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6840006010 tx=0x7f6840004270 comp rx=0 tx=0).stop 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 shutdown_connections 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68300afb70 0x7f68300b2080 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6844108920 0x7f68441a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6844103c20 0x7f68441999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 --2- 172.21.15.94:0/34566664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68441029e0 0x7f6844199470 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 >> 172.21.15.94:0/34566664 conn(0x7f68440fdbd0 msgr2=0x7f6844104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 shutdown_connections 2024-04-25T13:51:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:08.052+0000 7f684b6bd640 1 -- 172.21.15.94:0/34566664 wait complete. 2024-04-25T13:51:08.063 INFO:tasks.cephfs.filesystem:scrub status for tag:8880b6d8-a7ca-4614-bc03-ecc9950b5dfb - {'path': '/', 'tag': '8880b6d8-a7ca-4614-bc03-ecc9950b5dfb', 'options': 'recursive,force'} 2024-04-25T13:51:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:08 smithi094 ceph-mon[29195]: pgmap v627: 97 pgs: 97 active+clean; 52 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-04-25T13:51:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:08 smithi138 ceph-mon[32977]: pgmap v627: 97 pgs: 97 active+clean; 52 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-04-25T13:51:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:08 smithi137 ceph-mon[32568]: pgmap v627: 97 pgs: 97 active+clean; 52 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-04-25T13:51:08.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1137476 12.82 MB/sec execute 351 sec latency 17.810 ms 2024-04-25T13:51:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:09 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:51:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:51:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:09 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:51:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:51:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:09 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:51:09.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:51:09.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:09.807 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1139991 12.82 MB/sec execute 352 sec latency 97.348 ms 2024-04-25T13:51:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:10 smithi094 ceph-mon[29195]: pgmap v628: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:51:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:10 smithi138 ceph-mon[32977]: pgmap v628: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:51:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:10 smithi137 ceph-mon[32568]: pgmap v628: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:51:10.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1142009 12.81 MB/sec execute 353 sec latency 14.392 ms 2024-04-25T13:51:11.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:51:11.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:51:11.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:51:11.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1144410 12.82 MB/sec execute 354 sec latency 12.824 ms 2024-04-25T13:51:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:12 smithi094 ceph-mon[29195]: pgmap v629: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:51:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:12 smithi138 ceph-mon[32977]: pgmap v629: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:51:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:12 smithi137 ceph-mon[32568]: pgmap v629: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:51:12.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1147351 12.83 MB/sec execute 355 sec latency 85.408 ms 2024-04-25T13:51:13.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1148745 12.81 MB/sec execute 356 sec latency 94.334 ms 2024-04-25T13:51:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:14 smithi094 ceph-mon[29195]: pgmap v630: 97 pgs: 97 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-04-25T13:51:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:14 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:14 smithi138 ceph-mon[32977]: pgmap v630: 97 pgs: 97 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-04-25T13:51:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:14 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:14 smithi137 ceph-mon[32568]: pgmap v630: 97 pgs: 97 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-04-25T13:51:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:14 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:14.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1150944 12.81 MB/sec execute 357 sec latency 75.097 ms 2024-04-25T13:51:15.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:15.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:15.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:51:15.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:15.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:15.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:51:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:51:15.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1152696 12.79 MB/sec execute 358 sec latency 16.463 ms 2024-04-25T13:51:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:16 smithi094 ceph-mon[29195]: pgmap v631: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-04-25T13:51:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:16 smithi138 ceph-mon[32977]: pgmap v631: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-04-25T13:51:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:16 smithi137 ceph-mon[32568]: pgmap v631: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-04-25T13:51:16.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1154942 12.80 MB/sec execute 359 sec latency 16.154 ms 2024-04-25T13:51:17.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1157593 12.80 MB/sec execute 360 sec latency 17.395 ms 2024-04-25T13:51:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:18 smithi094 ceph-mon[29195]: pgmap v632: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T13:51:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:18 smithi138 ceph-mon[32977]: pgmap v632: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T13:51:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:18 smithi137 ceph-mon[32568]: pgmap v632: 97 pgs: 97 active+clean; 51 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T13:51:18.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1159212 12.78 MB/sec execute 361 sec latency 17.180 ms 2024-04-25T13:51:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- 172.21.15.94:0/910427327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148108990 msgr2=0x7fd14810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/910427327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148108990 0x7fd14810adb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd140009ca0 tx=0x7fd14002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- 172.21.15.94:0/910427327 shutdown_connections 2024-04-25T13:51:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/910427327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148108990 0x7fd14810adb0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/910427327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148103c90 0x7fd1481040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/910427327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd148103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- 172.21.15.94:0/910427327 >> 172.21.15.94:0/910427327 conn(0x7fd1480fdc60 msgr2=0x7fd1481000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- 172.21.15.94:0/910427327 shutdown_connections 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- 172.21.15.94:0/910427327 wait complete. 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 Processor -- start 2024-04-25T13:51:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.469+0000 7fd14e9f3640 1 -- start start 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd1480ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148108990 0x7fd1480fe810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810e560 con 0x7fd148103c90 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810e3e0 con 0x7fd148108990 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14e9f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd14810e6e0 con 0x7fd148102a50 2024-04-25T13:51:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57468/0 (socket says 172.21.15.94:57468) 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 -- 172.21.15.94:0/2618568431 learned_addr learned my addr 172.21.15.94:0/2618568431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd147fff640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd1480ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd14cf90640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148108990 0x7fd1480fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 -- 172.21.15.94:0/2618568431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 msgr2=0x7fd1480ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd1480ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 -- 172.21.15.94:0/2618568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148108990 msgr2=0x7fd1480fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148108990 0x7fd1480fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd1477fe640 1 -- 172.21.15.94:0/2618568431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1480fef40 con 0x7fd148103c90 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.470+0000 7fd147fff640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd1480ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd1457fa640 1 -- 172.21.15.94:0/2618568431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13000cd80 con 0x7fd148103c90 2024-04-25T13:51:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1481a4210 con 0x7fd148103c90 2024-04-25T13:51:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd1457fa640 1 -- 172.21.15.94:0/2618568431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1300108e0 con 0x7fd148103c90 2024-04-25T13:51:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd1457fa640 1 -- 172.21.15.94:0/2618568431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130018980 con 0x7fd148103c90 2024-04-25T13:51:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.471+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1481a4700 con 0x7fd148103c90 2024-04-25T13:51:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.473+0000 7fd155982740 1 Processor -- start 2024-04-25T13:51:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1457fa640 1 -- 172.21.15.94:0/2618568431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd130018b20 con 0x7fd148103c90 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 -- start start 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 0x564696782970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696782ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646966e8460 con 0x5646963c14d0 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646966a2590 con 0x5646963b92f0 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd155982740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646966e4420 con 0x5646963e21b0 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57472/0 (socket says 172.21.15.94:57472) 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 -- 172.21.15.94:0/3640983460 learned_addr learned my addr 172.21.15.94:0/3640983460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd147fff640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696782ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd14cf90640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 0x564696782970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 -- 172.21.15.94:0/3640983460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 msgr2=0x564696782970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 0x564696782970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 -- 172.21.15.94:0/3640983460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 msgr2=0x564696782ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696782ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1457fa640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 0x7fd1340b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd1477fe640 1 -- 172.21.15.94:0/3640983460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646966af0d0 con 0x5646963c14d0 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd147fff640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696782ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.474+0000 7fd14cf90640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 0x564696782970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd14cf90640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 0x7fd1340b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd1477fe640 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a7020 tx=0x7fd1300a3c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f7ca9d544f8d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd1457fa640 1 -- 172.21.15.94:0/2618568431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd1300df020 con 0x7fd148103c90 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd13e7fc640 1 -- 172.21.15.94:0/3640983460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a72e0 con 0x5646963c14d0 2024-04-25T13:51:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd13e7fc640 1 -- 172.21.15.94:0/3640983460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1300a7510 con 0x5646963c14d0 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.475+0000 7fd13e7fc640 1 -- 172.21.15.94:0/3640983460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a6c80 con 0x5646963c14d0 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- 172.21.15.94:0/3640983460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 msgr2=0x564696789df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd14cf90640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 0x7fd1340b1ff0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd140004af0 tx=0x7fd140004a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a7020 tx=0x7fd1300a3c20 comp rx=0 tx=0).stop 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- 172.21.15.94:0/3640983460 shutdown_connections 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963c14d0 0x564696789df0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696782ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 --2- 172.21.15.94:0/3640983460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963e21b0 0x564696782970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- 172.21.15.94:0/3640983460 >> 172.21.15.94:0/3640983460 conn(0x5646963d6300 msgr2=0x56469677e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- 172.21.15.94:0/3640983460 shutdown_connections 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- 172.21.15.94:0/3640983460 wait complete. 2024-04-25T13:51:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 -- start start 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.476+0000 7fd155982740 1 Processor -- start 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469678cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963e21b0 0x564696793ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56469678a780 con 0x5646963b92f0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56469666fb60 con 0x5646963e21b0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56469678a900 con 0x5646963c14d0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd155982740 10 client.0 _ll_register_callbacks cb 0x5646966942b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57482/0 (socket says 172.21.15.94:57482) 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 -- 172.21.15.94:0/2385531405 learned_addr learned my addr 172.21.15.94:0/2385531405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd14cf90640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963e21b0 0x564696793ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469678cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 -- 172.21.15.94:0/2385531405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 msgr2=0x56469678cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469678cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 -- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963e21b0 msgr2=0x564696793ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963e21b0 0x564696793ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 -- 172.21.15.94:0/2385531405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646963fa210 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd147fff640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd138012d00 tx=0x7fd13800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f89865f7db0b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.477+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138009d90 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd138012110 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1300a7510 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd138004050 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 10 client.18954 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646964c6620 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1300a7510 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd138026450 con 0x5646963b92f0 2024-04-25T13:51:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 10 client.18954 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 10 client.18954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 20 client.18954 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963a0d80 0x56469679a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 4 client.18954 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646966f6070 con 0x5646963a0d80 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.478+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963a0d80 0x56469679a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.479+0000 7fd1477fe640 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963a0d80 0x56469679a3e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30ccfb310000bb7 server_cookie=3a4007861a5cbf6e in_seq=0 out_seq=0 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.479+0000 7fd1277fe640 10 client.18954 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.479+0000 7fd1277fe640 1 -- 172.21.15.94:0/2385531405 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646966f6070 con 0x5646963a0d80 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.479+0000 7fd1277fe640 10 client.18954 handle_command_reply: tid=0 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.479+0000 7fd155982740 1 client.18954 shutdown 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 msgr2=0x56469678c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd138012d00 tx=0x7fd13800f790 comp rx=0 tx=0).stop 2024-04-25T13:51:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 shutdown_connections 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963a0d80 0x56469679a3e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963e21b0 0x564696793ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd1277fe640 0 client.18954 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469678cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 --2- 172.21.15.94:0/2385531405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x56469678c640 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 >> 172.21.15.94:0/2385531405 conn(0x5646963d6300 msgr2=0x564696781210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 shutdown_connections 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 1 -- 172.21.15.94:0/2385531405 wait complete. 2024-04-25T13:51:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.480+0000 7fd155982740 20 client.18954 trim_cache size 0 max 16384 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.501+0000 7fd155982740 1 Processor -- start 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.501+0000 7fd155982740 1 -- start start 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963a0d80 0x564696787190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469677e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ad6c0 con 0x5646963b92f0 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481af200 con 0x5646963a0d80 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd155982740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481add90 con 0x5646963c14d0 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57492/0 (socket says 172.21.15.94:57492) 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 -- 172.21.15.94:0/650517451 learned_addr learned my addr 172.21.15.94:0/650517451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd147fff640 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963a0d80 0x564696787190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd14cf90640 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469677e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 -- 172.21.15.94:0/650517451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 msgr2=0x56469677e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469677e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 -- 172.21.15.94:0/650517451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963a0d80 msgr2=0x564696787190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963a0d80 0x564696787190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 -- 172.21.15.94:0/650517451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469677e8a0 con 0x5646963b92f0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.502+0000 7fd1477fe640 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a7210 tx=0x7fd130000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76b127e4f6de3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.503+0000 7fd13dffb640 1 -- 172.21.15.94:0/650517451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a6e60 con 0x5646963b92f0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.503+0000 7fd13dffb640 1 -- 172.21.15.94:0/650517451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1300e28d0 con 0x5646963b92f0 2024-04-25T13:51:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.503+0000 7fd13dffb640 1 -- 172.21.15.94:0/650517451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a7050 con 0x5646963b92f0 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- 172.21.15.94:0/650517451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 msgr2=0x564696793a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a7210 tx=0x7fd130000bc0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- 172.21.15.94:0/650517451 shutdown_connections 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x56469677e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963b92f0 0x564696793a00 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 --2- 172.21.15.94:0/650517451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963a0d80 0x564696787190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- 172.21.15.94:0/650517451 >> 172.21.15.94:0/650517451 conn(0x5646963d6300 msgr2=0x56469677dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- 172.21.15.94:0/650517451 shutdown_connections 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- 172.21.15.94:0/650517451 wait complete. 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 -- start start 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.504+0000 7fd155982740 1 Processor -- start 2024-04-25T13:51:19.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963a0d80 0x564696790f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696791490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481aeb30 con 0x5646963a0d80 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ae460 con 0x5646963b92f0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1481ab6a0 con 0x5646963c14d0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd155982740 10 client.0 _ll_register_callbacks cb 0x7fd138009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36306/0 (socket says 172.21.15.94:36306) 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 -- 172.21.15.94:0/716137589 learned_addr learned my addr 172.21.15.94:0/716137589 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd1477fe640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696791490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd147fff640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963a0d80 0x564696790f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 -- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 msgr2=0x564696791490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696791490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 -- 172.21.15.94:0/716137589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963a0d80 msgr2=0x564696790f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963a0d80 0x564696790f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.505+0000 7fd14cf90640 1 -- 172.21.15.94:0/716137589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564696792930 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd14cf90640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd140035000 tx=0x7fd14002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c106db73ce4ae05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd1277fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1400307c0 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8385+0+0 (secure 0 0 0) 0x7fd140042030 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1300e28d0 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd140030960 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd155982740 10 client.28249 fetch_fsmap learned FSMap version 27 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.506+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564696793c30 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1300e28d0 con 0x5646963c14d0 2024-04-25T13:51:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd140055420 con 0x5646963c14d0 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 10 client.28249 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 10 client.28249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 20 client.28249 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963e21b0 0x564696804ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 4 client.28249 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646966efee0 con 0x5646963e21b0 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.507+0000 7fd147fff640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963e21b0 0x564696804ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.508+0000 7fd1277fe640 10 client.28249 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.508+0000 7fd147fff640 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963e21b0 0x564696804ef0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6fdb2bb77cf09b7 server_cookie=833587b0d1eadca6 in_seq=0 out_seq=0 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd1277fe640 1 -- 172.21.15.94:0/716137589 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646966efee0 con 0x5646963e21b0 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd1277fe640 10 client.28249 handle_command_reply: tid=0 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 client.28249 shutdown 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 msgr2=0x564696792370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd140035000 tx=0x7fd14002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 shutdown_connections 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646963e21b0 0x564696804ef0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646963c14d0 0x564696792370 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646963b92f0 0x564696791490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd1277fe640 0 client.28249 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 --2- 172.21.15.94:0/716137589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646963a0d80 0x564696790f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 >> 172.21.15.94:0/716137589 conn(0x5646963d6300 msgr2=0x56469678c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 shutdown_connections 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 1 -- 172.21.15.94:0/716137589 wait complete. 2024-04-25T13:51:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.695+0000 7fd155982740 20 client.28249 trim_cache size 0 max 16384 2024-04-25T13:51:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:51:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:51:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:51:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 msgr2=0x7fd1340b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 0x7fd1340b1ff0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd140004af0 tx=0x7fd140004a60 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 msgr2=0x7fd1480fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 shutdown_connections 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd1340afae0 0x7fd1340b1ff0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd148108990 0x7fd1480fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd148103c90 0x7fd1480fe2d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 --2- 172.21.15.94:0/2618568431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd148102a50 0x7fd1480ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 >> 172.21.15.94:0/2618568431 conn(0x7fd1480fdc60 msgr2=0x7fd14810a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 shutdown_connections 2024-04-25T13:51:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:19.696+0000 7fd14e9f3640 1 -- 172.21.15.94:0/2618568431 wait complete. 2024-04-25T13:51:19.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1161296 12.78 MB/sec execute 362 sec latency 105.393 ms 2024-04-25T13:51:20.201 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:51:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T13:51:20.201 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:51:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T13:51:20.201 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:51:19 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/3328495521 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b521ce1a40 con 0x55b521d29180 2024-04-25T13:51:20.202 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:51:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T13:51:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:20 smithi094 ceph-mon[29195]: pgmap v633: 97 pgs: 97 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:20 smithi094 ceph-mon[29195]: daemon mds.f restarted 2024-04-25T13:51:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:20 smithi138 ceph-mon[32977]: pgmap v633: 97 pgs: 97 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:20 smithi138 ceph-mon[32977]: daemon mds.f restarted 2024-04-25T13:51:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:20 smithi137 ceph-mon[32568]: pgmap v633: 97 pgs: 97 active+clean; 50 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:20 smithi137 ceph-mon[32568]: daemon mds.f restarted 2024-04-25T13:51:20.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1162899 12.77 MB/sec execute 363 sec latency 26.993 ms 2024-04-25T13:51:21.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:21 smithi094 ceph-mon[29195]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:51:21.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:21 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/3275100272,v1:172.21.15.138:6835/3275100272] up:boot 2024-04-25T13:51:21.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:21 smithi094 ceph-mon[29195]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:51:21.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:51:21.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:21 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:51:21.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:21 smithi138 ceph-mon[32977]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:51:21.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:21 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/3275100272,v1:172.21.15.138:6835/3275100272] up:boot 2024-04-25T13:51:21.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:21 smithi138 ceph-mon[32977]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:51:21.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:51:21.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:21 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:51:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:21 smithi137 ceph-mon[32568]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:51:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:21 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/3275100272,v1:172.21.15.138:6835/3275100272] up:boot 2024-04-25T13:51:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:21 smithi137 ceph-mon[32568]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:51:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:51:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:21 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:51:21.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1165400 12.78 MB/sec execute 364 sec latency 8.206 ms 2024-04-25T13:51:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:22 smithi094 ceph-mon[29195]: pgmap v635: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-04-25T13:51:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:22 smithi138 ceph-mon[32977]: pgmap v635: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-04-25T13:51:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:22 smithi137 ceph-mon[32568]: pgmap v635: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-04-25T13:51:22.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1167936 12.78 MB/sec execute 365 sec latency 15.867 ms 2024-04-25T13:51:23.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1169639 12.77 MB/sec execute 366 sec latency 16.710 ms 2024-04-25T13:51:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:24 smithi094 ceph-mon[29195]: pgmap v636: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-04-25T13:51:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:24 smithi138 ceph-mon[32977]: pgmap v636: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-04-25T13:51:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:24 smithi137 ceph-mon[32568]: pgmap v636: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-04-25T13:51:24.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1172228 12.77 MB/sec execute 367 sec latency 64.177 ms 2024-04-25T13:51:25.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1174479 12.76 MB/sec execute 368 sec latency 16.000 ms 2024-04-25T13:51:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:26 smithi094 ceph-mon[29195]: pgmap v637: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:26 smithi138 ceph-mon[32977]: pgmap v637: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:26 smithi137 ceph-mon[32568]: pgmap v637: 97 pgs: 97 active+clean; 42 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-04-25T13:51:26.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1176448 12.76 MB/sec execute 369 sec latency 12.934 ms 2024-04-25T13:51:27.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1179411 12.78 MB/sec execute 370 sec latency 11.554 ms 2024-04-25T13:51:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:28 smithi094 ceph-mon[29195]: pgmap v638: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-04-25T13:51:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:28 smithi138 ceph-mon[32977]: pgmap v638: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-04-25T13:51:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:28 smithi137 ceph-mon[32568]: pgmap v638: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-04-25T13:51:28.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1181655 12.76 MB/sec execute 371 sec latency 12.756 ms 2024-04-25T13:51:29.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1183318 12.76 MB/sec execute 372 sec latency 105.146 ms 2024-04-25T13:51:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:30 smithi094 ceph-mon[29195]: pgmap v639: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-04-25T13:51:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:30 smithi138 ceph-mon[32977]: pgmap v639: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-04-25T13:51:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:30 smithi137 ceph-mon[32568]: pgmap v639: 97 pgs: 97 active+clean; 43 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-04-25T13:51:30.808 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1186275 12.77 MB/sec execute 373 sec latency 12.893 ms 2024-04-25T13:51:31.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1187985 12.76 MB/sec execute 374 sec latency 24.816 ms 2024-04-25T13:51:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:32 smithi094 ceph-mon[29195]: pgmap v640: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T13:51:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:32 smithi138 ceph-mon[32977]: pgmap v640: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T13:51:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:32 smithi137 ceph-mon[32568]: pgmap v640: 97 pgs: 97 active+clean; 44 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T13:51:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- 172.21.15.94:0/2431750987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 msgr2=0x7f58b010bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- 172.21.15.94:0/2431750987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b010bce0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009ca0 tx=0x7f58a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- 172.21.15.94:0/2431750987 shutdown_connections 2024-04-25T13:51:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- 172.21.15.94:0/2431750987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b010bce0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- 172.21.15.94:0/2431750987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b01023b0 0x7f58b0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- 172.21.15.94:0/2431750987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b0101eb0 0x7f58b0106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- 172.21.15.94:0/2431750987 >> 172.21.15.94:0/2431750987 conn(0x7f58b00fd3c0 msgr2=0x7f58b00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- 172.21.15.94:0/2431750987 shutdown_connections 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- 172.21.15.94:0/2431750987 wait complete. 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 Processor -- start 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 -- start start 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b0101eb0 0x7f58b019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b01023b0 0x7f58b019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.799+0000 7f58b71ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b71ab640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010f640 con 0x7f58b01097d0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b71ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010f4c0 con 0x7f58b0101eb0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b71ab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58b010f7c0 con 0x7f58b01023b0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57074/0 (socket says 172.21.15.94:57074) 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 -- 172.21.15.94:0/3213909274 learned_addr learned my addr 172.21.15.94:0/3213909274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b4f47640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b0101eb0 0x7f58b019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58affff640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b01023b0 0x7f58b019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 -- 172.21.15.94:0/3213909274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b01023b0 msgr2=0x7f58b019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b01023b0 0x7f58b019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 -- 172.21.15.94:0/3213909274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b0101eb0 msgr2=0x7f58b019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b0101eb0 0x7f58b019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 -- 172.21.15.94:0/3213909274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b01a1cf0 con 0x7f58b01097d0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b5748640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f58a8004af0 tx=0x7f58a8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58adffb640 1 -- 172.21.15.94:0/3213909274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8003b70 con 0x7f58b01097d0 2024-04-25T13:51:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58adffb640 1 -- 172.21.15.94:0/3213909274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a8003d10 con 0x7f58b01097d0 2024-04-25T13:51:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b01a1f80 con 0x7f58b01097d0 2024-04-25T13:51:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.800+0000 7f58adffb640 1 -- 172.21.15.94:0/3213909274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a802fb20 con 0x7f58b01097d0 2024-04-25T13:51:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.801+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b01a2440 con 0x7f58b01097d0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.803+0000 7f58be058740 1 Processor -- start 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.803+0000 7f58adffb640 1 -- 172.21.15.94:0/3213909274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f58a8004580 con 0x7f58b01097d0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.803+0000 7f58be058740 1 -- start start 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a1a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 0x55c928a1aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c928980460 con 0x55c9286594d0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c92893a590 con 0x55c9286512f0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58be058740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c92897c420 con 0x55c92867a1b0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57080/0 (socket says 172.21.15.94:57080) 2024-04-25T13:51:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 -- 172.21.15.94:0/684123053 learned_addr learned my addr 172.21.15.94:0/684123053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58b4f47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 0x55c928a1aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a1a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58adffb640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 0x7f58980b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 -- 172.21.15.94:0/684123053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 msgr2=0x55c928a1a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a1a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 -- 172.21.15.94:0/684123053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 msgr2=0x55c928a1aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 0x55c928a1aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58affff640 1 -- 172.21.15.94:0/684123053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9289470d0 con 0x55c9286594d0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58b4f47640 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 0x55c928a1aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58b5748640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 0x7f58980b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.804+0000 7f58adffb640 1 -- 172.21.15.94:0/3213909274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f58a8100210 con 0x7f58b01097d0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.805+0000 7f58affff640 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009fd0 tx=0x7f58a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4013952c8b002cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.805+0000 7f5896ffd640 1 -- 172.21.15.94:0/684123053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4004030 con 0x55c9286594d0 2024-04-25T13:51:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.805+0000 7f5896ffd640 1 -- 172.21.15.94:0/684123053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a4005020 con 0x55c9286594d0 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.805+0000 7f5896ffd640 1 -- 172.21.15.94:0/684123053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4004b80 con 0x55c9286594d0 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- 172.21.15.94:0/684123053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 msgr2=0x55c928a21df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58b5748640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 0x7f58980b2070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c4640 tx=0x7f58a808eae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009fd0 tx=0x7f58a4004730 comp rx=0 tx=0).stop 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- 172.21.15.94:0/684123053 shutdown_connections 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286594d0 0x55c928a21df0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286512f0 0x55c928a1aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 --2- 172.21.15.94:0/684123053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a1a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- 172.21.15.94:0/684123053 >> 172.21.15.94:0/684123053 conn(0x55c92866e300 msgr2=0x55c928a16300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- 172.21.15.94:0/684123053 shutdown_connections 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- 172.21.15.94:0/684123053 wait complete. 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 -- start start 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.806+0000 7f58be058740 1 Processor -- start 2024-04-25T13:51:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9286512f0 0x55c928a24640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 0x55c928a2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c928a22840 con 0x55c92867a1b0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c928907b60 con 0x55c9286594d0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c928a229c0 con 0x55c9286512f0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 10 client.0 _ll_register_callbacks cb 0x55c92892c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 0x55c928a2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 0x55c928a2bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57084/0 (socket says 172.21.15.94:57084) 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58affff640 1 -- 172.21.15.94:0/798658083 learned_addr learned my addr 172.21.15.94:0/798658083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58affff640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9286512f0 0x55c928a24640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 -- 172.21.15.94:0/798658083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9286512f0 msgr2=0x55c928a24640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9286512f0 0x55c928a24640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 -- 172.21.15.94:0/798658083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 msgr2=0x55c928a2bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 0x55c928a2bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 -- 172.21.15.94:0/798658083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c928692210 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58b4f47640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a24b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c928a1bd50 tx=0x7f58a0012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518f04c242c841fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f588ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0004050 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f58a00041f0 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.807+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a4005020 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58a00043f0 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 10 client.28430 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c92875e620 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a4005020 con 0x55c9286594d0 2024-04-25T13:51:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f58a0003b70 con 0x55c9286594d0 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 10 client.28430 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 10 client.28430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 20 client.28430 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c928638d80 0x55c928a32330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 4 client.28430 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c92898e070 con 0x55c928638d80 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.808+0000 7f58affff640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c928638d80 0x55c928a32330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.809+0000 7f58affff640 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c928638d80 0x55c928a32330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32df9bcf215eae6 server_cookie=d45d650badb78225 in_seq=0 out_seq=0 2024-04-25T13:51:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.809+0000 7f588ffff640 10 client.28430 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:32.815 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1190369 12.76 MB/sec execute 375 sec latency 12.717 ms 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f588ffff640 1 -- 172.21.15.94:0/798658083 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c92898e070 con 0x55c928638d80 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f588ffff640 10 client.28430 handle_command_reply: tid=0 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 client.28430 shutdown 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 msgr2=0x55c928a24b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a24b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55c928a1bd50 tx=0x7f58a0012b40 comp rx=0 tx=0).stop 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 shutdown_connections 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c928638d80 0x55c928a32330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92867a1b0 0x55c928a2bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f588ffff640 0 client.28430 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a24b80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 --2- 172.21.15.94:0/798658083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9286512f0 0x55c928a24640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 >> 172.21.15.94:0/798658083 conn(0x55c92866e300 msgr2=0x55c928a19210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 shutdown_connections 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 1 -- 172.21.15.94:0/798658083 wait complete. 2024-04-25T13:51:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.810+0000 7f58be058740 20 client.28430 trim_cache size 0 max 16384 2024-04-25T13:51:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 Processor -- start 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 -- start start 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a16270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ab4a0 con 0x55c9286512f0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01acfe0 con 0x55c9286594d0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58be058740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01abb70 con 0x55c92867a1b0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57092/0 (socket says 172.21.15.94:57092) 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 -- 172.21.15.94:0/1278291059 learned_addr learned my addr 172.21.15.94:0/1278291059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58b4f47640 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a16270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 -- 172.21.15.94:0/1278291059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 msgr2=0x55c928a16270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a16270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 -- 172.21.15.94:0/1278291059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 msgr2=0x55c928a2ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a2ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.825+0000 7f58affff640 1 -- 172.21.15.94:0/1278291059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928a2bf40 con 0x55c9286512f0 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58affff640 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009730 tx=0x7f58a4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c7e30788f5c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58967fc640 1 -- 172.21.15.94:0/1278291059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4038070 con 0x55c9286512f0 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58967fc640 1 -- 172.21.15.94:0/1278291059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a400fc40 con 0x55c9286512f0 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58967fc640 1 -- 172.21.15.94:0/1278291059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40306e0 con 0x55c9286512f0 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 -- 172.21.15.94:0/1278291059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 msgr2=0x55c928a15d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009730 tx=0x7f58a4009cb0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 -- 172.21.15.94:0/1278291059 shutdown_connections 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a16270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a2ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 --2- 172.21.15.94:0/1278291059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a15d30 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 -- 172.21.15.94:0/1278291059 >> 172.21.15.94:0/1278291059 conn(0x55c92866e300 msgr2=0x55c928a1ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 -- 172.21.15.94:0/1278291059 shutdown_connections 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.826+0000 7f58be058740 1 -- 172.21.15.94:0/1278291059 wait complete. 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 -- start start 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 Processor -- start 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ac910 con 0x55c9286512f0 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01ac240 con 0x55c9286594d0 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58b01a9480 con 0x55c92867a1b0 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58be058740 10 client.0 _ll_register_callbacks cb 0x7f58a0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57106/0 (socket says 172.21.15.94:57106) 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58affff640 1 -- 172.21.15.94:0/3638432873 learned_addr learned my addr 172.21.15.94:0/3638432873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58b4f47640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.827+0000 7f58affff640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 -- 172.21.15.94:0/3638432873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 msgr2=0x55c928a2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a2a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 -- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 msgr2=0x55c928a29660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a29660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 -- 172.21.15.94:0/3638432873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c928a2ab50 con 0x55c9286512f0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58b4f47640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58affff640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f58a40117f0 tx=0x7f58a4009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b888d6a8dd695254 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4038040 con 0x55c9286512f0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f58a400f880 con 0x55c9286512f0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a400fc40 con 0x55c9286512f0 2024-04-25T13:51:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58a400fa80 con 0x55c9286512f0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58be058740 10 client.18984 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c928a9a6e0 con 0x55c9286512f0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a400fc40 con 0x55c9286512f0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.828+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f58a402f040 con 0x55c9286512f0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 10 client.18984 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 10 client.18984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 20 client.18984 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c92860e1c0 0x55c928a9cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 4 client.18984 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c928987ee0 con 0x55c92860e1c0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58b4f47640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c92860e1c0 0x55c928a9cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f588ffff640 10 client.18984 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:32.829+0000 7f58b4f47640 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c92860e1c0 0x55c928a9cd20 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d756c958a8222c1a server_cookie=5f8b7c7fff0bf97 in_seq=0 out_seq=0 2024-04-25T13:51:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.091+0000 7f588ffff640 1 -- 172.21.15.94:0/3638432873 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c928987ee0 con 0x55c92860e1c0 2024-04-25T13:51:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.091+0000 7f588ffff640 10 client.18984 handle_command_reply: tid=0 2024-04-25T13:51:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.091+0000 7f58be058740 1 client.18984 shutdown 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 msgr2=0x55c928a29120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f58a40117f0 tx=0x7f58a4009d20 comp rx=0 tx=0).stop 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 shutdown_connections 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c92860e1c0 0x55c928a9cd20 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92867a1b0 0x55c928a2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9286594d0 0x55c928a29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f588ffff640 0 client.18984 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 --2- 172.21.15.94:0/3638432873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9286512f0 0x55c928a29120 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 >> 172.21.15.94:0/3638432873 conn(0x55c92866e300 msgr2=0x55c928a15ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 shutdown_connections 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 1 -- 172.21.15.94:0/3638432873 wait complete. 2024-04-25T13:51:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58be058740 20 client.18984 trim_cache size 0 max 16384 2024-04-25T13:51:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:51:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:51:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:51:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 msgr2=0x7f58980b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.092+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 0x7f58980b2070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c4640 tx=0x7f58a808eae0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 msgr2=0x7f58b01a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f58a8004af0 tx=0x7f58a8031700 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 shutdown_connections 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afb60 0x7f58980b2070 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b01097d0 0x7f58b01a1680 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58b01023b0 0x7f58b019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 --2- 172.21.15.94:0/3213909274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b0101eb0 0x7f58b019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 >> 172.21.15.94:0/3213909274 conn(0x7f58b00fd3c0 msgr2=0x7f58b0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 shutdown_connections 2024-04-25T13:51:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:33.093+0000 7f58b71ab640 1 -- 172.21.15.94:0/3213909274 wait complete. 2024-04-25T13:51:33.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1192716 12.76 MB/sec execute 376 sec latency 93.510 ms 2024-04-25T13:51:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:34 smithi094 ceph-mon[29195]: pgmap v641: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-04-25T13:51:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:34 smithi138 ceph-mon[32977]: pgmap v641: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-04-25T13:51:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:34 smithi137 ceph-mon[32568]: pgmap v641: 97 pgs: 97 active+clean; 48 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-04-25T13:51:34.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1194105 12.74 MB/sec execute 377 sec latency 57.431 ms 2024-04-25T13:51:35.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1197037 12.76 MB/sec execute 378 sec latency 14.352 ms 2024-04-25T13:51:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:36 smithi094 ceph-mon[29195]: pgmap v642: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T13:51:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:36 smithi138 ceph-mon[32977]: pgmap v642: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T13:51:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:36 smithi137 ceph-mon[32568]: pgmap v642: 97 pgs: 97 active+clean; 45 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T13:51:36.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1199021 12.74 MB/sec execute 379 sec latency 24.938 ms 2024-04-25T13:51:37.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1200906 12.74 MB/sec execute 380 sec latency 72.648 ms 2024-04-25T13:51:38.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:51:38.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:51:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 -- 172.21.15.94:0/3027440458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 msgr2=0x7fc82410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 --2- 172.21.15.94:0/3027440458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc82410aba0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009be0 tx=0x7fc81c02a6c0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 -- 172.21.15.94:0/3027440458 shutdown_connections 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 --2- 172.21.15.94:0/3027440458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc82410aba0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 --2- 172.21.15.94:0/3027440458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824103930 0x7fc824103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 --2- 172.21.15.94:0/3027440458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824102f50 0x7fc824103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 -- 172.21.15.94:0/3027440458 >> 172.21.15.94:0/3027440458 conn(0x7fc8240fd350 msgr2=0x7fc8240ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 -- 172.21.15.94:0/3027440458 shutdown_connections 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.183+0000 7fc82b205640 1 -- 172.21.15.94:0/3027440458 wait complete. 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 Processor -- start 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 -- start start 2024-04-25T13:51:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 0x7fc8241021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 0x7fc82419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e520 con 0x7fc824104450 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e3a0 con 0x7fc824103930 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc82b205640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc82410e6a0 con 0x7fc824102f50 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 0x7fc8241021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 0x7fc8241021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43268/0 (socket says 172.21.15.94:43268) 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41140/0 (socket says 172.21.15.94:41140) 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc828fa1640 1 -- 172.21.15.94:0/811862014 learned_addr learned my addr 172.21.15.94:0/811862014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc823fff640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 0x7fc82419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 -- 172.21.15.94:0/811862014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 msgr2=0x7fc8241021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 0x7fc8241021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 -- 172.21.15.94:0/811862014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 msgr2=0x7fc82419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 0x7fc82419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc8297a2640 1 -- 172.21.15.94:0/811862014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8241a1c80 con 0x7fc824104450 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.184+0000 7fc823fff640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 0x7fc82419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc8297a2640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02abe0 tx=0x7fc81c030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc821ffb640 1 -- 172.21.15.94:0/811862014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c041020 con 0x7fc824104450 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8241a1f10 con 0x7fc824104450 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc821ffb640 1 -- 172.21.15.94:0/811862014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c0318f0 con 0x7fc824104450 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc821ffb640 1 -- 172.21.15.94:0/811862014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c036040 con 0x7fc824104450 2024-04-25T13:51:38.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.185+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8241a2320 con 0x7fc824104450 2024-04-25T13:51:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 Processor -- start 2024-04-25T13:51:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc821ffb640 1 -- 172.21.15.94:0/811862014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc81c044020 con 0x7fc824104450 2024-04-25T13:51:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 -- start start 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b3aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6a9fd80 con 0x55c3f67786b0 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6a598b0 con 0x55c3f6799390 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.188+0000 7fc832192740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6a9bd40 con 0x55c3f67704d0 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc821ffb640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 0x7fc8140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41156/0 (socket says 172.21.15.94:41156) 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc823fff640 1 -- 172.21.15.94:0/2868223343 learned_addr learned my addr 172.21.15.94:0/2868223343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc823fff640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b3aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc8297a2640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 0x7fc8140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc821ffb640 1 -- 172.21.15.94:0/811862014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fc81c0ff8b0 con 0x7fc824104450 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 -- 172.21.15.94:0/2868223343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 msgr2=0x55c3f6b3aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b3aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 -- 172.21.15.94:0/2868223343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 msgr2=0x55c3f6b3bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 -- 172.21.15.94:0/2868223343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f6a2aa40 con 0x55c3f67704d0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc823fff640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b3afe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc818009c80 tx=0x7fc818011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a761e72b7b9acdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc8297a2640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 0x7fc8140b2000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0c79f0 tx=0x7fc81c08e160 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.189+0000 7fc80effd640 1 -- 172.21.15.94:0/2868223343 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81801e070 con 0x55c3f67704d0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc80effd640 1 -- 172.21.15.94:0/2868223343 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818019ae0 con 0x55c3f67704d0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc80effd640 1 -- 172.21.15.94:0/2868223343 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81800e920 con 0x55c3f67704d0 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 -- 172.21.15.94:0/2868223343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 msgr2=0x55c3f6b3afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b3afe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc818009c80 tx=0x7fc818011810 comp rx=0 tx=0).stop 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 -- 172.21.15.94:0/2868223343 shutdown_connections 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b3bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b3afe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 --2- 172.21.15.94:0/2868223343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b3aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.190+0000 7fc832192740 1 -- 172.21.15.94:0/2868223343 >> 172.21.15.94:0/2868223343 conn(0x55c3f678d4e0 msgr2=0x55c3f6b36270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- 172.21.15.94:0/2868223343 shutdown_connections 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- 172.21.15.94:0/2868223343 wait complete. 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- start start 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 Processor -- start 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b44900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6b3c780 con 0x55c3f67786b0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6a26d10 con 0x55c3f6799390 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3f6b3c900 con 0x55c3f67704d0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.191+0000 7fc832192740 10 client.0 _ll_register_callbacks cb 0x55c3f6a53210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc823fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b44900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41168/0 (socket says 172.21.15.94:41168) 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc823fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b44900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43284/0 (socket says 172.21.15.94:43284) 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 -- 172.21.15.94:0/1939546592 learned_addr learned my addr 172.21.15.94:0/1939546592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc823fff640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 -- 172.21.15.94:0/1939546592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 msgr2=0x55c3f6b44900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b44900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 -- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 msgr2=0x55c3f6b4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 -- 172.21.15.94:0/1939546592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f687b7d0 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc828fa1640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc818009fa0 tx=0x7fc818019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f862d626e02bad31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc803fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818029020 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fc818005040 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818019ae0 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc818034050 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc832192740 10 client.18996 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.192+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c3f67b13f0 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818019ae0 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc818004cd0 con 0x55c3f67786b0 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 10 client.18996 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 20 client.18996 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f6757f60 0x55c3f6b52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 4 client.18996 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3f6aad990 con 0x55c3f6757f60 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc823fff640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f6757f60 0x55c3f6b52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc803fff640 10 client.18996 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.193+0000 7fc823fff640 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f6757f60 0x55c3f6b52050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f595754f783f6586 server_cookie=80251a6d8741fb24 in_seq=0 out_seq=0 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc803fff640 1 -- 172.21.15.94:0/1939546592 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3f6aad990 con 0x55c3f6757f60 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc803fff640 10 client.18996 handle_command_reply: tid=0 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 client.18996 shutdown 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 msgr2=0x55c3f6b44e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc818009fa0 tx=0x7fc818019660 comp rx=0 tx=0).stop 2024-04-25T13:51:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 shutdown_connections 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f6757f60 0x55c3f6b52050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f6799390 0x55c3f6b4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b44e40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 --2- 172.21.15.94:0/1939546592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f67704d0 0x55c3f6b44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc803fff640 0 client.18996 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.194+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 >> 172.21.15.94:0/1939546592 conn(0x55c3f678d4e0 msgr2=0x55c3f6b39180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.195+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 shutdown_connections 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.195+0000 7fc832192740 1 -- 172.21.15.94:0/1939546592 wait complete. 2024-04-25T13:51:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.195+0000 7fc832192740 20 client.18996 trim_cache size 0 max 16384 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 Processor -- start 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 -- start start 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 0x55c3f6b35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241ab4b0 con 0x55c3f67786b0 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241acff0 con 0x55c3f67704d0 2024-04-25T13:51:38.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc832192740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241abb80 con 0x55c3f6799390 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41176/0 (socket says 172.21.15.94:41176) 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 -- 172.21.15.94:0/2108751953 learned_addr learned my addr 172.21.15.94:0/2108751953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc823fff640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 0x55c3f6b35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc823fff640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 -- 172.21.15.94:0/2108751953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 msgr2=0x55c3f6b42f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b42f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 -- 172.21.15.94:0/2108751953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 msgr2=0x55c3f6b35c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 0x55c3f6b35c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc828fa1640 1 -- 172.21.15.94:0/2108751953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f67710f0 con 0x55c3f67786b0 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.214+0000 7fc823fff640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 0x55c3f6b35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.215+0000 7fc828fa1640 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fc818011d90 tx=0x7fc818011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910723c81362101e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.215+0000 7fc80e7fc640 1 -- 172.21.15.94:0/2108751953 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818029040 con 0x55c3f67786b0 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.215+0000 7fc80e7fc640 1 -- 172.21.15.94:0/2108751953 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8180054f0 con 0x55c3f67786b0 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.215+0000 7fc80e7fc640 1 -- 172.21.15.94:0/2108751953 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81801e040 con 0x55c3f67786b0 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- 172.21.15.94:0/2108751953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 msgr2=0x55c3f6b36150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fc818011d90 tx=0x7fc818011810 comp rx=0 tx=0).stop 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- 172.21.15.94:0/2108751953 shutdown_connections 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b42f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67786b0 0x55c3f6b36150 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- 172.21.15.94:0/2108751953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67704d0 0x55c3f6b35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- 172.21.15.94:0/2108751953 >> 172.21.15.94:0/2108751953 conn(0x55c3f678d4e0 msgr2=0x55c3f6b3f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- 172.21.15.94:0/2108751953 shutdown_connections 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- 172.21.15.94:0/2108751953 wait complete. 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- start start 2024-04-25T13:51:38.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 Processor -- start 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67786b0 0x55c3f6b47dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b48300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241ac920 con 0x55c3f67704d0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241ac250 con 0x55c3f67786b0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.216+0000 7fc832192740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241a9490 con 0x55c3f6799390 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc832192740 10 client.0 _ll_register_callbacks cb 0x7fc81801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41184/0 (socket says 172.21.15.94:41184) 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 -- 172.21.15.94:0/3359334491 learned_addr learned my addr 172.21.15.94:0/3359334491 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc828fa1640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67786b0 0x55c3f6b47dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b48300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 -- 172.21.15.94:0/3359334491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 msgr2=0x55c3f6b48300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b48300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 -- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67786b0 msgr2=0x55c3f6b47dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67786b0 0x55c3f6b47dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 -- 172.21.15.94:0/3359334491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f6a604b0 con 0x55c3f67704d0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc823fff640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc810000c00 tx=0x7fc810005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6471e4d3d1060ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc803fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81001a040 con 0x55c3f67704d0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fc810004bb0 con 0x55c3f67704d0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8180054f0 con 0x55c3f67704d0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc810004db0 con 0x55c3f67704d0 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc832192740 10 client.19002 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.217+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c3f6bb9410 con 0x55c3f67704d0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8180054f0 con 0x55c3f67704d0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc810030030 con 0x55c3f67704d0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 10 client.19002 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 20 client.19002 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f672d3a0 0x55c3f6bbbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 4 client.19002 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3f6aa7800 con 0x55c3f672d3a0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc828fa1640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f672d3a0 0x55c3f6bbbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc803fff640 10 client.19002 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.218+0000 7fc828fa1640 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f672d3a0 0x55c3f6bbbd90 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d5798bd4e939a0 server_cookie=34d20bb3c0b53783 in_seq=0 out_seq=0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc803fff640 1 -- 172.21.15.94:0/3359334491 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3f6aa7800 con 0x55c3f672d3a0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc803fff640 10 client.19002 handle_command_reply: tid=0 2024-04-25T13:51:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 client.19002 shutdown 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 msgr2=0x55c3f6b4b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc810000c00 tx=0x7fc810005f90 comp rx=0 tx=0).stop 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 shutdown_connections 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c3f672d3a0 0x55c3f6bbbd90 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3f6799390 0x55c3f6b48300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3f67786b0 0x55c3f6b47dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 --2- 172.21.15.94:0/3359334491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3f67704d0 0x55c3f6b4b240 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 >> 172.21.15.94:0/3359334491 conn(0x55c3f678d4e0 msgr2=0x55c3f6b35c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc803fff640 0 client.19002 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 shutdown_connections 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 1 -- 172.21.15.94:0/3359334491 wait complete. 2024-04-25T13:51:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.219+0000 7fc832192740 20 client.19002 trim_cache size 0 max 16384 2024-04-25T13:51:38.227 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:38.227 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:51:38.227 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:51:38.227 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 msgr2=0x7fc8140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 0x7fc8140b2000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0c79f0 tx=0x7fc81c08e160 comp rx=0 tx=0).stop 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 msgr2=0x7fc8241a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02abe0 tx=0x7fc81c030d20 comp rx=0 tx=0).stop 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 shutdown_connections 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc8140afaf0 0x7fc8140b2000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc824104450 0x7fc8241a1460 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc824103930 0x7fc82419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 --2- 172.21.15.94:0/811862014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc824102f50 0x7fc8241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 >> 172.21.15.94:0/811862014 conn(0x7fc8240fd350 msgr2=0x7fc824106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 shutdown_connections 2024-04-25T13:51:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.220+0000 7fc82b205640 1 -- 172.21.15.94:0/811862014 wait complete. 2024-04-25T13:51:38.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:51:38.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:38.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 -- 172.21.15.94:0/3297461580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c1041e0 msgr2=0x7f2a8c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/3297461580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c1041e0 0x7f2a8c102be0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009900 tx=0x7f2a7402a410 comp rx=0 tx=0).stop 2024-04-25T13:51:38.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 -- 172.21.15.94:0/3297461580 shutdown_connections 2024-04-25T13:51:38.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/3297461580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 0x7f2a8c10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/3297461580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c1041e0 0x7f2a8c102be0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/3297461580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c103d00 0x7f2a8c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 -- 172.21.15.94:0/3297461580 >> 172.21.15.94:0/3297461580 conn(0x7f2a8c0fd1c0 msgr2=0x7f2a8c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 -- 172.21.15.94:0/3297461580 shutdown_connections 2024-04-25T13:51:38.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.344+0000 7f2a8b59e640 1 -- 172.21.15.94:0/3297461580 wait complete. 2024-04-25T13:51:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 Processor -- start 2024-04-25T13:51:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 -- start start 2024-04-25T13:51:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 0x7f2a8c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c19ef00 0x7f2a8c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c10e320 con 0x7f2a8c103d00 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c10e1a0 con 0x7f2a8c108830 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8b59e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c10e4a0 con 0x7f2a8c19ef00 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41208/0 (socket says 172.21.15.94:41208) 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8a59c640 1 -- 172.21.15.94:0/1264711146 learned_addr learned my addr 172.21.15.94:0/1264711146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a83fff640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 0x7f2a8c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8ad9d640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c19ef00 0x7f2a8c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.345+0000 7f2a8a59c640 1 -- 172.21.15.94:0/1264711146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c19ef00 msgr2=0x7f2a8c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8a59c640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c19ef00 0x7f2a8c1a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8a59c640 1 -- 172.21.15.94:0/1264711146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 msgr2=0x7f2a8c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8a59c640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 0x7f2a8c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8a59c640 1 -- 172.21.15.94:0/1264711146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1a1b40 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8a59c640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9a0 tx=0x7f2a7c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00cd80 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c0035f0 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1a1e30 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c010690 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.346+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a2290 con 0x7f2a8c103d00 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.347+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c0691b0 con 0x7f2a8c103d00 2024-04-25T13:51:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.349+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2a7c004000 con 0x7f2a8c103d00 2024-04-25T13:51:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.349+0000 7f2a837fe640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 0x7f2a780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.349+0000 7f2a83fff640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 0x7f2a780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.349+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f2a7c0df040 con 0x7f2a8c103d00 2024-04-25T13:51:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.350+0000 7f2a83fff640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 0x7f2a780b1ff0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009230 tx=0x7f2a74031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.353+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a7c0042f0 con 0x7f2a8c103d00 2024-04-25T13:51:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:38 smithi094 ceph-mon[29195]: pgmap v643: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T13:51:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:38 smithi138 ceph-mon[32977]: pgmap v643: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T13:51:38.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.594+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8c106f80 con 0x7f2a8c103d00 2024-04-25T13:51:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.594+0000 7f2a837fe640 1 -- 172.21.15.94:0/1264711146 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7f2a8c106f80 con 0x7f2a8c103d00 2024-04-25T13:51:38.595 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:51:38.595 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:51:20.200062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28255":{"gid":28255,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3275100272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3275100272},{"type":"v1","addr":"172.21.15.138:6835","nonce":3275100272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:51:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 29 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 msgr2=0x7f2a780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 0x7f2a780b1ff0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009230 tx=0x7f2a74031400 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 msgr2=0x7f2a8c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9a0 tx=0x7f2a7c00ee60 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 shutdown_connections 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a780afae0 0x7f2a780b1ff0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8c19ef00 0x7f2a8c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a8c108830 0x7f2a8c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 --2- 172.21.15.94:0/1264711146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a8c103d00 0x7f2a8c101f40 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.596+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 >> 172.21.15.94:0/1264711146 conn(0x7f2a8c0fd1c0 msgr2=0x7f2a8c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.597+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 shutdown_connections 2024-04-25T13:51:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.597+0000 7f2a8b59e640 1 -- 172.21.15.94:0/1264711146 wait complete. 2024-04-25T13:51:38.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:51:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:38 smithi137 ceph-mon[32568]: pgmap v643: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.722+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/2615084471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f41033b0 msgr2=0x7fb1f41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.722+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/2615084471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f41033b0 0x7fb1f41037c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800aa10 tx=0x7fb1e802b720 comp rx=0 tx=0).stop 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/2615084471 shutdown_connections 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/2615084471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 0x7fb1f410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/2615084471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f41033b0 0x7fb1f41037c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/2615084471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f4102170 0x7fb1f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/2615084471 >> 172.21.15.94:0/2615084471 conn(0x7fb1f40fd340 msgr2=0x7fb1f40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/2615084471 shutdown_connections 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/2615084471 wait complete. 2024-04-25T13:51:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 Processor -- start 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.723+0000 7fb1f9cd1640 1 -- start start 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 0x7fb1f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 0x7fb1f419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f410d400 con 0x7fb1f4102170 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f410d280 con 0x7fb1f41033b0 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f410d580 con 0x7fb1f41080b0 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41218/0 (socket says 172.21.15.94:41218) 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 -- 172.21.15.94:0/4172782347 learned_addr learned my addr 172.21.15.94:0/4172782347 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 0x7fb1f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 0x7fb1f419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 -- 172.21.15.94:0/4172782347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 msgr2=0x7fb1f419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 0x7fb1f419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 -- 172.21.15.94:0/4172782347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 msgr2=0x7fb1f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 0x7fb1f4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 -- 172.21.15.94:0/4172782347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f419fb40 con 0x7fb1f4102170 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 0x7fb1f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9f0 tx=0x7fb1e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f0ff9640 1 -- 172.21.15.94:0/4172782347 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400ccc0 con 0x7fb1f4102170 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f419fe30 con 0x7fb1f4102170 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f0ff9640 1 -- 172.21.15.94:0/4172782347 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e40035f0 con 0x7fb1f4102170 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.724+0000 7fb1f0ff9640 1 -- 172.21.15.94:0/4172782347 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400ce60 con 0x7fb1f4102170 2024-04-25T13:51:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.725+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a02f0 con 0x7fb1f4102170 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb200b7f740 1 Processor -- start 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb200b7f740 1 -- start start 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb1f0ff9640 1 -- 172.21.15.94:0/4172782347 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb1e4010690 con 0x7fb1f4102170 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.727+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb200b7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562111eb0d80 con 0x562111baa390 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb200b7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562111e6aa20 con 0x562111b896b0 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb200b7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562111eacd40 con 0x562111b814d0 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41222/0 (socket says 172.21.15.94:41222) 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 -- 172.21.15.94:0/39380434 learned_addr learned my addr 172.21.15.94:0/39380434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 -- 172.21.15.94:0/39380434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 msgr2=0x562111f4cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f4cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 -- 172.21.15.94:0/39380434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 msgr2=0x562111f4da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f4da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 -- 172.21.15.94:0/39380434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562111e75c30 con 0x562111baa390 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009fd0 tx=0x7fb1ec0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4848bc809d7d8f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f0ff9640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 0x7fb1dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1d9ffb640 1 -- 172.21.15.94:0/39380434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec004d20 con 0x562111baa390 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 0x7fb1dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1d9ffb640 1 -- 172.21.15.94:0/39380434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ec004050 con 0x562111baa390 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.728+0000 7fb1d9ffb640 1 -- 172.21.15.94:0/39380434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec005280 con 0x562111baa390 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb1f0ff9640 1 -- 172.21.15.94:0/4172782347 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb1e40def80 con 0x7fb1f4102170 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 -- 172.21.15.94:0/39380434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 msgr2=0x562111f4c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb1f37fe640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 0x7fb1dc0b2000 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x562111f4c530 tx=0x7fb1e40e2a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009fd0 tx=0x7fb1ec0118d0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 -- 172.21.15.94:0/39380434 shutdown_connections 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f4da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f4cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 --2- 172.21.15.94:0/39380434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111baa390 0x562111f4c810 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.729+0000 7fb200b7f740 1 -- 172.21.15.94:0/39380434 >> 172.21.15.94:0/39380434 conn(0x562111b9e4e0 msgr2=0x562111f48020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 -- 172.21.15.94:0/39380434 shutdown_connections 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 -- 172.21.15.94:0/39380434 wait complete. 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 -- start start 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 Processor -- start 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.730+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111baa390 0x562111f5d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562111f4e530 con 0x562111b814d0 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562111e37dc0 con 0x562111b896b0 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562111f4e6b0 con 0x562111baa390 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41238/0 (socket says 172.21.15.94:41238) 2024-04-25T13:51:38.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1068383913 learned_addr learned my addr 172.21.15.94:0/1068383913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 10 client.0 _ll_register_callbacks cb 0x562111e53600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111baa390 msgr2=0x562111f5d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111baa390 0x562111f5d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1068383913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 msgr2=0x562111f56a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f56a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1068383913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562111bc23f0 con 0x562111b814d0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec01b040 tx=0x7fb1ec003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3d2f2304d03e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec028020 con 0x562111b814d0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1ec004050 con 0x562111b814d0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb1ec004c40 con 0x562111b814d0 2024-04-25T13:51:38.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.731+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec033050 con 0x562111b814d0 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 10 client.19020 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562111c8c7d0 con 0x562111b814d0 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ec004050 con 0x562111b814d0 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb1ec00f570 con 0x562111b814d0 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 10 client.19020 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 10 client.19020 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 20 client.19020 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111b68f60 0x562111f63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 4 client.19020 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562111ebe990 con 0x562111b68f60 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.732+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111b68f60 0x562111f63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.733+0000 7fb1d2ffd640 10 client.19020 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:38.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.733+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111b68f60 0x562111f63fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=735b030b6adf8265 server_cookie=546ec23e156b3d20 in_seq=0 out_seq=0 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.733+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1068383913 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562111ebe990 con 0x562111b68f60 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.733+0000 7fb1d2ffd640 10 client.19020 handle_command_reply: tid=0 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.733+0000 7fb200b7f740 1 client.19020 shutdown 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 msgr2=0x562111f56530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec01b040 tx=0x7fb1ec003e60 comp rx=0 tx=0).stop 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 shutdown_connections 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111b68f60 0x562111f63fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111baa390 0x562111f5d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb1d2ffd640 0 client.19020 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f56a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1068383913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f56530 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 >> 172.21.15.94:0/1068383913 conn(0x562111b9e4e0 msgr2=0x562111f4af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 shutdown_connections 2024-04-25T13:51:38.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 1 -- 172.21.15.94:0/1068383913 wait complete. 2024-04-25T13:51:38.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.734+0000 7fb200b7f740 20 client.19020 trim_cache size 0 max 16384 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 Processor -- start 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 -- start start 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b68f60 0x562111f48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f54d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41a9400 con 0x562111b814d0 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41aaf40 con 0x562111b896b0 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb200b7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41a9ad0 con 0x562111b68f60 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb1f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb1f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41242/0 (socket says 172.21.15.94:41242) 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb1f2ffd640 1 -- 172.21.15.94:0/1070943239 learned_addr learned my addr 172.21.15.94:0/1070943239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b68f60 0x562111f48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.748+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f54d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 -- 172.21.15.94:0/1070943239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b68f60 msgr2=0x562111f48160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b68f60 0x562111f48160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 -- 172.21.15.94:0/1070943239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 msgr2=0x562111f54d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f54d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 -- 172.21.15.94:0/1070943239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562111f55260 con 0x562111b814d0 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f54d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800a7d0 tx=0x7fb1e8030a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41369087c5bc6728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1d97fa640 1 -- 172.21.15.94:0/1070943239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8036040 con 0x562111b814d0 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1d97fa640 1 -- 172.21.15.94:0/1070943239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e804c070 con 0x562111b814d0 2024-04-25T13:51:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.749+0000 7fb1d97fa640 1 -- 172.21.15.94:0/1070943239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8003830 con 0x562111b814d0 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- 172.21.15.94:0/1070943239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 msgr2=0x562111f5d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800a7d0 tx=0x7fb1e8030a40 comp rx=0 tx=0).stop 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- 172.21.15.94:0/1070943239 shutdown_connections 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111f54d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b814d0 0x562111f5d960 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1070943239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b68f60 0x562111f48160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- 172.21.15.94:0/1070943239 >> 172.21.15.94:0/1070943239 conn(0x562111b9e4e0 msgr2=0x562111f47a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- 172.21.15.94:0/1070943239 shutdown_connections 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- 172.21.15.94:0/1070943239 wait complete. 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.750+0000 7fb200b7f740 1 -- start start 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 Processor -- start 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111fca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41aa870 con 0x562111b68f60 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41aa1a0 con 0x562111b896b0 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41a73e0 con 0x562111b814d0 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb200b7f740 10 client.0 _ll_register_callbacks cb 0x7fb1ec0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41246/0 (socket says 172.21.15.94:41246) 2024-04-25T13:51:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1863908907 learned_addr learned my addr 172.21.15.94:0/1863908907 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111fca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 msgr2=0x562111f5afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f5afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1863908907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 msgr2=0x562111fca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111fca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 -- 172.21.15.94:0/1863908907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562111fca880 con 0x562111b68f60 2024-04-25T13:51:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f5afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1f3fff640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00fb30 tx=0x7fb1ec004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b61b1b7683405f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.751+0000 7fb1d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec028070 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb1ec00fbb0 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e804c070 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec00fdb0 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 10 client.19026 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562111fcbaa0 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e804c070 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb1ec019a70 con 0x562111b68f60 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 10 client.19026 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 10 client.19026 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 20 client.19026 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111baa390 0x562111fce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 4 client.19026 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562111eb8800 con 0x562111baa390 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.752+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111baa390 0x562111fce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb1d2ffd640 10 client.19026 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb1f2ffd640 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111baa390 0x562111fce320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b735e9d08c3e10e3 server_cookie=17f1e24d32b96d1e in_seq=0 out_seq=0 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb1d2ffd640 1 -- 172.21.15.94:0/1863908907 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562111eb8800 con 0x562111baa390 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb1d2ffd640 10 client.19026 handle_command_reply: tid=0 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 client.19026 shutdown 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 msgr2=0x562111f5aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00fb30 tx=0x7fb1ec004810 comp rx=0 tx=0).stop 2024-04-25T13:51:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 shutdown_connections 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x562111baa390 0x562111fce320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562111b896b0 0x562111fca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb1d2ffd640 0 client.19026 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562111b814d0 0x562111f5afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 --2- 172.21.15.94:0/1863908907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562111b68f60 0x562111f5aaa0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 >> 172.21.15.94:0/1863908907 conn(0x562111b9e4e0 msgr2=0x562111f56100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 shutdown_connections 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.753+0000 7fb200b7f740 1 -- 172.21.15.94:0/1863908907 wait complete. 2024-04-25T13:51:38.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb200b7f740 20 client.19026 trim_cache size 0 max 16384 2024-04-25T13:51:38.762 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:38.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 msgr2=0x7fb1dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 0x7fb1dc0b2000 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x562111f4c530 tx=0x7fb1e40e2a60 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 msgr2=0x7fb1f4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9f0 tx=0x7fb1e400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 shutdown_connections 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb1dc0afaf0 0x7fb1dc0b2000 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f41080b0 0x7fb1f419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1f41033b0 0x7fb1f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 --2- 172.21.15.94:0/4172782347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4102170 0x7fb1f4198000 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 >> 172.21.15.94:0/4172782347 conn(0x7fb1f40fd340 msgr2=0x7fb1f4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 shutdown_connections 2024-04-25T13:51:38.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.754+0000 7fb1f9cd1640 1 -- 172.21.15.94:0/4172782347 wait complete. 2024-04-25T13:51:38.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:51:38.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1203459 12.74 MB/sec execute 381 sec latency 14.777 ms 2024-04-25T13:51:38.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 -- 172.21.15.94:0/1261638183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 msgr2=0x7fef28103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 --2- 172.21.15.94:0/1261638183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28103750 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fef18009900 tx=0x7fef1802a410 comp rx=0 tx=0).stop 2024-04-25T13:51:38.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 -- 172.21.15.94:0/1261638183 shutdown_connections 2024-04-25T13:51:38.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 --2- 172.21.15.94:0/1261638183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28108990 0x7fef2810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 --2- 172.21.15.94:0/1261638183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28103c90 0x7fef281040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 --2- 172.21.15.94:0/1261638183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28103750 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 -- 172.21.15.94:0/1261638183 >> 172.21.15.94:0/1261638183 conn(0x7fef280fdc40 msgr2=0x7fef281000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 -- 172.21.15.94:0/1261638183 shutdown_connections 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.880+0000 7fef2dae2640 1 -- 172.21.15.94:0/1261638183 wait complete. 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 Processor -- start 2024-04-25T13:51:38.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 -- start start 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28103c90 0x7fef2819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28108990 0x7fef281a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef2810e4e0 con 0x7fef28102a50 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef2810e360 con 0x7fef28103c90 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef2dae2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef2810e660 con 0x7fef28108990 2024-04-25T13:51:38.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41260/0 (socket says 172.21.15.94:41260) 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 -- 172.21.15.94:0/913091712 learned_addr learned my addr 172.21.15.94:0/913091712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef26ffd640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28103c90 0x7fef2819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 -- 172.21.15.94:0/913091712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28108990 msgr2=0x7fef281a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef27fff640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28108990 0x7fef281a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28108990 0x7fef281a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 -- 172.21.15.94:0/913091712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28103c90 msgr2=0x7fef2819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28103c90 0x7fef2819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.881+0000 7fef277fe640 1 -- 172.21.15.94:0/913091712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef281a1d50 con 0x7fef28102a50 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef277fe640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fef18006010 tx=0x7fef18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef24ff9640 1 -- 172.21.15.94:0/913091712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1803f030 con 0x7fef28102a50 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef24ff9640 1 -- 172.21.15.94:0/913091712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef18003aa0 con 0x7fef28102a50 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef281a1fe0 con 0x7fef28102a50 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef24ff9640 1 -- 172.21.15.94:0/913091712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18035070 con 0x7fef28102a50 2024-04-25T13:51:38.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.882+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a23e0 con 0x7fef28102a50 2024-04-25T13:51:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.884+0000 7fef24ff9640 1 -- 172.21.15.94:0/913091712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fef18042030 con 0x7fef28102a50 2024-04-25T13:51:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 Processor -- start 2024-04-25T13:51:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 -- start start 2024-04-25T13:51:38.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef24ff9640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 0x7fef100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef27fff640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 0x7fef100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92014d0 0x5610a95d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92096b0 0x5610a95d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5610a9530d80 con 0x5610a922a390 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef24ff9640 1 -- 172.21.15.94:0/913091712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fef180ff050 con 0x7fef28102a50 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5610a94eaa20 con 0x5610a92096b0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef3498d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610a952cd40 con 0x5610a92014d0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41276/0 (socket says 172.21.15.94:41276) 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef277fe640 1 -- 172.21.15.94:0/3288486099 learned_addr learned my addr 172.21.15.94:0/3288486099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef27fff640 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92096b0 0x5610a95d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.885+0000 7fef26ffd640 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92014d0 0x5610a95d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 -- 172.21.15.94:0/3288486099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92014d0 msgr2=0x5610a95d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92014d0 0x5610a95d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 -- 172.21.15.94:0/3288486099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92096b0 msgr2=0x5610a95d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92096b0 0x5610a95d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 -- 172.21.15.94:0/3288486099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a94f5c30 con 0x5610a922a390 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef27fff640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 0x7fef100b2000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef20005fd0 tx=0x7fef20013ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef277fe640 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fef18005f70 tx=0x7fef18003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a47b9a22cef31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef0dffb640 1 -- 172.21.15.94:0/3288486099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18049040 con 0x5610a922a390 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef0dffb640 1 -- 172.21.15.94:0/3288486099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef180c7530 con 0x5610a922a390 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.886+0000 7fef0dffb640 1 -- 172.21.15.94:0/3288486099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180c7790 con 0x5610a922a390 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- 172.21.15.94:0/3288486099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 msgr2=0x5610a95d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fef18005f70 tx=0x7fef18003f90 comp rx=0 tx=0).stop 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- 172.21.15.94:0/3288486099 shutdown_connections 2024-04-25T13:51:38.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92096b0 0x5610a95d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92014d0 0x5610a95d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 --2- 172.21.15.94:0/3288486099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95d0c20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- 172.21.15.94:0/3288486099 >> 172.21.15.94:0/3288486099 conn(0x5610a921e4e0 msgr2=0x5610a95c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- 172.21.15.94:0/3288486099 shutdown_connections 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- 172.21.15.94:0/3288486099 wait complete. 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 -- start start 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.887+0000 7fef3498d740 1 Processor -- start 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5610a9530d80 con 0x5610a92014d0 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5610a94b7dc0 con 0x5610a922a390 2024-04-25T13:51:38.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610a95d2700 con 0x5610a92096b0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 10 client.0 _ll_register_callbacks cb 0x5610a94d3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41278/0 (socket says 172.21.15.94:41278) 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 -- 172.21.15.94:0/1607302151 learned_addr learned my addr 172.21.15.94:0/1607302151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef26ffd640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef27fff640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 -- 172.21.15.94:0/1607302151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 msgr2=0x5610a95dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 -- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 msgr2=0x5610a95e1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95e1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 -- 172.21.15.94:0/1607302151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a92423f0 con 0x5610a92014d0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef277fe640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fef180c71e0 tx=0x7fef18031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f586602af81b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18049070 con 0x5610a92014d0 2024-04-25T13:51:38.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fef180c3b20 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.888+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180c7530 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef180d0390 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 10 client.19038 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610a930c7d0 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef180c7530 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef180e8310 con 0x5610a92014d0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 10 client.19038 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 10 client.19038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 20 client.19038 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91e8f60 0x5610a95e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 4 client.19038 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a953e990 con 0x5610a91e8f60 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.889+0000 7fef26ffd640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91e8f60 0x5610a95e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef06ffd640 10 client.19038 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef26ffd640 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91e8f60 0x5610a95e7f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5df4a7195a3bc0d8 server_cookie=d55caf111a63b942 in_seq=0 out_seq=0 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef06ffd640 1 -- 172.21.15.94:0/1607302151 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610a953e990 con 0x5610a91e8f60 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef06ffd640 10 client.19038 handle_command_reply: tid=0 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 client.19038 shutdown 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 msgr2=0x5610a95da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fef180c71e0 tx=0x7fef18031400 comp rx=0 tx=0).stop 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 shutdown_connections 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91e8f60 0x5610a95e7f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95e1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef3498d740 1 --2- 172.21.15.94:0/1607302151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95da620 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.890+0000 7fef06ffd640 0 client.19038 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.891+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 >> 172.21.15.94:0/1607302151 conn(0x5610a921e4e0 msgr2=0x5610a95caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.891+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 shutdown_connections 2024-04-25T13:51:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.891+0000 7fef3498d740 1 -- 172.21.15.94:0/1607302151 wait complete. 2024-04-25T13:51:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.891+0000 7fef3498d740 20 client.19038 trim_cache size 0 max 16384 2024-04-25T13:51:38.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 Processor -- start 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 -- start start 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef281a9350 con 0x5610a92014d0 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef281aae90 con 0x5610a922a390 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef3498d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef281a9a20 con 0x5610a92096b0 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41294/0 (socket says 172.21.15.94:41294) 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef277fe640 1 -- 172.21.15.94:0/2394598299 learned_addr learned my addr 172.21.15.94:0/2394598299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef27fff640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.905+0000 7fef26ffd640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 -- 172.21.15.94:0/2394598299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 msgr2=0x5610a95c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 -- 172.21.15.94:0/2394598299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 msgr2=0x5610a95da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 -- 172.21.15.94:0/2394598299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a95d5f30 con 0x5610a92014d0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef26ffd640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef277fe640 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fef180e8730 tx=0x7fef1803b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350d831a3fff2ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef0d7fa640 1 -- 172.21.15.94:0/2394598299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18049030 con 0x5610a92014d0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef0d7fa640 1 -- 172.21.15.94:0/2394598299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef180c7530 con 0x5610a92014d0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.906+0000 7fef0d7fa640 1 -- 172.21.15.94:0/2394598299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180e87e0 con 0x5610a92014d0 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- 172.21.15.94:0/2394598299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 msgr2=0x5610a95c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fef180e8730 tx=0x7fef1803b6b0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- 172.21.15.94:0/2394598299 shutdown_connections 2024-04-25T13:51:38.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a922a390 0x5610a95da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 --2- 172.21.15.94:0/2394598299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a92014d0 0x5610a95c7a50 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- 172.21.15.94:0/2394598299 >> 172.21.15.94:0/2394598299 conn(0x5610a921e4e0 msgr2=0x5610a95d8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- 172.21.15.94:0/2394598299 shutdown_connections 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- 172.21.15.94:0/2394598299 wait complete. 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.907+0000 7fef3498d740 1 -- start start 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 Processor -- start 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 0x5610a95de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef281aa7c0 con 0x5610a922a390 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef281aa0f0 con 0x5610a92014d0 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef281a7330 con 0x5610a92096b0 2024-04-25T13:51:38.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef3498d740 10 client.0 _ll_register_callbacks cb 0x7fef200098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef277fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 0x5610a95de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41296/0 (socket says 172.21.15.94:41296) 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef277fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 0x5610a95de300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43810/0 (socket says 172.21.15.94:43810) 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 -- 172.21.15.94:0/4019676789 learned_addr learned my addr 172.21.15.94:0/4019676789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef26ffd640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 -- 172.21.15.94:0/4019676789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 msgr2=0x5610a95de840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95de840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 -- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 msgr2=0x5610a95de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 0x5610a95de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 -- 172.21.15.94:0/4019676789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a95dfd30 con 0x5610a922a390 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef27fff640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fef20013450 tx=0x7fef20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e980991bcd891619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.908+0000 7fef06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20004b20 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180c7530 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fef20004cc0 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef20020a60 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 10 client.19044 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610a95e1070 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef180c7530 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef20028070 con 0x5610a922a390 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 10 client.19044 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 10 client.19044 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 20 client.19044 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91be3a0 0x5610a9652750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 4 client.19044 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:38.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.909+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610a9538800 con 0x5610a91be3a0 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef277fe640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91be3a0 0x5610a9652750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef06ffd640 10 client.19044 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef277fe640 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91be3a0 0x5610a9652750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9476a475a0760e4b server_cookie=271bbf8d810b72d1 in_seq=0 out_seq=0 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef06ffd640 1 -- 172.21.15.94:0/4019676789 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a9538800 con 0x5610a91be3a0 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef06ffd640 10 client.19044 handle_command_reply: tid=0 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 client.19044 shutdown 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 msgr2=0x5610a95df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fef20013450 tx=0x7fef20019000 comp rx=0 tx=0).stop 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 shutdown_connections 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5610a91be3a0 0x5610a9652750 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610a922a390 0x5610a95df770 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610a92096b0 0x5610a95de840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef06ffd640 0 client.19044 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 --2- 172.21.15.94:0/4019676789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610a92014d0 0x5610a95de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 >> 172.21.15.94:0/4019676789 conn(0x5610a921e4e0 msgr2=0x5610a95da1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 shutdown_connections 2024-04-25T13:51:38.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.910+0000 7fef3498d740 1 -- 172.21.15.94:0/4019676789 wait complete. 2024-04-25T13:51:38.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef3498d740 20 client.19044 trim_cache size 0 max 16384 2024-04-25T13:51:38.917 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 msgr2=0x7fef100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 0x7fef100b2000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef20005fd0 tx=0x7fef20013ac0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 msgr2=0x7fef28075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fef18006010 tx=0x7fef18004270 comp rx=0 tx=0).stop 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 shutdown_connections 2024-04-25T13:51:38.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef100afaf0 0x7fef100b2000 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef28108990 0x7fef281a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef28103c90 0x7fef2819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 --2- 172.21.15.94:0/913091712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef28102a50 0x7fef28075840 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 >> 172.21.15.94:0/913091712 conn(0x7fef280fdc40 msgr2=0x7fef28104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 shutdown_connections 2024-04-25T13:51:38.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:38.911+0000 7fef2dae2640 1 -- 172.21.15.94:0/913091712 wait complete. 2024-04-25T13:51:38.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.079+0000 7f5cf6a51640 1 -- 172.21.15.94:0/746332134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01080c0 msgr2=0x7f5cf010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.079+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/746332134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01080c0 0x7f5cf010a4e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800adb0 tx=0x7f5ce802b7a0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 -- 172.21.15.94:0/746332134 shutdown_connections 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/746332134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01080c0 0x7f5cf010a4e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/746332134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf01033c0 0x7f5cf01037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/746332134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf0102180 0x7f5cf0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 -- 172.21.15.94:0/746332134 >> 172.21.15.94:0/746332134 conn(0x7f5cf00fd350 msgr2=0x7f5cf00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 -- 172.21.15.94:0/746332134 shutdown_connections 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 -- 172.21.15.94:0/746332134 wait complete. 2024-04-25T13:51:39.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 Processor -- start 2024-04-25T13:51:39.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.080+0000 7f5cf6a51640 1 -- start start 2024-04-25T13:51:39.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 0x7f5cf019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01080c0 0x7f5cf01a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d410 con 0x7f5cf01033c0 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d290 con 0x7f5cf0102180 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d590 con 0x7f5cf01080c0 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41324/0 (socket says 172.21.15.94:41324) 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2666094257 learned_addr learned my addr 172.21.15.94:0/2666094257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01080c0 0x7f5cf01a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 0x7f5cf019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2666094257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01080c0 msgr2=0x7f5cf01a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01080c0 0x7f5cf01a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2666094257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 msgr2=0x7f5cf019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 0x7f5cf019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2666094257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf01a1ce0 con 0x7f5cf01033c0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 0x7f5cf019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400e9f0 tx=0x7f5ce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5ced7fa640 1 -- 172.21.15.94:0/2666094257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400cd80 con 0x7f5cf01033c0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5ced7fa640 1 -- 172.21.15.94:0/2666094257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce40108e0 con 0x7f5cf01033c0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01a1fd0 con 0x7f5cf01033c0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5ced7fa640 1 -- 172.21.15.94:0/2666094257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4018980 con 0x7f5cf01033c0 2024-04-25T13:51:39.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.081+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a2490 con 0x7f5cf01033c0 2024-04-25T13:51:39.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 Processor -- start 2024-04-25T13:51:39.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5ced7fa640 1 -- 172.21.15.94:0/2666094257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5ce4010430 con 0x7f5cf01033c0 2024-04-25T13:51:39.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 -- start start 2024-04-25T13:51:39.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c27810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c27d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a266b8bd80 con 0x55a2668646b0 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a266b45a20 con 0x55a266885390 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a266b87d40 con 0x55a26685c4d0 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cf4fee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c27810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41332/0 (socket says 172.21.15.94:41332) 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2533631499 learned_addr learned my addr 172.21.15.94:0/2533631499 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5ceffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c27d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2533631499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 msgr2=0x55a266c27d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c27d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2533631499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 msgr2=0x55a266c27810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c27810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2533631499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a266b50c30 con 0x55a2668646b0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.084+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c27810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5ced7fa640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 0x7f5cd80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40e71b0 tx=0x7f5ce40e7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c46b62b5bc78c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 0x7f5cd80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5ced7fa640 1 -- 172.21.15.94:0/2666094257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce40defc0 con 0x7f5cf01033c0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5cd67fc640 1 -- 172.21.15.94:0/2533631499 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40a32c0 con 0x55a2668646b0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5cd67fc640 1 -- 172.21.15.94:0/2533631499 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce4010720 con 0x55a2668646b0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.085+0000 7f5cd67fc640 1 -- 172.21.15.94:0/2533631499 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40a6b70 con 0x55a2668646b0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2533631499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 msgr2=0x55a266c28a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 0x7f5cd80b2080 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005cc0 tx=0x7f5ce8005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40e71b0 tx=0x7f5ce40e7670 comp rx=0 tx=0).stop 2024-04-25T13:51:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2533631499 shutdown_connections 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c28a20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c27d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2533631499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c27810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2533631499 >> 172.21.15.94:0/2533631499 conn(0x55a2668794e0 msgr2=0x55a266c23020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2533631499 shutdown_connections 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2533631499 wait complete. 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 -- start start 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.086+0000 7f5cfd9e7740 1 Processor -- start 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a266c29530 con 0x55a2668646b0 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a266b12dc0 con 0x55a266885390 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a266c296b0 con 0x55a26685c4d0 2024-04-25T13:51:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cfd9e7740 10 client.0 _ll_register_callbacks cb 0x55a266b2e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5ceffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cf4fee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41342/0 (socket says 172.21.15.94:41342) 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5ceffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43456/0 (socket says 172.21.15.94:43456) 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 -- 172.21.15.94:0/1843752245 learned_addr learned my addr 172.21.15.94:0/1843752245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 -- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 msgr2=0x55a266c31570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 -- 172.21.15.94:0/1843752245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 msgr2=0x55a266c38a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c38a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 -- 172.21.15.94:0/1843752245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26689d3f0 con 0x55a2668646b0 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5ceffff640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40a7100 tx=0x7f5ce400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df2d31f6a3a85fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5ccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.087+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40a6d50 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce4010a90 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a266b99990 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ce40a71d0 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 10 client.19056 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2669677d0 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a266b99990 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce40e2180 con 0x55a2668646b0 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 10 client.19056 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 10 client.19056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 20 client.19056 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a266843f60 0x55a266c3ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 4 client.19056 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a266b93800 con 0x55a266843f60 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.088+0000 7f5ceffff640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a266843f60 0x55a266c3ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.089+0000 7f5ccf7fe640 10 client.19056 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.089+0000 7f5ceffff640 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a266843f60 0x55a266c3ee60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3fdc36fc25f4222 server_cookie=49016d53555c6479 in_seq=0 out_seq=0 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.089+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/1843752245 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a266b93800 con 0x55a266843f60 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.089+0000 7f5ccf7fe640 10 client.19056 handle_command_reply: tid=0 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.089+0000 7f5cfd9e7740 1 client.19056 shutdown 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 msgr2=0x55a266c31ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40a7100 tx=0x7f5ce400cf20 comp rx=0 tx=0).stop 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 shutdown_connections 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a266843f60 0x55a266c3ee60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266c38a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266c31ab0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1843752245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266c31570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5ccf7fe640 0 client.19056 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 >> 172.21.15.94:0/1843752245 conn(0x55a2668794e0 msgr2=0x55a266c25f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 shutdown_connections 2024-04-25T13:51:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1843752245 wait complete. 2024-04-25T13:51:39.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.090+0000 7f5cfd9e7740 20 client.19056 trim_cache size 0 max 16384 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 Processor -- start 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 -- start start 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a26685c4d0 0x55a266c22a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 0x55a266c22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01ab600 con 0x55a266885390 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01ad140 con 0x55a26685c4d0 2024-04-25T13:51:39.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01abcd0 con 0x55a2668646b0 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cf4fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cf4fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41350/0 (socket says 172.21.15.94:41350) 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 0x55a266c22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5cf4fee640 1 -- 172.21.15.94:0/1759610097 learned_addr learned my addr 172.21.15.94:0/1759610097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.104+0000 7f5ceffff640 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a26685c4d0 0x55a266c22a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 -- 172.21.15.94:0/1759610097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 msgr2=0x55a266c22f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 0x55a266c22f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 -- 172.21.15.94:0/1759610097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a26685c4d0 msgr2=0x55a266c22a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a26685c4d0 0x55a266c22a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 -- 172.21.15.94:0/1759610097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a266c27240 con 0x55a266885390 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 0x55a266c22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8035000 tx=0x7f5ce8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e3001e1eaebe4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cd5ffb640 1 -- 172.21.15.94:0/1759610097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8058070 con 0x55a266885390 2024-04-25T13:51:39.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cd5ffb640 1 -- 172.21.15.94:0/1759610097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8003e20 con 0x55a266885390 2024-04-25T13:51:39.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.105+0000 7f5cd5ffb640 1 -- 172.21.15.94:0/1759610097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8031610 con 0x55a266885390 2024-04-25T13:51:39.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1759610097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 msgr2=0x55a266c21610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8035000 tx=0x7f5ce8042000 comp rx=0 tx=0).stop 2024-04-25T13:51:39.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1759610097 shutdown_connections 2024-04-25T13:51:39.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a266885390 0x55a266c21610 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2668646b0 0x55a266c22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/1759610097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a26685c4d0 0x55a266c22a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1759610097 >> 172.21.15.94:0/1759610097 conn(0x55a2668794e0 msgr2=0x55a266c2c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1759610097 shutdown_connections 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/1759610097 wait complete. 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 -- start start 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.106+0000 7f5cfd9e7740 1 Processor -- start 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266ca1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266ca8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01aca70 con 0x55a2668646b0 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01ac3a0 con 0x55a266885390 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a95e0 con 0x55a26685c4d0 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 10 client.0 _ll_register_callbacks cb 0x7f5ce800ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41362/0 (socket says 172.21.15.94:41362) 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2034179927 learned_addr learned my addr 172.21.15.94:0/2034179927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cf4fee640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266ca8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266ca1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 msgr2=0x55a266ca1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266ca1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2034179927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 msgr2=0x55a266ca8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266ca8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 -- 172.21.15.94:0/2034179927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a266ca9540 con 0x55a2668646b0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266ca1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cef7fe640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40e71b0 tx=0x7f5ce4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4994245415aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5ccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40a32c0 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce40e2180 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.107+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8003e20 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ce40a6b70 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 10 client.19062 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a266caa7c0 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8003e20 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce40afe20 con 0x55a2668646b0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 10 client.19062 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 10 client.19062 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 20 client.19062 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2668193a0 0x55a266cacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 4 client.19062 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a266b0e610 con 0x55a2668193a0 2024-04-25T13:51:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.108+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2668193a0 0x55a266cacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5ceffff640 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2668193a0 0x55a266cacfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=914e582a97c59c60 server_cookie=16979dac9d833230 in_seq=0 out_seq=0 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5ccf7fe640 10 client.19062 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5ccf7fe640 1 -- 172.21.15.94:0/2034179927 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a266b0e610 con 0x55a2668193a0 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5ccf7fe640 10 client.19062 handle_command_reply: tid=0 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 client.19062 shutdown 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 msgr2=0x55a266ca2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40e71b0 tx=0x7f5ce4018420 comp rx=0 tx=0).stop 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 shutdown_connections 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a2668193a0 0x55a266cacfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a266885390 0x55a266ca8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5ccf7fe640 0 client.19062 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2668646b0 0x55a266ca2020 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 --2- 172.21.15.94:0/2034179927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a26685c4d0 0x55a266ca1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 >> 172.21.15.94:0/2034179927 conn(0x55a2668794e0 msgr2=0x55a266c25e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 shutdown_connections 2024-04-25T13:51:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 1 -- 172.21.15.94:0/2034179927 wait complete. 2024-04-25T13:51:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.109+0000 7f5cfd9e7740 20 client.19062 trim_cache size 0 max 16384 2024-04-25T13:51:39.116 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 msgr2=0x7f5cd80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 0x7f5cd80b2080 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005cc0 tx=0x7f5ce8005c30 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 msgr2=0x7f5cf019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400e9f0 tx=0x7f5ce400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 shutdown_connections 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afb70 0x7f5cd80b2080 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01080c0 0x7f5cf01a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf01033c0 0x7f5cf019a6e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 --2- 172.21.15.94:0/2666094257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102180 0x7f5cf019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 >> 172.21.15.94:0/2666094257 conn(0x7f5cf00fd350 msgr2=0x7f5cf01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 shutdown_connections 2024-04-25T13:51:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.110+0000 7f5cf6a51640 1 -- 172.21.15.94:0/2666094257 wait complete. 2024-04-25T13:51:39.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:51:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 -- 172.21.15.94:0/852735690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de4101f00 msgr2=0x7f3de4106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 --2- 172.21.15.94:0/852735690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de4101f00 0x7f3de4106b70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009ca0 tx=0x7f3ddc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 -- 172.21.15.94:0/852735690 shutdown_connections 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 --2- 172.21.15.94:0/852735690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 0x7f3de410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 --2- 172.21.15.94:0/852735690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de41070b0 0x7f3de41094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 --2- 172.21.15.94:0/852735690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de4101f00 0x7f3de4106b70 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 -- 172.21.15.94:0/852735690 >> 172.21.15.94:0/852735690 conn(0x7f3de40fd340 msgr2=0x7f3de40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 -- 172.21.15.94:0/852735690 shutdown_connections 2024-04-25T13:51:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 -- 172.21.15.94:0/852735690 wait complete. 2024-04-25T13:51:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.237+0000 7f3dea205640 1 Processor -- start 2024-04-25T13:51:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 -- start start 2024-04-25T13:51:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de4101f00 0x7f3de419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 0x7f3de41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410f680 con 0x7f3de41070b0 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410f500 con 0x7f3de4109a10 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3dea205640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3de410f800 con 0x7f3de4101f00 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41392/0 (socket says 172.21.15.94:41392) 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 -- 172.21.15.94:0/3558090516 learned_addr learned my addr 172.21.15.94:0/3558090516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 0x7f3de41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de4101f00 0x7f3de419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 -- 172.21.15.94:0/3558090516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de4101f00 msgr2=0x7f3de419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de4101f00 0x7f3de419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 -- 172.21.15.94:0/3558090516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 msgr2=0x7f3de41a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 0x7f3de41a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 -- 172.21.15.94:0/3558090516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de41a1df0 con 0x7f3de41070b0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.238+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00e9f0 tx=0x7f3dcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.239+0000 7f3de0ff9640 1 -- 172.21.15.94:0/3558090516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00cd80 con 0x7f3de41070b0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.239+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41a20e0 con 0x7f3de41070b0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.239+0000 7f3de0ff9640 1 -- 172.21.15.94:0/3558090516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dcc0054d0 con 0x7f3de41070b0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.239+0000 7f3de0ff9640 1 -- 172.21.15.94:0/3558090516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc010690 con 0x7f3de41070b0 2024-04-25T13:51:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.239+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a2630 con 0x7f3de41070b0 2024-04-25T13:51:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.241+0000 7f3df1190740 1 Processor -- start 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.241+0000 7f3de0ff9640 1 -- 172.21.15.94:0/3558090516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3dcc004e90 con 0x7f3de41070b0 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.241+0000 7f3df1190740 1 -- start start 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 0x5555e74b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70ed6b0 0x5555e74b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555e7414d80 con 0x5555e70e54d0 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555e73cea20 con 0x5555e710e390 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3df1190740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555e7410d40 con 0x5555e70ed6b0 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 0x5555e74b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41394/0 (socket says 172.21.15.94:41394) 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 -- 172.21.15.94:0/3658513373 learned_addr learned my addr 172.21.15.94:0/3658513373 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70ed6b0 0x5555e74b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 -- 172.21.15.94:0/3658513373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70ed6b0 msgr2=0x5555e74b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70ed6b0 0x5555e74b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 -- 172.21.15.94:0/3658513373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 msgr2=0x5555e74b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 0x5555e74b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 -- 172.21.15.94:0/3658513373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e73d9c30 con 0x5555e70e54d0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 0x5555e74b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de0ff9640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 0x7f3dd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc035000 tx=0x7f3ddc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8a824b5eb8ab1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 0x7f3dd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3de0ff9640 1 -- 172.21.15.94:0/3558090516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f3dcc0df0d0 con 0x7f3de41070b0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.242+0000 7f3dd9ffb640 1 -- 172.21.15.94:0/3658513373 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc031650 con 0x5555e70e54d0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3dd9ffb640 1 -- 172.21.15.94:0/3658513373 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc036030 con 0x5555e70e54d0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3dd9ffb640 1 -- 172.21.15.94:0/3658513373 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004620 con 0x5555e70e54d0 2024-04-25T13:51:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 -- 172.21.15.94:0/3658513373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 msgr2=0x5555e74b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 0x7f3dd00b2000 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5555e74b1860 tx=0x7f3dd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc035000 tx=0x7f3ddc004ab0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 -- 172.21.15.94:0/3658513373 shutdown_connections 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70ed6b0 0x5555e74b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b0d90 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 --2- 172.21.15.94:0/3658513373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e710e390 0x5555e74b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 -- 172.21.15.94:0/3658513373 >> 172.21.15.94:0/3658513373 conn(0x5555e71024e0 msgr2=0x5555e74ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 -- 172.21.15.94:0/3658513373 shutdown_connections 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.243+0000 7f3df1190740 1 -- 172.21.15.94:0/3658513373 wait complete. 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 -- start start 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 Processor -- start 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 0x5555e74ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555e74b2350 con 0x5555e710e390 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555e739bdc0 con 0x5555e70ed6b0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555e74b24d0 con 0x5555e70e54d0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3df1190740 10 client.0 _ll_register_callbacks cb 0x5555e73b7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3de3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3de3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41400/0 (socket says 172.21.15.94:41400) 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3de3fff640 1 -- 172.21.15.94:0/3540866030 learned_addr learned my addr 172.21.15.94:0/3540866030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 0x5555e74ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.244+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 -- 172.21.15.94:0/3540866030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 msgr2=0x5555e74ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 0x5555e74ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 -- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 msgr2=0x5555e74ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 -- 172.21.15.94:0/3540866030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e71263f0 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 0x5555e74ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005880 tx=0x7f3dd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefc26501c81ca66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3dc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4009be0 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f3dd4018030 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc036030 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3dd4004020 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3df1190740 10 client.19074 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555e71f07d0 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.245+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc036030 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3dd4033430 con 0x5555e710e390 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 10 client.19074 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 10 client.19074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:51:39.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 20 client.19074 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70ccf60 0x5555e74c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 4 client.19074 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555e7422990 con 0x5555e70ccf60 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70ccf60 0x5555e74c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70ccf60 0x5555e74c8110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=861ef9ed1edadaa0 server_cookie=be21a5d3ad983b80 in_seq=0 out_seq=0 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.246+0000 7f3dc2ffd640 10 client.19074 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3540866030 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555e7422990 con 0x5555e70ccf60 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3dc2ffd640 10 client.19074 handle_command_reply: tid=0 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 client.19074 shutdown 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 msgr2=0x5555e74c17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005880 tx=0x7f3dd4004730 comp rx=0 tx=0).stop 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 shutdown_connections 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70ccf60 0x5555e74c8110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e710e390 0x5555e74c17b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 --2- 172.21.15.94:0/3540866030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e70e54d0 0x5555e74ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 >> 172.21.15.94:0/3540866030 conn(0x5555e71024e0 msgr2=0x5555e74aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3dc2ffd640 0 client.19074 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 shutdown_connections 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.247+0000 7f3df1190740 1 -- 172.21.15.94:0/3540866030 wait complete. 2024-04-25T13:51:39.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.248+0000 7f3df1190740 20 client.19074 trim_cache size 0 max 16384 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 Processor -- start 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 -- start start 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ab4e0 con 0x5555e70e54d0 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ad020 con 0x5555e70ed6b0 2024-04-25T13:51:39.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3df1190740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41abbb0 con 0x5555e710e390 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41406/0 (socket says 172.21.15.94:41406) 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3de37fe640 1 -- 172.21.15.94:0/1962727602 learned_addr learned my addr 172.21.15.94:0/1962727602 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.261+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de3fff640 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 -- 172.21.15.94:0/1962727602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 msgr2=0x5555e74b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 -- 172.21.15.94:0/1962727602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 msgr2=0x5555e74b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 -- 172.21.15.94:0/1962727602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e70cd190 con 0x5555e70e54d0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3de37fe640 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x5555e74b0530 tx=0x7f3ddc02f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300b064187640803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3dd97fa640 1 -- 172.21.15.94:0/1962727602 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc04a040 con 0x5555e70e54d0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3dd97fa640 1 -- 172.21.15.94:0/1962727602 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc036040 con 0x5555e70e54d0 2024-04-25T13:51:39.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.262+0000 7f3dd97fa640 1 -- 172.21.15.94:0/1962727602 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0552b0 con 0x5555e70e54d0 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- 172.21.15.94:0/1962727602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 msgr2=0x5555e74b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x5555e74b0530 tx=0x7f3ddc02f4c0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- 172.21.15.94:0/1962727602 shutdown_connections 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 --2- 172.21.15.94:0/1962727602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74b51f0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- 172.21.15.94:0/1962727602 >> 172.21.15.94:0/1962727602 conn(0x5555e71024e0 msgr2=0x5555e74ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- 172.21.15.94:0/1962727602 shutdown_connections 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- 172.21.15.94:0/1962727602 wait complete. 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.263+0000 7f3df1190740 1 -- start start 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 Processor -- start 2024-04-25T13:51:39.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac950 con 0x5555e70e54d0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41ac280 con 0x5555e70ed6b0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3de41a94c0 con 0x5555e710e390 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3df1190740 10 client.0 _ll_register_callbacks cb 0x7f3dd4012f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41422/0 (socket says 172.21.15.94:41422) 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 -- 172.21.15.94:0/3246609660 learned_addr learned my addr 172.21.15.94:0/3246609660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de3fff640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 -- 172.21.15.94:0/3246609660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 msgr2=0x5555e74bdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74bdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 -- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 msgr2=0x5555e74bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74bd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.264+0000 7f3de37fe640 1 -- 172.21.15.94:0/3246609660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e74be2c0 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3de37fe640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005e30 tx=0x7f3ddc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f426c26068c5ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004120 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f3ddc0045f0 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ddc036040 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ddc0047f0 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 10 client.19080 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555e752f500 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ddc036040 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ddc043070 con 0x5555e70e54d0 2024-04-25T13:51:39.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 10 client.19080 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 10 client.19080 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 20 client.19080 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70a23a0 0x5555e7531c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 4 client.19080 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555e741c800 con 0x5555e70a23a0 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.265+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70a23a0 0x5555e7531c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3dc2ffd640 10 client.19080 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3de2ffd640 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70a23a0 0x5555e7531c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16ec038ae6098058 server_cookie=6304c552905bef29 in_seq=0 out_seq=0 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3dc2ffd640 1 -- 172.21.15.94:0/3246609660 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555e741c800 con 0x5555e70a23a0 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3dc2ffd640 10 client.19080 handle_command_reply: tid=0 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 client.19080 shutdown 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 msgr2=0x5555e74c0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc005e30 tx=0x7f3ddc005fd0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 shutdown_connections 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555e70a23a0 0x5555e7531c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555e710e390 0x5555e74bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e70ed6b0 0x5555e74bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 --2- 172.21.15.94:0/3246609660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555e70e54d0 0x5555e74c0b90 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3dc2ffd640 0 client.19080 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 >> 172.21.15.94:0/3246609660 conn(0x5555e71024e0 msgr2=0x5555e74b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 shutdown_connections 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 1 -- 172.21.15.94:0/3246609660 wait complete. 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.266+0000 7f3df1190740 20 client.19080 trim_cache size 0 max 16384 2024-04-25T13:51:39.272 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:39.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 msgr2=0x7f3dd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 0x7f3dd00b2000 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5555e74b1860 tx=0x7f3dd4005f90 comp rx=0 tx=0).stop 2024-04-25T13:51:39.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 msgr2=0x7f3de419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00e9f0 tx=0x7f3dcc00edf0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 shutdown_connections 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd00afaf0 0x7f3dd00b2000 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de4109a10 0x7f3de41a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de41070b0 0x7f3de419a7f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 --2- 172.21.15.94:0/3558090516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3de4101f00 0x7f3de419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 >> 172.21.15.94:0/3558090516 conn(0x7f3de40fd340 msgr2=0x7f3de4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 shutdown_connections 2024-04-25T13:51:39.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.267+0000 7f3dea205640 1 -- 172.21.15.94:0/3558090516 wait complete. 2024-04-25T13:51:39.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:51:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 -- 172.21.15.94:0/1730165776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88103960 msgr2=0x7f1c88103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1730165776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88103960 0x7f1c88103dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009900 tx=0x7f1c7c02a410 comp rx=0 tx=0).stop 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 -- 172.21.15.94:0/1730165776 shutdown_connections 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1730165776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88104460 0x7f1c8810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1730165776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88103960 0x7f1c88103dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1730165776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88102f80 0x7f1c88103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 -- 172.21.15.94:0/1730165776 >> 172.21.15.94:0/1730165776 conn(0x7f1c880fd340 msgr2=0x7f1c880ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 -- 172.21.15.94:0/1730165776 shutdown_connections 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.396+0000 7f1c8e467640 1 -- 172.21.15.94:0/1730165776 wait complete. 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 Processor -- start 2024-04-25T13:51:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 -- start start 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88103960 0x7f1c8819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 0x7f1c881a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810e500 con 0x7f1c88102f80 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810e380 con 0x7f1c88103960 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8e467640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8810e680 con 0x7f1c88104460 2024-04-25T13:51:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41438/0 (socket says 172.21.15.94:41438) 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c87fff640 1 -- 172.21.15.94:0/1723206384 learned_addr learned my addr 172.21.15.94:0/1723206384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 0x7f1c881a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.397+0000 7f1c877fe640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88103960 0x7f1c8819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 -- 172.21.15.94:0/1723206384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 msgr2=0x7f1c881a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 0x7f1c881a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 -- 172.21.15.94:0/1723206384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88103960 msgr2=0x7f1c8819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88103960 0x7f1c8819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 -- 172.21.15.94:0/1723206384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c881a1d50 con 0x7f1c88102f80 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 0x7f1c881a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9f0 tx=0x7f1c7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c857fa640 1 -- 172.21.15.94:0/1723206384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800ccc0 con 0x7f1c88102f80 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c881a1fe0 con 0x7f1c88102f80 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c857fa640 1 -- 172.21.15.94:0/1723206384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c780035f0 con 0x7f1c88102f80 2024-04-25T13:51:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c857fa640 1 -- 172.21.15.94:0/1723206384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800ce60 con 0x7f1c88102f80 2024-04-25T13:51:39.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.398+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a24f0 con 0x7f1c88102f80 2024-04-25T13:51:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c857fa640 1 -- 172.21.15.94:0/1723206384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c78010690 con 0x7f1c88102f80 2024-04-25T13:51:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 Processor -- start 2024-04-25T13:51:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 -- start start 2024-04-25T13:51:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a719100d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a719101a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a719064d80 con 0x55a718d5e390 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a71901ea20 con 0x55a718d3d6b0 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c953f8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a719060d40 con 0x55a718d354d0 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a719100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41452/0 (socket says 172.21.15.94:41452) 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2231596307 learned_addr learned my addr 172.21.15.94:0/2231596307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c877fe640 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a719101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c857fa640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 0x7f1c700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.401+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2231596307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 msgr2=0x55a719100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a719100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2231596307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 msgr2=0x55a719101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a719101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2231596307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a719029c30 con 0x55a718d5e390 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a719100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c857fa640 1 -- 172.21.15.94:0/1723206384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f1c780defa0 con 0x7f1c88102f80 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 0x7f1c700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1c800117e0 tx=0x7f1c80011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a2444189d8cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c6e7fc640 1 -- 172.21.15.94:0/2231596307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000fb00 con 0x55a718d5e390 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c6e7fc640 1 -- 172.21.15.94:0/2231596307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c80010e00 con 0x55a718d5e390 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.402+0000 7f1c6e7fc640 1 -- 172.21.15.94:0/2231596307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001ca80 con 0x55a718d5e390 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- 172.21.15.94:0/2231596307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 msgr2=0x55a719100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 0x7f1c700b2000 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c80011490 tx=0x7f1c80021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1c800117e0 tx=0x7f1c80011ca0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- 172.21.15.94:0/2231596307 shutdown_connections 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a719101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a719100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2231596307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a719100810 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- 172.21.15.94:0/2231596307 >> 172.21.15.94:0/2231596307 conn(0x55a718d524e0 msgr2=0x55a7190fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- 172.21.15.94:0/2231596307 shutdown_connections 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- 172.21.15.94:0/2231596307 wait complete. 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.403+0000 7f1c953f8740 1 -- start start 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 Processor -- start 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 0x55a71910a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a71910aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a719102530 con 0x55a718d5e390 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a718febdc0 con 0x55a718d354d0 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7191026b0 con 0x55a718d3d6b0 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c953f8740 10 client.0 _ll_register_callbacks cb 0x55a719007600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41468/0 (socket says 172.21.15.94:41468) 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2107769750 learned_addr learned my addr 172.21.15.94:0/2107769750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 0x55a71910a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c877fe640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a71910aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2107769750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 msgr2=0x55a71910aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a71910aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 msgr2=0x55a71910a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 0x55a71910a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2107769750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a718d763f0 con 0x55a718d5e390 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.404+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 0x55a71910a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1c80011130 tx=0x7f1c8000fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a59065e2c0adea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800108d0 con 0x55a718d5e390 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f1c8002e070 con 0x55a718d5e390 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c80010e00 con 0x55a718d5e390 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c80010a70 con 0x55a718d5e390 2024-04-25T13:51:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 10 client.19092 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a718e407d0 con 0x55a718d5e390 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c80010e00 con 0x55a718d5e390 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1c800055a0 con 0x55a718d5e390 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 10 client.19092 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 10 client.19092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 20 client.19092 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718d1cf60 0x55a7191180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 4 client.19092 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.405+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a719072990 con 0x55a718d1cf60 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.406+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718d1cf60 0x55a7191180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.406+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718d1cf60 0x55a7191180e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18212b5ff2c90427 server_cookie=cc35dccf179322cb in_seq=0 out_seq=0 2024-04-25T13:51:39.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.406+0000 7f1c677fe640 10 client.19092 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c677fe640 1 -- 172.21.15.94:0/2107769750 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a719072990 con 0x55a718d1cf60 2024-04-25T13:51:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c677fe640 10 client.19092 handle_command_reply: tid=0 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 client.19092 shutdown 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 msgr2=0x55a7191119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1c80011130 tx=0x7f1c8000fa10 comp rx=0 tx=0).stop 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 shutdown_connections 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718d1cf60 0x55a7191180e0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a7191119e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a71910aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2107769750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d354d0 0x55a71910a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c677fe640 0 client.19092 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 >> 172.21.15.94:0/2107769750 conn(0x55a718d524e0 msgr2=0x55a7190fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 shutdown_connections 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 1 -- 172.21.15.94:0/2107769750 wait complete. 2024-04-25T13:51:39.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.408+0000 7f1c953f8740 20 client.19092 trim_cache size 0 max 16384 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 Processor -- start 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 -- start start 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a719105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 0x55a7191057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ab5c0 con 0x55a718d354d0 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ad100 con 0x55a718d5e390 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c953f8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881abc90 con 0x55a718d3d6b0 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41472/0 (socket says 172.21.15.94:41472) 2024-04-25T13:51:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.422+0000 7f1c87fff640 1 -- 172.21.15.94:0/1449520696 learned_addr learned my addr 172.21.15.94:0/1449520696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 0x55a7191057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c877fe640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a719105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 -- 172.21.15.94:0/1449520696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 msgr2=0x55a719105270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a719105270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 -- 172.21.15.94:0/1449520696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 msgr2=0x55a7191057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 0x55a7191057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 -- 172.21.15.94:0/1449520696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a719108630 con 0x55a718d354d0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 0x55a7191057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c87fff640 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1c780a70e0 tx=0x7f1c780e2a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0659ca5fc8c02b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c6dffb640 1 -- 172.21.15.94:0/1449520696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780a71b0 con 0x55a718d354d0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c6dffb640 1 -- 172.21.15.94:0/1449520696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c780e28d0 con 0x55a718d354d0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.423+0000 7f1c6dffb640 1 -- 172.21.15.94:0/1449520696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780a6b50 con 0x55a718d354d0 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- 172.21.15.94:0/1449520696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 msgr2=0x55a7190f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1c780a70e0 tx=0x7f1c780e2a80 comp rx=0 tx=0).stop 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- 172.21.15.94:0/1449520696 shutdown_connections 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d5e390 0x55a7191057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d3d6b0 0x55a719105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 --2- 172.21.15.94:0/1449520696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d354d0 0x55a7190f6d30 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- 172.21.15.94:0/1449520696 >> 172.21.15.94:0/1449520696 conn(0x55a718d524e0 msgr2=0x55a719100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- 172.21.15.94:0/1449520696 shutdown_connections 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- 172.21.15.94:0/1449520696 wait complete. 2024-04-25T13:51:39.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.424+0000 7f1c953f8740 1 -- start start 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 Processor -- start 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a71910e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a71910eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881aca30 con 0x55a718d5e390 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881ac360 con 0x55a718d3d6b0 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881a95a0 con 0x55a718d354d0 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c953f8740 10 client.0 _ll_register_callbacks cb 0x7f1c8001f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a71910eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41480/0 (socket says 172.21.15.94:41480) 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2138500467 learned_addr learned my addr 172.21.15.94:0/2138500467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a71910e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2138500467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 msgr2=0x55a71910e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a71910e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 msgr2=0x55a71910eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a71910eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 -- 172.21.15.94:0/2138500467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71910fff0 con 0x55a718d5e390 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c8ca04640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005940 tx=0x7f1c80009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ebd549e4519210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.425+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800108d0 con 0x55a718d5e390 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f1c8002e040 con 0x55a718d5e390 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c780e28d0 con 0x55a718d5e390 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c80009e00 con 0x55a718d5e390 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 10 client.19098 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a719111270 con 0x55a718d5e390 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c780e28d0 con 0x55a718d5e390 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1c800445f0 con 0x55a718d5e390 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 10 client.19098 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 20 client.19098 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718cf23a0 0x55a719182b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 4 client.19098 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a71906c800 con 0x55a718cf23a0 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.426+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718cf23a0 0x55a719182b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c677fe640 10 client.19098 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c87fff640 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718cf23a0 0x55a719182b30 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39748ae83ec5def1 server_cookie=ffe00e109dfa9c84 in_seq=0 out_seq=0 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c677fe640 1 -- 172.21.15.94:0/2138500467 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a71906c800 con 0x55a718cf23a0 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c677fe640 10 client.19098 handle_command_reply: tid=0 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 client.19098 shutdown 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 msgr2=0x55a71910fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005940 tx=0x7f1c80009500 comp rx=0 tx=0).stop 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 shutdown_connections 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a718cf23a0 0x55a719182b30 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a718d5e390 0x55a71910fa30 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a718d3d6b0 0x55a71910eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c677fe640 0 client.19098 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 --2- 172.21.15.94:0/2138500467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a718d354d0 0x55a71910e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 >> 172.21.15.94:0/2138500467 conn(0x55a718d524e0 msgr2=0x55a719100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 shutdown_connections 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 1 -- 172.21.15.94:0/2138500467 wait complete. 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.427+0000 7f1c953f8740 20 client.19098 trim_cache size 0 max 16384 2024-04-25T13:51:39.433 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:51:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 msgr2=0x7f1c700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 0x7f1c700b2000 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c80011490 tx=0x7f1c80021000 comp rx=0 tx=0).stop 2024-04-25T13:51:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 msgr2=0x7f1c88102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9f0 tx=0x7f1c7800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 shutdown_connections 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c700afaf0 0x7f1c700b2000 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c88104460 0x7f1c881a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c88103960 0x7f1c8819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 --2- 172.21.15.94:0/1723206384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c88102f80 0x7f1c88102240 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 >> 172.21.15.94:0/1723206384 conn(0x7f1c880fd340 msgr2=0x7f1c88104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 shutdown_connections 2024-04-25T13:51:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:39.428+0000 7f1c8e467640 1 -- 172.21.15.94:0/1723206384 wait complete. 2024-04-25T13:51:39.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1264711146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:39 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1264711146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:39 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1264711146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:51:39.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1204923 12.73 MB/sec execute 382 sec latency 116.106 ms 2024-04-25T13:51:40.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:51:40.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:40.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:51:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:40 smithi138 ceph-mon[32977]: pgmap v644: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-04-25T13:51:40.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 -- 172.21.15.94:0/882057413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb724103c30 msgr2=0x7fb724104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 --2- 172.21.15.94:0/882057413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb724103c30 0x7fb724104040 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb718009900 tx=0x7fb71802a410 comp rx=0 tx=0).stop 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 -- 172.21.15.94:0/882057413 shutdown_connections 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 --2- 172.21.15.94:0/882057413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724108930 0x7fb72410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 --2- 172.21.15.94:0/882057413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb724103c30 0x7fb724104040 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 --2- 172.21.15.94:0/882057413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7241029f0 0x7fb7241036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 -- 172.21.15.94:0/882057413 >> 172.21.15.94:0/882057413 conn(0x7fb7240fdc00 msgr2=0x7fb724100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 -- 172.21.15.94:0/882057413 shutdown_connections 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.562+0000 7fb729814640 1 -- 172.21.15.94:0/882057413 wait complete. 2024-04-25T13:51:40.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 Processor -- start 2024-04-25T13:51:40.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 -- start start 2024-04-25T13:51:40.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724103c30 0x7fb72410b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb724108930 0x7fb72410bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410f620 con 0x7fb7241029f0 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410f4a0 con 0x7fb724108930 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb729814640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410f7a0 con 0x7fb724103c30 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41498/0 (socket says 172.21.15.94:41498) 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.563+0000 7fb722ffd640 1 -- 172.21.15.94:0/209165613 learned_addr learned my addr 172.21.15.94:0/209165613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb7227fc640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724103c30 0x7fb72410b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb7237fe640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb724108930 0x7fb72410bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 -- 172.21.15.94:0/209165613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724103c30 msgr2=0x7fb72410b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724103c30 0x7fb72410b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 -- 172.21.15.94:0/209165613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb724108930 msgr2=0x7fb72410bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb724108930 0x7fb72410bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 -- 172.21.15.94:0/209165613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72419e0b0 con 0x7fb7241029f0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.564+0000 7fb722ffd640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e980 tx=0x7fb71400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.565+0000 7fb728812640 1 -- 172.21.15.94:0/209165613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400cd80 con 0x7fb7241029f0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.565+0000 7fb728812640 1 -- 172.21.15.94:0/209165613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7140108e0 con 0x7fb7241029f0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.565+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72410c500 con 0x7fb7241029f0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.565+0000 7fb728812640 1 -- 172.21.15.94:0/209165613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714018990 con 0x7fb7241029f0 2024-04-25T13:51:40.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.565+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241a8cb0 con 0x7fb7241029f0 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 Processor -- start 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 -- start start 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb728812640 1 -- 172.21.15.94:0/209165613 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb71401f020 con 0x7fb7241029f0 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 0x564a40f1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 0x564a40f1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a40e80b00 con 0x564a40b59730 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564a40e1c920 con 0x564a40b51550 2024-04-25T13:51:40.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.568+0000 7fb73079b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a40e7cac0 con 0x564a40b7a410 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41512/0 (socket says 172.21.15.94:41512) 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb7227fc640 1 -- 172.21.15.94:0/834556486 learned_addr learned my addr 172.21.15.94:0/834556486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb722ffd640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 0x564a40f1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb728812640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 0x7fb70c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb722ffd640 1 -- 172.21.15.94:0/834556486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 msgr2=0x564a40f1b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.569+0000 7fb7237fe640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 0x564a40f1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb722ffd640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 0x564a40f1b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb722ffd640 1 -- 172.21.15.94:0/834556486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 msgr2=0x564a40f22920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb722ffd640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb722ffd640 1 -- 172.21.15.94:0/834556486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a40c5a7e0 con 0x564a40b51550 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb7227fc640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb7237fe640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 0x564a40f1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb728812640 1 -- 172.21.15.94:0/209165613 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb7140dee10 con 0x7fb7241029f0 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb7237fe640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 0x7fb70c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb722ffd640 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 0x564a40f1ba20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7140dc840 tx=0x7fb714023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474164d38f80af53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb709ffb640 1 -- 172.21.15.94:0/834556486 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140a6ba0 con 0x564a40b51550 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb709ffb640 1 -- 172.21.15.94:0/834556486 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714010b70 con 0x564a40b51550 2024-04-25T13:51:40.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.570+0000 7fb709ffb640 1 -- 172.21.15.94:0/834556486 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714083e80 con 0x564a40b51550 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 -- 172.21.15.94:0/834556486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 msgr2=0x564a40f1ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 0x564a40f1ba20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7140dc840 tx=0x7fb714023000 comp rx=0 tx=0).stop 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb7237fe640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 0x7fb70c0b2080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x564a40f1c670 tx=0x7fb71c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 -- 172.21.15.94:0/834556486 shutdown_connections 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f22920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b51550 0x564a40f1ba20 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7140dc840 tx=0x7fb714023000 comp rx=0 tx=0).stop 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 --2- 172.21.15.94:0/834556486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b7a410 0x564a40f1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.571+0000 7fb73079b740 1 -- 172.21.15.94:0/834556486 >> 172.21.15.94:0/834556486 conn(0x564a40b6e560 msgr2=0x564a40f16e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.572+0000 7fb73079b740 1 -- 172.21.15.94:0/834556486 shutdown_connections 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.573+0000 7fb73079b740 1 -- 172.21.15.94:0/834556486 wait complete. 2024-04-25T13:51:40.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.573+0000 7fb73079b740 1 -- start start 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.573+0000 7fb73079b740 1 Processor -- start 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b59730 0x564a40f25300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b391f0 0x564a40f2c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a40f23310 con 0x564a40b7a410 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564a40e1a5d0 con 0x564a40b59730 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a40f23490 con 0x564a40b391f0 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb73079b740 10 client.0 _ll_register_callbacks cb 0x564a40e08080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41514/0 (socket says 172.21.15.94:41514) 2024-04-25T13:51:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 -- 172.21.15.94:0/4257974947 learned_addr learned my addr 172.21.15.94:0/4257974947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7237fe640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b391f0 0x564a40f2c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 -- 172.21.15.94:0/4257974947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b391f0 msgr2=0x564a40f2c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b391f0 0x564a40f2c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 -- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b59730 msgr2=0x564a40f25300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b59730 0x564a40f25300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.574+0000 7fb7227fc640 1 -- 172.21.15.94:0/4257974947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a40cbaf70 con 0x564a40b7a410 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb7227fc640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb718005ab0 tx=0x7fb718031d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debfc2b904166973 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718035040 con 0x564a40b7a410 2024-04-25T13:51:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb718004040 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb714010b70 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb718004240 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb73079b740 10 client.19110 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a40b52170 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.575+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb714010b70 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb718044020 con 0x564a40b7a410 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 10 client.19110 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 10 client.19110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 20 client.19110 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b0e630 0x564a40f32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 4 client.19110 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a40e8e710 con 0x564a40b0e630 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb722ffd640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b0e630 0x564a40f32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb722ffd640 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b0e630 0x564a40f32c10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a3b64d559e16b0 server_cookie=7665fcad69c99458 in_seq=0 out_seq=0 2024-04-25T13:51:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.576+0000 7fb702ffd640 10 client.19110 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.578+0000 7fb702ffd640 1 -- 172.21.15.94:0/4257974947 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a40e8e710 con 0x564a40b0e630 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.578+0000 7fb702ffd640 10 client.19110 handle_command_reply: tid=0 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.578+0000 7fb73079b740 1 client.19110 shutdown 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 msgr2=0x564a40f25840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb718005ab0 tx=0x7fb718031d90 comp rx=0 tx=0).stop 2024-04-25T13:51:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 shutdown_connections 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b0e630 0x564a40f32c10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b391f0 0x564a40f2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb702ffd640 0 client.19110 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b7a410 0x564a40f25840 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 --2- 172.21.15.94:0/4257974947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b59730 0x564a40f25300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 >> 172.21.15.94:0/4257974947 conn(0x564a40b6e560 msgr2=0x564a40f15ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 shutdown_connections 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 1 -- 172.21.15.94:0/4257974947 wait complete. 2024-04-25T13:51:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.579+0000 7fb73079b740 20 client.19110 trim_cache size 0 max 16384 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 Processor -- start 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 -- start start 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b0e630 0x564a40f167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241b1b80 con 0x564a40b59730 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241b36c0 con 0x564a40b391f0 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb73079b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241b2250 con 0x564a40b0e630 2024-04-25T13:51:40.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb7237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb7237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41522/0 (socket says 172.21.15.94:41522) 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb7237fe640 1 -- 172.21.15.94:0/2622678305 learned_addr learned my addr 172.21.15.94:0/2622678305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb7227fc640 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.605+0000 7fb722ffd640 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b0e630 0x564a40f167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 -- 172.21.15.94:0/2622678305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b0e630 msgr2=0x564a40f167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b0e630 0x564a40f167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 -- 172.21.15.94:0/2622678305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 msgr2=0x564a40f2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 -- 172.21.15.94:0/2622678305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a40f24380 con 0x564a40b59730 2024-04-25T13:51:40.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.606+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb71c012610 tx=0x7fb71c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c32ff647f700831 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.607+0000 7fb7097fa640 1 -- 172.21.15.94:0/2622678305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c01e4e0 con 0x564a40b59730 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.607+0000 7fb7097fa640 1 -- 172.21.15.94:0/2622678305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c01eb30 con 0x564a40b59730 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.607+0000 7fb7097fa640 1 -- 172.21.15.94:0/2622678305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c010b60 con 0x564a40b59730 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 -- 172.21.15.94:0/2622678305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 msgr2=0x564a40f16d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb71c012610 tx=0x7fb71c015000 comp rx=0 tx=0).stop 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 -- 172.21.15.94:0/2622678305 shutdown_connections 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b59730 0x564a40f16d30 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 --2- 172.21.15.94:0/2622678305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b0e630 0x564a40f167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 -- 172.21.15.94:0/2622678305 >> 172.21.15.94:0/2622678305 conn(0x564a40b6e560 msgr2=0x564a40f1f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 -- 172.21.15.94:0/2622678305 shutdown_connections 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.608+0000 7fb73079b740 1 -- 172.21.15.94:0/2622678305 wait complete. 2024-04-25T13:51:40.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.609+0000 7fb73079b740 1 -- start start 2024-04-25T13:51:40.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.609+0000 7fb73079b740 1 Processor -- start 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.609+0000 7fb73079b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 0x564a40f29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.609+0000 7fb73079b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f29d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb73079b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 0x564a40f98f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb73079b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241b2ff0 con 0x564a40b0e630 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb73079b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241b2920 con 0x564a40b391f0 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb73079b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241afb60 con 0x564a40b59730 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 0x564a40f29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f29d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 0x564a40f29820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41536/0 (socket says 172.21.15.94:41536) 2024-04-25T13:51:40.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7227fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f29d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:44014/0 (socket says 172.21.15.94:44014) 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb722ffd640 1 -- 172.21.15.94:0/2999245796 learned_addr learned my addr 172.21.15.94:0/2999245796 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 0x564a40f98f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb73079b740 10 client.0 _ll_register_callbacks cb 0x7fb718005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 -- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 msgr2=0x564a40f29d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f29d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 -- 172.21.15.94:0/2999245796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 msgr2=0x564a40f29820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 0x564a40f29820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.610+0000 7fb7237fe640 1 -- 172.21.15.94:0/2999245796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a40f994e0 con 0x564a40b59730 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb7237fe640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 0x564a40f98f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009230 tx=0x7fb71c01e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0787b9caa1d78e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0047d0 con 0x564a40b59730 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb71c004cc0 con 0x564a40b59730 2024-04-25T13:51:40.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71c01eb30 con 0x564a40b59730 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0108a0 con 0x564a40b59730 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb73079b740 10 client.28393 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.611+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a40f9a740 con 0x564a40b59730 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c005030 con 0x564a40b59730 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb71c030020 con 0x564a40b59730 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 10 client.28393 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 10 client.28393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 20 client.28393 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b7a410 0x564a40f9d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 4 client.28393 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.612+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb71c01eb30 con 0x564a40b7a410 2024-04-25T13:51:40.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.613+0000 7fb722ffd640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b7a410 0x564a40f9d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.613+0000 7fb722ffd640 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b7a410 0x564a40f9d060 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c565d11a29a6ef server_cookie=9d7fd460f0a37423 in_seq=0 out_seq=0 2024-04-25T13:51:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.613+0000 7fb702ffd640 10 client.28393 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.618+0000 7fb702ffd640 1 -- 172.21.15.94:0/2999245796 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb71c01eb30 con 0x564a40b7a410 2024-04-25T13:51:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.618+0000 7fb702ffd640 10 client.28393 handle_command_reply: tid=0 2024-04-25T13:51:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.618+0000 7fb73079b740 1 client.28393 shutdown 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 msgr2=0x564a40f98f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 0x564a40f98f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009230 tx=0x7fb71c01e9e0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 shutdown_connections 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564a40b7a410 0x564a40f9d060 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a40b59730 0x564a40f98f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40b391f0 0x564a40f29d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb702ffd640 0 client.28393 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 --2- 172.21.15.94:0/2999245796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a40b0e630 0x564a40f29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 >> 172.21.15.94:0/2999245796 conn(0x564a40b6e560 msgr2=0x564a40f19c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 shutdown_connections 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 1 -- 172.21.15.94:0/2999245796 wait complete. 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.619+0000 7fb73079b740 20 client.28393 trim_cache size 0 max 16384 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "73d5d96c-b988-43ff-afb4-8af7229d5f6d", 2024-04-25T13:51:40.620 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:51:40.621 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 msgr2=0x7fb70c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 0x7fb70c0b2080 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x564a40f1c670 tx=0x7fb71c004a40 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 msgr2=0x7fb72410d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e980 tx=0x7fb71400ee40 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 shutdown_connections 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb70c0afb70 0x7fb70c0b2080 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb724108930 0x7fb72410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb724103c30 0x7fb72410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 --2- 172.21.15.94:0/209165613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7241029f0 0x7fb72410d2c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.620+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 >> 172.21.15.94:0/209165613 conn(0x7fb7240fdc00 msgr2=0x7fb724104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.621+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 shutdown_connections 2024-04-25T13:51:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.621+0000 7fb729814640 1 -- 172.21.15.94:0/209165613 wait complete. 2024-04-25T13:51:40.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:40 smithi094 ceph-mon[29195]: pgmap v644: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-04-25T13:51:40.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:51:40.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:51:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:40 smithi137 ceph-mon[32568]: pgmap v644: 97 pgs: 97 active+clean; 41 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 313 op/s 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- 172.21.15.94:0/325761323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 msgr2=0x7f9590103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 --2- 172.21.15.94:0/325761323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590103de0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9584009900 tx=0x7f958402a410 comp rx=0 tx=0).stop 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- 172.21.15.94:0/325761323 shutdown_connections 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 --2- 172.21.15.94:0/325761323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f959010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 --2- 172.21.15.94:0/325761323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590103de0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 --2- 172.21.15.94:0/325761323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f9590103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- 172.21.15.94:0/325761323 >> 172.21.15.94:0/325761323 conn(0x7f95900fd350 msgr2=0x7f95900ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- 172.21.15.94:0/325761323 shutdown_connections 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- 172.21.15.94:0/325761323 wait complete. 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 Processor -- start 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.768+0000 7f95962f9640 1 -- start start 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f95901990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f95901a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f959010e540 con 0x7f9590103930 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f959010e3c0 con 0x7f9590102f50 2024-04-25T13:51:40.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f95962f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f959010e6c0 con 0x7f9590104470 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f95901990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41556/0 (socket says 172.21.15.94:41556) 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 -- 172.21.15.94:0/3486756222 learned_addr learned my addr 172.21.15.94:0/3486756222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f9594896640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f95901a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 -- 172.21.15.94:0/3486756222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 msgr2=0x7f95901a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f95901a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 -- 172.21.15.94:0/3486756222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 msgr2=0x7f95901990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f95901990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 -- 172.21.15.94:0/3486756222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9590101780 con 0x7f9590103930 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f9594896640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f95901a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:40.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958ffff640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f95901990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.769+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9584006010 tx=0x7f95840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.770+0000 7f958d7fa640 1 -- 172.21.15.94:0/3486756222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958403f030 con 0x7f9590103930 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.770+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9590101a10 con 0x7f9590103930 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.770+0000 7f958d7fa640 1 -- 172.21.15.94:0/3486756222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584031910 con 0x7f9590103930 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.770+0000 7f958d7fa640 1 -- 172.21.15.94:0/3486756222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584035070 con 0x7f9590103930 2024-04-25T13:51:40.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.770+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9590101f00 con 0x7f9590103930 2024-04-25T13:51:40.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f958d7fa640 1 -- 172.21.15.94:0/3486756222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9584042030 con 0x7f9590103930 2024-04-25T13:51:40.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 Processor -- start 2024-04-25T13:51:40.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 -- start start 2024-04-25T13:51:40.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942ef6b0 0x5641946b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564194616d80 con 0x564194310390 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5641945d08b0 con 0x5641942e74d0 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f959d1ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564194612d40 con 0x5641942ef6b0 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f9594896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f9594896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41572/0 (socket says 172.21.15.94:41572) 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f9594896640 1 -- 172.21.15.94:0/1466922742 learned_addr learned my addr 172.21.15.94:0/1466922742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f958ffff640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f958f7fe640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942ef6b0 0x5641946b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.773+0000 7f958d7fa640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 0x7f95780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 -- 172.21.15.94:0/1466922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942ef6b0 msgr2=0x5641946b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942ef6b0 0x5641946b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 -- 172.21.15.94:0/1466922742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 msgr2=0x5641946b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 -- 172.21.15.94:0/1466922742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641945a1a40 con 0x5641942e74d0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f9594896640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958d7fa640 1 -- 172.21.15.94:0/3486756222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f95840ff230 con 0x7f9590103930 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f9594896640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 0x7f95780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f958ffff640 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b1fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f959006b2f0 tx=0x7f95800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353d47b3bf688ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f95767fc640 1 -- 172.21.15.94:0/1466922742 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580004030 con 0x5641942e74d0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f9594896640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 0x7f95780b2000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5641946b2ab0 tx=0x7f9588004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f95767fc640 1 -- 172.21.15.94:0/1466922742 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95800054d0 con 0x5641942e74d0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.774+0000 7f95767fc640 1 -- 172.21.15.94:0/1466922742 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580004c20 con 0x5641942e74d0 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- 172.21.15.94:0/1466922742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 msgr2=0x5641946b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b1fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f959006b2f0 tx=0x7f95800118d0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- 172.21.15.94:0/1466922742 shutdown_connections 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942ef6b0 0x5641946b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b1fe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 --2- 172.21.15.94:0/1466922742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- 172.21.15.94:0/1466922742 >> 172.21.15.94:0/1466922742 conn(0x5641943044e0 msgr2=0x5641946ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- 172.21.15.94:0/1466922742 shutdown_connections 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- 172.21.15.94:0/1466922742 wait complete. 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.775+0000 7f959d1ad740 1 -- start start 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 Processor -- start 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 0x5641946bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5641946b3780 con 0x564194310390 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56419459dd10 con 0x5641942ef6b0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641946b3900 con 0x5641942e74d0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f959d1ad740 10 client.0 _ll_register_callbacks cb 0x5641945ca210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f9594896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 0x5641946bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43650/0 (socket says 172.21.15.94:43650) 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958ffff640 1 -- 172.21.15.94:0/3420771718 learned_addr learned my addr 172.21.15.94:0/3420771718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f9594896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41588/0 (socket says 172.21.15.94:41588) 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 -- 172.21.15.94:0/3420771718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 msgr2=0x5641946bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 -- 172.21.15.94:0/3420771718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 msgr2=0x5641946c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 -- 172.21.15.94:0/3420771718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641943f27d0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958ffff640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f9594896640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 0x5641946bbdf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95840fcce0 tx=0x7f9584102900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400d94a97700bc26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.776+0000 7f956f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584049070 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95800054d0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f95840c3900 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95840c6fc0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 10 client.28577 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641943283f0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95800054d0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95840c78d0 con 0x5641942ef6b0 2024-04-25T13:51:40.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 10 client.28577 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 10 client.28577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 20 client.28577 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942cef60 0x5641946c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 4 client.28577 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.777+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564194624990 con 0x5641942cef60 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.778+0000 7f958ffff640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942cef60 0x5641946c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.778+0000 7f956f7fe640 10 client.28577 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.778+0000 7f958ffff640 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942cef60 0x5641946c9210 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd668f5edb087ee server_cookie=9ee7efe81ed1c140 in_seq=0 out_seq=0 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f956f7fe640 1 -- 172.21.15.94:0/3420771718 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564194624990 con 0x5641942cef60 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f956f7fe640 10 client.28577 handle_command_reply: tid=0 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 client.28577 shutdown 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 msgr2=0x5641946bbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 0x5641946bbdf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95840fcce0 tx=0x7f9584102900 comp rx=0 tx=0).stop 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 shutdown_connections 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942cef60 0x5641946c9210 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564194310390 0x5641946c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942ef6b0 0x5641946bbdf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3420771718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641942e74d0 0x5641946bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f956f7fe640 0 client.28577 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 >> 172.21.15.94:0/3420771718 conn(0x5641943044e0 msgr2=0x5641946b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 shutdown_connections 2024-04-25T13:51:40.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 1 -- 172.21.15.94:0/3420771718 wait complete. 2024-04-25T13:51:40.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.779+0000 7f959d1ad740 20 client.28577 trim_cache size 0 max 16384 2024-04-25T13:51:40.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.798+0000 7f959d1ad740 1 Processor -- start 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.798+0000 7f959d1ad740 1 -- start start 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ab490 con 0x5641942ef6b0 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95901acfd0 con 0x5641942e74d0 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f959d1ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95901abb60 con 0x564194310390 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41592/0 (socket says 172.21.15.94:41592) 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f9594896640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 -- 172.21.15.94:0/3342714719 learned_addr learned my addr 172.21.15.94:0/3342714719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 -- 172.21.15.94:0/3342714719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 msgr2=0x5641946b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 -- 172.21.15.94:0/3342714719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 msgr2=0x5641946b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 -- 172.21.15.94:0/3342714719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641942e80f0 con 0x5641942ef6b0 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f9594896640 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958ffff640 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:40.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f958f7fe640 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f95840fd000 tx=0x7f95840d04d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222641d872c136c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.799+0000 7f9575ffb640 1 -- 172.21.15.94:0/3342714719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584049030 con 0x5641942ef6b0 2024-04-25T13:51:40.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f9575ffb640 1 -- 172.21.15.94:0/3342714719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95840c7180 con 0x5641942ef6b0 2024-04-25T13:51:40.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f9575ffb640 1 -- 172.21.15.94:0/3342714719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95840044e0 con 0x5641942ef6b0 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 -- 172.21.15.94:0/3342714719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 msgr2=0x5641946c2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f95840fd000 tx=0x7f95840d04d0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 -- 172.21.15.94:0/3342714719 shutdown_connections 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 --2- 172.21.15.94:0/3342714719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 -- 172.21.15.94:0/3342714719 >> 172.21.15.94:0/3342714719 conn(0x5641943044e0 msgr2=0x5641946b9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 -- 172.21.15.94:0/3342714719 shutdown_connections 2024-04-25T13:51:40.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.800+0000 7f959d1ad740 1 -- 172.21.15.94:0/3342714719 wait complete. 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 -- start start 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 Processor -- start 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946bde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ac900 con 0x5641942ef6b0 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95901ac230 con 0x5641942e74d0 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95901a9470 con 0x564194310390 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f959d1ad740 10 client.0 _ll_register_callbacks cb 0x7f9584005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f958f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41602/0 (socket says 172.21.15.94:41602) 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f958f7fe640 1 -- 172.21.15.94:0/4127453106 learned_addr learned my addr 172.21.15.94:0/4127453106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:40.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.801+0000 7f958ffff640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 -- 172.21.15.94:0/4127453106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 msgr2=0x5641946bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f9594896640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946bde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 -- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 msgr2=0x5641946bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 -- 172.21.15.94:0/4127453106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641945d74b0 con 0x5641942ef6b0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f958f7fe640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f95840c7c70 tx=0x7f95840d02f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10866e189132af2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584049070 con 0x5641942ef6b0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f958403bd50 con 0x5641942ef6b0 2024-04-25T13:51:40.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95840c7180 con 0x5641942ef6b0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95840c6de0 con 0x5641942ef6b0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f959d1ad740 10 client.19134 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641945d74b0 con 0x5641942ef6b0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95840c7180 con 0x5641942ef6b0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95840aeb10 con 0x5641942ef6b0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.802+0000 7f959d1ad740 10 client.19134 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f959d1ad740 10 client.19134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f959d1ad740 20 client.19134 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942a43a0 0x564194743c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f959d1ad740 4 client.19134 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56419461e800 con 0x5641942a43a0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f958ffff640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942a43a0 0x564194743c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f956f7fe640 10 client.19134 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.803+0000 7f958ffff640 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942a43a0 0x564194743c50 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3316ac505ebbb879 server_cookie=192e92ba1c8ec57 in_seq=0 out_seq=0 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.804+0000 7f956f7fe640 1 -- 172.21.15.94:0/4127453106 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56419461e800 con 0x5641942a43a0 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.804+0000 7f956f7fe640 10 client.19134 handle_command_reply: tid=0 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.804+0000 7f959d1ad740 1 client.19134 shutdown 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 msgr2=0x5641946c2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f95840c7c70 tx=0x7f95840d02f0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 shutdown_connections 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5641942a43a0 0x564194743c50 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564194310390 0x5641946bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5641942ef6b0 0x5641946c2c60 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f956f7fe640 0 client.19134 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:40.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 --2- 172.21.15.94:0/4127453106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641942e74d0 0x5641946bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 >> 172.21.15.94:0/4127453106 conn(0x5641943044e0 msgr2=0x5641946b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 shutdown_connections 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 1 -- 172.21.15.94:0/4127453106 wait complete. 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f959d1ad740 20 client.19134 trim_cache size 0 max 16384 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "73d5d96c-b988-43ff-afb4-8af7229d5f6d": { 2024-04-25T13:51:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:51:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "73d5d96c-b988-43ff-afb4-8af7229d5f6d", 2024-04-25T13:51:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:51:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:51:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:51:40.813 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 msgr2=0x7f95780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 0x7f95780b2000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5641946b2ab0 tx=0x7f9588004a40 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 msgr2=0x7f9590199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9584006010 tx=0x7f95840042f0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 shutdown_connections 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f95780afaf0 0x7f95780b2000 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590104470 0x7f95901a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9590103930 0x7f9590199600 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 --2- 172.21.15.94:0/3486756222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9590102f50 0x7f95901990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 >> 172.21.15.94:0/3486756222 conn(0x7f95900fd350 msgr2=0x7f9590104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.805+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 shutdown_connections 2024-04-25T13:51:40.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:51:40.806+0000 7f95962f9640 1 -- 172.21.15.94:0/3486756222 wait complete. 2024-04-25T13:51:40.815 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1207715 12.74 MB/sec execute 383 sec latency 16.868 ms 2024-04-25T13:51:40.817 INFO:tasks.cephfs.filesystem:scrub status for tag:73d5d96c-b988-43ff-afb4-8af7229d5f6d - {'path': '/', 'tag': '73d5d96c-b988-43ff-afb4-8af7229d5f6d', 'options': 'recursive,force'} 2024-04-25T13:51:41.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:41 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:51:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:41 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:41 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:51:41.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:41 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:41 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:51:41.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:41 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:41 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:51:41.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:41 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:41 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:51:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:41 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:41 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:51:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:41 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:51:41.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1209909 12.73 MB/sec execute 384 sec latency 56.229 ms 2024-04-25T13:51:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:42 smithi138 ceph-mon[32977]: pgmap v645: 97 pgs: 97 active+clean; 45 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-04-25T13:51:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:42 smithi137 ceph-mon[32568]: pgmap v645: 97 pgs: 97 active+clean; 45 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-04-25T13:51:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:42 smithi094 ceph-mon[29195]: pgmap v645: 97 pgs: 97 active+clean; 45 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-04-25T13:51:42.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1211986 12.73 MB/sec execute 385 sec latency 13.255 ms 2024-04-25T13:51:43.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1214890 12.74 MB/sec execute 386 sec latency 13.953 ms 2024-04-25T13:51:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:44 smithi138 ceph-mon[32977]: pgmap v646: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-04-25T13:51:44.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:44 smithi094 ceph-mon[29195]: pgmap v646: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-04-25T13:51:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:44 smithi137 ceph-mon[32568]: pgmap v646: 97 pgs: 97 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-04-25T13:51:44.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1216553 12.72 MB/sec execute 387 sec latency 77.502 ms 2024-04-25T13:51:45.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1218822 12.73 MB/sec execute 388 sec latency 19.763 ms 2024-04-25T13:51:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 -- 172.21.15.94:0/2652827872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70101e90 msgr2=0x7f7e70106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 --2- 172.21.15.94:0/2652827872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70101e90 0x7f7e70106b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009900 tx=0x7f7e6002a410 comp rx=0 tx=0).stop 2024-04-25T13:51:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 -- 172.21.15.94:0/2652827872 shutdown_connections 2024-04-25T13:51:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 --2- 172.21.15.94:0/2652827872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701097b0 0x7f7e7010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 --2- 172.21.15.94:0/2652827872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70102390 0x7f7e70109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 --2- 172.21.15.94:0/2652827872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70101e90 0x7f7e70106b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 -- 172.21.15.94:0/2652827872 >> 172.21.15.94:0/2652827872 conn(0x7f7e700fd3a0 msgr2=0x7f7e700ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 -- 172.21.15.94:0/2652827872 shutdown_connections 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.196+0000 7f7e76667640 1 -- 172.21.15.94:0/2652827872 wait complete. 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 Processor -- start 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 -- start start 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102390 0x7f7e7019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e701097b0 0x7f7e701a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f610 con 0x7f7e70102390 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f490 con 0x7f7e70101e90 2024-04-25T13:51:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e76667640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f790 con 0x7f7e701097b0 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42334/0 (socket says 172.21.15.94:42334) 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e6ffff640 1 -- 172.21.15.94:0/4140569838 learned_addr learned my addr 172.21.15.94:0/4140569838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e74c04640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e701097b0 0x7f7e701a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.197+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102390 0x7f7e7019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 -- 172.21.15.94:0/4140569838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e701097b0 msgr2=0x7f7e701a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e701097b0 0x7f7e701a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 -- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102390 msgr2=0x7f7e7019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102390 0x7f7e7019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 -- 172.21.15.94:0/4140569838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a1d50 con 0x7f7e70101e90 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006010 tx=0x7f7e600042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/4140569838 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6003f030 con 0x7f7e70101e90 2024-04-25T13:51:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/4140569838 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e60031910 con 0x7f7e70101e90 2024-04-25T13:51:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/4140569838 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60035070 con 0x7f7e70101e90 2024-04-25T13:51:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a1fe0 con 0x7f7e70101e90 2024-04-25T13:51:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.198+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a2440 con 0x7f7e70101e90 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.201+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/4140569838 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7e60042030 con 0x7f7e70101e90 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.201+0000 7f7e7d5f9740 1 Processor -- start 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.201+0000 7f7e7d5f9740 1 -- start start 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e60310970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff472f0 0x559e60310ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e60276460 con 0x559e5ff4f4d0 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e60230590 con 0x559e5ff701b0 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e60272420 con 0x559e5ff472f0 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e74c04640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e60310970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e74c04640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e60310970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42342/0 (socket says 172.21.15.94:42342) 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43004/0 (socket says 172.21.15.94:43004) 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e74c04640 1 -- 172.21.15.94:0/2056605297 learned_addr learned my addr 172.21.15.94:0/2056605297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff472f0 0x559e60310ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6d7fa640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 0x7f7e580b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2056605297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff472f0 msgr2=0x559e60310ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff472f0 0x559e60310ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2056605297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 msgr2=0x559e60310970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e60310970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2056605297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6023d0d0 con 0x559e5ff4f4d0 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e74c04640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 0x7f7e580b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/4140569838 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f7e600ff160 con 0x7f7e70101e90 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009bd0 tx=0x7f7e64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f675f4801a39fbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.202+0000 7f7e567fc640 1 -- 172.21.15.94:0/2056605297 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64004230 con 0x559e5ff4f4d0 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.203+0000 7f7e74c04640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 0x7f7e580b2070 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e68004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.203+0000 7f7e567fc640 1 -- 172.21.15.94:0/2056605297 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64009e40 con 0x559e5ff4f4d0 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.203+0000 7f7e567fc640 1 -- 172.21.15.94:0/2056605297 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64004c00 con 0x559e5ff4f4d0 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.203+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/2056605297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 msgr2=0x559e60317df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.203+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009bd0 tx=0x7f7e64011810 comp rx=0 tx=0).stop 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/2056605297 shutdown_connections 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff4f4d0 0x559e60317df0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff472f0 0x559e60310ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/2056605297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e60310970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/2056605297 >> 172.21.15.94:0/2056605297 conn(0x559e5ff64300 msgr2=0x559e6030c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/2056605297 shutdown_connections 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/2056605297 wait complete. 2024-04-25T13:51:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- start start 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 Processor -- start 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e6031ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 0x559e60321ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559e60318840 con 0x559e5ff701b0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e601fdb60 con 0x559e5ff472f0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e603189c0 con 0x559e5ff4f4d0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.204+0000 7f7e7d5f9740 10 client.0 _ll_register_callbacks cb 0x559e602222b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e74c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 0x559e60321ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42348/0 (socket says 172.21.15.94:42348) 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 -- 172.21.15.94:0/1719878937 learned_addr learned my addr 172.21.15.94:0/1719878937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e6031ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 -- 172.21.15.94:0/1719878937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 msgr2=0x559e6031ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e6031ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 -- 172.21.15.94:0/1719878937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 msgr2=0x559e60321ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 0x559e60321ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 -- 172.21.15.94:0/1719878937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5ff88210 con 0x559e5ff472f0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e74c04640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 0x559e60321ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e600fcbc0 tx=0x7f7e60102830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d137cb66075dceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60049070 con 0x559e5ff472f0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f7e600c38e0 con 0x559e5ff472f0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e60284070 con 0x559e5ff472f0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.205+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e600c7420 con 0x559e5ff472f0 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 10 client.28595 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e60054620 con 0x559e5ff472f0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e60284070 con 0x559e5ff472f0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7e600cff70 con 0x559e5ff472f0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 10 client.28595 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 10 client.28595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 20 client.28595 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff2ed80 0x559e603282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 4 client.28595 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6027dee0 con 0x559e5ff2ed80 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.206+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff2ed80 0x559e603282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.207+0000 7f7e4f7fe640 10 client.28595 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.207+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff2ed80 0x559e603282f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfee97fd12c179d8 server_cookie=f9d5a74f0c639c3d in_seq=0 out_seq=0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/1719878937 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e6027dee0 con 0x559e5ff2ed80 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e4f7fe640 10 client.28595 handle_command_reply: tid=0 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 client.28595 shutdown 2024-04-25T13:51:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 msgr2=0x559e6031a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e600fcbc0 tx=0x7f7e60102830 comp rx=0 tx=0).stop 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 shutdown_connections 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff2ed80 0x559e603282f0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff701b0 0x559e60321ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e4f7fe640 0 client.28595 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e6031ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1719878937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff472f0 0x559e6031a620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 >> 172.21.15.94:0/1719878937 conn(0x559e5ff64300 msgr2=0x559e6030f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 shutdown_connections 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1719878937 wait complete. 2024-04-25T13:51:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.208+0000 7f7e7d5f9740 20 client.28595 trim_cache size 0 max 16384 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 Processor -- start 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 -- start start 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e60321a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e6030f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ab460 con 0x559e5ff472f0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701acfa0 con 0x559e5ff701b0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701abb30 con 0x559e5ff4f4d0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e60321a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43022/0 (socket says 172.21.15.94:43022) 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6ffff640 1 -- 172.21.15.94:0/1658465192 learned_addr learned my addr 172.21.15.94:0/1658465192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e74c04640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e6030f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/1658465192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 msgr2=0x559e6030f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e6030f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/1658465192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 msgr2=0x559e6030c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/1658465192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e603181b0 con 0x559e5ff4f4d0 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.225+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.226+0000 7f7e74c04640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e6030f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.226+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e60321a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x559e6031b690 tx=0x7f7e64019550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e780a397c23333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.226+0000 7f7e55ffb640 1 -- 172.21.15.94:0/1658465192 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6401e040 con 0x559e5ff4f4d0 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.226+0000 7f7e55ffb640 1 -- 172.21.15.94:0/1658465192 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64009780 con 0x559e5ff4f4d0 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.226+0000 7f7e55ffb640 1 -- 172.21.15.94:0/1658465192 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6402b650 con 0x559e5ff4f4d0 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1658465192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 msgr2=0x559e60321a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e60321a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x559e6031b690 tx=0x7f7e64019550 comp rx=0 tx=0).stop 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1658465192 shutdown_connections 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff701b0 0x559e6030f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff4f4d0 0x559e60321a50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/1658465192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6030c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1658465192 >> 172.21.15.94:0/1658465192 conn(0x559e5ff64300 msgr2=0x559e6030bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1658465192 shutdown_connections 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/1658465192 wait complete. 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.227+0000 7f7e7d5f9740 1 -- start start 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 Processor -- start 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff4f4d0 0x559e6031f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff701b0 0x559e60320790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ac8d0 con 0x559e5ff472f0 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ac200 con 0x559e5ff4f4d0 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701a9440 con 0x559e5ff701b0 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e7d5f9740 10 client.0 _ll_register_callbacks cb 0x7f7e60005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43034/0 (socket says 172.21.15.94:43034) 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 -- 172.21.15.94:0/942206525 learned_addr learned my addr 172.21.15.94:0/942206525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff4f4d0 0x559e6031f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e74c04640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff701b0 0x559e60320790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 -- 172.21.15.94:0/942206525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff701b0 msgr2=0x559e60320790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff701b0 0x559e60320790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 -- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff4f4d0 msgr2=0x559e6031f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff4f4d0 0x559e6031f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 -- 172.21.15.94:0/942206525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e603154a0 con 0x559e5ff472f0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7e600042c0 tx=0x7f7e600c3b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b414f788c8c6269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.228+0000 7f7e4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60106070 con 0x559e5ff472f0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f7e600aedd0 con 0x559e5ff472f0 2024-04-25T13:51:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e64009780 con 0x559e5ff472f0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e60049070 con 0x559e5ff472f0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 10 client.19152 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e60321e10 con 0x559e5ff472f0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e64009780 con 0x559e5ff472f0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7e600e82e0 con 0x559e5ff472f0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 10 client.19152 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 10 client.19152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 20 client.19152 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff041c0 0x559e60392ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 4 client.19152 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e601f95f0 con 0x559e5ff041c0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.229+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff041c0 0x559e60392ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.230+0000 7f7e4f7fe640 10 client.19152 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.230+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff041c0 0x559e60392ac0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44227b335737d0 server_cookie=9cb5daf05b9cc74f in_seq=0 out_seq=0 2024-04-25T13:51:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.409+0000 7f7e4f7fe640 1 -- 172.21.15.94:0/942206525 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e601f95f0 con 0x559e5ff041c0 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.409+0000 7f7e4f7fe640 10 client.19152 handle_command_reply: tid=0 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.409+0000 7f7e7d5f9740 1 client.19152 shutdown 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 msgr2=0x559e6031f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7e600042c0 tx=0x7f7e600c3b90 comp rx=0 tx=0).stop 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 shutdown_connections 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559e5ff041c0 0x559e60392ac0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e5ff701b0 0x559e60320790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e4f7fe640 0 client.19152 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e5ff4f4d0 0x559e6031f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 --2- 172.21.15.94:0/942206525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559e5ff472f0 0x559e6031f370 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 >> 172.21.15.94:0/942206525 conn(0x559e5ff64300 msgr2=0x559e6031a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 shutdown_connections 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 1 -- 172.21.15.94:0/942206525 wait complete. 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.410+0000 7f7e7d5f9740 20 client.19152 trim_cache size 0 max 16384 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:51:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:51:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 msgr2=0x7f7e580b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 0x7f7e580b2070 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009fd0 tx=0x7f7e68004a40 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 msgr2=0x7f7e7019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006010 tx=0x7f7e600042f0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 shutdown_connections 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afb60 0x7f7e580b2070 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e701097b0 0x7f7e701a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102390 0x7f7e7019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 --2- 172.21.15.94:0/4140569838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70101e90 0x7f7e7019a210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 >> 172.21.15.94:0/4140569838 conn(0x7f7e700fd3a0 msgr2=0x7f7e70106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 shutdown_connections 2024-04-25T13:51:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:46.411+0000 7f7e76667640 1 -- 172.21.15.94:0/4140569838 wait complete. 2024-04-25T13:51:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:46 smithi138 ceph-mon[32977]: pgmap v647: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 371 op/s 2024-04-25T13:51:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:46 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:51:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:46 smithi137 ceph-mon[32568]: pgmap v647: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 371 op/s 2024-04-25T13:51:46.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:46 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:51:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:46 smithi094 ceph-mon[29195]: pgmap v647: 97 pgs: 97 active+clean; 47 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 371 op/s 2024-04-25T13:51:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:46 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:51:46.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1221232 12.73 MB/sec execute 389 sec latency 90.991 ms 2024-04-25T13:51:47.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1223364 12.72 MB/sec execute 390 sec latency 16.877 ms 2024-04-25T13:51:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:48 smithi138 ceph-mon[32977]: pgmap v648: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 280 op/s 2024-04-25T13:51:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:48 smithi137 ceph-mon[32568]: pgmap v648: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 280 op/s 2024-04-25T13:51:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:48 smithi094 ceph-mon[29195]: pgmap v648: 97 pgs: 97 active+clean; 50 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 280 op/s 2024-04-25T13:51:48.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1225875 12.73 MB/sec execute 391 sec latency 10.301 ms 2024-04-25T13:51:49.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1228837 12.74 MB/sec execute 392 sec latency 36.668 ms 2024-04-25T13:51:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:50 smithi138 ceph-mon[32977]: pgmap v649: 97 pgs: 97 active+clean; 52 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:51:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:50 smithi137 ceph-mon[32568]: pgmap v649: 97 pgs: 97 active+clean; 52 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:51:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:50 smithi094 ceph-mon[29195]: pgmap v649: 97 pgs: 97 active+clean; 52 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 297 op/s 2024-04-25T13:51:50.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1230426 12.72 MB/sec execute 393 sec latency 77.936 ms 2024-04-25T13:51:51.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1232860 12.73 MB/sec execute 394 sec latency 15.606 ms 2024-04-25T13:51:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:52 smithi138 ceph-mon[32977]: pgmap v650: 97 pgs: 97 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-04-25T13:51:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:52 smithi137 ceph-mon[32568]: pgmap v650: 97 pgs: 97 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-04-25T13:51:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:52 smithi094 ceph-mon[29195]: pgmap v650: 97 pgs: 97 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-04-25T13:51:52.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1235727 12.73 MB/sec execute 395 sec latency 12.902 ms 2024-04-25T13:51:53.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1237493 12.72 MB/sec execute 396 sec latency 13.151 ms 2024-04-25T13:51:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:54 smithi137 ceph-mon[32568]: pgmap v651: 97 pgs: 97 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2024-04-25T13:51:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:54 smithi094 ceph-mon[29195]: pgmap v651: 97 pgs: 97 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2024-04-25T13:51:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:54 smithi138 ceph-mon[32977]: pgmap v651: 97 pgs: 97 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2024-04-25T13:51:54.809 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1239922 12.73 MB/sec execute 397 sec latency 9.707 ms 2024-04-25T13:51:55.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1241967 12.71 MB/sec execute 398 sec latency 60.082 ms 2024-04-25T13:51:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:56 smithi137 ceph-mon[32568]: pgmap v652: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 424 op/s 2024-04-25T13:51:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:56 smithi094 ceph-mon[29195]: pgmap v652: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 424 op/s 2024-04-25T13:51:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:56 smithi138 ceph-mon[32977]: pgmap v652: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 424 op/s 2024-04-25T13:51:56.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1243899 12.71 MB/sec execute 399 sec latency 16.062 ms 2024-04-25T13:51:57.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1246892 12.73 MB/sec execute 400 sec latency 22.230 ms 2024-04-25T13:51:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:51:58 smithi137 ceph-mon[32568]: pgmap v653: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-04-25T13:51:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:51:58 smithi094 ceph-mon[29195]: pgmap v653: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-04-25T13:51:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:51:58 smithi138 ceph-mon[32977]: pgmap v653: 97 pgs: 97 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 380 op/s 2024-04-25T13:51:58.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1249546 12.73 MB/sec execute 401 sec latency 15.764 ms 2024-04-25T13:51:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 -- 172.21.15.94:0/971094953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074103c60 msgr2=0x7f6074104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 --2- 172.21.15.94:0/971094953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074103c60 0x7f6074104070 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002a410 comp rx=0 tx=0).stop 2024-04-25T13:51:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 -- 172.21.15.94:0/971094953 shutdown_connections 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 --2- 172.21.15.94:0/971094953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074108960 0x7f607410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 --2- 172.21.15.94:0/971094953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074103c60 0x7f6074104070 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 --2- 172.21.15.94:0/971094953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074102a20 0x7f6074103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 -- 172.21.15.94:0/971094953 >> 172.21.15.94:0/971094953 conn(0x7f60740fdc10 msgr2=0x7f60741000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 -- 172.21.15.94:0/971094953 shutdown_connections 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.510+0000 7f607b0ca640 1 -- 172.21.15.94:0/971094953 wait complete. 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 Processor -- start 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 -- start start 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074103c60 0x7f6074199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074108960 0x7f60741a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f607410e520 con 0x7f6074102a20 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f607410e3a0 con 0x7f6074108960 2024-04-25T13:51:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f607b0ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f607410e6a0 con 0x7f6074103c60 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f6078e66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f6078e66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49866/0 (socket says 172.21.15.94:49866) 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f6078e66640 1 -- 172.21.15.94:0/2529965283 learned_addr learned my addr 172.21.15.94:0/2529965283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f6079667640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074108960 0x7f60741a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.511+0000 7f6073fff640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074103c60 0x7f6074199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 -- 172.21.15.94:0/2529965283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074103c60 msgr2=0x7f6074199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074103c60 0x7f6074199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 -- 172.21.15.94:0/2529965283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074108960 msgr2=0x7f60741a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074108960 0x7f60741a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 -- 172.21.15.94:0/2529965283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60741a0c60 con 0x7f6074102a20 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6078e66640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f606800e9f0 tx=0x7f606800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6071ffb640 1 -- 172.21.15.94:0/2529965283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606800ccc0 con 0x7f6074102a20 2024-04-25T13:51:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6071ffb640 1 -- 172.21.15.94:0/2529965283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60680035f0 con 0x7f6074102a20 2024-04-25T13:51:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f6071ffb640 1 -- 172.21.15.94:0/2529965283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606800ce60 con 0x7f6074102a20 2024-04-25T13:51:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60741a0f50 con 0x7f6074102a20 2024-04-25T13:51:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.512+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60741a1410 con 0x7f6074102a20 2024-04-25T13:51:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.514+0000 7f6071ffb640 1 -- 172.21.15.94:0/2529965283 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6068003aa0 con 0x7f6074102a20 2024-04-25T13:51:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 Processor -- start 2024-04-25T13:51:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 -- start start 2024-04-25T13:51:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6071ffb640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 0x7f60640b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68306ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f454d0 0x557b6830ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b6826c460 con 0x557b67f661b0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b68226590 con 0x557b67f3d2f0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6081f7b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b68268420 con 0x557b67f454d0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6078e66640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 0x7f60640b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49874/0 (socket says 172.21.15.94:49874) 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 -- 172.21.15.94:0/1426128635 learned_addr learned my addr 172.21.15.94:0/1426128635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6073fff640 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68306ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6071ffb640 1 -- 172.21.15.94:0/2529965283 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f60680def40 con 0x7f6074102a20 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f454d0 0x557b6830ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 -- 172.21.15.94:0/1426128635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f454d0 msgr2=0x557b6830ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f454d0 0x557b6830ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 -- 172.21.15.94:0/1426128635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 msgr2=0x557b68306ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68306ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.515+0000 7f6079667640 1 -- 172.21.15.94:0/1426128635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b682330d0 con 0x557b67f661b0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.516+0000 7f6078e66640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 0x7f60640b2070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f60680a7310 tx=0x7f6068003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.516+0000 7f6079667640 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f606c013960 tx=0x7f606c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ebdeb771270641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.516+0000 7f605effd640 1 -- 172.21.15.94:0/1426128635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c004b20 con 0x557b67f661b0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.516+0000 7f605effd640 1 -- 172.21.15.94:0/1426128635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c003e20 con 0x557b67f661b0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.516+0000 7f605effd640 1 -- 172.21.15.94:0/1426128635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c004050 con 0x557b67f661b0 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- 172.21.15.94:0/1426128635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 msgr2=0x557b68306970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f606c013960 tx=0x7f606c009be0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- 172.21.15.94:0/1426128635 shutdown_connections 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f454d0 0x557b6830ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68306ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 --2- 172.21.15.94:0/1426128635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f661b0 0x557b68306970 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- 172.21.15.94:0/1426128635 >> 172.21.15.94:0/1426128635 conn(0x557b67f5a300 msgr2=0x557b68302300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- 172.21.15.94:0/1426128635 shutdown_connections 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- 172.21.15.94:0/1426128635 wait complete. 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 -- start start 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.517+0000 7f6081f7b740 1 Processor -- start 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68310aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f661b0 0x557b68317a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b6830e7e0 con 0x557b67f3d2f0 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b681f3b60 con 0x557b67f454d0 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b6830e960 con 0x557b67f661b0 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 10 client.0 _ll_register_callbacks cb 0x557b682182b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49876/0 (socket says 172.21.15.94:49876) 2024-04-25T13:51:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 -- 172.21.15.94:0/3068924647 learned_addr learned my addr 172.21.15.94:0/3068924647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6073fff640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68310aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f661b0 0x557b68317a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 -- 172.21.15.94:0/3068924647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f661b0 msgr2=0x557b68317a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f661b0 0x557b68317a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 -- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 msgr2=0x557b68310aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68310aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 -- 172.21.15.94:0/3068924647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b67f7e210 con 0x557b67f3d2f0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6079667640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f606c013500 tx=0x7f606c00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb4e643c82d639 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6053fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c004ca0 con 0x557b67f3d2f0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f606c003fd0 con 0x557b67f3d2f0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c003e20 con 0x557b67f3d2f0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.518+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f606c0041d0 con 0x557b67f3d2f0 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 10 client.19164 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b6804a620 con 0x557b67f3d2f0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c003e20 con 0x557b67f3d2f0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f606c0053f0 con 0x557b67f3d2f0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 10 client.19164 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 10 client.19164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 20 client.19164 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f24d80 0x557b6831df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 4 client.19164 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6827a070 con 0x557b67f24d80 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.519+0000 7f6073fff640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f24d80 0x557b6831df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.520+0000 7f6053fff640 10 client.19164 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.520+0000 7f6073fff640 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f24d80 0x557b6831df30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf60b7d7d00b7def server_cookie=4017cca6b49ba3f8 in_seq=0 out_seq=0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6053fff640 1 -- 172.21.15.94:0/3068924647 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b6827a070 con 0x557b67f24d80 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6053fff640 10 client.19164 handle_command_reply: tid=0 2024-04-25T13:51:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 client.19164 shutdown 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 msgr2=0x557b68310560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f606c013500 tx=0x7f606c00f8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 shutdown_connections 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f24d80 0x557b6831df30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f661b0 0x557b68317a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6053fff640 0 client.19164 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68310aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3068924647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f3d2f0 0x557b68310560 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 >> 172.21.15.94:0/3068924647 conn(0x557b67f5a300 msgr2=0x557b68305210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 shutdown_connections 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 1 -- 172.21.15.94:0/3068924647 wait complete. 2024-04-25T13:51:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.521+0000 7f6081f7b740 20 client.19164 trim_cache size 0 max 16384 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 Processor -- start 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 -- start start 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f24d80 0x557b68305810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68317990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60741aa510 con 0x557b67f454d0 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60741ac050 con 0x557b67f3d2f0 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6081f7b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60741aabe0 con 0x557b67f24d80 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49880/0 (socket says 172.21.15.94:49880) 2024-04-25T13:51:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 -- 172.21.15.94:0/236736668 learned_addr learned my addr 172.21.15.94:0/236736668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6073fff640 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68317990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f24d80 0x557b68305810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 -- 172.21.15.94:0/236736668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f24d80 msgr2=0x557b68305810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f24d80 0x557b68305810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 -- 172.21.15.94:0/236736668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 msgr2=0x557b68317990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68317990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.537+0000 7f6079667640 1 -- 172.21.15.94:0/236736668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6830e180 con 0x557b67f454d0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.538+0000 7f6079667640 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f606c00f810 tx=0x7f606c0131b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5250e65f317b632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.538+0000 7f605e7fc640 1 -- 172.21.15.94:0/236736668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c005bf0 con 0x557b67f454d0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.538+0000 7f605e7fc640 1 -- 172.21.15.94:0/236736668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c03a070 con 0x557b67f454d0 2024-04-25T13:51:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.538+0000 7f605e7fc640 1 -- 172.21.15.94:0/236736668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c003da0 con 0x557b67f454d0 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- 172.21.15.94:0/236736668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 msgr2=0x557b68323730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f606c00f810 tx=0x7f606c0131b0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- 172.21.15.94:0/236736668 shutdown_connections 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f454d0 0x557b68323730 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f3d2f0 0x557b68317990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- 172.21.15.94:0/236736668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f24d80 0x557b68305810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- 172.21.15.94:0/236736668 >> 172.21.15.94:0/236736668 conn(0x557b67f5a300 msgr2=0x557b68305100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- 172.21.15.94:0/236736668 shutdown_connections 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- 172.21.15.94:0/236736668 wait complete. 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- start start 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 Processor -- start 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68384830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60741ab980 con 0x557b67f24d80 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60741ab2b0 con 0x557b67f454d0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60741a84f0 con 0x557b67f3d2f0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.539+0000 7f6081f7b740 10 client.0 _ll_register_callbacks cb 0x7f606c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6073fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49892/0 (socket says 172.21.15.94:49892) 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6073fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50768/0 (socket says 172.21.15.94:50768) 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 -- 172.21.15.94:0/3315528554 learned_addr learned my addr 172.21.15.94:0/3315528554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68384830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 -- 172.21.15.94:0/3315528554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 msgr2=0x557b68317570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 -- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 msgr2=0x557b68384830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68384830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 -- 172.21.15.94:0/3315528554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b68384e20 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6073fff640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6079667640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f606c009270 tx=0x7f606c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e70b6d4da57312 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6053fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c004570 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f606c004710 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c03a070 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f606c020710 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6081f7b740 10 client.19170 fetch_fsmap learned FSMap version 29 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b683860b0 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.540+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c03a070 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f606c03f070 con 0x557b67f24d80 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 10 client.19170 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 10 client.19170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:51:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 20 client.19170 populate_metadata read hostname 'smithi094' 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f661b0 0x557b683889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 4 client.19170 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b68273ee0 con 0x557b67f661b0 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6073fff640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f661b0 0x557b683889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6073fff640 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f661b0 0x557b683889c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e6165d05f76b6b server_cookie=be8f02e0c3d2c8b2 in_seq=0 out_seq=0 2024-04-25T13:51:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.541+0000 7f6053fff640 10 client.19170 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.734+0000 7f6053fff640 1 -- 172.21.15.94:0/3315528554 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b68273ee0 con 0x557b67f661b0 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.734+0000 7f6053fff640 10 client.19170 handle_command_reply: tid=0 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.734+0000 7f6081f7b740 1 client.19170 shutdown 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 msgr2=0x557b68317030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f606c009270 tx=0x7f606c009fa0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 shutdown_connections 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557b67f661b0 0x557b683889c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6053fff640 0 client.19170 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b67f454d0 0x557b68384830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67f3d2f0 0x557b68317570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 --2- 172.21.15.94:0/3315528554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b67f24d80 0x557b68317030 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 >> 172.21.15.94:0/3315528554 conn(0x557b67f5a300 msgr2=0x557b68310130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 shutdown_connections 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 1 -- 172.21.15.94:0/3315528554 wait complete. 2024-04-25T13:51:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.735+0000 7f6081f7b740 20 client.19170 trim_cache size 0 max 16384 2024-04-25T13:51:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:51:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:51:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:51:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 msgr2=0x7f60640b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 0x7f60640b2070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f60680a7310 tx=0x7f6068003a10 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 msgr2=0x7f6074199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f606800e9f0 tx=0x7f606800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 shutdown_connections 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60640afb60 0x7f60640b2070 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074108960 0x7f60741a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6074103c60 0x7f6074199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 --2- 172.21.15.94:0/2529965283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6074102a20 0x7f6074199120 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 >> 172.21.15.94:0/2529965283 conn(0x7f60740fdc10 msgr2=0x7f6074104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 shutdown_connections 2024-04-25T13:51:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:51:59.736+0000 7f607b0ca640 1 -- 172.21.15.94:0/2529965283 wait complete. 2024-04-25T13:51:59.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1251095 12.71 MB/sec execute 402 sec latency 87.365 ms 2024-04-25T13:52:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:00 smithi137 ceph-mon[32568]: pgmap v654: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-04-25T13:52:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:00 smithi094 ceph-mon[29195]: pgmap v654: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-04-25T13:52:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:00 smithi138 ceph-mon[32977]: pgmap v654: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-04-25T13:52:00.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1253864 12.73 MB/sec execute 403 sec latency 57.806 ms 2024-04-25T13:52:01.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1256151 12.72 MB/sec execute 404 sec latency 12.087 ms 2024-04-25T13:52:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:02 smithi137 ceph-mon[32568]: pgmap v655: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-04-25T13:52:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:02 smithi094 ceph-mon[29195]: pgmap v655: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-04-25T13:52:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:02 smithi138 ceph-mon[32977]: pgmap v655: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-04-25T13:52:02.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1258078 12.71 MB/sec execute 405 sec latency 42.907 ms 2024-04-25T13:52:03.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1260991 12.73 MB/sec execute 406 sec latency 13.358 ms 2024-04-25T13:52:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:04 smithi137 ceph-mon[32568]: pgmap v656: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:52:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:04 smithi094 ceph-mon[29195]: pgmap v656: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:52:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:04 smithi138 ceph-mon[32977]: pgmap v656: 97 pgs: 97 active+clean; 46 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-04-25T13:52:04.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1263245 12.72 MB/sec execute 407 sec latency 25.064 ms 2024-04-25T13:52:05.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1264965 12.71 MB/sec execute 408 sec latency 62.155 ms 2024-04-25T13:52:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:06 smithi137 ceph-mon[32568]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-04-25T13:52:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:06 smithi094 ceph-mon[29195]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-04-25T13:52:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:06 smithi138 ceph-mon[32977]: pgmap v657: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-04-25T13:52:06.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1267950 12.72 MB/sec execute 409 sec latency 10.343 ms 2024-04-25T13:52:07.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1269909 12.71 MB/sec execute 410 sec latency 13.480 ms 2024-04-25T13:52:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:08 smithi137 ceph-mon[32568]: pgmap v658: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-04-25T13:52:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:08 smithi094 ceph-mon[29195]: pgmap v658: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-04-25T13:52:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:08 smithi138 ceph-mon[32977]: pgmap v658: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 354 op/s 2024-04-25T13:52:08.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1271946 12.71 MB/sec execute 411 sec latency 20.254 ms 2024-04-25T13:52:09.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1274683 12.72 MB/sec execute 412 sec latency 14.642 ms 2024-04-25T13:52:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:10 smithi137 ceph-mon[32568]: pgmap v659: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-04-25T13:52:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:10 smithi094 ceph-mon[29195]: pgmap v659: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-04-25T13:52:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:10 smithi138 ceph-mon[32977]: pgmap v659: 97 pgs: 97 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 349 op/s 2024-04-25T13:52:10.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1275747 12.70 MB/sec execute 413 sec latency 74.253 ms 2024-04-25T13:52:10.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:52:10.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 -- 172.21.15.94:0/323730875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4104460 msgr2=0x7f0ae410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/323730875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4104460 0x7f0ae410aba0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009ca0 tx=0x7f0adc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 -- 172.21.15.94:0/323730875 shutdown_connections 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/323730875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4104460 0x7f0ae410aba0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/323730875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 0x7f0ae4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/323730875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4102f40 0x7f0ae4103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 -- 172.21.15.94:0/323730875 >> 172.21.15.94:0/323730875 conn(0x7f0ae40fd340 msgr2=0x7f0ae40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 -- 172.21.15.94:0/323730875 shutdown_connections 2024-04-25T13:52:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.933+0000 7f0ae8a92640 1 -- 172.21.15.94:0/323730875 wait complete. 2024-04-25T13:52:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 Processor -- start 2024-04-25T13:52:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 -- start start 2024-04-25T13:52:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 0x7f0ae41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4104460 0x7f0ae41a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410e530 con 0x7f0ae4102f40 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410e3b0 con 0x7f0ae4104460 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae8a92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae410e6b0 con 0x7f0ae4103920 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53136/0 (socket says 172.21.15.94:53136) 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 -- 172.21.15.94:0/2857932272 learned_addr learned my addr 172.21.15.94:0/2857932272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 0x7f0ae41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4104460 0x7f0ae41a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 -- 172.21.15.94:0/2857932272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 msgr2=0x7f0ae41995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 0x7f0ae41995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 -- 172.21.15.94:0/2857932272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4104460 msgr2=0x7f0ae41a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4104460 0x7f0ae41a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.934+0000 7f0ae259c640 1 -- 172.21.15.94:0/2857932272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41a0bc0 con 0x7f0ae4102f40 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ae259c640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9f0 tx=0x7f0ad400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ad37fe640 1 -- 172.21.15.94:0/2857932272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400ccc0 con 0x7f0ae4102f40 2024-04-25T13:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ad37fe640 1 -- 172.21.15.94:0/2857932272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40108e0 con 0x7f0ae4102f40 2024-04-25T13:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a0eb0 con 0x7f0ae4102f40 2024-04-25T13:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ad37fe640 1 -- 172.21.15.94:0/2857932272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4018860 con 0x7f0ae4102f40 2024-04-25T13:52:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.935+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a1310 con 0x7f0ae4102f40 2024-04-25T13:52:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.937+0000 7f0aefa1c740 1 Processor -- start 2024-04-25T13:52:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.937+0000 7f0aefa1c740 1 -- start start 2024-04-25T13:52:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ad37fe640 1 -- 172.21.15.94:0/2857932272 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0ad4018a00 con 0x7f0ae4102f40 2024-04-25T13:52:10.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 0x55f83c935c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5694d0 0x55f83c9361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f83c898d80 con 0x55f83c5716b0 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f83c8528b0 con 0x55f83c5694d0 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f83c894d40 con 0x55f83c592390 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5694d0 0x55f83c9361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53140/0 (socket says 172.21.15.94:53140) 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 -- 172.21.15.94:0/549621999 learned_addr learned my addr 172.21.15.94:0/549621999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 0x55f83c935c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 -- 172.21.15.94:0/549621999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 msgr2=0x55f83c935c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 0x55f83c935c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 -- 172.21.15.94:0/549621999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5694d0 msgr2=0x55f83c9361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5694d0 0x55f83c9361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 -- 172.21.15.94:0/549621999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83c823a40 con 0x55f83c5716b0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 0x55f83c935c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ad37fe640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 0x7f0acc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009ca0 tx=0x7f0ad8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348e17c2673c1eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 0x7f0acc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.938+0000 7f0ad37fe640 1 -- 172.21.15.94:0/2857932272 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f0ad40def50 con 0x7f0ae4102f40 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0ad0ff9640 1 -- 172.21.15.94:0/549621999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8005220 con 0x55f83c5716b0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0ad0ff9640 1 -- 172.21.15.94:0/549621999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad8004c50 con 0x55f83c5716b0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0ad0ff9640 1 -- 172.21.15.94:0/549621999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80197a0 con 0x55f83c5716b0 2024-04-25T13:52:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0aefa1c740 1 -- 172.21.15.94:0/549621999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 msgr2=0x55f83c936e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 0x7f0acc0b2000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55f83c936cc0 tx=0x7f0adc02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.939+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009ca0 tx=0x7f0ad8011810 comp rx=0 tx=0).stop 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- 172.21.15.94:0/549621999 shutdown_connections 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5716b0 0x55f83c936e80 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5694d0 0x55f83c9361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/549621999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c592390 0x55f83c935c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- 172.21.15.94:0/549621999 >> 172.21.15.94:0/549621999 conn(0x55f83c5864e0 msgr2=0x55f83c92f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- 172.21.15.94:0/549621999 shutdown_connections 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- 172.21.15.94:0/549621999 wait complete. 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- start start 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 Processor -- start 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c93fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c940050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f83c937990 con 0x55f83c592390 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f83c81fd10 con 0x55f83c5716b0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f83c937b10 con 0x55f83c5694d0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0aefa1c740 10 client.0 _ll_register_callbacks cb 0x55f83c84c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0ae2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0ae2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53142/0 (socket says 172.21.15.94:53142) 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0ae2d9d640 1 -- 172.21.15.94:0/3901986477 learned_addr learned my addr 172.21.15.94:0/3901986477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.940+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c940050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c93fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 -- 172.21.15.94:0/3901986477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 msgr2=0x55f83c93fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c93fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 -- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 msgr2=0x55f83c940050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c940050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 -- 172.21.15.94:0/3901986477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83c6747d0 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0adc035000 tx=0x7f0adc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eba7e2770776126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc0043d0 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f0adc004570 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad8004c50 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0adc004770 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0aefa1c740 10 client.19182 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83c5aa3f0 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.941+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad8004c50 con 0x55f83c592390 2024-04-25T13:52:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0adc051070 con 0x55f83c592390 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 10 client.19182 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 10 client.19182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 20 client.19182 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c550f60 0x55f83c94d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 4 client.19182 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83c8a6990 con 0x55f83c550f60 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c550f60 0x55f83c94d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0ac9ffb640 10 client.19182 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.942+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c550f60 0x55f83c94d580 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2521951d6b039ac4 server_cookie=9edb040e7e65062e in_seq=0 out_seq=0 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/3901986477 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f83c8a6990 con 0x55f83c550f60 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0ac9ffb640 10 client.19182 handle_command_reply: tid=0 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 client.19182 shutdown 2024-04-25T13:52:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 msgr2=0x55f83c946fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0adc035000 tx=0x7f0adc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 shutdown_connections 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c550f60 0x55f83c94d580 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c592390 0x55f83c946fb0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c940050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3901986477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c93fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0ac9ffb640 0 client.19182 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 >> 172.21.15.94:0/3901986477 conn(0x55f83c5864e0 msgr2=0x55f83c932180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 shutdown_connections 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3901986477 wait complete. 2024-04-25T13:52:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.943+0000 7f0aefa1c740 20 client.19182 trim_cache size 0 max 16384 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.961+0000 7f0aefa1c740 1 Processor -- start 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.961+0000 7f0aefa1c740 1 -- start start 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c946ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c93e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41a91e0 con 0x55f83c5864e0 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41aad20 con 0x55f83c5716b0 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41a98b0 con 0x55f83c5694d0 2024-04-25T13:52:10.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53158/0 (socket says 172.21.15.94:53158) 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 -- 172.21.15.94:0/3956929888 learned_addr learned my addr 172.21.15.94:0/3956929888 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c946ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c93e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 -- 172.21.15.94:0/3956929888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 msgr2=0x55f83c946ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c946ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 -- 172.21.15.94:0/3956929888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 msgr2=0x55f83c93e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c93e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 -- 172.21.15.94:0/3956929888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83c56a0f0 con 0x55f83c5864e0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0ae259c640 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c6a0 tx=0x7f0ad4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0d6111808dc5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0acbfff640 1 -- 172.21.15.94:0/3956929888 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40a6f50 con 0x55f83c5864e0 2024-04-25T13:52:10.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0acbfff640 1 -- 172.21.15.94:0/3956929888 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40a7440 con 0x55f83c5864e0 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.962+0000 7f0acbfff640 1 -- 172.21.15.94:0/3956929888 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40e10d0 con 0x55f83c5864e0 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3956929888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 msgr2=0x55f83c92f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c6a0 tx=0x7f0ad4009740 comp rx=0 tx=0).stop 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3956929888 shutdown_connections 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c93e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5694d0 0x55f83c946ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/3956929888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5864e0 0x55f83c92f320 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3956929888 >> 172.21.15.94:0/3956929888 conn(0x55f83c92a4d0 msgr2=0x55f83c92ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3956929888 shutdown_connections 2024-04-25T13:52:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- 172.21.15.94:0/3956929888 wait complete. 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.963+0000 7f0aefa1c740 1 -- start start 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 Processor -- start 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c944890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5864e0 0x55f83c9457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41aa650 con 0x55f83c5694d0 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41a9f80 con 0x55f83c5716b0 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41a71c0 con 0x55f83c5864e0 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0aefa1c740 10 client.0 _ll_register_callbacks cb 0x7f0adc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53166/0 (socket says 172.21.15.94:53166) 2024-04-25T13:52:10.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 -- 172.21.15.94:0/1414291451 learned_addr learned my addr 172.21.15.94:0/1414291451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c944890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae2d9d640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5864e0 0x55f83c9457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 -- 172.21.15.94:0/1414291451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5864e0 msgr2=0x55f83c9457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5864e0 0x55f83c9457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 -- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 msgr2=0x55f83c944890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c944890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 -- 172.21.15.94:0/1414291451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83c8594b0 con 0x55f83c5694d0 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ae259c640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40dca00 tx=0x7f0ad400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceae5decf8957d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40a6f50 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.964+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f0ad40a1bd0 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad40a7440 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ad40b01f0 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 10 client.19188 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f83c946e90 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40a7440 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0ad40afdf0 con 0x55f83c5694d0 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 10 client.19188 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 10 client.19188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 20 client.19188 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c592390 0x55f83c933800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 4 client.19188 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f83c8a0800 con 0x55f83c592390 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.965+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c592390 0x55f83c933800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0ac9ffb640 10 client.19188 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0ae1d9b640 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c592390 0x55f83c933800 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f651c9ea0dfaea70 server_cookie=2f48c1f26647db66 in_seq=0 out_seq=0 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0ac9ffb640 1 -- 172.21.15.94:0/1414291451 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f83c8a0800 con 0x55f83c592390 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0ac9ffb640 10 client.19188 handle_command_reply: tid=0 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 client.19188 shutdown 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 msgr2=0x55f83c944350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40dca00 tx=0x7f0ad400ce60 comp rx=0 tx=0).stop 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 shutdown_connections 2024-04-25T13:52:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f83c592390 0x55f83c933800 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f83c5864e0 0x55f83c9457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f83c5716b0 0x55f83c944890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0ac9ffb640 0 client.19188 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 --2- 172.21.15.94:0/1414291451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f83c5694d0 0x55f83c944350 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 >> 172.21.15.94:0/1414291451 conn(0x55f83c92a4d0 msgr2=0x55f83c93e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 shutdown_connections 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 1 -- 172.21.15.94:0/1414291451 wait complete. 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.966+0000 7f0aefa1c740 20 client.19188 trim_cache size 0 max 16384 2024-04-25T13:52:10.975 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:52:10.976 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 msgr2=0x7f0acc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 0x7f0acc0b2000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55f83c936cc0 tx=0x7f0adc02f310 comp rx=0 tx=0).stop 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 msgr2=0x7f0ae4199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9f0 tx=0x7f0ad400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 shutdown_connections 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0acc0afaf0 0x7f0acc0b2000 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae4104460 0x7f0ae41a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4103920 0x7f0ae41995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 --2- 172.21.15.94:0/2857932272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae4102f40 0x7f0ae4199080 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 >> 172.21.15.94:0/2857932272 conn(0x7f0ae40fd340 msgr2=0x7f0ae4106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 shutdown_connections 2024-04-25T13:52:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:10.967+0000 7f0ae8a92640 1 -- 172.21.15.94:0/2857932272 wait complete. 2024-04-25T13:52:10.979 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:52:10.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/3477092440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44103c00 msgr2=0x7f8b44104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/3477092440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44103c00 0x7f8b44104010 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009900 tx=0x7f8b3002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/3477092440 shutdown_connections 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/3477092440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b44108900 0x7f8b4410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/3477092440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44103c00 0x7f8b44104010 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/3477092440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b441029c0 0x7f8b441036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/3477092440 >> 172.21.15.94:0/3477092440 conn(0x7f8b440fdbd0 msgr2=0x7f8b44100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/3477092440 shutdown_connections 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.090+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/3477092440 wait complete. 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 Processor -- start 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 -- start start 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b441029c0 0x7f8b44199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b44103c00 0x7f8b44199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410e4f0 con 0x7f8b44108900 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410e370 con 0x7f8b44103c00 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b4ae4a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4410e670 con 0x7f8b441029c0 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b493e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b493e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53180/0 (socket says 172.21.15.94:53180) 2024-04-25T13:52:11.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b493e7640 1 -- 172.21.15.94:0/823069640 learned_addr learned my addr 172.21.15.94:0/823069640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b48be6640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b441029c0 0x7f8b44199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.091+0000 7f8b43fff640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b44103c00 0x7f8b44199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 -- 172.21.15.94:0/823069640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b441029c0 msgr2=0x7f8b44199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b441029c0 0x7f8b44199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 -- 172.21.15.94:0/823069640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b44103c00 msgr2=0x7f8b44199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b44103c00 0x7f8b44199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 -- 172.21.15.94:0/823069640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b441a0c40 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b493e7640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00eb10 tx=0x7f8b3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004b20 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b3c004cc0 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b441a0f30 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c010670 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.092+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b441a1390 con 0x7f8b44108900 2024-04-25T13:52:11.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.093+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4406afa0 con 0x7f8b44108900 2024-04-25T13:52:11.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.094+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8b3c005020 con 0x7f8b44108900 2024-04-25T13:52:11.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.095+0000 7f8b41ffb640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 0x7f8b340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.095+0000 7f8b48be6640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 0x7f8b340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.095+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f8b3c0df230 con 0x7f8b44108900 2024-04-25T13:52:11.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.095+0000 7f8b48be6640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 0x7f8b340b2000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b441035c0 tx=0x7f8b38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.100+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b3c0a7d70 con 0x7f8b44108900 2024-04-25T13:52:11.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.333+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b440637d0 con 0x7f8b44108900 2024-04-25T13:52:11.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.334+0000 7f8b41ffb640 1 -- 172.21.15.94:0/823069640 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7f8b3c0a7670 con 0x7f8b44108900 2024-04-25T13:52:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:52:11.334 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:51:20.200062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28255":{"gid":28255,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3275100272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3275100272},{"type":"v1","addr":"172.21.15.138:6835","nonce":3275100272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:52:11.334 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 29 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 msgr2=0x7f8b340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 0x7f8b340b2000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b441035c0 tx=0x7f8b38005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 msgr2=0x7f8b441a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00eb10 tx=0x7f8b3c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 shutdown_connections 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8b340afaf0 0x7f8b340b2000 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b44108900 0x7f8b441a05d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b44103c00 0x7f8b44199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 --2- 172.21.15.94:0/823069640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b441029c0 0x7f8b44199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 >> 172.21.15.94:0/823069640 conn(0x7f8b440fdbd0 msgr2=0x7f8b441048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 shutdown_connections 2024-04-25T13:52:11.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.336+0000 7f8b4ae4a640 1 -- 172.21.15.94:0/823069640 wait complete. 2024-04-25T13:52:11.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.458+0000 7f13b9fb7640 1 -- 172.21.15.94:0/3880840592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b4104670 msgr2=0x7f13b410adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.458+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/3880840592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b4104670 0x7f13b410adc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 -- 172.21.15.94:0/3880840592 shutdown_connections 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/3880840592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b4104670 0x7f13b410adc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/3880840592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4103b10 0x7f13b4103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/3880840592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4102fb0 0x7f13b41033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 -- 172.21.15.94:0/3880840592 >> 172.21.15.94:0/3880840592 conn(0x7f13b40fd350 msgr2=0x7f13b40ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 -- 172.21.15.94:0/3880840592 shutdown_connections 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 -- 172.21.15.94:0/3880840592 wait complete. 2024-04-25T13:52:11.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 Processor -- start 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.459+0000 7f13b9fb7640 1 -- start start 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4102fb0 0x7f13b419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4103b10 0x7f13b419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410f840 con 0x7f13b41a1670 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410f6c0 con 0x7f13b4103b10 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410f9c0 con 0x7f13b4102fb0 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53198/0 (socket says 172.21.15.94:53198) 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 -- 172.21.15.94:0/1896877041 learned_addr learned my addr 172.21.15.94:0/1896877041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b37fe640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4102fb0 0x7f13b419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4103b10 0x7f13b419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 -- 172.21.15.94:0/1896877041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4102fb0 msgr2=0x7f13b419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4102fb0 0x7f13b419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 -- 172.21.15.94:0/1896877041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4103b10 msgr2=0x7f13b419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4103b10 0x7f13b419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 -- 172.21.15.94:0/1896877041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b41a4100 con 0x7f13b41a1670 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b3fff640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0097b0 tx=0x7f13ac031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b0ff9640 1 -- 172.21.15.94:0/1896877041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac003bf0 con 0x7f13b41a1670 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b0ff9640 1 -- 172.21.15.94:0/1896877041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac003d90 con 0x7f13b41a1670 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b41a4390 con 0x7f13b41a1670 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.460+0000 7f13b0ff9640 1 -- 172.21.15.94:0/1896877041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac02fc30 con 0x7f13b41a1670 2024-04-25T13:52:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.461+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a48e0 con 0x7f13b41a1670 2024-04-25T13:52:11.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 Processor -- start 2024-04-25T13:52:11.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b0ff9640 1 -- 172.21.15.94:0/1896877041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f13ac004040 con 0x7f13b41a1670 2024-04-25T13:52:11.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 -- start start 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3df0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3df0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3d54d80 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3d0ea20 con 0x55a7f3a4e390 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3d50d40 con 0x55a7f3a254d0 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53212/0 (socket says 172.21.15.94:53212) 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 -- 172.21.15.94:0/1158417701 learned_addr learned my addr 172.21.15.94:0/1158417701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3df0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b0ff9640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 0x7f139c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b37fe640 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3df0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 -- 172.21.15.94:0/1158417701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 msgr2=0x55a7f3df0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3df0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 -- 172.21.15.94:0/1158417701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 msgr2=0x55a7f3df0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3df0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b2ffd640 1 -- 172.21.15.94:0/1158417701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f3d19c30 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.464+0000 7f13b3fff640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 0x7f139c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f13b0ff9640 1 -- 172.21.15.94:0/1896877041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f13ac100340 con 0x7f13b41a1670 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f13a80058a0 tx=0x7f13a800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2798858dfe6ef0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f1399ffb640 1 -- 172.21.15.94:0/1158417701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8012d00 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f13b3fff640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 0x7f139c0b2000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0c4800 tx=0x7f13ac0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f1399ffb640 1 -- 172.21.15.94:0/1158417701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8010c20 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.465+0000 7f1399ffb640 1 -- 172.21.15.94:0/1158417701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a801e9e0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- 172.21.15.94:0/1158417701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 msgr2=0x55a7f3df1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f13a80058a0 tx=0x7f13a800b7c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- 172.21.15.94:0/1158417701 shutdown_connections 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df1a20 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3df0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/1158417701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3df0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- 172.21.15.94:0/1158417701 >> 172.21.15.94:0/1158417701 conn(0x55a7f3a424e0 msgr2=0x55a7f3dec020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- 172.21.15.94:0/1158417701 shutdown_connections 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- 172.21.15.94:0/1158417701 wait complete. 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 -- start start 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.466+0000 7f13c0e6e740 1 Processor -- start 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 0x55a7f3e019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3df2530 con 0x55a7f3a4e390 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3cdbdc0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7f3df26b0 con 0x55a7f3a254d0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13c0e6e740 10 client.0 _ll_register_callbacks cb 0x55a7f3cf7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 0x55a7f3e019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:49914/0 (socket says 172.21.15.94:49914) 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 -- 172.21.15.94:0/3627795711 learned_addr learned my addr 172.21.15.94:0/3627795711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 0x55a7f3e019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53228/0 (socket says 172.21.15.94:53228) 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 -- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 msgr2=0x55a7f3dfa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 -- 172.21.15.94:0/3627795711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 msgr2=0x55a7f3e019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 0x55a7f3e019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 -- 172.21.15.94:0/3627795711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f3a663f0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f13b37fe640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13a4012700 tx=0x7f13a4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289599c148ece1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f1392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.467+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004050 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a8010c20 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f13a40041f0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13a40043f0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 10 client.28655 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7f3b307d0 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8010c20 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13a4003b60 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 10 client.28655 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 10 client.28655 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 20 client.28655 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f3a0cf60 0x55a7f3e08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 4 client.28655 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f3d62990 con 0x55a7f3a0cf60 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.468+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f3a0cf60 0x55a7f3e08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.469+0000 7f1392ffd640 10 client.28655 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.469+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f3a0cf60 0x55a7f3e08240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=912e9554cde3c87 server_cookie=e7ef01f8d91c7b5f in_seq=0 out_seq=0 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.469+0000 7f1392ffd640 1 -- 172.21.15.94:0/3627795711 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7f3d62990 con 0x55a7f3a0cf60 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f1392ffd640 10 client.28655 handle_command_reply: tid=0 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 client.28655 shutdown 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 msgr2=0x55a7f3dfaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13a4012700 tx=0x7f13a4012bc0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 shutdown_connections 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f3a0cf60 0x55a7f3e08240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a4e390 0x55a7f3e019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f1392ffd640 0 client.28655 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfaa80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3627795711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 >> 172.21.15.94:0/3627795711 conn(0x55a7f3a424e0 msgr2=0x55a7f3deef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 shutdown_connections 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3627795711 wait complete. 2024-04-25T13:52:11.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.470+0000 7f13c0e6e740 20 client.28655 trim_cache size 0 max 16384 2024-04-25T13:52:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 Processor -- start 2024-04-25T13:52:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 -- start start 2024-04-25T13:52:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3deba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ad670 con 0x55a7f3a254d0 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41af1b0 con 0x55a7f3a4e390 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41add40 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53232/0 (socket says 172.21.15.94:53232) 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 -- 172.21.15.94:0/3867829979 learned_addr learned my addr 172.21.15.94:0/3867829979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b37fe640 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3deba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 -- 172.21.15.94:0/3867829979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a2d6b0 msgr2=0x55a7f3df57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 -- 172.21.15.94:0/3867829979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 msgr2=0x55a7f3deba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3deba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 -- 172.21.15.94:0/3867829979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f3df8630 con 0x55a7f3a254d0 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.484+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f13a8012fd0 tx=0x7f13a800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4816ba2b77d7639e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13997fa640 1 -- 172.21.15.94:0/3867829979 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8024040 con 0x55a7f3a254d0 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13997fa640 1 -- 172.21.15.94:0/3867829979 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8016070 con 0x55a7f3a254d0 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13997fa640 1 -- 172.21.15.94:0/3867829979 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80048a0 con 0x55a7f3a254d0 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3867829979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 msgr2=0x55a7f3df5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f13a8012fd0 tx=0x7f13a800bc20 comp rx=0 tx=0).stop 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3867829979 shutdown_connections 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a4e390 0x55a7f3deba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3df57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/3867829979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3df5270 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3867829979 >> 172.21.15.94:0/3867829979 conn(0x55a7f3a424e0 msgr2=0x55a7f3dfa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.485+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3867829979 shutdown_connections 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 -- 172.21.15.94:0/3867829979 wait complete. 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 -- start start 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 Processor -- start 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a4e390 0x55a7f3dff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41aeae0 con 0x55a7f3a254d0 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ae410 con 0x55a7f3a2d6b0 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ab650 con 0x55a7f3a4e390 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13c0e6e740 10 client.0 _ll_register_callbacks cb 0x7f13a40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53238/0 (socket says 172.21.15.94:53238) 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13b2ffd640 1 -- 172.21.15.94:0/833692778 learned_addr learned my addr 172.21.15.94:0/833692778 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.486+0000 7f13b37fe640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a4e390 0x55a7f3dff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 -- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a4e390 msgr2=0x55a7f3dff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a4e390 0x55a7f3dff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 -- 172.21.15.94:0/833692778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 msgr2=0x55a7f3dfe820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfe820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 -- 172.21.15.94:0/833692778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f3dffd10 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13b2ffd640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f13a80124f0 tx=0x7f13a8010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af3c101c63b96f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f1392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8024040 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f13a8005240 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a8016070 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13a8043050 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13c0e6e740 10 client.19212 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7f3e00f90 con 0x55a7f3a254d0 2024-04-25T13:52:11.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.487+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8016070 con 0x55a7f3a254d0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13a802a040 con 0x55a7f3a254d0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 10 client.19212 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 10 client.19212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 20 client.19212 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f39e23a0 0x55a7f3e72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 4 client.19212 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7f3d5c800 con 0x55a7f39e23a0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13b37fe640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f39e23a0 0x55a7f3e72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f1392ffd640 10 client.19212 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.488+0000 7f13b37fe640 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f39e23a0 0x55a7f3e72680 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51197a0ffad4d269 server_cookie=cac16aae95838268 in_seq=0 out_seq=0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f1392ffd640 1 -- 172.21.15.94:0/833692778 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7f3d5c800 con 0x55a7f39e23a0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f1392ffd640 10 client.19212 handle_command_reply: tid=0 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 client.19212 shutdown 2024-04-25T13:52:11.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 msgr2=0x55a7f3dfe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f13a80124f0 tx=0x7f13a8010a00 comp rx=0 tx=0).stop 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 shutdown_connections 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a7f39e23a0 0x55a7f3e72680 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7f3a4e390 0x55a7f3dff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7f3a2d6b0 0x55a7f3dfe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f1392ffd640 0 client.19212 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 --2- 172.21.15.94:0/833692778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7f3a254d0 0x55a7f3dfe2e0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 >> 172.21.15.94:0/833692778 conn(0x55a7f3a424e0 msgr2=0x55a7f3df5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 shutdown_connections 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 1 -- 172.21.15.94:0/833692778 wait complete. 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.489+0000 7f13c0e6e740 20 client.19212 trim_cache size 0 max 16384 2024-04-25T13:52:11.495 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 msgr2=0x7f139c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 0x7f139c0b2000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0c4800 tx=0x7f13ac0095c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 msgr2=0x7f13b41a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0097b0 tx=0x7f13ac031840 comp rx=0 tx=0).stop 2024-04-25T13:52:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 shutdown_connections 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f139c0afaf0 0x7f139c0b2000 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13b41a1670 0x7f13b41a3a90 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b4103b10 0x7f13b419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 --2- 172.21.15.94:0/1896877041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b4102fb0 0x7f13b419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 >> 172.21.15.94:0/1896877041 conn(0x7f13b40fd350 msgr2=0x7f13b40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 shutdown_connections 2024-04-25T13:52:11.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.490+0000 7f13b9fb7640 1 -- 172.21.15.94:0/1896877041 wait complete. 2024-04-25T13:52:11.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- 172.21.15.94:0/852839714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc1050b0 msgr2=0x7f60cc1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 --2- 172.21.15.94:0/852839714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc1050b0 0x7f60cc1054c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009900 tx=0x7f60c802a410 comp rx=0 tx=0).stop 2024-04-25T13:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- 172.21.15.94:0/852839714 shutdown_connections 2024-04-25T13:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 --2- 172.21.15.94:0/852839714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc102260 0x7f60cc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 --2- 172.21.15.94:0/852839714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 0x7f60cc101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 --2- 172.21.15.94:0/852839714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc1050b0 0x7f60cc1054c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- 172.21.15.94:0/852839714 >> 172.21.15.94:0/852839714 conn(0x7f60cc0fd380 msgr2=0x7f60cc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- 172.21.15.94:0/852839714 shutdown_connections 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- 172.21.15.94:0/852839714 wait complete. 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 Processor -- start 2024-04-25T13:52:11.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.616+0000 7f60d35e3640 1 -- start start 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 0x7f60cc104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc1050b0 0x7f60cc102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10f600 con 0x7f60cc102260 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10f480 con 0x7f60cc1050b0 2024-04-25T13:52:11.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d35e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc10f780 con 0x7f60cc101760 2024-04-25T13:52:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53246/0 (socket says 172.21.15.94:53246) 2024-04-25T13:52:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 -- 172.21.15.94:0/2790044001 learned_addr learned my addr 172.21.15.94:0/2790044001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d137f640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 0x7f60cc104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 -- 172.21.15.94:0/2790044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 msgr2=0x7f60cc104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 0x7f60cc104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 -- 172.21.15.94:0/2790044001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc1050b0 msgr2=0x7f60cc102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc1050b0 0x7f60cc102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 -- 172.21.15.94:0/2790044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc103570 con 0x7f60cc102260 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9f0 tx=0x7f60bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60c27fc640 1 -- 172.21.15.94:0/2790044001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cd80 con 0x7f60cc102260 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.617+0000 7f60c27fc640 1 -- 172.21.15.94:0/2790044001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60bc0108e0 con 0x7f60cc102260 2024-04-25T13:52:11.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.618+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc1a4310 con 0x7f60cc102260 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.618+0000 7f60c27fc640 1 -- 172.21.15.94:0/2790044001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc018980 con 0x7f60cc102260 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.618+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc1a47d0 con 0x7f60cc102260 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.620+0000 7f60da56a740 1 Processor -- start 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.620+0000 7f60da56a740 1 -- start start 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60c27fc640 1 -- 172.21.15.94:0/2790044001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f60bc010430 con 0x7f60cc102260 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b1577d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1578a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555b14dbd80 con 0x5555b11ac4d0 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555b1495a20 con 0x5555b11d5390 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60da56a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555b14d7d40 con 0x5555b11b46b0 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d1b80640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b1577d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d1b80640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:49948/0 (socket says 172.21.15.94:49948) 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d1b80640 1 -- 172.21.15.94:0/430109492 learned_addr learned my addr 172.21.15.94:0/430109492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1578a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 -- 172.21.15.94:0/430109492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 msgr2=0x5555b1578a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1578a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 -- 172.21.15.94:0/430109492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 msgr2=0x5555b1577810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 -- 172.21.15.94:0/430109492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b14a0c30 con 0x5555b11ac4d0 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1578a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d1b80640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60c27fc640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 0x7f60b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.621+0000 7f60d137f640 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b1577d90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f60c802a930 tx=0x7f60c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc62b8b2f93c6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.622+0000 7f60d1b80640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 0x7f60b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.622+0000 7f60c27fc640 1 -- 172.21.15.94:0/2790044001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f60bc0df040 con 0x7f60cc102260 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.622+0000 7f60b77fe640 1 -- 172.21.15.94:0/430109492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8035030 con 0x5555b11ac4d0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.622+0000 7f60b77fe640 1 -- 172.21.15.94:0/430109492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c8040070 con 0x5555b11ac4d0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.622+0000 7f60b77fe640 1 -- 172.21.15.94:0/430109492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8003830 con 0x5555b11ac4d0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- 172.21.15.94:0/430109492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 msgr2=0x5555b1577d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b1577d90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f60c802a930 tx=0x7f60c8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60d1b80640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 0x7f60b80b2000 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5555b1578860 tx=0x7f60c40099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- 172.21.15.94:0/430109492 shutdown_connections 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1578a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b1577d90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 --2- 172.21.15.94:0/430109492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1577810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- 172.21.15.94:0/430109492 >> 172.21.15.94:0/430109492 conn(0x5555b11c94e0 msgr2=0x5555b1573020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- 172.21.15.94:0/430109492 shutdown_connections 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- 172.21.15.94:0/430109492 wait complete. 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 -- start start 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.623+0000 7f60da56a740 1 Processor -- start 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1581ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555b15794d0 con 0x5555b11ac4d0 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555b1462dc0 con 0x5555b11d5390 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555b1579650 con 0x5555b11b46b0 2024-04-25T13:52:11.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53264/0 (socket says 172.21.15.94:53264) 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 -- 172.21.15.94:0/985550865 learned_addr learned my addr 172.21.15.94:0/985550865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60da56a740 10 client.0 _ll_register_callbacks cb 0x5555b147e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1581ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d1b80640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 -- 172.21.15.94:0/985550865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 msgr2=0x5555b1581ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1581ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 -- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 msgr2=0x5555b1588a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 -- 172.21.15.94:0/985550865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b11ed3f0 con 0x5555b11ac4d0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60d137f640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009900 tx=0x7f60c8031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9464a2dbe4700b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:11.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.624+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8035040 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c8040070 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f60c80317b0 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60c80041d0 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 10 client.19224 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555b12b77d0 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c8040070 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f60c8045630 con 0x5555b11ac4d0 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 10 client.19224 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 10 client.19224 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 20 client.19224 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b1193f60 0x5555b158ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 4 client.19224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b14e9990 con 0x5555b1193f60 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.625+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b1193f60 0x5555b158ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.626+0000 7f60b4ff9640 10 client.19224 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.626+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b1193f60 0x5555b158ee70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a58d9ec1d32f9f70 server_cookie=c7a38bea65e2ce8e in_seq=0 out_seq=0 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60b4ff9640 1 -- 172.21.15.94:0/985550865 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555b14e9990 con 0x5555b1193f60 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60b4ff9640 10 client.19224 handle_command_reply: tid=0 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 client.19224 shutdown 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 msgr2=0x5555b15815a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009900 tx=0x7f60c8031e20 comp rx=0 tx=0).stop 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 shutdown_connections 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b1193f60 0x5555b158ee70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11b46b0 0x5555b1581ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60b4ff9640 0 client.19224 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 --2- 172.21.15.94:0/985550865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11ac4d0 0x5555b15815a0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 >> 172.21.15.94:0/985550865 conn(0x5555b11c94e0 msgr2=0x5555b1575f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 shutdown_connections 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 1 -- 172.21.15.94:0/985550865 wait complete. 2024-04-25T13:52:11.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.627+0000 7f60da56a740 20 client.19224 trim_cache size 0 max 16384 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 Processor -- start 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 -- start start 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b1594500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1ad750 con 0x5555b11b46b0 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1af290 con 0x5555b11d5390 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60da56a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1ade20 con 0x5555b11ac4d0 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53280/0 (socket says 172.21.15.94:53280) 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 -- 172.21.15.94:0/3336843558 learned_addr learned my addr 172.21.15.94:0/3336843558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d137f640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b1594500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d1b80640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 -- 172.21.15.94:0/3336843558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 msgr2=0x5555b1594500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b1594500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 -- 172.21.15.94:0/3336843558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 msgr2=0x5555b1588960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 -- 172.21.15.94:0/3336843558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b1588ea0 con 0x5555b11b46b0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d1b80640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0dcaa0 tx=0x7f60bc0dc970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b911a165856834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.641+0000 7f60b6ffd640 1 -- 172.21.15.94:0/3336843558 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010720 con 0x5555b11b46b0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60b6ffd640 1 -- 172.21.15.94:0/3336843558 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60bc0a6f90 con 0x5555b11b46b0 2024-04-25T13:52:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60b6ffd640 1 -- 172.21.15.94:0/3336843558 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010a90 con 0x5555b11b46b0 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 -- 172.21.15.94:0/3336843558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 msgr2=0x5555b1594a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0dcaa0 tx=0x7f60bc0dc970 comp rx=0 tx=0).stop 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 -- 172.21.15.94:0/3336843558 shutdown_connections 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11d5390 0x5555b1588960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11b46b0 0x5555b1594a40 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 --2- 172.21.15.94:0/3336843558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b1594500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 -- 172.21.15.94:0/3336843558 >> 172.21.15.94:0/3336843558 conn(0x5555b11c94e0 msgr2=0x5555b1576490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 -- 172.21.15.94:0/3336843558 shutdown_connections 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.642+0000 7f60da56a740 1 -- 172.21.15.94:0/3336843558 wait complete. 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 -- start start 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 Processor -- start 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b15f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11b46b0 0x5555b15f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1aebc0 con 0x5555b11d5390 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1ae4f0 con 0x5555b11b46b0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc1ab730 con 0x5555b11ac4d0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60da56a740 10 client.0 _ll_register_callbacks cb 0x7f60c4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d1b80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d1b80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53288/0 (socket says 172.21.15.94:53288) 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d1b80640 1 -- 172.21.15.94:0/179195090 learned_addr learned my addr 172.21.15.94:0/179195090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d0b7e640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11b46b0 0x5555b15f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d137f640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b15f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.643+0000 7f60d1b80640 1 -- 172.21.15.94:0/179195090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 msgr2=0x5555b15f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d1b80640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b15f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d1b80640 1 -- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11b46b0 msgr2=0x5555b15f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d1b80640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11b46b0 0x5555b15f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d1b80640 1 -- 172.21.15.94:0/179195090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b15f6110 con 0x5555b11d5390 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d137f640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b15f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60d1b80640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f60c40049d0 tx=0x7f60c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca7a5e4b25b993f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c402d060 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f60c401ed40 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60bc0a6f90 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60c4032430 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 10 client.19230 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555b15f7440 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60bc0a6f90 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f60c4027070 con 0x5555b11d5390 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 10 client.19230 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 10 client.19230 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 20 client.19230 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b11693a0 0x5555b15f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 4 client.19230 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.644+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555b14e3800 con 0x5555b11693a0 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60d137f640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b11693a0 0x5555b15f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60b4ff9640 10 client.19230 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60d137f640 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b11693a0 0x5555b15f9e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e431b43e659d9020 server_cookie=ab42a6049c3ac61 in_seq=0 out_seq=0 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60b4ff9640 1 -- 172.21.15.94:0/179195090 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555b14e3800 con 0x5555b11693a0 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60b4ff9640 10 client.19230 handle_command_reply: tid=0 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 client.19230 shutdown 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 msgr2=0x5555b15f5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f60c40049d0 tx=0x7f60c4004a40 comp rx=0 tx=0).stop 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 shutdown_connections 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5555b11693a0 0x5555b15f9e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555b11d5390 0x5555b15f5b20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555b11b46b0 0x5555b15f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60b4ff9640 0 client.19230 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 --2- 172.21.15.94:0/179195090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555b11ac4d0 0x5555b15f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 >> 172.21.15.94:0/179195090 conn(0x5555b11c94e0 msgr2=0x5555b1576490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 shutdown_connections 2024-04-25T13:52:11.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.645+0000 7f60da56a740 1 -- 172.21.15.94:0/179195090 wait complete. 2024-04-25T13:52:11.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60da56a740 20 client.19230 trim_cache size 0 max 16384 2024-04-25T13:52:11.659 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:11.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 msgr2=0x7f60b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 0x7f60b80b2000 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5555b1578860 tx=0x7f60c40099b0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 msgr2=0x7f60cc102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9f0 tx=0x7f60bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 shutdown_connections 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f60b80afaf0 0x7f60b80b2000 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60cc1050b0 0x7f60cc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60cc102260 0x7f60cc102900 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 --2- 172.21.15.94:0/2790044001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60cc101760 0x7f60cc104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 >> 172.21.15.94:0/2790044001 conn(0x7f60cc0fd380 msgr2=0x7f60cc105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 shutdown_connections 2024-04-25T13:52:11.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.646+0000 7f60d35e3640 1 -- 172.21.15.94:0/2790044001 wait complete. 2024-04-25T13:52:11.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 -- 172.21.15.94:0/1093643545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 msgr2=0x7fb4c81036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/1093643545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c81036f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009900 tx=0x7fb4b802a410 comp rx=0 tx=0).stop 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 -- 172.21.15.94:0/1093643545 shutdown_connections 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/1093643545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 0x7fb4c810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/1093643545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 0x7fb4c8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/1093643545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c81036f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 -- 172.21.15.94:0/1093643545 >> 172.21.15.94:0/1093643545 conn(0x7fb4c80fdbe0 msgr2=0x7fb4c8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 -- 172.21.15.94:0/1093643545 shutdown_connections 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.779+0000 7fb4cdc51640 1 -- 172.21.15.94:0/1093643545 wait complete. 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 Processor -- start 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 -- start start 2024-04-25T13:52:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 0x7fb4c819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 0x7fb4c81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810e4f0 con 0x7fb4c81029f0 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810e370 con 0x7fb4c8103c30 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4cdc51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810e670 con 0x7fb4c8108930 2024-04-25T13:52:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53310/0 (socket says 172.21.15.94:53310) 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4c77fe640 1 -- 172.21.15.94:0/2975973005 learned_addr learned my addr 172.21.15.94:0/2975973005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 0x7fb4c819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.780+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 0x7fb4c81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 -- 172.21.15.94:0/2975973005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 msgr2=0x7fb4c81a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 0x7fb4c81a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 -- 172.21.15.94:0/2975973005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 msgr2=0x7fb4c819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 0x7fb4c819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 -- 172.21.15.94:0/2975973005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a1db0 con 0x7fb4c81029f0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c4ff9640 1 -- 172.21.15.94:0/2975973005 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b803f030 con 0x7fb4c81029f0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c4ff9640 1 -- 172.21.15.94:0/2975973005 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b8031910 con 0x7fb4c81029f0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a2040 con 0x7fb4c81029f0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4c4ff9640 1 -- 172.21.15.94:0/2975973005 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8035070 con 0x7fb4c81029f0 2024-04-25T13:52:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.781+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a2530 con 0x7fb4c81029f0 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 Processor -- start 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c4ff9640 1 -- 172.21.15.94:0/2975973005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb4b8042030 con 0x7fb4c81029f0 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 -- start start 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 0x55dfe0b95810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0b96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0af9d80 con 0x55dfe07d26b0 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0ab3a20 con 0x55dfe07f3390 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0af5d40 con 0x55dfe07ca4d0 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0b96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 0x55dfe0b95810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 0x55dfe0b95810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:50000/0 (socket says 172.21.15.94:50000) 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36168/0 (socket says 172.21.15.94:36168) 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c7fff640 1 -- 172.21.15.94:0/1757307578 learned_addr learned my addr 172.21.15.94:0/1757307578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0b96a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53316/0 (socket says 172.21.15.94:53316) 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 -- 172.21.15.94:0/1757307578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 msgr2=0x55dfe0b95810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 0x55dfe0b95810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 -- 172.21.15.94:0/1757307578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 msgr2=0x55dfe0b96a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0b96a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.784+0000 7fb4c77fe640 1 -- 172.21.15.94:0/1757307578 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe0abec30 con 0x55dfe07ca4d0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4c4ff9640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 0x7fb4b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 0x7fb4b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80fc9a0 tx=0x7fb4b8107220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c297d8c873d659e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4c4ff9640 1 -- 172.21.15.94:0/2975973005 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb4b80fef70 con 0x7fb4c81029f0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4adffb640 1 -- 172.21.15.94:0/1757307578 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80c3320 con 0x55dfe07ca4d0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4adffb640 1 -- 172.21.15.94:0/1757307578 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80c6f70 con 0x55dfe07ca4d0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.785+0000 7fb4adffb640 1 -- 172.21.15.94:0/1757307578 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80c6bd0 con 0x55dfe07ca4d0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1757307578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 msgr2=0x55dfe0b95d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 0x7fb4b00b2000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55dfe0b96860 tx=0x7fb4c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80fc9a0 tx=0x7fb4b8107220 comp rx=0 tx=0).stop 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1757307578 shutdown_connections 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0b96a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b95d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1757307578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07f3390 0x55dfe0b95810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1757307578 >> 172.21.15.94:0/1757307578 conn(0x55dfe07e74e0 msgr2=0x55dfe0b91020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1757307578 shutdown_connections 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1757307578 wait complete. 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.786+0000 7fb4d4be8740 1 -- start start 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 Processor -- start 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 0x55dfe0ba6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0b972f0 con 0x55dfe07f3390 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0a80dc0 con 0x55dfe07d26b0 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0b97470 con 0x55dfe07ca4d0 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4d4be8740 10 client.0 _ll_register_callbacks cb 0x55dfe0a9c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 0x55dfe0ba6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:50006/0 (socket says 172.21.15.94:50006) 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c6ffd640 1 -- 172.21.15.94:0/4015827889 learned_addr learned my addr 172.21.15.94:0/4015827889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 -- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 msgr2=0x55dfe0b9f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 -- 172.21.15.94:0/4015827889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 msgr2=0x55dfe0b9f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c7fff640 1 -- 172.21.15.94:0/4015827889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe080b3f0 con 0x55dfe07f3390 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.787+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 0x55dfe0ba6730 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000f610 tx=0x7fb4c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0db53819b24cea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0003970 con 0x55dfe07f3390 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb4c0017030 con 0x55dfe07f3390 2024-04-25T13:52:11.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b80c6f70 con 0x55dfe07f3390 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4c0003b10 con 0x55dfe07f3390 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 10 client.19242 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfe08d57d0 con 0x55dfe07f3390 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80c6f70 con 0x55dfe07f3390 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4c0004210 con 0x55dfe07f3390 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 10 client.19242 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 10 client.19242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 20 client.19242 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07b1f60 0x55dfe0bad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 4 client.19242 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.788+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe0b07990 con 0x55dfe07b1f60 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.789+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07b1f60 0x55dfe0bad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.789+0000 7fb4a6ffd640 10 client.19242 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:11.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.789+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07b1f60 0x55dfe0bad060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b59a32a86681a5ad server_cookie=2c32052adf1b3010 in_seq=0 out_seq=0 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/4015827889 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfe0b07990 con 0x55dfe07b1f60 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4a6ffd640 10 client.19242 handle_command_reply: tid=0 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 client.19242 shutdown 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 msgr2=0x55dfe0ba6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 0x55dfe0ba6730 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000f610 tx=0x7fb4c0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 shutdown_connections 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07b1f60 0x55dfe0bad060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07f3390 0x55dfe0ba6730 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0b9f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4a6ffd640 0 client.19242 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/4015827889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 >> 172.21.15.94:0/4015827889 conn(0x55dfe07e74e0 msgr2=0x55dfe0b90040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 shutdown_connections 2024-04-25T13:52:11.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 1 -- 172.21.15.94:0/4015827889 wait complete. 2024-04-25T13:52:11.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.790+0000 7fb4d4be8740 20 client.19242 trim_cache size 0 max 16384 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 Processor -- start 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 -- start start 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07b1f60 0x55dfe0b9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ab600 con 0x55dfe07d26b0 2024-04-25T13:52:11.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ad140 con 0x55dfe07ca4d0 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81abcd0 con 0x55dfe07b1f60 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53330/0 (socket says 172.21.15.94:53330) 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 -- 172.21.15.94:0/937235924 learned_addr learned my addr 172.21.15.94:0/937235924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07b1f60 0x55dfe0b9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 -- 172.21.15.94:0/937235924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07b1f60 msgr2=0x55dfe0b9d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07b1f60 0x55dfe0b9d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 -- 172.21.15.94:0/937235924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07ca4d0 msgr2=0x55dfe0b9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.804+0000 7fb4c7fff640 1 -- 172.21.15.94:0/937235924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe0b8c3a0 con 0x55dfe07d26b0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005c40 tx=0x7fb4c000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00a46aa85245e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4ad7fa640 1 -- 172.21.15.94:0/937235924 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c001f240 con 0x55dfe07d26b0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4ad7fa640 1 -- 172.21.15.94:0/937235924 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c001fcf0 con 0x55dfe07d26b0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4ad7fa640 1 -- 172.21.15.94:0/937235924 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00043c0 con 0x55dfe07d26b0 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4d4be8740 1 -- 172.21.15.94:0/937235924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 msgr2=0x55dfe0bb1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.805+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005c40 tx=0x7fb4c000f3c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- 172.21.15.94:0/937235924 shutdown_connections 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07d26b0 0x55dfe0bb1810 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07ca4d0 0x55dfe0b9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/937235924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07b1f60 0x55dfe0b9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- 172.21.15.94:0/937235924 >> 172.21.15.94:0/937235924 conn(0x55dfe07e74e0 msgr2=0x55dfe0b9a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- 172.21.15.94:0/937235924 shutdown_connections 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- 172.21.15.94:0/937235924 wait complete. 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- start start 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 Processor -- start 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0ba3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0c133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81aca70 con 0x55dfe07b1f60 2024-04-25T13:52:11.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81ac3a0 con 0x55dfe07d26b0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81a95e0 con 0x55dfe07ca4d0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4d4be8740 10 client.0 _ll_register_callbacks cb 0x7fb4c0009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53346/0 (socket says 172.21.15.94:53346) 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4c77fe640 1 -- 172.21.15.94:0/1074073725 learned_addr learned my addr 172.21.15.94:0/1074073725 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.806+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0ba3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c7fff640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0c133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 -- 172.21.15.94:0/1074073725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 msgr2=0x55dfe0c133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0c133c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 -- 172.21.15.94:0/1074073725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 msgr2=0x55dfe0ba2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 -- 172.21.15.94:0/1074073725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe0c13930 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4c6ffd640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0ba3360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55dfe0b8c870 tx=0x7fb4bc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd79265a43bfcce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc022020 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb4bc005220 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c001fcf0 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4bc005420 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4d4be8740 10 client.28522 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.807+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfe0c14c90 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c001fcf0 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4bc01e440 con 0x55dfe07ca4d0 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 10 client.28522 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 10 client.28522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 20 client.28522 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07f3390 0x55dfe0c17730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 4 client.28522 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfe0b01800 con 0x55dfe07f3390 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07f3390 0x55dfe0c17730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4c77fe640 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07f3390 0x55dfe0c17730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ec30e7734cbc370 server_cookie=25b8e15d82c4a785 in_seq=0 out_seq=0 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.808+0000 7fb4a6ffd640 10 client.28522 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4a6ffd640 1 -- 172.21.15.94:0/1074073725 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfe0b01800 con 0x55dfe07f3390 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4a6ffd640 10 client.28522 handle_command_reply: tid=0 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 client.28522 shutdown 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 msgr2=0x55dfe0ba3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0ba3360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55dfe0b8c870 tx=0x7fb4bc00f790 comp rx=0 tx=0).stop 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 shutdown_connections 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55dfe07f3390 0x55dfe0c17730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfe07d26b0 0x55dfe0c133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfe07ca4d0 0x55dfe0ba3360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 --2- 172.21.15.94:0/1074073725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dfe07b1f60 0x55dfe0ba2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4a6ffd640 0 client.28522 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 >> 172.21.15.94:0/1074073725 conn(0x55dfe07e74e0 msgr2=0x55dfe0b9d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 shutdown_connections 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 1 -- 172.21.15.94:0/1074073725 wait complete. 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4d4be8740 20 client.28522 trim_cache size 0 max 16384 2024-04-25T13:52:11.818 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:11.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 msgr2=0x7fb4b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 0x7fb4b00b2000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55dfe0b96860 tx=0x7fb4c0009500 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 msgr2=0x7fb4c806bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8004270 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 shutdown_connections 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4b00afaf0 0x7fb4b00b2000 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8108930 0x7fb4c81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c8103c30 0x7fb4c819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.809+0000 7fb4cdc51640 1 --2- 172.21.15.94:0/2975973005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4c81029f0 0x7fb4c806bd20 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.810+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 >> 172.21.15.94:0/2975973005 conn(0x7fb4c80fdbe0 msgr2=0x7fb4c8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.810+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 shutdown_connections 2024-04-25T13:52:11.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.810+0000 7fb4cdc51640 1 -- 172.21.15.94:0/2975973005 wait complete. 2024-04-25T13:52:11.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1278620 12.71 MB/sec execute 414 sec latency 8.655 ms 2024-04-25T13:52:11.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 -- 172.21.15.94:0/1849266527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 msgr2=0x7f8ee810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 --2- 172.21.15.94:0/1849266527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee810ad40 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009ca0 tx=0x7f8ee002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 -- 172.21.15.94:0/1849266527 shutdown_connections 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 --2- 172.21.15.94:0/1849266527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee810ad40 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 --2- 172.21.15.94:0/1849266527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee8103c20 0x7f8ee8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 --2- 172.21.15.94:0/1849266527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee81029e0 0x7f8ee81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 -- 172.21.15.94:0/1849266527 >> 172.21.15.94:0/1849266527 conn(0x7f8ee80fdbd0 msgr2=0x7f8ee8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.935+0000 7f8eee506640 1 -- 172.21.15.94:0/1849266527 shutdown_connections 2024-04-25T13:52:11.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 -- 172.21.15.94:0/1849266527 wait complete. 2024-04-25T13:52:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 Processor -- start 2024-04-25T13:52:11.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 -- start start 2024-04-25T13:52:11.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee81029e0 0x7f8ee81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee8103c20 0x7f8ee8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.936+0000 7f8eee506640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eee506640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e4f0 con 0x7f8ee8108920 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eee506640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e370 con 0x7f8ee81029e0 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eee506640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e670 con 0x7f8ee8103c20 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53362/0 (socket says 172.21.15.94:53362) 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee81029e0 0x7f8ee81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 -- 172.21.15.94:0/1730415627 learned_addr learned my addr 172.21.15.94:0/1730415627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee8103c20 0x7f8ee8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 -- 172.21.15.94:0/1730415627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee8103c20 msgr2=0x7f8ee8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee8103c20 0x7f8ee8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 -- 172.21.15.94:0/1730415627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee81029e0 msgr2=0x7f8ee81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee81029e0 0x7f8ee81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 -- 172.21.15.94:0/1730415627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee81a0f30 con 0x7f8ee8108920 2024-04-25T13:52:11.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eecaa3640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00097b0 tx=0x7f8ee0031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8ee57fa640 1 -- 172.21.15.94:0/1730415627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0003bf0 con 0x7f8ee8108920 2024-04-25T13:52:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee81a11c0 con 0x7f8ee8108920 2024-04-25T13:52:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8ee57fa640 1 -- 172.21.15.94:0/1730415627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee0003d90 con 0x7f8ee8108920 2024-04-25T13:52:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8ee57fa640 1 -- 172.21.15.94:0/1730415627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee002fc30 con 0x7f8ee8108920 2024-04-25T13:52:11.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.937+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a1710 con 0x7f8ee8108920 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 Processor -- start 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ee57fa640 1 -- 172.21.15.94:0/1730415627 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8ee0004040 con 0x7f8ee8108920 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 -- start start 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db07823d80 con 0x55db074fc6b0 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db077dda20 con 0x55db074f44d0 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db0781fd40 con 0x55db0751d390 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.940+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53370/0 (socket says 172.21.15.94:53370) 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 -- 172.21.15.94:0/395287649 learned_addr learned my addr 172.21.15.94:0/395287649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 -- 172.21.15.94:0/395287649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 msgr2=0x55db078bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 -- 172.21.15.94:0/395287649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 msgr2=0x55db078bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee57fa640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 0x7f8ed00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 -- 172.21.15.94:0/395287649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db077e8c30 con 0x55db074fc6b0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8eecaa3640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 0x7f8ed00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0058c0 tx=0x7f8edc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470ae3b69bad85b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ece7fc640 1 -- 172.21.15.94:0/395287649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00f810 con 0x55db074fc6b0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ee57fa640 1 -- 172.21.15.94:0/1730415627 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f8ee0100340 con 0x7f8ee8108920 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ece7fc640 1 -- 172.21.15.94:0/395287649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc00fe60 con 0x55db074fc6b0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.941+0000 7f8ece7fc640 1 -- 172.21.15.94:0/395287649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01fc30 con 0x55db074fc6b0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- 172.21.15.94:0/395287649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 msgr2=0x55db078c0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8eecaa3640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 0x7f8ed00b2000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00c8480 tx=0x7f8ee00f6ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0058c0 tx=0x7f8edc005850 comp rx=0 tx=0).stop 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- 172.21.15.94:0/395287649 shutdown_connections 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074fc6b0 0x55db078c0a20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/395287649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- 172.21.15.94:0/395287649 >> 172.21.15.94:0/395287649 conn(0x55db075114e0 msgr2=0x55db078bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- 172.21.15.94:0/395287649 shutdown_connections 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- 172.21.15.94:0/395287649 wait complete. 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.942+0000 7f8ef53bb740 1 -- start start 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 Processor -- start 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db078c1530 con 0x55db074f44d0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db077aadc0 con 0x55db074fc6b0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db078c16b0 con 0x55db0751d390 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ef53bb740 10 client.0 _ll_register_callbacks cb 0x55db077c6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53372/0 (socket says 172.21.15.94:53372) 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 -- 172.21.15.94:0/3921968687 learned_addr learned my addr 172.21.15.94:0/3921968687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 -- 172.21.15.94:0/3921968687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 msgr2=0x55db078d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 -- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 msgr2=0x55db078cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.943+0000 7f8ee77fe640 1 -- 172.21.15.94:0/3921968687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db075353f0 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009cc0 tx=0x7f8edc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4e34df9fac5293 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01f9b0 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f8edc01fb50 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc00fe60 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8edc01fd50 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 10 client.19260 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db075ff7d0 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc00fe60 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8edc01d020 con 0x55db074f44d0 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 10 client.19260 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 10 client.19260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 20 client.19260 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074dbf60 0x55db078d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 4 client.19260 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.944+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db07831990 con 0x55db074dbf60 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.945+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074dbf60 0x55db078d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.945+0000 7f8ec77fe640 10 client.19260 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.945+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074dbf60 0x55db078d9100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a86a1594f7c2fb4 server_cookie=9e350b90e042f8bd in_seq=0 out_seq=0 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ec77fe640 1 -- 172.21.15.94:0/3921968687 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db07831990 con 0x55db074dbf60 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ec77fe640 10 client.19260 handle_command_reply: tid=0 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 client.19260 shutdown 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 msgr2=0x55db078cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009cc0 tx=0x7f8edc009c30 comp rx=0 tx=0).stop 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 shutdown_connections 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074dbf60 0x55db078d9100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db0751d390 0x55db078d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/3921968687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db074f44d0 0x55db078cb7a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ec77fe640 0 client.19260 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 >> 172.21.15.94:0/3921968687 conn(0x55db075114e0 msgr2=0x55db078bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 shutdown_connections 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 1 -- 172.21.15.94:0/3921968687 wait complete. 2024-04-25T13:52:11.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.946+0000 7f8ef53bb740 20 client.19260 trim_cache size 0 max 16384 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 Processor -- start 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 -- start start 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074f44d0 0x55db078ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81a9200 con 0x55db0751d390 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81aad40 con 0x55db074fc6b0 2024-04-25T13:52:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81a98d0 con 0x55db074f44d0 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53378/0 (socket says 172.21.15.94:53378) 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 -- 172.21.15.94:0/1720117045 learned_addr learned my addr 172.21.15.94:0/1720117045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074f44d0 0x55db078ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 -- 172.21.15.94:0/1720117045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074f44d0 msgr2=0x55db078ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074f44d0 0x55db078ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 -- 172.21.15.94:0/1720117045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 msgr2=0x55db078d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.960+0000 7f8ee77fe640 1 -- 172.21.15.94:0/1720117045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db078c3dc0 con 0x55db0751d390 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.961+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0059f0 tx=0x7f8edc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d1e4fc6cd9bd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.961+0000 7f8ecdffb640 1 -- 172.21.15.94:0/1720117045 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc011030 con 0x55db0751d390 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.961+0000 7f8ecdffb640 1 -- 172.21.15.94:0/1720117045 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc0111d0 con 0x55db0751d390 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.961+0000 7f8ecdffb640 1 -- 172.21.15.94:0/1720117045 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00f9f0 con 0x55db0751d390 2024-04-25T13:52:11.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- 172.21.15.94:0/1720117045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 msgr2=0x55db078cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0059f0 tx=0x7f8edc005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- 172.21.15.94:0/1720117045 shutdown_connections 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cb370 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074fc6b0 0x55db078d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/1720117045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074f44d0 0x55db078ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- 172.21.15.94:0/1720117045 >> 172.21.15.94:0/1720117045 conn(0x55db075114e0 msgr2=0x55db078c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- 172.21.15.94:0/1720117045 shutdown_connections 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- 172.21.15.94:0/1720117045 wait complete. 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 -- start start 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.962+0000 7f8ef53bb740 1 Processor -- start 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 0x55db078cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81aa670 con 0x55db0751d390 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81a9fa0 con 0x55db074f44d0 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81a71e0 con 0x55db074fc6b0 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ef53bb740 10 client.0 _ll_register_callbacks cb 0x7f8ed8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53382/0 (socket says 172.21.15.94:53382) 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 0x55db078cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee77fe640 1 -- 172.21.15.94:0/954213420 learned_addr learned my addr 172.21.15.94:0/954213420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:11.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 -- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 msgr2=0x55db078d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 -- 172.21.15.94:0/954213420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 msgr2=0x55db078cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 -- 172.21.15.94:0/954213420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db0793f4f0 con 0x55db074fc6b0 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ee7fff640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 0x55db078cf670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55db078bbdb0 tx=0x7f8ed80039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c084fdc4b4503e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8005220 con 0x55db074fc6b0 2024-04-25T13:52:11.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.963+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f8ed80053c0 con 0x55db074fc6b0 2024-04-25T13:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc0111d0 con 0x55db074fc6b0 2024-04-25T13:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ed8030050 con 0x55db074fc6b0 2024-04-25T13:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 10 client.28540 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db07940780 con 0x55db074fc6b0 2024-04-25T13:52:11.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc0111d0 con 0x55db074fc6b0 2024-04-25T13:52:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ed80297d0 con 0x55db074fc6b0 2024-04-25T13:52:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 10 client.28540 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 10 client.28540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:52:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 20 client.28540 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:11.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074b13a0 0x55db079430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 4 client.28540 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db0782b800 con 0x55db074b13a0 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.964+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074b13a0 0x55db079430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ee77fe640 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074b13a0 0x55db079430c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e444f532c56b7461 server_cookie=7fed2a2bd724802f in_seq=0 out_seq=0 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ec77fe640 10 client.28540 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:11.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ec77fe640 1 -- 172.21.15.94:0/954213420 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db0782b800 con 0x55db074b13a0 2024-04-25T13:52:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ec77fe640 10 client.28540 handle_command_reply: tid=0 2024-04-25T13:52:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 client.28540 shutdown 2024-04-25T13:52:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 msgr2=0x55db078cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 0x55db078cf670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55db078bbdb0 tx=0x7f8ed80039c0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 shutdown_connections 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55db074b13a0 0x55db079430c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db0751d390 0x55db078cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ec77fe640 0 client.28540 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db074fc6b0 0x55db078cf670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 --2- 172.21.15.94:0/954213420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db074f44d0 0x55db078d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 >> 172.21.15.94:0/954213420 conn(0x55db075114e0 msgr2=0x55db078ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 shutdown_connections 2024-04-25T13:52:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 1 -- 172.21.15.94:0/954213420 wait complete. 2024-04-25T13:52:11.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.965+0000 7f8ef53bb740 20 client.28540 trim_cache size 0 max 16384 2024-04-25T13:52:11.975 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:11.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 msgr2=0x7f8ed00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 0x7f8ed00b2000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00c8480 tx=0x7f8ee00f6ed0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 msgr2=0x7f8ee81a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:11.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00097b0 tx=0x7f8ee0031840 comp rx=0 tx=0).stop 2024-04-25T13:52:11.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 shutdown_connections 2024-04-25T13:52:11.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ed00afaf0 0x7f8ed00b2000 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee8108920 0x7f8ee81a08c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ee8103c20 0x7f8ee8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 --2- 172.21.15.94:0/1730415627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee81029e0 0x7f8ee81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:11.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 >> 172.21.15.94:0/1730415627 conn(0x7f8ee80fdbd0 msgr2=0x7f8ee8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:11.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 shutdown_connections 2024-04-25T13:52:11.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:11.966+0000 7f8eee506640 1 -- 172.21.15.94:0/1730415627 wait complete. 2024-04-25T13:52:11.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:52:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 -- 172.21.15.94:0/822665228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c102a00 msgr2=0x7fb52c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 --2- 172.21.15.94:0/822665228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c102a00 0x7fb52c103700 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 -- 172.21.15.94:0/822665228 shutdown_connections 2024-04-25T13:52:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 --2- 172.21.15.94:0/822665228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c108940 0x7fb52c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 --2- 172.21.15.94:0/822665228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c103c40 0x7fb52c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 --2- 172.21.15.94:0/822665228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c102a00 0x7fb52c103700 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 -- 172.21.15.94:0/822665228 >> 172.21.15.94:0/822665228 conn(0x7fb52c0fdbf0 msgr2=0x7fb52c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 -- 172.21.15.94:0/822665228 shutdown_connections 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.094+0000 7fb53177a640 1 -- 172.21.15.94:0/822665228 wait complete. 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 Processor -- start 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 -- start start 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c102a00 0x7fb52c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c108940 0x7fb52c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10e500 con 0x7fb52c103c40 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10e380 con 0x7fb52c102a00 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb53177a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10e680 con 0x7fb52c108940 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53398/0 (socket says 172.21.15.94:53398) 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 -- 172.21.15.94:0/274788879 learned_addr learned my addr 172.21.15.94:0/274788879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52affd640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c102a00 0x7fb52c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c108940 0x7fb52c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 -- 172.21.15.94:0/274788879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c108940 msgr2=0x7fb52c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c108940 0x7fb52c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 -- 172.21.15.94:0/274788879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c102a00 msgr2=0x7fb52c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c102a00 0x7fb52c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.095+0000 7fb52a7fc640 1 -- 172.21.15.94:0/274788879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c1a1dc0 con 0x7fb52c103c40 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d770 tx=0x7fb51c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb513fff640 1 -- 172.21.15.94:0/274788879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c017070 con 0x7fb52c103c40 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c1a20b0 con 0x7fb52c103c40 2024-04-25T13:52:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb513fff640 1 -- 172.21.15.94:0/274788879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51c0054d0 con 0x7fb52c103c40 2024-04-25T13:52:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb513fff640 1 -- 172.21.15.94:0/274788879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c004c30 con 0x7fb52c103c40 2024-04-25T13:52:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.096+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a2600 con 0x7fb52c103c40 2024-04-25T13:52:12.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.098+0000 7fb53861f740 1 Processor -- start 2024-04-25T13:52:12.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 -- start start 2024-04-25T13:52:12.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb513fff640 1 -- 172.21.15.94:0/274788879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb51c004e10 con 0x7fb52c103c40 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b04074d0 0x55a1b07d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b040f6b0 0x55a1b07d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a1b0736d80 con 0x55a1b0430390 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06f0a20 con 0x55a1b04074d0 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb53861f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b0732d40 con 0x55a1b040f6b0 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53414/0 (socket says 172.21.15.94:53414) 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3025104265 learned_addr learned my addr 172.21.15.94:0/3025104265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52affd640 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b04074d0 0x55a1b07d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3025104265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b040f6b0 msgr2=0x55a1b07d3a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:52:12.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b040f6b0 0x55a1b07d3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3025104265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b04074d0 msgr2=0x55a1b07d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b04074d0 0x55a1b07d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3025104265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b06fbc30 con 0x55a1b0430390 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb524009fd0 tx=0x7fb5240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e74dbc5fa45164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb513fff640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 0x7fb5140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb5117fa640 1 -- 172.21.15.94:0/3025104265 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524004d20 con 0x55a1b0430390 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb5117fa640 1 -- 172.21.15.94:0/3025104265 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524004050 con 0x55a1b0430390 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.099+0000 7fb52affd640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 0x7fb5140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb5117fa640 1 -- 172.21.15.94:0/3025104265 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524005280 con 0x55a1b0430390 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb513fff640 1 -- 172.21.15.94:0/274788879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb51c0de6d0 con 0x7fb52c103c40 2024-04-25T13:52:12.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 -- 172.21.15.94:0/3025104265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 msgr2=0x55a1b07d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb524009fd0 tx=0x7fb5240118d0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb52affd640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 0x7fb5140b2000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb520031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 -- 172.21.15.94:0/3025104265 shutdown_connections 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b040f6b0 0x55a1b07d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b04074d0 0x55a1b07d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 --2- 172.21.15.94:0/3025104265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07d2810 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 -- 172.21.15.94:0/3025104265 >> 172.21.15.94:0/3025104265 conn(0x55a1b04244e0 msgr2=0x55a1b07ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 -- 172.21.15.94:0/3025104265 shutdown_connections 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.100+0000 7fb53861f740 1 -- 172.21.15.94:0/3025104265 wait complete. 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 -- start start 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 Processor -- start 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a1b07d4530 con 0x55a1b0430390 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06bddc0 con 0x55a1b040f6b0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b07d46b0 con 0x55a1b04074d0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb53861f740 10 client.0 _ll_register_callbacks cb 0x55a1b06d9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53426/0 (socket says 172.21.15.94:53426) 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3831221651 learned_addr learned my addr 172.21.15.94:0/3831221651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52affd640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3831221651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 msgr2=0x55a1b07dc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 msgr2=0x55a1b07dcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07dcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3831221651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b04483f0 con 0x55a1b0430390 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52affd640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:12.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.101+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb524003fd0 tx=0x7fb524004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a20f22ce97b15f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524028020 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb524004c50 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb524004050 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb524033050 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 10 client.19278 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1b05127d0 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524004050 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb52400f4e0 con 0x55a1b0430390 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 10 client.19278 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:12.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 10 client.19278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 20 client.19278 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.102+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03eef60 0x55a1b07ea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.103+0000 7fb53861f740 4 client.19278 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.103+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b0744990 con 0x55a1b03eef60 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.103+0000 7fb52affd640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03eef60 0x55a1b07ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.103+0000 7fb52affd640 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03eef60 0x55a1b07ea160 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a01eb185fe5cad server_cookie=8fc6ecb8daa57780 in_seq=0 out_seq=0 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.103+0000 7fb50eff5640 10 client.19278 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb50eff5640 1 -- 172.21.15.94:0/3831221651 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1b0744990 con 0x55a1b03eef60 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb50eff5640 10 client.19278 handle_command_reply: tid=0 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 client.19278 shutdown 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 msgr2=0x55a1b07e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb524003fd0 tx=0x7fb524004430 comp rx=0 tx=0).stop 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 shutdown_connections 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03eef60 0x55a1b07ea160 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07e3ac0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb50eff5640 0 client.19278 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 --2- 172.21.15.94:0/3831221651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 >> 172.21.15.94:0/3831221651 conn(0x55a1b04244e0 msgr2=0x55a1b07cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 shutdown_connections 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 1 -- 172.21.15.94:0/3831221651 wait complete. 2024-04-25T13:52:12.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.104+0000 7fb53861f740 20 client.19278 trim_cache size 0 max 16384 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.118+0000 7fb53861f740 1 Processor -- start 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.118+0000 7fb53861f740 1 -- start start 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.118+0000 7fb53861f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.118+0000 7fb53861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.118+0000 7fb53861f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb53861f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ab5c0 con 0x55a1b0430390 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb53861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ad100 con 0x55a1b040f6b0 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb53861f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1abc90 con 0x55a1b04074d0 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53436/0 (socket says 172.21.15.94:53436) 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 -- 172.21.15.94:0/1439592104 learned_addr learned my addr 172.21.15.94:0/1439592104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52affd640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 -- 172.21.15.94:0/1439592104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 msgr2=0x55a1b07dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 -- 172.21.15.94:0/1439592104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 msgr2=0x55a1b07e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07e39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 -- 172.21.15.94:0/1439592104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b07d2360 con 0x55a1b0430390 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb524009de0 tx=0x7fb524009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9285f1dce84c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb510ff9640 1 -- 172.21.15.94:0/1439592104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524028070 con 0x55a1b0430390 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb510ff9640 1 -- 172.21.15.94:0/1439592104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524004820 con 0x55a1b0430390 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.119+0000 7fb510ff9640 1 -- 172.21.15.94:0/1439592104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5240198e0 con 0x55a1b0430390 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- 172.21.15.94:0/1439592104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 msgr2=0x55a1b07dad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb524009de0 tx=0x7fb524009ea0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- 172.21.15.94:0/1439592104 shutdown_connections 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b0430390 0x55a1b07dad20 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b040f6b0 0x55a1b07e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 --2- 172.21.15.94:0/1439592104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b07dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- 172.21.15.94:0/1439592104 >> 172.21.15.94:0/1439592104 conn(0x55a1b04244e0 msgr2=0x55a1b07dc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- 172.21.15.94:0/1439592104 shutdown_connections 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- 172.21.15.94:0/1439592104 wait complete. 2024-04-25T13:52:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 -- start start 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.120+0000 7fb53861f740 1 Processor -- start 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b084cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b0430390 0x55a1b0854440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1aca30 con 0x55a1b040f6b0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ac360 con 0x55a1b0430390 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1a95a0 con 0x55a1b04074d0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 10 client.0 _ll_register_callbacks cb 0x7fb520009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53444/0 (socket says 172.21.15.94:53444) 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 -- 172.21.15.94:0/392812841 learned_addr learned my addr 172.21.15.94:0/392812841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52b7fe640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b0430390 0x55a1b0854440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52affd640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b084cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 -- 172.21.15.94:0/392812841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 msgr2=0x55a1b084cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b084cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 -- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b0430390 msgr2=0x55a1b0854440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b0430390 0x55a1b0854440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 -- 172.21.15.94:0/392812841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b07d7d80 con 0x55a1b040f6b0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb52a7fc640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0a2c40 tx=0x7fb51c0a63f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26ebc1c7e485e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb50eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0c0060 con 0x55a1b040f6b0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7fb51c0af6f0 con 0x55a1b040f6b0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb524004820 con 0x55a1b040f6b0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb51c020040 con 0x55a1b040f6b0 2024-04-25T13:52:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 10 client.19284 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.121+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1b0855b80 con 0x55a1b040f6b0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524004820 con 0x55a1b040f6b0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb51c0a2530 con 0x55a1b040f6b0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 10 client.19284 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 10 client.19284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 20 client.19284 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03c43a0 0x55a1b0858520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 4 client.19284 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1b073e800 con 0x55a1b03c43a0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb52affd640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03c43a0 0x55a1b0858520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb50eff5640 10 client.19284 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.122+0000 7fb52affd640 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03c43a0 0x55a1b0858520 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6ff93e8eed1f50 server_cookie=de1053dd0ba60acd in_seq=0 out_seq=0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb50eff5640 1 -- 172.21.15.94:0/392812841 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1b073e800 con 0x55a1b03c43a0 2024-04-25T13:52:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb50eff5640 10 client.19284 handle_command_reply: tid=0 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 client.19284 shutdown 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 msgr2=0x55a1b084d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0a2c40 tx=0x7fb51c0a63f0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 shutdown_connections 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a1b03c43a0 0x55a1b0858520 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1b0430390 0x55a1b0854440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1b040f6b0 0x55a1b084d4e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb50eff5640 0 client.19284 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 --2- 172.21.15.94:0/392812841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b04074d0 0x55a1b084cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 >> 172.21.15.94:0/392812841 conn(0x55a1b04244e0 msgr2=0x55a1b07dc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 shutdown_connections 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 1 -- 172.21.15.94:0/392812841 wait complete. 2024-04-25T13:52:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53861f740 20 client.19284 trim_cache size 0 max 16384 2024-04-25T13:52:12.131 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:12.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 msgr2=0x7fb5140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 0x7fb5140b2000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb520031400 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 msgr2=0x7fb52c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00d770 tx=0x7fb51c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 shutdown_connections 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5140afaf0 0x7fb5140b2000 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c108940 0x7fb52c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb52c103c40 0x7fb52c19a7c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 --2- 172.21.15.94:0/274788879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c102a00 0x7fb52c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.123+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 >> 172.21.15.94:0/274788879 conn(0x7fb52c0fdbf0 msgr2=0x7fb52c109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.124+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 shutdown_connections 2024-04-25T13:52:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:12.124+0000 7fb53177a640 1 -- 172.21.15.94:0/274788879 wait complete. 2024-04-25T13:52:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:12 smithi137 ceph-mon[32568]: pgmap v660: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 417 op/s 2024-04-25T13:52:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:12 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/823069640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:12 smithi094 ceph-mon[29195]: pgmap v660: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 417 op/s 2024-04-25T13:52:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:12 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/823069640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:12 smithi138 ceph-mon[32977]: pgmap v660: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 417 op/s 2024-04-25T13:52:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/823069640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:12.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1280660 12.70 MB/sec execute 415 sec latency 22.112 ms 2024-04-25T13:52:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 -- 172.21.15.94:0/521430381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c103c60 msgr2=0x7f5f1c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 --2- 172.21.15.94:0/521430381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c103c60 0x7f5f1c104070 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f1002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 -- 172.21.15.94:0/521430381 shutdown_connections 2024-04-25T13:52:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 --2- 172.21.15.94:0/521430381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c108960 0x7f5f1c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 --2- 172.21.15.94:0/521430381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c103c60 0x7f5f1c104070 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 --2- 172.21.15.94:0/521430381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 0x7f5f1c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 -- 172.21.15.94:0/521430381 >> 172.21.15.94:0/521430381 conn(0x7f5f1c0fdc10 msgr2=0x7f5f1c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 -- 172.21.15.94:0/521430381 shutdown_connections 2024-04-25T13:52:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.836+0000 7f5f20c51640 1 -- 172.21.15.94:0/521430381 wait complete. 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 Processor -- start 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 -- start start 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 0x7f5f1c06ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c103c60 0x7f5f1c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10e520 con 0x7f5f1c108960 2024-04-25T13:52:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10e3a0 con 0x7f5f1c102a20 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f20c51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c10e6a0 con 0x7f5f1c103c60 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53468/0 (socket says 172.21.15.94:53468) 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/2753457611 learned_addr learned my addr 172.21.15.94:0/2753457611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.837+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 0x7f5f1c06ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c103c60 0x7f5f1c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/2753457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c103c60 msgr2=0x7f5f1c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c103c60 0x7f5f1c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/2753457611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 msgr2=0x7f5f1c06ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 0x7f5f1c06ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/2753457611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c1a1cb0 con 0x7f5f1c108960 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400cce0 tx=0x7f5f1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f0b7fe640 1 -- 172.21.15.94:0/2753457611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400f450 con 0x7f5f1c108960 2024-04-25T13:52:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f0b7fe640 1 -- 172.21.15.94:0/2753457611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1400faa0 con 0x7f5f1c108960 2024-04-25T13:52:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c1a1fa0 con 0x7f5f1c108960 2024-04-25T13:52:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f0b7fe640 1 -- 172.21.15.94:0/2753457611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14018b70 con 0x7f5f1c108960 2024-04-25T13:52:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.838+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1a2400 con 0x7f5f1c108960 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f0b7fe640 1 -- 172.21.15.94:0/2753457611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5f14018d10 con 0x7f5f1c108960 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 Processor -- start 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 -- start start 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 0x5598f3993970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35d24d0 0x5598f399adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5598f38f9460 con 0x5598f35d24d0 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598f38b3590 con 0x5598f35f31b0 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f27be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598f38f5420 con 0x5598f35ca2f0 2024-04-25T13:52:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f19d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 0x5598f3993970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f1a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f1a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36298/0 (socket says 172.21.15.94:36298) 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f19d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 0x5598f3993970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:50178/0 (socket says 172.21.15.94:50178) 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f1a59c640 1 -- 172.21.15.94:0/513739571 learned_addr learned my addr 172.21.15.94:0/513739571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.841+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35d24d0 0x5598f399adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 -- 172.21.15.94:0/513739571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 msgr2=0x5598f3993970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 0x5598f3993970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 -- 172.21.15.94:0/513739571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35d24d0 msgr2=0x5598f399adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35d24d0 0x5598f399adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 -- 172.21.15.94:0/513739571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f38c00d0 con 0x5598f35ca2f0 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f0b7fe640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 0x7f5f040b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 0x7f5f040b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009c60 tx=0x7f5f0c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f6cd16f04d6726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f0b7fe640 1 -- 172.21.15.94:0/2753457611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5f140deff0 con 0x7f5f1c108960 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f08ff9640 1 -- 172.21.15.94:0/513739571 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c009e10 con 0x5598f35ca2f0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f08ff9640 1 -- 172.21.15.94:0/513739571 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f0c005020 con 0x5598f35ca2f0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.842+0000 7f5f08ff9640 1 -- 172.21.15.94:0/513739571 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0049e0 con 0x5598f35ca2f0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- 172.21.15.94:0/513739571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 msgr2=0x5598f3993ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 0x7f5f040b2070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009c60 tx=0x7f5f0c011810 comp rx=0 tx=0).stop 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- 172.21.15.94:0/513739571 shutdown_connections 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35d24d0 0x5598f399adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f3993ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 --2- 172.21.15.94:0/513739571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35f31b0 0x5598f3993970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- 172.21.15.94:0/513739571 >> 172.21.15.94:0/513739571 conn(0x5598f35e7300 msgr2=0x5598f398f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- 172.21.15.94:0/513739571 shutdown_connections 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- 172.21.15.94:0/513739571 wait complete. 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.843+0000 7f5f27be8740 1 -- start start 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 Processor -- start 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35ca2f0 0x5598f399d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f399db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5598f399b840 con 0x5598f35f31b0 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598f3880b60 con 0x5598f35ca2f0 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598f399b9c0 con 0x5598f35d24d0 2024-04-25T13:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f27be8740 10 client.0 _ll_register_callbacks cb 0x5598f38a52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53476/0 (socket says 172.21.15.94:53476) 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/280339719 learned_addr learned my addr 172.21.15.94:0/280339719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35ca2f0 0x5598f399d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f399db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/280339719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 msgr2=0x5598f399db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f399db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35ca2f0 msgr2=0x5598f399d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35ca2f0 0x5598f399d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 -- 172.21.15.94:0/280339719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f360b210 con 0x5598f35f31b0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5f140a73c0 tx=0x7f5f140a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8641c9647914f9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.844+0000 7f5f01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f140a7430 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5f140afe00 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f0c005020 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f140ab7e0 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 10 client.19296 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598f36d7620 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f0c005020 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5f140a6690 con 0x5598f35f31b0 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 10 client.19296 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 10 client.19296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 20 client.19296 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35b1d80 0x5598f39ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 4 client.19296 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f3907070 con 0x5598f35b1d80 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.845+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35b1d80 0x5598f39ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.846+0000 7f5f01ffb640 10 client.19296 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.846+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35b1d80 0x5598f39ab2f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c329f668e33c0c62 server_cookie=82e7af1be5960244 in_seq=0 out_seq=0 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f01ffb640 1 -- 172.21.15.94:0/280339719 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598f3907070 con 0x5598f35b1d80 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f01ffb640 10 client.19296 handle_command_reply: tid=0 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 client.19296 shutdown 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 msgr2=0x5598f39a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5f140a73c0 tx=0x7f5f140a6ba0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 shutdown_connections 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35b1d80 0x5598f39ab2f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35f31b0 0x5598f39a4a80 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f399db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 --2- 172.21.15.94:0/280339719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35ca2f0 0x5598f399d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f01ffb640 0 client.19296 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 >> 172.21.15.94:0/280339719 conn(0x5598f35e7300 msgr2=0x5598f3992210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 shutdown_connections 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 1 -- 172.21.15.94:0/280339719 wait complete. 2024-04-25T13:52:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.848+0000 7f5f27be8740 20 client.19296 trim_cache size 0 max 16384 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 Processor -- start 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 -- start start 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35b1d80 0x5598f398ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f398f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1ab3c0 con 0x5598f35ca2f0 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1acf00 con 0x5598f35b1d80 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f27be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1aba90 con 0x5598f35d24d0 2024-04-25T13:52:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53482/0 (socket says 172.21.15.94:53482) 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 -- 172.21.15.94:0/4173537315 learned_addr learned my addr 172.21.15.94:0/4173537315 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35b1d80 0x5598f398ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f398f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 -- 172.21.15.94:0/4173537315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 msgr2=0x5598f398f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f398f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 -- 172.21.15.94:0/4173537315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35b1d80 msgr2=0x5598f398ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35b1d80 0x5598f398ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.865+0000 7f5f19d9b640 1 -- 172.21.15.94:0/4173537315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f39a4ee0 con 0x5598f35ca2f0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.866+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5ec5b126921c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.866+0000 7f5f03fff640 1 -- 172.21.15.94:0/4173537315 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10035040 con 0x5598f35ca2f0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.866+0000 7f5f03fff640 1 -- 172.21.15.94:0/4173537315 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f100035f0 con 0x5598f35ca2f0 2024-04-25T13:52:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.866+0000 7f5f03fff640 1 -- 172.21.15.94:0/4173537315 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1003d450 con 0x5598f35ca2f0 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- 172.21.15.94:0/4173537315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 msgr2=0x5598f39a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f10009260 comp rx=0 tx=0).stop 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- 172.21.15.94:0/4173537315 shutdown_connections 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35d24d0 0x5598f398f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35ca2f0 0x5598f39a49a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- 172.21.15.94:0/4173537315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35b1d80 0x5598f398ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- 172.21.15.94:0/4173537315 >> 172.21.15.94:0/4173537315 conn(0x5598f35e7300 msgr2=0x5598f3997a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- 172.21.15.94:0/4173537315 shutdown_connections 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- 172.21.15.94:0/4173537315 wait complete. 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- start start 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 Processor -- start 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f39a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35d24d0 0x5598f39a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1ac830 con 0x5598f35b1d80 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1ac160 con 0x5598f35d24d0 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.867+0000 7f5f27be8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1a93a0 con 0x5598f35ca2f0 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f27be8740 10 client.0 _ll_register_callbacks cb 0x7f5f140077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53498/0 (socket says 172.21.15.94:53498) 2024-04-25T13:52:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 -- 172.21.15.94:0/3325864414 learned_addr learned my addr 172.21.15.94:0/3325864414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1ad9d640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35d24d0 0x5598f39a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f39a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 -- 172.21.15.94:0/3325864414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 msgr2=0x5598f39a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f39a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 -- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35d24d0 msgr2=0x5598f39a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35d24d0 0x5598f39a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 -- 172.21.15.94:0/3325864414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f39a3620 con 0x5598f35b1d80 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f1a59c640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c01b040 tx=0x7f5f0c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd146eae1f86f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c01d040 con 0x5598f35b1d80 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f5f0c016040 con 0x5598f35b1d80 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f100035f0 con 0x5598f35b1d80 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c0097a0 con 0x5598f35b1d80 2024-04-25T13:52:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f27be8740 10 client.19302 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.868+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598f39a4920 con 0x5598f35b1d80 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f100035f0 con 0x5598f35b1d80 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5f0c02f070 con 0x5598f35b1d80 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 10 client.19302 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 10 client.19302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 20 client.19302 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35f31b0 0x5598f3a159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 4 client.19302 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598f3900ee0 con 0x5598f35f31b0 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35f31b0 0x5598f3a159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f01ffb640 10 client.19302 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:12.869+0000 7f5f19d9b640 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35f31b0 0x5598f3a159f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5db3a96205d6f45 server_cookie=921ee09c09df2285 in_seq=0 out_seq=0 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f01ffb640 1 -- 172.21.15.94:0/3325864414 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598f3900ee0 con 0x5598f35f31b0 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f01ffb640 10 client.19302 handle_command_reply: tid=0 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 client.19302 shutdown 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 msgr2=0x5598f39a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c01b040 tx=0x7f5f0c009f80 comp rx=0 tx=0).stop 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 shutdown_connections 2024-04-25T13:52:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5598f35f31b0 0x5598f3a159f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f35d24d0 0x5598f39a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f35ca2f0 0x5598f39a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 --2- 172.21.15.94:0/3325864414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598f35b1d80 0x5598f39a1cb0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f01ffb640 0 client.19302 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 >> 172.21.15.94:0/3325864414 conn(0x5598f35e7300 msgr2=0x5598f3992100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 shutdown_connections 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.053+0000 7f5f27be8740 1 -- 172.21.15.94:0/3325864414 wait complete. 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f27be8740 20 client.19302 trim_cache size 0 max 16384 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:52:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:52:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:52:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:52:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 msgr2=0x7f5f040b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 0x7f5f040b2070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10031400 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 msgr2=0x7f5f1c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400cce0 tx=0x7f5f1400a620 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 shutdown_connections 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f040afb60 0x7f5f040b2070 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f1c108960 0x7f5f1c1a1520 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1c103c60 0x7f5f1c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 --2- 172.21.15.94:0/2753457611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f1c102a20 0x7f5f1c06ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 >> 172.21.15.94:0/2753457611 conn(0x7f5f1c0fdc10 msgr2=0x7f5f1c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 shutdown_connections 2024-04-25T13:52:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:13.054+0000 7f5f20c51640 1 -- 172.21.15.94:0/2753457611 wait complete. 2024-04-25T13:52:13.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:52:13.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:13.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2570970252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 msgr2=0x7f82c410ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2570970252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c410ad50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ca0 tx=0x7f82bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2570970252 shutdown_connections 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2570970252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c410ad50 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2570970252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c4103c30 0x7f82c4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2570970252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c41029f0 0x7f82c41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2570970252 >> 172.21.15.94:0/2570970252 conn(0x7f82c40fdbc0 msgr2=0x7f82c4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2570970252 shutdown_connections 2024-04-25T13:52:13.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.257+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2570970252 wait complete. 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 Processor -- start 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 -- start start 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 0x7f82c40757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 0x7f82c419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e500 con 0x7f82c4108930 2024-04-25T13:52:13.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e380 con 0x7f82c41029f0 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c8d5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e680 con 0x7f82c4103c30 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 0x7f82c40757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53504/0 (socket says 172.21.15.94:53504) 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c2d9d640 1 -- 172.21.15.94:0/2898007293 learned_addr learned my addr 172.21.15.94:0/2898007293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.258+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 0x7f82c419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 -- 172.21.15.94:0/2898007293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 msgr2=0x7f82c419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 0x7f82c419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 -- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 msgr2=0x7f82c41a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 -- 172.21.15.94:0/2898007293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a1d30 con 0x7f82c41029f0 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:13.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 0x7f82c419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c259c640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 0x7f82c40757d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82c41022c0 tx=0x7f82b400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82b37fe640 1 -- 172.21.15.94:0/2898007293 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b400cd00 con 0x7f82c41029f0 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82b37fe640 1 -- 172.21.15.94:0/2898007293 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b40108e0 con 0x7f82c41029f0 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a1fc0 con 0x7f82c41029f0 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82b37fe640 1 -- 172.21.15.94:0/2898007293 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40188c0 con 0x7f82c41029f0 2024-04-25T13:52:13.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.259+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a2480 con 0x7f82c41029f0 2024-04-25T13:52:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 Processor -- start 2024-04-25T13:52:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 -- start start 2024-04-25T13:52:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a6f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561c769cdb00 con 0x561c766c7410 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c76969920 con 0x561c766a6730 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82cfc10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c769c9ac0 con 0x561c7669e550 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82b37fe640 1 -- 172.21.15.94:0/2898007293 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f82b401f020 con 0x7f82c41029f0 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a6f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82c2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53514/0 (socket says 172.21.15.94:53514) 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82c2d9d640 1 -- 172.21.15.94:0/1890797727 learned_addr learned my addr 172.21.15.94:0/1890797727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.262+0000 7f82c259c640 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1890797727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 msgr2=0x561c76a68a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a68a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1890797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 msgr2=0x561c76a684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1890797727 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c767a77e0 con 0x561c766a6730 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a6f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fa0 tx=0x7f82b80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8938a037a490dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82b37fe640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 0x7f82ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82b0ff9640 1 -- 172.21.15.94:0/1890797727 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8004030 con 0x561c766a6730 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 0x7f82ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82b0ff9640 1 -- 172.21.15.94:0/1890797727 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b80054d0 con 0x561c766a6730 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82b0ff9640 1 -- 172.21.15.94:0/1890797727 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8004ca0 con 0x561c766a6730 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.263+0000 7f82b37fe640 1 -- 172.21.15.94:0/2898007293 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f82b40ded90 con 0x7f82c41029f0 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 0x7f82ac0b2000 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x561c76a69670 tx=0x7f82bc031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 -- 172.21.15.94:0/1890797727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 msgr2=0x561c76a6f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a6f920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fa0 tx=0x7f82b80118d0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 -- 172.21.15.94:0/1890797727 shutdown_connections 2024-04-25T13:52:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a6f920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a68a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1890797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766c7410 0x561c76a684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 -- 172.21.15.94:0/1890797727 >> 172.21.15.94:0/1890797727 conn(0x561c766bb560 msgr2=0x561c76a63e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.264+0000 7f82cfc10740 1 -- 172.21.15.94:0/1890797727 shutdown_connections 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 -- 172.21.15.94:0/1890797727 wait complete. 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 -- start start 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 Processor -- start 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766c7410 0x561c76a79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561c76a70370 con 0x561c766a6730 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c769675d0 con 0x561c766c7410 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c76a704f0 con 0x561c7669e550 2024-04-25T13:52:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82cfc10740 10 client.0 _ll_register_callbacks cb 0x561c76955080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.265+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36366/0 (socket says 172.21.15.94:36366) 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 -- 172.21.15.94:0/2008952810 learned_addr learned my addr 172.21.15.94:0/2008952810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766c7410 0x561c76a79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 -- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766c7410 msgr2=0x561c76a79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766c7410 0x561c76a79770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 -- 172.21.15.94:0/2008952810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 msgr2=0x561c76a72810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a72810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 -- 172.21.15.94:0/2008952810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c76807f70 con 0x561c7669e550 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82c259c640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c6a0 tx=0x7f82b40a3150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b687010fd89c7bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40a6ce0 con 0x561c7669e550 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f82b4010a90 con 0x561c7669e550 2024-04-25T13:52:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b80054d0 con 0x561c7669e550 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.266+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82b40a78d0 con 0x561c7669e550 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 10 client.28576 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c7669f170 con 0x561c7669e550 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b80054d0 con 0x561c7669e550 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f82b40afba0 con 0x561c7669e550 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 10 client.28576 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 10 client.28576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 20 client.28576 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c766861f0 0x561c76a7fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 4 client.28576 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c769db710 con 0x561c766861f0 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.267+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c766861f0 0x561c76a7fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.268+0000 7f82a9ffb640 10 client.28576 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.268+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c766861f0 0x561c76a7fdd0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eced178dcf9a3e36 server_cookie=ab84e71d16b16282 in_seq=0 out_seq=0 2024-04-25T13:52:13.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2008952810 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c769db710 con 0x561c766861f0 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82a9ffb640 10 client.28576 handle_command_reply: tid=0 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 client.28576 shutdown 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 msgr2=0x561c76a722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c6a0 tx=0x7f82b40a3150 comp rx=0 tx=0).stop 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 shutdown_connections 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c766861f0 0x561c76a7fdd0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766c7410 0x561c76a79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a72810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2008952810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c7669e550 0x561c76a722d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 >> 172.21.15.94:0/2008952810 conn(0x561c766bb560 msgr2=0x561c76a62ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82a9ffb640 0 client.28576 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 shutdown_connections 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 1 -- 172.21.15.94:0/2008952810 wait complete. 2024-04-25T13:52:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.270+0000 7f82cfc10740 20 client.28576 trim_cache size 0 max 16384 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 Processor -- start 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 -- start start 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7669e550 0x561c76a70890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ab550 con 0x561c766a6730 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ad090 con 0x561c7669e550 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82cfc10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41abc20 con 0x561c766c7410 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53542/0 (socket says 172.21.15.94:53542) 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1591361484 learned_addr learned my addr 172.21.15.94:0/1591361484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c259c640 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7669e550 0x561c76a70890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1591361484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 msgr2=0x561c76a84450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a84450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1591361484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7669e550 msgr2=0x561c76a70890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7669e550 0x561c76a70890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 -- 172.21.15.94:0/1591361484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c76a69290 con 0x561c766a6730 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.290+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fa0 tx=0x7f82b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adda5256f94111ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.291+0000 7f82abfff640 1 -- 172.21.15.94:0/1591361484 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8032070 con 0x561c766a6730 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.291+0000 7f82abfff640 1 -- 172.21.15.94:0/1591361484 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b802c070 con 0x561c766a6730 2024-04-25T13:52:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.291+0000 7f82abfff640 1 -- 172.21.15.94:0/1591361484 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b801d040 con 0x561c766a6730 2024-04-25T13:52:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.291+0000 7f82cfc10740 1 -- 172.21.15.94:0/1591361484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 msgr2=0x561c76a70dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009fa0 tx=0x7f82b8005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- 172.21.15.94:0/1591361484 shutdown_connections 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a84450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c766a6730 0x561c76a70dd0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- 172.21.15.94:0/1591361484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c7669e550 0x561c76a70890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- 172.21.15.94:0/1591361484 >> 172.21.15.94:0/1591361484 conn(0x561c766bb560 msgr2=0x561c76a6c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- 172.21.15.94:0/1591361484 shutdown_connections 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- 172.21.15.94:0/1591361484 wait complete. 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- start start 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 Processor -- start 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a76890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a77700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ac9c0 con 0x561c7669e550 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ac2f0 con 0x561c766a6730 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41a9530 con 0x561c766c7410 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82cfc10740 10 client.0 _ll_register_callbacks cb 0x7f82b400cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53550/0 (socket says 172.21.15.94:53550) 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82c259c640 1 -- 172.21.15.94:0/2867697201 learned_addr learned my addr 172.21.15.94:0/2867697201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.292+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a76890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c2d9d640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 -- 172.21.15.94:0/2867697201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 msgr2=0x561c76a77700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a77700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 -- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 msgr2=0x561c76a76890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a76890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 -- 172.21.15.94:0/2867697201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c76a77cc0 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82c259c640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f82b40095f0 tx=0x7f82b40ab3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b4647369c5096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40a6b20 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f82b40afba0 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b802c070 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82b40c8050 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82cfc10740 10 client.19320 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c76a78ff0 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b802c070 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.293+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f82b40a2b40 con 0x561c7669e550 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 10 client.19320 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 10 client.19320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 20 client.19320 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c7665b630 0x561c76aea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 4 client.19320 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c769d5580 con 0x561c7665b630 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c7665b630 0x561c76aea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82a9ffb640 10 client.19320 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.294+0000 7f82c1d9b640 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c7665b630 0x561c76aea6c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e4e67b0c1de46a server_cookie=c922e3ebfebb0506 in_seq=0 out_seq=0 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.295+0000 7f82a9ffb640 1 -- 172.21.15.94:0/2867697201 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c769d5580 con 0x561c7665b630 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.295+0000 7f82a9ffb640 10 client.19320 handle_command_reply: tid=0 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.295+0000 7f82cfc10740 1 client.19320 shutdown 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 msgr2=0x561c76a76350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f82b40095f0 tx=0x7f82b40ab3e0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 shutdown_connections 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561c7665b630 0x561c76aea6c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c766c7410 0x561c76a77700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82a9ffb640 0 client.19320 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c766a6730 0x561c76a76890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 --2- 172.21.15.94:0/2867697201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c7669e550 0x561c76a76350 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 >> 172.21.15.94:0/2867697201 conn(0x561c766bb560 msgr2=0x561c76a70890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 shutdown_connections 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 1 -- 172.21.15.94:0/2867697201 wait complete. 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82cfc10740 20 client.19320 trim_cache size 0 max 16384 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:52:13.303 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "711d06c7-846e-42d9-b13c-10b54e323157", 2024-04-25T13:52:13.303 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:52:13.303 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 msgr2=0x7f82ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 0x7f82ac0b2000 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x561c76a69670 tx=0x7f82bc031af0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 msgr2=0x7f82c40757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 0x7f82c40757d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82c41022c0 tx=0x7f82b400ee50 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 shutdown_connections 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82ac0afaf0 0x7f82ac0b2000 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82c4108930 0x7f82c41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82c4103c30 0x7f82c419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 --2- 172.21.15.94:0/2898007293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c41029f0 0x7f82c40757d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 >> 172.21.15.94:0/2898007293 conn(0x7f82c40fdbc0 msgr2=0x7f82c4104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 shutdown_connections 2024-04-25T13:52:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.296+0000 7f82c8d5e640 1 -- 172.21.15.94:0/2898007293 wait complete. 2024-04-25T13:52:13.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:52:13.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 -- 172.21.15.94:0/1185743348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c103c30 msgr2=0x7f977c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 --2- 172.21.15.94:0/1185743348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c103c30 0x7f977c104040 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 -- 172.21.15.94:0/1185743348 shutdown_connections 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 --2- 172.21.15.94:0/1185743348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c108930 0x7f977c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 --2- 172.21.15.94:0/1185743348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c103c30 0x7f977c104040 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 --2- 172.21.15.94:0/1185743348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c1029f0 0x7f977c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 -- 172.21.15.94:0/1185743348 >> 172.21.15.94:0/1185743348 conn(0x7f977c0fdc00 msgr2=0x7f977c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 -- 172.21.15.94:0/1185743348 shutdown_connections 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.425+0000 7f978112e640 1 -- 172.21.15.94:0/1185743348 wait complete. 2024-04-25T13:52:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 Processor -- start 2024-04-25T13:52:13.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 -- start start 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c103c30 0x7f977c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c108930 0x7f977c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.426+0000 7f978112e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10e570 con 0x7f977c1029f0 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f978112e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10e3f0 con 0x7f977c103c30 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f978112e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10e6f0 con 0x7f977c108930 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53562/0 (socket says 172.21.15.94:53562) 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 -- 172.21.15.94:0/3662983282 learned_addr learned my addr 172.21.15.94:0/3662983282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977a59c640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c103c30 0x7f977c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977b59e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c108930 0x7f977c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 -- 172.21.15.94:0/3662983282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c108930 msgr2=0x7f977c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c108930 0x7f977c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 -- 172.21.15.94:0/3662983282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c103c30 msgr2=0x7f977c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c103c30 0x7f977c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 -- 172.21.15.94:0/3662983282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a1d00 con 0x7f977c1029f0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f977ad9d640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c970 tx=0x7f976c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f9763fff640 1 -- 172.21.15.94:0/3662983282 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c007c70 con 0x7f977c1029f0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f9763fff640 1 -- 172.21.15.94:0/3662983282 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f976c007e10 con 0x7f977c1029f0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a1ff0 con 0x7f977c1029f0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f9763fff640 1 -- 172.21.15.94:0/3662983282 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c018760 con 0x7f977c1029f0 2024-04-25T13:52:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.427+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a24d0 con 0x7f977c1029f0 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 Processor -- start 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 -- start start 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc739fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc73ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558fdc69ed80 con 0x558fdc36f4d0 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558fdc6588b0 con 0x558fdc398390 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f9787fd9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558fdc69ad40 con 0x558fdc3776b0 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.430+0000 7f977b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:50262/0 (socket says 172.21.15.94:50262) 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 -- 172.21.15.94:0/395663562 learned_addr learned my addr 172.21.15.94:0/395663562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977a59c640 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc73ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f9763fff640 1 -- 172.21.15.94:0/3662983282 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f976c00fab0 con 0x7f977c1029f0 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 -- 172.21.15.94:0/395663562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 msgr2=0x558fdc73ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc73ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 -- 172.21.15.94:0/395663562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 msgr2=0x558fdc739fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc739fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 -- 172.21.15.94:0/395663562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fdc629a40 con 0x558fdc398390 2024-04-25T13:52:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f977b59e640 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9774009f20 tx=0x7f977400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebacf97fa3b0213c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f97617fa640 1 -- 172.21.15.94:0/395663562 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977401a070 con 0x558fdc398390 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f9763fff640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 0x7f97640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f97617fa640 1 -- 172.21.15.94:0/395663562 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97740158c0 con 0x558fdc398390 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.431+0000 7f97617fa640 1 -- 172.21.15.94:0/395663562 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400a820 con 0x558fdc398390 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f977ad9d640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 0x7f97640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9763fff640 1 -- 172.21.15.94:0/3662983282 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f976c0ded70 con 0x7f977c1029f0 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f977ad9d640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 0x7f97640b2000 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f976c0a6eb0 tx=0x7f976c074a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 -- 172.21.15.94:0/395663562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 msgr2=0x558fdc739a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9774009f20 tx=0x7f977400c8b0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 -- 172.21.15.94:0/395663562 shutdown_connections 2024-04-25T13:52:13.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc73ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc739fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 --2- 172.21.15.94:0/395663562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc398390 0x558fdc739a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 -- 172.21.15.94:0/395663562 >> 172.21.15.94:0/395663562 conn(0x558fdc38c4e0 msgr2=0x558fdc735270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 -- 172.21.15.94:0/395663562 shutdown_connections 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.432+0000 7f9787fd9740 1 -- 172.21.15.94:0/395663562 wait complete. 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 -- start start 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 Processor -- start 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 0x558fdc743960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 0x558fdc74ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558fdc73b780 con 0x558fdc398390 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558fdc625d10 con 0x558fdc36f4d0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558fdc73b900 con 0x558fdc3776b0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f9787fd9740 10 client.0 _ll_register_callbacks cb 0x558fdc652210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 0x558fdc74ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 0x558fdc743960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36414/0 (socket says 172.21.15.94:36414) 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 0x558fdc743960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:50278/0 (socket says 172.21.15.94:50278) 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.433+0000 7f977a59c640 1 -- 172.21.15.94:0/3200264287 learned_addr learned my addr 172.21.15.94:0/3200264287 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 -- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 msgr2=0x558fdc743960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 0x558fdc743960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 -- 172.21.15.94:0/3200264287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 msgr2=0x558fdc74ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 0x558fdc74ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 -- 172.21.15.94:0/3200264287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fdc47a7d0 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977b59e640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 0x558fdc74ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f977a59c640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f977002a930 tx=0x7f97700317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172bd27e1f61abd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f975eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035040 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f9770004450 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97740158c0 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97700035f0 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f9787fd9740 10 client.28597 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.434+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fdc3b03f0 con 0x558fdc3776b0 2024-04-25T13:52:13.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97740158c0 con 0x558fdc3776b0 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f977003c7b0 con 0x558fdc3776b0 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 10 client.28597 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 10 client.28597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 20 client.28597 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc356f60 0x558fdc751220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 4 client.28597 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.435+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fdc6ac990 con 0x558fdc356f60 2024-04-25T13:52:13.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.437+0000 7f977b59e640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc356f60 0x558fdc751220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f975eff5640 10 client.28597 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.438+0000 7f977b59e640 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc356f60 0x558fdc751220 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76bb1d5612515ac server_cookie=24f2b0315e50aa5c in_seq=0 out_seq=0 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f975eff5640 1 -- 172.21.15.94:0/3200264287 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558fdc6ac990 con 0x558fdc356f60 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f975eff5640 10 client.28597 handle_command_reply: tid=0 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 client.28597 shutdown 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 msgr2=0x558fdc743ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f977002a930 tx=0x7f97700317d0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 shutdown_connections 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc356f60 0x558fdc751220 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc398390 0x558fdc74ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc3776b0 0x558fdc743ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f975eff5640 0 client.28597 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 --2- 172.21.15.94:0/3200264287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc36f4d0 0x558fdc743960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 >> 172.21.15.94:0/3200264287 conn(0x558fdc38c4e0 msgr2=0x558fdc7343f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.439+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 shutdown_connections 2024-04-25T13:52:13.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.440+0000 7f9787fd9740 1 -- 172.21.15.94:0/3200264287 wait complete. 2024-04-25T13:52:13.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.440+0000 7f9787fd9740 20 client.28597 trim_cache size 0 max 16384 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 Processor -- start 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 -- start start 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc741f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc74acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1ab5b0 con 0x558fdc36f4d0 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1ad0f0 con 0x558fdc3776b0 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9787fd9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1abc80 con 0x558fdc356f60 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37352/0 (socket says 172.21.15.94:37352) 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 -- 172.21.15.94:0/807957964 learned_addr learned my addr 172.21.15.94:0/807957964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977b59e640 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc741f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977b59e640 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc74acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 -- 172.21.15.94:0/807957964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 msgr2=0x558fdc741f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc741f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 -- 172.21.15.94:0/807957964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 msgr2=0x558fdc74acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc74acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 -- 172.21.15.94:0/807957964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fdc3700f0 con 0x558fdc36f4d0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f977a59c640 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f977003cc00 tx=0x7f977002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3585ff870a7eac39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.458+0000 7f9760ff9640 1 -- 172.21.15.94:0/807957964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035030 con 0x558fdc36f4d0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9760ff9640 1 -- 172.21.15.94:0/807957964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9770054070 con 0x558fdc36f4d0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9760ff9640 1 -- 172.21.15.94:0/807957964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770003a80 con 0x558fdc36f4d0 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 -- 172.21.15.94:0/807957964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 msgr2=0x558fdc742440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f977003cc00 tx=0x7f977002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 -- 172.21.15.94:0/807957964 shutdown_connections 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc74acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc742440 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 --2- 172.21.15.94:0/807957964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc741f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 -- 172.21.15.94:0/807957964 >> 172.21.15.94:0/807957964 conn(0x558fdc38c4e0 msgr2=0x558fdc734ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 -- 172.21.15.94:0/807957964 shutdown_connections 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.459+0000 7f9787fd9740 1 -- 172.21.15.94:0/807957964 wait complete. 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 -- start start 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 Processor -- start 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc74b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc748450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1aca20 con 0x558fdc36f4d0 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1ac350 con 0x558fdc3776b0 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1a9590 con 0x558fdc356f60 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f9787fd9740 10 client.0 _ll_register_callbacks cb 0x7f9770031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37362/0 (socket says 172.21.15.94:37362) 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 -- 172.21.15.94:0/4097001858 learned_addr learned my addr 172.21.15.94:0/4097001858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977b59e640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc74b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977b59e640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc748450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 -- 172.21.15.94:0/4097001858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 msgr2=0x558fdc74b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc74b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 -- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 msgr2=0x558fdc748450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.460+0000 7f977a59c640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc748450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f977a59c640 1 -- 172.21.15.94:0/4097001858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fdc65f4b0 con 0x558fdc36f4d0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f977a59c640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f977003cb50 tx=0x7f9770031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95319fe5052ccbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035070 con 0x558fdc36f4d0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8385+0+0 (secure 0 0 0) 0x7f9770004450 con 0x558fdc36f4d0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9770054070 con 0x558fdc36f4d0 2024-04-25T13:52:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9770068050 con 0x558fdc36f4d0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 10 client.19341 fetch_fsmap learned FSMap version 29 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fdc7b8420 con 0x558fdc36f4d0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9770054070 con 0x558fdc36f4d0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f977004f8d0 con 0x558fdc36f4d0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 10 client.19341 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 20 client.19341 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc398390 0x558fdc7bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 4 client.19341 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.461+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fdc6a6800 con 0x558fdc398390 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.462+0000 7f977b59e640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc398390 0x558fdc7bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.462+0000 7f977b59e640 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc398390 0x558fdc7bade0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=189c16f97ce0ab84 server_cookie=efc1fb5d11b707f3 in_seq=0 out_seq=0 2024-04-25T13:52:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.462+0000 7f975eff5640 10 client.19341 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f975eff5640 1 -- 172.21.15.94:0/4097001858 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558fdc6a6800 con 0x558fdc398390 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f975eff5640 10 client.19341 handle_command_reply: tid=0 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 client.19341 shutdown 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 msgr2=0x558fdc747f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f977003cb50 tx=0x7f9770031df0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 shutdown_connections 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558fdc398390 0x558fdc7bade0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fdc3776b0 0x558fdc748450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fdc36f4d0 0x558fdc747f10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f975eff5640 0 client.19341 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 --2- 172.21.15.94:0/4097001858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558fdc356f60 0x558fdc74b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 >> 172.21.15.94:0/4097001858 conn(0x558fdc38c4e0 msgr2=0x558fdc738070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 shutdown_connections 2024-04-25T13:52:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 1 -- 172.21.15.94:0/4097001858 wait complete. 2024-04-25T13:52:13.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f9787fd9740 20 client.19341 trim_cache size 0 max 16384 2024-04-25T13:52:13.470 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:13.470 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: "711d06c7-846e-42d9-b13c-10b54e323157": { 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "711d06c7-846e-42d9-b13c-10b54e323157", 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:52:13.471 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 msgr2=0x7f97640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 0x7f97640b2000 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f976c0a6eb0 tx=0x7f976c074a60 comp rx=0 tx=0).stop 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 msgr2=0x7f977c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.463+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c970 tx=0x7f976c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 shutdown_connections 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97640afaf0 0x7f97640b2000 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f977c108930 0x7f977c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f977c103c30 0x7f977c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 --2- 172.21.15.94:0/3662983282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f977c1029f0 0x7f977c06bd20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 >> 172.21.15.94:0/3662983282 conn(0x7f977c0fdc00 msgr2=0x7f977c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 shutdown_connections 2024-04-25T13:52:13.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:13.464+0000 7f978112e640 1 -- 172.21.15.94:0/3662983282 wait complete. 2024-04-25T13:52:13.475 INFO:tasks.cephfs.filesystem:scrub status for tag:711d06c7-846e-42d9-b13c-10b54e323157 - {'path': '/', 'tag': '711d06c7-846e-42d9-b13c-10b54e323157', 'options': 'recursive,force'} 2024-04-25T13:52:13.746 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:52:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T13:52:13.746 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:52:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T13:52:13.747 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:52:13 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/2278660361 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afc195fa40 con 0x55afc19a7a80 2024-04-25T13:52:13.747 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:52:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T13:52:13.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1282450 12.70 MB/sec execute 416 sec latency 103.844 ms 2024-04-25T13:52:14.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: pgmap v661: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 333 op/s 2024-04-25T13:52:14.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:52:14.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:52:14.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.626 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:14 smithi094 ceph-mon[29195]: daemon mds.f restarted 2024-04-25T13:52:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: pgmap v661: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 333 op/s 2024-04-25T13:52:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:52:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:52:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:14 smithi137 ceph-mon[32568]: daemon mds.f restarted 2024-04-25T13:52:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: pgmap v661: 97 pgs: 97 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 333 op/s 2024-04-25T13:52:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:52:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:52:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:14 smithi138 ceph-mon[32977]: daemon mds.f restarted 2024-04-25T13:52:14.810 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1285238 12.70 MB/sec execute 417 sec latency 14.357 ms 2024-04-25T13:52:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:52:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/1434478747,v1:172.21.15.138:6835/1434478747] up:boot 2024-04-25T13:52:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:52:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:52:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:52:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:15 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/1434478747,v1:172.21.15.138:6835/1434478747] up:boot 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:52:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:15 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/1434478747,v1:172.21.15.138:6835/1434478747] up:boot 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:52:15.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:15 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:15.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1286479 12.68 MB/sec execute 418 sec latency 84.167 ms 2024-04-25T13:52:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:16 smithi138 ceph-mon[32977]: pgmap v663: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-04-25T13:52:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:52:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:16 smithi137 ceph-mon[32568]: pgmap v663: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-04-25T13:52:16.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:52:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:16 smithi094 ceph-mon[29195]: pgmap v663: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-04-25T13:52:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:52:16.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1289340 12.70 MB/sec execute 419 sec latency 13.831 ms 2024-04-25T13:52:17.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1291561 12.68 MB/sec execute 420 sec latency 23.074 ms 2024-04-25T13:52:17.965 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:17 smithi137 ceph-mon[32568]: pgmap v664: 97 pgs: 97 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 369 op/s 2024-04-25T13:52:17.965 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.965 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.966 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.966 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:17 smithi094 ceph-mon[29195]: pgmap v664: 97 pgs: 97 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 369 op/s 2024-04-25T13:52:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:17 smithi138 ceph-mon[32977]: pgmap v664: 97 pgs: 97 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 369 op/s 2024-04-25T13:52:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:18.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1293282 12.68 MB/sec execute 421 sec latency 36.765 ms 2024-04-25T13:52:19.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1295875 12.68 MB/sec execute 422 sec latency 27.320 ms 2024-04-25T13:52:19.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:19 smithi094 ceph-mon[29195]: pgmap v665: 97 pgs: 97 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-04-25T13:52:19.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:19.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:19 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:19.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:19 smithi138 ceph-mon[32977]: pgmap v665: 97 pgs: 97 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-04-25T13:52:19.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:19.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:19 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:19 smithi137 ceph-mon[32568]: pgmap v665: 97 pgs: 97 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-04-25T13:52:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:19 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:20.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1297005 12.67 MB/sec execute 423 sec latency 57.331 ms 2024-04-25T13:52:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:20 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:20.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:20.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:20.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:20 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:20 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:52:21.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1299867 12.68 MB/sec execute 424 sec latency 12.400 ms 2024-04-25T13:52:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:22 smithi094 ceph-mon[29195]: pgmap v666: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-25T13:52:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:22 smithi138 ceph-mon[32977]: pgmap v666: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-25T13:52:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:22 smithi137 ceph-mon[32568]: pgmap v666: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-04-25T13:52:22.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1301487 12.67 MB/sec execute 425 sec latency 16.597 ms 2024-04-25T13:52:23.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1303890 12.67 MB/sec execute 426 sec latency 10.080 ms 2024-04-25T13:52:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:24 smithi094 ceph-mon[29195]: pgmap v667: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:52:24.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:24 smithi138 ceph-mon[32977]: pgmap v667: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:52:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:24 smithi137 ceph-mon[32568]: pgmap v667: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 383 op/s 2024-04-25T13:52:24.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1306905 12.68 MB/sec execute 427 sec latency 18.230 ms 2024-04-25T13:52:25.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1308342 12.67 MB/sec execute 428 sec latency 135.587 ms 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 -- 172.21.15.94:0/3170603649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4103c60 msgr2=0x7f65c4104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 --2- 172.21.15.94:0/3170603649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4103c60 0x7f65c4104070 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009900 tx=0x7f65b802a410 comp rx=0 tx=0).stop 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 -- 172.21.15.94:0/3170603649 shutdown_connections 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 --2- 172.21.15.94:0/3170603649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4108960 0x7f65c410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 --2- 172.21.15.94:0/3170603649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4103c60 0x7f65c4104070 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 --2- 172.21.15.94:0/3170603649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4102a20 0x7f65c4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 -- 172.21.15.94:0/3170603649 >> 172.21.15.94:0/3170603649 conn(0x7f65c40fdc10 msgr2=0x7f65c41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 -- 172.21.15.94:0/3170603649 shutdown_connections 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.156+0000 7f65ca535640 1 -- 172.21.15.94:0/3170603649 wait complete. 2024-04-25T13:52:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 Processor -- start 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 -- start start 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4103c60 0x7f65c41a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4108960 0x7f65c41a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410e520 con 0x7f65c4102a20 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410e3a0 con 0x7f65c4103c60 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65ca535640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65c410e6a0 con 0x7f65c4108960 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54888/0 (socket says 172.21.15.94:54888) 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65c3fff640 1 -- 172.21.15.94:0/2623712000 learned_addr learned my addr 172.21.15.94:0/2623712000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65c37fe640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4103c60 0x7f65c41a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.157+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4108960 0x7f65c41a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 -- 172.21.15.94:0/2623712000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4108960 msgr2=0x7f65c41a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4108960 0x7f65c41a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 -- 172.21.15.94:0/2623712000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4103c60 msgr2=0x7f65c41a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4103c60 0x7f65c41a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 -- 172.21.15.94:0/2623712000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41a3f60 con 0x7f65c4102a20 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c3fff640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e9f0 tx=0x7f65b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c17fa640 1 -- 172.21.15.94:0/2623712000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b400ccc0 con 0x7f65c4102a20 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c17fa640 1 -- 172.21.15.94:0/2623712000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b40035f0 con 0x7f65c4102a20 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41a4250 con 0x7f65c4102a20 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65c17fa640 1 -- 172.21.15.94:0/2623712000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b400ce60 con 0x7f65c4102a20 2024-04-25T13:52:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.158+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a47a0 con 0x7f65c4102a20 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 Processor -- start 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 -- start start 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c17fa640 1 -- 172.21.15.94:0/2623712000 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f65b4003aa0 con 0x7f65c4102a20 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d79700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558181cd6460 con 0x5581819af4d0 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558181c90590 con 0x5581819a72f0 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65d13e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558181cd2420 con 0x5581819d01b0 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54898/0 (socket says 172.21.15.94:54898) 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 -- 172.21.15.94:0/2400913435 learned_addr learned my addr 172.21.15.94:0/2400913435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c3fff640 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d79700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 -- 172.21.15.94:0/2400913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 msgr2=0x558181d79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d79180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 -- 172.21.15.94:0/2400913435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 msgr2=0x558181d79700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d79700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c37fe640 1 -- 172.21.15.94:0/2400913435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558181c9d0d0 con 0x5581819af4d0 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.161+0000 7f65c17fa640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 0x7f65ac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65c37fe640 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f65b8031500 tx=0x7f65b8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9f94eb65564f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 0x7f65ac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65c17fa640 1 -- 172.21.15.94:0/2623712000 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f65b40df040 con 0x7f65c4102a20 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65aa7fc640 1 -- 172.21.15.94:0/2400913435 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8035030 con 0x5581819af4d0 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65aa7fc640 1 -- 172.21.15.94:0/2400913435 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b8040070 con 0x5581819af4d0 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.162+0000 7f65aa7fc640 1 -- 172.21.15.94:0/2400913435 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8003930 con 0x5581819af4d0 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- 172.21.15.94:0/2400913435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 msgr2=0x558181d80600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 0x7f65ac0b1ff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x558181d7a350 tx=0x7f65bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f65b8031500 tx=0x7f65b8009640 comp rx=0 tx=0).stop 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- 172.21.15.94:0/2400913435 shutdown_connections 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819af4d0 0x558181d80600 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d79700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 --2- 172.21.15.94:0/2400913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- 172.21.15.94:0/2400913435 >> 172.21.15.94:0/2400913435 conn(0x5581819c4300 msgr2=0x558181d74b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- 172.21.15.94:0/2400913435 shutdown_connections 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- 172.21.15.94:0/2400913435 wait complete. 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 -- start start 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.163+0000 7f65d13e0740 1 Processor -- start 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d82e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 0x558181d83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558181d80ff0 con 0x5581819d01b0 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558181c5db60 con 0x5581819a72f0 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558181d81170 con 0x5581819af4d0 2024-04-25T13:52:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 10 client.0 _ll_register_callbacks cb 0x558181c822b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54900/0 (socket says 172.21.15.94:54900) 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 -- 172.21.15.94:0/748919889 learned_addr learned my addr 172.21.15.94:0/748919889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d82e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c37fe640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 0x558181d83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 -- 172.21.15.94:0/748919889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 msgr2=0x558181d83360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 0x558181d83360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 -- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 msgr2=0x558181d82e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d82e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 -- 172.21.15.94:0/748919889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581819e8210 con 0x5581819d01b0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c37fe640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 0x558181d83360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c3fff640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d82e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009e90 tx=0x7f65bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5b256f0082fba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc010df0 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f65bc01b460 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.164+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b8040070 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f65bc01b660 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 10 client.19353 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558181ab4620 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b8040070 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f65bc024070 con 0x5581819d01b0 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 10 client.19353 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 10 client.19353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 20 client.19353 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55818198ed80 0x558181d90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 4 client.19353 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558181ce4070 con 0x55818198ed80 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.165+0000 7f65c3fff640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55818198ed80 0x558181d90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.166+0000 7f65a37fe640 10 client.19353 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.166+0000 7f65c3fff640 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55818198ed80 0x558181d90c10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc8132bf1a76dbf server_cookie=68e74c1f2ed191cf in_seq=0 out_seq=0 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65a37fe640 1 -- 172.21.15.94:0/748919889 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558181ce4070 con 0x55818198ed80 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65a37fe640 10 client.19353 handle_command_reply: tid=0 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 client.19353 shutdown 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 msgr2=0x558181d8a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009e90 tx=0x7f65bc004a40 comp rx=0 tx=0).stop 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 shutdown_connections 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55818198ed80 0x558181d90c10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d8a2c0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819af4d0 0x558181d83360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 --2- 172.21.15.94:0/748919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819a72f0 0x558181d82e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65a37fe640 0 client.19353 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 >> 172.21.15.94:0/748919889 conn(0x5581819c4300 msgr2=0x558181d6b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 shutdown_connections 2024-04-25T13:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 1 -- 172.21.15.94:0/748919889 wait complete. 2024-04-25T13:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.167+0000 7f65d13e0740 20 client.19353 trim_cache size 0 max 16384 2024-04-25T13:52:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 Processor -- start 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 -- start start 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41a93a0 con 0x5581819a72f0 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41aaee0 con 0x5581819af4d0 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.184+0000 7f65d13e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41a9a70 con 0x5581819d01b0 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54908/0 (socket says 172.21.15.94:54908) 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 -- 172.21.15.94:0/3253229165 learned_addr learned my addr 172.21.15.94:0/3253229165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c37fe640 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 -- 172.21.15.94:0/3253229165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 msgr2=0x558181d81510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d81510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 -- 172.21.15.94:0/3253229165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 msgr2=0x558181d8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 -- 172.21.15.94:0/3253229165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558181d8a720 con 0x5581819a72f0 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65c3fff640 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f65b40a7410 tx=0x7f65b40dc970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93de76b3953b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65a9ffb640 1 -- 172.21.15.94:0/3253229165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a6dd0 con 0x5581819a72f0 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65a9ffb640 1 -- 172.21.15.94:0/3253229165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b40a6f90 con 0x5581819a72f0 2024-04-25T13:52:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.185+0000 7f65a9ffb640 1 -- 172.21.15.94:0/3253229165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a6bf0 con 0x5581819a72f0 2024-04-25T13:52:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 -- 172.21.15.94:0/3253229165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 msgr2=0x558181d7caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f65b40a7410 tx=0x7f65b40dc970 comp rx=0 tx=0).stop 2024-04-25T13:52:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 -- 172.21.15.94:0/3253229165 shutdown_connections 2024-04-25T13:52:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819d01b0 0x558181d81510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 --2- 172.21.15.94:0/3253229165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819a72f0 0x558181d7caa0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 -- 172.21.15.94:0/3253229165 >> 172.21.15.94:0/3253229165 conn(0x5581819c4300 msgr2=0x558181d7c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 -- 172.21.15.94:0/3253229165 shutdown_connections 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.186+0000 7f65d13e0740 1 -- 172.21.15.94:0/3253229165 wait complete. 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 -- start start 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 Processor -- start 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819a72f0 0x558181d873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41aa810 con 0x5581819d01b0 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41aa140 con 0x5581819af4d0 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41a7380 con 0x5581819a72f0 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65d13e0740 10 client.0 _ll_register_callbacks cb 0x7f65bc009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54916/0 (socket says 172.21.15.94:54916) 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 -- 172.21.15.94:0/177050082 learned_addr learned my addr 172.21.15.94:0/177050082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c37fe640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c3fff640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819a72f0 0x558181d873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 -- 172.21.15.94:0/177050082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819a72f0 msgr2=0x558181d873b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819a72f0 0x558181d873b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 -- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 msgr2=0x558181d878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.187+0000 7f65c8ad2640 1 -- 172.21.15.94:0/177050082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558181d88de0 con 0x5581819d01b0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65c8ad2640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006010 tx=0x7f65bc010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d701edae2335c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc004cc0 con 0x5581819d01b0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f65bc02f3f0 con 0x5581819d01b0 2024-04-25T13:52:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b40a6f90 con 0x5581819d01b0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f65bc004e60 con 0x5581819d01b0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 10 client.19359 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558181d8a0e0 con 0x5581819d01b0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b40a6f90 con 0x5581819d01b0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f65bc024050 con 0x5581819d01b0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 10 client.19359 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 20 client.19359 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581819641c0 0x558181dfb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 4 client.19359 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.188+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558181cddee0 con 0x5581819641c0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.189+0000 7f65c3fff640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581819641c0 0x558181dfb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.189+0000 7f65a37fe640 10 client.19359 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.189+0000 7f65c3fff640 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581819641c0 0x558181dfb390 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69e383cc0864125 server_cookie=de6c8bcfd6ef4ec4 in_seq=0 out_seq=0 2024-04-25T13:52:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.374+0000 7f65a37fe640 1 -- 172.21.15.94:0/177050082 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558181cddee0 con 0x5581819641c0 2024-04-25T13:52:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.374+0000 7f65a37fe640 10 client.19359 handle_command_reply: tid=0 2024-04-25T13:52:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.374+0000 7f65d13e0740 1 client.19359 shutdown 2024-04-25T13:52:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 msgr2=0x558181d88820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006010 tx=0x7f65bc010e20 comp rx=0 tx=0).stop 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 shutdown_connections 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581819641c0 0x558181dfb390 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581819d01b0 0x558181d88820 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581819af4d0 0x558181d878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 --2- 172.21.15.94:0/177050082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581819a72f0 0x558181d873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65a37fe640 0 client.19359 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 >> 172.21.15.94:0/177050082 conn(0x5581819c4300 msgr2=0x558181d81510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 shutdown_connections 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 1 -- 172.21.15.94:0/177050082 wait complete. 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65d13e0740 20 client.19359 trim_cache size 0 max 16384 2024-04-25T13:52:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:52:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:52:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:52:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:52:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 msgr2=0x7f65ac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 0x7f65ac0b1ff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x558181d7a350 tx=0x7f65bc009500 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 msgr2=0x7f65c406ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.375+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e9f0 tx=0x7f65b400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 shutdown_connections 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65ac0afae0 0x7f65ac0b1ff0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65c4108960 0x7f65c41a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65c4103c60 0x7f65c41a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 --2- 172.21.15.94:0/2623712000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65c4102a20 0x7f65c406ae30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 >> 172.21.15.94:0/2623712000 conn(0x7f65c40fdc10 msgr2=0x7f65c4104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 shutdown_connections 2024-04-25T13:52:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:26.376+0000 7f65ca535640 1 -- 172.21.15.94:0/2623712000 wait complete. 2024-04-25T13:52:26.456 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:26 smithi137 ceph-mon[32568]: pgmap v668: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-04-25T13:52:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:26 smithi094 ceph-mon[29195]: pgmap v668: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-04-25T13:52:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:26 smithi138 ceph-mon[32977]: pgmap v668: 97 pgs: 97 active+clean; 42 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-04-25T13:52:26.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1310692 12.67 MB/sec execute 429 sec latency 65.562 ms 2024-04-25T13:52:27.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1312911 12.66 MB/sec execute 430 sec latency 18.361 ms 2024-04-25T13:52:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:28 smithi094 ceph-mon[29195]: pgmap v669: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-04-25T13:52:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:28 smithi138 ceph-mon[32977]: pgmap v669: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-04-25T13:52:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:28 smithi137 ceph-mon[32568]: pgmap v669: 97 pgs: 97 active+clean; 43 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-04-25T13:52:28.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1314800 12.66 MB/sec execute 431 sec latency 12.473 ms 2024-04-25T13:52:29.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1317660 12.67 MB/sec execute 432 sec latency 31.548 ms 2024-04-25T13:52:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:30 smithi094 ceph-mon[29195]: pgmap v670: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T13:52:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:30 smithi138 ceph-mon[32977]: pgmap v670: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T13:52:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:30 smithi137 ceph-mon[32568]: pgmap v670: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T13:52:30.811 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1319206 12.65 MB/sec execute 433 sec latency 32.848 ms 2024-04-25T13:52:31.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1321471 12.66 MB/sec execute 434 sec latency 81.105 ms 2024-04-25T13:52:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:32 smithi094 ceph-mon[29195]: pgmap v671: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-04-25T13:52:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:32 smithi138 ceph-mon[32977]: pgmap v671: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-04-25T13:52:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:32 smithi137 ceph-mon[32568]: pgmap v671: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-04-25T13:52:32.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1323696 12.65 MB/sec execute 435 sec latency 14.632 ms 2024-04-25T13:52:33.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1325444 12.64 MB/sec execute 436 sec latency 19.141 ms 2024-04-25T13:52:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:34 smithi094 ceph-mon[29195]: pgmap v672: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-04-25T13:52:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:34 smithi138 ceph-mon[32977]: pgmap v672: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-04-25T13:52:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:34 smithi137 ceph-mon[32568]: pgmap v672: 97 pgs: 97 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-04-25T13:52:34.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1328297 12.65 MB/sec execute 437 sec latency 9.340 ms 2024-04-25T13:52:35.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1330147 12.64 MB/sec execute 438 sec latency 20.852 ms 2024-04-25T13:52:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:36 smithi094 ceph-mon[29195]: pgmap v673: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 384 op/s 2024-04-25T13:52:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:36 smithi138 ceph-mon[32977]: pgmap v673: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 384 op/s 2024-04-25T13:52:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:36 smithi137 ceph-mon[32568]: pgmap v673: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 384 op/s 2024-04-25T13:52:36.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1332113 12.64 MB/sec execute 439 sec latency 59.407 ms 2024-04-25T13:52:37.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1334606 12.64 MB/sec execute 440 sec latency 12.546 ms 2024-04-25T13:52:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:38 smithi094 ceph-mon[29195]: pgmap v674: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-04-25T13:52:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:38 smithi138 ceph-mon[32977]: pgmap v674: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-04-25T13:52:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:38 smithi137 ceph-mon[32568]: pgmap v674: 97 pgs: 97 active+clean; 40 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-04-25T13:52:38.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1336215 12.63 MB/sec execute 441 sec latency 12.762 ms 2024-04-25T13:52:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 -- 172.21.15.94:0/732627061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc109a80 msgr2=0x7f87dc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 --2- 172.21.15.94:0/732627061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc109a80 0x7f87dc10bea0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009ca0 tx=0x7f87d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 -- 172.21.15.94:0/732627061 shutdown_connections 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 --2- 172.21.15.94:0/732627061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc109a80 0x7f87dc10bea0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 --2- 172.21.15.94:0/732627061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 0x7f87dc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 --2- 172.21.15.94:0/732627061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc103b40 0x7f87dc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 -- 172.21.15.94:0/732627061 >> 172.21.15.94:0/732627061 conn(0x7f87dc0fed30 msgr2=0x7f87dc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 -- 172.21.15.94:0/732627061 shutdown_connections 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.478+0000 7f87e2f46640 1 -- 172.21.15.94:0/732627061 wait complete. 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 Processor -- start 2024-04-25T13:52:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 -- start start 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 0x7f87dc074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc109a80 0x7f87dc075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10f670 con 0x7f87dc103b40 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10f4f0 con 0x7f87dc109a80 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e2f46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc10f7f0 con 0x7f87dc104d80 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e0ce2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e0ce2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41732/0 (socket says 172.21.15.94:41732) 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e0ce2640 1 -- 172.21.15.94:0/2023087071 learned_addr learned my addr 172.21.15.94:0/2023087071 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 0x7f87dc074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.479+0000 7f87e14e3640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc109a80 0x7f87dc075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 -- 172.21.15.94:0/2023087071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 msgr2=0x7f87dc074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 0x7f87dc074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 -- 172.21.15.94:0/2023087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc109a80 msgr2=0x7f87dc075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc109a80 0x7f87dc075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 -- 172.21.15.94:0/2023087071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc075b60 con 0x7f87dc103b40 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f87d0005fd0 tx=0x7f87d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87d9ffb640 1 -- 172.21.15.94:0/2023087071 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0017070 con 0x7f87dc103b40 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87d9ffb640 1 -- 172.21.15.94:0/2023087071 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d0009c30 con 0x7f87dc103b40 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc1a42a0 con 0x7f87dc103b40 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87d9ffb640 1 -- 172.21.15.94:0/2023087071 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003d10 con 0x7f87dc103b40 2024-04-25T13:52:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.480+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1a4760 con 0x7f87dc103b40 2024-04-25T13:52:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.482+0000 7f87d9ffb640 1 -- 172.21.15.94:0/2023087071 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f87d0003eb0 con 0x7f87dc103b40 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 Processor -- start 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 -- start start 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269b91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b91ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 0x55a269b98df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a269af7460 con 0x55a2697f11b0 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a269ab1590 con 0x55a2697c82f0 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e9df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a269af3420 con 0x55a2697d04d0 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87d9ffb640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 0x7f87cc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e14e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269b91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 0x55a269b98df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e14e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269b91970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41744/0 (socket says 172.21.15.94:41744) 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e14e3640 1 -- 172.21.15.94:0/1673843370 learned_addr learned my addr 172.21.15.94:0/1673843370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.483+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b91ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87e14e3640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 0x7f87cc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 -- 172.21.15.94:0/1673843370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 msgr2=0x55a269b91ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b91ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 -- 172.21.15.94:0/1673843370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 msgr2=0x55a269b91970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269b91970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87d9ffb640 1 -- 172.21.15.94:0/2023087071 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f87d00de920 con 0x7f87dc103b40 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 -- 172.21.15.94:0/1673843370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a269abe0d0 con 0x55a2697d04d0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87dbfff640 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 0x55a269b98df0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87c8005fd0 tx=0x7f87c800ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634976324ab96e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87e14e3640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 0x7f87cc0b1ff0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55a269b92b40 tx=0x7f87d40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87c6ffd640 1 -- 172.21.15.94:0/1673843370 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8003a90 con 0x55a2697d04d0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87c6ffd640 1 -- 172.21.15.94:0/1673843370 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c8015070 con 0x55a2697d04d0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.484+0000 7f87c6ffd640 1 -- 172.21.15.94:0/1673843370 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004c00 con 0x55a2697d04d0 2024-04-25T13:52:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 -- 172.21.15.94:0/1673843370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 msgr2=0x55a269b98df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 0x55a269b98df0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87c8005fd0 tx=0x7f87c800ba60 comp rx=0 tx=0).stop 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 -- 172.21.15.94:0/1673843370 shutdown_connections 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697d04d0 0x55a269b98df0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b91ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 --2- 172.21.15.94:0/1673843370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269b91970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 -- 172.21.15.94:0/1673843370 >> 172.21.15.94:0/1673843370 conn(0x55a2697e5300 msgr2=0x55a269b8d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 -- 172.21.15.94:0/1673843370 shutdown_connections 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.485+0000 7f87e9df9740 1 -- 172.21.15.94:0/1673843370 wait complete. 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 -- start start 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 Processor -- start 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b9b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697f11b0 0x55a269ba2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a269b99840 con 0x55a2697d04d0 2024-04-25T13:52:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a269a7eb60 con 0x55a2697c82f0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a269b999c0 con 0x55a2697f11b0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e9df9740 10 client.0 _ll_register_callbacks cb 0x55a269aa32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41758/0 (socket says 172.21.15.94:41758) 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87dbfff640 1 -- 172.21.15.94:0/2870792633 learned_addr learned my addr 172.21.15.94:0/2870792633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b9b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.486+0000 7f87e14e3640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697f11b0 0x55a269ba2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 -- 172.21.15.94:0/2870792633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697f11b0 msgr2=0x55a269ba2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697f11b0 0x55a269ba2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 -- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 msgr2=0x55a269b9b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b9b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 -- 172.21.15.94:0/2870792633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a269809210 con 0x55a2697d04d0 2024-04-25T13:52:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f87c8003ac0 tx=0x7f87c8003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b68a52cb8c78cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8028060 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f87c8011850 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c8015070 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f87c8003dd0 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87e9df9740 10 client.19371 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2698d5620 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.487+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c8015070 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f87c8022070 con 0x55a2697d04d0 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 10 client.19371 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 10 client.19371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 20 client.19371 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697afd80 0x55a269ba9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 4 client.19371 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a269b05070 con 0x55a2697afd80 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697afd80 0x55a269ba9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697afd80 0x55a269ba9000 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ec78e62e38e2f9e server_cookie=754937f55ece8e04 in_seq=0 out_seq=0 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.488+0000 7f87bbfff640 10 client.19371 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87bbfff640 1 -- 172.21.15.94:0/2870792633 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a269b05070 con 0x55a2697afd80 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87bbfff640 10 client.19371 handle_command_reply: tid=0 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 client.19371 shutdown 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 msgr2=0x55a269b9bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f87c8003ac0 tx=0x7f87c8003af0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 shutdown_connections 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697afd80 0x55a269ba9000 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697f11b0 0x55a269ba2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87bbfff640 0 client.19371 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b9bb90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2870792633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697c82f0 0x55a269b9b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 >> 172.21.15.94:0/2870792633 conn(0x55a2697e5300 msgr2=0x55a269b8c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.489+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 shutdown_connections 2024-04-25T13:52:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.490+0000 7f87e9df9740 1 -- 172.21.15.94:0/2870792633 wait complete. 2024-04-25T13:52:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.490+0000 7f87e9df9740 20 client.19371 trim_cache size 0 max 16384 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 Processor -- start 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 -- start start 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b95290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697f11b0 0x55a269b8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1ad700 con 0x55a2697d04d0 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1af240 con 0x55a2697f11b0 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e9df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1addd0 con 0x55a2697c82f0 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41772/0 (socket says 172.21.15.94:41772) 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 -- 172.21.15.94:0/2461610718 learned_addr learned my addr 172.21.15.94:0/2461610718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b95290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e14e3640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697f11b0 0x55a269b8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 -- 172.21.15.94:0/2461610718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 msgr2=0x55a269b95290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b95290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 -- 172.21.15.94:0/2461610718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697f11b0 msgr2=0x55a269b8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697f11b0 0x55a269b8d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87dbfff640 1 -- 172.21.15.94:0/2461610718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a269b91b20 con 0x55a2697d04d0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.507+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b95290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.508+0000 7f87dbfff640 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f87c800baa0 tx=0x7f87c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be73275ffa7a871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.508+0000 7f87c67fc640 1 -- 172.21.15.94:0/2461610718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8028070 con 0x55a2697d04d0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.508+0000 7f87c67fc640 1 -- 172.21.15.94:0/2461610718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c8015070 con 0x55a2697d04d0 2024-04-25T13:52:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.508+0000 7f87c67fc640 1 -- 172.21.15.94:0/2461610718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004c50 con 0x55a2697d04d0 2024-04-25T13:52:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- 172.21.15.94:0/2461610718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 msgr2=0x55a269b8cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f87c800baa0 tx=0x7f87c8004430 comp rx=0 tx=0).stop 2024-04-25T13:52:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- 172.21.15.94:0/2461610718 shutdown_connections 2024-04-25T13:52:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697f11b0 0x55a269b8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697d04d0 0x55a269b8cbe0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- 172.21.15.94:0/2461610718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b95290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- 172.21.15.94:0/2461610718 >> 172.21.15.94:0/2461610718 conn(0x55a2697e5300 msgr2=0x55a269b94b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- 172.21.15.94:0/2461610718 shutdown_connections 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- 172.21.15.94:0/2461610718 wait complete. 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- start start 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 Processor -- start 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b9f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 0x55a269b9ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1aeb70 con 0x55a2697f11b0 2024-04-25T13:52:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1ae4a0 con 0x55a2697d04d0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.509+0000 7f87e9df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1ab6e0 con 0x55a2697c82f0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e9df9740 10 client.0 _ll_register_callbacks cb 0x7f87d4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e0ce2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b9f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 0x55a269b9ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41788/0 (socket says 172.21.15.94:41788) 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 -- 172.21.15.94:0/819218929 learned_addr learned my addr 172.21.15.94:0/819218929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e0ce2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b9f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33350/0 (socket says 172.21.15.94:33350) 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 -- 172.21.15.94:0/819218929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 msgr2=0x55a269b9f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b9f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 -- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 msgr2=0x55a269b9ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 0x55a269b9ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 -- 172.21.15.94:0/819218929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a269ba13f0 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87dbfff640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 0x55a269b9ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e14e3640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f87d4031b50 tx=0x7f87d402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee1f3d28d18e91b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40043d0 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f87d4004570 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c8015070 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f87d4004770 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e9df9740 10 client.19377 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.510+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a269c11340 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c8015070 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f87d4030650 con 0x55a2697f11b0 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 10 client.19377 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 10 client.19377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 20 client.19377 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697851c0 0x55a269c13db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 4 client.19377 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a269afeee0 con 0x55a2697851c0 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697851c0 0x55a269c13db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87bbfff640 10 client.19377 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.511+0000 7f87e0ce2640 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697851c0 0x55a269c13db0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca7ed8bf71c1d2c server_cookie=7792483e6b42a076 in_seq=0 out_seq=0 2024-04-25T13:52:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.690+0000 7f87bbfff640 1 -- 172.21.15.94:0/819218929 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a269afeee0 con 0x55a2697851c0 2024-04-25T13:52:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.690+0000 7f87bbfff640 10 client.19377 handle_command_reply: tid=0 2024-04-25T13:52:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.690+0000 7f87e9df9740 1 client.19377 shutdown 2024-04-25T13:52:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 msgr2=0x55a269ba0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f87d4031b50 tx=0x7f87d402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 shutdown_connections 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a2697851c0 0x55a269c13db0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2697f11b0 0x55a269ba0e30 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2697d04d0 0x55a269b9ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87bbfff640 0 client.19377 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 --2- 172.21.15.94:0/819218929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2697c82f0 0x55a269b9f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 >> 172.21.15.94:0/819218929 conn(0x55a2697e5300 msgr2=0x55a269b8cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 shutdown_connections 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 1 -- 172.21.15.94:0/819218929 wait complete. 2024-04-25T13:52:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.691+0000 7f87e9df9740 20 client.19377 trim_cache size 0 max 16384 2024-04-25T13:52:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:52:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:52:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:52:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 msgr2=0x7f87cc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 0x7f87cc0b1ff0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55a269b92b40 tx=0x7f87d40095c0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 msgr2=0x7f87dc076850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f87d0005fd0 tx=0x7f87d000dde0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 shutdown_connections 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87cc0afae0 0x7f87cc0b1ff0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87dc109a80 0x7f87dc075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87dc104d80 0x7f87dc074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 --2- 172.21.15.94:0/2023087071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc103b40 0x7f87dc076850 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 >> 172.21.15.94:0/2023087071 conn(0x7f87dc0fed30 msgr2=0x7f87dc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 shutdown_connections 2024-04-25T13:52:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:39.692+0000 7f87e2f46640 1 -- 172.21.15.94:0/2023087071 wait complete. 2024-04-25T13:52:39.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1338790 12.64 MB/sec execute 442 sec latency 66.719 ms 2024-04-25T13:52:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:40 smithi094 ceph-mon[29195]: pgmap v675: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-04-25T13:52:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:40 smithi138 ceph-mon[32977]: pgmap v675: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-04-25T13:52:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:40 smithi137 ceph-mon[32568]: pgmap v675: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-04-25T13:52:40.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1340350 12.63 MB/sec execute 443 sec latency 23.405 ms 2024-04-25T13:52:41.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1342610 12.63 MB/sec execute 444 sec latency 64.628 ms 2024-04-25T13:52:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:42 smithi094 ceph-mon[29195]: pgmap v676: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 368 op/s 2024-04-25T13:52:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:42 smithi138 ceph-mon[32977]: pgmap v676: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 368 op/s 2024-04-25T13:52:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:42 smithi137 ceph-mon[32568]: pgmap v676: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 368 op/s 2024-04-25T13:52:42.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1344698 12.62 MB/sec execute 445 sec latency 17.693 ms 2024-04-25T13:52:43.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:52:43.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:52:43.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 -- 172.21.15.94:0/360904953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d0109890 msgr2=0x7f47d010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 --2- 172.21.15.94:0/360904953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d0109890 0x7f47d010bcb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009ca0 tx=0x7f47c802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 -- 172.21.15.94:0/360904953 shutdown_connections 2024-04-25T13:52:43.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 --2- 172.21.15.94:0/360904953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d0109890 0x7f47d010bcb0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 --2- 172.21.15.94:0/360904953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 0x7f47d0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 --2- 172.21.15.94:0/360904953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d01045d0 0x7f47d01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 -- 172.21.15.94:0/360904953 >> 172.21.15.94:0/360904953 conn(0x7f47d00fdbd0 msgr2=0x7f47d0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 -- 172.21.15.94:0/360904953 shutdown_connections 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.591+0000 7f47d6985640 1 -- 172.21.15.94:0/360904953 wait complete. 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 Processor -- start 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 -- start start 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 0x7f47d019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d0109890 0x7f47d01a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47d010f610 con 0x7f47d01045d0 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47d010f490 con 0x7f47d0106f30 2024-04-25T13:52:43.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d6985640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47d010f790 con 0x7f47d0109890 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37580/0 (socket says 172.21.15.94:37580) 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47cffff640 1 -- 172.21.15.94:0/2204461464 learned_addr learned my addr 172.21.15.94:0/2204461464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47d4f22640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d0109890 0x7f47d01a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.592+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 0x7f47d019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 -- 172.21.15.94:0/2204461464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d0109890 msgr2=0x7f47d01a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d0109890 0x7f47d01a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 -- 172.21.15.94:0/2204461464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 msgr2=0x7f47d019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 0x7f47d019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 -- 172.21.15.94:0/2204461464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d01a2080 con 0x7f47d01045d0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cffff640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c970 tx=0x7f47c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cd7fa640 1 -- 172.21.15.94:0/2204461464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0007bb0 con 0x7f47d01045d0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cd7fa640 1 -- 172.21.15.94:0/2204461464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c0007d50 con 0x7f47d01045d0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d01a2370 con 0x7f47d01045d0 2024-04-25T13:52:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47cd7fa640 1 -- 172.21.15.94:0/2204461464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0018730 con 0x7f47d01045d0 2024-04-25T13:52:43.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.593+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d01a2870 con 0x7f47d01045d0 2024-04-25T13:52:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.595+0000 7f47dd831740 1 Processor -- start 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47cd7fa640 1 -- 172.21.15.94:0/2204461464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f47c00188f0 con 0x7f47d01045d0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 -- start start 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116c34d0 0x564611a8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116cb6b0 0x564611a8ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646119f2d80 con 0x5646116ec390 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646119ac8b0 con 0x5646116cb6b0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47dd831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646119eed40 con 0x5646116c34d0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37594/0 (socket says 172.21.15.94:37594) 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 -- 172.21.15.94:0/1301512905 learned_addr learned my addr 172.21.15.94:0/1301512905 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47cffff640 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116c34d0 0x564611a8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116cb6b0 0x564611a8ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 -- 172.21.15.94:0/1301512905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116c34d0 msgr2=0x564611a8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116c34d0 0x564611a8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 -- 172.21.15.94:0/1301512905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116cb6b0 msgr2=0x564611a8ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116cb6b0 0x564611a8ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47d4f22640 1 -- 172.21.15.94:0/1301512905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461197da40 con 0x5646116ec390 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.596+0000 7f47cd7fa640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 0x7f47b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47cffff640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 0x7f47b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47d4f22640 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f47c8035000 tx=0x7f47c80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3682965b94b10429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47cd7fa640 1 -- 172.21.15.94:0/2204461464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f47c00dec40 con 0x7f47d01045d0 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47b67fc640 1 -- 172.21.15.94:0/1301512905 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8031710 con 0x5646116ec390 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47b67fc640 1 -- 172.21.15.94:0/1301512905 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c8036030 con 0x5646116ec390 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.597+0000 7f47b67fc640 1 -- 172.21.15.94:0/1301512905 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c802f660 con 0x5646116ec390 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- 172.21.15.94:0/1301512905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 msgr2=0x564611a8da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47cffff640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 0x7f47b80b2000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x564611a8d780 tx=0x7f47c00e2720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f47c8035000 tx=0x7f47c80098b0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- 172.21.15.94:0/1301512905 shutdown_connections 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116cb6b0 0x564611a8ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116c34d0 0x564611a8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 --2- 172.21.15.94:0/1301512905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a8da60 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- 172.21.15.94:0/1301512905 >> 172.21.15.94:0/1301512905 conn(0x5646116e04e0 msgr2=0x564611a89270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- 172.21.15.94:0/1301512905 shutdown_connections 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- 172.21.15.94:0/1301512905 wait complete. 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 -- start start 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.598+0000 7f47dd831740 1 Processor -- start 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a97c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564611a8f5a0 con 0x5646116ec390 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564611979d10 con 0x5646116c34d0 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564611a8f720 con 0x5646116cb6b0 2024-04-25T13:52:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47dd831740 10 client.0 _ll_register_callbacks cb 0x5646119a6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37606/0 (socket says 172.21.15.94:37606) 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 -- 172.21.15.94:0/885424761 learned_addr learned my addr 172.21.15.94:0/885424761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a97c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 -- 172.21.15.94:0/885424761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 msgr2=0x564611a97c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a97c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 -- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 msgr2=0x564611a976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 -- 172.21.15.94:0/885424761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646117ce7d0 con 0x5646116ec390 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47d4f22640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009620 tx=0x7f47c8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81fb7a2b426ec4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.599+0000 7f47af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8059060 con 0x5646116ec390 2024-04-25T13:52:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f47c8003d80 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c8036030 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47c8004b40 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 10 client.19389 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5646117043f0 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c8036030 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f47c8043070 con 0x5646116ec390 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 10 client.19389 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 10 client.19389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 20 client.19389 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116aaf60 0x564611aa4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 4 client.19389 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564611a00990 con 0x5646116aaf60 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.600+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116aaf60 0x564611aa4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.601+0000 7f47af7fe640 10 client.19389 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.601+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116aaf60 0x564611aa4c20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6665dc76b9440f08 server_cookie=22144fd03eb72861 in_seq=0 out_seq=0 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47af7fe640 1 -- 172.21.15.94:0/885424761 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564611a00990 con 0x5646116aaf60 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47af7fe640 10 client.19389 handle_command_reply: tid=0 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 client.19389 shutdown 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 msgr2=0x564611a9eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009620 tx=0x7f47c8031af0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 shutdown_connections 2024-04-25T13:52:43.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116aaf60 0x564611aa4c20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116ec390 0x564611a9eb80 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a97c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 --2- 172.21.15.94:0/885424761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47af7fe640 0 client.19389 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 >> 172.21.15.94:0/885424761 conn(0x5646116e04e0 msgr2=0x564611a8c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 shutdown_connections 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 1 -- 172.21.15.94:0/885424761 wait complete. 2024-04-25T13:52:43.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.602+0000 7f47dd831740 20 client.19389 trim_cache size 0 max 16384 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 Processor -- start 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 -- start start 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116ec390 0x564611a88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01ab420 con 0x5646116cb6b0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01acf60 con 0x5646116c34d0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47dd831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01abaf0 con 0x5646116ec390 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37620/0 (socket says 172.21.15.94:37620) 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 -- 172.21.15.94:0/507645340 learned_addr learned my addr 172.21.15.94:0/507645340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47d4f22640 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116ec390 0x564611a88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47d4f22640 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 -- 172.21.15.94:0/507645340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116ec390 msgr2=0x564611a88ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116ec390 0x564611a88ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 -- 172.21.15.94:0/507645340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 msgr2=0x564611a840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 -- 172.21.15.94:0/507645340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646116c40f0 con 0x5646116cb6b0 2024-04-25T13:52:43.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.621+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x564611a8a000 tx=0x7f47c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba950c88b482872e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47b5ffb640 1 -- 172.21.15.94:0/507645340 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4024070 con 0x5646116cb6b0 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47b5ffb640 1 -- 172.21.15.94:0/507645340 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c400fd40 con 0x5646116cb6b0 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47b5ffb640 1 -- 172.21.15.94:0/507645340 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4020940 con 0x5646116cb6b0 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 -- 172.21.15.94:0/507645340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 msgr2=0x564611a9eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x564611a8a000 tx=0x7f47c4005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 -- 172.21.15.94:0/507645340 shutdown_connections 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116ec390 0x564611a88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116cb6b0 0x564611a9eaa0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 --2- 172.21.15.94:0/507645340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116c34d0 0x564611a840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 -- 172.21.15.94:0/507645340 >> 172.21.15.94:0/507645340 conn(0x5646116e04e0 msgr2=0x564611a95d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 -- 172.21.15.94:0/507645340 shutdown_connections 2024-04-25T13:52:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.622+0000 7f47dd831740 1 -- 172.21.15.94:0/507645340 wait complete. 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 -- start start 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 Processor -- start 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a9b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116ec390 0x564611a9bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01ac890 con 0x5646116c34d0 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01ac1c0 con 0x5646116ec390 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01a9400 con 0x5646116cb6b0 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47dd831740 10 client.0 _ll_register_callbacks cb 0x7f47c40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47d4f22640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37628/0 (socket says 172.21.15.94:37628) 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47d4f22640 1 -- 172.21.15.94:0/2486518192 learned_addr learned my addr 172.21.15.94:0/2486518192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a9b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.623+0000 7f47d4f22640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116ec390 0x564611a9bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 -- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116ec390 msgr2=0x564611a9bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116ec390 0x564611a9bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 -- 172.21.15.94:0/2486518192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 msgr2=0x564611a9ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 -- 172.21.15.94:0/2486518192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646119b34b0 con 0x5646116cb6b0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47d4f22640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47cf7fe640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a9b610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x564611979e40 tx=0x7f47c400fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114a78cc9b57e400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4024050 con 0x5646116cb6b0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c400fd40 con 0x5646116cb6b0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f47c400f890 con 0x5646116cb6b0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47c400fad0 con 0x5646116cb6b0 2024-04-25T13:52:43.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47dd831740 10 client.28651 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564611b0c0b0 con 0x5646116cb6b0 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c400fd40 con 0x5646116cb6b0 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.624+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f47c4028a50 con 0x5646116cb6b0 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 10 client.28651 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 10 client.28651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 20 client.28651 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116803a0 0x564611b0e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 4 client.28651 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646119fa800 con 0x5646116803a0 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47d4f22640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116803a0 0x564611b0e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47af7fe640 10 client.28651 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:43.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.625+0000 7f47d4f22640 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116803a0 0x564611b0e9f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bea737a8ade75ca server_cookie=b087f6261a455b40 in_seq=0 out_seq=0 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.628+0000 7f47af7fe640 1 -- 172.21.15.94:0/2486518192 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646119fa800 con 0x5646116803a0 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47af7fe640 10 client.28651 handle_command_reply: tid=0 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 client.28651 shutdown 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 msgr2=0x564611a9b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a9b610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x564611979e40 tx=0x7f47c400fef0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 shutdown_connections 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646116803a0 0x564611b0e9f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646116ec390 0x564611a9bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646116cb6b0 0x564611a9b610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47af7fe640 0 client.28651 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 --2- 172.21.15.94:0/2486518192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646116c34d0 0x564611a9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 >> 172.21.15.94:0/2486518192 conn(0x5646116e04e0 msgr2=0x564611a88c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 shutdown_connections 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 1 -- 172.21.15.94:0/2486518192 wait complete. 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.629+0000 7f47dd831740 20 client.28651 trim_cache size 0 max 16384 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:52:43.633 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 msgr2=0x7f47b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 0x7f47b80b2000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x564611a8d780 tx=0x7f47c00e2720 comp rx=0 tx=0).stop 2024-04-25T13:52:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 msgr2=0x7f47d019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c970 tx=0x7f47c000ce30 comp rx=0 tx=0).stop 2024-04-25T13:52:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 shutdown_connections 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f47b80afaf0 0x7f47b80b2000 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d0109890 0x7f47d01a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d0106f30 0x7f47d019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 --2- 172.21.15.94:0/2204461464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47d01045d0 0x7f47d019a540 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 >> 172.21.15.94:0/2204461464 conn(0x7f47d00fdbd0 msgr2=0x7f47d0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 shutdown_connections 2024-04-25T13:52:43.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.630+0000 7f47d6985640 1 -- 172.21.15.94:0/2204461464 wait complete. 2024-04-25T13:52:43.641 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:52:43.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1815560289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f01029c0 msgr2=0x7f79f01036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1815560289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f01029c0 0x7f79f01036c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1815560289 shutdown_connections 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1815560289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f0108900 0x7f79f010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1815560289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0103c00 0x7f79f0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1815560289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f01029c0 0x7f79f01036c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1815560289 >> 172.21.15.94:0/1815560289 conn(0x7f79f00fdbd0 msgr2=0x7f79f0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1815560289 shutdown_connections 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1815560289 wait complete. 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 Processor -- start 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.752+0000 7f79f7c1a640 1 -- start start 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f01029c0 0x7f79f0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0108900 0x7f79f01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79f010e4c0 con 0x7f79f0103c00 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79f010e340 con 0x7f79f01029c0 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79f010e640 con 0x7f79f0108900 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37650/0 (socket says 172.21.15.94:37650) 2024-04-25T13:52:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 -- 172.21.15.94:0/1766722128 learned_addr learned my addr 172.21.15.94:0/1766722128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f61b7640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0108900 0x7f79f01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f59b6640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f01029c0 0x7f79f0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 -- 172.21.15.94:0/1766722128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0108900 msgr2=0x7f79f01a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0108900 0x7f79f01a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 -- 172.21.15.94:0/1766722128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f01029c0 msgr2=0x7f79f0199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f01029c0 0x7f79f0199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 -- 172.21.15.94:0/1766722128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f01a0c70 con 0x7f79f0103c00 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f51b5640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00e9f0 tx=0x7f79ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00cd80 con 0x7f79f0103c00 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ec0054d0 con 0x7f79f0103c00 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f01a0f60 con 0x7f79f0103c00 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec010690 con 0x7f79f0103c00 2024-04-25T13:52:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.753+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a1370 con 0x7f79f0103c00 2024-04-25T13:52:43.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.754+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f0107710 con 0x7f79f0103c00 2024-04-25T13:52:43.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.756+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f79ec004e90 con 0x7f79f0103c00 2024-04-25T13:52:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.756+0000 7f79e6ffd640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 0x7f79dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.757+0000 7f79f59b6640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 0x7f79dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.757+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f79ec0df0d0 con 0x7f79f0103c00 2024-04-25T13:52:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.757+0000 7f79f59b6640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 0x7f79dc0b2000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005ab0 tx=0x7f79e0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:43.763+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79ec0a7c10 con 0x7f79f0103c00 2024-04-25T13:52:43.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1346456 12.62 MB/sec execute 446 sec latency 13.811 ms 2024-04-25T13:52:44.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.001+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f002dd70 con 0x7f79f0103c00 2024-04-25T13:52:44.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.001+0000 7f79e6ffd640 1 -- 172.21.15.94:0/1766722128 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11941 (secure 0 0 0) 0x7f79ec0a7510 con 0x7f79f0103c00 2024-04-25T13:52:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:52:44.001 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:52:14.280433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:52:44.002 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-25T13:52:44.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.003+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 msgr2=0x7f79dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.003+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 0x7f79dc0b2000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005ab0 tx=0x7f79e0031400 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.003+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 msgr2=0x7f79f0199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.003+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00e9f0 tx=0x7f79ec00edf0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 shutdown_connections 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79dc0afaf0 0x7f79dc0b2000 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79f0108900 0x7f79f01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79f0103c00 0x7f79f0199670 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 --2- 172.21.15.94:0/1766722128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79f01029c0 0x7f79f0199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 >> 172.21.15.94:0/1766722128 conn(0x7f79f00fdbd0 msgr2=0x7f79f01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 shutdown_connections 2024-04-25T13:52:44.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.004+0000 7f79f7c1a640 1 -- 172.21.15.94:0/1766722128 wait complete. 2024-04-25T13:52:44.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:52:44.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 -- 172.21.15.94:0/3155948769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 msgr2=0x7f1a48104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 --2- 172.21.15.94:0/3155948769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a48104030 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009900 tx=0x7f1a3c02a410 comp rx=0 tx=0).stop 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 -- 172.21.15.94:0/3155948769 shutdown_connections 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 --2- 172.21.15.94:0/3155948769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 0x7f1a4810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 --2- 172.21.15.94:0/3155948769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a48104030 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 --2- 172.21.15.94:0/3155948769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 0x7f1a481036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 -- 172.21.15.94:0/3155948769 >> 172.21.15.94:0/3155948769 conn(0x7f1a480fdbd0 msgr2=0x7f1a48100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 -- 172.21.15.94:0/3155948769 shutdown_connections 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.126+0000 7f1a4d430640 1 -- 172.21.15.94:0/3155948769 wait complete. 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 Processor -- start 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 -- start start 2024-04-25T13:52:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 0x7f1a481994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 0x7f1a481a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810e4e0 con 0x7f1a48103c20 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810e360 con 0x7f1a48108920 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a4d430640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810e660 con 0x7f1a481029e0 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37668/0 (socket says 172.21.15.94:37668) 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a467fc640 1 -- 172.21.15.94:0/781683255 learned_addr learned my addr 172.21.15.94:0/781683255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a477fe640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 0x7f1a481a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.127+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 0x7f1a481994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 -- 172.21.15.94:0/781683255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 msgr2=0x7f1a481994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 0x7f1a481994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 -- 172.21.15.94:0/781683255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 msgr2=0x7f1a481a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 0x7f1a481a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 -- 172.21.15.94:0/781683255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a481a0f70 con 0x7f1a48103c20 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a467fc640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006010 tx=0x7f1a3c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a2ffff640 1 -- 172.21.15.94:0/781683255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c03f030 con 0x7f1a48103c20 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a481a1200 con 0x7f1a48103c20 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a2ffff640 1 -- 172.21.15.94:0/781683255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c003aa0 con 0x7f1a48103c20 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a2ffff640 1 -- 172.21.15.94:0/781683255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c035070 con 0x7f1a48103c20 2024-04-25T13:52:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.128+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a1740 con 0x7f1a48103c20 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 Processor -- start 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 -- start start 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a2ffff640 1 -- 172.21.15.94:0/781683255 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1a3c042030 con 0x7f1a48103c20 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651424624d0 0x56514282dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514246a6b0 0x56514282ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565142791d80 con 0x56514248b390 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56514274ba20 con 0x5651424624d0 2024-04-25T13:52:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.131+0000 7f1a543b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56514278dd40 con 0x56514246a6b0 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37672/0 (socket says 172.21.15.94:37672) 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 -- 172.21.15.94:0/1836750879 learned_addr learned my addr 172.21.15.94:0/1836750879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a467fc640 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514246a6b0 0x56514282ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651424624d0 0x56514282dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 -- 172.21.15.94:0/1836750879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514246a6b0 msgr2=0x56514282ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514246a6b0 0x56514282ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 -- 172.21.15.94:0/1836750879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651424624d0 msgr2=0x56514282dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651424624d0 0x56514282dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 -- 172.21.15.94:0/1836750879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565142756c30 con 0x56514248b390 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a477fe640 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aada07f175199cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a2ffff640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 0x7f1a300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a2d7fa640 1 -- 172.21.15.94:0/1836750879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40004d20 con 0x56514248b390 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 0x7f1a300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a2d7fa640 1 -- 172.21.15.94:0/1836750879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40004050 con 0x56514248b390 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a2ffff640 1 -- 172.21.15.94:0/781683255 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1a3c0ff230 con 0x7f1a48103c20 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.132+0000 7f1a2d7fa640 1 -- 172.21.15.94:0/1836750879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40005260 con 0x56514248b390 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- 172.21.15.94:0/1836750879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 msgr2=0x56514282d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 0x7f1a300b2000 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x56514282d530 tx=0x7f1a3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a400118d0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- 172.21.15.94:0/1836750879 shutdown_connections 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56514246a6b0 0x56514282ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651424624d0 0x56514282dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 --2- 172.21.15.94:0/1836750879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x56514282d810 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- 172.21.15.94:0/1836750879 >> 172.21.15.94:0/1836750879 conn(0x56514247f4e0 msgr2=0x565142829020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- 172.21.15.94:0/1836750879 shutdown_connections 2024-04-25T13:52:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- 172.21.15.94:0/1836750879 wait complete. 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.133+0000 7f1a543b1740 1 -- start start 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 Processor -- start 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x565142837620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514283eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56514282f530 con 0x56514246a6b0 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565142718dc0 con 0x56514248b390 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56514282f6b0 con 0x5651424624d0 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a543b1740 10 client.0 _ll_register_callbacks cb 0x565142734600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37686/0 (socket says 172.21.15.94:37686) 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 -- 172.21.15.94:0/922490772 learned_addr learned my addr 172.21.15.94:0/922490772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x565142837620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a477fe640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514283eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 -- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 msgr2=0x565142837620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x565142837620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 -- 172.21.15.94:0/922490772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 msgr2=0x56514283eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514283eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 -- 172.21.15.94:0/922490772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651424a33f0 con 0x56514246a6b0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a467fc640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c4990 tx=0x7f1a3c003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab8b3bf3d5f50eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.134+0000 7f1a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c049070 con 0x56514246a6b0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1a3c0b00b0 con 0x56514246a6b0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40004050 con 0x56514246a6b0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0c35e0 con 0x56514246a6b0 2024-04-25T13:52:44.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 10 client.19413 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56514256d7d0 con 0x56514246a6b0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40004050 con 0x56514246a6b0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1a3c0c7980 con 0x56514246a6b0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 10 client.19413 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 10 client.19413 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 20 client.19413 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x565142449f60 0x565142845260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 4 client.19413 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56514279f990 con 0x565142449f60 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.135+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x565142449f60 0x565142845260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.136+0000 7f1a2aff5640 10 client.19413 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.136+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x565142449f60 0x565142845260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4c4920fd7932f3d server_cookie=7fdf810cd5a6c02b in_seq=0 out_seq=0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.136+0000 7f1a2aff5640 1 -- 172.21.15.94:0/922490772 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56514279f990 con 0x565142449f60 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.136+0000 7f1a2aff5640 10 client.19413 handle_command_reply: tid=0 2024-04-25T13:52:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.136+0000 7f1a543b1740 1 client.19413 shutdown 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 msgr2=0x565142837b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0c4990 tx=0x7f1a3c003e80 comp rx=0 tx=0).stop 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 shutdown_connections 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x565142449f60 0x565142845260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514283eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142837b60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a2aff5640 0 client.19413 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 --2- 172.21.15.94:0/922490772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x565142837620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 >> 172.21.15.94:0/922490772 conn(0x56514247f4e0 msgr2=0x565142828040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 shutdown_connections 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 1 -- 172.21.15.94:0/922490772 wait complete. 2024-04-25T13:52:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.137+0000 7f1a543b1740 20 client.19413 trim_cache size 0 max 16384 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 Processor -- start 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 -- start start 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x5651428499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514284a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481aa420 con 0x56514246a6b0 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481abf60 con 0x56514248b390 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.150+0000 7f1a543b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481aaaf0 con 0x5651424624d0 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37694/0 (socket says 172.21.15.94:37694) 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 -- 172.21.15.94:0/3050565871 learned_addr learned my addr 172.21.15.94:0/3050565871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a477fe640 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514284a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x5651428499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 -- 172.21.15.94:0/3050565871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 msgr2=0x5651428499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x5651428499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 -- 172.21.15.94:0/3050565871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 msgr2=0x56514284a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514284a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 -- 172.21.15.94:0/3050565871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565142829580 con 0x56514246a6b0 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a467fc640 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c031440 tx=0x7f1a3c0ff5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b7c3a3703ce858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a2cff9640 1 -- 172.21.15.94:0/3050565871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c049070 con 0x56514246a6b0 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a2cff9640 1 -- 172.21.15.94:0/3050565871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c03b460 con 0x56514246a6b0 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.151+0000 7f1a2cff9640 1 -- 172.21.15.94:0/3050565871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0c6e90 con 0x56514246a6b0 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- 172.21.15.94:0/3050565871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 msgr2=0x565142849f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c031440 tx=0x7f1a3c0ff5e0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- 172.21.15.94:0/3050565871 shutdown_connections 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514248b390 0x56514284a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514246a6b0 0x565142849f00 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3050565871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x5651428499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- 172.21.15.94:0/3050565871 >> 172.21.15.94:0/3050565871 conn(0x56514247f4e0 msgr2=0x565142832350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- 172.21.15.94:0/3050565871 shutdown_connections 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- 172.21.15.94:0/3050565871 wait complete. 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 -- start start 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.152+0000 7f1a543b1740 1 Processor -- start 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x56514283e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 0x56514283e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481ab890 con 0x56514248b390 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481ab1c0 con 0x56514246a6b0 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481a8400 con 0x5651424624d0 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 10 client.0 _ll_register_callbacks cb 0x7f1a3c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37704/0 (socket says 172.21.15.94:37704) 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 -- 172.21.15.94:0/3794310055 learned_addr learned my addr 172.21.15.94:0/3794310055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a467fc640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 0x56514283e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x56514283e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 -- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 msgr2=0x56514283e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x56514283e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 -- 172.21.15.94:0/3794310055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 msgr2=0x56514283e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 0x56514283e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 -- 172.21.15.94:0/3794310055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651428ae0e0 con 0x56514248b390 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a467fc640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 0x56514283e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a477fe640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1a40014040 tx=0x7f1a40004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c428ebb9a6b69c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009a50 con 0x56514248b390 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1a40004050 con 0x56514248b390 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.153+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565142799800 con 0x56514248b390 2024-04-25T13:52:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a40004250 con 0x56514248b390 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 10 client.19419 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651428af360 con 0x56514248b390 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565142799800 con 0x56514248b390 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1a40005750 con 0x56514248b390 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 10 client.19419 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 10 client.19419 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 20 client.19419 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56514241f3a0 0x5651428b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 4 client.19419 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565142714610 con 0x56514241f3a0 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.154+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56514241f3a0 0x5651428b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a2aff5640 10 client.19419 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a46ffd640 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56514241f3a0 0x5651428b1b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d3f84e87561d18c server_cookie=c1de6f215d2d59e3 in_seq=0 out_seq=0 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a2aff5640 1 -- 172.21.15.94:0/3794310055 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565142714610 con 0x56514241f3a0 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a2aff5640 10 client.19419 handle_command_reply: tid=0 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 client.19419 shutdown 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 msgr2=0x5651428adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f1a40014040 tx=0x7f1a40004c30 comp rx=0 tx=0).stop 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 shutdown_connections 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56514241f3a0 0x5651428b1b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56514248b390 0x5651428adaa0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56514246a6b0 0x56514283e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a2aff5640 0 client.19419 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:52:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 --2- 172.21.15.94:0/3794310055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651424624d0 0x56514283e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 >> 172.21.15.94:0/3794310055 conn(0x56514247f4e0 msgr2=0x56514282be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 shutdown_connections 2024-04-25T13:52:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 1 -- 172.21.15.94:0/3794310055 wait complete. 2024-04-25T13:52:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.155+0000 7f1a543b1740 20 client.19419 trim_cache size 0 max 16384 2024-04-25T13:52:44.163 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 msgr2=0x7f1a300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 0x7f1a300b2000 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x56514282d530 tx=0x7f1a3800b390 comp rx=0 tx=0).stop 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 msgr2=0x7f1a481999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006010 tx=0x7f1a3c031a50 comp rx=0 tx=0).stop 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 shutdown_connections 2024-04-25T13:52:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a300afaf0 0x7f1a300b2000 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a48108920 0x7f1a481a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a48103c20 0x7f1a481999e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 --2- 172.21.15.94:0/781683255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a481029e0 0x7f1a481994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 >> 172.21.15.94:0/781683255 conn(0x7f1a480fdbd0 msgr2=0x7f1a48104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 shutdown_connections 2024-04-25T13:52:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.156+0000 7f1a4d430640 1 -- 172.21.15.94:0/781683255 wait complete. 2024-04-25T13:52:44.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:52:44.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 -- 172.21.15.94:0/2381128947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 msgr2=0x7fdf2c10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 --2- 172.21.15.94:0/2381128947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c10bc90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009ca0 tx=0x7fdf2402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 -- 172.21.15.94:0/2381128947 shutdown_connections 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 --2- 172.21.15.94:0/2381128947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c10bc90 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 --2- 172.21.15.94:0/2381128947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 --2- 172.21.15.94:0/2381128947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 -- 172.21.15.94:0/2381128947 >> 172.21.15.94:0/2381128947 conn(0x7fdf2c0fd350 msgr2=0x7fdf2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 -- 172.21.15.94:0/2381128947 shutdown_connections 2024-04-25T13:52:44.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.280+0000 7fdf33c34640 1 -- 172.21.15.94:0/2381128947 wait complete. 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 Processor -- start 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 -- start start 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c10f5f0 con 0x7fdf2c109780 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c10f470 con 0x7fdf2c102360 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.281+0000 7fdf33c34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c10f770 con 0x7fdf2c101e60 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37712/0 (socket says 172.21.15.94:37712) 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 -- 172.21.15.94:0/1877323022 learned_addr learned my addr 172.21.15.94:0/1877323022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf311cf640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf319d0640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 -- 172.21.15.94:0/1877323022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 msgr2=0x7fdf2c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 -- 172.21.15.94:0/1877323022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 msgr2=0x7fdf2c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 -- 172.21.15.94:0/1877323022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1a1ca0 con 0x7fdf2c109780 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf311cf640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf319d0640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf321d1640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004af0 tx=0x7fdf24031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf22ffd640 1 -- 172.21.15.94:0/1877323022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24003b70 con 0x7fdf2c109780 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1a1f30 con 0x7fdf2c109780 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.282+0000 7fdf22ffd640 1 -- 172.21.15.94:0/1877323022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf24003d10 con 0x7fdf2c109780 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.283+0000 7fdf22ffd640 1 -- 172.21.15.94:0/1877323022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2402fb20 con 0x7fdf2c109780 2024-04-25T13:52:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.283+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a23a0 con 0x7fdf2c109780 2024-04-25T13:52:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.285+0000 7fdf22ffd640 1 -- 172.21.15.94:0/1877323022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdf24004580 con 0x7fdf2c109780 2024-04-25T13:52:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.285+0000 7fdf3abc2740 1 Processor -- start 2024-04-25T13:52:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.285+0000 7fdf3abc2740 1 -- start start 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.285+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e81a390 0x56040ebbc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.285+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebbda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56040eb20d80 con 0x56040e7f14d0 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56040eadaa20 con 0x56040e81a390 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56040eb1cd40 con 0x56040e7f96b0 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf311cf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e81a390 0x56040ebbc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37726/0 (socket says 172.21.15.94:37726) 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 -- 172.21.15.94:0/4038925040 learned_addr learned my addr 172.21.15.94:0/4038925040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebbda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf22ffd640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 0x7fdf180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 -- 172.21.15.94:0/4038925040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 msgr2=0x56040ebbda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf321d1640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 0x7fdf180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebbda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 -- 172.21.15.94:0/4038925040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e81a390 msgr2=0x56040ebbc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e81a390 0x56040ebbc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 -- 172.21.15.94:0/4038925040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040eae5c30 con 0x56040e7f14d0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf22ffd640 1 -- 172.21.15.94:0/1877323022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdf24100280 con 0x7fdf2c109780 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdf280115c0 tx=0x7fdf28011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e455a35bd382380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.286+0000 7fdf17fff640 1 -- 172.21.15.94:0/4038925040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800f640 con 0x56040e7f14d0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.287+0000 7fdf17fff640 1 -- 172.21.15.94:0/4038925040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2800fc90 con 0x56040e7f14d0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.287+0000 7fdf321d1640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 0x7fdf180b2000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdf240c58a0 tx=0x7fdf24095ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.287+0000 7fdf17fff640 1 -- 172.21.15.94:0/4038925040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28019bf0 con 0x56040e7f14d0 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4038925040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 msgr2=0x56040ebbcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdf280115c0 tx=0x7fdf28011a80 comp rx=0 tx=0).stop 2024-04-25T13:52:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4038925040 shutdown_connections 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebbda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebbcd90 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4038925040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e81a390 0x56040ebbc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4038925040 >> 172.21.15.94:0/4038925040 conn(0x56040e80e4e0 msgr2=0x56040ebb8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4038925040 shutdown_connections 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4038925040 wait complete. 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 -- start start 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.288+0000 7fdf3abc2740 1 Processor -- start 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebc6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 0x56040ebcda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56040ebbe530 con 0x56040e7f96b0 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56040eaa7dc0 con 0x56040e7f14d0 2024-04-25T13:52:44.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56040ebbe6b0 con 0x56040e81a390 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 10 client.0 _ll_register_callbacks cb 0x56040eac3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37742/0 (socket says 172.21.15.94:37742) 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 -- 172.21.15.94:0/4040598317 learned_addr learned my addr 172.21.15.94:0/4040598317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebc6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 0x56040ebcda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 -- 172.21.15.94:0/4040598317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 msgr2=0x56040ebcda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 0x56040ebcda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 -- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 msgr2=0x56040ebc6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebc6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 -- 172.21.15.94:0/4040598317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040e8323f0 con 0x56040e7f96b0 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 0x56040ebcda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf319d0640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdf28004270 tx=0x7fdf280042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f04196ad032e476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf157fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28027060 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdf2800f850 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2800fc90 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.289+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf2800fe40 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 10 client.19431 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56040e8fc7d0 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2800fc90 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdf28005300 con 0x56040e7f96b0 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 10 client.19431 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 10 client.19431 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 20 client.19431 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e7d8f60 0x56040ebd4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 4 client.19431 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:44.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56040eb2e990 con 0x56040e7d8f60 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e7d8f60 0x56040ebd4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf157fa640 10 client.19431 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.290+0000 7fdf311cf640 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e7d8f60 0x56040ebd4070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a2fc9a6a6012c33 server_cookie=a242bd988e528b36 in_seq=0 out_seq=0 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.291+0000 7fdf157fa640 1 -- 172.21.15.94:0/4040598317 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56040eb2e990 con 0x56040e7d8f60 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.291+0000 7fdf157fa640 10 client.19431 handle_command_reply: tid=0 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.291+0000 7fdf3abc2740 1 client.19431 shutdown 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.291+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 msgr2=0x56040ebc6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.291+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdf28004270 tx=0x7fdf280042a0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 shutdown_connections 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e7d8f60 0x56040ebd4070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e81a390 0x56040ebcda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebc6aa0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf157fa640 0 client.19431 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/4040598317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebc6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 >> 172.21.15.94:0/4040598317 conn(0x56040e80e4e0 msgr2=0x56040ebbaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 shutdown_connections 2024-04-25T13:52:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 1 -- 172.21.15.94:0/4040598317 wait complete. 2024-04-25T13:52:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.292+0000 7fdf3abc2740 20 client.19431 trim_cache size 0 max 16384 2024-04-25T13:52:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 Processor -- start 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 -- start start 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7d8f60 0x56040ebb8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebc4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ab4a0 con 0x56040e7f14d0 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1acfe0 con 0x56040e7d8f60 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1abb70 con 0x56040e7f96b0 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37750/0 (socket says 172.21.15.94:37750) 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 -- 172.21.15.94:0/1859367496 learned_addr learned my addr 172.21.15.94:0/1859367496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf311cf640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7d8f60 0x56040ebb8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf311cf640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebc4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 -- 172.21.15.94:0/1859367496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 msgr2=0x56040ebc4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebc4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 -- 172.21.15.94:0/1859367496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7d8f60 msgr2=0x56040ebb8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7d8f60 0x56040ebb8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 -- 172.21.15.94:0/1859367496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040ebcde60 con 0x56040e7f14d0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf311cf640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebc4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf319d0640 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009d10 tx=0x7fdf28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73bcf61726b8f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf177fe640 1 -- 172.21.15.94:0/1859367496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28027040 con 0x56040e7f14d0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf177fe640 1 -- 172.21.15.94:0/1859367496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf28004510 con 0x56040e7f14d0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.306+0000 7fdf177fe640 1 -- 172.21.15.94:0/1859367496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800fab0 con 0x56040e7f14d0 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 -- 172.21.15.94:0/1859367496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 msgr2=0x56040ebcd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009d10 tx=0x7fdf28005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 -- 172.21.15.94:0/1859367496 shutdown_connections 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7f96b0 0x56040ebc4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f14d0 0x56040ebcd920 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/1859367496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7d8f60 0x56040ebb8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 -- 172.21.15.94:0/1859367496 >> 172.21.15.94:0/1859367496 conn(0x56040e80e4e0 msgr2=0x56040ebb7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 -- 172.21.15.94:0/1859367496 shutdown_connections 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.307+0000 7fdf3abc2740 1 -- 172.21.15.94:0/1859367496 wait complete. 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 -- start start 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 Processor -- start 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7d8f60 0x56040ebcadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebcb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ac910 con 0x56040e7f96b0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ac240 con 0x56040e7f14d0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1a9480 con 0x56040e7d8f60 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf3abc2740 10 client.0 _ll_register_callbacks cb 0x7fdf1c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf311cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf311cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37754/0 (socket says 172.21.15.94:37754) 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf311cf640 1 -- 172.21.15.94:0/2921480194 learned_addr learned my addr 172.21.15.94:0/2921480194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebcb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf311cf640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7d8f60 0x56040ebcadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 -- 172.21.15.94:0/2921480194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7d8f60 msgr2=0x56040ebcadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7d8f60 0x56040ebcadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 -- 172.21.15.94:0/2921480194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 msgr2=0x56040ebcc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf319d0640 1 -- 172.21.15.94:0/2921480194 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040ebcc800 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.308+0000 7fdf311cf640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf319d0640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebcb310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56040ebbe1e0 tx=0x7fdf280044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0558048395fb578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2803a070 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf28004510 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdf2800fa60 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf28046050 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 10 client.28883 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56040ebcd9f0 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf28004510 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdf28034020 con 0x56040e7f14d0 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 10 client.28883 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 10 client.28883 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 20 client.28883 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e81a390 0x56040ec3e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 4 client.28883 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.309+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56040eb28800 con 0x56040e81a390 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf311cf640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e81a390 0x56040ec3e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf311cf640 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e81a390 0x56040ec3e8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a72cccc0bfbc69d0 server_cookie=4ed623dede42d2ed in_seq=0 out_seq=0 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf157fa640 10 client.28883 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf157fa640 1 -- 172.21.15.94:0/2921480194 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56040eb28800 con 0x56040e81a390 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf157fa640 10 client.28883 handle_command_reply: tid=0 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 client.28883 shutdown 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 msgr2=0x56040ebcb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebcb310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56040ebbe1e0 tx=0x7fdf280044a0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 shutdown_connections 2024-04-25T13:52:44.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56040e81a390 0x56040ec3e8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56040e7f96b0 0x56040ebcc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56040e7f14d0 0x56040ebcb310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf157fa640 0 client.28883 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.310+0000 7fdf3abc2740 1 --2- 172.21.15.94:0/2921480194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56040e7d8f60 0x56040ebcadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 >> 172.21.15.94:0/2921480194 conn(0x56040e80e4e0 msgr2=0x56040ebbae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 shutdown_connections 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf3abc2740 1 -- 172.21.15.94:0/2921480194 wait complete. 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf3abc2740 20 client.28883 trim_cache size 0 max 16384 2024-04-25T13:52:44.318 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:44.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 msgr2=0x7fdf180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 0x7fdf180b2000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdf240c58a0 tx=0x7fdf24095ee0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 msgr2=0x7fdf2c1a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004af0 tx=0x7fdf24031700 comp rx=0 tx=0).stop 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 shutdown_connections 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf180afaf0 0x7fdf180b2000 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf2c109780 0x7fdf2c1a1630 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf2c102360 0x7fdf2c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 --2- 172.21.15.94:0/1877323022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101e60 0x7fdf2c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 >> 172.21.15.94:0/1877323022 conn(0x7fdf2c0fd350 msgr2=0x7fdf2c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 shutdown_connections 2024-04-25T13:52:44.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.311+0000 7fdf33c34640 1 -- 172.21.15.94:0/1877323022 wait complete. 2024-04-25T13:52:44.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:44 smithi094 ceph-mon[29195]: pgmap v677: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-04-25T13:52:44.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:44 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1766722128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:44.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:52:44.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 -- 172.21.15.94:0/106670080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 msgr2=0x7f98981054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 --2- 172.21.15.94:0/106670080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981054a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989402a410 comp rx=0 tx=0).stop 2024-04-25T13:52:44.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 -- 172.21.15.94:0/106670080 shutdown_connections 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 --2- 172.21.15.94:0/106670080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898102240 0x7f989810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 --2- 172.21.15.94:0/106670080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898101740 0x7f9898101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 --2- 172.21.15.94:0/106670080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981054a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 -- 172.21.15.94:0/106670080 >> 172.21.15.94:0/106670080 conn(0x7f98980fd340 msgr2=0x7f98980ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 -- 172.21.15.94:0/106670080 shutdown_connections 2024-04-25T13:52:44.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.439+0000 7f989f37d640 1 -- 172.21.15.94:0/106670080 wait complete. 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 Processor -- start 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 -- start start 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898101740 0x7f989819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898102240 0x7f989819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f989810f600 con 0x7f9898105090 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f989810f480 con 0x7f9898101740 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989f37d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f989810f780 con 0x7f9898102240 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989d91a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989d91a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37770/0 (socket says 172.21.15.94:37770) 2024-04-25T13:52:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989d91a640 1 -- 172.21.15.94:0/2886434454 learned_addr learned my addr 172.21.15.94:0/2886434454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989c918640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898102240 0x7f989819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.440+0000 7f989d119640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898101740 0x7f989819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 -- 172.21.15.94:0/2886434454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898102240 msgr2=0x7f989819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898102240 0x7f989819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 -- 172.21.15.94:0/2886434454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898101740 msgr2=0x7f989819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898101740 0x7f989819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 -- 172.21.15.94:0/2886434454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98981a1da0 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989d91a640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f989000eb10 tx=0x7f989000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f988e7fc640 1 -- 172.21.15.94:0/2886434454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890004b20 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f988e7fc640 1 -- 172.21.15.94:0/2886434454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890004cc0 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98981a2090 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f988e7fc640 1 -- 172.21.15.94:0/2886434454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890010670 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.441+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981a2590 con 0x7f9898105090 2024-04-25T13:52:44.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 Processor -- start 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f988e7fc640 1 -- 172.21.15.94:0/2886434454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9890010830 con 0x7f9898105090 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 -- start start 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e74e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e74fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56179e6b2d80 con 0x56179e3834d0 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56179e66ca20 con 0x56179e3ac390 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f98a622b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179e6aed40 con 0x56179e38b6b0 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.444+0000 7f989d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37780/0 (socket says 172.21.15.94:37780) 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989c918640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e74e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 -- 172.21.15.94:0/1960370802 learned_addr learned my addr 172.21.15.94:0/1960370802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989c918640 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e74fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f988e7fc640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 0x7f98840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 -- 172.21.15.94:0/1960370802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 msgr2=0x56179e74fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e74fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 -- 172.21.15.94:0/1960370802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 msgr2=0x56179e74e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e74e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 -- 172.21.15.94:0/1960370802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179e677c30 con 0x56179e3834d0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d91a640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 0x7f98840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f989d119640 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9894009230 tx=0x7f9894009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf4a4ae790277c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f988e7fc640 1 -- 172.21.15.94:0/2886434454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f98900df240 con 0x7f9898105090 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f98837fe640 1 -- 172.21.15.94:0/1960370802 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894035030 con 0x56179e3834d0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f98837fe640 1 -- 172.21.15.94:0/1960370802 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894040070 con 0x56179e3834d0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.445+0000 7f98837fe640 1 -- 172.21.15.94:0/1960370802 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894003830 con 0x56179e3834d0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 -- 172.21.15.94:0/1960370802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 msgr2=0x56179e74ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f989d91a640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 0x7f98840b2000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f98900a48f0 tx=0x7f9890071ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9894009230 tx=0x7f9894009620 comp rx=0 tx=0).stop 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 -- 172.21.15.94:0/1960370802 shutdown_connections 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e74fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e74ed90 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 --2- 172.21.15.94:0/1960370802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e74e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 -- 172.21.15.94:0/1960370802 >> 172.21.15.94:0/1960370802 conn(0x56179e3a04e0 msgr2=0x56179e74a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 -- 172.21.15.94:0/1960370802 shutdown_connections 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.446+0000 7f98a622b740 1 -- 172.21.15.94:0/1960370802 wait complete. 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 -- start start 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 Processor -- start 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e75cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e763e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56179e7504d0 con 0x56179e3834d0 2024-04-25T13:52:44.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56179e639dc0 con 0x56179e3ac390 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179e750650 con 0x56179e38b6b0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f98a622b740 10 client.0 _ll_register_callbacks cb 0x56179e655600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f989c918640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f989c918640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37796/0 (socket says 172.21.15.94:37796) 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f989c918640 1 -- 172.21.15.94:0/2203913635 learned_addr learned my addr 172.21.15.94:0/2203913635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.447+0000 7f989d119640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e75cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e763e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 -- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 msgr2=0x56179e75cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e75cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 -- 172.21.15.94:0/2203913635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 msgr2=0x56179e763e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e763e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 -- 172.21.15.94:0/2203913635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179e3c43f0 con 0x56179e3834d0 2024-04-25T13:52:44.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989d119640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e75cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f989c918640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f9888004550 tx=0x7f9888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8a9c3d0e07c2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f9880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888003e90 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9894040070 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9888005020 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9888005220 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f98a622b740 10 client.19449 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.448+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56179e48e7d0 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894040070 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f988801d030 con 0x56179e3834d0 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 10 client.19449 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 10 client.19449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 20 client.19449 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e36af60 0x56179e76a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 4 client.19449 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179e6c0990 con 0x56179e36af60 2024-04-25T13:52:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f989d119640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e36af60 0x56179e76a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f9880ff9640 10 client.19449 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:44.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.449+0000 7f989d119640 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e36af60 0x56179e76a700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5dcc341e9bef4ff server_cookie=e22c5e0b14f68a50 in_seq=0 out_seq=0 2024-04-25T13:52:44.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f9880ff9640 1 -- 172.21.15.94:0/2203913635 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56179e6c0990 con 0x56179e36af60 2024-04-25T13:52:44.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f9880ff9640 10 client.19449 handle_command_reply: tid=0 2024-04-25T13:52:44.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 client.19449 shutdown 2024-04-25T13:52:44.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 msgr2=0x56179e75c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f9888004550 tx=0x7f9888004730 comp rx=0 tx=0).stop 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 shutdown_connections 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e36af60 0x56179e76a700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3ac390 0x56179e763e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e75cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f9880ff9640 0 client.19449 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 --2- 172.21.15.94:0/2203913635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e75c9c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 >> 172.21.15.94:0/2203913635 conn(0x56179e3a04e0 msgr2=0x56179e74cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 shutdown_connections 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 1 -- 172.21.15.94:0/2203913635 wait complete. 2024-04-25T13:52:44.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.451+0000 7f98a622b740 20 client.19449 trim_cache size 0 max 16384 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 Processor -- start 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 -- start start 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 0x56179e749a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e38b6b0 0x56179e749f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98981a9350 con 0x56179e3834d0 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98981aae90 con 0x56179e38b6b0 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f98a622b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98981a9a20 con 0x56179e36af60 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37810/0 (socket says 172.21.15.94:37810) 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 -- 172.21.15.94:0/4195157091 learned_addr learned my addr 172.21.15.94:0/4195157091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989c918640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 0x56179e749a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989c918640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e38b6b0 0x56179e749f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 -- 172.21.15.94:0/4195157091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 msgr2=0x56179e749a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 0x56179e749a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 -- 172.21.15.94:0/4195157091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e38b6b0 msgr2=0x56179e749f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e38b6b0 0x56179e749f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 -- 172.21.15.94:0/4195157091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179e7582d0 con 0x56179e3834d0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989c918640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 0x56179e749a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f989d119640 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9894005f70 tx=0x7f9894005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39ba29caf366fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f9882ffd640 1 -- 172.21.15.94:0/4195157091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894035070 con 0x56179e3834d0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f9882ffd640 1 -- 172.21.15.94:0/4195157091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894040050 con 0x56179e3834d0 2024-04-25T13:52:44.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.466+0000 7f9882ffd640 1 -- 172.21.15.94:0/4195157091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894003b60 con 0x56179e3834d0 2024-04-25T13:52:44.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- 172.21.15.94:0/4195157091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 msgr2=0x56179e763df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9894005f70 tx=0x7f9894005fd0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- 172.21.15.94:0/4195157091 shutdown_connections 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e38b6b0 0x56179e749f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e3834d0 0x56179e763df0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 --2- 172.21.15.94:0/4195157091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e36af60 0x56179e749a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- 172.21.15.94:0/4195157091 >> 172.21.15.94:0/4195157091 conn(0x56179e3a04e0 msgr2=0x56179e75afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- 172.21.15.94:0/4195157091 shutdown_connections 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- 172.21.15.94:0/4195157091 wait complete. 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.467+0000 7f98a622b740 1 -- start start 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 Processor -- start 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e7d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98981aa7c0 con 0x56179e36af60 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98981aa0f0 con 0x56179e3834d0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98981a7330 con 0x56179e38b6b0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f98a622b740 10 client.0 _ll_register_callbacks cb 0x7f9888009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989d119640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37826/0 (socket says 172.21.15.94:37826) 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 -- 172.21.15.94:0/2918410424 learned_addr learned my addr 172.21.15.94:0/2918410424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989d119640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37110/0 (socket says 172.21.15.94:37110) 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e7d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 -- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 msgr2=0x56179e7d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e7d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 -- 172.21.15.94:0/2918410424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 msgr2=0x56179e75f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989c918640 1 -- 172.21.15.94:0/2918410424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179e7d0fa0 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.468+0000 7f989d119640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f989c918640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f988802b520 tx=0x7f9888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448ebeebdcf6ff7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98880047a0 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9888004940 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9894040050 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9888004b40 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 10 client.19455 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56179e7d20f0 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894040050 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9888034070 con 0x56179e36af60 2024-04-25T13:52:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 10 client.19455 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 10 client.19455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 20 client.19455 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.469+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e3ac390 0x56179e7d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 4 client.19455 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56179e6ba800 con 0x56179e3ac390 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f989d119640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e3ac390 0x56179e7d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f989d119640 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e3ac390 0x56179e7d4a30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eb21a6042fbc501 server_cookie=1f802562ae2cd995 in_seq=0 out_seq=0 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f9880ff9640 10 client.19455 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f9880ff9640 1 -- 172.21.15.94:0/2918410424 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56179e6ba800 con 0x56179e3ac390 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f9880ff9640 10 client.19455 handle_command_reply: tid=0 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 client.19455 shutdown 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 msgr2=0x56179e75edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f988802b520 tx=0x7f9888004730 comp rx=0 tx=0).stop 2024-04-25T13:52:44.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 shutdown_connections 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56179e3ac390 0x56179e7d4a30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179e38b6b0 0x56179e7d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56179e3834d0 0x56179e75f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 --2- 172.21.15.94:0/2918410424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56179e36af60 0x56179e75edd0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f9880ff9640 0 client.19455 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.470+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 >> 172.21.15.94:0/2918410424 conn(0x56179e3a04e0 msgr2=0x56179e7499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 shutdown_connections 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f98a622b740 1 -- 172.21.15.94:0/2918410424 wait complete. 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f98a622b740 20 client.19455 trim_cache size 0 max 16384 2024-04-25T13:52:44.477 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 msgr2=0x7f98840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 0x7f98840b2000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f98900a48f0 tx=0x7f9890071ea0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 msgr2=0x7f98981a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f989000eb10 tx=0x7f989000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 shutdown_connections 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98840afaf0 0x7f98840b2000 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9898105090 0x7f98981a1730 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9898102240 0x7f989819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 --2- 172.21.15.94:0/2886434454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9898101740 0x7f989819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 >> 172.21.15.94:0/2886434454 conn(0x7f98980fd340 msgr2=0x7f9898105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 shutdown_connections 2024-04-25T13:52:44.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.471+0000 7f989f37d640 1 -- 172.21.15.94:0/2886434454 wait complete. 2024-04-25T13:52:44.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:52:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:44 smithi138 ceph-mon[32977]: pgmap v677: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-04-25T13:52:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1766722128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 -- 172.21.15.94:0/3116797765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968109890 msgr2=0x7f696810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 --2- 172.21.15.94:0/3116797765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968109890 0x7f696810bcb0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6960009ca0 tx=0x7f696002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 -- 172.21.15.94:0/3116797765 shutdown_connections 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 --2- 172.21.15.94:0/3116797765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968109890 0x7f696810bcb0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 --2- 172.21.15.94:0/3116797765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6968106f30 0x7f6968109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 --2- 172.21.15.94:0/3116797765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69681045d0 0x7f69681069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 -- 172.21.15.94:0/3116797765 >> 172.21.15.94:0/3116797765 conn(0x7f69680fdbd0 msgr2=0x7f6968100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 -- 172.21.15.94:0/3116797765 shutdown_connections 2024-04-25T13:52:44.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.599+0000 7f696e6f2640 1 -- 172.21.15.94:0/3116797765 wait complete. 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 Processor -- start 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 -- start start 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 0x7f696819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 0x7f69681a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f696810f610 con 0x7f6968106f30 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f696810f490 con 0x7f6968109890 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696e6f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f696810f790 con 0x7f69681045d0 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37830/0 (socket says 172.21.15.94:37830) 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f696cc8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 0x7f69681a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f69677fe640 1 -- 172.21.15.94:0/274718810 learned_addr learned my addr 172.21.15.94:0/274718810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.600+0000 7f6967fff640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 0x7f696819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 -- 172.21.15.94:0/274718810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 msgr2=0x7f696819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 0x7f696819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 -- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 msgr2=0x7f696819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 -- 172.21.15.94:0/274718810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a2080 con 0x7f6968109890 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f69677fe640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f6967fff640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 0x7f696819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696cc8f640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 0x7f69681a1a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6960033040 tx=0x7f6960031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f69657fa640 1 -- 172.21.15.94:0/274718810 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960003b70 con 0x7f6968109890 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a2310 con 0x7f6968109890 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f69657fa640 1 -- 172.21.15.94:0/274718810 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6960003d10 con 0x7f6968109890 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f69657fa640 1 -- 172.21.15.94:0/274718810 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696002fa50 con 0x7f6968109890 2024-04-25T13:52:44.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.601+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681a28b0 con 0x7f6968109890 2024-04-25T13:52:44.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69657fa640 1 -- 172.21.15.94:0/274718810 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f696002fc10 con 0x7f6968109890 2024-04-25T13:52:44.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 Processor -- start 2024-04-25T13:52:44.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 -- start start 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x5584701511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470151e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5584700b0d80 con 0x55846fdaa390 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55847006aa20 con 0x55846fd814d0 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69755a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584700acd40 con 0x55846fd896b0 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37836/0 (socket says 172.21.15.94:37836) 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69677fe640 1 -- 172.21.15.94:0/4058106696 learned_addr learned my addr 172.21.15.94:0/4058106696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f6967fff640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x5584701511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.604+0000 7f69677fe640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69657fa640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 0x7f69500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 -- 172.21.15.94:0/4058106696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 msgr2=0x558470151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 -- 172.21.15.94:0/4058106696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 msgr2=0x5584701511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x5584701511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 -- 172.21.15.94:0/4058106696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558470075c30 con 0x55846fdaa390 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f6967fff640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x5584701511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f696cc8f640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 0x7f69500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69677fe640 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f6958013d20 tx=0x7f6958009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5c1f7a34c03e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f69657fa640 1 -- 172.21.15.94:0/274718810 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f69601000d0 con 0x7f6968109890 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f694e7fc640 1 -- 172.21.15.94:0/4058106696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800fc50 con 0x55846fdaa390 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f694e7fc640 1 -- 172.21.15.94:0/4058106696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695800fdf0 con 0x55846fdaa390 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.605+0000 7f694e7fc640 1 -- 172.21.15.94:0/4058106696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958010d00 con 0x55846fdaa390 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 -- 172.21.15.94:0/4058106696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 msgr2=0x558470150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f696cc8f640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 0x7f69500b2000 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f69600fdb80 tx=0x7f6960092ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f6958013d20 tx=0x7f6958009c30 comp rx=0 tx=0).stop 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 -- 172.21.15.94:0/4058106696 shutdown_connections 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x5584701511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 --2- 172.21.15.94:0/4058106696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x558470150c20 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 -- 172.21.15.94:0/4058106696 >> 172.21.15.94:0/4058106696 conn(0x55846fd9e4e0 msgr2=0x558470148020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 -- 172.21.15.94:0/4058106696 shutdown_connections 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.606+0000 7f69755a4740 1 -- 172.21.15.94:0/4058106696 wait complete. 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 -- start start 2024-04-25T13:52:44.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 Processor -- start 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 0x55847015af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fdaa390 0x558470161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558470152940 con 0x55846fd896b0 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558470037dc0 con 0x55846fd814d0 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558470152ac0 con 0x55846fdaa390 2024-04-25T13:52:44.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69755a4740 10 client.0 _ll_register_callbacks cb 0x558470053600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f69677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.607+0000 7f6967fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 0x55847015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37144/0 (socket says 172.21.15.94:37144) 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 -- 172.21.15.94:0/426966841 learned_addr learned my addr 172.21.15.94:0/426966841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fdaa390 0x558470161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 -- 172.21.15.94:0/426966841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fdaa390 msgr2=0x558470161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fdaa390 0x558470161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 -- 172.21.15.94:0/426966841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 msgr2=0x55847015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 0x55847015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 -- 172.21.15.94:0/426966841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846fdc23f0 con 0x55846fd814d0 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f6967fff640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 0x55847015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69677fe640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f69580097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838419db1d6cdcc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69477fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958010c90 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695800fdf0 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.608+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6958005050 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6958010e30 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 10 client.28913 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55846fe8c7d0 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695800fdf0 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6958020410 con 0x55846fd814d0 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 10 client.28913 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 10 client.28913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 20 client.28913 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd68f60 0x558470168390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 4 client.28913 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584700be990 con 0x55846fd68f60 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.609+0000 7f6967fff640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd68f60 0x558470168390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.610+0000 7f69477fe640 10 client.28913 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.610+0000 7f6967fff640 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd68f60 0x558470168390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a89c0eff18cb8cda server_cookie=8046f0dbe1c44112 in_seq=0 out_seq=0 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.610+0000 7f69477fe640 1 -- 172.21.15.94:0/426966841 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584700be990 con 0x55846fd68f60 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.610+0000 7f69477fe640 10 client.28913 handle_command_reply: tid=0 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.610+0000 7f69755a4740 1 client.28913 shutdown 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 msgr2=0x55847015aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6958006010 tx=0x7f69580097a0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 shutdown_connections 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd68f60 0x558470168390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fdaa390 0x558470161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd896b0 0x55847015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69477fe640 0 client.28913 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 --2- 172.21.15.94:0/426966841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015aa30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 >> 172.21.15.94:0/426966841 conn(0x55846fd9e4e0 msgr2=0x558470147040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 shutdown_connections 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 1 -- 172.21.15.94:0/426966841 wait complete. 2024-04-25T13:52:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.611+0000 7f69755a4740 20 client.28913 trim_cache size 0 max 16384 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 Processor -- start 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 -- start start 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 0x5584701557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fdaa390 0x558470146610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69681a9300 con 0x55846fd814d0 2024-04-25T13:52:44.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69681aae40 con 0x55846fdaa390 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69755a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69681a99d0 con 0x55846fd896b0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 0x5584701557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 0x5584701557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37856/0 (socket says 172.21.15.94:37856) 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69677fe640 1 -- 172.21.15.94:0/4089554699 learned_addr learned my addr 172.21.15.94:0/4089554699 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f69677fe640 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fdaa390 0x558470146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 -- 172.21.15.94:0/4089554699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fdaa390 msgr2=0x558470146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fdaa390 0x558470146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 -- 172.21.15.94:0/4089554699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 msgr2=0x5584701557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 0x5584701557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.626+0000 7f6967fff640 1 -- 172.21.15.94:0/4089554699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558470148bb0 con 0x55846fd896b0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.627+0000 7f6967fff640 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470155ce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x558470168aa0 tx=0x7f695c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25dce5f8fbb4a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.627+0000 7f694dffb640 1 -- 172.21.15.94:0/4089554699 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c012110 con 0x55846fd896b0 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.627+0000 7f694dffb640 1 -- 172.21.15.94:0/4089554699 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695c0054d0 con 0x55846fd896b0 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.627+0000 7f694dffb640 1 -- 172.21.15.94:0/4089554699 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c004a90 con 0x55846fd896b0 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- 172.21.15.94:0/4089554699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 msgr2=0x558470155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470155ce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x558470168aa0 tx=0x7f695c005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- 172.21.15.94:0/4089554699 shutdown_connections 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fdaa390 0x558470146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x558470155ce0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 --2- 172.21.15.94:0/4089554699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fd814d0 0x5584701557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- 172.21.15.94:0/4089554699 >> 172.21.15.94:0/4089554699 conn(0x55846fd9e4e0 msgr2=0x558470159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- 172.21.15.94:0/4089554699 shutdown_connections 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- 172.21.15.94:0/4089554699 wait complete. 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 -- start start 2024-04-25T13:52:44.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.628+0000 7f69755a4740 1 Processor -- start 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x55847015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69681aa770 con 0x55846fdaa390 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69681aa0a0 con 0x55846fd814d0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69681a72e0 con 0x55846fd896b0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69755a4740 10 client.0 _ll_register_callbacks cb 0x7f695c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f6967fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x55847015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f6967fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59784/0 (socket says 172.21.15.94:59784) 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f6967fff640 1 -- 172.21.15.94:0/1943136120 learned_addr learned my addr 172.21.15.94:0/1943136120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 -- 172.21.15.94:0/1943136120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 msgr2=0x55847015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 -- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 msgr2=0x55847015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 -- 172.21.15.94:0/1943136120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558470160440 con 0x55846fdaa390 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f6967fff640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69677fe640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x55847015fe80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6958010540 tx=0x7f6958009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12afcc375c51c35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69477fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.629+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800fe60 con 0x55846fdaa390 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6958020c60 con 0x55846fdaa390 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c0054d0 con 0x55846fdaa390 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f695802e440 con 0x55846fdaa390 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 10 client.19473 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584701616c0 con 0x55846fdaa390 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f695c0054d0 con 0x55846fdaa390 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6958043070 con 0x55846fdaa390 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 10 client.19473 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 10 client.19473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 20 client.19473 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd3e3a0 0x5584701d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 4 client.19473 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584700b8800 con 0x55846fd3e3a0 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.630+0000 7f6967fff640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd3e3a0 0x5584701d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69477fe640 10 client.19473 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f6967fff640 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd3e3a0 0x5584701d2be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da6d0b1f7039c625 server_cookie=d64e484d81a3fb37 in_seq=0 out_seq=0 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69477fe640 1 -- 172.21.15.94:0/1943136120 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584700b8800 con 0x55846fd3e3a0 2024-04-25T13:52:44.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69477fe640 10 client.19473 handle_command_reply: tid=0 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 client.19473 shutdown 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 msgr2=0x55847015fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x55847015fe80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6958010540 tx=0x7f6958009c30 comp rx=0 tx=0).stop 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 shutdown_connections 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55846fd3e3a0 0x5584701d2be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55846fdaa390 0x55847015fe80 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69477fe640 0 client.19473 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846fd896b0 0x55847015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 --2- 172.21.15.94:0/1943136120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55846fd814d0 0x55847015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 >> 172.21.15.94:0/1943136120 conn(0x55846fd9e4e0 msgr2=0x55847015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 shutdown_connections 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.631+0000 7f69755a4740 1 -- 172.21.15.94:0/1943136120 wait complete. 2024-04-25T13:52:44.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f69755a4740 20 client.19473 trim_cache size 0 max 16384 2024-04-25T13:52:44.639 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 msgr2=0x7f69500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 0x7f69500b2000 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f69600fdb80 tx=0x7f6960092ca0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 msgr2=0x7f69681a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 0x7f69681a1a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6960033040 tx=0x7f6960031680 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 shutdown_connections 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69500afaf0 0x7f69500b2000 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6968109890 0x7f69681a1a10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6968106f30 0x7f696819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 --2- 172.21.15.94:0/274718810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69681045d0 0x7f696819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 >> 172.21.15.94:0/274718810 conn(0x7f69680fdbd0 msgr2=0x7f6968105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 shutdown_connections 2024-04-25T13:52:44.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.632+0000 7f696e6f2640 1 -- 172.21.15.94:0/274718810 wait complete. 2024-04-25T13:52:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:44 smithi137 ceph-mon[32568]: pgmap v677: 97 pgs: 97 active+clean; 44 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-04-25T13:52:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:44 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1766722128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:52:44.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 -- 172.21.15.94:0/4087829928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc108920 msgr2=0x7f12cc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 --2- 172.21.15.94:0/4087829928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc108920 0x7f12cc10ad40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009ca0 tx=0x7f12c402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 -- 172.21.15.94:0/4087829928 shutdown_connections 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 --2- 172.21.15.94:0/4087829928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc108920 0x7f12cc10ad40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 --2- 172.21.15.94:0/4087829928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc103c20 0x7f12cc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 --2- 172.21.15.94:0/4087829928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc1029e0 0x7f12cc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 -- 172.21.15.94:0/4087829928 >> 172.21.15.94:0/4087829928 conn(0x7f12cc0fdbd0 msgr2=0x7f12cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 -- 172.21.15.94:0/4087829928 shutdown_connections 2024-04-25T13:52:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.758+0000 7f12d39a8640 1 -- 172.21.15.94:0/4087829928 wait complete. 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 Processor -- start 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 -- start start 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 0x7f12cc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc108920 0x7f12cc1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10e510 con 0x7f12cc103c20 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10e390 con 0x7f12cc1029e0 2024-04-25T13:52:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d39a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10e690 con 0x7f12cc108920 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d0f43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d0f43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37888/0 (socket says 172.21.15.94:37888) 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d1744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 0x7f12cc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d0f43640 1 -- 172.21.15.94:0/1198561126 learned_addr learned my addr 172.21.15.94:0/1198561126 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d1f45640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc108920 0x7f12cc1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.759+0000 7f12d0f43640 1 -- 172.21.15.94:0/1198561126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc108920 msgr2=0x7f12cc1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d0f43640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc108920 0x7f12cc1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d0f43640 1 -- 172.21.15.94:0/1198561126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 msgr2=0x7f12cc199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d0f43640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 0x7f12cc199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d0f43640 1 -- 172.21.15.94:0/1198561126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1a0f50 con 0x7f12cc103c20 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d1744640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 0x7f12cc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d0f43640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12c27fc640 1 -- 172.21.15.94:0/1198561126 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c800cd80 con 0x7f12cc103c20 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12c27fc640 1 -- 172.21.15.94:0/1198561126 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c80108e0 con 0x7f12cc103c20 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1a1240 con 0x7f12cc103c20 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12c27fc640 1 -- 172.21.15.94:0/1198561126 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8018980 con 0x7f12cc103c20 2024-04-25T13:52:44.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.760+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1a1790 con 0x7f12cc103c20 2024-04-25T13:52:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.762+0000 7f12c27fc640 1 -- 172.21.15.94:0/1198561126 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f12c8018b20 con 0x7f12cc103c20 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 Processor -- start 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 -- start start 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12c27fc640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 0x7f12b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86d9d4d0 0x559f8716afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f8716bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559f870ccd80 con 0x559f86d9d4d0 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f87086a20 con 0x559f86dc6390 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12da855740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f870c8d40 con 0x559f86da56b0 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12d0f43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86d9d4d0 0x559f8716afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12d1744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12c27fc640 1 -- 172.21.15.94:0/1198561126 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f12c80defd0 con 0x7f12cc103c20 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12d1744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37196/0 (socket says 172.21.15.94:37196) 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12d1744640 1 -- 172.21.15.94:0/2158608663 learned_addr learned my addr 172.21.15.94:0/2158608663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.763+0000 7f12d1f45640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 0x7f12b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 -- 172.21.15.94:0/2158608663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 msgr2=0x559f8716bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f8716bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 -- 172.21.15.94:0/2158608663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86d9d4d0 msgr2=0x559f8716afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86d9d4d0 0x559f8716afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 -- 172.21.15.94:0/2158608663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f87091c30 con 0x559f86dc6390 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1f45640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 0x7f12b80b2000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f12c40094e0 tx=0x7f12c4009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12d1744640 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0114f0 tx=0x7f12bc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afee7f4be63d65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12b77fe640 1 -- 172.21.15.94:0/2158608663 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc01e070 con 0x559f86dc6390 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12b77fe640 1 -- 172.21.15.94:0/2158608663 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc0035f0 con 0x559f86dc6390 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.764+0000 7f12b77fe640 1 -- 172.21.15.94:0/2158608663 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00f880 con 0x559f86dc6390 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- 172.21.15.94:0/2158608663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 msgr2=0x559f8716aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0114f0 tx=0x7f12bc0119b0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- 172.21.15.94:0/2158608663 shutdown_connections 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f8716bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86d9d4d0 0x559f8716afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 --2- 172.21.15.94:0/2158608663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86dc6390 0x559f8716aa20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- 172.21.15.94:0/2158608663 >> 172.21.15.94:0/2158608663 conn(0x559f86dba4e0 msgr2=0x559f87164020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- 172.21.15.94:0/2158608663 shutdown_connections 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- 172.21.15.94:0/2158608663 wait complete. 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 -- start start 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.765+0000 7f12da855740 1 Processor -- start 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f87174d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86dc6390 0x559f8717bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559f8716c740 con 0x559f86dc6390 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f87053dc0 con 0x559f86d9d4d0 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f8716c8c0 con 0x559f86da56b0 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12da855740 10 client.0 _ll_register_callbacks cb 0x559f8706f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1f45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86dc6390 0x559f8717bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37210/0 (socket says 172.21.15.94:37210) 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 -- 172.21.15.94:0/3839473688 learned_addr learned my addr 172.21.15.94:0/3839473688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d0f43640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f87174d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 -- 172.21.15.94:0/3839473688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 msgr2=0x559f87174d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f87174d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 -- 172.21.15.94:0/3839473688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86dc6390 msgr2=0x559f8717bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86dc6390 0x559f8717bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.766+0000 7f12d1744640 1 -- 172.21.15.94:0/3839473688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f86dde3f0 con 0x559f86d9d4d0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12d1744640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0114c0 tx=0x7f12bc0042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e871e6ba7a01e1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:52:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc026020 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc0035f0 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f12bc009be0 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12bc031050 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 10 client.28931 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f86ea87d0 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc0035f0 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f12bc021030 con 0x559f86d9d4d0 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 10 client.28931 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 10 client.28931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 20 client.28931 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86d84f60 0x559f87182200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 4 client.28931 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.767+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f870da990 con 0x559f86d84f60 2024-04-25T13:52:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.769+0000 7f12d0f43640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86d84f60 0x559f87182200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.769+0000 7f12b4ff9640 10 client.28931 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.769+0000 7f12d0f43640 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86d84f60 0x559f87182200 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2db7f16a57e985e server_cookie=40324efbccf6cfa5 in_seq=0 out_seq=0 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12b4ff9640 1 -- 172.21.15.94:0/3839473688 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f870da990 con 0x559f86d84f60 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12b4ff9640 10 client.28931 handle_command_reply: tid=0 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 client.28931 shutdown 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 msgr2=0x559f87174800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0114c0 tx=0x7f12bc0042b0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 shutdown_connections 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86d84f60 0x559f87182200 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86dc6390 0x559f8717bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12b4ff9640 0 client.28931 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86da56b0 0x559f87174d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 --2- 172.21.15.94:0/3839473688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87174800 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 >> 172.21.15.94:0/3839473688 conn(0x559f86dba4e0 msgr2=0x559f87163040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.770+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 shutdown_connections 2024-04-25T13:52:44.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.771+0000 7f12da855740 1 -- 172.21.15.94:0/3839473688 wait complete. 2024-04-25T13:52:44.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.771+0000 7f12da855740 20 client.28931 trim_cache size 0 max 16384 2024-04-25T13:52:44.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 Processor -- start 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 -- start start 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87173640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f871743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a9360 con 0x559f86da56b0 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1aaea0 con 0x559f86d9d4d0 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12da855740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a9a30 con 0x559f86d84f60 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12d1744640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12d1f45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f871743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12d1744640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59836/0 (socket says 172.21.15.94:59836) 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12d1744640 1 -- 172.21.15.94:0/28622981 learned_addr learned my addr 172.21.15.94:0/28622981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.786+0000 7f12d0f43640 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87173640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 -- 172.21.15.94:0/28622981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 msgr2=0x559f87173640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87173640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 -- 172.21.15.94:0/28622981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 msgr2=0x559f871743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f871743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 -- 172.21.15.94:0/28622981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f87175400 con 0x559f86d84f60 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12d1744640 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12bc004c20 tx=0x7f12bc0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e12318ea0cb4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12b6ffd640 1 -- 172.21.15.94:0/28622981 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc026040 con 0x559f86d84f60 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12b6ffd640 1 -- 172.21.15.94:0/28622981 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc003e10 con 0x559f86d84f60 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.787+0000 7f12b6ffd640 1 -- 172.21.15.94:0/28622981 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc01e040 con 0x559f86d84f60 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.788+0000 7f12da855740 1 -- 172.21.15.94:0/28622981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 msgr2=0x559f8716fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.788+0000 7f12da855740 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12bc004c20 tx=0x7f12bc0035f0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 -- 172.21.15.94:0/28622981 shutdown_connections 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f871743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d9d4d0 0x559f87173640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 --2- 172.21.15.94:0/28622981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d84f60 0x559f8716fb90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 -- 172.21.15.94:0/28622981 >> 172.21.15.94:0/28622981 conn(0x559f86dba4e0 msgr2=0x559f87172f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 -- 172.21.15.94:0/28622981 shutdown_connections 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 -- 172.21.15.94:0/28622981 wait complete. 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 -- start start 2024-04-25T13:52:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.789+0000 7f12da855740 1 Processor -- start 2024-04-25T13:52:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d84f60 0x559f87175970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f8717adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1aa7d0 con 0x559f86da56b0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1aa100 con 0x559f86d84f60 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a7340 con 0x559f86d9d4d0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 10 client.0 _ll_register_callbacks cb 0x7f12bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d1f45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f8717adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59850/0 (socket says 172.21.15.94:59850) 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 -- 172.21.15.94:0/2201346821 learned_addr learned my addr 172.21.15.94:0/2201346821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d1744640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d84f60 0x559f87175970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 -- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d84f60 msgr2=0x559f87175970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d84f60 0x559f87175970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 -- 172.21.15.94:0/2201346821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 msgr2=0x559f8717adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f8717adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 -- 172.21.15.94:0/2201346821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8717b3a0 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12d0f43640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12c80a4730 tx=0x7f12c8018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44000a5dad7e811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80c1060 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f12c808ffb0 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc003e10 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12c80c8050 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.791+0000 7f12da855740 10 client.28741 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f871e7480 con 0x559f86d9d4d0 2024-04-25T13:52:44.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc003e10 con 0x559f86d9d4d0 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f12c80c81f0 con 0x559f86d9d4d0 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 10 client.28741 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 10 client.28741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 20 client.28741 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86dc6390 0x559f871e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 4 client.28741 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f870d4800 con 0x559f86dc6390 2024-04-25T13:52:44.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.792+0000 7f12d1744640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86dc6390 0x559f871e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.793+0000 7f12b4ff9640 10 client.28741 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.793+0000 7f12d1744640 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86dc6390 0x559f871e9e20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da2f4a0744a3e8a server_cookie=26bc7c7c26301ae0 in_seq=0 out_seq=0 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2201346821 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f870d4800 con 0x559f86dc6390 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12b4ff9640 10 client.28741 handle_command_reply: tid=0 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 client.28741 shutdown 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 msgr2=0x559f87175eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12c80a4730 tx=0x7f12c8018420 comp rx=0 tx=0).stop 2024-04-25T13:52:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 shutdown_connections 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559f86dc6390 0x559f871e9e20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12b4ff9640 0 client.28741 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559f86da56b0 0x559f8717adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f86d9d4d0 0x559f87175eb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 --2- 172.21.15.94:0/2201346821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f86d84f60 0x559f87175970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 >> 172.21.15.94:0/2201346821 conn(0x559f86dba4e0 msgr2=0x559f87174380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.794+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 shutdown_connections 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.795+0000 7f12da855740 1 -- 172.21.15.94:0/2201346821 wait complete. 2024-04-25T13:52:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.795+0000 7f12da855740 20 client.28741 trim_cache size 0 max 16384 2024-04-25T13:52:44.799 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:52:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 msgr2=0x7f12b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 0x7f12b80b2000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f12c40094e0 tx=0x7f12c4009800 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 msgr2=0x7f12cc199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 shutdown_connections 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12b80afaf0 0x7f12b80b2000 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12cc108920 0x7f12cc1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12cc103c20 0x7f12cc199950 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 --2- 172.21.15.94:0/1198561126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12cc1029e0 0x7f12cc199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 >> 172.21.15.94:0/1198561126 conn(0x7f12cc0fdbd0 msgr2=0x7f12cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 shutdown_connections 2024-04-25T13:52:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:44.796+0000 7f12d39a8640 1 -- 172.21.15.94:0/1198561126 wait complete. 2024-04-25T13:52:44.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1348900 12.62 MB/sec execute 447 sec latency 28.195 ms 2024-04-25T13:52:45.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:52:45.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:45.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:52:45.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1350446 12.60 MB/sec execute 448 sec latency 15.789 ms 2024-04-25T13:52:45.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- 172.21.15.94:0/1172391792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 msgr2=0x7f1c68104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 --2- 172.21.15.94:0/1172391792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c68104030 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009900 tx=0x7f1c5002a410 comp rx=0 tx=0).stop 2024-04-25T13:52:45.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- 172.21.15.94:0/1172391792 shutdown_connections 2024-04-25T13:52:45.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 --2- 172.21.15.94:0/1172391792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c68108920 0x7f1c6810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 --2- 172.21.15.94:0/1172391792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c68104030 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 --2- 172.21.15.94:0/1172391792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c681029e0 0x7f1c681036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- 172.21.15.94:0/1172391792 >> 172.21.15.94:0/1172391792 conn(0x7f1c680fdbd0 msgr2=0x7f1c68100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- 172.21.15.94:0/1172391792 shutdown_connections 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- 172.21.15.94:0/1172391792 wait complete. 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 Processor -- start 2024-04-25T13:52:45.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.933+0000 7f1c6e425640 1 -- start start 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c681029e0 0x7f1c6806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c68108920 0x7f1c681a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810e4e0 con 0x7f1c68103c20 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810e360 con 0x7f1c681029e0 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6e425640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810e660 con 0x7f1c68108920 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37964/0 (socket says 172.21.15.94:37964) 2024-04-25T13:52:45.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 -- 172.21.15.94:0/428080356 learned_addr learned my addr 172.21.15.94:0/428080356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c67fff640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c681029e0 0x7f1c6806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c68108920 0x7f1c681a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 -- 172.21.15.94:0/428080356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c68108920 msgr2=0x7f1c681a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c68108920 0x7f1c681a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 -- 172.21.15.94:0/428080356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c681029e0 msgr2=0x7f1c6806a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c681029e0 0x7f1c6806a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 -- 172.21.15.94:0/428080356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c681a2050 con 0x7f1c68103c20 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.934+0000 7f1c677fe640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f1c50006010 tx=0x7f1c50031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.935+0000 7f1c657fa640 1 -- 172.21.15.94:0/428080356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5003f030 con 0x7f1c68103c20 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.935+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c681a22e0 con 0x7f1c68103c20 2024-04-25T13:52:45.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.935+0000 7f1c657fa640 1 -- 172.21.15.94:0/428080356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c50003aa0 con 0x7f1c68103c20 2024-04-25T13:52:45.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.935+0000 7f1c657fa640 1 -- 172.21.15.94:0/428080356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50035070 con 0x7f1c68103c20 2024-04-25T13:52:45.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.935+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c681a27d0 con 0x7f1c68103c20 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.937+0000 7f1c753aa740 1 Processor -- start 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.937+0000 7f1c753aa740 1 -- start start 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.937+0000 7f1c657fa640 1 -- 172.21.15.94:0/428080356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c50042030 con 0x7f1c68103c20 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.937+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 0x55808a8394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.937+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a839a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c753aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55808a79eb00 con 0x55808a477730 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c753aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55808a73a920 con 0x55808a498410 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c753aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55808a79aac0 con 0x55808a46f550 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37976/0 (socket says 172.21.15.94:37976) 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 -- 172.21.15.94:0/2598372186 learned_addr learned my addr 172.21.15.94:0/2598372186 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 0x55808a8394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c67fff640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a839a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 -- 172.21.15.94:0/2598372186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 msgr2=0x55808a839a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a839a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 -- 172.21.15.94:0/2598372186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 msgr2=0x55808a8394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 0x55808a8394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 -- 172.21.15.94:0/2598372186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55808a5787e0 con 0x55808a477730 2024-04-25T13:52:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 0x55808a8394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c677fe640 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1c501074a0 tx=0x7f1c50107590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8480f2384b2384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c657fa640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c5e7fc640 1 -- 172.21.15.94:0/2598372186 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c3d40 con 0x55808a477730 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.938+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c5e7fc640 1 -- 172.21.15.94:0/2598372186 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5010c030 con 0x55808a477730 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c5e7fc640 1 -- 172.21.15.94:0/2598372186 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c6e60 con 0x55808a477730 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c657fa640 1 -- 172.21.15.94:0/428080356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1c500ff2b0 con 0x7f1c68103c20 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 -- 172.21.15.94:0/2598372186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 msgr2=0x55808a840920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55808a83a670 tx=0x7f1c60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1c501074a0 tx=0x7f1c50107590 comp rx=0 tx=0).stop 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 -- 172.21.15.94:0/2598372186 shutdown_connections 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a840920 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a839a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.939+0000 7f1c753aa740 1 --2- 172.21.15.94:0/2598372186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a498410 0x55808a8394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- 172.21.15.94:0/2598372186 >> 172.21.15.94:0/2598372186 conn(0x55808a48c560 msgr2=0x55808a834e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- 172.21.15.94:0/2598372186 shutdown_connections 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- 172.21.15.94:0/2598372186 wait complete. 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- start start 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 Processor -- start 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a842f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 0x55808a843470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55808a841130 con 0x55808a498410 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55808a7385d0 con 0x55808a46f550 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.940+0000 7f1c753aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55808a8412b0 con 0x55808a477730 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c753aa740 10 client.0 _ll_register_callbacks cb 0x55808a726080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37990/0 (socket says 172.21.15.94:37990) 2024-04-25T13:52:45.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/968650357 learned_addr learned my addr 172.21.15.94:0/968650357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a842f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c677fe640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 0x55808a843470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/968650357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 msgr2=0x55808a843470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 0x55808a843470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 msgr2=0x55808a842f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a842f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/968650357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55808a5d8f70 con 0x55808a498410 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c677fe640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 0x55808a843470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009d40 tx=0x7f1c60004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b25dc80b8fdf80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001fc80 con 0x55808a498410 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1c60024450 con 0x55808a498410 2024-04-25T13:52:45.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.941+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5010c030 con 0x55808a498410 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c6001fe20 con 0x55808a498410 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 10 client.19500 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55808a470170 con 0x55808a498410 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5010c030 con 0x55808a498410 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1c60036810 con 0x55808a498410 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 10 client.19500 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 10 client.19500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 20 client.19500 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a4571f0 0x55808a8509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 4 client.19500 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55808a7ac710 con 0x55808a4571f0 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.942+0000 7f1c67fff640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a4571f0 0x55808a8509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.943+0000 7f1c477fe640 10 client.19500 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:45.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.943+0000 7f1c67fff640 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a4571f0 0x55808a8509b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca62d30912cf875 server_cookie=7409751861a921da in_seq=0 out_seq=0 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c477fe640 1 -- 172.21.15.94:0/968650357 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55808a7ac710 con 0x55808a4571f0 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c477fe640 10 client.19500 handle_command_reply: tid=0 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 client.19500 shutdown 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 msgr2=0x55808a84a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009d40 tx=0x7f1c60004ab0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 shutdown_connections 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a4571f0 0x55808a8509b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.944+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a84a3d0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a477730 0x55808a843470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 1 --2- 172.21.15.94:0/968650357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a842f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c477fe640 0 client.19500 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 >> 172.21.15.94:0/968650357 conn(0x55808a48c560 msgr2=0x55808a833ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 shutdown_connections 2024-04-25T13:52:45.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 1 -- 172.21.15.94:0/968650357 wait complete. 2024-04-25T13:52:45.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.945+0000 7f1c753aa740 20 client.19500 trim_cache size 0 max 16384 2024-04-25T13:52:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.962+0000 7f1c753aa740 1 Processor -- start 2024-04-25T13:52:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.962+0000 7f1c753aa740 1 -- start start 2024-04-25T13:52:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a855160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a498410 0x55808a855be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681a92e0 con 0x55808a477730 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681aae20 con 0x55808a46f550 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c753aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681a99b0 con 0x55808a498410 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38004/0 (socket says 172.21.15.94:38004) 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 -- 172.21.15.94:0/729529490 learned_addr learned my addr 172.21.15.94:0/729529490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c67fff640 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a855160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a498410 0x55808a855be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 -- 172.21.15.94:0/729529490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a498410 msgr2=0x55808a855be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a498410 0x55808a855be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 -- 172.21.15.94:0/729529490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 msgr2=0x55808a855160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a855160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 -- 172.21.15.94:0/729529490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55808a83a190 con 0x55808a477730 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c677fe640 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1c501076c0 tx=0x7f1c50005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944863a667ed6546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c5dffb640 1 -- 172.21.15.94:0/729529490 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50049020 con 0x55808a477730 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c5dffb640 1 -- 172.21.15.94:0/729529490 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5010c070 con 0x55808a477730 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.963+0000 7f1c5dffb640 1 -- 172.21.15.94:0/729529490 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c7040 con 0x55808a477730 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 -- 172.21.15.94:0/729529490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 msgr2=0x55808a8556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f1c501076c0 tx=0x7f1c50005e50 comp rx=0 tx=0).stop 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 -- 172.21.15.94:0/729529490 shutdown_connections 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a498410 0x55808a855be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a477730 0x55808a8556a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 --2- 172.21.15.94:0/729529490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a46f550 0x55808a855160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 -- 172.21.15.94:0/729529490 >> 172.21.15.94:0/729529490 conn(0x55808a48c560 msgr2=0x55808a83d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 -- 172.21.15.94:0/729529490 shutdown_connections 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.964+0000 7f1c753aa740 1 -- 172.21.15.94:0/729529490 wait complete. 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 -- start start 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 Processor -- start 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a8472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a477730 0x55808a847830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681aa750 con 0x55808a498410 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681aa080 con 0x55808a477730 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681a72c0 con 0x55808a46f550 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c753aa740 10 client.0 _ll_register_callbacks cb 0x7f1c60010a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c6c9c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a477730 0x55808a847830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c6c9c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38018/0 (socket says 172.21.15.94:38018) 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/3835781723 learned_addr learned my addr 172.21.15.94:0/3835781723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.965+0000 7f1c67fff640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a8472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/3835781723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 msgr2=0x55808a8472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a8472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a477730 msgr2=0x55808a847830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a477730 0x55808a847830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 -- 172.21.15.94:0/3835781723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55808a848d20 con 0x55808a498410 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c67fff640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a8472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c6c9c2640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1c60036e70 tx=0x7f1c60000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a8062e5dbe0a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001fc80 con 0x55808a498410 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5010c070 con 0x55808a498410 2024-04-25T13:52:45.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1c60004b20 con 0x55808a498410 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c60036a70 con 0x55808a498410 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c753aa740 10 client.19506 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55808a8b8e50 con 0x55808a498410 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.966+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5010c070 con 0x55808a498410 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6002cc50 con 0x55808a498410 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 10 client.19506 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 10 client.19506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 20 client.19506 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a42c630 0x55808a8bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 4 client.19506 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55808a7a6580 con 0x55808a42c630 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c67fff640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a42c630 0x55808a8bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c477fe640 10 client.19506 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:45.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.967+0000 7f1c67fff640 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a42c630 0x55808a8bb870 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f966b699a73b34a server_cookie=24ad01eda1ffc012 in_seq=0 out_seq=0 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c477fe640 1 -- 172.21.15.94:0/3835781723 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55808a7a6580 con 0x55808a42c630 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c477fe640 10 client.19506 handle_command_reply: tid=0 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 client.19506 shutdown 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 msgr2=0x55808a848760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1c60036e70 tx=0x7f1c60000f40 comp rx=0 tx=0).stop 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 shutdown_connections 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55808a42c630 0x55808a8bb870 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55808a498410 0x55808a848760 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55808a477730 0x55808a847830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 --2- 172.21.15.94:0/3835781723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55808a46f550 0x55808a8472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 >> 172.21.15.94:0/3835781723 conn(0x55808a48c560 msgr2=0x55808a837c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c477fe640 0 client.19506 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 shutdown_connections 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.969+0000 7f1c753aa740 1 -- 172.21.15.94:0/3835781723 wait complete. 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c753aa740 20 client.19506 trim_cache size 0 max 16384 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b2082834-ede3-4b85-ad20-8ce1b087f7a6", 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:45.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 msgr2=0x7f1c540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55808a83a670 tx=0x7f1c60009500 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 msgr2=0x7f1c6819f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f1c50006010 tx=0x7f1c50031a50 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 shutdown_connections 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c68108920 0x7f1c681a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c68103c20 0x7f1c6819f330 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 --2- 172.21.15.94:0/428080356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c681029e0 0x7f1c6806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 >> 172.21.15.94:0/428080356 conn(0x7f1c680fdbd0 msgr2=0x7f1c68104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 shutdown_connections 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:45.970+0000 7f1c6e425640 1 -- 172.21.15.94:0/428080356 wait complete. 2024-04-25T13:52:45.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:52:45.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:52:46.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.109+0000 7f20efed8640 1 -- 172.21.15.94:0/901793621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 msgr2=0x7f20e806a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.109+0000 7f20efed8640 1 --2- 172.21.15.94:0/901793621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e806a2a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e002a8f0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.115+0000 7f20efed8640 1 -- 172.21.15.94:0/901793621 shutdown_connections 2024-04-25T13:52:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.115+0000 7f20efed8640 1 --2- 172.21.15.94:0/901793621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e806a2a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.115+0000 7f20efed8640 1 --2- 172.21.15.94:0/901793621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e8103c20 0x7f20e8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.115+0000 7f20efed8640 1 --2- 172.21.15.94:0/901793621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e81029e0 0x7f20e81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.115+0000 7f20efed8640 1 -- 172.21.15.94:0/901793621 >> 172.21.15.94:0/901793621 conn(0x7f20e80fdbd0 msgr2=0x7f20e8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.120+0000 7f20efed8640 1 -- 172.21.15.94:0/901793621 shutdown_connections 2024-04-25T13:52:46.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.122+0000 7f20efed8640 1 -- 172.21.15.94:0/901793621 wait complete. 2024-04-25T13:52:46.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.122+0000 7f20efed8640 1 Processor -- start 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 -- start start 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e81029e0 0x7f20e80722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8103c20 0x7f20e8073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111820 con 0x7f20e8103c20 2024-04-25T13:52:46.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81116a0 con 0x7f20e8069e90 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.123+0000 7f20efed8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81119a0 con 0x7f20e81029e0 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.124+0000 7f20edc74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.124+0000 7f20edc74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37284/0 (socket says 172.21.15.94:37284) 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.124+0000 7f20edc74640 1 -- 172.21.15.94:0/3851566389 learned_addr learned my addr 172.21.15.94:0/3851566389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.124+0000 7f20ee475640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8103c20 0x7f20e8073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20ed473640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e81029e0 0x7f20e80722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 -- 172.21.15.94:0/3851566389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e81029e0 msgr2=0x7f20e80722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e81029e0 0x7f20e80722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 -- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8103c20 msgr2=0x7f20e8073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8103c20 0x7f20e8073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 -- 172.21.15.94:0/3851566389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e80738b0 con 0x7f20e8069e90 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20edc74640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d740 tx=0x7f20d800dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.125+0000 7f20e6ffd640 1 -- 172.21.15.94:0/3851566389 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8017070 con 0x7f20e8069e90 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.126+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e8073ba0 con 0x7f20e8069e90 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.126+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e80740f0 con 0x7f20e8069e90 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.126+0000 7f20e6ffd640 1 -- 172.21.15.94:0/3851566389 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d800de50 con 0x7f20e8069e90 2024-04-25T13:52:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.126+0000 7f20e6ffd640 1 -- 172.21.15.94:0/3851566389 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8003c20 con 0x7f20e8069e90 2024-04-25T13:52:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.131+0000 7f20f6d84740 1 Processor -- start 2024-04-25T13:52:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.132+0000 7f20f6d84740 1 -- start start 2024-04-25T13:52:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.132+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f345e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.132+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f345e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20f6d84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6f33bad80 con 0x55a6f30b4390 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20f6d84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6f33748b0 con 0x55a6f308b4d0 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20f6d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6f33b6d40 con 0x55a6f30936b0 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ee475640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f345e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59958/0 (socket says 172.21.15.94:59958) 2024-04-25T13:52:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 -- 172.21.15.94:0/3915402688 learned_addr learned my addr 172.21.15.94:0/3915402688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:46.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 -- 172.21.15.94:0/3915402688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 msgr2=0x55a6f345e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f345e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 -- 172.21.15.94:0/3915402688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 msgr2=0x55a6f345e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f345e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.133+0000 7f20ed473640 1 -- 172.21.15.94:0/3915402688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f3345a40 con 0x55a6f30936b0 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.134+0000 7f20ed473640 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009b90 tx=0x7f20dc00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99db60783bc9f5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.134+0000 7f20cffff640 1 -- 172.21.15.94:0/3915402688 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc016070 con 0x55a6f30936b0 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.134+0000 7f20cffff640 1 -- 172.21.15.94:0/3915402688 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20dc00ee70 con 0x55a6f30936b0 2024-04-25T13:52:46.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.134+0000 7f20cffff640 1 -- 172.21.15.94:0/3915402688 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0106c0 con 0x55a6f30936b0 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.134+0000 7f20e6ffd640 1 -- 172.21.15.94:0/3851566389 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f20d8010460 con 0x7f20e8069e90 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 -- 172.21.15.94:0/3915402688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 msgr2=0x55a6f345f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009b90 tx=0x7f20dc00e6c0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20e6ffd640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 0x7f20d00b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 -- 172.21.15.94:0/3915402688 shutdown_connections 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f345f480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f345e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 --2- 172.21.15.94:0/3915402688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f345e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 -- 172.21.15.94:0/3915402688 >> 172.21.15.94:0/3915402688 conn(0x55a6f30a84e0 msgr2=0x55a6f3459a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20ee475640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 0x7f20d00b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20e6ffd640 1 -- 172.21.15.94:0/3851566389 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f20d80de720 con 0x7f20e8069e90 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.135+0000 7f20f6d84740 1 -- 172.21.15.94:0/3915402688 shutdown_connections 2024-04-25T13:52:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 -- 172.21.15.94:0/3915402688 wait complete. 2024-04-25T13:52:46.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20ee475640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 0x7f20d00b2190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55a6f345f2c0 tx=0x7f20e0031940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 -- start start 2024-04-25T13:52:46.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 Processor -- start 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3468220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30b4390 0x55a6f346f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6f345ff90 con 0x55a6f30936b0 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6f3341d10 con 0x55a6f308b4d0 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.136+0000 7f20f6d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6f3460110 con 0x55a6f30b4390 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20f6d84740 10 client.0 _ll_register_callbacks cb 0x55a6f336e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:46.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38048/0 (socket says 172.21.15.94:38048) 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 -- 172.21.15.94:0/2840526435 learned_addr learned my addr 172.21.15.94:0/2840526435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ee475640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30b4390 0x55a6f346f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20edc74640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3468220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 -- 172.21.15.94:0/2840526435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30b4390 msgr2=0x55a6f346f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30b4390 0x55a6f346f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 -- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 msgr2=0x55a6f3468220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3468220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 -- 172.21.15.94:0/2840526435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f31967d0 con 0x55a6f30936b0 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20edc74640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3468220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20ed473640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f825333643f981ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc016050 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20dc00ee70 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f20dc0048c0 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20dc004ac0 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.137+0000 7f20f6d84740 10 client.19518 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6f30cc3f0 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20dc00ee70 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f20dc00f040 con 0x55a6f30936b0 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 10 client.19518 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 10 client.19518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 20 client.19518 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:46.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f3072f60 0x55a6f3475860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 4 client.19518 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f33c8990 con 0x55a6f3072f60 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20edc74640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f3072f60 0x55a6f3475860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.138+0000 7f20cd7fa640 10 client.19518 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.139+0000 7f20edc74640 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f3072f60 0x55a6f3475860 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d187b1e50148167 server_cookie=126c37cab3d7a52 in_seq=0 out_seq=0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.141+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2840526435 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6f33c8990 con 0x55a6f3072f60 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.141+0000 7f20cd7fa640 10 client.19518 handle_command_reply: tid=0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.141+0000 7f20f6d84740 1 client.19518 shutdown 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.141+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 msgr2=0x55a6f3468760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.141+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc0052e0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 shutdown_connections 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f3072f60 0x55a6f3475860 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30b4390 0x55a6f346f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30936b0 0x55a6f3468760 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20cd7fa640 0 client.19518 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2840526435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3468220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 >> 172.21.15.94:0/2840526435 conn(0x55a6f30a84e0 msgr2=0x55a6f34503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 shutdown_connections 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 1 -- 172.21.15.94:0/2840526435 wait complete. 2024-04-25T13:52:46.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.142+0000 7f20f6d84740 20 client.19518 trim_cache size 0 max 16384 2024-04-25T13:52:46.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 Processor -- start 2024-04-25T13:52:46.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 -- start start 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3459440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f346f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20d400e730 con 0x55a6f30b4390 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4010270 con 0x55a6f308b4d0 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20f6d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20d400ee00 con 0x55a6f30936b0 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38056/0 (socket says 172.21.15.94:38056) 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 -- 172.21.15.94:0/116866588 learned_addr learned my addr 172.21.15.94:0/116866588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20edc74640 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3459440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ed473640 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f346f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 -- 172.21.15.94:0/116866588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 msgr2=0x55a6f346f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f346f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 -- 172.21.15.94:0/116866588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 msgr2=0x55a6f3459440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3459440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 -- 172.21.15.94:0/116866588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f308c0f0 con 0x55a6f30b4390 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.161+0000 7f20ee475640 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119fce10dea497a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20cf7fe640 1 -- 172.21.15.94:0/116866588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0031e20 con 0x55a6f30b4390 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20cf7fe640 1 -- 172.21.15.94:0/116866588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0056070 con 0x55a6f30b4390 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20cf7fe640 1 -- 172.21.15.94:0/116866588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0051df0 con 0x55a6f30b4390 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 -- 172.21.15.94:0/116866588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 msgr2=0x55a6f3459980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e003f000 comp rx=0 tx=0).stop 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 -- 172.21.15.94:0/116866588 shutdown_connections 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f3459980 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f30936b0 0x55a6f346f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 --2- 172.21.15.94:0/116866588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f308b4d0 0x55a6f3459440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.162+0000 7f20f6d84740 1 -- 172.21.15.94:0/116866588 >> 172.21.15.94:0/116866588 conn(0x55a6f30a84e0 msgr2=0x55a6f3462f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- 172.21.15.94:0/116866588 shutdown_connections 2024-04-25T13:52:46.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- 172.21.15.94:0/116866588 wait complete. 2024-04-25T13:52:46.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- start start 2024-04-25T13:52:46.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 Processor -- start 2024-04-25T13:52:46.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f308b4d0 0x55a6f346c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f30936b0 0x55a6f346c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20d400fba0 con 0x55a6f30b4390 2024-04-25T13:52:46.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20d400f4d0 con 0x55a6f30936b0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20d400c710 con 0x55a6f308b4d0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20f6d84740 10 client.0 _ll_register_callbacks cb 0x7f20e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.163+0000 7f20ee475640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20edc74640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f308b4d0 0x55a6f346c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38070/0 (socket says 172.21.15.94:38070) 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 -- 172.21.15.94:0/2842239675 learned_addr learned my addr 172.21.15.94:0/2842239675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ed473640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f30936b0 0x55a6f346c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 -- 172.21.15.94:0/2842239675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f308b4d0 msgr2=0x55a6f346c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f308b4d0 0x55a6f346c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 -- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f30936b0 msgr2=0x55a6f346c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f30936b0 0x55a6f346c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 -- 172.21.15.94:0/2842239675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f337b4b0 con 0x55a6f30b4390 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20ee475640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f20e00094e0 tx=0x7f20e0051f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98cb5a743b7d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0051670 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f20e0030ac0 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0056070 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f20e0030cc0 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20f6d84740 10 client.19524 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6f34dcf50 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.164+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0056070 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f20e005a410 con 0x55a6f30b4390 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 10 client.19524 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 10 client.19524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 20 client.19524 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f30483a0 0x55a6f34df910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 4 client.19524 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:52:46.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6f33c2800 con 0x55a6f30483a0 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20edc74640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f30483a0 0x55a6f34df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20cd7fa640 10 client.19524 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.165+0000 7f20edc74640 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f30483a0 0x55a6f34df910 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85331551a4e2ff4 server_cookie=646d6a89052695dc in_seq=0 out_seq=0 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.167+0000 7f20cd7fa640 1 -- 172.21.15.94:0/2842239675 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a6f33c2800 con 0x55a6f30483a0 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.167+0000 7f20cd7fa640 10 client.19524 handle_command_reply: tid=0 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.167+0000 7f20f6d84740 1 client.19524 shutdown 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.167+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 msgr2=0x55a6f34dbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.167+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f20e00094e0 tx=0x7f20e0051f70 comp rx=0 tx=0).stop 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 shutdown_connections 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6f30483a0 0x55a6f34df910 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6f30b4390 0x55a6f34dbba0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20cd7fa640 0 client.19524 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6f30936b0 0x55a6f346c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 --2- 172.21.15.94:0/2842239675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6f308b4d0 0x55a6f346c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 >> 172.21.15.94:0/2842239675 conn(0x55a6f30a84e0 msgr2=0x55a6f3459310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 shutdown_connections 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 1 -- 172.21.15.94:0/2842239675 wait complete. 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20f6d84740 20 client.19524 trim_cache size 0 max 16384 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:52:46.172 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: "b2082834-ede3-4b85-ad20-8ce1b087f7a6": { 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b2082834-ede3-4b85-ad20-8ce1b087f7a6", 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:52:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 msgr2=0x7f20d00b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 0x7f20d00b2190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55a6f345f2c0 tx=0x7f20e0031940 comp rx=0 tx=0).stop 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 msgr2=0x7f20e8071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d740 tx=0x7f20d800dc00 comp rx=0 tx=0).stop 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 shutdown_connections 2024-04-25T13:52:46.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20d00afc80 0x7f20d00b2190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20e8103c20 0x7f20e8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e81029e0 0x7f20e80722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 --2- 172.21.15.94:0/3851566389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20e8069e90 0x7f20e8071da0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 >> 172.21.15.94:0/3851566389 conn(0x7f20e80fdbd0 msgr2=0x7f20e8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 shutdown_connections 2024-04-25T13:52:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:52:46.168+0000 7f20efed8640 1 -- 172.21.15.94:0/3851566389 wait complete. 2024-04-25T13:52:46.178 INFO:tasks.cephfs.filesystem:scrub status for tag:b2082834-ede3-4b85-ad20-8ce1b087f7a6 - {'path': '/', 'tag': 'b2082834-ede3-4b85-ad20-8ce1b087f7a6', 'options': 'recursive,force'} 2024-04-25T13:52:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:46 smithi094 ceph-mon[29195]: pgmap v678: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-04-25T13:52:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:46 smithi138 ceph-mon[32977]: pgmap v678: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-04-25T13:52:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:46 smithi137 ceph-mon[32568]: pgmap v678: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-04-25T13:52:46.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1353116 12.61 MB/sec execute 449 sec latency 79.273 ms 2024-04-25T13:52:47.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:47 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:52:47.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:52:47.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:47 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:52:47.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:52:47.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:47 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:52:47.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:52:47.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:52:47.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1355202 12.60 MB/sec execute 450 sec latency 18.257 ms 2024-04-25T13:52:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:48 smithi138 ceph-mon[32977]: pgmap v679: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-25T13:52:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:48 smithi137 ceph-mon[32568]: pgmap v679: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-25T13:52:48.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:48 smithi094 ceph-mon[29195]: pgmap v679: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-25T13:52:48.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1357220 12.60 MB/sec execute 451 sec latency 18.346 ms 2024-04-25T13:52:49.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1360207 12.61 MB/sec execute 452 sec latency 10.939 ms 2024-04-25T13:52:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:50 smithi138 ceph-mon[32977]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-04-25T13:52:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:50 smithi137 ceph-mon[32568]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-04-25T13:52:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:50 smithi094 ceph-mon[29195]: pgmap v680: 97 pgs: 97 active+clean; 39 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 319 op/s 2024-04-25T13:52:50.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1362461 12.60 MB/sec execute 453 sec latency 15.156 ms 2024-04-25T13:52:51.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:52:51.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:52:51.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:52:51.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1364112 12.60 MB/sec execute 454 sec latency 60.950 ms 2024-04-25T13:52:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:52 smithi138 ceph-mon[32977]: pgmap v681: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 376 op/s 2024-04-25T13:52:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:52 smithi137 ceph-mon[32568]: pgmap v681: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 376 op/s 2024-04-25T13:52:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:52 smithi094 ceph-mon[29195]: pgmap v681: 97 pgs: 97 active+clean; 43 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 376 op/s 2024-04-25T13:52:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 -- 172.21.15.94:0/2738979719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4102a00 msgr2=0x7fdad4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 --2- 172.21.15.94:0/2738979719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4102a00 0x7fdad4103700 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009900 tx=0x7fdac402a410 comp rx=0 tx=0).stop 2024-04-25T13:52:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 -- 172.21.15.94:0/2738979719 shutdown_connections 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 --2- 172.21.15.94:0/2738979719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4108940 0x7fdad410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 --2- 172.21.15.94:0/2738979719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4103c40 0x7fdad4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 --2- 172.21.15.94:0/2738979719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4102a00 0x7fdad4103700 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 -- 172.21.15.94:0/2738979719 >> 172.21.15.94:0/2738979719 conn(0x7fdad40fdbd0 msgr2=0x7fdad41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 -- 172.21.15.94:0/2738979719 shutdown_connections 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.792+0000 7fdad93b2640 1 -- 172.21.15.94:0/2738979719 wait complete. 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 Processor -- start 2024-04-25T13:52:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 -- start start 2024-04-25T13:52:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4102a00 0x7fdad419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4103c40 0x7fdad419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410f5f0 con 0x7fdad4108940 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410f470 con 0x7fdad4102a00 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad93b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdad410f770 con 0x7fdad4103c40 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38086/0 (socket says 172.21.15.94:38086) 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 -- 172.21.15.94:0/3707795324 learned_addr learned my addr 172.21.15.94:0/3707795324 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad27fc640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4103c40 0x7fdad419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4102a00 0x7fdad419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 -- 172.21.15.94:0/3707795324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4103c40 msgr2=0x7fdad419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4103c40 0x7fdad419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 -- 172.21.15.94:0/3707795324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4102a00 msgr2=0x7fdad419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4102a00 0x7fdad419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.793+0000 7fdad37fe640 1 -- 172.21.15.94:0/3707795324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41a1da0 con 0x7fdad4108940 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdad37fe640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00eb10 tx=0x7fdacc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdabbfff640 1 -- 172.21.15.94:0/3707795324 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc004b20 con 0x7fdad4108940 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdabbfff640 1 -- 172.21.15.94:0/3707795324 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdacc004cc0 con 0x7fdad4108940 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41a2090 con 0x7fdad4108940 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdabbfff640 1 -- 172.21.15.94:0/3707795324 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc010670 con 0x7fdad4108940 2024-04-25T13:52:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.794+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a2590 con 0x7fdad4108940 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.796+0000 7fdabbfff640 1 -- 172.21.15.94:0/3707795324 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdacc005020 con 0x7fdad4108940 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.796+0000 7fdae025e740 1 Processor -- start 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.796+0000 7fdae025e740 1 -- start start 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.796+0000 7fdae025e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdae025e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 0x55fcc455fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdae025e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4566df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdae025e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fcc44c5460 con 0x55fcc41bf1b0 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdae025e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fcc447f590 con 0x55fcc41962f0 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdae025e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fcc44c1420 con 0x55fcc419e4d0 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38094/0 (socket says 172.21.15.94:38094) 2024-04-25T13:52:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 0x55fcc455fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad27fc640 1 -- 172.21.15.94:0/2662884835 learned_addr learned my addr 172.21.15.94:0/2662884835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdabbfff640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 0x7fdabc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad27fc640 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4566df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad37fe640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 0x7fdabc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 -- 172.21.15.94:0/2662884835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 msgr2=0x55fcc4566df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4566df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 -- 172.21.15.94:0/2662884835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 msgr2=0x55fcc455f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 -- 172.21.15.94:0/2662884835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc448c0d0 con 0x55fcc41962f0 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad27fc640 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdabbfff640 1 -- 172.21.15.94:0/3707795324 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdacc0df280 con 0x7fdad4108940 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 0x55fcc455fef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdac402a930 tx=0x7fdac4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22afdbe80499f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdab97fa640 1 -- 172.21.15.94:0/2662884835 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4035030 con 0x55fcc41962f0 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.797+0000 7fdad37fe640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 0x7fdabc0b1ff0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0a3740 tx=0x7fdacc076040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdab97fa640 1 -- 172.21.15.94:0/2662884835 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac4040070 con 0x55fcc41962f0 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdab97fa640 1 -- 172.21.15.94:0/2662884835 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003850 con 0x55fcc41962f0 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 -- 172.21.15.94:0/2662884835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 msgr2=0x55fcc455fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 0x55fcc455fef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdac402a930 tx=0x7fdac4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 -- 172.21.15.94:0/2662884835 shutdown_connections 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4566df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41962f0 0x55fcc455fef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 --2- 172.21.15.94:0/2662884835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41bf1b0 0x55fcc455f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 -- 172.21.15.94:0/2662884835 >> 172.21.15.94:0/2662884835 conn(0x55fcc41b3300 msgr2=0x55fcc455b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 -- 172.21.15.94:0/2662884835 shutdown_connections 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.798+0000 7fdae025e740 1 -- 172.21.15.94:0/2662884835 wait complete. 2024-04-25T13:52:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 -- start start 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 Processor -- start 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc4569b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc4570ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fcc45677e0 con 0x55fcc41962f0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fcc444cb60 con 0x55fcc419e4d0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fcc4567960 con 0x55fcc41bf1b0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdae025e740 10 client.0 _ll_register_callbacks cb 0x55fcc44712b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38098/0 (socket says 172.21.15.94:38098) 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 -- 172.21.15.94:0/445027871 learned_addr learned my addr 172.21.15.94:0/445027871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc4569b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc4570ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 -- 172.21.15.94:0/445027871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 msgr2=0x55fcc4570ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.799+0000 7fdad27fc640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc4570ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdad27fc640 1 -- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 msgr2=0x55fcc4569b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdad27fc640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc4569b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdad27fc640 1 -- 172.21.15.94:0/445027871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc41d7210 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdad27fc640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdac8013f80 tx=0x7fdac8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ad7330ae15bb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8005020 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdac80051c0 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac4040070 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdac80053c0 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 10 client.19536 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fcc42a3620 con 0x55fcc41962f0 2024-04-25T13:52:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac4040070 con 0x55fcc41962f0 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdac8024530 con 0x55fcc41962f0 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 10 client.19536 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 10 client.19536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 20 client.19536 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc417dd80 0x55fcc45772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 4 client.19536 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.800+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc44d3070 con 0x55fcc417dd80 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.801+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc417dd80 0x55fcc45772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.801+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc417dd80 0x55fcc45772f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64de2ca2d913482 server_cookie=e9efb62d4987721a in_seq=0 out_seq=0 2024-04-25T13:52:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.801+0000 7fdab6ff5640 10 client.19536 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdab6ff5640 1 -- 172.21.15.94:0/445027871 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fcc44d3070 con 0x55fcc417dd80 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdab6ff5640 10 client.19536 handle_command_reply: tid=0 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 client.19536 shutdown 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 msgr2=0x55fcc4569640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdac8013f80 tx=0x7fdac8005f90 comp rx=0 tx=0).stop 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 shutdown_connections 2024-04-25T13:52:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc417dd80 0x55fcc45772f0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc4570ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc4569b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdab6ff5640 0 client.19536 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 --2- 172.21.15.94:0/445027871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc4569640 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.802+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 >> 172.21.15.94:0/445027871 conn(0x55fcc41b3300 msgr2=0x55fcc455e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.803+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 shutdown_connections 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.803+0000 7fdae025e740 1 -- 172.21.15.94:0/445027871 wait complete. 2024-04-25T13:52:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.803+0000 7fdae025e740 20 client.19536 trim_cache size 0 max 16384 2024-04-25T13:52:52.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1366563 12.60 MB/sec execute 455 sec latency 34.091 ms 2024-04-25T13:52:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 Processor -- start 2024-04-25T13:52:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 -- start start 2024-04-25T13:52:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4570a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41bf1b0 0x55fcc455e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ab540 con 0x55fcc41962f0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ad080 con 0x55fcc41bf1b0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdae025e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41abc10 con 0x55fcc419e4d0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38108/0 (socket says 172.21.15.94:38108) 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.818+0000 7fdad27fc640 1 -- 172.21.15.94:0/685246470 learned_addr learned my addr 172.21.15.94:0/685246470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4570a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41bf1b0 0x55fcc455e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 -- 172.21.15.94:0/685246470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 msgr2=0x55fcc4570a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4570a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 -- 172.21.15.94:0/685246470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41bf1b0 msgr2=0x55fcc455e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41bf1b0 0x55fcc455e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 -- 172.21.15.94:0/685246470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc4564550 con 0x55fcc41962f0 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdad27fc640 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fdac8013d10 tx=0x7fdac8016fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266276e27b4cefa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdab8ff9640 1 -- 172.21.15.94:0/685246470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac80240b0 con 0x55fcc41962f0 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdab8ff9640 1 -- 172.21.15.94:0/685246470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac8031040 con 0x55fcc41962f0 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.819+0000 7fdab8ff9640 1 -- 172.21.15.94:0/685246470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8022940 con 0x55fcc41962f0 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- 172.21.15.94:0/685246470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 msgr2=0x55fcc455b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fdac8013d10 tx=0x7fdac8016fe0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- 172.21.15.94:0/685246470 shutdown_connections 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc41bf1b0 0x55fcc455e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc419e4d0 0x55fcc4570a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 --2- 172.21.15.94:0/685246470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc455b3d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- 172.21.15.94:0/685246470 >> 172.21.15.94:0/685246470 conn(0x55fcc41b3300 msgr2=0x55fcc455acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- 172.21.15.94:0/685246470 shutdown_connections 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- 172.21.15.94:0/685246470 wait complete. 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 -- start start 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.820+0000 7fdae025e740 1 Processor -- start 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc456e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc456f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ac9b0 con 0x55fcc41962f0 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41ac2e0 con 0x55fcc419e4d0 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdad41a9520 con 0x55fcc41bf1b0 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 10 client.0 _ll_register_callbacks cb 0x7fdac8009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38110/0 (socket says 172.21.15.94:38110) 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 -- 172.21.15.94:0/458975749 learned_addr learned my addr 172.21.15.94:0/458975749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:52:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc456e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc456f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 -- 172.21.15.94:0/458975749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 msgr2=0x55fcc456f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc456f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 -- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 msgr2=0x55fcc456e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc456e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 -- 172.21.15.94:0/458975749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc456fe50 con 0x55fcc41962f0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdad27fc640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006010 tx=0x7fdac8025fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e734f2acefbe98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8025d80 con 0x55fcc41962f0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fdac80240d0 con 0x55fcc41962f0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac8031040 con 0x55fcc41962f0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdac8041050 con 0x55fcc41962f0 2024-04-25T13:52:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 10 client.19542 fetch_fsmap learned FSMap version 31 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.821+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fcc45df7e0 con 0x55fcc41962f0 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac8031040 con 0x55fcc41962f0 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdac8022c90 con 0x55fcc41962f0 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 10 client.19542 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 10 client.19542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 20 client.19542 populate_metadata read hostname 'smithi094' 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc41531c0 0x55fcc45e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 4 client.19542 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcc44ccee0 con 0x55fcc41531c0 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc41531c0 0x55fcc45e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdab6ff5640 10 client.19542 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:52.822+0000 7fdad2ffd640 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc41531c0 0x55fcc45e2130 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48557098a185f51 server_cookie=b978580d31db27d7 in_seq=0 out_seq=0 2024-04-25T13:52:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.008+0000 7fdab6ff5640 1 -- 172.21.15.94:0/458975749 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcc44ccee0 con 0x55fcc41531c0 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdab6ff5640 10 client.19542 handle_command_reply: tid=0 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 client.19542 shutdown 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 msgr2=0x55fcc456e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006010 tx=0x7fdac8025fa0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 shutdown_connections 2024-04-25T13:52:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fcc41531c0 0x55fcc45e2130 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fcc41bf1b0 0x55fcc456f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fcc419e4d0 0x55fcc456e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 --2- 172.21.15.94:0/458975749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcc41962f0 0x55fcc456e420 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 >> 172.21.15.94:0/458975749 conn(0x55fcc41b3300 msgr2=0x55fcc4569210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdab6ff5640 0 client.19542 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:52:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 shutdown_connections 2024-04-25T13:52:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 1 -- 172.21.15.94:0/458975749 wait complete. 2024-04-25T13:52:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.009+0000 7fdae025e740 20 client.19542 trim_cache size 0 max 16384 2024-04-25T13:52:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:52:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:52:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:52:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:52:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 msgr2=0x7fdabc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 0x7fdabc0b1ff0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0a3740 tx=0x7fdacc076040 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 msgr2=0x7fdad41a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00eb10 tx=0x7fdacc00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 shutdown_connections 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdabc0afae0 0x7fdabc0b1ff0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad4108940 0x7fdad41a1730 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad4103c40 0x7fdad419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 --2- 172.21.15.94:0/3707795324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad4102a00 0x7fdad419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 >> 172.21.15.94:0/3707795324 conn(0x7fdad40fdbd0 msgr2=0x7fdad4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 shutdown_connections 2024-04-25T13:52:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:52:53.010+0000 7fdad93b2640 1 -- 172.21.15.94:0/3707795324 wait complete. 2024-04-25T13:52:53.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1367843 12.59 MB/sec execute 456 sec latency 107.000 ms 2024-04-25T13:52:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:54 smithi137 ceph-mon[32568]: pgmap v682: 97 pgs: 97 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:52:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:54 smithi094 ceph-mon[29195]: pgmap v682: 97 pgs: 97 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:52:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:54 smithi138 ceph-mon[32977]: pgmap v682: 97 pgs: 97 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:52:54.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1370518 12.59 MB/sec execute 457 sec latency 14.237 ms 2024-04-25T13:52:55.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1372135 12.59 MB/sec execute 458 sec latency 14.430 ms 2024-04-25T13:52:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:56 smithi137 ceph-mon[32568]: pgmap v683: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-04-25T13:52:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:56 smithi094 ceph-mon[29195]: pgmap v683: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-04-25T13:52:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:56 smithi138 ceph-mon[32977]: pgmap v683: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-04-25T13:52:56.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1374437 12.59 MB/sec execute 459 sec latency 75.945 ms 2024-04-25T13:52:57.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1375574 12.61 MB/sec execute 460 sec latency 49.470 ms 2024-04-25T13:52:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:52:58 smithi137 ceph-mon[32568]: pgmap v684: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-04-25T13:52:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:52:58 smithi094 ceph-mon[29195]: pgmap v684: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-04-25T13:52:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:52:58 smithi138 ceph-mon[32977]: pgmap v684: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 344 op/s 2024-04-25T13:52:58.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1378525 12.62 MB/sec execute 461 sec latency 9.763 ms 2024-04-25T13:52:59.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1380485 12.61 MB/sec execute 462 sec latency 14.985 ms 2024-04-25T13:53:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:00 smithi137 ceph-mon[32568]: pgmap v685: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-04-25T13:53:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:00 smithi094 ceph-mon[29195]: pgmap v685: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-04-25T13:53:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:00 smithi138 ceph-mon[32977]: pgmap v685: 97 pgs: 97 active+clean; 35 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2024-04-25T13:53:00.812 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1382753 12.61 MB/sec execute 463 sec latency 15.272 ms 2024-04-25T13:53:01.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1385143 12.61 MB/sec execute 464 sec latency 56.533 ms 2024-04-25T13:53:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:02 smithi137 ceph-mon[32568]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T13:53:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:02 smithi094 ceph-mon[29195]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T13:53:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:02 smithi138 ceph-mon[32977]: pgmap v686: 97 pgs: 97 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T13:53:02.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1387091 12.61 MB/sec execute 465 sec latency 18.590 ms 2024-04-25T13:53:03.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1389762 12.61 MB/sec execute 466 sec latency 10.966 ms 2024-04-25T13:53:04.646 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:04 smithi137 ceph-mon[32568]: pgmap v687: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-04-25T13:53:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:04 smithi094 ceph-mon[29195]: pgmap v687: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-04-25T13:53:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:04 smithi138 ceph-mon[32977]: pgmap v687: 97 pgs: 97 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-04-25T13:53:04.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1392754 12.62 MB/sec execute 467 sec latency 13.949 ms 2024-04-25T13:53:05.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1394414 12.61 MB/sec execute 468 sec latency 56.079 ms 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 -- 172.21.15.94:0/628411011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1021b0 msgr2=0x7f74cc102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/628411011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1021b0 0x7f74cc102eb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009900 tx=0x7f74bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 -- 172.21.15.94:0/628411011 shutdown_connections 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/628411011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1080f0 0x7f74cc10a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/628411011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1033f0 0x7f74cc103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/628411011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1021b0 0x7f74cc102eb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 -- 172.21.15.94:0/628411011 >> 172.21.15.94:0/628411011 conn(0x7f74cc0fd380 msgr2=0x7f74cc0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 -- 172.21.15.94:0/628411011 shutdown_connections 2024-04-25T13:53:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.113+0000 7f74d0dc0640 1 -- 172.21.15.94:0/628411011 wait complete. 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 Processor -- start 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 -- start start 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1021b0 0x7f74cc19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1080f0 0x7f74cc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc10d440 con 0x7f74cc1033f0 2024-04-25T13:53:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc10d2c0 con 0x7f74cc1080f0 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74d0dc0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc10d5c0 con 0x7f74cc1021b0 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51420/0 (socket says 172.21.15.94:51420) 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 -- 172.21.15.94:0/366673399 learned_addr learned my addr 172.21.15.94:0/366673399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74cad9d640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1080f0 0x7f74cc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74ca59c640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1021b0 0x7f74cc19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 -- 172.21.15.94:0/366673399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1021b0 msgr2=0x7f74cc19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1021b0 0x7f74cc19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 -- 172.21.15.94:0/366673399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1080f0 msgr2=0x7f74cc1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1080f0 0x7f74cc1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 -- 172.21.15.94:0/366673399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc1a1d60 con 0x7f74cc1033f0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.114+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e9f0 tx=0x7f74c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.115+0000 7f74bb7fe640 1 -- 172.21.15.94:0/366673399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c000cd80 con 0x7f74cc1033f0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.115+0000 7f74bb7fe640 1 -- 172.21.15.94:0/366673399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c00108e0 con 0x7f74cc1033f0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.115+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc1a1ff0 con 0x7f74cc1033f0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.115+0000 7f74bb7fe640 1 -- 172.21.15.94:0/366673399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0018980 con 0x7f74cc1033f0 2024-04-25T13:53:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.115+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc1a24b0 con 0x7f74cc1033f0 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.117+0000 7f74d7c70740 1 Processor -- start 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.117+0000 7f74d7c70740 1 -- start start 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.117+0000 7f74bb7fe640 1 -- 172.21.15.94:0/366673399 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f74c0018b20 con 0x7f74cc1033f0 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560189835460 con 0x56018950e4d0 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601897ef590 con 0x56018952f1b0 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74d7c70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560189831420 con 0x5601895062f0 2024-04-25T13:53:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:51670/0 (socket says 172.21.15.94:51670) 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 -- 172.21.15.94:0/1676636208 learned_addr learned my addr 172.21.15.94:0/1676636208 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 -- 172.21.15.94:0/1676636208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 msgr2=0x5601898cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 -- 172.21.15.94:0/1676636208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 msgr2=0x5601898d6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898d6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 -- 172.21.15.94:0/1676636208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601897fc0d0 con 0x56018952f1b0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74bb7fe640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 0x7f74b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e480 tx=0x7f74c400a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f91658886fddd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74ca59c640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 0x7f74b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.118+0000 7f74bb7fe640 1 -- 172.21.15.94:0/366673399 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f74c00df020 con 0x7f74cc1033f0 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.119+0000 7f74b8ff9640 1 -- 172.21.15.94:0/1676636208 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004d20 con 0x56018952f1b0 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.119+0000 7f74b8ff9640 1 -- 172.21.15.94:0/1676636208 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c400da70 con 0x56018952f1b0 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.119+0000 7f74b8ff9640 1 -- 172.21.15.94:0/1676636208 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4005640 con 0x56018952f1b0 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- 172.21.15.94:0/1676636208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 msgr2=0x5601898cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74ca59c640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 0x7f74b40b1ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005ab0 tx=0x7f74bc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e480 tx=0x7f74c400a610 comp rx=0 tx=0).stop 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- 172.21.15.94:0/1676636208 shutdown_connections 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898d6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1676636208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x5601898cf970 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- 172.21.15.94:0/1676636208 >> 172.21.15.94:0/1676636208 conn(0x560189523300 msgr2=0x5601898cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- 172.21.15.94:0/1676636208 shutdown_connections 2024-04-25T13:53:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- 172.21.15.94:0/1676636208 wait complete. 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- start start 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 Processor -- start 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5601898d7840 con 0x56018952f1b0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601897bcb60 con 0x56018950e4d0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.120+0000 7f74d7c70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601898d79c0 con 0x5601895062f0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74d7c70740 10 client.0 _ll_register_callbacks cb 0x5601897e12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51440/0 (socket says 172.21.15.94:51440) 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74cad9d640 1 -- 172.21.15.94:0/1322993414 learned_addr learned my addr 172.21.15.94:0/1322993414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 -- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 msgr2=0x5601898d9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898d9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 -- 172.21.15.94:0/1322993414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 msgr2=0x5601898e0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 -- 172.21.15.94:0/1322993414 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560189547210 con 0x5601895062f0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74ca59c640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d9760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74bc02a930 tx=0x7f74bc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d8baac3a2f5212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035040 con 0x5601895062f0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f74bc004300 con 0x5601895062f0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c400da70 con 0x5601895062f0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74bc0035f0 con 0x5601895062f0 2024-04-25T13:53:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74d7c70740 10 client.28804 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.121+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560189613620 con 0x5601895062f0 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c400da70 con 0x5601895062f0 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f74bc044070 con 0x5601895062f0 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 10 client.28804 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 10 client.28804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 20 client.28804 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894edd80 0x5601898e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 4 client.28804 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560189843070 con 0x5601894edd80 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894edd80 0x5601898e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74b1ffb640 10 client.28804 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.122+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894edd80 0x5601898e73e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f17d9c0918f495 server_cookie=e2d54f5eb94a07a0 in_seq=0 out_seq=0 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.123+0000 7f74b1ffb640 1 -- 172.21.15.94:0/1322993414 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560189843070 con 0x5601894edd80 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74b1ffb640 10 client.28804 handle_command_reply: tid=0 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 client.28804 shutdown 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 msgr2=0x5601898d9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d9760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74bc02a930 tx=0x7f74bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 shutdown_connections 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894edd80 0x5601898e73e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898d9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74b1ffb640 0 client.28804 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1322993414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d9760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 >> 172.21.15.94:0/1322993414 conn(0x560189523300 msgr2=0x5601898ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 shutdown_connections 2024-04-25T13:53:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 1 -- 172.21.15.94:0/1322993414 wait complete. 2024-04-25T13:53:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.124+0000 7f74d7c70740 20 client.28804 trim_cache size 0 max 16384 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 Processor -- start 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 -- start start 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1ab500 con 0x56018952f1b0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1ad040 con 0x56018950e4d0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.140+0000 7f74d7c70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1abbd0 con 0x5601895062f0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51454/0 (socket says 172.21.15.94:51454) 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 -- 172.21.15.94:0/1198595948 learned_addr learned my addr 172.21.15.94:0/1198595948 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 -- 172.21.15.94:0/1198595948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 msgr2=0x5601898d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 -- 172.21.15.94:0/1198595948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 msgr2=0x5601898e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 -- 172.21.15.94:0/1198595948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601898d6f90 con 0x56018952f1b0 2024-04-25T13:53:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74cad9d640 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f74c4010010 tx=0x7f74c4005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c375a525b38a188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74b3fff640 1 -- 172.21.15.94:0/1198595948 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4028070 con 0x56018952f1b0 2024-04-25T13:53:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74b3fff640 1 -- 172.21.15.94:0/1198595948 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c4005520 con 0x56018952f1b0 2024-04-25T13:53:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.141+0000 7f74b3fff640 1 -- 172.21.15.94:0/1198595948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400d8f0 con 0x56018952f1b0 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 -- 172.21.15.94:0/1198595948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 msgr2=0x5601898ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f74c4010010 tx=0x7f74c4005990 comp rx=0 tx=0).stop 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 -- 172.21.15.94:0/1198595948 shutdown_connections 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018952f1b0 0x5601898ce810 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018950e4d0 0x5601898e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 --2- 172.21.15.94:0/1198595948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 -- 172.21.15.94:0/1198595948 >> 172.21.15.94:0/1198595948 conn(0x560189523300 msgr2=0x5601898ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 -- 172.21.15.94:0/1198595948 shutdown_connections 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.142+0000 7f74d7c70740 1 -- 172.21.15.94:0/1198595948 wait complete. 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 -- start start 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 Processor -- start 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x56018994d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1ac970 con 0x56018950e4d0 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1ac2a0 con 0x56018952f1b0 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1a94e0 con 0x5601895062f0 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74d7c70740 10 client.0 _ll_register_callbacks cb 0x7f74bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51468/0 (socket says 172.21.15.94:51468) 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 -- 172.21.15.94:0/4110601682 learned_addr learned my addr 172.21.15.94:0/4110601682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74ca59c640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74cad9d640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x56018994d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 -- 172.21.15.94:0/4110601682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 msgr2=0x5601898dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 -- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 msgr2=0x56018994d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.143+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x56018994d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74c9d9b640 1 -- 172.21.15.94:0/4110601682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018994de50 con 0x56018950e4d0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74c9d9b640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f74c00a74a0 tx=0x7f74c00a34d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f0d57840db16e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00a6c80 con 0x56018950e4d0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f74c00a37a0 con 0x56018950e4d0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c4005520 con 0x56018950e4d0 2024-04-25T13:53:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74c00a7020 con 0x56018950e4d0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74d7c70740 10 client.19560 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56018994f190 con 0x56018950e4d0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.144+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c4005520 con 0x56018950e4d0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f74c00e1170 con 0x56018950e4d0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 10 client.19560 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 10 client.19560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 20 client.19560 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894c31c0 0x560189951ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 4 client.19560 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56018983cee0 con 0x5601894c31c0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74ca59c640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894c31c0 0x560189951ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74b1ffb640 10 client.19560 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.145+0000 7f74ca59c640 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894c31c0 0x560189951ae0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95e281f43c38988 server_cookie=8408f692c0c690d3 in_seq=0 out_seq=0 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74b1ffb640 1 -- 172.21.15.94:0/4110601682 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56018983cee0 con 0x5601894c31c0 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74b1ffb640 10 client.19560 handle_command_reply: tid=0 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 client.19560 shutdown 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 msgr2=0x5601898e0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f74c00a74a0 tx=0x7f74c00a34d0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 shutdown_connections 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5601894c31c0 0x560189951ae0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56018952f1b0 0x56018994d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56018950e4d0 0x5601898e0240 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 --2- 172.21.15.94:0/4110601682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601895062f0 0x5601898dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74b1ffb640 0 client.19560 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 >> 172.21.15.94:0/4110601682 conn(0x560189523300 msgr2=0x5601898d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 shutdown_connections 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.329+0000 7f74d7c70740 1 -- 172.21.15.94:0/4110601682 wait complete. 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d7c70740 20 client.19560 trim_cache size 0 max 16384 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:53:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:53:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 msgr2=0x7f74b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 0x7f74b40b1ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005ab0 tx=0x7f74bc031400 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 msgr2=0x7f74cc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e9f0 tx=0x7f74c000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 shutdown_connections 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f74b40afae0 0x7f74b40b1ff0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74cc1080f0 0x7f74cc1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74cc1033f0 0x7f74cc19a760 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 --2- 172.21.15.94:0/366673399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74cc1021b0 0x7f74cc19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 >> 172.21.15.94:0/366673399 conn(0x7f74cc0fd380 msgr2=0x7f74cc1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 shutdown_connections 2024-04-25T13:53:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:06.330+0000 7f74d0dc0640 1 -- 172.21.15.94:0/366673399 wait complete. 2024-04-25T13:53:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:06 smithi137 ceph-mon[32568]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2024-04-25T13:53:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:06 smithi094 ceph-mon[29195]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2024-04-25T13:53:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:06 smithi138 ceph-mon[32977]: pgmap v688: 97 pgs: 97 active+clean; 27 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.7 MiB/s wr, 359 op/s 2024-04-25T13:53:06.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1396593 12.61 MB/sec execute 469 sec latency 104.055 ms 2024-04-25T13:53:07.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1398804 12.60 MB/sec execute 470 sec latency 14.087 ms 2024-04-25T13:53:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:08 smithi137 ceph-mon[32568]: pgmap v689: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-04-25T13:53:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:08 smithi094 ceph-mon[29195]: pgmap v689: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-04-25T13:53:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:08 smithi138 ceph-mon[32977]: pgmap v689: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-04-25T13:53:08.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1400716 12.60 MB/sec execute 471 sec latency 15.640 ms 2024-04-25T13:53:09.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1403588 12.61 MB/sec execute 472 sec latency 11.852 ms 2024-04-25T13:53:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:10 smithi137 ceph-mon[32568]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-25T13:53:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:10 smithi094 ceph-mon[29195]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-25T13:53:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:10 smithi138 ceph-mon[32977]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-25T13:53:10.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1405429 12.60 MB/sec execute 473 sec latency 13.885 ms 2024-04-25T13:53:11.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1407382 12.60 MB/sec execute 474 sec latency 105.074 ms 2024-04-25T13:53:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:12 smithi137 ceph-mon[32568]: pgmap v691: 97 pgs: 97 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:53:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:12 smithi094 ceph-mon[29195]: pgmap v691: 97 pgs: 97 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:53:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:12 smithi138 ceph-mon[32977]: pgmap v691: 97 pgs: 97 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 375 op/s 2024-04-25T13:53:12.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1409431 12.59 MB/sec execute 475 sec latency 45.072 ms 2024-04-25T13:53:13.814 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1411263 12.59 MB/sec execute 476 sec latency 24.637 ms 2024-04-25T13:53:14.526 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:14 smithi094 ceph-mon[29195]: pgmap v692: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:53:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:14 smithi137 ceph-mon[32568]: pgmap v692: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:53:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:14 smithi138 ceph-mon[32977]: pgmap v692: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:53:14.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1414319 12.60 MB/sec execute 477 sec latency 11.603 ms 2024-04-25T13:53:15.813 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1416590 12.59 MB/sec execute 478 sec latency 16.817 ms 2024-04-25T13:53:16.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:53:16.180 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:53:16.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- 172.21.15.94:0/667813130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d441029e0 msgr2=0x7f2d441036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/667813130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d441029e0 0x7f2d441036e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d3402a410 comp rx=0 tx=0).stop 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- 172.21.15.94:0/667813130 shutdown_connections 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/667813130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44108920 0x7f2d4410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/667813130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d44103c20 0x7f2d44104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/667813130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d441029e0 0x7f2d441036e0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- 172.21.15.94:0/667813130 >> 172.21.15.94:0/667813130 conn(0x7f2d440fdbd0 msgr2=0x7f2d44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- 172.21.15.94:0/667813130 shutdown_connections 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- 172.21.15.94:0/667813130 wait complete. 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 Processor -- start 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.295+0000 7f2d49dd7640 1 -- start start 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d441029e0 0x7f2d4406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44103c20 0x7f2d4419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4410e4e0 con 0x7f2d44108920 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4410e360 con 0x7f2d44103c20 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4410e660 con 0x7f2d441029e0 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51474/0 (socket says 172.21.15.94:51474) 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d441029e0 0x7f2d4406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 -- 172.21.15.94:0/3706753652 learned_addr learned my addr 172.21.15.94:0/3706753652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44103c20 0x7f2d4419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 -- 172.21.15.94:0/3706753652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d441029e0 msgr2=0x7f2d4406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d441029e0 0x7f2d4406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 -- 172.21.15.94:0/3706753652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44103c20 msgr2=0x7f2d4419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44103c20 0x7f2d4419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 -- 172.21.15.94:0/3706753652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d441a2090 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d43fff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00eb10 tx=0x7f2d3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d417fa640 1 -- 172.21.15.94:0/3706753652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004b20 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d441a2380 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d417fa640 1 -- 172.21.15.94:0/3706753652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d3c004cc0 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d417fa640 1 -- 172.21.15.94:0/3706753652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c010670 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.296+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d441a2880 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d417fa640 1 -- 172.21.15.94:0/3706753652 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2d3c005020 con 0x7f2d44108920 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 Processor -- start 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 -- start start 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0638a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0639c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b5b059dd80 con 0x55b5b02766b0 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5b05578b0 con 0x55b5b0297390 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d50d64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5b0599d40 con 0x55b5b026e4d0 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0639c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56478/0 (socket says 172.21.15.94:56478) 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0639c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51486/0 (socket says 172.21.15.94:51486) 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d437fe640 1 -- 172.21.15.94:0/2010126493 learned_addr learned my addr 172.21.15.94:0/2010126493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.299+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0638a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d417fa640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 0x7f2d300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 -- 172.21.15.94:0/2010126493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 msgr2=0x55b5b0638a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0638a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 -- 172.21.15.94:0/2010126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 msgr2=0x55b5b0639c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0639c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 -- 172.21.15.94:0/2010126493 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b0528a40 con 0x55b5b026e4d0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d43fff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 0x7f2d300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d437fe640 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009230 tx=0x7f2d34009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f99d3bed9d21eefa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d417fa640 1 -- 172.21.15.94:0/3706753652 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2d3c0df2e0 con 0x7f2d44108920 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d39ffb640 1 -- 172.21.15.94:0/2010126493 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34035030 con 0x55b5b026e4d0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d39ffb640 1 -- 172.21.15.94:0/2010126493 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d34040070 con 0x55b5b026e4d0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.300+0000 7f2d39ffb640 1 -- 172.21.15.94:0/2010126493 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34003830 con 0x55b5b026e4d0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- 172.21.15.94:0/2010126493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 msgr2=0x55b5b0638fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009230 tx=0x7f2d34009620 comp rx=0 tx=0).stop 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d43fff640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 0x7f2d300b2000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0a73c0 tx=0x7f2d3c0ab040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- 172.21.15.94:0/2010126493 shutdown_connections 2024-04-25T13:53:16.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0639c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0638fe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2010126493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0638a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- 172.21.15.94:0/2010126493 >> 172.21.15.94:0/2010126493 conn(0x55b5b028b4e0 msgr2=0x55b5b0634270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- 172.21.15.94:0/2010126493 shutdown_connections 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- 172.21.15.94:0/2010126493 wait complete. 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.301+0000 7f2d50d64740 1 -- start start 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 Processor -- start 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b026e4d0 0x55b5b0642940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0642e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b5b063a720 con 0x55b5b0297390 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5b0524d10 con 0x55b5b026e4d0 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5b063a8a0 con 0x55b5b02766b0 2024-04-25T13:53:16.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d50d64740 10 client.0 _ll_register_callbacks cb 0x55b5b0551210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51502/0 (socket says 172.21.15.94:51502) 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d3bfff640 1 -- 172.21.15.94:0/985883200 learned_addr learned my addr 172.21.15.94:0/985883200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b026e4d0 0x55b5b0642940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.302+0000 7f2d437fe640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0642e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 -- 172.21.15.94:0/985883200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 msgr2=0x55b5b0642e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0642e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 -- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b026e4d0 msgr2=0x55b5b0642940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b026e4d0 0x55b5b0642940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 -- 172.21.15.94:0/985883200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b03797d0 con 0x55b5b0297390 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00f480 tx=0x7f2d2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16aa3fabac04f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:16.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c011260 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2d2c01a040 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d34040070 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c012020 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d50d64740 10 client.19572 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5b02af3f0 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d34040070 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.303+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d2c029070 con 0x55b5b0297390 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 10 client.19572 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 10 client.19572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 20 client.19572 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b0255f60 0x55b5b06502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 4 client.19572 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b05ab990 con 0x55b5b0255f60 2024-04-25T13:53:16.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d437fe640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b0255f60 0x55b5b06502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d437fe640 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b0255f60 0x55b5b06502f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6840f45adfd2e67d server_cookie=da43fd2ce238a662 in_seq=0 out_seq=0 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.304+0000 7f2d22ffd640 10 client.19572 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d22ffd640 1 -- 172.21.15.94:0/985883200 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5b05ab990 con 0x55b5b0255f60 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d22ffd640 10 client.19572 handle_command_reply: tid=0 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 client.19572 shutdown 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 msgr2=0x55b5b0649de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00f480 tx=0x7f2d2c004730 comp rx=0 tx=0).stop 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 shutdown_connections 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b0255f60 0x55b5b06502f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b0297390 0x55b5b0649de0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0642e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.305+0000 7f2d22ffd640 0 client.19572 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:16.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.306+0000 7f2d50d64740 1 --2- 172.21.15.94:0/985883200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b026e4d0 0x55b5b0642940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.306+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 >> 172.21.15.94:0/985883200 conn(0x55b5b028b4e0 msgr2=0x55b5b0637180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.306+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 shutdown_connections 2024-04-25T13:53:16.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.306+0000 7f2d50d64740 1 -- 172.21.15.94:0/985883200 wait complete. 2024-04-25T13:53:16.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.306+0000 7f2d50d64740 20 client.19572 trim_cache size 0 max 16384 2024-04-25T13:53:16.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 Processor -- start 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 -- start start 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0633c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0634170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ab4d0 con 0x55b5b026e4d0 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ad010 con 0x55b5b0297390 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d50d64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441abba0 con 0x55b5b02766b0 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51508/0 (socket says 172.21.15.94:51508) 2024-04-25T13:53:16.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 -- 172.21.15.94:0/2538803458 learned_addr learned my addr 172.21.15.94:0/2538803458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d437fe640 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0633c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0634170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 -- 172.21.15.94:0/2538803458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 msgr2=0x55b5b0633c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0633c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 -- 172.21.15.94:0/2538803458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 msgr2=0x55b5b0634170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0634170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.324+0000 7f2d3bfff640 1 -- 172.21.15.94:0/2538803458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b026f0f0 con 0x55b5b026e4d0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.325+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c005eb0 tx=0x7f2d2c012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745386265dcc3a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.325+0000 7f2d397fa640 1 -- 172.21.15.94:0/2538803458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c02f070 con 0x55b5b026e4d0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.325+0000 7f2d397fa640 1 -- 172.21.15.94:0/2538803458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2c030040 con 0x55b5b026e4d0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.325+0000 7f2d397fa640 1 -- 172.21.15.94:0/2538803458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c005020 con 0x55b5b026e4d0 2024-04-25T13:53:16.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- 172.21.15.94:0/2538803458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 msgr2=0x55b5b0641610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c005eb0 tx=0x7f2d2c012da0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- 172.21.15.94:0/2538803458 shutdown_connections 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0634170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b02766b0 0x55b5b0633c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- 172.21.15.94:0/2538803458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b026e4d0 0x55b5b0641610 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- 172.21.15.94:0/2538803458 >> 172.21.15.94:0/2538803458 conn(0x55b5b028b4e0 msgr2=0x55b5b0640f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- 172.21.15.94:0/2538803458 shutdown_connections 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- 172.21.15.94:0/2538803458 wait complete. 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- start start 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 Processor -- start 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0648080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ac940 con 0x55b5b02766b0 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ac270 con 0x55b5b0297390 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441a94b0 con 0x55b5b026e4d0 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.326+0000 7f2d50d64740 10 client.0 _ll_register_callbacks cb 0x7f2d2c012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51514/0 (socket says 172.21.15.94:51514) 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56500/0 (socket says 172.21.15.94:56500) 2024-04-25T13:53:16.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 -- 172.21.15.94:0/678392854 learned_addr learned my addr 172.21.15.94:0/678392854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0648080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 -- 172.21.15.94:0/678392854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 msgr2=0x55b5b0646c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 -- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 msgr2=0x55b5b0648080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0648080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 -- 172.21.15.94:0/678392854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b055e4b0 con 0x55b5b02766b0 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d437fe640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d34009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6f54f51ca88c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:16.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34057060 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2d34004040 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c030040 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d34035040 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d50d64740 10 client.19578 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.327+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b5b0649800 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2c030040 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2d34045470 con 0x55b5b02766b0 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 10 client.19578 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 20 client.19578 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:16.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b022b3a0 0x55b5b06baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 4 client.19578 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5b05a5800 con 0x55b5b022b3a0 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b022b3a0 0x55b5b06baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d22ffd640 10 client.19578 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.328+0000 7f2d3bfff640 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b022b3a0 0x55b5b06baa10 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea27d28610412f0e server_cookie=15696939a1512ebc in_seq=0 out_seq=0 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d22ffd640 1 -- 172.21.15.94:0/678392854 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b5b05a5800 con 0x55b5b022b3a0 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d22ffd640 10 client.19578 handle_command_reply: tid=0 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 client.19578 shutdown 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 msgr2=0x55b5b0647150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d34009260 comp rx=0 tx=0).stop 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 shutdown_connections 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b5b022b3a0 0x55b5b06baa10 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b0297390 0x55b5b0648080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5b02766b0 0x55b5b0647150 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d22ffd640 0 client.19578 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 --2- 172.21.15.94:0/678392854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5b026e4d0 0x55b5b0646c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 >> 172.21.15.94:0/678392854 conn(0x55b5b028b4e0 msgr2=0x55b5b0633c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 shutdown_connections 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 1 -- 172.21.15.94:0/678392854 wait complete. 2024-04-25T13:53:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d50d64740 20 client.19578 trim_cache size 0 max 16384 2024-04-25T13:53:16.338 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:53:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:53:16.338 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 msgr2=0x7f2d300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 0x7f2d300b2000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0a73c0 tx=0x7f2d3c0ab040 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 msgr2=0x7f2d441a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.329+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00eb10 tx=0x7f2d3c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 shutdown_connections 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2d300afaf0 0x7f2d300b2000 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d44108920 0x7f2d441a1870 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d44103c20 0x7f2d4419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 --2- 172.21.15.94:0/3706753652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d441029e0 0x7f2d4406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 >> 172.21.15.94:0/3706753652 conn(0x7f2d440fdbd0 msgr2=0x7f2d44104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 shutdown_connections 2024-04-25T13:53:16.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.330+0000 7f2d49dd7640 1 -- 172.21.15.94:0/3706753652 wait complete. 2024-04-25T13:53:16.341 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:53:16.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:53:16.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 -- 172.21.15.94:0/3727040509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c108900 msgr2=0x7f211c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 --2- 172.21.15.94:0/3727040509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c108900 0x7f211c10ad20 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2114009ca0 tx=0x7f211402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 -- 172.21.15.94:0/3727040509 shutdown_connections 2024-04-25T13:53:16.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 --2- 172.21.15.94:0/3727040509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c108900 0x7f211c10ad20 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 --2- 172.21.15.94:0/3727040509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 0x7f211c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 --2- 172.21.15.94:0/3727040509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c1029c0 0x7f211c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 -- 172.21.15.94:0/3727040509 >> 172.21.15.94:0/3727040509 conn(0x7f211c0fdbd0 msgr2=0x7f211c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 -- 172.21.15.94:0/3727040509 shutdown_connections 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.451+0000 7f2120908640 1 -- 172.21.15.94:0/3727040509 wait complete. 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 Processor -- start 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 -- start start 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 0x7f211c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c108900 0x7f211c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10e4f0 con 0x7f211c1029c0 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10e370 con 0x7f211c108900 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.452+0000 7f2120908640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10e670 con 0x7f211c103c00 2024-04-25T13:53:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51534/0 (socket says 172.21.15.94:51534) 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f2113fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 0x7f211c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 -- 172.21.15.94:0/698419876 learned_addr learned my addr 172.21.15.94:0/698419876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211ad9d640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c108900 0x7f211c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 -- 172.21.15.94:0/698419876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 msgr2=0x7f211c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 0x7f211c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 -- 172.21.15.94:0/698419876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c108900 msgr2=0x7f211c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c108900 0x7f211c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 -- 172.21.15.94:0/698419876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c1a1cf0 con 0x7f211c1029c0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f211a59c640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f210c00e9f0 tx=0x7f210c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c00ccc0 con 0x7f211c1029c0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f210c0035f0 con 0x7f211c1029c0 2024-04-25T13:53:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c00ce60 con 0x7f211c1029c0 2024-04-25T13:53:16.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c1a1fe0 con 0x7f211c1029c0 2024-04-25T13:53:16.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.453+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c1a2440 con 0x7f211c1029c0 2024-04-25T13:53:16.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.454+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c06afa0 con 0x7f211c1029c0 2024-04-25T13:53:16.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.456+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f210c003aa0 con 0x7f211c1029c0 2024-04-25T13:53:16.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.456+0000 7f21137fe640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 0x7f21080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.457+0000 7f2113fff640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 0x7f21080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.457+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f210c0df030 con 0x7f211c1029c0 2024-04-25T13:53:16.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.457+0000 7f2113fff640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 0x7f21080b2000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f211c19b250 tx=0x7f2104005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.460+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f210c0a7b70 con 0x7f211c1029c0 2024-04-25T13:53:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:16 smithi094 ceph-mon[29195]: pgmap v693: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-04-25T13:53:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:16 smithi137 ceph-mon[32568]: pgmap v693: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-04-25T13:53:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.688+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f211c1a2780 con 0x7f211c1029c0 2024-04-25T13:53:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.688+0000 7f21137fe640 1 -- 172.21.15.94:0/698419876 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11941 (secure 0 0 0) 0x7f210c0a7470 con 0x7f211c1029c0 2024-04-25T13:53:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:53:16.689 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:52:14.280433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:53:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 msgr2=0x7f21080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 0x7f21080b2000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f211c19b250 tx=0x7f2104005f90 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 msgr2=0x7f211c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f210c00e9f0 tx=0x7f210c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 shutdown_connections 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f21080afaf0 0x7f21080b2000 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.690+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f211c108900 0x7f211c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.691+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f211c103c00 0x7f211c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.691+0000 7f2120908640 1 --2- 172.21.15.94:0/698419876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c1029c0 0x7f211c19a1b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.691+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 >> 172.21.15.94:0/698419876 conn(0x7f211c0fdbd0 msgr2=0x7f211c109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.691+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 shutdown_connections 2024-04-25T13:53:16.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.691+0000 7f2120908640 1 -- 172.21.15.94:0/698419876 wait complete. 2024-04-25T13:53:16.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:53:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:16 smithi138 ceph-mon[32977]: pgmap v693: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-04-25T13:53:16.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1418604 12.59 MB/sec execute 479 sec latency 22.047 ms 2024-04-25T13:53:16.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.831+0000 7fcf4831f640 1 -- 172.21.15.94:0/211303960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 msgr2=0x7fcf4010add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.831+0000 7fcf4831f640 1 --2- 172.21.15.94:0/211303960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf4010add0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009860 tx=0x7fcf3c02a360 comp rx=0 tx=0).stop 2024-04-25T13:53:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.836+0000 7fcf4831f640 1 -- 172.21.15.94:0/211303960 shutdown_connections 2024-04-25T13:53:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.836+0000 7fcf4831f640 1 --2- 172.21.15.94:0/211303960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf40110030 0x7fcf40110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.836+0000 7fcf4831f640 1 --2- 172.21.15.94:0/211303960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4010b310 0x7fcf4010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.836+0000 7fcf4831f640 1 --2- 172.21.15.94:0/211303960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf4010add0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.836+0000 7fcf4831f640 1 -- 172.21.15.94:0/211303960 >> 172.21.15.94:0/211303960 conn(0x7fcf400704e0 msgr2=0x7fcf40072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.841+0000 7fcf4831f640 1 -- 172.21.15.94:0/211303960 shutdown_connections 2024-04-25T13:53:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.841+0000 7fcf4831f640 1 -- 172.21.15.94:0/211303960 wait complete. 2024-04-25T13:53:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 Processor -- start 2024-04-25T13:53:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 -- start start 2024-04-25T13:53:16.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf401170d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 0x7fcf40117610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf40110030 0x7fcf40118570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4011e540 con 0x7fcf4010b310 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4011e3c0 con 0x7fcf4010a420 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.842+0000 7fcf4831f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4011e6c0 con 0x7fcf40110030 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.843+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf401170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.843+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf401170d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34068/0 (socket says 172.21.15.94:34068) 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.843+0000 7fcf460bb640 1 -- 172.21.15.94:0/2096885747 learned_addr learned my addr 172.21.15.94:0/2096885747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.843+0000 7fcf468bc640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf40110030 0x7fcf40118570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.843+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 0x7fcf40117610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.844+0000 7fcf458ba640 1 -- 172.21.15.94:0/2096885747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf40110030 msgr2=0x7fcf40118570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.844+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf40110030 0x7fcf40118570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.844+0000 7fcf458ba640 1 -- 172.21.15.94:0/2096885747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 msgr2=0x7fcf401170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.844+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf401170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.844+0000 7fcf458ba640 1 -- 172.21.15.94:0/2096885747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf40118be0 con 0x7fcf4010b310 2024-04-25T13:53:16.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.845+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 0x7fcf40117610 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800be20 tx=0x7fcf38009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.845+0000 7fcf377fe640 1 -- 172.21.15.94:0/2096885747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38019070 con 0x7fcf4010b310 2024-04-25T13:53:16.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.845+0000 7fcf377fe640 1 -- 172.21.15.94:0/2096885747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf380148f0 con 0x7fcf4010b310 2024-04-25T13:53:16.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.845+0000 7fcf377fe640 1 -- 172.21.15.94:0/2096885747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380128e0 con 0x7fcf4010b310 2024-04-25T13:53:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.846+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf40118ed0 con 0x7fcf4010b310 2024-04-25T13:53:16.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.846+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf401193d0 con 0x7fcf4010b310 2024-04-25T13:53:16.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.851+0000 7fcf4f1ce740 1 Processor -- start 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.851+0000 7fcf4f1ce740 1 -- start start 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.851+0000 7fcf377fe640 1 -- 172.21.15.94:0/2096885747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcf38014440 con 0x7fcf4010b310 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3086b0 0x55614b6cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55614b62fd80 con 0x55614b3086b0 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55614b5e9a20 con 0x55614b329390 2024-04-25T13:53:16.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55614b62bd40 con 0x55614b3004d0 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf458ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3086b0 0x55614b6cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf468bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56562/0 (socket says 172.21.15.94:56562) 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf468bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34072/0 (socket says 172.21.15.94:34072) 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 -- 172.21.15.94:0/1345549460 learned_addr learned my addr 172.21.15.94:0/1345549460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 -- 172.21.15.94:0/1345549460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 msgr2=0x55614b6cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.852+0000 7fcf460bb640 1 -- 172.21.15.94:0/1345549460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3086b0 msgr2=0x55614b6cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf460bb640 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3086b0 0x55614b6cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf460bb640 1 -- 172.21.15.94:0/1345549460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614b5f4c30 con 0x55614b3004d0 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf377fe640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 0x7fcf2c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf468bc640 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf377fe640 1 -- 172.21.15.94:0/2096885747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf380de400 con 0x7fcf4010b310 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf468bc640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 0x7fcf2c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf460bb640 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c000c00 tx=0x7fcf3c030c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1c6b8ce9425a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf34bfe640 1 -- 172.21.15.94:0/1345549460 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c041030 con 0x55614b3004d0 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf34bfe640 1 -- 172.21.15.94:0/1345549460 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c038040 con 0x55614b3004d0 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf468bc640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 0x7fcf2c0b2190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55614b6cc860 tx=0x7fcf3000b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.853+0000 7fcf34bfe640 1 -- 172.21.15.94:0/1345549460 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c037070 con 0x55614b3004d0 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1345549460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 msgr2=0x55614b6cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c000c00 tx=0x7fcf3c030c70 comp rx=0 tx=0).stop 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1345549460 shutdown_connections 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3086b0 0x55614b6cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6cbd90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1345549460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1345549460 >> 172.21.15.94:0/1345549460 conn(0x55614b31d4e0 msgr2=0x55614b6c7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1345549460 shutdown_connections 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.854+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1345549460 wait complete. 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 -- start start 2024-04-25T13:53:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 Processor -- start 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55614b6cd4d0 con 0x55614b3004d0 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55614b5b6dc0 con 0x55614b329390 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55614b6cd650 con 0x55614b3086b0 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51560/0 (socket says 172.21.15.94:51560) 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 -- 172.21.15.94:0/3214885361 learned_addr learned my addr 172.21.15.94:0/3214885361 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf468bc640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf4f1ce740 10 client.0 _ll_register_callbacks cb 0x55614b5d2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 -- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 msgr2=0x55614b6d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 -- 172.21.15.94:0/3214885361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 msgr2=0x55614b6dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.855+0000 7fcf460bb640 1 -- 172.21.15.94:0/3214885361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614b3413f0 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf460bb640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009830 tx=0x7fcf3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4668289d26e56333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c04c030 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf3c0305c0 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c038040 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c041040 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 10 client.19596 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55614b40b7d0 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c038040 con 0x55614b3004d0 2024-04-25T13:53:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcf3c0047c0 con 0x55614b3004d0 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 10 client.19596 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 10 client.19596 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 20 client.19596 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b2e7f60 0x55614b6e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 4 client.19596 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.856+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614b63d990 con 0x55614b2e7f60 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.857+0000 7fcf458ba640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b2e7f60 0x55614b6e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.857+0000 7fcf29ffb640 10 client.19596 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.857+0000 7fcf458ba640 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b2e7f60 0x55614b6e2e90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a74e22cf587cb64 server_cookie=ee0c4615176f999 in_seq=0 out_seq=0 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf29ffb640 1 -- 172.21.15.94:0/3214885361 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55614b63d990 con 0x55614b2e7f60 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf29ffb640 10 client.19596 handle_command_reply: tid=0 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 client.19596 shutdown 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 msgr2=0x55614b6d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009830 tx=0x7fcf3c004270 comp rx=0 tx=0).stop 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 shutdown_connections 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b2e7f60 0x55614b6e2e90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b329390 0x55614b6dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf29ffb640 0 client.19596 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/3214885361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6d55c0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 >> 172.21.15.94:0/3214885361 conn(0x55614b31d4e0 msgr2=0x55614b6c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 shutdown_connections 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/3214885361 wait complete. 2024-04-25T13:53:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.858+0000 7fcf4f1ce740 20 client.19596 trim_cache size 0 max 16384 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 Processor -- start 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 -- start start 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b2e7f60 0x55614b6d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c2af0 con 0x55614b3004d0 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c4630 con 0x55614b2e7f60 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c31c0 con 0x55614b3086b0 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51572/0 (socket says 172.21.15.94:51572) 2024-04-25T13:53:16.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 -- 172.21.15.94:0/1517931999 learned_addr learned my addr 172.21.15.94:0/1517931999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf460bb640 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b2e7f60 0x55614b6d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf468bc640 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 -- 172.21.15.94:0/1517931999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 msgr2=0x55614b6c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 -- 172.21.15.94:0/1517931999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b2e7f60 msgr2=0x55614b6d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b2e7f60 0x55614b6d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 -- 172.21.15.94:0/1517931999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614b6dcec0 con 0x55614b3004d0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.872+0000 7fcf458ba640 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fcf380a21f0 tx=0x7fcf380c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75f75bedadc9617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf2bfff640 1 -- 172.21.15.94:0/1517931999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380a6240 con 0x55614b3004d0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf2bfff640 1 -- 172.21.15.94:0/1517931999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf38014730 con 0x55614b3004d0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf2bfff640 1 -- 172.21.15.94:0/1517931999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380a6060 con 0x55614b3004d0 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1517931999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 msgr2=0x55614b6dc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fcf380a21f0 tx=0x7fcf380c9040 comp rx=0 tx=0).stop 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1517931999 shutdown_connections 2024-04-25T13:53:16.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3086b0 0x55614b6c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b3004d0 0x55614b6dc980 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/1517931999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b2e7f60 0x55614b6d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1517931999 >> 172.21.15.94:0/1517931999 conn(0x55614b31d4e0 msgr2=0x55614b6d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1517931999 shutdown_connections 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.873+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/1517931999 wait complete. 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 -- start start 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 Processor -- start 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b3086b0 0x55614b7491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c3f60 con 0x55614b2e7f60 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c3890 con 0x55614b3086b0 2024-04-25T13:53:16.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf401c0ad0 con 0x55614b3004d0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf4f1ce740 10 client.0 _ll_register_callbacks cb 0x7fcf30005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51588/0 (socket says 172.21.15.94:51588) 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 -- 172.21.15.94:0/2950852898 learned_addr learned my addr 172.21.15.94:0/2950852898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf468bc640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b3086b0 0x55614b7491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 -- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 msgr2=0x55614b6d9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6d9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 -- 172.21.15.94:0/2950852898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b3086b0 msgr2=0x55614b7491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b3086b0 0x55614b7491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.874+0000 7fcf460bb640 1 -- 172.21.15.94:0/2950852898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614b6dcab0 con 0x55614b2e7f60 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:16.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf460bb640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c004650 tx=0x7fcf3c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a615d8743016a515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c05f060 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf3c0048c0 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55614b637800 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c04c070 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 10 client.19602 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55614b74a5d0 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55614b637800 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcf3c003c20 con 0x55614b2e7f60 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 10 client.19602 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 10 client.19602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 20 client.19602 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b329390 0x55614b74cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:16.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 4 client.19602 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.875+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55614b5b2610 con 0x55614b329390 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b329390 0x55614b74cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf29ffb640 10 client.19602 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf458ba640 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b329390 0x55614b74cff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83a2778db02649f8 server_cookie=6134f774b3f1d121 in_seq=0 out_seq=0 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf29ffb640 1 -- 172.21.15.94:0/2950852898 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55614b5b2610 con 0x55614b329390 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf29ffb640 10 client.19602 handle_command_reply: tid=0 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 client.19602 shutdown 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 msgr2=0x55614b6d9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c004650 tx=0x7fcf3c005f10 comp rx=0 tx=0).stop 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 shutdown_connections 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55614b329390 0x55614b74cff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55614b3086b0 0x55614b7491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55614b3004d0 0x55614b6d9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 --2- 172.21.15.94:0/2950852898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55614b2e7f60 0x55614b6d9690 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 >> 172.21.15.94:0/2950852898 conn(0x55614b31d4e0 msgr2=0x55614b6d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf29ffb640 0 client.19602 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 shutdown_connections 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.876+0000 7fcf4f1ce740 1 -- 172.21.15.94:0/2950852898 wait complete. 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4f1ce740 20 client.19602 trim_cache size 0 max 16384 2024-04-25T13:53:16.884 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 msgr2=0x7fcf2c0b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 0x7fcf2c0b2190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55614b6cc860 tx=0x7fcf3000b2e0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 msgr2=0x7fcf40117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 0x7fcf40117610 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800be20 tx=0x7fcf38009f70 comp rx=0 tx=0).stop 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 shutdown_connections 2024-04-25T13:53:16.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf2c0afc80 0x7fcf2c0b2190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf40110030 0x7fcf40118570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf4010b310 0x7fcf40117610 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 --2- 172.21.15.94:0/2096885747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf4010a420 0x7fcf401170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 >> 172.21.15.94:0/2096885747 conn(0x7fcf400704e0 msgr2=0x7fcf4010c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 shutdown_connections 2024-04-25T13:53:16.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:16.877+0000 7fcf4831f640 1 -- 172.21.15.94:0/2096885747 wait complete. 2024-04-25T13:53:16.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:53:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 -- 172.21.15.94:0/3495497186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418102340 msgr2=0x7f94181090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 --2- 172.21.15.94:0/3495497186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418102340 0x7f94181090d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002a410 comp rx=0 tx=0).stop 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 -- 172.21.15.94:0/3495497186 shutdown_connections 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 --2- 172.21.15.94:0/3495497186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 0x7f941810bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 --2- 172.21.15.94:0/3495497186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418102340 0x7f94181090d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 --2- 172.21.15.94:0/3495497186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418101e40 0x7f9418106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 -- 172.21.15.94:0/3495497186 >> 172.21.15.94:0/3495497186 conn(0x7f94180fd350 msgr2=0x7f94180ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 -- 172.21.15.94:0/3495497186 shutdown_connections 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.010+0000 7f941d796640 1 -- 172.21.15.94:0/3495497186 wait complete. 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 Processor -- start 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 -- start start 2024-04-25T13:53:17.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418102340 0x7f941819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 0x7f94181a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f5b0 con 0x7f9418101e40 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f430 con 0x7f9418102340 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f941d796640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f941810f730 con 0x7f9418109760 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51612/0 (socket says 172.21.15.94:51612) 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f9416ffd640 1 -- 172.21.15.94:0/3657592059 learned_addr learned my addr 172.21.15.94:0/3657592059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.011+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418102340 0x7f941819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f94177fe640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 0x7f94181a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 -- 172.21.15.94:0/3657592059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 msgr2=0x7f94181a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 0x7f94181a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 -- 172.21.15.94:0/3657592059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418102340 msgr2=0x7f941819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418102340 0x7f941819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 -- 172.21.15.94:0/3657592059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a1d30 con 0x7f9418101e40 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9414ff9640 1 -- 172.21.15.94:0/3657592059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940800cd80 con 0x7f9418101e40 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9414ff9640 1 -- 172.21.15.94:0/3657592059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94080108e0 con 0x7f9418101e40 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a1fc0 con 0x7f9418101e40 2024-04-25T13:53:17.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f9414ff9640 1 -- 172.21.15.94:0/3657592059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408018980 con 0x7f9418101e40 2024-04-25T13:53:17.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.012+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a2480 con 0x7f9418101e40 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.014+0000 7f942464a740 1 Processor -- start 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 -- start start 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9414ff9640 1 -- 172.21.15.94:0/3657592059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9408010430 con 0x7f9418101e40 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 0x56332a39f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 0x56332a39fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56332a303d80 con 0x563329fdc6b0 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56332a2bda20 con 0x563329fd44d0 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f942464a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56332a2ffd40 con 0x563329ffd390 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 0x56332a39fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51624/0 (socket says 172.21.15.94:51624) 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f940e7fc640 1 -- 172.21.15.94:0/474747722 learned_addr learned my addr 172.21.15.94:0/474747722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f94177fe640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 0x56332a39f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 -- 172.21.15.94:0/474747722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 msgr2=0x56332a39f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 0x56332a39f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 -- 172.21.15.94:0/474747722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 msgr2=0x56332a3a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9416ffd640 1 -- 172.21.15.94:0/474747722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56332a2c8c30 con 0x563329fd44d0 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f9414ff9640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 0x7f94040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f940e7fc640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.015+0000 7f94177fe640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 0x56332a39f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f94177fe640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 0x7f94040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f9414ff9640 1 -- 172.21.15.94:0/3657592059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f94080def90 con 0x7f9418101e40 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f9416ffd640 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 0x56332a39fd90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94080dc9c0 tx=0x7f94080a3ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3651e0ed19c50686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f940d7fa640 1 -- 172.21.15.94:0/474747722 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080a7250 con 0x563329fd44d0 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f940d7fa640 1 -- 172.21.15.94:0/474747722 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9408010720 con 0x563329fd44d0 2024-04-25T13:53:17.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f94177fe640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 0x7f94040b2000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x56332a3a0860 tx=0x7f94100096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.016+0000 7f940d7fa640 1 -- 172.21.15.94:0/474747722 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080a6bf0 con 0x563329fd44d0 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- 172.21.15.94:0/474747722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 msgr2=0x56332a39fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 0x56332a39fd90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94080dc9c0 tx=0x7f94080a3ad0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- 172.21.15.94:0/474747722 shutdown_connections 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329fd44d0 0x56332a39fd90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 --2- 172.21.15.94:0/474747722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329ffd390 0x56332a39f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- 172.21.15.94:0/474747722 >> 172.21.15.94:0/474747722 conn(0x563329ff14e0 msgr2=0x56332a39b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- 172.21.15.94:0/474747722 shutdown_connections 2024-04-25T13:53:17.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- 172.21.15.94:0/474747722 wait complete. 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 -- start start 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.017+0000 7f942464a740 1 Processor -- start 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56332a3a1530 con 0x563329fdc6b0 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56332a28adc0 con 0x563329ffd390 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56332a3a16b0 con 0x563329fd44d0 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f942464a740 10 client.0 _ll_register_callbacks cb 0x56332a2a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f94177fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f94177fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34126/0 (socket says 172.21.15.94:34126) 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51628/0 (socket says 172.21.15.94:51628) 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f94177fe640 1 -- 172.21.15.94:0/3697036833 learned_addr learned my addr 172.21.15.94:0/3697036833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 -- 172.21.15.94:0/3697036833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 msgr2=0x56332a3a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 -- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 msgr2=0x56332a3b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 -- 172.21.15.94:0/3697036833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56332a0153f0 con 0x563329fdc6b0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f94177fe640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9400031440 tx=0x7f9400031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab4b706f201acf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.018+0000 7f93fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400035040 con 0x563329fdc6b0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9400004240 con 0x563329fdc6b0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56332a311990 con 0x563329fdc6b0 2024-04-25T13:53:17.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94000318f0 con 0x563329fdc6b0 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 10 client.19614 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56332a0df7d0 con 0x563329fdc6b0 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56332a311990 con 0x563329fdc6b0 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9400044020 con 0x563329fdc6b0 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 10 client.19614 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 10 client.19614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 20 client.19614 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329fbbf60 0x56332a3b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 4 client.19614 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56332a30b800 con 0x563329fbbf60 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.019+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329fbbf60 0x56332a3b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.020+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329fbbf60 0x56332a3b7680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b62196a7d5f32cd server_cookie=7508513b95faadfd in_seq=0 out_seq=0 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.020+0000 7f93fe7fc640 10 client.19614 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3697036833 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56332a30b800 con 0x563329fbbf60 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f93fe7fc640 10 client.19614 handle_command_reply: tid=0 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 client.19614 shutdown 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 msgr2=0x56332a3a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9400031440 tx=0x7f9400031e80 comp rx=0 tx=0).stop 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 shutdown_connections 2024-04-25T13:53:17.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329fbbf60 0x56332a3b7680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3a9b00 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 --2- 172.21.15.94:0/3697036833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f93fe7fc640 0 client.19614 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 >> 172.21.15.94:0/3697036833 conn(0x563329ff14e0 msgr2=0x56332a39df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 shutdown_connections 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 1 -- 172.21.15.94:0/3697036833 wait complete. 2024-04-25T13:53:17.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.021+0000 7f942464a740 20 client.19614 trim_cache size 0 max 16384 2024-04-25T13:53:17.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.034+0000 7f942464a740 1 Processor -- start 2024-04-25T13:53:17.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.034+0000 7f942464a740 1 -- start start 2024-04-25T13:53:17.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ab510 con 0x563329fdc6b0 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ad050 con 0x563329ffd390 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f942464a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94181abbe0 con 0x563329fd44d0 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51642/0 (socket says 172.21.15.94:51642) 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 -- 172.21.15.94:0/68269412 learned_addr learned my addr 172.21.15.94:0/68269412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f9416ffd640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f94177fe640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 -- 172.21.15.94:0/68269412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 msgr2=0x56332a3a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 -- 172.21.15.94:0/68269412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 msgr2=0x56332a3a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 -- 172.21.15.94:0/68269412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56332a3a76a0 con 0x563329fdc6b0 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f94177fe640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940e7fc640 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d38533aa4e15acf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.035+0000 7f940cff9640 1 -- 172.21.15.94:0/68269412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400035040 con 0x563329fdc6b0 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f940cff9640 1 -- 172.21.15.94:0/68269412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9400055020 con 0x563329fdc6b0 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f940cff9640 1 -- 172.21.15.94:0/68269412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400003850 con 0x563329fdc6b0 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 -- 172.21.15.94:0/68269412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 msgr2=0x56332a3b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002a410 comp rx=0 tx=0).stop 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 -- 172.21.15.94:0/68269412 shutdown_connections 2024-04-25T13:53:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0980 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 --2- 172.21.15.94:0/68269412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 -- 172.21.15.94:0/68269412 >> 172.21.15.94:0/68269412 conn(0x563329ff14e0 msgr2=0x56332a3a95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.036+0000 7f942464a740 1 -- 172.21.15.94:0/68269412 shutdown_connections 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 -- 172.21.15.94:0/68269412 wait complete. 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 -- start start 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 Processor -- start 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ac980 con 0x563329fdc6b0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ac2b0 con 0x563329ffd390 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94181a94f0 con 0x563329fd44d0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f942464a740 10 client.0 _ll_register_callbacks cb 0x7f9400009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f940e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51652/0 (socket says 172.21.15.94:51652) 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f940e7fc640 1 -- 172.21.15.94:0/3208925220 learned_addr learned my addr 172.21.15.94:0/3208925220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f94177fe640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.037+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 -- 172.21.15.94:0/3208925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 msgr2=0x56332a3b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 -- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 msgr2=0x56332a3acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 -- 172.21.15.94:0/3208925220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56332a3ad690 con 0x563329fdc6b0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f940e7fc640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9400009850 tx=0x7f9400031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f553f378e025a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940005d020 con 0x563329fdc6b0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9400050ce0 con 0x563329fdc6b0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9400055020 con 0x563329fdc6b0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9400035040 con 0x563329fdc6b0 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 10 client.19620 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56332a41e690 con 0x563329fdc6b0 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9400055020 con 0x563329fdc6b0 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9400044040 con 0x563329fdc6b0 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 10 client.19620 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 10 client.19620 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 20 client.19620 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329f913a0 0x56332a4210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 4 client.19620 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.038+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56332a286610 con 0x563329f913a0 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329f913a0 0x56332a4210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f9416ffd640 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329f913a0 0x56332a4210a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73709290798c0ded server_cookie=f662b26339d44a21 in_seq=0 out_seq=0 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f93fe7fc640 10 client.19620 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f93fe7fc640 1 -- 172.21.15.94:0/3208925220 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56332a286610 con 0x563329f913a0 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f93fe7fc640 10 client.19620 handle_command_reply: tid=0 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 client.19620 shutdown 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 msgr2=0x56332a3b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9400009850 tx=0x7f9400031b90 comp rx=0 tx=0).stop 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 shutdown_connections 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563329f913a0 0x56332a4210a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563329ffd390 0x56332a3acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f93fe7fc640 0 client.19620 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563329fdc6b0 0x56332a3b0860 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 --2- 172.21.15.94:0/3208925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563329fd44d0 0x56332a3b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.039+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 >> 172.21.15.94:0/3208925220 conn(0x563329ff14e0 msgr2=0x56332a3a95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 shutdown_connections 2024-04-25T13:53:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f942464a740 1 -- 172.21.15.94:0/3208925220 wait complete. 2024-04-25T13:53:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f942464a740 20 client.19620 trim_cache size 0 max 16384 2024-04-25T13:53:17.050 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 msgr2=0x7f94040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 0x7f94040b2000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x56332a3a0860 tx=0x7f94100096a0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 msgr2=0x7f941819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9f0 tx=0x7f940800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 shutdown_connections 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94040afaf0 0x7f94040b2000 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418109760 0x7f94181a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9418102340 0x7f941819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 --2- 172.21.15.94:0/3657592059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9418101e40 0x7f941819a1f0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 >> 172.21.15.94:0/3657592059 conn(0x7f94180fd350 msgr2=0x7f941810a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 shutdown_connections 2024-04-25T13:53:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.040+0000 7f941d796640 1 -- 172.21.15.94:0/3657592059 wait complete. 2024-04-25T13:53:17.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 -- 172.21.15.94:0/2114241207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8104480 msgr2=0x7efca810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 --2- 172.21.15.94:0/2114241207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8104480 0x7efca810abc0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7efca0009ca0 tx=0x7efca002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 -- 172.21.15.94:0/2114241207 shutdown_connections 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 --2- 172.21.15.94:0/2114241207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8104480 0x7efca810abc0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 --2- 172.21.15.94:0/2114241207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8103940 0x7efca8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 --2- 172.21.15.94:0/2114241207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8102f60 0x7efca8103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 -- 172.21.15.94:0/2114241207 >> 172.21.15.94:0/2114241207 conn(0x7efca80fd380 msgr2=0x7efca80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 -- 172.21.15.94:0/2114241207 shutdown_connections 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 -- 172.21.15.94:0/2114241207 wait complete. 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.169+0000 7efcaddce640 1 Processor -- start 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 -- start start 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 0x7efca8102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 0x7efca81a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efca810e550 con 0x7efca8103940 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efca810e3d0 con 0x7efca8102f60 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efcaddce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca810e6d0 con 0x7efca8104480 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efc9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efc9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51678/0 (socket says 172.21.15.94:51678) 2024-04-25T13:53:17.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 0x7efca8102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efc9ffff640 1 -- 172.21.15.94:0/3940152380 learned_addr learned my addr 172.21.15.94:0/3940152380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca7fff640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 0x7efca81a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 -- 172.21.15.94:0/3940152380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 msgr2=0x7efca81a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 0x7efca81a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 -- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 msgr2=0x7efca819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 -- 172.21.15.94:0/3940152380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca81a1d80 con 0x7efca8102f60 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca7fff640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 0x7efca81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efc9ffff640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.170+0000 7efca77fe640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 0x7efca8102220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efca806b2d0 tx=0x7efc9800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.171+0000 7efca57fa640 1 -- 172.21.15.94:0/3940152380 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800cc40 con 0x7efca8102f60 2024-04-25T13:53:17.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.171+0000 7efca57fa640 1 -- 172.21.15.94:0/3940152380 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9800cde0 con 0x7efca8102f60 2024-04-25T13:53:17.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.171+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81a2070 con 0x7efca8102f60 2024-04-25T13:53:17.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.171+0000 7efca57fa640 1 -- 172.21.15.94:0/3940152380 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98004a00 con 0x7efca8102f60 2024-04-25T13:53:17.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.171+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a2570 con 0x7efca8102f60 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.173+0000 7efcb4c7e740 1 Processor -- start 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.173+0000 7efcb4c7e740 1 -- start start 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.173+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94b94d0 0x55bfc98891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94c16b0 0x55bfc9889e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bfc97e8d80 con 0x55bfc94e2390 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfc97a2a20 con 0x55bfc94c16b0 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfc97e4d40 con 0x55bfc94b94d0 2024-04-25T13:53:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51690/0 (socket says 172.21.15.94:51690) 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 -- 172.21.15.94:0/1011976329 learned_addr learned my addr 172.21.15.94:0/1011976329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efc9ffff640 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94c16b0 0x55bfc9889e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 -- 172.21.15.94:0/1011976329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94b94d0 msgr2=0x55bfc98891a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94b94d0 0x55bfc98891a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 -- 172.21.15.94:0/1011976329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94c16b0 msgr2=0x55bfc9889e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94c16b0 0x55bfc9889e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 -- 172.21.15.94:0/1011976329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc97adc30 con 0x55bfc94e2390 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca57fa640 1 -- 172.21.15.94:0/3940152380 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efc98004ba0 con 0x7efca8102f60 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efca7fff640 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7efca0035000 tx=0x7efca002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726fd3fa4dce338a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efc9dffb640 1 -- 172.21.15.94:0/1011976329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0031450 con 0x55bfc94e2390 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.174+0000 7efc9dffb640 1 -- 172.21.15.94:0/1011976329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca0036040 con 0x55bfc94e2390 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efc9dffb640 1 -- 172.21.15.94:0/1011976329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0004610 con 0x55bfc94e2390 2024-04-25T13:53:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efca57fa640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 0x7efc940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efca57fa640 1 -- 172.21.15.94:0/3940152380 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7efc980dec10 con 0x7efca8102f60 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efca77fe640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 0x7efc940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1011976329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 msgr2=0x55bfc9888c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7efca0035000 tx=0x7efca002f410 comp rx=0 tx=0).stop 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efca77fe640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 0x7efc940b2000 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc980def70 tx=0x7efc9800cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1011976329 shutdown_connections 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94c16b0 0x55bfc9889e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94b94d0 0x55bfc98891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1011976329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94e2390 0x55bfc9888c20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1011976329 >> 172.21.15.94:0/1011976329 conn(0x55bfc94d64e0 msgr2=0x55bfc9880020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.175+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1011976329 shutdown_connections 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1011976329 wait complete. 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 -- start start 2024-04-25T13:53:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 Processor -- start 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94b94d0 0x55bfc9892880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94e2390 0x55bfc9899cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bfc988a760 con 0x55bfc94c16b0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfc976fdc0 con 0x55bfc94b94d0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfc988a8e0 con 0x55bfc94e2390 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efcb4c7e740 10 client.0 _ll_register_callbacks cb 0x55bfc978b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efc9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efc9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51692/0 (socket says 172.21.15.94:51692) 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efc9ffff640 1 -- 172.21.15.94:0/1227799573 learned_addr learned my addr 172.21.15.94:0/1227799573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.176+0000 7efca7fff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94b94d0 0x55bfc9892880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efca7fff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94e2390 0x55bfc9899cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 -- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94e2390 msgr2=0x55bfc9899cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94e2390 0x55bfc9899cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 -- 172.21.15.94:0/1227799573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94b94d0 msgr2=0x55bfc9892880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94b94d0 0x55bfc9892880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 -- 172.21.15.94:0/1227799573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc94fa3f0 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc9ffff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efc90012d20 tx=0x7efc9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0343673a7138d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90012110 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7efc90004050 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca0036040 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efc900122b0 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efcb4c7e740 10 client.19632 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfc95c47d0 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.177+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca0036040 con 0x55bfc94c16b0 2024-04-25T13:53:17.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7efc90010430 con 0x55bfc94c16b0 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 10 client.19632 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 10 client.19632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 20 client.19632 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94a0f60 0x55bfc98a0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 4 client.19632 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfc97f6990 con 0x55bfc94a0f60 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efca7fff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94a0f60 0x55bfc98a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efc86ffd640 10 client.19632 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.178+0000 7efca7fff640 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94a0f60 0x55bfc98a0510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0137b2cfdacb5df server_cookie=eeac5b1a30746a78 in_seq=0 out_seq=0 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efc86ffd640 1 -- 172.21.15.94:0/1227799573 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfc97f6990 con 0x55bfc94a0f60 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efc86ffd640 10 client.19632 handle_command_reply: tid=0 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 client.19632 shutdown 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 msgr2=0x55bfc9892dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efc90012d20 tx=0x7efc9000f790 comp rx=0 tx=0).stop 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 shutdown_connections 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94a0f60 0x55bfc98a0510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94e2390 0x55bfc9899cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94c16b0 0x55bfc9892dc0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/1227799573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94b94d0 0x55bfc9892880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 >> 172.21.15.94:0/1227799573 conn(0x55bfc94d64e0 msgr2=0x55bfc987f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efc86ffd640 0 client.19632 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 shutdown_connections 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 1 -- 172.21.15.94:0/1227799573 wait complete. 2024-04-25T13:53:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.179+0000 7efcb4c7e740 20 client.19632 trim_cache size 0 max 16384 2024-04-25T13:53:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 Processor -- start 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 -- start start 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc987ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc988d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a92e0 con 0x55bfc94b94d0 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efca81aae20 con 0x55bfc94e2390 2024-04-25T13:53:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a99b0 con 0x55bfc94c16b0 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51694/0 (socket says 172.21.15.94:51694) 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 -- 172.21.15.94:0/840817164 learned_addr learned my addr 172.21.15.94:0/840817164 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efc9ffff640 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc987ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc988d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 -- 172.21.15.94:0/840817164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 msgr2=0x55bfc987ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc987ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 -- 172.21.15.94:0/840817164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 msgr2=0x55bfc988d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc988d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.193+0000 7efca7fff640 1 -- 172.21.15.94:0/840817164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc988a670 con 0x55bfc94b94d0 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efca7fff640 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7efca0009a60 tx=0x7efca002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d57f1d37c2c87fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efc9d7fa640 1 -- 172.21.15.94:0/840817164 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca005d030 con 0x55bfc94b94d0 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efc9d7fa640 1 -- 172.21.15.94:0/840817164 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca0036040 con 0x55bfc94b94d0 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efc9d7fa640 1 -- 172.21.15.94:0/840817164 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0004300 con 0x55bfc94b94d0 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efcb4c7e740 1 -- 172.21.15.94:0/840817164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 msgr2=0x55bfc987fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.194+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7efca0009a60 tx=0x7efca002f3a0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- 172.21.15.94:0/840817164 shutdown_connections 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc988d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc987ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/840817164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc987fa50 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- 172.21.15.94:0/840817164 >> 172.21.15.94:0/840817164 conn(0x55bfc94d64e0 msgr2=0x55bfc987ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- 172.21.15.94:0/840817164 shutdown_connections 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- 172.21.15.94:0/840817164 wait complete. 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- start start 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 Processor -- start 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc9895e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc98964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efca81aa750 con 0x55bfc94b94d0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efca81aa080 con 0x55bfc94e2390 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca81a72c0 con 0x55bfc94c16b0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efcb4c7e740 10 client.0 _ll_register_callbacks cb 0x7efc90009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.195+0000 7efca7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51700/0 (socket says 172.21.15.94:51700) 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 -- 172.21.15.94:0/2766957322 learned_addr learned my addr 172.21.15.94:0/2766957322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc98964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc9ffff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc9895e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 -- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 msgr2=0x55bfc9895e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc9895e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 -- 172.21.15.94:0/2766957322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 msgr2=0x55bfc98964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc98964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 -- 172.21.15.94:0/2766957322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc99068f0 con 0x55bfc94b94d0 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc9ffff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc9895e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efca7fff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efca0006010 tx=0x7efca002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390eea7120489723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca005d040 con 0x55bfc94b94d0 2024-04-25T13:53:17.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7efca0003970 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca0036040 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efca0003b70 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efcb4c7e740 10 client.19638 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfc9907b80 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.196+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca0036040 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7efca0051040 con 0x55bfc94b94d0 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 10 client.19638 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 10 client.19638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 20 client.19638 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94763a0 0x55bfc990a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 4 client.19638 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfc97f0800 con 0x55bfc94763a0 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efc9ffff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94763a0 0x55bfc990a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efc86ffd640 10 client.19638 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.197+0000 7efc9ffff640 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94763a0 0x55bfc990a260 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44c8542f64fdafbc server_cookie=52ea8c2dfd896b0 in_seq=0 out_seq=0 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efc86ffd640 1 -- 172.21.15.94:0/2766957322 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfc97f0800 con 0x55bfc94763a0 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efc86ffd640 10 client.19638 handle_command_reply: tid=0 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 client.19638 shutdown 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 msgr2=0x55bfc98992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efca0006010 tx=0x7efca002f410 comp rx=0 tx=0).stop 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 shutdown_connections 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55bfc94763a0 0x55bfc990a260 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfc94e2390 0x55bfc98964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efc86ffd640 0 client.19638 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfc94c16b0 0x55bfc9895e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 --2- 172.21.15.94:0/2766957322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bfc94b94d0 0x55bfc98992e0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 >> 172.21.15.94:0/2766957322 conn(0x55bfc94d64e0 msgr2=0x55bfc9890e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 shutdown_connections 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 1 -- 172.21.15.94:0/2766957322 wait complete. 2024-04-25T13:53:17.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcb4c7e740 20 client.19638 trim_cache size 0 max 16384 2024-04-25T13:53:17.211 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 msgr2=0x7efc940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 0x7efc940b2000 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc980def70 tx=0x7efc9800cf90 comp rx=0 tx=0).stop 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 msgr2=0x7efca8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 0x7efca8102220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efca806b2d0 tx=0x7efc9800ee50 comp rx=0 tx=0).stop 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 shutdown_connections 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efc940afaf0 0x7efc940b2000 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8104480 0x7efca81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efca8103940 0x7efca819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 --2- 172.21.15.94:0/3940152380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efca8102f60 0x7efca8102220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.198+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 >> 172.21.15.94:0/3940152380 conn(0x7efca80fd380 msgr2=0x7efca8106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.199+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 shutdown_connections 2024-04-25T13:53:17.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.199+0000 7efcaddce640 1 -- 172.21.15.94:0/3940152380 wait complete. 2024-04-25T13:53:17.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 -- 172.21.15.94:0/2975170590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c1097a0 msgr2=0x7f7d3c10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 --2- 172.21.15.94:0/2975170590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c10bcb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009ca0 tx=0x7f7d3402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 -- 172.21.15.94:0/2975170590 shutdown_connections 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 --2- 172.21.15.94:0/2975170590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c10bcb0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 --2- 172.21.15.94:0/2975170590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 0x7f7d3c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 --2- 172.21.15.94:0/2975170590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c101e50 0x7f7d3c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 -- 172.21.15.94:0/2975170590 >> 172.21.15.94:0/2975170590 conn(0x7f7d3c0fd380 msgr2=0x7f7d3c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 -- 172.21.15.94:0/2975170590 shutdown_connections 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.330+0000 7f7d409c5640 1 -- 172.21.15.94:0/2975170590 wait complete. 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 Processor -- start 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 -- start start 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 0x7f7d3c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10f630 con 0x7f7d3c101e50 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10f4b0 con 0x7f7d3c1097a0 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d409c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10f7b0 con 0x7f7d3c106c30 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51718/0 (socket says 172.21.15.94:51718) 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 -- 172.21.15.94:0/435551192 learned_addr learned my addr 172.21.15.94:0/435551192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d33fff640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 0x7f7d3c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 -- 172.21.15.94:0/435551192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 msgr2=0x7f7d3c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 0x7f7d3c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 -- 172.21.15.94:0/435551192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c1097a0 msgr2=0x7f7d3c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 -- 172.21.15.94:0/435551192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1a1ce0 con 0x7f7d3c101e50 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.331+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9c0 tx=0x7f7d2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.332+0000 7f7d337fe640 1 -- 172.21.15.94:0/435551192 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00cd80 con 0x7f7d3c101e50 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.332+0000 7f7d337fe640 1 -- 172.21.15.94:0/435551192 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2c0054d0 con 0x7f7d3c101e50 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.332+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1a1fd0 con 0x7f7d3c101e50 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.332+0000 7f7d337fe640 1 -- 172.21.15.94:0/435551192 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c010430 con 0x7f7d3c101e50 2024-04-25T13:53:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.332+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c102b30 con 0x7f7d3c101e50 2024-04-25T13:53:17.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.334+0000 7f7d47878740 1 Processor -- start 2024-04-25T13:53:17.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 -- start start 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d337fe640 1 -- 172.21.15.94:0/435551192 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7d2c004fd0 con 0x7f7d3c101e50 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9ed3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9e36d80 con 0x55e2b9b30390 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9df0a20 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d47878740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9e32d40 con 0x55e2b9b074d0 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9ed3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51722/0 (socket says 172.21.15.94:51722) 2024-04-25T13:53:17.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d3ad9d640 1 -- 172.21.15.94:0/668188117 learned_addr learned my addr 172.21.15.94:0/668188117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 -- 172.21.15.94:0/668188117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 msgr2=0x55e2b9ed2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 -- 172.21.15.94:0/668188117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 msgr2=0x55e2b9ed2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d33fff640 1 -- 172.21.15.94:0/668188117 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b9dfbc30 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.335+0000 7f7d337fe640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 0x7f7d280b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 0x7f7d280b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d33fff640 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9ed3a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009b40 tx=0x7f7d240113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca6f79398f666f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d337fe640 1 -- 172.21.15.94:0/435551192 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7d2c0df140 con 0x7f7d3c101e50 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d30ff9640 1 -- 172.21.15.94:0/668188117 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24011e10 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d30ff9640 1 -- 172.21.15.94:0/668188117 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d24012900 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.336+0000 7f7d30ff9640 1 -- 172.21.15.94:0/668188117 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400f9a0 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- 172.21.15.94:0/668188117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 msgr2=0x55e2b9ed3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 0x7f7d280b6410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009a60 tx=0x7f7d3402f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9ed3a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009b40 tx=0x7f7d240113e0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- 172.21.15.94:0/668188117 shutdown_connections 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9ed3a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- 172.21.15.94:0/668188117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9ed2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- 172.21.15.94:0/668188117 >> 172.21.15.94:0/668188117 conn(0x55e2b9b244e0 msgr2=0x55e2b9ece020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- 172.21.15.94:0/668188117 shutdown_connections 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- 172.21.15.94:0/668188117 wait complete. 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 -- start start 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 Processor -- start 2024-04-25T13:53:17.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b074d0 0x55e2b9edc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.337+0000 7f7d47878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9ee39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9ed4530 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9dbddc0 con 0x55e2b9b074d0 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2b9ed46b0 con 0x55e2b9b30390 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 10 client.0 _ll_register_callbacks cb 0x55e2b9dd9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51724/0 (socket says 172.21.15.94:51724) 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 -- 172.21.15.94:0/1405852207 learned_addr learned my addr 172.21.15.94:0/1405852207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b074d0 0x55e2b9edc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9ee39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 -- 172.21.15.94:0/1405852207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 msgr2=0x55e2b9ee39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9ee39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 -- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b074d0 msgr2=0x55e2b9edc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b074d0 0x55e2b9edc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 -- 172.21.15.94:0/1405852207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b9b483f0 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d33fff640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f7d24012700 tx=0x7f7d24012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffecd2d62a5c8e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24028020 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7d24005070 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d24012900 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d24033050 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 10 client.19650 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.338+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2b9c127d0 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d24012900 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7d240035f0 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 10 client.19650 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 10 client.19650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 20 client.19650 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9aeef60 0x55e2b9ee9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 4 client.19650 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2b9e44990 con 0x55e2b9aeef60 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.339+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9aeef60 0x55e2b9ee9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d21ffb640 10 client.19650 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9aeef60 0x55e2b9ee9fe0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abe4f6ed5a8da6aa server_cookie=3d6b070585889a63 in_seq=0 out_seq=0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d21ffb640 1 -- 172.21.15.94:0/1405852207 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2b9e44990 con 0x55e2b9aeef60 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d21ffb640 10 client.19650 handle_command_reply: tid=0 2024-04-25T13:53:17.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 client.19650 shutdown 2024-04-25T13:53:17.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 msgr2=0x55e2b9edca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f7d24012700 tx=0x7f7d24012730 comp rx=0 tx=0).stop 2024-04-25T13:53:17.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 shutdown_connections 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9aeef60 0x55e2b9ee9fe0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9ee39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9edca90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 --2- 172.21.15.94:0/1405852207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b074d0 0x55e2b9edc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d21ffb640 0 client.19650 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.340+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 >> 172.21.15.94:0/1405852207 conn(0x55e2b9b244e0 msgr2=0x55e2b9ed0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.341+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 shutdown_connections 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.341+0000 7f7d47878740 1 -- 172.21.15.94:0/1405852207 wait complete. 2024-04-25T13:53:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.341+0000 7f7d47878740 20 client.19650 trim_cache size 0 max 16384 2024-04-25T13:53:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.354+0000 7f7d47878740 1 Processor -- start 2024-04-25T13:53:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.354+0000 7f7d47878740 1 -- start start 2024-04-25T13:53:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.354+0000 7f7d47878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.354+0000 7f7d47878740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9eee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d47878740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9eeed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d47878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1a9220 con 0x55e2b9b074d0 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d47878740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1aad60 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d47878740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1a98f0 con 0x55e2b9b30390 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51730/0 (socket says 172.21.15.94:51730) 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 -- 172.21.15.94:0/2829573651 learned_addr learned my addr 172.21.15.94:0/2829573651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9eeed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d33fff640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9eee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 -- 172.21.15.94:0/2829573651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 msgr2=0x55e2b9eeed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9eeed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 -- 172.21.15.94:0/2829573651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 msgr2=0x55e2b9eee7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9eee7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 -- 172.21.15.94:0/2829573651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b9edd150 con 0x55e2b9b074d0 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d33fff640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9eee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0057a0 tx=0x7f7d2c0049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55946c123036133b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d23fff640 1 -- 172.21.15.94:0/2829573651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0046c0 con 0x55e2b9b074d0 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d23fff640 1 -- 172.21.15.94:0/2829573651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2c0900b0 con 0x55e2b9b074d0 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.355+0000 7f7d23fff640 1 -- 172.21.15.94:0/2829573651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b03e0 con 0x55e2b9b074d0 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- 172.21.15.94:0/2829573651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 msgr2=0x55e2b9ed29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0057a0 tx=0x7f7d2c0049b0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- 172.21.15.94:0/2829573651 shutdown_connections 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b30390 0x55e2b9eeed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9eee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 --2- 172.21.15.94:0/2829573651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b074d0 0x55e2b9ed29c0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- 172.21.15.94:0/2829573651 >> 172.21.15.94:0/2829573651 conn(0x55e2b9b244e0 msgr2=0x55e2b9ed7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- 172.21.15.94:0/2829573651 shutdown_connections 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- 172.21.15.94:0/2829573651 wait complete. 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.356+0000 7f7d47878740 1 -- start start 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 Processor -- start 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9f4c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9f4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1aa690 con 0x55e2b9b30390 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1a9fc0 con 0x55e2b9b0f6b0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1a7200 con 0x55e2b9b074d0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d47878740 10 client.0 _ll_register_callbacks cb 0x7f7d24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51742/0 (socket says 172.21.15.94:51742) 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9f4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 -- 172.21.15.94:0/4089564072 learned_addr learned my addr 172.21.15.94:0/4089564072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9f4c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 -- 172.21.15.94:0/4089564072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 msgr2=0x55e2b9f4c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9f4c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 -- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 msgr2=0x55e2b9f4e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9f4e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 -- 172.21.15.94:0/4089564072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b9f51370 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d3ad9d640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7d34004af0 tx=0x7f7d340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfd4dfcb6236daf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.357+0000 7f7d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34057060 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7d34004310 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2b9e3e800 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d34004510 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 10 client.19656 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2b9f52660 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e2b9e3e800 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7d340305c0 con 0x55e2b9b30390 2024-04-25T13:53:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 10 client.19656 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 10 client.19656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 20 client.19656 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9ac43a0 0x55e2b9f550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 4 client.19656 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2b9db9610 con 0x55e2b9ac43a0 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.358+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9ac43a0 0x55e2b9f550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d3a59c640 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9ac43a0 0x55e2b9f550b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9935f1df645fb1ce server_cookie=9c7f39b6d751bff8 in_seq=0 out_seq=0 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d21ffb640 10 client.19656 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d21ffb640 1 -- 172.21.15.94:0/4089564072 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2b9db9610 con 0x55e2b9ac43a0 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d21ffb640 10 client.19656 handle_command_reply: tid=0 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 client.19656 shutdown 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 msgr2=0x55e2b9f50e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7d34004af0 tx=0x7f7d340095c0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 shutdown_connections 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e2b9ac43a0 0x55e2b9f550b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e2b9b30390 0x55e2b9f50e00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2b9b0f6b0 0x55e2b9f4e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d21ffb640 0 client.19656 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 --2- 172.21.15.94:0/4089564072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2b9b074d0 0x55e2b9f4c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 >> 172.21.15.94:0/4089564072 conn(0x55e2b9b244e0 msgr2=0x55e2b9ed0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 shutdown_connections 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 1 -- 172.21.15.94:0/4089564072 wait complete. 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.359+0000 7f7d47878740 20 client.19656 trim_cache size 0 max 16384 2024-04-25T13:53:17.367 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:17.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 msgr2=0x7f7d280b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 0x7f7d280b6410 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009a60 tx=0x7f7d3402f310 comp rx=0 tx=0).stop 2024-04-25T13:53:17.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 msgr2=0x7f7d3c19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9c0 tx=0x7f7d2c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 shutdown_connections 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d280b3f00 0x7f7d280b6410 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d3c1097a0 0x7f7d3c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c106c30 0x7f7d3c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 --2- 172.21.15.94:0/435551192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d3c101e50 0x7f7d3c19a1a0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 >> 172.21.15.94:0/435551192 conn(0x7f7d3c0fd380 msgr2=0x7f7d3c107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 shutdown_connections 2024-04-25T13:53:17.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.360+0000 7f7d409c5640 1 -- 172.21.15.94:0/435551192 wait complete. 2024-04-25T13:53:17.369 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/698419876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:17.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- 172.21.15.94:0/3413954303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90541027b0 msgr2=0x7f9054102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 --2- 172.21.15.94:0/3413954303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90541027b0 0x7f9054102c80 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f9044009900 tx=0x7f904402a410 comp rx=0 tx=0).stop 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- 172.21.15.94:0/3413954303 shutdown_connections 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 --2- 172.21.15.94:0/3413954303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 0x7f9054107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 --2- 172.21.15.94:0/3413954303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90541027b0 0x7f9054102c80 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 --2- 172.21.15.94:0/3413954303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9054101d70 0x7f90541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- 172.21.15.94:0/3413954303 >> 172.21.15.94:0/3413954303 conn(0x7f90540fd340 msgr2=0x7f90540ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- 172.21.15.94:0/3413954303 shutdown_connections 2024-04-25T13:53:17.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- 172.21.15.94:0/3413954303 wait complete. 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 Processor -- start 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.485+0000 7f905b3b4640 1 -- start start 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90541027b0 0x7f90541a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 0x7f90541a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9054115bf0 con 0x7f9054101d70 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9054115a70 con 0x7f9054107920 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9054115d70 con 0x7f90541027b0 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51760/0 (socket says 172.21.15.94:51760) 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 -- 172.21.15.94:0/2169326192 learned_addr learned my addr 172.21.15.94:0/2169326192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905894f640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90541027b0 0x7f90541a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059951640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 0x7f90541a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 -- 172.21.15.94:0/2169326192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90541027b0 msgr2=0x7f90541a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90541027b0 0x7f90541a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 -- 172.21.15.94:0/2169326192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 msgr2=0x7f90541a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 0x7f90541a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 -- 172.21.15.94:0/2169326192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90541a8340 con 0x7f9054101d70 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f9059150640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f905000eb90 tx=0x7f905000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f904a7fc640 1 -- 172.21.15.94:0/2169326192 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050004d20 con 0x7f9054101d70 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f904a7fc640 1 -- 172.21.15.94:0/2169326192 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905000ee90 con 0x7f9054101d70 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90541a8630 con 0x7f9054101d70 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f904a7fc640 1 -- 172.21.15.94:0/2169326192 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050010670 con 0x7f9054101d70 2024-04-25T13:53:17.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.486+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a8b30 con 0x7f9054101d70 2024-04-25T13:53:17.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.489+0000 7f9062261740 1 Processor -- start 2024-04-25T13:53:17.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.489+0000 7f9062261740 1 -- start start 2024-04-25T13:53:17.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d58d4d0 0x55706d958d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d959a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55706d8bcd80 con 0x55706d5b6390 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55706d876a20 con 0x55706d5956b0 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9062261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55706d8b8d40 con 0x55706d58d4d0 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f904a7fc640 1 -- 172.21.15.94:0/2169326192 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9050005020 con 0x7f9054101d70 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51768/0 (socket says 172.21.15.94:51768) 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f905894f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d959a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 -- 172.21.15.94:0/2351027764 learned_addr learned my addr 172.21.15.94:0/2351027764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 -- 172.21.15.94:0/2351027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d58d4d0 msgr2=0x55706d958d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d58d4d0 0x55706d958d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 -- 172.21.15.94:0/2351027764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 msgr2=0x55706d959a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d959a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 -- 172.21.15.94:0/2351027764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706d881c30 con 0x55706d5b6390 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f905894f640 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d959a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f9059951640 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ca0 tx=0x7f904c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee3385a73876aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.490+0000 7f904a7fc640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 0x7f90400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.491+0000 7f903f7fe640 1 -- 172.21.15.94:0/2351027764 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c01e070 con 0x55706d5b6390 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.491+0000 7f903f7fe640 1 -- 172.21.15.94:0/2351027764 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c009e50 con 0x55706d5b6390 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.491+0000 7f903f7fe640 1 -- 172.21.15.94:0/2351027764 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c00e9f0 con 0x55706d5b6390 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.491+0000 7f9059150640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 0x7f90400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.491+0000 7f904a7fc640 1 -- 172.21.15.94:0/2169326192 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f90500df2b0 con 0x7f9054101d70 2024-04-25T13:53:17.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- 172.21.15.94:0/2351027764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 msgr2=0x55706d958810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9059150640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 0x7f90400b2000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90500049d0 tx=0x7f9050074fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ca0 tx=0x7f904c011810 comp rx=0 tx=0).stop 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- 172.21.15.94:0/2351027764 shutdown_connections 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d959a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d58d4d0 0x55706d958d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 --2- 172.21.15.94:0/2351027764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d958810 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- 172.21.15.94:0/2351027764 >> 172.21.15.94:0/2351027764 conn(0x55706d5aa4e0 msgr2=0x55706d954020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- 172.21.15.94:0/2351027764 shutdown_connections 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- 172.21.15.94:0/2351027764 wait complete. 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 -- start start 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.492+0000 7f9062261740 1 Processor -- start 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d962560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d969a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55706d95a530 con 0x55706d5956b0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55706d843dc0 con 0x55706d58d4d0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55706d95a6b0 con 0x55706d5b6390 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 10 client.0 _ll_register_callbacks cb 0x55706d85f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51784/0 (socket says 172.21.15.94:51784) 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 -- 172.21.15.94:0/3670019342 learned_addr learned my addr 172.21.15.94:0/3670019342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9059951640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d962560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9059951640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d969a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 -- 172.21.15.94:0/3670019342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 msgr2=0x55706d969a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d969a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 -- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 msgr2=0x55706d962560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d962560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 -- 172.21.15.94:0/3670019342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706d5ce3f0 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9059951640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d962560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f905894f640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f904402a930 tx=0x7f9044031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d205083b48d5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f903cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044035040 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9044004040 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.493+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f904c009e50 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9044004240 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 10 client.19668 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55706d6987d0 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c009e50 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9044044020 con 0x55706d5956b0 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 10 client.19668 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 10 client.19668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 20 client.19668 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d574f60 0x55706d9701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 4 client.19668 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706d8ca990 con 0x55706d574f60 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.494+0000 7f9059951640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d574f60 0x55706d9701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.495+0000 7f903cff9640 10 client.19668 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.495+0000 7f9059951640 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d574f60 0x55706d9701b0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f187700d9f811649 server_cookie=81b0f2af895845ce in_seq=0 out_seq=0 2024-04-25T13:53:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.496+0000 7f903cff9640 1 -- 172.21.15.94:0/3670019342 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55706d8ca990 con 0x55706d574f60 2024-04-25T13:53:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.496+0000 7f903cff9640 10 client.19668 handle_command_reply: tid=0 2024-04-25T13:53:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.496+0000 7f9062261740 1 client.19668 shutdown 2024-04-25T13:53:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 msgr2=0x55706d962aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f904402a930 tx=0x7f9044031db0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 shutdown_connections 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d574f60 0x55706d9701b0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d969a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5956b0 0x55706d962aa0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f903cff9640 0 client.19668 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 --2- 172.21.15.94:0/3670019342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d962560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 >> 172.21.15.94:0/3670019342 conn(0x55706d5aa4e0 msgr2=0x55706d953040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 shutdown_connections 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 1 -- 172.21.15.94:0/3670019342 wait complete. 2024-04-25T13:53:17.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.497+0000 7f9062261740 20 client.19668 trim_cache size 0 max 16384 2024-04-25T13:53:17.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.510+0000 7f9062261740 1 Processor -- start 2024-04-25T13:53:17.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 -- start start 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d969920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d952b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b1b60 con 0x55706d58d4d0 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b36a0 con 0x55706d5956b0 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9062261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b2230 con 0x55706d5b6390 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51796/0 (socket says 172.21.15.94:51796) 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 -- 172.21.15.94:0/2757860115 learned_addr learned my addr 172.21.15.94:0/2757860115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f905894f640 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d969920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d952b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 -- 172.21.15.94:0/2757860115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 msgr2=0x55706d952b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d952b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 -- 172.21.15.94:0/2757860115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 msgr2=0x55706d969920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d969920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 -- 172.21.15.94:0/2757860115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706d969e60 con 0x55706d58d4d0 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f9059951640 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f904c009c40 tx=0x7f904c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e480c4e3f2131e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f903effd640 1 -- 172.21.15.94:0/2757860115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c01e040 con 0x55706d58d4d0 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.511+0000 7f903effd640 1 -- 172.21.15.94:0/2757860115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c00e8b0 con 0x55706d58d4d0 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f903effd640 1 -- 172.21.15.94:0/2757860115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c003930 con 0x55706d58d4d0 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 -- 172.21.15.94:0/2757860115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 msgr2=0x55706d952610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f904c009c40 tx=0x7f904c00e720 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 -- 172.21.15.94:0/2757860115 shutdown_connections 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5b6390 0x55706d952b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d5956b0 0x55706d969920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 --2- 172.21.15.94:0/2757860115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d58d4d0 0x55706d952610 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 -- 172.21.15.94:0/2757860115 >> 172.21.15.94:0/2757860115 conn(0x55706d5aa4e0 msgr2=0x55706d95d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 -- 172.21.15.94:0/2757860115 shutdown_connections 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.512+0000 7f9062261740 1 -- 172.21.15.94:0/2757860115 wait complete. 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 -- start start 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 Processor -- start 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d966a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5956b0 0x55706d966fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b2fd0 con 0x55706d5b6390 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b2900 con 0x55706d58d4d0 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90541afb40 con 0x55706d5956b0 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9062261740 10 client.0 _ll_register_callbacks cb 0x7f9044009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9059951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51798/0 (socket says 172.21.15.94:51798) 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9059951640 1 -- 172.21.15.94:0/3069455407 learned_addr learned my addr 172.21.15.94:0/3069455407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f9059951640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d966a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.513+0000 7f905894f640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5956b0 0x55706d966fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 -- 172.21.15.94:0/3069455407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5956b0 msgr2=0x55706d966fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5956b0 0x55706d966fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 -- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 msgr2=0x55706d966a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d966a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 -- 172.21.15.94:0/3069455407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706d9684c0 con 0x55706d5b6390 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9059951640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f904c006010 tx=0x7f904c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cd285cd4f5a842 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:17.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c01e040 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f904c00ea60 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f904c00e8b0 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f904c003e70 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9062261740 10 client.19674 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55706d969740 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c00e8b0 con 0x55706d5b6390 2024-04-25T13:53:17.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f904c0044f0 con 0x55706d5b6390 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9062261740 10 client.19674 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.514+0000 7f9062261740 10 client.19674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f9062261740 20 client.19674 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d54a3a0 0x55706d9da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f9062261740 4 client.19674 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55706d8c4800 con 0x55706d54a3a0 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f905894f640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d54a3a0 0x55706d9da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f903cff9640 10 client.19674 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:17.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.515+0000 7f905894f640 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d54a3a0 0x55706d9da5d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24557b3a3ec785e server_cookie=5201cddc20de704f in_seq=0 out_seq=0 2024-04-25T13:53:17.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f903cff9640 1 -- 172.21.15.94:0/3069455407 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55706d8c4800 con 0x55706d54a3a0 2024-04-25T13:53:17.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f903cff9640 10 client.19674 handle_command_reply: tid=0 2024-04-25T13:53:17.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 client.19674 shutdown 2024-04-25T13:53:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 msgr2=0x55706d967f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f904c006010 tx=0x7f904c005850 comp rx=0 tx=0).stop 2024-04-25T13:53:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 shutdown_connections 2024-04-25T13:53:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55706d54a3a0 0x55706d9da5d0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55706d5b6390 0x55706d967f00 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55706d5956b0 0x55706d966fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 --2- 172.21.15.94:0/3069455407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55706d58d4d0 0x55706d966a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 >> 172.21.15.94:0/3069455407 conn(0x55706d5aa4e0 msgr2=0x55706d956e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f903cff9640 0 client.19674 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 shutdown_connections 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 1 -- 172.21.15.94:0/3069455407 wait complete. 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.517+0000 7f9062261740 20 client.19674 trim_cache size 0 max 16384 2024-04-25T13:53:17.528 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:17.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 msgr2=0x7f90400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 0x7f90400b2000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90500049d0 tx=0x7f9050074fa0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 msgr2=0x7f90541a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f905000eb90 tx=0x7f905000c6a0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 shutdown_connections 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90400afaf0 0x7f90400b2000 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9054107920 0x7f90541a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90541027b0 0x7f90541a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 --2- 172.21.15.94:0/2169326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9054101d70 0x7f90541a0800 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 >> 172.21.15.94:0/2169326192 conn(0x7f90540fd340 msgr2=0x7f9054103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 shutdown_connections 2024-04-25T13:53:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:17.518+0000 7f905b3b4640 1 -- 172.21.15.94:0/2169326192 wait complete. 2024-04-25T13:53:17.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:17 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/698419876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/698419876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:17.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1421380 12.60 MB/sec execute 480 sec latency 89.688 ms 2024-04-25T13:53:18.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:53:18.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:53:18.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:53:18.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:18 smithi094 ceph-mon[29195]: pgmap v694: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-04-25T13:53:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:18 smithi137 ceph-mon[32568]: pgmap v694: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- 172.21.15.94:0/4002005238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 msgr2=0x7fb7cc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/4002005238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc1036e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009ca0 tx=0x7fb7c402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- 172.21.15.94:0/4002005238 shutdown_connections 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/4002005238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 0x7fb7cc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/4002005238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 0x7fb7cc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/4002005238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc1036e0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- 172.21.15.94:0/4002005238 >> 172.21.15.94:0/4002005238 conn(0x7fb7cc0fdbd0 msgr2=0x7fb7cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- 172.21.15.94:0/4002005238 shutdown_connections 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- 172.21.15.94:0/4002005238 wait complete. 2024-04-25T13:53:18.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 Processor -- start 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- start start 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 0x7fb7cc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 0x7fb7cc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e4e0 con 0x7fb7cc1029e0 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e360 con 0x7fb7cc103c20 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.648+0000 7fb7d2d00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e660 con 0x7fb7cc108920 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51808/0 (socket says 172.21.15.94:51808) 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/2615499295 learned_addr learned my addr 172.21.15.94:0/2615499295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 0x7fb7cc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d129d640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 0x7fb7cc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/2615499295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 msgr2=0x7fb7cc1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 0x7fb7cc1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/2615499295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 msgr2=0x7fb7cc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 0x7fb7cc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/2615499295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc1a2030 con 0x7fb7cc1029e0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40098d0 tx=0x7fb7c4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7c9ffb640 1 -- 172.21.15.94:0/2615499295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4003b70 con 0x7fb7cc1029e0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7c9ffb640 1 -- 172.21.15.94:0/2615499295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c4003d10 con 0x7fb7cc1029e0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1a22c0 con 0x7fb7cc1029e0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.649+0000 7fb7c9ffb640 1 -- 172.21.15.94:0/2615499295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c402fa50 con 0x7fb7cc1029e0 2024-04-25T13:53:18.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.650+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a2810 con 0x7fb7cc1029e0 2024-04-25T13:53:18.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7c9ffb640 1 -- 172.21.15.94:0/2615499295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb7c4004580 con 0x7fb7cc1029e0 2024-04-25T13:53:18.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 Processor -- start 2024-04-25T13:53:18.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 -- start start 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 0x55b921635a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b92163c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b92159ab00 con 0x55b921294410 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b921536920 con 0x55b921273730 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.652+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b921596ac0 con 0x55b92126b550 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51810/0 (socket says 172.21.15.94:51810) 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 -- 172.21.15.94:0/386557345 learned_addr learned my addr 172.21.15.94:0/386557345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b92163c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 0x55b921635a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7c9ffb640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 0x7fb7b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 -- 172.21.15.94:0/386557345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 msgr2=0x55b921635a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 0x55b921635a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 -- 172.21.15.94:0/386557345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 msgr2=0x55b92163c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b92163c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 -- 172.21.15.94:0/386557345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9213747e0 con 0x55b921294410 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b92163c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 0x55b921635a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 0x7fb7b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7c9ffb640 1 -- 172.21.15.94:0/2615499295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fb7c4100030 con 0x7fb7cc1029e0 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7d129d640 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009e20 tx=0x7fb7b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9302c59ceda264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7c2ffd640 1 -- 172.21.15.94:0/386557345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4017050 con 0x55b921294410 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7c2ffd640 1 -- 172.21.15.94:0/386557345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b4003c40 con 0x55b921294410 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.653+0000 7fb7c2ffd640 1 -- 172.21.15.94:0/386557345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400f3c0 con 0x55b921294410 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/386557345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 msgr2=0x55b9216354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d129d640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 0x7fb7b80b2000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4023000 tx=0x7fb7b4024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009e20 tx=0x7fb7b4011810 comp rx=0 tx=0).stop 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/386557345 shutdown_connections 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b92163c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b92126b550 0x55b921635a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/386557345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216354a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/386557345 >> 172.21.15.94:0/386557345 conn(0x55b921288560 msgr2=0x55b921630e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/386557345 shutdown_connections 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.654+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/386557345 wait complete. 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 -- start start 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 Processor -- start 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b92163f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b92163f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b92163d370 con 0x55b921294410 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9215345d0 con 0x55b92126b550 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b92163d4f0 con 0x55b921273730 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d9bb5740 10 client.0 _ll_register_callbacks cb 0x55b921522080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d0a9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b92163f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51824/0 (socket says 172.21.15.94:51824) 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7d129d640 1 -- 172.21.15.94:0/3660108798 learned_addr learned my addr 172.21.15.94:0/3660108798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.655+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b92163f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 -- 172.21.15.94:0/3660108798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 msgr2=0x55b92163f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b92163f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 -- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 msgr2=0x55b92163f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b92163f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 -- 172.21.15.94:0/3660108798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9213d4f70 con 0x55b921294410 2024-04-25T13:53:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b92163f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b92163f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d129d640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009fd0 tx=0x7fb7b4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc9e06d1636bc90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4036060 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fb7b40039c0 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b4003c40 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7b4017020 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d9bb5740 10 client.19686 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b92126c170 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b4003c40 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.656+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b400f8f0 con 0x55b921294410 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 10 client.19686 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:18.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 20 client.19686 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9212531f0 0x55b92164cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 4 client.19686 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9215a8710 con 0x55b9212531f0 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9212531f0 0x55b92164cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9212531f0 0x55b92164cb90 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e06e16ad37a7cb server_cookie=a181b0e18d03388a in_seq=0 out_seq=0 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.657+0000 7fb7abfff640 10 client.19686 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7abfff640 1 -- 172.21.15.94:0/3660108798 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b9215a8710 con 0x55b9212531f0 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7abfff640 10 client.19686 handle_command_reply: tid=0 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 client.19686 shutdown 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 msgr2=0x55b921646800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009fd0 tx=0x7fb7b4009830 comp rx=0 tx=0).stop 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 shutdown_connections 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b9212531f0 0x55b92164cb90 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b921646800 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b92163f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7abfff640 0 client.19686 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/3660108798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b92163f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 >> 172.21.15.94:0/3660108798 conn(0x55b921288560 msgr2=0x55b92162fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 shutdown_connections 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.658+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/3660108798 wait complete. 2024-04-25T13:53:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.659+0000 7fb7d9bb5740 20 client.19686 trim_cache size 0 max 16384 2024-04-25T13:53:18.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 Processor -- start 2024-04-25T13:53:18.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 -- start start 2024-04-25T13:53:18.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b921630f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921294410 0x55b921646720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ab580 con 0x55b92126b550 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ad0c0 con 0x55b921273730 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1abc50 con 0x55b921294410 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51832/0 (socket says 172.21.15.94:51832) 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/627985502 learned_addr learned my addr 172.21.15.94:0/627985502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d129d640 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921294410 0x55b921646720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b921630f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/627985502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921294410 msgr2=0x55b921646720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921294410 0x55b921646720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/627985502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 msgr2=0x55b921630f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b921630f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.678+0000 7fb7d0a9c640 1 -- 172.21.15.94:0/627985502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9216319c0 con 0x55b92126b550 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.679+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40fda90 tx=0x7fb7c40b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65c4298ccc2016a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.679+0000 7fb7c27fc640 1 -- 172.21.15.94:0/627985502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40c7be0 con 0x55b92126b550 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.679+0000 7fb7c27fc640 1 -- 172.21.15.94:0/627985502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c4036040 con 0x55b92126b550 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.679+0000 7fb7c27fc640 1 -- 172.21.15.94:0/627985502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40d12d0 con 0x55b92126b550 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/627985502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 msgr2=0x55b921639e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40fda90 tx=0x7fb7c40b1040 comp rx=0 tx=0).stop 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/627985502 shutdown_connections 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921294410 0x55b921646720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b921273730 0x55b921630f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/627985502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b92126b550 0x55b921639e10 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/627985502 >> 172.21.15.94:0/627985502 conn(0x55b921288560 msgr2=0x55b9216307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/627985502 shutdown_connections 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/627985502 wait complete. 2024-04-25T13:53:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- start start 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 Processor -- start 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b921643fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b921644500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ac9f0 con 0x55b921294410 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ac320 con 0x55b92126b550 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1a9560 con 0x55b921273730 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.680+0000 7fb7d9bb5740 10 client.0 _ll_register_callbacks cb 0x7fb7b401c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51848/0 (socket says 172.21.15.94:51848) 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 -- 172.21.15.94:0/235063962 learned_addr learned my addr 172.21.15.94:0/235063962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b921643fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7cbfff640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b921644500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 -- 172.21.15.94:0/235063962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 msgr2=0x55b921644500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b921644500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 -- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 msgr2=0x55b921643fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b921643fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 -- 172.21.15.94:0/235063962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9216b3470 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d129d640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb7b401c8f0 tx=0x7fb7b4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3017ec8c25911a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4036040 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fb7b40039c0 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c4036040 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7b404a050 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d9bb5740 10 client.19692 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.681+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b9216b4710 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c4036040 con 0x55b921294410 2024-04-25T13:53:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b4042020 con 0x55b921294410 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 10 client.19692 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 10 client.19692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 20 client.19692 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b921228630 0x55b9216b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 4 client.19692 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b9215a2580 con 0x55b921228630 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b921228630 0x55b9216b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7abfff640 10 client.19692 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.682+0000 7fb7d0a9c640 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b921228630 0x55b9216b7180 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae5667fa9c484d6 server_cookie=e149042f7c433d74 in_seq=0 out_seq=0 2024-04-25T13:53:18.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7abfff640 1 -- 172.21.15.94:0/235063962 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b9215a2580 con 0x55b921228630 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7abfff640 10 client.19692 handle_command_reply: tid=0 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 client.19692 shutdown 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 msgr2=0x55b9216b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb7b401c8f0 tx=0x7fb7b4009ac0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 shutdown_connections 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b921228630 0x55b9216b7180 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b921294410 0x55b9216b2f30 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b921273730 0x55b921644500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7abfff640 0 client.19692 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 --2- 172.21.15.94:0/235063962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b92126b550 0x55b921643fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.684+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 >> 172.21.15.94:0/235063962 conn(0x55b921288560 msgr2=0x55b921634660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 shutdown_connections 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d9bb5740 1 -- 172.21.15.94:0/235063962 wait complete. 2024-04-25T13:53:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d9bb5740 20 client.19692 trim_cache size 0 max 16384 2024-04-25T13:53:18.690 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:18.690 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:53:18.690 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "db7ae5b5-93e5-4b95-9a4e-70523a1329cc", 2024-04-25T13:53:18.690 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:53:18.690 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 msgr2=0x7fb7b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 0x7fb7b80b2000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4023000 tx=0x7fb7b4024000 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 msgr2=0x7fb7cc06a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40098d0 tx=0x7fb7c4031700 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 shutdown_connections 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb7b80afaf0 0x7fb7b80b2000 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7cc108920 0x7fb7cc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc103c20 0x7fb7cc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 --2- 172.21.15.94:0/2615499295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc1029e0 0x7fb7cc06a900 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 >> 172.21.15.94:0/2615499295 conn(0x7fb7cc0fdbd0 msgr2=0x7fb7cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 shutdown_connections 2024-04-25T13:53:18.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.685+0000 7fb7d2d00640 1 -- 172.21.15.94:0/2615499295 wait complete. 2024-04-25T13:53:18.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:53:18.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:53:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:18 smithi138 ceph-mon[32977]: pgmap v694: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-04-25T13:53:18.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1423379 12.59 MB/sec execute 481 sec latency 14.192 ms 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- 172.21.15.94:0/1535135014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 msgr2=0x7f5f78102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- 172.21.15.94:0/1535135014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f78102e70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400aa10 tx=0x7f5f7402b720 comp rx=0 tx=0).stop 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- 172.21.15.94:0/1535135014 shutdown_connections 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- 172.21.15.94:0/1535135014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 0x7f5f7810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- 172.21.15.94:0/1535135014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f781037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- 172.21.15.94:0/1535135014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f78102e70 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- 172.21.15.94:0/1535135014 >> 172.21.15.94:0/1535135014 conn(0x7f5f780fd340 msgr2=0x7f5f780ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- 172.21.15.94:0/1535135014 shutdown_connections 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- 172.21.15.94:0/1535135014 wait complete. 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 Processor -- start 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 -- start start 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.850+0000 7f5f80357640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f7819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 0x7f5f781a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810d400 con 0x7f5f78102170 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810d280 con 0x7f5f781080b0 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810d580 con 0x7f5f781033b0 2024-04-25T13:53:18.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51876/0 (socket says 172.21.15.94:51876) 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/2262500754 learned_addr learned my addr 172.21.15.94:0/2262500754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 0x7f5f781a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f7819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/2262500754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 msgr2=0x7f5f7819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f7819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/2262500754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 msgr2=0x7f5f781a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 0x7f5f781a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/2262500754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f781a3f90 con 0x7f5f78102170 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f7819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:18.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006010 tx=0x7f5f74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f6f7fe640 1 -- 172.21.15.94:0/2262500754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7403f070 con 0x7f5f78102170 2024-04-25T13:53:18.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f6f7fe640 1 -- 172.21.15.94:0/2262500754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f74003ca0 con 0x7f5f78102170 2024-04-25T13:53:18.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f781a4220 con 0x7f5f78102170 2024-04-25T13:53:18.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f6f7fe640 1 -- 172.21.15.94:0/2262500754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74036040 con 0x7f5f78102170 2024-04-25T13:53:18.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.851+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a4710 con 0x7f5f78102170 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.854+0000 7f5f87207740 1 Processor -- start 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.854+0000 7f5f6f7fe640 1 -- 172.21.15.94:0/2262500754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5f74042070 con 0x7f5f78102170 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.854+0000 7f5f87207740 1 -- start start 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fece390 0x563a9026fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a9026ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563a901d4d80 con 0x563a8fead6b0 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a9018e8b0 con 0x563a8fea54d0 2024-04-25T13:53:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f87207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a901d0d40 con 0x563a8fece390 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51880/0 (socket says 172.21.15.94:51880) 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/3092175536 learned_addr learned my addr 172.21.15.94:0/3092175536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a9026ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fece390 0x563a9026fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f6f7fe640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 0x7f5f640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/3092175536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fece390 msgr2=0x563a9026fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fece390 0x563a9026fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/3092175536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 msgr2=0x563a9026ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a9026ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/3092175536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a9015fa40 con 0x563a8fead6b0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 0x7f5f640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f6f7fe640 1 -- 172.21.15.94:0/2262500754 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5f740ff580 con 0x7f5f78102170 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009c20 tx=0x7f5f68010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b078acc7c34f0c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.855+0000 7f5f6cbfe640 1 -- 172.21.15.94:0/3092175536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004030 con 0x563a8fead6b0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.856+0000 7f5f6cbfe640 1 -- 172.21.15.94:0/3092175536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f680035f0 con 0x563a8fead6b0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.856+0000 7f5f6cbfe640 1 -- 172.21.15.94:0/3092175536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680126b0 con 0x563a8fead6b0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.856+0000 7f5f87207740 1 -- 172.21.15.94:0/3092175536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 msgr2=0x563a90270c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.856+0000 7f5f87207740 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009c20 tx=0x7f5f68010740 comp rx=0 tx=0).stop 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.856+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 0x7f5f640b2000 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x563a90270ab0 tx=0x7f5f70014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 -- 172.21.15.94:0/3092175536 shutdown_connections 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fead6b0 0x563a90270c70 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a9026ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- 172.21.15.94:0/3092175536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fece390 0x563a9026fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 -- 172.21.15.94:0/3092175536 >> 172.21.15.94:0/3092175536 conn(0x563a8fec24e0 msgr2=0x563a9026b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 -- 172.21.15.94:0/3092175536 shutdown_connections 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 -- 172.21.15.94:0/3092175536 wait complete. 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 -- start start 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 Processor -- start 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fece390 0x563a90279f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.857+0000 7f5f87207740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90280ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f87207740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563a90271780 con 0x563a8fe8cf60 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f87207740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a9015bd10 con 0x563a8fece390 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f87207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a90271900 con 0x563a8fea54d0 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f87207740 10 client.0 _ll_register_callbacks cb 0x563a90188210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e8f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90280ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56946/0 (socket says 172.21.15.94:56946) 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/3975466882 learned_addr learned my addr 172.21.15.94:0/3975466882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fece390 0x563a90279f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fece390 msgr2=0x563a90279f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fece390 0x563a90279f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/3975466882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 msgr2=0x563a90280ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90280ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 -- 172.21.15.94:0/3975466882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8ffb07d0 con 0x563a8fea54d0 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90280ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f74030550 tx=0x7f5f74102bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33750b9c44c728d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74049030 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5f740d0390 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.858+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f680035f0 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f740aee60 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 10 client.28936 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563a8fee63f0 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f680035f0 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5f7400b040 con 0x563a8fea54d0 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 10 client.28936 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 10 client.28936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 20 client.28936 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fe623a0 0x563a90287380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 4 client.28936 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a901e2990 con 0x563a8fe623a0 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.859+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fe623a0 0x563a90287380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.860+0000 7f5f61ffb640 10 client.28936 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.860+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fe623a0 0x563a90287380 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d89074f88649ea server_cookie=51d536af429ef7c5 in_seq=0 out_seq=0 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.860+0000 7f5f61ffb640 1 -- 172.21.15.94:0/3975466882 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a901e2990 con 0x563a8fe623a0 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.860+0000 7f5f61ffb640 10 client.28936 handle_command_reply: tid=0 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 client.28936 shutdown 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 msgr2=0x563a90279a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f74030550 tx=0x7f5f74102bb0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 shutdown_connections 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fe623a0 0x563a90287380 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90280ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f61ffb640 0 client.28936 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fece390 0x563a90279f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 --2- 172.21.15.94:0/3975466882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fea54d0 0x563a90279a40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 >> 172.21.15.94:0/3975466882 conn(0x563a8fec24e0 msgr2=0x563a9026a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 shutdown_connections 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 1 -- 172.21.15.94:0/3975466882 wait complete. 2024-04-25T13:53:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.861+0000 7f5f87207740 20 client.28936 trim_cache size 0 max 16384 2024-04-25T13:53:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.879+0000 7f5f87207740 1 Processor -- start 2024-04-25T13:53:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.879+0000 7f5f87207740 1 -- start start 2024-04-25T13:53:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.879+0000 7f5f87207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a902786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.879+0000 7f5f87207740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f87207740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a90280ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f87207740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ad730 con 0x563a8fe8cf60 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f87207740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781af270 con 0x563a8fea54d0 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f87207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ade00 con 0x563a8fe623a0 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51890/0 (socket says 172.21.15.94:51890) 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/4161040766 learned_addr learned my addr 172.21.15.94:0/4161040766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a902786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a90280ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/4161040766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 msgr2=0x563a902786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a902786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/4161040766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 msgr2=0x563a90280ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a90280ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 -- 172.21.15.94:0/4161040766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8fea60f0 con 0x563a8fe8cf60 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a90280ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005e00 tx=0x7f5f68009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470ab0e8819f6c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f63fff640 1 -- 172.21.15.94:0/4161040766 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680052b0 con 0x563a8fe8cf60 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f63fff640 1 -- 172.21.15.94:0/4161040766 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f68030070 con 0x563a8fe8cf60 2024-04-25T13:53:18.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.880+0000 7f5f63fff640 1 -- 172.21.15.94:0/4161040766 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68003e20 con 0x563a8fe8cf60 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- 172.21.15.94:0/4161040766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 msgr2=0x563a90279d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5f68005e00 tx=0x7f5f68009970 comp rx=0 tx=0).stop 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- 172.21.15.94:0/4161040766 shutdown_connections 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fea54d0 0x563a90280ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fe8cf60 0x563a90279d20 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 --2- 172.21.15.94:0/4161040766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a902786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- 172.21.15.94:0/4161040766 >> 172.21.15.94:0/4161040766 conn(0x563a8fec24e0 msgr2=0x563a90279610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- 172.21.15.94:0/4161040766 shutdown_connections 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- 172.21.15.94:0/4161040766 wait complete. 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 -- start start 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.881+0000 7f5f87207740 1 Processor -- start 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a9027c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fe8cf60 0x563a9027cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781aeba0 con 0x563a8fea54d0 2024-04-25T13:53:18.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ae4d0 con 0x563a8fe8cf60 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ab710 con 0x563a8fe623a0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f87207740 10 client.0 _ll_register_callbacks cb 0x7f5f74005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51900/0 (socket says 172.21.15.94:51900) 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 -- 172.21.15.94:0/748267741 learned_addr learned my addr 172.21.15.94:0/748267741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7d8f2640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fe8cf60 0x563a9027cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a9027c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 -- 172.21.15.94:0/748267741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 msgr2=0x563a9027c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a9027c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 -- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fe8cf60 msgr2=0x563a9027cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fe8cf60 0x563a9027cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 -- 172.21.15.94:0/748267741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a901954b0 con 0x563a8fea54d0 2024-04-25T13:53:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f7e8f4640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5f70010e00 tx=0x7f5f70009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a0ddf6a817a057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.882+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7001edb0 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f68030070 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5f7001c480 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f7001c680 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 10 client.19710 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563a902815d0 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f68030070 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5f70030920 con 0x563a8fea54d0 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 10 client.19710 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 20 client.19710 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fece390 0x563a902f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 4 client.19710 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a901dc800 con 0x563a8fece390 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.883+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fece390 0x563a902f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f7e0f3640 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fece390 0x563a902f1b30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5c46fd21009b7f server_cookie=e8da6fab164c47d in_seq=0 out_seq=0 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f61ffb640 10 client.19710 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f61ffb640 1 -- 172.21.15.94:0/748267741 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563a901dc800 con 0x563a8fece390 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f61ffb640 10 client.19710 handle_command_reply: tid=0 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 client.19710 shutdown 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 msgr2=0x563a9027fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5f70010e00 tx=0x7f5f70009500 comp rx=0 tx=0).stop 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 shutdown_connections 2024-04-25T13:53:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563a8fece390 0x563a902f1b30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563a8fea54d0 0x563a9027fe60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a8fe8cf60 0x563a9027cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f61ffb640 0 client.19710 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 --2- 172.21.15.94:0/748267741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a8fe623a0 0x563a9027c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 >> 172.21.15.94:0/748267741 conn(0x563a8fec24e0 msgr2=0x563a90279da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 shutdown_connections 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 1 -- 172.21.15.94:0/748267741 wait complete. 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.884+0000 7f5f87207740 20 client.19710 trim_cache size 0 max 16384 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:18.892 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: "db7ae5b5-93e5-4b95-9a4e-70523a1329cc": { 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "db7ae5b5-93e5-4b95-9a4e-70523a1329cc", 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:53:18.893 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 msgr2=0x7f5f640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 0x7f5f640b2000 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x563a90270ab0 tx=0x7f5f70014040 comp rx=0 tx=0).stop 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 msgr2=0x7f5f7819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006010 tx=0x7f5f74004270 comp rx=0 tx=0).stop 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 shutdown_connections 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f640afaf0 0x7f5f640b2000 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f781080b0 0x7f5f781a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f781033b0 0x7f5f7819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 --2- 172.21.15.94:0/2262500754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f78102170 0x7f5f7819c450 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 >> 172.21.15.94:0/2262500754 conn(0x7f5f780fd340 msgr2=0x7f5f78104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 shutdown_connections 2024-04-25T13:53:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:18.885+0000 7f5f80357640 1 -- 172.21.15.94:0/2262500754 wait complete. 2024-04-25T13:53:18.896 INFO:tasks.cephfs.filesystem:scrub status for tag:db7ae5b5-93e5-4b95-9a4e-70523a1329cc - {'path': '/', 'tag': 'db7ae5b5-93e5-4b95-9a4e-70523a1329cc', 'options': 'recursive,force'} 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- 172.21.15.94:0/3298743030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 msgr2=0x7fa948103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/3298743030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa948103750 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009900 tx=0x7fa93c02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- 172.21.15.94:0/3298743030 shutdown_connections 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/3298743030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 0x7fa94810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/3298743030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 0x7fa9481040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/3298743030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa948103750 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- 172.21.15.94:0/3298743030 >> 172.21.15.94:0/3298743030 conn(0x7fa9480fdc60 msgr2=0x7fa9481000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- 172.21.15.94:0/3298743030 shutdown_connections 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- 172.21.15.94:0/3298743030 wait complete. 2024-04-25T13:53:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 Processor -- start 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.429+0000 7fa94f0b1640 1 -- start start 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 0x7fa94819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 0x7fa9481a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa94810e550 con 0x7fa948102a50 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa94810e3d0 con 0x7fa948103c90 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa94810e6d0 con 0x7fa948108990 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51914/0 (socket says 172.21.15.94:51914) 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 -- 172.21.15.94:0/1096418214 learned_addr learned my addr 172.21.15.94:0/1096418214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa947fff640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 0x7fa94819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94d64e640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 0x7fa9481a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 -- 172.21.15.94:0/1096418214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 msgr2=0x7fa9481a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 0x7fa9481a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 -- 172.21.15.94:0/1096418214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 msgr2=0x7fa94819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 0x7fa94819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 -- 172.21.15.94:0/1096418214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9481a1d70 con 0x7fa948102a50 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006010 tx=0x7fa93c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa945ffb640 1 -- 172.21.15.94:0/1096418214 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c03f030 con 0x7fa948102a50 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa945ffb640 1 -- 172.21.15.94:0/1096418214 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93c031910 con 0x7fa948102a50 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9481a2000 con 0x7fa948102a50 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.430+0000 7fa945ffb640 1 -- 172.21.15.94:0/1096418214 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c035070 con 0x7fa948102a50 2024-04-25T13:53:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.431+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a24f0 con 0x7fa948102a50 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.433+0000 7fa945ffb640 1 -- 172.21.15.94:0/1096418214 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa93c042030 con 0x7fa948102a50 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa945ffb640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 0x7fa9300b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 Processor -- start 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 -- start start 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa945ffb640 1 -- 172.21.15.94:0/1096418214 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa93c0fef80 con 0x7fa948102a50 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa947fff640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 0x7fa9300b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beeaa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c2bee0b460 con 0x55c2beb051b0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c2bedc5590 con 0x55c2beadc2f0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa955bfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c2bee07420 con 0x55c2beae44d0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51924/0 (socket says 172.21.15.94:51924) 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 -- 172.21.15.94:0/2863904577 learned_addr learned my addr 172.21.15.94:0/2863904577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa947fff640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beeaa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94d64e640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa947fff640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 0x7fa9300b1ff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa94806b620 tx=0x7fa938011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 -- 172.21.15.94:0/2863904577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 msgr2=0x55c2beeb1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 -- 172.21.15.94:0/2863904577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 msgr2=0x55c2beeaa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beeaa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa94ce4d640 1 -- 172.21.15.94:0/2863904577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bedd20d0 con 0x55c2beb051b0 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.434+0000 7fa947fff640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beeaa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.435+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0c7170 tx=0x7fa93c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a02993bf95061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.435+0000 7fa92effd640 1 -- 172.21.15.94:0/2863904577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c049040 con 0x55c2beb051b0 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.435+0000 7fa92effd640 1 -- 172.21.15.94:0/2863904577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93c003d30 con 0x55c2beb051b0 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.435+0000 7fa92effd640 1 -- 172.21.15.94:0/2863904577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0cfd90 con 0x55c2beb051b0 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- 172.21.15.94:0/2863904577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 msgr2=0x55c2beea9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0c7170 tx=0x7fa93c009260 comp rx=0 tx=0).stop 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- 172.21.15.94:0/2863904577 shutdown_connections 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beeaa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- 172.21.15.94:0/2863904577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beb051b0 0x55c2beea9d80 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- 172.21.15.94:0/2863904577 >> 172.21.15.94:0/2863904577 conn(0x55c2beaf9300 msgr2=0x55c2beea1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- 172.21.15.94:0/2863904577 shutdown_connections 2024-04-25T13:53:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- 172.21.15.94:0/2863904577 wait complete. 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- start start 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 Processor -- start 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beb051b0 0x55c2beebad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c2beeb1a10 con 0x55c2beadc2f0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c2bed92b60 con 0x55c2beb051b0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.436+0000 7fa955bfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c2beeb1b90 con 0x55c2beae44d0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa955bfd740 10 client.0 _ll_register_callbacks cb 0x55c2bedb72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa947fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94d64e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beb051b0 0x55c2beebad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51938/0 (socket says 172.21.15.94:51938) 2024-04-25T13:53:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 -- 172.21.15.94:0/3377243349 learned_addr learned my addr 172.21.15.94:0/3377243349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 -- 172.21.15.94:0/3377243349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 msgr2=0x55c2beeb3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 -- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beb051b0 msgr2=0x55c2beebad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beb051b0 0x55c2beebad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 -- 172.21.15.94:0/3377243349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2beb1d210 con 0x55c2beadc2f0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa947fff640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0c34f0 tx=0x7fa93c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b611e4afc937001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa927fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c049050 con 0x55c2beadc2f0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa93c0cff90 con 0x55c2beadc2f0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93c003d30 con 0x55c2beadc2f0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa93c0e8300 con 0x55c2beadc2f0 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa955bfd740 10 client.19722 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.437+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2bebe9620 con 0x55c2beadc2f0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93c003d30 con 0x55c2beadc2f0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa93c0c2de0 con 0x55c2beadc2f0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 10 client.19722 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 10 client.19722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 20 client.19722 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beac3d80 0x55c2beec1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 4 client.19722 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2bee19070 con 0x55c2beac3d80 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa947fff640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beac3d80 0x55c2beec1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa927fff640 10 client.19722 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.438+0000 7fa947fff640 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beac3d80 0x55c2beec1360 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9c6f53de30d94a server_cookie=4aaf4e28cbaca469 in_seq=0 out_seq=0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa927fff640 1 -- 172.21.15.94:0/3377243349 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2bee19070 con 0x55c2beac3d80 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa927fff640 10 client.19722 handle_command_reply: tid=0 2024-04-25T13:53:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 client.19722 shutdown 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 msgr2=0x55c2beeb38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0c34f0 tx=0x7fa93c02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 shutdown_connections 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beac3d80 0x55c2beec1360 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beb051b0 0x55c2beebad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa927fff640 0 client.19722 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2beeb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 --2- 172.21.15.94:0/3377243349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb38d0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 >> 172.21.15.94:0/3377243349 conn(0x55c2beaf9300 msgr2=0x55c2beea0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 shutdown_connections 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 1 -- 172.21.15.94:0/3377243349 wait complete. 2024-04-25T13:53:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.440+0000 7fa955bfd740 20 client.19722 trim_cache size 0 max 16384 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.455+0000 7fa955bfd740 1 Processor -- start 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.455+0000 7fa955bfd740 1 -- start start 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beac3d80 0x55c2beeae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beebad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481a93b0 con 0x55c2beae44d0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481aaef0 con 0x55c2beadc2f0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa955bfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481a9a80 con 0x55c2beac3d80 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51944/0 (socket says 172.21.15.94:51944) 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 -- 172.21.15.94:0/587773876 learned_addr learned my addr 172.21.15.94:0/587773876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa947fff640 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beebad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beac3d80 0x55c2beeae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 -- 172.21.15.94:0/587773876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beac3d80 msgr2=0x55c2beeae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beac3d80 0x55c2beeae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 -- 172.21.15.94:0/587773876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 msgr2=0x55c2beebad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beebad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 -- 172.21.15.94:0/587773876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2beeb1bf0 con 0x55c2beae44d0 2024-04-25T13:53:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa94d64e640 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa940003c30 tx=0x7fa940004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff21a43e76c94d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa92e7fc640 1 -- 172.21.15.94:0/587773876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940003e60 con 0x55c2beae44d0 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa92e7fc640 1 -- 172.21.15.94:0/587773876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9400108e0 con 0x55c2beae44d0 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.456+0000 7fa92e7fc640 1 -- 172.21.15.94:0/587773876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94001ba00 con 0x55c2beae44d0 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 -- 172.21.15.94:0/587773876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 msgr2=0x55c2beeb3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa940003c30 tx=0x7fa940004410 comp rx=0 tx=0).stop 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 -- 172.21.15.94:0/587773876 shutdown_connections 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beae44d0 0x55c2beeb3bb0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beadc2f0 0x55c2beebad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 --2- 172.21.15.94:0/587773876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beac3d80 0x55c2beeae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 -- 172.21.15.94:0/587773876 >> 172.21.15.94:0/587773876 conn(0x55c2beaf9300 msgr2=0x55c2beeb34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 -- 172.21.15.94:0/587773876 shutdown_connections 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.457+0000 7fa955bfd740 1 -- 172.21.15.94:0/587773876 wait complete. 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 -- start start 2024-04-25T13:53:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 Processor -- start 2024-04-25T13:53:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beac3d80 0x55c2beeb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2bef276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481aa820 con 0x55c2beadc2f0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481aa150 con 0x55c2beac3d80 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481a7390 con 0x55c2beae44d0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa955bfd740 10 client.0 _ll_register_callbacks cb 0x7fa93c0cb070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51952/0 (socket says 172.21.15.94:51952) 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 -- 172.21.15.94:0/1424997686 learned_addr learned my addr 172.21.15.94:0/1424997686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beac3d80 0x55c2beeb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 -- 172.21.15.94:0/1424997686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 msgr2=0x55c2bef276c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa94d64e640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2bef276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2bef276c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 -- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beac3d80 msgr2=0x55c2beeb7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beac3d80 0x55c2beeb7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.458+0000 7fa947fff640 1 -- 172.21.15.94:0/1424997686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2beeb8880 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa947fff640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x55c2beeab160 tx=0x7fa938004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a1fac5a425e56e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938017040 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa938005020 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9400108e0 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa93803a050 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 10 client.19728 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2bef28d90 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9400108e0 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa938026040 con 0x55c2beadc2f0 2024-04-25T13:53:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 10 client.19728 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 10 client.19728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 20 client.19728 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beb051b0 0x55c2bef2b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 4 client.19728 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2bee12ee0 con 0x55c2beb051b0 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.459+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beb051b0 0x55c2bef2b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.460+0000 7fa927fff640 10 client.19728 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.460+0000 7fa94ce4d640 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beb051b0 0x55c2bef2b7b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eed09a63c8b6cd server_cookie=5fedf61fffbcee3f in_seq=0 out_seq=0 2024-04-25T13:53:19.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:19 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:53:19.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:19 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:19 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:53:19.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:19 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa927fff640 1 -- 172.21.15.94:0/1424997686 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2bee12ee0 con 0x55c2beb051b0 2024-04-25T13:53:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa927fff640 10 client.19728 handle_command_reply: tid=0 2024-04-25T13:53:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 client.19728 shutdown 2024-04-25T13:53:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 msgr2=0x55c2beeb8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x55c2beeab160 tx=0x7fa938004730 comp rx=0 tx=0).stop 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 shutdown_connections 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c2beb051b0 0x55c2bef2b7b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c2beae44d0 0x55c2bef276c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c2beadc2f0 0x55c2beeb8240 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa927fff640 0 client.19728 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.666+0000 7fa955bfd740 1 --2- 172.21.15.94:0/1424997686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2beac3d80 0x55c2beeb7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 >> 172.21.15.94:0/1424997686 conn(0x55c2beaf9300 msgr2=0x55c2beeb3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 shutdown_connections 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa955bfd740 1 -- 172.21.15.94:0/1424997686 wait complete. 2024-04-25T13:53:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa955bfd740 20 client.19728 trim_cache size 0 max 16384 2024-04-25T13:53:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:53:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:53:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:53:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 msgr2=0x7fa9300b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 0x7fa9300b1ff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa94806b620 tx=0x7fa938011810 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 msgr2=0x7fa94806bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.667+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006010 tx=0x7fa93c004270 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 shutdown_connections 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9300afae0 0x7fa9300b1ff0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948108990 0x7fa9481a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa948103c90 0x7fa94819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 --2- 172.21.15.94:0/1096418214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa948102a50 0x7fa94806bc80 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 >> 172.21.15.94:0/1096418214 conn(0x7fa9480fdc60 msgr2=0x7fa948104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 shutdown_connections 2024-04-25T13:53:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:19.668+0000 7fa94f0b1640 1 -- 172.21.15.94:0/1096418214 wait complete. 2024-04-25T13:53:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:19 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:53:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:19 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:19 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:53:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:19 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:19 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:53:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:19 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:19 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:53:19.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:19 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:19.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1425169 12.59 MB/sec execute 482 sec latency 130.393 ms 2024-04-25T13:53:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:20 smithi137 ceph-mon[32568]: pgmap v695: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-25T13:53:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:20 smithi094 ceph-mon[29195]: pgmap v695: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-25T13:53:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:20 smithi138 ceph-mon[32977]: pgmap v695: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-25T13:53:20.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1427719 12.59 MB/sec execute 483 sec latency 16.014 ms 2024-04-25T13:53:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:53:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:21 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:53:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:21 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:53:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:21 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:21.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1429453 12.58 MB/sec execute 484 sec latency 15.416 ms 2024-04-25T13:53:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:22 smithi137 ceph-mon[32568]: pgmap v696: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-04-25T13:53:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:22 smithi094 ceph-mon[29195]: pgmap v696: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-04-25T13:53:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:22 smithi138 ceph-mon[32977]: pgmap v696: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-04-25T13:53:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:53:22.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1432016 12.59 MB/sec execute 485 sec latency 63.691 ms 2024-04-25T13:53:23.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1434150 12.58 MB/sec execute 486 sec latency 19.591 ms 2024-04-25T13:53:24.548 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:24 smithi138 ceph-mon[32977]: pgmap v697: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-04-25T13:53:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:24 smithi137 ceph-mon[32568]: pgmap v697: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-04-25T13:53:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:24 smithi094 ceph-mon[29195]: pgmap v697: 97 pgs: 97 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-04-25T13:53:24.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1436039 12.58 MB/sec execute 487 sec latency 34.337 ms 2024-04-25T13:53:25.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1438901 12.59 MB/sec execute 488 sec latency 21.782 ms 2024-04-25T13:53:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:26 smithi137 ceph-mon[32568]: pgmap v698: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-25T13:53:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:26 smithi094 ceph-mon[29195]: pgmap v698: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-25T13:53:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:26 smithi138 ceph-mon[32977]: pgmap v698: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-25T13:53:26.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1440880 12.58 MB/sec execute 489 sec latency 18.711 ms 2024-04-25T13:53:27.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1443010 12.58 MB/sec execute 490 sec latency 49.044 ms 2024-04-25T13:53:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:28 smithi137 ceph-mon[32568]: pgmap v699: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-04-25T13:53:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:28 smithi094 ceph-mon[29195]: pgmap v699: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-04-25T13:53:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:28 smithi138 ceph-mon[32977]: pgmap v699: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-04-25T13:53:28.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1445799 12.58 MB/sec execute 491 sec latency 26.445 ms 2024-04-25T13:53:29.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1447890 12.58 MB/sec execute 492 sec latency 14.539 ms 2024-04-25T13:53:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:30 smithi137 ceph-mon[32568]: pgmap v700: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-04-25T13:53:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:30 smithi094 ceph-mon[29195]: pgmap v700: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-04-25T13:53:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:30 smithi138 ceph-mon[32977]: pgmap v700: 97 pgs: 97 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-04-25T13:53:30.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1450242 12.58 MB/sec execute 493 sec latency 13.020 ms 2024-04-25T13:53:31.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1453329 12.59 MB/sec execute 494 sec latency 19.861 ms 2024-04-25T13:53:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:32 smithi137 ceph-mon[32568]: pgmap v701: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-04-25T13:53:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:32 smithi094 ceph-mon[29195]: pgmap v701: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-04-25T13:53:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:32 smithi138 ceph-mon[32977]: pgmap v701: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 -- 172.21.15.94:0/2450136039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354108960 msgr2=0x7f135410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 --2- 172.21.15.94:0/2450136039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354108960 0x7f135410ad80 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f134c009ca0 tx=0x7f134c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 -- 172.21.15.94:0/2450136039 shutdown_connections 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 --2- 172.21.15.94:0/2450136039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354108960 0x7f135410ad80 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 --2- 172.21.15.94:0/2450136039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354103c60 0x7f1354104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 --2- 172.21.15.94:0/2450136039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f1354103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 -- 172.21.15.94:0/2450136039 >> 172.21.15.94:0/2450136039 conn(0x7f13540fdc10 msgr2=0x7f13541000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 -- 172.21.15.94:0/2450136039 shutdown_connections 2024-04-25T13:53:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.770+0000 7f135bdb2640 1 -- 172.21.15.94:0/2450136039 wait complete. 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 Processor -- start 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 -- start start 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f135419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354108960 0x7f13541a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f135410e550 con 0x7f1354103c60 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f135410e3d0 con 0x7f1354102a20 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135bdb2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f135410e6d0 con 0x7f1354108960 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44090/0 (socket says 172.21.15.94:44090) 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 -- 172.21.15.94:0/3991310869 learned_addr learned my addr 172.21.15.94:0/3991310869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f1359b4e640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f135419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135a34f640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354108960 0x7f13541a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 -- 172.21.15.94:0/3991310869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354108960 msgr2=0x7f13541a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354108960 0x7f13541a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 -- 172.21.15.94:0/3991310869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 msgr2=0x7f135419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f135419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f135934d640 1 -- 172.21.15.94:0/3991310869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541a1dc0 con 0x7f1354103c60 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.771+0000 7f1359b4e640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f135419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f135934d640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f134400e9f0 tx=0x7f134400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f134affd640 1 -- 172.21.15.94:0/3991310869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400cd80 con 0x7f1354103c60 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f134affd640 1 -- 172.21.15.94:0/3991310869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13440054d0 con 0x7f1354103c60 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541a20b0 con 0x7f1354103c60 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f134affd640 1 -- 172.21.15.94:0/3991310869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344010690 con 0x7f1354103c60 2024-04-25T13:53:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.772+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13541a25b0 con 0x7f1354103c60 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.774+0000 7f134affd640 1 -- 172.21.15.94:0/3991310869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1344004e90 con 0x7f1354103c60 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 Processor -- start 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 -- start start 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce72eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce735df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dcce694460 con 0x55dcce3652f0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dcce64e590 con 0x55dcce36d4d0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1362c63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcce690420 con 0x55dcce38e1b0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f135934d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce735df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f135a34f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f1359b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce72eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f135a34f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:44506/0 (socket says 172.21.15.94:44506) 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f135a34f640 1 -- 172.21.15.94:0/2269418089 learned_addr learned my addr 172.21.15.94:0/2269418089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.775+0000 7f134affd640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 0x7f13400b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 -- 172.21.15.94:0/2269418089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 msgr2=0x55dcce735df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce735df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 -- 172.21.15.94:0/2269418089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 msgr2=0x55dcce72eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce72eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 -- 172.21.15.94:0/2269418089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcce65b0d0 con 0x55dcce38e1b0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f134affd640 1 -- 172.21.15.94:0/3991310869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f13440df010 con 0x7f1354103c60 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 0x7f13400b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f134c004af0 tx=0x7f134c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c86337c8b244875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f133ffff640 1 -- 172.21.15.94:0/2269418089 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c042040 con 0x55dcce38e1b0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f135a34f640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 0x7f13400b1ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f134c04d000 tx=0x7f134c04e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f133ffff640 1 -- 172.21.15.94:0/2269418089 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f134c036070 con 0x55dcce38e1b0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.776+0000 7f133ffff640 1 -- 172.21.15.94:0/2269418089 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c02fbd0 con 0x55dcce38e1b0 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 -- 172.21.15.94:0/2269418089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 msgr2=0x55dcce72e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f134c004af0 tx=0x7f134c009800 comp rx=0 tx=0).stop 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 -- 172.21.15.94:0/2269418089 shutdown_connections 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce735df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce72eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 --2- 172.21.15.94:0/2269418089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce72e970 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 -- 172.21.15.94:0/2269418089 >> 172.21.15.94:0/2269418089 conn(0x55dcce382300 msgr2=0x55dcce72a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 -- 172.21.15.94:0/2269418089 shutdown_connections 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.777+0000 7f1362c63740 1 -- 172.21.15.94:0/2269418089 wait complete. 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 -- start start 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 Processor -- start 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce738680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dcce736840 con 0x55dcce36d4d0 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dcce61bb60 con 0x55dcce3652f0 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcce7369c0 con 0x55dcce38e1b0 2024-04-25T13:53:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1362c63740 10 client.0 _ll_register_callbacks cb 0x55dcce6402b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44112/0 (socket says 172.21.15.94:44112) 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135934d640 1 -- 172.21.15.94:0/1622653864 learned_addr learned my addr 172.21.15.94:0/1622653864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f1359b4e640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce738680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135a34f640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135934d640 1 -- 172.21.15.94:0/1622653864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 msgr2=0x55dcce73fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.778+0000 7f135934d640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f135934d640 1 -- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 msgr2=0x55dcce738680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f135934d640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce738680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f135934d640 1 -- 172.21.15.94:0/1622653864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcce3a6210 con 0x55dcce36d4d0 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f135a34f640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f1359b4e640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce738680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f135934d640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f134400c6a0 tx=0x7f1344009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7aec39a26d696a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f133d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440a6bc0 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f13440e10d0 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c036070 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1344005180 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f1362c63740 10 client.19740 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dcce472620 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.779+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f134c036070 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f13440a76b0 con 0x55dcce36d4d0 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 10 client.19740 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 10 client.19740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 20 client.19740 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce34cd80 0x55dcce746390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 4 client.19740 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcce6a2070 con 0x55dcce34cd80 2024-04-25T13:53:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1359b4e640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce34cd80 0x55dcce746390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f133d7fa640 10 client.19740 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.780+0000 7f1359b4e640 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce34cd80 0x55dcce746390 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319d8fc1781c48a0 server_cookie=acde9b26e1c08cc9 in_seq=0 out_seq=0 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f133d7fa640 1 -- 172.21.15.94:0/1622653864 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcce6a2070 con 0x55dcce34cd80 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f133d7fa640 10 client.19740 handle_command_reply: tid=0 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 client.19740 shutdown 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 msgr2=0x55dcce738bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f134400c6a0 tx=0x7f1344009310 comp rx=0 tx=0).stop 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 shutdown_connections 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce34cd80 0x55dcce746390 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce738bc0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 --2- 172.21.15.94:0/1622653864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce738680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f133d7fa640 0 client.19740 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 >> 172.21.15.94:0/1622653864 conn(0x55dcce382300 msgr2=0x55dcce729250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 shutdown_connections 2024-04-25T13:53:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 1 -- 172.21.15.94:0/1622653864 wait complete. 2024-04-25T13:53:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.783+0000 7f1362c63740 20 client.19740 trim_cache size 0 max 16384 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 Processor -- start 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 -- start start 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce72a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce728820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ab480 con 0x55dcce3652f0 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13541acfc0 con 0x55dcce36d4d0 2024-04-25T13:53:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1362c63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13541abb50 con 0x55dcce38e1b0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44124/0 (socket says 172.21.15.94:44124) 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 -- 172.21.15.94:0/301208464 learned_addr learned my addr 172.21.15.94:0/301208464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f135934d640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce72a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f135a34f640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce728820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 -- 172.21.15.94:0/301208464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 msgr2=0x55dcce728820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce728820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 -- 172.21.15.94:0/301208464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 msgr2=0x55dcce72a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.802+0000 7f1359b4e640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce72a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f1359b4e640 1 -- 172.21.15.94:0/301208464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcce72ae90 con 0x55dcce3652f0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f135934d640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce72a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f135a34f640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce728820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f1359b4e640 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f1354103620 tx=0x7f135000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef0109249e2306c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f133f7fe640 1 -- 172.21.15.94:0/301208464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135000fcf0 con 0x55dcce3652f0 2024-04-25T13:53:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f133f7fe640 1 -- 172.21.15.94:0/301208464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135000fe90 con 0x55dcce3652f0 2024-04-25T13:53:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.803+0000 7f133f7fe640 1 -- 172.21.15.94:0/301208464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350004dd0 con 0x55dcce3652f0 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- 172.21.15.94:0/301208464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 msgr2=0x55dcce729ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f1354103620 tx=0x7f135000f790 comp rx=0 tx=0).stop 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- 172.21.15.94:0/301208464 shutdown_connections 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce728820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce36d4d0 0x55dcce72a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 --2- 172.21.15.94:0/301208464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce3652f0 0x55dcce729ba0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- 172.21.15.94:0/301208464 >> 172.21.15.94:0/301208464 conn(0x55dcce382300 msgr2=0x55dcce732ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- 172.21.15.94:0/301208464 shutdown_connections 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- 172.21.15.94:0/301208464 wait complete. 2024-04-25T13:53:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 -- start start 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.804+0000 7f1362c63740 1 Processor -- start 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce73a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ac8f0 con 0x55dcce36d4d0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ac220 con 0x55dcce3652f0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13541a9460 con 0x55dcce38e1b0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1362c63740 10 client.0 _ll_register_callbacks cb 0x7f13440e27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce73a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f135934d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44130/0 (socket says 172.21.15.94:44130) 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f135934d640 1 -- 172.21.15.94:0/3625259852 learned_addr learned my addr 172.21.15.94:0/3625259852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f135a34f640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 -- 172.21.15.94:0/3625259852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 msgr2=0x55dcce73dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 -- 172.21.15.94:0/3625259852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 msgr2=0x55dcce73ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f1359b4e640 1 -- 172.21.15.94:0/3625259852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcce73e290 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.805+0000 7f135934d640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1359b4e640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce73a850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55dcce742ff0 tx=0x7f135000fcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fdb7b4a45bd54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135002f020 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1350022170 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f135000fe90 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f135003a050 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 10 client.29189 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dcce7ae790 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135000fe90 con 0x55dcce3652f0 2024-04-25T13:53:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f135002b650 con 0x55dcce3652f0 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 10 client.29189 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 10 client.29189 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 20 client.29189 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce3221c0 0x55dcce7b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 4 client.29189 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.806+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcce69bee0 con 0x55dcce3221c0 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.807+0000 7f135934d640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce3221c0 0x55dcce7b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.807+0000 7f133d7fa640 10 client.29189 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:32.807+0000 7f135934d640 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce3221c0 0x55dcce7b0fb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d639a7ca6d8a8c44 server_cookie=e74e6f7effbdf71c in_seq=0 out_seq=0 2024-04-25T13:53:32.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1454810 12.58 MB/sec execute 495 sec latency 81.630 ms 2024-04-25T13:53:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.063+0000 7f133d7fa640 1 -- 172.21.15.94:0/3625259852 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcce69bee0 con 0x55dcce3221c0 2024-04-25T13:53:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.063+0000 7f133d7fa640 10 client.29189 handle_command_reply: tid=0 2024-04-25T13:53:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.063+0000 7f1362c63740 1 client.29189 shutdown 2024-04-25T13:53:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 msgr2=0x55dcce73a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce73a850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55dcce742ff0 tx=0x7f135000fcf0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 shutdown_connections 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcce3221c0 0x55dcce7b0fb0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcce38e1b0 0x55dcce73dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcce36d4d0 0x55dcce73ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 --2- 172.21.15.94:0/3625259852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcce3652f0 0x55dcce73a850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 >> 172.21.15.94:0/3625259852 conn(0x55dcce382300 msgr2=0x55dcce729ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f133d7fa640 0 client.29189 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 shutdown_connections 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 1 -- 172.21.15.94:0/3625259852 wait complete. 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.064+0000 7f1362c63740 20 client.29189 trim_cache size 0 max 16384 2024-04-25T13:53:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:53:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:53:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:53:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 msgr2=0x7f13400b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 0x7f13400b1ff0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f134c04d000 tx=0x7f134c04e000 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 msgr2=0x7f135419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f134400e9f0 tx=0x7f134400edf0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 shutdown_connections 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f13400afae0 0x7f13400b1ff0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354108960 0x7f13541a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354103c60 0x7f135419a7c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 --2- 172.21.15.94:0/3991310869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102a20 0x7f135419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 >> 172.21.15.94:0/3991310869 conn(0x7f13540fdc10 msgr2=0x7f1354109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 shutdown_connections 2024-04-25T13:53:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:33.065+0000 7f135bdb2640 1 -- 172.21.15.94:0/3991310869 wait complete. 2024-04-25T13:53:33.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1456990 12.58 MB/sec execute 496 sec latency 146.288 ms 2024-04-25T13:53:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:34 smithi137 ceph-mon[32568]: pgmap v702: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-04-25T13:53:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:34 smithi094 ceph-mon[29195]: pgmap v702: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-04-25T13:53:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:34 smithi138 ceph-mon[32977]: pgmap v702: 97 pgs: 97 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.7 MiB/s wr, 319 op/s 2024-04-25T13:53:34.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1459646 12.58 MB/sec execute 497 sec latency 18.325 ms 2024-04-25T13:53:35.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1461633 12.58 MB/sec execute 498 sec latency 49.003 ms 2024-04-25T13:53:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:36 smithi137 ceph-mon[32568]: pgmap v703: 97 pgs: 97 active+clean; 34 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-04-25T13:53:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:36 smithi094 ceph-mon[29195]: pgmap v703: 97 pgs: 97 active+clean; 34 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-04-25T13:53:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:36 smithi138 ceph-mon[32977]: pgmap v703: 97 pgs: 97 active+clean; 34 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-04-25T13:53:36.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1464185 12.58 MB/sec execute 499 sec latency 16.993 ms 2024-04-25T13:53:37.818 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1466329 12.57 MB/sec execute 500 sec latency 60.840 ms 2024-04-25T13:53:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:38 smithi137 ceph-mon[32568]: pgmap v704: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-04-25T13:53:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:38 smithi094 ceph-mon[29195]: pgmap v704: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-04-25T13:53:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:38 smithi138 ceph-mon[32977]: pgmap v704: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-04-25T13:53:38.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1468093 12.57 MB/sec execute 501 sec latency 61.114 ms 2024-04-25T13:53:39.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1471075 12.58 MB/sec execute 502 sec latency 11.477 ms 2024-04-25T13:53:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:40 smithi137 ceph-mon[32568]: pgmap v705: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T13:53:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:40 smithi094 ceph-mon[29195]: pgmap v705: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T13:53:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:40 smithi138 ceph-mon[32977]: pgmap v705: 97 pgs: 97 active+clean; 35 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T13:53:40.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1473407 12.57 MB/sec execute 503 sec latency 12.744 ms 2024-04-25T13:53:41.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1475548 12.58 MB/sec execute 504 sec latency 15.205 ms 2024-04-25T13:53:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:42 smithi137 ceph-mon[32568]: pgmap v706: 97 pgs: 97 active+clean; 38 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 389 op/s 2024-04-25T13:53:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:42 smithi094 ceph-mon[29195]: pgmap v706: 97 pgs: 97 active+clean; 38 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 389 op/s 2024-04-25T13:53:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:42 smithi138 ceph-mon[32977]: pgmap v706: 97 pgs: 97 active+clean; 38 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 389 op/s 2024-04-25T13:53:42.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1478212 12.58 MB/sec execute 505 sec latency 61.991 ms 2024-04-25T13:53:43.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1480428 12.57 MB/sec execute 506 sec latency 45.423 ms 2024-04-25T13:53:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:44 smithi137 ceph-mon[32568]: pgmap v707: 97 pgs: 97 active+clean; 41 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T13:53:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:44 smithi094 ceph-mon[29195]: pgmap v707: 97 pgs: 97 active+clean; 41 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T13:53:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:44 smithi138 ceph-mon[32977]: pgmap v707: 97 pgs: 97 active+clean; 41 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T13:53:44.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1482456 12.57 MB/sec execute 507 sec latency 16.476 ms 2024-04-25T13:53:45.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1485308 12.58 MB/sec execute 508 sec latency 39.366 ms 2024-04-25T13:53:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 -- 172.21.15.94:0/2310699651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed741098d0 msgr2=0x7fed7410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/2310699651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed741098d0 0x7fed7410bcf0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009ca0 tx=0x7fed6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 -- 172.21.15.94:0/2310699651 shutdown_connections 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/2310699651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed741098d0 0x7fed7410bcf0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/2310699651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74106f70 0x7fed74109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/2310699651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed74104610 0x7fed74106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 -- 172.21.15.94:0/2310699651 >> 172.21.15.94:0/2310699651 conn(0x7fed740fdc10 msgr2=0x7fed741000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 -- 172.21.15.94:0/2310699651 shutdown_connections 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.165+0000 7fed7b3b7640 1 -- 172.21.15.94:0/2310699651 wait complete. 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 Processor -- start 2024-04-25T13:53:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 -- start start 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 0x7fed7419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 0x7fed7419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741098d0 0x7fed741a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410f630 con 0x7fed74106f70 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410f4b0 con 0x7fed74104610 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed7b3b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed7410f7b0 con 0x7fed741098d0 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed78952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 0x7fed7419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed79153640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 0x7fed7419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed78952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 0x7fed7419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37882/0 (socket says 172.21.15.94:37882) 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed78952640 1 -- 172.21.15.94:0/3675056035 learned_addr learned my addr 172.21.15.94:0/3675056035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed79954640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741098d0 0x7fed741a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.166+0000 7fed79153640 1 -- 172.21.15.94:0/3675056035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741098d0 msgr2=0x7fed741a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed79153640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741098d0 0x7fed741a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed79153640 1 -- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 msgr2=0x7fed7419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed79153640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 0x7fed7419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed79153640 1 -- 172.21.15.94:0/3675056035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed741a1d40 con 0x7fed74104610 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed79153640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 0x7fed7419a200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed70009b90 tx=0x7fed7000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed6a7fc640 1 -- 172.21.15.94:0/3675056035 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7001e070 con 0x7fed74104610 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed6a7fc640 1 -- 172.21.15.94:0/3675056035 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed700198c0 con 0x7fed74104610 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed74102040 con 0x7fed74104610 2024-04-25T13:53:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed6a7fc640 1 -- 172.21.15.94:0/3675056035 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700187d0 con 0x7fed74104610 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.167+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed741025e0 con 0x7fed74104610 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.170+0000 7fed6a7fc640 1 -- 172.21.15.94:0/3675056035 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fed70021070 con 0x7fed74104610 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 Processor -- start 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 -- start start 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aaa92f0 0x55db2ae7b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 0x55db2ae82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db2add8460 con 0x55db2aad21b0 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db2ad92590 con 0x55db2aab14d0 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed8226d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db2add4420 con 0x55db2aaa92f0 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.172+0000 7fed6a7fc640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 0x7fed600b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37886/0 (socket says 172.21.15.94:37886) 2024-04-25T13:53:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 -- 172.21.15.94:0/1279954078 learned_addr learned my addr 172.21.15.94:0/1279954078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed78952640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 0x55db2ae82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79153640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aaa92f0 0x55db2ae7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed6a7fc640 1 -- 172.21.15.94:0/3675056035 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fed700e2440 con 0x7fed74104610 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 -- 172.21.15.94:0/1279954078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aaa92f0 msgr2=0x55db2ae7b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aaa92f0 0x55db2ae7b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 -- 172.21.15.94:0/1279954078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 msgr2=0x55db2ae82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 0x55db2ae82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 -- 172.21.15.94:0/1279954078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2ad9f0d0 con 0x55db2aad21b0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 0x7fed600b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed78952640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 0x55db2ae82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fed7406b4a0 tx=0x7fed6c009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76d276a80bc4156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed5f7fe640 1 -- 172.21.15.94:0/1279954078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c003970 con 0x55db2aad21b0 2024-04-25T13:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed79954640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 0x7fed600b2070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fed6c04e000 tx=0x7fed6c051000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed5f7fe640 1 -- 172.21.15.94:0/1279954078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6c036040 con 0x55db2aad21b0 2024-04-25T13:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.173+0000 7fed5f7fe640 1 -- 172.21.15.94:0/1279954078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c02f660 con 0x55db2aad21b0 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 -- 172.21.15.94:0/1279954078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 msgr2=0x55db2ae7b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fed7406b4a0 tx=0x7fed6c009470 comp rx=0 tx=0).stop 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 -- 172.21.15.94:0/1279954078 shutdown_connections 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aab14d0 0x55db2ae82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aaa92f0 0x55db2ae7b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 --2- 172.21.15.94:0/1279954078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aad21b0 0x55db2ae7b180 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 -- 172.21.15.94:0/1279954078 >> 172.21.15.94:0/1279954078 conn(0x55db2aac6300 msgr2=0x55db2ae6e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 -- 172.21.15.94:0/1279954078 shutdown_connections 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.174+0000 7fed8226d740 1 -- 172.21.15.94:0/1279954078 wait complete. 2024-04-25T13:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.175+0000 7fed8226d740 1 -- start start 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.175+0000 7fed8226d740 1 Processor -- start 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.175+0000 7fed8226d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aaa92f0 0x55db2ae84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aab14d0 0x55db2ae853d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55db2ae83050 con 0x55db2aab14d0 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55db2ad5fb60 con 0x55db2aaa92f0 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db2ae831d0 con 0x55db2aad21b0 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed8226d740 10 client.0 _ll_register_callbacks cb 0x55db2ad842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57872/0 (socket says 172.21.15.94:57872) 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 -- 172.21.15.94:0/1045102940 learned_addr learned my addr 172.21.15.94:0/1045102940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79153640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aaa92f0 0x55db2ae84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed78952640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aab14d0 0x55db2ae853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 -- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aaa92f0 msgr2=0x55db2ae84e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aaa92f0 0x55db2ae84e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 -- 172.21.15.94:0/1045102940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aab14d0 msgr2=0x55db2ae853d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aab14d0 0x55db2ae853d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 -- 172.21.15.94:0/1045102940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2aaea210 con 0x55db2aad21b0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed79954640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed6c033ff0 tx=0x7fed6c02f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2350b3ca8388a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.176+0000 7fed5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c004720 con 0x55db2aad21b0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6c036040 con 0x55db2aad21b0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fed6c003970 con 0x55db2aad21b0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed6c003bb0 con 0x55db2aad21b0 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed8226d740 10 client.28987 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55db2abb6620 con 0x55db2aad21b0 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6c036040 con 0x55db2aad21b0 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fed6c030890 con 0x55db2aad21b0 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed8226d740 10 client.28987 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.177+0000 7fed8226d740 10 client.28987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed8226d740 20 client.28987 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa90d80 0x55db2ae92a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed8226d740 4 client.28987 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db2ade6070 con 0x55db2aa90d80 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed79153640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa90d80 0x55db2ae92a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed5cff9640 10 client.28987 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.178+0000 7fed79153640 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa90d80 0x55db2ae92a80 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365aeb89bcd06ef2 server_cookie=9ae7cb6d641c4097 in_seq=0 out_seq=0 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.179+0000 7fed5cff9640 1 -- 172.21.15.94:0/1045102940 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db2ade6070 con 0x55db2aa90d80 2024-04-25T13:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.179+0000 7fed5cff9640 10 client.28987 handle_command_reply: tid=0 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 client.28987 shutdown 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 msgr2=0x55db2ae8c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed6c033ff0 tx=0x7fed6c02f310 comp rx=0 tx=0).stop 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 shutdown_connections 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa90d80 0x55db2ae92a80 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aad21b0 0x55db2ae8c330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aab14d0 0x55db2ae853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed5cff9640 0 client.28987 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 --2- 172.21.15.94:0/1045102940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aaa92f0 0x55db2ae84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 >> 172.21.15.94:0/1045102940 conn(0x55db2aac6300 msgr2=0x55db2ae71210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 shutdown_connections 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 1 -- 172.21.15.94:0/1045102940 wait complete. 2024-04-25T13:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.180+0000 7fed8226d740 20 client.28987 trim_cache size 0 max 16384 2024-04-25T13:53:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.196+0000 7fed8226d740 1 Processor -- start 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.196+0000 7fed8226d740 1 -- start start 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2ae8c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed741a92a0 con 0x55db2aaa92f0 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed741aade0 con 0x55db2aad21b0 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed8226d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed741a9970 con 0x55db2aab14d0 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37914/0 (socket says 172.21.15.94:37914) 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 -- 172.21.15.94:0/797997727 learned_addr learned my addr 172.21.15.94:0/797997727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed78952640 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79954640 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2ae8c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 -- 172.21.15.94:0/797997727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 msgr2=0x55db2ae987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 -- 172.21.15.94:0/797997727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 msgr2=0x55db2ae8c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2ae8c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 -- 172.21.15.94:0/797997727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2ae69760 con 0x55db2aaa92f0 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed79153640 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fed700a6950 tx=0x7fed700dfd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabaf00c02e34a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.197+0000 7fed5effd640 1 -- 172.21.15.94:0/797997727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700aebd0 con 0x55db2aaa92f0 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed5effd640 1 -- 172.21.15.94:0/797997727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed700b3680 con 0x55db2aaa92f0 2024-04-25T13:53:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed5effd640 1 -- 172.21.15.94:0/797997727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70019a70 con 0x55db2aaa92f0 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 -- 172.21.15.94:0/797997727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 msgr2=0x55db2ae98280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fed700a6950 tx=0x7fed700dfd10 comp rx=0 tx=0).stop 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 -- 172.21.15.94:0/797997727 shutdown_connections 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2ae8c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 --2- 172.21.15.94:0/797997727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae98280 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 -- 172.21.15.94:0/797997727 >> 172.21.15.94:0/797997727 conn(0x55db2aac6300 msgr2=0x55db2ae7f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.198+0000 7fed8226d740 1 -- 172.21.15.94:0/797997727 shutdown_connections 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 -- 172.21.15.94:0/797997727 wait complete. 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 -- start start 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 Processor -- start 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae8bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2aef8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.199+0000 7fed8226d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed741aa710 con 0x55db2aaa92f0 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed8226d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed741aa040 con 0x55db2aad21b0 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed8226d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed741a7280 con 0x55db2aab14d0 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed8226d740 10 client.0 _ll_register_callbacks cb 0x7fed700b1f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37922/0 (socket says 172.21.15.94:37922) 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 -- 172.21.15.94:0/187061371 learned_addr learned my addr 172.21.15.94:0/187061371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79954640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2aef8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed78952640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae8bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 -- 172.21.15.94:0/187061371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 msgr2=0x55db2ae8bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae8bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 -- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 msgr2=0x55db2aef8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2aef8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 -- 172.21.15.94:0/187061371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2aef95b0 con 0x55db2aaa92f0 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed78952640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae8bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79954640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2aef8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed79153640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fed700cd010 tx=0x7fed700a6190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ebece8e4708033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700aebd0 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fed700cbad0 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed700b3680 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.200+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fed70019a70 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 10 client.19764 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55db2aefa840 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed700b3680 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fed700a9bc0 con 0x55db2aaa92f0 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 10 client.19764 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 10 client.19764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 20 client.19764 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa661c0 0x55db2aefd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 4 client.19764 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:53:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db2addfee0 con 0x55db2aa661c0 2024-04-25T13:53:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed78952640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa661c0 0x55db2aefd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.201+0000 7fed5cff9640 10 client.19764 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.202+0000 7fed78952640 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa661c0 0x55db2aefd200 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33b09a1850238d0 server_cookie=4dfe7fef47099eaf in_seq=0 out_seq=0 2024-04-25T13:53:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed5cff9640 1 -- 172.21.15.94:0/187061371 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db2addfee0 con 0x55db2aa661c0 2024-04-25T13:53:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed5cff9640 10 client.19764 handle_command_reply: tid=0 2024-04-25T13:53:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 client.19764 shutdown 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 msgr2=0x55db2ae8b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fed700cd010 tx=0x7fed700a6190 comp rx=0 tx=0).stop 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 shutdown_connections 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55db2aa661c0 0x55db2aefd200 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2aad21b0 0x55db2aef8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed5cff9640 0 client.19764 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db2aab14d0 0x55db2ae8bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 --2- 172.21.15.94:0/187061371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db2aaa92f0 0x55db2ae8b710 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 >> 172.21.15.94:0/187061371 conn(0x55db2aac6300 msgr2=0x55db2ae71100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 shutdown_connections 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 1 -- 172.21.15.94:0/187061371 wait complete. 2024-04-25T13:53:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.397+0000 7fed8226d740 20 client.19764 trim_cache size 0 max 16384 2024-04-25T13:53:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:53:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:53:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:53:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:53:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 msgr2=0x7fed600b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 0x7fed600b2070 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fed6c04e000 tx=0x7fed6c051000 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 msgr2=0x7fed7419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 0x7fed7419a200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed70009b90 tx=0x7fed7000b690 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 shutdown_connections 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed600afb60 0x7fed600b2070 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741098d0 0x7fed741a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed74106f70 0x7fed7419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 --2- 172.21.15.94:0/3675056035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74104610 0x7fed7419a200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 >> 172.21.15.94:0/3675056035 conn(0x7fed740fdc10 msgr2=0x7fed74105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 shutdown_connections 2024-04-25T13:53:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:46.398+0000 7fed7b3b7640 1 -- 172.21.15.94:0/3675056035 wait complete. 2024-04-25T13:53:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:46 smithi137 ceph-mon[32568]: pgmap v708: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 406 op/s 2024-04-25T13:53:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:46 smithi094 ceph-mon[29195]: pgmap v708: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 406 op/s 2024-04-25T13:53:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:46 smithi138 ceph-mon[32977]: pgmap v708: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.4 MiB/s wr, 406 op/s 2024-04-25T13:53:46.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1487391 12.57 MB/sec execute 509 sec latency 68.691 ms 2024-04-25T13:53:47.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1489122 12.57 MB/sec execute 510 sec latency 70.038 ms 2024-04-25T13:53:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:48 smithi137 ceph-mon[32568]: pgmap v709: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-04-25T13:53:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:48 smithi094 ceph-mon[29195]: pgmap v709: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-04-25T13:53:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:48 smithi138 ceph-mon[32977]: pgmap v709: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2024-04-25T13:53:48.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1492042 12.58 MB/sec execute 511 sec latency 19.992 ms 2024-04-25T13:53:48.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:53:48.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:53:49.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3327090903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04104b70 msgr2=0x7f6d04104f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3327090903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04104b70 0x7f6d04104f80 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf402a410 comp rx=0 tx=0).stop 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3327090903 shutdown_connections 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3327090903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 0x7f6d0410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3327090903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04104b70 0x7f6d04104f80 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3327090903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04103930 0x7f6d04104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.012+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3327090903 >> 172.21.15.94:0/3327090903 conn(0x7f6d040feb20 msgr2=0x7f6d04100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3327090903 shutdown_connections 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3327090903 wait complete. 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 Processor -- start 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- start start 2024-04-25T13:53:49.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04104b70 0x7f6d0419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 0x7f6d041a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f450 con 0x7f6d04103930 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f2d0 con 0x7f6d04109870 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0b3ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f5d0 con 0x7f6d04104b70 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0a3e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.013+0000 7f6d0a3e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37946/0 (socket says 172.21.15.94:37946) 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/3032607189 learned_addr learned my addr 172.21.15.94:0/3032607189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d09be7640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04104b70 0x7f6d0419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 0x7f6d041a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/3032607189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04104b70 msgr2=0x7f6d0419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04104b70 0x7f6d0419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/3032607189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 msgr2=0x7f6d041a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 0x7f6d041a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/3032607189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d041a1b60 con 0x7f6d04103930 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000e9f0 tx=0x7f6d0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6cfb7fe640 1 -- 172.21.15.94:0/3032607189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000ccc0 con 0x7f6d04103930 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6cfb7fe640 1 -- 172.21.15.94:0/3032607189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d000035f0 con 0x7f6d04103930 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041a1e50 con 0x7f6d04103930 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6cfb7fe640 1 -- 172.21.15.94:0/3032607189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000ce60 con 0x7f6d04103930 2024-04-25T13:53:49.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.014+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a22b0 con 0x7f6d04103930 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 Processor -- start 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6cfb7fe640 1 -- 172.21.15.94:0/3032607189 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6d00003aa0 con 0x7f6d04103930 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 -- start start 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d465aac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d4650fd80 con 0x559d461e86b0 2024-04-25T13:53:49.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559d464c98b0 con 0x559d46209390 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d135dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d4650bd40 con 0x559d461e04d0 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d09be7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d0abe9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d465aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d09be7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37948/0 (socket says 172.21.15.94:37948) 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d09be7640 1 -- 172.21.15.94:0/2089352900 learned_addr learned my addr 172.21.15.94:0/2089352900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.017+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 -- 172.21.15.94:0/2089352900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 msgr2=0x559d465ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 -- 172.21.15.94:0/2089352900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 msgr2=0x559d465aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d465aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 -- 172.21.15.94:0/2089352900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4649aa40 con 0x559d461e86b0 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d465aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6cfb7fe640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 0x7f6cf00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d09be7640 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4031500 tx=0x7f6cf4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38a56c33065a2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 0x7f6cf00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6cfb7fe640 1 -- 172.21.15.94:0/3032607189 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6d000deff0 con 0x7f6d04103930 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6cf8bfe640 1 -- 172.21.15.94:0/2089352900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4035030 con 0x559d461e86b0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6cf8bfe640 1 -- 172.21.15.94:0/2089352900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf4040070 con 0x559d461e86b0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.018+0000 7f6cf8bfe640 1 -- 172.21.15.94:0/2089352900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4003970 con 0x559d461e86b0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 -- 172.21.15.94:0/2089352900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 msgr2=0x559d465abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 0x7f6cf00b1ff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x559d465abca0 tx=0x7f6cfc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4031500 tx=0x7f6cf4009620 comp rx=0 tx=0).stop 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 -- 172.21.15.94:0/2089352900 shutdown_connections 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e86b0 0x559d465abe60 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 --2- 172.21.15.94:0/2089352900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d465aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 -- 172.21.15.94:0/2089352900 >> 172.21.15.94:0/2089352900 conn(0x559d461fd4e0 msgr2=0x559d465a6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 -- 172.21.15.94:0/2089352900 shutdown_connections 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.019+0000 7f6d135dc740 1 -- 172.21.15.94:0/2089352900 wait complete. 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 -- start start 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 Processor -- start 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d465ac910 con 0x559d46209390 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559d46496d10 con 0x559d461e86b0 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d465aca90 con 0x559d461e04d0 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d135dc740 10 client.0 _ll_register_callbacks cb 0x559d464c3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d0abe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d0abe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37956/0 (socket says 172.21.15.94:37956) 2024-04-25T13:53:49.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d0abe9640 1 -- 172.21.15.94:0/1347704515 learned_addr learned my addr 172.21.15.94:0/1347704515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 -- 172.21.15.94:0/1347704515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 msgr2=0x559d465b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 -- 172.21.15.94:0/1347704515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 msgr2=0x559d465bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d09be7640 1 -- 172.21.15.94:0/1347704515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d462eb7d0 con 0x559d461e86b0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.020+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465b5040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005ab0 tx=0x7f6cf40317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f321c9a2953c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4053060 con 0x559d461e86b0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf4040070 con 0x559d461e86b0 2024-04-25T13:53:49.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6cf40036f0 con 0x559d461e86b0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6cf4035070 con 0x559d461e86b0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 10 client.29216 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d462213f0 con 0x559d461e86b0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf4040070 con 0x559d461e86b0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6cf404d030 con 0x559d461e86b0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 10 client.29216 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 10 client.29216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 20 client.29216 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d461c7f60 0x559d465c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 4 client.29216 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.021+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d4651d990 con 0x559d461c7f60 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.022+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d461c7f60 0x559d465c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.022+0000 7f6cedffb640 10 client.29216 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:49.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.022+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d461c7f60 0x559d465c2060 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35c47e7717bb91f server_cookie=aae62a662e3d2ff9 in_seq=0 out_seq=0 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6cedffb640 1 -- 172.21.15.94:0/1347704515 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d4651d990 con 0x559d461c7f60 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6cedffb640 10 client.29216 handle_command_reply: tid=0 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 client.29216 shutdown 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 msgr2=0x559d465b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465b5040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005ab0 tx=0x7f6cf40317d0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 shutdown_connections 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d461c7f60 0x559d465c2060 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465b5040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1347704515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6cedffb640 0 client.29216 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 >> 172.21.15.94:0/1347704515 conn(0x559d461fd4e0 msgr2=0x559d465a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 shutdown_connections 2024-04-25T13:53:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 1 -- 172.21.15.94:0/1347704515 wait complete. 2024-04-25T13:53:49.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.026+0000 7f6d135dc740 20 client.29216 trim_cache size 0 max 16384 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.050+0000 7f6d135dc740 1 Processor -- start 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 -- start start 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ab2b0 con 0x559d46209390 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041acdf0 con 0x559d461e86b0 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d135dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ab980 con 0x559d461e04d0 2024-04-25T13:53:49.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37964/0 (socket says 172.21.15.94:37964) 2024-04-25T13:53:49.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0a3e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d09be7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 -- 172.21.15.94:0/3750889558 learned_addr learned my addr 172.21.15.94:0/3750889558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 -- 172.21.15.94:0/3750889558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 msgr2=0x559d465aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 -- 172.21.15.94:0/3750889558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 msgr2=0x559d465a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.051+0000 7f6d0abe9640 1 -- 172.21.15.94:0/3750889558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d461e10f0 con 0x559d46209390 2024-04-25T13:53:49.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.052+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003ba0 tx=0x7f6cfc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a43ee779ac6448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.052+0000 7f6ceffff640 1 -- 172.21.15.94:0/3750889558 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004100 con 0x559d46209390 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.052+0000 7f6ceffff640 1 -- 172.21.15.94:0/3750889558 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc02b020 con 0x559d46209390 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.052+0000 7f6ceffff640 1 -- 172.21.15.94:0/3750889558 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc027b40 con 0x559d46209390 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 -- 172.21.15.94:0/3750889558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 msgr2=0x559d465aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc003ba0 tx=0x7f6cfc01a000 comp rx=0 tx=0).stop 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 -- 172.21.15.94:0/3750889558 shutdown_connections 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d46209390 0x559d465aabd0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d461e86b0 0x559d465a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 --2- 172.21.15.94:0/3750889558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e04d0 0x559d465aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.053+0000 7f6d135dc740 1 -- 172.21.15.94:0/3750889558 >> 172.21.15.94:0/3750889558 conn(0x559d461fd4e0 msgr2=0x559d465b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- 172.21.15.94:0/3750889558 shutdown_connections 2024-04-25T13:53:49.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- 172.21.15.94:0/3750889558 wait complete. 2024-04-25T13:53:49.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- start start 2024-04-25T13:53:49.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 Processor -- start 2024-04-25T13:53:49.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e86b0 0x559d465bb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d46628820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ac720 con 0x559d461e04d0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041ac050 con 0x559d46209390 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041a9290 con 0x559d461e86b0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d135dc740 10 client.0 _ll_register_callbacks cb 0x7f6cfc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d0a3e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d0a3e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37976/0 (socket says 172.21.15.94:37976) 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/1985223676 learned_addr learned my addr 172.21.15.94:0/1985223676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.054+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e86b0 0x559d465bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0abe9640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d46628820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/1985223676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e86b0 msgr2=0x559d465bb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e86b0 0x559d465bb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 msgr2=0x559d46628820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d46628820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 -- 172.21.15.94:0/1985223676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d464d04b0 con 0x559d461e04d0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d0a3e8640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a34b0 tx=0x7f6d000a6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b9e32b90fc3c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d000a7b30 con 0x559d461e04d0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6d000037a0 con 0x559d461e04d0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.055+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc02b020 con 0x559d461e04d0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d000a7430 con 0x559d461e04d0 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 10 client.19782 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d46629f70 con 0x559d461e04d0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc02b020 con 0x559d461e04d0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6d000e1140 con 0x559d461e04d0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 10 client.19782 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 10 client.19782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 20 client.19782 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d4619d3a0 0x559d4662c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 4 client.19782 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d46517800 con 0x559d4619d3a0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.056+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d4619d3a0 0x559d4662c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.057+0000 7f6cedffb640 10 client.19782 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.057+0000 7f6d09be7640 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d4619d3a0 0x559d4662c930 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702524b36c87385 server_cookie=b289f776b585f014 in_seq=0 out_seq=0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6cedffb640 1 -- 172.21.15.94:0/1985223676 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d46517800 con 0x559d4619d3a0 2024-04-25T13:53:49.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6cedffb640 10 client.19782 handle_command_reply: tid=0 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 client.19782 shutdown 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 msgr2=0x559d465bacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a34b0 tx=0x7f6d000a6c60 comp rx=0 tx=0).stop 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 shutdown_connections 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559d4619d3a0 0x559d4662c930 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d46209390 0x559d46628820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d461e86b0 0x559d465bb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6cedffb640 0 client.19782 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 --2- 172.21.15.94:0/1985223676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d461e04d0 0x559d465bacc0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 >> 172.21.15.94:0/1985223676 conn(0x559d461fd4e0 msgr2=0x559d465a9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.058+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 shutdown_connections 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.059+0000 7f6d135dc740 1 -- 172.21.15.94:0/1985223676 wait complete. 2024-04-25T13:53:49.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.059+0000 7f6d135dc740 20 client.19782 trim_cache size 0 max 16384 2024-04-25T13:53:49.062 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:49.062 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:53:49.062 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:53:49.062 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 msgr2=0x7f6cf00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 0x7f6cf00b1ff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x559d465abca0 tx=0x7f6cfc009500 comp rx=0 tx=0).stop 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 msgr2=0x7f6d0419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000e9f0 tx=0x7f6d0000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 shutdown_connections 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6cf00afae0 0x7f6cf00b1ff0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d04109870 0x7f6d041a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d04104b70 0x7f6d0419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 --2- 172.21.15.94:0/3032607189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d04103930 0x7f6d0419a020 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 >> 172.21.15.94:0/3032607189 conn(0x7f6d040feb20 msgr2=0x7f6d04105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 shutdown_connections 2024-04-25T13:53:49.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.060+0000 7f6d0b3ea640 1 -- 172.21.15.94:0/3032607189 wait complete. 2024-04-25T13:53:49.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:53:49.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:53:49.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 -- 172.21.15.94:0/2962322644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 msgr2=0x7f03441036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 --2- 172.21.15.94:0/2962322644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f03441036c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402a410 comp rx=0 tx=0).stop 2024-04-25T13:53:49.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 -- 172.21.15.94:0/2962322644 shutdown_connections 2024-04-25T13:53:49.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 --2- 172.21.15.94:0/2962322644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 0x7f034410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 --2- 172.21.15.94:0/2962322644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 0x7f0344104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 --2- 172.21.15.94:0/2962322644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f03441036c0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 -- 172.21.15.94:0/2962322644 >> 172.21.15.94:0/2962322644 conn(0x7f03440fdbd0 msgr2=0x7f0344100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.187+0000 7f034984c640 1 -- 172.21.15.94:0/2962322644 shutdown_connections 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 -- 172.21.15.94:0/2962322644 wait complete. 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 Processor -- start 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 -- start start 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 0x7f034419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 0x7f03441a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e4c0 con 0x7f03441029c0 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e340 con 0x7f0344108900 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.188+0000 7f034984c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f034410e640 con 0x7f0344103c00 2024-04-25T13:53:49.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38002/0 (socket says 172.21.15.94:38002) 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 -- 172.21.15.94:0/615872977 learned_addr learned my addr 172.21.15.94:0/615872977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f03427fc640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 0x7f034419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f03437fe640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 0x7f03441a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 -- 172.21.15.94:0/615872977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 msgr2=0x7f034419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 0x7f034419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 -- 172.21.15.94:0/615872977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 msgr2=0x7f03441a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 0x7f03441a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 -- 172.21.15.94:0/615872977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03441a1d50 con 0x7f03441029c0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f0342ffd640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f0334006010 tx=0x7f0334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033403f030 con 0x7f03441029c0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a1fe0 con 0x7f03441029c0 2024-04-25T13:53:49.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0334003aa0 con 0x7f03441029c0 2024-04-25T13:53:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334035070 con 0x7f03441029c0 2024-04-25T13:53:49.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.189+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a23e0 con 0x7f03441029c0 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.190+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0344107710 con 0x7f03441029c0 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.192+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0334042030 con 0x7f03441029c0 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.193+0000 7f034884a640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 0x7f032c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.193+0000 7f03427fc640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 0x7f032c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.193+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f03340ff690 con 0x7f03441029c0 2024-04-25T13:53:49.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.193+0000 7f03427fc640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 0x7f032c0b2000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f034406a2a0 tx=0x7f033800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.197+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03340c7b20 con 0x7f03441029c0 2024-04-25T13:53:49.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.435+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f034402db50 con 0x7f03441029c0 2024-04-25T13:53:49.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.436+0000 7f034884a640 1 -- 172.21.15.94:0/615872977 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11941 (secure 0 0 0) 0x7f03340cbe20 con 0x7f03441029c0 2024-04-25T13:53:49.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:53:49.435 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:52:14.280433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:53:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-25T13:53:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 msgr2=0x7f032c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 0x7f032c0b2000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f034406a2a0 tx=0x7f033800b390 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 msgr2=0x7f034406a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f0334006010 tx=0x7f0334004270 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 shutdown_connections 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f032c0afaf0 0x7f032c0b2000 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0344108900 0x7f03441a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0344103c00 0x7f034419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 --2- 172.21.15.94:0/615872977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03441029c0 0x7f034406a900 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 >> 172.21.15.94:0/615872977 conn(0x7f03440fdbd0 msgr2=0x7f03441048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 shutdown_connections 2024-04-25T13:53:49.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.438+0000 7f034984c640 1 -- 172.21.15.94:0/615872977 wait complete. 2024-04-25T13:53:49.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 -- 172.21.15.94:0/278504708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4103c20 msgr2=0x7f9be4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/278504708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4103c20 0x7f9be4104030 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009900 tx=0x7f9bcc02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 -- 172.21.15.94:0/278504708 shutdown_connections 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/278504708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4108920 0x7f9be410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/278504708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4103c20 0x7f9be4104030 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/278504708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 0x7f9be41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 -- 172.21.15.94:0/278504708 >> 172.21.15.94:0/278504708 conn(0x7f9be40fdbd0 msgr2=0x7f9be4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 -- 172.21.15.94:0/278504708 shutdown_connections 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 -- 172.21.15.94:0/278504708 wait complete. 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.568+0000 7f9be8f0d640 1 Processor -- start 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 -- start start 2024-04-25T13:53:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 0x7f9be4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 0x7f9be41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410e500 con 0x7f9be4108920 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410e380 con 0x7f9be4103c20 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be8f0d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9be410e680 con 0x7f9be41029e0 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38010/0 (socket says 172.21.15.94:38010) 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 -- 172.21.15.94:0/1714345783 learned_addr learned my addr 172.21.15.94:0/1714345783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 0x7f9be41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be259c640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 0x7f9be4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 -- 172.21.15.94:0/1714345783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 msgr2=0x7f9be4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 0x7f9be4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 -- 172.21.15.94:0/1714345783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 msgr2=0x7f9be41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 0x7f9be41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 -- 172.21.15.94:0/1714345783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be41a0fa0 con 0x7f9be4108920 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 0x7f9be41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9be2d9d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00eb10 tx=0x7f9bdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.569+0000 7f9bdb7fe640 1 -- 172.21.15.94:0/1714345783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004b20 con 0x7f9be4108920 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.570+0000 7f9bdb7fe640 1 -- 172.21.15.94:0/1714345783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bdc004cc0 con 0x7f9be4108920 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.570+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be41a1290 con 0x7f9be4108920 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.570+0000 7f9bdb7fe640 1 -- 172.21.15.94:0/1714345783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010670 con 0x7f9be4108920 2024-04-25T13:53:49.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.570+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be41a1790 con 0x7f9be4108920 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.572+0000 7f9befdc2740 1 Processor -- start 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.572+0000 7f9bdb7fe640 1 -- 172.21.15.94:0/1714345783 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9bdc005020 con 0x7f9be4108920 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.572+0000 7f9befdc2740 1 -- start start 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da56390 0x55f50ddfaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50ddfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f50dd5cd80 con 0x55f50da356b0 2024-04-25T13:53:49.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f50dd16a20 con 0x55f50da56390 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9befdc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f50dd58d40 con 0x55f50da2d4d0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38012/0 (socket says 172.21.15.94:38012) 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 -- 172.21.15.94:0/2522497747 learned_addr learned my addr 172.21.15.94:0/2522497747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9be259c640 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50ddfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da56390 0x55f50ddfaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdb7fe640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 0x7f9bd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 -- 172.21.15.94:0/2522497747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 msgr2=0x55f50ddfafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50ddfafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9be2d9d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 0x7f9bd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 -- 172.21.15.94:0/2522497747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da56390 msgr2=0x55f50ddfaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da56390 0x55f50ddfaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 -- 172.21.15.94:0/2522497747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50dd21c30 con 0x55f50da356b0 2024-04-25T13:53:49.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdb7fe640 1 -- 172.21.15.94:0/1714345783 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9bdc0df330 con 0x7f9be4108920 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.573+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x55f50ddfba70 tx=0x7f9bcc032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a25903527a92791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9be2d9d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 0x7f9bd00b2000 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc018000 tx=0x7f9bdc09c0b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9bd8ff9640 1 -- 172.21.15.94:0/2522497747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc035040 con 0x55f50da356b0 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9bd8ff9640 1 -- 172.21.15.94:0/2522497747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc004230 con 0x55f50da356b0 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9bd8ff9640 1 -- 172.21.15.94:0/2522497747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc03d450 con 0x55f50da356b0 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9befdc2740 1 -- 172.21.15.94:0/2522497747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 msgr2=0x55f50ddfbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.574+0000 7f9befdc2740 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x55f50ddfba70 tx=0x7f9bcc032ed0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- 172.21.15.94:0/2522497747 shutdown_connections 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da356b0 0x55f50ddfbc30 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50ddfafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- 172.21.15.94:0/2522497747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da56390 0x55f50ddfaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- 172.21.15.94:0/2522497747 >> 172.21.15.94:0/2522497747 conn(0x55f50da4a4e0 msgr2=0x55f50ddf4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- 172.21.15.94:0/2522497747 shutdown_connections 2024-04-25T13:53:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- 172.21.15.94:0/2522497747 wait complete. 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- start start 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 Processor -- start 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da56390 0x55f50de09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f50ddfc6e0 con 0x55f50da2d4d0 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f50dce3dc0 con 0x55f50da356b0 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f50ddfc860 con 0x55f50da56390 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9befdc2740 10 client.0 _ll_register_callbacks cb 0x55f50dcff600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38028/0 (socket says 172.21.15.94:38028) 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9bdbfff640 1 -- 172.21.15.94:0/745111909 learned_addr learned my addr 172.21.15.94:0/745111909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.575+0000 7f9be259c640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da56390 0x55f50de09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 -- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da56390 msgr2=0x55f50de09a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da56390 0x55f50de09a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 -- 172.21.15.94:0/745111909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 msgr2=0x55f50de02ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 -- 172.21.15.94:0/745111909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50da6e3f0 con 0x55f50da2d4d0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9be259c640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0096c0 tx=0x7f9bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e44f1e5c7b761a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc041020 con 0x55f50da2d4d0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9bcc003ef0 con 0x55f50da2d4d0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc004230 con 0x55f50da2d4d0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc035070 con 0x55f50da2d4d0 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9befdc2740 10 client.19800 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f50db387d0 con 0x55f50da2d4d0 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.576+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc004230 con 0x55f50da2d4d0 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9bcc054030 con 0x55f50da2d4d0 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 10 client.19800 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 10 client.19800 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 20 client.19800 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da14f60 0x55f50de0fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 4 client.19800 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50dd6a990 con 0x55f50da14f60 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.577+0000 7f9be259c640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da14f60 0x55f50de0fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9be259c640 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da14f60 0x55f50de0fe60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69d35cbdbf8f7c12 server_cookie=8badee71e14d9dc6 in_seq=0 out_seq=0 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9bc9ffb640 10 client.19800 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/745111909 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f50dd6a990 con 0x55f50da14f60 2024-04-25T13:53:49.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9bc9ffb640 10 client.19800 handle_command_reply: tid=0 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 client.19800 shutdown 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 msgr2=0x55f50de02570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0096c0 tx=0x7f9bcc0046c0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 shutdown_connections 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da14f60 0x55f50de0fe60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da56390 0x55f50de09a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9bc9ffb640 0 client.19800 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 --2- 172.21.15.94:0/745111909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de02570 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.578+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 >> 172.21.15.94:0/745111909 conn(0x55f50da4a4e0 msgr2=0x55f50ddf2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.579+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 shutdown_connections 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.579+0000 7f9befdc2740 1 -- 172.21.15.94:0/745111909 wait complete. 2024-04-25T13:53:49.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.579+0000 7f9befdc2740 20 client.19800 trim_cache size 0 max 16384 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 Processor -- start 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 -- start start 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50de099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41a93b0 con 0x55f50da14f60 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41aaef0 con 0x55f50da356b0 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9befdc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41a9a80 con 0x55f50da2d4d0 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38030/0 (socket says 172.21.15.94:38030) 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9be259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50de099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 -- 172.21.15.94:0/1631320363 learned_addr learned my addr 172.21.15.94:0/1631320363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 -- 172.21.15.94:0/1631320363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 msgr2=0x55f50de099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50de099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 -- 172.21.15.94:0/1631320363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 msgr2=0x55f50de02140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 -- 172.21.15.94:0/1631320363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50ddfde80 con 0x55f50da14f60 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.593+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0319d0 tx=0x7f9bcc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e81d05f9f5e4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.594+0000 7f9bcbfff640 1 -- 172.21.15.94:0/1631320363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc041040 con 0x55f50da14f60 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.594+0000 7f9bcbfff640 1 -- 172.21.15.94:0/1631320363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc04f040 con 0x55f50da14f60 2024-04-25T13:53:49.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.594+0000 7f9bcbfff640 1 -- 172.21.15.94:0/1631320363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc000e60 con 0x55f50da14f60 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.594+0000 7f9befdc2740 1 -- 172.21.15.94:0/1631320363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 msgr2=0x55f50ddf4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.594+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0319d0 tx=0x7f9bcc02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- 172.21.15.94:0/1631320363 shutdown_connections 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de02140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da2d4d0 0x55f50de099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1631320363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da14f60 0x55f50ddf4160 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- 172.21.15.94:0/1631320363 >> 172.21.15.94:0/1631320363 conn(0x55f50da4a4e0 msgr2=0x55f50ddf3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- 172.21.15.94:0/1631320363 shutdown_connections 2024-04-25T13:53:49.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- 172.21.15.94:0/1631320363 wait complete. 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- start start 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 Processor -- start 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da14f60 0x55f50de06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de75fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41aa820 con 0x55f50da2d4d0 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41aa150 con 0x55f50da356b0 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41a7390 con 0x55f50da14f60 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9befdc2740 10 client.0 _ll_register_callbacks cb 0x7f9bcc0315a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9be259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9be259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38042/0 (socket says 172.21.15.94:38042) 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9be259c640 1 -- 172.21.15.94:0/1953048441 learned_addr learned my addr 172.21.15.94:0/1953048441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de75fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.595+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da14f60 0x55f50de06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 -- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da14f60 msgr2=0x55f50de06af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da14f60 0x55f50de06af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 -- 172.21.15.94:0/1953048441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 msgr2=0x55f50de75fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de75fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 -- 172.21.15.94:0/1953048441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50de764e0 con 0x55f50da2d4d0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9be259c640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9be41035e0 tx=0x7f9bd401e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27b96e015f1a524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4014b20 con 0x55f50da2d4d0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9bd4014cc0 con 0x55f50da2d4d0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc04f040 con 0x55f50da2d4d0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9bd4012450 con 0x55f50da2d4d0 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9befdc2740 10 client.19806 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f50de77810 con 0x55f50da2d4d0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.596+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc04f040 con 0x55f50da2d4d0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9bd4025070 con 0x55f50da2d4d0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 10 client.19806 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 10 client.19806 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 20 client.19806 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da56390 0x55f50de7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 4 client.19806 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f50dd64800 con 0x55f50da56390 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da56390 0x55f50de7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bc9ffb640 10 client.19806 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bdbfff640 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da56390 0x55f50de7a2d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8ddab1ddea69b7d server_cookie=f22f75b44af72e3c in_seq=0 out_seq=0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bc9ffb640 1 -- 172.21.15.94:0/1953048441 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f50dd64800 con 0x55f50da56390 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9bc9ffb640 10 client.19806 handle_command_reply: tid=0 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.597+0000 7f9befdc2740 1 client.19806 shutdown 2024-04-25T13:53:49.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 msgr2=0x55f50de07030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f9be41035e0 tx=0x7f9bd401e880 comp rx=0 tx=0).stop 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 shutdown_connections 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f50da56390 0x55f50de7a2d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f50da356b0 0x55f50de75fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f50da2d4d0 0x55f50de07030 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 --2- 172.21.15.94:0/1953048441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50da14f60 0x55f50de06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 >> 172.21.15.94:0/1953048441 conn(0x55f50da4a4e0 msgr2=0x55f50ddf6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9bc9ffb640 0 client.19806 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 shutdown_connections 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 1 -- 172.21.15.94:0/1953048441 wait complete. 2024-04-25T13:53:49.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9befdc2740 20 client.19806 trim_cache size 0 max 16384 2024-04-25T13:53:49.604 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 msgr2=0x7f9bd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 0x7f9bd00b2000 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc018000 tx=0x7f9bdc09c0b0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 msgr2=0x7f9be41a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00eb10 tx=0x7f9bdc00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 shutdown_connections 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9bd00afaf0 0x7f9bd00b2000 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9be4108920 0x7f9be41a0930 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9be4103c20 0x7f9be41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 --2- 172.21.15.94:0/1714345783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9be41029e0 0x7f9be4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.598+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 >> 172.21.15.94:0/1714345783 conn(0x7f9be40fdbd0 msgr2=0x7f9be4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.599+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 shutdown_connections 2024-04-25T13:53:49.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.599+0000 7f9be8f0d640 1 -- 172.21.15.94:0/1714345783 wait complete. 2024-04-25T13:53:49.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:53:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 -- 172.21.15.94:0/3448935178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 msgr2=0x7fcfc8102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 --2- 172.21.15.94:0/3448935178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc8102860 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009900 tx=0x7fcfbc02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 -- 172.21.15.94:0/3448935178 shutdown_connections 2024-04-25T13:53:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 --2- 172.21.15.94:0/3448935178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 0x7fcfc810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 --2- 172.21.15.94:0/3448935178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 0x7fcfc8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 --2- 172.21.15.94:0/3448935178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc8102860 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 -- 172.21.15.94:0/3448935178 >> 172.21.15.94:0/3448935178 conn(0x7fcfc80fd380 msgr2=0x7fcfc80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 -- 172.21.15.94:0/3448935178 shutdown_connections 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 -- 172.21.15.94:0/3448935178 wait complete. 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.725+0000 7fcfceef0640 1 Processor -- start 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 -- start start 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 0x7fcfc819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 0x7fcfc81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810e520 con 0x7fcfc8103f70 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810e3a0 con 0x7fcfc8104410 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfceef0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810e6a0 con 0x7fcfc8108ab0 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38076/0 (socket says 172.21.15.94:38076) 2024-04-25T13:53:49.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 -- 172.21.15.94:0/816157577 learned_addr learned my addr 172.21.15.94:0/816157577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 0x7fcfc819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 0x7fcfc81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 -- 172.21.15.94:0/816157577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 msgr2=0x7fcfc81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 0x7fcfc81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 -- 172.21.15.94:0/816157577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 msgr2=0x7fcfc819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 0x7fcfc819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 -- 172.21.15.94:0/816157577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc81a1d60 con 0x7fcfc8103f70 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006010 tx=0x7fcfbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfc5ffb640 1 -- 172.21.15.94:0/816157577 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc03f030 con 0x7fcfc8103f70 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.726+0000 7fcfc5ffb640 1 -- 172.21.15.94:0/816157577 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfbc031910 con 0x7fcfc8103f70 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.727+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc81a1ff0 con 0x7fcfc8103f70 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.727+0000 7fcfc5ffb640 1 -- 172.21.15.94:0/816157577 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc035070 con 0x7fcfc8103f70 2024-04-25T13:53:49.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.727+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc81a2400 con 0x7fcfc8103f70 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 Processor -- start 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfc5ffb640 1 -- 172.21.15.94:0/816157577 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcfbc042030 con 0x7fcfc8103f70 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 -- start start 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 0x5631940e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 0x5631940e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563194047d80 con 0x563193d206b0 2024-04-25T13:53:49.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563194001a20 con 0x563193d41390 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.729+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563194043d40 con 0x563193d184d0 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfcd48d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 0x5631940e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 0x5631940e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940e4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38090/0 (socket says 172.21.15.94:38090) 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfcd48d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 0x5631940e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38930/0 (socket says 172.21.15.94:38930) 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfc7fff640 1 -- 172.21.15.94:0/2323486306 learned_addr learned my addr 172.21.15.94:0/2323486306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2323486306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 msgr2=0x5631940e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 0x5631940e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2323486306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 msgr2=0x5631940e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2323486306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319400cc30 con 0x563193d184d0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfc5ffb640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 0x7fcfb80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 0x7fcfb80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfc5ffb640 1 -- 172.21.15.94:0/816157577 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcfbc0fefd0 con 0x7fcfc8103f70 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.730+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 0x5631940e3d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0fca00 tx=0x7fcfbc107320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40ba8c7a2c039c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2323486306 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc049070 con 0x563193d184d0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 0x7fcfb80b2000 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x5631940e4860 tx=0x7fcfc0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2323486306 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfbc0c6fd0 con 0x563193d184d0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2323486306 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0c3380 con 0x563193d184d0 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2323486306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 msgr2=0x5631940e3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 0x5631940e3d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0fca00 tx=0x7fcfbc107320 comp rx=0 tx=0).stop 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2323486306 shutdown_connections 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d184d0 0x5631940e3d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.731+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2323486306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d41390 0x5631940e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2323486306 >> 172.21.15.94:0/2323486306 conn(0x563193d354e0 msgr2=0x5631940df020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2323486306 shutdown_connections 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2323486306 wait complete. 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- start start 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 Processor -- start 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 0x5631940ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5631940e52f0 con 0x563193d184d0 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563193fcedc0 con 0x563193d206b0 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631940e5470 con 0x563193d41390 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfd5d9c740 10 client.0 _ll_register_callbacks cb 0x563193fea600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.732+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38106/0 (socket says 172.21.15.94:38106) 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2850238303 learned_addr learned my addr 172.21.15.94:0/2850238303 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 0x5631940ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2850238303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 msgr2=0x5631940f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 msgr2=0x5631940ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 0x5631940ed830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 -- 172.21.15.94:0/2850238303 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563193d593f0 con 0x563193d184d0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 0x5631940ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0b0050 tx=0x7fcfbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d492114c31169026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc049040 con 0x563193d184d0 2024-04-25T13:53:49.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcfbc0c7720 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfbc0c6fd0 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcfbc0c6c30 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfd5d9c740 10 client.19818 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.733+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563193e237d0 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfbc0c6fd0 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcfbc0cfde0 con 0x563193d184d0 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 10 client.19818 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 10 client.19818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 20 client.19818 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cfff60 0x5631940fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 4 client.19818 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563194055990 con 0x563193cfff60 2024-04-25T13:53:49.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cfff60 0x5631940fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfa7fff640 10 client.19818 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.734+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cfff60 0x5631940fac80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af0973d5ef04be98 server_cookie=8cc25a9e248f6799 in_seq=0 out_seq=0 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfa7fff640 1 -- 172.21.15.94:0/2850238303 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563194055990 con 0x563193cfff60 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfa7fff640 10 client.19818 handle_command_reply: tid=0 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 client.19818 shutdown 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 msgr2=0x5631940ed2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0b0050 tx=0x7fcfbc009260 comp rx=0 tx=0).stop 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 shutdown_connections 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cfff60 0x5631940fac80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d206b0 0x5631940ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/2850238303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d184d0 0x5631940ed2f0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfa7fff640 0 client.19818 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:49.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.735+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 >> 172.21.15.94:0/2850238303 conn(0x563193d354e0 msgr2=0x5631940e1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.736+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 shutdown_connections 2024-04-25T13:53:49.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.736+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/2850238303 wait complete. 2024-04-25T13:53:49.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.736+0000 7fcfd5d9c740 20 client.19818 trim_cache size 0 max 16384 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 Processor -- start 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 -- start start 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ab580 con 0x563193d206b0 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ad0c0 con 0x563193d184d0 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81abc50 con 0x563193d41390 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38116/0 (socket says 172.21.15.94:38116) 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1487057764 learned_addr learned my addr 172.21.15.94:0/1487057764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1487057764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 msgr2=0x5631940ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1487057764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 msgr2=0x5631940ecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940ecec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1487057764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631940e4430 con 0x563193d206b0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.750+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940ecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.751+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x5631940dfdb0 tx=0x7fcfb4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55492ab41801d21c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.751+0000 7fcfb27fc640 1 -- 172.21.15.94:0/1487057764 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4005220 con 0x563193d206b0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.751+0000 7fcfb27fc640 1 -- 172.21.15.94:0/1487057764 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb4004c50 con 0x563193d206b0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.751+0000 7fcfb27fc640 1 -- 172.21.15.94:0/1487057764 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40108f0 con 0x563193d206b0 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1487057764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 msgr2=0x5631940ed400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x5631940dfdb0 tx=0x7fcfb4005f90 comp rx=0 tx=0).stop 2024-04-25T13:53:49.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1487057764 shutdown_connections 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631940ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940ed400 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1487057764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940ecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1487057764 >> 172.21.15.94:0/1487057764 conn(0x563193d354e0 msgr2=0x5631940d9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1487057764 shutdown_connections 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1487057764 wait complete. 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 -- start start 2024-04-25T13:53:49.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.752+0000 7fcfd5d9c740 1 Processor -- start 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631941610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ac9f0 con 0x563193d206b0 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81ac320 con 0x563193d184d0 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81a9560 con 0x563193d41390 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 10 client.0 _ll_register_callbacks cb 0x7fcfb40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38128/0 (socket says 172.21.15.94:38128) 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1002530346 learned_addr learned my addr 172.21.15.94:0/1002530346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfccc8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfcd48d640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631941610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1002530346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 msgr2=0x5631941610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631941610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 msgr2=0x5631940f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 -- 172.21.15.94:0/1002530346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631940e8c00 con 0x563193d206b0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfc7fff640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004fd0 tx=0x7fcfb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cc8acf442b662a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4005020 con 0x563193d206b0 2024-04-25T13:53:49.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcfb40051c0 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb4004c50 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.753+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcfb4038050 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 10 client.19824 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563194162770 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb4004c50 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fcfb4029420 con 0x563193d206b0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 10 client.19824 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 10 client.19824 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 20 client.19824 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cd53a0 0x563194165130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 4 client.19824 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56319404f800 con 0x563193cd53a0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.754+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cd53a0 0x563194165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfa7fff640 10 client.19824 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfccc8c640 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cd53a0 0x563194165130 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=355144a27f9db876 server_cookie=b362e4d775b331fd in_seq=0 out_seq=0 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfa7fff640 1 -- 172.21.15.94:0/1002530346 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56319404f800 con 0x563193cd53a0 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfa7fff640 10 client.19824 handle_command_reply: tid=0 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 client.19824 shutdown 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 msgr2=0x5631940f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004fd0 tx=0x7fcfb4004740 comp rx=0 tx=0).stop 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 shutdown_connections 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x563193cd53a0 0x563194165130 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563193d41390 0x5631941610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563193d206b0 0x5631940f3500 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 --2- 172.21.15.94:0/1002530346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563193d184d0 0x5631940f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfa7fff640 0 client.19824 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 >> 172.21.15.94:0/1002530346 conn(0x563193d354e0 msgr2=0x5631940e1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 shutdown_connections 2024-04-25T13:53:49.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 1 -- 172.21.15.94:0/1002530346 wait complete. 2024-04-25T13:53:49.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.755+0000 7fcfd5d9c740 20 client.19824 trim_cache size 0 max 16384 2024-04-25T13:53:49.761 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 msgr2=0x7fcfb80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 0x7fcfb80b2000 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x5631940e4860 tx=0x7fcfc0009500 comp rx=0 tx=0).stop 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 msgr2=0x7fcfc819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006010 tx=0x7fcfbc004270 comp rx=0 tx=0).stop 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 shutdown_connections 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb80afaf0 0x7fcfb80b2000 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108ab0 0x7fcfc81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8104410 0x7fcfc819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 --2- 172.21.15.94:0/816157577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103f70 0x7fcfc819a220 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 >> 172.21.15.94:0/816157577 conn(0x7fcfc80fd380 msgr2=0x7fcfc8109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 shutdown_connections 2024-04-25T13:53:49.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.756+0000 7fcfceef0640 1 -- 172.21.15.94:0/816157577 wait complete. 2024-04-25T13:53:49.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:53:49.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1493817 12.57 MB/sec execute 512 sec latency 14.040 ms 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- 172.21.15.94:0/2503300883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108920 msgr2=0x7f79a410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 --2- 172.21.15.94:0/2503300883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108920 0x7f79a410ad40 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ca0 tx=0x7f799c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- 172.21.15.94:0/2503300883 shutdown_connections 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 --2- 172.21.15.94:0/2503300883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4108920 0x7f79a410ad40 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 --2- 172.21.15.94:0/2503300883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a4103c20 0x7f79a4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 --2- 172.21.15.94:0/2503300883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a41029e0 0x7f79a41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- 172.21.15.94:0/2503300883 >> 172.21.15.94:0/2503300883 conn(0x7f79a40fdbd0 msgr2=0x7f79a4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- 172.21.15.94:0/2503300883 shutdown_connections 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- 172.21.15.94:0/2503300883 wait complete. 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 Processor -- start 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.885+0000 7f79ac669640 1 -- start start 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a41029e0 0x7f79a41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a4108920 0x7f79a419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410d400 con 0x7f79a4103c20 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410d280 con 0x7f79a4108920 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410d580 con 0x7f79a41029e0 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38154/0 (socket says 172.21.15.94:38154) 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79aac06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a4108920 0x7f79a419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 -- 172.21.15.94:0/4211406398 learned_addr learned my addr 172.21.15.94:0/4211406398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79aa405640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a41029e0 0x7f79a41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 -- 172.21.15.94:0/4211406398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a41029e0 msgr2=0x7f79a41982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a41029e0 0x7f79a41982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 -- 172.21.15.94:0/4211406398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a4108920 msgr2=0x7f79a419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a4108920 0x7f79a419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 -- 172.21.15.94:0/4211406398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a419fe10 con 0x7f79a4103c20 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79a9c04640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f799400c890 tx=0x7f799400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f799b7fe640 1 -- 172.21.15.94:0/4211406398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994004220 con 0x7f79a4103c20 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f799b7fe640 1 -- 172.21.15.94:0/4211406398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79940043c0 con 0x7f79a4103c20 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.886+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41a0100 con 0x7f79a4103c20 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.887+0000 7f799b7fe640 1 -- 172.21.15.94:0/4211406398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994004a00 con 0x7f79a4103c20 2024-04-25T13:53:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.887+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a0650 con 0x7f79a4103c20 2024-04-25T13:53:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.889+0000 7f79b35f0740 1 Processor -- start 2024-04-25T13:53:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 -- start start 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 0x559323520d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x559323521a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559323484d80 con 0x5593231554d0 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55932343ea20 con 0x55932317e390 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79b35f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559323480d40 con 0x55932315d6b0 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f799b7fe640 1 -- 172.21.15.94:0/4211406398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f79940079b0 con 0x7f79a4103c20 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aa405640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 0x559323520d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aa405640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 0x559323520d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38164/0 (socket says 172.21.15.94:38164) 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38988/0 (socket says 172.21.15.94:38988) 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aa405640 1 -- 172.21.15.94:0/1483719653 learned_addr learned my addr 172.21.15.94:0/1483719653 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79a9c04640 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x559323521a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 -- 172.21.15.94:0/1483719653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 msgr2=0x559323521a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x559323521a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 -- 172.21.15.94:0/1483719653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 msgr2=0x559323520d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 0x559323520d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.890+0000 7f79aac06640 1 -- 172.21.15.94:0/1483719653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559323449c30 con 0x55932317e390 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f79aac06640 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f799c006010 tx=0x7f799c0098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aea5781b2852ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f799b7fe640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 0x7f79900b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f7998bfe640 1 -- 172.21.15.94:0/1483719653 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c031710 con 0x55932317e390 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f7998bfe640 1 -- 172.21.15.94:0/1483719653 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c036030 con 0x55932317e390 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f7998bfe640 1 -- 172.21.15.94:0/1483719653 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02f660 con 0x55932317e390 2024-04-25T13:53:49.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f799b7fe640 1 -- 172.21.15.94:0/4211406398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f79940de980 con 0x7f79a4103c20 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.891+0000 7f79aa405640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 0x7f79900b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- 172.21.15.94:0/1483719653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 msgr2=0x559323520810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79aa405640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 0x7f79900b4210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x559323520530 tx=0x7f79a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f799c006010 tx=0x7f799c0098b0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- 172.21.15.94:0/1483719653 shutdown_connections 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x559323521a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593231554d0 0x559323520d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 --2- 172.21.15.94:0/1483719653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x559323520810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- 172.21.15.94:0/1483719653 >> 172.21.15.94:0/1483719653 conn(0x5593231724e0 msgr2=0x55932351c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- 172.21.15.94:0/1483719653 shutdown_connections 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- 172.21.15.94:0/1483719653 wait complete. 2024-04-25T13:53:49.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 -- start start 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.892+0000 7f79b35f0740 1 Processor -- start 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x55932352a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559323522350 con 0x55932317e390 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55932340bdc0 con 0x5593231554d0 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593235224d0 con 0x55932315d6b0 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79b35f0740 10 client.0 _ll_register_callbacks cb 0x559323427600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38178/0 (socket says 172.21.15.94:38178) 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 -- 172.21.15.94:0/349890900 learned_addr learned my addr 172.21.15.94:0/349890900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aa405640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79a9c04640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x55932352a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 -- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 msgr2=0x55932352a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x55932352a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 -- 172.21.15.94:0/349890900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 msgr2=0x55932352a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 -- 172.21.15.94:0/349890900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593231963f0 con 0x55932317e390 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79a9c04640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x55932352a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f79aac06640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f799c0317f0 tx=0x7f799c031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8d330eb5f4f485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.893+0000 7f798dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c04a040 con 0x55932317e390 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f799c003b70 con 0x55932317e390 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c036030 con 0x55932317e390 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f799c003d70 con 0x55932317e390 2024-04-25T13:53:49.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 10 client.19836 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5593232607d0 con 0x55932317e390 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c036030 con 0x55932317e390 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f799c04d030 con 0x55932317e390 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 10 client.19836 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 10 client.19836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 20 client.19836 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55932313cf60 0x559323537dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 4 client.19836 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.894+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559323492990 con 0x55932313cf60 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.895+0000 7f79aa405640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55932313cf60 0x559323537dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.895+0000 7f79aa405640 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55932313cf60 0x559323537dc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b30f7172a99334f server_cookie=186ae4f7421130be in_seq=0 out_seq=0 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.895+0000 7f798dffb640 10 client.19836 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f798dffb640 1 -- 172.21.15.94:0/349890900 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559323492990 con 0x55932313cf60 2024-04-25T13:53:49.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f798dffb640 10 client.19836 handle_command_reply: tid=0 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 client.19836 shutdown 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 msgr2=0x5593235318d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f799c0317f0 tx=0x7f799c031820 comp rx=0 tx=0).stop 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 shutdown_connections 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55932313cf60 0x559323537dc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932317e390 0x5593235318d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932315d6b0 0x55932352a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 --2- 172.21.15.94:0/349890900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f798dffb640 0 client.19836 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 >> 172.21.15.94:0/349890900 conn(0x5593231724e0 msgr2=0x55932351b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 shutdown_connections 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 1 -- 172.21.15.94:0/349890900 wait complete. 2024-04-25T13:53:49.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.896+0000 7f79b35f0740 20 client.19836 trim_cache size 0 max 16384 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 Processor -- start 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 -- start start 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x55932352a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a9250 con 0x55932315d6b0 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41aad90 con 0x55932317e390 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79b35f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a9920 con 0x5593231554d0 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38184/0 (socket says 172.21.15.94:38184) 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 -- 172.21.15.94:0/917323951 learned_addr learned my addr 172.21.15.94:0/917323951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79aac06640 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x55932352a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79aa405640 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 -- 172.21.15.94:0/917323951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593231554d0 msgr2=0x55932352a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 -- 172.21.15.94:0/917323951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 msgr2=0x55932352a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x55932352a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.910+0000 7f79a9c04640 1 -- 172.21.15.94:0/917323951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593235286d0 con 0x55932315d6b0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.911+0000 7f79a9c04640 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f79940dc3e0 tx=0x7f79940ded60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57aa7dc395d2295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.911+0000 7f798ffff640 1 -- 172.21.15.94:0/917323951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799401d040 con 0x55932315d6b0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.911+0000 7f798ffff640 1 -- 172.21.15.94:0/917323951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994005290 con 0x55932315d6b0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.911+0000 7f798ffff640 1 -- 172.21.15.94:0/917323951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79940a27e0 con 0x55932315d6b0 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- 172.21.15.94:0/917323951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 msgr2=0x5593235317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f79940dc3e0 tx=0x7f79940ded60 comp rx=0 tx=0).stop 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- 172.21.15.94:0/917323951 shutdown_connections 2024-04-25T13:53:49.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55932317e390 0x55932352a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x5593235317f0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- 172.21.15.94:0/917323951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593231554d0 0x55932352a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- 172.21.15.94:0/917323951 >> 172.21.15.94:0/917323951 conn(0x5593231724e0 msgr2=0x559323525310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- 172.21.15.94:0/917323951 shutdown_connections 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- 172.21.15.94:0/917323951 wait complete. 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- start start 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 Processor -- start 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x5593235315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932317e390 0x55932352e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41aa6c0 con 0x55932315d6b0 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a9ff0 con 0x5593231554d0 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a7230 con 0x55932317e390 2024-04-25T13:53:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79b35f0740 10 client.0 _ll_register_callbacks cb 0x7f79a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79a9c04640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38186/0 (socket says 172.21.15.94:38186) 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79a9c04640 1 -- 172.21.15.94:0/3550904696 learned_addr learned my addr 172.21.15.94:0/3550904696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.912+0000 7f79aa405640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x5593235315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79aac06640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932317e390 0x55932352e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 -- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932317e390 msgr2=0x55932352e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932317e390 0x55932352e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 -- 172.21.15.94:0/3550904696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 msgr2=0x5593235315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x5593235315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 -- 172.21.15.94:0/3550904696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55932352ebf0 con 0x55932315d6b0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79a9c04640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f79940a6e00 tx=0x7f79940e4320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117693da39ef9e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799401d030 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f79940a27e0 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55932348c800 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79940a6e70 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79b35f0740 10 client.19842 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55932359f010 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55932348c800 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.913+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f79940c1080 con 0x55932315d6b0 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 10 client.19842 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 10 client.19842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 20 client.19842 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5593231123a0 0x5593235a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 4 client.19842 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559323407610 con 0x5593231123a0 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79aa405640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5593231123a0 0x5593235a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f798dffb640 10 client.19842 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79aa405640 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5593231123a0 0x5593235a1980 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f5e570bd7d0af4c server_cookie=842debba34e8a4e4 in_seq=0 out_seq=0 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f798dffb640 1 -- 172.21.15.94:0/3550904696 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559323407610 con 0x5593231123a0 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f798dffb640 10 client.19842 handle_command_reply: tid=0 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 client.19842 shutdown 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 msgr2=0x55932352e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f79940a6e00 tx=0x7f79940e4320 comp rx=0 tx=0).stop 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 shutdown_connections 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5593231123a0 0x5593235a1980 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55932317e390 0x55932352e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932315d6b0 0x55932352e170 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 --2- 172.21.15.94:0/3550904696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593231554d0 0x5593235315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f798dffb640 0 client.19842 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.914+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 >> 172.21.15.94:0/3550904696 conn(0x5593231724e0 msgr2=0x55932352a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 shutdown_connections 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79b35f0740 1 -- 172.21.15.94:0/3550904696 wait complete. 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79b35f0740 20 client.19842 trim_cache size 0 max 16384 2024-04-25T13:53:49.921 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 msgr2=0x7f79900b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 0x7f79900b4210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x559323520530 tx=0x7f79a0005f90 comp rx=0 tx=0).stop 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 msgr2=0x7f79a4198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f799400c890 tx=0x7f799400cd50 comp rx=0 tx=0).stop 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 shutdown_connections 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79900b1d00 0x7f79900b4210 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a4108920 0x7f79a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79a4103c20 0x7f79a4198810 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 --2- 172.21.15.94:0/4211406398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79a41029e0 0x7f79a41982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 >> 172.21.15.94:0/4211406398 conn(0x7f79a40fdbd0 msgr2=0x7f79a4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 shutdown_connections 2024-04-25T13:53:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:49.915+0000 7f79ac669640 1 -- 172.21.15.94:0/4211406398 wait complete. 2024-04-25T13:53:49.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 -- 172.21.15.94:0/896925587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 msgr2=0x7f2b8810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/896925587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8810ad50 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009ca0 tx=0x7f2b8002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 -- 172.21.15.94:0/896925587 shutdown_connections 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/896925587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8810ad50 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/896925587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b88103c30 0x7f2b88104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/896925587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b881029f0 0x7f2b881036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 -- 172.21.15.94:0/896925587 >> 172.21.15.94:0/896925587 conn(0x7f2b880fdc00 msgr2=0x7f2b88100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 -- 172.21.15.94:0/896925587 shutdown_connections 2024-04-25T13:53:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.049+0000 7f2b8ee27640 1 -- 172.21.15.94:0/896925587 wait complete. 2024-04-25T13:53:50.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 Processor -- start 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 -- start start 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b881029f0 0x7f2b88197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b88103c30 0x7f2b88198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d410 con 0x7f2b88108930 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d290 con 0x7f2b88103c30 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8ee27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d590 con 0x7f2b881029f0 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8d3c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b88103c30 0x7f2b88198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8d3c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38208/0 (socket says 172.21.15.94:38208) 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/1275715001 learned_addr learned my addr 172.21.15.94:0/1275715001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.050+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b881029f0 0x7f2b88197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/1275715001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b881029f0 msgr2=0x7f2b88197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b881029f0 0x7f2b88197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/1275715001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b88103c30 msgr2=0x7f2b88198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b88103c30 0x7f2b88198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/1275715001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b881a3f10 con 0x7f2b88108930 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2b800097b0 tx=0x7f2b80031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b85ffb640 1 -- 172.21.15.94:0/1275715001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80003bf0 con 0x7f2b88108930 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b85ffb640 1 -- 172.21.15.94:0/1275715001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b80003d90 con 0x7f2b88108930 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b85ffb640 1 -- 172.21.15.94:0/1275715001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8002fc30 con 0x7f2b88108930 2024-04-25T13:53:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.051+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881a41a0 con 0x7f2b88108930 2024-04-25T13:53:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.052+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881a46a0 con 0x7f2b88108930 2024-04-25T13:53:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.055+0000 7f2b85ffb640 1 -- 172.21.15.94:0/1275715001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2b800305c0 con 0x7f2b88108930 2024-04-25T13:53:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.055+0000 7f2b85ffb640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 0x7f2b780b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.056+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 0x7f2b780b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.056+0000 7f2b85ffb640 1 -- 172.21.15.94:0/1275715001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2b80100280 con 0x7f2b88108930 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.056+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 0x7f2b780b1fe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2b881035f0 tx=0x7f2b7c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 Processor -- start 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 -- start start 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 0x55e583cfc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583cfda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e583c60d80 con 0x55e5839314d0 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e583c1aa20 con 0x55e5839396b0 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b95db8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e583c5cd40 con 0x55e58395a390 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38212/0 (socket says 172.21.15.94:38212) 2024-04-25T13:53:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b87fff640 1 -- 172.21.15.94:0/4023161234 learned_addr learned my addr 172.21.15.94:0/4023161234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583cfda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.057+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 0x55e583cfc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 -- 172.21.15.94:0/4023161234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 msgr2=0x55e583cfc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 0x55e583cfc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 -- 172.21.15.94:0/4023161234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 msgr2=0x55e583cfda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583cfda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 -- 172.21.15.94:0/4023161234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e583c25c30 con 0x55e5839314d0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 0x55e583cfc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009ca0 tx=0x7f2b74011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c7e6452ecc7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b72ffd640 1 -- 172.21.15.94:0/4023161234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74005220 con 0x55e5839314d0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b72ffd640 1 -- 172.21.15.94:0/4023161234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b74004c50 con 0x55e5839314d0 2024-04-25T13:53:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.058+0000 7f2b72ffd640 1 -- 172.21.15.94:0/4023161234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740197a0 con 0x55e5839314d0 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 -- 172.21.15.94:0/4023161234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 msgr2=0x55e583cfcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009ca0 tx=0x7f2b74011810 comp rx=0 tx=0).stop 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 -- 172.21.15.94:0/4023161234 shutdown_connections 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583cfda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839314d0 0x55e583cfcd90 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4023161234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e58395a390 0x55e583cfc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 -- 172.21.15.94:0/4023161234 >> 172.21.15.94:0/4023161234 conn(0x55e58394e4e0 msgr2=0x55e583cf8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 -- 172.21.15.94:0/4023161234 shutdown_connections 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.059+0000 7f2b95db8740 1 -- 172.21.15.94:0/4023161234 wait complete. 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 -- start start 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 Processor -- start 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d06510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d06a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e583cfe530 con 0x55e5839396b0 2024-04-25T13:53:50.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e583be7dc0 con 0x55e58395a390 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e583cfe6b0 con 0x55e5839314d0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b95db8740 10 client.0 _ll_register_callbacks cb 0x55e583c03600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d06a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d06a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38226/0 (socket says 172.21.15.94:38226) 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b87fff640 1 -- 172.21.15.94:0/316708585 learned_addr learned my addr 172.21.15.94:0/316708585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.060+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d06510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/316708585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 msgr2=0x55e583d06510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d06510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/316708585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 msgr2=0x55e583d06a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d06a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/316708585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5839723f0 con 0x55e58395a390 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d0d9b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b800fdd30 tx=0x7f2b800c7ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acab3995881d0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b67fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b800e1020 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b74004c50 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2b8002fdd0 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b800c8dc0 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b95db8740 10 client.29294 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.061+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e583a3c7d0 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b74004c50 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2b800c8920 con 0x55e58395a390 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 10 client.29294 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 10 client.29294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:53:50.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 20 client.29294 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e583918f60 0x55e583d14240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 4 client.29294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e583c6e990 con 0x55e583918f60 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e583918f60 0x55e583d14240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e583918f60 0x55e583d14240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e2958c0e05ecb61 server_cookie=8d32b997c5a488ab in_seq=0 out_seq=0 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.062+0000 7f2b67fff640 10 client.29294 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b67fff640 1 -- 172.21.15.94:0/316708585 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e583c6e990 con 0x55e583918f60 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b67fff640 10 client.29294 handle_command_reply: tid=0 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 client.29294 shutdown 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 msgr2=0x55e583d0d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d0d9b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b800fdd30 tx=0x7f2b800c7ef0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 shutdown_connections 2024-04-25T13:53:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e583918f60 0x55e583d14240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d0d9b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d06a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 --2- 172.21.15.94:0/316708585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d06510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b67fff640 0 client.29294 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.063+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 >> 172.21.15.94:0/316708585 conn(0x55e58394e4e0 msgr2=0x55e583cfaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.064+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 shutdown_connections 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.064+0000 7f2b95db8740 1 -- 172.21.15.94:0/316708585 wait complete. 2024-04-25T13:53:50.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.064+0000 7f2b95db8740 20 client.29294 trim_cache size 0 max 16384 2024-04-25T13:53:50.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.077+0000 7f2b95db8740 1 Processor -- start 2024-04-25T13:53:50.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.077+0000 7f2b95db8740 1 -- start start 2024-04-25T13:53:50.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583cf8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00e730 con 0x55e5839396b0 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c010270 con 0x55e58395a390 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b95db8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00ee00 con 0x55e5839314d0 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38234/0 (socket says 172.21.15.94:38234) 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 -- 172.21.15.94:0/1835679442 learned_addr learned my addr 172.21.15.94:0/1835679442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583cf8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 -- 172.21.15.94:0/1835679442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 msgr2=0x55e583cf8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583cf8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 -- 172.21.15.94:0/1835679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 msgr2=0x55e583d05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 -- 172.21.15.94:0/1835679442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e583cfc3d0 con 0x55e5839396b0 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b87fff640 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2b74011d10 tx=0x7f2b74009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ae006e76860e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.078+0000 7f2b727fc640 1 -- 172.21.15.94:0/1835679442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74027070 con 0x55e5839396b0 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b727fc640 1 -- 172.21.15.94:0/1835679442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b74005390 con 0x55e5839396b0 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b727fc640 1 -- 172.21.15.94:0/1835679442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74019690 con 0x55e5839396b0 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 -- 172.21.15.94:0/1835679442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 msgr2=0x55e583d04d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2b74011d10 tx=0x7f2b74009800 comp rx=0 tx=0).stop 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 -- 172.21.15.94:0/1835679442 shutdown_connections 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58395a390 0x55e583d05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5839396b0 0x55e583d04d20 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 --2- 172.21.15.94:0/1835679442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583cf8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 -- 172.21.15.94:0/1835679442 >> 172.21.15.94:0/1835679442 conn(0x55e58394e4e0 msgr2=0x55e583cf7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 -- 172.21.15.94:0/1835679442 shutdown_connections 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.079+0000 7f2b95db8740 1 -- 172.21.15.94:0/1835679442 wait complete. 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 -- start start 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 Processor -- start 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00fba0 con 0x55e58395a390 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00f4d0 con 0x55e5839396b0 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c00c710 con 0x55e5839314d0 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b95db8740 10 client.0 _ll_register_callbacks cb 0x7f2b80009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:50.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b8d3c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b8d3c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38244/0 (socket says 172.21.15.94:38244) 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39050/0 (socket says 172.21.15.94:39050) 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b8d3c4640 1 -- 172.21.15.94:0/4048283654 learned_addr learned my addr 172.21.15.94:0/4048283654 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.080+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 -- 172.21.15.94:0/4048283654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 msgr2=0x55e583d0a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d0a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 -- 172.21.15.94:0/4048283654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 msgr2=0x55e583d0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 -- 172.21.15.94:0/4048283654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e583d01c20 con 0x55e5839396b0 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b8d3c4640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d0a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b87fff640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55e583d04bd0 tx=0x7f2b74003c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6096026a2e9a9bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b67fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74027050 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b74005390 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2b7402c450 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b74009870 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b95db8740 10 client.29300 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e583d0d100 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.081+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b74005390 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f2b74023400 con 0x55e5839396b0 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 10 client.29300 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 10 client.29300 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 20 client.29300 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5838ee3a0 0x55e583d7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 4 client.29300 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e583c68800 con 0x55e5838ee3a0 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5838ee3a0 0x55e583d7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b67fff640 10 client.29300 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.082+0000 7f2b8cbc3640 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5838ee3a0 0x55e583d7ea60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=914756637fd643d3 server_cookie=7d3120c553fa5240 in_seq=0 out_seq=0 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b67fff640 1 -- 172.21.15.94:0/4048283654 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e583c68800 con 0x55e5838ee3a0 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b67fff640 10 client.29300 handle_command_reply: tid=0 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 client.29300 shutdown 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 msgr2=0x55e583d0ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55e583d04bd0 tx=0x7f2b74003c50 comp rx=0 tx=0).stop 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 shutdown_connections 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5838ee3a0 0x55e583d7ea60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e58395a390 0x55e583d0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e5839396b0 0x55e583d0ac20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b67fff640 0 client.29300 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:53:50.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 --2- 172.21.15.94:0/4048283654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5839314d0 0x55e583d0a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 >> 172.21.15.94:0/4048283654 conn(0x55e58394e4e0 msgr2=0x55e583d04d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 shutdown_connections 2024-04-25T13:53:50.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 1 -- 172.21.15.94:0/4048283654 wait complete. 2024-04-25T13:53:50.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b95db8740 20 client.29300 trim_cache size 0 max 16384 2024-04-25T13:53:50.089 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 msgr2=0x7f2b780b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.083+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 0x7f2b780b1fe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2b881035f0 tx=0x7f2b7c00b390 comp rx=0 tx=0).stop 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 msgr2=0x7f2b8819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2b800097b0 tx=0x7f2b80031840 comp rx=0 tx=0).stop 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 shutdown_connections 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2b780afad0 0x7f2b780b1fe0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b88108930 0x7f2b8819f490 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b88103c30 0x7f2b88198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 --2- 172.21.15.94:0/1275715001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b881029f0 0x7f2b88197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 >> 172.21.15.94:0/1275715001 conn(0x7f2b880fdc00 msgr2=0x7f2b88104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 shutdown_connections 2024-04-25T13:53:50.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.084+0000 7f2b8ee27640 1 -- 172.21.15.94:0/1275715001 wait complete. 2024-04-25T13:53:50.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 -- 172.21.15.94:0/280063701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50103c20 msgr2=0x7f7e50104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 --2- 172.21.15.94:0/280063701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50103c20 0x7f7e50104030 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009900 tx=0x7f7e3802a410 comp rx=0 tx=0).stop 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 -- 172.21.15.94:0/280063701 shutdown_connections 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 --2- 172.21.15.94:0/280063701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e50108920 0x7f7e5010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 --2- 172.21.15.94:0/280063701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50103c20 0x7f7e50104030 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 --2- 172.21.15.94:0/280063701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e501029e0 0x7f7e501036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 -- 172.21.15.94:0/280063701 >> 172.21.15.94:0/280063701 conn(0x7f7e500fdbd0 msgr2=0x7f7e50100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 -- 172.21.15.94:0/280063701 shutdown_connections 2024-04-25T13:53:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 -- 172.21.15.94:0/280063701 wait complete. 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.211+0000 7f7e54fce640 1 Processor -- start 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 -- start start 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 0x7f7e50199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 0x7f7e50199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010e4e0 con 0x7f7e50108920 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010e360 con 0x7f7e50103c20 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e54fce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010e660 con 0x7f7e501029e0 2024-04-25T13:53:50.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38258/0 (socket says 172.21.15.94:38258) 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 0x7f7e50199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 -- 172.21.15.94:0/587622676 learned_addr learned my addr 172.21.15.94:0/587622676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 0x7f7e50199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 -- 172.21.15.94:0/587622676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 msgr2=0x7f7e50199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 0x7f7e50199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 -- 172.21.15.94:0/587622676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 msgr2=0x7f7e50199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 0x7f7e50199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4ed9d640 1 -- 172.21.15.94:0/587622676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e501a0f50 con 0x7f7e50108920 2024-04-25T13:53:50.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 0x7f7e50199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.212+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 0x7f7e50199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e4ed9d640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb10 tx=0x7f7e4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e47fff640 1 -- 172.21.15.94:0/587622676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48004b20 con 0x7f7e50108920 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e47fff640 1 -- 172.21.15.94:0/587622676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e48004cc0 con 0x7f7e50108920 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e501a1240 con 0x7f7e50108920 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e47fff640 1 -- 172.21.15.94:0/587622676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48010670 con 0x7f7e50108920 2024-04-25T13:53:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.213+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a1740 con 0x7f7e50108920 2024-04-25T13:53:50.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.215+0000 7f7e47fff640 1 -- 172.21.15.94:0/587622676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7e48010830 con 0x7f7e50108920 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 Processor -- start 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 -- start start 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c249a4390 0x555c24d46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c24caad80 con 0x555c249836b0 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c24c64a20 con 0x555c2497b4d0 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c24ca6d40 con 0x555c249a4390 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e4e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38272/0 (socket says 172.21.15.94:38272) 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e4e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39078/0 (socket says 172.21.15.94:39078) 2024-04-25T13:53:50.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e45d9b640 1 -- 172.21.15.94:0/675252189 learned_addr learned my addr 172.21.15.94:0/675252189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e47fff640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 0x7f7e3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c249a4390 0x555c24d46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.216+0000 7f7e4ed9d640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 0x7f7e3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 -- 172.21.15.94:0/675252189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c249a4390 msgr2=0x555c24d46810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c249a4390 0x555c24d46810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 -- 172.21.15.94:0/675252189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 msgr2=0x555c24d47a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 -- 172.21.15.94:0/675252189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c24c6fc30 con 0x555c2497b4d0 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e47fff640 1 -- 172.21.15.94:0/587622676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7e480df310 con 0x7f7e50108920 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7e501022b0 tx=0x7f7e400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf3aec5b0e2542f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e4ed9d640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 0x7f7e3c0b2000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e480dd0e0 tx=0x7f7e48011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e44d99640 1 -- 172.21.15.94:0/675252189 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40012450 con 0x555c2497b4d0 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e44d99640 1 -- 172.21.15.94:0/675252189 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e40012aa0 con 0x555c2497b4d0 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.217+0000 7f7e44d99640 1 -- 172.21.15.94:0/675252189 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000fa70 con 0x555c2497b4d0 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- 172.21.15.94:0/675252189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 msgr2=0x555c24d46d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7e501022b0 tx=0x7f7e400117b0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- 172.21.15.94:0/675252189 shutdown_connections 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d47a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d46d90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/675252189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c249a4390 0x555c24d46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- 172.21.15.94:0/675252189 >> 172.21.15.94:0/675252189 conn(0x555c249984e0 msgr2=0x555c24d42020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- 172.21.15.94:0/675252189 shutdown_connections 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- 172.21.15.94:0/675252189 wait complete. 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 -- start start 2024-04-25T13:53:50.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.218+0000 7f7e5be7f740 1 Processor -- start 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c2497b4d0 0x555c24d50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24d50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c24d48530 con 0x555c249a4390 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c24c31dc0 con 0x555c249836b0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c24d486b0 con 0x555c2497b4d0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e5be7f740 10 client.0 _ll_register_callbacks cb 0x555c24c4d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e4e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24d50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38286/0 (socket says 172.21.15.94:38286) 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 -- 172.21.15.94:0/641269848 learned_addr learned my addr 172.21.15.94:0/641269848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c2497b4d0 0x555c24d50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 -- 172.21.15.94:0/641269848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c2497b4d0 msgr2=0x555c24d50560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c2497b4d0 0x555c24d50560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 -- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 msgr2=0x555c24d50aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24d50aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 -- 172.21.15.94:0/641269848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c249bc3f0 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24d50aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009230 tx=0x7f7e3802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb59217da3eaaa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.219+0000 7f7e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3803f070 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7e380037f0 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e40012aa0 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e38035040 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 10 client.19872 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c24a867d0 con 0x555c249a4390 2024-04-25T13:53:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e40012aa0 con 0x555c249a4390 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7e3803ba20 con 0x555c249a4390 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 10 client.19872 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 10 client.19872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 20 client.19872 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c24962f60 0x555c24d5e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 4 client.19872 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c24cb8990 con 0x555c24962f60 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.220+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c24962f60 0x555c24d5e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.221+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c24962f60 0x555c24d5e2a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9cfb5ac600484bd server_cookie=518961f35ed7451f in_seq=0 out_seq=0 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.221+0000 7f7e35ffb640 10 client.19872 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e35ffb640 1 -- 172.21.15.94:0/641269848 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c24cb8990 con 0x555c24962f60 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e35ffb640 10 client.19872 handle_command_reply: tid=0 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 client.19872 shutdown 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 msgr2=0x555c24d57a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009230 tx=0x7f7e3802a410 comp rx=0 tx=0).stop 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 shutdown_connections 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c24962f60 0x555c24d5e2a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249a4390 0x555c24d57a00 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24d50aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e35ffb640 0 client.19872 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/641269848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c2497b4d0 0x555c24d50560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.223+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 >> 172.21.15.94:0/641269848 conn(0x555c249984e0 msgr2=0x555c24d44f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.224+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 shutdown_connections 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.224+0000 7f7e5be7f740 1 -- 172.21.15.94:0/641269848 wait complete. 2024-04-25T13:53:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.224+0000 7f7e5be7f740 20 client.19872 trim_cache size 0 max 16384 2024-04-25T13:53:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.238+0000 7f7e5be7f740 1 Processor -- start 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.238+0000 7f7e5be7f740 1 -- start start 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d41c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501a9310 con 0x555c249836b0 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501aae50 con 0x555c2497b4d0 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501a99e0 con 0x555c24962f60 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38302/0 (socket says 172.21.15.94:38302) 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 -- 172.21.15.94:0/58002033 learned_addr learned my addr 172.21.15.94:0/58002033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d41c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 -- 172.21.15.94:0/58002033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 msgr2=0x555c24d416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 -- 172.21.15.94:0/58002033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 msgr2=0x555c24d41c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d41c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 -- 172.21.15.94:0/58002033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c24d516a0 con 0x555c249836b0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7e380059d0 tx=0x7f7e3803bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b2d6e8b3abc3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.239+0000 7f7e37fff640 1 -- 172.21.15.94:0/58002033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3805a070 con 0x555c249836b0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e37fff640 1 -- 172.21.15.94:0/58002033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e3805b040 con 0x555c249836b0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e37fff640 1 -- 172.21.15.94:0/58002033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3803f070 con 0x555c249836b0 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 -- 172.21.15.94:0/58002033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 msgr2=0x555c24d42140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7e380059d0 tx=0x7f7e3803bf90 comp rx=0 tx=0).stop 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 -- 172.21.15.94:0/58002033 shutdown_connections 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c249836b0 0x555c24d42140 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c2497b4d0 0x555c24d41c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/58002033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 -- 172.21.15.94:0/58002033 >> 172.21.15.94:0/58002033 conn(0x555c249984e0 msgr2=0x555c24d4b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 -- 172.21.15.94:0/58002033 shutdown_connections 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.240+0000 7f7e5be7f740 1 -- 172.21.15.94:0/58002033 wait complete. 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 -- start start 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 Processor -- start 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 0x555c24d545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24dc4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501aa780 con 0x555c2497b4d0 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501aa0b0 con 0x555c249836b0 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501a72f0 con 0x555c24962f60 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e5be7f740 10 client.0 _ll_register_callbacks cb 0x7f7e38031820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e4e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 0x555c24d545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58254/0 (socket says 172.21.15.94:58254) 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 -- 172.21.15.94:0/2169093268 learned_addr learned my addr 172.21.15.94:0/2169093268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24dc4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 -- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 msgr2=0x555c24dc4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24dc4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 -- 172.21.15.94:0/2169093268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 msgr2=0x555c24d545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.241+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 0x555c24d545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e45d9b640 1 -- 172.21.15.94:0/2169093268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c24dc4b40 con 0x555c24962f60 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 0x555c24d545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e45d9b640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e38004210 tx=0x7f7e38005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223289c7bf1d908b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3805a070 con 0x555c24962f60 2024-04-25T13:53:50.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3805b040 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7e3803b560 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e3803f040 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 10 client.29101 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555c24dc5d60 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e3805b040 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7e38060070 con 0x555c24962f60 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 10 client.29101 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 10 client.29101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 20 client.29101 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c249a4390 0x555c24dc8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 4 client.29101 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.242+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c24cb2800 con 0x555c249a4390 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c249a4390 0x555c24dc8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e35ffb640 10 client.29101 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:50.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e4e59c640 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c249a4390 0x555c24dc8700 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db97c17b21e391f0 server_cookie=ff5f850f32b73d11 in_seq=0 out_seq=0 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e35ffb640 1 -- 172.21.15.94:0/2169093268 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c24cb2800 con 0x555c249a4390 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e35ffb640 10 client.29101 handle_command_reply: tid=0 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 client.29101 shutdown 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 msgr2=0x555c24d540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e38004210 tx=0x7f7e38005e30 comp rx=0 tx=0).stop 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 shutdown_connections 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c249a4390 0x555c24dc8700 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c249836b0 0x555c24dc4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e35ffb640 0 client.29101 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c2497b4d0 0x555c24d545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 --2- 172.21.15.94:0/2169093268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c24962f60 0x555c24d540a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.243+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 >> 172.21.15.94:0/2169093268 conn(0x555c249984e0 msgr2=0x555c24d44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 shutdown_connections 2024-04-25T13:53:50.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e5be7f740 1 -- 172.21.15.94:0/2169093268 wait complete. 2024-04-25T13:53:50.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e5be7f740 20 client.29101 trim_cache size 0 max 16384 2024-04-25T13:53:50.250 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 msgr2=0x7f7e3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 0x7f7e3c0b2000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e480dd0e0 tx=0x7f7e48011000 comp rx=0 tx=0).stop 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 msgr2=0x7f7e501a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb10 tx=0x7f7e4800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 shutdown_connections 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e3c0afaf0 0x7f7e3c0b2000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50108920 0x7f7e501a08e0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e50103c20 0x7f7e50199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 --2- 172.21.15.94:0/587622676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e501029e0 0x7f7e50199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:50.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 >> 172.21.15.94:0/587622676 conn(0x7f7e500fdbd0 msgr2=0x7f7e50104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:50.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 shutdown_connections 2024-04-25T13:53:50.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:50.244+0000 7f7e54fce640 1 -- 172.21.15.94:0/587622676 wait complete. 2024-04-25T13:53:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:50 smithi094 ceph-mon[29195]: pgmap v710: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-04-25T13:53:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:50 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/615872977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:50 smithi137 ceph-mon[32568]: pgmap v710: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-04-25T13:53:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:50 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/615872977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:50 smithi138 ceph-mon[32977]: pgmap v710: 97 pgs: 97 active+clean; 44 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-04-25T13:53:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:50 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/615872977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:53:50.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1496250 12.57 MB/sec execute 513 sec latency 10.186 ms 2024-04-25T13:53:51.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:53:51.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:53:51.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:53:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 -- 172.21.15.94:0/4170284735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534104680 msgr2=0x7fc53410add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 --2- 172.21.15.94:0/4170284735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534104680 0x7fc53410add0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009ca0 tx=0x7fc52c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 -- 172.21.15.94:0/4170284735 shutdown_connections 2024-04-25T13:53:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 --2- 172.21.15.94:0/4170284735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534104680 0x7fc53410add0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 --2- 172.21.15.94:0/4170284735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc534103b20 0x7fc534103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 --2- 172.21.15.94:0/4170284735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 0x7fc5341033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.384+0000 7fc53958f640 1 -- 172.21.15.94:0/4170284735 >> 172.21.15.94:0/4170284735 conn(0x7fc5340fd340 msgr2=0x7fc5340ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.385+0000 7fc53958f640 1 -- 172.21.15.94:0/4170284735 shutdown_connections 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.385+0000 7fc53958f640 1 -- 172.21.15.94:0/4170284735 wait complete. 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.385+0000 7fc53958f640 1 Processor -- start 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.385+0000 7fc53958f640 1 -- start start 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 0x7fc534199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc53419f4b0 0x7fc5341a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410e730 con 0x7fc534103b20 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410e5b0 con 0x7fc534102fc0 2024-04-25T13:53:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410e8b0 con 0x7fc53419f4b0 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38322/0 (socket says 172.21.15.94:38322) 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 -- 172.21.15.94:0/1910806237 learned_addr learned my addr 172.21.15.94:0/1910806237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc532ffd640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 0x7fc534199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5337fe640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc53419f4b0 0x7fc5341a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 -- 172.21.15.94:0/1910806237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc53419f4b0 msgr2=0x7fc5341a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc53419f4b0 0x7fc5341a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 -- 172.21.15.94:0/1910806237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 msgr2=0x7fc534199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 0x7fc534199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 -- 172.21.15.94:0/1910806237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341a1f40 con 0x7fc534103b20 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc5327fc640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e9f0 tx=0x7fc52800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc51bfff640 1 -- 172.21.15.94:0/1910806237 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52800cd80 con 0x7fc534103b20 2024-04-25T13:53:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341a2230 con 0x7fc534103b20 2024-04-25T13:53:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc51bfff640 1 -- 172.21.15.94:0/1910806237 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5280108e0 con 0x7fc534103b20 2024-04-25T13:53:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc51bfff640 1 -- 172.21.15.94:0/1910806237 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528018980 con 0x7fc534103b20 2024-04-25T13:53:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.386+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341a2780 con 0x7fc534103b20 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc51bfff640 1 -- 172.21.15.94:0/1910806237 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc528010430 con 0x7fc534103b20 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 Processor -- start 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 -- start start 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559122735b00 con 0x55912240e730 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5591226d1920 con 0x55912242f410 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc540444740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559122731ac0 con 0x559122406550 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc5327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39132/0 (socket says 172.21.15.94:39132) 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc5337fe640 1 -- 172.21.15.94:0/2728668483 learned_addr learned my addr 172.21.15.94:0/2728668483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.389+0000 7fc51bfff640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc532ffd640 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 -- 172.21.15.94:0/2728668483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 msgr2=0x5591227d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 -- 172.21.15.94:0/2728668483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 msgr2=0x5591227d7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227d7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc51bfff640 1 -- 172.21.15.94:0/1910806237 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc5280df070 con 0x7fc534103b20 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 -- 172.21.15.94:0/2728668483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912250f7e0 con 0x55912242f410 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc532ffd640 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009dc0 tx=0x7fc52c031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1fe4f9a9642dc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5337fe640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 0x7fc51c0b2000 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc52c046000 tx=0x7fc52c048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5197fa640 1 -- 172.21.15.94:0/2728668483 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004040 con 0x55912242f410 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5197fa640 1 -- 172.21.15.94:0/2728668483 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c036070 con 0x55912242f410 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.390+0000 7fc5197fa640 1 -- 172.21.15.94:0/2728668483 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c02fbf0 con 0x55912242f410 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 -- 172.21.15.94:0/2728668483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 msgr2=0x5591227d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009dc0 tx=0x7fc52c031550 comp rx=0 tx=0).stop 2024-04-25T13:53:51.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 -- 172.21.15.94:0/2728668483 shutdown_connections 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227d7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 --2- 172.21.15.94:0/2728668483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x5591227d04a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 -- 172.21.15.94:0/2728668483 >> 172.21.15.94:0/2728668483 conn(0x559122423560 msgr2=0x5591227cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 -- 172.21.15.94:0/2728668483 shutdown_connections 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.391+0000 7fc540444740 1 -- 172.21.15.94:0/2728668483 wait complete. 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 -- start start 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 Processor -- start 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 0x5591227da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5591227d82b0 con 0x55912242f410 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5591226cf5d0 con 0x559122406550 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591227d8430 con 0x55912240e730 2024-04-25T13:53:51.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc540444740 10 client.0 _ll_register_callbacks cb 0x5591226bd080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc532ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc532ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39134/0 (socket says 172.21.15.94:39134) 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38340/0 (socket says 172.21.15.94:38340) 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc532ffd640 1 -- 172.21.15.94:0/2086087717 learned_addr learned my addr 172.21.15.94:0/2086087717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5327fc640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 0x5591227da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 -- 172.21.15.94:0/2086087717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 msgr2=0x5591227da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 0x5591227da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 -- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 msgr2=0x5591227da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5337fe640 1 -- 172.21.15.94:0/2086087717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912256ff70 con 0x55912242f410 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc532ffd640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.392+0000 7fc5327fc640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 0x5591227da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:51.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc5337fe640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009ca0 tx=0x7fc52c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1390beeeb9c89a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c059070 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc52c004240 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc52c036070 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc52c004440 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 10 client.19890 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559122407170 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c036070 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc52c030c70 con 0x55912242f410 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 10 client.19890 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 10 client.19890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:51.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 20 client.19890 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223ee1f0 0x5591227e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 4 client.19890 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.393+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559122743710 con 0x5591223ee1f0 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.394+0000 7fc532ffd640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223ee1f0 0x5591227e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.394+0000 7fc516ff5640 10 client.19890 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.394+0000 7fc532ffd640 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223ee1f0 0x5591227e7830 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357184e909d46078 server_cookie=cbb68706c9debcd3 in_seq=0 out_seq=0 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc516ff5640 1 -- 172.21.15.94:0/2086087717 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559122743710 con 0x5591223ee1f0 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc516ff5640 10 client.19890 handle_command_reply: tid=0 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 client.19890 shutdown 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 msgr2=0x5591227e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009ca0 tx=0x7fc52c031a20 comp rx=0 tx=0).stop 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 shutdown_connections 2024-04-25T13:53:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223ee1f0 0x5591227e7830 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912242f410 0x5591227e1770 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912240e730 0x5591227da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc516ff5640 0 client.19890 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 --2- 172.21.15.94:0/2086087717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559122406550 0x5591227da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.395+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 >> 172.21.15.94:0/2086087717 conn(0x559122423560 msgr2=0x5591227caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.396+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 shutdown_connections 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.396+0000 7fc540444740 1 -- 172.21.15.94:0/2086087717 wait complete. 2024-04-25T13:53:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.396+0000 7fc540444740 20 client.19890 trim_cache size 0 max 16384 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 Processor -- start 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 -- start start 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912240e730 0x5591227e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912242f410 0x5591227cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341ab4e0 con 0x559122406550 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341ad020 con 0x55912240e730 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc540444740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341abbb0 con 0x55912242f410 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc532ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc532ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38354/0 (socket says 172.21.15.94:38354) 2024-04-25T13:53:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc532ffd640 1 -- 172.21.15.94:0/567086712 learned_addr learned my addr 172.21.15.94:0/567086712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc5337fe640 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912242f410 0x5591227cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.414+0000 7fc5327fc640 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912240e730 0x5591227e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 -- 172.21.15.94:0/567086712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912242f410 msgr2=0x5591227cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912242f410 0x5591227cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 -- 172.21.15.94:0/567086712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912240e730 msgr2=0x5591227e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912240e730 0x5591227e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 -- 172.21.15.94:0/567086712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591227d51d0 con 0x559122406550 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc532ffd640 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x5591227e7da0 tx=0x7fc524005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e800b6557118ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc518ff9640 1 -- 172.21.15.94:0/567086712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524004050 con 0x559122406550 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc518ff9640 1 -- 172.21.15.94:0/567086712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5240054d0 con 0x559122406550 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.415+0000 7fc518ff9640 1 -- 172.21.15.94:0/567086712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524004ca0 con 0x559122406550 2024-04-25T13:53:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- 172.21.15.94:0/567086712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 msgr2=0x5591227d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x5591227e7da0 tx=0x7fc524005f90 comp rx=0 tx=0).stop 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- 172.21.15.94:0/567086712 shutdown_connections 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55912242f410 0x5591227cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912240e730 0x5591227e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 --2- 172.21.15.94:0/567086712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559122406550 0x5591227d3ee0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- 172.21.15.94:0/567086712 >> 172.21.15.94:0/567086712 conn(0x559122423560 msgr2=0x5591227d88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- 172.21.15.94:0/567086712 shutdown_connections 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- 172.21.15.94:0/567086712 wait complete. 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 -- start start 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.416+0000 7fc540444740 1 Processor -- start 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227de5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x55912284d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341ac950 con 0x55912240e730 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341ac280 con 0x55912242f410 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5341a94c0 con 0x559122406550 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 10 client.0 _ll_register_callbacks cb 0x7fc52c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38366/0 (socket says 172.21.15.94:38366) 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 -- 172.21.15.94:0/3149211870 learned_addr learned my addr 172.21.15.94:0/3149211870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc532ffd640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227de5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5337fe640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x55912284d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 -- 172.21.15.94:0/3149211870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 msgr2=0x5591227de5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227de5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 -- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 msgr2=0x55912284d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x55912284d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 -- 172.21.15.94:0/3149211870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591227df170 con 0x55912240e730 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc532ffd640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227de5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc5327fc640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c6a0 tx=0x7fc52800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2437b87eff5ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc516ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528026040 con 0x55912240e730 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc5280e10f0 con 0x55912240e730 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.417+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5240054d0 con 0x55912240e730 2024-04-25T13:53:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5280a6fc0 con 0x55912240e730 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 10 client.19896 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55912284f050 con 0x55912240e730 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5240054d0 con 0x55912240e730 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fc528018b20 con 0x55912240e730 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 10 client.19896 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 10 client.19896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 20 client.19896 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223c3630 0x559122851a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 4 client.19896 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55912273d580 con 0x5591223c3630 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.418+0000 7fc532ffd640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223c3630 0x559122851a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.419+0000 7fc516ff5640 10 client.19896 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.419+0000 7fc532ffd640 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223c3630 0x559122851a70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60a902d06d77bed server_cookie=4399ea12cd466cc9 in_seq=0 out_seq=0 2024-04-25T13:53:51.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc516ff5640 1 -- 172.21.15.94:0/3149211870 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55912273d580 con 0x5591223c3630 2024-04-25T13:53:51.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc516ff5640 10 client.19896 handle_command_reply: tid=0 2024-04-25T13:53:51.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 client.19896 shutdown 2024-04-25T13:53:51.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 msgr2=0x5591227deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c6a0 tx=0x7fc52800cf20 comp rx=0 tx=0).stop 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 shutdown_connections 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5591223c3630 0x559122851a70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55912242f410 0x55912284d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc516ff5640 0 client.19896 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55912240e730 0x5591227deb00 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 --2- 172.21.15.94:0/3149211870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559122406550 0x5591227de5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 >> 172.21.15.94:0/3149211870 conn(0x559122423560 msgr2=0x5591227cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 shutdown_connections 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 1 -- 172.21.15.94:0/3149211870 wait complete. 2024-04-25T13:53:51.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.420+0000 7fc540444740 20 client.19896 trim_cache size 0 max 16384 2024-04-25T13:53:51.427 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:51.427 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:53:51.427 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "c2d1f867-a738-475e-a322-1300a8daa932", 2024-04-25T13:53:51.427 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:53:51.427 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:51.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 msgr2=0x7fc51c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 0x7fc51c0b2000 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc52c046000 tx=0x7fc52c048000 comp rx=0 tx=0).stop 2024-04-25T13:53:51.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 msgr2=0x7fc53419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e9f0 tx=0x7fc52800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 shutdown_connections 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc53419f4b0 0x7fc5341a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc534103b20 0x7fc53419a530 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 --2- 172.21.15.94:0/1910806237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc534102fc0 0x7fc534199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 >> 172.21.15.94:0/1910806237 conn(0x7fc5340fd340 msgr2=0x7fc5340fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 shutdown_connections 2024-04-25T13:53:51.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.421+0000 7fc53958f640 1 -- 172.21.15.94:0/1910806237 wait complete. 2024-04-25T13:53:51.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:53:51.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/2662197209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481033d0 msgr2=0x7f3d481037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/2662197209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481033d0 0x7f3d481037e0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009900 tx=0x7f3d3802a410 comp rx=0 tx=0).stop 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/2662197209 shutdown_connections 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/2662197209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d481080d0 0x7f3d4810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/2662197209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481033d0 0x7f3d481037e0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/2662197209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d48102190 0x7f3d48102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/2662197209 >> 172.21.15.94:0/2662197209 conn(0x7f3d480fd380 msgr2=0x7f3d480ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/2662197209 shutdown_connections 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.546+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/2662197209 wait complete. 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 Processor -- start 2024-04-25T13:53:51.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 -- start start 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d48102190 0x7f3d4819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d481033d0 0x7f3d4819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4810f630 con 0x7f3d481080d0 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4810f4b0 con 0x7f3d48102190 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4f5c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4810f7b0 con 0x7f3d481033d0 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4db62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4db62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38388/0 (socket says 172.21.15.94:38388) 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4db62640 1 -- 172.21.15.94:0/1162872418 learned_addr learned my addr 172.21.15.94:0/1162872418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4d361640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d48102190 0x7f3d4819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.547+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d481033d0 0x7f3d4819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 -- 172.21.15.94:0/1162872418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d481033d0 msgr2=0x7f3d4819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d481033d0 0x7f3d4819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 -- 172.21.15.94:0/1162872418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d48102190 msgr2=0x7f3d4819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d48102190 0x7f3d4819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 -- 172.21.15.94:0/1162872418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a1d90 con 0x7f3d481080d0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4db62640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000eb10 tx=0x7f3d4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d3e7fc640 1 -- 172.21.15.94:0/1162872418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004b20 con 0x7f3d481080d0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d3e7fc640 1 -- 172.21.15.94:0/1162872418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40004cc0 con 0x7f3d481080d0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a2080 con 0x7f3d481080d0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d3e7fc640 1 -- 172.21.15.94:0/1162872418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40010670 con 0x7f3d481080d0 2024-04-25T13:53:51.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.548+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a2580 con 0x7f3d481080d0 2024-04-25T13:53:51.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d3e7fc640 1 -- 172.21.15.94:0/1162872418 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3d40005020 con 0x7f3d481080d0 2024-04-25T13:53:51.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 Processor -- start 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 -- start start 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb488a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0be4d0 0x5634fb488fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5634fb3edd80 con 0x5634fb0c66b0 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5634fb3a78b0 con 0x5634fb0e7390 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d56479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634fb3e9d40 con 0x5634fb0be4d0 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38390/0 (socket says 172.21.15.94:38390) 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d4cb60640 1 -- 172.21.15.94:0/652890818 learned_addr learned my addr 172.21.15.94:0/652890818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.551+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb488a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4d361640 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0be4d0 0x5634fb488fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d3e7fc640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 0x7f3d340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 -- 172.21.15.94:0/652890818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0be4d0 msgr2=0x5634fb488fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0be4d0 0x5634fb488fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 -- 172.21.15.94:0/652890818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 msgr2=0x5634fb488a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb488a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 -- 172.21.15.94:0/652890818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634fb378a40 con 0x5634fb0c66b0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4db62640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 0x7f3d340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3d380096c0 tx=0x7f3d38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dd877f3c6187d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d3e7fc640 1 -- 172.21.15.94:0/1162872418 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d400df280 con 0x7f3d481080d0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d337fe640 1 -- 172.21.15.94:0/652890818 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38035040 con 0x5634fb0c66b0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d337fe640 1 -- 172.21.15.94:0/652890818 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38031650 con 0x5634fb0c66b0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.552+0000 7f3d337fe640 1 -- 172.21.15.94:0/652890818 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380039b0 con 0x5634fb0c66b0 2024-04-25T13:53:51.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 -- 172.21.15.94:0/652890818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 msgr2=0x5634fb489c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d4db62640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 0x7f3d340b2000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d400dd050 tx=0x7f3d40090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3d380096c0 tx=0x7f3d38000bc0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 -- 172.21.15.94:0/652890818 shutdown_connections 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb489c70 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0be4d0 0x5634fb488fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 --2- 172.21.15.94:0/652890818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb488a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 -- 172.21.15.94:0/652890818 >> 172.21.15.94:0/652890818 conn(0x5634fb0db4e0 msgr2=0x5634fb484270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 -- 172.21.15.94:0/652890818 shutdown_connections 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.553+0000 7f3d56479740 1 -- 172.21.15.94:0/652890818 wait complete. 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 -- start start 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 Processor -- start 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb496db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb4972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5634fb48a720 con 0x5634fb0e7390 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5634fb374d10 con 0x5634fb0be4d0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634fb48a8a0 con 0x5634fb0c66b0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d56479740 10 client.0 _ll_register_callbacks cb 0x5634fb3a1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38406/0 (socket says 172.21.15.94:38406) 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d4cb60640 1 -- 172.21.15.94:0/198901873 learned_addr learned my addr 172.21.15.94:0/198901873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb496db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.554+0000 7f3d4d361640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb4972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 -- 172.21.15.94:0/198901873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 msgr2=0x5634fb4972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb4972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 -- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 msgr2=0x5634fb496db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb496db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 -- 172.21.15.94:0/198901873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634fb1c97d0 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3d38003a00 tx=0x7f3d38009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23435221a47d89a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38035070 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d38031d10 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d38031650 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d38003de0 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d56479740 10 client.19908 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5634fb0ff3f0 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.555+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38031650 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3d3803d760 con 0x5634fb0e7390 2024-04-25T13:53:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 10 client.19908 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 10 client.19908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 20 client.19908 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb0a5f60 0x5634fb4a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 4 client.19908 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634fb3fb990 con 0x5634fb0a5f60 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d4d361640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb0a5f60 0x5634fb4a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d30ff9640 10 client.19908 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.556+0000 7f3d4d361640 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb0a5f60 0x5634fb4a42a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcaeb87772e947c6 server_cookie=2e792244eb8de9ea in_seq=0 out_seq=0 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d30ff9640 1 -- 172.21.15.94:0/198901873 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634fb3fb990 con 0x5634fb0a5f60 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d30ff9640 10 client.19908 handle_command_reply: tid=0 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 client.19908 shutdown 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 msgr2=0x5634fb49e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3d38003a00 tx=0x7f3d38009520 comp rx=0 tx=0).stop 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 shutdown_connections 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb0a5f60 0x5634fb4a42a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0e7390 0x5634fb49e1e0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb4972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d30ff9640 0 client.19908 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 --2- 172.21.15.94:0/198901873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb496db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 >> 172.21.15.94:0/198901873 conn(0x5634fb0db4e0 msgr2=0x5634fb487180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 shutdown_connections 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.557+0000 7f3d56479740 1 -- 172.21.15.94:0/198901873 wait complete. 2024-04-25T13:53:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.558+0000 7f3d56479740 20 client.19908 trim_cache size 0 max 16384 2024-04-25T13:53:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.576+0000 7f3d56479740 1 Processor -- start 2024-04-25T13:53:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.576+0000 7f3d56479740 1 -- start start 2024-04-25T13:53:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb495910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb483ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481a93a0 con 0x5634fb0be4d0 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481aaee0 con 0x5634fb0e7390 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d56479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481a9a70 con 0x5634fb0c66b0 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38422/0 (socket says 172.21.15.94:38422) 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 -- 172.21.15.94:0/2791146026 learned_addr learned my addr 172.21.15.94:0/2791146026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb483ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4d361640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb495910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 -- 172.21.15.94:0/2791146026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 msgr2=0x5634fb495910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb495910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 -- 172.21.15.94:0/2791146026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 msgr2=0x5634fb483ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb483ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 -- 172.21.15.94:0/2791146026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634fb0bf0f0 con 0x5634fb0be4d0 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4d361640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb495910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001e90 tx=0x7f3d38031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b891beedef8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d32ffd640 1 -- 172.21.15.94:0/2791146026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3805e020 con 0x5634fb0be4d0 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d32ffd640 1 -- 172.21.15.94:0/2791146026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3804a3f0 con 0x5634fb0be4d0 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.577+0000 7f3d32ffd640 1 -- 172.21.15.94:0/2791146026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38035070 con 0x5634fb0be4d0 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 -- 172.21.15.94:0/2791146026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 msgr2=0x5634fb4953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d38001e90 tx=0x7f3d38031400 comp rx=0 tx=0).stop 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 -- 172.21.15.94:0/2791146026 shutdown_connections 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0e7390 0x5634fb483ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0c66b0 0x5634fb495910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 --2- 172.21.15.94:0/2791146026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0be4d0 0x5634fb4953d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 -- 172.21.15.94:0/2791146026 >> 172.21.15.94:0/2791146026 conn(0x5634fb0db4e0 msgr2=0x5634fb48d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 -- 172.21.15.94:0/2791146026 shutdown_connections 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.578+0000 7f3d56479740 1 -- 172.21.15.94:0/2791146026 wait complete. 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 -- start start 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 Processor -- start 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb49a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0e7390 0x5634fb49bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481aa810 con 0x5634fb0c66b0 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481aa140 con 0x5634fb0be4d0 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481a7380 con 0x5634fb0e7390 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d56479740 10 client.0 _ll_register_callbacks cb 0x7f3d38032c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d4d361640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d4cb60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb49a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d4d361640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38436/0 (socket says 172.21.15.94:38436) 2024-04-25T13:53:51.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d4d361640 1 -- 172.21.15.94:0/4203609152 learned_addr learned my addr 172.21.15.94:0/4203609152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.579+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0e7390 0x5634fb49bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 -- 172.21.15.94:0/4203609152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0e7390 msgr2=0x5634fb49bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0e7390 0x5634fb49bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 -- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 msgr2=0x5634fb49a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb49a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 -- 172.21.15.94:0/4203609152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634fb3ae4b0 con 0x5634fb0c66b0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d4d361640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x5634fb488780 tx=0x7f3d4400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d67e548aa5f0f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44005020 con 0x5634fb0c66b0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d440051c0 con 0x5634fb0c66b0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5634fb3f5800 con 0x5634fb0c66b0 2024-04-25T13:53:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3d44030050 con 0x5634fb0c66b0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 10 client.19914 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5634fb49d480 con 0x5634fb0c66b0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5634fb3f5800 con 0x5634fb0c66b0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3d44020170 con 0x5634fb0c66b0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 10 client.19914 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 10 client.19914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.580+0000 7f3d56479740 20 client.19914 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb07b3a0 0x5634fb50e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d56479740 4 client.19914 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5634fb3704a0 con 0x5634fb07b3a0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb07b3a0 0x5634fb50e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d30ff9640 10 client.19914 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.581+0000 7f3d4cb60640 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb07b3a0 0x5634fb50e550 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c4bdde840a72fe server_cookie=d302807246772188 in_seq=0 out_seq=0 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d30ff9640 1 -- 172.21.15.94:0/4203609152 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5634fb3704a0 con 0x5634fb07b3a0 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d30ff9640 10 client.19914 handle_command_reply: tid=0 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 client.19914 shutdown 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 msgr2=0x5634fb49aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x5634fb488780 tx=0x7f3d4400bec0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 shutdown_connections 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5634fb07b3a0 0x5634fb50e550 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634fb0e7390 0x5634fb49bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634fb0c66b0 0x5634fb49aeb0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d30ff9640 0 client.19914 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 --2- 172.21.15.94:0/4203609152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634fb0be4d0 0x5634fb49a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 >> 172.21.15.94:0/4203609152 conn(0x5634fb0db4e0 msgr2=0x5634fb495310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 shutdown_connections 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 1 -- 172.21.15.94:0/4203609152 wait complete. 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.583+0000 7f3d56479740 20 client.19914 trim_cache size 0 max 16384 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "c2d1f867-a738-475e-a322-1300a8daa932": { 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "c2d1f867-a738-475e-a322-1300a8daa932", 2024-04-25T13:53:51.588 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:53:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:53:51.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:53:51.589 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:53:51.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 msgr2=0x7f3d340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 0x7f3d340b2000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d400dd050 tx=0x7f3d40090040 comp rx=0 tx=0).stop 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 msgr2=0x7f3d481a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000eb10 tx=0x7f3d4000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 shutdown_connections 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d340afaf0 0x7f3d340b2000 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d481080d0 0x7f3d481a1720 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d481033d0 0x7f3d4819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 --2- 172.21.15.94:0/1162872418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d48102190 0x7f3d4819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 >> 172.21.15.94:0/1162872418 conn(0x7f3d480fd380 msgr2=0x7f3d481040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 shutdown_connections 2024-04-25T13:53:51.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:53:51.584+0000 7f3d4f5c5640 1 -- 172.21.15.94:0/1162872418 wait complete. 2024-04-25T13:53:51.594 INFO:tasks.cephfs.filesystem:scrub status for tag:c2d1f867-a738-475e-a322-1300a8daa932 - {'path': '/', 'tag': 'c2d1f867-a738-475e-a322-1300a8daa932', 'options': 'recursive,force'} 2024-04-25T13:53:51.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1499200 12.58 MB/sec execute 514 sec latency 16.289 ms 2024-04-25T13:53:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:52 smithi137 ceph-mon[32568]: pgmap v711: 97 pgs: 97 active+clean; 43 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:53:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:52 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:53:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:52 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:53:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:52 smithi094 ceph-mon[29195]: pgmap v711: 97 pgs: 97 active+clean; 43 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:53:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:52 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:53:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:52 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:53:52.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:52 smithi138 ceph-mon[32977]: pgmap v711: 97 pgs: 97 active+clean; 43 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-04-25T13:53:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:52 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:53:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:52 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:53:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:53:52.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1500737 12.57 MB/sec execute 515 sec latency 54.316 ms 2024-04-25T13:53:53.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1503226 12.58 MB/sec execute 516 sec latency 14.903 ms 2024-04-25T13:53:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:54 smithi137 ceph-mon[32568]: pgmap v712: 97 pgs: 97 active+clean; 39 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-04-25T13:53:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:54 smithi094 ceph-mon[29195]: pgmap v712: 97 pgs: 97 active+clean; 39 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-04-25T13:53:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:54 smithi138 ceph-mon[32977]: pgmap v712: 97 pgs: 97 active+clean; 39 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-04-25T13:53:54.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1506117 12.58 MB/sec execute 517 sec latency 22.583 ms 2024-04-25T13:53:55.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1508003 12.57 MB/sec execute 518 sec latency 22.431 ms 2024-04-25T13:53:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:56 smithi137 ceph-mon[32568]: pgmap v713: 97 pgs: 97 active+clean; 40 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-04-25T13:53:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:53:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:56 smithi094 ceph-mon[29195]: pgmap v713: 97 pgs: 97 active+clean; 40 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-04-25T13:53:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:53:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:56 smithi138 ceph-mon[32977]: pgmap v713: 97 pgs: 97 active+clean; 40 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-04-25T13:53:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:53:56.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1510445 12.58 MB/sec execute 519 sec latency 20.083 ms 2024-04-25T13:53:57.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1513350 12.59 MB/sec execute 520 sec latency 30.831 ms 2024-04-25T13:53:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:53:58 smithi137 ceph-mon[32568]: pgmap v714: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-04-25T13:53:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:53:58 smithi094 ceph-mon[29195]: pgmap v714: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-04-25T13:53:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:53:58 smithi138 ceph-mon[32977]: pgmap v714: 97 pgs: 97 active+clean; 44 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 363 op/s 2024-04-25T13:53:58.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1514657 12.57 MB/sec execute 521 sec latency 79.065 ms 2024-04-25T13:53:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- 172.21.15.94:0/3652632144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 msgr2=0x7f53781033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 --2- 172.21.15.94:0/3652632144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f53781033a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f536c009900 tx=0x7f536c02a410 comp rx=0 tx=0).stop 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- 172.21.15.94:0/3652632144 shutdown_connections 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 --2- 172.21.15.94:0/3652632144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53781044b0 0x7f537810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 --2- 172.21.15.94:0/3652632144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5378103970 0x7f5378103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 --2- 172.21.15.94:0/3652632144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f53781033a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- 172.21.15.94:0/3652632144 >> 172.21.15.94:0/3652632144 conn(0x7f53780fd390 msgr2=0x7f53780ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- 172.21.15.94:0/3652632144 shutdown_connections 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- 172.21.15.94:0/3652632144 wait complete. 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 Processor -- start 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.505+0000 7f537f18f640 1 -- start start 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378103970 0x7f5378199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53781044b0 0x7f53781a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f537810e540 con 0x7f5378102f90 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f537810e3c0 con 0x7f53781044b0 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f537810e6c0 con 0x7f5378103970 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33338/0 (socket says 172.21.15.94:33338) 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 -- 172.21.15.94:0/3683785850 learned_addr learned my addr 172.21.15.94:0/3683785850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537d72c640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53781044b0 0x7f53781a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f5377fff640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378103970 0x7f5378199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 -- 172.21.15.94:0/3683785850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378103970 msgr2=0x7f5378199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378103970 0x7f5378199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 -- 172.21.15.94:0/3683785850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53781044b0 msgr2=0x7f53781a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53781044b0 0x7f53781a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 -- 172.21.15.94:0/3683785850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53781a0c90 con 0x7f5378102f90 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f536c006010 tx=0x7f536c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f5375ffb640 1 -- 172.21.15.94:0/3683785850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c03f030 con 0x7f5378102f90 2024-04-25T13:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.506+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53781a0f20 con 0x7f5378102f90 2024-04-25T13:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.507+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781a1380 con 0x7f5378102f90 2024-04-25T13:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.507+0000 7f5375ffb640 1 -- 172.21.15.94:0/3683785850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c031910 con 0x7f5378102f90 2024-04-25T13:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.507+0000 7f5375ffb640 1 -- 172.21.15.94:0/3683785850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c049060 con 0x7f5378102f90 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.509+0000 7f5375ffb640 1 -- 172.21.15.94:0/3683785850 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f536c042030 con 0x7f5378102f90 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.509+0000 7f5386043740 1 Processor -- start 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.509+0000 7f5386043740 1 -- start start 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.509+0000 7f5386043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c9836b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5386043740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c9837100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5386043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5386043740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592c979a460 con 0x5592c946b2f0 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5386043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592c9754590 con 0x5592c94734d0 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5386043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592c9796420 con 0x5592c94941b0 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c9837100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56490/0 (socket says 172.21.15.94:56490) 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 -- 172.21.15.94:0/2900356035 learned_addr learned my addr 172.21.15.94:0/2900356035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f537d72c640 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c9836b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5375ffb640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 0x7f53680b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 -- 172.21.15.94:0/2900356035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 msgr2=0x5592c9836b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c9836b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 -- 172.21.15.94:0/2900356035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 msgr2=0x5592c9837100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c9837100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 -- 172.21.15.94:0/2900356035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c97610d0 con 0x5592c94734d0 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f537cf2b640 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c9837100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f537d72c640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 0x7f53680b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5375ffb640 1 -- 172.21.15.94:0/3683785850 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f536c0feee0 con 0x7f5378102f90 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5377fff640 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5364009c90 tx=0x7f5364011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed058c7a5ab8744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.510+0000 7f5362ffd640 1 -- 172.21.15.94:0/2900356035 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364005220 con 0x5592c94734d0 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.511+0000 7f5362ffd640 1 -- 172.21.15.94:0/2900356035 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364004c50 con 0x5592c94734d0 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.511+0000 7f5362ffd640 1 -- 172.21.15.94:0/2900356035 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640197a0 con 0x5592c94734d0 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- 172.21.15.94:0/2900356035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 msgr2=0x5592c983e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f537d72c640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 0x7f53680b1ff0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5370009fd0 tx=0x7f5370004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5364009c90 tx=0x7f5364011810 comp rx=0 tx=0).stop 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- 172.21.15.94:0/2900356035 shutdown_connections 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c983e000 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c9837100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 --2- 172.21.15.94:0/2900356035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c9836b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- 172.21.15.94:0/2900356035 >> 172.21.15.94:0/2900356035 conn(0x5592c9488300 msgr2=0x5592c9830300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- 172.21.15.94:0/2900356035 shutdown_connections 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- 172.21.15.94:0/2900356035 wait complete. 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 -- start start 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.512+0000 7f5386043740 1 Processor -- start 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9840d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592c983ea50 con 0x5592c94941b0 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592c9721b60 con 0x5592c946b2f0 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592c983ebd0 con 0x5592c94734d0 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5386043740 10 client.0 _ll_register_callbacks cb 0x5592c97462b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33354/0 (socket says 172.21.15.94:33354) 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 -- 172.21.15.94:0/2323704780 learned_addr learned my addr 172.21.15.94:0/2323704780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537cf2b640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56500/0 (socket says 172.21.15.94:56500) 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f5377fff640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9840d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 -- 172.21.15.94:0/2323704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 msgr2=0x5592c9840d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9840d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 -- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 msgr2=0x5592c98407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537d72c640 1 -- 172.21.15.94:0/2323704780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c94ac210 con 0x5592c94941b0 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.513+0000 7f537cf2b640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f537d72c640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f537000fb90 tx=0x7f5370004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d7f295f992514d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537001eac0 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f537001ec60 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5364004c50 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f537001ee60 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 10 client.19926 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5592c9578620 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364004c50 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f53700284b0 con 0x5592c94941b0 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 10 client.19926 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 10 client.19926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 20 client.19926 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c9452d80 0x5592c984e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 4 client.19926 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.514+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c97a8070 con 0x5592c9452d80 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.515+0000 7f537cf2b640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c9452d80 0x5592c984e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.515+0000 7f5357fff640 10 client.19926 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.515+0000 7f537cf2b640 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c9452d80 0x5592c984e640 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e728d15eb4f6b3e server_cookie=d04c5a2448362234 in_seq=0 out_seq=0 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5357fff640 1 -- 172.21.15.94:0/2323704780 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592c97a8070 con 0x5592c9452d80 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5357fff640 10 client.19926 handle_command_reply: tid=0 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 client.19926 shutdown 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 msgr2=0x5592c9847c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f537000fb90 tx=0x7f5370004ab0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 shutdown_connections 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c9452d80 0x5592c984e640 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9847c90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9840d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5357fff640 0 client.19926 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 --2- 172.21.15.94:0/2323704780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.516+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 >> 172.21.15.94:0/2323704780 conn(0x5592c9488300 msgr2=0x5592c982f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.517+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 shutdown_connections 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.517+0000 7f5386043740 1 -- 172.21.15.94:0/2323704780 wait complete. 2024-04-25T13:53:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.517+0000 7f5386043740 20 client.19926 trim_cache size 0 max 16384 2024-04-25T13:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 Processor -- start 2024-04-25T13:53:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 -- start start 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c9847bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c982fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f535c00e730 con 0x5592c946b2f0 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f535c010270 con 0x5592c94734d0 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5386043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f535c00ee00 con 0x5592c94941b0 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33366/0 (socket says 172.21.15.94:33366) 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f537cf2b640 1 -- 172.21.15.94:0/3101840146 learned_addr learned my addr 172.21.15.94:0/3101840146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f537d72c640 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c982fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.532+0000 7f5377fff640 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c9847bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 -- 172.21.15.94:0/3101840146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 msgr2=0x5592c982fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c982fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 -- 172.21.15.94:0/3101840146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 msgr2=0x5592c9847bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c9847bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 -- 172.21.15.94:0/3101840146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c98480f0 con 0x5592c946b2f0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f536c02a930 tx=0x7f536c0314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1569b8ee4a31d80f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f53627fc640 1 -- 172.21.15.94:0/3101840146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c035070 con 0x5592c946b2f0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f53627fc640 1 -- 172.21.15.94:0/3101840146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0c6ee0 con 0x5592c946b2f0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.533+0000 7f53627fc640 1 -- 172.21.15.94:0/3101840146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0c7630 con 0x5592c946b2f0 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- 172.21.15.94:0/3101840146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 msgr2=0x5592c982af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f536c02a930 tx=0x7f536c0314c0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- 172.21.15.94:0/3101840146 shutdown_connections 2024-04-25T13:53:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94941b0 0x5592c982fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c94734d0 0x5592c9847bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 --2- 172.21.15.94:0/3101840146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c946b2f0 0x5592c982af40 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- 172.21.15.94:0/3101840146 >> 172.21.15.94:0/3101840146 conn(0x5592c9488300 msgr2=0x5592c983abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- 172.21.15.94:0/3101840146 shutdown_connections 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- 172.21.15.94:0/3101840146 wait complete. 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.534+0000 7f5386043740 1 -- start start 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 Processor -- start 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9845610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9846540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f535c00fba0 con 0x5592c94941b0 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f535c00f4d0 con 0x5592c946b2f0 2024-04-25T13:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f535c00c710 con 0x5592c94734d0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5386043740 10 client.0 _ll_register_callbacks cb 0x7f5370009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f537d72c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9846540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f537cf2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56522/0 (socket says 172.21.15.94:56522) 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f537cf2b640 1 -- 172.21.15.94:0/3396758800 learned_addr learned my addr 172.21.15.94:0/3396758800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.535+0000 7f5377fff640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9845610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 -- 172.21.15.94:0/3396758800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 msgr2=0x5592c9845610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9845610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 -- 172.21.15.94:0/3396758800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 msgr2=0x5592c9846540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9846540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 -- 172.21.15.94:0/3396758800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c9846b00 con 0x5592c946b2f0 2024-04-25T13:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f537cf2b640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f536c0fccb0 tx=0x7f536c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805e46b65971e1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5357fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c035040 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f536c0c7840 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c0c6ee0 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f536c0c6b40 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5386043740 10 client.29372 fetch_fsmap learned FSMap version 31 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5592c9847d90 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.536+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0c6ee0 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f536c0c1b60 con 0x5592c946b2f0 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 10 client.29372 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 10 client.29372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 20 client.29372 populate_metadata read hostname 'smithi094' 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c94281c0 0x5592c98b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 4 client.29372 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592c97a1ee0 con 0x5592c94281c0 2024-04-25T13:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5377fff640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c94281c0 0x5592c98b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5357fff640 10 client.29372 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.537+0000 7f5377fff640 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c94281c0 0x5592c98b8ff0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a360c35dc11981 server_cookie=12246b5461739830 in_seq=0 out_seq=0 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5357fff640 1 -- 172.21.15.94:0/3396758800 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592c97a1ee0 con 0x5592c94281c0 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5357fff640 10 client.29372 handle_command_reply: tid=0 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 client.29372 shutdown 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 msgr2=0x5592c98450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f536c0fccb0 tx=0x7f536c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 shutdown_connections 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592c94281c0 0x5592c98b8ff0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592c94941b0 0x5592c9846540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5357fff640 0 client.29372 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:53:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592c94734d0 0x5592c9845610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 --2- 172.21.15.94:0/3396758800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592c946b2f0 0x5592c98450d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.723+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 >> 172.21.15.94:0/3396758800 conn(0x5592c9488300 msgr2=0x5592c982fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.724+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 shutdown_connections 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.724+0000 7f5386043740 1 -- 172.21.15.94:0/3396758800 wait complete. 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.724+0000 7f5386043740 20 client.29372 trim_cache size 0 max 16384 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:53:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:53:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.724+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 msgr2=0x7f53680b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.724+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 0x7f53680b1ff0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5370009fd0 tx=0x7f5370004a40 comp rx=0 tx=0).stop 2024-04-25T13:53:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 msgr2=0x7f5378199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f536c006010 tx=0x7f536c004270 comp rx=0 tx=0).stop 2024-04-25T13:53:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 shutdown_connections 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53680afae0 0x7f53680b1ff0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53781044b0 0x7f53781a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378103970 0x7f5378199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 --2- 172.21.15.94:0/3683785850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5378102f90 0x7f5378199150 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 >> 172.21.15.94:0/3683785850 conn(0x7f53780fd390 msgr2=0x7f5378104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 shutdown_connections 2024-04-25T13:53:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:53:59.725+0000 7f537f18f640 1 -- 172.21.15.94:0/3683785850 wait complete. 2024-04-25T13:53:59.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1517178 12.58 MB/sec execute 522 sec latency 46.584 ms 2024-04-25T13:54:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:00 smithi137 ceph-mon[32568]: pgmap v715: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:54:00.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:00 smithi094 ceph-mon[29195]: pgmap v715: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:54:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:00 smithi138 ceph-mon[32977]: pgmap v715: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 358 op/s 2024-04-25T13:54:00.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1519383 12.57 MB/sec execute 523 sec latency 37.229 ms 2024-04-25T13:54:01.819 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1521272 12.57 MB/sec execute 524 sec latency 16.868 ms 2024-04-25T13:54:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:02 smithi137 ceph-mon[32568]: pgmap v716: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:54:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:02 smithi094 ceph-mon[29195]: pgmap v716: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:54:02.749 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:02 smithi138 ceph-mon[32977]: pgmap v716: 97 pgs: 97 active+clean; 45 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-04-25T13:54:02.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1524265 12.58 MB/sec execute 525 sec latency 12.833 ms 2024-04-25T13:54:03.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1525641 12.56 MB/sec execute 526 sec latency 107.159 ms 2024-04-25T13:54:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:04 smithi137 ceph-mon[32568]: pgmap v717: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-04-25T13:54:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:04 smithi094 ceph-mon[29195]: pgmap v717: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-04-25T13:54:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:04 smithi138 ceph-mon[32977]: pgmap v717: 97 pgs: 97 active+clean; 43 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-04-25T13:54:04.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1528125 12.57 MB/sec execute 527 sec latency 13.335 ms 2024-04-25T13:54:05.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1531088 12.58 MB/sec execute 528 sec latency 13.142 ms 2024-04-25T13:54:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:06 smithi137 ceph-mon[32568]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-04-25T13:54:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:06 smithi094 ceph-mon[29195]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-04-25T13:54:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:06 smithi138 ceph-mon[32977]: pgmap v718: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-04-25T13:54:06.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1532829 12.57 MB/sec execute 529 sec latency 14.952 ms 2024-04-25T13:54:07.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1535243 12.57 MB/sec execute 530 sec latency 10.773 ms 2024-04-25T13:54:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:08 smithi137 ceph-mon[32568]: pgmap v719: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-04-25T13:54:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:08 smithi094 ceph-mon[29195]: pgmap v719: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-04-25T13:54:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:08 smithi138 ceph-mon[32977]: pgmap v719: 97 pgs: 97 active+clean; 45 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-04-25T13:54:08.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1537431 12.57 MB/sec execute 531 sec latency 82.479 ms 2024-04-25T13:54:09.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1539000 12.56 MB/sec execute 532 sec latency 17.069 ms 2024-04-25T13:54:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:10 smithi137 ceph-mon[32568]: pgmap v720: 97 pgs: 97 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-04-25T13:54:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:10 smithi094 ceph-mon[29195]: pgmap v720: 97 pgs: 97 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-04-25T13:54:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:10 smithi138 ceph-mon[32977]: pgmap v720: 97 pgs: 97 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 308 op/s 2024-04-25T13:54:10.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1541831 12.57 MB/sec execute 533 sec latency 17.650 ms 2024-04-25T13:54:11.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1543402 12.56 MB/sec execute 534 sec latency 18.803 ms 2024-04-25T13:54:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:12 smithi137 ceph-mon[32568]: pgmap v721: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-04-25T13:54:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:12 smithi094 ceph-mon[29195]: pgmap v721: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-04-25T13:54:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:12 smithi138 ceph-mon[32977]: pgmap v721: 97 pgs: 97 active+clean; 48 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-04-25T13:54:12.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1545697 12.56 MB/sec execute 535 sec latency 15.265 ms 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.822+0000 7f7b9e815640 1 -- 172.21.15.94:0/3559526489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 msgr2=0x7f7b9810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.822+0000 7f7b9e815640 1 --2- 172.21.15.94:0/3559526489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9810bd00 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009ca0 tx=0x7f7b9002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.822+0000 7f7b9e815640 1 -- 172.21.15.94:0/3559526489 shutdown_connections 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.822+0000 7f7b9e815640 1 --2- 172.21.15.94:0/3559526489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9810bd00 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.822+0000 7f7b9e815640 1 --2- 172.21.15.94:0/3559526489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981017a0 0x7f7b98101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 --2- 172.21.15.94:0/3559526489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981050f0 0x7f7b98105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 -- 172.21.15.94:0/3559526489 >> 172.21.15.94:0/3559526489 conn(0x7f7b980fd3c0 msgr2=0x7f7b980ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 -- 172.21.15.94:0/3559526489 shutdown_connections 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 -- 172.21.15.94:0/3559526489 wait complete. 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 Processor -- start 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.823+0000 7f7b9e815640 1 -- start start 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981017a0 0x7f7b9819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981050f0 0x7f7b981a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9810f680 con 0x7f7b981022a0 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9810f500 con 0x7f7b981050f0 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9810f800 con 0x7f7b981017a0 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56490/0 (socket says 172.21.15.94:56490) 2024-04-25T13:54:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 -- 172.21.15.94:0/2825625553 learned_addr learned my addr 172.21.15.94:0/2825625553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981050f0 0x7f7b981a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b97fff640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981017a0 0x7f7b9819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 -- 172.21.15.94:0/2825625553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981017a0 msgr2=0x7f7b9819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981017a0 0x7f7b9819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 -- 172.21.15.94:0/2825625553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981050f0 msgr2=0x7f7b981a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981050f0 0x7f7b981a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 -- 172.21.15.94:0/2825625553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b981a1d20 con 0x7f7b981022a0 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e9f0 tx=0x7f7b8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b957fa640 1 -- 172.21.15.94:0/2825625553 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00cd80 con 0x7f7b981022a0 2024-04-25T13:54:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b957fa640 1 -- 172.21.15.94:0/2825625553 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c0108e0 con 0x7f7b981022a0 2024-04-25T13:54:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b981a1fb0 con 0x7f7b981022a0 2024-04-25T13:54:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b957fa640 1 -- 172.21.15.94:0/2825625553 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c018980 con 0x7f7b981022a0 2024-04-25T13:54:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.824+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b981a24a0 con 0x7f7b981022a0 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b957fa640 1 -- 172.21.15.94:0/2825625553 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7b8c010430 con 0x7f7b981022a0 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 Processor -- start 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 -- start start 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3ba1b0 0x55880c75a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c761df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55880c6c0460 con 0x55880c3912f0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55880c67a590 con 0x55880c3994d0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55880c6bc420 con 0x55880c3ba1b0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b957fa640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 0x7f7b800b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56494/0 (socket says 172.21.15.94:56494) 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b97fff640 1 -- 172.21.15.94:0/567130142 learned_addr learned my addr 172.21.15.94:0/567130142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.827+0000 7f7b9cdb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3ba1b0 0x55880c75a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b977fe640 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c761df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 0x7f7b800b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 -- 172.21.15.94:0/567130142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3ba1b0 msgr2=0x55880c75a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3ba1b0 0x55880c75a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 -- 172.21.15.94:0/567130142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 msgr2=0x55880c761df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c761df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b957fa640 1 -- 172.21.15.94:0/2825625553 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7b8c0def90 con 0x7f7b981022a0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 -- 172.21.15.94:0/567130142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880c6870d0 con 0x55880c3912f0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b97fff640 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800bc70 tx=0x7f7b880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebd0e6a8109083e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 0x7f7b800b1ff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b9002f500 tx=0x7f7b9002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b7e7fc640 1 -- 172.21.15.94:0/567130142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8801c680 con 0x55880c3912f0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b7e7fc640 1 -- 172.21.15.94:0/567130142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8800be20 con 0x55880c3912f0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.828+0000 7f7b7e7fc640 1 -- 172.21.15.94:0/567130142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88011aa0 con 0x55880c3912f0 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- 172.21.15.94:0/567130142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 msgr2=0x55880c75aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800bc70 tx=0x7f7b880107a0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- 172.21.15.94:0/567130142 shutdown_connections 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c761df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c75aef0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/567130142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3ba1b0 0x55880c75a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- 172.21.15.94:0/567130142 >> 172.21.15.94:0/567130142 conn(0x55880c3ae300 msgr2=0x55880c756300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- 172.21.15.94:0/567130142 shutdown_connections 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- 172.21.15.94:0/567130142 wait complete. 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.829+0000 7f7ba57a6740 1 -- start start 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 Processor -- start 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c764be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3ba1b0 0x55880c76bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55880c762840 con 0x55880c3912f0 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55880c647b60 con 0x55880c3ba1b0 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55880c7629c0 con 0x55880c3994d0 2024-04-25T13:54:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7ba57a6740 10 client.0 _ll_register_callbacks cb 0x55880c66c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56504/0 (socket says 172.21.15.94:56504) 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 -- 172.21.15.94:0/2610121193 learned_addr learned my addr 172.21.15.94:0/2610121193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c764be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3ba1b0 0x55880c76bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 -- 172.21.15.94:0/2610121193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 msgr2=0x55880c764be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c764be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 -- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3ba1b0 msgr2=0x55880c76bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3ba1b0 0x55880c76bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.830+0000 7f7b97fff640 1 -- 172.21.15.94:0/2610121193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880c3d2210 con 0x55880c3912f0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b97fff640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b5f0 tx=0x7f7b88005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2629e30a777e366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8802b060 con 0x55880c3912f0 2024-04-25T13:54:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7b880117a0 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8800be20 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b880119a0 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 10 client.19944 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55880c49e620 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8800be20 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8801c480 con 0x55880c3912f0 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 10 client.19944 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 10 client.19944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 20 client.19944 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c378d80 0x55880c771fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 4 client.19944 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.831+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55880c6ce070 con 0x55880c378d80 2024-04-25T13:54:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.832+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c378d80 0x55880c771fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.832+0000 7f7b777fe640 10 client.19944 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.832+0000 7f7b977fe640 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c378d80 0x55880c771fe0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402d9a3ae31bccd6 server_cookie=80747019b707c32d in_seq=0 out_seq=0 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7b777fe640 1 -- 172.21.15.94:0/2610121193 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55880c6ce070 con 0x55880c378d80 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7b777fe640 10 client.19944 handle_command_reply: tid=0 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 client.19944 shutdown 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 msgr2=0x55880c7646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b5f0 tx=0x7f7b88005270 comp rx=0 tx=0).stop 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 shutdown_connections 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c378d80 0x55880c771fe0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3ba1b0 0x55880c76bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c764be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7b777fe640 0 client.19944 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/2610121193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c7646a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 >> 172.21.15.94:0/2610121193 conn(0x55880c3ae300 msgr2=0x55880c759210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 shutdown_connections 2024-04-25T13:54:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 1 -- 172.21.15.94:0/2610121193 wait complete. 2024-04-25T13:54:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.833+0000 7f7ba57a6740 20 client.19944 trim_cache size 0 max 16384 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 Processor -- start 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 -- start start 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3912f0 0x55880c777bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c76b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ab560 con 0x55880c378d80 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ad0a0 con 0x55880c3994d0 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981abc30 con 0x55880c3912f0 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56518/0 (socket says 172.21.15.94:56518) 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 -- 172.21.15.94:0/1536765618 learned_addr learned my addr 172.21.15.94:0/1536765618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b977fe640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3912f0 0x55880c777bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c76b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 -- 172.21.15.94:0/1536765618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3912f0 msgr2=0x55880c777bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3912f0 0x55880c777bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 -- 172.21.15.94:0/1536765618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 msgr2=0x55880c76b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c76b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.849+0000 7f7b97fff640 1 -- 172.21.15.94:0/1536765618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880c75baf0 con 0x55880c378d80 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.850+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c76b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.850+0000 7f7b97fff640 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f7b88010ca0 tx=0x7f7b88010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9196244269f6e39a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.850+0000 7f7b7dffb640 1 -- 172.21.15.94:0/1536765618 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8802b040 con 0x55880c378d80 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.850+0000 7f7b7dffb640 1 -- 172.21.15.94:0/1536765618 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8801c480 con 0x55880c378d80 2024-04-25T13:54:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.850+0000 7f7b7dffb640 1 -- 172.21.15.94:0/1536765618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88011550 con 0x55880c378d80 2024-04-25T13:54:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- 172.21.15.94:0/1536765618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 msgr2=0x55880c777690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f7b88010ca0 tx=0x7f7b88010fd0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- 172.21.15.94:0/1536765618 shutdown_connections 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c3994d0 0x55880c76b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3912f0 0x55880c777bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/1536765618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c378d80 0x55880c777690 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- 172.21.15.94:0/1536765618 >> 172.21.15.94:0/1536765618 conn(0x55880c3ae300 msgr2=0x55880c75eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- 172.21.15.94:0/1536765618 shutdown_connections 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- 172.21.15.94:0/1536765618 wait complete. 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 -- start start 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.851+0000 7f7ba57a6740 1 Processor -- start 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c378d80 0x55880c76b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c7d8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ac9d0 con 0x55880c3912f0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ac300 con 0x55880c378d80 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981a9540 con 0x55880c3994d0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 10 client.0 _ll_register_callbacks cb 0x7f7b8c0a3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56520/0 (socket says 172.21.15.94:56520) 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 -- 172.21.15.94:0/223918488 learned_addr learned my addr 172.21.15.94:0/223918488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b97fff640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c378d80 0x55880c76b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c7d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 -- 172.21.15.94:0/223918488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 msgr2=0x55880c7d8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c7d8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 -- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c378d80 msgr2=0x55880c76b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c378d80 0x55880c76b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 -- 172.21.15.94:0/223918488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880c76bf30 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b9cdb2640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c7d8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b977fe640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0a7410 tx=0x7f7b8c0a3470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf7494969ff3ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b777fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0ab780 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8801c480 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.852+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7b8c0e10d0 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c0b0230 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 10 client.19950 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55880c7da000 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8801c480 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8c018cb0 con 0x55880c3912f0 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 10 client.19950 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 10 client.19950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 20 client.19950 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c3ba1b0 0x55880c7dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 4 client.19950 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55880c6c7ee0 con 0x55880c3ba1b0 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b97fff640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c3ba1b0 0x55880c7dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b777fe640 10 client.19950 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:12.853+0000 7f7b97fff640 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c3ba1b0 0x55880c7dc9a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b4967d17373c37 server_cookie=c3d12c93a449b67c in_seq=0 out_seq=0 2024-04-25T13:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.053+0000 7f7b777fe640 1 -- 172.21.15.94:0/223918488 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55880c6c7ee0 con 0x55880c3ba1b0 2024-04-25T13:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.053+0000 7f7b777fe640 10 client.19950 handle_command_reply: tid=0 2024-04-25T13:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.053+0000 7f7ba57a6740 1 client.19950 shutdown 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 msgr2=0x55880c76b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0a7410 tx=0x7f7b8c0a3470 comp rx=0 tx=0).stop 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 shutdown_connections 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55880c3ba1b0 0x55880c7dc9a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55880c3994d0 0x55880c7d8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55880c3912f0 0x55880c76b680 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 --2- 172.21.15.94:0/223918488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55880c378d80 0x55880c76b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7b777fe640 0 client.19950 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 >> 172.21.15.94:0/223918488 conn(0x55880c3ae300 msgr2=0x55880c764250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 shutdown_connections 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 1 -- 172.21.15.94:0/223918488 wait complete. 2024-04-25T13:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.054+0000 7f7ba57a6740 20 client.19950 trim_cache size 0 max 16384 2024-04-25T13:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 msgr2=0x7f7b800b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 0x7f7b800b1ff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b9002f500 tx=0x7f7b9002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 msgr2=0x7f7b9819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e9f0 tx=0x7f7b8c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 shutdown_connections 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b800afae0 0x7f7b800b1ff0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b981050f0 0x7f7b981a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b981022a0 0x7f7b9819a770 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 --2- 172.21.15.94:0/2825625553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b981017a0 0x7f7b9819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 >> 172.21.15.94:0/2825625553 conn(0x7f7b980fd3c0 msgr2=0x7f7b98107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 shutdown_connections 2024-04-25T13:54:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:13.055+0000 7f7b9e815640 1 -- 172.21.15.94:0/2825625553 wait complete. 2024-04-25T13:54:13.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1547396 12.54 MB/sec execute 536 sec latency 122.978 ms 2024-04-25T13:54:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:14 smithi137 ceph-mon[32568]: pgmap v722: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-04-25T13:54:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:14 smithi094 ceph-mon[29195]: pgmap v722: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-04-25T13:54:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:14 smithi138 ceph-mon[32977]: pgmap v722: 97 pgs: 97 active+clean; 50 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 349 op/s 2024-04-25T13:54:14.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1549531 12.55 MB/sec execute 537 sec latency 17.943 ms 2024-04-25T13:54:15.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1552536 12.56 MB/sec execute 538 sec latency 11.270 ms 2024-04-25T13:54:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:16 smithi137 ceph-mon[32568]: pgmap v723: 97 pgs: 97 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-04-25T13:54:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:16 smithi094 ceph-mon[29195]: pgmap v723: 97 pgs: 97 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-04-25T13:54:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:16 smithi138 ceph-mon[32977]: pgmap v723: 97 pgs: 97 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 440 op/s 2024-04-25T13:54:16.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1554839 12.55 MB/sec execute 539 sec latency 15.679 ms 2024-04-25T13:54:17.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1556857 12.55 MB/sec execute 540 sec latency 16.312 ms 2024-04-25T13:54:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:18 smithi137 ceph-mon[32568]: pgmap v724: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:54:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:18 smithi094 ceph-mon[29195]: pgmap v724: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:54:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:18 smithi138 ceph-mon[32977]: pgmap v724: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-04-25T13:54:18.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1559621 12.56 MB/sec execute 541 sec latency 93.617 ms 2024-04-25T13:54:19.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1561665 12.55 MB/sec execute 542 sec latency 19.742 ms 2024-04-25T13:54:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:20 smithi137 ceph-mon[32568]: pgmap v725: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-04-25T13:54:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:20 smithi094 ceph-mon[29195]: pgmap v725: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-04-25T13:54:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:20 smithi138 ceph-mon[32977]: pgmap v725: 97 pgs: 97 active+clean; 41 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-04-25T13:54:20.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1563759 12.55 MB/sec execute 543 sec latency 14.063 ms 2024-04-25T13:54:21.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:54:21.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:54:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 -- 172.21.15.94:0/3955707504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 msgr2=0x7f98641036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 --2- 172.21.15.94:0/3955707504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f98641036e0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ca0 tx=0x7f986002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 -- 172.21.15.94:0/3955707504 shutdown_connections 2024-04-25T13:54:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 --2- 172.21.15.94:0/3955707504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864108920 0x7f986410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 --2- 172.21.15.94:0/3955707504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864103c20 0x7f9864104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 --2- 172.21.15.94:0/3955707504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f98641036e0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 -- 172.21.15.94:0/3955707504 >> 172.21.15.94:0/3955707504 conn(0x7f98640fdbd0 msgr2=0x7f9864100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 -- 172.21.15.94:0/3955707504 shutdown_connections 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.713+0000 7f986bae7640 1 -- 172.21.15.94:0/3955707504 wait complete. 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 Processor -- start 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 -- start start 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864103c20 0x7f986419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864108920 0x7f98641a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f986410f600 con 0x7f98641029e0 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f986410f480 con 0x7f9864103c20 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986bae7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f986410f780 con 0x7f9864108920 2024-04-25T13:54:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f9869883640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f9869883640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36710/0 (socket says 172.21.15.94:36710) 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f9869883640 1 -- 172.21.15.94:0/3726208091 learned_addr learned my addr 172.21.15.94:0/3726208091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f9869082640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864103c20 0x7f986419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.714+0000 7f986a084640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864108920 0x7f98641a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 -- 172.21.15.94:0/3726208091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864108920 msgr2=0x7f98641a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864108920 0x7f98641a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 -- 172.21.15.94:0/3726208091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864103c20 msgr2=0x7f986419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864103c20 0x7f986419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 -- 172.21.15.94:0/3726208091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641a20a0 con 0x7f98641029e0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f9869883640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f986002ade0 tx=0x7f98600316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f985affd640 1 -- 172.21.15.94:0/3726208091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860003b70 con 0x7f98641029e0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f985affd640 1 -- 172.21.15.94:0/3726208091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860003d10 con 0x7f98641029e0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641a2330 con 0x7f98641029e0 2024-04-25T13:54:21.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f985affd640 1 -- 172.21.15.94:0/3726208091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986002fb90 con 0x7f98641029e0 2024-04-25T13:54:21.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.715+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a2880 con 0x7f98641029e0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f985affd640 1 -- 172.21.15.94:0/3726208091 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f986002fd30 con 0x7f98641029e0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 Processor -- start 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 -- start start 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec55aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec55afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec55bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612ec4bfd80 con 0x5612ec1b9390 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5612ec4798b0 con 0x5612ec1986b0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9872998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612ec4bbd40 con 0x5612ec1904d0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9869883640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec55afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9869082640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec55bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9869883640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec55afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60952/0 (socket says 172.21.15.94:60952) 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f9869883640 1 -- 172.21.15.94:0/364330351 learned_addr learned my addr 172.21.15.94:0/364330351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.718+0000 7f986a084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec55aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f985affd640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 0x7f98500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 -- 172.21.15.94:0/364330351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 msgr2=0x5612ec55afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec55afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 -- 172.21.15.94:0/364330351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 msgr2=0x5612ec55aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec55aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 -- 172.21.15.94:0/364330351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ec44aa40 con 0x5612ec1986b0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f986a084640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 0x7f98500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f985affd640 1 -- 172.21.15.94:0/3726208091 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9860100150 con 0x7f98641029e0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f9869082640 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec55bc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9854005fd0 tx=0x7f985400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395d545f5728003f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f984ffff640 1 -- 172.21.15.94:0/364330351 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854003bd0 con 0x5612ec1986b0 2024-04-25T13:54:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f984ffff640 1 -- 172.21.15.94:0/364330351 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9854015070 con 0x5612ec1986b0 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.719+0000 7f984ffff640 1 -- 172.21.15.94:0/364330351 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854004cd0 con 0x5612ec1986b0 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 -- 172.21.15.94:0/364330351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 msgr2=0x5612ec55bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f986a084640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 0x7f98500b2000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5612ec55bab0 tx=0x7f985c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec55bc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9854005fd0 tx=0x7f985400bb20 comp rx=0 tx=0).stop 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 -- 172.21.15.94:0/364330351 shutdown_connections 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec55bc70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec55afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 --2- 172.21.15.94:0/364330351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec55aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 -- 172.21.15.94:0/364330351 >> 172.21.15.94:0/364330351 conn(0x5612ec1ad4e0 msgr2=0x5612ec556270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 -- 172.21.15.94:0/364330351 shutdown_connections 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.720+0000 7f9872998740 1 -- 172.21.15.94:0/364330351 wait complete. 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 -- start start 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 Processor -- start 2024-04-25T13:54:21.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec564900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612ec55c780 con 0x5612ec1b9390 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5612ec446d10 con 0x5612ec1986b0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612ec55c900 con 0x5612ec1904d0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9872998740 10 client.0 _ll_register_callbacks cb 0x5612ec473210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f986a084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9869082640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9869082640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47372/0 (socket says 172.21.15.94:47372) 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9869883640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec564900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f986a084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36730/0 (socket says 172.21.15.94:36730) 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.721+0000 7f9869082640 1 -- 172.21.15.94:0/3595553437 learned_addr learned my addr 172.21.15.94:0/3595553437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 -- 172.21.15.94:0/3595553437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 msgr2=0x5612ec564900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec564900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 -- 172.21.15.94:0/3595553437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 msgr2=0x5612ec56bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 -- 172.21.15.94:0/3595553437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ec29b7d0 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f986a084640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869883640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec564900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9869082640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98540043c0 tx=0x7f9854003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed4f65fa3a42218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f984d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854028060 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9854011640 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9854015070 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9854011840 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9872998740 10 client.29402 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612ec1d13f0 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.722+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9854015070 con 0x5612ec1986b0 2024-04-25T13:54:21.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9854022070 con 0x5612ec1986b0 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 10 client.29402 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 10 client.29402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 20 client.29402 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec177f60 0x5612ec571ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 4 client.29402 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612ec4cd990 con 0x5612ec177f60 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9869883640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec177f60 0x5612ec571ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f984d7fa640 10 client.29402 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.723+0000 7f9869883640 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec177f60 0x5612ec571ec0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63810575c9447e5 server_cookie=ab3ea22d8d1ae67f in_seq=0 out_seq=0 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.724+0000 7f984d7fa640 1 -- 172.21.15.94:0/3595553437 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612ec4cd990 con 0x5612ec177f60 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.724+0000 7f984d7fa640 10 client.29402 handle_command_reply: tid=0 2024-04-25T13:54:21.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.724+0000 7f9872998740 1 client.29402 shutdown 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 msgr2=0x5612ec564e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98540043c0 tx=0x7f9854003bd0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 shutdown_connections 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec177f60 0x5612ec571ec0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec56bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f984d7fa640 0 client.29402 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec564e40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 --2- 172.21.15.94:0/3595553437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec564900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 >> 172.21.15.94:0/3595553437 conn(0x5612ec1ad4e0 msgr2=0x5612ec5553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 shutdown_connections 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 1 -- 172.21.15.94:0/3595553437 wait complete. 2024-04-25T13:54:21.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.725+0000 7f9872998740 20 client.29402 trim_cache size 0 max 16384 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.743+0000 7f9872998740 1 Processor -- start 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 -- start start 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec559cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1b9390 0x5612ec577630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98641ab500 con 0x5612ec1904d0 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98641ad040 con 0x5612ec1986b0 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9872998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98641abbd0 con 0x5612ec1b9390 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36744/0 (socket says 172.21.15.94:36744) 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 -- 172.21.15.94:0/309578993 learned_addr learned my addr 172.21.15.94:0/309578993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869082640 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec559cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f986a084640 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1b9390 0x5612ec577630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 -- 172.21.15.94:0/309578993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1b9390 msgr2=0x5612ec577630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1b9390 0x5612ec577630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 -- 172.21.15.94:0/309578993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 msgr2=0x5612ec559cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec559cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 -- 172.21.15.94:0/309578993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ec1910f0 con 0x5612ec1904d0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f9869883640 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9860030040 tx=0x7f98600fda80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec0abcb7b9c331d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.744+0000 7f984f7fe640 1 -- 172.21.15.94:0/309578993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600c8360 con 0x5612ec1904d0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f984f7fe640 1 -- 172.21.15.94:0/309578993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860036040 con 0x5612ec1904d0 2024-04-25T13:54:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f984f7fe640 1 -- 172.21.15.94:0/309578993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600c7d00 con 0x5612ec1904d0 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 -- 172.21.15.94:0/309578993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 msgr2=0x5612ec559780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9860030040 tx=0x7f98600fda80 comp rx=0 tx=0).stop 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 -- 172.21.15.94:0/309578993 shutdown_connections 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1b9390 0x5612ec577630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec559cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 --2- 172.21.15.94:0/309578993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1904d0 0x5612ec559780 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 -- 172.21.15.94:0/309578993 >> 172.21.15.94:0/309578993 conn(0x5612ec1ad4e0 msgr2=0x5612ec559070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 -- 172.21.15.94:0/309578993 shutdown_connections 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.745+0000 7f9872998740 1 -- 172.21.15.94:0/309578993 wait complete. 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 -- start start 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 Processor -- start 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec5696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98641ac970 con 0x5612ec1b9390 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98641ac2a0 con 0x5612ec1986b0 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98641a94e0 con 0x5612ec1904d0 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9872998740 10 client.0 _ll_register_callbacks cb 0x7f985c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f986a084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9869883640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f986a084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36748/0 (socket says 172.21.15.94:36748) 2024-04-25T13:54:21.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9869883640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60980/0 (socket says 172.21.15.94:60980) 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f986a084640 1 -- 172.21.15.94:0/283640958 learned_addr learned my addr 172.21.15.94:0/283640958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.746+0000 7f9869082640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec5696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 -- 172.21.15.94:0/283640958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 msgr2=0x5612ec569170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 -- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 msgr2=0x5612ec5696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec5696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 -- 172.21.15.94:0/283640958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ec4804b0 con 0x5612ec1b9390 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f9869883640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f986a084640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f985c003db0 tx=0x7f985c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa3c0da5f354c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f984d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c030060 con 0x5612ec1b9390 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f985c02b440 con 0x5612ec1b9390 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9860036040 con 0x5612ec1b9390 2024-04-25T13:54:21.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f985c02b640 con 0x5612ec1b9390 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f9872998740 10 client.19968 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612ec5d9460 con 0x5612ec1b9390 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.747+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860036040 con 0x5612ec1b9390 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f985c01c870 con 0x5612ec1b9390 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 10 client.19968 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 10 client.19968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 20 client.19968 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec14d3a0 0x5612ec5dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 4 client.19968 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612ec4c7800 con 0x5612ec14d3a0 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9869883640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec14d3a0 0x5612ec5dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f984d7fa640 10 client.19968 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:21.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9869883640 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec14d3a0 0x5612ec5dbe80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e63701bcb1d6efa server_cookie=ffd7f25e75764362 in_seq=0 out_seq=0 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f984d7fa640 1 -- 172.21.15.94:0/283640958 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5612ec4c7800 con 0x5612ec14d3a0 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f984d7fa640 10 client.19968 handle_command_reply: tid=0 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.748+0000 7f9872998740 1 client.19968 shutdown 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 msgr2=0x5612ec5d81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f985c003db0 tx=0x7f985c004730 comp rx=0 tx=0).stop 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 shutdown_connections 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5612ec14d3a0 0x5612ec5dbe80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612ec1b9390 0x5612ec5d81d0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612ec1986b0 0x5612ec5696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 --2- 172.21.15.94:0/283640958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612ec1904d0 0x5612ec569170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f984d7fa640 0 client.19968 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 >> 172.21.15.94:0/283640958 conn(0x5612ec1ad4e0 msgr2=0x5612ec559aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 shutdown_connections 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 1 -- 172.21.15.94:0/283640958 wait complete. 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f9872998740 20 client.19968 trim_cache size 0 max 16384 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:54:21.756 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 msgr2=0x7f98500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 0x7f98500b2000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5612ec55bab0 tx=0x7f985c0046c0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 msgr2=0x7f986419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.749+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f986002ade0 tx=0x7f98600316e0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 shutdown_connections 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98500afaf0 0x7f98500b2000 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9864108920 0x7f98641a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9864103c20 0x7f986419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 --2- 172.21.15.94:0/3726208091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98641029e0 0x7f986419a560 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 >> 172.21.15.94:0/3726208091 conn(0x7f98640fdbd0 msgr2=0x7f9864104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 shutdown_connections 2024-04-25T13:54:21.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.750+0000 7f986bae7640 1 -- 172.21.15.94:0/3726208091 wait complete. 2024-04-25T13:54:21.760 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:54:21.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:54:21.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1566670 12.56 MB/sec execute 544 sec latency 13.072 ms 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- 172.21.15.94:0/2760219748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90103c00 msgr2=0x7f1c90104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 --2- 172.21.15.94:0/2760219748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90103c00 0x7f1c90104010 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400d910 tx=0x7f1c8402c370 comp rx=0 tx=0).stop 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- 172.21.15.94:0/2760219748 shutdown_connections 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 --2- 172.21.15.94:0/2760219748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90108900 0x7f1c9010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 --2- 172.21.15.94:0/2760219748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90103c00 0x7f1c90104010 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 --2- 172.21.15.94:0/2760219748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 0x7f1c901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- 172.21.15.94:0/2760219748 >> 172.21.15.94:0/2760219748 conn(0x7f1c900fdbd0 msgr2=0x7f1c90100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- 172.21.15.94:0/2760219748 shutdown_connections 2024-04-25T13:54:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- 172.21.15.94:0/2760219748 wait complete. 2024-04-25T13:54:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 Processor -- start 2024-04-25T13:54:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.874+0000 7f1c96357640 1 -- start start 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 0x7f1c90198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 0x7f1c90198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010d3e0 con 0x7f1c90108900 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010d260 con 0x7f1c90103c00 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c96357640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9010d560 con 0x7f1c901029c0 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c8f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 0x7f1c90198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36768/0 (socket says 172.21.15.94:36768) 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 -- 172.21.15.94:0/3286371534 learned_addr learned my addr 172.21.15.94:0/3286371534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c8ffff640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 0x7f1c90198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 -- 172.21.15.94:0/3286371534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 msgr2=0x7f1c90198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 0x7f1c90198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 -- 172.21.15.94:0/3286371534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 msgr2=0x7f1c90198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 0x7f1c90198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 -- 172.21.15.94:0/3286371534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9019fb60 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c8f7fe640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 0x7f1c90198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.875+0000 7f1c948f4640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009fd0 tx=0x7f1c8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.876+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88017070 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.876+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c88003e20 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.876+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88010460 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.876+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9019fe50 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.876+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901a0260 con 0x7f1c90108900 2024-04-25T13:54:21.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.877+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9006afa0 con 0x7f1c90108900 2024-04-25T13:54:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.879+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c88003970 con 0x7f1c90108900 2024-04-25T13:54:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.879+0000 7f1c8d7fa640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 0x7f1c780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.880+0000 7f1c8ffff640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 0x7f1c780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.880+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1c880de940 con 0x7f1c90108900 2024-04-25T13:54:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.880+0000 7f1c8ffff640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 0x7f1c780b2000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c901035c0 tx=0x7f1c80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:21.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:21.883+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c880a6e30 con 0x7f1c90108900 2024-04-25T13:54:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.117+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c9002dc10 con 0x7f1c90108900 2024-04-25T13:54:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.118+0000 7f1c8d7fa640 1 -- 172.21.15.94:0/3286371534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11941 (secure 0 0 0) 0x7f1c880ab130 con 0x7f1c90108900 2024-04-25T13:54:22.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:54:22.118 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:52:14.280433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:54:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-25T13:54:22.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 msgr2=0x7f1c780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 0x7f1c780b2000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c901035c0 tx=0x7f1c80005f90 comp rx=0 tx=0).stop 2024-04-25T13:54:22.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 msgr2=0x7f1c9019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009fd0 tx=0x7f1c8800b260 comp rx=0 tx=0).stop 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 shutdown_connections 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c780afaf0 0x7f1c780b2000 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c90108900 0x7f1c9019f4f0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c90103c00 0x7f1c90198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 --2- 172.21.15.94:0/3286371534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c901029c0 0x7f1c90198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 >> 172.21.15.94:0/3286371534 conn(0x7f1c900fdbd0 msgr2=0x7f1c901048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 shutdown_connections 2024-04-25T13:54:22.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.120+0000 7f1c96357640 1 -- 172.21.15.94:0/3286371534 wait complete. 2024-04-25T13:54:22.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 -- 172.21.15.94:0/1192165046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8102a50 msgr2=0x7f8ed8103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 --2- 172.21.15.94:0/1192165046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8102a50 0x7f8ed8103750 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009900 tx=0x7f8ec802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 -- 172.21.15.94:0/1192165046 shutdown_connections 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 --2- 172.21.15.94:0/1192165046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8108990 0x7f8ed810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 --2- 172.21.15.94:0/1192165046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 0x7f8ed81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 --2- 172.21.15.94:0/1192165046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8102a50 0x7f8ed8103750 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 -- 172.21.15.94:0/1192165046 >> 172.21.15.94:0/1192165046 conn(0x7f8ed80fdc40 msgr2=0x7f8ed81000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 -- 172.21.15.94:0/1192165046 shutdown_connections 2024-04-25T13:54:22.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.260+0000 7f8ede04c640 1 -- 172.21.15.94:0/1192165046 wait complete. 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 Processor -- start 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 -- start start 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8102a50 0x7f8ed8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 0x7f8ed8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e530 con 0x7f8ed8108990 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e3b0 con 0x7f8ed8103c90 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ede04c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e6b0 con 0x7f8ed8102a50 2024-04-25T13:54:22.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36788/0 (socket says 172.21.15.94:36788) 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ed7fff640 1 -- 172.21.15.94:0/3061604401 learned_addr learned my addr 172.21.15.94:0/3061604401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 0x7f8ed8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.261+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8102a50 0x7f8ed8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 -- 172.21.15.94:0/3061604401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8102a50 msgr2=0x7f8ed8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8102a50 0x7f8ed8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 -- 172.21.15.94:0/3061604401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 msgr2=0x7f8ed8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 0x7f8ed8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 -- 172.21.15.94:0/3061604401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81a0c70 con 0x7f8ed8108990 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000eb10 tx=0x7f8ed000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed4ff9640 1 -- 172.21.15.94:0/3061604401 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004b20 con 0x7f8ed8108990 2024-04-25T13:54:22.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed4ff9640 1 -- 172.21.15.94:0/3061604401 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed0004cc0 con 0x7f8ed8108990 2024-04-25T13:54:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a0f60 con 0x7f8ed8108990 2024-04-25T13:54:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ed4ff9640 1 -- 172.21.15.94:0/3061604401 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0010670 con 0x7f8ed8108990 2024-04-25T13:54:22.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.262+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81a13c0 con 0x7f8ed8108990 2024-04-25T13:54:22.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 Processor -- start 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed4ff9640 1 -- 172.21.15.94:0/3061604401 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8ed0005020 con 0x7f8ed8108990 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 -- start start 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c510810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c510d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b5c474d80 con 0x556b5c14d6b0 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b5c42ea20 con 0x556b5c16e390 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b5c470d40 con 0x556b5c1454d0 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c510d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36794/0 (socket says 172.21.15.94:36794) 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/1029370956 learned_addr learned my addr 172.21.15.94:0/1029370956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c510810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.265+0000 7f8ed77fe640 1 -- 172.21.15.94:0/1029370956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 msgr2=0x556b5c510810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c510810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed77fe640 1 -- 172.21.15.94:0/1029370956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 msgr2=0x556b5c511a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed77fe640 1 -- 172.21.15.94:0/1029370956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5c439c30 con 0x556b5c1454d0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed4ff9640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 0x7f8ec00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c510d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009230 tx=0x7f8ec8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7fe53959460537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 0x7f8ec00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ed4ff9640 1 -- 172.21.15.94:0/3061604401 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8ed00df200 con 0x7f8ed8108990 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ebdffb640 1 -- 172.21.15.94:0/1029370956 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8035030 con 0x556b5c1454d0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ebdffb640 1 -- 172.21.15.94:0/1029370956 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8040070 con 0x556b5c1454d0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.266+0000 7f8ebdffb640 1 -- 172.21.15.94:0/1029370956 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8031800 con 0x556b5c1454d0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/1029370956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 msgr2=0x556b5c510d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 0x7f8ec00b2000 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x556b5c511860 tx=0x7f8ecc004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c510d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009230 tx=0x7f8ec8004040 comp rx=0 tx=0).stop 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/1029370956 shutdown_connections 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c511a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c510d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/1029370956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c510810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/1029370956 >> 172.21.15.94:0/1029370956 conn(0x556b5c1624e0 msgr2=0x556b5c50c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/1029370956 shutdown_connections 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/1029370956 wait complete. 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.267+0000 7f8ee4fdb740 1 -- start start 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 Processor -- start 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c521ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b5c5124d0 con 0x556b5c14d6b0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b5c3fbdc0 con 0x556b5c16e390 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b5c512650 con 0x556b5c1454d0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ee4fdb740 10 client.0 _ll_register_callbacks cb 0x556b5c417600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36798/0 (socket says 172.21.15.94:36798) 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3387337886 learned_addr learned my addr 172.21.15.94:0/3387337886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c521ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 msgr2=0x556b5c51a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3387337886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 msgr2=0x556b5c521ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c521ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3387337886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5c1863f0 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.268+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00edf0 tx=0x7f8ecc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6ae54a563dd0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc02d060 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8ecc01a070 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8040070 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc020b30 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 10 client.19986 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b5c2507d0 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8040070 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8ecc027030 con 0x556b5c14d6b0 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 10 client.19986 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 10 client.19986 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:54:22.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 20 client.19986 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c12cf60 0x556b5c528410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 4 client.19986 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.269+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b5c482990 con 0x556b5c12cf60 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.270+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c12cf60 0x556b5c528410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.270+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c12cf60 0x556b5c528410 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a77ce211d1cb068c server_cookie=cc48e1d4d8a4a89b in_seq=0 out_seq=0 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.270+0000 7f8eb6ffd640 10 client.19986 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/3387337886 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b5c482990 con 0x556b5c12cf60 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8eb6ffd640 10 client.19986 handle_command_reply: tid=0 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 client.19986 shutdown 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 msgr2=0x556b5c51ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00edf0 tx=0x7f8ecc005f90 comp rx=0 tx=0).stop 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 shutdown_connections 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c12cf60 0x556b5c528410 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c521ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ab60 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8eb6ffd640 0 client.19986 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3387337886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 >> 172.21.15.94:0/3387337886 conn(0x556b5c1624e0 msgr2=0x556b5c50ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 shutdown_connections 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3387337886 wait complete. 2024-04-25T13:54:22.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.271+0000 7f8ee4fdb740 20 client.19986 trim_cache size 0 max 16384 2024-04-25T13:54:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 Processor -- start 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 -- start start 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c518cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c50b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81aa470 con 0x556b5c14d6b0 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81abfb0 con 0x556b5c16e390 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81aab40 con 0x556b5c1454d0 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36814/0 (socket says 172.21.15.94:36814) 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3055470211 learned_addr learned my addr 172.21.15.94:0/3055470211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c50b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.290+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c518cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3055470211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 msgr2=0x556b5c518cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c518cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3055470211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 msgr2=0x556b5c50b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c50b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/3055470211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5c518730 con 0x556b5c14d6b0 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00ebe0 tx=0x7f8ecc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b962b9d11f83e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ebd7fa640 1 -- 172.21.15.94:0/3055470211 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc011030 con 0x556b5c14d6b0 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ebd7fa640 1 -- 172.21.15.94:0/3055470211 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc01ed30 con 0x556b5c14d6b0 2024-04-25T13:54:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.291+0000 7f8ebd7fa640 1 -- 172.21.15.94:0/3055470211 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0328b0 con 0x556b5c14d6b0 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3055470211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 msgr2=0x556b5c5191f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00ebe0 tx=0x7f8ecc00ec10 comp rx=0 tx=0).stop 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3055470211 shutdown_connections 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c50b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c5191f0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/3055470211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c518cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3055470211 >> 172.21.15.94:0/3055470211 conn(0x556b5c1624e0 msgr2=0x556b5c510790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3055470211 shutdown_connections 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.292+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/3055470211 wait complete. 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 -- start start 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 Processor -- start 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c51f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81ab8e0 con 0x556b5c14d6b0 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81ab210 con 0x556b5c16e390 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed81a8450 con 0x556b5c1454d0 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ee4fdb740 10 client.0 _ll_register_callbacks cb 0x7f8ecc00ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.293+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36816/0 (socket says 172.21.15.94:36816) 2024-04-25T13:54:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed6ffd640 1 -- 172.21.15.94:0/4141041294 learned_addr learned my addr 172.21.15.94:0/4141041294 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed7fff640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c51f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 -- 172.21.15.94:0/4141041294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 msgr2=0x556b5c51f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c51f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 -- 172.21.15.94:0/4141041294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 msgr2=0x556b5c51ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 -- 172.21.15.94:0/4141041294 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5c51fed0 con 0x556b5c1454d0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ed77fe640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51e540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802a930 tx=0x7f8ec8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f760717bdc9d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8eb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8056020 con 0x556b5c1454d0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ecc01ed30 con 0x556b5c1454d0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8ec8003a00 con 0x556b5c1454d0 2024-04-25T13:54:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8061050 con 0x556b5c1454d0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.294+0000 7f8ee4fdb740 10 client.29203 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b5c521220 con 0x556b5c1454d0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc01ed30 con 0x556b5c1454d0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8ec804e070 con 0x556b5c1454d0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 10 client.29203 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 10 client.29203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 20 client.29203 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c1023a0 0x556b5c594960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 4 client.29203 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b5c47c800 con 0x556b5c1023a0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.295+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c1023a0 0x556b5c594960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8eb6ffd640 10 client.29203 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ed6ffd640 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c1023a0 0x556b5c594960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ec6d377e3b152f9 server_cookie=aeb7b73883dfa03d in_seq=0 out_seq=0 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8eb6ffd640 1 -- 172.21.15.94:0/4141041294 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b5c47c800 con 0x556b5c1023a0 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8eb6ffd640 10 client.29203 handle_command_reply: tid=0 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 client.29203 shutdown 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 msgr2=0x556b5c51e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51e540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802a930 tx=0x7f8ec8009260 comp rx=0 tx=0).stop 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 shutdown_connections 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x556b5c1023a0 0x556b5c594960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b5c16e390 0x556b5c51f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8eb6ffd640 0 client.29203 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b5c14d6b0 0x556b5c51ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 --2- 172.21.15.94:0/4141041294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b5c1454d0 0x556b5c51e540 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 >> 172.21.15.94:0/4141041294 conn(0x556b5c1624e0 msgr2=0x556b5c518cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 shutdown_connections 2024-04-25T13:54:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 1 -- 172.21.15.94:0/4141041294 wait complete. 2024-04-25T13:54:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.296+0000 7f8ee4fdb740 20 client.29203 trim_cache size 0 max 16384 2024-04-25T13:54:22.303 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 msgr2=0x7f8ec00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 0x7f8ec00b2000 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x556b5c511860 tx=0x7f8ecc004730 comp rx=0 tx=0).stop 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 msgr2=0x7f8ed81a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000eb10 tx=0x7f8ed000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 shutdown_connections 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8ec00afaf0 0x7f8ec00b2000 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ed8108990 0x7f8ed81a0600 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ed8103c90 0x7f8ed8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 --2- 172.21.15.94:0/3061604401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed8102a50 0x7f8ed8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 >> 172.21.15.94:0/3061604401 conn(0x7f8ed80fdc40 msgr2=0x7f8ed8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 shutdown_connections 2024-04-25T13:54:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.297+0000 7f8ede04c640 1 -- 172.21.15.94:0/3061604401 wait complete. 2024-04-25T13:54:22.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:54:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 -- 172.21.15.94:0/1935278766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 msgr2=0x7ff6b81054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/1935278766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81054a0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009860 tx=0x7ff6b402a340 comp rx=0 tx=0).stop 2024-04-25T13:54:22.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 -- 172.21.15.94:0/1935278766 shutdown_connections 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/1935278766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8102240 0x7ff6b810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/1935278766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101740 0x7ff6b8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/1935278766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81054a0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 -- 172.21.15.94:0/1935278766 >> 172.21.15.94:0/1935278766 conn(0x7ff6b80fd340 msgr2=0x7ff6b80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 -- 172.21.15.94:0/1935278766 shutdown_connections 2024-04-25T13:54:22.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.431+0000 7ff6c04f0640 1 -- 172.21.15.94:0/1935278766 wait complete. 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.432+0000 7ff6c04f0640 1 Processor -- start 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.432+0000 7ff6c04f0640 1 -- start start 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 0x7ff6b819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810f600 con 0x7ff6b8105090 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810f480 con 0x7ff6b8101740 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6c04f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810f780 con 0x7ff6b8102240 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bea8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 0x7ff6b819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:32828/0 (socket says 172.21.15.94:32828) 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 -- 172.21.15.94:0/2167725123 learned_addr learned my addr 172.21.15.94:0/2167725123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 0x7ff6b819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47490/0 (socket says 172.21.15.94:47490) 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 -- 172.21.15.94:0/2167725123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 msgr2=0x7ff6b819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 0x7ff6b819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 -- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 msgr2=0x7ff6b81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bda8b640 1 -- 172.21.15.94:0/2167725123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b81a1da0 con 0x7ff6b8102240 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.433+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9f0 tx=0x7ff6a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6af7fe640 1 -- 172.21.15.94:0/2167725123 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ccc0 con 0x7ff6b8102240 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6af7fe640 1 -- 172.21.15.94:0/2167725123 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a80054d0 con 0x7ff6b8102240 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b81a2090 con 0x7ff6b8102240 2024-04-25T13:54:22.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6af7fe640 1 -- 172.21.15.94:0/2167725123 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8010430 con 0x7ff6b8102240 2024-04-25T13:54:22.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.434+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b81a2590 con 0x7ff6b8102240 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.437+0000 7ff6c739c740 1 Processor -- start 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.437+0000 7ff6c739c740 1 -- start start 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297370dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x56297370ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562973671d80 con 0x56297336b390 2024-04-25T13:54:22.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56297362ba20 con 0x5629733424d0 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6c739c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56297366dd40 con 0x56297334a6b0 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6af7fe640 1 -- 172.21.15.94:0/2167725123 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff6a8005020 con 0x7ff6b8102240 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36840/0 (socket says 172.21.15.94:36840) 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 -- 172.21.15.94:0/1888490709 learned_addr learned my addr 172.21.15.94:0/1888490709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6be28c640 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297370dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x56297370ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 -- 172.21.15.94:0/1888490709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 msgr2=0x56297370ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x56297370ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 -- 172.21.15.94:0/1888490709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 msgr2=0x56297370dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297370dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.438+0000 7ff6bea8d640 1 -- 172.21.15.94:0/1888490709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562973636c30 con 0x56297336b390 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00117e0 tx=0x7ff6b0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2894b32dd2ea0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6acbfe640 1 -- 172.21.15.94:0/1888490709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011310 con 0x56297336b390 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6acbfe640 1 -- 172.21.15.94:0/1888490709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b0003e20 con 0x56297336b390 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6acbfe640 1 -- 172.21.15.94:0/1888490709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00044f0 con 0x56297336b390 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6af7fe640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 0x7ff6a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6be28c640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 0x7ff6a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.439+0000 7ff6af7fe640 1 -- 172.21.15.94:0/2167725123 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff6a80def40 con 0x7ff6b8102240 2024-04-25T13:54:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 -- 172.21.15.94:0/1888490709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 msgr2=0x56297370d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6be28c640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 0x7ff6a40b2000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009830 tx=0x7ff6b402c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00117e0 tx=0x7ff6b0011ca0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 -- 172.21.15.94:0/1888490709 shutdown_connections 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x56297370ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297370dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1888490709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297336b390 0x56297370d810 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 -- 172.21.15.94:0/1888490709 >> 172.21.15.94:0/1888490709 conn(0x56297335f4e0 msgr2=0x562973709020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 -- 172.21.15.94:0/1888490709 shutdown_connections 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.440+0000 7ff6c739c740 1 -- 172.21.15.94:0/1888490709 wait complete. 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 -- start start 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 Processor -- start 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629733424d0 0x56297371fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297336b390 0x562973727310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56297370f530 con 0x56297334a6b0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5629735f8dc0 con 0x56297336b390 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56297370f6b0 con 0x5629733424d0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6c739c740 10 client.0 _ll_register_callbacks cb 0x562973614600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6bda8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6bda8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36846/0 (socket says 172.21.15.94:36846) 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6bda8b640 1 -- 172.21.15.94:0/745250881 learned_addr learned my addr 172.21.15.94:0/745250881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6be28c640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629733424d0 0x56297371fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.441+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297336b390 0x562973727310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 -- 172.21.15.94:0/745250881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629733424d0 msgr2=0x56297371fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629733424d0 0x56297371fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 -- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297336b390 msgr2=0x562973727310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297336b390 0x562973727310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 -- 172.21.15.94:0/745250881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629733833f0 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80a3400 tx=0x7ff6a80a3300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711fe6de865e5419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80dce60 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff6a80afd50 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b0003e20 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6a8005310 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6c739c740 10 client.20004 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56297344d7d0 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.442+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b0003e20 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff6a80a1b10 con 0x56297334a6b0 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 10 client.20004 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 10 client.20004 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 20 client.20004 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562973329f60 0x56297372e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 4 client.20004 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56297367f990 con 0x562973329f60 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.443+0000 7ff6be28c640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562973329f60 0x56297372e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.444+0000 7ff6a1ffb640 10 client.20004 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.444+0000 7ff6be28c640 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562973329f60 0x56297372e0f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b84235f824c48d87 server_cookie=f10b8d4e16a98bef in_seq=0 out_seq=0 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.444+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/745250881 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56297367f990 con 0x562973329f60 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.444+0000 7ff6a1ffb640 10 client.20004 handle_command_reply: tid=0 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 client.20004 shutdown 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 msgr2=0x5629737203b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80a3400 tx=0x7ff6a80a3300 comp rx=0 tx=0).stop 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 shutdown_connections 2024-04-25T13:54:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562973329f60 0x56297372e0f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6a1ffb640 0 client.20004 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56297336b390 0x562973727310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297334a6b0 0x5629737203b0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 --2- 172.21.15.94:0/745250881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629733424d0 0x56297371fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 >> 172.21.15.94:0/745250881 conn(0x56297335f4e0 msgr2=0x562973708040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 shutdown_connections 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 1 -- 172.21.15.94:0/745250881 wait complete. 2024-04-25T13:54:22.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.445+0000 7ff6c739c740 20 client.20004 trim_cache size 0 max 16384 2024-04-25T13:54:22.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.462+0000 7ff6c739c740 1 Processor -- start 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.462+0000 7ff6c739c740 1 -- start start 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.462+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.462+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297371ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6c739c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81a93f0 con 0x562973329f60 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6c739c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81aaf30 con 0x5629733424d0 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6c739c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81a9ac0 con 0x56297334a6b0 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36860/0 (socket says 172.21.15.94:36860) 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 -- 172.21.15.94:0/1069627557 learned_addr learned my addr 172.21.15.94:0/1069627557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297371ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 -- 172.21.15.94:0/1069627557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 msgr2=0x5629737123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 -- 172.21.15.94:0/1069627557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 msgr2=0x56297371ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297371ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 -- 172.21.15.94:0/1069627557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56297370d3d0 con 0x562973329f60 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6be28c640 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009860 tx=0x7ff6b403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63dd824137e52a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6a3fff640 1 -- 172.21.15.94:0/1069627557 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4054070 con 0x562973329f60 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6a3fff640 1 -- 172.21.15.94:0/1069627557 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b4030810 con 0x562973329f60 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.463+0000 7ff6a3fff640 1 -- 172.21.15.94:0/1069627557 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4037040 con 0x562973329f60 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 -- 172.21.15.94:0/1069627557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 msgr2=0x56297371e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009860 tx=0x7ff6b403e000 comp rx=0 tx=0).stop 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 -- 172.21.15.94:0/1069627557 shutdown_connections 2024-04-25T13:54:22.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x56297371ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 --2- 172.21.15.94:0/1069627557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x56297371e540 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 -- 172.21.15.94:0/1069627557 >> 172.21.15.94:0/1069627557 conn(0x56297335f4e0 msgr2=0x56297370d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 -- 172.21.15.94:0/1069627557 shutdown_connections 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.464+0000 7ff6c739c740 1 -- 172.21.15.94:0/1069627557 wait complete. 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 -- start start 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 Processor -- start 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x562973726020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81aa860 con 0x562973329f60 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81aa190 con 0x5629733424d0 2024-04-25T13:54:22.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81a73d0 con 0x56297334a6b0 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6c739c740 10 client.0 _ll_register_callbacks cb 0x7ff6a800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6be28c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36868/0 (socket says 172.21.15.94:36868) 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6be28c640 1 -- 172.21.15.94:0/2844257537 learned_addr learned my addr 172.21.15.94:0/2844257537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.465+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x562973726020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6bea8d640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 -- 172.21.15.94:0/2844257537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 msgr2=0x5629737944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 -- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 msgr2=0x562973726020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x562973726020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 -- 172.21.15.94:0/2844257537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562973794ac0 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x562973726020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6be28c640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4004650 tx=0x7ff6b4005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3213d2479e5615d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b405d060 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff6b4047070 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b4030810 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6b4054030 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6c739c740 10 client.20010 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.466+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562973795d30 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b4030810 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff6b404f660 con 0x562973329f60 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 10 client.20010 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 10 client.20010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 20 client.20010 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56297336b390 0x562973798700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 4 client.20010 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562973679800 con 0x56297336b390 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.467+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56297336b390 0x562973798700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6a1ffb640 10 client.20010 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6bda8b640 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56297336b390 0x562973798700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1281757d988c8a1 server_cookie=2291f3caadf1becb in_seq=0 out_seq=0 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6a1ffb640 1 -- 172.21.15.94:0/2844257537 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562973679800 con 0x56297336b390 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6a1ffb640 10 client.20010 handle_command_reply: tid=0 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 client.20010 shutdown 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 msgr2=0x562973725ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4004650 tx=0x7ff6b4005fa0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.475 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 shutdown_connections 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56297336b390 0x562973798700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56297334a6b0 0x5629737944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629733424d0 0x562973726020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6a1ffb640 0 client.20010 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 --2- 172.21.15.94:0/2844257537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562973329f60 0x562973725ae0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 >> 172.21.15.94:0/2844257537 conn(0x56297335f4e0 msgr2=0x5629737123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 shutdown_connections 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 1 -- 172.21.15.94:0/2844257537 wait complete. 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.468+0000 7ff6c739c740 20 client.20010 trim_cache size 0 max 16384 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 msgr2=0x7ff6a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 0x7ff6a40b2000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009830 tx=0x7ff6b402c040 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 msgr2=0x7ff6b819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9f0 tx=0x7ff6a800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 shutdown_connections 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6a40afaf0 0x7ff6a40b2000 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6b8105090 0x7ff6b81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8102240 0x7ff6b819a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 --2- 172.21.15.94:0/2167725123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6b8101740 0x7ff6b819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 >> 172.21.15.94:0/2167725123 conn(0x7ff6b80fd340 msgr2=0x7ff6b8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 shutdown_connections 2024-04-25T13:54:22.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.469+0000 7ff6c04f0640 1 -- 172.21.15.94:0/2167725123 wait complete. 2024-04-25T13:54:22.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:22 smithi094 ceph-mon[29195]: pgmap v726: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:54:22.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:22 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:22 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3286371534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:54:22.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 -- 172.21.15.94:0/634168265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8103c20 msgr2=0x7f90e8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 --2- 172.21.15.94:0/634168265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8103c20 0x7f90e8104030 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009900 tx=0x7f90d802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 -- 172.21.15.94:0/634168265 shutdown_connections 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 --2- 172.21.15.94:0/634168265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8108920 0x7f90e810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 --2- 172.21.15.94:0/634168265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8103c20 0x7f90e8104030 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 --2- 172.21.15.94:0/634168265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 0x7f90e81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 -- 172.21.15.94:0/634168265 >> 172.21.15.94:0/634168265 conn(0x7f90e80fdbd0 msgr2=0x7f90e8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 -- 172.21.15.94:0/634168265 shutdown_connections 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.598+0000 7f90efa30640 1 -- 172.21.15.94:0/634168265 wait complete. 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 Processor -- start 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 -- start start 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 0x7f90e806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8103c20 0x7f90e819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810e500 con 0x7f90e8108920 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810e380 con 0x7f90e8103c20 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90efa30640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810e680 con 0x7f90e81029e0 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36890/0 (socket says 172.21.15.94:36890) 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 -- 172.21.15.94:0/1278368936 learned_addr learned my addr 172.21.15.94:0/1278368936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8103c20 0x7f90e819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 0x7f90e806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 -- 172.21.15.94:0/1278368936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 msgr2=0x7f90e806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 0x7f90e806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 -- 172.21.15.94:0/1278368936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8103c20 msgr2=0x7f90e819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8103c20 0x7f90e819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 -- 172.21.15.94:0/1278368936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e81a2080 con 0x7f90e8108920 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.599+0000 7f90edfcd640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f90e000eb10 tx=0x7f90e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.600+0000 7f90de7fc640 1 -- 172.21.15.94:0/1278368936 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0004b20 con 0x7f90e8108920 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.600+0000 7f90de7fc640 1 -- 172.21.15.94:0/1278368936 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e0004cc0 con 0x7f90e8108920 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.600+0000 7f90de7fc640 1 -- 172.21.15.94:0/1278368936 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0010670 con 0x7f90e8108920 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.600+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e81a2370 con 0x7f90e8108920 2024-04-25T13:54:22.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.600+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e81a2870 con 0x7f90e8108920 2024-04-25T13:54:22.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.602+0000 7f90de7fc640 1 -- 172.21.15.94:0/1278368936 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f90e0005020 con 0x7f90e8108920 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.602+0000 7f90f69b4740 1 Processor -- start 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.602+0000 7f90f69b4740 1 -- start start 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b806b0 0x55c683f48e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c683ea7d80 con 0x55c683ba1390 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c683e61a20 con 0x55c683b784d0 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90f69b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c683ea3d40 con 0x55c683b806b0 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90de7fc640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 0x7f90d40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36894/0 (socket says 172.21.15.94:36894) 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 -- 172.21.15.94:0/4127396222 learned_addr learned my addr 172.21.15.94:0/4127396222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90edfcd640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 0x7f90d40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b806b0 0x55c683f48e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90de7fc640 1 -- 172.21.15.94:0/1278368936 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f90e00df200 con 0x7f90e8108920 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 -- 172.21.15.94:0/4127396222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b806b0 msgr2=0x55c683f48e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b806b0 0x55c683f48e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 -- 172.21.15.94:0/4127396222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 msgr2=0x55c683f481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 -- 172.21.15.94:0/4127396222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c683e6cc30 con 0x55c683ba1390 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90edfcd640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 0x7f90d40b2080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f90e00dcfd0 tx=0x7f90e0023040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.603+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f90d8031500 tx=0x7f90d8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0e447733295080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90d37fe640 1 -- 172.21.15.94:0/4127396222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8035070 con 0x55c683ba1390 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90d37fe640 1 -- 172.21.15.94:0/4127396222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d8004040 con 0x55c683ba1390 2024-04-25T13:54:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90d37fe640 1 -- 172.21.15.94:0/4127396222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8003a90 con 0x55c683ba1390 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 -- 172.21.15.94:0/4127396222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 msgr2=0x55c683f47c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f90d8031500 tx=0x7f90d8009560 comp rx=0 tx=0).stop 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 -- 172.21.15.94:0/4127396222 shutdown_connections 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b806b0 0x55c683f48e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 --2- 172.21.15.94:0/4127396222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683ba1390 0x55c683f47c20 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.604+0000 7f90f69b4740 1 -- 172.21.15.94:0/4127396222 >> 172.21.15.94:0/4127396222 conn(0x55c683b954e0 msgr2=0x55c683f3f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- 172.21.15.94:0/4127396222 shutdown_connections 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- 172.21.15.94:0/4127396222 wait complete. 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- start start 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 Processor -- start 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683ba1390 0x55c683f58df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c683f498e0 con 0x55c683b784d0 2024-04-25T13:54:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c683e2edc0 con 0x55c683b806b0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c683f49a60 con 0x55c683ba1390 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90f69b4740 10 client.0 _ll_register_callbacks cb 0x55c683e4a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36896/0 (socket says 172.21.15.94:36896) 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.605+0000 7f90ecfcb640 1 -- 172.21.15.94:0/1089112334 learned_addr learned my addr 172.21.15.94:0/1089112334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683ba1390 0x55c683f58df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 -- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683ba1390 msgr2=0x55c683f58df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683ba1390 0x55c683f58df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 -- 172.21.15.94:0/1089112334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 msgr2=0x55c683f51e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 -- 172.21.15.94:0/1089112334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c683bb93f0 con 0x55c683b784d0 2024-04-25T13:54:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f90d8047040 tx=0x7f90d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a1bad8575e8f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8035070 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f90d8003810 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.606+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d8004040 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f90d8003a30 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 10 client.20022 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c683c837d0 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d8004040 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f90d805a070 con 0x55c683b784d0 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 10 client.20022 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 10 client.20022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:54:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 20 client.20022 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683b5ff60 0x55c683f5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 4 client.20022 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c683eb5990 con 0x55c683b5ff60 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.607+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683b5ff60 0x55c683f5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.608+0000 7f90d0ff9640 10 client.20022 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.608+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683b5ff60 0x55c683f5f240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7d69269f0f660ff server_cookie=5da6c29267c69c82 in_seq=0 out_seq=0 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90d0ff9640 1 -- 172.21.15.94:0/1089112334 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c683eb5990 con 0x55c683b5ff60 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90d0ff9640 10 client.20022 handle_command_reply: tid=0 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 client.20022 shutdown 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 msgr2=0x55c683f51950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f90d8047040 tx=0x7f90d8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 shutdown_connections 2024-04-25T13:54:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683b5ff60 0x55c683f5f240 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683ba1390 0x55c683f58df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90d0ff9640 0 client.20022 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 --2- 172.21.15.94:0/1089112334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b784d0 0x55c683f51950 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 >> 172.21.15.94:0/1089112334 conn(0x55c683b954e0 msgr2=0x55c683f3e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 shutdown_connections 2024-04-25T13:54:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 1 -- 172.21.15.94:0/1089112334 wait complete. 2024-04-25T13:54:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.609+0000 7f90f69b4740 20 client.20022 trim_cache size 0 max 16384 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 Processor -- start 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 -- start start 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b784d0 0x55c683f50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81a9190 con 0x55c683b5ff60 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81aacd0 con 0x55c683b806b0 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90f69b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81a9860 con 0x55c683b784d0 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36898/0 (socket says 172.21.15.94:36898) 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.623+0000 7f90ecfcb640 1 -- 172.21.15.94:0/3232011665 learned_addr learned my addr 172.21.15.94:0/3232011665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b784d0 0x55c683f50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 -- 172.21.15.94:0/3232011665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b784d0 msgr2=0x55c683f50600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b784d0 0x55c683f50600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 -- 172.21.15.94:0/3232011665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 msgr2=0x55c683f51520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 -- 172.21.15.94:0/3232011665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c683f39d30 con 0x55c683b5ff60 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001eb0 tx=0x7f90d8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8d8155fe182eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90d2ffd640 1 -- 172.21.15.94:0/3232011665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8035030 con 0x55c683b5ff60 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90d2ffd640 1 -- 172.21.15.94:0/3232011665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d8061040 con 0x55c683b5ff60 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.624+0000 7f90d2ffd640 1 -- 172.21.15.94:0/3232011665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8060070 con 0x55c683b5ff60 2024-04-25T13:54:22.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- 172.21.15.94:0/3232011665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 msgr2=0x55c683f500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001eb0 tx=0x7f90d8031400 comp rx=0 tx=0).stop 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- 172.21.15.94:0/3232011665 shutdown_connections 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b806b0 0x55c683f51520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b784d0 0x55c683f50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 --2- 172.21.15.94:0/3232011665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b5ff60 0x55c683f500c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- 172.21.15.94:0/3232011665 >> 172.21.15.94:0/3232011665 conn(0x55c683b954e0 msgr2=0x55c683f4c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- 172.21.15.94:0/3232011665 shutdown_connections 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- 172.21.15.94:0/3232011665 wait complete. 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 -- start start 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.625+0000 7f90f69b4740 1 Processor -- start 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b5ff60 0x55c683f55220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81aa600 con 0x55c683b806b0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81a9f30 con 0x55c683b784d0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81a7170 con 0x55c683b5ff60 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90f69b4740 10 client.0 _ll_register_callbacks cb 0x7f90e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36906/0 (socket says 172.21.15.94:36906) 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 -- 172.21.15.94:0/340960822 learned_addr learned my addr 172.21.15.94:0/340960822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b5ff60 0x55c683f55220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 -- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b5ff60 msgr2=0x55c683f55220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b5ff60 0x55c683f55220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 -- 172.21.15.94:0/340960822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 msgr2=0x55c683f55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f55760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ecfcb640 1 -- 172.21.15.94:0/340960822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c683f58720 con 0x55c683b806b0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.626+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90ecfcb640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f90d8036040 tx=0x7f90d8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c11bec331d9325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8060070 con 0x55c683b806b0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f90d8003dd0 con 0x55c683b806b0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d8061040 con 0x55c683b806b0 2024-04-25T13:54:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f90d8003840 con 0x55c683b806b0 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 10 client.20028 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c683fc68a0 con 0x55c683b806b0 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d8061040 con 0x55c683b806b0 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f90d805a040 con 0x55c683b806b0 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 10 client.20028 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 10 client.20028 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 20 client.20028 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683ba1390 0x55c683fc9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 4 client.20028 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.627+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c683eaf800 con 0x55c683ba1390 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683ba1390 0x55c683fc9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90d0ff9640 10 client.20028 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90ed7cc640 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683ba1390 0x55c683fc9120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f3c52ec65e06626 server_cookie=fc01aaa9b2d427c6 in_seq=0 out_seq=0 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90d0ff9640 1 -- 172.21.15.94:0/340960822 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c683eaf800 con 0x55c683ba1390 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90d0ff9640 10 client.20028 handle_command_reply: tid=0 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 client.20028 shutdown 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 msgr2=0x55c683fc53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f90d8036040 tx=0x7f90d8009520 comp rx=0 tx=0).stop 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 shutdown_connections 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c683ba1390 0x55c683fc9120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c683b806b0 0x55c683fc53b0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90d0ff9640 0 client.20028 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c683b784d0 0x55c683f55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.628+0000 7f90f69b4740 1 --2- 172.21.15.94:0/340960822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c683b5ff60 0x55c683f55220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 >> 172.21.15.94:0/340960822 conn(0x55c683b954e0 msgr2=0x55c683f500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 shutdown_connections 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90f69b4740 1 -- 172.21.15.94:0/340960822 wait complete. 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90f69b4740 20 client.20028 trim_cache size 0 max 16384 2024-04-25T13:54:22.637 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 msgr2=0x7f90d40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 0x7f90d40b2080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f90e00dcfd0 tx=0x7f90e0023040 comp rx=0 tx=0).stop 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 msgr2=0x7f90e81a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f90e000eb10 tx=0x7f90e000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 shutdown_connections 2024-04-25T13:54:22.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90d40afb70 0x7f90d40b2080 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90e8108920 0x7f90e81a1860 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e8103c20 0x7f90e819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 --2- 172.21.15.94:0/1278368936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e81029e0 0x7f90e806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 >> 172.21.15.94:0/1278368936 conn(0x7f90e80fdbd0 msgr2=0x7f90e8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 shutdown_connections 2024-04-25T13:54:22.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.629+0000 7f90efa30640 1 -- 172.21.15.94:0/1278368936 wait complete. 2024-04-25T13:54:22.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:54:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:22 smithi137 ceph-mon[32568]: pgmap v726: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:54:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:22 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:22 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3286371534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:22 smithi138 ceph-mon[32977]: pgmap v726: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 404 op/s 2024-04-25T13:54:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:22 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:22 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3286371534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:22.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 -- 172.21.15.94:0/2846931248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01045e0 msgr2=0x7f6ca0106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/2846931248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01045e0 0x7f6ca0106a00 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009900 tx=0x7f6c9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 -- 172.21.15.94:0/2846931248 shutdown_connections 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/2846931248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca01098a0 0x7f6ca010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/2846931248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca0106f40 0x7f6ca0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/2846931248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01045e0 0x7f6ca0106a00 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.754+0000 7f6ca7a42640 1 -- 172.21.15.94:0/2846931248 >> 172.21.15.94:0/2846931248 conn(0x7f6ca00fdc00 msgr2=0x7f6ca0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- 172.21.15.94:0/2846931248 shutdown_connections 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- 172.21.15.94:0/2846931248 wait complete. 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 Processor -- start 2024-04-25T13:54:22.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- start start 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca0106f40 0x7f6ca019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010f5d0 con 0x7f6ca01098a0 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010f450 con 0x7f6ca0106f40 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.755+0000 7f6ca7a42640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca010f750 con 0x7f6ca01045e0 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca0106f40 0x7f6ca019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:32946/0 (socket says 172.21.15.94:32946) 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36914/0 (socket says 172.21.15.94:36914) 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca57de640 1 -- 172.21.15.94:0/3008490161 learned_addr learned my addr 172.21.15.94:0/3008490161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 -- 172.21.15.94:0/3008490161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 msgr2=0x7f6ca019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 -- 172.21.15.94:0/3008490161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca0106f40 msgr2=0x7f6ca019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca0106f40 0x7f6ca019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 -- 172.21.15.94:0/3008490161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca01a1d50 con 0x7f6ca01098a0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca5fdf640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005fd0 tx=0x7f6c9800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6c967fc640 1 -- 172.21.15.94:0/3008490161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800cdf0 con 0x7f6ca01098a0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6c967fc640 1 -- 172.21.15.94:0/3008490161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c98005020 con 0x7f6ca01098a0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca01a1fe0 con 0x7f6ca01098a0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6c967fc640 1 -- 172.21.15.94:0/3008490161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98010690 con 0x7f6ca01098a0 2024-04-25T13:54:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.756+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a2450 con 0x7f6ca01098a0 2024-04-25T13:54:22.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.759+0000 7f6cae9cb740 1 Processor -- start 2024-04-25T13:54:22.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.759+0000 7f6c967fc640 1 -- 172.21.15.94:0/3008490161 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6c98005460 con 0x7f6ca01098a0 2024-04-25T13:54:22.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 -- start start 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf53fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf540a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558dbf4a3d80 con 0x558dbf19d390 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558dbf45da20 con 0x558dbf1744d0 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558dbf49fd40 con 0x558dbf17c6b0 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36922/0 (socket says 172.21.15.94:36922) 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf53fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca4fdd640 1 -- 172.21.15.94:0/3454641428 learned_addr learned my addr 172.21.15.94:0/3454641428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf540a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6c967fc640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 0x7f6c8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 -- 172.21.15.94:0/3454641428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 msgr2=0x558dbf540a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf540a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 -- 172.21.15.94:0/3454641428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 msgr2=0x558dbf53f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca57de640 1 -- 172.21.15.94:0/3454641428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbf468c30 con 0x558dbf1744d0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.760+0000 7f6ca5fdf640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 0x7f6c8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.761+0000 7f6c967fc640 1 -- 172.21.15.94:0/3008490161 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6c980df3b0 con 0x7f6ca01098a0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.761+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf53fd90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c031440 tx=0x7f6c9c0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe5e4d1dc8d03ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.761+0000 7f6c8b7fe640 1 -- 172.21.15.94:0/3454641428 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c035070 con 0x558dbf1744d0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.761+0000 7f6c8b7fe640 1 -- 172.21.15.94:0/3454641428 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c0318a0 con 0x558dbf1744d0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.761+0000 7f6c8b7fe640 1 -- 172.21.15.94:0/3454641428 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003df0 con 0x558dbf1744d0 2024-04-25T13:54:22.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3454641428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 msgr2=0x558dbf53fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6ca5fdf640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 0x7f6c8c0b2000 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c980a4a60 tx=0x7f6c9809d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf53fd90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c031440 tx=0x7f6c9c0095a0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3454641428 shutdown_connections 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf540a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf53fd90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3454641428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf53f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3454641428 >> 172.21.15.94:0/3454641428 conn(0x558dbf1914e0 msgr2=0x558dbf53b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3454641428 shutdown_connections 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3454641428 wait complete. 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 -- start start 2024-04-25T13:54:22.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.762+0000 7f6cae9cb740 1 Processor -- start 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf549560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf550a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558dbf5414d0 con 0x558dbf19d390 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558dbf42adc0 con 0x558dbf1744d0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558dbf541650 con 0x558dbf17c6b0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6cae9cb740 10 client.0 _ll_register_callbacks cb 0x558dbf446600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:32970/0 (socket says 172.21.15.94:32970) 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf549560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 -- 172.21.15.94:0/392054343 learned_addr learned my addr 172.21.15.94:0/392054343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf550a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 -- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 msgr2=0x558dbf549560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf549560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 -- 172.21.15.94:0/392054343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 msgr2=0x558dbf550a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf550a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 -- 172.21.15.94:0/392054343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dbf1b53f0 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6ca57de640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c004730 tx=0x7f6c9c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526eba5bfd604dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6c88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.763+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c035030 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c0318a0 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6c9c003b70 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c9c04a040 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 10 client.29251 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558dbf27f7d0 con 0x558dbf17c6b0 2024-04-25T13:54:22.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c0318a0 con 0x558dbf17c6b0 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6c9c04d070 con 0x558dbf17c6b0 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 10 client.29251 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 10 client.29251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 20 client.29251 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf15bf60 0x558dbf556e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 4 client.29251 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dbf4b1990 con 0x558dbf15bf60 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.764+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf15bf60 0x558dbf556e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.765+0000 7f6c88ff9640 10 client.29251 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.765+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf15bf60 0x558dbf556e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e8a45e4b5bcd496 server_cookie=5a5c5741df59625d in_seq=0 out_seq=0 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.765+0000 7f6c88ff9640 1 -- 172.21.15.94:0/392054343 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558dbf4b1990 con 0x558dbf15bf60 2024-04-25T13:54:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.765+0000 7f6c88ff9640 10 client.29251 handle_command_reply: tid=0 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.765+0000 7f6cae9cb740 1 client.29251 shutdown 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 msgr2=0x558dbf549aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c004730 tx=0x7f6c9c005d50 comp rx=0 tx=0).stop 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 shutdown_connections 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf15bf60 0x558dbf556e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf550a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf549aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6c88ff9640 0 client.29251 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/392054343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf549560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 >> 172.21.15.94:0/392054343 conn(0x558dbf1914e0 msgr2=0x558dbf53df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 shutdown_connections 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 1 -- 172.21.15.94:0/392054343 wait complete. 2024-04-25T13:54:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.766+0000 7f6cae9cb740 20 client.29251 trim_cache size 0 max 16384 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 Processor -- start 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 -- start start 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf550920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf19d390 0x558dbf55c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01ab4c0 con 0x558dbf1744d0 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01ad000 con 0x558dbf19d390 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.782+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01abb90 con 0x558dbf17c6b0 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36938/0 (socket says 172.21.15.94:36938) 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 -- 172.21.15.94:0/1346139687 learned_addr learned my addr 172.21.15.94:0/1346139687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf19d390 0x558dbf55c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca57de640 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf550920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 -- 172.21.15.94:0/1346139687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 msgr2=0x558dbf550920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf550920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 -- 172.21.15.94:0/1346139687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf19d390 msgr2=0x558dbf55c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf19d390 0x558dbf55c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 -- 172.21.15.94:0/1346139687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbf550e60 con 0x558dbf1744d0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.783+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6c90013430 tx=0x7f6c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227b4035897c96a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.784+0000 7f6c8affd640 1 -- 172.21.15.94:0/1346139687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90015310 con 0x558dbf1744d0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.784+0000 7f6c8affd640 1 -- 172.21.15.94:0/1346139687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c90026e50 con 0x558dbf1744d0 2024-04-25T13:54:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.784+0000 7f6c8affd640 1 -- 172.21.15.94:0/1346139687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90004a90 con 0x558dbf1744d0 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- 172.21.15.94:0/1346139687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 msgr2=0x558dbf53e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6c90013430 tx=0x7f6c900046c0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- 172.21.15.94:0/1346139687 shutdown_connections 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf19d390 0x558dbf55c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf550920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/1346139687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf1744d0 0x558dbf53e530 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- 172.21.15.94:0/1346139687 >> 172.21.15.94:0/1346139687 conn(0x558dbf1914e0 msgr2=0x558dbf53de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- 172.21.15.94:0/1346139687 shutdown_connections 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- 172.21.15.94:0/1346139687 wait complete. 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.785+0000 7f6cae9cb740 1 -- start start 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 Processor -- start 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf54f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf5bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01ac930 con 0x558dbf19d390 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01ac260 con 0x558dbf1744d0 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01a94a0 con 0x558dbf17c6b0 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6cae9cb740 10 client.0 _ll_register_callbacks cb 0x7f6c90009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6ca4fdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf5bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6ca57de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:32980/0 (socket says 172.21.15.94:32980) 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6ca57de640 1 -- 172.21.15.94:0/3491634307 learned_addr learned my addr 172.21.15.94:0/3491634307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.786+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf54f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 -- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 msgr2=0x558dbf54f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf54f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 -- 172.21.15.94:0/3491634307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 msgr2=0x558dbf5bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf5bd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 -- 172.21.15.94:0/3491634307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dbf5502c0 con 0x558dbf17c6b0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6ca57de640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c03c690 tx=0x7f6c9c009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6b0a70694c068d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6c88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c066060 con 0x558dbf17c6b0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c90026e50 con 0x558dbf17c6b0 2024-04-25T13:54:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.787+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6c9c03cc90 con 0x558dbf17c6b0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6c9c035040 con 0x558dbf17c6b0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 10 client.29257 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558dbf5be5e0 con 0x558dbf17c6b0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c90026e50 con 0x558dbf17c6b0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6c9c04d040 con 0x558dbf17c6b0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 10 client.29257 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 10 client.29257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 20 client.29257 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf1313a0 0x558dbf5c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 4 client.29257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dbf4ab800 con 0x558dbf1313a0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.788+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf1313a0 0x558dbf5c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6c88ff9640 10 client.29257 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6ca4fdd640 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf1313a0 0x558dbf5c0f40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=475e9684885e9f9b server_cookie=d96e13c2adf1eee3 in_seq=0 out_seq=0 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6c88ff9640 1 -- 172.21.15.94:0/3491634307 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dbf4ab800 con 0x558dbf1313a0 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6c88ff9640 10 client.29257 handle_command_reply: tid=0 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 client.29257 shutdown 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 msgr2=0x558dbf54fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c03c690 tx=0x7f6c9c009300 comp rx=0 tx=0).stop 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 shutdown_connections 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x558dbf1313a0 0x558dbf5c0f40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558dbf19d390 0x558dbf5bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dbf17c6b0 0x558dbf54fa00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 --2- 172.21.15.94:0/3491634307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558dbf1744d0 0x558dbf54f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6c88ff9640 0 client.29257 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 >> 172.21.15.94:0/3491634307 conn(0x558dbf1914e0 msgr2=0x558dbf53de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 shutdown_connections 2024-04-25T13:54:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.789+0000 7f6cae9cb740 1 -- 172.21.15.94:0/3491634307 wait complete. 2024-04-25T13:54:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6cae9cb740 20 client.29257 trim_cache size 0 max 16384 2024-04-25T13:54:22.795 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 msgr2=0x7f6c8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 0x7f6c8c0b2000 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c980a4a60 tx=0x7f6c9809d040 comp rx=0 tx=0).stop 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 msgr2=0x7f6ca01a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005fd0 tx=0x7f6c9800ef10 comp rx=0 tx=0).stop 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 shutdown_connections 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6c8c0afaf0 0x7f6c8c0b2000 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ca01098a0 0x7f6ca01a16e0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ca0106f40 0x7f6ca019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 --2- 172.21.15.94:0/3008490161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca01045e0 0x7f6ca019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 >> 172.21.15.94:0/3008490161 conn(0x7f6ca00fdc00 msgr2=0x7f6ca01065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 shutdown_connections 2024-04-25T13:54:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.790+0000 7f6ca7a42640 1 -- 172.21.15.94:0/3008490161 wait complete. 2024-04-25T13:54:22.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:54:22.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1568265 12.55 MB/sec execute 545 sec latency 20.435 ms 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 -- 172.21.15.94:0/4074788642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00104d20 msgr2=0x7ffa00105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 --2- 172.21.15.94:0/4074788642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00104d20 0x7ffa00105130 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009900 tx=0x7ff9f002a410 comp rx=0 tx=0).stop 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 -- 172.21.15.94:0/4074788642 shutdown_connections 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 --2- 172.21.15.94:0/4074788642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00109a20 0x7ffa0010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 --2- 172.21.15.94:0/4074788642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00104d20 0x7ffa00105130 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 --2- 172.21.15.94:0/4074788642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00103ae0 0x7ffa001047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 -- 172.21.15.94:0/4074788642 >> 172.21.15.94:0/4074788642 conn(0x7ffa000fecd0 msgr2=0x7ffa00101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 -- 172.21.15.94:0/4074788642 shutdown_connections 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.917+0000 7ffa077ce640 1 -- 172.21.15.94:0/4074788642 wait complete. 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 Processor -- start 2024-04-25T13:54:22.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 -- start start 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00104d20 0x7ffa0019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00109a20 0x7ffa001a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0010f600 con 0x7ffa00103ae0 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0010f480 con 0x7ffa00104d20 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa077ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0010f780 con 0x7ffa00109a20 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36958/0 (socket says 172.21.15.94:36958) 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa0556a640 1 -- 172.21.15.94:0/3110105952 learned_addr learned my addr 172.21.15.94:0/3110105952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa04d69640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00104d20 0x7ffa0019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.918+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00109a20 0x7ffa001a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 -- 172.21.15.94:0/3110105952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00109a20 msgr2=0x7ffa001a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00109a20 0x7ffa001a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 -- 172.21.15.94:0/3110105952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00104d20 msgr2=0x7ffa0019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00104d20 0x7ffa0019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 -- 172.21.15.94:0/3110105952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa001a1d70 con 0x7ffa00103ae0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9f0 tx=0x7ff9fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ff9f67fc640 1 -- 172.21.15.94:0/3110105952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00ccc0 con 0x7ffa00103ae0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa00074e30 con 0x7ffa00103ae0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ff9f67fc640 1 -- 172.21.15.94:0/3110105952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0035f0 con 0x7ffa00103ae0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ff9f67fc640 1 -- 172.21.15.94:0/3110105952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00ce60 con 0x7ffa00103ae0 2024-04-25T13:54:22.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.919+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa00075380 con 0x7ffa00103ae0 2024-04-25T13:54:22.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.921+0000 7ffa0e67f740 1 Processor -- start 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ff9f67fc640 1 -- 172.21.15.94:0/3110105952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff9fc003aa0 con 0x7ffa00103ae0 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 -- start start 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f91810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f92a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f743ef5d80 con 0x55f743bc64d0 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f743eafa20 con 0x55f743bef390 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f743ef1d40 con 0x55f743bce6b0 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa05d6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f91810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa05d6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f91810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47656/0 (socket says 172.21.15.94:47656) 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa05d6b640 1 -- 172.21.15.94:0/1440984391 learned_addr learned my addr 172.21.15.94:0/1440984391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f92a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ff9f67fc640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 0x7ff9ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 -- 172.21.15.94:0/1440984391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 msgr2=0x55f743f92a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f92a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 -- 172.21.15.94:0/1440984391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 msgr2=0x55f743f91810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f91810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa0556a640 1 -- 172.21.15.94:0/1440984391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f743ebac30 con 0x55f743bc64d0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.922+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 0x7ff9ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ff9f67fc640 1 -- 172.21.15.94:0/3110105952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff9fc0deff0 con 0x7ffa00103ae0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f91d90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0dca20 tx=0x7ff9fc0a3a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff1d76f3d2a2d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/1440984391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a3120 con 0x55f743bc64d0 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/1440984391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 0x7ff9ec0b2000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.923+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/1440984391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a6ba0 con 0x55f743bc64d0 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1440984391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 msgr2=0x55f743f91d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f91d90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0dca20 tx=0x7ff9fc0a3a80 comp rx=0 tx=0).stop 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1440984391 shutdown_connections 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f92a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f91d90 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1440984391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f91810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1440984391 >> 172.21.15.94:0/1440984391 conn(0x55f743be34e0 msgr2=0x55f743f8d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1440984391 shutdown_connections 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1440984391 wait complete. 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 -- start start 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.924+0000 7ffa0e67f740 1 Processor -- start 2024-04-25T13:54:22.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bce6b0 0x55f743f9ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bef390 0x55f743fa2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f743f934d0 con 0x55f743bc64d0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f743e7cdc0 con 0x55f743bce6b0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f743f93650 con 0x55f743bef390 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0e67f740 10 client.0 _ll_register_callbacks cb 0x55f743e98600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36966/0 (socket says 172.21.15.94:36966) 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 -- 172.21.15.94:0/1038341671 learned_addr learned my addr 172.21.15.94:0/1038341671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bce6b0 0x55f743f9ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bef390 0x55f743fa2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 -- 172.21.15.94:0/1038341671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bef390 msgr2=0x55f743fa2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bef390 0x55f743fa2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 -- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bce6b0 msgr2=0x55f743f9ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bce6b0 0x55f743f9ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 -- 172.21.15.94:0/1038341671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f743c073f0 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0dcdc0 tx=0x7ff9fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f05a7d1dc9d40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ff9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.925+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a6d80 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff9fc0dbb20 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0a7200 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 10 client.20058 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f743cd17d0 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff9fc0a36c0 con 0x55f743bc64d0 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 10 client.20058 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 10 client.20058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 20 client.20058 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743badf60 0x55f743fa8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 4 client.20058 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f743f03990 con 0x55f743badf60 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.926+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743badf60 0x55f743fa8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.927+0000 7ff9e8ff9640 10 client.20058 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.927+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743badf60 0x55f743fa8db0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea77c8a1128546a server_cookie=3bb4bcc73ed968b6 in_seq=0 out_seq=0 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/1038341671 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f743f03990 con 0x55f743badf60 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ff9e8ff9640 10 client.20058 handle_command_reply: tid=0 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 client.20058 shutdown 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 msgr2=0x55f743f9b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0dcdc0 tx=0x7ff9fc0057f0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 shutdown_connections 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743badf60 0x55f743fa8db0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bef390 0x55f743fa2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bce6b0 0x55f743f9ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ff9e8ff9640 0 client.20058 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1038341671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9b4c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.928+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 >> 172.21.15.94:0/1038341671 conn(0x55f743be34e0 msgr2=0x55f743f8ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.929+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 shutdown_connections 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.929+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1038341671 wait complete. 2024-04-25T13:54:22.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.929+0000 7ffa0e67f740 20 client.20058 trim_cache size 0 max 16384 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.942+0000 7ffa0e67f740 1 Processor -- start 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.942+0000 7ffa0e67f740 1 -- start start 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.942+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.942+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f9b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001a9340 con 0x55f743bc64d0 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001aae80 con 0x55f743bef390 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001a9a10 con 0x55f743bce6b0 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36972/0 (socket says 172.21.15.94:36972) 2024-04-25T13:54:22.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 -- 172.21.15.94:0/1235333389 learned_addr learned my addr 172.21.15.94:0/1235333389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f9b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 -- 172.21.15.94:0/1235333389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 msgr2=0x55f743f9b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 -- 172.21.15.94:0/1235333389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 msgr2=0x55f743f9b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f9b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 -- 172.21.15.94:0/1235333389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f743f96c80 con 0x55f743bc64d0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa04d69640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ffa0556a640 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a73c0 tx=0x7ff9fc0e2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190686be92d6c9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ff9eaffd640 1 -- 172.21.15.94:0/1235333389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a2da0 con 0x55f743bc64d0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ff9eaffd640 1 -- 172.21.15.94:0/1235333389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.943+0000 7ff9eaffd640 1 -- 172.21.15.94:0/1235333389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0e28a0 con 0x55f743bc64d0 2024-04-25T13:54:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1235333389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 msgr2=0x55f743f9a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a73c0 tx=0x7ff9fc0e2c00 comp rx=0 tx=0).stop 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1235333389 shutdown_connections 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f743f9b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/1235333389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9a3c0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1235333389 >> 172.21.15.94:0/1235333389 conn(0x55f743be34e0 msgr2=0x55f743f99cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1235333389 shutdown_connections 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.944+0000 7ffa0e67f740 1 -- 172.21.15.94:0/1235333389 wait complete. 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 -- start start 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 Processor -- start 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f74400efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001aa7b0 con 0x55f743bc64d0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001aa0e0 con 0x55f743bef390 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001a7320 con 0x55f743bce6b0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0e67f740 10 client.0 _ll_register_callbacks cb 0x7ff9fc090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36982/0 (socket says 172.21.15.94:36982) 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 -- 172.21.15.94:0/3745080849 learned_addr learned my addr 172.21.15.94:0/3745080849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa05d6b640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f74400efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa04d69640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 -- 172.21.15.94:0/3745080849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 msgr2=0x55f743f9f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 -- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 msgr2=0x55f74400efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:22.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f74400efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.945+0000 7ffa0556a640 1 -- 172.21.15.94:0/3745080849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f743f9fa40 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0556a640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0ab9a0 tx=0x7ff9fc0a3210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530e92a8f7443ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a6d80 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff9fc0a36c0 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0a6ba0 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 10 client.20064 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f7440105a0 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a6f40 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff9fc0a1bc0 con 0x55f743bc64d0 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 10 client.20064 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:22.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 20 client.20064 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743b833a0 0x55f744012a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 4 client.20064 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.946+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f743efd800 con 0x55f743b833a0 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ffa04d69640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743b833a0 0x55f744012a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ff9e8ff9640 10 client.20064 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ffa04d69640 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743b833a0 0x55f744012a10 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861312948407b17f server_cookie=1b7b76901f5c951d in_seq=0 out_seq=0 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ff9e8ff9640 1 -- 172.21.15.94:0/3745080849 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f743efd800 con 0x55f743b833a0 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ff9e8ff9640 10 client.20064 handle_command_reply: tid=0 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ffa0e67f740 1 client.20064 shutdown 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 msgr2=0x55f743f9ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.947+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0ab9a0 tx=0x7ff9fc0a3210 comp rx=0 tx=0).stop 2024-04-25T13:54:22.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 shutdown_connections 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f743b833a0 0x55f744012a10 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f743bef390 0x55f74400efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ff9e8ff9640 0 client.20064 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f743bce6b0 0x55f743f9f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 --2- 172.21.15.94:0/3745080849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f743bc64d0 0x55f743f9ee90 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 >> 172.21.15.94:0/3745080849 conn(0x55f743be34e0 msgr2=0x55f743f8fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 shutdown_connections 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 1 -- 172.21.15.94:0/3745080849 wait complete. 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa0e67f740 20 client.20064 trim_cache size 0 max 16384 2024-04-25T13:54:22.955 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:22.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 msgr2=0x7ff9ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 0x7ff9ec0b2000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8004a40 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 msgr2=0x7ffa0019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00e9f0 tx=0x7ff9fc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 shutdown_connections 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff9ec0afaf0 0x7ff9ec0b2000 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa00109a20 0x7ffa001a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00104d20 0x7ffa0019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 --2- 172.21.15.94:0/3110105952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa00103ae0 0x7ffa0019a230 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.948+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 >> 172.21.15.94:0/3110105952 conn(0x7ffa000fecd0 msgr2=0x7ffa00105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:22.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.949+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 shutdown_connections 2024-04-25T13:54:22.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:22.949+0000 7ffa077ce640 1 -- 172.21.15.94:0/3110105952 wait complete. 2024-04-25T13:54:23.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:23.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:23.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:23.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:23.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:23.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:54:23.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1570489 12.55 MB/sec execute 546 sec latency 111.154 ms 2024-04-25T13:54:23.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:54:23.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:54:23.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 -- 172.21.15.94:0/2171525215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 msgr2=0x7f64841036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2171525215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f64841036f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6480009900 tx=0x7f648002a410 comp rx=0 tx=0).stop 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 -- 172.21.15.94:0/2171525215 shutdown_connections 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2171525215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484108930 0x7f648410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2171525215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484103c30 0x7f6484104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2171525215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f64841036f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 -- 172.21.15.94:0/2171525215 >> 172.21.15.94:0/2171525215 conn(0x7f64840fdbc0 msgr2=0x7f6484100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 -- 172.21.15.94:0/2171525215 shutdown_connections 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.076+0000 7f648c6a8640 1 -- 172.21.15.94:0/2171525215 wait complete. 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 Processor -- start 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 -- start start 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484103c30 0x7f648419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484108930 0x7f64841a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f620 con 0x7f64841029f0 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f4a0 con 0x7f6484108930 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648c6a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f7a0 con 0x7f6484103c30 2024-04-25T13:54:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44790/0 (socket says 172.21.15.94:44790) 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 -- 172.21.15.94:0/2266864141 learned_addr learned my addr 172.21.15.94:0/2266864141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648ac45640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484108930 0x7f64841a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f6489c43640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484103c30 0x7f648419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 -- 172.21.15.94:0/2266864141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484103c30 msgr2=0x7f648419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484103c30 0x7f648419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 -- 172.21.15.94:0/2266864141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484108930 msgr2=0x7f64841a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484108930 0x7f64841a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.077+0000 7f648a444640 1 -- 172.21.15.94:0/2266864141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a1cf0 con 0x7f64841029f0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f648a444640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6480006010 tx=0x7f6480004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f647b7fe640 1 -- 172.21.15.94:0/2266864141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648003f030 con 0x7f64841029f0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841a1f80 con 0x7f64841029f0 2024-04-25T13:54:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f647b7fe640 1 -- 172.21.15.94:0/2266864141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6480003aa0 con 0x7f64841029f0 2024-04-25T13:54:24.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f647b7fe640 1 -- 172.21.15.94:0/2266864141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480035070 con 0x7f64841029f0 2024-04-25T13:54:24.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.078+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a23e0 con 0x7f64841029f0 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.080+0000 7f6493636740 1 Processor -- start 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 -- start start 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f647b7fe640 1 -- 172.21.15.94:0/2266864141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6480042030 con 0x7f64841029f0 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa7714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa771a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa778920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5650aa6d6b00 con 0x5650aa3d0410 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650aa672920 con 0x5650aa3af730 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6493636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5650aa6d2ac0 con 0x5650aa3a7550 2024-04-25T13:54:24.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f648ac45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa7714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f648ac45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa7714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44802/0 (socket says 172.21.15.94:44802) 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f648ac45640 1 -- 172.21.15.94:0/1797118731 learned_addr learned my addr 172.21.15.94:0/1797118731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa778920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f648a444640 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa771a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 -- 172.21.15.94:0/1797118731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 msgr2=0x5650aa771a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa771a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 -- 172.21.15.94:0/1797118731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 msgr2=0x5650aa7714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa7714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.081+0000 7f6489c43640 1 -- 172.21.15.94:0/1797118731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aa4b07e0 con 0x5650aa3af730 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f647b7fe640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f6489c43640 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa778920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b80 tx=0x7f6474010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe4aac0bb7d94f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f648ac45640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f647b7fe640 1 -- 172.21.15.94:0/2266864141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f64800ff350 con 0x7f64841029f0 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f6478bfe640 1 -- 172.21.15.94:0/1797118731 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474005020 con 0x5650aa3af730 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f6478bfe640 1 -- 172.21.15.94:0/1797118731 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64740047a0 con 0x5650aa3af730 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.082+0000 7f6478bfe640 1 -- 172.21.15.94:0/1797118731 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740126b0 con 0x5650aa3af730 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- 172.21.15.94:0/1797118731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 msgr2=0x5650aa778920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f648ac45640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 0x7f64700b2000 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5650aa772670 tx=0x7f647c007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa778920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b80 tx=0x7f6474010740 comp rx=0 tx=0).stop 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- 172.21.15.94:0/1797118731 shutdown_connections 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa778920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa771a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 --2- 172.21.15.94:0/1797118731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa7714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- 172.21.15.94:0/1797118731 >> 172.21.15.94:0/1797118731 conn(0x5650aa3c4560 msgr2=0x5650aa76ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- 172.21.15.94:0/1797118731 shutdown_connections 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- 172.21.15.94:0/1797118731 wait complete. 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 -- start start 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.083+0000 7f6493636740 1 Processor -- start 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa77b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 0x5650aa782760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5650aa779370 con 0x5650aa3a7550 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650aa6705d0 con 0x5650aa3af730 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5650aa7794f0 con 0x5650aa3d0410 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 10 client.0 _ll_register_callbacks cb 0x5650aa65e080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6489c43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa77b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44810/0 (socket says 172.21.15.94:44810) 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 -- 172.21.15.94:0/1523621686 learned_addr learned my addr 172.21.15.94:0/1523621686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648ac45640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 0x5650aa782760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 -- 172.21.15.94:0/1523621686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 msgr2=0x5650aa782760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 0x5650aa782760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 -- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 msgr2=0x5650aa77b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa77b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 -- 172.21.15.94:0/1523621686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aa510f70 con 0x5650aa3a7550 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648ac45640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 0x5650aa782760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6489c43640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa77b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f648a444640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f64800c7720 tx=0x7f6480009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c1bb12ecbd31a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f646dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480049070 con 0x5650aa3a7550 2024-04-25T13:54:24.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6480003c50 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.084+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64740047a0 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f64800c6f00 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 10 client.20076 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5650aa3a8170 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64740047a0 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f64800c3a20 con 0x5650aa3a7550 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 10 client.20076 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 10 client.20076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 20 client.20076 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa38f1f0 0x5650aa788c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 4 client.20076 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650aa6e4710 con 0x5650aa38f1f0 2024-04-25T13:54:24.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.085+0000 7f6489c43640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa38f1f0 0x5650aa788c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.086+0000 7f6489c43640 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa38f1f0 0x5650aa788c10 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbc096b113091fb server_cookie=35bbf57436f04b98 in_seq=0 out_seq=0 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.086+0000 7f646dffb640 10 client.20076 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f646dffb640 1 -- 172.21.15.94:0/1523621686 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650aa6e4710 con 0x5650aa38f1f0 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f646dffb640 10 client.20076 handle_command_reply: tid=0 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 client.20076 shutdown 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 msgr2=0x5650aa77b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f64800c7720 tx=0x7f6480009260 comp rx=0 tx=0).stop 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 shutdown_connections 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa38f1f0 0x5650aa788c10 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3d0410 0x5650aa782760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa77b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 --2- 172.21.15.94:0/1523621686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3a7550 0x5650aa77b2c0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f646dffb640 0 client.20076 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 >> 172.21.15.94:0/1523621686 conn(0x5650aa3c4560 msgr2=0x5650aa76bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 shutdown_connections 2024-04-25T13:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 1 -- 172.21.15.94:0/1523621686 wait complete. 2024-04-25T13:54:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.088+0000 7f6493636740 20 client.20076 trim_cache size 0 max 16384 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.107+0000 7f6493636740 1 Processor -- start 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.107+0000 7f6493636740 1 -- start start 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa7703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa7826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64841ab470 con 0x5650aa3d0410 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f64841acfb0 con 0x5650aa3af730 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6493636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64841abb40 con 0x5650aa3a7550 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44818/0 (socket says 172.21.15.94:44818) 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 -- 172.21.15.94:0/2316578341 learned_addr learned my addr 172.21.15.94:0/2316578341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f6489c43640 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa7826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648a444640 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa7703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 -- 172.21.15.94:0/2316578341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 msgr2=0x5650aa7703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa7703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 -- 172.21.15.94:0/2316578341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 msgr2=0x5650aa7826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa7826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 -- 172.21.15.94:0/2316578341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aa778ac0 con 0x5650aa3d0410 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.108+0000 7f648ac45640 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f647c00f890 tx=0x7f647c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0e3817b6d2487c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.109+0000 7f646ffff640 1 -- 172.21.15.94:0/2316578341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c00fe10 con 0x5650aa3d0410 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.109+0000 7f646ffff640 1 -- 172.21.15.94:0/2316578341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647c003970 con 0x5650aa3d0410 2024-04-25T13:54:24.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.109+0000 7f646ffff640 1 -- 172.21.15.94:0/2316578341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c004370 con 0x5650aa3d0410 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.109+0000 7f6493636740 1 -- 172.21.15.94:0/2316578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 msgr2=0x5650aa78e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.109+0000 7f6493636740 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f647c00f890 tx=0x7f647c015000 comp rx=0 tx=0).stop 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- 172.21.15.94:0/2316578341 shutdown_connections 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3d0410 0x5650aa78e410 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3af730 0x5650aa7826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- 172.21.15.94:0/2316578341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa7703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- 172.21.15.94:0/2316578341 >> 172.21.15.94:0/2316578341 conn(0x5650aa3c4560 msgr2=0x5650aa76fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- 172.21.15.94:0/2316578341 shutdown_connections 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- 172.21.15.94:0/2316578341 wait complete. 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- start start 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 Processor -- start 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa77ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3d0410 0x5650aa781400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64841ac8e0 con 0x5650aa3af730 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f64841ac210 con 0x5650aa3d0410 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64841a9450 con 0x5650aa3a7550 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6493636740 10 client.0 _ll_register_callbacks cb 0x7f64800c2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6489c43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6489c43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44828/0 (socket says 172.21.15.94:44828) 2024-04-25T13:54:24.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.110+0000 7f6489c43640 1 -- 172.21.15.94:0/922735098 learned_addr learned my addr 172.21.15.94:0/922735098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f648a444640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa77ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f648ac45640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3d0410 0x5650aa781400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 -- 172.21.15.94:0/922735098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 msgr2=0x5650aa77ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa77ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 -- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3d0410 msgr2=0x5650aa781400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3d0410 0x5650aa781400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 -- 172.21.15.94:0/922735098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aa7819c0 con 0x5650aa3af730 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6489c43640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x5650aa76dbc0 tx=0x7f64740096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6919e8f6a3ea027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f646dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647402e070 con 0x5650aa3af730 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647c003970 con 0x5650aa3af730 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f6474004220 con 0x5650aa3af730 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6474039050 con 0x5650aa3af730 2024-04-25T13:54:24.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6493636740 10 client.20082 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5650aa7f9840 con 0x5650aa3af730 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.111+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647c003970 con 0x5650aa3af730 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f647400f040 con 0x5650aa3af730 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 10 client.20082 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 10 client.20082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 20 client.20082 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa364630 0x5650aa7fbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 4 client.20082 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5650aa6de580 con 0x5650aa364630 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f648a444640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa364630 0x5650aa7fbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f646dffb640 10 client.20082 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.112+0000 7f648a444640 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa364630 0x5650aa7fbec0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfa0ac87dcf1b3a server_cookie=b407dd8990a62c1a in_seq=0 out_seq=0 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f646dffb640 1 -- 172.21.15.94:0/922735098 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5650aa6de580 con 0x5650aa364630 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f646dffb640 10 client.20082 handle_command_reply: tid=0 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 client.20082 shutdown 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 msgr2=0x5650aa7804d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x5650aa76dbc0 tx=0x7f64740096f0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 shutdown_connections 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5650aa364630 0x5650aa7fbec0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650aa3d0410 0x5650aa781400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5650aa3af730 0x5650aa7804d0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f646dffb640 0 client.20082 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 --2- 172.21.15.94:0/922735098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650aa3a7550 0x5650aa77ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 >> 172.21.15.94:0/922735098 conn(0x5650aa3c4560 msgr2=0x5650aa77b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 shutdown_connections 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 1 -- 172.21.15.94:0/922735098 wait complete. 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.115+0000 7f6493636740 20 client.20082 trim_cache size 0 max 16384 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b426e45f-0448-4a73-b72c-e6486e85c0b0", 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:54:24.120 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 msgr2=0x7f64700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 0x7f64700b2000 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5650aa772670 tx=0x7f647c007520 comp rx=0 tx=0).stop 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 msgr2=0x7f648419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6480006010 tx=0x7f6480004270 comp rx=0 tx=0).stop 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 shutdown_connections 2024-04-25T13:54:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484108930 0x7f64841a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484103c30 0x7f648419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 --2- 172.21.15.94:0/2266864141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64841029f0 0x7f648419a2d0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 >> 172.21.15.94:0/2266864141 conn(0x7f64840fdbc0 msgr2=0x7f6484104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 shutdown_connections 2024-04-25T13:54:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.116+0000 7f648c6a8640 1 -- 172.21.15.94:0/2266864141 wait complete. 2024-04-25T13:54:24.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:54:24.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:54:24.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 -- 172.21.15.94:0/1098720009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec103fc0 msgr2=0x7f45ec1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 --2- 172.21.15.94:0/1098720009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec103fc0 0x7f45ec1043d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009900 tx=0x7f45dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:54:24.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 -- 172.21.15.94:0/1098720009 shutdown_connections 2024-04-25T13:54:24.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 --2- 172.21.15.94:0/1098720009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec10b9a0 0x7f45ec10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 --2- 172.21.15.94:0/1098720009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec108e70 0x7f45ec10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 --2- 172.21.15.94:0/1098720009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec103fc0 0x7f45ec1043d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 -- 172.21.15.94:0/1098720009 >> 172.21.15.94:0/1098720009 conn(0x7f45ec0fd340 msgr2=0x7f45ec0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 -- 172.21.15.94:0/1098720009 shutdown_connections 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.261+0000 7f45f2964640 1 -- 172.21.15.94:0/1098720009 wait complete. 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.262+0000 7f45f2964640 1 Processor -- start 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.262+0000 7f45f2964640 1 -- start start 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.262+0000 7f45f2964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.262+0000 7f45f2964640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 0x7f45ec19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.262+0000 7f45f2964640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 0x7f45ec1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45f2964640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec111800 con 0x7f45ec10b9a0 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45f2964640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec111680 con 0x7f45ec103fc0 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45f2964640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec111980 con 0x7f45ec108e70 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60894/0 (socket says 172.21.15.94:60894) 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 -- 172.21.15.94:0/2390874196 learned_addr learned my addr 172.21.15.94:0/2390874196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45f0f01640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 0x7f45ec1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 0x7f45ec19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 -- 172.21.15.94:0/2390874196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 msgr2=0x7f45ec19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 0x7f45ec19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 -- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 msgr2=0x7f45ec1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 0x7f45ec1a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 -- 172.21.15.94:0/2390874196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec1a3f80 con 0x7f45ec103fc0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45f0f01640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 0x7f45ec1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 0x7f45ec19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.263+0000 7f45ebfff640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45dc006010 tx=0x7f45dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.264+0000 7f45e97fa640 1 -- 172.21.15.94:0/2390874196 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc03f030 con 0x7f45ec103fc0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.264+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec1a4210 con 0x7f45ec103fc0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.264+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec1a4750 con 0x7f45ec103fc0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.264+0000 7f45e97fa640 1 -- 172.21.15.94:0/2390874196 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc031910 con 0x7f45ec103fc0 2024-04-25T13:54:24.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.264+0000 7f45e97fa640 1 -- 172.21.15.94:0/2390874196 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc035070 con 0x7f45ec103fc0 2024-04-25T13:54:24.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.268+0000 7f45e97fa640 1 -- 172.21.15.94:0/2390874196 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f45dc042030 con 0x7f45ec103fc0 2024-04-25T13:54:24.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.268+0000 7f45f9813740 1 Processor -- start 2024-04-25T13:54:24.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.268+0000 7f45e97fa640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 0x7f45d40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.269+0000 7f45e97fa640 1 -- 172.21.15.94:0/2390874196 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f45dc0feec0 con 0x7f45ec103fc0 2024-04-25T13:54:24.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.269+0000 7f45f9813740 1 -- start start 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.269+0000 7f45f0f01640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 0x7f45d40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af2dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56197ae92d80 con 0x56197ab6b6b0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56197ae4c8b0 con 0x56197ab634d0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36430/0 (socket says 172.21.15.94:36430) 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 -- 172.21.15.94:0/1764316522 learned_addr learned my addr 172.21.15.94:0/1764316522 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56197ae8ed40 con 0x56197ab8c390 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f0f01640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af2dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45f0f01640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 0x7f45d40b2080 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005fd0 tx=0x7f45e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 -- 172.21.15.94:0/1764316522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 msgr2=0x56197af2dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af2dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 -- 172.21.15.94:0/1764316522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 msgr2=0x56197af2ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.270+0000 7f45ebfff640 1 -- 172.21.15.94:0/1764316522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197ae1da40 con 0x56197ab8c390 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.271+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af2dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.271+0000 7f45ebfff640 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0c7000 tx=0x7f45dc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc51a03f349ca5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.271+0000 7f45d27fc640 1 -- 172.21.15.94:0/1764316522 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc049040 con 0x56197ab8c390 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.272+0000 7f45d27fc640 1 -- 172.21.15.94:0/1764316522 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc0c7a00 con 0x56197ab8c390 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.272+0000 7f45d27fc640 1 -- 172.21.15.94:0/1764316522 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0c77c0 con 0x56197ab8c390 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 msgr2=0x56197af2da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0c7000 tx=0x7f45dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 shutdown_connections 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af2dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 --2- 172.21.15.94:0/1764316522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab8c390 0x56197af2da60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.273+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 >> 172.21.15.94:0/1764316522 conn(0x56197ab804e0 msgr2=0x56197af29270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.274+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 shutdown_connections 2024-04-25T13:54:24.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.274+0000 7f45f9813740 1 -- 172.21.15.94:0/1764316522 wait complete. 2024-04-25T13:54:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 -- start start 2024-04-25T13:54:24.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 Processor -- start 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197af37b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab8c390 0x56197af3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56197af2f540 con 0x56197ab8c390 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56197ae19d10 con 0x56197ab634d0 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56197af2f6c0 con 0x56197ab6b6b0 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.275+0000 7f45f9813740 10 client.0 _ll_register_callbacks cb 0x56197ae46210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60912/0 (socket says 172.21.15.94:60912) 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 -- 172.21.15.94:0/3325635677 learned_addr learned my addr 172.21.15.94:0/3325635677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45f0f01640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab8c390 0x56197af3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197af37b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 -- 172.21.15.94:0/3325635677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 msgr2=0x56197af37b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197af37b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 -- 172.21.15.94:0/3325635677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab8c390 msgr2=0x56197af3eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab8c390 0x56197af3eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 -- 172.21.15.94:0/3325635677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56197ac6e7d0 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45dc032fd0 tx=0x7f45dc0c77e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271d560f518538c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.276+0000 7f45cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0e5070 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f45dc0cfee0 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45dc0c7a00 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45dc049050 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45f9813740 10 client.29534 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56197aba43f0 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.277+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc0c7a00 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f45dc0cb120 con 0x56197ab634d0 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 10 client.29534 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 10 client.29534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:24.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 20 client.29534 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:24.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab4af60 0x56197af4d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 4 client.29534 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:24.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56197aea0990 con 0x56197ab4af60 2024-04-25T13:54:24.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab4af60 0x56197af4d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.278+0000 7f45cb7fe640 10 client.29534 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.279+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab4af60 0x56197af4d5a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2a5cdfe5c5d958 server_cookie=dbb49b706dc2f1df in_seq=0 out_seq=0 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.280+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3325635677 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56197aea0990 con 0x56197ab4af60 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.280+0000 7f45cb7fe640 10 client.29534 handle_command_reply: tid=0 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.280+0000 7f45f9813740 1 client.29534 shutdown 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.280+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 msgr2=0x56197af37620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.280+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45dc032fd0 tx=0x7f45dc0c77e0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 shutdown_connections 2024-04-25T13:54:24.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab4af60 0x56197af4d5a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45cb7fe640 0 client.29534 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab8c390 0x56197af3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197af37b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 --2- 172.21.15.94:0/3325635677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab634d0 0x56197af37620 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 >> 172.21.15.94:0/3325635677 conn(0x56197ab804e0 msgr2=0x56197af283f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 shutdown_connections 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 1 -- 172.21.15.94:0/3325635677 wait complete. 2024-04-25T13:54:24.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.281+0000 7f45f9813740 20 client.29534 trim_cache size 0 max 16384 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 Processor -- start 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 -- start start 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af36c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab634d0 0x56197af2c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800e690 con 0x56197ab6b6b0 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80101d0 con 0x56197ab4af60 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f9813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800ed60 con 0x56197ab634d0 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af36c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45ebfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af36c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60918/0 (socket says 172.21.15.94:60918) 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45eb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab634d0 0x56197af2c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45ebfff640 1 -- 172.21.15.94:0/1383855519 learned_addr learned my addr 172.21.15.94:0/1383855519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 -- 172.21.15.94:0/1383855519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab634d0 msgr2=0x56197af2c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab634d0 0x56197af2c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 -- 172.21.15.94:0/1383855519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 msgr2=0x56197af36c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af36c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.306+0000 7f45f0f01640 1 -- 172.21.15.94:0/1383855519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197ab640f0 con 0x56197ab6b6b0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.307+0000 7f45f0f01640 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ccc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f45e400fed0 tx=0x7f45e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb36a2eab32c7838 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.307+0000 7f45d1ffb640 1 -- 172.21.15.94:0/1383855519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4003e00 con 0x56197ab6b6b0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.307+0000 7f45d1ffb640 1 -- 172.21.15.94:0/1383855519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e401e3f0 con 0x56197ab6b6b0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.307+0000 7f45d1ffb640 1 -- 172.21.15.94:0/1383855519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4028620 con 0x56197ab6b6b0 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- 172.21.15.94:0/1383855519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 msgr2=0x56197af2ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ccc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f45e400fed0 tx=0x7f45e4014000 comp rx=0 tx=0).stop 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- 172.21.15.94:0/1383855519 shutdown_connections 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab6b6b0 0x56197af2ccc0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab634d0 0x56197af2c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 --2- 172.21.15.94:0/1383855519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af36c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- 172.21.15.94:0/1383855519 >> 172.21.15.94:0/1383855519 conn(0x56197ab804e0 msgr2=0x56197af2c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- 172.21.15.94:0/1383855519 shutdown_connections 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- 172.21.15.94:0/1383855519 wait complete. 2024-04-25T13:54:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.308+0000 7f45f9813740 1 -- start start 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 Processor -- start 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af3b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197afb3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800fb00 con 0x56197ab634d0 2024-04-25T13:54:24.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800f430 con 0x56197ab4af60 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800c670 con 0x56197ab6b6b0 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44878/0 (socket says 172.21.15.94:44878) 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45eb7fe640 1 -- 172.21.15.94:0/3852927724 learned_addr learned my addr 172.21.15.94:0/3852927724 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af3b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f9813740 10 client.0 _ll_register_callbacks cb 0x7f45dc0c71a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45f0f01640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197afb3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.309+0000 7f45eb7fe640 1 -- 172.21.15.94:0/3852927724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 msgr2=0x56197afb3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197afb3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45eb7fe640 1 -- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 msgr2=0x56197af3b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af3b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45eb7fe640 1 -- 172.21.15.94:0/3852927724 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56197ae534b0 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af3b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45eb7fe640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x56197af2a000 tx=0x7f45e0014ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855217d20bedad00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0004230 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f45e00043d0 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e401e3f0 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45e001e4a0 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45f9813740 10 client.20100 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56197afb4e90 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.310+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e401e3f0 con 0x56197ab634d0 2024-04-25T13:54:24.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f45e0003c00 con 0x56197ab634d0 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 10 client.20100 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 10 client.20100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 20 client.20100 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab8c390 0x56197afb7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 4 client.20100 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56197ae9a800 con 0x56197ab8c390 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab8c390 0x56197afb7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45cb7fe640 10 client.20100 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.311+0000 7f45ebfff640 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab8c390 0x56197afb7810 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3c7c2333031ee3 server_cookie=eb8bf21133566fa9 in_seq=0 out_seq=0 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45cb7fe640 1 -- 172.21.15.94:0/3852927724 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56197ae9a800 con 0x56197ab8c390 2024-04-25T13:54:24.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45cb7fe640 10 client.20100 handle_command_reply: tid=0 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 client.20100 shutdown 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 msgr2=0x56197af3b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x56197af2a000 tx=0x7f45e0014ab0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 shutdown_connections 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56197ab8c390 0x56197afb7810 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56197ab6b6b0 0x56197afb3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45cb7fe640 0 client.20100 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197ab634d0 0x56197af3b5f0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 --2- 172.21.15.94:0/3852927724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56197ab4af60 0x56197af3b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 >> 172.21.15.94:0/3852927724 conn(0x56197ab804e0 msgr2=0x56197af2caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 shutdown_connections 2024-04-25T13:54:24.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 1 -- 172.21.15.94:0/3852927724 wait complete. 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f9813740 20 client.20100 trim_cache size 0 max 16384 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "b426e45f-0448-4a73-b72c-e6486e85c0b0": { 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b426e45f-0448-4a73-b72c-e6486e85c0b0", 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:54:24.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:54:24.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:54:24.320 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 msgr2=0x7f45d40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 0x7f45d40b2080 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005fd0 tx=0x7f45e4009500 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 msgr2=0x7f45ec19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.312+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45dc006010 tx=0x7f45dc004270 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 shutdown_connections 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45d40afb70 0x7f45d40b2080 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45ec10b9a0 0x7f45ec1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ec108e70 0x7f45ec19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 --2- 172.21.15.94:0/2390874196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ec103fc0 0x7f45ec19c440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 >> 172.21.15.94:0/2390874196 conn(0x7f45ec0fd340 msgr2=0x7f45ec10af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 shutdown_connections 2024-04-25T13:54:24.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:24.313+0000 7f45f2964640 1 -- 172.21.15.94:0/2390874196 wait complete. 2024-04-25T13:54:24.324 INFO:tasks.cephfs.filesystem:scrub status for tag:b426e45f-0448-4a73-b72c-e6486e85c0b0 - {'path': '/', 'tag': 'b426e45f-0448-4a73-b72c-e6486e85c0b0', 'options': 'recursive,force'} 2024-04-25T13:54:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:24 smithi137 ceph-mon[32568]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-04-25T13:54:24.647 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:24 smithi138 ceph-mon[32977]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-04-25T13:54:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:24 smithi094 ceph-mon[29195]: pgmap v727: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-04-25T13:54:24.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1572627 12.54 MB/sec execute 547 sec latency 17.574 ms 2024-04-25T13:54:25.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:25 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:54:25.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:54:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:25 smithi137 ceph-mon[32568]: pgmap v728: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-04-25T13:54:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:25 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:54:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:54:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:25 smithi094 ceph-mon[29195]: pgmap v728: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-04-25T13:54:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:25 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:54:25.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:54:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:25.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:25 smithi138 ceph-mon[32977]: pgmap v728: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-04-25T13:54:25.820 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1574614 12.54 MB/sec execute 548 sec latency 16.285 ms 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 -- 172.21.15.94:0/2122951096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654108960 msgr2=0x7f965410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 --2- 172.21.15.94:0/2122951096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654108960 0x7f965410ad80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f964c009ca0 tx=0x7f964c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 -- 172.21.15.94:0/2122951096 shutdown_connections 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 --2- 172.21.15.94:0/2122951096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654108960 0x7f965410ad80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 --2- 172.21.15.94:0/2122951096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654103c60 0x7f9654104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 --2- 172.21.15.94:0/2122951096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 0x7f9654103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 -- 172.21.15.94:0/2122951096 >> 172.21.15.94:0/2122951096 conn(0x7f96540fdc10 msgr2=0x7f96541000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 -- 172.21.15.94:0/2122951096 shutdown_connections 2024-04-25T13:54:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.155+0000 7f965c242640 1 -- 172.21.15.94:0/2122951096 wait complete. 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 Processor -- start 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 -- start start 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 0x7f965406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108960 0x7f96541a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e550 con 0x7f9654103c60 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e3d0 con 0x7f9654108960 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965c242640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e6d0 con 0x7f9654102a20 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f96597dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f96597dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44890/0 (socket says 172.21.15.94:44890) 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f96597dd640 1 -- 172.21.15.94:0/254080665 learned_addr learned my addr 172.21.15.94:0/254080665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f965a7df640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108960 0x7f96541a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.156+0000 7f9659fde640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 0x7f965406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 -- 172.21.15.94:0/254080665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 msgr2=0x7f965406ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 0x7f965406ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 -- 172.21.15.94:0/254080665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108960 msgr2=0x7f96541a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108960 0x7f96541a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 -- 172.21.15.94:0/254080665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541a1cf0 con 0x7f9654103c60 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f96597dd640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f964affd640 1 -- 172.21.15.94:0/254080665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400cd80 con 0x7f9654103c60 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96541a1fe0 con 0x7f9654103c60 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f964affd640 1 -- 172.21.15.94:0/254080665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440108e0 con 0x7f9654103c60 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f964affd640 1 -- 172.21.15.94:0/254080665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644018980 con 0x7f9654103c60 2024-04-25T13:54:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.157+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a2450 con 0x7f9654103c60 2024-04-25T13:54:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.160+0000 7f96631cb740 1 Processor -- start 2024-04-25T13:54:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.161+0000 7f964affd640 1 -- 172.21.15.94:0/254080665 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9644018b20 con 0x7f9654103c60 2024-04-25T13:54:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.161+0000 7f96631cb740 1 -- start start 2024-04-25T13:54:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e519c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e519cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607e5102460 con 0x5607e4dd32f0 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5607e50bc590 con 0x5607e4dfc1b0 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96631cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607e50fe420 con 0x5607e4ddb4d0 2024-04-25T13:54:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f964affd640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 0x7f96400b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96597dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f965a7df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e519c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f965a7df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e519c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60970/0 (socket says 172.21.15.94:60970) 2024-04-25T13:54:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f96597dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36492/0 (socket says 172.21.15.94:36492) 2024-04-25T13:54:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f965a7df640 1 -- 172.21.15.94:0/1530253609 learned_addr learned my addr 172.21.15.94:0/1530253609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f964affd640 1 -- 172.21.15.94:0/254080665 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f96440defe0 con 0x7f9654103c60 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.162+0000 7f965a7df640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 0x7f96400b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 -- 172.21.15.94:0/1530253609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 msgr2=0x5607e519c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e519c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 -- 172.21.15.94:0/1530253609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 msgr2=0x5607e519cef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e519cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 -- 172.21.15.94:0/1530253609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e50c90d0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f96597dd640 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f96440dca90 tx=0x7f964409be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb14105b582f60a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f963ffff640 1 -- 172.21.15.94:0/1530253609 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440a6fe0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f963ffff640 1 -- 172.21.15.94:0/1530253609 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440a74d0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.163+0000 7f963ffff640 1 -- 172.21.15.94:0/1530253609 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964401dcb0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 -- 172.21.15.94:0/1530253609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 msgr2=0x5607e51a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f965a7df640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 0x7f96400b1ff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5607e519db40 tx=0x7f964c02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f96440dca90 tx=0x7f964409be10 comp rx=0 tx=0).stop 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 -- 172.21.15.94:0/1530253609 shutdown_connections 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a3df0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e519cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.164+0000 7f96631cb740 1 --2- 172.21.15.94:0/1530253609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e519c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.165+0000 7f96631cb740 1 -- 172.21.15.94:0/1530253609 >> 172.21.15.94:0/1530253609 conn(0x5607e4df0300 msgr2=0x5607e5198300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.165+0000 7f96631cb740 1 -- 172.21.15.94:0/1530253609 shutdown_connections 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.165+0000 7f96631cb740 1 -- 172.21.15.94:0/1530253609 wait complete. 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.165+0000 7f96631cb740 1 -- start start 2024-04-25T13:54:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.165+0000 7f96631cb740 1 Processor -- start 2024-04-25T13:54:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 0x5607e51ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dfc1b0 0x5607e51b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607e51a8c50 con 0x5607e4dfc1b0 2024-04-25T13:54:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5607e5089b60 con 0x5607e4ddb4d0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607e51a8dd0 con 0x5607e4dd32f0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96631cb740 10 client.0 _ll_register_callbacks cb 0x5607e50ae2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f9659fde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f9659fde640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36494/0 (socket says 172.21.15.94:36494) 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f9659fde640 1 -- 172.21.15.94:0/3725724067 learned_addr learned my addr 172.21.15.94:0/3725724067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f965a7df640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dfc1b0 0x5607e51b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.166+0000 7f96597dd640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 0x5607e51ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 -- 172.21.15.94:0/3725724067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 msgr2=0x5607e51aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 -- 172.21.15.94:0/3725724067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dfc1b0 msgr2=0x5607e51b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dfc1b0 0x5607e51b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 -- 172.21.15.94:0/3725724067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e4e14210 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.167+0000 7f96597dd640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 0x5607e51ab080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96440a7180 tx=0x7f9644009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32df3b736ff4015e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f963d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440dc020 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f96440d8dd0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96440a74d0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96440b0280 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f96631cb740 10 client.29552 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.168+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607e4ee0620 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.169+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440a74d0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.169+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f96440a2e40 con 0x5607e4ddb4d0 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 10 client.29552 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 10 client.29552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 20 client.29552 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4dbad80 0x5607e51a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 4 client.29552 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e5110070 con 0x5607e4dbad80 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.170+0000 7f9659fde640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4dbad80 0x5607e51a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.171+0000 7f9659fde640 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4dbad80 0x5607e51a7c10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904681a848bd13be server_cookie=6057c8c94520902b in_seq=0 out_seq=0 2024-04-25T13:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.171+0000 7f963d7fa640 10 client.29552 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.172+0000 7f963d7fa640 1 -- 172.21.15.94:0/3725724067 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607e5110070 con 0x5607e4dbad80 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.172+0000 7f963d7fa640 10 client.29552 handle_command_reply: tid=0 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.172+0000 7f96631cb740 1 client.29552 shutdown 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.172+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 msgr2=0x5607e51ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.172+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 0x5607e51ab080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96440a7180 tx=0x7f9644009310 comp rx=0 tx=0).stop 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 shutdown_connections 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4dbad80 0x5607e51a7c10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dfc1b0 0x5607e51b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4ddb4d0 0x5607e51ab080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f963d7fa640 0 client.29552 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 --2- 172.21.15.94:0/3725724067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 >> 172.21.15.94:0/3725724067 conn(0x5607e4df0300 msgr2=0x5607e5197250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 shutdown_connections 2024-04-25T13:54:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 1 -- 172.21.15.94:0/3725724067 wait complete. 2024-04-25T13:54:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.173+0000 7f96631cb740 20 client.29552 trim_cache size 0 max 16384 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 Processor -- start 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 -- start start 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5197c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a93a0 con 0x5607e4dd32f0 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96541aaee0 con 0x5607e4dfc1b0 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f96631cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a9a70 con 0x5607e4ddb4d0 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.190+0000 7f9659fde640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44908/0 (socket says 172.21.15.94:44908) 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 -- 172.21.15.94:0/1528912900 learned_addr learned my addr 172.21.15.94:0/1528912900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f965a7df640 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f96597dd640 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 -- 172.21.15.94:0/1528912900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 msgr2=0x5607e51a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 -- 172.21.15.94:0/1528912900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 msgr2=0x5607e5197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 -- 172.21.15.94:0/1528912900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e51a3f60 con 0x5607e4dd32f0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f9659fde640 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f965000bb90 tx=0x7f9650005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9513d2943cb07d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f963f7fe640 1 -- 172.21.15.94:0/1528912900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650004e60 con 0x5607e4dd32f0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f963f7fe640 1 -- 172.21.15.94:0/1528912900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96500118f0 con 0x5607e4dd32f0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.191+0000 7f963f7fe640 1 -- 172.21.15.94:0/1528912900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965001c800 con 0x5607e4dd32f0 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- 172.21.15.94:0/1528912900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 msgr2=0x5607e51a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f965000bb90 tx=0x7f9650005410 comp rx=0 tx=0).stop 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- 172.21.15.94:0/1528912900 shutdown_connections 2024-04-25T13:54:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4ddb4d0 0x5607e51a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 --2- 172.21.15.94:0/1528912900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4dd32f0 0x5607e51a0ba0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- 172.21.15.94:0/1528912900 >> 172.21.15.94:0/1528912900 conn(0x5607e4df0300 msgr2=0x5607e519c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- 172.21.15.94:0/1528912900 shutdown_connections 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- 172.21.15.94:0/1528912900 wait complete. 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.192+0000 7f96631cb740 1 -- start start 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 Processor -- start 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5223520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96541aa810 con 0x5607e4ddb4d0 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96541aa140 con 0x5607e4dfc1b0 2024-04-25T13:54:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a7380 con 0x5607e4dd32f0 2024-04-25T13:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96631cb740 10 client.0 _ll_register_callbacks cb 0x7f96440a72a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44918/0 (socket says 172.21.15.94:44918) 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 -- 172.21.15.94:0/3986995380 learned_addr learned my addr 172.21.15.94:0/3986995380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f965a7df640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5223520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f9659fde640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 -- 172.21.15.94:0/3986995380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 msgr2=0x5607e51ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 -- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 msgr2=0x5607e5223520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5223520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 -- 172.21.15.94:0/3986995380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e51b1720 con 0x5607e4ddb4d0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f96597dd640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f96440a71d0 tx=0x7f96440a7060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84db0f7e804289e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.193+0000 7f963d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440dc070 con 0x5607e4ddb4d0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f96440a6430 con 0x5607e4ddb4d0 2024-04-25T13:54:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96500118f0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96440a74d0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 10 client.20115 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607e52248a0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96500118f0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f96440a67c0 con 0x5607e4ddb4d0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 10 client.20115 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 10 client.20115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 20 client.20115 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4d901c0 0x5607e5227100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 4 client.20115 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607e5109ee0 con 0x5607e4d901c0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.194+0000 7f9659fde640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4d901c0 0x5607e5227100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.195+0000 7f963d7fa640 10 client.20115 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.195+0000 7f9659fde640 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4d901c0 0x5607e5227100 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b9853394cd4a2e server_cookie=dd64fe36ad4b147c in_seq=0 out_seq=0 2024-04-25T13:54:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.429+0000 7f963d7fa640 1 -- 172.21.15.94:0/3986995380 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607e5109ee0 con 0x5607e4d901c0 2024-04-25T13:54:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.429+0000 7f963d7fa640 10 client.20115 handle_command_reply: tid=0 2024-04-25T13:54:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.429+0000 7f96631cb740 1 client.20115 shutdown 2024-04-25T13:54:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.429+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 msgr2=0x5607e51ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.429+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f96440a71d0 tx=0x7f96440a7060 comp rx=0 tx=0).stop 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 shutdown_connections 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5607e4d901c0 0x5607e5227100 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5607e4dfc1b0 0x5607e5223520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607e4ddb4d0 0x5607e51ae760 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f963d7fa640 0 client.20115 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 --2- 172.21.15.94:0/3986995380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e4dd32f0 0x5607e51ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 >> 172.21.15.94:0/3986995380 conn(0x5607e4df0300 msgr2=0x5607e51aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 shutdown_connections 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 1 -- 172.21.15.94:0/3986995380 wait complete. 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f96631cb740 20 client.20115 trim_cache size 0 max 16384 2024-04-25T13:54:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 msgr2=0x7f96400b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 0x7f96400b1ff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5607e519db40 tx=0x7f964c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 msgr2=0x7f965419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 shutdown_connections 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96400afae0 0x7f96400b1ff0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108960 0x7f96541a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103c60 0x7f965419ef80 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 --2- 172.21.15.94:0/254080665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654102a20 0x7f965406ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 >> 172.21.15.94:0/254080665 conn(0x7f96540fdc10 msgr2=0x7f9654104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 shutdown_connections 2024-04-25T13:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:26.430+0000 7f965c242640 1 -- 172.21.15.94:0/254080665 wait complete. 2024-04-25T13:54:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:54:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:54:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:54:26.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1577215 12.55 MB/sec execute 549 sec latency 119.197 ms 2024-04-25T13:54:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:27 smithi094 ceph-mon[29195]: pgmap v729: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 285 op/s 2024-04-25T13:54:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:27 smithi138 ceph-mon[32977]: pgmap v729: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 285 op/s 2024-04-25T13:54:27.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1578877 12.54 MB/sec execute 550 sec latency 18.442 ms 2024-04-25T13:54:27.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:27 smithi137 ceph-mon[32568]: pgmap v729: 97 pgs: 97 active+clean; 51 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 285 op/s 2024-04-25T13:54:28.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1581172 12.54 MB/sec execute 551 sec latency 49.661 ms 2024-04-25T13:54:29.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1583331 12.53 MB/sec execute 552 sec latency 23.043 ms 2024-04-25T13:54:29.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:29 smithi137 ceph-mon[32568]: pgmap v730: 97 pgs: 97 active+clean; 50 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-04-25T13:54:29.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:29 smithi094 ceph-mon[29195]: pgmap v730: 97 pgs: 97 active+clean; 50 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-04-25T13:54:29.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:29 smithi138 ceph-mon[32977]: pgmap v730: 97 pgs: 97 active+clean; 50 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-04-25T13:54:30.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1585120 12.53 MB/sec execute 553 sec latency 22.234 ms 2024-04-25T13:54:31.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1587948 12.54 MB/sec execute 554 sec latency 15.913 ms 2024-04-25T13:54:31.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:31 smithi137 ceph-mon[32568]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-04-25T13:54:31.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:31 smithi094 ceph-mon[29195]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-04-25T13:54:31.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:31 smithi138 ceph-mon[32977]: pgmap v731: 97 pgs: 97 active+clean; 46 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-04-25T13:54:32.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1589547 12.53 MB/sec execute 555 sec latency 13.924 ms 2024-04-25T13:54:33.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1591722 12.53 MB/sec execute 556 sec latency 55.803 ms 2024-04-25T13:54:33.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:33 smithi137 ceph-mon[32568]: pgmap v732: 97 pgs: 97 active+clean; 42 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:33.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:33 smithi094 ceph-mon[29195]: pgmap v732: 97 pgs: 97 active+clean; 42 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:33.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:33 smithi138 ceph-mon[32977]: pgmap v732: 97 pgs: 97 active+clean; 42 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:34.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1593714 12.52 MB/sec execute 557 sec latency 28.460 ms 2024-04-25T13:54:35.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1595714 12.52 MB/sec execute 558 sec latency 41.323 ms 2024-04-25T13:54:35.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:35 smithi137 ceph-mon[32568]: pgmap v733: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:54:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:35 smithi094 ceph-mon[29195]: pgmap v733: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:54:35.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:35 smithi138 ceph-mon[32977]: pgmap v733: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:54:36.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1598754 12.53 MB/sec execute 559 sec latency 14.863 ms 2024-04-25T13:54:37.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1600969 12.52 MB/sec execute 560 sec latency 18.365 ms 2024-04-25T13:54:37.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:37 smithi137 ceph-mon[32568]: pgmap v734: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:37 smithi094 ceph-mon[29195]: pgmap v734: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:37.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:37 smithi138 ceph-mon[32977]: pgmap v734: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-04-25T13:54:38.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1602878 12.52 MB/sec execute 561 sec latency 91.145 ms 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- 172.21.15.94:0/283248434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a901050e0 msgr2=0x7f8a901054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 --2- 172.21.15.94:0/283248434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a901050e0 0x7f8a901054f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009ca0 tx=0x7f8a8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- 172.21.15.94:0/283248434 shutdown_connections 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 --2- 172.21.15.94:0/283248434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 0x7f8a9010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 --2- 172.21.15.94:0/283248434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a90101790 0x7f8a90101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 --2- 172.21.15.94:0/283248434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a901050e0 0x7f8a901054f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- 172.21.15.94:0/283248434 >> 172.21.15.94:0/283248434 conn(0x7f8a900fd390 msgr2=0x7f8a900ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- 172.21.15.94:0/283248434 shutdown_connections 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- 172.21.15.94:0/283248434 wait complete. 2024-04-25T13:54:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 Processor -- start 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.531+0000 7f8a97b46640 1 -- start start 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 0x7f8a9019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 0x7f8a901a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9010f630 con 0x7f8a90101790 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9010f4b0 con 0x7f8a90102290 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9010f7b0 con 0x7f8a901050e0 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34382/0 (socket says 172.21.15.94:34382) 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 -- 172.21.15.94:0/1503617957 learned_addr learned my addr 172.21.15.94:0/1503617957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a950e1640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 0x7f8a9019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a960e3640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 0x7f8a901a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 -- 172.21.15.94:0/1503617957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 msgr2=0x7f8a901a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 0x7f8a901a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 -- 172.21.15.94:0/1503617957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 msgr2=0x7f8a9019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 0x7f8a9019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 -- 172.21.15.94:0/1503617957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a901a1dd0 con 0x7f8a90101790 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a960e3640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 0x7f8a901a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c02ade0 tx=0x7f8a8c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a86ffd640 1 -- 172.21.15.94:0/1503617957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c003b70 con 0x7f8a90101790 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a901a2060 con 0x7f8a90101790 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a86ffd640 1 -- 172.21.15.94:0/1503617957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8c003d10 con 0x7f8a90101790 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a86ffd640 1 -- 172.21.15.94:0/1503617957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c02fb90 con 0x7f8a90101790 2024-04-25T13:54:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.532+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901a2600 con 0x7f8a90101790 2024-04-25T13:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.535+0000 7f8a86ffd640 1 -- 172.21.15.94:0/1503617957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8a8c02fd30 con 0x7f8a90101790 2024-04-25T13:54:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.535+0000 7f8a9e9f6740 1 Processor -- start 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.535+0000 7f8a9e9f6740 1 -- start start 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb5290ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ecf4d0 0x557bb5297df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557bb51f6460 con 0x557bb4ef01b0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557bb51b0590 con 0x557bb4ec72f0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bb51f2420 con 0x557bb4ecf4d0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34386/0 (socket says 172.21.15.94:34386) 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb5290ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 -- 172.21.15.94:0/2244903627 learned_addr learned my addr 172.21.15.94:0/2244903627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a950e1640 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ecf4d0 0x557bb5297df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a86ffd640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 0x7f8a7c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 -- 172.21.15.94:0/2244903627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ecf4d0 msgr2=0x557bb5297df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ecf4d0 0x557bb5297df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 -- 172.21.15.94:0/2244903627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 msgr2=0x557bb5290ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb5290ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 -- 172.21.15.94:0/2244903627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb51bd0d0 con 0x557bb4ef01b0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a958e2640 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb5290ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a86ffd640 1 -- 172.21.15.94:0/1503617957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8a8c1001b0 con 0x7f8a90101790 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 0x7f8a7c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.536+0000 7f8a960e3640 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009dc0 tx=0x7f8a880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46850f13792ea512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.537+0000 7f8a7bfff640 1 -- 172.21.15.94:0/2244903627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800f420 con 0x557bb4ef01b0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.537+0000 7f8a960e3640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 0x7f8a7c0b1ff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a88022000 tx=0x7f8a8801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.537+0000 7f8a7bfff640 1 -- 172.21.15.94:0/2244903627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fa70 con 0x557bb4ef01b0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.537+0000 7f8a7bfff640 1 -- 172.21.15.94:0/2244903627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88012ad0 con 0x557bb4ef01b0 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/2244903627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 msgr2=0x557bb5290970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009dc0 tx=0x7f8a880117b0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/2244903627 shutdown_connections 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ecf4d0 0x557bb5297df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb5290ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/2244903627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ef01b0 0x557bb5290970 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/2244903627 >> 172.21.15.94:0/2244903627 conn(0x557bb4ee4300 msgr2=0x557bb528c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/2244903627 shutdown_connections 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/2244903627 wait complete. 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 -- start start 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 Processor -- start 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.538+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557bb5298840 con 0x557bb4ec72f0 2024-04-25T13:54:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557bb517db60 con 0x557bb4ecf4d0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bb52989c0 con 0x557bb4ef01b0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 10 client.0 _ll_register_callbacks cb 0x557bb51a22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34392/0 (socket says 172.21.15.94:34392) 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 -- 172.21.15.94:0/4004891758 learned_addr learned my addr 172.21.15.94:0/4004891758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a950e1640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a960e3640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 -- 172.21.15.94:0/4004891758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 msgr2=0x557bb52a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 -- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 msgr2=0x557bb529abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 -- 172.21.15.94:0/4004891758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb4f08210 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a950e1640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a958e2640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c004af0 tx=0x7f8a8c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12c382ef78b5f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a797fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0c44b0 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8a8c0c4880 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8800fa70 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.539+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c0c7d60 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 10 client.20127 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557bb4fd4620 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8800fa70 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8a8c004610 con 0x557bb4ec72f0 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 10 client.20127 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 10 client.20127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 20 client.20127 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4eaed80 0x557bb52a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 4 client.20127 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bb5204070 con 0x557bb4eaed80 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.540+0000 7f8a950e1640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4eaed80 0x557bb52a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a797fa640 10 client.20127 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a950e1640 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4eaed80 0x557bb52a8410 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc9267e36dd7c1 server_cookie=e99940dd7281da47 in_seq=0 out_seq=0 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a797fa640 1 -- 172.21.15.94:0/4004891758 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bb5204070 con 0x557bb4eaed80 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a797fa640 10 client.20127 handle_command_reply: tid=0 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a9e9f6740 1 client.20127 shutdown 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 msgr2=0x557bb529a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.541+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c004af0 tx=0x7f8a8c031a20 comp rx=0 tx=0).stop 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 shutdown_connections 2024-04-25T13:54:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4eaed80 0x557bb52a8410 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/4004891758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529a6a0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a797fa640 0 client.20127 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 >> 172.21.15.94:0/4004891758 conn(0x557bb4ee4300 msgr2=0x557bb528b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 shutdown_connections 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/4004891758 wait complete. 2024-04-25T13:54:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.542+0000 7f8a9e9f6740 20 client.20127 trim_cache size 0 max 16384 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 Processor -- start 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 -- start start 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb528c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb529a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901ab5b0 con 0x557bb4ecf4d0 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901ad0f0 con 0x557bb4ec72f0 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901abc80 con 0x557bb4ef01b0 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a950e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a950e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34408/0 (socket says 172.21.15.94:34408) 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a950e1640 1 -- 172.21.15.94:0/246918665 learned_addr learned my addr 172.21.15.94:0/246918665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a958e2640 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb528c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.557+0000 7f8a960e3640 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb529a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 -- 172.21.15.94:0/246918665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 msgr2=0x557bb529a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb529a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 -- 172.21.15.94:0/246918665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 msgr2=0x557bb528c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb528c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 -- 172.21.15.94:0/246918665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb529b710 con 0x557bb4ecf4d0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a950e1640 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f8a9019b330 tx=0x7f8a80011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f043f8e23c1b2c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a7b7fe640 1 -- 172.21.15.94:0/246918665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004220 con 0x557bb4ecf4d0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a7b7fe640 1 -- 172.21.15.94:0/246918665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a80024030 con 0x557bb4ecf4d0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.558+0000 7f8a7b7fe640 1 -- 172.21.15.94:0/246918665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004e40 con 0x557bb4ecf4d0 2024-04-25T13:54:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/246918665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 msgr2=0x557bb52a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f8a9019b330 tx=0x7f8a80011a60 comp rx=0 tx=0).stop 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/246918665 shutdown_connections 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb529a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ecf4d0 0x557bb52a1ad0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/246918665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ec72f0 0x557bb528c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/246918665 >> 172.21.15.94:0/246918665 conn(0x557bb4ee4300 msgr2=0x557bb528bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/246918665 shutdown_connections 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/246918665 wait complete. 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 -- start start 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.559+0000 7f8a9e9f6740 1 Processor -- start 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901aca20 con 0x557bb4ec72f0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901ac350 con 0x557bb4ecf4d0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901a9590 con 0x557bb4ef01b0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 10 client.0 _ll_register_callbacks cb 0x7f8a8c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34414/0 (socket says 172.21.15.94:34414) 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 -- 172.21.15.94:0/1958680817 learned_addr learned my addr 172.21.15.94:0/1958680817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a950e1640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a960e3640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 -- 172.21.15.94:0/1958680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 msgr2=0x557bb52a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 -- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 msgr2=0x557bb529f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 -- 172.21.15.94:0/1958680817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb52a0ab0 con 0x557bb4ec72f0 2024-04-25T13:54:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a958e2640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0c8720 tx=0x7f8a8c0d1450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ff61018a772dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a797fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0e2060 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f8a8c0d0fc0 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.560+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a80024030 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c0c7f40 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 10 client.20133 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557bb52a1db0 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a80024030 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f8a8c0c7850 con 0x557bb4ec72f0 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 10 client.20133 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 10 client.20133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 20 client.20133 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4e841c0 0x557bb5312dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 4 client.20133 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:54:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bb51fdee0 con 0x557bb4e841c0 2024-04-25T13:54:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.561+0000 7f8a950e1640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4e841c0 0x557bb5312dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.562+0000 7f8a797fa640 10 client.20133 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.562+0000 7f8a950e1640 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4e841c0 0x557bb5312dc0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a3ba64f1b64e9e server_cookie=ed47a99c4285a09e in_seq=0 out_seq=0 2024-04-25T13:54:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.741+0000 7f8a797fa640 1 -- 172.21.15.94:0/1958680817 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bb51fdee0 con 0x557bb4e841c0 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a797fa640 10 client.20133 handle_command_reply: tid=0 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 client.20133 shutdown 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 msgr2=0x557bb529f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0c8720 tx=0x7f8a8c0d1450 comp rx=0 tx=0).stop 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 shutdown_connections 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557bb4e841c0 0x557bb5312dc0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb4ef01b0 0x557bb52a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a797fa640 0 client.20133 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557bb4ecf4d0 0x557bb529f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 --2- 172.21.15.94:0/1958680817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557bb4ec72f0 0x557bb529f0d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 >> 172.21.15.94:0/1958680817 conn(0x557bb4ee4300 msgr2=0x557bb528f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 shutdown_connections 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 1 -- 172.21.15.94:0/1958680817 wait complete. 2024-04-25T13:54:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.742+0000 7f8a9e9f6740 20 client.20133 trim_cache size 0 max 16384 2024-04-25T13:54:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:54:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:54:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:54:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 msgr2=0x7f8a7c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 0x7f8a7c0b1ff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a88022000 tx=0x7f8a8801f000 comp rx=0 tx=0).stop 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 msgr2=0x7f8a9019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c02ade0 tx=0x7f8a8c0316e0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 shutdown_connections 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8a7c0afae0 0x7f8a7c0b1ff0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a901050e0 0x7f8a901a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a90102290 0x7f8a9019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 --2- 172.21.15.94:0/1503617957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a90101790 0x7f8a9019a290 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 >> 172.21.15.94:0/1503617957 conn(0x7f8a900fd390 msgr2=0x7f8a90105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 shutdown_connections 2024-04-25T13:54:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:39.743+0000 7f8a97b46640 1 -- 172.21.15.94:0/1503617957 wait complete. 2024-04-25T13:54:39.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1605067 12.52 MB/sec execute 562 sec latency 110.249 ms 2024-04-25T13:54:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:39 smithi137 ceph-mon[32568]: pgmap v735: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:54:39.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:39 smithi094 ceph-mon[29195]: pgmap v735: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:54:39.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:39 smithi138 ceph-mon[32977]: pgmap v735: 97 pgs: 97 active+clean; 44 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-04-25T13:54:40.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1607046 12.52 MB/sec execute 563 sec latency 23.751 ms 2024-04-25T13:54:41.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1609646 12.52 MB/sec execute 564 sec latency 14.447 ms 2024-04-25T13:54:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:41 smithi137 ceph-mon[32568]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2024-04-25T13:54:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:41 smithi094 ceph-mon[29195]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2024-04-25T13:54:41.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:41 smithi138 ceph-mon[32977]: pgmap v736: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2024-04-25T13:54:42.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1612541 12.53 MB/sec execute 565 sec latency 14.217 ms 2024-04-25T13:54:43.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1614258 12.52 MB/sec execute 566 sec latency 26.870 ms 2024-04-25T13:54:43.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:43 smithi137 ceph-mon[32568]: pgmap v737: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-04-25T13:54:43.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:43 smithi094 ceph-mon[29195]: pgmap v737: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-04-25T13:54:43.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:43 smithi138 ceph-mon[32977]: pgmap v737: 97 pgs: 97 active+clean; 49 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-04-25T13:54:44.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1616591 12.52 MB/sec execute 567 sec latency 96.705 ms 2024-04-25T13:54:45.821 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1619147 12.52 MB/sec execute 568 sec latency 14.956 ms 2024-04-25T13:54:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:45 smithi137 ceph-mon[32568]: pgmap v738: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:54:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:45 smithi094 ceph-mon[29195]: pgmap v738: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:54:45.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:45 smithi138 ceph-mon[32977]: pgmap v738: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 388 op/s 2024-04-25T13:54:46.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1620954 12.52 MB/sec execute 569 sec latency 12.317 ms 2024-04-25T13:54:47.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1623689 12.52 MB/sec execute 570 sec latency 58.108 ms 2024-04-25T13:54:47.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:47 smithi137 ceph-mon[32568]: pgmap v739: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:54:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:47 smithi094 ceph-mon[29195]: pgmap v739: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:54:47.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:47 smithi138 ceph-mon[32977]: pgmap v739: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-04-25T13:54:48.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1626276 12.52 MB/sec execute 571 sec latency 14.768 ms 2024-04-25T13:54:49.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1627644 12.51 MB/sec execute 572 sec latency 68.851 ms 2024-04-25T13:54:49.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:49 smithi137 ceph-mon[32568]: pgmap v740: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2024-04-25T13:54:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:49 smithi094 ceph-mon[29195]: pgmap v740: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2024-04-25T13:54:49.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:49 smithi138 ceph-mon[32977]: pgmap v740: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2024-04-25T13:54:50.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1630599 12.52 MB/sec execute 573 sec latency 11.132 ms 2024-04-25T13:54:51.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1632594 12.51 MB/sec execute 574 sec latency 83.888 ms 2024-04-25T13:54:51.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:51 smithi137 ceph-mon[32568]: pgmap v741: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-04-25T13:54:51.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:51 smithi094 ceph-mon[29195]: pgmap v741: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-04-25T13:54:51.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:51 smithi138 ceph-mon[32977]: pgmap v741: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 401 op/s 2024-04-25T13:54:52.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1634782 12.52 MB/sec execute 575 sec latency 13.196 ms 2024-04-25T13:54:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/247044265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74103c60 msgr2=0x7f1b74104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/247044265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74103c60 0x7f1b74104070 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009900 tx=0x7f1b6002a410 comp rx=0 tx=0).stop 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/247044265 shutdown_connections 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/247044265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 0x7f1b7410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/247044265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74103c60 0x7f1b74104070 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/247044265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74102a20 0x7f1b74103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.844+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/247044265 >> 172.21.15.94:0/247044265 conn(0x7f1b740fdc10 msgr2=0x7f1b741000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.845+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/247044265 shutdown_connections 2024-04-25T13:54:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.845+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/247044265 wait complete. 2024-04-25T13:54:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.845+0000 7f1b7ab1f640 1 Processor -- start 2024-04-25T13:54:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.845+0000 7f1b7ab1f640 1 -- start start 2024-04-25T13:54:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.845+0000 7f1b7ab1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 0x7f1b7419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 0x7f1b741a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410e520 con 0x7f1b74102a20 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410e3a0 con 0x7f1b74108960 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410e6a0 con 0x7f1b74103c60 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54208/0 (socket says 172.21.15.94:54208) 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 -- 172.21.15.94:0/1875366679 learned_addr learned my addr 172.21.15.94:0/1875366679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 0x7f1b7419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 0x7f1b741a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 -- 172.21.15.94:0/1875366679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 msgr2=0x7f1b7419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 0x7f1b7419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 -- 172.21.15.94:0/1875366679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 msgr2=0x7f1b741a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 0x7f1b741a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 -- 172.21.15.94:0/1875366679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741a1d40 con 0x7f1b74102a20 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 0x7f1b7419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9f0 tx=0x7f1b6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b71ffb640 1 -- 172.21.15.94:0/1875366679 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800ccc0 con 0x7f1b74102a20 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b71ffb640 1 -- 172.21.15.94:0/1875366679 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680035f0 con 0x7f1b74102a20 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741a1fd0 con 0x7f1b74102a20 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.846+0000 7f1b71ffb640 1 -- 172.21.15.94:0/1875366679 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800ce60 con 0x7f1b74102a20 2024-04-25T13:54:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.847+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a2490 con 0x7f1b74102a20 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.849+0000 7f1b71ffb640 1 -- 172.21.15.94:0/1875366679 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1b68003aa0 con 0x7f1b74102a20 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 Processor -- start 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 -- start start 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b71ffb640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 0x7f1b640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f624460 con 0x55cd2f2f52f0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f5de590 con 0x55cd2f31e1b0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b819d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f620420 con 0x55cd2f2fd4d0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 0x7f1b640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54218/0 (socket says 172.21.15.94:54218) 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b73fff640 1 -- 172.21.15.94:0/4229815318 learned_addr learned my addr 172.21.15.94:0/4229815318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b790bc640 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b71ffb640 1 -- 172.21.15.94:0/1875366679 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b680def80 con 0x7f1b74102a20 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b788bb640 1 -- 172.21.15.94:0/4229815318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 msgr2=0x55cd2f6ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b788bb640 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.850+0000 7f1b788bb640 1 -- 172.21.15.94:0/4229815318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 msgr2=0x55cd2f6c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b788bb640 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b788bb640 1 -- 172.21.15.94:0/4229815318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2f5eb0d0 con 0x55cd2f31e1b0 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 0x7f1b640b1ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005fd0 tx=0x7f1b6c009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b788bb640 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6c2d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b6801c000 tx=0x7f1b68021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1545ce46dee8aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b5effd640 1 -- 172.21.15.94:0/4229815318 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800be30 con 0x55cd2f31e1b0 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b5effd640 1 -- 172.21.15.94:0/4229815318 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680a6ed0 con 0x55cd2f31e1b0 2024-04-25T13:54:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.851+0000 7f1b5effd640 1 -- 172.21.15.94:0/4229815318 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b680ab770 con 0x55cd2f31e1b0 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- 172.21.15.94:0/4229815318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 msgr2=0x55cd2f6c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6c2d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b6801c000 tx=0x7f1b68021000 comp rx=0 tx=0).stop 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- 172.21.15.94:0/4229815318 shutdown_connections 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 --2- 172.21.15.94:0/4229815318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6c2d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- 172.21.15.94:0/4229815318 >> 172.21.15.94:0/4229815318 conn(0x55cd2f312300 msgr2=0x55cd2f6ba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- 172.21.15.94:0/4229815318 shutdown_connections 2024-04-25T13:54:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- 172.21.15.94:0/4229815318 wait complete. 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 -- start start 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.852+0000 7f1b819d1740 1 Processor -- start 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f6cac50 con 0x55cd2f2f52f0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f5abb60 con 0x55cd2f2fd4d0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd2f6cadd0 con 0x55cd2f31e1b0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b819d1740 10 client.0 _ll_register_callbacks cb 0x55cd2f5d02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54230/0 (socket says 172.21.15.94:54230) 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 -- 172.21.15.94:0/1967225940 learned_addr learned my addr 172.21.15.94:0/1967225940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b790bc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 -- 172.21.15.94:0/1967225940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f31e1b0 msgr2=0x55cd2f6d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 -- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 msgr2=0x55cd2f6cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b788bb640 1 -- 172.21.15.94:0/1967225940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2f336210 con 0x55cd2f2f52f0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.853+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f1b680dcd50 tx=0x7f1b6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234d86035ffced60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b680ab770 con 0x55cd2f2f52f0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b680afd90 con 0x55cd2f2f52f0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680a6ed0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b680aff90 con 0x55cd2f2f52f0 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 10 client.20145 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd2f402620 con 0x55cd2f2f52f0 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680a6ed0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1b68003fd0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 10 client.20145 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 10 client.20145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 20 client.20145 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2dcd80 0x55cd2f6da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 4 client.20145 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.854+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd2f632070 con 0x55cd2f2dcd80 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.855+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2dcd80 0x55cd2f6da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.855+0000 7f1b53fff640 10 client.20145 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.855+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2dcd80 0x55cd2f6da220 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e00196319065639 server_cookie=8d7407fac3a1da8c in_seq=0 out_seq=0 2024-04-25T13:54:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.858+0000 7f1b53fff640 1 -- 172.21.15.94:0/1967225940 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd2f632070 con 0x55cd2f2dcd80 2024-04-25T13:54:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.858+0000 7f1b53fff640 10 client.20145 handle_command_reply: tid=0 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.858+0000 7f1b819d1740 1 client.20145 shutdown 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 msgr2=0x55cd2f6cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f1b680dcd50 tx=0x7f1b6800b330 comp rx=0 tx=0).stop 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 shutdown_connections 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2dcd80 0x55cd2f6da220 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b53fff640 0 client.20145 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1967225940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6cc930 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 >> 172.21.15.94:0/1967225940 conn(0x55cd2f312300 msgr2=0x55cd2f6bd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 shutdown_connections 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 1 -- 172.21.15.94:0/1967225940 wait complete. 2024-04-25T13:54:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.859+0000 7f1b819d1740 20 client.20145 trim_cache size 0 max 16384 2024-04-25T13:54:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.874+0000 7f1b819d1740 1 Processor -- start 2024-04-25T13:54:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.874+0000 7f1b819d1740 1 -- start start 2024-04-25T13:54:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741a9310 con 0x55cd2f31e1b0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741aae50 con 0x55cd2f2fd4d0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b819d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741a99e0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54242/0 (socket says 172.21.15.94:54242) 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 -- 172.21.15.94:0/1482827089 learned_addr learned my addr 172.21.15.94:0/1482827089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b788bb640 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b73fff640 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 -- 172.21.15.94:0/1482827089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2f52f0 msgr2=0x55cd2f6c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 -- 172.21.15.94:0/1482827089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 msgr2=0x55cd2f6d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 -- 172.21.15.94:0/1482827089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2f6c77d0 con 0x55cd2f31e1b0 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b790bc640 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00ffd0 tx=0x7f1b6c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ca681529c67657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b5e7fc640 1 -- 172.21.15.94:0/1482827089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c01fd70 con 0x55cd2f31e1b0 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b5e7fc640 1 -- 172.21.15.94:0/1482827089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6c003e20 con 0x55cd2f31e1b0 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.875+0000 7f1b5e7fc640 1 -- 172.21.15.94:0/1482827089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c004530 con 0x55cd2f31e1b0 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 -- 172.21.15.94:0/1482827089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 msgr2=0x55cd2f6b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00ffd0 tx=0x7f1b6c009500 comp rx=0 tx=0).stop 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 -- 172.21.15.94:0/1482827089 shutdown_connections 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6b9cc0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 --2- 172.21.15.94:0/1482827089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 -- 172.21.15.94:0/1482827089 >> 172.21.15.94:0/1482827089 conn(0x55cd2f312300 msgr2=0x55cd2f6c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 -- 172.21.15.94:0/1482827089 shutdown_connections 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.876+0000 7f1b819d1740 1 -- 172.21.15.94:0/1482827089 wait complete. 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 -- start start 2024-04-25T13:54:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 Processor -- start 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741aa780 con 0x55cd2f2f52f0 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741aa0b0 con 0x55cd2f31e1b0 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741a72f0 con 0x55cd2f2fd4d0 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b819d1740 10 client.0 _ll_register_callbacks cb 0x7f1b680a3280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b788bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54248/0 (socket says 172.21.15.94:54248) 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b788bb640 1 -- 172.21.15.94:0/3934528427 learned_addr learned my addr 172.21.15.94:0/3934528427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b73fff640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.877+0000 7f1b790bc640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 -- 172.21.15.94:0/3934528427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 msgr2=0x55cd2f6d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 -- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 msgr2=0x55cd2f6d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 -- 172.21.15.94:0/3934528427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2f6d2ac0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b73fff640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b788bb640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004370 tx=0x7f1b680a2c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eabf7495d417f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b53fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b680a6ed0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b680a39b0 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c003e20 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b680ab770 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b819d1740 10 client.20151 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd2f6d3d50 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.878+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6c003e20 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1b680a6620 con 0x55cd2f2f52f0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 10 client.20151 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 10 client.20151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 20 client.20151 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2b21c0 0x55cd2f744780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 4 client.20151 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd2f62bee0 con 0x55cd2f2b21c0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b73fff640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2b21c0 0x55cd2f744780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b53fff640 10 client.20151 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:52.879+0000 7f1b73fff640 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2b21c0 0x55cd2f744780 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f658c99ca4d92b90 server_cookie=7e2d6f30b90a00d6 in_seq=0 out_seq=0 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.086+0000 7f1b53fff640 1 -- 172.21.15.94:0/3934528427 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd2f62bee0 con 0x55cd2f2b21c0 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.086+0000 7f1b53fff640 10 client.20151 handle_command_reply: tid=0 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.086+0000 7f1b819d1740 1 client.20151 shutdown 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.086+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 msgr2=0x55cd2f6d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.086+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004370 tx=0x7f1b680a2c80 comp rx=0 tx=0).stop 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 shutdown_connections 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55cd2f2b21c0 0x55cd2f744780 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd2f31e1b0 0x55cd2f6d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd2f2fd4d0 0x55cd2f6d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 --2- 172.21.15.94:0/3934528427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd2f2f52f0 0x55cd2f6d1100 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b53fff640 0 client.20151 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 >> 172.21.15.94:0/3934528427 conn(0x55cd2f312300 msgr2=0x55cd2f6b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 shutdown_connections 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 1 -- 172.21.15.94:0/3934528427 wait complete. 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.087+0000 7f1b819d1740 20 client.20151 trim_cache size 0 max 16384 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:54:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:54:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:54:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 msgr2=0x7f1b640b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 0x7f1b640b1ff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005fd0 tx=0x7f1b6c009e00 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 msgr2=0x7f1b7406ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9f0 tx=0x7f1b6800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 shutdown_connections 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b640afae0 0x7f1b640b1ff0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b74108960 0x7f1b741a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b74103c60 0x7f1b7419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 --2- 172.21.15.94:0/1875366679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b74102a20 0x7f1b7406ae30 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 >> 172.21.15.94:0/1875366679 conn(0x7f1b740fdc10 msgr2=0x7f1b74104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 shutdown_connections 2024-04-25T13:54:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:54:53.088+0000 7f1b7ab1f640 1 -- 172.21.15.94:0/1875366679 wait complete. 2024-04-25T13:54:53.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1637351 12.52 MB/sec execute 576 sec latency 113.020 ms 2024-04-25T13:54:53.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:53 smithi137 ceph-mon[32568]: pgmap v742: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-04-25T13:54:53.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:53 smithi094 ceph-mon[29195]: pgmap v742: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-04-25T13:54:53.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:53 smithi138 ceph-mon[32977]: pgmap v742: 97 pgs: 97 active+clean; 47 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-04-25T13:54:54.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:54:54.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 -- 172.21.15.94:0/723039269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 msgr2=0x7fbc8010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 --2- 172.21.15.94:0/723039269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc8010ad40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009900 tx=0x7fbc7802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 -- 172.21.15.94:0/723039269 shutdown_connections 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 --2- 172.21.15.94:0/723039269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc8010ad40 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 --2- 172.21.15.94:0/723039269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 0x7fbc80104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 --2- 172.21.15.94:0/723039269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 0x7fbc801036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 -- 172.21.15.94:0/723039269 >> 172.21.15.94:0/723039269 conn(0x7fbc800fdbd0 msgr2=0x7fbc80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 -- 172.21.15.94:0/723039269 shutdown_connections 2024-04-25T13:54:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.469+0000 7fbc85e27640 1 -- 172.21.15.94:0/723039269 wait complete. 2024-04-25T13:54:54.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 Processor -- start 2024-04-25T13:54:54.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 -- start start 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 0x7fbc801993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 0x7fbc80199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8010e4f0 con 0x7fbc80108920 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8010e370 con 0x7fbc80103c20 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.470+0000 7fbc85e27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8010e670 con 0x7fbc801029e0 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40344/0 (socket says 172.21.15.94:40344) 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 -- 172.21.15.94:0/1171646325 learned_addr learned my addr 172.21.15.94:0/1171646325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 0x7fbc801993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7effd640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 0x7fbc80199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 -- 172.21.15.94:0/1171646325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 msgr2=0x7fbc801993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 0x7fbc801993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 -- 172.21.15.94:0/1171646325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 msgr2=0x7fbc80199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 0x7fbc80199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 -- 172.21.15.94:0/1171646325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc801a0e80 con 0x7fbc80108920 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7ffff640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009230 tx=0x7fbc780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7cff9640 1 -- 172.21.15.94:0/1171646325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7803f030 con 0x7fbc80108920 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7cff9640 1 -- 172.21.15.94:0/1171646325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc78031910 con 0x7fbc80108920 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc801a1110 con 0x7fbc80108920 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc7cff9640 1 -- 172.21.15.94:0/1171646325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78035070 con 0x7fbc80108920 2024-04-25T13:54:54.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.471+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc801a1600 con 0x7fbc80108920 2024-04-25T13:54:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc7cff9640 1 -- 172.21.15.94:0/1171646325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbc78042030 con 0x7fbc80108920 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 Processor -- start 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 -- start start 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x562135531fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213516f6b0 0x562135532c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562135496d80 con 0x562135190390 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5621354508b0 con 0x5621351674d0 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562135492d40 con 0x56213516f6b0 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc7effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc7effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40360/0 (socket says 172.21.15.94:40360) 2024-04-25T13:54:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc7effd640 1 -- 172.21.15.94:0/2038383016 learned_addr learned my addr 172.21.15.94:0/2038383016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.474+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x562135531fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213516f6b0 0x562135532c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7cff9640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 0x7fbc680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7ffff640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 0x7fbc680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 -- 172.21.15.94:0/2038383016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213516f6b0 msgr2=0x562135532c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213516f6b0 0x562135532c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 -- 172.21.15.94:0/2038383016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 msgr2=0x562135531fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x562135531fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 -- 172.21.15.94:0/2038383016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562135421a40 con 0x562135190390 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7cff9640 1 -- 172.21.15.94:0/1171646325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbc780ff4c0 con 0x7fbc80108920 2024-04-25T13:54:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbc74013820 tx=0x7fbc740097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15c66540d155366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc65ffb640 1 -- 172.21.15.94:0/2038383016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74013da0 con 0x562135190390 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc65ffb640 1 -- 172.21.15.94:0/2038383016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74013170 con 0x562135190390 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.475+0000 7fbc65ffb640 1 -- 172.21.15.94:0/2038383016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004d50 con 0x562135190390 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2038383016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 msgr2=0x562135531a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc7ffff640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 0x7fbc680b2000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbc780c4c20 tx=0x7fbc7808ded0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbc74013820 tx=0x7fbc740097a0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2038383016 shutdown_connections 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213516f6b0 0x562135532c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x562135531fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2038383016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562135190390 0x562135531a60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2038383016 >> 172.21.15.94:0/2038383016 conn(0x5621351844e0 msgr2=0x56213552d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2038383016 shutdown_connections 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2038383016 wait complete. 2024-04-25T13:54:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.476+0000 7fbc8cdbc740 1 -- start start 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 Processor -- start 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562135190390 0x562135544f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562135533720 con 0x56213516f6b0 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56213541dd10 con 0x5621351674d0 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621355338a0 con 0x562135190390 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc8cdbc740 10 client.0 _ll_register_callbacks cb 0x56213544a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40364/0 (socket says 172.21.15.94:40364) 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/2732299960 learned_addr learned my addr 172.21.15.94:0/2732299960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562135190390 0x562135544f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/2732299960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562135190390 msgr2=0x562135544f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562135190390 0x562135544f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 msgr2=0x56213553da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.477+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/2732299960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621352727d0 con 0x56213516f6b0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbc70010b70 tx=0x7fbc7000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1874a19ac09cedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003970 con 0x56213516f6b0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbc70003b10 con 0x56213516f6b0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc74013170 con 0x56213516f6b0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc70003d50 con 0x56213516f6b0 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 10 client.20163 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:54.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621351a83f0 con 0x56213516f6b0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74013170 con 0x56213516f6b0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fbc7000e830 con 0x56213516f6b0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 10 client.20163 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 10 client.20163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 20 client.20163 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56213514ef60 0x56213554b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 4 client.20163 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.478+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621354a4990 con 0x56213514ef60 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.479+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56213514ef60 0x56213554b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.479+0000 7fbc5effd640 10 client.20163 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.479+0000 7fbc7effd640 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56213514ef60 0x56213554b410 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef13a3b02874283e server_cookie=68617f6390607093 in_seq=0 out_seq=0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc5effd640 1 -- 172.21.15.94:0/2732299960 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621354a4990 con 0x56213514ef60 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc5effd640 10 client.20163 handle_command_reply: tid=0 2024-04-25T13:54:54.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc8cdbc740 1 client.20163 shutdown 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 msgr2=0x56213553dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbc70010b70 tx=0x7fbc7000f7f0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 shutdown_connections 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.481+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56213514ef60 0x56213554b410 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc5effd640 0 client.20163 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562135190390 0x562135544f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213516f6b0 0x56213553dfc0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/2732299960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 >> 172.21.15.94:0/2732299960 conn(0x5621351844e0 msgr2=0x562135530180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 shutdown_connections 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/2732299960 wait complete. 2024-04-25T13:54:54.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.482+0000 7fbc8cdbc740 20 client.20163 trim_cache size 0 max 16384 2024-04-25T13:54:54.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 Processor -- start 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 -- start start 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x56213553a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801a91d0 con 0x56213514ef60 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801aad10 con 0x56213516f6b0 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801a98a0 con 0x5621351674d0 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40368/0 (socket says 172.21.15.94:40368) 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 -- 172.21.15.94:0/162445255 learned_addr learned my addr 172.21.15.94:0/162445255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x56213553a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 -- 172.21.15.94:0/162445255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621351674d0 msgr2=0x56213553da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 -- 172.21.15.94:0/162445255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 msgr2=0x56213553a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x56213553a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.501+0000 7fbc7effd640 1 -- 172.21.15.94:0/162445255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621351680f0 con 0x56213514ef60 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc7effd640 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbc740058c0 tx=0x7fbc74015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e64d5b902af3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc657fa640 1 -- 172.21.15.94:0/162445255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74013170 con 0x56213514ef60 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc657fa640 1 -- 172.21.15.94:0/162445255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74004800 con 0x56213514ef60 2024-04-25T13:54:54.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc657fa640 1 -- 172.21.15.94:0/162445255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400fb90 con 0x56213514ef60 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/162445255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 msgr2=0x562135536c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.502+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbc740058c0 tx=0x7fbc74015040 comp rx=0 tx=0).stop 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/162445255 shutdown_connections 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x56213553a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621351674d0 0x56213553da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/162445255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56213514ef60 0x562135536c30 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/162445255 >> 172.21.15.94:0/162445255 conn(0x5621351844e0 msgr2=0x562135539f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/162445255 shutdown_connections 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/162445255 wait complete. 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- start start 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 Processor -- start 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213514ef60 0x562135545530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x5621355425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801aa640 con 0x5621351674d0 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801a9f70 con 0x56213516f6b0 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc801a71b0 con 0x56213514ef60 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.503+0000 7fbc8cdbc740 10 client.0 _ll_register_callbacks cb 0x7fbc70007a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x5621355425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40374/0 (socket says 172.21.15.94:40374) 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/3719696565 learned_addr learned my addr 172.21.15.94:0/3719696565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7effd640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213514ef60 0x562135545530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/3719696565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213514ef60 msgr2=0x562135545530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213514ef60 0x562135545530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 msgr2=0x5621355425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x5621355425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 -- 172.21.15.94:0/3719696565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621354574b0 con 0x5621351674d0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc7f7fe640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007470 tx=0x7fbc700104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e6d62400520e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc5effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7002a040 con 0x5621351674d0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbc7000ee40 con 0x5621351674d0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc74004800 con 0x5621351674d0 2024-04-25T13:54:54.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbc7001a460 con 0x5621351674d0 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc8cdbc740 10 client.20169 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.504+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621355b2680 con 0x5621351674d0 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74004800 con 0x5621351674d0 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fbc7001a9d0 con 0x5621351674d0 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 10 client.20169 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 10 client.20169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 20 client.20169 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562135190390 0x5621355b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 4 client.20169 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56213549e800 con 0x562135190390 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc7effd640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562135190390 0x5621355b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc7effd640 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562135190390 0x5621355b50a0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38bb1c4625b741 server_cookie=1d066d5a2dd14b16 in_seq=0 out_seq=0 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.505+0000 7fbc5effd640 10 client.20169 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc5effd640 1 -- 172.21.15.94:0/3719696565 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56213549e800 con 0x562135190390 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc5effd640 10 client.20169 handle_command_reply: tid=0 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 client.20169 shutdown 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 msgr2=0x5621355420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007470 tx=0x7fbc700104b0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 shutdown_connections 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562135190390 0x5621355b50a0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56213516f6b0 0x5621355425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621351674d0 0x5621355420b0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc5effd640 0 client.20169 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 --2- 172.21.15.94:0/3719696565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56213514ef60 0x562135545530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 >> 172.21.15.94:0/3719696565 conn(0x5621351844e0 msgr2=0x56213552b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 shutdown_connections 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 1 -- 172.21.15.94:0/3719696565 wait complete. 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.507+0000 7fbc8cdbc740 20 client.20169 trim_cache size 0 max 16384 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:54:54.512 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 msgr2=0x7fbc680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 0x7fbc680b2000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbc780c4c20 tx=0x7fbc7808ded0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 msgr2=0x7fbc801a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009230 tx=0x7fbc780042f0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 shutdown_connections 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc680afaf0 0x7fbc680b2000 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc80108920 0x7fbc801a08c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc80103c20 0x7fbc80199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 --2- 172.21.15.94:0/1171646325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc801029e0 0x7fbc801993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 >> 172.21.15.94:0/1171646325 conn(0x7fbc800fdbd0 msgr2=0x7fbc80104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 shutdown_connections 2024-04-25T13:54:54.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.508+0000 7fbc85e27640 1 -- 172.21.15.94:0/1171646325 wait complete. 2024-04-25T13:54:54.518 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:54:54.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:54:54.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- 172.21.15.94:0/3232616234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7640102260 msgr2=0x7f764010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 --2- 172.21.15.94:0/3232616234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7640102260 0x7f764010bcc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f763802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:54.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- 172.21.15.94:0/3232616234 shutdown_connections 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 --2- 172.21.15.94:0/3232616234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7640102260 0x7f764010bcc0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 --2- 172.21.15.94:0/3232616234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f7640101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 --2- 172.21.15.94:0/3232616234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76401050b0 0x7f76401054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- 172.21.15.94:0/3232616234 >> 172.21.15.94:0/3232616234 conn(0x7f76400fd360 msgr2=0x7f76400ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- 172.21.15.94:0/3232616234 shutdown_connections 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- 172.21.15.94:0/3232616234 wait complete. 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 Processor -- start 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.634+0000 7f76478e0640 1 -- start start 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f764019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640102260 0x7f764019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f764010f640 con 0x7f76401050b0 2024-04-25T13:54:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f764010f4c0 con 0x7f7640101760 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76478e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f764010f7c0 con 0x7f7640102260 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f764567c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f764019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40390/0 (socket says 172.21.15.94:40390) 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 -- 172.21.15.94:0/2103722985 learned_addr learned my addr 172.21.15.94:0/2103722985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7644e7b640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640102260 0x7f764019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 -- 172.21.15.94:0/2103722985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640102260 msgr2=0x7f764019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640102260 0x7f764019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 -- 172.21.15.94:0/2103722985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 msgr2=0x7f764019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f764019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 -- 172.21.15.94:0/2103722985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a1ca0 con 0x7f76401050b0 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f764567c640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f764019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f7645e7d640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7638009230 tx=0x7f76380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.635+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763803f030 con 0x7f76401050b0 2024-04-25T13:54:54.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.636+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7638031910 con 0x7f76401050b0 2024-04-25T13:54:54.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.636+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a1f30 con 0x7f76401050b0 2024-04-25T13:54:54.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.636+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638035070 con 0x7f76401050b0 2024-04-25T13:54:54.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.636+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a2340 con 0x7f76401050b0 2024-04-25T13:54:54.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.637+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764010aa50 con 0x7f76401050b0 2024-04-25T13:54:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.638+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7638042030 con 0x7f76401050b0 2024-04-25T13:54:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.638+0000 7f76367fc640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 0x7f762c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.639+0000 7f764567c640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 0x7f762c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.639+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f76380ff4f0 con 0x7f76401050b0 2024-04-25T13:54:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.639+0000 7f764567c640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 0x7f762c0b2000 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f764006b300 tx=0x7f763c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.643+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76380c79e0 con 0x7f76401050b0 2024-04-25T13:54:54.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1638652 12.51 MB/sec execute 577 sec latency 46.630 ms 2024-04-25T13:54:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.876+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f764002dc70 con 0x7f76401050b0 2024-04-25T13:54:54.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.877+0000 7f76367fc640 1 -- 172.21.15.94:0/2103722985 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11941 (secure 0 0 0) 0x7f76380cbce0 con 0x7f76401050b0 2024-04-25T13:54:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:54:54.876 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:52:14.280433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:54:54.876 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 msgr2=0x7f762c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 0x7f762c0b2000 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f764006b300 tx=0x7f763c009500 comp rx=0 tx=0).stop 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 msgr2=0x7f76401a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7638009230 tx=0x7f76380042f0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 shutdown_connections 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f762c0afaf0 0x7f762c0b2000 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401050b0 0x7f76401a1630 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7640102260 0x7f764019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 --2- 172.21.15.94:0/2103722985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7640101760 0x7f764019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 >> 172.21.15.94:0/2103722985 conn(0x7f76400fd360 msgr2=0x7f7640107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 shutdown_connections 2024-04-25T13:54:54.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:54.879+0000 7f76478e0640 1 -- 172.21.15.94:0/2103722985 wait complete. 2024-04-25T13:54:54.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:54:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 -- 172.21.15.94:0/1173644279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988103c30 msgr2=0x7f5988104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 --2- 172.21.15.94:0/1173644279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988103c30 0x7f5988104040 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f597002e820 comp rx=0 tx=0).stop 2024-04-25T13:54:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 -- 172.21.15.94:0/1173644279 shutdown_connections 2024-04-25T13:54:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 --2- 172.21.15.94:0/1173644279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988108930 0x7f598810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 --2- 172.21.15.94:0/1173644279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988103c30 0x7f5988104040 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 --2- 172.21.15.94:0/1173644279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 0x7f59881036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 -- 172.21.15.94:0/1173644279 >> 172.21.15.94:0/1173644279 conn(0x7f59880fdc00 msgr2=0x7f5988100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 -- 172.21.15.94:0/1173644279 shutdown_connections 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.005+0000 7f598d849640 1 -- 172.21.15.94:0/1173644279 wait complete. 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 Processor -- start 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 -- start start 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 0x7f5988198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988103c30 0x7f59881985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f598810d410 con 0x7f5988108930 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f598810d290 con 0x7f5988103c30 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f598d849640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f598810d590 con 0x7f59881029f0 2024-04-25T13:54:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40418/0 (socket says 172.21.15.94:40418) 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 -- 172.21.15.94:0/2715480036 learned_addr learned my addr 172.21.15.94:0/2715480036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f5986ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 0x7f5988198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f597ffff640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988103c30 0x7f59881985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 -- 172.21.15.94:0/2715480036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 msgr2=0x7f5988198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 0x7f5988198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 -- 172.21.15.94:0/2715480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988103c30 msgr2=0x7f59881985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988103c30 0x7f59881985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 -- 172.21.15.94:0/2715480036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598819fb50 con 0x7f5988108930 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.006+0000 7f59877fe640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f598000da90 tx=0x7f598000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.007+0000 7f598c847640 1 -- 172.21.15.94:0/2715480036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598001b020 con 0x7f5988108930 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.007+0000 7f598c847640 1 -- 172.21.15.94:0/2715480036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598000bc30 con 0x7f5988108930 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.007+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598819fe40 con 0x7f5988108930 2024-04-25T13:54:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.007+0000 7f598c847640 1 -- 172.21.15.94:0/2715480036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980005950 con 0x7f5988108930 2024-04-25T13:54:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.007+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59881a0300 con 0x7f5988108930 2024-04-25T13:54:55.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.009+0000 7f59947d3740 1 Processor -- start 2024-04-25T13:54:55.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.009+0000 7f598c847640 1 -- 172.21.15.94:0/2715480036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5980004a40 con 0x7f5988108930 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.009+0000 7f59947d3740 1 -- start start 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.009+0000 7f59947d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.009+0000 7f59947d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a8fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f59947d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f59947d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564fd69f3d80 con 0x564fd66ed390 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f59947d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564fd69ada20 con 0x564fd66cc6b0 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f59947d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fd69efd40 con 0x564fd66c44d0 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40422/0 (socket says 172.21.15.94:40422) 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597ffff640 1 -- 172.21.15.94:0/3387143420 learned_addr learned my addr 172.21.15.94:0/3387143420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a8fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597ffff640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f598c847640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 0x7f59740b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 -- 172.21.15.94:0/3387143420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 msgr2=0x564fd6a90a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a90a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 -- 172.21.15.94:0/3387143420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 msgr2=0x564fd6a8f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 -- 172.21.15.94:0/3387143420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd69b8c30 con 0x564fd66c44d0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597ffff640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f59877fe640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 0x7f59740b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f598c847640 1 -- 172.21.15.94:0/2715480036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f59800dea70 con 0x7f5988108930 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f5986ffd640 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a8fd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59881022c0 tx=0x7f5978011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20b6c3d4fb48cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597dffb640 1 -- 172.21.15.94:0/3387143420 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978004230 con 0x564fd66c44d0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597dffb640 1 -- 172.21.15.94:0/3387143420 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5978009e10 con 0x564fd66c44d0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.010+0000 7f597dffb640 1 -- 172.21.15.94:0/3387143420 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978019680 con 0x564fd66c44d0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 -- 172.21.15.94:0/3387143420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 msgr2=0x564fd6a8fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59877fe640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 0x7f59740b2080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59800a6c40 tx=0x7f5980075a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a8fd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59881022c0 tx=0x7f5978011810 comp rx=0 tx=0).stop 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 -- 172.21.15.94:0/3387143420 shutdown_connections 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a8fd90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 --2- 172.21.15.94:0/3387143420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 -- 172.21.15.94:0/3387143420 >> 172.21.15.94:0/3387143420 conn(0x564fd66e14e0 msgr2=0x564fd6a8b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 -- 172.21.15.94:0/3387143420 shutdown_connections 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.011+0000 7f59947d3740 1 -- 172.21.15.94:0/3387143420 wait complete. 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 -- start start 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 Processor -- start 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a99560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a99aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564fd6a91530 con 0x564fd66ed390 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564fd697adc0 con 0x564fd66cc6b0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fd6a916b0 con 0x564fd66c44d0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f59947d3740 10 client.0 _ll_register_callbacks cb 0x564fd6996600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40430/0 (socket says 172.21.15.94:40430) 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f597ffff640 1 -- 172.21.15.94:0/4081613715 learned_addr learned my addr 172.21.15.94:0/4081613715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f597ffff640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a99560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.012+0000 7f5986ffd640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a99aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 -- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 msgr2=0x564fd6a99560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a99560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 -- 172.21.15.94:0/4081613715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 msgr2=0x564fd6a99aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a99aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 -- 172.21.15.94:0/4081613715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd67053f0 con 0x564fd66ed390 2024-04-25T13:54:55.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f597ffff640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f5970036410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9726e0e8967f6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f5966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970004260 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5970033040 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5978009e10 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5970004400 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f59947d3740 10 client.20187 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fd67cf7d0 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.013+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5978009e10 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f59700454e0 con 0x564fd66ed390 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 10 client.20187 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 10 client.20187 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 20 client.20187 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66abf60 0x564fd6aa7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 4 client.20187 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:55.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd6a01990 con 0x564fd66abf60 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f5986ffd640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66abf60 0x564fd6aa7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f5966ffd640 10 client.20187 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.014+0000 7f5986ffd640 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66abf60 0x564fd6aa7180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5af18407e08bb49 server_cookie=43df27754fb39309 in_seq=0 out_seq=0 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f5966ffd640 1 -- 172.21.15.94:0/4081613715 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564fd6a01990 con 0x564fd66abf60 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f5966ffd640 10 client.20187 handle_command_reply: tid=0 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f59947d3740 1 client.20187 shutdown 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 msgr2=0x564fd6aa0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f5970036410 comp rx=0 tx=0).stop 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 shutdown_connections 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.015+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66abf60 0x564fd6aa7180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6aa0a00 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a99aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 --2- 172.21.15.94:0/4081613715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a99560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f5966ffd640 0 client.20187 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 >> 172.21.15.94:0/4081613715 conn(0x564fd66e14e0 msgr2=0x564fd6a8df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 shutdown_connections 2024-04-25T13:54:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 1 -- 172.21.15.94:0/4081613715 wait complete. 2024-04-25T13:54:55.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.016+0000 7f59947d3740 20 client.20187 trim_cache size 0 max 16384 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 Processor -- start 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 -- start start 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66c44d0 0x564fd6a8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66cc6b0 0x564fd6aa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59881a9330 con 0x564fd66ed390 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aae70 con 0x564fd66c44d0 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.030+0000 7f59947d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59881a9a00 con 0x564fd66cc6b0 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40440/0 (socket says 172.21.15.94:40440) 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 -- 172.21.15.94:0/3622871231 learned_addr learned my addr 172.21.15.94:0/3622871231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f5986ffd640 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66cc6b0 0x564fd6aa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66c44d0 0x564fd6a8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 -- 172.21.15.94:0/3622871231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66cc6b0 msgr2=0x564fd6aa0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66cc6b0 0x564fd6aa0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 -- 172.21.15.94:0/3622871231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66c44d0 msgr2=0x564fd6a8a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66c44d0 0x564fd6a8a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 -- 172.21.15.94:0/3622871231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd6a9a6a0 con 0x564fd66ed390 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597ffff640 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f59700059d0 tx=0x7f597002e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d7f7f6d2517df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597d7fa640 1 -- 172.21.15.94:0/3622871231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970000e60 con 0x564fd66ed390 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597d7fa640 1 -- 172.21.15.94:0/3622871231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5970045c30 con 0x564fd66ed390 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.031+0000 7f597d7fa640 1 -- 172.21.15.94:0/3622871231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970003dd0 con 0x564fd66ed390 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- 172.21.15.94:0/3622871231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 msgr2=0x564fd6a8af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f59700059d0 tx=0x7f597002e820 comp rx=0 tx=0).stop 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- 172.21.15.94:0/3622871231 shutdown_connections 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a8af20 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66cc6b0 0x564fd6aa0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 --2- 172.21.15.94:0/3622871231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66c44d0 0x564fd6a8a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- 172.21.15.94:0/3622871231 >> 172.21.15.94:0/3622871231 conn(0x564fd66e14e0 msgr2=0x564fd6a8f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- 172.21.15.94:0/3622871231 shutdown_connections 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- 172.21.15.94:0/3622871231 wait complete. 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 -- start start 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.032+0000 7f59947d3740 1 Processor -- start 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a9e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aa7a0 con 0x564fd66ed390 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aa0d0 con 0x564fd66cc6b0 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59881a7310 con 0x564fd66c44d0 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 10 client.0 _ll_register_callbacks cb 0x7f5970009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40446/0 (socket says 172.21.15.94:40446) 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 -- 172.21.15.94:0/1082451343 learned_addr learned my addr 172.21.15.94:0/1082451343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f5986ffd640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a9e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 -- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 msgr2=0x564fd6a9e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a9e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 -- 172.21.15.94:0/1082451343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 msgr2=0x564fd6a9e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a9e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 -- 172.21.15.94:0/1082451343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd6a9fe40 con 0x564fd66ed390 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f597ffff640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f597002ed40 tx=0x7f59700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b989ef0d3afc53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f5966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970060070 con 0x564fd66ed390 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f59700037f0 con 0x564fd66ed390 2024-04-25T13:54:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.033+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5970045c30 con 0x564fd66ed390 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5970003b70 con 0x564fd66ed390 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 10 client.20193 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fd6b0f5d0 con 0x564fd66ed390 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5970045c30 con 0x564fd66ed390 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5970044ab0 con 0x564fd66ed390 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 10 client.20193 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 10 client.20193 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 20 client.20193 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66813a0 0x564fd6b11e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 4 client.20193 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fd69fb800 con 0x564fd66813a0 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.034+0000 7f5986ffd640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66813a0 0x564fd6b11e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f5966ffd640 10 client.20193 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f5986ffd640 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66813a0 0x564fd6b11e80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=484c5b90f8eefbac server_cookie=c0062a23a95bae38 in_seq=0 out_seq=0 2024-04-25T13:54:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f5966ffd640 1 -- 172.21.15.94:0/1082451343 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fd69fb800 con 0x564fd66813a0 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f5966ffd640 10 client.20193 handle_command_reply: tid=0 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 client.20193 shutdown 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 msgr2=0x564fd6a9f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f597002ed40 tx=0x7f59700046c0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 shutdown_connections 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x564fd66813a0 0x564fd6b11e80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564fd66ed390 0x564fd6a9f880 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fd66cc6b0 0x564fd6a9e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f5966ffd640 0 client.20193 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 --2- 172.21.15.94:0/1082451343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fd66c44d0 0x564fd6a9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 >> 172.21.15.94:0/1082451343 conn(0x564fd66e14e0 msgr2=0x564fd6a8de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 shutdown_connections 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 1 -- 172.21.15.94:0/1082451343 wait complete. 2024-04-25T13:54:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.035+0000 7f59947d3740 20 client.20193 trim_cache size 0 max 16384 2024-04-25T13:54:55.041 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 msgr2=0x7f59740b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 0x7f59740b2080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59800a6c40 tx=0x7f5980075a90 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 msgr2=0x7f598819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f598000da90 tx=0x7f598000df50 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 shutdown_connections 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f59740afb70 0x7f59740b2080 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5988108930 0x7f598819f4e0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5988103c30 0x7f59881985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 --2- 172.21.15.94:0/2715480036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59881029f0 0x7f5988198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 >> 172.21.15.94:0/2715480036 conn(0x7f59880fdc00 msgr2=0x7f5988104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 shutdown_connections 2024-04-25T13:54:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.036+0000 7f598d849640 1 -- 172.21.15.94:0/2715480036 wait complete. 2024-04-25T13:54:55.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:54:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 -- 172.21.15.94:0/364043055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 msgr2=0x7fa124103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 --2- 172.21.15.94:0/364043055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124103df0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa114009900 tx=0x7fa11402a410 comp rx=0 tx=0).stop 2024-04-25T13:54:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 -- 172.21.15.94:0/364043055 shutdown_connections 2024-04-25T13:54:55.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 --2- 172.21.15.94:0/364043055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104480 0x7fa12410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 --2- 172.21.15.94:0/364043055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124103df0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 --2- 172.21.15.94:0/364043055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124102f80 0x7fa124103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 -- 172.21.15.94:0/364043055 >> 172.21.15.94:0/364043055 conn(0x7fa1240fd380 msgr2=0x7fa1240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 -- 172.21.15.94:0/364043055 shutdown_connections 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.160+0000 7fa12b372640 1 -- 172.21.15.94:0/364043055 wait complete. 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 Processor -- start 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 -- start start 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124102f80 0x7fa124199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104480 0x7fa1241a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410e5a0 con 0x7fa124103960 2024-04-25T13:54:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410e420 con 0x7fa124104480 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12b372640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410e720 con 0x7fa124102f80 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40464/0 (socket says 172.21.15.94:40464) 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 -- 172.21.15.94:0/2861537580 learned_addr learned my addr 172.21.15.94:0/2861537580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12910e640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124102f80 0x7fa124199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12990f640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104480 0x7fa1241a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 -- 172.21.15.94:0/2861537580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124102f80 msgr2=0x7fa124199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124102f80 0x7fa124199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 -- 172.21.15.94:0/2861537580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104480 msgr2=0x7fa1241a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104480 0x7fa1241a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 -- 172.21.15.94:0/2861537580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1241a0c60 con 0x7fa124103960 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.161+0000 7fa12890d640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa114031440 tx=0x7fa114003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.162+0000 7fa11a7fc640 1 -- 172.21.15.94:0/2861537580 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11403f070 con 0x7fa124103960 2024-04-25T13:54:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.162+0000 7fa11a7fc640 1 -- 172.21.15.94:0/2861537580 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa114031c50 con 0x7fa124103960 2024-04-25T13:54:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.162+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1241a0ef0 con 0x7fa124103960 2024-04-25T13:54:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.162+0000 7fa11a7fc640 1 -- 172.21.15.94:0/2861537580 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114035030 con 0x7fa124103960 2024-04-25T13:54:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.162+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241a1350 con 0x7fa124103960 2024-04-25T13:54:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.164+0000 7fa132226740 1 Processor -- start 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 -- start start 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa11a7fc640 1 -- 172.21.15.94:0/2861537580 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa114042070 con 0x7fa124103960 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aee7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aee8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b04ae49d80 con 0x55b04ab226b0 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b04ae03a20 con 0x55b04ab43390 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa132226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b04ae45d40 con 0x55b04ab1a4d0 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aee8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37088/0 (socket says 172.21.15.94:37088) 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 -- 172.21.15.94:0/1883236219 learned_addr learned my addr 172.21.15.94:0/1883236219 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aee8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40478/0 (socket says 172.21.15.94:40478) 2024-04-25T13:54:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12910e640 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aee7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 -- 172.21.15.94:0/1883236219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 msgr2=0x55b04aee7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aee7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 -- 172.21.15.94:0/1883236219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 msgr2=0x55b04aee8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aee8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 -- 172.21.15.94:0/1883236219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04ae0ec30 con 0x55b04ab43390 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa12990f640 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009fd0 tx=0x7fa11c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f542e0cc1e612cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.165+0000 7fa11a7fc640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 0x7fa1100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa10f7fe640 1 -- 172.21.15.94:0/1883236219 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004d20 con 0x55b04ab43390 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa12910e640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 0x7fa1100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa10f7fe640 1 -- 172.21.15.94:0/1883236219 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c004050 con 0x55b04ab43390 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa10f7fe640 1 -- 172.21.15.94:0/1883236219 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c005280 con 0x55b04ab43390 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa11a7fc640 1 -- 172.21.15.94:0/2861537580 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa1140ff2e0 con 0x7fa124103960 2024-04-25T13:54:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa132226740 1 -- 172.21.15.94:0/1883236219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 msgr2=0x55b04aee7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa12910e640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 0x7fa1100b2080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55b04aee7740 tx=0x7fa120005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.166+0000 7fa132226740 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009fd0 tx=0x7fa11c0118d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- 172.21.15.94:0/1883236219 shutdown_connections 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aee8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aee7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- 172.21.15.94:0/1883236219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aee7a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- 172.21.15.94:0/1883236219 >> 172.21.15.94:0/1883236219 conn(0x55b04ab374e0 msgr2=0x55b04aee1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- 172.21.15.94:0/1883236219 shutdown_connections 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- 172.21.15.94:0/1883236219 wait complete. 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- start start 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 Processor -- start 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab226b0 0x55b04aef1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aef8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b04aee9740 con 0x55b04ab1a4d0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b04add0dc0 con 0x55b04ab43390 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b04aee98c0 con 0x55b04ab226b0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.167+0000 7fa132226740 10 client.0 _ll_register_callbacks cb 0x55b04adec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40480/0 (socket says 172.21.15.94:40480) 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 -- 172.21.15.94:0/1487715097 learned_addr learned my addr 172.21.15.94:0/1487715097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12990f640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aef8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12890d640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab226b0 0x55b04aef1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 -- 172.21.15.94:0/1487715097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab226b0 msgr2=0x55b04aef1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab226b0 0x55b04aef1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 -- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 msgr2=0x55b04aef8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aef8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 -- 172.21.15.94:0/1487715097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04ab5b3f0 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12990f640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aef8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa12910e640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fa120013010 tx=0x7fa120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4128a8aca449734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120009e60 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa120005020 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c004050 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa120005220 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa132226740 10 client.20205 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.168+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b04ac257d0 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c004050 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa120027070 con 0x55b04ab1a4d0 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 10 client.20205 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 10 client.20205 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:54:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 20 client.20205 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04ab01f60 0x55b04aeff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 4 client.20205 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04ae57990 con 0x55b04ab01f60 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa12890d640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04ab01f60 0x55b04aeff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa12890d640 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04ab01f60 0x55b04aeff630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b7c59b213d9a98a server_cookie=8e1bf3ace1c85452 in_seq=0 out_seq=0 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.169+0000 7fa10cff9640 10 client.20205 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.170+0000 7fa10cff9640 1 -- 172.21.15.94:0/1487715097 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b04ae57990 con 0x55b04ab01f60 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.170+0000 7fa10cff9640 10 client.20205 handle_command_reply: tid=0 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.170+0000 7fa132226740 1 client.20205 shutdown 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 msgr2=0x55b04aef17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fa120013010 tx=0x7fa120004730 comp rx=0 tx=0).stop 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 shutdown_connections 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04ab01f60 0x55b04aeff630 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab43390 0x55b04aef8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa10cff9640 0 client.20205 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab226b0 0x55b04aef1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 --2- 172.21.15.94:0/1487715097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab1a4d0 0x55b04aef17f0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 >> 172.21.15.94:0/1487715097 conn(0x55b04ab374e0 msgr2=0x55b04aee3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 shutdown_connections 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 1 -- 172.21.15.94:0/1487715097 wait complete. 2024-04-25T13:54:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.171+0000 7fa132226740 20 client.20205 trim_cache size 0 max 16384 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 Processor -- start 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 -- start start 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab1a4d0 0x55b04aef1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 0x55b04af03de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.184+0000 7fa132226740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9250 con 0x55b04ab226b0 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa132226740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241aad90 con 0x55b04ab1a4d0 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa132226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9920 con 0x55b04ab43390 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40490/0 (socket says 172.21.15.94:40490) 2024-04-25T13:54:55.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 -- 172.21.15.94:0/4061839848 learned_addr learned my addr 172.21.15.94:0/4061839848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12910e640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab1a4d0 0x55b04aef1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12990f640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 0x55b04af03de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 -- 172.21.15.94:0/4061839848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 msgr2=0x55b04af03de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 0x55b04af03de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 -- 172.21.15.94:0/4061839848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab1a4d0 msgr2=0x55b04aef1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab1a4d0 0x55b04aef1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 -- 172.21.15.94:0/4061839848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04aeebfd0 con 0x55b04ab226b0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12990f640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 0x55b04af03de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa12890d640 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fa1140c4a40 tx=0x7fa1140ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cce3d02dd0a99c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa10effd640 1 -- 172.21.15.94:0/4061839848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114049040 con 0x55b04ab226b0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa10effd640 1 -- 172.21.15.94:0/4061839848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1140cee50 con 0x55b04ab226b0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.185+0000 7fa10effd640 1 -- 172.21.15.94:0/4061839848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140c77d0 con 0x55b04ab226b0 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- 172.21.15.94:0/4061839848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 msgr2=0x55b04aef8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fa1140c4a40 tx=0x7fa1140ea040 comp rx=0 tx=0).stop 2024-04-25T13:54:55.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- 172.21.15.94:0/4061839848 shutdown_connections 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab43390 0x55b04af03de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab226b0 0x55b04aef8c20 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 --2- 172.21.15.94:0/4061839848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab1a4d0 0x55b04aef1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- 172.21.15.94:0/4061839848 >> 172.21.15.94:0/4061839848 conn(0x55b04ab374e0 msgr2=0x55b04aef13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- 172.21.15.94:0/4061839848 shutdown_connections 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- 172.21.15.94:0/4061839848 wait complete. 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 -- start start 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.186+0000 7fa132226740 1 Processor -- start 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aef8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab226b0 0x55b04aef8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241aa6c0 con 0x55b04ab43390 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9ff0 con 0x55b04ab226b0 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a7230 con 0x55b04ab1a4d0 2024-04-25T13:54:55.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa132226740 10 client.0 _ll_register_callbacks cb 0x7fa120009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40504/0 (socket says 172.21.15.94:40504) 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 -- 172.21.15.94:0/3690633099 learned_addr learned my addr 172.21.15.94:0/3690633099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12910e640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aef8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12890d640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab226b0 0x55b04aef8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 -- 172.21.15.94:0/3690633099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 msgr2=0x55b04aef8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aef8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 -- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab226b0 msgr2=0x55b04aef8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab226b0 0x55b04aef8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.187+0000 7fa12990f640 1 -- 172.21.15.94:0/3690633099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04aeecf10 con 0x55b04ab43390 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa12990f640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663eb3e5242fe051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa10cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004d00 con 0x55b04ab43390 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fa11c004050 con 0x55b04ab43390 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1140cee50 con 0x55b04ab43390 2024-04-25T13:54:55.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa11c004250 con 0x55b04ab43390 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa132226740 10 client.20211 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b04af67330 con 0x55b04ab43390 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.188+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1140cee50 con 0x55b04ab43390 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa11c00f4e0 con 0x55b04ab43390 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 10 client.20211 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 10 client.20211 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 20 client.20211 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04aad73a0 0x55b04af69b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 4 client.20211 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b04ae51800 con 0x55b04aad73a0 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa12910e640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04aad73a0 0x55b04af69b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa10cff9640 10 client.20211 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa12910e640 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04aad73a0 0x55b04af69b90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76fc4e8a0121142f server_cookie=cbf889382412b3e6 in_seq=0 out_seq=0 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa10cff9640 1 -- 172.21.15.94:0/3690633099 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b04ae51800 con 0x55b04aad73a0 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa10cff9640 10 client.20211 handle_command_reply: tid=0 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.189+0000 7fa132226740 1 client.20211 shutdown 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 msgr2=0x55b04af65c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006010 tx=0x7fa11c004900 comp rx=0 tx=0).stop 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 shutdown_connections 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b04aad73a0 0x55b04af69b90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b04ab43390 0x55b04af65c20 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b04ab226b0 0x55b04aef8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 --2- 172.21.15.94:0/3690633099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b04ab1a4d0 0x55b04aef8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 >> 172.21.15.94:0/3690633099 conn(0x55b04ab374e0 msgr2=0x55b04aee3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa10cff9640 0 client.20211 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 shutdown_connections 2024-04-25T13:54:55.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 1 -- 172.21.15.94:0/3690633099 wait complete. 2024-04-25T13:54:55.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa132226740 20 client.20211 trim_cache size 0 max 16384 2024-04-25T13:54:55.196 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 msgr2=0x7fa1100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 0x7fa1100b2080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55b04aee7740 tx=0x7fa120005f90 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 msgr2=0x7fa124199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa114031440 tx=0x7fa114003c10 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 shutdown_connections 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afb70 0x7fa1100b2080 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104480 0x7fa1241a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124103960 0x7fa124199660 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 --2- 172.21.15.94:0/2861537580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124102f80 0x7fa124199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.190+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 >> 172.21.15.94:0/2861537580 conn(0x7fa1240fd380 msgr2=0x7fa124104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.191+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 shutdown_connections 2024-04-25T13:54:55.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.191+0000 7fa12b372640 1 -- 172.21.15.94:0/2861537580 wait complete. 2024-04-25T13:54:55.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 -- 172.21.15.94:0/2306262560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648104d30 msgr2=0x7f9648105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 --2- 172.21.15.94:0/2306262560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648104d30 0x7f9648105140 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f9638009900 tx=0x7f963802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 -- 172.21.15.94:0/2306262560 shutdown_connections 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 --2- 172.21.15.94:0/2306262560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648109a30 0x7f964810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 --2- 172.21.15.94:0/2306262560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648104d30 0x7f9648105140 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 --2- 172.21.15.94:0/2306262560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f96481047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 -- 172.21.15.94:0/2306262560 >> 172.21.15.94:0/2306262560 conn(0x7f96480fece0 msgr2=0x7f9648101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 -- 172.21.15.94:0/2306262560 shutdown_connections 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.315+0000 7f9650514640 1 -- 172.21.15.94:0/2306262560 wait complete. 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 Processor -- start 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 -- start start 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648104d30 0x7f964819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 0x7f96481a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f964810f5f0 con 0x7f9648109a30 2024-04-25T13:54:55.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f964810f470 con 0x7f9648103af0 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f9650514640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f964810f770 con 0x7f9648104d30 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964e2b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 0x7f96481a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964e2b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37136/0 (socket says 172.21.15.94:37136) 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964e2b0640 1 -- 172.21.15.94:0/3490642330 learned_addr learned my addr 172.21.15.94:0/3490642330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 0x7f96481a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40524/0 (socket says 172.21.15.94:40524) 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.316+0000 7f964daaf640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648104d30 0x7f964819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 -- 172.21.15.94:0/3490642330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648104d30 msgr2=0x7f964819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648104d30 0x7f964819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 -- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 msgr2=0x7f96481a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 0x7f96481a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 -- 172.21.15.94:0/3490642330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96481a1d30 con 0x7f9648103af0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f964e2b0640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f963f7fe640 1 -- 172.21.15.94:0/3490642330 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400ccc0 con 0x7f9648103af0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96481a1fc0 con 0x7f9648103af0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f963f7fe640 1 -- 172.21.15.94:0/3490642330 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96440035f0 con 0x7f9648103af0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f963f7fe640 1 -- 172.21.15.94:0/3490642330 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400ce60 con 0x7f9648103af0 2024-04-25T13:54:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.317+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96481a2480 con 0x7f9648103af0 2024-04-25T13:54:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 Processor -- start 2024-04-25T13:54:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 -- start start 2024-04-25T13:54:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f963f7fe640 1 -- 172.21.15.94:0/3490642330 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9644003aa0 con 0x7f9648103af0 2024-04-25T13:54:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafa6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabe36b0 0x561dfafa7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dfaf0ad80 con 0x561dfac04390 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dfaec4a20 con 0x561dfabdb4d0 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f96573c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561dfaf06d40 con 0x561dfabe36b0 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964e2b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafa6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40536/0 (socket says 172.21.15.94:40536) 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964eab1640 1 -- 172.21.15.94:0/1959363407 learned_addr learned my addr 172.21.15.94:0/1959363407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964daaf640 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabe36b0 0x561dfafa7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.320+0000 7f964eab1640 1 -- 172.21.15.94:0/1959363407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabe36b0 msgr2=0x561dfafa7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964eab1640 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabe36b0 0x561dfafa7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964eab1640 1 -- 172.21.15.94:0/1959363407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 msgr2=0x561dfafa6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964eab1640 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafa6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964eab1640 1 -- 172.21.15.94:0/1959363407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfaecfc30 con 0x561dfac04390 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964e2b0640 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafa6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964eab1640 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9640009fd0 tx=0x7f96400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539fd39ef57ee556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f963f7fe640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 0x7f96340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f963cbfe640 1 -- 172.21.15.94:0/1959363407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640004d20 con 0x561dfac04390 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f963cbfe640 1 -- 172.21.15.94:0/1959363407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640004050 con 0x561dfac04390 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f963cbfe640 1 -- 172.21.15.94:0/1959363407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640005260 con 0x561dfac04390 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f964e2b0640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 0x7f96340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.321+0000 7f963f7fe640 1 -- 172.21.15.94:0/3490642330 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f96440df030 con 0x7f9648103af0 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- 172.21.15.94:0/1959363407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 msgr2=0x561dfafa6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f964e2b0640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 0x7f96340b2000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x561dfafa6530 tx=0x7f96440e2b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f9640009fd0 tx=0x7f96400118d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- 172.21.15.94:0/1959363407 shutdown_connections 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabe36b0 0x561dfafa7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafa6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 --2- 172.21.15.94:0/1959363407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfac04390 0x561dfafa6810 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- 172.21.15.94:0/1959363407 >> 172.21.15.94:0/1959363407 conn(0x561dfabf84e0 msgr2=0x561dfafa2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- 172.21.15.94:0/1959363407 shutdown_connections 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- 172.21.15.94:0/1959363407 wait complete. 2024-04-25T13:54:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 -- start start 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.322+0000 7f96573c8740 1 Processor -- start 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabdb4d0 0x561dfafb0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 0x561dfafb79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dfafa8530 con 0x561dfabe36b0 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dfae91dc0 con 0x561dfac04390 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561dfafa86b0 con 0x561dfabdb4d0 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 10 client.0 _ll_register_callbacks cb 0x561dfaead600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40552/0 (socket says 172.21.15.94:40552) 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 -- 172.21.15.94:0/2430381295 learned_addr learned my addr 172.21.15.94:0/2430381295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964eab1640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabdb4d0 0x561dfafb0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964eab1640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 0x561dfafb79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 -- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabdb4d0 msgr2=0x561dfafb0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabdb4d0 0x561dfafb0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 -- 172.21.15.94:0/2430381295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 msgr2=0x561dfafb79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 0x561dfafb79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 -- 172.21.15.94:0/2430381295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfac1c3f0 con 0x561dfabe36b0 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964eab1640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 0x561dfafb79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f964daaf640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9638005940 tx=0x7f96380317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a599b13d84362a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638035040 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9638004240 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.323+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9640004050 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9638004440 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 10 client.20223 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561dface67d0 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640004050 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9638004aa0 con 0x561dfabe36b0 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 10 client.20223 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 10 client.20223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:54:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 20 client.20223 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfabc2f60 0x561dfafbe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 4 client.20223 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dfaf18990 con 0x561dfabc2f60 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.324+0000 7f964eab1640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfabc2f60 0x561dfafbe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.325+0000 7f9631ffb640 10 client.20223 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.325+0000 7f964eab1640 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfabc2f60 0x561dfafbe1e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d329e07977fc78b6 server_cookie=8df6f7736a05ff0d in_seq=0 out_seq=0 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.325+0000 7f9631ffb640 1 -- 172.21.15.94:0/2430381295 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dfaf18990 con 0x561dfabc2f60 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.325+0000 7f9631ffb640 10 client.20223 handle_command_reply: tid=0 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.325+0000 7f96573c8740 1 client.20223 shutdown 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 msgr2=0x561dfafb0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9638005940 tx=0x7f96380317d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 shutdown_connections 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfabc2f60 0x561dfafbe1e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfac04390 0x561dfafb79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f9631ffb640 0 client.20223 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafb0a40 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 --2- 172.21.15.94:0/2430381295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabdb4d0 0x561dfafb0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 >> 172.21.15.94:0/2430381295 conn(0x561dfabf84e0 msgr2=0x561dfafa1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 shutdown_connections 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 1 -- 172.21.15.94:0/2430381295 wait complete. 2024-04-25T13:54:55.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.326+0000 7f96573c8740 20 client.20223 trim_cache size 0 max 16384 2024-04-25T13:54:55.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 Processor -- start 2024-04-25T13:54:55.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 -- start start 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafa5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafb78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96481ab5f0 con 0x561dfabe36b0 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96481ad130 con 0x561dfabdb4d0 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f96573c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96481abcc0 con 0x561dfabc2f60 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f964daaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafb78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f964eab1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40564/0 (socket says 172.21.15.94:40564) 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f964eab1640 1 -- 172.21.15.94:0/1952500460 learned_addr learned my addr 172.21.15.94:0/1952500460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.340+0000 7f964eab1640 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafa5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 -- 172.21.15.94:0/1952500460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 msgr2=0x561dfafa5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafa5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 -- 172.21.15.94:0/1952500460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 msgr2=0x561dfafb78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafb78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 -- 172.21.15.94:0/1952500460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfafabc80 con 0x561dfabe36b0 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964daaf640 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafb78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f964eab1640 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9640014040 tx=0x7f96400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5082b5f9a8e89fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f9633fff640 1 -- 172.21.15.94:0/1952500460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640003dc0 con 0x561dfabe36b0 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f9633fff640 1 -- 172.21.15.94:0/1952500460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640004370 con 0x561dfabe36b0 2024-04-25T13:54:55.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.341+0000 7f9633fff640 1 -- 172.21.15.94:0/1952500460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400053e0 con 0x561dfabe36b0 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- 172.21.15.94:0/1952500460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 msgr2=0x561dfafc39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9640014040 tx=0x7f96400118d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- 172.21.15.94:0/1952500460 shutdown_connections 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabe36b0 0x561dfafc39a0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabdb4d0 0x561dfafb78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 --2- 172.21.15.94:0/1952500460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafa5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- 172.21.15.94:0/1952500460 >> 172.21.15.94:0/1952500460 conn(0x561dfabf84e0 msgr2=0x561dfafa4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- 172.21.15.94:0/1952500460 shutdown_connections 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- 172.21.15.94:0/1952500460 wait complete. 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 -- start start 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.342+0000 7f96573c8740 1 Processor -- start 2024-04-25T13:54:55.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafb5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabe36b0 0x561dfafb6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96481aca60 con 0x561dfabdb4d0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96481ac390 con 0x561dfabe36b0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96481a95d0 con 0x561dfabc2f60 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 10 client.0 _ll_register_callbacks cb 0x7f96400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40576/0 (socket says 172.21.15.94:40576) 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 -- 172.21.15.94:0/3501726898 learned_addr learned my addr 172.21.15.94:0/3501726898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964eab1640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafb5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964eab1640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabe36b0 0x561dfafb6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 -- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 msgr2=0x561dfafb5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafb5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 -- 172.21.15.94:0/3501726898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabe36b0 msgr2=0x561dfafb6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabe36b0 0x561dfafb6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 -- 172.21.15.94:0/3501726898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfafb7e00 con 0x561dfabdb4d0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f964daaf640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9638000c00 tx=0x7f96380035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb799f52edb2547c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638035040 con 0x561dfabdb4d0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f96380046c0 con 0x561dfabdb4d0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.343+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9640004370 con 0x561dfabdb4d0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96380048c0 con 0x561dfabdb4d0 2024-04-25T13:54:55.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 10 client.20229 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561dfafb7bc0 con 0x561dfabdb4d0 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640004370 con 0x561dfabdb4d0 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9638003a40 con 0x561dfabdb4d0 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 10 client.20229 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 10 client.20229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 20 client.20229 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfac04390 0x561dfb028cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 4 client.20229 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dfaf12800 con 0x561dfac04390 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.344+0000 7f964eab1640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfac04390 0x561dfb028cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f9631ffb640 10 client.20229 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f964eab1640 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfac04390 0x561dfb028cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa6a16fcac74c15 server_cookie=27b912f6b0344acc in_seq=0 out_seq=0 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f9631ffb640 1 -- 172.21.15.94:0/3501726898 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dfaf12800 con 0x561dfac04390 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f9631ffb640 10 client.20229 handle_command_reply: tid=0 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 client.20229 shutdown 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 msgr2=0x561dfafb5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9638000c00 tx=0x7f96380035f0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 shutdown_connections 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x561dfac04390 0x561dfb028cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dfabe36b0 0x561dfafb6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dfabdb4d0 0x561dfafb5640 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 --2- 172.21.15.94:0/3501726898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561dfabc2f60 0x561dfafb5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f9631ffb640 0 client.20229 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:54:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 >> 172.21.15.94:0/3501726898 conn(0x561dfabf84e0 msgr2=0x561dfafa5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 shutdown_connections 2024-04-25T13:54:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 1 -- 172.21.15.94:0/3501726898 wait complete. 2024-04-25T13:54:55.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.345+0000 7f96573c8740 20 client.20229 trim_cache size 0 max 16384 2024-04-25T13:54:55.353 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:55.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 msgr2=0x7f96340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 0x7f96340b2000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x561dfafa6530 tx=0x7f96440e2b10 comp rx=0 tx=0).stop 2024-04-25T13:54:55.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 msgr2=0x7f964819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 shutdown_connections 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96340afaf0 0x7f96340b2000 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9648109a30 0x7f96481a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9648104d30 0x7f964819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 --2- 172.21.15.94:0/3490642330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9648103af0 0x7f964819a1f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 >> 172.21.15.94:0/3490642330 conn(0x7f96480fece0 msgr2=0x7f9648105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 shutdown_connections 2024-04-25T13:54:55.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.346+0000 7f9650514640 1 -- 172.21.15.94:0/3490642330 wait complete. 2024-04-25T13:54:55.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:54:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 -- 172.21.15.94:0/2203008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 msgr2=0x7f9710103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 --2- 172.21.15.94:0/2203008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710103df0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f9704009900 tx=0x7f970402a410 comp rx=0 tx=0).stop 2024-04-25T13:54:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 -- 172.21.15.94:0/2203008886 shutdown_connections 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 --2- 172.21.15.94:0/2203008886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 0x7f971010abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 --2- 172.21.15.94:0/2203008886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710103df0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 --2- 172.21.15.94:0/2203008886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 0x7f97101033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 -- 172.21.15.94:0/2203008886 >> 172.21.15.94:0/2203008886 conn(0x7f97100fd380 msgr2=0x7f97100ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 -- 172.21.15.94:0/2203008886 shutdown_connections 2024-04-25T13:54:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.473+0000 7f9714cb6640 1 -- 172.21.15.94:0/2203008886 wait complete. 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 Processor -- start 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 -- start start 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 0x7f9710199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 0x7f97101a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f971010e540 con 0x7f9710103980 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f971010e3c0 con 0x7f9710104480 2024-04-25T13:54:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f9714cb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f971010e6c0 con 0x7f9710102fa0 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.474+0000 7f970dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40594/0 (socket says 172.21.15.94:40594) 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 -- 172.21.15.94:0/1654574721 learned_addr learned my addr 172.21.15.94:0/1654574721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970ed9d640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 0x7f97101a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970e59c640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 0x7f9710199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 -- 172.21.15.94:0/1654574721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 msgr2=0x7f9710199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 0x7f9710199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 -- 172.21.15.94:0/1654574721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 msgr2=0x7f97101a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 0x7f97101a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 -- 172.21.15.94:0/1654574721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97101018d0 con 0x7f9710103980 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f97040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f96ff7fe640 1 -- 172.21.15.94:0/1654574721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403f030 con 0x7f9710103980 2024-04-25T13:54:55.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9710101b60 con 0x7f9710103980 2024-04-25T13:54:55.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101020a0 con 0x7f9710103980 2024-04-25T13:54:55.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f96ff7fe640 1 -- 172.21.15.94:0/1654574721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9704031910 con 0x7f9710103980 2024-04-25T13:54:55.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.475+0000 7f96ff7fe640 1 -- 172.21.15.94:0/1654574721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403f030 con 0x7f9710103980 2024-04-25T13:54:55.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 Processor -- start 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 -- start start 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f96ff7fe640 1 -- 172.21.15.94:0/1654574721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9704042030 con 0x7f9710103980 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561426d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561427a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55956138ad80 con 0x559561084390 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559561344a20 con 0x5595610636b0 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.478+0000 7f971bb6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559561386d40 con 0x55956105b4d0 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40608/0 (socket says 172.21.15.94:40608) 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970ed9d640 1 -- 172.21.15.94:0/260785278 learned_addr learned my addr 172.21.15.94:0/260785278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561426d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970dd9b640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561427a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 -- 172.21.15.94:0/260785278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 msgr2=0x559561427a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561427a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 -- 172.21.15.94:0/260785278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 msgr2=0x559561426810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 -- 172.21.15.94:0/260785278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956134fc30 con 0x55956105b4d0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970ed9d640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970e59c640 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561426d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f971006b4a0 tx=0x7f9700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e664776f022c118c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f96ff7fe640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 0x7f96f80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f96fcff9640 1 -- 172.21.15.94:0/260785278 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700004230 con 0x55956105b4d0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f970ed9d640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 0x7f96f80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f96fcff9640 1 -- 172.21.15.94:0/260785278 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9700009e10 con 0x55956105b4d0 2024-04-25T13:54:55.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f96fcff9640 1 -- 172.21.15.94:0/260785278 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700019680 con 0x55956105b4d0 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.479+0000 7f96ff7fe640 1 -- 172.21.15.94:0/1654574721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f97040fee30 con 0x7f9710103980 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 -- 172.21.15.94:0/260785278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 msgr2=0x559561426d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f970ed9d640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 0x7f96f80b2200 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x559561427860 tx=0x7f9708009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561426d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f971006b4a0 tx=0x7f9700011810 comp rx=0 tx=0).stop 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 -- 172.21.15.94:0/260785278 shutdown_connections 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561427a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561426d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 --2- 172.21.15.94:0/260785278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559561084390 0x559561426810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 -- 172.21.15.94:0/260785278 >> 172.21.15.94:0/260785278 conn(0x5595610784e0 msgr2=0x559561422020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 -- 172.21.15.94:0/260785278 shutdown_connections 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.480+0000 7f971bb6a740 1 -- 172.21.15.94:0/260785278 wait complete. 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 -- start start 2024-04-25T13:54:55.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 Processor -- start 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595610636b0 0x559561430a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x5595614379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559561428530 con 0x55956105b4d0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559561311dc0 con 0x559561084390 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595614286b0 con 0x5595610636b0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f971bb6a740 10 client.0 _ll_register_callbacks cb 0x55956132d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40610/0 (socket says 172.21.15.94:40610) 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 -- 172.21.15.94:0/2590860456 learned_addr learned my addr 172.21.15.94:0/2590860456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970dd9b640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595610636b0 0x559561430a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970ed9d640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x5595614379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 -- 172.21.15.94:0/2590860456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595610636b0 msgr2=0x559561430a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595610636b0 0x559561430a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 -- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 msgr2=0x5595614379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x5595614379b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.481+0000 7f970e59c640 1 -- 172.21.15.94:0/2590860456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956109c3f0 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f970e59c640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9700011d10 tx=0x7f9700003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8963d8a24eb9dd4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700028070 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9700005040 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9700009e10 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9700034050 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 10 client.20241 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5595611667d0 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9700009e10 con 0x55956105b4d0 2024-04-25T13:54:55.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f97000049f0 con 0x55956105b4d0 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 10 client.20241 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 10 client.20241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 20 client.20241 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559561042f60 0x55956143dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 4 client.20241 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.482+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559561398990 con 0x559561042f60 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.483+0000 7f970dd9b640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559561042f60 0x55956143dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.483+0000 7f96f5ffb640 10 client.20241 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.483+0000 7f970dd9b640 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559561042f60 0x55956143dfb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ad31977270f9b57 server_cookie=bdbb45b5acd42c4 in_seq=0 out_seq=0 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f96f5ffb640 1 -- 172.21.15.94:0/2590860456 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559561398990 con 0x559561042f60 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f96f5ffb640 10 client.20241 handle_command_reply: tid=0 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 client.20241 shutdown 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 msgr2=0x559561430510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9700011d10 tx=0x7f9700003820 comp rx=0 tx=0).stop 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 shutdown_connections 2024-04-25T13:54:55.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559561042f60 0x55956143dfb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x5595614379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595610636b0 0x559561430a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f96f5ffb640 0 client.20241 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2590860456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561430510 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 >> 172.21.15.94:0/2590860456 conn(0x5595610784e0 msgr2=0x559561424f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 shutdown_connections 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 1 -- 172.21.15.94:0/2590860456 wait complete. 2024-04-25T13:54:55.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.484+0000 7f971bb6a740 20 client.20241 trim_cache size 0 max 16384 2024-04-25T13:54:55.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 Processor -- start 2024-04-25T13:54:55.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 -- start start 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561425a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559561084390 0x5595614378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ab540 con 0x55956105b4d0 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ad080 con 0x5595610636b0 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f971bb6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97101abc10 con 0x559561084390 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40618/0 (socket says 172.21.15.94:40618) 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 -- 172.21.15.94:0/2805885321 learned_addr learned my addr 172.21.15.94:0/2805885321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970dd9b640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561425a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970ed9d640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559561084390 0x5595614378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 -- 172.21.15.94:0/2805885321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559561084390 msgr2=0x5595614378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559561084390 0x5595614378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 -- 172.21.15.94:0/2805885321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 msgr2=0x559561425a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561425a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970e59c640 1 -- 172.21.15.94:0/2805885321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559561437e10 con 0x55956105b4d0 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.498+0000 7f970dd9b640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561425a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.499+0000 7f970e59c640 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f97000058c0 tx=0x7f9700005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff329972ac98de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.499+0000 7f96f7fff640 1 -- 172.21.15.94:0/2805885321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970003d070 con 0x55956105b4d0 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.499+0000 7f96f7fff640 1 -- 172.21.15.94:0/2805885321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9700031030 con 0x55956105b4d0 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.499+0000 7f96f7fff640 1 -- 172.21.15.94:0/2805885321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700028070 con 0x55956105b4d0 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- 172.21.15.94:0/2805885321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 msgr2=0x559561425530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f97000058c0 tx=0x7f9700005f10 comp rx=0 tx=0).stop 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- 172.21.15.94:0/2805885321 shutdown_connections 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559561084390 0x5595614378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595610636b0 0x559561425a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- 172.21.15.94:0/2805885321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55956105b4d0 0x559561425530 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- 172.21.15.94:0/2805885321 >> 172.21.15.94:0/2805885321 conn(0x5595610784e0 msgr2=0x559561424e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- 172.21.15.94:0/2805885321 shutdown_connections 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- 172.21.15.94:0/2805885321 wait complete. 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- start start 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 Processor -- start 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561434a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x559561435e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ac9b0 con 0x5595610636b0 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97101ac2e0 con 0x559561084390 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97101a9520 con 0x55956105b4d0 2024-04-25T13:54:55.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f971bb6a740 10 client.0 _ll_register_callbacks cb 0x7f9708009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f970dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f970dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40634/0 (socket says 172.21.15.94:40634) 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f970dd9b640 1 -- 172.21.15.94:0/1990361025 learned_addr learned my addr 172.21.15.94:0/1990361025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f970e59c640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561434a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.500+0000 7f970ed9d640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x559561435e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 -- 172.21.15.94:0/1990361025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 msgr2=0x559561434a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561434a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 -- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 msgr2=0x559561435e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x559561435e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 -- 172.21.15.94:0/1990361025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595614363e0 con 0x5595610636b0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f970dd9b640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f97040c7290 tx=0x7f97041026d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbc2e421a12cc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704035040 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f97040c7560 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9700031030 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9704049040 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 10 client.20247 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559561437730 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9700031030 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f97040cb170 con 0x5595610636b0 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 10 client.20247 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 10 client.20247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 20 client.20247 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5595610183a0 0x5595614a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 4 client.20247 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:55.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.501+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559561392800 con 0x5595610183a0 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f970e59c640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5595610183a0 0x5595614a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f96f5ffb640 10 client.20247 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f970e59c640 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5595610183a0 0x5595614a8690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=750ccd9ee2ed8d45 server_cookie=8f6cad9f92452d44 in_seq=0 out_seq=0 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f96f5ffb640 1 -- 172.21.15.94:0/1990361025 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559561392800 con 0x5595610183a0 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f96f5ffb640 10 client.20247 handle_command_reply: tid=0 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 client.20247 shutdown 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 msgr2=0x559561434fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f97040c7290 tx=0x7f97041026d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 shutdown_connections 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5595610183a0 0x5595614a8690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559561084390 0x559561435e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f96f5ffb640 0 client.20247 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595610636b0 0x559561434fb0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.502+0000 7f971bb6a740 1 --2- 172.21.15.94:0/1990361025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55956105b4d0 0x559561434a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 >> 172.21.15.94:0/1990361025 conn(0x5595610784e0 msgr2=0x5595614255b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 shutdown_connections 2024-04-25T13:54:55.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f971bb6a740 1 -- 172.21.15.94:0/1990361025 wait complete. 2024-04-25T13:54:55.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f971bb6a740 20 client.20247 trim_cache size 0 max 16384 2024-04-25T13:54:55.509 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 msgr2=0x7f96f80b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 0x7f96f80b2200 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x559561427860 tx=0x7f9708009500 comp rx=0 tx=0).stop 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 msgr2=0x7f9710199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f97040042f0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 shutdown_connections 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96f80afcf0 0x7f96f80b2200 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9710104480 0x7f97101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9710103980 0x7f9710199690 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 --2- 172.21.15.94:0/1654574721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9710102fa0 0x7f9710199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 >> 172.21.15.94:0/1654574721 conn(0x7f97100fd380 msgr2=0x7f9710104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 shutdown_connections 2024-04-25T13:54:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.503+0000 7f9714cb6640 1 -- 172.21.15.94:0/1654574721 wait complete. 2024-04-25T13:54:55.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:54:55.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 -- 172.21.15.94:0/177075278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31781050a0 msgr2=0x7f31781054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 --2- 172.21.15.94:0/177075278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31781050a0 0x7f31781054b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3168009900 tx=0x7f316802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 -- 172.21.15.94:0/177075278 shutdown_connections 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 --2- 172.21.15.94:0/177075278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 --2- 172.21.15.94:0/177075278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3178101750 0x7f3178101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 --2- 172.21.15.94:0/177075278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31781050a0 0x7f31781054b0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.628+0000 7f317cd14640 1 -- 172.21.15.94:0/177075278 >> 172.21.15.94:0/177075278 conn(0x7f31780fd350 msgr2=0x7f31780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.629+0000 7f317cd14640 1 -- 172.21.15.94:0/177075278 shutdown_connections 2024-04-25T13:54:55.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.629+0000 7f317cd14640 1 -- 172.21.15.94:0/177075278 wait complete. 2024-04-25T13:54:55.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.629+0000 7f317cd14640 1 Processor -- start 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.629+0000 7f317cd14640 1 -- start start 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31781050a0 0x7f31781a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f317810f610 con 0x7f3178101750 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f317810f490 con 0x7f31781050a0 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f317810f790 con 0x7f3178102250 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40652/0 (socket says 172.21.15.94:40652) 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 -- 172.21.15.94:0/21938003 learned_addr learned my addr 172.21.15.94:0/21938003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f3176d9d640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31781050a0 0x7f31781a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f3175d9b640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 -- 172.21.15.94:0/21938003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 msgr2=0x7f317819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 -- 172.21.15.94:0/21938003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31781050a0 msgr2=0x7f31781a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31781050a0 0x7f31781a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 -- 172.21.15.94:0/21938003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31781a1d50 con 0x7f3178101750 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f3175d9b640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317659c640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f3168004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f31677fe640 1 -- 172.21.15.94:0/21938003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316803f030 con 0x7f3178101750 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f31677fe640 1 -- 172.21.15.94:0/21938003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3168031910 con 0x7f3178101750 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31781a1fe0 con 0x7f3178101750 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f31677fe640 1 -- 172.21.15.94:0/21938003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168035070 con 0x7f3178101750 2024-04-25T13:54:55.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.630+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31781a2440 con 0x7f3178101750 2024-04-25T13:54:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 Processor -- start 2024-04-25T13:54:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 -- start start 2024-04-25T13:54:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f31677fe640 1 -- 172.21.15.94:0/21938003 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3168042030 con 0x7f3178101750 2024-04-25T13:54:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefd7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefd8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564bcef3bd80 con 0x564bcec35390 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564bceef5a20 con 0x564bcec0c4d0 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3183bc9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564bcef37d40 con 0x564bcec146b0 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40660/0 (socket says 172.21.15.94:40660) 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3176d9d640 1 -- 172.21.15.94:0/1356381593 learned_addr learned my addr 172.21.15.94:0/1356381593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f317659c640 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefd7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.633+0000 7f3175d9b640 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefd8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 -- 172.21.15.94:0/1356381593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 msgr2=0x564bcefd8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefd8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 -- 172.21.15.94:0/1356381593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 msgr2=0x564bcefd7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefd7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 -- 172.21.15.94:0/1356381593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bcef00c30 con 0x564bcec35390 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3176d9d640 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3170009fd0 tx=0x7f31700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9069910643bc9f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f31677fe640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 0x7f31600b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f317659c640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 0x7f31600b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3164ff9640 1 -- 172.21.15.94:0/1356381593 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170004d20 con 0x564bcec35390 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3164ff9640 1 -- 172.21.15.94:0/1356381593 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3170004050 con 0x564bcec35390 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f3164ff9640 1 -- 172.21.15.94:0/1356381593 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170005280 con 0x564bcec35390 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.634+0000 7f31677fe640 1 -- 172.21.15.94:0/21938003 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f31680fefb0 con 0x7f3178101750 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 -- 172.21.15.94:0/1356381593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 msgr2=0x564bcefd7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f317659c640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 0x7f31600b2080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x564bcefd7530 tx=0x7f31680b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3170009fd0 tx=0x7f31700118d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 -- 172.21.15.94:0/1356381593 shutdown_connections 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefd8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefd7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 --2- 172.21.15.94:0/1356381593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec35390 0x564bcefd7810 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 -- 172.21.15.94:0/1356381593 >> 172.21.15.94:0/1356381593 conn(0x564bcec294e0 msgr2=0x564bcefd3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 -- 172.21.15.94:0/1356381593 shutdown_connections 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.635+0000 7f3183bc9740 1 -- 172.21.15.94:0/1356381593 wait complete. 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 -- start start 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 Processor -- start 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec35390 0x564bcefe89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564bcefd9530 con 0x564bcec146b0 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564bceec2dc0 con 0x564bcec35390 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564bcefd96b0 con 0x564bcec0c4d0 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3183bc9740 10 client.0 _ll_register_callbacks cb 0x564bceede600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40662/0 (socket says 172.21.15.94:40662) 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 -- 172.21.15.94:0/544071856 learned_addr learned my addr 172.21.15.94:0/544071856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3176d9d640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3176d9d640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec35390 0x564bcefe89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 -- 172.21.15.94:0/544071856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 msgr2=0x564bcefe1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 -- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec35390 msgr2=0x564bcefe89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec35390 0x564bcefe89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 -- 172.21.15.94:0/544071856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bcec4d3f0 con 0x564bcec146b0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f3175d9b640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f316c012700 tx=0x7f316c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32005a7aa2678d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.636+0000 7f315dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c004260 con 0x564bcec146b0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f316c004400 con 0x564bcec146b0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3170004050 con 0x564bcec146b0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f316c0119a0 con 0x564bcec146b0 2024-04-25T13:54:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 10 client.20259 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564bced177d0 con 0x564bcec146b0 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3170004050 con 0x564bcec146b0 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f316c00f440 con 0x564bcec146b0 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 10 client.20259 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 10 client.20259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 20 client.20259 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcebf3f60 0x564bcefef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 4 client.20259 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bcef49990 con 0x564bcebf3f60 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.637+0000 7f3176d9d640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcebf3f60 0x564bcefef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.638+0000 7f315dffb640 10 client.20259 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:55.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.638+0000 7f3176d9d640 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcebf3f60 0x564bcefef220 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c509f5bd8da45be1 server_cookie=8de74e84e20d1e8c in_seq=0 out_seq=0 2024-04-25T13:54:55.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f315dffb640 1 -- 172.21.15.94:0/544071856 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bcef49990 con 0x564bcebf3f60 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f315dffb640 10 client.20259 handle_command_reply: tid=0 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 client.20259 shutdown 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 msgr2=0x564bcefe1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f316c012700 tx=0x7f316c012bc0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 shutdown_connections 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcebf3f60 0x564bcefef220 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec35390 0x564bcefe89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe1a40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f315dffb640 0 client.20259 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 --2- 172.21.15.94:0/544071856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 >> 172.21.15.94:0/544071856 conn(0x564bcec294e0 msgr2=0x564bcefd2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 shutdown_connections 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 1 -- 172.21.15.94:0/544071856 wait complete. 2024-04-25T13:54:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.640+0000 7f3183bc9740 20 client.20259 trim_cache size 0 max 16384 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 Processor -- start 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 -- start start 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefdc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefdc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31781ab4c0 con 0x564bcebf3f60 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31781ad000 con 0x564bcec0c4d0 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3183bc9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31781abb90 con 0x564bcec146b0 2024-04-25T13:54:55.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40676/0 (socket says 172.21.15.94:40676) 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 -- 172.21.15.94:0/2731820635 learned_addr learned my addr 172.21.15.94:0/2731820635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3175d9b640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefdc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefdc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 -- 172.21.15.94:0/2731820635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 msgr2=0x564bcefdc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefdc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 -- 172.21.15.94:0/2731820635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 msgr2=0x564bcefdc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefdc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3176d9d640 1 -- 172.21.15.94:0/2731820635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bcefdf5f0 con 0x564bcebf3f60 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.654+0000 7f3175d9b640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.655+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3170009de0 tx=0x7f31700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec35bf65c8819f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.655+0000 7f315ffff640 1 -- 172.21.15.94:0/2731820635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170004280 con 0x564bcebf3f60 2024-04-25T13:54:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.655+0000 7f315ffff640 1 -- 172.21.15.94:0/2731820635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317002c030 con 0x564bcebf3f60 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.655+0000 7f315ffff640 1 -- 172.21.15.94:0/2731820635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170005330 con 0x564bcebf3f60 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- 172.21.15.94:0/2731820635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 msgr2=0x564bcefcdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3170009de0 tx=0x7f31700118d0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- 172.21.15.94:0/2731820635 shutdown_connections 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec146b0 0x564bcefdc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcec0c4d0 0x564bcefdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2731820635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcebf3f60 0x564bcefcdd30 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- 172.21.15.94:0/2731820635 >> 172.21.15.94:0/2731820635 conn(0x564bcec294e0 msgr2=0x564bcefe1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- 172.21.15.94:0/2731820635 shutdown_connections 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- 172.21.15.94:0/2731820635 wait complete. 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- start start 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 Processor -- start 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcebf3f60 0x564bcefe5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31781ac930 con 0x564bcec146b0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31781ac260 con 0x564bcebf3f60 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31781a94a0 con 0x564bcec0c4d0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3183bc9740 10 client.0 _ll_register_callbacks cb 0x7f316c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3176d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40690/0 (socket says 172.21.15.94:40690) 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3176d9d640 1 -- 172.21.15.94:0/2330314427 learned_addr learned my addr 172.21.15.94:0/2330314427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.656+0000 7f3175d9b640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcebf3f60 0x564bcefe5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 -- 172.21.15.94:0/2330314427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 msgr2=0x564bcefe5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 -- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcebf3f60 msgr2=0x564bcefe5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcebf3f60 0x564bcefe5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 -- 172.21.15.94:0/2330314427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bcefe6f80 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3176d9d640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3170009db0 tx=0x7f3170011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9f7a1721961ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f315dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170004da0 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f31700050b0 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317002c030 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f31700052b0 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3183bc9740 10 client.20265 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564bcefe8200 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.657+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317002c030 con 0x564bcec146b0 2024-04-25T13:54:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3170038070 con 0x564bcec146b0 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 10 client.20265 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 10 client.20265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 20 client.20265 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcec35390 0x564bcf059ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 4 client.20265 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bcef43800 con 0x564bcec35390 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3175d9b640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcec35390 0x564bcf059ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f315dffb640 10 client.20265 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3175d9b640 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcec35390 0x564bcf059ab0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892b9cbd77d29f86 server_cookie=7e53f4837a65c256 in_seq=0 out_seq=0 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f315dffb640 1 -- 172.21.15.94:0/2330314427 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bcef43800 con 0x564bcec35390 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f315dffb640 10 client.20265 handle_command_reply: tid=0 2024-04-25T13:54:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.658+0000 7f3183bc9740 1 client.20265 shutdown 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 msgr2=0x564bcefe69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3170009db0 tx=0x7f3170011e50 comp rx=0 tx=0).stop 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 shutdown_connections 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bcec35390 0x564bcf059ab0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bcec146b0 0x564bcefe69c0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bcec0c4d0 0x564bcefe5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f315dffb640 0 client.20265 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 --2- 172.21.15.94:0/2330314427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bcebf3f60 0x564bcefe5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 >> 172.21.15.94:0/2330314427 conn(0x564bcec294e0 msgr2=0x564bcefe1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 shutdown_connections 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 1 -- 172.21.15.94:0/2330314427 wait complete. 2024-04-25T13:54:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f3183bc9740 20 client.20265 trim_cache size 0 max 16384 2024-04-25T13:54:55.666 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:54:55.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 msgr2=0x7f31600b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 0x7f31600b2080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x564bcefd7530 tx=0x7f31680b0040 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 msgr2=0x7f317819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f3168004270 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 shutdown_connections 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f31600afb70 0x7f31600b2080 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31781050a0 0x7f31781a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3178102250 0x7f317819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 --2- 172.21.15.94:0/21938003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3178101750 0x7f317819a210 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 >> 172.21.15.94:0/21938003 conn(0x7f31780fd350 msgr2=0x7f3178105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 shutdown_connections 2024-04-25T13:54:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:55.659+0000 7f317cd14640 1 -- 172.21.15.94:0/21938003 wait complete. 2024-04-25T13:54:55.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1641476 12.52 MB/sec execute 578 sec latency 11.530 ms 2024-04-25T13:54:55.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:55 smithi137 ceph-mon[32568]: pgmap v743: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-25T13:54:55.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2103722985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:55.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:55 smithi094 ceph-mon[29195]: pgmap v743: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-25T13:54:55.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2103722985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:55.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:55 smithi138 ceph-mon[32977]: pgmap v743: 97 pgs: 97 active+clean; 43 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-25T13:54:55.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2103722985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:54:56.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:54:56.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:54:56.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 -- 172.21.15.94:0/1256582008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 msgr2=0x7f1b30104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1256582008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b30104030 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009900 tx=0x7f1b2002a410 comp rx=0 tx=0).stop 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 -- 172.21.15.94:0/1256582008 shutdown_connections 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1256582008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b30108920 0x7f1b3010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1256582008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b30104030 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1256582008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b301029e0 0x7f1b301036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 -- 172.21.15.94:0/1256582008 >> 172.21.15.94:0/1256582008 conn(0x7f1b300fdbd0 msgr2=0x7f1b30100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 -- 172.21.15.94:0/1256582008 shutdown_connections 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 -- 172.21.15.94:0/1256582008 wait complete. 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.789+0000 7f1b3811f640 1 Processor -- start 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 -- start start 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 0x7f1b300ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b30108920 0x7f1b300fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3010e4e0 con 0x7f1b30103c20 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3010e360 con 0x7f1b301029e0 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b3811f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3010e660 con 0x7f1b30108920 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40716/0 (socket says 172.21.15.94:40716) 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 -- 172.21.15.94:0/1043011097 learned_addr learned my addr 172.21.15.94:0/1043011097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 0x7f1b300ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b366bc640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b30108920 0x7f1b300fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 -- 172.21.15.94:0/1043011097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b30108920 msgr2=0x7f1b300fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b30108920 0x7f1b300fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 -- 172.21.15.94:0/1043011097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 msgr2=0x7f1b300ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 0x7f1b300ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 -- 172.21.15.94:0/1043011097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b300fed70 con 0x7f1b30103c20 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 0x7f1b300ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.790+0000 7f1b356ba640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006010 tx=0x7f1b200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.791+0000 7f1b26ffd640 1 -- 172.21.15.94:0/1043011097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2003f030 con 0x7f1b30103c20 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.791+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b301a45f0 con 0x7f1b30103c20 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.791+0000 7f1b26ffd640 1 -- 172.21.15.94:0/1043011097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b20031910 con 0x7f1b30103c20 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.791+0000 7f1b26ffd640 1 -- 172.21.15.94:0/1043011097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20035070 con 0x7f1b30103c20 2024-04-25T13:54:56.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.791+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b301a4a50 con 0x7f1b30103c20 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.793+0000 7f1b3efd1740 1 Processor -- start 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.793+0000 7f1b3efd1740 1 -- start start 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.793+0000 7f1b26ffd640 1 -- 172.21.15.94:0/1043011097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1b20042030 con 0x7f1b30103c20 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60acc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b60a2ab00 con 0x562b606fb550 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b609c6920 con 0x562b60724410 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b60a26ac0 con 0x562b60703730 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b35ebb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37290/0 (socket says 172.21.15.94:37290) 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 -- 172.21.15.94:0/822500106 learned_addr learned my addr 172.21.15.94:0/822500106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b356ba640 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60acc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 -- 172.21.15.94:0/822500106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 msgr2=0x562b60acc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60acc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 -- 172.21.15.94:0/822500106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 msgr2=0x562b60ac5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 -- 172.21.15.94:0/822500106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b608047e0 con 0x562b60724410 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b26ffd640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 0x7f1b1c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b366bc640 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009ca0 tx=0x7f1b28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980c10eda029164e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 0x7f1b1c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.794+0000 7f1b26ffd640 1 -- 172.21.15.94:0/1043011097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b200ff1a0 con 0x7f1b30103c20 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.795+0000 7f1b1bfff640 1 -- 172.21.15.94:0/822500106 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2801e070 con 0x562b60724410 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.795+0000 7f1b1bfff640 1 -- 172.21.15.94:0/822500106 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b28009e50 con 0x562b60724410 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.795+0000 7f1b1bfff640 1 -- 172.21.15.94:0/822500106 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800e9b0 con 0x562b60724410 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- 172.21.15.94:0/822500106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 msgr2=0x562b60ac54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 0x7f1b1c0b2000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x562b60ac6880 tx=0x7f1b2c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009ca0 tx=0x7f1b28011810 comp rx=0 tx=0).stop 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- 172.21.15.94:0/822500106 shutdown_connections 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60acc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/822500106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60ac54a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- 172.21.15.94:0/822500106 >> 172.21.15.94:0/822500106 conn(0x562b60718560 msgr2=0x562b60ac0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- 172.21.15.94:0/822500106 shutdown_connections 2024-04-25T13:54:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- 172.21.15.94:0/822500106 wait complete. 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 -- start start 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.796+0000 7f1b3efd1740 1 Processor -- start 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 0x562b60acf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60724410 0x562b60ad6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b60acd370 con 0x562b60703730 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562b609c45d0 con 0x562b606fb550 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562b60acd4f0 con 0x562b60724410 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 10 client.0 _ll_register_callbacks cb 0x562b609b2080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40742/0 (socket says 172.21.15.94:40742) 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b35ebb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 0x562b60acf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 -- 172.21.15.94:0/3815455242 learned_addr learned my addr 172.21.15.94:0/3815455242 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b366bc640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60724410 0x562b60ad6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 -- 172.21.15.94:0/3815455242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60724410 msgr2=0x562b60ad6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60724410 0x562b60ad6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 -- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 msgr2=0x562b60acf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 0x562b60acf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 -- 172.21.15.94:0/3815455242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b60864f70 con 0x562b60703730 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 0x562b60acf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b356ba640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f1b200c7370 tx=0x7f1b20005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea43e85677bd7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20049070 con 0x562b60703730 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b200b00b0 con 0x562b60703730 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.797+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b28009e50 con 0x562b60703730 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b200c7440 con 0x562b60703730 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 10 client.20277 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562b606fc170 con 0x562b60703730 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b28009e50 con 0x562b60703730 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1b200c3900 con 0x562b60703730 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 10 client.20277 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 10 client.20277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 20 client.20277 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606e31f0 0x562b60adcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 4 client.20277 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b60a38710 con 0x562b606e31f0 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.798+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606e31f0 0x562b60adcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.799+0000 7f1b197fa640 10 client.20277 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.799+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606e31f0 0x562b60adcb90 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8175d8beba31c9d server_cookie=28b28b98a10c10a0 in_seq=0 out_seq=0 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.799+0000 7f1b197fa640 1 -- 172.21.15.94:0/3815455242 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b60a38710 con 0x562b606e31f0 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.799+0000 7f1b197fa640 10 client.20277 handle_command_reply: tid=0 2024-04-25T13:54:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.799+0000 7f1b3efd1740 1 client.20277 shutdown 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 msgr2=0x562b60acf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f1b200c7370 tx=0x7f1b20005c90 comp rx=0 tx=0).stop 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 shutdown_connections 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606e31f0 0x562b60adcb90 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60724410 0x562b60ad6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60703730 0x562b60acf8d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/3815455242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b606fb550 0x562b60acf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 >> 172.21.15.94:0/3815455242 conn(0x562b60718560 msgr2=0x562b60abfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b197fa640 0 client.20277 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 shutdown_connections 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 1 -- 172.21.15.94:0/3815455242 wait complete. 2024-04-25T13:54:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.800+0000 7f1b3efd1740 20 client.20277 trim_cache size 0 max 16384 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 Processor -- start 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 -- start start 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60ac9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60acd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301ad6f0 con 0x562b606fb550 2024-04-25T13:54:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301af230 con 0x562b60724410 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301addc0 con 0x562b60703730 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40752/0 (socket says 172.21.15.94:40752) 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 -- 172.21.15.94:0/253529004 learned_addr learned my addr 172.21.15.94:0/253529004 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b356ba640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60ac9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b366bc640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60acd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 -- 172.21.15.94:0/253529004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 msgr2=0x562b60ac9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60ac9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 -- 172.21.15.94:0/253529004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 msgr2=0x562b60acd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60acd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b35ebb640 1 -- 172.21.15.94:0/253529004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b60ac66a0 con 0x562b606fb550 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.818+0000 7f1b366bc640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60acd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.819+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1b301022b0 tx=0x7f1b2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a5422e48a4acd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.819+0000 7f1b1b7fe640 1 -- 172.21.15.94:0/253529004 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c004b20 con 0x562b606fb550 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.819+0000 7f1b1b7fe640 1 -- 172.21.15.94:0/253529004 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c003e20 con 0x562b606fb550 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.819+0000 7f1b1b7fe640 1 -- 172.21.15.94:0/253529004 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c02a420 con 0x562b606fb550 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- 172.21.15.94:0/253529004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 msgr2=0x562b60ac97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1b301022b0 tx=0x7f1b2c016000 comp rx=0 tx=0).stop 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- 172.21.15.94:0/253529004 shutdown_connections 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60724410 0x562b60acd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b60703730 0x562b60ac9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/253529004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b606fb550 0x562b60ac97b0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- 172.21.15.94:0/253529004 >> 172.21.15.94:0/253529004 conn(0x562b60718560 msgr2=0x562b60ac8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- 172.21.15.94:0/253529004 shutdown_connections 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- 172.21.15.94:0/253529004 wait complete. 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 -- start start 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 Processor -- start 2024-04-25T13:54:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.820+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b606fb550 0x562b60ad5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 0x562b60ad29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301aeb60 con 0x562b60724410 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301ae490 con 0x562b60703730 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b301ab6d0 con 0x562b606fb550 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b3efd1740 10 client.0 _ll_register_callbacks cb 0x7f1b2003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b366bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b366bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40760/0 (socket says 172.21.15.94:40760) 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b366bc640 1 -- 172.21.15.94:0/861361316 learned_addr learned my addr 172.21.15.94:0/861361316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 0x562b60ad29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b606fb550 0x562b60ad5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 -- 172.21.15.94:0/861361316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b606fb550 msgr2=0x562b60ad5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b606fb550 0x562b60ad5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 -- 172.21.15.94:0/861361316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 msgr2=0x562b60ad3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 -- 172.21.15.94:0/861361316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b60b42f60 con 0x562b60703730 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b366bc640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b356ba640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 0x562b60ad29f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1b200c48e0 tx=0x7f1b200d0340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e708bd28415d548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.821+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20049030 con 0x562b60703730 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c003e20 con 0x562b60703730 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1b200c3c60 con 0x562b60703730 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1b200044e0 con 0x562b60703730 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 10 client.29726 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:56.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562b60b44200 con 0x562b60703730 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c003e20 con 0x562b60703730 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f1b200c3900 con 0x562b60703730 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 10 client.29726 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 10 client.29726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 20 client.29726 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606b8630 0x562b60b46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 4 client.29726 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562b60a32580 con 0x562b606b8630 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.822+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606b8630 0x562b60b46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.823+0000 7f1b197fa640 10 client.29726 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.823+0000 7f1b35ebb640 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606b8630 0x562b60b46860 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b4e374acb11350 server_cookie=6e9e4605f7970972 in_seq=0 out_seq=0 2024-04-25T13:54:56.828 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1644012 12.52 MB/sec execute 579 sec latency 16.365 ms 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b197fa640 1 -- 172.21.15.94:0/861361316 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562b60a32580 con 0x562b606b8630 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b197fa640 10 client.29726 handle_command_reply: tid=0 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 client.29726 shutdown 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 msgr2=0x562b60ad29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 0x562b60ad29f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1b200c48e0 tx=0x7f1b200d0340 comp rx=0 tx=0).stop 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 shutdown_connections 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562b606b8630 0x562b60b46860 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b60724410 0x562b60ad3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b60703730 0x562b60ad29f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b197fa640 0 client.29726 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 --2- 172.21.15.94:0/861361316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562b606fb550 0x562b60ad5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 >> 172.21.15.94:0/861361316 conn(0x562b60718560 msgr2=0x562b60acf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 shutdown_connections 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 1 -- 172.21.15.94:0/861361316 wait complete. 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.824+0000 7f1b3efd1740 20 client.29726 trim_cache size 0 max 16384 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "7e10cc9e-ef92-4a0b-af37-aab59791d51f", 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:54:56.830 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:56.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 msgr2=0x7f1b1c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 0x7f1b1c0b2000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x562b60ac6880 tx=0x7f1b2c009500 comp rx=0 tx=0).stop 2024-04-25T13:54:56.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 msgr2=0x7f1b300fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1b20006010 tx=0x7f1b200042f0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 shutdown_connections 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1b1c0afaf0 0x7f1b1c0b2000 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b30108920 0x7f1b300fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b30103c20 0x7f1b300fe100 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 --2- 172.21.15.94:0/1043011097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b301029e0 0x7f1b300ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 >> 172.21.15.94:0/1043011097 conn(0x7f1b300fdbd0 msgr2=0x7f1b30109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 shutdown_connections 2024-04-25T13:54:56.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.825+0000 7f1b3811f640 1 -- 172.21.15.94:0/1043011097 wait complete. 2024-04-25T13:54:56.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:54:56.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:54:56.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 -- 172.21.15.94:0/4102702378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 msgr2=0x7f7edc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 --2- 172.21.15.94:0/4102702378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc1036e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed802a410 comp rx=0 tx=0).stop 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 -- 172.21.15.94:0/4102702378 shutdown_connections 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 --2- 172.21.15.94:0/4102702378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc108920 0x7f7edc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 --2- 172.21.15.94:0/4102702378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc103c20 0x7f7edc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 --2- 172.21.15.94:0/4102702378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc1036e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 -- 172.21.15.94:0/4102702378 >> 172.21.15.94:0/4102702378 conn(0x7f7edc0fdbd0 msgr2=0x7f7edc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.947+0000 7f7ee3442640 1 -- 172.21.15.94:0/4102702378 shutdown_connections 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 -- 172.21.15.94:0/4102702378 wait complete. 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 Processor -- start 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 -- start start 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc103c20 0x7f7edc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc108920 0x7f7edc1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc10e4c0 con 0x7f7edc1029e0 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc10e340 con 0x7f7edc103c20 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee3442640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc10e640 con 0x7f7edc108920 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee11de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.948+0000 7f7ee11de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40780/0 (socket says 172.21.15.94:40780) 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 -- 172.21.15.94:0/3480092800 learned_addr learned my addr 172.21.15.94:0/3480092800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee19df640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc108920 0x7f7edc1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc103c20 0x7f7edc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 -- 172.21.15.94:0/3480092800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc108920 msgr2=0x7f7edc1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc108920 0x7f7edc1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 -- 172.21.15.94:0/3480092800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc103c20 msgr2=0x7f7edc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc103c20 0x7f7edc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 -- 172.21.15.94:0/3480092800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc1a20a0 con 0x7f7edc1029e0 2024-04-25T13:54:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee11de640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8006010 tx=0x7f7ed8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ed27fc640 1 -- 172.21.15.94:0/3480092800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed803f030 con 0x7f7edc1029e0 2024-04-25T13:54:56.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc1a2330 con 0x7f7edc1029e0 2024-04-25T13:54:56.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ed27fc640 1 -- 172.21.15.94:0/3480092800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed8003aa0 con 0x7f7edc1029e0 2024-04-25T13:54:56.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ed27fc640 1 -- 172.21.15.94:0/3480092800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8035070 con 0x7f7edc1029e0 2024-04-25T13:54:56.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.949+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc1a2820 con 0x7f7edc1029e0 2024-04-25T13:54:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ed27fc640 1 -- 172.21.15.94:0/3480092800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7ed8042030 con 0x7f7edc1029e0 2024-04-25T13:54:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 Processor -- start 2024-04-25T13:54:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 -- start start 2024-04-25T13:54:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333048c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c332facd80 con 0x55c332ca6390 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c332f668b0 con 0x55c332c856b0 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c332fa8d40 con 0x55c332c7d4d0 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40786/0 (socket says 172.21.15.94:40786) 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 -- 172.21.15.94:0/4106787120 learned_addr learned my addr 172.21.15.94:0/4106787120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333048c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee11de640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 -- 172.21.15.94:0/4106787120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 msgr2=0x55c333047fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 -- 172.21.15.94:0/4106787120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 msgr2=0x55c333048c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333048c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ed27fc640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 0x7f7ec80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee19df640 1 -- 172.21.15.94:0/4106787120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c332f37a40 con 0x55c332ca6390 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.952+0000 7f7ee11de640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ee11de640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 0x7f7ec80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ee19df640 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4010370 tx=0x7f7ed4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803165b5d5559600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ed27fc640 1 -- 172.21.15.94:0/3480092800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7ed80ff250 con 0x7f7edc1029e0 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ec77fe640 1 -- 172.21.15.94:0/4106787120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4016070 con 0x55c332ca6390 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ec77fe640 1 -- 172.21.15.94:0/4106787120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed4004d20 con 0x55c332ca6390 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.953+0000 7f7ec77fe640 1 -- 172.21.15.94:0/4106787120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4005390 con 0x55c332ca6390 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- 172.21.15.94:0/4106787120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 msgr2=0x55c333047a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7ee11de640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 0x7f7ec80b2000 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55c333047780 tx=0x7f7ed8003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4010370 tx=0x7f7ed4010830 comp rx=0 tx=0).stop 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- 172.21.15.94:0/4106787120 shutdown_connections 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333048c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/4106787120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332ca6390 0x55c333047a60 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- 172.21.15.94:0/4106787120 >> 172.21.15.94:0/4106787120 conn(0x55c332c9a4e0 msgr2=0x55c333043270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- 172.21.15.94:0/4106787120 shutdown_connections 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- 172.21.15.94:0/4106787120 wait complete. 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 -- start start 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.954+0000 7f7eea3ce740 1 Processor -- start 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 0x55c333051e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332ca6390 0x55c333058dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c333049780 con 0x55c332c7d4d0 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c332f33d10 con 0x55c332ca6390 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c333049900 con 0x55c332c856b0 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7eea3ce740 10 client.0 _ll_register_callbacks cb 0x55c332f60210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:56.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40794/0 (socket says 172.21.15.94:40794) 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 -- 172.21.15.94:0/1561642861 learned_addr learned my addr 172.21.15.94:0/1561642861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332ca6390 0x55c333058dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 0x55c333051e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 -- 172.21.15.94:0/1561642861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 msgr2=0x55c333051e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 0x55c333051e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 -- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332ca6390 msgr2=0x55c333058dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332ca6390 0x55c333058dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 -- 172.21.15.94:0/1561642861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c332d887d0 con 0x55c332c7d4d0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 0x55c333051e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4004cf0 tx=0x7f7ed4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319eae348b03eedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.955+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4016070 con 0x55c332c7d4d0 2024-04-25T13:54:56.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7ed4012710 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed4004d20 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ed4012910 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 10 client.20295 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c332cbe3f0 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed4004d20 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed4028070 con 0x55c332c7d4d0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 10 client.20295 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 10 client.20295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 20 client.20295 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332c64f60 0x55c33305eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 4 client.20295 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c332fba990 con 0x55c332c64f60 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.956+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332c64f60 0x55c33305eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7ec4ff9640 10 client.20295 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332c64f60 0x55c33305eef0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c88da4ea750dd4 server_cookie=aee0035eae7d1562 in_seq=0 out_seq=0 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/1561642861 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c332fba990 con 0x55c332c64f60 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7ec4ff9640 10 client.20295 handle_command_reply: tid=0 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7eea3ce740 1 client.20295 shutdown 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 msgr2=0x55c333051930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.957+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4004cf0 tx=0x7f7ed4005c50 comp rx=0 tx=0).stop 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 shutdown_connections 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332c64f60 0x55c33305eef0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332ca6390 0x55c333058dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7ec4ff9640 0 client.20295 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c856b0 0x55c333051e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1561642861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c7d4d0 0x55c333051930 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 >> 172.21.15.94:0/1561642861 conn(0x55c332c9a4e0 msgr2=0x55c3330423f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 shutdown_connections 2024-04-25T13:54:56.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1561642861 wait complete. 2024-04-25T13:54:56.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.958+0000 7f7eea3ce740 20 client.20295 trim_cache size 0 max 16384 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 Processor -- start 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 -- start start 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333050610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ab550 con 0x55c332c64f60 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ad090 con 0x55c332c856b0 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1abc20 con 0x55c332c7d4d0 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40808/0 (socket says 172.21.15.94:40808) 2024-04-25T13:54:56.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 -- 172.21.15.94:0/1204746870 learned_addr learned my addr 172.21.15.94:0/1204746870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333050610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 -- 172.21.15.94:0/1204746870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 msgr2=0x55c333047a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 -- 172.21.15.94:0/1204746870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 msgr2=0x55c333050610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333050610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 -- 172.21.15.94:0/1204746870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c332c7e0f0 con 0x55c332c64f60 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.977+0000 7f7ee19df640 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009ac0 tx=0x7f7ed40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1600cd1cc531799b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.978+0000 7f7ec6ffd640 1 -- 172.21.15.94:0/1204746870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4016070 con 0x55c332c64f60 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.978+0000 7f7ec6ffd640 1 -- 172.21.15.94:0/1204746870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed401f030 con 0x55c332c64f60 2024-04-25T13:54:56.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.978+0000 7f7ec6ffd640 1 -- 172.21.15.94:0/1204746870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4012d10 con 0x55c332c64f60 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1204746870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 msgr2=0x55c33304ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009ac0 tx=0x7f7ed40021f0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1204746870 shutdown_connections 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c856b0 0x55c333050610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c7d4d0 0x55c333047a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/1204746870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c64f60 0x55c33304ccf0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1204746870 >> 172.21.15.94:0/1204746870 conn(0x55c332c9a4e0 msgr2=0x55c33304ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1204746870 shutdown_connections 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- 172.21.15.94:0/1204746870 wait complete. 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- start start 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 Processor -- start 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c64f60 0x55c333055380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c7d4d0 0x55c3330558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ac9c0 con 0x55c332c856b0 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ac2f0 con 0x55c332c7d4d0 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.979+0000 7f7eea3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1a9530 con 0x55c332c64f60 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7eea3ce740 10 client.0 _ll_register_callbacks cb 0x7f7ecc00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40822/0 (socket says 172.21.15.94:40822) 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 -- 172.21.15.94:0/245247989 learned_addr learned my addr 172.21.15.94:0/245247989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee09dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c7d4d0 0x55c3330558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c64f60 0x55c333055380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 -- 172.21.15.94:0/245247989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c64f60 msgr2=0x55c333055380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c64f60 0x55c333055380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 -- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c7d4d0 msgr2=0x55c3330558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c7d4d0 0x55c3330558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 -- 172.21.15.94:0/245247989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c332f6d4b0 con 0x55c332c856b0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ee19df640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4005380 tx=0x7f7ed4010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825f7d220ca4e31a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4016040 con 0x55c332c856b0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7ed40059a0 con 0x55c332c856b0 2024-04-25T13:54:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed401f030 con 0x55c332c856b0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ed4005ba0 con 0x55c332c856b0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7eea3ce740 10 client.20301 fetch_fsmap learned FSMap version 31 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.980+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3330c6590 con 0x55c332c856b0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed401f030 con 0x55c332c856b0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed4012ce0 con 0x55c332c856b0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 10 client.20301 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 10 client.20301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 20 client.20301 populate_metadata read hostname 'smithi094' 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332ca6390 0x55c3330c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 4 client.20301 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c332fb4800 con 0x55c332ca6390 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332ca6390 0x55c3330c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:56.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7ec4ff9640 10 client.20301 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.981+0000 7f7ee09dd640 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332ca6390 0x55c3330c8e60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82013c303653903 server_cookie=b70a162712432537 in_seq=0 out_seq=0 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7ec4ff9640 1 -- 172.21.15.94:0/245247989 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c332fb4800 con 0x55c332ca6390 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7ec4ff9640 10 client.20301 handle_command_reply: tid=0 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 client.20301 shutdown 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 msgr2=0x55c3330c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4005380 tx=0x7f7ed4010d30 comp rx=0 tx=0).stop 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 shutdown_connections 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c332ca6390 0x55c3330c8e60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c332c856b0 0x55c3330c5130 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c332c7d4d0 0x55c3330558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7ec4ff9640 0 client.20301 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 --2- 172.21.15.94:0/245247989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c332c64f60 0x55c333055380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 >> 172.21.15.94:0/245247989 conn(0x55c332c9a4e0 msgr2=0x55c333051c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 shutdown_connections 2024-04-25T13:54:56.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 1 -- 172.21.15.94:0/245247989 wait complete. 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.982+0000 7f7eea3ce740 20 client.20301 trim_cache size 0 max 16384 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "7e10cc9e-ef92-4a0b-af37-aab59791d51f": { 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "7e10cc9e-ef92-4a0b-af37-aab59791d51f", 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:54:56.988 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:54:56.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 msgr2=0x7f7ec80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 0x7f7ec80b2000 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55c333047780 tx=0x7f7ed8003f90 comp rx=0 tx=0).stop 2024-04-25T13:54:56.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 msgr2=0x7f7edc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:56.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8006010 tx=0x7f7ed8031a50 comp rx=0 tx=0).stop 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 shutdown_connections 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7ec80afaf0 0x7f7ec80b2000 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7edc108920 0x7f7edc1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7edc103c20 0x7f7edc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 --2- 172.21.15.94:0/3480092800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc1029e0 0x7f7edc19a560 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 >> 172.21.15.94:0/3480092800 conn(0x7f7edc0fdbd0 msgr2=0x7f7edc109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 shutdown_connections 2024-04-25T13:54:56.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:54:56.983+0000 7f7ee3442640 1 -- 172.21.15.94:0/3480092800 wait complete. 2024-04-25T13:54:56.992 INFO:tasks.cephfs.filesystem:scrub status for tag:7e10cc9e-ef92-4a0b-af37-aab59791d51f - {'path': '/', 'tag': '7e10cc9e-ef92-4a0b-af37-aab59791d51f', 'options': 'recursive,force'} 2024-04-25T13:54:57.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1645547 12.51 MB/sec execute 580 sec latency 92.786 ms 2024-04-25T13:54:57.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:57 smithi137 ceph-mon[32568]: pgmap v744: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-04-25T13:54:57.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:57 smithi094 ceph-mon[29195]: pgmap v744: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-04-25T13:54:57.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:57 smithi138 ceph-mon[32977]: pgmap v744: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-04-25T13:54:58.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1648391 12.52 MB/sec execute 581 sec latency 47.782 ms 2024-04-25T13:54:58.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:58 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:54:58.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:58 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:58.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:58 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:54:58.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:58 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:58 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:54:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:58 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:58 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:54:58.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:58 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:58 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:54:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:58 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:58 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:54:58.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:58 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:54:59.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1649875 12.51 MB/sec execute 582 sec latency 82.118 ms 2024-04-25T13:54:59.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:54:59 smithi137 ceph-mon[32568]: pgmap v745: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-04-25T13:54:59.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:54:59 smithi094 ceph-mon[29195]: pgmap v745: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-04-25T13:54:59.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:54:59 smithi138 ceph-mon[32977]: pgmap v745: 97 pgs: 97 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 323 op/s 2024-04-25T13:55:00.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1652175 12.51 MB/sec execute 583 sec latency 22.376 ms 2024-04-25T13:55:01.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1654738 12.51 MB/sec execute 584 sec latency 21.426 ms 2024-04-25T13:55:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:01 smithi137 ceph-mon[32568]: pgmap v746: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-04-25T13:55:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:01 smithi094 ceph-mon[29195]: pgmap v746: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-04-25T13:55:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:01 smithi138 ceph-mon[32977]: pgmap v746: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-04-25T13:55:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:02.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1656727 12.51 MB/sec execute 585 sec latency 15.849 ms 2024-04-25T13:55:03.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1659354 12.51 MB/sec execute 586 sec latency 9.078 ms 2024-04-25T13:55:03.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:03 smithi137 ceph-mon[32568]: pgmap v747: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-04-25T13:55:03.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:03 smithi094 ceph-mon[29195]: pgmap v747: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-04-25T13:55:03.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:03 smithi138 ceph-mon[32977]: pgmap v747: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-04-25T13:55:04.822 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1661731 12.51 MB/sec execute 587 sec latency 65.130 ms 2024-04-25T13:55:05.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1663462 12.50 MB/sec execute 588 sec latency 17.342 ms 2024-04-25T13:55:05.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:05 smithi137 ceph-mon[32568]: pgmap v748: 97 pgs: 97 active+clean; 44 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-04-25T13:55:05.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:05 smithi094 ceph-mon[29195]: pgmap v748: 97 pgs: 97 active+clean; 44 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-04-25T13:55:05.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:05 smithi138 ceph-mon[32977]: pgmap v748: 97 pgs: 97 active+clean; 44 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.188+0000 7fae2d39d640 1 -- 172.21.15.94:0/4017654102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28102290 msgr2=0x7fae2810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.188+0000 7fae2d39d640 1 --2- 172.21.15.94:0/4017654102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28102290 0x7fae2810bcf0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fae20009ca0 tx=0x7fae2002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.188+0000 7fae2d39d640 1 -- 172.21.15.94:0/4017654102 shutdown_connections 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.188+0000 7fae2d39d640 1 --2- 172.21.15.94:0/4017654102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28102290 0x7fae2810bcf0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 --2- 172.21.15.94:0/4017654102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae28101790 0x7fae28101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 --2- 172.21.15.94:0/4017654102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae281050e0 0x7fae281054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 -- 172.21.15.94:0/4017654102 >> 172.21.15.94:0/4017654102 conn(0x7fae280fd390 msgr2=0x7fae280ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 -- 172.21.15.94:0/4017654102 shutdown_connections 2024-04-25T13:55:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 -- 172.21.15.94:0/4017654102 wait complete. 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 Processor -- start 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.189+0000 7fae2d39d640 1 -- start start 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae28102290 0x7fae28102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae281050e0 0x7fae28102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810f670 con 0x7fae28101790 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810f4f0 con 0x7fae281050e0 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae2810f7f0 con 0x7fae28102290 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42054/0 (socket says 172.21.15.94:42054) 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 -- 172.21.15.94:0/1436968468 learned_addr learned my addr 172.21.15.94:0/1436968468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae277fe640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae281050e0 0x7fae28102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae267fc640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae28102290 0x7fae28102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 -- 172.21.15.94:0/1436968468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae28102290 msgr2=0x7fae28102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae28102290 0x7fae28102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 -- 172.21.15.94:0/1436968468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae281050e0 msgr2=0x7fae28102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae281050e0 0x7fae28102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 -- 172.21.15.94:0/1436968468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae281035a0 con 0x7fae28101790 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae26ffd640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9f0 tx=0x7fae1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae0ffff640 1 -- 172.21.15.94:0/1436968468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800ccc0 con 0x7fae28101790 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae281a4270 con 0x7fae28101790 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae0ffff640 1 -- 172.21.15.94:0/1436968468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180035f0 con 0x7fae28101790 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae0ffff640 1 -- 172.21.15.94:0/1436968468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800ce60 con 0x7fae28101790 2024-04-25T13:55:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.190+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a4730 con 0x7fae28101790 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.193+0000 7fae3424f740 1 Processor -- start 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.193+0000 7fae3424f740 1 -- start start 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae0ffff640 1 -- 172.21.15.94:0/1436968468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fae18010690 con 0x7fae28101790 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff010970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff017df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2fef76460 con 0x55f2fec472f0 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2fef30590 con 0x55f2fec4f4d0 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae3424f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2fef72420 con 0x55f2fec701b0 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42068/0 (socket says 172.21.15.94:42068) 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 -- 172.21.15.94:0/3991942731 learned_addr learned my addr 172.21.15.94:0/3991942731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae267fc640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff017df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae277fe640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff010970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 -- 172.21.15.94:0/3991942731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 msgr2=0x55f2ff010970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff010970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 -- 172.21.15.94:0/3991942731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 msgr2=0x55f2ff017df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff017df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 -- 172.21.15.94:0/3991942731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fef3d0d0 con 0x55f2fec472f0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae267fc640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff017df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae277fe640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff010970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae26ffd640 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fae180e2f90 tx=0x7fae180e71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94c88e63a3157fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae0ffff640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 0x7fae100b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.194+0000 7fae0d7fa640 1 -- 172.21.15.94:0/3991942731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180f2070 con 0x55f2fec472f0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae277fe640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 0x7fae100b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae0d7fa640 1 -- 172.21.15.94:0/3991942731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180cccd0 con 0x55f2fec472f0 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae0d7fa640 1 -- 172.21.15.94:0/3991942731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18017a70 con 0x55f2fec472f0 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae0ffff640 1 -- 172.21.15.94:0/1436968468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fae180df010 con 0x7fae28101790 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 -- 172.21.15.94:0/3991942731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 msgr2=0x55f2ff010ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fae180e2f90 tx=0x7fae180e71a0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae277fe640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 0x7fae100b1ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55f2ff011b40 tx=0x7fae2002f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 -- 172.21.15.94:0/3991942731 shutdown_connections 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff017df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff010ef0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 --2- 172.21.15.94:0/3991942731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff010970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 -- 172.21.15.94:0/3991942731 >> 172.21.15.94:0/3991942731 conn(0x55f2fec64300 msgr2=0x55f2ff00c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.195+0000 7fae3424f740 1 -- 172.21.15.94:0/3991942731 shutdown_connections 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 -- 172.21.15.94:0/3991942731 wait complete. 2024-04-25T13:55:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 -- start start 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 Processor -- start 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff021b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2ff018840 con 0x55f2fec472f0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2feefdb60 con 0x55f2fec4f4d0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2ff0189c0 con 0x55f2fec701b0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae3424f740 10 client.0 _ll_register_callbacks cb 0x55f2fef222b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42084/0 (socket says 172.21.15.94:42084) 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae26ffd640 1 -- 172.21.15.94:0/2474198805 learned_addr learned my addr 172.21.15.94:0/2474198805 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae267fc640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.196+0000 7fae277fe640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff021b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 -- 172.21.15.94:0/2474198805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 msgr2=0x55f2ff021b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff021b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 -- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 msgr2=0x55f2ff01abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 -- 172.21.15.94:0/2474198805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fec88210 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae267fc640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fae180a73e0 tx=0x7fae180045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719a8878ea9d014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18016a20 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fae180037a0 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae180cccd0 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae18016c00 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 10 client.20313 fetch_fsmap learned FSMap version 31 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f2fed54620 con 0x55f2fec472f0 2024-04-25T13:55:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180cccd0 con 0x55f2fec472f0 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fae180a36e0 con 0x55f2fec472f0 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 10 client.20313 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 10 client.20313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.197+0000 7fae3424f740 20 client.20313 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec2ed80 0x55f2ff027f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae3424f740 4 client.20313 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2fef84070 con 0x55f2fec2ed80 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae267fc640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec2ed80 0x55f2ff027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae0aff5640 10 client.20313 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.198+0000 7fae267fc640 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec2ed80 0x55f2ff027f70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959bd302fc07ecdf server_cookie=f17ce6a4b824a2e4 in_seq=0 out_seq=0 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae0aff5640 1 -- 172.21.15.94:0/2474198805 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2fef84070 con 0x55f2fec2ed80 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae0aff5640 10 client.20313 handle_command_reply: tid=0 2024-04-25T13:55:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 client.20313 shutdown 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 msgr2=0x55f2ff01a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fae180a73e0 tx=0x7fae180045b0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 shutdown_connections 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec2ed80 0x55f2ff027f70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff021b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae0aff5640 0 client.20313 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 --2- 172.21.15.94:0/2474198805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff01a680 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 >> 172.21.15.94:0/2474198805 conn(0x55f2fec64300 msgr2=0x55f2ff00f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 shutdown_connections 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 1 -- 172.21.15.94:0/2474198805 wait complete. 2024-04-25T13:55:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.200+0000 7fae3424f740 20 client.20313 trim_cache size 0 max 16384 2024-04-25T13:55:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 Processor -- start 2024-04-25T13:55:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 -- start start 2024-04-25T13:55:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff021ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff019c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ad790 con 0x55f2fec472f0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae281af2d0 con 0x55f2fec4f4d0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae3424f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ade60 con 0x55f2fec701b0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42088/0 (socket says 172.21.15.94:42088) 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 -- 172.21.15.94:0/2416941242 learned_addr learned my addr 172.21.15.94:0/2416941242 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae267fc640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff021ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae277fe640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff019c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 -- 172.21.15.94:0/2416941242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 msgr2=0x55f2ff019c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff019c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 -- 172.21.15.94:0/2416941242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 msgr2=0x55f2ff021ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff021ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 -- 172.21.15.94:0/2416941242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ff011250 con 0x55f2fec472f0 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae277fe640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff019c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.216+0000 7fae26ffd640 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fae1800c6a0 tx=0x7fae180dcb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae6d5c886236a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae0cff9640 1 -- 172.21.15.94:0/2416941242 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18016de0 con 0x55f2fec472f0 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae0cff9640 1 -- 172.21.15.94:0/2416941242 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180ab800 con 0x55f2fec472f0 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae0cff9640 1 -- 172.21.15.94:0/2416941242 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1801f340 con 0x55f2fec472f0 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 -- 172.21.15.94:0/2416941242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 msgr2=0x55f2ff00c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fae1800c6a0 tx=0x7fae180dcb00 comp rx=0 tx=0).stop 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 -- 172.21.15.94:0/2416941242 shutdown_connections 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec701b0 0x55f2ff019c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec4f4d0 0x55f2ff021ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.217+0000 7fae3424f740 1 --2- 172.21.15.94:0/2416941242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec472f0 0x55f2ff00c320 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- 172.21.15.94:0/2416941242 >> 172.21.15.94:0/2416941242 conn(0x55f2fec64300 msgr2=0x55f2ff00bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- 172.21.15.94:0/2416941242 shutdown_connections 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- 172.21.15.94:0/2416941242 wait complete. 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- start start 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 Processor -- start 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec472f0 0x55f2ff01f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec701b0 0x55f2ff0205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae281aec00 con 0x55f2fec4f4d0 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ae530 con 0x55f2fec701b0 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ab770 con 0x55f2fec472f0 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae3424f740 10 client.0 _ll_register_callbacks cb 0x7fae180f0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42090/0 (socket says 172.21.15.94:42090) 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 -- 172.21.15.94:0/664429873 learned_addr learned my addr 172.21.15.94:0/664429873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae277fe640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec701b0 0x55f2ff0205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae26ffd640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec472f0 0x55f2ff01f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 -- 172.21.15.94:0/664429873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec472f0 msgr2=0x55f2ff01f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec472f0 0x55f2ff01f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 -- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec701b0 msgr2=0x55f2ff0205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec701b0 0x55f2ff0205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.218+0000 7fae267fc640 1 -- 172.21.15.94:0/664429873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ff020bb0 con 0x55f2fec4f4d0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae267fc640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000c00 tx=0x7fae1c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2752db0bc40adf75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c024070 con 0x55f2fec4f4d0 2024-04-25T13:55:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae180ab800 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fae1c004460 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fae1c030050 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 10 client.20319 fetch_fsmap learned FSMap version 31 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f2ff090150 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae180ab800 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fae18017ce0 con 0x55f2fec4f4d0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 10 client.20319 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 10 client.20319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 20 client.20319 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec041c0 0x55f2ff0929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 4 client.20319 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.219+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2fef7dee0 con 0x55f2fec041c0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.220+0000 7fae26ffd640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec041c0 0x55f2ff0929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.220+0000 7fae0aff5640 10 client.20319 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.220+0000 7fae26ffd640 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec041c0 0x55f2ff0929d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2aef4fb0e29181 server_cookie=3982cee40faf3832 in_seq=0 out_seq=0 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae0aff5640 1 -- 172.21.15.94:0/664429873 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2fef7dee0 con 0x55f2fec041c0 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae0aff5640 10 client.20319 handle_command_reply: tid=0 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 client.20319 shutdown 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 msgr2=0x55f2ff01f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000c00 tx=0x7fae1c012ba0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 shutdown_connections 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2fec041c0 0x55f2ff0929d0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2fec701b0 0x55f2ff0205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae0aff5640 0 client.20319 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2fec4f4d0 0x55f2ff01f710 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 --2- 172.21.15.94:0/664429873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2fec472f0 0x55f2ff01f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 >> 172.21.15.94:0/664429873 conn(0x55f2fec64300 msgr2=0x55f2ff01a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 shutdown_connections 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 1 -- 172.21.15.94:0/664429873 wait complete. 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.427+0000 7fae3424f740 20 client.20319 trim_cache size 0 max 16384 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 msgr2=0x7fae100b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 0x7fae100b1ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55f2ff011b40 tx=0x7fae2002f310 comp rx=0 tx=0).stop 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 msgr2=0x7fae28104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9f0 tx=0x7fae1800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 shutdown_connections 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fae100afae0 0x7fae100b1ff0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae281050e0 0x7fae28102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae28102290 0x7fae28102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 --2- 172.21.15.94:0/1436968468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae28101790 0x7fae28104290 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 >> 172.21.15.94:0/1436968468 conn(0x7fae280fd390 msgr2=0x7fae28107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 shutdown_connections 2024-04-25T13:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:06.428+0000 7fae2d39d640 1 -- 172.21.15.94:0/1436968468 wait complete. 2024-04-25T13:55:06.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1666030 12.51 MB/sec execute 589 sec latency 127.023 ms 2024-04-25T13:55:07.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1667815 12.50 MB/sec execute 590 sec latency 28.388 ms 2024-04-25T13:55:07.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:07 smithi137 ceph-mon[32568]: pgmap v749: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T13:55:07.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:07 smithi094 ceph-mon[29195]: pgmap v749: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T13:55:07.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:07 smithi138 ceph-mon[32977]: pgmap v749: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T13:55:08.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1670178 12.50 MB/sec execute 591 sec latency 14.716 ms 2024-04-25T13:55:09.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1672847 12.50 MB/sec execute 592 sec latency 61.110 ms 2024-04-25T13:55:09.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:09 smithi137 ceph-mon[32568]: pgmap v750: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 243 op/s 2024-04-25T13:55:09.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:09 smithi094 ceph-mon[29195]: pgmap v750: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 243 op/s 2024-04-25T13:55:09.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:09 smithi138 ceph-mon[32977]: pgmap v750: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 243 op/s 2024-04-25T13:55:10.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1674457 12.50 MB/sec execute 593 sec latency 51.965 ms 2024-04-25T13:55:11.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1677062 12.50 MB/sec execute 594 sec latency 8.868 ms 2024-04-25T13:55:11.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:11 smithi137 ceph-mon[32568]: pgmap v751: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-04-25T13:55:11.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:11 smithi094 ceph-mon[29195]: pgmap v751: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-04-25T13:55:11.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:11 smithi138 ceph-mon[32977]: pgmap v751: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 407 op/s 2024-04-25T13:55:12.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1679749 12.50 MB/sec execute 595 sec latency 61.155 ms 2024-04-25T13:55:13.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1681626 12.50 MB/sec execute 596 sec latency 15.820 ms 2024-04-25T13:55:13.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:13 smithi137 ceph-mon[32568]: pgmap v752: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-04-25T13:55:13.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:13 smithi094 ceph-mon[29195]: pgmap v752: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-04-25T13:55:13.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:13 smithi138 ceph-mon[32977]: pgmap v752: 97 pgs: 97 active+clean; 43 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2024-04-25T13:55:14.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1684024 12.50 MB/sec execute 597 sec latency 43.988 ms 2024-04-25T13:55:15.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1686412 12.50 MB/sec execute 598 sec latency 21.553 ms 2024-04-25T13:55:15.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:15 smithi137 ceph-mon[32568]: pgmap v753: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-04-25T13:55:15.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:15 smithi094 ceph-mon[29195]: pgmap v753: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-04-25T13:55:15.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:15 smithi138 ceph-mon[32977]: pgmap v753: 97 pgs: 97 active+clean; 46 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-04-25T13:55:16.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1688260 12.50 MB/sec execute 599 sec latency 14.157 ms 2024-04-25T13:55:17.823 INFO:tasks.workunit.client.0.smithi094.stdout: 1 cleanup 600 sec 2024-04-25T13:55:17.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:17 smithi137 ceph-mon[32568]: pgmap v754: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-04-25T13:55:17.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:17 smithi094 ceph-mon[29195]: pgmap v754: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-04-25T13:55:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:17 smithi138 ceph-mon[32977]: pgmap v754: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 328 op/s 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: 0 cleanup 600 sec 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Operation Count AvgLat MaxLat 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: ---------------------------------------- 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: NTCreateX 238966 1.010 146.277 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Close 175518 0.003 0.094 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Rename 10117 4.502 121.912 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Unlink 48261 1.611 81.616 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Deltree 6 75.260 100.836 2024-04-25T13:55:18.032 INFO:tasks.workunit.client.0.smithi094.stdout: Mkdir 3 0.009 0.012 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Qpathinfo 216579 0.714 127.013 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Qfileinfo 37950 0.001 0.027 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Qfsinfo 39713 0.137 5.428 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Sfileinfo 19462 0.016 0.985 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Find 83730 0.080 91.787 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: WriteX 119118 0.015 39.511 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: ReadX 374524 0.018 42.336 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: LockX 778 1.223 90.049 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: UnlockX 778 0.975 54.827 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: Flush 16745 3.225 116.100 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T13:55:18.033 INFO:tasks.workunit.client.0.smithi094.stdout:Throughput 12.4961 MB/sec 1 clients 1 procs max_latency=146.288 ms 2024-04-25T13:55:18.034 INFO:tasks.workunit.client.0.smithi094.stderr:+ dbench 10 2024-04-25T13:55:18.034 INFO:tasks.workunit.client.0.smithi094.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T13:55:18.034 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T13:55:18.034 INFO:tasks.workunit.client.0.smithi094.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T13:55:18.034 INFO:tasks.workunit.client.0.smithi094.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-25T13:55:18.134 INFO:tasks.workunit.client.0.smithi094.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-25T13:55:18.134 INFO:tasks.workunit.client.0.smithi094.stdout:releasing clients 2024-04-25T13:55:19.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 377 134.61 MB/sec warmup 1 sec latency 50.831 ms 2024-04-25T13:55:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.529+0000 7f3935d9b640 1 -- 172.21.15.94:0/3800536313 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0318a0 con 0x7f3938103c80 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.529+0000 7f393d0a7640 1 -- 172.21.15.94:0/3800536313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938103c80 msgr2=0x7f3938104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.529+0000 7f393d0a7640 1 --2- 172.21.15.94:0/3800536313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938103c80 0x7f3938104090 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c02a410 comp rx=0 tx=0).stop 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 -- 172.21.15.94:0/3800536313 shutdown_connections 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 --2- 172.21.15.94:0/3800536313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938108980 0x7f393810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 --2- 172.21.15.94:0/3800536313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938103c80 0x7f3938104090 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 --2- 172.21.15.94:0/3800536313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 0x7f3938103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 -- 172.21.15.94:0/3800536313 >> 172.21.15.94:0/3800536313 conn(0x7f39380fdc30 msgr2=0x7f39381000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 -- 172.21.15.94:0/3800536313 shutdown_connections 2024-04-25T13:55:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 -- 172.21.15.94:0/3800536313 wait complete. 2024-04-25T13:55:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.530+0000 7f393d0a7640 1 Processor -- start 2024-04-25T13:55:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 -- start start 2024-04-25T13:55:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 0x7f393819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 0x7f39381a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f393810e560 con 0x7f3938108980 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f393810e3e0 con 0x7f3938103c80 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393d0a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f393810e6e0 con 0x7f3938102a40 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 0x7f39381a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 0x7f393819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38984/0 (socket says 172.21.15.94:38984) 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 -- 172.21.15.94:0/2118772021 learned_addr learned my addr 172.21.15.94:0/2118772021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 0x7f39381a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51492/0 (socket says 172.21.15.94:51492) 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 -- 172.21.15.94:0/2118772021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 msgr2=0x7f393819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 0x7f393819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 -- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 msgr2=0x7f39381a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 0x7f39381a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 -- 172.21.15.94:0/2118772021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381a1d70 con 0x7f3938103c80 2024-04-25T13:55:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.531+0000 7f393659c640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.532+0000 7f391ffff640 1 -- 172.21.15.94:0/2118772021 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c03f030 con 0x7f3938103c80 2024-04-25T13:55:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.532+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39381a2000 con 0x7f3938103c80 2024-04-25T13:55:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.532+0000 7f391ffff640 1 -- 172.21.15.94:0/2118772021 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c003aa0 con 0x7f3938103c80 2024-04-25T13:55:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.532+0000 7f391ffff640 1 -- 172.21.15.94:0/2118772021 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c035070 con 0x7f3938103c80 2024-04-25T13:55:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.532+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a24f0 con 0x7f3938103c80 2024-04-25T13:55:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 Processor -- start 2024-04-25T13:55:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 -- start start 2024-04-25T13:55:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f391ffff640 1 -- 172.21.15.94:0/2118772021 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f392c042030 con 0x7f3938103c80 2024-04-25T13:55:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cbc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556989c1d460 con 0x5569898f64d0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556989bd7590 con 0x5569899171b0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3944030740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556989c19420 con 0x5569898ee2f0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f393659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f393759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f393659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51500/0 (socket says 172.21.15.94:51500) 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f393759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38992/0 (socket says 172.21.15.94:38992) 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f393659c640 1 -- 172.21.15.94:0/2694510041 learned_addr learned my addr 172.21.15.94:0/2694510041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.535+0000 7f3936d9d640 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cbc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 -- 172.21.15.94:0/2694510041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 msgr2=0x556989cbc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cbc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 -- 172.21.15.94:0/2694510041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 msgr2=0x556989cbbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 -- 172.21.15.94:0/2694510041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989be40d0 con 0x5569898f64d0 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393759e640 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f391ffff640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 0x7f39200b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393659c640 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f392c107430 tx=0x7f392c107520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8522bbbd424a29a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f393759e640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 0x7f39200b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f391ffff640 1 -- 172.21.15.94:0/2118772021 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f392c0ff240 con 0x7f3938103c80 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f391d7fa640 1 -- 172.21.15.94:0/2694510041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0c35f0 con 0x5569898f64d0 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f391d7fa640 1 -- 172.21.15.94:0/2694510041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c03b460 con 0x5569898f64d0 2024-04-25T13:55:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.536+0000 7f391d7fa640 1 -- 172.21.15.94:0/2694510041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0c6ea0 con 0x5569898f64d0 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- 172.21.15.94:0/2694510041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 msgr2=0x556989cc3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f393759e640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 0x7f39200b1ff0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x556989cbcf50 tx=0x7f3928009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f392c107430 tx=0x7f392c107520 comp rx=0 tx=0).stop 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- 172.21.15.94:0/2694510041 shutdown_connections 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cc3200 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cbc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 --2- 172.21.15.94:0/2694510041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569899171b0 0x556989cbbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- 172.21.15.94:0/2694510041 >> 172.21.15.94:0/2694510041 conn(0x55698990b300 msgr2=0x556989cb3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- 172.21.15.94:0/2694510041 shutdown_connections 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- 172.21.15.94:0/2694510041 wait complete. 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 -- start start 2024-04-25T13:55:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.537+0000 7f3944030740 1 Processor -- start 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989cc5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556989cc3a10 con 0x5569899171b0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556989ba4b60 con 0x5569898f64d0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556989cc3b90 con 0x5569898ee2f0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3944030740 10 client.0 _ll_register_callbacks cb 0x556989bc92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51508/0 (socket says 172.21.15.94:51508) 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 -- 172.21.15.94:0/330551975 learned_addr learned my addr 172.21.15.94:0/330551975 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393659c640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989cc5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f3936d9d640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 -- 172.21.15.94:0/330551975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 msgr2=0x556989cc5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 -- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 msgr2=0x556989cc5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989cc5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 -- 172.21.15.94:0/330551975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698992f210 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393659c640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989cc5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f393759e640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3928003c40 tx=0x7f3928004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef4810490dab9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f391aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.538+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800fe60 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f39280045c0 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556989c2b070 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f39280047c0 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 10 client.20331 fetch_fsmap learned FSMap version 31 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5569899fb620 con 0x5569899171b0 2024-04-25T13:55:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556989c2b070 con 0x5569899171b0 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3928026070 con 0x5569899171b0 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 10 client.20331 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 10 client.20331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 20 client.20331 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569898d5d80 0x556989cd3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 4 client.20331 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556989c24ee0 con 0x5569898d5d80 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.539+0000 7f3936d9d640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569898d5d80 0x556989cd3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.540+0000 7f391aff5640 10 client.20331 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.540+0000 7f3936d9d640 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569898d5d80 0x556989cd3650 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25763b6654930735 server_cookie=540addf35ab91fb1 in_seq=0 out_seq=0 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f391aff5640 1 -- 172.21.15.94:0/330551975 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556989c24ee0 con 0x5569898d5d80 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f391aff5640 10 client.20331 handle_command_reply: tid=0 2024-04-25T13:55:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 client.20331 shutdown 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 msgr2=0x556989cccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3928003c40 tx=0x7f3928004ab0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 shutdown_connections 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569898d5d80 0x556989cd3650 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569899171b0 0x556989cccbd0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f391aff5640 0 client.20331 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989cc5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 --2- 172.21.15.94:0/330551975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 >> 172.21.15.94:0/330551975 conn(0x55698990b300 msgr2=0x556989cb6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 shutdown_connections 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 1 -- 172.21.15.94:0/330551975 wait complete. 2024-04-25T13:55:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.542+0000 7f3944030740 20 client.20331 trim_cache size 0 max 16384 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.557+0000 7f3944030740 1 Processor -- start 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.557+0000 7f3944030740 1 -- start start 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a9340 con 0x5569898f64d0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aae80 con 0x5569898d5d80 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3944030740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a9a10 con 0x5569898ee2f0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39018/0 (socket says 172.21.15.94:39018) 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3936d9d640 1 -- 172.21.15.94:0/2167066964 learned_addr learned my addr 172.21.15.94:0/2167066964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393659c640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 -- 172.21.15.94:0/2167066964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 msgr2=0x556989cc0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 -- 172.21.15.94:0/2167066964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 msgr2=0x556989cbff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 -- 172.21.15.94:0/2167066964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989ccd030 con 0x5569898f64d0 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f3936d9d640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393659c640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f393759e640 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cccaf0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f392801e720 tx=0x7f3928003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79db6dca82c6a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.558+0000 7f391cff9640 1 -- 172.21.15.94:0/2167066964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928035020 con 0x5569898f64d0 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f391cff9640 1 -- 172.21.15.94:0/2167066964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928004410 con 0x5569898f64d0 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f391cff9640 1 -- 172.21.15.94:0/2167066964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928028a20 con 0x5569898f64d0 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 -- 172.21.15.94:0/2167066964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 msgr2=0x556989cccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cccaf0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f392801e720 tx=0x7f3928003a00 comp rx=0 tx=0).stop 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 -- 172.21.15.94:0/2167066964 shutdown_connections 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898f64d0 0x556989cccaf0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cc0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 --2- 172.21.15.94:0/2167066964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898d5d80 0x556989cbff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 -- 172.21.15.94:0/2167066964 >> 172.21.15.94:0/2167066964 conn(0x55698990b300 msgr2=0x556989cc5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 -- 172.21.15.94:0/2167066964 shutdown_connections 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.559+0000 7f3944030740 1 -- 172.21.15.94:0/2167066964 wait complete. 2024-04-25T13:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 -- start start 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 Processor -- start 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989ccb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aa7b0 con 0x5569898d5d80 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aa0e0 con 0x5569898f64d0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a7320 con 0x5569898ee2f0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3944030740 10 client.0 _ll_register_callbacks cb 0x7f3928009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51534/0 (socket says 172.21.15.94:51534) 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 -- 172.21.15.94:0/1797480661 learned_addr learned my addr 172.21.15.94:0/1797480661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f393659c640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f393759e640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989ccb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 -- 172.21.15.94:0/1797480661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 msgr2=0x556989cca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 -- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 msgr2=0x556989ccb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989ccb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.560+0000 7f3936d9d640 1 -- 172.21.15.94:0/1797480661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556989ccb7a0 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3936d9d640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f39300126d0 tx=0x7f3930012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91111217ab68754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393001eb40 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f393001ece0 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3928004410 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3930005020 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 10 client.20337 fetch_fsmap learned FSMap version 31 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556989cccb00 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928004410 con 0x5569898d5d80 2024-04-25T13:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3928011d20 con 0x5569898d5d80 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 10 client.20337 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 10 client.20337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 20 client.20337 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569899171b0 0x556989d3de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 4 client.20337 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.561+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556989ba05f0 con 0x5569899171b0 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.562+0000 7f393659c640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569899171b0 0x556989d3de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.562+0000 7f391aff5640 10 client.20337 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:19.562+0000 7f393659c640 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569899171b0 0x556989d3de80 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ff18aef961170a server_cookie=4565697ef45160a in_seq=0 out_seq=0 2024-04-25T13:55:19.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:19 smithi137 ceph-mon[32568]: pgmap v755: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-04-25T13:55:19.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:19 smithi094 ceph-mon[29195]: pgmap v755: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-04-25T13:55:19.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:19 smithi138 ceph-mon[32977]: pgmap v755: 97 pgs: 97 active+clean; 45 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-04-25T13:55:20.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 549 92.73 MB/sec warmup 2 sec latency 439.209 ms 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f391aff5640 1 -- 172.21.15.94:0/1797480661 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556989ba05f0 con 0x5569899171b0 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f391aff5640 10 client.20337 handle_command_reply: tid=0 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 client.20337 shutdown 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 msgr2=0x556989cc9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f39300126d0 tx=0x7f3930012b90 comp rx=0 tx=0).stop 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 shutdown_connections 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5569899171b0 0x556989d3de80 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5569898f64d0 0x556989ccb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569898ee2f0 0x556989cca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f391aff5640 0 client.20337 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 --2- 172.21.15.94:0/1797480661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569898d5d80 0x556989cc9dc0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 >> 172.21.15.94:0/1797480661 conn(0x55698990b300 msgr2=0x556989cc3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 shutdown_connections 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 1 -- 172.21.15.94:0/1797480661 wait complete. 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.169+0000 7f3944030740 20 client.20337 trim_cache size 0 max 16384 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:55:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:55:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 msgr2=0x7f39200b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 0x7f39200b1ff0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x556989cbcf50 tx=0x7f3928009500 comp rx=0 tx=0).stop 2024-04-25T13:55:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 msgr2=0x7f393819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c031a50 comp rx=0 tx=0).stop 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 shutdown_connections 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39200afae0 0x7f39200b1ff0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938108980 0x7f39381a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938103c80 0x7f393819a770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 --2- 172.21.15.94:0/2118772021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3938102a40 0x7f393819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 >> 172.21.15.94:0/2118772021 conn(0x7f39380fdc30 msgr2=0x7f3938109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 shutdown_connections 2024-04-25T13:55:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:20.170+0000 7f393d0a7640 1 -- 172.21.15.94:0/2118772021 wait complete. 2024-04-25T13:55:21.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 975 73.40 MB/sec warmup 3 sec latency 475.650 ms 2024-04-25T13:55:21.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:21 smithi137 ceph-mon[32568]: pgmap v756: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:55:21.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:21 smithi094 ceph-mon[29195]: pgmap v756: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:55:21.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:21 smithi138 ceph-mon[32977]: pgmap v756: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-04-25T13:55:22.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 1512 65.18 MB/sec warmup 4 sec latency 92.227 ms 2024-04-25T13:55:22.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:22 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:55:22.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:22 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:55:22.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:22 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:55:23.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 2372 55.70 MB/sec warmup 5 sec latency 68.007 ms 2024-04-25T13:55:23.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:23 smithi137 ceph-mon[32568]: pgmap v757: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-04-25T13:55:23.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:23 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:23.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:23 smithi094 ceph-mon[29195]: pgmap v757: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-04-25T13:55:23.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:23 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:23.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:23 smithi138 ceph-mon[32977]: pgmap v757: 97 pgs: 97 active+clean; 49 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-04-25T13:55:23.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:23 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:24.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 2835 53.86 MB/sec warmup 6 sec latency 62.768 ms 2024-04-25T13:55:24.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:24 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:24.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:24 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:24.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:24 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:55:24.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:24 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:24.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:24 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:24.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:24 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:55:24.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:24 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:24.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:24 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:24.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:24 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:55:25.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3375 51.54 MB/sec warmup 7 sec latency 124.552 ms 2024-04-25T13:55:26.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3754 49.32 MB/sec warmup 8 sec latency 84.125 ms 2024-04-25T13:55:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:26 smithi137 ceph-mon[32568]: pgmap v758: 97 pgs: 97 active+clean; 34 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-04-25T13:55:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:26 smithi094 ceph-mon[29195]: pgmap v758: 97 pgs: 97 active+clean; 34 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-04-25T13:55:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:26 smithi138 ceph-mon[32977]: pgmap v758: 97 pgs: 97 active+clean; 34 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 498 op/s 2024-04-25T13:55:26.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:55:26.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:55:27.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 -- 172.21.15.94:0/86984580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4103c20 msgr2=0x7fada4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/86984580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4103c20 0x7fada4104030 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fad98009900 tx=0x7fad9802a410 comp rx=0 tx=0).stop 2024-04-25T13:55:27.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 -- 172.21.15.94:0/86984580 shutdown_connections 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/86984580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada4108920 0x7fada410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/86984580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4103c20 0x7fada4104030 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/86984580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada41029e0 0x7fada41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 -- 172.21.15.94:0/86984580 >> 172.21.15.94:0/86984580 conn(0x7fada40fdbd0 msgr2=0x7fada4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 -- 172.21.15.94:0/86984580 shutdown_connections 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.109+0000 7fadaa7a8640 1 -- 172.21.15.94:0/86984580 wait complete. 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 Processor -- start 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 -- start start 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada41029e0 0x7fada4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada4103c20 0x7fada41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada410e500 con 0x7fada4108920 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fada410e380 con 0x7fada41029e0 2024-04-25T13:55:27.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fadaa7a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fada410e680 con 0x7fada4103c20 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.110+0000 7fada8d45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36128/0 (socket says 172.21.15.94:36128) 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 -- 172.21.15.94:0/2438754702 learned_addr learned my addr 172.21.15.94:0/2438754702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada3fff640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada41029e0 0x7fada4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada37fe640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada4103c20 0x7fada41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 -- 172.21.15.94:0/2438754702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada4103c20 msgr2=0x7fada41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada4103c20 0x7fada41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 -- 172.21.15.94:0/2438754702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada41029e0 msgr2=0x7fada4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada41029e0 0x7fada4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 -- 172.21.15.94:0/2438754702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada41a0fa0 con 0x7fada4108920 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada8d45640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00eb10 tx=0x7fad9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada17fa640 1 -- 172.21.15.94:0/2438754702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c004b20 con 0x7fada4108920 2024-04-25T13:55:27.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada41a1290 con 0x7fada4108920 2024-04-25T13:55:27.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada17fa640 1 -- 172.21.15.94:0/2438754702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c004cc0 con 0x7fada4108920 2024-04-25T13:55:27.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fada17fa640 1 -- 172.21.15.94:0/2438754702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c010670 con 0x7fada4108920 2024-04-25T13:55:27.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.111+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41a17e0 con 0x7fada4108920 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada17fa640 1 -- 172.21.15.94:0/2438754702 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fad9c005020 con 0x7fada4108920 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 Processor -- start 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 -- start start 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433043a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c794d0 0x560433043fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560432fa8d80 con 0x560432c816b0 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560432f628b0 con 0x560432ca2390 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fadb165c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560432fa4d40 con 0x560432c794d0 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36144/0 (socket says 172.21.15.94:36144) 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 -- 172.21.15.94:0/1139286695 learned_addr learned my addr 172.21.15.94:0/1139286695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433043a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada3fff640 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c794d0 0x560433043fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada17fa640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 0x7fad8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 -- 172.21.15.94:0/1139286695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c794d0 msgr2=0x560433043fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c794d0 0x560433043fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 -- 172.21.15.94:0/1139286695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 msgr2=0x560433043a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433043a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada37fe640 1 -- 172.21.15.94:0/1139286695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560432f33a40 con 0x560432c816b0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.114+0000 7fada8d45640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 0x7fad8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.115+0000 7fada37fe640 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fad980096c0 tx=0x7fad98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd4265002f116bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.115+0000 7fada17fa640 1 -- 172.21.15.94:0/2438754702 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad9c0df280 con 0x7fada4108920 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.115+0000 7fad8a7fc640 1 -- 172.21.15.94:0/1139286695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98035040 con 0x560432c816b0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.115+0000 7fad8a7fc640 1 -- 172.21.15.94:0/1139286695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad98031650 con 0x560432c816b0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.115+0000 7fad8a7fc640 1 -- 172.21.15.94:0/1139286695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98003b30 con 0x560432c816b0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- 172.21.15.94:0/1139286695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 msgr2=0x560433044c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fada8d45640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 0x7fad8c0b2000 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0dd050 tx=0x7fad9c0ca040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fad980096c0 tx=0x7fad98000bc0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- 172.21.15.94:0/1139286695 shutdown_connections 2024-04-25T13:55:27.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c816b0 0x560433044c70 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c794d0 0x560433043fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 --2- 172.21.15.94:0/1139286695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433043a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- 172.21.15.94:0/1139286695 >> 172.21.15.94:0/1139286695 conn(0x560432c964e0 msgr2=0x56043303f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- 172.21.15.94:0/1139286695 shutdown_connections 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- 172.21.15.94:0/1139286695 wait complete. 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 -- start start 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.116+0000 7fadb165c740 1 Processor -- start 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433054dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560433045720 con 0x560432c794d0 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560432f2fd10 con 0x560432ca2390 2024-04-25T13:55:27.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5604330458a0 con 0x560432c816b0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 10 client.0 _ll_register_callbacks cb 0x560432f5c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36150/0 (socket says 172.21.15.94:36150) 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 -- 172.21.15.94:0/961088382 learned_addr learned my addr 172.21.15.94:0/961088382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada3fff640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433054dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 -- 172.21.15.94:0/961088382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 msgr2=0x56043304de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 -- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 msgr2=0x560433054dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433054dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 -- 172.21.15.94:0/961088382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560432d847d0 con 0x560432c794d0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fada37fe640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fad98001eb0 tx=0x7fad98009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0d01fe57c82f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fad837fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98035070 con 0x560432c794d0 2024-04-25T13:55:27.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad9803b970 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.117+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad98031650 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad9803bb70 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 10 client.20349 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560432cba3f0 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad98031650 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad9804dca0 con 0x560432c794d0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 10 client.20349 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 10 client.20349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 20 client.20349 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432c60f60 0x56043305ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 4 client.20349 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560432fb6990 con 0x560432c60f60 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.118+0000 7fada3fff640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432c60f60 0x56043305ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.119+0000 7fad837fe640 10 client.20349 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:27.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.119+0000 7fada3fff640 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432c60f60 0x56043305ae80 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aaa839f76a60e34 server_cookie=49a8577f6f70db82 in_seq=0 out_seq=0 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fad837fe640 1 -- 172.21.15.94:0/961088382 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560432fb6990 con 0x560432c60f60 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fad837fe640 10 client.20349 handle_command_reply: tid=0 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 client.20349 shutdown 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 msgr2=0x56043304d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fad98001eb0 tx=0x7fad98009520 comp rx=0 tx=0).stop 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 shutdown_connections 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432c60f60 0x56043305ae80 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432ca2390 0x560433054dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fad837fe640 0 client.20349 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 --2- 172.21.15.94:0/961088382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x56043304d920 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 >> 172.21.15.94:0/961088382 conn(0x560432c964e0 msgr2=0x560433042180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 shutdown_connections 2024-04-25T13:55:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 1 -- 172.21.15.94:0/961088382 wait complete. 2024-04-25T13:55:27.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.121+0000 7fadb165c740 20 client.20349 trim_cache size 0 max 16384 2024-04-25T13:55:27.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3937 44.48 MB/sec warmup 9 sec latency 45.609 ms 2024-04-25T13:55:27.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 Processor -- start 2024-04-25T13:55:27.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 -- start start 2024-04-25T13:55:27.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c60f60 0x56043304c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada41a9300 con 0x560432c794d0 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fada41aaf00 con 0x560432c60f60 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fadb165c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fada41a99d0 con 0x560432c816b0 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36164/0 (socket says 172.21.15.94:36164) 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 -- 172.21.15.94:0/3781689855 learned_addr learned my addr 172.21.15.94:0/3781689855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada37fe640 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c60f60 0x56043304c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada37fe640 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 -- 172.21.15.94:0/3781689855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 msgr2=0x56043304dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 -- 172.21.15.94:0/3781689855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c60f60 msgr2=0x56043304c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c60f60 0x56043304c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 -- 172.21.15.94:0/3781689855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560432c7a0f0 con 0x560432c794d0 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.140+0000 7fada3fff640 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fada41035e0 tx=0x7fad9400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c97a6385e2091a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.141+0000 7fad89ffb640 1 -- 172.21.15.94:0/3781689855 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94005220 con 0x560432c794d0 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.141+0000 7fad89ffb640 1 -- 172.21.15.94:0/3781689855 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94024070 con 0x560432c794d0 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.141+0000 7fad89ffb640 1 -- 172.21.15.94:0/3781689855 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940205a0 con 0x560432c794d0 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.141+0000 7fadb165c740 1 -- 172.21.15.94:0/3781689855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 msgr2=0x5604330486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.141+0000 7fadb165c740 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fada41035e0 tx=0x7fad9400b390 comp rx=0 tx=0).stop 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- 172.21.15.94:0/3781689855 shutdown_connections 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c816b0 0x56043304dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x5604330486e0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- 172.21.15.94:0/3781689855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c60f60 0x56043304c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- 172.21.15.94:0/3781689855 >> 172.21.15.94:0/3781689855 conn(0x560432c964e0 msgr2=0x56043304d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- 172.21.15.94:0/3781689855 shutdown_connections 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- 172.21.15.94:0/3781689855 wait complete. 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- start start 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 Processor -- start 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 0x560433052b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c816b0 0x560433053f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fada41aa830 con 0x560432c794d0 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fada41aa0a0 con 0x560432c816b0 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fada41a72e0 con 0x560432c60f60 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.142+0000 7fadb165c740 10 client.0 _ll_register_callbacks cb 0x7fad9803e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36170/0 (socket says 172.21.15.94:36170) 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada3fff640 1 -- 172.21.15.94:0/2184645731 learned_addr learned my addr 172.21.15.94:0/2184645731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 0x560433052b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c816b0 0x560433053f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 -- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c816b0 msgr2=0x560433053f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c816b0 0x560433053f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 -- 172.21.15.94:0/2184645731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 msgr2=0x560433053090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 -- 172.21.15.94:0/2184645731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560432f694b0 con 0x560432c60f60 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada3fff640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fada37fe640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 0x560433052b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad9805a4d0 tx=0x7fad98032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ccedec56fcc3743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fad837fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98035040 con 0x560432c60f60 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad98004220 con 0x560432c60f60 2024-04-25T13:55:27.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad94024070 con 0x560432c60f60 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad9803b980 con 0x560432c60f60 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fadb165c740 10 client.29548 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.143+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604330556c0 con 0x560432c60f60 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94024070 con 0x560432c60f60 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad980035f0 con 0x560432c60f60 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 10 client.29548 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 10 client.29548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 20 client.29548 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432ca2390 0x5604330c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 4 client.29548 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560432fb0800 con 0x560432ca2390 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fada3fff640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432ca2390 0x5604330c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fada3fff640 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432ca2390 0x5604330c56c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3520afd1956bc0da server_cookie=1fb7a28a23423db9 in_seq=0 out_seq=0 2024-04-25T13:55:27.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.144+0000 7fad837fe640 10 client.29548 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fad837fe640 1 -- 172.21.15.94:0/2184645731 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560432fb0800 con 0x560432ca2390 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fad837fe640 10 client.29548 handle_command_reply: tid=0 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 client.29548 shutdown 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 msgr2=0x560433052b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 0x560433052b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad9805a4d0 tx=0x7fad98032e90 comp rx=0 tx=0).stop 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 shutdown_connections 2024-04-25T13:55:27.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560432ca2390 0x5604330c56c0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560432c816b0 0x560433053f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fad837fe640 0 client.29548 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560432c794d0 0x560433053090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 --2- 172.21.15.94:0/2184645731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560432c60f60 0x560433052b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 >> 172.21.15.94:0/2184645731 conn(0x560432c964e0 msgr2=0x56043303d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 shutdown_connections 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 1 -- 172.21.15.94:0/2184645731 wait complete. 2024-04-25T13:55:27.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.150+0000 7fadb165c740 20 client.29548 trim_cache size 0 max 16384 2024-04-25T13:55:27.153 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:55:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:55:27.153 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:55:27.153 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 msgr2=0x7fad8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 0x7fad8c0b2000 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0dd050 tx=0x7fad9c0ca040 comp rx=0 tx=0).stop 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 msgr2=0x7fada41a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00eb10 tx=0x7fad9c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 shutdown_connections 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad8c0afaf0 0x7fad8c0b2000 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada4108920 0x7fada41a0930 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada4103c20 0x7fada41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 --2- 172.21.15.94:0/2438754702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada41029e0 0x7fada4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 >> 172.21.15.94:0/2438754702 conn(0x7fada40fdbd0 msgr2=0x7fada4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 shutdown_connections 2024-04-25T13:55:27.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.151+0000 7fadaa7a8640 1 -- 172.21.15.94:0/2438754702 wait complete. 2024-04-25T13:55:27.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:55:27.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:55:27.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- 172.21.15.94:0/39393594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d88109870 msgr2=0x7f7d8810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 --2- 172.21.15.94:0/39393594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d88109870 0x7f7d8810bc90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009ca0 tx=0x7f7d8002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- 172.21.15.94:0/39393594 shutdown_connections 2024-04-25T13:55:27.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 --2- 172.21.15.94:0/39393594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d88109870 0x7f7d8810bc90 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 --2- 172.21.15.94:0/39393594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 0x7f7d88109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 --2- 172.21.15.94:0/39393594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d881045b0 0x7f7d881069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- 172.21.15.94:0/39393594 >> 172.21.15.94:0/39393594 conn(0x7f7d880fdbd0 msgr2=0x7f7d88100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- 172.21.15.94:0/39393594 shutdown_connections 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- 172.21.15.94:0/39393594 wait complete. 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 Processor -- start 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.274+0000 7f7d8e796640 1 -- start start 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 0x7f7d8819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d88109870 0x7f7d881a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8810f5d0 con 0x7f7d881045b0 2024-04-25T13:55:27.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8810f450 con 0x7f7d88106f10 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8810f750 con 0x7f7d88109870 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36184/0 (socket says 172.21.15.94:36184) 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 -- 172.21.15.94:0/2537428314 learned_addr learned my addr 172.21.15.94:0/2537428314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d877fe640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 0x7f7d8819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8cd33640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d88109870 0x7f7d881a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 -- 172.21.15.94:0/2537428314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d88109870 msgr2=0x7f7d881a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d88109870 0x7f7d881a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 -- 172.21.15.94:0/2537428314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 msgr2=0x7f7d8819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 0x7f7d8819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 -- 172.21.15.94:0/2537428314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d881a1d10 con 0x7f7d881045b0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d87fff640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9a0 tx=0x7f7d7800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800cd80 con 0x7f7d881045b0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d780035f0 con 0x7f7d881045b0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d881a1fa0 con 0x7f7d881045b0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78010690 con 0x7f7d881045b0 2024-04-25T13:55:27.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.275+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a2460 con 0x7f7d881045b0 2024-04-25T13:55:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.277+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d880ff6f0 con 0x7f7d881045b0 2024-04-25T13:55:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.278+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7d78004000 con 0x7f7d881045b0 2024-04-25T13:55:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.278+0000 7f7d857fa640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 0x7f7d700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.278+0000 7f7d877fe640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 0x7f7d700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.278+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f7d780df010 con 0x7f7d881045b0 2024-04-25T13:55:27.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.279+0000 7f7d877fe640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 0x7f7d700b2000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d8819b270 tx=0x7f7d7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.284+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d780042f0 con 0x7f7d881045b0 2024-04-25T13:55:27.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.515+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d880637a0 con 0x7f7d881045b0 2024-04-25T13:55:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.516+0000 7f7d857fa640 1 -- 172.21.15.94:0/2537428314 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11948 (secure 0 0 0) 0x7f7d780a7b50 con 0x7f7d881045b0 2024-04-25T13:55:27.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:55:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:55:20.694859+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:55:27.516 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 32 2024-04-25T13:55:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 msgr2=0x7f7d700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 0x7f7d700b2000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d8819b270 tx=0x7f7d7c005f90 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 msgr2=0x7f7d8819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9a0 tx=0x7f7d7800ee60 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 shutdown_connections 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7d700afaf0 0x7f7d700b2000 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d88109870 0x7f7d881a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d88106f10 0x7f7d8819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 --2- 172.21.15.94:0/2537428314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d881045b0 0x7f7d8819a1d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 >> 172.21.15.94:0/2537428314 conn(0x7f7d880fdbd0 msgr2=0x7f7d88105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.518+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 shutdown_connections 2024-04-25T13:55:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.519+0000 7f7d8e796640 1 -- 172.21.15.94:0/2537428314 wait complete. 2024-04-25T13:55:27.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:55:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 -- 172.21.15.94:0/3713902838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad38108920 msgr2=0x7fad3810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 --2- 172.21.15.94:0/3713902838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad38108920 0x7fad3810ad40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fad30009ca0 tx=0x7fad3002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 -- 172.21.15.94:0/3713902838 shutdown_connections 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 --2- 172.21.15.94:0/3713902838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad38108920 0x7fad3810ad40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 --2- 172.21.15.94:0/3713902838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 --2- 172.21.15.94:0/3713902838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381029e0 0x7fad381036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 -- 172.21.15.94:0/3713902838 >> 172.21.15.94:0/3713902838 conn(0x7fad380fdbd0 msgr2=0x7fad38100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 -- 172.21.15.94:0/3713902838 shutdown_connections 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.667+0000 7fad3f013640 1 -- 172.21.15.94:0/3713902838 wait complete. 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 Processor -- start 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 -- start start 2024-04-25T13:55:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad38108920 0x7fad381a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810e4f0 con 0x7fad381029e0 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810e370 con 0x7fad38108920 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3f013640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad3810e670 con 0x7fad38103c20 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36212/0 (socket says 172.21.15.94:36212) 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 -- 172.21.15.94:0/3190338235 learned_addr learned my addr 172.21.15.94:0/3190338235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad38108920 0x7fad381a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad37fff640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 -- 172.21.15.94:0/3190338235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 msgr2=0x7fad38199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 -- 172.21.15.94:0/3190338235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad38108920 msgr2=0x7fad381a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad38108920 0x7fad381a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad3cdaf640 1 -- 172.21.15.94:0/3190338235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381a0f00 con 0x7fad381029e0 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.668+0000 7fad37fff640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00e9f0 tx=0x7fad2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad35ffb640 1 -- 172.21.15.94:0/3190338235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00ccc0 con 0x7fad381029e0 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad35ffb640 1 -- 172.21.15.94:0/3190338235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad2c0035f0 con 0x7fad381029e0 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381a11f0 con 0x7fad381029e0 2024-04-25T13:55:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad35ffb640 1 -- 172.21.15.94:0/3190338235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00ce60 con 0x7fad381029e0 2024-04-25T13:55:27.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.669+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad381a1740 con 0x7fad381029e0 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.671+0000 7fad35ffb640 1 -- 172.21.15.94:0/3190338235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fad2c003aa0 con 0x7fad381029e0 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.671+0000 7fad45f9f740 1 Processor -- start 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.671+0000 7fad45f9f740 1 -- start start 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e7236a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e694d0 0x5618e7236fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618e7198d80 con 0x5618e6e716b0 2024-04-25T13:55:27.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5618e7152a20 con 0x5618e6e92390 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad45f9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618e7194d40 con 0x5618e6e694d0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad3d5b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e7236a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad35ffb640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 0x7fad280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36218/0 (socket says 172.21.15.94:36218) 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e694d0 0x5618e7236fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 -- 172.21.15.94:0/3511917416 learned_addr learned my addr 172.21.15.94:0/3511917416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 0x7fad280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 -- 172.21.15.94:0/3511917416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e694d0 msgr2=0x5618e7236fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e694d0 0x5618e7236fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 -- 172.21.15.94:0/3511917416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 msgr2=0x5618e7236a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e7236a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 -- 172.21.15.94:0/3511917416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e715dc30 con 0x5618e6e716b0 2024-04-25T13:55:27.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad35ffb640 1 -- 172.21.15.94:0/3190338235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad2c0df060 con 0x7fad381029e0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.672+0000 7fad37fff640 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bcc0 tx=0x7fad2400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a258bed457764d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad22ffd640 1 -- 172.21.15.94:0/3511917416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24005020 con 0x5618e6e716b0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 0x7fad280b2000 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fad3002f500 tx=0x7fad3002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad22ffd640 1 -- 172.21.15.94:0/3511917416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad240047a0 con 0x5618e6e716b0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad22ffd640 1 -- 172.21.15.94:0/3511917416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240116a0 con 0x5618e6e716b0 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 -- 172.21.15.94:0/3511917416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 msgr2=0x5618e7237c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bcc0 tx=0x7fad2400f730 comp rx=0 tx=0).stop 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 -- 172.21.15.94:0/3511917416 shutdown_connections 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e716b0 0x5618e7237c30 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e694d0 0x5618e7236fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 --2- 172.21.15.94:0/3511917416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e7236a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.673+0000 7fad45f9f740 1 -- 172.21.15.94:0/3511917416 >> 172.21.15.94:0/3511917416 conn(0x5618e6e864e0 msgr2=0x5618e7230020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 -- 172.21.15.94:0/3511917416 shutdown_connections 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 -- 172.21.15.94:0/3511917416 wait complete. 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 -- start start 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 Processor -- start 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e716b0 0x5618e7240bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.674+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e92390 0x5618e7247b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad45f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5618e7238740 con 0x5618e6e694d0 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad45f9f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5618e711fdc0 con 0x5618e6e716b0 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad45f9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618e72388c0 con 0x5618e6e92390 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad45f9f740 10 client.0 _ll_register_callbacks cb 0x5618e713b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36234/0 (socket says 172.21.15.94:36234) 2024-04-25T13:55:27.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 -- 172.21.15.94:0/799320426 learned_addr learned my addr 172.21.15.94:0/799320426 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad37fff640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e716b0 0x5618e7240bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e92390 0x5618e7247b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 -- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e92390 msgr2=0x5618e7247b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e92390 0x5618e7247b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 -- 172.21.15.94:0/799320426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e716b0 msgr2=0x5618e7240bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e716b0 0x5618e7240bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 -- 172.21.15.94:0/799320426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e6eaa3f0 con 0x5618e6e694d0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad17fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00e9c0 tx=0x7fad2c0a6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b60d87b38c78d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0a7ba0 con 0x5618e6e694d0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad2c0a7910 con 0x5618e6e694d0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad240047a0 con 0x5618e6e694d0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.675+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad2c0a74a0 con 0x5618e6e694d0 2024-04-25T13:55:27.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 10 client.20373 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618e6f747d0 con 0x5618e6e694d0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad240047a0 con 0x5618e6e694d0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad2c0aff00 con 0x5618e6e694d0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 10 client.20373 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 10 client.20373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 20 client.20373 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e50f60 0x5618e724e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 4 client.20373 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618e71a6990 con 0x5618e6e50f60 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.676+0000 7fad37fff640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e50f60 0x5618e724e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.677+0000 7fad37fff640 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e50f60 0x5618e724e440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10665f5920626d33 server_cookie=d989e118700f528 in_seq=0 out_seq=0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.677+0000 7fad17fff640 10 client.20373 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.677+0000 7fad17fff640 1 -- 172.21.15.94:0/799320426 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618e71a6990 con 0x5618e6e50f60 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.677+0000 7fad17fff640 10 client.20373 handle_command_reply: tid=0 2024-04-25T13:55:27.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.677+0000 7fad45f9f740 1 client.20373 shutdown 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 msgr2=0x5618e72406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00e9c0 tx=0x7fad2c0a6d20 comp rx=0 tx=0).stop 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 shutdown_connections 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e50f60 0x5618e724e440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e92390 0x5618e7247b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e716b0 0x5618e7240bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad17fff640 0 client.20373 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 --2- 172.21.15.94:0/799320426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72406b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 >> 172.21.15.94:0/799320426 conn(0x5618e6e864e0 msgr2=0x5618e722f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 shutdown_connections 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 1 -- 172.21.15.94:0/799320426 wait complete. 2024-04-25T13:55:27.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.678+0000 7fad45f9f740 20 client.20373 trim_cache size 0 max 16384 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 Processor -- start 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 -- start start 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7236ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e723b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad381a9380 con 0x5618e6e694d0 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad381aaf80 con 0x5618e6e92390 2024-04-25T13:55:27.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad45f9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad381a9a50 con 0x5618e6e716b0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.692+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36236/0 (socket says 172.21.15.94:36236) 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 -- 172.21.15.94:0/433645729 learned_addr learned my addr 172.21.15.94:0/433645729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad37fff640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7236ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e723b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 -- 172.21.15.94:0/433645729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 msgr2=0x5618e7236ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7236ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 -- 172.21.15.94:0/433645729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 msgr2=0x5618e723b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e723b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 -- 172.21.15.94:0/433645729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e723e7d0 con 0x5618e6e694d0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad37fff640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7236ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000c00 tx=0x7fad2c003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fc3936c4300300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad227fc640 1 -- 172.21.15.94:0/433645729 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0a7ba0 con 0x5618e6e694d0 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad227fc640 1 -- 172.21.15.94:0/433645729 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad2c06c070 con 0x5618e6e694d0 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.693+0000 7fad227fc640 1 -- 172.21.15.94:0/433645729 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0aebd0 con 0x5618e6e694d0 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- 172.21.15.94:0/433645729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 msgr2=0x5618e72369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000c00 tx=0x7fad2c003e70 comp rx=0 tx=0).stop 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- 172.21.15.94:0/433645729 shutdown_connections 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e92390 0x5618e723b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7236ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 --2- 172.21.15.94:0/433645729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e694d0 0x5618e72369a0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- 172.21.15.94:0/433645729 >> 172.21.15.94:0/433645729 conn(0x5618e6e864e0 msgr2=0x5618e7240650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- 172.21.15.94:0/433645729 shutdown_connections 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- 172.21.15.94:0/433645729 wait complete. 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 -- start start 2024-04-25T13:55:27.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.694+0000 7fad45f9f740 1 Processor -- start 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e694d0 0x5618e7244920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7244e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad381aa8b0 con 0x5618e6e92390 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad381aa120 con 0x5618e6e694d0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad381a7360 con 0x5618e6e716b0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad45f9f740 10 client.0 _ll_register_callbacks cb 0x7fad2c0a6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36250/0 (socket says 172.21.15.94:36250) 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 -- 172.21.15.94:0/2675578089 learned_addr learned my addr 172.21.15.94:0/2675578089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e694d0 0x5618e7244920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad37fff640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7244e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 -- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 msgr2=0x5618e7244e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7244e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 -- 172.21.15.94:0/2675578089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e694d0 msgr2=0x5618e7244920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e694d0 0x5618e7244920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 -- 172.21.15.94:0/2675578089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e7246290 con 0x5618e6e92390 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad3d5b0640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fad30009ca0 tx=0x7fad300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33ea57ddecf1c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.695+0000 7fad17fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3002f660 con 0x5618e6e92390 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fad3002f800 con 0x5618e6e92390 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad2c06c070 con 0x5618e6e92390 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad3002fa00 con 0x5618e6e92390 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 10 client.20379 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618e7247580 con 0x5618e6e92390 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad2c06c070 con 0x5618e6e92390 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fad3004a040 con 0x5618e6e92390 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 10 client.20379 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 10 client.20379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 20 client.20379 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e263a0 0x5618e72b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 4 client.20379 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.696+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618e71a0800 con 0x5618e6e263a0 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e263a0 0x5618e72b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad17fff640 10 client.20379 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad3cdaf640 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e263a0 0x5618e72b9030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3031969f4e9ef2a server_cookie=99b60e5917fa19f0 in_seq=0 out_seq=0 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad17fff640 1 -- 172.21.15.94:0/2675578089 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618e71a0800 con 0x5618e6e263a0 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad17fff640 10 client.20379 handle_command_reply: tid=0 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 client.20379 shutdown 2024-04-25T13:55:27.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 msgr2=0x5618e7245cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fad30009ca0 tx=0x7fad300095c0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 shutdown_connections 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618e6e263a0 0x5618e72b9030 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618e6e92390 0x5618e7245cd0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618e6e716b0 0x5618e7244e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 --2- 172.21.15.94:0/2675578089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618e6e694d0 0x5618e7244920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 >> 172.21.15.94:0/2675578089 conn(0x5618e6e864e0 msgr2=0x5618e723b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad17fff640 0 client.20379 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 shutdown_connections 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 1 -- 172.21.15.94:0/2675578089 wait complete. 2024-04-25T13:55:27.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.697+0000 7fad45f9f740 20 client.20379 trim_cache size 0 max 16384 2024-04-25T13:55:27.705 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:55:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 msgr2=0x7fad280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 0x7fad280b2000 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fad3002f500 tx=0x7fad3002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 msgr2=0x7fad381993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00e9f0 tx=0x7fad2c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 shutdown_connections 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fad280afaf0 0x7fad280b2000 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad38108920 0x7fad381a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad38103c20 0x7fad38199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 --2- 172.21.15.94:0/3190338235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad381029e0 0x7fad381993c0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 >> 172.21.15.94:0/3190338235 conn(0x7fad380fdbd0 msgr2=0x7fad38104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 shutdown_connections 2024-04-25T13:55:27.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.698+0000 7fad3f013640 1 -- 172.21.15.94:0/3190338235 wait complete. 2024-04-25T13:55:27.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:55:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 -- 172.21.15.94:0/3134159346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 msgr2=0x7f3038103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 --2- 172.21.15.94:0/3134159346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038103350 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3028009900 tx=0x7f302802a410 comp rx=0 tx=0).stop 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 -- 172.21.15.94:0/3134159346 shutdown_connections 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 --2- 172.21.15.94:0/3134159346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 0x7f303810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 --2- 172.21.15.94:0/3134159346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 0x7f3038103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 --2- 172.21.15.94:0/3134159346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038103350 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 -- 172.21.15.94:0/3134159346 >> 172.21.15.94:0/3134159346 conn(0x7f30380fd340 msgr2=0x7f30380ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 -- 172.21.15.94:0/3134159346 shutdown_connections 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.822+0000 7f303e0d1640 1 -- 172.21.15.94:0/3134159346 wait complete. 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 Processor -- start 2024-04-25T13:55:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 -- start start 2024-04-25T13:55:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 0x7f3038199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 0x7f30381a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f303810e520 con 0x7f3038102f40 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f303810e3a0 con 0x7f3038104460 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f303e0d1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f303810e6a0 con 0x7f3038103920 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36254/0 (socket says 172.21.15.94:36254) 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f30377fe640 1 -- 172.21.15.94:0/2702184096 learned_addr learned my addr 172.21.15.94:0/2702184096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f3037fff640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 0x7f30381a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.823+0000 7f3036ffd640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 0x7f3038199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 -- 172.21.15.94:0/2702184096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 msgr2=0x7f3038199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 0x7f3038199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 -- 172.21.15.94:0/2702184096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 msgr2=0x7f30381a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 0x7f30381a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 -- 172.21.15.94:0/2702184096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30381a0c90 con 0x7f3038102f40 2024-04-25T13:55:27.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f30377fe640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3028031440 tx=0x7f3028003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f3034ff9640 1 -- 172.21.15.94:0/2702184096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302803f070 con 0x7f3038102f40 2024-04-25T13:55:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f3034ff9640 1 -- 172.21.15.94:0/2702184096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3028031c50 con 0x7f3038102f40 2024-04-25T13:55:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30381a0f20 con 0x7f3038102f40 2024-04-25T13:55:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f3034ff9640 1 -- 172.21.15.94:0/2702184096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028035030 con 0x7f3038102f40 2024-04-25T13:55:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.824+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30381a1330 con 0x7f3038102f40 2024-04-25T13:55:27.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 Processor -- start 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3034ff9640 1 -- 172.21.15.94:0/2702184096 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3028042070 con 0x7f3038102f40 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 -- start start 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 0x55cdc54dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc54dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cdc5441d80 con 0x55cdc51124d0 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cdc53fba20 con 0x55cdc513b390 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3044f80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cdc543dd40 con 0x55cdc511a6b0 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3037fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 0x55cdc54dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36262/0 (socket says 172.21.15.94:36262) 2024-04-25T13:55:27.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f30377fe640 1 -- 172.21.15.94:0/3182952402 learned_addr learned my addr 172.21.15.94:0/3182952402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.827+0000 7f3036ffd640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc54dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 -- 172.21.15.94:0/3182952402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 msgr2=0x55cdc54dea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc54dea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 -- 172.21.15.94:0/3182952402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 msgr2=0x55cdc54dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 0x55cdc54dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f3034ff9640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 0x7f30200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 -- 172.21.15.94:0/3182952402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc5406c30 con 0x55cdc51124d0 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f3036ffd640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc54dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f3037fff640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 0x55cdc54dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f3037fff640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 0x7f30200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f30377fe640 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f30280c3f60 tx=0x7f30280c4050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b9f4b77dfaf65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f3034ff9640 1 -- 172.21.15.94:0/2702184096 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f30280ff4b0 con 0x7f3038102f40 2024-04-25T13:55:27.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f301dffb640 1 -- 172.21.15.94:0/3182952402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028049030 con 0x55cdc51124d0 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f301dffb640 1 -- 172.21.15.94:0/3182952402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30280d4880 con 0x55cdc51124d0 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.828+0000 7f301dffb640 1 -- 172.21.15.94:0/3182952402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280c7710 con 0x55cdc51124d0 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- 172.21.15.94:0/3182952402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 msgr2=0x55cdc54ddd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3037fff640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 0x7f30200b2000 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55cdc54de860 tx=0x7f3030009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f30280c3f60 tx=0x7f30280c4050 comp rx=0 tx=0).stop 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- 172.21.15.94:0/3182952402 shutdown_connections 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc54dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc51124d0 0x55cdc54ddd90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 --2- 172.21.15.94:0/3182952402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc513b390 0x55cdc54dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- 172.21.15.94:0/3182952402 >> 172.21.15.94:0/3182952402 conn(0x55cdc512f4e0 msgr2=0x55cdc54d9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- 172.21.15.94:0/3182952402 shutdown_connections 2024-04-25T13:55:27.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- 172.21.15.94:0/3182952402 wait complete. 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.829+0000 7f3044f80740 1 -- start start 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 Processor -- start 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc54e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 0x55cdc54f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cdc54df2f0 con 0x55cdc511a6b0 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cdc53c8dc0 con 0x55cdc51124d0 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cdc54df470 con 0x55cdc513b390 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3044f80740 10 client.0 _ll_register_callbacks cb 0x55cdc53e4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36270/0 (socket says 172.21.15.94:36270) 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 -- 172.21.15.94:0/429115232 learned_addr learned my addr 172.21.15.94:0/429115232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f30377fe640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc54e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3037fff640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 0x55cdc54f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 -- 172.21.15.94:0/429115232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 msgr2=0x55cdc54f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 0x55cdc54f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 -- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 msgr2=0x55cdc54e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc54e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3036ffd640 1 -- 172.21.15.94:0/429115232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc51533f0 con 0x55cdc511a6b0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f3037fff640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 0x55cdc54f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.830+0000 7f30377fe640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc54e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3036ffd640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f302c010820 tx=0x7f302c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356ced062ee8a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c00fc50 con 0x55cdc511a6b0 2024-04-25T13:55:27.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f302c00fdf0 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cdc544f990 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f302c005020 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 10 client.20391 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdc521d7d0 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cdc544f990 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f302c023430 con 0x55cdc511a6b0 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 10 client.20391 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 10 client.20391 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 20 client.20391 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc50f9f60 0x55cdc54f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 4 client.20391 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.831+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc5449800 con 0x55cdc50f9f60 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.832+0000 7f30377fe640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc50f9f60 0x55cdc54f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.832+0000 7f30377fe640 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc50f9f60 0x55cdc54f72e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd43cba606d23306 server_cookie=64c97289b7d927df in_seq=0 out_seq=0 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.832+0000 7f3016ffd640 10 client.20391 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3016ffd640 1 -- 172.21.15.94:0/429115232 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cdc5449800 con 0x55cdc50f9f60 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3016ffd640 10 client.20391 handle_command_reply: tid=0 2024-04-25T13:55:27.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 client.20391 shutdown 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 msgr2=0x55cdc54e9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f302c010820 tx=0x7f302c010ce0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 shutdown_connections 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc50f9f60 0x55cdc54f72e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc513b390 0x55cdc54f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc511a6b0 0x55cdc54e9ac0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3016ffd640 0 client.20391 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 --2- 172.21.15.94:0/429115232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc54e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 >> 172.21.15.94:0/429115232 conn(0x55cdc512f4e0 msgr2=0x55cdc54dbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 shutdown_connections 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 1 -- 172.21.15.94:0/429115232 wait complete. 2024-04-25T13:55:27.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.833+0000 7f3044f80740 20 client.20391 trim_cache size 0 max 16384 2024-04-25T13:55:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 Processor -- start 2024-04-25T13:55:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 -- start start 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc51124d0 0x55cdc54e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc511a6b0 0x55cdc54e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30381a93c0 con 0x55cdc50f9f60 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30381aafc0 con 0x55cdc511a6b0 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3044f80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30381a9a90 con 0x55cdc51124d0 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36276/0 (socket says 172.21.15.94:36276) 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 -- 172.21.15.94:0/2815967929 learned_addr learned my addr 172.21.15.94:0/2815967929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3037fff640 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc511a6b0 0x55cdc54e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f3036ffd640 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc51124d0 0x55cdc54e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 -- 172.21.15.94:0/2815967929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc51124d0 msgr2=0x55cdc54e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc51124d0 0x55cdc54e5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 -- 172.21.15.94:0/2815967929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc511a6b0 msgr2=0x55cdc54e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc511a6b0 0x55cdc54e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.847+0000 7f30377fe640 1 -- 172.21.15.94:0/2815967929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc54ea620 con 0x55cdc50f9f60 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f30377fe640 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f30280d4a40 tx=0x7f30280c40c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a8e29a6ed5a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f301d7fa640 1 -- 172.21.15.94:0/2815967929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028049040 con 0x55cdc50f9f60 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f301d7fa640 1 -- 172.21.15.94:0/2815967929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3028102cc0 con 0x55cdc50f9f60 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f301d7fa640 1 -- 172.21.15.94:0/2815967929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280d4a70 con 0x55cdc50f9f60 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 -- 172.21.15.94:0/2815967929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 msgr2=0x55cdc54e82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f30280d4a40 tx=0x7f30280c40c0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 -- 172.21.15.94:0/2815967929 shutdown_connections 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc511a6b0 0x55cdc54e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc51124d0 0x55cdc54e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.848+0000 7f3044f80740 1 --2- 172.21.15.94:0/2815967929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc54e82b0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- 172.21.15.94:0/2815967929 >> 172.21.15.94:0/2815967929 conn(0x55cdc512f4e0 msgr2=0x55cdc54e7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- 172.21.15.94:0/2815967929 shutdown_connections 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- 172.21.15.94:0/2815967929 wait complete. 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- start start 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 Processor -- start 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc55588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc555f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30381aa8f0 con 0x55cdc50f9f60 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30381aa160 con 0x55cdc51124d0 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30381a73a0 con 0x55cdc511a6b0 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f3044f80740 10 client.0 _ll_register_callbacks cb 0x7f302c00a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.849+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3037fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc555f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3036ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc55588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36278/0 (socket says 172.21.15.94:36278) 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 -- 172.21.15.94:0/414980385 learned_addr learned my addr 172.21.15.94:0/414980385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 -- 172.21.15.94:0/414980385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 msgr2=0x55cdc555f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc555f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 -- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 msgr2=0x55cdc55588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc55588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 -- 172.21.15.94:0/414980385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc555fd90 con 0x55cdc50f9f60 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f30377fe640 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f30280cfff0 tx=0x7f302803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438cd424974610f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028049070 con 0x55cdc50f9f60 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f30280fcfc0 con 0x55cdc50f9f60 2024-04-25T13:55:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3028102cc0 con 0x55cdc50f9f60 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30280d4c70 con 0x55cdc50f9f60 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3044f80740 10 client.20397 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdc5561010 con 0x55cdc50f9f60 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.850+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3028102cc0 con 0x55cdc50f9f60 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f30280dcce0 con 0x55cdc50f9f60 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 10 client.20397 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 10 client.20397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 20 client.20397 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc513b390 0x55cdc5563670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 4 client.20397 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdc53c4610 con 0x55cdc513b390 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3036ffd640 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc513b390 0x55cdc5563670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3016ffd640 10 client.20397 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:55:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3036ffd640 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc513b390 0x55cdc5563670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27188388d8658a5b server_cookie=75f3dead6372e7e3 in_seq=0 out_seq=0 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3016ffd640 1 -- 172.21.15.94:0/414980385 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdc53c4610 con 0x55cdc513b390 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3016ffd640 10 client.20397 handle_command_reply: tid=0 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.851+0000 7f3044f80740 1 client.20397 shutdown 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 msgr2=0x55cdc55583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f30280cfff0 tx=0x7f302803b460 comp rx=0 tx=0).stop 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 shutdown_connections 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55cdc513b390 0x55cdc5563670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc511a6b0 0x55cdc555f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdc51124d0 0x55cdc55588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3016ffd640 0 client.20397 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 --2- 172.21.15.94:0/414980385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cdc50f9f60 0x55cdc55583a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 >> 172.21.15.94:0/414980385 conn(0x55cdc512f4e0 msgr2=0x55cdc54dbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 shutdown_connections 2024-04-25T13:55:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 1 -- 172.21.15.94:0/414980385 wait complete. 2024-04-25T13:55:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f3044f80740 20 client.20397 trim_cache size 0 max 16384 2024-04-25T13:55:27.859 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 msgr2=0x7f30200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 0x7f30200b2000 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55cdc54de860 tx=0x7f3030009500 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 msgr2=0x7f3038199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3028031440 tx=0x7f3028003c10 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 shutdown_connections 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30200afaf0 0x7f30200b2000 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3038104460 0x7f30381a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103920 0x7f3038199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 --2- 172.21.15.94:0/2702184096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3038102f40 0x7f3038199150 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 >> 172.21.15.94:0/2702184096 conn(0x7f30380fd340 msgr2=0x7f3038104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 shutdown_connections 2024-04-25T13:55:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.852+0000 7f303e0d1640 1 -- 172.21.15.94:0/2702184096 wait complete. 2024-04-25T13:55:27.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:55:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 -- 172.21.15.94:0/3012008669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8103ae0 msgr2=0x7f37c81047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/3012008669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8103ae0 0x7f37c81047e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009900 tx=0x7f37b802a410 comp rx=0 tx=0).stop 2024-04-25T13:55:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 -- 172.21.15.94:0/3012008669 shutdown_connections 2024-04-25T13:55:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/3012008669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8109a20 0x7f37c810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/3012008669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 0x7f37c8105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/3012008669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8103ae0 0x7f37c81047e0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 -- 172.21.15.94:0/3012008669 >> 172.21.15.94:0/3012008669 conn(0x7f37c80fecd0 msgr2=0x7f37c8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 -- 172.21.15.94:0/3012008669 shutdown_connections 2024-04-25T13:55:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.979+0000 7f37ccd1c640 1 -- 172.21.15.94:0/3012008669 wait complete. 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 Processor -- start 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 -- start start 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8103ae0 0x7f37c819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 0x7f37c819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37c810f5e0 con 0x7f37c8109a20 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37c810f460 con 0x7f37c8104d20 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37ccd1c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37c810f760 con 0x7f37c8103ae0 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36302/0 (socket says 172.21.15.94:36302) 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37c6d9d640 1 -- 172.21.15.94:0/99499594 learned_addr learned my addr 172.21.15.94:0/99499594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.980+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 0x7f37c819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c659c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8103ae0 0x7f37c819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 -- 172.21.15.94:0/99499594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8103ae0 msgr2=0x7f37c819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8103ae0 0x7f37c819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 -- 172.21.15.94:0/99499594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 msgr2=0x7f37c819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 0x7f37c819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 -- 172.21.15.94:0/99499594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c81a1d50 con 0x7f37c8109a20 2024-04-25T13:55:27.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37c6d9d640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f37c000eaa0 tx=0x7f37c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37b77fe640 1 -- 172.21.15.94:0/99499594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c000cdb0 con 0x7f37c8109a20 2024-04-25T13:55:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37b77fe640 1 -- 172.21.15.94:0/99499594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c0005020 con 0x7f37c8109a20 2024-04-25T13:55:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37b77fe640 1 -- 172.21.15.94:0/99499594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00116d0 con 0x7f37c8109a20 2024-04-25T13:55:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c81a1fe0 con 0x7f37c8109a20 2024-04-25T13:55:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.981+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a24a0 con 0x7f37c8109a20 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.983+0000 7f37d3bcc740 1 Processor -- start 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.983+0000 7f37b77fe640 1 -- 172.21.15.94:0/99499594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f37c00051d0 con 0x7f37c8109a20 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 -- start start 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17647e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e17647fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e1763e2d80 con 0x55e1760b34d0 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e17639ca20 con 0x55e1760dc390 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1763ded40 con 0x55e1760bb6b0 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17647e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36306/0 (socket says 172.21.15.94:36306) 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 -- 172.21.15.94:0/2492893495 learned_addr learned my addr 172.21.15.94:0/2492893495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e17647fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37b77fe640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 0x7f37b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 -- 172.21.15.94:0/2492893495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 msgr2=0x55e17647fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e17647fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 -- 172.21.15.94:0/2492893495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 msgr2=0x55e17647e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17647e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.984+0000 7f37c659c640 1 -- 172.21.15.94:0/2492893495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1763a7c30 con 0x55e1760b34d0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37c6d9d640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 0x7f37b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37c659c640 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cfb94311db8989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37b77fe640 1 -- 172.21.15.94:0/99499594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37c00dee10 con 0x7f37c8109a20 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37b4ff9640 1 -- 172.21.15.94:0/2492893495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b803f070 con 0x55e1760b34d0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37b4ff9640 1 -- 172.21.15.94:0/2492893495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8004040 con 0x55e1760b34d0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.985+0000 7f37b4ff9640 1 -- 172.21.15.94:0/2492893495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8035070 con 0x55e1760b34d0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2492893495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 msgr2=0x55e17647ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37c6d9d640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 0x7f37b00b2000 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a32d0 tx=0x7f37c00d59a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b8032b20 comp rx=0 tx=0).stop 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2492893495 shutdown_connections 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e17647fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e17647ed90 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2492893495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17647e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2492893495 >> 172.21.15.94:0/2492893495 conn(0x55e1760d04e0 msgr2=0x55e17647a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2492893495 shutdown_connections 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2492893495 wait complete. 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 -- start start 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.986+0000 7f37d3bcc740 1 Processor -- start 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e176488a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17648f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e1764804d0 con 0x55e1760b34d0 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e176369dc0 con 0x55e1760dc390 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e176480650 con 0x55e1760bb6b0 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 10 client.0 _ll_register_callbacks cb 0x55e176385600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36320/0 (socket says 172.21.15.94:36320) 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2621135835 learned_addr learned my addr 172.21.15.94:0/2621135835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c659c640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e176488a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 msgr2=0x55e176488a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e176488a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2621135835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 msgr2=0x55e17648f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:27.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17648f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2621135835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1760f43f0 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f37bc004620 tx=0x7f37bc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82299eeeec5e4d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37adffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc011940 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37bc009040 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.987+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b8004040 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f37bc011ae0 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 10 client.20409 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e1761be7d0 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8004040 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f37bc0051e0 con 0x55e1760b34d0 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 10 client.20409 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 10 client.20409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:55:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 20 client.20409 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e17609af60 0x55e1764961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 4 client.20409 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1763f0990 con 0x55e17609af60 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.988+0000 7f37c659c640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e17609af60 0x55e1764961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37adffb640 10 client.20409 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37c659c640 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e17609af60 0x55e1764961e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=594bda13a087aea server_cookie=c18fac3c78add460 in_seq=0 out_seq=0 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37adffb640 1 -- 172.21.15.94:0/2621135835 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e1763f0990 con 0x55e17609af60 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37adffb640 10 client.20409 handle_command_reply: tid=0 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37d3bcc740 1 client.20409 shutdown 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 msgr2=0x55e176488520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.990+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f37bc004620 tx=0x7f37bc003a80 comp rx=0 tx=0).stop 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 shutdown_connections 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e17609af60 0x55e1764961e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760dc390 0x55e17648f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37adffb640 0 client.20409 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760bb6b0 0x55e176488a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2621135835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760b34d0 0x55e176488520 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 >> 172.21.15.94:0/2621135835 conn(0x55e1760d04e0 msgr2=0x55e17647cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 shutdown_connections 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2621135835 wait complete. 2024-04-25T13:55:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:27.991+0000 7f37d3bcc740 20 client.20409 trim_cache size 0 max 16384 2024-04-25T13:55:28.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 Processor -- start 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 -- start start 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760b34d0 0x55e176488800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760dc390 0x55e17649af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ab540 con 0x55e1760bb6b0 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ad140 con 0x55e1760b34d0 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81abc10 con 0x55e1760dc390 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36336/0 (socket says 172.21.15.94:36336) 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 -- 172.21.15.94:0/3459406775 learned_addr learned my addr 172.21.15.94:0/3459406775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760b34d0 0x55e176488800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760dc390 0x55e17649af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 -- 172.21.15.94:0/3459406775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760dc390 msgr2=0x55e17649af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760dc390 0x55e17649af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 -- 172.21.15.94:0/3459406775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760b34d0 msgr2=0x55e176488800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760b34d0 0x55e176488800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 -- 172.21.15.94:0/3459406775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e176475500 con 0x55e1760bb6b0 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.005+0000 7f37c659c640 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f37b802a930 tx=0x7f37b802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b183255d6e8bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.006+0000 7f37affff640 1 -- 172.21.15.94:0/3459406775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8054070 con 0x55e1760bb6b0 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.006+0000 7f37affff640 1 -- 172.21.15.94:0/3459406775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8005030 con 0x55e1760bb6b0 2024-04-25T13:55:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.006+0000 7f37affff640 1 -- 172.21.15.94:0/3459406775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b803f070 con 0x55e1760bb6b0 2024-04-25T13:55:28.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.006+0000 7f37d3bcc740 1 -- 172.21.15.94:0/3459406775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 msgr2=0x55e17649aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.006+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f37b802a930 tx=0x7f37b802a410 comp rx=0 tx=0).stop 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- 172.21.15.94:0/3459406775 shutdown_connections 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760dc390 0x55e17649af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760bb6b0 0x55e17649aa20 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/3459406775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760b34d0 0x55e176488800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- 172.21.15.94:0/3459406775 >> 172.21.15.94:0/3459406775 conn(0x55e1760d04e0 msgr2=0x55e1764880f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- 172.21.15.94:0/3459406775 shutdown_connections 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- 172.21.15.94:0/3459406775 wait complete. 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- start start 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 Processor -- start 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760b34d0 0x55e17648e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760bb6b0 0x55e17648e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81aca70 con 0x55e1760dc390 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ac2e0 con 0x55e1760bb6b0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81a9520 con 0x55e1760b34d0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37d3bcc740 10 client.0 _ll_register_callbacks cb 0x7f37bc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37c5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36346/0 (socket says 172.21.15.94:36346) 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2682381691 learned_addr learned my addr 172.21.15.94:0/2682381691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37c659c640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760bb6b0 0x55e17648e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.007+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760b34d0 0x55e17648e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760b34d0 msgr2=0x55e17648e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760b34d0 0x55e17648e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2682381691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760bb6b0 msgr2=0x55e17648e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760bb6b0 0x55e17648e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 -- 172.21.15.94:0/2682381691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e176500f20 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37c5d9b640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009730 tx=0x7f37bc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaedf8d82f2332f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc01f910 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37bc01fab0 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b8005030 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f37bc01fcb0 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37d3bcc740 10 client.20415 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e1765020e0 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8005030 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f37bc042400 con 0x55e1760dc390 2024-04-25T13:55:28.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.008+0000 7f37d3bcc740 10 client.20415 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37d3bcc740 10 client.20415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37d3bcc740 20 client.20415 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1760703a0 0x55e176504990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37d3bcc740 4 client.20415 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1763ea800 con 0x55e1760703a0 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37c659c640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1760703a0 0x55e176504990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37c659c640 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1760703a0 0x55e176504990 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76f4ed1c3823e201 server_cookie=ef3d310cf7cd9c4b in_seq=0 out_seq=0 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37adffb640 10 client.20415 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:55:28.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37adffb640 1 -- 172.21.15.94:0/2682381691 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1763ea800 con 0x55e1760703a0 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.009+0000 7f37adffb640 10 client.20415 handle_command_reply: tid=0 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 client.20415 shutdown 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 msgr2=0x55e1765009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009730 tx=0x7f37bc009ac0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 shutdown_connections 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e1760703a0 0x55e176504990 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1760dc390 0x55e1765009e0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e1760bb6b0 0x55e17648e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37adffb640 0 client.20415 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 --2- 172.21.15.94:0/2682381691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1760b34d0 0x55e17648e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 >> 172.21.15.94:0/2682381691 conn(0x55e1760d04e0 msgr2=0x55e17647d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 shutdown_connections 2024-04-25T13:55:28.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 1 -- 172.21.15.94:0/2682381691 wait complete. 2024-04-25T13:55:28.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37d3bcc740 20 client.20415 trim_cache size 0 max 16384 2024-04-25T13:55:28.017 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 msgr2=0x7f37b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 0x7f37b00b2000 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a32d0 tx=0x7f37c00d59a0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 msgr2=0x7f37c81a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f37c000eaa0 tx=0x7f37c000ef60 comp rx=0 tx=0).stop 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 shutdown_connections 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37b00afaf0 0x7f37b00b2000 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c8109a20 0x7f37c81a16e0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c8104d20 0x7f37c819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 --2- 172.21.15.94:0/99499594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37c8103ae0 0x7f37c819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 >> 172.21.15.94:0/99499594 conn(0x7f37c80fecd0 msgr2=0x7f37c8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 shutdown_connections 2024-04-25T13:55:28.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.010+0000 7f37ccd1c640 1 -- 172.21.15.94:0/99499594 wait complete. 2024-04-25T13:55:28.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:55:28.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 4104 40.09 MB/sec warmup 10 sec latency 88.768 ms 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 -- 172.21.15.94:0/3543095029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04108920 msgr2=0x7fce0410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3543095029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04108920 0x7fce0410ad40 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009ca0 tx=0x7fcdfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 -- 172.21.15.94:0/3543095029 shutdown_connections 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3543095029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04108920 0x7fce0410ad40 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3543095029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103c20 0x7fce04104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3543095029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce041029e0 0x7fce041036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 -- 172.21.15.94:0/3543095029 >> 172.21.15.94:0/3543095029 conn(0x7fce040fdbd0 msgr2=0x7fce04100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 -- 172.21.15.94:0/3543095029 shutdown_connections 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.135+0000 7fce08aa7640 1 -- 172.21.15.94:0/3543095029 wait complete. 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 Processor -- start 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 -- start start 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce041029e0 0x7fce041993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04108920 0x7fce041a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce0410e510 con 0x7fce04103c20 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce0410e390 con 0x7fce04108920 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce08aa7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce0410e690 con 0x7fce041029e0 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36354/0 (socket says 172.21.15.94:36354) 2024-04-25T13:55:28.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 -- 172.21.15.94:0/3481255182 learned_addr learned my addr 172.21.15.94:0/3481255182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04108920 0x7fce041a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce0259c640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce041029e0 0x7fce041993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 -- 172.21.15.94:0/3481255182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce041029e0 msgr2=0x7fce041993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce041029e0 0x7fce041993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 -- 172.21.15.94:0/3481255182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04108920 msgr2=0x7fce041a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04108920 0x7fce041a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.136+0000 7fce01d9b640 1 -- 172.21.15.94:0/3481255182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041a0f20 con 0x7fce04103c20 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800c970 tx=0x7fcdf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fcdf37fe640 1 -- 172.21.15.94:0/3481255182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8007bb0 con 0x7fce04103c20 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fcdf37fe640 1 -- 172.21.15.94:0/3481255182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf8007d50 con 0x7fce04103c20 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fcdf37fe640 1 -- 172.21.15.94:0/3481255182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800f450 con 0x7fce04103c20 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce041a1210 con 0x7fce04103c20 2024-04-25T13:55:28.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.137+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a1710 con 0x7fce04103c20 2024-04-25T13:55:28.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.139+0000 7fce0fa30740 1 Processor -- start 2024-04-25T13:55:28.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 -- start start 2024-04-25T13:55:28.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fcdf37fe640 1 -- 172.21.15.94:0/3481255182 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcdf8004f20 con 0x7fce04103c20 2024-04-25T13:55:28.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555fc14fd80 con 0x5555fbe286b0 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555fc109a20 con 0x5555fbe49390 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0fa30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555fc14bd40 con 0x5555fbe204d0 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce02d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36366/0 (socket says 172.21.15.94:36366) 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce02d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:45948/0 (socket says 172.21.15.94:45948) 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 -- 172.21.15.94:0/1632139666 learned_addr learned my addr 172.21.15.94:0/1632139666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce0259c640 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 -- 172.21.15.94:0/1632139666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 msgr2=0x5555fc1ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 -- 172.21.15.94:0/1632139666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 msgr2=0x5555fc1eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce01d9b640 1 -- 172.21.15.94:0/1632139666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fc114c30 con 0x5555fbe286b0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.140+0000 7fce02d9d640 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fcdf37fe640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 0x7fcdec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80e2b80 tx=0x7fcdf80e7520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9fa752f8c63601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 0x7fcdec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fcdf0ff9640 1 -- 172.21.15.94:0/1632139666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80a2d50 con 0x5555fbe286b0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fcdf37fe640 1 -- 172.21.15.94:0/3481255182 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcdf80dea50 con 0x7fce04103c20 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fcdf0ff9640 1 -- 172.21.15.94:0/1632139666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf80055f0 con 0x5555fbe286b0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.141+0000 7fcdf0ff9640 1 -- 172.21.15.94:0/1632139666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80a6600 con 0x5555fbe286b0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- 172.21.15.94:0/1632139666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 msgr2=0x5555fc1eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80e2b80 tx=0x7fcdf80e7520 comp rx=0 tx=0).stop 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 0x7fcdec0b2000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5555fc1ec860 tx=0x7fcdfc02f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- 172.21.15.94:0/1632139666 shutdown_connections 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe286b0 0x5555fc1eca20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1632139666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe49390 0x5555fc1eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- 172.21.15.94:0/1632139666 >> 172.21.15.94:0/1632139666 conn(0x5555fbe3d4e0 msgr2=0x5555fc1e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- 172.21.15.94:0/1632139666 shutdown_connections 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- 172.21.15.94:0/1632139666 wait complete. 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 -- start start 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.142+0000 7fce0fa30740 1 Processor -- start 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 0x5555fc1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5555fc1ed530 con 0x5555fbe49390 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555fc0d6dc0 con 0x5555fbe204d0 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555fc1ed6b0 con 0x5555fbe286b0 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0fa30740 10 client.0 _ll_register_callbacks cb 0x5555fc0f2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:28.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 0x5555fc1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36372/0 (socket says 172.21.15.94:36372) 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 -- 172.21.15.94:0/3616390973 learned_addr learned my addr 172.21.15.94:0/3616390973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 -- 172.21.15.94:0/3616390973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 msgr2=0x5555fc1f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 -- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 msgr2=0x5555fc1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 0x5555fc1f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 -- 172.21.15.94:0/3616390973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fbe613f0 con 0x5555fbe49390 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce0259c640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 0x5555fc1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc035000 tx=0x7fcdfc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2430dc057a6a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.143+0000 7fcde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:28.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004450 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcdfc0045f0 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555fc15d990 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc0047f0 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 10 client.20427 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5555fbf2b7d0 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5555fc15d990 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcdfc051070 con 0x5555fbe49390 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 10 client.20427 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 10 client.20427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 20 client.20427 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe07f60 0x5555fc203230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 4 client.20427 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555fc157800 con 0x5555fbe07f60 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.144+0000 7fce0259c640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe07f60 0x5555fc203230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.145+0000 7fcde9ffb640 10 client.20427 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:55:28.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.145+0000 7fce0259c640 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe07f60 0x5555fc203230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76e0c26ee078c345 server_cookie=5448578261b27046 in_seq=0 out_seq=0 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3616390973 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555fc157800 con 0x5555fbe07f60 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fcde9ffb640 10 client.20427 handle_command_reply: tid=0 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 client.20427 shutdown 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 msgr2=0x5555fc1fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc035000 tx=0x7fcdfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 shutdown_connections 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe07f60 0x5555fc203230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe49390 0x5555fc1fc9b0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fcde9ffb640 0 client.20427 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3616390973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe204d0 0x5555fc1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 >> 172.21.15.94:0/3616390973 conn(0x5555fbe3d4e0 msgr2=0x5555fc1e9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 shutdown_connections 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 1 -- 172.21.15.94:0/3616390973 wait complete. 2024-04-25T13:55:28.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.146+0000 7fce0fa30740 20 client.20427 trim_cache size 0 max 16384 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 Processor -- start 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 -- start start 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe07f60 0x5555fc1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce041aa3e0 con 0x5555fbe204d0 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce041abfe0 con 0x5555fbe07f60 2024-04-25T13:55:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.160+0000 7fce0fa30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce041aaab0 con 0x5555fbe286b0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36374/0 (socket says 172.21.15.94:36374) 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 -- 172.21.15.94:0/1724950727 learned_addr learned my addr 172.21.15.94:0/1724950727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce0259c640 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe07f60 0x5555fc1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce02d9d640 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 -- 172.21.15.94:0/1724950727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 msgr2=0x5555fc1e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 -- 172.21.15.94:0/1724950727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe07f60 msgr2=0x5555fc1e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe07f60 0x5555fc1e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 -- 172.21.15.94:0/1724950727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fc1fce10 con 0x5555fbe204d0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fce01d9b640 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80dc820 tx=0x7fcdf800a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5531e75e29d35928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fcdebfff640 1 -- 172.21.15.94:0/1724950727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801d040 con 0x5555fbe204d0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fcdebfff640 1 -- 172.21.15.94:0/1724950727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf80a7590 con 0x5555fbe204d0 2024-04-25T13:55:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.161+0000 7fcdebfff640 1 -- 172.21.15.94:0/1724950727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80a67e0 con 0x5555fbe204d0 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- 172.21.15.94:0/1724950727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 msgr2=0x5555fc1fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80dc820 tx=0x7fcdf800a4b0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- 172.21.15.94:0/1724950727 shutdown_connections 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe286b0 0x5555fc1e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe204d0 0x5555fc1fc8d0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 --2- 172.21.15.94:0/1724950727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe07f60 0x5555fc1e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- 172.21.15.94:0/1724950727 >> 172.21.15.94:0/1724950727 conn(0x5555fbe3d4e0 msgr2=0x5555fc1f01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- 172.21.15.94:0/1724950727 shutdown_connections 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- 172.21.15.94:0/1724950727 wait complete. 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 -- start start 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.162+0000 7fce0fa30740 1 Processor -- start 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe286b0 0x5555fc2695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ab910 con 0x5555fbe07f60 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ab180 con 0x5555fbe286b0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce041a83c0 con 0x5555fbe204d0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 10 client.0 _ll_register_callbacks cb 0x7fcdfc02f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36382/0 (socket says 172.21.15.94:36382) 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 -- 172.21.15.94:0/3389965797 learned_addr learned my addr 172.21.15.94:0/3389965797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce02d9d640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe286b0 0x5555fc2695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 -- 172.21.15.94:0/3389965797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 msgr2=0x5555fc1f99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1f99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 -- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe286b0 msgr2=0x5555fc2695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe286b0 0x5555fc2695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 -- 172.21.15.94:0/3389965797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fc1fc990 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0259c640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x5555fc1eb530 tx=0x7fcdf40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bede2c14dd66bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fcde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40207a0 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcdf4020940 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.163+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf80a7590 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcdf40134e0 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 10 client.20433 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5555fc26a9e0 con 0x5555fbe07f60 2024-04-25T13:55:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf80a7590 con 0x5555fbe07f60 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcdf4032650 con 0x5555fbe07f60 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 10 client.20433 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 10 client.20433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 20 client.20433 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe49390 0x5555fc26d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 4 client.20433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555fc0d2610 con 0x5555fbe49390 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.164+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe49390 0x5555fc26d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fcde9ffb640 10 client.20433 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce01d9b640 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe49390 0x5555fc26d3b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b5bea9fb1a37908 server_cookie=228fd3cd16f5c702 in_seq=0 out_seq=0 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fcde9ffb640 1 -- 172.21.15.94:0/3389965797 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555fc0d2610 con 0x5555fbe49390 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fcde9ffb640 10 client.20433 handle_command_reply: tid=0 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 client.20433 shutdown 2024-04-25T13:55:28.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 msgr2=0x5555fc1f9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x5555fc1eb530 tx=0x7fcdf40039c0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 shutdown_connections 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5555fbe49390 0x5555fc26d3b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555fbe286b0 0x5555fc2695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fcde9ffb640 0 client.20433 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555fbe204d0 0x5555fc1f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 --2- 172.21.15.94:0/3389965797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5555fbe07f60 0x5555fc1f9490 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 >> 172.21.15.94:0/3389965797 conn(0x5555fbe3d4e0 msgr2=0x5555fc1f3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 shutdown_connections 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 1 -- 172.21.15.94:0/3389965797 wait complete. 2024-04-25T13:55:28.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.165+0000 7fce0fa30740 20 client.20433 trim_cache size 0 max 16384 2024-04-25T13:55:28.175 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 msgr2=0x7fcdec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 0x7fcdec0b2000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5555fc1ec860 tx=0x7fcdfc02f4a0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 msgr2=0x7fce04199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800c970 tx=0x7fcdf800ce30 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 shutdown_connections 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcdec0afaf0 0x7fcdec0b2000 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04108920 0x7fce041a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce04103c20 0x7fce04199920 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 --2- 172.21.15.94:0/3481255182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce041029e0 0x7fce041993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 >> 172.21.15.94:0/3481255182 conn(0x7fce040fdbd0 msgr2=0x7fce04104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 shutdown_connections 2024-04-25T13:55:28.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.166+0000 7fce08aa7640 1 -- 172.21.15.94:0/3481255182 wait complete. 2024-04-25T13:55:28.177 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:28 smithi094 ceph-mon[29195]: pgmap v759: 97 pgs: 97 active+clean; 36 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-04-25T13:55:28.177 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:28 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2537428314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:55:28.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.291+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/515547334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28102f50 msgr2=0x7f5b28103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.291+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/515547334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28102f50 0x7f5b28103360 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009900 tx=0x7f5b2402a410 comp rx=0 tx=0).stop 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/515547334 shutdown_connections 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/515547334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 0x7f5b2810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/515547334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28103930 0x7f5b28103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/515547334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28102f50 0x7f5b28103360 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/515547334 >> 172.21.15.94:0/515547334 conn(0x7f5b280fd350 msgr2=0x7f5b280ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/515547334 shutdown_connections 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/515547334 wait complete. 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 Processor -- start 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 -- start start 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28102f50 0x7f5b2819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.292+0000 7f5b2fd4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 0x7f5b281a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2fd4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2810e4d0 con 0x7f5b28103930 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2fd4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2810e350 con 0x7f5b28104450 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2fd4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2810e650 con 0x7f5b28102f50 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36394/0 (socket says 172.21.15.94:36394) 2024-04-25T13:55:28.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/64315622 learned_addr learned my addr 172.21.15.94:0/64315622 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28102f50 0x7f5b2819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 0x7f5b281a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/64315622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28102f50 msgr2=0x7f5b2819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28102f50 0x7f5b2819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/64315622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 msgr2=0x7f5b281a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 0x7f5b281a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/64315622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b281a1d30 con 0x7f5b28103930 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d950 tx=0x7f5b1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b1effd640 1 -- 172.21.15.94:0/64315622 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18017070 con 0x7f5b28103930 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b1effd640 1 -- 172.21.15.94:0/64315622 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18009c30 con 0x7f5b28103930 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b281a1fc0 con 0x7f5b28103930 2024-04-25T13:55:28.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b1effd640 1 -- 172.21.15.94:0/64315622 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800f620 con 0x7f5b28103930 2024-04-25T13:55:28.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.293+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b281a2500 con 0x7f5b28103930 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.296+0000 7f5b36c04740 1 Processor -- start 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.296+0000 7f5b36c04740 1 -- start start 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.296+0000 7f5b1effd640 1 -- 172.21.15.94:0/64315622 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5b18010460 con 0x7f5b28103930 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b77d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564bc8adbd80 con 0x564bc87b46b0 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564bc8a95a20 con 0x564bc87d5390 2024-04-25T13:55:28.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b36c04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564bc8ad7d40 con 0x564bc87ac4d0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46004/0 (socket says 172.21.15.94:46004) 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/1710131313 learned_addr learned my addr 172.21.15.94:0/1710131313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b77d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/1710131313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 msgr2=0x564bc8b77d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b77d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/1710131313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 msgr2=0x564bc8b78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b78a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/1710131313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8aa0c30 con 0x564bc87d5390 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b1effd640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 0x7f5b140b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.297+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009fd0 tx=0x7f5b200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64861d682fd2676b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.298+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 0x7f5b140b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.298+0000 7f5b13fff640 1 -- 172.21.15.94:0/1710131313 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004d20 con 0x564bc87d5390 2024-04-25T13:55:28.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.298+0000 7f5b13fff640 1 -- 172.21.15.94:0/1710131313 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b20004050 con 0x564bc87d5390 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.298+0000 7f5b13fff640 1 -- 172.21.15.94:0/1710131313 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20005280 con 0x564bc87d5390 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.298+0000 7f5b1effd640 1 -- 172.21.15.94:0/64315622 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5b180de800 con 0x7f5b28103930 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- 172.21.15.94:0/1710131313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 msgr2=0x564bc8b77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 0x7f5b140b2080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009230 tx=0x7f5b24031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009fd0 tx=0x7f5b200118d0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- 172.21.15.94:0/1710131313 shutdown_connections 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b77d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 --2- 172.21.15.94:0/1710131313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b77810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- 172.21.15.94:0/1710131313 >> 172.21.15.94:0/1710131313 conn(0x564bc87c94e0 msgr2=0x564bc8b73020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- 172.21.15.94:0/1710131313 shutdown_connections 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- 172.21.15.94:0/1710131313 wait complete. 2024-04-25T13:55:28.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 -- start start 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.299+0000 7f5b36c04740 1 Processor -- start 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b81b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564bc8b79530 con 0x564bc87d5390 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564bc8a62dc0 con 0x564bc87b46b0 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564bc8b796b0 con 0x564bc87ac4d0 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b36c04740 10 client.0 _ll_register_callbacks cb 0x564bc8a7e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36414/0 (socket says 172.21.15.94:36414) 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b81b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/3148088539 learned_addr learned my addr 172.21.15.94:0/3148088539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/3148088539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 msgr2=0x564bc8b81630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b81630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 msgr2=0x564bc8b81b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b81b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/3148088539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc87ed3f0 con 0x564bc87d5390 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5b2001b040 tx=0x7f5b200044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5564bcef3c845b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b117fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.300+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20028020 con 0x564bc87d5390 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5b20004c50 con 0x564bc87d5390 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b20004050 con 0x564bc87d5390 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5b20033050 con 0x564bc87d5390 2024-04-25T13:55:28.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 10 client.20445 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564bc88b77d0 con 0x564bc87d5390 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b20004050 con 0x564bc87d5390 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f5b2000f4e0 con 0x564bc87d5390 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 10 client.20445 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 10 client.20445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 20 client.20445 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc8793f60 0x564bc8b8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 4 client.20445 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc8ae9990 con 0x564bc8793f60 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.301+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc8793f60 0x564bc8b8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.302+0000 7f5b117fa640 10 client.20445 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:28.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.302+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc8793f60 0x564bc8b8f760 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2bc48b483c913a server_cookie=2d326918dc5adde3 in_seq=0 out_seq=0 2024-04-25T13:55:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b117fa640 1 -- 172.21.15.94:0/3148088539 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bc8ae9990 con 0x564bc8793f60 2024-04-25T13:55:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b117fa640 10 client.20445 handle_command_reply: tid=0 2024-04-25T13:55:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 client.20445 shutdown 2024-04-25T13:55:28.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 msgr2=0x564bc8b88ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5b2001b040 tx=0x7f5b200044b0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 shutdown_connections 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc8793f60 0x564bc8b8f760 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b88ad0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3148088539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b81630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 >> 172.21.15.94:0/3148088539 conn(0x564bc87c94e0 msgr2=0x564bc8b75f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b117fa640 0 client.20445 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 shutdown_connections 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 1 -- 172.21.15.94:0/3148088539 wait complete. 2024-04-25T13:55:28.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.306+0000 7f5b36c04740 20 client.20445 trim_cache size 0 max 16384 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 Processor -- start 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 -- start start 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281ab5f0 con 0x564bc87d5390 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281ad1f0 con 0x564bc87b46b0 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.321+0000 7f5b36c04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281abcc0 con 0x564bc87ac4d0 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36422/0 (socket says 172.21.15.94:36422) 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/2740091412 learned_addr learned my addr 172.21.15.94:0/2740091412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/2740091412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 msgr2=0x564bc8b808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b808e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/2740091412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 msgr2=0x564bc8b889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 -- 172.21.15.94:0/2740091412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8b7f750 con 0x564bc87d5390 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005c40 tx=0x7f5b20005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83972d0265eabee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b137fe640 1 -- 172.21.15.94:0/2740091412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20028070 con 0x564bc87d5390 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b137fe640 1 -- 172.21.15.94:0/2740091412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b20004520 con 0x564bc87d5390 2024-04-25T13:55:28.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.322+0000 7f5b137fe640 1 -- 172.21.15.94:0/2740091412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200198e0 con 0x564bc87d5390 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 -- 172.21.15.94:0/2740091412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 msgr2=0x564bc8b80e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005c40 tx=0x7f5b20005f10 comp rx=0 tx=0).stop 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 -- 172.21.15.94:0/2740091412 shutdown_connections 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87d5390 0x564bc8b80e20 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87b46b0 0x564bc8b889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 --2- 172.21.15.94:0/2740091412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 -- 172.21.15.94:0/2740091412 >> 172.21.15.94:0/2740091412 conn(0x564bc87c94e0 msgr2=0x564bc8b7c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 -- 172.21.15.94:0/2740091412 shutdown_connections 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.323+0000 7f5b36c04740 1 -- 172.21.15.94:0/2740091412 wait complete. 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 -- start start 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 Processor -- start 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b87ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281acb20 con 0x564bc87b46b0 2024-04-25T13:55:28.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281ac390 con 0x564bc87d5390 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281a95d0 con 0x564bc87ac4d0 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b36c04740 10 client.0 _ll_register_callbacks cb 0x7f5b180e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2d2e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36430/0 (socket says 172.21.15.94:36430) 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/3595900389 learned_addr learned my addr 172.21.15.94:0/3595900389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2e2ea640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b87ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/3595900389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 msgr2=0x564bc8b848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.324+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b848b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 msgr2=0x564bc8b87ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b87ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b2d2e8640 1 -- 172.21.15.94:0/3595900389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc8b882a0 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b2d2e8640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5b180a2d70 tx=0x7f5b1800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6d1e77a224a4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18020070 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5b180a6160 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b20004520 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5b180bf020 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b36c04740 10 client.20451 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564bc8bf60d0 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b20004520 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.325+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f5b18010750 con 0x564bc87b46b0 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 10 client.20451 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 10 client.20451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:28.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 20 client.20451 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc87693a0 0x564bc8bf8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 4 client.20451 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bc8ae3800 con 0x564bc87693a0 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc87693a0 0x564bc8bf8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b117fa640 10 client.20451 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:28.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.326+0000 7f5b2dae9640 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc87693a0 0x564bc8bf8b80 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1290f68b0c10b9e0 server_cookie=5ac94b1f3fa23954 in_seq=0 out_seq=0 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.330+0000 7f5b117fa640 1 -- 172.21.15.94:0/3595900389 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bc8ae3800 con 0x564bc87693a0 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.330+0000 7f5b117fa640 10 client.20451 handle_command_reply: tid=0 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.330+0000 7f5b36c04740 1 client.20451 shutdown 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 msgr2=0x564bc8b84df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5b180a2d70 tx=0x7f5b1800bf60 comp rx=0 tx=0).stop 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 shutdown_connections 2024-04-25T13:55:28.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564bc87693a0 0x564bc8bf8b80 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564bc87d5390 0x564bc8b87ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564bc87b46b0 0x564bc8b84df0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 --2- 172.21.15.94:0/3595900389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564bc87ac4d0 0x564bc8b848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 >> 172.21.15.94:0/3595900389 conn(0x564bc87c94e0 msgr2=0x564bc8b75e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b117fa640 0 client.20451 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 shutdown_connections 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 1 -- 172.21.15.94:0/3595900389 wait complete. 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b36c04740 20 client.20451 trim_cache size 0 max 16384 2024-04-25T13:55:28.334 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 msgr2=0x7f5b140b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 0x7f5b140b2080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009230 tx=0x7f5b24031400 comp rx=0 tx=0).stop 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 msgr2=0x7f5b2819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d950 tx=0x7f5b1800de10 comp rx=0 tx=0).stop 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.331+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 shutdown_connections 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b140afb70 0x7f5b140b2080 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b28104450 0x7f5b281a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b28103930 0x7f5b2819a730 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 --2- 172.21.15.94:0/64315622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b28102f50 0x7f5b2819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 >> 172.21.15.94:0/64315622 conn(0x7f5b280fd350 msgr2=0x7f5b28109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 shutdown_connections 2024-04-25T13:55:28.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:28.332+0000 7f5b2fd4d640 1 -- 172.21.15.94:0/64315622 wait complete. 2024-04-25T13:55:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:28 smithi137 ceph-mon[32568]: pgmap v759: 97 pgs: 97 active+clean; 36 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-04-25T13:55:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:28 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2537428314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:55:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:28 smithi138 ceph-mon[32977]: pgmap v759: 97 pgs: 97 active+clean; 36 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 479 op/s 2024-04-25T13:55:28.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:28 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2537428314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:55:29.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 4313 36.75 MB/sec warmup 11 sec latency 189.547 ms 2024-04-25T13:55:29.343 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:55:29.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:55:29.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 -- 172.21.15.94:0/343424374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 msgr2=0x7fc9e0103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 --2- 172.21.15.94:0/343424374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e0103dd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009900 tx=0x7fc9d402a410 comp rx=0 tx=0).stop 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 -- 172.21.15.94:0/343424374 shutdown_connections 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 --2- 172.21.15.94:0/343424374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 0x7fc9e010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 --2- 172.21.15.94:0/343424374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e0103dd0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 --2- 172.21.15.94:0/343424374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 0x7fc9e0103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 -- 172.21.15.94:0/343424374 >> 172.21.15.94:0/343424374 conn(0x7fc9e00fd340 msgr2=0x7fc9e00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 -- 172.21.15.94:0/343424374 shutdown_connections 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 -- 172.21.15.94:0/343424374 wait complete. 2024-04-25T13:55:29.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.463+0000 7fc9e5420640 1 Processor -- start 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 -- start start 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 0x7fc9e019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 0x7fc9e01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010e570 con 0x7fc9e0103960 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010e3f0 con 0x7fc9e0104460 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9e5420640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010e6f0 con 0x7fc9e0102f80 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 0x7fc9e019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36442/0 (socket says 172.21.15.94:36442) 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 -- 172.21.15.94:0/2598121003 learned_addr learned my addr 172.21.15.94:0/2598121003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 0x7fc9e01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 -- 172.21.15.94:0/2598121003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 msgr2=0x7fc9e019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 0x7fc9e019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 -- 172.21.15.94:0/2598121003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 msgr2=0x7fc9e01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 0x7fc9e01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.464+0000 7fc9de7fc640 1 -- 172.21.15.94:0/2598121003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e01a1d60 con 0x7fc9e0103960 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4031440 tx=0x7fc9d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9c7fff640 1 -- 172.21.15.94:0/2598121003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d403f070 con 0x7fc9e0103960 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9c7fff640 1 -- 172.21.15.94:0/2598121003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d4031c50 con 0x7fc9e0103960 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e01a1ff0 con 0x7fc9e0103960 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9c7fff640 1 -- 172.21.15.94:0/2598121003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4035030 con 0x7fc9e0103960 2024-04-25T13:55:29.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.465+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a2400 con 0x7fc9e0103960 2024-04-25T13:55:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 Processor -- start 2024-04-25T13:55:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9c7fff640 1 -- 172.21.15.94:0/2598121003 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc9d4042070 con 0x7fc9e0103960 2024-04-25T13:55:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 -- start start 2024-04-25T13:55:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b179920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a16b0d7b00 con 0x55a16ada8550 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a16b073920 con 0x55a16add1410 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a16b0d3ac0 con 0x55a16adb0730 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36450/0 (socket says 172.21.15.94:36450) 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 -- 172.21.15.94:0/3299751865 learned_addr learned my addr 172.21.15.94:0/3299751865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46056/0 (socket says 172.21.15.94:46056) 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b179920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 -- 172.21.15.94:0/3299751865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 msgr2=0x55a16b179920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b179920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 -- 172.21.15.94:0/3299751865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 msgr2=0x55a16b1724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9deffd640 1 -- 172.21.15.94:0/3299751865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16aeb17e0 con 0x55a16ada8550 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.468+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009c60 tx=0x7fc9d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcb5fad42c1e01b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9c7fff640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 0x7fc9c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9c57fa640 1 -- 172.21.15.94:0/3299751865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0004220 con 0x55a16ada8550 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 0x7fc9c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9c57fa640 1 -- 172.21.15.94:0/3299751865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d00043c0 con 0x55a16ada8550 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9c57fa640 1 -- 172.21.15.94:0/3299751865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0019780 con 0x55a16ada8550 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.469+0000 7fc9c7fff640 1 -- 172.21.15.94:0/2598121003 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc9d40ff3c0 con 0x7fc9e0103960 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3299751865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 msgr2=0x55a16b172a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009c60 tx=0x7fc9d0011810 comp rx=0 tx=0).stop 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 0x7fc9c80b2000 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55a16b173670 tx=0x7fc9d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3299751865 shutdown_connections 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b179920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b172a20 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3299751865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16add1410 0x55a16b1724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3299751865 >> 172.21.15.94:0/3299751865 conn(0x55a16adc5560 msgr2=0x55a16b16de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3299751865 shutdown_connections 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3299751865 wait complete. 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 -- start start 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.470+0000 7fc9ec3ad740 1 Processor -- start 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b17c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16add1410 0x55a16b1836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a16b17a370 con 0x55a16ada8550 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a16b0715d0 con 0x55a16adb0730 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a16b17a4f0 con 0x55a16add1410 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9ec3ad740 10 client.0 _ll_register_callbacks cb 0x55a16b05f080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36454/0 (socket says 172.21.15.94:36454) 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 -- 172.21.15.94:0/2302412159 learned_addr learned my addr 172.21.15.94:0/2302412159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b17c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16add1410 0x55a16b1836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 -- 172.21.15.94:0/2302412159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16add1410 msgr2=0x55a16b1836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16add1410 0x55a16b1836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 -- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 msgr2=0x55a16b17c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b17c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 -- 172.21.15.94:0/2302412159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16af11f70 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9deffd640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009c30 tx=0x7fc9d00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38287d94f42a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.471+0000 7fc9c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0028070 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc9d00035f0 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d00043c0 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9d0034050 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9ec3ad740 10 client.20463 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16ada9170 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d00043c0 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc9d0004a80 con 0x55a16ada8550 2024-04-25T13:55:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.472+0000 7fc9ec3ad740 10 client.20463 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9ec3ad740 10 client.20463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9ec3ad740 20 client.20463 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad901f0 0x55a16b189a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9ec3ad740 4 client.20463 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16b0e5710 con 0x55a16ad901f0 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad901f0 0x55a16b189a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9c2ff5640 10 client.20463 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.473+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad901f0 0x55a16b189a60 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31cbb009a032ba server_cookie=3ca6f01f7522491e in_seq=0 out_seq=0 2024-04-25T13:55:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.477+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/2302412159 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a16b0e5710 con 0x55a16ad901f0 2024-04-25T13:55:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.477+0000 7fc9c2ff5640 10 client.20463 handle_command_reply: tid=0 2024-04-25T13:55:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.477+0000 7fc9ec3ad740 1 client.20463 shutdown 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 msgr2=0x55a16b17c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009c30 tx=0x7fc9d00052f0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 shutdown_connections 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad901f0 0x55a16b189a60 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16add1410 0x55a16b1836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b17c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/2302412159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16ada8550 0x55a16b17c240 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9c2ff5640 0 client.20463 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 >> 172.21.15.94:0/2302412159 conn(0x55a16adc5560 msgr2=0x55a16b16cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 shutdown_connections 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/2302412159 wait complete. 2024-04-25T13:55:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.478+0000 7fc9ec3ad740 20 client.20463 trim_cache size 0 max 16384 2024-04-25T13:55:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.508+0000 7fc9ec3ad740 1 Processor -- start 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.508+0000 7fc9ec3ad740 1 -- start start 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16ada8550 0x55a16b171660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b18f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ab480 con 0x55a16add1410 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ad080 con 0x55a16adb0730 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01abb50 con 0x55a16ada8550 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36462/0 (socket says 172.21.15.94:36462) 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16ada8550 0x55a16b171660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 -- 172.21.15.94:0/1488904411 learned_addr learned my addr 172.21.15.94:0/1488904411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b18f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 -- 172.21.15.94:0/1488904411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16ada8550 msgr2=0x55a16b171660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16ada8550 0x55a16b171660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 -- 172.21.15.94:0/1488904411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 msgr2=0x55a16b18f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b18f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.509+0000 7fc9df7fe640 1 -- 172.21.15.94:0/1488904411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16b17c8e0 con 0x55a16add1410 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.510+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8003c90 tx=0x7fc9d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d357af8ef632559c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.510+0000 7fc9c4ff9640 1 -- 172.21.15.94:0/1488904411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800fe60 con 0x55a16add1410 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.510+0000 7fc9c4ff9640 1 -- 172.21.15.94:0/1488904411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d8003cc0 con 0x55a16add1410 2024-04-25T13:55:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.510+0000 7fc9c4ff9640 1 -- 172.21.15.94:0/1488904411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8029e10 con 0x55a16add1410 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/1488904411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 msgr2=0x55a16b18f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8003c90 tx=0x7fc9d801a000 comp rx=0 tx=0).stop 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/1488904411 shutdown_connections 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b18f7a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16adb0730 0x55a16b18f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/1488904411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16ada8550 0x55a16b171660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.511+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/1488904411 >> 172.21.15.94:0/1488904411 conn(0x55a16adc5560 msgr2=0x55a16b17a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/1488904411 shutdown_connections 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/1488904411 wait complete. 2024-04-25T13:55:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- start start 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 Processor -- start 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b181460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b1efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ac9b0 con 0x55a16add1410 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ac220 con 0x55a16ada8550 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01a9460 con 0x55a16adb0730 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.512+0000 7fc9ec3ad740 10 client.0 _ll_register_callbacks cb 0x7fc9d0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46068/0 (socket says 172.21.15.94:46068) 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 -- 172.21.15.94:0/3463433001 learned_addr learned my addr 172.21.15.94:0/3463433001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b181460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9df7fe640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b1efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 -- 172.21.15.94:0/3463433001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 msgr2=0x55a16b181460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b181460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 -- 172.21.15.94:0/3463433001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 msgr2=0x55a16b1efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b1efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 -- 172.21.15.94:0/3463433001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16b1f0340 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9deffd640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0004ed0 tx=0x7fc9d00096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3af755b59c8d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.513+0000 7fc9c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0028070 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc9d0019c10 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d8003cc0 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9d0019780 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9ec3ad740 10 client.29909 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16b1f1560 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d0019e10 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.514+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc9d0004380 con 0x55a16ada8550 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 10 client.29909 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 10 client.29909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 20 client.29909 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad65630 0x55a16b1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 4 client.29909 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc9d8003cc0 con 0x55a16ad65630 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad65630 0x55a16b1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9c2ff5640 10 client.29909 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.515+0000 7fc9de7fc640 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad65630 0x55a16b1f4060 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5190a20696ef9f8 server_cookie=1af5ea6da20c6b1b in_seq=0 out_seq=0 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9c2ff5640 1 -- 172.21.15.94:0/3463433001 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc9d8003cc0 con 0x55a16ad65630 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9c2ff5640 10 client.29909 handle_command_reply: tid=0 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 client.29909 shutdown 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 msgr2=0x55a16b180f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0004ed0 tx=0x7fc9d00096d0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 shutdown_connections 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a16ad65630 0x55a16b1f4060 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a16add1410 0x55a16b1efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a16adb0730 0x55a16b181460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9c2ff5640 0 client.29909 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 --2- 172.21.15.94:0/3463433001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16ada8550 0x55a16b180f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 >> 172.21.15.94:0/3463433001 conn(0x55a16adc5560 msgr2=0x55a16b1713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 shutdown_connections 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 1 -- 172.21.15.94:0/3463433001 wait complete. 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.539+0000 7fc9ec3ad740 20 client.29909 trim_cache size 0 max 16384 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "7ae72420-f76a-4a66-a70a-6d1c6b8209b1", 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:55:29.541 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:55:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 msgr2=0x7fc9c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 0x7fc9c80b2000 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55a16b173670 tx=0x7fc9d8009500 comp rx=0 tx=0).stop 2024-04-25T13:55:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 msgr2=0x7fc9e019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4031440 tx=0x7fc9d4003c10 comp rx=0 tx=0).stop 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 shutdown_connections 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9c80afaf0 0x7fc9c80b2000 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9e0104460 0x7fc9e01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e0103960 0x7fc9e019a7b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 --2- 172.21.15.94:0/2598121003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e0102f80 0x7fc9e019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 >> 172.21.15.94:0/2598121003 conn(0x7fc9e00fd340 msgr2=0x7fc9e0109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 shutdown_connections 2024-04-25T13:55:29.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.540+0000 7fc9e5420640 1 -- 172.21.15.94:0/2598121003 wait complete. 2024-04-25T13:55:29.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:55:29.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:55:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 -- 172.21.15.94:0/3614145352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 msgr2=0x7f3df01052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 --2- 172.21.15.94:0/3614145352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01052c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002a410 comp rx=0 tx=0).stop 2024-04-25T13:55:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 -- 172.21.15.94:0/3614145352 shutdown_connections 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 --2- 172.21.15.94:0/3614145352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 0x7f3df010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 --2- 172.21.15.94:0/3614145352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 0x7f3df01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 --2- 172.21.15.94:0/3614145352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01052c0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 -- 172.21.15.94:0/3614145352 >> 172.21.15.94:0/3614145352 conn(0x7f3df00fd1c0 msgr2=0x7f3df00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 -- 172.21.15.94:0/3614145352 shutdown_connections 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.666+0000 7f3df559d640 1 -- 172.21.15.94:0/3614145352 wait complete. 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 Processor -- start 2024-04-25T13:55:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 -- start start 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 0x7f3df019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 0x7f3df019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010f420 con 0x7f3df0104eb0 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010f2a0 con 0x7f3df0102060 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df559d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010f5a0 con 0x7f3df0101560 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36498/0 (socket says 172.21.15.94:36498) 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 -- 172.21.15.94:0/290383798 learned_addr learned my addr 172.21.15.94:0/290383798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3def7fe640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 0x7f3df019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3deffff640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 0x7f3df019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 -- 172.21.15.94:0/290383798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 msgr2=0x7f3df019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 0x7f3df019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 -- 172.21.15.94:0/290383798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 msgr2=0x7f3df019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 0x7f3df019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.667+0000 7f3df4d9c640 1 -- 172.21.15.94:0/290383798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a1bd0 con 0x7f3df0104eb0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3df4d9c640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3de800eb10 tx=0x7f3de800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3ded7fa640 1 -- 172.21.15.94:0/290383798 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8004b20 con 0x7f3df0104eb0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3ded7fa640 1 -- 172.21.15.94:0/290383798 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de8004cc0 con 0x7f3df0104eb0 2024-04-25T13:55:29.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a1ec0 con 0x7f3df0104eb0 2024-04-25T13:55:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3ded7fa640 1 -- 172.21.15.94:0/290383798 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8010670 con 0x7f3df0104eb0 2024-04-25T13:55:29.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.668+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a22d0 con 0x7f3df0104eb0 2024-04-25T13:55:29.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.670+0000 7f3ded7fa640 1 -- 172.21.15.94:0/290383798 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3de8010830 con 0x7f3df0104eb0 2024-04-25T13:55:29.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 Processor -- start 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 -- start start 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5065c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5066e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5558b4fcad80 con 0x5558b4ca36b0 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5558b4f848b0 con 0x5558b4cc4390 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3dfd785740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558b4fc6d40 con 0x5558b4c9b4d0 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5066e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3deffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3deffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56012/0 (socket says 172.21.15.94:56012) 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3deffff640 1 -- 172.21.15.94:0/1442496313 learned_addr learned my addr 172.21.15.94:0/1442496313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5065c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3ded7fa640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 0x7f3dd80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3df4d9c640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 0x7f3dd80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3deffff640 1 -- 172.21.15.94:0/1442496313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 msgr2=0x5558b5065c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.671+0000 7f3deffff640 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5065c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3deffff640 1 -- 172.21.15.94:0/1442496313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 msgr2=0x5558b5066e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3deffff640 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5066e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3deffff640 1 -- 172.21.15.94:0/1442496313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b4f55a40 con 0x5558b4c9b4d0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5066e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3ded7fa640 1 -- 172.21.15.94:0/290383798 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3de80df250 con 0x7f3df0104eb0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3deffff640 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009230 tx=0x7f3de0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6c68dbb102562f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3dd67fc640 1 -- 172.21.15.94:0/1442496313 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0035030 con 0x5558b4c9b4d0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3dd67fc640 1 -- 172.21.15.94:0/1442496313 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0044070 con 0x5558b4c9b4d0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.672+0000 7f3dd67fc640 1 -- 172.21.15.94:0/1442496313 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0040b40 con 0x5558b4c9b4d0 2024-04-25T13:55:29.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 -- 172.21.15.94:0/1442496313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 msgr2=0x5558b50661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009230 tx=0x7f3de0009620 comp rx=0 tx=0).stop 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3df4d9c640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 0x7f3dd80b1ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3de8014040 tx=0x7f3de806efe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 -- 172.21.15.94:0/1442496313 shutdown_connections 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5066e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b50661d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1442496313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5065c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 -- 172.21.15.94:0/1442496313 >> 172.21.15.94:0/1442496313 conn(0x5558b4cb84e0 msgr2=0x5558b5061460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 -- 172.21.15.94:0/1442496313 shutdown_connections 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.673+0000 7f3dfd785740 1 -- 172.21.15.94:0/1442496313 wait complete. 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 -- start start 2024-04-25T13:55:29.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 Processor -- start 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 0x5558b506fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4ca36b0 0x5558b506fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5076f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5558b5067910 con 0x5558b4c9b4d0 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5558b4f51d10 con 0x5558b4cc4390 2024-04-25T13:55:29.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558b5067a90 con 0x5558b4ca36b0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3dfd785740 10 client.0 _ll_register_callbacks cb 0x5558b4f7e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 0x5558b506fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 0x5558b506fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36510/0 (socket says 172.21.15.94:36510) 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.674+0000 7f3def7fe640 1 -- 172.21.15.94:0/81290750 learned_addr learned my addr 172.21.15.94:0/81290750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3deffff640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4ca36b0 0x5558b506fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 -- 172.21.15.94:0/81290750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4ca36b0 msgr2=0x5558b506fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4ca36b0 0x5558b506fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 -- 172.21.15.94:0/81290750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 msgr2=0x5558b506fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 0x5558b506fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 -- 172.21.15.94:0/81290750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b4da67d0 con 0x5558b4cc4390 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3def7fe640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5076f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3de4004620 tx=0x7f3de40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46db68ecc21ea924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3dcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:55:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de401a040 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3de4013110 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de0044070 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.675+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3de4013310 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 10 client.29921 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558b4cdc3f0 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0044070 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3de4011960 con 0x5558b4cc4390 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 10 client.29921 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 10 client.29921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 20 client.29921 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c82f60 0x5558b507d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 4 client.29921 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558b4fd8990 con 0x5558b4c82f60 2024-04-25T13:55:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.676+0000 7f3deffff640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c82f60 0x5558b507d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.677+0000 7f3dcf7fe640 10 client.29921 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.677+0000 7f3deffff640 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c82f60 0x5558b507d490 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536a4b7b815b605c server_cookie=2f9a6f21c985ca8 in_seq=0 out_seq=0 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.681+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/81290750 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558b4fd8990 con 0x5558b4c82f60 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dcf7fe640 10 client.29921 handle_command_reply: tid=0 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 client.29921 shutdown 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 msgr2=0x5558b5076f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5076f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3de4004620 tx=0x7f3de40046c0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 shutdown_connections 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c82f60 0x5558b507d490 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b5076f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4ca36b0 0x5558b506fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 --2- 172.21.15.94:0/81290750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4c9b4d0 0x5558b506fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dcf7fe640 0 client.29921 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 >> 172.21.15.94:0/81290750 conn(0x5558b4cb84e0 msgr2=0x5558b5064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 shutdown_connections 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 1 -- 172.21.15.94:0/81290750 wait complete. 2024-04-25T13:55:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.682+0000 7f3dfd785740 20 client.29921 trim_cache size 0 max 16384 2024-04-25T13:55:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 Processor -- start 2024-04-25T13:55:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 -- start start 2024-04-25T13:55:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5081c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01a91a0 con 0x5558b4ca36b0 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01aada0 con 0x5558b4cc4390 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3dfd785740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01a9870 con 0x5558b4c9b4d0 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3deffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3deffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36524/0 (socket says 172.21.15.94:36524) 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5081c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3deffff640 1 -- 172.21.15.94:0/1376165746 learned_addr learned my addr 172.21.15.94:0/1376165746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.702+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 -- 172.21.15.94:0/1376165746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 msgr2=0x5558b5081c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5081c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 -- 172.21.15.94:0/1376165746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 msgr2=0x5558b50821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 -- 172.21.15.94:0/1376165746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b4c9c0f0 con 0x5558b4ca36b0 2024-04-25T13:55:29.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3deffff640 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005f70 tx=0x7f3de002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4088911c09bc8ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3dd5ffb640 1 -- 172.21.15.94:0/1376165746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de005e070 con 0x5558b4ca36b0 2024-04-25T13:55:29.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3dd5ffb640 1 -- 172.21.15.94:0/1376165746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0058070 con 0x5558b4ca36b0 2024-04-25T13:55:29.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.703+0000 7f3dd5ffb640 1 -- 172.21.15.94:0/1376165746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0035040 con 0x5558b4ca36b0 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- 172.21.15.94:0/1376165746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 msgr2=0x5558b5076ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005f70 tx=0x7f3de002a410 comp rx=0 tx=0).stop 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- 172.21.15.94:0/1376165746 shutdown_connections 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5076ee0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1376165746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5081c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- 172.21.15.94:0/1376165746 >> 172.21.15.94:0/1376165746 conn(0x5558b4cb84e0 msgr2=0x5558b506e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- 172.21.15.94:0/1376165746 shutdown_connections 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- 172.21.15.94:0/1376165746 wait complete. 2024-04-25T13:55:29.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.704+0000 7f3dfd785740 1 -- start start 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 Processor -- start 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5075940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01aa6d0 con 0x5558b4ca36b0 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01a9f40 con 0x5558b4cc4390 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01a7180 con 0x5558b4c9b4d0 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3dfd785740 10 client.0 _ll_register_callbacks cb 0x7f3de40042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36536/0 (socket says 172.21.15.94:36536) 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 -- 172.21.15.94:0/1263664673 learned_addr learned my addr 172.21.15.94:0/1263664673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 -- 172.21.15.94:0/1263664673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 msgr2=0x5558b5075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 -- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 msgr2=0x5558b50763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.705+0000 7f3deffff640 1 -- 172.21.15.94:0/1263664673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b4f8b4b0 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3deffff640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331a3fe134479367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de005e070 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3de0004240 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de0058070 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3de0035040 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 10 client.20487 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558b50e4a80 con 0x5558b4ca36b0 2024-04-25T13:55:29.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0058070 con 0x5558b4ca36b0 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3de0052af0 con 0x5558b4ca36b0 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 10 client.20487 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 10 client.20487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 20 client.20487 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c583a0 0x5558b50e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 4 client.20487 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.706+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558b4fd2800 con 0x5558b4c583a0 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.707+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c583a0 0x5558b50e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.707+0000 7f3dcf7fe640 10 client.20487 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.707+0000 7f3def7fe640 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c583a0 0x5558b50e70a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396723f5df38054a server_cookie=4b6273b32bab5eaa in_seq=0 out_seq=0 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.711+0000 7f3dcf7fe640 1 -- 172.21.15.94:0/1263664673 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5558b4fd2800 con 0x5558b4c583a0 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.711+0000 7f3dcf7fe640 10 client.20487 handle_command_reply: tid=0 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.711+0000 7f3dfd785740 1 client.20487 shutdown 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 msgr2=0x5558b5075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002ab90 comp rx=0 tx=0).stop 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 shutdown_connections 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5558b4c583a0 0x5558b50e70a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5558b4cc4390 0x5558b50763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5558b4ca36b0 0x5558b5075e80 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dcf7fe640 0 client.20487 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 --2- 172.21.15.94:0/1263664673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558b4c9b4d0 0x5558b5075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 >> 172.21.15.94:0/1263664673 conn(0x5558b4cb84e0 msgr2=0x5558b5064260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 shutdown_connections 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 1 -- 172.21.15.94:0/1263664673 wait complete. 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3dfd785740 20 client.20487 trim_cache size 0 max 16384 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (166 inodes in the stack)", 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "7ae72420-f76a-4a66-a70a-6d1c6b8209b1": { 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "7ae72420-f76a-4a66-a70a-6d1c6b8209b1", 2024-04-25T13:55:29.715 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:55:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:55:29.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:55:29.716 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:55:29.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 msgr2=0x7f3dd80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 0x7f3dd80b1ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3de8014040 tx=0x7f3de806efe0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 msgr2=0x7f3df01a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3de800eb10 tx=0x7f3de800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 shutdown_connections 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3dd80afae0 0x7f3dd80b1ff0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3df0104eb0 0x7f3df01a1560 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3df0102060 0x7f3df019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 --2- 172.21.15.94:0/290383798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df0101560 0x7f3df019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 >> 172.21.15.94:0/290383798 conn(0x7f3df00fd1c0 msgr2=0x7f3df0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 shutdown_connections 2024-04-25T13:55:29.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:29.712+0000 7f3df559d640 1 -- 172.21.15.94:0/290383798 wait complete. 2024-04-25T13:55:29.723 INFO:tasks.cephfs.filesystem:scrub status for tag:7ae72420-f76a-4a66-a70a-6d1c6b8209b1 - {'path': '/', 'tag': '7ae72420-f76a-4a66-a70a-6d1c6b8209b1', 'options': 'recursive,force'} 2024-04-25T13:55:30.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 4617 34.77 MB/sec warmup 12 sec latency 143.420 ms 2024-04-25T13:55:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: pgmap v760: 97 pgs: 97 active+clean; 37 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: pgmap v760: 97 pgs: 97 active+clean; 37 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:30.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: pgmap v760: 97 pgs: 97 active+clean; 37 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-04-25T13:55:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 5155 34.39 MB/sec warmup 13 sec latency 1132.594 ms 2024-04-25T13:55:31.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:55:31.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:31 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:31 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:31.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:31 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:55:32.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 5912 33.80 MB/sec warmup 14 sec latency 2132.633 ms 2024-04-25T13:55:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:32 smithi137 ceph-mon[32568]: pgmap v761: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-04-25T13:55:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:32 smithi094 ceph-mon[29195]: pgmap v761: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-04-25T13:55:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:32 smithi138 ceph-mon[32977]: pgmap v761: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-04-25T13:55:33.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 6597 35.53 MB/sec warmup 15 sec latency 2407.088 ms 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 -- 172.21.15.94:0/2469520048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 msgr2=0x7feb14102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 --2- 172.21.15.94:0/2469520048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb14102e30 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802a410 comp rx=0 tx=0).stop 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 -- 172.21.15.94:0/2469520048 shutdown_connections 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 --2- 172.21.15.94:0/2469520048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14108ae0 0x7feb1410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 --2- 172.21.15.94:0/2469520048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb14102e30 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 --2- 172.21.15.94:0/2469520048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14103fa0 0x7feb14102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 -- 172.21.15.94:0/2469520048 >> 172.21.15.94:0/2469520048 conn(0x7feb140fd390 msgr2=0x7feb140ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 -- 172.21.15.94:0/2469520048 shutdown_connections 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.276+0000 7feb1a11f640 1 -- 172.21.15.94:0/2469520048 wait complete. 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 Processor -- start 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 -- start start 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 0x7feb1419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14108ae0 0x7feb141a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410f650 con 0x7feb14104440 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410f4d0 con 0x7feb14108ae0 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb1a11f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410f7d0 con 0x7feb14103fa0 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36556/0 (socket says 172.21.15.94:36556) 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 -- 172.21.15.94:0/53522333 learned_addr learned my addr 172.21.15.94:0/53522333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb137fe640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 0x7feb1419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb13fff640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14108ae0 0x7feb141a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 -- 172.21.15.94:0/53522333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 msgr2=0x7feb1419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 0x7feb1419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 -- 172.21.15.94:0/53522333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14108ae0 msgr2=0x7feb141a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14108ae0 0x7feb141a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.277+0000 7feb12ffd640 1 -- 172.21.15.94:0/53522333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb141a1d20 con 0x7feb14104440 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb137fe640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 0x7feb1419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb12ffd640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb10ff9640 1 -- 172.21.15.94:0/53522333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0803f030 con 0x7feb14104440 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb10ff9640 1 -- 172.21.15.94:0/53522333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb08031910 con 0x7feb14104440 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb10ff9640 1 -- 172.21.15.94:0/53522333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08035070 con 0x7feb14104440 2024-04-25T13:55:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb141a1fb0 con 0x7feb14104440 2024-04-25T13:55:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.278+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a23b0 con 0x7feb14104440 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.280+0000 7feb10ff9640 1 -- 172.21.15.94:0/53522333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7feb08042030 con 0x7feb14104440 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.280+0000 7feb20fcf740 1 Processor -- start 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.280+0000 7feb20fcf740 1 -- start start 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 0x55a6c12b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6c121f460 con 0x55a6c0ef84d0 2024-04-25T13:55:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6c11d9590 con 0x55a6c0f191b0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb20fcf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6c121b420 con 0x55a6c0ef02f0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb13fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb13fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46168/0 (socket says 172.21.15.94:46168) 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 0x55a6c12b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb13fff640 1 -- 172.21.15.94:0/275629738 learned_addr learned my addr 172.21.15.94:0/275629738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb10ff9640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 0x7feafc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 -- 172.21.15.94:0/275629738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 msgr2=0x55a6c12b9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 -- 172.21.15.94:0/275629738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 msgr2=0x55a6c12c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb137fe640 1 -- 172.21.15.94:0/275629738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c11e60d0 con 0x55a6c0ef02f0 2024-04-25T13:55:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb13fff640 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb12ffd640 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.281+0000 7feb13fff640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 0x7feafc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.282+0000 7feb10ff9640 1 -- 172.21.15.94:0/53522333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7feb080ff1b0 con 0x7feb14104440 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.282+0000 7feb137fe640 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 0x55a6c12b9ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb04009fa0 tx=0x7feb040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9debbfe94984e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.282+0000 7feaf9ffb640 1 -- 172.21.15.94:0/275629738 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04004030 con 0x55a6c0ef02f0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.282+0000 7feaf9ffb640 1 -- 172.21.15.94:0/275629738 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb040054d0 con 0x55a6c0ef02f0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.282+0000 7feaf9ffb640 1 -- 172.21.15.94:0/275629738 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04004c80 con 0x55a6c0ef02f0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- 172.21.15.94:0/275629738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 msgr2=0x55a6c12b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 0x55a6c12b9ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb04009fa0 tx=0x7feb040118d0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb13fff640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 0x7feafc0b1ff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55a6c12bab40 tx=0x7feb0c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- 172.21.15.94:0/275629738 shutdown_connections 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef02f0 0x55a6c12b9ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 --2- 172.21.15.94:0/275629738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- 172.21.15.94:0/275629738 >> 172.21.15.94:0/275629738 conn(0x55a6c0f0d300 msgr2=0x55a6c12b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- 172.21.15.94:0/275629738 shutdown_connections 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- 172.21.15.94:0/275629738 wait complete. 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.283+0000 7feb20fcf740 1 -- start start 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 Processor -- start 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 0x55a6c12cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a6c12c1840 con 0x55a6c0ef02f0 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6c11a6b60 con 0x55a6c0ef84d0 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6c12c19c0 con 0x55a6c0f191b0 2024-04-25T13:55:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb20fcf740 10 client.0 _ll_register_callbacks cb 0x55a6c11cb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36582/0 (socket says 172.21.15.94:36582) 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 -- 172.21.15.94:0/1023945866 learned_addr learned my addr 172.21.15.94:0/1023945866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb12ffd640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb13fff640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 0x55a6c12cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 -- 172.21.15.94:0/1023945866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 msgr2=0x55a6c12cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 0x55a6c12cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 -- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef84d0 msgr2=0x55a6c12c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 -- 172.21.15.94:0/1023945866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c0f31210 con 0x55a6c0ef02f0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb13fff640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 0x55a6c12cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feb137fe640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feb04004630 tx=0x7feb0400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9828bd07d48423ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.284+0000 7feaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04028020 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7feb0400fdf0 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb040054d0 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb04033050 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 10 client.20499 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6c0ffd620 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb040054d0 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7feb04023470 con 0x55a6c0ef02f0 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 10 client.20499 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 10 client.20499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 20 client.20499 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ed7d80 0x55a6c12d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 4 client.20499 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6c122d070 con 0x55a6c0ed7d80 2024-04-25T13:55:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.285+0000 7feb12ffd640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ed7d80 0x55a6c12d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.286+0000 7feaf2ffd640 10 client.20499 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.286+0000 7feb12ffd640 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ed7d80 0x55a6c12d11d0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f8ae56431891e8f server_cookie=5b8ab3bff01499ff in_seq=0 out_seq=0 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.288+0000 7feaf2ffd640 1 -- 172.21.15.94:0/1023945866 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6c122d070 con 0x55a6c0ed7d80 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.288+0000 7feaf2ffd640 10 client.20499 handle_command_reply: tid=0 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.288+0000 7feb20fcf740 1 client.20499 shutdown 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.288+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 msgr2=0x55a6c12c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.288+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7feb04004630 tx=0x7feb0400fc00 comp rx=0 tx=0).stop 2024-04-25T13:55:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 shutdown_connections 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ed7d80 0x55a6c12d11d0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0f191b0 0x55a6c12cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feaf2ffd640 0 client.20499 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 --2- 172.21.15.94:0/1023945866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c3670 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 >> 172.21.15.94:0/1023945866 conn(0x55a6c0f0d300 msgr2=0x55a6c12b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 shutdown_connections 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 1 -- 172.21.15.94:0/1023945866 wait complete. 2024-04-25T13:55:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.289+0000 7feb20fcf740 20 client.20499 trim_cache size 0 max 16384 2024-04-25T13:55:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 Processor -- start 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 -- start start 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 0x55a6c12be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ad700 con 0x55a6c0f191b0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feb141af300 con 0x55a6c0ef02f0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb20fcf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb141addd0 con 0x55a6c0ef84d0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36594/0 (socket says 172.21.15.94:36594) 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 -- 172.21.15.94:0/3698796532 learned_addr learned my addr 172.21.15.94:0/3698796532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb137fe640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 0x55a6c12be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb12ffd640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 -- 172.21.15.94:0/3698796532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 msgr2=0x55a6c12b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 -- 172.21.15.94:0/3698796532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 msgr2=0x55a6c12be160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 0x55a6c12be160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb13fff640 1 -- 172.21.15.94:0/3698796532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c12b8d50 con 0x55a6c0f191b0 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.305+0000 7feb137fe640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 0x55a6c12be160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.306+0000 7feb13fff640 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7feb0c011630 tx=0x7feb0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a2f7b725e1ab86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.306+0000 7feaf97fa640 1 -- 172.21.15.94:0/3698796532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c004d20 con 0x55a6c0f191b0 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.306+0000 7feaf97fa640 1 -- 172.21.15.94:0/3698796532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c01cb10 con 0x55a6c0f191b0 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.306+0000 7feaf97fa640 1 -- 172.21.15.94:0/3698796532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c005880 con 0x55a6c0f191b0 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- 172.21.15.94:0/3698796532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 msgr2=0x55a6c12c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7feb0c011630 tx=0x7feb0c019000 comp rx=0 tx=0).stop 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- 172.21.15.94:0/3698796532 shutdown_connections 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0f191b0 0x55a6c12c3240 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 --2- 172.21.15.94:0/3698796532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0ef02f0 0x55a6c12be160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- 172.21.15.94:0/3698796532 >> 172.21.15.94:0/3698796532 conn(0x55a6c0f0d300 msgr2=0x55a6c12b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- 172.21.15.94:0/3698796532 shutdown_connections 2024-04-25T13:55:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- 172.21.15.94:0/3698796532 wait complete. 2024-04-25T13:55:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 -- start start 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.307+0000 7feb20fcf740 1 Processor -- start 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb141aec30 con 0x55a6c0ef02f0 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ae4a0 con 0x55a6c0f191b0 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ab6e0 con 0x55a6c0ef84d0 2024-04-25T13:55:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 10 client.0 _ll_register_callbacks cb 0x7feb08102b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36602/0 (socket says 172.21.15.94:36602) 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 -- 172.21.15.94:0/13946350 learned_addr learned my addr 172.21.15.94:0/13946350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb12ffd640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb13fff640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 -- 172.21.15.94:0/13946350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 msgr2=0x55a6c12c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 -- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 msgr2=0x55a6c12c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 -- 172.21.15.94:0/13946350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c12c8d70 con 0x55a6c0ef02f0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb12ffd640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb137fe640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7feb04009250 tx=0x7feb04009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a58b0c44019adc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04028070 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7feb04005040 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.308+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c01cb10 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb0401d040 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 10 client.20505 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6c12ca000 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c01cb10 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7feb04036030 con 0x55a6c0ef02f0 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 10 client.20505 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 10 client.20505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 20 client.20505 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ead1c0 0x55a6c133b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 4 client.20505 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:55:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6c1226ee0 con 0x55a6c0ead1c0 2024-04-25T13:55:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.309+0000 7feb12ffd640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ead1c0 0x55a6c133b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.310+0000 7feaf2ffd640 10 client.20505 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.310+0000 7feb12ffd640 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ead1c0 0x55a6c133b980 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c3d3dfcc08efce server_cookie=e321a2e3de16b82 in_seq=0 out_seq=0 2024-04-25T13:55:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.826+0000 7feaf2ffd640 1 -- 172.21.15.94:0/13946350 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6c1226ee0 con 0x55a6c0ead1c0 2024-04-25T13:55:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.826+0000 7feaf2ffd640 10 client.20505 handle_command_reply: tid=0 2024-04-25T13:55:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.826+0000 7feb20fcf740 1 client.20505 shutdown 2024-04-25T13:55:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.826+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 msgr2=0x55a6c12c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.826+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7feb04009250 tx=0x7feb04009e20 comp rx=0 tx=0).stop 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 shutdown_connections 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a6c0ead1c0 0x55a6c133b980 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6c0f191b0 0x55a6c12c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6c0ef84d0 0x55a6c12c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 --2- 172.21.15.94:0/13946350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a6c0ef02f0 0x55a6c12c73b0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 >> 172.21.15.94:0/13946350 conn(0x55a6c0f0d300 msgr2=0x55a6c12d63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feaf2ffd640 0 client.20505 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 shutdown_connections 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 1 -- 172.21.15.94:0/13946350 wait complete. 2024-04-25T13:55:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.827+0000 7feb20fcf740 20 client.20505 trim_cache size 0 max 16384 2024-04-25T13:55:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:55:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:55:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:55:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 msgr2=0x7feafc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 0x7feafc0b1ff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55a6c12bab40 tx=0x7feb0c004a40 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 msgr2=0x7feb1419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb080042f0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 shutdown_connections 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feafc0afae0 0x7feafc0b1ff0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb14108ae0 0x7feb141a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb14104440 0x7feb1419a720 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 --2- 172.21.15.94:0/53522333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb14103fa0 0x7feb1419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 >> 172.21.15.94:0/53522333 conn(0x7feb140fd390 msgr2=0x7feb14104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 shutdown_connections 2024-04-25T13:55:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:33.828+0000 7feb1a11f640 1 -- 172.21.15.94:0/53522333 wait complete. 2024-04-25T13:55:34.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 6950 34.92 MB/sec warmup 16 sec latency 289.557 ms 2024-04-25T13:55:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:34 smithi137 ceph-mon[32568]: pgmap v762: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 434 op/s 2024-04-25T13:55:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:34 smithi094 ceph-mon[29195]: pgmap v762: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 434 op/s 2024-04-25T13:55:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:34 smithi138 ceph-mon[32977]: pgmap v762: 97 pgs: 97 active+clean; 41 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 434 op/s 2024-04-25T13:55:35.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 7230 33.90 MB/sec warmup 17 sec latency 77.688 ms 2024-04-25T13:55:36.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 7243 32.04 MB/sec warmup 18 sec latency 723.327 ms 2024-04-25T13:55:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:36 smithi137 ceph-mon[32568]: pgmap v763: 97 pgs: 97 active+clean; 44 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 675 op/s 2024-04-25T13:55:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:36.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:36 smithi094 ceph-mon[29195]: pgmap v763: 97 pgs: 97 active+clean; 44 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 675 op/s 2024-04-25T13:55:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:36 smithi138 ceph-mon[32977]: pgmap v763: 97 pgs: 97 active+clean; 44 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 675 op/s 2024-04-25T13:55:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:37.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 7402 30.75 MB/sec warmup 19 sec latency 77.317 ms 2024-04-25T13:55:37.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:37.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:37.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:55:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:55:37.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:37.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:37.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:55:38.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 7568 29.33 MB/sec warmup 20 sec latency 142.877 ms 2024-04-25T13:55:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:38 smithi137 ceph-mon[32568]: pgmap v764: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-04-25T13:55:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:38 smithi094 ceph-mon[29195]: pgmap v764: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-04-25T13:55:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:38 smithi138 ceph-mon[32977]: pgmap v764: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 525 op/s 2024-04-25T13:55:39.134 INFO:tasks.workunit.client.0.smithi094.stdout: 10 7828 28.33 MB/sec warmup 21 sec latency 115.378 ms 2024-04-25T13:55:40.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 8214 28.10 MB/sec warmup 22 sec latency 69.823 ms 2024-04-25T13:55:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:40 smithi137 ceph-mon[32568]: pgmap v765: 97 pgs: 97 active+clean; 95 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.2 MiB/s wr, 541 op/s 2024-04-25T13:55:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:40 smithi094 ceph-mon[29195]: pgmap v765: 97 pgs: 97 active+clean; 95 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.2 MiB/s wr, 541 op/s 2024-04-25T13:55:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:40 smithi138 ceph-mon[32977]: pgmap v765: 97 pgs: 97 active+clean; 95 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.2 MiB/s wr, 541 op/s 2024-04-25T13:55:41.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 8806 27.83 MB/sec warmup 23 sec latency 85.421 ms 2024-04-25T13:55:42.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 9395 27.67 MB/sec warmup 24 sec latency 48.956 ms 2024-04-25T13:55:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:42 smithi137 ceph-mon[32568]: pgmap v766: 97 pgs: 97 active+clean; 172 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 18 MiB/s wr, 682 op/s 2024-04-25T13:55:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:42 smithi094 ceph-mon[29195]: pgmap v766: 97 pgs: 97 active+clean; 172 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 18 MiB/s wr, 682 op/s 2024-04-25T13:55:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:42 smithi138 ceph-mon[32977]: pgmap v766: 97 pgs: 97 active+clean; 172 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 18 MiB/s wr, 682 op/s 2024-04-25T13:55:43.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 9920 28.11 MB/sec warmup 25 sec latency 68.581 ms 2024-04-25T13:55:44.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 10391 28.66 MB/sec warmup 26 sec latency 122.963 ms 2024-04-25T13:55:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:44 smithi137 ceph-mon[32568]: pgmap v767: 97 pgs: 97 active+clean; 171 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 619 op/s 2024-04-25T13:55:44.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:44 smithi094 ceph-mon[29195]: pgmap v767: 97 pgs: 97 active+clean; 171 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 619 op/s 2024-04-25T13:55:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:44 smithi138 ceph-mon[32977]: pgmap v767: 97 pgs: 97 active+clean; 171 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 17 MiB/s wr, 619 op/s 2024-04-25T13:55:45.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 10699 28.29 MB/sec warmup 27 sec latency 124.089 ms 2024-04-25T13:55:46.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 10880 27.49 MB/sec warmup 28 sec latency 81.640 ms 2024-04-25T13:55:46.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:46 smithi137 ceph-mon[32568]: pgmap v768: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-04-25T13:55:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:46 smithi094 ceph-mon[29195]: pgmap v768: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-04-25T13:55:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:46 smithi138 ceph-mon[32977]: pgmap v768: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 18 MiB/s wr, 715 op/s 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/1147792895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 msgr2=0x7f4ac41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/1147792895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac41037b0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009900 tx=0x7f4ab402a410 comp rx=0 tx=0).stop 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/1147792895 shutdown_connections 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/1147792895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 0x7f4ac410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/1147792895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 0x7f4ac4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/1147792895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac41037b0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.930+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/1147792895 >> 172.21.15.94:0/1147792895 conn(0x7f4ac40fdca0 msgr2=0x7f4ac4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.931+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/1147792895 shutdown_connections 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.931+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/1147792895 wait complete. 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.931+0000 7f4ac8cf2640 1 Processor -- start 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.931+0000 7f4ac8cf2640 1 -- start start 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 0x7f4ac419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 0x7f4ac41a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac410e520 con 0x7f4ac4102ab0 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac410e3a0 con 0x7f4ac41089f0 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac410e6a0 con 0x7f4ac4103cf0 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40978/0 (socket says 172.21.15.94:40978) 2024-04-25T13:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 -- 172.21.15.94:0/3567336349 learned_addr learned my addr 172.21.15.94:0/3567336349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 0x7f4ac41a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 0x7f4ac419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 -- 172.21.15.94:0/3567336349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 msgr2=0x7f4ac419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 0x7f4ac419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 -- 172.21.15.94:0/3567336349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 msgr2=0x7f4ac41a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 0x7f4ac41a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 -- 172.21.15.94:0/3567336349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac41a1d70 con 0x7f4ac4102ab0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac259c640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006010 tx=0x7f4ab4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4abbfff640 1 -- 172.21.15.94:0/3567336349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab403f030 con 0x7f4ac4102ab0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4abbfff640 1 -- 172.21.15.94:0/3567336349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab4003aa0 con 0x7f4ac4102ab0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac41a2000 con 0x7f4ac4102ab0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.932+0000 7f4abbfff640 1 -- 172.21.15.94:0/3567336349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035070 con 0x7f4ac4102ab0 2024-04-25T13:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.933+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac41a24f0 con 0x7f4ac4102ab0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.935+0000 7f4acfba5740 1 Processor -- start 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 -- start start 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4abbfff640 1 -- 172.21.15.94:0/3567336349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4ab4042030 con 0x7f4ac4102ab0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c33970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 0x559751c33ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559751b99460 con 0x55975186a2f0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559751b53590 con 0x5597518931b0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4acfba5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559751b95420 con 0x5597518724d0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 0x559751c33ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 0x559751c33ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40980/0 (socket says 172.21.15.94:40980) 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac259c640 1 -- 172.21.15.94:0/1468836328 learned_addr learned my addr 172.21.15.94:0/1468836328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c33970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c3adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1468836328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 msgr2=0x559751c3adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c3adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1468836328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 msgr2=0x559751c33ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 0x559751c33ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.936+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1468836328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559751b600d0 con 0x5597518931b0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c33970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009f00 tx=0x7f4abc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb439f789c2a85ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4abbfff640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 0x7f4ab00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4ab8d99640 1 -- 172.21.15.94:0/1468836328 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc004d20 con 0x5597518931b0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4ac259c640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 0x7f4ab00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4ab8d99640 1 -- 172.21.15.94:0/1468836328 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc004030 con 0x5597518931b0 2024-04-25T13:55:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4ab8d99640 1 -- 172.21.15.94:0/1468836328 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc005240 con 0x5597518931b0 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.937+0000 7f4abbfff640 1 -- 172.21.15.94:0/3567336349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4ab40ff3a0 con 0x7f4ac4102ab0 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- 172.21.15.94:0/1468836328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 msgr2=0x559751c33970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4ac259c640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 0x7f4ab00b1ff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40c7770 tx=0x7f4ab4102e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c33970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009f00 tx=0x7f4abc011810 comp rx=0 tx=0).stop 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- 172.21.15.94:0/1468836328 shutdown_connections 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55975186a2f0 0x559751c33ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1468836328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c33970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- 172.21.15.94:0/1468836328 >> 172.21.15.94:0/1468836328 conn(0x559751887300 msgr2=0x559751c2f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- 172.21.15.94:0/1468836328 shutdown_connections 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- 172.21.15.94:0/1468836328 wait complete. 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- start start 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 Processor -- start 2024-04-25T13:55:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55975186a2f0 0x559751c3d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559751c3b840 con 0x5597518724d0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559751b20b60 con 0x5597518931b0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.938+0000 7f4acfba5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559751c3b9c0 con 0x55975186a2f0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4acfba5740 10 client.0 _ll_register_callbacks cb 0x559751b452b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40996/0 (socket says 172.21.15.94:40996) 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 -- 172.21.15.94:0/2540905036 learned_addr learned my addr 172.21.15.94:0/2540905036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55975186a2f0 0x559751c3d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 -- 172.21.15.94:0/2540905036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55975186a2f0 msgr2=0x559751c3d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55975186a2f0 0x559751c3d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 -- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 msgr2=0x559751c44ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c44ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 -- 172.21.15.94:0/2540905036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597518ab210 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00af00 tx=0x7f4aac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c75ff599ea2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011850 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4aac0119f0 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc004030 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4aac011bf0 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4acfba5740 10 client.20517 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.939+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559751977620 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc004030 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4aac024450 con 0x5597518724d0 2024-04-25T13:55:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 10 client.20517 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 10 client.20517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 20 client.20517 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559751851d80 0x559751c4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 4 client.20517 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559751ba7070 con 0x559751851d80 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559751851d80 0x559751c4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4aa9ffb640 10 client.20517 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.940+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559751851d80 0x559751c4b440 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1d4a9d53a6877e server_cookie=2406e1d96f8ffbfa in_seq=0 out_seq=0 2024-04-25T13:55:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/2540905036 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559751ba7070 con 0x559751851d80 2024-04-25T13:55:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4aa9ffb640 10 client.20517 handle_command_reply: tid=0 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 client.20517 shutdown 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 msgr2=0x559751c3db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00af00 tx=0x7f4aac00f730 comp rx=0 tx=0).stop 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 shutdown_connections 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559751851d80 0x559751c4b440 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597518931b0 0x559751c44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c3db80 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4aa9ffb640 0 client.20517 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 --2- 172.21.15.94:0/2540905036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55975186a2f0 0x559751c3d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 >> 172.21.15.94:0/2540905036 conn(0x559751887300 msgr2=0x559751c2e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 shutdown_connections 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 1 -- 172.21.15.94:0/2540905036 wait complete. 2024-04-25T13:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.944+0000 7f4acfba5740 20 client.20517 trim_cache size 0 max 16384 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 Processor -- start 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 -- start start 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c2eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41ab5b0 con 0x559751851d80 2024-04-25T13:55:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41ad1b0 con 0x55975186a2f0 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4acfba5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41abc80 con 0x5597518724d0 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4ac2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4ac2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41012/0 (socket says 172.21.15.94:41012) 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.959+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1847116335 learned_addr learned my addr 172.21.15.94:0/1847116335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c2eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1847116335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 msgr2=0x559751c2eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c2eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1847116335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 msgr2=0x559751c44a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c44a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/1847116335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559751c44f40 con 0x559751851d80 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:55:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e10b378dcab7e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4aabfff640 1 -- 172.21.15.94:0/1847116335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc01e040 con 0x559751851d80 2024-04-25T13:55:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4aabfff640 1 -- 172.21.15.94:0/1847116335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc004840 con 0x559751851d80 2024-04-25T13:55:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.960+0000 7f4aabfff640 1 -- 172.21.15.94:0/1847116335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc005460 con 0x559751851d80 2024-04-25T13:55:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- 172.21.15.94:0/1847116335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 msgr2=0x559751c3c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc011810 comp rx=0 tx=0).stop 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- 172.21.15.94:0/1847116335 shutdown_connections 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597518724d0 0x559751c2eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 --2- 172.21.15.94:0/1847116335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559751851d80 0x559751c3c4a0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- 172.21.15.94:0/1847116335 >> 172.21.15.94:0/1847116335 conn(0x559751887300 msgr2=0x559751c3bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- 172.21.15.94:0/1847116335 shutdown_connections 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- 172.21.15.94:0/1847116335 wait complete. 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 -- start start 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.961+0000 7f4acfba5740 1 Processor -- start 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559751851d80 0x559751c42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41acae0 con 0x5597518724d0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41ac350 con 0x55975186a2f0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41a9590 con 0x559751851d80 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 10 client.0 _ll_register_callbacks cb 0x7f4aac005f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41014/0 (socket says 172.21.15.94:41014) 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/4092571264 learned_addr learned my addr 172.21.15.94:0/4092571264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559751851d80 0x559751c42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/4092571264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559751851d80 msgr2=0x559751c42290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559751851d80 0x559751c42290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 msgr2=0x559751c427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 -- 172.21.15.94:0/4092571264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559751c43c70 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4ac2d9d640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76b1c3494687221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0049f0 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4abc030040 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.962+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc004840 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4abc004b90 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 10 client.20523 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559751cb37b0 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc004840 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4abc041070 con 0x5597518724d0 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 10 client.20523 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 10 client.20523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 20 client.20523 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597518931b0 0x559751cb6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 4 client.20523 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:55:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559751ba0ee0 con 0x5597518931b0 2024-04-25T13:55:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.963+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597518931b0 0x559751cb6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.964+0000 7f4ab9d9b640 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597518931b0 0x559751cb6030 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297c986fc005cc95 server_cookie=f7a2c5a360339be9 in_seq=0 out_seq=0 2024-04-25T13:55:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:46.964+0000 7f4aa9ffb640 10 client.20523 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:47.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 11055 26.79 MB/sec warmup 29 sec latency 146.580 ms 2024-04-25T13:55:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.443+0000 7f4aa9ffb640 1 -- 172.21.15.94:0/4092571264 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559751ba0ee0 con 0x5597518931b0 2024-04-25T13:55:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.443+0000 7f4aa9ffb640 10 client.20523 handle_command_reply: tid=0 2024-04-25T13:55:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 client.20523 shutdown 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 msgr2=0x559751c436b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc003f10 comp rx=0 tx=0).stop 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 shutdown_connections 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597518931b0 0x559751cb6030 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597518724d0 0x559751c436b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975186a2f0 0x559751c427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 --2- 172.21.15.94:0/4092571264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559751851d80 0x559751c42290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4aa9ffb640 0 client.20523 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 >> 172.21.15.94:0/4092571264 conn(0x559751887300 msgr2=0x559751c2eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 shutdown_connections 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 1 -- 172.21.15.94:0/4092571264 wait complete. 2024-04-25T13:55:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4acfba5740 20 client.20523 trim_cache size 0 max 16384 2024-04-25T13:55:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:55:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:55:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:55:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 msgr2=0x7f4ab00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 0x7f4ab00b1ff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40c7770 tx=0x7f4ab4102e80 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 msgr2=0x7f4ac4075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006010 tx=0x7f4ab4004270 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 shutdown_connections 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ab00afae0 0x7f4ab00b1ff0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac41089f0 0x7f4ac41a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ac4103cf0 0x7f4ac419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 --2- 172.21.15.94:0/3567336349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac4102ab0 0x7f4ac4075840 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.444+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 >> 172.21.15.94:0/3567336349 conn(0x7f4ac40fdca0 msgr2=0x7f4ac41049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.445+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 shutdown_connections 2024-04-25T13:55:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:55:47.445+0000 7f4ac8cf2640 1 -- 172.21.15.94:0/3567336349 wait complete. 2024-04-25T13:55:48.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 11276 26.10 MB/sec warmup 30 sec latency 261.806 ms 2024-04-25T13:55:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:48 smithi137 ceph-mon[32568]: pgmap v769: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 466 op/s 2024-04-25T13:55:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:48 smithi094 ceph-mon[29195]: pgmap v769: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 466 op/s 2024-04-25T13:55:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:48 smithi138 ceph-mon[32977]: pgmap v769: 97 pgs: 97 active+clean; 182 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 17 MiB/s wr, 466 op/s 2024-04-25T13:55:49.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 11616 25.79 MB/sec warmup 31 sec latency 83.563 ms 2024-04-25T13:55:50.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 12131 25.68 MB/sec warmup 32 sec latency 62.385 ms 2024-04-25T13:55:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:50 smithi137 ceph-mon[32568]: pgmap v770: 97 pgs: 97 active+clean; 183 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 395 op/s 2024-04-25T13:55:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:50 smithi094 ceph-mon[29195]: pgmap v770: 97 pgs: 97 active+clean; 183 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 395 op/s 2024-04-25T13:55:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:50 smithi138 ceph-mon[32977]: pgmap v770: 97 pgs: 97 active+clean; 183 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 395 op/s 2024-04-25T13:55:51.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 12603 25.52 MB/sec warmup 33 sec latency 146.936 ms 2024-04-25T13:55:52.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 13134 25.70 MB/sec warmup 34 sec latency 55.650 ms 2024-04-25T13:55:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:52 smithi137 ceph-mon[32568]: pgmap v771: 97 pgs: 97 active+clean; 180 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 520 op/s 2024-04-25T13:55:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:52 smithi094 ceph-mon[29195]: pgmap v771: 97 pgs: 97 active+clean; 180 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 520 op/s 2024-04-25T13:55:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:52 smithi138 ceph-mon[32977]: pgmap v771: 97 pgs: 97 active+clean; 180 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 520 op/s 2024-04-25T13:55:53.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 13663 26.05 MB/sec warmup 35 sec latency 60.347 ms 2024-04-25T13:55:54.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 14040 26.18 MB/sec warmup 36 sec latency 77.433 ms 2024-04-25T13:55:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:54 smithi137 ceph-mon[32568]: pgmap v772: 97 pgs: 97 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-04-25T13:55:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:54 smithi094 ceph-mon[29195]: pgmap v772: 97 pgs: 97 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-04-25T13:55:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:54 smithi138 ceph-mon[32977]: pgmap v772: 97 pgs: 97 active+clean; 178 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-04-25T13:55:55.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 14341 25.87 MB/sec warmup 37 sec latency 83.979 ms 2024-04-25T13:55:56.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 14457 25.29 MB/sec warmup 38 sec latency 414.369 ms 2024-04-25T13:55:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:56 smithi137 ceph-mon[32568]: pgmap v773: 97 pgs: 97 active+clean; 233 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 490 op/s 2024-04-25T13:55:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:56 smithi094 ceph-mon[29195]: pgmap v773: 97 pgs: 97 active+clean; 233 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 490 op/s 2024-04-25T13:55:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:56 smithi138 ceph-mon[32977]: pgmap v773: 97 pgs: 97 active+clean; 233 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.1 MiB/s wr, 490 op/s 2024-04-25T13:55:57.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 14677 24.86 MB/sec warmup 39 sec latency 200.649 ms 2024-04-25T13:55:58.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 15033 24.80 MB/sec warmup 40 sec latency 128.881 ms 2024-04-25T13:55:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:55:58 smithi137 ceph-mon[32568]: pgmap v774: 97 pgs: 97 active+clean; 238 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-25T13:55:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:55:58 smithi094 ceph-mon[29195]: pgmap v774: 97 pgs: 97 active+clean; 238 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-25T13:55:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:55:58 smithi138 ceph-mon[32977]: pgmap v774: 97 pgs: 97 active+clean; 238 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.5 MiB/s wr, 396 op/s 2024-04-25T13:55:59.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 15429 24.48 MB/sec warmup 41 sec latency 98.331 ms 2024-04-25T13:55:59.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:55:59.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:55:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 -- 172.21.15.94:0/3758199987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 msgr2=0x7f1f401036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/3758199987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f401036e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009900 tx=0x7f1f3c02a410 comp rx=0 tx=0).stop 2024-04-25T13:55:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 -- 172.21.15.94:0/3758199987 shutdown_connections 2024-04-25T13:55:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/3758199987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 0x7f1f4010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/3758199987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 0x7f1f40104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/3758199987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f401036e0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 -- 172.21.15.94:0/3758199987 >> 172.21.15.94:0/3758199987 conn(0x7f1f400fdbd0 msgr2=0x7f1f40100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 -- 172.21.15.94:0/3758199987 shutdown_connections 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.845+0000 7f1f47fb2640 1 -- 172.21.15.94:0/3758199987 wait complete. 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 Processor -- start 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 -- start start 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 0x7f1f401999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 0x7f1f401a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010e4e0 con 0x7f1f401029e0 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010e360 con 0x7f1f40108920 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f47fb2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010e660 con 0x7f1f40103c20 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39428/0 (socket says 172.21.15.94:39428) 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2401184294 learned_addr learned my addr 172.21.15.94:0/2401184294 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f4554d640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 0x7f1f401999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f4654f640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 0x7f1f401a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2401184294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 msgr2=0x7f1f401999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 0x7f1f401999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2401184294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 msgr2=0x7f1f401a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.846+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 0x7f1f401a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2401184294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f401a0fb0 con 0x7f1f401029e0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f36ffd640 1 -- 172.21.15.94:0/2401184294 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c03f030 con 0x7f1f401029e0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f401a1240 con 0x7f1f401029e0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f36ffd640 1 -- 172.21.15.94:0/2401184294 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c031910 con 0x7f1f401029e0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f36ffd640 1 -- 172.21.15.94:0/2401184294 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c035070 con 0x7f1f401029e0 2024-04-25T13:55:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.847+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401a1780 con 0x7f1f401029e0 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.850+0000 7f1f36ffd640 1 -- 172.21.15.94:0/2401184294 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1f3c042030 con 0x7f1f401029e0 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.850+0000 7f1f4ee68740 1 Processor -- start 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.850+0000 7f1f4ee68740 1 -- start start 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 0x55b0b27aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b27aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23ec6b0 0x55b0b27afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0b2713d80 con 0x55b0b23ec6b0 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0b26cd8b0 con 0x55b0b23e44d0 2024-04-25T13:55:59.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0b270fd40 con 0x55b0b240d390 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f36ffd640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 0x7f1f2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4654f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 0x55b0b27aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b27aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4554d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23ec6b0 0x55b0b27afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4654f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 0x55b0b27aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58092/0 (socket says 172.21.15.94:58092) 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4654f640 1 -- 172.21.15.94:0/1648369703 learned_addr learned my addr 172.21.15.94:0/1648369703 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4654f640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 0x7f1f2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f36ffd640 1 -- 172.21.15.94:0/2401184294 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1f3c0fefc0 con 0x7f1f401029e0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 -- 172.21.15.94:0/1648369703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 msgr2=0x55b0b27aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 0x55b0b27aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 -- 172.21.15.94:0/1648369703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23ec6b0 msgr2=0x55b0b27afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23ec6b0 0x55b0b27afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 -- 172.21.15.94:0/1648369703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b269ea40 con 0x55b0b23e44d0 2024-04-25T13:55:59.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f4654f640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 0x7f1f2c0b2000 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55b0b27afab0 tx=0x7f1f38009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.851+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b27aefe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0fca70 tx=0x7f1f3c096000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec1e6255cd48833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.852+0000 7f1f2bfff640 1 -- 172.21.15.94:0/1648369703 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c049070 con 0x55b0b23e44d0 2024-04-25T13:55:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.852+0000 7f1f2bfff640 1 -- 172.21.15.94:0/1648369703 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c0c6fc0 con 0x55b0b23e44d0 2024-04-25T13:55:59.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.852+0000 7f1f2bfff640 1 -- 172.21.15.94:0/1648369703 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0d0260 con 0x55b0b23e44d0 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1648369703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 msgr2=0x55b0b27aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b27aefe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0fca70 tx=0x7f1f3c096000 comp rx=0 tx=0).stop 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1648369703 shutdown_connections 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23ec6b0 0x55b0b27afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b27aefe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1648369703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b240d390 0x55b0b27aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1648369703 >> 172.21.15.94:0/1648369703 conn(0x55b0b24014e0 msgr2=0x55b0b27aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1648369703 shutdown_connections 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1648369703 wait complete. 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 -- start start 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.853+0000 7f1f4ee68740 1 Processor -- start 2024-04-25T13:55:59.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b240d390 0x55b0b27bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0b26cd8b0 con 0x55b0b23e44d0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0b269ad10 con 0x55b0b240d390 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0b27b0540 con 0x55b0b23ec6b0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 10 client.0 _ll_register_callbacks cb 0x55b0b26c7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39454/0 (socket says 172.21.15.94:39454) 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 -- 172.21.15.94:0/4242983179 learned_addr learned my addr 172.21.15.94:0/4242983179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4554d640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4654f640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b240d390 0x55b0b27bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 -- 172.21.15.94:0/4242983179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 msgr2=0x55b0b27b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 -- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b240d390 msgr2=0x55b0b27bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b240d390 0x55b0b27bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 -- 172.21.15.94:0/4242983179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b24ef7d0 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c4720 tx=0x7f1f3c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18ef1ab7905caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c099020 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1f3c09a220 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.854+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c0c6fc0 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c049050 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 10 client.20535 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0b24253f0 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c0c6fc0 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1f3c0cfe60 con 0x55b0b23e44d0 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 10 client.20535 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 10 client.20535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 20 client.20535 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:59.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b23cbf60 0x55b0b27c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 4 client.20535 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b2721990 con 0x55b0b23cbf60 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.855+0000 7f1f4554d640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b23cbf60 0x55b0b27c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.856+0000 7f1f297fa640 10 client.20535 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.856+0000 7f1f4554d640 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b23cbf60 0x55b0b27c5b30 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2760772fbf1cb15 server_cookie=ecb385974944a794 in_seq=0 out_seq=0 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.858+0000 7f1f297fa640 1 -- 172.21.15.94:0/4242983179 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0b2721990 con 0x55b0b23cbf60 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.858+0000 7f1f297fa640 10 client.20535 handle_command_reply: tid=0 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.858+0000 7f1f4ee68740 1 client.20535 shutdown 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.858+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 msgr2=0x55b0b27b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.858+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c4720 tx=0x7f1f3c009260 comp rx=0 tx=0).stop 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 shutdown_connections 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b23cbf60 0x55b0b27c5b30 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f297fa640 0 client.20535 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:59.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b240d390 0x55b0b27bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/4242983179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27b8520 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 >> 172.21.15.94:0/4242983179 conn(0x55b0b24014e0 msgr2=0x55b0b27ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 shutdown_connections 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 1 -- 172.21.15.94:0/4242983179 wait complete. 2024-04-25T13:55:59.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.859+0000 7f1f4ee68740 20 client.20535 trim_cache size 0 max 16384 2024-04-25T13:55:59.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 Processor -- start 2024-04-25T13:55:59.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 -- start start 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23cbf60 0x55b0b27b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401a9320 con 0x55b0b23e44d0 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401aaf20 con 0x55b0b23cbf60 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.877+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401a99f0 con 0x55b0b23ec6b0 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39458/0 (socket says 172.21.15.94:39458) 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 -- 172.21.15.94:0/1604522660 learned_addr learned my addr 172.21.15.94:0/1604522660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4654f640 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23cbf60 0x55b0b27b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 -- 172.21.15.94:0/1604522660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 msgr2=0x55b0b27bfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27bfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 -- 172.21.15.94:0/1604522660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23cbf60 msgr2=0x55b0b27b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23cbf60 0x55b0b27b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 -- 172.21.15.94:0/1604522660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b23e50f0 con 0x55b0b23e44d0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f4554d640 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1f4019a510 tx=0x7f1f30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7c40b3250a6590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f2b7fe640 1 -- 172.21.15.94:0/1604522660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30010430 con 0x55b0b23e44d0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f2b7fe640 1 -- 172.21.15.94:0/1604522660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30010a80 con 0x55b0b23e44d0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.878+0000 7f1f2b7fe640 1 -- 172.21.15.94:0/1604522660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f300206b0 con 0x55b0b23e44d0 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1604522660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 msgr2=0x55b0b27bf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1f4019a510 tx=0x7f1f30005f90 comp rx=0 tx=0).stop 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1604522660 shutdown_connections 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b27bfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23e44d0 0x55b0b27bf950 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/1604522660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23cbf60 0x55b0b27b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1604522660 >> 172.21.15.94:0/1604522660 conn(0x55b0b24014e0 msgr2=0x55b0b27b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1604522660 shutdown_connections 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.879+0000 7f1f4ee68740 1 -- 172.21.15.94:0/1604522660 wait complete. 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 -- start start 2024-04-25T13:55:59.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 Processor -- start 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b2829080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b282ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401aa850 con 0x55b0b23cbf60 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401aa0c0 con 0x55b0b23e44d0 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f401a7300 con 0x55b0b23ec6b0 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4ee68740 10 client.0 _ll_register_callbacks cb 0x7f1f30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:55:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f45d4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39468/0 (socket says 172.21.15.94:39468) 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2029156924 learned_addr learned my addr 172.21.15.94:0/2029156924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4554d640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b2829080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.880+0000 7f1f4654f640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b282ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2029156924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 msgr2=0x55b0b282ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b282ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 msgr2=0x55b0b2829080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b2829080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 -- 172.21.15.94:0/2029156924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b26d44b0 con 0x55b0b23cbf60 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f45d4e640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c2e20 tx=0x7f1f3c0ff2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accdc293be0e7794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c099040 con 0x55b0b23cbf60 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f1f3c0c7920 con 0x55b0b23cbf60 2024-04-25T13:55:59.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f30010a80 con 0x55b0b23cbf60 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c0c3370 con 0x55b0b23cbf60 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f4ee68740 10 client.20541 fetch_fsmap learned FSMap version 32 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0b26d44b0 con 0x55b0b23cbf60 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.881+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30010a80 con 0x55b0b23cbf60 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f1f3c0bbce0 con 0x55b0b23cbf60 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 10 client.20541 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 10 client.20541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 20 client.20541 populate_metadata read hostname 'smithi094' 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b240d390 0x55b0b2833d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 4 client.20541 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0b271b800 con 0x55b0b240d390 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4554d640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b240d390 0x55b0b2833d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:59.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f297fa640 10 client.20541 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:59.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.882+0000 7f1f4554d640 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b240d390 0x55b0b2833d20 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983ee97833aa8e0b server_cookie=67ab3427932efc80 in_seq=0 out_seq=0 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.887+0000 7f1f297fa640 1 -- 172.21.15.94:0/2029156924 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0b271b800 con 0x55b0b240d390 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f297fa640 10 client.20541 handle_command_reply: tid=0 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 client.20541 shutdown 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 msgr2=0x55b0b2828b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0c2e20 tx=0x7f1f3c0ff2c0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 shutdown_connections 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b0b240d390 0x55b0b2833d20 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b23ec6b0 0x55b0b282ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b23e44d0 0x55b0b2829080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f297fa640 0 client.20541 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:55:59.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 --2- 172.21.15.94:0/2029156924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b23cbf60 0x55b0b2828b40 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 >> 172.21.15.94:0/2029156924 conn(0x55b0b24014e0 msgr2=0x55b0b27adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 shutdown_connections 2024-04-25T13:55:59.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 1 -- 172.21.15.94:0/2029156924 wait complete. 2024-04-25T13:55:59.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.888+0000 7f1f4ee68740 20 client.20541 trim_cache size 0 max 16384 2024-04-25T13:55:59.889 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:55:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:55:59.890 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:55:59.890 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 msgr2=0x7f1f2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 0x7f1f2c0b2000 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55b0b27afab0 tx=0x7f1f38009500 comp rx=0 tx=0).stop 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 msgr2=0x7f1f40199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c004270 comp rx=0 tx=0).stop 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 shutdown_connections 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1f2c0afaf0 0x7f1f2c0b2000 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f40108920 0x7f1f401a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f40103c20 0x7f1f401999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 --2- 172.21.15.94:0/2401184294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f401029e0 0x7f1f40199470 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:59.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 >> 172.21.15.94:0/2401184294 conn(0x7f1f400fdbd0 msgr2=0x7f1f40104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:59.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 shutdown_connections 2024-04-25T13:55:59.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:55:59.889+0000 7f1f47fb2640 1 -- 172.21.15.94:0/2401184294 wait complete. 2024-04-25T13:55:59.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:55:59.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:00.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 -- 172.21.15.94:0/1910214338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70103c50 msgr2=0x7fec70104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 --2- 172.21.15.94:0/1910214338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70103c50 0x7fec70104060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fec64009900 tx=0x7fec6402a410 comp rx=0 tx=0).stop 2024-04-25T13:56:00.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 -- 172.21.15.94:0/1910214338 shutdown_connections 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 --2- 172.21.15.94:0/1910214338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70108950 0x7fec7010ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 --2- 172.21.15.94:0/1910214338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70103c50 0x7fec70104060 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 --2- 172.21.15.94:0/1910214338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec70103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 -- 172.21.15.94:0/1910214338 >> 172.21.15.94:0/1910214338 conn(0x7fec700fdc00 msgr2=0x7fec701000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 -- 172.21.15.94:0/1910214338 shutdown_connections 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.016+0000 7fec75dd7640 1 -- 172.21.15.94:0/1910214338 wait complete. 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 Processor -- start 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 -- start start 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec7006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70103c50 0x7fec7019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec7010e530 con 0x7fec70108950 2024-04-25T13:56:00.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec7010e3b0 con 0x7fec70103c50 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec75dd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec7010e6b0 con 0x7fec70102a10 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.017+0000 7fec6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec7006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39486/0 (socket says 172.21.15.94:39486) 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6effd640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70103c50 0x7fec7019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 -- 172.21.15.94:0/3555891879 learned_addr learned my addr 172.21.15.94:0/3555891879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 -- 172.21.15.94:0/3555891879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 msgr2=0x7fec7006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec7006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 -- 172.21.15.94:0/3555891879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70103c50 msgr2=0x7fec7019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70103c50 0x7fec7019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 -- 172.21.15.94:0/3555891879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec701a1dc0 con 0x7fec70108950 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec7006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6ffff640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fec6800eb10 tx=0x7fec6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68004b20 con 0x7fec70108950 2024-04-25T13:56:00.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec701a20b0 con 0x7fec70108950 2024-04-25T13:56:00.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68004cc0 con 0x7fec70108950 2024-04-25T13:56:00.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68010670 con 0x7fec70108950 2024-04-25T13:56:00.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.018+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a2600 con 0x7fec70108950 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.019+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec70107760 con 0x7fec70108950 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.021+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fec68005000 con 0x7fec70108950 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.021+0000 7fec6cff9640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 0x7fec580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.022+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 0x7fec580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.022+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fec680df420 con 0x7fec70108950 2024-04-25T13:56:00.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.022+0000 7fec6f7fe640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 0x7fec580b2000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec70103610 tx=0x7fec60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.027+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec68053ff0 con 0x7fec70108950 2024-04-25T13:56:00.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 15965 24.33 MB/sec warmup 42 sec latency 74.325 ms 2024-04-25T13:56:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.271+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec7002db50 con 0x7fec70108950 2024-04-25T13:56:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.271+0000 7fec6cff9640 1 -- 172.21.15.94:0/3555891879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11948 (secure 0 0 0) 0x7fec680a7f60 con 0x7fec70108950 2024-04-25T13:56:00.271 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:56:00.271 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:55:20.694859+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:56:00.271 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 32 2024-04-25T13:56:00.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.273+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 msgr2=0x7fec580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.273+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 0x7fec580b2000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec70103610 tx=0x7fec60005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.273+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 msgr2=0x7fec701a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.273+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fec6800eb10 tx=0x7fec6800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 shutdown_connections 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fec580afaf0 0x7fec580b2000 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec70108950 0x7fec701a15a0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec70103c50 0x7fec7019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 --2- 172.21.15.94:0/3555891879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec70102a10 0x7fec7006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 >> 172.21.15.94:0/3555891879 conn(0x7fec700fdc00 msgr2=0x7fec70104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 shutdown_connections 2024-04-25T13:56:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.274+0000 7fec75dd7640 1 -- 172.21.15.94:0/3555891879 wait complete. 2024-04-25T13:56:00.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:56:00.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:00 smithi137 ceph-mon[32568]: pgmap v775: 97 pgs: 97 active+clean; 237 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 407 op/s 2024-04-25T13:56:00.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- 172.21.15.94:0/1328847860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181098b0 msgr2=0x7fbe1810bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1328847860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181098b0 0x7fbe1810bcd0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009ca0 tx=0x7fbe1002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- 172.21.15.94:0/1328847860 shutdown_connections 2024-04-25T13:56:00.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1328847860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181098b0 0x7fbe1810bcd0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1328847860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe18109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1328847860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181045f0 0x7fbe18106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- 172.21.15.94:0/1328847860 >> 172.21.15.94:0/1328847860 conn(0x7fbe180fdbf0 msgr2=0x7fbe181000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- 172.21.15.94:0/1328847860 shutdown_connections 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- 172.21.15.94:0/1328847860 wait complete. 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 Processor -- start 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.402+0000 7fbe1e213640 1 -- start start 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe1819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 0x7fbe181a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810f630 con 0x7fbe181045f0 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810f4b0 con 0x7fbe181098b0 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe1e213640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810f7b0 con 0x7fbe18106f50 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39498/0 (socket says 172.21.15.94:39498) 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 -- 172.21.15.94:0/1916134729 learned_addr learned my addr 172.21.15.94:0/1916134729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe17fff640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 0x7fbe181a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe1819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 -- 172.21.15.94:0/1916134729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 msgr2=0x7fbe1819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe1819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 -- 172.21.15.94:0/1916134729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 msgr2=0x7fbe181a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 0x7fbe181a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.403+0000 7fbe177fe640 1 -- 172.21.15.94:0/1916134729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181a1d20 con 0x7fbe181045f0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe1819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe17fff640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 0x7fbe181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe177fe640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00e9c0 tx=0x7fbe0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe14ff9640 1 -- 172.21.15.94:0/1916134729 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00cd80 con 0x7fbe181045f0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe14ff9640 1 -- 172.21.15.94:0/1916134729 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c0054d0 con 0x7fbe181045f0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe181a1fb0 con 0x7fbe181045f0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe14ff9640 1 -- 172.21.15.94:0/1916134729 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c010430 con 0x7fbe181045f0 2024-04-25T13:56:00.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.404+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a2470 con 0x7fbe181045f0 2024-04-25T13:56:00.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 Processor -- start 2024-04-25T13:56:00.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 -- start start 2024-04-25T13:56:00.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe14ff9640 1 -- 172.21.15.94:0/1916134729 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbe0c0105d0 con 0x7fbe181045f0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 0x55b672782d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672783a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b6726e6d80 con 0x55b6723b74d0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6726a0a20 con 0x55b6723e0390 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe2519f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6726e2d40 con 0x55b6723bf6b0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 0x55b672782d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33898/0 (socket says 172.21.15.94:33898) 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.407+0000 7fbe17fff640 1 -- 172.21.15.94:0/47426203 learned_addr learned my addr 172.21.15.94:0/47426203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672783a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 -- 172.21.15.94:0/47426203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 msgr2=0x55b672783a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672783a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 -- 172.21.15.94:0/47426203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 msgr2=0x55b672782d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 0x55b672782d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 -- 172.21.15.94:0/47426203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6726abc30 con 0x55b6723e0390 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672783a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe14ff9640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 0x7fbe000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe177fe640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 0x55b672782d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe17fff640 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe18075eb0 tx=0x7fbe1002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8e850d2d4ade06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe177fe640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 0x7fbe000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbe14ff9640 1 -- 172.21.15.94:0/1916134729 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbe0c0df060 con 0x7fbe181045f0 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbdfdffb640 1 -- 172.21.15.94:0/47426203 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10031450 con 0x55b6723e0390 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbdfdffb640 1 -- 172.21.15.94:0/47426203 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe10036040 con 0x55b6723e0390 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.408+0000 7fbdfdffb640 1 -- 172.21.15.94:0/47426203 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100045b0 con 0x55b6723e0390 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 -- 172.21.15.94:0/47426203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 msgr2=0x55b672782810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe177fe640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 0x7fbe000b2000 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c6a0 tx=0x7fbe0c0e2bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe18075eb0 tx=0x7fbe1002f410 comp rx=0 tx=0).stop 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 -- 172.21.15.94:0/47426203 shutdown_connections 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672783a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723b74d0 0x55b672782d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 --2- 172.21.15.94:0/47426203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723e0390 0x55b672782810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 -- 172.21.15.94:0/47426203 >> 172.21.15.94:0/47426203 conn(0x55b6723d44e0 msgr2=0x55b67277e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 -- 172.21.15.94:0/47426203 shutdown_connections 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.409+0000 7fbe2519f740 1 -- 172.21.15.94:0/47426203 wait complete. 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 -- start start 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 Processor -- start 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723b74d0 0x55b67278c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 0x55b672793810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b672784350 con 0x55b6723e0390 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b67266ddc0 con 0x55b6723bf6b0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6727844d0 con 0x55b6723b74d0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe2519f740 10 client.0 _ll_register_callbacks cb 0x55b672689600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 0x55b672793810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33900/0 (socket says 172.21.15.94:33900) 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 0x55b672793810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39514/0 (socket says 172.21.15.94:39514) 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe16ffd640 1 -- 172.21.15.94:0/3956199251 learned_addr learned my addr 172.21.15.94:0/3956199251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.410+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723b74d0 0x55b67278c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 -- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723b74d0 msgr2=0x55b67278c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723b74d0 0x55b67278c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 -- 172.21.15.94:0/3956199251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 msgr2=0x55b672793810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 0x55b672793810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 -- 172.21.15.94:0/3956199251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6723f83f0 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe08013d30 tx=0x7fbe080117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2aacfceeddd581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08004250 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbe080043f0 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe10036040 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbe08005020 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe2519f740 10 client.29999 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.411+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6724c27d0 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe10036040 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbe0801a070 con 0x55b6723bf6b0 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 10 client.29999 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 10 client.29999 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:56:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 20 client.29999 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b67239ef60 0x55b67279a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 4 client.29999 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6726f4990 con 0x55b67239ef60 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.412+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b67239ef60 0x55b67279a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbdf6ffd640 10 client.29999 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b67239ef60 0x55b67279a550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d91f3d373f5e87bf server_cookie=453cef0631ee775a in_seq=0 out_seq=0 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/3956199251 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b6726f4990 con 0x55b67239ef60 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbdf6ffd640 10 client.29999 handle_command_reply: tid=0 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbe2519f740 1 client.29999 shutdown 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 msgr2=0x55b67278c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.413+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe08013d30 tx=0x7fbe080117b0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 shutdown_connections 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b67239ef60 0x55b67279a550 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6723e0390 0x55b672793810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723bf6b0 0x55b67278c8b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbdf6ffd640 0 client.29999 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3956199251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723b74d0 0x55b67278c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 >> 172.21.15.94:0/3956199251 conn(0x55b6723d44e0 msgr2=0x55b67277d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 shutdown_connections 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 1 -- 172.21.15.94:0/3956199251 wait complete. 2024-04-25T13:56:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.414+0000 7fbe2519f740 20 client.29999 trim_cache size 0 max 16384 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 Processor -- start 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 -- start start 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672793730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b67278bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ab4e0 con 0x55b67239ef60 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ad0e0 con 0x55b6723b74d0 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe2519f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181abbb0 con 0x55b6723bf6b0 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39520/0 (socket says 172.21.15.94:39520) 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe17fff640 1 -- 172.21.15.94:0/3500224345 learned_addr learned my addr 172.21.15.94:0/3500224345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672793730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.428+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b67278bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 -- 172.21.15.94:0/3500224345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 msgr2=0x55b67278bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b67278bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 -- 172.21.15.94:0/3500224345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 msgr2=0x55b672793730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672793730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 -- 172.21.15.94:0/3500224345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b672787c80 con 0x55b67239ef60 2024-04-25T13:56:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672793730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbe17fff640 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006010 tx=0x7fbe1002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1df918c321e704b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbdfd7fa640 1 -- 172.21.15.94:0/3500224345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10004500 con 0x55b67239ef60 2024-04-25T13:56:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbdfd7fa640 1 -- 172.21.15.94:0/3500224345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe10036070 con 0x55b67239ef60 2024-04-25T13:56:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.429+0000 7fbdfd7fa640 1 -- 172.21.15.94:0/3500224345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1002f880 con 0x55b67239ef60 2024-04-25T13:56:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- 172.21.15.94:0/3500224345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 msgr2=0x55b67278b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006010 tx=0x7fbe1002f3a0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- 172.21.15.94:0/3500224345 shutdown_connections 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b67278bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672793730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 --2- 172.21.15.94:0/3500224345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b67278b600 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- 172.21.15.94:0/3500224345 >> 172.21.15.94:0/3500224345 conn(0x55b6723d44e0 msgr2=0x55b67278a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- 172.21.15.94:0/3500224345 shutdown_connections 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- 172.21.15.94:0/3500224345 wait complete. 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.430+0000 7fbe2519f740 1 -- start start 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 Processor -- start 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672791130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672792060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aca10 con 0x55b67239ef60 2024-04-25T13:56:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ac280 con 0x55b6723b74d0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a94c0 con 0x55b6723bf6b0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe2519f740 10 client.0 _ll_register_callbacks cb 0x7fbe08009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39524/0 (socket says 172.21.15.94:39524) 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 -- 172.21.15.94:0/960813166 learned_addr learned my addr 172.21.15.94:0/960813166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672791130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672792060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 -- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 msgr2=0x55b672792060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672792060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 -- 172.21.15.94:0/960813166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 msgr2=0x55b672791130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672791130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 -- 172.21.15.94:0/960813166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b672793c70 con 0x55b67239ef60 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbe17fff640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fbe1002ade0 tx=0x7fbe10000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcd47cc8916c39c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.431+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10031450 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fbe100315f0 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe10036070 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbe100317f0 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 10 client.20565 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6727936d0 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe10036070 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fbe10051040 con 0x55b67239ef60 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 10 client.20565 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 10 client.20565 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 20 client.20565 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b6723e0390 0x55b672804a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 4 client.20565 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6726ee800 con 0x55b6723e0390 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.432+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b6723e0390 0x55b672804a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbdf6ffd640 10 client.20565 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe16ffd640 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b6723e0390 0x55b672804a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d1f7356a31d4c16 server_cookie=74490ba7fe161616 in_seq=0 out_seq=0 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbdf6ffd640 1 -- 172.21.15.94:0/960813166 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6726ee800 con 0x55b6723e0390 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbdf6ffd640 10 client.20565 handle_command_reply: tid=0 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 client.20565 shutdown 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 msgr2=0x55b672790bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fbe1002ade0 tx=0x7fbe10000f40 comp rx=0 tx=0).stop 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 shutdown_connections 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55b6723e0390 0x55b672804a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6723bf6b0 0x55b672792060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6723b74d0 0x55b672791130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbdf6ffd640 0 client.20565 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:00.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 --2- 172.21.15.94:0/960813166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b67239ef60 0x55b672790bf0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 >> 172.21.15.94:0/960813166 conn(0x55b6723d44e0 msgr2=0x55b67278b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 shutdown_connections 2024-04-25T13:56:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.433+0000 7fbe2519f740 1 -- 172.21.15.94:0/960813166 wait complete. 2024-04-25T13:56:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe2519f740 20 client.20565 trim_cache size 0 max 16384 2024-04-25T13:56:00.439 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 msgr2=0x7fbe000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 0x7fbe000b2000 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c6a0 tx=0x7fbe0c0e2bd0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 msgr2=0x7fbe1819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00e9c0 tx=0x7fbe0c00ee80 comp rx=0 tx=0).stop 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 shutdown_connections 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe000afaf0 0x7fbe000b2000 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe181098b0 0x7fbe181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18106f50 0x7fbe1819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 --2- 172.21.15.94:0/1916134729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181045f0 0x7fbe1819a1e0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 >> 172.21.15.94:0/1916134729 conn(0x7fbe180fdbf0 msgr2=0x7fbe181050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 shutdown_connections 2024-04-25T13:56:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.434+0000 7fbe1e213640 1 -- 172.21.15.94:0/1916134729 wait complete. 2024-04-25T13:56:00.443 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:00 smithi094 ceph-mon[29195]: pgmap v775: 97 pgs: 97 active+clean; 237 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 407 op/s 2024-04-25T13:56:00.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:56:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:00 smithi138 ceph-mon[32977]: pgmap v775: 97 pgs: 97 active+clean; 237 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.9 MiB/s wr, 407 op/s 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 -- 172.21.15.94:0/2765462177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1050e0 msgr2=0x7f86bc1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 --2- 172.21.15.94:0/2765462177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1050e0 0x7f86bc1054f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009900 tx=0x7f86b802a410 comp rx=0 tx=0).stop 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 -- 172.21.15.94:0/2765462177 shutdown_connections 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 --2- 172.21.15.94:0/2765462177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc102290 0x7f86bc10bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 --2- 172.21.15.94:0/2765462177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 0x7f86bc101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 --2- 172.21.15.94:0/2765462177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1050e0 0x7f86bc1054f0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 -- 172.21.15.94:0/2765462177 >> 172.21.15.94:0/2765462177 conn(0x7f86bc0fd390 msgr2=0x7f86bc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 -- 172.21.15.94:0/2765462177 shutdown_connections 2024-04-25T13:56:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.553+0000 7f86c3f34640 1 -- 172.21.15.94:0/2765462177 wait complete. 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 Processor -- start 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 -- start start 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 0x7f86bc104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc1050e0 0x7f86bc102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10f650 con 0x7f86bc102290 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10f4d0 con 0x7f86bc1050e0 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c3f34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10f7d0 con 0x7f86bc101790 2024-04-25T13:56:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39546/0 (socket says 172.21.15.94:39546) 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 -- 172.21.15.94:0/1581903187 learned_addr learned my addr 172.21.15.94:0/1581903187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 0x7f86bc104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc1050e0 0x7f86bc102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 -- 172.21.15.94:0/1581903187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 msgr2=0x7f86bc104290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 0x7f86bc104290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.554+0000 7f86c14cf640 1 -- 172.21.15.94:0/1581903187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc1050e0 msgr2=0x7f86bc102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc1050e0 0x7f86bc102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86c14cf640 1 -- 172.21.15.94:0/1581903187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc1035a0 con 0x7f86bc102290 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86b2ffd640 1 -- 172.21.15.94:0/1581903187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00cd80 con 0x7f86bc102290 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86b2ffd640 1 -- 172.21.15.94:0/1581903187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac0054d0 con 0x7f86bc102290 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1a4370 con 0x7f86bc102290 2024-04-25T13:56:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86b2ffd640 1 -- 172.21.15.94:0/1581903187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010690 con 0x7f86bc102290 2024-04-25T13:56:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.555+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a47e0 con 0x7f86bc102290 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.558+0000 7f86cade9740 1 Processor -- start 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.558+0000 7f86cade9740 1 -- start start 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.558+0000 7f86b2ffd640 1 -- 172.21.15.94:0/1581903187 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f86ac004e90 con 0x7f86bc102290 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559690241460 con 0x55968ff3b1b0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596901fb590 con 0x55968ff122f0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86cade9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55969023d420 con 0x55968ff1a4d0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39556/0 (socket says 172.21.15.94:39556) 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 -- 172.21.15.94:0/1006174349 learned_addr learned my addr 172.21.15.94:0/1006174349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 -- 172.21.15.94:0/1006174349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 msgr2=0x5596902e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 -- 172.21.15.94:0/1006174349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 msgr2=0x5596902dbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902dbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 -- 172.21.15.94:0/1006174349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596902080d0 con 0x55968ff3b1b0 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86b2ffd640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 0x7f86a80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fd0 tx=0x7f86b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce34d714a5590d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.559+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 0x7f86a80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.560+0000 7f86b2ffd640 1 -- 172.21.15.94:0/1581903187 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f86ac0df0d0 con 0x7f86bc102290 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.560+0000 7f86a7fff640 1 -- 172.21.15.94:0/1006174349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4004d20 con 0x55968ff3b1b0 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.560+0000 7f86a7fff640 1 -- 172.21.15.94:0/1006174349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b4004050 con 0x55968ff3b1b0 2024-04-25T13:56:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.560+0000 7f86a7fff640 1 -- 172.21.15.94:0/1006174349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4005260 con 0x55968ff3b1b0 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- 172.21.15.94:0/1006174349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 msgr2=0x5596902db970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 0x7f86a80b1ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005ab0 tx=0x7f86b8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fd0 tx=0x7f86b40118d0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- 172.21.15.94:0/1006174349 shutdown_connections 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 --2- 172.21.15.94:0/1006174349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902db970 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- 172.21.15.94:0/1006174349 >> 172.21.15.94:0/1006174349 conn(0x55968ff2f300 msgr2=0x5596902d7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- 172.21.15.94:0/1006174349 shutdown_connections 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- 172.21.15.94:0/1006174349 wait complete. 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.561+0000 7f86cade9740 1 -- start start 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 Processor -- start 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 0x5596902ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5596902e3840 con 0x55968ff122f0 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596901c8b60 con 0x55968ff3b1b0 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596902e39c0 con 0x55968ff1a4d0 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86cade9740 10 client.0 _ll_register_callbacks cb 0x5596901ed2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c1cd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c1cd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39564/0 (socket says 172.21.15.94:39564) 2024-04-25T13:56:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c1cd0640 1 -- 172.21.15.94:0/1249939211 learned_addr learned my addr 172.21.15.94:0/1249939211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 0x5596902ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 -- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 msgr2=0x5596902ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 0x5596902ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 -- 172.21.15.94:0/1249939211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 msgr2=0x5596902e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.562+0000 7f86c14cf640 1 -- 172.21.15.94:0/1249939211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968ff53210 con 0x55968ff1a4d0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 0x5596902ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e5c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0dcb80 tx=0x7f86ac004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea051f3044ad026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0a6c80 con 0x55968ff1a4d0 2024-04-25T13:56:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f86ac0e2d60 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b4004050 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86ac005180 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86cade9740 10 client.29740 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55969001f620 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.563+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ac0a7020 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f86ac0afee0 con 0x55968ff1a4d0 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 10 client.29740 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 10 client.29740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 20 client.29740 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fef9d80 0x5596902f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 4 client.29740 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86b4004050 con 0x55968fef9d80 2024-04-25T13:56:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fef9d80 0x5596902f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86a57fa640 10 client.29740 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.564+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fef9d80 0x5596902f33d0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd2734703d46278 server_cookie=ff869974bd731310 in_seq=0 out_seq=0 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 -- 172.21.15.94:0/1529412093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 msgr2=0x7ff3bc10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/1529412093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc10abb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009ca0 tx=0x7ff3b402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 -- 172.21.15.94:0/1529412093 shutdown_connections 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/1529412093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc10abb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/1529412093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/1529412093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 -- 172.21.15.94:0/1529412093 >> 172.21.15.94:0/1529412093 conn(0x7ff3bc0fd340 msgr2=0x7ff3bc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 -- 172.21.15.94:0/1529412093 shutdown_connections 2024-04-25T13:56:00.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 -- 172.21.15.94:0/1529412093 wait complete. 2024-04-25T13:56:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86a57fa640 1 -- 172.21.15.94:0/1249939211 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f86b4004050 con 0x55968fef9d80 2024-04-25T13:56:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86a57fa640 10 client.29740 handle_command_reply: tid=0 2024-04-25T13:56:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 client.29740 shutdown 2024-04-25T13:56:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 msgr2=0x5596902e5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e5c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0dcb80 tx=0x7f86ac004990 comp rx=0 tx=0).stop 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 shutdown_connections 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fef9d80 0x5596902f33d0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff3b1b0 0x5596902ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e5c20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86a57fa640 0 client.29740 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 --2- 172.21.15.94:0/1249939211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff122f0 0x5596902e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.570+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 >> 172.21.15.94:0/1249939211 conn(0x55968ff2f300 msgr2=0x5596902d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.570+0000 7ff3c3b34640 1 Processor -- start 2024-04-25T13:56:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.571+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 shutdown_connections 2024-04-25T13:56:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.571+0000 7f86cade9740 1 -- 172.21.15.94:0/1249939211 wait complete. 2024-04-25T13:56:00.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 -- start start 2024-04-25T13:56:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.571+0000 7f86cade9740 20 client.29740 trim_cache size 0 max 16384 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc1a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10e530 con 0x7ff3bc103940 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10e3b0 con 0x7ff3bc104460 2024-04-25T13:56:00.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c3b34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10e6b0 con 0x7ff3bc102f60 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39582/0 (socket says 172.21.15.94:39582) 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 -- 172.21.15.94:0/71788575 learned_addr learned my addr 172.21.15.94:0/71788575 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc1a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 -- 172.21.15.94:0/71788575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 msgr2=0x7ff3bc19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 -- 172.21.15.94:0/71788575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 msgr2=0x7ff3bc1a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc1a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c10cf640 1 -- 172.21.15.94:0/71788575 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc1a3f30 con 0x7ff3bc103940 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.571+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e9f0 tx=0x7ff3ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3b2ffd640 1 -- 172.21.15.94:0/71788575 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00cd80 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3b2ffd640 1 -- 172.21.15.94:0/71788575 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3ac0108e0 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc1a4220 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3b2ffd640 1 -- 172.21.15.94:0/71788575 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac018980 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.572+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc1a4770 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.574+0000 7ff3b2ffd640 1 -- 172.21.15.94:0/71788575 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff3ac010430 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.575+0000 7ff3b2ffd640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 0x7ff3a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.575+0000 7ff3b2ffd640 1 -- 172.21.15.94:0/71788575 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff3ac0df070 con 0x7ff3bc103940 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.575+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 0x7ff3a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.575+0000 7ff3ca9e4740 1 Processor -- start 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 -- start start 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 0x7ff3a80b2000 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc069bd0 tx=0x7ff3b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428ee25390 0x56428f1d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 0x56428f1d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56428f12bd80 con 0x56428edfc4d0 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56428f0e5a20 con 0x56428ee25390 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56428f127d40 con 0x56428ee046b0 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 0x56428f1d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39586/0 (socket says 172.21.15.94:39586) 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2890576403 learned_addr learned my addr 172.21.15.94:0/2890576403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428ee25390 0x56428f1d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 -- 172.21.15.94:0/2890576403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428ee25390 msgr2=0x56428f1d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428ee25390 0x56428f1d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 -- 172.21.15.94:0/2890576403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 msgr2=0x56428f1d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c20d1640 1 -- 172.21.15.94:0/2890576403 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428f0f0c30 con 0x56428ee046b0 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.576+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.577+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 0x56428f1d1230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031b50 tx=0x7ff3b402f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12119ee33d66ac73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.577+0000 7ff3a7fff640 1 -- 172.21.15.94:0/2890576403 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4031650 con 0x56428ee046b0 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.577+0000 7ff3a7fff640 1 -- 172.21.15.94:0/2890576403 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b4036030 con 0x56428ee046b0 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.577+0000 7ff3a7fff640 1 -- 172.21.15.94:0/2890576403 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40046d0 con 0x56428ee046b0 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2890576403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 msgr2=0x56428f1d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 0x56428f1d1230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031b50 tx=0x7ff3b402f410 comp rx=0 tx=0).stop 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2890576403 shutdown_connections 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee046b0 0x56428f1d1230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428edfc4d0 0x56428f1d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2890576403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428ee25390 0x56428f1d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2890576403 >> 172.21.15.94:0/2890576403 conn(0x56428ee194e0 msgr2=0x56428f1cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2890576403 shutdown_connections 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2890576403 wait complete. 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 -- start start 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.578+0000 7ff3ca9e4740 1 Processor -- start 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee25390 0x56428f1e1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56428f1d1b60 con 0x56428ee046b0 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56428f0b2dc0 con 0x56428edfc4d0 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56428f1d1ce0 con 0x56428ee25390 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3ca9e4740 10 client.0 _ll_register_callbacks cb 0x56428f0ce600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39590/0 (socket says 172.21.15.94:39590) 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c18d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2638162307 learned_addr learned my addr 172.21.15.94:0/2638162307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee25390 0x56428f1e1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2638162307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee25390 msgr2=0x56428f1e1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee25390 0x56428f1e1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 msgr2=0x56428f1d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2638162307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56428ee3d3f0 con 0x56428ee046b0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5478b1442dd1cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.579+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0c1060 con 0x56428ee046b0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff3ac0b0090 con 0x56428ee046b0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b4036030 con 0x56428ee046b0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac0c8050 con 0x56428ee046b0 2024-04-25T13:56:00.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3ca9e4740 10 client.20589 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56428ef077d0 con 0x56428ee046b0 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.580+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b4036030 con 0x56428ee046b0 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff3ac0c81f0 con 0x56428ee046b0 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 10 client.20589 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 10 client.20589 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 20 client.20589 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ede3f60 0x56428f1e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 4 client.20589 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.581+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56428f139990 con 0x56428ede3f60 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.582+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ede3f60 0x56428f1e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.582+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ede3f60 0x56428f1e78d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b77760f11f43859 server_cookie=ace431febf8e6d99 in_seq=0 out_seq=0 2024-04-25T13:56:00.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.582+0000 7ff3a57fa640 10 client.20589 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3a57fa640 1 -- 172.21.15.94:0/2638162307 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56428f139990 con 0x56428ede3f60 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3a57fa640 10 client.20589 handle_command_reply: tid=0 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 client.20589 shutdown 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 msgr2=0x56428f1da0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000c00 tx=0x7ff3ac010860 comp rx=0 tx=0).stop 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 shutdown_connections 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ede3f60 0x56428f1e78d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3a57fa640 0 client.20589 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee25390 0x56428f1e1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee046b0 0x56428f1da0b0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2638162307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 >> 172.21.15.94:0/2638162307 conn(0x56428ee194e0 msgr2=0x56428f1c2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 shutdown_connections 2024-04-25T13:56:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2638162307 wait complete. 2024-04-25T13:56:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.584+0000 7ff3ca9e4740 20 client.20589 trim_cache size 0 max 16384 2024-04-25T13:56:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 Processor -- start 2024-04-25T13:56:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 -- start start 2024-04-25T13:56:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902d7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ad7a0 con 0x55968ff3b1b0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1af3a0 con 0x55968ff122f0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86cade9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ade70 con 0x55968ff1a4d0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39604/0 (socket says 172.21.15.94:39604) 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c1cd0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902d7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 -- 172.21.15.94:0/1494235020 learned_addr learned my addr 172.21.15.94:0/1494235020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c14cf640 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 -- 172.21.15.94:0/1494235020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 msgr2=0x5596902e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 -- 172.21.15.94:0/1494235020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 msgr2=0x5596902d7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902d7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c24d1640 1 -- 172.21.15.94:0/1494235020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596902e0590 con 0x55968ff3b1b0 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.591+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902d7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.592+0000 7f86c24d1640 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f86b4000f80 tx=0x7f86b4004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdc7c6321639ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.592+0000 7f86a77fe640 1 -- 172.21.15.94:0/1494235020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4028070 con 0x55968ff3b1b0 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.592+0000 7f86a77fe640 1 -- 172.21.15.94:0/1494235020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b4005bd0 con 0x55968ff3b1b0 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.592+0000 7f86a77fe640 1 -- 172.21.15.94:0/1494235020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b40052f0 con 0x55968ff3b1b0 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- 172.21.15.94:0/1494235020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 msgr2=0x5596902e42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f86b4000f80 tx=0x7f86b4004f80 comp rx=0 tx=0).stop 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- 172.21.15.94:0/1494235020 shutdown_connections 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff3b1b0 0x5596902e42a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff1a4d0 0x5596902e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 --2- 172.21.15.94:0/1494235020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902d7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- 172.21.15.94:0/1494235020 >> 172.21.15.94:0/1494235020 conn(0x55968ff2f300 msgr2=0x5596902d6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- 172.21.15.94:0/1494235020 shutdown_connections 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- 172.21.15.94:0/1494235020 wait complete. 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.593+0000 7f86cade9740 1 -- start start 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 Processor -- start 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff3b1b0 0x5596902eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1aecd0 con 0x55968ff1a4d0 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ae540 con 0x55968ff122f0 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ab780 con 0x55968ff3b1b0 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86cade9740 10 client.0 _ll_register_callbacks cb 0x7f86ac00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86c14cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86c14cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39606/0 (socket says 172.21.15.94:39606) 2024-04-25T13:56:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86c14cf640 1 -- 172.21.15.94:0/881635183 learned_addr learned my addr 172.21.15.94:0/881635183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.594+0000 7f86c24d1640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff3b1b0 0x5596902eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 -- 172.21.15.94:0/881635183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff3b1b0 msgr2=0x5596902eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff3b1b0 0x5596902eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 -- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 msgr2=0x5596902e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 -- 172.21.15.94:0/881635183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596902eb240 con 0x55968ff1a4d0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86c14cf640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0a3590 tx=0x7f86ac0a6980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3d5b5f79800067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0a6c80 con 0x55968ff1a4d0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f86ac0a3090 con 0x55968ff1a4d0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b4005bd0 con 0x55968ff1a4d0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86ac0b03c0 con 0x55968ff1a4d0 2024-04-25T13:56:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86cade9740 10 client.20595 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5596902ec540 con 0x55968ff1a4d0 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b4005bd0 con 0x55968ff1a4d0 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.595+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f86ac0c8200 con 0x55968ff1a4d0 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 10 client.20595 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 10 client.20595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 20 client.20595 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fecf1c0 0x55969035deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 4 client.20595 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55969024f070 con 0x55968fecf1c0 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fecf1c0 0x55969035deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86a57fa640 10 client.20595 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:00.596+0000 7f86c1cd0640 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fecf1c0 0x55969035deb0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1b653b78bb85e4 server_cookie=afdb856c2abd96cb in_seq=0 out_seq=0 2024-04-25T13:56:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 Processor -- start 2024-04-25T13:56:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 -- start start 2024-04-25T13:56:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee194e0 0x56428f1d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1a92e0 con 0x56428ede3f60 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1aaee0 con 0x56428edfc4d0 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1a99b0 con 0x56428ee194e0 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39614/0 (socket says 172.21.15.94:39614) 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2369113588 learned_addr learned my addr 172.21.15.94:0/2369113588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.600+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee194e0 0x56428f1d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2369113588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee194e0 msgr2=0x56428f1d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee194e0 0x56428f1d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2369113588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 msgr2=0x56428f1d8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 -- 172.21.15.94:0/2369113588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428f1cbd90 con 0x56428ede3f60 2024-04-25T13:56:00.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0a34e0 tx=0x7ff3ac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e10acefd975ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3a77fe640 1 -- 172.21.15.94:0/2369113588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0a7280 con 0x56428ede3f60 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3a77fe640 1 -- 172.21.15.94:0/2369113588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3ac0ab860 con 0x56428ede3f60 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.601+0000 7ff3a77fe640 1 -- 172.21.15.94:0/2369113588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0a6180 con 0x56428ede3f60 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2369113588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 msgr2=0x56428f1d4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0a34e0 tx=0x7ff3ac009310 comp rx=0 tx=0).stop 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2369113588 shutdown_connections 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1d8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ede3f60 0x56428f1d4f90 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/2369113588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ee194e0 0x56428f1d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2369113588 >> 172.21.15.94:0/2369113588 conn(0x56428f1e7bc0 msgr2=0x56428f1d9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2369113588 shutdown_connections 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.602+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/2369113588 wait complete. 2024-04-25T13:56:00.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 -- start start 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 Processor -- start 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 0x56428f1dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1aa810 con 0x56428ee194e0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1aa080 con 0x56428edfc4d0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1a7340 con 0x56428ede3f60 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3ca9e4740 10 client.0 _ll_register_callbacks cb 0x7ff3ac0a6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c20d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c18d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 0x56428f1dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c20d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39628/0 (socket says 172.21.15.94:39628) 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c18d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 0x56428f1dd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58320/0 (socket says 172.21.15.94:58320) 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c10cf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.603+0000 7ff3c18d0640 1 -- 172.21.15.94:0/1906291289 learned_addr learned my addr 172.21.15.94:0/1906291289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 -- 172.21.15.94:0/1906291289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 msgr2=0x56428f1dd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 0x56428f1dd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 -- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 msgr2=0x56428f1ddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1ddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 -- 172.21.15.94:0/1906291289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56428f24e1a0 con 0x56428ee194e0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c10cf640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3c20d1640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009620 tx=0x7ff3b4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e9d13679a60a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4054030 con 0x56428ee194e0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3ac0ab860 con 0x56428ee194e0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7ff3b4003c10 con 0x56428ee194e0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3b404a040 con 0x56428ee194e0 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3ca9e4740 10 client.20601 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.604+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56428f24f470 con 0x56428ee194e0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3ac0ab860 con 0x56428ee194e0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7ff3b4043070 con 0x56428ee194e0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 10 client.20601 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 10 client.20601 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 20 client.20601 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ee046b0 0x56428f251b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 4 client.20601 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56428f133800 con 0x56428ee046b0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ee046b0 0x56428f251b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3a57fa640 10 client.20601 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.605+0000 7ff3c18d0640 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ee046b0 0x56428f251b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc5974957887a63a server_cookie=1174cf57d1de4aff in_seq=0 out_seq=0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3a57fa640 1 -- 172.21.15.94:0/1906291289 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56428f133800 con 0x56428ee046b0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3a57fa640 10 client.20601 handle_command_reply: tid=0 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 client.20601 shutdown 2024-04-25T13:56:00.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 msgr2=0x56428f24dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009620 tx=0x7ff3b4003ba0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 shutdown_connections 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56428ee046b0 0x56428f251b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56428ee194e0 0x56428f24dc30 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56428edfc4d0 0x56428f1ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3a57fa640 0 client.20601 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 --2- 172.21.15.94:0/1906291289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56428ede3f60 0x56428f1dd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 >> 172.21.15.94:0/1906291289 conn(0x56428f1e7bc0 msgr2=0x56428f1d9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 shutdown_connections 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 1 -- 172.21.15.94:0/1906291289 wait complete. 2024-04-25T13:56:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.606+0000 7ff3ca9e4740 20 client.20601 trim_cache size 0 max 16384 2024-04-25T13:56:00.615 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 msgr2=0x7ff3a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 0x7ff3a80b2000 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc069bd0 tx=0x7ff3b8005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 msgr2=0x7ff3bc19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00e9f0 tx=0x7ff3ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 shutdown_connections 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3a80afaf0 0x7ff3a80b2000 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3bc104460 0x7ff3bc1a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3bc103940 0x7ff3bc19c930 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 --2- 172.21.15.94:0/71788575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3bc102f60 0x7ff3bc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 >> 172.21.15.94:0/71788575 conn(0x7ff3bc0fd340 msgr2=0x7ff3bc0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 shutdown_connections 2024-04-25T13:56:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.607+0000 7ff3c3b34640 1 -- 172.21.15.94:0/71788575 wait complete. 2024-04-25T13:56:00.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:56:00.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.772+0000 7f53cb59e640 1 -- 172.21.15.94:0/2680314575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc10b180 msgr2=0x7f53cc10b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.772+0000 7f53cb59e640 1 --2- 172.21.15.94:0/2680314575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc10b180 0x7f53cc10b590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009900 tx=0x7f53bc02a440 comp rx=0 tx=0).stop 2024-04-25T13:56:00.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.782+0000 7f53cb59e640 1 -- 172.21.15.94:0/2680314575 shutdown_connections 2024-04-25T13:56:00.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.782+0000 7f53cb59e640 1 --2- 172.21.15.94:0/2680314575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc10fe80 0x7f53cc1122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.782+0000 7f53cb59e640 1 --2- 172.21.15.94:0/2680314575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc10b180 0x7f53cc10b590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.782+0000 7f53cb59e640 1 --2- 172.21.15.94:0/2680314575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc109f40 0x7f53cc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.782+0000 7f53cb59e640 1 -- 172.21.15.94:0/2680314575 >> 172.21.15.94:0/2680314575 conn(0x7f53cc078b50 msgr2=0x7f53cc1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.785+0000 7f53cb59e640 1 -- 172.21.15.94:0/2680314575 shutdown_connections 2024-04-25T13:56:00.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.788+0000 7f53cb59e640 1 -- 172.21.15.94:0/2680314575 wait complete. 2024-04-25T13:56:00.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.789+0000 7f53cb59e640 1 Processor -- start 2024-04-25T13:56:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 -- start start 2024-04-25T13:56:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc109f40 0x7f53cc074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc10fe80 0x7f53cc081a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc06e340 con 0x7f53cc10b180 2024-04-25T13:56:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc06e1c0 con 0x7f53cc10fe80 2024-04-25T13:56:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.793+0000 7f53cb59e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc06e4c0 con 0x7f53cc109f40 2024-04-25T13:56:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.794+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.794+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39646/0 (socket says 172.21.15.94:39646) 2024-04-25T13:56:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.794+0000 7f53c9d9b640 1 -- 172.21.15.94:0/3110509903 learned_addr learned my addr 172.21.15.94:0/3110509903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.795+0000 7f53c9d9b640 1 -- 172.21.15.94:0/3110509903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc109f40 msgr2=0x7f53cc074840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.795+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc109f40 0x7f53cc074840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.795+0000 7f53c9d9b640 1 -- 172.21.15.94:0/3110509903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc10fe80 msgr2=0x7f53cc081a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.795+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc10fe80 0x7f53cc081a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.795+0000 7f53c9d9b640 1 -- 172.21.15.94:0/3110509903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc082290 con 0x7f53cc10b180 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.796+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f53bc02aa30 tx=0x7f53bc030bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.796+0000 7f53bb7fe640 1 -- 172.21.15.94:0/3110509903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc040070 con 0x7f53cc10b180 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.796+0000 7f53bb7fe640 1 -- 172.21.15.94:0/3110509903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc004220 con 0x7f53cc10b180 2024-04-25T13:56:00.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.796+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc082520 con 0x7f53cc10b180 2024-04-25T13:56:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.796+0000 7f53bb7fe640 1 -- 172.21.15.94:0/3110509903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc035030 con 0x7f53cc10b180 2024-04-25T13:56:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.797+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc082ae0 con 0x7f53cc10b180 2024-04-25T13:56:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.800+0000 7f53d7e82740 1 Processor -- start 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 -- start start 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fba09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fba0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a67fb04d80 con 0x55a67f7dd6b0 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a67fabea20 con 0x55a67f7fe390 2024-04-25T13:56:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.802+0000 7f53d7e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fb00d40 con 0x55a67f7d54d0 2024-04-25T13:56:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.806+0000 7f53cad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fba09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.807+0000 7f53bb7fe640 1 -- 172.21.15.94:0/3110509903 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f53bc0035f0 con 0x7f53cc10b180 2024-04-25T13:56:00.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.808+0000 7f53bb7fe640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 0x7f53b40b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.808+0000 7f53bb7fe640 1 -- 172.21.15.94:0/3110509903 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f53bc0ffdd0 con 0x7f53cc10b180 2024-04-25T13:56:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.809+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.809+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39658/0 (socket says 172.21.15.94:39658) 2024-04-25T13:56:00.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.809+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1449258386 learned_addr learned my addr 172.21.15.94:0/1449258386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.810+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1449258386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 msgr2=0x55a67fba0f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.810+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fba0f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.810+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1449258386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 msgr2=0x55a67fba09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.810+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fba09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.810+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1449258386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67fac9c30 con 0x55a67f7dd6b0 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.811+0000 7f53cad9d640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 0x7f53b40b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.811+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0c8140 tx=0x7f53bc003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07f9645cecb0b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.813+0000 7f53b8ff9640 1 -- 172.21.15.94:0/1449258386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0c81b0 con 0x55a67f7dd6b0 2024-04-25T13:56:00.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.814+0000 7f53b8ff9640 1 -- 172.21.15.94:0/1449258386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc0cc560 con 0x55a67f7dd6b0 2024-04-25T13:56:00.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.815+0000 7f53d7e82740 1 -- 172.21.15.94:0/1449258386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 msgr2=0x55a67fba1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.815+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0c8140 tx=0x7f53bc003dd0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.818+0000 7f53cad9d640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 0x7f53b40b2130 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009fd0 tx=0x7f53c4004af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.821+0000 7f53d7e82740 1 -- 172.21.15.94:0/1449258386 shutdown_connections 2024-04-25T13:56:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.821+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fba1bf0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.821+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fba0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.821+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1449258386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fba09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.821+0000 7f53d7e82740 1 -- 172.21.15.94:0/1449258386 >> 172.21.15.94:0/1449258386 conn(0x55a67f7f24e0 msgr2=0x55a67fb9c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.830+0000 7f53d7e82740 1 -- 172.21.15.94:0/1449258386 shutdown_connections 2024-04-25T13:56:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.832+0000 7f53d7e82740 1 -- 172.21.15.94:0/1449258386 wait complete. 2024-04-25T13:56:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.832+0000 7f53d7e82740 1 -- start start 2024-04-25T13:56:00.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.832+0000 7f53d7e82740 1 Processor -- start 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7fe390 0x55a67fbb1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a67fba24c0 con 0x55a67f7fe390 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a67fa8bdc0 con 0x55a67f7dd6b0 2024-04-25T13:56:00.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a67fba2640 con 0x55a67f7d54d0 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53d7e82740 10 client.0 _ll_register_callbacks cb 0x55a67faa7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58370/0 (socket says 172.21.15.94:58370) 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.835+0000 7f53ca59c640 1 -- 172.21.15.94:0/399803881 learned_addr learned my addr 172.21.15.94:0/399803881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.836+0000 7f53ca59c640 1 -- 172.21.15.94:0/399803881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7dd6b0 msgr2=0x55a67fbaa9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.836+0000 7f53ca59c640 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaa9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.836+0000 7f53ca59c640 1 -- 172.21.15.94:0/399803881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7fe390 msgr2=0x55a67fbb1920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:56:00.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.836+0000 7f53ca59c640 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7fe390 0x55a67fbb1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.836+0000 7f53ca59c640 1 -- 172.21.15.94:0/399803881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67f8163f0 con 0x55a67f7d54d0 2024-04-25T13:56:00.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.841+0000 7f53ca59c640 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009d80 tx=0x7f53c000b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9b6ba43702bb7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.841+0000 7f53b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:00.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.845+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0011420 con 0x55a67f7d54d0 2024-04-25T13:56:00.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.845+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc0cc560 con 0x55a67f7d54d0 2024-04-25T13:56:00.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.845+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f53c00115c0 con 0x55a67f7d54d0 2024-04-25T13:56:00.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.845+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53c00117c0 con 0x55a67f7d54d0 2024-04-25T13:56:00.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.846+0000 7f53d7e82740 10 client.29770 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.846+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a67f8e07d0 con 0x55a67f7d54d0 2024-04-25T13:56:00.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.847+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc0cc560 con 0x55a67f7d54d0 2024-04-25T13:56:00.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.847+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f53c001bc10 con 0x55a67f7d54d0 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 10 client.29770 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 10 client.29770 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 20 client.29770 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67f7bcf60 0x55a67fbb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 4 client.29770 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:00.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.848+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67fb12990 con 0x55a67f7bcf60 2024-04-25T13:56:00.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.849+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67f7bcf60 0x55a67fbb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.855+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67f7bcf60 0x55a67fbb8390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad26be54e09abf2 server_cookie=a0d5b804daf430f7 in_seq=0 out_seq=0 2024-04-25T13:56:00.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.857+0000 7f53b1ffb640 10 client.29770 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:00.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.858+0000 7f53b1ffb640 1 -- 172.21.15.94:0/399803881 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a67fb12990 con 0x55a67f7bcf60 2024-04-25T13:56:00.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.858+0000 7f53b1ffb640 10 client.29770 handle_command_reply: tid=0 2024-04-25T13:56:00.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.859+0000 7f53d7e82740 1 client.29770 shutdown 2024-04-25T13:56:00.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.864+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 msgr2=0x55a67fbaa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.864+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009d80 tx=0x7f53c000b4b0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 shutdown_connections 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67f7bcf60 0x55a67fbb8390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7fe390 0x55a67fbb1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 --2- 172.21.15.94:0/399803881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7d54d0 0x55a67fbaa480 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009d80 tx=0x7f53c000b4b0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 >> 172.21.15.94:0/399803881 conn(0x55a67f7f24e0 msgr2=0x55a67fb9b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.865+0000 7f53b1ffb640 0 client.29770 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:00.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.867+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 shutdown_connections 2024-04-25T13:56:00.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.871+0000 7f53d7e82740 1 -- 172.21.15.94:0/399803881 wait complete. 2024-04-25T13:56:00.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.875+0000 7f53d7e82740 20 client.29770 trim_cache size 0 max 16384 2024-04-25T13:56:00.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.912+0000 7f53d7e82740 1 Processor -- start 2024-04-25T13:56:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 -- start start 2024-04-25T13:56:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 0x55a67fb9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7923a0 0x55a67fb9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac00e890 con 0x55a67f7923a0 2024-04-25T13:56:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac010490 con 0x55a67f7fe390 2024-04-25T13:56:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.913+0000 7f53d7e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac00ef60 con 0x55a67f7dd6b0 2024-04-25T13:56:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 0x55a67fb9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34056/0 (socket says 172.21.15.94:34056) 2024-04-25T13:56:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 0x55a67fb9a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58372/0 (socket says 172.21.15.94:58372) 2024-04-25T13:56:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53ca59c640 1 -- 172.21.15.94:0/1159218366 learned_addr learned my addr 172.21.15.94:0/1159218366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.915+0000 7f53cad9d640 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7923a0 0x55a67fb9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.916+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1159218366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 msgr2=0x55a67fb9a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.916+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 0x55a67fb9a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.916+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1159218366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7923a0 msgr2=0x55a67fb9af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.916+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7923a0 0x55a67fb9af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.916+0000 7f53c9d9b640 1 -- 172.21.15.94:0/1159218366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67fba5d90 con 0x55a67f7fe390 2024-04-25T13:56:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.917+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a67fbab4f0 tx=0x7f53bc0c41a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80556420caefef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.921+0000 7f53b3fff640 1 -- 172.21.15.94:0/1159218366 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0c81b0 con 0x55a67f7fe390 2024-04-25T13:56:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.921+0000 7f53b3fff640 1 -- 172.21.15.94:0/1159218366 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc0d1010 con 0x55a67f7fe390 2024-04-25T13:56:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.921+0000 7f53b3fff640 1 -- 172.21.15.94:0/1159218366 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0038e0 con 0x55a67f7fe390 2024-04-25T13:56:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.922+0000 7f53d7e82740 1 -- 172.21.15.94:0/1159218366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 msgr2=0x55a67fbb18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.922+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a67fbab4f0 tx=0x7f53bc0c41a0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.923+0000 7f53d7e82740 1 -- 172.21.15.94:0/1159218366 shutdown_connections 2024-04-25T13:56:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.923+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7923a0 0x55a67fb9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.923+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7fe390 0x55a67fbb18b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.923+0000 7f53d7e82740 1 --2- 172.21.15.94:0/1159218366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7dd6b0 0x55a67fb9a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.923+0000 7f53d7e82740 1 -- 172.21.15.94:0/1159218366 >> 172.21.15.94:0/1159218366 conn(0x55a67f7f24e0 msgr2=0x55a67fba8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.930+0000 7f53d7e82740 1 -- 172.21.15.94:0/1159218366 shutdown_connections 2024-04-25T13:56:00.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.931+0000 7f53d7e82740 1 -- 172.21.15.94:0/1159218366 wait complete. 2024-04-25T13:56:00.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 -- start start 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 Processor -- start 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7923a0 0x55a67fbaea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7fe390 0x55a67fbafee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac00fdc0 con 0x55a67f7dd6b0 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac00f630 con 0x55a67f7923a0 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac00c870 con 0x55a67f7fe390 2024-04-25T13:56:00.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.932+0000 7f53d7e82740 10 client.0 _ll_register_callbacks cb 0x7f53c0006e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39664/0 (socket says 172.21.15.94:39664) 2024-04-25T13:56:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 -- 172.21.15.94:0/2963047973 learned_addr learned my addr 172.21.15.94:0/2963047973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 -- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7fe390 msgr2=0x55a67fbafee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7fe390 0x55a67fbafee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 -- 172.21.15.94:0/2963047973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7923a0 msgr2=0x55a67fbaea70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7923a0 0x55a67fbaea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.933+0000 7f53c9d9b640 1 -- 172.21.15.94:0/2963047973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67fbb04a0 con 0x55a67f7dd6b0 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.934+0000 7f53c9d9b640 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0c7940 tx=0x7f53bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf09dc57e4b1185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.934+0000 7f53b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.935+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0e7060 con 0x55a67f7dd6b0 2024-04-25T13:56:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.935+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc0d1010 con 0x55a67f7dd6b0 2024-04-25T13:56:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.935+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f53bc0c47a0 con 0x55a67f7dd6b0 2024-04-25T13:56:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.936+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53bc0cc310 con 0x55a67f7dd6b0 2024-04-25T13:56:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.937+0000 7f53d7e82740 10 client.20613 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.937+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a67fbb1720 con 0x55a67f7dd6b0 2024-04-25T13:56:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc0d1010 con 0x55a67f7dd6b0 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f53bc03a040 con 0x55a67f7dd6b0 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 10 client.20613 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 10 client.20613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 20 client.20613 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67fc20820 0x55a67fc22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 4 client.20613 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.938+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a67fba1360 con 0x55a67fc20820 2024-04-25T13:56:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.939+0000 7f53ca59c640 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67fc20820 0x55a67fc22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:00.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.951+0000 7f53ca59c640 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67fc20820 0x55a67fc22c40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49c0e4d906de83fa server_cookie=246949f7e749b0b0 in_seq=0 out_seq=0 2024-04-25T13:56:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.951+0000 7f53b1ffb640 10 client.20613 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.951+0000 7f53b1ffb640 1 -- 172.21.15.94:0/2963047973 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a67fba1360 con 0x55a67fc20820 2024-04-25T13:56:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.951+0000 7f53b1ffb640 10 client.20613 handle_command_reply: tid=0 2024-04-25T13:56:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.951+0000 7f53d7e82740 1 client.20613 shutdown 2024-04-25T13:56:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.955+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 msgr2=0x55a67fbaefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.955+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0c7940 tx=0x7f53bc009260 comp rx=0 tx=0).stop 2024-04-25T13:56:00.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 shutdown_connections 2024-04-25T13:56:00.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a67fc20820 0x55a67fc22c40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a67f7fe390 0x55a67fbafee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a67f7dd6b0 0x55a67fbaefb0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 --2- 172.21.15.94:0/2963047973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a67f7923a0 0x55a67fbaea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 >> 172.21.15.94:0/2963047973 conn(0x55a67f7f24e0 msgr2=0x55a67fb9eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.958+0000 7f53b1ffb640 0 client.20613 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.959+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 shutdown_connections 2024-04-25T13:56:00.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.962+0000 7f53d7e82740 1 -- 172.21.15.94:0/2963047973 wait complete. 2024-04-25T13:56:00.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.964+0000 7f53d7e82740 20 client.20613 trim_cache size 0 max 16384 2024-04-25T13:56:00.962 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:00.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.965+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 msgr2=0x7f53b40b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.965+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 0x7f53b40b2130 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009fd0 tx=0x7f53c4004af0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.965+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 msgr2=0x7f53cc07f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.965+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f53bc02aa30 tx=0x7f53bc030bc0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 shutdown_connections 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53b40afc20 0x7f53b40b2130 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53cc10fe80 0x7f53cc081a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53cc10b180 0x7f53cc07f410 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 --2- 172.21.15.94:0/3110509903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc109f40 0x7f53cc074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.966+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 >> 172.21.15.94:0/3110509903 conn(0x7f53cc078b50 msgr2=0x7f53cc10be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.967+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 shutdown_connections 2024-04-25T13:56:00.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:00.967+0000 7f53cb59e640 1 -- 172.21.15.94:0/3110509903 wait complete. 2024-04-25T13:56:00.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:56:01.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- 172.21.15.94:0/62332353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 msgr2=0x7f37341036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 --2- 172.21.15.94:0/62332353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f37341036e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f3730009900 tx=0x7f373002a410 comp rx=0 tx=0).stop 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- 172.21.15.94:0/62332353 shutdown_connections 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 --2- 172.21.15.94:0/62332353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734108920 0x7f373410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 --2- 172.21.15.94:0/62332353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734103c20 0x7f3734104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 --2- 172.21.15.94:0/62332353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f37341036e0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- 172.21.15.94:0/62332353 >> 172.21.15.94:0/62332353 conn(0x7f37340fdbd0 msgr2=0x7f3734100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- 172.21.15.94:0/62332353 shutdown_connections 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- 172.21.15.94:0/62332353 wait complete. 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 Processor -- start 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.107+0000 7f373c786640 1 -- start start 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734103c20 0x7f373419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734108920 0x7f37341a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e4e0 con 0x7f37341029e0 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e360 con 0x7f3734108920 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373c786640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f373410e660 con 0x7f3734103c20 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39692/0 (socket says 172.21.15.94:39692) 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 -- 172.21.15.94:0/4114866927 learned_addr learned my addr 172.21.15.94:0/4114866927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373ad23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734108920 0x7f37341a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f3739d21640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734103c20 0x7f373419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 -- 172.21.15.94:0/4114866927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734103c20 msgr2=0x7f373419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734103c20 0x7f373419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 -- 172.21.15.94:0/4114866927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734108920 msgr2=0x7f37341a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734108920 0x7f37341a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 -- 172.21.15.94:0/4114866927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3734199160 con 0x7f37341029e0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f373a522640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f3730006010 tx=0x7f3730004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.108+0000 7f372b7fe640 1 -- 172.21.15.94:0/4114866927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373003f030 con 0x7f37341029e0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.109+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341993f0 con 0x7f37341029e0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.109+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3734199930 con 0x7f37341029e0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.109+0000 7f372b7fe640 1 -- 172.21.15.94:0/4114866927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3730031910 con 0x7f37341029e0 2024-04-25T13:56:01.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.109+0000 7f372b7fe640 1 -- 172.21.15.94:0/4114866927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730035070 con 0x7f37341029e0 2024-04-25T13:56:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f374363e740 1 Processor -- start 2024-04-25T13:56:01.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f372b7fe640 1 -- 172.21.15.94:0/4114866927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3730042030 con 0x7f37341029e0 2024-04-25T13:56:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 -- start start 2024-04-25T13:56:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556427836390 0x556427bd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427bd8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 0x556427bd9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556427b3cd80 con 0x55642780d4d0 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556427af6a20 con 0x5564278156b0 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f374363e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556427b38d40 con 0x556427836390 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.113+0000 7f372b7fe640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f3739d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 0x556427bd9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f372b7fe640 1 -- 172.21.15.94:0/4114866927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37300feed0 con 0x7f37341029e0 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f3739d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 0x556427bd9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34082/0 (socket says 172.21.15.94:34082) 2024-04-25T13:56:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f3739d21640 1 -- 172.21.15.94:0/1366638308 learned_addr learned my addr 172.21.15.94:0/1366638308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f373a522640 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427bd8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.114+0000 7f373ad23640 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556427836390 0x556427bd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f86a57fa640 1 -- 172.21.15.94:0/881635183 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55969024f070 con 0x55968fecf1c0 2024-04-25T13:56:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f86a57fa640 10 client.20595 handle_command_reply: tid=0 2024-04-25T13:56:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f86cade9740 1 client.20595 shutdown 2024-04-25T13:56:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 msgr2=0x5596902e9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.112+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0a3590 tx=0x7f86ac0a6980 comp rx=0 tx=0).stop 2024-04-25T13:56:01.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373ad23640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 -- 172.21.15.94:0/1366638308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556427836390 msgr2=0x556427bd8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556427836390 0x556427bd8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 -- 172.21.15.94:0/1366638308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 msgr2=0x556427bd9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 0x556427bd9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 -- 172.21.15.94:0/1366638308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556427b01c30 con 0x55642780d4d0 2024-04-25T13:56:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f373a522640 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427bd8d90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f37300c6b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5695945492257d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f3728ff9640 1 -- 172.21.15.94:0/1366638308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730049070 con 0x55642780d4d0 2024-04-25T13:56:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f3728ff9640 1 -- 172.21.15.94:0/1366638308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37300d0110 con 0x55642780d4d0 2024-04-25T13:56:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.115+0000 7f3728ff9640 1 -- 172.21.15.94:0/1366638308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37300cfee0 con 0x55642780d4d0 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 shutdown_connections 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55968fecf1c0 0x55969035deb0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55968ff3b1b0 0x5596902eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55968ff1a4d0 0x5596902e9d50 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 --2- 172.21.15.94:0/881635183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55968ff122f0 0x5596902e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 >> 172.21.15.94:0/881635183 conn(0x55968ff2f300 msgr2=0x5596902e3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86a57fa640 0 client.20595 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 shutdown_connections 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 1 -- 172.21.15.94:0/881635183 wait complete. 2024-04-25T13:56:01.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.116+0000 7f86cade9740 20 client.20595 trim_cache size 0 max 16384 2024-04-25T13:56:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:56:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:56:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:56:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:56:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 msgr2=0x7f86a80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 0x7f86a80b1ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005ab0 tx=0x7f86b8031400 comp rx=0 tx=0).stop 2024-04-25T13:56:01.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 -- 172.21.15.94:0/1366638308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 msgr2=0x556427bd8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427bd8d90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f37300c6b90 comp rx=0 tx=0).stop 2024-04-25T13:56:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 msgr2=0x7f86bc102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00edf0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f373ad23640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 0x7f37200b2080 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x556427bd9860 tx=0x7f372c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 shutdown_connections 2024-04-25T13:56:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86a80afae0 0x7f86a80b1ff0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86bc1050e0 0x7f86bc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc102290 0x7f86bc102930 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 --2- 172.21.15.94:0/1581903187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86bc101790 0x7f86bc104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 >> 172.21.15.94:0/1581903187 conn(0x7f86bc0fd390 msgr2=0x7f86bc105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 shutdown_connections 2024-04-25T13:56:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f86c3f34640 1 -- 172.21.15.94:0/1581903187 wait complete. 2024-04-25T13:56:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 -- 172.21.15.94:0/1366638308 shutdown_connections 2024-04-25T13:56:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564278156b0 0x556427bd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427bd8d90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 --2- 172.21.15.94:0/1366638308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556427836390 0x556427bd8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 -- 172.21.15.94:0/1366638308 >> 172.21.15.94:0/1366638308 conn(0x55642782a4e0 msgr2=0x556427bd4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 -- 172.21.15.94:0/1366638308 shutdown_connections 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.117+0000 7f374363e740 1 -- 172.21.15.94:0/1366638308 wait complete. 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 -- start start 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 Processor -- start 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556427836390 0x556427be9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556427bda2f0 con 0x55642780d4d0 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556427ac3dc0 con 0x556427836390 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556427bda470 con 0x5564278156b0 2024-04-25T13:56:01.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f374363e740 10 client.0 _ll_register_callbacks cb 0x556427adf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39714/0 (socket says 172.21.15.94:39714) 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f373a522640 1 -- 172.21.15.94:0/3187642771 learned_addr learned my addr 172.21.15.94:0/3187642771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f3739d21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.118+0000 7f373ad23640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556427836390 0x556427be9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 -- 172.21.15.94:0/3187642771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 msgr2=0x556427be27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 -- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556427836390 msgr2=0x556427be9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556427836390 0x556427be9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 -- 172.21.15.94:0/3187642771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642784e3f0 con 0x55642780d4d0 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f3739d21640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:01.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f373a522640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f373003d040 tx=0x7f37300314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c829e801cdd932bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37300e0020 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37300cfe90 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37300d0110 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3730049070 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f374363e740 10 client.20625 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564279187d0 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.119+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37300d0110 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f37300c1af0 con 0x55642780d4d0 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 10 client.20625 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 10 client.20625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 20 client.20625 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5564277f4f60 0x556427befdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 4 client.20625 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556427b4a990 con 0x5564277f4f60 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.120+0000 7f3739d21640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5564277f4f60 0x556427befdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.121+0000 7f3739d21640 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5564277f4f60 0x556427befdb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de990f372b0b415d server_cookie=3957c260fbd69128 in_seq=0 out_seq=0 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.121+0000 7f371dffb640 10 client.20625 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f371dffb640 1 -- 172.21.15.94:0/3187642771 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556427b4a990 con 0x5564277f4f60 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f371dffb640 10 client.20625 handle_command_reply: tid=0 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 client.20625 shutdown 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 msgr2=0x556427be2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f373003d040 tx=0x7f37300314c0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 shutdown_connections 2024-04-25T13:56:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5564277f4f60 0x556427befdb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556427836390 0x556427be9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f371dffb640 0 client.20625 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 --2- 172.21.15.94:0/3187642771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55642780d4d0 0x556427be2290 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 >> 172.21.15.94:0/3187642771 conn(0x55642782a4e0 msgr2=0x556427bd6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 shutdown_connections 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 1 -- 172.21.15.94:0/3187642771 wait complete. 2024-04-25T13:56:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.122+0000 7f374363e740 20 client.20625 trim_cache size 0 max 16384 2024-04-25T13:56:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 Processor -- start 2024-04-25T13:56:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 -- start start 2024-04-25T13:56:01.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f371800e730 con 0x5564278156b0 2024-04-25T13:56:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3718010330 con 0x55642780d4d0 2024-04-25T13:56:01.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.136+0000 7f374363e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f371800ee00 con 0x5564277f4f60 2024-04-25T13:56:01.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 16271 24.32 MB/sec warmup 43 sec latency 484.036 ms 2024-04-25T13:56:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373a522640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373ad23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34100/0 (socket says 172.21.15.94:34100) 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373ad23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39720/0 (socket says 172.21.15.94:39720) 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 -- 172.21.15.94:0/3725639887 learned_addr learned my addr 172.21.15.94:0/3725639887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373a522640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58416/0 (socket says 172.21.15.94:58416) 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 -- 172.21.15.94:0/3725639887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 msgr2=0x556427be0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 -- 172.21.15.94:0/3725639887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 msgr2=0x556427be23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 -- 172.21.15.94:0/3725639887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556427bddba0 con 0x55642780d4d0 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373a522640 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f373ad23640 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f3739d21640 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3724009530 tx=0x7f3724005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8b5d337cfa83561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f371ffff640 1 -- 172.21.15.94:0/3725639887 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724005220 con 0x55642780d4d0 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f371ffff640 1 -- 172.21.15.94:0/3725639887 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3724004c50 con 0x55642780d4d0 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.137+0000 7f371ffff640 1 -- 172.21.15.94:0/3725639887 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724010a20 con 0x55642780d4d0 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 -- 172.21.15.94:0/3725639887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 msgr2=0x556427be1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3724009530 tx=0x7f3724005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 -- 172.21.15.94:0/3725639887 shutdown_connections 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564278156b0 0x556427be23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be1e60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 --2- 172.21.15.94:0/3725639887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564277f4f60 0x556427be0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 -- 172.21.15.94:0/3725639887 >> 172.21.15.94:0/3725639887 conn(0x55642782a4e0 msgr2=0x556427be08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 -- 172.21.15.94:0/3725639887 shutdown_connections 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.138+0000 7f374363e740 1 -- 172.21.15.94:0/3725639887 wait complete. 2024-04-25T13:56:01.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 -- start start 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 Processor -- start 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f371800fc60 con 0x5564277f4f60 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f371800f4d0 con 0x55642780d4d0 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f371800c710 con 0x5564278156b0 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f374363e740 10 client.0 _ll_register_callbacks cb 0x7f37240097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f373a522640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39724/0 (socket says 172.21.15.94:39724) 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.139+0000 7f373a522640 1 -- 172.21.15.94:0/1954349563 learned_addr learned my addr 172.21.15.94:0/1954349563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373ad23640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f3739d21640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 -- 172.21.15.94:0/1954349563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 msgr2=0x556427be82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 -- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 msgr2=0x556427be7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 -- 172.21.15.94:0/1954349563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556427c562d0 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f373a522640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f37300c38c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b4dc8cdb2539f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f371dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37300e0040 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f37300cb170 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3724004c50 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f37300c6cb0 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f374363e740 10 client.20631 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556427c562d0 con 0x5564277f4f60 2024-04-25T13:56:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.140+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3724004c50 con 0x5564277f4f60 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3730031ac0 con 0x5564277f4f60 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 10 client.20631 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 10 client.20631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 20 client.20631 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556427836390 0x556427c59b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 4 client.20631 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556427b44800 con 0x556427836390 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f3739d21640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556427836390 0x556427c59b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f371dffb640 10 client.20631 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.141+0000 7f3739d21640 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556427836390 0x556427c59b30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36513623422e8bc7 server_cookie=315fb7c140b3d861 in_seq=0 out_seq=0 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f371dffb640 1 -- 172.21.15.94:0/1954349563 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556427b44800 con 0x556427836390 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f371dffb640 10 client.20631 handle_command_reply: tid=0 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 client.20631 shutdown 2024-04-25T13:56:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 msgr2=0x556427be7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f37300c38c0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 shutdown_connections 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556427836390 0x556427c59b30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564278156b0 0x556427be82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642780d4d0 0x556427be7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f371dffb640 0 client.20631 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 --2- 172.21.15.94:0/1954349563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564277f4f60 0x556427be7850 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 >> 172.21.15.94:0/1954349563 conn(0x55642782a4e0 msgr2=0x556427bd6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 shutdown_connections 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 1 -- 172.21.15.94:0/1954349563 wait complete. 2024-04-25T13:56:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f374363e740 20 client.20631 trim_cache size 0 max 16384 2024-04-25T13:56:01.148 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 msgr2=0x7f37200b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 0x7f37200b2080 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x556427bd9860 tx=0x7f372c004ab0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 msgr2=0x7f373406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f3730006010 tx=0x7f3730004270 comp rx=0 tx=0).stop 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 shutdown_connections 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3734108920 0x7f37341a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3734103c20 0x7f373419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 --2- 172.21.15.94:0/4114866927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37341029e0 0x7f373406a960 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.142+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 >> 172.21.15.94:0/4114866927 conn(0x7f37340fdbd0 msgr2=0x7f3734104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.143+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 shutdown_connections 2024-04-25T13:56:01.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.143+0000 7f373c786640 1 -- 172.21.15.94:0/4114866927 wait complete. 2024-04-25T13:56:01.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:56:01.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 -- 172.21.15.94:0/1313767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 msgr2=0x7f3d1c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 --2- 172.21.15.94:0/1313767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c10ad50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009ca0 tx=0x7f3d1402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 -- 172.21.15.94:0/1313767003 shutdown_connections 2024-04-25T13:56:01.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 --2- 172.21.15.94:0/1313767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c10ad50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 --2- 172.21.15.94:0/1313767003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 --2- 172.21.15.94:0/1313767003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 0x7f3d1c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 -- 172.21.15.94:0/1313767003 >> 172.21.15.94:0/1313767003 conn(0x7f3d1c0fdc00 msgr2=0x7f3d1c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 -- 172.21.15.94:0/1313767003 shutdown_connections 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.274+0000 7f3d2372e640 1 -- 172.21.15.94:0/1313767003 wait complete. 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 Processor -- start 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 -- start start 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 0x7f3d1c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10e520 con 0x7f3d1c108930 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10e3a0 con 0x7f3d1c103c30 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d2372e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c10e6a0 con 0x7f3d1c1029f0 2024-04-25T13:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d21ccb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d21ccb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39738/0 (socket says 172.21.15.94:39738) 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d214ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 0x7f3d1c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d21ccb640 1 -- 172.21.15.94:0/696218971 learned_addr learned my addr 172.21.15.94:0/696218971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.275+0000 7f3d21ccb640 1 -- 172.21.15.94:0/696218971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 msgr2=0x7f3d1c19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d21ccb640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 0x7f3d1c19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d21ccb640 1 -- 172.21.15.94:0/696218971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 msgr2=0x7f3d1c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d21ccb640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d21ccb640 1 -- 172.21.15.94:0/696218971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c1a1d30 con 0x7f3d1c108930 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d21ccb640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004af0 tx=0x7f3d14031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d127fc640 1 -- 172.21.15.94:0/696218971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14003b70 con 0x7f3d1c108930 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c1a1fc0 con 0x7f3d1c108930 2024-04-25T13:56:01.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d127fc640 1 -- 172.21.15.94:0/696218971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d14003d10 con 0x7f3d1c108930 2024-04-25T13:56:01.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d127fc640 1 -- 172.21.15.94:0/696218971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1402fb20 con 0x7f3d1c108930 2024-04-25T13:56:01.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.276+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c1a2430 con 0x7f3d1c108930 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.279+0000 7f3d127fc640 1 -- 172.21.15.94:0/696218971 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3d1402fcc0 con 0x7f3d1c108930 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.279+0000 7f3d2a5d9740 1 Processor -- start 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.279+0000 7f3d2a5d9740 1 -- start start 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86ceae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556c86c49d80 con 0x556c8691a4d0 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c86c03a20 con 0x556c86943390 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c86c45d40 con 0x556c869226b0 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34148/0 (socket says 172.21.15.94:34148) 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 -- 172.21.15.94:0/1716171191 learned_addr learned my addr 172.21.15.94:0/1716171191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d214ca640 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86ceae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d127fc640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 0x7f3d080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 -- 172.21.15.94:0/1716171191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 msgr2=0x556c86ceae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86ceae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 -- 172.21.15.94:0/1716171191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 msgr2=0x556c86cea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 -- 172.21.15.94:0/1716171191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c86c0ec30 con 0x556c86943390 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d21ccb640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 0x7f3d080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c011c00 tx=0x7f3d0c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe505f912426eb8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.280+0000 7f3d127fc640 1 -- 172.21.15.94:0/696218971 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d14100430 con 0x7f3d1c108930 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.281+0000 7f3d077fe640 1 -- 172.21.15.94:0/1716171191 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00fc50 con 0x556c86943390 2024-04-25T13:56:01.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.281+0000 7f3d077fe640 1 -- 172.21.15.94:0/1716171191 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c00fdf0 con 0x556c86943390 2024-04-25T13:56:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.281+0000 7f3d21ccb640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 0x7f3d080b2000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d140fe200 tx=0x7f3d140f6fc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.281+0000 7f3d077fe640 1 -- 172.21.15.94:0/1716171191 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e5f0 con 0x556c86943390 2024-04-25T13:56:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1716171191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 msgr2=0x556c86ce9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c011c00 tx=0x7f3d0c009ba0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1716171191 shutdown_connections 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86ceae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1716171191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86ce9c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1716171191 >> 172.21.15.94:0/1716171191 conn(0x556c869374e0 msgr2=0x556c86ce1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1716171191 shutdown_connections 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1716171191 wait complete. 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 -- start start 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.282+0000 7f3d2a5d9740 1 Processor -- start 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cf3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86cfae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556c86ceb940 con 0x556c8691a4d0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c86bd0dc0 con 0x556c86943390 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c86cebac0 con 0x556c869226b0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d2a5d9740 10 client.0 _ll_register_callbacks cb 0x556c86bec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39766/0 (socket says 172.21.15.94:39766) 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 -- 172.21.15.94:0/871299322 learned_addr learned my addr 172.21.15.94:0/871299322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d214ca640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cf3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86cfae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 -- 172.21.15.94:0/871299322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 msgr2=0x556c86cf3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cf3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 -- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 msgr2=0x556c86cfae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86cfae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 -- 172.21.15.94:0/871299322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8695b3f0 con 0x556c8691a4d0 2024-04-25T13:56:01.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0042d0 tx=0x7f3d0c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73abe8386787c8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.283+0000 7f3d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c01f440 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d0c019040 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c00fdf0 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d0c01f5e0 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 10 client.20643 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c86a257d0 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c00fdf0 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3d0c01d020 con 0x556c8691a4d0 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 10 client.20643 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 10 client.20643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 20 client.20643 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:01.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.284+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c86901f60 0x556c86d013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.285+0000 7f3d2a5d9740 4 client.20643 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.285+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c86c57990 con 0x556c86901f60 2024-04-25T13:56:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.285+0000 7f3d214ca640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c86901f60 0x556c86d013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.286+0000 7f3d04ff9640 10 client.20643 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:01.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.286+0000 7f3d214ca640 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c86901f60 0x556c86d013d0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c8dcb27f01db5e server_cookie=f6ac594f7f894ea4 in_seq=0 out_seq=0 2024-04-25T13:56:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.292+0000 7f3d04ff9640 1 -- 172.21.15.94:0/871299322 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c86c57990 con 0x556c86901f60 2024-04-25T13:56:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.292+0000 7f3d04ff9640 10 client.20643 handle_command_reply: tid=0 2024-04-25T13:56:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.292+0000 7f3d2a5d9740 1 client.20643 shutdown 2024-04-25T13:56:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 msgr2=0x556c86cf39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0042d0 tx=0x7f3d0c005850 comp rx=0 tx=0).stop 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 shutdown_connections 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c86901f60 0x556c86d013d0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c86943390 0x556c86cfae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cf3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/871299322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c8691a4d0 0x556c86cf39c0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 >> 172.21.15.94:0/871299322 conn(0x556c869374e0 msgr2=0x556c86ce0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d04ff9640 0 client.20643 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 shutdown_connections 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/871299322 wait complete. 2024-04-25T13:56:01.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.293+0000 7f3d2a5d9740 20 client.20643 trim_cache size 0 max 16384 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 Processor -- start 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 -- start start 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86ce9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1a9350 con 0x556c86943390 2024-04-25T13:56:01.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1aaf50 con 0x556c8691a4d0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1a9a20 con 0x556c869226b0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d20cc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39768/0 (socket says 172.21.15.94:39768) 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d20cc9640 1 -- 172.21.15.94:0/447253928 learned_addr learned my addr 172.21.15.94:0/447253928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86ce9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.307+0000 7f3d214ca640 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 -- 172.21.15.94:0/447253928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 msgr2=0x556c86cea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 -- 172.21.15.94:0/447253928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 msgr2=0x556c86ce9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86ce9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 -- 172.21.15.94:0/447253928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c86cf1af0 con 0x556c86943390 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00fca0 tx=0x7f3d0c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c63255d10c91b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d06ffd640 1 -- 172.21.15.94:0/447253928 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c01f440 con 0x556c86943390 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d06ffd640 1 -- 172.21.15.94:0/447253928 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c009e90 con 0x556c86943390 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.308+0000 7f3d06ffd640 1 -- 172.21.15.94:0/447253928 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c030300 con 0x556c86943390 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/447253928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 msgr2=0x556c86cf3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00fca0 tx=0x7f3d0c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/447253928 shutdown_connections 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c86943390 0x556c86cf3590 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c869226b0 0x556c86cea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/447253928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86ce9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/447253928 >> 172.21.15.94:0/447253928 conn(0x556c869374e0 msgr2=0x556c86cee780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/447253928 shutdown_connections 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/447253928 wait complete. 2024-04-25T13:56:01.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 -- start start 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.309+0000 7f3d2a5d9740 1 Processor -- start 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86cf7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c86943390 0x556c86cf9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1aa880 con 0x556c869226b0 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1aa0f0 con 0x556c8691a4d0 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1a7330 con 0x556c86943390 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d2a5d9740 10 client.0 _ll_register_callbacks cb 0x7f3d18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39774/0 (socket says 172.21.15.94:39774) 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 -- 172.21.15.94:0/1843021038 learned_addr learned my addr 172.21.15.94:0/1843021038 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:01.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86cf7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c86943390 0x556c86cf9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 -- 172.21.15.94:0/1843021038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c86943390 msgr2=0x556c86cf9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c86943390 0x556c86cf9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 -- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 msgr2=0x556c86cf7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86cf7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 -- 172.21.15.94:0/1843021038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c86cf9700 con 0x556c869226b0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d214ca640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x556c86ce9940 tx=0x7f3d18010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fabb65605f2613d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.310+0000 7f3d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18023020 con 0x556c869226b0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f3d180035f0 con 0x556c869226b0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c009e90 con 0x556c869226b0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d1802e050 con 0x556c869226b0 2024-04-25T13:56:01.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 10 client.20649 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c86d697e0 con 0x556c869226b0 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c009e90 con 0x556c869226b0 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f3d1801f620 con 0x556c869226b0 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 10 client.20649 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 10 client.20649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 20 client.20649 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c868d73a0 0x556c86d6c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 4 client.20649 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.311+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c86c51800 con 0x556c868d73a0 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.312+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c868d73a0 0x556c86d6c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.312+0000 7f3d04ff9640 10 client.20649 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:01.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.312+0000 7f3d20cc9640 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c868d73a0 0x556c86d6c2c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1659d61a702eec server_cookie=9a430f8ae85908f5 in_seq=0 out_seq=0 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.314+0000 7f3d04ff9640 1 -- 172.21.15.94:0/1843021038 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c86c51800 con 0x556c868d73a0 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.314+0000 7f3d04ff9640 10 client.20649 handle_command_reply: tid=0 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.314+0000 7f3d2a5d9740 1 client.20649 shutdown 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.314+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 msgr2=0x556c86cf8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.314+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x556c86ce9940 tx=0x7f3d18010aa0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 shutdown_connections 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556c868d73a0 0x556c86d6c2c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c86943390 0x556c86cf9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c869226b0 0x556c86cf8210 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 --2- 172.21.15.94:0/1843021038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c8691a4d0 0x556c86cf7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 >> 172.21.15.94:0/1843021038 conn(0x556c869374e0 msgr2=0x556c86cee780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d04ff9640 0 client.20649 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:01.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 shutdown_connections 2024-04-25T13:56:01.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 1 -- 172.21.15.94:0/1843021038 wait complete. 2024-04-25T13:56:01.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2a5d9740 20 client.20649 trim_cache size 0 max 16384 2024-04-25T13:56:01.319 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 msgr2=0x7f3d080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 0x7f3d080b2000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d140fe200 tx=0x7f3d140f6fc0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 msgr2=0x7f3d1c1a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004af0 tx=0x7f3d14031700 comp rx=0 tx=0).stop 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 shutdown_connections 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3d080afaf0 0x7f3d080b2000 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d1c108930 0x7f3d1c1a16c0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d1c103c30 0x7f3d1c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 --2- 172.21.15.94:0/696218971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d1c1029f0 0x7f3d1c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:01.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.315+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 >> 172.21.15.94:0/696218971 conn(0x7f3d1c0fdc00 msgr2=0x7f3d1c109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:01.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.316+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 shutdown_connections 2024-04-25T13:56:01.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:01.316+0000 7f3d2372e640 1 -- 172.21.15.94:0/696218971 wait complete. 2024-04-25T13:56:01.324 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:01 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3555891879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:01.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:01 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3555891879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:01.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:01 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3555891879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:02.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 16837 24.54 MB/sec warmup 44 sec latency 81.776 ms 2024-04-25T13:56:02.326 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:56:02.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:56:02.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:56:02.348 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:02 smithi094 ceph-mon[29195]: pgmap v776: 97 pgs: 97 active+clean; 243 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-25T13:56:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:02 smithi137 ceph-mon[32568]: pgmap v776: 97 pgs: 97 active+clean; 243 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-25T13:56:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 -- 172.21.15.94:0/1273572532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d41029f0 msgr2=0x7fd5d41036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/1273572532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d41029f0 0x7fd5d41036f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009900 tx=0x7fd5c402a410 comp rx=0 tx=0).stop 2024-04-25T13:56:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 -- 172.21.15.94:0/1273572532 shutdown_connections 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/1273572532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 0x7fd5d410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/1273572532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d4103c30 0x7fd5d4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/1273572532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d41029f0 0x7fd5d41036f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 -- 172.21.15.94:0/1273572532 >> 172.21.15.94:0/1273572532 conn(0x7fd5d40fdbc0 msgr2=0x7fd5d4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 -- 172.21.15.94:0/1273572532 shutdown_connections 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.453+0000 7fd5d89b2640 1 -- 172.21.15.94:0/1273572532 wait complete. 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 Processor -- start 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 -- start start 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d41029f0 0x7fd5d419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 0x7fd5d41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d410e4d0 con 0x7fd5d4103c30 2024-04-25T13:56:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d410e350 con 0x7fd5d41029f0 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d89b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d410e650 con 0x7fd5d4108930 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39802/0 (socket says 172.21.15.94:39802) 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/2247394111 learned_addr learned my addr 172.21.15.94:0/2247394111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d259c640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d41029f0 0x7fd5d419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.454+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 0x7fd5d41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/2247394111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 msgr2=0x7fd5d41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 0x7fd5d41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/2247394111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d41029f0 msgr2=0x7fd5d419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d41029f0 0x7fd5d419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/2247394111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41a1d90 con 0x7fd5d4103c30 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5c37fe640 1 -- 172.21.15.94:0/2247394111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cd80 con 0x7fd5d4103c30 2024-04-25T13:56:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5c37fe640 1 -- 172.21.15.94:0/2247394111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c80108e0 con 0x7fd5d4103c30 2024-04-25T13:56:02.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a2080 con 0x7fd5d4103c30 2024-04-25T13:56:02.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5c37fe640 1 -- 172.21.15.94:0/2247394111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8018980 con 0x7fd5d4103c30 2024-04-25T13:56:02.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.455+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a25d0 con 0x7fd5d4103c30 2024-04-25T13:56:02.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.457+0000 7fd5c37fe640 1 -- 172.21.15.94:0/2247394111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd5c8018b20 con 0x7fd5d4103c30 2024-04-25T13:56:02.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.458+0000 7fd5c37fe640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 0x7fd5bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.458+0000 7fd5df865740 1 Processor -- start 2024-04-25T13:56:02.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.458+0000 7fd5c37fe640 1 -- 172.21.15.94:0/2247394111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd5c80df020 con 0x7fd5d4103c30 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.458+0000 7fd5df865740 1 -- start start 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.458+0000 7fd5d259c640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 0x7fd5bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bacdfc9b00 con 0x55bacdcc3410 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bacdf65920 con 0x55bacdca2730 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5df865740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bacdfc5ac0 con 0x55bacdc9a550 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34204/0 (socket says 172.21.15.94:34204) 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/60927119 learned_addr learned my addr 172.21.15.94:0/60927119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace064a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58490/0 (socket says 172.21.15.94:58490) 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d259c640 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d259c640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 0x7fd5bc0b2000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005ab0 tx=0x7fd5c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/60927119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 msgr2=0x55bace064a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace064a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/60927119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 msgr2=0x55bace0644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/60927119 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacdda37e0 con 0x55bacdca2730 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009fd0 tx=0x7fd5cc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639629222075f839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.459+0000 7fd5c0ff9640 1 -- 172.21.15.94:0/60927119 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc004d20 con 0x55bacdca2730 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.460+0000 7fd5c0ff9640 1 -- 172.21.15.94:0/60927119 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x55bacdca2730 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.460+0000 7fd5c0ff9640 1 -- 172.21.15.94:0/60927119 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc005280 con 0x55bacdca2730 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.460+0000 7fd5df865740 1 -- 172.21.15.94:0/60927119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 msgr2=0x55bace06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.460+0000 7fd5df865740 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009fd0 tx=0x7fd5cc0118d0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- 172.21.15.94:0/60927119 shutdown_connections 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace06b920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace064a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- 172.21.15.94:0/60927119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- 172.21.15.94:0/60927119 >> 172.21.15.94:0/60927119 conn(0x55bacdcb7560 msgr2=0x55bace05fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- 172.21.15.94:0/60927119 shutdown_connections 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- 172.21.15.94:0/60927119 wait complete. 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- start start 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 Processor -- start 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace079b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bace06c370 con 0x55bacdca2730 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bacdf635d0 con 0x55bacdcc3410 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bace06c4f0 con 0x55bacdc9a550 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.461+0000 7fd5df865740 10 client.0 _ll_register_callbacks cb 0x55bacdf51080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace079b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58506/0 (socket says 172.21.15.94:58506) 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace072bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39814/0 (socket says 172.21.15.94:39814) 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d259c640 1 -- 172.21.15.94:0/3953542991 learned_addr learned my addr 172.21.15.94:0/3953542991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/3953542991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 msgr2=0x55bace072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/3953542991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 msgr2=0x55bace072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace072bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/3953542991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacde03f70 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace079b10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc01b040 tx=0x7fd5cc004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d099eec111af2d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc028020 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd5cc004410 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc004050 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc033050 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5df865740 10 client.30095 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.462+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bacdc9b170 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd5cc00f4e0 con 0x55bacdcc3410 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 10 client.30095 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 10 client.30095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 20 client.30095 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc821f0 0x55bace07ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 4 client.30095 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacdfd7710 con 0x55bacdc821f0 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.463+0000 7fd5d259c640 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc821f0 0x55bace07ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.464+0000 7fd5b9ffb640 10 client.30095 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.464+0000 7fd5d259c640 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc821f0 0x55bace07ff40 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4899b2f3a6cb3f server_cookie=d4b9159f3394494c in_seq=0 out_seq=0 2024-04-25T13:56:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.467+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/3953542991 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bacdfd7710 con 0x55bacdc821f0 2024-04-25T13:56:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.467+0000 7fd5b9ffb640 10 client.30095 handle_command_reply: tid=0 2024-04-25T13:56:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.467+0000 7fd5df865740 1 client.30095 shutdown 2024-04-25T13:56:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 msgr2=0x55bace079b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace079b10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc01b040 tx=0x7fd5cc004c70 comp rx=0 tx=0).stop 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 shutdown_connections 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc821f0 0x55bace07ff40 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace079b10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace072bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 --2- 172.21.15.94:0/3953542991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5b9ffb640 0 client.30095 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 >> 172.21.15.94:0/3953542991 conn(0x55bacdcb7560 msgr2=0x55bace05eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 shutdown_connections 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 1 -- 172.21.15.94:0/3953542991 wait complete. 2024-04-25T13:56:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.468+0000 7fd5df865740 20 client.30095 trim_cache size 0 max 16384 2024-04-25T13:56:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.489+0000 7fd5df865740 1 Processor -- start 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 -- start start 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace063340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace085700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41a9450 con 0x55bacdca2730 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41ab050 con 0x55bacdcc3410 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5df865740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41a9b20 con 0x55bacdc9a550 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39820/0 (socket says 172.21.15.94:39820) 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/3085472721 learned_addr learned my addr 172.21.15.94:0/3085472721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace085700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d259c640 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/3085472721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 msgr2=0x55bace063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace063340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/3085472721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 msgr2=0x55bace085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace085700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 -- 172.21.15.94:0/3085472721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bace070910 con 0x55bacdca2730 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.490+0000 7fd5d1d9b640 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80dcdf0 tx=0x7fd5c80af040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80a670b85bf75cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5bbfff640 1 -- 172.21.15.94:0/3085472721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80b02c0 con 0x55bacdca2730 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5bbfff640 1 -- 172.21.15.94:0/3085472721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c8010a90 con 0x55bacdca2730 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5bbfff640 1 -- 172.21.15.94:0/3085472721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80a3df0 con 0x55bacdca2730 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 -- 172.21.15.94:0/3085472721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 msgr2=0x55bace079a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80dcdf0 tx=0x7fd5c80af040 comp rx=0 tx=0).stop 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 -- 172.21.15.94:0/3085472721 shutdown_connections 2024-04-25T13:56:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdcc3410 0x55bace085700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdca2730 0x55bace079a30 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 --2- 172.21.15.94:0/3085472721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace063340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.491+0000 7fd5df865740 1 -- 172.21.15.94:0/3085472721 >> 172.21.15.94:0/3085472721 conn(0x55bacdcb7560 msgr2=0x55bace062c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- 172.21.15.94:0/3085472721 shutdown_connections 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- 172.21.15.94:0/3085472721 wait complete. 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- start start 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 Processor -- start 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace0e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace0e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41aa980 con 0x55bacdcc3410 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41aa1f0 con 0x55bacdca2730 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41a7430 con 0x55bacdc9a550 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5df865740 10 client.0 _ll_register_callbacks cb 0x7fd5cc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5d2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5d1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace0e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5d2d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39828/0 (socket says 172.21.15.94:39828) 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5d259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace0e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.492+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/1973833070 learned_addr learned my addr 172.21.15.94:0/1973833070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/1973833070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 msgr2=0x55bace0e2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace0e2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 msgr2=0x55bace0e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace0e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 -- 172.21.15.94:0/1973833070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bace0e74c0 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d259c640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace0e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5d2d9d640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00f980 tx=0x7fd5cc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c59580889e4d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03b020 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd5cc00fb70 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c8010a90 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc028040 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5df865740 10 client.20667 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bace0e8780 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c8010a90 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.493+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd5cc041720 con 0x55bacdcc3410 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 10 client.20667 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 10 client.20667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 20 client.20667 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc57630 0x55bace0eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 4 client.20667 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bacdfd1580 con 0x55bacdc57630 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5d259c640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc57630 0x55bace0eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5b9ffb640 10 client.20667 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.494+0000 7fd5d259c640 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc57630 0x55bace0eb190 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0dc4b1f5a219836 server_cookie=f4c16e83bb7b4b5a in_seq=0 out_seq=0 2024-04-25T13:56:02.500 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:02 smithi138 ceph-mon[32977]: pgmap v776: 97 pgs: 97 active+clean; 243 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 516 op/s 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5b9ffb640 1 -- 172.21.15.94:0/1973833070 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bacdfd1580 con 0x55bacdc57630 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5b9ffb640 10 client.20667 handle_command_reply: tid=0 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 client.20667 shutdown 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 msgr2=0x55bace0e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00f980 tx=0x7fd5cc00f710 comp rx=0 tx=0).stop 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 shutdown_connections 2024-04-25T13:56:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bacdc57630 0x55bace0eb190 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bacdcc3410 0x55bace0e6d60 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bacdca2730 0x55bace0e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 --2- 172.21.15.94:0/1973833070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bacdc9a550 0x55bace0e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 >> 172.21.15.94:0/1973833070 conn(0x55bacdcb7560 msgr2=0x55bace063660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.497+0000 7fd5b9ffb640 0 client.20667 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 shutdown_connections 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5df865740 1 -- 172.21.15.94:0/1973833070 wait complete. 2024-04-25T13:56:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5df865740 20 client.20667 trim_cache size 0 max 16384 2024-04-25T13:56:02.503 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:56:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:56:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "622f81f3-ec49-4966-a73c-5b9939260c14", 2024-04-25T13:56:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:56:02.503 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 msgr2=0x7fd5bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 0x7fd5bc0b2000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4005ab0 tx=0x7fd5c4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 msgr2=0x7fd5d419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 shutdown_connections 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5bc0afaf0 0x7fd5bc0b2000 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d4108930 0x7fd5d41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4103c30 0x7fd5d419a790 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 --2- 172.21.15.94:0/2247394111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5d41029f0 0x7fd5d419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 >> 172.21.15.94:0/2247394111 conn(0x7fd5d40fdbc0 msgr2=0x7fd5d4109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 shutdown_connections 2024-04-25T13:56:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.498+0000 7fd5d89b2640 1 -- 172.21.15.94:0/2247394111 wait complete. 2024-04-25T13:56:02.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:56:02.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.671+0000 7f54128b6640 1 -- 172.21.15.94:0/1796989194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c103c20 msgr2=0x7f540c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.671+0000 7f54128b6640 1 --2- 172.21.15.94:0/1796989194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c103c20 0x7f540c104030 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402a410 comp rx=0 tx=0).stop 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.671+0000 7f54128b6640 1 -- 172.21.15.94:0/1796989194 shutdown_connections 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.671+0000 7f54128b6640 1 --2- 172.21.15.94:0/1796989194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 0x7f540c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 --2- 172.21.15.94:0/1796989194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c103c20 0x7f540c104030 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 --2- 172.21.15.94:0/1796989194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1029e0 0x7f540c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 -- 172.21.15.94:0/1796989194 >> 172.21.15.94:0/1796989194 conn(0x7f540c0fdbd0 msgr2=0x7f540c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 -- 172.21.15.94:0/1796989194 shutdown_connections 2024-04-25T13:56:02.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 -- 172.21.15.94:0/1796989194 wait complete. 2024-04-25T13:56:02.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 Processor -- start 2024-04-25T13:56:02.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.672+0000 7f54128b6640 1 -- start start 2024-04-25T13:56:02.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c103c20 0x7f540c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 0x7f540c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f540c10e500 con 0x7f540c1029e0 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f540c10e380 con 0x7f540c108920 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f540c10e680 con 0x7f540c103c20 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39848/0 (socket says 172.21.15.94:39848) 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c103c20 0x7f540c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 -- 172.21.15.94:0/1292150636 learned_addr learned my addr 172.21.15.94:0/1292150636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f5410e53640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 0x7f540c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 -- 172.21.15.94:0/1292150636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c103c20 msgr2=0x7f540c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c103c20 0x7f540c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 -- 172.21.15.94:0/1292150636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 msgr2=0x7f540c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 0x7f540c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 -- 172.21.15.94:0/1292150636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1a2080 con 0x7f540c1029e0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f540bfff640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e9f0 tx=0x7f53fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54097fa640 1 -- 172.21.15.94:0/1292150636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00ccc0 con 0x7f540c1029e0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54097fa640 1 -- 172.21.15.94:0/1292150636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc0035f0 con 0x7f540c1029e0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a2370 con 0x7f540c1029e0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.673+0000 7f54097fa640 1 -- 172.21.15.94:0/1292150636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00ce60 con 0x7f540c1029e0 2024-04-25T13:56:02.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.674+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a2870 con 0x7f540c1029e0 2024-04-25T13:56:02.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 Processor -- start 2024-04-25T13:56:02.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f54097fa640 1 -- 172.21.15.94:0/1292150636 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f53fc003aa0 con 0x7f540c1029e0 2024-04-25T13:56:02.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 -- start start 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597273efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597237c6b0 0x55597273fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5559726a3d80 con 0x55597239d390 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55597265d8b0 con 0x5559723744d0 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f541976f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55597269fd40 con 0x55597237c6b0 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39864/0 (socket says 172.21.15.94:39864) 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 -- 172.21.15.94:0/1590958168 learned_addr learned my addr 172.21.15.94:0/1590958168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f540bfff640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597273efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f540b7fe640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597237c6b0 0x55597273fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 -- 172.21.15.94:0/1590958168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597237c6b0 msgr2=0x55597273fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f54097fa640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 0x7f53f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597237c6b0 0x55597273fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 -- 172.21.15.94:0/1590958168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 msgr2=0x55597273efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597273efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f5410e53640 1 -- 172.21.15.94:0/1590958168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55597262ea40 con 0x55597239d390 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.677+0000 7f540bfff640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597273efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f54097fa640 1 -- 172.21.15.94:0/1292150636 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f53fc0deff0 con 0x7f540c1029e0 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f5410e53640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 0x7f53f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f5410e53640 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f54040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56197867ebfb64fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f54027fc640 1 -- 172.21.15.94:0/1590958168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404004050 con 0x55597239d390 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f54027fc640 1 -- 172.21.15.94:0/1590958168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5404004720 con 0x55597239d390 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.678+0000 7f54027fc640 1 -- 172.21.15.94:0/1590958168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404019400 con 0x55597239d390 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- 172.21.15.94:0/1590958168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 msgr2=0x55597273ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f5410e53640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 0x7f53f80b2000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f540401d000 tx=0x7f5404019680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f54040118d0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- 172.21.15.94:0/1590958168 shutdown_connections 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597237c6b0 0x55597273fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597273efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 --2- 172.21.15.94:0/1590958168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x55597273ea60 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- 172.21.15.94:0/1590958168 >> 172.21.15.94:0/1590958168 conn(0x5559723914e0 msgr2=0x55597273a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- 172.21.15.94:0/1590958168 shutdown_connections 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- 172.21.15.94:0/1590958168 wait complete. 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 -- start start 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.679+0000 7f541976f740 1 Processor -- start 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559723744d0 0x55597274cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555972740780 con 0x55597239d390 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55597262ad10 con 0x55597237c6b0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555972740900 con 0x5559723744d0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 10 client.0 _ll_register_callbacks cb 0x555972657210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39866/0 (socket says 172.21.15.94:39866) 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 -- 172.21.15.94:0/3905845973 learned_addr learned my addr 172.21.15.94:0/3905845973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f540b7fe640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f540bfff640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559723744d0 0x55597274cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 -- 172.21.15.94:0/3905845973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559723744d0 msgr2=0x55597274cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559723744d0 0x55597274cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 -- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 msgr2=0x55597274d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 -- 172.21.15.94:0/3905845973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55597247f7d0 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f540b7fe640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f5410e53640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5404003a80 tx=0x7f5404017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c6857f79b4924e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f53eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404033070 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5404004720 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.680+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f540400fc00 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f540403f050 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 10 client.20679 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559723b53f0 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5404004720 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f540402f750 con 0x55597239d390 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 10 client.20679 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:02.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 10 client.20679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 20 client.20679 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55597235bf60 0x55597275a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 4 client.20679 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559726b1990 con 0x55597235bf60 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.681+0000 7f540bfff640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55597235bf60 0x55597275a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.682+0000 7f53eb7fe640 10 client.20679 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.682+0000 7f540bfff640 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55597235bf60 0x55597275a550 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a964112e2bba3c3d server_cookie=bb75de7f80f05cd5 in_seq=0 out_seq=0 2024-04-25T13:56:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.684+0000 7f53eb7fe640 1 -- 172.21.15.94:0/3905845973 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559726b1990 con 0x55597235bf60 2024-04-25T13:56:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f53eb7fe640 10 client.20679 handle_command_reply: tid=0 2024-04-25T13:56:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 client.20679 shutdown 2024-04-25T13:56:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 msgr2=0x555972754110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5404003a80 tx=0x7f5404017000 comp rx=0 tx=0).stop 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 shutdown_connections 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55597235bf60 0x55597275a550 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597239d390 0x555972754110 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 --2- 172.21.15.94:0/3905845973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559723744d0 0x55597274cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f53eb7fe640 0 client.20679 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 >> 172.21.15.94:0/3905845973 conn(0x5559723914e0 msgr2=0x5559727393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 shutdown_connections 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 1 -- 172.21.15.94:0/3905845973 wait complete. 2024-04-25T13:56:02.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.685+0000 7f541976f740 20 client.20679 trim_cache size 0 max 16384 2024-04-25T13:56:02.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.709+0000 7f541976f740 1 Processor -- start 2024-04-25T13:56:02.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.709+0000 7f541976f740 1 -- start start 2024-04-25T13:56:02.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f540bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f540bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39874/0 (socket says 172.21.15.94:39874) 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f540bfff640 1 -- 172.21.15.94:0/3304458121 learned_addr learned my addr 172.21.15.94:0/3304458121 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x555972739b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a93d0 con 0x5559723744d0 2024-04-25T13:56:02.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aafd0 con 0x55597237c6b0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.710+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a9aa0 con 0x55597239d390 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f5410e53640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x555972739b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540b7fe640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 -- 172.21.15.94:0/3304458121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 msgr2=0x555972739b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x555972739b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 -- 172.21.15.94:0/3304458121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 msgr2=0x55597274b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 -- 172.21.15.94:0/3304458121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559723750f0 con 0x5559723744d0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540b7fe640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f540bfff640 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0a73c0 tx=0x7f53fc0e2ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ba09ed86dc0f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f5401ffb640 1 -- 172.21.15.94:0/3304458121 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a6f40 con 0x5559723744d0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f5401ffb640 1 -- 172.21.15.94:0/3304458121 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc0a7b30 con 0x5559723744d0 2024-04-25T13:56:02.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.711+0000 7f5401ffb640 1 -- 172.21.15.94:0/3304458121 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a36c0 con 0x5559723744d0 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.712+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 msgr2=0x55597274b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.712+0000 7f541976f740 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0a73c0 tx=0x7f53fc0e2ad0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.713+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 shutdown_connections 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.713+0000 7f541976f740 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x555972739b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.713+0000 7f541976f740 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55597237c6b0 0x55597274b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.713+0000 7f541976f740 1 --2- 172.21.15.94:0/3304458121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559723744d0 0x55597274b290 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.713+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 >> 172.21.15.94:0/3304458121 conn(0x5559723914e0 msgr2=0x55597273e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.714+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 shutdown_connections 2024-04-25T13:56:02.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 -- 172.21.15.94:0/3304458121 wait complete. 2024-04-25T13:56:02.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 -- start start 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 Processor -- start 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597274f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x5559727544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aa900 con 0x55597237c6b0 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aa170 con 0x5559723744d0 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1a73b0 con 0x55597239d390 2024-04-25T13:56:02.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f541976f740 10 client.0 _ll_register_callbacks cb 0x7f5404028070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f540b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f540b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39880/0 (socket says 172.21.15.94:39880) 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f540b7fe640 1 -- 172.21.15.94:0/4051314982 learned_addr learned my addr 172.21.15.94:0/4051314982 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.715+0000 7f540bfff640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597274f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 -- 172.21.15.94:0/4051314982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 msgr2=0x5559727544e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T13:56:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x5559727544e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 -- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 msgr2=0x55597274f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597274f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 -- 172.21.15.94:0/4051314982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559726644b0 con 0x55597237c6b0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f540b7fe640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f53f4031500 tx=0x7f53f40318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb908f45b11d09d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f53eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4035040 con 0x55597237c6b0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f53f4004250 con 0x55597237c6b0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.716+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc0a7b30 con 0x55597237c6b0 2024-04-25T13:56:02.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53f4004450 con 0x55597237c6b0 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 10 client.20685 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559727c1600 con 0x55597237c6b0 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc0a7b30 con 0x55597237c6b0 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f53f403c760 con 0x55597237c6b0 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 10 client.20685 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 10 client.20685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 20 client.20685 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5559723313a0 0x5559727c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 4 client.20685 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:56:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.717+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559726ab800 con 0x5559723313a0 2024-04-25T13:56:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.718+0000 7f540bfff640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5559723313a0 0x5559727c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.719+0000 7f53eb7fe640 10 client.20685 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.719+0000 7f540bfff640 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5559723313a0 0x5559727c4130 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdaf4d5389474da server_cookie=60fde035f31fdbc in_seq=0 out_seq=0 2024-04-25T13:56:02.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.723+0000 7f53eb7fe640 1 -- 172.21.15.94:0/4051314982 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5559726ab800 con 0x5559723313a0 2024-04-25T13:56:02.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.723+0000 7f53eb7fe640 10 client.20685 handle_command_reply: tid=0 2024-04-25T13:56:02.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.723+0000 7f541976f740 1 client.20685 shutdown 2024-04-25T13:56:02.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.724+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 msgr2=0x55597274f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.724+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f53f4031500 tx=0x7f53f40318f0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 shutdown_connections 2024-04-25T13:56:02.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5559723313a0 0x5559727c4130 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55597239d390 0x5559727544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55597237c6b0 0x55597274f6a0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 --2- 172.21.15.94:0/4051314982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559723744d0 0x55597274f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 >> 172.21.15.94:0/4051314982 conn(0x5559723914e0 msgr2=0x55597273e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f53eb7fe640 0 client.20685 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 shutdown_connections 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 1 -- 172.21.15.94:0/4051314982 wait complete. 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.726+0000 7f541976f740 20 client.20685 trim_cache size 0 max 16384 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (456 inodes in the stack)", 2024-04-25T13:56:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "622f81f3-ec49-4966-a73c-5b9939260c14": { 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "622f81f3-ec49-4966-a73c-5b9939260c14", 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:56:02.728 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 msgr2=0x7f53f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 0x7f53f80b2000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f540401d000 tx=0x7f5404019680 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 msgr2=0x7f540c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e9f0 tx=0x7f53fc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 shutdown_connections 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f53f80afaf0 0x7f53f80b2000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f540c108920 0x7f540c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c103c20 0x7f540c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 --2- 172.21.15.94:0/1292150636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f540c1029e0 0x7f540c06a920 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:02.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 >> 172.21.15.94:0/1292150636 conn(0x7f540c0fdbd0 msgr2=0x7f540c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:02.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 shutdown_connections 2024-04-25T13:56:02.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:02.727+0000 7f54128b6640 1 -- 172.21.15.94:0/1292150636 wait complete. 2024-04-25T13:56:02.740 INFO:tasks.cephfs.filesystem:scrub status for tag:622f81f3-ec49-4966-a73c-5b9939260c14 - {'path': '/', 'tag': '622f81f3-ec49-4966-a73c-5b9939260c14', 'options': 'recursive,force'} 2024-04-25T13:56:03.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 17285 24.63 MB/sec warmup 45 sec latency 85.647 ms 2024-04-25T13:56:03.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:56:03.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:03.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:03 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:56:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:03.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:03.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:03.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:03 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:56:03.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:03.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:03 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 17646 24.62 MB/sec warmup 46 sec latency 52.520 ms 2024-04-25T13:56:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: pgmap v777: 97 pgs: 97 active+clean; 242 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-04-25T13:56:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:04 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: pgmap v777: 97 pgs: 97 active+clean; 242 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-04-25T13:56:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:04 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: pgmap v777: 97 pgs: 97 active+clean; 242 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:04.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:04 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:05.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 17932 24.38 MB/sec warmup 47 sec latency 94.421 ms 2024-04-25T13:56:06.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 18131 24.11 MB/sec warmup 48 sec latency 259.320 ms 2024-04-25T13:56:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:06 smithi137 ceph-mon[32568]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-04-25T13:56:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:06 smithi094 ceph-mon[29195]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-04-25T13:56:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:06 smithi138 ceph-mon[32977]: pgmap v778: 97 pgs: 97 active+clean; 236 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.3 MiB/s wr, 603 op/s 2024-04-25T13:56:07.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 18410 23.82 MB/sec warmup 49 sec latency 239.349 ms 2024-04-25T13:56:08.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 18819 23.79 MB/sec warmup 50 sec latency 75.610 ms 2024-04-25T13:56:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:08 smithi137 ceph-mon[32568]: pgmap v779: 97 pgs: 97 active+clean; 232 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 568 op/s 2024-04-25T13:56:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:08 smithi094 ceph-mon[29195]: pgmap v779: 97 pgs: 97 active+clean; 232 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 568 op/s 2024-04-25T13:56:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:08 smithi138 ceph-mon[32977]: pgmap v779: 97 pgs: 97 active+clean; 232 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 568 op/s 2024-04-25T13:56:09.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 19288 23.61 MB/sec warmup 51 sec latency 79.108 ms 2024-04-25T13:56:10.135 INFO:tasks.workunit.client.0.smithi094.stdout: 10 19821 23.75 MB/sec warmup 52 sec latency 54.328 ms 2024-04-25T13:56:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:10 smithi137 ceph-mon[32568]: pgmap v780: 97 pgs: 97 active+clean; 231 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-04-25T13:56:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:10 smithi094 ceph-mon[29195]: pgmap v780: 97 pgs: 97 active+clean; 231 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-04-25T13:56:10.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:10 smithi138 ceph-mon[32977]: pgmap v780: 97 pgs: 97 active+clean; 231 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-04-25T13:56:11.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 20266 23.91 MB/sec warmup 53 sec latency 161.844 ms 2024-04-25T13:56:11.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:11.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:11.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:11.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:11.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:11.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:12.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 20779 23.97 MB/sec warmup 54 sec latency 97.892 ms 2024-04-25T13:56:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:12 smithi137 ceph-mon[32568]: pgmap v781: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-25T13:56:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:12 smithi094 ceph-mon[29195]: pgmap v781: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-25T13:56:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:12.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:12 smithi138 ceph-mon[32977]: pgmap v781: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.3 MiB/s wr, 658 op/s 2024-04-25T13:56:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:13.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 21123 23.95 MB/sec warmup 55 sec latency 63.689 ms 2024-04-25T13:56:14.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 21484 23.86 MB/sec warmup 56 sec latency 67.440 ms 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.221+0000 7f4fc319e640 1 -- 172.21.15.94:0/2147026491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc109a80 msgr2=0x7f4fbc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.221+0000 7f4fc319e640 1 --2- 172.21.15.94:0/2147026491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc109a80 0x7f4fbc10bea0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009ca0 tx=0x7f4fb402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 -- 172.21.15.94:0/2147026491 shutdown_connections 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 --2- 172.21.15.94:0/2147026491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc109a80 0x7f4fbc10bea0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 --2- 172.21.15.94:0/2147026491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc104d80 0x7f4fbc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 --2- 172.21.15.94:0/2147026491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc103b40 0x7f4fbc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 -- 172.21.15.94:0/2147026491 >> 172.21.15.94:0/2147026491 conn(0x7f4fbc0fed30 msgr2=0x7f4fbc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 -- 172.21.15.94:0/2147026491 shutdown_connections 2024-04-25T13:56:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 -- 172.21.15.94:0/2147026491 wait complete. 2024-04-25T13:56:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 Processor -- start 2024-04-25T13:56:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.222+0000 7f4fc319e640 1 -- start start 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc103b40 0x7f4fbc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 0x7f4fbc1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10f670 con 0x7f4fbc104d80 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10f4f0 con 0x7f4fbc103b40 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc319e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10f7f0 con 0x7f4fbc109a80 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46170/0 (socket says 172.21.15.94:46170) 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 -- 172.21.15.94:0/390129204 learned_addr learned my addr 172.21.15.94:0/390129204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc103b40 0x7f4fbc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc173b640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 0x7f4fbc1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 -- 172.21.15.94:0/390129204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 msgr2=0x7f4fbc1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 0x7f4fbc1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 -- 172.21.15.94:0/390129204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc103b40 msgr2=0x7f4fbc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc103b40 0x7f4fbc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 -- 172.21.15.94:0/390129204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc1a1d20 con 0x7f4fbc104d80 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fc173b640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 0x7f4fbc1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.223+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.224+0000 7f4fb9ffb640 1 -- 172.21.15.94:0/390129204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800cd80 con 0x7f4fbc104d80 2024-04-25T13:56:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.224+0000 7f4fb9ffb640 1 -- 172.21.15.94:0/390129204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa80108e0 con 0x7f4fbc104d80 2024-04-25T13:56:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.224+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc1a1fb0 con 0x7f4fbc104d80 2024-04-25T13:56:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.224+0000 7f4fb9ffb640 1 -- 172.21.15.94:0/390129204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8018980 con 0x7f4fbc104d80 2024-04-25T13:56:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.224+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1a24a0 con 0x7f4fbc104d80 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 Processor -- start 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 -- start start 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fddddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c0fd3c460 con 0x555c0fa361b0 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c0fcf6590 con 0x555c0fa0d2f0 2024-04-25T13:56:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fca04c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c0fd38420 con 0x555c0fa154d0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fc173b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fb9ffb640 1 -- 172.21.15.94:0/390129204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4fa8010430 con 0x7f4fbc104d80 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fc173b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46172/0 (socket says 172.21.15.94:46172) 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fc173b640 1 -- 172.21.15.94:0/254528809 learned_addr learned my addr 172.21.15.94:0/254528809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.227+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fddddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 -- 172.21.15.94:0/254528809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 msgr2=0x555c0fddddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fddddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 -- 172.21.15.94:0/254528809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa0d2f0 msgr2=0x555c0fdd6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 -- 172.21.15.94:0/254528809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0fd030d0 con 0x555c0fa361b0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fc173b640 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4031a90 tx=0x7f4fb402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27501162d60b1998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fb9ffb640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 0x7f4fac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fa6ffd640 1 -- 172.21.15.94:0/254528809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031650 con 0x555c0fa361b0 2024-04-25T13:56:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.228+0000 7f4fb9ffb640 1 -- 172.21.15.94:0/390129204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4fa80df040 con 0x7f4fbc104d80 2024-04-25T13:56:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.229+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 0x7f4fac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.229+0000 7f4fa6ffd640 1 -- 172.21.15.94:0/254528809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4036070 con 0x555c0fa361b0 2024-04-25T13:56:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.229+0000 7f4fa6ffd640 1 -- 172.21.15.94:0/254528809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004660 con 0x555c0fa361b0 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- 172.21.15.94:0/254528809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 msgr2=0x555c0fdd6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 0x7f4fac0b1ff0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x555c0fdd7d50 tx=0x7f4fb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4031a90 tx=0x7f4fb402f420 comp rx=0 tx=0).stop 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- 172.21.15.94:0/254528809 shutdown_connections 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fddddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 --2- 172.21.15.94:0/254528809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa361b0 0x555c0fdd6970 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- 172.21.15.94:0/254528809 >> 172.21.15.94:0/254528809 conn(0x555c0fa2a300 msgr2=0x555c0fdd2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- 172.21.15.94:0/254528809 shutdown_connections 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- 172.21.15.94:0/254528809 wait complete. 2024-04-25T13:56:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 -- start start 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.230+0000 7f4fca04c740 1 Processor -- start 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c0fdde7e0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c0fcc3b60 con 0x555c0fa154d0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c0fdde960 con 0x555c0fa361b0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fca04c740 10 client.0 _ll_register_callbacks cb 0x555c0fce82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fc173b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52668/0 (socket says 172.21.15.94:52668) 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46182/0 (socket says 172.21.15.94:46182) 2024-04-25T13:56:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fbbfff640 1 -- 172.21.15.94:0/2330164285 learned_addr learned my addr 172.21.15.94:0/2330164285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.231+0000 7f4fc173b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60010/0 (socket says 172.21.15.94:60010) 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 -- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 msgr2=0x555c0fde0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 -- 172.21.15.94:0/2330164285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 msgr2=0x555c0fde06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 -- 172.21.15.94:0/2330164285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0fa4e210 con 0x555c0fa361b0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fc173b640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009ca0 tx=0x7f4fb4009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384cef7aa4c9e79b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4f9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4049040 con 0x555c0fa361b0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4fb4031750 con 0x555c0fa361b0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb4036070 con 0x555c0fa361b0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fb40043e0 con 0x555c0fa361b0 2024-04-25T13:56:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fca04c740 10 client.29851 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.232+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c0fb1a620 con 0x555c0fa361b0 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4036070 con 0x555c0fa361b0 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4fb404c070 con 0x555c0fa361b0 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 10 client.29851 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 10 client.29851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 20 client.29851 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9f4d80 0x555c0fdedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 4 client.29851 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0fd4a070 con 0x555c0f9f4d80 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9f4d80 0x555c0fdedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9f4d80 0x555c0fdedfc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61906cd81b73aac2 server_cookie=95b2a4542fbd2306 in_seq=0 out_seq=0 2024-04-25T13:56:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.233+0000 7f4f9bfff640 10 client.29851 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4f9bfff640 1 -- 172.21.15.94:0/2330164285 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c0fd4a070 con 0x555c0f9f4d80 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4f9bfff640 10 client.29851 handle_command_reply: tid=0 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 client.29851 shutdown 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 msgr2=0x555c0fde7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009ca0 tx=0x7f4fb4009470 comp rx=0 tx=0).stop 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 shutdown_connections 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9f4d80 0x555c0fdedfc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa361b0 0x555c0fde7b70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa154d0 0x555c0fde0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2330164285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4f9bfff640 0 client.29851 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 >> 172.21.15.94:0/2330164285 conn(0x555c0fa2a300 msgr2=0x555c0fdd5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 shutdown_connections 2024-04-25T13:56:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 1 -- 172.21.15.94:0/2330164285 wait complete. 2024-04-25T13:56:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.237+0000 7f4fca04c740 20 client.29851 trim_cache size 0 max 16384 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 Processor -- start 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 -- start start 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fdd5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ab530 con 0x555c0fa0d2f0 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ad130 con 0x555c0fa361b0 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fca04c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1abc00 con 0x555c0fa154d0 2024-04-25T13:56:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46198/0 (socket says 172.21.15.94:46198) 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/2657834925 learned_addr learned my addr 172.21.15.94:0/2657834925 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc173b640 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fdd5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/2657834925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 msgr2=0x555c0fde7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/2657834925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 msgr2=0x555c0fdd5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fdd5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/2657834925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0fde1740 con 0x555c0fa0d2f0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.254+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x555c0fdd3090 tx=0x7f4fb001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be02ed05ec2565d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.255+0000 7f4fa67fc640 1 -- 172.21.15.94:0/2657834925 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb001ddb0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.255+0000 7f4fa67fc640 1 -- 172.21.15.94:0/2657834925 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb002a8d0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.255+0000 7f4fa67fc640 1 -- 172.21.15.94:0/2657834925 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002b3d0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.255+0000 7f4fca04c740 1 -- 172.21.15.94:0/2657834925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 msgr2=0x555c0fdd23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x555c0fdd3090 tx=0x7f4fb001b000 comp rx=0 tx=0).stop 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- 172.21.15.94:0/2657834925 shutdown_connections 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fdd5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- 172.21.15.94:0/2657834925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fdd23d0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- 172.21.15.94:0/2657834925 >> 172.21.15.94:0/2657834925 conn(0x555c0fa2a300 msgr2=0x555c0fdd1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- 172.21.15.94:0/2657834925 shutdown_connections 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- 172.21.15.94:0/2657834925 wait complete. 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- start start 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 Processor -- start 2024-04-25T13:56:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fde6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1aca60 con 0x555c0fa0d2f0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ac2d0 con 0x555c0fa361b0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1a9510 con 0x555c0fa154d0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.256+0000 7f4fca04c740 10 client.0 _ll_register_callbacks cb 0x7f4fb0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46204/0 (socket says 172.21.15.94:46204) 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/1195288806 learned_addr learned my addr 172.21.15.94:0/1195288806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc173b640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fde6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/1195288806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 msgr2=0x555c0fde56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 msgr2=0x555c0fde6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fde6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 -- 172.21.15.94:0/1195288806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0fde6bc0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc173b640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fde6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fc0f3a640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f4fb001efd0 tx=0x7f4fb001de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffa8491b041cd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4f9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002aa80 con 0x555c0fa0d2f0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f4fb002ac20 con 0x555c0fa0d2f0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.257+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb002a8d0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fb002ae20 con 0x555c0fa0d2f0 2024-04-25T13:56:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 10 client.20703 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c0fde7e50 con 0x555c0fa0d2f0 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb002a8d0 con 0x555c0fa0d2f0 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f4fb0004b80 con 0x555c0fa0d2f0 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 10 client.20703 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 10 client.20703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 20 client.20703 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9ca1c0 0x555c0fe58ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 4 client.20703 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c0fd43ee0 con 0x555c0f9ca1c0 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.258+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9ca1c0 0x555c0fe58ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.259+0000 7f4f9bfff640 10 client.20703 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.259+0000 7f4fbbfff640 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9ca1c0 0x555c0fe58ab0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe6721aa5033c5e server_cookie=96dd3962d052a5e9 in_seq=0 out_seq=0 2024-04-25T13:56:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:14 smithi137 ceph-mon[32568]: pgmap v782: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 557 op/s 2024-04-25T13:56:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:14 smithi094 ceph-mon[29195]: pgmap v782: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 557 op/s 2024-04-25T13:56:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:14 smithi138 ceph-mon[32977]: pgmap v782: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 557 op/s 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4f9bfff640 1 -- 172.21.15.94:0/1195288806 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c0fd43ee0 con 0x555c0f9ca1c0 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4f9bfff640 10 client.20703 handle_command_reply: tid=0 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 client.20703 shutdown 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 msgr2=0x555c0fde5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f4fb001efd0 tx=0x7f4fb001de60 comp rx=0 tx=0).stop 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 shutdown_connections 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555c0f9ca1c0 0x555c0fe58ab0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0fa361b0 0x555c0fde6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4f9bfff640 0 client.20703 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c0fa154d0 0x555c0fde56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 --2- 172.21.15.94:0/1195288806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0fa0d2f0 0x555c0fde5190 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 >> 172.21.15.94:0/1195288806 conn(0x555c0fa2a300 msgr2=0x555c0fdded30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 shutdown_connections 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 1 -- 172.21.15.94:0/1195288806 wait complete. 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.828+0000 7f4fca04c740 20 client.20703 trim_cache size 0 max 16384 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:56:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:56:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 msgr2=0x7f4fac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 0x7f4fac0b1ff0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x555c0fdd7d50 tx=0x7f4fb0005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 msgr2=0x7f4fbc19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 shutdown_connections 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fac0afae0 0x7f4fac0b1ff0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fbc109a80 0x7f4fbc1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fbc104d80 0x7f4fbc19a720 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 --2- 172.21.15.94:0/390129204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fbc103b40 0x7f4fbc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 >> 172.21.15.94:0/390129204 conn(0x7f4fbc0fed30 msgr2=0x7f4fbc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 shutdown_connections 2024-04-25T13:56:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:14.829+0000 7f4fc319e640 1 -- 172.21.15.94:0/390129204 wait complete. 2024-04-25T13:56:15.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 21676 23.59 MB/sec warmup 57 sec latency 316.592 ms 2024-04-25T13:56:16.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 21933 23.40 MB/sec warmup 58 sec latency 130.524 ms 2024-04-25T13:56:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:16 smithi137 ceph-mon[32568]: pgmap v783: 97 pgs: 97 active+clean; 245 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 652 op/s 2024-04-25T13:56:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:16 smithi094 ceph-mon[29195]: pgmap v783: 97 pgs: 97 active+clean; 245 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 652 op/s 2024-04-25T13:56:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:16 smithi138 ceph-mon[32977]: pgmap v783: 97 pgs: 97 active+clean; 245 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 652 op/s 2024-04-25T13:56:17.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 22184 23.29 MB/sec warmup 59 sec latency 340.169 ms 2024-04-25T13:56:18.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 22668 23.19 MB/sec warmup 60 sec latency 82.723 ms 2024-04-25T13:56:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:18 smithi137 ceph-mon[32568]: pgmap v784: 97 pgs: 97 active+clean; 241 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-04-25T13:56:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:18 smithi094 ceph-mon[29195]: pgmap v784: 97 pgs: 97 active+clean; 241 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-04-25T13:56:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:18 smithi138 ceph-mon[32977]: pgmap v784: 97 pgs: 97 active+clean; 241 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-04-25T13:56:19.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 23087 23.05 MB/sec warmup 61 sec latency 73.473 ms 2024-04-25T13:56:20.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 23619 23.29 MB/sec warmup 62 sec latency 62.330 ms 2024-04-25T13:56:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:20 smithi137 ceph-mon[32568]: pgmap v785: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 398 op/s 2024-04-25T13:56:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:20 smithi094 ceph-mon[29195]: pgmap v785: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 398 op/s 2024-04-25T13:56:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:20 smithi138 ceph-mon[32977]: pgmap v785: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 398 op/s 2024-04-25T13:56:21.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 24133 23.39 MB/sec warmup 63 sec latency 64.762 ms 2024-04-25T13:56:22.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 24438 23.31 MB/sec warmup 64 sec latency 209.089 ms 2024-04-25T13:56:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:22 smithi137 ceph-mon[32568]: pgmap v786: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-04-25T13:56:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:22 smithi094 ceph-mon[29195]: pgmap v786: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-04-25T13:56:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:22 smithi138 ceph-mon[32977]: pgmap v786: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-04-25T13:56:23.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 24785 23.30 MB/sec warmup 65 sec latency 58.196 ms 2024-04-25T13:56:24.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 25072 23.17 MB/sec warmup 66 sec latency 208.996 ms 2024-04-25T13:56:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:24 smithi137 ceph-mon[32568]: pgmap v787: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-04-25T13:56:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:24 smithi094 ceph-mon[29195]: pgmap v787: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-04-25T13:56:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:24 smithi138 ceph-mon[32977]: pgmap v787: 97 pgs: 97 active+clean; 235 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-04-25T13:56:25.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 25343 23.03 MB/sec warmup 67 sec latency 225.718 ms 2024-04-25T13:56:25.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:25.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:25.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:56:25.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:25.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:25.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:56:25.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:25.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:25.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:56:26.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 25713 23.02 MB/sec warmup 68 sec latency 66.312 ms 2024-04-25T13:56:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:26 smithi137 ceph-mon[32568]: pgmap v788: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-04-25T13:56:26.397 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:26 smithi138 ceph-mon[32977]: pgmap v788: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-04-25T13:56:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:26 smithi094 ceph-mon[29195]: pgmap v788: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-04-25T13:56:27.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 26007 22.80 MB/sec warmup 69 sec latency 252.829 ms 2024-04-25T13:56:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 -- 172.21.15.94:0/924536985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c108960 msgr2=0x7fc08c10ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 --2- 172.21.15.94:0/924536985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c108960 0x7fc08c10ad80 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc084009ca0 tx=0x7fc08402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 -- 172.21.15.94:0/924536985 shutdown_connections 2024-04-25T13:56:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 --2- 172.21.15.94:0/924536985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c108960 0x7fc08c10ad80 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 --2- 172.21.15.94:0/924536985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c103c60 0x7fc08c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 --2- 172.21.15.94:0/924536985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c102a20 0x7fc08c103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 -- 172.21.15.94:0/924536985 >> 172.21.15.94:0/924536985 conn(0x7fc08c0fdc10 msgr2=0x7fc08c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 -- 172.21.15.94:0/924536985 shutdown_connections 2024-04-25T13:56:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.941+0000 7fc094477640 1 -- 172.21.15.94:0/924536985 wait complete. 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 Processor -- start 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 -- start start 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c102a20 0x7fc08c06ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c108960 0x7fc08c1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10e530 con 0x7fc08c103c60 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10e3b0 con 0x7fc08c102a20 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc094477640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10e6b0 con 0x7fc08c108960 2024-04-25T13:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc091a12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc091a12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37446/0 (socket says 172.21.15.94:37446) 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc091a12640 1 -- 172.21.15.94:0/2406236234 learned_addr learned my addr 172.21.15.94:0/2406236234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc092a14640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c108960 0x7fc08c1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.942+0000 7fc092213640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c102a20 0x7fc08c06ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 -- 172.21.15.94:0/2406236234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c108960 msgr2=0x7fc08c1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c108960 0x7fc08c1a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 -- 172.21.15.94:0/2406236234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c102a20 msgr2=0x7fc08c06ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c102a20 0x7fc08c06ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 -- 172.21.15.94:0/2406236234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1a1cd0 con 0x7fc08c103c60 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc091a12640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9f0 tx=0x7fc07c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc0837fe640 1 -- 172.21.15.94:0/2406236234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cd80 con 0x7fc08c103c60 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c1a1fc0 con 0x7fc08c103c60 2024-04-25T13:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a2480 con 0x7fc08c103c60 2024-04-25T13:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc0837fe640 1 -- 172.21.15.94:0/2406236234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c0108e0 con 0x7fc08c103c60 2024-04-25T13:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.943+0000 7fc0837fe640 1 -- 172.21.15.94:0/2406236234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c026060 con 0x7fc08c103c60 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.946+0000 7fc09b405740 1 Processor -- start 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.946+0000 7fc0837fe640 1 -- 172.21.15.94:0/2406236234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc07c018820 con 0x7fc08c103c60 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 -- start start 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 0x55dc4e45f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0994d0 0x55dc4e466200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e3c0460 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e37a590 con 0x55dc4e0994d0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc09b405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e3bc420 con 0x55dc4e0912f0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37458/0 (socket says 172.21.15.94:37458) 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092213640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 0x55dc4e45f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 -- 172.21.15.94:0/2358939802 learned_addr learned my addr 172.21.15.94:0/2358939802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc091a12640 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0994d0 0x55dc4e466200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc0837fe640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 -- 172.21.15.94:0/2358939802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 msgr2=0x55dc4e45f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 0x55dc4e45f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 -- 172.21.15.94:0/2358939802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0994d0 msgr2=0x55dc4e466200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0994d0 0x55dc4e466200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092a14640 1 -- 172.21.15.94:0/2358939802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4e3870d0 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.947+0000 7fc092213640 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 0x55dc4e45f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc0837fe640 1 -- 172.21.15.94:0/2406236234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc07c0def40 con 0x7fc08c103c60 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc092a14640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc092a14640 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc084035000 tx=0x7fc08402f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2ef7e30e6f2bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc080bfe640 1 -- 172.21.15.94:0/2358939802 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084031790 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc080bfe640 1 -- 172.21.15.94:0/2358939802 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084036030 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.948+0000 7fc080bfe640 1 -- 172.21.15.94:0/2358939802 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402fa80 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 -- 172.21.15.94:0/2358939802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 msgr2=0x55dc4e45ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc084035000 tx=0x7fc08402f4c0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc092a14640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 0x7fc0780b1ff0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc08404a000 tx=0x7fc08404b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 -- 172.21.15.94:0/2358939802 shutdown_connections 2024-04-25T13:56:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0994d0 0x55dc4e466200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0912f0 0x55dc4e45f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 --2- 172.21.15.94:0/2358939802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e45ed80 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 -- 172.21.15.94:0/2358939802 >> 172.21.15.94:0/2358939802 conn(0x55dc4e0ae300 msgr2=0x55dc4e456300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 -- 172.21.15.94:0/2358939802 shutdown_connections 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.949+0000 7fc09b405740 1 -- 172.21.15.94:0/2358939802 wait complete. 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 -- start start 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 Processor -- start 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e468e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e46fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e466ad0 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e347b60 con 0x55dc4e0912f0 2024-04-25T13:56:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc4e466c50 con 0x55dc4e0994d0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc09b405740 10 client.0 _ll_register_callbacks cb 0x55dc4e36c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e46fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41506/0 (socket says 172.21.15.94:41506) 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 -- 172.21.15.94:0/1706858782 learned_addr learned my addr 172.21.15.94:0/1706858782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e46fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37468/0 (socket says 172.21.15.94:37468) 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc091a12640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e468e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 -- 172.21.15.94:0/1706858782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 msgr2=0x55dc4e468e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e468e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 -- 172.21.15.94:0/1706858782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 msgr2=0x55dc4e46fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e46fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.950+0000 7fc092213640 1 -- 172.21.15.94:0/1706858782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4e0d2210 con 0x55dc4e0912f0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc092213640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc08800b9d0 tx=0x7fc08800be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba3f50503c8191a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088011440 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc084036030 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc0880115e0 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0880117e0 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 10 client.30146 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc4e19e620 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084036030 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc08801b8a0 con 0x55dc4e0912f0 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 10 client.30146 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 10 client.30146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 20 client.30146 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e078d80 0x55dc4e476830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 4 client.30146 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.951+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4e3ce070 con 0x55dc4e078d80 2024-04-25T13:56:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.952+0000 7fc091a12640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e078d80 0x55dc4e476830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.952+0000 7fc075ffb640 10 client.30146 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.952+0000 7fc091a12640 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e078d80 0x55dc4e476830 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6c4c5cf8656d81 server_cookie=ae5a6cec2be30040 in_seq=0 out_seq=0 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.956+0000 7fc075ffb640 1 -- 172.21.15.94:0/1706858782 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc4e3ce070 con 0x55dc4e078d80 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.956+0000 7fc075ffb640 10 client.30146 handle_command_reply: tid=0 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 client.30146 shutdown 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 msgr2=0x55dc4e468900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc08800b9d0 tx=0x7fc08800be90 comp rx=0 tx=0).stop 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 shutdown_connections 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e078d80 0x55dc4e476830 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0ba1b0 0x55dc4e46fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e468e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc075ffb640 0 client.30146 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 --2- 172.21.15.94:0/1706858782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e468900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 >> 172.21.15.94:0/1706858782 conn(0x55dc4e0ae300 msgr2=0x55dc4e455250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 shutdown_connections 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 1 -- 172.21.15.94:0/1706858782 wait complete. 2024-04-25T13:56:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.957+0000 7fc09b405740 20 client.30146 trim_cache size 0 max 16384 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 Processor -- start 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 -- start start 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e4684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc07000e730 con 0x55dc4e078d80 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc070010330 con 0x55dc4e0912f0 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc09b405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc07000ee00 con 0x55dc4e0994d0 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37484/0 (socket says 172.21.15.94:37484) 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 -- 172.21.15.94:0/2917310935 learned_addr learned my addr 172.21.15.94:0/2917310935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc091a12640 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092a14640 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e4684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 -- 172.21.15.94:0/2917310935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 msgr2=0x55dc4e4684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e4684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 -- 172.21.15.94:0/2917310935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 msgr2=0x55dc4e46fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 -- 172.21.15.94:0/2917310935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4e4638a0 con 0x55dc4e078d80 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.973+0000 7fc092213640 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc088001eb0 tx=0x7fc088004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f2cb8ac48b695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.974+0000 7fc077fff640 1 -- 172.21.15.94:0/2917310935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088028070 con 0x55dc4e078d80 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.974+0000 7fc077fff640 1 -- 172.21.15.94:0/2917310935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc088011e30 con 0x55dc4e078d80 2024-04-25T13:56:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.974+0000 7fc077fff640 1 -- 172.21.15.94:0/2917310935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880055c0 con 0x55dc4e078d80 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- 172.21.15.94:0/2917310935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 msgr2=0x55dc4e4563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc088001eb0 tx=0x7fc088004590 comp rx=0 tx=0).stop 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- 172.21.15.94:0/2917310935 shutdown_connections 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e0994d0 0x55dc4e4684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- 172.21.15.94:0/2917310935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e078d80 0x55dc4e4563d0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- 172.21.15.94:0/2917310935 >> 172.21.15.94:0/2917310935 conn(0x55dc4e0ae300 msgr2=0x55dc4e455cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- 172.21.15.94:0/2917310935 shutdown_connections 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- 172.21.15.94:0/2917310935 wait complete. 2024-04-25T13:56:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- start start 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 Processor -- start 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e078d80 0x55dc4e46b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc07000fc60 con 0x55dc4e0994d0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc07000f4d0 con 0x55dc4e0912f0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc07000c710 con 0x55dc4e078d80 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc09b405740 10 client.0 _ll_register_callbacks cb 0x7fc0880042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.975+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37498/0 (socket says 172.21.15.94:37498) 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 -- 172.21.15.94:0/4067513309 learned_addr learned my addr 172.21.15.94:0/4067513309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092213640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e078d80 0x55dc4e46b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc091a12640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 -- 172.21.15.94:0/4067513309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e078d80 msgr2=0x55dc4e46b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e078d80 0x55dc4e46b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 -- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 msgr2=0x55dc4e46bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 -- 172.21.15.94:0/4067513309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4e470200 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc091a12640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc092a14640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc084009800 tx=0x7fc084031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39fb92607edb29b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08405f070 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fc08402f870 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc088011e30 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc08406b050 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc09b405740 10 client.20721 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.976+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc4e4deba0 con 0x55dc4e0994d0 2024-04-25T13:56:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc088011e30 con 0x55dc4e0994d0 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fc0840305c0 con 0x55dc4e0994d0 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 10 client.20721 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 10 client.20721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 20 client.20721 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e0ba1b0 0x55dc4e4e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 4 client.20721 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc4e3c7ee0 con 0x55dc4e0ba1b0 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc092213640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e0ba1b0 0x55dc4e4e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc075ffb640 10 client.20721 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:27.977+0000 7fc092213640 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e0ba1b0 0x55dc4e4e1290 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa376e79919d683 server_cookie=2f2599dc5e9b02e6 in_seq=0 out_seq=0 2024-04-25T13:56:28.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 26408 22.70 MB/sec warmup 70 sec latency 140.994 ms 2024-04-25T13:56:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:28 smithi137 ceph-mon[32568]: pgmap v789: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc075ffb640 1 -- 172.21.15.94:0/4067513309 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc4e3c7ee0 con 0x55dc4e0ba1b0 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc075ffb640 10 client.20721 handle_command_reply: tid=0 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc09b405740 1 client.20721 shutdown 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 msgr2=0x55dc4e46e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc084009800 tx=0x7fc084031880 comp rx=0 tx=0).stop 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 shutdown_connections 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.463+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dc4e0ba1b0 0x55dc4e4e1290 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc4e0994d0 0x55dc4e46e9f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc4e0912f0 0x55dc4e46bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 --2- 172.21.15.94:0/4067513309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc4e078d80 0x55dc4e46b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc075ffb640 0 client.20721 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 >> 172.21.15.94:0/4067513309 conn(0x55dc4e0ae300 msgr2=0x55dc4e468440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 shutdown_connections 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 1 -- 172.21.15.94:0/4067513309 wait complete. 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc09b405740 20 client.20721 trim_cache size 0 max 16384 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:56:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:56:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:56:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 msgr2=0x7fc0780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.464+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 0x7fc0780b1ff0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc08404a000 tx=0x7fc08404b000 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 msgr2=0x7fc08c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9f0 tx=0x7fc07c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 shutdown_connections 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc08c108960 0x7fc08c1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c103c60 0x7fc08c19ef40 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 --2- 172.21.15.94:0/2406236234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc08c102a20 0x7fc08c06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 >> 172.21.15.94:0/2406236234 conn(0x7fc08c0fdc10 msgr2=0x7fc08c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 shutdown_connections 2024-04-25T13:56:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:28.465+0000 7fc094477640 1 -- 172.21.15.94:0/2406236234 wait complete. 2024-04-25T13:56:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:28 smithi094 ceph-mon[29195]: pgmap v789: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:56:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:28 smithi138 ceph-mon[32977]: pgmap v789: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:56:29.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 26783 22.66 MB/sec warmup 71 sec latency 420.103 ms 2024-04-25T13:56:30.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 27291 22.89 MB/sec warmup 72 sec latency 48.855 ms 2024-04-25T13:56:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:30 smithi137 ceph-mon[32568]: pgmap v790: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-04-25T13:56:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:30 smithi094 ceph-mon[29195]: pgmap v790: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-04-25T13:56:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:30 smithi138 ceph-mon[32977]: pgmap v790: 97 pgs: 97 active+clean; 240 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2024-04-25T13:56:31.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 27765 22.87 MB/sec warmup 73 sec latency 82.433 ms 2024-04-25T13:56:32.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 28018 22.77 MB/sec warmup 74 sec latency 207.477 ms 2024-04-25T13:56:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:32 smithi137 ceph-mon[32568]: pgmap v791: 97 pgs: 97 active+clean; 235 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-04-25T13:56:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:32 smithi094 ceph-mon[29195]: pgmap v791: 97 pgs: 97 active+clean; 235 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-04-25T13:56:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:32 smithi138 ceph-mon[32977]: pgmap v791: 97 pgs: 97 active+clean; 235 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-04-25T13:56:32.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:56:32.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 -- 172.21.15.94:0/95818231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810101740 msgr2=0x7f5810101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 --2- 172.21.15.94:0/95818231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810101740 0x7f5810101bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5808009be0 tx=0x7f580802a6c0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 -- 172.21.15.94:0/95818231 shutdown_connections 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 --2- 172.21.15.94:0/95818231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810102240 0x7f581010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 --2- 172.21.15.94:0/95818231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810101740 0x7f5810101bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 --2- 172.21.15.94:0/95818231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 -- 172.21.15.94:0/95818231 >> 172.21.15.94:0/95818231 conn(0x7f58100fd340 msgr2=0x7f58100ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 -- 172.21.15.94:0/95818231 shutdown_connections 2024-04-25T13:56:32.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.862+0000 7f5816a51640 1 -- 172.21.15.94:0/95818231 wait complete. 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.863+0000 7f5816a51640 1 Processor -- start 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.863+0000 7f5816a51640 1 -- start start 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 0x7f581019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 0x7f581019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5810111800 con 0x7f5810101740 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5810111680 con 0x7f5810102240 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5816a51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5810111980 con 0x7f5810105090 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 0x7f581019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5814fee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5814fee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40084/0 (socket says 172.21.15.94:40084) 2024-04-25T13:56:32.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 0x7f581019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41542/0 (socket says 172.21.15.94:41542) 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f5814fee640 1 -- 172.21.15.94:0/1660608720 learned_addr learned my addr 172.21.15.94:0/1660608720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 0x7f581019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 -- 172.21.15.94:0/1660608720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 msgr2=0x7f58101a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 -- 172.21.15.94:0/1660608720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 msgr2=0x7f581019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 0x7f581019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.864+0000 7f580ffff640 1 -- 172.21.15.94:0/1660608720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58101a3fa0 con 0x7f5810101740 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f580ffff640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 0x7f581019c460 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5810069dc0 tx=0x7f580000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f580d7fa640 1 -- 172.21.15.94:0/1660608720 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800009e60 con 0x7f5810101740 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101a4290 con 0x7f5810101740 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101a4830 con 0x7f5810101740 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f580d7fa640 1 -- 172.21.15.94:0/1660608720 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580000ccc0 con 0x7f5810101740 2024-04-25T13:56:32.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.865+0000 7f580d7fa640 1 -- 172.21.15.94:0/1660608720 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800003e20 con 0x7f5810101740 2024-04-25T13:56:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 Processor -- start 2024-04-25T13:56:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 -- start start 2024-04-25T13:56:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555baf620d80 con 0x555baf2f96b0 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555baf5da8b0 con 0x555baf31a390 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.868+0000 7f581d9e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555baf61cd40 con 0x555baf2f14d0 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f5814fee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f5814fee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41546/0 (socket says 172.21.15.94:41546) 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f5814fee640 1 -- 172.21.15.94:0/674686316 learned_addr learned my addr 172.21.15.94:0/674686316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 -- 172.21.15.94:0/674686316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 msgr2=0x555baf6bbfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6bbfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 -- 172.21.15.94:0/674686316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 msgr2=0x555baf6bba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 -- 172.21.15.94:0/674686316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baf5aba40 con 0x555baf2f96b0 2024-04-25T13:56:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f5814fee640 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580d7fa640 1 -- 172.21.15.94:0/1660608720 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f58000035f0 con 0x7f5810101740 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.869+0000 7f580f7fe640 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6bcc70 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f580802acb0 tx=0x7f5808005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5645664c7549e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f58067fc640 1 -- 172.21.15.94:0/674686316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808041030 con 0x555baf2f96b0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f58067fc640 1 -- 172.21.15.94:0/674686316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58080305e0 con 0x555baf2f96b0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f58067fc640 1 -- 172.21.15.94:0/674686316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808036040 con 0x555baf2f96b0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f580d7fa640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 0x7f57fc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f5814fee640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 0x7f57fc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.870+0000 7f580d7fa640 1 -- 172.21.15.94:0/1660608720 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f58000deec0 con 0x7f5810101740 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f5814fee640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 0x7f57fc0b2080 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x555baf6bcab0 tx=0x7f57f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- 172.21.15.94:0/674686316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 msgr2=0x555baf6bcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6bcc70 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f580802acb0 tx=0x7f5808005c50 comp rx=0 tx=0).stop 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- 172.21.15.94:0/674686316 shutdown_connections 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6bcc70 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 --2- 172.21.15.94:0/674686316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf31a390 0x555baf6bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- 172.21.15.94:0/674686316 >> 172.21.15.94:0/674686316 conn(0x555baf30e4e0 msgr2=0x555baf6b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- 172.21.15.94:0/674686316 shutdown_connections 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- 172.21.15.94:0/674686316 wait complete. 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.871+0000 7f581d9e9740 1 -- start start 2024-04-25T13:56:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 Processor -- start 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf6ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555baf6bd720 con 0x555baf2f14d0 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555baf5a7d10 con 0x555baf2f96b0 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555baf6bd8a0 con 0x555baf31a390 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f581d9e9740 10 client.0 _ll_register_callbacks cb 0x555baf5d4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37544/0 (socket says 172.21.15.94:37544) 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 -- 172.21.15.94:0/1439993789 learned_addr learned my addr 172.21.15.94:0/1439993789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:32.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580f7fe640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f5814fee640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf6ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 -- 172.21.15.94:0/1439993789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 msgr2=0x555baf6ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf6ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 -- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 msgr2=0x555baf6c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.872+0000 7f580ffff640 1 -- 172.21.15.94:0/1439993789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baf3fc7d0 con 0x555baf2f14d0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f580ffff640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f58000a4570 tx=0x7f58000a6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce0121f420f3121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000ab6b0 con 0x555baf2f14d0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f58000a6770 con 0x555baf2f14d0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58080305e0 con 0x555baf2f14d0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f58000b0160 con 0x555baf2f14d0 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f581d9e9740 10 client.20733 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:32.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555baf3323f0 con 0x555baf2f14d0 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58080305e0 con 0x555baf2f14d0 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.873+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f58000a6060 con 0x555baf2f14d0 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 10 client.20733 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 10 client.20733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 20 client.20733 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2d8f60 0x555baf6d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 4 client.20733 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555baf62e990 con 0x555baf2d8f60 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f580f7fe640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2d8f60 0x555baf6d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f57ef7fe640 10 client.20733 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.874+0000 7f580f7fe640 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2d8f60 0x555baf6d32d0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b137c952c697ba server_cookie=ab36998eee5f4d26 in_seq=0 out_seq=0 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f57ef7fe640 1 -- 172.21.15.94:0/1439993789 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555baf62e990 con 0x555baf2d8f60 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f57ef7fe640 10 client.20733 handle_command_reply: tid=0 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 client.20733 shutdown 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 msgr2=0x555baf6c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f58000a4570 tx=0x7f58000a6ed0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 shutdown_connections 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2d8f60 0x555baf6d32d0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf6ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 --2- 172.21.15.94:0/1439993789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f14d0 0x555baf6c5960 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f57ef7fe640 0 client.20733 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:32.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 >> 172.21.15.94:0/1439993789 conn(0x555baf30e4e0 msgr2=0x555baf6b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 shutdown_connections 2024-04-25T13:56:32.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.877+0000 7f581d9e9740 1 -- 172.21.15.94:0/1439993789 wait complete. 2024-04-25T13:56:32.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.878+0000 7f581d9e9740 20 client.20733 trim_cache size 0 max 16384 2024-04-25T13:56:32.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 Processor -- start 2024-04-25T13:56:32.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 -- start start 2024-04-25T13:56:32.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000e730 con 0x555baf31a390 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0010330 con 0x555baf2f96b0 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.897+0000 7f581d9e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000ee00 con 0x555baf2f14d0 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37556/0 (socket says 172.21.15.94:37556) 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 -- 172.21.15.94:0/730441812 learned_addr learned my addr 172.21.15.94:0/730441812 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f580f7fe640 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f580ffff640 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 -- 172.21.15.94:0/730441812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 msgr2=0x555baf6b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 -- 172.21.15.94:0/730441812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 msgr2=0x555baf6c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 -- 172.21.15.94:0/730441812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baf2f20f0 con 0x555baf31a390 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5814fee640 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f57f8004290 tx=0x7f57f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ce2ef89ddeaf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5805ffb640 1 -- 172.21.15.94:0/730441812 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8004380 con 0x555baf31a390 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5805ffb640 1 -- 172.21.15.94:0/730441812 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57f80054d0 con 0x555baf31a390 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.898+0000 7f5805ffb640 1 -- 172.21.15.94:0/730441812 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f800f740 con 0x555baf31a390 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 -- 172.21.15.94:0/730441812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 msgr2=0x555baf6c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f57f8004290 tx=0x7f57f801b000 comp rx=0 tx=0).stop 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 -- 172.21.15.94:0/730441812 shutdown_connections 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf31a390 0x555baf6c4440 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f96b0 0x555baf6c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 --2- 172.21.15.94:0/730441812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf2f14d0 0x555baf6b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 -- 172.21.15.94:0/730441812 >> 172.21.15.94:0/730441812 conn(0x555baf30e4e0 msgr2=0x555baf6b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 -- 172.21.15.94:0/730441812 shutdown_connections 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.899+0000 7f581d9e9740 1 -- 172.21.15.94:0/730441812 wait complete. 2024-04-25T13:56:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 -- start start 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 Processor -- start 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 0x555baf6c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf739670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000fc60 con 0x555baf2f96b0 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000f4d0 con 0x555baf2f14d0 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57f000c710 con 0x555baf31a390 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f581d9e9740 10 client.0 _ll_register_callbacks cb 0x7f58000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37564/0 (socket says 172.21.15.94:37564) 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 -- 172.21.15.94:0/156714021 learned_addr learned my addr 172.21.15.94:0/156714021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580ffff640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 0x555baf6c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f5814fee640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf739670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 -- 172.21.15.94:0/156714021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 msgr2=0x555baf739670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf739670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 -- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 msgr2=0x555baf6c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 0x555baf6c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580f7fe640 1 -- 172.21.15.94:0/156714021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baf5e14b0 con 0x555baf2f96b0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.900+0000 7f580ffff640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 0x555baf6c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f580f7fe640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f5808030800 tx=0x7f580802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6123aae4d4c3ff6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808053070 con 0x555baf2f96b0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f5808030c70 con 0x555baf2f96b0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f80054d0 con 0x555baf2f96b0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5808036040 con 0x555baf2f96b0 2024-04-25T13:56:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 10 client.20739 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555baf73a910 con 0x555baf2f96b0 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57f80054d0 con 0x555baf2f96b0 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f5808056030 con 0x555baf2f96b0 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 10 client.20739 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 10 client.20739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 20 client.20739 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2ae3a0 0x555baf73cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 4 client.20739 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.901+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555baf628800 con 0x555baf2ae3a0 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.902+0000 7f580ffff640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2ae3a0 0x555baf73cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.902+0000 7f57ef7fe640 10 client.20739 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.902+0000 7f580ffff640 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2ae3a0 0x555baf73cf90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cebe3d939c83abb1 server_cookie=45dc9397114a3ce8 in_seq=0 out_seq=0 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f57ef7fe640 1 -- 172.21.15.94:0/156714021 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555baf628800 con 0x555baf2ae3a0 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f57ef7fe640 10 client.20739 handle_command_reply: tid=0 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 client.20739 shutdown 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 msgr2=0x555baf6c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f5808030800 tx=0x7f580802a6c0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 shutdown_connections 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555baf2ae3a0 0x555baf73cf90 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555baf31a390 0x555baf739670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555baf2f96b0 0x555baf6c9600 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 --2- 172.21.15.94:0/156714021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555baf2f14d0 0x555baf6c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f57ef7fe640 0 client.20739 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 >> 172.21.15.94:0/156714021 conn(0x555baf30e4e0 msgr2=0x555baf6c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 shutdown_connections 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 1 -- 172.21.15.94:0/156714021 wait complete. 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.908+0000 7f581d9e9740 20 client.20739 trim_cache size 0 max 16384 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:56:32.910 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 msgr2=0x7f57fc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 0x7f57fc0b2080 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x555baf6bcab0 tx=0x7f57f8009260 comp rx=0 tx=0).stop 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 msgr2=0x7f581019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 0x7f581019c460 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5810069dc0 tx=0x7f580000ed90 comp rx=0 tx=0).stop 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 shutdown_connections 2024-04-25T13:56:32.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57fc0afb70 0x7f57fc0b2080 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5810105090 0x7f58101a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5810102240 0x7f581019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 --2- 172.21.15.94:0/1660608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5810101740 0x7f581019c460 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 >> 172.21.15.94:0/1660608720 conn(0x7f58100fd340 msgr2=0x7f5810105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 shutdown_connections 2024-04-25T13:56:32.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:32.909+0000 7f5816a51640 1 -- 172.21.15.94:0/1660608720 wait complete. 2024-04-25T13:56:32.920 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:56:32.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/429648595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64101730 msgr2=0x7f2a64101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/429648595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64101730 0x7f2a64101ba0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a5002a410 comp rx=0 tx=0).stop 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/429648595 shutdown_connections 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/429648595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64102230 0x7f2a6410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/429648595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64101730 0x7f2a64101ba0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/429648595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64105080 0x7f2a64105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/429648595 >> 172.21.15.94:0/429648595 conn(0x7f2a640fd330 msgr2=0x7f2a640ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/429648595 shutdown_connections 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.035+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/429648595 wait complete. 2024-04-25T13:56:33.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 Processor -- start 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 -- start start 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 0x7f2a641041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64102230 0x7f2a641028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410f5f0 con 0x7f2a64105080 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410f470 con 0x7f2a64102230 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6aa9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6410f770 con 0x7f2a64101730 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6903c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6903c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37572/0 (socket says 172.21.15.94:37572) 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6903c640 1 -- 172.21.15.94:0/1120957283 learned_addr learned my addr 172.21.15.94:0/1120957283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a63fff640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64102230 0x7f2a641028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.036+0000 7f2a6883b640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 0x7f2a641041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 -- 172.21.15.94:0/1120957283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 msgr2=0x7f2a641041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 0x7f2a641041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 -- 172.21.15.94:0/1120957283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64102230 msgr2=0x7f2a641028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64102230 0x7f2a641028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 -- 172.21.15.94:0/1120957283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a64103540 con 0x7f2a64105080 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6883b640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 0x7f2a641041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6903c640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009fd0 tx=0x7f2a5c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x7f2a64105080 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c003e20 con 0x7f2a64105080 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a4340 con 0x7f2a64105080 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c010460 con 0x7f2a64105080 2024-04-25T13:56:33.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.037+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a4750 con 0x7f2a64105080 2024-04-25T13:56:33.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.038+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a64069310 con 0x7f2a64105080 2024-04-25T13:56:33.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.039+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2a5c003970 con 0x7f2a64105080 2024-04-25T13:56:33.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.040+0000 7f2a61ffb640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 0x7f2a540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.040+0000 7f2a6883b640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 0x7f2a540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.040+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f2a5c0dea70 con 0x7f2a64105080 2024-04-25T13:56:33.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.041+0000 7f2a6883b640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 0x7f2a540b2000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2a64069bf0 tx=0x7f2a58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.045+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a5c0a75b0 con 0x7f2a64105080 2024-04-25T13:56:33.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 28353 22.74 MB/sec warmup 75 sec latency 58.431 ms 2024-04-25T13:56:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.277+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a640637a0 con 0x7f2a64105080 2024-04-25T13:56:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.278+0000 7f2a61ffb640 1 -- 172.21.15.94:0/1120957283 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11948 (secure 0 0 0) 0x7f2a5c0a6eb0 con 0x7f2a64105080 2024-04-25T13:56:33.278 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:56:33.278 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:55:20.694859+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28588":{"gid":28588,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1434478747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1434478747},{"type":"v1","addr":"172.21.15.138:6835","nonce":1434478747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:56:33.278 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 32 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 msgr2=0x7f2a540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 0x7f2a540b2000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2a64069bf0 tx=0x7f2a58005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 msgr2=0x7f2a64102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009fd0 tx=0x7f2a5c00b260 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 shutdown_connections 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2a540afaf0 0x7f2a540b2000 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a64105080 0x7f2a64102e10 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a64102230 0x7f2a641028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 --2- 172.21.15.94:0/1120957283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a64101730 0x7f2a641041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 >> 172.21.15.94:0/1120957283 conn(0x7f2a640fd330 msgr2=0x7f2a64105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 shutdown_connections 2024-04-25T13:56:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.280+0000 7f2a6aa9f640 1 -- 172.21.15.94:0/1120957283 wait complete. 2024-04-25T13:56:33.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:56:33.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 -- 172.21.15.94:0/380445769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc108920 msgr2=0x7f0ccc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/380445769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc108920 0x7f0ccc10ad40 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009ca0 tx=0x7f0cc402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 -- 172.21.15.94:0/380445769 shutdown_connections 2024-04-25T13:56:33.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/380445769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc108920 0x7f0ccc10ad40 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/380445769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc103c20 0x7f0ccc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/380445769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 -- 172.21.15.94:0/380445769 >> 172.21.15.94:0/380445769 conn(0x7f0ccc0fdbd0 msgr2=0x7f0ccc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 -- 172.21.15.94:0/380445769 shutdown_connections 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.410+0000 7f0cd1b08640 1 -- 172.21.15.94:0/380445769 wait complete. 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 Processor -- start 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 -- start start 2024-04-25T13:56:33.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc103c20 0x7f0ccc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc108920 0x7f0ccc1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f610 con 0x7f0ccc1029e0 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f490 con 0x7f0ccc108920 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0cd1b08640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f790 con 0x7f0ccc103c20 2024-04-25T13:56:33.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37590/0 (socket says 172.21.15.94:37590) 2024-04-25T13:56:33.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0ccb7fe640 1 -- 172.21.15.94:0/230590060 learned_addr learned my addr 172.21.15.94:0/230590060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc108920 0x7f0ccc1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.411+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc103c20 0x7f0ccc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 -- 172.21.15.94:0/230590060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc103c20 msgr2=0x7f0ccc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc103c20 0x7f0ccc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 -- 172.21.15.94:0/230590060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc108920 msgr2=0x7f0ccc1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc108920 0x7f0ccc1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 -- 172.21.15.94:0/230590060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a2020 con 0x7f0ccc1029e0 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005d00 tx=0x7f0cbc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0cc8ff9640 1 -- 172.21.15.94:0/230590060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc017070 con 0x7f0ccc1029e0 2024-04-25T13:56:33.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0cc8ff9640 1 -- 172.21.15.94:0/230590060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cbc009c30 con 0x7f0ccc1029e0 2024-04-25T13:56:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0cc8ff9640 1 -- 172.21.15.94:0/230590060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00f620 con 0x7f0ccc1029e0 2024-04-25T13:56:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc10b890 con 0x7f0ccc1029e0 2024-04-25T13:56:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.412+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc10bde0 con 0x7f0ccc1029e0 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.414+0000 7f0cc8ff9640 1 -- 172.21.15.94:0/230590060 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0cbc01a070 con 0x7f0ccc1029e0 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.414+0000 7f0cd89b7740 1 Processor -- start 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 -- start start 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba2f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563cba257d80 con 0x563cb9f51390 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563cba211a20 con 0x563cb9f306b0 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563cba253d40 con 0x563cb9f284d0 2024-04-25T13:56:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37592/0 (socket says 172.21.15.94:37592) 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba2f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 -- 172.21.15.94:0/3442736909 learned_addr learned my addr 172.21.15.94:0/3442736909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cc8ff9640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 0x7f0cb40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 -- 172.21.15.94:0/3442736909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 msgr2=0x563cba2f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 -- 172.21.15.94:0/3442736909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 msgr2=0x563cba2f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba2f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 -- 172.21.15.94:0/3442736909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cba21cc30 con 0x563cb9f51390 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba2f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0cc8ff9640 1 -- 172.21.15.94:0/230590060 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f0cbc0de5f0 con 0x7f0ccc1029e0 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.415+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 0x7f0cb40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.416+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4035000 tx=0x7f0cc402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7cb8f742b48f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.416+0000 7f0cb1ffb640 1 -- 172.21.15.94:0/3442736909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4003970 con 0x563cb9f51390 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.416+0000 7f0cb1ffb640 1 -- 172.21.15.94:0/3442736909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc4036070 con 0x563cb9f51390 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.416+0000 7f0cb1ffb640 1 -- 172.21.15.94:0/3442736909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc402fa80 con 0x563cb9f51390 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- 172.21.15.94:0/3442736909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 msgr2=0x563cba2f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 0x7f0cb40b2080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0cc404f000 tx=0x7f0cc4052000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4035000 tx=0x7f0cc402f450 comp rx=0 tx=0).stop 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- 172.21.15.94:0/3442736909 shutdown_connections 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba2f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/3442736909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba2f3810 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- 172.21.15.94:0/3442736909 >> 172.21.15.94:0/3442736909 conn(0x563cb9f454e0 msgr2=0x563cba2ef020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- 172.21.15.94:0/3442736909 shutdown_connections 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- 172.21.15.94:0/3442736909 wait complete. 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 -- start start 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.417+0000 7f0cd89b7740 1 Processor -- start 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f51390 0x563cba3049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563cba2f54d0 con 0x563cb9f306b0 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563cba1dedc0 con 0x563cb9f51390 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563cba2f5650 con 0x563cb9f284d0 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 10 client.0 _ll_register_callbacks cb 0x563cba1fa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37600/0 (socket says 172.21.15.94:37600) 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 -- 172.21.15.94:0/892214950 learned_addr learned my addr 172.21.15.94:0/892214950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f51390 0x563cba3049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 -- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 msgr2=0x563cba2fd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2fd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 -- 172.21.15.94:0/892214950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f51390 msgr2=0x563cba3049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f51390 0x563cba3049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 -- 172.21.15.94:0/892214950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb9f693f0 con 0x563cb9f306b0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0012720 tx=0x7f0cc0012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44f7fb0cd5648e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0caaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004030 con 0x563cb9f306b0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f0cc00041d0 con 0x563cb9f306b0 2024-04-25T13:56:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.418+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc4036070 con 0x563cb9f306b0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cc0004410 con 0x563cb9f306b0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 10 client.20757 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563cba0337d0 con 0x563cb9f306b0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc4036070 con 0x563cb9f306b0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f0cc0003bc0 con 0x563cb9f306b0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 10 client.20757 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 10 client.20757 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 20 client.20757 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9f0ff60 0x563cba30b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 4 client.20757 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cba265990 con 0x563cb9f0ff60 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.419+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9f0ff60 0x563cba30b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0caaffd640 10 client.20757 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9f0ff60 0x563cba30b2c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd6e99bb9aa717f8 server_cookie=b758393f24ced41d in_seq=0 out_seq=0 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0caaffd640 1 -- 172.21.15.94:0/892214950 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cba265990 con 0x563cb9f0ff60 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0caaffd640 10 client.20757 handle_command_reply: tid=0 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0cd89b7740 1 client.20757 shutdown 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 msgr2=0x563cba2fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0012720 tx=0x7f0cc0012be0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 shutdown_connections 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.420+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9f0ff60 0x563cba30b2c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f51390 0x563cba3049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2fda70 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/892214950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba2fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0caaffd640 0 client.20757 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 >> 172.21.15.94:0/892214950 conn(0x563cb9f454e0 msgr2=0x563cba2f1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 shutdown_connections 2024-04-25T13:56:33.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 1 -- 172.21.15.94:0/892214950 wait complete. 2024-04-25T13:56:33.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.421+0000 7f0cd89b7740 20 client.20757 trim_cache size 0 max 16384 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.434+0000 7f0cd89b7740 1 Processor -- start 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 -- start start 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f284d0 0x563cba2eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f51390 0x563cba2f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a91c0 con 0x563cb9f306b0 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aadc0 con 0x563cb9f284d0 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a9890 con 0x563cb9f51390 2024-04-25T13:56:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37602/0 (socket says 172.21.15.94:37602) 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 -- 172.21.15.94:0/2637847654 learned_addr learned my addr 172.21.15.94:0/2637847654 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f284d0 0x563cba2eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f51390 0x563cba2f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 -- 172.21.15.94:0/2637847654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f51390 msgr2=0x563cba2f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f51390 0x563cba2f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 -- 172.21.15.94:0/2637847654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f284d0 msgr2=0x563cba2eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f284d0 0x563cba2eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 -- 172.21.15.94:0/2637847654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cba2e92b0 con 0x563cb9f306b0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.435+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0026610 tx=0x7f0cc0005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa2a8cfaf69022a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.436+0000 7f0cb17fa640 1 -- 172.21.15.94:0/2637847654 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0026690 con 0x563cb9f306b0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.436+0000 7f0cb17fa640 1 -- 172.21.15.94:0/2637847654 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc0026830 con 0x563cb9f306b0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.436+0000 7f0cb17fa640 1 -- 172.21.15.94:0/2637847654 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00035f0 con 0x563cb9f306b0 2024-04-25T13:56:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- 172.21.15.94:0/2637847654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 msgr2=0x563cba2eef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0026610 tx=0x7f0cc0005870 comp rx=0 tx=0).stop 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- 172.21.15.94:0/2637847654 shutdown_connections 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f51390 0x563cba2f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f306b0 0x563cba2eef90 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/2637847654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f284d0 0x563cba2eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- 172.21.15.94:0/2637847654 >> 172.21.15.94:0/2637847654 conn(0x563cb9f454e0 msgr2=0x563cba2f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- 172.21.15.94:0/2637847654 shutdown_connections 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- 172.21.15.94:0/2637847654 wait complete. 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 -- start start 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.437+0000 7f0cd89b7740 1 Processor -- start 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba300d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba301260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aa6f0 con 0x563cb9f51390 2024-04-25T13:56:33.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a9f60 con 0x563cb9f306b0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a71a0 con 0x563cb9f284d0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 10 client.0 _ll_register_callbacks cb 0x7f0cc0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37614/0 (socket says 172.21.15.94:37614) 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 -- 172.21.15.94:0/360783053 learned_addr learned my addr 172.21.15.94:0/360783053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba300d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccaffd640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba301260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 -- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 msgr2=0x563cba300d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba300d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 -- 172.21.15.94:0/360783053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 msgr2=0x563cba301260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba301260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 -- 172.21.15.94:0/360783053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cba304220 con 0x563cb9f51390 2024-04-25T13:56:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0ccbfff640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x563cba2f4860 tx=0x7f0cc402af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413960efb731dd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0caaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4058040 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f0cc402f660 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.438+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc0026830 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cc4009360 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 10 client.20763 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563cba372b20 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc0026830 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f0cc4004650 con 0x563cb9f51390 2024-04-25T13:56:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 10 client.20763 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 10 client.20763 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 20 client.20763 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9ee53a0 0x563cba375040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 4 client.20763 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cba25f800 con 0x563cb9ee53a0 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.439+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9ee53a0 0x563cba375040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0caaffd640 10 client.20763 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0ccb7fe640 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9ee53a0 0x563cba375040 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c69414d92da3688 server_cookie=3a0ba6e0bdc41ee1 in_seq=0 out_seq=0 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0caaffd640 1 -- 172.21.15.94:0/360783053 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cba25f800 con 0x563cb9ee53a0 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0caaffd640 10 client.20763 handle_command_reply: tid=0 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 client.20763 shutdown 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 msgr2=0x563cba371660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x563cba2f4860 tx=0x7f0cc402af40 comp rx=0 tx=0).stop 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 shutdown_connections 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x563cb9ee53a0 0x563cba375040 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb9f51390 0x563cba371660 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563cb9f306b0 0x563cba301260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 --2- 172.21.15.94:0/360783053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563cb9f284d0 0x563cba300d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 >> 172.21.15.94:0/360783053 conn(0x563cb9f454e0 msgr2=0x563cba2ee9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0caaffd640 0 client.20763 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 shutdown_connections 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 1 -- 172.21.15.94:0/360783053 wait complete. 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.440+0000 7f0cd89b7740 20 client.20763 trim_cache size 0 max 16384 2024-04-25T13:56:33.448 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 msgr2=0x7f0cb40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 0x7f0cb40b2080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0cc404f000 tx=0x7f0cc4052000 comp rx=0 tx=0).stop 2024-04-25T13:56:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 msgr2=0x7f0ccc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005d00 tx=0x7f0cbc00dde0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 shutdown_connections 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0cb40afb70 0x7f0cb40b2080 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ccc108920 0x7f0ccc1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ccc103c20 0x7f0ccc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 --2- 172.21.15.94:0/230590060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ccc1029e0 0x7f0ccc19a4e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 >> 172.21.15.94:0/230590060 conn(0x7f0ccc0fdbd0 msgr2=0x7f0ccc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 shutdown_connections 2024-04-25T13:56:33.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.441+0000 7f0cd1b08640 1 -- 172.21.15.94:0/230590060 wait complete. 2024-04-25T13:56:33.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:56:33.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 -- 172.21.15.94:0/3519301662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 msgr2=0x7f34c8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/3519301662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c8104030 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009900 tx=0x7f34bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 -- 172.21.15.94:0/3519301662 shutdown_connections 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/3519301662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 0x7f34c810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/3519301662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c8104030 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/3519301662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 0x7f34c81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.569+0000 7f34ce5ce640 1 -- 172.21.15.94:0/3519301662 >> 172.21.15.94:0/3519301662 conn(0x7f34c80fdbd0 msgr2=0x7f34c8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.570+0000 7f34ce5ce640 1 -- 172.21.15.94:0/3519301662 shutdown_connections 2024-04-25T13:56:33.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.570+0000 7f34ce5ce640 1 -- 172.21.15.94:0/3519301662 wait complete. 2024-04-25T13:56:33.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.570+0000 7f34ce5ce640 1 Processor -- start 2024-04-25T13:56:33.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.570+0000 7f34ce5ce640 1 -- start start 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 0x7f34c806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 0x7f34c81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e560 con 0x7f34c8103c20 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e3e0 con 0x7f34c81029e0 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ce5ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e6e0 con 0x7f34c8108920 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34076/0 (socket says 172.21.15.94:34076) 2024-04-25T13:56:33.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 -- 172.21.15.94:0/369760923 learned_addr learned my addr 172.21.15.94:0/369760923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c7fff640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 0x7f34c806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 0x7f34c81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 -- 172.21.15.94:0/369760923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 msgr2=0x7f34c81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 0x7f34c81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 -- 172.21.15.94:0/369760923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 msgr2=0x7f34c806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 0x7f34c806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 -- 172.21.15.94:0/369760923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c81a2030 con 0x7f34c8103c20 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c77fe640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031440 tx=0x7f34bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.571+0000 7f34c57fa640 1 -- 172.21.15.94:0/369760923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc03f070 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.572+0000 7f34c57fa640 1 -- 172.21.15.94:0/369760923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34bc031c50 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.572+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81a22c0 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.572+0000 7f34c57fa640 1 -- 172.21.15.94:0/369760923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc035030 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.572+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81a2800 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 Processor -- start 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 -- start start 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34c57fa640 1 -- 172.21.15.94:0/369760923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f34bc042070 con 0x7f34c8103c20 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d37a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e789c9ad80 con 0x55e7899736b0 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e789c54a20 con 0x55e78996b4d0 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.574+0000 7f34d5481740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e789c96d40 con 0x55e789994390 2024-04-25T13:56:33.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ccb6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ccb6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38718/0 (socket says 172.21.15.94:38718) 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42494/0 (socket says 172.21.15.94:42494) 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d37a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ccb6b640 1 -- 172.21.15.94:0/4173558786 learned_addr learned my addr 172.21.15.94:0/4173558786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 -- 172.21.15.94:0/4173558786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 msgr2=0x55e789d36810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 -- 172.21.15.94:0/4173558786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 msgr2=0x55e789d37a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d37a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 -- 172.21.15.94:0/4173558786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e789c5fc30 con 0x55e78996b4d0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c57fa640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 0x7f34b00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c7fff640 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f34c81035e0 tx=0x7f34b800bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea9beb859ac4b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 0x7f34b00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34c57fa640 1 -- 172.21.15.94:0/369760923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f34bc0ff410 con 0x7f34c8103c20 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ae7fc640 1 -- 172.21.15.94:0/4173558786 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8003da0 con 0x55e78996b4d0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ae7fc640 1 -- 172.21.15.94:0/4173558786 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b8015070 con 0x55e78996b4d0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.575+0000 7f34ae7fc640 1 -- 172.21.15.94:0/4173558786 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b80116a0 con 0x55e78996b4d0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 -- 172.21.15.94:0/4173558786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 msgr2=0x55e789d36d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f34c81035e0 tx=0x7f34b800bbe0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 0x7f34b00b2080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009fd0 tx=0x7f34c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 -- 172.21.15.94:0/4173558786 shutdown_connections 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d37a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d36d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 --2- 172.21.15.94:0/4173558786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d36810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 -- 172.21.15.94:0/4173558786 >> 172.21.15.94:0/4173558786 conn(0x55e7899884e0 msgr2=0x55e789d32020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 -- 172.21.15.94:0/4173558786 shutdown_connections 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.576+0000 7f34d5481740 1 -- 172.21.15.94:0/4173558786 wait complete. 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 -- start start 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 Processor -- start 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d47a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e789d38530 con 0x55e7899736b0 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e789c21dc0 con 0x55e78996b4d0 2024-04-25T13:56:33.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e789d386b0 con 0x55e789994390 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34096/0 (socket says 172.21.15.94:34096) 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34d5481740 10 client.0 _ll_register_callbacks cb 0x55e789c3d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34c77fe640 1 -- 172.21.15.94:0/2374594171 learned_addr learned my addr 172.21.15.94:0/2374594171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34c7fff640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.577+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d47a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 -- 172.21.15.94:0/2374594171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 msgr2=0x55e789d47a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d47a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 -- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 msgr2=0x55e789d405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 -- 172.21.15.94:0/2374594171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7899ac3f0 con 0x55e7899736b0 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:33.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34c77fe640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0c4b70 tx=0x7f34bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5283878d288b2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc049070 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f34bc03b460 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b8015070 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34bc0c37c0 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34d5481740 10 client.20775 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e789a767d0 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.578+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b8015070 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f34bc0c3b90 con 0x55e7899736b0 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 10 client.20775 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 10 client.20775 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 20 client.20775 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e789952f60 0x55e789d4e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 4 client.20775 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e789ca8990 con 0x55e789952f60 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34c7fff640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e789952f60 0x55e789d4e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34c7fff640 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e789952f60 0x55e789d4e360 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97957d1dd03eebb4 server_cookie=1e462c23a8df8f1d in_seq=0 out_seq=0 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.579+0000 7f34a77fe640 10 client.20775 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.580+0000 7f34a77fe640 1 -- 172.21.15.94:0/2374594171 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e789ca8990 con 0x55e789952f60 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.580+0000 7f34a77fe640 10 client.20775 handle_command_reply: tid=0 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.580+0000 7f34d5481740 1 client.20775 shutdown 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 msgr2=0x55e789d40ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0c4b70 tx=0x7f34bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 shutdown_connections 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e789952f60 0x55e789d4e360 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e789994390 0x55e789d47a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d40ae0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 --2- 172.21.15.94:0/2374594171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e78996b4d0 0x55e789d405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34a77fe640 0 client.20775 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 >> 172.21.15.94:0/2374594171 conn(0x55e7899884e0 msgr2=0x55e789d34f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 shutdown_connections 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 1 -- 172.21.15.94:0/2374594171 wait complete. 2024-04-25T13:56:33.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.581+0000 7f34d5481740 20 client.20775 trim_cache size 0 max 16384 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.594+0000 7f34d5481740 1 Processor -- start 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 -- start start 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d31a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 0x55e789d3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81ab500 con 0x55e7899736b0 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81ad100 con 0x55e789994390 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34d5481740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81abbd0 con 0x55e78996b4d0 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34102/0 (socket says 172.21.15.94:34102) 2024-04-25T13:56:33.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 -- 172.21.15.94:0/3713728297 learned_addr learned my addr 172.21.15.94:0/3713728297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 0x55e789d3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c7fff640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d31a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 -- 172.21.15.94:0/3713728297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 msgr2=0x55e789d31a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d31a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 -- 172.21.15.94:0/3713728297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 msgr2=0x55e789d3ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 0x55e789d3ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 -- 172.21.15.94:0/3713728297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e789d362e0 con 0x55e7899736b0 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 0x55e789d3ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34c77fe640 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f34bc02a9d0 tx=0x7f34bc03b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9acac9fa8561fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34adffb640 1 -- 172.21.15.94:0/3713728297 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc049030 con 0x55e7899736b0 2024-04-25T13:56:33.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34adffb640 1 -- 172.21.15.94:0/3713728297 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34bc0e0070 con 0x55e7899736b0 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.595+0000 7f34adffb640 1 -- 172.21.15.94:0/3713728297 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0c7070 con 0x55e7899736b0 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 -- 172.21.15.94:0/3713728297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 msgr2=0x55e789d31f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f34bc02a9d0 tx=0x7f34bc03b790 comp rx=0 tx=0).stop 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 -- 172.21.15.94:0/3713728297 shutdown_connections 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e789994390 0x55e789d3ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7899736b0 0x55e789d31f90 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 --2- 172.21.15.94:0/3713728297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d31a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 -- 172.21.15.94:0/3713728297 >> 172.21.15.94:0/3713728297 conn(0x55e7899884e0 msgr2=0x55e789d3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 -- 172.21.15.94:0/3713728297 shutdown_connections 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.596+0000 7f34d5481740 1 -- 172.21.15.94:0/3713728297 wait complete. 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 -- start start 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 Processor -- start 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7899736b0 0x55e789d44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81aca30 con 0x55e789994390 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81ac2a0 con 0x55e7899736b0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34c81a94e0 con 0x55e78996b4d0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34d5481740 10 client.0 _ll_register_callbacks cb 0x7f34bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34ccb6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34ccb6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34108/0 (socket says 172.21.15.94:34108) 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34ccb6b640 1 -- 172.21.15.94:0/1715018072 learned_addr learned my addr 172.21.15.94:0/1715018072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34c77fe640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7899736b0 0x55e789d44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.597+0000 7f34c7fff640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 -- 172.21.15.94:0/1715018072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 msgr2=0x55e789d43fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d43fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 -- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7899736b0 msgr2=0x55e789d44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7899736b0 0x55e789d44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 -- 172.21.15.94:0/1715018072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e789db4c40 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34ccb6b640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f34c0003f50 tx=0x7f34c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c058e837188a9ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000fe10 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f34c0004b20 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34bc0e0070 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34c0004d20 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34d5481740 10 client.20781 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e789db5ec0 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.598+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34bc0e0070 con 0x55e789994390 2024-04-25T13:56:33.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f34c0028070 con 0x55e789994390 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 10 client.20781 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 10 client.20781 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 20 client.20781 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e7899283a0 0x55e789db8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 4 client.20781 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e789ca2800 con 0x55e7899283a0 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34c7fff640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e7899283a0 0x55e789db8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34c7fff640 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e7899283a0 0x55e789db8960 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=faae4d7ea80e36fc server_cookie=641b833d841e1735 in_seq=0 out_seq=0 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.599+0000 7f34a77fe640 10 client.20781 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34a77fe640 1 -- 172.21.15.94:0/1715018072 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e789ca2800 con 0x55e7899283a0 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34a77fe640 10 client.20781 handle_command_reply: tid=0 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 client.20781 shutdown 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 msgr2=0x55e789db4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f34c0003f50 tx=0x7f34c0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 shutdown_connections 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e7899283a0 0x55e789db8960 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e789994390 0x55e789db4700 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7899736b0 0x55e789d44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34a77fe640 0 client.20781 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 --2- 172.21.15.94:0/1715018072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e78996b4d0 0x55e789d43fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 >> 172.21.15.94:0/1715018072 conn(0x55e7899884e0 msgr2=0x55e789d3ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 shutdown_connections 2024-04-25T13:56:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 1 -- 172.21.15.94:0/1715018072 wait complete. 2024-04-25T13:56:33.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34d5481740 20 client.20781 trim_cache size 0 max 16384 2024-04-25T13:56:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 msgr2=0x7f34b00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 0x7f34b00b2080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009fd0 tx=0x7f34c0009500 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 msgr2=0x7f34c819f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.600+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031440 tx=0x7f34bc003c10 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 shutdown_connections 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f34b00afb70 0x7f34b00b2080 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34c8108920 0x7f34c81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34c8103c20 0x7f34c819f2c0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 --2- 172.21.15.94:0/369760923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c81029e0 0x7f34c806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 >> 172.21.15.94:0/369760923 conn(0x7f34c80fdbd0 msgr2=0x7f34c8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 shutdown_connections 2024-04-25T13:56:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.601+0000 7f34ce5ce640 1 -- 172.21.15.94:0/369760923 wait complete. 2024-04-25T13:56:33.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:56:33.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 -- 172.21.15.94:0/1618855891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c108920 msgr2=0x7f9b8c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 --2- 172.21.15.94:0/1618855891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c108920 0x7f9b8c10ad40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009ca0 tx=0x7f9b8402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 -- 172.21.15.94:0/1618855891 shutdown_connections 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 --2- 172.21.15.94:0/1618855891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c108920 0x7f9b8c10ad40 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 --2- 172.21.15.94:0/1618855891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c103c20 0x7f9b8c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 --2- 172.21.15.94:0/1618855891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 -- 172.21.15.94:0/1618855891 >> 172.21.15.94:0/1618855891 conn(0x7f9b8c0fdbd0 msgr2=0x7f9b8c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 -- 172.21.15.94:0/1618855891 shutdown_connections 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.730+0000 7f9b9470d640 1 -- 172.21.15.94:0/1618855891 wait complete. 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 Processor -- start 2024-04-25T13:56:33.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 -- start start 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c103c20 0x7f9b8c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c108920 0x7f9b8c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10e510 con 0x7f9b8c1029e0 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10e390 con 0x7f9b8c108920 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b9470d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c10e690 con 0x7f9b8c103c20 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34134/0 (socket says 172.21.15.94:34134) 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 -- 172.21.15.94:0/3281157624 learned_addr learned my addr 172.21.15.94:0/3281157624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b92caa640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c108920 0x7f9b8c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c103c20 0x7f9b8c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 -- 172.21.15.94:0/3281157624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c103c20 msgr2=0x7f9b8c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c103c20 0x7f9b8c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 -- 172.21.15.94:0/3281157624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c108920 msgr2=0x7f9b8c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c108920 0x7f9b8c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.731+0000 7f9b924a9640 1 -- 172.21.15.94:0/3281157624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c1a2030 con 0x7f9b8c1029e0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005d00 tx=0x7f9b7c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b837fe640 1 -- 172.21.15.94:0/3281157624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c017070 con 0x7f9b8c1029e0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b837fe640 1 -- 172.21.15.94:0/3281157624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c009c30 con 0x7f9b8c1029e0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c1a2320 con 0x7f9b8c1029e0 2024-04-25T13:56:33.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b837fe640 1 -- 172.21.15.94:0/3281157624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00f620 con 0x7f9b8c1029e0 2024-04-25T13:56:33.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.732+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c1a2820 con 0x7f9b8c1029e0 2024-04-25T13:56:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.734+0000 7f9b837fe640 1 -- 172.21.15.94:0/3281157624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9b7c01a070 con 0x7f9b8c1029e0 2024-04-25T13:56:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.734+0000 7f9b9b5be740 1 Processor -- start 2024-04-25T13:56:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 -- start start 2024-04-25T13:56:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c06810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556073b6ad80 con 0x55607383b4d0 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556073b24a20 con 0x556073864390 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556073b66d40 con 0x5560738436b0 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b92caa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c06810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34144/0 (socket says 172.21.15.94:34144) 2024-04-25T13:56:33.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 -- 172.21.15.94:0/1330318229 learned_addr learned my addr 172.21.15.94:0/1330318229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b837fe640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 0x7f9b780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 -- 172.21.15.94:0/1330318229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 msgr2=0x556073c07a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c07a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 -- 172.21.15.94:0/1330318229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 msgr2=0x556073c06810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c06810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b924a9640 1 -- 172.21.15.94:0/1330318229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556073b2fc30 con 0x55607383b4d0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b92caa640 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b92caa640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 0x7f9b780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.735+0000 7f9b837fe640 1 -- 172.21.15.94:0/3281157624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9b7c0de670 con 0x7f9b8c1029e0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.736+0000 7f9b924a9640 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c063270 tx=0x7f9b7c010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e0251f65ccf36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.736+0000 7f9b80ff9640 1 -- 172.21.15.94:0/1330318229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0a6930 con 0x55607383b4d0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.736+0000 7f9b80ff9640 1 -- 172.21.15.94:0/1330318229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c0a6670 con 0x55607383b4d0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.736+0000 7f9b80ff9640 1 -- 172.21.15.94:0/1330318229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0a62d0 con 0x55607383b4d0 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1330318229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 msgr2=0x556073c06d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b92caa640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 0x7f9b780b2000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x556073c07860 tx=0x7f9b840095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c063270 tx=0x7f9b7c010c00 comp rx=0 tx=0).stop 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1330318229 shutdown_connections 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c07a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55607383b4d0 0x556073c06d90 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1330318229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c06810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1330318229 >> 172.21.15.94:0/1330318229 conn(0x5560738584e0 msgr2=0x556073c02020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1330318229 shutdown_connections 2024-04-25T13:56:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1330318229 wait complete. 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 -- start start 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.737+0000 7f9b9b5be740 1 Processor -- start 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c10a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556073c084d0 con 0x556073864390 2024-04-25T13:56:33.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556073af1dc0 con 0x55607383b4d0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556073c08650 con 0x5560738436b0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 10 client.0 _ll_register_callbacks cb 0x556073b0d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34154/0 (socket says 172.21.15.94:34154) 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 -- 172.21.15.94:0/1672148955 learned_addr learned my addr 172.21.15.94:0/1672148955 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42566/0 (socket says 172.21.15.94:42566) 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c10a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 -- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 msgr2=0x556073c10a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c10a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 -- 172.21.15.94:0/1672148955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 msgr2=0x556073c104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 -- 172.21.15.94:0/1672148955 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607387c3f0 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b92caa640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f9b84035000 tx=0x7f9b8402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9e9cfbda2f06f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8402f310 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9b84004540 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c0a6670 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9b84004740 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 10 client.20793 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.738+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560739467d0 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c0a6670 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9b840306e0 con 0x556073864390 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 10 client.20793 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 10 client.20793 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:56:33.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 20 client.20793 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556073822f60 0x556073c1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 4 client.20793 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556073b78990 con 0x556073822f60 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.739+0000 7f9b924a9640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556073822f60 0x556073c1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b75ffb640 10 client.20793 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b924a9640 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556073822f60 0x556073c1e240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4503ac30542da343 server_cookie=663b1a164b707d77 in_seq=0 out_seq=0 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b75ffb640 1 -- 172.21.15.94:0/1672148955 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556073b78990 con 0x556073822f60 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b75ffb640 10 client.20793 handle_command_reply: tid=0 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b9b5be740 1 client.20793 shutdown 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 msgr2=0x556073c17990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f9b84035000 tx=0x7f9b8402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 shutdown_connections 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.740+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556073822f60 0x556073c1e240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c17990 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c10a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b75ffb640 0 client.20793 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/1672148955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 >> 172.21.15.94:0/1672148955 conn(0x5560738584e0 msgr2=0x556073c04f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 shutdown_connections 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 1 -- 172.21.15.94:0/1672148955 wait complete. 2024-04-25T13:56:33.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.741+0000 7f9b9b5be740 20 client.20793 trim_cache size 0 max 16384 2024-04-25T13:56:33.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 Processor -- start 2024-04-25T13:56:33.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 -- start start 2024-04-25T13:56:33.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c0f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ab4f0 con 0x556073864390 2024-04-25T13:56:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ad0f0 con 0x55607383b4d0 2024-04-25T13:56:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.755+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1abbc0 con 0x5560738436b0 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34160/0 (socket says 172.21.15.94:34160) 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b924a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c0f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 -- 172.21.15.94:0/2839807922 learned_addr learned my addr 172.21.15.94:0/2839807922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 -- 172.21.15.94:0/2839807922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 msgr2=0x556073c178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 -- 172.21.15.94:0/2839807922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 msgr2=0x556073c0f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c0f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 -- 172.21.15.94:0/2839807922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556073c17df0 con 0x556073864390 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b92caa640 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b84004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae116c7d3bc0cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b77fff640 1 -- 172.21.15.94:0/2839807922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84060030 con 0x556073864390 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b77fff640 1 -- 172.21.15.94:0/2839807922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b84036040 con 0x556073864390 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.756+0000 7f9b77fff640 1 -- 172.21.15.94:0/2839807922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8402f660 con 0x556073864390 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- 172.21.15.94:0/2839807922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 msgr2=0x556073c01760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b84004990 comp rx=0 tx=0).stop 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- 172.21.15.94:0/2839807922 shutdown_connections 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556073864390 0x556073c01760 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560738436b0 0x556073c178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/2839807922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55607383b4d0 0x556073c0f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- 172.21.15.94:0/2839807922 >> 172.21.15.94:0/2839807922 conn(0x5560738584e0 msgr2=0x556073c0ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- 172.21.15.94:0/2839807922 shutdown_connections 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- 172.21.15.94:0/2839807922 wait complete. 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.757+0000 7f9b9b5be740 1 -- start start 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 Processor -- start 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55607383b4d0 0x556073c14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1aca20 con 0x5560738436b0 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ac290 con 0x556073864390 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1a94d0 con 0x55607383b4d0 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b9b5be740 10 client.0 _ll_register_callbacks cb 0x7f9b840099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34172/0 (socket says 172.21.15.94:34172) 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 -- 172.21.15.94:0/3308709574 learned_addr learned my addr 172.21.15.94:0/3308709574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b92caa640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55607383b4d0 0x556073c14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 -- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55607383b4d0 msgr2=0x556073c14830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55607383b4d0 0x556073c14830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 -- 172.21.15.94:0/3308709574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 msgr2=0x556073c845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 -- 172.21.15.94:0/3308709574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556073c84b20 con 0x5560738436b0 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b91ca8640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005d00 tx=0x7f9b88012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ffda30f49cddcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.758+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88021030 con 0x5560738436b0 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f9b88010c30 con 0x5560738436b0 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b84036040 con 0x5560738436b0 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9b88010e70 con 0x5560738436b0 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 10 client.20799 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556073c85de0 con 0x5560738436b0 2024-04-25T13:56:33.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b84036040 con 0x5560738436b0 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f9b88004cc0 con 0x5560738436b0 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 10 client.20799 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 10 client.20799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 20 client.20799 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5560737f83a0 0x556073c88880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 4 client.20799 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556073b72800 con 0x5560737f83a0 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.759+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5560737f83a0 0x556073c88880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b75ffb640 10 client.20799 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b924a9640 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5560737f83a0 0x556073c88880 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aaead53712ff8368 server_cookie=2641ae36ab232b8 in_seq=0 out_seq=0 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b75ffb640 1 -- 172.21.15.94:0/3308709574 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556073b72800 con 0x5560737f83a0 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b75ffb640 10 client.20799 handle_command_reply: tid=0 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 client.20799 shutdown 2024-04-25T13:56:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 msgr2=0x556073c14d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005d00 tx=0x7f9b88012b60 comp rx=0 tx=0).stop 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 shutdown_connections 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5560737f83a0 0x556073c88880 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b75ffb640 0 client.20799 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556073864390 0x556073c845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5560738436b0 0x556073c14d70 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 --2- 172.21.15.94:0/3308709574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55607383b4d0 0x556073c14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 >> 172.21.15.94:0/3308709574 conn(0x5560738584e0 msgr2=0x556073c04ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 shutdown_connections 2024-04-25T13:56:33.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.760+0000 7f9b9b5be740 1 -- 172.21.15.94:0/3308709574 wait complete. 2024-04-25T13:56:33.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9b5be740 20 client.20799 trim_cache size 0 max 16384 2024-04-25T13:56:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 msgr2=0x7f9b780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 0x7f9b780b2000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x556073c07860 tx=0x7f9b840095c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 msgr2=0x7f9b8c06a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005d00 tx=0x7f9b7c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 shutdown_connections 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9b780afaf0 0x7f9b780b2000 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b8c108920 0x7f9b8c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b8c103c20 0x7f9b8c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 --2- 172.21.15.94:0/3281157624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8c1029e0 0x7f9b8c06a900 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 >> 172.21.15.94:0/3281157624 conn(0x7f9b8c0fdbd0 msgr2=0x7f9b8c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 shutdown_connections 2024-04-25T13:56:33.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.761+0000 7f9b9470d640 1 -- 172.21.15.94:0/3281157624 wait complete. 2024-04-25T13:56:33.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:56:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2612686797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc1080d0 msgr2=0x7f42bc10a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2612686797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc1080d0 0x7f42bc10a4f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009ca0 tx=0x7f42b402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2612686797 shutdown_connections 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2612686797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc1080d0 0x7f42bc10a4f0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2612686797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 0x7f42bc1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2612686797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc102190 0x7f42bc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2612686797 >> 172.21.15.94:0/2612686797 conn(0x7f42bc0fd380 msgr2=0x7f42bc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2612686797 shutdown_connections 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2612686797 wait complete. 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.903+0000 7f42c2d1f640 1 Processor -- start 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 -- start start 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 0x7f42bc19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc1080d0 0x7f42bc1a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc10e540 con 0x7f42bc102190 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc10e3c0 con 0x7f42bc1080d0 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c2d1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc10e6c0 con 0x7f42bc1033d0 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34188/0 (socket says 172.21.15.94:34188) 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 -- 172.21.15.94:0/2137614885 learned_addr learned my addr 172.21.15.94:0/2137614885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42bbfff640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 0x7f42bc19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c12bc640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc1080d0 0x7f42bc1a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 -- 172.21.15.94:0/2137614885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 msgr2=0x7f42bc19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 0x7f42bc19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 -- 172.21.15.94:0/2137614885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc1080d0 msgr2=0x7f42bc1a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc1080d0 0x7f42bc1a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 -- 172.21.15.94:0/2137614885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1a3f40 con 0x7f42bc102190 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.904+0000 7f42c0abb640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9f0 tx=0x7f42b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.905+0000 7f42b9ffb640 1 -- 172.21.15.94:0/2137614885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000ccc0 con 0x7f42bc102190 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.905+0000 7f42b9ffb640 1 -- 172.21.15.94:0/2137614885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b00035f0 con 0x7f42bc102190 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.905+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc1a4230 con 0x7f42bc102190 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.905+0000 7f42b9ffb640 1 -- 172.21.15.94:0/2137614885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000ce60 con 0x7f42bc102190 2024-04-25T13:56:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.905+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc1a47d0 con 0x7f42bc102190 2024-04-25T13:56:33.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.907+0000 7f42b9ffb640 1 -- 172.21.15.94:0/2137614885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f42b0003aa0 con 0x7f42bc102190 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 Processor -- start 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 -- start start 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56494ebd4d80 con 0x56494e8ad6b0 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56494eb8ea20 con 0x56494e8ce390 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56494ebd0d40 con 0x56494e8a54d0 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38828/0 (socket says 172.21.15.94:38828) 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c0abb640 1 -- 172.21.15.94:0/1263944206 learned_addr learned my addr 172.21.15.94:0/1263944206 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42b9ffb640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 0x7f42ac0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 -- 172.21.15.94:0/1263944206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 msgr2=0x56494ec70d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 -- 172.21.15.94:0/1263944206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 msgr2=0x56494ec71a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec71a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c12bc640 1 -- 172.21.15.94:0/1263944206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494eb99c30 con 0x56494e8ce390 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.908+0000 7f42c0abb640 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42c12bc640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 0x7f42ac0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42b9ffb640 1 -- 172.21.15.94:0/2137614885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f42b00def40 con 0x7f42bc102190 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec70810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42b4031b50 tx=0x7f42b40098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3649a06d5e9e6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42c12bc640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 0x7f42ac0b2080 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x56494ec721e0 tx=0x7f42b404e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42a6ffd640 1 -- 172.21.15.94:0/1263944206 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4042040 con 0x56494e8ce390 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42a6ffd640 1 -- 172.21.15.94:0/1263944206 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x56494e8ce390 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.909+0000 7f42a6ffd640 1 -- 172.21.15.94:0/1263944206 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b402fc10 con 0x56494e8ce390 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- 172.21.15.94:0/1263944206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 msgr2=0x56494ec70810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec70810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f42b4031b50 tx=0x7f42b40098d0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- 172.21.15.94:0/1263944206 shutdown_connections 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8a54d0 0x56494ec70d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1263944206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec70810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- 172.21.15.94:0/1263944206 >> 172.21.15.94:0/1263944206 conn(0x56494e8c24e0 msgr2=0x56494ec6c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- 172.21.15.94:0/1263944206 shutdown_connections 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- 172.21.15.94:0/1263944206 wait complete. 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 -- start start 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.910+0000 7f42c9bce740 1 Processor -- start 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 0x56494ec81a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56494ec72530 con 0x56494e8ad6b0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56494eb5bdc0 con 0x56494e8a54d0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56494ec726b0 con 0x56494e8ce390 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c9bce740 10 client.0 _ll_register_callbacks cb 0x56494eb77600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34206/0 (socket says 172.21.15.94:34206) 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 -- 172.21.15.94:0/3076701840 learned_addr learned my addr 172.21.15.94:0/3076701840 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c12bc640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 0x56494ec81a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 -- 172.21.15.94:0/3076701840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 msgr2=0x56494ec81a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 0x56494ec81a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 -- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 msgr2=0x56494ec7a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 -- 172.21.15.94:0/3076701840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494e8e63f0 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c12bc640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 0x56494ec81a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42c0abb640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f42bbfff640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f42a800ba60 tx=0x7f42a800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c568da97bad9fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.911+0000 7f429bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8005020 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f42a80051c0 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b4036070 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42a8005400 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 10 client.20811 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56494e9b07d0 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f42a801b600 con 0x56494e8ad6b0 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 10 client.20811 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 10 client.20811 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:56:33.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 20 client.20811 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e88cf60 0x56494ec88330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 4 client.20811 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.912+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56494ebe2990 con 0x56494e88cf60 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.913+0000 7f42c0abb640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e88cf60 0x56494ec88330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.913+0000 7f429bfff640 10 client.20811 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.913+0000 7f42c0abb640 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e88cf60 0x56494ec88330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8fab543e33d904d server_cookie=43ad0471194a67ae in_seq=0 out_seq=0 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f429bfff640 1 -- 172.21.15.94:0/3076701840 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56494ebe2990 con 0x56494e88cf60 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f429bfff640 10 client.20811 handle_command_reply: tid=0 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 client.20811 shutdown 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 msgr2=0x56494ec7ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f42a800ba60 tx=0x7f42a800bf20 comp rx=0 tx=0).stop 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 shutdown_connections 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e88cf60 0x56494ec88330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ce390 0x56494ec81a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ad6b0 0x56494ec7ab30 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f429bfff640 0 client.20811 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3076701840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 >> 172.21.15.94:0/3076701840 conn(0x56494e8c24e0 msgr2=0x56494ec6b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 shutdown_connections 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 1 -- 172.21.15.94:0/3076701840 wait complete. 2024-04-25T13:56:33.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.914+0000 7f42c9bce740 20 client.20811 trim_cache size 0 max 16384 2024-04-25T13:56:33.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.929+0000 7f42c9bce740 1 Processor -- start 2024-04-25T13:56:33.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 -- start start 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 0x56494ec78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ad660 con 0x56494e8a54d0 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1af260 con 0x56494e8ce390 2024-04-25T13:56:33.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1add30 con 0x56494e8ad6b0 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c12bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c12bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42648/0 (socket says 172.21.15.94:42648) 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c12bc640 1 -- 172.21.15.94:0/3960895310 learned_addr learned my addr 172.21.15.94:0/3960895310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42c0abb640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 0x56494ec78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.930+0000 7f42bbfff640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 -- 172.21.15.94:0/3960895310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 msgr2=0x56494ec79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 -- 172.21.15.94:0/3960895310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 msgr2=0x56494ec78d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 0x56494ec78d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 -- 172.21.15.94:0/3960895310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494ec72770 con 0x56494e8ce390 2024-04-25T13:56:33.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c0abb640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 0x56494ec78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42c12bc640 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009ca0 tx=0x7f42b40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd01f1a5e1763a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42a67fc640 1 -- 172.21.15.94:0/3960895310 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4059070 con 0x56494e8ce390 2024-04-25T13:56:33.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42a67fc640 1 -- 172.21.15.94:0/3960895310 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x56494e8ce390 2024-04-25T13:56:33.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.931+0000 7f42a67fc640 1 -- 172.21.15.94:0/3960895310 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4042040 con 0x56494e8ce390 2024-04-25T13:56:33.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 -- 172.21.15.94:0/3960895310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 msgr2=0x56494ec8cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009ca0 tx=0x7f42b40095c0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 -- 172.21.15.94:0/3960895310 shutdown_connections 2024-04-25T13:56:33.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8ce390 0x56494ec8cae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 --2- 172.21.15.94:0/3960895310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8a54d0 0x56494ec78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 -- 172.21.15.94:0/3960895310 >> 172.21.15.94:0/3960895310 conn(0x56494e8c24e0 msgr2=0x56494ec75310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 -- 172.21.15.94:0/3960895310 shutdown_connections 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.932+0000 7f42c9bce740 1 -- 172.21.15.94:0/3960895310 wait complete. 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c9bce740 1 -- start start 2024-04-25T13:56:33.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c9bce740 1 Processor -- start 2024-04-25T13:56:33.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c9bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 0x56494ecee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.933+0000 7f42c0abb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42660/0 (socket says 172.21.15.94:42660) 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38854/0 (socket says 172.21.15.94:38854) 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c9bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1aeb90 con 0x56494e8ce390 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c9bce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ae400 con 0x56494e8a54d0 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c9bce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1ab640 con 0x56494e8ad6b0 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c9bce740 10 client.0 _ll_register_callbacks cb 0x7f42a80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c0abb640 1 -- 172.21.15.94:0/1725848556 learned_addr learned my addr 172.21.15.94:0/1725848556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 0x56494ecee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 -- 172.21.15.94:0/1725848556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 msgr2=0x56494ec7e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 -- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 msgr2=0x56494ec7de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42c12bc640 1 -- 172.21.15.94:0/1725848556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494ec81350 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.934+0000 7f42bbfff640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.935+0000 7f42c12bc640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 0x56494ecee690 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f42b402ade0 tx=0x7f42b4031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298500d18fde1ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.935+0000 7f429bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.935+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4062060 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.935+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f42b4004040 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.935+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b4036070 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42b4059030 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f42c9bce740 10 client.20817 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56494ecf8390 con 0x56494e8ce390 2024-04-25T13:56:33.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x56494e8ce390 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f42b4030ba0 con 0x56494e8ce390 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f42c9bce740 10 client.20817 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f42c9bce740 10 client.20817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.936+0000 7f42c9bce740 20 client.20817 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.937+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e8623a0 0x56494ecfa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.937+0000 7f42c9bce740 4 client.20817 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.937+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56494ebdc800 con 0x56494e8623a0 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.937+0000 7f42c0abb640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e8623a0 0x56494ecfa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c0abb640 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e8623a0 0x56494ecfa7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cad86f6763958afa server_cookie=9dfde98c19ead5b8 in_seq=0 out_seq=0 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f429bfff640 10 client.20817 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:33.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f429bfff640 1 -- 172.21.15.94:0/1725848556 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56494ebdc800 con 0x56494e8623a0 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f429bfff640 10 client.20817 handle_command_reply: tid=0 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 client.20817 shutdown 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 msgr2=0x56494ecee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 0x56494ecee690 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f42b402ade0 tx=0x7f42b4031690 comp rx=0 tx=0).stop 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 shutdown_connections 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56494e8623a0 0x56494ecfa7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56494e8ce390 0x56494ecee690 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56494e8ad6b0 0x56494ec7e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 --2- 172.21.15.94:0/1725848556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56494e8a54d0 0x56494ec7de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 >> 172.21.15.94:0/1725848556 conn(0x56494e8c24e0 msgr2=0x56494ec78d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f429bfff640 0 client.20817 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 shutdown_connections 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 1 -- 172.21.15.94:0/1725848556 wait complete. 2024-04-25T13:56:33.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.939+0000 7f42c9bce740 20 client.20817 trim_cache size 0 max 16384 2024-04-25T13:56:33.950 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 msgr2=0x7f42ac0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 0x7f42ac0b2080 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x56494ec721e0 tx=0x7f42b404e000 comp rx=0 tx=0).stop 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 msgr2=0x7f42bc19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9f0 tx=0x7f42b000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 shutdown_connections 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42ac0afb70 0x7f42ac0b2080 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42bc1080d0 0x7f42bc1a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc1033d0 0x7f42bc19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 --2- 172.21.15.94:0/2137614885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42bc102190 0x7f42bc19c400 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 >> 172.21.15.94:0/2137614885 conn(0x7f42bc0fd380 msgr2=0x7f42bc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 shutdown_connections 2024-04-25T13:56:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:33.941+0000 7f42c2d1f640 1 -- 172.21.15.94:0/2137614885 wait complete. 2024-04-25T13:56:33.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:56:34.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.082+0000 7f15c4fee640 1 -- 172.21.15.94:0/1720797965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 msgr2=0x7f15c010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.082+0000 7f15c4fee640 1 --2- 172.21.15.94:0/1720797965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c010ad40 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009ca0 tx=0x7f15b802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 -- 172.21.15.94:0/1720797965 shutdown_connections 2024-04-25T13:56:34.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 --2- 172.21.15.94:0/1720797965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c010ad40 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 --2- 172.21.15.94:0/1720797965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0103c20 0x7f15c0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 --2- 172.21.15.94:0/1720797965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c01029e0 0x7f15c01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 -- 172.21.15.94:0/1720797965 >> 172.21.15.94:0/1720797965 conn(0x7f15c00fdbd0 msgr2=0x7f15c0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 -- 172.21.15.94:0/1720797965 shutdown_connections 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.083+0000 7f15c4fee640 1 -- 172.21.15.94:0/1720797965 wait complete. 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 Processor -- start 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 -- start start 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 0x7f15c0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 0x7f15c00720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e4f0 con 0x7f15c0108920 2024-04-25T13:56:34.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e370 con 0x7f15c0103c20 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15c4fee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010e670 con 0x7f15c01029e0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 0x7f15c00720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 0x7f15c0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 0x7f15c00720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42666/0 (socket says 172.21.15.94:42666) 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15bdd9b640 1 -- 172.21.15.94:0/489974167 learned_addr learned my addr 172.21.15.94:0/489974167 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15bed9d640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 -- 172.21.15.94:0/489974167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 msgr2=0x7f15c00720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 0x7f15c00720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 -- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 msgr2=0x7f15c0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.084+0000 7f15be59c640 1 -- 172.21.15.94:0/489974167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c00736c0 con 0x7f15c01029e0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15be59c640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 0x7f15c0071bb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15c01022b0 tx=0x7f15b000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15af7fe640 1 -- 172.21.15.94:0/489974167 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000cc40 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15af7fe640 1 -- 172.21.15.94:0/489974167 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b000cde0 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15af7fe640 1 -- 172.21.15.94:0/489974167 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0010430 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c00739b0 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.085+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c0073eb0 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15af7fe640 1 -- 172.21.15.94:0/489974167 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f15b0003ed0 con 0x7f15c01029e0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 Processor -- start 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 -- start start 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d3e99d3d80 con 0x55d3e96ac6b0 2024-04-25T13:56:34.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3e998da20 con 0x55d3e96a44d0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.088+0000 7f15cbf73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d3e99cfd40 con 0x55d3e96cd390 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15af7fe640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 0x7f15a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34256/0 (socket says 172.21.15.94:34256) 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1139456239 learned_addr learned my addr 172.21.15.94:0/1139456239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bed9d640 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bed9d640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 0x7f15a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15af7fe640 1 -- 172.21.15.94:0/489974167 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f15b00df0b0 con 0x7f15c01029e0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15be59c640 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1139456239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 msgr2=0x55d3e9a6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1139456239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 msgr2=0x55d3e9a6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a6fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1139456239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e9998c30 con 0x55d3e96ac6b0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009cf0 tx=0x7f15b40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1877c5ae5fc064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15bed9d640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 0x7f15a80b2000 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55d3e9a70860 tx=0x7f15b8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15acff9640 1 -- 172.21.15.94:0/1139456239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005020 con 0x55d3e96ac6b0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15acff9640 1 -- 172.21.15.94:0/1139456239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40047a0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.089+0000 7f15acff9640 1 -- 172.21.15.94:0/1139456239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b401a700 con 0x55d3e96ac6b0 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 -- 172.21.15.94:0/1139456239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 msgr2=0x55d3e9a70a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009cf0 tx=0x7f15b40119a0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 -- 172.21.15.94:0/1139456239 shutdown_connections 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a70a20 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1139456239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.090+0000 7f15cbf73740 1 -- 172.21.15.94:0/1139456239 >> 172.21.15.94:0/1139456239 conn(0x55d3e96c14e0 msgr2=0x55d3e9a6b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- 172.21.15.94:0/1139456239 shutdown_connections 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- 172.21.15.94:0/1139456239 wait complete. 2024-04-25T13:56:34.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- start start 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 Processor -- start 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9a71530 con 0x55d3e96a44d0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3e995adc0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.091+0000 7f15cbf73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9a716b0 con 0x55d3e96cd390 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15cbf73740 10 client.0 _ll_register_callbacks cb 0x55d3e9976600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34266/0 (socket says 172.21.15.94:34266) 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 -- 172.21.15.94:0/2234848801 learned_addr learned my addr 172.21.15.94:0/2234848801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15bed9d640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 -- 172.21.15.94:0/2234848801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 msgr2=0x55d3e9a80910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a80910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 -- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96ac6b0 msgr2=0x55d3e9a799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 -- 172.21.15.94:0/2234848801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e96e53f0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15be59c640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f15b00a71f0 tx=0x7f15b00a6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4de968d6f65583e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b00c0020 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f15b00afec0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.092+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b40047a0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15b00a74f0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 10 client.20829 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e97af7d0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40047a0 con 0x55d3e96a44d0 2024-04-25T13:56:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f15b00a6750 con 0x55d3e96a44d0 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 10 client.20829 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 10 client.20829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 20 client.20829 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e968bf60 0x55d3e9a872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 4 client.20829 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e99e1990 con 0x55d3e968bf60 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.093+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e968bf60 0x55d3e9a872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.094+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e968bf60 0x55d3e9a872d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16623cd43d86ab8a server_cookie=f106a3ad11b0a060 in_seq=0 out_seq=0 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.094+0000 7f15a5ffb640 10 client.20829 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15a5ffb640 1 -- 172.21.15.94:0/2234848801 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d3e99e1990 con 0x55d3e968bf60 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15a5ffb640 10 client.20829 handle_command_reply: tid=0 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 client.20829 shutdown 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 msgr2=0x55d3e9a79470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f15b00a71f0 tx=0x7f15b00a6d20 comp rx=0 tx=0).stop 2024-04-25T13:56:34.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 shutdown_connections 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e968bf60 0x55d3e9a872d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96cd390 0x55d3e9a80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2234848801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96a44d0 0x55d3e9a79470 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 >> 172.21.15.94:0/2234848801 conn(0x55d3e96c14e0 msgr2=0x55d3e9a6a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15a5ffb640 0 client.20829 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 shutdown_connections 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.096+0000 7f15cbf73740 1 -- 172.21.15.94:0/2234848801 wait complete. 2024-04-25T13:56:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.097+0000 7f15cbf73740 20 client.20829 trim_cache size 0 max 16384 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 Processor -- start 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 -- start start 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96a44d0 0x55d3e9a77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01b18c0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01b34c0 con 0x55d3e96cd390 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.114+0000 7f15cbf73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01b1f90 con 0x55d3e96a44d0 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34280/0 (socket says 172.21.15.94:34280) 2024-04-25T13:56:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 -- 172.21.15.94:0/2757213106 learned_addr learned my addr 172.21.15.94:0/2757213106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bed9d640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42684/0 (socket says 172.21.15.94:42684) 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15be59c640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96a44d0 0x55d3e9a77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 -- 172.21.15.94:0/2757213106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96a44d0 msgr2=0x55d3e9a77cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96a44d0 0x55d3e9a77cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 -- 172.21.15.94:0/2757213106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 msgr2=0x55d3e9a79950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 -- 172.21.15.94:0/2757213106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e9a6f3d0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bed9d640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009c90 tx=0x7f15b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a271988e95e9b882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.115+0000 7f15a7fff640 1 -- 172.21.15.94:0/2757213106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4011050 con 0x55d3e96ac6b0 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.116+0000 7f15a7fff640 1 -- 172.21.15.94:0/2757213106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40044f0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.116+0000 7f15a7fff640 1 -- 172.21.15.94:0/2757213106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005040 con 0x55d3e96ac6b0 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- 172.21.15.94:0/2757213106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 msgr2=0x55d3e9a79410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009c90 tx=0x7f15b40046c0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- 172.21.15.94:0/2757213106 shutdown_connections 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96cd390 0x55d3e9a79950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a79410 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 --2- 172.21.15.94:0/2757213106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96a44d0 0x55d3e9a77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- 172.21.15.94:0/2757213106 >> 172.21.15.94:0/2757213106 conn(0x55d3e96c14e0 msgr2=0x55d3e9a6a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- 172.21.15.94:0/2757213106 shutdown_connections 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- 172.21.15.94:0/2757213106 wait complete. 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 -- start start 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.117+0000 7f15cbf73740 1 Processor -- start 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 0x55d3e9af19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01b2df0 con 0x55d3e96cd390 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01b2660 con 0x55d3e96a44d0 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01af8a0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15be59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38906/0 (socket says 172.21.15.94:38906) 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1526594245 learned_addr learned my addr 172.21.15.94:0/1526594245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15cbf73740 10 client.0 _ll_register_callbacks cb 0x7f15b00a7350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:34.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bed9d640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 0x55d3e9af19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 msgr2=0x55d3e9a7d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a7d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1526594245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 msgr2=0x55d3e9af19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 0x55d3e9af19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 -- 172.21.15.94:0/1526594245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e9af1f60 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bed9d640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 0x55d3e9af19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15bdd9b640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b40099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ac7fc98143bfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.118+0000 7f15a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b403a060 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f15b401ac20 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b40044f0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15b4005c70 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 10 client.29986 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e9ae9810 con 0x55d3e96ac6b0 2024-04-25T13:56:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40044f0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f15b402cce0 con 0x55d3e96ac6b0 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 10 client.29986 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 10 client.29986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 20 client.29986 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e96613a0 0x55d3e9aec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 4 client.29986 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.119+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3e99db800 con 0x55d3e96613a0 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.120+0000 7f15be59c640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e96613a0 0x55d3e9aec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.120+0000 7f15a5ffb640 10 client.29986 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.120+0000 7f15be59c640 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e96613a0 0x55d3e9aec050 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbf791300512a63 server_cookie=2f55080c31b32b51 in_seq=0 out_seq=0 2024-04-25T13:56:34.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15a5ffb640 1 -- 172.21.15.94:0/1526594245 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3e99db800 con 0x55d3e96613a0 2024-04-25T13:56:34.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15a5ffb640 10 client.29986 handle_command_reply: tid=0 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 client.29986 shutdown 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 msgr2=0x55d3e9a7d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000c00 tx=0x7f15b40099c0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 shutdown_connections 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d3e96613a0 0x55d3e9aec050 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3e96cd390 0x55d3e9af19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15a5ffb640 0 client.29986 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3e96ac6b0 0x55d3e9a7d570 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 --2- 172.21.15.94:0/1526594245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3e96a44d0 0x55d3e9a7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.122+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 >> 172.21.15.94:0/1526594245 conn(0x55d3e96c14e0 msgr2=0x55d3e9a6f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 shutdown_connections 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15cbf73740 1 -- 172.21.15.94:0/1526594245 wait complete. 2024-04-25T13:56:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15cbf73740 20 client.29986 trim_cache size 0 max 16384 2024-04-25T13:56:34.126 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 msgr2=0x7f15a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 0x7f15a80b2000 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55d3e9a70860 tx=0x7f15b8009360 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 msgr2=0x7f15c0071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 0x7f15c0071bb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15c01022b0 tx=0x7f15b000ee50 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 shutdown_connections 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f15a80afaf0 0x7f15a80b2000 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15c0108920 0x7f15c0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15c0103c20 0x7f15c00720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 --2- 172.21.15.94:0/489974167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c01029e0 0x7f15c0071bb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 >> 172.21.15.94:0/489974167 conn(0x7f15c00fdbd0 msgr2=0x7f15c0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:34.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 shutdown_connections 2024-04-25T13:56:34.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:34.123+0000 7f15c4fee640 1 -- 172.21.15.94:0/489974167 wait complete. 2024-04-25T13:56:34.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 28684 22.69 MB/sec warmup 76 sec latency 79.527 ms 2024-04-25T13:56:34.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:34 smithi094 ceph-mon[29195]: pgmap v792: 97 pgs: 97 active+clean; 236 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-04-25T13:56:34.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1120957283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:34 smithi138 ceph-mon[32977]: pgmap v792: 97 pgs: 97 active+clean; 236 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-04-25T13:56:34.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1120957283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:34 smithi137 ceph-mon[32568]: pgmap v792: 97 pgs: 97 active+clean; 236 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-04-25T13:56:34.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1120957283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:35.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:56:35.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:56:35.136 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:56:35.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 28972 22.58 MB/sec warmup 77 sec latency 122.994 ms 2024-04-25T13:56:35.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 -- 172.21.15.94:0/918538888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0107940 msgr2=0x7fafc0107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 --2- 172.21.15.94:0/918538888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0107940 0x7fafc0107d50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009ca0 tx=0x7fafb802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 -- 172.21.15.94:0/918538888 shutdown_connections 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 --2- 172.21.15.94:0/918538888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0107940 0x7fafc0107d50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 --2- 172.21.15.94:0/918538888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 0x7fafc0102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 --2- 172.21.15.94:0/918538888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0101d90 0x7fafc01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 -- 172.21.15.94:0/918538888 >> 172.21.15.94:0/918538888 conn(0x7fafc00fd380 msgr2=0x7fafc00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 -- 172.21.15.94:0/918538888 shutdown_connections 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.260+0000 7fafc63f3640 1 -- 172.21.15.94:0/918538888 wait complete. 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 Processor -- start 2024-04-25T13:56:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 -- start start 2024-04-25T13:56:35.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 0x7fafc01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 0x7fafc01a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0115c10 con 0x7fafc0101d90 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0115a90 con 0x7fafc0107940 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc63f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0115d90 con 0x7fafc01027d0 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34300/0 (socket says 172.21.15.94:34300) 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafbffff640 1 -- 172.21.15.94:0/3565471865 learned_addr learned my addr 172.21.15.94:0/3565471865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.261+0000 7fafc4990640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 0x7fafc01a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 0x7fafc01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 -- 172.21.15.94:0/3565471865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 msgr2=0x7fafc01a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 0x7fafc01a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 -- 172.21.15.94:0/3565471865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 msgr2=0x7fafc01a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 0x7fafc01a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 -- 172.21.15.94:0/3565471865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc01a82e0 con 0x7fafc0101d90 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafc4990640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 0x7fafc01a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbffff640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9f0 tx=0x7fafb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbd7fa640 1 -- 172.21.15.94:0/3565471865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb000cd80 con 0x7fafc0101d90 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc01a85d0 con 0x7fafc0101d90 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc01a8b70 con 0x7fafc0101d90 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbd7fa640 1 -- 172.21.15.94:0/3565471865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb00108e0 con 0x7fafc0101d90 2024-04-25T13:56:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.262+0000 7fafbd7fa640 1 -- 172.21.15.94:0/3565471865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00188e0 con 0x7fafc0101d90 2024-04-25T13:56:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 Processor -- start 2024-04-25T13:56:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 -- start start 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafbd7fa640 1 -- 172.21.15.94:0/3565471865 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fafb0018a80 con 0x7fafc0101d90 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 0x55f18291fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f182926920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f182884b00 con 0x55f18257e410 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f182820920 con 0x55f18255d730 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafcd37d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f182880ac0 con 0x55f182555550 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafc4990640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.265+0000 7fafbf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f182926920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafc4990640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34312/0 (socket says 172.21.15.94:34312) 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafc4990640 1 -- 172.21.15.94:0/3895795145 learned_addr learned my addr 172.21.15.94:0/3895795145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbffff640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 0x55f18291fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 -- 172.21.15.94:0/3895795145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 msgr2=0x55f18291fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 0x55f18291fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 -- 172.21.15.94:0/3895795145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 msgr2=0x55f18291f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 -- 172.21.15.94:0/3895795145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18265e7e0 con 0x55f18255d730 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbffff640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 0x55f18291fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafc4990640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbd7fa640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 0x7fafa80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafc4990640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 0x7fafa80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbd7fa640 1 -- 172.21.15.94:0/3565471865 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fafb00def10 con 0x7fafc0101d90 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f182926920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafb400bbc0 tx=0x7fafb40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c142d93d14ff66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafa67fc640 1 -- 172.21.15.94:0/3895795145 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004030 con 0x55f18255d730 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafa67fc640 1 -- 172.21.15.94:0/3895795145 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40041d0 con 0x55f18255d730 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.266+0000 7fafa67fc640 1 -- 172.21.15.94:0/3895795145 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40116a0 con 0x55f18255d730 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 -- 172.21.15.94:0/3895795145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 msgr2=0x55f182926920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafc4990640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 0x7fafa80b2000 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009a60 tx=0x7fafb802f490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f182926920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafb400bbc0 tx=0x7fafb40107a0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 -- 172.21.15.94:0/3895795145 shutdown_connections 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f182926920 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f182555550 0x55f18291fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 --2- 172.21.15.94:0/3895795145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f18291f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 -- 172.21.15.94:0/3895795145 >> 172.21.15.94:0/3895795145 conn(0x55f182572560 msgr2=0x55f18291ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 -- 172.21.15.94:0/3895795145 shutdown_connections 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.267+0000 7fafcd37d740 1 -- 172.21.15.94:0/3895795145 wait complete. 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 -- start start 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 Processor -- start 2024-04-25T13:56:35.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 0x55f1829292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18255d730 0x55f182929810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f182927370 con 0x55f18257e410 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f18281e5d0 con 0x55f182555550 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1829274f0 con 0x55f18255d730 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafcd37d740 10 client.0 _ll_register_callbacks cb 0x55f18280c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafc4990640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 0x55f1829292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafc4990640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34320/0 (socket says 172.21.15.94:34320) 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafc4990640 1 -- 172.21.15.94:0/1023824047 learned_addr learned my addr 172.21.15.94:0/1023824047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.268+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18255d730 0x55f182929810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 -- 172.21.15.94:0/1023824047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18255d730 msgr2=0x55f182929810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18255d730 0x55f182929810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 -- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 msgr2=0x55f1829292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 0x55f1829292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 -- 172.21.15.94:0/1023824047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1826bef70 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafbffff640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 0x55f1829292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafc4990640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009ca0 tx=0x7fafb802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181daf70295ba269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7faf9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8004310 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fafb80044b0 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb40041d0 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafb80046b0 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafcd37d740 10 client.20847 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f182556170 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.269+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40041d0 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fafb8052070 con 0x55f18257e410 2024-04-25T13:56:35.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 10 client.20847 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 10 client.20847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 20 client.20847 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f18253d1f0 0x55f182936da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 4 client.20847 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f182892710 con 0x55f18253d1f0 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafbffff640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f18253d1f0 0x55f182936da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7faf9f7fe640 10 client.20847 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.270+0000 7fafbffff640 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f18253d1f0 0x55f182936da0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c0db5271fb019f server_cookie=858b4086dc47cee7 in_seq=0 out_seq=0 2024-04-25T13:56:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1023824047 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f182892710 con 0x55f18253d1f0 2024-04-25T13:56:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7faf9f7fe640 10 client.20847 handle_command_reply: tid=0 2024-04-25T13:56:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 client.20847 shutdown 2024-04-25T13:56:35.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 msgr2=0x55f182930770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009ca0 tx=0x7fafb802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 shutdown_connections 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f18253d1f0 0x55f182936da0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f18257e410 0x55f182930770 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18255d730 0x55f182929810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1023824047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f182555550 0x55f1829292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 >> 172.21.15.94:0/1023824047 conn(0x55f182572560 msgr2=0x55f182919ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7faf9f7fe640 0 client.20847 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 shutdown_connections 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.273+0000 7fafcd37d740 1 -- 172.21.15.94:0/1023824047 wait complete. 2024-04-25T13:56:35.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.274+0000 7fafcd37d740 20 client.20847 trim_cache size 0 max 16384 2024-04-25T13:56:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 Processor -- start 2024-04-25T13:56:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 -- start start 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18291ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f182930690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00e730 con 0x55f182555550 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafac010330 con 0x55f18255d730 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafcd37d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00ee00 con 0x55f18257e410 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafbf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18291ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.294+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34324/0 (socket says 172.21.15.94:34324) 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbffff640 1 -- 172.21.15.94:0/2991975580 learned_addr learned my addr 172.21.15.94:0/2991975580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafc4990640 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f182930690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 -- 172.21.15.94:0/2991975580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 msgr2=0x55f182930690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f182930690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 -- 172.21.15.94:0/2991975580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 msgr2=0x55f18291a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 -- 172.21.15.94:0/2991975580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1829240f0 con 0x55f18255d730 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbffff640 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18291ad30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb400b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0ffa3effadf737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafa5ffb640 1 -- 172.21.15.94:0/2991975580 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb402a070 con 0x55f18255d730 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafa5ffb640 1 -- 172.21.15.94:0/2991975580 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40054f0 con 0x55f18255d730 2024-04-25T13:56:35.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.295+0000 7fafa5ffb640 1 -- 172.21.15.94:0/2991975580 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4011440 con 0x55f18255d730 2024-04-25T13:56:35.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- 172.21.15.94:0/2991975580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 msgr2=0x55f18291ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18291ad30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c00 tx=0x7fafb400b940 comp rx=0 tx=0).stop 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- 172.21.15.94:0/2991975580 shutdown_connections 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f182930690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18291ad30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 --2- 172.21.15.94:0/2991975580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f18291a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- 172.21.15.94:0/2991975580 >> 172.21.15.94:0/2991975580 conn(0x55f182572560 msgr2=0x55f182927920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- 172.21.15.94:0/2991975580 shutdown_connections 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- 172.21.15.94:0/2991975580 wait complete. 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.296+0000 7fafcd37d740 1 -- start start 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 Processor -- start 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18292db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f18292e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00fc60 con 0x55f182555550 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00f4d0 con 0x55f18255d730 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafac00c710 con 0x55f18257e410 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafcd37d740 10 client.0 _ll_register_callbacks cb 0x55f18280c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34326/0 (socket says 172.21.15.94:34326) 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 -- 172.21.15.94:0/1796013929 learned_addr learned my addr 172.21.15.94:0/1796013929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18292db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 -- 172.21.15.94:0/1796013929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 msgr2=0x55f18292e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f18292e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 -- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 msgr2=0x55f18292db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18292db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.297+0000 7fafbffff640 1 -- 172.21.15.94:0/1796013929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18299d130 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafbffff640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fafb00dcce0 tx=0x7fafb00e2970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67457a4a67e4193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00a7400 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fafb00a1da0 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb40054f0 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafb00ab700 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 10 client.20853 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f18299e440 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40054f0 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fafb00a2f00 con 0x55f182555550 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 10 client.20853 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 10 client.20853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:35.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 20 client.20853 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f182512630 0x55f1829a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 4 client.20853 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.298+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f18288c580 con 0x55f182512630 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.299+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f182512630 0x55f1829a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.299+0000 7fafbf7fe640 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f182512630 0x55f1829a0e90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d3beb0b7ce1f8 server_cookie=c9f47ec77d901bd0 in_seq=0 out_seq=0 2024-04-25T13:56:35.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.299+0000 7faf9f7fe640 10 client.20853 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7faf9f7fe640 1 -- 172.21.15.94:0/1796013929 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f18288c580 con 0x55f182512630 2024-04-25T13:56:35.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7faf9f7fe640 10 client.20853 handle_command_reply: tid=0 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 client.20853 shutdown 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 msgr2=0x55f182930fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fafb00dcce0 tx=0x7fafb00e2970 comp rx=0 tx=0).stop 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 shutdown_connections 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f182512630 0x55f1829a0e90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f18257e410 0x55f18292e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7faf9f7fe640 0 client.20853 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f18255d730 0x55f18292db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 --2- 172.21.15.94:0/1796013929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f182555550 0x55f182930fb0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 >> 172.21.15.94:0/1796013929 conn(0x55f182572560 msgr2=0x55f18291dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 shutdown_connections 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 1 -- 172.21.15.94:0/1796013929 wait complete. 2024-04-25T13:56:35.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.303+0000 7fafcd37d740 20 client.20853 trim_cache size 0 max 16384 2024-04-25T13:56:35.307 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:56:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:56:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "bd670531-51b6-47f0-84cb-07060f405538", 2024-04-25T13:56:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:56:35.307 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 msgr2=0x7fafa80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 0x7fafa80b2000 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009a60 tx=0x7fafb802f490 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 msgr2=0x7fafc01a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9f0 tx=0x7fafb000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 shutdown_connections 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fafa80afaf0 0x7fafa80b2000 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc0107940 0x7fafc01a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01027d0 0x7fafc01a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 --2- 172.21.15.94:0/3565471865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc0101d90 0x7fafc01a07a0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 >> 172.21.15.94:0/3565471865 conn(0x7fafc00fd380 msgr2=0x7fafc0103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 shutdown_connections 2024-04-25T13:56:35.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.304+0000 7fafc63f3640 1 -- 172.21.15.94:0/3565471865 wait complete. 2024-04-25T13:56:35.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:56:35.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 -- 172.21.15.94:0/3269337658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0103c30 msgr2=0x7f28c0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 --2- 172.21.15.94:0/3269337658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0103c30 0x7f28c0104040 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009900 tx=0x7f28b002a410 comp rx=0 tx=0).stop 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 -- 172.21.15.94:0/3269337658 shutdown_connections 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 --2- 172.21.15.94:0/3269337658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0108930 0x7f28c010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 --2- 172.21.15.94:0/3269337658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0103c30 0x7f28c0104040 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 --2- 172.21.15.94:0/3269337658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 -- 172.21.15.94:0/3269337658 >> 172.21.15.94:0/3269337658 conn(0x7f28c00fdc00 msgr2=0x7f28c0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 -- 172.21.15.94:0/3269337658 shutdown_connections 2024-04-25T13:56:35.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.442+0000 7f28c866e640 1 -- 172.21.15.94:0/3269337658 wait complete. 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 Processor -- start 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 -- start start 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 0x7f28c01a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010e560 con 0x7f28c0108930 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010e3e0 con 0x7f28c0103c30 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c866e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28c010e6e0 con 0x7f28c01029f0 2024-04-25T13:56:35.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 0x7f28c01a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c5c09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c640a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 0x7f28c01a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34344/0 (socket says 172.21.15.94:34344) 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c5c09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42758/0 (socket says 172.21.15.94:42758) 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1844625833 learned_addr learned my addr 172.21.15.94:0/1844625833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.443+0000 7f28c640a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39008/0 (socket says 172.21.15.94:39008) 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 -- 172.21.15.94:0/1844625833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 msgr2=0x7f28c019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 -- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 msgr2=0x7f28c01a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 0x7f28c01a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 -- 172.21.15.94:0/1844625833 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c01a1d40 con 0x7f28c01029f0 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c5c09640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c640a640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28c01022c0 tx=0x7f28bc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28b77fe640 1 -- 172.21.15.94:0/1844625833 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00cc40 con 0x7f28c01029f0 2024-04-25T13:56:35.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c01a1fd0 con 0x7f28c01029f0 2024-04-25T13:56:35.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28b77fe640 1 -- 172.21.15.94:0/1844625833 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28bc00cde0 con 0x7f28c01029f0 2024-04-25T13:56:35.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.444+0000 7f28b77fe640 1 -- 172.21.15.94:0/1844625833 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc010430 con 0x7f28c01029f0 2024-04-25T13:56:35.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.445+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c01a2490 con 0x7f28c01029f0 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.447+0000 7f28cf601740 1 Processor -- start 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 -- start start 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28b77fe640 1 -- 172.21.15.94:0/1844625833 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f28bc004030 con 0x7f28c01029f0 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 0x5572da35c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f956b0 0x5572da35d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572da2bcd80 con 0x5572d9fb6390 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572da2768b0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28cf601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572da2b8d40 con 0x5572d9f956b0 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c640a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 0x5572da35c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34350/0 (socket says 172.21.15.94:34350) 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1076465201 learned_addr learned my addr 172.21.15.94:0/1076465201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c5c09640 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f956b0 0x5572da35d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1076465201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f956b0 msgr2=0x5572da35d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f956b0 0x5572da35d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1076465201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 msgr2=0x5572da35c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 0x5572da35c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1076465201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572da247a40 con 0x5572d9fb6390 2024-04-25T13:56:35.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.448+0000 7f28c640a640 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 0x5572da35c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011570 tx=0x7f28b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefaa7f5986a9243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28b77fe640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 0x7f28ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28b4bfe640 1 -- 172.21.15.94:0/1076465201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004b20 con 0x5572d9fb6390 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28b77fe640 1 -- 172.21.15.94:0/1844625833 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f28bc0df010 con 0x7f28c01029f0 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28b4bfe640 1 -- 172.21.15.94:0/1076465201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8004cc0 con 0x5572d9fb6390 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.449+0000 7f28c640a640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 0x7f28ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28b4bfe640 1 -- 172.21.15.94:0/1076465201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8005420 con 0x5572d9fb6390 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 -- 172.21.15.94:0/1076465201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 msgr2=0x5572da35be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011570 tx=0x7f28b8011a30 comp rx=0 tx=0).stop 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28c640a640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 0x7f28ac0b2000 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5572da35bb90 tx=0x7f28bc0e2af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 -- 172.21.15.94:0/1076465201 shutdown_connections 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f956b0 0x5572da35d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f8d4d0 0x5572da35c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 --2- 172.21.15.94:0/1076465201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da35be70 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 -- 172.21.15.94:0/1076465201 >> 172.21.15.94:0/1076465201 conn(0x5572d9faa4e0 msgr2=0x5572da353270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 -- 172.21.15.94:0/1076465201 shutdown_connections 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.450+0000 7f28cf601740 1 -- 172.21.15.94:0/1076465201 wait complete. 2024-04-25T13:56:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28cf601740 1 -- start start 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28cf601740 1 Processor -- start 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28cf601740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28cf601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da366170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28cf601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9fb6390 0x5572da36d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.451+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34356/0 (socket says 172.21.15.94:34356) 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572da35dc30 con 0x5572d9f8d4d0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572da243d10 con 0x5572d9f956b0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1073128231 learned_addr learned my addr 172.21.15.94:0/1073128231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572da35ddb0 con 0x5572d9fb6390 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28cf601740 10 client.0 _ll_register_callbacks cb 0x5572da270210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c5c09640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da366170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9fb6390 0x5572da36d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1073128231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9fb6390 msgr2=0x5572da36d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9fb6390 0x5572da36d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 msgr2=0x5572da366170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da366170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572da0987d0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f28b801b040 tx=0x7f28b800fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0f76f21259fa2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b802b020 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8004cc0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.452+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f28b8005330 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28b8036050 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 10 client.20865 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5572d9fce3f0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8004cc0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f28b8027960 con 0x5572d9f8d4d0 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 10 client.20865 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 10 client.20865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 20 client.20865 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f74f60 0x5572da373470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 4 client.20865 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572da2ca990 con 0x5572d9f74f60 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.453+0000 7f28c5c09640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f74f60 0x5572da373470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.454+0000 7f28a9ffb640 10 client.20865 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.454+0000 7f28c5c09640 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f74f60 0x5572da373470 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c42ef38ebc5bd4e server_cookie=a2398cb257a4750b in_seq=0 out_seq=0 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.456+0000 7f28a9ffb640 1 -- 172.21.15.94:0/1073128231 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572da2ca990 con 0x5572d9f74f60 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.456+0000 7f28a9ffb640 10 client.20865 handle_command_reply: tid=0 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.456+0000 7f28cf601740 1 client.20865 shutdown 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 msgr2=0x5572da365c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f28b801b040 tx=0x7f28b800fd80 comp rx=0 tx=0).stop 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 shutdown_connections 2024-04-25T13:56:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f74f60 0x5572da373470 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9fb6390 0x5572da36d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da366170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 --2- 172.21.15.94:0/1073128231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9f8d4d0 0x5572da365c30 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28a9ffb640 0 client.20865 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 >> 172.21.15.94:0/1073128231 conn(0x5572d9faa4e0 msgr2=0x5572da3523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 shutdown_connections 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 1 -- 172.21.15.94:0/1073128231 wait complete. 2024-04-25T13:56:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.457+0000 7f28cf601740 20 client.20865 trim_cache size 0 max 16384 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 Processor -- start 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 -- start start 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da3733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da360890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01a9270 con 0x5572d9fb6390 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01aae70 con 0x5572d9f956b0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28cf601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01a9940 con 0x5572d9f8d4d0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34368/0 (socket says 172.21.15.94:34368) 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2593535568 learned_addr learned my addr 172.21.15.94:0/2593535568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da3733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.478+0000 7f28c5c09640 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da360890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2593535568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 msgr2=0x5572da3733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da3733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2593535568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 msgr2=0x5572da360890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da360890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2593535568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d9f8e0f0 con 0x5572d9fb6390 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011570 tx=0x7f28b8005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a59d42f958c00e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28abfff640 1 -- 172.21.15.94:0/2593535568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b802b040 con 0x5572d9fb6390 2024-04-25T13:56:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28abfff640 1 -- 172.21.15.94:0/2593535568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8004e50 con 0x5572d9fb6390 2024-04-25T13:56:35.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.479+0000 7f28abfff640 1 -- 172.21.15.94:0/2593535568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80056d0 con 0x5572d9fb6390 2024-04-25T13:56:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- 172.21.15.94:0/2593535568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 msgr2=0x5572da360dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011570 tx=0x7f28b8005cf0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- 172.21.15.94:0/2593535568 shutdown_connections 2024-04-25T13:56:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da360dd0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da360890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 --2- 172.21.15.94:0/2593535568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da3733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- 172.21.15.94:0/2593535568 >> 172.21.15.94:0/2593535568 conn(0x5572d9faa4e0 msgr2=0x5572da35be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- 172.21.15.94:0/2593535568 shutdown_connections 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- 172.21.15.94:0/2593535568 wait complete. 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 -- start start 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.480+0000 7f28cf601740 1 Processor -- start 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da369310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da3699b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01aa7a0 con 0x5572d9fb6390 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01aa010 con 0x5572d9f956b0 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01a7250 con 0x5572d9f8d4d0 2024-04-25T13:56:35.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 10 client.0 _ll_register_callbacks cb 0x7f28b8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39038/0 (socket says 172.21.15.94:39038) 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c5c09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da369310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2086726749 learned_addr learned my addr 172.21.15.94:0/2086726749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da3699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 msgr2=0x5572da369310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da369310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2086726749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 msgr2=0x5572da3699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da3699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 -- 172.21.15.94:0/2086726749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572da27d4b0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28c6c0b640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28b800f990 tx=0x7f28b801b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eedc3605bb6a5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:35.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b803e030 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8004e50 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.481+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7f28b80058c0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28b8005ac0 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 10 client.30019 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5572da3da950 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8004e50 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f28b8025850 con 0x5572d9f8d4d0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 10 client.30019 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 10 client.30019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 20 client.30019 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f4a3a0 0x5572da3dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 4 client.30019 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572da2c4800 con 0x5572d9f4a3a0 2024-04-25T13:56:35.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.482+0000 7f28c5c09640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f4a3a0 0x5572da3dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.483+0000 7f28c5c09640 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f4a3a0 0x5572da3dd280 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830411d2f44e71ed server_cookie=7e0081232e3c4841 in_seq=0 out_seq=0 2024-04-25T13:56:35.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.483+0000 7f28a9ffb640 10 client.30019 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28a9ffb640 1 -- 172.21.15.94:0/2086726749 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5572da2c4800 con 0x5572d9f4a3a0 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28a9ffb640 10 client.30019 handle_command_reply: tid=0 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 client.30019 shutdown 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 msgr2=0x5572da36c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28b800f990 tx=0x7f28b801b040 comp rx=0 tx=0).stop 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 shutdown_connections 2024-04-25T13:56:35.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d9f4a3a0 0x5572da3dd280 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d9fb6390 0x5572da3699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28a9ffb640 0 client.30019 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d9f956b0 0x5572da369310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 --2- 172.21.15.94:0/2086726749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d9f8d4d0 0x5572da36c790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 >> 172.21.15.94:0/2086726749 conn(0x5572d9faa4e0 msgr2=0x5572da35bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 shutdown_connections 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 1 -- 172.21.15.94:0/2086726749 wait complete. 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.487+0000 7f28cf601740 20 client.30019 trim_cache size 0 max 16384 2024-04-25T13:56:35.490 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (309 inodes in the stack)", 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "bd670531-51b6-47f0-84cb-07060f405538": { 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "bd670531-51b6-47f0-84cb-07060f405538", 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:56:35.491 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 msgr2=0x7f28ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 0x7f28ac0b2000 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5572da35bb90 tx=0x7f28bc0e2af0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 msgr2=0x7f28c006bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28c01022c0 tx=0x7f28bc00ee50 comp rx=0 tx=0).stop 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 shutdown_connections 2024-04-25T13:56:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f28ac0afaf0 0x7f28ac0b2000 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28c0108930 0x7f28c01a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28c0103c30 0x7f28c019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 --2- 172.21.15.94:0/1844625833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c01029f0 0x7f28c006bc50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 >> 172.21.15.94:0/1844625833 conn(0x7f28c00fdc00 msgr2=0x7f28c0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 shutdown_connections 2024-04-25T13:56:35.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:56:35.488+0000 7f28c866e640 1 -- 172.21.15.94:0/1844625833 wait complete. 2024-04-25T13:56:35.501 INFO:tasks.cephfs.filesystem:scrub status for tag:bd670531-51b6-47f0-84cb-07060f405538 - {'path': '/', 'tag': 'bd670531-51b6-47f0-84cb-07060f405538', 'options': 'recursive,force'} 2024-04-25T13:56:36.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 29316 22.53 MB/sec warmup 78 sec latency 67.576 ms 2024-04-25T13:56:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: pgmap v793: 97 pgs: 97 active+clean; 234 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 559 op/s 2024-04-25T13:56:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:56:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:36.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:36 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: pgmap v793: 97 pgs: 97 active+clean; 234 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 559 op/s 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:36.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:36 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: pgmap v793: 97 pgs: 97 active+clean; 234 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 559 op/s 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:36.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:36.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:36 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 29682 22.37 MB/sec warmup 79 sec latency 194.947 ms 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:56:37.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:37 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:37.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:37.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:56:37.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:37 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:37.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:56:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:37 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:56:38.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 30092 22.32 MB/sec warmup 80 sec latency 66.901 ms 2024-04-25T13:56:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:38 smithi094 ceph-mon[29195]: pgmap v794: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-04-25T13:56:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:38 smithi138 ceph-mon[32977]: pgmap v794: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-04-25T13:56:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:38 smithi137 ceph-mon[32568]: pgmap v794: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 506 op/s 2024-04-25T13:56:39.136 INFO:tasks.workunit.client.0.smithi094.stdout: 10 30638 22.49 MB/sec warmup 81 sec latency 39.679 ms 2024-04-25T13:56:40.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 31122 22.58 MB/sec warmup 82 sec latency 67.722 ms 2024-04-25T13:56:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:40 smithi094 ceph-mon[29195]: pgmap v795: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 511 op/s 2024-04-25T13:56:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:40 smithi138 ceph-mon[32977]: pgmap v795: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 511 op/s 2024-04-25T13:56:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:40 smithi137 ceph-mon[32568]: pgmap v795: 97 pgs: 97 active+clean; 237 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 511 op/s 2024-04-25T13:56:41.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 31539 22.58 MB/sec warmup 83 sec latency 79.842 ms 2024-04-25T13:56:41.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:41 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:41.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:41 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:41.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:41 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:41.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:41 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 -- 172.21.15.94:0/2395886581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 msgr2=0x7fd3d01064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 --2- 172.21.15.94:0/2395886581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d01064e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ca0 tx=0x7fd3c802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 -- 172.21.15.94:0/2395886581 shutdown_connections 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 --2- 172.21.15.94:0/2395886581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d01095e0 0x7fd3d010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 --2- 172.21.15.94:0/2395886581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d0106ab0 0x7fd3d0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 --2- 172.21.15.94:0/2395886581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d01064e0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 -- 172.21.15.94:0/2395886581 >> 172.21.15.94:0/2395886581 conn(0x7fd3d00fda40 msgr2=0x7fd3d00ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 -- 172.21.15.94:0/2395886581 shutdown_connections 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.567+0000 7fd3d5413640 1 -- 172.21.15.94:0/2395886581 wait complete. 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 Processor -- start 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 -- start start 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0106ab0 0x7fd3d019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 0x7fd3d01a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f420 con 0x7fd3d01040c0 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f2a0 con 0x7fd3d01095e0 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3d5413640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010f5a0 con 0x7fd3d0106ab0 2024-04-25T13:56:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34392/0 (socket says 172.21.15.94:34392) 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3152615375 learned_addr learned my addr 172.21.15.94:0/3152615375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.568+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 0x7fd3d01a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0106ab0 0x7fd3d019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3152615375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0106ab0 msgr2=0x7fd3d019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0106ab0 0x7fd3d019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3152615375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 msgr2=0x7fd3d01a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 0x7fd3d01a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3152615375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01a1b10 con 0x7fd3d01040c0 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 0x7fd3d01a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd3c802ade0 tx=0x7fd3c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3152615375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802f380 con 0x7fd3d01040c0 2024-04-25T13:56:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a1da0 con 0x7fd3d01040c0 2024-04-25T13:56:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3152615375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c80317c0 con 0x7fd3d01040c0 2024-04-25T13:56:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3152615375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802fc60 con 0x7fd3d01040c0 2024-04-25T13:56:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.569+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a2210 con 0x7fd3d01040c0 2024-04-25T13:56:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.571+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3152615375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd3c8004040 con 0x7fd3d01040c0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 Processor -- start 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 -- start start 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb701b0 0x556addf14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb4f4d0 0x556addf1c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556adde76460 con 0x556addb472f0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556adde30590 con 0x556addb4f4d0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556adde72420 con 0x556addb701b0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3b7fff640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 0x7fd3b80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 0x7fd3b80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34396/0 (socket says 172.21.15.94:34396) 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3ceffd640 1 -- 172.21.15.94:0/360550696 learned_addr learned my addr 172.21.15.94:0/360550696 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb701b0 0x556addf14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb4f4d0 0x556addf1c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.572+0000 7fd3b7fff640 1 -- 172.21.15.94:0/3152615375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd3c8100240 con 0x7fd3d01040c0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 -- 172.21.15.94:0/360550696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb701b0 msgr2=0x556addf14f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb701b0 0x556addf14f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 -- 172.21.15.94:0/360550696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb4f4d0 msgr2=0x556addf1c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb4f4d0 0x556addf1c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 -- 172.21.15.94:0/360550696 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adde3d0d0 con 0x556addb472f0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 0x7fd3b80b2190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x556addf160f0 tx=0x7fd3c0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd3c808f010 tx=0x7fd3c80f6dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e54faad410f86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3b57fa640 1 -- 172.21.15.94:0/360550696 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80c7fd0 con 0x556addb472f0 2024-04-25T13:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3b57fa640 1 -- 172.21.15.94:0/360550696 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c8036070 con 0x556addb472f0 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.573+0000 7fd3b57fa640 1 -- 172.21.15.94:0/360550696 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80c7df0 con 0x556addb472f0 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- 172.21.15.94:0/360550696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 msgr2=0x556addf154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd3c808f010 tx=0x7fd3c80f6dd0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- 172.21.15.94:0/360550696 shutdown_connections 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb4f4d0 0x556addf1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb472f0 0x556addf154a0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/360550696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb701b0 0x556addf14f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- 172.21.15.94:0/360550696 >> 172.21.15.94:0/360550696 conn(0x556addb64300 msgr2=0x556addf0c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- 172.21.15.94:0/360550696 shutdown_connections 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- 172.21.15.94:0/360550696 wait complete. 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 -- start start 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.574+0000 7fd3dc39d740 1 Processor -- start 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf1e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556adde76460 con 0x556addb701b0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556adddfdb60 con 0x556addb472f0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556addf1cbb0 con 0x556addb4f4d0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3dc39d740 10 client.0 _ll_register_callbacks cb 0x556adde222b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34410/0 (socket says 172.21.15.94:34410) 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/87395870 learned_addr learned my addr 172.21.15.94:0/87395870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf1e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/87395870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 msgr2=0x556addf1ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf1ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 msgr2=0x556addf1e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf1e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/87395870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556addb88210 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0004250 tx=0x7fd3c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6063b9dce52e42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.575+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c001d400 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd3c001d5a0 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c8036070 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3c001d7a0 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 10 client.20883 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556addc54620 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c8036070 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd3c001c070 con 0x556addb701b0 2024-04-25T13:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 10 client.20883 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 10 client.20883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 20 client.20883 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb2ed80 0x556addf2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 4 client.20883 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556adde84070 con 0x556addb2ed80 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.576+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb2ed80 0x556addf2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.577+0000 7fd3b2ff5640 10 client.20883 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.577+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb2ed80 0x556addf2c6d0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7f61ccd5e4c43 server_cookie=969230ed554e1325 in_seq=0 out_seq=0 2024-04-25T13:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/87395870 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556adde84070 con 0x556addb2ed80 2024-04-25T13:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3b2ff5640 10 client.20883 handle_command_reply: tid=0 2024-04-25T13:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 client.20883 shutdown 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 msgr2=0x556addf25c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0004250 tx=0x7fd3c0004730 comp rx=0 tx=0).stop 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 shutdown_connections 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb2ed80 0x556addf2c6d0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb701b0 0x556addf25c60 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3b2ff5640 0 client.20883 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/87395870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf1e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 >> 172.21.15.94:0/87395870 conn(0x556addb64300 msgr2=0x556addf0f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 shutdown_connections 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 1 -- 172.21.15.94:0/87395870 wait complete. 2024-04-25T13:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.580+0000 7fd3dc39d740 20 client.20883 trim_cache size 0 max 16384 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 Processor -- start 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 -- start start 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb2ed80 0x556addf196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb472f0 0x556addf25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01a90b0 con 0x556addb4f4d0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01aacb0 con 0x556addb2ed80 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01a9780 con 0x556addb472f0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.595+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34422/0 (socket says 172.21.15.94:34422) 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb2ed80 0x556addf196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/4147630016 learned_addr learned my addr 172.21.15.94:0/4147630016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb472f0 0x556addf25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/4147630016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb472f0 msgr2=0x556addf25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb472f0 0x556addf25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/4147630016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb2ed80 msgr2=0x556addf196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb2ed80 0x556addf196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 -- 172.21.15.94:0/4147630016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556addf1f3c0 con 0x556addb4f4d0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0014040 tx=0x7fd3c000ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8dd623f3961d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3b4ff9640 1 -- 172.21.15.94:0/4147630016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c001d400 con 0x556addb4f4d0 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3b4ff9640 1 -- 172.21.15.94:0/4147630016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c001d5a0 con 0x556addb4f4d0 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.596+0000 7fd3b4ff9640 1 -- 172.21.15.94:0/4147630016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0005280 con 0x556addb4f4d0 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- 172.21.15.94:0/4147630016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 msgr2=0x556addf31e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0014040 tx=0x7fd3c000ae10 comp rx=0 tx=0).stop 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- 172.21.15.94:0/4147630016 shutdown_connections 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb4f4d0 0x556addf31e90 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb472f0 0x556addf25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/4147630016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb2ed80 0x556addf196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- 172.21.15.94:0/4147630016 >> 172.21.15.94:0/4147630016 conn(0x556addb64300 msgr2=0x556addf1cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- 172.21.15.94:0/4147630016 shutdown_connections 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- 172.21.15.94:0/4147630016 wait complete. 2024-04-25T13:56:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- start start 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 Processor -- start 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf92a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01aa5e0 con 0x556addb2ed80 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01a9e50 con 0x556addb472f0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01a7090 con 0x556addb4f4d0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.597+0000 7fd3dc39d740 10 client.0 _ll_register_callbacks cb 0x7fd3c000aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34430/0 (socket says 172.21.15.94:34430) 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3260324611 learned_addr learned my addr 172.21.15.94:0/3260324611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3cf7fe640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf92a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3260324611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 msgr2=0x556addf92a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf92a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 msgr2=0x556addf25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf25230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 -- 172.21.15.94:0/3260324611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556addf93080 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3ceffd640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000f80 tx=0x7fd3c80c8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c15f08ad86f07a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80c7fd0 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fd3c80b10d0 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c001d5a0 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3c80c7df0 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3dc39d740 10 client.20889 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556addf94320 con 0x556addb2ed80 2024-04-25T13:56:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.598+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c001d5a0 con 0x556addb2ed80 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fd3c8004330 con 0x556addb2ed80 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 10 client.20889 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 10 client.20889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 20 client.20889 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb701b0 0x556addf96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 4 client.20889 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556adde7dee0 con 0x556addb701b0 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb701b0 0x556addf96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3b2ff5640 10 client.20889 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:41.599+0000 7fd3ce7fc640 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb701b0 0x556addf96d30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a721f3f4b16b68 server_cookie=e8da1d4cec0ea03e in_seq=0 out_seq=0 2024-04-25T13:56:41.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:41 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:41.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:41 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:42.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 31703 22.47 MB/sec warmup 84 sec latency 491.151 ms 2024-04-25T13:56:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3b2ff5640 1 -- 172.21.15.94:0/3260324611 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556adde7dee0 con 0x556addb701b0 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3b2ff5640 10 client.20889 handle_command_reply: tid=0 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 client.20889 shutdown 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 msgr2=0x556addf24cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000f80 tx=0x7fd3c80c8190 comp rx=0 tx=0).stop 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 shutdown_connections 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556addb701b0 0x556addf96d30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556addb4f4d0 0x556addf92a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556addb472f0 0x556addf25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 --2- 172.21.15.94:0/3260324611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556addb2ed80 0x556addf24cf0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3b2ff5640 0 client.20889 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 >> 172.21.15.94:0/3260324611 conn(0x556addb64300 msgr2=0x556addf1d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 shutdown_connections 2024-04-25T13:56:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.155+0000 7fd3dc39d740 1 -- 172.21.15.94:0/3260324611 wait complete. 2024-04-25T13:56:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3dc39d740 20 client.20889 trim_cache size 0 max 16384 2024-04-25T13:56:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:56:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:56:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:56:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 msgr2=0x7fd3b80b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 0x7fd3b80b2190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x556addf160f0 tx=0x7fd3c0005d20 comp rx=0 tx=0).stop 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 msgr2=0x7fd3d019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd3c802ade0 tx=0x7fd3c8003ba0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 shutdown_connections 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3b80afc80 0x7fd3b80b2190 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3d01095e0 0x7fd3d01a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0106ab0 0x7fd3d019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.156+0000 7fd3d5413640 1 --2- 172.21.15.94:0/3152615375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01040c0 0x7fd3d019a080 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.157+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 >> 172.21.15.94:0/3152615375 conn(0x7fd3d00fda40 msgr2=0x7fd3d0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.157+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 shutdown_connections 2024-04-25T13:56:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:42.157+0000 7fd3d5413640 1 -- 172.21.15.94:0/3152615375 wait complete. 2024-04-25T13:56:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:42 smithi094 ceph-mon[29195]: pgmap v796: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 615 op/s 2024-04-25T13:56:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:56:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:42 smithi138 ceph-mon[32977]: pgmap v796: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 615 op/s 2024-04-25T13:56:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:42.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:56:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:42 smithi137 ceph-mon[32568]: pgmap v796: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.0 MiB/s wr, 615 op/s 2024-04-25T13:56:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:56:43.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 32017 22.36 MB/sec warmup 85 sec latency 554.017 ms 2024-04-25T13:56:44.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 32314 22.30 MB/sec warmup 86 sec latency 126.774 ms 2024-04-25T13:56:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:44 smithi094 ceph-mon[29195]: pgmap v797: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 370 op/s 2024-04-25T13:56:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:44 smithi138 ceph-mon[32977]: pgmap v797: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 370 op/s 2024-04-25T13:56:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:44 smithi137 ceph-mon[32568]: pgmap v797: 97 pgs: 97 active+clean; 244 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 370 op/s 2024-04-25T13:56:45.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 32647 22.28 MB/sec warmup 87 sec latency 117.582 ms 2024-04-25T13:56:46.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 33127 22.22 MB/sec warmup 88 sec latency 77.646 ms 2024-04-25T13:56:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:46 smithi094 ceph-mon[29195]: pgmap v798: 97 pgs: 97 active+clean; 241 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2024-04-25T13:56:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:46 smithi138 ceph-mon[32977]: pgmap v798: 97 pgs: 97 active+clean; 241 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2024-04-25T13:56:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:46 smithi137 ceph-mon[32568]: pgmap v798: 97 pgs: 97 active+clean; 241 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 622 op/s 2024-04-25T13:56:47.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 33387 22.13 MB/sec warmup 89 sec latency 135.958 ms 2024-04-25T13:56:48.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 33866 22.13 MB/sec warmup 90 sec latency 67.276 ms 2024-04-25T13:56:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:48 smithi094 ceph-mon[29195]: pgmap v799: 97 pgs: 97 active+clean; 237 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 588 op/s 2024-04-25T13:56:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:48 smithi138 ceph-mon[32977]: pgmap v799: 97 pgs: 97 active+clean; 237 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 588 op/s 2024-04-25T13:56:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:48 smithi137 ceph-mon[32568]: pgmap v799: 97 pgs: 97 active+clean; 237 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 588 op/s 2024-04-25T13:56:49.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 34252 22.17 MB/sec warmup 91 sec latency 87.176 ms 2024-04-25T13:56:50.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 34669 22.23 MB/sec warmup 92 sec latency 103.803 ms 2024-04-25T13:56:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:50 smithi094 ceph-mon[29195]: pgmap v800: 97 pgs: 97 active+clean; 232 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T13:56:50.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:50 smithi138 ceph-mon[32977]: pgmap v800: 97 pgs: 97 active+clean; 232 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T13:56:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:50 smithi137 ceph-mon[32568]: pgmap v800: 97 pgs: 97 active+clean; 232 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T13:56:51.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 35057 22.21 MB/sec warmup 93 sec latency 103.546 ms 2024-04-25T13:56:52.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 35292 22.10 MB/sec warmup 94 sec latency 283.459 ms 2024-04-25T13:56:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:52 smithi094 ceph-mon[29195]: pgmap v801: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-25T13:56:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:52 smithi138 ceph-mon[32977]: pgmap v801: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-25T13:56:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:52 smithi137 ceph-mon[32568]: pgmap v801: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-25T13:56:53.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 35532 21.96 MB/sec warmup 95 sec latency 370.145 ms 2024-04-25T13:56:54.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 35889 21.96 MB/sec warmup 96 sec latency 92.253 ms 2024-04-25T13:56:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:54 smithi094 ceph-mon[29195]: pgmap v802: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-04-25T13:56:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:54 smithi138 ceph-mon[32977]: pgmap v802: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-04-25T13:56:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:54 smithi137 ceph-mon[32568]: pgmap v802: 97 pgs: 97 active+clean; 240 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-04-25T13:56:55.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 36266 21.97 MB/sec warmup 97 sec latency 83.916 ms 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- 172.21.15.94:0/90396315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60109a70 msgr2=0x7fcf6010be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 --2- 172.21.15.94:0/90396315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60109a70 0x7fcf6010be90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009ca0 tx=0x7fcf5802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- 172.21.15.94:0/90396315 shutdown_connections 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 --2- 172.21.15.94:0/90396315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60109a70 0x7fcf6010be90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 --2- 172.21.15.94:0/90396315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60104d70 0x7fcf60105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 --2- 172.21.15.94:0/90396315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60103b30 0x7fcf60104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- 172.21.15.94:0/90396315 >> 172.21.15.94:0/90396315 conn(0x7fcf600fed00 msgr2=0x7fcf601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- 172.21.15.94:0/90396315 shutdown_connections 2024-04-25T13:56:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- 172.21.15.94:0/90396315 wait complete. 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 Processor -- start 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.262+0000 7fcf6729f640 1 -- start start 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60103b30 0x7fcf6019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60109a70 0x7fcf601a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f660 con 0x7fcf60104d70 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f4e0 con 0x7fcf60103b30 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f7e0 con 0x7fcf60109a70 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41830/0 (socket says 172.21.15.94:41830) 2024-04-25T13:56:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60103b30 0x7fcf6019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 -- 172.21.15.94:0/401980563 learned_addr learned my addr 172.21.15.94:0/401980563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6583c640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60109a70 0x7fcf601a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 -- 172.21.15.94:0/401980563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60109a70 msgr2=0x7fcf601a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60109a70 0x7fcf601a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 -- 172.21.15.94:0/401980563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60103b30 msgr2=0x7fcf6019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60103b30 0x7fcf6019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 -- 172.21.15.94:0/401980563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601a1d10 con 0x7fcf60104d70 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6483a640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf567fc640 1 -- 172.21.15.94:0/401980563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000cd80 con 0x7fcf60104d70 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf567fc640 1 -- 172.21.15.94:0/401980563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf500108e0 con 0x7fcf60104d70 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a1fa0 con 0x7fcf60104d70 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf567fc640 1 -- 172.21.15.94:0/401980563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50018980 con 0x7fcf60104d70 2024-04-25T13:56:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.263+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a2490 con 0x7fcf60104d70 2024-04-25T13:56:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 Processor -- start 2024-04-25T13:56:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf567fc640 1 -- 172.21.15.94:0/401980563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcf50010430 con 0x7fcf60104d70 2024-04-25T13:56:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 -- start start 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea71a6a460 con 0x55ea717641b0 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ea71a24590 con 0x55ea7173b2f0 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea71a66420 con 0x55ea717434d0 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41846/0 (socket says 172.21.15.94:41846) 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54982/0 (socket says 172.21.15.94:54982) 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.266+0000 7fcf6583c640 1 -- 172.21.15.94:0/3869917531 learned_addr learned my addr 172.21.15.94:0/3869917531 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6483a640 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 -- 172.21.15.94:0/3869917531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 msgr2=0x55ea71b0bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 -- 172.21.15.94:0/3869917531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 msgr2=0x55ea71b04ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 -- 172.21.15.94:0/3869917531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea71a310d0 con 0x55ea717641b0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf567fc640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 0x7fcf4c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fcf58035000 tx=0x7fcf5802f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a7a46399ade39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf6503b640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 0x7fcf4c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf567fc640 1 -- 172.21.15.94:0/401980563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf500def90 con 0x7fcf60104d70 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf4b7fe640 1 -- 172.21.15.94:0/3869917531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58031650 con 0x55ea717641b0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf4b7fe640 1 -- 172.21.15.94:0/3869917531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58036030 con 0x55ea717641b0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.267+0000 7fcf4b7fe640 1 -- 172.21.15.94:0/3869917531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580047a0 con 0x55ea717641b0 2024-04-25T13:56:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6503b640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 0x7fcf4c0b1ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55ea71b05d50 tx=0x7fcf5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3869917531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 msgr2=0x55ea71b04970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fcf58035000 tx=0x7fcf5802f410 comp rx=0 tx=0).stop 2024-04-25T13:56:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3869917531 shutdown_connections 2024-04-25T13:56:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b04ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3869917531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b04970 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3869917531 >> 172.21.15.94:0/3869917531 conn(0x55ea71758300 msgr2=0x55ea71b00300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3869917531 shutdown_connections 2024-04-25T13:56:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.268+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3869917531 wait complete. 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 -- start start 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 Processor -- start 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea71b0c660 con 0x55ea717641b0 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ea719f1b60 con 0x55ea7173b2f0 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea71b0c7e0 con 0x55ea717434d0 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6e22d740 10 client.0 _ll_register_callbacks cb 0x55ea71a162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41862/0 (socket says 172.21.15.94:41862) 2024-04-25T13:56:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6583c640 1 -- 172.21.15.94:0/1792332609 learned_addr learned my addr 172.21.15.94:0/1792332609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6483a640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.269+0000 7fcf6503b640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 -- 172.21.15.94:0/1792332609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 msgr2=0x55ea71b0ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 -- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 msgr2=0x55ea71b0e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b0e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 -- 172.21.15.94:0/1792332609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea7177c210 con 0x55ea717641b0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6583c640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009bf0 tx=0x7fcf58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdb5f894f4d5879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5804a040 con 0x55ea717641b0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf58003b70 con 0x55ea717641b0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58036030 con 0x55ea717641b0 2024-04-25T13:56:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf58009360 con 0x55ea717641b0 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6e22d740 10 client.20901 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea71848620 con 0x55ea717641b0 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.270+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58036030 con 0x55ea717641b0 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcf5804d030 con 0x55ea717641b0 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 10 client.20901 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 10 client.20901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 20 client.20901 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea71722d80 0x55ea71b1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 4 client.20901 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea71a78070 con 0x55ea71722d80 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6503b640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea71722d80 0x55ea71b1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf48ff9640 10 client.20901 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.271+0000 7fcf6503b640 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea71722d80 0x55ea71b1bff0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83bca50d9d3bf2d server_cookie=c281812cfe8cb603 in_seq=0 out_seq=0 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf48ff9640 1 -- 172.21.15.94:0/1792332609 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea71a78070 con 0x55ea71722d80 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf48ff9640 10 client.20901 handle_command_reply: tid=0 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 client.20901 shutdown 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 msgr2=0x55ea71b159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009bf0 tx=0x7fcf58004750 comp rx=0 tx=0).stop 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 shutdown_connections 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea71722d80 0x55ea71b1bff0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717641b0 0x55ea71b159f0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf48ff9640 0 client.20901 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea717434d0 0x55ea71b0ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/1792332609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea7173b2f0 0x55ea71b0e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 >> 172.21.15.94:0/1792332609 conn(0x55ea71758300 msgr2=0x55ea71aff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 shutdown_connections 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 1 -- 172.21.15.94:0/1792332609 wait complete. 2024-04-25T13:56:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.275+0000 7fcf6e22d740 20 client.20901 trim_cache size 0 max 16384 2024-04-25T13:56:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 Processor -- start 2024-04-25T13:56:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 -- start start 2024-04-25T13:56:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 0x55ea71b15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ab4e0 con 0x55ea717434d0 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ad0e0 con 0x55ea71722d80 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.291+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601abbb0 con 0x55ea7173b2f0 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41872/0 (socket says 172.21.15.94:41872) 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54992/0 (socket says 172.21.15.94:54992) 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6483a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 0x55ea71b15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 -- 172.21.15.94:0/3339640294 learned_addr learned my addr 172.21.15.94:0/3339640294 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 -- 172.21.15.94:0/3339640294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 msgr2=0x55ea71b15910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 0x55ea71b15910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 -- 172.21.15.94:0/3339640294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 msgr2=0x55ea71b0ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 -- 172.21.15.94:0/3339640294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea71b086a0 con 0x55ea717434d0 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6483a640 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 0x55ea71b15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000f80 tx=0x7fcf5802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5fe1c0e23dc918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf4affd640 1 -- 172.21.15.94:0/3339640294 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009360 con 0x55ea717434d0 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf4affd640 1 -- 172.21.15.94:0/3339640294 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58036070 con 0x55ea717434d0 2024-04-25T13:56:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.292+0000 7fcf4affd640 1 -- 172.21.15.94:0/3339640294 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58030b50 con 0x55ea717434d0 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3339640294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 msgr2=0x55ea71b0cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000f80 tx=0x7fcf5802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3339640294 shutdown_connections 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea717434d0 0x55ea71b0cbb0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea7173b2f0 0x55ea71b15910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3339640294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea71722d80 0x55ea71b0ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3339640294 >> 172.21.15.94:0/3339640294 conn(0x55ea71758300 msgr2=0x55ea71b0e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3339640294 shutdown_connections 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.293+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3339640294 wait complete. 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 -- start start 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 Processor -- start 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 0x55ea71b13390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea717434d0 0x55ea71b142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aca10 con 0x55ea7173b2f0 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ac280 con 0x55ea717434d0 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a94c0 con 0x55ea71722d80 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6e22d740 10 client.0 _ll_register_callbacks cb 0x7fcf5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:56:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6483a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 0x55ea71b13390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6503b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6483a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 0x55ea71b13390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41878/0 (socket says 172.21.15.94:41878) 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6483a640 1 -- 172.21.15.94:0/3833067156 learned_addr learned my addr 172.21.15.94:0/3833067156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:33238/0 (socket says 172.21.15.94:33238) 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.294+0000 7fcf6583c640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea717434d0 0x55ea71b142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 -- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea717434d0 msgr2=0x55ea71b142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea717434d0 0x55ea71b142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 -- 172.21.15.94:0/3833067156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 msgr2=0x55ea71b13390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 0x55ea71b13390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 -- 172.21.15.94:0/3833067156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea71b823d0 con 0x55ea71722d80 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6503b640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d50 tx=0x7fcf5c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9871f3b460d4532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c02ba50 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58036070 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8385+0+0 (secure 0 0 0) 0x7fcf5c0035f0 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c0037f0 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6e22d740 10 client.30052 fetch_fsmap learned FSMap version 32 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.295+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea71b157f0 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58036070 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7fcf5c034ab0 con 0x55ea71722d80 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 10 client.30052 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 10 client.30052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 20 client.30052 populate_metadata read hostname 'smithi094' 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea717641b0 0x55ea71b8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 4 client.30052 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea71a71ee0 con 0x55ea717641b0 2024-04-25T13:56:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.296+0000 7fcf6483a640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea717641b0 0x55ea71b8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.297+0000 7fcf48ff9640 10 client.30052 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.297+0000 7fcf6483a640 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea717641b0 0x55ea71b8f110 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7cbea6ff9ebde18 server_cookie=d04a581040914c6a in_seq=0 out_seq=0 2024-04-25T13:56:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf48ff9640 1 -- 172.21.15.94:0/3833067156 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea71a71ee0 con 0x55ea717641b0 2024-04-25T13:56:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf48ff9640 10 client.30052 handle_command_reply: tid=0 2024-04-25T13:56:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 client.30052 shutdown 2024-04-25T13:56:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 msgr2=0x55ea71b12e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d50 tx=0x7fcf5c0046c0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 shutdown_connections 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ea717641b0 0x55ea71b8f110 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea717434d0 0x55ea71b142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7173b2f0 0x55ea71b13390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 --2- 172.21.15.94:0/3833067156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea71722d80 0x55ea71b12e50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 >> 172.21.15.94:0/3833067156 conn(0x55ea71758300 msgr2=0x55ea71b0cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf48ff9640 0 client.30052 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 shutdown_connections 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 1 -- 172.21.15.94:0/3833067156 wait complete. 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.740+0000 7fcf6e22d740 20 client.30052 trim_cache size 0 max 16384 2024-04-25T13:56:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:56:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:56:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:56:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:56:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 msgr2=0x7fcf4c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 0x7fcf4c0b1ff0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x55ea71b05d50 tx=0x7fcf5c005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 msgr2=0x7fcf6019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 shutdown_connections 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcf4c0afae0 0x7fcf4c0b1ff0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf60109a70 0x7fcf601a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf60104d70 0x7fcf6019a780 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.741+0000 7fcf6729f640 1 --2- 172.21.15.94:0/401980563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf60103b30 0x7fcf6019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.742+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 >> 172.21.15.94:0/401980563 conn(0x7fcf600fed00 msgr2=0x7fcf60105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.742+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 shutdown_connections 2024-04-25T13:56:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:56:55.742+0000 7fcf6729f640 1 -- 172.21.15.94:0/401980563 wait complete. 2024-04-25T13:56:56.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 36579 21.87 MB/sec warmup 98 sec latency 239.973 ms 2024-04-25T13:56:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:56 smithi138 ceph-mon[32977]: pgmap v803: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 612 op/s 2024-04-25T13:56:56.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:56 smithi138 ceph-mon[32977]: daemon mds.f restarted 2024-04-25T13:56:56.247 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:56:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T13:56:56.247 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:56:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T13:56:56.247 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:56:55 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/3682207329 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563efb465a40 con 0x563efb4ada80 2024-04-25T13:56:56.247 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:56:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T13:56:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:56 smithi094 ceph-mon[29195]: pgmap v803: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 612 op/s 2024-04-25T13:56:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:56 smithi094 ceph-mon[29195]: daemon mds.f restarted 2024-04-25T13:56:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:56 smithi137 ceph-mon[32568]: pgmap v803: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 612 op/s 2024-04-25T13:56:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:56 smithi137 ceph-mon[32568]: daemon mds.f restarted 2024-04-25T13:56:57.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 36853 21.77 MB/sec warmup 99 sec latency 83.852 ms 2024-04-25T13:56:57.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:57 smithi094 ceph-mon[29195]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:56:57.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:57 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/2422946890,v1:172.21.15.138:6835/2422946890] up:boot 2024-04-25T13:56:57.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:57 smithi094 ceph-mon[29195]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:56:57.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:57 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:57.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:57 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:56:57.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:57 smithi138 ceph-mon[32977]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:56:57.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:57 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/2422946890,v1:172.21.15.138:6835/2422946890] up:boot 2024-04-25T13:56:57.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:57 smithi138 ceph-mon[32977]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:56:57.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:57 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:57.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:57 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:56:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:57 smithi137 ceph-mon[32568]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:56:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:57 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/2422946890,v1:172.21.15.138:6835/2422946890] up:boot 2024-04-25T13:56:57.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:57 smithi137 ceph-mon[32568]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:56:57.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:57 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:57.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:57 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:56:58.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 37279 21.79 MB/sec warmup 100 sec latency 257.813 ms 2024-04-25T13:56:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:56:58 smithi094 ceph-mon[29195]: pgmap v805: 97 pgs: 97 active+clean; 245 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-04-25T13:56:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:56:58 smithi138 ceph-mon[32977]: pgmap v805: 97 pgs: 97 active+clean; 245 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-04-25T13:56:58.543 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:56:58 smithi137 ceph-mon[32568]: pgmap v805: 97 pgs: 97 active+clean; 245 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-04-25T13:56:59.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 37673 21.79 MB/sec warmup 101 sec latency 148.746 ms 2024-04-25T13:57:00.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 38044 21.80 MB/sec warmup 102 sec latency 89.465 ms 2024-04-25T13:57:00.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:00 smithi094 ceph-mon[29195]: pgmap v806: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 415 op/s 2024-04-25T13:57:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:00 smithi138 ceph-mon[32977]: pgmap v806: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 415 op/s 2024-04-25T13:57:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:00 smithi137 ceph-mon[32568]: pgmap v806: 97 pgs: 97 active+clean; 246 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 415 op/s 2024-04-25T13:57:01.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 38461 21.81 MB/sec warmup 103 sec latency 81.614 ms 2024-04-25T13:57:02.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 38837 21.80 MB/sec warmup 104 sec latency 84.726 ms 2024-04-25T13:57:02.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:02 smithi094 ceph-mon[29195]: pgmap v807: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-04-25T13:57:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:02 smithi138 ceph-mon[32977]: pgmap v807: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-04-25T13:57:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:02 smithi137 ceph-mon[32568]: pgmap v807: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-04-25T13:57:03.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 39108 21.69 MB/sec warmup 105 sec latency 276.568 ms 2024-04-25T13:57:04.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 39480 21.75 MB/sec warmup 106 sec latency 96.957 ms 2024-04-25T13:57:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:04 smithi094 ceph-mon[29195]: pgmap v808: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-25T13:57:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:04 smithi138 ceph-mon[32977]: pgmap v808: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-25T13:57:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:04 smithi137 ceph-mon[32568]: pgmap v808: 97 pgs: 97 active+clean; 236 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-04-25T13:57:05.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 39852 21.71 MB/sec warmup 107 sec latency 68.018 ms 2024-04-25T13:57:05.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:57:05.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 -- 172.21.15.94:0/2530852641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a8108930 msgr2=0x7f00a810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 --2- 172.21.15.94:0/2530852641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a8108930 0x7f00a810ad50 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009ca0 tx=0x7f00a002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 -- 172.21.15.94:0/2530852641 shutdown_connections 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 --2- 172.21.15.94:0/2530852641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a8108930 0x7f00a810ad50 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 --2- 172.21.15.94:0/2530852641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 --2- 172.21.15.94:0/2530852641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a81029f0 0x7f00a81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 -- 172.21.15.94:0/2530852641 >> 172.21.15.94:0/2530852641 conn(0x7f00a80fdc00 msgr2=0x7f00a8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 -- 172.21.15.94:0/2530852641 shutdown_connections 2024-04-25T13:57:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.621+0000 7f00accaf640 1 -- 172.21.15.94:0/2530852641 wait complete. 2024-04-25T13:57:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 Processor -- start 2024-04-25T13:57:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 -- start start 2024-04-25T13:57:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8108930 0x7f00a81a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00a810e520 con 0x7f00a81029f0 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00a810e3a0 con 0x7f00a8103c30 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.622+0000 7f00accaf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00a810e6a0 con 0x7f00a8108930 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60334/0 (socket says 172.21.15.94:60334) 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 -- 172.21.15.94:0/824893910 learned_addr learned my addr 172.21.15.94:0/824893910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f009dd9b640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8108930 0x7f00a81a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 -- 172.21.15.94:0/824893910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8108930 msgr2=0x7f00a81a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8108930 0x7f00a81a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 -- 172.21.15.94:0/824893910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 msgr2=0x7f00a8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 -- 172.21.15.94:0/824893910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81a0c20 con 0x7f00a81029f0 2024-04-25T13:57:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f009dd9b640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00a659c640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f009800e9f0 tx=0x7f009800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f009ffff640 1 -- 172.21.15.94:0/824893910 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009800ccc0 con 0x7f00a81029f0 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f009ffff640 1 -- 172.21.15.94:0/824893910 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00980035f0 con 0x7f00a81029f0 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a81a0f10 con 0x7f00a81029f0 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f009ffff640 1 -- 172.21.15.94:0/824893910 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009800ce60 con 0x7f00a81029f0 2024-04-25T13:57:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.623+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81a1320 con 0x7f00a81029f0 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.626+0000 7f00b3b64740 1 Processor -- start 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.626+0000 7f009ffff640 1 -- 172.21.15.94:0/824893910 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0098010690 con 0x7f00a81029f0 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.626+0000 7f00b3b64740 1 -- start start 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560c0d01bd80 con 0x560c0ccec4d0 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560c0cfd58b0 con 0x560c0ccf46b0 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00b3b64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c0d017d40 con 0x560c0cd15390 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51270/0 (socket says 172.21.15.94:51270) 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f009dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 -- 172.21.15.94:0/459282044 learned_addr learned my addr 172.21.15.94:0/459282044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60340/0 (socket says 172.21.15.94:60340) 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 -- 172.21.15.94:0/459282044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 msgr2=0x560c0d0b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 -- 172.21.15.94:0/459282044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 msgr2=0x560c0d0b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f009ffff640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 -- 172.21.15.94:0/459282044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0cfa6a40 con 0x560c0cd15390 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f009dd9b640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f009ffff640 1 -- 172.21.15.94:0/824893910 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f00980df0d0 con 0x7f00a81029f0 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.627+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00a002f390 tx=0x7f00a002f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77af75ad62188a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.628+0000 7f009dd9b640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00a819a180 tx=0x7f0090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.628+0000 7f009cd99640 1 -- 172.21.15.94:0/459282044 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0031650 con 0x560c0cd15390 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.628+0000 7f009cd99640 1 -- 172.21.15.94:0/459282044 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a0036040 con 0x560c0cd15390 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.628+0000 7f009cd99640 1 -- 172.21.15.94:0/459282044 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a00046c0 con 0x560c0cd15390 2024-04-25T13:57:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- 172.21.15.94:0/459282044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 msgr2=0x560c0d0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00a002f390 tx=0x7f00a002f3c0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- 172.21.15.94:0/459282044 shutdown_connections 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 --2- 172.21.15.94:0/459282044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0cd15390 0x560c0d0b6a60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- 172.21.15.94:0/459282044 >> 172.21.15.94:0/459282044 conn(0x560c0cd094e0 msgr2=0x560c0d0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- 172.21.15.94:0/459282044 shutdown_connections 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- 172.21.15.94:0/459282044 wait complete. 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 -- start start 2024-04-25T13:57:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.629+0000 7f00b3b64740 1 Processor -- start 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560c0d0b8720 con 0x560c0cd15390 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560c0cfa2d10 con 0x560c0ccf46b0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c0d0b88a0 con 0x560c0ccec4d0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00b3b64740 10 client.0 _ll_register_callbacks cb 0x560c0cfcf210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60354/0 (socket says 172.21.15.94:60354) 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f009dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 -- 172.21.15.94:0/695361187 learned_addr learned my addr 172.21.15.94:0/695361187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 -- 172.21.15.94:0/695361187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 msgr2=0x560c0d0c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 -- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 msgr2=0x560c0d0c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 -- 172.21.15.94:0/695361187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0cdf77d0 con 0x560c0cd15390 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f00a002a900 tx=0x7f00a00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ee8f8574688f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f008dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.630+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0052020 con 0x560c0cd15390 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f00a00043e0 con 0x560c0cd15390 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a0036040 con 0x560c0cd15390 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00a005d050 con 0x560c0cd15390 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 10 client.20925 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:05.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c0cd2d3f0 con 0x560c0cd15390 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a0036040 con 0x560c0cd15390 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f00a004d030 con 0x560c0cd15390 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 10 client.20925 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 10 client.20925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 20 client.20925 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0ccd3f60 0x560c0d0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 4 client.20925 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0d029990 con 0x560c0ccd3f60 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.631+0000 7f00a659c640 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0ccd3f60 0x560c0d0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.632+0000 7f008dffb640 10 client.20925 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.632+0000 7f00a659c640 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0ccd3f60 0x560c0d0cdea0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b934ce598eeace5 server_cookie=26f441d8c04d7d56 in_seq=0 out_seq=0 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.633+0000 7f008dffb640 1 -- 172.21.15.94:0/695361187 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c0d029990 con 0x560c0ccd3f60 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.633+0000 7f008dffb640 10 client.20925 handle_command_reply: tid=0 2024-04-25T13:57:05.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.633+0000 7f00b3b64740 1 client.20925 shutdown 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 msgr2=0x560c0d0c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f00a002a900 tx=0x7f00a00093e0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 shutdown_connections 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0ccd3f60 0x560c0d0cdea0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0cd15390 0x560c0d0c7de0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0ccf46b0 0x560c0d0c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f008dffb640 0 client.20925 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 --2- 172.21.15.94:0/695361187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 >> 172.21.15.94:0/695361187 conn(0x560c0cd094e0 msgr2=0x560c0d0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 shutdown_connections 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 1 -- 172.21.15.94:0/695361187 wait complete. 2024-04-25T13:57:05.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.634+0000 7f00b3b64740 20 client.20925 trim_cache size 0 max 16384 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 Processor -- start 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 -- start start 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81a9330 con 0x560c0ccf46b0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81aaf30 con 0x560c0cd094e0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.652+0000 7f00b3b64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81a9a00 con 0x560c0ccec4d0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60358/0 (socket says 172.21.15.94:60358) 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f009dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 -- 172.21.15.94:0/454029205 learned_addr learned my addr 172.21.15.94:0/454029205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a659c640 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 -- 172.21.15.94:0/454029205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 msgr2=0x560c0d0c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 -- 172.21.15.94:0/454029205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 msgr2=0x560c0d0bf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0bf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 -- 172.21.15.94:0/454029205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0cced0f0 con 0x560c0ccf46b0 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f00a0006010 tx=0x7f00a0009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d4801bc0fbc69a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f008ffff640 1 -- 172.21.15.94:0/454029205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0052070 con 0x560c0ccf46b0 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f008ffff640 1 -- 172.21.15.94:0/454029205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a0036070 con 0x560c0ccf46b0 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.653+0000 7f008ffff640 1 -- 172.21.15.94:0/454029205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a004a040 con 0x560c0ccf46b0 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- 172.21.15.94:0/454029205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 msgr2=0x560c0d0d35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f00a0006010 tx=0x7f00a0009450 comp rx=0 tx=0).stop 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- 172.21.15.94:0/454029205 shutdown_connections 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccf46b0 0x560c0d0d35a0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccec4d0 0x560c0d0c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 --2- 172.21.15.94:0/454029205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0bf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- 172.21.15.94:0/454029205 >> 172.21.15.94:0/454029205 conn(0x560c0d0b71f0 msgr2=0x560c0d0b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- 172.21.15.94:0/454029205 shutdown_connections 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- 172.21.15.94:0/454029205 wait complete. 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.654+0000 7f00b3b64740 1 -- start start 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 Processor -- start 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccf46b0 0x560c0d0c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81aa860 con 0x560c0ccec4d0 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81aa0d0 con 0x560c0cd094e0 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81a7310 con 0x560c0ccf46b0 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00b3b64740 10 client.0 _ll_register_callbacks cb 0x7f00980e2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60360/0 (socket says 172.21.15.94:60360) 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 -- 172.21.15.94:0/560031291 learned_addr learned my addr 172.21.15.94:0/560031291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a6d9d640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f009dd9b640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccf46b0 0x560c0d0c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 -- 172.21.15.94:0/560031291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccf46b0 msgr2=0x560c0d0c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccf46b0 0x560c0d0c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 -- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 msgr2=0x560c0d0c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 -- 172.21.15.94:0/560031291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0cfdc4b0 con 0x560c0ccec4d0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f00a659c640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f00980a7210 tx=0x7f00980a6f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6de6d830e596fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f008dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.655+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980a7020 con 0x560c0ccec4d0 2024-04-25T13:57:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f00980e2d90 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a0036070 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00980a7c10 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 10 client.20931 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c0d0c85a0 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00a0036070 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f00980e7030 con 0x560c0ccec4d0 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 10 client.20931 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 10 client.20931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 20 client.20931 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0cd15390 0x560c0d1385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 4 client.20931 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c0d023800 con 0x560c0cd15390 2024-04-25T13:57:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.656+0000 7f009dd9b640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0cd15390 0x560c0d1385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.657+0000 7f008dffb640 10 client.20931 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.657+0000 7f009dd9b640 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0cd15390 0x560c0d1385a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e548730a5a81bf server_cookie=e1338a8a78ce6306 in_seq=0 out_seq=0 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f008dffb640 1 -- 172.21.15.94:0/560031291 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560c0d023800 con 0x560c0cd15390 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f008dffb640 10 client.20931 handle_command_reply: tid=0 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 client.20931 shutdown 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 msgr2=0x560c0d0c5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f00980a7210 tx=0x7f00980a6f20 comp rx=0 tx=0).stop 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 shutdown_connections 2024-04-25T13:57:05.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560c0cd15390 0x560c0d1385a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f008dffb640 0 client.20931 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0cd094e0 0x560c0d0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c0ccf46b0 0x560c0d0c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 --2- 172.21.15.94:0/560031291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560c0ccec4d0 0x560c0d0c5a50 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 >> 172.21.15.94:0/560031291 conn(0x560c0d0b71f0 msgr2=0x560c0d0b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 shutdown_connections 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.660+0000 7f00b3b64740 1 -- 172.21.15.94:0/560031291 wait complete. 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00b3b64740 20 client.20931 trim_cache size 0 max 16384 2024-04-25T13:57:05.666 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:05.667 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:57:05.667 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:57:05.667 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 msgr2=0x7f00940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00a819a180 tx=0x7f0090005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 msgr2=0x7f00a81990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f009800e9f0 tx=0x7f009800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 shutdown_connections 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f00940afaf0 0x7f00940b2000 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8108930 0x7f00a81a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00a8103c30 0x7f00a8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 --2- 172.21.15.94:0/824893910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00a81029f0 0x7f00a81990e0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 >> 172.21.15.94:0/824893910 conn(0x7f00a80fdc00 msgr2=0x7f00a8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 shutdown_connections 2024-04-25T13:57:05.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.661+0000 7f00accaf640 1 -- 172.21.15.94:0/824893910 wait complete. 2024-04-25T13:57:05.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:57:05.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 -- 172.21.15.94:0/922635015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 msgr2=0x7f876c1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 --2- 172.21.15.94:0/922635015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c1037b0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f8760009900 tx=0x7f876002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 -- 172.21.15.94:0/922635015 shutdown_connections 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 --2- 172.21.15.94:0/922635015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c1080a0 0x7f876c10a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 --2- 172.21.15.94:0/922635015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c1037b0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 --2- 172.21.15.94:0/922635015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c102160 0x7f876c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 -- 172.21.15.94:0/922635015 >> 172.21.15.94:0/922635015 conn(0x7f876c0fd330 msgr2=0x7f876c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.790+0000 7f877231f640 1 -- 172.21.15.94:0/922635015 shutdown_connections 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.791+0000 7f877231f640 1 -- 172.21.15.94:0/922635015 wait complete. 2024-04-25T13:57:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.791+0000 7f877231f640 1 Processor -- start 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.791+0000 7f877231f640 1 -- start start 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.791+0000 7f877231f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 0x7f876c10ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c1080a0 0x7f876c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10e500 con 0x7f876c1033a0 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10e380 con 0x7f876c102160 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10e680 con 0x7f876c1080a0 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60378/0 (socket says 172.21.15.94:60378) 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 -- 172.21.15.94:0/2719323476 learned_addr learned my addr 172.21.15.94:0/2719323476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876bfff640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 0x7f876c10ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f87708bc640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c1080a0 0x7f876c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 -- 172.21.15.94:0/2719323476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c1080a0 msgr2=0x7f876c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c1080a0 0x7f876c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 -- 172.21.15.94:0/2719323476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 msgr2=0x7f876c10ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 0x7f876c10ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 -- 172.21.15.94:0/2719323476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c1a1d30 con 0x7f876c1033a0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876bfff640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 0x7f876c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f876b7fe640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f87600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876003f030 con 0x7f876c1033a0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8760031910 con 0x7f876c1033a0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760035070 con 0x7f876c1033a0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.792+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c1a1fc0 con 0x7f876c1033a0 2024-04-25T13:57:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.793+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1a2370 con 0x7f876c1033a0 2024-04-25T13:57:05.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.794+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c069310 con 0x7f876c1033a0 2024-04-25T13:57:05.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.795+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8760042030 con 0x7f876c1033a0 2024-04-25T13:57:05.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.796+0000 7f87697fa640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 0x7f87540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.796+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f87600ff180 con 0x7f876c1033a0 2024-04-25T13:57:05.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.796+0000 7f876bfff640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 0x7f87540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.796+0000 7f876bfff640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 0x7f87540b2000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f876c102d60 tx=0x7f875c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:05.801+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87600c7670 con 0x7f876c1033a0 2024-04-25T13:57:06.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.055+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f876c0637a0 con 0x7f876c1033a0 2024-04-25T13:57:06.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.056+0000 7f87697fa640 1 -- 172.21.15.94:0/2719323476 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7f87600cb970 con 0x7f876c1033a0 2024-04-25T13:57:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:57:06.056 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:56:56.183005+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30058":{"gid":30058,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2422946890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2422946890},{"type":"v1","addr":"172.21.15.138:6835","nonce":2422946890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24502,24488,24475,24512,24557]},"id":1}]} 2024-04-25T13:57:06.057 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 msgr2=0x7f87540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 0x7f87540b2000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f876c102d60 tx=0x7f875c005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 msgr2=0x7f876c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f87600042f0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 shutdown_connections 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87540afaf0 0x7f87540b2000 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f876c1080a0 0x7f876c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c1033a0 0x7f876c19ef60 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 --2- 172.21.15.94:0/2719323476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f876c102160 0x7f876c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 >> 172.21.15.94:0/2719323476 conn(0x7f876c0fd330 msgr2=0x7f876c104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 shutdown_connections 2024-04-25T13:57:06.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.059+0000 7f877231f640 1 -- 172.21.15.94:0/2719323476 wait complete. 2024-04-25T13:57:06.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:57:06.137 INFO:tasks.workunit.client.0.smithi094.stdout: 10 40247 21.67 MB/sec warmup 108 sec latency 77.641 ms 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 -- 172.21.15.94:0/789017020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30108930 msgr2=0x7fab3010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 --2- 172.21.15.94:0/789017020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30108930 0x7fab3010ad50 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fab28009ca0 tx=0x7fab2802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 -- 172.21.15.94:0/789017020 shutdown_connections 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 --2- 172.21.15.94:0/789017020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30108930 0x7fab3010ad50 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 --2- 172.21.15.94:0/789017020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30103c30 0x7fab30104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 --2- 172.21.15.94:0/789017020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 0x7fab301036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 -- 172.21.15.94:0/789017020 >> 172.21.15.94:0/789017020 conn(0x7fab300fdbe0 msgr2=0x7fab30100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 -- 172.21.15.94:0/789017020 shutdown_connections 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.204+0000 7fab34b34640 1 -- 172.21.15.94:0/789017020 wait complete. 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 Processor -- start 2024-04-25T13:57:06.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 -- start start 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 0x7fab3019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30108930 0x7fab301a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010f620 con 0x7fab30103c30 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010f4a0 con 0x7fab301029f0 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab34b34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010f7a0 con 0x7fab30108930 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60384/0 (socket says 172.21.15.94:60384) 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.205+0000 7fab27fff640 1 -- 172.21.15.94:0/2224772584 learned_addr learned my addr 172.21.15.94:0/2224772584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30108930 0x7fab301a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab2e59c640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 0x7fab3019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 -- 172.21.15.94:0/2224772584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30108930 msgr2=0x7fab301a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30108930 0x7fab301a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 -- 172.21.15.94:0/2224772584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 msgr2=0x7fab3019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 0x7fab3019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 -- 172.21.15.94:0/2224772584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab301a1d60 con 0x7fab30103c30 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab27fff640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fab1800e9f0 tx=0x7fab1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab277fe640 1 -- 172.21.15.94:0/2224772584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1800cd80 con 0x7fab30103c30 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab301a1ff0 con 0x7fab30103c30 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab277fe640 1 -- 172.21.15.94:0/2224772584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab180054d0 con 0x7fab30103c30 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab277fe640 1 -- 172.21.15.94:0/2224772584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18010690 con 0x7fab30103c30 2024-04-25T13:57:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.206+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a24e0 con 0x7fab30103c30 2024-04-25T13:57:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab277fe640 1 -- 172.21.15.94:0/2224772584 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fab18010830 con 0x7fab30103c30 2024-04-25T13:57:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 Processor -- start 2024-04-25T13:57:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 -- start start 2024-04-25T13:57:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 0x560085e97810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085ad46b0 0x560085e98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560085dfbd80 con 0x560085acc4d0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560085db5a20 con 0x560085af5390 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.209+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560085df7d40 con 0x560085ad46b0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60386/0 (socket says 172.21.15.94:60386) 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 -- 172.21.15.94:0/2181287159 learned_addr learned my addr 172.21.15.94:0/2181287159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 0x560085e97810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab27fff640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085ad46b0 0x560085e98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab277fe640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 0x7fab1c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 0x7fab1c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 -- 172.21.15.94:0/2181287159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085ad46b0 msgr2=0x560085e98a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085ad46b0 0x560085e98a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 -- 172.21.15.94:0/2181287159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 msgr2=0x560085e97810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 0x560085e97810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 -- 172.21.15.94:0/2181287159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560085dc0c30 con 0x560085acc4d0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 0x560085e97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab277fe640 1 -- 172.21.15.94:0/2224772584 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fab180df050 con 0x7fab30103c30 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2e59c640 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c60 tx=0x7fab20011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf65e2871696459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 0x7fab1c0b2000 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fab2802f500 tx=0x7fab2802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab24ff9640 1 -- 172.21.15.94:0/2181287159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20004220 con 0x560085acc4d0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab24ff9640 1 -- 172.21.15.94:0/2181287159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200043c0 con 0x560085acc4d0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.210+0000 7fab24ff9640 1 -- 172.21.15.94:0/2181287159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20004ce0 con 0x560085acc4d0 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 -- 172.21.15.94:0/2181287159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 msgr2=0x560085e97d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c60 tx=0x7fab20011810 comp rx=0 tx=0).stop 2024-04-25T13:57:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 -- 172.21.15.94:0/2181287159 shutdown_connections 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085ad46b0 0x560085e98a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085acc4d0 0x560085e97d90 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/2181287159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085af5390 0x560085e97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 -- 172.21.15.94:0/2181287159 >> 172.21.15.94:0/2181287159 conn(0x560085ae94e0 msgr2=0x560085e93020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 -- 172.21.15.94:0/2181287159 shutdown_connections 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.211+0000 7fab3b9e7740 1 -- 172.21.15.94:0/2181287159 wait complete. 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 -- start start 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 Processor -- start 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 0x560085ea1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085af5390 0x560085ea89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560085e99530 con 0x560085ad46b0 2024-04-25T13:57:06.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560085d82dc0 con 0x560085acc4d0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560085e996b0 con 0x560085af5390 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab3b9e7740 10 client.0 _ll_register_callbacks cb 0x560085d9e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab2e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 0x560085ea1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60392/0 (socket says 172.21.15.94:60392) 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab27fff640 1 -- 172.21.15.94:0/224986285 learned_addr learned my addr 172.21.15.94:0/224986285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.212+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085af5390 0x560085ea89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 -- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085af5390 msgr2=0x560085ea89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085af5390 0x560085ea89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 -- 172.21.15.94:0/224986285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 msgr2=0x560085ea1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 0x560085ea1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 -- 172.21.15.94:0/224986285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560085b0d3f0 con 0x560085ad46b0 2024-04-25T13:57:06.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab2e59c640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 0x560085ea1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab27fff640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c6a0 tx=0x7fab180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8640ceda7897efea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180a6ca0 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fab180e10d0 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab200043c0 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab180a7040 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab3b9e7740 10 client.20949 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560085bd77d0 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.213+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab200043c0 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab18010b20 con 0x560085ad46b0 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 10 client.20949 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 10 client.20949 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 20 client.20949 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085ab3f60 0x560085eaf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 4 client.20949 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560085e09990 con 0x560085ab3f60 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab2e59c640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085ab3f60 0x560085eaf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.214+0000 7fab15ffb640 10 client.20949 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab2e59c640 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085ab3f60 0x560085eaf270 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c1f0c5ae44e7e94 server_cookie=ff6a55bebea15e57 in_seq=0 out_seq=0 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab15ffb640 1 -- 172.21.15.94:0/224986285 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560085e09990 con 0x560085ab3f60 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab15ffb640 10 client.20949 handle_command_reply: tid=0 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab3b9e7740 1 client.20949 shutdown 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 msgr2=0x560085ea1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.215+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c6a0 tx=0x7fab180048f0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 shutdown_connections 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085ab3f60 0x560085eaf270 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085af5390 0x560085ea89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085ad46b0 0x560085ea1a40 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab15ffb640 0 client.20949 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/224986285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085acc4d0 0x560085ea1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 >> 172.21.15.94:0/224986285 conn(0x560085ae94e0 msgr2=0x560085e95f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 shutdown_connections 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 1 -- 172.21.15.94:0/224986285 wait complete. 2024-04-25T13:57:06.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.216+0000 7fab3b9e7740 20 client.20949 trim_cache size 0 max 16384 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 Processor -- start 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 -- start start 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085e91d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ab500 con 0x560085af5390 2024-04-25T13:57:06.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ad100 con 0x560085ad46b0 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab301abbd0 con 0x560085acc4d0 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60408/0 (socket says 172.21.15.94:60408) 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 -- 172.21.15.94:0/3363905733 learned_addr learned my addr 172.21.15.94:0/3363905733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab27fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2e59c640 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085e91d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 -- 172.21.15.94:0/3363905733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 msgr2=0x560085e91d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085e91d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 -- 172.21.15.94:0/3363905733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 msgr2=0x560085ea88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab2ed9d640 1 -- 172.21.15.94:0/3363905733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560085e9bdc0 con 0x560085af5390 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.232+0000 7fab27fff640 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.233+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fab28009ca0 tx=0x7fab280093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12571f6be243adaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.233+0000 7fab17fff640 1 -- 172.21.15.94:0/3363905733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28051070 con 0x560085af5390 2024-04-25T13:57:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.233+0000 7fab17fff640 1 -- 172.21.15.94:0/3363905733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28036050 con 0x560085af5390 2024-04-25T13:57:06.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.233+0000 7fab17fff640 1 -- 172.21.15.94:0/3363905733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2802fec0 con 0x560085af5390 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- 172.21.15.94:0/3363905733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 msgr2=0x560085e95e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fab28009ca0 tx=0x7fab280093d0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- 172.21.15.94:0/3363905733 shutdown_connections 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085e95e20 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/3363905733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085e91d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- 172.21.15.94:0/3363905733 >> 172.21.15.94:0/3363905733 conn(0x560085ae94e0 msgr2=0x560085e91610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- 172.21.15.94:0/3363905733 shutdown_connections 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- 172.21.15.94:0/3363905733 wait complete. 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- start start 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 Processor -- start 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085ea5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab301aca30 con 0x560085af5390 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ac2a0 con 0x560085ad46b0 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.234+0000 7fab3b9e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab301a94e0 con 0x560085acc4d0 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab3b9e7740 10 client.0 _ll_register_callbacks cb 0x7fab1800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60412/0 (socket says 172.21.15.94:60412) 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 -- 172.21.15.94:0/4140235006 learned_addr learned my addr 172.21.15.94:0/4140235006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2e59c640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085ea5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab27fff640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 -- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 msgr2=0x560085ea5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085ea5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 -- 172.21.15.94:0/4140235006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 msgr2=0x560085ea6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 -- 172.21.15.94:0/4140235006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560085ea7740 con 0x560085af5390 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab2ed9d640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fab2802ade0 tx=0x7fab2802fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d8a5132f1ac240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2805a070 con 0x560085af5390 2024-04-25T13:57:06.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fab28004450 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28036050 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab28051050 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab3b9e7740 10 client.20955 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.235+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560085ea89c0 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28036050 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab2804a050 con 0x560085af5390 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 10 client.20955 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 10 client.20955 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 20 client.20955 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085a893a0 0x560085f19b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 4 client.20955 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560085e03800 con 0x560085a893a0 2024-04-25T13:57:06.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab2e59c640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085a893a0 0x560085f19b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab15ffb640 10 client.20955 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab2e59c640 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085a893a0 0x560085f19b20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b4829368ff493f4 server_cookie=b3387cce36348438 in_seq=0 out_seq=0 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.236+0000 7fab15ffb640 1 -- 172.21.15.94:0/4140235006 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560085e03800 con 0x560085a893a0 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab15ffb640 10 client.20955 handle_command_reply: tid=0 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 client.20955 shutdown 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 msgr2=0x560085ea7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fab2802ade0 tx=0x7fab2802fef0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 shutdown_connections 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560085a893a0 0x560085f19b20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560085af5390 0x560085ea7180 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab15ffb640 0 client.20955 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560085ad46b0 0x560085ea6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 --2- 172.21.15.94:0/4140235006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560085acc4d0 0x560085ea5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 >> 172.21.15.94:0/4140235006 conn(0x560085ae94e0 msgr2=0x560085e96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 shutdown_connections 2024-04-25T13:57:06.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 1 -- 172.21.15.94:0/4140235006 wait complete. 2024-04-25T13:57:06.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab3b9e7740 20 client.20955 trim_cache size 0 max 16384 2024-04-25T13:57:06.245 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:06.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 msgr2=0x7fab1c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 0x7fab1c0b2000 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fab2802f500 tx=0x7fab2802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 msgr2=0x7fab3019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fab1800e9f0 tx=0x7fab1800edf0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 shutdown_connections 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab1c0afaf0 0x7fab1c0b2000 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab30108930 0x7fab301a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30103c30 0x7fab3019a760 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 --2- 172.21.15.94:0/2224772584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab301029f0 0x7fab3019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.237+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 >> 172.21.15.94:0/2224772584 conn(0x7fab300fdbe0 msgr2=0x7fab30104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.238+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 shutdown_connections 2024-04-25T13:57:06.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.238+0000 7fab34b34640 1 -- 172.21.15.94:0/2224772584 wait complete. 2024-04-25T13:57:06.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:57:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 -- 172.21.15.94:0/544730964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd324103c30 msgr2=0x7fd324104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 --2- 172.21.15.94:0/544730964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd324103c30 0x7fd324104040 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fd310009860 tx=0x7fd31002a340 comp rx=0 tx=0).stop 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 -- 172.21.15.94:0/544730964 shutdown_connections 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 --2- 172.21.15.94:0/544730964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 0x7fd32410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 --2- 172.21.15.94:0/544730964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd324103c30 0x7fd324104040 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 --2- 172.21.15.94:0/544730964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3241029f0 0x7fd3241036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 -- 172.21.15.94:0/544730964 >> 172.21.15.94:0/544730964 conn(0x7fd3240fdc00 msgr2=0x7fd324100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 -- 172.21.15.94:0/544730964 shutdown_connections 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.370+0000 7fd32b09f640 1 -- 172.21.15.94:0/544730964 wait complete. 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 Processor -- start 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 -- start start 2024-04-25T13:57:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 0x7fd32419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 0x7fd3241a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410e4f0 con 0x7fd3241029f0 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410e370 con 0x7fd324103c30 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd32b09f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd32410e670 con 0x7fd324108930 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60438/0 (socket says 172.21.15.94:60438) 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd328e3b640 1 -- 172.21.15.94:0/62704631 learned_addr learned my addr 172.21.15.94:0/62704631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.371+0000 7fd323fff640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 0x7fd32419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd32963c640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 0x7fd3241a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 -- 172.21.15.94:0/62704631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 msgr2=0x7fd3241a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 0x7fd3241a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 -- 172.21.15.94:0/62704631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 msgr2=0x7fd32419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 0x7fd32419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 -- 172.21.15.94:0/62704631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3241a1d30 con 0x7fd3241029f0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd323fff640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 0x7fd32419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd328e3b640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fd31800e980 tx=0x7fd31800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd321ffb640 1 -- 172.21.15.94:0/62704631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800cd80 con 0x7fd3241029f0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd321ffb640 1 -- 172.21.15.94:0/62704631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3180108e0 con 0x7fd3241029f0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3241a1fc0 con 0x7fd3241029f0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd321ffb640 1 -- 172.21.15.94:0/62704631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318018940 con 0x7fd3241029f0 2024-04-25T13:57:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.372+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3241a2480 con 0x7fd3241029f0 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd321ffb640 1 -- 172.21.15.94:0/62704631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd31801f020 con 0x7fd3241029f0 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 Processor -- start 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 -- start start 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 0x55fe39a77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a77d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe399dbd80 con 0x55fe396b46b0 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe39995a20 con 0x55fe396ac4d0 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd33202c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe399d7d40 con 0x55fe396d5390 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a77d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd323fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd32963c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 0x55fe39a77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd323fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a78a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60446/0 (socket says 172.21.15.94:60446) 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a77d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58018/0 (socket says 172.21.15.94:58018) 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd323fff640 1 -- 172.21.15.94:0/1785117377 learned_addr learned my addr 172.21.15.94:0/1785117377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.375+0000 7fd321ffb640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 0x7fd3140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 -- 172.21.15.94:0/1785117377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 msgr2=0x55fe39a77d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a77d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 -- 172.21.15.94:0/1785117377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 msgr2=0x55fe39a78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a78a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 -- 172.21.15.94:0/1785117377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe399a0c30 con 0x55fe396d5390 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd321ffb640 1 -- 172.21.15.94:0/62704631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd3180dedf0 con 0x7fd3241029f0 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 0x7fd3140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd32963c640 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 0x55fe39a77810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009fd0 tx=0x7fd31c011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c19b26b45bbfaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd30effd640 1 -- 172.21.15.94:0/1785117377 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c017070 con 0x55fe396d5390 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd30effd640 1 -- 172.21.15.94:0/1785117377 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31c00fbf0 con 0x55fe396d5390 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.376+0000 7fd30effd640 1 -- 172.21.15.94:0/1785117377 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c01b9a0 con 0x55fe396d5390 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- 172.21.15.94:0/1785117377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 msgr2=0x55fe39a77810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 0x55fe39a77810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009fd0 tx=0x7fd31c011920 comp rx=0 tx=0).stop 2024-04-25T13:57:06.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd32963c640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 0x7fd3140b2000 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd31c022000 tx=0x7fd31c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- 172.21.15.94:0/1785117377 shutdown_connections 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a77d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 --2- 172.21.15.94:0/1785117377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396d5390 0x55fe39a77810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- 172.21.15.94:0/1785117377 >> 172.21.15.94:0/1785117377 conn(0x55fe396c94e0 msgr2=0x55fe39a73020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- 172.21.15.94:0/1785117377 shutdown_connections 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- 172.21.15.94:0/1785117377 wait complete. 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.377+0000 7fd33202c740 1 -- start start 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 Processor -- start 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 0x55fe39a81590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 0x55fe39a81ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fe39a79530 con 0x55fe396d5390 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe39962dc0 con 0x55fe396b46b0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe39a796b0 con 0x55fe396ac4d0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd33202c740 10 client.0 _ll_register_callbacks cb 0x55fe3997e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd32963c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 0x55fe39a81ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd32963c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60454/0 (socket says 172.21.15.94:60454) 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd32963c640 1 -- 172.21.15.94:0/4170427133 learned_addr learned my addr 172.21.15.94:0/4170427133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.378+0000 7fd328e3b640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 0x55fe39a81590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 -- 172.21.15.94:0/4170427133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 msgr2=0x55fe39a81590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 0x55fe39a81590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 -- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 msgr2=0x55fe39a81ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 0x55fe39a81ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 -- 172.21.15.94:0/4170427133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe396ed3f0 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd328e3b640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 0x55fe39a81590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd323fff640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 0x55fe39a81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd32963c640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd31c004a70 tx=0x7fd31c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c03231453f324a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c017050 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd31c01b720 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31c00fbf0 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd31c01b920 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd33202c740 10 client.20967 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fe397b77d0 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31c00fbf0 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.379+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd31c026840 con 0x55fe396d5390 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 10 client.20967 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 10 client.20967 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 20 client.20967 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe39693f60 0x55fe39a8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 4 client.20967 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe399e9990 con 0x55fe39693f60 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd328e3b640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe39693f60 0x55fe39a8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd328e3b640 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe39693f60 0x55fe39a8f000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e94bed38ca5a3d45 server_cookie=d560e640c3d93d27 in_seq=0 out_seq=0 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.380+0000 7fd303fff640 10 client.20967 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd303fff640 1 -- 172.21.15.94:0/4170427133 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe399e9990 con 0x55fe39693f60 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd303fff640 10 client.20967 handle_command_reply: tid=0 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 client.20967 shutdown 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 msgr2=0x55fe39a88a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd31c004a70 tx=0x7fd31c004aa0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 shutdown_connections 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe39693f60 0x55fe39a8f000 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396d5390 0x55fe39a88a30 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396b46b0 0x55fe39a81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd303fff640 0 client.20967 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 --2- 172.21.15.94:0/4170427133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396ac4d0 0x55fe39a81590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 >> 172.21.15.94:0/4170427133 conn(0x55fe396c94e0 msgr2=0x55fe39a75f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 shutdown_connections 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 1 -- 172.21.15.94:0/4170427133 wait complete. 2024-04-25T13:57:06.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.381+0000 7fd33202c740 20 client.20967 trim_cache size 0 max 16384 2024-04-25T13:57:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.396+0000 7fd33202c740 1 Processor -- start 2024-04-25T13:57:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.396+0000 7fd33202c740 1 -- start start 2024-04-25T13:57:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.396+0000 7fd33202c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd33202c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a88950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd33202c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396b46b0 0x55fe39a75e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd33202c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241a9340 con 0x55fe39693f60 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd33202c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241aaf40 con 0x55fe396ac4d0 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd33202c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241a9a10 con 0x55fe396b46b0 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60458/0 (socket says 172.21.15.94:60458) 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a88950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 -- 172.21.15.94:0/450359853 learned_addr learned my addr 172.21.15.94:0/450359853 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd32963c640 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396b46b0 0x55fe39a75e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 -- 172.21.15.94:0/450359853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396b46b0 msgr2=0x55fe39a75e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396b46b0 0x55fe39a75e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 -- 172.21.15.94:0/450359853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 msgr2=0x55fe39a88950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a88950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 -- 172.21.15.94:0/450359853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe39a88e90 con 0x55fe39693f60 2024-04-25T13:57:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd328e3b640 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd318000c00 tx=0x7fd31800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e13ea1562b86f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd30e7fc640 1 -- 172.21.15.94:0/450359853 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180a6d40 con 0x55fe39693f60 2024-04-25T13:57:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd30e7fc640 1 -- 172.21.15.94:0/450359853 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3180a7930 con 0x55fe39693f60 2024-04-25T13:57:06.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.397+0000 7fd30e7fc640 1 -- 172.21.15.94:0/450359853 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318010d20 con 0x55fe39693f60 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- 172.21.15.94:0/450359853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 msgr2=0x55fe39a803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd318000c00 tx=0x7fd31800ef70 comp rx=0 tx=0).stop 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- 172.21.15.94:0/450359853 shutdown_connections 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe396b46b0 0x55fe39a75e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a88950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- 172.21.15.94:0/450359853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe39693f60 0x55fe39a803c0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- 172.21.15.94:0/450359853 >> 172.21.15.94:0/450359853 conn(0x55fe396c94e0 msgr2=0x55fe39a7fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- 172.21.15.94:0/450359853 shutdown_connections 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- 172.21.15.94:0/450359853 wait complete. 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- start start 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 Processor -- start 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241aa870 con 0x55fe396b46b0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241aa0e0 con 0x55fe396ac4d0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3241a7320 con 0x55fe39693f60 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.399+0000 7fd33202c740 10 client.0 _ll_register_callbacks cb 0x7fd318005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd32963c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd32963c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a86fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60470/0 (socket says 172.21.15.94:60470) 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd32963c640 1 -- 172.21.15.94:0/2088108368 learned_addr learned my addr 172.21.15.94:0/2088108368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51440/0 (socket says 172.21.15.94:51440) 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a860e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58040/0 (socket says 172.21.15.94:58040) 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 -- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 msgr2=0x55fe39a860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 -- 172.21.15.94:0/2088108368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 msgr2=0x55fe39a86fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a86fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 -- 172.21.15.94:0/2088108368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe39a87560 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd328e3b640 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3180df220 tx=0x7fd3180a6c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3442f803614977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd303fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3180a7230 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd3180a34c0 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3180a7930 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd3180b0090 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd33202c740 10 client.30118 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.400+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fe39a88850 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3180a7930 con 0x55fe39693f60 2024-04-25T13:57:06.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd3180afc00 con 0x55fe39693f60 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 10 client.30118 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 10 client.30118 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 20 client.30118 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe396d5390 0x55fe39af96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 4 client.30118 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe399e3800 con 0x55fe396d5390 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd323fff640 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe396d5390 0x55fe39af96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.401+0000 7fd303fff640 10 client.30118 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd323fff640 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe396d5390 0x55fe39af96f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e81c5a4a9a69f942 server_cookie=771f16b5579d3366 in_seq=0 out_seq=0 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd303fff640 1 -- 172.21.15.94:0/2088108368 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe399e3800 con 0x55fe396d5390 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd303fff640 10 client.30118 handle_command_reply: tid=0 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 client.30118 shutdown 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 msgr2=0x55fe39a85ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3180df220 tx=0x7fd3180a6c80 comp rx=0 tx=0).stop 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 shutdown_connections 2024-04-25T13:57:06.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55fe396d5390 0x55fe39af96f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fe396b46b0 0x55fe39a86fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe396ac4d0 0x55fe39a860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd303fff640 0 client.30118 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 --2- 172.21.15.94:0/2088108368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe39693f60 0x55fe39a85ba0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 >> 172.21.15.94:0/2088108368 conn(0x55fe396c94e0 msgr2=0x55fe39a71610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 shutdown_connections 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 1 -- 172.21.15.94:0/2088108368 wait complete. 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.402+0000 7fd33202c740 20 client.30118 trim_cache size 0 max 16384 2024-04-25T13:57:06.408 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 msgr2=0x7fd3140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 0x7fd3140b2000 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd31c022000 tx=0x7fd31c023000 comp rx=0 tx=0).stop 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 msgr2=0x7fd32419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fd31800e980 tx=0x7fd31800ee40 comp rx=0 tx=0).stop 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 shutdown_connections 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd3140afaf0 0x7fd3140b2000 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd324108930 0x7fd3241a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd324103c30 0x7fd32419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 --2- 172.21.15.94:0/62704631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3241029f0 0x7fd32419a1f0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 >> 172.21.15.94:0/62704631 conn(0x7fd3240fdc00 msgr2=0x7fd324109e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 shutdown_connections 2024-04-25T13:57:06.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.403+0000 7fd32b09f640 1 -- 172.21.15.94:0/62704631 wait complete. 2024-04-25T13:57:06.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:06 smithi094 ceph-mon[29195]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-04-25T13:57:06.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:06 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2719323476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:06.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:57:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:06 smithi138 ceph-mon[32977]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-04-25T13:57:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:06 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2719323476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:06.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 -- 172.21.15.94:0/141284329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face4101760 msgr2=0x7face4101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 --2- 172.21.15.94:0/141284329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face4101760 0x7face4101bd0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7facd8009900 tx=0x7facd802a410 comp rx=0 tx=0).stop 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 -- 172.21.15.94:0/141284329 shutdown_connections 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 --2- 172.21.15.94:0/141284329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4102260 0x7face410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 --2- 172.21.15.94:0/141284329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face4101760 0x7face4101bd0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 --2- 172.21.15.94:0/141284329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face41050b0 0x7face41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 -- 172.21.15.94:0/141284329 >> 172.21.15.94:0/141284329 conn(0x7face40fd380 msgr2=0x7face40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 -- 172.21.15.94:0/141284329 shutdown_connections 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.536+0000 7face9e68640 1 -- 172.21.15.94:0/141284329 wait complete. 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 Processor -- start 2024-04-25T13:57:06.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 -- start start 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4101760 0x7face419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 0x7face419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7face410f620 con 0x7face41050b0 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7face410f4a0 con 0x7face4101760 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.537+0000 7face9e68640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7face410f7a0 con 0x7face4102260 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60490/0 (socket says 172.21.15.94:60490) 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 -- 172.21.15.94:0/1697633966 learned_addr learned my addr 172.21.15.94:0/1697633966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face2ffd640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 0x7face419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face37fe640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4101760 0x7face419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 -- 172.21.15.94:0/1697633966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 msgr2=0x7face419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 0x7face419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 -- 172.21.15.94:0/1697633966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4101760 msgr2=0x7face419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4101760 0x7face419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 -- 172.21.15.94:0/1697633966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face41a1d60 con 0x7face41050b0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face2ffd640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 0x7face419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face3fff640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7facdc00eb10 tx=0x7facdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face0ff9640 1 -- 172.21.15.94:0/1697633966 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc004b20 con 0x7face41050b0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.538+0000 7face0ff9640 1 -- 172.21.15.94:0/1697633966 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc004cc0 con 0x7face41050b0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.539+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face41a1ff0 con 0x7face41050b0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.539+0000 7face0ff9640 1 -- 172.21.15.94:0/1697633966 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc010670 con 0x7face41050b0 2024-04-25T13:57:06.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.539+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face41027e0 con 0x7face41050b0 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.541+0000 7facf0df2740 1 Processor -- start 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face0ff9640 1 -- 172.21.15.94:0/1697633966 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7facdc005020 con 0x7face41050b0 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 -- start start 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8fed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 0x558df8fedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558df8f51d80 con 0x558df8c2a6b0 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558df8f0ba20 con 0x558df8c4b390 2024-04-25T13:57:06.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7facf0df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558df8f4dd40 con 0x558df8c224d0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60504/0 (socket says 172.21.15.94:60504) 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 -- 172.21.15.94:0/3701825218 learned_addr learned my addr 172.21.15.94:0/3701825218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face37fe640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 0x558df8fedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8fed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face0ff9640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 0x7faccc0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 -- 172.21.15.94:0/3701825218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 msgr2=0x558df8fedd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face3fff640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 0x7faccc0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 0x558df8fedd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 -- 172.21.15.94:0/3701825218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 msgr2=0x558df8fed810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8fed810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face2ffd640 1 -- 172.21.15.94:0/3701825218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df8f16c30 con 0x558df8c2a6b0 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.542+0000 7face37fe640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 0x558df8fedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.543+0000 7face0ff9640 1 -- 172.21.15.94:0/1697633966 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7facdc0df260 con 0x7face41050b0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.543+0000 7face2ffd640 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x558df8fee860 tx=0x7facd8032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7028ee115ffc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.543+0000 7facc9ffb640 1 -- 172.21.15.94:0/3701825218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8035040 con 0x558df8c2a6b0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.543+0000 7facc9ffb640 1 -- 172.21.15.94:0/3701825218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd80035f0 con 0x558df8c2a6b0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.543+0000 7facc9ffb640 1 -- 172.21.15.94:0/3701825218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80044e0 con 0x558df8c2a6b0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- 172.21.15.94:0/3701825218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 msgr2=0x558df8feea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7face3fff640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 0x7faccc0b2010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7facdc0dd030 tx=0x7facdc09bfe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x558df8fee860 tx=0x7facd8032ed0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- 172.21.15.94:0/3701825218 shutdown_connections 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8feea20 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c224d0 0x558df8fedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 --2- 172.21.15.94:0/3701825218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8fed810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- 172.21.15.94:0/3701825218 >> 172.21.15.94:0/3701825218 conn(0x558df8c3f4e0 msgr2=0x558df8fe9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- 172.21.15.94:0/3701825218 shutdown_connections 2024-04-25T13:57:06.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- 172.21.15.94:0/3701825218 wait complete. 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 -- start start 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.544+0000 7facf0df2740 1 Processor -- start 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 0x558df8ff7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558df8fef4d0 con 0x558df8c224d0 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558df8ed8dc0 con 0x558df8c2a6b0 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558df8fef650 con 0x558df8c4b390 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facf0df2740 10 client.0 _ll_register_callbacks cb 0x558df8ef4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60516/0 (socket says 172.21.15.94:60516) 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 -- 172.21.15.94:0/622283103 learned_addr learned my addr 172.21.15.94:0/622283103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face37fe640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 0x558df8ff7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 -- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 msgr2=0x558df8ffeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 -- 172.21.15.94:0/622283103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 msgr2=0x558df8ff7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 0x558df8ff7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 -- 172.21.15.94:0/622283103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df8c633f0 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face37fe640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 0x558df8ff7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7face2ffd640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7facd8005ab0 tx=0x7facd8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956cd581e2baeab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.545+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8035040 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd80035f0 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7facd8004200 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7facd8000e60 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facf0df2740 10 client.20985 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558df8d2d7d0 con 0x558df8c224d0 2024-04-25T13:57:06.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.546+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd80035f0 con 0x558df8c224d0 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7facd8052400 con 0x558df8c224d0 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 10 client.20985 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 10 client.20985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 20 client.20985 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8c09f60 0x558df9004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 4 client.20985 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df8f5f990 con 0x558df8c09f60 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7face37fe640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8c09f60 0x558df9004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7facc2ffd640 10 client.20985 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.547+0000 7face37fe640 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8c09f60 0x558df9004f10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=877e80cb667bb3da server_cookie=935aa7567bd80532 in_seq=0 out_seq=0 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facc2ffd640 1 -- 172.21.15.94:0/622283103 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558df8f5f990 con 0x558df8c09f60 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facc2ffd640 10 client.20985 handle_command_reply: tid=0 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 client.20985 shutdown 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 msgr2=0x558df8ff7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7facd8005ab0 tx=0x7facd8031850 comp rx=0 tx=0).stop 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 shutdown_connections 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8c09f60 0x558df9004f10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c2a6b0 0x558df8ff7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facc2ffd640 0 client.20985 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 --2- 172.21.15.94:0/622283103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8ff7620 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.548+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 >> 172.21.15.94:0/622283103 conn(0x558df8c3f4e0 msgr2=0x558df8febf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.549+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 shutdown_connections 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.549+0000 7facf0df2740 1 -- 172.21.15.94:0/622283103 wait complete. 2024-04-25T13:57:06.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.549+0000 7facf0df2740 20 client.20985 trim_cache size 0 max 16384 2024-04-25T13:57:06.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.562+0000 7facf0df2740 1 Processor -- start 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.562+0000 7facf0df2740 1 -- start start 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c2a6b0 0x558df8fe8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8febe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7face41b1b30 con 0x558df8c224d0 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7face41b3730 con 0x558df8c4b390 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facf0df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7face41b2200 con 0x558df8c2a6b0 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60522/0 (socket says 172.21.15.94:60522) 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 -- 172.21.15.94:0/11184355 learned_addr learned my addr 172.21.15.94:0/11184355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face37fe640 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c2a6b0 0x558df8fe8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8febe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 -- 172.21.15.94:0/11184355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c2a6b0 msgr2=0x558df8fe8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c2a6b0 0x558df8fe8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 -- 172.21.15.94:0/11184355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 msgr2=0x558df8febe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8febe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 -- 172.21.15.94:0/11184355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df8fe9bb0 con 0x558df8c224d0 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7face2ffd640 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7facd8031500 tx=0x7facd8032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cc4711df6d89e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.563+0000 7facc97fa640 1 -- 172.21.15.94:0/11184355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8035040 con 0x558df8c224d0 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facc97fa640 1 -- 172.21.15.94:0/11184355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd804e070 con 0x558df8c224d0 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facc97fa640 1 -- 172.21.15.94:0/11184355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8004420 con 0x558df8c224d0 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 -- 172.21.15.94:0/11184355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 msgr2=0x558df8fe8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7facd8031500 tx=0x7facd8032ed0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 -- 172.21.15.94:0/11184355 shutdown_connections 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c4b390 0x558df8febe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c2a6b0 0x558df8fe8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 --2- 172.21.15.94:0/11184355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c224d0 0x558df8fe8a50 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 -- 172.21.15.94:0/11184355 >> 172.21.15.94:0/11184355 conn(0x558df8c3f4e0 msgr2=0x558df8ff2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 -- 172.21.15.94:0/11184355 shutdown_connections 2024-04-25T13:57:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.564+0000 7facf0df2740 1 -- 172.21.15.94:0/11184355 wait complete. 2024-04-25T13:57:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 -- start start 2024-04-25T13:57:06.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 Processor -- start 2024-04-25T13:57:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c224d0 0x558df8ffc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7face41b3060 con 0x558df8c2a6b0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7face41b28d0 con 0x558df8c224d0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7face41afb10 con 0x558df8c4b390 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7facf0df2740 10 client.0 _ll_register_callbacks cb 0x7facd80315a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.565+0000 7face37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c224d0 0x558df8ffc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60530/0 (socket says 172.21.15.94:60530) 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 -- 172.21.15.94:0/3403426835 learned_addr learned my addr 172.21.15.94:0/3403426835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face2ffd640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 -- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 msgr2=0x558df8ffd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 -- 172.21.15.94:0/3403426835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c224d0 msgr2=0x558df8ffc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c224d0 0x558df8ffc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 -- 172.21.15.94:0/3403426835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df8ffe4e0 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7face37fe640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7facd4005fd0 tx=0x7facd400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4921e8057de1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40200b0 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7facd4020250 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd804e070 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7facd4020450 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facf0df2740 10 client.20991 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.566+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558df906c1d0 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd804e070 con 0x558df8c2a6b0 2024-04-25T13:57:06.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7facd4011b10 con 0x558df8c2a6b0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 10 client.20991 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 10 client.20991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 20 client.20991 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8bdf3a0 0x558df906ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 4 client.20991 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df8f59800 con 0x558df8bdf3a0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7face2ffd640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8bdf3a0 0x558df906ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7facc2ffd640 10 client.20991 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.567+0000 7face2ffd640 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8bdf3a0 0x558df906ebf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32922fd1ecbaffe0 server_cookie=a0f5c3604a074c0b in_seq=0 out_seq=0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facc2ffd640 1 -- 172.21.15.94:0/3403426835 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df8f59800 con 0x558df8bdf3a0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facc2ffd640 10 client.20991 handle_command_reply: tid=0 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 client.20991 shutdown 2024-04-25T13:57:06.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 msgr2=0x558df8ffcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7facd4005fd0 tx=0x7facd400bf10 comp rx=0 tx=0).stop 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 shutdown_connections 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558df8bdf3a0 0x558df906ebf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facc2ffd640 0 client.20991 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df8c4b390 0x558df8ffd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558df8c2a6b0 0x558df8ffcdd0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 --2- 172.21.15.94:0/3403426835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df8c224d0 0x558df8ffc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 >> 172.21.15.94:0/3403426835 conn(0x558df8c3f4e0 msgr2=0x558df8febe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 shutdown_connections 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 1 -- 172.21.15.94:0/3403426835 wait complete. 2024-04-25T13:57:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.568+0000 7facf0df2740 20 client.20991 trim_cache size 0 max 16384 2024-04-25T13:57:06.575 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 msgr2=0x7faccc0b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 0x7faccc0b2010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7facdc0dd030 tx=0x7facdc09bfe0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 msgr2=0x7face41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7facdc00eb10 tx=0x7facdc00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 shutdown_connections 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faccc0afb00 0x7faccc0b2010 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face41050b0 0x7face41a16f0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7face4102260 0x7face419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 --2- 172.21.15.94:0/1697633966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7face4101760 0x7face419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 >> 172.21.15.94:0/1697633966 conn(0x7face40fd380 msgr2=0x7face4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 shutdown_connections 2024-04-25T13:57:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.569+0000 7face9e68640 1 -- 172.21.15.94:0/1697633966 wait complete. 2024-04-25T13:57:06.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:57:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:06 smithi137 ceph-mon[32568]: pgmap v809: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-04-25T13:57:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:06 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2719323476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:06.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.700+0000 7f919b049640 1 -- 172.21.15.94:0/3464847391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194108920 msgr2=0x7f919410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.700+0000 7f919b049640 1 --2- 172.21.15.94:0/3464847391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194108920 0x7f919410ad40 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f918c009ca0 tx=0x7f918c02ecd0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- 172.21.15.94:0/3464847391 shutdown_connections 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- 172.21.15.94:0/3464847391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194108920 0x7f919410ad40 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- 172.21.15.94:0/3464847391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9194103c20 0x7f9194104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- 172.21.15.94:0/3464847391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91941029e0 0x7f91941036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- 172.21.15.94:0/3464847391 >> 172.21.15.94:0/3464847391 conn(0x7f91940fdbd0 msgr2=0x7f9194100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- 172.21.15.94:0/3464847391 shutdown_connections 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- 172.21.15.94:0/3464847391 wait complete. 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 Processor -- start 2024-04-25T13:57:06.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- start start 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941029e0 0x7f91941982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 0x7f919419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f919410d3a0 con 0x7f9194103c20 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f919410d220 con 0x7f91941029e0 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.701+0000 7f919b049640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f919410d520 con 0x7f9194108920 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9198de5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941029e0 0x7f91941982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 0x7f919419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60548/0 (socket says 172.21.15.94:60548) 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9193fff640 1 -- 172.21.15.94:0/4078276411 learned_addr learned my addr 172.21.15.94:0/4078276411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 -- 172.21.15.94:0/4078276411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941029e0 msgr2=0x7f91941982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941029e0 0x7f91941982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 -- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 msgr2=0x7f9194198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 -- 172.21.15.94:0/4078276411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919419fe00 con 0x7f9194108920 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9193fff640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f91995e6640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 0x7f919419f790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f918c02f1f0 tx=0x7f918c003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9191ffb640 1 -- 172.21.15.94:0/4078276411 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c03c040 con 0x7f9194108920 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941a0090 con 0x7f9194108920 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9191ffb640 1 -- 172.21.15.94:0/4078276411 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918c047070 con 0x7f9194108920 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.702+0000 7f9191ffb640 1 -- 172.21.15.94:0/4078276411 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0378e0 con 0x7f9194108920 2024-04-25T13:57:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.703+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a05d0 con 0x7f9194108920 2024-04-25T13:57:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 Processor -- start 2024-04-25T13:57:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9191ffb640 1 -- 172.21.15.94:0/4078276411 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f918c037a80 con 0x7f9194108920 2024-04-25T13:57:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 -- start start 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a159810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d966b0 0x560f1a15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f1a0bdd80 con 0x560f19db7390 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f1a077a20 con 0x560f19d966b0 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f1a0b9d40 con 0x560f19d8e4d0 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9198de5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60560/0 (socket says 172.21.15.94:60560) 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9198de5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51548/0 (socket says 172.21.15.94:51548) 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9193fff640 1 -- 172.21.15.94:0/2523253009 learned_addr learned my addr 172.21.15.94:0/2523253009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.706+0000 7f9193fff640 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d966b0 0x560f1a15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9191ffb640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 0x7f91840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 -- 172.21.15.94:0/2523253009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d966b0 msgr2=0x560f1a15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d966b0 0x560f1a15aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 -- 172.21.15.94:0/2523253009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 msgr2=0x560f1a159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 -- 172.21.15.94:0/2523253009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1a082c30 con 0x560f19d8e4d0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f91995e6640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 0x7f91840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9191ffb640 1 -- 172.21.15.94:0/4078276411 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f918c103940 con 0x7f9194108920 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f9198de5640 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f91941022b0 tx=0x7f9188005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9094fbdc0831882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f917effd640 1 -- 172.21.15.94:0/2523253009 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188009b10 con 0x560f19d8e4d0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f917effd640 1 -- 172.21.15.94:0/2523253009 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188009cb0 con 0x560f19d8e4d0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.707+0000 7f917effd640 1 -- 172.21.15.94:0/2523253009 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188003e10 con 0x560f19d8e4d0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 -- 172.21.15.94:0/2523253009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 msgr2=0x560f1a159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91995e6640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 0x7f91840b2080 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f918c003fd0 tx=0x7f918c092330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f91941022b0 tx=0x7f9188005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 -- 172.21.15.94:0/2523253009 shutdown_connections 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d966b0 0x560f1a15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a159d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/2523253009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 -- 172.21.15.94:0/2523253009 >> 172.21.15.94:0/2523253009 conn(0x560f19dab4e0 msgr2=0x560f1a155020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 -- 172.21.15.94:0/2523253009 shutdown_connections 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.708+0000 7f91a1fd3740 1 -- 172.21.15.94:0/2523253009 wait complete. 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 -- start start 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 Processor -- start 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a163a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a16a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f1a15b530 con 0x560f19d8e4d0 2024-04-25T13:57:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f1a044dc0 con 0x560f19db7390 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f1a15b6b0 con 0x560f19d966b0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f91a1fd3740 10 client.0 _ll_register_callbacks cb 0x560f1a060600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60564/0 (socket says 172.21.15.94:60564) 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f9193fff640 1 -- 172.21.15.94:0/3769578810 learned_addr learned my addr 172.21.15.94:0/3769578810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f9198de5640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.709+0000 7f9193fff640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a16a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 -- 172.21.15.94:0/3769578810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 msgr2=0x560f1a163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a163a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 -- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 msgr2=0x560f1a16a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a16a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 -- 172.21.15.94:0/3769578810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f19dcf3f0 con 0x560f19d8e4d0 2024-04-25T13:57:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9198de5640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9193fff640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f9180013fd0 tx=0x7f9180015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b334d4efd90b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180003e80 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f9180005020 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9188009cb0 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9180005220 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f91a1fd3740 10 client.21003 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f19e997d0 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188009cb0 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f918002a070 con 0x560f19d8e4d0 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.710+0000 7f91a1fd3740 10 client.21003 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f91a1fd3740 10 client.21003 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f91a1fd3740 20 client.21003 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d75f60 0x560f1a1712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f91a1fd3740 4 client.21003 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f1a0cb990 con 0x560f19d75f60 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f9198de5640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d75f60 0x560f1a1712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f9198de5640 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d75f60 0x560f1a1712b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=744f2b44a2084ed9 server_cookie=7278e09671dfacab in_seq=0 out_seq=0 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.711+0000 7f9173fff640 10 client.21003 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f9173fff640 1 -- 172.21.15.94:0/3769578810 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f1a0cb990 con 0x560f19d75f60 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f9173fff640 10 client.21003 handle_command_reply: tid=0 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 client.21003 shutdown 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 msgr2=0x560f1a163500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f9180013fd0 tx=0x7f9180015040 comp rx=0 tx=0).stop 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 shutdown_connections 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d75f60 0x560f1a1712b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a16a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3769578810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d8e4d0 0x560f1a163500 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.712+0000 7f9173fff640 0 client.21003 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.713+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 >> 172.21.15.94:0/3769578810 conn(0x560f19dab4e0 msgr2=0x560f1a157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.713+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 shutdown_connections 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.713+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3769578810 wait complete. 2024-04-25T13:57:06.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.713+0000 7f91a1fd3740 20 client.21003 trim_cache size 0 max 16384 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 Processor -- start 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 -- start start 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a1630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91941a91a0 con 0x560f19d966b0 2024-04-25T13:57:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91941aada0 con 0x560f19db7390 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91941a9870 con 0x560f19d8e4d0 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f9198de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f9198de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60568/0 (socket says 172.21.15.94:60568) 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f9198de5640 1 -- 172.21.15.94:0/3078778494 learned_addr learned my addr 172.21.15.94:0/3078778494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f9193fff640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.727+0000 7f9193fff640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a1630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 -- 172.21.15.94:0/3078778494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 msgr2=0x560f1a1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 -- 172.21.15.94:0/3078778494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 msgr2=0x560f1a1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a1630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 -- 172.21.15.94:0/3078778494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1a155b60 con 0x560f19d966b0 2024-04-25T13:57:06.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9193fff640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f9198de5640 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f91941022b0 tx=0x7f918800d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d5d17101fc0be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f917e7fc640 1 -- 172.21.15.94:0/3078778494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188009c30 con 0x560f19d966b0 2024-04-25T13:57:06.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f917e7fc640 1 -- 172.21.15.94:0/3078778494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918802f070 con 0x560f19d966b0 2024-04-25T13:57:06.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.728+0000 7f917e7fc640 1 -- 172.21.15.94:0/3078778494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918802a590 con 0x560f19d966b0 2024-04-25T13:57:06.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3078778494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 msgr2=0x560f1a154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f91941022b0 tx=0x7f918800d3f0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3078778494 shutdown_connections 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19db7390 0x560f1a1630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19d966b0 0x560f1a154f20 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/3078778494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d8e4d0 0x560f1a1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3078778494 >> 172.21.15.94:0/3078778494 conn(0x560f19dab4e0 msgr2=0x560f1a15e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3078778494 shutdown_connections 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- 172.21.15.94:0/3078778494 wait complete. 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.729+0000 7f91a1fd3740 1 -- start start 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 Processor -- start 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d8e4d0 0x560f1a166c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a1671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91941aa6d0 con 0x560f19db7390 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91941a9f40 con 0x560f19d8e4d0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91941a7180 con 0x560f19d966b0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f91a1fd3740 10 client.0 _ll_register_callbacks cb 0x7f9180009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9198de5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a1671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9193fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d8e4d0 0x560f1a166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9193fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60578/0 (socket says 172.21.15.94:60578) 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9193fff640 1 -- 172.21.15.94:0/1367457291 learned_addr learned my addr 172.21.15.94:0/1367457291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.730+0000 7f9198de5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a1671b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51566/0 (socket says 172.21.15.94:51566) 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 -- 172.21.15.94:0/1367457291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 msgr2=0x560f1a1671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a1671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 -- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d8e4d0 msgr2=0x560f1a166c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d8e4d0 0x560f1a166c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 -- 172.21.15.94:0/1367457291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1a16a170 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9193fff640 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f9180015010 tx=0x7f91800095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcce6fb576fd432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9173fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180030040 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f918000ebc0 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f918802f070 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9180004060 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f91a1fd3740 10 client.21009 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.731+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f1a1d89d0 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918802f070 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9180036070 con 0x560f19db7390 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 10 client.21009 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 10 client.21009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:57:06.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 20 client.21009 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d4b3a0 0x560f1a1db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 4 client.21009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f1a0c5800 con 0x560f19d4b3a0 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.732+0000 7f9198de5640 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d4b3a0 0x560f1a1db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f9198de5640 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d4b3a0 0x560f1a1db420 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d3bb86722a0b478 server_cookie=33b1989ad270336 in_seq=0 out_seq=0 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f9173fff640 10 client.21009 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f9173fff640 1 -- 172.21.15.94:0/1367457291 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f1a0c5800 con 0x560f19d4b3a0 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f9173fff640 10 client.21009 handle_command_reply: tid=0 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 client.21009 shutdown 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 msgr2=0x560f1a1d7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f9180015010 tx=0x7f91800095e0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 shutdown_connections 2024-04-25T13:57:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560f19d4b3a0 0x560f1a1db420 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f19db7390 0x560f1a1d7610 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f19d966b0 0x560f1a1671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f91a1fd3740 1 --2- 172.21.15.94:0/1367457291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f19d8e4d0 0x560f1a166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.733+0000 7f9173fff640 0 client.21009 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.734+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 >> 172.21.15.94:0/1367457291 conn(0x560f19dab4e0 msgr2=0x560f1a154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.734+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 shutdown_connections 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.734+0000 7f91a1fd3740 1 -- 172.21.15.94:0/1367457291 wait complete. 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.734+0000 7f91a1fd3740 20 client.21009 trim_cache size 0 max 16384 2024-04-25T13:57:06.739 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:06.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 msgr2=0x7f91840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 0x7f91840b2080 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f918c003fd0 tx=0x7f918c092330 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 msgr2=0x7f919419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 0x7f919419f790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f918c02f1f0 tx=0x7f918c003ee0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 shutdown_connections 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91840afb70 0x7f91840b2080 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9194108920 0x7f919419f790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9194103c20 0x7f9194198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 --2- 172.21.15.94:0/4078276411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91941029e0 0x7f91941982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 >> 172.21.15.94:0/4078276411 conn(0x7f91940fdbd0 msgr2=0x7f9194104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 shutdown_connections 2024-04-25T13:57:06.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.735+0000 7f919b049640 1 -- 172.21.15.94:0/4078276411 wait complete. 2024-04-25T13:57:06.748 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:57:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.867+0000 7fe578501640 1 -- 172.21.15.94:0/3913187948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570108920 msgr2=0x7fe57010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.867+0000 7fe578501640 1 --2- 172.21.15.94:0/3913187948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570108920 0x7fe57010ad40 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fe568009ca0 tx=0x7fe56802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- 172.21.15.94:0/3913187948 shutdown_connections 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- 172.21.15.94:0/3913187948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe570108920 0x7fe57010ad40 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- 172.21.15.94:0/3913187948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570103c20 0x7fe570104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- 172.21.15.94:0/3913187948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5701029e0 0x7fe5701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- 172.21.15.94:0/3913187948 >> 172.21.15.94:0/3913187948 conn(0x7fe5700fdbd0 msgr2=0x7fe570100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- 172.21.15.94:0/3913187948 shutdown_connections 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- 172.21.15.94:0/3913187948 wait complete. 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 Processor -- start 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- start start 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570103c20 0x7fe570199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 0x7fe5701a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e510 con 0x7fe5701029e0 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e390 con 0x7fe570108920 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.868+0000 7fe578501640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe57010e690 con 0x7fe570103c20 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60606/0 (socket says 172.21.15.94:60606) 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 -- 172.21.15.94:0/4228446158 learned_addr learned my addr 172.21.15.94:0/4228446158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe575a9c640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570103c20 0x7fe570199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe576a9e640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 0x7fe5701a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 -- 172.21.15.94:0/4228446158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570103c20 msgr2=0x7fe570199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570103c20 0x7fe570199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 -- 172.21.15.94:0/4228446158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 msgr2=0x7fe5701a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 0x7fe5701a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 -- 172.21.15.94:0/4228446158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570069f90 con 0x7fe5701029e0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe576a9e640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 0x7fe5701a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe57629d640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c380 tx=0x7fe56c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe5677fe640 1 -- 172.21.15.94:0/4228446158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004030 con 0x7fe5701029e0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57006a280 con 0x7fe5701029e0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe5677fe640 1 -- 172.21.15.94:0/4228446158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56c0041d0 con 0x7fe5701029e0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe5677fe640 1 -- 172.21.15.94:0/4228446158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c01e790 con 0x7fe5701029e0 2024-04-25T13:57:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.869+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57006a780 con 0x7fe5701029e0 2024-04-25T13:57:06.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.872+0000 7fe57f3b0740 1 Processor -- start 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 -- start start 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21f1390 0x558ca2593810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2593d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe5677fe640 1 -- 172.21.15.94:0/4228446158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe56c01e950 con 0x7fe5701029e0 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558ca24f7d80 con 0x558ca21d06b0 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ca24b1a20 con 0x558ca21c84d0 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ca24f3d40 con 0x558ca21f1390 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60616/0 (socket says 172.21.15.94:60616) 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 -- 172.21.15.94:0/3379531983 learned_addr learned my addr 172.21.15.94:0/3379531983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe576a9e640 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21f1390 0x558ca2593810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe57629d640 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2593d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 -- 172.21.15.94:0/3379531983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21f1390 msgr2=0x558ca2593810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21f1390 0x558ca2593810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 -- 172.21.15.94:0/3379531983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 msgr2=0x558ca2593d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2593d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 -- 172.21.15.94:0/3379531983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca24bcc30 con 0x558ca21d06b0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.873+0000 7fe575a9c640 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fe560009c90 tx=0x7fe560011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db31d2cfdb4ad174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe564bfe640 1 -- 172.21.15.94:0/3379531983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560005220 con 0x558ca21d06b0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe564bfe640 1 -- 172.21.15.94:0/3379531983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560004c50 con 0x558ca21d06b0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe564bfe640 1 -- 172.21.15.94:0/3379531983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5600196d0 con 0x558ca21d06b0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe5677fe640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe576a9e640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.874+0000 7fe5677fe640 1 -- 172.21.15.94:0/4228446158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fe56c0e6fe0 con 0x7fe5701029e0 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- 172.21.15.94:0/3379531983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 msgr2=0x558ca2594a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe576a9e640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x558ca2594860 tx=0x7fe568009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fe560009c90 tx=0x7fe560011810 comp rx=0 tx=0).stop 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- 172.21.15.94:0/3379531983 shutdown_connections 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21d06b0 0x558ca2594a20 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2593d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/3379531983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21f1390 0x558ca2593810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- 172.21.15.94:0/3379531983 >> 172.21.15.94:0/3379531983 conn(0x558ca21e54e0 msgr2=0x558ca258f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- 172.21.15.94:0/3379531983 shutdown_connections 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- 172.21.15.94:0/3379531983 wait complete. 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 -- start start 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 Processor -- start 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca259d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.875+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca259db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558ca2595530 con 0x558ca21f1390 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ca247edc0 con 0x558ca21c84d0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ca25956b0 con 0x558ca21d06b0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 10 client.0 _ll_register_callbacks cb 0x558ca249a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60620/0 (socket says 172.21.15.94:60620) 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 -- 172.21.15.94:0/2291026041 learned_addr learned my addr 172.21.15.94:0/2291026041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca259db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57629d640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca259d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 -- 172.21.15.94:0/2291026041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 msgr2=0x558ca259db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca259db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 -- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 msgr2=0x558ca259d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca259d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 -- 172.21.15.94:0/2291026041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca22093f0 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe568035000 tx=0x7fe56802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efde7a502ef1832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568004250 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fe5680043f0 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560004c50 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5680045f0 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 10 client.21021 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.876+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558ca22d37d0 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560004c50 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe568052070 con 0x558ca21f1390 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 10 client.21021 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 10 client.21021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 20 client.21021 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21aff60 0x558ca25ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 4 client.21021 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca2505990 con 0x558ca21aff60 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57629d640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21aff60 0x558ca25ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe57629d640 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21aff60 0x558ca25ab3a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a913e2410c896469 server_cookie=4c872d594b48fbd2 in_seq=0 out_seq=0 2024-04-25T13:57:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.877+0000 7fe559ffb640 10 client.21021 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.879+0000 7fe559ffb640 1 -- 172.21.15.94:0/2291026041 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ca2505990 con 0x558ca21aff60 2024-04-25T13:57:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.879+0000 7fe559ffb640 10 client.21021 handle_command_reply: tid=0 2024-04-25T13:57:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 client.21021 shutdown 2024-04-25T13:57:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 msgr2=0x558ca25a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe568035000 tx=0x7fe56802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 shutdown_connections 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21aff60 0x558ca25ab3a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4a00 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca259db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe559ffb640 0 client.21021 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2291026041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca259d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 >> 172.21.15.94:0/2291026041 conn(0x558ca21e54e0 msgr2=0x558ca258e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 shutdown_connections 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2291026041 wait complete. 2024-04-25T13:57:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.880+0000 7fe57f3b0740 20 client.21021 trim_cache size 0 max 16384 2024-04-25T13:57:06.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.894+0000 7fe57f3b0740 1 Processor -- start 2024-04-25T13:57:06.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.894+0000 7fe57f3b0740 1 -- start start 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.894+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2592530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.894+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca2592a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701a92a0 con 0x558ca21f1390 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701aaea0 con 0x558ca21c84d0 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701a9970 con 0x558ca21d06b0 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60628/0 (socket says 172.21.15.94:60628) 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe576a9e640 1 -- 172.21.15.94:0/2911570709 learned_addr learned my addr 172.21.15.94:0/2911570709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57629d640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2592530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca2592a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 -- 172.21.15.94:0/2911570709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 msgr2=0x558ca2592530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2592530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 -- 172.21.15.94:0/2911570709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 msgr2=0x558ca25a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 -- 172.21.15.94:0/2911570709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca25a4e60 con 0x558ca21d06b0 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca2592a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558ca259e640 tx=0x7fe560009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72216b0a97710dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.895+0000 7fe57629d640 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2592530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.896+0000 7fe55bfff640 1 -- 172.21.15.94:0/2911570709 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560005270 con 0x558ca21d06b0 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.896+0000 7fe55bfff640 1 -- 172.21.15.94:0/2911570709 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560005410 con 0x558ca21d06b0 2024-04-25T13:57:06.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.896+0000 7fe55bfff640 1 -- 172.21.15.94:0/2911570709 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560019720 con 0x558ca21d06b0 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911570709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 msgr2=0x558ca2592a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca2592a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558ca259e640 tx=0x7fe560009850 comp rx=0 tx=0).stop 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911570709 shutdown_connections 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21d06b0 0x558ca2592a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911570709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21c84d0 0x558ca2592530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911570709 >> 172.21.15.94:0/2911570709 conn(0x558ca21e54e0 msgr2=0x558ca2591e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911570709 shutdown_connections 2024-04-25T13:57:06.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911570709 wait complete. 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 -- start start 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 Processor -- start 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21c84d0 0x558ca25a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21d06b0 0x558ca25a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.897+0000 7fe57f3b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701aa7d0 con 0x558ca21f1390 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701aa040 con 0x558ca21d06b0 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701a7280 con 0x558ca21c84d0 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 10 client.0 _ll_register_callbacks cb 0x7fe568009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60632/0 (socket says 172.21.15.94:60632) 2024-04-25T13:57:06.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57629d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21c84d0 0x558ca25a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 -- 172.21.15.94:0/2911789799 learned_addr learned my addr 172.21.15.94:0/2911789799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe575a9c640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21d06b0 0x558ca25a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 -- 172.21.15.94:0/2911789799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21c84d0 msgr2=0x558ca25a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21c84d0 0x558ca25a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 -- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21d06b0 msgr2=0x558ca25a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21d06b0 0x558ca25a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 -- 172.21.15.94:0/2911789799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca25a39c0 con 0x558ca21f1390 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe576a9e640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe568030780 tx=0x7fe568030880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f078f96dea9f2c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568003d40 con 0x558ca21f1390 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fe568030c80 con 0x558ca21f1390 2024-04-25T13:57:06.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560005410 con 0x558ca21f1390 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5680317f0 con 0x558ca21f1390 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 10 client.21027 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.898+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558ca2613710 con 0x558ca21f1390 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560005410 con 0x558ca21f1390 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe568052040 con 0x558ca21f1390 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 10 client.21027 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 10 client.21027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 20 client.21027 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21853a0 0x558ca2616090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 4 client.21027 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ca24ff800 con 0x558ca21853a0 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57629d640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21853a0 0x558ca2616090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:06.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe559ffb640 10 client.21027 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.899+0000 7fe57629d640 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21853a0 0x558ca2616090 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ecc68f280e36d2 server_cookie=76eec8be91c144ce in_seq=0 out_seq=0 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.901+0000 7fe559ffb640 1 -- 172.21.15.94:0/2911789799 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ca24ff800 con 0x558ca21853a0 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.901+0000 7fe559ffb640 10 client.21027 handle_command_reply: tid=0 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.901+0000 7fe57f3b0740 1 client.21027 shutdown 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 msgr2=0x558ca25a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fe568030780 tx=0x7fe568030880 comp rx=0 tx=0).stop 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 shutdown_connections 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ca21853a0 0x558ca2616090 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ca21f1390 0x558ca25a3400 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca21d06b0 0x558ca25a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe559ffb640 0 client.21027 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 --2- 172.21.15.94:0/2911789799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ca21c84d0 0x558ca25a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 >> 172.21.15.94:0/2911789799 conn(0x558ca21e54e0 msgr2=0x558ca2592850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 shutdown_connections 2024-04-25T13:57:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 1 -- 172.21.15.94:0/2911789799 wait complete. 2024-04-25T13:57:06.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe57f3b0740 20 client.21027 trim_cache size 0 max 16384 2024-04-25T13:57:06.907 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 msgr2=0x7fe55c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x558ca2594860 tx=0x7fe568009360 comp rx=0 tx=0).stop 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 msgr2=0x7fe570199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.902+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c380 tx=0x7fe56c00c840 comp rx=0 tx=0).stop 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 shutdown_connections 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe570108920 0x7fe5701a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe570103c20 0x7fe570199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 --2- 172.21.15.94:0/4228446158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5701029e0 0x7fe570199410 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 >> 172.21.15.94:0/4228446158 conn(0x7fe5700fdbd0 msgr2=0x7fe570104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 shutdown_connections 2024-04-25T13:57:06.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:06.903+0000 7fe578501640 1 -- 172.21.15.94:0/4228446158 wait complete. 2024-04-25T13:57:07.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 40674 21.73 MB/sec warmup 109 sec latency 70.340 ms 2024-04-25T13:57:07.915 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:57:07.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:57:07.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- 172.21.15.94:0/1390422960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8101e30 msgr2=0x7f88d8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 --2- 172.21.15.94:0/1390422960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8101e30 0x7f88d8102240 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009900 tx=0x7f88c802a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- 172.21.15.94:0/1390422960 shutdown_connections 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 --2- 172.21.15.94:0/1390422960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8109780 0x7f88d810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 --2- 172.21.15.94:0/1390422960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8106c10 0x7f88d81090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 --2- 172.21.15.94:0/1390422960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8101e30 0x7f88d8102240 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- 172.21.15.94:0/1390422960 >> 172.21.15.94:0/1390422960 conn(0x7f88d80fd340 msgr2=0x7f88d80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- 172.21.15.94:0/1390422960 shutdown_connections 2024-04-25T13:57:08.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- 172.21.15.94:0/1390422960 wait complete. 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 Processor -- start 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.043+0000 7f88dd75e640 1 -- start start 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 0x7f88d819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8106c10 0x7f88d819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f5e0 con 0x7f88d8109780 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f460 con 0x7f88d8101e30 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88dd75e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f760 con 0x7f88d8106c10 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 0x7f88d819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60644/0 (socket says 172.21.15.94:60644) 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d77fe640 1 -- 172.21.15.94:0/2506030003 learned_addr learned my addr 172.21.15.94:0/2506030003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d67fc640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8106c10 0x7f88d819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 -- 172.21.15.94:0/2506030003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8106c10 msgr2=0x7f88d819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8106c10 0x7f88d819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 -- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 msgr2=0x7f88d81a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 -- 172.21.15.94:0/2506030003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d81a1d70 con 0x7f88d8101e30 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.044+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 0x7f88d819a230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006010 tx=0x7f88c8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.045+0000 7f88bffff640 1 -- 172.21.15.94:0/2506030003 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c803f070 con 0x7f88d8101e30 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.045+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d81a2000 con 0x7f88d8101e30 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.045+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d81a2540 con 0x7f88d8101e30 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.045+0000 7f88bffff640 1 -- 172.21.15.94:0/2506030003 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c8031910 con 0x7f88d8101e30 2024-04-25T13:57:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.045+0000 7f88bffff640 1 -- 172.21.15.94:0/2506030003 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8035070 con 0x7f88d8101e30 2024-04-25T13:57:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.047+0000 7f88e460d740 1 Processor -- start 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.047+0000 7f88e460d740 1 -- start start 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5d7410 0x56236b9784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88bffff640 1 -- 172.21.15.94:0/2506030003 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f88c80037f0 con 0x7f88d8101e30 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5b6730 0x56236b97f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56236b8ddb00 con 0x56236b5ae550 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56236b879920 con 0x56236b5d7410 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88e460d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56236b8d9ac0 con 0x56236b5b6730 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5d7410 0x56236b9784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60652/0 (socket says 172.21.15.94:60652) 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 -- 172.21.15.94:0/1466600031 learned_addr learned my addr 172.21.15.94:0/1466600031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d67fc640 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5b6730 0x56236b97f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 -- 172.21.15.94:0/1466600031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5b6730 msgr2=0x56236b97f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5b6730 0x56236b97f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 -- 172.21.15.94:0/1466600031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5d7410 msgr2=0x56236b9784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5d7410 0x56236b9784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 -- 172.21.15.94:0/1466600031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236b6b77e0 con 0x56236b5ae550 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f88c8102fc0 tx=0x7f88c81070e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b81baf3d62b4466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.048+0000 7f88bffff640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 0x7f88c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.049+0000 7f88bd7fa640 1 -- 172.21.15.94:0/1466600031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80c3940 con 0x56236b5ae550 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.049+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 0x7f88c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.049+0000 7f88bd7fa640 1 -- 172.21.15.94:0/1466600031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c810c070 con 0x56236b5ae550 2024-04-25T13:57:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.049+0000 7f88bd7fa640 1 -- 172.21.15.94:0/1466600031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80c6a60 con 0x56236b5ae550 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.049+0000 7f88bffff640 1 -- 172.21.15.94:0/2506030003 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f88c80feeb0 con 0x7f88d8101e30 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- 172.21.15.94:0/1466600031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 msgr2=0x56236b978a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 0x7f88c00b2000 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x56236b979670 tx=0x7f88d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f88c8102fc0 tx=0x7f88c81070e0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- 172.21.15.94:0/1466600031 shutdown_connections 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5b6730 0x56236b97f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b978a20 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 --2- 172.21.15.94:0/1466600031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5d7410 0x56236b9784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- 172.21.15.94:0/1466600031 >> 172.21.15.94:0/1466600031 conn(0x56236b5cb560 msgr2=0x56236b973e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- 172.21.15.94:0/1466600031 shutdown_connections 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- 172.21.15.94:0/1466600031 wait complete. 2024-04-25T13:57:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 -- start start 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.050+0000 7f88e460d740 1 Processor -- start 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b982580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56236b980130 con 0x56236b5ae550 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56236b8775d0 con 0x56236b5b6730 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56236b9802b0 con 0x56236b5d7410 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88e460d740 10 client.0 _ll_register_callbacks cb 0x56236b865080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60662/0 (socket says 172.21.15.94:60662) 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 -- 172.21.15.94:0/856733232 learned_addr learned my addr 172.21.15.94:0/856733232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d67fc640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d77fe640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 -- 172.21.15.94:0/856733232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 msgr2=0x56236b989470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 -- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 msgr2=0x56236b982580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b982580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d6ffd640 1 -- 172.21.15.94:0/856733232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236b717f70 con 0x56236b5ae550 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.051+0000 7f88d67fc640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b982580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f88c80fc960 tx=0x7f88c80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cacf8ef2d639755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8048070 con 0x56236b5ae550 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f88c8102ae0 con 0x56236b5ae550 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c810c070 con 0x56236b5ae550 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88c80c6c40 con 0x56236b5ae550 2024-04-25T13:57:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 10 client.21039 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56236b5af170 con 0x56236b5ae550 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c810c070 con 0x56236b5ae550 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f88c804b060 con 0x56236b5ae550 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 10 client.21039 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 10 client.21039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 20 client.21039 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236b5961f0 0x56236b993a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 4 client.21039 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.052+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236b8eb710 con 0x56236b5961f0 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.053+0000 7f88d67fc640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236b5961f0 0x56236b993a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.053+0000 7f88baff5640 10 client.21039 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.053+0000 7f88d67fc640 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236b5961f0 0x56236b993a90 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaca5f216d306090 server_cookie=3b1e98263eb46e92 in_seq=0 out_seq=0 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88baff5640 1 -- 172.21.15.94:0/856733232 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56236b8eb710 con 0x56236b5961f0 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88baff5640 10 client.21039 handle_command_reply: tid=0 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 client.21039 shutdown 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 msgr2=0x56236b982040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f88c80fc960 tx=0x7f88c80314a0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 shutdown_connections 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236b5961f0 0x56236b993a90 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88baff5640 0 client.21039 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b982580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 --2- 172.21.15.94:0/856733232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5ae550 0x56236b982040 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.056+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 >> 172.21.15.94:0/856733232 conn(0x56236b5cb560 msgr2=0x56236b972ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.057+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 shutdown_connections 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.057+0000 7f88e460d740 1 -- 172.21.15.94:0/856733232 wait complete. 2024-04-25T13:57:08.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.057+0000 7f88e460d740 20 client.21039 trim_cache size 0 max 16384 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 Processor -- start 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 -- start start 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b56b630 0x56236b998280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81a93a0 con 0x56236b5b6730 2024-04-25T13:57:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81aafa0 con 0x56236b56b630 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88e460d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81a9a70 con 0x56236b5d7410 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88d67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60676/0 (socket says 172.21.15.94:60676) 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88d6ffd640 1 -- 172.21.15.94:0/2720690456 learned_addr learned my addr 172.21.15.94:0/2720690456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.076+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b56b630 0x56236b998280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 -- 172.21.15.94:0/2720690456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b56b630 msgr2=0x56236b998280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b56b630 0x56236b998280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 -- 172.21.15.94:0/2720690456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 msgr2=0x56236b982320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 -- 172.21.15.94:0/2720690456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236b97fb20 con 0x56236b5d7410 2024-04-25T13:57:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88d67fc640 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56236b9941d0 tx=0x7f88cc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd66ae7ccafc3d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88bcff9640 1 -- 172.21.15.94:0/2720690456 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc004230 con 0x56236b5d7410 2024-04-25T13:57:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88bcff9640 1 -- 172.21.15.94:0/2720690456 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc0043d0 con 0x56236b5d7410 2024-04-25T13:57:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.077+0000 7f88bcff9640 1 -- 172.21.15.94:0/2720690456 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc003bf0 con 0x56236b5d7410 2024-04-25T13:57:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 -- 172.21.15.94:0/2720690456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 msgr2=0x56236b989400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56236b9941d0 tx=0x7f88cc005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 -- 172.21.15.94:0/2720690456 shutdown_connections 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b56b630 0x56236b998280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b989400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 --2- 172.21.15.94:0/2720690456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b5b6730 0x56236b982320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 -- 172.21.15.94:0/2720690456 >> 172.21.15.94:0/2720690456 conn(0x56236b5cb560 msgr2=0x56236b981c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 -- 172.21.15.94:0/2720690456 shutdown_connections 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.078+0000 7f88e460d740 1 -- 172.21.15.94:0/2720690456 wait complete. 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 -- start start 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 Processor -- start 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b56b630 0x56236b986670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b986bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81aa8d0 con 0x56236b56b630 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81aa140 con 0x56236b5b6730 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88d81a7380 con 0x56236b5d7410 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.080+0000 7f88e460d740 10 client.0 _ll_register_callbacks cb 0x7f88c8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b986bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51696/0 (socket says 172.21.15.94:51696) 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 -- 172.21.15.94:0/2287499451 learned_addr learned my addr 172.21.15.94:0/2287499451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b56b630 0x56236b986670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 -- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 msgr2=0x56236b986bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b986bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 -- 172.21.15.94:0/2287499451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b56b630 msgr2=0x56236b986670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b56b630 0x56236b986670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 -- 172.21.15.94:0/2287499451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236b990440 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88d77fe640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88d000ffb0 tx=0x7f88d0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5867e2e737256eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.081+0000 7f88baff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0011c90 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f88d0004b20 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc0043d0 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88d0011e30 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 10 client.30187 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56236b988fd0 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc0043d0 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f88d0034430 con 0x56236b5d7410 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 10 client.30187 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 10 client.30187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 20 client.30187 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236ba007f0 0x56236ba02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 4 client.30187 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.082+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56236b9810e0 con 0x56236ba007f0 2024-04-25T13:57:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.083+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236ba007f0 0x56236ba02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.083+0000 7f88baff5640 10 client.30187 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.083+0000 7f88d6ffd640 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236ba007f0 0x56236ba02c10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d24a04f2aa402c server_cookie=2c11cb2ad81c6bef in_seq=0 out_seq=0 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.092+0000 7f88baff5640 1 -- 172.21.15.94:0/2287499451 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56236b9810e0 con 0x56236ba007f0 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.092+0000 7f88baff5640 10 client.30187 handle_command_reply: tid=0 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.092+0000 7f88e460d740 1 client.30187 shutdown 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 msgr2=0x56236b987a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88d000ffb0 tx=0x7f88d0004a40 comp rx=0 tx=0).stop 2024-04-25T13:57:08.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 shutdown_connections 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56236ba007f0 0x56236ba02c10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b5d7410 0x56236b987a70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56236b5b6730 0x56236b986bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 --2- 172.21.15.94:0/2287499451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56236b56b630 0x56236b986670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88baff5640 0 client.30187 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 >> 172.21.15.94:0/2287499451 conn(0x56236b5cb560 msgr2=0x56236b976c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 shutdown_connections 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 1 -- 172.21.15.94:0/2287499451 wait complete. 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88e460d740 20 client.30187 trim_cache size 0 max 16384 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:57:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "57d949db-b5c7-415b-b545-358b601f1a23", 2024-04-25T13:57:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:57:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 msgr2=0x7f88c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 0x7f88c00b2000 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x56236b979670 tx=0x7f88d0009500 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 msgr2=0x7f88d819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 0x7f88d819a230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006010 tx=0x7f88c8004270 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 shutdown_connections 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88c00afaf0 0x7f88c00b2000 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d8109780 0x7f88d81a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88d8106c10 0x7f88d819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 --2- 172.21.15.94:0/2506030003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d8101e30 0x7f88d819a230 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 >> 172.21.15.94:0/2506030003 conn(0x7f88d80fd340 msgr2=0x7f88d8108d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 shutdown_connections 2024-04-25T13:57:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.093+0000 7f88dd75e640 1 -- 172.21.15.94:0/2506030003 wait complete. 2024-04-25T13:57:08.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:57:08.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:57:08.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 41077 21.68 MB/sec warmup 110 sec latency 183.212 ms 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 -- 172.21.15.94:0/3014401939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840104270 msgr2=0x7f6840106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 --2- 172.21.15.94:0/3014401939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840104270 0x7f6840106690 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f683c009900 tx=0x7f683c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 -- 172.21.15.94:0/3014401939 shutdown_connections 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 --2- 172.21.15.94:0/3014401939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 0x7f684010bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 --2- 172.21.15.94:0/3014401939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840106c60 0x7f68401090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 --2- 172.21.15.94:0/3014401939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840104270 0x7f6840106690 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 -- 172.21.15.94:0/3014401939 >> 172.21.15.94:0/3014401939 conn(0x7f68400fdbd0 msgr2=0x7f6840100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 -- 172.21.15.94:0/3014401939 shutdown_connections 2024-04-25T13:57:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 -- 172.21.15.94:0/3014401939 wait complete. 2024-04-25T13:57:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.226+0000 7f6847b46640 1 Processor -- start 2024-04-25T13:57:08.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 -- start start 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840104270 0x7f684019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 0x7f68401a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f684010f5e0 con 0x7f6840106c60 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f684010f460 con 0x7f6840109770 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6847b46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f684010f760 con 0x7f6840104270 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60696/0 (socket says 172.21.15.94:60696) 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 -- 172.21.15.94:0/3838357887 learned_addr learned my addr 172.21.15.94:0/3838357887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68460e3640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 0x7f68401a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68458e2640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840104270 0x7f684019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 -- 172.21.15.94:0/3838357887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840104270 msgr2=0x7f684019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840104270 0x7f684019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 -- 172.21.15.94:0/3838357887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 msgr2=0x7f68401a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 0x7f68401a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 -- 172.21.15.94:0/3838357887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68401a1e10 con 0x7f6840106c60 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f68450e1640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9f0 tx=0x7f683000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.227+0000 7f6836ffd640 1 -- 172.21.15.94:0/3838357887 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683000cd80 con 0x7f6840106c60 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.228+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68401a2100 con 0x7f6840106c60 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.228+0000 7f6836ffd640 1 -- 172.21.15.94:0/3838357887 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68300054d0 con 0x7f6840106c60 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.228+0000 7f6836ffd640 1 -- 172.21.15.94:0/3838357887 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830010690 con 0x7f6840106c60 2024-04-25T13:57:08.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.228+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68401a2650 con 0x7f6840106c60 2024-04-25T13:57:08.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.230+0000 7f684e9fb740 1 Processor -- start 2024-04-25T13:57:08.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.230+0000 7f684e9fb740 1 -- start start 2024-04-25T13:57:08.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.230+0000 7f6836ffd640 1 -- 172.21.15.94:0/3838357887 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6830010830 con 0x7f6840106c60 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56240a9a0d80 con 0x56240a6714d0 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56240a95a8b0 con 0x56240a6796b0 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f684e9fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56240a99cd40 con 0x56240a69a390 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68450e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51726/0 (socket says 172.21.15.94:51726) 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa3bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60712/0 (socket says 172.21.15.94:60712) 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 -- 172.21.15.94:0/2711196788 learned_addr learned my addr 172.21.15.94:0/2711196788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 -- 172.21.15.94:0/2711196788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 msgr2=0x56240aa3cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 -- 172.21.15.94:0/2711196788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 msgr2=0x56240aa3bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa3bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68460e3640 1 -- 172.21.15.94:0/2711196788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a92ba40 con 0x56240a69a390 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.231+0000 7f68450e1640 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f6836ffd640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 0x7f682c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f68460e3640 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f68380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ad59ec4721bfb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f68458e2640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 0x7f682c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f6836ffd640 1 -- 172.21.15.94:0/3838357887 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f68300df040 con 0x7f6840106c60 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f682bfff640 1 -- 172.21.15.94:0/2711196788 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838004d20 con 0x56240a69a390 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f682bfff640 1 -- 172.21.15.94:0/2711196788 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838004050 con 0x56240a69a390 2024-04-25T13:57:08.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.232+0000 7f682bfff640 1 -- 172.21.15.94:0/2711196788 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838005280 con 0x56240a69a390 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 -- 172.21.15.94:0/2711196788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 msgr2=0x56240aa3ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f68458e2640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 0x7f682c0b2000 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f683c009230 tx=0x7f683c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f68380118d0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 -- 172.21.15.94:0/2711196788 shutdown_connections 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa3bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2711196788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a69a390 0x56240aa3ba60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.233+0000 7f684e9fb740 1 -- 172.21.15.94:0/2711196788 >> 172.21.15.94:0/2711196788 conn(0x56240a68e4e0 msgr2=0x56240aa37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.234+0000 7f684e9fb740 1 -- 172.21.15.94:0/2711196788 shutdown_connections 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.234+0000 7f684e9fb740 1 -- 172.21.15.94:0/2711196788 wait complete. 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.234+0000 7f684e9fb740 1 -- start start 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.234+0000 7f684e9fb740 1 Processor -- start 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6796b0 0x56240aa46070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 0x56240aa4cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56240aa3d780 con 0x56240a6714d0 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56240a927d10 con 0x56240a69a390 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56240aa3d900 con 0x56240a6796b0 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f684e9fb740 10 client.0 _ll_register_callbacks cb 0x56240a954210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60726/0 (socket says 172.21.15.94:60726) 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 -- 172.21.15.94:0/2666405775 learned_addr learned my addr 172.21.15.94:0/2666405775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68460e3640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 0x56240aa4cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68450e1640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6796b0 0x56240aa46070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 -- 172.21.15.94:0/2666405775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6796b0 msgr2=0x56240aa46070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6796b0 0x56240aa46070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 -- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 msgr2=0x56240aa4cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 0x56240aa4cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 -- 172.21.15.94:0/2666405775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240a77c7d0 con 0x56240a6714d0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68460e3640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 0x56240aa4cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68458e2640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f683c009900 tx=0x7f683c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36544a44565f4054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68297fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.235+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c04d070 con 0x56240a6714d0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f683c004440 con 0x56240a6714d0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6838004050 con 0x56240a6714d0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f683c059050 con 0x56240a6714d0 2024-04-25T13:57:08.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 10 client.21057 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56240a6b23f0 con 0x56240a6714d0 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838004050 con 0x56240a6714d0 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f683c004c10 con 0x56240a6714d0 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 10 client.21057 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 10 client.21057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 20 client.21057 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a658f60 0x56240aa53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 4 client.21057 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56240a9ae990 con 0x56240a658f60 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.236+0000 7f68450e1640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a658f60 0x56240aa53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.237+0000 7f68297fa640 10 client.21057 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.237+0000 7f68450e1640 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a658f60 0x56240aa53470 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a2484fb7ed40b3 server_cookie=ad575d9ad02a1ecd in_seq=0 out_seq=0 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.240+0000 7f68297fa640 1 -- 172.21.15.94:0/2666405775 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56240a9ae990 con 0x56240a658f60 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.240+0000 7f68297fa640 10 client.21057 handle_command_reply: tid=0 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.240+0000 7f684e9fb740 1 client.21057 shutdown 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 msgr2=0x56240aa45b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f683c009900 tx=0x7f683c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 shutdown_connections 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a658f60 0x56240aa53470 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a69a390 0x56240aa4cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6796b0 0x56240aa46070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f68297fa640 0 client.21057 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2666405775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6714d0 0x56240aa45b30 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 >> 172.21.15.94:0/2666405775 conn(0x56240a68e4e0 msgr2=0x56240aa363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 shutdown_connections 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 1 -- 172.21.15.94:0/2666405775 wait complete. 2024-04-25T13:57:08.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.241+0000 7f684e9fb740 20 client.21057 trim_cache size 0 max 16384 2024-04-25T13:57:08.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 Processor -- start 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 -- start start 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6714d0 0x56240aa3a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ab660 con 0x56240a68e4e0 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ad260 con 0x56240a6796b0 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f684e9fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68401abd30 con 0x56240a6714d0 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60728/0 (socket says 172.21.15.94:60728) 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 -- 172.21.15.94:0/2454783759 learned_addr learned my addr 172.21.15.94:0/2454783759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68460e3640 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68450e1640 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6714d0 0x56240aa3a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 -- 172.21.15.94:0/2454783759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6714d0 msgr2=0x56240aa3a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6714d0 0x56240aa3a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 -- 172.21.15.94:0/2454783759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 msgr2=0x56240aa3acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 -- 172.21.15.94:0/2454783759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240a6720f0 con 0x56240a68e4e0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.261+0000 7f68458e2640 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f683c0035f0 tx=0x7f683c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5167bf2ba2d2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.262+0000 7f682b7fe640 1 -- 172.21.15.94:0/2454783759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c061030 con 0x56240a68e4e0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.262+0000 7f682b7fe640 1 -- 172.21.15.94:0/2454783759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c004240 con 0x56240a68e4e0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.262+0000 7f682b7fe640 1 -- 172.21.15.94:0/2454783759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c04d050 con 0x56240a68e4e0 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.262+0000 7f684e9fb740 1 -- 172.21.15.94:0/2454783759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 msgr2=0x56240aa44b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.262+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f683c0035f0 tx=0x7f683c0039f0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- 172.21.15.94:0/2454783759 shutdown_connections 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6796b0 0x56240aa3acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a6714d0 0x56240aa3a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- 172.21.15.94:0/2454783759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a68e4e0 0x56240aa44b80 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- 172.21.15.94:0/2454783759 >> 172.21.15.94:0/2454783759 conn(0x56240aa46240 msgr2=0x56240aa3a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- 172.21.15.94:0/2454783759 shutdown_connections 2024-04-25T13:57:08.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- 172.21.15.94:0/2454783759 wait complete. 2024-04-25T13:57:08.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- start start 2024-04-25T13:57:08.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 Processor -- start 2024-04-25T13:57:08.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 0x56240aa4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a68e4e0 0x56240aa4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68401acb90 con 0x56240a6796b0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ac400 con 0x56240a6714d0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.263+0000 7f684e9fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a9640 con 0x56240a68e4e0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f684e9fb740 10 client.0 _ll_register_callbacks cb 0x7f683c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60744/0 (socket says 172.21.15.94:60744) 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 -- 172.21.15.94:0/1823412354 learned_addr learned my addr 172.21.15.94:0/1823412354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68458e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 0x56240aa4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68460e3640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a68e4e0 0x56240aa4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 -- 172.21.15.94:0/1823412354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a68e4e0 msgr2=0x56240aa4cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a68e4e0 0x56240aa4cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 -- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 msgr2=0x56240aa4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 0x56240aa4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 -- 172.21.15.94:0/1823412354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240a9614b0 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68458e2640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 0x56240aa4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68450e1640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f683000c6a0 tx=0x7f68300df2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b5b98dba7d0d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68297fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68300a6c90 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f68300a37b0 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f683c004240 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f68300a7030 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.264+0000 7f684e9fb740 10 client.21063 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56240aaba9d0 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c004240 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f68300a7780 con 0x56240a6796b0 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 10 client.21063 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 10 client.21063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 20 client.21063 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a69a390 0x56240aabd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 4 client.21063 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56240a9a8800 con 0x56240a69a390 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.265+0000 7f68458e2640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a69a390 0x56240aabd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.266+0000 7f68297fa640 10 client.21063 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.266+0000 7f68458e2640 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a69a390 0x56240aabd430 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74444119090c5bbb server_cookie=319872f8d8e75d26 in_seq=0 out_seq=0 2024-04-25T13:57:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f68297fa640 1 -- 172.21.15.94:0/1823412354 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56240a9a8800 con 0x56240a69a390 2024-04-25T13:57:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f68297fa640 10 client.21063 handle_command_reply: tid=0 2024-04-25T13:57:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 client.21063 shutdown 2024-04-25T13:57:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 msgr2=0x56240aa4c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f683000c6a0 tx=0x7f68300df2b0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 shutdown_connections 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56240a69a390 0x56240aabd430 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56240a68e4e0 0x56240aa4cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56240a6796b0 0x56240aa4c6a0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 --2- 172.21.15.94:0/1823412354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56240a6714d0 0x56240aa4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f68297fa640 0 client.21063 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 >> 172.21.15.94:0/1823412354 conn(0x56240aa46240 msgr2=0x56240aa3aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 shutdown_connections 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 1 -- 172.21.15.94:0/1823412354 wait complete. 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.269+0000 7f684e9fb740 20 client.21063 trim_cache size 0 max 16384 2024-04-25T13:57:08.272 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (66 inodes in the stack)", 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "57d949db-b5c7-415b-b545-358b601f1a23": { 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "57d949db-b5c7-415b-b545-358b601f1a23", 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:57:08.273 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 msgr2=0x7f682c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 0x7f682c0b2000 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f683c009230 tx=0x7f683c031400 comp rx=0 tx=0).stop 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 msgr2=0x7f684019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9f0 tx=0x7f683000edf0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 shutdown_connections 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f682c0afaf0 0x7f682c0b2000 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840109770 0x7f68401a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6840106c60 0x7f684019a810 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 --2- 172.21.15.94:0/3838357887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840104270 0x7f684019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 >> 172.21.15.94:0/3838357887 conn(0x7f68400fdbd0 msgr2=0x7f6840106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 shutdown_connections 2024-04-25T13:57:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:08.270+0000 7f6847b46640 1 -- 172.21.15.94:0/3838357887 wait complete. 2024-04-25T13:57:08.282 INFO:tasks.cephfs.filesystem:scrub status for tag:57d949db-b5c7-415b-b545-358b601f1a23 - {'path': '/', 'tag': '57d949db-b5c7-415b-b545-358b601f1a23', 'options': 'recursive,force'} 2024-04-25T13:57:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:08 smithi094 ceph-mon[29195]: pgmap v810: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T13:57:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:08 smithi138 ceph-mon[32977]: pgmap v810: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T13:57:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:08 smithi137 ceph-mon[32568]: pgmap v810: 97 pgs: 97 active+clean; 244 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T13:57:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 -- 172.21.15.94:0/930007303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44108ae0 msgr2=0x7f0d4410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/930007303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44108ae0 0x7f0d4410af00 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009ca0 tx=0x7f0d3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 -- 172.21.15.94:0/930007303 shutdown_connections 2024-04-25T13:57:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/930007303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44108ae0 0x7f0d4410af00 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/930007303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44104440 0x7f0d44102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/930007303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 0x7f0d44102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.851+0000 7f0d4b49f640 1 -- 172.21.15.94:0/930007303 >> 172.21.15.94:0/930007303 conn(0x7f0d440fd390 msgr2=0x7f0d440ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- 172.21.15.94:0/930007303 shutdown_connections 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- 172.21.15.94:0/930007303 wait complete. 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 Processor -- start 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- start start 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 0x7f0d4419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44108ae0 0x7f0d441a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410e560 con 0x7f0d44104440 2024-04-25T13:57:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410e3e0 con 0x7f0d44108ae0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d4b49f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410e6e0 con 0x7f0d44103fa0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d48a3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.852+0000 7f0d48a3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60762/0 (socket says 172.21.15.94:60762) 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 -- 172.21.15.94:0/153423911 learned_addr learned my addr 172.21.15.94:0/153423911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d4923b640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 0x7f0d4419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44108ae0 0x7f0d441a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 -- 172.21.15.94:0/153423911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 msgr2=0x7f0d4419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 0x7f0d4419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 -- 172.21.15.94:0/153423911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44108ae0 msgr2=0x7f0d441a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44108ae0 0x7f0d441a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 -- 172.21.15.94:0/153423911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d441a1d10 con 0x7f0d44104440 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9f0 tx=0x7f0d3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d3a7fc640 1 -- 172.21.15.94:0/153423911 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400cd80 con 0x7f0d44104440 2024-04-25T13:57:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d3a7fc640 1 -- 172.21.15.94:0/153423911 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340108e0 con 0x7f0d44104440 2024-04-25T13:57:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d441a1fa0 con 0x7f0d44104440 2024-04-25T13:57:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d3a7fc640 1 -- 172.21.15.94:0/153423911 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34018980 con 0x7f0d44104440 2024-04-25T13:57:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.853+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a2490 con 0x7f0d44104440 2024-04-25T13:57:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.856+0000 7f0d5242f740 1 Processor -- start 2024-04-25T13:57:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.856+0000 7f0d5242f740 1 -- start start 2024-04-25T13:57:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d3a7fc640 1 -- 172.21.15.94:0/153423911 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0d34010430 con 0x7f0d44104440 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f3dfa52460 con 0x55f3df7232f0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f3dfa0c590 con 0x55f3df74c1b0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d5242f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3dfa4e420 con 0x55f3df72b4d0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d49a3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d49a3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58368/0 (socket says 172.21.15.94:58368) 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d49a3c640 1 -- 172.21.15.94:0/1734798097 learned_addr learned my addr 172.21.15.94:0/1734798097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 -- 172.21.15.94:0/1734798097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 msgr2=0x55f3dfaf3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 -- 172.21.15.94:0/1734798097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 msgr2=0x55f3dfaec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 -- 172.21.15.94:0/1734798097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3dfa190d0 con 0x55f3df7232f0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.857+0000 7f0d4923b640 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaecef0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bbc0 tx=0x7f0d400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1ffc3a33535690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d3a7fc640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 0x7f0d300b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d2f7fe640 1 -- 172.21.15.94:0/1734798097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40004030 con 0x55f3df7232f0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 0x7f0d300b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d2f7fe640 1 -- 172.21.15.94:0/1734798097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d400035f0 con 0x55f3df7232f0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d2f7fe640 1 -- 172.21.15.94:0/1734798097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400116a0 con 0x55f3df7232f0 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.858+0000 7f0d3a7fc640 1 -- 172.21.15.94:0/153423911 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f0d340deff0 con 0x7f0d44104440 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- 172.21.15.94:0/1734798097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 msgr2=0x55f3dfaecef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaecef0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bbc0 tx=0x7f0d400107a0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 0x7f0d300b1ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c02f500 tx=0x7f0d3c02f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- 172.21.15.94:0/1734798097 shutdown_connections 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaecef0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 --2- 172.21.15.94:0/1734798097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- 172.21.15.94:0/1734798097 >> 172.21.15.94:0/1734798097 conn(0x55f3df740300 msgr2=0x55f3dfae8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- 172.21.15.94:0/1734798097 shutdown_connections 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- 172.21.15.94:0/1734798097 wait complete. 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.859+0000 7f0d5242f740 1 -- start start 2024-04-25T13:57:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 Processor -- start 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaf6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f3dfaf4840 con 0x55f3df7232f0 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f3df9d9b60 con 0x55f3df74c1b0 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3dfaf49c0 con 0x55f3df72b4d0 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60788/0 (socket says 172.21.15.94:60788) 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d4923b640 1 -- 172.21.15.94:0/880414495 learned_addr learned my addr 172.21.15.94:0/880414495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.860+0000 7f0d5242f740 10 client.0 _ll_register_callbacks cb 0x55f3df9fe2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaf6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 -- 172.21.15.94:0/880414495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 msgr2=0x55f3dfaf9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 -- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 msgr2=0x55f3dfaf6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaf6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 -- 172.21.15.94:0/880414495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3df764210 con 0x55f3df7232f0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d4923b640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004230 tx=0x7f0d40004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514583cd108b5a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4002a060 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f0d4000bd70 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d400035f0 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d40003b10 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d5242f740 10 client.21075 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.861+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f3df830620 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d400035f0 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0d40004b00 con 0x55f3df7232f0 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 10 client.21075 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 10 client.21075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 20 client.21075 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df70ad80 0x55f3dfb0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 4 client.21075 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3dfa60070 con 0x55f3df70ad80 2024-04-25T13:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df70ad80 0x55f3dfb0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d2cff9640 10 client.21075 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.862+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df70ad80 0x55f3dfb0c670 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d7d9e2fae5823e server_cookie=6de1597c13f00acc in_seq=0 out_seq=0 2024-04-25T13:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d2cff9640 1 -- 172.21.15.94:0/880414495 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3dfa60070 con 0x55f3df70ad80 2024-04-25T13:57:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d2cff9640 10 client.21075 handle_command_reply: tid=0 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 client.21075 shutdown 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 msgr2=0x55f3dfaf9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004230 tx=0x7f0d40004260 comp rx=0 tx=0).stop 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 shutdown_connections 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df70ad80 0x55f3dfb0c670 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df74c1b0 0x55f3dfaf6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfaf9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 --2- 172.21.15.94:0/880414495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfaf9a80 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d2cff9640 0 client.21075 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 >> 172.21.15.94:0/880414495 conn(0x55f3df740300 msgr2=0x55f3dfaeb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 shutdown_connections 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 1 -- 172.21.15.94:0/880414495 wait complete. 2024-04-25T13:57:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.866+0000 7f0d5242f740 20 client.21075 trim_cache size 0 max 16384 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 Processor -- start 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 -- start start 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df7232f0 0x55f3dfae8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfb10e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441a93d0 con 0x55f3df70ad80 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aafd0 con 0x55f3df7232f0 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d5242f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441a9aa0 con 0x55f3df72b4d0 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60796/0 (socket says 172.21.15.94:60796) 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 -- 172.21.15.94:0/4016686098 learned_addr learned my addr 172.21.15.94:0/4016686098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df7232f0 0x55f3dfae8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d49a3c640 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfb10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 -- 172.21.15.94:0/4016686098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 msgr2=0x55f3dfb10e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfb10e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 -- 172.21.15.94:0/4016686098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df7232f0 msgr2=0x55f3dfae8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df7232f0 0x55f3dfae8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.884+0000 7f0d4923b640 1 -- 172.21.15.94:0/4016686098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3dfaf41b0 con 0x55f3df70ad80 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.885+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b4e0 tx=0x7f0d400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d32ceb5307869a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.885+0000 7f0d2effd640 1 -- 172.21.15.94:0/4016686098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4002a070 con 0x55f3df70ad80 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.885+0000 7f0d2effd640 1 -- 172.21.15.94:0/4016686098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40003cb0 con 0x55f3df70ad80 2024-04-25T13:57:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.885+0000 7f0d2effd640 1 -- 172.21.15.94:0/4016686098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40004980 con 0x55f3df70ad80 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- 172.21.15.94:0/4016686098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 msgr2=0x55f3dfafa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b4e0 tx=0x7f0d400107a0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- 172.21.15.94:0/4016686098 shutdown_connections 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df72b4d0 0x55f3dfb10e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df7232f0 0x55f3dfae8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4016686098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df70ad80 0x55f3dfafa010 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- 172.21.15.94:0/4016686098 >> 172.21.15.94:0/4016686098 conn(0x55f3df740300 msgr2=0x55f3dfae7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- 172.21.15.94:0/4016686098 shutdown_connections 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- 172.21.15.94:0/4016686098 wait complete. 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 -- start start 2024-04-25T13:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 Processor -- start 2024-04-25T13:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 0x55f3dfb1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.886+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d5242f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df72b4d0 0x55f3dfb1c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d5242f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aa900 con 0x55f3df7232f0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d5242f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aa170 con 0x55f3df72b4d0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d5242f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441a73b0 con 0x55f3df70ad80 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d5242f740 10 client.0 _ll_register_callbacks cb 0x7f0d4000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d48a3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d48a3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60804/0 (socket says 172.21.15.94:60804) 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 0x55f3dfb1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d49a3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df72b4d0 0x55f3dfb1c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d48a3a640 1 -- 172.21.15.94:0/4131634554 learned_addr learned my addr 172.21.15.94:0/4131634554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 -- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df72b4d0 msgr2=0x55f3dfb1c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df72b4d0 0x55f3dfb1c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 -- 172.21.15.94:0/4131634554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 msgr2=0x55f3dfb1b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 -- 172.21.15.94:0/4131634554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3dfb1cb70 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d4923b640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 0x55f3dfb1b140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004ed0 tx=0x7f0d4000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7979e444804a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.887+0000 7f0d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4003c070 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40003cb0 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f0d40011dc0 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d4002a040 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 10 client.30220 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f3dfb1de00 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40003cb0 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0d4003f030 con 0x55f3df70ad80 2024-04-25T13:57:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 10 client.30220 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 10 client.30220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 20 client.30220 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df74c1b0 0x55f3dfb76950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 4 client.30220 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3dfa59ee0 con 0x55f3df74c1b0 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.888+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df74c1b0 0x55f3dfb76950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.889+0000 7f0d2cff9640 10 client.30220 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:08.889+0000 7f0d48a3a640 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df74c1b0 0x55f3dfb76950 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5180496e34a9ddfe server_cookie=89a09e8cf99212e in_seq=0 out_seq=0 2024-04-25T13:57:09.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 41302 21.60 MB/sec warmup 111 sec latency 188.221 ms 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d2cff9640 1 -- 172.21.15.94:0/4131634554 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3dfa59ee0 con 0x55f3df74c1b0 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d2cff9640 10 client.30220 handle_command_reply: tid=0 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 client.30220 shutdown 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 msgr2=0x55f3dfb1b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 0x55f3dfb1b140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d40004ed0 tx=0x7f0d4000b8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 shutdown_connections 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f3df74c1b0 0x55f3dfb76950 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3df72b4d0 0x55f3dfb1c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3df7232f0 0x55f3dfb1b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 --2- 172.21.15.94:0/4131634554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3df70ad80 0x55f3dfb1b140 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d2cff9640 0 client.30220 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 >> 172.21.15.94:0/4131634554 conn(0x55f3df740300 msgr2=0x55f3dfb11850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 shutdown_connections 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 1 -- 172.21.15.94:0/4131634554 wait complete. 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.421+0000 7f0d5242f740 20 client.30220 trim_cache size 0 max 16384 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:57:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 msgr2=0x7f0d300b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 0x7f0d300b1ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c02f500 tx=0x7f0d3c02f310 comp rx=0 tx=0).stop 2024-04-25T13:57:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 msgr2=0x7f0d4419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9f0 tx=0x7f0d3400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 shutdown_connections 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0d300afae0 0x7f0d300b1ff0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d44108ae0 0x7f0d441a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d44104440 0x7f0d4419a710 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 --2- 172.21.15.94:0/153423911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d44103fa0 0x7f0d4419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 >> 172.21.15.94:0/153423911 conn(0x7f0d440fd390 msgr2=0x7f0d44102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 shutdown_connections 2024-04-25T13:57:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:09.422+0000 7f0d4b49f640 1 -- 172.21.15.94:0/153423911 wait complete. 2024-04-25T13:57:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:57:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:09.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:09.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:09 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:57:09.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:09.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:09 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:09.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:09.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:09 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 41588 21.57 MB/sec warmup 112 sec latency 280.535 ms 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: pgmap v811: 97 pgs: 97 active+clean; 245 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 467 op/s 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:10 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: pgmap v811: 97 pgs: 97 active+clean; 245 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 467 op/s 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:10 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: pgmap v811: 97 pgs: 97 active+clean; 245 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.3 MiB/s wr, 467 op/s 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:10.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:10 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:11.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 42083 21.65 MB/sec warmup 113 sec latency 102.448 ms 2024-04-25T13:57:12.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 42483 21.58 MB/sec warmup 114 sec latency 95.431 ms 2024-04-25T13:57:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:12 smithi094 ceph-mon[29195]: pgmap v812: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-04-25T13:57:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:12 smithi138 ceph-mon[32977]: pgmap v812: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-04-25T13:57:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:12 smithi137 ceph-mon[32568]: pgmap v812: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-04-25T13:57:13.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 42765 21.55 MB/sec warmup 115 sec latency 200.601 ms 2024-04-25T13:57:14.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 43106 21.57 MB/sec warmup 116 sec latency 74.194 ms 2024-04-25T13:57:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:14 smithi094 ceph-mon[29195]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2024-04-25T13:57:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:14 smithi138 ceph-mon[32977]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2024-04-25T13:57:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:14 smithi137 ceph-mon[32568]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 404 op/s 2024-04-25T13:57:15.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 43531 21.55 MB/sec warmup 117 sec latency 98.405 ms 2024-04-25T13:57:16.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 43896 21.55 MB/sec warmup 118 sec latency 91.136 ms 2024-04-25T13:57:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:16 smithi094 ceph-mon[29195]: pgmap v814: 97 pgs: 97 active+clean; 246 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 648 op/s 2024-04-25T13:57:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:16 smithi138 ceph-mon[32977]: pgmap v814: 97 pgs: 97 active+clean; 246 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 648 op/s 2024-04-25T13:57:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:16 smithi137 ceph-mon[32568]: pgmap v814: 97 pgs: 97 active+clean; 246 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 648 op/s 2024-04-25T13:57:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:16.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:17.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 44286 21.53 MB/sec warmup 119 sec latency 62.046 ms 2024-04-25T13:57:17.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:17.518 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:17.518 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:17.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:17.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:17.519 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:17.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:17.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:17.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:18 smithi138 ceph-mon[32977]: pgmap v815: 97 pgs: 97 active+clean; 246 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-04-25T13:57:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:18 smithi137 ceph-mon[32568]: pgmap v815: 97 pgs: 97 active+clean; 246 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-04-25T13:57:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:18 smithi094 ceph-mon[29195]: pgmap v815: 97 pgs: 97 active+clean; 246 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 543 op/s 2024-04-25T13:57:19.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 44966 18.68 MB/sec execute 1 sec latency 236.755 ms 2024-04-25T13:57:20.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 45434 24.01 MB/sec execute 2 sec latency 51.496 ms 2024-04-25T13:57:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:20 smithi138 ceph-mon[32977]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-04-25T13:57:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:20 smithi137 ceph-mon[32568]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-04-25T13:57:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:20 smithi094 ceph-mon[29195]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-04-25T13:57:21.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 45815 21.76 MB/sec execute 3 sec latency 68.923 ms 2024-04-25T13:57:22.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 46150 20.43 MB/sec execute 4 sec latency 110.316 ms 2024-04-25T13:57:22.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:22 smithi138 ceph-mon[32977]: pgmap v817: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 -- 172.21.15.94:0/3527768891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa290106f70 msgr2=0x7fa290109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 --2- 172.21.15.94:0/3527768891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa290106f70 0x7fa290109390 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fa278009900 tx=0x7fa27802a410 comp rx=0 tx=0).stop 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 -- 172.21.15.94:0/3527768891 shutdown_connections 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 --2- 172.21.15.94:0/3527768891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2901098d0 0x7fa29010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 --2- 172.21.15.94:0/3527768891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa290106f70 0x7fa290109390 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 --2- 172.21.15.94:0/3527768891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 -- 172.21.15.94:0/3527768891 >> 172.21.15.94:0/3527768891 conn(0x7fa2900fdc10 msgr2=0x7fa2901000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 -- 172.21.15.94:0/3527768891 shutdown_connections 2024-04-25T13:57:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.524+0000 7fa29505a640 1 -- 172.21.15.94:0/3527768891 wait complete. 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 Processor -- start 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 -- start start 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa290106f70 0x7fa290102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa29010f650 con 0x7fa2901098d0 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa29010f4d0 con 0x7fa290104610 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa29505a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa29010f7d0 con 0x7fa290106f70 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44322/0 (socket says 172.21.15.94:44322) 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 -- 172.21.15.94:0/4155800790 learned_addr learned my addr 172.21.15.94:0/4155800790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa287fff640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa290106f70 0x7fa290102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 -- 172.21.15.94:0/4155800790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa290106f70 msgr2=0x7fa290102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa290106f70 0x7fa290102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 -- 172.21.15.94:0/4155800790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 msgr2=0x7fa290103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 -- 172.21.15.94:0/4155800790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa290102d70 con 0x7fa2901098d0 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.525+0000 7fa28f59e640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9d0 tx=0x7fa28800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.526+0000 7fa28cd99640 1 -- 172.21.15.94:0/4155800790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800cd80 con 0x7fa2901098d0 2024-04-25T13:57:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.526+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2901a4340 con 0x7fa2901098d0 2024-04-25T13:57:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.526+0000 7fa28cd99640 1 -- 172.21.15.94:0/4155800790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa288005020 con 0x7fa2901098d0 2024-04-25T13:57:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.526+0000 7fa28cd99640 1 -- 172.21.15.94:0/4155800790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880116d0 con 0x7fa2901098d0 2024-04-25T13:57:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.526+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901a4800 con 0x7fa2901098d0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.528+0000 7fa28cd99640 1 -- 172.21.15.94:0/4155800790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa2880051d0 con 0x7fa2901098d0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 Processor -- start 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 -- start start 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47172970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47179df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bd470d8460 con 0x55bd46da92f0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bd47092590 con 0x55bd46dd21b0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28cd99640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 0x7fa27c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa29bfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd470d4420 con 0x55bd46db14d0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44326/0 (socket says 172.21.15.94:44326) 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28ed9d640 1 -- 172.21.15.94:0/2114802534 learned_addr learned my addr 172.21.15.94:0/2114802534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28f59e640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 0x7fa27c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa287fff640 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47172970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa28cd99640 1 -- 172.21.15.94:0/4155800790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa2880df150 con 0x7fa2901098d0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.529+0000 7fa287fff640 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47179df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 -- 172.21.15.94:0/2114802534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 msgr2=0x55bd47179df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47179df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 -- 172.21.15.94:0/2114802534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 msgr2=0x55bd47172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47172970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 -- 172.21.15.94:0/2114802534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4709f0d0 con 0x55bd46da92f0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa2800114f0 tx=0x7fa2800119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947b5c168bca85da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa28f59e640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 0x7fa27c0b1ff0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa2880a36c0 tx=0x7fa28806db60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa2857fa640 1 -- 172.21.15.94:0/2114802534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28001e070 con 0x55bd46da92f0 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa2857fa640 1 -- 172.21.15.94:0/2114802534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2800035f0 con 0x55bd46da92f0 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.530+0000 7fa2857fa640 1 -- 172.21.15.94:0/2114802534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28000f7a0 con 0x55bd46da92f0 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 -- 172.21.15.94:0/2114802534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 msgr2=0x55bd47172ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa2800114f0 tx=0x7fa2800119b0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 -- 172.21.15.94:0/2114802534 shutdown_connections 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47179df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd47172ef0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2114802534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47172970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 -- 172.21.15.94:0/2114802534 >> 172.21.15.94:0/2114802534 conn(0x55bd46dc6300 msgr2=0x55bd4716e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 -- 172.21.15.94:0/2114802534 shutdown_connections 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.531+0000 7fa29bfea740 1 -- 172.21.15.94:0/2114802534 wait complete. 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 -- start start 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 Processor -- start 2024-04-25T13:57:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd4717cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47183a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bd4717a840 con 0x55bd46da92f0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bd4705fb60 con 0x55bd46dd21b0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd4717a9c0 con 0x55bd46db14d0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa29bfea740 10 client.0 _ll_register_callbacks cb 0x55bd470842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44330/0 (socket says 172.21.15.94:44330) 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa287fff640 1 -- 172.21.15.94:0/1606023306 learned_addr learned my addr 172.21.15.94:0/1606023306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa287fff640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47183a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd4717cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.532+0000 7fa287fff640 1 -- 172.21.15.94:0/1606023306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 msgr2=0x55bd4717cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa287fff640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd4717cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa287fff640 1 -- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 msgr2=0x55bd47183a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa287fff640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47183a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa287fff640 1 -- 172.21.15.94:0/1606023306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd46dea210 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa287fff640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fa278031500 tx=0x7fa27802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83df04e53e320e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa2767fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278035040 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa2780035f0 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2800035f0 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa2780037f0 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa29bfea740 10 client.21093 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bd46eb6620 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.533+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2800035f0 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa27803d870 con 0x55bd46da92f0 2024-04-25T13:57:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 10 client.21093 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 10 client.21093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 20 client.21093 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d90d80 0x55bd4718a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 4 client.21093 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd470e6070 con 0x55bd46d90d80 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d90d80 0x55bd4718a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d90d80 0x55bd4718a370 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eded080111a0997a server_cookie=4549d1c54dafbf9d in_seq=0 out_seq=0 2024-04-25T13:57:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.534+0000 7fa2767fc640 10 client.21093 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa2767fc640 1 -- 172.21.15.94:0/1606023306 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd470e6070 con 0x55bd46d90d80 2024-04-25T13:57:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa2767fc640 10 client.21093 handle_command_reply: tid=0 2024-04-25T13:57:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 client.21093 shutdown 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 msgr2=0x55bd4717c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fa278031500 tx=0x7fa27802a410 comp rx=0 tx=0).stop 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 shutdown_connections 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d90d80 0x55bd4718a370 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd47183a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd4717cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1606023306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c5e0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa2767fc640 0 client.21093 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 >> 172.21.15.94:0/1606023306 conn(0x55bd46dc6300 msgr2=0x55bd47171210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 shutdown_connections 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 1 -- 172.21.15.94:0/1606023306 wait complete. 2024-04-25T13:57:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.542+0000 7fa29bfea740 20 client.21093 trim_cache size 0 max 16384 2024-04-25T13:57:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.559+0000 7fa29bfea740 1 Processor -- start 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.559+0000 7fa29bfea740 1 -- start start 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47183a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd4717ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901ad820 con 0x55bd46da92f0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901af420 con 0x55bd46dd21b0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa29bfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901adef0 con 0x55bd46db14d0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44346/0 (socket says 172.21.15.94:44346) 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 -- 172.21.15.94:0/1728371788 learned_addr learned my addr 172.21.15.94:0/1728371788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47183a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd4717ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 -- 172.21.15.94:0/1728371788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 msgr2=0x55bd47183a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47183a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 -- 172.21.15.94:0/1728371788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 msgr2=0x55bd4717ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd4717ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 -- 172.21.15.94:0/1728371788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4717d660 con 0x55bd46da92f0 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa287fff640 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa278033000 tx=0x7fa278000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a93291d7d3ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.560+0000 7fa284ff9640 1 -- 172.21.15.94:0/1728371788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278035070 con 0x55bd46da92f0 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.561+0000 7fa284ff9640 1 -- 172.21.15.94:0/1728371788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa27805a040 con 0x55bd46da92f0 2024-04-25T13:57:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.561+0000 7fa284ff9640 1 -- 172.21.15.94:0/1728371788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278059020 con 0x55bd46da92f0 2024-04-25T13:57:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.561+0000 7fa29bfea740 1 -- 172.21.15.94:0/1728371788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 msgr2=0x55bd4717c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.561+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa278033000 tx=0x7fa278000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- 172.21.15.94:0/1728371788 shutdown_connections 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46dd21b0 0x55bd4717ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47183a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- 172.21.15.94:0/1728371788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46da92f0 0x55bd4717c8c0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- 172.21.15.94:0/1728371788 >> 172.21.15.94:0/1728371788 conn(0x55bd46dc6300 msgr2=0x55bd4717c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- 172.21.15.94:0/1728371788 shutdown_connections 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- 172.21.15.94:0/1728371788 wait complete. 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- start start 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 Processor -- start 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46da92f0 0x55bd47180e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47181380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901aed50 con 0x55bd46dd21b0 2024-04-25T13:57:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901ae5c0 con 0x55bd46da92f0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901ab800 con 0x55bd46db14d0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa29bfea740 10 client.0 _ll_register_callbacks cb 0x7fa278005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44360/0 (socket says 172.21.15.94:44360) 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.562+0000 7fa287fff640 1 -- 172.21.15.94:0/2600417444 learned_addr learned my addr 172.21.15.94:0/2600417444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47181380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46da92f0 0x55bd47180e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 -- 172.21.15.94:0/2600417444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 msgr2=0x55bd47181380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47181380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 -- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46da92f0 msgr2=0x55bd47180e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46da92f0 0x55bd47180e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 -- 172.21.15.94:0/2600417444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd47182870 con 0x55bd46dd21b0 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47181380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa287fff640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fa2780316d0 tx=0x7fa278009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2653c99df5fe97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa2767fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278035070 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa2780035f0 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27805a040 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa278059070 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa29bfea740 10 client.21099 fetch_fsmap learned FSMap version 34 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bd47183b00 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.563+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa27805a040 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa27804c8e0 con 0x55bd46dd21b0 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 10 client.21099 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 10 client.21099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 20 client.21099 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d661c0 0x55bd471f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 4 client.21099 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:57:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd470dfee0 con 0x55bd46d661c0 2024-04-25T13:57:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d661c0 0x55bd471f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa2767fc640 10 client.21099 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:22.564+0000 7fa28ed9d640 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d661c0 0x55bd471f4c00 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84d146a37bd3086 server_cookie=1e20501e3e5724df in_seq=0 out_seq=0 2024-04-25T13:57:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:22 smithi137 ceph-mon[32568]: pgmap v817: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T13:57:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:22 smithi094 ceph-mon[29195]: pgmap v817: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T13:57:23.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 46358 19.25 MB/sec execute 5 sec latency 538.497 ms 2024-04-25T13:57:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa2767fc640 1 -- 172.21.15.94:0/2600417444 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd470dfee0 con 0x55bd46d661c0 2024-04-25T13:57:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa2767fc640 10 client.21099 handle_command_reply: tid=0 2024-04-25T13:57:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 client.21099 shutdown 2024-04-25T13:57:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 msgr2=0x55bd471822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fa2780316d0 tx=0x7fa278009680 comp rx=0 tx=0).stop 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 shutdown_connections 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bd46d661c0 0x55bd471f4c00 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bd46dd21b0 0x55bd471822b0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd46db14d0 0x55bd47181380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 --2- 172.21.15.94:0/2600417444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bd46da92f0 0x55bd47180e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 >> 172.21.15.94:0/2600417444 conn(0x55bd46dc6300 msgr2=0x55bd4717ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.236+0000 7fa2767fc640 0 client.21099 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 shutdown_connections 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29bfea740 1 -- 172.21.15.94:0/2600417444 wait complete. 2024-04-25T13:57:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29bfea740 20 client.21099 trim_cache size 0 max 16384 2024-04-25T13:57:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:57:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:57:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:57:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 msgr2=0x7fa27c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 0x7fa27c0b1ff0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa2880a36c0 tx=0x7fa28806db60 comp rx=0 tx=0).stop 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 msgr2=0x7fa290102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9d0 tx=0x7fa28800ee90 comp rx=0 tx=0).stop 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 shutdown_connections 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa27c0afae0 0x7fa27c0b1ff0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2901098d0 0x7fa290102640 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa290106f70 0x7fa290102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 --2- 172.21.15.94:0/4155800790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa290104610 0x7fa290103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.237+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 >> 172.21.15.94:0/4155800790 conn(0x7fa2900fdc10 msgr2=0x7fa290105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.238+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 shutdown_connections 2024-04-25T13:57:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:23.238+0000 7fa29505a640 1 -- 172.21.15.94:0/4155800790 wait complete. 2024-04-25T13:57:24.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 46744 19.71 MB/sec execute 6 sec latency 584.182 ms 2024-04-25T13:57:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:24 smithi138 ceph-mon[32977]: pgmap v818: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-04-25T13:57:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:24 smithi137 ceph-mon[32568]: pgmap v818: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-04-25T13:57:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:24 smithi094 ceph-mon[29195]: pgmap v818: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-04-25T13:57:25.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 47064 19.11 MB/sec execute 7 sec latency 216.980 ms 2024-04-25T13:57:25.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:25 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:25.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:25 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:25.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:25 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:26.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 47392 18.64 MB/sec execute 8 sec latency 178.215 ms 2024-04-25T13:57:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:26 smithi138 ceph-mon[32977]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 648 op/s 2024-04-25T13:57:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:26.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:26.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:57:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:26 smithi137 ceph-mon[32568]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 648 op/s 2024-04-25T13:57:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:57:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:26 smithi094 ceph-mon[29195]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 648 op/s 2024-04-25T13:57:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:57:27.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 47740 19.15 MB/sec execute 9 sec latency 57.808 ms 2024-04-25T13:57:27.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:27 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:27 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:27 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:28.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 48136 18.89 MB/sec execute 10 sec latency 236.986 ms 2024-04-25T13:57:28.543 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:28 smithi137 ceph-mon[32568]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T13:57:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:28 smithi094 ceph-mon[29195]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T13:57:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:28 smithi138 ceph-mon[32977]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T13:57:29.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 48436 18.66 MB/sec execute 11 sec latency 77.732 ms 2024-04-25T13:57:30.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 48873 19.53 MB/sec execute 12 sec latency 65.568 ms 2024-04-25T13:57:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:30 smithi137 ceph-mon[32568]: pgmap v821: 97 pgs: 97 active+clean; 256 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-04-25T13:57:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:30 smithi094 ceph-mon[29195]: pgmap v821: 97 pgs: 97 active+clean; 256 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-04-25T13:57:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:30 smithi138 ceph-mon[32977]: pgmap v821: 97 pgs: 97 active+clean; 256 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-04-25T13:57:31.138 INFO:tasks.workunit.client.0.smithi094.stdout: 10 49282 19.60 MB/sec execute 13 sec latency 66.317 ms 2024-04-25T13:57:32.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 49628 19.43 MB/sec execute 14 sec latency 74.107 ms 2024-04-25T13:57:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:32 smithi137 ceph-mon[32568]: pgmap v822: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-25T13:57:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:32 smithi094 ceph-mon[29195]: pgmap v822: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-25T13:57:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:32 smithi138 ceph-mon[32977]: pgmap v822: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-25T13:57:33.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 49916 19.16 MB/sec execute 15 sec latency 277.344 ms 2024-04-25T13:57:34.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 50266 19.22 MB/sec execute 16 sec latency 131.037 ms 2024-04-25T13:57:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:34 smithi137 ceph-mon[32568]: pgmap v823: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2024-04-25T13:57:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:34 smithi094 ceph-mon[29195]: pgmap v823: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2024-04-25T13:57:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:34 smithi138 ceph-mon[32977]: pgmap v823: 97 pgs: 97 active+clean; 248 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 475 op/s 2024-04-25T13:57:35.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 50640 19.12 MB/sec execute 17 sec latency 88.108 ms 2024-04-25T13:57:36.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 51008 19.18 MB/sec execute 18 sec latency 175.067 ms 2024-04-25T13:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 -- 172.21.15.94:0/3584967980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 msgr2=0x7f4ff0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 --2- 172.21.15.94:0/3584967980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff0104070 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 -- 172.21.15.94:0/3584967980 shutdown_connections 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 --2- 172.21.15.94:0/3584967980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0108960 0x7f4ff010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 --2- 172.21.15.94:0/3584967980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff0104070 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 --2- 172.21.15.94:0/3584967980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0102a20 0x7f4ff0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 -- 172.21.15.94:0/3584967980 >> 172.21.15.94:0/3584967980 conn(0x7f4ff00fdc10 msgr2=0x7f4ff01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 -- 172.21.15.94:0/3584967980 shutdown_connections 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.353+0000 7f4ff7708640 1 -- 172.21.15.94:0/3584967980 wait complete. 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 Processor -- start 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 -- start start 2024-04-25T13:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 0x7f4ff006ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0108960 0x7f4ff01a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff010e550 con 0x7f4ff0103c60 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff010e3d0 con 0x7f4ff0108960 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff7708640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff010e6d0 con 0x7f4ff0102a20 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44164/0 (socket says 172.21.15.94:44164) 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/2690147630 learned_addr learned my addr 172.21.15.94:0/2690147630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0108960 0x7f4ff01a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.354+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 0x7f4ff006ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/2690147630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 msgr2=0x7f4ff006ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 0x7f4ff006ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/2690147630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0108960 msgr2=0x7f4ff01a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0108960 0x7f4ff01a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/2690147630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff01a1d20 con 0x7f4ff0103c60 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 0x7f4ff006ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006010 tx=0x7f4fe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4fe67fc640 1 -- 172.21.15.94:0/2690147630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe003f030 con 0x7f4ff0103c60 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4fe67fc640 1 -- 172.21.15.94:0/2690147630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe0031910 con 0x7f4ff0103c60 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff01a1fb0 con 0x7f4ff0103c60 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4fe67fc640 1 -- 172.21.15.94:0/2690147630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0035070 con 0x7f4ff0103c60 2024-04-25T13:57:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.355+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff01a23c0 con 0x7f4ff0103c60 2024-04-25T13:57:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 Processor -- start 2024-04-25T13:57:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 -- start start 2024-04-25T13:57:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556169eb5460 con 0x556169b862f0 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556169e6f590 con 0x556169baf1b0 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556169eb1420 con 0x556169b8e4d0 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44180/0 (socket says 172.21.15.94:44180) 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3667776423 learned_addr learned my addr 172.21.15.94:0/3667776423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4fe67fc640 1 -- 172.21.15.94:0/2690147630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4fe0042030 con 0x7f4ff0103c60 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 -- 172.21.15.94:0/3667776423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 msgr2=0x556169f5b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 -- 172.21.15.94:0/3667776423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 msgr2=0x556169f54300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff5ca5640 1 -- 172.21.15.94:0/3667776423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556169e7c0d0 con 0x556169baf1b0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.358+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f53d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009f00 tx=0x7f4fe8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4ce141e5803e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4fe67fc640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 0x7f4fdc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4fdb7fe640 1 -- 172.21.15.94:0/3667776423 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8004d20 con 0x556169baf1b0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 0x7f4fdc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4fdb7fe640 1 -- 172.21.15.94:0/3667776423 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe8004030 con 0x556169baf1b0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4fdb7fe640 1 -- 172.21.15.94:0/3667776423 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8005280 con 0x556169baf1b0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.359+0000 7f4fe67fc640 1 -- 172.21.15.94:0/2690147630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f4fe00ff2b0 con 0x7f4ff0103c60 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 0x7f4fdc0b1ff0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x556169f55160 tx=0x7f4fec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3667776423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 msgr2=0x556169f53d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f53d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009f00 tx=0x7f4fe8011810 comp rx=0 tx=0).stop 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3667776423 shutdown_connections 2024-04-25T13:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f54300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3667776423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f53d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3667776423 >> 172.21.15.94:0/3667776423 conn(0x556169ba3300 msgr2=0x556169f4b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3667776423 shutdown_connections 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3667776423 wait complete. 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 -- start start 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.360+0000 7f4ffe5b6740 1 Processor -- start 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556169f5bc50 con 0x556169b862f0 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556169e3cb60 con 0x556169baf1b0 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556169f5bdd0 con 0x556169b8e4d0 2024-04-25T13:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ffe5b6740 10 client.0 _ll_register_callbacks cb 0x556169e612b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44190/0 (socket says 172.21.15.94:44190) 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3378948509 learned_addr learned my addr 172.21.15.94:0/3378948509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3378948509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 msgr2=0x556169f5e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 msgr2=0x556169f650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3378948509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556169bc7210 con 0x556169b862f0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f4fec011a90 tx=0x7f4fec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96476153faea0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.361+0000 7f4fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec009c70 con 0x556169b862f0 2024-04-25T13:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f4fec01d640 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe8004030 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4fec01d840 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 10 client.21111 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556169c93620 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe8004030 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec026030 con 0x556169b862f0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 10 client.21111 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 10 client.21111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 20 client.21111 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169b6dd80 0x556169f6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 4 client.21111 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.362+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556169ec3070 con 0x556169b6dd80 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.363+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169b6dd80 0x556169f6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.363+0000 7f4fd8ff9640 10 client.21111 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.363+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169b6dd80 0x556169f6ba10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24c7fdd2f8ed78e server_cookie=bc253539ed00def2 in_seq=0 out_seq=0 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3378948509 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556169ec3070 con 0x556169b6dd80 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4fd8ff9640 10 client.21111 handle_command_reply: tid=0 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 client.21111 shutdown 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 msgr2=0x556169f5dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f4fec011a90 tx=0x7f4fec004730 comp rx=0 tx=0).stop 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 shutdown_connections 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169b6dd80 0x556169f6ba10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169baf1b0 0x556169f650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4fd8ff9640 0 client.21111 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f5e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3378948509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f5dc00 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.368+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 >> 172.21.15.94:0/3378948509 conn(0x556169ba3300 msgr2=0x556169f4a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.369+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 shutdown_connections 2024-04-25T13:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.369+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3378948509 wait complete. 2024-04-25T13:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.369+0000 7f4ffe5b6740 20 client.21111 trim_cache size 0 max 16384 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.384+0000 7f4ffe5b6740 1 Processor -- start 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.384+0000 7f4ffe5b6740 1 -- start start 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.384+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.384+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f71210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01a9200 con 0x556169b862f0 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01aada0 con 0x556169b6dd80 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01a98d0 con 0x556169b8e4d0 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44192/0 (socket says 172.21.15.94:44192) 2024-04-25T13:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/4079352415 learned_addr learned my addr 172.21.15.94:0/4079352415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f4e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58556/0 (socket says 172.21.15.94:58556) 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f71210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/4079352415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 msgr2=0x556169f71210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f71210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/4079352415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 msgr2=0x556169f4e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f4e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/4079352415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556169f54f80 con 0x556169b862f0 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00c7680 tx=0x7f4fe00b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab077fa3c67e4c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4fdaffd640 1 -- 172.21.15.94:0/4079352415 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0049040 con 0x556169b862f0 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4fdaffd640 1 -- 172.21.15.94:0/4079352415 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe00c7df0 con 0x556169b862f0 2024-04-25T13:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.385+0000 7f4fdaffd640 1 -- 172.21.15.94:0/4079352415 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00c3d40 con 0x556169b862f0 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/4079352415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 msgr2=0x556169f65030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00c7680 tx=0x7f4fe00b0040 comp rx=0 tx=0).stop 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/4079352415 shutdown_connections 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b8e4d0 0x556169f71210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b862f0 0x556169f65030 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/4079352415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f4e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/4079352415 >> 172.21.15.94:0/4079352415 conn(0x556169ba3300 msgr2=0x556169f4e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/4079352415 shutdown_connections 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.386+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/4079352415 wait complete. 2024-04-25T13:57:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 -- start start 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 Processor -- start 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 0x556169f62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 0x556169f63dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01aa6d0 con 0x556169b8e4d0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01a9fa0 con 0x556169b6dd80 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff01a71e0 con 0x556169b862f0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.387+0000 7f4ffe5b6740 10 client.0 _ll_register_callbacks cb 0x7f4fec012f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 0x556169f62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff5ca5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 0x556169f63dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff4ca3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 0x556169f62e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34402/0 (socket says 172.21.15.94:34402) 2024-04-25T13:57:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58558/0 (socket says 172.21.15.94:58558) 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff4ca3640 1 -- 172.21.15.94:0/3947760687 learned_addr learned my addr 172.21.15.94:0/3947760687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3947760687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 msgr2=0x556169f62e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 0x556169f62e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3947760687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 msgr2=0x556169f63dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 0x556169f63dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 -- 172.21.15.94:0/3947760687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556169f64380 con 0x556169b6dd80 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff5ca5640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 0x556169f63dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ff54a4640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4fec031e50 tx=0x7f4fec009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a663907c8bba315a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec01dbf0 con 0x556169b6dd80 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe00c7df0 con 0x556169b6dd80 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f4fec01dd90 con 0x556169b6dd80 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4fec004040 con 0x556169b6dd80 2024-04-25T13:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ffe5b6740 10 client.30548 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.388+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556169fd3dc0 con 0x556169b6dd80 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe00c7df0 con 0x556169b6dd80 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec026040 con 0x556169b6dd80 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 10 client.30548 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 10 client.30548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 20 client.30548 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169baf1b0 0x556169fd6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 4 client.30548 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556169ebcee0 con 0x556169baf1b0 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.389+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169baf1b0 0x556169fd6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.390+0000 7f4fd8ff9640 10 client.30548 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.390+0000 7f4ff4ca3640 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169baf1b0 0x556169fd6780 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dede1e3395accc21 server_cookie=18cbacbfe4468a96 in_seq=0 out_seq=0 2024-04-25T13:57:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:36 smithi137 ceph-mon[32568]: pgmap v824: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-04-25T13:57:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:36 smithi094 ceph-mon[29195]: pgmap v824: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-04-25T13:57:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:36 smithi138 ceph-mon[32977]: pgmap v824: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-04-25T13:57:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4fd8ff9640 1 -- 172.21.15.94:0/3947760687 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556169ebcee0 con 0x556169baf1b0 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4fd8ff9640 10 client.30548 handle_command_reply: tid=0 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 client.30548 shutdown 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 msgr2=0x556169f62950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4fec031e50 tx=0x7f4fec009d60 comp rx=0 tx=0).stop 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 shutdown_connections 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556169baf1b0 0x556169fd6780 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556169b8e4d0 0x556169f63dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4fd8ff9640 0 client.30548 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556169b862f0 0x556169f62e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 --2- 172.21.15.94:0/3947760687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556169b6dd80 0x556169f62950 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 >> 172.21.15.94:0/3947760687 conn(0x556169ba3300 msgr2=0x556169f4eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 shutdown_connections 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 1 -- 172.21.15.94:0/3947760687 wait complete. 2024-04-25T13:57:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.906+0000 7f4ffe5b6740 20 client.30548 trim_cache size 0 max 16384 2024-04-25T13:57:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:57:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:57:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:57:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 msgr2=0x7f4fdc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 0x7f4fdc0b1ff0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x556169f55160 tx=0x7f4fec005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 msgr2=0x7f4ff019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006010 tx=0x7f4fe00042f0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 shutdown_connections 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fdc0afae0 0x7f4fdc0b1ff0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ff0108960 0x7f4ff01a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff0103c60 0x7f4ff019efb0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 --2- 172.21.15.94:0/2690147630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff0102a20 0x7f4ff006ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 >> 172.21.15.94:0/2690147630 conn(0x7f4ff00fdc10 msgr2=0x7f4ff0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 shutdown_connections 2024-04-25T13:57:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:36.907+0000 7f4ff7708640 1 -- 172.21.15.94:0/2690147630 wait complete. 2024-04-25T13:57:37.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 51207 18.91 MB/sec execute 19 sec latency 357.163 ms 2024-04-25T13:57:37.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:37 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:37.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:37 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:37 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:38.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 51583 18.88 MB/sec execute 20 sec latency 204.577 ms 2024-04-25T13:57:38.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:57:38.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:57:38.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 -- 172.21.15.94:0/2371008216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658102f80 msgr2=0x7ff658103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/2371008216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658102f80 0x7ff658103390 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff654009900 tx=0x7ff65402a420 comp rx=0 tx=0).stop 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 -- 172.21.15.94:0/2371008216 shutdown_connections 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/2371008216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658104460 0x7ff65810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/2371008216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff658103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/2371008216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658102f80 0x7ff658103390 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.415+0000 7ff65f7c5640 1 -- 172.21.15.94:0/2371008216 >> 172.21.15.94:0/2371008216 conn(0x7ff6580fd340 msgr2=0x7ff6580ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.416+0000 7ff65f7c5640 1 -- 172.21.15.94:0/2371008216 shutdown_connections 2024-04-25T13:57:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.416+0000 7ff65f7c5640 1 -- 172.21.15.94:0/2371008216 wait complete. 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.416+0000 7ff65f7c5640 1 Processor -- start 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.416+0000 7ff65f7c5640 1 -- start start 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff65819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 0x7ff6581a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810e4e0 con 0x7ff658104460 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810e360 con 0x7ff658103960 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65f7c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff65810e660 con 0x7ff658102f80 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65dd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 0x7ff6581a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65d561640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65d561640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34412/0 (socket says 172.21.15.94:34412) 2024-04-25T13:57:38.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65d561640 1 -- 172.21.15.94:0/1430422521 learned_addr learned my addr 172.21.15.94:0/1430422521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65cd60640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff65819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.417+0000 7ff65dd62640 1 -- 172.21.15.94:0/1430422521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 msgr2=0x7ff658102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65dd62640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65dd62640 1 -- 172.21.15.94:0/1430422521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 msgr2=0x7ff65819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65dd62640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff65819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65dd62640 1 -- 172.21.15.94:0/1430422521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6581a8340 con 0x7ff658104460 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65d561640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65cd60640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff65819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff65dd62640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 0x7ff6581a7b70 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fd0 tx=0x7ff65000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff64e7fc640 1 -- 172.21.15.94:0/1430422521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65000ce20 con 0x7ff658104460 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff64e7fc640 1 -- 172.21.15.94:0/1430422521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff650005020 con 0x7ff658104460 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.418+0000 7ff64e7fc640 1 -- 172.21.15.94:0/1430422521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650010690 con 0x7ff658104460 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.419+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6581a8630 con 0x7ff658104460 2024-04-25T13:57:38.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.419+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6581a8b30 con 0x7ff658104460 2024-04-25T13:57:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.422+0000 7ff66667b740 1 Processor -- start 2024-04-25T13:57:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.422+0000 7ff64e7fc640 1 -- 172.21.15.94:0/1430422521 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff650010830 con 0x7ff658104460 2024-04-25T13:57:38.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.422+0000 7ff66667b740 1 -- start start 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d20321afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d20321bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d20317fd80 con 0x55d202e504d0 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2031398b0 con 0x55d202e79390 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff66667b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d20317bd40 con 0x55d202e586b0 2024-04-25T13:57:38.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58592/0 (socket says 172.21.15.94:58592) 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 -- 172.21.15.94:0/941113639 learned_addr learned my addr 172.21.15.94:0/941113639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d20321bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 -- 172.21.15.94:0/941113639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 msgr2=0x55d20321bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d20321bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 -- 172.21.15.94:0/941113639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 msgr2=0x55d20321afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff64e7fc640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 0x7ff6440b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d20321afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.423+0000 7ff65cd60640 1 -- 172.21.15.94:0/941113639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d20310aa40 con 0x55d202e79390 2024-04-25T13:57:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff64e7fc640 1 -- 172.21.15.94:0/1430422521 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff6500df410 con 0x7ff658104460 2024-04-25T13:57:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff65cd60640 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff648012840 tx=0x7ff648009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64aebb638467c4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff65dd62640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 0x7ff6440b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff6437fe640 1 -- 172.21.15.94:0/941113639 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648012dc0 con 0x55d202e79390 2024-04-25T13:57:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff6437fe640 1 -- 172.21.15.94:0/941113639 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648010c80 con 0x55d202e79390 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.424+0000 7ff6437fe640 1 -- 172.21.15.94:0/941113639 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64801eb90 con 0x55d202e79390 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff66667b740 1 -- 172.21.15.94:0/941113639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 msgr2=0x55d20321aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff65dd62640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 0x7ff6440b2100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff65001a040 tx=0x7ff6500a3040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff66667b740 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff648012840 tx=0x7ff648009740 comp rx=0 tx=0).stop 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff66667b740 1 -- 172.21.15.94:0/941113639 shutdown_connections 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff66667b740 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d20321bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.425+0000 7ff66667b740 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d20321afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 --2- 172.21.15.94:0/941113639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20321aa60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 -- 172.21.15.94:0/941113639 >> 172.21.15.94:0/941113639 conn(0x55d202e6d4e0 msgr2=0x55d203216270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 -- 172.21.15.94:0/941113639 shutdown_connections 2024-04-25T13:57:38.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 -- 172.21.15.94:0/941113639 wait complete. 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 -- start start 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.426+0000 7ff66667b740 1 Processor -- start 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d203224d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44262/0 (socket says 172.21.15.94:44262) 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff65cd60640 1 -- 172.21.15.94:0/2329279652 learned_addr learned my addr 172.21.15.94:0/2329279652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20322bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d20321c780 con 0x55d202e504d0 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d203106d10 con 0x55d202e79390 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d20321c900 con 0x55d202e586b0 2024-04-25T13:57:38.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff66667b740 10 client.0 _ll_register_callbacks cb 0x55d203133210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff65d561640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d203224d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.427+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20322bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 -- 172.21.15.94:0/2329279652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 msgr2=0x55d203224d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d203224d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 -- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 msgr2=0x55d20322bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20322bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d202f5b7d0 con 0x55d202e504d0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65d561640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d203224d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff6480096e0 tx=0x7ff648009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc73907c1c5b4e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648004320 con 0x55d202e504d0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff64801e930 con 0x55d202e504d0 2024-04-25T13:57:38.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff648010c80 con 0x55d202e504d0 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff648033050 con 0x55d202e504d0 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff66667b740 10 client.21126 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d202e913f0 con 0x55d202e504d0 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.428+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648010c80 con 0x55d202e504d0 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff648027020 con 0x55d202e504d0 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 10 client.21126 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 10 client.21126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 20 client.21126 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e37f60 0x55d203231da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 4 client.21126 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:38.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.429+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d20318d990 con 0x55d202e37f60 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.430+0000 7ff65d561640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e37f60 0x55d203231da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.430+0000 7ff640ff9640 10 client.21126 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.430+0000 7ff65d561640 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e37f60 0x55d203231da0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6ab833b11d4381 server_cookie=dd0e9b62dfc006db in_seq=0 out_seq=0 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff640ff9640 1 -- 172.21.15.94:0/2329279652 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d20318d990 con 0x55d202e37f60 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff640ff9640 10 client.21126 handle_command_reply: tid=0 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 client.21126 shutdown 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 msgr2=0x55d2032247e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff6480096e0 tx=0x7ff648009d30 comp rx=0 tx=0).stop 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 shutdown_connections 2024-04-25T13:57:38.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e37f60 0x55d203231da0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff640ff9640 0 client.21126 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e79390 0x55d20322bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e586b0 0x55d203224d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 --2- 172.21.15.94:0/2329279652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e504d0 0x55d2032247e0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 >> 172.21.15.94:0/2329279652 conn(0x55d202e6d4e0 msgr2=0x55d2032153f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 shutdown_connections 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 1 -- 172.21.15.94:0/2329279652 wait complete. 2024-04-25T13:57:38.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.432+0000 7ff66667b740 20 client.21126 trim_cache size 0 max 16384 2024-04-25T13:57:38.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 Processor -- start 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 -- start start 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e37f60 0x55d203222f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d20322bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581b1bb0 con 0x55d202e6d4e0 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581b3750 con 0x55d202e37f60 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff66667b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581b2280 con 0x55d202e504d0 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.460+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44276/0 (socket says 172.21.15.94:44276) 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 -- 172.21.15.94:0/2906233350 learned_addr learned my addr 172.21.15.94:0/2906233350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65d561640 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e37f60 0x55d203222f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d20322bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 -- 172.21.15.94:0/2906233350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 msgr2=0x55d20322bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d20322bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 -- 172.21.15.94:0/2906233350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e37f60 msgr2=0x55d203222f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e37f60 0x55d203222f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 -- 172.21.15.94:0/2906233350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d202e510f0 con 0x55d202e6d4e0 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff65cd60640 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff648012650 tx=0x7ff6480127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aa6615357f3d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff642ffd640 1 -- 172.21.15.94:0/2906233350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648010c80 con 0x55d202e6d4e0 2024-04-25T13:57:38.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff642ffd640 1 -- 172.21.15.94:0/2906233350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648009a50 con 0x55d202e6d4e0 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.461+0000 7ff642ffd640 1 -- 172.21.15.94:0/2906233350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64801ee60 con 0x55d202e6d4e0 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 -- 172.21.15.94:0/2906233350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 msgr2=0x55d20321fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff648012650 tx=0x7ff6480127c0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 -- 172.21.15.94:0/2906233350 shutdown_connections 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d20322bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e37f60 0x55d203222f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 --2- 172.21.15.94:0/2906233350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e6d4e0 0x55d20321fba0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 -- 172.21.15.94:0/2906233350 >> 172.21.15.94:0/2906233350 conn(0x55d2032320f0 msgr2=0x55d203219aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 -- 172.21.15.94:0/2906233350 shutdown_connections 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.462+0000 7ff66667b740 1 -- 172.21.15.94:0/2906233350 wait complete. 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 -- start start 2024-04-25T13:57:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 Processor -- start 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d203229b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e6d4e0 0x55d203298150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581b3080 con 0x55d202e37f60 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581b2950 con 0x55d202e6d4e0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff65cd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44284/0 (socket says 172.21.15.94:44284) 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff65cd60640 1 -- 172.21.15.94:0/573525968 learned_addr learned my addr 172.21.15.94:0/573525968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.463+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581afb90 con 0x55d202e504d0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff66667b740 10 client.0 _ll_register_callbacks cb 0x7ff648009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65d561640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d203229b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e6d4e0 0x55d203298150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 -- 172.21.15.94:0/573525968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 msgr2=0x55d203229b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d203229b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 -- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e6d4e0 msgr2=0x55d203298150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e6d4e0 0x55d203298150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 -- 172.21.15.94:0/573525968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2031404b0 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65d561640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d203229b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff65cd60640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff648004710 tx=0x7ff648009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5610986f629c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64801e750 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff648009a50 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.464+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff64803d070 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.465+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff64801e8f0 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.465+0000 7ff66667b740 10 client.21132 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.465+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d203299790 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.465+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648009a50 con 0x55d202e37f60 2024-04-25T13:57:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.466+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff648040020 con 0x55d202e37f60 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.466+0000 7ff66667b740 10 client.21132 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff66667b740 10 client.21132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff66667b740 20 client.21132 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e586b0 0x55d20329c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff66667b740 4 client.21132 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d203216340 con 0x55d202e586b0 2024-04-25T13:57:38.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff65d561640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e586b0 0x55d20329c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff65d561640 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e586b0 0x55d20329c0e0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a665a91c1efa368 server_cookie=f02bc51452843deb in_seq=0 out_seq=0 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.467+0000 7ff640ff9640 10 client.21132 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff640ff9640 1 -- 172.21.15.94:0/573525968 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d203216340 con 0x55d202e586b0 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff640ff9640 10 client.21132 handle_command_reply: tid=0 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 client.21132 shutdown 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 msgr2=0x55d2032295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff648004710 tx=0x7ff648009f60 comp rx=0 tx=0).stop 2024-04-25T13:57:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 shutdown_connections 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d202e586b0 0x55d20329c0e0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d202e6d4e0 0x55d203298150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d202e504d0 0x55d203229b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 --2- 172.21.15.94:0/573525968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d202e37f60 0x55d2032295d0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 >> 172.21.15.94:0/573525968 conn(0x55d2032320f0 msgr2=0x55d2032153f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff640ff9640 0 client.21132 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.470+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 shutdown_connections 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff66667b740 1 -- 172.21.15.94:0/573525968 wait complete. 2024-04-25T13:57:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff66667b740 20 client.21132 trim_cache size 0 max 16384 2024-04-25T13:57:38.476 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:38.476 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:57:38.476 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:57:38.476 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 msgr2=0x7ff6440b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 0x7ff6440b2100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff65001a040 tx=0x7ff6500a3040 comp rx=0 tx=0).stop 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 msgr2=0x7ff6581a7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 0x7ff6581a7b70 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff650009fd0 tx=0x7ff65000ef10 comp rx=0 tx=0).stop 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 shutdown_connections 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff6440afbf0 0x7ff6440b2100 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff658104460 0x7ff6581a7b70 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.471+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff658103960 0x7ff65819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.472+0000 7ff65f7c5640 1 --2- 172.21.15.94:0/1430422521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff658102f80 0x7ff658102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.472+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 >> 172.21.15.94:0/1430422521 conn(0x7ff6580fd340 msgr2=0x7ff658104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.472+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 shutdown_connections 2024-04-25T13:57:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.472+0000 7ff65f7c5640 1 -- 172.21.15.94:0/1430422521 wait complete. 2024-04-25T13:57:38.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:57:38.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3471678920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 msgr2=0x7fbe28103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3471678920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28103dc0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009900 tx=0x7fbe1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3471678920 shutdown_connections 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3471678920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 0x7fbe2810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3471678920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28103dc0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3471678920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 0x7fbe281033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3471678920 >> 172.21.15.94:0/3471678920 conn(0x7fbe280fd330 msgr2=0x7fbe280ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3471678920 shutdown_connections 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.624+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3471678920 wait complete. 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 Processor -- start 2024-04-25T13:57:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 -- start start 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 0x7fbe281990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 0x7fbe281a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810e4f0 con 0x7fbe28103970 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810e370 con 0x7fbe28102f90 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe2e2af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2810e670 con 0x7fbe28104450 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44302/0 (socket says 172.21.15.94:44302) 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe277fe640 1 -- 172.21.15.94:0/3945015762 learned_addr learned my addr 172.21.15.94:0/3945015762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.625+0000 7fbe27fff640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 0x7fbe281990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe2c84c640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 0x7fbe281a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 -- 172.21.15.94:0/3945015762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 msgr2=0x7fbe281a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 0x7fbe281a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 -- 172.21.15.94:0/3945015762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 msgr2=0x7fbe281990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 0x7fbe281990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 -- 172.21.15.94:0/3945015762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe281a0c10 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe277fe640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006010 tx=0x7fbe1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c03f030 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe281a0ea0 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1c031910 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c035070 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.626+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a1300 con 0x7fbe28103970 2024-04-25T13:57:38.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.627+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe28107930 con 0x7fbe28103970 2024-04-25T13:57:38.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.630+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbe1c042030 con 0x7fbe28103970 2024-04-25T13:57:38.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.630+0000 7fbe257fa640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 0x7fbe100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:38.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.630+0000 7fbe27fff640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 0x7fbe100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:38.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.630+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fbe1c0ff760 con 0x7fbe28103970 2024-04-25T13:57:38.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.631+0000 7fbe27fff640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 0x7fbe100b2080 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe28069b90 tx=0x7fbe18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:38.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.635+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe1c0c7c50 con 0x7fbe28103970 2024-04-25T13:57:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:38 smithi137 ceph-mon[32568]: pgmap v825: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-04-25T13:57:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:38 smithi094 ceph-mon[29195]: pgmap v825: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-04-25T13:57:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:38 smithi138 ceph-mon[32977]: pgmap v825: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 393 op/s 2024-04-25T13:57:38.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.896+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe2802db50 con 0x7fbe28103970 2024-04-25T13:57:38.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.897+0000 7fbe257fa640 1 -- 172.21.15.94:0/3945015762 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11944 (secure 0 0 0) 0x7fbe1c0cbf50 con 0x7fbe28103970 2024-04-25T13:57:38.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:57:38.897 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:57:35.758775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30058":{"gid":30058,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2422946890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2422946890},{"type":"v1","addr":"172.21.15.138:6835","nonce":2422946890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24488,24502,24475,24557,24512]},"id":1}]} 2024-04-25T13:57:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-25T13:57:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 msgr2=0x7fbe100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 0x7fbe100b2080 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe28069b90 tx=0x7fbe18005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 msgr2=0x7fbe28199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006010 tx=0x7fbe1c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 shutdown_connections 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe100afb70 0x7fbe100b2080 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe28104450 0x7fbe281a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe28103970 0x7fbe28199610 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 --2- 172.21.15.94:0/3945015762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe28102f90 0x7fbe281990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 >> 172.21.15.94:0/3945015762 conn(0x7fbe280fd330 msgr2=0x7fbe28104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 shutdown_connections 2024-04-25T13:57:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:38.899+0000 7fbe2e2af640 1 -- 172.21.15.94:0/3945015762 wait complete. 2024-04-25T13:57:38.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 -- 172.21.15.94:0/1092458588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 msgr2=0x7f015010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 --2- 172.21.15.94:0/1092458588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f015010ad50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0148009be0 tx=0x7f014802a6c0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 -- 172.21.15.94:0/1092458588 shutdown_connections 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 --2- 172.21.15.94:0/1092458588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f015010ad50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 --2- 172.21.15.94:0/1092458588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f0150104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 --2- 172.21.15.94:0/1092458588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 0x7f01501036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 -- 172.21.15.94:0/1092458588 >> 172.21.15.94:0/1092458588 conn(0x7f01500fdc00 msgr2=0x7f0150100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 -- 172.21.15.94:0/1092458588 shutdown_connections 2024-04-25T13:57:39.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.035+0000 7f0155e97640 1 -- 172.21.15.94:0/1092458588 wait complete. 2024-04-25T13:57:39.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 Processor -- start 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 -- start start 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 0x7f015019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f01501a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f015010f620 con 0x7f01501029f0 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f015010f4a0 con 0x7f0150103c30 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f0155e97640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f015010f7a0 con 0x7f0150108930 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f014effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f014effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58646/0 (socket says 172.21.15.94:58646) 2024-04-25T13:57:39.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f014effd640 1 -- 172.21.15.94:0/126298926 learned_addr learned my addr 172.21.15.94:0/126298926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f014ffff640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f01501a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.036+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 0x7f015019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 -- 172.21.15.94:0/126298926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 msgr2=0x7f01501a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f01501a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 -- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 msgr2=0x7f015019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 0x7f015019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 -- 172.21.15.94:0/126298926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01501a1d10 con 0x7f0150103c30 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014effd640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f014400e9f0 tx=0x7f014400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014cff9640 1 -- 172.21.15.94:0/126298926 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014400ccc0 con 0x7f0150103c30 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014cff9640 1 -- 172.21.15.94:0/126298926 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f014400ce60 con 0x7f0150103c30 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01501a1fa0 con 0x7f0150103c30 2024-04-25T13:57:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f014cff9640 1 -- 172.21.15.94:0/126298926 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144010430 con 0x7f0150103c30 2024-04-25T13:57:39.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.037+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01501a2440 con 0x7f0150103c30 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014cff9640 1 -- 172.21.15.94:0/126298926 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f01440046c0 con 0x7f0150103c30 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 Processor -- start 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 -- start start 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014cff9640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5483dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014ffff640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5483ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561f547a1d80 con 0x561f5449b390 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014cff9640 1 -- 172.21.15.94:0/126298926 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f01440df0c0 con 0x7f0150103c30 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561f5475ba20 con 0x561f544724d0 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f015ce22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f5479dd40 con 0x561f5447a6b0 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44320/0 (socket says 172.21.15.94:44320) 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.040+0000 7f014f7fe640 1 -- 172.21.15.94:0/563483052 learned_addr learned my addr 172.21.15.94:0/563483052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014effd640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5483dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014ffff640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5483ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 -- 172.21.15.94:0/563483052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 msgr2=0x561f5483ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5483ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 -- 172.21.15.94:0/563483052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 msgr2=0x561f5483dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5483dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 -- 172.21.15.94:0/563483052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f54766c30 con 0x561f5449b390 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014effd640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5483dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014ffff640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5483ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014ffff640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 0x7f01380b2000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0148004af0 tx=0x7f0148005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f014f7fe640 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c60 tx=0x7f0140011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ef0c8e0888b376 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f0135ffb640 1 -- 172.21.15.94:0/563483052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140004220 con 0x561f5449b390 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f0135ffb640 1 -- 172.21.15.94:0/563483052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01400043c0 con 0x561f5449b390 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.041+0000 7f0135ffb640 1 -- 172.21.15.94:0/563483052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140004d10 con 0x561f5449b390 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 -- 172.21.15.94:0/563483052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 msgr2=0x561f5483d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c60 tx=0x7f0140011810 comp rx=0 tx=0).stop 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 -- 172.21.15.94:0/563483052 shutdown_connections 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5483ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5483dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 --2- 172.21.15.94:0/563483052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f5483d810 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 -- 172.21.15.94:0/563483052 >> 172.21.15.94:0/563483052 conn(0x561f5448f4e0 msgr2=0x561f54839020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 -- 172.21.15.94:0/563483052 shutdown_connections 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.042+0000 7f015ce22740 1 -- 172.21.15.94:0/563483052 wait complete. 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 -- start start 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 Processor -- start 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f544724d0 0x561f54847530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f5449b390 0x561f5484e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561f5483f530 con 0x561f5447a6b0 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561f54728dc0 con 0x561f5449b390 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f5483f6b0 con 0x561f544724d0 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f015ce22740 10 client.0 _ll_register_callbacks cb 0x561f54744600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f014effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f014effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44334/0 (socket says 172.21.15.94:44334) 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f014ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f5449b390 0x561f5484e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f014effd640 1 -- 172.21.15.94:0/1602542751 learned_addr learned my addr 172.21.15.94:0/1602542751 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.043+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f544724d0 0x561f54847530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 -- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f544724d0 msgr2=0x561f54847530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f544724d0 0x561f54847530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 -- 172.21.15.94:0/1602542751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f5449b390 msgr2=0x561f5484e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f5449b390 0x561f5484e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 -- 172.21.15.94:0/1602542751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f544b33f0 con 0x561f5447a6b0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f014effd640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f014400c6a0 tx=0x7f01440a6f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe8f5410ed9e96b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f012effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01440a7010 con 0x561f5447a6b0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f01440a7760 con 0x561f5447a6b0 2024-04-25T13:57:39.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01400043c0 con 0x561f5447a6b0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f01440a7c00 con 0x561f5447a6b0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f015ce22740 10 client.21150 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561f5457d7d0 con 0x561f5447a6b0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.044+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01400043c0 con 0x561f5447a6b0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f01440afed0 con 0x561f5447a6b0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 10 client.21150 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 10 client.21150 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 20 client.21150 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f54459f60 0x561f548551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 4 client.21150 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f547af990 con 0x561f54459f60 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f54459f60 0x561f548551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f012effd640 10 client.21150 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:39.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.045+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f54459f60 0x561f548551f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ea4a7085d136c1b server_cookie=90a446cedfbac4bf in_seq=0 out_seq=0 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f012effd640 1 -- 172.21.15.94:0/1602542751 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f547af990 con 0x561f54459f60 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f012effd640 10 client.21150 handle_command_reply: tid=0 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f015ce22740 1 client.21150 shutdown 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 msgr2=0x561f54847a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f014400c6a0 tx=0x7f01440a6f10 comp rx=0 tx=0).stop 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 shutdown_connections 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.046+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f54459f60 0x561f548551f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f5449b390 0x561f5484e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f012effd640 0 client.21150 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5447a6b0 0x561f54847a70 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 --2- 172.21.15.94:0/1602542751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f544724d0 0x561f54847530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 >> 172.21.15.94:0/1602542751 conn(0x561f5448f4e0 msgr2=0x561f54838040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 shutdown_connections 2024-04-25T13:57:39.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 1 -- 172.21.15.94:0/1602542751 wait complete. 2024-04-25T13:57:39.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.047+0000 7f015ce22740 20 client.21150 trim_cache size 0 max 16384 2024-04-25T13:57:39.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.063+0000 7f015ce22740 1 Processor -- start 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.063+0000 7f015ce22740 1 -- start start 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01501ab550 con 0x561f5449b390 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01501ad0f0 con 0x561f544724d0 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f015ce22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01501abc20 con 0x561f5447a6b0 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58664/0 (socket says 172.21.15.94:58664) 2024-04-25T13:57:39.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44340/0 (socket says 172.21.15.94:44340) 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 -- 172.21.15.94:0/2506194973 learned_addr learned my addr 172.21.15.94:0/2506194973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014effd640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 -- 172.21.15.94:0/2506194973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 msgr2=0x561f5484e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 -- 172.21.15.94:0/2506194973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 msgr2=0x561f54859970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 -- 172.21.15.94:0/2506194973 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f54845650 con 0x561f544724d0 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014effd640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014ffff640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.064+0000 7f014f7fe640 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c60 tx=0x7f0140009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0deab49f207d88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.065+0000 7f01357fa640 1 -- 172.21.15.94:0/2506194973 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014001d040 con 0x561f544724d0 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.065+0000 7f01357fa640 1 -- 172.21.15.94:0/2506194973 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01400054e0 con 0x561f544724d0 2024-04-25T13:57:39.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.065+0000 7f01357fa640 1 -- 172.21.15.94:0/2506194973 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140003870 con 0x561f544724d0 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.065+0000 7f015ce22740 1 -- 172.21.15.94:0/2506194973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 msgr2=0x561f54847810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.065+0000 7f015ce22740 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0140009c60 tx=0x7f0140009600 comp rx=0 tx=0).stop 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 -- 172.21.15.94:0/2506194973 shutdown_connections 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f54859970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 --2- 172.21.15.94:0/2506194973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f54847810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 -- 172.21.15.94:0/2506194973 >> 172.21.15.94:0/2506194973 conn(0x561f5448f4e0 msgr2=0x561f54847100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 -- 172.21.15.94:0/2506194973 shutdown_connections 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 -- 172.21.15.94:0/2506194973 wait complete. 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 -- start start 2024-04-25T13:57:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.066+0000 7f015ce22740 1 Processor -- start 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f548bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01501aca20 con 0x561f5449b390 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01501ac2f0 con 0x561f544724d0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01501a9530 con 0x561f5447a6b0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f015ce22740 10 client.0 _ll_register_callbacks cb 0x7f01440a2ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f548bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58672/0 (socket says 172.21.15.94:58672) 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 -- 172.21.15.94:0/1076285269 learned_addr learned my addr 172.21.15.94:0/1076285269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014effd640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 -- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 msgr2=0x561f5484bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 -- 172.21.15.94:0/1076285269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 msgr2=0x561f548bb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f548bb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 -- 172.21.15.94:0/1076285269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5484c1f0 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014ffff640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f548bb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f014f7fe640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0140006010 tx=0x7f0140009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4eddbf76453b0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.067+0000 7f012effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014003b020 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f014002c360 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01400054e0 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0140046050 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f015ce22740 10 client.30590 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561f548bca70 con 0x561f544724d0 2024-04-25T13:57:39.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.068+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01400054e0 con 0x561f544724d0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f014002cce0 con 0x561f544724d0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 10 client.30590 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 10 client.30590 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 20 client.30590 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f5442f3a0 0x561f548bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 4 client.30590 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f547a9800 con 0x561f5442f3a0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f014effd640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f5442f3a0 0x561f548bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f012effd640 10 client.30590 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f014effd640 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f5442f3a0 0x561f548bf230 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45e710c0f25ee917 server_cookie=7899b9b9067b1e6d in_seq=0 out_seq=0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f012effd640 1 -- 172.21.15.94:0/1076285269 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f547a9800 con 0x561f5442f3a0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.069+0000 7f012effd640 10 client.30590 handle_command_reply: tid=0 2024-04-25T13:57:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 client.30590 shutdown 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 msgr2=0x561f5484b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0140006010 tx=0x7f0140009b60 comp rx=0 tx=0).stop 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 shutdown_connections 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x561f5442f3a0 0x561f548bf230 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561f5449b390 0x561f548bb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f012effd640 0 client.30590 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f5447a6b0 0x561f5484bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 --2- 172.21.15.94:0/1076285269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f544724d0 0x561f5484b6c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 >> 172.21.15.94:0/1076285269 conn(0x561f5448f4e0 msgr2=0x561f5483be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 shutdown_connections 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 1 -- 172.21.15.94:0/1076285269 wait complete. 2024-04-25T13:57:39.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f015ce22740 20 client.30590 trim_cache size 0 max 16384 2024-04-25T13:57:39.076 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:39.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 msgr2=0x7f01380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 0x7f01380b2000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0148004af0 tx=0x7f0148005fd0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 msgr2=0x7f015019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f014400e9f0 tx=0x7f014400edf0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 shutdown_connections 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0150108930 0x7f01501a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0150103c30 0x7f015019a710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 --2- 172.21.15.94:0/126298926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01501029f0 0x7f015019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.070+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 >> 172.21.15.94:0/126298926 conn(0x7f01500fdc00 msgr2=0x7f0150104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.071+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 shutdown_connections 2024-04-25T13:57:39.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.071+0000 7f0155e97640 1 -- 172.21.15.94:0/126298926 wait complete. 2024-04-25T13:57:39.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:57:39.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 51940 18.87 MB/sec execute 21 sec latency 267.311 ms 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.209+0000 7fc641100640 1 -- 172.21.15.94:0/4012828146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c1098a0 msgr2=0x7fc63c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.209+0000 7fc641100640 1 --2- 172.21.15.94:0/4012828146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c1098a0 0x7fc63c10bcc0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fc634009ca0 tx=0x7fc63402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 -- 172.21.15.94:0/4012828146 shutdown_connections 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 --2- 172.21.15.94:0/4012828146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c1098a0 0x7fc63c10bcc0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 --2- 172.21.15.94:0/4012828146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c106f40 0x7fc63c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 --2- 172.21.15.94:0/4012828146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 -- 172.21.15.94:0/4012828146 >> 172.21.15.94:0/4012828146 conn(0x7fc63c0fdbc0 msgr2=0x7fc63c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 -- 172.21.15.94:0/4012828146 shutdown_connections 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.210+0000 7fc641100640 1 -- 172.21.15.94:0/4012828146 wait complete. 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 Processor -- start 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 -- start start 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c1098a0 0x7fc63c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c10f620 con 0x7fc63c106f40 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c10f4a0 con 0x7fc63c1098a0 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc641100640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c10f7a0 con 0x7fc63c1045e0 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc633fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc633fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44358/0 (socket says 172.21.15.94:44358) 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc633fff640 1 -- 172.21.15.94:0/3218482080 learned_addr learned my addr 172.21.15.94:0/3218482080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63b59e640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c1098a0 0x7fc63c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 -- 172.21.15.94:0/3218482080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c1098a0 msgr2=0x7fc63c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c1098a0 0x7fc63c1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 -- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 msgr2=0x7fc63c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc63ad9d640 1 -- 172.21.15.94:0/3218482080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c1a1d30 con 0x7fc63c1045e0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.211+0000 7fc633fff640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c19a1f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c940 tx=0x7fc62c00ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc638d99640 1 -- 172.21.15.94:0/3218482080 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c007bb0 con 0x7fc63c1045e0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc638d99640 1 -- 172.21.15.94:0/3218482080 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc62c007d50 con 0x7fc63c1045e0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c1a1fc0 con 0x7fc63c1045e0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc638d99640 1 -- 172.21.15.94:0/3218482080 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c004db0 con 0x7fc63c1045e0 2024-04-25T13:57:39.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.212+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1a2480 con 0x7fc63c1045e0 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 Processor -- start 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 -- start start 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc638d99640 1 -- 172.21.15.94:0/3218482080 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc62c00f490 con 0x7fc63c1045e0 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0250810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0251a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e3c01b4d80 con 0x55e3bfe854d0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3c016ea20 con 0x55e3bfeae390 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc647fb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e3c01b0d40 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc633fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0251a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc63b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0250810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44372/0 (socket says 172.21.15.94:44372) 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.215+0000 7fc63ad9d640 1 -- 172.21.15.94:0/3577696280 learned_addr learned my addr 172.21.15.94:0/3577696280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 -- 172.21.15.94:0/3577696280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 msgr2=0x55e3c0250810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0250810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 -- 172.21.15.94:0/3577696280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 msgr2=0x55e3c0250d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 -- 172.21.15.94:0/3577696280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c0179c30 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc638d99640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 0x7fc6280b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc63b59e640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 0x7fc6280b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc633fff640 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0251a20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc63c19b290 tx=0x7fc6240119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df469fd3ae5c958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc638d99640 1 -- 172.21.15.94:0/3218482080 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fc62c0de7e0 con 0x7fc63c1045e0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc6317fa640 1 -- 172.21.15.94:0/3577696280 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004030 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc6317fa640 1 -- 172.21.15.94:0/3577696280 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc624005020 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.216+0000 7fc6317fa640 1 -- 172.21.15.94:0/3577696280 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004d30 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- 172.21.15.94:0/3577696280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 msgr2=0x55e3c0251a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc63b59e640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 0x7fc6280b2080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55e3c0251860 tx=0x7fc634009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0251a20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc63c19b290 tx=0x7fc6240119a0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- 172.21.15.94:0/3577696280 shutdown_connections 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0251a20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe854d0 0x55e3c0250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 --2- 172.21.15.94:0/3577696280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0250810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- 172.21.15.94:0/3577696280 >> 172.21.15.94:0/3577696280 conn(0x55e3bfea24e0 msgr2=0x55e3c024c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- 172.21.15.94:0/3577696280 shutdown_connections 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- 172.21.15.94:0/3577696280 wait complete. 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.217+0000 7fc647fb1740 1 -- start start 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 Processor -- start 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0261990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e3c0252530 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3c013bdc0 con 0x55e3bfeae390 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e3c02526b0 con 0x55e3bfe854d0 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc647fb1740 10 client.0 _ll_register_callbacks cb 0x55e3c0157600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44382/0 (socket says 172.21.15.94:44382) 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 -- 172.21.15.94:0/1737998442 learned_addr learned my addr 172.21.15.94:0/1737998442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc63b59e640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0261990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 -- 172.21.15.94:0/1737998442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 msgr2=0x55e3c025a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 -- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 msgr2=0x55e3c0261990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0261990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc633fff640 1 -- 172.21.15.94:0/1737998442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3bfec63f0 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.218+0000 7fc63b59e640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0261990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc633fff640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc62401b040 tx=0x7fc624005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d3976661a441a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624027020 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fc624016040 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624005020 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc624032050 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 10 client.21168 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3bff907d0 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc624005020 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc624003ac0 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 10 client.21168 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 10 client.21168 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:57:39.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 20 client.21168 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfe6cf60 0x55e3c02680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 4 client.21168 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.219+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c01c2990 con 0x55e3bfe6cf60 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.220+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfe6cf60 0x55e3c02680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.220+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfe6cf60 0x55e3c02680a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9dfc2623b2a6a6b server_cookie=807dc76d816a7a4 in_seq=0 out_seq=0 2024-04-25T13:57:39.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.220+0000 7fc6227fc640 10 client.21168 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc6227fc640 1 -- 172.21.15.94:0/1737998442 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3c01c2990 con 0x55e3bfe6cf60 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc6227fc640 10 client.21168 handle_command_reply: tid=0 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 client.21168 shutdown 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 msgr2=0x55e3c025aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc62401b040 tx=0x7fc624005480 comp rx=0 tx=0).stop 2024-04-25T13:57:39.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 shutdown_connections 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfe6cf60 0x55e3c02680a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfeae390 0x55e3c0261990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025aaa0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc6227fc640 0 client.21168 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 --2- 172.21.15.94:0/1737998442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.221+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 >> 172.21.15.94:0/1737998442 conn(0x55e3bfea24e0 msgr2=0x55e3c024b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.222+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 shutdown_connections 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.222+0000 7fc647fb1740 1 -- 172.21.15.94:0/1737998442 wait complete. 2024-04-25T13:57:39.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.222+0000 7fc647fb1740 20 client.21168 trim_cache size 0 max 16384 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 Processor -- start 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 -- start start 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0258cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1ab4f0 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1ad090 con 0x55e3bfe6cf60 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc647fb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1abbc0 con 0x55e3bfe854d0 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc63b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0258cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.237+0000 7fc633fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58728/0 (socket says 172.21.15.94:58728) 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc633fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34548/0 (socket says 172.21.15.94:34548) 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0258cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44394/0 (socket says 172.21.15.94:44394) 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 -- 172.21.15.94:0/447200234 learned_addr learned my addr 172.21.15.94:0/447200234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 -- 172.21.15.94:0/447200234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 msgr2=0x55e3c0255830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 -- 172.21.15.94:0/447200234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 msgr2=0x55e3c0258cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0258cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 -- 172.21.15.94:0/447200234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c0252410 con 0x55e3bfe6cf60 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc633fff640 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0dc290 tx=0x7fc62c0c9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e782c554d6cb0788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc630ff9640 1 -- 172.21.15.94:0/447200234 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c01d070 con 0x55e3bfe6cf60 2024-04-25T13:57:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc630ff9640 1 -- 172.21.15.94:0/447200234 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc62c0046c0 con 0x55e3bfe6cf60 2024-04-25T13:57:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.238+0000 7fc630ff9640 1 -- 172.21.15.94:0/447200234 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0e4040 con 0x55e3bfe6cf60 2024-04-25T13:57:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 -- 172.21.15.94:0/447200234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 msgr2=0x55e3c02552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0dc290 tx=0x7fc62c0c9040 comp rx=0 tx=0).stop 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 -- 172.21.15.94:0/447200234 shutdown_connections 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c0258cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c0255830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 --2- 172.21.15.94:0/447200234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c02552f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 -- 172.21.15.94:0/447200234 >> 172.21.15.94:0/447200234 conn(0x55e3bfea24e0 msgr2=0x55e3c0250790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.239+0000 7fc647fb1740 1 -- 172.21.15.94:0/447200234 shutdown_connections 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 -- 172.21.15.94:0/447200234 wait complete. 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 -- start start 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 Processor -- start 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c0260ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1ac9c0 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1ac290 con 0x55e3bfe6cf60 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1a94d0 con 0x55e3bfe854d0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc647fb1740 10 client.0 _ll_register_callbacks cb 0x7fc62c0e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc63b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc633fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c0260ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc63ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c0260ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58732/0 (socket says 172.21.15.94:58732) 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc633fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34562/0 (socket says 172.21.15.94:34562) 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc63ad9d640 1 -- 172.21.15.94:0/2976527461 learned_addr learned my addr 172.21.15.94:0/2976527461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.240+0000 7fc63b59e640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44398/0 (socket says 172.21.15.94:44398) 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 -- 172.21.15.94:0/2976527461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 msgr2=0x55e3c025d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 -- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 msgr2=0x55e3c0260ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c0260ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 -- 172.21.15.94:0/2976527461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c025e320 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc63b59e640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fc634006010 tx=0x7fc6340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d331fc6fa40f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc6227fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63402f660 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3c01bc800 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fc63402f800 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc63402fa00 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc647fb1740 10 client.21174 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3c02cf490 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.241+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3c01bc800 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc63405b070 con 0x55e3bfe8d6b0 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 10 client.21174 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 10 client.21174 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 20 client.21174 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfeae390 0x55e3c02d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 4 client.21174 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3c0137610 con 0x55e3bfeae390 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfeae390 0x55e3c02d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc63ad9d640 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfeae390 0x55e3c02d1e80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af5c17e7d0dd6221 server_cookie=55ec404abb32556a in_seq=0 out_seq=0 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.242+0000 7fc6227fc640 10 client.21174 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc6227fc640 1 -- 172.21.15.94:0/2976527461 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3c0137610 con 0x55e3bfeae390 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc6227fc640 10 client.21174 handle_command_reply: tid=0 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 client.21174 shutdown 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 msgr2=0x55e3c025dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fc634006010 tx=0x7fc6340095c0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 shutdown_connections 2024-04-25T13:57:39.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e3bfeae390 0x55e3c02d1e80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e3bfe8d6b0 0x55e3c025dd60 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e3bfe854d0 0x55e3c025d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc6227fc640 0 client.21174 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 --2- 172.21.15.94:0/2976527461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3bfe6cf60 0x55e3c0260ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 >> 172.21.15.94:0/2976527461 conn(0x55e3bfea24e0 msgr2=0x55e3c0250790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 shutdown_connections 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 1 -- 172.21.15.94:0/2976527461 wait complete. 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.243+0000 7fc647fb1740 20 client.21174 trim_cache size 0 max 16384 2024-04-25T13:57:39.249 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 msgr2=0x7fc6280b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 0x7fc6280b2080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55e3c0251860 tx=0x7fc634009450 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 msgr2=0x7fc63c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c19a1f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00c940 tx=0x7fc62c00ce00 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 shutdown_connections 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc6280afb70 0x7fc6280b2080 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc63c1098a0 0x7fc63c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c106f40 0x7fc63c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 --2- 172.21.15.94:0/3218482080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc63c1045e0 0x7fc63c19a1f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 >> 172.21.15.94:0/3218482080 conn(0x7fc63c0fdbc0 msgr2=0x7fc63c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 shutdown_connections 2024-04-25T13:57:39.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.244+0000 7fc641100640 1 -- 172.21.15.94:0/3218482080 wait complete. 2024-04-25T13:57:39.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 -- 172.21.15.94:0/2755582143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c103ee0 msgr2=0x7fb60c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 --2- 172.21.15.94:0/2755582143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c103ee0 0x7fb60c1027c0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009900 tx=0x7fb5fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 -- 172.21.15.94:0/2755582143 shutdown_connections 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 --2- 172.21.15.94:0/2755582143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c108a10 0x7fb60c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 --2- 172.21.15.94:0/2755582143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c1043c0 0x7fb60c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 --2- 172.21.15.94:0/2755582143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c103ee0 0x7fb60c1027c0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 -- 172.21.15.94:0/2755582143 >> 172.21.15.94:0/2755582143 conn(0x7fb60c0fd340 msgr2=0x7fb60c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 -- 172.21.15.94:0/2755582143 shutdown_connections 2024-04-25T13:57:39.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.392+0000 7fb6121f3640 1 -- 172.21.15.94:0/2755582143 wait complete. 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 Processor -- start 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 -- start start 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c103ee0 0x7fb60c102140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c19f0e0 0x7fb60c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10e4f0 con 0x7fb60c108a10 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10e370 con 0x7fb60c103ee0 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb6121f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10e670 con 0x7fb60c19f0e0 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb60affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb60affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb60affd640 1 -- 172.21.15.94:0/3617944259 learned_addr learned my addr 172.21.15.94:0/3617944259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c103ee0 0x7fb60c102140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.393+0000 7fb60bfff640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c19f0e0 0x7fb60c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 -- 172.21.15.94:0/3617944259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c19f0e0 msgr2=0x7fb60c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c19f0e0 0x7fb60c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 -- 172.21.15.94:0/3617944259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c103ee0 msgr2=0x7fb60c102140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c103ee0 0x7fb60c102140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 -- 172.21.15.94:0/3617944259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c1989f0 con 0x7fb60c108a10 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb60affd640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb60000d950 tx=0x7fb60000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb608ff9640 1 -- 172.21.15.94:0/3617944259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600017070 con 0x7fb60c108a10 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb608ff9640 1 -- 172.21.15.94:0/3617944259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb600009c30 con 0x7fb60c108a10 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c198ce0 con 0x7fb60c108a10 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb608ff9640 1 -- 172.21.15.94:0/3617944259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60000f620 con 0x7fb60c108a10 2024-04-25T13:57:39.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.394+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c1991c0 con 0x7fb60c108a10 2024-04-25T13:57:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.397+0000 7fb61917c740 1 Processor -- start 2024-04-25T13:57:39.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.397+0000 7fb61917c740 1 -- start start 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.397+0000 7fb608ff9640 1 -- 172.21.15.94:0/3617944259 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb60000f7c0 con 0x7fb60c108a10 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1b9390 0x5614aa55b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa55ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614aa4bfd80 con 0x5614aa1904d0 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614aa479a20 con 0x5614aa1986b0 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb61917c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614aa4bbd40 con 0x5614aa1b9390 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1b9390 0x5614aa55b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44422/0 (socket says 172.21.15.94:44422) 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 -- 172.21.15.94:0/965916201 learned_addr learned my addr 172.21.15.94:0/965916201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60affd640 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa55ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 -- 172.21.15.94:0/965916201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1b9390 msgr2=0x5614aa55b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1b9390 0x5614aa55b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 -- 172.21.15.94:0/965916201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 msgr2=0x5614aa55ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa55ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 -- 172.21.15.94:0/965916201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614aa484c30 con 0x5614aa1904d0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb608ff9640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 0x7fb5f40b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.398+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fb60c069cb0 tx=0x7fb5fc005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600eccfc572ddf9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb5f1ffb640 1 -- 172.21.15.94:0/965916201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc03f070 con 0x5614aa1904d0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb60bfff640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 0x7fb5f40b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb5f1ffb640 1 -- 172.21.15.94:0/965916201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc004380 con 0x5614aa1904d0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb5f1ffb640 1 -- 172.21.15.94:0/965916201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc035070 con 0x5614aa1904d0 2024-04-25T13:57:39.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb608ff9640 1 -- 172.21.15.94:0/3617944259 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb6000de900 con 0x7fb60c108a10 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb61917c740 1 -- 172.21.15.94:0/965916201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 msgr2=0x5614aa55bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb61917c740 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fb60c069cb0 tx=0x7fb5fc005df0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.399+0000 7fb60bfff640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 0x7fb5f40b2100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5614aa55c860 tx=0x7fb604009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 -- 172.21.15.94:0/965916201 shutdown_connections 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa55ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa55bd90 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 --2- 172.21.15.94:0/965916201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1b9390 0x5614aa55b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 -- 172.21.15.94:0/965916201 >> 172.21.15.94:0/965916201 conn(0x5614aa1ad4e0 msgr2=0x5614aa557020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 -- 172.21.15.94:0/965916201 shutdown_connections 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 -- 172.21.15.94:0/965916201 wait complete. 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 -- start start 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.400+0000 7fb61917c740 1 Processor -- start 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1986b0 0x5614aa565a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1b9390 0x5614aa56c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5614aa55d4d0 con 0x5614aa1904d0 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614aa446dc0 con 0x5614aa1b9390 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5614aa55d650 con 0x5614aa1986b0 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44432/0 (socket says 172.21.15.94:44432) 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 -- 172.21.15.94:0/1485055747 learned_addr learned my addr 172.21.15.94:0/1485055747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 10 client.0 _ll_register_callbacks cb 0x5614aa462600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60affd640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1986b0 0x5614aa565a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 -- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1986b0 msgr2=0x5614aa565a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1986b0 0x5614aa565a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 -- 172.21.15.94:0/1485055747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1b9390 msgr2=0x5614aa56c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1b9390 0x5614aa56c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 -- 172.21.15.94:0/1485055747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614aa1d13f0 con 0x5614aa1904d0 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0037d0 tx=0x7fb5fc02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f20418800ddf814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb5eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc035070 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5fc004380 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.401+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb5fc004180 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc03f040 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 10 client.21186 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5614aa29b7d0 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc004380 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb5fc0449c0 con 0x5614aa1904d0 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 10 client.21186 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 10 client.21186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 20 client.21186 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa177f60 0x5614aa572e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 4 client.21186 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614aa4cd990 con 0x5614aa177f60 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.402+0000 7fb60affd640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa177f60 0x5614aa572e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.403+0000 7fb5eaffd640 10 client.21186 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.403+0000 7fb60affd640 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa177f60 0x5614aa572e10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d01a30670e5a18b9 server_cookie=b5b51b02b32f1f72 in_seq=0 out_seq=0 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb5eaffd640 1 -- 172.21.15.94:0/1485055747 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614aa4cd990 con 0x5614aa177f60 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb5eaffd640 10 client.21186 handle_command_reply: tid=0 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 client.21186 shutdown 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 msgr2=0x5614aa565520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0037d0 tx=0x7fb5fc02ab10 comp rx=0 tx=0).stop 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 shutdown_connections 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa177f60 0x5614aa572e10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb5eaffd640 0 client.21186 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1b9390 0x5614aa56c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1986b0 0x5614aa565a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 --2- 172.21.15.94:0/1485055747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1904d0 0x5614aa565520 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.404+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 >> 172.21.15.94:0/1485055747 conn(0x5614aa1ad4e0 msgr2=0x5614aa559f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.405+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 shutdown_connections 2024-04-25T13:57:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.405+0000 7fb61917c740 1 -- 172.21.15.94:0/1485055747 wait complete. 2024-04-25T13:57:39.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.405+0000 7fb61917c740 20 client.21186 trim_cache size 0 max 16384 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.421+0000 7fb61917c740 1 Processor -- start 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.421+0000 7fb61917c740 1 -- start start 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1904d0 0x5614aa56c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 0x5614aa556720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 0x5614aa556720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34612/0 (socket says 172.21.15.94:34612) 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 -- 172.21.15.94:0/3878959395 learned_addr learned my addr 172.21.15.94:0/3878959395 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1ad730 con 0x5614aa1986b0 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1af2d0 con 0x5614aa1904d0 2024-04-25T13:57:39.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1ade00 con 0x5614aa177f60 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60affd640 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1904d0 0x5614aa56c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 -- 172.21.15.94:0/3878959395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1904d0 msgr2=0x5614aa56c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1904d0 0x5614aa56c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 -- 172.21.15.94:0/3878959395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 msgr2=0x5614aa556720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 0x5614aa556720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60b7fe640 1 -- 172.21.15.94:0/3878959395 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614aa55b3d0 con 0x5614aa177f60 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.422+0000 7fb60bfff640 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 0x5614aa556720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.423+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc044f00 tx=0x7fb5fc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4853c890c7db3c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.423+0000 7fb5f17fa640 1 -- 172.21.15.94:0/3878959395 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc05b070 con 0x5614aa177f60 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.423+0000 7fb5f17fa640 1 -- 172.21.15.94:0/3878959395 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc0035f0 con 0x5614aa177f60 2024-04-25T13:57:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.423+0000 7fb5f17fa640 1 -- 172.21.15.94:0/3878959395 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc035040 con 0x5614aa177f60 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 msgr2=0x5614aa565800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc044f00 tx=0x7fb5fc005740 comp rx=0 tx=0).stop 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 shutdown_connections 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa1986b0 0x5614aa556720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1904d0 0x5614aa56c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 --2- 172.21.15.94:0/3878959395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa177f60 0x5614aa565800 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 >> 172.21.15.94:0/3878959395 conn(0x5614aa1ad4e0 msgr2=0x5614aa5650f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 shutdown_connections 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.424+0000 7fb61917c740 1 -- 172.21.15.94:0/3878959395 wait complete. 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 -- start start 2024-04-25T13:57:39.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 Processor -- start 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1904d0 0x5614aa56a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa56b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1aec00 con 0x5614aa177f60 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1ae4d0 con 0x5614aa1986b0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1ab710 con 0x5614aa1904d0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb61917c740 10 client.0 _ll_register_callbacks cb 0x7fb604009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.425+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1904d0 0x5614aa56a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44442/0 (socket says 172.21.15.94:44442) 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 -- 172.21.15.94:0/2996992929 learned_addr learned my addr 172.21.15.94:0/2996992929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60bfff640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa56b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 -- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1904d0 msgr2=0x5614aa56a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1904d0 0x5614aa56a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 -- 172.21.15.94:0/2996992929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 msgr2=0x5614aa56b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa56b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 -- 172.21.15.94:0/2996992929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614aa56ce20 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60bfff640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa56b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb60b7fe640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc044d90 tx=0x7fb5fc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6386a6de2c518481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb5eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc05b070 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5fc0035f0 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb5fc004180 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc067050 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb61917c740 10 client.21192 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.426+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5614aa56c790 con 0x5614aa177f60 2024-04-25T13:57:39.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc0035f0 con 0x5614aa177f60 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb5fc03b4b0 con 0x5614aa177f60 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 10 client.21192 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 10 client.21192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 20 client.21192 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa1b9390 0x5614aa5dd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 4 client.21192 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614aa5645c0 con 0x5614aa1b9390 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb60affd640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa1b9390 0x5614aa5dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.427+0000 7fb5eaffd640 10 client.21192 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb60affd640 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa1b9390 0x5614aa5dd390 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1caabe3ba8b5dc0 server_cookie=656f84bd632ac5ec in_seq=0 out_seq=0 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb5eaffd640 1 -- 172.21.15.94:0/2996992929 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614aa5645c0 con 0x5614aa1b9390 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb5eaffd640 10 client.21192 handle_command_reply: tid=0 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 client.21192 shutdown 2024-04-25T13:57:39.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 msgr2=0x5614aa569cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc044d90 tx=0x7fb5fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 shutdown_connections 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5614aa1b9390 0x5614aa5dd390 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614aa1986b0 0x5614aa56b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb5eaffd640 0 client.21192 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5614aa1904d0 0x5614aa56a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 --2- 172.21.15.94:0/2996992929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614aa177f60 0x5614aa569cb0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 >> 172.21.15.94:0/2996992929 conn(0x5614aa1ad4e0 msgr2=0x5614aa555610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 shutdown_connections 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 1 -- 172.21.15.94:0/2996992929 wait complete. 2024-04-25T13:57:39.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.428+0000 7fb61917c740 20 client.21192 trim_cache size 0 max 16384 2024-04-25T13:57:39.435 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:39.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 msgr2=0x7fb5f40b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 0x7fb5f40b2100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5614aa55c860 tx=0x7fb604009500 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 msgr2=0x7fb60c19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb60000d950 tx=0x7fb60000de10 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 shutdown_connections 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5f40afbf0 0x7fb5f40b2100 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb60c19f0e0 0x7fb60c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb60c108a10 0x7fb60c19eba0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 --2- 172.21.15.94:0/3617944259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb60c103ee0 0x7fb60c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 >> 172.21.15.94:0/3617944259 conn(0x7fb60c0fd340 msgr2=0x7fb60c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 shutdown_connections 2024-04-25T13:57:39.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.429+0000 7fb6121f3640 1 -- 172.21.15.94:0/3617944259 wait complete. 2024-04-25T13:57:39.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.570+0000 7fdeef372640 1 -- 172.21.15.94:0/1536696464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 msgr2=0x7fdee8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.570+0000 7fdeef372640 1 --2- 172.21.15.94:0/1536696464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee8102e70 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009900 tx=0x7fdee402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 -- 172.21.15.94:0/1536696464 shutdown_connections 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 --2- 172.21.15.94:0/1536696464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 0x7fdee810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 --2- 172.21.15.94:0/1536696464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 0x7fdee81037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 --2- 172.21.15.94:0/1536696464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee8102e70 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 -- 172.21.15.94:0/1536696464 >> 172.21.15.94:0/1536696464 conn(0x7fdee80fd340 msgr2=0x7fdee80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 -- 172.21.15.94:0/1536696464 shutdown_connections 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 -- 172.21.15.94:0/1536696464 wait complete. 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 Processor -- start 2024-04-25T13:57:39.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 -- start start 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.571+0000 7fdeef372640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 0x7fdee819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 0x7fdee81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810f610 con 0x7fdee8102170 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810f490 con 0x7fdee81080b0 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdee810f790 con 0x7fdee81033b0 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44450/0 (socket says 172.21.15.94:44450) 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 -- 172.21.15.94:0/1137892127 learned_addr learned my addr 172.21.15.94:0/1137892127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed90f640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 0x7fdee81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeec90d640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 0x7fdee819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 -- 172.21.15.94:0/1137892127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 msgr2=0x7fdee819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 0x7fdee819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 -- 172.21.15.94:0/1137892127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 msgr2=0x7fdee81a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 0x7fdee81a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 -- 172.21.15.94:0/1137892127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee81a1d70 con 0x7fdee8102170 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeed10e640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006010 tx=0x7fdee4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdede7fc640 1 -- 172.21.15.94:0/1137892127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee403f030 con 0x7fdee8102170 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee81a2000 con 0x7fdee8102170 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdede7fc640 1 -- 172.21.15.94:0/1137892127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee4031910 con 0x7fdee8102170 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdede7fc640 1 -- 172.21.15.94:0/1137892127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4035070 con 0x7fdee8102170 2024-04-25T13:57:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.572+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee81a24f0 con 0x7fdee8102170 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.575+0000 7fdef6223740 1 Processor -- start 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.575+0000 7fdef6223740 1 -- start start 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.575+0000 7fdede7fc640 1 -- 172.21.15.94:0/1137892127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdee4042030 con 0x7fdee8102170 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556749715d80 con 0x5567493e64d0 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5567496cfa20 con 0x55674940f390 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdef6223740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556749711d40 con 0x5567493ee6b0 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed10e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58806/0 (socket says 172.21.15.94:58806) 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed10e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497becb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44458/0 (socket says 172.21.15.94:44458) 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 -- 172.21.15.94:0/2634950326 learned_addr learned my addr 172.21.15.94:0/2634950326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeec90d640 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 -- 172.21.15.94:0/2634950326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 msgr2=0x5567497c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 -- 172.21.15.94:0/2634950326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 msgr2=0x5567497becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.576+0000 7fdeed90f640 1 -- 172.21.15.94:0/2634950326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567496dac30 con 0x55674940f390 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fdede7fc640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 0x7fded40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009fd0 tx=0x7fdee00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af305fdee50d945a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fdeed10e640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 0x7fded40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fded37fe640 1 -- 172.21.15.94:0/2634950326 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004d20 con 0x55674940f390 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fdede7fc640 1 -- 172.21.15.94:0/1137892127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fdee40fefb0 con 0x7fdee8102170 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fded37fe640 1 -- 172.21.15.94:0/2634950326 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee0004050 con 0x55674940f390 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.577+0000 7fded37fe640 1 -- 172.21.15.94:0/2634950326 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0005280 con 0x55674940f390 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- 172.21.15.94:0/2634950326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 msgr2=0x5567497bc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdeed10e640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 0x7fded40b2000 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdee40c71a0 tx=0x7fdee4102a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009fd0 tx=0x7fdee00118d0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- 172.21.15.94:0/2634950326 shutdown_connections 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 --2- 172.21.15.94:0/2634950326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497bc6e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- 172.21.15.94:0/2634950326 >> 172.21.15.94:0/2634950326 conn(0x5567494034e0 msgr2=0x5567497acfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- 172.21.15.94:0/2634950326 shutdown_connections 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- 172.21.15.94:0/2634950326 wait complete. 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 -- start start 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.578+0000 7fdef6223740 1 Processor -- start 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5567497c1f60 con 0x5567493e64d0 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55674969cdc0 con 0x55674940f390 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567497c20e0 con 0x5567493ee6b0 2024-04-25T13:57:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdef6223740 10 client.0 _ll_register_callbacks cb 0x5567496b8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44466/0 (socket says 172.21.15.94:44466) 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 -- 172.21.15.94:0/2516730629 learned_addr learned my addr 172.21.15.94:0/2516730629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeec90d640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 -- 172.21.15.94:0/2516730629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 msgr2=0x5567497c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 -- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 msgr2=0x5567497cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 -- 172.21.15.94:0/2516730629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567494273f0 con 0x5567493e64d0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fdeed90f640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003eb0 tx=0x7fdee0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49202e041adaaac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.579+0000 7fded0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004d90 con 0x5567493e64d0 2024-04-25T13:57:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fdee0005040 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee0004050 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdee0005240 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 10 client.21204 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5567494f17d0 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee0004050 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdee000f8d0 con 0x5567493e64d0 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 10 client.21204 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 10 client.21204 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 20 client.21204 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493cdf60 0x5567497d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 4 client.21204 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556749723990 con 0x5567493cdf60 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.580+0000 7fdeec90d640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493cdf60 0x5567497d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.581+0000 7fded0ff9640 10 client.21204 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.581+0000 7fdeec90d640 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493cdf60 0x5567497d1700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c96dd4b2ed9a73d9 server_cookie=f65853f56882b44a in_seq=0 out_seq=0 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fded0ff9640 1 -- 172.21.15.94:0/2516730629 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556749723990 con 0x5567493cdf60 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fded0ff9640 10 client.21204 handle_command_reply: tid=0 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 client.21204 shutdown 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 msgr2=0x5567497c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003eb0 tx=0x7fdee0004770 comp rx=0 tx=0).stop 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 shutdown_connections 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493cdf60 0x5567497d1700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fded0ff9640 0 client.21204 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 --2- 172.21.15.94:0/2516730629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497c3d00 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 >> 172.21.15.94:0/2516730629 conn(0x5567494034e0 msgr2=0x5567497ab610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 shutdown_connections 2024-04-25T13:57:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 1 -- 172.21.15.94:0/2516730629 wait complete. 2024-04-25T13:57:39.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.582+0000 7fdef6223740 20 client.21204 trim_cache size 0 max 16384 2024-04-25T13:57:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 Processor -- start 2024-04-25T13:57:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 -- start start 2024-04-25T13:57:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81a9330 con 0x5567493e64d0 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81aaed0 con 0x55674940f390 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.597+0000 7fdef6223740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81a9a00 con 0x5567493ee6b0 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44472/0 (socket says 172.21.15.94:44472) 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeed90f640 1 -- 172.21.15.94:0/579867190 learned_addr learned my addr 172.21.15.94:0/579867190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeed90f640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 -- 172.21.15.94:0/579867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 msgr2=0x5567497c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 -- 172.21.15.94:0/579867190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 msgr2=0x5567497bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 -- 172.21.15.94:0/579867190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567497bec80 con 0x5567493ee6b0 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeed90f640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fdeec90d640 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497bfb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5567497d1e40 tx=0x7fded8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5dafb50e0021bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fded2ffd640 1 -- 172.21.15.94:0/579867190 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004230 con 0x5567493ee6b0 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fded2ffd640 1 -- 172.21.15.94:0/579867190 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded80043d0 con 0x5567493ee6b0 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.598+0000 7fded2ffd640 1 -- 172.21.15.94:0/579867190 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004e50 con 0x5567493ee6b0 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 -- 172.21.15.94:0/579867190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 msgr2=0x5567497bfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497bfb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5567497d1e40 tx=0x7fded8005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 -- 172.21.15.94:0/579867190 shutdown_connections 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55674940f390 0x5567497c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493ee6b0 0x5567497bfb80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 --2- 172.21.15.94:0/579867190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567493e64d0 0x5567497bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 -- 172.21.15.94:0/579867190 >> 172.21.15.94:0/579867190 conn(0x5567494034e0 msgr2=0x5567497c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 -- 172.21.15.94:0/579867190 shutdown_connections 2024-04-25T13:57:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.599+0000 7fdef6223740 1 -- 172.21.15.94:0/579867190 wait complete. 2024-04-25T13:57:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 -- start start 2024-04-25T13:57:39.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 Processor -- start 2024-04-25T13:57:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493e64d0 0x5567497ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 0x5567497c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81aa800 con 0x55674940f390 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81aa0d0 con 0x5567493ee6b0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81a7310 con 0x5567493e64d0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdef6223740 10 client.0 _ll_register_callbacks cb 0x7fdee4102960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdeec90d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 0x5567497c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdeed90f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44478/0 (socket says 172.21.15.94:44478) 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdeed90f640 1 -- 172.21.15.94:0/3047425123 learned_addr learned my addr 172.21.15.94:0/3047425123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.600+0000 7fdeed90f640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493e64d0 0x5567497ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 -- 172.21.15.94:0/3047425123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493e64d0 msgr2=0x5567497ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493e64d0 0x5567497ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 -- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 msgr2=0x5567497c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 0x5567497c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 -- 172.21.15.94:0/3047425123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556749837aa0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeec90d640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 0x5567497c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdeed90f640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fdee0002230 tx=0x7fdee00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad6f57a41c415e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fded0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee00040c0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fdee0004fa0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded80043d0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdee000f4e0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdef6223740 10 client.21210 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.601+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556749838cd0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded80043d0 con 0x55674940f390 2024-04-25T13:57:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdee0005920 con 0x55674940f390 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 10 client.21210 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 10 client.21210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 20 client.21210 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493a33a0 0x55674983b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 4 client.21210 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55674971d800 con 0x5567493a33a0 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fdeec90d640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493a33a0 0x55674983b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.602+0000 7fded0ff9640 10 client.21210 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdeec90d640 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493a33a0 0x55674983b550 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8112e41afa31cf19 server_cookie=cbdc91aacc4fbc62 in_seq=0 out_seq=0 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fded0ff9640 1 -- 172.21.15.94:0/3047425123 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55674971d800 con 0x5567493a33a0 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fded0ff9640 10 client.21210 handle_command_reply: tid=0 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 client.21210 shutdown 2024-04-25T13:57:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 msgr2=0x5567497c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fdee0002230 tx=0x7fdee00042f0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 shutdown_connections 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5567493a33a0 0x55674983b550 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55674940f390 0x5567497c7960 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567493ee6b0 0x5567497c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fded0ff9640 0 client.21210 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 --2- 172.21.15.94:0/3047425123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567493e64d0 0x5567497ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 >> 172.21.15.94:0/3047425123 conn(0x5567494034e0 msgr2=0x5567497b94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 shutdown_connections 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 1 -- 172.21.15.94:0/3047425123 wait complete. 2024-04-25T13:57:39.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.603+0000 7fdef6223740 20 client.21210 trim_cache size 0 max 16384 2024-04-25T13:57:39.610 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 msgr2=0x7fded40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 0x7fded40b2000 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdee40c71a0 tx=0x7fdee4102a90 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 msgr2=0x7fdee819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdee4006010 tx=0x7fdee4004270 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 shutdown_connections 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fded40afaf0 0x7fded40b2000 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdee81080b0 0x7fdee81a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee81033b0 0x7fdee819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 --2- 172.21.15.94:0/1137892127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee8102170 0x7fdee819a230 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 >> 172.21.15.94:0/1137892127 conn(0x7fdee80fd340 msgr2=0x7fdee8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 shutdown_connections 2024-04-25T13:57:39.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.604+0000 7fdeef372640 1 -- 172.21.15.94:0/1137892127 wait complete. 2024-04-25T13:57:39.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:39 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3945015762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:39.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:57:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:39 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3945015762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:39.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 -- 172.21.15.94:0/831282626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 msgr2=0x7f5900103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 --2- 172.21.15.94:0/831282626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900103370 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009900 tx=0x7f58f002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:39.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 -- 172.21.15.94:0/831282626 shutdown_connections 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 --2- 172.21.15.94:0/831282626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900104480 0x7f590010abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 --2- 172.21.15.94:0/831282626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900103940 0x7f5900103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 --2- 172.21.15.94:0/831282626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900103370 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 -- 172.21.15.94:0/831282626 >> 172.21.15.94:0/831282626 conn(0x7f59000fd380 msgr2=0x7f59000ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 -- 172.21.15.94:0/831282626 shutdown_connections 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.737+0000 7f590551f640 1 -- 172.21.15.94:0/831282626 wait complete. 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 Processor -- start 2024-04-25T13:57:39.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 -- start start 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900103940 0x7f59001996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900104480 0x7f59001a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f590010e510 con 0x7f5900102f60 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f590010e390 con 0x7f5900103940 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f590551f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f590010e690 con 0x7f5900104480 2024-04-25T13:57:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44494/0 (socket says 172.21.15.94:44494) 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 -- 172.21.15.94:0/2452866049 learned_addr learned my addr 172.21.15.94:0/2452866049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900103940 0x7f59001996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900104480 0x7f59001a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 -- 172.21.15.94:0/2452866049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900104480 msgr2=0x7f59001a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900104480 0x7f59001a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 -- 172.21.15.94:0/2452866049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900103940 msgr2=0x7f59001996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900103940 0x7f59001996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.738+0000 7f58feffd640 1 -- 172.21.15.94:0/2452866049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59001a0cc0 con 0x7f5900102f60 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f58feffd640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f58f0006010 tx=0x7f58f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f58e7fff640 1 -- 172.21.15.94:0/2452866049 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f003f030 con 0x7f5900102f60 2024-04-25T13:57:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f58e7fff640 1 -- 172.21.15.94:0/2452866049 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0031910 con 0x7f5900102f60 2024-04-25T13:57:39.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001a0f50 con 0x7f5900102f60 2024-04-25T13:57:39.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f58e7fff640 1 -- 172.21.15.94:0/2452866049 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0035070 con 0x7f5900102f60 2024-04-25T13:57:39.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.739+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a1350 con 0x7f5900102f60 2024-04-25T13:57:39.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3945015762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f58e7fff640 1 -- 172.21.15.94:0/2452866049 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f58f0042030 con 0x7f5900102f60 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 Processor -- start 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 -- start start 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a08a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56459596bd80 con 0x56459563c4d0 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564595925a20 con 0x564595665390 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.741+0000 7f590c3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564595967d40 con 0x5645956446b0 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44496/0 (socket says 172.21.15.94:44496) 2024-04-25T13:57:39.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58feffd640 1 -- 172.21.15.94:0/2931950986 learned_addr learned my addr 172.21.15.94:0/2931950986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a08a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 -- 172.21.15.94:0/2931950986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 msgr2=0x564595a08a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a08a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58e7fff640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 0x7f58e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 -- 172.21.15.94:0/2931950986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 msgr2=0x564595a07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 -- 172.21.15.94:0/2931950986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595930c30 con 0x564595665390 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58feffd640 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58e7fff640 1 -- 172.21.15.94:0/2452866049 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f58f00ff060 con 0x7f5900102f60 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 0x7f58e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a07810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009e10 tx=0x7f58f800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de9df6a12efddb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58e57fa640 1 -- 172.21.15.94:0/2931950986 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f801c980 con 0x564595665390 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58e57fa640 1 -- 172.21.15.94:0/2931950986 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f801cba0 con 0x564595665390 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.742+0000 7f58e57fa640 1 -- 172.21.15.94:0/2931950986 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f801b6c0 con 0x564595665390 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.743+0000 7f590c3d3740 1 -- 172.21.15.94:0/2931950986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 msgr2=0x564595a07810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.743+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 0x7f58e80b2000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f58f801b000 tx=0x7f58f801bdf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.743+0000 7f590c3d3740 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a07810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009e10 tx=0x7f58f800f730 comp rx=0 tx=0).stop 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 -- 172.21.15.94:0/2931950986 shutdown_connections 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a08a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 --2- 172.21.15.94:0/2931950986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a07810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 -- 172.21.15.94:0/2931950986 >> 172.21.15.94:0/2931950986 conn(0x5645956594e0 msgr2=0x564595a03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 -- 172.21.15.94:0/2931950986 shutdown_connections 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 -- 172.21.15.94:0/2931950986 wait complete. 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 -- start start 2024-04-25T13:57:39.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 Processor -- start 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.744+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 0x564595a114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a18980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564595a09530 con 0x5645956446b0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5645958f2dc0 con 0x564595665390 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564595a096b0 con 0x56459563c4d0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f590c3d3740 10 client.0 _ll_register_callbacks cb 0x56459590e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58fe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 0x564595a114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58fe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44506/0 (socket says 172.21.15.94:44506) 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58fe7fc640 1 -- 172.21.15.94:0/364555516 learned_addr learned my addr 172.21.15.94:0/364555516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58ff7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a18980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 -- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 msgr2=0x564595a18980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a18980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 -- 172.21.15.94:0/364555516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 msgr2=0x564595a11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 -- 172.21.15.94:0/364555516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459567d3f0 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58feffd640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 0x564595a114e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031440 tx=0x7f58f003b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb4d55e9f78be5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.745+0000 7f58e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0049070 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f58f00c3730 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f801cba0 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58f00c6df0 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 10 client.30352 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5645957477d0 con 0x56459563c4d0 2024-04-25T13:57:39.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00c6fb0 con 0x56459563c4d0 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f58f0101110 con 0x56459563c4d0 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 10 client.30352 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 10 client.30352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 20 client.30352 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.746+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564595623f60 0x564595a1f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.747+0000 7f590c3d3740 4 client.30352 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.747+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58f801cba0 con 0x564595623f60 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.747+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564595623f60 0x564595a1f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.747+0000 7f58e2ff5640 10 client.30352 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.747+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564595623f60 0x564595a1f1f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581a5e07f5bea748 server_cookie=189701c3e18e2988 in_seq=0 out_seq=0 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.748+0000 7f58e2ff5640 1 -- 172.21.15.94:0/364555516 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f58f801cba0 con 0x564595623f60 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.748+0000 7f58e2ff5640 10 client.30352 handle_command_reply: tid=0 2024-04-25T13:57:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 client.30352 shutdown 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 msgr2=0x564595a114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 0x564595a114e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031440 tx=0x7f58f003b460 comp rx=0 tx=0).stop 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 shutdown_connections 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564595623f60 0x564595a1f1f0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a18980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645956446b0 0x564595a11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f58e2ff5640 0 client.30352 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 --2- 172.21.15.94:0/364555516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56459563c4d0 0x564595a114e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 >> 172.21.15.94:0/364555516 conn(0x5645956594e0 msgr2=0x564595a02040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 shutdown_connections 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 1 -- 172.21.15.94:0/364555516 wait complete. 2024-04-25T13:57:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.749+0000 7f590c3d3740 20 client.30352 trim_cache size 0 max 16384 2024-04-25T13:57:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 Processor -- start 2024-04-25T13:57:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 -- start start 2024-04-25T13:57:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645956446b0 0x564595a110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564595665390 0x564595a115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59001a9370 con 0x56459563c4d0 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59001aaf10 con 0x5645956446b0 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f590c3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59001a9a40 con 0x564595665390 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44512/0 (socket says 172.21.15.94:44512) 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.764+0000 7f58feffd640 1 -- 172.21.15.94:0/1270289532 learned_addr learned my addr 172.21.15.94:0/1270289532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645956446b0 0x564595a110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564595665390 0x564595a115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 -- 172.21.15.94:0/1270289532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564595665390 msgr2=0x564595a115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564595665390 0x564595a115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 -- 172.21.15.94:0/1270289532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645956446b0 msgr2=0x564595a110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645956446b0 0x564595a110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 -- 172.21.15.94:0/1270289532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595a0cc40 con 0x56459563c4d0 2024-04-25T13:57:39.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58feffd640 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f58f003d040 tx=0x7f58f003b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e27365d48609c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58e4ff9640 1 -- 172.21.15.94:0/1270289532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0049030 con 0x56459563c4d0 2024-04-25T13:57:39.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58e4ff9640 1 -- 172.21.15.94:0/1270289532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00c7ba0 con 0x56459563c4d0 2024-04-25T13:57:39.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.765+0000 7f58e4ff9640 1 -- 172.21.15.94:0/1270289532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f00c6c10 con 0x56459563c4d0 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- 172.21.15.94:0/1270289532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 msgr2=0x564595a10430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f58f003d040 tx=0x7f58f003b4f0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- 172.21.15.94:0/1270289532 shutdown_connections 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564595665390 0x564595a115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645956446b0 0x564595a110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1270289532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a10430 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- 172.21.15.94:0/1270289532 >> 172.21.15.94:0/1270289532 conn(0x5645956594e0 msgr2=0x564595a0fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- 172.21.15.94:0/1270289532 shutdown_connections 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- 172.21.15.94:0/1270289532 wait complete. 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.766+0000 7f590c3d3740 1 -- start start 2024-04-25T13:57:39.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 Processor -- start 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a15cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59001aa840 con 0x56459563c4d0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59001aa110 con 0x564595665390 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59001a7350 con 0x5645956446b0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f590c3d3740 10 client.0 _ll_register_callbacks cb 0x7f58f0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44528/0 (socket says 172.21.15.94:44528) 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 -- 172.21.15.94:0/1806081007 learned_addr learned my addr 172.21.15.94:0/1806081007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a15cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 -- 172.21.15.94:0/1806081007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 msgr2=0x564595a15cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a15cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 -- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 msgr2=0x564595a16bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a16bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 -- 172.21.15.94:0/1806081007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564595a171b0 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58ff7fe640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a16bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58feffd640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031440 tx=0x7f58f00c3960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0b225d99d020ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.767+0000 7f58e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0049030 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f58f00c3a90 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f00c7ba0 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58f00c6c10 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 10 client.21228 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564595a171b0 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f00c7ba0 con 0x56459563c4d0 2024-04-25T13:57:39.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f58f00e82e0 con 0x56459563c4d0 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 10 client.21228 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 10 client.21228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 20 client.21228 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645955f93a0 0x564595a8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 4 client.21228 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564595979990 con 0x5645955f93a0 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.768+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645955f93a0 0x564595a8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.769+0000 7f58e2ff5640 10 client.21228 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:39.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.769+0000 7f58fe7fc640 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645955f93a0 0x564595a8de80 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41818c613a4980d4 server_cookie=933c8ff88d3029e7 in_seq=0 out_seq=0 2024-04-25T13:57:39.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f58e2ff5640 1 -- 172.21.15.94:0/1806081007 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564595979990 con 0x5645955f93a0 2024-04-25T13:57:39.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f58e2ff5640 10 client.21228 handle_command_reply: tid=0 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 client.21228 shutdown 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 msgr2=0x564595a15780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031440 tx=0x7f58f00c3960 comp rx=0 tx=0).stop 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 shutdown_connections 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645955f93a0 0x564595a8de80 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564595665390 0x564595a16bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645956446b0 0x564595a15cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f58e2ff5640 0 client.21228 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 --2- 172.21.15.94:0/1806081007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56459563c4d0 0x564595a15780 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 >> 172.21.15.94:0/1806081007 conn(0x5645956594e0 msgr2=0x564595a05ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 shutdown_connections 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 1 -- 172.21.15.94:0/1806081007 wait complete. 2024-04-25T13:57:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590c3d3740 20 client.21228 trim_cache size 0 max 16384 2024-04-25T13:57:39.777 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 msgr2=0x7f58e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 0x7f58e80b2000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f58f801b000 tx=0x7f58f801bdf0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 msgr2=0x7f5900199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f58f0006010 tx=0x7f58f0004270 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 shutdown_connections 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58e80afaf0 0x7f58e80b2000 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900104480 0x7f59001a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5900103940 0x7f59001996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 --2- 172.21.15.94:0/2452866049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5900102f60 0x7f5900199180 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.772+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 >> 172.21.15.94:0/2452866049 conn(0x7f59000fd380 msgr2=0x7f5900104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.773+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 shutdown_connections 2024-04-25T13:57:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:39.773+0000 7f590551f640 1 -- 172.21.15.94:0/2452866049 wait complete. 2024-04-25T13:57:40.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 52372 19.26 MB/sec execute 22 sec latency 67.193 ms 2024-04-25T13:57:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:40 smithi137 ceph-mon[32568]: pgmap v826: 97 pgs: 97 active+clean; 257 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-04-25T13:57:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:40 smithi094 ceph-mon[29195]: pgmap v826: 97 pgs: 97 active+clean; 257 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-04-25T13:57:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:40 smithi138 ceph-mon[32977]: pgmap v826: 97 pgs: 97 active+clean; 257 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-04-25T13:57:40.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:57:40.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:57:40.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:57:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 -- 172.21.15.94:0/279015050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8108920 msgr2=0x7f49e810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 --2- 172.21.15.94:0/279015050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8108920 0x7f49e810ad40 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ca0 tx=0x7f49e002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 -- 172.21.15.94:0/279015050 shutdown_connections 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 --2- 172.21.15.94:0/279015050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e8108920 0x7f49e810ad40 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 --2- 172.21.15.94:0/279015050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 --2- 172.21.15.94:0/279015050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e81029e0 0x7f49e81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 -- 172.21.15.94:0/279015050 >> 172.21.15.94:0/279015050 conn(0x7f49e80fdbd0 msgr2=0x7f49e8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 -- 172.21.15.94:0/279015050 shutdown_connections 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.908+0000 7f49ef051640 1 -- 172.21.15.94:0/279015050 wait complete. 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 Processor -- start 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 -- start start 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 0x7f49e8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e8108920 0x7f49e81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810e510 con 0x7f49e81029e0 2024-04-25T13:57:40.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810e390 con 0x7f49e8103c20 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ef051640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810e690 con 0x7f49e8108920 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 0x7f49e8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 0x7f49e8075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44542/0 (socket says 172.21.15.94:44542) 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ecded640 1 -- 172.21.15.94:0/2617527817 learned_addr learned my addr 172.21.15.94:0/2617527817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.909+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e8108920 0x7f49e81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 -- 172.21.15.94:0/2617527817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e8108920 msgr2=0x7f49e81a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e8108920 0x7f49e81a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 -- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 msgr2=0x7f49e8075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 0x7f49e8075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 -- 172.21.15.94:0/2617527817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e81a1d60 con 0x7f49e8103c20 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e7fff640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e819efd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e9f0 tx=0x7f49d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e5ffb640 1 -- 172.21.15.94:0/2617527817 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d400ccc0 con 0x7f49e8103c20 2024-04-25T13:57:40.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e5ffb640 1 -- 172.21.15.94:0/2617527817 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49d40108e0 con 0x7f49e8103c20 2024-04-25T13:57:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e81a1ff0 con 0x7f49e8103c20 2024-04-25T13:57:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49e5ffb640 1 -- 172.21.15.94:0/2617527817 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4018880 con 0x7f49e8103c20 2024-04-25T13:57:40.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.910+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81a24e0 con 0x7f49e8103c20 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.913+0000 7f49e5ffb640 1 -- 172.21.15.94:0/2617527817 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f49d4010430 con 0x7f49e8103c20 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.913+0000 7f49f5fe2740 1 Processor -- start 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.913+0000 7f49f5fe2740 1 -- start start 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d23474cb00 con 0x55d23441d550 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2346e8920 con 0x55d234425730 2024-04-25T13:57:40.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d234748ac0 con 0x55d234446410 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34784/0 (socket says 172.21.15.94:34784) 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44544/0 (socket says 172.21.15.94:44544) 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 -- 172.21.15.94:0/1055559688 learned_addr learned my addr 172.21.15.94:0/1055559688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49e5ffb640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 0x7f49d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 0x7f49d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 -- 172.21.15.94:0/1055559688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 msgr2=0x55d2347ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347ee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 -- 172.21.15.94:0/1055559688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 msgr2=0x55d2347e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49e5ffb640 1 -- 172.21.15.94:0/2617527817 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f49d40df010 con 0x7f49e8103c20 2024-04-25T13:57:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.914+0000 7f49ed5ee640 1 -- 172.21.15.94:0/1055559688 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2345267e0 con 0x55d234446410 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49ecded640 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ca0 tx=0x7f49e0031820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e353c0cd920602c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49d2ffd640 1 -- 172.21.15.94:0/1055559688 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0004d40 con 0x55d234446410 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 0x7f49d80b2000 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49e004e000 tx=0x7f49e004c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49d2ffd640 1 -- 172.21.15.94:0/1055559688 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e0036070 con 0x55d234446410 2024-04-25T13:57:40.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.915+0000 7f49d2ffd640 1 -- 172.21.15.94:0/1055559688 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e002fa80 con 0x55d234446410 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 -- 172.21.15.94:0/1055559688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 msgr2=0x55d2347e74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ca0 tx=0x7f49e0031820 comp rx=0 tx=0).stop 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 -- 172.21.15.94:0/1055559688 shutdown_connections 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/1055559688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347e74a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.916+0000 7f49f5fe2740 1 -- 172.21.15.94:0/1055559688 >> 172.21.15.94:0/1055559688 conn(0x55d23443a560 msgr2=0x55d2347e2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.917+0000 7f49f5fe2740 1 -- 172.21.15.94:0/1055559688 shutdown_connections 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.917+0000 7f49f5fe2740 1 -- 172.21.15.94:0/1055559688 wait complete. 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.917+0000 7f49f5fe2740 1 -- start start 2024-04-25T13:57:40.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.917+0000 7f49f5fe2740 1 Processor -- start 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d2347ef1f0 con 0x55d23441d550 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2346e65d0 con 0x55d234425730 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2347ef370 con 0x55d234446410 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49f5fe2740 10 client.0 _ll_register_callbacks cb 0x55d2346d4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ecded640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44556/0 (socket says 172.21.15.94:44556) 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ecded640 1 -- 172.21.15.94:0/4038370463 learned_addr learned my addr 172.21.15.94:0/4038370463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49e7fff640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 -- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 msgr2=0x55d2347f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347f16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 -- 172.21.15.94:0/4038370463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 msgr2=0x55d2347f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ed5ee640 1 -- 172.21.15.94:0/4038370463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d234586f70 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.918+0000 7f49ecded640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49e7fff640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347f8620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005e30 tx=0x7f49e002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4433c9123710eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0004640 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f49e002f800 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e0036070 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f49e002fa00 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49f5fe2740 10 client.30367 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.919+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d23441e170 con 0x55d234446410 2024-04-25T13:57:40.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e004f070 con 0x55d234446410 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f49e0030a10 con 0x55d234446410 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 10 client.30367 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 10 client.30367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 20 client.30367 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2344051f0 0x55d2347fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 4 client.30367 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f49e0036070 con 0x55d2344051f0 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49ecded640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2344051f0 0x55d2347fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.920+0000 7f49c7fff640 10 client.30367 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:40.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.921+0000 7f49ecded640 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2344051f0 0x55d2347fe7a0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb14d0161a832ba1 server_cookie=1b781001e0e60468 in_seq=0 out_seq=0 2024-04-25T13:57:40.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49c7fff640 1 -- 172.21.15.94:0/4038370463 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f49e0036070 con 0x55d2344051f0 2024-04-25T13:57:40.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49c7fff640 10 client.30367 handle_command_reply: tid=0 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 client.30367 shutdown 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 msgr2=0x55d2347f8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347f8620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005e30 tx=0x7f49e002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 shutdown_connections 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2344051f0 0x55d2347fe7a0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d234446410 0x55d2347f8620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/4038370463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d23441d550 0x55d2347f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49c7fff640 0 client.30367 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 >> 172.21.15.94:0/4038370463 conn(0x55d23443a560 msgr2=0x55d2347e1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 shutdown_connections 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 1 -- 172.21.15.94:0/4038370463 wait complete. 2024-04-25T13:57:40.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.926+0000 7f49f5fe2740 20 client.30367 trim_cache size 0 max 16384 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 Processor -- start 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 -- start start 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347e6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81ab610 con 0x55d234446410 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81ad1b0 con 0x55d234425730 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81abce0 con 0x55d23441d550 2024-04-25T13:57:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ed5ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ecded640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ed5ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44570/0 (socket says 172.21.15.94:44570) 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ed5ee640 1 -- 172.21.15.94:0/3754521587 learned_addr learned my addr 172.21.15.94:0/3754521587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ecded640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34806/0 (socket says 172.21.15.94:34806) 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49e7fff640 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ed5ee640 1 -- 172.21.15.94:0/3754521587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 msgr2=0x55d2347f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.945+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49ed5ee640 1 -- 172.21.15.94:0/3754521587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 msgr2=0x55d2347e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49ed5ee640 1 -- 172.21.15.94:0/3754521587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2347e3970 con 0x55d234446410 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49ecded640 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009a60 tx=0x7f49e002f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549c490a6e8ed9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49d27fc640 1 -- 172.21.15.94:0/3754521587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e002f660 con 0x55d234446410 2024-04-25T13:57:40.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49d27fc640 1 -- 172.21.15.94:0/3754521587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e0036070 con 0x55d234446410 2024-04-25T13:57:40.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.946+0000 7f49d27fc640 1 -- 172.21.15.94:0/3754521587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e00306b0 con 0x55d234446410 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- 172.21.15.94:0/3754521587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 msgr2=0x55d2347e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009a60 tx=0x7f49e002f390 comp rx=0 tx=0).stop 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- 172.21.15.94:0/3754521587 shutdown_connections 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234446410 0x55d2347e6880 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234425730 0x55d2347e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/3754521587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- 172.21.15.94:0/3754521587 >> 172.21.15.94:0/3754521587 conn(0x55d23443a560 msgr2=0x55d2347e5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- 172.21.15.94:0/3754521587 shutdown_connections 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- 172.21.15.94:0/3754521587 wait complete. 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 -- start start 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.947+0000 7f49f5fe2740 1 Processor -- start 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234446410 0x55d2347f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81acae0 con 0x55d234425730 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81ac3b0 con 0x55d234446410 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81a95f0 con 0x55d23441d550 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49f5fe2740 10 client.0 _ll_register_callbacks cb 0x7f49e0031670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44572/0 (socket says 172.21.15.94:44572) 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 -- 172.21.15.94:0/357612960 learned_addr learned my addr 172.21.15.94:0/357612960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49ecded640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49ed5ee640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234446410 0x55d2347f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 -- 172.21.15.94:0/357612960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 msgr2=0x55d2347f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 -- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234446410 msgr2=0x55d2347f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234446410 0x55d2347f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 -- 172.21.15.94:0/357612960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2347f7a70 con 0x55d234425730 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49e7fff640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f49d40a3aa0 tx=0x7f49d400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8417f92c87864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.948+0000 7f49c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d40a7010 con 0x55d234425730 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f49d40aeb80 con 0x55d234425730 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e0036070 con 0x55d234425730 2024-04-25T13:57:40.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f49d40a7500 con 0x55d234425730 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 10 client.21246 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d2347f8e00 con 0x55d234425730 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e0036070 con 0x55d234425730 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f49d40c8210 con 0x55d234425730 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 10 client.21246 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 10 client.21246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 20 client.21246 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2343da630 0x55d234869030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 4 client.21246 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.949+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d23475a710 con 0x55d2343da630 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.950+0000 7f49ecded640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2343da630 0x55d234869030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.950+0000 7f49c7fff640 10 client.21246 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.950+0000 7f49ecded640 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2343da630 0x55d234869030 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4659698f16ddff18 server_cookie=aa1236d9fb5824a8 in_seq=0 out_seq=0 2024-04-25T13:57:40.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49c7fff640 1 -- 172.21.15.94:0/357612960 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d23475a710 con 0x55d2343da630 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49c7fff640 10 client.21246 handle_command_reply: tid=0 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 client.21246 shutdown 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 msgr2=0x55d2347f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f49d40a3aa0 tx=0x7f49d400ce60 comp rx=0 tx=0).stop 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 shutdown_connections 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2343da630 0x55d234869030 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234446410 0x55d2347f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49c7fff640 0 client.21246 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d234425730 0x55d2347f6580 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 --2- 172.21.15.94:0/357612960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d23441d550 0x55d2347f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 >> 172.21.15.94:0/357612960 conn(0x55d23443a560 msgr2=0x55d2347eff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 shutdown_connections 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.963+0000 7f49f5fe2740 1 -- 172.21.15.94:0/357612960 wait complete. 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49f5fe2740 20 client.21246 trim_cache size 0 max 16384 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "6beaebc5-976f-4f58-992e-b3b88da73e52", 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:57:40.965 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:40.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 msgr2=0x7f49d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 0x7f49d80b2000 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49e004e000 tx=0x7f49e004c000 comp rx=0 tx=0).stop 2024-04-25T13:57:40.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 msgr2=0x7f49e819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e819efd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e9f0 tx=0x7f49d400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 shutdown_connections 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f49d80afaf0 0x7f49d80b2000 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49e8108920 0x7f49e81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8103c20 0x7f49e819efd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 --2- 172.21.15.94:0/2617527817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49e81029e0 0x7f49e8075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 >> 172.21.15.94:0/2617527817 conn(0x7f49e80fdbd0 msgr2=0x7f49e8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 shutdown_connections 2024-04-25T13:57:40.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:40.964+0000 7f49ef051640 1 -- 172.21.15.94:0/2617527817 wait complete. 2024-04-25T13:57:40.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:57:40.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 -- 172.21.15.94:0/3337767795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 msgr2=0x7f3f801027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3337767795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f801027c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009900 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 -- 172.21.15.94:0/3337767795 shutdown_connections 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3337767795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 0x7f3f8010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3337767795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f801043c0 0x7f3f80102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3337767795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f801027c0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 -- 172.21.15.94:0/3337767795 >> 172.21.15.94:0/3337767795 conn(0x7f3f800fd340 msgr2=0x7f3f800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 -- 172.21.15.94:0/3337767795 shutdown_connections 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.098+0000 7f3f8817a640 1 -- 172.21.15.94:0/3337767795 wait complete. 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 Processor -- start 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 -- start start 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 0x7f3f80199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8019e1d0 0x7f3f801a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010e4f0 con 0x7f3f80103ee0 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010e370 con 0x7f3f80108a10 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f8817a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8010e670 con 0x7f3f8019e1d0 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44600/0 (socket says 172.21.15.94:44600) 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 -- 172.21.15.94:0/3477954325 learned_addr learned my addr 172.21.15.94:0/3477954325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85715640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 0x7f3f80199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f86717640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8019e1d0 0x7f3f801a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 -- 172.21.15.94:0/3477954325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8019e1d0 msgr2=0x7f3f801a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8019e1d0 0x7f3f801a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 -- 172.21.15.94:0/3477954325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 msgr2=0x7f3f80199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 0x7f3f80199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 -- 172.21.15.94:0/3477954325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f801a0c60 con 0x7f3f80103ee0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.099+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006010 tx=0x7f3f7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.100+0000 7f3f76ffd640 1 -- 172.21.15.94:0/3477954325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c03f030 con 0x7f3f80103ee0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.100+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f801a0ef0 con 0x7f3f80103ee0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.100+0000 7f3f76ffd640 1 -- 172.21.15.94:0/3477954325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c031910 con 0x7f3f80103ee0 2024-04-25T13:57:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.100+0000 7f3f76ffd640 1 -- 172.21.15.94:0/3477954325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035070 con 0x7f3f80103ee0 2024-04-25T13:57:41.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.100+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f80101890 con 0x7f3f80103ee0 2024-04-25T13:57:41.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.102+0000 7f3f76ffd640 1 -- 172.21.15.94:0/3477954325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3f7c042030 con 0x7f3f80103ee0 2024-04-25T13:57:41.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.102+0000 7f3f8f020740 1 Processor -- start 2024-04-25T13:57:41.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 -- start start 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e47efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e47fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56477e3e3d80 con 0x56477e0dd390 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56477e39d8b0 con 0x56477e0bc6b0 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f8f020740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56477e3dfd40 con 0x56477e0b44d0 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e47fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44604/0 (socket says 172.21.15.94:44604) 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f86717640 1 -- 172.21.15.94:0/1297661468 learned_addr learned my addr 172.21.15.94:0/1297661468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85f16640 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e47efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f76ffd640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 0x7f3f6c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 -- 172.21.15.94:0/1297661468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 msgr2=0x56477e47efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e47efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 -- 172.21.15.94:0/1297661468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 msgr2=0x56477e47ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f85715640 1 -- 172.21.15.94:0/1297661468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477e36ea40 con 0x56477e0bc6b0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f86717640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 0x7f3f6c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.103+0000 7f3f76ffd640 1 -- 172.21.15.94:0/3477954325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f3f7c0fefe0 con 0x7f3f80103ee0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.104+0000 7f3f85715640 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e47fc70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009c00 tx=0x7f3f70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a261a050e6c4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.104+0000 7f3f6bfff640 1 -- 172.21.15.94:0/1297661468 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70004230 con 0x56477e0bc6b0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.104+0000 7f3f86717640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 0x7f3f6c0b4240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x56477e47fab0 tx=0x7f3f78004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.104+0000 7f3f6bfff640 1 -- 172.21.15.94:0/1297661468 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f700043d0 con 0x56477e0bc6b0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.104+0000 7f3f6bfff640 1 -- 172.21.15.94:0/1297661468 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70004c10 con 0x56477e0bc6b0 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- 172.21.15.94:0/1297661468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 msgr2=0x56477e47fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e47fc70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009c00 tx=0x7f3f70011810 comp rx=0 tx=0).stop 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- 172.21.15.94:0/1297661468 shutdown_connections 2024-04-25T13:57:41.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e47fc70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e47efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 --2- 172.21.15.94:0/1297661468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- 172.21.15.94:0/1297661468 >> 172.21.15.94:0/1297661468 conn(0x56477e0d14e0 msgr2=0x56477e47a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- 172.21.15.94:0/1297661468 shutdown_connections 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- 172.21.15.94:0/1297661468 wait complete. 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 -- start start 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.105+0000 7f3f8f020740 1 Processor -- start 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e4888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e488e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56477e480780 con 0x56477e0dd390 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56477e36ad10 con 0x56477e0bc6b0 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56477e480900 con 0x56477e0b44d0 2024-04-25T13:57:41.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f8f020740 10 client.0 _ll_register_callbacks cb 0x56477e397210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e488e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85f16640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e4888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44612/0 (socket says 172.21.15.94:44612) 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f86717640 1 -- 172.21.15.94:0/106827205 learned_addr learned my addr 172.21.15.94:0/106827205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 -- 172.21.15.94:0/106827205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 msgr2=0x56477e4888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e4888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 -- 172.21.15.94:0/106827205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 msgr2=0x56477e48fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 -- 172.21.15.94:0/106827205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56477e1bf7d0 con 0x56477e0bc6b0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f86717640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85f16640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e4888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f85715640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e488e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f70011d10 tx=0x7f3f70019ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42fe370373d681a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.106+0000 7f3f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:57:41.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70028020 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f3f70024620 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f700043d0 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f70033050 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 10 client.30686 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56477e0f53f0 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f700043d0 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f3f70018070 con 0x56477e0bc6b0 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 10 client.30686 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 10 client.30686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 20 client.30686 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e09bf60 0x56477e496070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 4 client.30686 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56477e3f1990 con 0x56477e09bf60 2024-04-25T13:57:41.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.107+0000 7f3f85f16640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e09bf60 0x56477e496070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.108+0000 7f3f697fa640 10 client.30686 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:41.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.108+0000 7f3f85f16640 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e09bf60 0x56477e496070 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc21faf2e540451f server_cookie=5f815f05cf06f24c in_seq=0 out_seq=0 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f697fa640 1 -- 172.21.15.94:0/106827205 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56477e3f1990 con 0x56477e09bf60 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f697fa640 10 client.30686 handle_command_reply: tid=0 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 client.30686 shutdown 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 msgr2=0x56477e488e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e488e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f70011d10 tx=0x7f3f70019ad0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 shutdown_connections 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e09bf60 0x56477e496070 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e48fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e488e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 --2- 172.21.15.94:0/106827205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0b44d0 0x56477e4888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 >> 172.21.15.94:0/106827205 conn(0x56477e0d14e0 msgr2=0x56477e47d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f697fa640 0 client.30686 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 shutdown_connections 2024-04-25T13:57:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 1 -- 172.21.15.94:0/106827205 wait complete. 2024-04-25T13:57:41.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.111+0000 7f3f8f020740 20 client.30686 trim_cache size 0 max 16384 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 Processor -- start 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 -- start start 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 0x56477e479c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0bc6b0 0x56477e48fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801a9310 con 0x56477e0dd390 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801aaeb0 con 0x56477e0b44d0 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f8f020740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801a99e0 con 0x56477e0bc6b0 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44620/0 (socket says 172.21.15.94:44620) 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 -- 172.21.15.94:0/2356147425 learned_addr learned my addr 172.21.15.94:0/2356147425 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f85f16640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 0x56477e479c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f85715640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0bc6b0 0x56477e48fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 -- 172.21.15.94:0/2356147425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0bc6b0 msgr2=0x56477e48fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0bc6b0 0x56477e48fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 -- 172.21.15.94:0/2356147425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 msgr2=0x56477e479c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 0x56477e479c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f86717640 1 -- 172.21.15.94:0/2356147425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477e0b50f0 con 0x56477e0dd390 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.131+0000 7f3f85f16640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 0x56477e479c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.132+0000 7f3f86717640 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f3f780099b0 tx=0x7f3f78018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6ff636e15ec694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.132+0000 7f3f6b7fe640 1 -- 172.21.15.94:0/2356147425 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7801fa00 con 0x56477e0dd390 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.132+0000 7f3f6b7fe640 1 -- 172.21.15.94:0/2356147425 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7801fba0 con 0x56477e0dd390 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.132+0000 7f3f6b7fe640 1 -- 172.21.15.94:0/2356147425 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7802a600 con 0x56477e0dd390 2024-04-25T13:57:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- 172.21.15.94:0/2356147425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 msgr2=0x56477e47a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f3f780099b0 tx=0x7f3f78018000 comp rx=0 tx=0).stop 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- 172.21.15.94:0/2356147425 shutdown_connections 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0dd390 0x56477e47a150 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0bc6b0 0x56477e48fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- 172.21.15.94:0/2356147425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0b44d0 0x56477e479c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- 172.21.15.94:0/2356147425 >> 172.21.15.94:0/2356147425 conn(0x56477e0d14e0 msgr2=0x56477e483620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- 172.21.15.94:0/2356147425 shutdown_connections 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- 172.21.15.94:0/2356147425 wait complete. 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 -- start start 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 Processor -- start 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.133+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e48c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 0x56477e48f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801aa7e0 con 0x56477e0b44d0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801aa0b0 con 0x56477e0bc6b0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801a72f0 con 0x56477e0dd390 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 10 client.0 _ll_register_callbacks cb 0x7f3f78009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44630/0 (socket says 172.21.15.94:44630) 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 -- 172.21.15.94:0/3639796325 learned_addr learned my addr 172.21.15.94:0/3639796325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f86717640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 0x56477e48f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85715640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e48c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 -- 172.21.15.94:0/3639796325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 msgr2=0x56477e48f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 0x56477e48f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 -- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 msgr2=0x56477e48c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e48c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 -- 172.21.15.94:0/3639796325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56477e3a44b0 con 0x56477e0b44d0 2024-04-25T13:57:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f86717640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 0x56477e48f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f85f16640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c031500 tx=0x7f3f7c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0adf65ff778f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c049040 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f3f7c1010e0 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7801fba0 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c0c6e20 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 10 client.21264 fetch_fsmap learned FSMap version 36 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.134+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56477e4fca40 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7801fba0 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f3f7c0cfdf0 con 0x56477e0b44d0 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 10 client.21264 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 10 client.21264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 20 client.21264 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:41.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e0713a0 0x56477e4ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 4 client.21264 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:57:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56477e3eb800 con 0x56477e0713a0 2024-04-25T13:57:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.135+0000 7f3f85715640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e0713a0 0x56477e4ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.136+0000 7f3f697fa640 10 client.21264 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.136+0000 7f3f85715640 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e0713a0 0x56477e4ff3d0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9eeac7ac16a410 server_cookie=fe0accf77605bcc3 in_seq=0 out_seq=0 2024-04-25T13:57:41.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 52782 19.25 MB/sec execute 23 sec latency 61.982 ms 2024-04-25T13:57:41.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f697fa640 1 -- 172.21.15.94:0/3639796325 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56477e3eb800 con 0x56477e0713a0 2024-04-25T13:57:41.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f697fa640 10 client.21264 handle_command_reply: tid=0 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 client.21264 shutdown 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 msgr2=0x56477e48bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c031500 tx=0x7f3f7c009620 comp rx=0 tx=0).stop 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 shutdown_connections 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56477e0713a0 0x56477e4ff3d0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56477e0dd390 0x56477e48f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56477e0bc6b0 0x56477e48c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 --2- 172.21.15.94:0/3639796325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56477e0b44d0 0x56477e48bd20 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f697fa640 0 client.21264 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 >> 172.21.15.94:0/3639796325 conn(0x56477e0d14e0 msgr2=0x56477e479c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 shutdown_connections 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 1 -- 172.21.15.94:0/3639796325 wait complete. 2024-04-25T13:57:41.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8f020740 20 client.21264 trim_cache size 0 max 16384 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (405 inodes in the stack)", 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "6beaebc5-976f-4f58-992e-b3b88da73e52": { 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "6beaebc5-976f-4f58-992e-b3b88da73e52", 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:57:41.144 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:57:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.140+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 msgr2=0x7f3f6c0b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 0x7f3f6c0b4240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x56477e47fab0 tx=0x7f3f78004ab0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 msgr2=0x7f3f80198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006010 tx=0x7f3f7c004270 comp rx=0 tx=0).stop 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 shutdown_connections 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3f6c0b1d30 0x7f3f6c0b4240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8019e1d0 0x7f3f801a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f80108a10 0x7f3f80199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 --2- 172.21.15.94:0/3477954325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f80103ee0 0x7f3f80198d50 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 >> 172.21.15.94:0/3477954325 conn(0x7f3f800fd340 msgr2=0x7f3f800ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 shutdown_connections 2024-04-25T13:57:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:57:41.141+0000 7f3f8817a640 1 -- 172.21.15.94:0/3477954325 wait complete. 2024-04-25T13:57:41.155 INFO:tasks.cephfs.filesystem:scrub status for tag:6beaebc5-976f-4f58-992e-b3b88da73e52 - {'path': '/', 'tag': '6beaebc5-976f-4f58-992e-b3b88da73e52', 'options': 'recursive,force'} 2024-04-25T13:57:42.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 53062 19.04 MB/sec execute 24 sec latency 138.324 ms 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: pgmap v827: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:57:42.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:42 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: pgmap v827: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-04-25T13:57:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:57:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:57:42.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:42 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: pgmap v827: 97 pgs: 97 active+clean; 253 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:57:42.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:42 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:43.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 53402 19.20 MB/sec execute 25 sec latency 90.756 ms 2024-04-25T13:57:43.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:43.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:57:44.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 53696 18.89 MB/sec execute 26 sec latency 219.669 ms 2024-04-25T13:57:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:44 smithi137 ceph-mon[32568]: pgmap v828: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-04-25T13:57:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:44 smithi094 ceph-mon[29195]: pgmap v828: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-04-25T13:57:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:44 smithi138 ceph-mon[32977]: pgmap v828: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-04-25T13:57:45.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 54086 18.92 MB/sec execute 27 sec latency 124.794 ms 2024-04-25T13:57:46.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 54453 18.85 MB/sec execute 28 sec latency 69.201 ms 2024-04-25T13:57:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:46 smithi137 ceph-mon[32568]: pgmap v829: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-04-25T13:57:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:46 smithi094 ceph-mon[29195]: pgmap v829: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-04-25T13:57:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:46 smithi138 ceph-mon[32977]: pgmap v829: 97 pgs: 97 active+clean; 250 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-04-25T13:57:47.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 54820 19.02 MB/sec execute 29 sec latency 72.035 ms 2024-04-25T13:57:47.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:47 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:47.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:47 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:47.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:47 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:57:48.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 55241 19.10 MB/sec execute 30 sec latency 128.059 ms 2024-04-25T13:57:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:48 smithi137 ceph-mon[32568]: pgmap v830: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-04-25T13:57:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:48 smithi094 ceph-mon[29195]: pgmap v830: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-04-25T13:57:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:48 smithi138 ceph-mon[32977]: pgmap v830: 97 pgs: 97 active+clean; 254 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 558 op/s 2024-04-25T13:57:49.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 55582 19.20 MB/sec execute 31 sec latency 134.755 ms 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 -- 172.21.15.94:0/2049409741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 msgr2=0x7fb4ec104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/2049409741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec104090 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009900 tx=0x7fb4d402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 -- 172.21.15.94:0/2049409741 shutdown_connections 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/2049409741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec108980 0x7fb4ec10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/2049409741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec104090 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/2049409741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec102a40 0x7fb4ec103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 -- 172.21.15.94:0/2049409741 >> 172.21.15.94:0/2049409741 conn(0x7fb4ec0fdc30 msgr2=0x7fb4ec1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 -- 172.21.15.94:0/2049409741 shutdown_connections 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.007+0000 7fb4f13a8640 1 -- 172.21.15.94:0/2049409741 wait complete. 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 Processor -- start 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 -- start start 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec102a40 0x7fb4ec19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec108980 0x7fb4ec1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec10e570 con 0x7fb4ec103c80 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec10e3f0 con 0x7fb4ec108980 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4f13a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec10e6f0 con 0x7fb4ec102a40 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57908/0 (socket says 172.21.15.94:57908) 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 -- 172.21.15.94:0/791793002 learned_addr learned my addr 172.21.15.94:0/791793002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec102a40 0x7fb4ec19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec108980 0x7fb4ec1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 -- 172.21.15.94:0/791793002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec102a40 msgr2=0x7fb4ec19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec102a40 0x7fb4ec19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 -- 172.21.15.94:0/791793002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec108980 msgr2=0x7fb4ec1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec108980 0x7fb4ec1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.008+0000 7fb4e3fff640 1 -- 172.21.15.94:0/791793002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec1a1d20 con 0x7fb4ec103c80 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4006010 tx=0x7fb4d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4e8ff9640 1 -- 172.21.15.94:0/791793002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d403f030 con 0x7fb4ec103c80 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4e8ff9640 1 -- 172.21.15.94:0/791793002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d4031910 con 0x7fb4ec103c80 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec1a1fb0 con 0x7fb4ec103c80 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4e8ff9640 1 -- 172.21.15.94:0/791793002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4035070 con 0x7fb4ec103c80 2024-04-25T13:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.009+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec1a23c0 con 0x7fb4ec103c80 2024-04-25T13:57:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 Processor -- start 2024-04-25T13:57:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 -- start start 2024-04-25T13:57:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4e8ff9640 1 -- 172.21.15.94:0/791793002 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb4d4042030 con 0x7fb4ec103c80 2024-04-25T13:57:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657531d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665716c4d0 0x556657539200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556657493460 con 0x55665716c4d0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55665744d590 con 0x5566571642f0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4f8251740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55665748f420 con 0x55665718d1b0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4e3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665716c4d0 0x556657539200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4eaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4eb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657531d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4eaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35314/0 (socket says 172.21.15.94:35314) 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.012+0000 7fb4eaffd640 1 -- 172.21.15.94:0/245763238 learned_addr learned my addr 172.21.15.94:0/245763238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 -- 172.21.15.94:0/245763238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 msgr2=0x556657531d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657531d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 -- 172.21.15.94:0/245763238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665716c4d0 msgr2=0x556657539200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665716c4d0 0x556657539200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 -- 172.21.15.94:0/245763238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665745a0d0 con 0x5566571642f0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657531d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4e8ff9640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 0x7fb4d80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009c60 tx=0x7fb4dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4181a76449ea42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 0x7fb4d80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4e8ff9640 1 -- 172.21.15.94:0/791793002 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb4d40ff1a0 con 0x7fb4ec103c80 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4e17fa640 1 -- 172.21.15.94:0/245763238 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc004220 con 0x5566571642f0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4e17fa640 1 -- 172.21.15.94:0/245763238 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc0043c0 con 0x5566571642f0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.013+0000 7fb4e17fa640 1 -- 172.21.15.94:0/245763238 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc004d30 con 0x5566571642f0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 -- 172.21.15.94:0/245763238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 msgr2=0x556657532300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009c60 tx=0x7fb4dc011810 comp rx=0 tx=0).stop 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 0x7fb4d80b1ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x556657532f50 tx=0x7fb4e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 -- 172.21.15.94:0/245763238 shutdown_connections 2024-04-25T13:57:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665716c4d0 0x556657539200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566571642f0 0x556657532300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 --2- 172.21.15.94:0/245763238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657531d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 -- 172.21.15.94:0/245763238 >> 172.21.15.94:0/245763238 conn(0x556657181300 msgr2=0x556657529300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 -- 172.21.15.94:0/245763238 shutdown_connections 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.014+0000 7fb4f8251740 1 -- 172.21.15.94:0/245763238 wait complete. 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 -- start start 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 Processor -- start 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556657539c50 con 0x55665718d1b0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55665741ab60 con 0x55665716c4d0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556657539dd0 con 0x5566571642f0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4f8251740 10 client.0 _ll_register_callbacks cb 0x55665743f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57926/0 (socket says 172.21.15.94:57926) 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 -- 172.21.15.94:0/116415357 learned_addr learned my addr 172.21.15.94:0/116415357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 -- 172.21.15.94:0/116415357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 msgr2=0x55665753ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 -- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 msgr2=0x55665753bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4eb7fe640 1 -- 172.21.15.94:0/116415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566571a5210 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.015+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4003c40 tx=0x7fb4e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d14b8bc8298709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400fe60 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb4e40045c0 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4dc0043c0 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4e40047c0 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 10 client.21276 fetch_fsmap learned FSMap version 37 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556657271620 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc0043c0 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e4026070 con 0x55665718d1b0 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 10 client.21276 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:57:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 10 client.21276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 20 client.21276 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55665714bd80 0x556657549a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 4 client.21276 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.016+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566574a1070 con 0x55665714bd80 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.017+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55665714bd80 0x556657549a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.017+0000 7fb4d27fc640 10 client.21276 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.017+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55665714bd80 0x556657549a50 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d678ed9160898d0b server_cookie=c1b162ac42c7e0e3 in_seq=0 out_seq=0 2024-04-25T13:57:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4d27fc640 1 -- 172.21.15.94:0/116415357 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566574a1070 con 0x55665714bd80 2024-04-25T13:57:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4d27fc640 10 client.21276 handle_command_reply: tid=0 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 client.21276 shutdown 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 msgr2=0x556657542f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4003c40 tx=0x7fb4e4004ab0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 shutdown_connections 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55665714bd80 0x556657549a50 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x556657542f00 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 --2- 172.21.15.94:0/116415357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4d27fc640 0 client.21276 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 >> 172.21.15.94:0/116415357 conn(0x556657181300 msgr2=0x55665752c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 shutdown_connections 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 1 -- 172.21.15.94:0/116415357 wait complete. 2024-04-25T13:57:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.024+0000 7fb4f8251740 20 client.21276 trim_cache size 0 max 16384 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.039+0000 7fb4f8251740 1 Processor -- start 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.039+0000 7fb4f8251740 1 -- start start 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x556657532240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657535ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1a92a0 con 0x5566571642f0 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1aade0 con 0x55665716c4d0 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4f8251740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1a9970 con 0x55665718d1b0 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57932/0 (socket says 172.21.15.94:57932) 2024-04-25T13:57:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 -- 172.21.15.94:0/3342361185 learned_addr learned my addr 172.21.15.94:0/3342361185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eb7fe640 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657535ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x556657532240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 -- 172.21.15.94:0/3342361185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 msgr2=0x556657535ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657535ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 -- 172.21.15.94:0/3342361185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 msgr2=0x556657532240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x556657532240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 -- 172.21.15.94:0/3342361185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665753c660 con 0x5566571642f0 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00a010 tx=0x7fb4dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f48b6cc008fff2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4e0ff9640 1 -- 172.21.15.94:0/3342361185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc01d040 con 0x5566571642f0 2024-04-25T13:57:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4e0ff9640 1 -- 172.21.15.94:0/3342361185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc02c310 con 0x5566571642f0 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.040+0000 7fb4e0ff9640 1 -- 172.21.15.94:0/3342361185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc003850 con 0x5566571642f0 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 -- 172.21.15.94:0/3342361185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 msgr2=0x556657531d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00a010 tx=0x7fb4dc005850 comp rx=0 tx=0).stop 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 -- 172.21.15.94:0/3342361185 shutdown_connections 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55665718d1b0 0x556657535ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x556657532240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 --2- 172.21.15.94:0/3342361185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566571642f0 0x556657531d00 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.041+0000 7fb4f8251740 1 -- 172.21.15.94:0/3342361185 >> 172.21.15.94:0/3342361185 conn(0x556657181300 msgr2=0x55665753a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- 172.21.15.94:0/3342361185 shutdown_connections 2024-04-25T13:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- 172.21.15.94:0/3342361185 wait complete. 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- start start 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 Processor -- start 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x5566575afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1aa710 con 0x55665718d1b0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1aa040 con 0x55665716c4d0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1a7280 con 0x5566571642f0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4f8251740 10 client.0 _ll_register_callbacks cb 0x7fb4e4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x5566575afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x5566575afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57934/0 (socket says 172.21.15.94:57934) 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4eaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4eb7fe640 1 -- 172.21.15.94:0/1610651881 learned_addr learned my addr 172.21.15.94:0/1610651881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.042+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 -- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 msgr2=0x55665753fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 -- 172.21.15.94:0/1610651881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 msgr2=0x5566575afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x5566575afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 -- 172.21.15.94:0/1610651881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566575b0370 con 0x5566571642f0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4eaffd640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753f630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55665752d530 tx=0x7fb4dc0050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e9ac3ad2b070cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc01d040 con 0x5566571642f0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb4dc005250 con 0x5566571642f0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4dc02c310 con 0x5566571642f0 2024-04-25T13:57:50.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc005450 con 0x5566571642f0 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4f8251740 10 client.30409 fetch_fsmap learned FSMap version 37 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5566575b15e0 con 0x5566571642f0 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.043+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc02c310 con 0x5566571642f0 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e4011d20 con 0x5566571642f0 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 10 client.30409 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 10 client.30409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 20 client.30409 populate_metadata read hostname 'smithi094' 2024-04-25T13:57:50.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5566571211c0 0x5566575b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 4 client.30409 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55665749aee0 con 0x5566571211c0 2024-04-25T13:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5566571211c0 0x5566575b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4d27fc640 10 client.30409 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.044+0000 7fb4e3fff640 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5566571211c0 0x5566575b3db0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbc30c7395dfb70 server_cookie=dc8d8f31b1356d2d in_seq=0 out_seq=0 2024-04-25T13:57:50.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 55927 19.16 MB/sec execute 32 sec latency 81.997 ms 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4d27fc640 1 -- 172.21.15.94:0/1610651881 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55665749aee0 con 0x5566571211c0 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4d27fc640 10 client.30409 handle_command_reply: tid=0 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 client.30409 shutdown 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 msgr2=0x55665753f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753f630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55665752d530 tx=0x7fb4dc0050d0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 shutdown_connections 2024-04-25T13:57:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5566571211c0 0x5566575b3db0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55665718d1b0 0x5566575afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4d27fc640 0 client.30409 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55665716c4d0 0x55665753fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 --2- 172.21.15.94:0/1610651881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566571642f0 0x55665753f630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 >> 172.21.15.94:0/1610651881 conn(0x556657181300 msgr2=0x55665753a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 shutdown_connections 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 1 -- 172.21.15.94:0/1610651881 wait complete. 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f8251740 20 client.30409 trim_cache size 0 max 16384 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:57:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:57:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:57:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:57:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 msgr2=0x7fb4d80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 0x7fb4d80b1ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x556657532f50 tx=0x7fb4e4009500 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 msgr2=0x7fb4ec19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4006010 tx=0x7fb4d40042f0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 shutdown_connections 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb4d80afae0 0x7fb4d80b1ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4ec108980 0x7fb4ec1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4ec103c80 0x7fb4ec19a720 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 --2- 172.21.15.94:0/791793002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4ec102a40 0x7fb4ec19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.620+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 >> 172.21.15.94:0/791793002 conn(0x7fb4ec0fdc30 msgr2=0x7fb4ec10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.621+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 shutdown_connections 2024-04-25T13:57:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:57:50.621+0000 7fb4f13a8640 1 -- 172.21.15.94:0/791793002 wait complete. 2024-04-25T13:57:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:50 smithi137 ceph-mon[32568]: pgmap v831: 97 pgs: 97 active+clean; 252 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-04-25T13:57:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:50 smithi094 ceph-mon[29195]: pgmap v831: 97 pgs: 97 active+clean; 252 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-04-25T13:57:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:50 smithi138 ceph-mon[32977]: pgmap v831: 97 pgs: 97 active+clean; 252 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-04-25T13:57:51.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 56152 18.93 MB/sec execute 33 sec latency 362.949 ms 2024-04-25T13:57:51.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:51.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:51.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:51.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:51.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:51.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:52.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 56527 18.91 MB/sec execute 34 sec latency 117.773 ms 2024-04-25T13:57:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:52 smithi137 ceph-mon[32568]: pgmap v832: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-04-25T13:57:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:52 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:52 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:52.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:52 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:57:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:52 smithi094 ceph-mon[29195]: pgmap v832: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-04-25T13:57:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:52 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:52 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:52 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:57:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:52 smithi138 ceph-mon[32977]: pgmap v832: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 621 op/s 2024-04-25T13:57:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:52 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:52 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:52.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:52 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:57:53.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 56921 19.09 MB/sec execute 35 sec latency 75.093 ms 2024-04-25T13:57:54.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 57310 19.08 MB/sec execute 36 sec latency 210.141 ms 2024-04-25T13:57:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:54 smithi137 ceph-mon[32568]: pgmap v833: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 373 op/s 2024-04-25T13:57:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:54 smithi094 ceph-mon[29195]: pgmap v833: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 373 op/s 2024-04-25T13:57:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:54 smithi138 ceph-mon[32977]: pgmap v833: 97 pgs: 97 active+clean; 261 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 373 op/s 2024-04-25T13:57:55.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 57608 18.91 MB/sec execute 37 sec latency 110.399 ms 2024-04-25T13:57:56.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 58023 19.01 MB/sec execute 38 sec latency 59.320 ms 2024-04-25T13:57:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:56 smithi137 ceph-mon[32568]: pgmap v834: 97 pgs: 97 active+clean; 266 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 621 op/s 2024-04-25T13:57:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:56 smithi094 ceph-mon[29195]: pgmap v834: 97 pgs: 97 active+clean; 266 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 621 op/s 2024-04-25T13:57:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:56 smithi138 ceph-mon[32977]: pgmap v834: 97 pgs: 97 active+clean; 266 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 621 op/s 2024-04-25T13:57:57.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 58408 19.16 MB/sec execute 39 sec latency 78.693 ms 2024-04-25T13:57:58.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 58813 19.14 MB/sec execute 40 sec latency 87.186 ms 2024-04-25T13:57:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:57:58 smithi137 ceph-mon[32568]: pgmap v835: 97 pgs: 97 active+clean; 255 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 541 op/s 2024-04-25T13:57:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:57:58 smithi094 ceph-mon[29195]: pgmap v835: 97 pgs: 97 active+clean; 255 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 541 op/s 2024-04-25T13:57:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:57:58 smithi138 ceph-mon[32977]: pgmap v835: 97 pgs: 97 active+clean; 255 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 541 op/s 2024-04-25T13:57:59.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 59158 19.20 MB/sec execute 41 sec latency 169.475 ms 2024-04-25T13:58:00.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 59486 19.09 MB/sec execute 42 sec latency 147.124 ms 2024-04-25T13:58:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:00 smithi137 ceph-mon[32568]: pgmap v836: 97 pgs: 97 active+clean; 253 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-04-25T13:58:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:00 smithi094 ceph-mon[29195]: pgmap v836: 97 pgs: 97 active+clean; 253 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-04-25T13:58:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:00 smithi138 ceph-mon[32977]: pgmap v836: 97 pgs: 97 active+clean; 253 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-04-25T13:58:01.139 INFO:tasks.workunit.client.0.smithi094.stdout: 10 59890 19.11 MB/sec execute 43 sec latency 94.304 ms 2024-04-25T13:58:02.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 60259 19.27 MB/sec execute 44 sec latency 84.640 ms 2024-04-25T13:58:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:02 smithi137 ceph-mon[32568]: pgmap v837: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 594 op/s 2024-04-25T13:58:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:02 smithi094 ceph-mon[29195]: pgmap v837: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 594 op/s 2024-04-25T13:58:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:02 smithi138 ceph-mon[32977]: pgmap v837: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 594 op/s 2024-04-25T13:58:03.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 60710 19.35 MB/sec execute 45 sec latency 61.093 ms 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 -- 172.21.15.94:0/975588489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 msgr2=0x7f4164103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 --2- 172.21.15.94:0/975588489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f4164103730 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f4154009900 tx=0x7f415402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 -- 172.21.15.94:0/975588489 shutdown_connections 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 --2- 172.21.15.94:0/975588489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 0x7f416410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 --2- 172.21.15.94:0/975588489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 0x7f4164104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 --2- 172.21.15.94:0/975588489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f4164103730 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 -- 172.21.15.94:0/975588489 >> 172.21.15.94:0/975588489 conn(0x7f41640fdc20 msgr2=0x7f41641000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 -- 172.21.15.94:0/975588489 shutdown_connections 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.727+0000 7f4169cfa640 1 -- 172.21.15.94:0/975588489 wait complete. 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 Processor -- start 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 -- start start 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 0x7f416419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 0x7f41641a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f416410f660 con 0x7f4164102a30 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f416410f4e0 con 0x7f4164108970 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4169cfa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f416410f7e0 con 0x7f4164103c70 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47266/0 (socket says 172.21.15.94:47266) 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f41637fe640 1 -- 172.21.15.94:0/2205419971 learned_addr learned my addr 172.21.15.94:0/2205419971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.728+0000 7f4163fff640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 0x7f41641a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 0x7f416419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 -- 172.21.15.94:0/2205419971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 msgr2=0x7f416419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 0x7f416419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 -- 172.21.15.94:0/2205419971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 msgr2=0x7f41641a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 0x7f41641a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 -- 172.21.15.94:0/2205419971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41641a1d70 con 0x7f4164102a30 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f41637fe640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f4154006010 tx=0x7f4154004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4160ff9640 1 -- 172.21.15.94:0/2205419971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415403f030 con 0x7f4164102a30 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4160ff9640 1 -- 172.21.15.94:0/2205419971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154003aa0 con 0x7f4164102a30 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41641a2000 con 0x7f4164102a30 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4160ff9640 1 -- 172.21.15.94:0/2205419971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154035070 con 0x7f4164102a30 2024-04-25T13:58:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.729+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41641a24f0 con 0x7f4164102a30 2024-04-25T13:58:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 Processor -- start 2024-04-25T13:58:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 -- start start 2024-04-25T13:58:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4160ff9640 1 -- 172.21.15.94:0/2205419971 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4154042030 con 0x7f4164102a30 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae4196e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415a52f0 0x55ae4196eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae418d4460 con 0x55ae415ad4d0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae4188e590 con 0x55ae415a52f0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4170ba8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae418d0420 con 0x55ae415ce1b0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4162ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4162ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47280/0 (socket says 172.21.15.94:47280) 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f41637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415a52f0 0x55ae4196eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4162ffd640 1 -- 172.21.15.94:0/2905400152 learned_addr learned my addr 172.21.15.94:0/2905400152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.732+0000 7f4163fff640 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae4196e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 -- 172.21.15.94:0/2905400152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 msgr2=0x55ae4196e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae4196e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 -- 172.21.15.94:0/2905400152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415a52f0 msgr2=0x55ae4196eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415a52f0 0x55ae4196eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 -- 172.21.15.94:0/2905400152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4189b0d0 con 0x55ae415ad4d0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f415800cbb0 tx=0x7f415800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51fe7a42ec5f0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4160ff9640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 0x7f414c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4149ffb640 1 -- 172.21.15.94:0/2905400152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158010630 con 0x55ae415ad4d0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4163fff640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 0x7f414c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4149ffb640 1 -- 172.21.15.94:0/2905400152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4158010c80 con 0x55ae415ad4d0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4149ffb640 1 -- 172.21.15.94:0/2905400152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415801ab60 con 0x55ae415ad4d0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.733+0000 7f4160ff9640 1 -- 172.21.15.94:0/2205419971 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f41540ff350 con 0x7f4164102a30 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4163fff640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 0x7f414c0b1ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55ae4196fb40 tx=0x7f415c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- 172.21.15.94:0/2905400152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 msgr2=0x55ae41975df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f415800cbb0 tx=0x7f415800e720 comp rx=0 tx=0).stop 2024-04-25T13:58:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- 172.21.15.94:0/2905400152 shutdown_connections 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ad4d0 0x55ae41975df0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415a52f0 0x55ae4196eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2905400152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae4196e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- 172.21.15.94:0/2905400152 >> 172.21.15.94:0/2905400152 conn(0x55ae415c2300 msgr2=0x55ae4196a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- 172.21.15.94:0/2905400152 shutdown_connections 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- 172.21.15.94:0/2905400152 wait complete. 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.734+0000 7f4170ba8740 1 -- start start 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 Processor -- start 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415a52f0 0x55ae419786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae41978be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae41976840 con 0x55ae415ce1b0 2024-04-25T13:58:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae4185bb60 con 0x55ae415ad4d0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae419769c0 con 0x55ae415a52f0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4170ba8740 10 client.0 _ll_register_callbacks cb 0x55ae418802b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47296/0 (socket says 172.21.15.94:47296) 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 -- 172.21.15.94:0/4107311446 learned_addr learned my addr 172.21.15.94:0/4107311446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4162ffd640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae41978be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f41637fe640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415a52f0 0x55ae419786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 -- 172.21.15.94:0/4107311446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415a52f0 msgr2=0x55ae419786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415a52f0 0x55ae419786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 -- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 msgr2=0x55ae41978be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae41978be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.735+0000 7f4163fff640 1 -- 172.21.15.94:0/4107311446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae415e6210 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4163fff640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f415c009cd0 tx=0x7f415c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7c6e73a87f1178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c00f830 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4158010c80 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f415c00f9d0 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f415c00fbd0 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4170ba8740 10 client.21294 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ae416b2620 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.736+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4158010c80 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f415c026070 con 0x55ae415ce1b0 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 10 client.21294 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 10 client.21294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 20 client.21294 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae4158cd80 0x55ae419868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 4 client.21294 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae418e2070 con 0x55ae4158cd80 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f41637fe640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae4158cd80 0x55ae419868c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f4142ffd640 10 client.21294 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.737+0000 7f41637fe640 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae4158cd80 0x55ae419868c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a4c0ddb054d638 server_cookie=ec4ae96023175a43 in_seq=0 out_seq=0 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4142ffd640 1 -- 172.21.15.94:0/4107311446 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae418e2070 con 0x55ae4158cd80 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4142ffd640 10 client.21294 handle_command_reply: tid=0 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 client.21294 shutdown 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 msgr2=0x55ae4197fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f415c009cd0 tx=0x7f415c004ab0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 shutdown_connections 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae4158cd80 0x55ae419868c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415ce1b0 0x55ae4197fad0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae41978be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 --2- 172.21.15.94:0/4107311446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415a52f0 0x55ae419786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 >> 172.21.15.94:0/4107311446 conn(0x55ae415c2300 msgr2=0x55ae41969250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4142ffd640 0 client.21294 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 shutdown_connections 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 1 -- 172.21.15.94:0/4107311446 wait complete. 2024-04-25T13:58:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.739+0000 7f4170ba8740 20 client.21294 trim_cache size 0 max 16384 2024-04-25T13:58:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 Processor -- start 2024-04-25T13:58:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 -- start start 2024-04-25T13:58:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ad4d0 0x55ae41973050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 0x55ae41976d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41641ab4f0 con 0x55ae415a52f0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f41641ad030 con 0x55ae415ce1b0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4170ba8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41641abbc0 con 0x55ae415ad4d0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47304/0 (socket says 172.21.15.94:47304) 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 -- 172.21.15.94:0/3194932833 learned_addr learned my addr 172.21.15.94:0/3194932833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4162ffd640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ad4d0 0x55ae41973050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4163fff640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 0x55ae41976d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 -- 172.21.15.94:0/3194932833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ad4d0 msgr2=0x55ae41973050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ad4d0 0x55ae41973050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 -- 172.21.15.94:0/3194932833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 msgr2=0x55ae41976d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 0x55ae41976d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 -- 172.21.15.94:0/3194932833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4196e4f0 con 0x55ae415a52f0 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f4163fff640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 0x55ae41976d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41637fe640 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f4154031500 tx=0x7f41540ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409255c40c315f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.755+0000 7f41497fa640 1 -- 172.21.15.94:0/3194932833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154049040 con 0x55ae415a52f0 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f41497fa640 1 -- 172.21.15.94:0/3194932833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41540c72a0 con 0x55ae415a52f0 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f41497fa640 1 -- 172.21.15.94:0/3194932833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41540c7790 con 0x55ae415a52f0 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 -- 172.21.15.94:0/3194932833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 msgr2=0x55ae41972b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f4154031500 tx=0x7f41540ea040 comp rx=0 tx=0).stop 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 -- 172.21.15.94:0/3194932833 shutdown_connections 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ce1b0 0x55ae41976d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ad4d0 0x55ae41973050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 --2- 172.21.15.94:0/3194932833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae41972b10 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 -- 172.21.15.94:0/3194932833 >> 172.21.15.94:0/3194932833 conn(0x55ae415c2300 msgr2=0x55ae41971ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 -- 172.21.15.94:0/3194932833 shutdown_connections 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.756+0000 7f4170ba8740 1 -- 172.21.15.94:0/3194932833 wait complete. 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 -- start start 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 Processor -- start 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae4197c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae419ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41641ac960 con 0x55ae415a52f0 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f41641ac290 con 0x55ae415ad4d0 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41641a94d0 con 0x55ae415ce1b0 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4170ba8740 10 client.0 _ll_register_callbacks cb 0x7f415c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f41637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47312/0 (socket says 172.21.15.94:47312) 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f41637fe640 1 -- 172.21.15.94:0/2869844449 learned_addr learned my addr 172.21.15.94:0/2869844449 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae4197c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.757+0000 7f4163fff640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae419ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 -- 172.21.15.94:0/2869844449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 msgr2=0x55ae419ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae419ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 -- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 msgr2=0x55ae4197c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae4197c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 -- 172.21.15.94:0/2869844449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4197f290 con 0x55ae415a52f0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f41637fe640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f41540fd120 tx=0x7f4154000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82516ad55535f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154049070 con 0x55ae415a52f0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f415403bd90 con 0x55ae415a52f0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41540c72a0 con 0x55ae415a52f0 2024-04-25T13:58:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f41540c6f00 con 0x55ae415a52f0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 10 client.21300 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ae419ee0d0 con 0x55ae415a52f0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41540c72a0 con 0x55ae415a52f0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4154003c50 con 0x55ae415a52f0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 10 client.21300 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 10 client.21300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 20 client.21300 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae415621c0 0x55ae419f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 4 client.21300 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.758+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae418dbee0 con 0x55ae415621c0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.759+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae415621c0 0x55ae419f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.759+0000 7f4142ffd640 10 client.21300 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:03.759+0000 7f4162ffd640 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae415621c0 0x55ae419f0ae0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678e2155aa2615ae server_cookie=e7c4802c8c67923c in_seq=0 out_seq=0 2024-04-25T13:58:04.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 60940 19.20 MB/sec execute 46 sec latency 221.892 ms 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.489+0000 7f4142ffd640 1 -- 172.21.15.94:0/2869844449 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae418dbee0 con 0x55ae415621c0 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.489+0000 7f4142ffd640 10 client.21300 handle_command_reply: tid=0 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.489+0000 7f4170ba8740 1 client.21300 shutdown 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 msgr2=0x55ae4197bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f41540fd120 tx=0x7f4154000bc0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 shutdown_connections 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ae415621c0 0x55ae419f0ae0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4142ffd640 0 client.21300 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae415ce1b0 0x55ae419ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae415ad4d0 0x55ae4197c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 --2- 172.21.15.94:0/2869844449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae415a52f0 0x55ae4197bd90 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 >> 172.21.15.94:0/2869844449 conn(0x55ae415c2300 msgr2=0x55ae41971ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 shutdown_connections 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 1 -- 172.21.15.94:0/2869844449 wait complete. 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.490+0000 7f4170ba8740 20 client.21300 trim_cache size 0 max 16384 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:58:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:58:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:58:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:58:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 msgr2=0x7f414c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 0x7f414c0b1ff0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55ae4196fb40 tx=0x7f415c009500 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 msgr2=0x7f416419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f4154006010 tx=0x7f4154004270 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 shutdown_connections 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f414c0afae0 0x7f414c0b1ff0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4164108970 0x7f41641a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164103c70 0x7f416419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 --2- 172.21.15.94:0/2205419971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4164102a30 0x7f416419a2a0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 >> 172.21.15.94:0/2205419971 conn(0x7f41640fdc20 msgr2=0x7f4164104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 shutdown_connections 2024-04-25T13:58:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:04.491+0000 7f4169cfa640 1 -- 172.21.15.94:0/2205419971 wait complete. 2024-04-25T13:58:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:04 smithi137 ceph-mon[32568]: pgmap v838: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-04-25T13:58:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:04 smithi094 ceph-mon[29195]: pgmap v838: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-04-25T13:58:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:04 smithi138 ceph-mon[32977]: pgmap v838: 97 pgs: 97 active+clean; 257 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-04-25T13:58:05.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 61200 19.04 MB/sec execute 47 sec latency 364.854 ms 2024-04-25T13:58:06.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 61572 19.05 MB/sec execute 48 sec latency 106.548 ms 2024-04-25T13:58:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:06 smithi137 ceph-mon[32568]: pgmap v839: 97 pgs: 97 active+clean; 260 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 561 op/s 2024-04-25T13:58:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:06 smithi094 ceph-mon[29195]: pgmap v839: 97 pgs: 97 active+clean; 260 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 561 op/s 2024-04-25T13:58:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:06 smithi138 ceph-mon[32977]: pgmap v839: 97 pgs: 97 active+clean; 260 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.5 MiB/s wr, 561 op/s 2024-04-25T13:58:07.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 62004 19.16 MB/sec execute 49 sec latency 59.377 ms 2024-04-25T13:58:08.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 62393 19.17 MB/sec execute 50 sec latency 101.990 ms 2024-04-25T13:58:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:08 smithi137 ceph-mon[32568]: pgmap v840: 97 pgs: 97 active+clean; 262 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-04-25T13:58:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:08 smithi094 ceph-mon[29195]: pgmap v840: 97 pgs: 97 active+clean; 262 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-04-25T13:58:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:08 smithi138 ceph-mon[32977]: pgmap v840: 97 pgs: 97 active+clean; 262 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 402 op/s 2024-04-25T13:58:09.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 62730 19.23 MB/sec execute 51 sec latency 196.076 ms 2024-04-25T13:58:10.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 63120 19.31 MB/sec execute 52 sec latency 98.155 ms 2024-04-25T13:58:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:10 smithi137 ceph-mon[32568]: pgmap v841: 97 pgs: 97 active+clean; 263 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 424 op/s 2024-04-25T13:58:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:10 smithi094 ceph-mon[29195]: pgmap v841: 97 pgs: 97 active+clean; 263 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 424 op/s 2024-04-25T13:58:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:10 smithi138 ceph-mon[32977]: pgmap v841: 97 pgs: 97 active+clean; 263 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 424 op/s 2024-04-25T13:58:11.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 63512 19.18 MB/sec execute 53 sec latency 78.779 ms 2024-04-25T13:58:11.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:58:11.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:58:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.274+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/624500419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc103c20 msgr2=0x7fb2cc104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.274+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/624500419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc103c20 0x7fb2cc104030 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009900 tx=0x7fb2bc02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.274+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/624500419 shutdown_connections 2024-04-25T13:58:11.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/624500419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc108920 0x7fb2cc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/624500419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc103c20 0x7fb2cc104030 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/624500419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc1029e0 0x7fb2cc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/624500419 >> 172.21.15.94:0/624500419 conn(0x7fb2cc0fdbd0 msgr2=0x7fb2cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/624500419 shutdown_connections 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/624500419 wait complete. 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 Processor -- start 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- start start 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc1029e0 0x7fb2cc06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc103c20 0x7fb2cc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10e500 con 0x7fb2cc108920 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10e380 con 0x7fb2cc1029e0 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.275+0000 7fb2d3ad9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10e680 con 0x7fb2cc103c20 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47334/0 (socket says 172.21.15.94:47334) 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 -- 172.21.15.94:0/1044877630 learned_addr learned my addr 172.21.15.94:0/1044877630 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d1875640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc1029e0 0x7fb2cc06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d1074640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc103c20 0x7fb2cc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 -- 172.21.15.94:0/1044877630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc103c20 msgr2=0x7fb2cc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc103c20 0x7fb2cc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 -- 172.21.15.94:0/1044877630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc1029e0 msgr2=0x7fb2cc06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc1029e0 0x7fb2cc06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 -- 172.21.15.94:0/1044877630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1a2080 con 0x7fb2cc108920 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d2076640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400efd0 tx=0x7fb2c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2c2ffd640 1 -- 172.21.15.94:0/1044877630 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4003b70 con 0x7fb2cc108920 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2c2ffd640 1 -- 172.21.15.94:0/1044877630 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c4003d10 con 0x7fb2cc108920 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2c2ffd640 1 -- 172.21.15.94:0/1044877630 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40108a0 con 0x7fb2cc108920 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1a2370 con 0x7fb2cc108920 2024-04-25T13:58:11.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.276+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc1a2870 con 0x7fb2cc108920 2024-04-25T13:58:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.279+0000 7fb2da981740 1 Processor -- start 2024-04-25T13:58:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.279+0000 7fb2c2ffd640 1 -- 172.21.15.94:0/1044877630 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb2c4004020 con 0x7fb2cc108920 2024-04-25T13:58:11.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.279+0000 7fb2da981740 1 -- start start 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b219a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556e4b17ed80 con 0x556e4ae4f4d0 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556e4b1388b0 con 0x556e4ae576b0 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2da981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e4b17ad40 con 0x556e4ae78390 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40028/0 (socket says 172.21.15.94:40028) 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47346/0 (socket says 172.21.15.94:47346) 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 -- 172.21.15.94:0/2324356637 learned_addr learned my addr 172.21.15.94:0/2324356637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b219a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2c2ffd640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 0x7fb2b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d2076640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 0x7fb2b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 -- 172.21.15.94:0/2324356637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 msgr2=0x556e4b219a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b219a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 -- 172.21.15.94:0/2324356637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 msgr2=0x556e4b219fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1074640 1 -- 172.21.15.94:0/2324356637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4b109a40 con 0x556e4ae576b0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2d1875640 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.280+0000 7fb2c2ffd640 1 -- 172.21.15.94:0/1044877630 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb2c40df050 con 0x7fb2cc108920 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.281+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0096c0 tx=0x7fb2bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f289343851721be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.281+0000 7fb2d2076640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 0x7fb2b80b2000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40a35b0 tx=0x7fb2c4090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.281+0000 7fb2b7fff640 1 -- 172.21.15.94:0/2324356637 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc035040 con 0x556e4ae576b0 2024-04-25T13:58:11.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.281+0000 7fb2b7fff640 1 -- 172.21.15.94:0/2324356637 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc004030 con 0x556e4ae576b0 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.281+0000 7fb2b7fff640 1 -- 172.21.15.94:0/2324356637 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003850 con 0x556e4ae576b0 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- 172.21.15.94:0/2324356637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 msgr2=0x556e4b21ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0096c0 tx=0x7fb2bc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- 172.21.15.94:0/2324356637 shutdown_connections 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b21ac70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b219fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 --2- 172.21.15.94:0/2324356637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b219a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- 172.21.15.94:0/2324356637 >> 172.21.15.94:0/2324356637 conn(0x556e4ae6c4e0 msgr2=0x556e4b215270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- 172.21.15.94:0/2324356637 shutdown_connections 2024-04-25T13:58:11.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- 172.21.15.94:0/2324356637 wait complete. 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 -- start start 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.282+0000 7fb2da981740 1 Processor -- start 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b223e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b22ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556e4b21b720 con 0x556e4ae4f4d0 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556e4b105d10 con 0x556e4ae576b0 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e4b21b8a0 con 0x556e4ae78390 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2da981740 10 client.0 _ll_register_callbacks cb 0x556e4b132210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b223e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47354/0 (socket says 172.21.15.94:47354) 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 -- 172.21.15.94:0/2759281022 learned_addr learned my addr 172.21.15.94:0/2759281022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b22ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 -- 172.21.15.94:0/2759281022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 msgr2=0x556e4b22ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b22ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 -- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 msgr2=0x556e4b223e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b223e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 -- 172.21.15.94:0/2759281022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4af5a7d0 con 0x556e4ae4f4d0 2024-04-25T13:58:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1875640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b223e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2d1074640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc001eb0 tx=0x7fb2bc032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d08d1eb07f9a704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.283+0000 7fb2b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc035040 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc004030 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb2bc0037f0 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb2bc0039f0 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 10 client.21312 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556e4ae903f0 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc004030 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb2bc04bba0 con 0x556e4ae4f4d0 2024-04-25T13:58:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 10 client.21312 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 10 client.21312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 20 client.21312 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae36f60 0x556e4b230ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 4 client.21312 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e4b18c990 con 0x556e4ae36f60 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.284+0000 7fb2d1875640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae36f60 0x556e4b230ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.285+0000 7fb2b57fa640 10 client.21312 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.285+0000 7fb2d1875640 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae36f60 0x556e4b230ea0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa05ab958c73d54e server_cookie=8e90366c818b9ea4 in_seq=0 out_seq=0 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2b57fa640 1 -- 172.21.15.94:0/2759281022 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e4b18c990 con 0x556e4ae36f60 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2b57fa640 10 client.21312 handle_command_reply: tid=0 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 client.21312 shutdown 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 msgr2=0x556e4b223940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc001eb0 tx=0x7fb2bc032dd0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 shutdown_connections 2024-04-25T13:58:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae36f60 0x556e4b230ea0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae78390 0x556e4b22ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2b57fa640 0 client.21312 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae576b0 0x556e4b223e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 --2- 172.21.15.94:0/2759281022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae4f4d0 0x556e4b223940 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 >> 172.21.15.94:0/2759281022 conn(0x556e4ae6c4e0 msgr2=0x556e4b218180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 shutdown_connections 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 1 -- 172.21.15.94:0/2759281022 wait complete. 2024-04-25T13:58:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.291+0000 7fb2da981740 20 client.21312 trim_cache size 0 max 16384 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.310+0000 7fb2da981740 1 Processor -- start 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.310+0000 7fb2da981740 1 -- start start 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b2225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ab480 con 0x556e4ae576b0 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1acfc0 con 0x556e4ae78390 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2da981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1abb50 con 0x556e4ae4f4d0 2024-04-25T13:58:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47356/0 (socket says 172.21.15.94:47356) 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 -- 172.21.15.94:0/3361381885 learned_addr learned my addr 172.21.15.94:0/3361381885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1074640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1074640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b2225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 -- 172.21.15.94:0/3361381885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 msgr2=0x556e4b2225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b2225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 -- 172.21.15.94:0/3361381885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 msgr2=0x556e4b22ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 -- 172.21.15.94:0/3361381885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e4ae500f0 con 0x556e4ae576b0 2024-04-25T13:58:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1074640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b2225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2d1875640 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x556e4b219780 tx=0x7fb2c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69643280bcd00efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.311+0000 7fb2b77fe640 1 -- 172.21.15.94:0/3361381885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004050 con 0x556e4ae576b0 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.312+0000 7fb2b77fe640 1 -- 172.21.15.94:0/3361381885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c80054d0 con 0x556e4ae576b0 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.312+0000 7fb2b77fe640 1 -- 172.21.15.94:0/3361381885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004da0 con 0x556e4ae576b0 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.312+0000 7fb2da981740 1 -- 172.21.15.94:0/3361381885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 msgr2=0x556e4b223c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.312+0000 7fb2da981740 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x556e4b219780 tx=0x7fb2c8005f90 comp rx=0 tx=0).stop 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 -- 172.21.15.94:0/3361381885 shutdown_connections 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b223c20 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 --2- 172.21.15.94:0/3361381885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b2225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 -- 172.21.15.94:0/3361381885 >> 172.21.15.94:0/3361381885 conn(0x556e4ae6c4e0 msgr2=0x556e4b223510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 -- 172.21.15.94:0/3361381885 shutdown_connections 2024-04-25T13:58:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 -- 172.21.15.94:0/3361381885 wait complete. 2024-04-25T13:58:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 -- start start 2024-04-25T13:58:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.313+0000 7fb2da981740 1 Processor -- start 2024-04-25T13:58:11.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b229e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ac8f0 con 0x556e4ae576b0 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ac220 con 0x556e4ae78390 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a9460 con 0x556e4ae4f4d0 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1074640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b229e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47362/0 (socket says 172.21.15.94:47362) 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 -- 172.21.15.94:0/4076611784 learned_addr learned my addr 172.21.15.94:0/4076611784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1074640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2da981740 10 client.0 _ll_register_callbacks cb 0x7fb2c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 -- 172.21.15.94:0/4076611784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 msgr2=0x556e4b229e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b229e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 -- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 msgr2=0x556e4b22a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 -- 172.21.15.94:0/4076611784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e4b13f4b0 con 0x556e4ae576b0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2d1875640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8012600 tx=0x7fb2c8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437094381a64cbf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.314+0000 7fb2b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8005230 con 0x556e4ae576b0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fb2c8004b20 con 0x556e4ae576b0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c80054d0 con 0x556e4ae576b0 2024-04-25T13:58:11.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb2c8004d20 con 0x556e4ae576b0 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 10 client.21318 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556e4b2980f0 con 0x556e4ae576b0 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c80054d0 con 0x556e4ae576b0 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb2c8003a10 con 0x556e4ae576b0 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 10 client.21318 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 10 client.21318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 20 client.21318 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae0c3a0 0x556e4b29ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 4 client.21318 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:58:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.315+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e4b186800 con 0x556e4ae0c3a0 2024-04-25T13:58:11.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.316+0000 7fb2d1074640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae0c3a0 0x556e4b29ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.316+0000 7fb2b57fa640 10 client.21318 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:11.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.316+0000 7fb2d1074640 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae0c3a0 0x556e4b29ab10 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cdbe0549b835c2 server_cookie=3389b1625b936623 in_seq=0 out_seq=0 2024-04-25T13:58:11.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2b57fa640 1 -- 172.21.15.94:0/4076611784 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e4b186800 con 0x556e4ae0c3a0 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2b57fa640 10 client.21318 handle_command_reply: tid=0 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 client.21318 shutdown 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 msgr2=0x556e4b22a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8012600 tx=0x7fb2c8004700 comp rx=0 tx=0).stop 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 shutdown_connections 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556e4ae0c3a0 0x556e4b29ab10 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e4ae78390 0x556e4b22a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e4ae576b0 0x556e4b22a360 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2b57fa640 0 client.21318 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:11.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 --2- 172.21.15.94:0/4076611784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e4ae4f4d0 0x556e4b229e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 >> 172.21.15.94:0/4076611784 conn(0x556e4ae6c4e0 msgr2=0x556e4b2139c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 shutdown_connections 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.320+0000 7fb2da981740 1 -- 172.21.15.94:0/4076611784 wait complete. 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2da981740 20 client.21318 trim_cache size 0 max 16384 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:58:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:58:11.327 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 msgr2=0x7fb2b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 0x7fb2b80b2000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40a35b0 tx=0x7fb2c4090040 comp rx=0 tx=0).stop 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 msgr2=0x7fb2cc1a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400efd0 tx=0x7fb2c400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 shutdown_connections 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2b80afaf0 0x7fb2b80b2000 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2cc108920 0x7fb2cc1a1860 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2cc103c20 0x7fb2cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 --2- 172.21.15.94:0/1044877630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2cc1029e0 0x7fb2cc06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 >> 172.21.15.94:0/1044877630 conn(0x7fb2cc0fdbd0 msgr2=0x7fb2cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 shutdown_connections 2024-04-25T13:58:11.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.321+0000 7fb2d3ad9640 1 -- 172.21.15.94:0/1044877630 wait complete. 2024-04-25T13:58:11.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:58:11.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 -- 172.21.15.94:0/3716825510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 msgr2=0x7fba4c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 --2- 172.21.15.94:0/3716825510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c10be60 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fba44009ca0 tx=0x7fba4402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 -- 172.21.15.94:0/3716825510 shutdown_connections 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 --2- 172.21.15.94:0/3716825510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c10be60 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 --2- 172.21.15.94:0/3716825510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c104d40 0x7fba4c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 --2- 172.21.15.94:0/3716825510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c103b00 0x7fba4c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 -- 172.21.15.94:0/3716825510 >> 172.21.15.94:0/3716825510 conn(0x7fba4c0fed10 msgr2=0x7fba4c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 -- 172.21.15.94:0/3716825510 shutdown_connections 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.450+0000 7fba511b2640 1 -- 172.21.15.94:0/3716825510 wait complete. 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 Processor -- start 2024-04-25T13:58:11.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 -- start start 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 0x7fba4c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 0x7fba4c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f630 con 0x7fba4c109a40 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f4b0 con 0x7fba4c103b00 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba511b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f7b0 con 0x7fba4c104d40 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba4b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba4ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 0x7fba4c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba4b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47380/0 (socket says 172.21.15.94:47380) 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.451+0000 7fba4b59e640 1 -- 172.21.15.94:0/8262253 learned_addr learned my addr 172.21.15.94:0/8262253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba43fff640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 0x7fba4c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 -- 172.21.15.94:0/8262253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 msgr2=0x7fba4c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 0x7fba4c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 -- 172.21.15.94:0/8262253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 msgr2=0x7fba4c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 0x7fba4c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 -- 172.21.15.94:0/8262253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c1a1d10 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba43fff640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 0x7fba4c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4ad9d640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 0x7fba4c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba4b59e640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fba44004af0 tx=0x7fba44031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44003b70 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c1a1fa0 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba44003d10 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4402fb20 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.452+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c1a2460 con 0x7fba4c109a40 2024-04-25T13:58:11.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.453+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c108850 con 0x7fba4c109a40 2024-04-25T13:58:11.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.455+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fba44004580 con 0x7fba4c109a40 2024-04-25T13:58:11.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.455+0000 7fba48d99640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 0x7fba380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.456+0000 7fba4ad9d640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 0x7fba380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.456+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fba44100570 con 0x7fba4c109a40 2024-04-25T13:58:11.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.456+0000 7fba4ad9d640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 0x7fba380b2000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fba4c104700 tx=0x7fba3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.460+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba440c90b0 con 0x7fba4c109a40 2024-04-25T13:58:11.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.731+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba4c02db50 con 0x7fba4c109a40 2024-04-25T13:58:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.732+0000 7fba48d99640 1 -- 172.21.15.94:0/8262253 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7fba440c89b0 con 0x7fba4c109a40 2024-04-25T13:58:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:58:11.732 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:57:45.761472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30058":{"gid":30058,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2422946890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2422946890},{"type":"v1","addr":"172.21.15.138:6835","nonce":2422946890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:58:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-04-25T13:58:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 msgr2=0x7fba380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 0x7fba380b2000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fba4c104700 tx=0x7fba3c005f90 comp rx=0 tx=0).stop 2024-04-25T13:58:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 msgr2=0x7fba4c1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fba44004af0 tx=0x7fba44031700 comp rx=0 tx=0).stop 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 shutdown_connections 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fba380afaf0 0x7fba380b2000 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba4c109a40 0x7fba4c1a16a0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba4c104d40 0x7fba4c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 --2- 172.21.15.94:0/8262253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba4c103b00 0x7fba4c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 >> 172.21.15.94:0/8262253 conn(0x7fba4c0fed10 msgr2=0x7fba4c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 shutdown_connections 2024-04-25T13:58:11.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.734+0000 7fba511b2640 1 -- 172.21.15.94:0/8262253 wait complete. 2024-04-25T13:58:11.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.865+0000 7f5acbb2e640 1 -- 172.21.15.94:0/493802463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4104d40 msgr2=0x7f5ac4105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.865+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/493802463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4104d40 0x7f5ac4105150 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009900 tx=0x7f5ab402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- 172.21.15.94:0/493802463 shutdown_connections 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/493802463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 0x7f5ac410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/493802463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4104d40 0x7f5ac4105150 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/493802463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4103b00 0x7f5ac4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- 172.21.15.94:0/493802463 >> 172.21.15.94:0/493802463 conn(0x7f5ac40fecf0 msgr2=0x7f5ac41011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- 172.21.15.94:0/493802463 shutdown_connections 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- 172.21.15.94:0/493802463 wait complete. 2024-04-25T13:58:11.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 Processor -- start 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- start start 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4104d40 0x7f5ac419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 0x7f5ac41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac410f630 con 0x7f5ac4103b00 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac410f4b0 con 0x7f5ac4104d40 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.866+0000 7f5acbb2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac410f7b0 con 0x7f5ac4109a40 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47394/0 (socket says 172.21.15.94:47394) 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1239831365 learned_addr learned my addr 172.21.15.94:0/1239831365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 0x7f5ac41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4104d40 0x7f5ac419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1239831365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 msgr2=0x7f5ac41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 0x7f5ac41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1239831365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4104d40 msgr2=0x7f5ac419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4104d40 0x7f5ac419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1239831365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac41a1d50 con 0x7f5ac4103b00 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9f0 tx=0x7f5ac000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5abaffd640 1 -- 172.21.15.94:0/1239831365 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000ccc0 con 0x7f5ac4103b00 2024-04-25T13:58:11.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5abaffd640 1 -- 172.21.15.94:0/1239831365 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac00035f0 con 0x7f5ac4103b00 2024-04-25T13:58:11.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac41a1fe0 con 0x7f5ac4103b00 2024-04-25T13:58:11.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5abaffd640 1 -- 172.21.15.94:0/1239831365 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000ce60 con 0x7f5ac4103b00 2024-04-25T13:58:11.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.867+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41a24a0 con 0x7f5ac4103b00 2024-04-25T13:58:11.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.870+0000 7f5abaffd640 1 -- 172.21.15.94:0/1239831365 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5ac0003aa0 con 0x7f5ac4103b00 2024-04-25T13:58:11.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.870+0000 7f5ad29dc740 1 Processor -- start 2024-04-25T13:58:11.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.870+0000 7f5ad29dc740 1 -- start start 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 0x55842ffd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffd6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55842ff39d80 con 0x55842fc33390 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55842fef3a20 con 0x55842fc0a4d0 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55842ff35d40 con 0x55842fc126b0 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5abaffd640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 0x7f5ab00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5aca0cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 0x55842ffd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5aca0cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 0x55842ffd5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47396/0 (socket says 172.21.15.94:47396) 2024-04-25T13:58:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40110/0 (socket says 172.21.15.94:40110) 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5aca0cb640 1 -- 172.21.15.94:0/1141371303 learned_addr learned my addr 172.21.15.94:0/1141371303 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 0x7f5ab00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffd6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5abaffd640 1 -- 172.21.15.94:0/1239831365 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f5ac00def90 con 0x7f5ac4103b00 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1141371303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 msgr2=0x55842ffd6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffd6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1141371303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 msgr2=0x55842ffd5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 0x55842ffd5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1141371303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842fefec30 con 0x55842fc0a4d0 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffd6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.871+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c6a0 tx=0x7f5ac0003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5356443cbbdc4cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.872+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 0x7f5ab00b2000 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55842ffd6860 tx=0x7f5abc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.872+0000 7f5aaffff640 1 -- 172.21.15.94:0/1141371303 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00a6ee0 con 0x55842fc0a4d0 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.872+0000 7f5aaffff640 1 -- 172.21.15.94:0/1141371303 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac00a7ad0 con 0x55842fc0a4d0 2024-04-25T13:58:11.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.872+0000 7f5aaffff640 1 -- 172.21.15.94:0/1141371303 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00afda0 con 0x55842fc0a4d0 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1141371303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 msgr2=0x55842ffd5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c6a0 tx=0x7f5ac0003e70 comp rx=0 tx=0).stop 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1141371303 shutdown_connections 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffd6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc0a4d0 0x55842ffd5d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1141371303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc33390 0x55842ffd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1141371303 >> 172.21.15.94:0/1141371303 conn(0x55842fc274e0 msgr2=0x55842ffd1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1141371303 shutdown_connections 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1141371303 wait complete. 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 -- start start 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.873+0000 7f5ad29dc740 1 Processor -- start 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffdfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842ffe6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55842ffd74d0 con 0x55842fc0a4d0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55842fec0dc0 con 0x55842fc33390 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55842ffd7650 con 0x55842fc126b0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ad29dc740 10 client.0 _ll_register_callbacks cb 0x55842fedc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47400/0 (socket says 172.21.15.94:47400) 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5aca0cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842ffe6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 -- 172.21.15.94:0/2892729197 learned_addr learned my addr 172.21.15.94:0/2892729197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffdfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 -- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 msgr2=0x55842ffdfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffdfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 -- 172.21.15.94:0/2892729197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 msgr2=0x55842ffe6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842ffe6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac98ca640 1 -- 172.21.15.94:0/2892729197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842fc4b3f0 con 0x55842fc0a4d0 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffdfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.874+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842ffe6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a4640 tx=0x7f5ac00df2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79447fb69b295489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5aad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:11.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00c6060 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f5ac00a39c0 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac00a7ad0 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5ac00a6ee0 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5ad29dc740 10 client.21336 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55842fd157d0 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.875+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac00a7ad0 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5ac00a2d40 con 0x55842fc0a4d0 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 10 client.21336 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 10 client.21336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 20 client.21336 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbf1f60 0x55842ffece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 4 client.21336 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:11.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55842ff47990 con 0x55842fbf1f60 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbf1f60 0x55842ffece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5aad7fa640 10 client.21336 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.876+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbf1f60 0x55842ffece80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bd12bcad3e04a56 server_cookie=f34143d7b698f018 in_seq=0 out_seq=0 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5aad7fa640 1 -- 172.21.15.94:0/2892729197 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55842ff47990 con 0x55842fbf1f60 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5aad7fa640 10 client.21336 handle_command_reply: tid=0 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 client.21336 shutdown 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 msgr2=0x55842ffdf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a4640 tx=0x7f5ac00df2e0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 shutdown_connections 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbf1f60 0x55842ffece80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842ffe6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc126b0 0x55842ffdfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/2892729197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffdf590 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 >> 172.21.15.94:0/2892729197 conn(0x55842fc274e0 msgr2=0x55842ffd0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.877+0000 7f5aad7fa640 0 client.21336 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:11.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.878+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 shutdown_connections 2024-04-25T13:58:11.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.878+0000 7f5ad29dc740 1 -- 172.21.15.94:0/2892729197 wait complete. 2024-04-25T13:58:11.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.878+0000 7f5ad29dc740 20 client.21336 trim_cache size 0 max 16384 2024-04-25T13:58:11.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 Processor -- start 2024-04-25T13:58:11.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 -- start start 2024-04-25T13:58:11.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc0a4d0 0x55842ffdf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842fff1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ab600 con 0x55842fc126b0 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ad140 con 0x55842fc33390 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.892+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41abcd0 con 0x55842fc0a4d0 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47402/0 (socket says 172.21.15.94:47402) 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 -- 172.21.15.94:0/1787736319 learned_addr learned my addr 172.21.15.94:0/1787736319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842fff1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc0a4d0 0x55842ffdf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 -- 172.21.15.94:0/1787736319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc0a4d0 msgr2=0x55842ffdf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc0a4d0 0x55842ffdf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 -- 172.21.15.94:0/1787736319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 msgr2=0x55842fff1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842fff1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 -- 172.21.15.94:0/1787736319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842ffcc780 con 0x55842fc126b0 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4031500 tx=0x7f5ab40315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96e4fccecd723a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5aaf7fe640 1 -- 172.21.15.94:0/1787736319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4035040 con 0x55842fc126b0 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5aaf7fe640 1 -- 172.21.15.94:0/1787736319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab40319d0 con 0x55842fc126b0 2024-04-25T13:58:11.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.893+0000 7f5aaf7fe640 1 -- 172.21.15.94:0/1787736319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4003850 con 0x55842fc126b0 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1787736319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 msgr2=0x55842ffdf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4031500 tx=0x7f5ab40315a0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1787736319 shutdown_connections 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc33390 0x55842fff1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc126b0 0x55842ffdf6a0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1787736319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc0a4d0 0x55842ffdf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1787736319 >> 172.21.15.94:0/1787736319 conn(0x55842fc274e0 msgr2=0x55842ffda330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1787736319 shutdown_connections 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.894+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1787736319 wait complete. 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 -- start start 2024-04-25T13:58:11.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 Processor -- start 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 0x55842ffe3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc33390 0x558430053020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41aca70 con 0x55842fc0a4d0 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41ac3a0 con 0x55842fc126b0 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41a95e0 con 0x55842fc33390 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ad29dc740 10 client.0 _ll_register_callbacks cb 0x7f5abc011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47412/0 (socket says 172.21.15.94:47412) 2024-04-25T13:58:11.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1587598565 learned_addr learned my addr 172.21.15.94:0/1587598565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 0x55842ffe3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5aca0cb640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc33390 0x558430053020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc33390 msgr2=0x558430053020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc33390 0x558430053020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1587598565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 msgr2=0x55842ffe3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 0x55842ffe3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.895+0000 7f5ac98ca640 1 -- 172.21.15.94:0/1587598565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842ffe6350 con 0x55842fc0a4d0 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 0x55842ffe3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:11.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ac98ca640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a70d0 tx=0x7f5ac00a2bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352e3a5787385da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00a6d20 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f5ac00a6840 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab40319d0 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5ac00a7ad0 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 10 client.21342 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558430054510 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab40319d0 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5ac00a6160 con 0x55842fc0a4d0 2024-04-25T13:58:11.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 10 client.21342 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 10 client.21342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 20 client.21342 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.896+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbc73a0 0x558430056d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ad29dc740 4 client.21342 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55842ff41800 con 0x55842fbc73a0 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbc73a0 0x558430056d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5aad7fa640 10 client.21342 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ac90c9640 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbc73a0 0x558430056d40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=909944213a2cf467 server_cookie=43a674c1bbde106c in_seq=0 out_seq=0 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5aad7fa640 1 -- 172.21.15.94:0/1587598565 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55842ff41800 con 0x55842fbc73a0 2024-04-25T13:58:11.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5aad7fa640 10 client.21342 handle_command_reply: tid=0 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ad29dc740 1 client.21342 shutdown 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 msgr2=0x55842ffe2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.897+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00a70d0 tx=0x7f5ac00a2bf0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 shutdown_connections 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55842fbc73a0 0x558430056d40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55842fc33390 0x558430053020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55842fc126b0 0x55842ffe3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 --2- 172.21.15.94:0/1587598565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55842fc0a4d0 0x55842ffe2e50 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5aad7fa640 0 client.21342 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 >> 172.21.15.94:0/1587598565 conn(0x55842fc274e0 msgr2=0x55842ffdf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 shutdown_connections 2024-04-25T13:58:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 1 -- 172.21.15.94:0/1587598565 wait complete. 2024-04-25T13:58:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5ad29dc740 20 client.21342 trim_cache size 0 max 16384 2024-04-25T13:58:11.909 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 msgr2=0x7f5ab00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 0x7f5ab00b2000 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55842ffd6860 tx=0x7f5abc009500 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 msgr2=0x7f5ac419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000e9f0 tx=0x7f5ac000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 shutdown_connections 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ab00afaf0 0x7f5ab00b2000 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4109a40 0x7f5ac41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ac4104d40 0x7f5ac419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 --2- 172.21.15.94:0/1239831365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ac4103b00 0x7f5ac419a210 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 >> 172.21.15.94:0/1239831365 conn(0x7f5ac40fecf0 msgr2=0x7f5ac4105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 shutdown_connections 2024-04-25T13:58:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:11.898+0000 7f5acbb2e640 1 -- 172.21.15.94:0/1239831365 wait complete. 2024-04-25T13:58:11.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:58:12.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3309682168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 msgr2=0x7f97b8105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3309682168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b8105140 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009900 tx=0x7f97a802a410 comp rx=0 tx=0).stop 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3309682168 shutdown_connections 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3309682168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 0x7f97b810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3309682168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b8105140 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3309682168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3309682168 >> 172.21.15.94:0/3309682168 conn(0x7f97b80fece0 msgr2=0x7f97b8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3309682168 shutdown_connections 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.031+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3309682168 wait complete. 2024-04-25T13:58:12.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 Processor -- start 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 -- start start 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 0x7f97b81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97b810f610 con 0x7f97b8104d30 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97b810f490 con 0x7f97b8109a30 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bf7c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97b810f790 con 0x7f97b8103af0 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bcd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bcd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47426/0 (socket says 172.21.15.94:47426) 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bcd60640 1 -- 172.21.15.94:0/3394562320 learned_addr learned my addr 172.21.15.94:0/3394562320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bd561640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.032+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 0x7f97b81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 -- 172.21.15.94:0/3394562320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 msgr2=0x7f97b819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 -- 172.21.15.94:0/3394562320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 msgr2=0x7f97b81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 0x7f97b81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 -- 172.21.15.94:0/3394562320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b81a1d80 con 0x7f97b8104d30 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bd561640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f97a8006010 tx=0x7f97a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97ae7fc640 1 -- 172.21.15.94:0/3394562320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a803f030 con 0x7f97b8104d30 2024-04-25T13:58:12.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b81a2010 con 0x7f97b8104d30 2024-04-25T13:58:12.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97ae7fc640 1 -- 172.21.15.94:0/3394562320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a8031910 con 0x7f97b8104d30 2024-04-25T13:58:12.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97ae7fc640 1 -- 172.21.15.94:0/3394562320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8035070 con 0x7f97b8104d30 2024-04-25T13:58:12.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.033+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b81a2500 con 0x7f97b8104d30 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97ae7fc640 1 -- 172.21.15.94:0/3394562320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f97a8042030 con 0x7f97b8104d30 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 Processor -- start 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 -- start start 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 0x55e8e716e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e716ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dab6b0 0x55e8e716fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e8e70d2d80 con 0x55e8e6dcc390 2024-04-25T13:58:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e708ca20 con 0x55e8e6da34d0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97c6679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8e70ced40 con 0x55e8e6dab6b0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 0x55e8e716e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 0x55e8e716e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47430/0 (socket says 172.21.15.94:47430) 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bdd62640 1 -- 172.21.15.94:0/934171507 learned_addr learned my addr 172.21.15.94:0/934171507 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e716ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97ae7fc640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bcd60640 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dab6b0 0x55e8e716fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97ae7fc640 1 -- 172.21.15.94:0/3394562320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f97a80ff180 con 0x7f97b8104d30 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 -- 172.21.15.94:0/934171507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dab6b0 msgr2=0x55e8e716fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dab6b0 0x55e8e716fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 -- 172.21.15.94:0/934171507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 msgr2=0x55e8e716e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 0x55e8e716e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.036+0000 7f97bd561640 1 -- 172.21.15.94:0/934171507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e7097c30 con 0x55e8e6da34d0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.037+0000 7f97bd561640 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e716ed90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97b81033c0 tx=0x7f97b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee5485dda4f2d121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.037+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 0x7f97a40b2000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55e8e716f860 tx=0x7f97b0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.037+0000 7f97a37fe640 1 -- 172.21.15.94:0/934171507 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b401e070 con 0x55e8e6da34d0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.037+0000 7f97a37fe640 1 -- 172.21.15.94:0/934171507 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b40035f0 con 0x55e8e6da34d0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.037+0000 7f97a37fe640 1 -- 172.21.15.94:0/934171507 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400f780 con 0x55e8e6da34d0 2024-04-25T13:58:12.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- 172.21.15.94:0/934171507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 msgr2=0x55e8e716ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e716ed90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97b81033c0 tx=0x7f97b40118d0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- 172.21.15.94:0/934171507 shutdown_connections 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dab6b0 0x55e8e716fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e716ed90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 --2- 172.21.15.94:0/934171507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dcc390 0x55e8e716e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- 172.21.15.94:0/934171507 >> 172.21.15.94:0/934171507 conn(0x55e8e6dc04e0 msgr2=0x55e8e716a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- 172.21.15.94:0/934171507 shutdown_connections 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- 172.21.15.94:0/934171507 wait complete. 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.038+0000 7f97c6679740 1 -- start start 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 Processor -- start 2024-04-25T13:58:12.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 0x55e8e717fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7170530 con 0x55e8e6dab6b0 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8e7059dc0 con 0x55e8e6da34d0 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8e71706b0 con 0x55e8e6dcc390 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97c6679740 10 client.0 _ll_register_callbacks cb 0x55e8e7075600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bcd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bd561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bcd60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47436/0 (socket says 172.21.15.94:47436) 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bd561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40168/0 (socket says 172.21.15.94:40168) 2024-04-25T13:58:12.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bcd60640 1 -- 172.21.15.94:0/3718812127 learned_addr learned my addr 172.21.15.94:0/3718812127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.039+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 0x55e8e717fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 -- 172.21.15.94:0/3718812127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 msgr2=0x55e8e717fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 0x55e8e717fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 -- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 msgr2=0x55e8e7178560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 -- 172.21.15.94:0/3718812127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e6de43f0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 0x55e8e717fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bd561640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f97a80c7600 tx=0x7f97a80c7550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fdc3cd208f38bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8049070 con 0x55e8e6dab6b0 2024-04-25T13:58:12.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f97a80c78d0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b40035f0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97a80c7180 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97c6679740 10 client.21354 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.040+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e8e6eae7d0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b40035f0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97a80cff90 con 0x55e8e6dab6b0 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 10 client.21354 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 10 client.21354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 20 client.21354 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6d8af60 0x55e8e71861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 4 client.21354 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:12.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e70e0990 con 0x55e8e6d8af60 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.041+0000 7f97bd561640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6d8af60 0x55e8e71861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.042+0000 7f97a0ff9640 10 client.21354 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.042+0000 7f97bd561640 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6d8af60 0x55e8e71861b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aab3ca09d85e412e server_cookie=286db949558896de in_seq=0 out_seq=0 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97a0ff9640 1 -- 172.21.15.94:0/3718812127 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8e70e0990 con 0x55e8e6d8af60 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97a0ff9640 10 client.21354 handle_command_reply: tid=0 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 client.21354 shutdown 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 msgr2=0x55e8e7178aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f97a80c7600 tx=0x7f97a80c7550 comp rx=0 tx=0).stop 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 shutdown_connections 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6d8af60 0x55e8e71861b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6dcc390 0x55e8e717fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97a0ff9640 0 client.21354 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7178aa0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 --2- 172.21.15.94:0/3718812127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e7178560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 >> 172.21.15.94:0/3718812127 conn(0x55e8e6dc04e0 msgr2=0x55e8e716cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 shutdown_connections 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 1 -- 172.21.15.94:0/3718812127 wait complete. 2024-04-25T13:58:12.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.043+0000 7f97c6679740 20 client.21354 trim_cache size 0 max 16384 2024-04-25T13:58:12.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 Processor -- start 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 -- start start 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6d8af60 0x55e8e71732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e717f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81ab4b0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81acff0 con 0x55e8e6da34d0 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.057+0000 7f97c6679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81abb80 con 0x55e8e6d8af60 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47442/0 (socket says 172.21.15.94:47442) 2024-04-25T13:58:12.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 -- 172.21.15.94:0/3213115254 learned_addr learned my addr 172.21.15.94:0/3213115254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e717f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bd561640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6d8af60 0x55e8e71732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 -- 172.21.15.94:0/3213115254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6d8af60 msgr2=0x55e8e71732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6d8af60 0x55e8e71732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 -- 172.21.15.94:0/3213115254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 msgr2=0x55e8e717f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e717f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 -- 172.21.15.94:0/3213115254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e71642b0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bcd60640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e717f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97bdd62640 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f97b0004260 tx=0x7f97b0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e09f4830713f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97a2ffd640 1 -- 172.21.15.94:0/3213115254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0010bb0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97a2ffd640 1 -- 172.21.15.94:0/3213115254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b0027030 con 0x55e8e6dab6b0 2024-04-25T13:58:12.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.058+0000 7f97a2ffd640 1 -- 172.21.15.94:0/3213115254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b002b3d0 con 0x55e8e6dab6b0 2024-04-25T13:58:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- 172.21.15.94:0/3213115254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 msgr2=0x55e8e7173830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f97b0004260 tx=0x7f97b0009be0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- 172.21.15.94:0/3213115254 shutdown_connections 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e7173830 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6da34d0 0x55e8e717f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 --2- 172.21.15.94:0/3213115254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6d8af60 0x55e8e71732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- 172.21.15.94:0/3213115254 >> 172.21.15.94:0/3213115254 conn(0x55e8e6dc04e0 msgr2=0x55e8e716e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- 172.21.15.94:0/3213115254 shutdown_connections 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- 172.21.15.94:0/3213115254 wait complete. 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 -- start start 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.060+0000 7f97c6679740 1 Processor -- start 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 0x55e8e717e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6da34d0 0x55e8e717ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81ac920 con 0x55e8e6dab6b0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81ac250 con 0x55e8e6d8af60 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97b81a9490 con 0x55e8e6da34d0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 10 client.0 _ll_register_callbacks cb 0x7f97a80c3620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 0x55e8e717e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bdd62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47456/0 (socket says 172.21.15.94:47456) 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bdd62640 1 -- 172.21.15.94:0/2608793411 learned_addr learned my addr 172.21.15.94:0/2608793411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bcd60640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6da34d0 0x55e8e717ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 -- 172.21.15.94:0/2608793411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6da34d0 msgr2=0x55e8e717ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6da34d0 0x55e8e717ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 -- 172.21.15.94:0/2608793411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 msgr2=0x55e8e71ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 -- 172.21.15.94:0/2608793411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e71eca90 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bdd62640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97bd561640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 0x55e8e717e9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011dd0 tx=0x7f97b40035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e5369e6936450c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b401e040 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f97b4003bd0 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.061+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b0027030 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97b4009bf0 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 10 client.30788 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e8e71edc70 con 0x55e8e6d8af60 2024-04-25T13:58:12.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b0027030 con 0x55e8e6d8af60 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97b402b5e0 con 0x55e8e6d8af60 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 10 client.30788 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 10 client.30788 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 20 client.30788 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6dcc390 0x55e8e71f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 4 client.30788 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8e70da800 con 0x55e8e6dcc390 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.062+0000 7f97bcd60640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6dcc390 0x55e8e71f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97a0ff9640 10 client.30788 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97bcd60640 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6dcc390 0x55e8e71f03f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d23061a803f89aa7 server_cookie=abdffbf54caff14f in_seq=0 out_seq=0 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97a0ff9640 1 -- 172.21.15.94:0/2608793411 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8e70da800 con 0x55e8e6dcc390 2024-04-25T13:58:12.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97a0ff9640 10 client.30788 handle_command_reply: tid=0 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 client.30788 shutdown 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 msgr2=0x55e8e717e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 0x55e8e717e9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011dd0 tx=0x7f97b40035f0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 shutdown_connections 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e8e6dcc390 0x55e8e71f03f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e8e6dab6b0 0x55e8e71ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8e6da34d0 0x55e8e717ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 --2- 172.21.15.94:0/2608793411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8e6d8af60 0x55e8e717e9d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97a0ff9640 0 client.30788 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 >> 172.21.15.94:0/2608793411 conn(0x55e8e6dc04e0 msgr2=0x55e8e716e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 shutdown_connections 2024-04-25T13:58:12.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 1 -- 172.21.15.94:0/2608793411 wait complete. 2024-04-25T13:58:12.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.063+0000 7f97c6679740 20 client.30788 trim_cache size 0 max 16384 2024-04-25T13:58:12.073 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 msgr2=0x7f97a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 0x7f97a40b2000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55e8e716f860 tx=0x7f97b0004ab0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 msgr2=0x7f97b819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f97a8006010 tx=0x7f97a80042f0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 shutdown_connections 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8109a30 0x7f97b81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97b8104d30 0x7f97b819a780 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 --2- 172.21.15.94:0/3394562320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b8103af0 0x7f97b819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 >> 172.21.15.94:0/3394562320 conn(0x7f97b80fece0 msgr2=0x7f97b8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 shutdown_connections 2024-04-25T13:58:12.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.064+0000 7f97bf7c5640 1 -- 172.21.15.94:0/3394562320 wait complete. 2024-04-25T13:58:12.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:58:12.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 63855 19.22 MB/sec execute 54 sec latency 60.656 ms 2024-04-25T13:58:12.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 -- 172.21.15.94:0/1831247407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 msgr2=0x7f6f2c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 --2- 172.21.15.94:0/1831247407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c101bb0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009900 tx=0x7f6f1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 -- 172.21.15.94:0/1831247407 shutdown_connections 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 --2- 172.21.15.94:0/1831247407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 --2- 172.21.15.94:0/1831247407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c101bb0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 --2- 172.21.15.94:0/1831247407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.197+0000 7f6f34300640 1 -- 172.21.15.94:0/1831247407 >> 172.21.15.94:0/1831247407 conn(0x7f6f2c0fd340 msgr2=0x7f6f2c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.198+0000 7f6f34300640 1 -- 172.21.15.94:0/1831247407 shutdown_connections 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.198+0000 7f6f34300640 1 -- 172.21.15.94:0/1831247407 wait complete. 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.198+0000 7f6f34300640 1 Processor -- start 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.198+0000 7f6f34300640 1 -- start start 2024-04-25T13:58:12.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c10f600 con 0x7f6f2c101740 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c10f480 con 0x7f6f2c102240 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f34300640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c10f780 con 0x7f6f2c105090 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3289d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3289d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42080/0 (socket says 172.21.15.94:42080) 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40204/0 (socket says 172.21.15.94:40204) 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3289d640 1 -- 172.21.15.94:0/873123122 learned_addr learned my addr 172.21.15.94:0/873123122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 -- 172.21.15.94:0/873123122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 msgr2=0x7f6f2c102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 -- 172.21.15.94:0/873123122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 msgr2=0x7f6f2c1028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 -- 172.21.15.94:0/873123122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c103550 con 0x7f6f2c101740 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3289d640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3189b640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.199+0000 7f6f3209c640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c104240 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c069bf0 tx=0x7f6f2800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.200+0000 7f6f237fe640 1 -- 172.21.15.94:0/873123122 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28017070 con 0x7f6f2c101740 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.200+0000 7f6f237fe640 1 -- 172.21.15.94:0/873123122 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f28009c00 con 0x7f6f2c101740 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.200+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2c199ec0 con 0x7f6f2c101740 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.200+0000 7f6f237fe640 1 -- 172.21.15.94:0/873123122 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800f620 con 0x7f6f2c101740 2024-04-25T13:58:12.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.200+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2c19a410 con 0x7f6f2c101740 2024-04-25T13:58:12.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f237fe640 1 -- 172.21.15.94:0/873123122 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6f2801a070 con 0x7f6f2c101740 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 Processor -- start 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 -- start start 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aefdca8d80 con 0x55aefd9a2390 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aefdc62a20 con 0x55aefd9816b0 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.203+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aefdca4d40 con 0x55aefd9794d0 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47474/0 (socket says 172.21.15.94:47474) 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40206/0 (socket says 172.21.15.94:40206) 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 -- 172.21.15.94:0/3963201992 learned_addr learned my addr 172.21.15.94:0/3963201992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f237fe640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 0x7f6f180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 -- 172.21.15.94:0/3963201992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 msgr2=0x55aefdd44d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd44d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 -- 172.21.15.94:0/3963201992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 msgr2=0x55aefdd44810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 -- 172.21.15.94:0/3963201992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefdc6dc30 con 0x55aefd9816b0 2024-04-25T13:58:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f237fe640 1 -- 172.21.15.94:0/873123122 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f6f280de960 con 0x7f6f2c101740 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 0x7f6f180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c005ab0 tx=0x7f6f1c005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb8b69aa6d7e409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f20bfe640 1 -- 172.21.15.94:0/3963201992 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c035040 con 0x55aefd9816b0 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.204+0000 7f6f3289d640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 0x7f6f180b2000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55aefdd45860 tx=0x7f6f24004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f20bfe640 1 -- 172.21.15.94:0/3963201992 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1c031820 con 0x55aefd9816b0 2024-04-25T13:58:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f20bfe640 1 -- 172.21.15.94:0/3963201992 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c003850 con 0x55aefd9816b0 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3963201992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 msgr2=0x55aefdd45a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c005ab0 tx=0x7f6f1c005eb0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3963201992 shutdown_connections 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd45a20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3963201992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd44810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.205+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3963201992 >> 172.21.15.94:0/3963201992 conn(0x55aefd9964e0 msgr2=0x55aefdd40020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3963201992 shutdown_connections 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3963201992 wait complete. 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- start start 2024-04-25T13:58:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 Processor -- start 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 0x55aefdd4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd464d0 con 0x55aefd9a2390 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aefdc2fdc0 con 0x55aefd9794d0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aefdd46650 con 0x55aefd9816b0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.206+0000 7f6f3b1b1740 10 client.0 _ll_register_callbacks cb 0x55aefdc4b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3289d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 0x55aefdd4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40208/0 (socket says 172.21.15.94:40208) 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 -- 172.21.15.94:0/3817149949 learned_addr learned my addr 172.21.15.94:0/3817149949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 -- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 msgr2=0x55aefdd4ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 0x55aefdd4ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 -- 172.21.15.94:0/3817149949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 msgr2=0x55aefdd55a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd55a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 -- 172.21.15.94:0/3817149949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefd9ba3f0 con 0x55aefd9794d0 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 0x55aefdd4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f28000c00 tx=0x7f6f2800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4ef0fd8a2735b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28021030 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f1c031820 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f6f280af770 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.207+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6f280a6c20 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 10 client.30800 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aefda847d0 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1c031820 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6f2808f070 con 0x55aefd9794d0 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 10 client.30800 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 10 client.30800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:58:12.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 20 client.30800 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd960f60 0x55aefdd5c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 4 client.30800 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aefdcb6990 con 0x55aefd960f60 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.208+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd960f60 0x55aefdd5c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.209+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd960f60 0x55aefdd5c290 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea417e0bba72143f server_cookie=37387260d8d96efb in_seq=0 out_seq=0 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.209+0000 7f6f15ffb640 10 client.30800 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.209+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3817149949 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aefdcb6990 con 0x55aefd960f60 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.209+0000 7f6f15ffb640 10 client.30800 handle_command_reply: tid=0 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.209+0000 7f6f3b1b1740 1 client.30800 shutdown 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 msgr2=0x55aefdd4e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f28000c00 tx=0x7f6f2800bed0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 shutdown_connections 2024-04-25T13:58:12.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd960f60 0x55aefdd5c290 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9a2390 0x55aefdd55a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9816b0 0x55aefdd4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3817149949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9794d0 0x55aefdd4e590 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f15ffb640 0 client.30800 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 >> 172.21.15.94:0/3817149949 conn(0x55aefd9964e0 msgr2=0x55aefdd42f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 shutdown_connections 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3817149949 wait complete. 2024-04-25T13:58:12.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.210+0000 7f6f3b1b1740 20 client.30800 trim_cache size 0 max 16384 2024-04-25T13:58:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.225+0000 7f6f3b1b1740 1 Processor -- start 2024-04-25T13:58:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 -- start start 2024-04-25T13:58:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1a9290 con 0x55aefd960f60 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1aadd0 con 0x55aefd9816b0 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1a9960 con 0x55aefd9794d0 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47494/0 (socket says 172.21.15.94:47494) 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 -- 172.21.15.94:0/3906948995 learned_addr learned my addr 172.21.15.94:0/3906948995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3289d640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 -- 172.21.15.94:0/3906948995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 msgr2=0x55aefdd61f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd61f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 -- 172.21.15.94:0/3906948995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 msgr2=0x55aefdd42e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd42e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 -- 172.21.15.94:0/3906948995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefdd58c00 con 0x55aefd960f60 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f6f280a6de0 tx=0x7f6f280e40c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e315f4bbe0986bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.226+0000 7f6f17fff640 1 -- 172.21.15.94:0/3906948995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28021070 con 0x55aefd960f60 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f17fff640 1 -- 172.21.15.94:0/3906948995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f280a6960 con 0x55aefd960f60 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f17fff640 1 -- 172.21.15.94:0/3906948995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f280a65c0 con 0x55aefd960f60 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3906948995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 msgr2=0x55aefdd61a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f6f280a6de0 tx=0x7f6f280e40c0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3906948995 shutdown_connections 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefdd42e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd9794d0 0x55aefdd61f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3906948995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd960f60 0x55aefdd61a50 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3906948995 >> 172.21.15.94:0/3906948995 conn(0x55aefd9964e0 msgr2=0x55aefdd49310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.227+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3906948995 shutdown_connections 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3906948995 wait complete. 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 -- start start 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 Processor -- start 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd960f60 0x55aefdd541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefddc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1aa700 con 0x55aefd9794d0 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1aa030 con 0x55aefd9816b0 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c1a7270 con 0x55aefd960f60 2024-04-25T13:58:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3b1b1740 10 client.0 _ll_register_callbacks cb 0x7f6f2800df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3189b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47500/0 (socket says 172.21.15.94:47500) 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3189b640 1 -- 172.21.15.94:0/3983319579 learned_addr learned my addr 172.21.15.94:0/3983319579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3289d640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefddc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.228+0000 7f6f3189b640 1 -- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd960f60 msgr2=0x55aefdd541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd960f60 0x55aefdd541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd960f60 0x55aefdd541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3189b640 1 -- 172.21.15.94:0/3983319579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 msgr2=0x55aefddc25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefddc25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3189b640 1 -- 172.21.15.94:0/3983319579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefddc2bb0 con 0x55aefd9794d0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3189b640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009900 tx=0x7f6f1c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc35d64d143cbb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:12.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c05f060 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f6f1c03ea70 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f280a6960 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6f1c035050 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 10 client.21378 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aefddc3e50 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f280a6960 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6f1c059ce0 con 0x55aefd9794d0 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 10 client.21378 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 10 client.21378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 20 client.21378 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd9a2390 0x55aefddc63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 4 client.21378 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.229+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aefdcb0800 con 0x55aefd9a2390 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd9a2390 0x55aefddc63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f15ffb640 10 client.21378 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3209c640 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd9a2390 0x55aefddc63d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f9b7733f0f0d593 server_cookie=297a601f9c909573 in_seq=0 out_seq=0 2024-04-25T13:58:12.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f15ffb640 1 -- 172.21.15.94:0/3983319579 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aefdcb0800 con 0x55aefd9a2390 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f15ffb640 10 client.21378 handle_command_reply: tid=0 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 client.21378 shutdown 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 msgr2=0x55aefdd54720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009900 tx=0x7f6f1c032ed0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 shutdown_connections 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55aefd9a2390 0x55aefddc63d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aefd9816b0 0x55aefddc25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f15ffb640 0 client.21378 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aefd9794d0 0x55aefdd54720 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 --2- 172.21.15.94:0/3983319579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aefd960f60 0x55aefdd541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 >> 172.21.15.94:0/3983319579 conn(0x55aefd9964e0 msgr2=0x55aefdd42e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 shutdown_connections 2024-04-25T13:58:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.230+0000 7f6f3b1b1740 1 -- 172.21.15.94:0/3983319579 wait complete. 2024-04-25T13:58:12.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f3b1b1740 20 client.21378 trim_cache size 0 max 16384 2024-04-25T13:58:12.241 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 msgr2=0x7f6f180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 0x7f6f180b2000 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55aefdd45860 tx=0x7f6f24004a40 comp rx=0 tx=0).stop 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 msgr2=0x7f6f2c104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c104240 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c069bf0 tx=0x7f6f2800dde0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 shutdown_connections 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6f180afaf0 0x7f6f180b2000 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c105090 0x7f6f2c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f2c102240 0x7f6f2c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 --2- 172.21.15.94:0/873123122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f2c101740 0x7f6f2c104240 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 >> 172.21.15.94:0/873123122 conn(0x7f6f2c0fd340 msgr2=0x7f6f2c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 shutdown_connections 2024-04-25T13:58:12.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.231+0000 7f6f34300640 1 -- 172.21.15.94:0/873123122 wait complete. 2024-04-25T13:58:12.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:58:12.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 -- 172.21.15.94:0/1850677345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 msgr2=0x7fedec1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 --2- 172.21.15.94:0/1850677345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec1036f0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 -- 172.21.15.94:0/1850677345 shutdown_connections 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 --2- 172.21.15.94:0/1850677345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec108930 0x7fedec10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 --2- 172.21.15.94:0/1850677345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec103c30 0x7fedec104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 --2- 172.21.15.94:0/1850677345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec1036f0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 -- 172.21.15.94:0/1850677345 >> 172.21.15.94:0/1850677345 conn(0x7fedec0fdc00 msgr2=0x7fedec100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 -- 172.21.15.94:0/1850677345 shutdown_connections 2024-04-25T13:58:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.371+0000 7fedf09f3640 1 -- 172.21.15.94:0/1850677345 wait complete. 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.372+0000 7fedf09f3640 1 Processor -- start 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.372+0000 7fedf09f3640 1 -- start start 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.372+0000 7fedf09f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.372+0000 7fedf09f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 0x7fedec19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.372+0000 7fedf09f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedf09f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10e4f0 con 0x7fedec1029f0 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedf09f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10e370 con 0x7fedec108930 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedf09f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10e670 con 0x7fedec103c30 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedead9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedead9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40244/0 (socket says 172.21.15.94:40244) 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedead9d640 1 -- 172.21.15.94:0/2081554089 learned_addr learned my addr 172.21.15.94:0/2081554089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47522/0 (socket says 172.21.15.94:47522) 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 0x7fedec19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 -- 172.21.15.94:0/2081554089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 msgr2=0x7fedec1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 -- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 msgr2=0x7fedec19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 -- 172.21.15.94:0/2081554089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1a1d80 con 0x7fedec103c30 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fedead9d640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.373+0000 7fede1d9b640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 0x7fedec19a780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9f0 tx=0x7fedd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.374+0000 7fede3fff640 1 -- 172.21.15.94:0/2081554089 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd400ccc0 con 0x7fedec103c30 2024-04-25T13:58:12.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.374+0000 7fede3fff640 1 -- 172.21.15.94:0/2081554089 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd40108e0 con 0x7fedec103c30 2024-04-25T13:58:12.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.374+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1a2070 con 0x7fedec103c30 2024-04-25T13:58:12.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.374+0000 7fede3fff640 1 -- 172.21.15.94:0/2081554089 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4018880 con 0x7fedec103c30 2024-04-25T13:58:12.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.374+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a25c0 con 0x7fedec103c30 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.376+0000 7fedf797a740 1 Processor -- start 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.376+0000 7fede3fff640 1 -- 172.21.15.94:0/2081554089 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fedd4018a20 con 0x7fedec103c30 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.376+0000 7fedf797a740 1 -- start start 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a357810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a358a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56081a2bbd80 con 0x560819fb5390 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56081a275a20 con 0x560819f946b0 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedf797a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56081a2b7d40 con 0x560819f8c4d0 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a357810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42156/0 (socket says 172.21.15.94:42156) 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 -- 172.21.15.94:0/1281523016 learned_addr learned my addr 172.21.15.94:0/1281523016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedead9d640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a357810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47536/0 (socket says 172.21.15.94:47536) 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fede1d9b640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a358a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 -- 172.21.15.94:0/1281523016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 msgr2=0x56081a358a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a358a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 -- 172.21.15.94:0/1281523016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 msgr2=0x56081a357810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a357810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedea59c640 1 -- 172.21.15.94:0/1281523016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081a280c30 con 0x560819f8c4d0 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fede3fff640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 0x7fedd80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.377+0000 7fedead9d640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 0x7fedd80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fedea59c640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feddc009230 tx=0x7feddc009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f197d33e06ff93df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fede3fff640 1 -- 172.21.15.94:0/2081554089 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fedd40df060 con 0x7fedec103c30 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fede1d9b640 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a358a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fede0d99640 1 -- 172.21.15.94:0/1281523016 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc035030 con 0x560819f8c4d0 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fede0d99640 1 -- 172.21.15.94:0/1281523016 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc040070 con 0x560819f8c4d0 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.378+0000 7fede0d99640 1 -- 172.21.15.94:0/1281523016 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc0038c0 con 0x560819f8c4d0 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 -- 172.21.15.94:0/1281523016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 msgr2=0x56081a357d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedead9d640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 0x7fedd80b2080 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x56081a358860 tx=0x7fede4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feddc009230 tx=0x7feddc009620 comp rx=0 tx=0).stop 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 -- 172.21.15.94:0/1281523016 shutdown_connections 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a358a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a357d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 --2- 172.21.15.94:0/1281523016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a357810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 -- 172.21.15.94:0/1281523016 >> 172.21.15.94:0/1281523016 conn(0x560819fa94e0 msgr2=0x56081a353020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 -- 172.21.15.94:0/1281523016 shutdown_connections 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.379+0000 7fedf797a740 1 -- 172.21.15.94:0/1281523016 wait complete. 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 -- start start 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 Processor -- start 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a361590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a361ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56081a3594d0 con 0x560819fb5390 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56081a242dc0 con 0x560819f8c4d0 2024-04-25T13:58:12.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56081a359650 con 0x560819f946b0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedf797a740 10 client.0 _ll_register_callbacks cb 0x56081a25e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fede1d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a361ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedea59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a361590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47546/0 (socket says 172.21.15.94:47546) 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.380+0000 7fedead9d640 1 -- 172.21.15.94:0/296813940 learned_addr learned my addr 172.21.15.94:0/296813940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 -- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 msgr2=0x56081a361590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a361590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 -- 172.21.15.94:0/296813940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 msgr2=0x56081a368a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 -- 172.21.15.94:0/296813940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560819fcd3f0 con 0x560819f946b0 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedea59c640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a361590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedead9d640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fede1d9b640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a361ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fedd40a7480 tx=0x7fedd40a6ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b67fcd00298e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40ab7f0 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fedd4010a90 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc040070 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fedd40b02a0 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedf797a740 10 client.30508 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.381+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56081a0977d0 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc040070 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fedd40a2e60 con 0x560819f946b0 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 10 client.30508 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 10 client.30508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 20 client.30508 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819f73f60 0x56081a36f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 4 client.30508 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56081a2c9990 con 0x560819f73f60 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedea59c640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819f73f60 0x56081a36f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedd1ffb640 10 client.30508 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.382+0000 7fedea59c640 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819f73f60 0x56081a36f1d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52ffadfe53f2e2cd server_cookie=df5a4bb244d9aeb1 in_seq=0 out_seq=0 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.383+0000 7fedd1ffb640 1 -- 172.21.15.94:0/296813940 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56081a2c9990 con 0x560819f73f60 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.383+0000 7fedd1ffb640 10 client.30508 handle_command_reply: tid=0 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.383+0000 7fedf797a740 1 client.30508 shutdown 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 msgr2=0x56081a361ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a361ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fedd40a7480 tx=0x7fedd40a6ce0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 shutdown_connections 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819f73f60 0x56081a36f1d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819fb5390 0x56081a368a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a361ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedd1ffb640 0 client.30508 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 --2- 172.21.15.94:0/296813940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a361590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 >> 172.21.15.94:0/296813940 conn(0x560819fa94e0 msgr2=0x56081a355f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 shutdown_connections 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 1 -- 172.21.15.94:0/296813940 wait complete. 2024-04-25T13:58:12.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.384+0000 7fedf797a740 20 client.30508 trim_cache size 0 max 16384 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 Processor -- start 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 -- start start 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a368950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a351b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ab520 con 0x560819f73f60 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ad060 con 0x560819f946b0 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedf797a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1abbf0 con 0x560819f8c4d0 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47554/0 (socket says 172.21.15.94:47554) 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedea59c640 1 -- 172.21.15.94:0/402014854 learned_addr learned my addr 172.21.15.94:0/402014854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fede1d9b640 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a368950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.398+0000 7fedead9d640 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a351b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 -- 172.21.15.94:0/402014854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 msgr2=0x56081a368950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a368950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 -- 172.21.15.94:0/402014854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 msgr2=0x56081a351b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a351b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 -- 172.21.15.94:0/402014854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081a3571a0 con 0x560819f73f60 2024-04-25T13:58:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedea59c640 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7feddc005f70 tx=0x7feddc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bbd4cf5e3672d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedd3fff640 1 -- 172.21.15.94:0/402014854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc035040 con 0x560819f73f60 2024-04-25T13:58:12.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedd3fff640 1 -- 172.21.15.94:0/402014854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc040040 con 0x560819f73f60 2024-04-25T13:58:12.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.399+0000 7fedd3fff640 1 -- 172.21.15.94:0/402014854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc0506c0 con 0x560819f73f60 2024-04-25T13:58:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- 172.21.15.94:0/402014854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 msgr2=0x56081a351610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7feddc005f70 tx=0x7feddc009260 comp rx=0 tx=0).stop 2024-04-25T13:58:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- 172.21.15.94:0/402014854 shutdown_connections 2024-04-25T13:58:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f946b0 0x56081a351b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f8c4d0 0x56081a368950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 --2- 172.21.15.94:0/402014854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a351610 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- 172.21.15.94:0/402014854 >> 172.21.15.94:0/402014854 conn(0x560819fa94e0 msgr2=0x56081a357790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- 172.21.15.94:0/402014854 shutdown_connections 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- 172.21.15.94:0/402014854 wait complete. 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.400+0000 7fedf797a740 1 -- start start 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 Processor -- start 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a365d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a3d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ac990 con 0x560819f73f60 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1ac2c0 con 0x560819f8c4d0 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedec1a9500 con 0x560819f946b0 2024-04-25T13:58:12.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedf797a740 10 client.0 _ll_register_callbacks cb 0x7fedd400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fede1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a365d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47562/0 (socket says 172.21.15.94:47562) 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 -- 172.21.15.94:0/1562951537 learned_addr learned my addr 172.21.15.94:0/1562951537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedead9d640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a3d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 -- 172.21.15.94:0/1562951537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 msgr2=0x56081a3d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a3d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 -- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 msgr2=0x56081a365d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a365d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 -- 172.21.15.94:0/1562951537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081a366400 con 0x560819f73f60 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fede1d9b640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a365d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedea59c640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7feddc009850 tx=0x7feddc003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100748b1dc4e6b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.401+0000 7fedd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc035040 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7feddc004230 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc040040 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feddc004430 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 10 client.21396 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56081a3d6af0 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feddc003aa0 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7feddc052690 con 0x560819f73f60 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 10 client.21396 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 10 client.21396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 20 client.21396 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819fb5390 0x56081a3d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 4 client.21396 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.402+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7feddc040040 con 0x560819fb5390 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fede1d9b640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819fb5390 0x56081a3d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedd1ffb640 10 client.21396 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fede1d9b640 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819fb5390 0x56081a3d9070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56815fd9357360b4 server_cookie=975a3205e7adee87 in_seq=0 out_seq=0 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedd1ffb640 1 -- 172.21.15.94:0/1562951537 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7feddc040040 con 0x560819fb5390 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedd1ffb640 10 client.21396 handle_command_reply: tid=0 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 client.21396 shutdown 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 msgr2=0x56081a365850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7feddc009850 tx=0x7feddc003f40 comp rx=0 tx=0).stop 2024-04-25T13:58:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 shutdown_connections 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x560819fb5390 0x56081a3d9070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560819f946b0 0x56081a3d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedd1ffb640 0 client.21396 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819f8c4d0 0x56081a365d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 --2- 172.21.15.94:0/1562951537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560819f73f60 0x56081a365850 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 >> 172.21.15.94:0/1562951537 conn(0x560819fa94e0 msgr2=0x56081a351610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 shutdown_connections 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.403+0000 7fedf797a740 1 -- 172.21.15.94:0/1562951537 wait complete. 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf797a740 20 client.21396 trim_cache size 0 max 16384 2024-04-25T13:58:12.410 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 msgr2=0x7fedd80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 0x7fedd80b2080 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x56081a358860 tx=0x7fede4009500 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 msgr2=0x7fedec19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 0x7fedec19a780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9f0 tx=0x7fedd400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 shutdown_connections 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fedd80afb70 0x7fedd80b2080 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedec108930 0x7fedec1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedec103c30 0x7fedec19a780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 --2- 172.21.15.94:0/2081554089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedec1029f0 0x7fedec19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 >> 172.21.15.94:0/2081554089 conn(0x7fedec0fdc00 msgr2=0x7fedec06b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 shutdown_connections 2024-04-25T13:58:12.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.404+0000 7fedf09f3640 1 -- 172.21.15.94:0/2081554089 wait complete. 2024-04-25T13:58:12.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:58:12.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 -- 172.21.15.94:0/939320938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed40108920 msgr2=0x7fed4010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 --2- 172.21.15.94:0/939320938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed40108920 0x7fed4010ad40 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fed38009ca0 tx=0x7fed3802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 -- 172.21.15.94:0/939320938 shutdown_connections 2024-04-25T13:58:12.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 --2- 172.21.15.94:0/939320938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed40108920 0x7fed4010ad40 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 --2- 172.21.15.94:0/939320938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40103c20 0x7fed40104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 --2- 172.21.15.94:0/939320938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed401029e0 0x7fed401036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 -- 172.21.15.94:0/939320938 >> 172.21.15.94:0/939320938 conn(0x7fed400fdbd0 msgr2=0x7fed40100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 -- 172.21.15.94:0/939320938 shutdown_connections 2024-04-25T13:58:12.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.545+0000 7fed45e51640 1 -- 172.21.15.94:0/939320938 wait complete. 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 Processor -- start 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 -- start start 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 0x7fed40074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40108920 0x7fed401aa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed4006cc80 con 0x7fed401029e0 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed4006cb00 con 0x7fed40103c20 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed45e51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed4006ce00 con 0x7fed40108920 2024-04-25T13:58:12.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.546+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47582/0 (socket says 172.21.15.94:47582) 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 -- 172.21.15.94:0/3400252111 learned_addr learned my addr 172.21.15.94:0/3400252111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3ffff640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40108920 0x7fed401aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3effd640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 0x7fed40074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 -- 172.21.15.94:0/3400252111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40108920 msgr2=0x7fed401aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40108920 0x7fed401aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 -- 172.21.15.94:0/3400252111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 msgr2=0x7fed40074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 0x7fed40074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 -- 172.21.15.94:0/3400252111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed401aa800 con 0x7fed401029e0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3effd640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 0x7fed40074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fed30009ec0 tx=0x7fed3000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3cff9640 1 -- 172.21.15.94:0/3400252111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000cd80 con 0x7fed401029e0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed3cff9640 1 -- 172.21.15.94:0/3400252111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed300137d0 con 0x7fed401029e0 2024-04-25T13:58:12.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.547+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed400fe180 con 0x7fed401029e0 2024-04-25T13:58:12.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.548+0000 7fed3cff9640 1 -- 172.21.15.94:0/3400252111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003850 con 0x7fed401029e0 2024-04-25T13:58:12.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.548+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed400fe660 con 0x7fed401029e0 2024-04-25T13:58:12.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.551+0000 7fed3cff9640 1 -- 172.21.15.94:0/3400252111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fed30022020 con 0x7fed401029e0 2024-04-25T13:58:12.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 Processor -- start 2024-04-25T13:58:12.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 -- start start 2024-04-25T13:58:12.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded30020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede95c4d0 0x563eded305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563edec8bd80 con 0x563ede95c4d0 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563edec45a20 con 0x563ede985390 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed4cde9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563edec87d40 con 0x563ede9646b0 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3cff9640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 0x7fed280b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded30020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded30020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40290/0 (socket says 172.21.15.94:40290) 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42218/0 (socket says 172.21.15.94:42218) 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.552+0000 7fed3ffff640 1 -- 172.21.15.94:0/801570075 learned_addr learned my addr 172.21.15.94:0/801570075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede95c4d0 0x563eded305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3ffff640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 0x7fed280b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3cff9640 1 -- 172.21.15.94:0/3400252111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fed3000f420 con 0x7fed401029e0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 -- 172.21.15.94:0/801570075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 msgr2=0x563eded30020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded30020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 -- 172.21.15.94:0/801570075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede95c4d0 msgr2=0x563eded305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede95c4d0 0x563eded305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 -- 172.21.15.94:0/801570075 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edec50c30 con 0x563ede9646b0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.553+0000 7fed3effd640 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed34009c00 tx=0x7fed340117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed10a1ce09a24a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.554+0000 7fed25ffb640 1 -- 172.21.15.94:0/801570075 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34012450 con 0x563ede9646b0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.554+0000 7fed3ffff640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 0x7fed280b2100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fed38004af0 tx=0x7fed3802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.554+0000 7fed25ffb640 1 -- 172.21.15.94:0/801570075 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34012aa0 con 0x563ede9646b0 2024-04-25T13:58:12.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.554+0000 7fed25ffb640 1 -- 172.21.15.94:0/801570075 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400fa70 con 0x563ede9646b0 2024-04-25T13:58:12.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 -- 172.21.15.94:0/801570075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 msgr2=0x563eded31230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed34009c00 tx=0x7fed340117b0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 -- 172.21.15.94:0/801570075 shutdown_connections 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede9646b0 0x563eded31230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede95c4d0 0x563eded305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 --2- 172.21.15.94:0/801570075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded30020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 -- 172.21.15.94:0/801570075 >> 172.21.15.94:0/801570075 conn(0x563ede9794e0 msgr2=0x563eded2b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 -- 172.21.15.94:0/801570075 shutdown_connections 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.555+0000 7fed4cde9740 1 -- 172.21.15.94:0/801570075 wait complete. 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 -- start start 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 Processor -- start 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 0x563eded40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563eded31d40 con 0x563ede9646b0 2024-04-25T13:58:12.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563edec12dc0 con 0x563ede95c4d0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eded31ec0 con 0x563ede985390 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.556+0000 7fed4cde9740 10 client.0 _ll_register_callbacks cb 0x563edec2e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40298/0 (socket says 172.21.15.94:40298) 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 -- 172.21.15.94:0/2585135314 learned_addr learned my addr 172.21.15.94:0/2585135314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3effd640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3ffff640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 0x563eded40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 -- 172.21.15.94:0/2585135314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 msgr2=0x563eded40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 0x563eded40f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 -- 172.21.15.94:0/2585135314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 msgr2=0x563eded3a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 -- 172.21.15.94:0/2585135314 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ede99d3f0 con 0x563ede95c4d0 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3ffff640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 0x563eded40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3effd640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fed300a7350 tx=0x7fed30017890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3ce1040ebf6f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.557+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30071c50 con 0x563ede95c4d0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fed300a7560 con 0x563ede95c4d0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34012aa0 con 0x563ede95c4d0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed300ab9d0 con 0x563ede95c4d0 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 10 client.30833 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563edea677d0 con 0x563ede95c4d0 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed300ab2d0 con 0x563ede95c4d0 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fed300717b0 con 0x563ede95c4d0 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 10 client.30833 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 10 client.30833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 20 client.30833 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede943f60 0x563eded477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 4 client.30833 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.558+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed34012aa0 con 0x563ede943f60 2024-04-25T13:58:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.559+0000 7fed3effd640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede943f60 0x563eded477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.559+0000 7fed1effd640 10 client.30833 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:12.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.559+0000 7fed3effd640 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede943f60 0x563eded477e0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35d66b4649176f9 server_cookie=aaf10e2443ed3ef0 in_seq=0 out_seq=0 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.562+0000 7fed1effd640 1 -- 172.21.15.94:0/2585135314 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fed34012aa0 con 0x563ede943f60 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.562+0000 7fed1effd640 10 client.30833 handle_command_reply: tid=0 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 client.30833 shutdown 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 msgr2=0x563eded39b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fed300a7350 tx=0x7fed30017890 comp rx=0 tx=0).stop 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 shutdown_connections 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede943f60 0x563eded477e0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede985390 0x563eded40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed1effd640 0 client.30833 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 --2- 172.21.15.94:0/2585135314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede95c4d0 0x563eded39b30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 >> 172.21.15.94:0/2585135314 conn(0x563ede9794e0 msgr2=0x563eded2e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 shutdown_connections 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 1 -- 172.21.15.94:0/2585135314 wait complete. 2024-04-25T13:58:12.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.563+0000 7fed4cde9740 20 client.30833 trim_cache size 0 max 16384 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.577+0000 7fed4cde9740 1 Processor -- start 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.577+0000 7fed4cde9740 1 -- start start 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded2b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b1b00 con 0x563ede9646b0 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b3640 con 0x563ede985390 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed4cde9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b21d0 con 0x563ede95c4d0 2024-04-25T13:58:12.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47614/0 (socket says 172.21.15.94:47614) 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 -- 172.21.15.94:0/1097119656 learned_addr learned my addr 172.21.15.94:0/1097119656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded2b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3ffff640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 -- 172.21.15.94:0/1097119656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 msgr2=0x563eded2b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded2b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 -- 172.21.15.94:0/1097119656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 msgr2=0x563eded2e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded2e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 -- 172.21.15.94:0/1097119656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eded2c3c0 con 0x563ede9646b0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded2b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.578+0000 7fed3effd640 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fed34011cb0 tx=0x7fed34009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4e82c2519e62c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.579+0000 7fed257fa640 1 -- 172.21.15.94:0/1097119656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34009820 con 0x563ede9646b0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.579+0000 7fed257fa640 1 -- 172.21.15.94:0/1097119656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34012a50 con 0x563ede9646b0 2024-04-25T13:58:12.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.579+0000 7fed257fa640 1 -- 172.21.15.94:0/1097119656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340056b0 con 0x563ede9646b0 2024-04-25T13:58:12.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- 172.21.15.94:0/1097119656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 msgr2=0x563eded2b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fed34011cb0 tx=0x7fed34009b70 comp rx=0 tx=0).stop 2024-04-25T13:58:12.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- 172.21.15.94:0/1097119656 shutdown_connections 2024-04-25T13:58:12.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563eded2e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded2b7a0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1097119656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded2b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- 172.21.15.94:0/1097119656 >> 172.21.15.94:0/1097119656 conn(0x563ede9794e0 msgr2=0x563eded34880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- 172.21.15.94:0/1097119656 shutdown_connections 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- 172.21.15.94:0/1097119656 wait complete. 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 -- start start 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.580+0000 7fed4cde9740 1 Processor -- start 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563ededadb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b2f70 con 0x563ede9646b0 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b28a0 con 0x563ede985390 2024-04-25T13:58:12.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed401afae0 con 0x563ede95c4d0 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed4cde9740 10 client.0 _ll_register_callbacks cb 0x7fed30018fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42234/0 (socket says 172.21.15.94:42234) 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 -- 172.21.15.94:0/1389794254 learned_addr learned my addr 172.21.15.94:0/1389794254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47622/0 (socket says 172.21.15.94:47622) 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3ffff640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563ededadb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 -- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 msgr2=0x563ededadb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563ededadb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 -- 172.21.15.94:0/1389794254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 msgr2=0x563eded3da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:12.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 -- 172.21.15.94:0/1389794254 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ededae080 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3effd640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed3f7fe640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed30004770 tx=0x7fed30013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc4709596660efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.581+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300ab2d0 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fed30071be0 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34012a50 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed300af680 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 10 client.30532 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563ededaf1f0 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34012a50 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fed300a6cd0 con 0x563ede95c4d0 2024-04-25T13:58:12.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 10 client.30532 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 10 client.30532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 20 client.30532 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede9193a0 0x563ededb1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 4 client.30532 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563eded38f30 con 0x563ede9193a0 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.582+0000 7fed3effd640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede9193a0 0x563ededb1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.583+0000 7fed1effd640 10 client.30532 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:12.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.583+0000 7fed3effd640 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede9193a0 0x563ededb1b90 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efa9a92d18ba193 server_cookie=f010da9104bb294c in_seq=0 out_seq=0 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed1effd640 1 -- 172.21.15.94:0/1389794254 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563eded38f30 con 0x563ede9193a0 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed1effd640 10 client.30532 handle_command_reply: tid=0 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 client.30532 shutdown 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 msgr2=0x563eded3d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed30004770 tx=0x7fed30013f50 comp rx=0 tx=0).stop 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 shutdown_connections 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563ede9193a0 0x563ededb1b90 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ede985390 0x563ededadb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ede9646b0 0x563eded3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed1effd640 0 client.30532 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 --2- 172.21.15.94:0/1389794254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ede95c4d0 0x563eded3d530 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 >> 172.21.15.94:0/1389794254 conn(0x563ede9794e0 msgr2=0x563eded2b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 shutdown_connections 2024-04-25T13:58:12.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 1 -- 172.21.15.94:0/1389794254 wait complete. 2024-04-25T13:58:12.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.587+0000 7fed4cde9740 20 client.30532 trim_cache size 0 max 16384 2024-04-25T13:58:12.591 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 msgr2=0x7fed280b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 0x7fed280b2100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fed38004af0 tx=0x7fed3802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 msgr2=0x7fed40073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fed30009ec0 tx=0x7fed3000edc0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 shutdown_connections 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fed280afbf0 0x7fed280b2100 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed40108920 0x7fed401aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103c20 0x7fed40074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 --2- 172.21.15.94:0/3400252111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed401029e0 0x7fed40073f50 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:12.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 >> 172.21.15.94:0/3400252111 conn(0x7fed400fdbd0 msgr2=0x7fed40069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:12.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 shutdown_connections 2024-04-25T13:58:12.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:12.588+0000 7fed45e51640 1 -- 172.21.15.94:0/3400252111 wait complete. 2024-04-25T13:58:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:12 smithi137 ceph-mon[32568]: pgmap v842: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-04-25T13:58:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:12 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/8262253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:12 smithi094 ceph-mon[29195]: pgmap v842: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-04-25T13:58:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:12 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/8262253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:12 smithi138 ceph-mon[32977]: pgmap v842: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.0 MiB/s wr, 586 op/s 2024-04-25T13:58:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/8262253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:13.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 64296 19.29 MB/sec execute 55 sec latency 61.586 ms 2024-04-25T13:58:13.600 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:58:13.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:58:13.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:58:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 -- 172.21.15.94:0/2792455083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 msgr2=0x7fa5b81037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/2792455083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b81037c0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009900 tx=0x7fa5a802a410 comp rx=0 tx=0).stop 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 -- 172.21.15.94:0/2792455083 shutdown_connections 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/2792455083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b81080b0 0x7fa5b810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/2792455083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b81037c0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/2792455083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b8102170 0x7fa5b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 -- 172.21.15.94:0/2792455083 >> 172.21.15.94:0/2792455083 conn(0x7fa5b80fd340 msgr2=0x7fa5b80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 -- 172.21.15.94:0/2792455083 shutdown_connections 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.722+0000 7fa5bfea8640 1 -- 172.21.15.94:0/2792455083 wait complete. 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 Processor -- start 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 -- start start 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b8102170 0x7fa5b8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 0x7fa5b81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810dc90 con 0x7fa5b81033b0 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810db10 con 0x7fa5b8102170 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bfea8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b810de10 con 0x7fa5b81080b0 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39286/0 (socket says 172.21.15.94:39286) 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bd443640 1 -- 172.21.15.94:0/272512978 learned_addr learned my addr 172.21.15.94:0/272512978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.723+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b8102170 0x7fa5b8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5be445640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 0x7fa5b81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 -- 172.21.15.94:0/272512978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 msgr2=0x7fa5b81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 0x7fa5b81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 -- 172.21.15.94:0/272512978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b8102170 msgr2=0x7fa5b8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b8102170 0x7fa5b8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 -- 172.21.15.94:0/272512978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b81a0c80 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5be445640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 0x7fa5b81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bd443640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006010 tx=0x7fa5a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5aeffd640 1 -- 172.21.15.94:0/272512978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a803f030 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5aeffd640 1 -- 172.21.15.94:0/272512978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a8031910 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b81a0f10 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5aeffd640 1 -- 172.21.15.94:0/272512978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8035070 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.724+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b81a1320 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5aeffd640 1 -- 172.21.15.94:0/272512978 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa5a8042030 con 0x7fa5b81033b0 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 Processor -- start 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 -- start start 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce3e410 0x55565d1df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce1d730 0x55565d1e6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55565d144b00 con 0x55565ce15550 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55565d0e0920 con 0x55565ce1d730 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55565d140ac0 con 0x55565ce3e410 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.727+0000 7fa5aeffd640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 0x7fa5a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39288/0 (socket says 172.21.15.94:39288) 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 -- 172.21.15.94:0/1953772457 learned_addr learned my addr 172.21.15.94:0/1953772457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bd443640 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce1d730 0x55565d1e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5be445640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 0x7fa5a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5be445640 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce3e410 0x55565d1df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5aeffd640 1 -- 172.21.15.94:0/272512978 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa5a80ff2b0 con 0x7fa5b81033b0 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 -- 172.21.15.94:0/1953772457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce3e410 msgr2=0x55565d1df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce3e410 0x55565d1df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 -- 172.21.15.94:0/1953772457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce1d730 msgr2=0x55565d1e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce1d730 0x55565d1e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 -- 172.21.15.94:0/1953772457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565cf1e7e0 con 0x55565ce15550 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5be445640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 0x7fa5a40b2000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55565d1e0670 tx=0x7fa5b00098e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009c60 tx=0x7fa5b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca23212a683460d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5a3fff640 1 -- 172.21.15.94:0/1953772457 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004220 con 0x55565ce15550 2024-04-25T13:58:13.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5a3fff640 1 -- 172.21.15.94:0/1953772457 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b40043c0 con 0x55565ce15550 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.728+0000 7fa5a3fff640 1 -- 172.21.15.94:0/1953772457 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4004c00 con 0x55565ce15550 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1953772457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 msgr2=0x55565d1dfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009c60 tx=0x7fa5b4011810 comp rx=0 tx=0).stop 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1953772457 shutdown_connections 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce1d730 0x55565d1e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1dfa20 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1953772457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce3e410 0x55565d1df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.729+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1953772457 >> 172.21.15.94:0/1953772457 conn(0x55565ce32560 msgr2=0x55565d1dae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1953772457 shutdown_connections 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1953772457 wait complete. 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- start start 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 Processor -- start 2024-04-25T13:58:13.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 0x55565d1e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55565d1e7370 con 0x55565ce15550 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55565d0de5d0 con 0x55565ce3e410 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55565d1e74f0 con 0x55565ce1d730 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.730+0000 7fa5c6e36740 10 client.0 _ll_register_callbacks cb 0x55565d0cc080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bdc44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 0x55565d1e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bdc44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39298/0 (socket says 172.21.15.94:39298) 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bdc44640 1 -- 172.21.15.94:0/3292493731 learned_addr learned my addr 172.21.15.94:0/3292493731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5be445640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 -- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 msgr2=0x55565d1f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 -- 172.21.15.94:0/3292493731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 msgr2=0x55565d1e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 -- 172.21.15.94:0/3292493731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565cf7ef70 con 0x55565ce1d730 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5bd443640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 0x55565d1e97d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80fd080 tx=0x7fa5a8030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90046f89e65e3b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8049040 con 0x55565ce1d730 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b40043c0 con 0x55565ce1d730 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa5a80c3ce0 con 0x55565ce1d730 2024-04-25T13:58:13.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa5a80044e0 con 0x55565ce1d730 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.731+0000 7fa5c6e36740 10 client.30544 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55565ce16170 con 0x55565ce1d730 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b40043c0 con 0x55565ce1d730 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa5a80b0070 con 0x55565ce1d730 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 10 client.30544 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 10 client.30544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 20 client.30544 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdfd1f0 0x55565d1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 4 client.30544 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55565d152710 con 0x55565cdfd1f0 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.732+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdfd1f0 0x55565d1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.733+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdfd1f0 0x55565d1f6c90 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789f2a8b50c972cc server_cookie=782107c7208919a3 in_seq=0 out_seq=0 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.733+0000 7fa5a17fa640 10 client.30544 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.736+0000 7fa5a17fa640 1 -- 172.21.15.94:0/3292493731 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55565d152710 con 0x55565cdfd1f0 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.736+0000 7fa5a17fa640 10 client.30544 handle_command_reply: tid=0 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.736+0000 7fa5c6e36740 1 client.30544 shutdown 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.736+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 msgr2=0x55565d1e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.736+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 0x55565d1e97d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80fd080 tx=0x7fa5a8030000 comp rx=0 tx=0).stop 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 shutdown_connections 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdfd1f0 0x55565d1f6c90 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce1d730 0x55565d1e97d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/3292493731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce15550 0x55565d1e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5a17fa640 0 client.30544 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 >> 172.21.15.94:0/3292493731 conn(0x55565ce32560 msgr2=0x55565d1d9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 shutdown_connections 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 1 -- 172.21.15.94:0/3292493731 wait complete. 2024-04-25T13:58:13.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.737+0000 7fa5c6e36740 20 client.30544 trim_cache size 0 max 16384 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 Processor -- start 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 -- start start 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aa360 con 0x55565ce1d730 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81abea0 con 0x55565ce3e410 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aaa30 con 0x55565ce15550 2024-04-25T13:58:13.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39314/0 (socket says 172.21.15.94:39314) 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 -- 172.21.15.94:0/366287898 learned_addr learned my addr 172.21.15.94:0/366287898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5be445640 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 -- 172.21.15.94:0/366287898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 msgr2=0x55565d1e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1e3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 -- 172.21.15.94:0/366287898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 msgr2=0x55565d1f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.756+0000 7fa5bd443640 1 -- 172.21.15.94:0/366287898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565d1e82c0 con 0x55565ce1d730 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.757+0000 7fa5bd443640 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fa5a803b5a0 tx=0x7fa5a80c3660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5450f913d3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.757+0000 7fa5a37fe640 1 -- 172.21.15.94:0/366287898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8049070 con 0x55565ce1d730 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.757+0000 7fa5a37fe640 1 -- 172.21.15.94:0/366287898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a80c7200 con 0x55565ce1d730 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.757+0000 7fa5a37fe640 1 -- 172.21.15.94:0/366287898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80c3ce0 con 0x55565ce1d730 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- 172.21.15.94:0/366287898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 msgr2=0x55565d1e2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fa5a803b5a0 tx=0x7fa5a80c3660 comp rx=0 tx=0).stop 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- 172.21.15.94:0/366287898 shutdown_connections 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1e2dc0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/366287898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- 172.21.15.94:0/366287898 >> 172.21.15.94:0/366287898 conn(0x55565ce32560 msgr2=0x55565d1e26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- 172.21.15.94:0/366287898 shutdown_connections 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- 172.21.15.94:0/366287898 wait complete. 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- start start 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 Processor -- start 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ab7d0 con 0x55565ce1d730 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ab100 con 0x55565ce3e410 2024-04-25T13:58:13.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.758+0000 7fa5c6e36740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81a8340 con 0x55565ce15550 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5c6e36740 10 client.0 _ll_register_callbacks cb 0x7fa5a80c35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39324/0 (socket says 172.21.15.94:39324) 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 -- 172.21.15.94:0/1445730717 learned_addr learned my addr 172.21.15.94:0/1445730717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5be445640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 -- 172.21.15.94:0/1445730717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 msgr2=0x55565d1f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 -- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 msgr2=0x55565d1edf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1edf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 -- 172.21.15.94:0/1445730717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565d25d030 con 0x55565ce1d730 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5bd443640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8032af0 tx=0x7fa5a80b0320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f645ada6d89afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8049070 con 0x55565ce1d730 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fa5a80e8050 con 0x55565ce1d730 2024-04-25T13:58:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a80c7200 con 0x55565ce1d730 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa5a80c7040 con 0x55565ce1d730 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5c6e36740 10 client.21432 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.759+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55565d25d030 con 0x55565ce1d730 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a80c7200 con 0x55565ce1d730 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa5a80aeb90 con 0x55565ce1d730 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 10 client.21432 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 10 client.21432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 20 client.21432 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdd2630 0x55565d260ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 4 client.21432 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55565d14c580 con 0x55565cdd2630 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdd2630 0x55565d260ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5a17fa640 10 client.21432 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.760+0000 7fa5bdc44640 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdd2630 0x55565d260ad0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53250711fb7dab7 server_cookie=3166798237cd5b8c in_seq=0 out_seq=0 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.763+0000 7fa5a17fa640 1 -- 172.21.15.94:0/1445730717 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55565d14c580 con 0x55565cdd2630 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.763+0000 7fa5a17fa640 10 client.21432 handle_command_reply: tid=0 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.763+0000 7fa5c6e36740 1 client.21432 shutdown 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.763+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 msgr2=0x55565d1ed9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.763+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8032af0 tx=0x7fa5a80b0320 comp rx=0 tx=0).stop 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 shutdown_connections 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565cdd2630 0x55565d260ad0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565ce3e410 0x55565d1edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565ce1d730 0x55565d1ed9f0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 --2- 172.21.15.94:0/1445730717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565ce15550 0x55565d1f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5a17fa640 0 client.21432 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 >> 172.21.15.94:0/1445730717 conn(0x55565ce32560 msgr2=0x55565d1ddc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 shutdown_connections 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 1 -- 172.21.15.94:0/1445730717 wait complete. 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5c6e36740 20 client.21432 trim_cache size 0 max 16384 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "13327f9b-12d0-40bc-b853-e90e8cc67006", 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:58:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 msgr2=0x7fa5a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 0x7fa5a40b2000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55565d1e0670 tx=0x7fa5b00098e0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 msgr2=0x7fa5b8199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006010 tx=0x7fa5a80042f0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 shutdown_connections 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5a40afaf0 0x7fa5a40b2000 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b81080b0 0x7fa5b81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b81033b0 0x7fa5b8199680 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 --2- 172.21.15.94:0/272512978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b8102170 0x7fa5b8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 >> 172.21.15.94:0/272512978 conn(0x7fa5b80fd340 msgr2=0x7fa5b80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 shutdown_connections 2024-04-25T13:58:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.764+0000 7fa5bfea8640 1 -- 172.21.15.94:0/272512978 wait complete. 2024-04-25T13:58:13.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:58:13.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:58:13.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 -- 172.21.15.94:0/3844432566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc105cf0 msgr2=0x7fd5dc1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/3844432566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc1069f0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009900 tx=0x7fd5cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 -- 172.21.15.94:0/3844432566 shutdown_connections 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/3844432566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/3844432566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc106f30 0x7fd5dc107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/3844432566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc1069f0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 -- 172.21.15.94:0/3844432566 >> 172.21.15.94:0/3844432566 conn(0x7fd5dc100f00 msgr2=0x7fd5dc103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 -- 172.21.15.94:0/3844432566 shutdown_connections 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.951+0000 7fd5e1a49640 1 -- 172.21.15.94:0/3844432566 wait complete. 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 Processor -- start 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 -- start start 2024-04-25T13:58:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc106f30 0x7fd5dc19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1117f0 con 0x7fd5dc10bc30 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc111670 con 0x7fd5dc105cf0 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5e1a49640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc111970 con 0x7fd5dc106f30 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5db7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39338/0 (socket says 172.21.15.94:39338) 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5db7fe640 1 -- 172.21.15.94:0/811426268 learned_addr learned my addr 172.21.15.94:0/811426268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5daffd640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.952+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc106f30 0x7fd5dc19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 -- 172.21.15.94:0/811426268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc106f30 msgr2=0x7fd5dc19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc106f30 0x7fd5dc19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 -- 172.21.15.94:0/811426268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc105cf0 msgr2=0x7fd5dc19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 -- 172.21.15.94:0/811426268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc1a3f90 con 0x7fd5dc10bc30 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400cad0 tx=0x7fd5d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5e0a47640 1 -- 172.21.15.94:0/811426268 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400ae20 con 0x7fd5dc10bc30 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5dc1a4280 con 0x7fd5dc10bc30 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5e0a47640 1 -- 172.21.15.94:0/811426268 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4005020 con 0x7fd5dc10bc30 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5e0a47640 1 -- 172.21.15.94:0/811426268 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400f6b0 con 0x7fd5dc10bc30 2024-04-25T13:58:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.953+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5dc1a47d0 con 0x7fd5dc10bc30 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.955+0000 7fd5e89d1740 1 Processor -- start 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e0a47640 1 -- 172.21.15.94:0/811426268 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd5d4004b20 con 0x7fd5dc10bc30 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 -- start start 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 0x557aa95b9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557aa951ed80 con 0x557aa91f76b0 2024-04-25T13:58:13.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557aa94d88b0 con 0x557aa91ef4d0 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557aa951ad40 con 0x557aa9218390 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5daffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 0x557aa95b9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39348/0 (socket says 172.21.15.94:39348) 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 -- 172.21.15.94:0/427781574 learned_addr learned my addr 172.21.15.94:0/427781574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 -- 172.21.15.94:0/427781574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 msgr2=0x557aa95b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.956+0000 7fd5d3fff640 1 -- 172.21.15.94:0/427781574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 msgr2=0x557aa95b9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 0x557aa95b9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d3fff640 1 -- 172.21.15.94:0/427781574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa94a9a40 con 0x557aa91f76b0 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5daffd640 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 0x557aa95b9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:13.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5e0a47640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 0x7fd5c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009fd0 tx=0x7fd5c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6a71659acf6a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5daffd640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 0x7fd5c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5e0a47640 1 -- 172.21.15.94:0/811426268 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd5d40debf0 con 0x7fd5dc10bc30 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d1ffb640 1 -- 172.21.15.94:0/427781574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004030 con 0x557aa91f76b0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d1ffb640 1 -- 172.21.15.94:0/427781574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c4005020 con 0x557aa91f76b0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.957+0000 7fd5d1ffb640 1 -- 172.21.15.94:0/427781574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004b90 con 0x557aa91f76b0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- 172.21.15.94:0/427781574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 msgr2=0x557aa95bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009fd0 tx=0x7fd5c4004730 comp rx=0 tx=0).stop 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5daffd640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 0x7fd5c80b2000 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc005ab0 tx=0x7fd5cc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- 172.21.15.94:0/427781574 shutdown_connections 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95bac70 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91ef4d0 0x557aa95b9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/427781574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- 172.21.15.94:0/427781574 >> 172.21.15.94:0/427781574 conn(0x557aa920c4e0 msgr2=0x557aa95b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- 172.21.15.94:0/427781574 shutdown_connections 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- 172.21.15.94:0/427781574 wait complete. 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.958+0000 7fd5e89d1740 1 -- start start 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 Processor -- start 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 0x557aa95c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557aa95bb780 con 0x557aa91ef4d0 2024-04-25T13:58:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557aa94a5d10 con 0x557aa9218390 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557aa95bb900 con 0x557aa91f76b0 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5e89d1740 10 client.0 _ll_register_callbacks cb 0x557aa94d2210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5d3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 0x557aa95c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39362/0 (socket says 172.21.15.94:39362) 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5daffd640 1 -- 172.21.15.94:0/1015532008 learned_addr learned my addr 172.21.15.94:0/1015532008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.959+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 -- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 msgr2=0x557aa95cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 -- 172.21.15.94:0/1015532008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 msgr2=0x557aa95c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 -- 172.21.15.94:0/1015532008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa92fa7d0 con 0x557aa91f76b0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5daffd640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 0x557aa95c3ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009fd0 tx=0x7fd5c4005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5412b4b4dfe23828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004310 con 0x557aa91f76b0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c4005020 con 0x557aa91f76b0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd5c4004910 con 0x557aa91f76b0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd5c40044b0 con 0x557aa91f76b0 2024-04-25T13:58:13.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5e89d1740 10 client.30559 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.960+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557aa92303f0 con 0x557aa91f76b0 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c4005020 con 0x557aa91f76b0 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd5c40035f0 con 0x557aa91f76b0 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 10 client.30559 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 10 client.30559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 20 client.30559 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91d6f60 0x557aa95d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 4 client.30559 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aa952c990 con 0x557aa91d6f60 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5daffd640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91d6f60 0x557aa95d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.961+0000 7fd5baffd640 10 client.30559 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.962+0000 7fd5daffd640 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91d6f60 0x557aa95d1030 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74b33b2f7355a8a server_cookie=1b79c263bfabbabe in_seq=0 out_seq=0 2024-04-25T13:58:13.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5baffd640 1 -- 172.21.15.94:0/1015532008 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557aa952c990 con 0x557aa91d6f60 2024-04-25T13:58:13.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5baffd640 10 client.30559 handle_command_reply: tid=0 2024-04-25T13:58:13.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 client.30559 shutdown 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 msgr2=0x557aa95c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 0x557aa95c3ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009fd0 tx=0x7fd5c4005780 comp rx=0 tx=0).stop 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 shutdown_connections 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91d6f60 0x557aa95d1030 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91f76b0 0x557aa95c3ec0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5baffd640 0 client.30559 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/1015532008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 >> 172.21.15.94:0/1015532008 conn(0x557aa920c4e0 msgr2=0x557aa95b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 shutdown_connections 2024-04-25T13:58:13.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 1 -- 172.21.15.94:0/1015532008 wait complete. 2024-04-25T13:58:13.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.964+0000 7fd5e89d1740 20 client.30559 trim_cache size 0 max 16384 2024-04-25T13:58:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.983+0000 7fd5e89d1740 1 Processor -- start 2024-04-25T13:58:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.983+0000 7fd5e89d1740 1 -- start start 2024-04-25T13:58:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.983+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91f76b0 0x557aa95b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1ad780 con 0x557aa91ef4d0 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1af2c0 con 0x557aa91f76b0 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1ade50 con 0x557aa9218390 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39372/0 (socket says 172.21.15.94:39372) 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 -- 172.21.15.94:0/2799805775 learned_addr learned my addr 172.21.15.94:0/2799805775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91f76b0 0x557aa95b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 -- 172.21.15.94:0/2799805775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 msgr2=0x557aa95b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 -- 172.21.15.94:0/2799805775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91f76b0 msgr2=0x557aa95b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91f76b0 0x557aa95b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 -- 172.21.15.94:0/2799805775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa91f00f0 con 0x557aa91ef4d0 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5daffd640 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009900 tx=0x7fd5cc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b443b3e4c0a7930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5d17fa640 1 -- 172.21.15.94:0/2799805775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc035040 con 0x557aa91ef4d0 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5d17fa640 1 -- 172.21.15.94:0/2799805775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc0043c0 con 0x557aa91ef4d0 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.984+0000 7fd5d17fa640 1 -- 172.21.15.94:0/2799805775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc03d720 con 0x557aa91ef4d0 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 -- 172.21.15.94:0/2799805775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 msgr2=0x557aa95c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009900 tx=0x7fd5cc03f040 comp rx=0 tx=0).stop 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 -- 172.21.15.94:0/2799805775 shutdown_connections 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa9218390 0x557aa95b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa91f76b0 0x557aa95b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/2799805775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91ef4d0 0x557aa95c29f0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 -- 172.21.15.94:0/2799805775 >> 172.21.15.94:0/2799805775 conn(0x557aa920c4e0 msgr2=0x557aa95b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 -- 172.21.15.94:0/2799805775 shutdown_connections 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.985+0000 7fd5e89d1740 1 -- 172.21.15.94:0/2799805775 wait complete. 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 -- start start 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 Processor -- start 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91ef4d0 0x557aa95c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1aebf0 con 0x557aa91f76b0 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1ae520 con 0x557aa9218390 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1ab760 con 0x557aa91ef4d0 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5e89d1740 10 client.0 _ll_register_callbacks cb 0x7fd5cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39380/0 (socket says 172.21.15.94:39380) 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 -- 172.21.15.94:0/410553911 learned_addr learned my addr 172.21.15.94:0/410553911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5db7fe640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5daffd640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91ef4d0 0x557aa95c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 -- 172.21.15.94:0/410553911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91ef4d0 msgr2=0x557aa95c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91ef4d0 0x557aa95c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 -- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 msgr2=0x557aa95c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.986+0000 7fd5d3fff640 1 -- 172.21.15.94:0/410553911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa94df4b0 con 0x557aa91f76b0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5d3fff640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009820 tx=0x7fd5c4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece430d73ef7e846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004310 con 0x557aa91f76b0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fd5c40035f0 con 0x557aa91f76b0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc0043c0 con 0x557aa91f76b0 2024-04-25T13:58:13.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd5c4041050 con 0x557aa91f76b0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 10 client.21450 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557aa95cb580 con 0x557aa91f76b0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc0043c0 con 0x557aa91f76b0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd5c40048d0 con 0x557aa91f76b0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 10 client.21450 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 10 client.21450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 20 client.21450 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91ac3a0 0x557aa963b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 4 client.21450 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.987+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557aa9526800 con 0x557aa91ac3a0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.988+0000 7fd5daffd640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91ac3a0 0x557aa963b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.988+0000 7fd5baffd640 10 client.21450 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.988+0000 7fd5daffd640 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91ac3a0 0x557aa963b6d0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353451fa27135a23 server_cookie=269dd5b3edfb8cf1 in_seq=0 out_seq=0 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5baffd640 1 -- 172.21.15.94:0/410553911 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x557aa9526800 con 0x557aa91ac3a0 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5baffd640 10 client.21450 handle_command_reply: tid=0 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 client.21450 shutdown 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 msgr2=0x557aa95c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009820 tx=0x7fd5c4005710 comp rx=0 tx=0).stop 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 shutdown_connections 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557aa91ac3a0 0x557aa963b6d0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557aa9218390 0x557aa95c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557aa91f76b0 0x557aa95c9040 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 --2- 172.21.15.94:0/410553911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557aa91ef4d0 0x557aa95c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.990+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 >> 172.21.15.94:0/410553911 conn(0x557aa920c4e0 msgr2=0x557aa95c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5baffd640 0 client.21450 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 shutdown_connections 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e89d1740 1 -- 172.21.15.94:0/410553911 wait complete. 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e89d1740 20 client.21450 trim_cache size 0 max 16384 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (90 inodes in the stack)", 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stdout: "13327f9b-12d0-40bc-b853-e90e8cc67006": { 2024-04-25T13:58:13.996 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:58:13.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "13327f9b-12d0-40bc-b853-e90e8cc67006", 2024-04-25T13:58:13.997 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:58:13.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:58:13.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:58:13.997 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 msgr2=0x7fd5c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 0x7fd5c80b2000 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc005ab0 tx=0x7fd5cc031400 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 msgr2=0x7fd5dc1a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400cad0 tx=0x7fd5d400cf90 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 shutdown_connections 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd5c80afaf0 0x7fd5c80b2000 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5dc10bc30 0x7fd5dc1a3920 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5dc106f30 0x7fd5dc19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 --2- 172.21.15.94:0/811426268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5dc105cf0 0x7fd5dc19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 >> 172.21.15.94:0/811426268 conn(0x7fd5dc100f00 msgr2=0x7fd5dc107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:13.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 shutdown_connections 2024-04-25T13:58:13.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:13.991+0000 7fd5e1a49640 1 -- 172.21.15.94:0/811426268 wait complete. 2024-04-25T13:58:14.002 INFO:tasks.cephfs.filesystem:scrub status for tag:13327f9b-12d0-40bc-b853-e90e8cc67006 - {'path': '/', 'tag': '13327f9b-12d0-40bc-b853-e90e8cc67006', 'options': 'recursive,force'} 2024-04-25T13:58:14.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 64592 19.32 MB/sec execute 56 sec latency 319.721 ms 2024-04-25T13:58:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:14 smithi137 ceph-mon[32568]: pgmap v843: 97 pgs: 97 active+clean; 252 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T13:58:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:14 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:58:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:14 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:14.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:14 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:14 smithi094 ceph-mon[29195]: pgmap v843: 97 pgs: 97 active+clean; 252 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T13:58:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:14 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:58:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:14 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:14.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:14 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:14 smithi138 ceph-mon[32977]: pgmap v843: 97 pgs: 97 active+clean; 252 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T13:58:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:14 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:58:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:14 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:14.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:14 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 64881 19.21 MB/sec execute 57 sec latency 341.572 ms 2024-04-25T13:58:15.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:15.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:15.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:15.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:16.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 65290 19.23 MB/sec execute 58 sec latency 65.957 ms 2024-04-25T13:58:16.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:16 smithi094 ceph-mon[29195]: pgmap v844: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.3 MiB/s wr, 588 op/s 2024-04-25T13:58:16.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:16 smithi137 ceph-mon[32568]: pgmap v844: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.3 MiB/s wr, 588 op/s 2024-04-25T13:58:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:16 smithi138 ceph-mon[32977]: pgmap v844: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.3 MiB/s wr, 588 op/s 2024-04-25T13:58:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:17.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 65725 19.29 MB/sec execute 59 sec latency 101.392 ms 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 -- 172.21.15.94:0/3436907423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 msgr2=0x7ff93410e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 --2- 172.21.15.94:0/3436907423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff93410e0a0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009ca0 tx=0x7ff92c02a8e0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 -- 172.21.15.94:0/3436907423 shutdown_connections 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 --2- 172.21.15.94:0/3436907423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff93410e0a0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 --2- 172.21.15.94:0/3436907423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934106f80 0x7ff934107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 --2- 172.21.15.94:0/3436907423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934105d40 0x7ff934106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 -- 172.21.15.94:0/3436907423 >> 172.21.15.94:0/3436907423 conn(0x7ff934100f10 msgr2=0x7ff9341033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 -- 172.21.15.94:0/3436907423 shutdown_connections 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.605+0000 7ff93a172640 1 -- 172.21.15.94:0/3436907423 wait complete. 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.606+0000 7ff93a172640 1 Processor -- start 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.606+0000 7ff93a172640 1 -- start start 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934106f80 0x7ff9341a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff9341b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff934078a70 con 0x7ff93410bc80 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340788f0 con 0x7ff934106f80 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff93a172640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff934078bf0 con 0x7ff934105d40 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53452/0 (socket says 172.21.15.94:53452) 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 -- 172.21.15.94:0/2527125307 learned_addr learned my addr 172.21.15.94:0/2527125307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:17.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff9341b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff932ffd640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934106f80 0x7ff9341a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 -- 172.21.15.94:0/2527125307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934106f80 msgr2=0x7ff9341a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934106f80 0x7ff9341a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 -- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 msgr2=0x7ff9341b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff9341b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.607+0000 7ff9337fe640 1 -- 172.21.15.94:0/2527125307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341b0b50 con 0x7ff934105d40 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.608+0000 7ff9337fe640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff924006fd0 tx=0x7ff92400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.609+0000 7ff930ff9640 1 -- 172.21.15.94:0/2527125307 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924011830 con 0x7ff934105d40 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.609+0000 7ff930ff9640 1 -- 172.21.15.94:0/2527125307 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff924016070 con 0x7ff934105d40 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.609+0000 7ff930ff9640 1 -- 172.21.15.94:0/2527125307 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924010e00 con 0x7ff934105d40 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.609+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341b0e40 con 0x7ff934105d40 2024-04-25T13:58:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.609+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341b12d0 con 0x7ff934105d40 2024-04-25T13:58:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.612+0000 7ff94101d740 1 Processor -- start 2024-04-25T13:58:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff930ff9640 1 -- 172.21.15.94:0/2527125307 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff92401b460 con 0x7ff934105d40 2024-04-25T13:58:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 -- start start 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b136def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a4b12d3460 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4b128d590 con 0x55a4b0fa42f0 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff94101d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4b12cf420 con 0x55a4b0fac4d0 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff932ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff932ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53456/0 (socket says 172.21.15.94:53456) 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.613+0000 7ff932ffd640 1 -- 172.21.15.94:0/3521470892 learned_addr learned my addr 172.21.15.94:0/3521470892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b136def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff933fff640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff930ff9640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 0x7ff91c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 -- 172.21.15.94:0/3521470892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 msgr2=0x55a4b136def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b136def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 -- 172.21.15.94:0/3521470892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 msgr2=0x55a4b136d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 -- 172.21.15.94:0/3521470892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b129a0d0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff933fff640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff933fff640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 0x7ff91c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff930ff9640 1 -- 172.21.15.94:0/2527125307 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff9240de1d0 con 0x7ff934105d40 2024-04-25T13:58:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.614+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff928009ca0 tx=0x7ff928011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8e6265e983f68f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.615+0000 7ff919ffb640 1 -- 172.21.15.94:0/3521470892 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928005220 con 0x55a4b0fac4d0 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.615+0000 7ff919ffb640 1 -- 172.21.15.94:0/3521470892 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff928004c50 con 0x55a4b0fac4d0 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.615+0000 7ff919ffb640 1 -- 172.21.15.94:0/3521470892 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280197a0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 -- 172.21.15.94:0/3521470892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 msgr2=0x55a4b1374df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff928009ca0 tx=0x7ff928011810 comp rx=0 tx=0).stop 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff933fff640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 0x7ff91c0b2070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55a4b136eb40 tx=0x7ff92c03e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 -- 172.21.15.94:0/3521470892 shutdown_connections 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fac4d0 0x55a4b1374df0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b136def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 --2- 172.21.15.94:0/3521470892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 -- 172.21.15.94:0/3521470892 >> 172.21.15.94:0/3521470892 conn(0x55a4b0fc1300 msgr2=0x55a4b1369300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 -- 172.21.15.94:0/3521470892 shutdown_connections 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.616+0000 7ff94101d740 1 -- 172.21.15.94:0/3521470892 wait complete. 2024-04-25T13:58:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 -- start start 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 Processor -- start 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b1377b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b137eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff94101d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a4b1375840 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.617+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b137eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53462/0 (socket says 172.21.15.94:53462) 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 -- 172.21.15.94:0/4091489968 learned_addr learned my addr 172.21.15.94:0/4091489968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff932ffd640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b1377b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4b125ab60 con 0x55a4b0fac4d0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4b13759c0 con 0x55a4b0fa42f0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff94101d740 10 client.0 _ll_register_callbacks cb 0x55a4b127f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 -- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 msgr2=0x55a4b1377b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b1377b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 -- 172.21.15.94:0/4091489968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 msgr2=0x55a4b137eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b137eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b0fe5210 con 0x55a4b0fa42f0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff928004c50 con 0x55a4b0fa42f0 2024-04-25T13:58:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff9337fe640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff92400bde0 tx=0x7ff9240a25f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec9a526b5ce2a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff912ffd640 10 client.30577 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff912ffd640 1 -- 172.21.15.94:0/4091489968 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240de030 con 0x55a4b0fa42f0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.618+0000 7ff912ffd640 1 -- 172.21.15.94:0/4091489968 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff9240aefe0 con 0x55a4b0fa42f0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff912ffd640 1 -- 172.21.15.94:0/4091489968 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff9240af200 con 0x55a4b0fa42f0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 10 client.30577 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55a4b10b1620 con 0x55a4b0fa42f0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff912ffd640 1 -- 172.21.15.94:0/4091489968 <== mon.1 v2:172.21.15.138:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff9240e1b80 con 0x55a4b0fa42f0 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 10 client.30577 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 10 client.30577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 20 client.30577 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f8bd80 0x55a4b1385150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 4 client.30577 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b12e1070 con 0x55a4b0f8bd80 2024-04-25T13:58:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.619+0000 7ff932ffd640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f8bd80 0x55a4b1385150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.620+0000 7ff912ffd640 10 client.30577 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.620+0000 7ff932ffd640 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f8bd80 0x55a4b1385150 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24605ddb53e2daea server_cookie=48f08c7fad6f2cc5 in_seq=0 out_seq=0 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.626+0000 7ff912ffd640 1 -- 172.21.15.94:0/4091489968 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4b12e1070 con 0x55a4b0f8bd80 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.626+0000 7ff912ffd640 10 client.30577 handle_command_reply: tid=0 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.626+0000 7ff94101d740 1 client.30577 shutdown 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 msgr2=0x55a4b1377610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff92400bde0 tx=0x7ff9240a25f0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 shutdown_connections 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f8bd80 0x55a4b1385150 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b137eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b1377b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 --2- 172.21.15.94:0/4091489968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1377610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 >> 172.21.15.94:0/4091489968 conn(0x55a4b0fc1300 msgr2=0x55a4b136c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff912ffd640 0 client.30577 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 shutdown_connections 2024-04-25T13:58:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 1 -- 172.21.15.94:0/4091489968 wait complete. 2024-04-25T13:58:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.627+0000 7ff94101d740 20 client.30577 trim_cache size 0 max 16384 2024-04-25T13:58:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 Processor -- start 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 -- start start 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1368c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b13691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341ba310 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341bbe50 con 0x55a4b0fac4d0 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.644+0000 7ff94101d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341ba9e0 con 0x55a4b0fa42f0 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39424/0 (socket says 172.21.15.94:39424) 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 -- 172.21.15.94:0/3089586953 learned_addr learned my addr 172.21.15.94:0/3089586953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b13691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1368c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 -- 172.21.15.94:0/3089586953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 msgr2=0x55a4b1368c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1368c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 -- 172.21.15.94:0/3089586953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 msgr2=0x55a4b13691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b13691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 -- 172.21.15.94:0/3089586953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b1374e50 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1368c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b13691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff933fff640 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff92c035000 tx=0x7ff92c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673ca51fe0088bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.645+0000 7ff9197fa640 1 -- 172.21.15.94:0/3089586953 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c030eb0 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff9197fa640 1 -- 172.21.15.94:0/3089586953 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c036050 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff9197fa640 1 -- 172.21.15.94:0/3089586953 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c045470 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 -- 172.21.15.94:0/3089586953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 msgr2=0x55a4b136c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff92c035000 tx=0x7ff92c041000 comp rx=0 tx=0).stop 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 -- 172.21.15.94:0/3089586953 shutdown_connections 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fcd1b0 0x55a4b136c100 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fac4d0 0x55a4b13691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 --2- 172.21.15.94:0/3089586953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fa42f0 0x55a4b1368c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 -- 172.21.15.94:0/3089586953 >> 172.21.15.94:0/3089586953 conn(0x55a4b0fc1300 msgr2=0x55a4b1371a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 -- 172.21.15.94:0/3089586953 shutdown_connections 2024-04-25T13:58:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.646+0000 7ff94101d740 1 -- 172.21.15.94:0/3089586953 wait complete. 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 -- start start 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 Processor -- start 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b137adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fcd1b0 0x55a4b13eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341bb780 con 0x55a4b0fac4d0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341bb0b0 con 0x55a4b0fa42f0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9341b82f0 con 0x55a4b0fcd1b0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff94101d740 10 client.0 _ll_register_callbacks cb 0x7ff9240e19a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff932ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff932ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39434/0 (socket says 172.21.15.94:39434) 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff932ffd640 1 -- 172.21.15.94:0/3778449035 learned_addr learned my addr 172.21.15.94:0/3778449035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b137adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.647+0000 7ff933fff640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fcd1b0 0x55a4b13eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 -- 172.21.15.94:0/3778449035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fcd1b0 msgr2=0x55a4b13eb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fcd1b0 0x55a4b13eb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 -- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 msgr2=0x55a4b137adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b137adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 -- 172.21.15.94:0/3778449035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b137e2b0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff932ffd640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff928009bc0 tx=0x7ff9280098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefdcffe36d07e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928004040 con 0x55a4b0fac4d0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff9280041e0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c036050 con 0x55a4b0fac4d0 2024-04-25T13:58:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff9280043e0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff94101d740 10 client.21468 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a4b13ec8d0 con 0x55a4b0fac4d0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.648+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c036050 con 0x55a4b0fac4d0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff928003780 con 0x55a4b0fac4d0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 10 client.21468 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 10 client.21468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 20 client.21468 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f611c0 0x55a4b13ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 4 client.21468 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4b12daee0 con 0x55a4b0f611c0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f611c0 0x55a4b13ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff9337fe640 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f611c0 0x55a4b13ef090 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b45b7680f02a048 server_cookie=29fdaf8d742ad5c6 in_seq=0 out_seq=0 2024-04-25T13:58:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:17.649+0000 7ff912ffd640 10 client.21468 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff912ffd640 1 -- 172.21.15.94:0/3778449035 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4b12daee0 con 0x55a4b0f611c0 2024-04-25T13:58:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff912ffd640 10 client.21468 handle_command_reply: tid=0 2024-04-25T13:58:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 client.21468 shutdown 2024-04-25T13:58:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 msgr2=0x55a4b137b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff928009bc0 tx=0x7ff9280098e0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 shutdown_connections 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a4b0f611c0 0x55a4b13ef090 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff912ffd640 0 client.21468 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4b0fcd1b0 0x55a4b13eb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a4b0fac4d0 0x55a4b137b2f0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 --2- 172.21.15.94:0/3778449035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4b0fa42f0 0x55a4b137adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.138+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 >> 172.21.15.94:0/3778449035 conn(0x55a4b0fc1300 msgr2=0x55a4b136c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 shutdown_connections 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff94101d740 1 -- 172.21.15.94:0/3778449035 wait complete. 2024-04-25T13:58:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff94101d740 20 client.21468 trim_cache size 0 max 16384 2024-04-25T13:58:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:58:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:58:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:58:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 msgr2=0x7ff91c0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 0x7ff91c0b2070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55a4b136eb40 tx=0x7ff92c03e040 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 msgr2=0x7ff9341a9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.139+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff924006fd0 tx=0x7ff92400b8e0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 shutdown_connections 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff91c0afb60 0x7ff91c0b2070 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93410bc80 0x7ff9341b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff934106f80 0x7ff9341a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 --2- 172.21.15.94:0/2527125307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff934105d40 0x7ff9341a9010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 >> 172.21.15.94:0/2527125307 conn(0x7ff934100f10 msgr2=0x7ff934101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 shutdown_connections 2024-04-25T13:58:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:18.140+0000 7ff93a172640 1 -- 172.21.15.94:0/2527125307 wait complete. 2024-04-25T13:58:18.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 65937 19.14 MB/sec execute 60 sec latency 297.652 ms 2024-04-25T13:58:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:18 smithi137 ceph-mon[32568]: pgmap v845: 97 pgs: 97 active+clean; 257 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 514 op/s 2024-04-25T13:58:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:18 smithi094 ceph-mon[29195]: pgmap v845: 97 pgs: 97 active+clean; 257 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 514 op/s 2024-04-25T13:58:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:18 smithi138 ceph-mon[32977]: pgmap v845: 97 pgs: 97 active+clean; 257 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 514 op/s 2024-04-25T13:58:19.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 66255 19.19 MB/sec execute 61 sec latency 203.344 ms 2024-04-25T13:58:20.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 66630 19.25 MB/sec execute 62 sec latency 143.562 ms 2024-04-25T13:58:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:20 smithi137 ceph-mon[32568]: pgmap v846: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 459 op/s 2024-04-25T13:58:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:20 smithi094 ceph-mon[29195]: pgmap v846: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 459 op/s 2024-04-25T13:58:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:20 smithi138 ceph-mon[32977]: pgmap v846: 97 pgs: 97 active+clean; 256 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 459 op/s 2024-04-25T13:58:21.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 67009 19.19 MB/sec execute 63 sec latency 77.857 ms 2024-04-25T13:58:21.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:21 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:21.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:21 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:21 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:21.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:21 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:21 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:21.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:21 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:22.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 67356 19.16 MB/sec execute 64 sec latency 68.475 ms 2024-04-25T13:58:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:22 smithi137 ceph-mon[32568]: pgmap v847: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-04-25T13:58:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:22 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:22 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:22.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:22 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:22 smithi094 ceph-mon[29195]: pgmap v847: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-04-25T13:58:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:22 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:22 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:22 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:22 smithi138 ceph-mon[32977]: pgmap v847: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 697 op/s 2024-04-25T13:58:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:22 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:22 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:22 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:23.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 67791 19.25 MB/sec execute 65 sec latency 42.123 ms 2024-04-25T13:58:24.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 68137 19.29 MB/sec execute 66 sec latency 133.516 ms 2024-04-25T13:58:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:24 smithi137 ceph-mon[32568]: pgmap v848: 97 pgs: 97 active+clean; 246 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 544 op/s 2024-04-25T13:58:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:24 smithi094 ceph-mon[29195]: pgmap v848: 97 pgs: 97 active+clean; 246 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 544 op/s 2024-04-25T13:58:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:24 smithi138 ceph-mon[32977]: pgmap v848: 97 pgs: 97 active+clean; 246 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 544 op/s 2024-04-25T13:58:25.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 68450 19.16 MB/sec execute 67 sec latency 241.172 ms 2024-04-25T13:58:26.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 68832 19.23 MB/sec execute 68 sec latency 70.872 ms 2024-04-25T13:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:26 smithi137 ceph-mon[32568]: pgmap v849: 97 pgs: 97 active+clean; 247 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-04-25T13:58:26.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:26 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:26 smithi094 ceph-mon[29195]: pgmap v849: 97 pgs: 97 active+clean; 247 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-04-25T13:58:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:26 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:26 smithi138 ceph-mon[32977]: pgmap v849: 97 pgs: 97 active+clean; 247 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-04-25T13:58:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:26 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:27.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 69289 19.26 MB/sec execute 69 sec latency 102.288 ms 2024-04-25T13:58:27.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:27.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:58:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:27.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:27.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:58:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:58:27.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:27.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:27.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:58:28.140 INFO:tasks.workunit.client.0.smithi094.stdout: 10 69653 19.29 MB/sec execute 70 sec latency 75.366 ms 2024-04-25T13:58:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:28 smithi137 ceph-mon[32568]: pgmap v850: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-04-25T13:58:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:28 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:58:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:28 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:58:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:28 smithi094 ceph-mon[29195]: pgmap v850: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-04-25T13:58:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:28 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:58:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:28 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:58:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:28 smithi138 ceph-mon[32977]: pgmap v850: 97 pgs: 97 active+clean; 251 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-04-25T13:58:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:28 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T13:58:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:28 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:58:29.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 70131 19.45 MB/sec execute 71 sec latency 57.367 ms 2024-04-25T13:58:30.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 70395 19.39 MB/sec execute 72 sec latency 175.719 ms 2024-04-25T13:58:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:30 smithi137 ceph-mon[32568]: pgmap v851: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-04-25T13:58:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:30 smithi094 ceph-mon[29195]: pgmap v851: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-04-25T13:58:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:30 smithi138 ceph-mon[32977]: pgmap v851: 97 pgs: 97 active+clean; 250 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-04-25T13:58:31.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 70752 19.35 MB/sec execute 73 sec latency 90.353 ms 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 -- 172.21.15.94:0/3511377329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 msgr2=0x7f227410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 --2- 172.21.15.94:0/3511377329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f227410ada0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2264009ca0 tx=0x7f226402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 -- 172.21.15.94:0/3511377329 shutdown_connections 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 --2- 172.21.15.94:0/3511377329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f227410ada0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 --2- 172.21.15.94:0/3511377329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 0x7f2274104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 --2- 172.21.15.94:0/3511377329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 0x7f2274103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 -- 172.21.15.94:0/3511377329 >> 172.21.15.94:0/3511377329 conn(0x7f22740fdc30 msgr2=0x7f22741000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 -- 172.21.15.94:0/3511377329 shutdown_connections 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.242+0000 7f227b9c0640 1 -- 172.21.15.94:0/3511377329 wait complete. 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 Processor -- start 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 -- start start 2024-04-25T13:58:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 0x7f22741990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 0x7f22741995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f227410e570 con 0x7f2274108980 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f227410e3f0 con 0x7f2274102a40 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f227b9c0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f227410e6f0 con 0x7f2274103c80 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.243+0000 7f2279f5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35378/0 (socket says 172.21.15.94:35378) 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 -- 172.21.15.94:0/159640776 learned_addr learned my addr 172.21.15.94:0/159640776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2278f5b640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 0x7f22741995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f227975c640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 0x7f22741990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 -- 172.21.15.94:0/159640776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 msgr2=0x7f22741995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 0x7f22741995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 -- 172.21.15.94:0/159640776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 msgr2=0x7f22741990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 0x7f22741990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 -- 172.21.15.94:0/159640776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22741a0bf0 con 0x7f2274108980 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f2279f5d640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f2264004af0 tx=0x7f2264031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f22727fc640 1 -- 172.21.15.94:0/159640776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264003b70 con 0x7f2274108980 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f22727fc640 1 -- 172.21.15.94:0/159640776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2264003d10 con 0x7f2274108980 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22741a0e80 con 0x7f2274108980 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f22727fc640 1 -- 172.21.15.94:0/159640776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226402fb20 con 0x7f2274108980 2024-04-25T13:58:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.244+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741a12e0 con 0x7f2274108980 2024-04-25T13:58:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.247+0000 7f2282870740 1 Processor -- start 2024-04-25T13:58:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.247+0000 7f22727fc640 1 -- 172.21.15.94:0/159640776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2264004580 con 0x7f2274108980 2024-04-25T13:58:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 -- start start 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d104d0 0x5647d70d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5647d7037460 con 0x5647d6d082f0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5647d6ff1590 con 0x5647d6d311b0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2282870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d7033420 con 0x5647d6d104d0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35390/0 (socket says 172.21.15.94:35390) 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 -- 172.21.15.94:0/3517372779 learned_addr learned my addr 172.21.15.94:0/3517372779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2278f5b640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2278f5b640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d104d0 0x5647d70d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f22727fc640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 0x7f225c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 -- 172.21.15.94:0/3517372779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d104d0 msgr2=0x5647d70d8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d104d0 0x5647d70d8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2279f5d640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 0x7f225c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 -- 172.21.15.94:0/3517372779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 msgr2=0x5647d70d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f227975c640 1 -- 172.21.15.94:0/3517372779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d6ffe0d0 con 0x5647d6d082f0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f2278f5b640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.248+0000 7f22727fc640 1 -- 172.21.15.94:0/159640776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f2264100290 con 0x7f2274108980 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.249+0000 7f227975c640 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f226c009be0 tx=0x7f226c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b54d27c6e3c43d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.249+0000 7f225b7fe640 1 -- 172.21.15.94:0/3517372779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c004020 con 0x5647d6d082f0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.249+0000 7f2279f5d640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 0x7f225c0b1ff0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22640c83d0 tx=0x7f22640f6e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.249+0000 7f225b7fe640 1 -- 172.21.15.94:0/3517372779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c0054d0 con 0x5647d6d082f0 2024-04-25T13:58:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.249+0000 7f225b7fe640 1 -- 172.21.15.94:0/3517372779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c004b60 con 0x5647d6d082f0 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- 172.21.15.94:0/3517372779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 msgr2=0x5647d70d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f226c009be0 tx=0x7f226c0119a0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- 172.21.15.94:0/3517372779 shutdown_connections 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d104d0 0x5647d70d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d082f0 0x5647d70d1ef0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 --2- 172.21.15.94:0/3517372779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- 172.21.15.94:0/3517372779 >> 172.21.15.94:0/3517372779 conn(0x5647d6d25300 msgr2=0x5647d70cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- 172.21.15.94:0/3517372779 shutdown_connections 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- 172.21.15.94:0/3517372779 wait complete. 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 -- start start 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.250+0000 7f2282870740 1 Processor -- start 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5647d70d97e0 con 0x5647d6d104d0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5647d6fbeb60 con 0x5647d6d311b0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d70d9960 con 0x5647d6d082f0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 10 client.0 _ll_register_callbacks cb 0x5647d6fe32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35394/0 (socket says 172.21.15.94:35394) 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 -- 172.21.15.94:0/907746653 learned_addr learned my addr 172.21.15.94:0/907746653 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2278f5b640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2278f5b640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 -- 172.21.15.94:0/907746653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 msgr2=0x5647d70db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 -- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 msgr2=0x5647d70e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 -- 172.21.15.94:0/907746653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d6d49210 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2278f5b640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f227975c640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f226c01c040 tx=0x7f226c01aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27bb027c2e8dc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2258ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c027020 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f226c023400 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.251+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c0054d0 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f226c032050 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 10 client.21480 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5647d6e15620 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c0054d0 con 0x5647d6d104d0 2024-04-25T13:58:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f226c023c70 con 0x5647d6d104d0 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 10 client.21480 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 10 client.21480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 20 client.21480 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cefd80 0x5647d70e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 4 client.21480 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d7045070 con 0x5647d6cefd80 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.252+0000 7f2278f5b640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cefd80 0x5647d70e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.253+0000 7f2258ff9640 10 client.21480 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.253+0000 7f2278f5b640 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cefd80 0x5647d70e9010 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5097a8f4e74e127 server_cookie=d89592d3a5625bef in_seq=0 out_seq=0 2024-04-25T13:58:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.254+0000 7f2258ff9640 1 -- 172.21.15.94:0/907746653 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647d7045070 con 0x5647d6cefd80 2024-04-25T13:58:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2258ff9640 10 client.21480 handle_command_reply: tid=0 2024-04-25T13:58:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 client.21480 shutdown 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 msgr2=0x5647d70dbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f226c01c040 tx=0x7f226c01aa70 comp rx=0 tx=0).stop 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 shutdown_connections 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cefd80 0x5647d70e9010 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d311b0 0x5647d70e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d104d0 0x5647d70dbaf0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 --2- 172.21.15.94:0/907746653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 >> 172.21.15.94:0/907746653 conn(0x5647d6d25300 msgr2=0x5647d70d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2258ff9640 0 client.21480 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 shutdown_connections 2024-04-25T13:58:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 1 -- 172.21.15.94:0/907746653 wait complete. 2024-04-25T13:58:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.255+0000 7f2282870740 20 client.21480 trim_cache size 0 max 16384 2024-04-25T13:58:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 Processor -- start 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 -- start start 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d70da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22741a9290 con 0x5647d6d311b0 2024-04-25T13:58:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aadd0 con 0x5647d6d104d0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2282870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22741a9960 con 0x5647d6d082f0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35400/0 (socket says 172.21.15.94:35400) 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 -- 172.21.15.94:0/2982122153 learned_addr learned my addr 172.21.15.94:0/2982122153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f227975c640 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d70da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 -- 172.21.15.94:0/2982122153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 msgr2=0x5647d70d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 -- 172.21.15.94:0/2982122153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 msgr2=0x5647d70da270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d70da270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.271+0000 7f2278f5b640 1 -- 172.21.15.94:0/2982122153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d6ceffb0 con 0x5647d6d311b0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.272+0000 7f2278f5b640 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43d4af237152bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.272+0000 7f225affd640 1 -- 172.21.15.94:0/2982122153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226802a030 con 0x5647d6d311b0 2024-04-25T13:58:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.272+0000 7f225affd640 1 -- 172.21.15.94:0/2982122153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22680130d0 con 0x5647d6d311b0 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.272+0000 7f225affd640 1 -- 172.21.15.94:0/2982122153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268011930 con 0x5647d6d311b0 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- 172.21.15.94:0/2982122153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 msgr2=0x5647d70d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f22680058c0 tx=0x7f2268009ae0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- 172.21.15.94:0/2982122153 shutdown_connections 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d70d0100 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d70da270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- 172.21.15.94:0/2982122153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d70d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- 172.21.15.94:0/2982122153 >> 172.21.15.94:0/2982122153 conn(0x5647d6d25300 msgr2=0x5647d70d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- 172.21.15.94:0/2982122153 shutdown_connections 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- 172.21.15.94:0/2982122153 wait complete. 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- start start 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 Processor -- start 2024-04-25T13:58:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d714bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d714c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aa700 con 0x5647d6d311b0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aa030 con 0x5647d6d104d0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22741a7270 con 0x5647d6d082f0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.273+0000 7f2282870740 10 client.0 _ll_register_callbacks cb 0x7f2268009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35416/0 (socket says 172.21.15.94:35416) 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f227975c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d714c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 -- 172.21.15.94:0/1474157228 learned_addr learned my addr 172.21.15.94:0/1474157228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d714bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 -- 172.21.15.94:0/1474157228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 msgr2=0x5647d714bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d714bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 -- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 msgr2=0x5647d714c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d714c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 -- 172.21.15.94:0/1474157228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d71535a0 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2278f5b640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f22680044a0 tx=0x7f2268009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae8f85e9e70d621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2258ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226802a040 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f2268003ac0 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22680130d0 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2268003cc0 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2282870740 10 client.21486 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5647d7154830 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.274+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22680130d0 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2268004a00 con 0x5647d6d311b0 2024-04-25T13:58:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 10 client.21486 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 10 client.21486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 20 client.21486 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cc51c0 0x5647d7157110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 4 client.21486 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647d703eee0 con 0x5647d6cc51c0 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f227975c640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cc51c0 0x5647d7157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f2258ff9640 10 client.21486 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.275+0000 7f227975c640 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cc51c0 0x5647d7157110 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae47c39199d4bc84 server_cookie=dada46f3756a8ddd in_seq=0 out_seq=0 2024-04-25T13:58:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2258ff9640 1 -- 172.21.15.94:0/1474157228 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647d703eee0 con 0x5647d6cc51c0 2024-04-25T13:58:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2258ff9640 10 client.21486 handle_command_reply: tid=0 2024-04-25T13:58:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 client.21486 shutdown 2024-04-25T13:58:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 msgr2=0x5647d7152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f22680044a0 tx=0x7f2268009bd0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 shutdown_connections 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5647d6cc51c0 0x5647d7157110 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5647d6d311b0 0x5647d7152fe0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2258ff9640 0 client.21486 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647d6d104d0 0x5647d714c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 --2- 172.21.15.94:0/1474157228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647d6d082f0 0x5647d714bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.884+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 >> 172.21.15.94:0/1474157228 conn(0x5647d6d25300 msgr2=0x5647d70d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 shutdown_connections 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f2282870740 1 -- 172.21.15.94:0/1474157228 wait complete. 2024-04-25T13:58:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f2282870740 20 client.21486 trim_cache size 0 max 16384 2024-04-25T13:58:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:58:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:58:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:58:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 msgr2=0x7f225c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 0x7f225c0b1ff0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22640c83d0 tx=0x7f22640f6e20 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 msgr2=0x7f22741a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f2264004af0 tx=0x7f2264031700 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 shutdown_connections 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f225c0afae0 0x7f225c0b1ff0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2274108980 0x7f22741a0580 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2274103c80 0x7f22741995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 --2- 172.21.15.94:0/159640776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2274102a40 0x7f22741990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 >> 172.21.15.94:0/159640776 conn(0x7f22740fdc30 msgr2=0x7f2274104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 shutdown_connections 2024-04-25T13:58:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:31.885+0000 7f227b9c0640 1 -- 172.21.15.94:0/159640776 wait complete. 2024-04-25T13:58:32.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 70902 19.14 MB/sec execute 74 sec latency 381.802 ms 2024-04-25T13:58:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:32 smithi137 ceph-mon[32568]: pgmap v852: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-04-25T13:58:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:32 smithi094 ceph-mon[29195]: pgmap v852: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-04-25T13:58:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:32 smithi138 ceph-mon[32977]: pgmap v852: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 605 op/s 2024-04-25T13:58:33.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 71305 19.21 MB/sec execute 75 sec latency 76.870 ms 2024-04-25T13:58:34.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 71651 19.25 MB/sec execute 76 sec latency 109.113 ms 2024-04-25T13:58:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:34 smithi094 ceph-mon[29195]: pgmap v853: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 322 op/s 2024-04-25T13:58:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:34 smithi138 ceph-mon[32977]: pgmap v853: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 322 op/s 2024-04-25T13:58:34.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:34 smithi137 ceph-mon[32568]: pgmap v853: 97 pgs: 97 active+clean; 260 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 322 op/s 2024-04-25T13:58:35.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 71975 19.14 MB/sec execute 77 sec latency 157.743 ms 2024-04-25T13:58:36.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 72394 19.23 MB/sec execute 78 sec latency 69.078 ms 2024-04-25T13:58:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:36 smithi094 ceph-mon[29195]: pgmap v854: 97 pgs: 97 active+clean; 253 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-04-25T13:58:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:36 smithi138 ceph-mon[32977]: pgmap v854: 97 pgs: 97 active+clean; 253 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-04-25T13:58:36.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:36 smithi137 ceph-mon[32568]: pgmap v854: 97 pgs: 97 active+clean; 253 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-04-25T13:58:37.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 72925 19.28 MB/sec execute 79 sec latency 63.041 ms 2024-04-25T13:58:38.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 73298 19.37 MB/sec execute 80 sec latency 76.574 ms 2024-04-25T13:58:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:38 smithi094 ceph-mon[29195]: pgmap v855: 97 pgs: 97 active+clean; 247 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-04-25T13:58:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:38 smithi138 ceph-mon[32977]: pgmap v855: 97 pgs: 97 active+clean; 247 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-04-25T13:58:38.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:38 smithi137 ceph-mon[32568]: pgmap v855: 97 pgs: 97 active+clean; 247 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-04-25T13:58:39.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 73709 19.39 MB/sec execute 81 sec latency 74.972 ms 2024-04-25T13:58:40.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 73961 19.35 MB/sec execute 82 sec latency 229.476 ms 2024-04-25T13:58:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:40 smithi094 ceph-mon[29195]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-04-25T13:58:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:40 smithi138 ceph-mon[32977]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-04-25T13:58:40.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:40 smithi137 ceph-mon[32568]: pgmap v856: 97 pgs: 97 active+clean; 246 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 500 op/s 2024-04-25T13:58:41.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 74269 19.26 MB/sec execute 83 sec latency 197.174 ms 2024-04-25T13:58:42.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 74651 19.30 MB/sec execute 84 sec latency 86.480 ms 2024-04-25T13:58:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:42 smithi094 ceph-mon[29195]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-04-25T13:58:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:42 smithi138 ceph-mon[32977]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-04-25T13:58:42.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:42 smithi137 ceph-mon[32568]: pgmap v857: 97 pgs: 97 active+clean; 255 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2024-04-25T13:58:43.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 75045 19.32 MB/sec execute 85 sec latency 83.426 ms 2024-04-25T13:58:43.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:43 smithi094 ceph-mon[29195]: pgmap v858: 97 pgs: 97 active+clean; 256 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 478 op/s 2024-04-25T13:58:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:43 smithi138 ceph-mon[32977]: pgmap v858: 97 pgs: 97 active+clean; 256 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 478 op/s 2024-04-25T13:58:43.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:43 smithi137 ceph-mon[32568]: pgmap v858: 97 pgs: 97 active+clean; 256 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 478 op/s 2024-04-25T13:58:44.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:58:44.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:58:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 -- 172.21.15.94:0/670311548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5974103c20 msgr2=0x7f5974104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 --2- 172.21.15.94:0/670311548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5974103c20 0x7f5974104030 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f5968009900 tx=0x7f596802a410 comp rx=0 tx=0).stop 2024-04-25T13:58:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 -- 172.21.15.94:0/670311548 shutdown_connections 2024-04-25T13:58:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 --2- 172.21.15.94:0/670311548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974108920 0x7f597410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 --2- 172.21.15.94:0/670311548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5974103c20 0x7f5974104030 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 --2- 172.21.15.94:0/670311548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59741029e0 0x7f59741036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 -- 172.21.15.94:0/670311548 >> 172.21.15.94:0/670311548 conn(0x7f59740fdbd0 msgr2=0x7f5974100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 -- 172.21.15.94:0/670311548 shutdown_connections 2024-04-25T13:58:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.129+0000 7f5979451640 1 -- 172.21.15.94:0/670311548 wait complete. 2024-04-25T13:58:44.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 Processor -- start 2024-04-25T13:58:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 -- start start 2024-04-25T13:58:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974103c20 0x7f597419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5974108920 0x7f59741a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f597410e4e0 con 0x7f59741029e0 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f597410e360 con 0x7f5974103c20 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5979451640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f597410e660 con 0x7f5974108920 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5972ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5972ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59974/0 (socket says 172.21.15.94:59974) 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f5972ffd640 1 -- 172.21.15.94:0/2952193647 learned_addr learned my addr 172.21.15.94:0/2952193647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f59727fc640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974103c20 0x7f597419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.130+0000 7f59737fe640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5974108920 0x7f59741a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 -- 172.21.15.94:0/2952193647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5974108920 msgr2=0x7f59741a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5974108920 0x7f59741a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 -- 172.21.15.94:0/2952193647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974103c20 msgr2=0x7f597419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974103c20 0x7f597419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 -- 172.21.15.94:0/2952193647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59741a2070 con 0x7f59741029e0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5972ffd640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9f0 tx=0x7f596400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f595bfff640 1 -- 172.21.15.94:0/2952193647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400ccc0 con 0x7f59741029e0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f595bfff640 1 -- 172.21.15.94:0/2952193647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59640035f0 con 0x7f59741029e0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59741a2360 con 0x7f59741029e0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f595bfff640 1 -- 172.21.15.94:0/2952193647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400ce60 con 0x7f59741029e0 2024-04-25T13:58:44.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.131+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741a28b0 con 0x7f59741029e0 2024-04-25T13:58:44.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 Processor -- start 2024-04-25T13:58:44.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 -- start start 2024-04-25T13:58:44.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f595bfff640 1 -- 172.21.15.94:0/2952193647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5964010690 con 0x7f59741029e0 2024-04-25T13:58:44.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127be3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127be4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563127b48d80 con 0x5631278194d0 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563127b028b0 con 0x5631278216b0 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59803d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563127b44d40 con 0x563127842390 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59737fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59737fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51276/0 (socket says 172.21.15.94:51276) 2024-04-25T13:58:44.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59737fe640 1 -- 172.21.15.94:0/3026926604 learned_addr learned my addr 172.21.15.94:0/3026926604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f5972ffd640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127be3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.134+0000 7f59727fc640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127be4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 -- 172.21.15.94:0/3026926604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 msgr2=0x563127be4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127be4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 -- 172.21.15.94:0/3026926604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 msgr2=0x563127be3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127be3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 -- 172.21.15.94:0/3026926604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563127ad3a40 con 0x563127842390 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f5972ffd640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127be3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59737fe640 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f596c009fd0 tx=0x7f596c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a98a7bd2e23e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59597fa640 1 -- 172.21.15.94:0/3026926604 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c004d20 con 0x563127842390 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f595bfff640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 0x7f595c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59597fa640 1 -- 172.21.15.94:0/3026926604 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c004050 con 0x563127842390 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.135+0000 7f59597fa640 1 -- 172.21.15.94:0/3026926604 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c005280 con 0x563127842390 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.136+0000 7f595bfff640 1 -- 172.21.15.94:0/2952193647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f59640deff0 con 0x7f59741029e0 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.136+0000 7f5972ffd640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 0x7f595c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 -- 172.21.15.94:0/3026926604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 msgr2=0x563127be3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f5972ffd640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 0x7f595c0b2000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f59640dc9f0 tx=0x7f5964014000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f596c009fd0 tx=0x7f596c0118d0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 -- 172.21.15.94:0/3026926604 shutdown_connections 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127be4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127be3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 --2- 172.21.15.94:0/3026926604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127be3a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 -- 172.21.15.94:0/3026926604 >> 172.21.15.94:0/3026926604 conn(0x5631278364e0 msgr2=0x563127bdf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 -- 172.21.15.94:0/3026926604 shutdown_connections 2024-04-25T13:58:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.137+0000 7f59803d6740 1 -- 172.21.15.94:0/3026926604 wait complete. 2024-04-25T13:58:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 -- start start 2024-04-25T13:58:44.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 Processor -- start 2024-04-25T13:58:44.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127bf4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563127be5780 con 0x5631278194d0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563127acfd10 con 0x5631278216b0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563127be5900 con 0x563127842390 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59803d6740 10 client.0 _ll_register_callbacks cb 0x563127afc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59988/0 (socket says 172.21.15.94:59988) 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.138+0000 7f59737fe640 1 -- 172.21.15.94:0/1164060527 learned_addr learned my addr 172.21.15.94:0/1164060527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59737fe640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127bf4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 -- 172.21.15.94:0/1164060527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 msgr2=0x563127bf4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127bf4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 -- 172.21.15.94:0/1164060527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 msgr2=0x563127bed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 -- 172.21.15.94:0/1164060527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631279247d0 con 0x5631278216b0 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59737fe640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59727fc640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5968005ab0 tx=0x7f5968031d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e51e457ab877761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f5956ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968035070 con 0x5631278216b0 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f5968004040 con 0x5631278216b0 2024-04-25T13:58:44.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c004050 con 0x5631278216b0 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.139+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5968004240 con 0x5631278216b0 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 10 client.30920 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56312785a3f0 con 0x5631278216b0 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c004050 con 0x5631278216b0 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f596803b430 con 0x5631278216b0 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 10 client.30920 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 10 client.30920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 20 client.30920 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563127800f60 0x563127bfb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 4 client.30920 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:44.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.140+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563127b56990 con 0x563127800f60 2024-04-25T13:58:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.141+0000 7f59737fe640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563127800f60 0x563127bfb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.141+0000 7f59737fe640 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563127800f60 0x563127bfb640 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4e4ba21f581e4a server_cookie=37bd24b5e6767b15 in_seq=0 out_seq=0 2024-04-25T13:58:44.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.141+0000 7f5956ff5640 10 client.30920 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:44.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 75545 19.36 MB/sec execute 86 sec latency 78.033 ms 2024-04-25T13:58:44.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f5956ff5640 1 -- 172.21.15.94:0/1164060527 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563127b56990 con 0x563127800f60 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f5956ff5640 10 client.30920 handle_command_reply: tid=0 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 client.30920 shutdown 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 msgr2=0x563127bede40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5968005ab0 tx=0x7f5968031d40 comp rx=0 tx=0).stop 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 shutdown_connections 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563127800f60 0x563127bfb640 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563127842390 0x563127bf4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 --2- 172.21.15.94:0/1164060527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f5956ff5640 0 client.30920 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 >> 172.21.15.94:0/1164060527 conn(0x5631278364e0 msgr2=0x563127bde3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 shutdown_connections 2024-04-25T13:58:44.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 1 -- 172.21.15.94:0/1164060527 wait complete. 2024-04-25T13:58:44.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.144+0000 7f59803d6740 20 client.30920 trim_cache size 0 max 16384 2024-04-25T13:58:44.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 Processor -- start 2024-04-25T13:58:44.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 -- start start 2024-04-25T13:58:44.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a9350 con 0x563127842390 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59741aae90 con 0x5631278216b0 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59803d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a9a20 con 0x5631278194d0 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59268/0 (socket says 172.21.15.94:59268) 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60002/0 (socket says 172.21.15.94:60002) 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59727fc640 1 -- 172.21.15.94:0/4943412 learned_addr learned my addr 172.21.15.94:0/4943412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.164+0000 7f59737fe640 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 -- 172.21.15.94:0/4943412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 msgr2=0x563127bed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 -- 172.21.15.94:0/4943412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 msgr2=0x563127bede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 -- 172.21.15.94:0/4943412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312781a0f0 con 0x563127842390 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59727fc640 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f59737fe640 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f596c005c40 tx=0x7f596c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeeebeed3cd3a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f5958ff9640 1 -- 172.21.15.94:0/4943412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c004630 con 0x563127842390 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f5958ff9640 1 -- 172.21.15.94:0/4943412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c004050 con 0x563127842390 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.165+0000 7f5958ff9640 1 -- 172.21.15.94:0/4943412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c005770 con 0x563127842390 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- 172.21.15.94:0/4943412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 msgr2=0x563127bebf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f596c005c40 tx=0x7f596c003f50 comp rx=0 tx=0).stop 2024-04-25T13:58:44.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- 172.21.15.94:0/4943412 shutdown_connections 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563127842390 0x563127bebf70 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631278216b0 0x563127bede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 --2- 172.21.15.94:0/4943412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- 172.21.15.94:0/4943412 >> 172.21.15.94:0/4943412 conn(0x5631278364e0 msgr2=0x563127be8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- 172.21.15.94:0/4943412 shutdown_connections 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- 172.21.15.94:0/4943412 wait complete. 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 -- start start 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.166+0000 7f59803d6740 1 Processor -- start 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bf3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563127842390 0x563127bf07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59741aa7c0 con 0x5631278216b0 2024-04-25T13:58:44.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f59741aa0f0 con 0x563127842390 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a7330 con 0x5631278194d0 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 10 client.0 _ll_register_callbacks cb 0x7f5964005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60004/0 (socket says 172.21.15.94:60004) 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 -- 172.21.15.94:0/3519674552 learned_addr learned my addr 172.21.15.94:0/3519674552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59737fe640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563127842390 0x563127bf07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59737fe640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bf3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 -- 172.21.15.94:0/3519674552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 msgr2=0x563127bf3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bf3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 -- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563127842390 msgr2=0x563127bf07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563127842390 0x563127bf07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 -- 172.21.15.94:0/3519674552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563127b094b0 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59727fc640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f5968005ab0 tx=0x7f5968009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c1dfc80b90bcfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f5956ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968035070 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f596803b430 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c004050 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.167+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f596805e050 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 10 client.21504 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563127c62430 con 0x5631278216b0 2024-04-25T13:58:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c004050 con 0x5631278216b0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5968003aa0 con 0x5631278216b0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 10 client.21504 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 10 client.21504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 20 client.21504 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5631277d63a0 0x563127c64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 4 client.21504 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563127b50800 con 0x5631277d63a0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59737fe640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5631277d63a0 0x563127c64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f5956ff5640 10 client.21504 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.168+0000 7f59737fe640 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5631277d63a0 0x563127c64e50 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98730112eb05997 server_cookie=40497c002fb6346 in_seq=0 out_seq=0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f5956ff5640 1 -- 172.21.15.94:0/3519674552 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563127b50800 con 0x5631277d63a0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f5956ff5640 10 client.21504 handle_command_reply: tid=0 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 client.21504 shutdown 2024-04-25T13:58:44.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 msgr2=0x563127bf4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f5968005ab0 tx=0x7f5968009780 comp rx=0 tx=0).stop 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 shutdown_connections 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5631277d63a0 0x563127c64e50 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563127842390 0x563127bf07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5631278216b0 0x563127bf4190 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 --2- 172.21.15.94:0/3519674552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631278194d0 0x563127bf3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f5956ff5640 0 client.21504 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.170+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 >> 172.21.15.94:0/3519674552 conn(0x5631278364e0 msgr2=0x563127bed8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 shutdown_connections 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f59803d6740 1 -- 172.21.15.94:0/3519674552 wait complete. 2024-04-25T13:58:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f59803d6740 20 client.21504 trim_cache size 0 max 16384 2024-04-25T13:58:44.181 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:44.181 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:58:44.181 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:58:44.181 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 msgr2=0x7f595c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 0x7f595c0b2000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f59640dc9f0 tx=0x7f5964014000 comp rx=0 tx=0).stop 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 msgr2=0x7f597406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9f0 tx=0x7f596400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 shutdown_connections 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f595c0afaf0 0x7f595c0b2000 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5974108920 0x7f59741a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974103c20 0x7f597419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 --2- 172.21.15.94:0/2952193647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59741029e0 0x7f597406a890 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.171+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 >> 172.21.15.94:0/2952193647 conn(0x7f59740fdbd0 msgr2=0x7f5974104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.172+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 shutdown_connections 2024-04-25T13:58:44.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.172+0000 7f5979451640 1 -- 172.21.15.94:0/2952193647 wait complete. 2024-04-25T13:58:44.184 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:58:44.185 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:58:44.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 -- 172.21.15.94:0/1789044341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 msgr2=0x7f23f8106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/1789044341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f8106610 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009900 tx=0x7f23f402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:44.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 -- 172.21.15.94:0/1789044341 shutdown_connections 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/1789044341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8109750 0x7f23f810bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/1789044341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8106be0 0x7f23f81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/1789044341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f8106610 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 -- 172.21.15.94:0/1789044341 >> 172.21.15.94:0/1789044341 conn(0x7f23f80fdbd0 msgr2=0x7f23f8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 -- 172.21.15.94:0/1789044341 shutdown_connections 2024-04-25T13:58:44.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.306+0000 7f23ffc0c640 1 -- 172.21.15.94:0/1789044341 wait complete. 2024-04-25T13:58:44.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 Processor -- start 2024-04-25T13:58:44.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 -- start start 2024-04-25T13:58:44.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 0x7f23f81a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f590 con 0x7f23f81041f0 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f410 con 0x7f23f8109750 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.307+0000 7f23ffc0c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f710 con 0x7f23f8106be0 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd9a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51328/0 (socket says 172.21.15.94:51328) 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd9a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60030/0 (socket says 172.21.15.94:60030) 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 -- 172.21.15.94:0/3077503350 learned_addr learned my addr 172.21.15.94:0/3077503350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fe1a9640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 0x7f23f81a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 -- 172.21.15.94:0/3077503350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 msgr2=0x7f23f81a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 0x7f23f81a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 -- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 msgr2=0x7f23f819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 -- 172.21.15.94:0/3077503350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f81a1d60 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fe1a9640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 0x7f23f81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd9a8640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23fd1a7640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23e800ea60 tx=0x7f23e800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.308+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e800cd20 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.309+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f81a1ff0 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.309+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e80035f0 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.309+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8010690 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.309+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f81a2530 con 0x7f23f8106be0 2024-04-25T13:58:44.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.310+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f80ff6f0 con 0x7f23f8106be0 2024-04-25T13:58:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.311+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f23e80037a0 con 0x7f23f8106be0 2024-04-25T13:58:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.312+0000 7f23eeffd640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 0x7f23e40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.312+0000 7f23fd9a8640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 0x7f23e40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.312+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f23e80df070 con 0x7f23f8106be0 2024-04-25T13:58:44.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.312+0000 7f23fd9a8640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 0x7f23e40b2080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009230 tx=0x7f23f4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.315+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23e8053ff0 con 0x7f23f8106be0 2024-04-25T13:58:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.617+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f802db50 con 0x7f23f8106be0 2024-04-25T13:58:44.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.618+0000 7f23eeffd640 1 -- 172.21.15.94:0/3077503350 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7f23e80a7c50 con 0x7f23f8106be0 2024-04-25T13:58:44.618 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:58:44.618 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:57:45.761472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30058":{"gid":30058,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2422946890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2422946890},{"type":"v1","addr":"172.21.15.138:6835","nonce":2422946890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24512,24557,24475,24488,24502]},"id":1}]} 2024-04-25T13:58:44.618 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 msgr2=0x7f23e40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 0x7f23e40b2080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009230 tx=0x7f23f4031400 comp rx=0 tx=0).stop 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 msgr2=0x7f23f819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23e800ea60 tx=0x7f23e800ee60 comp rx=0 tx=0).stop 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 shutdown_connections 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e40afb70 0x7f23e40b2080 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8109750 0x7f23f81a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8106be0 0x7f23f819a760 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 --2- 172.21.15.94:0/3077503350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f81041f0 0x7f23f819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 >> 172.21.15.94:0/3077503350 conn(0x7f23f80fdbd0 msgr2=0x7f23f8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 shutdown_connections 2024-04-25T13:58:44.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.621+0000 7f23ffc0c640 1 -- 172.21.15.94:0/3077503350 wait complete. 2024-04-25T13:58:44.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:58:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 -- 172.21.15.94:0/689267598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1670103c20 msgr2=0x7f1670104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 --2- 172.21.15.94:0/689267598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1670103c20 0x7f1670104030 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f166402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:44.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 -- 172.21.15.94:0/689267598 shutdown_connections 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 --2- 172.21.15.94:0/689267598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670108920 0x7f167010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 --2- 172.21.15.94:0/689267598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1670103c20 0x7f1670104030 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 --2- 172.21.15.94:0/689267598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16701029e0 0x7f16701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 -- 172.21.15.94:0/689267598 >> 172.21.15.94:0/689267598 conn(0x7f16700fdbd0 msgr2=0x7f1670100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 -- 172.21.15.94:0/689267598 shutdown_connections 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.757+0000 7f1676813640 1 -- 172.21.15.94:0/689267598 wait complete. 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 Processor -- start 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 -- start start 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670103c20 0x7f167019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670108920 0x7f16701a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f167010e510 con 0x7f16701029e0 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f167010e390 con 0x7f1670108920 2024-04-25T13:58:44.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1676813640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f167010e690 con 0x7f1670103c20 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f166ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f166ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60048/0 (socket says 172.21.15.94:60048) 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f166ffff640 1 -- 172.21.15.94:0/2000928011 learned_addr learned my addr 172.21.15.94:0/2000928011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f1674db0640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670108920 0x7f16701a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.758+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670103c20 0x7f167019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 -- 172.21.15.94:0/2000928011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670103c20 msgr2=0x7f167019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670103c20 0x7f167019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 -- 172.21.15.94:0/2000928011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670108920 msgr2=0x7f16701a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670108920 0x7f16701a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 -- 172.21.15.94:0/2000928011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16701a2060 con 0x7f16701029e0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166ffff640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f166000e980 tx=0x7f166000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166d7fa640 1 -- 172.21.15.94:0/2000928011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000cd80 con 0x7f16701029e0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166d7fa640 1 -- 172.21.15.94:0/2000928011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16600108e0 con 0x7f16701029e0 2024-04-25T13:58:44.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16701a2350 con 0x7f16701029e0 2024-04-25T13:58:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f166d7fa640 1 -- 172.21.15.94:0/2000928011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000cd80 con 0x7f16701029e0 2024-04-25T13:58:44.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.759+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16701a2850 con 0x7f16701029e0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 Processor -- start 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 -- start start 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843d90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561843cf4d80 con 0x5618439c54d0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561843caea20 con 0x5618439cd6b0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f167d79f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561843cf0d40 con 0x5618439ee390 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f166d7fa640 1 -- 172.21.15.94:0/2000928011 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f166001f020 con 0x7f16701029e0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f166ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843d90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f166ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60050/0 (socket says 172.21.15.94:60050) 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f166ffff640 1 -- 172.21.15.94:0/2956101823 learned_addr learned my addr 172.21.15.94:0/2956101823 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 -- 172.21.15.94:0/2956101823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 msgr2=0x561843d91a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d91a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 -- 172.21.15.94:0/2956101823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 msgr2=0x561843d90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.762+0000 7f1674db0640 1 -- 172.21.15.94:0/2956101823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561843cb9c30 con 0x5618439ee390 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f1674db0640 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843d90810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1668009f00 tx=0x7f1668011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0e57ad3f04f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f166d7fa640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 0x7f16580b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f16567fc640 1 -- 172.21.15.94:0/2956101823 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668004d20 con 0x5618439ee390 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f16567fc640 1 -- 172.21.15.94:0/2956101823 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1668004030 con 0x5618439ee390 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f16567fc640 1 -- 172.21.15.94:0/2956101823 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668005280 con 0x5618439ee390 2024-04-25T13:58:44.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f166ffff640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 0x7f16580b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.763+0000 7f166d7fa640 1 -- 172.21.15.94:0/2000928011 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f16600de910 con 0x7f16701029e0 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- 172.21.15.94:0/2956101823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 msgr2=0x561843d90810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f166ffff640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 0x7f16580b2200 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16600a6ce0 tx=0x7f16600e4550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843d90810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1668009f00 tx=0x7f1668011810 comp rx=0 tx=0).stop 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- 172.21.15.94:0/2956101823 shutdown_connections 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 --2- 172.21.15.94:0/2956101823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843d90810 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- 172.21.15.94:0/2956101823 >> 172.21.15.94:0/2956101823 conn(0x5618439e24e0 msgr2=0x561843d8c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- 172.21.15.94:0/2956101823 shutdown_connections 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- 172.21.15.94:0/2956101823 wait complete. 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 -- start start 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.764+0000 7f167d79f740 1 Processor -- start 2024-04-25T13:58:44.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843da1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561843d92530 con 0x5618439c54d0 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561843c7bdc0 con 0x5618439cd6b0 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561843d926b0 con 0x5618439ee390 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f167d79f740 10 client.0 _ll_register_callbacks cb 0x561843c97600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60060/0 (socket says 172.21.15.94:60060) 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 -- 172.21.15.94:0/2405187775 learned_addr learned my addr 172.21.15.94:0/2405187775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843da1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 -- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 msgr2=0x561843da1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843da1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 -- 172.21.15.94:0/2405187775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 msgr2=0x561843d9aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d9aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 -- 172.21.15.94:0/2405187775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561843a063f0 con 0x5618439c54d0 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f1674db0640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f1668009270 tx=0x7f16680043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e3ae946a26ce95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f164f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.765+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166802d060 con 0x5618439c54d0 2024-04-25T13:58:44.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f1668005040 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1668004030 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1668005240 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 10 client.21522 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561843ad07d0 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1668004030 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1668024070 con 0x5618439c54d0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 10 client.21522 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 10 client.21522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 20 client.21522 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439acf60 0x561843da7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 4 client.21522 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561843d02990 con 0x5618439acf60 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.766+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439acf60 0x561843da7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.767+0000 7f164f7fe640 10 client.21522 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.767+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439acf60 0x561843da7f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16198282f1291b30 server_cookie=296abe9e9866e319 in_seq=0 out_seq=0 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.767+0000 7f164f7fe640 1 -- 172.21.15.94:0/2405187775 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561843d02990 con 0x5618439acf60 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.767+0000 7f164f7fe640 10 client.21522 handle_command_reply: tid=0 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 client.21522 shutdown 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 msgr2=0x561843d9a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f1668009270 tx=0x7f16680043e0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 shutdown_connections 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439acf60 0x561843da7f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439ee390 0x561843da1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439cd6b0 0x561843d9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f164f7fe640 0 client.21522 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 --2- 172.21.15.94:0/2405187775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439c54d0 0x561843d9a4c0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 >> 172.21.15.94:0/2405187775 conn(0x5618439e24e0 msgr2=0x561843d8b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 shutdown_connections 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 1 -- 172.21.15.94:0/2405187775 wait complete. 2024-04-25T13:58:44.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.768+0000 7f167d79f740 20 client.21522 trim_cache size 0 max 16384 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.787+0000 7f167d79f740 1 Processor -- start 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 -- start start 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843d9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ab560 con 0x5618439cd6b0 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ad0a0 con 0x5618439ee390 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f167d79f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16701abc30 con 0x5618439c54d0 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60064/0 (socket says 172.21.15.94:60064) 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 -- 172.21.15.94:0/2865495345 learned_addr learned my addr 172.21.15.94:0/2865495345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f1674db0640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843d9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f1674db0640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 -- 172.21.15.94:0/2865495345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 msgr2=0x561843d87190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d87190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 -- 172.21.15.94:0/2865495345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 msgr2=0x561843d9a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843d9a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 -- 172.21.15.94:0/2865495345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561843d91840 con 0x5618439cd6b0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f1674db0640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843d9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.788+0000 7f166f7fe640 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f166402a930 tx=0x7f1664031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fdb53991169a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.789+0000 7f1655ffb640 1 -- 172.21.15.94:0/2865495345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664035040 con 0x5618439cd6b0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.789+0000 7f1655ffb640 1 -- 172.21.15.94:0/2865495345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f166404b070 con 0x5618439cd6b0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.789+0000 7f1655ffb640 1 -- 172.21.15.94:0/2865495345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640035f0 con 0x5618439cd6b0 2024-04-25T13:58:44.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.789+0000 7f167d79f740 1 -- 172.21.15.94:0/2865495345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 msgr2=0x561843d9a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.789+0000 7f167d79f740 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f166402a930 tx=0x7f1664031870 comp rx=0 tx=0).stop 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- 172.21.15.94:0/2865495345 shutdown_connections 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843d9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9a090 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- 172.21.15.94:0/2865495345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d87190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- 172.21.15.94:0/2865495345 >> 172.21.15.94:0/2865495345 conn(0x5618439e24e0 msgr2=0x561843d951b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- 172.21.15.94:0/2865495345 shutdown_connections 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- 172.21.15.94:0/2865495345 wait complete. 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- start start 2024-04-25T13:58:44.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 Processor -- start 2024-04-25T13:58:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843e0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac9d0 con 0x5618439cd6b0 2024-04-25T13:58:44.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac300 con 0x5618439ee390 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a9540 con 0x5618439c54d0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.790+0000 7f167d79f740 10 client.0 _ll_register_callbacks cb 0x7f16600e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51386/0 (socket says 172.21.15.94:51386) 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 -- 172.21.15.94:0/3598822430 learned_addr learned my addr 172.21.15.94:0/3598822430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843e0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 -- 172.21.15.94:0/3598822430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 msgr2=0x561843e0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843e0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 -- 172.21.15.94:0/3598822430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 msgr2=0x561843d9e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 -- 172.21.15.94:0/3598822430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561843e0e7c0 con 0x5618439c54d0 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f166f7fe640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f1674db0640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1668011d10 tx=0x7f16680043e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976465a123096c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f164f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166802d070 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f166800fb50 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166404b070 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.791+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f166800fd50 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 10 client.30640 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561843e0f9d0 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f166404b070 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f166803b040 con 0x5618439c54d0 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 10 client.30640 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 10 client.30640 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 20 client.30640 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439823a0 0x561843e12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 4 client.30640 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561843cfc800 con 0x5618439823a0 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.792+0000 7f166f7fe640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439823a0 0x561843e12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f166f7fe640 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439823a0 0x561843e12320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb3bc20eb397992d server_cookie=6debcb4f6f37405 in_seq=0 out_seq=0 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f164f7fe640 10 client.30640 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f164f7fe640 1 -- 172.21.15.94:0/3598822430 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561843cfc800 con 0x5618439823a0 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f164f7fe640 10 client.30640 handle_command_reply: tid=0 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 client.30640 shutdown 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 msgr2=0x561843d9dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1668011d10 tx=0x7f16680043e0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 shutdown_connections 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5618439823a0 0x561843e12320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618439ee390 0x561843e0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5618439cd6b0 0x561843d9e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 --2- 172.21.15.94:0/3598822430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618439c54d0 0x561843d9dfe0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f164f7fe640 0 client.30640 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 >> 172.21.15.94:0/3598822430 conn(0x5618439e24e0 msgr2=0x561843d9a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 shutdown_connections 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 1 -- 172.21.15.94:0/3598822430 wait complete. 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.793+0000 7f167d79f740 20 client.30640 trim_cache size 0 max 16384 2024-04-25T13:58:44.800 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 msgr2=0x7f16580b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 0x7f16580b2200 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16600a6ce0 tx=0x7f16600e4550 comp rx=0 tx=0).stop 2024-04-25T13:58:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 msgr2=0x7f167006a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f166000e980 tx=0x7f166000ee40 comp rx=0 tx=0).stop 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 shutdown_connections 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16580afcf0 0x7f16580b2200 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1670108920 0x7f16701a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1670103c20 0x7f167019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 --2- 172.21.15.94:0/2000928011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16701029e0 0x7f167006a900 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 >> 172.21.15.94:0/2000928011 conn(0x7f16700fdbd0 msgr2=0x7f1670104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 shutdown_connections 2024-04-25T13:58:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.794+0000 7f1676813640 1 -- 172.21.15.94:0/2000928011 wait complete. 2024-04-25T13:58:44.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:58:44.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 -- 172.21.15.94:0/967098749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80104d20 msgr2=0x7f7e80105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/967098749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80104d20 0x7f7e80105130 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009900 tx=0x7f7e7002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:44.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 -- 172.21.15.94:0/967098749 shutdown_connections 2024-04-25T13:58:44.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/967098749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80109a20 0x7f7e8010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/967098749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80104d20 0x7f7e80105130 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/967098749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80103ae0 0x7f7e801047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 -- 172.21.15.94:0/967098749 >> 172.21.15.94:0/967098749 conn(0x7f7e800fecd0 msgr2=0x7f7e80101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 -- 172.21.15.94:0/967098749 shutdown_connections 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 -- 172.21.15.94:0/967098749 wait complete. 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.924+0000 7f7e87f0e640 1 Processor -- start 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 -- start start 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 0x7f7e8019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80109a20 0x7f7e801a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f600 con 0x7f7e80103ae0 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f480 con 0x7f7e80104d20 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e87f0e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f780 con 0x7f7e80109a20 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e85caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 0x7f7e8019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59386/0 (socket says 172.21.15.94:59386) 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e85caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 0x7f7e8019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60098/0 (socket says 172.21.15.94:60098) 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 -- 172.21.15.94:0/2601017464 learned_addr learned my addr 172.21.15.94:0/2601017464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e864ab640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80109a20 0x7f7e801a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 -- 172.21.15.94:0/2601017464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80109a20 msgr2=0x7f7e801a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80109a20 0x7f7e801a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 -- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 msgr2=0x7f7e8019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 0x7f7e8019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.925+0000 7f7e854a9640 1 -- 172.21.15.94:0/2601017464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801a1d80 con 0x7f7e80104d20 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006010 tx=0x7f7e700042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e76ffd640 1 -- 172.21.15.94:0/2601017464 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7003f030 con 0x7f7e80104d20 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e76ffd640 1 -- 172.21.15.94:0/2601017464 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e70031910 con 0x7f7e80104d20 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801a2010 con 0x7f7e80104d20 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e76ffd640 1 -- 172.21.15.94:0/2601017464 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70035070 con 0x7f7e80104d20 2024-04-25T13:58:44.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.926+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a2500 con 0x7f7e80104d20 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.929+0000 7f7e76ffd640 1 -- 172.21.15.94:0/2601017464 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7e70042030 con 0x7f7e80104d20 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.929+0000 7f7e8edbd740 1 Processor -- start 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.929+0000 7f7e8edbd740 1 -- start start 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.929+0000 7f7e76ffd640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 0x7f7e6c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b11c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 0x55f347b121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e76ffd640 1 -- 172.21.15.94:0/2601017464 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f7e700ff1e0 con 0x7f7e80104d20 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e864ab640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 0x7f7e6c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b12e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f347a71d80 con 0x55f3477424d0 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f347a2ba20 con 0x55f34774a6b0 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f347a6dd40 con 0x55f34776b390 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 0x55f347b121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b11c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 0x55f347b121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60108/0 (socket says 172.21.15.94:60108) 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e854a9640 1 -- 172.21.15.94:0/1724935725 learned_addr learned my addr 172.21.15.94:0/1724935725 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e864ab640 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 -- 172.21.15.94:0/1724935725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 msgr2=0x55f347b12e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b12e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 -- 172.21.15.94:0/1724935725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 msgr2=0x55f347b121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 0x55f347b121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 -- 172.21.15.94:0/1724935725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f347a36c30 con 0x55f34776b390 2024-04-25T13:58:44.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e864ab640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 0x7f7e6c0b2000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005fd0 tx=0x7f7e78013b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.930+0000 7f7e85caa640 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b11c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e801033b0 tx=0x7f7e7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ff58e88e5105cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.931+0000 7f7e6bfff640 1 -- 172.21.15.94:0/1724935725 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c004220 con 0x55f34776b390 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.931+0000 7f7e6bfff640 1 -- 172.21.15.94:0/1724935725 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c009e10 con 0x55f34776b390 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.931+0000 7f7e6bfff640 1 -- 172.21.15.94:0/1724935725 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c018610 con 0x55f34776b390 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- 172.21.15.94:0/1724935725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 msgr2=0x55f347b11c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b11c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e801033b0 tx=0x7f7e7c011810 comp rx=0 tx=0).stop 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- 172.21.15.94:0/1724935725 shutdown_connections 2024-04-25T13:58:44.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b12e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3477424d0 0x55f347b121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/1724935725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b11c20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- 172.21.15.94:0/1724935725 >> 172.21.15.94:0/1724935725 conn(0x55f34775f4e0 msgr2=0x55f347b09020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- 172.21.15.94:0/1724935725 shutdown_connections 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- 172.21.15.94:0/1724935725 wait complete. 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 -- start start 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.932+0000 7f7e8edbd740 1 Processor -- start 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f347b13940 con 0x55f34774a6b0 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f3479f8dc0 con 0x55f3477424d0 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f347b13ac0 con 0x55f34776b390 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60122/0 (socket says 172.21.15.94:60122) 2024-04-25T13:58:44.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e854a9640 1 -- 172.21.15.94:0/2279269629 learned_addr learned my addr 172.21.15.94:0/2279269629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e8edbd740 10 client.0 _ll_register_callbacks cb 0x55f347a14600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e864ab640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 -- 172.21.15.94:0/2279269629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 msgr2=0x55f347b22dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b22dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 -- 172.21.15.94:0/2279269629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 msgr2=0x55f347b1bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 -- 172.21.15.94:0/2279269629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3477833f0 con 0x55f3477424d0 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e85caa640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1b990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c011d10 tx=0x7f7e7c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f5afa627e90be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.933+0000 7f7e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c027020 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f7e7c005040 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c009e10 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c032050 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 10 client.30962 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f34784d7d0 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c009e10 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7e7c004ab0 con 0x55f3477424d0 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 10 client.30962 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 10 client.30962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 20 client.30962 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f347729f60 0x55f347b293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 4 client.30962 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.934+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f347a7f990 con 0x55f347729f60 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.935+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f347729f60 0x55f347b293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.935+0000 7f7e697fa640 10 client.30962 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.935+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f347729f60 0x55f347b293c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1512dc42be2a4ba3 server_cookie=6f05815be8cb32b8 in_seq=0 out_seq=0 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.936+0000 7f7e697fa640 1 -- 172.21.15.94:0/2279269629 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f347a7f990 con 0x55f347729f60 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.936+0000 7f7e697fa640 10 client.30962 handle_command_reply: tid=0 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.936+0000 7f7e8edbd740 1 client.30962 shutdown 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.936+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 msgr2=0x55f347b1b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.936+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1b990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c011d10 tx=0x7f7e7c003820 comp rx=0 tx=0).stop 2024-04-25T13:58:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 shutdown_connections 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f347729f60 0x55f347b293c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b1bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2279269629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1b990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e697fa640 0 client.30962 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 >> 172.21.15.94:0/2279269629 conn(0x55f34775f4e0 msgr2=0x55f347b08040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 shutdown_connections 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2279269629 wait complete. 2024-04-25T13:58:44.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.937+0000 7f7e8edbd740 20 client.30962 trim_cache size 0 max 16384 2024-04-25T13:58:44.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.964+0000 7f7e8edbd740 1 Processor -- start 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.964+0000 7f7e8edbd740 1 -- start start 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801a92e0 con 0x55f34774a6b0 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801aae20 con 0x55f3477424d0 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e8edbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801a99b0 con 0x55f34776b390 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60132/0 (socket says 172.21.15.94:60132) 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 -- 172.21.15.94:0/2363652789 learned_addr learned my addr 172.21.15.94:0/2363652789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e85caa640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e864ab640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 -- 172.21.15.94:0/2363652789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 msgr2=0x55f347b2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 -- 172.21.15.94:0/2363652789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 msgr2=0x55f347b1bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 -- 172.21.15.94:0/2363652789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f347b19e60 con 0x55f34774a6b0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e85caa640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e854a9640 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7e700fcc40 tx=0x7f7e700ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9624cbd4baf35020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.965+0000 7f7e6b7fe640 1 -- 172.21.15.94:0/2363652789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700e0070 con 0x55f34774a6b0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.966+0000 7f7e6b7fe640 1 -- 172.21.15.94:0/2363652789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e700c7620 con 0x55f34774a6b0 2024-04-25T13:58:44.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.966+0000 7f7e6b7fe640 1 -- 172.21.15.94:0/2363652789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70049070 con 0x55f34774a6b0 2024-04-25T13:58:44.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2363652789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 msgr2=0x55f347b22ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7e700fcc40 tx=0x7f7e700ea040 comp rx=0 tx=0).stop 2024-04-25T13:58:44.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2363652789 shutdown_connections 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34776b390 0x55f347b2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34774a6b0 0x55f347b22ce0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/2363652789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3477424d0 0x55f347b1bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2363652789 >> 172.21.15.94:0/2363652789 conn(0x55f34775f4e0 msgr2=0x55f347b1b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2363652789 shutdown_connections 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- 172.21.15.94:0/2363652789 wait complete. 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 -- start start 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.967+0000 7f7e8edbd740 1 Processor -- start 2024-04-25T13:58:44.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 0x55f347b1fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34776b390 0x55f347b8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e85caa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 0x55f347b1fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e854a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59410/0 (socket says 172.21.15.94:59410) 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e854a9640 1 -- 172.21.15.94:0/3984557259 learned_addr learned my addr 172.21.15.94:0/3984557259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801aa750 con 0x55f34776b390 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e864ab640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34776b390 0x55f347b8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801aa080 con 0x55f34774a6b0 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.968+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801a72c0 con 0x55f3477424d0 2024-04-25T13:58:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e8edbd740 10 client.0 _ll_register_callbacks cb 0x7f7e70005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 -- 172.21.15.94:0/3984557259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 msgr2=0x55f347b1fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 0x55f347b1fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 -- 172.21.15.94:0/3984557259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34776b390 msgr2=0x55f347b8f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34776b390 0x55f347b8f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f347b23220 con 0x55f34774a6b0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e85caa640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 0x55f347b1fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e854a9640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e700fcfb0 tx=0x7f7e7003b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b848b510f9aed422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700e0050 con 0x55f34774a6b0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f7e700c7a90 con 0x55f34774a6b0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.969+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e700c7620 con 0x55f34774a6b0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7e70049070 con 0x55f34774a6b0 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 10 client.30968 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f347b90a70 con 0x55f34774a6b0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e700c7620 con 0x55f34774a6b0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7e700c2f90 con 0x55f34774a6b0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 10 client.30968 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 10 client.30968 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 20 client.30968 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f3476ff3a0 0x55f347b93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 4 client.30968 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f347a79800 con 0x55f3476ff3a0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.970+0000 7f7e85caa640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f3476ff3a0 0x55f347b93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e697fa640 10 client.30968 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e85caa640 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f3476ff3a0 0x55f347b93400 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f3b769a54e3b6a0 server_cookie=8d0717e7e54afd44 in_seq=0 out_seq=0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e697fa640 1 -- 172.21.15.94:0/3984557259 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f347a79800 con 0x55f3476ff3a0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e697fa640 10 client.30968 handle_command_reply: tid=0 2024-04-25T13:58:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 client.30968 shutdown 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 msgr2=0x55f347b1ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e700fcfb0 tx=0x7f7e7003b460 comp rx=0 tx=0).stop 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 shutdown_connections 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55f3476ff3a0 0x55f347b93400 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f34776b390 0x55f347b8f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e697fa640 0 client.30968 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f34774a6b0 0x55f347b1ff70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 --2- 172.21.15.94:0/3984557259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3477424d0 0x55f347b1fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 >> 172.21.15.94:0/3984557259 conn(0x55f34775f4e0 msgr2=0x55f347b1a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 shutdown_connections 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 1 -- 172.21.15.94:0/3984557259 wait complete. 2024-04-25T13:58:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.971+0000 7f7e8edbd740 20 client.30968 trim_cache size 0 max 16384 2024-04-25T13:58:44.978 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 msgr2=0x7f7e6c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 0x7f7e6c0b2000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005fd0 tx=0x7f7e78013b80 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 msgr2=0x7f7e8019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006010 tx=0x7f7e700042f0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 shutdown_connections 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e6c0afaf0 0x7f7e6c0b2000 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e80109a20 0x7f7e801a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e80104d20 0x7f7e8019a780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 --2- 172.21.15.94:0/2601017464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e80103ae0 0x7f7e8019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 >> 172.21.15.94:0/2601017464 conn(0x7f7e800fecd0 msgr2=0x7f7e80105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 shutdown_connections 2024-04-25T13:58:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:44.972+0000 7f7e87f0e640 1 -- 172.21.15.94:0/2601017464 wait complete. 2024-04-25T13:58:44.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.005+0000 7fde01c15640 1 -- 172.21.15.94:0/2011321269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc101f50 msgr2=0x7fddfc106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.005+0000 7fde01c15640 1 --2- 172.21.15.94:0/2011321269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc101f50 0x7fddfc106bc0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fddec009ca0 tx=0x7fddec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 -- 172.21.15.94:0/2011321269 shutdown_connections 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 --2- 172.21.15.94:0/2011321269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc109a60 0x7fddfc10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 --2- 172.21.15.94:0/2011321269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 --2- 172.21.15.94:0/2011321269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc101f50 0x7fddfc106bc0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 -- 172.21.15.94:0/2011321269 >> 172.21.15.94:0/2011321269 conn(0x7fddfc0fd390 msgr2=0x7fddfc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 -- 172.21.15.94:0/2011321269 shutdown_connections 2024-04-25T13:58:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 -- 172.21.15.94:0/2011321269 wait complete. 2024-04-25T13:58:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.006+0000 7fde01c15640 1 Processor -- start 2024-04-25T13:58:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 -- start start 2024-04-25T13:58:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc101f50 0x7fddfc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10f6b0 con 0x7fddfc109a60 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10f530 con 0x7fddfc107100 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fde01c15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10f830 con 0x7fddfc101f50 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60152/0 (socket says 172.21.15.94:60152) 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfbfff640 1 -- 172.21.15.94:0/2028115509 learned_addr learned my addr 172.21.15.94:0/2028115509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc101f50 0x7fddfc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 -- 172.21.15.94:0/2028115509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc101f50 msgr2=0x7fddfc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc101f50 0x7fddfc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 -- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 msgr2=0x7fddfc1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfaffd640 1 -- 172.21.15.94:0/2028115509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc1a1df0 con 0x7fddfc107100 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.007+0000 7fddfbfff640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fddfaffd640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc19a7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9f0 tx=0x7fdde400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fddf8ff9640 1 -- 172.21.15.94:0/2028115509 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde400ccc0 con 0x7fddfc107100 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc1a20e0 con 0x7fddfc107100 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fddf8ff9640 1 -- 172.21.15.94:0/2028115509 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdde40108e0 con 0x7fddfc107100 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fddf8ff9640 1 -- 172.21.15.94:0/2028115509 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4018880 con 0x7fddfc107100 2024-04-25T13:58:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.008+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a2680 con 0x7fddfc107100 2024-04-25T13:58:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 Processor -- start 2024-04-25T13:58:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 -- start start 2024-04-25T13:58:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 0x5644def45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb7f4d0 0x5644def4c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644deea6460 con 0x5644deb7f4d0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5644dee60590 con 0x5644deb772f0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fde08ba3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644deea2420 con 0x5644deba01b0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddf8ff9640 1 -- 172.21.15.94:0/2028115509 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdde4018a20 con 0x7fddfc107100 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51472/0 (socket says 172.21.15.94:51472) 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddfbfff640 1 -- 172.21.15.94:0/1415410700 learned_addr learned my addr 172.21.15.94:0/1415410700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 0x5644def45300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.011+0000 7fddfaffd640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb7f4d0 0x5644def4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 -- 172.21.15.94:0/1415410700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 msgr2=0x5644def45300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 0x5644def45300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 -- 172.21.15.94:0/1415410700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb7f4d0 msgr2=0x5644def4c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb7f4d0 0x5644def4c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 -- 172.21.15.94:0/1415410700 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644dee6d0d0 con 0x5644deba01b0 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 0x5644def45300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfbfff640 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011600 tx=0x7fddf4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef7c7943ec2db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddf8ff9640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 0x7fdde80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 0x7fdde80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddf8ff9640 1 -- 172.21.15.94:0/2028115509 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fdde40df040 con 0x7fddfc107100 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddf1ffb640 1 -- 172.21.15.94:0/1415410700 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf401e070 con 0x5644deba01b0 2024-04-25T13:58:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddf1ffb640 1 -- 172.21.15.94:0/1415410700 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf400faf0 con 0x5644deba01b0 2024-04-25T13:58:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.012+0000 7fddf1ffb640 1 -- 172.21.15.94:0/1415410700 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4019b90 con 0x5644deba01b0 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 -- 172.21.15.94:0/1415410700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 msgr2=0x5644def44d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 0x7fdde80b2070 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5644def46160 tx=0x7fddec009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011600 tx=0x7fddf4009f90 comp rx=0 tx=0).stop 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 -- 172.21.15.94:0/1415410700 shutdown_connections 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb7f4d0 0x5644def4c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb772f0 0x5644def45300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1415410700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deba01b0 0x5644def44d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 -- 172.21.15.94:0/1415410700 >> 172.21.15.94:0/1415410700 conn(0x5644deb94300 msgr2=0x5644def3c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 -- 172.21.15.94:0/1415410700 shutdown_connections 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.013+0000 7fde08ba3740 1 -- 172.21.15.94:0/1415410700 wait complete. 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 -- start start 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 Processor -- start 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def4f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def56010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644def4cc50 con 0x5644deba01b0 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5644dee2db60 con 0x5644deb7f4d0 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644def4cdd0 con 0x5644deb772f0 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fde08ba3740 10 client.0 _ll_register_callbacks cb 0x5644dee522b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.014+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51482/0 (socket says 172.21.15.94:51482) 2024-04-25T13:58:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 -- 172.21.15.94:0/3126187073 learned_addr learned my addr 172.21.15.94:0/3126187073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfaffd640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def4f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfbfff640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def56010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 -- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 msgr2=0x5644def4f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def4f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 -- 172.21.15.94:0/3126187073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 msgr2=0x5644def56010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def56010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 -- 172.21.15.94:0/3126187073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644debb8210 con 0x5644deb772f0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfbfff640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def56010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddec009ca0 tx=0x7fddec02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d271deb519f7e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fdddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec003e40 con 0x5644deb772f0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fddec004250 con 0x5644deb772f0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.015+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf400faf0 con 0x5644deb772f0 2024-04-25T13:58:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fddec004450 con 0x5644deb772f0 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 10 client.30670 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5644dec84620 con 0x5644deb772f0 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf400faf0 con 0x5644deb772f0 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fddec051070 con 0x5644deb772f0 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 10 client.30670 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 10 client.30670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 20 client.30670 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb5ed80 0x5644def5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 4 client.30670 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644deeb4070 con 0x5644deb5ed80 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.016+0000 7fddfaffd640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb5ed80 0x5644def5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.017+0000 7fdddaffd640 10 client.30670 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.017+0000 7fddfaffd640 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb5ed80 0x5644def5cbe0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=206b34a0baebfc72 server_cookie=2cc2b63c938b9c8 in_seq=0 out_seq=0 2024-04-25T13:58:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fdddaffd640 1 -- 172.21.15.94:0/3126187073 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644deeb4070 con 0x5644deb5ed80 2024-04-25T13:58:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fdddaffd640 10 client.30670 handle_command_reply: tid=0 2024-04-25T13:58:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 client.30670 shutdown 2024-04-25T13:58:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 msgr2=0x5644def4eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddec009ca0 tx=0x7fddec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 shutdown_connections 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb5ed80 0x5644def5cbe0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def56010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def4f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 --2- 172.21.15.94:0/3126187073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def4eb70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 >> 172.21.15.94:0/3126187073 conn(0x5644deb94300 msgr2=0x5644def3f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.019+0000 7fdddaffd640 0 client.30670 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.020+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 shutdown_connections 2024-04-25T13:58:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.020+0000 7fde08ba3740 1 -- 172.21.15.94:0/3126187073 wait complete. 2024-04-25T13:58:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.020+0000 7fde08ba3740 20 client.30670 trim_cache size 0 max 16384 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 Processor -- start 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 -- start start 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def49650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a9400 con 0x5644deba01b0 2024-04-25T13:58:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1aaf40 con 0x5644deb7f4d0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fde08ba3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a9ad0 con 0x5644deb772f0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60176/0 (socket says 172.21.15.94:60176) 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fddfbfff640 1 -- 172.21.15.94:0/5327750 learned_addr learned my addr 172.21.15.94:0/5327750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fddfaffd640 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.035+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def49650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 -- 172.21.15.94:0/5327750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 msgr2=0x5644def49650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def49650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 -- 172.21.15.94:0/5327750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 msgr2=0x5644def55fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def55fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 -- 172.21.15.94:0/5327750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644def4f770 con 0x5644deba01b0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddfbfff640 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011600 tx=0x7fddf40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce462e938a27349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddf17fa640 1 -- 172.21.15.94:0/5327750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf401e070 con 0x5644deba01b0 2024-04-25T13:58:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddf17fa640 1 -- 172.21.15.94:0/5327750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf401f040 con 0x5644deba01b0 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.036+0000 7fddf17fa640 1 -- 172.21.15.94:0/5327750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf40038f0 con 0x5644deba01b0 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- 172.21.15.94:0/5327750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 msgr2=0x5644def3f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011600 tx=0x7fddf40056d0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- 172.21.15.94:0/5327750 shutdown_connections 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deba01b0 0x5644def3f810 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deb7f4d0 0x5644def55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 --2- 172.21.15.94:0/5327750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb772f0 0x5644def49650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- 172.21.15.94:0/5327750 >> 172.21.15.94:0/5327750 conn(0x5644deb94300 msgr2=0x5644def3f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- 172.21.15.94:0/5327750 shutdown_connections 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- 172.21.15.94:0/5327750 wait complete. 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.037+0000 7fde08ba3740 1 -- start start 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 Processor -- start 2024-04-25T13:58:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb7f4d0 0x5644def55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deba01b0 0x5644defc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1aa870 con 0x5644deb772f0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1aa1a0 con 0x5644deba01b0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a73e0 con 0x5644deb7f4d0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fde08ba3740 10 client.0 _ll_register_callbacks cb 0x7fddf4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60184/0 (socket says 172.21.15.94:60184) 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 -- 172.21.15.94:0/1073138959 learned_addr learned my addr 172.21.15.94:0/1073138959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfbfff640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deba01b0 0x5644defc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfaffd640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb7f4d0 0x5644def55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 -- 172.21.15.94:0/1073138959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb7f4d0 msgr2=0x5644def55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb7f4d0 0x5644def55500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 -- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deba01b0 msgr2=0x5644defc2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deba01b0 0x5644defc2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.038+0000 7fddfb7fe640 1 -- 172.21.15.94:0/1073138959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644defc34c0 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fddfb7fe640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fddec009600 tx=0x7fddec041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c594cc7dc1aaafef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec004b40 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7fddec004ce0 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf401f040 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fddec030a40 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fde08ba3740 10 client.21564 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5644defc4720 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf401f040 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.039+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fddec051040 con 0x5644deb772f0 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 10 client.21564 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 10 client.21564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 20 client.21564 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb341c0 0x5644defc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 4 client.21564 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644deeadee0 con 0x5644deb341c0 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fddfaffd640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb341c0 0x5644defc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fdddaffd640 10 client.21564 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.040+0000 7fddfaffd640 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb341c0 0x5644defc7160 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70a8b99ce7ef176 server_cookie=d2d11bf571c41e33 in_seq=0 out_seq=0 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 -- 172.21.15.94:0/2303097715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc1023d0 msgr2=0x7f16bc1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/2303097715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc1023d0 0x7f16bc1047f0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009900 tx=0x7f16ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 -- 172.21.15.94:0/2303097715 shutdown_connections 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/2303097715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc107690 0x7f16bc107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/2303097715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 0x7f16bc107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/2303097715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc1023d0 0x7f16bc1047f0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.104+0000 7f16c1d4d640 1 -- 172.21.15.94:0/2303097715 >> 172.21.15.94:0/2303097715 conn(0x7f16bc0fdbc0 msgr2=0x7f16bc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.105+0000 7f16c1d4d640 1 -- 172.21.15.94:0/2303097715 shutdown_connections 2024-04-25T13:58:45.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.105+0000 7f16c1d4d640 1 -- 172.21.15.94:0/2303097715 wait complete. 2024-04-25T13:58:45.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.105+0000 7f16c1d4d640 1 Processor -- start 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.105+0000 7f16c1d4d640 1 -- start start 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc1023d0 0x7f16bc1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 0x7f16bc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc115c10 con 0x7f16bc107690 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc115a90 con 0x7f16bc104d30 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc115d90 con 0x7f16bc1023d0 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60196/0 (socket says 172.21.15.94:60196) 2024-04-25T13:58:45.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc1023d0 0x7f16bc1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 -- 172.21.15.94:0/1503201104 learned_addr learned my addr 172.21.15.94:0/1503201104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16baffd640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 0x7f16bc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 -- 172.21.15.94:0/1503201104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc1023d0 msgr2=0x7f16bc1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc1023d0 0x7f16bc1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 -- 172.21.15.94:0/1503201104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 msgr2=0x7f16bc1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 0x7f16bc1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 -- 172.21.15.94:0/1503201104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1a8360 con 0x7f16bc107690 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f16b400eb40 tx=0x7f16b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16b8ff9640 1 -- 172.21.15.94:0/1503201104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004b20 con 0x7f16bc107690 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16b8ff9640 1 -- 172.21.15.94:0/1503201104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b4004cc0 con 0x7f16bc107690 2024-04-25T13:58:45.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc1a8650 con 0x7f16bc107690 2024-04-25T13:58:45.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16b8ff9640 1 -- 172.21.15.94:0/1503201104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4010720 con 0x7f16bc107690 2024-04-25T13:58:45.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.106+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a8ba0 con 0x7f16bc107690 2024-04-25T13:58:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16b8ff9640 1 -- 172.21.15.94:0/1503201104 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f16b4005560 con 0x7f16bc107690 2024-04-25T13:58:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 Processor -- start 2024-04-25T13:58:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 -- start start 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788af390 0x55d678c51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c51d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d678bb5d80 con 0x55d67888e6b0 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d678b6fa20 con 0x55d6788864d0 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16c8c03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d678bb1d40 con 0x55d6788af390 2024-04-25T13:58:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16baffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16baffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60204/0 (socket says 172.21.15.94:60204) 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16baffd640 1 -- 172.21.15.94:0/985085797 learned_addr learned my addr 172.21.15.94:0/985085797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c51d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.109+0000 7f16baffd640 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788af390 0x55d678c51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 -- 172.21.15.94:0/985085797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788af390 msgr2=0x55d678c51810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788af390 0x55d678c51810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 -- 172.21.15.94:0/985085797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 msgr2=0x55d678c51d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c51d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 -- 172.21.15.94:0/985085797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d678b7ac30 con 0x55d67888e6b0 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16b8ff9640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 0x7f16a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16baffd640 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009fd0 tx=0x7f16b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c550c45d5dd7b4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 0x7f16a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16b8ff9640 1 -- 172.21.15.94:0/1503201104 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f16b40df400 con 0x7f16bc107690 2024-04-25T13:58:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16a1ffb640 1 -- 172.21.15.94:0/985085797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004030 con 0x55d67888e6b0 2024-04-25T13:58:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16a1ffb640 1 -- 172.21.15.94:0/985085797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0005020 con 0x55d67888e6b0 2024-04-25T13:58:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.110+0000 7f16a1ffb640 1 -- 172.21.15.94:0/985085797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004bc0 con 0x55d67888e6b0 2024-04-25T13:58:45.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- 172.21.15.94:0/985085797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 msgr2=0x55d678c52a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 0x7f16a40b2000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009230 tx=0x7f16ac031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009fd0 tx=0x7f16b0004730 comp rx=0 tx=0).stop 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- 172.21.15.94:0/985085797 shutdown_connections 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c52a20 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c51d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 --2- 172.21.15.94:0/985085797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788af390 0x55d678c51810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- 172.21.15.94:0/985085797 >> 172.21.15.94:0/985085797 conn(0x55d6788a34e0 msgr2=0x55d678c4d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- 172.21.15.94:0/985085797 shutdown_connections 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- 172.21.15.94:0/985085797 wait complete. 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 -- start start 2024-04-25T13:58:45.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.111+0000 7f16c8c03740 1 Processor -- start 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c5b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788af390 0x55d678c62a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d678c53530 con 0x55d67888e6b0 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d678b3cdc0 con 0x55d6788af390 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d678c536b0 con 0x55d6788864d0 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 10 client.0 _ll_register_callbacks cb 0x55d678b58600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60212/0 (socket says 172.21.15.94:60212) 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 -- 172.21.15.94:0/1594933621 learned_addr learned my addr 172.21.15.94:0/1594933621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788af390 0x55d678c62a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c5b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 -- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 msgr2=0x55d678c5b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c5b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 -- 172.21.15.94:0/1594933621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788af390 msgr2=0x55d678c62a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788af390 0x55d678c62a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 -- 172.21.15.94:0/1594933621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6788c73f0 con 0x55d67888e6b0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16baffd640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f16b00055d0 tx=0x7f16b0005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71928627cf02fc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f169affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:45.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004c10 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f16b0004db0 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.112+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b0005020 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f16b000f790 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 10 client.21576 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d6789917d0 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0005020 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f16b0026020 con 0x55d67888e6b0 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 10 client.21576 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 10 client.21576 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 20 client.21576 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d67886df60 0x55d678c69010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 4 client.21576 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:45.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d678bc3990 con 0x55d67886df60 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.113+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d67886df60 0x55d678c69010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.114+0000 7f169affd640 10 client.21576 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.114+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d67886df60 0x55d678c69010 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aeb34e2ec0d92554 server_cookie=e7e41e96439b595b in_seq=0 out_seq=0 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.114+0000 7f169affd640 1 -- 172.21.15.94:0/1594933621 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d678bc3990 con 0x55d67886df60 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.114+0000 7f169affd640 10 client.21576 handle_command_reply: tid=0 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 client.21576 shutdown 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 msgr2=0x55d678c5bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f16b00055d0 tx=0x7f16b0005600 comp rx=0 tx=0).stop 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 shutdown_connections 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d67886df60 0x55d678c69010 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788af390 0x55d678c62a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f169affd640 0 client.21576 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:45.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c5bb30 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1594933621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c5b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 >> 172.21.15.94:0/1594933621 conn(0x55d6788a34e0 msgr2=0x55d678c4ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 shutdown_connections 2024-04-25T13:58:45.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 1 -- 172.21.15.94:0/1594933621 wait complete. 2024-04-25T13:58:45.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.115+0000 7f16c8c03740 20 client.21576 trim_cache size 0 max 16384 2024-04-25T13:58:45.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.128+0000 7f16c8c03740 1 Processor -- start 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.128+0000 7f16c8c03740 1 -- start start 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d67886df60 0x55d678c56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b1b80 con 0x55d67888e6b0 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b36c0 con 0x55d67886df60 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16c8c03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b2250 con 0x55d6788864d0 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60220/0 (socket says 172.21.15.94:60220) 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 -- 172.21.15.94:0/89687680 learned_addr learned my addr 172.21.15.94:0/89687680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d67886df60 0x55d678c56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 -- 172.21.15.94:0/89687680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 msgr2=0x55d678c56890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 -- 172.21.15.94:0/89687680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d67886df60 msgr2=0x55d678c56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d67886df60 0x55d678c56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 -- 172.21.15.94:0/89687680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d678c5c730 con 0x55d67888e6b0 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16bbfff640 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a77d0 tx=0x7f16b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e01a36a035352f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16a17fa640 1 -- 172.21.15.94:0/89687680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40a7f40 con 0x55d67888e6b0 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16a17fa640 1 -- 172.21.15.94:0/89687680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b40a7840 con 0x55d67888e6b0 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.129+0000 7f16a17fa640 1 -- 172.21.15.94:0/89687680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40b0450 con 0x55d67888e6b0 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- 172.21.15.94:0/89687680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 msgr2=0x55d678c4ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a77d0 tx=0x7f16b400cf60 comp rx=0 tx=0).stop 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- 172.21.15.94:0/89687680 shutdown_connections 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678c4ca50 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6788864d0 0x55d678c56890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 --2- 172.21.15.94:0/89687680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d67886df60 0x55d678c56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- 172.21.15.94:0/89687680 >> 172.21.15.94:0/89687680 conn(0x55d6788a34e0 msgr2=0x55d678c51790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- 172.21.15.94:0/89687680 shutdown_connections 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- 172.21.15.94:0/89687680 wait complete. 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.130+0000 7f16c8c03740 1 -- start start 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 Processor -- start 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d67886df60 0x55d678c5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b2ff0 con 0x55d67888e6b0 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b2920 con 0x55d6788864d0 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1afb60 con 0x55d67886df60 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16c8c03740 10 client.0 _ll_register_callbacks cb 0x7f16b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60236/0 (socket says 172.21.15.94:60236) 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 -- 172.21.15.94:0/1262118484 learned_addr learned my addr 172.21.15.94:0/1262118484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16baffd640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d67886df60 0x55d678c5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 -- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d67886df60 msgr2=0x55d678c5ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d67886df60 0x55d678c5ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 -- 172.21.15.94:0/1262118484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 msgr2=0x55d678c5f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c5f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.131+0000 7f16bbfff640 1 -- 172.21.15.94:0/1262118484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d678c622e0 con 0x55d67888e6b0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16bbfff640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a4ab0 tx=0x7f16b40abbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0570e13658d5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:45.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40a7f40 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f16b40a3e30 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b40a7840 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f16b40a3c20 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 10 client.21582 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d678cd0800 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b40a7840 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f16b408a030 con 0x55d67888e6b0 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 10 client.21582 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 10 client.21582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 20 client.21582 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d6788af390 0x55d678cd3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 4 client.21582 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:45.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d678bbd800 con 0x55d6788af390 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.132+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d6788af390 0x55d678cd3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f169affd640 10 client.21582 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16bb7fe640 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d6788af390 0x55d678cd3210 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=846cef8e86157e12 server_cookie=f54ec405371747da in_seq=0 out_seq=0 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f169affd640 1 -- 172.21.15.94:0/1262118484 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d678bbd800 con 0x55d6788af390 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f169affd640 10 client.21582 handle_command_reply: tid=0 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 client.21582 shutdown 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 msgr2=0x55d678ccf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a4ab0 tx=0x7f16b40abbf0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 shutdown_connections 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d6788af390 0x55d678cd3210 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d67888e6b0 0x55d678ccf370 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f169affd640 0 client.21582 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:58:45.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6788864d0 0x55d678c5f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 --2- 172.21.15.94:0/1262118484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d67886df60 0x55d678c5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 >> 172.21.15.94:0/1262118484 conn(0x55d6788a34e0 msgr2=0x55d678c51790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 shutdown_connections 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 1 -- 172.21.15.94:0/1262118484 wait complete. 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.133+0000 7f16c8c03740 20 client.21582 trim_cache size 0 max 16384 2024-04-25T13:58:45.145 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:45.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 msgr2=0x7f16a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 0x7f16a40b2000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009230 tx=0x7f16ac031400 comp rx=0 tx=0).stop 2024-04-25T13:58:45.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 msgr2=0x7f16bc1a7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f16b400eb40 tx=0x7f16b400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 shutdown_connections 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16a40afaf0 0x7f16a40b2000 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16bc107690 0x7f16bc1a7cf0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16bc104d30 0x7f16bc1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 --2- 172.21.15.94:0/1503201104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc1023d0 0x7f16bc1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 >> 172.21.15.94:0/1503201104 conn(0x7f16bc0fdbc0 msgr2=0x7f16bc1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 shutdown_connections 2024-04-25T13:58:45.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.134+0000 7f16c1d4d640 1 -- 172.21.15.94:0/1503201104 wait complete. 2024-04-25T13:58:45.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 75797 19.31 MB/sec execute 87 sec latency 248.936 ms 2024-04-25T13:58:45.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:58:45.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.402+0000 7f731da20640 1 -- 172.21.15.94:0/1390758448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318103b00 msgr2=0x7f7318104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.402+0000 7f731da20640 1 --2- 172.21.15.94:0/1390758448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318103b00 0x7f7318104800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7310009ca0 tx=0x7f731002a8f0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.405+0000 7f731da20640 1 -- 172.21.15.94:0/1390758448 shutdown_connections 2024-04-25T13:58:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.405+0000 7f731da20640 1 --2- 172.21.15.94:0/1390758448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318109a40 0x7f731810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.405+0000 7f731da20640 1 --2- 172.21.15.94:0/1390758448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7318104d40 0x7f7318105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.405+0000 7f731da20640 1 --2- 172.21.15.94:0/1390758448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318103b00 0x7f7318104800 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7310009ca0 tx=0x7f731002a8f0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.405+0000 7f731da20640 1 -- 172.21.15.94:0/1390758448 >> 172.21.15.94:0/1390758448 conn(0x7f73180fecf0 msgr2=0x7f73181011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.410+0000 7f731da20640 1 -- 172.21.15.94:0/1390758448 shutdown_connections 2024-04-25T13:58:45.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.411+0000 7f731da20640 1 -- 172.21.15.94:0/1390758448 wait complete. 2024-04-25T13:58:45.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.413+0000 7f731da20640 1 Processor -- start 2024-04-25T13:58:45.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.413+0000 7f731da20640 1 -- start start 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.413+0000 7f731da20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 0x7f7318113400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f731da20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 0x7f7318113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f731da20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7318118920 0x7f7318118d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f731da20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7318076960 con 0x7f7318104d40 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f731da20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73180767e0 con 0x7f7318109a40 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f731da20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7318076ae0 con 0x7f7318118920 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f73167fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 0x7f7318113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f73167fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 0x7f7318113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59546/0 (socket says 172.21.15.94:59546) 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f73167fc640 1 -- 172.21.15.94:0/2949859080 learned_addr learned my addr 172.21.15.94:0/2949859080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.414+0000 7f7316ffd640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 0x7f7318113400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 -- 172.21.15.94:0/2949859080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7318118920 msgr2=0x7f7318118d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7318118920 0x7f7318118d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 -- 172.21.15.94:0/2949859080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 msgr2=0x7f7318113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 0x7f7318113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 -- 172.21.15.94:0/2949859080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7318119460 con 0x7f7318104d40 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f7316ffd640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 0x7f7318113400 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f73181031a0 tx=0x7f7310003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.415+0000 7f731ca1e640 1 -- 172.21.15.94:0/2949859080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310004040 con 0x7f7318104d40 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.416+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181196f0 con 0x7f7318104d40 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.416+0000 7f731ca1e640 1 -- 172.21.15.94:0/2949859080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731002d070 con 0x7f7318104d40 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.416+0000 7f731ca1e640 1 -- 172.21.15.94:0/2949859080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310030c10 con 0x7f7318104d40 2024-04-25T13:58:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.416+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7318119ce0 con 0x7f7318104d40 2024-04-25T13:58:45.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.421+0000 7f731ca1e640 1 -- 172.21.15.94:0/2949859080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f731004e440 con 0x7f7318104d40 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.422+0000 7f731ca1e640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 0x7f73000b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.422+0000 7f73167fc640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 0x7f73000b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.423+0000 7f731ca1e640 1 -- 172.21.15.94:0/2949859080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f7310100f30 con 0x7f7318104d40 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.425+0000 7f73167fc640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 0x7f73000b2140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7308005fd0 tx=0x7f730800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 Processor -- start 2024-04-25T13:58:45.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 -- start start 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e4836390 0x55c3e4bd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4bd9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4b3cd80 con 0x55c3e4836390 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.426+0000 7f73249ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4af6a20 con 0x55c3e48156b0 2024-04-25T13:58:45.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73249ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4b38d40 con 0x55c3e480d4d0 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51606/0 (socket says 172.21.15.94:51606) 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 -- 172.21.15.94:0/404485092 learned_addr learned my addr 172.21.15.94:0/404485092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73177fe640 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4bd9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 -- 172.21.15.94:0/404485092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 msgr2=0x55c3e4bd9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4bd9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 -- 172.21.15.94:0/404485092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e4836390 msgr2=0x55c3e4bd8810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e4836390 0x55c3e4bd8810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.427+0000 7f73167fc640 1 -- 172.21.15.94:0/404485092 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e4b01c30 con 0x55c3e480d4d0 2024-04-25T13:58:45.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.428+0000 7f73167fc640 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f730800bf70 tx=0x7f7308012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a575658666e5a1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.430+0000 7f72fdffb640 1 -- 172.21.15.94:0/404485092 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308022070 con 0x55c3e480d4d0 2024-04-25T13:58:45.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.430+0000 7f72fdffb640 1 -- 172.21.15.94:0/404485092 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73080047a0 con 0x55c3e480d4d0 2024-04-25T13:58:45.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.430+0000 7f72fdffb640 1 -- 172.21.15.94:0/404485092 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730801e6b0 con 0x55c3e480d4d0 2024-04-25T13:58:45.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.432+0000 7f73249ac740 1 -- 172.21.15.94:0/404485092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 msgr2=0x55c3e4bd8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.432+0000 7f73249ac740 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f730800bf70 tx=0x7f7308012000 comp rx=0 tx=0).stop 2024-04-25T13:58:45.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.436+0000 7f73249ac740 1 -- 172.21.15.94:0/404485092 shutdown_connections 2024-04-25T13:58:45.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.436+0000 7f73249ac740 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4bd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.436+0000 7f73249ac740 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bd8d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.436+0000 7f73249ac740 1 --2- 172.21.15.94:0/404485092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e4836390 0x55c3e4bd8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.436+0000 7f73249ac740 1 -- 172.21.15.94:0/404485092 >> 172.21.15.94:0/404485092 conn(0x55c3e482a4e0 msgr2=0x55c3e4bd4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.437+0000 7f73249ac740 1 -- 172.21.15.94:0/404485092 shutdown_connections 2024-04-25T13:58:45.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.437+0000 7f73249ac740 1 -- 172.21.15.94:0/404485092 wait complete. 2024-04-25T13:58:45.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.438+0000 7f73249ac740 1 -- start start 2024-04-25T13:58:45.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.438+0000 7f73249ac740 1 Processor -- start 2024-04-25T13:58:45.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4be2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4bda530 con 0x55c3e480d4d0 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4ac3dc0 con 0x55c3e48156b0 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3e4bda6b0 con 0x55c3e4836390 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.439+0000 7f73249ac740 10 client.0 _ll_register_callbacks cb 0x55c3e4adf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.440+0000 7f73177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.440+0000 7f73167fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4be2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.440+0000 7f73177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51622/0 (socket says 172.21.15.94:51622) 2024-04-25T13:58:45.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.440+0000 7f73177fe640 1 -- 172.21.15.94:0/946779311 learned_addr learned my addr 172.21.15.94:0/946779311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.441+0000 7f73177fe640 1 -- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 msgr2=0x55c3e4be2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.441+0000 7f73177fe640 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4be2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.442+0000 7f73177fe640 1 -- 172.21.15.94:0/946779311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 msgr2=0x55c3e4be2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.442+0000 7f73177fe640 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.442+0000 7f73177fe640 1 -- 172.21.15.94:0/946779311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e484e3f0 con 0x55c3e4836390 2024-04-25T13:58:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.443+0000 7f73177fe640 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f730c00d160 tx=0x7f730c00d620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3377f804c498f88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.443+0000 7f72f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.445+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c00de50 con 0x55c3e4836390 2024-04-25T13:58:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.445+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f730c0125f0 con 0x55c3e4836390 2024-04-25T13:58:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.445+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73080047a0 con 0x55c3e4836390 2024-04-25T13:58:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.446+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f730c0127f0 con 0x55c3e4836390 2024-04-25T13:58:45.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.446+0000 7f73249ac740 10 client.30700 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.446+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3e49187d0 con 0x55c3e4836390 2024-04-25T13:58:45.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.450+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73080047a0 con 0x55c3e4836390 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f730c013cd0 con 0x55c3e4836390 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 10 client.30700 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 10 client.30700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 20 client.30700 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47f4f60 0x55c3e4bf04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 4 client.30700 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.451+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3e4b4a990 con 0x55c3e47f4f60 2024-04-25T13:58:45.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.452+0000 7f7316ffd640 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47f4f60 0x55c3e4bf04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.455+0000 7f7316ffd640 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47f4f60 0x55c3e4bf04f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cac497033628d8e server_cookie=ec3a20e387a5b0b5 in_seq=0 out_seq=0 2024-04-25T13:58:45.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.455+0000 7f72f6ffd640 10 client.30700 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.457+0000 7f72f6ffd640 1 -- 172.21.15.94:0/946779311 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3e4b4a990 con 0x55c3e47f4f60 2024-04-25T13:58:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.457+0000 7f72f6ffd640 10 client.30700 handle_command_reply: tid=0 2024-04-25T13:58:45.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.457+0000 7f73249ac740 1 client.30700 shutdown 2024-04-25T13:58:45.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.458+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 msgr2=0x55c3e4be9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.458+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f730c00d160 tx=0x7f730c00d620 comp rx=0 tx=0).stop 2024-04-25T13:58:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 shutdown_connections 2024-04-25T13:58:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47f4f60 0x55c3e4bf04f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e4836390 0x55c3e4be9b00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e48156b0 0x55c3e4be2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 --2- 172.21.15.94:0/946779311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 >> 172.21.15.94:0/946779311 conn(0x55c3e482a4e0 msgr2=0x55c3e4bd6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f72f6ffd640 0 client.30700 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 shutdown_connections 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.459+0000 7f73249ac740 1 -- 172.21.15.94:0/946779311 wait complete. 2024-04-25T13:58:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.460+0000 7f73249ac740 20 client.30700 trim_cache size 0 max 16384 2024-04-25T13:58:45.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 Processor -- start 2024-04-25T13:58:45.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 -- start start 2024-04-25T13:58:45.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4be9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60274/0 (socket says 172.21.15.94:60274) 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 -- 172.21.15.94:0/1674378716 learned_addr learned my addr 172.21.15.94:0/1674378716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f730400e6c0 con 0x55c3e48156b0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f7316ffd640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7304010200 con 0x55c3e4836390 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f730400ed90 con 0x55c3e480d4d0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73177fe640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4be9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 -- 172.21.15.94:0/1674378716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 msgr2=0x55c3e4bdd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bdd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.482+0000 7f73167fc640 1 -- 172.21.15.94:0/1674378716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 msgr2=0x55c3e4be9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f73167fc640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4be9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f73167fc640 1 -- 172.21.15.94:0/1674378716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e4be9f60 con 0x55c3e48156b0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f7316ffd640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f73167fc640 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f7308011fd0 tx=0x7f730800b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6c2d3cf071d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f72fd7fa640 1 -- 172.21.15.94:0/1674378716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308022040 con 0x55c3e48156b0 2024-04-25T13:58:45.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f72fd7fa640 1 -- 172.21.15.94:0/1674378716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7308004730 con 0x55c3e48156b0 2024-04-25T13:58:45.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.483+0000 7f72fd7fa640 1 -- 172.21.15.94:0/1674378716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730801e770 con 0x55c3e48156b0 2024-04-25T13:58:45.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 msgr2=0x55c3e4bdd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f7308011fd0 tx=0x7f730800b570 comp rx=0 tx=0).stop 2024-04-25T13:58:45.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 shutdown_connections 2024-04-25T13:58:45.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4be9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e48156b0 0x55c3e4bdd890 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 --2- 172.21.15.94:0/1674378716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e480d4d0 0x55c3e4bdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 >> 172.21.15.94:0/1674378716 conn(0x55c3e482a4e0 msgr2=0x55c3e4be2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 shutdown_connections 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- 172.21.15.94:0/1674378716 wait complete. 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.484+0000 7f73249ac740 1 -- start start 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 Processor -- start 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e48156b0 0x55c3e4be6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4c56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f730400fb30 con 0x55c3e480d4d0 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f730400f460 con 0x55c3e4836390 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f730400c750 con 0x55c3e48156b0 2024-04-25T13:58:45.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60276/0 (socket says 172.21.15.94:60276) 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 -- 172.21.15.94:0/236545165 learned_addr learned my addr 172.21.15.94:0/236545165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73249ac740 10 client.0 _ll_register_callbacks cb 0x7f730c00b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73167fc640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e48156b0 0x55c3e4be6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f73177fe640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4c56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 -- 172.21.15.94:0/236545165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e48156b0 msgr2=0x55c3e4be6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e48156b0 0x55c3e4be6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 -- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 msgr2=0x55c3e4c56890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4c56890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.485+0000 7f7316ffd640 1 -- 172.21.15.94:0/236545165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e4be9c00 con 0x55c3e480d4d0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f7316ffd640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f73100c5390 tx=0x7f73100c9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466ce2489345d14b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73100e2020 con 0x55c3e480d4d0 2024-04-25T13:58:45.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7f73100fea40 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308004730 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73100ea050 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 10 client.21591 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3e4c57dc0 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7308004730 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f73100e5ce0 con 0x55c3e480d4d0 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 10 client.21591 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 10 client.21591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:58:45.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 20 client.21591 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47ca3a0 0x55c3e4c5a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 4 client.21591 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.486+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3e4b44800 con 0x55c3e47ca3a0 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f73167fc640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47ca3a0 0x55c3e4c5a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f72f6ffd640 10 client.21591 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f73167fc640 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47ca3a0 0x55c3e4c5a730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc400ec5b2176c1e server_cookie=24700449da93826b in_seq=0 out_seq=0 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f72f6ffd640 1 -- 172.21.15.94:0/236545165 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3e4b44800 con 0x55c3e47ca3a0 2024-04-25T13:58:45.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f72f6ffd640 10 client.21591 handle_command_reply: tid=0 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.487+0000 7f73249ac740 1 client.21591 shutdown 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 msgr2=0x55c3e4be6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f73100c5390 tx=0x7f73100c9ad0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 shutdown_connections 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c3e47ca3a0 0x55c3e4c5a730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f72f6ffd640 0 client.21591 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3e4836390 0x55c3e4c56890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3e48156b0 0x55c3e4be6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 --2- 172.21.15.94:0/236545165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3e480d4d0 0x55c3e4be6750 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 >> 172.21.15.94:0/236545165 conn(0x55c3e482a4e0 msgr2=0x55c3e4be2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 shutdown_connections 2024-04-25T13:58:45.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 1 -- 172.21.15.94:0/236545165 wait complete. 2024-04-25T13:58:45.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f73249ac740 20 client.21591 trim_cache size 0 max 16384 2024-04-25T13:58:45.497 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 msgr2=0x7f73000b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 0x7f73000b2140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7308005fd0 tx=0x7f730800b2c0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 msgr2=0x7f7318113400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 0x7f7318113400 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f73181031a0 tx=0x7f7310003ba0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 shutdown_connections 2024-04-25T13:58:45.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f73000afc30 0x7f73000b2140 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7318118920 0x7f7318118d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7318109a40 0x7f7318113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 --2- 172.21.15.94:0/2949859080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7318104d40 0x7f7318113400 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 >> 172.21.15.94:0/2949859080 conn(0x7f73180fecf0 msgr2=0x7f7318105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.488+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 shutdown_connections 2024-04-25T13:58:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.489+0000 7f731da20640 1 -- 172.21.15.94:0/2949859080 wait complete. 2024-04-25T13:58:45.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fdddaffd640 1 -- 172.21.15.94:0/1073138959 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644deeadee0 con 0x5644deb341c0 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fdddaffd640 10 client.21564 handle_command_reply: tid=0 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 client.21564 shutdown 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 msgr2=0x5644def54fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fddec009600 tx=0x7fddec041000 comp rx=0 tx=0).stop 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 shutdown_connections 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5644deb341c0 0x5644defc7160 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644deba01b0 0x5644defc2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fdddaffd640 0 client.21564 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644deb7f4d0 0x5644def55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 --2- 172.21.15.94:0/1073138959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644deb772f0 0x5644def54fc0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 >> 172.21.15.94:0/1073138959 conn(0x5644deb94300 msgr2=0x5644def4e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 shutdown_connections 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.583+0000 7fde08ba3740 1 -- 172.21.15.94:0/1073138959 wait complete. 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde08ba3740 20 client.21564 trim_cache size 0 max 16384 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:58:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:58:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 msgr2=0x7fdde80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 0x7fdde80b2070 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5644def46160 tx=0x7fddec009360 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 msgr2=0x7fddfc19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc19a7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9f0 tx=0x7fdde400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 shutdown_connections 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdde80afb60 0x7fdde80b2070 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddfc109a60 0x7fddfc1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc107100 0x7fddfc19a7f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 --2- 172.21.15.94:0/2028115509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc101f50 0x7fddfc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 >> 172.21.15.94:0/2028115509 conn(0x7fddfc0fd390 msgr2=0x7fddfc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 shutdown_connections 2024-04-25T13:58:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:45.584+0000 7fde01c15640 1 -- 172.21.15.94:0/2028115509 wait complete. 2024-04-25T13:58:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 -- 172.21.15.94:0/2822596827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c103af0 msgr2=0x7ff30c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 --2- 172.21.15.94:0/2822596827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c103af0 0x7ff30c1047f0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009900 tx=0x7ff2fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 -- 172.21.15.94:0/2822596827 shutdown_connections 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 --2- 172.21.15.94:0/2822596827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 0x7ff30c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 --2- 172.21.15.94:0/2822596827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c104d30 0x7ff30c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 --2- 172.21.15.94:0/2822596827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c103af0 0x7ff30c1047f0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 -- 172.21.15.94:0/2822596827 >> 172.21.15.94:0/2822596827 conn(0x7ff30c0fece0 msgr2=0x7ff30c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 -- 172.21.15.94:0/2822596827 shutdown_connections 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.615+0000 7ff313851640 1 -- 172.21.15.94:0/2822596827 wait complete. 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 Processor -- start 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 -- start start 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 0x7ff30c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 0x7ff30c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10f5d0 con 0x7ff30c104d30 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10f450 con 0x7ff30c103af0 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff313851640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10f750 con 0x7ff30c109a30 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60292/0 (socket says 172.21.15.94:60292) 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff3115ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 0x7ff30c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff310dec640 1 -- 172.21.15.94:0/4014918634 learned_addr learned my addr 172.21.15.94:0/4014918634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.616+0000 7ff311dee640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 0x7ff30c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 -- 172.21.15.94:0/4014918634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 msgr2=0x7ff30c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 0x7ff30c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 -- 172.21.15.94:0/4014918634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 msgr2=0x7ff30c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 0x7ff30c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 -- 172.21.15.94:0/4014918634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c1a1d70 con 0x7ff30c104d30 2024-04-25T13:58:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff3115ed640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 0x7ff30c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff310dec640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff3027fc640 1 -- 172.21.15.94:0/4014918634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800cd80 con 0x7ff30c104d30 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1a2060 con 0x7ff30c104d30 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff3027fc640 1 -- 172.21.15.94:0/4014918634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3080108e0 con 0x7ff30c104d30 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff3027fc640 1 -- 172.21.15.94:0/4014918634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018980 con 0x7ff30c104d30 2024-04-25T13:58:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.617+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a25b0 con 0x7ff30c104d30 2024-04-25T13:58:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff31a7e5740 1 Processor -- start 2024-04-25T13:58:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff31a7e5740 1 -- start start 2024-04-25T13:58:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff3027fc640 1 -- 172.21.15.94:0/4014918634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff308010430 con 0x7ff30c104d30 2024-04-25T13:58:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec866d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.620+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 0x562fec867a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562fec7cad80 con 0x562fec4c4390 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562fec784a20 con 0x562fec49b4d0 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562fec7c6d40 con 0x562fec4a36b0 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60298/0 (socket says 172.21.15.94:60298) 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 -- 172.21.15.94:0/798348342 learned_addr learned my addr 172.21.15.94:0/798348342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff3115ed640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec866d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff310dec640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 0x562fec867a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 -- 172.21.15.94:0/798348342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 msgr2=0x562fec867a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 0x562fec867a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 -- 172.21.15.94:0/798348342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 msgr2=0x562fec866d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec866d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 -- 172.21.15.94:0/798348342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fec78fc30 con 0x562fec4c4390 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff310dec640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 0x562fec867a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff311dee640 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff304009fd0 tx=0x7ff3040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90195b68a151956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff3027fc640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 0x7ff2f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff2f77fe640 1 -- 172.21.15.94:0/798348342 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304004d20 con 0x562fec4c4390 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff2f77fe640 1 -- 172.21.15.94:0/798348342 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff304004050 con 0x562fec4c4390 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff2f77fe640 1 -- 172.21.15.94:0/798348342 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304005280 con 0x562fec4c4390 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.621+0000 7ff3115ed640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 0x7ff2f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff3027fc640 1 -- 172.21.15.94:0/4014918634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff3080def90 con 0x7ff30c104d30 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 -- 172.21.15.94:0/798348342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 msgr2=0x562fec866810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff3115ed640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 0x7ff2f80b2000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005ab0 tx=0x7ff2fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff304009fd0 tx=0x7ff3040118d0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 -- 172.21.15.94:0/798348342 shutdown_connections 2024-04-25T13:58:45.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4a36b0 0x562fec867a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec866d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/798348342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4c4390 0x562fec866810 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 -- 172.21.15.94:0/798348342 >> 172.21.15.94:0/798348342 conn(0x562fec4b84e0 msgr2=0x562fec862020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 -- 172.21.15.94:0/798348342 shutdown_connections 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.622+0000 7ff31a7e5740 1 -- 172.21.15.94:0/798348342 wait complete. 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 -- start start 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 Processor -- start 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec870620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec877ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562fec868530 con 0x562fec4a36b0 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562fec751dc0 con 0x562fec49b4d0 2024-04-25T13:58:45.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562fec8686b0 con 0x562fec4c4390 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff31a7e5740 10 client.0 _ll_register_callbacks cb 0x562fec76d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60310/0 (socket says 172.21.15.94:60310) 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff310dec640 1 -- 172.21.15.94:0/1818276018 learned_addr learned my addr 172.21.15.94:0/1818276018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff3115ed640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec870620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.623+0000 7ff311dee640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec877ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 -- 172.21.15.94:0/1818276018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 msgr2=0x562fec877ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec877ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 -- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 msgr2=0x562fec870620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec870620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 -- 172.21.15.94:0/1818276018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fec4dc3f0 con 0x562fec4a36b0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff310dec640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7ff3080a7410 tx=0x7ff3080a7350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d340e56d4636c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080a6f90 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff3080e2580 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff304004050 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff3080a7480 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff31a7e5740 10 client.21603 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562fec5a67d0 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff304004050 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.624+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff308018c90 con 0x562fec4a36b0 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 10 client.21603 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 10 client.21603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 20 client.21603 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec482f60 0x562fec87e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 4 client.21603 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fec7d8990 con 0x562fec482f60 2024-04-25T13:58:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff3115ed640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec482f60 0x562fec87e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff2f4ff9640 10 client.21603 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.625+0000 7ff3115ed640 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec482f60 0x562fec87e380 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a498d239c8f3d0 server_cookie=5e73bfd988a36a80 in_seq=0 out_seq=0 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/1818276018 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562fec7d8990 con 0x562fec482f60 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff2f4ff9640 10 client.21603 handle_command_reply: tid=0 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 client.21603 shutdown 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 msgr2=0x562fec870b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7ff3080a7410 tx=0x7ff3080a7350 comp rx=0 tx=0).stop 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 shutdown_connections 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec482f60 0x562fec87e380 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec877ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec870b60 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff2f4ff9640 0 client.21603 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/1818276018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec870620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 >> 172.21.15.94:0/1818276018 conn(0x562fec4b84e0 msgr2=0x562fec864f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 shutdown_connections 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 1 -- 172.21.15.94:0/1818276018 wait complete. 2024-04-25T13:58:45.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.630+0000 7ff31a7e5740 20 client.21603 trim_cache size 0 max 16384 2024-04-25T13:58:45.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 Processor -- start 2024-04-25T13:58:45.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 -- start start 2024-04-25T13:58:45.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec86f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec883040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1ab590 con 0x562fec4a36b0 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1ad0d0 con 0x562fec49b4d0 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1abc60 con 0x562fec4c4390 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff3115ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec86f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff310dec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60316/0 (socket says 172.21.15.94:60316) 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff310dec640 1 -- 172.21.15.94:0/2332837702 learned_addr learned my addr 172.21.15.94:0/2332837702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.644+0000 7ff311dee640 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec883040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 -- 172.21.15.94:0/2332837702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 msgr2=0x562fec883040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec883040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 -- 172.21.15.94:0/2332837702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 msgr2=0x562fec86f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec86f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 -- 172.21.15.94:0/2332837702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fec8663d0 con 0x562fec4a36b0 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff3115ed640 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec86f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff310dec640 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c6a0 tx=0x7ff30800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2858ceef1b90cac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff2f6ffd640 1 -- 172.21.15.94:0/2332837702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080a6bf0 con 0x562fec4a36b0 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff2f6ffd640 1 -- 172.21.15.94:0/2332837702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3080a6f90 con 0x562fec4a36b0 2024-04-25T13:58:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.645+0000 7ff2f6ffd640 1 -- 172.21.15.94:0/2332837702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080a63c0 con 0x562fec4a36b0 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2332837702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 msgr2=0x562fec882b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c6a0 tx=0x7ff30800c7d0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2332837702 shutdown_connections 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec4c4390 0x562fec883040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec882b00 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2332837702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec49b4d0 0x562fec86f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2332837702 >> 172.21.15.94:0/2332837702 conn(0x562fec4b84e0 msgr2=0x562fec86ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2332837702 shutdown_connections 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.646+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2332837702 wait complete. 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 -- start start 2024-04-25T13:58:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 Processor -- start 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec8e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 0x562fec8e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1aca00 con 0x562fec4a36b0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.647+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1ac330 con 0x562fec4c4390 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff31a7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1a9570 con 0x562fec49b4d0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff31a7e5740 10 client.0 _ll_register_callbacks cb 0x7ff3080a3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff311dee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 0x562fec8e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51700/0 (socket says 172.21.15.94:51700) 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff311dee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 0x562fec8e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59618/0 (socket says 172.21.15.94:59618) 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 -- 172.21.15.94:0/2441200079 learned_addr learned my addr 172.21.15.94:0/2441200079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff310dec640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec8e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 -- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 msgr2=0x562fec8e8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 0x562fec8e8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 -- 172.21.15.94:0/2441200079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 msgr2=0x562fec8e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec8e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 -- 172.21.15.94:0/2441200079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fec86bda0 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff3115ed640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0096c0 tx=0x7ff2fc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae8c733fd395d0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.648+0000 7ff2f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc035040 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3080a6f90 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8385+0+0 (secure 0 0 0) 0x7ff2fc054680 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc064050 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 10 client.30724 fetch_fsmap learned FSMap version 37 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562fec8ea230 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3080a6f90 con 0x562fec49b4d0 2024-04-25T13:58:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff2fc044070 con 0x562fec49b4d0 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 10 client.30724 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 10 client.30724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 20 client.30724 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec4583a0 0x562fec8ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 4 client.30724 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fec7d2800 con 0x562fec4583a0 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.649+0000 7ff310dec640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec4583a0 0x562fec8ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.650+0000 7ff2f4ff9640 10 client.30724 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.650+0000 7ff310dec640 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec4583a0 0x562fec8ecc00 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80fa1de545cc1df server_cookie=42a7ca67211481be in_seq=0 out_seq=0 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff2f4ff9640 1 -- 172.21.15.94:0/2441200079 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fec7d2800 con 0x562fec4583a0 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff2f4ff9640 10 client.30724 handle_command_reply: tid=0 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 client.30724 shutdown 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 msgr2=0x562fec8e1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0096c0 tx=0x7ff2fc009260 comp rx=0 tx=0).stop 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 shutdown_connections 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562fec4583a0 0x562fec8ecc00 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562fec4c4390 0x562fec8e8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fec4a36b0 0x562fec8e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff2f4ff9640 0 client.30724 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:45.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 --2- 172.21.15.94:0/2441200079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562fec49b4d0 0x562fec8e1650 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.656+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 >> 172.21.15.94:0/2441200079 conn(0x562fec4b84e0 msgr2=0x562fec864ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 shutdown_connections 2024-04-25T13:58:45.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff31a7e5740 1 -- 172.21.15.94:0/2441200079 wait complete. 2024-04-25T13:58:45.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff31a7e5740 20 client.30724 trim_cache size 0 max 16384 2024-04-25T13:58:45.658 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 msgr2=0x7ff2f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 0x7ff2f80b2000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005ab0 tx=0x7ff2fc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 msgr2=0x7ff30c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 shutdown_connections 2024-04-25T13:58:45.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff2f80afaf0 0x7ff2f80b2000 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c109a30 0x7ff30c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff30c104d30 0x7ff30c19a770 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 --2- 172.21.15.94:0/4014918634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c103af0 0x7ff30c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 >> 172.21.15.94:0/4014918634 conn(0x7ff30c0fece0 msgr2=0x7ff30c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 shutdown_connections 2024-04-25T13:58:45.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:45.657+0000 7ff313851640 1 -- 172.21.15.94:0/4014918634 wait complete. 2024-04-25T13:58:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:45 smithi137 ceph-mon[32568]: pgmap v859: 97 pgs: 97 active+clean; 255 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-04-25T13:58:45.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:45 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3077503350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:45 smithi094 ceph-mon[29195]: pgmap v859: 97 pgs: 97 active+clean; 255 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-04-25T13:58:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:45 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3077503350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:45.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-mon[32977]: pgmap v859: 97 pgs: 97 active+clean; 255 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-04-25T13:58:45.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3077503350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:45.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T13:58:45.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T13:58:45.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/3015828940 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cb7d8fa40 con 0x558cb7dd7180 2024-04-25T13:58:45.998 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 13:58:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T13:58:46.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 75991 19.24 MB/sec execute 88 sec latency 341.264 ms 2024-04-25T13:58:46.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:58:46.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:58:46.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.798+0000 7fdfa8e20640 1 -- 172.21.15.94:0/787301647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101740 msgr2=0x7fdfa4101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.798+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/787301647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101740 0x7fdfa4101bb0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009900 tx=0x7fdf8c02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 -- 172.21.15.94:0/787301647 shutdown_connections 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/787301647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4102240 0x7fdfa410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/787301647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101740 0x7fdfa4101bb0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/787301647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4105090 0x7fdfa41054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 -- 172.21.15.94:0/787301647 >> 172.21.15.94:0/787301647 conn(0x7fdfa40fd340 msgr2=0x7fdfa40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 -- 172.21.15.94:0/787301647 shutdown_connections 2024-04-25T13:58:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.799+0000 7fdfa8e20640 1 -- 172.21.15.94:0/787301647 wait complete. 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 Processor -- start 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 -- start start 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 0x7fdfa419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4105090 0x7fdfa41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f600 con 0x7fdfa4102240 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f480 con 0x7fdfa4105090 2024-04-25T13:58:46.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.800+0000 7fdfa8e20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa410f780 con 0x7fdfa4101740 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60330/0 (socket says 172.21.15.94:60330) 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2340890533 learned_addr learned my addr 172.21.15.94:0/2340890533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdfa259c640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 0x7fdfa419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4105090 0x7fdfa41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2340890533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 msgr2=0x7fdfa419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 0x7fdfa419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2340890533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4105090 msgr2=0x7fdfa41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4105090 0x7fdfa41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2340890533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa41a1d80 con 0x7fdfa4102240 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdfa259c640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 0x7fdfa419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.801+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c006010 tx=0x7fdf8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.802+0000 7fdf9b7fe640 1 -- 172.21.15.94:0/2340890533 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c03f030 con 0x7fdfa4102240 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.802+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa41a2010 con 0x7fdfa4102240 2024-04-25T13:58:46.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.802+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa41a2550 con 0x7fdfa4102240 2024-04-25T13:58:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.802+0000 7fdf9b7fe640 1 -- 172.21.15.94:0/2340890533 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c031910 con 0x7fdfa4102240 2024-04-25T13:58:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.802+0000 7fdf9b7fe640 1 -- 172.21.15.94:0/2340890533 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c035070 con 0x7fdfa4102240 2024-04-25T13:58:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 Processor -- start 2024-04-25T13:58:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdf9b7fe640 1 -- 172.21.15.94:0/2340890533 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdf8c042030 con 0x7fdfa4102240 2024-04-25T13:58:46.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 -- start start 2024-04-25T13:58:46.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d85f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572d8555b00 con 0x5572d822e730 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572d84f1920 con 0x5572d8226550 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572d8551ac0 con 0x5572d824f410 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60342/0 (socket says 172.21.15.94:60342) 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfa259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2632363137 learned_addr learned my addr 172.21.15.94:0/2632363137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.805+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d85f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9b7fe640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 0x7fdf900b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2632363137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 msgr2=0x5572d85f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d85f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2632363137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 msgr2=0x5572d85f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 -- 172.21.15.94:0/2632363137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d832f7e0 con 0x5572d822e730 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdfa259c640 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 0x7fdf900b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0e7670 tx=0x7fdf8c0c3bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e91dbd939aba372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf9b7fe640 1 -- 172.21.15.94:0/2340890533 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdf8c0feff0 con 0x7fdfa4102240 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf98ff9640 1 -- 172.21.15.94:0/2632363137 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c049070 con 0x5572d822e730 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 0x7fdf900b2080 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf98ff9640 1 -- 172.21.15.94:0/2632363137 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c0c6ff0 con 0x5572d822e730 2024-04-25T13:58:46.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.806+0000 7fdf98ff9640 1 -- 172.21.15.94:0/2632363137 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0c72b0 con 0x5572d822e730 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 -- 172.21.15.94:0/2632363137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 msgr2=0x5572d85f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0e7670 tx=0x7fdf8c0c3bf0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 -- 172.21.15.94:0/2632363137 shutdown_connections 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85f7920 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/2632363137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d85f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 -- 172.21.15.94:0/2632363137 >> 172.21.15.94:0/2632363137 conn(0x5572d8243560 msgr2=0x5572d85ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.807+0000 7fdfafdaf740 1 -- 172.21.15.94:0/2632363137 shutdown_connections 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 -- 172.21.15.94:0/2632363137 wait complete. 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 -- start start 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 Processor -- start 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d8601420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5572d85f8130 con 0x5572d822e730 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572d84ef5d0 con 0x5572d8226550 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572d85f82b0 con 0x5572d824f410 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdfafdaf740 10 client.0 _ll_register_callbacks cb 0x5572d84dd080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:46.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60354/0 (socket says 172.21.15.94:60354) 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.808+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1575354197 learned_addr learned my addr 172.21.15.94:0/1575354197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d8601420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1575354197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 msgr2=0x5572d8601420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d8601420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 msgr2=0x5572d85f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1575354197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d838ff70 con 0x5572d822e730 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0fcaa0 tx=0x7fdf8c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0bcb21b05c9134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c049070 con 0x5572d822e730 2024-04-25T13:58:46.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdf8c0fbb20 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c0c6ff0 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c0c6c50 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdfafdaf740 10 client.21627 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.809+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572d8227170 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c0c6ff0 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdf8c0c3770 con 0x5572d822e730 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 10 client.21627 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 10 client.21627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 20 client.21627 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d820e1f0 0x5572d8607760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 4 client.21627 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572d8563710 con 0x5572d820e1f0 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.810+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d820e1f0 0x5572d8607760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.811+0000 7fdf89ffb640 10 client.21627 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.811+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d820e1f0 0x5572d8607760 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d994a6bff82a4d server_cookie=7fa59cb225106e47 in_seq=0 out_seq=0 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1575354197 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572d8563710 con 0x5572d820e1f0 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdf89ffb640 10 client.21627 handle_command_reply: tid=0 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 client.21627 shutdown 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 msgr2=0x5572d85fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0fcaa0 tx=0x7fdf8c02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 shutdown_connections 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d820e1f0 0x5572d8607760 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d8601420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85fa4c0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdf89ffb640 0 client.21627 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1575354197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 >> 172.21.15.94:0/1575354197 conn(0x5572d8243560 msgr2=0x5572d85eaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 shutdown_connections 2024-04-25T13:58:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1575354197 wait complete. 2024-04-25T13:58:46.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.816+0000 7fdfafdaf740 20 client.21627 trim_cache size 0 max 16384 2024-04-25T13:58:46.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 Processor -- start 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 -- start start 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d860d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400e730 con 0x5572d822e730 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84010270 con 0x5572d8226550 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400ee00 con 0x5572d824f410 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60364/0 (socket says 172.21.15.94:60364) 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 -- 172.21.15.94:0/609508063 learned_addr learned my addr 172.21.15.94:0/609508063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfa259c640 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d860d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 -- 172.21.15.94:0/609508063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 msgr2=0x5572d860d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d860d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 -- 172.21.15.94:0/609508063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 msgr2=0x5572d85f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.836+0000 7fdf9bfff640 1 -- 172.21.15.94:0/609508063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d85f16a0 con 0x5572d822e730 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.837+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0fcdc0 tx=0x7fdf8c0c1b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee439e57f99693f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.837+0000 7fdf8bfff640 1 -- 172.21.15.94:0/609508063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c049020 con 0x5572d822e730 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.837+0000 7fdf8bfff640 1 -- 172.21.15.94:0/609508063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c1028a0 con 0x5572d822e730 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.837+0000 7fdf8bfff640 1 -- 172.21.15.94:0/609508063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0c6e30 con 0x5572d822e730 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- 172.21.15.94:0/609508063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 msgr2=0x5572d860cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0fcdc0 tx=0x7fdf8c0c1b00 comp rx=0 tx=0).stop 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- 172.21.15.94:0/609508063 shutdown_connections 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d824f410 0x5572d860d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d860cf20 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/609508063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d8226550 0x5572d85f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- 172.21.15.94:0/609508063 >> 172.21.15.94:0/609508063 conn(0x5572d8243560 msgr2=0x5572d85f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- 172.21.15.94:0/609508063 shutdown_connections 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- 172.21.15.94:0/609508063 wait complete. 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- start start 2024-04-25T13:58:46.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 Processor -- start 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d8226550 0x5572d85ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d824f410 0x5572d86008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400fba0 con 0x5572d822e730 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400f4d0 con 0x5572d824f410 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8400c710 con 0x5572d8226550 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfafdaf740 10 client.0 _ll_register_callbacks cb 0x7fdf8c1070e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60370/0 (socket says 172.21.15.94:60370) 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1600113915 learned_addr learned my addr 172.21.15.94:0/1600113915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.838+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d8226550 0x5572d85ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdfa2d9d640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d824f410 0x5572d86008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1600113915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d8226550 msgr2=0x5572d85ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d8226550 0x5572d85ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d824f410 msgr2=0x5572d86008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d824f410 0x5572d86008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 -- 172.21.15.94:0/1600113915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d8600e70 con 0x5572d822e730 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf9bfff640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c03b460 tx=0x7fdf8c0c3c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0bb03a5691ae7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c049070 con 0x5572d822e730 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdf8c0d0000 con 0x5572d822e730 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c1028a0 con 0x5572d822e730 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c0c6e30 con 0x5572d822e730 2024-04-25T13:58:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdfafdaf740 10 client.21633 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572d8600e70 con 0x5572d822e730 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.839+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf8c1028a0 con 0x5572d822e730 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdf8c0c77d0 con 0x5572d822e730 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 10 client.21633 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 10 client.21633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 20 client.21633 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d81e3630 0x5572d8672090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 4 client.21633 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5572d855d580 con 0x5572d81e3630 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d81e3630 0x5572d8672090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdfa259c640 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d81e3630 0x5572d8672090 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597803d5c6a9003b server_cookie=7e81d217000a49a8 in_seq=0 out_seq=0 2024-04-25T13:58:46.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.840+0000 7fdf89ffb640 10 client.21633 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdf89ffb640 1 -- 172.21.15.94:0/1600113915 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5572d855d580 con 0x5572d81e3630 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdf89ffb640 10 client.21633 handle_command_reply: tid=0 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 client.21633 shutdown 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 msgr2=0x5572d85ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c03b460 tx=0x7fdf8c0c3c60 comp rx=0 tx=0).stop 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 shutdown_connections 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5572d81e3630 0x5572d8672090 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572d824f410 0x5572d86008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572d822e730 0x5572d85ff980 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdf89ffb640 0 client.21633 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 --2- 172.21.15.94:0/1600113915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572d8226550 0x5572d85ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 >> 172.21.15.94:0/1600113915 conn(0x5572d8243560 msgr2=0x5572d85f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 shutdown_connections 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 1 -- 172.21.15.94:0/1600113915 wait complete. 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfafdaf740 20 client.21633 trim_cache size 0 max 16384 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "e1008845-4d84-4229-87c0-d8775ed11725", 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:58:46.848 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 msgr2=0x7fdf900b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 0x7fdf900b2080 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009fd0 tx=0x7fdf9c004a40 comp rx=0 tx=0).stop 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 msgr2=0x7fdfa419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.845+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c006010 tx=0x7fdf8c0042f0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 shutdown_connections 2024-04-25T13:58:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdf900afb70 0x7fdf900b2080 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa4105090 0x7fdfa41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4102240 0x7fdfa419a780 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 --2- 172.21.15.94:0/2340890533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101740 0x7fdfa419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 >> 172.21.15.94:0/2340890533 conn(0x7fdfa40fd340 msgr2=0x7fdfa4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 shutdown_connections 2024-04-25T13:58:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.846+0000 7fdfa8e20640 1 -- 172.21.15.94:0/2340890533 wait complete. 2024-04-25T13:58:46.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:58:46.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:58:46.875 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:46 smithi094 ceph-mon[29195]: daemon mds.f restarted 2024-04-25T13:58:46.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:46 smithi137 ceph-mon[32568]: daemon mds.f restarted 2024-04-25T13:58:46.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/3936512817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64102230 msgr2=0x7f4d6410bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/3936512817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64102230 0x7f4d6410bc90 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009900 tx=0x7f4d6002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/3936512817 shutdown_connections 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/3936512817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64102230 0x7f4d6410bc90 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/3936512817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64101730 0x7f4d64101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/3936512817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d64105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/3936512817 >> 172.21.15.94:0/3936512817 conn(0x7f4d640fd350 msgr2=0x7f4d640ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/3936512817 shutdown_connections 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.976+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/3936512817 wait complete. 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 Processor -- start 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 -- start start 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64102230 0x7f4d6419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d641a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410f610 con 0x7f4d64101730 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410f490 con 0x7f4d64102230 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6c7c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410f790 con 0x7f4d64105080 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60386/0 (socket says 172.21.15.94:60386) 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 -- 172.21.15.94:0/4086558803 learned_addr learned my addr 172.21.15.94:0/4086558803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d69d60640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64102230 0x7f4d6419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d641a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 -- 172.21.15.94:0/4086558803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 msgr2=0x7f4d641a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d641a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 -- 172.21.15.94:0/4086558803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64102230 msgr2=0x7f4d6419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64102230 0x7f4d6419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6a561640 1 -- 172.21.15.94:0/4086558803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d641a1c90 con 0x7f4d64101730 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.977+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d641a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00eb50 tx=0x7f4d5c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d5b7fe640 1 -- 172.21.15.94:0/4086558803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c003b70 con 0x7f4d64101730 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d641a1f80 con 0x7f4d64101730 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d5b7fe640 1 -- 172.21.15.94:0/4086558803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c00ee50 con 0x7f4d64101730 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d5b7fe640 1 -- 172.21.15.94:0/4086558803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c010790 con 0x7f4d64101730 2024-04-25T13:58:46.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.978+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641a23e0 con 0x7f4d64101730 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.980+0000 7f4d73679740 1 Processor -- start 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.980+0000 7f4d73679740 1 -- start start 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.980+0000 7f4d5b7fe640 1 -- 172.21.15.94:0/4086558803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4d5c004020 con 0x7f4d64101730 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52e9be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e52dfcd80 con 0x560e52acd4d0 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e52db68b0 con 0x560e52af6390 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d73679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e52df8d40 con 0x560e52ad56b0 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6ad62640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52e9be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60400/0 (socket says 172.21.15.94:60400) 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51762/0 (socket says 172.21.15.94:51762) 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6ad62640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52e9be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59702/0 (socket says 172.21.15.94:59702) 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 -- 172.21.15.94:0/3957294380 learned_addr learned my addr 172.21.15.94:0/3957294380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d5b7fe640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 0x7f4d500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 -- 172.21.15.94:0/3957294380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 msgr2=0x560e52e9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 -- 172.21.15.94:0/3957294380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 msgr2=0x560e52e9be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52e9be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6a561640 1 -- 172.21.15.94:0/3957294380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e52d87a40 con 0x560e52acd4d0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d69d60640 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 0x7f4d500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.981+0000 7f4d5b7fe640 1 -- 172.21.15.94:0/4086558803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4d5c0df570 con 0x7f4d64101730 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.982+0000 7f4d6a561640 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0dcf10 tx=0x7f4d5c0a4130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e7c7acd02e4baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.982+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 0x7f4d500b2000 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4d60031500 tx=0x7f4d60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.982+0000 7f4d58ff9640 1 -- 172.21.15.94:0/3957294380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0a7780 con 0x560e52acd4d0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.982+0000 7f4d58ff9640 1 -- 172.21.15.94:0/3957294380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c08ee50 con 0x560e52acd4d0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.982+0000 7f4d58ff9640 1 -- 172.21.15.94:0/3957294380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0acef0 con 0x560e52acd4d0 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- 172.21.15.94:0/3957294380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 msgr2=0x560e52e9c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0dcf10 tx=0x7f4d5c0a4130 comp rx=0 tx=0).stop 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- 172.21.15.94:0/3957294380 shutdown_connections 2024-04-25T13:58:46.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52ad56b0 0x560e52e9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52e9c3f0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- 172.21.15.94:0/3957294380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52e9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- 172.21.15.94:0/3957294380 >> 172.21.15.94:0/3957294380 conn(0x560e52aea4e0 msgr2=0x560e52e93270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- 172.21.15.94:0/3957294380 shutdown_connections 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- 172.21.15.94:0/3957294380 wait complete. 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- start start 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 Processor -- start 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52acd4d0 0x560e52ea5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52ead180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e52e9db90 con 0x560e52ad56b0 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.983+0000 7f4d73679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e52d83d10 con 0x560e52acd4d0 2024-04-25T13:58:46.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d73679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e52e9dd10 con 0x560e52af6390 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d73679740 10 client.0 _ll_register_callbacks cb 0x560e52db0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60404/0 (socket says 172.21.15.94:60404) 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 -- 172.21.15.94:0/1474063758 learned_addr learned my addr 172.21.15.94:0/1474063758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52acd4d0 0x560e52ea5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52ead180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 -- 172.21.15.94:0/1474063758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 msgr2=0x560e52ead180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52ead180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 -- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52acd4d0 msgr2=0x560e52ea5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52acd4d0 0x560e52ea5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 -- 172.21.15.94:0/1474063758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e52bd87d0 con 0x560e52ad56b0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d69d60640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f4d54012720 tx=0x7f4d54012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c5af17b3bfa5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:46.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54004030 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4d540041d0 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5c08ee50 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4d54004410 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d73679740 10 client.21645 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.984+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e52b0e3f0 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c08ee50 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4d54003bc0 con 0x560e52ad56b0 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 10 client.21645 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 10 client.21645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 20 client.21645 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52ab4f60 0x560e52eb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 4 client.21645 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e52e0a990 con 0x560e52ab4f60 2024-04-25T13:58:46.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d6a561640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52ab4f60 0x560e52eb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d4dffb640 10 client.21645 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.985+0000 7f4d6a561640 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52ab4f60 0x560e52eb36a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0840b54d7fb2bf2 server_cookie=15059f9ff44f5b77 in_seq=0 out_seq=0 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d4dffb640 1 -- 172.21.15.94:0/1474063758 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e52e0a990 con 0x560e52ab4f60 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d4dffb640 10 client.21645 handle_command_reply: tid=0 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 client.21645 shutdown 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 msgr2=0x560e52ea6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f4d54012720 tx=0x7f4d54012be0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 shutdown_connections 2024-04-25T13:58:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52ab4f60 0x560e52eb36a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52ead180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d4dffb640 0 client.21645 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea6220 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 --2- 172.21.15.94:0/1474063758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52acd4d0 0x560e52ea5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 >> 172.21.15.94:0/1474063758 conn(0x560e52aea4e0 msgr2=0x560e52e96180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 shutdown_connections 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 1 -- 172.21.15.94:0/1474063758 wait complete. 2024-04-25T13:58:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:46.989+0000 7f4d73679740 20 client.21645 trim_cache size 0 max 16384 2024-04-25T13:58:46.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:46 smithi138 ceph-mon[32977]: daemon mds.f restarted 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 Processor -- start 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 -- start start 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52acd4d0 0x560e52ea10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52ead0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641a91e0 con 0x560e52ad56b0 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641aad20 con 0x560e52af6390 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d73679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641a98b0 con 0x560e52acd4d0 2024-04-25T13:58:47.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.008+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60408/0 (socket says 172.21.15.94:60408) 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 -- 172.21.15.94:0/474888047 learned_addr learned my addr 172.21.15.94:0/474888047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d6a561640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52acd4d0 0x560e52ea10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52ead0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 -- 172.21.15.94:0/474888047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52acd4d0 msgr2=0x560e52ea10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52acd4d0 0x560e52ea10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 -- 172.21.15.94:0/474888047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 msgr2=0x560e52ead0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52ead0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 -- 172.21.15.94:0/474888047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e52ace0f0 con 0x560e52ad56b0 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52ead0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d69d60640 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f4d54026610 tx=0x7f4d540057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f5ea8b5c1efb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:47.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d4ffff640 1 -- 172.21.15.94:0/474888047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54026750 con 0x560e52ad56b0 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d4ffff640 1 -- 172.21.15.94:0/474888047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d54003f90 con 0x560e52ad56b0 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.009+0000 7f4d4ffff640 1 -- 172.21.15.94:0/474888047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540035f0 con 0x560e52ad56b0 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- 172.21.15.94:0/474888047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 msgr2=0x560e52ea4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f4d54026610 tx=0x7f4d540057d0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- 172.21.15.94:0/474888047 shutdown_connections 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52af6390 0x560e52ead0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52ad56b0 0x560e52ea4a20 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 --2- 172.21.15.94:0/474888047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52acd4d0 0x560e52ea10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- 172.21.15.94:0/474888047 >> 172.21.15.94:0/474888047 conn(0x560e52aea4e0 msgr2=0x560e52ea4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- 172.21.15.94:0/474888047 shutdown_connections 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- 172.21.15.94:0/474888047 wait complete. 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.010+0000 7f4d73679740 1 -- start start 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 Processor -- start 2024-04-25T13:58:47.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52ad56b0 0x560e52eaab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52f1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641aa650 con 0x560e52acd4d0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641a9f80 con 0x560e52ad56b0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641a71c0 con 0x560e52af6390 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d73679740 10 client.0 _ll_register_callbacks cb 0x7f4d54009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60410/0 (socket says 172.21.15.94:60410) 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 -- 172.21.15.94:0/4256266363 learned_addr learned my addr 172.21.15.94:0/4256266363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52f1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d69d60640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52ad56b0 0x560e52eaab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 -- 172.21.15.94:0/4256266363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 msgr2=0x560e52f1e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:47.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52f1e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 -- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52ad56b0 msgr2=0x560e52eaab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52ad56b0 0x560e52eaab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 -- 172.21.15.94:0/4256266363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e52dbd4b0 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6ad62640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52f1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d6a561640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00cba0 tx=0x7f4d5c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d78907b97085b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0a7120 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.011+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4d5c0ad130 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d54003f90 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c0a7300 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 10 client.21651 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e52f1f100 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d54003f90 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4d5c0a8ce0 con 0x560e52acd4d0 2024-04-25T13:58:47.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 10 client.21651 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 10 client.21651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 20 client.21651 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52a8a3a0 0x560e52f21630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 4 client.21651 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e52e04800 con 0x560e52a8a3a0 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.012+0000 7f4d69d60640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52a8a3a0 0x560e52f21630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.013+0000 7f4d4dffb640 10 client.21651 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:47.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.013+0000 7f4d69d60640 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52a8a3a0 0x560e52f21630 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0ac3d633567e9e server_cookie=34a207312084e073 in_seq=0 out_seq=0 2024-04-25T13:58:47.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d4dffb640 1 -- 172.21.15.94:0/4256266363 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560e52e04800 con 0x560e52a8a3a0 2024-04-25T13:58:47.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d4dffb640 10 client.21651 handle_command_reply: tid=0 2024-04-25T13:58:47.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 client.21651 shutdown 2024-04-25T13:58:47.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 msgr2=0x560e52eaa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00cba0 tx=0x7f4d5c004830 comp rx=0 tx=0).stop 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 shutdown_connections 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560e52a8a3a0 0x560e52f21630 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e52af6390 0x560e52f1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d4dffb640 0 client.21651 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.015+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e52ad56b0 0x560e52eaab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d73679740 1 --2- 172.21.15.94:0/4256266363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e52acd4d0 0x560e52eaa5f0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 >> 172.21.15.94:0/4256266363 conn(0x560e52aea4e0 msgr2=0x560e52e919c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 shutdown_connections 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d73679740 1 -- 172.21.15.94:0/4256266363 wait complete. 2024-04-25T13:58:47.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d73679740 20 client.21651 trim_cache size 0 max 16384 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (355 inodes in the stack)", 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "e1008845-4d84-4229-87c0-d8775ed11725": { 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "e1008845-4d84-4229-87c0-d8775ed11725", 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:58:47.021 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:58:47.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 msgr2=0x7f4d500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:47.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 0x7f4d500b2000 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4d60031500 tx=0x7f4d60009260 comp rx=0 tx=0).stop 2024-04-25T13:58:47.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 msgr2=0x7f4d6419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:47.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00eb50 tx=0x7f4d5c00c6a0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 shutdown_connections 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d500afaf0 0x7f4d500b2000 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d64105080 0x7f4d641a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d64102230 0x7f4d6419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 --2- 172.21.15.94:0/4086558803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d64101730 0x7f4d6419a150 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 >> 172.21.15.94:0/4086558803 conn(0x7f4d640fd350 msgr2=0x7f4d64107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 shutdown_connections 2024-04-25T13:58:47.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:58:47.016+0000 7f4d6c7c5640 1 -- 172.21.15.94:0/4086558803 wait complete. 2024-04-25T13:58:47.027 INFO:tasks.cephfs.filesystem:scrub status for tag:e1008845-4d84-4229-87c0-d8775ed11725 - {'path': '/', 'tag': 'e1008845-4d84-4229-87c0-d8775ed11725', 'options': 'recursive,force'} 2024-04-25T13:58:47.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 76353 19.22 MB/sec execute 89 sec latency 77.139 ms 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: pgmap v860: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6834/677168211,v1:172.21.15.138:6835/677168211] up:boot 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:58:47.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:47.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:47 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: pgmap v860: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-04-25T13:58:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:58:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6834/677168211,v1:172.21.15.138:6835/677168211] up:boot 2024-04-25T13:58:47.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:47.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:47 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: pgmap v860: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6834/677168211,v1:172.21.15.138:6835/677168211] up:boot 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:47.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:47.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:47.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:47 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 76702 19.22 MB/sec execute 90 sec latency 111.577 ms 2024-04-25T13:58:48.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:58:48.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:48 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:58:48.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:58:48.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:48 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:58:48.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:48 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:49.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 77049 19.22 MB/sec execute 91 sec latency 54.929 ms 2024-04-25T13:58:49.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:49.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:49 smithi137 ceph-mon[32568]: pgmap v862: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-04-25T13:58:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:49.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:49 smithi094 ceph-mon[29195]: pgmap v862: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-04-25T13:58:49.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:58:49.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:49 smithi138 ceph-mon[32977]: pgmap v862: 97 pgs: 97 active+clean; 260 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 449 op/s 2024-04-25T13:58:50.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 77367 19.22 MB/sec execute 92 sec latency 291.271 ms 2024-04-25T13:58:51.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 77700 19.17 MB/sec execute 93 sec latency 131.830 ms 2024-04-25T13:58:51.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:51 smithi137 ceph-mon[32568]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 630 op/s 2024-04-25T13:58:51.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:51 smithi094 ceph-mon[29195]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 630 op/s 2024-04-25T13:58:51.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:51 smithi138 ceph-mon[32977]: pgmap v863: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 630 op/s 2024-04-25T13:58:52.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 78099 19.21 MB/sec execute 94 sec latency 55.457 ms 2024-04-25T13:58:53.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 78515 19.28 MB/sec execute 95 sec latency 79.526 ms 2024-04-25T13:58:53.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:53 smithi137 ceph-mon[32568]: pgmap v864: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 636 op/s 2024-04-25T13:58:53.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:53 smithi094 ceph-mon[29195]: pgmap v864: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 636 op/s 2024-04-25T13:58:53.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:53 smithi138 ceph-mon[32977]: pgmap v864: 97 pgs: 97 active+clean; 246 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 636 op/s 2024-04-25T13:58:54.141 INFO:tasks.workunit.client.0.smithi094.stdout: 10 78995 19.30 MB/sec execute 96 sec latency 54.436 ms 2024-04-25T13:58:55.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 79267 19.26 MB/sec execute 97 sec latency 246.253 ms 2024-04-25T13:58:55.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:55 smithi137 ceph-mon[32568]: pgmap v865: 97 pgs: 97 active+clean; 253 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2024-04-25T13:58:55.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:55 smithi094 ceph-mon[29195]: pgmap v865: 97 pgs: 97 active+clean; 253 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2024-04-25T13:58:55.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:55 smithi138 ceph-mon[32977]: pgmap v865: 97 pgs: 97 active+clean; 253 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.8 MiB/s wr, 669 op/s 2024-04-25T13:58:56.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 79628 19.29 MB/sec execute 98 sec latency 107.733 ms 2024-04-25T13:58:56.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:56.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:56.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:56.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:56.897 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:56.897 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:56.898 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:56.898 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:57.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 80033 19.33 MB/sec execute 99 sec latency 93.160 ms 2024-04-25T13:58:57.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:57 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:58:57.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:57 smithi094 ceph-mon[29195]: pgmap v866: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-04-25T13:58:57.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:57 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:58:57.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:57 smithi138 ceph-mon[32977]: pgmap v866: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-04-25T13:58:58.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 80459 19.33 MB/sec execute 100 sec latency 53.372 ms 2024-04-25T13:58:58.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:57 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:58:58.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:57 smithi137 ceph-mon[32568]: pgmap v866: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 603 op/s 2024-04-25T13:58:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.690+0000 7f0c6949f640 1 -- 172.21.15.94:0/2897024842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64108980 msgr2=0x7f0c6410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.690+0000 7f0c6949f640 1 --2- 172.21.15.94:0/2897024842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64108980 0x7f0c6410ada0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009ca0 tx=0x7f0c5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 -- 172.21.15.94:0/2897024842 shutdown_connections 2024-04-25T13:58:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 --2- 172.21.15.94:0/2897024842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64108980 0x7f0c6410ada0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 --2- 172.21.15.94:0/2897024842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64103c80 0x7f0c64104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 --2- 172.21.15.94:0/2897024842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64102a40 0x7f0c64103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 -- 172.21.15.94:0/2897024842 >> 172.21.15.94:0/2897024842 conn(0x7f0c640fdc30 msgr2=0x7f0c641000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 -- 172.21.15.94:0/2897024842 shutdown_connections 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 -- 172.21.15.94:0/2897024842 wait complete. 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.691+0000 7f0c6949f640 1 Processor -- start 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 -- start start 2024-04-25T13:58:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64103c80 0x7f0c6419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 0x7f0c641a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6410e570 con 0x7f0c64102a40 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6410e3f0 con 0x7f0c64103c80 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c6949f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6410e6f0 con 0x7f0c64108980 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49646/0 (socket says 172.21.15.94:49646) 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3891982106 learned_addr learned my addr 172.21.15.94:0/3891982106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64103c80 0x7f0c6419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c637fe640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 0x7f0c641a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3891982106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 msgr2=0x7f0c641a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 0x7f0c641a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3891982106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64103c80 msgr2=0x7f0c6419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64103c80 0x7f0c6419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3891982106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c641a1d40 con 0x7f0c64102a40 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.692+0000 7f0c637fe640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 0x7f0c641a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9f0 tx=0x7f0c5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c60ff9640 1 -- 172.21.15.94:0/3891982106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400ccc0 con 0x7f0c64102a40 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c641a1fd0 con 0x7f0c64102a40 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c60ff9640 1 -- 172.21.15.94:0/3891982106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c540035f0 con 0x7f0c64102a40 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c60ff9640 1 -- 172.21.15.94:0/3891982106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400ce60 con 0x7f0c64102a40 2024-04-25T13:58:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.693+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c641a2490 con 0x7f0c64102a40 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 Processor -- start 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 -- start start 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c60ff9640 1 -- 172.21.15.94:0/3891982106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0c54010690 con 0x7f0c64102a40 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d084970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d084ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f6cfea460 con 0x555f6cce41b0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555f6cfa4590 con 0x555f6ccc34d0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c7042c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f6cfe6420 con 0x555f6ccbb2f0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d084970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d084ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d084970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49650/0 (socket says 172.21.15.94:49650) 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c637fe640 1 -- 172.21.15.94:0/575290676 learned_addr learned my addr 172.21.15.94:0/575290676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.696+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 -- 172.21.15.94:0/575290676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 msgr2=0x555f6d08bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 -- 172.21.15.94:0/575290676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 msgr2=0x555f6d084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d084970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 -- 172.21.15.94:0/575290676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6cfb10d0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c60ff9640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 0x7f0c500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d084ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0c540dca70 tx=0x7f0c540e71a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef820b30c75516fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c637fe640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 0x7f0c500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c597fa640 1 -- 172.21.15.94:0/575290676 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540a3340 con 0x555f6ccbb2f0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c60ff9640 1 -- 172.21.15.94:0/3891982106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0c540df040 con 0x7f0c64102a40 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c597fa640 1 -- 172.21.15.94:0/575290676 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c540a6f90 con 0x555f6ccbb2f0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.697+0000 7f0c597fa640 1 -- 172.21.15.94:0/575290676 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540a6bf0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.698+0000 7f0c7042c740 1 -- 172.21.15.94:0/575290676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 msgr2=0x555f6d084ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.698+0000 7f0c637fe640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 0x7f0c500b1ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0094e0 tx=0x7f0c5c031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.698+0000 7f0c7042c740 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d084ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0c540dca70 tx=0x7f0c540e71a0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- 172.21.15.94:0/575290676 shutdown_connections 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d084ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- 172.21.15.94:0/575290676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d084970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- 172.21.15.94:0/575290676 >> 172.21.15.94:0/575290676 conn(0x555f6ccd8300 msgr2=0x555f6d080300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- 172.21.15.94:0/575290676 shutdown_connections 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- 172.21.15.94:0/575290676 wait complete. 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- start start 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 Processor -- start 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d095a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f6d08c830 con 0x555f6cce41b0 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555f6cf71b60 con 0x555f6ccc34d0 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f6d08c9b0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c7042c740 10 client.0 _ll_register_callbacks cb 0x555f6cf962b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d095a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52152/0 (socket says 172.21.15.94:52152) 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3853295558 learned_addr learned my addr 172.21.15.94:0/3853295558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.699+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d095a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49664/0 (socket says 172.21.15.94:49664) 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 msgr2=0x555f6d08eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3853295558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 msgr2=0x555f6d095a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d095a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 -- 172.21.15.94:0/3853295558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6ccfc210 con 0x555f6ccbb2f0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0c540dce10 tx=0x7f0c54003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec47b760ff9c561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:58:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c540a6dd0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c540a6f90 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0c5401ade0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0c540a7250 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c7042c740 10 client.30778 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.700+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555f6cdc8620 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c540a6f90 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0c540a76e0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 10 client.30778 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 10 client.30778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 20 client.30778 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cca2d80 0x555f6d09bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 4 client.30778 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f6cff8070 con 0x555f6cca2d80 2024-04-25T13:58:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.701+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cca2d80 0x555f6d09bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.702+0000 7f0c4a7fc640 10 client.30778 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.702+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cca2d80 0x555f6d09bed0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755e9e93d881536b server_cookie=c6b0bd7ae00fd134 in_seq=0 out_seq=0 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3853295558 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f6cff8070 con 0x555f6cca2d80 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c4a7fc640 10 client.30778 handle_command_reply: tid=0 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 client.30778 shutdown 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 msgr2=0x555f6d08e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0c540dce10 tx=0x7f0c54003e70 comp rx=0 tx=0).stop 2024-04-25T13:58:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 shutdown_connections 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cca2d80 0x555f6d09bed0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6cce41b0 0x555f6d095a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccc34d0 0x555f6d08eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3853295558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d08e5e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c4a7fc640 0 client.30778 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 >> 172.21.15.94:0/3853295558 conn(0x555f6ccd8300 msgr2=0x555f6d083210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 shutdown_connections 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 1 -- 172.21.15.94:0/3853295558 wait complete. 2024-04-25T13:58:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.709+0000 7f0c7042c740 20 client.30778 trim_cache size 0 max 16384 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 Processor -- start 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 -- start start 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccbb2f0 0x555f6d083890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6cce41b0 0x555f6d07fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641ad7e0 con 0x555f6ccc34d0 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641af320 con 0x555f6ccbb2f0 2024-04-25T13:58:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c7042c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641adeb0 con 0x555f6cce41b0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49678/0 (socket says 172.21.15.94:49678) 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/2895805209 learned_addr learned my addr 172.21.15.94:0/2895805209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c637fe640 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6cce41b0 0x555f6d07fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.727+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccbb2f0 0x555f6d083890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/2895805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6cce41b0 msgr2=0x555f6d07fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6cce41b0 0x555f6d07fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/2895805209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccbb2f0 msgr2=0x555f6d083890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccbb2f0 0x555f6d083890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/2895805209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6d089470 con 0x555f6ccc34d0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x555f6d09c500 tx=0x7f0c4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c2bacbf1e37996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c58ff9640 1 -- 172.21.15.94:0/2895805209 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0200f0 con 0x555f6ccc34d0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c58ff9640 1 -- 172.21.15.94:0/2895805209 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c025070 con 0x555f6ccc34d0 2024-04-25T13:58:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.728+0000 7f0c58ff9640 1 -- 172.21.15.94:0/2895805209 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0107b0 con 0x555f6ccc34d0 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- 172.21.15.94:0/2895805209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 msgr2=0x555f6d095a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x555f6d09c500 tx=0x7f0c4c005f90 comp rx=0 tx=0).stop 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- 172.21.15.94:0/2895805209 shutdown_connections 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6cce41b0 0x555f6d07fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d095a10 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 --2- 172.21.15.94:0/2895805209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6ccbb2f0 0x555f6d083890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- 172.21.15.94:0/2895805209 >> 172.21.15.94:0/2895805209 conn(0x555f6ccd8300 msgr2=0x555f6d083180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- 172.21.15.94:0/2895805209 shutdown_connections 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- 172.21.15.94:0/2895805209 wait complete. 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 -- start start 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.729+0000 7f0c7042c740 1 Processor -- start 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d093490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6cce41b0 0x555f6d094900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641aec50 con 0x555f6ccc34d0 2024-04-25T13:58:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641ae580 con 0x555f6cce41b0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641ab7c0 con 0x555f6ccbb2f0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c7042c740 10 client.0 _ll_register_callbacks cb 0x7f0c54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49686/0 (socket says 172.21.15.94:49686) 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/3016027636 learned_addr learned my addr 172.21.15.94:0/3016027636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c637fe640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6cce41b0 0x555f6d094900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d093490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/3016027636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 msgr2=0x555f6d093490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d093490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6cce41b0 msgr2=0x555f6d094900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6cce41b0 0x555f6d094900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 -- 172.21.15.94:0/3016027636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6d094ec0 con 0x555f6ccc34d0 2024-04-25T13:58:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c5a7fc640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0096e0 tx=0x7f0c4c004010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bf86c201549e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.730+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c010530 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0c4c0106d0 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c025070 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0c4c0108d0 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 10 client.21669 fetch_fsmap learned FSMap version 39 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555f6d1042f0 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c025070 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0c4c020880 con 0x555f6ccc34d0 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 10 client.21669 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 10 client.21669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 20 client.21669 populate_metadata read hostname 'smithi094' 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cc781c0 0x555f6d106c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 4 client.21669 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:58:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f6d0852f0 con 0x555f6cc781c0 2024-04-25T13:58:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.731+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cc781c0 0x555f6d106c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.732+0000 7f0c4a7fc640 10 client.21669 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:58.732+0000 7f0c62ffd640 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cc781c0 0x555f6d106c70 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb399ad3f8e500b3 server_cookie=4768762f18ad866e in_seq=0 out_seq=0 2024-04-25T13:58:59.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 80753 19.31 MB/sec execute 101 sec latency 217.815 ms 2024-04-25T13:58:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c4a7fc640 1 -- 172.21.15.94:0/3016027636 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f6d0852f0 con 0x555f6cc781c0 2024-04-25T13:58:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c4a7fc640 10 client.21669 handle_command_reply: tid=0 2024-04-25T13:58:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 client.21669 shutdown 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 msgr2=0x555f6d0939d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0096e0 tx=0x7f0c4c004010 comp rx=0 tx=0).stop 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 shutdown_connections 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555f6cc781c0 0x555f6d106c70 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c4a7fc640 0 client.21669 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555f6cce41b0 0x555f6d094900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f6ccc34d0 0x555f6d0939d0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 --2- 172.21.15.94:0/3016027636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f6ccbb2f0 0x555f6d093490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 >> 172.21.15.94:0/3016027636 conn(0x555f6ccd8300 msgr2=0x555f6d08e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 shutdown_connections 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 1 -- 172.21.15.94:0/3016027636 wait complete. 2024-04-25T13:58:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.283+0000 7f0c7042c740 20 client.21669 trim_cache size 0 max 16384 2024-04-25T13:58:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:58:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:58:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:58:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 msgr2=0x7f0c500b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 0x7f0c500b1ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0094e0 tx=0x7f0c5c031af0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 msgr2=0x7f0c6406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9f0 tx=0x7f0c5400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 shutdown_connections 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0c500afae0 0x7f0c500b1ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c64108980 0x7f0c641a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c64103c80 0x7f0c6419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 --2- 172.21.15.94:0/3891982106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c64102a40 0x7f0c6406bce0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 >> 172.21.15.94:0/3891982106 conn(0x7f0c640fdc30 msgr2=0x7f0c64104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 shutdown_connections 2024-04-25T13:58:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:58:59.284+0000 7f0c6949f640 1 -- 172.21.15.94:0/3891982106 wait complete. 2024-04-25T13:58:59.944 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:58:59 smithi137 ceph-mon[32568]: pgmap v867: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:58:59.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:58:59 smithi094 ceph-mon[29195]: pgmap v867: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:58:59.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:58:59 smithi138 ceph-mon[32977]: pgmap v867: 97 pgs: 97 active+clean; 249 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 520 op/s 2024-04-25T13:59:00.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 81034 19.26 MB/sec execute 102 sec latency 310.147 ms 2024-04-25T13:59:01.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 81386 19.24 MB/sec execute 103 sec latency 117.200 ms 2024-04-25T13:59:01.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:01 smithi094 ceph-mon[29195]: pgmap v868: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-04-25T13:59:01.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:01 smithi138 ceph-mon[32977]: pgmap v868: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-04-25T13:59:02.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 81837 19.30 MB/sec execute 104 sec latency 60.571 ms 2024-04-25T13:59:02.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:01 smithi137 ceph-mon[32568]: pgmap v868: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-04-25T13:59:03.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 82223 19.32 MB/sec execute 105 sec latency 54.182 ms 2024-04-25T13:59:03.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:03 smithi094 ceph-mon[29195]: pgmap v869: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 415 op/s 2024-04-25T13:59:03.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:03 smithi138 ceph-mon[32977]: pgmap v869: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 415 op/s 2024-04-25T13:59:04.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 82649 19.32 MB/sec execute 106 sec latency 80.991 ms 2024-04-25T13:59:04.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:03 smithi137 ceph-mon[32568]: pgmap v869: 97 pgs: 97 active+clean; 257 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 415 op/s 2024-04-25T13:59:05.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 82964 19.37 MB/sec execute 107 sec latency 114.574 ms 2024-04-25T13:59:05.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:05 smithi094 ceph-mon[29195]: pgmap v870: 97 pgs: 97 active+clean; 250 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-04-25T13:59:05.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:05 smithi138 ceph-mon[32977]: pgmap v870: 97 pgs: 97 active+clean; 250 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-04-25T13:59:06.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 83294 19.33 MB/sec execute 108 sec latency 182.673 ms 2024-04-25T13:59:06.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:05 smithi137 ceph-mon[32568]: pgmap v870: 97 pgs: 97 active+clean; 250 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.7 MiB/s wr, 595 op/s 2024-04-25T13:59:07.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 83661 19.34 MB/sec execute 109 sec latency 67.886 ms 2024-04-25T13:59:07.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:07 smithi094 ceph-mon[29195]: pgmap v871: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T13:59:07.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:07 smithi138 ceph-mon[32977]: pgmap v871: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T13:59:08.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 84091 19.36 MB/sec execute 110 sec latency 53.353 ms 2024-04-25T13:59:08.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:07 smithi137 ceph-mon[32568]: pgmap v871: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T13:59:09.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 84477 19.36 MB/sec execute 111 sec latency 104.969 ms 2024-04-25T13:59:09.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:09 smithi094 ceph-mon[29195]: pgmap v872: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 488 op/s 2024-04-25T13:59:09.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:09 smithi138 ceph-mon[32977]: pgmap v872: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 488 op/s 2024-04-25T13:59:10.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 84855 19.37 MB/sec execute 112 sec latency 81.118 ms 2024-04-25T13:59:10.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:09 smithi137 ceph-mon[32568]: pgmap v872: 97 pgs: 97 active+clean; 251 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 488 op/s 2024-04-25T13:59:11.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 85155 19.35 MB/sec execute 113 sec latency 306.573 ms 2024-04-25T13:59:11.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:11 smithi094 ceph-mon[29195]: pgmap v873: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-04-25T13:59:11.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:11 smithi138 ceph-mon[32977]: pgmap v873: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-04-25T13:59:12.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:11 smithi137 ceph-mon[32568]: pgmap v873: 97 pgs: 97 active+clean; 258 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-04-25T13:59:12.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 85603 19.41 MB/sec execute 114 sec latency 52.197 ms 2024-04-25T13:59:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/2997177154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28108110 msgr2=0x7f6a2810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/2997177154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28108110 0x7f6a2810a530 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000adb0 tx=0x7f6a2002b7a0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/2997177154 shutdown_connections 2024-04-25T13:59:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/2997177154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28108110 0x7f6a2810a530 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/2997177154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28103410 0x7f6a28103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/2997177154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 0x7f6a28102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/2997177154 >> 172.21.15.94:0/2997177154 conn(0x7f6a280fd3c0 msgr2=0x7f6a280ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/2997177154 shutdown_connections 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.385+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/2997177154 wait complete. 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 Processor -- start 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 -- start start 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 0x7f6a281996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28108110 0x7f6a28198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810d460 con 0x7f6a28103410 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810d2e0 con 0x7f6a281021d0 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2f7b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810d5e0 con 0x7f6a28108110 2024-04-25T13:59:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34260/0 (socket says 172.21.15.94:34260) 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1267296217 learned_addr learned my addr 172.21.15.94:0/1267296217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 0x7f6a281996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28108110 0x7f6a28198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1267296217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28108110 msgr2=0x7f6a28198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28108110 0x7f6a28198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1267296217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 msgr2=0x7f6a281996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 0x7f6a281996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.386+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1267296217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a281989c0 con 0x7f6a28103410 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800d950 tx=0x7f6a1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a1e7fc640 1 -- 172.21.15.94:0/1267296217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18017070 con 0x7f6a28103410 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a1e7fc640 1 -- 172.21.15.94:0/1267296217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a18009c30 con 0x7f6a28103410 2024-04-25T13:59:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a281a4280 con 0x7f6a28103410 2024-04-25T13:59:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a1e7fc640 1 -- 172.21.15.94:0/1267296217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800f620 con 0x7f6a28103410 2024-04-25T13:59:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.387+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a4770 con 0x7f6a28103410 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a1e7fc640 1 -- 172.21.15.94:0/1267296217 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6a18010460 con 0x7f6a28103410 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 Processor -- start 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 -- start start 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3876e1b0 0x558d38b0e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 0x558d38b0eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d38a74460 con 0x558d3874d4d0 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558d38a2e590 con 0x558d387452f0 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a36669740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d38a70420 con 0x558d3876e1b0 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34274/0 (socket says 172.21.15.94:34274) 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 0x558d38b0eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/808141387 learned_addr learned my addr 172.21.15.94:0/808141387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3876e1b0 0x558d38b0e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a1e7fc640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 0x7f6a140b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 0x7f6a140b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 -- 172.21.15.94:0/808141387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3876e1b0 msgr2=0x558d38b0e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3876e1b0 0x558d38b0e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 -- 172.21.15.94:0/808141387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 msgr2=0x558d38b15df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2d54e640 1 -- 172.21.15.94:0/808141387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d38a3b0d0 con 0x558d387452f0 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.390+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a1e7fc640 1 -- 172.21.15.94:0/1267296217 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6a180de780 con 0x7f6a28103410 2024-04-25T13:59:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 0x558d38b0eef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a28102dd0 tx=0x7f6a2400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c641fc4b13aed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 0x7f6a140b2070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005cc0 tx=0x7f6a2002b7a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a137fe640 1 -- 172.21.15.94:0/808141387 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24003ba0 con 0x558d387452f0 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a137fe640 1 -- 172.21.15.94:0/808141387 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a24015070 con 0x558d387452f0 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.391+0000 7f6a137fe640 1 -- 172.21.15.94:0/808141387 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240116a0 con 0x558d387452f0 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- 172.21.15.94:0/808141387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 msgr2=0x558d38b0eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 0x558d38b0eef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a28102dd0 tx=0x7f6a2400bb20 comp rx=0 tx=0).stop 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- 172.21.15.94:0/808141387 shutdown_connections 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3874d4d0 0x558d38b15df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d387452f0 0x558d38b0eef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 --2- 172.21.15.94:0/808141387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3876e1b0 0x558d38b0e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- 172.21.15.94:0/808141387 >> 172.21.15.94:0/808141387 conn(0x558d38762300 msgr2=0x558d38b0a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- 172.21.15.94:0/808141387 shutdown_connections 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- 172.21.15.94:0/808141387 wait complete. 2024-04-25T13:59:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 -- start start 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.392+0000 7f6a36669740 1 Processor -- start 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3874d4d0 0x558d38b18b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 0x558d38b1faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d38b16840 con 0x558d387452f0 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558d389fbb60 con 0x558d3876e1b0 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d38b169c0 con 0x558d3874d4d0 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a36669740 10 client.0 _ll_register_callbacks cb 0x558d38a202b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34276/0 (socket says 172.21.15.94:34276) 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 -- 172.21.15.94:0/3847469270 learned_addr learned my addr 172.21.15.94:0/3847469270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 0x558d38b1faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3874d4d0 0x558d38b18b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 -- 172.21.15.94:0/3847469270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3874d4d0 msgr2=0x558d38b18b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3874d4d0 0x558d38b18b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 -- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 msgr2=0x558d38b1faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 0x558d38b1faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 -- 172.21.15.94:0/3847469270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d38786210 con 0x558d387452f0 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 0x558d38b1faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d84887a7a29657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402a060 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.393+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6a2401a470 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a24015070 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a2401ae60 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 10 client.21681 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558d38852620 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a24015070 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6a24011b00 con 0x558d387452f0 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 10 client.21681 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 10 client.21681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 20 client.21681 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3872cd80 0x558d38b26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 4 client.21681 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d38a82070 con 0x558d3872cd80 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.394+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3872cd80 0x558d38b26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.395+0000 7f6a10ff9640 10 client.21681 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.395+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3872cd80 0x558d38b26100 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e67e6d67540346 server_cookie=9f0ebf428625c61 in_seq=0 out_seq=0 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a10ff9640 1 -- 172.21.15.94:0/3847469270 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d38a82070 con 0x558d3872cd80 2024-04-25T13:59:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a10ff9640 10 client.21681 handle_command_reply: tid=0 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 client.21681 shutdown 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 msgr2=0x558d38b18650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24005760 comp rx=0 tx=0).stop 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 shutdown_connections 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3872cd80 0x558d38b26100 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3876e1b0 0x558d38b1faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a10ff9640 0 client.21681 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3874d4d0 0x558d38b18b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 --2- 172.21.15.94:0/3847469270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b18650 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.396+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 >> 172.21.15.94:0/3847469270 conn(0x558d38762300 msgr2=0x558d38b0d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.397+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 shutdown_connections 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.397+0000 7f6a36669740 1 -- 172.21.15.94:0/3847469270 wait complete. 2024-04-25T13:59:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.397+0000 7f6a36669740 20 client.21681 trim_cache size 0 max 16384 2024-04-25T13:59:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.413+0000 7f6a36669740 1 Processor -- start 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.413+0000 7f6a36669740 1 -- start start 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 0x558d38b1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b0d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281ad810 con 0x558d3872cd80 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281af350 con 0x558d3874d4d0 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a36669740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281adee0 con 0x558d387452f0 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34280/0 (socket says 172.21.15.94:34280) 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 -- 172.21.15.94:0/2420264164 learned_addr learned my addr 172.21.15.94:0/2420264164 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b0d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 0x558d38b1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 -- 172.21.15.94:0/2420264164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 msgr2=0x558d38b1fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 0x558d38b1fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 -- 172.21.15.94:0/2420264164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 msgr2=0x558d38b0d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b0d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 -- 172.21.15.94:0/2420264164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d38b13500 con 0x558d3872cd80 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 0x558d38b1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6a240043b0 tx=0x7f6a2400b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6c27ef45b584b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a12ffd640 1 -- 172.21.15.94:0/2420264164 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402a050 con 0x558d3872cd80 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a12ffd640 1 -- 172.21.15.94:0/2420264164 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2401f070 con 0x558d3872cd80 2024-04-25T13:59:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.414+0000 7f6a12ffd640 1 -- 172.21.15.94:0/2420264164 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401a8a0 con 0x558d3872cd80 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 -- 172.21.15.94:0/2420264164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 msgr2=0x558d38b0a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f6a240043b0 tx=0x7f6a2400b520 comp rx=0 tx=0).stop 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 -- 172.21.15.94:0/2420264164 shutdown_connections 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b0d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d387452f0 0x558d38b1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 --2- 172.21.15.94:0/2420264164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d3872cd80 0x558d38b0a360 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 -- 172.21.15.94:0/2420264164 >> 172.21.15.94:0/2420264164 conn(0x558d38762300 msgr2=0x558d38b09c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 -- 172.21.15.94:0/2420264164 shutdown_connections 2024-04-25T13:59:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.415+0000 7f6a36669740 1 -- 172.21.15.94:0/2420264164 wait complete. 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 -- start start 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 Processor -- start 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 0x558d38b1cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281aec80 con 0x558d387452f0 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281ae5b0 con 0x558d3874d4d0 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a281ab7f0 con 0x558d3872cd80 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a36669740 10 client.0 _ll_register_callbacks cb 0x7f6a180e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a2cd4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34284/0 (socket says 172.21.15.94:34284) 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1291740738 learned_addr learned my addr 172.21.15.94:0/1291740738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a2dd4f640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.416+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 0x558d38b1cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1291740738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 msgr2=0x558d38b1cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 0x558d38b1cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 msgr2=0x558d38b1e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b1e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 -- 172.21.15.94:0/1291740738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d38b1ff50 con 0x558d387452f0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 0x558d38b1cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a2cd4d640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000c00 tx=0x7f6a180a3340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22aa9abd5280187d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18020040 con 0x558d387452f0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6a180af790 con 0x558d387452f0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2401f070 con 0x558d387452f0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a18009de0 con 0x558d387452f0 2024-04-25T13:59:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 10 client.21687 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558d38b0a8a0 con 0x558d387452f0 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2401f070 con 0x558d387452f0 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6a180af040 con 0x558d387452f0 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 10 client.21687 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 10 client.21687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 20 client.21687 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3876e1b0 0x558d38b907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 4 client.21687 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.417+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d38a7bee0 con 0x558d3876e1b0 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.418+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3876e1b0 0x558d38b907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.418+0000 7f6a10ff9640 10 client.21687 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.418+0000 7f6a2d54e640 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3876e1b0 0x558d38b907c0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cbd1d5514cf740 server_cookie=e61bf376d1adc1ec in_seq=0 out_seq=0 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a10ff9640 1 -- 172.21.15.94:0/1291740738 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d38a7bee0 con 0x558d3876e1b0 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a10ff9640 10 client.21687 handle_command_reply: tid=0 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 client.21687 shutdown 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 msgr2=0x558d38b1d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000c00 tx=0x7f6a180a3340 comp rx=0 tx=0).stop 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 shutdown_connections 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558d3876e1b0 0x558d38b907c0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d3874d4d0 0x558d38b1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d387452f0 0x558d38b1d430 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 --2- 172.21.15.94:0/1291740738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d3872cd80 0x558d38b1cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 >> 172.21.15.94:0/1291740738 conn(0x558d38762300 msgr2=0x558d38b181b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.984+0000 7f6a10ff9640 0 client.21687 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.985+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 shutdown_connections 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.985+0000 7f6a36669740 1 -- 172.21.15.94:0/1291740738 wait complete. 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.985+0000 7f6a36669740 20 client.21687 trim_cache size 0 max 16384 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:59:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:59:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 msgr2=0x7f6a140b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 0x7f6a140b2070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005cc0 tx=0x7f6a2002b7a0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 msgr2=0x7f6a28197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800d950 tx=0x7f6a1800de10 comp rx=0 tx=0).stop 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 shutdown_connections 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6a140afb60 0x7f6a140b2070 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a28108110 0x7f6a28198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a28103410 0x7f6a28197d50 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 --2- 172.21.15.94:0/1267296217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a281021d0 0x7f6a281996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 >> 172.21.15.94:0/1267296217 conn(0x7f6a280fd3c0 msgr2=0x7f6a281040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 shutdown_connections 2024-04-25T13:59:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:12.986+0000 7f6a2f7b2640 1 -- 172.21.15.94:0/1267296217 wait complete. 2024-04-25T13:59:13.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 85845 19.35 MB/sec execute 115 sec latency 544.555 ms 2024-04-25T13:59:13.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:13 smithi094 ceph-mon[29195]: pgmap v874: 97 pgs: 97 active+clean; 259 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T13:59:13.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:13 smithi138 ceph-mon[32977]: pgmap v874: 97 pgs: 97 active+clean; 259 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T13:59:14.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:13 smithi137 ceph-mon[32568]: pgmap v874: 97 pgs: 97 active+clean; 259 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T13:59:14.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 86219 19.33 MB/sec execute 116 sec latency 94.317 ms 2024-04-25T13:59:15.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 86609 19.39 MB/sec execute 117 sec latency 56.683 ms 2024-04-25T13:59:15.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:15 smithi094 ceph-mon[29195]: pgmap v875: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 641 op/s 2024-04-25T13:59:15.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:15 smithi138 ceph-mon[32977]: pgmap v875: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 641 op/s 2024-04-25T13:59:16.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:15 smithi137 ceph-mon[32568]: pgmap v875: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 641 op/s 2024-04-25T13:59:16.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 86893 19.35 MB/sec execute 118 sec latency 210.603 ms 2024-04-25T13:59:17.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:17.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:17.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 87286 19.33 MB/sec execute 119 sec latency 88.987 ms 2024-04-25T13:59:17.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 -- 172.21.15.94:0/4272135988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 msgr2=0x7ff410102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 --2- 172.21.15.94:0/4272135988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410102e70 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff400009900 tx=0x7ff40002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 -- 172.21.15.94:0/4272135988 shutdown_connections 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 --2- 172.21.15.94:0/4272135988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 0x7ff41010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 --2- 172.21.15.94:0/4272135988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff4101037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 --2- 172.21.15.94:0/4272135988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410102e70 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 -- 172.21.15.94:0/4272135988 >> 172.21.15.94:0/4272135988 conn(0x7ff4100fd380 msgr2=0x7ff4100ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 -- 172.21.15.94:0/4272135988 shutdown_connections 2024-04-25T13:59:17.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.148+0000 7ff41499e640 1 -- 172.21.15.94:0/4272135988 wait complete. 2024-04-25T13:59:17.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 Processor -- start 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 -- start start 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff410198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 0x7ff41019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.149+0000 7ff41499e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff41010d400 con 0x7ff410102170 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff41499e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff41010d280 con 0x7ff4101033b0 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff41499e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff41010d580 con 0x7ff4101080b0 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47654/0 (socket says 172.21.15.94:47654) 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 -- 172.21.15.94:0/1302390299 learned_addr learned my addr 172.21.15.94:0/1302390299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff410198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 0x7ff41019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 -- 172.21.15.94:0/1302390299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 msgr2=0x7ff41019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 0x7ff41019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 -- 172.21.15.94:0/1302390299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 msgr2=0x7ff410198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff410198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 -- 172.21.15.94:0/1302390299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41019fb50 con 0x7ff410102170 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff410198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff40e59c640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff400006010 tx=0x7ff400004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff3ff7fe640 1 -- 172.21.15.94:0/1302390299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40003f030 con 0x7ff410102170 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff3ff7fe640 1 -- 172.21.15.94:0/1302390299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff400031910 con 0x7ff410102170 2024-04-25T13:59:17.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff3ff7fe640 1 -- 172.21.15.94:0/1302390299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400035070 con 0x7ff410102170 2024-04-25T13:59:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41019fde0 con 0x7ff410102170 2024-04-25T13:59:17.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.150+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101a0240 con 0x7ff410102170 2024-04-25T13:59:17.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.153+0000 7ff41b84b740 1 Processor -- start 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.153+0000 7ff41b84b740 1 -- start start 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9c64d0 0x557c6cd90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ce6b0 0x557c6cd91c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557c6ccf5d80 con 0x557c6c9c64d0 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557c6ccaf8b0 con 0x557c6c9ef390 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff41b84b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c6ccf1d40 con 0x557c6c9ce6b0 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff3ff7fe640 1 -- 172.21.15.94:0/1302390299 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff400042030 con 0x7ff410102170 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60386/0 (socket says 172.21.15.94:60386) 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 -- 172.21.15.94:0/831247950 learned_addr learned my addr 172.21.15.94:0/831247950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40e59c640 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9c64d0 0x557c6cd90fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ce6b0 0x557c6cd91c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 -- 172.21.15.94:0/831247950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ce6b0 msgr2=0x557c6cd91c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ce6b0 0x557c6cd91c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 -- 172.21.15.94:0/831247950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9c64d0 msgr2=0x557c6cd90fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9c64d0 0x557c6cd90fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 -- 172.21.15.94:0/831247950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6cc80a40 con 0x557c6c9ef390 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e530 tx=0x7ff40800a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a0f5a2ca3b28f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.154+0000 7ff3ff7fe640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 0x7ff3f80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.155+0000 7ff3fcff9640 1 -- 172.21.15.94:0/831247950 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40801e070 con 0x557c6c9ef390 2024-04-25T13:59:17.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.155+0000 7ff3fcff9640 1 -- 172.21.15.94:0/831247950 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff408003e40 con 0x557c6c9ef390 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.155+0000 7ff3fcff9640 1 -- 172.21.15.94:0/831247950 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40800dc00 con 0x557c6c9ef390 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.155+0000 7ff40e59c640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 0x7ff3f80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.155+0000 7ff3ff7fe640 1 -- 172.21.15.94:0/1302390299 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff4000fef30 con 0x7ff410102170 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- 172.21.15.94:0/831247950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 msgr2=0x557c6cd90a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff40e59c640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 0x7ff3f80b2080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x557c6cd90780 tx=0x7ff400102a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e530 tx=0x7ff40800a610 comp rx=0 tx=0).stop 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- 172.21.15.94:0/831247950 shutdown_connections 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ce6b0 0x557c6cd91c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9c64d0 0x557c6cd90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- 172.21.15.94:0/831247950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd90a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- 172.21.15.94:0/831247950 >> 172.21.15.94:0/831247950 conn(0x557c6c9e34e0 msgr2=0x557c6cd8c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- 172.21.15.94:0/831247950 shutdown_connections 2024-04-25T13:59:17.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- 172.21.15.94:0/831247950 wait complete. 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- start start 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 Processor -- start 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ef390 0x557c6cda4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557c6cd92780 con 0x557c6c9ef390 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557c6cc7cd10 con 0x557c6c9ce6b0 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c6cd92900 con 0x557c6c9c64d0 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.156+0000 7ff41b84b740 10 client.0 _ll_register_callbacks cb 0x557c6cca9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34608/0 (socket says 172.21.15.94:34608) 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60396/0 (socket says 172.21.15.94:60396) 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 -- 172.21.15.94:0/578398045 learned_addr learned my addr 172.21.15.94:0/578398045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ef390 0x557c6cda4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 -- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 msgr2=0x557c6cd9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 -- 172.21.15.94:0/578398045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ef390 msgr2=0x557c6cda4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ef390 0x557c6cda4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 -- 172.21.15.94:0/578398045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6cad17d0 con 0x557c6c9c64d0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff40800aaf0 tx=0x7ff40800acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f7d25cec4868ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40801e040 con 0x557c6c9c64d0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff408004050 con 0x557c6c9c64d0 2024-04-25T13:59:17.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c6cd03990 con 0x557c6c9c64d0 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff408004e50 con 0x557c6c9c64d0 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff41b84b740 10 client.30811 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.157+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c6ca073f0 con 0x557c6c9c64d0 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557c6cd03990 con 0x557c6c9c64d0 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff408024070 con 0x557c6c9c64d0 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 10 client.30811 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 10 client.30811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 20 client.30811 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9adf60 0x557c6cdaa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 4 client.30811 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:17.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6ccfd800 con 0x557c6c9adf60 2024-04-25T13:59:17.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9adf60 0x557c6cdaa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9adf60 0x557c6cdaa310 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef71033f0b57f4f4 server_cookie=90cbdd67ad3c567c in_seq=0 out_seq=0 2024-04-25T13:59:17.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.158+0000 7ff3f5ffb640 10 client.30811 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:17.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/578398045 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c6ccfd800 con 0x557c6c9adf60 2024-04-25T13:59:17.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff3f5ffb640 10 client.30811 handle_command_reply: tid=0 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 client.30811 shutdown 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 msgr2=0x557c6cd9cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff40800aaf0 tx=0x7ff40800acf0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 shutdown_connections 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9adf60 0x557c6cdaa310 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ef390 0x557c6cda4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff3f5ffb640 0 client.30811 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 --2- 172.21.15.94:0/578398045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd9cbd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 >> 172.21.15.94:0/578398045 conn(0x557c6c9e34e0 msgr2=0x557c6cd8b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 shutdown_connections 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 1 -- 172.21.15.94:0/578398045 wait complete. 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.162+0000 7ff41b84b740 20 client.30811 trim_cache size 0 max 16384 2024-04-25T13:59:17.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 Processor -- start 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 -- start start 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101a92b0 con 0x557c6c9ce6b0 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101aadf0 con 0x557c6c9ef390 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.183+0000 7ff41b84b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101a9980 con 0x557c6c9c64d0 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47684/0 (socket says 172.21.15.94:47684) 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 -- 172.21.15.94:0/748944432 learned_addr learned my addr 172.21.15.94:0/748944432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 -- 172.21.15.94:0/748944432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 msgr2=0x557c6cd95950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd95950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 -- 172.21.15.94:0/748944432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 msgr2=0x557c6cd9ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd9ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 -- 172.21.15.94:0/748944432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6c9c70f0 con 0x557c6c9ce6b0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x557c6cd9dc40 tx=0x7ff404005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31b6dc38f1c2e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff3f7fff640 1 -- 172.21.15.94:0/748944432 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404005220 con 0x557c6c9ce6b0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff3f7fff640 1 -- 172.21.15.94:0/748944432 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff404025030 con 0x557c6c9ce6b0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.184+0000 7ff3f7fff640 1 -- 172.21.15.94:0/748944432 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404021460 con 0x557c6c9ce6b0 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- 172.21.15.94:0/748944432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 msgr2=0x557c6cd9b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x557c6cd9dc40 tx=0x7ff404005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- 172.21.15.94:0/748944432 shutdown_connections 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9ef390 0x557c6cd9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9b780 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 --2- 172.21.15.94:0/748944432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9c64d0 0x557c6cd95950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- 172.21.15.94:0/748944432 >> 172.21.15.94:0/748944432 conn(0x557c6c9e34e0 msgr2=0x557c6cd9c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- 172.21.15.94:0/748944432 shutdown_connections 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- 172.21.15.94:0/748944432 wait complete. 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.185+0000 7ff41b84b740 1 -- start start 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 Processor -- start 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9c64d0 0x557c6cd9f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 0x557c6ce105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101aa720 con 0x557c6c9ce6b0 2024-04-25T13:59:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101aa050 con 0x557c6c9c64d0 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4101a7290 con 0x557c6c9ef390 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff41b84b740 10 client.0 _ll_register_callbacks cb 0x7ff40801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47694/0 (socket says 172.21.15.94:47694) 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 -- 172.21.15.94:0/2869737190 learned_addr learned my addr 172.21.15.94:0/2869737190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9c64d0 0x557c6cd9f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 0x557c6ce105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 -- 172.21.15.94:0/2869737190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 msgr2=0x557c6ce105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 0x557c6ce105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 -- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9c64d0 msgr2=0x557c6cd9f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9c64d0 0x557c6cd9f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 -- 172.21.15.94:0/2869737190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6ccb64b0 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 0x557c6ce105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff40dd9b640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff404012280 tx=0x7ff404005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d6cfc4906a58e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.186+0000 7ff3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40402e070 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff404025030 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff404004050 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff40403a050 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 10 client.21705 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c6ce118a0 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff404025030 con 0x557c6c9ce6b0 2024-04-25T13:59:17.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff4040035f0 con 0x557c6c9ce6b0 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 10 client.21705 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 10 client.21705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 20 client.21705 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9833a0 0x557c6ce14240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 4 client.21705 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c6cc784a0 con 0x557c6c9833a0 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.187+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9833a0 0x557c6ce14240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.188+0000 7ff3f5ffb640 10 client.21705 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.188+0000 7ff40ed9d640 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9833a0 0x557c6ce14240 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c6463f3f42c157d server_cookie=8a7840bd8fe608db in_seq=0 out_seq=0 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff3f5ffb640 1 -- 172.21.15.94:0/2869737190 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c6cc784a0 con 0x557c6c9833a0 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff3f5ffb640 10 client.21705 handle_command_reply: tid=0 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 client.21705 shutdown 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 msgr2=0x557c6cd9fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff404012280 tx=0x7ff404005020 comp rx=0 tx=0).stop 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 shutdown_connections 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557c6c9833a0 0x557c6ce14240 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c6c9ef390 0x557c6ce105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c6c9ce6b0 0x557c6cd9fae0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 --2- 172.21.15.94:0/2869737190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c6c9c64d0 0x557c6cd9f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff3f5ffb640 0 client.21705 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 >> 172.21.15.94:0/2869737190 conn(0x557c6c9e34e0 msgr2=0x557c6cd9cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 shutdown_connections 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.191+0000 7ff41b84b740 1 -- 172.21.15.94:0/2869737190 wait complete. 2024-04-25T13:59:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41b84b740 20 client.21705 trim_cache size 0 max 16384 2024-04-25T13:59:17.200 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:59:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:59:17.200 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 msgr2=0x7ff3f80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 0x7ff3f80b2080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x557c6cd90780 tx=0x7ff400102a10 comp rx=0 tx=0).stop 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 msgr2=0x7ff410198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff400006010 tx=0x7ff400004270 comp rx=0 tx=0).stop 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 shutdown_connections 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff3f80afb70 0x7ff3f80b2080 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4101080b0 0x7ff41019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4101033b0 0x7ff410198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 --2- 172.21.15.94:0/1302390299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff410102170 0x7ff410198010 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 >> 172.21.15.94:0/1302390299 conn(0x7ff4100fd380 msgr2=0x7ff410104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 shutdown_connections 2024-04-25T13:59:17.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.192+0000 7ff41499e640 1 -- 172.21.15.94:0/1302390299 wait complete. 2024-04-25T13:59:17.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:59:17.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- 172.21.15.94:0/3392301291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c108900 msgr2=0x7f654c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 --2- 172.21.15.94:0/3392301291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c108900 0x7f654c10ad20 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f6544009ca0 tx=0x7f654402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- 172.21.15.94:0/3392301291 shutdown_connections 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 --2- 172.21.15.94:0/3392301291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c108900 0x7f654c10ad20 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 --2- 172.21.15.94:0/3392301291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c103c00 0x7f654c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 --2- 172.21.15.94:0/3392301291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 0x7f654c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- 172.21.15.94:0/3392301291 >> 172.21.15.94:0/3392301291 conn(0x7f654c0fdbd0 msgr2=0x7f654c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- 172.21.15.94:0/3392301291 shutdown_connections 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- 172.21.15.94:0/3392301291 wait complete. 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 Processor -- start 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.320+0000 7f6553908640 1 -- start start 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 0x7f654c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c108900 0x7f654c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10e4f0 con 0x7f654c103c00 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10e370 con 0x7f654c1029c0 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6553908640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10e670 con 0x7f654c108900 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47700/0 (socket says 172.21.15.94:47700) 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 -- 172.21.15.94:0/1459096850 learned_addr learned my addr 172.21.15.94:0/1459096850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f65516a4640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 0x7f654c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6551ea5640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c108900 0x7f654c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 -- 172.21.15.94:0/1459096850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c108900 msgr2=0x7f654c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c108900 0x7f654c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 -- 172.21.15.94:0/1459096850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 msgr2=0x7f654c06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 0x7f654c06a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 -- 172.21.15.94:0/1459096850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c1a1cf0 con 0x7f654c103c00 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f6550ea3640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e9f0 tx=0x7f653c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.321+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c00cd80 con 0x7f654c103c00 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.322+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c1a1fe0 con 0x7f654c103c00 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.322+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f653c0054d0 con 0x7f654c103c00 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.322+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c010690 con 0x7f654c103c00 2024-04-25T13:59:17.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.322+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c1a2440 con 0x7f654c103c00 2024-04-25T13:59:17.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.322+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654c107710 con 0x7f654c103c00 2024-04-25T13:59:17.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.324+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f653c004e50 con 0x7f654c103c00 2024-04-25T13:59:17.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.325+0000 7f65427fc640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 0x7f65380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.325+0000 7f65516a4640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 0x7f65380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.325+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f653c0df2e0 con 0x7f654c103c00 2024-04-25T13:59:17.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.325+0000 7f65516a4640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 0x7f65380b2000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f654c1035c0 tx=0x7f6548005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.330+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f653c053ff0 con 0x7f654c103c00 2024-04-25T13:59:17.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.571+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f654c02db50 con 0x7f654c103c00 2024-04-25T13:59:17.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.571+0000 7f65427fc640 1 -- 172.21.15.94:0/1459096850 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f653c0a7e20 con 0x7f654c103c00 2024-04-25T13:59:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:59:17.571 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:59:17.572 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 msgr2=0x7f65380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 0x7f65380b2000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f654c1035c0 tx=0x7f6548005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 msgr2=0x7f654c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e9f0 tx=0x7f653c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 shutdown_connections 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65380afaf0 0x7f65380b2000 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654c108900 0x7f654c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f654c103c00 0x7f654c19ef60 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 --2- 172.21.15.94:0/1459096850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f654c1029c0 0x7f654c06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 >> 172.21.15.94:0/1459096850 conn(0x7f654c0fdbd0 msgr2=0x7f654c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 shutdown_connections 2024-04-25T13:59:17.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.574+0000 7f6553908640 1 -- 172.21.15.94:0/1459096850 wait complete. 2024-04-25T13:59:17.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:59:17.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/4075626454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 msgr2=0x7fd0b81047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/4075626454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b81047e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009900 tx=0x7fd0a802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/4075626454 shutdown_connections 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/4075626454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 0x7fd0b810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/4075626454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/4075626454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b81047e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/4075626454 >> 172.21.15.94:0/4075626454 conn(0x7fd0b80fecd0 msgr2=0x7fd0b8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/4075626454 shutdown_connections 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.703+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/4075626454 wait complete. 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 Processor -- start 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 -- start start 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 0x7fd0b8075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b810f5e0 con 0x7fd0b8103ae0 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b810f460 con 0x7fd0b8104d20 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0bd6b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b810f760 con 0x7fd0b8109a20 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47724/0 (socket says 172.21.15.94:47724) 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0b6ffd640 1 -- 172.21.15.94:0/1309132297 learned_addr learned my addr 172.21.15.94:0/1309132297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.704+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 0x7fd0b8075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 -- 172.21.15.94:0/1309132297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 msgr2=0x7fd0b8075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 0x7fd0b8075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 -- 172.21.15.94:0/1309132297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 msgr2=0x7fd0b8074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 -- 172.21.15.94:0/1309132297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b8075b60 con 0x7fd0b8103ae0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8006010 tx=0x7fd0a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd09ffff640 1 -- 172.21.15.94:0/1309132297 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a803f030 con 0x7fd0b8103ae0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b81a42d0 con 0x7fd0b8103ae0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b81a4730 con 0x7fd0b8103ae0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd09ffff640 1 -- 172.21.15.94:0/1309132297 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a8031910 con 0x7fd0b8103ae0 2024-04-25T13:59:17.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.705+0000 7fd09ffff640 1 -- 172.21.15.94:0/1309132297 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8035070 con 0x7fd0b8103ae0 2024-04-25T13:59:17.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.708+0000 7fd0c456a740 1 Processor -- start 2024-04-25T13:59:17.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.708+0000 7fd0c456a740 1 -- start start 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.708+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 0x55f330c3a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0c456a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f330b9ed80 con 0x55f3308776b0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0c456a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f330b58a20 con 0x55f330898390 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0c456a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f330b9ad40 con 0x55f33086f4d0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd09ffff640 1 -- 172.21.15.94:0/1309132297 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd0a8042030 con 0x7fd0b8103ae0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 0x55f330c3a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47726/0 (socket says 172.21.15.94:47726) 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b67fc640 1 -- 172.21.15.94:0/4205649968 learned_addr learned my addr 172.21.15.94:0/4205649968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 -- 172.21.15.94:0/4205649968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 msgr2=0x55f330c3ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c3ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 -- 172.21.15.94:0/4205649968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 msgr2=0x55f330c3ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 -- 172.21.15.94:0/4205649968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f330b63c30 con 0x55f330898390 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 0x55f330c3a810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009f00 tx=0x7fd0b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea5023bcbd9c79e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.709+0000 7fd09ffff640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 0x7fd0a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 0x7fd0a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd09d7fa640 1 -- 172.21.15.94:0/4205649968 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004d20 con 0x55f330898390 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd09d7fa640 1 -- 172.21.15.94:0/4205649968 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b0004030 con 0x55f330898390 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd09d7fa640 1 -- 172.21.15.94:0/4205649968 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0005240 con 0x55f330898390 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd09ffff640 1 -- 172.21.15.94:0/1309132297 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd0a80fef80 con 0x7fd0b8103ae0 2024-04-25T13:59:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd0c456a740 1 -- 172.21.15.94:0/4205649968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 msgr2=0x55f330c3a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd0b6ffd640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 0x7fd0a00b2000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55f330c3a530 tx=0x7fd0a80b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.710+0000 7fd0c456a740 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 0x55f330c3a810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009f00 tx=0x7fd0b0011810 comp rx=0 tx=0).stop 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- 172.21.15.94:0/4205649968 shutdown_connections 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f3308776b0 0x55f330c3ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- 172.21.15.94:0/4205649968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f330898390 0x55f330c3a810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- 172.21.15.94:0/4205649968 >> 172.21.15.94:0/4205649968 conn(0x55f33088c4e0 msgr2=0x55f330c36020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- 172.21.15.94:0/4205649968 shutdown_connections 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- 172.21.15.94:0/4205649968 wait complete. 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- start start 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 Processor -- start 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c44530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c44a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f330c3c530 con 0x55f330898390 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f330b25dc0 con 0x55f3308776b0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f330c3c6b0 con 0x55f33086f4d0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0c456a740 10 client.0 _ll_register_callbacks cb 0x55f330b41600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0b67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c44a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47732/0 (socket says 172.21.15.94:47732) 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.711+0000 7fd0b77fe640 1 -- 172.21.15.94:0/92142769 learned_addr learned my addr 172.21.15.94:0/92142769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c44530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 -- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 msgr2=0x55f330c44530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c44530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 -- 172.21.15.94:0/92142769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 msgr2=0x55f330c44a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c44a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 -- 172.21.15.94:0/92142769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3308b03f0 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c44a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00056e0 tx=0x7fd0b0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411b360b3e86898f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd09aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b001e040 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd0b0004c40 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b0004030 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd0b0004e40 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0c456a740 10 client.21723 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.712+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f33097a7d0 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b0004030 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd0b0030070 con 0x55f330898390 2024-04-25T13:59:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 10 client.21723 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 10 client.21723 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 20 client.21723 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330856f60 0x55f330c52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 4 client.21723 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f330bac990 con 0x55f330856f60 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330856f60 0x55f330c52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd09aff5640 10 client.21723 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.713+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330856f60 0x55f330c52050 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5b3ee9fb921f3e3 server_cookie=34875118ed9df568 in_seq=0 out_seq=0 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd09aff5640 1 -- 172.21.15.94:0/92142769 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f330bac990 con 0x55f330856f60 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd09aff5640 10 client.21723 handle_command_reply: tid=0 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 client.21723 shutdown 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 msgr2=0x55f330c4b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00056e0 tx=0x7fd0b0009be0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 shutdown_connections 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330856f60 0x55f330c52050 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd09aff5640 0 client.21723 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330898390 0x55f330c4b9d0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c44a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 --2- 172.21.15.94:0/92142769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c44530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 >> 172.21.15.94:0/92142769 conn(0x55f33088c4e0 msgr2=0x55f330c38f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 shutdown_connections 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.714+0000 7fd0c456a740 1 -- 172.21.15.94:0/92142769 wait complete. 2024-04-25T13:59:17.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.715+0000 7fd0c456a740 20 client.21723 trim_cache size 0 max 16384 2024-04-25T13:59:17.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 Processor -- start 2024-04-25T13:59:17.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 -- start start 2024-04-25T13:59:17.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81ad720 con 0x55f330856f60 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81af260 con 0x55f3308776b0 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0c456a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81addf0 con 0x55f33086f4d0 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47744/0 (socket says 172.21.15.94:47744) 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 -- 172.21.15.94:0/2891648959 learned_addr learned my addr 172.21.15.94:0/2891648959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 -- 172.21.15.94:0/2891648959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 msgr2=0x55f330c4b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 -- 172.21.15.94:0/2891648959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 msgr2=0x55f330c56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c56d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.729+0000 7fd0b77fe640 1 -- 172.21.15.94:0/2891648959 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f330c4be30 con 0x55f330856f60 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.730+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0005d70 tx=0x7fd0b0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa362f316dd2b498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.730+0000 7fd09cff9640 1 -- 172.21.15.94:0/2891648959 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0036040 con 0x55f330856f60 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.730+0000 7fd09cff9640 1 -- 172.21.15.94:0/2891648959 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b0011d10 con 0x55f330856f60 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.730+0000 7fd09cff9640 1 -- 172.21.15.94:0/2891648959 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0005240 con 0x55f330856f60 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- 172.21.15.94:0/2891648959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 msgr2=0x55f330c56840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0005d70 tx=0x7fd0b0009b30 comp rx=0 tx=0).stop 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- 172.21.15.94:0/2891648959 shutdown_connections 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330c56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- 172.21.15.94:0/2891648959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c56840 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- 172.21.15.94:0/2891648959 >> 172.21.15.94:0/2891648959 conn(0x55f33088c4e0 msgr2=0x55f330c3f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- 172.21.15.94:0/2891648959 shutdown_connections 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- 172.21.15.94:0/2891648959 wait complete. 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- start start 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 Processor -- start 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330cb85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81aeb90 con 0x55f330856f60 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81ae4c0 con 0x55f3308776b0 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81ab700 con 0x55f33086f4d0 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0c456a740 10 client.0 _ll_register_callbacks cb 0x7fd0ac009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c4ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47752/0 (socket says 172.21.15.94:47752) 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0b77fe640 1 -- 172.21.15.94:0/682268750 learned_addr learned my addr 172.21.15.94:0/682268750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.731+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330cb85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 -- 172.21.15.94:0/682268750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 msgr2=0x55f330cb85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330cb85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 -- 172.21.15.94:0/682268750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 msgr2=0x55f330c4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c4ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 -- 172.21.15.94:0/682268750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f330cb8b60 con 0x55f33086f4d0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0b67fc640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f330c52790 tx=0x7fd0ac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eef780f9baa5d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd09aff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0220f0 con 0x55f33086f4d0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd0ac022290 con 0x55f33086f4d0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b0011d10 con 0x55f33086f4d0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.732+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd0ac032050 con 0x55f33086f4d0 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 10 client.30838 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f330cb9e40 con 0x55f33086f4d0 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b0011d10 con 0x55f33086f4d0 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd0ac0104a0 con 0x55f33086f4d0 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 10 client.30838 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 10 client.30838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 20 client.30838 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330898390 0x55f330cbc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 4 client.30838 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f330ba6800 con 0x55f330898390 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.733+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330898390 0x55f330cbc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd09aff5640 10 client.30838 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:17.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0b77fe640 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330898390 0x55f330cbc830 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb4176e820c92c92 server_cookie=5278143fc913f5b1 in_seq=0 out_seq=0 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd09aff5640 1 -- 172.21.15.94:0/682268750 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f330ba6800 con 0x55f330898390 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd09aff5640 10 client.30838 handle_command_reply: tid=0 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 client.30838 shutdown 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 msgr2=0x55f330c4b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f330c52790 tx=0x7fd0ac00f790 comp rx=0 tx=0).stop 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 shutdown_connections 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f330898390 0x55f330cbc830 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3308776b0 0x55f330cb85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f33086f4d0 0x55f330c4b090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd09aff5640 0 client.30838 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:17.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 --2- 172.21.15.94:0/682268750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f330856f60 0x55f330c4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 >> 172.21.15.94:0/682268750 conn(0x55f33088c4e0 msgr2=0x55f330c38ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 shutdown_connections 2024-04-25T13:59:17.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 1 -- 172.21.15.94:0/682268750 wait complete. 2024-04-25T13:59:17.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.734+0000 7fd0c456a740 20 client.30838 trim_cache size 0 max 16384 2024-04-25T13:59:17.741 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 msgr2=0x7fd0a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 0x7fd0a00b2000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55f330c3a530 tx=0x7fd0a80b0040 comp rx=0 tx=0).stop 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 msgr2=0x7fd0b8076850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8006010 tx=0x7fd0a8004270 comp rx=0 tx=0).stop 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 shutdown_connections 2024-04-25T13:59:17.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0a00afaf0 0x7fd0a00b2000 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0b8109a20 0x7fd0b8075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0b8104d20 0x7fd0b8074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 --2- 172.21.15.94:0/1309132297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b8103ae0 0x7fd0b8076850 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 >> 172.21.15.94:0/1309132297 conn(0x7fd0b80fecd0 msgr2=0x7fd0b8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 shutdown_connections 2024-04-25T13:59:17.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.735+0000 7fd0bd6b6640 1 -- 172.21.15.94:0/1309132297 wait complete. 2024-04-25T13:59:17.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:17 smithi094 ceph-mon[29195]: pgmap v876: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-04-25T13:59:17.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1459096850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:17.748 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:59:17.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 -- 172.21.15.94:0/834946112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648102fc0 msgr2=0x7f86481033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 --2- 172.21.15.94:0/834946112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648102fc0 0x7f86481033d0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f8638009900 tx=0x7f863802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:17.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 -- 172.21.15.94:0/834946112 shutdown_connections 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 --2- 172.21.15.94:0/834946112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648104680 0x7f864810add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 --2- 172.21.15.94:0/834946112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8648103b20 0x7f8648103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 --2- 172.21.15.94:0/834946112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648102fc0 0x7f86481033d0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 -- 172.21.15.94:0/834946112 >> 172.21.15.94:0/834946112 conn(0x7f86480fd340 msgr2=0x7f86480ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 -- 172.21.15.94:0/834946112 shutdown_connections 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.866+0000 7f864e7b7640 1 -- 172.21.15.94:0/834946112 wait complete. 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 Processor -- start 2024-04-25T13:59:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 -- start start 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648102fc0 0x7f864810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f864810c0d0 0x7f864810c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f850 con 0x7f8648103b20 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f6d0 con 0x7f864810c0d0 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864e7b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f864810f9d0 con 0x7f8648102fc0 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47772/0 (socket says 172.21.15.94:47772) 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 -- 172.21.15.94:0/886346306 learned_addr learned my addr 172.21.15.94:0/886346306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f864cd54640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f864810c0d0 0x7f864810c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f8647fff640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648102fc0 0x7f864810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 -- 172.21.15.94:0/886346306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648102fc0 msgr2=0x7f864810d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648102fc0 0x7f864810d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 -- 172.21.15.94:0/886346306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f864810c0d0 msgr2=0x7f864810c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f864810c0d0 0x7f864810c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.867+0000 7f86477fe640 1 -- 172.21.15.94:0/886346306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481a4140 con 0x7f8648103b20 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f86477fe640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f863c00ea20 tx=0x7f863c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f86457fa640 1 -- 172.21.15.94:0/886346306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c00cd80 con 0x7f8648103b20 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f86457fa640 1 -- 172.21.15.94:0/886346306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c0108e0 con 0x7f8648103b20 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481a4430 con 0x7f8648103b20 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f86457fa640 1 -- 172.21.15.94:0/886346306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c018840 con 0x7f8648103b20 2024-04-25T13:59:17.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.868+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a4980 con 0x7f8648103b20 2024-04-25T13:59:17.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.870+0000 7f865566c740 1 Processor -- start 2024-04-25T13:59:17.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.870+0000 7f865566c740 1 -- start start 2024-04-25T13:59:17.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.870+0000 7f86457fa640 1 -- 172.21.15.94:0/886346306 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f863c0189e0 con 0x7f8648103b20 2024-04-25T13:59:17.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d223810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d223d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d224a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d4d187d80 con 0x555d4ce606b0 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d4d141a20 con 0x555d4ce81390 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f865566c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d4d183d40 con 0x555d4ce584d0 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f86477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d224a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f864cd54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d223810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d223d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f864cd54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d223810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60496/0 (socket says 172.21.15.94:60496) 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f864cd54640 1 -- 172.21.15.94:0/343217659 learned_addr learned my addr 172.21.15.94:0/343217659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 -- 172.21.15.94:0/343217659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 msgr2=0x555d4d223810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d223810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 -- 172.21.15.94:0/343217659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 msgr2=0x555d4d224a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d224a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 -- 172.21.15.94:0/343217659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4d14cc30 con 0x555d4ce584d0 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f86457fa640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 0x7f86300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f864cd54640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 0x7f86300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f8647fff640 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d223d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8638006010 tx=0x7f8638032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4295cc9de740d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.871+0000 7f86457fa640 1 -- 172.21.15.94:0/886346306 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f863c0df050 con 0x7f8648103b20 2024-04-25T13:59:17.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f862e7fc640 1 -- 172.21.15.94:0/343217659 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863803f070 con 0x555d4ce584d0 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f862e7fc640 1 -- 172.21.15.94:0/343217659 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8638004040 con 0x555d4ce584d0 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f862e7fc640 1 -- 172.21.15.94:0/343217659 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638035070 con 0x555d4ce584d0 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 -- 172.21.15.94:0/343217659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 msgr2=0x555d4d223d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d223d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8638006010 tx=0x7f8638032b20 comp rx=0 tx=0).stop 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f864cd54640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 0x7f86300b2000 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x555d4d224860 tx=0x7f8640009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 -- 172.21.15.94:0/343217659 shutdown_connections 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d224a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d223d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.872+0000 7f865566c740 1 --2- 172.21.15.94:0/343217659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d223810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- 172.21.15.94:0/343217659 >> 172.21.15.94:0/343217659 conn(0x555d4ce754e0 msgr2=0x555d4d21f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- 172.21.15.94:0/343217659 shutdown_connections 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- 172.21.15.94:0/343217659 wait complete. 2024-04-25T13:59:17.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- start start 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 Processor -- start 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d22db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d234a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d4d2254d0 con 0x555d4ce606b0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d4d10edc0 con 0x555d4ce81390 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.873+0000 7f865566c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d4d225650 con 0x555d4ce584d0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f865566c740 10 client.0 _ll_register_callbacks cb 0x555d4d12a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f86477fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d22db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34754/0 (socket says 172.21.15.94:34754) 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 -- 172.21.15.94:0/1197781807 learned_addr learned my addr 172.21.15.94:0/1197781807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f864cd54640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d234a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 -- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 msgr2=0x555d4d234a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d234a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 -- 172.21.15.94:0/1197781807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 msgr2=0x555d4d22db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d22db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 -- 172.21.15.94:0/1197781807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4ce993f0 con 0x555d4ce584d0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f86477fe640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d22db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f8647fff640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f863802a930 tx=0x7f8638005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2808831f6d8fb1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f86277fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.874+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863803f070 con 0x555d4ce584d0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f86380315a0 con 0x555d4ce584d0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8638004040 con 0x555d4ce584d0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8638035040 con 0x555d4ce584d0 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 10 client.30850 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d4cf637d0 con 0x555d4ce584d0 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8638004040 con 0x555d4ce584d0 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8638003710 con 0x555d4ce584d0 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 10 client.30850 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 10 client.30850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 20 client.30850 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce3ff60 0x555d4d23aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 4 client.30850 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.875+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4d195990 con 0x555d4ce3ff60 2024-04-25T13:59:17.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.876+0000 7f86477fe640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce3ff60 0x555d4d23aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.876+0000 7f86277fe640 10 client.30850 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.876+0000 7f86477fe640 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce3ff60 0x555d4d23aee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72f93027bfae21be server_cookie=e2fdb11eeefb8949 in_seq=0 out_seq=0 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f86277fe640 1 -- 172.21.15.94:0/1197781807 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d4d195990 con 0x555d4ce3ff60 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f86277fe640 10 client.30850 handle_command_reply: tid=0 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 client.30850 shutdown 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 msgr2=0x555d4d22d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f863802a930 tx=0x7f8638005d50 comp rx=0 tx=0).stop 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 shutdown_connections 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce3ff60 0x555d4d23aee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce81390 0x555d4d234a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d22db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f86277fe640 0 client.30850 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 --2- 172.21.15.94:0/1197781807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d22d5f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 >> 172.21.15.94:0/1197781807 conn(0x555d4ce754e0 msgr2=0x555d4d221f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 shutdown_connections 2024-04-25T13:59:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 1 -- 172.21.15.94:0/1197781807 wait complete. 2024-04-25T13:59:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.877+0000 7f865566c740 20 client.30850 trim_cache size 0 max 16384 2024-04-25T13:59:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.894+0000 7f865566c740 1 Processor -- start 2024-04-25T13:59:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 -- start start 2024-04-25T13:59:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d21eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce606b0 0x555d4d2349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ad760 con 0x555d4ce81390 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86481af2a0 con 0x555d4ce606b0 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f865566c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ade30 con 0x555d4ce584d0 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f864cd54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f864cd54640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47798/0 (socket says 172.21.15.94:47798) 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f864cd54640 1 -- 172.21.15.94:0/1166171497 learned_addr learned my addr 172.21.15.94:0/1166171497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.895+0000 7f86477fe640 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce606b0 0x555d4d2349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 -- 172.21.15.94:0/1166171497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 msgr2=0x555d4d21eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d21eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 -- 172.21.15.94:0/1166171497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce606b0 msgr2=0x555d4d2349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce606b0 0x555d4d2349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 -- 172.21.15.94:0/1166171497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4d234ef0 con 0x555d4ce81390 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f864cd54640 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f8640011420 tx=0x7f8640019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223214a7830c554b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f862dffb640 1 -- 172.21.15.94:0/1166171497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640003970 con 0x555d4ce81390 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f862dffb640 1 -- 172.21.15.94:0/1166171497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8640004030 con 0x555d4ce81390 2024-04-25T13:59:17.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.896+0000 7f862dffb640 1 -- 172.21.15.94:0/1166171497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640010810 con 0x555d4ce81390 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- 172.21.15.94:0/1166171497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 msgr2=0x555d4d22d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f8640011420 tx=0x7f8640019000 comp rx=0 tx=0).stop 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- 172.21.15.94:0/1166171497 shutdown_connections 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce81390 0x555d4d22d1c0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce606b0 0x555d4d2349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 --2- 172.21.15.94:0/1166171497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce584d0 0x555d4d21eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- 172.21.15.94:0/1166171497 >> 172.21.15.94:0/1166171497 conn(0x555d4ce754e0 msgr2=0x555d4d21e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- 172.21.15.94:0/1166171497 shutdown_connections 2024-04-25T13:59:17.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- 172.21.15.94:0/1166171497 wait complete. 2024-04-25T13:59:17.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.898+0000 7f865566c740 1 -- start start 2024-04-25T13:59:17.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 Processor -- start 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce584d0 0x555d4d231820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d231d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86481aebd0 con 0x555d4ce606b0 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ae500 con 0x555d4ce584d0 2024-04-25T13:59:17.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ab740 con 0x555d4ce81390 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f865566c740 10 client.0 _ll_register_callbacks cb 0x7f863c0e2a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f864cd54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f864cd54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34768/0 (socket says 172.21.15.94:34768) 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f864cd54640 1 -- 172.21.15.94:0/2511044054 learned_addr learned my addr 172.21.15.94:0/2511044054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f86477fe640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d231d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.899+0000 7f8647fff640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce584d0 0x555d4d231820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 -- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce584d0 msgr2=0x555d4d231820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce584d0 0x555d4d231820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 -- 172.21.15.94:0/2511044054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 msgr2=0x555d4d231d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d231d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 -- 172.21.15.94:0/2511044054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4d2323d0 con 0x555d4ce81390 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f864cd54640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f864000fe20 tx=0x7f8640004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2769ee429cccbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f86277fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:17.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400045f0 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8640004030 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8640003970 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.900+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8640004e50 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 10 client.30853 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d4d2aaf00 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8640004030 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8640030070 con 0x555d4ce81390 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 10 client.30853 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 10 client.30853 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 20 client.30853 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce153a0 0x555d4d2ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 4 client.30853 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:17.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.901+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d4d18f800 con 0x555d4ce153a0 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.903+0000 7f8647fff640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce153a0 0x555d4d2ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.903+0000 7f8647fff640 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce153a0 0x555d4d2ad820 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=574e1930b378bb51 server_cookie=a84f55943fbc329d in_seq=0 out_seq=0 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.903+0000 7f86277fe640 10 client.30853 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f86277fe640 1 -- 172.21.15.94:0/2511044054 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d4d18f800 con 0x555d4ce153a0 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f86277fe640 10 client.30853 handle_command_reply: tid=0 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 client.30853 shutdown 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 msgr2=0x555d4d2a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f864000fe20 tx=0x7f8640004ba0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 shutdown_connections 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555d4ce153a0 0x555d4d2ad820 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f86277fe640 0 client.30853 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4ce81390 0x555d4d2a98c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d4ce606b0 0x555d4d231d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 --2- 172.21.15.94:0/2511044054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d4ce584d0 0x555d4d231820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 >> 172.21.15.94:0/2511044054 conn(0x555d4ce754e0 msgr2=0x555d4d22c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.904+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 shutdown_connections 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f865566c740 1 -- 172.21.15.94:0/2511044054 wait complete. 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f865566c740 20 client.30853 trim_cache size 0 max 16384 2024-04-25T13:59:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 msgr2=0x7f86300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 0x7f86300b2000 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x555d4d224860 tx=0x7f8640009500 comp rx=0 tx=0).stop 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 msgr2=0x7f864810bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f863c00ea20 tx=0x7f863c00eee0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 shutdown_connections 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f86300afaf0 0x7f86300b2000 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f864810c0d0 0x7f864810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648103b20 0x7f864810bb90 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 --2- 172.21.15.94:0/886346306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8648102fc0 0x7f864810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 >> 172.21.15.94:0/886346306 conn(0x7f86480fd340 msgr2=0x7f86480ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 shutdown_connections 2024-04-25T13:59:17.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:17.905+0000 7f864e7b7640 1 -- 172.21.15.94:0/886346306 wait complete. 2024-04-25T13:59:17.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:59:17.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:17 smithi138 ceph-mon[32977]: pgmap v876: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-04-25T13:59:17.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1459096850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:18.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 -- 172.21.15.94:0/1959608603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684103ae0 msgr2=0x7fb6841047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/1959608603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684103ae0 0x7fb6841047e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fb678009900 tx=0x7fb67802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:18.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 -- 172.21.15.94:0/1959608603 shutdown_connections 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/1959608603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684109a20 0x7fb68410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/1959608603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 0x7fb684105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/1959608603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684103ae0 0x7fb6841047e0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 -- 172.21.15.94:0/1959608603 >> 172.21.15.94:0/1959608603 conn(0x7fb6840fecd0 msgr2=0x7fb684101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 -- 172.21.15.94:0/1959608603 shutdown_connections 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.038+0000 7fb68ae1a640 1 -- 172.21.15.94:0/1959608603 wait complete. 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 Processor -- start 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 -- start start 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684103ae0 0x7fb68419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 0x7fb68419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb68410f5e0 con 0x7fb684109a20 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb68410f460 con 0x7fb684103ae0 2024-04-25T13:59:18.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb68ae1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb68410f760 con 0x7fb684104d20 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb6893b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684103ae0 0x7fb68419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb6893b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47814/0 (socket says 172.21.15.94:47814) 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb6893b7640 1 -- 172.21.15.94:0/2046704527 learned_addr learned my addr 172.21.15.94:0/2046704527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.039+0000 7fb683fff640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 0x7fb68419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 -- 172.21.15.94:0/2046704527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 msgr2=0x7fb68419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 0x7fb68419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 -- 172.21.15.94:0/2046704527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684103ae0 msgr2=0x7fb68419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684103ae0 0x7fb68419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 -- 172.21.15.94:0/2046704527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6841a1d50 con 0x7fb684109a20 2024-04-25T13:59:18.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb6893b7640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00eb10 tx=0x7fb67c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb681ffb640 1 -- 172.21.15.94:0/2046704527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c004b20 con 0x7fb684109a20 2024-04-25T13:59:18.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb681ffb640 1 -- 172.21.15.94:0/2046704527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb67c004cc0 con 0x7fb684109a20 2024-04-25T13:59:18.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6841a1fe0 con 0x7fb684109a20 2024-04-25T13:59:18.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb681ffb640 1 -- 172.21.15.94:0/2046704527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c010670 con 0x7fb684109a20 2024-04-25T13:59:18.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.040+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6841a2450 con 0x7fb684109a20 2024-04-25T13:59:18.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.042+0000 7fb691da2740 1 Processor -- start 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.042+0000 7fb681ffb640 1 -- 172.21.15.94:0/2046704527 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb67c005020 con 0x7fb684109a20 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 -- start start 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3ee7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b246b0 0x560ff3ee8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ff3e4bd80 con 0x560ff3b1c4d0 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ff3e05a20 con 0x560ff3b246b0 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb691da2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ff3e47d40 con 0x560ff3b45390 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47816/0 (socket says 172.21.15.94:47816) 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 -- 172.21.15.94:0/599200128 learned_addr learned my addr 172.21.15.94:0/599200128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb683fff640 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b246b0 0x560ff3ee8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb683fff640 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3ee7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb681ffb640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 0x7fb6740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 -- 172.21.15.94:0/599200128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 msgr2=0x560ff3ee7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3ee7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 -- 172.21.15.94:0/599200128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b246b0 msgr2=0x560ff3ee8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b246b0 0x560ff3ee8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 -- 172.21.15.94:0/599200128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff3e10c30 con 0x560ff3b1c4d0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb6893b7640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 0x7fb6740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb681ffb640 1 -- 172.21.15.94:0/2046704527 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb67c0df250 con 0x7fb684109a20 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.043+0000 7fb688bb6640 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fb678009230 tx=0x7fb678009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99608d4cbe60611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.044+0000 7fb66effd640 1 -- 172.21.15.94:0/599200128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678035030 con 0x560ff3b1c4d0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.044+0000 7fb66effd640 1 -- 172.21.15.94:0/599200128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb678040070 con 0x560ff3b1c4d0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.044+0000 7fb66effd640 1 -- 172.21.15.94:0/599200128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678003830 con 0x560ff3b1c4d0 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- 172.21.15.94:0/599200128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 msgr2=0x560ff3ee7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fb678009230 tx=0x7fb678009620 comp rx=0 tx=0).stop 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb6893b7640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 0x7fb6740b2000 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0dd020 tx=0x7fb67c05e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- 172.21.15.94:0/599200128 shutdown_connections 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b246b0 0x560ff3ee8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b1c4d0 0x560ff3ee7d90 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 --2- 172.21.15.94:0/599200128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3ee7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- 172.21.15.94:0/599200128 >> 172.21.15.94:0/599200128 conn(0x560ff3b394e0 msgr2=0x560ff3ee3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- 172.21.15.94:0/599200128 shutdown_connections 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- 172.21.15.94:0/599200128 wait complete. 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 -- start start 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.045+0000 7fb691da2740 1 Processor -- start 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3ef89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ff3ee94d0 con 0x560ff3b45390 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ff3dd2dc0 con 0x560ff3b1c4d0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ff3ee9650 con 0x560ff3b246b0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb691da2740 10 client.0 _ll_register_callbacks cb 0x560ff3dee600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb683fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3ef89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34790/0 (socket says 172.21.15.94:34790) 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb683fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 -- 172.21.15.94:0/2409922317 learned_addr learned my addr 172.21.15.94:0/2409922317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 -- 172.21.15.94:0/2409922317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 msgr2=0x560ff3ef1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 -- 172.21.15.94:0/2409922317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 msgr2=0x560ff3ef89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3ef89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 -- 172.21.15.94:0/2409922317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff3b5d3f0 con 0x560ff3b246b0 2024-04-25T13:59:18.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb683fff640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3ef89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb67802a930 tx=0x7fb67802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f42cf209983292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb663fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678035070 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.046+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb6780315a0 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb678040070 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb6780317a0 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 10 client.30865 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ff3c277d0 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb678040070 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb678045590 con 0x560ff3b246b0 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 10 client.30865 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 10 client.30865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 20 client.30865 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3b03f60 0x560ff3efee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 4 client.30865 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff3e59990 con 0x560ff3b03f60 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.047+0000 7fb683fff640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3b03f60 0x560ff3efee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.048+0000 7fb663fff640 10 client.30865 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.048+0000 7fb683fff640 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3b03f60 0x560ff3efee00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd8435176f663579 server_cookie=98af4db057702eae in_seq=0 out_seq=0 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.048+0000 7fb663fff640 1 -- 172.21.15.94:0/2409922317 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ff3e59990 con 0x560ff3b03f60 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.048+0000 7fb663fff640 10 client.30865 handle_command_reply: tid=0 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.048+0000 7fb691da2740 1 client.30865 shutdown 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 msgr2=0x560ff3ef1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb67802a930 tx=0x7fb67802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 shutdown_connections 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3b03f60 0x560ff3efee00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3ef89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3ef1a50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb663fff640 0 client.30865 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 --2- 172.21.15.94:0/2409922317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 >> 172.21.15.94:0/2409922317 conn(0x560ff3b394e0 msgr2=0x560ff3ee5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 shutdown_connections 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 1 -- 172.21.15.94:0/2409922317 wait complete. 2024-04-25T13:59:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.049+0000 7fb691da2740 20 client.30865 trim_cache size 0 max 16384 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 Processor -- start 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 -- start start 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ede190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3eec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841ab490 con 0x560ff3b45390 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841acfd0 con 0x560ff3b1c4d0 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb691da2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841abb60 con 0x560ff3b246b0 2024-04-25T13:59:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb683fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb683fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47820/0 (socket says 172.21.15.94:47820) 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb683fff640 1 -- 172.21.15.94:0/3200024357 learned_addr learned my addr 172.21.15.94:0/3200024357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb683fff640 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ede190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.063+0000 7fb688bb6640 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3eec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 -- 172.21.15.94:0/3200024357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 msgr2=0x560ff3eec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3eec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 -- 172.21.15.94:0/3200024357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 msgr2=0x560ff3ede190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ede190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 -- 172.21.15.94:0/3200024357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff3ef20f0 con 0x560ff3b45390 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb683fff640 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fb670005eb0 tx=0x7fb6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c01e80af24d92b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb66e7fc640 1 -- 172.21.15.94:0/3200024357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67002d070 con 0x560ff3b45390 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb66e7fc640 1 -- 172.21.15.94:0/3200024357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6700044d0 con 0x560ff3b45390 2024-04-25T13:59:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.064+0000 7fb66e7fc640 1 -- 172.21.15.94:0/3200024357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670005310 con 0x560ff3b45390 2024-04-25T13:59:18.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- 172.21.15.94:0/3200024357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 msgr2=0x560ff3eec830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fb670005eb0 tx=0x7fb6700046c0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- 172.21.15.94:0/3200024357 shutdown_connections 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b45390 0x560ff3eec830 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b246b0 0x560ff3eec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 --2- 172.21.15.94:0/3200024357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ede190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- 172.21.15.94:0/3200024357 >> 172.21.15.94:0/3200024357 conn(0x560ff3b394e0 msgr2=0x560ff3ee7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- 172.21.15.94:0/3200024357 shutdown_connections 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- 172.21.15.94:0/3200024357 wait complete. 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 -- start start 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.065+0000 7fb691da2740 1 Processor -- start 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3f65000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841ac900 con 0x560ff3b246b0 2024-04-25T13:59:18.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841ac230 con 0x560ff3b1c4d0 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6841a9470 con 0x560ff3b45390 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 10 client.0 _ll_register_callbacks cb 0x7fb670009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47832/0 (socket says 172.21.15.94:47832) 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 -- 172.21.15.94:0/2550217039 learned_addr learned my addr 172.21.15.94:0/2550217039 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb683fff640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb683fff640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3f65000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 -- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 msgr2=0x560ff3f65000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3f65000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 -- 172.21.15.94:0/2550217039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 msgr2=0x560ff3ef5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 -- 172.21.15.94:0/2550217039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff3f65570 con 0x560ff3b246b0 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb688bb6640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fb678031440 tx=0x7fb678045d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536948d5fcb5ec04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb663fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678035040 con 0x560ff3b246b0 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb678004460 con 0x560ff3b246b0 2024-04-25T13:59:18.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.066+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6700044d0 con 0x560ff3b246b0 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb678067050 con 0x560ff3b246b0 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 10 client.21765 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ff3f66790 con 0x560ff3b246b0 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6700044d0 con 0x560ff3b246b0 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb678003cd0 con 0x560ff3b246b0 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 10 client.21765 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 10 client.21765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 20 client.21765 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3ad93a0 0x560ff3f691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 4 client.21765 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ff3e53800 con 0x560ff3ad93a0 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.067+0000 7fb683fff640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3ad93a0 0x560ff3f691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb663fff640 10 client.21765 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb683fff640 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3ad93a0 0x560ff3f691b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=918ec5b82274b009 server_cookie=1a5d90b10016c2dc in_seq=0 out_seq=0 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb663fff640 1 -- 172.21.15.94:0/2550217039 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ff3e53800 con 0x560ff3ad93a0 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb663fff640 10 client.21765 handle_command_reply: tid=0 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 client.21765 shutdown 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 msgr2=0x560ff3ef5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fb678031440 tx=0x7fb678045d80 comp rx=0 tx=0).stop 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 shutdown_connections 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x560ff3ad93a0 0x560ff3f691b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ff3b45390 0x560ff3f65000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb663fff640 0 client.21765 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ff3b246b0 0x560ff3ef5560 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 --2- 172.21.15.94:0/2550217039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ff3b1c4d0 0x560ff3ef5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 >> 172.21.15.94:0/2550217039 conn(0x560ff3b394e0 msgr2=0x560ff3ef1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 shutdown_connections 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 1 -- 172.21.15.94:0/2550217039 wait complete. 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.068+0000 7fb691da2740 20 client.21765 trim_cache size 0 max 16384 2024-04-25T13:59:18.077 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 msgr2=0x7fb6740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 0x7fb6740b2000 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0dd020 tx=0x7fb67c05e040 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 msgr2=0x7fb6841a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00eb10 tx=0x7fb67c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 shutdown_connections 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6740afaf0 0x7fb6740b2000 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb684109a20 0x7fb6841a16e0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb684104d20 0x7fb68419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 --2- 172.21.15.94:0/2046704527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb684103ae0 0x7fb68419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 >> 172.21.15.94:0/2046704527 conn(0x7fb6840fecd0 msgr2=0x7fb684105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 shutdown_connections 2024-04-25T13:59:18.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.069+0000 7fb68ae1a640 1 -- 172.21.15.94:0/2046704527 wait complete. 2024-04-25T13:59:18.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:59:18.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:17 smithi137 ceph-mon[32568]: pgmap v876: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 517 op/s 2024-04-25T13:59:18.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:17 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1459096850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:18.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 87657 19.33 MB/sec execute 120 sec latency 66.528 ms 2024-04-25T13:59:18.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 -- 172.21.15.94:0/3779079184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 msgr2=0x7f25b81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3779079184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b81036e0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009900 tx=0x7f25a802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 -- 172.21.15.94:0/3779079184 shutdown_connections 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3779079184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 0x7f25b810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3779079184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 0x7f25b8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3779079184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b81036e0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 -- 172.21.15.94:0/3779079184 >> 172.21.15.94:0/3779079184 conn(0x7f25b80fdbd0 msgr2=0x7f25b8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 -- 172.21.15.94:0/3779079184 shutdown_connections 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.196+0000 7f25bd90d640 1 -- 172.21.15.94:0/3779079184 wait complete. 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 Processor -- start 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 -- start start 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 0x7f25b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 0x7f25b81a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810e4c0 con 0x7f25b81029e0 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810e340 con 0x7f25b8108920 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25bd90d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810e640 con 0x7f25b8103c20 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47850/0 (socket says 172.21.15.94:47850) 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3016016171 learned_addr learned my addr 172.21.15.94:0/3016016171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25b77fe640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 0x7f25b81a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25affff640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 0x7f25b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.197+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3016016171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 msgr2=0x7f25b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 0x7f25b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3016016171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 msgr2=0x7f25b81a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 0x7f25b81a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3016016171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b81a2020 con 0x7f25b81029e0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25bc90b640 1 -- 172.21.15.94:0/3016016171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a803f030 con 0x7f25b81029e0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25bc90b640 1 -- 172.21.15.94:0/3016016171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a8003aa0 con 0x7f25b81029e0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25bc90b640 1 -- 172.21.15.94:0/3016016171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8035070 con 0x7f25b81029e0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b81a22b0 con 0x7f25b81029e0 2024-04-25T13:59:18.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.198+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a2750 con 0x7f25b81029e0 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25bc90b640 1 -- 172.21.15.94:0/3016016171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f25a8042030 con 0x7f25b81029e0 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 Processor -- start 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 -- start start 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e61a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55de27dc4d80 con 0x55de27abe390 2024-04-25T13:59:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de27d7ea20 con 0x55de27a9d6b0 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25c47c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de27dc0d40 con 0x55de27a954d0 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47854/0 (socket says 172.21.15.94:47854) 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25b77fe640 1 -- 172.21.15.94:0/1111881967 learned_addr learned my addr 172.21.15.94:0/1111881967 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25affff640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e61a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.201+0000 7f25bc90b640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 0x7f25a40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 -- 172.21.15.94:0/1111881967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 msgr2=0x55de27e60d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e60d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 -- 172.21.15.94:0/1111881967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 msgr2=0x55de27e61a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e61a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 -- 172.21.15.94:0/1111881967 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de27d89c30 con 0x55de27abe390 2024-04-25T13:59:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25affff640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25bc90b640 1 -- 172.21.15.94:0/3016016171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f25a80ff190 con 0x7f25b81029e0 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 0x7f25a40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25b77fe640 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f25b0003b10 tx=0x7f25b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfac29a8c118fbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25adffb640 1 -- 172.21.15.94:0/1111881967 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0024070 con 0x55de27abe390 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25adffb640 1 -- 172.21.15.94:0/1111881967 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b0010cb0 con 0x55de27abe390 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.202+0000 7f25adffb640 1 -- 172.21.15.94:0/1111881967 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b001ed00 con 0x55de27abe390 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 -- 172.21.15.94:0/1111881967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 msgr2=0x55de27e60810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25b77fe640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 0x7f25a40b2080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f25b0021000 tx=0x7f25b001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f25b0003b10 tx=0x7f25b0009f90 comp rx=0 tx=0).stop 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 -- 172.21.15.94:0/1111881967 shutdown_connections 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 --2- 172.21.15.94:0/1111881967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27abe390 0x55de27e60810 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 -- 172.21.15.94:0/1111881967 >> 172.21.15.94:0/1111881967 conn(0x55de27ab24e0 msgr2=0x55de27e5c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 -- 172.21.15.94:0/1111881967 shutdown_connections 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.203+0000 7f25c47c3740 1 -- 172.21.15.94:0/1111881967 wait complete. 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 -- start start 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 Processor -- start 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27abe390 0x55de27e71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55de27e62530 con 0x55de27a954d0 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55de27d4bdc0 con 0x55de27a9d6b0 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de27e626b0 con 0x55de27abe390 2024-04-25T13:59:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25c47c3740 10 client.0 _ll_register_callbacks cb 0x55de27d67600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47856/0 (socket says 172.21.15.94:47856) 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25b6ffd640 1 -- 172.21.15.94:0/2876356335 learned_addr learned my addr 172.21.15.94:0/2876356335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25affff640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.204+0000 7f25b77fe640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27abe390 0x55de27e71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 -- 172.21.15.94:0/2876356335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27abe390 msgr2=0x55de27e71a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27abe390 0x55de27e71a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 -- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 msgr2=0x55de27e6aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 -- 172.21.15.94:0/2876356335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de27ad63f0 con 0x55de27a954d0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f25a8031440 tx=0x7f25a8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f56c48a75dbde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8049070 con 0x55de27a954d0 2024-04-25T13:59:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f25a8003c50 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b0010cb0 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f25a80c6d40 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25c47c3740 10 client.21777 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55de27ba07d0 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b0010cb0 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.205+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f25a80c3860 con 0x55de27a954d0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 10 client.21777 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 10 client.21777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 20 client.21777 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27a7cf60 0x55de27e78220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 4 client.21777 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de27dd2990 con 0x55de27a7cf60 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25affff640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27a7cf60 0x55de27e78220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f25affff640 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27a7cf60 0x55de27e78220 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b03e9fd87de8c00e server_cookie=c29f7db2df700538 in_seq=0 out_seq=0 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.206+0000 7f2596ffd640 10 client.21777 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f2596ffd640 1 -- 172.21.15.94:0/2876356335 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de27dd2990 con 0x55de27a7cf60 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f2596ffd640 10 client.21777 handle_command_reply: tid=0 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 client.21777 shutdown 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 msgr2=0x55de27e6a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f25a8031440 tx=0x7f25a8003f90 comp rx=0 tx=0).stop 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 shutdown_connections 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27a7cf60 0x55de27e78220 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27abe390 0x55de27e71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f2596ffd640 0 client.21777 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 --2- 172.21.15.94:0/2876356335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a954d0 0x55de27e6a560 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 >> 172.21.15.94:0/2876356335 conn(0x55de27ab24e0 msgr2=0x55de27e5b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 shutdown_connections 2024-04-25T13:59:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.207+0000 7f25c47c3740 1 -- 172.21.15.94:0/2876356335 wait complete. 2024-04-25T13:59:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.208+0000 7f25c47c3740 20 client.21777 trim_cache size 0 max 16384 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 Processor -- start 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 -- start start 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e6a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81ab510 con 0x55de27a7cf60 2024-04-25T13:59:18.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81ad050 con 0x55de27a9d6b0 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25c47c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81abbe0 con 0x55de27a954d0 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47870/0 (socket says 172.21.15.94:47870) 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3697468621 learned_addr learned my addr 172.21.15.94:0/3697468621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.222+0000 7f25b77fe640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25affff640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e6a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3697468621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 msgr2=0x55de27e6a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e6a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3697468621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 msgr2=0x55de27e6a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 -- 172.21.15.94:0/3697468621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de27e5cbb0 con 0x55de27a7cf60 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b77fe640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f25a80ff160 tx=0x7f25a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43fae3bb094de05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25ad7fa640 1 -- 172.21.15.94:0/3697468621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80e0030 con 0x55de27a7cf60 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25ad7fa640 1 -- 172.21.15.94:0/3697468621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a80c7cd0 con 0x55de27a7cf60 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.223+0000 7f25ad7fa640 1 -- 172.21.15.94:0/3697468621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8049070 con 0x55de27a7cf60 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- 172.21.15.94:0/3697468621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 msgr2=0x55de27e693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f25a80ff160 tx=0x7f25a8009260 comp rx=0 tx=0).stop 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- 172.21.15.94:0/3697468621 shutdown_connections 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a9d6b0 0x55de27e6a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a954d0 0x55de27e6a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 --2- 172.21.15.94:0/3697468621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a7cf60 0x55de27e693c0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- 172.21.15.94:0/3697468621 >> 172.21.15.94:0/3697468621 conn(0x55de27ab24e0 msgr2=0x55de27e68cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- 172.21.15.94:0/3697468621 shutdown_connections 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- 172.21.15.94:0/3697468621 wait complete. 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.224+0000 7f25c47c3740 1 -- start start 2024-04-25T13:59:18.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 Processor -- start 2024-04-25T13:59:18.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 0x55de27e6e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a954d0 0x55de27e6ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81ac980 con 0x55de27a9d6b0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81ac2b0 con 0x55de27a954d0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81a94f0 con 0x55de27a7cf60 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25c47c3740 10 client.0 _ll_register_callbacks cb 0x7f25a8102960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47882/0 (socket says 172.21.15.94:47882) 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 0x55de27e6e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 -- 172.21.15.94:0/4078033393 learned_addr learned my addr 172.21.15.94:0/4078033393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25affff640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a954d0 0x55de27e6ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 -- 172.21.15.94:0/4078033393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 msgr2=0x55de27e6e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 0x55de27e6e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 -- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a954d0 msgr2=0x55de27e6ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a954d0 0x55de27e6ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 -- 172.21.15.94:0/4078033393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de27e6ff60 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 0x55de27e6e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f25b77fe640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009840 tx=0x7f25b0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d079f734adf081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.225+0000 7f2596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0024040 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f25b00049b0 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a80c7cd0 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f25b0004bb0 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 10 client.21783 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55de27e71250 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a80c7cd0 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f25b003a070 con 0x55de27a9d6b0 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 10 client.21783 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 10 client.21783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 20 client.21783 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27abe390 0x55de27ee2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 4 client.21783 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de27dcc800 con 0x55de27abe390 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.226+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27abe390 0x55de27ee2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f2596ffd640 10 client.21783 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25b6ffd640 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27abe390 0x55de27ee2970 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4127ecbec3b133ff server_cookie=22e43fab628f8f84 in_seq=0 out_seq=0 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f2596ffd640 1 -- 172.21.15.94:0/4078033393 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de27dcc800 con 0x55de27abe390 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f2596ffd640 10 client.21783 handle_command_reply: tid=0 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 client.21783 shutdown 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 msgr2=0x55de27e6f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009840 tx=0x7f25b0009ce0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 shutdown_connections 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55de27abe390 0x55de27ee2970 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de27a9d6b0 0x55de27e6f9a0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f2596ffd640 0 client.21783 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de27a954d0 0x55de27e6ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 --2- 172.21.15.94:0/4078033393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de27a7cf60 0x55de27e6e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 >> 172.21.15.94:0/4078033393 conn(0x55de27ab24e0 msgr2=0x55de27e5ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 shutdown_connections 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.227+0000 7f25c47c3740 1 -- 172.21.15.94:0/4078033393 wait complete. 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25c47c3740 20 client.21783 trim_cache size 0 max 16384 2024-04-25T13:59:18.234 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:18.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 msgr2=0x7f25a40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 0x7f25a40b2080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f25b0021000 tx=0x7f25b001e000 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 msgr2=0x7f25b819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a8031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 shutdown_connections 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f25a40afb70 0x7f25a40b2080 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b8108920 0x7f25b81a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b8103c20 0x7f25b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 --2- 172.21.15.94:0/3016016171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25b81029e0 0x7f25b819a590 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 >> 172.21.15.94:0/3016016171 conn(0x7f25b80fdbd0 msgr2=0x7f25b8109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 shutdown_connections 2024-04-25T13:59:18.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.228+0000 7f25bd90d640 1 -- 172.21.15.94:0/3016016171 wait complete. 2024-04-25T13:59:18.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- 172.21.15.94:0/2317996375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 msgr2=0x7f69f81036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2317996375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f81036f0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009900 tx=0x7f69f402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- 172.21.15.94:0/2317996375 shutdown_connections 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2317996375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2317996375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 0x7f69f8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2317996375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f81036f0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- 172.21.15.94:0/2317996375 >> 172.21.15.94:0/2317996375 conn(0x7f69f80fdc00 msgr2=0x7f69f8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- 172.21.15.94:0/2317996375 shutdown_connections 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- 172.21.15.94:0/2317996375 wait complete. 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 Processor -- start 2024-04-25T13:59:18.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.359+0000 7f69ff49f640 1 -- start start 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 0x7f69f819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f81a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810e4f0 con 0x7f69f81029f0 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810e370 con 0x7f69f8108930 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810e670 con 0x7f69f8103c30 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47902/0 (socket says 172.21.15.94:47902) 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 -- 172.21.15.94:0/2625815968 learned_addr learned my addr 172.21.15.94:0/2625815968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fda3c640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f81a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fca3a640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 0x7f69f819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 -- 172.21.15.94:0/2625815968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 msgr2=0x7f69f819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 0x7f69f819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 -- 172.21.15.94:0/2625815968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 msgr2=0x7f69f81a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f81a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 -- 172.21.15.94:0/2625815968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f81a1d80 con 0x7f69f81029f0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fda3c640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69fd23b640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f69f4006010 tx=0x7f69f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ee7fc640 1 -- 172.21.15.94:0/2625815968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f403f030 con 0x7f69f81029f0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ee7fc640 1 -- 172.21.15.94:0/2625815968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f4031910 con 0x7f69f81029f0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81a2010 con 0x7f69f81029f0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ee7fc640 1 -- 172.21.15.94:0/2625815968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4035070 con 0x7f69f81029f0 2024-04-25T13:59:18.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.360+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a2500 con 0x7f69f81029f0 2024-04-25T13:59:18.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.363+0000 7f6a0642d740 1 Processor -- start 2024-04-25T13:59:18.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 -- start start 2024-04-25T13:59:18.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69ee7fc640 1 -- 172.21.15.94:0/2625815968 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f69f4042030 con 0x7f69f81029f0 2024-04-25T13:59:18.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e965ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9e95bed80 con 0x55f9e928f4d0 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9578a20 con 0x55f9e92976b0 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f6a0642d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e95bad40 con 0x55f9e92b8390 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fda3c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47908/0 (socket says 172.21.15.94:47908) 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fda3c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34894/0 (socket says 172.21.15.94:34894) 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 -- 172.21.15.94:0/825715217 learned_addr learned my addr 172.21.15.94:0/825715217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fca3a640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e965ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 -- 172.21.15.94:0/825715217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 msgr2=0x55f9e965a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 -- 172.21.15.94:0/825715217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 msgr2=0x55f9e965ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e965ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fd23b640 1 -- 172.21.15.94:0/825715217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e9583c30 con 0x55f9e928f4d0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fda3c640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.364+0000 7f69fca3a640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e965ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69ee7fc640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 0x7f69e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69fda3c640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 0x7f69e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69fd23b640 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f69f41071e0 tx=0x7f69f41072d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adec48912f71ec81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69ee7fc640 1 -- 172.21.15.94:0/2625815968 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f69f40feff0 con 0x7f69f81029f0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69e37fe640 1 -- 172.21.15.94:0/825715217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4049020 con 0x55f9e928f4d0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69e37fe640 1 -- 172.21.15.94:0/825715217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f40c6f40 con 0x55f9e928f4d0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.365+0000 7f69e37fe640 1 -- 172.21.15.94:0/825715217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f40c32f0 con 0x55f9e928f4d0 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- 172.21.15.94:0/825715217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 msgr2=0x55f9e965ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f69f41071e0 tx=0x7f69f41072d0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f69fda3c640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 0x7f69e40b2000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55f9e965b860 tx=0x7f69f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- 172.21.15.94:0/825715217 shutdown_connections 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e965ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e965ad90 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 --2- 172.21.15.94:0/825715217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e965a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- 172.21.15.94:0/825715217 >> 172.21.15.94:0/825715217 conn(0x55f9e92ac4e0 msgr2=0x55f9e9656020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- 172.21.15.94:0/825715217 shutdown_connections 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- 172.21.15.94:0/825715217 wait complete. 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.366+0000 7f6a0642d740 1 -- start start 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 Processor -- start 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e96647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e966b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9e965c2f0 con 0x55f9e928f4d0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f9e9545dc0 con 0x55f9e92976b0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e965c470 con 0x55f9e92b8390 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f6a0642d740 10 client.0 _ll_register_callbacks cb 0x55f9e9561600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47916/0 (socket says 172.21.15.94:47916) 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 -- 172.21.15.94:0/829881410 learned_addr learned my addr 172.21.15.94:0/829881410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fca3a640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e96647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fda3c640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e966b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 -- 172.21.15.94:0/829881410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 msgr2=0x55f9e966b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e966b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 -- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 msgr2=0x55f9e96647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e96647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fd23b640 1 -- 172.21.15.94:0/829881410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e92d03f0 con 0x55f9e928f4d0 2024-04-25T13:59:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.367+0000 7f69fca3a640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e96647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.368+0000 7f69fd23b640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f69f40fcaa0 tx=0x7f69f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfa969e9184ae61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.368+0000 7f69e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.368+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4049040 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.368+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f69f4003cc0 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.368+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f40c6f40 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69f40e1050 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 10 client.21795 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f9e939a7d0 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f40c6f40 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f69f40c7690 con 0x55f9e928f4d0 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 10 client.21795 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 10 client.21795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:18.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 20 client.21795 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e9276f60 0x55f9e9671f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 4 client.21795 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e95cc990 con 0x55f9e9276f60 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.369+0000 7f69fca3a640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e9276f60 0x55f9e9671f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.370+0000 7f69e0ff9640 10 client.21795 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:18.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.370+0000 7f69fca3a640 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e9276f60 0x55f9e9671f10 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18b97cbe10a5a57 server_cookie=be08730dc1543c2a in_seq=0 out_seq=0 2024-04-25T13:59:18.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f69e0ff9640 1 -- 172.21.15.94:0/829881410 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9e95cc990 con 0x55f9e9276f60 2024-04-25T13:59:18.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f69e0ff9640 10 client.21795 handle_command_reply: tid=0 2024-04-25T13:59:18.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 client.21795 shutdown 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 msgr2=0x55f9e96642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f69f40fcaa0 tx=0x7f69f4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 shutdown_connections 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e9276f60 0x55f9e9671f10 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e966b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e96647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 --2- 172.21.15.94:0/829881410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96642b0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 >> 172.21.15.94:0/829881410 conn(0x55f9e92ac4e0 msgr2=0x55f9e9658f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f69e0ff9640 0 client.21795 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 shutdown_connections 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 1 -- 172.21.15.94:0/829881410 wait complete. 2024-04-25T13:59:18.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.374+0000 7f6a0642d740 20 client.21795 trim_cache size 0 max 16384 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 Processor -- start 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 -- start start 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e966b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e9659530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ab5a0 con 0x55f9e928f4d0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ad0e0 con 0x55f9e92976b0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f6a0642d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81abc70 con 0x55f9e92b8390 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f69fd23b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47920/0 (socket says 172.21.15.94:47920) 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f69fd23b640 1 -- 172.21.15.94:0/1306193965 learned_addr learned my addr 172.21.15.94:0/1306193965 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.388+0000 7f69fca3a640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e966b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fda3c640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e9659530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 -- 172.21.15.94:0/1306193965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 msgr2=0x55f9e9659530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e9659530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 -- 172.21.15.94:0/1306193965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 msgr2=0x55f9e966b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e966b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 -- 172.21.15.94:0/1306193965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e9663300 con 0x55f9e928f4d0 2024-04-25T13:59:18.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fda3c640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e9659530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f69f40cb800 tx=0x7f69f40fcad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf9a9c4820bf5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69e2ffd640 1 -- 172.21.15.94:0/1306193965 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f40e5070 con 0x55f9e928f4d0 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69e2ffd640 1 -- 172.21.15.94:0/1306193965 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f41028a0 con 0x55f9e928f4d0 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.389+0000 7f69e2ffd640 1 -- 172.21.15.94:0/1306193965 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4049070 con 0x55f9e928f4d0 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- 172.21.15.94:0/1306193965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 msgr2=0x55f9e96648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f69f40cb800 tx=0x7f69f40fcad0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- 172.21.15.94:0/1306193965 shutdown_connections 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e92b8390 0x55f9e9659530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e966b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1306193965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e928f4d0 0x55f9e96648a0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- 172.21.15.94:0/1306193965 >> 172.21.15.94:0/1306193965 conn(0x55f9e92ac4e0 msgr2=0x55f9e9658e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- 172.21.15.94:0/1306193965 shutdown_connections 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- 172.21.15.94:0/1306193965 wait complete. 2024-04-25T13:59:18.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 -- start start 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.390+0000 7f6a0642d740 1 Processor -- start 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e928f4d0 0x55f9e966b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e9669ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81aca10 con 0x55f9e92b8390 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ac340 con 0x55f9e92976b0 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81a9580 con 0x55f9e928f4d0 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f6a0642d740 10 client.0 _ll_register_callbacks cb 0x7f69e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fda3c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e9669ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fda3c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47924/0 (socket says 172.21.15.94:47924) 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fda3c640 1 -- 172.21.15.94:0/1996784585 learned_addr learned my addr 172.21.15.94:0/1996784585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:18.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e928f4d0 0x55f9e966b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 -- 172.21.15.94:0/1996784585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e928f4d0 msgr2=0x55f9e966b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e928f4d0 0x55f9e966b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 -- 172.21.15.94:0/1996784585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 msgr2=0x55f9e966a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 -- 172.21.15.94:0/1996784585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e966a620 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fda3c640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69fca3a640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e9669ba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55f9e9672680 tx=0x7f69e8012410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31cb0389c77f5a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.391+0000 7f69e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8004250 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f41028a0 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f69e80043f0 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69e8012710 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 10 client.31232 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f9e96d8fd0 con 0x55f9e92976b0 2024-04-25T13:59:18.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f41028a0 con 0x55f9e92976b0 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f69e8003b60 con 0x55f9e92976b0 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 10 client.31232 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 10 client.31232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 20 client.31232 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e924c3a0 0x55f9e96db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 4 client.31232 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9e95c6800 con 0x55f9e924c3a0 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.392+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e924c3a0 0x55f9e96db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.393+0000 7f69e0ff9640 10 client.31232 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:18.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.393+0000 7f69fd23b640 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e924c3a0 0x55f9e96db8e0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff21687558202a4d server_cookie=be3c99ee004c191f in_seq=0 out_seq=0 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f69e0ff9640 1 -- 172.21.15.94:0/1996784585 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9e95c6800 con 0x55f9e924c3a0 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f69e0ff9640 10 client.31232 handle_command_reply: tid=0 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 client.31232 shutdown 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 msgr2=0x55f9e9669ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e9669ba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55f9e9672680 tx=0x7f69e8012410 comp rx=0 tx=0).stop 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 shutdown_connections 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f9e924c3a0 0x55f9e96db8e0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9e92b8390 0x55f9e966a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9e92976b0 0x55f9e9669ba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f69e0ff9640 0 client.31232 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 --2- 172.21.15.94:0/1996784585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e928f4d0 0x55f9e966b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.395+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 >> 172.21.15.94:0/1996784585 conn(0x55f9e92ac4e0 msgr2=0x55f9e9662c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 shutdown_connections 2024-04-25T13:59:18.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f6a0642d740 1 -- 172.21.15.94:0/1996784585 wait complete. 2024-04-25T13:59:18.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f6a0642d740 20 client.31232 trim_cache size 0 max 16384 2024-04-25T13:59:18.401 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:18.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 msgr2=0x7f69e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 0x7f69e40b2000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55f9e965b860 tx=0x7f69f0009500 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 msgr2=0x7f69f806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f69f4006010 tx=0x7f69f4004270 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 shutdown_connections 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f69e40afaf0 0x7f69e40b2000 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8108930 0x7f69f81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f8103c30 0x7f69f819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 --2- 172.21.15.94:0/2625815968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f81029f0 0x7f69f806bce0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 >> 172.21.15.94:0/2625815968 conn(0x7f69f80fdc00 msgr2=0x7f69f8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 shutdown_connections 2024-04-25T13:59:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:18.396+0000 7f69ff49f640 1 -- 172.21.15.94:0/2625815968 wait complete. 2024-04-25T13:59:19.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 88096 19.35 MB/sec execute 121 sec latency 67.636 ms 2024-04-25T13:59:19.410 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:59:19.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:19.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 -- 172.21.15.94:0/1545405334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 msgr2=0x7f84701054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 --2- 172.21.15.94:0/1545405334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701054a0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f846c009900 tx=0x7f846c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 -- 172.21.15.94:0/1545405334 shutdown_connections 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 --2- 172.21.15.94:0/1545405334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 0x7f847010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 --2- 172.21.15.94:0/1545405334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 --2- 172.21.15.94:0/1545405334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701054a0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 -- 172.21.15.94:0/1545405334 >> 172.21.15.94:0/1545405334 conn(0x7f84700fd340 msgr2=0x7f84700ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 -- 172.21.15.94:0/1545405334 shutdown_connections 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.532+0000 7f847784a640 1 -- 172.21.15.94:0/1545405334 wait complete. 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 Processor -- start 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 -- start start 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 0x7f8470198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f847010f5e0 con 0x7f8470105090 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f847010f460 con 0x7f8470102240 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f847784a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f847010f760 con 0x7f8470101740 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f8475de7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f8474de5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 0x7f8470198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f8475de7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47942/0 (socket says 172.21.15.94:47942) 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.533+0000 7f8475de7640 1 -- 172.21.15.94:0/2163234148 learned_addr learned my addr 172.21.15.94:0/2163234148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f84755e6640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 -- 172.21.15.94:0/2163234148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 msgr2=0x7f8470198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 -- 172.21.15.94:0/2163234148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 msgr2=0x7f8470198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 0x7f8470198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 -- 172.21.15.94:0/2163234148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84701a82d0 con 0x7f8470105090 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f84755e6640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f8475de7640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f846800eb10 tx=0x7f846800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f84667fc640 1 -- 172.21.15.94:0/2163234148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800ce20 con 0x7f8470105090 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f84667fc640 1 -- 172.21.15.94:0/2163234148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468005020 con 0x7f8470105090 2024-04-25T13:59:19.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84701a85c0 con 0x7f8470105090 2024-04-25T13:59:19.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f84667fc640 1 -- 172.21.15.94:0/2163234148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468010690 con 0x7f8470105090 2024-04-25T13:59:19.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.534+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84701a8ac0 con 0x7f8470105090 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f84667fc640 1 -- 172.21.15.94:0/2163234148 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8468005510 con 0x7f8470105090 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 Processor -- start 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 -- start start 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff29a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff30920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f1ffe8eb00 con 0x55f1ffb88410 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1ffe2a920 con 0x55f1ffb5f550 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f847e7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1ffe8aac0 con 0x55f1ffb67730 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f84755e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff29a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47944/0 (socket says 172.21.15.94:47944) 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f8474de5640 1 -- 172.21.15.94:0/2744154154 learned_addr learned my addr 172.21.15.94:0/2744154154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f84667fc640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 0x7f845c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f8475de7640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 0x7f845c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.537+0000 7f8474de5640 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff30920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 -- 172.21.15.94:0/2744154154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 msgr2=0x55f1fff30920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff30920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 -- 172.21.15.94:0/2744154154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 msgr2=0x55f1fff29a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff29a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 -- 172.21.15.94:0/2744154154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ffc687e0 con 0x55f1ffb88410 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f84755e6640 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff29a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f84667fc640 1 -- 172.21.15.94:0/2163234148 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f84680df2c0 con 0x7f8470105090 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8474de5640 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f846000af40 tx=0x7f846000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c14692d8fb94fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f8475de7640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 0x7f845c0b2000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f84680a4970 tx=0x7f8468072e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f845b7fe640 1 -- 172.21.15.94:0/2744154154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000f650 con 0x55f1ffb88410 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f845b7fe640 1 -- 172.21.15.94:0/2744154154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460017030 con 0x55f1ffb88410 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.538+0000 7f845b7fe640 1 -- 172.21.15.94:0/2744154154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846001b630 con 0x55f1ffb88410 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- 172.21.15.94:0/2744154154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 msgr2=0x55f1fff294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f846000af40 tx=0x7f846000a860 comp rx=0 tx=0).stop 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- 172.21.15.94:0/2744154154 shutdown_connections 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff30920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff29a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 --2- 172.21.15.94:0/2744154154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff294a0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- 172.21.15.94:0/2744154154 >> 172.21.15.94:0/2744154154 conn(0x55f1ffb7c560 msgr2=0x55f1fff24e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- 172.21.15.94:0/2744154154 shutdown_connections 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- 172.21.15.94:0/2744154154 wait complete. 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.539+0000 7f847e7de740 1 -- start start 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 Processor -- start 2024-04-25T13:59:19.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f1fff31370 con 0x55f1ffb88410 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1ffe285d0 con 0x55f1ffb5f550 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1fff314f0 con 0x55f1ffb67730 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f847e7de740 10 client.0 _ll_register_callbacks cb 0x55f1ffe16080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47946/0 (socket says 172.21.15.94:47946) 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f8474de5640 1 -- 172.21.15.94:0/327226892 learned_addr learned my addr 172.21.15.94:0/327226892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f8474de5640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.540+0000 7f84755e6640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 -- 172.21.15.94:0/327226892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 msgr2=0x55f1fff337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 -- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 msgr2=0x55f1fff33290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff33290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 -- 172.21.15.94:0/327226892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ffcc8f70 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8474de5640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f846000aa20 tx=0x7f846000a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dda06fb32c21f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000f450 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f846000f5f0 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8460017030 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f846000f7f0 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f847e7de740 10 client.21813 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f1ffb60170 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.541+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460017030 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f846001c8a0 con 0x55f1ffb88410 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 10 client.21813 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 10 client.21813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 20 client.21813 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb471f0 0x55f1fff40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 4 client.21813 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ffe9c710 con 0x55f1ffb471f0 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f84755e6640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb471f0 0x55f1fff40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f8458ff9640 10 client.21813 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.542+0000 7f84755e6640 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb471f0 0x55f1fff40920 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416a28a73b7e7ef8 server_cookie=517e4099a98e930f in_seq=0 out_seq=0 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f8458ff9640 1 -- 172.21.15.94:0/327226892 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1ffe9c710 con 0x55f1ffb471f0 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f8458ff9640 10 client.21813 handle_command_reply: tid=0 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 client.21813 shutdown 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 msgr2=0x55f1fff3a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f846000aa20 tx=0x7f846000a310 comp rx=0 tx=0).stop 2024-04-25T13:59:19.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 shutdown_connections 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb471f0 0x55f1fff40920 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3a730 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 --2- 172.21.15.94:0/327226892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 >> 172.21.15.94:0/327226892 conn(0x55f1ffb7c560 msgr2=0x55f1fff23ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f8458ff9640 0 client.21813 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 shutdown_connections 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 1 -- 172.21.15.94:0/327226892 wait complete. 2024-04-25T13:59:19.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.544+0000 7f847e7de740 20 client.21813 trim_cache size 0 max 16384 2024-04-25T13:59:19.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 Processor -- start 2024-04-25T13:59:19.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 -- start start 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff3a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b1b30 con 0x55f1ffb88410 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b3670 con 0x55f1ffb5f550 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.562+0000 7f847e7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b2200 con 0x55f1ffb67730 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47954/0 (socket says 172.21.15.94:47954) 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 -- 172.21.15.94:0/3017072841 learned_addr learned my addr 172.21.15.94:0/3017072841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff31fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 -- 172.21.15.94:0/3017072841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 msgr2=0x55f1fff3a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f84755e6640 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff3a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff3a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 -- 172.21.15.94:0/3017072841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 msgr2=0x55f1fff31fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff31fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 -- 172.21.15.94:0/3017072841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1fff2e130 con 0x55f1ffb88410 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f8474de5640 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f846001cc70 tx=0x7f8460000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60d210517b4867f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f845affd640 1 -- 172.21.15.94:0/3017072841 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000a400 con 0x55f1ffb88410 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f845affd640 1 -- 172.21.15.94:0/3017072841 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460017070 con 0x55f1ffb88410 2024-04-25T13:59:19.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.563+0000 7f845affd640 1 -- 172.21.15.94:0/3017072841 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460012cd0 con 0x55f1ffb88410 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 -- 172.21.15.94:0/3017072841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 msgr2=0x55f1fff3ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f846001cc70 tx=0x7f8460000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 -- 172.21.15.94:0/3017072841 shutdown_connections 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb88410 0x55f1fff3ab90 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb67730 0x55f1fff3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 --2- 172.21.15.94:0/3017072841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff31fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 -- 172.21.15.94:0/3017072841 >> 172.21.15.94:0/3017072841 conn(0x55f1ffb7c560 msgr2=0x55f1fff31890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 -- 172.21.15.94:0/3017072841 shutdown_connections 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.564+0000 7f847e7de740 1 -- 172.21.15.94:0/3017072841 wait complete. 2024-04-25T13:59:19.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 -- start start 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 Processor -- start 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb88410 0x55f1fff3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b2fa0 con 0x55f1ffb67730 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b28d0 con 0x55f1ffb5f550 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84701afb10 con 0x55f1ffb88410 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f847e7de740 10 client.0 _ll_register_callbacks cb 0x7f8460012850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f84755e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f84755e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47958/0 (socket says 172.21.15.94:47958) 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f84755e6640 1 -- 172.21.15.94:0/3861241555 learned_addr learned my addr 172.21.15.94:0/3861241555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f8474de5640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.565+0000 7f8474de5640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb88410 0x55f1fff3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 -- 172.21.15.94:0/3861241555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb88410 msgr2=0x55f1fff3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb88410 0x55f1fff3a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 -- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 msgr2=0x55f1fff398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 -- 172.21.15.94:0/3861241555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1fffa6e00 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f84755e6640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f846c009230 tx=0x7f846c0041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7457a9047cd5e862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c035040 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f846c005040 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8460017070 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f846c005240 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 10 client.21819 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f1fffa8190 con 0x55f1ffb67730 2024-04-25T13:59:19.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460017070 con 0x55f1ffb67730 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f846c0046c0 con 0x55f1ffb67730 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 10 client.21819 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 10 client.21819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.566+0000 7f847e7de740 20 client.21819 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb1c630 0x55f1fffaab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f847e7de740 4 client.21819 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f1ffe96580 con 0x55f1ffb1c630 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f8474de5640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb1c630 0x55f1fffaab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f8458ff9640 10 client.21819 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.567+0000 7f8474de5640 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb1c630 0x55f1fffaab70 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7865f9c3fc4b4266 server_cookie=64295ecd9f7a3ee3 in_seq=0 out_seq=0 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f8458ff9640 1 -- 172.21.15.94:0/3861241555 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f1ffe96580 con 0x55f1ffb1c630 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f8458ff9640 10 client.21819 handle_command_reply: tid=0 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 client.21819 shutdown 2024-04-25T13:59:19.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 msgr2=0x55f1fff39e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f846c009230 tx=0x7f846c0041f0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 shutdown_connections 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f1ffb1c630 0x55f1fffaab70 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1ffb88410 0x55f1fff3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1ffb67730 0x55f1fff39e30 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f8458ff9640 0 client.21819 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 --2- 172.21.15.94:0/3861241555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1ffb5f550 0x55f1fff398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 >> 172.21.15.94:0/3861241555 conn(0x55f1ffb7c560 msgr2=0x55f1fff27c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 shutdown_connections 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.569+0000 7f847e7de740 1 -- 172.21.15.94:0/3861241555 wait complete. 2024-04-25T13:59:19.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847e7de740 20 client.21819 trim_cache size 0 max 16384 2024-04-25T13:59:19.575 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:19.575 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:59:19.575 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "15e1e09e-9d4f-4808-8898-6eaf4617c6ee", 2024-04-25T13:59:19.575 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:59:19.575 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 msgr2=0x7f845c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 0x7f845c0b2000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f84680a4970 tx=0x7f8468072e50 comp rx=0 tx=0).stop 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 msgr2=0x7f84701a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f846800eb10 tx=0x7f846800ef10 comp rx=0 tx=0).stop 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 shutdown_connections 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f845c0afaf0 0x7f845c0b2000 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8470105090 0x7f84701a7ce0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8470102240 0x7f8470198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 --2- 172.21.15.94:0/2163234148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8470101740 0x7f8470198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 >> 172.21.15.94:0/2163234148 conn(0x7f84700fd340 msgr2=0x7f84700fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 shutdown_connections 2024-04-25T13:59:19.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.570+0000 7f847784a640 1 -- 172.21.15.94:0/2163234148 wait complete. 2024-04-25T13:59:19.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:19.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 -- 172.21.15.94:0/1121487278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa0103c30 msgr2=0x7f3aa0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 --2- 172.21.15.94:0/1121487278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa0103c30 0x7f3aa0104040 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009900 tx=0x7f3a8802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 -- 172.21.15.94:0/1121487278 shutdown_connections 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 --2- 172.21.15.94:0/1121487278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0108930 0x7f3aa010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 --2- 172.21.15.94:0/1121487278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa0103c30 0x7f3aa0104040 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 --2- 172.21.15.94:0/1121487278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa01029f0 0x7f3aa01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 -- 172.21.15.94:0/1121487278 >> 172.21.15.94:0/1121487278 conn(0x7f3aa00fdc00 msgr2=0x7f3aa0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 -- 172.21.15.94:0/1121487278 shutdown_connections 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 -- 172.21.15.94:0/1121487278 wait complete. 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.702+0000 7f3aa541a640 1 Processor -- start 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 -- start start 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0103c30 0x7f3aa01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa0108930 0x7f3aa01a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa010e520 con 0x7f3aa01029f0 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa010e3a0 con 0x7f3aa0108930 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3aa541a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa010e6a0 con 0x7f3aa0103c30 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47968/0 (socket says 172.21.15.94:47968) 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 -- 172.21.15.94:0/3167039259 learned_addr learned my addr 172.21.15.94:0/3167039259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa0108930 0x7f3aa01a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a967fc640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0103c30 0x7f3aa01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 -- 172.21.15.94:0/3167039259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0103c30 msgr2=0x7f3aa01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0103c30 0x7f3aa01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 -- 172.21.15.94:0/3167039259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa0108930 msgr2=0x7f3aa01a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa0108930 0x7f3aa01a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 -- 172.21.15.94:0/3167039259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa01a0c30 con 0x7f3aa01029f0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9effd640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e9f0 tx=0x7f3a9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.703+0000 7f3a9cff9640 1 -- 172.21.15.94:0/3167039259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000ccc0 con 0x7f3aa01029f0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.704+0000 7f3a9cff9640 1 -- 172.21.15.94:0/3167039259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a900035f0 con 0x7f3aa01029f0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.704+0000 7f3a9cff9640 1 -- 172.21.15.94:0/3167039259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000ce60 con 0x7f3aa01029f0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.704+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa01a0f20 con 0x7f3aa01029f0 2024-04-25T13:59:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.704+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa01a13e0 con 0x7f3aa01029f0 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.706+0000 7f3a9cff9640 1 -- 172.21.15.94:0/3167039259 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3a90003aa0 con 0x7f3aa01029f0 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.706+0000 7f3aac3a1740 1 Processor -- start 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.706+0000 7f3aac3a1740 1 -- start start 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdaac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d18bd0dd80 con 0x55d18b9de4d0 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d18bcc78b0 con 0x55d18b9e66b0 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d18bd09d40 con 0x55d18ba07390 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:60740/0 (socket says 172.21.15.94:60740) 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47984/0 (socket says 172.21.15.94:47984) 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 -- 172.21.15.94:0/837045402 learned_addr learned my addr 172.21.15.94:0/837045402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdaac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9cff9640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 0x7f3a8c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 -- 172.21.15.94:0/837045402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 msgr2=0x55d18bdaac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdaac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 -- 172.21.15.94:0/837045402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 msgr2=0x55d18bdab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 -- 172.21.15.94:0/837045402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18bc98a40 con 0x55d18b9e66b0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 0x7f3a8c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9effd640 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a9cff9640 1 -- 172.21.15.94:0/3167039259 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a900def40 con 0x7f3aa01029f0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.707+0000 7f3a967fc640 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3a88031500 tx=0x7f3a88009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0cdd01bad3be48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 0x7f3a8c0b2080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55d18bdabcc0 tx=0x7f3a98004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3a957fa640 1 -- 172.21.15.94:0/837045402 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88035030 con 0x55d18b9e66b0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3a957fa640 1 -- 172.21.15.94:0/837045402 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a88040070 con 0x55d18b9e66b0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3a957fa640 1 -- 172.21.15.94:0/837045402 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88003950 con 0x55d18b9e66b0 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3aac3a1740 1 -- 172.21.15.94:0/837045402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 msgr2=0x55d18bdabe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.708+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3a88031500 tx=0x7f3a88009640 comp rx=0 tx=0).stop 2024-04-25T13:59:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- 172.21.15.94:0/837045402 shutdown_connections 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdabe80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/837045402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdaac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- 172.21.15.94:0/837045402 >> 172.21.15.94:0/837045402 conn(0x55d18b9fb4e0 msgr2=0x55d18bda4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- 172.21.15.94:0/837045402 shutdown_connections 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- 172.21.15.94:0/837045402 wait complete. 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- start start 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 Processor -- start 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d18bdac930 con 0x55d18b9de4d0 2024-04-25T13:59:19.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d18bc94d10 con 0x55d18b9e66b0 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d18bdacab0 con 0x55d18ba07390 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3aac3a1740 10 client.0 _ll_register_callbacks cb 0x55d18bcc1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47988/0 (socket says 172.21.15.94:47988) 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.709+0000 7f3a9effd640 1 -- 172.21.15.94:0/4063156099 learned_addr learned my addr 172.21.15.94:0/4063156099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a967fc640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 -- 172.21.15.94:0/4063156099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 msgr2=0x55d18bdbbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 -- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 msgr2=0x55d18bdb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 -- 172.21.15.94:0/4063156099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18bae97d0 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a967fc640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a9effd640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f3a900a7080 tx=0x7f3a900a2be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3017e324205bb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a867fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900dce60 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a900e2de0 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a88040070 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a900a6cd0 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3aac3a1740 10 client.21831 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d18ba1f3f0 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.710+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a88040070 con 0x55d18b9de4d0 2024-04-25T13:59:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3a900e10d0 con 0x55d18b9de4d0 2024-04-25T13:59:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 10 client.21831 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 10 client.21831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 20 client.21831 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b9c5f60 0x55d18bdc2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 4 client.21831 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18bd1b990 con 0x55d18b9c5f60 2024-04-25T13:59:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3a967fc640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b9c5f60 0x55d18bdc2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3a967fc640 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b9c5f60 0x55d18bdc2510 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f9a93b47c51650 server_cookie=3445a954c1ca2a7d in_seq=0 out_seq=0 2024-04-25T13:59:19.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.711+0000 7f3a867fc640 10 client.21831 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3a867fc640 1 -- 172.21.15.94:0/4063156099 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d18bd1b990 con 0x55d18b9c5f60 2024-04-25T13:59:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3a867fc640 10 client.21831 handle_command_reply: tid=0 2024-04-25T13:59:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 client.21831 shutdown 2024-04-25T13:59:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 msgr2=0x55d18bdb4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f3a900a7080 tx=0x7f3a900a2be0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 shutdown_connections 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b9c5f60 0x55d18bdc2510 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/4063156099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb4af0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 >> 172.21.15.94:0/4063156099 conn(0x55d18b9fb4e0 msgr2=0x55d18bda33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3a867fc640 0 client.21831 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 shutdown_connections 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 1 -- 172.21.15.94:0/4063156099 wait complete. 2024-04-25T13:59:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.716+0000 7f3aac3a1740 20 client.21831 trim_cache size 0 max 16384 2024-04-25T13:59:19.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:19 smithi094 ceph-mon[29195]: pgmap v877: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-04-25T13:59:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 Processor -- start 2024-04-25T13:59:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 -- start start 2024-04-25T13:59:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a92e0 con 0x55d18b9de4d0 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01aae20 con 0x55d18b9e66b0 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.735+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a99b0 con 0x55d18ba07390 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48002/0 (socket says 172.21.15.94:48002) 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 -- 172.21.15.94:0/2247971620 learned_addr learned my addr 172.21.15.94:0/2247971620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a967fc640 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 -- 172.21.15.94:0/2247971620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 msgr2=0x55d18bdbbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 -- 172.21.15.94:0/2247971620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 msgr2=0x55d18bdb4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 -- 172.21.15.94:0/2247971620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18b9df0f0 con 0x55d18b9de4d0 2024-04-25T13:59:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a9effd640 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f3a900a45f0 tx=0x7f3a9000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54267eb005f60bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a94ff9640 1 -- 172.21.15.94:0/2247971620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900dce60 con 0x55d18b9de4d0 2024-04-25T13:59:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a94ff9640 1 -- 172.21.15.94:0/2247971620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a900a6e90 con 0x55d18b9de4d0 2024-04-25T13:59:19.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.736+0000 7f3a94ff9640 1 -- 172.21.15.94:0/2247971620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a900a6af0 con 0x55d18b9de4d0 2024-04-25T13:59:19.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 -- 172.21.15.94:0/2247971620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 msgr2=0x55d18bdb3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f3a900a45f0 tx=0x7f3a9000c7d0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 -- 172.21.15.94:0/2247971620 shutdown_connections 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18ba07390 0x55d18bdbbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9e66b0 0x55d18bdb4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/2247971620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18b9de4d0 0x55d18bdb3b40 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 -- 172.21.15.94:0/2247971620 >> 172.21.15.94:0/2247971620 conn(0x55d18b9fb4e0 msgr2=0x55d18bda70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 -- 172.21.15.94:0/2247971620 shutdown_connections 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.737+0000 7f3aac3a1740 1 -- 172.21.15.94:0/2247971620 wait complete. 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 -- start start 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 Processor -- start 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9de4d0 0x55d18bdbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18b9e66b0 0x55d18bdb87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01aa750 con 0x55d18ba07390 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01aa080 con 0x55d18b9de4d0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa01a72c0 con 0x55d18b9e66b0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3aac3a1740 10 client.0 _ll_register_callbacks cb 0x7f3a900046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9de4d0 0x55d18bdbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48006/0 (socket says 172.21.15.94:48006) 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 -- 172.21.15.94:0/1512215383 learned_addr learned my addr 172.21.15.94:0/1512215383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a967fc640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18b9e66b0 0x55d18bdb87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 -- 172.21.15.94:0/1512215383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18b9e66b0 msgr2=0x55d18bdb87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18b9e66b0 0x55d18bdb87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 -- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9de4d0 msgr2=0x55d18bdbbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:19.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9de4d0 0x55d18bdbbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 -- 172.21.15.94:0/1512215383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18bcce4b0 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a9f7fe640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f3a98013c50 tx=0x7f3a98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2bc24e09147318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.738+0000 7f3a867fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801d430 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a9801d5d0 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a900a6e90 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a9801d7d0 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 10 client.21837 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d18be296a0 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a900a6e90 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3a9801fa80 con 0x55d18ba07390 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 10 client.21837 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 10 client.21837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:19.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 20 client.21837 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b99b3a0 0x55d18be2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 4 client.21837 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.739+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d18bd15800 con 0x55d18b99b3a0 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.740+0000 7f3a9effd640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b99b3a0 0x55d18be2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.740+0000 7f3a9effd640 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b99b3a0 0x55d18be2c060 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3300672af2e98b0b server_cookie=d0cb930e871c545a in_seq=0 out_seq=0 2024-04-25T13:59:19.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.740+0000 7f3a867fc640 10 client.21837 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.742+0000 7f3a867fc640 1 -- 172.21.15.94:0/1512215383 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d18bd15800 con 0x55d18b99b3a0 2024-04-25T13:59:19.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.742+0000 7f3a867fc640 10 client.21837 handle_command_reply: tid=0 2024-04-25T13:59:19.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.742+0000 7f3aac3a1740 1 client.21837 shutdown 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 msgr2=0x55d18bdb8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f3a98013c50 tx=0x7f3a98004ab0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 shutdown_connections 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d18b99b3a0 0x55d18be2c060 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d18ba07390 0x55d18bdb8cf0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d18b9e66b0 0x55d18bdb87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 --2- 172.21.15.94:0/1512215383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18b9de4d0 0x55d18bdbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 >> 172.21.15.94:0/1512215383 conn(0x55d18b9fb4e0 msgr2=0x55d18bdb38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3a867fc640 0 client.21837 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 shutdown_connections 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 1 -- 172.21.15.94:0/1512215383 wait complete. 2024-04-25T13:59:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aac3a1740 20 client.21837 trim_cache size 0 max 16384 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (289 inodes in the stack)", 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "15e1e09e-9d4f-4808-8898-6eaf4617c6ee": { 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "15e1e09e-9d4f-4808-8898-6eaf4617c6ee", 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:59:19.748 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 msgr2=0x7f3a8c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 0x7f3a8c0b2080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55d18bdabcc0 tx=0x7f3a98004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 msgr2=0x7f3aa0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e9f0 tx=0x7f3a9000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 shutdown_connections 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a8c0afb70 0x7f3a8c0b2080 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3aa0108930 0x7f3aa01a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3aa0103c30 0x7f3aa01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 --2- 172.21.15.94:0/3167039259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3aa01029f0 0x7f3aa0199160 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.743+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 >> 172.21.15.94:0/3167039259 conn(0x7f3aa00fdc00 msgr2=0x7f3aa0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.744+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 shutdown_connections 2024-04-25T13:59:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:19.744+0000 7f3aa541a640 1 -- 172.21.15.94:0/3167039259 wait complete. 2024-04-25T13:59:19.755 INFO:tasks.cephfs.filesystem:scrub status for tag:15e1e09e-9d4f-4808-8898-6eaf4617c6ee - {'path': '/', 'tag': '15e1e09e-9d4f-4808-8898-6eaf4617c6ee', 'options': 'recursive,force'} 2024-04-25T13:59:19.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:19 smithi138 ceph-mon[32977]: pgmap v877: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-04-25T13:59:20.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:19 smithi137 ceph-mon[32568]: pgmap v877: 97 pgs: 97 active+clean; 247 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-04-25T13:59:20.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 88440 19.39 MB/sec execute 122 sec latency 105.157 ms 2024-04-25T13:59:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:59:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:20.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.976 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:20 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:59:20.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.999 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.999 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.999 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:20.999 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:20.999 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:20 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 88733 19.36 MB/sec execute 123 sec latency 191.459 ms 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.144 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:21.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.145 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:20 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:21 smithi094 ceph-mon[29195]: pgmap v878: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-04-25T13:59:21.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:21 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:21.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:21 smithi138 ceph-mon[32977]: pgmap v878: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-04-25T13:59:21.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:21 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:22.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:21 smithi137 ceph-mon[32568]: pgmap v878: 97 pgs: 97 active+clean; 252 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 632 op/s 2024-04-25T13:59:22.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:21 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:22.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 89198 19.41 MB/sec execute 124 sec latency 52.801 ms 2024-04-25T13:59:23.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 89578 19.38 MB/sec execute 125 sec latency 95.684 ms 2024-04-25T13:59:23.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:23 smithi094 ceph-mon[29195]: pgmap v879: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-25T13:59:23.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:23 smithi138 ceph-mon[32977]: pgmap v879: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-25T13:59:24.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:23 smithi137 ceph-mon[32568]: pgmap v879: 97 pgs: 97 active+clean; 253 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.9 MiB/s wr, 510 op/s 2024-04-25T13:59:24.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 89985 19.42 MB/sec execute 126 sec latency 70.096 ms 2024-04-25T13:59:25.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 90382 19.46 MB/sec execute 127 sec latency 96.404 ms 2024-04-25T13:59:25.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:25 smithi094 ceph-mon[29195]: pgmap v880: 97 pgs: 97 active+clean; 260 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-04-25T13:59:25.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:25 smithi138 ceph-mon[32977]: pgmap v880: 97 pgs: 97 active+clean; 260 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-04-25T13:59:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 -- 172.21.15.94:0/3968672567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724106f70 msgr2=0x7f8724109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 --2- 172.21.15.94:0/3968672567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724106f70 0x7f8724109390 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f8718009900 tx=0x7f871802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 -- 172.21.15.94:0/3968672567 shutdown_connections 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 --2- 172.21.15.94:0/3968672567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 0x7f872410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 --2- 172.21.15.94:0/3968672567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724106f70 0x7f8724109390 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 --2- 172.21.15.94:0/3968672567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724104610 0x7f8724106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 -- 172.21.15.94:0/3968672567 >> 172.21.15.94:0/3968672567 conn(0x7f87240fdc10 msgr2=0x7f87241000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 -- 172.21.15.94:0/3968672567 shutdown_connections 2024-04-25T13:59:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.092+0000 7f8729fa8640 1 -- 172.21.15.94:0/3968672567 wait complete. 2024-04-25T13:59:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.093+0000 7f8729fa8640 1 Processor -- start 2024-04-25T13:59:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.093+0000 7f8729fa8640 1 -- start start 2024-04-25T13:59:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.093+0000 7f8729fa8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.093+0000 7f8729fa8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724106f70 0x7f8724102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 0x7f8724102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f872410f650 con 0x7f8724104610 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f872410f4d0 con 0x7f8724106f70 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f872410f7d0 con 0x7f87241098d0 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52874/0 (socket says 172.21.15.94:52874) 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 -- 172.21.15.94:0/248571988 learned_addr learned my addr 172.21.15.94:0/248571988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8722ffd640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724106f70 0x7f8724102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8723fff640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 0x7f8724102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 -- 172.21.15.94:0/248571988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 msgr2=0x7f8724102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 0x7f8724102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 -- 172.21.15.94:0/248571988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724106f70 msgr2=0x7f8724102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724106f70 0x7f8724102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 -- 172.21.15.94:0/248571988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8724102d70 con 0x7f8724104610 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f87237fe640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f871400e9c0 tx=0x7f871400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8720ff9640 1 -- 172.21.15.94:0/248571988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400cd80 con 0x7f8724104610 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8720ff9640 1 -- 172.21.15.94:0/248571988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140054d0 con 0x7f8724104610 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87241a4340 con 0x7f8724104610 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8720ff9640 1 -- 172.21.15.94:0/248571988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714010430 con 0x7f8724104610 2024-04-25T13:59:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.094+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241a4800 con 0x7f8724104610 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.097+0000 7f8720ff9640 1 -- 172.21.15.94:0/248571988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f87140105d0 con 0x7f8724104610 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.097+0000 7f8730e55740 1 Processor -- start 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.097+0000 7f8730e55740 1 -- start start 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d6605970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6605ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d660cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623d656b460 con 0x5623d62444d0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623d6525590 con 0x5623d62651b0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8730e55740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623d6567420 con 0x5623d623c2f0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8720ff9640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 0x7f870c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d660cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6605ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d660cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52888/0 (socket says 172.21.15.94:52888) 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8722ffd640 1 -- 172.21.15.94:0/871092379 learned_addr learned my addr 172.21.15.94:0/871092379 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8723fff640 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d6605970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8723fff640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 0x7f870c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f8720ff9640 1 -- 172.21.15.94:0/248571988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87140df0d0 con 0x7f8724104610 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 -- 172.21.15.94:0/871092379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 msgr2=0x5623d6605970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d6605970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 -- 172.21.15.94:0/871092379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 msgr2=0x5623d660cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d660cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.098+0000 7f87237fe640 1 -- 172.21.15.94:0/871092379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d65320d0 con 0x5623d623c2f0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.099+0000 7f87237fe640 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6605ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f87140a3cd0 tx=0x7f87140049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6252466c85d4115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.099+0000 7f8723fff640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 0x7f870c0b1ff0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x5623d6606b40 tx=0x7f871c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.099+0000 7f8709ffb640 1 -- 172.21.15.94:0/871092379 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140a6e60 con 0x5623d623c2f0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.099+0000 7f8709ffb640 1 -- 172.21.15.94:0/871092379 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140046c0 con 0x5623d623c2f0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.099+0000 7f8709ffb640 1 -- 172.21.15.94:0/871092379 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140a6c80 con 0x5623d623c2f0 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 -- 172.21.15.94:0/871092379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 msgr2=0x5623d6605ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6605ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f87140a3cd0 tx=0x7f87140049b0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 -- 172.21.15.94:0/871092379 shutdown_connections 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d660cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6605ef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 --2- 172.21.15.94:0/871092379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d6605970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 -- 172.21.15.94:0/871092379 >> 172.21.15.94:0/871092379 conn(0x5623d6259300 msgr2=0x5623d6601300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 -- 172.21.15.94:0/871092379 shutdown_connections 2024-04-25T13:59:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.100+0000 7f8730e55740 1 -- 172.21.15.94:0/871092379 wait complete. 2024-04-25T13:59:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 -- start start 2024-04-25T13:59:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 Processor -- start 2024-04-25T13:59:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d660f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d660fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6616a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5623d660d780 con 0x5623d623c2f0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623d64f2b60 con 0x5623d62444d0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8730e55740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623d660d900 con 0x5623d62651b0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d660fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.101+0000 7f8723fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6616a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d660fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53682/0 (socket says 172.21.15.94:53682) 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8722ffd640 1 -- 172.21.15.94:0/2924557411 learned_addr learned my addr 172.21.15.94:0/2924557411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8730e55740 10 client.0 _ll_register_callbacks cb 0x5623d65172b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f87237fe640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d660f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 -- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 msgr2=0x5623d660fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d660fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 -- 172.21.15.94:0/2924557411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 msgr2=0x5623d660f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d660f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 -- 172.21.15.94:0/2924557411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d627d210 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8723fff640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6616a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f872406b4a0 tx=0x7f871c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ec5e8c12b8ba8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c011220 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623d6579070 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f871c004b20 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.102+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f871c004d20 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 10 client.30940 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623d6349620 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5623d6579070 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f871c030070 con 0x5623d62651b0 2024-04-25T13:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 10 client.30940 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 10 client.30940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 20 client.30940 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d6223d80 0x5623d661d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 4 client.30940 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.103+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623d6572ee0 con 0x5623d6223d80 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.104+0000 7f87237fe640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d6223d80 0x5623d661d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.104+0000 7f8702ffd640 10 client.30940 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.104+0000 7f87237fe640 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d6223d80 0x5623d661d340 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc69aac9c8f4b25 server_cookie=517c5fcd32faf270 in_seq=0 out_seq=0 2024-04-25T13:59:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8702ffd640 1 -- 172.21.15.94:0/2924557411 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623d6572ee0 con 0x5623d6223d80 2024-04-25T13:59:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8702ffd640 10 client.30940 handle_command_reply: tid=0 2024-04-25T13:59:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 client.30940 shutdown 2024-04-25T13:59:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 msgr2=0x5623d6616a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6616a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f872406b4a0 tx=0x7f871c004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 shutdown_connections 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d6223d80 0x5623d661d340 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6616a60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d660fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 --2- 172.21.15.94:0/2924557411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d660f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8702ffd640 0 client.30940 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.107+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 >> 172.21.15.94:0/2924557411 conn(0x5623d6259300 msgr2=0x5623d6600250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.108+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 shutdown_connections 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.108+0000 7f8730e55740 1 -- 172.21.15.94:0/2924557411 wait complete. 2024-04-25T13:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.108+0000 7f8730e55740 20 client.30940 trim_cache size 0 max 16384 2024-04-25T13:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 Processor -- start 2024-04-25T13:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 -- start start 2024-04-25T13:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d6616980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6604100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87241ad840 con 0x5623d623c2f0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87241af380 con 0x5623d62444d0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8730e55740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87241adf10 con 0x5623d62651b0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f87237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f87237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52894/0 (socket says 172.21.15.94:52894) 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f87237fe640 1 -- 172.21.15.94:0/511214285 learned_addr learned my addr 172.21.15.94:0/511214285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8723fff640 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6604100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.124+0000 7f8722ffd640 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d6616980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 -- 172.21.15.94:0/511214285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 msgr2=0x5623d6604100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6604100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 -- 172.21.15.94:0/511214285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 msgr2=0x5623d6616980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d6616980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 -- 172.21.15.94:0/511214285 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d660a530 con 0x5623d623c2f0 2024-04-25T13:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87237fe640 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f87140a7210 tx=0x7f8714014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f257d573016157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87097fa640 1 -- 172.21.15.94:0/511214285 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140a7550 con 0x5623d623c2f0 2024-04-25T13:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87097fa640 1 -- 172.21.15.94:0/511214285 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140a7020 con 0x5623d623c2f0 2024-04-25T13:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.125+0000 7f87097fa640 1 -- 172.21.15.94:0/511214285 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140afee0 con 0x5623d623c2f0 2024-04-25T13:59:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- 172.21.15.94:0/511214285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 msgr2=0x5623d66012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f87140a7210 tx=0x7f8714014000 comp rx=0 tx=0).stop 2024-04-25T13:59:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- 172.21.15.94:0/511214285 shutdown_connections 2024-04-25T13:59:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d62651b0 0x5623d6604100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62444d0 0x5623d6616980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 --2- 172.21.15.94:0/511214285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d623c2f0 0x5623d66012f0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- 172.21.15.94:0/511214285 >> 172.21.15.94:0/511214285 conn(0x5623d6259300 msgr2=0x5623d6600be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- 172.21.15.94:0/511214285 shutdown_connections 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- 172.21.15.94:0/511214285 wait complete. 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.126+0000 7f8730e55740 1 -- start start 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 Processor -- start 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6613fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d66153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87241aecb0 con 0x5623d62444d0 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87241ae5e0 con 0x5623d62651b0 2024-04-25T13:59:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87241ab820 con 0x5623d623c2f0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8730e55740 10 client.0 _ll_register_callbacks cb 0x7f871c011980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f87237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6613fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52900/0 (socket says 172.21.15.94:52900) 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 -- 172.21.15.94:0/36330909 learned_addr learned my addr 172.21.15.94:0/36330909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8723fff640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d66153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 -- 172.21.15.94:0/36330909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 msgr2=0x5623d6613fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6613fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 -- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 msgr2=0x5623d66153e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d66153e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f8722ffd640 1 -- 172.21.15.94:0/36330909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d6616ec0 con 0x5623d62444d0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.127+0000 7f87237fe640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6613fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8722ffd640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f8718031440 tx=0x7f87180317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d17a2886103313a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871804a060 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8718004450 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87140a7020 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8718035040 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8730e55740 10 client.21855 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623d6616b00 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140a7020 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.128+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f871803c5f0 con 0x5623d62444d0 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 10 client.21855 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 10 client.21855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 20 client.21855 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d61f91c0 0x5623d6687d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 4 client.21855 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623d64ee5f0 con 0x5623d61f91c0 2024-04-25T13:59:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f87237fe640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d61f91c0 0x5623d6687d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f87237fe640 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d61f91c0 0x5623d6687d50 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c25ae150af9c1a7 server_cookie=3af606ef70ab4f82 in_seq=0 out_seq=0 2024-04-25T13:59:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.129+0000 7f8702ffd640 10 client.21855 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:26.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:25 smithi137 ceph-mon[32568]: pgmap v880: 97 pgs: 97 active+clean; 260 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.4 MiB/s wr, 659 op/s 2024-04-25T13:59:26.142 INFO:tasks.workunit.client.0.smithi094.stdout: 10 90696 19.44 MB/sec execute 128 sec latency 238.757 ms 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.788+0000 7f8702ffd640 1 -- 172.21.15.94:0/36330909 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623d64ee5f0 con 0x5623d61f91c0 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.788+0000 7f8702ffd640 10 client.21855 handle_command_reply: tid=0 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.788+0000 7f8730e55740 1 client.21855 shutdown 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 msgr2=0x5623d6614520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f8718031440 tx=0x7f87180317d0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 shutdown_connections 2024-04-25T13:59:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5623d61f91c0 0x5623d6687d50 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8702ffd640 0 client.21855 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623d62651b0 0x5623d66153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623d62444d0 0x5623d6614520 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 --2- 172.21.15.94:0/36330909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623d623c2f0 0x5623d6613fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 >> 172.21.15.94:0/36330909 conn(0x5623d6259300 msgr2=0x5623d65ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 shutdown_connections 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 1 -- 172.21.15.94:0/36330909 wait complete. 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.789+0000 7f8730e55740 20 client.21855 trim_cache size 0 max 16384 2024-04-25T13:59:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:59:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:59:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:59:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:59:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 msgr2=0x7f870c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 0x7f870c0b1ff0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x5623d6606b40 tx=0x7f871c009500 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 msgr2=0x7f8724103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f871400e9c0 tx=0x7f871400ee80 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 shutdown_connections 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f870c0afae0 0x7f870c0b1ff0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241098d0 0x7f8724102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8724106f70 0x7f8724102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 --2- 172.21.15.94:0/248571988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8724104610 0x7f8724103a60 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 >> 172.21.15.94:0/248571988 conn(0x7f87240fdc10 msgr2=0x7f8724105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 shutdown_connections 2024-04-25T13:59:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:26.790+0000 7f8729fa8640 1 -- 172.21.15.94:0/248571988 wait complete. 2024-04-25T13:59:26.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:59:26.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:59:26.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:59:26.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:59:26.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T13:59:26.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:59:26.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:59:26.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:59:26.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:59:26.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T13:59:27.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:59:27.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:59:27.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:59:27.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:59:27.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T13:59:27.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 90816 19.33 MB/sec execute 129 sec latency 420.435 ms 2024-04-25T13:59:27.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: pgmap v881: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 408 op/s 2024-04-25T13:59:27.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: pgmap v881: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 408 op/s 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:27.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:27.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:59:28.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: pgmap v881: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 408 op/s 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:28.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T13:59:28.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 91155 19.33 MB/sec execute 130 sec latency 89.813 ms 2024-04-25T13:59:28.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:28 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:59:28.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:28 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:59:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:28 smithi094 ceph-mon[29195]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:59:28.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:28 smithi094 ceph-mon[29195]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:59:28.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:28 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:59:28.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:28 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:59:28.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:28 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:59:28.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:28 smithi138 ceph-mon[32977]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:59:29.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:28 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T13:59:29.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:28 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi138 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-25T13:59:29.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:28 smithi137 ceph-mon[32568]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T13:59:29.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:28 smithi137 ceph-mon[32568]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T13:59:29.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 91518 19.28 MB/sec execute 131 sec latency 79.940 ms 2024-04-25T13:59:29.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:29 smithi094 ceph-mon[29195]: pgmap v882: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-04-25T13:59:29.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:29 smithi138 ceph-mon[32977]: pgmap v882: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-04-25T13:59:30.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:29 smithi137 ceph-mon[32568]: pgmap v882: 97 pgs: 97 active+clean; 259 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-04-25T13:59:30.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 91862 19.28 MB/sec execute 132 sec latency 82.986 ms 2024-04-25T13:59:31.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 92161 19.28 MB/sec execute 133 sec latency 275.280 ms 2024-04-25T13:59:31.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:31 smithi094 ceph-mon[29195]: pgmap v883: 97 pgs: 97 active+clean; 247 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-04-25T13:59:31.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:31 smithi138 ceph-mon[32977]: pgmap v883: 97 pgs: 97 active+clean; 247 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-04-25T13:59:32.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:31 smithi137 ceph-mon[32568]: pgmap v883: 97 pgs: 97 active+clean; 247 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 640 op/s 2024-04-25T13:59:32.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 92572 19.31 MB/sec execute 134 sec latency 86.686 ms 2024-04-25T13:59:33.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 92995 19.32 MB/sec execute 135 sec latency 78.298 ms 2024-04-25T13:59:33.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:33 smithi094 ceph-mon[29195]: pgmap v884: 97 pgs: 97 active+clean; 246 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-04-25T13:59:33.998 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:33 smithi138 ceph-mon[32977]: pgmap v884: 97 pgs: 97 active+clean; 246 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-04-25T13:59:34.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:33 smithi137 ceph-mon[32568]: pgmap v884: 97 pgs: 97 active+clean; 246 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-04-25T13:59:34.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 93381 19.33 MB/sec execute 136 sec latency 90.213 ms 2024-04-25T13:59:35.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 93772 19.38 MB/sec execute 137 sec latency 105.994 ms 2024-04-25T13:59:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:35 smithi094 ceph-mon[29195]: pgmap v885: 97 pgs: 97 active+clean; 250 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2024-04-25T13:59:35.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:35 smithi138 ceph-mon[32977]: pgmap v885: 97 pgs: 97 active+clean; 250 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2024-04-25T13:59:36.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:35 smithi137 ceph-mon[32568]: pgmap v885: 97 pgs: 97 active+clean; 250 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 617 op/s 2024-04-25T13:59:36.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 94045 19.34 MB/sec execute 138 sec latency 355.561 ms 2024-04-25T13:59:37.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 94362 19.31 MB/sec execute 139 sec latency 368.271 ms 2024-04-25T13:59:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:37 smithi094 ceph-mon[29195]: pgmap v886: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T13:59:37.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:37 smithi138 ceph-mon[32977]: pgmap v886: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T13:59:38.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:37 smithi137 ceph-mon[32568]: pgmap v886: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T13:59:38.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 94717 19.30 MB/sec execute 140 sec latency 180.328 ms 2024-04-25T13:59:39.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 95134 19.32 MB/sec execute 141 sec latency 85.315 ms 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 -- 172.21.15.94:0/922321655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0102a50 msgr2=0x7f90b0103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 --2- 172.21.15.94:0/922321655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0102a50 0x7f90b0103750 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009ca0 tx=0x7f90a002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 -- 172.21.15.94:0/922321655 shutdown_connections 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 --2- 172.21.15.94:0/922321655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0108990 0x7f90b010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 --2- 172.21.15.94:0/922321655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0103c90 0x7f90b01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 --2- 172.21.15.94:0/922321655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0102a50 0x7f90b0103750 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 -- 172.21.15.94:0/922321655 >> 172.21.15.94:0/922321655 conn(0x7f90b00fdc60 msgr2=0x7f90b01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 -- 172.21.15.94:0/922321655 shutdown_connections 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.901+0000 7f90b4d46640 1 -- 172.21.15.94:0/922321655 wait complete. 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 Processor -- start 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 -- start start 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0108990 0x7f90b01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010e550 con 0x7f90b0102a50 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010e3d0 con 0x7f90b0108990 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90b4d46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010e6d0 con 0x7f90b0103c90 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43012/0 (socket says 172.21.15.94:43012) 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:36360/0 (socket says 172.21.15.94:36360) 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90ae59c640 1 -- 172.21.15.94:0/9650856 learned_addr learned my addr 172.21.15.94:0/9650856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90aed9d640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0108990 0x7f90b01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 -- 172.21.15.94:0/9650856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0108990 msgr2=0x7f90b01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0108990 0x7f90b01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 -- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 msgr2=0x7f90b0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.902+0000 7f90a7fff640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90a7fff640 1 -- 172.21.15.94:0/9650856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b01a0cb0 con 0x7f90b0103c90 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90ae59c640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90a7fff640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9f0 tx=0x7f909800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90a77fe640 1 -- 172.21.15.94:0/9650856 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800ccc0 con 0x7f90b0103c90 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90a77fe640 1 -- 172.21.15.94:0/9650856 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9098005020 con 0x7f90b0103c90 2024-04-25T13:59:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b01a0fa0 con 0x7f90b0103c90 2024-04-25T13:59:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90a77fe640 1 -- 172.21.15.94:0/9650856 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004a00 con 0x7f90b0103c90 2024-04-25T13:59:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.903+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01a13b0 con 0x7f90b0103c90 2024-04-25T13:59:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 Processor -- start 2024-04-25T13:59:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 -- start start 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x555720012b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55571ff76460 con 0x55571fc472f0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55571ff30590 con 0x55571fc701b0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55571ff72420 con 0x55571fc4f4d0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90a77fe640 1 -- 172.21.15.94:0/9650856 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9098004ba0 con 0x7f90b0103c90 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43020/0 (socket says 172.21.15.94:43020) 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x555720012b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90ae59c640 1 -- 172.21.15.94:0/2578351558 learned_addr learned my addr 172.21.15.94:0/2578351558 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.906+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 -- 172.21.15.94:0/2578351558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 msgr2=0x55572001a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 -- 172.21.15.94:0/2578351558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 msgr2=0x555720012b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x555720012b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 -- 172.21.15.94:0/2578351558 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571ff3d0d0 con 0x55571fc472f0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90aed9d640 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x555720012b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004ab0 tx=0x7f90a002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6def0ce51d2ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90a77fe640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 0x7f909c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90a4ff9640 1 -- 172.21.15.94:0/2578351558 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0031450 con 0x55571fc472f0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90a77fe640 1 -- 172.21.15.94:0/9650856 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90980dec50 con 0x7f90b0103c90 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90aed9d640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 0x7f909c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90a4ff9640 1 -- 172.21.15.94:0/2578351558 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a0036040 con 0x55571fc472f0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.907+0000 7f90a4ff9640 1 -- 172.21.15.94:0/2578351558 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00045f0 con 0x55571fc472f0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.908+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2578351558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 msgr2=0x555720013100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.908+0000 7f90aed9d640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 0x7f909c0b1ff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x555720013d50 tx=0x7f90a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.908+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004ab0 tx=0x7f90a002f310 comp rx=0 tx=0).stop 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2578351558 shutdown_connections 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x555720013100 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2578351558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x555720012b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2578351558 >> 172.21.15.94:0/2578351558 conn(0x55571fc64300 msgr2=0x55572000c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2578351558 shutdown_connections 2024-04-25T13:59:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2578351558 wait complete. 2024-04-25T13:59:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 -- start start 2024-04-25T13:59:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.909+0000 7f90bbbf6740 1 Processor -- start 2024-04-25T13:59:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555720010580 con 0x55571fc472f0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55571fefdb60 con 0x55571fc701b0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555720010700 con 0x55571fc4f4d0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90bbbf6740 10 client.0 _ll_register_callbacks cb 0x55571ff222b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43022/0 (socket says 172.21.15.94:43022) 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47334/0 (socket says 172.21.15.94:47334) 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90ae59c640 1 -- 172.21.15.94:0/2964764532 learned_addr learned my addr 172.21.15.94:0/2964764532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 -- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 msgr2=0x5557200218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 -- 172.21.15.94:0/2964764532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 msgr2=0x55572001aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 -- 172.21.15.94:0/2964764532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571fc88210 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001afe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90980dc700 tx=0x7f9098009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc0dd3020f23e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.910+0000 7f9095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909801c070 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90980054f0 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a0036040 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90980a69e0 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 10 client.30958 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55571fd54620 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a0036040 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f90980ab0b0 con 0x55571fc4f4d0 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 10 client.30958 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 10 client.30958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 20 client.30958 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc2ed80 0x555720028140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 4 client.30958 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55571ff84070 con 0x55571fc2ed80 2024-04-25T13:59:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.911+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc2ed80 0x555720028140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.912+0000 7f9095ffb640 10 client.30958 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.912+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc2ed80 0x555720028140 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4137835e426d574 server_cookie=99ae186902d33288 in_seq=0 out_seq=0 2024-04-25T13:59:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f9095ffb640 1 -- 172.21.15.94:0/2964764532 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55571ff84070 con 0x55571fc2ed80 2024-04-25T13:59:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f9095ffb640 10 client.30958 handle_command_reply: tid=0 2024-04-25T13:59:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 client.30958 shutdown 2024-04-25T13:59:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 msgr2=0x55572001afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001afe0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90980dc700 tx=0x7f9098009f60 comp rx=0 tx=0).stop 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 shutdown_connections 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc2ed80 0x555720028140 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f9095ffb640 0 client.30958 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001afe0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2964764532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 >> 172.21.15.94:0/2964764532 conn(0x55571fc64300 msgr2=0x55572000f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 shutdown_connections 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.915+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2964764532 wait complete. 2024-04-25T13:59:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.916+0000 7f90bbbf6740 20 client.30958 trim_cache size 0 max 16384 2024-04-25T13:59:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 Processor -- start 2024-04-25T13:59:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 -- start start 2024-04-25T13:59:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 0x55572000c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc4f4d0 0x555720021880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01ad6f0 con 0x55571fc701b0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01af230 con 0x55571fc4f4d0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01addc0 con 0x55571fc472f0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43032/0 (socket says 172.21.15.94:43032) 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 -- 172.21.15.94:0/4129731865 learned_addr learned my addr 172.21.15.94:0/4129731865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90a7fff640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc4f4d0 0x555720021880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90ae59c640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 0x55572000c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 -- 172.21.15.94:0/4129731865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 msgr2=0x55572000c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 0x55572000c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 -- 172.21.15.94:0/4129731865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc4f4d0 msgr2=0x555720021880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc4f4d0 0x555720021880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90aed9d640 1 -- 172.21.15.94:0/4129731865 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557200184e0 con 0x55571fc701b0 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.936+0000 7f90ae59c640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 0x55572000c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.937+0000 7f90aed9d640 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f90a8004380 tx=0x7f90a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a59773045dc5d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.937+0000 7f9097fff640 1 -- 172.21.15.94:0/4129731865 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8003a70 con 0x55571fc701b0 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.937+0000 7f9097fff640 1 -- 172.21.15.94:0/4129731865 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a802d070 con 0x55571fc701b0 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.937+0000 7f9097fff640 1 -- 172.21.15.94:0/4129731865 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8028400 con 0x55571fc701b0 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- 172.21.15.94:0/4129731865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 msgr2=0x555720013ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f90a8004380 tx=0x7f90a8017000 comp rx=0 tx=0).stop 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- 172.21.15.94:0/4129731865 shutdown_connections 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc701b0 0x555720013ef0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc4f4d0 0x555720021880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/4129731865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc472f0 0x55572000c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- 172.21.15.94:0/4129731865 >> 172.21.15.94:0/4129731865 conn(0x55571fc64300 msgr2=0x55572000bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- 172.21.15.94:0/4129731865 shutdown_connections 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- 172.21.15.94:0/4129731865 wait complete. 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- start start 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 Processor -- start 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01aeb60 con 0x55571fc472f0 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01ae490 con 0x55571fc701b0 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01ab6d0 con 0x55571fc4f4d0 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90bbbf6740 10 client.0 _ll_register_callbacks cb 0x7f90980e2420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.938+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43034/0 (socket says 172.21.15.94:43034) 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 -- 172.21.15.94:0/2662202646 learned_addr learned my addr 172.21.15.94:0/2662202646 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90aed9d640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 -- 172.21.15.94:0/2662202646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 msgr2=0x55572001d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 -- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 msgr2=0x5557200204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 -- 172.21.15.94:0/2662202646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555720020aa0 con 0x55571fc472f0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90ae59c640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f90a00094e0 tx=0x7f90a0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dcef8fe275ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f9095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00316e0 con 0x55571fc472f0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90a0003970 con 0x55571fc472f0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a802d070 con 0x55571fc472f0 2024-04-25T13:59:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90a0003b70 con 0x55571fc472f0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90bbbf6740 10 client.21873 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5557200904c0 con 0x55571fc472f0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.939+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a802d070 con 0x55571fc472f0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f90a0044070 con 0x55571fc472f0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 10 client.21873 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 10 client.21873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 20 client.21873 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc041c0 0x555720092bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 4 client.21873 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55571ff7dee0 con 0x55571fc041c0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc041c0 0x555720092bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f90a7fff640 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc041c0 0x555720092bd0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f210b8be8a9e77 server_cookie=96aa32b3a526e879 in_seq=0 out_seq=0 2024-04-25T13:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:39.940+0000 7f9095ffb640 10 client.21873 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:39.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:39 smithi094 ceph-mon[29195]: pgmap v887: 97 pgs: 97 active+clean; 254 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-04-25T13:59:39.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:39 smithi138 ceph-mon[32977]: pgmap v887: 97 pgs: 97 active+clean; 254 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-04-25T13:59:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:39 smithi137 ceph-mon[32568]: pgmap v887: 97 pgs: 97 active+clean; 254 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 506 op/s 2024-04-25T13:59:40.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 95428 19.27 MB/sec execute 142 sec latency 183.534 ms 2024-04-25T13:59:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.476+0000 7f9095ffb640 1 -- 172.21.15.94:0/2662202646 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55571ff7dee0 con 0x55571fc041c0 2024-04-25T13:59:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.476+0000 7f9095ffb640 10 client.21873 handle_command_reply: tid=0 2024-04-25T13:59:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.476+0000 7f90bbbf6740 1 client.21873 shutdown 2024-04-25T13:59:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 msgr2=0x55572001d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f90a00094e0 tx=0x7f90a0005bf0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 shutdown_connections 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55571fc041c0 0x555720092bd0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55571fc701b0 0x5557200204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f9095ffb640 0 client.21873 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571fc4f4d0 0x55572001d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 --2- 172.21.15.94:0/2662202646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55571fc472f0 0x55572001d0b0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 >> 172.21.15.94:0/2662202646 conn(0x55571fc64300 msgr2=0x55572000f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 shutdown_connections 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 1 -- 172.21.15.94:0/2662202646 wait complete. 2024-04-25T13:59:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.477+0000 7f90bbbf6740 20 client.21873 trim_cache size 0 max 16384 2024-04-25T13:59:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:59:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:59:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:59:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 msgr2=0x7f909c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 0x7f909c0b1ff0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x555720013d50 tx=0x7f90a8005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 msgr2=0x7f90b01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9f0 tx=0x7f909800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 shutdown_connections 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f909c0afae0 0x7f909c0b1ff0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90b0108990 0x7f90b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0103c90 0x7f90b01996b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 --2- 172.21.15.94:0/9650856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90b0102a50 0x7f90b0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 >> 172.21.15.94:0/9650856 conn(0x7f90b00fdc60 msgr2=0x7f90b0104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 shutdown_connections 2024-04-25T13:59:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:40.478+0000 7f90b4d46640 1 -- 172.21.15.94:0/9650856 wait complete. 2024-04-25T13:59:41.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 95669 19.25 MB/sec execute 143 sec latency 376.567 ms 2024-04-25T13:59:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:41 smithi094 ceph-mon[29195]: pgmap v888: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 590 op/s 2024-04-25T13:59:41.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:41 smithi138 ceph-mon[32977]: pgmap v888: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 590 op/s 2024-04-25T13:59:42.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:41 smithi137 ceph-mon[32568]: pgmap v888: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 590 op/s 2024-04-25T13:59:42.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 96108 19.29 MB/sec execute 144 sec latency 63.762 ms 2024-04-25T13:59:43.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 96530 19.30 MB/sec execute 145 sec latency 76.225 ms 2024-04-25T13:59:43.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:43 smithi094 ceph-mon[29195]: pgmap v889: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 343 op/s 2024-04-25T13:59:43.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:43 smithi138 ceph-mon[32977]: pgmap v889: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 343 op/s 2024-04-25T13:59:44.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:43 smithi137 ceph-mon[32568]: pgmap v889: 97 pgs: 97 active+clean; 254 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 343 op/s 2024-04-25T13:59:44.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 96940 19.32 MB/sec execute 146 sec latency 78.539 ms 2024-04-25T13:59:45.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 97358 19.40 MB/sec execute 147 sec latency 61.580 ms 2024-04-25T13:59:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:45 smithi094 ceph-mon[29195]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-04-25T13:59:45.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:45 smithi138 ceph-mon[32977]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-04-25T13:59:46.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:45 smithi137 ceph-mon[32568]: pgmap v890: 97 pgs: 97 active+clean; 252 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.3 MiB/s wr, 584 op/s 2024-04-25T13:59:46.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 97682 19.35 MB/sec execute 148 sec latency 148.783 ms 2024-04-25T13:59:47.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 97947 19.31 MB/sec execute 149 sec latency 291.723 ms 2024-04-25T13:59:48.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:47 smithi137 ceph-mon[32568]: pgmap v891: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 518 op/s 2024-04-25T13:59:48.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 98301 19.32 MB/sec execute 150 sec latency 80.207 ms 2024-04-25T13:59:48.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:47 smithi094 ceph-mon[29195]: pgmap v891: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 518 op/s 2024-04-25T13:59:48.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:47 smithi138 ceph-mon[32977]: pgmap v891: 97 pgs: 97 active+clean; 248 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 518 op/s 2024-04-25T13:59:49.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 98697 19.31 MB/sec execute 151 sec latency 68.758 ms 2024-04-25T13:59:49.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:49.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:49.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 -- 172.21.15.94:0/1814779830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 msgr2=0x7fa69410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 --2- 172.21.15.94:0/1814779830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa69410ad40 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009ca0 tx=0x7fa68c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 -- 172.21.15.94:0/1814779830 shutdown_connections 2024-04-25T13:59:49.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 --2- 172.21.15.94:0/1814779830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa69410ad40 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 --2- 172.21.15.94:0/1814779830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa694103c20 0x7fa694104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 --2- 172.21.15.94:0/1814779830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6941029e0 0x7fa6941036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.873+0000 7fa699906640 1 -- 172.21.15.94:0/1814779830 >> 172.21.15.94:0/1814779830 conn(0x7fa6940fdbd0 msgr2=0x7fa694100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.874+0000 7fa699906640 1 -- 172.21.15.94:0/1814779830 shutdown_connections 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.874+0000 7fa699906640 1 -- 172.21.15.94:0/1814779830 wait complete. 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.874+0000 7fa699906640 1 Processor -- start 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.874+0000 7fa699906640 1 -- start start 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6941029e0 0x7fa69419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa694103c20 0x7fa69419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410e510 con 0x7fa694108920 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410e390 con 0x7fa694103c20 2024-04-25T13:59:49.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410e690 con 0x7fa6941029e0 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50958/0 (socket says 172.21.15.94:50958) 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 -- 172.21.15.94:0/3921558115 learned_addr learned my addr 172.21.15.94:0/3921558115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa692ffd640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6941029e0 0x7fa69419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa694103c20 0x7fa69419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 -- 172.21.15.94:0/3921558115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6941029e0 msgr2=0x7fa69419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6941029e0 0x7fa69419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 -- 172.21.15.94:0/3921558115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa694103c20 msgr2=0x7fa69419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa694103c20 0x7fa69419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 -- 172.21.15.94:0/3921558115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6941a2030 con 0x7fa694108920 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa6937fe640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fa68c004af0 tx=0x7fa68c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa698904640 1 -- 172.21.15.94:0/3921558115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c003b70 con 0x7fa694108920 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa698904640 1 -- 172.21.15.94:0/3921558115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa68c003d10 con 0x7fa694108920 2024-04-25T13:59:49.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6941a22c0 con 0x7fa694108920 2024-04-25T13:59:49.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa698904640 1 -- 172.21.15.94:0/3921558115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c02fb20 con 0x7fa694108920 2024-04-25T13:59:49.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.875+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941a2810 con 0x7fa694108920 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa698904640 1 -- 172.21.15.94:0/3921558115 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa68c004580 con 0x7fa694108920 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 Processor -- start 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 -- start start 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fd2a84d80 con 0x559fd27554d0 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fd2a3e8b0 con 0x559fd275d6b0 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fd2a80d40 con 0x559fd277e390 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.878+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49060/0 (socket says 172.21.15.94:49060) 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 -- 172.21.15.94:0/2058543297 learned_addr learned my addr 172.21.15.94:0/2058543297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa698904640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 0x7fa6800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 -- 172.21.15.94:0/2058543297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 msgr2=0x559fd2b25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 -- 172.21.15.94:0/2058543297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 msgr2=0x559fd2b243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 -- 172.21.15.94:0/2058543297 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd2a0fa40 con 0x559fd277e390 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa6937fe640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 0x7fa6800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa698904640 1 -- 172.21.15.94:0/3921558115 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa68c100290 con 0x7fa694108920 2024-04-25T13:59:49.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa67c011810 tx=0x7fa67c0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ef955138ab5152 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.879+0000 7fa6937fe640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 0x7fa6800b2000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0c8660 tx=0x7fa68c04a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.882+0000 7fa689ffb640 1 -- 172.21.15.94:0/2058543297 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c004040 con 0x559fd277e390 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.882+0000 7fa689ffb640 1 -- 172.21.15.94:0/2058543297 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c011e50 con 0x559fd277e390 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.882+0000 7fa689ffb640 1 -- 172.21.15.94:0/2058543297 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0044e0 con 0x559fd277e390 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.882+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2058543297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 msgr2=0x559fd2b23e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.882+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa67c011810 tx=0x7fa67c0097d0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2058543297 shutdown_connections 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2058543297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b23e70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2058543297 >> 172.21.15.94:0/2058543297 conn(0x559fd27724e0 msgr2=0x559fd2b1b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2058543297 shutdown_connections 2024-04-25T13:59:49.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2058543297 wait complete. 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 -- start start 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.883+0000 7fa6a07bc740 1 Processor -- start 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd27554d0 0x559fd2b2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559fd2b25b90 con 0x559fd275d6b0 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559fd2a0bd10 con 0x559fd27554d0 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fd2b25d10 con 0x559fd277e390 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 10 client.0 _ll_register_callbacks cb 0x559fd2a38210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50976/0 (socket says 172.21.15.94:50976) 2024-04-25T13:59:49.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 -- 172.21.15.94:0/2661645255 learned_addr learned my addr 172.21.15.94:0/2661645255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd27554d0 0x559fd2b2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 -- 172.21.15.94:0/2661645255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 msgr2=0x559fd2b35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 -- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd27554d0 msgr2=0x559fd2b2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd27554d0 0x559fd2b2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 -- 172.21.15.94:0/2661645255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd28607d0 con 0x559fd275d6b0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa692ffd640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fa684012700 tx=0x7fa684012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844163dfb104904f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa684004050 con 0x559fd275d6b0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa6840041f0 con 0x559fd275d6b0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.884+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c011e50 con 0x559fd275d6b0 2024-04-25T13:59:49.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa6840043f0 con 0x559fd275d6b0 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 10 client.21885 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559fd27963f0 con 0x559fd275d6b0 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c011e50 con 0x559fd275d6b0 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa684003ae0 con 0x559fd275d6b0 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 10 client.21885 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 10 client.21885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 20 client.21885 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd273cf60 0x559fd2b3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 4 client.21885 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:49.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.885+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fd2a92990 con 0x559fd273cf60 2024-04-25T13:59:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.886+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd273cf60 0x559fd2b3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.886+0000 7fa672ffd640 10 client.21885 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:49.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.886+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd273cf60 0x559fd2b3b740 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c957259bcb72d076 server_cookie=6300c9ee8591183d in_seq=0 out_seq=0 2024-04-25T13:59:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.889+0000 7fa672ffd640 1 -- 172.21.15.94:0/2661645255 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fd2a92990 con 0x559fd273cf60 2024-04-25T13:59:49.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.889+0000 7fa672ffd640 10 client.21885 handle_command_reply: tid=0 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.889+0000 7fa6a07bc740 1 client.21885 shutdown 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 msgr2=0x559fd2b2e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fa684012700 tx=0x7fa684012bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 shutdown_connections 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd273cf60 0x559fd2b3b740 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd277e390 0x559fd2b35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd275d6b0 0x559fd2b2e2f0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/2661645255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd27554d0 0x559fd2b2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 >> 172.21.15.94:0/2661645255 conn(0x559fd27724e0 msgr2=0x559fd2b1e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.890+0000 7fa672ffd640 0 client.21885 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.891+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 shutdown_connections 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.891+0000 7fa6a07bc740 1 -- 172.21.15.94:0/2661645255 wait complete. 2024-04-25T13:59:49.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.891+0000 7fa6a07bc740 20 client.21885 trim_cache size 0 max 16384 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 Processor -- start 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 -- start start 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd27554d0 0x559fd2b1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941a92d0 con 0x559fd277e390 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941aae10 con 0x559fd275d6b0 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941a99a0 con 0x559fd27554d0 2024-04-25T13:59:49.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50982/0 (socket says 172.21.15.94:50982) 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 -- 172.21.15.94:0/3019131176 learned_addr learned my addr 172.21.15.94:0/3019131176 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa692ffd640 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b1b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd27554d0 0x559fd2b1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 -- 172.21.15.94:0/3019131176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd27554d0 msgr2=0x559fd2b1aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd27554d0 0x559fd2b1aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 -- 172.21.15.94:0/3019131176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 msgr2=0x559fd2b1b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b1b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.911+0000 7fa68a7fc640 1 -- 172.21.15.94:0/3019131176 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd27560f0 con 0x559fd277e390 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.912+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009810 tx=0x7fa67c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd5e38c49481f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.912+0000 7fa6897fa640 1 -- 172.21.15.94:0/3019131176 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c004230 con 0x559fd277e390 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.912+0000 7fa6897fa640 1 -- 172.21.15.94:0/3019131176 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c0308b0 con 0x559fd277e390 2024-04-25T13:59:49.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.912+0000 7fa6897fa640 1 -- 172.21.15.94:0/3019131176 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00f9f0 con 0x559fd277e390 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- 172.21.15.94:0/3019131176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 msgr2=0x559fd2b2c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009810 tx=0x7fa67c009620 comp rx=0 tx=0).stop 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- 172.21.15.94:0/3019131176 shutdown_connections 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd277e390 0x559fd2b2c380 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd275d6b0 0x559fd2b1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/3019131176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd27554d0 0x559fd2b1aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- 172.21.15.94:0/3019131176 >> 172.21.15.94:0/3019131176 conn(0x559fd27724e0 msgr2=0x559fd2b28a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- 172.21.15.94:0/3019131176 shutdown_connections 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- 172.21.15.94:0/3019131176 wait complete. 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- start start 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 Processor -- start 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd275d6b0 0x559fd2b30fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd277e390 0x559fd2b31510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941aa740 con 0x559fd27554d0 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.913+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941aa070 con 0x559fd277e390 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa6a07bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6941a72b0 con 0x559fd275d6b0 2024-04-25T13:59:49.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa6a07bc740 10 client.0 _ll_register_callbacks cb 0x7fa684009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50996/0 (socket says 172.21.15.94:50996) 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 -- 172.21.15.94:0/1506056738 learned_addr learned my addr 172.21.15.94:0/1506056738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa692ffd640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd275d6b0 0x559fd2b30fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd277e390 0x559fd2b31510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 -- 172.21.15.94:0/1506056738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd275d6b0 msgr2=0x559fd2b30fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd275d6b0 0x559fd2b30fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 -- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd277e390 msgr2=0x559fd2b31510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd277e390 0x559fd2b31510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 -- 172.21.15.94:0/1506056738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd2a454b0 con 0x559fd27554d0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa68a7fc640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009710 tx=0x7fa67c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b155e269b5b23216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c030400 con 0x559fd27554d0 2024-04-25T13:59:49.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa67c029040 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa67c0308b0 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa67c0305a0 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa6a07bc740 10 client.21891 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.914+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559fd2ba2790 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c0308b0 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa67c03c040 con 0x559fd27554d0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 10 client.21891 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 10 client.21891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 20 client.21891 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd27123a0 0x559fd2ba4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 4 client.21891 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fd2a8c800 con 0x559fd27123a0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa692ffd640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd27123a0 0x559fd2ba4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa692ffd640 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd27123a0 0x559fd2ba4f90 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155c77ec7e19076d server_cookie=796ec18c47109ceb in_seq=0 out_seq=0 2024-04-25T13:59:49.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.915+0000 7fa672ffd640 10 client.21891 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa672ffd640 1 -- 172.21.15.94:0/1506056738 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559fd2a8c800 con 0x559fd27123a0 2024-04-25T13:59:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa672ffd640 10 client.21891 handle_command_reply: tid=0 2024-04-25T13:59:49.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 client.21891 shutdown 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 msgr2=0x559fd2b34450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009710 tx=0x7fa67c009b70 comp rx=0 tx=0).stop 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 shutdown_connections 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559fd27123a0 0x559fd2ba4f90 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd277e390 0x559fd2b31510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa672ffd640 0 client.21891 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd275d6b0 0x559fd2b30fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 --2- 172.21.15.94:0/1506056738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559fd27554d0 0x559fd2b34450 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 >> 172.21.15.94:0/1506056738 conn(0x559fd27724e0 msgr2=0x559fd2b1ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 shutdown_connections 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 1 -- 172.21.15.94:0/1506056738 wait complete. 2024-04-25T13:59:49.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.919+0000 7fa6a07bc740 20 client.21891 trim_cache size 0 max 16384 2024-04-25T13:59:49.924 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:49.924 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T13:59:49.924 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T13:59:49.924 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 msgr2=0x7fa6800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 0x7fa6800b2000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0c8660 tx=0x7fa68c04a000 comp rx=0 tx=0).stop 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 msgr2=0x7fa6941a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fa68c004af0 tx=0x7fa68c031700 comp rx=0 tx=0).stop 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 shutdown_connections 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6800afaf0 0x7fa6800b2000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa694108920 0x7fa6941a19c0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa694103c20 0x7fa69419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 --2- 172.21.15.94:0/3921558115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6941029e0 0x7fa69419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 >> 172.21.15.94:0/3921558115 conn(0x7fa6940fdbd0 msgr2=0x7fa694109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 shutdown_connections 2024-04-25T13:59:49.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:49.920+0000 7fa699906640 1 -- 172.21.15.94:0/3921558115 wait complete. 2024-04-25T13:59:49.931 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:59:49.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 -- 172.21.15.94:0/2707708112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80104d40 msgr2=0x7f8c80105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 --2- 172.21.15.94:0/2707708112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80104d40 0x7f8c80105150 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009900 tx=0x7f8c7402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 -- 172.21.15.94:0/2707708112 shutdown_connections 2024-04-25T13:59:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 --2- 172.21.15.94:0/2707708112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80109a40 0x7f8c8010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 --2- 172.21.15.94:0/2707708112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80104d40 0x7f8c80105150 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 --2- 172.21.15.94:0/2707708112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 0x7f8c80104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 -- 172.21.15.94:0/2707708112 >> 172.21.15.94:0/2707708112 conn(0x7f8c800fed10 msgr2=0x7f8c801011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 -- 172.21.15.94:0/2707708112 shutdown_connections 2024-04-25T13:59:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 -- 172.21.15.94:0/2707708112 wait complete. 2024-04-25T13:59:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.045+0000 7f8c86014640 1 Processor -- start 2024-04-25T13:59:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 -- start start 2024-04-25T13:59:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 0x7f8c8019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80104d40 0x7f8c8019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010f630 con 0x7f8c80109a40 2024-04-25T13:59:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010f4b0 con 0x7f8c80103b00 2024-04-25T13:59:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c86014640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010f7b0 con 0x7f8c80104d40 2024-04-25T13:59:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 0x7f8c8019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80104d40 0x7f8c8019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51020/0 (socket says 172.21.15.94:51020) 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 -- 172.21.15.94:0/1716678253 learned_addr learned my addr 172.21.15.94:0/1716678253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 -- 172.21.15.94:0/1716678253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80104d40 msgr2=0x7f8c8019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80104d40 0x7f8c8019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 -- 172.21.15.94:0/1716678253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 msgr2=0x7f8c8019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 0x7f8c8019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.046+0000 7f8c7ffff640 1 -- 172.21.15.94:0/1716678253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c801a1d70 con 0x7f8c80109a40 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c7ffff640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800eaa0 tx=0x7f8c7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800cdb0 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c78005020 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c801a2060 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780116d0 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.047+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801a25b0 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.048+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c80108850 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.049+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8c780051d0 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.050+0000 7f8c7cff9640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 0x7f8c680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.050+0000 7f8c7f7fe640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 0x7f8c680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.050+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8c780df080 con 0x7f8c80109a40 2024-04-25T13:59:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.050+0000 7f8c7f7fe640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 0x7f8c680b2000 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005fd0 tx=0x7f8c7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.054+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c78016030 con 0x7f8c80109a40 2024-04-25T13:59:50.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:49 smithi137 ceph-mon[32568]: pgmap v892: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T13:59:50.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 99076 19.30 MB/sec execute 152 sec latency 117.703 ms 2024-04-25T13:59:50.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:49 smithi094 ceph-mon[29195]: pgmap v892: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T13:59:50.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:49 smithi138 ceph-mon[32977]: pgmap v892: 97 pgs: 97 active+clean; 249 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T13:59:50.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.299+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c8002db50 con 0x7f8c80109a40 2024-04-25T13:59:50.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.300+0000 7f8c7cff9640 1 -- 172.21.15.94:0/1716678253 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f8c780a7610 con 0x7f8c80109a40 2024-04-25T13:59:50.300 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T13:59:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T13:59:50.301 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T13:59:50.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 msgr2=0x7f8c680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 0x7f8c680b2000 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005fd0 tx=0x7f8c7000b410 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 msgr2=0x7f8c801a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800eaa0 tx=0x7f8c7800ef60 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 shutdown_connections 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8c680afaf0 0x7f8c680b2000 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c80109a40 0x7f8c801a1700 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c80104d40 0x7f8c8019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 --2- 172.21.15.94:0/1716678253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c80103b00 0x7f8c8019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 >> 172.21.15.94:0/1716678253 conn(0x7f8c800fed10 msgr2=0x7f8c80105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.302+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 shutdown_connections 2024-04-25T13:59:50.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.303+0000 7f8c86014640 1 -- 172.21.15.94:0/1716678253 wait complete. 2024-04-25T13:59:50.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T13:59:50.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 -- 172.21.15.94:0/1128139723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 msgr2=0x7f83681047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 --2- 172.21.15.94:0/1128139723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f83681047f0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f8364009900 tx=0x7f836402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:50.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 -- 172.21.15.94:0/1128139723 shutdown_connections 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 --2- 172.21.15.94:0/1128139723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 0x7f836810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 --2- 172.21.15.94:0/1128139723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 0x7f8368105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 --2- 172.21.15.94:0/1128139723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f83681047f0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 -- 172.21.15.94:0/1128139723 >> 172.21.15.94:0/1128139723 conn(0x7f83680fed00 msgr2=0x7f8368101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 -- 172.21.15.94:0/1128139723 shutdown_connections 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.431+0000 7f8370061640 1 -- 172.21.15.94:0/1128139723 wait complete. 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 Processor -- start 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 -- start start 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 0x7f836819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 0x7f83681a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f836810f5f0 con 0x7f8368103af0 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f836810f470 con 0x7f8368109a30 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f8370061640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f836810f770 con 0x7f8368104d30 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51032/0 (socket says 172.21.15.94:51032) 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 -- 172.21.15.94:0/644150675 learned_addr learned my addr 172.21.15.94:0/644150675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836e5fe640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 0x7f83681a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836d5fc640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 0x7f836819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 -- 172.21.15.94:0/644150675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 msgr2=0x7f836819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 0x7f836819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 -- 172.21.15.94:0/644150675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 msgr2=0x7f83681a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 0x7f83681a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 -- 172.21.15.94:0/644150675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a1d90 con 0x7f8368103af0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.432+0000 7f836ddfd640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f8364006010 tx=0x7f8364004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.433+0000 7f835effd640 1 -- 172.21.15.94:0/644150675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836403f030 con 0x7f8368103af0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.433+0000 7f835effd640 1 -- 172.21.15.94:0/644150675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8364031910 con 0x7f8368103af0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.433+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a2020 con 0x7f8368103af0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.433+0000 7f835effd640 1 -- 172.21.15.94:0/644150675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364035070 con 0x7f8368103af0 2024-04-25T13:59:50.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.433+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a2510 con 0x7f8368103af0 2024-04-25T13:59:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 Processor -- start 2024-04-25T13:59:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 -- start start 2024-04-25T13:59:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f7a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f7ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc11eded80 con 0x55bc11bb76b0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc11e98a20 con 0x55bc11baf4d0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f8376feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc11edad40 con 0x55bc11bd8390 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f835effd640 1 -- 172.21.15.94:0/644150675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8364042030 con 0x7f8368103af0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51044/0 (socket says 172.21.15.94:51044) 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f836d5fc640 1 -- 172.21.15.94:0/2036391686 learned_addr learned my addr 172.21.15.94:0/2036391686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.436+0000 7f836e5fe640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f7a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836ddfd640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f7ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 -- 172.21.15.94:0/2036391686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 msgr2=0x55bc11f7a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f7a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 -- 172.21.15.94:0/2036391686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 msgr2=0x55bc11f7ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f7ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 -- 172.21.15.94:0/2036391686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc11ea3c30 con 0x55bc11bb76b0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836ddfd640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f7ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836d5fc640 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c50 tx=0x7f8358010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9757d047af2e059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f8353fff640 1 -- 172.21.15.94:0/2036391686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358005020 con 0x55bc11bb76b0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f8353fff640 1 -- 172.21.15.94:0/2036391686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83580047a0 con 0x55bc11bb76b0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f8353fff640 1 -- 172.21.15.94:0/2036391686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580126b0 con 0x55bc11bb76b0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f835effd640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 0x7f83540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f836e5fe640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 0x7f83540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.437+0000 7f835effd640 1 -- 172.21.15.94:0/644150675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f83640ff080 con 0x7f8368103af0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 -- 172.21.15.94:0/2036391686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 msgr2=0x55bc11f7ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f836e5fe640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 0x7f83540b2000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55bc11f7b860 tx=0x7f8360009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c50 tx=0x7f8358010740 comp rx=0 tx=0).stop 2024-04-25T13:59:50.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 -- 172.21.15.94:0/2036391686 shutdown_connections 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f7ba20 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f7ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 --2- 172.21.15.94:0/2036391686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f7a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 -- 172.21.15.94:0/2036391686 >> 172.21.15.94:0/2036391686 conn(0x55bc11bcc4e0 msgr2=0x55bc11f76020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 -- 172.21.15.94:0/2036391686 shutdown_connections 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.438+0000 7f8376feb740 1 -- 172.21.15.94:0/2036391686 wait complete. 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 -- start start 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 Processor -- start 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bb76b0 0x55bc11f84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc11f7c530 con 0x55bc11bd8390 2024-04-25T13:59:50.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bc11e65dc0 con 0x55bc11baf4d0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc11f7c6b0 con 0x55bc11bb76b0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.439+0000 7f8376feb740 10 client.0 _ll_register_callbacks cb 0x55bc11e81600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836ddfd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51048/0 (socket says 172.21.15.94:51048) 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 -- 172.21.15.94:0/3754877239 learned_addr learned my addr 172.21.15.94:0/3754877239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836d5fc640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bb76b0 0x55bc11f84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 -- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bb76b0 msgr2=0x55bc11f84b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bb76b0 0x55bc11f84b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 -- 172.21.15.94:0/3754877239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 msgr2=0x55bc11f845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 -- 172.21.15.94:0/3754877239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc11bf03f0 con 0x55bc11bd8390 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f836e5fe640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f8360009cd0 tx=0x7f8360004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6b14a94c892283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f83517fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836000f830 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f836000f9d0 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83580047a0 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f836000fbd0 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f8376feb740 10 client.21909 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc11cba7d0 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.440+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83580047a0 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8360026070 con 0x55bc11bd8390 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 10 client.21909 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 10 client.21909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:59:50.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 20 client.21909 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b96f60 0x55bc11f923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 4 client.21909 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc11eec990 con 0x55bc11b96f60 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f836ddfd640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b96f60 0x55bc11f923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f836ddfd640 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b96f60 0x55bc11f923a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84ba791e49e3b05c server_cookie=cee3ea276eb74aa0 in_seq=0 out_seq=0 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.441+0000 7f83517fa640 10 client.21909 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f83517fa640 1 -- 172.21.15.94:0/3754877239 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc11eec990 con 0x55bc11b96f60 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f83517fa640 10 client.21909 handle_command_reply: tid=0 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 client.21909 shutdown 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 msgr2=0x55bc11f8ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f8360009cd0 tx=0x7f8360004ab0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 shutdown_connections 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b96f60 0x55bc11f923a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bd8390 0x55bc11f8ba90 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bb76b0 0x55bc11f84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 --2- 172.21.15.94:0/3754877239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f83517fa640 0 client.21909 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 >> 172.21.15.94:0/3754877239 conn(0x55bc11bcc4e0 msgr2=0x55bc11f78f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 shutdown_connections 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.442+0000 7f8376feb740 1 -- 172.21.15.94:0/3754877239 wait complete. 2024-04-25T13:59:50.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.443+0000 7f8376feb740 20 client.21909 trim_cache size 0 max 16384 2024-04-25T13:59:50.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 Processor -- start 2024-04-25T13:59:50.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 -- start start 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ab510 con 0x55bc11bb76b0 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ad050 con 0x55bc11baf4d0 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f8376feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83681abbe0 con 0x55bc11bd8390 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51058/0 (socket says 172.21.15.94:51058) 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 -- 172.21.15.94:0/4198267184 learned_addr learned my addr 172.21.15.94:0/4198267184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836e5fe640 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836ddfd640 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 -- 172.21.15.94:0/4198267184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 msgr2=0x55bc11f97b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f97b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 -- 172.21.15.94:0/4198267184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 msgr2=0x55bc11f922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.457+0000 7f836d5fc640 1 -- 172.21.15.94:0/4198267184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc11f826d0 con 0x55bc11bb76b0 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.458+0000 7f836d5fc640 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f8358010c40 tx=0x7f8358009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a1ef55565f46f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.458+0000 7f83537fe640 1 -- 172.21.15.94:0/4198267184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358004040 con 0x55bc11bb76b0 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.458+0000 7f83537fe640 1 -- 172.21.15.94:0/4198267184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358005030 con 0x55bc11bb76b0 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.458+0000 7f83537fe640 1 -- 172.21.15.94:0/4198267184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358004b00 con 0x55bc11bb76b0 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- 172.21.15.94:0/4198267184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 msgr2=0x55bc11f8b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f8358010c40 tx=0x7f8358009ed0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- 172.21.15.94:0/4198267184 shutdown_connections 2024-04-25T13:59:50.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f97b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8b9b0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- 172.21.15.94:0/4198267184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- 172.21.15.94:0/4198267184 >> 172.21.15.94:0/4198267184 conn(0x55bc11bcc4e0 msgr2=0x55bc11f7f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- 172.21.15.94:0/4198267184 shutdown_connections 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- 172.21.15.94:0/4198267184 wait complete. 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- start start 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 Processor -- start 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f8a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ac980 con 0x55bc11bb76b0 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83681ac2b0 con 0x55bc11baf4d0 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.459+0000 7f8376feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83681a94f0 con 0x55bc11bd8390 2024-04-25T13:59:50.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f8376feb740 10 client.0 _ll_register_callbacks cb 0x7f8360009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51074/0 (socket says 172.21.15.94:51074) 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 -- 172.21.15.94:0/295213926 learned_addr learned my addr 172.21.15.94:0/295213926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836e5fe640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836ddfd640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f8a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 -- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 msgr2=0x55bc11f8ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f8ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 -- 172.21.15.94:0/295213926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 msgr2=0x55bc11f8a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f8a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 -- 172.21.15.94:0/295213926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc11f8bd40 con 0x55bc11bb76b0 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836e5fe640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f836d5fc640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8358005d90 tx=0x7f8358009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31de62f0b6ea6178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f83517fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358034060 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f83580053f0 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358005030 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f83580055f0 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f8376feb740 10 client.21915 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.460+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc11ff97b0 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358005030 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f835800f040 con 0x55bc11bb76b0 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 10 client.21915 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 10 client.21915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T13:59:50.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 20 client.21915 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b6c3a0 0x55bc11ffbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 4 client.21915 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc11ee6800 con 0x55bc11b6c3a0 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f836ddfd640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b6c3a0 0x55bc11ffbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f83517fa640 10 client.21915 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.461+0000 7f836ddfd640 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b6c3a0 0x55bc11ffbf90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80f50855aa1a7909 server_cookie=32aab69380c6f868 in_seq=0 out_seq=0 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f83517fa640 1 -- 172.21.15.94:0/295213926 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc11ee6800 con 0x55bc11b6c3a0 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f83517fa640 10 client.21915 handle_command_reply: tid=0 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 client.21915 shutdown 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 msgr2=0x55bc11f8a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8358005d90 tx=0x7f8358009e30 comp rx=0 tx=0).stop 2024-04-25T13:59:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 shutdown_connections 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bc11b6c3a0 0x55bc11ffbf90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc11bd8390 0x55bc11f8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f83517fa640 0 client.21915 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc11bb76b0 0x55bc11f8a5c0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 --2- 172.21.15.94:0/295213926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc11baf4d0 0x55bc11f8a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 >> 172.21.15.94:0/295213926 conn(0x55bc11bcc4e0 msgr2=0x55bc11f7f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 shutdown_connections 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 1 -- 172.21.15.94:0/295213926 wait complete. 2024-04-25T13:59:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.462+0000 7f8376feb740 20 client.21915 trim_cache size 0 max 16384 2024-04-25T13:59:50.472 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:50.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 msgr2=0x7f83540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 0x7f83540b2000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55bc11f7b860 tx=0x7f8360009500 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 msgr2=0x7f836819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f8364006010 tx=0x7f8364004270 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 shutdown_connections 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f83540afaf0 0x7f83540b2000 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368109a30 0x7f83681a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8368104d30 0x7f836819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 --2- 172.21.15.94:0/644150675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8368103af0 0x7f836819a2c0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 >> 172.21.15.94:0/644150675 conn(0x7f83680fed00 msgr2=0x7f8368105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 shutdown_connections 2024-04-25T13:59:50.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.463+0000 7f8370061640 1 -- 172.21.15.94:0/644150675 wait complete. 2024-04-25T13:59:50.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T13:59:50.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 -- 172.21.15.94:0/482353806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48108920 msgr2=0x7fbc4810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 --2- 172.21.15.94:0/482353806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48108920 0x7fbc4810ad40 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009ca0 tx=0x7fbc4002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 -- 172.21.15.94:0/482353806 shutdown_connections 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 --2- 172.21.15.94:0/482353806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48108920 0x7fbc4810ad40 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 --2- 172.21.15.94:0/482353806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48103c20 0x7fbc48104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 --2- 172.21.15.94:0/482353806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 0x7fbc481036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 -- 172.21.15.94:0/482353806 >> 172.21.15.94:0/482353806 conn(0x7fbc480fdbd0 msgr2=0x7fbc48100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 -- 172.21.15.94:0/482353806 shutdown_connections 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.591+0000 7fbc500af640 1 -- 172.21.15.94:0/482353806 wait complete. 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 Processor -- start 2024-04-25T13:59:50.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 -- start start 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 0x7fbc4819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48108920 0x7fbc481a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810e540 con 0x7fbc48103c20 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810e3c0 con 0x7fbc481029e0 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc500af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810e6c0 con 0x7fbc48108920 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc4d64a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc4d64a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51090/0 (socket says 172.21.15.94:51090) 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc4d64a640 1 -- 172.21.15.94:0/513302784 learned_addr learned my addr 172.21.15.94:0/513302784 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 0x7fbc4819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.592+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48108920 0x7fbc481a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 -- 172.21.15.94:0/513302784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48108920 msgr2=0x7fbc481a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48108920 0x7fbc481a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 -- 172.21.15.94:0/513302784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 msgr2=0x7fbc4819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 0x7fbc4819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 -- 172.21.15.94:0/513302784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc481a2010 con 0x7fbc48103c20 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9f0 tx=0x7fbc3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc3effd640 1 -- 172.21.15.94:0/513302784 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800cd80 con 0x7fbc48103c20 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc3effd640 1 -- 172.21.15.94:0/513302784 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380108e0 con 0x7fbc48103c20 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc481a2300 con 0x7fbc48103c20 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc3effd640 1 -- 172.21.15.94:0/513302784 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38018980 con 0x7fbc48103c20 2024-04-25T13:59:50.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.593+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a2850 con 0x7fbc48103c20 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.595+0000 7fbc56f67740 1 Processor -- start 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 -- start start 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc3effd640 1 -- 172.21.15.94:0/513302784 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbc38010430 con 0x7fbc48103c20 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8ab390 0x56554ac4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a88a6b0 0x56554ac4ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56554abb1d80 con 0x56554a8824d0 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56554ab6ba20 con 0x56554a8ab390 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc56f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56554abadd40 con 0x56554a88a6b0 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4e64c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8ab390 0x56554ac4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51096/0 (socket says 172.21.15.94:51096) 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 -- 172.21.15.94:0/3773625659 learned_addr learned my addr 172.21.15.94:0/3773625659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a88a6b0 0x56554ac4ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 -- 172.21.15.94:0/3773625659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a88a6b0 msgr2=0x56554ac4ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a88a6b0 0x56554ac4ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 -- 172.21.15.94:0/3773625659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8ab390 msgr2=0x56554ac4d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8ab390 0x56554ac4d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.596+0000 7fbc4de4b640 1 -- 172.21.15.94:0/3773625659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554ab76c30 con 0x56554a8824d0 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc3effd640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 0x7fbc340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009c60 tx=0x7fbc44011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b28e16f252e4ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 0x7fbc340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc33fff640 1 -- 172.21.15.94:0/3773625659 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44004220 con 0x56554a8824d0 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc33fff640 1 -- 172.21.15.94:0/3773625659 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc440043c0 con 0x56554a8824d0 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc3effd640 1 -- 172.21.15.94:0/513302784 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbc380def90 con 0x7fbc48103c20 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.597+0000 7fbc33fff640 1 -- 172.21.15.94:0/3773625659 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44004cf0 con 0x56554a8824d0 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- 172.21.15.94:0/3773625659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 msgr2=0x56554ac4dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009c60 tx=0x7fbc44011810 comp rx=0 tx=0).stop 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 0x7fbc340b2000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x56554ac4e860 tx=0x7fbc40031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- 172.21.15.94:0/3773625659 shutdown_connections 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a88a6b0 0x56554ac4ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac4dd90 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3773625659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8ab390 0x56554ac4d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- 172.21.15.94:0/3773625659 >> 172.21.15.94:0/3773625659 conn(0x56554a89f4e0 msgr2=0x56554ac49020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- 172.21.15.94:0/3773625659 shutdown_connections 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- 172.21.15.94:0/3773625659 wait complete. 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 -- start start 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.598+0000 7fbc56f67740 1 Processor -- start 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac57a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56554ac4f530 con 0x56554a8824d0 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56554ab38dc0 con 0x56554a88a6b0 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56554ac4f6b0 con 0x56554a8ab390 2024-04-25T13:59:50.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc56f67740 10 client.0 _ll_register_callbacks cb 0x56554ab54600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51112/0 (socket says 172.21.15.94:51112) 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 -- 172.21.15.94:0/1289523230 learned_addr learned my addr 172.21.15.94:0/1289523230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac57a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 -- 172.21.15.94:0/1289523230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 msgr2=0x56554ac5e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 -- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 msgr2=0x56554ac57a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac57a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 -- 172.21.15.94:0/1289523230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554a8c33f0 con 0x56554a8824d0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009c30 tx=0x7fbc44019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5979b90d271967f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.599+0000 7fbc317fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44027030 con 0x56554a8824d0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbc44019d30 con 0x56554a8824d0 2024-04-25T13:59:50.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc440043c0 con 0x56554a8824d0 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbc4401d040 con 0x56554a8824d0 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 10 client.21927 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56554a98d7d0 con 0x56554a8824d0 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc440043c0 con 0x56554a8824d0 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbc44005150 con 0x56554a8824d0 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 10 client.21927 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 10 client.21927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 20 client.21927 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a869f60 0x56554ac64fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 4 client.21927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:50.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.600+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554abbf990 con 0x56554a869f60 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.601+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a869f60 0x56554ac64fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.601+0000 7fbc317fa640 10 client.21927 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.601+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a869f60 0x56554ac64fa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7c025f010f824c2 server_cookie=6796b2bc0f0790d8 in_seq=0 out_seq=0 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc317fa640 1 -- 172.21.15.94:0/1289523230 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56554abbf990 con 0x56554a869f60 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc317fa640 10 client.21927 handle_command_reply: tid=0 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 client.21927 shutdown 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 msgr2=0x56554ac57510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009c30 tx=0x7fbc44019b40 comp rx=0 tx=0).stop 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 shutdown_connections 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a869f60 0x56554ac64fa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac57a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc317fa640 0 client.21927 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:50.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 --2- 172.21.15.94:0/1289523230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57510 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 >> 172.21.15.94:0/1289523230 conn(0x56554a89f4e0 msgr2=0x56554ac4bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 shutdown_connections 2024-04-25T13:59:50.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.602+0000 7fbc56f67740 1 -- 172.21.15.94:0/1289523230 wait complete. 2024-04-25T13:59:50.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.603+0000 7fbc56f67740 20 client.21927 trim_cache size 0 max 16384 2024-04-25T13:59:50.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 Processor -- start 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 -- start start 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac5e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ad780 con 0x56554a8824d0 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481af2c0 con 0x56554a88a6b0 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc56f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ade50 con 0x56554a8ab390 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51118/0 (socket says 172.21.15.94:51118) 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 -- 172.21.15.94:0/843730619 learned_addr learned my addr 172.21.15.94:0/843730619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac5e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 -- 172.21.15.94:0/843730619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 msgr2=0x56554ac4c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac4c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.617+0000 7fbc4de4b640 1 -- 172.21.15.94:0/843730619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 msgr2=0x56554ac5e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac5e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc4de4b640 1 -- 172.21.15.94:0/843730619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554ac5ee10 con 0x56554a8824d0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fbc44011850 tx=0x7fbc44005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e312b33fb52509e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc337fe640 1 -- 172.21.15.94:0/843730619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44027070 con 0x56554a8824d0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc337fe640 1 -- 172.21.15.94:0/843730619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4402e740 con 0x56554a8824d0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.618+0000 7fbc337fe640 1 -- 172.21.15.94:0/843730619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44019bb0 con 0x56554a8824d0 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- 172.21.15.94:0/843730619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 msgr2=0x56554ac57b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fbc44011850 tx=0x7fbc44005780 comp rx=0 tx=0).stop 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- 172.21.15.94:0/843730619 shutdown_connections 2024-04-25T13:59:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac4c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a88a6b0 0x56554ac5e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 --2- 172.21.15.94:0/843730619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a8824d0 0x56554ac57b10 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- 172.21.15.94:0/843730619 >> 172.21.15.94:0/843730619 conn(0x56554a89f4e0 msgr2=0x56554ac4be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- 172.21.15.94:0/843730619 shutdown_connections 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- 172.21.15.94:0/843730619 wait complete. 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 -- start start 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.619+0000 7fbc56f67740 1 Processor -- start 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8824d0 0x56554ac5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481aebf0 con 0x56554a88a6b0 2024-04-25T13:59:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ae520 con 0x56554a8824d0 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ab760 con 0x56554a8ab390 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 10 client.0 _ll_register_callbacks cb 0x7fbc38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4de4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8824d0 0x56554ac5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51132/0 (socket says 172.21.15.94:51132) 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 -- 172.21.15.94:0/3563226428 learned_addr learned my addr 172.21.15.94:0/3563226428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 -- 172.21.15.94:0/3563226428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 msgr2=0x56554ac5d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 -- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8824d0 msgr2=0x56554ac5bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8824d0 0x56554ac5bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 -- 172.21.15.94:0/3563226428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554ac5d710 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4e64c640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc4d64a640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fbc380dc9f0 tx=0x7fbc380e2a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bd59f625c68f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc317fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38010720 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbc38010a90 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.620+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc4402e740 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbc380a6f90 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 10 client.21933 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56554ac5ea90 con 0x56554a88a6b0 2024-04-25T13:59:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc4402e740 con 0x56554a88a6b0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbc380a3710 con 0x56554a88a6b0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 10 client.21933 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 10 client.21933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 20 client.21933 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a83f3a0 0x56554accf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 4 client.21933 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56554abb9800 con 0x56554a83f3a0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.621+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a83f3a0 0x56554accf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc317fa640 10 client.21933 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc4de4b640 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a83f3a0 0x56554accf9c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49b101e2e12490c server_cookie=1348acdec3786ae5 in_seq=0 out_seq=0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc317fa640 1 -- 172.21.15.94:0/3563226428 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56554abb9800 con 0x56554a83f3a0 2024-04-25T13:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc317fa640 10 client.21933 handle_command_reply: tid=0 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 client.21933 shutdown 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 msgr2=0x56554ac5c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fbc380dc9f0 tx=0x7fbc380e2a70 comp rx=0 tx=0).stop 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 shutdown_connections 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56554a83f3a0 0x56554accf9c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56554a8ab390 0x56554ac5d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56554a88a6b0 0x56554ac5c290 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 --2- 172.21.15.94:0/3563226428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554a8824d0 0x56554ac5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc317fa640 0 client.21933 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T13:59:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 >> 172.21.15.94:0/3563226428 conn(0x56554a89f4e0 msgr2=0x56554ac4c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 shutdown_connections 2024-04-25T13:59:50.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 1 -- 172.21.15.94:0/3563226428 wait complete. 2024-04-25T13:59:50.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.622+0000 7fbc56f67740 20 client.21933 trim_cache size 0 max 16384 2024-04-25T13:59:50.630 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 msgr2=0x7fbc340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 0x7fbc340b2000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x56554ac4e860 tx=0x7fbc40031af0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 msgr2=0x7fbc4819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e9f0 tx=0x7fbc3800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 shutdown_connections 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbc340afaf0 0x7fbc340b2000 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc48108920 0x7fbc481a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc48103c20 0x7fbc4819aa10 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 --2- 172.21.15.94:0/513302784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc481029e0 0x7fbc4819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 >> 172.21.15.94:0/513302784 conn(0x7fbc480fdbd0 msgr2=0x7fbc48109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 shutdown_connections 2024-04-25T13:59:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.623+0000 7fbc500af640 1 -- 172.21.15.94:0/513302784 wait complete. 2024-04-25T13:59:50.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 -- 172.21.15.94:0/3348268925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 msgr2=0x7f2360104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 --2- 172.21.15.94:0/3348268925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f2360104050 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f2354009900 tx=0x7f235402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 -- 172.21.15.94:0/3348268925 shutdown_connections 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 --2- 172.21.15.94:0/3348268925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 0x7f236010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 --2- 172.21.15.94:0/3348268925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f2360104050 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 --2- 172.21.15.94:0/3348268925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 -- 172.21.15.94:0/3348268925 >> 172.21.15.94:0/3348268925 conn(0x7f23600fdbf0 msgr2=0x7f23601000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 -- 172.21.15.94:0/3348268925 shutdown_connections 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.748+0000 7f2366813640 1 -- 172.21.15.94:0/3348268925 wait complete. 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 Processor -- start 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 -- start start 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 0x7f23601a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f236010e530 con 0x7f2360103c40 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f236010e3b0 con 0x7f2360102a00 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.749+0000 7f2366813640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f236010e6b0 con 0x7f2360108940 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51162/0 (socket says 172.21.15.94:51162) 2024-04-25T13:59:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 -- 172.21.15.94:0/2918498790 learned_addr learned my addr 172.21.15.94:0/2918498790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f2364db0640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 0x7f23601a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 -- 172.21.15.94:0/2918498790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 msgr2=0x7f23601a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 0x7f23601a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 -- 172.21.15.94:0/2918498790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 msgr2=0x7f2360199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 -- 172.21.15.94:0/2918498790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23601a0cd0 con 0x7f2360103c40 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235ffff640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235f7fe640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f23540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235d7fa640 1 -- 172.21.15.94:0/2918498790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235403f030 con 0x7f2360103c40 2024-04-25T13:59:50.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23601a0f60 con 0x7f2360103c40 2024-04-25T13:59:50.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235d7fa640 1 -- 172.21.15.94:0/2918498790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2354031910 con 0x7f2360103c40 2024-04-25T13:59:50.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f235d7fa640 1 -- 172.21.15.94:0/2918498790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354035070 con 0x7f2360103c40 2024-04-25T13:59:50.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.750+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23601a1370 con 0x7f2360103c40 2024-04-25T13:59:50.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f235d7fa640 1 -- 172.21.15.94:0/2918498790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2354042030 con 0x7f2360103c40 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 Processor -- start 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 -- start start 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a21ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d844981d80 con 0x55d84467b390 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d84493ba20 con 0x55d84465a6b0 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f236d7a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d84497dd40 con 0x55d8446524d0 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.753+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49242/0 (socket says 172.21.15.94:49242) 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51174/0 (socket says 172.21.15.94:51174) 2024-04-25T13:59:50.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 -- 172.21.15.94:0/2407663619 learned_addr learned my addr 172.21.15.94:0/2407663619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235f7fe640 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a21ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235d7fa640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 0x7f23480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 -- 172.21.15.94:0/2407663619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 msgr2=0x55d844a21ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a21ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 -- 172.21.15.94:0/2407663619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 msgr2=0x55d844a23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 -- 172.21.15.94:0/2407663619 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d844946c30 con 0x55d8446524d0 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f2364db0640 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f2364db0640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 0x7f23480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235d7fa640 1 -- 172.21.15.94:0/2918498790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23540ff070 con 0x7f2360103c40 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f235ffff640 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23500114f0 tx=0x7f23500119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e9ea094e6080150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f23467fc640 1 -- 172.21.15.94:0/2407663619 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350005020 con 0x55d8446524d0 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f23467fc640 1 -- 172.21.15.94:0/2407663619 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23500047a0 con 0x55d8446524d0 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.754+0000 7f23467fc640 1 -- 172.21.15.94:0/2407663619 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000f880 con 0x55d8446524d0 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f2364db0640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 0x7f23480b2000 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55d844a19770 tx=0x7f2358004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 -- 172.21.15.94:0/2407663619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 msgr2=0x55d844a236d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23500114f0 tx=0x7f23500119b0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 -- 172.21.15.94:0/2407663619 shutdown_connections 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a21ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a236d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2407663619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 -- 172.21.15.94:0/2407663619 >> 172.21.15.94:0/2407663619 conn(0x55d84466f4e0 msgr2=0x55d844a189d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 -- 172.21.15.94:0/2407663619 shutdown_connections 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.755+0000 7f236d7a0740 1 -- 172.21.15.94:0/2407663619 wait complete. 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 -- start start 2024-04-25T13:59:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 Processor -- start 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8446524d0 0x55d844a2b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 0x55d844a2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d844a22920 con 0x55d84467b390 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d844908dc0 con 0x55d8446524d0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d844a22aa0 con 0x55d84465a6b0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f236d7a0740 10 client.0 _ll_register_callbacks cb 0x55d844924600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51176/0 (socket says 172.21.15.94:51176) 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f2364db0640 1 -- 172.21.15.94:0/2802334655 learned_addr learned my addr 172.21.15.94:0/2802334655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 0x55d844a2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.756+0000 7f235ffff640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8446524d0 0x55d844a2b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 -- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 msgr2=0x55d844a2bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 0x55d844a2bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 -- 172.21.15.94:0/2802334655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8446524d0 msgr2=0x55d844a2b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8446524d0 0x55d844a2b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 -- 172.21.15.94:0/2802334655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8446933f0 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f235f7fe640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 0x55d844a2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f2364db0640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f23580209a0 tx=0x7f2358004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ee565c65190ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f233f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358010bc0 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23500047a0 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f235801b440 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f235801b680 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f236d7a0740 10 client.21945 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d84475d7d0 con 0x55d84467b390 2024-04-25T13:59:50.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.757+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23500047a0 con 0x55d84467b390 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f2358027020 con 0x55d84467b390 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 10 client.21945 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 10 client.21945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 20 client.21945 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844639f60 0x55d844a39660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 4 client.21945 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84498f990 con 0x55d844639f60 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f235ffff640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844639f60 0x55d844a39660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f233f7fe640 10 client.21945 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.758+0000 7f235ffff640 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844639f60 0x55d844a39660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=596ef4e6a166621c server_cookie=49556a57681cd59d in_seq=0 out_seq=0 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f233f7fe640 1 -- 172.21.15.94:0/2802334655 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d84498f990 con 0x55d844639f60 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f233f7fe640 10 client.21945 handle_command_reply: tid=0 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 client.21945 shutdown 2024-04-25T13:59:50.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 msgr2=0x55d844a32d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f23580209a0 tx=0x7f2358004ab0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 shutdown_connections 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844639f60 0x55d844a39660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32d40 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f233f7fe640 0 client.21945 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84465a6b0 0x55d844a2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 --2- 172.21.15.94:0/2802334655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8446524d0 0x55d844a2b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 >> 172.21.15.94:0/2802334655 conn(0x55d84466f4e0 msgr2=0x55d844a189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 shutdown_connections 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 1 -- 172.21.15.94:0/2802334655 wait complete. 2024-04-25T13:59:50.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.759+0000 7f236d7a0740 20 client.21945 trim_cache size 0 max 16384 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 Processor -- start 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 -- start start 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23601a93a0 con 0x55d84467b390 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23601aaee0 con 0x55d84465a6b0 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f236d7a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23601a9a70 con 0x55d8446524d0 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f235ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51192/0 (socket says 172.21.15.94:51192) 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f2364db0640 1 -- 172.21.15.94:0/668098860 learned_addr learned my addr 172.21.15.94:0/668098860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.774+0000 7f235f7fe640 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 -- 172.21.15.94:0/668098860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 msgr2=0x55d844a2a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 -- 172.21.15.94:0/668098860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 msgr2=0x55d844a32c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 -- 172.21.15.94:0/668098860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d844a331a0 con 0x55d8446524d0 2024-04-25T13:59:50.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f2364db0640 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f235ffff640 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a29ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2350009380 tx=0x7f2350005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e848cfa3e750b1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f2345ffb640 1 -- 172.21.15.94:0/668098860 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350031070 con 0x55d8446524d0 2024-04-25T13:59:50.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f2345ffb640 1 -- 172.21.15.94:0/668098860 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2350004500 con 0x55d8446524d0 2024-04-25T13:59:50.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.775+0000 7f2345ffb640 1 -- 172.21.15.94:0/668098860 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350003ac0 con 0x55d8446524d0 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- 172.21.15.94:0/668098860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 msgr2=0x55d844a29ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a29ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2350009380 tx=0x7f2350005d50 comp rx=0 tx=0).stop 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- 172.21.15.94:0/668098860 shutdown_connections 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84467b390 0x55d844a32c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 --2- 172.21.15.94:0/668098860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8446524d0 0x55d844a29ec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- 172.21.15.94:0/668098860 >> 172.21.15.94:0/668098860 conn(0x55d84466f4e0 msgr2=0x55d844a26780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- 172.21.15.94:0/668098860 shutdown_connections 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- 172.21.15.94:0/668098860 wait complete. 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 -- start start 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.776+0000 7f236d7a0740 1 Processor -- start 2024-04-25T13:59:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84467b390 0x55d844a304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23601aa810 con 0x55d84460f3a0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23601aa140 con 0x55d84465a6b0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23601a7380 con 0x55d84467b390 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f236d7a0740 10 client.0 _ll_register_callbacks cb 0x7f2358009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51194/0 (socket says 172.21.15.94:51194) 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 -- 172.21.15.94:0/1744575295 learned_addr learned my addr 172.21.15.94:0/1744575295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f235ffff640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f235f7fe640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84467b390 0x55d844a304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 -- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84467b390 msgr2=0x55d844a304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84467b390 0x55d844a304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 -- 172.21.15.94:0/1744575295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 msgr2=0x55d844a2ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 -- 172.21.15.94:0/1744575295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d844a31950 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f235ffff640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f2364db0640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f2358029b10 tx=0x7f23580299c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b400f258861d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f233f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.777+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358040060 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2350004500 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f2358029d00 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2358037040 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 10 client.21951 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d844a32c40 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2350004500 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f235803a030 con 0x55d84460f3a0 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 10 client.21951 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 10 client.21951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 20 client.21951 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844aa1eb0 0x55d844aa42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 4 client.21951 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d844989800 con 0x55d844aa1eb0 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.778+0000 7f235ffff640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844aa1eb0 0x55d844aa42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f235ffff640 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844aa1eb0 0x55d844aa42d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8db0f92c84e286c server_cookie=da3f12bf5cd6da36 in_seq=0 out_seq=0 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f233f7fe640 10 client.21951 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f233f7fe640 1 -- 172.21.15.94:0/1744575295 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d844989800 con 0x55d844aa1eb0 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f233f7fe640 10 client.21951 handle_command_reply: tid=0 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 client.21951 shutdown 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 msgr2=0x55d844a31390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f2358029b10 tx=0x7f23580299c0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 shutdown_connections 2024-04-25T13:59:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d844aa1eb0 0x55d844aa42d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d84460f3a0 0x55d844a31390 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d84467b390 0x55d844a304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 --2- 172.21.15.94:0/1744575295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d84465a6b0 0x55d844a2ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f233f7fe640 0 client.21951 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 >> 172.21.15.94:0/1744575295 conn(0x55d84466f4e0 msgr2=0x55d844a26780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 shutdown_connections 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 1 -- 172.21.15.94:0/1744575295 wait complete. 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.779+0000 7f236d7a0740 20 client.21951 trim_cache size 0 max 16384 2024-04-25T13:59:50.786 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 msgr2=0x7f23480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 0x7f23480b2000 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55d844a19770 tx=0x7f2358004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 msgr2=0x7f23601996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f23540042f0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 shutdown_connections 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23480afaf0 0x7f23480b2000 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2360108940 0x7f23601a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2360103c40 0x7f23601996d0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 --2- 172.21.15.94:0/2918498790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2360102a00 0x7f2360199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 >> 172.21.15.94:0/2918498790 conn(0x7f23600fdbf0 msgr2=0x7f2360104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 shutdown_connections 2024-04-25T13:59:50.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.780+0000 7f2366813640 1 -- 172.21.15.94:0/2918498790 wait complete. 2024-04-25T13:59:50.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- 172.21.15.94:0/2942242207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb8103c20 msgr2=0x7f4eb8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 --2- 172.21.15.94:0/2942242207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb8103c20 0x7f4eb8104030 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- 172.21.15.94:0/2942242207 shutdown_connections 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 --2- 172.21.15.94:0/2942242207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8108920 0x7f4eb810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 --2- 172.21.15.94:0/2942242207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb8103c20 0x7f4eb8104030 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 --2- 172.21.15.94:0/2942242207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb81029e0 0x7f4eb81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- 172.21.15.94:0/2942242207 >> 172.21.15.94:0/2942242207 conn(0x7f4eb80fdbd0 msgr2=0x7f4eb8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- 172.21.15.94:0/2942242207 shutdown_connections 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- 172.21.15.94:0/2942242207 wait complete. 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 Processor -- start 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.907+0000 7f4ec0764640 1 -- start start 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8103c20 0x7f4eb819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 0x7f4eb81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e510 con 0x7f4eb81029e0 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e390 con 0x7f4eb8108920 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb810e690 con 0x7f4eb8103c20 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51214/0 (socket says 172.21.15.94:51214) 2024-04-25T13:59:50.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 -- 172.21.15.94:0/200148414 learned_addr learned my addr 172.21.15.94:0/200148414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8103c20 0x7f4eb819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebed01640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 0x7f4eb81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 -- 172.21.15.94:0/200148414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8103c20 msgr2=0x7f4eb819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8103c20 0x7f4eb819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 -- 172.21.15.94:0/200148414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 msgr2=0x7f4eb81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 0x7f4eb81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 -- 172.21.15.94:0/200148414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a2050 con 0x7f4eb81029e0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebed01640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 0x7f4eb81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ebe500640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400e9f0 tx=0x7f4eb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4eaf7fe640 1 -- 172.21.15.94:0/200148414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400ccc0 con 0x7f4eb81029e0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4eaf7fe640 1 -- 172.21.15.94:0/200148414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb40035f0 con 0x7f4eb81029e0 2024-04-25T13:59:50.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a2340 con 0x7f4eb81029e0 2024-04-25T13:59:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.908+0000 7f4eaf7fe640 1 -- 172.21.15.94:0/200148414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400ce60 con 0x7f4eb81029e0 2024-04-25T13:59:50.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.909+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a2890 con 0x7f4eb81029e0 2024-04-25T13:59:50.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.911+0000 7f4ec7613740 1 Processor -- start 2024-04-25T13:59:50.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.911+0000 7f4eaf7fe640 1 -- 172.21.15.94:0/200148414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4eb4003aa0 con 0x7f4eb81029e0 2024-04-25T13:59:50.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.911+0000 7f4ec7613740 1 -- start start 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.911+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.911+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 0x561bcc804a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ec7613740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561bcc767d80 con 0x561bcc461390 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ec7613740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561bcc721a20 con 0x561bcc4384d0 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ec7613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561bcc763d40 con 0x561bcc4406b0 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebe500640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebdcff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 0x561bcc804a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebe500640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55992/0 (socket says 172.21.15.94:55992) 2024-04-25T13:59:50.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebdcff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 0x561bcc804a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49282/0 (socket says 172.21.15.94:49282) 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebe500640 1 -- 172.21.15.94:0/2791829995 learned_addr learned my addr 172.21.15.94:0/2791829995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51218/0 (socket says 172.21.15.94:51218) 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 -- 172.21.15.94:0/2791829995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 msgr2=0x561bcc804a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 0x561bcc804a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 -- 172.21.15.94:0/2791829995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 msgr2=0x561bcc803d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 -- 172.21.15.94:0/2791829995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcc72cc30 con 0x561bcc461390 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebe500640 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4eaf7fe640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 0x7f4ea40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebed01640 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009f90 tx=0x7f4eb0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173391ca1882dbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4ebe500640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 0x7f4ea40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.912+0000 7f4eacff9640 1 -- 172.21.15.94:0/2791829995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004d20 con 0x561bcc461390 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4eaf7fe640 1 -- 172.21.15.94:0/200148414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4eb40df060 con 0x7f4eb81029e0 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4eacff9640 1 -- 172.21.15.94:0/2791829995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb00119b0 con 0x561bcc461390 2024-04-25T13:59:50.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4eacff9640 1 -- 172.21.15.94:0/2791829995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0005580 con 0x561bcc461390 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 -- 172.21.15.94:0/2791829995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 msgr2=0x561bcc803810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ebe500640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 0x7f4ea40b2000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40a3520 tx=0x7f4eb40e2b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009f90 tx=0x7f4eb0010860 comp rx=0 tx=0).stop 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 -- 172.21.15.94:0/2791829995 shutdown_connections 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc4406b0 0x561bcc804a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc803d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 --2- 172.21.15.94:0/2791829995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc461390 0x561bcc803810 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.913+0000 7f4ec7613740 1 -- 172.21.15.94:0/2791829995 >> 172.21.15.94:0/2791829995 conn(0x561bcc4554e0 msgr2=0x561bcc7ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- 172.21.15.94:0/2791829995 shutdown_connections 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- 172.21.15.94:0/2791829995 wait complete. 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- start start 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 Processor -- start 2024-04-25T13:59:50.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc80d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc461390 0x561bcc814a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561bcc805530 con 0x561bcc4406b0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561bcc6eedc0 con 0x561bcc4384d0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561bcc8056b0 con 0x561bcc461390 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ebdcff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ec7613740 10 client.0 _ll_register_callbacks cb 0x561bcc70a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc80d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ebdcff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51222/0 (socket says 172.21.15.94:51222) 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ebdcff640 1 -- 172.21.15.94:0/1232815528 learned_addr learned my addr 172.21.15.94:0/1232815528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.914+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc461390 0x561bcc814a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 -- 172.21.15.94:0/1232815528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc461390 msgr2=0x561bcc814a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc461390 0x561bcc814a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 -- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 msgr2=0x561bcc80d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc80d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 -- 172.21.15.94:0/1232815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcc4793f0 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc80d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea802a930 tx=0x7f4ea8031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168fded2249f09b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8035040 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4ea8004040 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb00119b0 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ea8004240 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ec7613740 10 client.21963 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bcc5437d0 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.915+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb00119b0 con 0x561bcc4406b0 2024-04-25T13:59:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4ea8044020 con 0x561bcc4406b0 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 10 client.21963 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 10 client.21963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 20 client.21963 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc41ff60 0x561bcc81b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 4 client.21963 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bcc775990 con 0x561bcc41ff60 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc41ff60 0x561bcc81b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ea1ffb640 10 client.21963 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.916+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc41ff60 0x561bcc81b240 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=443d8fe93a7120f2 server_cookie=4c92c9fbbfcb6a57 in_seq=0 out_seq=0 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1232815528 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561bcc775990 con 0x561bcc41ff60 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ea1ffb640 10 client.21963 handle_command_reply: tid=0 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 client.21963 shutdown 2024-04-25T13:59:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 msgr2=0x561bcc80dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea802a930 tx=0x7f4ea8031e80 comp rx=0 tx=0).stop 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 shutdown_connections 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc41ff60 0x561bcc81b240 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc461390 0x561bcc814a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ea1ffb640 0 client.21963 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc80dac0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1232815528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc80d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 >> 172.21.15.94:0/1232815528 conn(0x561bcc4554e0 msgr2=0x561bcc7fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 shutdown_connections 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 1 -- 172.21.15.94:0/1232815528 wait complete. 2024-04-25T13:59:50.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.917+0000 7f4ec7613740 20 client.21963 trim_cache size 0 max 16384 2024-04-25T13:59:50.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 Processor -- start 2024-04-25T13:59:50.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 -- start start 2024-04-25T13:59:50.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc80bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc814940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81ab560 con 0x561bcc4406b0 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81ad0a0 con 0x561bcc4384d0 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ec7613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81abc30 con 0x561bcc41ff60 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51224/0 (socket says 172.21.15.94:51224) 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 -- 172.21.15.94:0/3571277123 learned_addr learned my addr 172.21.15.94:0/3571277123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc80bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc814940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 -- 172.21.15.94:0/3571277123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 msgr2=0x561bcc80bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc80bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 -- 172.21.15.94:0/3571277123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 msgr2=0x561bcc814940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc814940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.931+0000 7f4ebed01640 1 -- 172.21.15.94:0/3571277123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bcc803360 con 0x561bcc4406b0 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.932+0000 7f4ebed01640 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a60 tx=0x7f4eb0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dcd44f487ac701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.932+0000 7f4ea3fff640 1 -- 172.21.15.94:0/3571277123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0025040 con 0x561bcc4406b0 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.932+0000 7f4ea3fff640 1 -- 172.21.15.94:0/3571277123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb0026040 con 0x561bcc4406b0 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.932+0000 7f4ea3fff640 1 -- 172.21.15.94:0/3571277123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0005090 con 0x561bcc4406b0 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- 172.21.15.94:0/3571277123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 msgr2=0x561bcc81f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a60 tx=0x7f4eb0005000 comp rx=0 tx=0).stop 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- 172.21.15.94:0/3571277123 shutdown_connections 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc81f9f0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc814940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- 172.21.15.94:0/3571277123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc80bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- 172.21.15.94:0/3571277123 >> 172.21.15.94:0/3571277123 conn(0x561bcc4554e0 msgr2=0x561bcc808270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- 172.21.15.94:0/3571277123 shutdown_connections 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- 172.21.15.94:0/3571277123 wait complete. 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- start start 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 Processor -- start 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc811fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc812500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81ac9d0 con 0x561bcc4406b0 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81ac300 con 0x561bcc4384d0 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81a9540 con 0x561bcc41ff60 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ec7613740 10 client.0 _ll_register_callbacks cb 0x7f4ea8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.933+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51230/0 (socket says 172.21.15.94:51230) 2024-04-25T13:59:50.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 -- 172.21.15.94:0/1852164068 learned_addr learned my addr 172.21.15.94:0/1852164068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc811fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc812500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 -- 172.21.15.94:0/1852164068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 msgr2=0x561bcc811fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc811fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 -- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 msgr2=0x561bcc812500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc812500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 -- 172.21.15.94:0/1852164068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcc814e80 con 0x561bcc4406b0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ebed01640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0011010 tx=0x7f4eb0025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5818a5f5140db8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002e600 con 0x561bcc4406b0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4eb002e7a0 con 0x561bcc4406b0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb0026040 con 0x561bcc4406b0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4eb002e9a0 con 0x561bcc4406b0 2024-04-25T13:59:50.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ec7613740 10 client.21969 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561bcc814a80 con 0x561bcc4406b0 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.934+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb0026040 con 0x561bcc4406b0 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4eb002b040 con 0x561bcc4406b0 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 10 client.21969 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 10 client.21969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 20 client.21969 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc461390 0x561bcc885900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 4 client.21969 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bcc76f800 con 0x561bcc461390 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc461390 0x561bcc885900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ea1ffb640 10 client.21969 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ebdcff640 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc461390 0x561bcc885900 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3e60118130f54d7 server_cookie=763602a04ef6a565 in_seq=0 out_seq=0 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ea1ffb640 1 -- 172.21.15.94:0/1852164068 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bcc76f800 con 0x561bcc461390 2024-04-25T13:59:50.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ea1ffb640 10 client.21969 handle_command_reply: tid=0 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.935+0000 7f4ec7613740 1 client.21969 shutdown 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 msgr2=0x561bcc813430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0011010 tx=0x7f4eb0025040 comp rx=0 tx=0).stop 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 shutdown_connections 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x561bcc461390 0x561bcc885900 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561bcc4406b0 0x561bcc813430 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bcc4384d0 0x561bcc812500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ea1ffb640 0 client.21969 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 --2- 172.21.15.94:0/1852164068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bcc41ff60 0x561bcc811fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 >> 172.21.15.94:0/1852164068 conn(0x561bcc4554e0 msgr2=0x561bcc801ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 shutdown_connections 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 1 -- 172.21.15.94:0/1852164068 wait complete. 2024-04-25T13:59:50.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec7613740 20 client.21969 trim_cache size 0 max 16384 2024-04-25T13:59:50.943 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 msgr2=0x7f4ea40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 0x7f4ea40b2000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40a3520 tx=0x7f4eb40e2b40 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 msgr2=0x7f4eb806a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400e9f0 tx=0x7f4eb400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 shutdown_connections 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4ea40afaf0 0x7f4ea40b2000 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8108920 0x7f4eb81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4eb8103c20 0x7f4eb819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 --2- 172.21.15.94:0/200148414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4eb81029e0 0x7f4eb806a920 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 >> 172.21.15.94:0/200148414 conn(0x7f4eb80fdbd0 msgr2=0x7f4eb8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 shutdown_connections 2024-04-25T13:59:50.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:50.936+0000 7f4ec0764640 1 -- 172.21.15.94:0/200148414 wait complete. 2024-04-25T13:59:50.946 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:50 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1716678253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:50.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:59:51.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.085+0000 7fd7147f8640 1 -- 172.21.15.94:0/3625248829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c110030 msgr2=0x7fd70c110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.085+0000 7fd7147f8640 1 --2- 172.21.15.94:0/3625248829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c110030 0x7fd70c110440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd704009ca0 tx=0x7fd70402a8d0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.086+0000 7fd7147f8640 1 -- 172.21.15.94:0/3625248829 shutdown_connections 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.086+0000 7fd7147f8640 1 --2- 172.21.15.94:0/3625248829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c110030 0x7fd70c110440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.086+0000 7fd7147f8640 1 --2- 172.21.15.94:0/3625248829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10b310 0x7fd70c10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.086+0000 7fd7147f8640 1 --2- 172.21.15.94:0/3625248829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10a420 0x7fd70c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.086+0000 7fd7147f8640 1 -- 172.21.15.94:0/3625248829 >> 172.21.15.94:0/3625248829 conn(0x7fd70c0704e0 msgr2=0x7fd70c072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.090+0000 7fd7147f8640 1 -- 172.21.15.94:0/3625248829 shutdown_connections 2024-04-25T13:59:51.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.093+0000 7fd7147f8640 1 -- 172.21.15.94:0/3625248829 wait complete. 2024-04-25T13:59:51.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.094+0000 7fd7147f8640 1 Processor -- start 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 -- start start 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10b310 0x7fd70c117550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c110030 0x7fd70c1184b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11e510 con 0x7fd70c10a420 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11e390 con 0x7fd70c110030 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.098+0000 7fd7147f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11e690 con 0x7fd70c10b310 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51246/0 (socket says 172.21.15.94:51246) 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 -- 172.21.15.94:0/299857625 learned_addr learned my addr 172.21.15.94:0/299857625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 -- 172.21.15.94:0/299857625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10b310 msgr2=0x7fd70c117550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10b310 0x7fd70c117550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 -- 172.21.15.94:0/299857625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c110030 msgr2=0x7fd70c1184b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c110030 0x7fd70c1184b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.099+0000 7fd712594640 1 -- 172.21.15.94:0/299857625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c118b20 con 0x7fd70c10a420 2024-04-25T13:59:51.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.101+0000 7fd712594640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00d770 tx=0x7fd6fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.106+0000 7fd7037fe640 1 -- 172.21.15.94:0/299857625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc017070 con 0x7fd70c10a420 2024-04-25T13:59:51.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.106+0000 7fd7037fe640 1 -- 172.21.15.94:0/299857625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc009c30 con 0x7fd70c10a420 2024-04-25T13:59:51.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.106+0000 7fd7037fe640 1 -- 172.21.15.94:0/299857625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00f620 con 0x7fd70c10a420 2024-04-25T13:59:51.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.106+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c118e10 con 0x7fd70c10a420 2024-04-25T13:59:51.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.106+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c119390 con 0x7fd70c10a420 2024-04-25T13:59:51.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.113+0000 7fd7037fe640 1 -- 172.21.15.94:0/299857625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd6fc01a070 con 0x7fd70c10a420 2024-04-25T13:59:51.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.115+0000 7fd7037fe640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 0x7fd6f80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.115+0000 7fd7037fe640 1 -- 172.21.15.94:0/299857625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd6fc0e1170 con 0x7fd70c10a420 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.117+0000 7fd711d93640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 0x7fd6f80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.117+0000 7fd71b788740 1 Processor -- start 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 -- start start 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f3449810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f344aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5563f33add80 con 0x5563f30866b0 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563f3367a20 con 0x5563f307e4d0 2024-04-25T13:59:51.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.118+0000 7fd71b788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563f33a9d40 con 0x5563f30a7390 2024-04-25T13:59:51.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.121+0000 7fd711d93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.121+0000 7fd711d93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56032/0 (socket says 172.21.15.94:56032) 2024-04-25T13:59:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.121+0000 7fd711d93640 1 -- 172.21.15.94:0/2011128899 learned_addr learned my addr 172.21.15.94:0/2011128899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.121+0000 7fd712594640 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f3449810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.122+0000 7fd711d93640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 0x7fd6f80b2190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd70c1180b0 tx=0x7fd708006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.123+0000 7fd712d95640 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f344aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.125+0000 7fd711d93640 1 -- 172.21.15.94:0/2011128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 msgr2=0x5563f3449810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.125+0000 7fd711d93640 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f3449810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.125+0000 7fd711d93640 1 -- 172.21.15.94:0/2011128899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 msgr2=0x5563f344aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.125+0000 7fd711d93640 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f344aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.125+0000 7fd711d93640 1 -- 172.21.15.94:0/2011128899 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f3372c30 con 0x5563f307e4d0 2024-04-25T13:59:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.126+0000 7fd711d93640 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7080137c0 tx=0x7fd708013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea27552dd02c770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.128+0000 7fd700ff9640 1 -- 172.21.15.94:0/2011128899 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708019040 con 0x5563f307e4d0 2024-04-25T13:59:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.128+0000 7fd700ff9640 1 -- 172.21.15.94:0/2011128899 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd708022070 con 0x5563f307e4d0 2024-04-25T13:59:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.128+0000 7fd700ff9640 1 -- 172.21.15.94:0/2011128899 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70801dd60 con 0x5563f307e4d0 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 -- 172.21.15.94:0/2011128899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 msgr2=0x5563f3449d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7080137c0 tx=0x7fd708013000 comp rx=0 tx=0).stop 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 -- 172.21.15.94:0/2011128899 shutdown_connections 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f344aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3449d90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 --2- 172.21.15.94:0/2011128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f3449810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.131+0000 7fd71b788740 1 -- 172.21.15.94:0/2011128899 >> 172.21.15.94:0/2011128899 conn(0x5563f309b4e0 msgr2=0x5563f3445020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.137+0000 7fd71b788740 1 -- 172.21.15.94:0/2011128899 shutdown_connections 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.137+0000 7fd71b788740 1 -- 172.21.15.94:0/2011128899 wait complete. 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.137+0000 7fd71b788740 1 -- start start 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.137+0000 7fd71b788740 1 Processor -- start 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f34538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f345a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5563f344b530 con 0x5563f30866b0 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd712594640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd712594640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56040/0 (socket says 172.21.15.94:56040) 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd712594640 1 -- 172.21.15.94:0/415380885 learned_addr learned my addr 172.21.15.94:0/415380885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563f3334dc0 con 0x5563f307e4d0 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563f344b6b0 con 0x5563f30a7390 2024-04-25T13:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd71b788740 10 client.0 _ll_register_callbacks cb 0x5563f3350600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.138+0000 7fd712d95640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f345a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 -- 172.21.15.94:0/415380885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 msgr2=0x5563f345a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f345a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 -- 172.21.15.94:0/415380885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 msgr2=0x5563f34538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f34538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 -- 172.21.15.94:0/415380885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f30bf3f0 con 0x5563f307e4d0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd712594640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0a5f90 tx=0x7fd6fc010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d5cb83b4f2d24a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.139+0000 7fd6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0c0060 con 0x5563f307e4d0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd6fc0a5470 con 0x5563f307e4d0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd708022070 con 0x5563f307e4d0 2024-04-25T13:59:51.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd6fc0c5050 con 0x5563f307e4d0 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd71b788740 10 client.31406 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.140+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5563f31897d0 con 0x5563f307e4d0 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.141+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc0a6700 con 0x5563f307e4d0 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd6fc0a4cd0 con 0x5563f307e4d0 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 10 client.31406 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 10 client.31406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 20 client.31406 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f3065f60 0x5563f3461160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 4 client.31406 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.142+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd708022070 con 0x5563f3065f60 2024-04-25T13:59:51.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.143+0000 7fd711d93640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f3065f60 0x5563f3461160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 99504 19.35 MB/sec execute 153 sec latency 105.298 ms 2024-04-25T13:59:51.146 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:50 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1716678253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:51.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.146+0000 7fd711d93640 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f3065f60 0x5563f3461160 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96f2e6e1eeba57 server_cookie=5fac84ed6fe55de3 in_seq=0 out_seq=0 2024-04-25T13:59:51.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.147+0000 7fd6f5ffb640 10 client.31406 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/415380885 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd708022070 con 0x5563f3065f60 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd6f5ffb640 10 client.31406 handle_command_reply: tid=0 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 client.31406 shutdown 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 msgr2=0x5563f34533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0a5f90 tx=0x7fd6fc010f00 comp rx=0 tx=0).stop 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 shutdown_connections 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f3065f60 0x5563f3461160 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30a7390 0x5563f345a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30866b0 0x5563f34538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 --2- 172.21.15.94:0/415380885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f34533b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.151+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 >> 172.21.15.94:0/415380885 conn(0x5563f309b4e0 msgr2=0x5563f3444040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.152+0000 7fd6f5ffb640 0 client.31406 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.152+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 shutdown_connections 2024-04-25T13:59:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.152+0000 7fd71b788740 1 -- 172.21.15.94:0/415380885 wait complete. 2024-04-25T13:59:51.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.152+0000 7fd71b788740 20 client.31406 trim_cache size 0 max 16384 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.172+0000 7fd71b788740 1 Processor -- start 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.172+0000 7fd71b788740 1 -- start start 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.172+0000 7fd71b788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.172+0000 7fd71b788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f34521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.172+0000 7fd71b788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f30a7390 0x5563f34449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd71b788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1aba90 con 0x5563f307e4d0 2024-04-25T13:59:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd71b788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1ad5d0 con 0x5563f30a7390 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51260/0 (socket says 172.21.15.94:51260) 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 -- 172.21.15.94:0/3966858500 learned_addr learned my addr 172.21.15.94:0/3966858500 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd711d93640 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f34521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712d95640 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f30a7390 0x5563f34449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1ac160 con 0x5563f30866b0 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 -- 172.21.15.94:0/3966858500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 msgr2=0x5563f34521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f34521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 -- 172.21.15.94:0/3966858500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f30a7390 msgr2=0x5563f34449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f30a7390 0x5563f34449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.173+0000 7fd712594640 1 -- 172.21.15.94:0/3966858500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f3449d70 con 0x5563f307e4d0 2024-04-25T13:59:51.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.174+0000 7fd712594640 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0a5eb0 tx=0x7fd6fc0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6f7e6771eb4640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.174+0000 7fd6f7fff640 1 -- 172.21.15.94:0/3966858500 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0c0070 con 0x5563f307e4d0 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.174+0000 7fd6f7fff640 1 -- 172.21.15.94:0/3966858500 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc0a6000 con 0x5563f307e4d0 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.174+0000 7fd6f7fff640 1 -- 172.21.15.94:0/3966858500 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0e2060 con 0x5563f307e4d0 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 msgr2=0x5563f3451cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0a5eb0 tx=0x7fd6fc0a5770 comp rx=0 tx=0).stop 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 shutdown_connections 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f30a7390 0x5563f34449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f34521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 --2- 172.21.15.94:0/3966858500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f307e4d0 0x5563f3451cb0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 >> 172.21.15.94:0/3966858500 conn(0x5563f309b4e0 msgr2=0x5563f344e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 shutdown_connections 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.175+0000 7fd71b788740 1 -- 172.21.15.94:0/3966858500 wait complete. 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 -- start start 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 Processor -- start 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3457030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f3457570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 0x5563f34584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1acf00 con 0x5563f30a7390 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1ac830 con 0x5563f307e4d0 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1a9b20 con 0x5563f30866b0 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd71b788740 10 client.0 _ll_register_callbacks cb 0x7fd6fc0e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:51.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.176+0000 7fd712594640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3457030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd712d95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 0x5563f34584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd712594640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3457030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56058/0 (socket says 172.21.15.94:56058) 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd712594640 1 -- 172.21.15.94:0/4235234885 learned_addr learned my addr 172.21.15.94:0/4235234885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f3457570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 -- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 msgr2=0x5563f3457030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3457030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 -- 172.21.15.94:0/4235234885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 msgr2=0x5563f34584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 0x5563f34584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 -- 172.21.15.94:0/4235234885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f3458a60 con 0x5563f30866b0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd712d95640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 0x5563f34584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd711d93640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f3457570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd708009f10 tx=0x7fd708006dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a165e97c4df2a496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708019040 con 0x5563f30866b0 2024-04-25T13:59:51.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6fc0a6000 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.177+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd7080059c0 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd70803f050 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 10 client.31072 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5563f3459d30 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc0a6000 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd708012070 con 0x5563f30866b0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 10 client.31072 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 10 client.31072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 20 client.31072 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f303b3a0 0x5563f34cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.178+0000 7fd71b788740 4 client.31072 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.179+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563f33bb990 con 0x5563f303b3a0 2024-04-25T13:59:51.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.179+0000 7fd712594640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f303b3a0 0x5563f34cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.179+0000 7fd712594640 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f303b3a0 0x5563f34cb580 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa01251dba558e99 server_cookie=220cc68ee14cf6ea in_seq=0 out_seq=0 2024-04-25T13:59:51.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.179+0000 7fd6f5ffb640 10 client.31072 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.183+0000 7fd6f5ffb640 1 -- 172.21.15.94:0/4235234885 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563f33bb990 con 0x5563f303b3a0 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.183+0000 7fd6f5ffb640 10 client.31072 handle_command_reply: tid=0 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.183+0000 7fd71b788740 1 client.31072 shutdown 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.183+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 msgr2=0x5563f3457570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.183+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f3457570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd708009f10 tx=0x7fd708006dd0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 shutdown_connections 2024-04-25T13:59:51.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5563f303b3a0 0x5563f34cb580 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563f30a7390 0x5563f34584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563f30866b0 0x5563f3457570 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 --2- 172.21.15.94:0/4235234885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563f307e4d0 0x5563f3457030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 >> 172.21.15.94:0/4235234885 conn(0x5563f309b4e0 msgr2=0x5563f3451cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd6f5ffb640 0 client.31072 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 shutdown_connections 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 1 -- 172.21.15.94:0/4235234885 wait complete. 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.184+0000 7fd71b788740 20 client.31072 trim_cache size 0 max 16384 2024-04-25T13:59:51.185 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 msgr2=0x7fd6f80b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 0x7fd6f80b2190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd70c1180b0 tx=0x7fd708006c20 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 msgr2=0x7fd70c117010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00d770 tx=0x7fd6fc00dc30 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 shutdown_connections 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6f80afc80 0x7fd6f80b2190 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c110030 0x7fd70c1184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10b310 0x7fd70c117550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 --2- 172.21.15.94:0/299857625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c10a420 0x7fd70c117010 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 >> 172.21.15.94:0/299857625 conn(0x7fd70c0704e0 msgr2=0x7fd70c10c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 shutdown_connections 2024-04-25T13:59:51.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:51.185+0000 7fd7147f8640 1 -- 172.21.15.94:0/299857625 wait complete. 2024-04-25T13:59:51.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:50 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1716678253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:52.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:51 smithi137 ceph-mon[32568]: pgmap v893: 97 pgs: 97 active+clean; 251 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-04-25T13:59:52.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 99884 19.37 MB/sec execute 154 sec latency 258.966 ms 2024-04-25T13:59:52.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:59:52.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:52.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:59:52.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:51 smithi094 ceph-mon[29195]: pgmap v893: 97 pgs: 97 active+clean; 251 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-04-25T13:59:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:51 smithi138 ceph-mon[32977]: pgmap v893: 97 pgs: 97 active+clean; 251 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-04-25T13:59:52.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 -- 172.21.15.94:0/3338223301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 msgr2=0x7f4218103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 --2- 172.21.15.94:0/3338223301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f4218103e80 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f4208009900 tx=0x7f420802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 -- 172.21.15.94:0/3338223301 shutdown_connections 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 --2- 172.21.15.94:0/3338223301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 0x7f421810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 --2- 172.21.15.94:0/3338223301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f4218103e80 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 --2- 172.21.15.94:0/3338223301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 0x7f4218103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 -- 172.21.15.94:0/3338223301 >> 172.21.15.94:0/3338223301 conn(0x7f42180fda40 msgr2=0x7f42180ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 -- 172.21.15.94:0/3338223301 shutdown_connections 2024-04-25T13:59:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.334+0000 7f421e99d640 1 -- 172.21.15.94:0/3338223301 wait complete. 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.335+0000 7f421e99d640 1 Processor -- start 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.335+0000 7f421e99d640 1 -- start start 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.335+0000 7f421e99d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 0x7f421819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e99d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e99d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 0x7f42181a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e99d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f421810e360 con 0x7f4218103a70 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e99d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f421810e1e0 con 0x7f4218102830 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e99d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f421810e4e0 con 0x7f4218108770 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51286/0 (socket says 172.21.15.94:51286) 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 -- 172.21.15.94:0/3386565721 learned_addr learned my addr 172.21.15.94:0/3386565721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d99b640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 0x7f421819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421e19c640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 0x7f42181a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 -- 172.21.15.94:0/3386565721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 msgr2=0x7f42181a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 0x7f42181a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 -- 172.21.15.94:0/3386565721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 msgr2=0x7f421819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 0x7f421819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 -- 172.21.15.94:0/3386565721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42181a1b60 con 0x7f4218103a70 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.336+0000 7f421d19a640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f42080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.337+0000 7f420effd640 1 -- 172.21.15.94:0/3386565721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420803f030 con 0x7f4218103a70 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.337+0000 7f420effd640 1 -- 172.21.15.94:0/3386565721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4208031910 con 0x7f4218103a70 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.337+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42181a1df0 con 0x7f4218103a70 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.337+0000 7f420effd640 1 -- 172.21.15.94:0/3386565721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208035070 con 0x7f4218103a70 2024-04-25T13:59:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.337+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42181a2220 con 0x7f4218103a70 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.340+0000 7f4226b86740 1 Processor -- start 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.340+0000 7f4226b86740 1 -- start start 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f420effd640 1 -- 172.21.15.94:0/3386565721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4208042030 con 0x7f4218103a70 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 0x55a92e158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a92e0bdb00 con 0x55a92ddb7410 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a92e059920 con 0x55a92dd8e550 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f4226b86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a92e0b9ac0 con 0x55a92dd96730 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421e19c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421e19c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51292/0 (socket says 172.21.15.94:51292) 2024-04-25T13:59:52.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421e19c640 1 -- 172.21.15.94:0/3428045699 learned_addr learned my addr 172.21.15.94:0/3428045699 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 0x55a92e158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d19a640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 -- 172.21.15.94:0/3428045699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 msgr2=0x55a92e15fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e15fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 -- 172.21.15.94:0/3428045699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 msgr2=0x55a92e158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 -- 172.21.15.94:0/3428045699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92de977e0 con 0x55a92dd8e550 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421e19c640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:52.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d19a640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e15fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f420effd640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 0x7f42040b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.341+0000 7f421d99b640 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 0x55a92e158be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4218102100 tx=0x7f4214011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8d9723d348f4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.342+0000 7f421e19c640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 0x7f42040b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.342+0000 7f420effd640 1 -- 172.21.15.94:0/3386565721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f42080ff1a0 con 0x7f4218103a70 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.342+0000 7f4203fff640 1 -- 172.21.15.94:0/3428045699 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214009d70 con 0x55a92dd8e550 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.342+0000 7f4203fff640 1 -- 172.21.15.94:0/3428045699 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4214005020 con 0x55a92dd8e550 2024-04-25T13:59:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.342+0000 7f4203fff640 1 -- 172.21.15.94:0/3428045699 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214004ad0 con 0x55a92dd8e550 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- 172.21.15.94:0/3428045699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 msgr2=0x55a92e158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f421e19c640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 0x7f42040b2140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55a92e159830 tx=0x7f4210009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 0x55a92e158be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4218102100 tx=0x7f4214011810 comp rx=0 tx=0).stop 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- 172.21.15.94:0/3428045699 shutdown_connections 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e15fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd8e550 0x55a92e158be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 --2- 172.21.15.94:0/3428045699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- 172.21.15.94:0/3428045699 >> 172.21.15.94:0/3428045699 conn(0x55a92ddab560 msgr2=0x55a92e153ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- 172.21.15.94:0/3428045699 shutdown_connections 2024-04-25T13:59:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- 172.21.15.94:0/3428045699 wait complete. 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.343+0000 7f4226b86740 1 -- start start 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 Processor -- start 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e166de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92ddb7410 0x55a92e16dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a92e160530 con 0x55a92dd8e550 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a92e0575d0 con 0x55a92dd96730 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a92e1606b0 con 0x55a92ddb7410 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f421d99b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f421d99b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51300/0 (socket says 172.21.15.94:51300) 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f421d99b640 1 -- 172.21.15.94:0/2917487456 learned_addr learned my addr 172.21.15.94:0/2917487456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f421d19a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e166de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f4226b86740 10 client.0 _ll_register_callbacks cb 0x55a92e045080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.344+0000 7f421e19c640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92ddb7410 0x55a92e16dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 -- 172.21.15.94:0/2917487456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92ddb7410 msgr2=0x55a92e16dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92ddb7410 0x55a92e16dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 -- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 msgr2=0x55a92e166de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e166de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 -- 172.21.15.94:0/2917487456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92def7f70 con 0x55a92dd8e550 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d19a640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e166de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f421d99b640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f421401b040 tx=0x7f4214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f68597cf9f74430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f42017fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214029020 con 0x55a92dd8e550 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4214005020 con 0x55a92dd8e550 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.345+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4214025470 con 0x55a92dd8e550 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4214034050 con 0x55a92dd8e550 2024-04-25T13:59:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 10 client.21993 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a92dd8f170 con 0x55a92dd8e550 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4214004e70 con 0x55a92dd8e550 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f42140194e0 con 0x55a92dd8e550 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 10 client.21993 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 10 client.21993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 20 client.21993 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd761f0 0x55a92e178b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 4 client.21993 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4214005020 con 0x55a92dd761f0 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.346+0000 7f421d19a640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd761f0 0x55a92e178b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.347+0000 7f42017fa640 10 client.21993 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.347+0000 7f421d19a640 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd761f0 0x55a92e178b20 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a252c64f477d6d9 server_cookie=cc8baac798c6d678 in_seq=0 out_seq=0 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f42017fa640 1 -- 172.21.15.94:0/2917487456 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4214005020 con 0x55a92dd761f0 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f42017fa640 10 client.21993 handle_command_reply: tid=0 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 client.21993 shutdown 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 msgr2=0x55a92e1668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f421401b040 tx=0x7f4214004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 shutdown_connections 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd761f0 0x55a92e178b20 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92ddb7410 0x55a92e16dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e166de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 --2- 172.21.15.94:0/2917487456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e1668a0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 >> 172.21.15.94:0/2917487456 conn(0x55a92ddab560 msgr2=0x55a92e152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f42017fa640 0 client.21993 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 shutdown_connections 2024-04-25T13:59:52.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.350+0000 7f4226b86740 1 -- 172.21.15.94:0/2917487456 wait complete. 2024-04-25T13:59:52.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.351+0000 7f4226b86740 20 client.21993 trim_cache size 0 max 16384 2024-04-25T13:59:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 Processor -- start 2024-04-25T13:59:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 -- start start 2024-04-25T13:59:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e16dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92ddb7410 0x55a92e152560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42181a9130 con 0x55a92dd8e550 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42181aac70 con 0x55a92ddb7410 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.377+0000 7f4226b86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42181a9800 con 0x55a92dd96730 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51312/0 (socket says 172.21.15.94:51312) 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 -- 172.21.15.94:0/1363211044 learned_addr learned my addr 172.21.15.94:0/1363211044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d19a640 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e16dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421e19c640 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92ddb7410 0x55a92e152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 -- 172.21.15.94:0/1363211044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 msgr2=0x55a92e16dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e16dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 -- 172.21.15.94:0/1363211044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92ddb7410 msgr2=0x55a92e152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92ddb7410 0x55a92e152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 -- 172.21.15.94:0/1363211044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92e15d340 con 0x55a92dd8e550 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f421d99b640 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f42140058c0 tx=0x7f42140096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa7b64e18c75dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f42037fe640 1 -- 172.21.15.94:0/1363211044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140097a0 con 0x55a92dd8e550 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f42037fe640 1 -- 172.21.15.94:0/1363211044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4214004e70 con 0x55a92dd8e550 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.378+0000 7f42037fe640 1 -- 172.21.15.94:0/1363211044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214025920 con 0x55a92dd8e550 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 -- 172.21.15.94:0/1363211044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 msgr2=0x55a92e161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f42140058c0 tx=0x7f42140096e0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 -- 172.21.15.94:0/1363211044 shutdown_connections 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92ddb7410 0x55a92e152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd96730 0x55a92e16dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 --2- 172.21.15.94:0/1363211044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92dd8e550 0x55a92e161160 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 -- 172.21.15.94:0/1363211044 >> 172.21.15.94:0/1363211044 conn(0x55a92ddab560 msgr2=0x55a92e160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 -- 172.21.15.94:0/1363211044 shutdown_connections 2024-04-25T13:59:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.379+0000 7f4226b86740 1 -- 172.21.15.94:0/1363211044 wait complete. 2024-04-25T13:59:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 -- start start 2024-04-25T13:59:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 Processor -- start 2024-04-25T13:59:52.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd8e550 0x55a92e16abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e16b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42181aa5a0 con 0x55a92ddb7410 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42181a9ed0 con 0x55a92dd96730 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42181a7110 con 0x55a92dd8e550 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f4226b86740 10 client.0 _ll_register_callbacks cb 0x7f4214005570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f421e19c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f421e19c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51322/0 (socket says 172.21.15.94:51322) 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f421e19c640 1 -- 172.21.15.94:0/4285657915 learned_addr learned my addr 172.21.15.94:0/4285657915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f421d99b640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd8e550 0x55a92e16abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.380+0000 7f421d19a640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e16b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 -- 172.21.15.94:0/4285657915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd8e550 msgr2=0x55a92e16abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd8e550 0x55a92e16abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 -- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 msgr2=0x55a92e16b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e16b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 -- 172.21.15.94:0/4285657915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92e1df1d0 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f421e19c640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f4210010f90 tx=0x7f4210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8546e4c7d028ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f42017fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421002e060 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f421001eab0 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.381+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4214004e70 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f421001ecb0 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 10 client.21999 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a92e1e0460 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4214004e70 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f421001d530 con 0x55a92ddb7410 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 10 client.21999 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 10 client.21999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 20 client.21999 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd4b630 0x55a92e1e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 4 client.21999 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.382+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a92e0cb710 con 0x55a92dd4b630 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.383+0000 7f421d99b640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd4b630 0x55a92e1e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.384+0000 7f421d99b640 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd4b630 0x55a92e1e2eb0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43021ef17473d1a4 server_cookie=9be2f152b119a670 in_seq=0 out_seq=0 2024-04-25T13:59:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.385+0000 7f42017fa640 10 client.21999 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f42017fa640 1 -- 172.21.15.94:0/4285657915 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a92e0cb710 con 0x55a92dd4b630 2024-04-25T13:59:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f42017fa640 10 client.21999 handle_command_reply: tid=0 2024-04-25T13:59:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 client.21999 shutdown 2024-04-25T13:59:52.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 msgr2=0x55a92e1dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f4210010f90 tx=0x7f4210004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 shutdown_connections 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a92dd4b630 0x55a92e1e2eb0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a92ddb7410 0x55a92e1dec90 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a92dd96730 0x55a92e16b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f42017fa640 0 client.21999 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 --2- 172.21.15.94:0/4285657915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a92dd8e550 0x55a92e16abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 >> 172.21.15.94:0/4285657915 conn(0x55a92ddab560 msgr2=0x55a92e166300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 shutdown_connections 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 1 -- 172.21.15.94:0/4285657915 wait complete. 2024-04-25T13:59:52.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.415+0000 7f4226b86740 20 client.21999 trim_cache size 0 max 16384 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "363120c3-6148-4e84-8dfd-2f11d6ce91ad", 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 msgr2=0x7f42040b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 0x7f42040b2140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55a92e159830 tx=0x7f4210009500 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 msgr2=0x7f421819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f42080042f0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 shutdown_connections 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f42040afc30 0x7f42040b2140 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4218108770 0x7f42181a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4218103a70 0x7f421819a560 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 --2- 172.21.15.94:0/3386565721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4218102830 0x7f421819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 >> 172.21.15.94:0/3386565721 conn(0x7f42180fda40 msgr2=0x7f42180ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 shutdown_connections 2024-04-25T13:59:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.416+0000 7f421e99d640 1 -- 172.21.15.94:0/3386565721 wait complete. 2024-04-25T13:59:52.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:52.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.587+0000 7fc0a34e2640 1 -- 172.21.15.94:0/3267247616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c1050b0 msgr2=0x7fc09c1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.587+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/3267247616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c1050b0 0x7fc09c1054c0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc098009900 tx=0x7fc09802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- 172.21.15.94:0/3267247616 shutdown_connections 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/3267247616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c102260 0x7fc09c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/3267247616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 0x7fc09c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/3267247616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c1050b0 0x7fc09c1054c0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- 172.21.15.94:0/3267247616 >> 172.21.15.94:0/3267247616 conn(0x7fc09c0fd380 msgr2=0x7fc09c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- 172.21.15.94:0/3267247616 shutdown_connections 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- 172.21.15.94:0/3267247616 wait complete. 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 Processor -- start 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- start start 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 0x7fc09c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c1050b0 0x7fc09c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c10f620 con 0x7fc09c102260 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c10f4a0 con 0x7fc09c1050b0 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.588+0000 7fc0a34e2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c10f7a0 con 0x7fc09c101760 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51340/0 (socket says 172.21.15.94:51340) 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/2195975931 learned_addr learned my addr 172.21.15.94:0/2195975931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a127e640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 0x7fc09c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c1050b0 0x7fc09c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/2195975931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 msgr2=0x7fc09c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 0x7fc09c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/2195975931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c1050b0 msgr2=0x7fc09c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c1050b0 0x7fc09c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/2195975931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c1a1db0 con 0x7fc09c102260 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e9f0 tx=0x7fc08c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0927fc640 1 -- 172.21.15.94:0/2195975931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00cd80 con 0x7fc09c102260 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c1a20a0 con 0x7fc09c102260 2024-04-25T13:59:52.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0927fc640 1 -- 172.21.15.94:0/2195975931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c0054d0 con 0x7fc09c102260 2024-04-25T13:59:52.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0927fc640 1 -- 172.21.15.94:0/2195975931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c010690 con 0x7fc09c102260 2024-04-25T13:59:52.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.589+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1a25a0 con 0x7fc09c102260 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0927fc640 1 -- 172.21.15.94:0/2195975931 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc08c004e90 con 0x7fc09c102260 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 Processor -- start 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 -- start start 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0927fc640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 0x7fc0880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 0x55815f8e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55815f849d80 con 0x55815f5226b0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55815f8038b0 con 0x55815f543390 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.592+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55815f845d40 con 0x55815f51a4d0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 0x7fc0880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0927fc640 1 -- 172.21.15.94:0/2195975931 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc08c0df0d0 con 0x7fc09c102260 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 0x55815f8e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 0x55815f8e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51350/0 (socket says 172.21.15.94:51350) 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a127e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a1a7f640 1 -- 172.21.15.94:0/1462214540 learned_addr learned my addr 172.21.15.94:0/1462214540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1462214540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 msgr2=0x55815f8e4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8e4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1462214540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 msgr2=0x55815f8e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 0x55815f8e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1462214540 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815f7d4a40 con 0x55815f51a4d0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 0x7fc0880b2000 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc094011d00 tx=0x7fc094011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8e4fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0dcea0 tx=0x7fc08c0a6f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4869a1dcfc95129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.593+0000 7fc0877fe640 1 -- 172.21.15.94:0/1462214540 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c04e730 con 0x55815f51a4d0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0877fe640 1 -- 172.21.15.94:0/1462214540 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c0a7020 con 0x55815f51a4d0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0877fe640 1 -- 172.21.15.94:0/1462214540 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0b0370 con 0x55815f51a4d0 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1462214540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 msgr2=0x55815f8e4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8e4fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0dcea0 tx=0x7fc08c0a6f20 comp rx=0 tx=0).stop 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1462214540 shutdown_connections 2024-04-25T13:59:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f5226b0 0x55815f8e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8e4fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1462214540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8e4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1462214540 >> 172.21.15.94:0/1462214540 conn(0x55815f5374e0 msgr2=0x55815f8e0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1462214540 shutdown_connections 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.594+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1462214540 wait complete. 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 -- start start 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 Processor -- start 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55815f8e6780 con 0x55815f51a4d0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55815f7d0d10 con 0x55815f543390 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55815f8e6900 con 0x55815f5226b0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0aa38d740 10 client.0 _ll_register_callbacks cb 0x55815f7fd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0a127e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0a127e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51366/0 (socket says 172.21.15.94:51366) 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0a127e640 1 -- 172.21.15.94:0/2895122213 learned_addr learned my addr 172.21.15.94:0/2895122213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.595+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 -- 172.21.15.94:0/2895122213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 msgr2=0x55815f8eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 -- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 msgr2=0x55815f8f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 -- 172.21.15.94:0/2895122213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815f6257d0 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0a127e640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fc098009230 tx=0x7fc098031dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e8d11d2051fb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc084ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098035040 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc098004040 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08c0a7020 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc098004240 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0aa38d740 10 client.22011 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.596+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55815f55b3f0 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c0a7020 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc098044020 con 0x55815f51a4d0 2024-04-25T13:59:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 10 client.22011 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 10 client.22011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 20 client.22011 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f501f60 0x55815f8fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 4 client.22011 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55815f857990 con 0x55815f501f60 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.597+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f501f60 0x55815f8fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.598+0000 7fc084ff9640 10 client.22011 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.598+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f501f60 0x55815f8fc260 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba290fa5d2e34421 server_cookie=8824164651d6996c in_seq=0 out_seq=0 2024-04-25T13:59:52.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.601+0000 7fc084ff9640 1 -- 172.21.15.94:0/2895122213 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55815f857990 con 0x55815f501f60 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.601+0000 7fc084ff9640 10 client.22011 handle_command_reply: tid=0 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.601+0000 7fc0aa38d740 1 client.22011 shutdown 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 msgr2=0x55815f8ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fc098009230 tx=0x7fc098031dd0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 shutdown_connections 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f501f60 0x55815f8fc260 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f543390 0x55815f8f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc084ff9640 0 client.22011 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/2895122213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f51a4d0 0x55815f8ee940 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 >> 172.21.15.94:0/2895122213 conn(0x55815f5374e0 msgr2=0x55815f8e3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 shutdown_connections 2024-04-25T13:59:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 1 -- 172.21.15.94:0/2895122213 wait complete. 2024-04-25T13:59:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.602+0000 7fc0aa38d740 20 client.22011 trim_cache size 0 max 16384 2024-04-25T13:59:52.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.620+0000 7fc0aa38d740 1 Processor -- start 2024-04-25T13:59:52.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.620+0000 7fc0aa38d740 1 -- start start 2024-04-25T13:59:52.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f5226b0 0x55815f8f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1ab580 con 0x55815f543390 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1ad0c0 con 0x55815f5226b0 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1abc50 con 0x55815f51a4d0 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51380/0 (socket says 172.21.15.94:51380) 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 -- 172.21.15.94:0/3567496666 learned_addr learned my addr 172.21.15.94:0/3567496666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a127e640 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f5226b0 0x55815f8f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 -- 172.21.15.94:0/3567496666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 msgr2=0x55815f8eec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8eec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 -- 172.21.15.94:0/3567496666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f5226b0 msgr2=0x55815f8f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f5226b0 0x55815f8f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 -- 172.21.15.94:0/3567496666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815f51b0f0 con 0x55815f543390 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fc0940117e0 tx=0x7fc094021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2751f04a4bd9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc086ffd640 1 -- 172.21.15.94:0/3567496666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09401ee10 con 0x55815f543390 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc086ffd640 1 -- 172.21.15.94:0/3567496666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc094011d30 con 0x55815f543390 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.621+0000 7fc086ffd640 1 -- 172.21.15.94:0/3567496666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094005780 con 0x55815f543390 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 -- 172.21.15.94:0/3567496666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 msgr2=0x55815f8f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fc0940117e0 tx=0x7fc094021000 comp rx=0 tx=0).stop 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 -- 172.21.15.94:0/3567496666 shutdown_connections 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f8f6240 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f5226b0 0x55815f8f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/3567496666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f51a4d0 0x55815f8eec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 -- 172.21.15.94:0/3567496666 >> 172.21.15.94:0/3567496666 conn(0x55815f5374e0 msgr2=0x55815f8ee510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 -- 172.21.15.94:0/3567496666 shutdown_connections 2024-04-25T13:59:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.622+0000 7fc0aa38d740 1 -- 172.21.15.94:0/3567496666 wait complete. 2024-04-25T13:59:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 -- start start 2024-04-25T13:59:52.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 Processor -- start 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1ac9f0 con 0x55815f543390 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1ac320 con 0x55815f51a4d0 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1a9560 con 0x55815f5226b0 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0aa38d740 10 client.0 _ll_register_callbacks cb 0x7fc098009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a127e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a0a7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a127e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56168/0 (socket says 172.21.15.94:56168) 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a1a7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51388/0 (socket says 172.21.15.94:51388) 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.623+0000 7fc0a127e640 1 -- 172.21.15.94:0/1779699842 learned_addr learned my addr 172.21.15.94:0/1779699842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 msgr2=0x55815f8f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1779699842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 msgr2=0x55815f9625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 -- 172.21.15.94:0/1779699842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55815f80a4b0 con 0x55815f5226b0 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a127e640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a1a7f640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:52.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0a0a7d640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8f3960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0dcb30 tx=0x7fc08c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7294b3d137d84f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc084ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0c8070 con 0x55815f5226b0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc08c0a3b00 con 0x55815f5226b0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc094011d30 con 0x55815f5226b0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc08c005180 con 0x55815f5226b0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0aa38d740 10 client.31105 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55815f963910 con 0x55815f5226b0 2024-04-25T13:59:52.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.624+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc094011d30 con 0x55815f5226b0 2024-04-25T13:59:52.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc08c0a37a0 con 0x55815f5226b0 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 10 client.31105 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 10 client.31105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 20 client.31105 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f4d73a0 0x55815f965e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 4 client.31105 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55815f851800 con 0x55815f4d73a0 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0a127e640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f4d73a0 0x55815f965e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc084ff9640 10 client.31105 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.625+0000 7fc0a127e640 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f4d73a0 0x55815f965e90 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699aa613a9b1b0b4 server_cookie=8af9327030da3e78 in_seq=0 out_seq=0 2024-04-25T13:59:52.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc084ff9640 1 -- 172.21.15.94:0/1779699842 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55815f851800 con 0x55815f4d73a0 2024-04-25T13:59:52.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc084ff9640 10 client.31105 handle_command_reply: tid=0 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 client.31105 shutdown 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 msgr2=0x55815f8f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8f3960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0dcb30 tx=0x7fc08c009310 comp rx=0 tx=0).stop 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 shutdown_connections 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55815f4d73a0 0x55815f965e90 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55815f543390 0x55815f9625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55815f5226b0 0x55815f8f3960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 --2- 172.21.15.94:0/1779699842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55815f51a4d0 0x55815f8f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc084ff9640 0 client.31105 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 >> 172.21.15.94:0/1779699842 conn(0x55815f5374e0 msgr2=0x55815f8de9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 shutdown_connections 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 1 -- 172.21.15.94:0/1779699842 wait complete. 2024-04-25T13:59:52.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.630+0000 7fc0aa38d740 20 client.31105 trim_cache size 0 max 16384 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (229 inodes in the stack)", 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "363120c3-6148-4e84-8dfd-2f11d6ce91ad": { 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "363120c3-6148-4e84-8dfd-2f11d6ce91ad", 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T13:59:52.638 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T13:59:52.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 msgr2=0x7fc0880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 0x7fc0880b2000 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc094011d00 tx=0x7fc094011440 comp rx=0 tx=0).stop 2024-04-25T13:59:52.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 msgr2=0x7fc09c19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:52.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e9f0 tx=0x7fc08c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 shutdown_connections 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc0880afaf0 0x7fc0880b2000 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc09c1050b0 0x7fc09c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc09c102260 0x7fc09c19a7b0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 --2- 172.21.15.94:0/2195975931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc09c101760 0x7fc09c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 >> 172.21.15.94:0/2195975931 conn(0x7fc09c0fd380 msgr2=0x7fc09c105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 shutdown_connections 2024-04-25T13:59:52.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T13:59:52.631+0000 7fc0a34e2640 1 -- 172.21.15.94:0/2195975931 wait complete. 2024-04-25T13:59:52.641 INFO:tasks.cephfs.filesystem:scrub status for tag:363120c3-6148-4e84-8dfd-2f11d6ce91ad - {'path': '/', 'tag': '363120c3-6148-4e84-8dfd-2f11d6ce91ad', 'options': 'recursive,force'} 2024-04-25T13:59:53.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:52 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T13:59:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:52 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:52 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:53.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:52 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.143 INFO:tasks.workunit.client.0.smithi094.stdout: 10 100167 19.32 MB/sec execute 155 sec latency 147.052 ms 2024-04-25T13:59:53.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:52 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T13:59:53.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:52 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:53.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:52 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:53.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:52 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:52 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T13:59:53.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:52 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:53.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:52 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:53.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:52 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- 172.21.15.94:0/360201480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041080f0 msgr2=0x7f530410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- 172.21.15.94:0/360201480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041080f0 0x7f530410a510 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f5300009900 tx=0x7f530002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- 172.21.15.94:0/360201480 shutdown_connections 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- 172.21.15.94:0/360201480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041080f0 0x7f530410a510 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- 172.21.15.94:0/360201480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041033f0 0x7f5304103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- 172.21.15.94:0/360201480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041021b0 0x7f5304102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- 172.21.15.94:0/360201480 >> 172.21.15.94:0/360201480 conn(0x7f53040fd380 msgr2=0x7f53040ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- 172.21.15.94:0/360201480 shutdown_connections 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- 172.21.15.94:0/360201480 wait complete. 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 Processor -- start 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 -- start start 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.578+0000 7f530b300640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041080f0 0x7f53041a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530b300640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f530410dcd0 con 0x7f53041033f0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530b300640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f530410db50 con 0x7f53041021b0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530b300640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f530410de50 con 0x7f53041080f0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530909c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530909c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35486/0 (socket says 172.21.15.94:35486) 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530909c640 1 -- 172.21.15.94:0/1133138290 learned_addr learned my addr 172.21.15.94:0/1133138290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59854/0 (socket says 172.21.15.94:59854) 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530989d640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041080f0 0x7f53041a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 -- 172.21.15.94:0/1133138290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041080f0 msgr2=0x7f53041a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041080f0 0x7f53041a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 -- 172.21.15.94:0/1133138290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 msgr2=0x7f530410f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 -- 172.21.15.94:0/1133138290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53041a1cc0 con 0x7f53041033f0 2024-04-25T13:59:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530909c640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530889b640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e9f0 tx=0x7f52f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f52fa7fc640 1 -- 172.21.15.94:0/1133138290 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400ccc0 con 0x7f53041033f0 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a1fb0 con 0x7f53041033f0 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a2470 con 0x7f53041033f0 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f52fa7fc640 1 -- 172.21.15.94:0/1133138290 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f40108e0 con 0x7f53041033f0 2024-04-25T13:59:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.579+0000 7f52fa7fc640 1 -- 172.21.15.94:0/1133138290 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4018880 con 0x7f53041033f0 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.582+0000 7f53121b1740 1 Processor -- start 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.582+0000 7f53121b1740 1 -- start start 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184e41b0 0x563818884970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184c34d0 0x56381888bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5638187ea460 con 0x5638184bb2f0 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5638187a4590 con 0x5638184c34d0 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f53121b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5638187e6420 con 0x5638184e41b0 2024-04-25T13:59:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59860/0 (socket says 172.21.15.94:59860) 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 -- 172.21.15.94:0/1605094840 learned_addr learned my addr 172.21.15.94:0/1605094840 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f52fa7fc640 1 -- 172.21.15.94:0/1133138290 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f52f4018a20 con 0x7f53041033f0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530989d640 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184e41b0 0x563818884970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 -- 172.21.15.94:0/1605094840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184e41b0 msgr2=0x563818884970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184e41b0 0x563818884970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 -- 172.21.15.94:0/1605094840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184c34d0 msgr2=0x56381888bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184c34d0 0x56381888bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 -- 172.21.15.94:0/1605094840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638187b10d0 con 0x5638184bb2f0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.583+0000 7f530909c640 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009a20 tx=0x7f52fc011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e9ddb5621110d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1605094840 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc01e070 con 0x5638184bb2f0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1605094840 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc00fe70 con 0x5638184bb2f0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f52ef7fe640 1 -- 172.21.15.94:0/1605094840 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc019d10 con 0x5638184bb2f0 2024-04-25T13:59:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f52fa7fc640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 0x7f52f00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f530989d640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 0x7f52f00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f52fa7fc640 1 -- 172.21.15.94:0/1133138290 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f52f40defd0 con 0x7f53041033f0 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f53121b1740 1 -- 172.21.15.94:0/1605094840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 msgr2=0x563818884ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f530989d640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 0x7f52f00b1ff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5300031440 tx=0x7f53000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.584+0000 7f53121b1740 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009a20 tx=0x7f52fc011d90 comp rx=0 tx=0).stop 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- 172.21.15.94:0/1605094840 shutdown_connections 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184c34d0 0x56381888bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x563818884ef0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- 172.21.15.94:0/1605094840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184e41b0 0x563818884970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- 172.21.15.94:0/1605094840 >> 172.21.15.94:0/1605094840 conn(0x5638184d8300 msgr2=0x563818880300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- 172.21.15.94:0/1605094840 shutdown_connections 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- 172.21.15.94:0/1605094840 wait complete. 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- start start 2024-04-25T13:59:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 Processor -- start 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 0x56381888ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818895b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56381888c840 con 0x5638184bb2f0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563818771b60 con 0x5638184e41b0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.585+0000 7f53121b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56381888c9c0 con 0x5638184c34d0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f53121b1740 10 client.0 _ll_register_callbacks cb 0x5638187962b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530909c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530909c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59866/0 (socket says 172.21.15.94:59866) 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530909c640 1 -- 172.21.15.94:0/4195664852 learned_addr learned my addr 172.21.15.94:0/4195664852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 0x56381888ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530989d640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818895b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 -- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 msgr2=0x563818895b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818895b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 -- 172.21.15.94:0/4195664852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 msgr2=0x56381888e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 -- 172.21.15.94:0/4195664852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638184fc210 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530909c640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f530889b640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 0x56381888ebe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52f40dca80 tx=0x7f52f40a6f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a84eba94f0448c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f52ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab7c0 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f52f40e1110 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc00fe70 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.586+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f52f40b0270 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 10 client.31117 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638185c8620 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52fc00fe70 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f52f40a2e30 con 0x5638184c34d0 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 10 client.31117 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 10 client.31117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 20 client.31117 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184a2d80 0x56381889c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 4 client.31117 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638187f8070 con 0x5638184a2d80 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.587+0000 7f530909c640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184a2d80 0x56381889c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.588+0000 7f52ecff9640 10 client.31117 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.588+0000 7f530909c640 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184a2d80 0x56381889c2e0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb375f0773a6227d server_cookie=4b42faf107c9c10 in_seq=0 out_seq=0 2024-04-25T13:59:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f52ecff9640 1 -- 172.21.15.94:0/4195664852 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638187f8070 con 0x5638184a2d80 2024-04-25T13:59:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f52ecff9640 10 client.31117 handle_command_reply: tid=0 2024-04-25T13:59:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 client.31117 shutdown 2024-04-25T13:59:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 msgr2=0x56381888ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 0x56381888ebe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52f40dca80 tx=0x7f52f40a6f10 comp rx=0 tx=0).stop 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 shutdown_connections 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184a2d80 0x56381889c2e0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818895b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f52ecff9640 0 client.31117 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184c34d0 0x56381888ebe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 --2- 172.21.15.94:0/4195664852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184bb2f0 0x56381888e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 >> 172.21.15.94:0/4195664852 conn(0x5638184d8300 msgr2=0x563818883210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 shutdown_connections 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 1 -- 172.21.15.94:0/4195664852 wait complete. 2024-04-25T13:59:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.590+0000 7f53121b1740 20 client.31117 trim_cache size 0 max 16384 2024-04-25T13:59:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 Processor -- start 2024-04-25T13:59:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 -- start start 2024-04-25T13:59:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x56381887fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818883100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ab480 con 0x5638184c34d0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53041acfc0 con 0x5638184e41b0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f53121b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53041abb50 con 0x5638184bb2f0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59872/0 (socket says 172.21.15.94:59872) 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f530889b640 1 -- 172.21.15.94:0/109565548 learned_addr learned my addr 172.21.15.94:0/109565548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f530909c640 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x56381887fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.606+0000 7f530989d640 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818883100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 -- 172.21.15.94:0/109565548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 msgr2=0x56381887fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x56381887fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 -- 172.21.15.94:0/109565548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 msgr2=0x563818883100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818883100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 -- 172.21.15.94:0/109565548 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563818889540 con 0x5638184c34d0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f530889b640 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a4730 tx=0x7f52f400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41a301ad6333345 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f52eeffd640 1 -- 172.21.15.94:0/109565548 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab7c0 con 0x5638184c34d0 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f52eeffd640 1 -- 172.21.15.94:0/109565548 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f40a6720 con 0x5638184c34d0 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.607+0000 7f52eeffd640 1 -- 172.21.15.94:0/109565548 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40a6440 con 0x5638184c34d0 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- 172.21.15.94:0/109565548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 msgr2=0x5638188801b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a4730 tx=0x7f52f400ce60 comp rx=0 tx=0).stop 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- 172.21.15.94:0/109565548 shutdown_connections 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818883100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x5638188801b0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 --2- 172.21.15.94:0/109565548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x56381887fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- 172.21.15.94:0/109565548 >> 172.21.15.94:0/109565548 conn(0x5638184d8300 msgr2=0x563818887bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- 172.21.15.94:0/109565548 shutdown_connections 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- 172.21.15.94:0/109565548 wait complete. 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 -- start start 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.608+0000 7f53121b1740 1 Processor -- start 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x563818892670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818893ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ac8f0 con 0x5638184c34d0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53041ac220 con 0x5638184e41b0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53041a9460 con 0x5638184bb2f0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 10 client.0 _ll_register_callbacks cb 0x7f52f400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59876/0 (socket says 172.21.15.94:59876) 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 -- 172.21.15.94:0/3097552012 learned_addr learned my addr 172.21.15.94:0/3097552012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530989d640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818893ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530909c640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x563818892670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 -- 172.21.15.94:0/3097552012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 msgr2=0x563818892670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x563818892670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 -- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 msgr2=0x563818893ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818893ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 -- 172.21.15.94:0/3097552012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638188940a0 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530909c640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x563818892670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f530889b640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a3540 tx=0x7f52f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559b96735d4d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f52ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40c5020 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f52f40c8330 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.609+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f40a6720 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f52f40a7290 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 10 client.22035 fetch_fsmap learned FSMap version 39 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563818895330 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f40a6720 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f52f40a63a0 con 0x5638184c34d0 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 10 client.22035 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 10 client.22035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 20 client.22035 populate_metadata read hostname 'smithi094' 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184781c0 0x56381890af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 4 client.22035 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638187f1ee0 con 0x5638184781c0 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.610+0000 7f530909c640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184781c0 0x56381890af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.611+0000 7f52ecff9640 10 client.22035 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:53.611+0000 7f530909c640 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184781c0 0x56381890af40 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4782e4ec26fc402b server_cookie=76c8b97a0cf0cd5a in_seq=0 out_seq=0 2024-04-25T13:59:54.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 100379 19.27 MB/sec execute 156 sec latency 358.232 ms 2024-04-25T13:59:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f52ecff9640 1 -- 172.21.15.94:0/3097552012 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638187f1ee0 con 0x5638184781c0 2024-04-25T13:59:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f52ecff9640 10 client.22035 handle_command_reply: tid=0 2024-04-25T13:59:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 client.22035 shutdown 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 msgr2=0x563818892bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a3540 tx=0x7f52f4010740 comp rx=0 tx=0).stop 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 shutdown_connections 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5638184781c0 0x56381890af40 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f52ecff9640 0 client.22035 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638184e41b0 0x563818893ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638184c34d0 0x563818892bb0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 --2- 172.21.15.94:0/3097552012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638184bb2f0 0x563818892670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 >> 172.21.15.94:0/3097552012 conn(0x5638184d8300 msgr2=0x563818883100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 shutdown_connections 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.203+0000 7f53121b1740 1 -- 172.21.15.94:0/3097552012 wait complete. 2024-04-25T13:59:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.204+0000 7f53121b1740 20 client.22035 trim_cache size 0 max 16384 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.204+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 msgr2=0x7f52f00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.204+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 0x7f52f00b1ff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5300031440 tx=0x7f53000314a0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.204+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 msgr2=0x7f530419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.204+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e9f0 tx=0x7f52f400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 shutdown_connections 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f52f00afae0 0x7f52f00b1ff0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53041080f0 0x7f53041a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53041033f0 0x7f530419ef50 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 --2- 172.21.15.94:0/1133138290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53041021b0 0x7f530410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 >> 172.21.15.94:0/1133138290 conn(0x7f53040fd380 msgr2=0x7f5304108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 shutdown_connections 2024-04-25T13:59:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T13:59:54.205+0000 7f530b300640 1 -- 172.21.15.94:0/1133138290 wait complete. 2024-04-25T13:59:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: pgmap v894: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-04-25T13:59:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:53 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: pgmap v894: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:53 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: pgmap v894: 97 pgs: 97 active+clean; 255 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 451 op/s 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T13:59:54.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:54.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:53 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:55.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 100728 19.31 MB/sec execute 157 sec latency 370.831 ms 2024-04-25T13:59:55.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:54 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:55.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:54 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:55.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:54 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:56.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 101084 19.30 MB/sec execute 158 sec latency 97.267 ms 2024-04-25T13:59:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:55 smithi094 ceph-mon[29195]: pgmap v895: 97 pgs: 97 active+clean; 258 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-25T13:59:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:55 smithi138 ceph-mon[32977]: pgmap v895: 97 pgs: 97 active+clean; 258 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-25T13:59:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:55 smithi137 ceph-mon[32568]: pgmap v895: 97 pgs: 97 active+clean; 258 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-25T13:59:57.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 101372 19.27 MB/sec execute 159 sec latency 230.754 ms 2024-04-25T13:59:58.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 101674 19.24 MB/sec execute 160 sec latency 112.546 ms 2024-04-25T13:59:58.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:57 smithi094 ceph-mon[29195]: pgmap v896: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-04-25T13:59:58.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:57 smithi138 ceph-mon[32977]: pgmap v896: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-04-25T13:59:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:57 smithi137 ceph-mon[32568]: pgmap v896: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-04-25T13:59:59.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 102095 19.24 MB/sec execute 161 sec latency 76.793 ms 2024-04-25T14:00:00.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 102531 19.27 MB/sec execute 162 sec latency 85.387 ms 2024-04-25T14:00:00.193 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 13:59:59 smithi137 ceph-mon[32568]: pgmap v897: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-04-25T14:00:00.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 13:59:59 smithi094 ceph-mon[29195]: pgmap v897: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-04-25T14:00:00.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 13:59:59 smithi138 ceph-mon[32977]: pgmap v897: 97 pgs: 97 active+clean; 257 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-04-25T14:00:01.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 103015 19.33 MB/sec execute 163 sec latency 62.877 ms 2024-04-25T14:00:01.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:00 smithi094 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T14:00:01.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:00 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:01.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:00 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T14:00:01.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:00 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:01.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:00 smithi137 ceph-mon[32568]: overall HEALTH_OK 2024-04-25T14:00:01.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:00 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:02.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 103326 19.29 MB/sec execute 164 sec latency 174.037 ms 2024-04-25T14:00:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:01 smithi094 ceph-mon[29195]: pgmap v898: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-04-25T14:00:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:01 smithi138 ceph-mon[32977]: pgmap v898: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-04-25T14:00:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:02.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:01 smithi137 ceph-mon[32568]: pgmap v898: 97 pgs: 97 active+clean; 249 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-04-25T14:00:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:02.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:02.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:03.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 103696 19.31 MB/sec execute 165 sec latency 186.068 ms 2024-04-25T14:00:04.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 104085 19.34 MB/sec execute 166 sec latency 77.070 ms 2024-04-25T14:00:04.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:03 smithi094 ceph-mon[29195]: pgmap v899: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-04-25T14:00:04.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:03 smithi138 ceph-mon[32977]: pgmap v899: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-04-25T14:00:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:03 smithi137 ceph-mon[32568]: pgmap v899: 97 pgs: 97 active+clean; 250 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-04-25T14:00:05.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 104443 19.36 MB/sec execute 167 sec latency 64.604 ms 2024-04-25T14:00:06.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 104750 19.31 MB/sec execute 168 sec latency 102.463 ms 2024-04-25T14:00:06.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:05 smithi094 ceph-mon[29195]: pgmap v900: 97 pgs: 97 active+clean; 255 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-04-25T14:00:06.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:05 smithi138 ceph-mon[32977]: pgmap v900: 97 pgs: 97 active+clean; 255 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-04-25T14:00:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:05 smithi137 ceph-mon[32568]: pgmap v900: 97 pgs: 97 active+clean; 255 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 645 op/s 2024-04-25T14:00:07.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 105003 19.29 MB/sec execute 169 sec latency 268.682 ms 2024-04-25T14:00:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2858890016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 msgr2=0x7fc1b0104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2858890016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b0104840 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a410 comp rx=0 tx=0).stop 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2858890016 shutdown_connections 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2858890016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 0x7fc1b010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2858890016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 0x7fc1b0105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2858890016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b0104840 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2858890016 >> 172.21.15.94:0/2858890016 conn(0x7fc1b00fed30 msgr2=0x7fc1b01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2858890016 shutdown_connections 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.314+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2858890016 wait complete. 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 Processor -- start 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 -- start start 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 0x7fc1b019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 0x7fc1b01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010f690 con 0x7fc1b0103b40 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010f510 con 0x7fc1b0109a80 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b69b7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010f810 con 0x7fc1b0104d80 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41424/0 (socket says 172.21.15.94:41424) 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1affff640 1 -- 172.21.15.94:0/2491471019 learned_addr learned my addr 172.21.15.94:0/2491471019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 0x7fc1b019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.315+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 0x7fc1b01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 -- 172.21.15.94:0/2491471019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 msgr2=0x7fc1b019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 0x7fc1b019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 -- 172.21.15.94:0/2491471019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 msgr2=0x7fc1b01a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 0x7fc1b01a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 -- 172.21.15.94:0/2491471019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b01a1dc0 con 0x7fc1b0103b40 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1affff640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0031440 tx=0x7fc1a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1ad7fa640 1 -- 172.21.15.94:0/2491471019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a003f070 con 0x7fc1b0103b40 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b01a2050 con 0x7fc1b0103b40 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1ad7fa640 1 -- 172.21.15.94:0/2491471019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0031c50 con 0x7fc1b0103b40 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1ad7fa640 1 -- 172.21.15.94:0/2491471019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035030 con 0x7fc1b0103b40 2024-04-25T14:00:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.316+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01a2540 con 0x7fc1b0103b40 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.318+0000 7fc1ad7fa640 1 -- 172.21.15.94:0/2491471019 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc1a0042070 con 0x7fc1b0103b40 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 Processor -- start 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 -- start start 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 0x55ca0653d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 0x55ca06544df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca064a3460 con 0x55ca061742f0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca0645d590 con 0x55ca0617c4d0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca0649f420 con 0x55ca0619d1b0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41430/0 (socket says 172.21.15.94:41430) 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 -- 172.21.15.94:0/1174161221 learned_addr learned my addr 172.21.15.94:0/1174161221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 0x55ca06544df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 0x55ca0653d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1ad7fa640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 0x7fc1980b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 0x7fc1980b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 -- 172.21.15.94:0/1174161221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 msgr2=0x55ca0653d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 0x55ca0653d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 -- 172.21.15.94:0/1174161221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 msgr2=0x55ca06544df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 0x55ca06544df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1affff640 1 -- 172.21.15.94:0/1174161221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0646a0d0 con 0x55ca061742f0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 0x55ca06544df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.319+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 0x55ca0653d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1ad7fa640 1 -- 172.21.15.94:0/2491471019 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1a00ff3d0 con 0x7fc1b0103b40 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1affff640 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00c7850 tx=0x7fc1a004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cab6d24a268f1c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 0x7fc1980b1ff0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55ca0653eb40 tx=0x7fc1a8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1967fc640 1 -- 172.21.15.94:0/1174161221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0031e00 con 0x55ca061742f0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1967fc640 1 -- 172.21.15.94:0/1174161221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a00c73d0 con 0x55ca061742f0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.320+0000 7fc1967fc640 1 -- 172.21.15.94:0/1174161221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00cbbc0 con 0x55ca061742f0 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1174161221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 msgr2=0x55ca0653def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00c7850 tx=0x7fc1a004b000 comp rx=0 tx=0).stop 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1174161221 shutdown_connections 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0617c4d0 0x55ca06544df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0653def0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1174161221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0619d1b0 0x55ca0653d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1174161221 >> 172.21.15.94:0/1174161221 conn(0x55ca06191300 msgr2=0x55ca06539300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1174161221 shutdown_connections 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1174161221 wait complete. 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 -- start start 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.321+0000 7fc1bd86d740 1 Processor -- start 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 0x55ca065477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0619d1b0 0x55ca0654e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca064a3460 con 0x55ca0617c4d0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ca0642ab60 con 0x55ca0619d1b0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca06545600 con 0x55ca061742f0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1bd86d740 10 client.0 _ll_register_callbacks cb 0x55ca0644f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 0x55ca065477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53600/0 (socket says 172.21.15.94:53600) 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 0x55ca065477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41438/0 (socket says 172.21.15.94:41438) 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 -- 172.21.15.94:0/556840001 learned_addr learned my addr 172.21.15.94:0/556840001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0619d1b0 0x55ca0654e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 -- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0619d1b0 msgr2=0x55ca0654e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0619d1b0 0x55ca0654e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 -- 172.21.15.94:0/556840001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 msgr2=0x55ca065477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 0x55ca065477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.322+0000 7fc1affff640 1 -- 172.21.15.94:0/556840001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca061b5210 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1affff640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00fd1a0 tx=0x7fc1a002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60954c32d1278969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0049040 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1a00ea0d0 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a00c73d0 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1a0031e00 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 10 client.31135 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca06281620 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a00c73d0 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc1a00fe070 con 0x55ca061742f0 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 10 client.31135 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 10 client.31135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.323+0000 7fc1bd86d740 20 client.31135 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0615bd80 0x55ca06554bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc1bd86d740 4 client.31135 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca064b1070 con 0x55ca0615bd80 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0615bd80 0x55ca06554bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc18f7fe640 10 client.31135 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.324+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0615bd80 0x55ca06554bd0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef6c20b38a161d5 server_cookie=ee855effb22dfb27 in_seq=0 out_seq=0 2024-04-25T14:00:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.327+0000 7fc18f7fe640 1 -- 172.21.15.94:0/556840001 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca064b1070 con 0x55ca0615bd80 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.327+0000 7fc18f7fe640 10 client.31135 handle_command_reply: tid=0 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.327+0000 7fc1bd86d740 1 client.31135 shutdown 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.327+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 msgr2=0x55ca06547290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.327+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00fd1a0 tx=0x7fc1a002a410 comp rx=0 tx=0).stop 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 shutdown_connections 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0615bd80 0x55ca06554bd0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc18f7fe640 0 client.31135 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0619d1b0 0x55ca0654e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca0617c4d0 0x55ca065477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/556840001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca061742f0 0x55ca06547290 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 >> 172.21.15.94:0/556840001 conn(0x55ca06191300 msgr2=0x55ca0653c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 shutdown_connections 2024-04-25T14:00:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 1 -- 172.21.15.94:0/556840001 wait complete. 2024-04-25T14:00:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.328+0000 7fc1bd86d740 20 client.31135 trim_cache size 0 max 16384 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 Processor -- start 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 -- start start 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0653c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca06546870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01ab610 con 0x55ca061742f0 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01ad150 con 0x55ca0615bd80 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01abce0 con 0x55ca0617c4d0 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41452/0 (socket says 172.21.15.94:41452) 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1af7fe640 1 -- 172.21.15.94:0/214472762 learned_addr learned my addr 172.21.15.94:0/214472762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.345+0000 7fc1affff640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0653c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca06546870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 -- 172.21.15.94:0/214472762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 msgr2=0x55ca06546870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca06546870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 -- 172.21.15.94:0/214472762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 msgr2=0x55ca0653c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0653c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 -- 172.21.15.94:0/214472762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0653d480 con 0x55ca061742f0 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca06546870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x55ca0653a090 tx=0x7fc1a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2329df8a982ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc195ffb640 1 -- 172.21.15.94:0/214472762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4005020 con 0x55ca061742f0 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc195ffb640 1 -- 172.21.15.94:0/214472762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a4024030 con 0x55ca061742f0 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.346+0000 7fc195ffb640 1 -- 172.21.15.94:0/214472762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4004a00 con 0x55ca061742f0 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- 172.21.15.94:0/214472762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 msgr2=0x55ca0654e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x55ca0653a090 tx=0x7fc1a4005f90 comp rx=0 tx=0).stop 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- 172.21.15.94:0/214472762 shutdown_connections 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca06546870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654e650 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/214472762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0653c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- 172.21.15.94:0/214472762 >> 172.21.15.94:0/214472762 conn(0x55ca06191300 msgr2=0x55ca0653c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- 172.21.15.94:0/214472762 shutdown_connections 2024-04-25T14:00:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- 172.21.15.94:0/214472762 wait complete. 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 -- start start 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.347+0000 7fc1bd86d740 1 Processor -- start 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0654beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca0654d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01aca80 con 0x55ca061742f0 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01ac3b0 con 0x55ca0615bd80 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01a95f0 con 0x55ca0617c4d0 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1bd86d740 10 client.0 _ll_register_callbacks cb 0x7fc1a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41462/0 (socket says 172.21.15.94:41462) 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 -- 172.21.15.94:0/1883376355 learned_addr learned my addr 172.21.15.94:0/1883376355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1affff640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0654beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1b4f54640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca0654d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 -- 172.21.15.94:0/1883376355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 msgr2=0x55ca0654d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca0654d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 -- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 msgr2=0x55ca0654beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0654beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 -- 172.21.15.94:0/1883376355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca06281620 con 0x55ca061742f0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc1af7fe640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4023040 tx=0x7fc1a400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4847005739b3eee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.348+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40122d0 con 0x55ca061742f0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1a4005020 con 0x55ca061742f0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a4024030 con 0x55ca061742f0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1a4005220 con 0x55ca061742f0 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 10 client.22053 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca0654e8d0 con 0x55ca061742f0 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a4024030 con 0x55ca061742f0 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc1a40344e0 con 0x55ca061742f0 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 10 client.22053 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 10 client.22053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 20 client.22053 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0619d1b0 0x55ca065bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 4 client.22053 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca064aaee0 con 0x55ca0619d1b0 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.349+0000 7fc1affff640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0619d1b0 0x55ca065bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.350+0000 7fc18f7fe640 10 client.22053 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:07.350+0000 7fc1affff640 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0619d1b0 0x55ca065bf5f0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fb7b288ab8b200 server_cookie=8bb2334311c07534 in_seq=0 out_seq=0 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.057+0000 7fc18f7fe640 1 -- 172.21.15.94:0/1883376355 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca064aaee0 con 0x55ca0619d1b0 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.057+0000 7fc18f7fe640 10 client.22053 handle_command_reply: tid=0 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.057+0000 7fc1bd86d740 1 client.22053 shutdown 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 msgr2=0x55ca0654c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4023040 tx=0x7fc1a400f9f0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 shutdown_connections 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ca0619d1b0 0x55ca065bf5f0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc18f7fe640 0 client.22053 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca0617c4d0 0x55ca0654d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca061742f0 0x55ca0654c3f0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 --2- 172.21.15.94:0/1883376355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca0615bd80 0x55ca0654beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 >> 172.21.15.94:0/1883376355 conn(0x55ca06191300 msgr2=0x55ca06547270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 shutdown_connections 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 1 -- 172.21.15.94:0/1883376355 wait complete. 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1bd86d740 20 client.22053 trim_cache size 0 max 16384 2024-04-25T14:00:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:00:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:00:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:00:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 msgr2=0x7fc1980b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 0x7fc1980b1ff0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55ca0653eb40 tx=0x7fc1a8004ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 msgr2=0x7fc1b019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.058+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0031440 tx=0x7fc1a0003c10 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 shutdown_connections 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1980afae0 0x7fc1980b1ff0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0109a80 0x7fc1b01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0104d80 0x7fc1b019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 --2- 172.21.15.94:0/2491471019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b0103b40 0x7fc1b019a2f0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 >> 172.21.15.94:0/2491471019 conn(0x7fc1b00fed30 msgr2=0x7fc1b0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 shutdown_connections 2024-04-25T14:00:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:08.059+0000 7fc1b69b7640 1 -- 172.21.15.94:0/2491471019 wait complete. 2024-04-25T14:00:08.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 105148 19.21 MB/sec execute 170 sec latency 432.969 ms 2024-04-25T14:00:08.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:07 smithi094 ceph-mon[29195]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-04-25T14:00:08.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:07 smithi138 ceph-mon[32977]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-04-25T14:00:08.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:07 smithi137 ceph-mon[32568]: pgmap v901: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-04-25T14:00:09.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 105548 19.23 MB/sec execute 171 sec latency 151.755 ms 2024-04-25T14:00:10.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 105982 19.24 MB/sec execute 172 sec latency 75.522 ms 2024-04-25T14:00:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:09 smithi094 ceph-mon[29195]: pgmap v902: 97 pgs: 97 active+clean; 257 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 392 op/s 2024-04-25T14:00:10.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:09 smithi138 ceph-mon[32977]: pgmap v902: 97 pgs: 97 active+clean; 257 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 392 op/s 2024-04-25T14:00:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:09 smithi137 ceph-mon[32568]: pgmap v902: 97 pgs: 97 active+clean; 257 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 392 op/s 2024-04-25T14:00:11.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 106424 19.24 MB/sec execute 173 sec latency 70.561 ms 2024-04-25T14:00:12.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 106752 19.24 MB/sec execute 174 sec latency 206.784 ms 2024-04-25T14:00:12.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:11 smithi094 ceph-mon[29195]: pgmap v903: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-04-25T14:00:12.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:11 smithi138 ceph-mon[32977]: pgmap v903: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-04-25T14:00:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:11 smithi137 ceph-mon[32568]: pgmap v903: 97 pgs: 97 active+clean; 249 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-04-25T14:00:13.144 INFO:tasks.workunit.client.0.smithi094.stdout: 10 107182 19.28 MB/sec execute 175 sec latency 76.922 ms 2024-04-25T14:00:14.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 107588 19.31 MB/sec execute 176 sec latency 88.683 ms 2024-04-25T14:00:14.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:13 smithi094 ceph-mon[29195]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-04-25T14:00:14.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:13 smithi138 ceph-mon[32977]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-04-25T14:00:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:13 smithi137 ceph-mon[32568]: pgmap v904: 97 pgs: 97 active+clean; 245 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-04-25T14:00:15.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 107912 19.32 MB/sec execute 177 sec latency 60.262 ms 2024-04-25T14:00:16.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 108236 19.29 MB/sec execute 178 sec latency 96.199 ms 2024-04-25T14:00:16.204 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:15 smithi137 ceph-mon[32568]: pgmap v905: 97 pgs: 97 active+clean; 252 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2024-04-25T14:00:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:15 smithi094 ceph-mon[29195]: pgmap v905: 97 pgs: 97 active+clean; 252 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2024-04-25T14:00:16.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:15 smithi138 ceph-mon[32977]: pgmap v905: 97 pgs: 97 active+clean; 252 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2024-04-25T14:00:17.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 108458 19.22 MB/sec execute 179 sec latency 206.997 ms 2024-04-25T14:00:18.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 108747 19.21 MB/sec execute 180 sec latency 109.692 ms 2024-04-25T14:00:18.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:17 smithi094 ceph-mon[29195]: pgmap v906: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-04-25T14:00:18.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:17 smithi138 ceph-mon[32977]: pgmap v906: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-04-25T14:00:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:17 smithi137 ceph-mon[32568]: pgmap v906: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-04-25T14:00:19.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 109202 19.24 MB/sec execute 181 sec latency 56.285 ms 2024-04-25T14:00:20.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 109656 19.25 MB/sec execute 182 sec latency 92.343 ms 2024-04-25T14:00:20.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:19 smithi094 ceph-mon[29195]: pgmap v907: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-04-25T14:00:20.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:19 smithi138 ceph-mon[32977]: pgmap v907: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-04-25T14:00:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:19 smithi137 ceph-mon[32568]: pgmap v907: 97 pgs: 97 active+clean; 254 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-04-25T14:00:21.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 110033 19.25 MB/sec execute 183 sec latency 62.239 ms 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 -- 172.21.15.94:0/1239193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8102a40 msgr2=0x7f23f8103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1239193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8102a40 0x7f23f8103740 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009900 tx=0x7f23f402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 -- 172.21.15.94:0/1239193159 shutdown_connections 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1239193159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 0x7f23f810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1239193159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8103c80 0x7f23f8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1239193159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8102a40 0x7f23f8103740 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 -- 172.21.15.94:0/1239193159 >> 172.21.15.94:0/1239193159 conn(0x7f23f80fdc30 msgr2=0x7f23f81000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 -- 172.21.15.94:0/1239193159 shutdown_connections 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.173+0000 7f23fec6e640 1 -- 172.21.15.94:0/1239193159 wait complete. 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 Processor -- start 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 -- start start 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 0x7f23f819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 0x7f23f81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810e540 con 0x7f23f8103c80 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810e3c0 con 0x7f23f8108980 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fec6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810e6c0 con 0x7f23f8102a40 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 0x7f23f819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55512/0 (socket says 172.21.15.94:55512) 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23effff640 1 -- 172.21.15.94:0/1362458345 learned_addr learned my addr 172.21.15.94:0/1362458345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fd20b640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 0x7f23f81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 -- 172.21.15.94:0/1362458345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 msgr2=0x7f23f81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 0x7f23f81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 -- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 msgr2=0x7f23f819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23fca0a640 1 -- 172.21.15.94:0/1362458345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f81a1d80 con 0x7f23f8102a40 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.174+0000 7f23effff640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23fca0a640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 0x7f23f819a240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23f402a930 tx=0x7f23f4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23edffb640 1 -- 172.21.15.94:0/1362458345 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f403f070 con 0x7f23f8102a40 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f81a2010 con 0x7f23f8102a40 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23edffb640 1 -- 172.21.15.94:0/1362458345 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f4031910 con 0x7f23f8102a40 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23edffb640 1 -- 172.21.15.94:0/1362458345 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4035070 con 0x7f23f8102a40 2024-04-25T14:00:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.175+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f81a2520 con 0x7f23f8102a40 2024-04-25T14:00:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.177+0000 7f2405c01740 1 Processor -- start 2024-04-25T14:00:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23edffb640 1 -- 172.21.15.94:0/1362458345 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f23f40037f0 con 0x7f23f8102a40 2024-04-25T14:00:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 -- start start 2024-04-25T14:00:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 0x557f46ed3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46edadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f46e39460 con 0x557f46b331b0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f46df3590 con 0x557f46b124d0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f2405c01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f46e35420 con 0x557f46b0a2f0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fca0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 0x557f46ed3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55526/0 (socket says 172.21.15.94:55526) 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 -- 172.21.15.94:0/865550379 learned_addr learned my addr 172.21.15.94:0/865550379 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23effff640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46edadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 -- 172.21.15.94:0/865550379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 msgr2=0x557f46ed3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 0x557f46ed3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 -- 172.21.15.94:0/865550379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 msgr2=0x557f46edadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46edadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 -- 172.21.15.94:0/865550379 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f46e000d0 con 0x557f46b331b0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23effff640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46edadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fca0a640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 0x557f46ed3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23edffb640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 0x7f23e00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fca0a640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 0x7f23e00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.178+0000 7f23fd20b640 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009fd0 tx=0x7f23f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09b52f8a85e7b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.179+0000 7f23edffb640 1 -- 172.21.15.94:0/1362458345 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23f40fede0 con 0x7f23f8102a40 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.179+0000 7f23deffd640 1 -- 172.21.15.94:0/865550379 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0004d20 con 0x557f46b331b0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.179+0000 7f23deffd640 1 -- 172.21.15.94:0/865550379 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f0004050 con 0x557f46b331b0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.179+0000 7f23deffd640 1 -- 172.21.15.94:0/865550379 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0005240 con 0x557f46b331b0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- 172.21.15.94:0/865550379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 msgr2=0x557f46ed3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f23fca0a640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 0x7f23e00b1ff0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x557f46ed4d50 tx=0x7f23f4107040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009fd0 tx=0x7f23f00118d0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- 172.21.15.94:0/865550379 shutdown_connections 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46edadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b0a2f0 0x557f46ed3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- 172.21.15.94:0/865550379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ed3970 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- 172.21.15.94:0/865550379 >> 172.21.15.94:0/865550379 conn(0x557f46b27300 msgr2=0x557f46ecf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- 172.21.15.94:0/865550379 shutdown_connections 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- 172.21.15.94:0/865550379 wait complete. 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 -- start start 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 Processor -- start 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46edd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.180+0000 7f2405c01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b124d0 0x557f46eddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f46edb840 con 0x557f46b331b0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f46dc0b60 con 0x557f46b0a2f0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f46edb9c0 con 0x557f46b124d0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 10 client.0 _ll_register_callbacks cb 0x557f46de52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55528/0 (socket says 172.21.15.94:55528) 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 -- 172.21.15.94:0/3197855441 learned_addr learned my addr 172.21.15.94:0/3197855441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46edd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23effff640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b124d0 0x557f46eddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 -- 172.21.15.94:0/3197855441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b124d0 msgr2=0x557f46eddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b124d0 0x557f46eddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 -- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 msgr2=0x557f46edd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46edd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 -- 172.21.15.94:0/3197855441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f46b4b210 con 0x557f46b331b0 2024-04-25T14:00:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23fd20b640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009de0 tx=0x7f23f0005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a640967f1cfe12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00045c0 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23f0004a40 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.181+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f0004050 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23f000f6f0 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 10 client.22065 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557f46c17620 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f0004050 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f23f0028900 con 0x557f46b331b0 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 10 client.22065 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 10 client.22065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 20 client.22065 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46af1d80 0x557f46eeb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 4 client.22065 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f46e47070 con 0x557f46af1d80 2024-04-25T14:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.182+0000 7f23effff640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46af1d80 0x557f46eeb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.183+0000 7f23d7fff640 10 client.22065 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.183+0000 7f23effff640 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46af1d80 0x557f46eeb0f0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703fa49983a1dc41 server_cookie=e4bf49b78207f251 in_seq=0 out_seq=0 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.187+0000 7f23d7fff640 1 -- 172.21.15.94:0/3197855441 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f46e47070 con 0x557f46af1d80 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.187+0000 7f23d7fff640 10 client.22065 handle_command_reply: tid=0 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.187+0000 7f2405c01740 1 client.22065 shutdown 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 msgr2=0x557f46ee4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009de0 tx=0x7f23f0005ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 shutdown_connections 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46af1d80 0x557f46eeb0f0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b331b0 0x557f46ee4ae0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b124d0 0x557f46eddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 --2- 172.21.15.94:0/3197855441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46edd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f23d7fff640 0 client.22065 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 >> 172.21.15.94:0/3197855441 conn(0x557f46b27300 msgr2=0x557f46ece250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 shutdown_connections 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 1 -- 172.21.15.94:0/3197855441 wait complete. 2024-04-25T14:00:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.188+0000 7f2405c01740 20 client.22065 trim_cache size 0 max 16384 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 Processor -- start 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 -- start start 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46ed8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46eceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.205+0000 7f2405c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81ab4f0 con 0x557f46b0a2f0 2024-04-25T14:00:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f2405c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81ad030 con 0x557f46b124d0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f2405c01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81abbc0 con 0x557f46b331b0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55540/0 (socket says 172.21.15.94:55540) 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 -- 172.21.15.94:0/588178269 learned_addr learned my addr 172.21.15.94:0/588178269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23effff640 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46ed8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46eceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 -- 172.21.15.94:0/588178269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 msgr2=0x557f46eceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46eceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 -- 172.21.15.94:0/588178269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 msgr2=0x557f46ed8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46ed8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 -- 172.21.15.94:0/588178269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f46ede6b0 con 0x557f46b0a2f0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23fd20b640 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005d70 tx=0x7f23f0029f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be77c4ba58540a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23de7fc640 1 -- 172.21.15.94:0/588178269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0035040 con 0x557f46b0a2f0 2024-04-25T14:00:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23de7fc640 1 -- 172.21.15.94:0/588178269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f000fd40 con 0x557f46b0a2f0 2024-04-25T14:00:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.206+0000 7f23de7fc640 1 -- 172.21.15.94:0/588178269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0028910 con 0x557f46b0a2f0 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- 172.21.15.94:0/588178269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 msgr2=0x557f46ed7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005d70 tx=0x7f23f0029f50 comp rx=0 tx=0).stop 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- 172.21.15.94:0/588178269 shutdown_connections 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46eceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b124d0 0x557f46ed8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 --2- 172.21.15.94:0/588178269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b0a2f0 0x557f46ed7af0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- 172.21.15.94:0/588178269 >> 172.21.15.94:0/588178269 conn(0x557f46b27300 msgr2=0x557f46edbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- 172.21.15.94:0/588178269 shutdown_connections 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- 172.21.15.94:0/588178269 wait complete. 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 -- start start 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.208+0000 7f2405c01740 1 Processor -- start 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46ee11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46f55860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81ac960 con 0x557f46b124d0 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81ac290 con 0x557f46b0a2f0 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81a94d0 con 0x557f46b331b0 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 10 client.0 _ll_register_callbacks cb 0x7f23e8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55552/0 (socket says 172.21.15.94:55552) 2024-04-25T14:00:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 -- 172.21.15.94:0/2505307404 learned_addr learned my addr 172.21.15.94:0/2505307404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23fd20b640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46ee11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23fd20b640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46f55860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 -- 172.21.15.94:0/2505307404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 msgr2=0x557f46f55860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46f55860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 -- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 msgr2=0x557f46ee11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46ee11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 -- 172.21.15.94:0/2505307404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f46f55dd0 con 0x557f46b124d0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23effff640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x557f46ed0090 tx=0x7f23e8012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4434bdafb450efca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8020bb0 con 0x557f46b124d0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23e8020d50 con 0x557f46b124d0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.209+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f000fd40 con 0x557f46b124d0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23e80220b0 con 0x557f46b124d0 2024-04-25T14:00:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 10 client.22071 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557f46f57160 con 0x557f46b124d0 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f000fd40 con 0x557f46b124d0 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f23e8010660 con 0x557f46b124d0 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 10 client.22071 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 10 client.22071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 20 client.22071 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46ac71c0 0x557f46f59b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 4 client.22071 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f46e40ee0 con 0x557f46ac71c0 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.210+0000 7f23fd20b640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46ac71c0 0x557f46f59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.211+0000 7f23d7fff640 10 client.22071 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.211+0000 7f23fd20b640 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46ac71c0 0x557f46f59b80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e728971a9711fc68 server_cookie=2ea332e2fe3791c1 in_seq=0 out_seq=0 2024-04-25T14:00:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.754+0000 7f23d7fff640 1 -- 172.21.15.94:0/2505307404 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f46e40ee0 con 0x557f46ac71c0 2024-04-25T14:00:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.754+0000 7f23d7fff640 10 client.22071 handle_command_reply: tid=0 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 client.22071 shutdown 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 msgr2=0x557f46ee1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x557f46ed0090 tx=0x7f23e8012f90 comp rx=0 tx=0).stop 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 shutdown_connections 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557f46ac71c0 0x557f46f59b80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f23d7fff640 0 client.22071 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f46b331b0 0x557f46f55860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f46b124d0 0x557f46ee1730 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 --2- 172.21.15.94:0/2505307404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f46b0a2f0 0x557f46ee11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 >> 172.21.15.94:0/2505307404 conn(0x557f46b27300 msgr2=0x557f46edd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 shutdown_connections 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 1 -- 172.21.15.94:0/2505307404 wait complete. 2024-04-25T14:00:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.755+0000 7f2405c01740 20 client.22071 trim_cache size 0 max 16384 2024-04-25T14:00:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:00:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:00:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:00:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:00:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 msgr2=0x7f23e00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 0x7f23e00b1ff0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x557f46ed4d50 tx=0x7f23f4107040 comp rx=0 tx=0).stop 2024-04-25T14:00:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 msgr2=0x7f23f819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 0x7f23f819a240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23f402a930 tx=0x7f23f4004270 comp rx=0 tx=0).stop 2024-04-25T14:00:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 shutdown_connections 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23e00afae0 0x7f23e00b1ff0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23f8108980 0x7f23f81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23f8103c80 0x7f23f819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 --2- 172.21.15.94:0/1362458345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23f8102a40 0x7f23f819a240 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 >> 172.21.15.94:0/1362458345 conn(0x7f23f80fdc30 msgr2=0x7f23f80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 shutdown_connections 2024-04-25T14:00:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:21.756+0000 7f23fec6e640 1 -- 172.21.15.94:0/1362458345 wait complete. 2024-04-25T14:00:22.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 110189 19.23 MB/sec execute 184 sec latency 338.261 ms 2024-04-25T14:00:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:21 smithi094 ceph-mon[29195]: pgmap v908: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-04-25T14:00:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:21 smithi138 ceph-mon[32977]: pgmap v908: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-04-25T14:00:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:21 smithi137 ceph-mon[32568]: pgmap v908: 97 pgs: 97 active+clean; 256 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-04-25T14:00:22.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:22.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:22.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 -- 172.21.15.94:0/3427835350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 msgr2=0x7f9460107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 --2- 172.21.15.94:0/3427835350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f9460107d30 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f9458009ca0 tx=0x7f945802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 -- 172.21.15.94:0/3427835350 shutdown_connections 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 --2- 172.21.15.94:0/3427835350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f9460107d30 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 --2- 172.21.15.94:0/3427835350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94601027b0 0x7f9460102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 --2- 172.21.15.94:0/3427835350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460101d70 0x7f94601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 -- 172.21.15.94:0/3427835350 >> 172.21.15.94:0/3427835350 conn(0x7f94600fd340 msgr2=0x7f94600ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 -- 172.21.15.94:0/3427835350 shutdown_connections 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.777+0000 7f9465e96640 1 -- 172.21.15.94:0/3427835350 wait complete. 2024-04-25T14:00:22.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.778+0000 7f9465e96640 1 Processor -- start 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.778+0000 7f9465e96640 1 -- start start 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94601027b0 0x7f94601a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f94601a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9460115bf0 con 0x7f9460107920 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9460115a70 con 0x7f94601027b0 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f9465e96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9460115d70 con 0x7f9460101d70 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58114/0 (socket says 172.21.15.94:58114) 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 -- 172.21.15.94:0/1721119088 learned_addr learned my addr 172.21.15.94:0/1721119088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945ffff640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f94601a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945effd640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94601027b0 0x7f94601a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 -- 172.21.15.94:0/1721119088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94601027b0 msgr2=0x7f94601a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94601027b0 0x7f94601a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 -- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 msgr2=0x7f94601a7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f94601a7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 -- 172.21.15.94:0/1721119088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a82c0 con 0x7f9460101d70 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.779+0000 7f945f7fe640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9f0 tx=0x7f945000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.780+0000 7f945cff9640 1 -- 172.21.15.94:0/1721119088 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945000cd80 con 0x7f9460101d70 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.780+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a85b0 con 0x7f9460101d70 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.780+0000 7f945cff9640 1 -- 172.21.15.94:0/1721119088 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94500108e0 con 0x7f9460101d70 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.780+0000 7f945cff9640 1 -- 172.21.15.94:0/1721119088 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450018980 con 0x7f9460101d70 2024-04-25T14:00:22.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.780+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a8b50 con 0x7f9460101d70 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.783+0000 7f946ce26740 1 Processor -- start 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 -- start start 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x562395829fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x56239582ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56239578ed80 con 0x56239545f4d0 2024-04-25T14:00:22.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623957488b0 con 0x5623954676b0 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f946ce26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56239578ad40 con 0x562395488390 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945cff9640 1 -- 172.21.15.94:0/1721119088 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9450010430 con 0x7f9460101d70 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58128/0 (socket says 172.21.15.94:58128) 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945ffff640 1 -- 172.21.15.94:0/3991877971 learned_addr learned my addr 172.21.15.94:0/3991877971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945effd640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x56239582ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.784+0000 7f945f7fe640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x562395829fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 -- 172.21.15.94:0/3991877971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 msgr2=0x56239582ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x56239582ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 -- 172.21.15.94:0/3991877971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 msgr2=0x562395829fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x562395829fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 -- 172.21.15.94:0/3991877971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562395719a40 con 0x562395488390 2024-04-25T14:00:22.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945f7fe640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x562395829fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945cff9640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 0x7f94480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.785+0000 7f945ffff640 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9458035000 tx=0x7f945802f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4796092e7a5509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.786+0000 7f945f7fe640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 0x7f94480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.786+0000 7f945cff9640 1 -- 172.21.15.94:0/1721119088 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f94500deff0 con 0x7f9460101d70 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.786+0000 7f9445ffb640 1 -- 172.21.15.94:0/3991877971 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458031650 con 0x562395488390 2024-04-25T14:00:22.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.786+0000 7f9445ffb640 1 -- 172.21.15.94:0/3991877971 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458036030 con 0x562395488390 2024-04-25T14:00:22.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.786+0000 7f9445ffb640 1 -- 172.21.15.94:0/3991877971 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580047a0 con 0x562395488390 2024-04-25T14:00:22.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.787+0000 7f946ce26740 1 -- 172.21.15.94:0/3991877971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 msgr2=0x562395829a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.787+0000 7f946ce26740 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9458035000 tx=0x7f945802f410 comp rx=0 tx=0).stop 2024-04-25T14:00:22.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.787+0000 7f945f7fe640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 0x7f94480b2080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f94500a7410 tx=0x7f945000cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 -- 172.21.15.94:0/3991877971 shutdown_connections 2024-04-25T14:00:22.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x56239582ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x562395829fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 --2- 172.21.15.94:0/3991877971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x562395829a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 -- 172.21.15.94:0/3991877971 >> 172.21.15.94:0/3991877971 conn(0x56239547c4e0 msgr2=0x562395825270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 -- 172.21.15.94:0/3991877971 shutdown_connections 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 -- 172.21.15.94:0/3991877971 wait complete. 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 -- start start 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.788+0000 7f946ce26740 1 Processor -- start 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56239545f4d0 0x562395844790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623954676b0 0x562395844cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56239582b5a0 con 0x5623954676b0 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562395715d10 con 0x56239545f4d0 2024-04-25T14:00:22.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56239582b720 con 0x562395488390 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.789+0000 7f946ce26740 10 client.0 _ll_register_callbacks cb 0x562395742210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58138/0 (socket says 172.21.15.94:58138) 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 -- 172.21.15.94:0/850119902 learned_addr learned my addr 172.21.15.94:0/850119902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56239545f4d0 0x562395844790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945effd640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623954676b0 0x562395844cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 -- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56239545f4d0 msgr2=0x562395844790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56239545f4d0 0x562395844790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 -- 172.21.15.94:0/850119902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623954676b0 msgr2=0x562395844cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623954676b0 0x562395844cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 -- 172.21.15.94:0/850119902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239556a7d0 con 0x562395488390 2024-04-25T14:00:22.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f945ffff640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9458009620 tx=0x7f9458004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af7ae65e9d6e863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f943effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.790+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458003d80 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f945802f660 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9458036030 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9458004b40 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 10 client.31168 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623954a03f0 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458036030 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9458043070 con 0x562395488390 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 10 client.31168 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 10 client.31168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 20 client.31168 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562395446f60 0x562395851cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 4 client.31168 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.791+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239579c990 con 0x562395446f60 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.792+0000 7f945effd640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562395446f60 0x562395851cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.792+0000 7f945effd640 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562395446f60 0x562395851cf0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3cf83ea6a85edb server_cookie=f2c0a5b4c07f5076 in_seq=0 out_seq=0 2024-04-25T14:00:22.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.792+0000 7f943effd640 10 client.31168 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.794+0000 7f943effd640 1 -- 172.21.15.94:0/850119902 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56239579c990 con 0x562395446f60 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.794+0000 7f943effd640 10 client.31168 handle_command_reply: tid=0 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.794+0000 7f946ce26740 1 client.31168 shutdown 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 msgr2=0x56239584bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9458009620 tx=0x7f9458004040 comp rx=0 tx=0).stop 2024-04-25T14:00:22.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 shutdown_connections 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562395446f60 0x562395851cf0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562395488390 0x56239584bc30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5623954676b0 0x562395844cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 --2- 172.21.15.94:0/850119902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56239545f4d0 0x562395844790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f943effd640 0 client.31168 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 >> 172.21.15.94:0/850119902 conn(0x56239547c4e0 msgr2=0x5623958243f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 shutdown_connections 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 1 -- 172.21.15.94:0/850119902 wait complete. 2024-04-25T14:00:22.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.795+0000 7f946ce26740 20 client.31168 trim_cache size 0 max 16384 2024-04-25T14:00:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 Processor -- start 2024-04-25T14:00:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 -- start start 2024-04-25T14:00:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623954676b0 0x562395828800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562395488390 0x5623958574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94601a9260 con 0x56239545f4d0 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aada0 con 0x562395488390 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f946ce26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94601a9930 con 0x5623954676b0 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f945ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f945ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55606/0 (socket says 172.21.15.94:55606) 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f945ffff640 1 -- 172.21.15.94:0/1502196501 learned_addr learned my addr 172.21.15.94:0/1502196501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f945effd640 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623954676b0 0x562395828800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.819+0000 7f945ffff640 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562395488390 0x5623958574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 -- 172.21.15.94:0/1502196501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623954676b0 msgr2=0x562395828800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623954676b0 0x562395828800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 -- 172.21.15.94:0/1502196501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562395488390 msgr2=0x5623958574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562395488390 0x5623958574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 -- 172.21.15.94:0/1502196501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623954600f0 con 0x56239545f4d0 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f945ffff640 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f94580314d0 tx=0x7f9458005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad14d953b33f8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f94457fa640 1 -- 172.21.15.94:0/1502196501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458030ce0 con 0x56239545f4d0 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f94457fa640 1 -- 172.21.15.94:0/1502196501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458036040 con 0x56239545f4d0 2024-04-25T14:00:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.820+0000 7f94457fa640 1 -- 172.21.15.94:0/1502196501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945802f8c0 con 0x56239545f4d0 2024-04-25T14:00:22.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 -- 172.21.15.94:0/1502196501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 msgr2=0x56239582ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f94580314d0 tx=0x7f9458005fd0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 -- 172.21.15.94:0/1502196501 shutdown_connections 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562395488390 0x5623958574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623954676b0 0x562395828800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 --2- 172.21.15.94:0/1502196501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56239545f4d0 0x56239582ecd0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 -- 172.21.15.94:0/1502196501 >> 172.21.15.94:0/1502196501 conn(0x56239547c4e0 msgr2=0x5623958280f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 -- 172.21.15.94:0/1502196501 shutdown_connections 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.821+0000 7f946ce26740 1 -- 172.21.15.94:0/1502196501 wait complete. 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 -- start start 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 Processor -- start 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239545f4d0 0x562395849230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x562395849770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aa6d0 con 0x562395488390 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aa000 con 0x5623954676b0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9460073250 con 0x56239545f4d0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f946ce26740 10 client.0 _ll_register_callbacks cb 0x7f9458009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f945ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.822+0000 7f945ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55618/0 (socket says 172.21.15.94:55618) 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 -- 172.21.15.94:0/1727171112 learned_addr learned my addr 172.21.15.94:0/1727171112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945effd640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x562395849770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239545f4d0 0x562395849230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 -- 172.21.15.94:0/1727171112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239545f4d0 msgr2=0x562395849230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239545f4d0 0x562395849230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 -- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 msgr2=0x562395849770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x562395849770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 -- 172.21.15.94:0/1727171112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239574f4b0 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f945ffff640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f94580316a0 tx=0x7f9458030860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed5dd541a14fbae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f943effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945802f660 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9458073070 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.823+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9458036040 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f945802f800 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 10 client.22089 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5623958b9370 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945804b040 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9458076020 con 0x562395488390 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 10 client.22089 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 10 client.22089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 20 client.22089 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56239541c3a0 0x5623958bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 4 client.22089 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9458036040 con 0x56239541c3a0 2024-04-25T14:00:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.824+0000 7f945effd640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56239541c3a0 0x5623958bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.825+0000 7f943effd640 10 client.22089 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:22.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.825+0000 7f945effd640 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56239541c3a0 0x5623958bb990 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5aba46f38c50c36 server_cookie=d5895a88030b2ed6 in_seq=0 out_seq=0 2024-04-25T14:00:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f943effd640 1 -- 172.21.15.94:0/1727171112 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9458036040 con 0x56239541c3a0 2024-04-25T14:00:22.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f943effd640 10 client.22089 handle_command_reply: tid=0 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 client.22089 shutdown 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 msgr2=0x5623958b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f94580316a0 tx=0x7f9458030860 comp rx=0 tx=0).stop 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 shutdown_connections 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56239541c3a0 0x5623958bb990 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562395488390 0x5623958b8050 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623954676b0 0x562395849770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 --2- 172.21.15.94:0/1727171112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56239545f4d0 0x562395849230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f943effd640 0 client.22089 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 >> 172.21.15.94:0/1727171112 conn(0x56239547c4e0 msgr2=0x5623958435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 shutdown_connections 2024-04-25T14:00:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 1 -- 172.21.15.94:0/1727171112 wait complete. 2024-04-25T14:00:22.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.831+0000 7f946ce26740 20 client.22089 trim_cache size 0 max 16384 2024-04-25T14:00:22.833 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:22.833 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:00:22.833 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:00:22.833 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 msgr2=0x7f94480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 0x7f94480b2080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f94500a7410 tx=0x7f945000cf20 comp rx=0 tx=0).stop 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 msgr2=0x7f94601a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9f0 tx=0x7f945000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 shutdown_connections 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f94480afb70 0x7f94480b2080 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9460107920 0x7f94601a7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94601027b0 0x7f94601a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 --2- 172.21.15.94:0/1721119088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9460101d70 0x7f94601a0780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 >> 172.21.15.94:0/1721119088 conn(0x7f94600fd340 msgr2=0x7f9460103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 shutdown_connections 2024-04-25T14:00:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.832+0000 7f9465e96640 1 -- 172.21.15.94:0/1721119088 wait complete. 2024-04-25T14:00:22.845 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:00:22.845 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 -- 172.21.15.94:0/2305633743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918103c00 msgr2=0x7f3918104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 --2- 172.21.15.94:0/2305633743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918103c00 0x7f3918104010 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f3908009900 tx=0x7f390802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 -- 172.21.15.94:0/2305633743 shutdown_connections 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 --2- 172.21.15.94:0/2305633743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918108900 0x7f391810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 --2- 172.21.15.94:0/2305633743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918103c00 0x7f3918104010 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 --2- 172.21.15.94:0/2305633743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39181029c0 0x7f39181036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.984+0000 7f391f6e7640 1 -- 172.21.15.94:0/2305633743 >> 172.21.15.94:0/2305633743 conn(0x7f39180fdbd0 msgr2=0x7f3918100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.985+0000 7f391f6e7640 1 -- 172.21.15.94:0/2305633743 shutdown_connections 2024-04-25T14:00:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.985+0000 7f391f6e7640 1 -- 172.21.15.94:0/2305633743 wait complete. 2024-04-25T14:00:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.985+0000 7f391f6e7640 1 Processor -- start 2024-04-25T14:00:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 -- start start 2024-04-25T14:00:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181029c0 0x7f3918073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 0x7f39180743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f391806ce20 con 0x7f3918108900 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f391806cca0 con 0x7f39181029c0 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391f6e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f391806cfa0 con 0x7f3918103c00 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391dc84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391dc84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55644/0 (socket says 172.21.15.94:55644) 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391dc84640 1 -- 172.21.15.94:0/1427606174 learned_addr learned my addr 172.21.15.94:0/1427606174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391d483640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181029c0 0x7f3918073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391cc82640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 0x7f39180743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.986+0000 7f391dc84640 1 -- 172.21.15.94:0/1427606174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 msgr2=0x7f39180743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391dc84640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 0x7f39180743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391dc84640 1 -- 172.21.15.94:0/1427606174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181029c0 msgr2=0x7f3918073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391dc84640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181029c0 0x7f3918073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391dc84640 1 -- 172.21.15.94:0/1427606174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180fe270 con 0x7f3918108900 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391cc82640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 0x7f39180743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391dc84640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f391000eaa0 tx=0x7f391000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000cdb0 con 0x7f3918108900 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39180fe560 con 0x7f3918108900 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39180feab0 con 0x7f3918108900 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3910005020 con 0x7f3918108900 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.987+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391001f060 con 0x7f3918108900 2024-04-25T14:00:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.988+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc0052c0 con 0x7f3918108900 2024-04-25T14:00:22.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.990+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3910004a40 con 0x7f3918108900 2024-04-25T14:00:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.991+0000 7f390e7fc640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 0x7f39040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.991+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f39100df210 con 0x7f3918108900 2024-04-25T14:00:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.991+0000 7f391d483640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 0x7f39040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.992+0000 7f391d483640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 0x7f39040b2000 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f39181035c0 tx=0x7f3914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:22.995+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3910016030 con 0x7f3918108900 2024-04-25T14:00:23.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 110592 19.23 MB/sec execute 185 sec latency 95.562 ms 2024-04-25T14:00:23.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.237+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38fc0055b0 con 0x7f3918108900 2024-04-25T14:00:23.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.238+0000 7f390e7fc640 1 -- 172.21.15.94:0/1427606174 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f39100a7d50 con 0x7f3918108900 2024-04-25T14:00:23.238 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:00:23.238 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:00:23.238 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:00:23.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 msgr2=0x7f39040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 0x7f39040b2000 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f39181035c0 tx=0x7f3914005f90 comp rx=0 tx=0).stop 2024-04-25T14:00:23.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 msgr2=0x7f39181a9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f391000eaa0 tx=0x7f391000ef60 comp rx=0 tx=0).stop 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 shutdown_connections 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39040afaf0 0x7f39040b2000 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3918108900 0x7f39181a9ed0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3918103c00 0x7f39180743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 --2- 172.21.15.94:0/1427606174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181029c0 0x7f3918073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 >> 172.21.15.94:0/1427606174 conn(0x7f39180fdbd0 msgr2=0x7f3918069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 shutdown_connections 2024-04-25T14:00:23.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.240+0000 7f391f6e7640 1 -- 172.21.15.94:0/1427606174 wait complete. 2024-04-25T14:00:23.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:00:23.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 -- 172.21.15.94:0/3922712566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c105090 msgr2=0x7f441c1054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 --2- 172.21.15.94:0/3922712566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c105090 0x7f441c1054a0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f4418009900 tx=0x7f441802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:23.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 -- 172.21.15.94:0/3922712566 shutdown_connections 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 --2- 172.21.15.94:0/3922712566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c102240 0x7f441c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 --2- 172.21.15.94:0/3922712566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c101740 0x7f441c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 --2- 172.21.15.94:0/3922712566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c105090 0x7f441c1054a0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.366+0000 7f442428c640 1 -- 172.21.15.94:0/3922712566 >> 172.21.15.94:0/3922712566 conn(0x7f441c0fd340 msgr2=0x7f441c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 -- 172.21.15.94:0/3922712566 shutdown_connections 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 -- 172.21.15.94:0/3922712566 wait complete. 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 Processor -- start 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 -- start start 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.367+0000 7f442428c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c102240 0x7f441c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f442428c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c105090 0x7f441c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f442428c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10f600 con 0x7f441c101740 2024-04-25T14:00:23.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f442428c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10f480 con 0x7f441c105090 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f442428c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10f780 con 0x7f441c102240 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4421827640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c102240 0x7f441c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55660/0 (socket says 172.21.15.94:55660) 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 -- 172.21.15.94:0/3864879800 learned_addr learned my addr 172.21.15.94:0/3864879800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422829640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c105090 0x7f441c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 -- 172.21.15.94:0/3864879800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c102240 msgr2=0x7f441c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c102240 0x7f441c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 -- 172.21.15.94:0/3864879800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c105090 msgr2=0x7f441c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c105090 0x7f441c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 -- 172.21.15.94:0/3864879800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c1a1da0 con 0x7f441c101740 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f4422028640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4418006010 tx=0x7f4418004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f44137fe640 1 -- 172.21.15.94:0/3864879800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441803f030 con 0x7f441c101740 2024-04-25T14:00:23.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f44137fe640 1 -- 172.21.15.94:0/3864879800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4418003aa0 con 0x7f441c101740 2024-04-25T14:00:23.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c1a2030 con 0x7f441c101740 2024-04-25T14:00:23.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.368+0000 7f44137fe640 1 -- 172.21.15.94:0/3864879800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418035070 con 0x7f441c101740 2024-04-25T14:00:23.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.369+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a2520 con 0x7f441c101740 2024-04-25T14:00:23.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.371+0000 7f442b210740 1 Processor -- start 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 -- start start 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139b0390 0x560a13d52810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560a13cb6d80 con 0x560a139874d0 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a13c70a20 con 0x560a1398f6b0 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f442b210740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a13cb2d40 con 0x560a139b0390 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4422829640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139b0390 0x560a13d52810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:34214/0 (socket says 172.21.15.94:34214) 2024-04-25T14:00:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f44137fe640 1 -- 172.21.15.94:0/3864879800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4418042030 con 0x7f441c101740 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 -- 172.21.15.94:0/3312645103 learned_addr learned my addr 172.21.15.94:0/3312645103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4422028640 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 -- 172.21.15.94:0/3312645103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139b0390 msgr2=0x560a13d52810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139b0390 0x560a13d52810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 -- 172.21.15.94:0/3312645103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 msgr2=0x560a13d52d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d52d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 -- 172.21.15.94:0/3312645103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a13c7bc30 con 0x560a1398f6b0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.372+0000 7f4421827640 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f440c004650 tx=0x7f440c00bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751250073640b900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f4410bfe640 1 -- 172.21.15.94:0/3312645103 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003bd0 con 0x560a1398f6b0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f4410bfe640 1 -- 172.21.15.94:0/3312645103 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c015070 con 0x560a1398f6b0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f4410bfe640 1 -- 172.21.15.94:0/3312645103 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004d80 con 0x560a1398f6b0 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f44137fe640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 0x7f44080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f4422829640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 0x7f44080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.373+0000 7f44137fe640 1 -- 172.21.15.94:0/3864879800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f44180ff270 con 0x7f441c101740 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- 172.21.15.94:0/3312645103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 msgr2=0x560a13d53a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f4422829640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 0x7f44080b2000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x560a13d53860 tx=0x7f4414009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f440c004650 tx=0x7f440c00bba0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- 172.21.15.94:0/3312645103 shutdown_connections 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d53a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d52d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- 172.21.15.94:0/3312645103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139b0390 0x560a13d52810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- 172.21.15.94:0/3312645103 >> 172.21.15.94:0/3312645103 conn(0x560a139a44e0 msgr2=0x560a13d4e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- 172.21.15.94:0/3312645103 shutdown_connections 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- 172.21.15.94:0/3312645103 wait complete. 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- start start 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 Processor -- start 2024-04-25T14:00:23.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 0x560a13d5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560a13d54530 con 0x560a139b0390 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a13c3ddc0 con 0x560a139874d0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.374+0000 7f442b210740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a13d546b0 con 0x560a1398f6b0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f442b210740 10 client.0 _ll_register_callbacks cb 0x560a13c59600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422829640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422829640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55676/0 (socket says 172.21.15.94:55676) 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422829640 1 -- 172.21.15.94:0/693401210 learned_addr learned my addr 172.21.15.94:0/693401210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 0x560a13d5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4421827640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 -- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 msgr2=0x560a13d5caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d5caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 -- 172.21.15.94:0/693401210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 msgr2=0x560a13d63a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 -- 172.21.15.94:0/693401210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a139c83f0 con 0x560a139874d0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422829640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4422028640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 0x560a13d5c560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x560a13d52530 tx=0x7f441802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271757843e8a0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418049070 con 0x560a139874d0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4418003c50 con 0x560a139874d0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c015070 con 0x560a139874d0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f44180c6ed0 con 0x560a139874d0 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f442b210740 10 client.31535 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.375+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a13a927d0 con 0x560a139874d0 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c015070 con 0x560a139874d0 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f44180b00b0 con 0x560a139874d0 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 10 client.31535 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 10 client.31535 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 20 client.31535 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a1396ef60 0x560a13d6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 4 client.31535 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a13cc4990 con 0x560a1396ef60 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.376+0000 7f4421827640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a1396ef60 0x560a13d6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.377+0000 7f4405ffb640 10 client.31535 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.377+0000 7f4421827640 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a1396ef60 0x560a13d6a230 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11678d998564ce60 server_cookie=7fd0c748c855c174 in_seq=0 out_seq=0 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.377+0000 7f4405ffb640 1 -- 172.21.15.94:0/693401210 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a13cc4990 con 0x560a1396ef60 2024-04-25T14:00:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.377+0000 7f4405ffb640 10 client.31535 handle_command_reply: tid=0 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.377+0000 7f442b210740 1 client.31535 shutdown 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 msgr2=0x560a13d5c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 0x560a13d5c560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x560a13d52530 tx=0x7f441802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 shutdown_connections 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a1396ef60 0x560a13d6a230 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d5caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 --2- 172.21.15.94:0/693401210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139874d0 0x560a13d5c560 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f4405ffb640 0 client.31535 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 >> 172.21.15.94:0/693401210 conn(0x560a139a44e0 msgr2=0x560a13d4d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 shutdown_connections 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 1 -- 172.21.15.94:0/693401210 wait complete. 2024-04-25T14:00:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.378+0000 7f442b210740 20 client.31535 trim_cache size 0 max 16384 2024-04-25T14:00:23.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 Processor -- start 2024-04-25T14:00:23.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 -- start start 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139874d0 0x560a13d5b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d5c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1ab4d0 con 0x560a139b0390 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1ad010 con 0x560a1398f6b0 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f442b210740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1abba0 con 0x560a139874d0 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55680/0 (socket says 172.21.15.94:55680) 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 -- 172.21.15.94:0/2536747813 learned_addr learned my addr 172.21.15.94:0/2536747813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4421827640 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d5c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 -- 172.21.15.94:0/2536747813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139874d0 msgr2=0x560a13d5b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139874d0 0x560a13d5b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 -- 172.21.15.94:0/2536747813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 msgr2=0x560a13d5c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d5c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 -- 172.21.15.94:0/2536747813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a13d5d160 con 0x560a139b0390 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.393+0000 7f4422829640 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f44140042e0 tx=0x7f441401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e69d288b4b6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f4407fff640 1 -- 172.21.15.94:0/2536747813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414004b20 con 0x560a139b0390 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f4407fff640 1 -- 172.21.15.94:0/2536747813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4414004cc0 con 0x560a139b0390 2024-04-25T14:00:23.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f4407fff640 1 -- 172.21.15.94:0/2536747813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414005870 con 0x560a139b0390 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 -- 172.21.15.94:0/2536747813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 msgr2=0x560a13d5c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f44140042e0 tx=0x7f441401a000 comp rx=0 tx=0).stop 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 -- 172.21.15.94:0/2536747813 shutdown_connections 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139b0390 0x560a13d5c670 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a1398f6b0 0x560a13d5c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 --2- 172.21.15.94:0/2536747813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a139874d0 0x560a13d5b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.394+0000 7f442b210740 1 -- 172.21.15.94:0/2536747813 >> 172.21.15.94:0/2536747813 conn(0x560a139a44e0 msgr2=0x560a13d5ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- 172.21.15.94:0/2536747813 shutdown_connections 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- 172.21.15.94:0/2536747813 wait complete. 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- start start 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 Processor -- start 2024-04-25T14:00:23.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 0x560a13dd0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1ac940 con 0x560a139874d0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1ac270 con 0x560a139b0390 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1a94b0 con 0x560a1398f6b0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f442b210740 10 client.0 _ll_register_callbacks cb 0x7f4418005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f4422028640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f4422028640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55692/0 (socket says 172.21.15.94:55692) 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f4422028640 1 -- 172.21.15.94:0/2797301330 learned_addr learned my addr 172.21.15.94:0/2797301330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f4421827640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.395+0000 7f4422829640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 0x560a13dd0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 -- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 msgr2=0x560a13d602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 -- 172.21.15.94:0/2797301330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 msgr2=0x560a13dd0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 0x560a13dd0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 -- 172.21.15.94:0/2797301330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a13d632b0 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422829640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 0x560a13dd0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4422028640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f44180b0420 tx=0x7f44180c7d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb26e4237ffee9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418049030 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f44180d0080 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4414004cc0 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f44180e7050 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 10 client.22113 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a13dd1a20 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4414004cc0 con 0x560a139874d0 2024-04-25T14:00:23.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f44180c69c0 con 0x560a139874d0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 10 client.22113 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 10 client.22113 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.396+0000 7f442b210740 20 client.22113 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a139443a0 0x560a13dd4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 4 client.22113 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a13cbe800 con 0x560a139443a0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f4421827640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a139443a0 0x560a13dd4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f4405ffb640 10 client.22113 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f4421827640 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a139443a0 0x560a13dd4280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8219f97166293ea8 server_cookie=56c278ee981a1e7 in_seq=0 out_seq=0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f4405ffb640 1 -- 172.21.15.94:0/2797301330 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a13cbe800 con 0x560a139443a0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f4405ffb640 10 client.22113 handle_command_reply: tid=0 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 client.22113 shutdown 2024-04-25T14:00:23.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 msgr2=0x560a13d5fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f44180b0420 tx=0x7f44180c7d40 comp rx=0 tx=0).stop 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 shutdown_connections 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.397+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x560a139443a0 0x560a13dd4280 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a139b0390 0x560a13dd0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a1398f6b0 0x560a13d602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f4405ffb640 0 client.22113 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 --2- 172.21.15.94:0/2797301330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560a139874d0 0x560a13d5fdb0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 >> 172.21.15.94:0/2797301330 conn(0x560a139a44e0 msgr2=0x560a13d5c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 shutdown_connections 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 1 -- 172.21.15.94:0/2797301330 wait complete. 2024-04-25T14:00:23.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442b210740 20 client.22113 trim_cache size 0 max 16384 2024-04-25T14:00:23.405 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:23.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 msgr2=0x7f44080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 0x7f44080b2000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x560a13d53860 tx=0x7f4414009500 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 msgr2=0x7f441c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f4418006010 tx=0x7f4418004270 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 shutdown_connections 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f44080afaf0 0x7f44080b2000 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f441c105090 0x7f441c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441c102240 0x7f441c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 --2- 172.21.15.94:0/3864879800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f441c101740 0x7f441c19a260 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 >> 172.21.15.94:0/3864879800 conn(0x7f441c0fd340 msgr2=0x7f441c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 shutdown_connections 2024-04-25T14:00:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.398+0000 7f442428c640 1 -- 172.21.15.94:0/3864879800 wait complete. 2024-04-25T14:00:23.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 -- 172.21.15.94:0/2551842840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 msgr2=0x7f9c241036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/2551842840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c241036e0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009900 tx=0x7f9c1402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 -- 172.21.15.94:0/2551842840 shutdown_connections 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/2551842840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 0x7f9c2410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/2551842840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 0x7f9c24104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/2551842840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c241036e0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 -- 172.21.15.94:0/2551842840 >> 172.21.15.94:0/2551842840 conn(0x7f9c240fdbd0 msgr2=0x7f9c24100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 -- 172.21.15.94:0/2551842840 shutdown_connections 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.532+0000 7f9c29d6c640 1 -- 172.21.15.94:0/2551842840 wait complete. 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 Processor -- start 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 -- start start 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 0x7f9c2419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 0x7f9c241a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410e4e0 con 0x7f9c241029e0 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410e360 con 0x7f9c24108920 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c29d6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410e660 con 0x7f9c24103c20 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32776/0 (socket says 172.21.15.94:32776) 2024-04-25T14:00:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c237fe640 1 -- 172.21.15.94:0/1617560712 learned_addr learned my addr 172.21.15.94:0/1617560712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c23fff640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 0x7f9c241a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.533+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 0x7f9c2419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 -- 172.21.15.94:0/1617560712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 msgr2=0x7f9c2419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 0x7f9c2419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 -- 172.21.15.94:0/1617560712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 msgr2=0x7f9c241a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 0x7f9c241a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 -- 172.21.15.94:0/1617560712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c241a20e0 con 0x7f9c241029e0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c237fe640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006010 tx=0x7f9c14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c20ff9640 1 -- 172.21.15.94:0/1617560712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1403f030 con 0x7f9c241029e0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c20ff9640 1 -- 172.21.15.94:0/1617560712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c14031910 con 0x7f9c241029e0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c241a2370 con 0x7f9c241029e0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c20ff9640 1 -- 172.21.15.94:0/1617560712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14035070 con 0x7f9c241029e0 2024-04-25T14:00:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.534+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c241a2860 con 0x7f9c241029e0 2024-04-25T14:00:23.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 Processor -- start 2024-04-25T14:00:23.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 -- start start 2024-04-25T14:00:23.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c20ff9640 1 -- 172.21.15.94:0/1617560712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9c14042030 con 0x7f9c241029e0 2024-04-25T14:00:23.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a80cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d044d0 0x55a0a80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a0a8033d80 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0a7feda20 con 0x55a0a7d044d0 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c30c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a0a802fd40 con 0x55a0a7d2d390 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32782/0 (socket says 172.21.15.94:32782) 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 -- 172.21.15.94:0/2731568922 learned_addr learned my addr 172.21.15.94:0/2731568922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d044d0 0x55a0a80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c23fff640 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a80cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 -- 172.21.15.94:0/2731568922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 msgr2=0x55a0a80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a80cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 -- 172.21.15.94:0/2731568922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d044d0 msgr2=0x55a0a80cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d044d0 0x55a0a80cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 -- 172.21.15.94:0/2731568922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a7ff8c30 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c20ff9640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 0x7f9c0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.537+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009ca0 tx=0x7f9c18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf80bf2fdd103936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.538+0000 7f9c23fff640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 0x7f9c0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.538+0000 7f9c09ffb640 1 -- 172.21.15.94:0/2731568922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18005220 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.538+0000 7f9c20ff9640 1 -- 172.21.15.94:0/1617560712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9c140fef70 con 0x7f9c241029e0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.538+0000 7f9c09ffb640 1 -- 172.21.15.94:0/2731568922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c18004c50 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.538+0000 7f9c09ffb640 1 -- 172.21.15.94:0/2731568922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18019760 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- 172.21.15.94:0/2731568922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 msgr2=0x55a0a80d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c23fff640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 0x7f9c0c0b2000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55a0a80d0860 tx=0x7f9c1c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009ca0 tx=0x7f9c18011810 comp rx=0 tx=0).stop 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- 172.21.15.94:0/2731568922 shutdown_connections 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d0a20 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d044d0 0x55a0a80cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2731568922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a80cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- 172.21.15.94:0/2731568922 >> 172.21.15.94:0/2731568922 conn(0x55a0a7d214e0 msgr2=0x55a0a80cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- 172.21.15.94:0/2731568922 shutdown_connections 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- 172.21.15.94:0/2731568922 wait complete. 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 -- start start 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 Processor -- start 2024-04-25T14:00:23.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.539+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a0a80d1530 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0a7fbadc0 con 0x55a0a7d2d390 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a0a80d16b0 con 0x55a0a7d044d0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 10 client.0 _ll_register_callbacks cb 0x55a0a7fd6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32786/0 (socket says 172.21.15.94:32786) 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56108/0 (socket says 172.21.15.94:56108) 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 -- 172.21.15.94:0/250208327 learned_addr learned my addr 172.21.15.94:0/250208327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c23fff640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 -- 172.21.15.94:0/250208327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 msgr2=0x55a0a80d9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 -- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 msgr2=0x55a0a80e09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80e09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 -- 172.21.15.94:0/250208327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a7d453f0 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c237fe640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f9c180050b0 tx=0x7f9c180050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3415c9eeb4a264d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18029070 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9c180196a0 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c18004c50 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9c18035050 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.540+0000 7f9c30c16740 10 client.22125 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0a7e0f7d0 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c18004c50 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9c18023770 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 10 client.22125 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 10 client.22125 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 20 client.22125 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cebf60 0x55a0a80e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 4 client.22125 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a8041990 con 0x55a0a7cebf60 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c237fe640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cebf60 0x55a0a80e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c237fe640 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cebf60 0x55a0a80e6f70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4233203bcc470e33 server_cookie=9f176107f36b930b in_seq=0 out_seq=0 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.541+0000 7f9c02ffd640 10 client.22125 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.542+0000 7f9c02ffd640 1 -- 172.21.15.94:0/250208327 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0a8041990 con 0x55a0a7cebf60 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.542+0000 7f9c02ffd640 10 client.22125 handle_command_reply: tid=0 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.542+0000 7f9c30c16740 1 client.22125 shutdown 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.542+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 msgr2=0x55a0a80d9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.542+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f9c180050b0 tx=0x7f9c180050e0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 shutdown_connections 2024-04-25T14:00:23.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cebf60 0x55a0a80e6f70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80e09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80d9a70 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c02ffd640 0 client.22125 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 --2- 172.21.15.94:0/250208327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d044d0 0x55a0a80d9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 >> 172.21.15.94:0/250208327 conn(0x55a0a7d214e0 msgr2=0x55a0a80ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 shutdown_connections 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 1 -- 172.21.15.94:0/250208327 wait complete. 2024-04-25T14:00:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.543+0000 7f9c30c16740 20 client.22125 trim_cache size 0 max 16384 2024-04-25T14:00:23.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 Processor -- start 2024-04-25T14:00:23.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 -- start start 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80e08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241ad750 con 0x55a0a7d044d0 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241af290 con 0x55a0a7d2d390 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c30c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241ade20 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32802/0 (socket says 172.21.15.94:32802) 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c237fe640 1 -- 172.21.15.94:0/2206549309 learned_addr learned my addr 172.21.15.94:0/2206549309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80e08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.557+0000 7f9c23fff640 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 -- 172.21.15.94:0/2206549309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d0c6b0 msgr2=0x55a0a80e08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80e08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 -- 172.21.15.94:0/2206549309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 msgr2=0x55a0a80d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 -- 172.21.15.94:0/2206549309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a80d4c80 con 0x55a0a7d044d0 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f9c140c46d0 tx=0x7f9c140b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d074d02e08e043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c097fa640 1 -- 172.21.15.94:0/2206549309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14049040 con 0x55a0a7d044d0 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c097fa640 1 -- 172.21.15.94:0/2206549309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c140c6f70 con 0x55a0a7d044d0 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.558+0000 7f9c097fa640 1 -- 172.21.15.94:0/2206549309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14003d30 con 0x55a0a7d044d0 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- 172.21.15.94:0/2206549309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 msgr2=0x55a0a80cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f9c140c46d0 tx=0x7f9c140b0040 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- 172.21.15.94:0/2206549309 shutdown_connections 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d2d390 0x55a0a80d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80e08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2206549309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80cae80 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- 172.21.15.94:0/2206549309 >> 172.21.15.94:0/2206549309 conn(0x55a0a7d214e0 msgr2=0x55a0a80ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- 172.21.15.94:0/2206549309 shutdown_connections 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- 172.21.15.94:0/2206549309 wait complete. 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 -- start start 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.559+0000 7f9c30c16740 1 Processor -- start 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a814d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241aebc0 con 0x55a0a7d044d0 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241ae4f0 con 0x55a0a7d0c6b0 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241ab730 con 0x55a0a7d2d390 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 10 client.0 _ll_register_callbacks cb 0x7f9c14102920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32804/0 (socket says 172.21.15.94:32804) 2024-04-25T14:00:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 -- 172.21.15.94:0/2694049142 learned_addr learned my addr 172.21.15.94:0/2694049142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c23fff640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 -- 172.21.15.94:0/2694049142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 msgr2=0x55a0a814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 -- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d0c6b0 msgr2=0x55a0a80ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 -- 172.21.15.94:0/2694049142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a80e0e30 con 0x55a0a7d044d0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c23fff640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c237fe640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9c140c73f0 tx=0x7f9c140314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff2f8a0d17a1a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14049070 con 0x55a0a7d044d0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9c14101110 con 0x55a0a7d044d0 2024-04-25T14:00:23.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c140c6f70 con 0x55a0a7d044d0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9c140c7460 con 0x55a0a7d044d0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 10 client.22131 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.560+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0a814e990 con 0x55a0a7d044d0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c140c6f70 con 0x55a0a7d044d0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9c140e8260 con 0x55a0a7d044d0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 10 client.22131 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 10 client.22131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 20 client.22131 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cc13a0 0x55a0a8151380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 4 client.22131 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0a803b800 con 0x55a0a7cc13a0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cc13a0 0x55a0a8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c02ffd640 10 client.22131 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:23.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.561+0000 7f9c22ffd640 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cc13a0 0x55a0a8151380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53f882137066f0de server_cookie=a543c3af72bc15f1 in_seq=0 out_seq=0 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c02ffd640 1 -- 172.21.15.94:0/2694049142 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0a803b800 con 0x55a0a7cc13a0 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c02ffd640 10 client.22131 handle_command_reply: tid=0 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 client.22131 shutdown 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 msgr2=0x55a0a80dd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9c140c73f0 tx=0x7f9c140314c0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 shutdown_connections 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a0a7cc13a0 0x55a0a8151380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a0a7d2d390 0x55a0a814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0a7d0c6b0 0x55a0a80ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c02ffd640 0 client.22131 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:23.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 --2- 172.21.15.94:0/2694049142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a0a7d044d0 0x55a0a80dd980 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 >> 172.21.15.94:0/2694049142 conn(0x55a0a7d214e0 msgr2=0x55a0a80cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 shutdown_connections 2024-04-25T14:00:23.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 1 -- 172.21.15.94:0/2694049142 wait complete. 2024-04-25T14:00:23.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.562+0000 7f9c30c16740 20 client.22131 trim_cache size 0 max 16384 2024-04-25T14:00:23.569 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:23.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 msgr2=0x7f9c0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 0x7f9c0c0b2000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55a0a80d0860 tx=0x7f9c1c009500 comp rx=0 tx=0).stop 2024-04-25T14:00:23.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 msgr2=0x7f9c2419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006010 tx=0x7f9c14004270 comp rx=0 tx=0).stop 2024-04-25T14:00:23.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 shutdown_connections 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f9c0c0afaf0 0x7f9c0c0b2000 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c24108920 0x7f9c241a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c24103c20 0x7f9c2419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 --2- 172.21.15.94:0/1617560712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c241029e0 0x7f9c2419a5a0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 >> 172.21.15.94:0/1617560712 conn(0x7f9c240fdbd0 msgr2=0x7f9c24109e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 shutdown_connections 2024-04-25T14:00:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.563+0000 7f9c29d6c640 1 -- 172.21.15.94:0/1617560712 wait complete. 2024-04-25T14:00:23.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:00:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 -- 172.21.15.94:0/3953300757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c101740 msgr2=0x7fa05c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 --2- 172.21.15.94:0/3953300757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c101740 0x7fa05c101bb0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002a410 comp rx=0 tx=0).stop 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 -- 172.21.15.94:0/3953300757 shutdown_connections 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 --2- 172.21.15.94:0/3953300757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c102240 0x7fa05c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 --2- 172.21.15.94:0/3953300757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c101740 0x7fa05c101bb0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 --2- 172.21.15.94:0/3953300757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 0x7fa05c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 -- 172.21.15.94:0/3953300757 >> 172.21.15.94:0/3953300757 conn(0x7fa05c0fd340 msgr2=0x7fa05c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 -- 172.21.15.94:0/3953300757 shutdown_connections 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.690+0000 7fa062106640 1 -- 172.21.15.94:0/3953300757 wait complete. 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 Processor -- start 2024-04-25T14:00:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 -- start start 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 0x7fa05c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 0x7fa05c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10f600 con 0x7fa05c102240 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10f480 con 0x7fa05c105090 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa062106640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10f780 con 0x7fa05c101740 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa05b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 0x7fa05c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32818/0 (socket says 172.21.15.94:32818) 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa05affd640 1 -- 172.21.15.94:0/390866042 learned_addr learned my addr 172.21.15.94:0/390866042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.691+0000 7fa05bfff640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 0x7fa05c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 -- 172.21.15.94:0/390866042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 msgr2=0x7fa05c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 0x7fa05c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 -- 172.21.15.94:0/390866042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 msgr2=0x7fa05c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 0x7fa05c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 -- 172.21.15.94:0/390866042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1a1d70 con 0x7fa05c102240 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 0x7fa05c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa05affd640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa050006010 tx=0x7fa0500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa058ff9640 1 -- 172.21.15.94:0/390866042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05003f030 con 0x7fa05c102240 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa058ff9640 1 -- 172.21.15.94:0/390866042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa050031910 con 0x7fa05c102240 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c1a2000 con 0x7fa05c102240 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa058ff9640 1 -- 172.21.15.94:0/390866042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050035070 con 0x7fa05c102240 2024-04-25T14:00:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.692+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1a24f0 con 0x7fa05c102240 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa058ff9640 1 -- 172.21.15.94:0/390866042 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa050042030 con 0x7fa05c102240 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 Processor -- start 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 -- start start 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b727d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b3646b0 0x56363b728a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56363b68bd80 con 0x56363b385390 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56363b645a20 con 0x56363b3646b0 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa068fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56363b687d40 con 0x56363b35c4d0 2024-04-25T14:00:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa058ff9640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 0x7fa0440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa05bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32828/0 (socket says 172.21.15.94:32828) 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa05bfff640 1 -- 172.21.15.94:0/4104396533 learned_addr learned my addr 172.21.15.94:0/4104396533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b727d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.695+0000 7fa05affd640 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b3646b0 0x56363b728a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 0x7fa0440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa058ff9640 1 -- 172.21.15.94:0/390866042 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0500ff1f0 con 0x7fa05c102240 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 -- 172.21.15.94:0/4104396533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 msgr2=0x56363b727d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b727d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 -- 172.21.15.94:0/4104396533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b3646b0 msgr2=0x56363b728a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b3646b0 0x56363b728a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 -- 172.21.15.94:0/4104396533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363b650c30 con 0x56363b385390 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fa054005c40 tx=0x7fa05400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10b43f8110ce431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa05bfff640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 0x7fa0440b2000 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa05400f8c0 tx=0x7fa054021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.696+0000 7fa041ffb640 1 -- 172.21.15.94:0/4104396533 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05402a070 con 0x56363b385390 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa041ffb640 1 -- 172.21.15.94:0/4104396533 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054004440 con 0x56363b385390 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa041ffb640 1 -- 172.21.15.94:0/4104396533 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05401d920 con 0x56363b385390 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa068fb8740 1 -- 172.21.15.94:0/4104396533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 msgr2=0x56363b727810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fa054005c40 tx=0x7fa05400f3c0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa068fb8740 1 -- 172.21.15.94:0/4104396533 shutdown_connections 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.697+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b3646b0 0x56363b728a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b727d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4104396533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b385390 0x56363b727810 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- 172.21.15.94:0/4104396533 >> 172.21.15.94:0/4104396533 conn(0x56363b3794e0 msgr2=0x56363b723020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- 172.21.15.94:0/4104396533 shutdown_connections 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- 172.21.15.94:0/4104396533 wait complete. 2024-04-25T14:00:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- start start 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 Processor -- start 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b731550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b385390 0x56363b7389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56363b7294d0 con 0x56363b3646b0 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56363b612dc0 con 0x56363b385390 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56363b729650 con 0x56363b35c4d0 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.698+0000 7fa068fb8740 10 client.0 _ll_register_callbacks cb 0x56363b62e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32834/0 (socket says 172.21.15.94:32834) 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 -- 172.21.15.94:0/858053926 learned_addr learned my addr 172.21.15.94:0/858053926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b731550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05bfff640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b385390 0x56363b7389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 -- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 msgr2=0x56363b731550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b731550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 -- 172.21.15.94:0/858053926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b385390 msgr2=0x56363b7389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b385390 0x56363b7389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 -- 172.21.15.94:0/858053926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363b39d3f0 con 0x56363b3646b0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa05affd640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa0500c73e0 tx=0x7fa0500c36d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7445edcf65252a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa03affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050049040 con 0x56363b3646b0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0500c7b40 con 0x56363b3646b0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa054004440 con 0x56363b3646b0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa0500c71f0 con 0x56363b3646b0 2024-04-25T14:00:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa068fb8740 10 client.22143 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56363b4677d0 con 0x56363b3646b0 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.699+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054004440 con 0x56363b3646b0 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa0500aeb80 con 0x56363b3646b0 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 10 client.22143 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 10 client.22143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 20 client.22143 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b343f60 0x56363b73f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 4 client.22143 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363b699990 con 0x56363b343f60 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b343f60 0x56363b73f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa03affd640 10 client.22143 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.700+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b343f60 0x56363b73f190 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=102f5b23e95416a9 server_cookie=5f72e985e4b87b3d in_seq=0 out_seq=0 2024-04-25T14:00:23.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa03affd640 1 -- 172.21.15.94:0/858053926 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56363b699990 con 0x56363b343f60 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa03affd640 10 client.22143 handle_command_reply: tid=0 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 client.22143 shutdown 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 msgr2=0x56363b731a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa0500c73e0 tx=0x7fa0500c36d0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 shutdown_connections 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b343f60 0x56363b73f190 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b385390 0x56363b7389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa03affd640 0 client.22143 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b731a90 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 --2- 172.21.15.94:0/858053926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b35c4d0 0x56363b731550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 >> 172.21.15.94:0/858053926 conn(0x56363b3794e0 msgr2=0x56363b725f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 shutdown_connections 2024-04-25T14:00:23.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.701+0000 7fa068fb8740 1 -- 172.21.15.94:0/858053926 wait complete. 2024-04-25T14:00:23.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.702+0000 7fa068fb8740 20 client.22143 trim_cache size 0 max 16384 2024-04-25T14:00:23.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 Processor -- start 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 -- start start 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b722720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b722c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1a9360 con 0x56363b3646b0 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aaea0 con 0x56363b35c4d0 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa068fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1a9a30 con 0x56363b385390 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32842/0 (socket says 172.21.15.94:32842) 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 -- 172.21.15.94:0/4185718871 learned_addr learned my addr 172.21.15.94:0/4185718871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b722720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05bfff640 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b722c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 -- 172.21.15.94:0/4185718871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 msgr2=0x56363b722c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b722c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 -- 172.21.15.94:0/4185718871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 msgr2=0x56363b722720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b722720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.716+0000 7fa05affd640 1 -- 172.21.15.94:0/4185718871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363b72bdc0 con 0x56363b3646b0 2024-04-25T14:00:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.717+0000 7fa05affd640 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fa0500c7670 tx=0x7fa0500d0360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baa348c18177425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.717+0000 7fa0417fa640 1 -- 172.21.15.94:0/4185718871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050049070 con 0x56363b3646b0 2024-04-25T14:00:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.717+0000 7fa0417fa640 1 -- 172.21.15.94:0/4185718871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0500d0000 con 0x56363b3646b0 2024-04-25T14:00:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.717+0000 7fa0417fa640 1 -- 172.21.15.94:0/4185718871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0500c3140 con 0x56363b3646b0 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- 172.21.15.94:0/4185718871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 msgr2=0x56363b738910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fa0500c7670 tx=0x7fa0500d0360 comp rx=0 tx=0).stop 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- 172.21.15.94:0/4185718871 shutdown_connections 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b722c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b738910 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- 172.21.15.94:0/4185718871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b722720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- 172.21.15.94:0/4185718871 >> 172.21.15.94:0/4185718871 conn(0x56363b3794e0 msgr2=0x56363b72c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- 172.21.15.94:0/4185718871 shutdown_connections 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- 172.21.15.94:0/4185718871 wait complete. 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- start start 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 Processor -- start 2024-04-25T14:00:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b735b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b7add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aa7d0 con 0x56363b3646b0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aa100 con 0x56363b35c4d0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.718+0000 7fa068fb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1a7340 con 0x56363b385390 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa068fb8740 10 client.0 _ll_register_callbacks cb 0x7fa0500ea050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32852/0 (socket says 172.21.15.94:32852) 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 -- 172.21.15.94:0/3877723875 learned_addr learned my addr 172.21.15.94:0/3877723875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b735b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05bfff640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b7add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 -- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 msgr2=0x56363b7add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b7add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 -- 172.21.15.94:0/3877723875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 msgr2=0x56363b735b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b735b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 -- 172.21.15.94:0/3877723875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363b7ae2b0 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa05affd640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fa05003d040 tx=0x7fa0500c36d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b15d6e8c14a1a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa03affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050049070 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0500c6940 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0500d0000 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa0500c7030 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa068fb8740 10 client.22149 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.719+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56363b7ae2b0 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0500d0000 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa0500c63b0 con 0x56363b3646b0 2024-04-25T14:00:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 10 client.22149 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 10 client.22149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 20 client.22149 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b3193a0 0x56363b7b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 4 client.22149 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56363b693800 con 0x56363b3193a0 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b3193a0 0x56363b7b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa03affd640 10 client.22149 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.720+0000 7fa05b7fe640 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b3193a0 0x56363b7b1b30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1213c5fe5808cb43 server_cookie=d5a67a95f2dd9d00 in_seq=0 out_seq=0 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa03affd640 1 -- 172.21.15.94:0/3877723875 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56363b693800 con 0x56363b3193a0 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa03affd640 10 client.22149 handle_command_reply: tid=0 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 client.22149 shutdown 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 msgr2=0x56363b736090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fa05003d040 tx=0x7fa0500c36d0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 shutdown_connections 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56363b3193a0 0x56363b7b1b30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56363b385390 0x56363b7add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363b3646b0 0x56363b736090 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 --2- 172.21.15.94:0/3877723875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363b35c4d0 0x56363b735b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa03affd640 0 client.22149 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 >> 172.21.15.94:0/3877723875 conn(0x56363b3794e0 msgr2=0x56363b721610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 shutdown_connections 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 1 -- 172.21.15.94:0/3877723875 wait complete. 2024-04-25T14:00:23.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa068fb8740 20 client.22149 trim_cache size 0 max 16384 2024-04-25T14:00:23.731 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 msgr2=0x7fa0440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 0x7fa0440b2000 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa05400f8c0 tx=0x7fa054021000 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 msgr2=0x7fa05c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa050006010 tx=0x7fa0500042f0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 shutdown_connections 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0440afaf0 0x7fa0440b2000 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa05c105090 0x7fa05c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa05c102240 0x7fa05c19a770 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 --2- 172.21.15.94:0/390866042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa05c101740 0x7fa05c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.721+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 >> 172.21.15.94:0/390866042 conn(0x7fa05c0fd340 msgr2=0x7fa05c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.722+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 shutdown_connections 2024-04-25T14:00:23.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.722+0000 7fa062106640 1 -- 172.21.15.94:0/390866042 wait complete. 2024-04-25T14:00:23.734 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 -- 172.21.15.94:0/3716636757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec103940 msgr2=0x7f23ec103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/3716636757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec103940 0x7f23ec103dd0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009900 tx=0x7f23d802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 -- 172.21.15.94:0/3716636757 shutdown_connections 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/3716636757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec104460 0x7f23ec10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/3716636757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec103940 0x7f23ec103dd0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/3716636757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 -- 172.21.15.94:0/3716636757 >> 172.21.15.94:0/3716636757 conn(0x7f23ec0fd340 msgr2=0x7f23ec0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 -- 172.21.15.94:0/3716636757 shutdown_connections 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.847+0000 7f23f2fd7640 1 -- 172.21.15.94:0/3716636757 wait complete. 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 Processor -- start 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 -- start start 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 0x7f23ec199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e530 con 0x7f23ec104460 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e3b0 con 0x7f23ec103940 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f2fd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec10e6b0 con 0x7f23ec102f60 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32874/0 (socket says 172.21.15.94:32874) 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 -- 172.21.15.94:0/1636474935 learned_addr learned my addr 172.21.15.94:0/1636474935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23ebfff640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 0x7f23ec199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 -- 172.21.15.94:0/1636474935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 msgr2=0x7f23ec199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 -- 172.21.15.94:0/1636474935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 msgr2=0x7f23ec199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 0x7f23ec199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f1574640 1 -- 172.21.15.94:0/1636474935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec1a0c40 con 0x7f23ec104460 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23f0d73640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.848+0000 7f23ebfff640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 0x7f23ec199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23f1574640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23e9ffb640 1 -- 172.21.15.94:0/1636474935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004b20 con 0x7f23ec104460 2024-04-25T14:00:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23e9ffb640 1 -- 172.21.15.94:0/1636474935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e4004cc0 con 0x7f23ec104460 2024-04-25T14:00:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23e9ffb640 1 -- 172.21.15.94:0/1636474935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4010670 con 0x7f23ec104460 2024-04-25T14:00:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1a0f30 con 0x7f23ec104460 2024-04-25T14:00:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.849+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1a1340 con 0x7f23ec104460 2024-04-25T14:00:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.851+0000 7f23e9ffb640 1 -- 172.21.15.94:0/1636474935 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f23e4005020 con 0x7f23ec104460 2024-04-25T14:00:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 Processor -- start 2024-04-25T14:00:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 -- start start 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23e9ffb640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e23fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5596c5d85d80 con 0x5596c5a564d0 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596c5d3fa20 con 0x5596c5a5e6b0 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f9f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596c5d81d40 con 0x5596c5a7f390 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23ebfff640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23e9ffb640 1 -- 172.21.15.94:0/1636474935 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23e40df280 con 0x7f23ec104460 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e23fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56214/0 (socket says 172.21.15.94:56214) 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f0d73640 1 -- 172.21.15.94:0/3238002014 learned_addr learned my addr 172.21.15.94:0/3238002014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.852+0000 7f23f1574640 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 -- 172.21.15.94:0/3238002014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 msgr2=0x5596c5e24c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e24c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 -- 172.21.15.94:0/3238002014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 msgr2=0x5596c5e23fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e23fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 -- 172.21.15.94:0/3238002014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c5d4ac30 con 0x5596c5a7f390 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f1574640 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e24c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23ebfff640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 0x7f23dc0b2000 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009230 tx=0x7f23d8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23e0010450 tx=0x7f23e0010850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3670e96b84493ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23d6ffd640 1 -- 172.21.15.94:0/3238002014 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0011700 con 0x5596c5a7f390 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23d6ffd640 1 -- 172.21.15.94:0/3238002014 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0011d50 con 0x5596c5a7f390 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.853+0000 7f23d6ffd640 1 -- 172.21.15.94:0/3238002014 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0019e30 con 0x5596c5a7f390 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- 172.21.15.94:0/3238002014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 msgr2=0x5596c5e23a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23e0010450 tx=0x7f23e0010850 comp rx=0 tx=0).stop 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- 172.21.15.94:0/3238002014 shutdown_connections 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e24c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e23fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3238002014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a7f390 0x5596c5e23a20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- 172.21.15.94:0/3238002014 >> 172.21.15.94:0/3238002014 conn(0x5596c5a734e0 msgr2=0x5596c5e1d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- 172.21.15.94:0/3238002014 shutdown_connections 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- 172.21.15.94:0/3238002014 wait complete. 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 -- start start 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.854+0000 7f23f9f65740 1 Processor -- start 2024-04-25T14:00:23.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e34bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5596c5e25740 con 0x5596c5a5e6b0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596c5d0cdc0 con 0x5596c5a7f390 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596c5e258c0 con 0x5596c5a564d0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f9f65740 10 client.0 _ll_register_callbacks cb 0x5596c5d28600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32892/0 (socket says 172.21.15.94:32892) 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 -- 172.21.15.94:0/3174328682 learned_addr learned my addr 172.21.15.94:0/3174328682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23f1574640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e34bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 -- 172.21.15.94:0/3174328682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 msgr2=0x5596c5e2d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e2d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 -- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 msgr2=0x5596c5e34bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e34bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.855+0000 7f23ebfff640 1 -- 172.21.15.94:0/3174328682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c5a973f0 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23ebfff640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f23d8031500 tx=0x7f23d8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b4c0af02dc9df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d804f030 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0011d50 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23d8046040 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23d8035040 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 10 client.22161 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596c5b617d0 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0011d50 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f23d8049040 con 0x5596c5a5e6b0 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 10 client.22161 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 10 client.22161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 20 client.22161 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a3df60 0x5596c5e3b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 4 client.22161 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.856+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596c5d93990 con 0x5596c5a3df60 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.857+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a3df60 0x5596c5e3b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.857+0000 7f23cbfff640 10 client.22161 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.857+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a3df60 0x5596c5e3b460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef61d678430b7004 server_cookie=ef7d774bee5427c3 in_seq=0 out_seq=0 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23cbfff640 1 -- 172.21.15.94:0/3174328682 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596c5d93990 con 0x5596c5a3df60 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23cbfff640 10 client.22161 handle_command_reply: tid=0 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 client.22161 shutdown 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 msgr2=0x5596c5e2dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f23d8031500 tx=0x7f23d8031400 comp rx=0 tx=0).stop 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 shutdown_connections 2024-04-25T14:00:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a3df60 0x5596c5e3b460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e34bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a5e6b0 0x5596c5e2dc50 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23cbfff640 0 client.22161 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3174328682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e2d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 >> 172.21.15.94:0/3174328682 conn(0x5596c5a734e0 msgr2=0x5596c5e1c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 shutdown_connections 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 1 -- 172.21.15.94:0/3174328682 wait complete. 2024-04-25T14:00:23.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.858+0000 7f23f9f65740 20 client.22161 trim_cache size 0 max 16384 2024-04-25T14:00:23.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 Processor -- start 2024-04-25T14:00:23.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 -- start start 2024-04-25T14:00:23.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a5e6b0 0x5596c5e34ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e40150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1a9220 con 0x5596c5a564d0 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1aad60 con 0x5596c5a7f390 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f9f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1a98f0 con 0x5596c5a5e6b0 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32904/0 (socket says 172.21.15.94:32904) 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 -- 172.21.15.94:0/2681625561 learned_addr learned my addr 172.21.15.94:0/2681625561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23ebfff640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a5e6b0 0x5596c5e34ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f1574640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e40150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 -- 172.21.15.94:0/2681625561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a5e6b0 msgr2=0x5596c5e34ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a5e6b0 0x5596c5e34ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 -- 172.21.15.94:0/2681625561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 msgr2=0x5596c5e40150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e40150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 -- 172.21.15.94:0/2681625561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c5e35010 con 0x5596c5a564d0 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f1574640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e40150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.873+0000 7f23f0d73640 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f23e0010d50 tx=0x7f23e0010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18751719fcab5641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23d67fc640 1 -- 172.21.15.94:0/2681625561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0005040 con 0x5596c5a564d0 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23d67fc640 1 -- 172.21.15.94:0/2681625561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0019720 con 0x5596c5a564d0 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23d67fc640 1 -- 172.21.15.94:0/2681625561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e002c500 con 0x5596c5a564d0 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 -- 172.21.15.94:0/2681625561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 msgr2=0x5596c5e3fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f23e0010d50 tx=0x7f23e0010e00 comp rx=0 tx=0).stop 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 -- 172.21.15.94:0/2681625561 shutdown_connections 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a7f390 0x5596c5e40150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a5e6b0 0x5596c5e34ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 --2- 172.21.15.94:0/2681625561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a564d0 0x5596c5e3fc10 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 -- 172.21.15.94:0/2681625561 >> 172.21.15.94:0/2681625561 conn(0x5596c5a734e0 msgr2=0x5596c5e284a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 -- 172.21.15.94:0/2681625561 shutdown_connections 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.874+0000 7f23f9f65740 1 -- 172.21.15.94:0/2681625561 wait complete. 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 -- start start 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 Processor -- start 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e321d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1aa690 con 0x5596c5a7f390 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1a9fc0 con 0x5596c5a5e6b0 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1a7200 con 0x5596c5a564d0 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f9f65740 10 client.0 _ll_register_callbacks cb 0x7f23d80057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f1574640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f1574640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32920/0 (socket says 172.21.15.94:32920) 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f0d73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23f1574640 1 -- 172.21.15.94:0/3633031014 learned_addr learned my addr 172.21.15.94:0/3633031014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.875+0000 7f23ebfff640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 -- 172.21.15.94:0/3633031014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 msgr2=0x5596c5e31c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e31c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 -- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 msgr2=0x5596c5e321d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e321d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 -- 172.21.15.94:0/3633031014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c5e336c0 con 0x5596c5a7f390 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e31c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f1574640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a4930 tx=0x7f23e40a70d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcb45838d55c744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40a71d0 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23e40a7920 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e0019720 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23e40a7dc0 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 10 client.22167 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596c5e34a10 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e0019720 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f23e40b0090 con 0x5596c5a7f390 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 10 client.22167 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 10 client.22167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 20 client.22167 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a133a0 0x5596c5ea5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 4 client.22167 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:23.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.876+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596c5d8d800 con 0x5596c5a133a0 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a133a0 0x5596c5ea5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f0d73640 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a133a0 0x5596c5ea5c20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87208a08360c5476 server_cookie=5dbd2858645459d5 in_seq=0 out_seq=0 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23cbfff640 10 client.22167 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23cbfff640 1 -- 172.21.15.94:0/3633031014 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596c5d8d800 con 0x5596c5a133a0 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23cbfff640 10 client.22167 handle_command_reply: tid=0 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 client.22167 shutdown 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 msgr2=0x5596c5e33100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a4930 tx=0x7f23e40a70d0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 shutdown_connections 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5596c5a133a0 0x5596c5ea5c20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596c5a7f390 0x5596c5e33100 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c5a5e6b0 0x5596c5e321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23cbfff640 0 client.22167 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 --2- 172.21.15.94:0/3633031014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596c5a564d0 0x5596c5e31c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.877+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 >> 172.21.15.94:0/3633031014 conn(0x5596c5a734e0 msgr2=0x5596c5e1fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 shutdown_connections 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f9f65740 1 -- 172.21.15.94:0/3633031014 wait complete. 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f9f65740 20 client.22167 trim_cache size 0 max 16384 2024-04-25T14:00:23.886 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:23.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 msgr2=0x7f23dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 0x7f23dc0b2000 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009230 tx=0x7f23d8005db0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 msgr2=0x7f23ec1a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 shutdown_connections 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec104460 0x7f23ec1a05d0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ec103940 0x7f23ec199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 --2- 172.21.15.94:0/1636474935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102f60 0x7f23ec199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 >> 172.21.15.94:0/1636474935 conn(0x7f23ec0fd340 msgr2=0x7f23ec104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 shutdown_connections 2024-04-25T14:00:23.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:23.878+0000 7f23f2fd7640 1 -- 172.21.15.94:0/1636474935 wait complete. 2024-04-25T14:00:23.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 -- 172.21.15.94:0/3823138779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 msgr2=0x7fdd08106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 --2- 172.21.15.94:0/3823138779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd08106a00 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009900 tx=0x7fdd0402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 -- 172.21.15.94:0/3823138779 shutdown_connections 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 --2- 172.21.15.94:0/3823138779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 0x7fdd0810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 --2- 172.21.15.94:0/3823138779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 0x7fdd08109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 --2- 172.21.15.94:0/3823138779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd08106a00 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 -- 172.21.15.94:0/3823138779 >> 172.21.15.94:0/3823138779 conn(0x7fdd080fdc00 msgr2=0x7fdd08100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 -- 172.21.15.94:0/3823138779 shutdown_connections 2024-04-25T14:00:24.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.008+0000 7fdd0f346640 1 -- 172.21.15.94:0/3823138779 wait complete. 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 Processor -- start 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 -- start start 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 0x7fdd0819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 0x7fdd081a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f5f0 con 0x7fdd081045e0 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f470 con 0x7fdd08106f40 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0f346640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0810f770 con 0x7fdd081098a0 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32942/0 (socket says 172.21.15.94:32942) 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/296260207 learned_addr learned my addr 172.21.15.94:0/296260207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 0x7fdd0819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 0x7fdd081a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/296260207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 msgr2=0x7fdd081a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 0x7fdd081a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/296260207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 msgr2=0x7fdd0819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 0x7fdd0819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/296260207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd081a1d90 con 0x7fdd081045e0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.009+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006010 tx=0x7fdd04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.010+0000 7fdcfe7fc640 1 -- 172.21.15.94:0/296260207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0403f030 con 0x7fdd081045e0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.010+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd081a2020 con 0x7fdd081045e0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.010+0000 7fdcfe7fc640 1 -- 172.21.15.94:0/296260207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd04031910 con 0x7fdd081045e0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.010+0000 7fdcfe7fc640 1 -- 172.21.15.94:0/296260207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04035070 con 0x7fdd081045e0 2024-04-25T14:00:24.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.010+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd081a2510 con 0x7fdd081045e0 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdcfe7fc640 1 -- 172.21.15.94:0/296260207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdd04042030 con 0x7fdd081045e0 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 Processor -- start 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 -- start start 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d451496d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451497a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d4513fad80 con 0x55d4510f4390 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d4513b4a20 con 0x55d4510cb4d0 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.012+0000 7fdd161f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d4513f6d40 con 0x55d4510d36b0 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32956/0 (socket says 172.21.15.94:32956) 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/2372004803 learned_addr learned my addr 172.21.15.94:0/2372004803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d451496d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451497a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdcfe7fc640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 0x7fdcf40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/2372004803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 msgr2=0x55d451497a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451497a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/2372004803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 msgr2=0x55d451496d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d451496d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/2372004803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4513bfc30 con 0x55d4510f4390 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451497a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdcfe7fc640 1 -- 172.21.15.94:0/296260207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdd040ff0b0 con 0x7fdd081045e0 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009ea0 tx=0x7fdd00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c26d960a481650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 0x7fdcf40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdcf37fe640 1 -- 172.21.15.94:0/2372004803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000116c0 con 0x55d4510f4390 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdcf37fe640 1 -- 172.21.15.94:0/2372004803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0001b490 con 0x55d4510f4390 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.013+0000 7fdcf37fe640 1 -- 172.21.15.94:0/2372004803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000106d0 con 0x55d4510f4390 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 -- 172.21.15.94:0/2372004803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 msgr2=0x55d451496810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 0x7fdcf40b2000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdd00010010 tx=0x7fdd00010d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009ea0 tx=0x7fdd00009f90 comp rx=0 tx=0).stop 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 -- 172.21.15.94:0/2372004803 shutdown_connections 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451497a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d451496d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2372004803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510f4390 0x55d451496810 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 -- 172.21.15.94:0/2372004803 >> 172.21.15.94:0/2372004803 conn(0x55d4510e84e0 msgr2=0x55d451492020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 -- 172.21.15.94:0/2372004803 shutdown_connections 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.014+0000 7fdd161f6740 1 -- 172.21.15.94:0/2372004803 wait complete. 2024-04-25T14:00:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 -- start start 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 Processor -- start 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d4514a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510f4390 0x55d4514a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d451498530 con 0x55d4510cb4d0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d451381dc0 con 0x55d4510f4390 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d4514986b0 con 0x55d4510d36b0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd161f6740 10 client.0 _ll_register_callbacks cb 0x55d45139d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32964/0 (socket says 172.21.15.94:32964) 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2474256264 learned_addr learned my addr 172.21.15.94:0/2474256264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d4514a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.015+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510f4390 0x55d4514a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2474256264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 msgr2=0x55d4514a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d4514a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510f4390 msgr2=0x55d4514a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510f4390 0x55d4514a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2474256264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45110c3f0 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fdd04031500 tx=0x7fdd0402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cf607dd854c195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04049040 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdd04003d10 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd0001b490 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd040c6e40 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd161f6740 10 client.22179 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d4511d67d0 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0001b490 con 0x55d4510cb4d0 2024-04-25T14:00:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.016+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdd040c3780 con 0x55d4510cb4d0 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 10 client.22179 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 10 client.22179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 20 client.22179 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510b2f60 0x55d4514ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 4 client.22179 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d451408990 con 0x55d4510b2f60 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510b2f60 0x55d4514ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdcf0ff9640 10 client.22179 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:24.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.017+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510b2f60 0x55d4514ae190 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a29e1959e9771c server_cookie=668ec298b2dc8b9 in_seq=0 out_seq=0 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2474256264 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d451408990 con 0x55d4510b2f60 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdcf0ff9640 10 client.22179 handle_command_reply: tid=0 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 client.22179 shutdown 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 msgr2=0x55d4514a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fdd04031500 tx=0x7fdd0402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 shutdown_connections 2024-04-25T14:00:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510b2f60 0x55d4514ae190 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510f4390 0x55d4514a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdcf0ff9640 0 client.22179 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d4514a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2474256264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510cb4d0 0x55d4514a05c0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 >> 172.21.15.94:0/2474256264 conn(0x55d4510e84e0 msgr2=0x55d451494f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 shutdown_connections 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.021+0000 7fdd161f6740 1 -- 172.21.15.94:0/2474256264 wait complete. 2024-04-25T14:00:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.022+0000 7fdd161f6740 20 client.22179 trim_cache size 0 max 16384 2024-04-25T14:00:24.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 Processor -- start 2024-04-25T14:00:24.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 -- start start 2024-04-25T14:00:24.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510b2f60 0x55d451492160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510cb4d0 0x55d4514a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081ab510 con 0x55d4510d36b0 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081ad050 con 0x55d4510b2f60 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd161f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081abbe0 con 0x55d4510cb4d0 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.036+0000 7fdd0d8e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32974/0 (socket says 172.21.15.94:32974) 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/4239829971 learned_addr learned my addr 172.21.15.94:0/4239829971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510cb4d0 0x55d4514a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510b2f60 0x55d451492160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/4239829971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510cb4d0 msgr2=0x55d4514a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510cb4d0 0x55d4514a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/4239829971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510b2f60 msgr2=0x55d451492160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510b2f60 0x55d451492160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 -- 172.21.15.94:0/4239829971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d451496220 con 0x55d4510d36b0 2024-04-25T14:00:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001a740 tx=0x7fdd00026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f31310fd08f0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdcf2ffd640 1 -- 172.21.15.94:0/4239829971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0002c040 con 0x55d4510d36b0 2024-04-25T14:00:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdcf2ffd640 1 -- 172.21.15.94:0/4239829971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd00036070 con 0x55d4510d36b0 2024-04-25T14:00:24.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.037+0000 7fdcf2ffd640 1 -- 172.21.15.94:0/4239829971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00010790 con 0x55d4510d36b0 2024-04-25T14:00:24.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 -- 172.21.15.94:0/4239829971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 msgr2=0x55d4514a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001a740 tx=0x7fdd00026000 comp rx=0 tx=0).stop 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 -- 172.21.15.94:0/4239829971 shutdown_connections 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510d36b0 0x55d4514a06d0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510cb4d0 0x55d4514a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 --2- 172.21.15.94:0/4239829971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510b2f60 0x55d451492160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 -- 172.21.15.94:0/4239829971 >> 172.21.15.94:0/4239829971 conn(0x55d4510e84e0 msgr2=0x55d451491a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 -- 172.21.15.94:0/4239829971 shutdown_connections 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.038+0000 7fdd161f6740 1 -- 172.21.15.94:0/4239829971 wait complete. 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 -- start start 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 Processor -- start 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d4514a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451514530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081ac980 con 0x55d4510b2f60 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081ac2b0 con 0x55d4510cb4d0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081a94f0 con 0x55d4510d36b0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd161f6740 10 client.0 _ll_register_callbacks cb 0x7fdd04005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd0d0e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32990/0 (socket says 172.21.15.94:32990) 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2721274316 learned_addr learned my addr 172.21.15.94:0/2721274316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d4514a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.039+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451514530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2721274316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 msgr2=0x55d451514530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451514530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 msgr2=0x55d4514a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d4514a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 -- 172.21.15.94:0/2721274316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d451514a70 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d8e3640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451514530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d4514a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd0d0e2640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b01b0 tx=0x7fdd0403b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31beabaef2f8e3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04049070 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdd040d00d0 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd00036070 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd040c6c60 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 10 client.22185 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d451515c90 con 0x55d4510b2f60 2024-04-25T14:00:24.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd00036070 con 0x55d4510b2f60 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdd040e8210 con 0x55d4510b2f60 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 10 client.22185 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 10 client.22185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 20 client.22185 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510f4390 0x55d4515184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 4 client.22185 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.040+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d451402800 con 0x55d4510f4390 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.041+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510f4390 0x55d4515184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.041+0000 7fdcf0ff9640 10 client.22185 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.041+0000 7fdd0c8e1640 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510f4390 0x55d4515184f0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f823340c3dd341c7 server_cookie=847a4b5edfed783f in_seq=0 out_seq=0 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.043+0000 7fdcf0ff9640 1 -- 172.21.15.94:0/2721274316 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d451402800 con 0x55d4510f4390 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.043+0000 7fdcf0ff9640 10 client.22185 handle_command_reply: tid=0 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.043+0000 7fdd161f6740 1 client.22185 shutdown 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.043+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 msgr2=0x55d4514a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.043+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b01b0 tx=0x7fdd0403b4d0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 shutdown_connections 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d4510f4390 0x55d4515184f0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d4510d36b0 0x55d451514530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d4510cb4d0 0x55d4514a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 --2- 172.21.15.94:0/2721274316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d4510b2f60 0x55d4514a40f0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 >> 172.21.15.94:0/2721274316 conn(0x55d4510e84e0 msgr2=0x55d4514a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdcf0ff9640 0 client.22185 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:24.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 shutdown_connections 2024-04-25T14:00:24.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 1 -- 172.21.15.94:0/2721274316 wait complete. 2024-04-25T14:00:24.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd161f6740 20 client.22185 trim_cache size 0 max 16384 2024-04-25T14:00:24.049 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 msgr2=0x7fdcf40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 0x7fdcf40b2000 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdd00010010 tx=0x7fdd00010d80 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 msgr2=0x7fdd0819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fdd04006010 tx=0x7fdd04004270 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 shutdown_connections 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdcf40afaf0 0x7fdcf40b2000 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd081098a0 0x7fdd081a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdd08106f40 0x7fdd0819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 --2- 172.21.15.94:0/296260207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd081045e0 0x7fdd0819a250 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:24.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.044+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 >> 172.21.15.94:0/296260207 conn(0x7fdd080fdc00 msgr2=0x7fdd081051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.045+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 shutdown_connections 2024-04-25T14:00:24.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:24.045+0000 7fdd0f346640 1 -- 172.21.15.94:0/296260207 wait complete. 2024-04-25T14:00:24.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 111015 19.28 MB/sec execute 186 sec latency 71.497 ms 2024-04-25T14:00:24.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:23 smithi094 ceph-mon[29195]: pgmap v909: 97 pgs: 97 active+clean; 258 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-04-25T14:00:24.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:23 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1427606174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:24.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:23 smithi138 ceph-mon[32977]: pgmap v909: 97 pgs: 97 active+clean; 258 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-04-25T14:00:24.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:23 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1427606174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:23 smithi137 ceph-mon[32568]: pgmap v909: 97 pgs: 97 active+clean; 258 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 315 op/s 2024-04-25T14:00:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:23 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1427606174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:25.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:00:25.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:00:25.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:00:25.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 111378 19.27 MB/sec execute 187 sec latency 56.823 ms 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- 172.21.15.94:0/657345303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5741098a0 msgr2=0x7fe57410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 --2- 172.21.15.94:0/657345303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5741098a0 0x7fe57410bcc0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009ca0 tx=0x7fe56c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- 172.21.15.94:0/657345303 shutdown_connections 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 --2- 172.21.15.94:0/657345303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5741098a0 0x7fe57410bcc0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 --2- 172.21.15.94:0/657345303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe574106f40 0x7fe574109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 --2- 172.21.15.94:0/657345303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 0x7fe574106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- 172.21.15.94:0/657345303 >> 172.21.15.94:0/657345303 conn(0x7fe5740fdc00 msgr2=0x7fe574100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- 172.21.15.94:0/657345303 shutdown_connections 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- 172.21.15.94:0/657345303 wait complete. 2024-04-25T14:00:25.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 Processor -- start 2024-04-25T14:00:25.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.176+0000 7fe579bd6640 1 -- start start 2024-04-25T14:00:25.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 0x7fe574103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5741098a0 0x7fe574102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe57410f670 con 0x7fe574106f40 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe57410f4f0 con 0x7fe5741098a0 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe579bd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe57410f7f0 con 0x7fe5741045e0 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33010/0 (socket says 172.21.15.94:33010) 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 -- 172.21.15.94:0/1951143274 learned_addr learned my addr 172.21.15.94:0/1951143274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe573fff640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5741098a0 0x7fe574102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe5737fe640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 0x7fe574103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 -- 172.21.15.94:0/1951143274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 msgr2=0x7fe574103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 0x7fe574103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 -- 172.21.15.94:0/1951143274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5741098a0 msgr2=0x7fe574102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5741098a0 0x7fe574102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 -- 172.21.15.94:0/1951143274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574102d40 con 0x7fe574106f40 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.177+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9f0 tx=0x7fe56400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.178+0000 7fe570ff9640 1 -- 172.21.15.94:0/1951143274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400cd80 con 0x7fe574106f40 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.178+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5741a42e0 con 0x7fe574106f40 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.178+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741a47d0 con 0x7fe574106f40 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.178+0000 7fe570ff9640 1 -- 172.21.15.94:0/1951143274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5640108e0 con 0x7fe574106f40 2024-04-25T14:00:25.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.178+0000 7fe570ff9640 1 -- 172.21.15.94:0/1951143274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564018980 con 0x7fe574106f40 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.180+0000 7fe580a85740 1 Processor -- start 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.180+0000 7fe580a85740 1 -- start start 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555958ffb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555958ffba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe570ff9640 1 -- 172.21.15.94:0/1951143274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe564010430 con 0x7fe574106f40 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555958f60b00 con 0x555958c39730 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555958efc920 con 0x555958c5a410 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe580a85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555958f5cac0 con 0x555958c31550 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe573fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555958ffb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33022/0 (socket says 172.21.15.94:33022) 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 -- 172.21.15.94:0/2934130880 learned_addr learned my addr 172.21.15.94:0/2934130880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe5737fe640 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555958ffba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 -- 172.21.15.94:0/2934130880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 msgr2=0x555958ffba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555958ffba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 -- 172.21.15.94:0/2934130880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 msgr2=0x555958ffb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555958ffb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 -- 172.21.15.94:0/2934130880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555958d3a7e0 con 0x555958c39730 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe572ffd640 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fe5640e7260 tx=0x7fe5640e7720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6524658cdcbaf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe570ff9640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.181+0000 7fe573fff640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe559ffb640 1 -- 172.21.15.94:0/2934130880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640f2070 con 0x555958c39730 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe570ff9640 1 -- 172.21.15.94:0/1951143274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe5640defd0 con 0x7fe574106f40 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe559ffb640 1 -- 172.21.15.94:0/2934130880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5640e12e0 con 0x555958c39730 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe559ffb640 1 -- 172.21.15.94:0/2934130880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640a6e10 con 0x555958c39730 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe580a85740 1 -- 172.21.15.94:0/2934130880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 msgr2=0x555959002920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe573fff640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005e30 tx=0x7fe56c031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.182+0000 7fe580a85740 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fe5640e7260 tx=0x7fe5640e7720 comp rx=0 tx=0).stop 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- 172.21.15.94:0/2934130880 shutdown_connections 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959002920 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555958ffba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- 172.21.15.94:0/2934130880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555958ffb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- 172.21.15.94:0/2934130880 >> 172.21.15.94:0/2934130880 conn(0x555958c4e560 msgr2=0x555958ff6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- 172.21.15.94:0/2934130880 shutdown_connections 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- 172.21.15.94:0/2934130880 wait complete. 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- start start 2024-04-25T14:00:25.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 Processor -- start 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x5559590052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555959003370 con 0x555958c39730 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555958efa5d0 con 0x555958c5a410 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5559590034f0 con 0x555958c31550 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.183+0000 7fe580a85740 10 client.0 _ll_register_callbacks cb 0x555958ee8080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33036/0 (socket says 172.21.15.94:33036) 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 -- 172.21.15.94:0/1455951850 learned_addr learned my addr 172.21.15.94:0/1455951850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe5737fe640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x5559590052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe573fff640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 -- 172.21.15.94:0/1455951850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 msgr2=0x5559590052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x5559590052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 -- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 msgr2=0x55595900c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 -- 172.21.15.94:0/1455951850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555958d9af70 con 0x555958c39730 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe573fff640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe572ffd640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a34f0 tx=0x7fe5640e2880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dbadf2fd86557c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640a6c30 con 0x555958c39730 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe5640e2be0 con 0x555958c39730 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555958f6e710 con 0x555958c39730 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe5640a6e10 con 0x555958c39730 2024-04-25T14:00:25.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe580a85740 10 client.22197 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555958c32170 con 0x555958c39730 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.184+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555958f6e710 con 0x555958c39730 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe5640a3750 con 0x555958c39730 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 10 client.22197 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 10 client.22197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 20 client.22197 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958c191f0 0x555959012820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 4 client.22197 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555958f68580 con 0x555958c191f0 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe5737fe640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958c191f0 0x555959012820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe552ffd640 10 client.22197 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.185+0000 7fe5737fe640 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958c191f0 0x555959012820 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9efe614c2154b43f server_cookie=bd779053736d422c in_seq=0 out_seq=0 2024-04-25T14:00:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe552ffd640 1 -- 172.21.15.94:0/1455951850 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555958f68580 con 0x555958c191f0 2024-04-25T14:00:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe552ffd640 10 client.22197 handle_command_reply: tid=0 2024-04-25T14:00:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 client.22197 shutdown 2024-04-25T14:00:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 msgr2=0x555959005800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a34f0 tx=0x7fe5640e2880 comp rx=0 tx=0).stop 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 shutdown_connections 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958c191f0 0x555959012820 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959005800 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe552ffd640 0 client.22197 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 --2- 172.21.15.94:0/1455951850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x5559590052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 >> 172.21.15.94:0/1455951850 conn(0x555958c4e560 msgr2=0x555958ff5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 shutdown_connections 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.192+0000 7fe580a85740 1 -- 172.21.15.94:0/1455951850 wait complete. 2024-04-25T14:00:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.193+0000 7fe580a85740 20 client.22197 trim_cache size 0 max 16384 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 Processor -- start 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 -- start start 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c39730 0x555959003890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555959003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ad820 con 0x555958c31550 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741af360 con 0x555958c5a410 2024-04-25T14:00:25.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe580a85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741adef0 con 0x555958c39730 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.211+0000 7fe5737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33042/0 (socket says 172.21.15.94:33042) 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 -- 172.21.15.94:0/148600238 learned_addr learned my addr 172.21.15.94:0/148600238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe572ffd640 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c39730 0x555959003890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe573fff640 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555959003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 -- 172.21.15.94:0/148600238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c39730 msgr2=0x555959003890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c39730 0x555959003890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 -- 172.21.15.94:0/148600238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 msgr2=0x555959003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555959003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 -- 172.21.15.94:0/148600238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555959000170 con 0x555958c31550 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5737fe640 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fe57406bc70 tx=0x7fe56800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9a8b172ddc0bff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5597fa640 1 -- 172.21.15.94:0/148600238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800bd40 con 0x555958c31550 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5597fa640 1 -- 172.21.15.94:0/148600238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5680054d0 con 0x555958c31550 2024-04-25T14:00:25.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.212+0000 7fe5597fa640 1 -- 172.21.15.94:0/148600238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680201b0 con 0x555958c31550 2024-04-25T14:00:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 -- 172.21.15.94:0/148600238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 msgr2=0x555958ff6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fe57406bc70 tx=0x7fe56800b390 comp rx=0 tx=0).stop 2024-04-25T14:00:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 -- 172.21.15.94:0/148600238 shutdown_connections 2024-04-25T14:00:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x555959003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c39730 0x555959003890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 --2- 172.21.15.94:0/148600238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c31550 0x555958ff6f00 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 -- 172.21.15.94:0/148600238 >> 172.21.15.94:0/148600238 conn(0x555958c4e560 msgr2=0x555958ff67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 -- 172.21.15.94:0/148600238 shutdown_connections 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.213+0000 7fe580a85740 1 -- 172.21.15.94:0/148600238 wait complete. 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 -- start start 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 Processor -- start 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555959009450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741aec90 con 0x555958c39730 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ae5c0 con 0x555958c5a410 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741ab800 con 0x555958c31550 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe580a85740 10 client.0 _ll_register_callbacks cb 0x7fe568004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33052/0 (socket says 172.21.15.94:33052) 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 -- 172.21.15.94:0/3747968026 learned_addr learned my addr 172.21.15.94:0/3747968026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe5737fe640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555959009450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe573fff640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 -- 172.21.15.94:0/3747968026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 msgr2=0x555959009450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555959009450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 -- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 msgr2=0x55595900a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.214+0000 7fe572ffd640 1 -- 172.21.15.94:0/3747968026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595900ae80 con 0x555958c39730 2024-04-25T14:00:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe572ffd640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fe564005620 tx=0x7fe5640df2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39df87f21d68d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640a6c30 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe5640e25c0 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5680054d0 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe5640a7290 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 10 client.22203 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55595907a9d0 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5680054d0 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe5640a77c0 con 0x555958c39730 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 10 client.22203 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 10 client.22203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 20 client.22203 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958bee630 0x55595907ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 4 client.22203 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:00:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.215+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555958ee3ac0 con 0x555958bee630 2024-04-25T14:00:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.216+0000 7fe5737fe640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958bee630 0x55595907ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.216+0000 7fe552ffd640 10 client.22203 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.216+0000 7fe5737fe640 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958bee630 0x55595907ce60 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71931bee9bd2db2f server_cookie=fd0a3130dc2220db in_seq=0 out_seq=0 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe552ffd640 1 -- 172.21.15.94:0/3747968026 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555958ee3ac0 con 0x555958bee630 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe552ffd640 10 client.22203 handle_command_reply: tid=0 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 client.22203 shutdown 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 msgr2=0x555959009990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fe564005620 tx=0x7fe5640df2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 shutdown_connections 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555958bee630 0x55595907ce60 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555958c5a410 0x55595900a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555958c39730 0x555959009990 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 --2- 172.21.15.94:0/3747968026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555958c31550 0x555959009450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe552ffd640 0 client.22203 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 >> 172.21.15.94:0/3747968026 conn(0x555958c4e560 msgr2=0x555958ff6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 shutdown_connections 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 1 -- 172.21.15.94:0/3747968026 wait complete. 2024-04-25T14:00:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.220+0000 7fe580a85740 20 client.22203 trim_cache size 0 max 16384 2024-04-25T14:00:25.224 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:00:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "7c7e1272-d3af-44e3-8598-a77f0b0f6c33", 2024-04-25T14:00:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:00:25.224 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 msgr2=0x7fe55c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe56c005e30 tx=0x7fe56c031af0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 msgr2=0x7fe5741020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9f0 tx=0x7fe56400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 shutdown_connections 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe55c0afaf0 0x7fe55c0b2000 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5741098a0 0x7fe574102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe574106f40 0x7fe5741020d0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 --2- 172.21.15.94:0/1951143274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5741045e0 0x7fe574103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 >> 172.21.15.94:0/1951143274 conn(0x7fe5740fdc00 msgr2=0x7fe5741050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 shutdown_connections 2024-04-25T14:00:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.221+0000 7fe579bd6640 1 -- 172.21.15.94:0/1951143274 wait complete. 2024-04-25T14:00:25.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:25.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:25.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- 172.21.15.94:0/598468263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c108920 msgr2=0x7fc65c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 --2- 172.21.15.94:0/598468263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c108920 0x7fc65c10ad40 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc65402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- 172.21.15.94:0/598468263 shutdown_connections 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 --2- 172.21.15.94:0/598468263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c108920 0x7fc65c10ad40 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 --2- 172.21.15.94:0/598468263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c103c20 0x7fc65c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 --2- 172.21.15.94:0/598468263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c1029e0 0x7fc65c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- 172.21.15.94:0/598468263 >> 172.21.15.94:0/598468263 conn(0x7fc65c0fdbd0 msgr2=0x7fc65c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- 172.21.15.94:0/598468263 shutdown_connections 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- 172.21.15.94:0/598468263 wait complete. 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 Processor -- start 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.351+0000 7fc662fbe640 1 -- start start 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c1029e0 0x7fc65c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 0x7fc65c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10f610 con 0x7fc65c103c20 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10f490 con 0x7fc65c1029e0 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10f790 con 0x7fc65c108920 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33080/0 (socket says 172.21.15.94:33080) 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 -- 172.21.15.94:0/4100813884 learned_addr learned my addr 172.21.15.94:0/4100813884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc660d5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c1029e0 0x7fc65c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc66155b640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 0x7fc65c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 -- 172.21.15.94:0/4100813884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 msgr2=0x7fc65c1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 0x7fc65c1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 -- 172.21.15.94:0/4100813884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c1029e0 msgr2=0x7fc65c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c1029e0 0x7fc65c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 -- 172.21.15.94:0/4100813884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c1a2040 con 0x7fc65c103c20 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc66155b640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 0x7fc65c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc65bfff640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fc64800e9f0 tx=0x7fc64800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc659ffb640 1 -- 172.21.15.94:0/4100813884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800cd80 con 0x7fc65c103c20 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1a2330 con 0x7fc65c103c20 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc659ffb640 1 -- 172.21.15.94:0/4100813884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6480108e0 con 0x7fc65c103c20 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.352+0000 7fc659ffb640 1 -- 172.21.15.94:0/4100813884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648018980 con 0x7fc65c103c20 2024-04-25T14:00:25.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.353+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a2880 con 0x7fc65c103c20 2024-04-25T14:00:25.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.355+0000 7fc659ffb640 1 -- 172.21.15.94:0/4100813884 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc648010430 con 0x7fc65c103c20 2024-04-25T14:00:25.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.355+0000 7fc669e72740 1 Processor -- start 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 -- start start 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be84390 0x55899c225a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c225fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55899c18ad80 con 0x55899be636b0 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899c1448b0 con 0x55899be5b4d0 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc669e72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55899c186d40 con 0x55899be84390 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc66155b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be84390 0x55899c225a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c225fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33082/0 (socket says 172.21.15.94:33082) 2024-04-25T14:00:25.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc65bfff640 1 -- 172.21.15.94:0/1632379592 learned_addr learned my addr 172.21.15.94:0/1632379592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc659ffb640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 0x7fc64c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 -- 172.21.15.94:0/1632379592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be84390 msgr2=0x55899c225a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be84390 0x55899c225a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 -- 172.21.15.94:0/1632379592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 msgr2=0x55899c226c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc660d5a640 1 -- 172.21.15.94:0/1632379592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899c115a40 con 0x55899be5b4d0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc66155b640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 0x7fc64c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.356+0000 7fc659ffb640 1 -- 172.21.15.94:0/4100813884 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc6480df040 con 0x7fc65c103c20 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.357+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c225fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc65c1035e0 tx=0x7fc6500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5700e8331d169795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.357+0000 7fc66155b640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 0x7fc64c0b2000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55899c226ab0 tx=0x7fc6540095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.357+0000 7fc646ffd640 1 -- 172.21.15.94:0/1632379592 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65001c480 con 0x55899be5b4d0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.357+0000 7fc646ffd640 1 -- 172.21.15.94:0/1632379592 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6500118f0 con 0x55899be5b4d0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.357+0000 7fc646ffd640 1 -- 172.21.15.94:0/1632379592 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65001a7b0 con 0x55899be5b4d0 2024-04-25T14:00:25.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 -- 172.21.15.94:0/1632379592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 msgr2=0x55899c225fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c225fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc65c1035e0 tx=0x7fc6500107a0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 -- 172.21.15.94:0/1632379592 shutdown_connections 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c226c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c225fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 --2- 172.21.15.94:0/1632379592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be84390 0x55899c225a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 -- 172.21.15.94:0/1632379592 >> 172.21.15.94:0/1632379592 conn(0x55899be784e0 msgr2=0x55899c221270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 -- 172.21.15.94:0/1632379592 shutdown_connections 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.358+0000 7fc669e72740 1 -- 172.21.15.94:0/1632379592 wait complete. 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 -- start start 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 Processor -- start 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c22f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be84390 0x55899c236de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55899c227780 con 0x55899be84390 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899c111d10 con 0x55899be5b4d0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55899c227900 con 0x55899be636b0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc669e72740 10 client.0 _ll_register_callbacks cb 0x55899c13e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc660d5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c22f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56390/0 (socket says 172.21.15.94:56390) 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc65bfff640 1 -- 172.21.15.94:0/624790872 learned_addr learned my addr 172.21.15.94:0/624790872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.359+0000 7fc660d5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c22f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56454/0 (socket says 172.21.15.94:56454) 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc66155b640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be84390 0x55899c236de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 -- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 msgr2=0x55899c22f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c22f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 -- 172.21.15.94:0/624790872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be84390 msgr2=0x55899c236de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be84390 0x55899c236de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 -- 172.21.15.94:0/624790872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899bf667d0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc65bfff640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6480a7410 tx=0x7fc648018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e09da6ed9a4ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc63bfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480a7250 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc6480e10d0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6500118f0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc6480a6bf0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc669e72740 10 client.31291 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.360+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55899be9c3f0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6500118f0 con 0x55899be636b0 2024-04-25T14:00:25.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc6480aebb0 con 0x55899be636b0 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 10 client.31291 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 10 client.31291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 20 client.31291 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be42f60 0x55899c23d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 4 client.31291 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899c198990 con 0x55899be42f60 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc660d5a640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be42f60 0x55899c23d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc660d5a640 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be42f60 0x55899c23d3d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b68f7da632e3b7e server_cookie=f3dc945a5db0fdf0 in_seq=0 out_seq=0 2024-04-25T14:00:25.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.361+0000 7fc63bfff640 10 client.31291 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.366+0000 7fc63bfff640 1 -- 172.21.15.94:0/624790872 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55899c198990 con 0x55899be42f60 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.366+0000 7fc63bfff640 10 client.31291 handle_command_reply: tid=0 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.366+0000 7fc669e72740 1 client.31291 shutdown 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.366+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 msgr2=0x55899c22fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.366+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc6480a7410 tx=0x7fc648018420 comp rx=0 tx=0).stop 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 shutdown_connections 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be42f60 0x55899c23d3d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be84390 0x55899c236de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be636b0 0x55899c22fe80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 --2- 172.21.15.94:0/624790872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be5b4d0 0x55899c22f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc63bfff640 0 client.31291 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 >> 172.21.15.94:0/624790872 conn(0x55899be784e0 msgr2=0x55899c224180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 shutdown_connections 2024-04-25T14:00:25.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 1 -- 172.21.15.94:0/624790872 wait complete. 2024-04-25T14:00:25.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.367+0000 7fc669e72740 20 client.31291 trim_cache size 0 max 16384 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 Processor -- start 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 -- start start 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c220c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c236d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1ad710 con 0x55899be636b0 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1af250 con 0x55899be84390 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc669e72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1adde0 con 0x55899be5b4d0 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33100/0 (socket says 172.21.15.94:33100) 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc65bfff640 1 -- 172.21.15.94:0/1091153320 learned_addr learned my addr 172.21.15.94:0/1091153320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.386+0000 7fc66155b640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c236d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c220c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 -- 172.21.15.94:0/1091153320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 msgr2=0x55899c220c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c220c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 -- 172.21.15.94:0/1091153320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 msgr2=0x55899c236d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c236d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 -- 172.21.15.94:0/1091153320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899be5c0f0 con 0x55899be636b0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c220c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fc6480dcaf0 tx=0x7fc6480e2bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e807facf2adc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc6467fc640 1 -- 172.21.15.94:0/1091153320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480a6dd0 con 0x55899be636b0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc6467fc640 1 -- 172.21.15.94:0/1091153320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6480a6f90 con 0x55899be636b0 2024-04-25T14:00:25.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.387+0000 7fc6467fc640 1 -- 172.21.15.94:0/1091153320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648018b20 con 0x55899be636b0 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- 172.21.15.94:0/1091153320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 msgr2=0x55899c221150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fc6480dcaf0 tx=0x7fc6480e2bb0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- 172.21.15.94:0/1091153320 shutdown_connections 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c236d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c221150 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 --2- 172.21.15.94:0/1091153320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c220c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- 172.21.15.94:0/1091153320 >> 172.21.15.94:0/1091153320 conn(0x55899be784e0 msgr2=0x55899c22df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- 172.21.15.94:0/1091153320 shutdown_connections 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- 172.21.15.94:0/1091153320 wait complete. 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 -- start start 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.388+0000 7fc669e72740 1 Processor -- start 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c234830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c235ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1aeb80 con 0x55899be636b0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1ae4b0 con 0x55899be84390 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1ab6f0 con 0x55899be5b4d0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc669e72740 10 client.0 _ll_register_callbacks cb 0x7fc6480e2810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33102/0 (socket says 172.21.15.94:33102) 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 -- 172.21.15.94:0/1806570742 learned_addr learned my addr 172.21.15.94:0/1806570742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c234830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc66155b640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c235ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 -- 172.21.15.94:0/1806570742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 msgr2=0x55899c234830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c234830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 -- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 msgr2=0x55899c235ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:25.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c235ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.389+0000 7fc65bfff640 1 -- 172.21.15.94:0/1806570742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899c14b4b0 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc65bfff640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fc6480e2c40 tx=0x7fc6480dcb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade18d4fae854707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480c7060 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc648018b20 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6480a6f90 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc6480a6dd0 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 10 client.22221 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55899c237320 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6480a6f90 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc6480a6150 con 0x55899be636b0 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 10 client.22221 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 10 client.22221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:25.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 20 client.22221 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be183a0 0x55899c2a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 4 client.22221 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.390+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55899c192800 con 0x55899be183a0 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.391+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be183a0 0x55899c2a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.391+0000 7fc63bfff640 10 client.22221 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.391+0000 7fc660d5a640 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be183a0 0x55899c2a7750 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34768b6bd3c5ced server_cookie=ec5fdf9a632676c2 in_seq=0 out_seq=0 2024-04-25T14:00:25.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc63bfff640 1 -- 172.21.15.94:0/1806570742 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55899c192800 con 0x55899be183a0 2024-04-25T14:00:25.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc63bfff640 10 client.22221 handle_command_reply: tid=0 2024-04-25T14:00:25.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 client.22221 shutdown 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 msgr2=0x55899c234d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fc6480e2c40 tx=0x7fc6480dcb20 comp rx=0 tx=0).stop 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 shutdown_connections 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55899be183a0 0x55899c2a7750 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899be84390 0x55899c235ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55899be636b0 0x55899c234d70 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 --2- 172.21.15.94:0/1806570742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55899be5b4d0 0x55899c234830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 >> 172.21.15.94:0/1806570742 conn(0x55899be784e0 msgr2=0x55899c220c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc63bfff640 0 client.22221 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 shutdown_connections 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 1 -- 172.21.15.94:0/1806570742 wait complete. 2024-04-25T14:00:25.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.394+0000 7fc669e72740 20 client.22221 trim_cache size 0 max 16384 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (291 inodes in the stack)", 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "7c7e1272-d3af-44e3-8598-a77f0b0f6c33": { 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "7c7e1272-d3af-44e3-8598-a77f0b0f6c33", 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:00:25.400 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:25.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 msgr2=0x7fc64c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 0x7fc64c0b2000 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55899c226ab0 tx=0x7fc6540095c0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 msgr2=0x7fc65c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:25.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fc64800e9f0 tx=0x7fc64800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 shutdown_connections 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc64c0afaf0 0x7fc64c0b2000 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c108920 0x7fc65c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c103c20 0x7fc65c19aa40 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 --2- 172.21.15.94:0/4100813884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c1029e0 0x7fc65c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 >> 172.21.15.94:0/4100813884 conn(0x7fc65c0fdbd0 msgr2=0x7fc65c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 shutdown_connections 2024-04-25T14:00:25.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:25.395+0000 7fc662fbe640 1 -- 172.21.15.94:0/4100813884 wait complete. 2024-04-25T14:00:25.407 INFO:tasks.cephfs.filesystem:scrub status for tag:7c7e1272-d3af-44e3-8598-a77f0b0f6c33 - {'path': '/', 'tag': '7c7e1272-d3af-44e3-8598-a77f0b0f6c33', 'options': 'recursive,force'} 2024-04-25T14:00:26.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 111692 19.26 MB/sec execute 188 sec latency 62.860 ms 2024-04-25T14:00:26.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: pgmap v910: 97 pgs: 97 active+clean; 256 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:25 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: pgmap v910: 97 pgs: 97 active+clean; 256 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:25 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: pgmap v910: 97 pgs: 97 active+clean; 256 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:26.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:25 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:27.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 111873 19.19 MB/sec execute 189 sec latency 163.029 ms 2024-04-25T14:00:27.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:26 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:27.251 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:26 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:27.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:26 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:28.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 112222 19.20 MB/sec execute 190 sec latency 132.493 ms 2024-04-25T14:00:28.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:27 smithi094 ceph-mon[29195]: pgmap v911: 97 pgs: 97 active+clean; 251 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-04-25T14:00:28.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:27 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:28.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:27 smithi138 ceph-mon[32977]: pgmap v911: 97 pgs: 97 active+clean; 251 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-04-25T14:00:28.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:27 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:27 smithi137 ceph-mon[32568]: pgmap v911: 97 pgs: 97 active+clean; 251 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 504 op/s 2024-04-25T14:00:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:27 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:29.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 112600 19.21 MB/sec execute 191 sec latency 53.858 ms 2024-04-25T14:00:29.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:28 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:29.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:28 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:29.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:28 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:00:29.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:28 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:29.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:28 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:29.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:28 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:00:29.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:28 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:29.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:28 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:29.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:28 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:00:30.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 113063 19.19 MB/sec execute 192 sec latency 83.441 ms 2024-04-25T14:00:30.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:29 smithi094 ceph-mon[29195]: pgmap v912: 97 pgs: 97 active+clean; 250 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-04-25T14:00:30.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:29 smithi138 ceph-mon[32977]: pgmap v912: 97 pgs: 97 active+clean; 250 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-04-25T14:00:30.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:29 smithi137 ceph-mon[32568]: pgmap v912: 97 pgs: 97 active+clean; 250 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-04-25T14:00:31.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 113403 19.18 MB/sec execute 193 sec latency 74.253 ms 2024-04-25T14:00:31.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:30 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:31.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:30 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:31.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:30 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:31.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:30 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:31.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:30 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:31.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:30 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:32.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 113749 19.23 MB/sec execute 194 sec latency 134.503 ms 2024-04-25T14:00:32.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:31 smithi094 ceph-mon[29195]: pgmap v913: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-04-25T14:00:32.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:32.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:32.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:00:32.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:31 smithi138 ceph-mon[32977]: pgmap v913: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-04-25T14:00:32.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:32.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:32.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:00:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:31 smithi137 ceph-mon[32568]: pgmap v913: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-04-25T14:00:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:00:33.145 INFO:tasks.workunit.client.0.smithi094.stdout: 10 114063 19.21 MB/sec execute 195 sec latency 266.110 ms 2024-04-25T14:00:34.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 114464 19.24 MB/sec execute 196 sec latency 64.664 ms 2024-04-25T14:00:34.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:33 smithi094 ceph-mon[29195]: pgmap v914: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-04-25T14:00:34.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:33 smithi138 ceph-mon[32977]: pgmap v914: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-04-25T14:00:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:33 smithi137 ceph-mon[32568]: pgmap v914: 97 pgs: 97 active+clean; 257 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-04-25T14:00:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 -- 172.21.15.94:0/1651785266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c108980 msgr2=0x7f5d6c10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 --2- 172.21.15.94:0/1651785266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c108980 0x7f5d6c10ada0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009ca0 tx=0x7f5d6402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 -- 172.21.15.94:0/1651785266 shutdown_connections 2024-04-25T14:00:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 --2- 172.21.15.94:0/1651785266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c108980 0x7f5d6c10ada0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 --2- 172.21.15.94:0/1651785266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 0x7f5d6c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 --2- 172.21.15.94:0/1651785266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c102a40 0x7f5d6c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 -- 172.21.15.94:0/1651785266 >> 172.21.15.94:0/1651785266 conn(0x7f5d6c0fdc30 msgr2=0x7f5d6c1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 -- 172.21.15.94:0/1651785266 shutdown_connections 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.859+0000 7f5d70d84640 1 -- 172.21.15.94:0/1651785266 wait complete. 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 Processor -- start 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 -- start start 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 0x7f5d6c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c108980 0x7f5d6c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10e570 con 0x7f5d6c102a40 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10e3f0 con 0x7f5d6c108980 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d70d84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10e6f0 con 0x7f5d6c103c80 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40600/0 (socket says 172.21.15.94:40600) 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3343386035 learned_addr learned my addr 172.21.15.94:0/3343386035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c108980 0x7f5d6c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.860+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 0x7f5d6c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3343386035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 msgr2=0x7f5d6c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 0x7f5d6c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3343386035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c108980 msgr2=0x7f5d6c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c108980 0x7f5d6c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3343386035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c1a1d40 con 0x7f5d6c102a40 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005fd0 tx=0x7f5d5c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/3343386035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c017070 con 0x7f5d6c102a40 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/3343386035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d5c009c30 con 0x7f5d6c102a40 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c1a1fd0 con 0x7f5d6c102a40 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/3343386035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c003d10 con 0x7f5d6c102a40 2024-04-25T14:00:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.861+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c1a24e0 con 0x7f5d6c102a40 2024-04-25T14:00:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/3343386035 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5d5c003eb0 con 0x7f5d6c102a40 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 Processor -- start 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 -- start start 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a000bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0013200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b59ff6d460 con 0x55b59fc3e2f0 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b59ff27590 con 0x55b59fc671b0 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b59ff69420 con 0x55b59fc464d0 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40614/0 (socket says 172.21.15.94:40614) 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 -- 172.21.15.94:0/169454216 learned_addr learned my addr 172.21.15.94:0/169454216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a000bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0013200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d5b7fe640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 0x7f5d540b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 -- 172.21.15.94:0/169454216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 msgr2=0x55b5a0013200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0013200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 -- 172.21.15.94:0/169454216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 msgr2=0x55b5a000bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.864+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a000bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d6a59c640 1 -- 172.21.15.94:0/169454216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59ff340d0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a000bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 0x7f5d540b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d5b7fe640 1 -- 172.21.15.94:0/3343386035 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5d5c0de890 con 0x7f5d6c102a40 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c092520 tx=0x7f5d5c06d1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1818f8bbbe82405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d58ff9640 1 -- 172.21.15.94:0/169454216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0a6620 con 0x55b59fc3e2f0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 0x7f5d540b2070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55b5a000cf50 tx=0x7f5d640095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d58ff9640 1 -- 172.21.15.94:0/169454216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d5c092580 con 0x55b59fc3e2f0 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.865+0000 7f5d58ff9640 1 -- 172.21.15.94:0/169454216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0a6440 con 0x55b59fc3e2f0 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- 172.21.15.94:0/169454216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 msgr2=0x55b5a000c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c092520 tx=0x7f5d5c06d1f0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- 172.21.15.94:0/169454216 shutdown_connections 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0013200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a000c300 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/169454216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a000bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- 172.21.15.94:0/169454216 >> 172.21.15.94:0/169454216 conn(0x55b59fc5b300 msgr2=0x55b5a0003300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- 172.21.15.94:0/169454216 shutdown_connections 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- 172.21.15.94:0/169454216 wait complete. 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 -- start start 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.866+0000 7f5d77c3b740 1 Processor -- start 2024-04-25T14:00:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0015ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a001ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0013bf0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b59fef4b60 con 0x55b59fc671b0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0013d70 con 0x55b59fc464d0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 10 client.0 _ll_register_callbacks cb 0x55b59ff192b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40628/0 (socket says 172.21.15.94:40628) 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3554337922 learned_addr learned my addr 172.21.15.94:0/3554337922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a001ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0015ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3554337922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 msgr2=0x55b5a0015ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0015ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 msgr2=0x55b5a001ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a001ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3554337922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59fc7f210 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0d6e00 tx=0x7f5d5c0a67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f11c96b3536ad82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c08f040 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5d5c0a26c0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.867+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b59ff7b070 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c0c9040 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 10 client.22233 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b59fd4b620 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b59ff7b070 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5d5c0a2f60 con 0x55b59fc3e2f0 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 10 client.22233 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 10 client.22233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 20 client.22233 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fc25d80 0x55b5a0023280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 4 client.22233 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59ff74ee0 con 0x55b59fc25d80 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.868+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fc25d80 0x55b5a0023280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.869+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fc25d80 0x55b5a0023280 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d986f63cf107d440 server_cookie=557f2699c78354d0 in_seq=0 out_seq=0 2024-04-25T14:00:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.869+0000 7f5d51ffb640 10 client.22233 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d51ffb640 1 -- 172.21.15.94:0/3554337922 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b59ff74ee0 con 0x55b59fc25d80 2024-04-25T14:00:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d51ffb640 10 client.22233 handle_command_reply: tid=0 2024-04-25T14:00:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 client.22233 shutdown 2024-04-25T14:00:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 msgr2=0x55b5a0015990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0d6e00 tx=0x7f5d5c0a67e0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 shutdown_connections 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fc25d80 0x55b5a0023280 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a001ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a0015ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3554337922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0015990 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 >> 172.21.15.94:0/3554337922 conn(0x55b59fc5b300 msgr2=0x55b5a0006210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d51ffb640 0 client.22233 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 shutdown_connections 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3554337922 wait complete. 2024-04-25T14:00:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.873+0000 7f5d77c3b740 20 client.22233 trim_cache size 0 max 16384 2024-04-25T14:00:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 Processor -- start 2024-04-25T14:00:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 -- start start 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a0006100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a93a0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aaee0 con 0x55b59fc671b0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.889+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a9a70 con 0x55b59fc464d0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40632/0 (socket says 172.21.15.94:40632) 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3025278441 learned_addr learned my addr 172.21.15.94:0/3025278441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a0006100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3025278441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 msgr2=0x55b5a001cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3025278441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 msgr2=0x55b5a0006100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a0006100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 -- 172.21.15.94:0/3025278441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a00108a0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a0006100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0a2d50 tx=0x7f5d5c0a6850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ad1f8b9148f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d53fff640 1 -- 172.21.15.94:0/3025278441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c08f040 con 0x55b59fc3e2f0 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d53fff640 1 -- 172.21.15.94:0/3025278441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d5c0a73d0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.890+0000 7f5d53fff640 1 -- 172.21.15.94:0/3025278441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c092180 con 0x55b59fc3e2f0 2024-04-25T14:00:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3025278441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 msgr2=0x55b5a00032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0a2d50 tx=0x7f5d5c0a6850 comp rx=0 tx=0).stop 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3025278441 shutdown_connections 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a0006100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/3025278441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a00032e0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3025278441 >> 172.21.15.94:0/3025278441 conn(0x55b59fc5b300 msgr2=0x55b5a0002bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3025278441 shutdown_connections 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- 172.21.15.94:0/3025278441 wait complete. 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 -- start start 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.891+0000 7f5d77c3b740 1 Processor -- start 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a00895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aa810 con 0x55b59fc3e2f0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aa140 con 0x55b59fc671b0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a7380 con 0x55b59fc464d0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 10 client.0 _ll_register_callbacks cb 0x7f5d64009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40640/0 (socket says 172.21.15.94:40640) 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 -- 172.21.15.94:0/4058504385 learned_addr learned my addr 172.21.15.94:0/4058504385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6ad9d640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a00895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 -- 172.21.15.94:0/4058504385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 msgr2=0x55b5a001a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 -- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 msgr2=0x55b5a00895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a00895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 -- 172.21.15.94:0/4058504385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a0089b10 con 0x55b59fc3e2f0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d6a59c640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c000c00 tx=0x7f5d5c0a67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3033cb7f701bc619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c08f040 con 0x55b59fc3e2f0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5d5c0af8a0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c0a73d0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.892+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5d5c0a6620 con 0x55b59fc3e2f0 2024-04-25T14:00:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 10 client.22239 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b5a0089b10 con 0x55b59fc3e2f0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d5c0a73d0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5d5c0041a0 con 0x55b59fc3e2f0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 10 client.22239 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 10 client.22239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 20 client.22239 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fbfb1c0 0x55b5a008d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 4 client.22239 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b59fef05f0 con 0x55b59fbfb1c0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fbfb1c0 0x55b5a008d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d69d9b640 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fbfb1c0 0x55b5a008d430 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87bf42481c6ae99 server_cookie=80d6966a0d020857 in_seq=0 out_seq=0 2024-04-25T14:00:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:34.893+0000 7f5d51ffb640 10 client.22239 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:35.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 114746 19.23 MB/sec execute 197 sec latency 201.686 ms 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d51ffb640 1 -- 172.21.15.94:0/4058504385 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b59fef05f0 con 0x55b59fbfb1c0 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d51ffb640 10 client.22239 handle_command_reply: tid=0 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 client.22239 shutdown 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 msgr2=0x55b5a0019de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c000c00 tx=0x7f5d5c0a67e0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 shutdown_connections 2024-04-25T14:00:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b59fbfb1c0 0x55b5a008d430 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b59fc671b0 0x55b5a00895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b59fc464d0 0x55b5a001a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d51ffb640 0 client.22239 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 --2- 172.21.15.94:0/4058504385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b59fc3e2f0 0x55b5a0019de0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 >> 172.21.15.94:0/4058504385 conn(0x55b59fc5b300 msgr2=0x55b5a00153d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 shutdown_connections 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 1 -- 172.21.15.94:0/4058504385 wait complete. 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.463+0000 7f5d77c3b740 20 client.22239 trim_cache size 0 max 16384 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:00:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:00:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:00:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 msgr2=0x7f5d540b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 0x7f5d540b2070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55b5a000cf50 tx=0x7f5d640095c0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 msgr2=0x7f5d6c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c005fd0 tx=0x7f5d5c00dde0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 shutdown_connections 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5d540afb60 0x7f5d540b2070 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d6c108980 0x7f5d6c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d6c103c80 0x7f5d6c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 --2- 172.21.15.94:0/3343386035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d6c102a40 0x7f5d6c06bce0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 >> 172.21.15.94:0/3343386035 conn(0x7f5d6c0fdc30 msgr2=0x7f5d6c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 shutdown_connections 2024-04-25T14:00:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:35.464+0000 7f5d70d84640 1 -- 172.21.15.94:0/3343386035 wait complete. 2024-04-25T14:00:36.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 115033 19.19 MB/sec execute 198 sec latency 363.976 ms 2024-04-25T14:00:36.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:35 smithi094 ceph-mon[29195]: pgmap v915: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-04-25T14:00:36.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:35 smithi138 ceph-mon[32977]: pgmap v915: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-04-25T14:00:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:35 smithi137 ceph-mon[32568]: pgmap v915: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.4 MiB/s wr, 653 op/s 2024-04-25T14:00:37.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 115298 19.17 MB/sec execute 199 sec latency 110.449 ms 2024-04-25T14:00:38.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 115514 19.12 MB/sec execute 200 sec latency 238.214 ms 2024-04-25T14:00:38.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:37 smithi094 ceph-mon[29195]: pgmap v916: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 436 op/s 2024-04-25T14:00:38.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:37 smithi138 ceph-mon[32977]: pgmap v916: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 436 op/s 2024-04-25T14:00:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:37 smithi137 ceph-mon[32568]: pgmap v916: 97 pgs: 97 active+clean; 263 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 436 op/s 2024-04-25T14:00:39.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 115936 19.15 MB/sec execute 201 sec latency 77.536 ms 2024-04-25T14:00:40.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 116283 19.15 MB/sec execute 202 sec latency 104.933 ms 2024-04-25T14:00:40.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:39 smithi138 ceph-mon[32977]: pgmap v917: 97 pgs: 97 active+clean; 264 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-04-25T14:00:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:39 smithi137 ceph-mon[32568]: pgmap v917: 97 pgs: 97 active+clean; 264 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-04-25T14:00:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:39 smithi094 ceph-mon[29195]: pgmap v917: 97 pgs: 97 active+clean; 264 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-04-25T14:00:41.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 116792 19.13 MB/sec execute 203 sec latency 50.354 ms 2024-04-25T14:00:42.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 117195 19.20 MB/sec execute 204 sec latency 93.941 ms 2024-04-25T14:00:42.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:41 smithi138 ceph-mon[32977]: pgmap v918: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 638 op/s 2024-04-25T14:00:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:41 smithi137 ceph-mon[32568]: pgmap v918: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 638 op/s 2024-04-25T14:00:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:41 smithi094 ceph-mon[29195]: pgmap v918: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.3 MiB/s wr, 638 op/s 2024-04-25T14:00:43.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 117516 19.17 MB/sec execute 205 sec latency 159.642 ms 2024-04-25T14:00:44.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 117932 19.21 MB/sec execute 206 sec latency 79.359 ms 2024-04-25T14:00:44.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:43 smithi138 ceph-mon[32977]: pgmap v919: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 482 op/s 2024-04-25T14:00:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:43 smithi137 ceph-mon[32568]: pgmap v919: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 482 op/s 2024-04-25T14:00:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:43 smithi094 ceph-mon[29195]: pgmap v919: 97 pgs: 97 active+clean; 253 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 482 op/s 2024-04-25T14:00:45.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 118243 19.22 MB/sec execute 207 sec latency 100.344 ms 2024-04-25T14:00:46.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 118639 19.21 MB/sec execute 208 sec latency 45.625 ms 2024-04-25T14:00:46.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:45 smithi138 ceph-mon[32977]: pgmap v920: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-04-25T14:00:46.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:45 smithi137 ceph-mon[32568]: pgmap v920: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-04-25T14:00:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:45 smithi094 ceph-mon[29195]: pgmap v920: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 600 op/s 2024-04-25T14:00:47.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 118896 19.17 MB/sec execute 209 sec latency 189.959 ms 2024-04-25T14:00:48.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 119180 19.15 MB/sec execute 210 sec latency 282.854 ms 2024-04-25T14:00:48.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:47 smithi138 ceph-mon[32977]: pgmap v921: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-04-25T14:00:48.275 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:47 smithi094 ceph-mon[29195]: pgmap v921: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-04-25T14:00:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:47 smithi137 ceph-mon[32568]: pgmap v921: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/1589819204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 msgr2=0x7f8cd4109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/1589819204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4109390 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009900 tx=0x7f8cc402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/1589819204 shutdown_connections 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/1589819204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd41098d0 0x7f8cd410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/1589819204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4109390 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/1589819204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd4104610 0x7f8cd4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/1589819204 >> 172.21.15.94:0/1589819204 conn(0x7f8cd40fdc10 msgr2=0x7f8cd41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/1589819204 shutdown_connections 2024-04-25T14:00:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.563+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/1589819204 wait complete. 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 Processor -- start 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 -- start start 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd4104610 0x7f8cd4103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd41098d0 0x7f8cd4102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410f630 con 0x7f8cd4106f70 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410f4b0 con 0x7f8cd41098d0 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cdaf1f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410f7b0 con 0x7f8cd4104610 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40446/0 (socket says 172.21.15.94:40446) 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 -- 172.21.15.94:0/2297428170 learned_addr learned my addr 172.21.15.94:0/2297428170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd4104610 0x7f8cd4103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd41098d0 0x7f8cd4102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 -- 172.21.15.94:0/2297428170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd4104610 msgr2=0x7f8cd4103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd4104610 0x7f8cd4103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 -- 172.21.15.94:0/2297428170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd41098d0 msgr2=0x7f8cd4102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd41098d0 0x7f8cd4102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.564+0000 7f8ccbfff640 1 -- 172.21.15.94:0/2297428170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd4102d70 con 0x7f8cd4106f70 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006010 tx=0x7f8cc4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8cc9ffb640 1 -- 172.21.15.94:0/2297428170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc403f030 con 0x7f8cd4106f70 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd41a4320 con 0x7f8cd4106f70 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8cc9ffb640 1 -- 172.21.15.94:0/2297428170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4003aa0 con 0x7f8cd4106f70 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8cc9ffb640 1 -- 172.21.15.94:0/2297428170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4035070 con 0x7f8cd4106f70 2024-04-25T14:00:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.565+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a4780 con 0x7f8cd4106f70 2024-04-25T14:00:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.567+0000 7f8cc9ffb640 1 -- 172.21.15.94:0/2297428170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8cc4042030 con 0x7f8cd4106f70 2024-04-25T14:00:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 Processor -- start 2024-04-25T14:00:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 -- start start 2024-04-25T14:00:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 0x55ff133c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ff13322460 con 0x55ff1301c1b0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff132dc590 con 0x55ff12ffb4d0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ff1331e420 con 0x55ff12ff32f0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cc9ffb640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 0x7f8cbc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40456/0 (socket says 172.21.15.94:40456) 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2190180850 learned_addr learned my addr 172.21.15.94:0/2190180850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd8cbb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 0x55ff133c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 0x7f8cbc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cc9ffb640 1 -- 172.21.15.94:0/2297428170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8cc40ff230 con 0x7f8cd4106f70 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2190180850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 msgr2=0x55ff133bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.568+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133bcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2190180850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 msgr2=0x55ff133c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 0x55ff133c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2190180850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff132e90d0 con 0x55ff1301c1b0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 0x55ff133c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc012d60 tx=0x7f8ccc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343261e9deabc47b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 0x7f8cbc0b1ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009fd0 tx=0x7f8ccc017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cbaffd640 1 -- 172.21.15.94:0/2190180850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc023070 con 0x55ff1301c1b0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cbaffd640 1 -- 172.21.15.94:0/2190180850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc0125a0 con 0x55ff1301c1b0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.569+0000 7f8cbaffd640 1 -- 172.21.15.94:0/2190180850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc003bd0 con 0x55ff1301c1b0 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2190180850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 msgr2=0x55ff133bc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc012d60 tx=0x7f8ccc010740 comp rx=0 tx=0).stop 2024-04-25T14:00:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2190180850 shutdown_connections 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ffb4d0 0x55ff133c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2190180850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff1301c1b0 0x55ff133bc970 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2190180850 >> 172.21.15.94:0/2190180850 conn(0x55ff13010300 msgr2=0x55ff133b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2190180850 shutdown_connections 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2190180850 wait complete. 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 -- start start 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.570+0000 7f8ce1dd3740 1 Processor -- start 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ffb4d0 0x55ff133c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1301c1b0 0x55ff133cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ff133c47e0 con 0x55ff12ff32f0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff132a9b60 con 0x55ff1301c1b0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ff133c4960 con 0x55ff12ffb4d0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 10 client.0 _ll_register_callbacks cb 0x55ff132ce2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40460/0 (socket says 172.21.15.94:40460) 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 -- 172.21.15.94:0/1112360450 learned_addr learned my addr 172.21.15.94:0/1112360450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ffb4d0 0x55ff133c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1301c1b0 0x55ff133cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 -- 172.21.15.94:0/1112360450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ffb4d0 msgr2=0x55ff133c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ffb4d0 0x55ff133c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 -- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1301c1b0 msgr2=0x55ff133cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1301c1b0 0x55ff133cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 -- 172.21.15.94:0/1112360450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff13034210 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0012700 tx=0x7f8cd0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69020462e2d5a193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000f920 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8cd000fac0 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.571+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ccc0125a0 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8cd000fcc0 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 10 client.22251 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ff13100620 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc0125a0 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8cd001c420 con 0x55ff12ff32f0 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 10 client.22251 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 10 client.22251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 20 client.22251 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff12fdad80 0x55ff133d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 4 client.22251 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff13330070 con 0x55ff12fdad80 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.572+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff12fdad80 0x55ff133d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.573+0000 7f8cb3fff640 10 client.22251 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.573+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff12fdad80 0x55ff133d44e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2145b1724b6d1f server_cookie=5c9766673d2bf5fe in_seq=0 out_seq=0 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.574+0000 7f8cb3fff640 1 -- 172.21.15.94:0/1112360450 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff13330070 con 0x55ff12fdad80 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.574+0000 7f8cb3fff640 10 client.22251 handle_command_reply: tid=0 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.574+0000 7f8ce1dd3740 1 client.22251 shutdown 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 msgr2=0x55ff133c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0012700 tx=0x7f8cd0012bc0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 shutdown_connections 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff12fdad80 0x55ff133d44e0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff1301c1b0 0x55ff133cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8cb3fff640 0 client.22251 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ffb4d0 0x55ff133c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/1112360450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ff32f0 0x55ff133c66a0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 >> 172.21.15.94:0/1112360450 conn(0x55ff13010300 msgr2=0x55ff133bb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 shutdown_connections 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/1112360450 wait complete. 2024-04-25T14:00:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.575+0000 7f8ce1dd3740 20 client.22251 trim_cache size 0 max 16384 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 Processor -- start 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 -- start start 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12fdad80 0x55ff133b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ff32f0 0x55ff133cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41ad6d0 con 0x55ff12ffb4d0 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41af210 con 0x55ff12ff32f0 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41adda0 con 0x55ff12fdad80 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40464/0 (socket says 172.21.15.94:40464) 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 -- 172.21.15.94:0/3046880400 learned_addr learned my addr 172.21.15.94:0/3046880400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8ccbfff640 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ff32f0 0x55ff133cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12fdad80 0x55ff133b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 -- 172.21.15.94:0/3046880400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12fdad80 msgr2=0x55ff133b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12fdad80 0x55ff133b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 -- 172.21.15.94:0/3046880400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ff32f0 msgr2=0x55ff133cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ff32f0 0x55ff133cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.591+0000 7f8cd94bc640 1 -- 172.21.15.94:0/3046880400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff133c1520 con 0x55ff12ffb4d0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.592+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc010c40 tx=0x7f8ccc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1442be1802bc64b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.592+0000 7f8cba7fc640 1 -- 172.21.15.94:0/3046880400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc023040 con 0x55ff12ffb4d0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.592+0000 7f8cba7fc640 1 -- 172.21.15.94:0/3046880400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc009d30 con 0x55ff12ffb4d0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.592+0000 7f8cba7fc640 1 -- 172.21.15.94:0/3046880400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc005500 con 0x55ff12ffb4d0 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/3046880400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 msgr2=0x55ff133bb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc010c40 tx=0x7f8ccc005bd0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/3046880400 shutdown_connections 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133bb100 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12ff32f0 0x55ff133cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/3046880400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12fdad80 0x55ff133b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/3046880400 >> 172.21.15.94:0/3046880400 conn(0x55ff13010300 msgr2=0x55ff133b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/3046880400 shutdown_connections 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/3046880400 wait complete. 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- start start 2024-04-25T14:00:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 Processor -- start 2024-04-25T14:00:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12fdad80 0x55ff133cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.593+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41aeb40 con 0x55ff12ffb4d0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41ae470 con 0x55ff12fdad80 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd41ab6b0 con 0x55ff12ff32f0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 10 client.0 _ll_register_callbacks cb 0x7f8cd0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40480/0 (socket says 172.21.15.94:40480) 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2286889170 learned_addr learned my addr 172.21.15.94:0/2286889170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12fdad80 0x55ff133cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2286889170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 msgr2=0x55ff133cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12fdad80 msgr2=0x55ff133cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12fdad80 0x55ff133cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 -- 172.21.15.94:0/2286889170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff133cdfa0 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cd94bc640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc003dc0 tx=0x7f8ccc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5446ba0e8ef7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc023040 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8ccc005250 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ccc009d30 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc040050 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 10 client.22257 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.594+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ff133cdcc0 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc009d30 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8ccc01f940 con 0x55ff12ffb4d0 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 10 client.22257 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 10 client.22257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 20 client.22257 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff1301c1b0 0x55ff1343ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 4 client.22257 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff13329ee0 con 0x55ff1301c1b0 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff1301c1b0 0x55ff1343ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8cb3fff640 10 client.22257 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:48.595+0000 7f8cd8cbb640 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff1301c1b0 0x55ff1343ef50 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6703249c62a61b00 server_cookie=109d512e4e0495a3 in_seq=0 out_seq=0 2024-04-25T14:00:49.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 119477 19.15 MB/sec execute 211 sec latency 343.077 ms 2024-04-25T14:00:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.272+0000 7f8cb3fff640 1 -- 172.21.15.94:0/2286889170 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff13329ee0 con 0x55ff1301c1b0 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.272+0000 7f8cb3fff640 10 client.22257 handle_command_reply: tid=0 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.272+0000 7f8ce1dd3740 1 client.22257 shutdown 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 msgr2=0x55ff133cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc003dc0 tx=0x7f8ccc010740 comp rx=0 tx=0).stop 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 shutdown_connections 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ff1301c1b0 0x55ff1343ef50 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8cb3fff640 0 client.22257 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ff12ffb4d0 0x55ff133cc640 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ff12ff32f0 0x55ff133cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 --2- 172.21.15.94:0/2286889170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff12fdad80 0x55ff133cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 >> 172.21.15.94:0/2286889170 conn(0x55ff13010300 msgr2=0x55ff133c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 shutdown_connections 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 1 -- 172.21.15.94:0/2286889170 wait complete. 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.273+0000 7f8ce1dd3740 20 client.22257 trim_cache size 0 max 16384 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:00:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:00:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 msgr2=0x7f8cbc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 0x7f8cbc0b1ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009fd0 tx=0x7f8ccc017000 comp rx=0 tx=0).stop 2024-04-25T14:00:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 msgr2=0x7f8cd4102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006010 tx=0x7f8cc4031a50 comp rx=0 tx=0).stop 2024-04-25T14:00:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 shutdown_connections 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8cbc0afae0 0x7f8cbc0b1ff0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd41098d0 0x7f8cd4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cd4106f70 0x7f8cd4102100 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 --2- 172.21.15.94:0/2297428170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cd4104610 0x7f8cd4103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 >> 172.21.15.94:0/2297428170 conn(0x7f8cd40fdc10 msgr2=0x7f8cd41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 shutdown_connections 2024-04-25T14:00:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:00:49.274+0000 7f8cdaf1f640 1 -- 172.21.15.94:0/2297428170 wait complete. 2024-04-25T14:00:50.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 119731 19.12 MB/sec execute 212 sec latency 501.852 ms 2024-04-25T14:00:50.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:49 smithi138 ceph-mon[32977]: pgmap v922: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-04-25T14:00:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:49 smithi137 ceph-mon[32568]: pgmap v922: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-04-25T14:00:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:49 smithi094 ceph-mon[29195]: pgmap v922: 97 pgs: 97 active+clean; 262 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 457 op/s 2024-04-25T14:00:51.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 120283 19.11 MB/sec execute 213 sec latency 52.166 ms 2024-04-25T14:00:52.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 120735 19.19 MB/sec execute 214 sec latency 65.059 ms 2024-04-25T14:00:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:51 smithi138 ceph-mon[32977]: pgmap v923: 97 pgs: 97 active+clean; 267 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-04-25T14:00:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:51 smithi137 ceph-mon[32568]: pgmap v923: 97 pgs: 97 active+clean; 267 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-04-25T14:00:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:51 smithi094 ceph-mon[29195]: pgmap v923: 97 pgs: 97 active+clean; 267 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-04-25T14:00:53.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 121010 19.15 MB/sec execute 215 sec latency 302.041 ms 2024-04-25T14:00:54.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 121465 19.22 MB/sec execute 216 sec latency 85.252 ms 2024-04-25T14:00:54.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:53 smithi138 ceph-mon[32977]: pgmap v924: 97 pgs: 97 active+clean; 266 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-04-25T14:00:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:53 smithi137 ceph-mon[32568]: pgmap v924: 97 pgs: 97 active+clean; 266 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-04-25T14:00:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:53 smithi094 ceph-mon[29195]: pgmap v924: 97 pgs: 97 active+clean; 266 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-04-25T14:00:55.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 121779 19.22 MB/sec execute 217 sec latency 70.585 ms 2024-04-25T14:00:55.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:55.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 -- 172.21.15.94:0/701746880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84102830 msgr2=0x7f0f84103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 --2- 172.21.15.94:0/701746880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84102830 0x7f0f84103530 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8002a410 comp rx=0 tx=0).stop 2024-04-25T14:00:55.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 -- 172.21.15.94:0/701746880 shutdown_connections 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 --2- 172.21.15.94:0/701746880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84108770 0x7f0f8410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 --2- 172.21.15.94:0/701746880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84103a70 0x7f0f84103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 --2- 172.21.15.94:0/701746880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84102830 0x7f0f84103530 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 -- 172.21.15.94:0/701746880 >> 172.21.15.94:0/701746880 conn(0x7f0f840fda20 msgr2=0x7f0f840ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 -- 172.21.15.94:0/701746880 shutdown_connections 2024-04-25T14:00:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.524+0000 7f0f8b030640 1 -- 172.21.15.94:0/701746880 wait complete. 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 Processor -- start 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 -- start start 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 0x7f0f8419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103a70 0x7f0f8419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 0x7f0f841a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410f460 con 0x7f0f84108770 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410f2e0 con 0x7f0f84103a70 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8b030640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410f5e0 con 0x7f0f84102830 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8a02e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 0x7f0f8419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8a82f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 0x7f0f841a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8a02e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 0x7f0f8419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46746/0 (socket says 172.21.15.94:46746) 2024-04-25T14:00:55.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8a02e640 1 -- 172.21.15.94:0/1778390021 learned_addr learned my addr 172.21.15.94:0/1778390021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.525+0000 7f0f8982d640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103a70 0x7f0f8419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 -- 172.21.15.94:0/1778390021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 msgr2=0x7f0f8419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 0x7f0f8419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 -- 172.21.15.94:0/1778390021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103a70 msgr2=0x7f0f8419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103a70 0x7f0f8419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 -- 172.21.15.94:0/1778390021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a1bb0 con 0x7f0f84108770 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8a82f640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 0x7f0f841a1540 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009fd0 tx=0x7f0f7c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f7b7fe640 1 -- 172.21.15.94:0/1778390021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00ce20 con 0x7f0f84108770 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f7b7fe640 1 -- 172.21.15.94:0/1778390021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c005020 con 0x7f0f84108770 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a1ea0 con 0x7f0f84108770 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f7b7fe640 1 -- 172.21.15.94:0/1778390021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c010690 con 0x7f0f84108770 2024-04-25T14:00:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.526+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a22b0 con 0x7f0f84108770 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.528+0000 7f0f7b7fe640 1 -- 172.21.15.94:0/1778390021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0f7c0054d0 con 0x7f0f84108770 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 Processor -- start 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 -- start start 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed34390 0x559a4f0d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a4f03ad80 con 0x559a4ed136b0 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a4eff48b0 con 0x559a4ed0b4d0 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f93216740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a4f036d40 con 0x559a4ed34390 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38538/0 (socket says 172.21.15.94:38538) 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8982d640 1 -- 172.21.15.94:0/535443313 learned_addr learned my addr 172.21.15.94:0/535443313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f7b7fe640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 0x7f0f700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8982d640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed34390 0x559a4f0d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8a02e640 1 -- 172.21.15.94:0/535443313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed34390 msgr2=0x559a4f0d5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed34390 0x559a4f0d5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.529+0000 7f0f8a02e640 1 -- 172.21.15.94:0/535443313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 msgr2=0x559a4f0d6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8a02e640 1 -- 172.21.15.94:0/535443313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4efc5a40 con 0x559a4ed0b4d0 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8982d640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:55.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8a82f640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 0x7f0f700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f7b7fe640 1 -- 172.21.15.94:0/1778390021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0f7c0df380 con 0x7f0f84108770 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0d61d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80031cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87a866b8636a7fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f78bfe640 1 -- 172.21.15.94:0/535443313 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035030 con 0x559a4ed0b4d0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f8a82f640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 0x7f0f700b1ff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0dd150 tx=0x7f0f7c0ca040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f78bfe640 1 -- 172.21.15.94:0/535443313 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f80040070 con 0x559a4ed0b4d0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.530+0000 7f0f78bfe640 1 -- 172.21.15.94:0/535443313 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80003af0 con 0x559a4ed0b4d0 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- 172.21.15.94:0/535443313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 msgr2=0x559a4f0d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0d61d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80031cc0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- 172.21.15.94:0/535443313 shutdown_connections 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed136b0 0x559a4f0d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0d61d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 --2- 172.21.15.94:0/535443313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed34390 0x559a4f0d5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- 172.21.15.94:0/535443313 >> 172.21.15.94:0/535443313 conn(0x559a4ed284e0 msgr2=0x559a4f0d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- 172.21.15.94:0/535443313 shutdown_connections 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- 172.21.15.94:0/535443313 wait complete. 2024-04-25T14:00:55.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 -- start start 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.531+0000 7f0f93216740 1 Processor -- start 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a4f0d7910 con 0x559a4ed34390 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a4efc1d10 con 0x559a4ed0b4d0 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a4f0d7a90 con 0x559a4ed136b0 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f93216740 10 client.0 _ll_register_callbacks cb 0x559a4efee210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38540/0 (socket says 172.21.15.94:38540) 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8982d640 1 -- 172.21.15.94:0/4282067902 learned_addr learned my addr 172.21.15.94:0/4282067902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8982d640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 -- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 msgr2=0x559a4f0dfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0dfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 -- 172.21.15.94:0/4282067902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 msgr2=0x559a4f0e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 -- 172.21.15.94:0/4282067902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4ee167d0 con 0x559a4ed136b0 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8982d640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:55.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0e0060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003740 tx=0x7f0f8002ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c453a0b78b90540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.532+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035040 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f80040070 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0f80045470 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0f80031870 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 10 client.31339 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559a4ed4c3f0 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f80040070 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0f8003c740 con 0x559a4ed136b0 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 10 client.31339 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 10 client.31339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 20 client.31339 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ecf2f60 0x559a4f0ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 4 client.31339 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a4f048990 con 0x559a4ecf2f60 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.533+0000 7f0f8982d640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ecf2f60 0x559a4f0ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.534+0000 7f0f6dffb640 10 client.31339 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.534+0000 7f0f8982d640 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ecf2f60 0x559a4f0ed080 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8363362fc0f5b9 server_cookie=7d0c391066bdb8c8 in_seq=0 out_seq=0 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.535+0000 7f0f6dffb640 1 -- 172.21.15.94:0/4282067902 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a4f048990 con 0x559a4ecf2f60 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.535+0000 7f0f6dffb640 10 client.31339 handle_command_reply: tid=0 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.535+0000 7f0f93216740 1 client.31339 shutdown 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 msgr2=0x559a4f0e0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0e0060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003740 tx=0x7f0f8002ab10 comp rx=0 tx=0).stop 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 shutdown_connections 2024-04-25T14:00:55.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ecf2f60 0x559a4f0ed080 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ed34390 0x559a4f0e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f6dffb640 0 client.31339 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0e0060 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 --2- 172.21.15.94:0/4282067902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0dfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 >> 172.21.15.94:0/4282067902 conn(0x559a4ed284e0 msgr2=0x559a4f0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 shutdown_connections 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 1 -- 172.21.15.94:0/4282067902 wait complete. 2024-04-25T14:00:55.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.536+0000 7f0f93216740 20 client.31339 trim_cache size 0 max 16384 2024-04-25T14:00:55.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 Processor -- start 2024-04-25T14:00:55.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 -- start start 2024-04-25T14:00:55.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0deb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ab290 con 0x559a4ecf2f60 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841acdd0 con 0x559a4ed0b4d0 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f93216740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ab960 con 0x559a4ed136b0 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38544/0 (socket says 172.21.15.94:38544) 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 -- 172.21.15.94:0/3617341932 learned_addr learned my addr 172.21.15.94:0/3617341932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 -- 172.21.15.94:0/3617341932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 msgr2=0x559a4f0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 -- 172.21.15.94:0/3617341932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 msgr2=0x559a4f0deb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0deb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.555+0000 7f0f8982d640 1 -- 172.21.15.94:0/3617341932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4ed0c0f0 con 0x559a4ecf2f60 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.556+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f0f740058c0 tx=0x7f0f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba65fb2711025aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.556+0000 7f0f6ffff640 1 -- 172.21.15.94:0/3617341932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7401c040 con 0x559a4ecf2f60 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.556+0000 7f0f6ffff640 1 -- 172.21.15.94:0/3617341932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f74011110 con 0x559a4ecf2f60 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.556+0000 7f0f6ffff640 1 -- 172.21.15.94:0/3617341932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74005310 con 0x559a4ecf2f60 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- 172.21.15.94:0/3617341932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 msgr2=0x559a4f0daee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f0f740058c0 tx=0x7f0f740046c0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- 172.21.15.94:0/3617341932 shutdown_connections 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed136b0 0x559a4f0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed0b4d0 0x559a4f0deb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- 172.21.15.94:0/3617341932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0daee0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- 172.21.15.94:0/3617341932 >> 172.21.15.94:0/3617341932 conn(0x559a4ed284e0 msgr2=0x559a4f0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- 172.21.15.94:0/3617341932 shutdown_connections 2024-04-25T14:00:55.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- 172.21.15.94:0/3617341932 wait complete. 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- start start 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 Processor -- start 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed0b4d0 0x559a4f0e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed136b0 0x559a4f1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ac700 con 0x559a4ecf2f60 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ac030 con 0x559a4ed136b0 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841a9270 con 0x559a4ed0b4d0 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f93216740 10 client.0 _ll_register_callbacks cb 0x7f0f7400a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f8982d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38554/0 (socket says 172.21.15.94:38554) 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f8982d640 1 -- 172.21.15.94:0/3770487239 learned_addr learned my addr 172.21.15.94:0/3770487239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed0b4d0 0x559a4f0e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.557+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed136b0 0x559a4f1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 -- 172.21.15.94:0/3770487239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed0b4d0 msgr2=0x559a4f0e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed0b4d0 0x559a4f0e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 -- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed136b0 msgr2=0x559a4f1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed136b0 0x559a4f1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 -- 172.21.15.94:0/3770487239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4effb4b0 con 0x559a4ecf2f60 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f8982d640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f0f7402a040 tx=0x7f0f740117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76121e2b66ec481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740112c0 con 0x559a4ecf2f60 2024-04-25T14:00:55.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0f74011460 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f74011110 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0f74013020 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 10 client.22275 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559a4f154750 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f74011110 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0f740257f0 con 0x559a4ecf2f60 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 10 client.22275 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 10 client.22275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 20 client.22275 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.558+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ed34390 0x559a4f157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.559+0000 7f0f93216740 4 client.22275 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.559+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a4f042800 con 0x559a4ed34390 2024-04-25T14:00:55.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.559+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ed34390 0x559a4f157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.559+0000 7f0f6dffb640 10 client.22275 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.559+0000 7f0f8a02e640 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ed34390 0x559a4f157060 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f320e5c21e933f server_cookie=9f39b2b68d23bcc5 in_seq=0 out_seq=0 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f6dffb640 1 -- 172.21.15.94:0/3770487239 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a4f042800 con 0x559a4ed34390 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f6dffb640 10 client.22275 handle_command_reply: tid=0 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 client.22275 shutdown 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 msgr2=0x559a4f0e64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f0f7402a040 tx=0x7f0f740117f0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 shutdown_connections 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559a4ed34390 0x559a4f157060 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a4ed136b0 0x559a4f1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a4ed0b4d0 0x559a4f0e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f6dffb640 0 client.22275 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 --2- 172.21.15.94:0/3770487239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a4ecf2f60 0x559a4f0e64a0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 >> 172.21.15.94:0/3770487239 conn(0x559a4ed284e0 msgr2=0x559a4f0d4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 shutdown_connections 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 1 -- 172.21.15.94:0/3770487239 wait complete. 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.565+0000 7f0f93216740 20 client.22275 trim_cache size 0 max 16384 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:00:55.571 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:55.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 msgr2=0x7f0f700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 0x7f0f700b1ff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0dd150 tx=0x7f0f7c0ca040 comp rx=0 tx=0).stop 2024-04-25T14:00:55.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 msgr2=0x7f0f841a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 0x7f0f841a1540 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009fd0 tx=0x7f0f7c00ef10 comp rx=0 tx=0).stop 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 shutdown_connections 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0f700afae0 0x7f0f700b1ff0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84108770 0x7f0f841a1540 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f84103a70 0x7f0f8419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 --2- 172.21.15.94:0/1778390021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f84102830 0x7f0f8419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 >> 172.21.15.94:0/1778390021 conn(0x7f0f840fda20 msgr2=0x7f0f84104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 shutdown_connections 2024-04-25T14:00:55.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.566+0000 7f0f8b030640 1 -- 172.21.15.94:0/1778390021 wait complete. 2024-04-25T14:00:55.576 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:00:55.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 -- 172.21.15.94:0/1016515266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 msgr2=0x7f23ac111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1016515266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac111b80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009860 tx=0x7f23a802a340 comp rx=0 tx=0).stop 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 -- 172.21.15.94:0/1016515266 shutdown_connections 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1016515266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac114520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1016515266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac111b80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1016515266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 0x7f23ac10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 -- 172.21.15.94:0/1016515266 >> 172.21.15.94:0/1016515266 conn(0x7f23ac0704e0 msgr2=0x7f23ac072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 -- 172.21.15.94:0/1016515266 shutdown_connections 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.694+0000 7f23b39ce640 1 -- 172.21.15.94:0/1016515266 wait complete. 2024-04-25T14:00:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 Processor -- start 2024-04-25T14:00:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 -- start start 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 0x7f23ac1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac117e90 con 0x7f23ac1120c0 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac117d10 con 0x7f23ac109fc0 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b39ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac118010 con 0x7f23ac10f760 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b1f6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b176a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 0x7f23ac1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b1f6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38578/0 (socket says 172.21.15.94:38578) 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.695+0000 7f23b1f6b640 1 -- 172.21.15.94:0/1081527301 learned_addr learned my addr 172.21.15.94:0/1081527301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b0f69640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 -- 172.21.15.94:0/1081527301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 msgr2=0x7f23ac1a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac1a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 -- 172.21.15.94:0/1081527301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 msgr2=0x7f23ac1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 0x7f23ac1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 -- 172.21.15.94:0/1081527301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac1aa610 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b0f69640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac1a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23b1f6b640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f23a400eb10 tx=0x7f23a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a400ce20 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.696+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a4005020 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.697+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4010690 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.697+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac1aa900 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.697+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1aae50 con 0x7f23ac1120c0 2024-04-25T14:00:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.698+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac072000 con 0x7f23ac1120c0 2024-04-25T14:00:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.700+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f23a4005510 con 0x7f23ac1120c0 2024-04-25T14:00:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.701+0000 7f23a27fc640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 0x7f23980b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.702+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23a40df4c0 con 0x7f23ac1120c0 2024-04-25T14:00:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.702+0000 7f23b176a640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 0x7f23980b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.702+0000 7f23b176a640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 0x7f23980b2100 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f23ac10b0c0 tx=0x7f239c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.707+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23a4053ff0 con 0x7f23ac1120c0 2024-04-25T14:00:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.953+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23ac02db50 con 0x7f23ac1120c0 2024-04-25T14:00:55.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.954+0000 7f23a27fc640 1 -- 172.21.15.94:0/1081527301 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f23a40a7fa0 con 0x7f23ac1120c0 2024-04-25T14:00:55.953 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:00:55.953 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:00:55.953 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 msgr2=0x7f23980b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 0x7f23980b2100 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f23ac10b0c0 tx=0x7f239c009260 comp rx=0 tx=0).stop 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 msgr2=0x7f23ac1a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f23a400eb10 tx=0x7f23a400ef10 comp rx=0 tx=0).stop 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 shutdown_connections 2024-04-25T14:00:55.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23980afbf0 0x7f23980b2100 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ac1120c0 0x7f23ac1a9fa0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ac10f760 0x7f23ac1a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 --2- 172.21.15.94:0/1081527301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23ac109fc0 0x7f23ac1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 >> 172.21.15.94:0/1081527301 conn(0x7f23ac0704e0 msgr2=0x7f23ac110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 shutdown_connections 2024-04-25T14:00:55.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:55.956+0000 7f23b39ce640 1 -- 172.21.15.94:0/1081527301 wait complete. 2024-04-25T14:00:55.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:00:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2792308242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 msgr2=0x7f2c9c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2792308242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c10ad40 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009ca0 tx=0x7f2c9402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2792308242 shutdown_connections 2024-04-25T14:00:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2792308242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c10ad40 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2792308242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c103c20 0x7f2c9c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2792308242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c1029e0 0x7f2c9c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2792308242 >> 172.21.15.94:0/2792308242 conn(0x7f2c9c0fdbd0 msgr2=0x7f2c9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2792308242 shutdown_connections 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.082+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2792308242 wait complete. 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 Processor -- start 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 -- start start 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c1029e0 0x7f2c9c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c103c20 0x7f2c9c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c10e510 con 0x7f2c9c108920 2024-04-25T14:00:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c10e390 con 0x7f2c9c103c20 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca3a1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c10e690 con 0x7f2c9c1029e0 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca1fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca1fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38608/0 (socket says 172.21.15.94:38608) 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/2929597637 learned_addr learned my addr 172.21.15.94:0/2929597637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c103c20 0x7f2c9c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.083+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c1029e0 0x7f2c9c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/2929597637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c1029e0 msgr2=0x7f2c9c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c1029e0 0x7f2c9c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/2929597637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c103c20 msgr2=0x7f2c9c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c103c20 0x7f2c9c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/2929597637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c1a0f20 con 0x7f2c9c108920 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004af0 tx=0x7f2c94031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2c927fc640 1 -- 172.21.15.94:0/2929597637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003b70 con 0x7f2c9c108920 2024-04-25T14:00:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c1a11b0 con 0x7f2c9c108920 2024-04-25T14:00:56.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2c927fc640 1 -- 172.21.15.94:0/2929597637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c94003d10 con 0x7f2c9c108920 2024-04-25T14:00:56.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2c927fc640 1 -- 172.21.15.94:0/2929597637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9402fb20 con 0x7f2c9c108920 2024-04-25T14:00:56.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.084+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c1a1750 con 0x7f2c9c108920 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 Processor -- start 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2c927fc640 1 -- 172.21.15.94:0/2929597637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2c94004580 con 0x7f2c9c108920 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 -- start start 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624dddb7d80 con 0x5624dda884d0 2024-04-25T14:00:56.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624ddd71a20 con 0x5624ddab1390 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.087+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624dddb3d40 con 0x5624dda906b0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38616/0 (socket says 172.21.15.94:38616) 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 -- 172.21.15.94:0/733741464 learned_addr learned my addr 172.21.15.94:0/733741464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 -- 172.21.15.94:0/733741464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 msgr2=0x5624dde53960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde53960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 -- 172.21.15.94:0/733741464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 msgr2=0x5624dde54ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde54ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 -- 172.21.15.94:0/733741464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ddd7cc30 con 0x5624dda884d0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2c927fc640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 0x7f2c880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 0x7f2c880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2c927fc640 1 -- 172.21.15.94:0/2929597637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f2c94100290 con 0x7f2c9c108920 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.088+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009c60 tx=0x7f2c98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d4579a6039348f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.089+0000 7f2c877fe640 1 -- 172.21.15.94:0/733741464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98009e10 con 0x5624dda884d0 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.089+0000 7f2c877fe640 1 -- 172.21.15.94:0/733741464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c98005020 con 0x5624dda884d0 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.089+0000 7f2c877fe640 1 -- 172.21.15.94:0/733741464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004a00 con 0x5624dda884d0 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- 172.21.15.94:0/733741464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 msgr2=0x5624dde53420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009c60 tx=0x7f2c98011810 comp rx=0 tx=0).stop 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 0x7f2c880b2000 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009fd0 tx=0x7f2c8c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- 172.21.15.94:0/733741464 shutdown_connections 2024-04-25T14:00:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda884d0 0x5624dde53420 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/733741464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- 172.21.15.94:0/733741464 >> 172.21.15.94:0/733741464 conn(0x5624ddaa54e0 msgr2=0x5624dde4f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- 172.21.15.94:0/733741464 shutdown_connections 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- 172.21.15.94:0/733741464 wait complete. 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 -- start start 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.090+0000 7f2caa9a1740 1 Processor -- start 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 0x5624dde661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624dde640a0 con 0x5624ddab1390 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624ddd3edc0 con 0x5624dda906b0 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624dde64220 con 0x5624dda884d0 2024-04-25T14:00:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2caa9a1740 10 client.0 _ll_register_callbacks cb 0x5624ddd5a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca0fb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 0x5624dde661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46810/0 (socket says 172.21.15.94:46810) 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca0fb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 0x5624dde661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36660/0 (socket says 172.21.15.94:36660) 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 -- 172.21.15.94:0/2614074684 learned_addr learned my addr 172.21.15.94:0/2614074684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca1fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 -- 172.21.15.94:0/2614074684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 msgr2=0x5624dde661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 0x5624dde661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 -- 172.21.15.94:0/2614074684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 msgr2=0x5624dde6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca17b6640 1 -- 172.21.15.94:0/2614074684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ddac93f0 con 0x5624dda884d0 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.091+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c98012fd0 tx=0x7f2c98004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f6891d1770d3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2c84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98028020 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f2c98004b60 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c98005020 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2c98033050 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2caa9a1740 10 client.31363 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624ddb937d0 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.092+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c98005020 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f2c98023be0 con 0x5624dda884d0 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 10 client.31363 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 10 client.31363 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 20 client.31363 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda6ff60 0x5624dde73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 4 client.31363 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624dddc5990 con 0x5624dda6ff60 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda6ff60 0x5624dde73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda6ff60 0x5624dde73750 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cff51dd70b230af3 server_cookie=ae9381cff213b663 in_seq=0 out_seq=0 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.093+0000 7f2c84ff9640 10 client.31363 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.094+0000 7f2c84ff9640 1 -- 172.21.15.94:0/2614074684 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624dddc5990 con 0x5624dda6ff60 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.094+0000 7f2c84ff9640 10 client.31363 handle_command_reply: tid=0 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.094+0000 7f2caa9a1740 1 client.31363 shutdown 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 msgr2=0x5624dde65ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c98012fd0 tx=0x7f2c98004970 comp rx=0 tx=0).stop 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 shutdown_connections 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda6ff60 0x5624dde73750 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda906b0 0x5624dde661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/2614074684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde65ca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2c84ff9640 0 client.31363 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 >> 172.21.15.94:0/2614074684 conn(0x5624ddaa54e0 msgr2=0x5624dde4d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 shutdown_connections 2024-04-25T14:00:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 1 -- 172.21.15.94:0/2614074684 wait complete. 2024-04-25T14:00:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.095+0000 7f2caa9a1740 20 client.31363 trim_cache size 0 max 16384 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 Processor -- start 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 -- start start 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde56020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1a92e0 con 0x5624dda906b0 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1aae20 con 0x5624ddab1390 2024-04-25T14:00:56.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1a99b0 con 0x5624dda884d0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.109+0000 7f2ca0fb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38638/0 (socket says 172.21.15.94:38638) 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 -- 172.21.15.94:0/1146844587 learned_addr learned my addr 172.21.15.94:0/1146844587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde56020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 -- 172.21.15.94:0/1146844587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 msgr2=0x5624dde56020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde56020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 -- 172.21.15.94:0/1146844587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 msgr2=0x5624dde538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 -- 172.21.15.94:0/1146844587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624dde58e40 con 0x5624dda906b0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde56020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0058c0 tx=0x7f2c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39d6d515a75c806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2c86ffd640 1 -- 172.21.15.94:0/1146844587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c011dc0 con 0x5624dda906b0 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2c86ffd640 1 -- 172.21.15.94:0/1146844587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c0054d0 con 0x5624dda906b0 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.110+0000 7f2c86ffd640 1 -- 172.21.15.94:0/1146844587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004e40 con 0x5624dda906b0 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- 172.21.15.94:0/1146844587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 msgr2=0x5624dde64b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0058c0 tx=0x7f2c8c0046c0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- 172.21.15.94:0/1146844587 shutdown_connections 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624ddab1390 0x5624dde538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624dda906b0 0x5624dde64b80 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/1146844587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda884d0 0x5624dde56020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- 172.21.15.94:0/1146844587 >> 172.21.15.94:0/1146844587 conn(0x5624ddaa54e0 msgr2=0x5624dde531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- 172.21.15.94:0/1146844587 shutdown_connections 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- 172.21.15.94:0/1146844587 wait complete. 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 -- start start 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.111+0000 7f2caa9a1740 1 Processor -- start 2024-04-25T14:00:56.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda884d0 0x5624dde6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1aa750 con 0x5624ddab1390 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1aa080 con 0x5624dda884d0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1a72c0 con 0x5624dda906b0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2caa9a1740 10 client.0 _ll_register_callbacks cb 0x7f2c8c009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38652/0 (socket says 172.21.15.94:38652) 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca17b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda884d0 0x5624dde6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/3008514744 learned_addr learned my addr 172.21.15.94:0/3008514744 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca0fb5640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 msgr2=0x5624dde6b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde6b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/3008514744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda884d0 msgr2=0x5624dde6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda884d0 0x5624dde6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 -- 172.21.15.94:0/3008514744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624dde6c3e0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2ca1fb7640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f2c940fdd40 tx=0x7f2c940c8020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132bdfd5fe398b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.112+0000 7f2c84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940e1070 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f2c940c4c30 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c8c0054d0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2c9402fed0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 10 client.22299 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624ddedaad0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c0054d0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f2c940c89c0 con 0x5624ddab1390 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 10 client.22299 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 10 client.22299 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 20 client.22299 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda453a0 0x5624ddedd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 4 client.22299 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624dddbf800 con 0x5624dda453a0 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.113+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda453a0 0x5624ddedd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2ca17b6640 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda453a0 0x5624ddedd480 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb4e90beb8cb1611 server_cookie=a8b8a0b0bf9b7ec6 in_seq=0 out_seq=0 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2c84ff9640 10 client.22299 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2c84ff9640 1 -- 172.21.15.94:0/3008514744 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624dddbf800 con 0x5624dda453a0 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2c84ff9640 10 client.22299 handle_command_reply: tid=0 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 client.22299 shutdown 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 msgr2=0x5624dde6bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f2c940fdd40 tx=0x7f2c940c8020 comp rx=0 tx=0).stop 2024-04-25T14:00:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 shutdown_connections 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5624dda453a0 0x5624ddedd480 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ddab1390 0x5624dde6bc80 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624dda906b0 0x5624dde6b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2c84ff9640 0 client.22299 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 --2- 172.21.15.94:0/3008514744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624dda884d0 0x5624dde6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 >> 172.21.15.94:0/3008514744 conn(0x5624ddaa54e0 msgr2=0x5624dde657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 shutdown_connections 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 1 -- 172.21.15.94:0/3008514744 wait complete. 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.114+0000 7f2caa9a1740 20 client.22299 trim_cache size 0 max 16384 2024-04-25T14:00:56.120 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:56.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 msgr2=0x7f2c880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 0x7f2c880b2000 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009fd0 tx=0x7f2c8c005850 comp rx=0 tx=0).stop 2024-04-25T14:00:56.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 msgr2=0x7f2c9c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004af0 tx=0x7f2c94031700 comp rx=0 tx=0).stop 2024-04-25T14:00:56.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 shutdown_connections 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2c880afaf0 0x7f2c880b2000 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c9c108920 0x7f2c9c1a08b0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c9c103c20 0x7f2c9c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 --2- 172.21.15.94:0/2929597637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c9c1029e0 0x7f2c9c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 >> 172.21.15.94:0/2929597637 conn(0x7f2c9c0fdbd0 msgr2=0x7f2c9c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 shutdown_connections 2024-04-25T14:00:56.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.115+0000 7f2ca3a1a640 1 -- 172.21.15.94:0/2929597637 wait complete. 2024-04-25T14:00:56.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:00:56.146 INFO:tasks.workunit.client.0.smithi094.stdout: 10 122133 19.20 MB/sec execute 218 sec latency 71.963 ms 2024-04-25T14:00:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:55 smithi094 ceph-mon[29195]: pgmap v925: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-04-25T14:00:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:55 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1081527301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 -- 172.21.15.94:0/1826061273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 msgr2=0x7f6898103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 --2- 172.21.15.94:0/1826061273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f6898103700 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f6888009900 tx=0x7f688802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 -- 172.21.15.94:0/1826061273 shutdown_connections 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 --2- 172.21.15.94:0/1826061273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898108940 0x7f689810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 --2- 172.21.15.94:0/1826061273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898103c40 0x7f6898104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 --2- 172.21.15.94:0/1826061273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f6898103700 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 -- 172.21.15.94:0/1826061273 >> 172.21.15.94:0/1826061273 conn(0x7f68980fdbf0 msgr2=0x7f68981000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 -- 172.21.15.94:0/1826061273 shutdown_connections 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.242+0000 7f68a04c6640 1 -- 172.21.15.94:0/1826061273 wait complete. 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 Processor -- start 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 -- start start 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898103c40 0x7f689819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898108940 0x7f68981a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f689810f630 con 0x7f6898102a00 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f689810f4b0 con 0x7f6898108940 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f68a04c6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f689810f7b0 con 0x7f6898103c40 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f689e262640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f689e262640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38668/0 (socket says 172.21.15.94:38668) 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f689e262640 1 -- 172.21.15.94:0/2671040760 learned_addr learned my addr 172.21.15.94:0/2671040760 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f689da61640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898103c40 0x7f689819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.243+0000 7f689ea63640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898108940 0x7f68981a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 -- 172.21.15.94:0/2671040760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898103c40 msgr2=0x7f689819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898103c40 0x7f689819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 -- 172.21.15.94:0/2671040760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898108940 msgr2=0x7f68981a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898108940 0x7f68981a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 -- 172.21.15.94:0/2671040760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68981a1e00 con 0x7f6898102a00 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f689e262640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f6888006010 tx=0x7f6888004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f688f7fe640 1 -- 172.21.15.94:0/2671040760 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688803f030 con 0x7f6898102a00 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f688f7fe640 1 -- 172.21.15.94:0/2671040760 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6888003aa0 con 0x7f6898102a00 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f688f7fe640 1 -- 172.21.15.94:0/2671040760 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888035070 con 0x7f6898102a00 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68981a2090 con 0x7f6898102a00 2024-04-25T14:00:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.244+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981a25d0 con 0x7f6898102a00 2024-04-25T14:00:56.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:55 smithi138 ceph-mon[32977]: pgmap v925: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-04-25T14:00:56.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1081527301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f688f7fe640 1 -- 172.21.15.94:0/2671040760 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6888042030 con 0x7f6898102a00 2024-04-25T14:00:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 Processor -- start 2024-04-25T14:00:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 -- start start 2024-04-25T14:00:56.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bdc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a868146b0 0x562a86bdce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a86b3bd80 con 0x562a86835390 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a86af5a20 con 0x562a868146b0 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f68a7377740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a86b37d40 con 0x562a8680c4d0 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689e262640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bdc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689da61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a868146b0 0x562a86bdce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689e262640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bdc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46846/0 (socket says 172.21.15.94:46846) 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38676/0 (socket says 172.21.15.94:38676) 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.247+0000 7f689e262640 1 -- 172.21.15.94:0/67913613 learned_addr learned my addr 172.21.15.94:0/67913613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 -- 172.21.15.94:0/67913613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 msgr2=0x562a86bdc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bdc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 -- 172.21.15.94:0/67913613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a868146b0 msgr2=0x562a86bdce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a868146b0 0x562a86bdce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 -- 172.21.15.94:0/67913613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a86b00c30 con 0x562a86835390 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f688f7fe640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 0x7f68840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689e262640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 0x7f68840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f689ea63640 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f6890009f30 tx=0x7f6890010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02d5f7924fb45e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f688f7fe640 1 -- 172.21.15.94:0/2671040760 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68880ff170 con 0x7f6898102a00 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f688cbfe640 1 -- 172.21.15.94:0/67913613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890016070 con 0x562a86835390 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f688cbfe640 1 -- 172.21.15.94:0/67913613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890004d20 con 0x562a86835390 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.248+0000 7f688cbfe640 1 -- 172.21.15.94:0/67913613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900127d0 con 0x562a86835390 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 -- 172.21.15.94:0/67913613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 msgr2=0x562a86bdbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f6890009f30 tx=0x7f6890010800 comp rx=0 tx=0).stop 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f689e262640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 0x7f68840b2080 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f68880c75f0 tx=0x7f68880ea040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 -- 172.21.15.94:0/67913613 shutdown_connections 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a868146b0 0x562a86bdce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bdc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 --2- 172.21.15.94:0/67913613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bdbc20 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 -- 172.21.15.94:0/67913613 >> 172.21.15.94:0/67913613 conn(0x562a868294e0 msgr2=0x562a86bd7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 -- 172.21.15.94:0/67913613 shutdown_connections 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.249+0000 7f68a7377740 1 -- 172.21.15.94:0/67913613 wait complete. 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 -- start start 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 Processor -- start 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86be5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bece00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a86bdd940 con 0x562a86835390 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a86ac2dc0 con 0x562a8680c4d0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a86bddac0 con 0x562a868146b0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f68a7377740 10 client.0 _ll_register_callbacks cb 0x562a86ade600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689da61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689ea63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86be5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689da61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46850/0 (socket says 172.21.15.94:46850) 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689da61640 1 -- 172.21.15.94:0/1405028086 learned_addr learned my addr 172.21.15.94:0/1405028086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.250+0000 7f689da61640 1 -- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 msgr2=0x562a86be5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f689da61640 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86be5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f689da61640 1 -- 172.21.15.94:0/1405028086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 msgr2=0x562a86bece00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f689da61640 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bece00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f689da61640 1 -- 172.21.15.94:0/1405028086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8684d3f0 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f689da61640 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6894012700 tx=0x7f6894012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb36ef9dba81eae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894004030 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6890004d20 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68940041d0 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f68940043d0 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f68a7377740 10 client.31381 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562a869177d0 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890004d20 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.251+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f68940039e0 con 0x562a868146b0 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 10 client.31381 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 10 client.31381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:00:56.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 20 client.31381 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a867f3f60 0x562a86bf3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 4 client.31381 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a86b49990 con 0x562a867f3f60 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f689ea63640 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a867f3f60 0x562a86bf3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f6881ffb640 10 client.31381 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.252+0000 7f689ea63640 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a867f3f60 0x562a86bf3680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0f16294ab3b3f1d server_cookie=501df5f97d891f66 in_seq=0 out_seq=0 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f6881ffb640 1 -- 172.21.15.94:0/1405028086 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a86b49990 con 0x562a867f3f60 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f6881ffb640 10 client.31381 handle_command_reply: tid=0 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 client.31381 shutdown 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 msgr2=0x562a86be5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6894012700 tx=0x7f6894012bc0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 shutdown_connections 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a867f3f60 0x562a86bf3680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a86835390 0x562a86bece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86be5ea0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f6881ffb640 0 client.31381 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 --2- 172.21.15.94:0/1405028086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86be5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 >> 172.21.15.94:0/1405028086 conn(0x562a868294e0 msgr2=0x562a86bd2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 shutdown_connections 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 1 -- 172.21.15.94:0/1405028086 wait complete. 2024-04-25T14:00:56.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.253+0000 7f68a7377740 20 client.31381 trim_cache size 0 max 16384 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 Processor -- start 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 -- start start 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86becd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86bd1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68981a93f0 con 0x562a867f3f60 2024-04-25T14:00:56.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68981aaf30 con 0x562a8680c4d0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f68a7377740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68981a9ac0 con 0x562a868146b0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38696/0 (socket says 172.21.15.94:38696) 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f689ea63640 1 -- 172.21.15.94:0/2895419750 learned_addr learned my addr 172.21.15.94:0/2895419750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.268+0000 7f689da61640 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86becd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86bd1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 -- 172.21.15.94:0/2895419750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 msgr2=0x562a86bd1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86bd1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 -- 172.21.15.94:0/2895419750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 msgr2=0x562a86becd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86becd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 -- 172.21.15.94:0/2895419750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a86be10f0 con 0x562a867f3f60 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f689ea63640 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6890010d00 tx=0x7f6890009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3aa7383889c095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f6883fff640 1 -- 172.21.15.94:0/2895419750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890016040 con 0x562a867f3f60 2024-04-25T14:00:56.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f6883fff640 1 -- 172.21.15.94:0/2895419750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689002d040 con 0x562a867f3f60 2024-04-25T14:00:56.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.269+0000 7f6883fff640 1 -- 172.21.15.94:0/2895419750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890023040 con 0x562a867f3f60 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- 172.21.15.94:0/2895419750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 msgr2=0x562a86be4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6890010d00 tx=0x7f6890009600 comp rx=0 tx=0).stop 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- 172.21.15.94:0/2895419750 shutdown_connections 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a868146b0 0x562a86bd1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a8680c4d0 0x562a86becd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 --2- 172.21.15.94:0/2895419750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a867f3f60 0x562a86be4840 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- 172.21.15.94:0/2895419750 >> 172.21.15.94:0/2895419750 conn(0x562a868294e0 msgr2=0x562a86be4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- 172.21.15.94:0/2895419750 shutdown_connections 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- 172.21.15.94:0/2895419750 wait complete. 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 -- start start 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.270+0000 7f68a7377740 1 Processor -- start 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a867f3f60 0x562a86bea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68981aa860 con 0x562a868146b0 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68981aa190 con 0x562a867f3f60 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68981a73d0 con 0x562a8680c4d0 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 10 client.0 _ll_register_callbacks cb 0x7f6894009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38700/0 (socket says 172.21.15.94:38700) 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 -- 172.21.15.94:0/3912278396 learned_addr learned my addr 172.21.15.94:0/3912278396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689da61640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a867f3f60 0x562a86bea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 -- 172.21.15.94:0/3912278396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 msgr2=0x562a86bea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 -- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a867f3f60 msgr2=0x562a86bea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a867f3f60 0x562a86bea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 -- 172.21.15.94:0/3912278396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a86bed260 con 0x562a868146b0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f689ea63640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f6890005cc0 tx=0x7f6890010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bc9dd3a34c9666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f6881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890016030 con 0x562a868146b0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f689002d1f0 con 0x562a868146b0 2024-04-25T14:00:56.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.271+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689002d040 con 0x562a868146b0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f689002d3f0 con 0x562a868146b0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 10 client.22317 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562a86bece80 con 0x562a868146b0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689002d040 con 0x562a868146b0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f689003a020 con 0x562a868146b0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 10 client.22317 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 10 client.22317 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 20 client.22317 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a86835390 0x562a86c5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 4 client.22317 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a86b43800 con 0x562a86835390 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.272+0000 7f689da61640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a86835390 0x562a86c5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f6881ffb640 10 client.22317 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:56.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f689da61640 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a86835390 0x562a86c5dd90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e89e8c85dfc0f17b server_cookie=6dfc6baef61366a6 in_seq=0 out_seq=0 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f6881ffb640 1 -- 172.21.15.94:0/3912278396 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a86b43800 con 0x562a86835390 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f6881ffb640 10 client.22317 handle_command_reply: tid=0 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 client.22317 shutdown 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 msgr2=0x562a86beb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f6890005cc0 tx=0x7f6890010fd0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 shutdown_connections 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562a86835390 0x562a86c5dd90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a868146b0 0x562a86beb810 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a8680c4d0 0x562a86bea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 --2- 172.21.15.94:0/3912278396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a867f3f60 0x562a86bea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f6881ffb640 0 client.22317 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 >> 172.21.15.94:0/3912278396 conn(0x562a868294e0 msgr2=0x562a86bda230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 shutdown_connections 2024-04-25T14:00:56.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 1 -- 172.21.15.94:0/3912278396 wait complete. 2024-04-25T14:00:56.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.273+0000 7f68a7377740 20 client.22317 trim_cache size 0 max 16384 2024-04-25T14:00:56.279 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 msgr2=0x7f68840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 0x7f68840b2080 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f68880c75f0 tx=0x7f68880ea040 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 msgr2=0x7f689819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f6888006010 tx=0x7f6888004270 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 shutdown_connections 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68840afb70 0x7f68840b2080 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6898108940 0x7f68981a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6898103c40 0x7f689819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 --2- 172.21.15.94:0/2671040760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6898102a00 0x7f689819a330 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 >> 172.21.15.94:0/2671040760 conn(0x7f68980fdbf0 msgr2=0x7f6898104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 shutdown_connections 2024-04-25T14:00:56.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.274+0000 7f68a04c6640 1 -- 172.21.15.94:0/2671040760 wait complete. 2024-04-25T14:00:56.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:00:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:55 smithi137 ceph-mon[32568]: pgmap v925: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 596 op/s 2024-04-25T14:00:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:55 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1081527301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 -- 172.21.15.94:0/1947858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c1050a0 msgr2=0x7f1d6c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/1947858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1054b0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009900 tx=0x7f1d6802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 -- 172.21.15.94:0/1947858201 shutdown_connections 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/1947858201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 0x7f1d6c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/1947858201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c101750 0x7f1d6c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/1947858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1054b0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 -- 172.21.15.94:0/1947858201 >> 172.21.15.94:0/1947858201 conn(0x7f1d6c0fd350 msgr2=0x7f1d6c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 -- 172.21.15.94:0/1947858201 shutdown_connections 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.440+0000 7f1d73b2e640 1 -- 172.21.15.94:0/1947858201 wait complete. 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 Processor -- start 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 -- start start 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 0x7f1d6c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c10f610 con 0x7f1d6c101750 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c10f490 con 0x7f1d6c102250 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d73b2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c10f790 con 0x7f1d6c1050a0 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38716/0 (socket says 172.21.15.94:38716) 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 -- 172.21.15.94:0/2106209172 learned_addr learned my addr 172.21.15.94:0/2106209172 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d710c9640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 0x7f1d6c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d720cb640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 -- 172.21.15.94:0/2106209172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 msgr2=0x7f1d6c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 -- 172.21.15.94:0/2106209172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 msgr2=0x7f1d6c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 0x7f1d6c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d718ca640 1 -- 172.21.15.94:0/2106209172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c1a1d80 con 0x7f1d6c101750 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.441+0000 7f1d720cb640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d718ca640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d62ffd640 1 -- 172.21.15.94:0/2106209172 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6803f030 con 0x7f1d6c101750 2024-04-25T14:00:56.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c1a2010 con 0x7f1d6c101750 2024-04-25T14:00:56.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d62ffd640 1 -- 172.21.15.94:0/2106209172 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d68031910 con 0x7f1d6c101750 2024-04-25T14:00:56.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d62ffd640 1 -- 172.21.15.94:0/2106209172 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68035070 con 0x7f1d6c101750 2024-04-25T14:00:56.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.442+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1a2500 con 0x7f1d6c101750 2024-04-25T14:00:56.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.444+0000 7f1d62ffd640 1 -- 172.21.15.94:0/2106209172 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1d68042030 con 0x7f1d6c101750 2024-04-25T14:00:56.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 Processor -- start 2024-04-25T14:00:56.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 -- start start 2024-04-25T14:00:56.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db76f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db76fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d8db6d3d80 con 0x55d8db3ac6b0 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8db68da20 con 0x55d8db3a44d0 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8db6cfd40 con 0x55d8db3cd390 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d710c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db76fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d710c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38726/0 (socket says 172.21.15.94:38726) 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d720cb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db76f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d710c9640 1 -- 172.21.15.94:0/3109785910 learned_addr learned my addr 172.21.15.94:0/3109785910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d62ffd640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 0x7f1d580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.445+0000 7f1d720cb640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 0x7f1d580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 -- 172.21.15.94:0/3109785910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 msgr2=0x55d8db76f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db76f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 -- 172.21.15.94:0/3109785910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 msgr2=0x55d8db76fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db76fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 -- 172.21.15.94:0/3109785910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8db698c30 con 0x55d8db3ac6b0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db76fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d62ffd640 1 -- 172.21.15.94:0/2106209172 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d680ff090 con 0x7f1d6c101750 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24f802d1995ce48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d720cb640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 0x7f1d580b2000 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55d8db770860 tx=0x7f1d64004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d57fff640 1 -- 172.21.15.94:0/3109785910 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c005220 con 0x55d8db3ac6b0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d57fff640 1 -- 172.21.15.94:0/3109785910 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c004c50 con 0x55d8db3ac6b0 2024-04-25T14:00:56.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.446+0000 7f1d57fff640 1 -- 172.21.15.94:0/3109785910 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0196b0 con 0x55d8db3ac6b0 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3109785910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 msgr2=0x55d8db770a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c011810 comp rx=0 tx=0).stop 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3109785910 shutdown_connections 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db770a20 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db76fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3109785910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db76f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3109785910 >> 172.21.15.94:0/3109785910 conn(0x55d8db3c14e0 msgr2=0x55d8db76b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3109785910 shutdown_connections 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3109785910 wait complete. 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 -- start start 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.447+0000 7f1d7a9dc740 1 Processor -- start 2024-04-25T14:00:56.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db7795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db7809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d8db771530 con 0x55d8db3ac6b0 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8db65adc0 con 0x55d8db3a44d0 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8db7716b0 con 0x55d8db3cd390 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d7a9dc740 10 client.0 _ll_register_callbacks cb 0x55d8db676600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38738/0 (socket says 172.21.15.94:38738) 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 -- 172.21.15.94:0/3040537783 learned_addr learned my addr 172.21.15.94:0/3040537783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db7795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d720cb640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db7809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 -- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 msgr2=0x55d8db7809f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db7809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 -- 172.21.15.94:0/3040537783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 msgr2=0x55d8db7795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db7795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 -- 172.21.15.94:0/3040537783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8db3e53f0 con 0x55d8db3ac6b0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.448+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01b040 tx=0x7f1d5c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1431d6a3573e0c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c029070 con 0x55d8db3ac6b0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5c004c50 con 0x55d8db3ac6b0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d5c023840 con 0x55d8db3ac6b0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1d5c035050 con 0x55d8db3ac6b0 2024-04-25T14:00:56.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 10 client.22329 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d8db4af7d0 con 0x55d8db3ac6b0 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c004c50 con 0x55d8db3ac6b0 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1d5c019690 con 0x55d8db3ac6b0 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 10 client.22329 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 10 client.22329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 20 client.22329 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db38bf60 0x55d8db786ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 4 client.22329 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.449+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8db6e1990 con 0x55d8db38bf60 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.450+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db38bf60 0x55d8db786ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.450+0000 7f1d557fa640 10 client.22329 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.450+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db38bf60 0x55d8db786ff0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5ac7df988568249 server_cookie=9aba81d8aa5dfdfc in_seq=0 out_seq=0 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d557fa640 1 -- 172.21.15.94:0/3040537783 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d8db6e1990 con 0x55d8db38bf60 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d557fa640 10 client.22329 handle_command_reply: tid=0 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 client.22329 shutdown 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 msgr2=0x55d8db779b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c01b040 tx=0x7f1d5c0048f0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 shutdown_connections 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db38bf60 0x55d8db786ff0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3cd390 0x55d8db7809f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d557fa640 0 client.22329 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db3ac6b0 0x55d8db779b00 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3040537783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db7795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 >> 172.21.15.94:0/3040537783 conn(0x55d8db3c14e0 msgr2=0x55d8db76a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 shutdown_connections 2024-04-25T14:00:56.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3040537783 wait complete. 2024-04-25T14:00:56.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.451+0000 7f1d7a9dc740 20 client.22329 trim_cache size 0 max 16384 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.465+0000 7f1d7a9dc740 1 Processor -- start 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.465+0000 7f1d7a9dc740 1 -- start start 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3a44d0 0x55d8db780910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 0x55d8db78b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1ab520 con 0x55d8db38bf60 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1ad060 con 0x55d8db3ac6b0 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1abbf0 con 0x55d8db3a44d0 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38752/0 (socket says 172.21.15.94:38752) 2024-04-25T14:00:56.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 -- 172.21.15.94:0/4038664635 learned_addr learned my addr 172.21.15.94:0/4038664635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d710c9640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3a44d0 0x55d8db780910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d720cb640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 0x55d8db78b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 -- 172.21.15.94:0/4038664635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3a44d0 msgr2=0x55d8db780910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3a44d0 0x55d8db780910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 -- 172.21.15.94:0/4038664635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 msgr2=0x55d8db78b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 0x55d8db78b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 -- 172.21.15.94:0/4038664635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8db76f3d0 con 0x55d8db38bf60 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d720cb640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 0x55d8db78b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d718ca640 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f1d680c3550 tx=0x7f1d680fc9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d649b0eba6d95484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d577fe640 1 -- 172.21.15.94:0/4038664635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68049040 con 0x55d8db38bf60 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d577fe640 1 -- 172.21.15.94:0/4038664635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d680c7bd0 con 0x55d8db38bf60 2024-04-25T14:00:56.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.466+0000 7f1d577fe640 1 -- 172.21.15.94:0/4038664635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680c3760 con 0x55d8db38bf60 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/4038664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 msgr2=0x55d8db7783c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f1d680c3550 tx=0x7f1d680fc9c0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/4038664635 shutdown_connections 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3ac6b0 0x55d8db78b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3a44d0 0x55d8db780910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/4038664635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db7783c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/4038664635 >> 172.21.15.94:0/4038664635 conn(0x55d8db3c14e0 msgr2=0x55d8db777cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/4038664635 shutdown_connections 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.467+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/4038664635 wait complete. 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 -- start start 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 Processor -- start 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db77dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3ac6b0 0x55d8db7ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1ac990 con 0x55d8db38bf60 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1ac2c0 con 0x55d8db3a44d0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1a9500 con 0x55d8db3ac6b0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d7a9dc740 10 client.0 _ll_register_callbacks cb 0x7f1d680c3390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38766/0 (socket says 172.21.15.94:38766) 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 -- 172.21.15.94:0/3076874703 learned_addr learned my addr 172.21.15.94:0/3076874703 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db77dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d720cb640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3ac6b0 0x55d8db7ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 -- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3ac6b0 msgr2=0x55d8db7ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3ac6b0 0x55d8db7ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 -- 172.21.15.94:0/3076874703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 msgr2=0x55d8db77dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db77dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.468+0000 7f1d718ca640 1 -- 172.21.15.94:0/3076874703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8db780e50 con 0x55d8db38bf60 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d718ca640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d680c7120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad10eb2baf53c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68049040 con 0x55d8db38bf60 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d680c7730 con 0x55d8db38bf60 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d680c7bd0 con 0x55d8db38bf60 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1d680c7930 con 0x55d8db38bf60 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 10 client.22335 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d8db7ee670 con 0x55d8db38bf60 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d680c7bd0 con 0x55d8db38bf60 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1d680c67c0 con 0x55d8db38bf60 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 10 client.22335 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 10 client.22335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 20 client.22335 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db3cd390 0x55d8db7f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 4 client.22335 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.469+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8db6db800 con 0x55d8db3cd390 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db3cd390 0x55d8db7f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d557fa640 10 client.22335 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d710c9640 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db3cd390 0x55d8db7f1030 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feb4de8b2e2d8e3a server_cookie=70e1a2e21794775c in_seq=0 out_seq=0 2024-04-25T14:00:56.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d557fa640 1 -- 172.21.15.94:0/3076874703 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8db6db800 con 0x55d8db3cd390 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d557fa640 10 client.22335 handle_command_reply: tid=0 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 client.22335 shutdown 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 msgr2=0x55d8db77d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d680c7120 comp rx=0 tx=0).stop 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 shutdown_connections 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55d8db3cd390 0x55d8db7f1030 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8db3ac6b0 0x55d8db7ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d557fa640 0 client.22335 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8db3a44d0 0x55d8db77dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 --2- 172.21.15.94:0/3076874703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8db38bf60 0x55d8db77d6d0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 >> 172.21.15.94:0/3076874703 conn(0x55d8db3c14e0 msgr2=0x55d8db76de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 shutdown_connections 2024-04-25T14:00:56.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.470+0000 7f1d7a9dc740 1 -- 172.21.15.94:0/3076874703 wait complete. 2024-04-25T14:00:56.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d7a9dc740 20 client.22335 trim_cache size 0 max 16384 2024-04-25T14:00:56.478 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 msgr2=0x7f1d580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 0x7f1d580b2000 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55d8db770860 tx=0x7f1d64004ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 msgr2=0x7f1d6c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68004270 comp rx=0 tx=0).stop 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 shutdown_connections 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d580afaf0 0x7f1d580b2000 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d6c1050a0 0x7f1d6c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d6c102250 0x7f1d6c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 --2- 172.21.15.94:0/2106209172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d6c101750 0x7f1d6c19a240 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 >> 172.21.15.94:0/2106209172 conn(0x7f1d6c0fd350 msgr2=0x7f1d6c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 shutdown_connections 2024-04-25T14:00:56.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.471+0000 7f1d73b2e640 1 -- 172.21.15.94:0/2106209172 wait complete. 2024-04-25T14:00:56.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:00:56.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 -- 172.21.15.94:0/71511984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 msgr2=0x7fa5801036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 --2- 172.21.15.94:0/71511984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa5801036f0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fa574009900 tx=0x7fa57402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 -- 172.21.15.94:0/71511984 shutdown_connections 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 --2- 172.21.15.94:0/71511984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580108930 0x7fa58010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 --2- 172.21.15.94:0/71511984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580103c30 0x7fa580104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 --2- 172.21.15.94:0/71511984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa5801036f0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 -- 172.21.15.94:0/71511984 >> 172.21.15.94:0/71511984 conn(0x7fa5800fdc00 msgr2=0x7fa580100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 -- 172.21.15.94:0/71511984 shutdown_connections 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.598+0000 7fa586cb6640 1 -- 172.21.15.94:0/71511984 wait complete. 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 Processor -- start 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 -- start start 2024-04-25T14:00:56.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580103c30 0x7fa58019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580108930 0x7fa5801a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010e520 con 0x7fa5801029f0 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010e3a0 con 0x7fa580108930 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa586cb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010e6a0 con 0x7fa580103c30 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38780/0 (socket says 172.21.15.94:38780) 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 -- 172.21.15.94:0/284867040 learned_addr learned my addr 172.21.15.94:0/284867040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa57ffff640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580103c30 0x7fa58019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa585253640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580108930 0x7fa5801a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 -- 172.21.15.94:0/284867040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580103c30 msgr2=0x7fa58019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580103c30 0x7fa58019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 -- 172.21.15.94:0/284867040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580108930 msgr2=0x7fa5801a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.599+0000 7fa584a52640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580108930 0x7fa5801a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa584a52640 1 -- 172.21.15.94:0/284867040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5801a1d70 con 0x7fa5801029f0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa584a52640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fa574031440 tx=0x7fa574003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa57dffb640 1 -- 172.21.15.94:0/284867040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57403f070 con 0x7fa5801029f0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa57dffb640 1 -- 172.21.15.94:0/284867040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa574031c50 con 0x7fa5801029f0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa57dffb640 1 -- 172.21.15.94:0/284867040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574035030 con 0x7fa5801029f0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5801a2000 con 0x7fa5801029f0 2024-04-25T14:00:56.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.600+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801a24a0 con 0x7fa5801029f0 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.602+0000 7fa58db6b740 1 Processor -- start 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 -- start start 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa57dffb640 1 -- 172.21.15.94:0/284867040 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa574042070 con 0x7fa5801029f0 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83ee6b0 0x55b0b87b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0b8715d80 con 0x55b0b83e64d0 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0b86cfa20 con 0x55b0b83ee6b0 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa58db6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0b8711d40 con 0x55b0b840f390 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38796/0 (socket says 172.21.15.94:38796) 2024-04-25T14:00:56.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 -- 172.21.15.94:0/275539109 learned_addr learned my addr 172.21.15.94:0/275539109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa585253640 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa57ffff640 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83ee6b0 0x55b0b87b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 -- 172.21.15.94:0/275539109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 msgr2=0x55b0b87b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 -- 172.21.15.94:0/275539109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83ee6b0 msgr2=0x55b0b87b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83ee6b0 0x55b0b87b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 -- 172.21.15.94:0/275539109 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b86dac30 con 0x55b0b83e64d0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa57dffb640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 0x7fa5700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa584a52640 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fa574105610 tx=0x7fa574105700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b0f2a0b3520ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.603+0000 7fa585253640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 0x7fa5700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa57dffb640 1 -- 172.21.15.94:0/284867040 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa5740ff4a0 con 0x7fa5801029f0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa56affd640 1 -- 172.21.15.94:0/275539109 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740c3850 con 0x55b0b83e64d0 2024-04-25T14:00:56.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa56affd640 1 -- 172.21.15.94:0/275539109 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa574031e00 con 0x55b0b83e64d0 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa56affd640 1 -- 172.21.15.94:0/275539109 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740c7100 con 0x55b0b83e64d0 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa58db6b740 1 -- 172.21.15.94:0/275539109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 msgr2=0x55b0b87b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa58db6b740 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fa574105610 tx=0x7fa574105700 comp rx=0 tx=0).stop 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.604+0000 7fa585253640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 0x7fa5700b2000 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55b0b87b2860 tx=0x7fa578009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 -- 172.21.15.94:0/275539109 shutdown_connections 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83ee6b0 0x55b0b87b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83e64d0 0x55b0b87b1d90 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- 172.21.15.94:0/275539109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 -- 172.21.15.94:0/275539109 >> 172.21.15.94:0/275539109 conn(0x55b0b84034e0 msgr2=0x55b0b87ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 -- 172.21.15.94:0/275539109 shutdown_connections 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 -- 172.21.15.94:0/275539109 wait complete. 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 -- start start 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 Processor -- start 2024-04-25T14:00:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.605+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0b87b32f0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0b869cdc0 con 0x55b0b840f390 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0b87b3470 con 0x55b0b83e64d0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 10 client.0 _ll_register_callbacks cb 0x55b0b86b8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38812/0 (socket says 172.21.15.94:38812) 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa57ffff640 1 -- 172.21.15.94:0/3378567744 learned_addr learned my addr 172.21.15.94:0/3378567744 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa585253640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 -- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 msgr2=0x55b0b87c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 -- 172.21.15.94:0/3378567744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 msgr2=0x55b0b87bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 -- 172.21.15.94:0/3378567744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b84273f0 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa584a52640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa574102b60 tx=0x7fa57403b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e6a1dfd980c2de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa55ffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574049040 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa57403b460 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0b8723990 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa5740e1050 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 10 client.31414 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.606+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b0b84f17d0 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b0b8723990 con 0x55b0b83e64d0 2024-04-25T14:00:56.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa5740c7bf0 con 0x55b0b83e64d0 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 10 client.31414 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 10 client.31414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 20 client.31414 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83cdf60 0x55b0b87c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 4 client.31414 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b871d800 con 0x55b0b83cdf60 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.607+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83cdf60 0x55b0b87c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa55ffff640 10 client.31414 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83cdf60 0x55b0b87c8b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64a0f09e71a858c8 server_cookie=1bf9e3fef5e7c30f in_seq=0 out_seq=0 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa55ffff640 1 -- 172.21.15.94:0/3378567744 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0b871d800 con 0x55b0b83cdf60 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa55ffff640 10 client.31414 handle_command_reply: tid=0 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 client.31414 shutdown 2024-04-25T14:00:56.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 msgr2=0x55b0b87bb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa574102b60 tx=0x7fa57403b800 comp rx=0 tx=0).stop 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 shutdown_connections 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83cdf60 0x55b0b87c8b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3378567744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb290 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa55ffff640 0 client.31414 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.608+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 >> 172.21.15.94:0/3378567744 conn(0x55b0b84034e0 msgr2=0x55b0b87ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.609+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 shutdown_connections 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.609+0000 7fa58db6b740 1 -- 172.21.15.94:0/3378567744 wait complete. 2024-04-25T14:00:56.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.609+0000 7fa58db6b740 20 client.31414 trim_cache size 0 max 16384 2024-04-25T14:00:56.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 Processor -- start 2024-04-25T14:00:56.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 -- start start 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ab540 con 0x55b0b83ee6b0 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ad080 con 0x55b0b83e64d0 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.622+0000 7fa58db6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801abc10 con 0x55b0b840f390 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38828/0 (socket says 172.21.15.94:38828) 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 -- 172.21.15.94:0/3068296088 learned_addr learned my addr 172.21.15.94:0/3068296088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa585253640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa584a52640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 -- 172.21.15.94:0/3068296088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 msgr2=0x55b0b87b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 -- 172.21.15.94:0/3068296088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 msgr2=0x55b0b87bb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 -- 172.21.15.94:0/3068296088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b87bc3d0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa584a52640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x55b0b87c9210 tx=0x7fa56c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da799a2d610f4a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa56a7fc640 1 -- 172.21.15.94:0/3068296088 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c004230 con 0x55b0b83ee6b0 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa56a7fc640 1 -- 172.21.15.94:0/3068296088 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c0043d0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.623+0000 7fa56a7fc640 1 -- 172.21.15.94:0/3068296088 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c010a90 con 0x55b0b83ee6b0 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- 172.21.15.94:0/3068296088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 msgr2=0x55b0b87c26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x55b0b87c9210 tx=0x7fa56c005f90 comp rx=0 tx=0).stop 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- 172.21.15.94:0/3068296088 shutdown_connections 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b840f390 0x55b0b87b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87c26c0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3068296088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b83e64d0 0x55b0b87bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- 172.21.15.94:0/3068296088 >> 172.21.15.94:0/3068296088 conn(0x55b0b84034e0 msgr2=0x55b0b87afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- 172.21.15.94:0/3068296088 shutdown_connections 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- 172.21.15.94:0/3068296088 wait complete. 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 -- start start 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.624+0000 7fa58db6b740 1 Processor -- start 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ac9b0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ac2e0 con 0x55b0b840f390 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801a9520 con 0x55b0b83e64d0 2024-04-25T14:00:56.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa58db6b740 10 client.0 _ll_register_callbacks cb 0x7fa56c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38842/0 (socket says 172.21.15.94:38842) 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 -- 172.21.15.94:0/3796803392 learned_addr learned my addr 172.21.15.94:0/3796803392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa584a52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa585253640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 -- 172.21.15.94:0/3796803392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 msgr2=0x55b0b87bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 -- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 msgr2=0x55b0b87c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 -- 172.21.15.94:0/3796803392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b87c1300 con 0x55b0b83ee6b0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa57ffff640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0124c0 tx=0x7fa56c012200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54c54e4f788592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.625+0000 7fa55ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0109d0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa56c010b70 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c0043d0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa56c010d70 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 10 client.22353 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b0b87c2620 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c0043d0 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa56c020340 con 0x55b0b83ee6b0 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 10 client.22353 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 10 client.22353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 20 client.22353 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83a33a0 0x55b0b8833220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 4 client.22353 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:56.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0b8698610 con 0x55b0b83a33a0 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.626+0000 7fa584a52640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83a33a0 0x55b0b8833220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa55ffff640 10 client.22353 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa584a52640 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83a33a0 0x55b0b8833220 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e3626527144b295 server_cookie=3cc9fd9cef2b6809 in_seq=0 out_seq=0 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa55ffff640 1 -- 172.21.15.94:0/3796803392 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0b8698610 con 0x55b0b83a33a0 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa55ffff640 10 client.22353 handle_command_reply: tid=0 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 client.22353 shutdown 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 msgr2=0x55b0b87bfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0124c0 tx=0x7fa56c012200 comp rx=0 tx=0).stop 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 shutdown_connections 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55b0b83a33a0 0x55b0b8833220 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0b840f390 0x55b0b87c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0b83ee6b0 0x55b0b87bfe10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa55ffff640 0 client.22353 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:00:56.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 --2- 172.21.15.94:0/3796803392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0b83e64d0 0x55b0b87bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 >> 172.21.15.94:0/3796803392 conn(0x55b0b84034e0 msgr2=0x55b0b87b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 shutdown_connections 2024-04-25T14:00:56.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 1 -- 172.21.15.94:0/3796803392 wait complete. 2024-04-25T14:00:56.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.627+0000 7fa58db6b740 20 client.22353 trim_cache size 0 max 16384 2024-04-25T14:00:56.634 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:56.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 msgr2=0x7fa5700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 0x7fa5700b2000 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55b0b87b2860 tx=0x7fa578009600 comp rx=0 tx=0).stop 2024-04-25T14:00:56.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 msgr2=0x7fa58006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fa574031440 tx=0x7fa574003c10 comp rx=0 tx=0).stop 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 shutdown_connections 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5700afaf0 0x7fa5700b2000 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa580108930 0x7fa5801a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa580103c30 0x7fa58019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 --2- 172.21.15.94:0/284867040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5801029f0 0x7fa58006bce0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 >> 172.21.15.94:0/284867040 conn(0x7fa5800fdc00 msgr2=0x7fa580104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 shutdown_connections 2024-04-25T14:00:56.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.628+0000 7fa586cb6640 1 -- 172.21.15.94:0/284867040 wait complete. 2024-04-25T14:00:56.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 -- 172.21.15.94:0/3019279371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818102f20 msgr2=0x7f0818103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 --2- 172.21.15.94:0/3019279371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818102f20 0x7f0818103330 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 -- 172.21.15.94:0/3019279371 shutdown_connections 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 --2- 172.21.15.94:0/3019279371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818104460 0x7f081810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 --2- 172.21.15.94:0/3019279371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f0818103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 --2- 172.21.15.94:0/3019279371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0818102f20 0x7f0818103330 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 -- 172.21.15.94:0/3019279371 >> 172.21.15.94:0/3019279371 conn(0x7f08180fd340 msgr2=0x7f08180ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 -- 172.21.15.94:0/3019279371 shutdown_connections 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.762+0000 7f081d9c5640 1 -- 172.21.15.94:0/3019279371 wait complete. 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.763+0000 7f081d9c5640 1 Processor -- start 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.763+0000 7f081d9c5640 1 -- start start 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.763+0000 7f081d9c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818102f20 0x7f0818199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.763+0000 7f081d9c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f081d9c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 0x7f08181a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f081d9c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e500 con 0x7f081819f2e0 2024-04-25T14:00:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f081d9c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e380 con 0x7f0818102f20 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f081d9c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e680 con 0x7f0818103900 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f08177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 0x7f08181a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46974/0 (socket says 172.21.15.94:46974) 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 -- 172.21.15.94:0/1882118429 learned_addr learned my addr 172.21.15.94:0/1882118429 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f0816ffd640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818102f20 0x7f0818199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 -- 172.21.15.94:0/1882118429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818102f20 msgr2=0x7f0818199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818102f20 0x7f0818199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 -- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 msgr2=0x7f08181a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 0x7f08181a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 -- 172.21.15.94:0/1882118429 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a1d70 con 0x7f0818103900 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f08177fe640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 0x7f08181a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.764+0000 7f080ffff640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f080000e9f0 tx=0x7f080000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.765+0000 7f081c9c3640 1 -- 172.21.15.94:0/1882118429 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080000cd80 con 0x7f0818103900 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.765+0000 7f081c9c3640 1 -- 172.21.15.94:0/1882118429 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08000054d0 con 0x7f0818103900 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.765+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a2060 con 0x7f0818103900 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.765+0000 7f081c9c3640 1 -- 172.21.15.94:0/1882118429 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800010690 con 0x7f0818103900 2024-04-25T14:00:56.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.765+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a2560 con 0x7f0818103900 2024-04-25T14:00:56.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 Processor -- start 2024-04-25T14:00:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 -- start start 2024-04-25T14:00:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0d1840d80 con 0x55c0d15114d0 2024-04-25T14:00:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0d17faa20 con 0x55c0d15196b0 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0824879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0d183cd40 con 0x55c0d153a390 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f081c9c3640 1 -- 172.21.15.94:0/1882118429 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0800010830 con 0x7f0818103900 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f0816ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46978/0 (socket says 172.21.15.94:46978) 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 -- 172.21.15.94:0/4260689133 learned_addr learned my addr 172.21.15.94:0/4260689133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 -- 172.21.15.94:0/4260689133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 msgr2=0x55c0d18dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 -- 172.21.15.94:0/4260689133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 msgr2=0x55c0d18dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.768+0000 7f08177fe640 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f08177fe640 1 -- 172.21.15.94:0/4260689133 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d1805c30 con 0x55c0d153a390 2024-04-25T14:00:56.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f08177fe640 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0810010750 tx=0x7f0810010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad47286780658bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f080dffb640 1 -- 172.21.15.94:0/4260689133 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000ead0 con 0x55c0d153a390 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f081c9c3640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 0x7f08040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f080dffb640 1 -- 172.21.15.94:0/4260689133 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0810010e60 con 0x55c0d153a390 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f080dffb640 1 -- 172.21.15.94:0/4260689133 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100113e0 con 0x55c0d153a390 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.769+0000 7f081c9c3640 1 -- 172.21.15.94:0/1882118429 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f08000df0b0 con 0x7f0818103900 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0816ffd640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 0x7f08040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 -- 172.21.15.94:0/4260689133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 msgr2=0x55c0d18dc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0810010750 tx=0x7f0810010c10 comp rx=0 tx=0).stop 2024-04-25T14:00:56.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0816ffd640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 0x7f08040b2000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0808005ab0 tx=0x7f0808031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 -- 172.21.15.94:0/4260689133 shutdown_connections 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 --2- 172.21.15.94:0/4260689133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18dc810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.770+0000 7f0824879740 1 -- 172.21.15.94:0/4260689133 >> 172.21.15.94:0/4260689133 conn(0x55c0d152e4e0 msgr2=0x55c0d18d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.771+0000 7f0824879740 1 -- 172.21.15.94:0/4260689133 shutdown_connections 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.771+0000 7f0824879740 1 -- 172.21.15.94:0/4260689133 wait complete. 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.771+0000 7f0824879740 1 -- start start 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.771+0000 7f0824879740 1 Processor -- start 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.771+0000 7f0824879740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0d18de530 con 0x55c0d15114d0 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0d17c7dc0 con 0x55c0d15196b0 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0d18de6b0 con 0x55c0d153a390 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0824879740 10 client.0 _ll_register_callbacks cb 0x55c0d17e3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38884/0 (socket says 172.21.15.94:38884) 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 -- 172.21.15.94:0/50802224 learned_addr learned my addr 172.21.15.94:0/50802224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f080ffff640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 -- 172.21.15.94:0/50802224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 msgr2=0x55c0d18f1f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18f1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 -- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 msgr2=0x55c0d18eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f0816ffd640 1 -- 172.21.15.94:0/50802224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d15523f0 con 0x55c0d15114d0 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.772+0000 7f080ffff640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.773+0000 7f0816ffd640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970b02de4ba9164b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.773+0000 7f07f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808035040 con 0x55c0d15114d0 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810010e60 con 0x55c0d15114d0 2024-04-25T14:00:56.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0808004300 con 0x55c0d15114d0 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f08080035f0 con 0x55c0d15114d0 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f0824879740 10 client.22365 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.774+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c0d161c7d0 con 0x55c0d15114d0 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0810010e60 con 0x55c0d15114d0 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0808044070 con 0x55c0d15114d0 2024-04-25T14:00:56.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 10 client.22365 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 10 client.22365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 20 client.22365 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14f8f60 0x55c0d18f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 4 client.22365 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d184e990 con 0x55c0d14f8f60 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.775+0000 7f080ffff640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14f8f60 0x55c0d18f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.776+0000 7f07f6ffd640 10 client.22365 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:56.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.776+0000 7f080ffff640 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14f8f60 0x55c0d18f87c0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638ca1fdbd8d3a22 server_cookie=a79f0f9f58fb9a16 in_seq=0 out_seq=0 2024-04-25T14:00:56.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f07f6ffd640 1 -- 172.21.15.94:0/50802224 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0d184e990 con 0x55c0d14f8f60 2024-04-25T14:00:56.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f07f6ffd640 10 client.22365 handle_command_reply: tid=0 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 client.22365 shutdown 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 msgr2=0x55c0d18eaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 shutdown_connections 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14f8f60 0x55c0d18f87c0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d153a390 0x55c0d18f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d15196b0 0x55c0d18eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f07f6ffd640 0 client.22365 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 --2- 172.21.15.94:0/50802224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15114d0 0x55c0d18eaaa0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 >> 172.21.15.94:0/50802224 conn(0x55c0d152e4e0 msgr2=0x55c0d18d7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 shutdown_connections 2024-04-25T14:00:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.782+0000 7f0824879740 1 -- 172.21.15.94:0/50802224 wait complete. 2024-04-25T14:00:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.783+0000 7f0824879740 20 client.22365 trim_cache size 0 max 16384 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 Processor -- start 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 -- start start 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d18ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08181a93a0 con 0x55c0d15196b0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08181aaee0 con 0x55c0d153a390 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0824879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08181a9a70 con 0x55c0d15114d0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38892/0 (socket says 172.21.15.94:38892) 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 -- 172.21.15.94:0/548187592 learned_addr learned my addr 172.21.15.94:0/548187592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f0816ffd640 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 -- 172.21.15.94:0/548187592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 msgr2=0x55c0d18e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 -- 172.21.15.94:0/548187592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 msgr2=0x55c0d18ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d18ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.801+0000 7f080ffff640 1 -- 172.21.15.94:0/548187592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d18e9af0 con 0x55c0d15196b0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.802+0000 7f080ffff640 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f08000dcb60 tx=0x7f0800014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299ea82db7ec93a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.802+0000 7f080d7fa640 1 -- 172.21.15.94:0/548187592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000a7bf0 con 0x55c0d15196b0 2024-04-25T14:00:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.802+0000 7f080d7fa640 1 -- 172.21.15.94:0/548187592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08000a74f0 con 0x55c0d15196b0 2024-04-25T14:00:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.802+0000 7f080d7fa640 1 -- 172.21.15.94:0/548187592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000a3780 con 0x55c0d15196b0 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- 172.21.15.94:0/548187592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 msgr2=0x55c0d18f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f08000dcb60 tx=0x7f0800014000 comp rx=0 tx=0).stop 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- 172.21.15.94:0/548187592 shutdown_connections 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d18ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1ed0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 --2- 172.21.15.94:0/548187592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- 172.21.15.94:0/548187592 >> 172.21.15.94:0/548187592 conn(0x55c0d152e4e0 msgr2=0x55c0d18ea670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- 172.21.15.94:0/548187592 shutdown_connections 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- 172.21.15.94:0/548187592 wait complete. 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.803+0000 7f0824879740 1 -- start start 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 Processor -- start 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18f1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d195ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08181aa810 con 0x55c0d15196b0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08181aa140 con 0x55c0d153a390 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08181a7380 con 0x55c0d15114d0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0824879740 10 client.0 _ll_register_callbacks cb 0x7f0808005e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f080ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38906/0 (socket says 172.21.15.94:38906) 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f080ffff640 1 -- 172.21.15.94:0/3361389100 learned_addr learned my addr 172.21.15.94:0/3361389100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f0816ffd640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f08177fe640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d195ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.804+0000 7f080ffff640 1 -- 172.21.15.94:0/3361389100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 msgr2=0x55c0d18f1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f080ffff640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18f1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f080ffff640 1 -- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 msgr2=0x55c0d195ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f080ffff640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d195ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f080ffff640 1 -- 172.21.15.94:0/3361389100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d195f240 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f080ffff640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f08000a6e60 tx=0x7f08000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5eeca9c48f2ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f07f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000c0070 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f08000afec0 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08000a74f0 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f08000b0140 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f0824879740 10 client.22371 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.805+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c0d1960560 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08000a74f0 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f08000a6750 con 0x55c0d15196b0 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 10 client.22371 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 10 client.22371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 20 client.22371 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14ce3a0 0x55c0d1962e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 4 client.22371 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.806+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0d18e0f00 con 0x55c0d14ce3a0 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.808+0000 7f0816ffd640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14ce3a0 0x55c0d1962e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.808+0000 7f07f6ffd640 10 client.22371 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.808+0000 7f0816ffd640 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14ce3a0 0x55c0d1962e60 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7398c73c981f4bce server_cookie=c08298a279de5ac8 in_seq=0 out_seq=0 2024-04-25T14:00:56.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.813+0000 7f07f6ffd640 1 -- 172.21.15.94:0/3361389100 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0d18e0f00 con 0x55c0d14ce3a0 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.813+0000 7f07f6ffd640 10 client.22371 handle_command_reply: tid=0 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.813+0000 7f0824879740 1 client.22371 shutdown 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.813+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 msgr2=0x55c0d18f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.813+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f08000a6e60 tx=0x7f08000048f0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 shutdown_connections 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55c0d14ce3a0 0x55c0d1962e60 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0d153a390 0x55c0d195ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f07f6ffd640 0 client.22371 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0d15196b0 0x55c0d18f1640 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 --2- 172.21.15.94:0/3361389100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0d15114d0 0x55c0d18f1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 >> 172.21.15.94:0/3361389100 conn(0x55c0d152e4e0 msgr2=0x55c0d18dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 shutdown_connections 2024-04-25T14:00:56.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 1 -- 172.21.15.94:0/3361389100 wait complete. 2024-04-25T14:00:56.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f0824879740 20 client.22371 trim_cache size 0 max 16384 2024-04-25T14:00:56.815 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:00:56.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 msgr2=0x7f08040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 0x7f08040b2000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0808005ab0 tx=0x7f0808031400 comp rx=0 tx=0).stop 2024-04-25T14:00:56.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 msgr2=0x7f081819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:56.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f080000e9f0 tx=0x7f080000edf0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 shutdown_connections 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f08040afaf0 0x7f08040b2000 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f081819f2e0 0x7f08181a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0818103900 0x7f081819a360 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 --2- 172.21.15.94:0/1882118429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0818102f20 0x7f0818199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 >> 172.21.15.94:0/1882118429 conn(0x7f08180fd340 msgr2=0x7f081810a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 shutdown_connections 2024-04-25T14:00:56.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:56.814+0000 7f081d9c5640 1 -- 172.21.15.94:0/1882118429 wait complete. 2024-04-25T14:00:57.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 122347 19.14 MB/sec execute 219 sec latency 110.305 ms 2024-04-25T14:00:57.826 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:00:57.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:00:57.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:00:57.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.944+0000 7fddacaf9640 1 -- 172.21.15.94:0/263672644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8101740 msgr2=0x7fdda8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.944+0000 7fddacaf9640 1 --2- 172.21.15.94:0/263672644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8101740 0x7fdda8101bb0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009900 tx=0x7fdd9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:00:57.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 -- 172.21.15.94:0/263672644 shutdown_connections 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 --2- 172.21.15.94:0/263672644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8102240 0x7fdda810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 --2- 172.21.15.94:0/263672644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8101740 0x7fdda8101bb0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 --2- 172.21.15.94:0/263672644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8105090 0x7fdda81054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 -- 172.21.15.94:0/263672644 >> 172.21.15.94:0/263672644 conn(0x7fdda80fd340 msgr2=0x7fdda80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 -- 172.21.15.94:0/263672644 shutdown_connections 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 -- 172.21.15.94:0/263672644 wait complete. 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 Processor -- start 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.945+0000 7fddacaf9640 1 -- start start 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 0x7fdda819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8102240 0x7fdda819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810f620 con 0x7fdda8105090 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810f4a0 con 0x7fdda8101740 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fddacaf9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810f7a0 con 0x7fdda8102240 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38932/0 (socket says 172.21.15.94:38932) 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 0x7fdda819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 -- 172.21.15.94:0/3318209674 learned_addr learned my addr 172.21.15.94:0/3318209674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8102240 0x7fdda819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 -- 172.21.15.94:0/3318209674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8102240 msgr2=0x7fdda819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8102240 0x7fdda819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 -- 172.21.15.94:0/3318209674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 msgr2=0x7fdda819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 0x7fdda819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 -- 172.21.15.94:0/3318209674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda81a1da0 con 0x7fdda8105090 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda659c640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 0x7fdda819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.946+0000 7fdda6d9d640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fdda000eb10 tx=0x7fdda000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.947+0000 7fdd977fe640 1 -- 172.21.15.94:0/3318209674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0004b20 con 0x7fdda8105090 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.947+0000 7fdd977fe640 1 -- 172.21.15.94:0/3318209674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda0004cc0 con 0x7fdda8105090 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.947+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda81a2090 con 0x7fdda8105090 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.947+0000 7fdd977fe640 1 -- 172.21.15.94:0/3318209674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0010670 con 0x7fdda8105090 2024-04-25T14:00:57.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.947+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda81a2590 con 0x7fdda8105090 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.950+0000 7fddb39ae740 1 Processor -- start 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.950+0000 7fdd977fe640 1 -- 172.21.15.94:0/3318209674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdda0010830 con 0x7fdda8105090 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.950+0000 7fddb39ae740 1 -- start start 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.950+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb090ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb090ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fddb39ae740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fb09064b00 con 0x55fb08d5e410 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fddb39ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fb09000920 con 0x55fb08d35550 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fddb39ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb09060ac0 con 0x55fb08d3d730 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb090ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb090ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38940/0 (socket says 172.21.15.94:38940) 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda5d9b640 1 -- 172.21.15.94:0/4131016910 learned_addr learned my addr 172.21.15.94:0/4131016910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb090ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdd977fe640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 0x7fdd900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 -- 172.21.15.94:0/4131016910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 msgr2=0x55fb09106920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09106920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 -- 172.21.15.94:0/4131016910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 msgr2=0x55fb090ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb090ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 -- 172.21.15.94:0/4131016910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb08e3e7e0 con 0x55fb08d35550 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdd977fe640 1 -- 172.21.15.94:0/3318209674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdda00df260 con 0x7fdda8105090 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda659c640 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb090ffa20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdda8069dc0 tx=0x7fdd980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1d2b1a637db4af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.951+0000 7fdda6d9d640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 0x7fdd900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.952+0000 7fdd94ff9640 1 -- 172.21.15.94:0/4131016910 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004030 con 0x55fb08d35550 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.952+0000 7fdd94ff9640 1 -- 172.21.15.94:0/4131016910 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd980041d0 con 0x55fb08d35550 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.952+0000 7fdd94ff9640 1 -- 172.21.15.94:0/4131016910 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800f740 con 0x55fb08d35550 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 -- 172.21.15.94:0/4131016910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 msgr2=0x55fb090ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fdda6d9d640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 0x7fdd900b2000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdda00dd030 tx=0x7fdda0014000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb090ffa20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdda8069dc0 tx=0x7fdd980118d0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 -- 172.21.15.94:0/4131016910 shutdown_connections 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb090ffa20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 --2- 172.21.15.94:0/4131016910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb090ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 -- 172.21.15.94:0/4131016910 >> 172.21.15.94:0/4131016910 conn(0x55fb08d52560 msgr2=0x55fb090fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 -- 172.21.15.94:0/4131016910 shutdown_connections 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.953+0000 7fddb39ae740 1 -- 172.21.15.94:0/4131016910 wait complete. 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 -- start start 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 Processor -- start 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d5e410 0x55fb09110740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fb09107370 con 0x55fb08d3d730 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fb08ffe5d0 con 0x55fb08d35550 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb091074f0 con 0x55fb08d5e410 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fddb39ae740 10 client.0 _ll_register_callbacks cb 0x55fb08fec080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38954/0 (socket says 172.21.15.94:38954) 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36928/0 (socket says 172.21.15.94:36928) 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda659c640 1 -- 172.21.15.94:0/2392908879 learned_addr learned my addr 172.21.15.94:0/2392908879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:57.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d5e410 0x55fb09110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2392908879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d5e410 msgr2=0x55fb09110740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d5e410 0x55fb09110740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2392908879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 msgr2=0x55fb091097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2392908879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb08e9ef70 con 0x55fb08d35550 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.954+0000 7fdda659c640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c031440 tx=0x7fdd9c009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=339e90f990064cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c03f030 con 0x55fb08d35550 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdd9c004020 con 0x55fb08d35550 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd980041d0 con 0x55fb08d35550 2024-04-25T14:00:57.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c035070 con 0x55fb08d35550 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fddb39ae740 10 client.31805 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb08d36170 con 0x55fb08d35550 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd980041d0 con 0x55fb08d35550 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdd9c03a5d0 con 0x55fb08d35550 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fddb39ae740 10 client.31805 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.955+0000 7fddb39ae740 10 client.31805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fddb39ae740 20 client.31805 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08d1d1f0 0x55fb09116ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fddb39ae740 4 client.31805 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb09072710 con 0x55fb08d1d1f0 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fdda659c640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08d1d1f0 0x55fb09116ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fdd8dffb640 10 client.31805 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.956+0000 7fdda659c640 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08d1d1f0 0x55fb09116ae0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadcc223fb2c5ddd server_cookie=57f80ce15c4c7e84 in_seq=0 out_seq=0 2024-04-25T14:00:57.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fdd8dffb640 1 -- 172.21.15.94:0/2392908879 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb09072710 con 0x55fb08d1d1f0 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fdd8dffb640 10 client.31805 handle_command_reply: tid=0 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 client.31805 shutdown 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 msgr2=0x55fb091092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c031440 tx=0x7fdd9c009620 comp rx=0 tx=0).stop 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 shutdown_connections 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08d1d1f0 0x55fb09116ae0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d5e410 0x55fb09110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb091097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2392908879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb091092a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fdd8dffb640 0 client.31805 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 >> 172.21.15.94:0/2392908879 conn(0x55fb08d52560 msgr2=0x55fb090f9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 shutdown_connections 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.959+0000 7fddb39ae740 1 -- 172.21.15.94:0/2392908879 wait complete. 2024-04-25T14:00:57.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.960+0000 7fddb39ae740 20 client.31805 trim_cache size 0 max 16384 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 Processor -- start 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 -- start start 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb09103d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09109580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81a9340 con 0x55fb08d5e410 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81aae80 con 0x55fb08d35550 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fddb39ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81a9a10 con 0x55fb08d3d730 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38958/0 (socket says 172.21.15.94:38958) 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2648566352 learned_addr learned my addr 172.21.15.94:0/2648566352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb09103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2648566352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 msgr2=0x55fb09109580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09109580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2648566352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 msgr2=0x55fb09103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb09103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.985+0000 7fdda5d9b640 1 -- 172.21.15.94:0/2648566352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb090fb9c0 con 0x55fb08d5e410 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.986+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c03afd0 tx=0x7fdd9c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c12ca0f1ad569f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.986+0000 7fdd8ffff640 1 -- 172.21.15.94:0/2648566352 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c03f070 con 0x55fb08d5e410 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.986+0000 7fdd8ffff640 1 -- 172.21.15.94:0/2648566352 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9c05c040 con 0x55fb08d5e410 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.986+0000 7fdd8ffff640 1 -- 172.21.15.94:0/2648566352 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c05b040 con 0x55fb08d5e410 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- 172.21.15.94:0/2648566352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 msgr2=0x55fb0911b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c03afd0 tx=0x7fdd9c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- 172.21.15.94:0/2648566352 shutdown_connections 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d5e410 0x55fb0911b260 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d3d730 0x55fb09109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- 172.21.15.94:0/2648566352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d35550 0x55fb09103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- 172.21.15.94:0/2648566352 >> 172.21.15.94:0/2648566352 conn(0x55fb08d52560 msgr2=0x55fb09108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- 172.21.15.94:0/2648566352 shutdown_connections 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- 172.21.15.94:0/2648566352 wait complete. 2024-04-25T14:00:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- start start 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 Processor -- start 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d35550 0x55fb0910d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d5e410 0x55fb0910e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81aa7b0 con 0x55fb08d3d730 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81aa0e0 con 0x55fb08d5e410 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.987+0000 7fddb39ae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81a7320 con 0x55fb08d35550 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fddb39ae740 10 client.0 _ll_register_callbacks cb 0x7fdd9c031770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:57.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38964/0 (socket says 172.21.15.94:38964) 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 -- 172.21.15.94:0/3105745062 learned_addr learned my addr 172.21.15.94:0/3105745062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d35550 0x55fb0910d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d5e410 0x55fb0910e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 -- 172.21.15.94:0/3105745062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d35550 msgr2=0x55fb0910d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d35550 0x55fb0910d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 -- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d5e410 msgr2=0x55fb0910e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d5e410 0x55fb0910e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 -- 172.21.15.94:0/3105745062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb0910ed60 con 0x55fb08d3d730 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdda659c640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009250 tx=0x7fdd98016330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee9d33ad6af03e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:57.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdd8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98005040 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdd98016040 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9c05c040 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdd9803e050 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fddb39ae740 10 client.22389 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.988+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fb09110090 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9c05c040 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdd980035f0 con 0x55fb08d3d730 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 10 client.22389 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 10 client.22389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 20 client.22389 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:57.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08cf2630 0x55fb09181410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 4 client.22389 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:00:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb0906c580 con 0x55fb08cf2630 2024-04-25T14:00:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08cf2630 0x55fb09181410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fdd8dffb640 10 client.22389 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:57.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.989+0000 7fdda5d9b640 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08cf2630 0x55fb09181410 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d428f64c4b5757 server_cookie=43048bed32ec04d9 in_seq=0 out_seq=0 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fdd8dffb640 1 -- 172.21.15.94:0/3105745062 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb0906c580 con 0x55fb08cf2630 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fdd8dffb640 10 client.22389 handle_command_reply: tid=0 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 client.22389 shutdown 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 msgr2=0x55fb0910d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009250 tx=0x7fdd98016330 comp rx=0 tx=0).stop 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 shutdown_connections 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55fb08cf2630 0x55fb09181410 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb08d5e410 0x55fb0910e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb08d3d730 0x55fb0910d8e0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fdd8dffb640 0 client.22389 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 --2- 172.21.15.94:0/3105745062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb08d35550 0x55fb0910d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 >> 172.21.15.94:0/3105745062 conn(0x55fb08d52560 msgr2=0x55fb090fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 shutdown_connections 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 1 -- 172.21.15.94:0/3105745062 wait complete. 2024-04-25T14:00:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.994+0000 7fddb39ae740 20 client.22389 trim_cache size 0 max 16384 2024-04-25T14:00:57.999 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:57.999 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:00:57.999 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "15e690a7-8875-4aec-94f3-61181f2e19b8", 2024-04-25T14:00:57.999 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:00:57.999 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 msgr2=0x7fdd900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 0x7fdd900b2000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdda00dd030 tx=0x7fdda0014000 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 msgr2=0x7fdda81a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fdda000eb10 tx=0x7fdda000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 shutdown_connections 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdd900afaf0 0x7fdd900b2000 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdda8105090 0x7fdda81a1730 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda8102240 0x7fdda819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 --2- 172.21.15.94:0/3318209674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdda8101740 0x7fdda819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 >> 172.21.15.94:0/3318209674 conn(0x7fdda80fd340 msgr2=0x7fdda8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 shutdown_connections 2024-04-25T14:00:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:57.995+0000 7fddacaf9640 1 -- 172.21.15.94:0/3318209674 wait complete. 2024-04-25T14:00:58.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:58.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.128+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3774997469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 msgr2=0x7fc2c010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.128+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3774997469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c010ad40 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009ca0 tx=0x7fc2b802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3774997469 shutdown_connections 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3774997469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c010ad40 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3774997469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 0x7fc2c0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3774997469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3774997469 >> 172.21.15.94:0/3774997469 conn(0x7fc2c00fdbd0 msgr2=0x7fc2c0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3774997469 shutdown_connections 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3774997469 wait complete. 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 Processor -- start 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- start start 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 0x7fc2c019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c010e510 con 0x7fc2c0108920 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c010e390 con 0x7fc2c0103c20 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.129+0000 7fc2c83a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c010e690 con 0x7fc2c01029e0 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6945640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6945640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38982/0 (socket says 172.21.15.94:38982) 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47070/0 (socket says 172.21.15.94:47070) 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6945640 1 -- 172.21.15.94:0/3014753658 learned_addr learned my addr 172.21.15.94:0/3014753658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 0x7fc2c019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 -- 172.21.15.94:0/3014753658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 msgr2=0x7fc2c019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 0x7fc2c019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 -- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 msgr2=0x7fc2c01a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 -- 172.21.15.94:0/3014753658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c01a2030 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6945640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.130+0000 7fc2c6144640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2c01022b0 tx=0x7fc2bc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.131+0000 7fc2b77fe640 1 -- 172.21.15.94:0/3014753658 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00cc40 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.131+0000 7fc2b77fe640 1 -- 172.21.15.94:0/3014753658 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc00cde0 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.131+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c01a2320 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.131+0000 7fc2b77fe640 1 -- 172.21.15.94:0/3014753658 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc010430 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.131+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c01a2820 con 0x7fc2c01029e0 2024-04-25T14:00:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.133+0000 7fc2b77fe640 1 -- 172.21.15.94:0/3014753658 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc2bc003680 con 0x7fc2c01029e0 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.134+0000 7fc2cf252740 1 Processor -- start 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.134+0000 7fc2cf252740 1 -- start start 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2b77fe640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 0x7fc2ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 0x56310bd91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56310bcf6d80 con 0x56310b9f0390 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56310bcb08b0 con 0x56310b9cf6b0 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2cf252740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56310bcf2d40 con 0x56310b9c74d0 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 0x56310bd91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6945640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 0x7fc2ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 0x56310bd91fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47084/0 (socket says 172.21.15.94:47084) 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38986/0 (socket says 172.21.15.94:38986) 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c5943640 1 -- 172.21.15.94:0/2628471911 learned_addr learned my addr 172.21.15.94:0/2628471911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6945640 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 -- 172.21.15.94:0/2628471911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 msgr2=0x56310bd91fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 0x56310bd91fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 -- 172.21.15.94:0/2628471911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 msgr2=0x56310bd92c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd92c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2c6144640 1 -- 172.21.15.94:0/2628471911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56310bc81a40 con 0x56310b9f0390 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.135+0000 7fc2b77fe640 1 -- 172.21.15.94:0/3014753658 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2bc0defe0 con 0x7fc2c01029e0 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.136+0000 7fc2c6945640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 0x7fc2ac0b2000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b802f400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.136+0000 7fc2c6144640 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a4670 tx=0x7fc2bc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42624dbd7e6a5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.136+0000 7fc2b4bfe640 1 -- 172.21.15.94:0/2628471911 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0a6f10 con 0x56310b9f0390 2024-04-25T14:00:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.136+0000 7fc2b4bfe640 1 -- 172.21.15.94:0/2628471911 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0a7b00 con 0x56310b9f0390 2024-04-25T14:00:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.136+0000 7fc2b4bfe640 1 -- 172.21.15.94:0/2628471911 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0a7400 con 0x56310b9f0390 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 -- 172.21.15.94:0/2628471911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 msgr2=0x56310bd91a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a4670 tx=0x7fc2bc021000 comp rx=0 tx=0).stop 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 -- 172.21.15.94:0/2628471911 shutdown_connections 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd92c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9c74d0 0x56310bd91fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 --2- 172.21.15.94:0/2628471911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd91a60 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 -- 172.21.15.94:0/2628471911 >> 172.21.15.94:0/2628471911 conn(0x56310b9e44e0 msgr2=0x56310bd8d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 -- 172.21.15.94:0/2628471911 shutdown_connections 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.137+0000 7fc2cf252740 1 -- 172.21.15.94:0/2628471911 wait complete. 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 -- start start 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 Processor -- start 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bda2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56310bd93780 con 0x56310b9c74d0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56310bc7dd10 con 0x56310b9cf6b0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56310bd93900 con 0x56310b9f0390 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2cf252740 10 client.0 _ll_register_callbacks cb 0x56310bcaa210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2c6945640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bda2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36972/0 (socket says 172.21.15.94:36972) 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2c5943640 1 -- 172.21.15.94:0/1390380923 learned_addr learned my addr 172.21.15.94:0/1390380923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.138+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 -- 172.21.15.94:0/1390380923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 msgr2=0x56310bda2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bda2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 -- 172.21.15.94:0/1390380923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 msgr2=0x56310bd9b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 -- 172.21.15.94:0/1390380923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56310bad27d0 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c6945640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bda2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2c5943640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009fd0 tx=0x7fc2b0012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f379a863a2ae04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000f920 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc0a7b00 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.139+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2b000fac0 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2b000fcc0 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 10 client.31823 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56310ba083f0 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0a7b00 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc2b001c420 con 0x56310b9cf6b0 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 10 client.31823 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:58.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 10 client.31823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 20 client.31823 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9aef60 0x56310bda9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 4 client.31823 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56310bd04990 con 0x56310b9aef60 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.140+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9aef60 0x56310bda9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.141+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9aef60 0x56310bda9360 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ee1e3ede4f5ea4 server_cookie=775bc586f69cf0b3 in_seq=0 out_seq=0 2024-04-25T14:00:58.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.141+0000 7fc2a9ffb640 10 client.31823 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1390380923 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56310bd04990 con 0x56310b9aef60 2024-04-25T14:00:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2a9ffb640 10 client.31823 handle_command_reply: tid=0 2024-04-25T14:00:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 client.31823 shutdown 2024-04-25T14:00:58.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 msgr2=0x56310bd9be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009fd0 tx=0x7fc2b0012bd0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 shutdown_connections 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9aef60 0x56310bda9360 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bda2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bd9be50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2a9ffb640 0 client.31823 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1390380923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 >> 172.21.15.94:0/1390380923 conn(0x56310b9e44e0 msgr2=0x56310bd90180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 shutdown_connections 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 1 -- 172.21.15.94:0/1390380923 wait complete. 2024-04-25T14:00:58.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.144+0000 7fc2cf252740 20 client.31823 trim_cache size 0 max 16384 2024-04-25T14:00:58.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 122628 19.13 MB/sec execute 220 sec latency 199.015 ms 2024-04-25T14:00:58.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 Processor -- start 2024-04-25T14:00:58.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 -- start start 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bda2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bd8cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ab500 con 0x56310b9c74d0 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ad040 con 0x56310b9cf6b0 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2cf252740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01abbd0 con 0x56310b9f0390 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39008/0 (socket says 172.21.15.94:39008) 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.165+0000 7fc2c6144640 1 -- 172.21.15.94:0/1018709335 learned_addr learned my addr 172.21.15.94:0/1018709335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6945640 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bd8cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c5943640 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bda2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 -- 172.21.15.94:0/1018709335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 msgr2=0x56310bd8cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bd8cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 -- 172.21.15.94:0/1018709335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 msgr2=0x56310bda2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bda2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 -- 172.21.15.94:0/1018709335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56310b9c80f0 con 0x56310b9c74d0 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a4670 tx=0x7fc2bc016ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880478f79b33f51b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2abfff640 1 -- 172.21.15.94:0/1018709335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0a32c0 con 0x56310b9c74d0 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2abfff640 1 -- 172.21.15.94:0/1018709335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0a7b00 con 0x56310b9c74d0 2024-04-25T14:00:58.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.166+0000 7fc2abfff640 1 -- 172.21.15.94:0/1018709335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0afdd0 con 0x56310b9c74d0 2024-04-25T14:00:58.369 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:57 smithi138 ceph-mon[32977]: pgmap v926: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 488 op/s 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 -- 172.21.15.94:0/1018709335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 msgr2=0x56310bd9a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a4670 tx=0x7fc2bc016ea0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 -- 172.21.15.94:0/1018709335 shutdown_connections 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9f0390 0x56310bd8cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9cf6b0 0x56310bda2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1018709335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9c74d0 0x56310bd9a610 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 -- 172.21.15.94:0/1018709335 >> 172.21.15.94:0/1018709335 conn(0x56310b9e44e0 msgr2=0x56310bd99f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 -- 172.21.15.94:0/1018709335 shutdown_connections 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.167+0000 7fc2cf252740 1 -- 172.21.15.94:0/1018709335 wait complete. 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 -- start start 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 Processor -- start 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 0x56310bda2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ac970 con 0x56310b9f0390 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01ac2a0 con 0x56310b9c74d0 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01a94e0 con 0x56310b9cf6b0 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2cf252740 10 client.0 _ll_register_callbacks cb 0x7fc2b0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6144640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 0x56310bda2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c5943640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47116/0 (socket says 172.21.15.94:47116) 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39018/0 (socket says 172.21.15.94:39018) 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c5943640 1 -- 172.21.15.94:0/1588514738 learned_addr learned my addr 172.21.15.94:0/1588514738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 -- 172.21.15.94:0/1588514738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 msgr2=0x56310bd9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 -- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 msgr2=0x56310bda2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 0x56310bda2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6945640 1 -- 172.21.15.94:0/1588514738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56310bcb74b0 con 0x56310b9f0390 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 0x56310bda2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.168+0000 7fc2c5943640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2c6945640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80094e0 tx=0x7fc2b8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bdac129fe2e98e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b805a060 con 0x56310b9f0390 2024-04-25T14:00:58.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2b8046070 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc0a7b00 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2b802f660 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 10 client.22407 fetch_fsmap learned FSMap version 39 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56310be10880 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0a7b00 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc2b80308a0 con 0x56310b9f0390 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 10 client.22407 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 10 client.22407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 20 client.22407 populate_metadata read hostname 'smithi094' 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9843a0 0x56310be132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 4 client.22407 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.169+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56310bcfe800 con 0x56310b9843a0 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.170+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9843a0 0x56310be132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:58.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.170+0000 7fc2a9ffb640 10 client.22407 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.170+0000 7fc2c6144640 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9843a0 0x56310be132a0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e3ff1d1d7559b server_cookie=f4fd6e100097ce0f in_seq=0 out_seq=0 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2a9ffb640 1 -- 172.21.15.94:0/1588514738 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56310bcfe800 con 0x56310b9843a0 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2a9ffb640 10 client.22407 handle_command_reply: tid=0 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 client.22407 shutdown 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 msgr2=0x56310bd9fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80094e0 tx=0x7fc2b8031af0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 shutdown_connections 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56310b9843a0 0x56310be132a0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56310b9f0390 0x56310bd9fe80 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56310b9cf6b0 0x56310bd9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2a9ffb640 0 client.22407 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 --2- 172.21.15.94:0/1588514738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56310b9c74d0 0x56310bda2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 >> 172.21.15.94:0/1588514738 conn(0x56310b9e44e0 msgr2=0x56310bd8cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 shutdown_connections 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.173+0000 7fc2cf252740 1 -- 172.21.15.94:0/1588514738 wait complete. 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2cf252740 20 client.22407 trim_cache size 0 max 16384 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (370 inodes in the stack)", 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "15e690a7-8875-4aec-94f3-61181f2e19b8": { 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "15e690a7-8875-4aec-94f3-61181f2e19b8", 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:00:58.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:00:58.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:00:58.380 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:00:58.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 msgr2=0x7fc2ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 0x7fc2ac0b2000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b802f400 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 msgr2=0x7fc2c006a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2c01022b0 tx=0x7fc2bc00ee50 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 shutdown_connections 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2ac0afaf0 0x7fc2ac0b2000 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2c0108920 0x7fc2c01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c0103c20 0x7fc2c019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 --2- 172.21.15.94:0/3014753658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2c01029e0 0x7fc2c006a900 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 >> 172.21.15.94:0/3014753658 conn(0x7fc2c00fdbd0 msgr2=0x7fc2c0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 shutdown_connections 2024-04-25T14:00:58.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:00:58.174+0000 7fc2c83a8640 1 -- 172.21.15.94:0/3014753658 wait complete. 2024-04-25T14:00:58.382 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:57 smithi094 ceph-mon[29195]: pgmap v926: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 488 op/s 2024-04-25T14:00:58.382 INFO:tasks.cephfs.filesystem:scrub status for tag:15e690a7-8875-4aec-94f3-61181f2e19b8 - {'path': '/', 'tag': '15e690a7-8875-4aec-94f3-61181f2e19b8', 'options': 'recursive,force'} 2024-04-25T14:00:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:57 smithi137 ceph-mon[32568]: pgmap v926: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 488 op/s 2024-04-25T14:00:59.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 123001 19.13 MB/sec execute 221 sec latency 103.429 ms 2024-04-25T14:00:59.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:00:59 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:00:59.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:00:59.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:00:59 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:59.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:00:59 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:00.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 123438 19.13 MB/sec execute 222 sec latency 130.428 ms 2024-04-25T14:01:00.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:00 smithi137 ceph-mon[32568]: pgmap v927: 97 pgs: 97 active+clean; 260 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 493 op/s 2024-04-25T14:01:00.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:00 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:00.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:00 smithi094 ceph-mon[29195]: pgmap v927: 97 pgs: 97 active+clean; 260 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 493 op/s 2024-04-25T14:01:00.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:00 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:00 smithi138 ceph-mon[32977]: pgmap v927: 97 pgs: 97 active+clean; 260 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 493 op/s 2024-04-25T14:01:00.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:00 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:01.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 123976 19.15 MB/sec execute 223 sec latency 59.001 ms 2024-04-25T14:01:02.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:02 smithi137 ceph-mon[32568]: pgmap v928: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-04-25T14:01:02.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 124427 19.22 MB/sec execute 224 sec latency 74.541 ms 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 -- 172.21.15.94:0/1693065808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1741046d0 msgr2=0x7fa17410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 --2- 172.21.15.94:0/1693065808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1741046d0 0x7fa17410ae20 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009ca0 tx=0x7fa16c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 -- 172.21.15.94:0/1693065808 shutdown_connections 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 --2- 172.21.15.94:0/1693065808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1741046d0 0x7fa17410ae20 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 --2- 172.21.15.94:0/1693065808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 0x7fa174104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 --2- 172.21.15.94:0/1693065808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa174103010 0x7fa174103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 -- 172.21.15.94:0/1693065808 >> 172.21.15.94:0/1693065808 conn(0x7fa1740fd390 msgr2=0x7fa1740ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 -- 172.21.15.94:0/1693065808 shutdown_connections 2024-04-25T14:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 -- 172.21.15.94:0/1693065808 wait complete. 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.378+0000 7fa17b842640 1 Processor -- start 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 -- start start 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 0x7fa174199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 0x7fa1741a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410e7a0 con 0x7fa174103010 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410e620 con 0x7fa174103b70 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa17410e920 con 0x7fa17419e3c0 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39038/0 (socket says 172.21.15.94:39038) 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 -- 172.21.15.94:0/3378437721 learned_addr learned my addr 172.21.15.94:0/3378437721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa178ddd640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 0x7fa174199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 0x7fa1741a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 -- 172.21.15.94:0/3378437721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 msgr2=0x7fa1741a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 0x7fa1741a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 -- 172.21.15.94:0/3378437721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 msgr2=0x7fa174199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 0x7fa174199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 -- 172.21.15.94:0/3378437721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1741a0e50 con 0x7fa174103010 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 0x7fa1741a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa1795de640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e9f0 tx=0x7fa17000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa16a7fc640 1 -- 172.21.15.94:0/3378437721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000ccc0 con 0x7fa174103010 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa16a7fc640 1 -- 172.21.15.94:0/3378437721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1700054d0 con 0x7fa174103010 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.379+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1741a1140 con 0x7fa174103010 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.380+0000 7fa16a7fc640 1 -- 172.21.15.94:0/3378437721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170010430 con 0x7fa174103010 2024-04-25T14:01:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.380+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a1640 con 0x7fa174103010 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.382+0000 7fa16a7fc640 1 -- 172.21.15.94:0/3378437721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa1700046c0 con 0x7fa174103010 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.382+0000 7fa1827d0740 1 Processor -- start 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.382+0000 7fa1827d0740 1 -- start start 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657761970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646576c7460 con 0x5646573a04d0 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564657681590 con 0x5646573c11b0 2024-04-25T14:01:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1827d0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646576c3420 con 0x5646573982f0 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa178ddd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa179ddf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657761970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47144/0 (socket says 172.21.15.94:47144) 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa178ddd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39040/0 (socket says 172.21.15.94:39040) 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 -- 172.21.15.94:0/607034356 learned_addr learned my addr 172.21.15.94:0/607034356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa16a7fc640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 0x7fa1600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 -- 172.21.15.94:0/607034356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 msgr2=0x564657761970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657761970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 -- 172.21.15.94:0/607034356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 msgr2=0x564657768df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 -- 172.21.15.94:0/607034356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465768e0d0 con 0x5646573982f0 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa179ddf640 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657761970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa178ddd640 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 0x7fa1600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa16a7fc640 1 -- 172.21.15.94:0/3378437721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa1700df0d0 con 0x7fa174103010 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.383+0000 7fa1795de640 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa170049a50 tx=0x7fa170021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc7fe5608f8e967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa15f7fe640 1 -- 172.21.15.94:0/607034356 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700a72e0 con 0x5646573982f0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa15f7fe640 1 -- 172.21.15.94:0/607034356 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa170049a80 con 0x5646573982f0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa15f7fe640 1 -- 172.21.15.94:0/607034356 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170013190 con 0x5646573982f0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa1827d0740 1 -- 172.21.15.94:0/607034356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 msgr2=0x564657761ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 0x7fa1600b1ff0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x564657762b40 tx=0x7fa16c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa1827d0740 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa170049a50 tx=0x7fa170021000 comp rx=0 tx=0).stop 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa1827d0740 1 -- 172.21.15.94:0/607034356 shutdown_connections 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa1827d0740 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573a04d0 0x564657768df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.384+0000 7fa1827d0740 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573982f0 0x564657761ef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 --2- 172.21.15.94:0/607034356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657761970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- 172.21.15.94:0/607034356 >> 172.21.15.94:0/607034356 conn(0x5646573b5300 msgr2=0x56465775d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- 172.21.15.94:0/607034356 shutdown_connections 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- 172.21.15.94:0/607034356 wait complete. 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- start start 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 Processor -- start 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573982f0 0x56465776d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 0x564657774c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646577697e0 con 0x5646573c11b0 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56465764eb60 con 0x5646573982f0 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564657769960 con 0x5646573a04d0 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa1827d0740 10 client.0 _ll_register_callbacks cb 0x5646576732b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa179ddf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 0x564657774c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.385+0000 7fa178ddd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47158/0 (socket says 172.21.15.94:47158) 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 -- 172.21.15.94:0/3854434166 learned_addr learned my addr 172.21.15.94:0/3854434166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa1795de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573982f0 0x56465776d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 -- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573982f0 msgr2=0x56465776d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573982f0 0x56465776d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 -- 172.21.15.94:0/3854434166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 msgr2=0x564657774c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 0x564657774c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 -- 172.21.15.94:0/3854434166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646573d9210 con 0x5646573a04d0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 0x564657774c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa178ddd640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa164012720 tx=0x7fa164012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9117872cfafb7a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa15cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164004030 con 0x5646573a04d0 2024-04-25T14:01:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa1640041d0 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646576d5070 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa164004410 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa1827d0740 10 client.31483 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5646574a5620 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.386+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646576d5070 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa164003b80 con 0x5646573a04d0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 10 client.31483 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 10 client.31483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 20 client.31483 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56465737fd80 0x56465777b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 4 client.31483 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646576ceee0 con 0x56465737fd80 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1795de640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56465737fd80 0x56465777b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa15cff9640 10 client.31483 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.387+0000 7fa1795de640 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56465737fd80 0x56465777b510 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acb5df4cfdff225 server_cookie=2b44e1f141885258 in_seq=0 out_seq=0 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa15cff9640 1 -- 172.21.15.94:0/3854434166 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646576ceee0 con 0x56465737fd80 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa15cff9640 10 client.31483 handle_command_reply: tid=0 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 client.31483 shutdown 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 msgr2=0x56465776dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa164012720 tx=0x7fa164012be0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 shutdown_connections 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56465737fd80 0x56465777b510 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573c11b0 0x564657774c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x56465776dd00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3854434166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573982f0 0x56465776d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa15cff9640 0 client.31483 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 >> 172.21.15.94:0/3854434166 conn(0x5646573b5300 msgr2=0x564657760210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 shutdown_connections 2024-04-25T14:01:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.391+0000 7fa1827d0740 1 -- 172.21.15.94:0/3854434166 wait complete. 2024-04-25T14:01:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.392+0000 7fa1827d0740 20 client.31483 trim_cache size 0 max 16384 2024-04-25T14:01:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 Processor -- start 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 -- start start 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657765f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657774b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741a92e0 con 0x5646573982f0 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741aae20 con 0x5646573c11b0 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1827d0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741a99b0 con 0x5646573a04d0 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39064/0 (socket says 172.21.15.94:39064) 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 -- 172.21.15.94:0/3989064482 learned_addr learned my addr 172.21.15.94:0/3989064482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa178ddd640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657765f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657774b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 -- 172.21.15.94:0/3989064482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 msgr2=0x564657765f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657765f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 -- 172.21.15.94:0/3989064482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 msgr2=0x564657774b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657774b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 -- 172.21.15.94:0/3989064482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646577750c0 con 0x5646573982f0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa179ddf640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657774b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.407+0000 7fa1795de640 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fa1700dcb80 tx=0x7fa1700e2bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af75181923308364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa15effd640 1 -- 172.21.15.94:0/3989064482 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700a6c80 con 0x5646573982f0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa15effd640 1 -- 172.21.15.94:0/3989064482 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1700a7020 con 0x5646573982f0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa15effd640 1 -- 172.21.15.94:0/3989064482 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700a6e60 con 0x5646573982f0 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 -- 172.21.15.94:0/3989064482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 msgr2=0x564657765a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fa1700dcb80 tx=0x7fa1700e2bb0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 -- 172.21.15.94:0/3989064482 shutdown_connections 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x564657774b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657765f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 --2- 172.21.15.94:0/3989064482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657765a20 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.408+0000 7fa1827d0740 1 -- 172.21.15.94:0/3989064482 >> 172.21.15.94:0/3989064482 conn(0x5646573b5300 msgr2=0x5646577618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- 172.21.15.94:0/3989064482 shutdown_connections 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- 172.21.15.94:0/3989064482 wait complete. 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- start start 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 Processor -- start 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657772130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x5646577e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741aa750 con 0x5646573982f0 2024-04-25T14:01:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741aa080 con 0x5646573c11b0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741a72c0 con 0x5646573a04d0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1827d0740 10 client.0 _ll_register_callbacks cb 0x7fa164009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1795de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39068/0 (socket says 172.21.15.94:39068) 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa1795de640 1 -- 172.21.15.94:0/4008739470 learned_addr learned my addr 172.21.15.94:0/4008739470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.409+0000 7fa179ddf640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x5646577e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa178ddd640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657772130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 -- 172.21.15.94:0/4008739470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 msgr2=0x564657772130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657772130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 -- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 msgr2=0x5646577e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x5646577e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 -- 172.21.15.94:0/4008739470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646577e1e20 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1795de640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fa17008a010 tx=0x7fa170004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1493221b410328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa15cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700a6e60 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa1700a2e80 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1700a7020 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa1700a6c80 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1827d0740 10 client.22425 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5646577e30b0 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.410+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1700a7020 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa1700e2ce0 con 0x5646573982f0 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 10 client.22425 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 10 client.22425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 20 client.22425 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646573551c0 0x5646577e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 4 client.22425 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56465764a5f0 con 0x5646573551c0 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa178ddd640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646573551c0 0x5646577e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa15cff9640 10 client.22425 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:02.411+0000 7fa178ddd640 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646573551c0 0x5646577e5420 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b93403fb2736b91 server_cookie=99e1de7c10262d3b in_seq=0 out_seq=0 2024-04-25T14:01:02.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:02 smithi094 ceph-mon[29195]: pgmap v928: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-04-25T14:01:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:02 smithi138 ceph-mon[32977]: pgmap v928: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-04-25T14:01:03.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 124547 19.16 MB/sec execute 225 sec latency 578.629 ms 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa15cff9640 1 -- 172.21.15.94:0/4008739470 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56465764a5f0 con 0x5646573551c0 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa15cff9640 10 client.22425 handle_command_reply: tid=0 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 client.22425 shutdown 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 msgr2=0x564657771bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fa17008a010 tx=0x7fa170004a60 comp rx=0 tx=0).stop 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 shutdown_connections 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646573551c0 0x5646577e5420 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.151+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646573c11b0 0x5646577e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646573a04d0 0x564657772130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 1 --2- 172.21.15.94:0/4008739470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646573982f0 0x564657771bf0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 >> 172.21.15.94:0/4008739470 conn(0x5646573b5300 msgr2=0x56465776d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa15cff9640 0 client.22425 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 shutdown_connections 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 1 -- 172.21.15.94:0/4008739470 wait complete. 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa1827d0740 20 client.22425 trim_cache size 0 max 16384 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:01:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:01:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 msgr2=0x7fa1600b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 0x7fa1600b1ff0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x564657762b40 tx=0x7fa16c0095c0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 msgr2=0x7fa174198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.152+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e9f0 tx=0x7fa17000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 shutdown_connections 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1600afae0 0x7fa1600b1ff0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17419e3c0 0x7fa1741a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa174103b70 0x7fa174199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 --2- 172.21.15.94:0/3378437721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa174103010 0x7fa174198f50 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 >> 172.21.15.94:0/3378437721 conn(0x7fa1740fd390 msgr2=0x7fa1740fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 shutdown_connections 2024-04-25T14:01:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:03.153+0000 7fa17b842640 1 -- 172.21.15.94:0/3378437721 wait complete. 2024-04-25T14:01:04.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 124885 19.16 MB/sec execute 226 sec latency 634.330 ms 2024-04-25T14:01:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:04 smithi137 ceph-mon[32568]: pgmap v929: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-04-25T14:01:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:04 smithi094 ceph-mon[29195]: pgmap v929: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-04-25T14:01:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:04 smithi138 ceph-mon[32977]: pgmap v929: 97 pgs: 97 active+clean; 265 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-04-25T14:01:05.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 125265 19.22 MB/sec execute 227 sec latency 73.124 ms 2024-04-25T14:01:06.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 125529 19.17 MB/sec execute 228 sec latency 86.494 ms 2024-04-25T14:01:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:06 smithi137 ceph-mon[32568]: pgmap v930: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-04-25T14:01:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:06 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:06 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:06 smithi094 ceph-mon[29195]: pgmap v930: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-04-25T14:01:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:06 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:06 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:06 smithi138 ceph-mon[32977]: pgmap v930: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-04-25T14:01:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:06 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:06 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:07.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 125837 19.13 MB/sec execute 229 sec latency 65.091 ms 2024-04-25T14:01:07.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:07 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:07.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:07 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:07.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:07 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:07.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:07 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:07.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:07 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:07.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:07 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:07.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:07 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:07.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:07 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:07.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:07 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:08.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 126089 19.11 MB/sec execute 230 sec latency 183.260 ms 2024-04-25T14:01:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:08 smithi137 ceph-mon[32568]: pgmap v931: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T14:01:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:08 smithi094 ceph-mon[29195]: pgmap v931: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T14:01:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:08 smithi138 ceph-mon[32977]: pgmap v931: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 555 op/s 2024-04-25T14:01:09.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 126453 19.10 MB/sec execute 231 sec latency 82.429 ms 2024-04-25T14:01:10.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 126963 19.13 MB/sec execute 232 sec latency 53.754 ms 2024-04-25T14:01:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:10 smithi137 ceph-mon[32568]: pgmap v932: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-04-25T14:01:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:10 smithi094 ceph-mon[29195]: pgmap v932: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-04-25T14:01:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:10 smithi138 ceph-mon[32977]: pgmap v932: 97 pgs: 97 active+clean; 258 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-04-25T14:01:11.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 127482 19.15 MB/sec execute 233 sec latency 82.455 ms 2024-04-25T14:01:12.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 127968 19.22 MB/sec execute 234 sec latency 58.193 ms 2024-04-25T14:01:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:12 smithi137 ceph-mon[32568]: pgmap v933: 97 pgs: 97 active+clean; 257 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-04-25T14:01:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:12 smithi094 ceph-mon[29195]: pgmap v933: 97 pgs: 97 active+clean; 257 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-04-25T14:01:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:12 smithi138 ceph-mon[32977]: pgmap v933: 97 pgs: 97 active+clean; 257 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 612 op/s 2024-04-25T14:01:13.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 128317 19.22 MB/sec execute 235 sec latency 183.898 ms 2024-04-25T14:01:14.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 128645 19.23 MB/sec execute 236 sec latency 272.216 ms 2024-04-25T14:01:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:14 smithi137 ceph-mon[32568]: pgmap v934: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-04-25T14:01:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:14 smithi094 ceph-mon[29195]: pgmap v934: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-04-25T14:01:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:14 smithi138 ceph-mon[32977]: pgmap v934: 97 pgs: 97 active+clean; 261 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-04-25T14:01:15.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 128935 19.23 MB/sec execute 237 sec latency 60.734 ms 2024-04-25T14:01:16.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 129219 19.18 MB/sec execute 238 sec latency 73.594 ms 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 -- 172.21.15.94:0/2459048475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0104d80 msgr2=0x7fc9a0105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/2459048475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0104d80 0x7fc9a0105190 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fc994009900 tx=0x7fc99402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 -- 172.21.15.94:0/2459048475 shutdown_connections 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/2459048475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0109a80 0x7fc9a010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/2459048475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0104d80 0x7fc9a0105190 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/2459048475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0103b40 0x7fc9a0104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.262+0000 7fc9a54d1640 1 -- 172.21.15.94:0/2459048475 >> 172.21.15.94:0/2459048475 conn(0x7fc9a00fed30 msgr2=0x7fc9a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.263+0000 7fc9a54d1640 1 -- 172.21.15.94:0/2459048475 shutdown_connections 2024-04-25T14:01:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.263+0000 7fc9a54d1640 1 -- 172.21.15.94:0/2459048475 wait complete. 2024-04-25T14:01:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.263+0000 7fc9a54d1640 1 Processor -- start 2024-04-25T14:01:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.263+0000 7fc9a54d1640 1 -- start start 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 0x7fc9a019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0109a80 0x7fc9a01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010f690 con 0x7fc9a0103b40 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010f510 con 0x7fc9a0104d80 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc9a54d1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010f810 con 0x7fc9a0109a80 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39788/0 (socket says 172.21.15.94:39788) 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 -- 172.21.15.94:0/1480747727 learned_addr learned my addr 172.21.15.94:0/1480747727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 0x7fc9a019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0109a80 0x7fc9a01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 -- 172.21.15.94:0/1480747727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0109a80 msgr2=0x7fc9a01a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0109a80 0x7fc9a01a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 -- 172.21.15.94:0/1480747727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 msgr2=0x7fc9a019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 0x7fc9a019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 -- 172.21.15.94:0/1480747727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a01a1d50 con 0x7fc9a0103b40 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 0x7fc9a019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.264+0000 7fc99effd640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fc99000ead0 tx=0x7fc99000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.265+0000 7fc987fff640 1 -- 172.21.15.94:0/1480747727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ce40 con 0x7fc9a0103b40 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.265+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a01a1fe0 con 0x7fc9a0103b40 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.265+0000 7fc987fff640 1 -- 172.21.15.94:0/1480747727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc990011920 con 0x7fc9a0103b40 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.265+0000 7fc987fff640 1 -- 172.21.15.94:0/1480747727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99001aae0 con 0x7fc9a0103b40 2024-04-25T14:01:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.265+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a01a2480 con 0x7fc9a0103b40 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.267+0000 7fc9ac37e740 1 Processor -- start 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.267+0000 7fc987fff640 1 -- 172.21.15.94:0/1480747727 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc99001ac80 con 0x7fc9a0103b40 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.267+0000 7fc9ac37e740 1 -- start start 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad09ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad10df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f82ac6f460 con 0x55f82a9691b0 2024-04-25T14:01:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f82ac29590 con 0x55f82a9484d0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f82ac6b420 con 0x55f82a9402f0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39794/0 (socket says 172.21.15.94:39794) 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 -- 172.21.15.94:0/4040656877 learned_addr learned my addr 172.21.15.94:0/4040656877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad10df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99effd640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad09ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 -- 172.21.15.94:0/4040656877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 msgr2=0x55f82ad09ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad09ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 -- 172.21.15.94:0/4040656877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 msgr2=0x55f82ad10df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad10df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc987fff640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 0x7fc9880b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 -- 172.21.15.94:0/4040656877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82ac360d0 con 0x55f82a9691b0 2024-04-25T14:01:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad10df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 0x7fc9880b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc987fff640 1 -- 172.21.15.94:0/1480747727 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc9900df1a0 con 0x7fc9a0103b40 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.268+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fc998009c20 tx=0x7fc998011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7e73131d720408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.269+0000 7fc9857fa640 1 -- 172.21.15.94:0/4040656877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99801b850 con 0x55f82a9691b0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.269+0000 7fc9857fa640 1 -- 172.21.15.94:0/4040656877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc998019d80 con 0x55f82a9691b0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.269+0000 7fc9857fa640 1 -- 172.21.15.94:0/4040656877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99800ed50 con 0x55f82a9691b0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- 172.21.15.94:0/4040656877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 msgr2=0x55f82ad09970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 0x7fc9880b1ff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc998009fd0 tx=0x7fc998021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fc998009c20 tx=0x7fc998011810 comp rx=0 tx=0).stop 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- 172.21.15.94:0/4040656877 shutdown_connections 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad10df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/4040656877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9691b0 0x55f82ad09970 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- 172.21.15.94:0/4040656877 >> 172.21.15.94:0/4040656877 conn(0x55f82a95d300 msgr2=0x55f82ad05300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- 172.21.15.94:0/4040656877 shutdown_connections 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- 172.21.15.94:0/4040656877 wait complete. 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 -- start start 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.270+0000 7fc9ac37e740 1 Processor -- start 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad13be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 0x55f82ad1aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f82ad11840 con 0x55f82a9402f0 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f82abf6b60 con 0x55f82a9484d0 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f82ad119c0 con 0x55f82a9691b0 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc9ac37e740 10 client.0 _ll_register_callbacks cb 0x55f82ac1b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39802/0 (socket says 172.21.15.94:39802) 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 -- 172.21.15.94:0/768579854 learned_addr learned my addr 172.21.15.94:0/768579854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad13be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 0x55f82ad1aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 -- 172.21.15.94:0/768579854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 msgr2=0x55f82ad1aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 0x55f82ad1aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 -- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 msgr2=0x55f82ad13be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad13be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 -- 172.21.15.94:0/768579854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f82a981210 con 0x55f82a9402f0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 0x55f82ad1aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad13be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc99effd640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fc99000eaa0 tx=0x7fc9900a34a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e13da60da7d238b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc982ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.271+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9900a6d50 con 0x55f82a9402f0 2024-04-25T14:01:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc990011ad0 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc998019d80 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9900a70f0 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 10 client.22437 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f82aa4d620 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc998019d80 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc990022030 con 0x55f82a9402f0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 10 client.22437 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 10 client.22437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 20 client.22437 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a927d80 0x55f82ad21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 4 client.22437 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f82ac7d070 con 0x55f82a927d80 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.272+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a927d80 0x55f82ad21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.273+0000 7fc982ff5640 10 client.22437 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.273+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a927d80 0x55f82ad21300 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700e02b48f70e0dc server_cookie=435dfbaa10f408d7 in_seq=0 out_seq=0 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc982ff5640 1 -- 172.21.15.94:0/768579854 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f82ac7d070 con 0x55f82a927d80 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc982ff5640 10 client.22437 handle_command_reply: tid=0 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 client.22437 shutdown 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 msgr2=0x55f82ad136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fc99000eaa0 tx=0x7fc9900a34a0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 shutdown_connections 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a927d80 0x55f82ad21300 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9691b0 0x55f82ad1aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc982ff5640 0 client.22437 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.276+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9484d0 0x55f82ad13be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/768579854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9402f0 0x55f82ad136a0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 >> 172.21.15.94:0/768579854 conn(0x55f82a95d300 msgr2=0x55f82ad08210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 shutdown_connections 2024-04-25T14:01:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc9ac37e740 1 -- 172.21.15.94:0/768579854 wait complete. 2024-04-25T14:01:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.277+0000 7fc9ac37e740 20 client.22437 trim_cache size 0 max 16384 2024-04-25T14:01:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 Processor -- start 2024-04-25T14:01:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 -- start start 2024-04-25T14:01:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a927d80 0x55f82ad26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ab5d0 con 0x55f82a9484d0 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ad110 con 0x55f82a927d80 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01abca0 con 0x55f82a9402f0 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc99f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.292+0000 7fc99f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39804/0 (socket says 172.21.15.94:39804) 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 -- 172.21.15.94:0/1331985917 learned_addr learned my addr 172.21.15.94:0/1331985917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99effd640 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a927d80 0x55f82ad26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 -- 172.21.15.94:0/1331985917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 msgr2=0x55f82ad26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad26fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 -- 172.21.15.94:0/1331985917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a927d80 msgr2=0x55f82ad26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a927d80 0x55f82ad26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 -- 172.21.15.94:0/1331985917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82ad0e5c0 con 0x55f82a9484d0 2024-04-25T14:01:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fc99801b6e0 tx=0x7fc998027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25581af0614142fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc984ff9640 1 -- 172.21.15.94:0/1331985917 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99802c040 con 0x55f82a9484d0 2024-04-25T14:01:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc984ff9640 1 -- 172.21.15.94:0/1331985917 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc998037070 con 0x55f82a9484d0 2024-04-25T14:01:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.293+0000 7fc984ff9640 1 -- 172.21.15.94:0/1331985917 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998005570 con 0x55f82a9484d0 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 -- 172.21.15.94:0/1331985917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 msgr2=0x55f82ad08100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fc99801b6e0 tx=0x7fc998027000 comp rx=0 tx=0).stop 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 -- 172.21.15.94:0/1331985917 shutdown_connections 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a9484d0 0x55f82ad08100 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9402f0 0x55f82ad26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/1331985917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a927d80 0x55f82ad26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 -- 172.21.15.94:0/1331985917 >> 172.21.15.94:0/1331985917 conn(0x55f82a95d300 msgr2=0x55f82ad11cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 -- 172.21.15.94:0/1331985917 shutdown_connections 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.294+0000 7fc9ac37e740 1 -- 172.21.15.94:0/1331985917 wait complete. 2024-04-25T14:01:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 -- start start 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 Processor -- start 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9402f0 0x55f82ad179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9484d0 0x55f82ad18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01aca40 con 0x55f82a927d80 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01ac370 con 0x55f82a9402f0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a01a95b0 con 0x55f82a9484d0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc9ac37e740 10 client.0 _ll_register_callbacks cb 0x7fc990009240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc99effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39808/0 (socket says 172.21.15.94:39808) 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc99effd640 1 -- 172.21.15.94:0/2472102856 learned_addr learned my addr 172.21.15.94:0/2472102856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9402f0 0x55f82ad179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.295+0000 7fc99f7fe640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9484d0 0x55f82ad18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 -- 172.21.15.94:0/2472102856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9484d0 msgr2=0x55f82ad18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9484d0 0x55f82ad18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 -- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9402f0 msgr2=0x55f82ad179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9402f0 0x55f82ad179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 -- 172.21.15.94:0/2472102856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f82ad18ec0 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc99effd640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c700 tx=0x7fc9900b0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedd76bb3fc9b9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9900a7ce0 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc990011680 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc998037070 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9900900f0 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 10 client.22443 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f82ad1a1c0 con 0x55f82a927d80 2024-04-25T14:01:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc998037070 con 0x55f82a927d80 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc990022070 con 0x55f82a927d80 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 10 client.22443 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 10 client.22443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 20 client.22443 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a9691b0 0x55f82ad8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 4 client.22443 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.296+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f82ac76ee0 con 0x55f82a9691b0 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.297+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a9691b0 0x55f82ad8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.297+0000 7fc982ff5640 10 client.22443 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.297+0000 7fc99e7fc640 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a9691b0 0x55f82ad8b970 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261505be44048d48 server_cookie=9a9d0cdf0efe3f99 in_seq=0 out_seq=0 2024-04-25T14:01:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:16 smithi137 ceph-mon[32568]: pgmap v935: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 604 op/s 2024-04-25T14:01:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:16 smithi094 ceph-mon[29195]: pgmap v935: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 604 op/s 2024-04-25T14:01:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:16 smithi138 ceph-mon[32977]: pgmap v935: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.5 MiB/s wr, 604 op/s 2024-04-25T14:01:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc982ff5640 1 -- 172.21.15.94:0/2472102856 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f82ac76ee0 con 0x55f82a9691b0 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc982ff5640 10 client.22443 handle_command_reply: tid=0 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 client.22443 shutdown 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 msgr2=0x55f82ad17490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c700 tx=0x7fc9900b0440 comp rx=0 tx=0).stop 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 shutdown_connections 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f82a9691b0 0x55f82ad8b970 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f82a9484d0 0x55f82ad18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f82a9402f0 0x55f82ad179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 --2- 172.21.15.94:0/2472102856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f82a927d80 0x55f82ad17490 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc982ff5640 0 client.22443 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 >> 172.21.15.94:0/2472102856 conn(0x55f82a95d300 msgr2=0x55f82ad08100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 shutdown_connections 2024-04-25T14:01:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 1 -- 172.21.15.94:0/2472102856 wait complete. 2024-04-25T14:01:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.898+0000 7fc9ac37e740 20 client.22443 trim_cache size 0 max 16384 2024-04-25T14:01:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:01:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:01:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:01:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:01:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 msgr2=0x7fc9880b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 0x7fc9880b1ff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc998009fd0 tx=0x7fc998021000 comp rx=0 tx=0).stop 2024-04-25T14:01:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 msgr2=0x7fc9a019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fc99000ead0 tx=0x7fc99000ef90 comp rx=0 tx=0).stop 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 shutdown_connections 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc9880afae0 0x7fc9880b1ff0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9a0109a80 0x7fc9a01a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0104d80 0x7fc9a019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 --2- 172.21.15.94:0/1480747727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9a0103b40 0x7fc9a019a210 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 >> 172.21.15.94:0/1480747727 conn(0x7fc9a00fed30 msgr2=0x7fc9a0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 shutdown_connections 2024-04-25T14:01:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:16.899+0000 7fc9a54d1640 1 -- 172.21.15.94:0/1480747727 wait complete. 2024-04-25T14:01:17.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 129420 19.13 MB/sec execute 239 sec latency 383.790 ms 2024-04-25T14:01:18.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 129716 19.13 MB/sec execute 240 sec latency 95.280 ms 2024-04-25T14:01:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:18 smithi137 ceph-mon[32568]: pgmap v936: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 345 op/s 2024-04-25T14:01:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:18 smithi094 ceph-mon[29195]: pgmap v936: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 345 op/s 2024-04-25T14:01:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:18 smithi138 ceph-mon[32977]: pgmap v936: 97 pgs: 97 active+clean; 270 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 345 op/s 2024-04-25T14:01:19.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 130059 19.12 MB/sec execute 241 sec latency 191.454 ms 2024-04-25T14:01:20.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 130558 19.14 MB/sec execute 242 sec latency 60.978 ms 2024-04-25T14:01:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:20 smithi137 ceph-mon[32568]: pgmap v937: 97 pgs: 97 active+clean; 270 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-04-25T14:01:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:20 smithi094 ceph-mon[29195]: pgmap v937: 97 pgs: 97 active+clean; 270 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-04-25T14:01:20.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:20 smithi138 ceph-mon[32977]: pgmap v937: 97 pgs: 97 active+clean; 270 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-04-25T14:01:21.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 131072 19.15 MB/sec execute 243 sec latency 63.022 ms 2024-04-25T14:01:22.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 131519 19.22 MB/sec execute 244 sec latency 70.247 ms 2024-04-25T14:01:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:22 smithi137 ceph-mon[32568]: pgmap v938: 97 pgs: 97 active+clean; 263 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-04-25T14:01:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:22 smithi094 ceph-mon[29195]: pgmap v938: 97 pgs: 97 active+clean; 263 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-04-25T14:01:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:22 smithi138 ceph-mon[32977]: pgmap v938: 97 pgs: 97 active+clean; 263 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.2 MiB/s wr, 575 op/s 2024-04-25T14:01:23.147 INFO:tasks.workunit.client.0.smithi094.stdout: 10 131933 19.24 MB/sec execute 245 sec latency 65.656 ms 2024-04-25T14:01:24.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 132200 19.23 MB/sec execute 246 sec latency 250.421 ms 2024-04-25T14:01:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:24 smithi137 ceph-mon[32568]: pgmap v939: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:01:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:24 smithi094 ceph-mon[29195]: pgmap v939: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:01:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:24 smithi138 ceph-mon[32977]: pgmap v939: 97 pgs: 97 active+clean; 259 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:01:25.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 132458 19.22 MB/sec execute 247 sec latency 71.871 ms 2024-04-25T14:01:26.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 132743 19.17 MB/sec execute 248 sec latency 127.339 ms 2024-04-25T14:01:26.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:26 smithi138 ceph-mon[32977]: pgmap v940: 97 pgs: 97 active+clean; 268 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-04-25T14:01:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:26 smithi137 ceph-mon[32568]: pgmap v940: 97 pgs: 97 active+clean; 268 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-04-25T14:01:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:26 smithi094 ceph-mon[29195]: pgmap v940: 97 pgs: 97 active+clean; 268 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.4 MiB/s wr, 602 op/s 2024-04-25T14:01:27.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 133062 19.14 MB/sec execute 249 sec latency 110.345 ms 2024-04-25T14:01:28.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 133414 19.15 MB/sec execute 250 sec latency 78.479 ms 2024-04-25T14:01:28.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:28.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:28 smithi137 ceph-mon[32568]: pgmap v941: 97 pgs: 97 active+clean; 268 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 490 op/s 2024-04-25T14:01:28.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:28 smithi094 ceph-mon[29195]: pgmap v941: 97 pgs: 97 active+clean; 268 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 490 op/s 2024-04-25T14:01:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:28 smithi138 ceph-mon[32977]: pgmap v941: 97 pgs: 97 active+clean; 268 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 490 op/s 2024-04-25T14:01:28.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 -- 172.21.15.94:0/4068671505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 msgr2=0x7f14ec104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 --2- 172.21.15.94:0/4068671505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec104030 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:28.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 -- 172.21.15.94:0/4068671505 shutdown_connections 2024-04-25T14:01:28.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 --2- 172.21.15.94:0/4068671505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 0x7f14ec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 --2- 172.21.15.94:0/4068671505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec104030 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 --2- 172.21.15.94:0/4068671505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 0x7f14ec1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 -- 172.21.15.94:0/4068671505 >> 172.21.15.94:0/4068671505 conn(0x7f14ec0fdbd0 msgr2=0x7f14ec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 -- 172.21.15.94:0/4068671505 shutdown_connections 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.526+0000 7f14f399d640 1 -- 172.21.15.94:0/4068671505 wait complete. 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 Processor -- start 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 -- start start 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 0x7f14ec1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 0x7f14ec1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10e500 con 0x7f14ec103c20 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10e380 con 0x7f14ec108920 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f399d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10e680 con 0x7f14ec1029e0 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42520/0 (socket says 172.21.15.94:42520) 2024-04-25T14:01:28.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f1739640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 0x7f14ec1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 -- 172.21.15.94:0/1556459159 learned_addr learned my addr 172.21.15.94:0/1556459159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 0x7f14ec1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 -- 172.21.15.94:0/1556459159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 msgr2=0x7f14ec1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 0x7f14ec1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 -- 172.21.15.94:0/1556459159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 msgr2=0x7f14ec1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 0x7f14ec1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 -- 172.21.15.94:0/1556459159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec1a0d00 con 0x7f14ec103c20 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.527+0000 7f14f0f38640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.528+0000 7f14e27fc640 1 -- 172.21.15.94:0/1556459159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc046070 con 0x7f14ec103c20 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.528+0000 7f14e27fc640 1 -- 172.21.15.94:0/1556459159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc031910 con 0x7f14ec103c20 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.528+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec074dd0 con 0x7f14ec103c20 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.528+0000 7f14e27fc640 1 -- 172.21.15.94:0/1556459159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc035050 con 0x7f14ec103c20 2024-04-25T14:01:28.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.528+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec0752c0 con 0x7f14ec103c20 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 Processor -- start 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 -- start start 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14e27fc640 1 -- 172.21.15.94:0/1556459159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f14dc049070 con 0x7f14ec103c20 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164bf7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558164b5cd80 con 0x5581648356b0 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558164b168b0 con 0x55816482d4d0 2024-04-25T14:01:28.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14fa84f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558164b58d40 con 0x558164856390 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14f1f3a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14f0f38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14f1f3a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40760/0 (socket says 172.21.15.94:40760) 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14f1f3a640 1 -- 172.21.15.94:0/724393517 learned_addr learned my addr 172.21.15.94:0/724393517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.531+0000 7f14f1739640 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164bf7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 -- 172.21.15.94:0/724393517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 msgr2=0x558164bf7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164bf7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 -- 172.21.15.94:0/724393517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 msgr2=0x558164bf8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 -- 172.21.15.94:0/724393517 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558164ae7a40 con 0x558164856390 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14e27fc640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 0x7f14d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009fd0 tx=0x7f14e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d49baea72307c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14f1739640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 0x7f14d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14e27fc640 1 -- 172.21.15.94:0/1556459159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f14dc100740 con 0x7f14ec103c20 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14d77fe640 1 -- 172.21.15.94:0/724393517 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004d20 con 0x558164856390 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14d77fe640 1 -- 172.21.15.94:0/724393517 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e8004050 con 0x558164856390 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.532+0000 7f14d77fe640 1 -- 172.21.15.94:0/724393517 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8005280 con 0x558164856390 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- 172.21.15.94:0/724393517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 msgr2=0x558164bf7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14f1739640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 0x7f14d80b2000 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x558164bf7780 tx=0x7f14e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009fd0 tx=0x7f14e80118d0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- 172.21.15.94:0/724393517 shutdown_connections 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164bf7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 --2- 172.21.15.94:0/724393517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf7a60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- 172.21.15.94:0/724393517 >> 172.21.15.94:0/724393517 conn(0x55816484a4e0 msgr2=0x558164bf3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- 172.21.15.94:0/724393517 shutdown_connections 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- 172.21.15.94:0/724393517 wait complete. 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.533+0000 7f14fa84f740 1 -- start start 2024-04-25T14:01:28.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 Processor -- start 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55816482d4d0 0x558164c01a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558164bf9780 con 0x558164856390 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558164ae3d10 con 0x5581648356b0 2024-04-25T14:01:28.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558164bf9900 con 0x55816482d4d0 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14fa84f740 10 client.0 _ll_register_callbacks cb 0x558164b10210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42534/0 (socket says 172.21.15.94:42534) 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 -- 172.21.15.94:0/590872293 learned_addr learned my addr 172.21.15.94:0/590872293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1739640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55816482d4d0 0x558164c01a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f0f38640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 -- 172.21.15.94:0/590872293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55816482d4d0 msgr2=0x558164c01a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55816482d4d0 0x558164c01a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 -- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 msgr2=0x558164c01f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c01f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.534+0000 7f14f1f3a640 1 -- 172.21.15.94:0/590872293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581649387d0 con 0x558164856390 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f14e801b040 tx=0x7f14e80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cba210a2cb9f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8028020 con 0x558164856390 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f14e8004c50 con 0x558164856390 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8004050 con 0x558164856390 2024-04-25T14:01:28.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f14e8033050 con 0x558164856390 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 10 client.22455 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55816486e3f0 con 0x558164856390 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14e8004050 con 0x558164856390 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f14e800f4e0 con 0x558164856390 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 10 client.22455 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 10 client.22455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 20 client.22455 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558164814f60 0x558164c0f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 4 client.22455 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.535+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558164b6a990 con 0x558164814f60 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.536+0000 7f14f1739640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558164814f60 0x558164c0f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.536+0000 7f14d4ff9640 10 client.22455 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.536+0000 7f14f1739640 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558164814f60 0x558164c0f200 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cd20da271ccb04 server_cookie=ae8540f5a2c527ab in_seq=0 out_seq=0 2024-04-25T14:01:28.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14d4ff9640 1 -- 172.21.15.94:0/590872293 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558164b6a990 con 0x558164814f60 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14d4ff9640 10 client.22455 handle_command_reply: tid=0 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 client.22455 shutdown 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 msgr2=0x558164c08ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f14e801b040 tx=0x7f14e80044b0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 shutdown_connections 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558164814f60 0x558164c0f200 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558164856390 0x558164c08ed0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 --2- 172.21.15.94:0/590872293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55816482d4d0 0x558164c01a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14d4ff9640 0 client.22455 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 >> 172.21.15.94:0/590872293 conn(0x55816484a4e0 msgr2=0x558164bf6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 shutdown_connections 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 1 -- 172.21.15.94:0/590872293 wait complete. 2024-04-25T14:01:28.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.538+0000 7f14fa84f740 20 client.22455 trim_cache size 0 max 16384 2024-04-25T14:01:28.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.556+0000 7f14fa84f740 1 Processor -- start 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.556+0000 7f14fa84f740 1 -- start start 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164c02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1a93a0 con 0x5581648356b0 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1aaee0 con 0x55816482d4d0 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14fa84f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1a9a70 con 0x558164856390 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42550/0 (socket says 172.21.15.94:42550) 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 -- 172.21.15.94:0/3757900040 learned_addr learned my addr 172.21.15.94:0/3757900040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f1739640 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164c02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 -- 172.21.15.94:0/3757900040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 msgr2=0x558164bf3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 -- 172.21.15.94:0/3757900040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 msgr2=0x558164c02140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164c02140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 -- 172.21.15.94:0/3757900040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816482e0f0 con 0x5581648356b0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0c4cb0 tx=0x7f14dc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d7f2d179b0d500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14d6ffd640 1 -- 172.21.15.94:0/3757900040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0c8580 con 0x5581648356b0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14d6ffd640 1 -- 172.21.15.94:0/3757900040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc0d1320 con 0x5581648356b0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.557+0000 7f14d6ffd640 1 -- 172.21.15.94:0/3757900040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0c83a0 con 0x5581648356b0 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 -- 172.21.15.94:0/3757900040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 msgr2=0x558164bf2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0c4cb0 tx=0x7f14dc005c90 comp rx=0 tx=0).stop 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 -- 172.21.15.94:0/3757900040 shutdown_connections 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164bf3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5581648356b0 0x558164bf2c30 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3757900040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55816482d4d0 0x558164c02140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 -- 172.21.15.94:0/3757900040 >> 172.21.15.94:0/3757900040 conn(0x55816484a4e0 msgr2=0x558164c01a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 -- 172.21.15.94:0/3757900040 shutdown_connections 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.558+0000 7f14fa84f740 1 -- 172.21.15.94:0/3757900040 wait complete. 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 -- start start 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 Processor -- start 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c05870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164c75560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1aa810 con 0x55816482d4d0 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1aa140 con 0x5581648356b0 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1a7380 con 0x558164856390 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14fa84f740 10 client.0 _ll_register_callbacks cb 0x7f14dc106420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:28.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42558/0 (socket says 172.21.15.94:42558) 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 -- 172.21.15.94:0/3078770836 learned_addr learned my addr 172.21.15.94:0/3078770836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c05870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1f3a640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164c75560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 -- 172.21.15.94:0/3078770836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 msgr2=0x558164c75560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164c75560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 -- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 msgr2=0x558164c05870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c05870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.559+0000 7f14f1739640 1 -- 172.21.15.94:0/3078770836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558164b1d4b0 con 0x55816482d4d0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14f1739640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x558164bf4000 tx=0x7f14e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87018315cc863abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4005020 con 0x55816482d4d0 2024-04-25T14:01:28.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f14e40051c0 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558164b64800 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f14e40053c0 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 10 client.22461 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558164c768f0 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558164b64800 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f14e4003710 con 0x55816482d4d0 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 10 client.22461 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 10 client.22461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 20 client.22461 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581647ea3a0 0x558164c79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 4 client.22461 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.560+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558164adf4a0 con 0x5581647ea3a0 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.561+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581647ea3a0 0x558164c79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.561+0000 7f14d4ff9640 10 client.22461 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.561+0000 7f14f0f38640 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581647ea3a0 0x558164c79270 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3a48239bc6c935 server_cookie=f3d96ce73f97bdaf in_seq=0 out_seq=0 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14d4ff9640 1 -- 172.21.15.94:0/3078770836 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558164adf4a0 con 0x5581647ea3a0 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14d4ff9640 10 client.22461 handle_command_reply: tid=0 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 client.22461 shutdown 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 msgr2=0x558164c05330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x558164bf4000 tx=0x7f14e40046c0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 shutdown_connections 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5581647ea3a0 0x558164c79270 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558164856390 0x558164c75560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581648356b0 0x558164c05870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14d4ff9640 0 client.22461 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 --2- 172.21.15.94:0/3078770836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55816482d4d0 0x558164c05330 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 >> 172.21.15.94:0/3078770836 conn(0x55816484a4e0 msgr2=0x558164bf2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 shutdown_connections 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.563+0000 7f14fa84f740 1 -- 172.21.15.94:0/3078770836 wait complete. 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14fa84f740 20 client.22461 trim_cache size 0 max 16384 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:01:28.574 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:01:28.575 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:01:28.575 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:01:28.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 msgr2=0x7f14d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 0x7f14d80b2000 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x558164bf7780 tx=0x7f14e4005f90 comp rx=0 tx=0).stop 2024-04-25T14:01:28.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 msgr2=0x7f14ec199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc004270 comp rx=0 tx=0).stop 2024-04-25T14:01:28.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 shutdown_connections 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f14d80afaf0 0x7f14d80b2000 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14ec108920 0x7f14ec1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec103c20 0x7f14ec199700 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 --2- 172.21.15.94:0/1556459159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec1029e0 0x7f14ec1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 >> 172.21.15.94:0/1556459159 conn(0x7f14ec0fdbd0 msgr2=0x7f14ec104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 shutdown_connections 2024-04-25T14:01:28.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.564+0000 7f14f399d640 1 -- 172.21.15.94:0/1556459159 wait complete. 2024-04-25T14:01:28.577 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:01:28.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 -- 172.21.15.94:0/3350281857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 msgr2=0x7fd6281036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/3350281857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd6281036c0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009900 tx=0x7fd61c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 -- 172.21.15.94:0/3350281857 shutdown_connections 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/3350281857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 0x7fd62810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/3350281857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 0x7fd628104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/3350281857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd6281036c0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 -- 172.21.15.94:0/3350281857 >> 172.21.15.94:0/3350281857 conn(0x7fd6280fdbd0 msgr2=0x7fd628100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:28.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 -- 172.21.15.94:0/3350281857 shutdown_connections 2024-04-25T14:01:28.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.733+0000 7fd62cf6b640 1 -- 172.21.15.94:0/3350281857 wait complete. 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 Processor -- start 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 -- start start 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 0x7fd628199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 0x7fd6281a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810e4c0 con 0x7fd6281029c0 2024-04-25T14:01:28.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810e340 con 0x7fd628103c00 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62cf6b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810e640 con 0x7fd628108900 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42582/0 (socket says 172.21.15.94:42582) 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 -- 172.21.15.94:0/959079127 learned_addr learned my addr 172.21.15.94:0/959079127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd625d9b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 0x7fd628199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd626d9d640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 0x7fd6281a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 -- 172.21.15.94:0/959079127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 msgr2=0x7fd6281a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 0x7fd6281a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 -- 172.21.15.94:0/959079127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 msgr2=0x7fd628199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:28.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 0x7fd628199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.734+0000 7fd62659c640 1 -- 172.21.15.94:0/959079127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281a0c60 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd62659c640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fd61c006010 tx=0x7fd61c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c03f030 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a0ef0 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd61c031910 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c035070 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.735+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a1350 con 0x7fd6281029c0 2024-04-25T14:01:28.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.736+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628107180 con 0x7fd6281029c0 2024-04-25T14:01:28.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.738+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd61c042030 con 0x7fd6281029c0 2024-04-25T14:01:28.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.738+0000 7fd6177fe640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 0x7fd6100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:28.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.738+0000 7fd625d9b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 0x7fd6100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:28.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.738+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd61c0ff440 con 0x7fd6281029c0 2024-04-25T14:01:28.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.739+0000 7fd625d9b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 0x7fd6100b2080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd62819a1c0 tx=0x7fd618005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:28.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.743+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd61c0c7f80 con 0x7fd6281029c0 2024-04-25T14:01:28.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.983+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd628107540 con 0x7fd6281029c0 2024-04-25T14:01:29.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.984+0000 7fd6177fe640 1 -- 172.21.15.94:0/959079127 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7fd61c0c7880 con 0x7fd6281029c0 2024-04-25T14:01:29.112 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:01:29.112 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:01:29.113 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:01:29.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 msgr2=0x7fd6100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 0x7fd6100b2080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd62819a1c0 tx=0x7fd618005f90 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 msgr2=0x7fd628199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fd61c006010 tx=0x7fd61c004270 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 shutdown_connections 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd6100afb70 0x7fd6100b2080 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd628108900 0x7fd6281a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd628103c00 0x7fd628199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 --2- 172.21.15.94:0/959079127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6281029c0 0x7fd628199120 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 >> 172.21.15.94:0/959079127 conn(0x7fd6280fdbd0 msgr2=0x7fd6281048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 shutdown_connections 2024-04-25T14:01:29.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:28.986+0000 7fd62cf6b640 1 -- 172.21.15.94:0/959079127 wait complete. 2024-04-25T14:01:29.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:01:29.148 INFO:tasks.workunit.client.0.smithi094.stdout: 10 133750 19.14 MB/sec execute 251 sec latency 231.308 ms 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 -- 172.21.15.94:0/3697566721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc105090 msgr2=0x7fa3cc1054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 --2- 172.21.15.94:0/3697566721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc105090 0x7fa3cc1054a0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009900 tx=0x7fa3c802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 -- 172.21.15.94:0/3697566721 shutdown_connections 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 --2- 172.21.15.94:0/3697566721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc102240 0x7fa3cc10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 --2- 172.21.15.94:0/3697566721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc101740 0x7fa3cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 --2- 172.21.15.94:0/3697566721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc105090 0x7fa3cc1054a0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.253+0000 7fa3d360c640 1 -- 172.21.15.94:0/3697566721 >> 172.21.15.94:0/3697566721 conn(0x7fa3cc0fd340 msgr2=0x7fa3cc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.254+0000 7fa3d360c640 1 -- 172.21.15.94:0/3697566721 shutdown_connections 2024-04-25T14:01:29.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.254+0000 7fa3d360c640 1 -- 172.21.15.94:0/3697566721 wait complete. 2024-04-25T14:01:29.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.254+0000 7fa3d360c640 1 Processor -- start 2024-04-25T14:01:29.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.254+0000 7fa3d360c640 1 -- start start 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 0x7fa3cc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 0x7fa3cc19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc105090 0x7fa3cc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10f680 con 0x7fa3cc102240 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10f500 con 0x7fa3cc101740 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d360c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc10f800 con 0x7fa3cc105090 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 0x7fa3cc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 0x7fa3cc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58864/0 (socket says 172.21.15.94:58864) 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d13a8640 1 -- 172.21.15.94:0/2630147133 learned_addr learned my addr 172.21.15.94:0/2630147133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.255+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 0x7fa3cc19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc105090 0x7fa3cc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/2630147133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc105090 msgr2=0x7fa3cc1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc105090 0x7fa3cc1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/2630147133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 msgr2=0x7fa3cc19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 0x7fa3cc19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/2630147133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc1a1dd0 con 0x7fa3cc102240 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 0x7fa3cc19a7d0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9f0 tx=0x7fa3bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3c27fc640 1 -- 172.21.15.94:0/2630147133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00ccc0 con 0x7fa3cc102240 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc1a20c0 con 0x7fa3cc102240 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3c27fc640 1 -- 172.21.15.94:0/2630147133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3bc005020 con 0x7fa3cc102240 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3c27fc640 1 -- 172.21.15.94:0/2630147133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004a00 con 0x7fa3cc102240 2024-04-25T14:01:29.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.256+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1a2690 con 0x7fa3cc102240 2024-04-25T14:01:29.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 Processor -- start 2024-04-25T14:01:29.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3c27fc640 1 -- 172.21.15.94:0/2630147133 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa3bc004ba0 con 0x7fa3cc102240 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 -- start start 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed895810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed895d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5637ed7f9d80 con 0x5637ed4d26b0 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5637ed7b3a20 con 0x5637ed4ca4d0 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.261+0000 7fa3da599740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5637ed7f5d40 con 0x5637ed4f3390 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42606/0 (socket says 172.21.15.94:42606) 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/1963953984 learned_addr learned my addr 172.21.15.94:0/1963953984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed895d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed895810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3c27fc640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 0x7fa3b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/1963953984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 msgr2=0x5637ed895810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed895810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/1963953984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 msgr2=0x5637ed895d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed895d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/1963953984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ed7bec30 con 0x5637ed4d26b0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 0x7fa3b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3c27fc640 1 -- 172.21.15.94:0/2630147133 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa3bc0ded60 con 0x7fa3cc102240 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.262+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc018bf0 tx=0x7fa3bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9e11b486c43330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.263+0000 7fa3b77fe640 1 -- 172.21.15.94:0/1963953984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0a6af0 con 0x5637ed4d26b0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.263+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 0x7fa3b80b2000 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x5637ed896860 tx=0x7fa3c400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.263+0000 7fa3b77fe640 1 -- 172.21.15.94:0/1963953984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3bc018c50 con 0x5637ed4d26b0 2024-04-25T14:01:29.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.263+0000 7fa3b77fe640 1 -- 172.21.15.94:0/1963953984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc01b170 con 0x5637ed4d26b0 2024-04-25T14:01:29.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 -- 172.21.15.94:0/1963953984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 msgr2=0x5637ed896a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc018bf0 tx=0x7fa3bc01f000 comp rx=0 tx=0).stop 2024-04-25T14:01:29.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 -- 172.21.15.94:0/1963953984 shutdown_connections 2024-04-25T14:01:29.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed896a20 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed895d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 --2- 172.21.15.94:0/1963953984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed895810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.264+0000 7fa3da599740 1 -- 172.21.15.94:0/1963953984 >> 172.21.15.94:0/1963953984 conn(0x5637ed4e74e0 msgr2=0x5637ed891020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.265+0000 7fa3da599740 1 -- 172.21.15.94:0/1963953984 shutdown_connections 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.265+0000 7fa3da599740 1 -- 172.21.15.94:0/1963953984 wait complete. 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.265+0000 7fa3da599740 1 -- start start 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.265+0000 7fa3da599740 1 Processor -- start 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4d26b0 0x5637ed89faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4f3390 0x5637ed8a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5637ed8974d0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5637ed780dc0 con 0x5637ed4f3390 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5637ed897650 con 0x5637ed4d26b0 2024-04-25T14:01:29.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3da599740 10 client.0 _ll_register_callbacks cb 0x5637ed79c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42622/0 (socket says 172.21.15.94:42622) 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 -- 172.21.15.94:0/1121972802 learned_addr learned my addr 172.21.15.94:0/1121972802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4d26b0 0x5637ed89faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4f3390 0x5637ed8a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 -- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4d26b0 msgr2=0x5637ed89faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4d26b0 0x5637ed89faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 -- 172.21.15.94:0/1121972802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4f3390 msgr2=0x5637ed8a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4f3390 0x5637ed8a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.266+0000 7fa3d13a8640 1 -- 172.21.15.94:0/1121972802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ed50b3f0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006010 tx=0x7fa3c8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d5b9284b8a09fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8035070 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa3c8004240 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5637ed807990 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3c8004440 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3da599740 10 client.22479 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5637ed5d57d0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.267+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5637ed807990 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa3c8004b00 con 0x5637ed4ca4d0 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 10 client.22479 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 10 client.22479 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 20 client.22479 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4b1f60 0x5637ed8ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 4 client.22479 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:29.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ed801800 con 0x5637ed4b1f60 2024-04-25T14:01:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4b1f60 0x5637ed8ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.268+0000 7fa3b4ff9640 10 client.22479 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:01:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.269+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4b1f60 0x5637ed8ad1f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41e8cace2d1353c0 server_cookie=a0fd934fcbb0f5ce in_seq=0 out_seq=0 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/1121972802 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637ed801800 con 0x5637ed4b1f60 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3b4ff9640 10 client.22479 handle_command_reply: tid=0 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 client.22479 shutdown 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 msgr2=0x5637ed89f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006010 tx=0x7fa3c8031850 comp rx=0 tx=0).stop 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 shutdown_connections 2024-04-25T14:01:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4b1f60 0x5637ed8ad1f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4f3390 0x5637ed8a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4d26b0 0x5637ed89faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 --2- 172.21.15.94:0/1121972802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed89f560 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 >> 172.21.15.94:0/1121972802 conn(0x5637ed4e74e0 msgr2=0x5637ed890040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3b4ff9640 0 client.22479 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 shutdown_connections 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 1 -- 172.21.15.94:0/1121972802 wait complete. 2024-04-25T14:01:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.270+0000 7fa3da599740 20 client.22479 trim_cache size 0 max 16384 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.286+0000 7fa3da599740 1 Processor -- start 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.286+0000 7fa3da599740 1 -- start start 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4d26b0 0x5637ed8a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed89dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1ab5e0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1ad120 con 0x5637ed4d26b0 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3da599740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1abcb0 con 0x5637ed4f3390 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42628/0 (socket says 172.21.15.94:42628) 2024-04-25T14:01:29.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 -- 172.21.15.94:0/2229224568 learned_addr learned my addr 172.21.15.94:0/2229224568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed89dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4d26b0 0x5637ed8a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 -- 172.21.15.94:0/2229224568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 msgr2=0x5637ed89dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed89dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 -- 172.21.15.94:0/2229224568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4d26b0 msgr2=0x5637ed8a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4d26b0 0x5637ed8a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 -- 172.21.15.94:0/2229224568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ed89feb0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.287+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8031440 tx=0x7fa3c8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c3637548616d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.288+0000 7fa3b6ffd640 1 -- 172.21.15.94:0/2229224568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8035030 con 0x5637ed4ca4d0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.288+0000 7fa3b6ffd640 1 -- 172.21.15.94:0/2229224568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c80038e0 con 0x5637ed4ca4d0 2024-04-25T14:01:29.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.288+0000 7fa3b6ffd640 1 -- 172.21.15.94:0/2229224568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8003b80 con 0x5637ed4ca4d0 2024-04-25T14:01:29.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- 172.21.15.94:0/2229224568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 msgr2=0x5637ed894850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8031440 tx=0x7fa3c8031a50 comp rx=0 tx=0).stop 2024-04-25T14:01:29.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- 172.21.15.94:0/2229224568 shutdown_connections 2024-04-25T14:01:29.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed89dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4d26b0 0x5637ed8a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 --2- 172.21.15.94:0/2229224568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4ca4d0 0x5637ed894850 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- 172.21.15.94:0/2229224568 >> 172.21.15.94:0/2229224568 conn(0x5637ed4e74e0 msgr2=0x5637ed89f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- 172.21.15.94:0/2229224568 shutdown_connections 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- 172.21.15.94:0/2229224568 wait complete. 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 -- start start 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.289+0000 7fa3da599740 1 Processor -- start 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed8a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed8a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1aca50 con 0x5637ed4d26b0 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1ac380 con 0x5637ed4ca4d0 2024-04-25T14:01:29.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc1a95c0 con 0x5637ed4f3390 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3da599740 10 client.0 _ll_register_callbacks cb 0x7fa3bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42640/0 (socket says 172.21.15.94:42640) 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/795104204 learned_addr learned my addr 172.21.15.94:0/795104204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed8a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 msgr2=0x5637ed8a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed8a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/795104204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 msgr2=0x5637ed8a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed8a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d0ba7640 1 -- 172.21.15.94:0/795104204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ed8a6e60 con 0x5637ed4d26b0 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.290+0000 7fa3d1ba9640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed8a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3d0ba7640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9c0 tx=0x7fa3bc004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5ee4f86c9aaf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc01c040 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa3bc0054f0 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c80038e0 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc0a6910 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 10 client.22485 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5637ed8a68b0 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c80038e0 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa3bc0a3430 con 0x5637ed4d26b0 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 10 client.22485 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 10 client.22485 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.291+0000 7fa3da599740 20 client.22485 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.292+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4873a0 0x5637ed91b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.292+0000 7fa3da599740 4 client.22485 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:29.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.292+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637ed77c610 con 0x5637ed4873a0 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.292+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4873a0 0x5637ed91b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.293+0000 7fa3b4ff9640 10 client.22485 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.293+0000 7fa3d13a8640 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4873a0 0x5637ed91b980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da0dd05d99a1c555 server_cookie=fbd63f04ee319ae7 in_seq=0 out_seq=0 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.293+0000 7fa3b4ff9640 1 -- 172.21.15.94:0/795104204 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637ed77c610 con 0x5637ed4873a0 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.293+0000 7fa3b4ff9640 10 client.22485 handle_command_reply: tid=0 2024-04-25T14:01:29.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.293+0000 7fa3da599740 1 client.22485 shutdown 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.294+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 msgr2=0x5637ed8a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.294+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9c0 tx=0x7fa3bc004e90 comp rx=0 tx=0).stop 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 shutdown_connections 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5637ed4873a0 0x5637ed91b980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637ed4f3390 0x5637ed8a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5637ed4d26b0 0x5637ed8a2300 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3b4ff9640 0 client.22485 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 --2- 172.21.15.94:0/795104204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637ed4ca4d0 0x5637ed8a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 >> 172.21.15.94:0/795104204 conn(0x5637ed4e74e0 msgr2=0x5637ed89f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 shutdown_connections 2024-04-25T14:01:29.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.295+0000 7fa3da599740 1 -- 172.21.15.94:0/795104204 wait complete. 2024-04-25T14:01:29.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3da599740 20 client.22485 trim_cache size 0 max 16384 2024-04-25T14:01:29.299 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 msgr2=0x7fa3b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 0x7fa3b80b2000 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x5637ed896860 tx=0x7fa3c400c040 comp rx=0 tx=0).stop 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 msgr2=0x7fa3cc19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 0x7fa3cc19a7d0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e9f0 tx=0x7fa3bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 shutdown_connections 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa3b80afaf0 0x7fa3b80b2000 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3cc105090 0x7fa3cc1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3cc102240 0x7fa3cc19a7d0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 --2- 172.21.15.94:0/2630147133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3cc101740 0x7fa3cc19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.296+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 >> 172.21.15.94:0/2630147133 conn(0x7fa3cc0fd340 msgr2=0x7fa3cc105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.297+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 shutdown_connections 2024-04-25T14:01:29.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.297+0000 7fa3d360c640 1 -- 172.21.15.94:0/2630147133 wait complete. 2024-04-25T14:01:29.309 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:29 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.309 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:29 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/959079127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:01:29.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:29 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:29 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/959079127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 -- 172.21.15.94:0/2784994935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac1085f0 msgr2=0x7f58ac10ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 --2- 172.21.15.94:0/2784994935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac1085f0 0x7f58ac10ab00 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009ca0 tx=0x7f58a402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 -- 172.21.15.94:0/2784994935 shutdown_connections 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 --2- 172.21.15.94:0/2784994935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac1085f0 0x7f58ac10ab00 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 --2- 172.21.15.94:0/2784994935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac105a80 0x7f58ac107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 --2- 172.21.15.94:0/2784994935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac103090 0x7f58ac1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 -- 172.21.15.94:0/2784994935 >> 172.21.15.94:0/2784994935 conn(0x7f58ac0fcf20 msgr2=0x7f58ac0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 -- 172.21.15.94:0/2784994935 shutdown_connections 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.475+0000 7f58b0f46640 1 -- 172.21.15.94:0/2784994935 wait complete. 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 Processor -- start 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 -- start start 2024-04-25T14:01:29.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac103090 0x7f58ac1025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac1085f0 0x7f58ac1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10e510 con 0x7f58ac105a80 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10e390 con 0x7f58ac103090 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58b0f46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10e690 con 0x7f58ac1085f0 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42660/0 (socket says 172.21.15.94:42660) 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58a3fff640 1 -- 172.21.15.94:0/4289519399 learned_addr learned my addr 172.21.15.94:0/4289519399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58aad9d640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac1085f0 0x7f58ac1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.476+0000 7f58aa59c640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac103090 0x7f58ac1025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 -- 172.21.15.94:0/4289519399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac1085f0 msgr2=0x7f58ac1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac1085f0 0x7f58ac1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 -- 172.21.15.94:0/4289519399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac103090 msgr2=0x7f58ac1025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac103090 0x7f58ac1025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 -- 172.21.15.94:0/4289519399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac1a1ce0 con 0x7f58ac105a80 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a3fff640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f589400d950 tx=0x7f589400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a37fe640 1 -- 172.21.15.94:0/4289519399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894017030 con 0x7f58ac105a80 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac1a1fd0 con 0x7f58ac105a80 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a37fe640 1 -- 172.21.15.94:0/4289519399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5894004020 con 0x7f58ac105a80 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58a37fe640 1 -- 172.21.15.94:0/4289519399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940047a0 con 0x7f58ac105a80 2024-04-25T14:01:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.477+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac1a2440 con 0x7f58ac105a80 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 Processor -- start 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 -- start start 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58a37fe640 1 -- 172.21.15.94:0/4289519399 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5894009a30 con 0x7f58ac105a80 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564f41433d80 con 0x564f4110c6b0 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564f413eda20 con 0x564f411044d0 2024-04-25T14:01:29.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58b7df7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f4142fd40 con 0x564f4112d390 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58aad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58aad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414d3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40898/0 (socket says 172.21.15.94:40898) 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.480+0000 7f58aad9d640 1 -- 172.21.15.94:0/1602972871 learned_addr learned my addr 172.21.15.94:0/1602972871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 -- 172.21.15.94:0/1602972871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 msgr2=0x564f414d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 -- 172.21.15.94:0/1602972871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 msgr2=0x564f414d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 -- 172.21.15.94:0/1602972871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f413f8c30 con 0x564f411044d0 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a3fff640 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a37fe640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 0x7f58980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aa59c640 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414d41a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f58ac101390 tx=0x7f589c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fe87ab8301bf74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58aad9d640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 0x7f58980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a0ff9640 1 -- 172.21.15.94:0/1602972871 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c009d70 con 0x564f411044d0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a0ff9640 1 -- 172.21.15.94:0/1602972871 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589c005020 con 0x564f411044d0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a0ff9640 1 -- 172.21.15.94:0/1602972871 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c004ad0 con 0x564f411044d0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.481+0000 7f58a37fe640 1 -- 172.21.15.94:0/4289519399 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f58940de670 con 0x7f58ac105a80 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58aad9d640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 0x7f58980b2000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x564f414d4c70 tx=0x7f58a4009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 -- 172.21.15.94:0/1602972871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 msgr2=0x564f414d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414d41a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f58ac101390 tx=0x7f589c011810 comp rx=0 tx=0).stop 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 -- 172.21.15.94:0/1602972871 shutdown_connections 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414d41a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 --2- 172.21.15.94:0/1602972871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.482+0000 7f58b7df7740 1 -- 172.21.15.94:0/1602972871 >> 172.21.15.94:0/1602972871 conn(0x564f411214e0 msgr2=0x564f414cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.483+0000 7f58b7df7740 1 -- 172.21.15.94:0/1602972871 shutdown_connections 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.483+0000 7f58b7df7740 1 -- 172.21.15.94:0/1602972871 wait complete. 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.483+0000 7f58b7df7740 1 -- start start 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.483+0000 7f58b7df7740 1 Processor -- start 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564f414d5940 con 0x564f4110c6b0 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564f413badc0 con 0x564f411044d0 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f414d5ac0 con 0x564f4112d390 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58b7df7740 10 client.0 _ll_register_callbacks cb 0x564f413d6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58aad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42670/0 (socket says 172.21.15.94:42670) 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58aad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40912/0 (socket says 172.21.15.94:40912) 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58922/0 (socket says 172.21.15.94:58922) 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 -- 172.21.15.94:0/275351435 learned_addr learned my addr 172.21.15.94:0/275351435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 -- 172.21.15.94:0/275351435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 msgr2=0x564f414e4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 -- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 msgr2=0x564f414dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 -- 172.21.15.94:0/275351435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f411453f0 con 0x564f4110c6b0 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58aad9d640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f58a3fff640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f58940a67b0 tx=0x7f5894005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85b020b3c5df8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.484+0000 7f5891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940a6400 con 0x564f4110c6b0 2024-04-25T14:01:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f58940041d0 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589c005020 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f58940a6220 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 10 client.22497 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f4120f7d0 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589c005020 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f58940a2d40 con 0x564f4110c6b0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 10 client.22497 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 10 client.22497 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 20 client.22497 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f410ebf60 0x564f414eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 4 client.22497 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f41441990 con 0x564f410ebf60 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.485+0000 7f58aa59c640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f410ebf60 0x564f414eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.486+0000 7f5891ffb640 10 client.22497 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:01:29.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.486+0000 7f58aa59c640 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f410ebf60 0x564f414eb670 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f2e620dbf96ea47 server_cookie=f2502f9b938e2023 in_seq=0 out_seq=0 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f5891ffb640 1 -- 172.21.15.94:0/275351435 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f41441990 con 0x564f410ebf60 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f5891ffb640 10 client.22497 handle_command_reply: tid=0 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 client.22497 shutdown 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 msgr2=0x564f414dded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f58940a67b0 tx=0x7f5894005bf0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 shutdown_connections 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f410ebf60 0x564f414eb670 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f414e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f4110c6b0 0x564f414dded0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f5891ffb640 0 client.22497 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:01:29.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 --2- 172.21.15.94:0/275351435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f411044d0 0x564f414dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 >> 172.21.15.94:0/275351435 conn(0x564f411214e0 msgr2=0x564f414cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.487+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 shutdown_connections 2024-04-25T14:01:29.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.488+0000 7f58b7df7740 1 -- 172.21.15.94:0/275351435 wait complete. 2024-04-25T14:01:29.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.488+0000 7f58b7df7740 20 client.22497 trim_cache size 0 max 16384 2024-04-25T14:01:29.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:29 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:29 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/959079127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 Processor -- start 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 -- start start 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f411044d0 0x564f414dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 0x564f414efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1a9310 con 0x564f410c13a0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1aae50 con 0x564f4112d390 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.515+0000 7f58b7df7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1a99e0 con 0x564f411044d0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aa59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f411044d0 0x564f414dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 0x564f414efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58934/0 (socket says 172.21.15.94:58934) 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58a3fff640 1 -- 172.21.15.94:0/2633690733 learned_addr learned my addr 172.21.15.94:0/2633690733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 -- 172.21.15.94:0/2633690733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f411044d0 msgr2=0x564f414dc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f411044d0 0x564f414dc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 -- 172.21.15.94:0/2633690733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 msgr2=0x564f414dc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 -- 172.21.15.94:0/2633690733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f414dbae0 con 0x564f410c13a0 2024-04-25T14:01:29.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58a3fff640 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f58aad9d640 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 0x564f414efe20 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006010 tx=0x7f58a4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bafaeaa472acf5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f5893fff640 1 -- 172.21.15.94:0/2633690733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4004240 con 0x564f410c13a0 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f5893fff640 1 -- 172.21.15.94:0/2633690733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a4036050 con 0x564f410c13a0 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.516+0000 7f5893fff640 1 -- 172.21.15.94:0/2633690733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40305c0 con 0x564f410c13a0 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 -- 172.21.15.94:0/2633690733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 msgr2=0x564f414efe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 0x564f414efe20 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f58a4006010 tx=0x7f58a4044000 comp rx=0 tx=0).stop 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 -- 172.21.15.94:0/2633690733 shutdown_connections 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f410c13a0 0x564f414efe20 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f4112d390 0x564f414dc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 --2- 172.21.15.94:0/2633690733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f411044d0 0x564f414dc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 -- 172.21.15.94:0/2633690733 >> 172.21.15.94:0/2633690733 conn(0x564f411214e0 msgr2=0x564f414d8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 -- 172.21.15.94:0/2633690733 shutdown_connections 2024-04-25T14:01:29.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.517+0000 7f58b7df7740 1 -- 172.21.15.94:0/2633690733 wait complete. 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 -- start start 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 Processor -- start 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f410c13a0 0x564f414e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f415519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1aa780 con 0x564f411044d0 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1aa0b0 con 0x564f410c13a0 2024-04-25T14:01:29.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac1a72f0 con 0x564f4112d390 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58b7df7740 10 client.0 _ll_register_callbacks cb 0x7f58940e4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42690/0 (socket says 172.21.15.94:42690) 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58a3fff640 1 -- 172.21.15.94:0/3968800918 learned_addr learned my addr 172.21.15.94:0/3968800918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58aa59c640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f410c13a0 0x564f414e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.518+0000 7f58aad9d640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f415519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 -- 172.21.15.94:0/3968800918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 msgr2=0x564f415519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f415519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 -- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f410c13a0 msgr2=0x564f414e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f410c13a0 0x564f414e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 -- 172.21.15.94:0/3968800918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f41551f90 con 0x564f411044d0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58a3fff640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f5894005fd0 tx=0x7f5894004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee49e1342f9c7b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940a6400 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f58940a5f20 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a4036050 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f58940deb20 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 10 client.22503 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f415531f0 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58a4036050 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f58940a2420 con 0x564f411044d0 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 10 client.22503 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 10 client.22503 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 20 client.22503 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f41553990 0x564f41555db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 4 client.22503 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.519+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f4143b800 con 0x564f41553990 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58aa59c640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f41553990 0x564f41555db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58aa59c640 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f41553990 0x564f41555db0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f2a5f72b117d1dd server_cookie=2927650c8e1948a0 in_seq=0 out_seq=0 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f5891ffb640 10 client.22503 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f5891ffb640 1 -- 172.21.15.94:0/3968800918 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f4143b800 con 0x564f41553990 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f5891ffb640 10 client.22503 handle_command_reply: tid=0 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 client.22503 shutdown 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 msgr2=0x564f414e3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f5894005fd0 tx=0x7f5894004400 comp rx=0 tx=0).stop 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 shutdown_connections 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x564f41553990 0x564f41555db0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f4112d390 0x564f415519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f5891ffb640 0 client.22503 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f411044d0 0x564f414e3b50 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.520+0000 7f58b7df7740 1 --2- 172.21.15.94:0/3968800918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f410c13a0 0x564f414e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 >> 172.21.15.94:0/3968800918 conn(0x564f411214e0 msgr2=0x564f414dc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 shutdown_connections 2024-04-25T14:01:29.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b7df7740 1 -- 172.21.15.94:0/3968800918 wait complete. 2024-04-25T14:01:29.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b7df7740 20 client.22503 trim_cache size 0 max 16384 2024-04-25T14:01:29.527 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 msgr2=0x7f58980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 0x7f58980b2000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x564f414d4c70 tx=0x7f58a4009380 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 msgr2=0x7f58ac19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f589400d950 tx=0x7f589400de10 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 shutdown_connections 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f58980afaf0 0x7f58980b2000 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58ac1085f0 0x7f58ac1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58ac105a80 0x7f58ac19ef70 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 --2- 172.21.15.94:0/4289519399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58ac103090 0x7f58ac1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 >> 172.21.15.94:0/4289519399 conn(0x7f58ac0fcf20 msgr2=0x7f58ac103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 shutdown_connections 2024-04-25T14:01:29.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.521+0000 7f58b0f46640 1 -- 172.21.15.94:0/4289519399 wait complete. 2024-04-25T14:01:29.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:01:29.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 -- 172.21.15.94:0/2931690874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 msgr2=0x7f88a4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 --2- 172.21.15.94:0/2931690874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4103700 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).stop 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 -- 172.21.15.94:0/2931690874 shutdown_connections 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 --2- 172.21.15.94:0/2931690874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 0x7f88a410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 --2- 172.21.15.94:0/2931690874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 0x7f88a4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 --2- 172.21.15.94:0/2931690874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4103700 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 -- 172.21.15.94:0/2931690874 >> 172.21.15.94:0/2931690874 conn(0x7f88a40fdbf0 msgr2=0x7f88a41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 -- 172.21.15.94:0/2931690874 shutdown_connections 2024-04-25T14:01:29.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.650+0000 7f88abb0d640 1 -- 172.21.15.94:0/2931690874 wait complete. 2024-04-25T14:01:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 Processor -- start 2024-04-25T14:01:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 -- start start 2024-04-25T14:01:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 0x7f88a419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 0x7f88a41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410e530 con 0x7f88a4102a00 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410e3b0 con 0x7f88a4103c40 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88abb0d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410e6b0 con 0x7f88a4108940 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42714/0 (socket says 172.21.15.94:42714) 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88a98a9640 1 -- 172.21.15.94:0/720101530 learned_addr learned my addr 172.21.15.94:0/720101530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88a90a8640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 0x7f88a419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.651+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 0x7f88a41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 -- 172.21.15.94:0/720101530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 msgr2=0x7f88a41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 0x7f88a41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 -- 172.21.15.94:0/720101530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 msgr2=0x7f88a419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 0x7f88a419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 -- 172.21.15.94:0/720101530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a41a1db0 con 0x7f88a4102a00 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88a98a9640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f88a0031440 tx=0x7f88a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f889affd640 1 -- 172.21.15.94:0/720101530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a003f070 con 0x7f88a4102a00 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a41a2040 con 0x7f88a4102a00 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a2580 con 0x7f88a4102a00 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f889affd640 1 -- 172.21.15.94:0/720101530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a003b910 con 0x7f88a4102a00 2024-04-25T14:01:29.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.652+0000 7f889affd640 1 -- 172.21.15.94:0/720101530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0049060 con 0x7f88a4102a00 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.655+0000 7f889affd640 1 -- 172.21.15.94:0/720101530 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f88a0042070 con 0x7f88a4102a00 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.655+0000 7f88b29c3740 1 Processor -- start 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.655+0000 7f88b29c3740 1 -- start start 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f889affd640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 0x7f88900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556e0141bd80 con 0x556e010ec4d0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556e013d5a20 con 0x556e01115390 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88b29c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e01417d40 con 0x556e010f46b0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88a90a8640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 0x7f88900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40954/0 (socket says 172.21.15.94:40954) 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 -- 172.21.15.94:0/2231459598 learned_addr learned my addr 172.21.15.94:0/2231459598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f889affd640 1 -- 172.21.15.94:0/720101530 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f88a00ff470 con 0x7f88a4102a00 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 -- 172.21.15.94:0/2231459598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 msgr2=0x556e014b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 -- 172.21.15.94:0/2231459598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 msgr2=0x556e014b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88aa0aa640 1 -- 172.21.15.94:0/2231459598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e013e0c30 con 0x556e010f46b0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88a98a9640 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.656+0000 7f88a90a8640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 0x7f88900b2000 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f88a40751e0 tx=0x7f8894005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.657+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f889c009270 tx=0x7f889c009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e01ffca727952f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.657+0000 7f888ffff640 1 -- 172.21.15.94:0/2231459598 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c004b20 con 0x556e010f46b0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.657+0000 7f888ffff640 1 -- 172.21.15.94:0/2231459598 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c003e20 con 0x556e010f46b0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.657+0000 7f888ffff640 1 -- 172.21.15.94:0/2231459598 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c004030 con 0x556e010f46b0 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- 172.21.15.94:0/2231459598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 msgr2=0x556e014b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f889c009270 tx=0x7f889c009a70 comp rx=0 tx=0).stop 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- 172.21.15.94:0/2231459598 shutdown_connections 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014b8a20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2231459598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- 172.21.15.94:0/2231459598 >> 172.21.15.94:0/2231459598 conn(0x556e011094e0 msgr2=0x556e014b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- 172.21.15.94:0/2231459598 shutdown_connections 2024-04-25T14:01:29.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- 172.21.15.94:0/2231459598 wait complete. 2024-04-25T14:01:29.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 -- start start 2024-04-25T14:01:29.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.658+0000 7f88b29c3740 1 Processor -- start 2024-04-25T14:01:29.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556e014b9530 con 0x556e010ec4d0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556e013a2dc0 con 0x556e01115390 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e014b96b0 con 0x556e010f46b0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88b29c3740 10 client.0 _ll_register_callbacks cb 0x556e013be600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88aa0aa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42724/0 (socket says 172.21.15.94:42724) 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88aa0aa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58980/0 (socket says 172.21.15.94:58980) 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a98a9640 1 -- 172.21.15.94:0/2673989435 learned_addr learned my addr 172.21.15.94:0/2673989435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 -- 172.21.15.94:0/2673989435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 msgr2=0x556e014c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 -- 172.21.15.94:0/2673989435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 msgr2=0x556e014c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a90a8640 1 -- 172.21.15.94:0/2673989435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0112d3f0 con 0x556e010f46b0 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.659+0000 7f88a98a9640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.660+0000 7f88a90a8640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014c1bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x556e014b3db0 tx=0x7f8894004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9155e6aff7e31f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.660+0000 7f888d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.660+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894004230 con 0x556e010f46b0 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.660+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f88940043d0 con 0x556e010f46b0 2024-04-25T14:01:29.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.660+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c003e20 con 0x556e010f46b0 2024-04-25T14:01:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f889401c870 con 0x556e010f46b0 2024-04-25T14:01:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 10 client.31579 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e011f77d0 con 0x556e010f46b0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c003e20 con 0x556e010f46b0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f8894003ce0 con 0x556e010f46b0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 10 client.31579 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 10 client.31579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 20 client.31579 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e010d3f60 0x556e014cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 4 client.31579 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e01429990 con 0x556e010d3f60 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.664+0000 7f88a98a9640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e010d3f60 0x556e014cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.665+0000 7f888d7fa640 10 client.31579 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.665+0000 7f88a98a9640 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e010d3f60 0x556e014cc6a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a20d132b8899c23 server_cookie=bf574edc7c9a72b2 in_seq=0 out_seq=0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f888d7fa640 1 -- 172.21.15.94:0/2673989435 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e01429990 con 0x556e010d3f60 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f888d7fa640 10 client.31579 handle_command_reply: tid=0 2024-04-25T14:01:29.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 client.31579 shutdown 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 msgr2=0x556e014c1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014c1bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x556e014b3db0 tx=0x7f8894004730 comp rx=0 tx=0).stop 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 shutdown_connections 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e010d3f60 0x556e014cc6a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e01115390 0x556e014c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014c1bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f888d7fa640 0 client.31579 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 --2- 172.21.15.94:0/2673989435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010ec4d0 0x556e014c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 >> 172.21.15.94:0/2673989435 conn(0x556e011094e0 msgr2=0x556e014b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 shutdown_connections 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 1 -- 172.21.15.94:0/2673989435 wait complete. 2024-04-25T14:01:29.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.666+0000 7f88b29c3740 20 client.31579 trim_cache size 0 max 16384 2024-04-25T14:01:29.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 Processor -- start 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 -- start start 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014bc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f888800e730 con 0x556e010d3f60 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8888010270 con 0x556e010ec4d0 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88b29c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f888800ee00 con 0x556e010f46b0 2024-04-25T14:01:29.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.680+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42736/0 (socket says 172.21.15.94:42736) 2024-04-25T14:01:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a90a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 -- 172.21.15.94:0/526584527 learned_addr learned my addr 172.21.15.94:0/526584527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 -- 172.21.15.94:0/526584527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 msgr2=0x556e014bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 -- 172.21.15.94:0/526584527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 msgr2=0x556e014bc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014bc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 -- 172.21.15.94:0/526584527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e014b9770 con 0x556e010d3f60 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f88a98a9640 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f88a00b0010 tx=0x7f88a003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6987a754ecb143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f888f7fe640 1 -- 172.21.15.94:0/526584527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a00c7960 con 0x556e010d3f60 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f888f7fe640 1 -- 172.21.15.94:0/526584527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00cbc60 con 0x556e010d3f60 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.681+0000 7f888f7fe640 1 -- 172.21.15.94:0/526584527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a00c7bc0 con 0x556e010d3f60 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 -- 172.21.15.94:0/526584527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 msgr2=0x556e014bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f88a00b0010 tx=0x7f88a003f000 comp rx=0 tx=0).stop 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 -- 172.21.15.94:0/526584527 shutdown_connections 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014bc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 --2- 172.21.15.94:0/526584527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014bc3f0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 -- 172.21.15.94:0/526584527 >> 172.21.15.94:0/526584527 conn(0x556e011094e0 msgr2=0x556e014b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 -- 172.21.15.94:0/526584527 shutdown_connections 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.682+0000 7f88b29c3740 1 -- 172.21.15.94:0/526584527 wait complete. 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 -- start start 2024-04-25T14:01:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 Processor -- start 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f888800fba0 con 0x556e010d3f60 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f888800f4d0 con 0x556e010ec4d0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f888800c710 con 0x556e010f46b0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88b29c3740 10 client.0 _ll_register_callbacks cb 0x7f88940099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88a98a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42744/0 (socket says 172.21.15.94:42744) 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88a98a9640 1 -- 172.21.15.94:0/1402569333 learned_addr learned my addr 172.21.15.94:0/1402569333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88a90a8640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.683+0000 7f88aa0aa640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 -- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 msgr2=0x556e014d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 -- 172.21.15.94:0/1402569333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 msgr2=0x556e014d5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014d5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 -- 172.21.15.94:0/1402569333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e014d3fa0 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88a98a9640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036040 tx=0x7f88a00c3ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c040648e061510bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f888d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a00fcfe0 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f88a00c3f50 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a00cbc60 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88a00c7960 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88b29c3740 10 client.22521 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556e01536790 con 0x556e010d3f60 2024-04-25T14:01:29.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.684+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00cbc60 con 0x556e010d3f60 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f88a00d0280 con 0x556e010d3f60 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 10 client.22521 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 10 client.22521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 20 client.22521 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e01115390 0x556e01538f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 4 client.22521 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e01423800 con 0x556e01115390 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88a90a8640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e01115390 0x556e01538f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f888d7fa640 10 client.22521 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88a90a8640 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e01115390 0x556e01538f10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af2ab37b7c3e5f4 server_cookie=f5cdfb07f5037695 in_seq=0 out_seq=0 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f888d7fa640 1 -- 172.21.15.94:0/1402569333 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e01423800 con 0x556e01115390 2024-04-25T14:01:29.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f888d7fa640 10 client.22521 handle_command_reply: tid=0 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.685+0000 7f88b29c3740 1 client.22521 shutdown 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 msgr2=0x556e014d51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036040 tx=0x7f88a00c3ea0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 shutdown_connections 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556e01115390 0x556e01538f10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e010f46b0 0x556e014d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e010ec4d0 0x556e014d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 --2- 172.21.15.94:0/1402569333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e010d3f60 0x556e014d51f0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f888d7fa640 0 client.22521 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 >> 172.21.15.94:0/1402569333 conn(0x556e011094e0 msgr2=0x556e014b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 shutdown_connections 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 1 -- 172.21.15.94:0/1402569333 wait complete. 2024-04-25T14:01:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88b29c3740 20 client.22521 trim_cache size 0 max 16384 2024-04-25T14:01:29.693 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:01:29.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 msgr2=0x7f88900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 0x7f88900b2000 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f88a40751e0 tx=0x7f8894005f90 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 msgr2=0x7f88a4075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.686+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f88a0031440 tx=0x7f88a0003c10 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 shutdown_connections 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f88900afaf0 0x7f88900b2000 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108940 0x7f88a41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88a4103c40 0x7f88a419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 --2- 172.21.15.94:0/720101530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88a4102a00 0x7f88a4075840 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 >> 172.21.15.94:0/720101530 conn(0x7f88a40fdbf0 msgr2=0x7f88a4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 shutdown_connections 2024-04-25T14:01:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.687+0000 7f88abb0d640 1 -- 172.21.15.94:0/720101530 wait complete. 2024-04-25T14:01:29.699 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:01:29.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 -- 172.21.15.94:0/747688791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0810bc40 msgr2=0x7f1f0810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 --2- 172.21.15.94:0/747688791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0810bc40 0x7f1f0810e060 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009900 tx=0x7f1f0402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 -- 172.21.15.94:0/747688791 shutdown_connections 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 --2- 172.21.15.94:0/747688791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f0810bc40 0x7f1f0810e060 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 --2- 172.21.15.94:0/747688791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f08106f40 0x7f1f08107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 --2- 172.21.15.94:0/747688791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08105d00 0x7f1f08106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 -- 172.21.15.94:0/747688791 >> 172.21.15.94:0/747688791 conn(0x7f1f08100ef0 msgr2=0x7f1f081033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 -- 172.21.15.94:0/747688791 shutdown_connections 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.821+0000 7f1f107f3640 1 -- 172.21.15.94:0/747688791 wait complete. 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 Processor -- start 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 -- start start 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08106f40 0x7f1f0819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f0810bc40 0x7f1f081a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08111810 con 0x7f1f08105d00 2024-04-25T14:01:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08111690 con 0x7f1f0810bc40 2024-04-25T14:01:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f107f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08111990 con 0x7f1f08106f40 2024-04-25T14:01:29.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42758/0 (socket says 172.21.15.94:42758) 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 -- 172.21.15.94:0/101239367 learned_addr learned my addr 172.21.15.94:0/101239367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08106f40 0x7f1f0819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f0810bc40 0x7f1f081a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 -- 172.21.15.94:0/101239367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08106f40 msgr2=0x7f1f0819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08106f40 0x7f1f0819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 -- 172.21.15.94:0/101239367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f0810bc40 msgr2=0x7f1f081a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f0810bc40 0x7f1f081a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.822+0000 7f1f0e58f640 1 -- 172.21.15.94:0/101239367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f081a3ef0 con 0x7f1f08105d00 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000eb90 tx=0x7f1f0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1eff7fe640 1 -- 172.21.15.94:0/101239367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00003b70 con 0x7f1f08105d00 2024-04-25T14:01:29.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1eff7fe640 1 -- 172.21.15.94:0/101239367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f0000ee90 con 0x7f1f08105d00 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1eff7fe640 1 -- 172.21.15.94:0/101239367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000197b0 con 0x7f1f08105d00 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f081a41e0 con 0x7f1f08105d00 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.823+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a4730 con 0x7f1f08105d00 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.825+0000 7f1f17780740 1 Processor -- start 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 -- start start 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4d54d0 0x56425f8a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56425f804d80 con 0x56425f4fe390 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56425f7bea20 con 0x56425f4d54d0 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f17780740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56425f800d40 con 0x56425f4dd6b0 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1eff7fe640 1 -- 172.21.15.94:0/101239367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1f00004030 con 0x7f1f08105d00 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42768/0 (socket says 172.21.15.94:42768) 2024-04-25T14:01:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3686383212 learned_addr learned my addr 172.21.15.94:0/3686383212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4d54d0 0x56425f8a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3686383212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 msgr2=0x56425f8a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3686383212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4d54d0 msgr2=0x56425f8a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.826+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4d54d0 0x56425f8a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3686383212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425f7c9c30 con 0x56425f4fe390 2024-04-25T14:01:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009900 tx=0x7f1f04009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f43e72368f5c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1eff7fe640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 0x7f1ef40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1efcff9640 1 -- 172.21.15.94:0/3686383212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04035030 con 0x56425f4fe390 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1efcff9640 1 -- 172.21.15.94:0/3686383212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f04040070 con 0x56425f4fe390 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1efcff9640 1 -- 172.21.15.94:0/3686383212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003ab0 con 0x56425f4fe390 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 0x7f1ef40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.827+0000 7f1eff7fe640 1 -- 172.21.15.94:0/101239367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1f000df540 con 0x7f1f08105d00 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- 172.21.15.94:0/3686383212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 msgr2=0x56425f8a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f0e58f640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 0x7f1ef40b2000 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1f00005fa0 tx=0x7f1f00000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009900 tx=0x7f1f04009620 comp rx=0 tx=0).stop 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- 172.21.15.94:0/3686383212 shutdown_connections 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4d54d0 0x56425f8a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 --2- 172.21.15.94:0/3686383212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4fe390 0x56425f8a0810 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- 172.21.15.94:0/3686383212 >> 172.21.15.94:0/3686383212 conn(0x56425f4f24e0 msgr2=0x56425f89c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- 172.21.15.94:0/3686383212 shutdown_connections 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- 172.21.15.94:0/3686383212 wait complete. 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 -- start start 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.828+0000 7f1f17780740 1 Processor -- start 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4fe390 0x56425f8b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56425f8a24d0 con 0x56425f4d54d0 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56425f78bdc0 con 0x56425f4fe390 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56425f8a2650 con 0x56425f4dd6b0 2024-04-25T14:01:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f17780740 10 client.0 _ll_register_callbacks cb 0x56425f7a7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42780/0 (socket says 172.21.15.94:42780) 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3324167953 learned_addr learned my addr 172.21.15.94:0/3324167953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4fe390 0x56425f8b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3324167953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 msgr2=0x56425f8aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4fe390 msgr2=0x56425f8b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4fe390 0x56425f8b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 -- 172.21.15.94:0/3324167953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425f5163f0 con 0x56425f4d54d0 2024-04-25T14:01:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402a930 tx=0x7f1f0402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f170ccf98382b66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.829+0000 7f1ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04035050 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1f04003830 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f04040070 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1f04003a30 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 10 client.22533 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56425f5e07d0 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f04040070 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1f04053a40 con 0x56425f4d54d0 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 10 client.22533 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 10 client.22533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 20 client.22533 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4bcf60 0x56425f8b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 4 client.22533 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425f812990 con 0x56425f4bcf60 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.830+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4bcf60 0x56425f8b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.831+0000 7f1ef1ffb640 10 client.22533 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.831+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4bcf60 0x56425f8b7e20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bb41050266646d7 server_cookie=f77edbfc75419c7e in_seq=0 out_seq=0 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/3324167953 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56425f812990 con 0x56425f4bcf60 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1ef1ffb640 10 client.22533 handle_command_reply: tid=0 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 client.22533 shutdown 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 msgr2=0x56425f8aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402a930 tx=0x7f1f0402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 shutdown_connections 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4bcf60 0x56425f8b7e20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4fe390 0x56425f8b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1ef1ffb640 0 client.22533 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 --2- 172.21.15.94:0/3324167953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8aa530 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 >> 172.21.15.94:0/3324167953 conn(0x56425f4f24e0 msgr2=0x56425f89b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 shutdown_connections 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 1 -- 172.21.15.94:0/3324167953 wait complete. 2024-04-25T14:01:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.832+0000 7f1f17780740 20 client.22533 trim_cache size 0 max 16384 2024-04-25T14:01:29.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 Processor -- start 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 -- start start 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081ad680 con 0x56425f4d54d0 2024-04-25T14:01:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081af1c0 con 0x56425f4bcf60 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f17780740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081add50 con 0x56425f4dd6b0 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42788/0 (socket says 172.21.15.94:42788) 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/3480359594 learned_addr learned my addr 172.21.15.94:0/3480359594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/3480359594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 msgr2=0x56425f8bc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8bc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/3480359594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 msgr2=0x56425f8aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.847+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/3480359594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425f8a0360 con 0x56425f4d54d0 2024-04-25T14:01:29.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.848+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x56425f89cdb0 tx=0x7f1ef8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a331328b592e7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.848+0000 7f1ef3fff640 1 -- 172.21.15.94:0/3480359594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8004230 con 0x56425f4d54d0 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.848+0000 7f1ef3fff640 1 -- 172.21.15.94:0/3480359594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef80043d0 con 0x56425f4d54d0 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.848+0000 7f1ef3fff640 1 -- 172.21.15.94:0/3480359594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80205d0 con 0x56425f4d54d0 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- 172.21.15.94:0/3480359594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 msgr2=0x56425f8b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x56425f89cdb0 tx=0x7f1ef8005f90 comp rx=0 tx=0).stop 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- 172.21.15.94:0/3480359594 shutdown_connections 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4dd6b0 0x56425f8bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4d54d0 0x56425f8b1960 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- 172.21.15.94:0/3480359594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- 172.21.15.94:0/3480359594 >> 172.21.15.94:0/3480359594 conn(0x56425f4f24e0 msgr2=0x56425f8aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- 172.21.15.94:0/3480359594 shutdown_connections 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- 172.21.15.94:0/3480359594 wait complete. 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- start start 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 Processor -- start 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4dd6b0 0x56425f8b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081aeaf0 con 0x56425f4dd6b0 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081ae420 con 0x56425f4bcf60 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.849+0000 7f1f17780740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f081ab660 con 0x56425f4d54d0 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f17780740 10 client.0 _ll_register_callbacks cb 0x7f1f000e2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41032/0 (socket says 172.21.15.94:41032) 2024-04-25T14:01:29.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/2340129643 learned_addr learned my addr 172.21.15.94:0/2340129643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0ed90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4dd6b0 0x56425f8b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 msgr2=0x56425f8af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/2340129643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4dd6b0 msgr2=0x56425f8b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4dd6b0 0x56425f8b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 -- 172.21.15.94:0/2340129643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425f8a5cc0 con 0x56425f4d54d0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1f0dd8e640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8005700 tx=0x7f1ef8005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0c3ef33dc234fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.850+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80035f0 con 0x56425f4d54d0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef80043d0 con 0x56425f4d54d0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1ef8003790 con 0x56425f4d54d0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1ef8003990 con 0x56425f4d54d0 2024-04-25T14:01:29.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 10 client.31603 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56425f8b1cd0 con 0x56425f4d54d0 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef80043d0 con 0x56425f4d54d0 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1ef8034500 con 0x56425f4d54d0 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 10 client.31603 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 10 client.31603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 20 client.31603 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4fe390 0x56425f9227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 4 client.31603 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.851+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56425f80c800 con 0x56425f4fe390 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4fe390 0x56425f9227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1f0ed90640 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4fe390 0x56425f9227f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62ea222be72bd0c7 server_cookie=c6ca2042e4f2f8c4 in_seq=0 out_seq=0 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1ef1ffb640 10 client.31603 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1ef1ffb640 1 -- 172.21.15.94:0/2340129643 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56425f80c800 con 0x56425f4fe390 2024-04-25T14:01:29.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1ef1ffb640 10 client.31603 handle_command_reply: tid=0 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.852+0000 7f1f17780740 1 client.31603 shutdown 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 msgr2=0x56425f8af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8005700 tx=0x7f1ef8005730 comp rx=0 tx=0).stop 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 shutdown_connections 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56425f4fe390 0x56425f9227f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56425f4dd6b0 0x56425f8b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1ef1ffb640 0 client.31603 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56425f4d54d0 0x56425f8af700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 --2- 172.21.15.94:0/2340129643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56425f4bcf60 0x56425f8af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 >> 172.21.15.94:0/2340129643 conn(0x56425f4f24e0 msgr2=0x56425f89eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 shutdown_connections 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 1 -- 172.21.15.94:0/2340129643 wait complete. 2024-04-25T14:01:29.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f17780740 20 client.31603 trim_cache size 0 max 16384 2024-04-25T14:01:29.871 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:01:29.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 msgr2=0x7f1ef40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 0x7f1ef40b2000 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1f00005fa0 tx=0x7f1f00000f40 comp rx=0 tx=0).stop 2024-04-25T14:01:29.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 msgr2=0x7f1f0819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000eb90 tx=0x7f1f0000c6a0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 shutdown_connections 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1ef40afaf0 0x7f1ef40b2000 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f0810bc40 0x7f1f081a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f08106f40 0x7f1f0819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.853+0000 7f1f107f3640 1 --2- 172.21.15.94:0/101239367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f08105d00 0x7f1f0819c3b0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.854+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 >> 172.21.15.94:0/101239367 conn(0x7f1f08100ef0 msgr2=0x7f1f08107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.854+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 shutdown_connections 2024-04-25T14:01:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.854+0000 7f1f107f3640 1 -- 172.21.15.94:0/101239367 wait complete. 2024-04-25T14:01:29.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:01:29.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 -- 172.21.15.94:0/3992081852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 msgr2=0x7f30c01090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/3992081852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c01090f0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009900 tx=0x7f30ac02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 -- 172.21.15.94:0/3992081852 shutdown_connections 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/3992081852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0109780 0x7f30c010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/3992081852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c01090f0 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/3992081852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0101e30 0x7f30c0102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 -- 172.21.15.94:0/3992081852 >> 172.21.15.94:0/3992081852 conn(0x7f30c00fd340 msgr2=0x7f30c00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 -- 172.21.15.94:0/3992081852 shutdown_connections 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.993+0000 7f30c6e0c640 1 -- 172.21.15.94:0/3992081852 wait complete. 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 Processor -- start 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 -- start start 2024-04-25T14:01:29.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0101e30 0x7f30c019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 0x7f30c01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010f610 con 0x7f30c0106c10 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010f490 con 0x7f30c0101e30 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c6e0c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30c010f790 con 0x7f30c0109780 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42814/0 (socket says 172.21.15.94:42814) 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0101e30 0x7f30c019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30bffff640 1 -- 172.21.15.94:0/4218865578 learned_addr learned my addr 172.21.15.94:0/4218865578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.994+0000 7f30c53a9640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 0x7f30c01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 -- 172.21.15.94:0/4218865578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 msgr2=0x7f30c01a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 0x7f30c01a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 -- 172.21.15.94:0/4218865578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0101e30 msgr2=0x7f30c019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0101e30 0x7f30c019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 -- 172.21.15.94:0/4218865578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c01a1d80 con 0x7f30c0106c10 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30c53a9640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 0x7f30c01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bffff640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f30ac006010 tx=0x7f30ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bdffb640 1 -- 172.21.15.94:0/4218865578 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac03f030 con 0x7f30c0106c10 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c01a2010 con 0x7f30c0106c10 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bdffb640 1 -- 172.21.15.94:0/4218865578 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac003aa0 con 0x7f30c0106c10 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30bdffb640 1 -- 172.21.15.94:0/4218865578 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac035070 con 0x7f30c0106c10 2024-04-25T14:01:29.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.995+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c01a2550 con 0x7f30c0106c10 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 Processor -- start 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 -- start start 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67836b0 0x5629c6b47a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5629c6aaad80 con 0x5629c67836b0 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5629c6a64a20 con 0x5629c677b4d0 2024-04-25T14:01:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30cdd97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629c6aa6d40 con 0x5629c67a4390 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30bdffb640 1 -- 172.21.15.94:0/4218865578 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f30ac042030 con 0x7f30c0106c10 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41054/0 (socket says 172.21.15.94:41054) 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 -- 172.21.15.94:0/1663932106 learned_addr learned my addr 172.21.15.94:0/1663932106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59088/0 (socket says 172.21.15.94:59088) 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30bffff640 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67836b0 0x5629c6b47a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 -- 172.21.15.94:0/1663932106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 msgr2=0x5629c6b46d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 -- 172.21.15.94:0/1663932106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67836b0 msgr2=0x5629c6b47a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67836b0 0x5629c6b47a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c53a9640 1 -- 172.21.15.94:0/1663932106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c6a6fc30 con 0x5629c67a4390 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.998+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30c53a9640 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f30b80107e0 tx=0x7f30b8010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f424330d189d078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30bdffb640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 0x7f30b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30aaffd640 1 -- 172.21.15.94:0/1663932106 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8003b70 con 0x5629c67a4390 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 0x7f30b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30aaffd640 1 -- 172.21.15.94:0/1663932106 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b800ede0 con 0x5629c67a4390 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30bdffb640 1 -- 172.21.15.94:0/4218865578 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f30ac0ff250 con 0x7f30c0106c10 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:29.999+0000 7f30aaffd640 1 -- 172.21.15.94:0/1663932106 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80046b0 con 0x5629c67a4390 2024-04-25T14:01:30.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- 172.21.15.94:0/1663932106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 msgr2=0x5629c6b46810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 0x7f30b00b2000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5629c6b46530 tx=0x7f30b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f30b80107e0 tx=0x7f30b8010ca0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- 172.21.15.94:0/1663932106 shutdown_connections 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67836b0 0x5629c6b47a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c677b4d0 0x5629c6b46d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 --2- 172.21.15.94:0/1663932106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c67a4390 0x5629c6b46810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- 172.21.15.94:0/1663932106 >> 172.21.15.94:0/1663932106 conn(0x5629c67984e0 msgr2=0x5629c6b42020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- 172.21.15.94:0/1663932106 shutdown_connections 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- 172.21.15.94:0/1663932106 wait complete. 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.000+0000 7f30cdd97740 1 -- start start 2024-04-25T14:01:30.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 Processor -- start 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 0x5629c6b57a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5629c6b48580 con 0x5629c67a4390 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5629c6a31dc0 con 0x5629c67836b0 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629c6b48700 con 0x5629c677b4d0 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30cdd97740 10 client.0 _ll_register_callbacks cb 0x5629c6a4d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c53a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 0x5629c6b57a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41058/0 (socket says 172.21.15.94:41058) 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 -- 172.21.15.94:0/3764821524 learned_addr learned my addr 172.21.15.94:0/3764821524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30bffff640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 -- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 msgr2=0x5629c6b50af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 -- 172.21.15.94:0/3764821524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 msgr2=0x5629c6b57a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 0x5629c6b57a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c4ba8640 1 -- 172.21.15.94:0/3764821524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c67bc3f0 con 0x5629c677b4d0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.001+0000 7f30c53a9640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 0x5629c6b57a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f30b401e950 tx=0x7f30b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4710bce31b80121c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b402e060 con 0x5629c677b4d0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f30b4018030 con 0x5629c677b4d0 2024-04-25T14:01:30.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b800ede0 con 0x5629c677b4d0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f30b40098f0 con 0x5629c677b4d0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f30cdd97740 10 client.31615 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629c68867d0 con 0x5629c677b4d0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b800ede0 con 0x5629c677b4d0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.002+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f30b401c5d0 con 0x5629c677b4d0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 10 client.31615 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 10 client.31615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 20 client.31615 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c6762f60 0x5629c6b5e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 4 client.31615 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c6ab8990 con 0x5629c6762f60 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30bffff640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c6762f60 0x5629c6b5e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f309ffff640 10 client.31615 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.003+0000 7f30bffff640 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c6762f60 0x5629c6b5e380 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664e6e5735241b55 server_cookie=7e288cb7b7bc9ed7 in_seq=0 out_seq=0 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f309ffff640 1 -- 172.21.15.94:0/3764821524 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629c6ab8990 con 0x5629c6762f60 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f309ffff640 10 client.31615 handle_command_reply: tid=0 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 client.31615 shutdown 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 msgr2=0x5629c6b505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f30b401e950 tx=0x7f30b4004730 comp rx=0 tx=0).stop 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 shutdown_connections 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c6762f60 0x5629c6b5e380 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c67a4390 0x5629c6b57a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f309ffff640 0 client.31615 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3764821524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c677b4d0 0x5629c6b505b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 >> 172.21.15.94:0/3764821524 conn(0x5629c67984e0 msgr2=0x5629c6b41040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 shutdown_connections 2024-04-25T14:01:30.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.005+0000 7f30cdd97740 1 -- 172.21.15.94:0/3764821524 wait complete. 2024-04-25T14:01:30.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.006+0000 7f30cdd97740 20 client.31615 trim_cache size 0 max 16384 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 -- 172.21.15.94:0/2115216849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40102a40 msgr2=0x7fcd40103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/2115216849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40102a40 0x7fcd40103740 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd3002a410 comp rx=0 tx=0).stop 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 -- 172.21.15.94:0/2115216849 shutdown_connections 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/2115216849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40108980 0x7fcd4010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/2115216849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd40104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/2115216849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40102a40 0x7fcd40103740 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 -- 172.21.15.94:0/2115216849 >> 172.21.15.94:0/2115216849 conn(0x7fcd400fdc30 msgr2=0x7fcd401000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 -- 172.21.15.94:0/2115216849 shutdown_connections 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.009+0000 7fcd45fb2640 1 -- 172.21.15.94:0/2115216849 wait complete. 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 Processor -- start 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 -- start start 2024-04-25T14:01:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 0x7fcd4019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4010e540 con 0x7fcd40108980 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4010e3c0 con 0x7fcd40102a40 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd45fb2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4010e6c0 con 0x7fcd40103c80 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 0x7fcd4019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41072/0 (socket says 172.21.15.94:41072) 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3effd640 1 -- 172.21.15.94:0/3955087471 learned_addr learned my addr 172.21.15.94:0/3955087471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.010+0000 7fcd3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42854/0 (socket says 172.21.15.94:42854) 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 -- 172.21.15.94:0/3955087471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 msgr2=0x7fcd4019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 -- 172.21.15.94:0/3955087471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 msgr2=0x7fcd4019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 0x7fcd4019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 -- 172.21.15.94:0/3955087471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd401a3fd0 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3effd640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 0x7fcd4019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3ffff640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ec50 tx=0x7fcd3800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3cff9640 1 -- 172.21.15.94:0/3955087471 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38003b70 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3cff9640 1 -- 172.21.15.94:0/3955087471 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd38003d10 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd401a42c0 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd3cff9640 1 -- 172.21.15.94:0/3955087471 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38010830 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.011+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd401a47c0 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.014+0000 7fcd4ce64740 1 Processor -- start 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd3cff9640 1 -- 172.21.15.94:0/3955087471 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcd38003ec0 con 0x7fcd40108980 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 -- start start 2024-04-25T14:01:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592a6713460 con 0x5592a63e42f0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592a66cd590 con 0x5592a63ec4d0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.015+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592a670f420 con 0x5592a640d1b0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42866/0 (socket says 172.21.15.94:42866) 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/4016899516 learned_addr learned my addr 172.21.15.94:0/4016899516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3effd640 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/4016899516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 msgr2=0x5592a67ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3cff9640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 0x7fcd280b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/4016899516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 msgr2=0x5592a67b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/4016899516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a66da0d0 con 0x5592a63e42f0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3ffff640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 0x7fcd280b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009230 tx=0x7fcd30032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aed6f5ee8a07dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd3cff9640 1 -- 172.21.15.94:0/3955087471 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcd380df1d0 con 0x7fcd40108980 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.016+0000 7fcd25ffb640 1 -- 172.21.15.94:0/4016899516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30035070 con 0x5592a63e42f0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd25ffb640 1 -- 172.21.15.94:0/4016899516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd300317a0 con 0x5592a63e42f0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd25ffb640 1 -- 172.21.15.94:0/4016899516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30003e10 con 0x5592a63e42f0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 -- 172.21.15.94:0/4016899516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 msgr2=0x5592a67adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009230 tx=0x7fcd30032f90 comp rx=0 tx=0).stop 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd3ffff640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 0x7fcd280b1ff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd380dcc80 tx=0x7fcd38071e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 -- 172.21.15.94:0/4016899516 shutdown_connections 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67adef0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.017+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/4016899516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- 172.21.15.94:0/4016899516 >> 172.21.15.94:0/4016899516 conn(0x5592a6401300 msgr2=0x5592a67a9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- 172.21.15.94:0/4016899516 shutdown_connections 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- 172.21.15.94:0/4016899516 wait complete. 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- start start 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 Processor -- start 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5592a67b57e0 con 0x5592a63e42f0 2024-04-25T14:01:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592a669ab60 con 0x5592a63ec4d0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.018+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5592a67b5960 con 0x5592a640d1b0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd4ce64740 10 client.0 _ll_register_callbacks cb 0x5592a66bf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42870/0 (socket says 172.21.15.94:42870) 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 -- 172.21.15.94:0/3651998847 learned_addr learned my addr 172.21.15.94:0/3651998847 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 -- 172.21.15.94:0/3651998847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 msgr2=0x5592a67bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 -- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 msgr2=0x5592a67b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 -- 172.21.15.94:0/3651998847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a6425210 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd3effd640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fcd34011fd0 tx=0x7fcd340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a2c6f6eade4352 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd1effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34005020 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcd340051c0 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd300317a0 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.019+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcd340053c0 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 10 client.22563 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592a64f1620 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd300317a0 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcd34020660 con 0x5592a63e42f0 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 10 client.22563 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 10 client.22563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 20 client.22563 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63cbd80 0x5592a67c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 4 client.22563 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a6721070 con 0x5592a63cbd80 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.020+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63cbd80 0x5592a67c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.021+0000 7fcd1effd640 10 client.22563 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.021+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63cbd80 0x5592a67c52f0 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dffcffd6b6106f65 server_cookie=579189f5c9ac2388 in_seq=0 out_seq=0 2024-04-25T14:01:30.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 Processor -- start 2024-04-25T14:01:30.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 -- start start 2024-04-25T14:01:30.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b4f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42872/0 (socket says 172.21.15.94:42872) 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b4f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01ab530 con 0x5629c677b4d0 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01ad070 con 0x5629c67836b0 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30bffff640 1 -- 172.21.15.94:0/3492656091 learned_addr learned my addr 172.21.15.94:0/3492656091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.024+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01abc00 con 0x5629c6762f60 2024-04-25T14:01:30.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30c53a9640 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 -- 172.21.15.94:0/3492656091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 msgr2=0x5629c6b4f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b4f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 -- 172.21.15.94:0/3492656091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 msgr2=0x5629c6b50180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 -- 172.21.15.94:0/3492656091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c6b4bc80 con 0x5629c677b4d0 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30bffff640 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0c7440 tx=0x7f30ac101760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5035e4bcc3a22e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30aa7fc640 1 -- 172.21.15.94:0/3492656091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0fe040 con 0x5629c677b4d0 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30aa7fc640 1 -- 172.21.15.94:0/3492656091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30ac03b460 con 0x5629c677b4d0 2024-04-25T14:01:30.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.025+0000 7f30aa7fc640 1 -- 172.21.15.94:0/3492656091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac049040 con 0x5629c677b4d0 2024-04-25T14:01:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd1effd640 1 -- 172.21.15.94:0/3651998847 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592a6721070 con 0x5592a63cbd80 2024-04-25T14:01:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd1effd640 10 client.22563 handle_command_reply: tid=0 2024-04-25T14:01:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 client.22563 shutdown 2024-04-25T14:01:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 msgr2=0x5592a67b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fcd34011fd0 tx=0x7fcd340046c0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 msgr2=0x5629c6b579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0c7440 tx=0x7f30ac101760 comp rx=0 tx=0).stop 2024-04-25T14:01:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 shutdown_connections 2024-04-25T14:01:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63cbd80 0x5592a67c52f0 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a640d1b0 0x5592a67bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a63ec4d0 0x5592a67b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/3651998847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67b75f0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 shutdown_connections 2024-04-25T14:01:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b579e0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3492656091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b4f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 >> 172.21.15.94:0/3651998847 conn(0x5592a6401300 msgr2=0x5592a67ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 >> 172.21.15.94:0/3492656091 conn(0x5629c67984e0 msgr2=0x5629c6b4ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd1effd640 0 client.22563 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 shutdown_connections 2024-04-25T14:01:30.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 shutdown_connections 2024-04-25T14:01:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.026+0000 7fcd4ce64740 1 -- 172.21.15.94:0/3651998847 wait complete. 2024-04-25T14:01:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.026+0000 7f30cdd97740 1 -- 172.21.15.94:0/3492656091 wait complete. 2024-04-25T14:01:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.027+0000 7fcd4ce64740 20 client.22563 trim_cache size 0 max 16384 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.027+0000 7f30cdd97740 1 -- start start 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.027+0000 7f30cdd97740 1 Processor -- start 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.027+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.027+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01ac9a0 con 0x5629c677b4d0 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01ac2d0 con 0x5629c67836b0 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30c01a9510 con 0x5629c6762f60 2024-04-25T14:01:30.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 10 client.0 _ll_register_callbacks cb 0x7f30b4009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30c4ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42878/0 (socket says 172.21.15.94:42878) 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 -- 172.21.15.94:0/3628569466 learned_addr learned my addr 172.21.15.94:0/3628569466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30c53a9640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 -- 172.21.15.94:0/3628569466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 msgr2=0x5629c6b553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 -- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 msgr2=0x5629c6b56810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b56810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 -- 172.21.15.94:0/3628569466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c6b56dd0 con 0x5629c677b4d0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30bffff640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0c3040 tx=0x7f30ac0c36c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ef70da8bfa2786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f309ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:30.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac049070 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f30ac003c50 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.028+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629c6ab2800 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f30ac0c7510 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 10 client.22569 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629c6bc66f0 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5629c6ab2800 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f30ac0aebe0 con 0x5629c677b4d0 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 10 client.22569 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 10 client.22569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:30.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 20 client.22569 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c67a4390 0x5629c6bc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 4 client.22569 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629c6a2d610 con 0x5629c67a4390 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.029+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c67a4390 0x5629c6bc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.030+0000 7f309ffff640 10 client.22569 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.030+0000 7f30c4ba8640 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c67a4390 0x5629c6bc90e0 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b4bdd657671141 server_cookie=2583630493a69694 in_seq=0 out_seq=0 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f309ffff640 1 -- 172.21.15.94:0/3628569466 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629c6a2d610 con 0x5629c67a4390 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f309ffff640 10 client.22569 handle_command_reply: tid=0 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 client.22569 shutdown 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 msgr2=0x5629c6b558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0c3040 tx=0x7f30ac0c36c0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 shutdown_connections 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5629c67a4390 0x5629c6bc90e0 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5629c67836b0 0x5629c6b56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629c677b4d0 0x5629c6b558e0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 --2- 172.21.15.94:0/3628569466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629c6762f60 0x5629c6b553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 >> 172.21.15.94:0/3628569466 conn(0x5629c67984e0 msgr2=0x5629c6b50160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f309ffff640 0 client.22569 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 shutdown_connections 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.032+0000 7f30cdd97740 1 -- 172.21.15.94:0/3628569466 wait complete. 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30cdd97740 20 client.22569 trim_cache size 0 max 16384 2024-04-25T14:01:30.048 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:01:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 msgr2=0x7f30b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 0x7f30b00b2000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5629c6b46530 tx=0x7f30b4005f90 comp rx=0 tx=0).stop 2024-04-25T14:01:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 msgr2=0x7f30c019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f30ac006010 tx=0x7f30ac031a50 comp rx=0 tx=0).stop 2024-04-25T14:01:30.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 shutdown_connections 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30b00afaf0 0x7f30b00b2000 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30c0109780 0x7f30c01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c0106c10 0x7f30c019a780 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 --2- 172.21.15.94:0/4218865578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0101e30 0x7f30c019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 >> 172.21.15.94:0/4218865578 conn(0x7f30c00fd340 msgr2=0x7f30c010a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 shutdown_connections 2024-04-25T14:01:30.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:30.033+0000 7f30c6e0c640 1 -- 172.21.15.94:0/4218865578 wait complete. 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 Processor -- start 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 -- start start 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ad7b0 con 0x5592a63e42f0 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401af2f0 con 0x5592a640d1b0 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ade80 con 0x5592a63ec4d0 2024-04-25T14:01:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41114/0 (socket says 172.21.15.94:41114) 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42894/0 (socket says 172.21.15.94:42894) 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/2526533600 learned_addr learned my addr 172.21.15.94:0/2526533600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 -- 172.21.15.94:0/2526533600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 msgr2=0x5592a67bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 -- 172.21.15.94:0/2526533600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 msgr2=0x5592a67ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.044+0000 7fcd3effd640 1 -- 172.21.15.94:0/2526533600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a67ad480 con 0x5592a63e42f0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.045+0000 7fcd3effd640 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fcd34011fa0 tx=0x7fcd34005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9f0fe2e11c1e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.045+0000 7fcd257fa640 1 -- 172.21.15.94:0/2526533600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34005d30 con 0x5592a63e42f0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.045+0000 7fcd257fa640 1 -- 172.21.15.94:0/2526533600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd34005560 con 0x5592a63e42f0 2024-04-25T14:01:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.045+0000 7fcd257fa640 1 -- 172.21.15.94:0/2526533600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3402cb40 con 0x5592a63e42f0 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- 172.21.15.94:0/2526533600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 msgr2=0x5592a67a93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fcd34011fa0 tx=0x7fcd34005bb0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- 172.21.15.94:0/2526533600 shutdown_connections 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/2526533600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67a93d0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- 172.21.15.94:0/2526533600 >> 172.21.15.94:0/2526533600 conn(0x5592a6401300 msgr2=0x5592a67a8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- 172.21.15.94:0/2526533600 shutdown_connections 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- 172.21.15.94:0/2526533600 wait complete. 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.046+0000 7fcd4ce64740 1 -- start start 2024-04-25T14:01:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 Processor -- start 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401aec20 con 0x5592a63e42f0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ae550 con 0x5592a640d1b0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd401ab790 con 0x5592a63ec4d0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd4ce64740 10 client.0 _ll_register_callbacks cb 0x7fcd3400f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42902/0 (socket says 172.21.15.94:42902) 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3effd640 1 -- 172.21.15.94:0/505706874 learned_addr learned my addr 172.21.15.94:0/505706874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3effd640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 msgr2=0x5592a67bd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67bd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/505706874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 msgr2=0x5592a67bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 -- 172.21.15.94:0/505706874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a67b2500 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3effd640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.047+0000 7fcd3f7fe640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bc370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009850 tx=0x7fcd300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f0ef11f7fd00fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.048+0000 7fcd1effd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30057070 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcd30004040 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd34005560 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcd30063050 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd4ce64740 10 client.31639 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.050+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592a67be9f0 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd34005560 con 0x5592a63ec4d0 2024-04-25T14:01:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcd30003b30 con 0x5592a63ec4d0 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 10 client.31639 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 10 client.31639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 20 client.31639 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63a11c0 0x5592a682f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 4 client.31639 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592a671aee0 con 0x5592a63a11c0 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd3effd640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63a11c0 0x5592a682f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.051+0000 7fcd1effd640 10 client.31639 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.052+0000 7fcd3effd640 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63a11c0 0x5592a682f6b0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b85476b0813b9fe server_cookie=b68b15f470733e9c in_seq=0 out_seq=0 2024-04-25T14:01:30.149 INFO:tasks.workunit.client.0.smithi094.stdout: 10 134220 19.17 MB/sec execute 252 sec latency 114.569 ms 2024-04-25T14:01:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:30 smithi137 ceph-mon[32568]: pgmap v942: 97 pgs: 97 active+clean; 264 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-04-25T14:01:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:30.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:01:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:30 smithi094 ceph-mon[29195]: pgmap v942: 97 pgs: 97 active+clean; 264 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-04-25T14:01:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:01:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:30 smithi138 ceph-mon[32977]: pgmap v942: 97 pgs: 97 active+clean; 264 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 485 op/s 2024-04-25T14:01:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:30.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:30.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd1effd640 1 -- 172.21.15.94:0/505706874 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592a671aee0 con 0x5592a63a11c0 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd1effd640 10 client.31639 handle_command_reply: tid=0 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 client.31639 shutdown 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 msgr2=0x5592a67bc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bc370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009850 tx=0x7fcd300046c0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 shutdown_connections 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5592a63a11c0 0x5592a682f6b0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592a640d1b0 0x5592a67bd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5592a63ec4d0 0x5592a67bc370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 --2- 172.21.15.94:0/505706874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5592a63e42f0 0x5592a67bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd1effd640 0 client.31639 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 >> 172.21.15.94:0/505706874 conn(0x5592a6401300 msgr2=0x5592a67ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 shutdown_connections 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.615+0000 7fcd4ce64740 1 -- 172.21.15.94:0/505706874 wait complete. 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd4ce64740 20 client.31639 trim_cache size 0 max 16384 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:01:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:01:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:01:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:01:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 msgr2=0x7fcd280b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 0x7fcd280b1ff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd380dcc80 tx=0x7fcd38071e30 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 msgr2=0x7fcd401a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ec50 tx=0x7fcd3800c6a0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 shutdown_connections 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcd280afae0 0x7fcd280b1ff0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd40108980 0x7fcd401a3960 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd40103c80 0x7fcd4019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 --2- 172.21.15.94:0/3955087471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd40102a40 0x7fcd4019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 >> 172.21.15.94:0/3955087471 conn(0x7fcd400fdc30 msgr2=0x7fcd40100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 shutdown_connections 2024-04-25T14:01:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:30.616+0000 7fcd45fb2640 1 -- 172.21.15.94:0/3955087471 wait complete. 2024-04-25T14:01:31.055 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:01:31.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:01:31.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:01:31.149 INFO:tasks.workunit.client.0.smithi094.stdout: 10 134477 19.11 MB/sec execute 253 sec latency 355.121 ms 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.177+0000 7f1c221e0640 1 -- 172.21.15.94:0/2252954216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 msgr2=0x7f1c1c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.177+0000 7f1c221e0640 1 --2- 172.21.15.94:0/2252954216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c1036f0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 -- 172.21.15.94:0/2252954216 shutdown_connections 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 --2- 172.21.15.94:0/2252954216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 0x7f1c1c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 --2- 172.21.15.94:0/2252954216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 0x7f1c1c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 --2- 172.21.15.94:0/2252954216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c1036f0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 -- 172.21.15.94:0/2252954216 >> 172.21.15.94:0/2252954216 conn(0x7f1c1c0fdc00 msgr2=0x7f1c1c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 -- 172.21.15.94:0/2252954216 shutdown_connections 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 -- 172.21.15.94:0/2252954216 wait complete. 2024-04-25T14:01:31.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 Processor -- start 2024-04-25T14:01:31.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.178+0000 7f1c221e0640 1 -- start start 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 0x7f1c1c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 0x7f1c1c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e4f0 con 0x7f1c1c1029f0 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e370 con 0x7f1c1c108930 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e670 con 0x7f1c1c103c30 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42912/0 (socket says 172.21.15.94:42912) 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 -- 172.21.15.94:0/4208380635 learned_addr learned my addr 172.21.15.94:0/4208380635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 0x7f1c1c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1affd640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 0x7f1c1c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 -- 172.21.15.94:0/4208380635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 msgr2=0x7f1c1c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 0x7f1c1c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 -- 172.21.15.94:0/4208380635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 msgr2=0x7f1c1c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 0x7f1c1c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 -- 172.21.15.94:0/4208380635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c1a0c90 con 0x7f1c1c1029f0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006010 tx=0x7f1c0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c18ff9640 1 -- 172.21.15.94:0/4208380635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c03f030 con 0x7f1c1c1029f0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c18ff9640 1 -- 172.21.15.94:0/4208380635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c031910 con 0x7f1c1c1029f0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c18ff9640 1 -- 172.21.15.94:0/4208380635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035070 con 0x7f1c1c1029f0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.179+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a0f20 con 0x7f1c1c1029f0 2024-04-25T14:01:31.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.180+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a1380 con 0x7f1c1c1029f0 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.182+0000 7f1c29168740 1 Processor -- start 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c18ff9640 1 -- 172.21.15.94:0/4208380635 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c0c042030 con 0x7f1c1c1029f0 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 -- start start 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 0x564f0d2074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d20e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564f0d16cb00 con 0x564f0ce3d550 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564f0d108920 con 0x564f0ce66410 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c29168740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f0d168ac0 con 0x564f0ce45730 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 0x564f0d2074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42918/0 (socket says 172.21.15.94:42918) 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1b7fe640 1 -- 172.21.15.94:0/3571546713 learned_addr learned my addr 172.21.15.94:0/3571546713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1affd640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d20e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 -- 172.21.15.94:0/3571546713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 msgr2=0x564f0d20e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d20e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 -- 172.21.15.94:0/3571546713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 msgr2=0x564f0d207a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1bfff640 1 -- 172.21.15.94:0/3571546713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0cf467e0 con 0x564f0ce66410 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1affd640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d20e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c18ff9640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 0x7f1c040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.183+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 0x7f1c040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.184+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 0x564f0d2074a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009f00 tx=0x7f1c14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7c7bd4195a9d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.184+0000 7f1c18ff9640 1 -- 172.21.15.94:0/4208380635 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c0c0fefe0 con 0x7f1c1c1029f0 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.184+0000 7f1c01ffb640 1 -- 172.21.15.94:0/3571546713 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004d20 con 0x564f0ce66410 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.184+0000 7f1c01ffb640 1 -- 172.21.15.94:0/3571546713 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14004030 con 0x564f0ce66410 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.184+0000 7f1c01ffb640 1 -- 172.21.15.94:0/3571546713 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005280 con 0x564f0ce66410 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- 172.21.15.94:0/3571546713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 msgr2=0x564f0d2074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c1b7fe640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 0x7f1c040b2080 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0c34a0 tx=0x7f1c0c102ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 0x564f0d2074a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009f00 tx=0x7f1c14011810 comp rx=0 tx=0).stop 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- 172.21.15.94:0/3571546713 shutdown_connections 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d20e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce3d550 0x564f0d207a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 --2- 172.21.15.94:0/3571546713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce66410 0x564f0d2074a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- 172.21.15.94:0/3571546713 >> 172.21.15.94:0/3571546713 conn(0x564f0ce5a560 msgr2=0x564f0d202e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- 172.21.15.94:0/3571546713 shutdown_connections 2024-04-25T14:01:31.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- 172.21.15.94:0/3571546713 wait complete. 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 -- start start 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.185+0000 7f1c29168740 1 Processor -- start 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d211300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d211840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564f0d20f370 con 0x564f0ce66410 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564f0d1065d0 con 0x564f0ce3d550 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f0d20f4f0 con 0x564f0ce45730 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c29168740 10 client.0 _ll_register_callbacks cb 0x564f0d0f4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42920/0 (socket says 172.21.15.94:42920) 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1024128584 learned_addr learned my addr 172.21.15.94:0/1024128584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1affd640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d211840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d211300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1024128584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 msgr2=0x564f0d211840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d211840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 msgr2=0x564f0d211300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d211300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1024128584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0cfa6f70 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1affd640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d211840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f1c14004000 tx=0x7f1c140047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87158d21137504a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.186+0000 7f1bfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401e040 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c14004a40 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14004030 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1c14004c40 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 10 client.22587 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f0ce3e170 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14004030 con 0x564f0ce66410 2024-04-25T14:01:31.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1c14030070 con 0x564f0ce66410 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 10 client.22587 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 10 client.22587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 20 client.22587 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce251f0 0x564f0d21ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 4 client.22587 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.187+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f0d17a710 con 0x564f0ce251f0 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.188+0000 7f1c1affd640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce251f0 0x564f0d21ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.188+0000 7f1bfaffd640 10 client.22587 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.188+0000 7f1c1affd640 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce251f0 0x564f0d21ea70 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637128072632d545 server_cookie=9d3a5774cda093a4 in_seq=0 out_seq=0 2024-04-25T14:01:31.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1bfaffd640 1 -- 172.21.15.94:0/1024128584 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f0d17a710 con 0x564f0ce251f0 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1bfaffd640 10 client.22587 handle_command_reply: tid=0 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 client.22587 shutdown 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 msgr2=0x564f0d2187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f1c14004000 tx=0x7f1c140047d0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 shutdown_connections 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce251f0 0x564f0d21ea70 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce66410 0x564f0d2187a0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d211840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1bfaffd640 0 client.22587 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.190+0000 7f1c29168740 1 --2- 172.21.15.94:0/1024128584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d211300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.191+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 >> 172.21.15.94:0/1024128584 conn(0x564f0ce5a560 msgr2=0x564f0d201ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.191+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 shutdown_connections 2024-04-25T14:01:31.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.191+0000 7f1c29168740 1 -- 172.21.15.94:0/1024128584 wait complete. 2024-04-25T14:01:31.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.191+0000 7f1c29168740 20 client.22587 trim_cache size 0 max 16384 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 Processor -- start 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 -- start start 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce3d550 0x564f0d20ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce45730 0x564f0d2188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1aa470 con 0x564f0ce251f0 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1abfb0 con 0x564f0ce45730 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c29168740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1aab40 con 0x564f0ce3d550 2024-04-25T14:01:31.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42936/0 (socket says 172.21.15.94:42936) 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1990412033 learned_addr learned my addr 172.21.15.94:0/1990412033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce45730 0x564f0d2188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1affd640 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce3d550 0x564f0d20ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1990412033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce3d550 msgr2=0x564f0d20ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce3d550 0x564f0d20ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1990412033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce45730 msgr2=0x564f0d2188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce45730 0x564f0d2188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.211+0000 7f1c1bfff640 1 -- 172.21.15.94:0/1990412033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0d1fe600 con 0x564f0ce251f0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.212+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005d70 tx=0x7f1c14009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b304a3292074c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.212+0000 7f1c017fa640 1 -- 172.21.15.94:0/1990412033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14036040 con 0x564f0ce251f0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.212+0000 7f1c017fa640 1 -- 172.21.15.94:0/1990412033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14003d60 con 0x564f0ce251f0 2024-04-25T14:01:31.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.212+0000 7f1c017fa640 1 -- 172.21.15.94:0/1990412033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004de0 con 0x564f0ce251f0 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- 172.21.15.94:0/1990412033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 msgr2=0x564f0d20bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005d70 tx=0x7f1c14009520 comp rx=0 tx=0).stop 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- 172.21.15.94:0/1990412033 shutdown_connections 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce45730 0x564f0d2188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce3d550 0x564f0d20ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 --2- 172.21.15.94:0/1990412033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d20bdc0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- 172.21.15.94:0/1990412033 >> 172.21.15.94:0/1990412033 conn(0x564f0ce5a560 msgr2=0x564f0d20f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- 172.21.15.94:0/1990412033 shutdown_connections 2024-04-25T14:01:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- 172.21.15.94:0/1990412033 wait complete. 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 -- start start 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.213+0000 7f1c29168740 1 Processor -- start 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d285310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d287860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ab8e0 con 0x564f0ce251f0 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ab210 con 0x564f0ce3d550 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1a8450 con 0x564f0ce45730 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 10 client.0 _ll_register_callbacks cb 0x7f1c14004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42952/0 (socket says 172.21.15.94:42952) 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 -- 172.21.15.94:0/2639300658 learned_addr learned my addr 172.21.15.94:0/2639300658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1affd640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d285310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 -- 172.21.15.94:0/2639300658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 msgr2=0x564f0d287860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d287860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 -- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 msgr2=0x564f0d285310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d285310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 -- 172.21.15.94:0/2639300658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0d287da0 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c1bfff640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1c14011d10 tx=0x7f1c14005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623cdb0209ab0197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1bfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14036030 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c140049d0 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.214+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14003d60 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1c1400f480 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 10 client.22593 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564f0d288f00 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14003d60 con 0x564f0ce251f0 2024-04-25T14:01:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1c1403c070 con 0x564f0ce251f0 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 10 client.22593 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 10 client.22593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 20 client.22593 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce66410 0x564f0d28b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 4 client.22593 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f0d174580 con 0x564f0ce66410 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.215+0000 7f1c1affd640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce66410 0x564f0d28b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.216+0000 7f1bfaffd640 10 client.22593 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.216+0000 7f1c1affd640 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce66410 0x564f0d28b7c0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7e7273711e8ec4 server_cookie=957eb47e8cff4cf4 in_seq=0 out_seq=0 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.218+0000 7f1bfaffd640 1 -- 172.21.15.94:0/2639300658 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f0d174580 con 0x564f0ce66410 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.218+0000 7f1bfaffd640 10 client.22593 handle_command_reply: tid=0 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.218+0000 7f1c29168740 1 client.22593 shutdown 2024-04-25T14:01:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 msgr2=0x564f0d282dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1c14011d10 tx=0x7f1c14005b30 comp rx=0 tx=0).stop 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 shutdown_connections 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564f0ce66410 0x564f0d28b7c0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f0ce45730 0x564f0d287860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f0ce3d550 0x564f0d285310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1bfaffd640 0 client.22593 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 --2- 172.21.15.94:0/2639300658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564f0ce251f0 0x564f0d282dc0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 >> 172.21.15.94:0/2639300658 conn(0x564f0ce5a560 msgr2=0x564f0d206660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 shutdown_connections 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 1 -- 172.21.15.94:0/2639300658 wait complete. 2024-04-25T14:01:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c29168740 20 client.22593 trim_cache size 0 max 16384 2024-04-25T14:01:31.224 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:01:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:01:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f57481e5-46b7-4e8c-afe2-a14e566aaec0", 2024-04-25T14:01:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:01:31.224 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 msgr2=0x7f1c040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 0x7f1c040b2080 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0c34a0 tx=0x7f1c0c102ac0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 msgr2=0x7f1c1c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006010 tx=0x7f1c0c004270 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 shutdown_connections 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c040afb70 0x7f1c040b2080 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c1c108930 0x7f1c1c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c103c30 0x7f1c1c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 --2- 172.21.15.94:0/4208380635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c1029f0 0x7f1c1c199150 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.219+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 >> 172.21.15.94:0/4208380635 conn(0x7f1c1c0fdc00 msgr2=0x7f1c1c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.220+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 shutdown_connections 2024-04-25T14:01:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.220+0000 7f1c221e0640 1 -- 172.21.15.94:0/4208380635 wait complete. 2024-04-25T14:01:31.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:31.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 -- 172.21.15.94:0/4282585310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 msgr2=0x7f5098103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 --2- 172.21.15.94:0/4282585310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f5098103700 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 -- 172.21.15.94:0/4282585310 shutdown_connections 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 --2- 172.21.15.94:0/4282585310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098108940 0x7f509810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 --2- 172.21.15.94:0/4282585310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098103c40 0x7f5098104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 --2- 172.21.15.94:0/4282585310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f5098103700 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 -- 172.21.15.94:0/4282585310 >> 172.21.15.94:0/4282585310 conn(0x7f50980fdbf0 msgr2=0x7f50981000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 -- 172.21.15.94:0/4282585310 shutdown_connections 2024-04-25T14:01:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 -- 172.21.15.94:0/4282585310 wait complete. 2024-04-25T14:01:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.348+0000 7f509ef2d640 1 Processor -- start 2024-04-25T14:01:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 -- start start 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098103c40 0x7f509819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098108940 0x7f50981a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f509810f630 con 0x7f5098102a00 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f509810f4b0 con 0x7f5098108940 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ef2d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f509810f7b0 con 0x7f5098103c40 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42960/0 (socket says 172.21.15.94:42960) 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 -- 172.21.15.94:0/1553597667 learned_addr learned my addr 172.21.15.94:0/1553597667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509d4ca640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098108940 0x7f50981a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f5097fff640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098103c40 0x7f509819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 -- 172.21.15.94:0/1553597667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098103c40 msgr2=0x7f509819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098103c40 0x7f509819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 -- 172.21.15.94:0/1553597667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098108940 msgr2=0x7f50981a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098108940 0x7f50981a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 -- 172.21.15.94:0/1553597667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50981a1d90 con 0x7f5098102a00 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.349+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.350+0000 7f5095ffb640 1 -- 172.21.15.94:0/1553597667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c03f030 con 0x7f5098102a00 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.350+0000 7f5095ffb640 1 -- 172.21.15.94:0/1553597667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508c031910 con 0x7f5098102a00 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.350+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50981a2020 con 0x7f5098102a00 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.350+0000 7f5095ffb640 1 -- 172.21.15.94:0/1553597667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c035070 con 0x7f5098102a00 2024-04-25T14:01:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.350+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a2510 con 0x7f5098102a00 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 Processor -- start 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 -- start start 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556416c15d80 con 0x5564168e64d0 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556416bcf8b0 con 0x5564168ee6b0 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556416c11d40 con 0x55641690f390 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.352+0000 7f5095ffb640 1 -- 172.21.15.94:0/1553597667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f508c042030 con 0x7f5098102a00 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:59220/0 (socket says 172.21.15.94:59220) 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42966/0 (socket says 172.21.15.94:42966) 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 -- 172.21.15.94:0/3816966529 learned_addr learned my addr 172.21.15.94:0/3816966529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509d4ca640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 -- 172.21.15.94:0/3816966529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 msgr2=0x556416cb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cb0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 -- 172.21.15.94:0/3816966529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 msgr2=0x556416cb0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 -- 172.21.15.94:0/3816966529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556416ba0a40 con 0x5564168ee6b0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509d4ca640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509ccc9640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5095ffb640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 0x7f50880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5097fff640 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5084009c90 tx=0x7f5084011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=367af2f8a9e3760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f509d4ca640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 0x7f50880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5095ffb640 1 -- 172.21.15.94:0/1553597667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f508c0fef80 con 0x7f5098102a00 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.353+0000 7f5082ffd640 1 -- 172.21.15.94:0/3816966529 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084005220 con 0x5564168ee6b0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f5082ffd640 1 -- 172.21.15.94:0/3816966529 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084004c50 con 0x5564168ee6b0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f5082ffd640 1 -- 172.21.15.94:0/3816966529 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084019690 con 0x5564168ee6b0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 -- 172.21.15.94:0/3816966529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 msgr2=0x556416cb1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5084009c90 tx=0x7f5084011810 comp rx=0 tx=0).stop 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f509d4ca640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 0x7f50880b2000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x556416cb1ab0 tx=0x7f5090009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 -- 172.21.15.94:0/3816966529 shutdown_connections 2024-04-25T14:01:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cb1c70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/3816966529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 -- 172.21.15.94:0/3816966529 >> 172.21.15.94:0/3816966529 conn(0x5564169034e0 msgr2=0x556416cac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 -- 172.21.15.94:0/3816966529 shutdown_connections 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.354+0000 7f50a5ddd740 1 -- 172.21.15.94:0/3816966529 wait complete. 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 -- start start 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 Processor -- start 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cbae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556416cb2780 con 0x5564168e64d0 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556416b9cd10 con 0x5564168ee6b0 2024-04-25T14:01:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556416cb2900 con 0x55641690f390 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.355+0000 7f50a5ddd740 10 client.0 _ll_register_callbacks cb 0x556416bc9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42970/0 (socket says 172.21.15.94:42970) 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 -- 172.21.15.94:0/1376631719 learned_addr learned my addr 172.21.15.94:0/1376631719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5097fff640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cbae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509d4ca640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 -- 172.21.15.94:0/1376631719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 msgr2=0x556416cc1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 -- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 msgr2=0x556416cbae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cbae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 -- 172.21.15.94:0/1376631719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564169f17d0 con 0x5564168e64d0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f508c031440 tx=0x7f508c03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3b13d7043b71d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5077fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c049070 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f508c102c10 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5084004c50 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f508c0c7240 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f50a5ddd740 10 client.22605 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5564169273f0 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.356+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084004c50 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f508c1010d0 con 0x5564168e64d0 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 10 client.22605 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 10 client.22605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 20 client.22605 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168cdf60 0x556416cc80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 4 client.22605 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556416c23990 con 0x5564168cdf60 2024-04-25T14:01:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f5097fff640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168cdf60 0x556416cc80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f5097fff640 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168cdf60 0x556416cc80d0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900e9a7c1d487b27 server_cookie=1391e699512d8a99 in_seq=0 out_seq=0 2024-04-25T14:01:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.357+0000 7f5077fff640 10 client.22605 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f5077fff640 1 -- 172.21.15.94:0/1376631719 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556416c23990 con 0x5564168cdf60 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f5077fff640 10 client.22605 handle_command_reply: tid=0 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 client.22605 shutdown 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 msgr2=0x556416cba8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f508c031440 tx=0x7f508c03b460 comp rx=0 tx=0).stop 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 shutdown_connections 2024-04-25T14:01:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168cdf60 0x556416cc80d0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168ee6b0 0x556416cbae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f5077fff640 0 client.22605 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1376631719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168e64d0 0x556416cba8e0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.361+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 >> 172.21.15.94:0/1376631719 conn(0x5564169034e0 msgr2=0x556416cab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.362+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 shutdown_connections 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.362+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1376631719 wait complete. 2024-04-25T14:01:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.362+0000 7f50a5ddd740 20 client.22605 trim_cache size 0 max 16384 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 Processor -- start 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 -- start start 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168e64d0 0x556416caf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ab530 con 0x5564168ee6b0 2024-04-25T14:01:31.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ad070 con 0x5564168e64d0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50981abc00 con 0x55641690f390 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.382+0000 7f5097fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42984/0 (socket says 172.21.15.94:42984) 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 -- 172.21.15.94:0/49057609 learned_addr learned my addr 172.21.15.94:0/49057609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f509d4ca640 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f509ccc9640 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168e64d0 0x556416caf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 -- 172.21.15.94:0/49057609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 msgr2=0x556416cc1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 -- 172.21.15.94:0/49057609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168e64d0 msgr2=0x556416caf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168e64d0 0x556416caf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 -- 172.21.15.94:0/49057609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564168e70f0 con 0x5564168ee6b0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f5097fff640 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f50840058c0 tx=0x7f5084005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b812f4e2e8e4d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f50827fc640 1 -- 172.21.15.94:0/49057609 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50840053d0 con 0x5564168ee6b0 2024-04-25T14:01:31.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f50827fc640 1 -- 172.21.15.94:0/49057609 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084005570 con 0x5564168ee6b0 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.383+0000 7f50827fc640 1 -- 172.21.15.94:0/49057609 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084003ab0 con 0x5564168ee6b0 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- 172.21.15.94:0/49057609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 msgr2=0x556416cafcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f50840058c0 tx=0x7f5084005730 comp rx=0 tx=0).stop 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- 172.21.15.94:0/49057609 shutdown_connections 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55641690f390 0x556416cc1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cafcc0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/49057609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564168e64d0 0x556416caf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- 172.21.15.94:0/49057609 >> 172.21.15.94:0/49057609 conn(0x5564169034e0 msgr2=0x556416caf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- 172.21.15.94:0/49057609 shutdown_connections 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- 172.21.15.94:0/49057609 wait complete. 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.384+0000 7f50a5ddd740 1 -- start start 2024-04-25T14:01:31.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 Processor -- start 2024-04-25T14:01:31.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564168e64d0 0x556416cbecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55641690f390 0x556416d2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ac9a0 con 0x5564168ee6b0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ac2d0 con 0x55641690f390 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50981a9510 con 0x5564168e64d0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f50a5ddd740 10 client.0 _ll_register_callbacks cb 0x7f508c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42996/0 (socket says 172.21.15.94:42996) 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 -- 172.21.15.94:0/1881385637 learned_addr learned my addr 172.21.15.94:0/1881385637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f509d4ca640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55641690f390 0x556416d2e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564168e64d0 0x556416cbecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 -- 172.21.15.94:0/1881385637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564168e64d0 msgr2=0x556416cbecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564168e64d0 0x556416cbecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 -- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55641690f390 msgr2=0x556416d2e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:31.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55641690f390 0x556416d2e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.385+0000 7f5097fff640 1 -- 172.21.15.94:0/1881385637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556416bd64b0 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5097fff640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f5084003630 tx=0x7f5084009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a48807e12d782a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508403c070 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5084003a30 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556416cb5150 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5084048050 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 10 client.22611 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556416d2f910 con 0x5564168ee6b0 2024-04-25T14:01:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556416cb5150 con 0x5564168ee6b0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f50840364c0 con 0x5564168ee6b0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 10 client.22611 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 10 client.22611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 20 client.22611 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168a33a0 0x556416d320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 4 client.22611 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556416c1d800 con 0x5564168a33a0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.386+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168a33a0 0x556416d320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.387+0000 7f5077fff640 10 client.22611 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.387+0000 7f509ccc9640 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168a33a0 0x556416d320e0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346274b986316327 server_cookie=847b727df747e5dc in_seq=0 out_seq=0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.389+0000 7f5077fff640 1 -- 172.21.15.94:0/1881385637 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556416c1d800 con 0x5564168a33a0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.389+0000 7f5077fff640 10 client.22611 handle_command_reply: tid=0 2024-04-25T14:01:31.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.389+0000 7f50a5ddd740 1 client.22611 shutdown 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 msgr2=0x556416cbf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f5084003630 tx=0x7f5084009820 comp rx=0 tx=0).stop 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 shutdown_connections 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5564168a33a0 0x556416d320e0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55641690f390 0x556416d2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564168ee6b0 0x556416cbf210 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f5077fff640 0 client.22611 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 --2- 172.21.15.94:0/1881385637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564168e64d0 0x556416cbecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 >> 172.21.15.94:0/1881385637 conn(0x5564169034e0 msgr2=0x556416cafaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 shutdown_connections 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 1 -- 172.21.15.94:0/1881385637 wait complete. 2024-04-25T14:01:31.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f50a5ddd740 20 client.22611 trim_cache size 0 max 16384 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (484 inodes in the stack)", 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "f57481e5-46b7-4e8c-afe2-a14e566aaec0": { 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f57481e5-46b7-4e8c-afe2-a14e566aaec0", 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:01:31.395 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:01:31.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 msgr2=0x7f50880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 0x7f50880b2000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x556416cb1ab0 tx=0x7f5090009500 comp rx=0 tx=0).stop 2024-04-25T14:01:31.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 msgr2=0x7f509819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c004270 comp rx=0 tx=0).stop 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 shutdown_connections 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50880afaf0 0x7f50880b2000 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5098108940 0x7f50981a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5098103c40 0x7f509819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 --2- 172.21.15.94:0/1553597667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5098102a00 0x7f509819a250 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.390+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 >> 172.21.15.94:0/1553597667 conn(0x7f50980fdbf0 msgr2=0x7f5098104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.391+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 shutdown_connections 2024-04-25T14:01:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:01:31.391+0000 7f509ef2d640 1 -- 172.21.15.94:0/1553597667 wait complete. 2024-04-25T14:01:31.402 INFO:tasks.cephfs.filesystem:scrub status for tag:f57481e5-46b7-4e8c-afe2-a14e566aaec0 - {'path': '/', 'tag': 'f57481e5-46b7-4e8c-afe2-a14e566aaec0', 'options': 'recursive,force'} 2024-04-25T14:01:32.149 INFO:tasks.workunit.client.0.smithi094.stdout: 10 134881 19.14 MB/sec execute 254 sec latency 82.202 ms 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: pgmap v943: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 575 op/s 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:32 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: pgmap v943: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 575 op/s 2024-04-25T14:01:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:01:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:01:32.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:32 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: pgmap v943: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 575 op/s 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:32.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:32 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:33.149 INFO:tasks.workunit.client.0.smithi094.stdout: 10 135330 19.20 MB/sec execute 255 sec latency 72.122 ms 2024-04-25T14:01:33.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:33 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:33.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:33 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:33.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:33 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:34.149 INFO:tasks.workunit.client.0.smithi094.stdout: 10 135576 19.18 MB/sec execute 256 sec latency 248.202 ms 2024-04-25T14:01:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:34 smithi137 ceph-mon[32568]: pgmap v944: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2024-04-25T14:01:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:34 smithi094 ceph-mon[29195]: pgmap v944: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2024-04-25T14:01:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:34 smithi138 ceph-mon[32977]: pgmap v944: 97 pgs: 97 active+clean; 271 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2024-04-25T14:01:35.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 135897 19.19 MB/sec execute 257 sec latency 91.514 ms 2024-04-25T14:01:36.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 136155 19.17 MB/sec execute 258 sec latency 73.811 ms 2024-04-25T14:01:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:36 smithi137 ceph-mon[32568]: pgmap v945: 97 pgs: 97 active+clean; 261 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 629 op/s 2024-04-25T14:01:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:36 smithi094 ceph-mon[29195]: pgmap v945: 97 pgs: 97 active+clean; 261 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 629 op/s 2024-04-25T14:01:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:36 smithi138 ceph-mon[32977]: pgmap v945: 97 pgs: 97 active+clean; 261 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 629 op/s 2024-04-25T14:01:37.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 136453 19.12 MB/sec execute 259 sec latency 83.568 ms 2024-04-25T14:01:38.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 136712 19.10 MB/sec execute 260 sec latency 63.089 ms 2024-04-25T14:01:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:38 smithi137 ceph-mon[32568]: pgmap v946: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 548 op/s 2024-04-25T14:01:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:38 smithi094 ceph-mon[29195]: pgmap v946: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 548 op/s 2024-04-25T14:01:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:38 smithi138 ceph-mon[32977]: pgmap v946: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 548 op/s 2024-04-25T14:01:39.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 136942 19.06 MB/sec execute 261 sec latency 317.561 ms 2024-04-25T14:01:40.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 137425 19.09 MB/sec execute 262 sec latency 122.023 ms 2024-04-25T14:01:40.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:40 smithi137 ceph-mon[32568]: pgmap v947: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 547 op/s 2024-04-25T14:01:40.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:40 smithi094 ceph-mon[29195]: pgmap v947: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 547 op/s 2024-04-25T14:01:40.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:40 smithi138 ceph-mon[32977]: pgmap v947: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 547 op/s 2024-04-25T14:01:41.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 138007 19.12 MB/sec execute 263 sec latency 68.433 ms 2024-04-25T14:01:41.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:41 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:41.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:41 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:41.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:41 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:41.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:41 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:41.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:41 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:41.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:41 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:42.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 138545 19.17 MB/sec execute 264 sec latency 65.149 ms 2024-04-25T14:01:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:42 smithi137 ceph-mon[32568]: pgmap v948: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-04-25T14:01:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:42.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:42.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:42 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:01:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:42 smithi094 ceph-mon[29195]: pgmap v948: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-04-25T14:01:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:42.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:42 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:01:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:42 smithi138 ceph-mon[32977]: pgmap v948: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-04-25T14:01:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:42.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:42 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:01:43.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 138978 19.23 MB/sec execute 265 sec latency 81.552 ms 2024-04-25T14:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.764+0000 7fb2eda13640 1 -- 172.21.15.94:0/4008884345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e8110070 msgr2=0x7fb2e8110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.764+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4008884345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e8110070 0x7fb2e8110480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009860 tx=0x7fb2d802a360 comp rx=0 tx=0).stop 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.765+0000 7fb2eda13640 1 -- 172.21.15.94:0/4008884345 shutdown_connections 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.765+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4008884345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e8110070 0x7fb2e8110480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.765+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4008884345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e810b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.765+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4008884345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2e810a420 0x7fb2e810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.765+0000 7fb2eda13640 1 -- 172.21.15.94:0/4008884345 >> 172.21.15.94:0/4008884345 conn(0x7fb2e8070540 msgr2=0x7fb2e80729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.766+0000 7fb2eda13640 1 -- 172.21.15.94:0/4008884345 shutdown_connections 2024-04-25T14:01:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.768+0000 7fb2eda13640 1 -- 172.21.15.94:0/4008884345 wait complete. 2024-04-25T14:01:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.769+0000 7fb2eda13640 1 Processor -- start 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 -- start start 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e810a420 0x7fb2e81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2e8110070 0x7fb2e81a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8113cb0 con 0x7fb2e810b350 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8113b30 con 0x7fb2e810a420 2024-04-25T14:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2eda13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8113e30 con 0x7fb2e8110070 2024-04-25T14:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59598/0 (socket says 172.21.15.94:59598) 2024-04-25T14:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2e67fc640 1 -- 172.21.15.94:0/4036268061 learned_addr learned my addr 172.21.15.94:0/4036268061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.770+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e810a420 0x7fb2e81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.771+0000 7fb2e67fc640 1 -- 172.21.15.94:0/4036268061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2e8110070 msgr2=0x7fb2e81a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.771+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2e8110070 0x7fb2e81a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.771+0000 7fb2e67fc640 1 -- 172.21.15.94:0/4036268061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e810a420 msgr2=0x7fb2e81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.771+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e810a420 0x7fb2e81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.771+0000 7fb2e67fc640 1 -- 172.21.15.94:0/4036268061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e81a83d0 con 0x7fb2e810b350 2024-04-25T14:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.772+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d50 tx=0x7fb2e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.772+0000 7fb2eca11640 1 -- 172.21.15.94:0/4036268061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003b70 con 0x7fb2e810b350 2024-04-25T14:01:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.773+0000 7fb2eca11640 1 -- 172.21.15.94:0/4036268061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e0003d10 con 0x7fb2e810b350 2024-04-25T14:01:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.773+0000 7fb2eca11640 1 -- 172.21.15.94:0/4036268061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0010820 con 0x7fb2e810b350 2024-04-25T14:01:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.774+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e81a86c0 con 0x7fb2e810b350 2024-04-25T14:01:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.775+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a8bc0 con 0x7fb2e810b350 2024-04-25T14:01:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.779+0000 7fb2f499e740 1 Processor -- start 2024-04-25T14:01:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.780+0000 7fb2eca11640 1 -- 172.21.15.94:0/4036268061 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb2e00109c0 con 0x7fb2e810b350 2024-04-25T14:01:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.781+0000 7fb2eca11640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 0x7fb2d00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 -- start start 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8382d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b834642f0 0x556b8382def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b83793460 con 0x556b8348d1b0 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b8374d590 con 0x556b834642f0 2024-04-25T14:01:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.783+0000 7fb2f499e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b8378f420 con 0x556b8346c4d0 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2eca11640 1 -- 172.21.15.94:0/4036268061 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb2e00a4c10 con 0x7fb2e810b350 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59494/0 (socket says 172.21.15.94:59494) 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e77fe640 1 -- 172.21.15.94:0/2178911001 learned_addr learned my addr 172.21.15.94:0/2178911001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 0x7fb2d00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8382d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.785+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b834642f0 0x556b8382def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.788+0000 7fb2e77fe640 1 -- 172.21.15.94:0/2178911001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b834642f0 msgr2=0x556b8382def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.788+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b834642f0 0x556b8382def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.788+0000 7fb2e77fe640 1 -- 172.21.15.94:0/2178911001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 msgr2=0x556b8382d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.788+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8382d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.788+0000 7fb2e77fe640 1 -- 172.21.15.94:0/2178911001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8375a0d0 con 0x556b8346c4d0 2024-04-25T14:01:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.791+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80057c0 tx=0x7fb2d8030f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27200c7a35a6b4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.794+0000 7fb2cdffb640 1 -- 172.21.15.94:0/2178911001 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d804d070 con 0x556b8346c4d0 2024-04-25T14:01:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.794+0000 7fb2cdffb640 1 -- 172.21.15.94:0/2178911001 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8044070 con 0x556b8346c4d0 2024-04-25T14:01:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 -- 172.21.15.94:0/2178911001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 msgr2=0x556b83834df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80057c0 tx=0x7fb2d8030f70 comp rx=0 tx=0).stop 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 -- 172.21.15.94:0/2178911001 shutdown_connections 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b83834df0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b834642f0 0x556b8382def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 --2- 172.21.15.94:0/2178911001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8382d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 -- 172.21.15.94:0/2178911001 >> 172.21.15.94:0/2178911001 conn(0x556b83481300 msgr2=0x556b83829300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.796+0000 7fb2f499e740 1 -- 172.21.15.94:0/2178911001 shutdown_connections 2024-04-25T14:01:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.799+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 0x7fb2d00b2180 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80095f0 tx=0x7fb2d8044000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.799+0000 7fb2f499e740 1 -- 172.21.15.94:0/2178911001 wait complete. 2024-04-25T14:01:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.800+0000 7fb2f499e740 1 -- start start 2024-04-25T14:01:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.800+0000 7fb2f499e740 1 Processor -- start 2024-04-25T14:01:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.800+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b83837650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8346c4d0 0x556b83837b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b83835830 con 0x556b8348d1b0 2024-04-25T14:01:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b8371ab60 con 0x556b8346c4d0 2024-04-25T14:01:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b838359b0 con 0x556b834642f0 2024-04-25T14:01:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2e77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59610/0 (socket says 172.21.15.94:59610) 2024-04-25T14:01:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2e77fe640 1 -- 172.21.15.94:0/1311742142 learned_addr learned my addr 172.21.15.94:0/1311742142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.801+0000 7fb2f499e740 10 client.0 _ll_register_callbacks cb 0x556b8373f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b83837650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 -- 172.21.15.94:0/1311742142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 msgr2=0x556b83837650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b83837650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 -- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8346c4d0 msgr2=0x556b83837b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8346c4d0 0x556b83837b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 -- 172.21.15.94:0/1311742142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b834a5210 con 0x556b8348d1b0 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.802+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802c040 tx=0x7fb2d8049d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d33fcaa6b86e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.803+0000 7fb2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.803+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d804d040 con 0x556b8348d1b0 2024-04-25T14:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.804+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d8044070 con 0x556b8348d1b0 2024-04-25T14:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.804+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb2d8032a10 con 0x556b8348d1b0 2024-04-25T14:01:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.806+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb2d8030980 con 0x556b8348d1b0 2024-04-25T14:01:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.806+0000 7fb2f499e740 10 client.22623 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.807+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b83571620 con 0x556b8348d1b0 2024-04-25T14:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.809+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8044070 con 0x556b8348d1b0 2024-04-25T14:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.809+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d803d050 con 0x556b8348d1b0 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.809+0000 7fb2f499e740 10 client.22623 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.809+0000 7fb2f499e740 10 client.22623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.809+0000 7fb2f499e740 20 client.22623 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.810+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b8344bd80 0x556b838455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.810+0000 7fb2f499e740 4 client.22623 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.810+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b837a1070 con 0x556b8344bd80 2024-04-25T14:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.810+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b8344bd80 0x556b838455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.818+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b8344bd80 0x556b838455b0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da19ebd24f2e944 server_cookie=4c404308ece2506a in_seq=0 out_seq=0 2024-04-25T14:01:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.818+0000 7fb2c6ffd640 10 client.22623 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.831+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/1311742142 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b837a1070 con 0x556b8344bd80 2024-04-25T14:01:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.831+0000 7fb2c6ffd640 10 client.22623 handle_command_reply: tid=0 2024-04-25T14:01:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.831+0000 7fb2f499e740 1 client.22623 shutdown 2024-04-25T14:01:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.831+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 msgr2=0x556b8383eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.831+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802c040 tx=0x7fb2d8049d70 comp rx=0 tx=0).stop 2024-04-25T14:01:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 shutdown_connections 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b8344bd80 0x556b838455b0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8348d1b0 0x556b8383eaf0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8346c4d0 0x556b83837b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2c6ffd640 0 client.22623 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 --2- 172.21.15.94:0/1311742142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b83837650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 >> 172.21.15.94:0/1311742142 conn(0x556b83481300 msgr2=0x556b83828250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 shutdown_connections 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 1 -- 172.21.15.94:0/1311742142 wait complete. 2024-04-25T14:01:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.832+0000 7fb2f499e740 20 client.22623 trim_cache size 0 max 16384 2024-04-25T14:01:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.852+0000 7fb2f499e740 1 Processor -- start 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.852+0000 7fb2f499e740 1 -- start start 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b838320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b83827f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400e890 con 0x556b8346c4d0 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d40103d0 con 0x556b8348d1b0 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2f499e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400ef60 con 0x556b834642f0 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59624/0 (socket says 172.21.15.94:59624) 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 -- 172.21.15.94:0/3384213175 learned_addr learned my addr 172.21.15.94:0/3384213175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b83827f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b838320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 -- 172.21.15.94:0/3384213175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 msgr2=0x556b838320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b838320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 -- 172.21.15.94:0/3384213175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 msgr2=0x556b83827f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b83827f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 -- 172.21.15.94:0/3384213175 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8383ef50 con 0x556b8346c4d0 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.853+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d20 tx=0x7fb2e00a4610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b214ddd429b014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.854+0000 7fb2cd7fa640 1 -- 172.21.15.94:0/3384213175 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00a8130 con 0x556b8346c4d0 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.854+0000 7fb2cd7fa640 1 -- 172.21.15.94:0/3384213175 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e00c0070 con 0x556b8346c4d0 2024-04-25T14:01:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.854+0000 7fb2cd7fa640 1 -- 172.21.15.94:0/3384213175 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00a41f0 con 0x556b8346c4d0 2024-04-25T14:01:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 -- 172.21.15.94:0/3384213175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 msgr2=0x556b8383ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d20 tx=0x7fb2e00a4610 comp rx=0 tx=0).stop 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 -- 172.21.15.94:0/3384213175 shutdown_connections 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b83827f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b8346c4d0 0x556b8383ea10 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 --2- 172.21.15.94:0/3384213175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b834642f0 0x556b838320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 -- 172.21.15.94:0/3384213175 >> 172.21.15.94:0/3384213175 conn(0x556b83481300 msgr2=0x556b83827820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 -- 172.21.15.94:0/3384213175 shutdown_connections 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.855+0000 7fb2f499e740 1 -- 172.21.15.94:0/3384213175 wait complete. 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 -- start start 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 Processor -- start 2024-04-25T14:01:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 0x556b8383b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b8383bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b838ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400fd00 con 0x556b834642f0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2e6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 0x556b8383b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2e6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 0x556b8383b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59634/0 (socket says 172.21.15.94:59634) 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2e67fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b8383bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2e6ffd640 1 -- 172.21.15.94:0/918497634 learned_addr learned my addr 172.21.15.94:0/918497634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.856+0000 7fb2f499e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400f630 con 0x556b8348d1b0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d400c870 con 0x556b8346c4d0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e77fe640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b838ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2f499e740 10 client.0 _ll_register_callbacks cb 0x7fb2d804b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 -- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 msgr2=0x556b838ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b838ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 -- 172.21.15.94:0/918497634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 msgr2=0x556b8383b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 0x556b8383b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 -- 172.21.15.94:0/918497634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8383eb80 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2e67fc640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b8383bbc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009420 tx=0x7fb2e00ca0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e65bc2a38b23ac30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00b5020 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb2e00a72f0 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e00c0070 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb2e00a8130 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.857+0000 7fb2f499e740 10 client.31684 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b838accf0 con 0x556b8346c4d0 2024-04-25T14:01:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e00c0070 con 0x556b8346c4d0 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb2e0083ce0 con 0x556b8346c4d0 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 10 client.31684 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 10 client.31684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 20 client.31684 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b834211c0 0x556b838af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 4 client.31684 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.858+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b8379aee0 con 0x556b834211c0 2024-04-25T14:01:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.860+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b834211c0 0x556b838af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.863+0000 7fb2c6ffd640 10 client.31684 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:43.863+0000 7fb2e6ffd640 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b834211c0 0x556b838af6d0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb7637678ec3b2a server_cookie=95fed56e42e3a2d8 in_seq=0 out_seq=0 2024-04-25T14:01:44.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 139235 19.21 MB/sec execute 266 sec latency 303.828 ms 2024-04-25T14:01:44.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:44 smithi137 ceph-mon[32568]: pgmap v949: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 528 op/s 2024-04-25T14:01:44.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:44 smithi094 ceph-mon[29195]: pgmap v949: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 528 op/s 2024-04-25T14:01:44.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:44 smithi138 ceph-mon[32977]: pgmap v949: 97 pgs: 97 active+clean; 268 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 528 op/s 2024-04-25T14:01:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2c6ffd640 1 -- 172.21.15.94:0/918497634 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b8379aee0 con 0x556b834211c0 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2c6ffd640 10 client.31684 handle_command_reply: tid=0 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 client.31684 shutdown 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 msgr2=0x556b8383bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b8383bbc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009420 tx=0x7fb2e00ca0e0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 shutdown_connections 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556b834211c0 0x556b838af6d0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b8348d1b0 0x556b838ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2c6ffd640 0 client.31684 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b8346c4d0 0x556b8383bbc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 --2- 172.21.15.94:0/918497634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b834642f0 0x556b8383b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 >> 172.21.15.94:0/918497634 conn(0x556b83481300 msgr2=0x556b83828250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 shutdown_connections 2024-04-25T14:01:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 1 -- 172.21.15.94:0/918497634 wait complete. 2024-04-25T14:01:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.640+0000 7fb2f499e740 20 client.31684 trim_cache size 0 max 16384 2024-04-25T14:01:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:01:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:01:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:01:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:01:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 msgr2=0x7fb2d00b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 0x7fb2d00b2180 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80095f0 tx=0x7fb2d8044000 comp rx=0 tx=0).stop 2024-04-25T14:01:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 msgr2=0x7fb2e81a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009d50 tx=0x7fb2e000c6a0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 shutdown_connections 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb2d00afc70 0x7fb2d00b2180 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2e8110070 0x7fb2e81a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e810b350 0x7fb2e81a0dd0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 --2- 172.21.15.94:0/4036268061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e810a420 0x7fb2e81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 >> 172.21.15.94:0/4036268061 conn(0x7fb2e8070540 msgr2=0x7fb2e810e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 shutdown_connections 2024-04-25T14:01:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:44.641+0000 7fb2eda13640 1 -- 172.21.15.94:0/4036268061 wait complete. 2024-04-25T14:01:45.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 139417 19.20 MB/sec execute 267 sec latency 531.718 ms 2024-04-25T14:01:46.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 139623 19.16 MB/sec execute 268 sec latency 96.954 ms 2024-04-25T14:01:46.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:46 smithi137 ceph-mon[32568]: pgmap v950: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T14:01:46.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:46 smithi094 ceph-mon[29195]: pgmap v950: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T14:01:46.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:46 smithi138 ceph-mon[32977]: pgmap v950: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 657 op/s 2024-04-25T14:01:47.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 139863 19.11 MB/sec execute 269 sec latency 72.990 ms 2024-04-25T14:01:48.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 140162 19.08 MB/sec execute 270 sec latency 128.074 ms 2024-04-25T14:01:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:48 smithi137 ceph-mon[32568]: pgmap v951: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-04-25T14:01:48.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:48 smithi094 ceph-mon[29195]: pgmap v951: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-04-25T14:01:48.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:48 smithi138 ceph-mon[32977]: pgmap v951: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.1 MiB/s wr, 424 op/s 2024-04-25T14:01:49.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 140417 19.04 MB/sec execute 271 sec latency 192.431 ms 2024-04-25T14:01:50.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 140871 19.08 MB/sec execute 272 sec latency 80.098 ms 2024-04-25T14:01:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:50 smithi137 ceph-mon[32568]: pgmap v952: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-04-25T14:01:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:50 smithi094 ceph-mon[29195]: pgmap v952: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-04-25T14:01:50.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:50 smithi138 ceph-mon[32977]: pgmap v952: 97 pgs: 97 active+clean; 273 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-04-25T14:01:51.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 141343 19.09 MB/sec execute 273 sec latency 89.688 ms 2024-04-25T14:01:52.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 141780 19.08 MB/sec execute 274 sec latency 69.552 ms 2024-04-25T14:01:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:52 smithi137 ceph-mon[32568]: pgmap v953: 97 pgs: 97 active+clean; 262 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-04-25T14:01:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:52 smithi094 ceph-mon[29195]: pgmap v953: 97 pgs: 97 active+clean; 262 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-04-25T14:01:52.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:52 smithi138 ceph-mon[32977]: pgmap v953: 97 pgs: 97 active+clean; 262 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 539 op/s 2024-04-25T14:01:53.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 142247 19.13 MB/sec execute 275 sec latency 59.327 ms 2024-04-25T14:01:54.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 142635 19.18 MB/sec execute 276 sec latency 227.282 ms 2024-04-25T14:01:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:54 smithi137 ceph-mon[32568]: pgmap v954: 97 pgs: 97 active+clean; 267 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T14:01:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:54 smithi094 ceph-mon[29195]: pgmap v954: 97 pgs: 97 active+clean; 267 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T14:01:54.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:54 smithi138 ceph-mon[32977]: pgmap v954: 97 pgs: 97 active+clean; 267 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-04-25T14:01:55.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 142929 19.18 MB/sec execute 277 sec latency 234.368 ms 2024-04-25T14:01:56.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 143144 19.15 MB/sec execute 278 sec latency 81.056 ms 2024-04-25T14:01:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:56 smithi137 ceph-mon[32568]: pgmap v955: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-04-25T14:01:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:56 smithi094 ceph-mon[29195]: pgmap v955: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-04-25T14:01:56.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:56 smithi138 ceph-mon[32977]: pgmap v955: 97 pgs: 97 active+clean; 276 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 553 op/s 2024-04-25T14:01:57.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 143376 19.11 MB/sec execute 279 sec latency 81.801 ms 2024-04-25T14:01:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- 172.21.15.94:0/1490109194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604109a80 msgr2=0x7f460410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 --2- 172.21.15.94:0/1490109194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604109a80 0x7f460410bea0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009ca0 tx=0x7f45fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- 172.21.15.94:0/1490109194 shutdown_connections 2024-04-25T14:01:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 --2- 172.21.15.94:0/1490109194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604109a80 0x7f460410bea0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 --2- 172.21.15.94:0/1490109194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604104d80 0x7f4604105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 --2- 172.21.15.94:0/1490109194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604103b40 0x7f4604104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- 172.21.15.94:0/1490109194 >> 172.21.15.94:0/1490109194 conn(0x7f46040fed30 msgr2=0x7f46041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- 172.21.15.94:0/1490109194 shutdown_connections 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- 172.21.15.94:0/1490109194 wait complete. 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 Processor -- start 2024-04-25T14:01:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.744+0000 7f460a9c5640 1 -- start start 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604103b40 0x7f460419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604109a80 0x7f46041a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f460410f6c0 con 0x7f4604104d80 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f460410f540 con 0x7f4604109a80 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f460a9c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f460410f840 con 0x7f4604103b40 2024-04-25T14:01:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40016/0 (socket says 172.21.15.94:40016) 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 -- 172.21.15.94:0/176304235 learned_addr learned my addr 172.21.15.94:0/176304235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f4608f62640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604109a80 0x7f46041a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f4603fff640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604103b40 0x7f460419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 -- 172.21.15.94:0/176304235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604103b40 msgr2=0x7f460419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604103b40 0x7f460419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 -- 172.21.15.94:0/176304235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604109a80 msgr2=0x7f46041a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604109a80 0x7f46041a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 -- 172.21.15.94:0/176304235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46041a1d20 con 0x7f4604104d80 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.745+0000 7f46037fe640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d770 tx=0x7f45f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.746+0000 7f46017fa640 1 -- 172.21.15.94:0/176304235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8017070 con 0x7f4604104d80 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.746+0000 7f46017fa640 1 -- 172.21.15.94:0/176304235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f80054d0 con 0x7f4604104d80 2024-04-25T14:01:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.746+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46041a1fb0 con 0x7f4604104d80 2024-04-25T14:01:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.746+0000 7f46017fa640 1 -- 172.21.15.94:0/176304235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004c30 con 0x7f4604104d80 2024-04-25T14:01:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.746+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041a2450 con 0x7f4604104d80 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.748+0000 7f46017fa640 1 -- 172.21.15.94:0/176304235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f45f8004e10 con 0x7f4604104d80 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.748+0000 7f4611876740 1 Processor -- start 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.748+0000 7f4611876740 1 -- start start 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513cf0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051392f4d0 0x560513cf7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560513c56460 con 0x5605139272f0 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560513c10590 con 0x5605139501b0 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4611876740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560513c52420 con 0x56051392f4d0 2024-04-25T14:01:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40026/0 (socket says 172.21.15.94:40026) 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 -- 172.21.15.94:0/4282345930 learned_addr learned my addr 172.21.15.94:0/4282345930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4608f62640 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513cf0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f46017fa640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 0x7f45ec0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f46037fe640 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051392f4d0 0x560513cf7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4608f62640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 0x7f45ec0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 -- 172.21.15.94:0/4282345930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051392f4d0 msgr2=0x560513cf7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051392f4d0 0x560513cf7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 -- 172.21.15.94:0/4282345930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 msgr2=0x560513cf0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513cf0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 -- 172.21.15.94:0/4282345930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560513c1d0d0 con 0x5605139272f0 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f46017fa640 1 -- 172.21.15.94:0/176304235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f45f80de7e0 con 0x7f4604104d80 2024-04-25T14:01:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.749+0000 7f4603fff640 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009bf0 tx=0x7f45f40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491f8c2747d7c9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4608f62640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 0x7f45ec0b2070 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x560513cf1b40 tx=0x7f45fc02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f45ea7fc640 1 -- 172.21.15.94:0/4282345930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4004030 con 0x5605139272f0 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f45ea7fc640 1 -- 172.21.15.94:0/4282345930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f40041d0 con 0x5605139272f0 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f45ea7fc640 1 -- 172.21.15.94:0/4282345930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40126b0 con 0x5605139272f0 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 -- 172.21.15.94:0/4282345930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 msgr2=0x560513cf0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009bf0 tx=0x7f45f40117b0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 -- 172.21.15.94:0/4282345930 shutdown_connections 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051392f4d0 0x560513cf7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605139272f0 0x560513cf0ef0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.750+0000 7f4611876740 1 --2- 172.21.15.94:0/4282345930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513cf0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- 172.21.15.94:0/4282345930 >> 172.21.15.94:0/4282345930 conn(0x560513944300 msgr2=0x560513cec300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- 172.21.15.94:0/4282345930 shutdown_connections 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- 172.21.15.94:0/4282345930 wait complete. 2024-04-25T14:01:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- start start 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 Processor -- start 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cfa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513d01a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560513cf8840 con 0x56051392f4d0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560513bddb60 con 0x5605139501b0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560513cf89c0 con 0x5605139272f0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.751+0000 7f4611876740 10 client.0 _ll_register_callbacks cb 0x560513c022b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40032/0 (socket says 172.21.15.94:40032) 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 -- 172.21.15.94:0/4126224991 learned_addr learned my addr 172.21.15.94:0/4126224991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4608f62640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513d01a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4603fff640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cfa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 -- 172.21.15.94:0/4126224991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 msgr2=0x560513cfa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cfa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 -- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 msgr2=0x560513d01a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513d01a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 -- 172.21.15.94:0/4126224991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560513968210 con 0x56051392f4d0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4608f62640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513d01a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f46037fe640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f45f80a2c40 tx=0x7f45f80a6a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92187918c1465d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f45e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8020040 con 0x56051392f4d0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f45f80a2e50 con 0x56051392f4d0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f40041d0 con 0x56051392f4d0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45f80a66d0 con 0x56051392f4d0 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4611876740 10 client.22641 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.752+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560513a34620 con 0x56051392f4d0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f40041d0 con 0x56051392f4d0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f45f80af040 con 0x56051392f4d0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 10 client.22641 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 10 client.22641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 20 client.22641 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56051390ed80 0x560513d082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 4 client.22641 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560513c64070 con 0x56051390ed80 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4603fff640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56051390ed80 0x560513d082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f4603fff640 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56051390ed80 0x560513d082f0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3678f1b7a2a780b7 server_cookie=623fe297176c71ac in_seq=0 out_seq=0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.753+0000 7f45e37fe640 10 client.22641 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.755+0000 7f45e37fe640 1 -- 172.21.15.94:0/4126224991 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560513c64070 con 0x56051390ed80 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.755+0000 7f45e37fe640 10 client.22641 handle_command_reply: tid=0 2024-04-25T14:01:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.755+0000 7f4611876740 1 client.22641 shutdown 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 msgr2=0x560513cfab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f45f80a2c40 tx=0x7f45f80a6a50 comp rx=0 tx=0).stop 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 shutdown_connections 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56051390ed80 0x560513d082f0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139501b0 0x560513d01a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513cfab90 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f45e37fe640 0 client.22641 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 --2- 172.21.15.94:0/4126224991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cfa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 >> 172.21.15.94:0/4126224991 conn(0x560513944300 msgr2=0x560513cef210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 shutdown_connections 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 1 -- 172.21.15.94:0/4126224991 wait complete. 2024-04-25T14:01:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.756+0000 7f4611876740 20 client.22641 trim_cache size 0 max 16384 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 Processor -- start 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 -- start start 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cf4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56051392f4d0 0x560513cef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46041ab520 con 0x56051390ed80 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46041ad060 con 0x56051392f4d0 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4611876740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46041abbf0 con 0x5605139272f0 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40038/0 (socket says 172.21.15.94:40038) 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 -- 172.21.15.94:0/569198823 learned_addr learned my addr 172.21.15.94:0/569198823 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f46037fe640 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cf4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4608f62640 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56051392f4d0 0x560513cef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 -- 172.21.15.94:0/569198823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 msgr2=0x560513cf4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cf4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 -- 172.21.15.94:0/569198823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56051392f4d0 msgr2=0x560513cef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56051392f4d0 0x560513cef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.772+0000 7f4603fff640 1 -- 172.21.15.94:0/569198823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560513cecc60 con 0x56051390ed80 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.773+0000 7f4603fff640 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x560513cf1d50 tx=0x7f45f4003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309454c9724453e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.773+0000 7f45e9ffb640 1 -- 172.21.15.94:0/569198823 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f402a070 con 0x56051390ed80 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.773+0000 7f45e9ffb640 1 -- 172.21.15.94:0/569198823 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f40035f0 con 0x56051390ed80 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.773+0000 7f45e9ffb640 1 -- 172.21.15.94:0/569198823 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f401f040 con 0x56051390ed80 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- 172.21.15.94:0/569198823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 msgr2=0x560513cf4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x560513cf1d50 tx=0x7f45f4003df0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- 172.21.15.94:0/569198823 shutdown_connections 2024-04-25T14:01:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56051392f4d0 0x560513cef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605139272f0 0x560513cf4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- 172.21.15.94:0/569198823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051390ed80 0x560513cf4aa0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- 172.21.15.94:0/569198823 >> 172.21.15.94:0/569198823 conn(0x560513944300 msgr2=0x560513cf08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- 172.21.15.94:0/569198823 shutdown_connections 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- 172.21.15.94:0/569198823 wait complete. 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- start start 2024-04-25T14:01:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 Processor -- start 2024-04-25T14:01:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051390ed80 0x560513cfe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139272f0 0x560513cfe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46041ac990 con 0x56051392f4d0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46041ac2c0 con 0x5605139272f0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46041a9500 con 0x56051390ed80 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.774+0000 7f4611876740 10 client.0 _ll_register_callbacks cb 0x7f45f80a2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40042/0 (socket says 172.21.15.94:40042) 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4603fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051390ed80 0x560513cfe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 -- 172.21.15.94:0/3295824861 learned_addr learned my addr 172.21.15.94:0/3295824861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f46037fe640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139272f0 0x560513cfe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 -- 172.21.15.94:0/3295824861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051390ed80 msgr2=0x560513cfe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051390ed80 0x560513cfe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 -- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139272f0 msgr2=0x560513cfe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139272f0 0x560513cfe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 -- 172.21.15.94:0/3295824861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560513d6ec20 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4608f62640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f45fc035000 tx=0x7f45fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf29eb8515d69d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f45e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc02f810 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f45fc02f9b0 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f40035f0 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f45fc05c050 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4611876740 10 client.22647 fetch_fsmap learned FSMap version 39 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.775+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560513d6ff20 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f40035f0 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f45fc04a070 con 0x56051392f4d0 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 10 client.22647 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 10 client.22647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 20 client.22647 populate_metadata read hostname 'smithi094' 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5605139501b0 0x560513d72920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 4 client.22647 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560513c5dee0 con 0x5605139501b0 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4603fff640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5605139501b0 0x560513d72920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f45e37fe640 10 client.22647 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:57.776+0000 7f4603fff640 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5605139501b0 0x560513d72920 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7178bbb994f7ecc2 server_cookie=a5d9c12e5ed22c7 in_seq=0 out_seq=0 2024-04-25T14:01:58.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 143634 19.06 MB/sec execute 280 sec latency 217.304 ms 2024-04-25T14:01:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:01:58 smithi137 ceph-mon[32568]: pgmap v956: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-04-25T14:01:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.458+0000 7f45e37fe640 1 -- 172.21.15.94:0/3295824861 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560513c5dee0 con 0x5605139501b0 2024-04-25T14:01:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.458+0000 7f45e37fe640 10 client.22647 handle_command_reply: tid=0 2024-04-25T14:01:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.458+0000 7f4611876740 1 client.22647 shutdown 2024-04-25T14:01:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 msgr2=0x560513d6e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f45fc035000 tx=0x7f45fc0095c0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 shutdown_connections 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5605139501b0 0x560513d72920 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56051392f4d0 0x560513d6e690 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f45e37fe640 0 client.22647 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605139272f0 0x560513cfe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 --2- 172.21.15.94:0/3295824861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56051390ed80 0x560513cfe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 >> 172.21.15.94:0/3295824861 conn(0x560513944300 msgr2=0x560513cf08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 shutdown_connections 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 1 -- 172.21.15.94:0/3295824861 wait complete. 2024-04-25T14:01:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.459+0000 7f4611876740 20 client.22647 trim_cache size 0 max 16384 2024-04-25T14:01:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:01:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:01:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:01:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 msgr2=0x7f45ec0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 0x7f45ec0b2070 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x560513cf1b40 tx=0x7f45fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 msgr2=0x7f460419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d770 tx=0x7f45f800dc30 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 shutdown_connections 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f45ec0afb60 0x7f45ec0b2070 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4604109a80 0x7f46041a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4604104d80 0x7f460419a720 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 --2- 172.21.15.94:0/176304235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4604103b40 0x7f460419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 >> 172.21.15.94:0/176304235 conn(0x7f46040fed30 msgr2=0x7f4604105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 shutdown_connections 2024-04-25T14:01:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:01:58.460+0000 7f460a9c5640 1 -- 172.21.15.94:0/176304235 wait complete. 2024-04-25T14:01:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:01:58 smithi094 ceph-mon[29195]: pgmap v956: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-04-25T14:01:58.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:01:58 smithi138 ceph-mon[32977]: pgmap v956: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 440 op/s 2024-04-25T14:01:59.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 143779 19.01 MB/sec execute 281 sec latency 395.433 ms 2024-04-25T14:02:00.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 144119 19.02 MB/sec execute 282 sec latency 255.405 ms 2024-04-25T14:02:00.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:00 smithi137 ceph-mon[32568]: pgmap v957: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 407 op/s 2024-04-25T14:02:00.586 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:00 smithi138 ceph-mon[32977]: pgmap v957: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 407 op/s 2024-04-25T14:02:00.586 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:00 smithi094 ceph-mon[29195]: pgmap v957: 97 pgs: 97 active+clean; 278 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 407 op/s 2024-04-25T14:02:01.150 INFO:tasks.workunit.client.0.smithi094.stdout: 10 144606 19.04 MB/sec execute 283 sec latency 100.643 ms 2024-04-25T14:02:01.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:01.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 -- 172.21.15.94:0/3443025199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 msgr2=0x7f395c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 --2- 172.21.15.94:0/3443025199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c10ad40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f3954009ca0 tx=0x7f395402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 -- 172.21.15.94:0/3443025199 shutdown_connections 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 --2- 172.21.15.94:0/3443025199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c10ad40 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 --2- 172.21.15.94:0/3443025199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c103c20 0x7f395c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.528+0000 7f396460d640 1 --2- 172.21.15.94:0/3443025199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c1029e0 0x7f395c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- 172.21.15.94:0/3443025199 >> 172.21.15.94:0/3443025199 conn(0x7f395c0fdbd0 msgr2=0x7f395c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- 172.21.15.94:0/3443025199 shutdown_connections 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- 172.21.15.94:0/3443025199 wait complete. 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 Processor -- start 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- start start 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c1029e0 0x7f395c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c103c20 0x7f395c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10e510 con 0x7f395c108920 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10e390 con 0x7f395c1029e0 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.529+0000 7f396460d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10e690 con 0x7f395c103c20 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40060/0 (socket says 172.21.15.94:40060) 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 -- 172.21.15.94:0/1607927371 learned_addr learned my addr 172.21.15.94:0/1607927371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c103c20 0x7f395c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f39623a9640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c1029e0 0x7f395c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 -- 172.21.15.94:0/1607927371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c103c20 msgr2=0x7f395c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c103c20 0x7f395c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 -- 172.21.15.94:0/1607927371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c1029e0 msgr2=0x7f395c19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c1029e0 0x7f395c19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 -- 172.21.15.94:0/1607927371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c1a4280 con 0x7f395c108920 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f3962baa640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f3954004af0 tx=0x7f3954031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f39537fe640 1 -- 172.21.15.94:0/1607927371 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954003b70 con 0x7f395c108920 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f39537fe640 1 -- 172.21.15.94:0/1607927371 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3954003d10 con 0x7f395c108920 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c1a4510 con 0x7f395c108920 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f39537fe640 1 -- 172.21.15.94:0/1607927371 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395402fb20 con 0x7f395c108920 2024-04-25T14:02:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.530+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c1a4a60 con 0x7f395c108920 2024-04-25T14:02:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.533+0000 7f39537fe640 1 -- 172.21.15.94:0/1607927371 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3954004580 con 0x7f395c108920 2024-04-25T14:02:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 Processor -- start 2024-04-25T14:02:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 -- start start 2024-04-25T14:02:01.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b3efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f39537fe640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 0x7f39480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ca1aa3d80 con 0x555ca179d390 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ca1a5d8b0 con 0x555ca177c6b0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.534+0000 7f396b595740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ca1a9fd40 con 0x555ca17744d0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f39623a9640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 0x7f39480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40072/0 (socket says 172.21.15.94:40072) 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3962baa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 -- 172.21.15.94:0/215181334 learned_addr learned my addr 172.21.15.94:0/215181334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b3efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f39537fe640 1 -- 172.21.15.94:0/1607927371 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3954100290 con 0x7f395c108920 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 -- 172.21.15.94:0/215181334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 msgr2=0x555ca1b3efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b3efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 -- 172.21.15.94:0/215181334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 msgr2=0x555ca1b3fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b3fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 -- 172.21.15.94:0/215181334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca1a2ea40 con 0x555ca179d390 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3962baa640 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b3fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f39623a9640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 0x7f39480b2000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f395c1035e0 tx=0x7f3958005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3961ba8640 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f394c013820 tx=0x7f394c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4cfa03e8330e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3950bfe640 1 -- 172.21.15.94:0/215181334 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c013da0 con 0x555ca179d390 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3950bfe640 1 -- 172.21.15.94:0/215181334 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c013120 con 0x555ca179d390 2024-04-25T14:02:01.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.535+0000 7f3950bfe640 1 -- 172.21.15.94:0/215181334 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c010830 con 0x555ca179d390 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 -- 172.21.15.94:0/215181334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 msgr2=0x555ca1b3ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f394c013820 tx=0x7f394c009570 comp rx=0 tx=0).stop 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 -- 172.21.15.94:0/215181334 shutdown_connections 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b3fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b3efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 --2- 172.21.15.94:0/215181334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b3ea60 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 -- 172.21.15.94:0/215181334 >> 172.21.15.94:0/215181334 conn(0x555ca17914e0 msgr2=0x555ca1b3a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.536+0000 7f396b595740 1 -- 172.21.15.94:0/215181334 shutdown_connections 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 -- 172.21.15.94:0/215181334 wait complete. 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 -- start start 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 Processor -- start 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555ca1b40720 con 0x555ca179d390 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ca1a2ad10 con 0x555ca177c6b0 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ca1b408a0 con 0x555ca17744d0 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f396b595740 10 client.0 _ll_register_callbacks cb 0x555ca1a57210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:01.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f39623a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40078/0 (socket says 172.21.15.94:40078) 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f39623a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51776/0 (socket says 172.21.15.94:51776) 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f3962baa640 1 -- 172.21.15.94:0/970279799 learned_addr learned my addr 172.21.15.94:0/970279799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.537+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 -- 172.21.15.94:0/970279799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 msgr2=0x555ca1b48a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 -- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 msgr2=0x555ca1b48f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b48f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 -- 172.21.15.94:0/970279799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca187f7d0 con 0x555ca179d390 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f39623a9640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3961ba8640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:01.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3962baa640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f3954004530 tx=0x7f39540c84a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad542fce2f04517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39540c9800 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f39540c4c30 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c013120 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f39540c81e0 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f396b595740 10 client.22659 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.538+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555ca17b53f0 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c013120 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f39540c48d0 con 0x555ca179d390 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 10 client.22659 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 10 client.22659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 20 client.22659 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca175bf60 0x555ca1b56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 4 client.22659 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:01.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.539+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca1ab1990 con 0x555ca175bf60 2024-04-25T14:02:01.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.540+0000 7f39623a9640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca175bf60 0x555ca1b56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.540+0000 7f3945ffb640 10 client.22659 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:01.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.540+0000 7f39623a9640 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca175bf60 0x555ca1b56310 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e45f5778d01140 server_cookie=d7fbfb63e07e44f1 in_seq=0 out_seq=0 2024-04-25T14:02:01.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f3945ffb640 1 -- 172.21.15.94:0/970279799 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ca1ab1990 con 0x555ca175bf60 2024-04-25T14:02:01.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f3945ffb640 10 client.22659 handle_command_reply: tid=0 2024-04-25T14:02:01.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 client.22659 shutdown 2024-04-25T14:02:01.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 msgr2=0x555ca1b4fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f3954004530 tx=0x7f39540c84a0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 shutdown_connections 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca175bf60 0x555ca1b56310 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1b4fdc0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 --2- 172.21.15.94:0/970279799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b48a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f3945ffb640 0 client.22659 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 >> 172.21.15.94:0/970279799 conn(0x555ca17914e0 msgr2=0x555ca1b393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 shutdown_connections 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 1 -- 172.21.15.94:0/970279799 wait complete. 2024-04-25T14:02:01.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.546+0000 7f396b595740 20 client.22659 trim_cache size 0 max 16384 2024-04-25T14:02:01.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 Processor -- start 2024-04-25T14:02:01.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 -- start start 2024-04-25T14:02:01.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca179d390 0x555ca1b41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1ad600 con 0x555ca177c6b0 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1af140 con 0x555ca179d390 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f396b595740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1adcd0 con 0x555ca17744d0 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f3961ba8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40086/0 (socket says 172.21.15.94:40086) 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f3961ba8640 1 -- 172.21.15.94:0/3407356168 learned_addr learned my addr 172.21.15.94:0/3407356168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f3962baa640 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca179d390 0x555ca1b41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.567+0000 7f39623a9640 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 -- 172.21.15.94:0/3407356168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 msgr2=0x555ca1b40dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b40dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 -- 172.21.15.94:0/3407356168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca179d390 msgr2=0x555ca1b41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca179d390 0x555ca1b41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 -- 172.21.15.94:0/3407356168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca17750f0 con 0x555ca177c6b0 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3961ba8640 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x555ca1b3fab0 tx=0x7f394c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caed9a293e94eb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3947fff640 1 -- 172.21.15.94:0/3407356168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0053c0 con 0x555ca177c6b0 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3947fff640 1 -- 172.21.15.94:0/3407356168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c024070 con 0x555ca177c6b0 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.568+0000 7f3947fff640 1 -- 172.21.15.94:0/3407356168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c003960 con 0x555ca177c6b0 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- 172.21.15.94:0/3407356168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 msgr2=0x555ca1b4fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x555ca1b3fab0 tx=0x7f394c009f60 comp rx=0 tx=0).stop 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- 172.21.15.94:0/3407356168 shutdown_connections 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca179d390 0x555ca1b41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca177c6b0 0x555ca1b4fce0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 --2- 172.21.15.94:0/3407356168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b40dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- 172.21.15.94:0/3407356168 >> 172.21.15.94:0/3407356168 conn(0x555ca17914e0 msgr2=0x555ca1b47020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- 172.21.15.94:0/3407356168 shutdown_connections 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- 172.21.15.94:0/3407356168 wait complete. 2024-04-25T14:02:01.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 -- start start 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.569+0000 7f396b595740 1 Processor -- start 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1aea70 con 0x555ca179d390 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1ae3a0 con 0x555ca177c6b0 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1ab5e0 con 0x555ca17744d0 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 10 client.0 _ll_register_callbacks cb 0x7f3954009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40088/0 (socket says 172.21.15.94:40088) 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 -- 172.21.15.94:0/520844790 learned_addr learned my addr 172.21.15.94:0/520844790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f39623a9640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3961ba8640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b4cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 -- 172.21.15.94:0/520844790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 msgr2=0x555ca1b4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b4ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 -- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 msgr2=0x555ca1b4cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b4cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 -- 172.21.15.94:0/520844790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca1a644b0 con 0x555ca179d390 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f39623a9640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3962baa640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f39540094e0 tx=0x7f395402fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aaf233fb5bf02b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39540d1530 con 0x555ca179d390 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f39540c4c30 con 0x555ca179d390 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c024070 con 0x555ca179d390 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f39540c86d0 con 0x555ca179d390 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 10 client.22665 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:01.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.570+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555ca1bbd8e0 con 0x555ca179d390 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c024070 con 0x555ca179d390 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f39540c6620 con 0x555ca179d390 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 10 client.22665 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 10 client.22665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 20 client.22665 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca17313a0 0x555ca1bc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 4 client.22665 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ca1aab800 con 0x555ca17313a0 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f39623a9640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca17313a0 0x555ca1bc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f3945ffb640 10 client.22665 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.571+0000 7f39623a9640 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca17313a0 0x555ca1bc0140 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f6786842dfb4b2 server_cookie=1cc4927a67c76612 in_seq=0 out_seq=0 2024-04-25T14:02:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f3945ffb640 1 -- 172.21.15.94:0/520844790 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ca1aab800 con 0x555ca17313a0 2024-04-25T14:02:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f3945ffb640 10 client.22665 handle_command_reply: tid=0 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 client.22665 shutdown 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 msgr2=0x555ca1bbc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f39540094e0 tx=0x7f395402fef0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 shutdown_connections 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555ca17313a0 0x555ca1bc0140 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555ca179d390 0x555ca1bbc6b0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ca177c6b0 0x555ca1b4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 --2- 172.21.15.94:0/520844790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ca17744d0 0x555ca1b4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f3945ffb640 0 client.22665 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 >> 172.21.15.94:0/520844790 conn(0x555ca17914e0 msgr2=0x555ca1b3d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 shutdown_connections 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.573+0000 7f396b595740 1 -- 172.21.15.94:0/520844790 wait complete. 2024-04-25T14:02:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396b595740 20 client.22665 trim_cache size 0 max 16384 2024-04-25T14:02:01.579 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:02:01.579 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:02:01.579 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 msgr2=0x7f39480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 0x7f39480b2000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f395c1035e0 tx=0x7f3958005f90 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 msgr2=0x7f395c1a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f3954004af0 tx=0x7f3954031700 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 shutdown_connections 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39480afaf0 0x7f39480b2000 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c108920 0x7f395c1a3c10 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f395c103c20 0x7f395c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 --2- 172.21.15.94:0/1607927371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f395c1029e0 0x7f395c19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 >> 172.21.15.94:0/1607927371 conn(0x7f395c0fdbd0 msgr2=0x7f395c109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 shutdown_connections 2024-04-25T14:02:01.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.574+0000 7f396460d640 1 -- 172.21.15.94:0/1607927371 wait complete. 2024-04-25T14:02:01.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:02:01.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.701+0000 7f8da8b84640 1 -- 172.21.15.94:0/2718388897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4102a00 msgr2=0x7f8da4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.701+0000 7f8da8b84640 1 --2- 172.21.15.94:0/2718388897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4102a00 0x7f8da4103700 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d9402a410 comp rx=0 tx=0).stop 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 -- 172.21.15.94:0/2718388897 shutdown_connections 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 --2- 172.21.15.94:0/2718388897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 0x7f8da410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 --2- 172.21.15.94:0/2718388897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4103c40 0x7f8da4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 --2- 172.21.15.94:0/2718388897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4102a00 0x7f8da4103700 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 -- 172.21.15.94:0/2718388897 >> 172.21.15.94:0/2718388897 conn(0x7f8da40fdbf0 msgr2=0x7f8da41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 -- 172.21.15.94:0/2718388897 shutdown_connections 2024-04-25T14:02:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 -- 172.21.15.94:0/2718388897 wait complete. 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 Processor -- start 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 -- start start 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 0x7f8da40758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.702+0000 7f8da8b84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 0x7f8da41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410e4e0 con 0x7f8da4103c40 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410e360 con 0x7f8da4102a00 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8da410e660 con 0x7f8da4108940 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40104/0 (socket says 172.21.15.94:40104) 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 -- 172.21.15.94:0/1879252768 learned_addr learned my addr 172.21.15.94:0/1879252768 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da259c640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 0x7f8da40758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da2d9d640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 0x7f8da41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 -- 172.21.15.94:0/1879252768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 msgr2=0x7f8da41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 0x7f8da41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 -- 172.21.15.94:0/1879252768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 msgr2=0x7f8da40758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 0x7f8da40758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 -- 172.21.15.94:0/1879252768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da41a1d60 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da259c640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 0x7f8da40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9bfff640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9f0 tx=0x7f8d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00cd80 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d8c0108e0 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c018980 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41a1ff0 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.703+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41a24b0 con 0x7f8da4103c40 2024-04-25T14:02:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.704+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da4107750 con 0x7f8da4103c40 2024-04-25T14:02:01.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.706+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8d8c010430 con 0x7f8da4103c40 2024-04-25T14:02:01.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.706+0000 7f8d9b7fe640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 0x7f8d900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:01.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.707+0000 7f8da259c640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 0x7f8d900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:01.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.707+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f8d8c0df550 con 0x7f8da4103c40 2024-04-25T14:02:01.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.707+0000 7f8da259c640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 0x7f8d900b2000 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8d94005ab0 tx=0x7f8d94031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:01.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.711+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d8c0a8090 con 0x7f8da4103c40 2024-04-25T14:02:01.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.967+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8da402db50 con 0x7f8da4103c40 2024-04-25T14:02:01.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.967+0000 7f8d9b7fe640 1 -- 172.21.15.94:0/1879252768 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f8d8c0b0110 con 0x7f8da4103c40 2024-04-25T14:02:01.966 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:02:01.967 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:02:01.967 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 msgr2=0x7f8d900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 0x7f8d900b2000 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8d94005ab0 tx=0x7f8d94031400 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 msgr2=0x7f8da419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9f0 tx=0x7f8d8c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 shutdown_connections 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f8d900afaf0 0x7f8d900b2000 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da4108940 0x7f8da41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da4103c40 0x7f8da419ef90 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 --2- 172.21.15.94:0/1879252768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8da4102a00 0x7f8da40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 >> 172.21.15.94:0/1879252768 conn(0x7f8da40fdbf0 msgr2=0x7f8da4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 shutdown_connections 2024-04-25T14:02:01.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:01.970+0000 7f8da8b84640 1 -- 172.21.15.94:0/1879252768 wait complete. 2024-04-25T14:02:01.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:02:02.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 -- 172.21.15.94:0/2682518145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c110030 msgr2=0x7f754c110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 --2- 172.21.15.94:0/2682518145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c110030 0x7f754c110440 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f753c009900 tx=0x7f753c02a410 comp rx=0 tx=0).stop 2024-04-25T14:02:02.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 -- 172.21.15.94:0/2682518145 shutdown_connections 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 --2- 172.21.15.94:0/2682518145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c110030 0x7f754c110440 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 --2- 172.21.15.94:0/2682518145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c10b310 0x7f754c10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 --2- 172.21.15.94:0/2682518145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 -- 172.21.15.94:0/2682518145 >> 172.21.15.94:0/2682518145 conn(0x7f754c0704e0 msgr2=0x7f754c072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 -- 172.21.15.94:0/2682518145 shutdown_connections 2024-04-25T14:02:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.107+0000 7f75522af640 1 -- 172.21.15.94:0/2682518145 wait complete. 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.108+0000 7f75522af640 1 Processor -- start 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.108+0000 7f75522af640 1 -- start start 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c10b310 0x7f754c1a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c110030 0x7f754c1b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f754c11e490 con 0x7f754c10b310 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f754c11e310 con 0x7f754c110030 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f75522af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f754c11e610 con 0x7f754c10a420 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51826/0 (socket says 172.21.15.94:51826) 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 -- 172.21.15.94:0/4251871368 learned_addr learned my addr 172.21.15.94:0/4251871368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f755084c640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c110030 0x7f754c1b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 -- 172.21.15.94:0/4251871368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c110030 msgr2=0x7f754c1b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c110030 0x7f754c1b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 -- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c10b310 msgr2=0x7f754c1a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c10b310 0x7f754c1a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.109+0000 7f754bfff640 1 -- 172.21.15.94:0/4251871368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c1b0ab0 con 0x7f754c10a420 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.110+0000 7f754bfff640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f754400eb90 tx=0x7f754400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.110+0000 7f75497fa640 1 -- 172.21.15.94:0/4251871368 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544010630 con 0x7f754c10a420 2024-04-25T14:02:02.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.110+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c1b0da0 con 0x7f754c10a420 2024-04-25T14:02:02.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.110+0000 7f75497fa640 1 -- 172.21.15.94:0/4251871368 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754400ee90 con 0x7f754c10a420 2024-04-25T14:02:02.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.110+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1b12c0 con 0x7f754c10a420 2024-04-25T14:02:02.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.111+0000 7f75497fa640 1 -- 172.21.15.94:0/4251871368 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544010630 con 0x7f754c10a420 2024-04-25T14:02:02.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.113+0000 7f7559163740 1 Processor -- start 2024-04-25T14:02:02.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.114+0000 7f7559163740 1 -- start start 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a71f91ed80 con 0x55a71f5f76b0 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a71f8d8a20 con 0x55a71f5ef4d0 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f7559163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a71f91ad40 con 0x55a71f618390 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51842/0 (socket says 172.21.15.94:51842) 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 -- 172.21.15.94:0/4271962131 learned_addr learned my addr 172.21.15.94:0/4271962131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f75497fa640 1 -- 172.21.15.94:0/4251871368 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7544018800 con 0x7f754c10a420 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 -- 172.21.15.94:0/4271962131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 msgr2=0x55a71f9bad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:02:02.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9bad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 -- 172.21.15.94:0/4271962131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 msgr2=0x55a71f9bba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.115+0000 7f755084c640 1 -- 172.21.15.94:0/4271962131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71f8e3c30 con 0x55a71f618390 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f755084c640 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f753c009900 tx=0x7f753c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6cd209e4f088d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f75327fc640 1 -- 172.21.15.94:0/4271962131 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c035030 con 0x55a71f618390 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f75497fa640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 0x7f75340b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f75327fc640 1 -- 172.21.15.94:0/4271962131 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753c040070 con 0x55a71f618390 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f75327fc640 1 -- 172.21.15.94:0/4271962131 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c003b50 con 0x55a71f618390 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.116+0000 7f754bfff640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 0x7f75340b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f75497fa640 1 -- 172.21.15.94:0/4251871368 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f75440def40 con 0x7f754c10a420 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 -- 172.21.15.94:0/4271962131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 msgr2=0x55a71f9ba810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f753c009900 tx=0x7f753c009680 comp rx=0 tx=0).stop 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f754bfff640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 0x7f75340b2200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f754400cba0 tx=0x7f754400cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 -- 172.21.15.94:0/4271962131 shutdown_connections 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9bba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 --2- 172.21.15.94:0/4271962131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9ba810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 -- 172.21.15.94:0/4271962131 >> 172.21.15.94:0/4271962131 conn(0x55a71f60c4e0 msgr2=0x55a71f9b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.117+0000 7f7559163740 1 -- 172.21.15.94:0/4271962131 shutdown_connections 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.118+0000 7f7559163740 1 -- 172.21.15.94:0/4271962131 wait complete. 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.118+0000 7f7559163740 1 -- start start 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 Processor -- start 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a71f9bc4d0 con 0x55a71f5ef4d0 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a71f8a5dc0 con 0x55a71f618390 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a71f9bc650 con 0x55a71f5f76b0 2024-04-25T14:02:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f7559163740 10 client.0 _ll_register_callbacks cb 0x55a71f8c1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f755084c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f755084c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40130/0 (socket says 172.21.15.94:40130) 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f755084c640 1 -- 172.21.15.94:0/139921257 learned_addr learned my addr 172.21.15.94:0/139921257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f755084c640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.119+0000 7f754b7fe640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 -- 172.21.15.94:0/139921257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 msgr2=0x55a71f9dca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9dca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 -- 172.21.15.94:0/139921257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 msgr2=0x55a71f9d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 -- 172.21.15.94:0/139921257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71f6303f0 con 0x55a71f5f76b0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f754b7fe640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9d5ad0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f754c1aa0c0 tx=0x7f754000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e81a8c58aea1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f752b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:02:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540014070 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f754000e880 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.120+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f753c040070 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.121+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7540020050 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.121+0000 7f7559163740 10 client.31738 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.121+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a71f6fa7d0 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.121+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753c040070 con 0x55a71f5f76b0 2024-04-25T14:02:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f754000fc70 con 0x55a71f5f76b0 2024-04-25T14:02:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 10 client.31738 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 10 client.31738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:02:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 20 client.31738 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5d6f60 0x55a71f9e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 4 client.31738 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a71f92c990 con 0x55a71f5d6f60 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.122+0000 7f755084c640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5d6f60 0x55a71f9e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.123+0000 7f755084c640 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5d6f60 0x55a71f9e32e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56b3351743f9bed3 server_cookie=a9c20d8b5f090825 in_seq=0 out_seq=0 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.123+0000 7f752b7fe640 10 client.31738 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f752b7fe640 1 -- 172.21.15.94:0/139921257 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a71f92c990 con 0x55a71f5d6f60 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f752b7fe640 10 client.31738 handle_command_reply: tid=0 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 client.31738 shutdown 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 msgr2=0x55a71f9d5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9d5ad0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f754c1aa0c0 tx=0x7f754000c7e0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 shutdown_connections 2024-04-25T14:02:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5d6f60 0x55a71f9e32e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9dca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9d5ad0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 --2- 172.21.15.94:0/139921257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 >> 172.21.15.94:0/139921257 conn(0x55a71f60c4e0 msgr2=0x55a71f9b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.124+0000 7f752b7fe640 0 client.31738 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.125+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 shutdown_connections 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.125+0000 7f7559163740 1 -- 172.21.15.94:0/139921257 wait complete. 2024-04-25T14:02:02.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.125+0000 7f7559163740 20 client.31738 trim_cache size 0 max 16384 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 Processor -- start 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 -- start start 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b1a90 con 0x55a71f5f76b0 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b35d0 con 0x55a71f5ef4d0 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f7559163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b2160 con 0x55a71f618390 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f754b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f755084c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f755084c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9e7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55082/0 (socket says 172.21.15.94:55082) 2024-04-25T14:02:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f755084c640 1 -- 172.21.15.94:0/4174251046 learned_addr learned my addr 172.21.15.94:0/4174251046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.144+0000 7f755084c640 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 -- 172.21.15.94:0/4174251046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 msgr2=0x55a71f9e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 -- 172.21.15.94:0/4174251046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 msgr2=0x55a71f9e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 -- 172.21.15.94:0/4174251046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71f9dce90 con 0x55a71f5f76b0 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f754b7fe640 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9dc950 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f7540019680 tx=0x7f754000cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933c5838f0990e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f7531ffb640 1 -- 172.21.15.94:0/4174251046 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540014040 con 0x55a71f5f76b0 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f7531ffb640 1 -- 172.21.15.94:0/4174251046 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754000a350 con 0x55a71f5f76b0 2024-04-25T14:02:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.145+0000 7f7531ffb640 1 -- 172.21.15.94:0/4174251046 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000e8e0 con 0x55a71f5f76b0 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 -- 172.21.15.94:0/4174251046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 msgr2=0x55a71f9dc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9dc950 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f7540019680 tx=0x7f754000cfd0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 -- 172.21.15.94:0/4174251046 shutdown_connections 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f618390 0x55a71f9e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5f76b0 0x55a71f9dc950 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 --2- 172.21.15.94:0/4174251046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f5ef4d0 0x55a71f9e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 -- 172.21.15.94:0/4174251046 >> 172.21.15.94:0/4174251046 conn(0x55a71f60c4e0 msgr2=0x55a71f9bf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 -- 172.21.15.94:0/4174251046 shutdown_connections 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.146+0000 7f7559163740 1 -- 172.21.15.94:0/4174251046 wait complete. 2024-04-25T14:02:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 -- start start 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 Processor -- start 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9db100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b2f00 con 0x55a71f5ef4d0 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1b2830 con 0x55a71f618390 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f754c118830 con 0x55a71f5f76b0 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f7559163740 10 client.0 _ll_register_callbacks cb 0x7f7540005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f755084c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f755084c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40150/0 (socket says 172.21.15.94:40150) 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f754b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f755084c640 1 -- 172.21.15.94:0/2725089109 learned_addr learned my addr 172.21.15.94:0/2725089109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f755084c640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9db100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.147+0000 7f754b7fe640 1 -- 172.21.15.94:0/2725089109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 msgr2=0x55a71f9db100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f754b7fe640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9db100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f754b7fe640 1 -- 172.21.15.94:0/2725089109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 msgr2=0x55a71f9d9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f754b7fe640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f754b7fe640 1 -- 172.21.15.94:0/2725089109 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71f9db6c0 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f754b7fe640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9da1d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7540000c00 tx=0x7f75400198e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f08a06d62d4d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f752b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540030070 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f754000a350 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f754000e680 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f754003c050 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f7559163740 10 client.31744 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a71f9dc940 con 0x55a71f5f76b0 2024-04-25T14:02:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.148+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754000a350 con 0x55a71f5f76b0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f75400035f0 con 0x55a71f5f76b0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 10 client.31744 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 10 client.31744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 20 client.31744 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5ac3a0 0x55a71fa4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 4 client.31744 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a71f926800 con 0x55a71f5ac3a0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f755084c640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5ac3a0 0x55a71fa4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f752b7fe640 10 client.31744 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f755084c640 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5ac3a0 0x55a71fa4dad0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=454b84bfacc3ab88 server_cookie=c8738b7591c6ce91 in_seq=0 out_seq=0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f752b7fe640 1 -- 172.21.15.94:0/2725089109 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a71f926800 con 0x55a71f5ac3a0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f752b7fe640 10 client.31744 handle_command_reply: tid=0 2024-04-25T14:02:02.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 1 client.31744 shutdown 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.149+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 msgr2=0x55a71f9da1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9da1d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7540000c00 tx=0x7f75400198e0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 shutdown_connections 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55a71f5ac3a0 0x55a71fa4dad0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a71f618390 0x55a71f9db100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f752b7fe640 0 client.31744 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a71f5f76b0 0x55a71f9da1d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 --2- 172.21.15.94:0/2725089109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a71f5ef4d0 0x55a71f9d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 >> 172.21.15.94:0/2725089109 conn(0x55a71f60c4e0 msgr2=0x55a71f9bf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 shutdown_connections 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 1 -- 172.21.15.94:0/2725089109 wait complete. 2024-04-25T14:02:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f7559163740 20 client.31744 trim_cache size 0 max 16384 2024-04-25T14:02:02.155 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:02.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 msgr2=0x7f75340b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 0x7f75340b2200 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f754400cba0 tx=0x7f754400cf90 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 msgr2=0x7f754c1a9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f754400eb90 tx=0x7f754400c6a0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 shutdown_connections 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75340afcf0 0x7f75340b2200 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f754c110030 0x7f754c1b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c10b310 0x7f754c1a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 --2- 172.21.15.94:0/4251871368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c10a420 0x7f754c1a9020 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 >> 172.21.15.94:0/4251871368 conn(0x7f754c0704e0 msgr2=0x7f754c10c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 shutdown_connections 2024-04-25T14:02:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.150+0000 7f75522af640 1 -- 172.21.15.94:0/4251871368 wait complete. 2024-04-25T14:02:02.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 145153 19.06 MB/sec execute 284 sec latency 56.425 ms 2024-04-25T14:02:02.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:02:02.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.286+0000 7faafb9e3640 1 -- 172.21.15.94:0/936515572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 msgr2=0x7faaf410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.286+0000 7faafb9e3640 1 --2- 172.21.15.94:0/936515572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf410bcc0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7faaec009ca0 tx=0x7faaec02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.286+0000 7faafb9e3640 1 -- 172.21.15.94:0/936515572 shutdown_connections 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.286+0000 7faafb9e3640 1 --2- 172.21.15.94:0/936515572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf410bcc0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.286+0000 7faafb9e3640 1 --2- 172.21.15.94:0/936515572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 0x7faaf4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 --2- 172.21.15.94:0/936515572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 -- 172.21.15.94:0/936515572 >> 172.21.15.94:0/936515572 conn(0x7faaf40fd380 msgr2=0x7faaf40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 -- 172.21.15.94:0/936515572 shutdown_connections 2024-04-25T14:02:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 -- 172.21.15.94:0/936515572 wait complete. 2024-04-25T14:02:02.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 Processor -- start 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 -- start start 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 0x7faaf419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.287+0000 7faafb9e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faafb9e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410f620 con 0x7faaf4102260 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faafb9e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410f4a0 con 0x7faaf4101760 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faafb9e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410f7a0 con 0x7faaf41050b0 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40168/0 (socket says 172.21.15.94:40168) 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 -- 172.21.15.94:0/1351145467 learned_addr learned my addr 172.21.15.94:0/1351145467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf977f640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 0x7faaf419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf9f80640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 -- 172.21.15.94:0/1351145467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 msgr2=0x7faaf41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 -- 172.21.15.94:0/1351145467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 msgr2=0x7faaf419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 0x7faaf419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 -- 172.21.15.94:0/1351145467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf41a1d10 con 0x7faaf4102260 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf9f80640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7faae400e9f0 tx=0x7faae400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaea7fc640 1 -- 172.21.15.94:0/1351145467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400cd80 con 0x7faaf4102260 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaea7fc640 1 -- 172.21.15.94:0/1351145467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae40108e0 con 0x7faaf4102260 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41a1fa0 con 0x7faaf4102260 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faaea7fc640 1 -- 172.21.15.94:0/1351145467 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4018980 con 0x7faaf4102260 2024-04-25T14:02:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.288+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a2490 con 0x7faaf4102260 2024-04-25T14:02:02.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 Processor -- start 2024-04-25T14:02:02.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaea7fc640 1 -- 172.21.15.94:0/1351145467 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7faae4010430 con 0x7faaf4102260 2024-04-25T14:02:02.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 -- start start 2024-04-25T14:02:02.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56238fe79d80 con 0x56238fb73390 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56238fe33a20 con 0x56238fb4a4d0 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7fab02969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56238fe75d40 con 0x56238fb526b0 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf977f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40184/0 (socket says 172.21.15.94:40184) 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf977f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55126/0 (socket says 172.21.15.94:55126) 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf9f80640 1 -- 172.21.15.94:0/3350277478 learned_addr learned my addr 172.21.15.94:0/3350277478 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.291+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 -- 172.21.15.94:0/3350277478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 msgr2=0x56238ff16a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff16a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 -- 172.21.15.94:0/3350277478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 msgr2=0x56238ff15d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaea7fc640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 0x7faae00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 -- 172.21.15.94:0/3350277478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238fe3ec30 con 0x56238fb73390 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf977f640 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 0x7faae00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7faaec031a90 tx=0x7faaec02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4270c1f2e80b7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faaea7fc640 1 -- 172.21.15.94:0/1351145467 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7faae40df040 con 0x7faaf4102260 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faadf7fe640 1 -- 172.21.15.94:0/3350277478 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec031650 con 0x56238fb73390 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faadf7fe640 1 -- 172.21.15.94:0/3350277478 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec036070 con 0x56238fb73390 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.292+0000 7faadf7fe640 1 -- 172.21.15.94:0/3350277478 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004660 con 0x56238fb73390 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 -- 172.21.15.94:0/3350277478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 msgr2=0x56238ff15810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7faaf8f7e640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 0x7faae00b2000 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faae40a46f0 tx=0x7faae40e2b20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7faaec031a90 tx=0x7faaec02f420 comp rx=0 tx=0).stop 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 -- 172.21.15.94:0/3350277478 shutdown_connections 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff16a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff15d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 --2- 172.21.15.94:0/3350277478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff15810 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 -- 172.21.15.94:0/3350277478 >> 172.21.15.94:0/3350277478 conn(0x56238fb674e0 msgr2=0x56238ff11020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 -- 172.21.15.94:0/3350277478 shutdown_connections 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.293+0000 7fab02969740 1 -- 172.21.15.94:0/3350277478 wait complete. 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 -- start start 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 Processor -- start 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff1f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56238ff174d0 con 0x56238fb73390 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56238fe00dc0 con 0x56238fb4a4d0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56238ff17650 con 0x56238fb526b0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7fab02969740 10 client.0 _ll_register_callbacks cb 0x56238fe1c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40198/0 (socket says 172.21.15.94:40198) 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7faaf9f80640 1 -- 172.21.15.94:0/2298789460 learned_addr learned my addr 172.21.15.94:0/2298789460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7faaf9f80640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.294+0000 7faaf977f640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 -- 172.21.15.94:0/2298789460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 msgr2=0x56238ff1faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff1faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 -- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 msgr2=0x56238ff1f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff1f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 -- 172.21.15.94:0/2298789460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238fb8b3f0 con 0x56238fb73390 2024-04-25T14:02:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faaf9f80640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7faaec0096a0 tx=0x7faaec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b62890ab6b56b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec049040 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7faaec031450 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec036070 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faaec031650 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7fab02969740 10 client.22695 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56238fc557d0 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec036070 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7faaec04f070 con 0x56238fb73390 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.295+0000 7fab02969740 10 client.22695 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7fab02969740 10 client.22695 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:02:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7fab02969740 20 client.22695 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb31f60 0x56238ff2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7fab02969740 4 client.22695 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56238fe87990 con 0x56238fb31f60 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7faaf977f640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb31f60 0x56238ff2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7faadcff9640 10 client.22695 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.296+0000 7faaf977f640 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb31f60 0x56238ff2ce50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4496fc6916f9896 server_cookie=608d420c31a3abc in_seq=0 out_seq=0 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.297+0000 7faadcff9640 1 -- 172.21.15.94:0/2298789460 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56238fe87990 con 0x56238fb31f60 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.297+0000 7faadcff9640 10 client.22695 handle_command_reply: tid=0 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.297+0000 7fab02969740 1 client.22695 shutdown 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.297+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 msgr2=0x56238ff26a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.297+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7faaec0096a0 tx=0x7faaec005c50 comp rx=0 tx=0).stop 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 shutdown_connections 2024-04-25T14:02:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb31f60 0x56238ff2ce50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb73390 0x56238ff26a00 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7faadcff9640 0 client.22695 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff1faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 --2- 172.21.15.94:0/2298789460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb4a4d0 0x56238ff1f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 >> 172.21.15.94:0/2298789460 conn(0x56238fb674e0 msgr2=0x56238ff10040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 shutdown_connections 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 1 -- 172.21.15.94:0/2298789460 wait complete. 2024-04-25T14:02:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.298+0000 7fab02969740 20 client.22695 trim_cache size 0 max 16384 2024-04-25T14:02:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.312+0000 7fab02969740 1 Processor -- start 2024-04-25T14:02:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.312+0000 7fab02969740 1 -- start start 2024-04-25T14:02:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.312+0000 7fab02969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7fab02969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb4a4d0 0x56238ff1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7fab02969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7fab02969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ab580 con 0x56238fb526b0 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7fab02969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ad0c0 con 0x56238fb31f60 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7fab02969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41abc50 con 0x56238fb4a4d0 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40202/0 (socket says 172.21.15.94:40202) 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 -- 172.21.15.94:0/532074758 learned_addr learned my addr 172.21.15.94:0/532074758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf977f640 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb4a4d0 0x56238ff1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 -- 172.21.15.94:0/532074758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb4a4d0 msgr2=0x56238ff1e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb4a4d0 0x56238ff1e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 -- 172.21.15.94:0/532074758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 msgr2=0x56238ff1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 -- 172.21.15.94:0/532074758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238ff16840 con 0x56238fb526b0 2024-04-25T14:02:02.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faaf9f80640 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x56238ff05980 tx=0x7faaec02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f82d65b6fab9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faadeffd640 1 -- 172.21.15.94:0/532074758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0317f0 con 0x56238fb526b0 2024-04-25T14:02:02.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faadeffd640 1 -- 172.21.15.94:0/532074758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec036070 con 0x56238fb526b0 2024-04-25T14:02:02.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.313+0000 7faadeffd640 1 -- 172.21.15.94:0/532074758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec02fce0 con 0x56238fb526b0 2024-04-25T14:02:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 -- 172.21.15.94:0/532074758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 msgr2=0x56238ff314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x56238ff05980 tx=0x7faaec02bfd0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 -- 172.21.15.94:0/532074758 shutdown_connections 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb526b0 0x56238ff314a0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb4a4d0 0x56238ff1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 --2- 172.21.15.94:0/532074758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 -- 172.21.15.94:0/532074758 >> 172.21.15.94:0/532074758 conn(0x56238fb674e0 msgr2=0x56238ff1a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 -- 172.21.15.94:0/532074758 shutdown_connections 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.314+0000 7fab02969740 1 -- 172.21.15.94:0/532074758 wait complete. 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 -- start start 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 Processor -- start 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff23140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ac9f0 con 0x56238fb4a4d0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ac320 con 0x56238fb31f60 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41a9560 con 0x56238fb526b0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7fab02969740 10 client.0 _ll_register_callbacks cb 0x7faaf000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7faaf977f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7faaf977f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40210/0 (socket says 172.21.15.94:40210) 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7faaf977f640 1 -- 172.21.15.94:0/3693253245 learned_addr learned my addr 172.21.15.94:0/3693253245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff23140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.315+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 -- 172.21.15.94:0/3693253245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 msgr2=0x56238ff245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 -- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 msgr2=0x56238ff23140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff23140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 -- 172.21.15.94:0/3693253245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238ff24b70 con 0x56238fb4a4d0 2024-04-25T14:02:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff23140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faaf977f640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c010 tx=0x7faaf00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e2b97b5a76e6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0005220 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7faaf00053c0 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec036070 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faaf002f050 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 10 client.22701 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56238ff25ed0 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec036070 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7faaf00116d0 con 0x56238fb4a4d0 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 10 client.22701 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 10 client.22701 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:02:02.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 20 client.22701 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb73390 0x56238ff97410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 4 client.22701 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.316+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56238fe81800 con 0x56238fb73390 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb73390 0x56238ff97410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faadcff9640 10 client.22701 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faaf9f80640 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb73390 0x56238ff97410 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=418c9d08e70207ca server_cookie=542843d46e5ab5d9 in_seq=0 out_seq=0 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faadcff9640 1 -- 172.21.15.94:0/3693253245 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56238fe81800 con 0x56238fb73390 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faadcff9640 10 client.22701 handle_command_reply: tid=0 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 client.22701 shutdown 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 msgr2=0x56238ff23680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c010 tx=0x7faaf00035f0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 shutdown_connections 2024-04-25T14:02:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56238fb73390 0x56238ff97410 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56238fb526b0 0x56238ff245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7faadcff9640 0 client.22701 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.317+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56238fb4a4d0 0x56238ff23680 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7fab02969740 1 --2- 172.21.15.94:0/3693253245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56238fb31f60 0x56238ff23140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 >> 172.21.15.94:0/3693253245 conn(0x56238fb674e0 msgr2=0x56238ff1dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 shutdown_connections 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7fab02969740 1 -- 172.21.15.94:0/3693253245 wait complete. 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7fab02969740 20 client.22701 trim_cache size 0 max 16384 2024-04-25T14:02:02.323 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 msgr2=0x7faae00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 0x7faae00b2000 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faae40a46f0 tx=0x7faae40e2b20 comp rx=0 tx=0).stop 2024-04-25T14:02:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 msgr2=0x7faaf419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7faae400e9f0 tx=0x7faae400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 shutdown_connections 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7faae00afaf0 0x7faae00b2000 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf41050b0 0x7faaf41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaf4102260 0x7faaf419a710 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 --2- 172.21.15.94:0/1351145467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf4101760 0x7faaf419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 >> 172.21.15.94:0/1351145467 conn(0x7faaf40fd380 msgr2=0x7faaf4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 shutdown_connections 2024-04-25T14:02:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.318+0000 7faafb9e3640 1 -- 172.21.15.94:0/1351145467 wait complete. 2024-04-25T14:02:02.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:02 smithi094 ceph-mon[29195]: pgmap v958: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 579 op/s 2024-04-25T14:02:02.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:02 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1879252768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:02.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:02:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:02 smithi137 ceph-mon[32568]: pgmap v958: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 579 op/s 2024-04-25T14:02:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:02 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1879252768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:02.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 -- 172.21.15.94:0/2193307699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 msgr2=0x7fefa010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2193307699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa010ad60 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fef98009ca0 tx=0x7fef9802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 -- 172.21.15.94:0/2193307699 shutdown_connections 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2193307699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa010ad60 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2193307699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0103c40 0x7fefa0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2193307699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0102a00 0x7fefa0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 -- 172.21.15.94:0/2193307699 >> 172.21.15.94:0/2193307699 conn(0x7fefa00fdbf0 msgr2=0x7fefa01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 -- 172.21.15.94:0/2193307699 shutdown_connections 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.452+0000 7fefa5b72640 1 -- 172.21.15.94:0/2193307699 wait complete. 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 Processor -- start 2024-04-25T14:02:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 -- start start 2024-04-25T14:02:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0102a00 0x7fefa01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 0x7fefa0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010e530 con 0x7fefa0108940 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010e3b0 con 0x7fefa0103c40 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fefa5b72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010e6b0 con 0x7fefa0102a00 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fef9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fef9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40218/0 (socket says 172.21.15.94:40218) 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fef9ffff640 1 -- 172.21.15.94:0/2237914237 learned_addr learned my addr 172.21.15.94:0/2237914237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.453+0000 7fef9effd640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 0x7fefa0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0102a00 0x7fefa01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 -- 172.21.15.94:0/2237914237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0102a00 msgr2=0x7fefa01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0102a00 0x7fefa01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 -- 172.21.15.94:0/2237914237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 msgr2=0x7fefa0199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 0x7fefa0199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 -- 172.21.15.94:0/2237914237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01a0bd0 con 0x7fefa0108940 2024-04-25T14:02:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9effd640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 0x7fefa0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9ffff640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fef980097b0 tx=0x7fef98031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9cff9640 1 -- 172.21.15.94:0/2237914237 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98003bf0 con 0x7fefa0108940 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9cff9640 1 -- 172.21.15.94:0/2237914237 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98003d90 con 0x7fefa0108940 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fef9cff9640 1 -- 172.21.15.94:0/2237914237 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9802fc30 con 0x7fefa0108940 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01a0e60 con 0x7fefa0108940 2024-04-25T14:02:02.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.454+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01a1320 con 0x7fefa0108940 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 Processor -- start 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9cff9640 1 -- 172.21.15.94:0/2237914237 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fef98004040 con 0x7fefa0108940 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 -- start start 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5563bf725d80 con 0x5563bf41f390 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563bf6dfa20 con 0x5563bf3f64d0 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fefaca23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563bf721d40 con 0x5563bf3fe6b0 2024-04-25T14:02:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40230/0 (socket says 172.21.15.94:40230) 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 -- 172.21.15.94:0/3245773898 learned_addr learned my addr 172.21.15.94:0/3245773898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9cff9640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 0x7fef880b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 -- 172.21.15.94:0/3245773898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 msgr2=0x5563bf7c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 -- 172.21.15.94:0/3245773898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 msgr2=0x5563bf7c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.457+0000 7fef9effd640 1 -- 172.21.15.94:0/3245773898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bf6eac30 con 0x5563bf41f390 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef9ffff640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 0x7fef880b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef9effd640 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fef940104f0 tx=0x7fef940109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69421c2c41885cee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef9cff9640 1 -- 172.21.15.94:0/2237914237 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef981002a0 con 0x7fefa0108940 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef85ffb640 1 -- 172.21.15.94:0/3245773898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94004250 con 0x5563bf41f390 2024-04-25T14:02:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef85ffb640 1 -- 172.21.15.94:0/3245773898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef940043f0 con 0x5563bf41f390 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.458+0000 7fef85ffb640 1 -- 172.21.15.94:0/3245773898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94011a20 con 0x5563bf41f390 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- 172.21.15.94:0/3245773898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 msgr2=0x5563bf7c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fef9ffff640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 0x7fef880b2080 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef9804d000 tx=0x7fef98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fef940104f0 tx=0x7fef940109b0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- 172.21.15.94:0/3245773898 shutdown_connections 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 --2- 172.21.15.94:0/3245773898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7c1810 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- 172.21.15.94:0/3245773898 >> 172.21.15.94:0/3245773898 conn(0x5563bf4134e0 msgr2=0x5563bf7bd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- 172.21.15.94:0/3245773898 shutdown_connections 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- 172.21.15.94:0/3245773898 wait complete. 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 -- start start 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.459+0000 7fefaca23740 1 Processor -- start 2024-04-25T14:02:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3f64d0 0x5563bf7cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3fe6b0 0x5563bf7cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5563bf7c3530 con 0x5563bf41f390 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563bf6acdc0 con 0x5563bf3fe6b0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563bf7c36b0 con 0x5563bf3f64d0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 10 client.0 _ll_register_callbacks cb 0x5563bf6c8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40234/0 (socket says 172.21.15.94:40234) 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 -- 172.21.15.94:0/1511190574 learned_addr learned my addr 172.21.15.94:0/1511190574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3f64d0 0x5563bf7cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3fe6b0 0x5563bf7cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 -- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3f64d0 msgr2=0x5563bf7cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3f64d0 0x5563bf7cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 -- 172.21.15.94:0/1511190574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3fe6b0 msgr2=0x5563bf7cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3fe6b0 0x5563bf7cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 -- 172.21.15.94:0/1511190574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bf4373f0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef9effd640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fef940043a0 tx=0x7fef94013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961dcef5a587d8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef7effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9402e060 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef940035f0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.460+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef940043f0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef940037f0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 10 client.22713 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5563bf5017d0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef940043f0 con 0x5563bf41f390 2024-04-25T14:02:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fef94033860 con 0x5563bf41f390 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 10 client.22713 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 10 client.22713 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 20 client.22713 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf3ddf60 0x5563bf7d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 4 client.22713 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563bf733990 con 0x5563bf3ddf60 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.461+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf3ddf60 0x5563bf7d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.462+0000 7fef7effd640 10 client.22713 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.462+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf3ddf60 0x5563bf7d8f20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b23dd0bfa7b319e0 server_cookie=c0a89521750e54a5 in_seq=0 out_seq=0 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.462+0000 7fef7effd640 1 -- 172.21.15.94:0/1511190574 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563bf733990 con 0x5563bf3ddf60 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.462+0000 7fef7effd640 10 client.22713 handle_command_reply: tid=0 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.462+0000 7fefaca23740 1 client.22713 shutdown 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 msgr2=0x5563bf7d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fef940043a0 tx=0x7fef94013040 comp rx=0 tx=0).stop 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 shutdown_connections 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf3ddf60 0x5563bf7d8f20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf41f390 0x5563bf7d2a10 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fef7effd640 0 client.22713 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3fe6b0 0x5563bf7cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 --2- 172.21.15.94:0/1511190574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3f64d0 0x5563bf7cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 >> 172.21.15.94:0/1511190574 conn(0x5563bf4134e0 msgr2=0x5563bf7bff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 shutdown_connections 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 1 -- 172.21.15.94:0/1511190574 wait complete. 2024-04-25T14:02:02.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.463+0000 7fefaca23740 20 client.22713 trim_cache size 0 max 16384 2024-04-25T14:02:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 Processor -- start 2024-04-25T14:02:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 -- start start 2024-04-25T14:02:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 0x5563bf7c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ab510 con 0x5563bf3fe6b0 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ad050 con 0x5563bf3f64d0 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fefaca23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01abbe0 con 0x5563bf3ddf60 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fef9f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fef9effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40248/0 (socket says 172.21.15.94:40248) 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fef9f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55192/0 (socket says 172.21.15.94:55192) 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.478+0000 7fef9effd640 1 -- 172.21.15.94:0/2263783980 learned_addr learned my addr 172.21.15.94:0/2263783980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9effd640 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 0x5563bf7c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 -- 172.21.15.94:0/2263783980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 msgr2=0x5563bf7c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 0x5563bf7c17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 -- 172.21.15.94:0/2263783980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 msgr2=0x5563bf7d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 -- 172.21.15.94:0/2263783980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bf7d2e70 con 0x5563bf3f64d0 2024-04-25T14:02:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9effd640 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 0x5563bf7c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5563bf7d9550 tx=0x7fef90005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8dfa3974187d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef857fa640 1 -- 172.21.15.94:0/2263783980 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90023070 con 0x5563bf3f64d0 2024-04-25T14:02:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef857fa640 1 -- 172.21.15.94:0/2263783980 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef90010430 con 0x5563bf3f64d0 2024-04-25T14:02:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.479+0000 7fef857fa640 1 -- 172.21.15.94:0/2263783980 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900044e0 con 0x5563bf3f64d0 2024-04-25T14:02:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 -- 172.21.15.94:0/2263783980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 msgr2=0x5563bf7c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5563bf7d9550 tx=0x7fef90005f90 comp rx=0 tx=0).stop 2024-04-25T14:02:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 -- 172.21.15.94:0/2263783980 shutdown_connections 2024-04-25T14:02:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3f64d0 0x5563bf7c1d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 --2- 172.21.15.94:0/2263783980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3ddf60 0x5563bf7c17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 -- 172.21.15.94:0/2263783980 >> 172.21.15.94:0/2263783980 conn(0x5563bf4134e0 msgr2=0x5563bf7cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 -- 172.21.15.94:0/2263783980 shutdown_connections 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.480+0000 7fefaca23740 1 -- 172.21.15.94:0/2263783980 wait complete. 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 -- start start 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 Processor -- start 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 0x5563bf7cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ac980 con 0x5563bf3f64d0 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ac2b0 con 0x5563bf3ddf60 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01a94f0 con 0x5563bf3fe6b0 2024-04-25T14:02:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fefaca23740 10 client.0 _ll_register_callbacks cb 0x7fef900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fef9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fef9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40258/0 (socket says 172.21.15.94:40258) 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fef9f7fe640 1 -- 172.21.15.94:0/68293064 learned_addr learned my addr 172.21.15.94:0/68293064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fef9effd640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 0x5563bf7cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.481+0000 7fef9effd640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 -- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 msgr2=0x5563bf7d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 -- 172.21.15.94:0/68293064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 msgr2=0x5563bf7d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 -- 172.21.15.94:0/68293064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bf7d15f0 con 0x5563bf3ddf60 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef9effd640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 0x5563bf7cfbc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef94033c60 tx=0x7fef9400be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a52cf4ffc1a7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef7effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94039040 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef90010430 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef94004360 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef9402e040 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fefaca23740 10 client.32129 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5563bf7d2870 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.482+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef90010430 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fef9401a720 con 0x5563bf3ddf60 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 10 client.32129 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 10 client.32129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 20 client.32129 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf41f390 0x5563bf843490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 4 client.32129 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563bf72d800 con 0x5563bf41f390 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf41f390 0x5563bf843490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fef7effd640 10 client.32129 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.483+0000 7fef9f7fe640 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf41f390 0x5563bf843490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2cd0fee7e716138 server_cookie=e3244b3fbb02f9fb in_seq=0 out_seq=0 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fef7effd640 1 -- 172.21.15.94:0/68293064 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563bf72d800 con 0x5563bf41f390 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fef7effd640 10 client.32129 handle_command_reply: tid=0 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 client.32129 shutdown 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 msgr2=0x5563bf7cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 0x5563bf7cfbc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef94033c60 tx=0x7fef9400be70 comp rx=0 tx=0).stop 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 shutdown_connections 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5563bf41f390 0x5563bf843490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563bf3fe6b0 0x5563bf7d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563bf3f64d0 0x5563bf7d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 --2- 172.21.15.94:0/68293064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563bf3ddf60 0x5563bf7cfbc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fef7effd640 0 client.32129 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 >> 172.21.15.94:0/68293064 conn(0x5563bf4134e0 msgr2=0x5563bf7cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 shutdown_connections 2024-04-25T14:02:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 1 -- 172.21.15.94:0/68293064 wait complete. 2024-04-25T14:02:02.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefaca23740 20 client.32129 trim_cache size 0 max 16384 2024-04-25T14:02:02.492 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 msgr2=0x7fef880b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 0x7fef880b2080 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef9804d000 tx=0x7fef98005fd0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 msgr2=0x7fefa01a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fef980097b0 tx=0x7fef98031840 comp rx=0 tx=0).stop 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.484+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 shutdown_connections 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef880afb70 0x7fef880b2080 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0108940 0x7fefa01a0560 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0103c40 0x7fefa0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 --2- 172.21.15.94:0/2237914237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0102a00 0x7fefa01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 >> 172.21.15.94:0/2237914237 conn(0x7fefa00fdbf0 msgr2=0x7fefa0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 shutdown_connections 2024-04-25T14:02:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.485+0000 7fefa5b72640 1 -- 172.21.15.94:0/2237914237 wait complete. 2024-04-25T14:02:02.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:02:02.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:02 smithi138 ceph-mon[32977]: pgmap v958: 97 pgs: 97 active+clean; 263 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.2 MiB/s wr, 579 op/s 2024-04-25T14:02:02.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:02 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1879252768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:02.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 -- 172.21.15.94:0/1672050223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 msgr2=0x7f243410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 --2- 172.21.15.94:0/1672050223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f243410ad40 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f2430009900 tx=0x7f243002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 -- 172.21.15.94:0/1672050223 shutdown_connections 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 --2- 172.21.15.94:0/1672050223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f243410ad40 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 --2- 172.21.15.94:0/1672050223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f2434104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 --2- 172.21.15.94:0/1672050223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f24341036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 -- 172.21.15.94:0/1672050223 >> 172.21.15.94:0/1672050223 conn(0x7f24340fdbd0 msgr2=0x7f2434100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 -- 172.21.15.94:0/1672050223 shutdown_connections 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.616+0000 7f243b692640 1 -- 172.21.15.94:0/1672050223 wait complete. 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 Processor -- start 2024-04-25T14:02:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 -- start start 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f243419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f24341a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e510 con 0x7f2434108920 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e390 con 0x7f2434103c20 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243b692640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f243410e690 con 0x7f24341029e0 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f2438c2d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f243942e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f243419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f2438c2d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55220/0 (socket says 172.21.15.94:55220) 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f2439c2f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f24341a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.617+0000 7f2438c2d640 1 -- 172.21.15.94:0/4219765672 learned_addr learned my addr 172.21.15.94:0/4219765672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 -- 172.21.15.94:0/4219765672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 msgr2=0x7f243419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f243419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 -- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 msgr2=0x7f24341a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f24341a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 -- 172.21.15.94:0/4219765672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24341a20f0 con 0x7f2434103c20 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f243942e640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f243419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f2438c2d640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9f0 tx=0x7f242400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f242a7fc640 1 -- 172.21.15.94:0/4219765672 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400cd80 con 0x7f2434103c20 2024-04-25T14:02:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f242a7fc640 1 -- 172.21.15.94:0/4219765672 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24240108e0 con 0x7f2434103c20 2024-04-25T14:02:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24341a23e0 con 0x7f2434103c20 2024-04-25T14:02:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f242a7fc640 1 -- 172.21.15.94:0/4219765672 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424018980 con 0x7f2434103c20 2024-04-25T14:02:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.618+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24341a2930 con 0x7f2434103c20 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.621+0000 7f242a7fc640 1 -- 172.21.15.94:0/4219765672 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2424010430 con 0x7f2434103c20 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 Processor -- start 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 -- start start 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 0x55e566536d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e56649ad80 con 0x55e5661736b0 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e566454a20 con 0x55e566194390 2024-04-25T14:02:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f244261b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e566496d40 con 0x55e56616b4d0 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2438c2d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2438c2d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40280/0 (socket says 172.21.15.94:40280) 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2438c2d640 1 -- 172.21.15.94:0/3865545490 learned_addr learned my addr 172.21.15.94:0/3865545490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55236/0 (socket says 172.21.15.94:55236) 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f243942e640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 0x55e566536d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f242a7fc640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 0x7f24200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 -- 172.21.15.94:0/3865545490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 msgr2=0x55e566536d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 0x55e566536d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 -- 172.21.15.94:0/3865545490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 msgr2=0x55e566537a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2439c2f640 1 -- 172.21.15.94:0/3865545490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56645fc30 con 0x55e566194390 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f243942e640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 0x55e566536d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.622+0000 7f2438c2d640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f242a7fc640 1 -- 172.21.15.94:0/4219765672 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f24240deef0 con 0x7f2434103c20 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f2439c2f640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 0x7f24200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2430005ab0 tx=0x7f2430031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238392a69814811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f241f7fe640 1 -- 172.21.15.94:0/3865545490 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243004f070 con 0x55e566194390 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f241f7fe640 1 -- 172.21.15.94:0/3865545490 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430004240 con 0x55e566194390 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f241f7fe640 1 -- 172.21.15.94:0/3865545490 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430035040 con 0x55e566194390 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.623+0000 7f2439c2f640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 0x7f24200b2000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2430006010 tx=0x7f2430049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- 172.21.15.94:0/3865545490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 msgr2=0x55e566536810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2430005ab0 tx=0x7f2430031b10 comp rx=0 tx=0).stop 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- 172.21.15.94:0/3865545490 shutdown_connections 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661736b0 0x55e566537a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e56616b4d0 0x55e566536d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 --2- 172.21.15.94:0/3865545490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566536810 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- 172.21.15.94:0/3865545490 >> 172.21.15.94:0/3865545490 conn(0x55e5661884e0 msgr2=0x55e566532020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- 172.21.15.94:0/3865545490 shutdown_connections 2024-04-25T14:02:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- 172.21.15.94:0/3865545490 wait complete. 2024-04-25T14:02:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 -- start start 2024-04-25T14:02:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.624+0000 7f244261b740 1 Processor -- start 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566540b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f243942e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f243942e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40288/0 (socket says 172.21.15.94:40288) 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f243942e640 1 -- 172.21.15.94:0/732984361 learned_addr learned my addr 172.21.15.94:0/732984361 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566547a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e566538530 con 0x55e56616b4d0 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e566421dc0 con 0x55e566194390 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e5665386b0 con 0x55e5661736b0 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f244261b740 10 client.0 _ll_register_callbacks cb 0x55e56643d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f2438c2d640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566540b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f2439c2f640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566547a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.625+0000 7f243942e640 1 -- 172.21.15.94:0/732984361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 msgr2=0x55e566540b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f243942e640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566540b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f243942e640 1 -- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 msgr2=0x55e566547a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f243942e640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566547a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f243942e640 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5661ac3f0 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f243942e640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f242c005fd0 tx=0x7f242c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f6df384398821e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f241cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c003970 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f242c003b10 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2430004240 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f242c003d10 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f244261b740 10 client.22731 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.626+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e5662767d0 con 0x55e56616b4d0 2024-04-25T14:02:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430004240 con 0x55e56616b4d0 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f242c0106d0 con 0x55e56616b4d0 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 10 client.22731 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 10 client.22731 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 20 client.22731 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e566152f60 0x55e56654e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 4 client.22731 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5664a8990 con 0x55e566152f60 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.627+0000 7f2438c2d640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e566152f60 0x55e56654e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.628+0000 7f241cff9640 10 client.22731 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.628+0000 7f2438c2d640 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e566152f60 0x55e56654e240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b52314a74fcc34e server_cookie=38b296a846b883aa in_seq=0 out_seq=0 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f241cff9640 1 -- 172.21.15.94:0/732984361 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5664a8990 con 0x55e566152f60 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f241cff9640 10 client.22731 handle_command_reply: tid=0 2024-04-25T14:02:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 client.22731 shutdown 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 msgr2=0x55e566540600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f242c005fd0 tx=0x7f242c012820 comp rx=0 tx=0).stop 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 shutdown_connections 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e566152f60 0x55e56654e240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f241cff9640 0 client.22731 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566547a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566540b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 --2- 172.21.15.94:0/732984361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e56616b4d0 0x55e566540600 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.629+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 >> 172.21.15.94:0/732984361 conn(0x55e5661884e0 msgr2=0x55e566534f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.630+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 shutdown_connections 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.630+0000 7f244261b740 1 -- 172.21.15.94:0/732984361 wait complete. 2024-04-25T14:02:02.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.630+0000 7f244261b740 20 client.22731 trim_cache size 0 max 16384 2024-04-25T14:02:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 Processor -- start 2024-04-25T14:02:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 -- start start 2024-04-25T14:02:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566532160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e56653ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ab5e0 con 0x55e5661283a0 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ad120 con 0x55e566194390 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f244261b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24341abcb0 con 0x55e5661736b0 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40292/0 (socket says 172.21.15.94:40292) 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 -- 172.21.15.94:0/3653454514 learned_addr learned my addr 172.21.15.94:0/3653454514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2438c2d640 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e56653ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f243942e640 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566532160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 -- 172.21.15.94:0/3653454514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 msgr2=0x55e566532160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566532160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 -- 172.21.15.94:0/3653454514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 msgr2=0x55e56653ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e56653ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.647+0000 7f2439c2f640 1 -- 172.21.15.94:0/3653454514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56653bd60 con 0x55e5661283a0 2024-04-25T14:02:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.648+0000 7f2439c2f640 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f24300314a0 tx=0x7f243002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1645206fd0ffe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.648+0000 7f241effd640 1 -- 172.21.15.94:0/3653454514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430035070 con 0x55e5661283a0 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.648+0000 7f241effd640 1 -- 172.21.15.94:0/3653454514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430057020 con 0x55e5661283a0 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.648+0000 7f241effd640 1 -- 172.21.15.94:0/3653454514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243004f040 con 0x55e5661283a0 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- 172.21.15.94:0/3653454514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 msgr2=0x55e56653f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f24300314a0 tx=0x7f243002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- 172.21.15.94:0/3653454514 shutdown_connections 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e56653f260 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e56653ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 --2- 172.21.15.94:0/3653454514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566532160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- 172.21.15.94:0/3653454514 >> 172.21.15.94:0/3653454514 conn(0x55e5661884e0 msgr2=0x55e566531a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- 172.21.15.94:0/3653454514 shutdown_connections 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- 172.21.15.94:0/3653454514 wait complete. 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.649+0000 7f244261b740 1 -- start start 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 Processor -- start 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566544820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566545750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24341aca50 con 0x55e5661283a0 2024-04-25T14:02:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24341ac380 con 0x55e566194390 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24341a95c0 con 0x55e5661736b0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f244261b740 10 client.0 _ll_register_callbacks cb 0x7f242c00f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f243942e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f243942e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40304/0 (socket says 172.21.15.94:40304) 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f2438c2d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566544820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f243942e640 1 -- 172.21.15.94:0/185077399 learned_addr learned my addr 172.21.15.94:0/185077399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.650+0000 7f2439c2f640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566545750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 -- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 msgr2=0x55e566545750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566545750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 -- 172.21.15.94:0/185077399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 msgr2=0x55e5665442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 -- 172.21.15.94:0/185077399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e566545d10 con 0x55e5661736b0 2024-04-25T14:02:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f243942e640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f2438c2d640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566544820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f24240a4650 tx=0x7f2424018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a362b658bde3ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424010720 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2430057020 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f24240afd00 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f24240a6ef0 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 10 client.31792 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e566547060 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2430057020 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f24240a1b70 con 0x55e5661736b0 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 10 client.31792 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 10 client.31792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:02:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 20 client.31792 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.651+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5665b6b00 0x55e5665b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f244261b740 4 client.31792 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5664a2800 con 0x55e5665b6b00 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f243942e640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5665b6b00 0x55e5665b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f241cff9640 10 client.31792 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f243942e640 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5665b6b00 0x55e5665b8f20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c6554ce0fb29b29 server_cookie=fa9d6488fbf4fdce in_seq=0 out_seq=0 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f241cff9640 1 -- 172.21.15.94:0/185077399 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5664a2800 con 0x55e5665b6b00 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f241cff9640 10 client.31792 handle_command_reply: tid=0 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.652+0000 7f244261b740 1 client.31792 shutdown 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 msgr2=0x55e566544820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566544820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f24240a4650 tx=0x7f2424018420 comp rx=0 tx=0).stop 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 shutdown_connections 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55e5665b6b00 0x55e5665b8f20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e566194390 0x55e566545750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5661736b0 0x55e566544820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 --2- 172.21.15.94:0/185077399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5661283a0 0x55e5665442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f241cff9640 0 client.31792 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 >> 172.21.15.94:0/185077399 conn(0x55e5661884e0 msgr2=0x55e5665401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 shutdown_connections 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 1 -- 172.21.15.94:0/185077399 wait complete. 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f244261b740 20 client.31792 trim_cache size 0 max 16384 2024-04-25T14:02:02.658 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 msgr2=0x7f24200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 0x7f24200b2000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2430006010 tx=0x7f2430049000 comp rx=0 tx=0).stop 2024-04-25T14:02:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 msgr2=0x7f243419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9f0 tx=0x7f242400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 shutdown_connections 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f24200afaf0 0x7f24200b2000 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2434108920 0x7f24341a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2434103c20 0x7f243419aaf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 --2- 172.21.15.94:0/4219765672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24341029e0 0x7f243419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 >> 172.21.15.94:0/4219765672 conn(0x7f24340fdbd0 msgr2=0x7f2434069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 shutdown_connections 2024-04-25T14:02:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.653+0000 7f243b692640 1 -- 172.21.15.94:0/4219765672 wait complete. 2024-04-25T14:02:02.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:02:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 -- 172.21.15.94:0/3496242888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa034108920 msgr2=0x7fa03410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 --2- 172.21.15.94:0/3496242888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa034108920 0x7fa03410ad40 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009ca0 tx=0x7fa02c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 -- 172.21.15.94:0/3496242888 shutdown_connections 2024-04-25T14:02:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 --2- 172.21.15.94:0/3496242888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa034108920 0x7fa03410ad40 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 --2- 172.21.15.94:0/3496242888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034103c20 0x7fa034104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 --2- 172.21.15.94:0/3496242888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0341029e0 0x7fa0341036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 -- 172.21.15.94:0/3496242888 >> 172.21.15.94:0/3496242888 conn(0x7fa0340fdbd0 msgr2=0x7fa034100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 -- 172.21.15.94:0/3496242888 shutdown_connections 2024-04-25T14:02:02.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.788+0000 7fa03adf3640 1 -- 172.21.15.94:0/3496242888 wait complete. 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 Processor -- start 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 -- start start 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa034103c20 0x7fa0340fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034108920 0x7fa0340fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410e540 con 0x7fa0341029e0 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410e3c0 con 0x7fa034103c20 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa03adf3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410e6c0 con 0x7fa034108920 2024-04-25T14:02:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa038b8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa033fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa034103c20 0x7fa0340fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa038b8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40314/0 (socket says 172.21.15.94:40314) 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.789+0000 7fa038b8f640 1 -- 172.21.15.94:0/1272421659 learned_addr learned my addr 172.21.15.94:0/1272421659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa039390640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034108920 0x7fa0340fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 -- 172.21.15.94:0/1272421659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034108920 msgr2=0x7fa0340fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034108920 0x7fa0340fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 -- 172.21.15.94:0/1272421659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa034103c20 msgr2=0x7fa0340fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa034103c20 0x7fa0340fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 -- 172.21.15.94:0/1272421659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340fed70 con 0x7fa0341029e0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa038b8f640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fa02800e9f0 tx=0x7fa02800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa031ffb640 1 -- 172.21.15.94:0/1272421659 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800ccc0 con 0x7fa0341029e0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0341a4590 con 0x7fa0341029e0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa031ffb640 1 -- 172.21.15.94:0/1272421659 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0280035f0 con 0x7fa0341029e0 2024-04-25T14:02:02.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa031ffb640 1 -- 172.21.15.94:0/1272421659 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800ce60 con 0x7fa0341029e0 2024-04-25T14:02:02.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.790+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0341a4a80 con 0x7fa0341029e0 2024-04-25T14:02:02.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa031ffb640 1 -- 172.21.15.94:0/1272421659 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa028010690 con 0x7fa0341029e0 2024-04-25T14:02:02.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 Processor -- start 2024-04-25T14:02:02.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 -- start start 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 0x55ef60bc2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef60b26d80 con 0x55ef607ff6b0 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef60ae0a20 con 0x55ef60820390 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa041d81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef60b22d40 con 0x55ef607f74d0 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa033fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa033fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40328/0 (socket says 172.21.15.94:40328) 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa033fff640 1 -- 172.21.15.94:0/3626217084 learned_addr learned my addr 172.21.15.94:0/3626217084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa039390640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 0x55ef60bc2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.793+0000 7fa031ffb640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 0x7fa0240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 -- 172.21.15.94:0/3626217084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 msgr2=0x55ef60bc2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bc2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 -- 172.21.15.94:0/3626217084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 msgr2=0x55ef60bc2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 0x55ef60bc2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 -- 172.21.15.94:0/3626217084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef60aebc30 con 0x55ef607ff6b0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bc2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa039390640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 0x55ef60bc2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa039390640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 0x7fa0240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa031ffb640 1 -- 172.21.15.94:0/1272421659 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0280defc0 con 0x7fa0341029e0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa033fff640 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fa020009c90 tx=0x7fa020011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be78c246da68482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa01effd640 1 -- 172.21.15.94:0/3626217084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020005220 con 0x55ef607ff6b0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa01effd640 1 -- 172.21.15.94:0/3626217084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa020004c50 con 0x55ef607ff6b0 2024-04-25T14:02:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.794+0000 7fa01effd640 1 -- 172.21.15.94:0/3626217084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200197a0 con 0x55ef607ff6b0 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- 172.21.15.94:0/3626217084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 msgr2=0x55ef60bc3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fa020009c90 tx=0x7fa020011810 comp rx=0 tx=0).stop 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa039390640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 0x7fa0240b2000 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ef60bc3860 tx=0x7fa02c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- 172.21.15.94:0/3626217084 shutdown_connections 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607ff6b0 0x55ef60bc3a20 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bc2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 --2- 172.21.15.94:0/3626217084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef60820390 0x55ef60bc2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- 172.21.15.94:0/3626217084 >> 172.21.15.94:0/3626217084 conn(0x55ef608144e0 msgr2=0x55ef60bbe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- 172.21.15.94:0/3626217084 shutdown_connections 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- 172.21.15.94:0/3626217084 wait complete. 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 -- start start 2024-04-25T14:02:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.795+0000 7fa041d81740 1 Processor -- start 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bcc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60bd39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ef60bc4530 con 0x55ef60820390 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef60aaddc0 con 0x55ef607ff6b0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef60bc46b0 con 0x55ef607f74d0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa041d81740 10 client.0 _ll_register_callbacks cb 0x55ef60ac9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa039390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60bd39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55300/0 (socket says 172.21.15.94:55300) 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 -- 172.21.15.94:0/3278089462 learned_addr learned my addr 172.21.15.94:0/3278089462 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bcc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 -- 172.21.15.94:0/3278089462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 msgr2=0x55ef60bcc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bcc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 -- 172.21.15.94:0/3278089462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 msgr2=0x55ef60bd39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60bd39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 -- 172.21.15.94:0/3278089462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef608383f0 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa033fff640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa02001b040 tx=0x7fa020005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3fe539dcc8a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.796+0000 7fa013fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020029070 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0200198b0 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa020004c50 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa020035050 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 10 client.32159 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ef609027d0 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa020004c50 con 0x55ef607ff6b0 2024-04-25T14:02:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa0200235f0 con 0x55ef607ff6b0 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 10 client.32159 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 10 client.32159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 20 client.32159 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607def60 0x55ef60bd9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 4 client.32159 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.797+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef60b34990 con 0x55ef607def60 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.798+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607def60 0x55ef60bd9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.798+0000 7fa013fff640 10 client.32159 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:02.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.798+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607def60 0x55ef60bd9f60 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35b0f78d140ddad server_cookie=20b48400d3bb0e7d in_seq=0 out_seq=0 2024-04-25T14:02:02.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.800+0000 7fa013fff640 1 -- 172.21.15.94:0/3278089462 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef60b34990 con 0x55ef607def60 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.800+0000 7fa013fff640 10 client.32159 handle_command_reply: tid=0 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.800+0000 7fa041d81740 1 client.32159 shutdown 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 msgr2=0x55ef60bcca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa02001b040 tx=0x7fa020005230 comp rx=0 tx=0).stop 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 shutdown_connections 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607def60 0x55ef60bd9f60 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60bd39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bcca50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 --2- 172.21.15.94:0/3278089462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bcc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa013fff640 0 client.32159 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 >> 172.21.15.94:0/3278089462 conn(0x55ef608144e0 msgr2=0x55ef60bc0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 shutdown_connections 2024-04-25T14:02:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 1 -- 172.21.15.94:0/3278089462 wait complete. 2024-04-25T14:02:02.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.801+0000 7fa041d81740 20 client.32159 trim_cache size 0 max 16384 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 Processor -- start 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 -- start start 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef60820390 0x55ef60bde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341ad7a0 con 0x55ef607f74d0 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341af2e0 con 0x55ef607ff6b0 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa041d81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341ade70 con 0x55ef60820390 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa033fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa039390640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef60820390 0x55ef60bde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40336/0 (socket says 172.21.15.94:40336) 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 -- 172.21.15.94:0/3153285190 learned_addr learned my addr 172.21.15.94:0/3153285190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 -- 172.21.15.94:0/3153285190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef60820390 msgr2=0x55ef60bde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef60820390 0x55ef60bde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 -- 172.21.15.94:0/3153285190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 msgr2=0x55ef60bd38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 -- 172.21.15.94:0/3153285190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef60bd3e10 con 0x55ef607f74d0 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa033fff640 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.817+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fa0280a7390 tx=0x7fa0280e2aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f2ce126fa56a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa01e7fc640 1 -- 172.21.15.94:0/3153285190 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0280a71d0 con 0x55ef607f74d0 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa01e7fc640 1 -- 172.21.15.94:0/3153285190 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0280a6f10 con 0x55ef607f74d0 2024-04-25T14:02:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa01e7fc640 1 -- 172.21.15.94:0/3153285190 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0280a6b70 con 0x55ef607f74d0 2024-04-25T14:02:02.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa041d81740 1 -- 172.21.15.94:0/3153285190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 msgr2=0x55ef60bcb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa041d81740 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fa0280a7390 tx=0x7fa0280e2aa0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa041d81740 1 -- 172.21.15.94:0/3153285190 shutdown_connections 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.818+0000 7fa041d81740 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef60820390 0x55ef60bde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 --2- 172.21.15.94:0/3153285190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef607f74d0 0x55ef60bcb430 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- 172.21.15.94:0/3153285190 >> 172.21.15.94:0/3153285190 conn(0x55ef608144e0 msgr2=0x55ef60bcad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- 172.21.15.94:0/3153285190 shutdown_connections 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- 172.21.15.94:0/3153285190 wait complete. 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- start start 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 Processor -- start 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bd28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341aec10 con 0x55ef60820390 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341ae540 con 0x55ef607ff6b0 2024-04-25T14:02:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341ab780 con 0x55ef607f74d0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.819+0000 7fa041d81740 10 client.0 _ll_register_callbacks cb 0x7fa020005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa039390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bd28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa039390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40338/0 (socket says 172.21.15.94:40338) 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa039390640 1 -- 172.21.15.94:0/3545748960 learned_addr learned my addr 172.21.15.94:0/3545748960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa033fff640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 -- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 msgr2=0x55ef60bd2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 -- 172.21.15.94:0/3545748960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 msgr2=0x55ef60c402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 -- 172.21.15.94:0/3545748960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef60c40800 con 0x55ef607f74d0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa039390640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa038b8f640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bd28b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa0280ab970 tx=0x7fa0280e26c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a8ab1b3250f0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa013fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0280a6d50 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0280a6f10 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa0280a3690 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa0280a7400 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa041d81740 10 client.31810 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.820+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ef60c419a0 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0280a6f10 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa0280afdd0 con 0x55ef607f74d0 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 10 client.31810 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 10 client.31810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 20 client.31810 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607b43a0 0x55ef60c44300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 4 client.31810 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef60b2e800 con 0x55ef607b43a0 2024-04-25T14:02:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa033fff640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607b43a0 0x55ef60c44300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.821+0000 7fa013fff640 10 client.31810 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:02.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.822+0000 7fa033fff640 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607b43a0 0x55ef60c44300 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e57a28f1d84ccc2 server_cookie=877db880ffc3aed0 in_seq=0 out_seq=0 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.826+0000 7fa013fff640 1 -- 172.21.15.94:0/3545748960 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef60b2e800 con 0x55ef607b43a0 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.826+0000 7fa013fff640 10 client.31810 handle_command_reply: tid=0 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.826+0000 7fa041d81740 1 client.31810 shutdown 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 msgr2=0x55ef60bd28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bd28b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa0280ab970 tx=0x7fa0280e26c0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 shutdown_connections 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55ef607b43a0 0x55ef60c44300 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef60820390 0x55ef60c402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef607ff6b0 0x55ef60bd2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 --2- 172.21.15.94:0/3545748960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef607f74d0 0x55ef60bd28b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa013fff640 0 client.31810 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 >> 172.21.15.94:0/3545748960 conn(0x55ef608144e0 msgr2=0x55ef60bc0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 shutdown_connections 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 1 -- 172.21.15.94:0/3545748960 wait complete. 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa041d81740 20 client.31810 trim_cache size 0 max 16384 2024-04-25T14:02:02.828 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 msgr2=0x7fa0240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.827+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 0x7fa0240b2000 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ef60bc3860 tx=0x7fa02c0095c0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 msgr2=0x7fa0340ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fa02800e9f0 tx=0x7fa02800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 shutdown_connections 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa0240afaf0 0x7fa0240b2000 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa034108920 0x7fa0340fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa034103c20 0x7fa0340fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 --2- 172.21.15.94:0/1272421659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0341029e0 0x7fa0340ffa60 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 >> 172.21.15.94:0/1272421659 conn(0x7fa0340fdbd0 msgr2=0x7fa034109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 shutdown_connections 2024-04-25T14:02:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:02.828+0000 7fa03adf3640 1 -- 172.21.15.94:0/1272421659 wait complete. 2024-04-25T14:02:03.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 145686 19.12 MB/sec execute 285 sec latency 57.493 ms 2024-04-25T14:02:03.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:02:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:02:03.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:02:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 -- 172.21.15.94:0/4133196444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808108920 msgr2=0x7f180810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 --2- 172.21.15.94:0/4133196444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808108920 0x7f180810ad40 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f180002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 -- 172.21.15.94:0/4133196444 shutdown_connections 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 --2- 172.21.15.94:0/4133196444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808108920 0x7f180810ad40 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 --2- 172.21.15.94:0/4133196444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808103c20 0x7f1808104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 --2- 172.21.15.94:0/4133196444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f18081036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 -- 172.21.15.94:0/4133196444 >> 172.21.15.94:0/4133196444 conn(0x7f18080fdbd0 msgr2=0x7f1808100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 -- 172.21.15.94:0/4133196444 shutdown_connections 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 -- 172.21.15.94:0/4133196444 wait complete. 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.961+0000 7f180d5f3640 1 Processor -- start 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 -- start start 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f180819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 0x7f180819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808108920 0x7f18081a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f180810f610 con 0x7f1808103c20 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f180810f490 con 0x7f18081029e0 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f180d5f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f180810f790 con 0x7f1808108920 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 0x7f180819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f180819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f180819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38754/0 (socket says 172.21.15.94:38754) 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f1806ffd640 1 -- 172.21.15.94:0/3215147335 learned_addr learned my addr 172.21.15.94:0/3215147335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18077fe640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808108920 0x7f18081a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 -- 172.21.15.94:0/3215147335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808108920 msgr2=0x7f18081a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808108920 0x7f18081a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 -- 172.21.15.94:0/3215147335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 msgr2=0x7f180819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f180819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 -- 172.21.15.94:0/3215147335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081a2040 con 0x7f1808103c20 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f18067fc640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 0x7f180819aa40 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.962+0000 7f17fffff640 1 -- 172.21.15.94:0/3215147335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000ccc0 con 0x7f1808103c20 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.963+0000 7f17fffff640 1 -- 172.21.15.94:0/3215147335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f0005020 con 0x7f1808103c20 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.963+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081a2330 con 0x7f1808103c20 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.963+0000 7f17fffff640 1 -- 172.21.15.94:0/3215147335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0004a00 con 0x7f1808103c20 2024-04-25T14:02:03.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.963+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081a2830 con 0x7f1808103c20 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.965+0000 7f181457e740 1 Processor -- start 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.965+0000 7f181457e740 1 -- start start 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.965+0000 7f17fffff640 1 -- 172.21.15.94:0/3215147335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f17f0004ba0 con 0x7f1808103c20 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 0x55879d10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55879d072b00 con 0x55879cd4b730 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55879d00e920 con 0x55879cd6c410 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f181457e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55879d06eac0 con 0x55879cd43550 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 0x55879d10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50712/0 (socket says 172.21.15.94:50712) 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47194/0 (socket says 172.21.15.94:47194) 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f1806ffd640 1 -- 172.21.15.94:0/1130584064 learned_addr learned my addr 172.21.15.94:0/1130584064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 -- 172.21.15.94:0/1130584064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 msgr2=0x55879d10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 -- 172.21.15.94:0/1130584064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 msgr2=0x55879d10d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 0x55879d10d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 -- 172.21.15.94:0/1130584064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879ce4c7e0 con 0x55879cd4b730 2024-04-25T14:02:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f17fffff640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 0x7f17f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f1806ffd640 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18077fe640 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 0x55879d10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18077fe640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 0x7f17f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.966+0000 7f18067fc640 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f17f00dc620 tx=0x7f17f00a3680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c236acc020bc38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.967+0000 7f17fffff640 1 -- 172.21.15.94:0/3215147335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f17f00debf0 con 0x7f1808103c20 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.967+0000 7f17fd7fa640 1 -- 172.21.15.94:0/1130584064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001c020 con 0x55879cd4b730 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.967+0000 7f17fd7fa640 1 -- 172.21.15.94:0/1130584064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f00e2e50 con 0x55879cd4b730 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.967+0000 7f17fd7fa640 1 -- 172.21.15.94:0/1130584064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00a6980 con 0x55879cd4b730 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- 172.21.15.94:0/1130584064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 msgr2=0x55879d114920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f18077fe640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 0x7f17f40b2000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1800031440 tx=0x7f18000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f17f00dc620 tx=0x7f17f00a3680 comp rx=0 tx=0).stop 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- 172.21.15.94:0/1130584064 shutdown_connections 2024-04-25T14:02:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd4b730 0x55879d114920 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd43550 0x55879d10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 --2- 172.21.15.94:0/1130584064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd6c410 0x55879d10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- 172.21.15.94:0/1130584064 >> 172.21.15.94:0/1130584064 conn(0x55879cd60560 msgr2=0x55879d108e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- 172.21.15.94:0/1130584064 shutdown_connections 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- 172.21.15.94:0/1130584064 wait complete. 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.968+0000 7f181457e740 1 -- start start 2024-04-25T14:02:03.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 Processor -- start 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d117800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55879d115370 con 0x55879cd43550 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55879d00c5d0 con 0x55879cd4b730 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55879d1154f0 con 0x55879cd6c410 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47200/0 (socket says 172.21.15.94:47200) 2024-04-25T14:02:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 -- 172.21.15.94:0/3083957079 learned_addr learned my addr 172.21.15.94:0/3083957079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f181457e740 10 client.0 _ll_register_callbacks cb 0x55879cffa080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f18077fe640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f18067fc640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 -- 172.21.15.94:0/3083957079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 msgr2=0x55879d11e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 -- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 msgr2=0x55879d117800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d117800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.969+0000 7f1806ffd640 1 -- 172.21.15.94:0/3083957079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879ceacf70 con 0x55879cd43550 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f1806ffd640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f17f8013d00 tx=0x7f17f80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2274b23b2cf05cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f17ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8003b70 con 0x55879cd43550 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f17f8003d10 con 0x55879cd43550 2024-04-25T14:02:03.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f00e2e50 con 0x55879cd43550 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17f8004020 con 0x55879cd43550 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f181457e740 10 client.22767 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55879cd44170 con 0x55879cd43550 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.970+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f00e2e50 con 0x55879cd43550 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f17f801a070 con 0x55879cd43550 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 10 client.22767 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 10 client.22767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 20 client.22767 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd2b1f0 0x55879d124d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 4 client.22767 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55879d080710 con 0x55879cd2b1f0 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f18067fc640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd2b1f0 0x55879d124d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f17ee7fc640 10 client.22767 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:03.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.971+0000 7f18067fc640 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd2b1f0 0x55879d124d30 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69aaa355dce9ed7e server_cookie=8ea2a56475a89a67 in_seq=0 out_seq=0 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f17ee7fc640 1 -- 172.21.15.94:0/3083957079 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55879d080710 con 0x55879cd2b1f0 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f17ee7fc640 10 client.22767 handle_command_reply: tid=0 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 client.22767 shutdown 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 msgr2=0x55879d1172c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f17f8013d00 tx=0x7f17f80117b0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 shutdown_connections 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd2b1f0 0x55879d124d30 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f17ee7fc640 0 client.22767 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 --2- 172.21.15.94:0/3083957079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d1172c0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 >> 172.21.15.94:0/3083957079 conn(0x55879cd60560 msgr2=0x55879d107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 shutdown_connections 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 1 -- 172.21.15.94:0/3083957079 wait complete. 2024-04-25T14:02:03.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.978+0000 7f181457e740 20 client.22767 trim_cache size 0 max 16384 2024-04-25T14:02:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.997+0000 7f181457e740 1 Processor -- start 2024-04-25T14:02:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 -- start start 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd43550 0x55879d10c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd4b730 0x55879d10c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18081ab590 con 0x55879cd6c410 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18081ad0d0 con 0x55879cd43550 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f181457e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18081abc60 con 0x55879cd4b730 2024-04-25T14:02:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47204/0 (socket says 172.21.15.94:47204) 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 -- 172.21.15.94:0/2459865130 learned_addr learned my addr 172.21.15.94:0/2459865130 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f1806ffd640 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd43550 0x55879d10c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18067fc640 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd4b730 0x55879d10c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 -- 172.21.15.94:0/2459865130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd4b730 msgr2=0x55879d10c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd4b730 0x55879d10c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 -- 172.21.15.94:0/2459865130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd43550 msgr2=0x55879d10c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.998+0000 7f18077fe640 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd43550 0x55879d10c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.999+0000 7f18077fe640 1 -- 172.21.15.94:0/2459865130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879d114af0 con 0x55879cd6c410 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.999+0000 7f18077fe640 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f1800040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57299bf7fcd85db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.999+0000 7f17fcff9640 1 -- 172.21.15.94:0/2459865130 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800035040 con 0x55879cd6c410 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.999+0000 7f17fcff9640 1 -- 172.21.15.94:0/2459865130 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18000035f0 con 0x55879cd6c410 2024-04-25T14:02:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:03.999+0000 7f17fcff9640 1 -- 172.21.15.94:0/2459865130 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180003d6b0 con 0x55879cd6c410 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- 172.21.15.94:0/2459865130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 msgr2=0x55879d11e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f1800040000 comp rx=0 tx=0).stop 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- 172.21.15.94:0/2459865130 shutdown_connections 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd6c410 0x55879d11e680 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd4b730 0x55879d10c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 --2- 172.21.15.94:0/2459865130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd43550 0x55879d10c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- 172.21.15.94:0/2459865130 >> 172.21.15.94:0/2459865130 conn(0x55879cd60560 msgr2=0x55879d10bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- 172.21.15.94:0/2459865130 shutdown_connections 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- 172.21.15.94:0/2459865130 wait complete. 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 -- start start 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.000+0000 7f181457e740 1 Processor -- start 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d11c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18081aca00 con 0x55879cd43550 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18081ac330 con 0x55879cd4b730 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18081a9570 con 0x55879cd6c410 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f181457e740 10 client.0 _ll_register_callbacks cb 0x7f17f8013940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47208/0 (socket says 172.21.15.94:47208) 2024-04-25T14:02:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 -- 172.21.15.94:0/1189542889 learned_addr learned my addr 172.21.15.94:0/1189542889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f18077fe640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f18067fc640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d11c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 -- 172.21.15.94:0/1189542889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 msgr2=0x55879d11d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 -- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 msgr2=0x55879d11c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d11c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 -- 172.21.15.94:0/1189542889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879d11dcc0 con 0x55879cd43550 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f1806ffd640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f17f801b850 tx=0x7f17f80138b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997c34f86d04e250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.001+0000 7f17ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8028070 con 0x55879cd43550 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f17f8004a00 con 0x55879cd43550 2024-04-25T14:02:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18000035f0 con 0x55879cd43550 2024-04-25T14:02:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17f8004c00 con 0x55879cd43550 2024-04-25T14:02:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 10 client.22773 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55879d11eff0 con 0x55879cd43550 2024-04-25T14:02:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18000035f0 con 0x55879cd43550 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f17f801a070 con 0x55879cd43550 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 10 client.22773 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 10 client.22773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 20 client.22773 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd00630 0x55879d193a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 4 client.22773 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55879d07a580 con 0x55879cd00630 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.002+0000 7f18067fc640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd00630 0x55879d193a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.003+0000 7f17ee7fc640 10 client.22773 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.003+0000 7f18067fc640 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd00630 0x55879d193a50 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59ef675581425f4 server_cookie=7bb257000416bf5 in_seq=0 out_seq=0 2024-04-25T14:02:04.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.006+0000 7f17ee7fc640 1 -- 172.21.15.94:0/1189542889 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55879d07a580 con 0x55879cd00630 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.006+0000 7f17ee7fc640 10 client.22773 handle_command_reply: tid=0 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.006+0000 7f181457e740 1 client.22773 shutdown 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.006+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 msgr2=0x55879d11c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.006+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f17f801b850 tx=0x7f17f80138b0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 shutdown_connections 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55879cd00630 0x55879d193a50 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55879cd6c410 0x55879d11d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55879cd4b730 0x55879d11c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 --2- 172.21.15.94:0/1189542889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55879cd43550 0x55879d11c290 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f17ee7fc640 0 client.22773 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 >> 172.21.15.94:0/1189542889 conn(0x55879cd60560 msgr2=0x55879d10c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 shutdown_connections 2024-04-25T14:02:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 1 -- 172.21.15.94:0/1189542889 wait complete. 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f181457e740 20 client.22773 trim_cache size 0 max 16384 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "05d8fd15-9105-47e4-9466-16ed7e3d3b0f", 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:02:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 msgr2=0x7f17f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 0x7f17f40b2000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1800031440 tx=0x7f18000314a0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 msgr2=0x7f180819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 0x7f180819aa40 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 shutdown_connections 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f17f40afaf0 0x7f17f40b2000 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1808108920 0x7f18081a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1808103c20 0x7f180819aa40 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 --2- 172.21.15.94:0/3215147335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18081029e0 0x7f180819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 >> 172.21.15.94:0/3215147335 conn(0x7f18080fdbd0 msgr2=0x7f1808104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 shutdown_connections 2024-04-25T14:02:04.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.007+0000 7f180d5f3640 1 -- 172.21.15.94:0/3215147335 wait complete. 2024-04-25T14:02:04.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:04.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- 172.21.15.94:0/2471324998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808102a50 msgr2=0x7f8808103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 --2- 172.21.15.94:0/2471324998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808102a50 0x7f8808103750 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- 172.21.15.94:0/2471324998 shutdown_connections 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 --2- 172.21.15.94:0/2471324998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808108990 0x7f880810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 --2- 172.21.15.94:0/2471324998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 0x7f88081040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 --2- 172.21.15.94:0/2471324998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808102a50 0x7f8808103750 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- 172.21.15.94:0/2471324998 >> 172.21.15.94:0/2471324998 conn(0x7f88080fdc40 msgr2=0x7f88081000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- 172.21.15.94:0/2471324998 shutdown_connections 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- 172.21.15.94:0/2471324998 wait complete. 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 Processor -- start 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.134+0000 7f880db84640 1 -- start start 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808102a50 0x7f880819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 0x7f880819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f880810f600 con 0x7f8808108990 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f880810f480 con 0x7f8808103c90 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f880810f780 con 0x7f8808102a50 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47220/0 (socket says 172.21.15.94:47220) 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 -- 172.21.15.94:0/1827979321 learned_addr learned my addr 172.21.15.94:0/1827979321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8806ffd640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 0x7f880819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f88077fe640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808102a50 0x7f880819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 -- 172.21.15.94:0/1827979321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808102a50 msgr2=0x7f880819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808102a50 0x7f880819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 -- 172.21.15.94:0/1827979321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 msgr2=0x7f880819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 0x7f880819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 -- 172.21.15.94:0/1827979321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081a1d60 con 0x7f8808108990 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8807fff640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f880000eb10 tx=0x7f880000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8804ff9640 1 -- 172.21.15.94:0/1827979321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004b20 con 0x7f8808108990 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8804ff9640 1 -- 172.21.15.94:0/1827979321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8800004cc0 con 0x7f8808108990 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081a1ff0 con 0x7f8808108990 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.135+0000 7f8804ff9640 1 -- 172.21.15.94:0/1827979321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800010670 con 0x7f8808108990 2024-04-25T14:02:04.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.136+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a24b0 con 0x7f8808108990 2024-04-25T14:02:04.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 Processor -- start 2024-04-25T14:02:04.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8804ff9640 1 -- 172.21.15.94:0/1827979321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8800005020 con 0x7f8808108990 2024-04-25T14:02:04.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 -- start start 2024-04-25T14:02:04.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55df28146d80 con 0x55df27e1f6b0 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55df281008b0 con 0x55df27e40390 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8814a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df28142d40 con 0x55df27e174d0 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f88077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f88077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50746/0 (socket says 172.21.15.94:50746) 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f88077fe640 1 -- 172.21.15.94:0/2089241705 learned_addr learned my addr 172.21.15.94:0/2089241705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.139+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f8804ff9640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 0x7f87f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 -- 172.21.15.94:0/2089241705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 msgr2=0x55df281e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 -- 172.21.15.94:0/2089241705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 msgr2=0x55df281e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 -- 172.21.15.94:0/2089241705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df280d1a40 con 0x55df27e174d0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f8807fff640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 0x7f87f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f8804ff9640 1 -- 172.21.15.94:0/1827979321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f88000df200 con 0x7f8808108990 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f88077fe640 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009230 tx=0x7f87f8009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e1263a23a4d6562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f87edffb640 1 -- 172.21.15.94:0/2089241705 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8035030 con 0x55df27e174d0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f87edffb640 1 -- 172.21.15.94:0/2089241705 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f8040070 con 0x55df27e174d0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.140+0000 7f87edffb640 1 -- 172.21.15.94:0/2089241705 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8003970 con 0x55df27e174d0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- 172.21.15.94:0/2089241705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 msgr2=0x55df281e1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8807fff640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 0x7f87f00b2000 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88000dcc60 tx=0x7f880009bf80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009230 tx=0x7f87f8009620 comp rx=0 tx=0).stop 2024-04-25T14:02:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- 172.21.15.94:0/2089241705 shutdown_connections 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281e1fe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2089241705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281e1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- 172.21.15.94:0/2089241705 >> 172.21.15.94:0/2089241705 conn(0x55df27e344e0 msgr2=0x55df281dd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- 172.21.15.94:0/2089241705 shutdown_connections 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- 172.21.15.94:0/2089241705 wait complete. 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.141+0000 7f8814a3b740 1 -- start start 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 Processor -- start 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55df281e3720 con 0x55df27e1f6b0 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55df280cdd10 con 0x55df27e40390 2024-04-25T14:02:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df281e38a0 con 0x55df27e174d0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8814a3b740 10 client.0 _ll_register_callbacks cb 0x55df280fa210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47242/0 (socket says 172.21.15.94:47242) 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 -- 172.21.15.94:0/2992745541 learned_addr learned my addr 172.21.15.94:0/2992745541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 -- 172.21.15.94:0/2992745541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 msgr2=0x55df281eb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281eb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 -- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 msgr2=0x55df281f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.142+0000 7f88077fe640 1 -- 172.21.15.94:0/2992745541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df27f227d0 con 0x55df27e1f6b0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f88077fe640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f87f8005ab0 tx=0x7f87f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1f22bec5c825af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8035070 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87f80317b0 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f8040070 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87f80036f0 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 10 client.22785 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55df27e583f0 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f8040070 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f87f80459a0 con 0x55df27e1f6b0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 10 client.22785 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 10 client.22785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 20 client.22785 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dfef60 0x55df281f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 4 client.22785 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.143+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df28154990 con 0x55df27dfef60 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.144+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dfef60 0x55df281f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.144+0000 7f87e6ffd640 10 client.22785 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.144+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dfef60 0x55df281f8e60 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a3e9e9c054f38 server_cookie=40d467076e233d4 in_seq=0 out_seq=0 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.146+0000 7f87e6ffd640 1 -- 172.21.15.94:0/2992745541 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df28154990 con 0x55df27dfef60 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.146+0000 7f87e6ffd640 10 client.22785 handle_command_reply: tid=0 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.146+0000 7f8814a3b740 1 client.22785 shutdown 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 msgr2=0x55df281ebe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f87f8005ab0 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 shutdown_connections 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dfef60 0x55df281f8e60 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281ebe40 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f87e6ffd640 0 client.22785 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2992745541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 >> 172.21.15.94:0/2992745541 conn(0x55df27e344e0 msgr2=0x55df281e0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 shutdown_connections 2024-04-25T14:02:04.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 1 -- 172.21.15.94:0/2992745541 wait complete. 2024-04-25T14:02:04.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.147+0000 7f8814a3b740 20 client.22785 trim_cache size 0 max 16384 2024-04-25T14:02:04.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 146138 19.18 MB/sec execute 286 sec latency 89.446 ms 2024-04-25T14:02:04.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 Processor -- start 2024-04-25T14:02:04.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 -- start start 2024-04-25T14:02:04.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e174d0 0x55df281e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e1f6b0 0x55df281ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ab550 con 0x55df27e40390 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ad090 con 0x55df27e174d0 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.166+0000 7f8814a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88081abc20 con 0x55df27e1f6b0 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47250/0 (socket says 172.21.15.94:47250) 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 -- 172.21.15.94:0/2526685264 learned_addr learned my addr 172.21.15.94:0/2526685264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e174d0 0x55df281e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f88077fe640 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e1f6b0 0x55df281ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 -- 172.21.15.94:0/2526685264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e1f6b0 msgr2=0x55df281ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e1f6b0 0x55df281ea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 -- 172.21.15.94:0/2526685264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e174d0 msgr2=0x55df281e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e174d0 0x55df281e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 -- 172.21.15.94:0/2526685264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df27e180f0 con 0x55df27e40390 2024-04-25T14:02:04.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f8806ffd640 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x55df281f3650 tx=0x7f87fc009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a796d74ecdd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f87ed7fa640 1 -- 172.21.15.94:0/2526685264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc01a040 con 0x55df27e40390 2024-04-25T14:02:04.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f87ed7fa640 1 -- 172.21.15.94:0/2526685264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87fc0047a0 con 0x55df27e40390 2024-04-25T14:02:04.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.167+0000 7f87ed7fa640 1 -- 172.21.15.94:0/2526685264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0226a0 con 0x55df27e40390 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- 172.21.15.94:0/2526685264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 msgr2=0x55df281dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x55df281f3650 tx=0x7f87fc009c60 comp rx=0 tx=0).stop 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- 172.21.15.94:0/2526685264 shutdown_connections 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e40390 0x55df281dcc10 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e1f6b0 0x55df281ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 --2- 172.21.15.94:0/2526685264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e174d0 0x55df281e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- 172.21.15.94:0/2526685264 >> 172.21.15.94:0/2526685264 conn(0x55df27e344e0 msgr2=0x55df281e6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- 172.21.15.94:0/2526685264 shutdown_connections 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- 172.21.15.94:0/2526685264 wait complete. 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.168+0000 7f8814a3b740 1 -- start start 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 Processor -- start 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281ef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ac9c0 con 0x55df27e1f6b0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ac2f0 con 0x55df27e40390 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a9530 con 0x55df27e174d0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8814a3b740 10 client.0 _ll_register_callbacks cb 0x7f87f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47262/0 (socket says 172.21.15.94:47262) 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 -- 172.21.15.94:0/3176025123 learned_addr learned my addr 172.21.15.94:0/3176025123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8806ffd640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f8806ffd640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 -- 172.21.15.94:0/3176025123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 msgr2=0x55df281ef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281ef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 -- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 msgr2=0x55df281f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:04.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.169+0000 7f88077fe640 1 -- 172.21.15.94:0/3176025123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df281074b0 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f88077fe640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f87f8045dc0 tx=0x7f87f8031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a0ab008bf6e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f805b070 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87f8003ca0 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc0047a0 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87f8066050 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 10 client.22791 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55df281f2350 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87fc0047a0 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f87f803c5d0 con 0x55df27e1f6b0 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 10 client.22791 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:04.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 10 client.22791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 20 client.22791 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dd43a0 0x55df28263420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 4 client.22791 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.170+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df2814e800 con 0x55df27dd43a0 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.171+0000 7f8806ffd640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dd43a0 0x55df28263420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.171+0000 7f87e6ffd640 10 client.22791 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.171+0000 7f8806ffd640 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dd43a0 0x55df28263420 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfb877c210aa25e server_cookie=186b671084c996f8 in_seq=0 out_seq=0 2024-04-25T14:02:04.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.202+0000 7f87e6ffd640 1 -- 172.21.15.94:0/3176025123 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55df2814e800 con 0x55df27dd43a0 2024-04-25T14:02:04.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.202+0000 7f87e6ffd640 10 client.22791 handle_command_reply: tid=0 2024-04-25T14:02:04.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.207+0000 7f8814a3b740 1 client.22791 shutdown 2024-04-25T14:02:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.221+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 msgr2=0x55df281efda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.221+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f87f8045dc0 tx=0x7f87f8031a60 comp rx=0 tx=0).stop 2024-04-25T14:02:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.225+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 shutdown_connections 2024-04-25T14:02:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.225+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55df27dd43a0 0x55df28263420 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:04 smithi094 ceph-mon[29195]: pgmap v959: 97 pgs: 97 active+clean; 259 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-04-25T14:02:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.226+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df27e40390 0x55df281f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.226+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df27e1f6b0 0x55df281efda0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.226+0000 7f8814a3b740 1 --2- 172.21.15.94:0/3176025123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df27e174d0 0x55df281ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.226+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 >> 172.21.15.94:0/3176025123 conn(0x55df27e344e0 msgr2=0x55df281e9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.226+0000 7f87e6ffd640 0 client.22791 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:04.227 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:04.227 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (94 inodes in the stack)", 2024-04-25T14:02:04.227 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:02:04.227 INFO:teuthology.orchestra.run.smithi094.stdout: "05d8fd15-9105-47e4-9466-16ed7e3d3b0f": { 2024-04-25T14:02:04.227 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "05d8fd15-9105-47e4-9466-16ed7e3d3b0f", 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.227+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 shutdown_connections 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.227+0000 7f8814a3b740 1 -- 172.21.15.94:0/3176025123 wait complete. 2024-04-25T14:02:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.228+0000 7f8814a3b740 20 client.22791 trim_cache size 0 max 16384 2024-04-25T14:02:04.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.234+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 msgr2=0x7f87f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.234+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 0x7f87f00b2000 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88000dcc60 tx=0x7f880009bf80 comp rx=0 tx=0).stop 2024-04-25T14:02:04.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.234+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 msgr2=0x7f88081a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:04.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.234+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f880000eb10 tx=0x7f880000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 shutdown_connections 2024-04-25T14:02:04.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87f00afaf0 0x7f87f00b2000 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8808108990 0x7f88081a16f0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808103c90 0x7f880819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 --2- 172.21.15.94:0/1827979321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8808102a50 0x7f880819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:04.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.240+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 >> 172.21.15.94:0/1827979321 conn(0x7f88080fdc40 msgr2=0x7f8808104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.246+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 shutdown_connections 2024-04-25T14:02:04.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:04.247+0000 7f880db84640 1 -- 172.21.15.94:0/1827979321 wait complete. 2024-04-25T14:02:04.269 INFO:tasks.cephfs.filesystem:scrub status for tag:05d8fd15-9105-47e4-9466-16ed7e3d3b0f - {'path': '/', 'tag': '05d8fd15-9105-47e4-9466-16ed7e3d3b0f', 'options': 'recursive,force'} 2024-04-25T14:02:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:04 smithi137 ceph-mon[32568]: pgmap v959: 97 pgs: 97 active+clean; 259 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-04-25T14:02:04.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:04 smithi138 ceph-mon[32977]: pgmap v959: 97 pgs: 97 active+clean; 259 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-04-25T14:02:05.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 146328 19.14 MB/sec execute 287 sec latency 292.450 ms 2024-04-25T14:02:05.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:05.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:05 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:02:05.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:05.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:05 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:05.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:05.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:05 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 146566 19.13 MB/sec execute 288 sec latency 74.176 ms 2024-04-25T14:02:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: pgmap v960: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 587 op/s 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:06.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:06 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:06.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: pgmap v960: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 587 op/s 2024-04-25T14:02:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:06 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:06.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: pgmap v960: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 587 op/s 2024-04-25T14:02:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:06.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:06 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:07.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 146792 19.10 MB/sec execute 289 sec latency 92.404 ms 2024-04-25T14:02:07.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:07 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:07.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:07 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:07.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:07 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:08.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 147033 19.05 MB/sec execute 290 sec latency 115.065 ms 2024-04-25T14:02:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:08 smithi137 ceph-mon[32568]: pgmap v961: 97 pgs: 97 active+clean; 271 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-25T14:02:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:08 smithi094 ceph-mon[29195]: pgmap v961: 97 pgs: 97 active+clean; 271 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-25T14:02:08.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:08 smithi138 ceph-mon[32977]: pgmap v961: 97 pgs: 97 active+clean; 271 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 527 op/s 2024-04-25T14:02:09.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 147394 19.03 MB/sec execute 291 sec latency 97.292 ms 2024-04-25T14:02:10.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 147632 19.01 MB/sec execute 292 sec latency 216.515 ms 2024-04-25T14:02:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:10 smithi137 ceph-mon[32568]: pgmap v962: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 516 op/s 2024-04-25T14:02:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:10 smithi094 ceph-mon[29195]: pgmap v962: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 516 op/s 2024-04-25T14:02:10.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:10 smithi138 ceph-mon[32977]: pgmap v962: 97 pgs: 97 active+clean; 271 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 516 op/s 2024-04-25T14:02:11.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 148087 19.02 MB/sec execute 293 sec latency 88.647 ms 2024-04-25T14:02:11.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:11.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:11.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:11.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:11.498 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 -- 172.21.15.94:0/6311347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0101790 msgr2=0x7f5cf0101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/6311347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0101790 0x7f5cf0101c00 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009900 tx=0x7f5ce402a410 comp rx=0 tx=0).stop 2024-04-25T14:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 -- 172.21.15.94:0/6311347 shutdown_connections 2024-04-25T14:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/6311347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0102290 0x7f5cf010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/6311347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0101790 0x7f5cf0101c00 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/6311347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 0x7f5cf01054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 -- 172.21.15.94:0/6311347 >> 172.21.15.94:0/6311347 conn(0x7f5cf00fd390 msgr2=0x7f5cf00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 -- 172.21.15.94:0/6311347 shutdown_connections 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.561+0000 7f5cf72a7640 1 -- 172.21.15.94:0/6311347 wait complete. 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 Processor -- start 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 -- start start 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0101790 0x7f5cf019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 0x7f5cf01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010f670 con 0x7f5cf0102290 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010f4f0 con 0x7f5cf0101790 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf72a7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010f7f0 con 0x7f5cf01050e0 2024-04-25T14:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47272/0 (socket says 172.21.15.94:47272) 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 -- 172.21.15.94:0/3399405964 learned_addr learned my addr 172.21.15.94:0/3399405964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0101790 0x7f5cf019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 0x7f5cf01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 -- 172.21.15.94:0/3399405964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 msgr2=0x7f5cf01a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 0x7f5cf01a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 -- 172.21.15.94:0/3399405964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0101790 msgr2=0x7f5cf019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0101790 0x7f5cf019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.562+0000 7f5cf4842640 1 -- 172.21.15.94:0/3399405964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf01a1d90 con 0x7f5cf0102290 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4006010 tx=0x7f5ce40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cee7fc640 1 -- 172.21.15.94:0/3399405964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce403f030 con 0x7f5cf0102290 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cee7fc640 1 -- 172.21.15.94:0/3399405964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce4031910 con 0x7f5cf0102290 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cee7fc640 1 -- 172.21.15.94:0/3399405964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4035070 con 0x7f5cf0102290 2024-04-25T14:02:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01a2020 con 0x7f5cf0102290 2024-04-25T14:02:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.563+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a2510 con 0x7f5cf0102290 2024-04-25T14:02:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.565+0000 7f5cee7fc640 1 -- 172.21.15.94:0/3399405964 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5ce4042030 con 0x7f5cf0102290 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.565+0000 7f5cfe231740 1 Processor -- start 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.565+0000 7f5cfe231740 1 -- start start 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833ec9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559833e2b460 con 0x559833afc2f0 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559833de5590 con 0x559833b044d0 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cfe231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559833e27420 con 0x559833b251b0 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47282/0 (socket says 172.21.15.94:47282) 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833ec9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5043640 1 -- 172.21.15.94:0/2097045936 learned_addr learned my addr 172.21.15.94:0/2097045936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cee7fc640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 0x7f5cd80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf4842640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 -- 172.21.15.94:0/2097045936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 msgr2=0x559833ed1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 -- 172.21.15.94:0/2097045936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 msgr2=0x559833eca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 -- 172.21.15.94:0/2097045936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559833df20d0 con 0x559833b251b0 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf4842640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5043640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cee7fc640 1 -- 172.21.15.94:0/3399405964 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce40ff180 con 0x7f5cf0102290 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.566+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 0x7f5cd80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.567+0000 7f5cf5844640 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833ec9d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009ca0 tx=0x7f5ce8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7f61032b789d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.567+0000 7f5cd77fe640 1 -- 172.21.15.94:0/2097045936 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8017050 con 0x559833b251b0 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.567+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 0x7f5cd80b1ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8023000 tx=0x7f5ce8024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.567+0000 7f5cd77fe640 1 -- 172.21.15.94:0/2097045936 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8003c40 con 0x559833b251b0 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.567+0000 7f5cd77fe640 1 -- 172.21.15.94:0/2097045936 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800f3c0 con 0x559833b251b0 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- 172.21.15.94:0/2097045936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 msgr2=0x559833ec9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833ec9d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009ca0 tx=0x7f5ce8011810 comp rx=0 tx=0).stop 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- 172.21.15.94:0/2097045936 shutdown_connections 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833eca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 --2- 172.21.15.94:0/2097045936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833ec9d80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- 172.21.15.94:0/2097045936 >> 172.21.15.94:0/2097045936 conn(0x559833b19300 msgr2=0x559833ec1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- 172.21.15.94:0/2097045936 shutdown_connections 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- 172.21.15.94:0/2097045936 wait complete. 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 -- start start 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.568+0000 7f5cfe231740 1 Processor -- start 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833edaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559833ed1c50 con 0x559833afc2f0 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559833db2b60 con 0x559833b044d0 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559833ed1dd0 con 0x559833b251b0 2024-04-25T14:02:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cfe231740 10 client.0 _ll_register_callbacks cb 0x559833dd72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47290/0 (socket says 172.21.15.94:47290) 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 -- 172.21.15.94:0/3329867791 learned_addr learned my addr 172.21.15.94:0/3329867791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833edaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 -- 172.21.15.94:0/3329867791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 msgr2=0x559833edaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833edaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 -- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 msgr2=0x559833ed3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 -- 172.21.15.94:0/3329867791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559833b3d210 con 0x559833afc2f0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0011d10 tx=0x7f5ce000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70579aeb4e019a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.569+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0004b20 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8003c40 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce0017050 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0004cc0 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 10 client.22803 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559833c09620 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8003c40 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce0022070 con 0x559833afc2f0 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 10 client.22803 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 10 client.22803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 20 client.22803 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833ae3d80 0x559833ee16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 4 client.22803 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559833e39070 con 0x559833ae3d80 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.570+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833ae3d80 0x559833ee16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.571+0000 7f5cd4ff9640 10 client.22803 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.571+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833ae3d80 0x559833ee16e0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de918ee2d24033f server_cookie=7be461392dc64e73 in_seq=0 out_seq=0 2024-04-25T14:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.574+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3329867791 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559833e39070 con 0x559833ae3d80 2024-04-25T14:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.574+0000 7f5cd4ff9640 10 client.22803 handle_command_reply: tid=0 2024-04-25T14:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.574+0000 7f5cfe231740 1 client.22803 shutdown 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 msgr2=0x559833ed3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0011d10 tx=0x7f5ce000f790 comp rx=0 tx=0).stop 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 shutdown_connections 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833ae3d80 0x559833ee16e0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833b251b0 0x559833edaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cd4ff9640 0 client.22803 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3329867791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833afc2f0 0x559833ed3a80 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 >> 172.21.15.94:0/3329867791 conn(0x559833b19300 msgr2=0x559833ec0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 shutdown_connections 2024-04-25T14:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 1 -- 172.21.15.94:0/3329867791 wait complete. 2024-04-25T14:02:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.575+0000 7f5cfe231740 20 client.22803 trim_cache size 0 max 16384 2024-04-25T14:02:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.590+0000 7f5cfe231740 1 Processor -- start 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.590+0000 7f5cfe231740 1 -- start start 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833edae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a92c0 con 0x559833ae3d80 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01aae00 con 0x559833b044d0 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cfe231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a9990 con 0x559833afc2f0 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47304/0 (socket says 172.21.15.94:47304) 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 -- 172.21.15.94:0/3809120642 learned_addr learned my addr 172.21.15.94:0/3809120642 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833edae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 -- 172.21.15.94:0/3809120642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 msgr2=0x559833edae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833edae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 -- 172.21.15.94:0/3809120642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 msgr2=0x559833ed3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 -- 172.21.15.94:0/3809120642 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559833ece930 con 0x559833ae3d80 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009600 tx=0x7f5ce0011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b005e7cae754a36d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cd6ffd640 1 -- 172.21.15.94:0/3809120642 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0031070 con 0x559833ae3d80 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cd6ffd640 1 -- 172.21.15.94:0/3809120642 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce00049d0 con 0x559833ae3d80 2024-04-25T14:02:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.591+0000 7f5cd6ffd640 1 -- 172.21.15.94:0/3809120642 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0028040 con 0x559833ae3d80 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 -- 172.21.15.94:0/3809120642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 msgr2=0x559833ec12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009600 tx=0x7f5ce0011980 comp rx=0 tx=0).stop 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 -- 172.21.15.94:0/3809120642 shutdown_connections 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833b044d0 0x559833ed3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833edae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3809120642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833ae3d80 0x559833ec12f0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 -- 172.21.15.94:0/3809120642 >> 172.21.15.94:0/3809120642 conn(0x559833b19300 msgr2=0x559833ec0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 -- 172.21.15.94:0/3809120642 shutdown_connections 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.592+0000 7f5cfe231740 1 -- 172.21.15.94:0/3809120642 wait complete. 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 -- start start 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 Processor -- start 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833ae3d80 0x559833ed8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833ed8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01aa730 con 0x559833b044d0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01aa060 con 0x559833ae3d80 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a72a0 con 0x559833afc2f0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cfe231740 10 client.0 _ll_register_callbacks cb 0x7f5ce0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47320/0 (socket says 172.21.15.94:47320) 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 -- 172.21.15.94:0/3626937885 learned_addr learned my addr 172.21.15.94:0/3626937885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5043640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833ae3d80 0x559833ed8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf4842640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833ed8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 -- 172.21.15.94:0/3626937885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 msgr2=0x559833ed8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833ed8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 -- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833ae3d80 msgr2=0x559833ed8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833ae3d80 0x559833ed8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.593+0000 7f5cf5844640 1 -- 172.21.15.94:0/3626937885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559833edb380 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cf5844640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x559833ecaf50 tx=0x7f5ce8004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb49a5a4b310d288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8005450 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5ce80055f0 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce00049d0 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ce8017020 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 10 client.22809 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559833edad70 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce00049d0 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce801c5e0 con 0x559833b044d0 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 10 client.22809 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 10 client.22809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 20 client.22809 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833b251b0 0x559833f4be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 4 client.22809 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.594+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559833e32ee0 con 0x559833b251b0 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.595+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833b251b0 0x559833f4be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.595+0000 7f5cd4ff9640 10 client.22809 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:11.595+0000 7f5cf5043640 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833b251b0 0x559833f4be30 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4568178f54264a20 server_cookie=79f8b72887a07090 in_seq=0 out_seq=0 2024-04-25T14:02:12.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 148469 19.03 MB/sec execute 294 sec latency 413.096 ms 2024-04-25T14:02:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cd4ff9640 1 -- 172.21.15.94:0/3626937885 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559833e32ee0 con 0x559833b251b0 2024-04-25T14:02:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cd4ff9640 10 client.22809 handle_command_reply: tid=0 2024-04-25T14:02:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 client.22809 shutdown 2024-04-25T14:02:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 msgr2=0x559833ed9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x559833ecaf50 tx=0x7f5ce8004120 comp rx=0 tx=0).stop 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 shutdown_connections 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559833b251b0 0x559833f4be30 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559833b044d0 0x559833ed9680 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cd4ff9640 0 client.22809 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559833afc2f0 0x559833ed8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 --2- 172.21.15.94:0/3626937885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559833ae3d80 0x559833ed8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 >> 172.21.15.94:0/3626937885 conn(0x559833b19300 msgr2=0x559833ebf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 shutdown_connections 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 1 -- 172.21.15.94:0/3626937885 wait complete. 2024-04-25T14:02:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.195+0000 7f5cfe231740 20 client.22809 trim_cache size 0 max 16384 2024-04-25T14:02:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:02:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:02:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:02:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 msgr2=0x7f5cd80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 0x7f5cd80b1ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8023000 tx=0x7f5ce8024000 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 msgr2=0x7f5cf019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4006010 tx=0x7f5ce40042f0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 shutdown_connections 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5cd80afae0 0x7f5cd80b1ff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf01050e0 0x7f5cf01a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cf0102290 0x7f5cf019a790 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 --2- 172.21.15.94:0/3399405964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cf0101790 0x7f5cf019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 >> 172.21.15.94:0/3399405964 conn(0x7f5cf00fd390 msgr2=0x7f5cf0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 shutdown_connections 2024-04-25T14:02:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:12.196+0000 7f5cf72a7640 1 -- 172.21.15.94:0/3399405964 wait complete. 2024-04-25T14:02:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:12 smithi137 ceph-mon[32568]: pgmap v963: 97 pgs: 97 active+clean; 267 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-04-25T14:02:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:12.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:12 smithi094 ceph-mon[29195]: pgmap v963: 97 pgs: 97 active+clean; 267 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-04-25T14:02:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:12.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:12 smithi138 ceph-mon[32977]: pgmap v963: 97 pgs: 97 active+clean; 267 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-04-25T14:02:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:12.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:13.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 148921 19.03 MB/sec execute 295 sec latency 362.876 ms 2024-04-25T14:02:14.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 149311 19.06 MB/sec execute 296 sec latency 117.931 ms 2024-04-25T14:02:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:14 smithi137 ceph-mon[32568]: pgmap v964: 97 pgs: 97 active+clean; 272 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-04-25T14:02:14.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:14 smithi094 ceph-mon[29195]: pgmap v964: 97 pgs: 97 active+clean; 272 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-04-25T14:02:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:14 smithi138 ceph-mon[32977]: pgmap v964: 97 pgs: 97 active+clean; 272 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 418 op/s 2024-04-25T14:02:15.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 149590 19.09 MB/sec execute 297 sec latency 325.393 ms 2024-04-25T14:02:16.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 149852 19.07 MB/sec execute 298 sec latency 82.247 ms 2024-04-25T14:02:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:16 smithi137 ceph-mon[32568]: pgmap v965: 97 pgs: 97 active+clean; 265 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-04-25T14:02:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:16 smithi094 ceph-mon[29195]: pgmap v965: 97 pgs: 97 active+clean; 265 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-04-25T14:02:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:16 smithi138 ceph-mon[32977]: pgmap v965: 97 pgs: 97 active+clean; 265 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-04-25T14:02:17.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 150093 19.06 MB/sec execute 299 sec latency 113.468 ms 2024-04-25T14:02:18.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 150296 19.02 MB/sec execute 300 sec latency 75.890 ms 2024-04-25T14:02:18.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:18 smithi137 ceph-mon[32568]: pgmap v966: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 534 op/s 2024-04-25T14:02:18.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:18 smithi094 ceph-mon[29195]: pgmap v966: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 534 op/s 2024-04-25T14:02:18.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:18 smithi138 ceph-mon[32977]: pgmap v966: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 534 op/s 2024-04-25T14:02:19.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 150535 18.99 MB/sec execute 301 sec latency 87.550 ms 2024-04-25T14:02:20.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 150858 18.96 MB/sec execute 302 sec latency 158.642 ms 2024-04-25T14:02:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:20 smithi137 ceph-mon[32568]: pgmap v967: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 522 op/s 2024-04-25T14:02:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:20 smithi094 ceph-mon[29195]: pgmap v967: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 522 op/s 2024-04-25T14:02:20.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:20 smithi138 ceph-mon[32977]: pgmap v967: 97 pgs: 97 active+clean; 263 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 522 op/s 2024-04-25T14:02:21.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 151161 18.93 MB/sec execute 303 sec latency 177.850 ms 2024-04-25T14:02:22.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 151730 18.98 MB/sec execute 304 sec latency 87.171 ms 2024-04-25T14:02:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:22 smithi137 ceph-mon[32568]: pgmap v968: 97 pgs: 97 active+clean; 258 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2024-04-25T14:02:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:22 smithi094 ceph-mon[29195]: pgmap v968: 97 pgs: 97 active+clean; 258 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2024-04-25T14:02:22.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:22 smithi138 ceph-mon[32977]: pgmap v968: 97 pgs: 97 active+clean; 258 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2024-04-25T14:02:23.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 152350 19.01 MB/sec execute 305 sec latency 65.323 ms 2024-04-25T14:02:24.151 INFO:tasks.workunit.client.0.smithi094.stdout: 10 152812 19.06 MB/sec execute 306 sec latency 60.581 ms 2024-04-25T14:02:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:24 smithi137 ceph-mon[32568]: pgmap v969: 97 pgs: 97 active+clean; 264 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:02:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:24 smithi094 ceph-mon[29195]: pgmap v969: 97 pgs: 97 active+clean; 264 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:02:24.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:24 smithi138 ceph-mon[32977]: pgmap v969: 97 pgs: 97 active+clean; 264 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:02:25.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 153142 19.09 MB/sec execute 307 sec latency 263.674 ms 2024-04-25T14:02:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 -- 172.21.15.94:0/328412633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50881021c0 msgr2=0x7f5088102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 --2- 172.21.15.94:0/328412633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50881021c0 0x7f5088102ec0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f5078009900 tx=0x7f507802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 -- 172.21.15.94:0/328412633 shutdown_connections 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 --2- 172.21.15.94:0/328412633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 0x7f508810a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 --2- 172.21.15.94:0/328412633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5088103400 0x7f5088103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 --2- 172.21.15.94:0/328412633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50881021c0 0x7f5088102ec0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 -- 172.21.15.94:0/328412633 >> 172.21.15.94:0/328412633 conn(0x7f50880fd390 msgr2=0x7f50880ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 -- 172.21.15.94:0/328412633 shutdown_connections 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.297+0000 7f508d384640 1 -- 172.21.15.94:0/328412633 wait complete. 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 Processor -- start 2024-04-25T14:02:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 -- start start 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50881021c0 0x7f50880ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 0x7f50880fde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d450 con 0x7f5088103400 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d2d0 con 0x7f50881021c0 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f508d384640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d5d0 con 0x7f5088108100 2024-04-25T14:02:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35458/0 (socket says 172.21.15.94:35458) 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f50867fc640 1 -- 172.21.15.94:0/3945469214 learned_addr learned my addr 172.21.15.94:0/3945469214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.298+0000 7f50877fe640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 0x7f50880fde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f5086ffd640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50881021c0 0x7f50880ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 -- 172.21.15.94:0/3945469214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 msgr2=0x7f50880fde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 0x7f50880fde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 -- 172.21.15.94:0/3945469214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50881021c0 msgr2=0x7f50880ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50881021c0 0x7f50880ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 -- 172.21.15.94:0/3945469214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50880fe530 con 0x7f5088103400 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f50867fc640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f506ffff640 1 -- 172.21.15.94:0/3945469214 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c00cd80 con 0x7f5088103400 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881a43c0 con 0x7f5088103400 2024-04-25T14:02:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f506ffff640 1 -- 172.21.15.94:0/3945469214 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c0108e0 con 0x7f5088103400 2024-04-25T14:02:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a48b0 con 0x7f5088103400 2024-04-25T14:02:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.299+0000 7f506ffff640 1 -- 172.21.15.94:0/3945469214 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c018980 con 0x7f5088103400 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.302+0000 7f506ffff640 1 -- 172.21.15.94:0/3945469214 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f507c018b20 con 0x7f5088103400 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.302+0000 7f5094234740 1 Processor -- start 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.302+0000 7f5094234740 1 -- start start 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.302+0000 7f5094234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.302+0000 7f5094234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f5094234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f5094234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555e3860f460 con 0x555e382e84d0 2024-04-25T14:02:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f5094234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555e385c9590 con 0x555e382e02f0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f5094234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555e3860b420 con 0x555e383091b0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f506ffff640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 0x7f50700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35464/0 (socket says 172.21.15.94:35464) 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 -- 172.21.15.94:0/1685855214 learned_addr learned my addr 172.21.15.94:0/1685855214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f5086ffd640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50877fe640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50877fe640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 0x7f50700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f506ffff640 1 -- 172.21.15.94:0/3945469214 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f507c0df0d0 con 0x7f5088103400 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 -- 172.21.15.94:0/1685855214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 msgr2=0x555e386a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 -- 172.21.15.94:0/1685855214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 msgr2=0x555e386a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 -- 172.21.15.94:0/1685855214 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e385d60d0 con 0x555e382e84d0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50877fe640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50867fc640 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f507c016000 tx=0x7f507c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d441455e4160d307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f506d7fa640 1 -- 172.21.15.94:0/1685855214 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0a7c10 con 0x555e382e84d0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.303+0000 7f50877fe640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 0x7f50700b1ff0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x555e386aab40 tx=0x7f508000a610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f506d7fa640 1 -- 172.21.15.94:0/1685855214 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c0ab8c0 con 0x555e382e84d0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f506d7fa640 1 -- 172.21.15.94:0/1685855214 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0a7770 con 0x555e382e84d0 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 -- 172.21.15.94:0/1685855214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 msgr2=0x555e386b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f507c016000 tx=0x7f507c018420 comp rx=0 tx=0).stop 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 -- 172.21.15.94:0/1685855214 shutdown_connections 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b0df0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 --2- 172.21.15.94:0/1685855214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.304+0000 7f5094234740 1 -- 172.21.15.94:0/1685855214 >> 172.21.15.94:0/1685855214 conn(0x555e382fd300 msgr2=0x555e386a5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- 172.21.15.94:0/1685855214 shutdown_connections 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- 172.21.15.94:0/1685855214 wait complete. 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- start start 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 Processor -- start 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555e386b17e0 con 0x555e383091b0 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555e38596b60 con 0x555e382e02f0 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555e386b1960 con 0x555e382e84d0 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.305+0000 7f5094234740 10 client.0 _ll_register_callbacks cb 0x555e385bb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50867fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50867fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:44264/0 (socket says 172.21.15.94:44264) 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50867fc640 1 -- 172.21.15.94:0/3896752657 learned_addr learned my addr 172.21.15.94:0/3896752657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50877fe640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 -- 172.21.15.94:0/3896752657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 msgr2=0x555e386b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 -- 172.21.15.94:0/3896752657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 msgr2=0x555e386baa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386baa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 -- 172.21.15.94:0/3896752657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e38321210 con 0x555e382e02f0 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50877fe640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f50867fc640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f5086ffd640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b3590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f507802a930 tx=0x7f50780317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef704727ab0a142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f506aff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.306+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078035040 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c0ab8c0 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5078004240 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5078004440 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 10 client.32228 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555e383ed620 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c0ab8c0 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f50780049c0 con 0x555e382e02f0 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 10 client.32228 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 10 client.32228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 20 client.32228 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e382c7d80 0x555e386c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 4 client.32228 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.307+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e3861d070 con 0x555e382c7d80 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.308+0000 7f50867fc640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e382c7d80 0x555e386c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.308+0000 7f506aff5640 10 client.32228 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.308+0000 7f50867fc640 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e382c7d80 0x555e386c1240 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e395c1276cb036 server_cookie=2acab71d9a4db638 in_seq=0 out_seq=0 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f506aff5640 1 -- 172.21.15.94:0/3896752657 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e3861d070 con 0x555e382c7d80 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f506aff5640 10 client.32228 handle_command_reply: tid=0 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 client.32228 shutdown 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 msgr2=0x555e386b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b3590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f507802a930 tx=0x7f50780317d0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 shutdown_connections 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e382c7d80 0x555e386c1240 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f506aff5640 0 client.32228 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 --2- 172.21.15.94:0/3896752657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b3590 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.309+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 >> 172.21.15.94:0/3896752657 conn(0x555e382fd300 msgr2=0x555e386a8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.310+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 shutdown_connections 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.310+0000 7f5094234740 1 -- 172.21.15.94:0/3896752657 wait complete. 2024-04-25T14:02:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.310+0000 7f5094234740 20 client.32228 trim_cache size 0 max 16384 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.325+0000 7f5094234740 1 Processor -- start 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.325+0000 7f5094234740 1 -- start start 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386a5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ad8c0 con 0x555e383091b0 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50881af400 con 0x555e382e02f0 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5094234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50881adf90 con 0x555e382e84d0 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35486/0 (socket says 172.21.15.94:35486) 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 -- 172.21.15.94:0/1374434288 learned_addr learned my addr 172.21.15.94:0/1374434288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f5086ffd640 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50867fc640 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 -- 172.21.15.94:0/1374434288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 msgr2=0x555e386a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 -- 172.21.15.94:0/1374434288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 msgr2=0x555e386a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 -- 172.21.15.94:0/1374434288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e386b11b0 con 0x555e383091b0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f50877fe640 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f5080003aa0 tx=0x7f5080015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43e67e7a938d934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.326+0000 7f506cff9640 1 -- 172.21.15.94:0/1374434288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800045d0 con 0x555e383091b0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f506cff9640 1 -- 172.21.15.94:0/1374434288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5080004770 con 0x555e383091b0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f506cff9640 1 -- 172.21.15.94:0/1374434288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080005690 con 0x555e383091b0 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 -- 172.21.15.94:0/1374434288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 msgr2=0x555e386a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f5080003aa0 tx=0x7f5080015000 comp rx=0 tx=0).stop 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 -- 172.21.15.94:0/1374434288 shutdown_connections 2024-04-25T14:02:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e383091b0 0x555e386a8100 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e382e84d0 0x555e386a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 --2- 172.21.15.94:0/1374434288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 -- 172.21.15.94:0/1374434288 >> 172.21.15.94:0/1374434288 conn(0x555e382fd300 msgr2=0x555e386adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 -- 172.21.15.94:0/1374434288 shutdown_connections 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.327+0000 7f5094234740 1 -- 172.21.15.94:0/1374434288 wait complete. 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 -- start start 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 Processor -- start 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50881aed30 con 0x555e382e84d0 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ae660 con 0x555e382e02f0 2024-04-25T14:02:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ab8a0 con 0x555e383091b0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5094234740 10 client.0 _ll_register_callbacks cb 0x7f508000a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35492/0 (socket says 172.21.15.94:35492) 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 -- 172.21.15.94:0/351286337 learned_addr learned my addr 172.21.15.94:0/351286337 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f5086ffd640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50877fe640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 -- 172.21.15.94:0/351286337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 msgr2=0x555e386b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 -- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 msgr2=0x555e386b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.328+0000 7f50867fc640 1 -- 172.21.15.94:0/351286337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e386b9060 con 0x555e382e84d0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f50867fc640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f507c0a7210 tx=0x7f507c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d470d078234d1516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0ab8c0 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f507c0a3b00 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5080004770 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f507c010430 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 10 client.22827 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555e386ba360 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5080004770 con 0x555e382e84d0 2024-04-25T14:02:25.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f507c0a2e80 con 0x555e382e84d0 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 10 client.22827 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 10 client.22827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 20 client.22827 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e3829d1c0 0x555e3872b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 4 client.22827 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:25.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e38616ee0 con 0x555e3829d1c0 2024-04-25T14:02:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.329+0000 7f5086ffd640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e3829d1c0 0x555e3872b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.330+0000 7f5086ffd640 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e3829d1c0 0x555e3872b9e0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26cdcd0f876ce015 server_cookie=e89efe0999f80bfe in_seq=0 out_seq=0 2024-04-25T14:02:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:25.330+0000 7f506aff5640 10 client.22827 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.054+0000 7f506aff5640 1 -- 172.21.15.94:0/351286337 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e38616ee0 con 0x555e3829d1c0 2024-04-25T14:02:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.054+0000 7f506aff5640 10 client.22827 handle_command_reply: tid=0 2024-04-25T14:02:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.054+0000 7f5094234740 1 client.22827 shutdown 2024-04-25T14:02:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.054+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 msgr2=0x555e386b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.054+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f507c0a7210 tx=0x7f507c00cf20 comp rx=0 tx=0).stop 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 shutdown_connections 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555e3829d1c0 0x555e3872b9e0 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f506aff5640 0 client.22827 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e383091b0 0x555e386b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555e382e84d0 0x555e386b7b70 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 --2- 172.21.15.94:0/351286337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382e02f0 0x555e386b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 >> 172.21.15.94:0/351286337 conn(0x555e382fd300 msgr2=0x555e386a8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 shutdown_connections 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 1 -- 172.21.15.94:0/351286337 wait complete. 2024-04-25T14:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f5094234740 20 client.22827 trim_cache size 0 max 16384 2024-04-25T14:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 msgr2=0x7f50700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 0x7f50700b1ff0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x555e386aab40 tx=0x7f508000a610 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 msgr2=0x7f50880fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.055+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 shutdown_connections 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f50700afae0 0x7f50700b1ff0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088108100 0x7f50880fde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5088103400 0x7f50880fd8c0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 --2- 172.21.15.94:0/3945469214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50881021c0 0x7f50880ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 >> 172.21.15.94:0/3945469214 conn(0x7f50880fd390 msgr2=0x7f5088109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 shutdown_connections 2024-04-25T14:02:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:26.056+0000 7f508d384640 1 -- 172.21.15.94:0/3945469214 wait complete. 2024-04-25T14:02:26.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 153286 19.06 MB/sec execute 308 sec latency 381.305 ms 2024-04-25T14:02:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:26 smithi137 ceph-mon[32568]: pgmap v970: 97 pgs: 97 active+clean; 268 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-04-25T14:02:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:26 smithi094 ceph-mon[29195]: pgmap v970: 97 pgs: 97 active+clean; 268 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-04-25T14:02:26.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:26 smithi138 ceph-mon[32977]: pgmap v970: 97 pgs: 97 active+clean; 268 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-04-25T14:02:27.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 153580 19.06 MB/sec execute 309 sec latency 55.496 ms 2024-04-25T14:02:28.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 153781 19.02 MB/sec execute 310 sec latency 129.842 ms 2024-04-25T14:02:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:28 smithi137 ceph-mon[32568]: pgmap v971: 97 pgs: 97 active+clean; 272 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-04-25T14:02:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:28 smithi094 ceph-mon[29195]: pgmap v971: 97 pgs: 97 active+clean; 272 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-04-25T14:02:28.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:28 smithi138 ceph-mon[32977]: pgmap v971: 97 pgs: 97 active+clean; 272 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.8 MiB/s wr, 379 op/s 2024-04-25T14:02:29.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 154051 18.99 MB/sec execute 311 sec latency 104.422 ms 2024-04-25T14:02:30.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 154308 18.95 MB/sec execute 312 sec latency 154.956 ms 2024-04-25T14:02:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:30 smithi094 ceph-mon[29195]: pgmap v972: 97 pgs: 97 active+clean; 271 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T14:02:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:30 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:30 smithi138 ceph-mon[32977]: pgmap v972: 97 pgs: 97 active+clean; 271 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T14:02:30.499 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:30 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:30 smithi137 ceph-mon[32568]: pgmap v972: 97 pgs: 97 active+clean; 271 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 327 op/s 2024-04-25T14:02:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:30 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:31.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 154662 18.93 MB/sec execute 313 sec latency 165.134 ms 2024-04-25T14:02:32.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 155146 18.96 MB/sec execute 314 sec latency 75.545 ms 2024-04-25T14:02:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:32 smithi094 ceph-mon[29195]: pgmap v973: 97 pgs: 97 active+clean; 277 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T14:02:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:32 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:32 smithi138 ceph-mon[32977]: pgmap v973: 97 pgs: 97 active+clean; 277 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T14:02:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:32 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:32 smithi137 ceph-mon[32568]: pgmap v973: 97 pgs: 97 active+clean; 277 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-04-25T14:02:32.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:32.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:32 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 155697 18.99 MB/sec execute 315 sec latency 65.432 ms 2024-04-25T14:02:33.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:33.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:33.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:33 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:33.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:33.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:33 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:33.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:33 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:02:34.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 156199 19.02 MB/sec execute 316 sec latency 71.243 ms 2024-04-25T14:02:34.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:34.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:34.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 -- 172.21.15.94:0/2911110908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 msgr2=0x7f96541027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 --2- 172.21.15.94:0/2911110908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f96541027d0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f9644009900 tx=0x7f964402a410 comp rx=0 tx=0).stop 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 -- 172.21.15.94:0/2911110908 shutdown_connections 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 --2- 172.21.15.94:0/2911110908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654108a20 0x7f965410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 --2- 172.21.15.94:0/2911110908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96541043d0 0x7f9654102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 --2- 172.21.15.94:0/2911110908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f96541027d0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 -- 172.21.15.94:0/2911110908 >> 172.21.15.94:0/2911110908 conn(0x7f96540fd350 msgr2=0x7f96540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 -- 172.21.15.94:0/2911110908 shutdown_connections 2024-04-25T14:02:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.397+0000 7f9659c9f640 1 -- 172.21.15.94:0/2911110908 wait complete. 2024-04-25T14:02:34.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.398+0000 7f9659c9f640 1 Processor -- start 2024-04-25T14:02:34.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.398+0000 7f9659c9f640 1 -- start start 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654108a20 0x7f9654108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654109490 0x7f96541098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e500 con 0x7f9654103ef0 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e380 con 0x7f9654109490 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9659c9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e680 con 0x7f9654108a20 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34636/0 (socket says 172.21.15.94:34636) 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 -- 172.21.15.94:0/538451246 learned_addr learned my addr 172.21.15.94:0/538451246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9652ffd640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654108a20 0x7f9654108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9653fff640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654109490 0x7f96541098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 -- 172.21.15.94:0/538451246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654108a20 msgr2=0x7f9654108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654108a20 0x7f9654108f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 -- 172.21.15.94:0/538451246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654109490 msgr2=0x7f96541098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654109490 0x7f96541098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 -- 172.21.15.94:0/538451246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541a3f30 con 0x7f9654103ef0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f96537fe640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f9644032fd0 tx=0x7f9644004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.399+0000 7f9650ff9640 1 -- 172.21.15.94:0/538451246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964403f030 con 0x7f9654103ef0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.400+0000 7f9650ff9640 1 -- 172.21.15.94:0/538451246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9644004490 con 0x7f9654103ef0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.400+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96541a41c0 con 0x7f9654103ef0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.400+0000 7f9650ff9640 1 -- 172.21.15.94:0/538451246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644035070 con 0x7f9654103ef0 2024-04-25T14:02:34.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.400+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a46b0 con 0x7f9654103ef0 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 Processor -- start 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 -- start start 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640020ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc4d6b0 0x55a640021c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a63ff74d80 con 0x55a63fc6e390 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a63ff2e8b0 con 0x55a63fc454d0 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.403+0000 7f9660c2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a63ff70d40 con 0x55a63fc4d6b0 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34640/0 (socket says 172.21.15.94:34640) 2024-04-25T14:02:34.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 -- 172.21.15.94:0/2674997489 learned_addr learned my addr 172.21.15.94:0/2674997489 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9650ff9640 1 -- 172.21.15.94:0/538451246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9644042030 con 0x7f9654103ef0 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 -- 172.21.15.94:0/2674997489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc4d6b0 msgr2=0x55a640021c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc4d6b0 0x55a640021c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.404+0000 7f9653fff640 1 -- 172.21.15.94:0/2674997489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 msgr2=0x55a640020ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f96537fe640 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640020ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f9653fff640 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640020ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f9653fff640 1 -- 172.21.15.94:0/2674997489 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63feffa40 con 0x55a63fc6e390 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f9653fff640 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f964c0039f0 tx=0x7f964c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d1845eef6878b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f9639ffb640 1 -- 172.21.15.94:0/2674997489 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c03c470 con 0x55a63fc6e390 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.405+0000 7f9639ffb640 1 -- 172.21.15.94:0/2674997489 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c037070 con 0x55a63fc6e390 2024-04-25T14:02:34.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.406+0000 7f9639ffb640 1 -- 172.21.15.94:0/2674997489 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c0328b0 con 0x55a63fc6e390 2024-04-25T14:02:34.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.406+0000 7f9650ff9640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 0x7f963c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.406+0000 7f96537fe640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 0x7f963c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.406+0000 7f9650ff9640 1 -- 172.21.15.94:0/538451246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f96440ff100 con 0x7f9654103ef0 2024-04-25T14:02:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- 172.21.15.94:0/2674997489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 msgr2=0x55a640020a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f96537fe640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 0x7f963c0b2000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f96440c7290 tx=0x7f96440c2d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f964c0039f0 tx=0x7f964c003eb0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- 172.21.15.94:0/2674997489 shutdown_connections 2024-04-25T14:02:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc4d6b0 0x55a640021c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640020ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2674997489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc6e390 0x55a640020a70 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- 172.21.15.94:0/2674997489 >> 172.21.15.94:0/2674997489 conn(0x55a63fc624e0 msgr2=0x55a64000b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- 172.21.15.94:0/2674997489 shutdown_connections 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- 172.21.15.94:0/2674997489 wait complete. 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.407+0000 7f9660c2d740 1 -- start start 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 Processor -- start 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640017220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a640022790 con 0x55a63fc4d6b0 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a63fefbd10 con 0x55a63fc454d0 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a640022910 con 0x55a63fc6e390 2024-04-25T14:02:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9660c2d740 10 client.0 _ll_register_callbacks cb 0x55a63ff28210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9652ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9652ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34656/0 (socket says 172.21.15.94:34656) 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9652ffd640 1 -- 172.21.15.94:0/2604189040 learned_addr learned my addr 172.21.15.94:0/2604189040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.408+0000 7f9653fff640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9653fff640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640017220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 -- 172.21.15.94:0/2604189040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 msgr2=0x55a64001e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 -- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 msgr2=0x55a640017220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640017220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 -- 172.21.15.94:0/2604189040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63fd507d0 con 0x55a63fc4d6b0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9652ffd640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f96480116f0 tx=0x7f9648011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea6d8bf60dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004460 con 0x55a63fc4d6b0 2024-04-25T14:02:34.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f964c037070 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f964800f920 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.409+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f964800fb20 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 10 client.22839 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a63fc863f0 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c037070 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9648018070 con 0x55a63fc4d6b0 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 10 client.22839 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 10 client.22839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 20 client.22839 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a63fc2cf60 0x55a640027880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 4 client.22839 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a63ff82990 con 0x55a63fc2cf60 2024-04-25T14:02:34.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.410+0000 7f9653fff640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a63fc2cf60 0x55a640027880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.411+0000 7f9653fff640 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a63fc2cf60 0x55a640027880 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82090f1eb95d4b3e server_cookie=edf57aab66befe4c in_seq=0 out_seq=0 2024-04-25T14:02:34.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.411+0000 7f9632ffd640 10 client.22839 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:34.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.417+0000 7f9632ffd640 1 -- 172.21.15.94:0/2604189040 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a63ff82990 con 0x55a63fc2cf60 2024-04-25T14:02:34.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.417+0000 7f9632ffd640 10 client.22839 handle_command_reply: tid=0 2024-04-25T14:02:34.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 client.22839 shutdown 2024-04-25T14:02:34.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 msgr2=0x55a640017760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f96480116f0 tx=0x7f9648011bb0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 shutdown_connections 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a63fc2cf60 0x55a640027880 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc4d6b0 0x55a640017760 secure :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f96480116f0 tx=0x7f9648011bb0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 --2- 172.21.15.94:0/2604189040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640017220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 >> 172.21.15.94:0/2604189040 conn(0x55a63fc624e0 msgr2=0x55a64000a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.418+0000 7f9632ffd640 0 client.22839 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.419+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 shutdown_connections 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.419+0000 7f9660c2d740 1 -- 172.21.15.94:0/2604189040 wait complete. 2024-04-25T14:02:34.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.419+0000 7f9660c2d740 20 client.22839 trim_cache size 0 max 16384 2024-04-25T14:02:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.440+0000 7f9660c2d740 1 Processor -- start 2024-04-25T14:02:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.440+0000 7f9660c2d740 1 -- start start 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a6400277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a640012100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a4ea0 con 0x55a63fc023a0 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a69e0 con 0x55a63fc454d0 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9660c2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a5570 con 0x55a63fc6e390 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9652ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a640012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34672/0 (socket says 172.21.15.94:34672) 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 -- 172.21.15.94:0/723137244 learned_addr learned my addr 172.21.15.94:0/723137244 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a6400277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 -- 172.21.15.94:0/723137244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 msgr2=0x55a640012100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a640012100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 -- 172.21.15.94:0/723137244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 msgr2=0x55a6400277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a6400277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 -- 172.21.15.94:0/723137244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a640017900 con 0x55a63fc023a0 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9652ffd640 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a640012100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.441+0000 7f9653fff640 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x55a64000bbf0 tx=0x7f964c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6cb885f7ddc0df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.442+0000 7f96397fa640 1 -- 172.21.15.94:0/723137244 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c004a00 con 0x55a63fc023a0 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.442+0000 7f96397fa640 1 -- 172.21.15.94:0/723137244 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c037040 con 0x55a63fc023a0 2024-04-25T14:02:34.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.442+0000 7f96397fa640 1 -- 172.21.15.94:0/723137244 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c04e6b0 con 0x55a63fc023a0 2024-04-25T14:02:34.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- 172.21.15.94:0/723137244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 msgr2=0x55a640012640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x55a64000bbf0 tx=0x7f964c005c50 comp rx=0 tx=0).stop 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- 172.21.15.94:0/723137244 shutdown_connections 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a640012640 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a640012100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 --2- 172.21.15.94:0/723137244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a6400277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- 172.21.15.94:0/723137244 >> 172.21.15.94:0/723137244 conn(0x55a63fc624e0 msgr2=0x55a6400113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- 172.21.15.94:0/723137244 shutdown_connections 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- 172.21.15.94:0/723137244 wait complete. 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 -- start start 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.443+0000 7f9660c2d740 1 Processor -- start 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a6400188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640018e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a6310 con 0x55a63fc023a0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96541a5c40 con 0x55a63fc454d0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9654073340 con 0x55a63fc6e390 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9660c2d740 10 client.0 _ll_register_callbacks cb 0x7f9648009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a6400188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a6400188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34688/0 (socket says 172.21.15.94:34688) 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9653fff640 1 -- 172.21.15.94:0/3672661810 learned_addr learned my addr 172.21.15.94:0/3672661810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640018e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9653fff640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 -- 172.21.15.94:0/3672661810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 msgr2=0x55a64001da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 -- 172.21.15.94:0/3672661810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 msgr2=0x55a6400188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a6400188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.444+0000 7f9652ffd640 1 -- 172.21.15.94:0/3672661810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63fc460f0 con 0x55a63fc454d0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9652ffd640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640018e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9648008000 tx=0x7f964801a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1483f6957586619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:02:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648027040 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f96480052d0 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f964c037040 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9648005510 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9660c2d740 10 client.32249 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.445+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a63ff354b0 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f964c037040 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9648034070 con 0x55a63fc454d0 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 10 client.32249 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 10 client.32249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 20 client.32249 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a64008f210 0x55a640091630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 4 client.32249 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a63ff7c800 con 0x55a64008f210 2024-04-25T14:02:34.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.446+0000 7f9653fff640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a64008f210 0x55a640091630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.447+0000 7f9653fff640 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a64008f210 0x55a640091630 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65bfe1849a1cb6c server_cookie=abd87e54753c7f63 in_seq=0 out_seq=0 2024-04-25T14:02:34.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.447+0000 7f9632ffd640 10 client.32249 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.449+0000 7f9632ffd640 1 -- 172.21.15.94:0/3672661810 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a63ff7c800 con 0x55a64008f210 2024-04-25T14:02:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.449+0000 7f9632ffd640 10 client.32249 handle_command_reply: tid=0 2024-04-25T14:02:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.449+0000 7f9660c2d740 1 client.32249 shutdown 2024-04-25T14:02:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 msgr2=0x55a640018e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640018e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9648008000 tx=0x7f964801a990 comp rx=0 tx=0).stop 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 shutdown_connections 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a64008f210 0x55a640091630 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fc6e390 0x55a64001da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a63fc454d0 0x55a640018e20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 --2- 172.21.15.94:0/3672661810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a63fc023a0 0x55a6400188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9632ffd640 0 client.32249 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 >> 172.21.15.94:0/3672661810 conn(0x55a63fc624e0 msgr2=0x55a640015770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 shutdown_connections 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 1 -- 172.21.15.94:0/3672661810 wait complete. 2024-04-25T14:02:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9660c2d740 20 client.32249 trim_cache size 0 max 16384 2024-04-25T14:02:34.453 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:34.453 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:02:34.453 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:02:34.453 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 msgr2=0x7f963c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 0x7f963c0b2000 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f96440c7290 tx=0x7f96440c2d30 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 msgr2=0x7f965410a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.450+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f9644032fd0 tx=0x7f9644004270 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 shutdown_connections 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f963c0afaf0 0x7f963c0b2000 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9654109490 0x7f96541098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9654108a20 0x7f9654108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 --2- 172.21.15.94:0/538451246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9654103ef0 0x7f965410a8b0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 >> 172.21.15.94:0/538451246 conn(0x7f96540fd350 msgr2=0x7f96540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 shutdown_connections 2024-04-25T14:02:34.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.451+0000 7f9659c9f640 1 -- 172.21.15.94:0/538451246 wait complete. 2024-04-25T14:02:34.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:02:34.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.581+0000 7f6575461640 1 -- 172.21.15.94:0/1692044241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6570108900 msgr2=0x7f657010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.581+0000 7f6575461640 1 --2- 172.21.15.94:0/1692044241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6570108900 0x7f657010ad20 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f6560009ca0 tx=0x7f656002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 -- 172.21.15.94:0/1692044241 shutdown_connections 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 --2- 172.21.15.94:0/1692044241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6570108900 0x7f657010ad20 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 --2- 172.21.15.94:0/1692044241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570103c00 0x7f6570104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 --2- 172.21.15.94:0/1692044241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65701029c0 0x7f65701036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 -- 172.21.15.94:0/1692044241 >> 172.21.15.94:0/1692044241 conn(0x7f65700fdbd0 msgr2=0x7f6570100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 -- 172.21.15.94:0/1692044241 shutdown_connections 2024-04-25T14:02:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 -- 172.21.15.94:0/1692044241 wait complete. 2024-04-25T14:02:34.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.582+0000 7f6575461640 1 Processor -- start 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 -- start start 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 0x7f65700fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 0x7f65700fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f657010e4d0 con 0x7f65701029c0 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f657010e350 con 0x7f6570103c00 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f6575461640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f657010e650 con 0x7f6570108900 2024-04-25T14:02:34.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34694/0 (socket says 172.21.15.94:34694) 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 -- 172.21.15.94:0/763266193 learned_addr learned my addr 172.21.15.94:0/763266193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656e7fc640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 0x7f65700fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656f7fe640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 0x7f65700fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 -- 172.21.15.94:0/763266193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 msgr2=0x7f65700fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 0x7f65700fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 -- 172.21.15.94:0/763266193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 msgr2=0x7f65700fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 0x7f65700fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656effd640 1 -- 172.21.15.94:0/763266193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65700fed70 con 0x7f65701029c0 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656e7fc640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 0x7f65700fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.583+0000 7f656f7fe640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 0x7f65700fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f656effd640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9f0 tx=0x7f656800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656800ccc0 con 0x7f65701029c0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701a4260 con 0x7f65701029c0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65680035f0 con 0x7f65701029c0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656800ce60 con 0x7f65701029c0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.584+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65701a4750 con 0x7f65701029c0 2024-04-25T14:02:34.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.585+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65701a4a10 con 0x7f65701029c0 2024-04-25T14:02:34.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.587+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6568003aa0 con 0x7f65701029c0 2024-04-25T14:02:34.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.587+0000 7f6557fff640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 0x7f65580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.587+0000 7f656e7fc640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 0x7f65580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.587+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f65680df0a0 con 0x7f65701029c0 2024-04-25T14:02:34.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.588+0000 7f656e7fc640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 0x7f65580b2000 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f65700ff400 tx=0x7f6564005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.591+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65680a7be0 con 0x7f65701029c0 2024-04-25T14:02:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:34 smithi137 ceph-mon[32568]: pgmap v974: 97 pgs: 97 active+clean; 262 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T14:02:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:34 smithi094 ceph-mon[29195]: pgmap v974: 97 pgs: 97 active+clean; 262 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T14:02:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:34 smithi138 ceph-mon[32977]: pgmap v974: 97 pgs: 97 active+clean; 262 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-04-25T14:02:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.827+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65701a4be0 con 0x7f65701029c0 2024-04-25T14:02:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.828+0000 7f6557fff640 1 -- 172.21.15.94:0/763266193 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f65680a74e0 con 0x7f65701029c0 2024-04-25T14:02:34.828 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:02:34.828 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:02:34.828 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:02:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 msgr2=0x7f65580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 0x7f65580b2000 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f65700ff400 tx=0x7f6564005f90 comp rx=0 tx=0).stop 2024-04-25T14:02:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 msgr2=0x7f65700ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9f0 tx=0x7f656800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 shutdown_connections 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f65580afaf0 0x7f65580b2000 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6570108900 0x7f65700fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6570103c00 0x7f65700fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 --2- 172.21.15.94:0/763266193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65701029c0 0x7f65700ffa60 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 >> 172.21.15.94:0/763266193 conn(0x7f65700fdbd0 msgr2=0x7f6570109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 shutdown_connections 2024-04-25T14:02:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:34.830+0000 7f6575461640 1 -- 172.21.15.94:0/763266193 wait complete. 2024-04-25T14:02:34.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:02:35.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.015+0000 7f4cf0d27640 1 -- 172.21.15.94:0/3665389668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec101d70 msgr2=0x7f4cec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.015+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/3665389668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec101d70 0x7f4cec1021c0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd402a420 comp rx=0 tx=0).stop 2024-04-25T14:02:35.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.020+0000 7f4cf0d27640 1 -- 172.21.15.94:0/3665389668 shutdown_connections 2024-04-25T14:02:35.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.020+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/3665389668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cec107920 0x7f4cec107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.020+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/3665389668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec1027b0 0x7f4cec102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.020+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/3665389668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec101d70 0x7f4cec1021c0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.020+0000 7f4cf0d27640 1 -- 172.21.15.94:0/3665389668 >> 172.21.15.94:0/3665389668 conn(0x7f4cec0fd340 msgr2=0x7f4cec0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.022+0000 7f4cf0d27640 1 -- 172.21.15.94:0/3665389668 shutdown_connections 2024-04-25T14:02:35.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.023+0000 7f4cf0d27640 1 -- 172.21.15.94:0/3665389668 wait complete. 2024-04-25T14:02:35.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 Processor -- start 2024-04-25T14:02:35.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 -- start start 2024-04-25T14:02:35.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cec101d70 0x7f4cec071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 0x7f4cec071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 0x7f4cec072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec115ca0 con 0x7f4cec1027b0 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec115b20 con 0x7f4cec101d70 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.024+0000 7f4cf0d27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec115e20 con 0x7f4cec107920 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.025+0000 7f4cead9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 0x7f4cec072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.025+0000 7f4cead9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 0x7f4cec072f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52666/0 (socket says 172.21.15.94:52666) 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.025+0000 7f4cead9d640 1 -- 172.21.15.94:0/1567872359 learned_addr learned my addr 172.21.15.94:0/1567872359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.025+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 0x7f4cec071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/1567872359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 msgr2=0x7f4cec072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 0x7f4cec072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/1567872359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cec101d70 msgr2=0x7f4cec071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cec101d70 0x7f4cec071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/1567872359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec0735c0 con 0x7f4cec1027b0 2024-04-25T14:02:35.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.026+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 0x7f4cec071fb0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c970 tx=0x7f4cdc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.027+0000 7f4ce3fff640 1 -- 172.21.15.94:0/1567872359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007bb0 con 0x7f4cec1027b0 2024-04-25T14:02:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.027+0000 7f4ce3fff640 1 -- 172.21.15.94:0/1567872359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cdc007d50 con 0x7f4cec1027b0 2024-04-25T14:02:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.027+0000 7f4ce3fff640 1 -- 172.21.15.94:0/1567872359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004df0 con 0x7f4cec1027b0 2024-04-25T14:02:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.028+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec0738b0 con 0x7f4cec1027b0 2024-04-25T14:02:35.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.028+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec073e30 con 0x7f4cec1027b0 2024-04-25T14:02:35.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.032+0000 7f4cf7bd7740 1 Processor -- start 2024-04-25T14:02:35.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.033+0000 7f4cf7bd7740 1 -- start start 2024-04-25T14:02:35.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef180810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef180d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5578ef0e4d80 con 0x5578eedbd6b0 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5578ef09ea20 con 0x5578eedde390 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.034+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578ef0e0d40 con 0x5578eedb54d0 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.035+0000 7f4ce3fff640 1 -- 172.21.15.94:0/1567872359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4cdc004f90 con 0x7f4cec1027b0 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.035+0000 7f4ce1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.035+0000 7f4ce1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34718/0 (socket says 172.21.15.94:34718) 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.035+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/4260172190 learned_addr learned my addr 172.21.15.94:0/4260172190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.036+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef180d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.036+0000 7f4ce3fff640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 0x7f4cd80b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cead9d640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 0x7f4cd80b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cea59c640 1 -- 172.21.15.94:0/4260172190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 msgr2=0x5578ef180810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef180810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cea59c640 1 -- 172.21.15.94:0/4260172190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 msgr2=0x5578ef181a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4cea59c640 1 -- 172.21.15.94:0/4260172190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ef0a9c30 con 0x5578eedb54d0 2024-04-25T14:02:35.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.037+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.038+0000 7f4ce3fff640 1 -- 172.21.15.94:0/1567872359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4cdc0de8a0 con 0x7f4cec1027b0 2024-04-25T14:02:35.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.038+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef180d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009230 tx=0x7f4cd4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d679ff6b6cfb4410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.039+0000 7f4cead9d640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 0x7f4cd80b2230 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x5578ef181860 tx=0x7f4ce4009a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.040+0000 7f4ce0d99640 1 -- 172.21.15.94:0/4260172190 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd403e070 con 0x5578eedb54d0 2024-04-25T14:02:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.040+0000 7f4ce0d99640 1 -- 172.21.15.94:0/4260172190 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40054b0 con 0x5578eedb54d0 2024-04-25T14:02:35.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.040+0000 7f4ce0d99640 1 -- 172.21.15.94:0/4260172190 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4035040 con 0x5578eedb54d0 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.041+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4260172190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 msgr2=0x5578ef180d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.041+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef180d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009230 tx=0x7f4cd4000bc0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.042+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4260172190 shutdown_connections 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.042+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef181a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.042+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef180d90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.042+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4260172190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef180810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.042+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4260172190 >> 172.21.15.94:0/4260172190 conn(0x5578eedd24e0 msgr2=0x5578ef17c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.044+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4260172190 shutdown_connections 2024-04-25T14:02:35.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.045+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4260172190 wait complete. 2024-04-25T14:02:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.045+0000 7f4cf7bd7740 1 -- start start 2024-04-25T14:02:35.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.045+0000 7f4cf7bd7740 1 Processor -- start 2024-04-25T14:02:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.046+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.046+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef18ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.046+0000 7f4cea59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.046+0000 7f4cea59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52670/0 (socket says 172.21.15.94:52670) 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cea59c640 1 -- 172.21.15.94:0/4170484235 learned_addr learned my addr 172.21.15.94:0/4170484235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef191ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5578ef1824d0 con 0x5578eedbd6b0 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5578ef06bdc0 con 0x5578eedde390 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578ef182650 con 0x5578eedb54d0 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef18ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.047+0000 7f4cf7bd7740 10 client.0 _ll_register_callbacks cb 0x5578ef087600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4cead9d640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef191ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 msgr2=0x5578ef18a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 msgr2=0x5578ef191ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef191ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578eedf63f0 con 0x5578eedbd6b0 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4cead9d640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef191ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:35.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.048+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.049+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef18ab60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a2d00 tx=0x7f4cdc0a64b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8eeec3118d65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.049+0000 7f4cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.049+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc01d040 con 0x5578eedbd6b0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.049+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4cdc0a5ee0 con 0x5578eedbd6b0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.049+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd40054b0 con 0x5578eedbd6b0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.050+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc0a6c80 con 0x5578eedbd6b0 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.050+0000 7f4cf7bd7740 10 client.22863 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.050+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578eeec07d0 con 0x5578eedbd6b0 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40054b0 con 0x5578eedbd6b0 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4cdc0a5950 con 0x5578eedbd6b0 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 10 client.22863 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 10 client.22863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 20 client.22863 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed9cf60 0x5578ef198360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 4 client.22863 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:35.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.051+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ef0f2990 con 0x5578eed9cf60 2024-04-25T14:02:35.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.052+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed9cf60 0x5578ef198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.053+0000 7f4cea59c640 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed9cf60 0x5578ef198360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7342a4bd6b9255c0 server_cookie=a9f236a812008d5 in_seq=0 out_seq=0 2024-04-25T14:02:35.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.053+0000 7f4cd1ffb640 10 client.22863 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:35.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.053+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/4170484235 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578ef0f2990 con 0x5578eed9cf60 2024-04-25T14:02:35.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.053+0000 7f4cd1ffb640 10 client.22863 handle_command_reply: tid=0 2024-04-25T14:02:35.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.053+0000 7f4cf7bd7740 1 client.22863 shutdown 2024-04-25T14:02:35.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.054+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 msgr2=0x5578ef18ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.054+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef18ab60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a2d00 tx=0x7f4cdc0a64b0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 shutdown_connections 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed9cf60 0x5578ef198360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedde390 0x5578ef191ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef18ab60 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/4170484235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef18a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 >> 172.21.15.94:0/4170484235 conn(0x5578eedd24e0 msgr2=0x5578ef17ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cd1ffb640 0 client.22863 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 shutdown_connections 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/4170484235 wait complete. 2024-04-25T14:02:35.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.055+0000 7f4cf7bd7740 20 client.22863 trim_cache size 0 max 16384 2024-04-25T14:02:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 Processor -- start 2024-04-25T14:02:35.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 -- start start 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef176d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedbd6b0 0x5578ef188cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1a9460 con 0x5578eedde390 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1aafa0 con 0x5578eedbd6b0 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1a9b30 con 0x5578eedb54d0 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34724/0 (socket says 172.21.15.94:34724) 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cead9d640 1 -- 172.21.15.94:0/1733544192 learned_addr learned my addr 172.21.15.94:0/1733544192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedbd6b0 0x5578ef188cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.075+0000 7f4cea59c640 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef176d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 -- 172.21.15.94:0/1733544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 msgr2=0x5578ef176d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef176d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 -- 172.21.15.94:0/1733544192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedbd6b0 msgr2=0x5578ef188cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedbd6b0 0x5578ef188cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 -- 172.21.15.94:0/1733544192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ef18b760 con 0x5578eedde390 2024-04-25T14:02:35.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cead9d640 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40167e0 tx=0x7f4ce4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1e4796336522bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cd3fff640 1 -- 172.21.15.94:0/1733544192 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4009e60 con 0x5578eedde390 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cd3fff640 1 -- 172.21.15.94:0/1733544192 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce4025070 con 0x5578eedde390 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.076+0000 7f4cd3fff640 1 -- 172.21.15.94:0/1733544192 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4020d20 con 0x5578eedde390 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/1733544192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 msgr2=0x5578ef1891f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40167e0 tx=0x7f4ce4013000 comp rx=0 tx=0).stop 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/1733544192 shutdown_connections 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedde390 0x5578ef1891f0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedbd6b0 0x5578ef188cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/1733544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedb54d0 0x5578ef176d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/1733544192 >> 172.21.15.94:0/1733544192 conn(0x5578eedd24e0 msgr2=0x5578ef1853f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/1733544192 shutdown_connections 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.077+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/1733544192 wait complete. 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 -- start start 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 Processor -- start 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedb54d0 0x5578ef190140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedde390 0x5578ef1fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1aa8d0 con 0x5578eedbd6b0 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1aa200 con 0x5578eedb54d0 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec1a7440 con 0x5578eedde390 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4ce1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cf7bd7740 10 client.0 _ll_register_callbacks cb 0x7f4cdc0a65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4ce1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34730/0 (socket says 172.21.15.94:34730) 2024-04-25T14:02:35.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/2484668726 learned_addr learned my addr 172.21.15.94:0/2484668726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cea59c640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedb54d0 0x5578ef190140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.078+0000 7f4cead9d640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedde390 0x5578ef1fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedde390 msgr2=0x5578ef1fe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedde390 0x5578ef1fe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/2484668726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedb54d0 msgr2=0x5578ef190140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedb54d0 0x5578ef190140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 -- 172.21.15.94:0/2484668726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ef1fec30 con 0x5578eedbd6b0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4ce1d9b640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a6980 tx=0x7f4cdc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef514a33c67b11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc01d040 con 0x5578eedbd6b0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4cdc0a22d0 con 0x5578eedbd6b0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce4025070 con 0x5578eedbd6b0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc0ab030 con 0x5578eedbd6b0 2024-04-25T14:02:35.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cf7bd7740 10 client.22869 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578ef1ffe80 con 0x5578eedbd6b0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.079+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce4025070 con 0x5578eedbd6b0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4cdc0b7070 con 0x5578eedbd6b0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 10 client.22869 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 10 client.22869 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 20 client.22869 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed723a0 0x5578ef202760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 4 client.22869 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578ef184f00 con 0x5578eed723a0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.080+0000 7f4cea59c640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed723a0 0x5578ef202760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cd1ffb640 10 client.22869 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cea59c640 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed723a0 0x5578ef202760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=973c20007bc3e851 server_cookie=a634821a69c3e2d6 in_seq=0 out_seq=0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cd1ffb640 1 -- 172.21.15.94:0/2484668726 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578ef184f00 con 0x5578eed723a0 2024-04-25T14:02:35.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cd1ffb640 10 client.22869 handle_command_reply: tid=0 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 client.22869 shutdown 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 msgr2=0x5578ef190680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a6980 tx=0x7f4cdc00a340 comp rx=0 tx=0).stop 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 shutdown_connections 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x5578eed723a0 0x5578ef202760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578eedde390 0x5578ef1fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578eedbd6b0 0x5578ef190680 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 --2- 172.21.15.94:0/2484668726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578eedb54d0 0x5578ef190140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 >> 172.21.15.94:0/2484668726 conn(0x5578eedd24e0 msgr2=0x5578ef1853f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cd1ffb640 0 client.22869 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 shutdown_connections 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 1 -- 172.21.15.94:0/2484668726 wait complete. 2024-04-25T14:02:35.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.081+0000 7f4cf7bd7740 20 client.22869 trim_cache size 0 max 16384 2024-04-25T14:02:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 msgr2=0x7f4cd80b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 0x7f4cd80b2230 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x5578ef181860 tx=0x7f4ce4009a50 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 msgr2=0x7f4cec071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 0x7f4cec071fb0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c970 tx=0x7f4cdc00ce30 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 shutdown_connections 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4cd80afd20 0x7f4cd80b2230 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cec107920 0x7f4cec072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cec1027b0 0x7f4cec071fb0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 --2- 172.21.15.94:0/1567872359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cec101d70 0x7f4cec071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 >> 172.21.15.94:0/1567872359 conn(0x7f4cec0fd340 msgr2=0x7f4cec103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 shutdown_connections 2024-04-25T14:02:35.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.082+0000 7f4cf0d27640 1 -- 172.21.15.94:0/1567872359 wait complete. 2024-04-25T14:02:35.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:02:35.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 156501 19.04 MB/sec execute 317 sec latency 211.592 ms 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3703416091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a00 msgr2=0x7fdc18103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3703416091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a00 0x7fdc18103700 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009900 tx=0x7fdc0c02a410 comp rx=0 tx=0).stop 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3703416091 shutdown_connections 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3703416091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18108940 0x7fdc1810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3703416091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18103c40 0x7fdc18104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3703416091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18102a00 0x7fdc18103700 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3703416091 >> 172.21.15.94:0/3703416091 conn(0x7fdc180fdbf0 msgr2=0x7fdc181000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3703416091 shutdown_connections 2024-04-25T14:02:35.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3703416091 wait complete. 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 Processor -- start 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.208+0000 7fdc1edf3640 1 -- start start 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 0x7fdc1819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108940 0x7fdc181a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d3e0 con 0x7fdc18103c40 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d260 con 0x7fdc18102a00 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1edf3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1810d560 con 0x7fdc18108940 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34746/0 (socket says 172.21.15.94:34746) 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 -- 172.21.15.94:0/3425368772 learned_addr learned my addr 172.21.15.94:0/3425368772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 0x7fdc1819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1d390640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108940 0x7fdc181a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 -- 172.21.15.94:0/3425368772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108940 msgr2=0x7fdc181a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108940 0x7fdc181a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 -- 172.21.15.94:0/3425368772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 msgr2=0x7fdc1819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 0x7fdc1819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 -- 172.21.15.94:0/3425368772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc181a3f60 con 0x7fdc18103c40 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 0x7fdc1819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.209+0000 7fdc17fff640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d950 tx=0x7fdc0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.210+0000 7fdc15ffb640 1 -- 172.21.15.94:0/3425368772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04017030 con 0x7fdc18103c40 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.210+0000 7fdc15ffb640 1 -- 172.21.15.94:0/3425368772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc04004020 con 0x7fdc18103c40 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.210+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc181a4250 con 0x7fdc18103c40 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.210+0000 7fdc15ffb640 1 -- 172.21.15.94:0/3425368772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc040047a0 con 0x7fdc18103c40 2024-04-25T14:02:35.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.210+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc181a47f0 con 0x7fdc18103c40 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 Processor -- start 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 -- start start 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b770d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 0x559d4b771a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d4b6d4d80 con 0x559d4b3ce390 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559d4b68ea20 con 0x559d4b3ad6b0 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc25d79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d4b6d0d40 con 0x559d4b3a54d0 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34760/0 (socket says 172.21.15.94:34760) 2024-04-25T14:02:35.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc1d390640 1 -- 172.21.15.94:0/955909758 learned_addr learned my addr 172.21.15.94:0/955909758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc15ffb640 1 -- 172.21.15.94:0/3425368772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdc04009a30 con 0x7fdc18103c40 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.213+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b770d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 -- 172.21.15.94:0/955909758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 msgr2=0x559d4b770d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b770d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 -- 172.21.15.94:0/955909758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 msgr2=0x559d4b771a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc17fff640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 0x559d4b771a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 0x559d4b771a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 -- 172.21.15.94:0/955909758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4b699c30 con 0x559d4b3ce390 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc17fff640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 0x559d4b771a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1d390640 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fdc10003a50 tx=0x7fdc1000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150e5e74707d426d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc02ffd640 1 -- 172.21.15.94:0/955909758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004690 con 0x559d4b3ce390 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc02ffd640 1 -- 172.21.15.94:0/955909758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10004830 con 0x559d4b3ce390 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc02ffd640 1 -- 172.21.15.94:0/955909758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10016470 con 0x559d4b3ce390 2024-04-25T14:02:35.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc15ffb640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 0x7fdc080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 0x7fdc080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.214+0000 7fdc15ffb640 1 -- 172.21.15.94:0/3425368772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdc040de640 con 0x7fdc18103c40 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 -- 172.21.15.94:0/955909758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 msgr2=0x559d4b770810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 0x7fdc080b2080 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005ab0 tx=0x7fdc0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fdc10003a50 tx=0x7fdc1000b710 comp rx=0 tx=0).stop 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 -- 172.21.15.94:0/955909758 shutdown_connections 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ad6b0 0x559d4b771a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b770d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 --2- 172.21.15.94:0/955909758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ce390 0x559d4b770810 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 -- 172.21.15.94:0/955909758 >> 172.21.15.94:0/955909758 conn(0x559d4b3c24e0 msgr2=0x559d4b76c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 -- 172.21.15.94:0/955909758 shutdown_connections 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.215+0000 7fdc25d79740 1 -- 172.21.15.94:0/955909758 wait complete. 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 -- start start 2024-04-25T14:02:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 Processor -- start 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 0x559d4b781a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d4b772530 con 0x559d4b3ad6b0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559d4b65bdc0 con 0x559d4b3ce390 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d4b7726b0 con 0x559d4b3a54d0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc25d79740 10 client.0 _ll_register_callbacks cb 0x559d4b677600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34774/0 (socket says 172.21.15.94:34774) 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc17fff640 1 -- 172.21.15.94:0/2371877932 learned_addr learned my addr 172.21.15.94:0/2371877932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc1d390640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 0x559d4b781a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.216+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 -- 172.21.15.94:0/2371877932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 msgr2=0x559d4b77a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 -- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 msgr2=0x559d4b781a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 0x559d4b781a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 -- 172.21.15.94:0/2371877932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4b3e63f0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc1d390640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 0x559d4b781a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc17fff640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fdc040a6a10 tx=0x7fdc040a62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd64f24ea74889b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc040aae30 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdc040041d0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc10004830 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc040af8e0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 10 client.22881 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d4b4b07d0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10004830 con 0x559d4b3ad6b0 2024-04-25T14:02:35.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdc040af4e0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 10 client.22881 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 10 client.22881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 20 client.22881 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.217+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b38cf60 0x559d4b788170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.218+0000 7fdc25d79740 4 client.22881 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.218+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d4b6e2990 con 0x559d4b38cf60 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.218+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b38cf60 0x559d4b788170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.218+0000 7fdbf7fff640 10 client.22881 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.218+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b38cf60 0x559d4b788170 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64e2bf1131529da4 server_cookie=11a214e9098d9a17 in_seq=0 out_seq=0 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdbf7fff640 1 -- 172.21.15.94:0/2371877932 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d4b6e2990 con 0x559d4b38cf60 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdbf7fff640 10 client.22881 handle_command_reply: tid=0 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 client.22881 shutdown 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 msgr2=0x559d4b77ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fdc040a6a10 tx=0x7fdc040a62b0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 shutdown_connections 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b38cf60 0x559d4b788170 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b3ce390 0x559d4b781a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdbf7fff640 0 client.22881 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b77ab70 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 --2- 172.21.15.94:0/2371877932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 >> 172.21.15.94:0/2371877932 conn(0x559d4b3c24e0 msgr2=0x559d4b76ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 shutdown_connections 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 1 -- 172.21.15.94:0/2371877932 wait complete. 2024-04-25T14:02:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.219+0000 7fdc25d79740 20 client.22881 trim_cache size 0 max 16384 2024-04-25T14:02:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 Processor -- start 2024-04-25T14:02:35.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 -- start start 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b775390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b781980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ad760 con 0x559d4b3ad6b0 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181af2a0 con 0x559d4b38cf60 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.233+0000 7fdc25d79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ade30 con 0x559d4b3a54d0 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34782/0 (socket says 172.21.15.94:34782) 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 -- 172.21.15.94:0/1609180681 learned_addr learned my addr 172.21.15.94:0/1609180681 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc17fff640 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b781980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b775390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 -- 172.21.15.94:0/1609180681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 msgr2=0x559d4b781980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b781980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 -- 172.21.15.94:0/1609180681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 msgr2=0x559d4b775390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b775390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 -- 172.21.15.94:0/1609180681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4b772410 con 0x559d4b3ad6b0 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc1d390640 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fdc10003a20 tx=0x7fdc10005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b76dc21e301b7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc027fc640 1 -- 172.21.15.94:0/1609180681 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004690 con 0x559d4b3ad6b0 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc027fc640 1 -- 172.21.15.94:0/1609180681 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10004830 con 0x559d4b3ad6b0 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.234+0000 7fdc027fc640 1 -- 172.21.15.94:0/1609180681 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10016870 con 0x559d4b3ad6b0 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- 172.21.15.94:0/1609180681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 msgr2=0x559d4b7758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fdc10003a20 tx=0x7fdc10005990 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- 172.21.15.94:0/1609180681 shutdown_connections 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7758d0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b781980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 --2- 172.21.15.94:0/1609180681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b775390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- 172.21.15.94:0/1609180681 >> 172.21.15.94:0/1609180681 conn(0x559d4b3c24e0 msgr2=0x559d4b778cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- 172.21.15.94:0/1609180681 shutdown_connections 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- 172.21.15.94:0/1609180681 wait complete. 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.235+0000 7fdc25d79740 1 -- start start 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 Processor -- start 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b77e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181aebd0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ae500 con 0x559d4b38cf60 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc181ab740 con 0x559d4b3a54d0 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc25d79740 10 client.0 _ll_register_callbacks cb 0x7fdc040046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34784/0 (socket says 172.21.15.94:34784) 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 -- 172.21.15.94:0/3013770080 learned_addr learned my addr 172.21.15.94:0/3013770080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc17fff640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b77e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 -- 172.21.15.94:0/3013770080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 msgr2=0x559d4b77ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 -- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 msgr2=0x559d4b77e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b77e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.236+0000 7fdc1d390640 1 -- 172.21.15.94:0/3013770080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4b7ee9e0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc1d390640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fdc10004010 tx=0x7fdc10004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a81d592b9ec494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004690 con 0x559d4b3ad6b0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdc1002f070 con 0x559d4b3ad6b0 2024-04-25T14:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc10004830 con 0x559d4b3ad6b0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc10004f60 con 0x559d4b3ad6b0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 10 client.22887 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d4b7efac0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10004830 con 0x559d4b3ad6b0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdc10016ca0 con 0x559d4b3ad6b0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 10 client.22887 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 10 client.22887 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 20 client.22887 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b3ce390 0x559d4b7f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 4 client.22887 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.237+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d4b6dc800 con 0x559d4b3ce390 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b3ce390 0x559d4b7f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdbf7fff640 10 client.22887 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc1cb8f640 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b3ce390 0x559d4b7f2300 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79771aaf8f174a11 server_cookie=e8adf91f48c8a0c4 in_seq=0 out_seq=0 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdbf7fff640 1 -- 172.21.15.94:0/3013770080 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d4b6dc800 con 0x559d4b3ce390 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdbf7fff640 10 client.22887 handle_command_reply: tid=0 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 client.22887 shutdown 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 msgr2=0x559d4b7ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fdc10004010 tx=0x7fdc10004490 comp rx=0 tx=0).stop 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 shutdown_connections 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x559d4b3ce390 0x559d4b7f2300 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d4b3ad6b0 0x559d4b7ee4a0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdbf7fff640 0 client.22887 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d4b3a54d0 0x559d4b77ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 --2- 172.21.15.94:0/3013770080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4b38cf60 0x559d4b77e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.238+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 >> 172.21.15.94:0/3013770080 conn(0x559d4b3c24e0 msgr2=0x559d4b778cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 shutdown_connections 2024-04-25T14:02:35.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc25d79740 1 -- 172.21.15.94:0/3013770080 wait complete. 2024-04-25T14:02:35.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc25d79740 20 client.22887 trim_cache size 0 max 16384 2024-04-25T14:02:35.246 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 msgr2=0x7fdc080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 0x7fdc080b2080 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005ab0 tx=0x7fdc0c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 msgr2=0x7fdc1819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d950 tx=0x7fdc0400de10 comp rx=0 tx=0).stop 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 shutdown_connections 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdc080afb70 0x7fdc080b2080 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc18108940 0x7fdc181a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc18103c40 0x7fdc1819c960 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 --2- 172.21.15.94:0/3425368772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc18102a00 0x7fdc1819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 >> 172.21.15.94:0/3425368772 conn(0x7fdc180fdbf0 msgr2=0x7fdc18104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 shutdown_connections 2024-04-25T14:02:35.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.239+0000 7fdc1edf3640 1 -- 172.21.15.94:0/3425368772 wait complete. 2024-04-25T14:02:35.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:02:35.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 -- 172.21.15.94:0/3300969951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710103960 msgr2=0x7f7710103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 --2- 172.21.15.94:0/3300969951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710103960 0x7f7710103dd0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009900 tx=0x7f76f802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 -- 172.21.15.94:0/3300969951 shutdown_connections 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 --2- 172.21.15.94:0/3300969951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710104460 0x7f771010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 --2- 172.21.15.94:0/3300969951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710103960 0x7f7710103dd0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 --2- 172.21.15.94:0/3300969951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 -- 172.21.15.94:0/3300969951 >> 172.21.15.94:0/3300969951 conn(0x7f77100fd340 msgr2=0x7f77100ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 -- 172.21.15.94:0/3300969951 shutdown_connections 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 -- 172.21.15.94:0/3300969951 wait complete. 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.389+0000 7f77171a8640 1 Processor -- start 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 -- start start 2024-04-25T14:02:35.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710103960 0x7f771019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f771010e570 con 0x7f7710104460 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f771010e3f0 con 0x7f7710103960 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f77171a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f771010e6f0 con 0x7f7710102f80 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7714f44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34808/0 (socket says 172.21.15.94:34808) 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 -- 172.21.15.94:0/101786897 learned_addr learned my addr 172.21.15.94:0/101786897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f770ffff640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710103960 0x7f771019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 -- 172.21.15.94:0/101786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 msgr2=0x7f7710102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 -- 172.21.15.94:0/101786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710103960 msgr2=0x7f771019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710103960 0x7f771019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 -- 172.21.15.94:0/101786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77101a1d60 con 0x7f7710104460 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7714f44640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.390+0000 7f7715745640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f770800eb10 tx=0x7f770800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.391+0000 7f770dffb640 1 -- 172.21.15.94:0/101786897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708004b20 con 0x7f7710104460 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.391+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101a1ff0 con 0x7f7710104460 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.391+0000 7f770dffb640 1 -- 172.21.15.94:0/101786897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7708004cc0 con 0x7f7710104460 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.391+0000 7f770dffb640 1 -- 172.21.15.94:0/101786897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708010690 con 0x7f7710104460 2024-04-25T14:02:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.391+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a24b0 con 0x7f7710104460 2024-04-25T14:02:35.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.393+0000 7f770dffb640 1 -- 172.21.15.94:0/101786897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7708005000 con 0x7f7710104460 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.393+0000 7f771e054740 1 Processor -- start 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 -- start start 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e74191f390 0x55e741cc1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e741c25d80 con 0x55e7418f64d0 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e741bdfa20 con 0x55e7418fe6b0 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f771e054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e741c21d40 con 0x55e74191f390 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34814/0 (socket says 172.21.15.94:34814) 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 -- 172.21.15.94:0/46006012 learned_addr learned my addr 172.21.15.94:0/46006012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f770ffff640 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f770ffff640 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e74191f390 0x55e741cc1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 -- 172.21.15.94:0/46006012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e74191f390 msgr2=0x55e741cc1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e74191f390 0x55e741cc1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 -- 172.21.15.94:0/46006012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 msgr2=0x55e741cc2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cc2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 -- 172.21.15.94:0/46006012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e741beac30 con 0x55e7418f64d0 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f770dffb640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 0x7f76fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f7714f44640 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f77000114f0 tx=0x7f77000119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14d927a92645dae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f770ffff640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 0x7f76fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.394+0000 7f770dffb640 1 -- 172.21.15.94:0/101786897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f77080df260 con 0x7f7710104460 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f7706ffd640 1 -- 172.21.15.94:0/46006012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001e070 con 0x55e7418f64d0 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f7706ffd640 1 -- 172.21.15.94:0/46006012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77000035f0 con 0x55e7418f64d0 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f7706ffd640 1 -- 172.21.15.94:0/46006012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770000f800 con 0x55e7418f64d0 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f771e054740 1 -- 172.21.15.94:0/46006012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 msgr2=0x55e741cc1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f770ffff640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 0x7f76fc0b2000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004770 tx=0x7f76f8005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.395+0000 7f771e054740 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f77000114f0 tx=0x7f77000119b0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 -- 172.21.15.94:0/46006012 shutdown_connections 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cc2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7418f64d0 0x55e741cc1d90 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- 172.21.15.94:0/46006012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e74191f390 0x55e741cc1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 -- 172.21.15.94:0/46006012 >> 172.21.15.94:0/46006012 conn(0x55e7419134e0 msgr2=0x55e741cbd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 -- 172.21.15.94:0/46006012 shutdown_connections 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 -- 172.21.15.94:0/46006012 wait complete. 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 -- start start 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 Processor -- start 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 0x55e741ccb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418fe6b0 0x55e741ccba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.396+0000 7f771e054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e741cc3530 con 0x55e74191f390 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e741bacdc0 con 0x55e7418f64d0 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e741cc36b0 con 0x55e7418fe6b0 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 10 client.0 _ll_register_callbacks cb 0x55e741bc8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34824/0 (socket says 172.21.15.94:34824) 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7714f44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 0x55e741ccb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 -- 172.21.15.94:0/4217905592 learned_addr learned my addr 172.21.15.94:0/4217905592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f770ffff640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418fe6b0 0x55e741ccba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 -- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418fe6b0 msgr2=0x55e741ccba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418fe6b0 0x55e741ccba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 -- 172.21.15.94:0/4217905592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 msgr2=0x55e741ccb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 0x55e741ccb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 -- 172.21.15.94:0/4217905592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7419373f0 con 0x55e74191f390 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7714f44640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 0x55e741ccb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f7715745640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f77080049d0 tx=0x7f77080a6e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2d084e64464b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f76effff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080c1060 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f77080b0070 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77000035f0 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f77080a76a0 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 10 client.22899 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.397+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e741a017d0 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77000035f0 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7708082ce0 con 0x55e74191f390 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 10 client.22899 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 10 client.22899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 20 client.22899 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418ddf60 0x55e741cd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 4 client.22899 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e741c33990 con 0x55e7418ddf60 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f7714f44640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418ddf60 0x55e741cd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f76effff640 10 client.22899 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.398+0000 7f7714f44640 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418ddf60 0x55e741cd9220 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deab58aa0bea55ae server_cookie=c64a528959bb527c in_seq=0 out_seq=0 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f76effff640 1 -- 172.21.15.94:0/4217905592 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e741c33990 con 0x55e7418ddf60 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f76effff640 10 client.22899 handle_command_reply: tid=0 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 client.22899 shutdown 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 msgr2=0x55e741cd29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f77080049d0 tx=0x7f77080a6e10 comp rx=0 tx=0).stop 2024-04-25T14:02:35.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 shutdown_connections 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418ddf60 0x55e741cd9220 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd29a0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418fe6b0 0x55e741ccba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 --2- 172.21.15.94:0/4217905592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418f64d0 0x55e741ccb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f76effff640 0 client.22899 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 >> 172.21.15.94:0/4217905592 conn(0x55e7419134e0 msgr2=0x55e741cbff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 shutdown_connections 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.399+0000 7f771e054740 1 -- 172.21.15.94:0/4217905592 wait complete. 2024-04-25T14:02:35.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.400+0000 7f771e054740 20 client.22899 trim_cache size 0 max 16384 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 Processor -- start 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 -- start start 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741cca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77101ab4b0 con 0x55e74191f390 2024-04-25T14:02:35.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77101acff0 con 0x55e7418fe6b0 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f771e054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77101abb80 con 0x55e7418f64d0 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34832/0 (socket says 172.21.15.94:34832) 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 -- 172.21.15.94:0/3268412813 learned_addr learned my addr 172.21.15.94:0/3268412813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7714f44640 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741cca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f770ffff640 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 -- 172.21.15.94:0/3268412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 msgr2=0x55e741cca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741cca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 -- 172.21.15.94:0/3268412813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 msgr2=0x55e741cd28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.414+0000 7f7715745640 1 -- 172.21.15.94:0/3268412813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e741cd2e00 con 0x55e74191f390 2024-04-25T14:02:35.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.415+0000 7f7715745640 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f77080dcd10 tx=0x7f77080902c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1739801ea32f3f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.415+0000 7f77067fc640 1 -- 172.21.15.94:0/3268412813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080a6900 con 0x55e74191f390 2024-04-25T14:02:35.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.415+0000 7f77067fc640 1 -- 172.21.15.94:0/3268412813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77080de030 con 0x55e74191f390 2024-04-25T14:02:35.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.415+0000 7f77067fc640 1 -- 172.21.15.94:0/3268412813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080b0070 con 0x55e74191f390 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- 172.21.15.94:0/3268412813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 msgr2=0x55e741cbc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f77080dcd10 tx=0x7f77080902c0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- 172.21.15.94:0/3268412813 shutdown_connections 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cbc9e0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 --2- 172.21.15.94:0/3268412813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741cca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- 172.21.15.94:0/3268412813 >> 172.21.15.94:0/3268412813 conn(0x55e7419134e0 msgr2=0x55e741cc9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- 172.21.15.94:0/3268412813 shutdown_connections 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- 172.21.15.94:0/3268412813 wait complete. 2024-04-25T14:02:35.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.416+0000 7f771e054740 1 -- start start 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 Processor -- start 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741ccfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77101ac920 con 0x55e74191f390 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77101ac250 con 0x55e7418fe6b0 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77101a9490 con 0x55e7418f64d0 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f771e054740 10 client.0 _ll_register_callbacks cb 0x7f7708005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f7714f44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741ccfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f7715745640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34838/0 (socket says 172.21.15.94:34838) 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f7715745640 1 -- 172.21.15.94:0/2886223678 learned_addr learned my addr 172.21.15.94:0/2886223678 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.417+0000 7f770ffff640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 -- 172.21.15.94:0/2886223678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 msgr2=0x55e741cd02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 -- 172.21.15.94:0/2886223678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 msgr2=0x55e741cd1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 -- 172.21.15.94:0/2886223678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e741cd1740 con 0x55e7418f64d0 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7715745640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f7714f44640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741ccfd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f77000114c0 tx=0x7f7700003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f91983dc11ae70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f76effff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001e040 con 0x55e7418f64d0 2024-04-25T14:02:35.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7700003da0 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77080de030 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7700039050 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f771e054740 10 client.31954 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e741cd2a30 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.418+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77080de030 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f770002b360 con 0x55e7418f64d0 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 10 client.31954 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 10 client.31954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 20 client.31954 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418b33a0 0x55e741d47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 4 client.31954 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:35.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e741c2d800 con 0x55e7418b33a0 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f770ffff640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418b33a0 0x55e741d47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f76effff640 10 client.31954 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.419+0000 7f770ffff640 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418b33a0 0x55e741d47da0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c79175d51f5b6bc4 server_cookie=c193eead27e8dc0f in_seq=0 out_seq=0 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f76effff640 1 -- 172.21.15.94:0/2886223678 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e741c2d800 con 0x55e7418b33a0 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f76effff640 10 client.31954 handle_command_reply: tid=0 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 client.31954 shutdown 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 msgr2=0x55e741ccfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741ccfd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f77000114c0 tx=0x7f7700003680 comp rx=0 tx=0).stop 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 shutdown_connections 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55e7418b33a0 0x55e741d47da0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e74191f390 0x55e741cd1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f76effff640 0 client.31954 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7418fe6b0 0x55e741cd02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 --2- 172.21.15.94:0/2886223678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7418f64d0 0x55e741ccfd80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 >> 172.21.15.94:0/2886223678 conn(0x55e7419134e0 msgr2=0x55e741cbfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 shutdown_connections 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 1 -- 172.21.15.94:0/2886223678 wait complete. 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f771e054740 20 client.31954 trim_cache size 0 max 16384 2024-04-25T14:02:35.427 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 msgr2=0x7f76fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 0x7f76fc0b2000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004770 tx=0x7f76f8005d70 comp rx=0 tx=0).stop 2024-04-25T14:02:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 msgr2=0x7f77101a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.420+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f770800eb10 tx=0x7f770800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 shutdown_connections 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f76fc0afaf0 0x7f76fc0b2000 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710104460 0x7f77101a1540 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7710103960 0x7f771019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 --2- 172.21.15.94:0/101786897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7710102f80 0x7f7710102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 >> 172.21.15.94:0/101786897 conn(0x7f77100fd340 msgr2=0x7f7710104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 shutdown_connections 2024-04-25T14:02:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.421+0000 7f77171a8640 1 -- 172.21.15.94:0/101786897 wait complete. 2024-04-25T14:02:35.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/763266193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:35.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:02:35.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 -- 172.21.15.94:0/2351507808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 msgr2=0x7fce2810bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2351507808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce2810bcc0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fce18009900 tx=0x7fce1802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 -- 172.21.15.94:0/2351507808 shutdown_connections 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2351507808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce2810bcc0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2351507808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce28109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2351507808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce28106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 -- 172.21.15.94:0/2351507808 >> 172.21.15.94:0/2351507808 conn(0x7fce280fdc00 msgr2=0x7fce28100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 -- 172.21.15.94:0/2351507808 shutdown_connections 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.546+0000 7fce2ea27640 1 -- 172.21.15.94:0/2351507808 wait complete. 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 Processor -- start 2024-04-25T14:02:35.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 -- start start 2024-04-25T14:02:35.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce2819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce2819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce2810f620 con 0x7fce281098a0 2024-04-25T14:02:35.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce2810f4a0 con 0x7fce281045e0 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2ea27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce2810f7a0 con 0x7fce28106f40 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2cfc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce27fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce2819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2cfc4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34850/0 (socket says 172.21.15.94:34850) 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce2cfc4640 1 -- 172.21.15.94:0/2999531480 learned_addr learned my addr 172.21.15.94:0/2999531480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.547+0000 7fce277fe640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce2819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 -- 172.21.15.94:0/2999531480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 msgr2=0x7fce2819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce2819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 -- 172.21.15.94:0/2999531480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 msgr2=0x7fce2819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce2819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 -- 172.21.15.94:0/2999531480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce281a1d40 con 0x7fce281098a0 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce277fe640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce2819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce27fff640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce2819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:35.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2cfc4640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fce18009230 tx=0x7fce180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce257fa640 1 -- 172.21.15.94:0/2999531480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1803f030 con 0x7fce281098a0 2024-04-25T14:02:35.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce281a1fd0 con 0x7fce281098a0 2024-04-25T14:02:35.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce257fa640 1 -- 172.21.15.94:0/2999531480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce180318f0 con 0x7fce281098a0 2024-04-25T14:02:35.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce257fa640 1 -- 172.21.15.94:0/2999531480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18035070 con 0x7fce281098a0 2024-04-25T14:02:35.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.548+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce281a2430 con 0x7fce281098a0 2024-04-25T14:02:35.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.550+0000 7fce257fa640 1 -- 172.21.15.94:0/2999531480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fce18042030 con 0x7fce281098a0 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 Processor -- start 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 -- start start 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b792390 0x56112bb38c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56112ba98d80 con 0x56112b7694d0 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56112ba52a20 con 0x56112b792390 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce359b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56112ba94d40 con 0x56112b7716b0 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34866/0 (socket says 172.21.15.94:34866) 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 -- 172.21.15.94:0/3844218425 learned_addr learned my addr 172.21.15.94:0/3844218425 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce277fe640 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b792390 0x56112bb38c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce257fa640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 0x7fce100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce277fe640 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 -- 172.21.15.94:0/3844218425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 msgr2=0x56112bb39e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb39e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 -- 172.21.15.94:0/3844218425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b792390 msgr2=0x56112bb38c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b792390 0x56112bb38c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce27fff640 1 -- 172.21.15.94:0/3844218425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112ba5dc30 con 0x56112b7694d0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.551+0000 7fce2cfc4640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 0x7fce100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.552+0000 7fce27fff640 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fce20011820 tx=0x7fce20011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bfe7463ed2532d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.552+0000 7fce257fa640 1 -- 172.21.15.94:0/2999531480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fce180ff1b0 con 0x7fce281098a0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.552+0000 7fce0e7fc640 1 -- 172.21.15.94:0/3844218425 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20011220 con 0x56112b7694d0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.552+0000 7fce0e7fc640 1 -- 172.21.15.94:0/3844218425 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce20019430 con 0x56112b7694d0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.552+0000 7fce0e7fc640 1 -- 172.21.15.94:0/3844218425 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20010690 con 0x56112b7694d0 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- 172.21.15.94:0/3844218425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 msgr2=0x56112bb391a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce2cfc4640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 0x7fce100b2000 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fce180c4940 tx=0x7fce18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fce20011820 tx=0x7fce20011ce0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- 172.21.15.94:0/3844218425 shutdown_connections 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb39e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb391a0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 --2- 172.21.15.94:0/3844218425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b792390 0x56112bb38c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- 172.21.15.94:0/3844218425 >> 172.21.15.94:0/3844218425 conn(0x56112b7864e0 msgr2=0x56112bb30020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- 172.21.15.94:0/3844218425 shutdown_connections 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- 172.21.15.94:0/3844218425 wait complete. 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 -- start start 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.553+0000 7fce359b9740 1 Processor -- start 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56112bb3a940 con 0x56112b792390 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56112ba1fdc0 con 0x56112b7694d0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56112bb3aac0 con 0x56112b7716b0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce359b9740 10 client.0 _ll_register_callbacks cb 0x56112ba3b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34872/0 (socket says 172.21.15.94:34872) 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 -- 172.21.15.94:0/592981101 learned_addr learned my addr 172.21.15.94:0/592981101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce27fff640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 -- 172.21.15.94:0/592981101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 msgr2=0x56112bb42e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb42e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 -- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 msgr2=0x56112bb428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 -- 172.21.15.94:0/592981101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112b7aa3f0 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce277fe640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fce1c013df0 tx=0x7fce1c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1546af68d231c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce077fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c011e80 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.554+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fce1c005020 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce20019430 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fce1c005220 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 10 client.22917 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56112b8747d0 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce20019430 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fce1c022470 con 0x56112b792390 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 10 client.22917 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 10 client.22917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 20 client.22917 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b750f60 0x56112bb50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 4 client.22917 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56112baa6990 con 0x56112b750f60 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.555+0000 7fce27fff640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b750f60 0x56112bb50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.556+0000 7fce077fe640 10 client.22917 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.556+0000 7fce27fff640 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b750f60 0x56112bb50540 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e4c9c66b62fb492 server_cookie=fe3525a12da25c13 in_seq=0 out_seq=0 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce077fe640 1 -- 172.21.15.94:0/592981101 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56112baa6990 con 0x56112b750f60 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce077fe640 10 client.22917 handle_command_reply: tid=0 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 client.22917 shutdown 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 msgr2=0x56112bb49d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fce1c013df0 tx=0x7fce1c015040 comp rx=0 tx=0).stop 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 shutdown_connections 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b750f60 0x56112bb50540 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b792390 0x56112bb49d80 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b7716b0 0x56112bb42e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce077fe640 0 client.22917 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 --2- 172.21.15.94:0/592981101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 >> 172.21.15.94:0/592981101 conn(0x56112b7864e0 msgr2=0x56112bb32f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 shutdown_connections 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 1 -- 172.21.15.94:0/592981101 wait complete. 2024-04-25T14:02:35.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.557+0000 7fce359b9740 20 client.22917 trim_cache size 0 max 16384 2024-04-25T14:02:35.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.571+0000 7fce359b9740 1 Processor -- start 2024-04-25T14:02:35.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.571+0000 7fce359b9740 1 -- start start 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7716b0 0x56112bb49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb41670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce281a9380 con 0x56112b7694d0 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce281aaec0 con 0x56112b7716b0 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce359b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce281a9a50 con 0x56112b792390 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34878/0 (socket says 172.21.15.94:34878) 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 -- 172.21.15.94:0/2407469237 learned_addr learned my addr 172.21.15.94:0/2407469237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce27fff640 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7716b0 0x56112bb49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb41670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 -- 172.21.15.94:0/2407469237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 msgr2=0x56112bb41670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb41670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 -- 172.21.15.94:0/2407469237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7716b0 msgr2=0x56112bb49d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7716b0 0x56112bb49d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 -- 172.21.15.94:0/2407469237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112bb30480 con 0x56112b7694d0 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.572+0000 7fce277fe640 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006010 tx=0x7fce1c013ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411dd02494222171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.573+0000 7fce0dffb640 1 -- 172.21.15.94:0/2407469237 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c032030 con 0x56112b7694d0 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.573+0000 7fce0dffb640 1 -- 172.21.15.94:0/2407469237 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce1c011e80 con 0x56112b7694d0 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.573+0000 7fce0dffb640 1 -- 172.21.15.94:0/2407469237 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c022630 con 0x56112b7694d0 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.573+0000 7fce359b9740 1 -- 172.21.15.94:0/2407469237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 msgr2=0x56112bb41130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.573+0000 7fce359b9740 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006010 tx=0x7fce1c013ad0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- 172.21.15.94:0/2407469237 shutdown_connections 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb41670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7716b0 0x56112bb49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- 172.21.15.94:0/2407469237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7694d0 0x56112bb41130 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- 172.21.15.94:0/2407469237 >> 172.21.15.94:0/2407469237 conn(0x56112b7864e0 msgr2=0x56112bb3d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- 172.21.15.94:0/2407469237 shutdown_connections 2024-04-25T14:02:35.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- 172.21.15.94:0/2407469237 wait complete. 2024-04-25T14:02:35.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- start start 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 Processor -- start 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb47af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb48f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce281aa7f0 con 0x56112b7716b0 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce281aa120 con 0x56112b7694d0 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.574+0000 7fce359b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce281a7360 con 0x56112b792390 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce359b9740 10 client.0 _ll_register_callbacks cb 0x7fce1c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34882/0 (socket says 172.21.15.94:34882) 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 -- 172.21.15.94:0/352049829 learned_addr learned my addr 172.21.15.94:0/352049829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce277fe640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb48f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce277fe640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb47af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 -- 172.21.15.94:0/352049829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 msgr2=0x56112bb48f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb48f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 -- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 msgr2=0x56112bb47af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb47af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 -- 172.21.15.94:0/352049829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112bb49520 con 0x56112b7716b0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce277fe640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb47af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce27fff640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fce20009a00 tx=0x7fce20019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb43b40506e960c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce077fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20019e60 con 0x56112b7716b0 2024-04-25T14:02:35.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fce20011040 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56112baa0800 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fce20011240 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce359b9740 10 client.22923 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.575+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56112bbb8da0 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56112baa0800 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fce20003cd0 con 0x56112b7716b0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 10 client.22923 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 10 client.22923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 20 client.22923 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b7263a0 0x56112bbbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 4 client.22923 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56112ba1b610 con 0x56112b7263a0 2024-04-25T14:02:35.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.576+0000 7fce277fe640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b7263a0 0x56112bbbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce277fe640 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b7263a0 0x56112bbbb5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbcf80f938fe6695 server_cookie=bacec20a4d124358 in_seq=0 out_seq=0 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce077fe640 10 client.22923 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce077fe640 1 -- 172.21.15.94:0/352049829 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56112ba1b610 con 0x56112b7263a0 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce077fe640 10 client.22923 handle_command_reply: tid=0 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 client.22923 shutdown 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 msgr2=0x56112bb48030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fce20009a00 tx=0x7fce20019570 comp rx=0 tx=0).stop 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 shutdown_connections 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56112b7263a0 0x56112bbbb5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112b792390 0x56112bb48f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce077fe640 0 client.22923 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56112b7716b0 0x56112bb48030 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 --2- 172.21.15.94:0/352049829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56112b7694d0 0x56112bb47af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 >> 172.21.15.94:0/352049829 conn(0x56112b7864e0 msgr2=0x56112bb41130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 shutdown_connections 2024-04-25T14:02:35.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 1 -- 172.21.15.94:0/352049829 wait complete. 2024-04-25T14:02:35.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.577+0000 7fce359b9740 20 client.22923 trim_cache size 0 max 16384 2024-04-25T14:02:35.584 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 msgr2=0x7fce100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 0x7fce100b2000 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fce180c4940 tx=0x7fce18009260 comp rx=0 tx=0).stop 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 msgr2=0x7fce281a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fce18009230 tx=0x7fce180042f0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 shutdown_connections 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fce100afaf0 0x7fce100b2000 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281098a0 0x7fce281a16d0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce28106f40 0x7fce2819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 --2- 172.21.15.94:0/2999531480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce281045e0 0x7fce2819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 >> 172.21.15.94:0/2999531480 conn(0x7fce280fdc00 msgr2=0x7fce281050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 shutdown_connections 2024-04-25T14:02:35.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.578+0000 7fce2ea27640 1 -- 172.21.15.94:0/2999531480 wait complete. 2024-04-25T14:02:35.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:02:35.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/763266193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.703+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/1277253092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8103c20 msgr2=0x7fa6f8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.703+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/1277253092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8103c20 0x7fa6f8104030 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009900 tx=0x7fa6e802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.703+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/1277253092 shutdown_connections 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/1277253092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8108920 0x7fa6f810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/1277253092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8103c20 0x7fa6f8104030 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/1277253092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/1277253092 >> 172.21.15.94:0/1277253092 conn(0x7fa6f80fdbd0 msgr2=0x7fa6f8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/1277253092 shutdown_connections 2024-04-25T14:02:35.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/1277253092 wait complete. 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 Processor -- start 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.704+0000 7fa6ff8f9640 1 -- start start 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8103c20 0x7fa6f8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f810e510 con 0x7fa6f8108920 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f810e390 con 0x7fa6f81029e0 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f810e690 con 0x7fa6f8103c20 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34896/0 (socket says 172.21.15.94:34896) 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 -- 172.21.15.94:0/702733542 learned_addr learned my addr 172.21.15.94:0/702733542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f8199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46088/0 (socket says 172.21.15.94:46088) 2024-04-25T14:02:35.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fce94640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8103c20 0x7fa6f8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 -- 172.21.15.94:0/702733542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8103c20 msgr2=0x7fa6f8199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8103c20 0x7fa6f8199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 -- 172.21.15.94:0/702733542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 msgr2=0x7fa6f8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 -- 172.21.15.94:0/702733542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f81a0f70 con 0x7fa6f8108920 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6fde96640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000eb10 tx=0x7fa6f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ee7fc640 1 -- 172.21.15.94:0/702733542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0004b20 con 0x7fa6f8108920 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ee7fc640 1 -- 172.21.15.94:0/702733542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f0004cc0 con 0x7fa6f8108920 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f81a1260 con 0x7fa6f8108920 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ee7fc640 1 -- 172.21.15.94:0/702733542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0010670 con 0x7fa6f8108920 2024-04-25T14:02:35.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.705+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81a17b0 con 0x7fa6f8108920 2024-04-25T14:02:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa6ee7fc640 1 -- 172.21.15.94:0/702733542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa6f0005020 con 0x7fa6f8108920 2024-04-25T14:02:35.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 Processor -- start 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 -- start start 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a40e7cd80 con 0x558a40b4d4d0 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a40e36a20 con 0x558a40b76390 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.708+0000 7fa7067a9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a40e78d40 con 0x558a40b556b0 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34898/0 (socket says 172.21.15.94:34898) 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 -- 172.21.15.94:0/2472356619 learned_addr learned my addr 172.21.15.94:0/2472356619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6ee7fc640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 0x7fa6e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 -- 172.21.15.94:0/2472356619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 msgr2=0x558a40f18b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f18b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 -- 172.21.15.94:0/2472356619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 msgr2=0x558a40f19ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f19ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 -- 172.21.15.94:0/2472356619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a40e41c30 con 0x558a40b4d4d0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fde96640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 0x7fa6e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009c60 tx=0x7fa6f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb6da619e9e1518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6ee7fc640 1 -- 172.21.15.94:0/702733542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa6f00df230 con 0x7fa6f8108920 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6e37fe640 1 -- 172.21.15.94:0/2472356619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4004220 con 0x558a40b4d4d0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6e37fe640 1 -- 172.21.15.94:0/2472356619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40043c0 con 0x558a40b4d4d0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.709+0000 7fa6e37fe640 1 -- 172.21.15.94:0/2472356619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4004ce0 con 0x558a40b4d4d0 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 -- 172.21.15.94:0/2472356619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 msgr2=0x558a40f18400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa6fde96640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 0x7fa6e40b2000 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0015020 tx=0x7fa6f0013330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009c60 tx=0x7fa6f4011810 comp rx=0 tx=0).stop 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 -- 172.21.15.94:0/2472356619 shutdown_connections 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f18b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f18400 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2472356619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 -- 172.21.15.94:0/2472356619 >> 172.21.15.94:0/2472356619 conn(0x558a40b6a4e0 msgr2=0x558a40f14020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 -- 172.21.15.94:0/2472356619 shutdown_connections 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.710+0000 7fa7067a9740 1 -- 172.21.15.94:0/2472356619 wait complete. 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 -- start start 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 Processor -- start 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f26e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 0x558a40f2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a40f24b90 con 0x558a40b76390 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a40e03dc0 con 0x558a40b4d4d0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a40f24d10 con 0x558a40b556b0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa7067a9740 10 client.0 _ll_register_callbacks cb 0x558a40e1f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa6fce94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 0x558a40f2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa6fce94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa6fce94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f268f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46112/0 (socket says 172.21.15.94:46112) 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa6fce94640 1 -- 172.21.15.94:0/840269020 learned_addr learned my addr 172.21.15.94:0/840269020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.711+0000 7fa6fd695640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f26e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 -- 172.21.15.94:0/840269020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 msgr2=0x558a40f26e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f26e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 -- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 msgr2=0x558a40f268f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f268f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 -- 172.21.15.94:0/840269020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a40b8e3f0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fd695640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f26e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6fce94640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 0x558a40f2dd90 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80096c0 tx=0x7fa6e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834651216dd9a8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8035040 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa6e8004320 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f40043c0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa6e80035f0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 10 client.22935 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a40c587d0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40043c0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa6e803d6f0 con 0x558a40b76390 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 10 client.22935 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 10 client.22935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 20 client.22935 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.712+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b34f60 0x558a40f34560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.713+0000 7fa7067a9740 4 client.22935 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.713+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a40e8a990 con 0x558a40b34f60 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.713+0000 7fa6fd695640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b34f60 0x558a40f34560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.713+0000 7fa6e0ff9640 10 client.22935 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:35.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.713+0000 7fa6fd695640 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b34f60 0x558a40f34560 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3d987fccda665b server_cookie=62830557859cff16 in_seq=0 out_seq=0 2024-04-25T14:02:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/840269020 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a40e8a990 con 0x558a40b34f60 2024-04-25T14:02:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa6e0ff9640 10 client.22935 handle_command_reply: tid=0 2024-04-25T14:02:35.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 client.22935 shutdown 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 msgr2=0x558a40f2dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 0x558a40f2dd90 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80096c0 tx=0x7fa6e802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 shutdown_connections 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b34f60 0x558a40f34560 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b76390 0x558a40f2dd90 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f26e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa6e0ff9640 0 client.22935 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 --2- 172.21.15.94:0/840269020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f268f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.715+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 >> 172.21.15.94:0/840269020 conn(0x558a40b6a4e0 msgr2=0x558a40f13040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.716+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 shutdown_connections 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.716+0000 7fa7067a9740 1 -- 172.21.15.94:0/840269020 wait complete. 2024-04-25T14:02:35.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.716+0000 7fa7067a9740 20 client.22935 trim_cache size 0 max 16384 2024-04-25T14:02:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 Processor -- start 2024-04-25T14:02:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 -- start start 2024-04-25T14:02:35.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f38d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81a93c0 con 0x558a40b4d4d0 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81aaf00 con 0x558a40b76390 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa7067a9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81a9a90 con 0x558a40b556b0 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34904/0 (socket says 172.21.15.94:34904) 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 -- 172.21.15.94:0/164055253 learned_addr learned my addr 172.21.15.94:0/164055253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fd695640 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f38d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 -- 172.21.15.94:0/164055253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 msgr2=0x558a40f2dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f2dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 -- 172.21.15.94:0/164055253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 msgr2=0x558a40f38d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f38d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 -- 172.21.15.94:0/164055253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a40f219a0 con 0x558a40b4d4d0 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6fce94640 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001eb0 tx=0x7fa6e8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a1e2e9afecddfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.730+0000 7fa6e2ffd640 1 -- 172.21.15.94:0/164055253 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8035070 con 0x558a40b4d4d0 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa6e2ffd640 1 -- 172.21.15.94:0/164055253 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e803dd30 con 0x558a40b4d4d0 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa6e2ffd640 1 -- 172.21.15.94:0/164055253 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8059040 con 0x558a40b4d4d0 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 -- 172.21.15.94:0/164055253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 msgr2=0x558a40f25820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001eb0 tx=0x7fa6e8031400 comp rx=0 tx=0).stop 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 -- 172.21.15.94:0/164055253 shutdown_connections 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b76390 0x558a40f38d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b556b0 0x558a40f2dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 --2- 172.21.15.94:0/164055253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b4d4d0 0x558a40f25820 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 -- 172.21.15.94:0/164055253 >> 172.21.15.94:0/164055253 conn(0x558a40b6a4e0 msgr2=0x558a40f25110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 -- 172.21.15.94:0/164055253 shutdown_connections 2024-04-25T14:02:35.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.731+0000 7fa7067a9740 1 -- 172.21.15.94:0/164055253 wait complete. 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 -- start start 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 Processor -- start 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f2b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b76390 0x558a40f2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81aa830 con 0x558a40b556b0 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81aa160 con 0x558a40b4d4d0 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81a73a0 con 0x558a40b76390 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa7067a9740 10 client.0 _ll_register_callbacks cb 0x7fa6e8032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa6fd695640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34910/0 (socket says 172.21.15.94:34910) 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa6fd695640 1 -- 172.21.15.94:0/2142738571 learned_addr learned my addr 172.21.15.94:0/2142738571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f2b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.732+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b76390 0x558a40f2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 -- 172.21.15.94:0/2142738571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b76390 msgr2=0x558a40f2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b76390 0x558a40f2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 -- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 msgr2=0x558a40f2b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f2b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 -- 172.21.15.94:0/2142738571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a40f2cfd0 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6fd695640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009930 tx=0x7fa6f4011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53de8a91230778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f401d040 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa6f4005240 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e803dd30 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa6f403a050 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa7067a9740 10 client.22941 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a40f9c900 con 0x558a40b556b0 2024-04-25T14:02:35.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.733+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e803dd30 con 0x558a40b556b0 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa6f4023040 con 0x558a40b556b0 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 10 client.22941 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 10 client.22941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 20 client.22941 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b0a3a0 0x558a40f9f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 4 client.22941 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a40e84800 con 0x558a40b0a3a0 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b0a3a0 0x558a40f9f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa6e0ff9640 10 client.22941 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:35.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.734+0000 7fa6fce94640 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b0a3a0 0x558a40f9f160 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d091c8eee41ca8c server_cookie=666da1a3b5691512 in_seq=0 out_seq=0 2024-04-25T14:02:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.738+0000 7fa6e0ff9640 1 -- 172.21.15.94:0/2142738571 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a40e84800 con 0x558a40b0a3a0 2024-04-25T14:02:35.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.738+0000 7fa6e0ff9640 10 client.22941 handle_command_reply: tid=0 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.738+0000 7fa7067a9740 1 client.22941 shutdown 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 msgr2=0x558a40f2bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009930 tx=0x7fa6f4011f20 comp rx=0 tx=0).stop 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 shutdown_connections 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a40b0a3a0 0x558a40f9f160 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a40b76390 0x558a40f2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6e0ff9640 0 client.22941 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a40b556b0 0x558a40f2bae0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 --2- 172.21.15.94:0/2142738571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a40b4d4d0 0x558a40f2b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 >> 172.21.15.94:0/2142738571 conn(0x558a40b6a4e0 msgr2=0x558a40f16e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 shutdown_connections 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 1 -- 172.21.15.94:0/2142738571 wait complete. 2024-04-25T14:02:35.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa7067a9740 20 client.22941 trim_cache size 0 max 16384 2024-04-25T14:02:35.742 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 msgr2=0x7fa6e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 0x7fa6e40b2000 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0015020 tx=0x7fa6f0013330 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 msgr2=0x7fa6f81a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000eb10 tx=0x7fa6f000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 shutdown_connections 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6e40afaf0 0x7fa6e40b2000 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6f8108920 0x7fa6f81a0900 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6f8103c20 0x7fa6f8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 --2- 172.21.15.94:0/702733542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6f81029e0 0x7fa6f8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.739+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 >> 172.21.15.94:0/702733542 conn(0x7fa6f80fdbd0 msgr2=0x7fa6f8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.740+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 shutdown_connections 2024-04-25T14:02:35.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:35.740+0000 7fa6ff8f9640 1 -- 172.21.15.94:0/702733542 wait complete. 2024-04-25T14:02:35.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/763266193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:36.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 156841 19.06 MB/sec execute 318 sec latency 250.331 ms 2024-04-25T14:02:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:36 smithi137 ceph-mon[32568]: pgmap v975: 97 pgs: 97 active+clean; 269 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-04-25T14:02:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:36 smithi094 ceph-mon[29195]: pgmap v975: 97 pgs: 97 active+clean; 269 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-04-25T14:02:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:36 smithi138 ceph-mon[32977]: pgmap v975: 97 pgs: 97 active+clean; 269 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 547 op/s 2024-04-25T14:02:36.751 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:02:36.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:02:36.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 -- 172.21.15.94:0/1704100848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0102a00 msgr2=0x7f90c0103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1704100848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0102a00 0x7f90c0103700 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009900 tx=0x7f90b402a410 comp rx=0 tx=0).stop 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 -- 172.21.15.94:0/1704100848 shutdown_connections 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1704100848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0108940 0x7f90c010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1704100848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1704100848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0102a00 0x7f90c0103700 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 -- 172.21.15.94:0/1704100848 >> 172.21.15.94:0/1704100848 conn(0x7f90c00fdbf0 msgr2=0x7f90c01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 -- 172.21.15.94:0/1704100848 shutdown_connections 2024-04-25T14:02:36.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.869+0000 7f90c6c49640 1 -- 172.21.15.94:0/1704100848 wait complete. 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 Processor -- start 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 -- start start 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0102a00 0x7f90c01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010e500 con 0x7f90c0108940 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010e380 con 0x7f90c0103c40 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c6c49640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010e680 con 0x7f90c0102a00 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c51e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c51e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34934/0 (socket says 172.21.15.94:34934) 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c51e6640 1 -- 172.21.15.94:0/1610217867 learned_addr learned my addr 172.21.15.94:0/1610217867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:36.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90bffff640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.870+0000 7f90c49e5640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0102a00 0x7f90c01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 -- 172.21.15.94:0/1610217867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0102a00 msgr2=0x7f90c01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0102a00 0x7f90c01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 -- 172.21.15.94:0/1610217867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 msgr2=0x7f90c01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 -- 172.21.15.94:0/1610217867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01a0ce0 con 0x7f90c0108940 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90bffff640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c51e6640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f90b800eb10 tx=0x7f90b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90bdffb640 1 -- 172.21.15.94:0/1610217867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004b20 con 0x7f90c0108940 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90bdffb640 1 -- 172.21.15.94:0/1610217867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b8004cc0 con 0x7f90c0108940 2024-04-25T14:02:36.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90bdffb640 1 -- 172.21.15.94:0/1610217867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8010670 con 0x7f90c0108940 2024-04-25T14:02:36.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01a0fd0 con 0x7f90c0108940 2024-04-25T14:02:36.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.871+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a1430 con 0x7f90c0108940 2024-04-25T14:02:36.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90bdffb640 1 -- 172.21.15.94:0/1610217867 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f90b8005020 con 0x7f90c0108940 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 Processor -- start 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 -- start start 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde1284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde128a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dcde08db00 con 0x55dcddd66730 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dcde029920 con 0x55dcddd87410 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcde089ac0 con 0x55dcddd5e550 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34942/0 (socket says 172.21.15.94:34942) 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90bffff640 1 -- 172.21.15.94:0/2966964503 learned_addr learned my addr 172.21.15.94:0/2966964503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.874+0000 7f90bffff640 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde1284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bdffb640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 0x7f90b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90c49e5640 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde128a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90c51e6640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 0x7f90b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 -- 172.21.15.94:0/2966964503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 msgr2=0x55dcde128a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde128a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 -- 172.21.15.94:0/2966964503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 msgr2=0x55dcde1284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde1284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 -- 172.21.15.94:0/2966964503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdde677e0 con 0x55dcddd66730 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bdffb640 1 -- 172.21.15.94:0/1610217867 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90b80df2b0 con 0x7f90c0108940 2024-04-25T14:02:36.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90bffff640 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f90ac012810 tx=0x7f90ac009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b35b592d98a34a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90c51e6640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 0x7f90b00b2000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f90b80dd080 tx=0x7f90b80d5e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90aaffd640 1 -- 172.21.15.94:0/2966964503 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac012d90 con 0x55dcddd66730 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90aaffd640 1 -- 172.21.15.94:0/2966964503 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac010c80 con 0x55dcddd66730 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.875+0000 7f90aaffd640 1 -- 172.21.15.94:0/2966964503 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01eb80 con 0x55dcddd66730 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- 172.21.15.94:0/2966964503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 msgr2=0x55dcde12f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f90ac012810 tx=0x7f90ac009740 comp rx=0 tx=0).stop 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- 172.21.15.94:0/2966964503 shutdown_connections 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde12f920 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde128a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/2966964503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde1284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- 172.21.15.94:0/2966964503 >> 172.21.15.94:0/2966964503 conn(0x55dcddd7b560 msgr2=0x55dcde123e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- 172.21.15.94:0/2966964503 shutdown_connections 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- 172.21.15.94:0/2966964503 wait complete. 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.876+0000 7f90cdbd3740 1 -- start start 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 Processor -- start 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde132780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd87410 0x55dcde1396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dcde130370 con 0x55dcddd5e550 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dcde0275d0 con 0x55dcddd66730 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcde1304f0 con 0x55dcddd87410 2024-04-25T14:02:36.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90cdbd3740 10 client.0 _ll_register_callbacks cb 0x55dcde015080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34956/0 (socket says 172.21.15.94:34956) 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 -- 172.21.15.94:0/1400303485 learned_addr learned my addr 172.21.15.94:0/1400303485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90c49e5640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde132780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd87410 0x55dcde1396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 -- 172.21.15.94:0/1400303485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd87410 msgr2=0x55dcde1396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd87410 0x55dcde1396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 -- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 msgr2=0x55dcde132780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.877+0000 7f90bffff640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde132780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90bffff640 1 -- 172.21.15.94:0/1400303485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcddec7f70 con 0x55dcddd5e550 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90c49e5640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde132780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:36.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90bffff640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f90ac011010 tx=0x7f90ac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac54f8afcbd46f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac010ac0 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90ac01eb70 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90ac010c80 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90ac01ed70 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 10 client.22953 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dcddd5f170 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac010c80 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f90ac027070 con 0x55dcddd5e550 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 10 client.22953 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 10 client.22953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:36.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 20 client.22953 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd461f0 0x55dcde13f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 4 client.22953 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.878+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcde09b710 con 0x55dcddd461f0 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.879+0000 7f90c49e5640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd461f0 0x55dcde13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.879+0000 7f90c49e5640 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd461f0 0x55dcde13f860 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43107d48d2e19bd8 server_cookie=5698c1fe336eae7e in_seq=0 out_seq=0 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.879+0000 7f909ffff640 10 client.22953 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f909ffff640 1 -- 172.21.15.94:0/1400303485 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcde09b710 con 0x55dcddd461f0 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f909ffff640 10 client.22953 handle_command_reply: tid=0 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 client.22953 shutdown 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 msgr2=0x55dcde132240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f90ac011010 tx=0x7f90ac009680 comp rx=0 tx=0).stop 2024-04-25T14:02:36.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 shutdown_connections 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd461f0 0x55dcde13f860 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd87410 0x55dcde1396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde132780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f909ffff640 0 client.22953 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/1400303485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd5e550 0x55dcde132240 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 >> 172.21.15.94:0/1400303485 conn(0x55dcddd7b560 msgr2=0x55dcde122ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 shutdown_connections 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.882+0000 7f90cdbd3740 1 -- 172.21.15.94:0/1400303485 wait complete. 2024-04-25T14:02:36.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.883+0000 7f90cdbd3740 20 client.22953 trim_cache size 0 max 16384 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.901+0000 7f90cdbd3740 1 Processor -- start 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.901+0000 7f90cdbd3740 1 -- start start 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde132520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde139670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01aa5c0 con 0x55dcddd87410 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01ac100 con 0x55dcddd66730 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01aac90 con 0x55dcddd5e550 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34960/0 (socket says 172.21.15.94:34960) 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 -- 172.21.15.94:0/806977900 learned_addr learned my addr 172.21.15.94:0/806977900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde132520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90c49e5640 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 -- 172.21.15.94:0/806977900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 msgr2=0x55dcde132520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde132520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 -- 172.21.15.94:0/806977900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 msgr2=0x55dcde139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 -- 172.21.15.94:0/806977900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcde123620 con 0x55dcddd87410 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90bffff640 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0095b0 tx=0x7f90ac01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432231d36762ab73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90aa7fc640 1 -- 172.21.15.94:0/806977900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01eb70 con 0x55dcddd87410 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90aa7fc640 1 -- 172.21.15.94:0/806977900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac01e740 con 0x55dcddd87410 2024-04-25T14:02:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.902+0000 7f90aa7fc640 1 -- 172.21.15.94:0/806977900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004da0 con 0x55dcddd87410 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 -- 172.21.15.94:0/806977900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 msgr2=0x55dcde139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0095b0 tx=0x7f90ac01b040 comp rx=0 tx=0).stop 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 -- 172.21.15.94:0/806977900 shutdown_connections 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd87410 0x55dcde139bb0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd66730 0x55dcde139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/806977900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde132520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 -- 172.21.15.94:0/806977900 >> 172.21.15.94:0/806977900 conn(0x55dcddd7b560 msgr2=0x55dcde131e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 -- 172.21.15.94:0/806977900 shutdown_connections 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.903+0000 7f90cdbd3740 1 -- 172.21.15.94:0/806977900 wait complete. 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 -- start start 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 Processor -- start 2024-04-25T14:02:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde138460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01aba30 con 0x55dcddd66730 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01ab360 con 0x55dcddd87410 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01a85a0 con 0x55dcddd5e550 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90cdbd3740 10 client.0 _ll_register_callbacks cb 0x7f90ac010860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90c49e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90c49e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34974/0 (socket says 172.21.15.94:34974) 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90c49e5640 1 -- 172.21.15.94:0/3231925816 learned_addr learned my addr 172.21.15.94:0/3231925816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90bffff640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.904+0000 7f90bffff640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 -- 172.21.15.94:0/3231925816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 msgr2=0x55dcde136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 -- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 msgr2=0x55dcde138460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde138460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 -- 172.21.15.94:0/3231925816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcde138a20 con 0x55dcddd66730 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90bffff640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90c49e5640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005f70 tx=0x7f90b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6977637494a619f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f909ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4035040 con 0x55dcddd66730 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f90b4005040 con 0x55dcddd66730 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90ac01e740 con 0x55dcddd66730 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90b4054050 con 0x55dcddd66730 2024-04-25T14:02:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90cdbd3740 10 client.22959 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dcde139db0 con 0x55dcddd66730 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.905+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac01e740 con 0x55dcddd66730 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f90b4045320 con 0x55dcddd66730 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 10 client.22959 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 10 client.22959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 20 client.22959 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd1b630 0x55dcde1aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 4 client.22959 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcde095580 con 0x55dcddd1b630 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90bffff640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd1b630 0x55dcde1aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f909ffff640 10 client.22959 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.906+0000 7f90bffff640 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd1b630 0x55dcde1aa320 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da7093e0c0614d3 server_cookie=a99f1d3d73f7a775 in_seq=0 out_seq=0 2024-04-25T14:02:36.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f909ffff640 1 -- 172.21.15.94:0/3231925816 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcde095580 con 0x55dcddd1b630 2024-04-25T14:02:36.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f909ffff640 10 client.22959 handle_command_reply: tid=0 2024-04-25T14:02:36.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 client.22959 shutdown 2024-04-25T14:02:36.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 msgr2=0x55dcde137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005f70 tx=0x7f90b4004040 comp rx=0 tx=0).stop 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 shutdown_connections 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55dcddd1b630 0x55dcde1aa320 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f909ffff640 0 client.22959 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dcddd87410 0x55dcde138460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcddd66730 0x55dcde137530 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 --2- 172.21.15.94:0/3231925816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcddd5e550 0x55dcde136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 >> 172.21.15.94:0/3231925816 conn(0x55dcddd7b560 msgr2=0x55dcde1224c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 shutdown_connections 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 1 -- 172.21.15.94:0/3231925816 wait complete. 2024-04-25T14:02:36.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.910+0000 7f90cdbd3740 20 client.22959 trim_cache size 0 max 16384 2024-04-25T14:02:36.916 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:36.916 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:02:36.916 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "41f35605-fcf8-4e82-8c69-ec9a95abce7e", 2024-04-25T14:02:36.916 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:02:36.916 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 msgr2=0x7f90b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 0x7f90b00b2000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f90b80dd080 tx=0x7f90b80d5e40 comp rx=0 tx=0).stop 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 msgr2=0x7f90c01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f90b800eb10 tx=0x7f90b800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 shutdown_connections 2024-04-25T14:02:36.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f90b00afaf0 0x7f90b00b2000 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90c0108940 0x7f90c01a0670 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90c0103c40 0x7f90c01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 --2- 172.21.15.94:0/1610217867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c0102a00 0x7f90c01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 >> 172.21.15.94:0/1610217867 conn(0x7f90c00fdbf0 msgr2=0x7f90c0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 shutdown_connections 2024-04-25T14:02:36.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:36.911+0000 7f90c6c49640 1 -- 172.21.15.94:0/1610217867 wait complete. 2024-04-25T14:02:36.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:36.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- 172.21.15.94:0/2269816749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 msgr2=0x7fb65810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 --2- 172.21.15.94:0/2269816749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb65810ad40 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fb650009ca0 tx=0x7fb65002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- 172.21.15.94:0/2269816749 shutdown_connections 2024-04-25T14:02:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 --2- 172.21.15.94:0/2269816749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb65810ad40 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 --2- 172.21.15.94:0/2269816749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658103c20 0x7fb658104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 --2- 172.21.15.94:0/2269816749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6581029e0 0x7fb6581036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- 172.21.15.94:0/2269816749 >> 172.21.15.94:0/2269816749 conn(0x7fb6580fdbd0 msgr2=0x7fb658100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- 172.21.15.94:0/2269816749 shutdown_connections 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- 172.21.15.94:0/2269816749 wait complete. 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 Processor -- start 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.052+0000 7fb65ef00640 1 -- start start 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6581029e0 0x7fb65819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658103c20 0x7fb65819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e510 con 0x7fb658108920 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e390 con 0x7fb6581029e0 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65ef00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810e690 con 0x7fb658103c20 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34990/0 (socket says 172.21.15.94:34990) 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 -- 172.21.15.94:0/801770230 learned_addr learned my addr 172.21.15.94:0/801770230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6581029e0 0x7fb65819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb657fff640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658103c20 0x7fb65819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 -- 172.21.15.94:0/801770230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658103c20 msgr2=0x7fb65819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658103c20 0x7fb65819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 -- 172.21.15.94:0/801770230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6581029e0 msgr2=0x7fb65819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6581029e0 0x7fb65819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 -- 172.21.15.94:0/801770230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6581a2060 con 0x7fb658108920 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb65d49d640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fb650004af0 tx=0x7fb650031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.053+0000 7fb655ffb640 1 -- 172.21.15.94:0/801770230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650003b70 con 0x7fb658108920 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.054+0000 7fb655ffb640 1 -- 172.21.15.94:0/801770230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb650003d10 con 0x7fb658108920 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.054+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a22f0 con 0x7fb658108920 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.054+0000 7fb655ffb640 1 -- 172.21.15.94:0/801770230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65002fb60 con 0x7fb658108920 2024-04-25T14:02:37.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.054+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a2840 con 0x7fb658108920 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 Processor -- start 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb655ffb640 1 -- 172.21.15.94:0/801770230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb65002fd00 con 0x7fb658108920 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 -- start start 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 0x560884835a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608844736b0 0x560884836c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56088479ad80 con 0x56088446b4d0 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608847548b0 con 0x560884494390 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb665db5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560884796d40 con 0x5608844736b0 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb657fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 0x560884835a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35006/0 (socket says 172.21.15.94:35006) 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.057+0000 7fb65cc9c640 1 -- 172.21.15.94:0/110857998 learned_addr learned my addr 172.21.15.94:0/110857998 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb657fff640 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608844736b0 0x560884836c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb655ffb640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 0x7fb6480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 -- 172.21.15.94:0/110857998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608844736b0 msgr2=0x560884836c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608844736b0 0x560884836c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 -- 172.21.15.94:0/110857998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 msgr2=0x560884835a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 0x560884835a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65d49d640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 0x7fb6480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 -- 172.21.15.94:0/110857998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560884725a40 con 0x56088446b4d0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb657fff640 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 0x560884835a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb655ffb640 1 -- 172.21.15.94:0/801770230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb650100290 con 0x7fb658108920 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009c60 tx=0x7fb64c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1f8a2db0d6eb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb642ffd640 1 -- 172.21.15.94:0/110857998 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004220 con 0x56088446b4d0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb642ffd640 1 -- 172.21.15.94:0/110857998 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0043c0 con 0x56088446b4d0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.058+0000 7fb642ffd640 1 -- 172.21.15.94:0/110857998 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0197d0 con 0x56088446b4d0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 -- 172.21.15.94:0/110857998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 msgr2=0x560884835fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009c60 tx=0x7fb64c011810 comp rx=0 tx=0).stop 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb65d49d640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 0x7fb6480b2000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb6500c8660 tx=0x7fb650097190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 -- 172.21.15.94:0/110857998 shutdown_connections 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608844736b0 0x560884836c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884835fe0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 --2- 172.21.15.94:0/110857998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560884494390 0x560884835a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 -- 172.21.15.94:0/110857998 >> 172.21.15.94:0/110857998 conn(0x5608844884e0 msgr2=0x560884831270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 -- 172.21.15.94:0/110857998 shutdown_connections 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.059+0000 7fb665db5740 1 -- 172.21.15.94:0/110857998 wait complete. 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 -- start start 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 Processor -- start 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x56088483f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884494390 0x560884846d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560884837780 con 0x56088446b4d0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560884721d10 con 0x5608844736b0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560884837900 con 0x560884494390 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb665db5740 10 client.0 _ll_register_callbacks cb 0x56088474e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb657fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x56088483f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46224/0 (socket says 172.21.15.94:46224) 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb65cc9c640 1 -- 172.21.15.94:0/1450985122 learned_addr learned my addr 172.21.15.94:0/1450985122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb657fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x56088483f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35012/0 (socket says 172.21.15.94:35012) 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb657fff640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884494390 0x560884846d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb65cc9c640 1 -- 172.21.15.94:0/1450985122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884494390 msgr2=0x560884846d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.060+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884494390 0x560884846d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb65cc9c640 1 -- 172.21.15.94:0/1450985122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 msgr2=0x56088483f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x56088483f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb65cc9c640 1 -- 172.21.15.94:0/1450985122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608845767d0 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb64c01b040 tx=0x7fb64c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547fa605bedc6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb637fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c028070 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c0043c0 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb64c005250 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb64c034050 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb665db5740 10 client.32360 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5608844ac3f0 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.061+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0043c0 con 0x5608844736b0 2024-04-25T14:02:37.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb64c004b70 con 0x5608844736b0 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 10 client.32360 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 10 client.32360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 20 client.32360 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884452f60 0x56088484cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 4 client.32360 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608847a8990 con 0x560884452f60 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb657fff640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884452f60 0x56088484cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb637fff640 10 client.32360 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:37.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.062+0000 7fb657fff640 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884452f60 0x56088484cfd0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf78c14c23d9b69 server_cookie=a9850afb95169a46 in_seq=0 out_seq=0 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb637fff640 1 -- 172.21.15.94:0/1450985122 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5608847a8990 con 0x560884452f60 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb637fff640 10 client.32360 handle_command_reply: tid=0 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 client.32360 shutdown 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 msgr2=0x56088483fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb64c01b040 tx=0x7fb64c0038a0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 shutdown_connections 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884452f60 0x56088484cfd0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884494390 0x560884846d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x56088483fdf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb637fff640 0 client.32360 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:37.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 --2- 172.21.15.94:0/1450985122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x56088483f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 >> 172.21.15.94:0/1450985122 conn(0x5608844884e0 msgr2=0x560884834180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 shutdown_connections 2024-04-25T14:02:37.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 1 -- 172.21.15.94:0/1450985122 wait complete. 2024-04-25T14:02:37.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.066+0000 7fb665db5740 20 client.32360 trim_cache size 0 max 16384 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 Processor -- start 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 -- start start 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884452f60 0x5608848313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x5608848471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ab530 con 0x56088446b4d0 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ad070 con 0x5608844736b0 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb665db5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581abc00 con 0x560884452f60 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35020/0 (socket says 172.21.15.94:35020) 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 -- 172.21.15.94:0/3254206521 learned_addr learned my addr 172.21.15.94:0/3254206521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb657fff640 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884452f60 0x5608848313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb657fff640 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x5608848471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 -- 172.21.15.94:0/3254206521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884452f60 msgr2=0x5608848313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884452f60 0x5608848313b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 -- 172.21.15.94:0/3254206521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 msgr2=0x5608848471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x5608848471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.085+0000 7fb65cc9c640 1 -- 172.21.15.94:0/3254206521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56088446c0f0 con 0x56088446b4d0 2024-04-25T14:02:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.086+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fb64c004f10 tx=0x7fb64c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c201fa6317bc88b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.086+0000 7fb6427fc640 1 -- 172.21.15.94:0/3254206521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c028070 con 0x56088446b4d0 2024-04-25T14:02:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.086+0000 7fb6427fc640 1 -- 172.21.15.94:0/3254206521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c031030 con 0x56088446b4d0 2024-04-25T14:02:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.086+0000 7fb6427fc640 1 -- 172.21.15.94:0/3254206521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c019670 con 0x56088446b4d0 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 -- 172.21.15.94:0/3254206521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 msgr2=0x560884846c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fb64c004f10 tx=0x7fb64c011810 comp rx=0 tx=0).stop 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 -- 172.21.15.94:0/3254206521 shutdown_connections 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x5608848471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56088446b4d0 0x560884846c70 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 --2- 172.21.15.94:0/3254206521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560884452f60 0x5608848313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 -- 172.21.15.94:0/3254206521 >> 172.21.15.94:0/3254206521 conn(0x5608844884e0 msgr2=0x560884830ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 -- 172.21.15.94:0/3254206521 shutdown_connections 2024-04-25T14:02:37.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.087+0000 7fb665db5740 1 -- 172.21.15.94:0/3254206521 wait complete. 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 -- start start 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 Processor -- start 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56088446b4d0 0x560884844010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x560884844550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ac9a0 con 0x560884452f60 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ac2d0 con 0x5608844736b0 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581a9510 con 0x56088446b4d0 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb665db5740 10 client.0 _ll_register_callbacks cb 0x7fb644009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb657fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb657fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35036/0 (socket says 172.21.15.94:35036) 2024-04-25T14:02:37.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb657fff640 1 -- 172.21.15.94:0/1550460090 learned_addr learned my addr 172.21.15.94:0/1550460090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.088+0000 7fb657fff640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x560884844550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56088446b4d0 0x560884844010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 -- 172.21.15.94:0/1550460090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56088446b4d0 msgr2=0x560884844010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56088446b4d0 0x560884844010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 -- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 msgr2=0x560884844550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x560884844550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 -- 172.21.15.94:0/1550460090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56088475b4b0 con 0x560884452f60 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb657fff640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x560884836ab0 tx=0x7fb64c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2f7f998a07c297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb637fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64401a040 con 0x560884452f60 2024-04-25T14:02:37.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb644003cf0 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.089+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c031030 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb644005020 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 10 client.22977 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5608848b43c0 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c031030 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb644004bd0 con 0x560884452f60 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 10 client.22977 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 10 client.22977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 20 client.22977 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884494390 0x5608848b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 4 client.22977 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:37.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608847a2800 con 0x560884494390 2024-04-25T14:02:37.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.090+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884494390 0x5608848b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:37.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.091+0000 7fb637fff640 10 client.22977 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:37.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.091+0000 7fb65cc9c640 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884494390 0x5608848b6de0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7153d0dda61d7504 server_cookie=3a4400d9a41b1ca in_seq=0 out_seq=0 2024-04-25T14:02:37.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb637fff640 1 -- 172.21.15.94:0/1550460090 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5608847a2800 con 0x560884494390 2024-04-25T14:02:37.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb637fff640 10 client.22977 handle_command_reply: tid=0 2024-04-25T14:02:37.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 client.22977 shutdown 2024-04-25T14:02:37.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 msgr2=0x560884847490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x560884836ab0 tx=0x7fb64c0042e0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 shutdown_connections 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560884494390 0x5608848b6de0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608844736b0 0x560884844550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56088446b4d0 0x560884844010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 --2- 172.21.15.94:0/1550460090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560884452f60 0x560884847490 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb637fff640 0 client.22977 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 >> 172.21.15.94:0/1550460090 conn(0x5608844884e0 msgr2=0x56088483df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 shutdown_connections 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.094+0000 7fb665db5740 1 -- 172.21.15.94:0/1550460090 wait complete. 2024-04-25T14:02:37.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb665db5740 20 client.22977 trim_cache size 0 max 16384 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (390 inodes in the stack)", 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "41f35605-fcf8-4e82-8c69-ec9a95abce7e": { 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "41f35605-fcf8-4e82-8c69-ec9a95abce7e", 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:02:37.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:02:37.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:02:37.100 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 msgr2=0x7fb6480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 0x7fb6480b2000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb6500c8660 tx=0x7fb650097190 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 msgr2=0x7fb6581a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fb650004af0 tx=0x7fb650031700 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 shutdown_connections 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6480afaf0 0x7fb6480b2000 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658108920 0x7fb6581a19f0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658103c20 0x7fb65819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 --2- 172.21.15.94:0/801770230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6581029e0 0x7fb65819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:37.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 >> 172.21.15.94:0/801770230 conn(0x7fb6580fdbd0 msgr2=0x7fb658109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 shutdown_connections 2024-04-25T14:02:37.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:02:37.095+0000 7fb65ef00640 1 -- 172.21.15.94:0/801770230 wait complete. 2024-04-25T14:02:37.107 INFO:tasks.cephfs.filesystem:scrub status for tag:41f35605-fcf8-4e82-8c69-ec9a95abce7e - {'path': '/', 'tag': '41f35605-fcf8-4e82-8c69-ec9a95abce7e', 'options': 'recursive,force'} 2024-04-25T14:02:37.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 157090 19.06 MB/sec execute 319 sec latency 125.563 ms 2024-04-25T14:02:38.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 157282 19.02 MB/sec execute 320 sec latency 179.572 ms 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: pgmap v976: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:38 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: pgmap v976: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2024-04-25T14:02:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:38 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: pgmap v976: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:38.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:38 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:39.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 157532 18.99 MB/sec execute 321 sec latency 107.604 ms 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 -- 172.21.15.94:0/2038181909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc109a80 msgr2=0x7fc2cc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2038181909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc109a80 0x7fc2cc10bea0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009ca0 tx=0x7fc2c402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 -- 172.21.15.94:0/2038181909 shutdown_connections 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2038181909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc109a80 0x7fc2cc10bea0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2038181909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc104d80 0x7fc2cc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2038181909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc103b40 0x7fc2cc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 -- 172.21.15.94:0/2038181909 >> 172.21.15.94:0/2038181909 conn(0x7fc2cc0fed30 msgr2=0x7fc2cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 -- 172.21.15.94:0/2038181909 shutdown_connections 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.163+0000 7fc2d3127640 1 -- 172.21.15.94:0/2038181909 wait complete. 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 Processor -- start 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 -- start start 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc103b40 0x7fc2cc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc109a80 0x7fc2cc1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10f670 con 0x7fc2cc104d80 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10f4f0 con 0x7fc2cc103b40 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d3127640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10f7f0 con 0x7fc2cc109a80 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35050/0 (socket says 172.21.15.94:35050) 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 -- 172.21.15.94:0/2606545923 learned_addr learned my addr 172.21.15.94:0/2606545923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc103b40 0x7fc2cc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc109a80 0x7fc2cc1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 -- 172.21.15.94:0/2606545923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc109a80 msgr2=0x7fc2cc1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc109a80 0x7fc2cc1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 -- 172.21.15.94:0/2606545923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc103b40 msgr2=0x7fc2cc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc103b40 0x7fc2cc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 -- 172.21.15.94:0/2606545923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc1a1d50 con 0x7fc2cc104d80 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.164+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9f0 tx=0x7fc2bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.165+0000 7fc2c1ffb640 1 -- 172.21.15.94:0/2606545923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00cd80 con 0x7fc2cc104d80 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.165+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1a1fe0 con 0x7fc2cc104d80 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.165+0000 7fc2c1ffb640 1 -- 172.21.15.94:0/2606545923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0108e0 con 0x7fc2cc104d80 2024-04-25T14:02:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.165+0000 7fc2c1ffb640 1 -- 172.21.15.94:0/2606545923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018980 con 0x7fc2cc104d80 2024-04-25T14:02:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.165+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1a24d0 con 0x7fc2cc104d80 2024-04-25T14:02:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.167+0000 7fc2c1ffb640 1 -- 172.21.15.94:0/2606545923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc2bc018b20 con 0x7fc2cc104d80 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 Processor -- start 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 -- start start 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c1ffb640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 0x7fc2b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 0x562496f2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 0x7fc2b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c1ffb640 1 -- 172.21.15.94:0/2606545923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2bc0df090 con 0x7fc2cc104d80 2024-04-25T14:02:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f34df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562496e93460 con 0x562496b642f0 2024-04-25T14:02:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562496e4d590 con 0x562496b8d1b0 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562496e8f420 con 0x562496b6c4d0 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35056/0 (socket says 172.21.15.94:35056) 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 0x562496f2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c3fff640 1 -- 172.21.15.94:0/157314082 learned_addr learned my addr 172.21.15.94:0/157314082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f34df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/157314082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 msgr2=0x562496f34df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f34df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/157314082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 msgr2=0x562496f2def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/157314082 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562496e5a0d0 con 0x562496b8d1b0 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.168+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.169+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 0x7fc2b40b1ff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005e30 tx=0x7fc2c4009a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.169+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 0x562496f2d970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc103410 tx=0x7fc2c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30603e00bb8ce290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.169+0000 7fc2b2ffd640 1 -- 172.21.15.94:0/157314082 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004220 con 0x562496b8d1b0 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.169+0000 7fc2b2ffd640 1 -- 172.21.15.94:0/157314082 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c8009e10 con 0x562496b8d1b0 2024-04-25T14:02:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.169+0000 7fc2b2ffd640 1 -- 172.21.15.94:0/157314082 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8018610 con 0x562496b8d1b0 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/157314082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 msgr2=0x562496f2d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 0x562496f2d970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc103410 tx=0x7fc2c8011810 comp rx=0 tx=0).stop 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/157314082 shutdown_connections 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f34df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/157314082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b8d1b0 0x562496f2d970 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/157314082 >> 172.21.15.94:0/157314082 conn(0x562496b81300 msgr2=0x562496f29300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/157314082 shutdown_connections 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/157314082 wait complete. 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 -- start start 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.170+0000 7fc2d9fd6740 1 Processor -- start 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 0x562496f40390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b8d1b0 0x562496f472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562496f3e050 con 0x562496b642f0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562496e1ab60 con 0x562496b6c4d0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562496f3e1d0 con 0x562496b8d1b0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d9fd6740 10 client.0 _ll_register_callbacks cb 0x562496e3f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 0x562496f40390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35068/0 (socket says 172.21.15.94:35068) 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/4196611637 learned_addr learned my addr 172.21.15.94:0/4196611637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b8d1b0 0x562496f472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/4196611637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b8d1b0 msgr2=0x562496f472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b8d1b0 0x562496f472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 msgr2=0x562496f40390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 0x562496f40390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 -- 172.21.15.94:0/4196611637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562496ba5210 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 0x562496f40390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005270 tx=0x7fc2c80052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6698230581e7a192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.171+0000 7fc2abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8027020 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2c80035f0 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c8009e10 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2c8032050 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 10 client.22989 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562496c71620 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c8009e10 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c8004ab0 con 0x562496b642f0 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 10 client.22989 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 10 client.22989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 20 client.22989 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b4bd80 0x562496f4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 4 client.22989 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562496ea1070 con 0x562496b4bd80 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.172+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b4bd80 0x562496f4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.173+0000 7fc2abfff640 10 client.22989 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.173+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b4bd80 0x562496f4d8c0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7060d92e0d6760 server_cookie=7e061da30ef10547 in_seq=0 out_seq=0 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2abfff640 1 -- 172.21.15.94:0/4196611637 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562496ea1070 con 0x562496b4bd80 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2abfff640 10 client.22989 handle_command_reply: tid=0 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 client.22989 shutdown 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 msgr2=0x562496f3fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005270 tx=0x7fc2c80052a0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 shutdown_connections 2024-04-25T14:02:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b4bd80 0x562496f4d8c0 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2abfff640 0 client.22989 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b8d1b0 0x562496f472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b6c4d0 0x562496f40390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/4196611637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f3fe50 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 >> 172.21.15.94:0/4196611637 conn(0x562496b81300 msgr2=0x562496f28250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 shutdown_connections 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.174+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/4196611637 wait complete. 2024-04-25T14:02:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.175+0000 7fc2d9fd6740 20 client.22989 trim_cache size 0 max 16384 2024-04-25T14:02:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.190+0000 7fc2d9fd6740 1 Processor -- start 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.190+0000 7fc2d9fd6740 1 -- start start 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b4bd80 0x562496f28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f3fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a93e0 con 0x562496b642f0 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1aaf20 con 0x562496b4bd80 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a9ab0 con 0x562496b6c4d0 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35080/0 (socket says 172.21.15.94:35080) 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 -- 172.21.15.94:0/3169281075 learned_addr learned my addr 172.21.15.94:0/3169281075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b4bd80 0x562496f28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f3fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 -- 172.21.15.94:0/3169281075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 msgr2=0x562496f3fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f3fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 -- 172.21.15.94:0/3169281075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b4bd80 msgr2=0x562496f28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b4bd80 0x562496f28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 -- 172.21.15.94:0/3169281075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562496f2e6d0 con 0x562496b642f0 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f3fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a7180 tx=0x7fc2bc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca2a72b42915d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.191+0000 7fc2b27fc640 1 -- 172.21.15.94:0/3169281075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0b0330 con 0x562496b642f0 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2b27fc640 1 -- 172.21.15.94:0/3169281075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0dcc00 con 0x562496b642f0 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2b27fc640 1 -- 172.21.15.94:0/3169281075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0a30a0 con 0x562496b642f0 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/3169281075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 msgr2=0x562496f290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0a7180 tx=0x7fc2bc018420 comp rx=0 tx=0).stop 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/3169281075 shutdown_connections 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b6c4d0 0x562496f3fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b642f0 0x562496f290e0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/3169281075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b4bd80 0x562496f28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/3169281075 >> 172.21.15.94:0/3169281075 conn(0x562496b81300 msgr2=0x562496f31ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/3169281075 shutdown_connections 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.192+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/3169281075 wait complete. 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 -- start start 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 Processor -- start 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b4bd80 0x562496f467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 0x562496f43370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1aa850 con 0x562496b6c4d0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1aa180 con 0x562496b642f0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a73c0 con 0x562496b4bd80 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d9fd6740 10 client.0 _ll_register_callbacks cb 0x7fc2bc0a2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d16c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d16c4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35090/0 (socket says 172.21.15.94:35090) 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d16c4640 1 -- 172.21.15.94:0/2771941017 learned_addr learned my addr 172.21.15.94:0/2771941017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 0x562496f43370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.193+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b4bd80 0x562496f467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 -- 172.21.15.94:0/2771941017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b4bd80 msgr2=0x562496f467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b4bd80 0x562496f467f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 -- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 msgr2=0x562496f43370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 0x562496f43370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 -- 172.21.15.94:0/2771941017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562496fb3c60 con 0x562496b6c4d0 2024-04-25T14:02:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2c3fff640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 0x562496f43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d16c4640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4035000 tx=0x7fc2c4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2094ee4520fad3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2abfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40045d0 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc2c4004770 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc0dcc00 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc2c402f660 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d9fd6740 10 client.22995 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562496fb4e10 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.194+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0dcc00 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c40309d0 con 0x562496b6c4d0 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 10 client.22995 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 10 client.22995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 20 client.22995 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b8d1b0 0x562496fb7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 4 client.22995 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562496e9aee0 con 0x562496b8d1b0 2024-04-25T14:02:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b8d1b0 0x562496fb7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2abfff640 10 client.22995 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.195+0000 7fc2d0ec3640 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b8d1b0 0x562496fb7810 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe88c54e10840b6b server_cookie=2636eb0fa6298b7a in_seq=0 out_seq=0 2024-04-25T14:02:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:39 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:39 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:39 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.883+0000 7fc2abfff640 1 -- 172.21.15.94:0/2771941017 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562496e9aee0 con 0x562496b8d1b0 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.883+0000 7fc2abfff640 10 client.22995 handle_command_reply: tid=0 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.883+0000 7fc2d9fd6740 1 client.22995 shutdown 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 msgr2=0x562496f438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4035000 tx=0x7fc2c4045000 comp rx=0 tx=0).stop 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 shutdown_connections 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562496b8d1b0 0x562496fb7810 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562496b6c4d0 0x562496f438d0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2abfff640 0 client.22995 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562496b642f0 0x562496f43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 --2- 172.21.15.94:0/2771941017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562496b4bd80 0x562496f467f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 >> 172.21.15.94:0/2771941017 conn(0x562496b81300 msgr2=0x562496f28ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 shutdown_connections 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 1 -- 172.21.15.94:0/2771941017 wait complete. 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.884+0000 7fc2d9fd6740 20 client.22995 trim_cache size 0 max 16384 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:02:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 msgr2=0x7fc2b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 0x7fc2b40b1ff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005e30 tx=0x7fc2c4009a20 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 msgr2=0x7fc2cc19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9f0 tx=0x7fc2bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 shutdown_connections 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2b40afae0 0x7fc2b40b1ff0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2cc109a80 0x7fc2cc1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2cc104d80 0x7fc2cc19a7c0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 --2- 172.21.15.94:0/2606545923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2cc103b40 0x7fc2cc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 >> 172.21.15.94:0/2606545923 conn(0x7fc2cc0fed30 msgr2=0x7fc2cc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 shutdown_connections 2024-04-25T14:02:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:39.885+0000 7fc2d3127640 1 -- 172.21.15.94:0/2606545923 wait complete. 2024-04-25T14:02:40.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 157649 18.95 MB/sec execute 322 sec latency 332.815 ms 2024-04-25T14:02:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:40 smithi137 ceph-mon[32568]: pgmap v977: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-04-25T14:02:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:40 smithi094 ceph-mon[29195]: pgmap v977: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-04-25T14:02:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:40 smithi138 ceph-mon[32977]: pgmap v977: 97 pgs: 97 active+clean; 273 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-04-25T14:02:41.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 157876 18.91 MB/sec execute 323 sec latency 308.303 ms 2024-04-25T14:02:42.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 158295 18.89 MB/sec execute 324 sec latency 64.330 ms 2024-04-25T14:02:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:42 smithi137 ceph-mon[32568]: pgmap v978: 97 pgs: 97 active+clean; 278 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-04-25T14:02:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:42 smithi094 ceph-mon[29195]: pgmap v978: 97 pgs: 97 active+clean; 278 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-04-25T14:02:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:42 smithi138 ceph-mon[32977]: pgmap v978: 97 pgs: 97 active+clean; 278 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-04-25T14:02:43.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 158778 18.93 MB/sec execute 325 sec latency 62.189 ms 2024-04-25T14:02:44.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 159390 18.96 MB/sec execute 326 sec latency 55.747 ms 2024-04-25T14:02:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:44 smithi137 ceph-mon[32568]: pgmap v979: 97 pgs: 97 active+clean; 274 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 457 op/s 2024-04-25T14:02:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:44 smithi094 ceph-mon[29195]: pgmap v979: 97 pgs: 97 active+clean; 274 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 457 op/s 2024-04-25T14:02:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:44 smithi138 ceph-mon[32977]: pgmap v979: 97 pgs: 97 active+clean; 274 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 457 op/s 2024-04-25T14:02:45.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 159836 19.00 MB/sec execute 327 sec latency 76.742 ms 2024-04-25T14:02:46.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 160131 19.00 MB/sec execute 328 sec latency 262.214 ms 2024-04-25T14:02:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:46 smithi137 ceph-mon[32568]: pgmap v980: 97 pgs: 97 active+clean; 261 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-04-25T14:02:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:46 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:46 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:46 smithi094 ceph-mon[29195]: pgmap v980: 97 pgs: 97 active+clean; 261 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-04-25T14:02:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:46 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:46 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:46 smithi138 ceph-mon[32977]: pgmap v980: 97 pgs: 97 active+clean; 261 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-04-25T14:02:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:46 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:46 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:47.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 160487 19.03 MB/sec execute 329 sec latency 66.054 ms 2024-04-25T14:02:47.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:47 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:47.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:47 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:47.648 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:47 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:02:47.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:47 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:47.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:47 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:47.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:47 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:02:47.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:47 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:47 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:47.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:47 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:02:48.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 160705 19.01 MB/sec execute 330 sec latency 173.269 ms 2024-04-25T14:02:48.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:48 smithi094 ceph-mon[29195]: pgmap v981: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-04-25T14:02:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:48 smithi137 ceph-mon[32568]: pgmap v981: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-04-25T14:02:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:48 smithi138 ceph-mon[32977]: pgmap v981: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.6 MiB/s wr, 574 op/s 2024-04-25T14:02:49.152 INFO:tasks.workunit.client.0.smithi094.stdout: 10 160936 18.97 MB/sec execute 331 sec latency 73.428 ms 2024-04-25T14:02:50.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 161212 18.95 MB/sec execute 332 sec latency 74.163 ms 2024-04-25T14:02:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:50 smithi137 ceph-mon[32568]: pgmap v982: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-04-25T14:02:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:50 smithi094 ceph-mon[29195]: pgmap v982: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-04-25T14:02:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:50 smithi138 ceph-mon[32977]: pgmap v982: 97 pgs: 97 active+clean; 266 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-04-25T14:02:51.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 161491 18.92 MB/sec execute 333 sec latency 170.910 ms 2024-04-25T14:02:52.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 161868 18.90 MB/sec execute 334 sec latency 91.957 ms 2024-04-25T14:02:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:52 smithi137 ceph-mon[32568]: pgmap v983: 97 pgs: 97 active+clean; 271 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-04-25T14:02:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:52 smithi094 ceph-mon[29195]: pgmap v983: 97 pgs: 97 active+clean; 271 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-04-25T14:02:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:52 smithi138 ceph-mon[32977]: pgmap v983: 97 pgs: 97 active+clean; 271 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 644 op/s 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 -- 172.21.15.94:0/2774634174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb81044b0 msgr2=0x7efeb810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 --2- 172.21.15.94:0/2774634174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb81044b0 0x7efeb810abf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009ca0 tx=0x7efeb002a8e0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 -- 172.21.15.94:0/2774634174 shutdown_connections 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 --2- 172.21.15.94:0/2774634174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb81044b0 0x7efeb810abf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 --2- 172.21.15.94:0/2774634174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb8103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 --2- 172.21.15.94:0/2774634174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb8102fd0 0x7efeb81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 -- 172.21.15.94:0/2774634174 >> 172.21.15.94:0/2774634174 conn(0x7efeb80fd390 msgr2=0x7efeb80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.000+0000 7efec0314640 1 -- 172.21.15.94:0/2774634174 shutdown_connections 2024-04-25T14:02:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.001+0000 7efec0314640 1 -- 172.21.15.94:0/2774634174 wait complete. 2024-04-25T14:02:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.001+0000 7efec0314640 1 Processor -- start 2024-04-25T14:02:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.001+0000 7efec0314640 1 -- start start 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 0x7efeb819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb81044b0 0x7efeb81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810f6c0 con 0x7efeb81039b0 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810f540 con 0x7efeb81044b0 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efec0314640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810f840 con 0x7efeb8102fd0 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebe0b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 0x7efeb819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58646/0 (socket says 172.21.15.94:58646) 2024-04-25T14:02:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 -- 172.21.15.94:0/3188987349 learned_addr learned my addr 172.21.15.94:0/3188987349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebe8b1640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb81044b0 0x7efeb81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 -- 172.21.15.94:0/3188987349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 msgr2=0x7efeb819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 0x7efeb819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 -- 172.21.15.94:0/3188987349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb81044b0 msgr2=0x7efeb81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb81044b0 0x7efeb81a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 -- 172.21.15.94:0/3188987349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81a1d40 con 0x7efeb81039b0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 0x7efeb819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efebd8af640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7efea800e9f0 tx=0x7efea800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.002+0000 7efeaf7fe640 1 -- 172.21.15.94:0/3188987349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea800ccc0 con 0x7efeb81039b0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.003+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb810b6f0 con 0x7efeb81039b0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.003+0000 7efeaf7fe640 1 -- 172.21.15.94:0/3188987349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea80108e0 con 0x7efeb81039b0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.003+0000 7efeaf7fe640 1 -- 172.21.15.94:0/3188987349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8018880 con 0x7efeb81039b0 2024-04-25T14:02:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.003+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb810bd10 con 0x7efeb81039b0 2024-04-25T14:02:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.006+0000 7efeaf7fe640 1 -- 172.21.15.94:0/3188987349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efea8010430 con 0x7efeb81039b0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.006+0000 7efec71c7740 1 Processor -- start 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.006+0000 7efec71c7740 1 -- start start 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.006+0000 7efec71c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efec71c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a365ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efec71c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efec71c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55700a2cb460 con 0x557009fa44d0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efec71c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55700a285590 con 0x557009fc51b0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efec71c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55700a2c7420 con 0x557009f9c2f0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43186/0 (socket says 172.21.15.94:43186) 2024-04-25T14:02:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 -- 172.21.15.94:0/3215019698 learned_addr learned my addr 172.21.15.94:0/3215019698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a365ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 -- 172.21.15.94:0/3215019698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 msgr2=0x55700a365ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a365ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 -- 172.21.15.94:0/3215019698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 msgr2=0x55700a36cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 -- 172.21.15.94:0/3215019698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700a2920d0 con 0x557009fc51b0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a365ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efeaf7fe640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 0x7efea40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 0x7efea40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efebe8b1640 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efeb0035000 tx=0x7efeb0009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d41a642ed489c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efeaf7fe640 1 -- 172.21.15.94:0/3188987349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efea80df0b0 con 0x7efeb81039b0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.007+0000 7efeacbfe640 1 -- 172.21.15.94:0/3215019698 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0031bd0 con 0x557009fc51b0 2024-04-25T14:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efeacbfe640 1 -- 172.21.15.94:0/3215019698 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb0036030 con 0x557009fc51b0 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efeacbfe640 1 -- 172.21.15.94:0/3215019698 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0004580 con 0x557009fc51b0 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 -- 172.21.15.94:0/3215019698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 msgr2=0x55700a365970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 0x7efea40b1ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efeb400dfd0 tx=0x7efeb400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efeb0035000 tx=0x7efeb0009400 comp rx=0 tx=0).stop 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 -- 172.21.15.94:0/3215019698 shutdown_connections 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a365ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 --2- 172.21.15.94:0/3215019698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a365970 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.008+0000 7efec71c7740 1 -- 172.21.15.94:0/3215019698 >> 172.21.15.94:0/3215019698 conn(0x557009fb9300 msgr2=0x55700a361300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.009+0000 7efec71c7740 1 -- 172.21.15.94:0/3215019698 shutdown_connections 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.009+0000 7efec71c7740 1 -- 172.21.15.94:0/3215019698 wait complete. 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.009+0000 7efec71c7740 1 -- start start 2024-04-25T14:02:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.009+0000 7efec71c7740 1 Processor -- start 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.009+0000 7efec71c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a36f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a376b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55700a36d7e0 con 0x557009fa44d0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55700a252b60 con 0x557009fc51b0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55700a36d960 con 0x557009f9c2f0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebe0b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a36f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58662/0 (socket says 172.21.15.94:58662) 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 -- 172.21.15.94:0/937566869 learned_addr learned my addr 172.21.15.94:0/937566869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 10 client.0 _ll_register_callbacks cb 0x55700a2772b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebe8b1640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a376b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 -- 172.21.15.94:0/937566869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 msgr2=0x55700a36f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a36f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 -- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 msgr2=0x55700a376b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a376b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 -- 172.21.15.94:0/937566869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557009fdd210 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efebd8af640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7efea80a71f0 tx=0x7efea80e2780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20cb75c5984ce99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80a72c0 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb0036030 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.010+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efea8010a90 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efea80a6c60 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 10 client.23007 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55700a0a9620 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb0036030 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efea80e1070 con 0x557009fa44d0 2024-04-25T14:02:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 10 client.23007 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 10 client.23007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 20 client.23007 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f83d80 0x55700a37d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 4 client.23007 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55700a2d9070 con 0x557009f83d80 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.011+0000 7efebe0b0640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f83d80 0x55700a37d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.012+0000 7efea1ffb640 10 client.23007 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.012+0000 7efebe0b0640 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f83d80 0x55700a37d840 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a76995d96fb0f4 server_cookie=5e70ae8e75e6c829 in_seq=0 out_seq=0 2024-04-25T14:02:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.018+0000 7efea1ffb640 1 -- 172.21.15.94:0/937566869 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55700a2d9070 con 0x557009f83d80 2024-04-25T14:02:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.018+0000 7efea1ffb640 10 client.23007 handle_command_reply: tid=0 2024-04-25T14:02:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.018+0000 7efec71c7740 1 client.23007 shutdown 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.018+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 msgr2=0x55700a36fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.018+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7efea80a71f0 tx=0x7efea80e2780 comp rx=0 tx=0).stop 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 shutdown_connections 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f83d80 0x55700a37d840 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a376b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a36fbb0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 --2- 172.21.15.94:0/937566869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a36f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 >> 172.21.15.94:0/937566869 conn(0x557009fb9300 msgr2=0x55700a360250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efea1ffb640 0 client.23007 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 shutdown_connections 2024-04-25T14:02:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 1 -- 172.21.15.94:0/937566869 wait complete. 2024-04-25T14:02:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.019+0000 7efec71c7740 20 client.23007 trim_cache size 0 max 16384 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 Processor -- start 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 -- start start 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fa44d0 0x55700a36a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009fc51b0 0x55700a360cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81a9330 con 0x557009f9c2f0 2024-04-25T14:02:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81aae70 con 0x557009fa44d0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efec71c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81a9a00 con 0x557009fc51b0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.037+0000 7efebe0b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58666/0 (socket says 172.21.15.94:58666) 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 -- 172.21.15.94:0/21542727 learned_addr learned my addr 172.21.15.94:0/21542727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe8b1640 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009fc51b0 0x55700a360cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebd8af640 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fa44d0 0x55700a36a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 -- 172.21.15.94:0/21542727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009fc51b0 msgr2=0x55700a360cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009fc51b0 0x55700a360cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 -- 172.21.15.94:0/21542727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fa44d0 msgr2=0x55700a36a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fa44d0 0x55700a36a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 -- 172.21.15.94:0/21542727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55700a361e90 con 0x557009f9c2f0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efebe0b0640 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7efeb4019650 tx=0x7efeb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc276fc01199a3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efea3fff640 1 -- 172.21.15.94:0/21542727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4020040 con 0x557009f9c2f0 2024-04-25T14:02:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efea3fff640 1 -- 172.21.15.94:0/21542727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb4021040 con 0x557009f9c2f0 2024-04-25T14:02:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.038+0000 7efea3fff640 1 -- 172.21.15.94:0/21542727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40279a0 con 0x557009f9c2f0 2024-04-25T14:02:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 -- 172.21.15.94:0/21542727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 msgr2=0x55700a369b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7efeb4019650 tx=0x7efeb4017000 comp rx=0 tx=0).stop 2024-04-25T14:02:53.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 -- 172.21.15.94:0/21542727 shutdown_connections 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009fc51b0 0x55700a360cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fa44d0 0x55700a36a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 --2- 172.21.15.94:0/21542727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009f9c2f0 0x55700a369b70 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 -- 172.21.15.94:0/21542727 >> 172.21.15.94:0/21542727 conn(0x557009fb9300 msgr2=0x55700a36dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 -- 172.21.15.94:0/21542727 shutdown_connections 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.039+0000 7efec71c7740 1 -- 172.21.15.94:0/21542727 wait complete. 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 -- start start 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 Processor -- start 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a3732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a374730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81aa7a0 con 0x557009fa44d0 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81aa0d0 con 0x557009fc51b0 2024-04-25T14:02:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81a7310 con 0x557009f9c2f0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efec71c7740 10 client.0 _ll_register_callbacks cb 0x7efea800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58668/0 (socket says 172.21.15.94:58668) 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 -- 172.21.15.94:0/3526517569 learned_addr learned my addr 172.21.15.94:0/3526517569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebe0b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a3732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebe8b1640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a374730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 -- 172.21.15.94:0/3526517569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 msgr2=0x55700a3732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a3732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 -- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 msgr2=0x55700a374730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a374730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.040+0000 7efebd8af640 1 -- 172.21.15.94:0/3526517569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55700a374cf0 con 0x557009fa44d0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efebd8af640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7efea80a3570 tx=0x7efea8010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1967d0f57ab21c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80a6e40 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efea80a2e60 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb4021040 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efea80a72c0 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efec71c7740 10 client.23013 fetch_fsmap learned FSMap version 39 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55700a375ff0 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb4021040 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efea80a3060 con 0x557009fa44d0 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efec71c7740 10 client.23013 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:02:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.041+0000 7efec71c7740 10 client.23013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efec71c7740 20 client.23013 populate_metadata read hostname 'smithi094' 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f591c0 0x55700a3e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efec71c7740 4 client.23013 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55700a2d2ee0 con 0x557009f591c0 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f591c0 0x55700a3e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efea1ffb640 10 client.23013 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.042+0000 7efebe0b0640 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f591c0 0x55700a3e7fa0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d80ff938a19e94 server_cookie=c3749604287ac222 in_seq=0 out_seq=0 2024-04-25T14:02:53.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 162370 18.94 MB/sec execute 335 sec latency 117.810 ms 2024-04-25T14:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.619+0000 7efea1ffb640 1 -- 172.21.15.94:0/3526517569 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55700a2d2ee0 con 0x557009f591c0 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efea1ffb640 10 client.23013 handle_command_reply: tid=0 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 client.23013 shutdown 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 msgr2=0x55700a373800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7efea80a3570 tx=0x7efea8010ca0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 shutdown_connections 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557009f591c0 0x55700a3e7fa0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557009fc51b0 0x55700a374730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efea1ffb640 0 client.23013 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557009fa44d0 0x55700a373800 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 --2- 172.21.15.94:0/3526517569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557009f9c2f0 0x55700a3732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 >> 172.21.15.94:0/3526517569 conn(0x557009fb9300 msgr2=0x55700a36f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 shutdown_connections 2024-04-25T14:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 1 -- 172.21.15.94:0/3526517569 wait complete. 2024-04-25T14:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.620+0000 7efec71c7740 20 client.23013 trim_cache size 0 max 16384 2024-04-25T14:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:02:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:02:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 msgr2=0x7efea40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 0x7efea40b1ff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efeb400dfd0 tx=0x7efeb400f040 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 msgr2=0x7efeb819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7efea800e9f0 tx=0x7efea800edf0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 shutdown_connections 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efea40afae0 0x7efea40b1ff0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efeb81044b0 0x7efeb81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb81039b0 0x7efeb819a740 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 --2- 172.21.15.94:0/3188987349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeb8102fd0 0x7efeb819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 >> 172.21.15.94:0/3188987349 conn(0x7efeb80fd390 msgr2=0x7efeb8106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 shutdown_connections 2024-04-25T14:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:02:53.621+0000 7efec0314640 1 -- 172.21.15.94:0/3188987349 wait complete. 2024-04-25T14:02:54.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 162736 18.95 MB/sec execute 336 sec latency 574.037 ms 2024-04-25T14:02:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:54 smithi137 ceph-mon[32568]: pgmap v984: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T14:02:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:54 smithi094 ceph-mon[29195]: pgmap v984: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T14:02:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:54 smithi138 ceph-mon[32977]: pgmap v984: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T14:02:55.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 163306 18.97 MB/sec execute 337 sec latency 46.742 ms 2024-04-25T14:02:56.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 163530 18.98 MB/sec execute 338 sec latency 401.923 ms 2024-04-25T14:02:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:56 smithi137 ceph-mon[32568]: pgmap v985: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 626 op/s 2024-04-25T14:02:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:56 smithi094 ceph-mon[29195]: pgmap v985: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 626 op/s 2024-04-25T14:02:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:56 smithi138 ceph-mon[32977]: pgmap v985: 97 pgs: 97 active+clean; 269 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.4 MiB/s wr, 626 op/s 2024-04-25T14:02:57.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 163871 18.99 MB/sec execute 339 sec latency 70.498 ms 2024-04-25T14:02:58.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 164204 19.00 MB/sec execute 340 sec latency 104.415 ms 2024-04-25T14:02:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:02:58 smithi137 ceph-mon[32568]: pgmap v986: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-04-25T14:02:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:02:58 smithi094 ceph-mon[29195]: pgmap v986: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-04-25T14:02:58.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:02:58 smithi138 ceph-mon[32977]: pgmap v986: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-04-25T14:02:59.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 164440 18.98 MB/sec execute 341 sec latency 134.721 ms 2024-04-25T14:03:00.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 164706 18.95 MB/sec execute 342 sec latency 57.442 ms 2024-04-25T14:03:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:00 smithi137 ceph-mon[32568]: pgmap v987: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 479 op/s 2024-04-25T14:03:00.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:00 smithi094 ceph-mon[29195]: pgmap v987: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 479 op/s 2024-04-25T14:03:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:00 smithi138 ceph-mon[32977]: pgmap v987: 97 pgs: 97 active+clean; 263 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 479 op/s 2024-04-25T14:03:01.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 164947 18.92 MB/sec execute 343 sec latency 218.527 ms 2024-04-25T14:03:02.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 165257 18.89 MB/sec execute 344 sec latency 191.666 ms 2024-04-25T14:03:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:02 smithi137 ceph-mon[32568]: pgmap v988: 97 pgs: 97 active+clean; 267 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-04-25T14:03:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:02 smithi094 ceph-mon[29195]: pgmap v988: 97 pgs: 97 active+clean; 267 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-04-25T14:03:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:02 smithi138 ceph-mon[32977]: pgmap v988: 97 pgs: 97 active+clean; 267 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-04-25T14:03:03.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 165719 18.91 MB/sec execute 345 sec latency 99.725 ms 2024-04-25T14:03:04.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 166296 18.96 MB/sec execute 346 sec latency 71.179 ms 2024-04-25T14:03:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:04 smithi137 ceph-mon[32568]: pgmap v989: 97 pgs: 97 active+clean; 265 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 489 op/s 2024-04-25T14:03:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:04 smithi094 ceph-mon[29195]: pgmap v989: 97 pgs: 97 active+clean; 265 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 489 op/s 2024-04-25T14:03:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:04 smithi138 ceph-mon[32977]: pgmap v989: 97 pgs: 97 active+clean; 265 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 489 op/s 2024-04-25T14:03:05.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 166851 18.98 MB/sec execute 347 sec latency 64.489 ms 2024-04-25T14:03:06.153 INFO:tasks.workunit.client.0.smithi094.stdout: 10 167097 18.98 MB/sec execute 348 sec latency 300.861 ms 2024-04-25T14:03:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:06 smithi137 ceph-mon[32568]: pgmap v990: 97 pgs: 97 active+clean; 271 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-04-25T14:03:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:06 smithi094 ceph-mon[29195]: pgmap v990: 97 pgs: 97 active+clean; 271 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 -- 172.21.15.94:0/3417507551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e01050e0 msgr2=0x7f55e01054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 --2- 172.21.15.94:0/3417507551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e01050e0 0x7f55e01054f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009900 tx=0x7f55d002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 -- 172.21.15.94:0/3417507551 shutdown_connections 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 --2- 172.21.15.94:0/3417507551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e0102290 0x7f55e010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 --2- 172.21.15.94:0/3417507551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 0x7f55e0101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 --2- 172.21.15.94:0/3417507551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e01050e0 0x7f55e01054f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 -- 172.21.15.94:0/3417507551 >> 172.21.15.94:0/3417507551 conn(0x7f55e00fd390 msgr2=0x7f55e00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.733+0000 7f55e5557640 1 -- 172.21.15.94:0/3417507551 shutdown_connections 2024-04-25T14:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.734+0000 7f55e5557640 1 -- 172.21.15.94:0/3417507551 wait complete. 2024-04-25T14:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 Processor -- start 2024-04-25T14:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 -- start start 2024-04-25T14:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 0x7f55e0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e01050e0 0x7f55e01a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f6a0 con 0x7f55e01050e0 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f520 con 0x7f55e0101790 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55e5557640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f820 con 0x7f55e0102290 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58876/0 (socket says 172.21.15.94:58876) 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 0x7f55e0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.735+0000 7f55de7fc640 1 -- 172.21.15.94:0/188181402 learned_addr learned my addr 172.21.15.94:0/188181402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55df7fe640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e01050e0 0x7f55e01a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 -- 172.21.15.94:0/188181402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 msgr2=0x7f55e0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 0x7f55e0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 -- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e01050e0 msgr2=0x7f55e01a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e01050e0 0x7f55e01a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 -- 172.21.15.94:0/188181402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e01a8210 con 0x7f55e0102290 2024-04-25T14:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55de7fc640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e9f0 tx=0x7f55d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55c7fff640 1 -- 172.21.15.94:0/188181402 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d400ccc0 con 0x7f55e0102290 2024-04-25T14:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55c7fff640 1 -- 172.21.15.94:0/188181402 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d400ce60 con 0x7f55e0102290 2024-04-25T14:03:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.736+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01a8500 con 0x7f55e0102290 2024-04-25T14:03:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.737+0000 7f55c7fff640 1 -- 172.21.15.94:0/188181402 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4010430 con 0x7f55e0102290 2024-04-25T14:03:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.737+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e01a89d0 con 0x7f55e0102290 2024-04-25T14:03:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.740+0000 7f55c7fff640 1 -- 172.21.15.94:0/188181402 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f55d40046c0 con 0x7f55e0102290 2024-04-25T14:03:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.741+0000 7f55c7fff640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 0x7f55c80b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.741+0000 7f55c7fff640 1 -- 172.21.15.94:0/188181402 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55d40df0c0 con 0x7f55e0102290 2024-04-25T14:03:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55deffd640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 0x7f55c80b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55ec406740 1 Processor -- start 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55ec406740 1 -- start start 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55deffd640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 0x7f55c80b20a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009850 tx=0x7f55d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55ec406740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f630f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55ec406740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.742+0000 7f55ec406740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55ec406740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f63057460 con 0x560f62d282f0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55ec406740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f63011590 con 0x560f62d511b0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55ec406740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f63053420 con 0x560f62d304d0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58892/0 (socket says 172.21.15.94:58892) 2024-04-25T14:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55df7fe640 1 -- 172.21.15.94:0/2427240270 learned_addr learned my addr 172.21.15.94:0/2427240270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55de7fc640 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.743+0000 7f55deffd640 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f630f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.744+0000 7f55df7fe640 1 -- 172.21.15.94:0/2427240270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 msgr2=0x560f630f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.744+0000 7f55df7fe640 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f630f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.744+0000 7f55df7fe640 1 -- 172.21.15.94:0/2427240270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 msgr2=0x560f630f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.744+0000 7f55df7fe640 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.744+0000 7f55df7fe640 1 -- 172.21.15.94:0/2427240270 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6301e0d0 con 0x560f62d304d0 2024-04-25T14:03:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.745+0000 7f55df7fe640 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f55d8011440 tx=0x7f55d8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a052e1b641781240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.746+0000 7f55c57fa640 1 -- 172.21.15.94:0/2427240270 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8003c30 con 0x560f62d304d0 2024-04-25T14:03:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.746+0000 7f55c57fa640 1 -- 172.21.15.94:0/2427240270 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d8003dd0 con 0x560f62d304d0 2024-04-25T14:03:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.746+0000 7f55c57fa640 1 -- 172.21.15.94:0/2427240270 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d800fae0 con 0x560f62d304d0 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.747+0000 7f55ec406740 1 -- 172.21.15.94:0/2427240270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 msgr2=0x560f630f8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.747+0000 7f55ec406740 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f55d8011440 tx=0x7f55d8011900 comp rx=0 tx=0).stop 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.747+0000 7f55ec406740 1 -- 172.21.15.94:0/2427240270 shutdown_connections 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.747+0000 7f55ec406740 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d304d0 0x560f630f8df0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.747+0000 7f55ec406740 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.748+0000 7f55ec406740 1 --2- 172.21.15.94:0/2427240270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f630f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.748+0000 7f55ec406740 1 -- 172.21.15.94:0/2427240270 >> 172.21.15.94:0/2427240270 conn(0x560f62d45300 msgr2=0x560f630ed300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:06.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:06 smithi138 ceph-mon[32977]: pgmap v990: 97 pgs: 97 active+clean; 271 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 574 op/s 2024-04-25T14:03:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.748+0000 7f55ec406740 1 -- 172.21.15.94:0/2427240270 shutdown_connections 2024-04-25T14:03:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.748+0000 7f55ec406740 1 -- 172.21.15.94:0/2427240270 wait complete. 2024-04-25T14:03:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.749+0000 7f55ec406740 1 -- start start 2024-04-25T14:03:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.749+0000 7f55ec406740 1 Processor -- start 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f630fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560f630f9890 con 0x560f62d304d0 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560f62fdeb60 con 0x560f62d511b0 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f630f9a10 con 0x560f62d282f0 2024-04-25T14:03:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.750+0000 7f55ec406740 10 client.0 _ll_register_callbacks cb 0x560f630032b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39648/0 (socket says 172.21.15.94:39648) 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 -- 172.21.15.94:0/1507654255 learned_addr learned my addr 172.21.15.94:0/1507654255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55deffd640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55de7fc640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f630fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 -- 172.21.15.94:0/1507654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 msgr2=0x560f630fb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630fb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 -- 172.21.15.94:0/1507654255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 msgr2=0x560f630fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f630fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.751+0000 7f55df7fe640 1 -- 172.21.15.94:0/1507654255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f62d69210 con 0x560f62d511b0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55df7fe640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55d800f580 tx=0x7f55d8004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6f74962ce320f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8029020 con 0x560f62d511b0 2024-04-25T14:03:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55d800fa20 con 0x560f62d511b0 2024-04-25T14:03:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d8003dd0 con 0x560f62d511b0 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55d8034050 con 0x560f62d511b0 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55ec406740 10 client.32411 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.752+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560f62e35620 con 0x560f62d511b0 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d8003dd0 con 0x560f62d511b0 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f55d8005040 con 0x560f62d511b0 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 10 client.32411 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 10 client.32411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 20 client.32411 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62d0fd80 0x560f63109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 4 client.32411 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f63065070 con 0x560f62d0fd80 2024-04-25T14:03:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.753+0000 7f55deffd640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62d0fd80 0x560f63109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.754+0000 7f55c2ff5640 10 client.32411 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.754+0000 7f55deffd640 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62d0fd80 0x560f63109130 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d3f378c4dcaa9d server_cookie=1c4f7a1f5a77ca4c in_seq=0 out_seq=0 2024-04-25T14:03:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.758+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1507654255 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f63065070 con 0x560f62d0fd80 2024-04-25T14:03:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.758+0000 7f55c2ff5640 10 client.32411 handle_command_reply: tid=0 2024-04-25T14:03:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.758+0000 7f55ec406740 1 client.32411 shutdown 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.758+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 msgr2=0x560f63102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.758+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55d800f580 tx=0x7f55d8004ce0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 shutdown_connections 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62d0fd80 0x560f63109130 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f63102b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f630fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 --2- 172.21.15.94:0/1507654255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 >> 172.21.15.94:0/1507654255 conn(0x560f62d45300 msgr2=0x560f630ec250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55c2ff5640 0 client.32411 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 shutdown_connections 2024-04-25T14:03:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 1 -- 172.21.15.94:0/1507654255 wait complete. 2024-04-25T14:03:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.759+0000 7f55ec406740 20 client.32411 trim_cache size 0 max 16384 2024-04-25T14:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 Processor -- start 2024-04-25T14:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 -- start start 2024-04-25T14:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d304d0 0x560f63102a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d511b0 0x560f630ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b1a10 con 0x560f62d282f0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b3550 con 0x560f62d304d0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.778+0000 7f55ec406740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b20e0 con 0x560f62d511b0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40240/0 (socket says 172.21.15.94:40240) 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 -- 172.21.15.94:0/756735586 learned_addr learned my addr 172.21.15.94:0/756735586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55df7fe640 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d511b0 0x560f630ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55de7fc640 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d304d0 0x560f63102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 -- 172.21.15.94:0/756735586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d511b0 msgr2=0x560f630ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d511b0 0x560f630ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 -- 172.21.15.94:0/756735586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d304d0 msgr2=0x560f63102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d304d0 0x560f63102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 -- 172.21.15.94:0/756735586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f630f1b20 con 0x560f62d282f0 2024-04-25T14:03:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55deffd640 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009900 tx=0x7f55d0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4699ebfb2c40f692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55c4ff9640 1 -- 172.21.15.94:0/756735586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0035040 con 0x560f62d282f0 2024-04-25T14:03:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55c4ff9640 1 -- 172.21.15.94:0/756735586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d0058070 con 0x560f62d282f0 2024-04-25T14:03:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.779+0000 7f55c4ff9640 1 -- 172.21.15.94:0/756735586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00314a0 con 0x560f62d282f0 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- 172.21.15.94:0/756735586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 msgr2=0x560f630fbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009900 tx=0x7f55d0040000 comp rx=0 tx=0).stop 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- 172.21.15.94:0/756735586 shutdown_connections 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d511b0 0x560f630ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d304d0 0x560f63102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 --2- 172.21.15.94:0/756735586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d282f0 0x560f630fbde0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- 172.21.15.94:0/756735586 >> 172.21.15.94:0/756735586 conn(0x560f62d45300 msgr2=0x560f630fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- 172.21.15.94:0/756735586 shutdown_connections 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- 172.21.15.94:0/756735586 wait complete. 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.781+0000 7f55ec406740 1 -- start start 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 Processor -- start 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f63100000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f6316f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b2e80 con 0x560f62d304d0 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b27b0 con 0x560f62d511b0 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01af9f0 con 0x560f62d282f0 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55ec406740 10 client.0 _ll_register_callbacks cb 0x7f55d4005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55deffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58928/0 (socket says 172.21.15.94:58928) 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55deffd640 1 -- 172.21.15.94:0/1546025138 learned_addr learned my addr 172.21.15.94:0/1546025138 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55de7fc640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f63100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.782+0000 7f55df7fe640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f6316f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 -- 172.21.15.94:0/1546025138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 msgr2=0x560f630ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 -- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 msgr2=0x560f6316f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f6316f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 -- 172.21.15.94:0/1546025138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f63102fd0 con 0x560f62d304d0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55deffd640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55de7fc640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f63100000 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a71a0 tx=0x7f55d40a7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28597dbd755ebd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40c0070 con 0x560f62d304d0 2024-04-25T14:03:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55d40afe70 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0058070 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55d40ab850 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.783+0000 7f55ec406740 10 client.23031 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560f63170b30 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40b0300 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f55d40a6170 con 0x560f62d304d0 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 10 client.23031 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 10 client.23031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 20 client.23031 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62ce51c0 0x560f63173520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 4 client.23031 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.784+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f55d0058070 con 0x560f62ce51c0 2024-04-25T14:03:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.786+0000 7f55deffd640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62ce51c0 0x560f63173520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.787+0000 7f55deffd640 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62ce51c0 0x560f63173520 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d42c975e266603f server_cookie=7b04092e9fe2d08 in_seq=0 out_seq=0 2024-04-25T14:03:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:06.787+0000 7f55c2ff5640 10 client.23031 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:07.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:07.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 167367 18.99 MB/sec execute 349 sec latency 222.095 ms 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 -- 172.21.15.94:0/1531933485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324102240 msgr2=0x7f732410bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 --2- 172.21.15.94:0/1531933485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324102240 0x7f732410bca0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f731c009ca0 tx=0x7f731c02a8e0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 -- 172.21.15.94:0/1531933485 shutdown_connections 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 --2- 172.21.15.94:0/1531933485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324102240 0x7f732410bca0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 --2- 172.21.15.94:0/1531933485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324101740 0x7f7324101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 --2- 172.21.15.94:0/1531933485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f73241054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.311+0000 7f732a8ee640 1 -- 172.21.15.94:0/1531933485 >> 172.21.15.94:0/1531933485 conn(0x7f73240fd340 msgr2=0x7f73240ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.312+0000 7f732a8ee640 1 -- 172.21.15.94:0/1531933485 shutdown_connections 2024-04-25T14:03:07.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.312+0000 7f732a8ee640 1 -- 172.21.15.94:0/1531933485 wait complete. 2024-04-25T14:03:07.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.313+0000 7f732a8ee640 1 Processor -- start 2024-04-25T14:03:07.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.313+0000 7f732a8ee640 1 -- start start 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 0x7f7324104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f732410f6d0 con 0x7f7324105090 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f732410f550 con 0x7f7324102240 2024-04-25T14:03:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f732a8ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f732410f850 con 0x7f7324101740 2024-04-25T14:03:07.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f73237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40276/0 (socket says 172.21.15.94:40276) 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f73237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39698/0 (socket says 172.21.15.94:39698) 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f7328e8b640 1 -- 172.21.15.94:0/2090094541 learned_addr learned my addr 172.21.15.94:0/2090094541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.314+0000 7f7323fff640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 0x7f7324104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 -- 172.21.15.94:0/2090094541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 msgr2=0x7f7324104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 0x7f7324104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 -- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 msgr2=0x7f7324102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 -- 172.21.15.94:0/2090094541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73241a9fd0 con 0x7f7324102240 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f7323fff640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 0x7f7324104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f73237fe640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f731400e9f0 tx=0x7f731400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.315+0000 7f7328e8b640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:07.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.316+0000 7f73217fa640 1 -- 172.21.15.94:0/2090094541 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400ccc0 con 0x7f7324102240 2024-04-25T14:03:07.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.316+0000 7f73217fa640 1 -- 172.21.15.94:0/2090094541 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73140108e0 con 0x7f7324102240 2024-04-25T14:03:07.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.316+0000 7f73217fa640 1 -- 172.21.15.94:0/2090094541 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140188e0 con 0x7f7324102240 2024-04-25T14:03:07.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.316+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241aa2c0 con 0x7f7324102240 2024-04-25T14:03:07.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.316+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241aa840 con 0x7f7324102240 2024-04-25T14:03:07.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.321+0000 7f73217fa640 1 -- 172.21.15.94:0/2090094541 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7314010430 con 0x7f7324102240 2024-04-25T14:03:07.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.323+0000 7f73217fa640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 0x7f730c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.323+0000 7f73217fa640 1 -- 172.21.15.94:0/2090094541 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f73140df010 con 0x7f7324102240 2024-04-25T14:03:07.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.323+0000 7f7323fff640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 0x7f730c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 Processor -- start 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 -- start start 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f7323fff640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 0x7f730c0b21d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417864390 0x562417c05a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c05fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562417b6ad80 con 0x56241783b4d0 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562417b248b0 con 0x562417864390 2024-04-25T14:03:07.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f733179e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562417b66d40 con 0x5624178436b0 2024-04-25T14:03:07.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f73237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c05fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58946/0 (socket says 172.21.15.94:58946) 2024-04-25T14:03:07.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.324+0000 7f7328e8b640 1 -- 172.21.15.94:0/757320204 learned_addr learned my addr 172.21.15.94:0/757320204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 -- 172.21.15.94:0/757320204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417864390 msgr2=0x562417c05a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417864390 0x562417c05a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 -- 172.21.15.94:0/757320204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 msgr2=0x562417c05fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c05fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 -- 172.21.15.94:0/757320204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562417af5a40 con 0x5624178436b0 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f7328e8b640 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f731c009940 tx=0x7f731c009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f3dfc151ff15e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.325+0000 7f730a7fc640 1 -- 172.21.15.94:0/757320204 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c003970 con 0x5624178436b0 2024-04-25T14:03:07.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.326+0000 7f730a7fc640 1 -- 172.21.15.94:0/757320204 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731c036040 con 0x5624178436b0 2024-04-25T14:03:07.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.326+0000 7f730a7fc640 1 -- 172.21.15.94:0/757320204 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c030840 con 0x5624178436b0 2024-04-25T14:03:07.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 -- 172.21.15.94:0/757320204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 msgr2=0x562417c06c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f731c009940 tx=0x7f731c009970 comp rx=0 tx=0).stop 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 -- 172.21.15.94:0/757320204 shutdown_connections 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c06c70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c05fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 --2- 172.21.15.94:0/757320204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417864390 0x562417c05a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.327+0000 7f733179e740 1 -- 172.21.15.94:0/757320204 >> 172.21.15.94:0/757320204 conn(0x5624178584e0 msgr2=0x562417c01270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.328+0000 7f733179e740 1 -- 172.21.15.94:0/757320204 shutdown_connections 2024-04-25T14:03:07.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.328+0000 7f733179e740 1 -- 172.21.15.94:0/757320204 wait complete. 2024-04-25T14:03:07.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.329+0000 7f733179e740 1 -- start start 2024-04-25T14:03:07.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.329+0000 7f733179e740 1 Processor -- start 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56241783b4d0 0x562417c0f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562417864390 0x562417c16d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562417c07770 con 0x5624178436b0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f73237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f73237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40302/0 (socket says 172.21.15.94:40302) 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f73237fe640 1 -- 172.21.15.94:0/2560176472 learned_addr learned my addr 172.21.15.94:0/2560176472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f7323fff640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56241783b4d0 0x562417c0f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562417af1d10 con 0x56241783b4d0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.330+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562417c078f0 con 0x562417864390 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f733179e740 10 client.0 _ll_register_callbacks cb 0x562417b1e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f7328e8b640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562417864390 0x562417c16d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 -- 172.21.15.94:0/2560176472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562417864390 msgr2=0x562417c16d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562417864390 0x562417c16d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 -- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56241783b4d0 msgr2=0x562417c0f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56241783b4d0 0x562417c0f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624179467d0 con 0x5624178436b0 2024-04-25T14:03:07.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73237fe640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f73140dca10 tx=0x7f7314010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331080d834b8154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.331+0000 7f73037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:07.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140ab7a0 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c036040 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f73140a39e0 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73140b0250 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f733179e740 10 client.23040 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56241787c3f0 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.332+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731c036040 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f73140c1ce0 con 0x5624178436b0 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 10 client.23040 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 10 client.23040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 20 client.23040 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417822f60 0x562417c1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 4 client.23040 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:07.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.333+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562417b78990 con 0x562417822f60 2024-04-25T14:03:07.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.334+0000 7f7323fff640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417822f60 0x562417c1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.336+0000 7f7323fff640 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417822f60 0x562417c1d240 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3aa254f40af2d server_cookie=f65e403b7220d28c in_seq=0 out_seq=0 2024-04-25T14:03:07.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.337+0000 7f73037fe640 10 client.23040 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55c2ff5640 1 -- 172.21.15.94:0/1546025138 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f55d0058070 con 0x560f62ce51c0 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55c2ff5640 10 client.23031 handle_command_reply: tid=0 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 client.23031 shutdown 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 msgr2=0x560f63100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f63100000 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a71a0 tx=0x7f55d40a7a40 comp rx=0 tx=0).stop 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 shutdown_connections 2024-04-25T14:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560f62ce51c0 0x560f63173520 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f62d511b0 0x560f6316f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560f62d304d0 0x560f63100000 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 --2- 172.21.15.94:0/1546025138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f62d282f0 0x560f630ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 >> 172.21.15.94:0/1546025138 conn(0x560f62d45300 msgr2=0x560f630ecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55c2ff5640 0 client.23031 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 shutdown_connections 2024-04-25T14:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f55ec406740 1 -- 172.21.15.94:0/1546025138 wait complete. 2024-04-25T14:03:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f73037fe640 1 -- 172.21.15.94:0/2560176472 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562417b78990 con 0x562417822f60 2024-04-25T14:03:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f73037fe640 10 client.23040 handle_command_reply: tid=0 2024-04-25T14:03:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.483+0000 7f733179e740 1 client.23040 shutdown 2024-04-25T14:03:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55ec406740 20 client.23031 trim_cache size 0 max 16384 2024-04-25T14:03:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:03:07.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 msgr2=0x562417c0fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f73140dca10 tx=0x7f7314010860 comp rx=0 tx=0).stop 2024-04-25T14:03:07.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 shutdown_connections 2024-04-25T14:03:07.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417822f60 0x562417c1d240 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562417864390 0x562417c16d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417c0fe20 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f73037fe640 0 client.23040 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 --2- 172.21.15.94:0/2560176472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56241783b4d0 0x562417c0f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 >> 172.21.15.94:0/2560176472 conn(0x5624178584e0 msgr2=0x562417c003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 shutdown_connections 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 1 -- 172.21.15.94:0/2560176472 wait complete. 2024-04-25T14:03:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f733179e740 20 client.23040 trim_cache size 0 max 16384 2024-04-25T14:03:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 msgr2=0x7f55c80b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 0x7f55c80b20a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009850 tx=0x7f55d0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 msgr2=0x7f55e0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e9f0 tx=0x7f55d400edf0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 shutdown_connections 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55c80afb90 0x7f55c80b20a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55e01050e0 0x7f55e01a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e0102290 0x7f55e0198410 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 --2- 172.21.15.94:0/188181402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e0101790 0x7f55e0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.484+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 >> 172.21.15.94:0/188181402 conn(0x7f55e00fd390 msgr2=0x7f55e00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.485+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 shutdown_connections 2024-04-25T14:03:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:07.485+0000 7f55e5557640 1 -- 172.21.15.94:0/188181402 wait complete. 2024-04-25T14:03:07.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.506+0000 7f733179e740 1 Processor -- start 2024-04-25T14:03:07.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.506+0000 7f733179e740 1 -- start start 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.506+0000 7f733179e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f733179e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56241783b4d0 0x562417c0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f733179e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f733179e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b1be0 con 0x5624178436b0 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f733179e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73241c4740 con 0x562417822f60 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f733179e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b22b0 con 0x56241783b4d0 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40310/0 (socket says 172.21.15.94:40310) 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7323fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 -- 172.21.15.94:0/1005175963 learned_addr learned my addr 172.21.15.94:0/1005175963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f73237fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56241783b4d0 0x562417c0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 -- 172.21.15.94:0/1005175963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56241783b4d0 msgr2=0x562417c0e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56241783b4d0 0x562417c0e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 -- 172.21.15.94:0/1005175963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 msgr2=0x562417c0df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c0df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 -- 172.21.15.94:0/1005175963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241783c0f0 con 0x5624178436b0 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.507+0000 7f7328e8b640 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f731c0099e0 tx=0x7f731c031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f7380705241d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f7309ffb640 1 -- 172.21.15.94:0/1005175963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c052070 con 0x5624178436b0 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f7309ffb640 1 -- 172.21.15.94:0/1005175963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731c036070 con 0x5624178436b0 2024-04-25T14:03:07.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f7309ffb640 1 -- 172.21.15.94:0/1005175963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0045e0 con 0x5624178436b0 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 -- 172.21.15.94:0/1005175963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 msgr2=0x562417bff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f731c0099e0 tx=0x7f731c031830 comp rx=0 tx=0).stop 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 -- 172.21.15.94:0/1005175963 shutdown_connections 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624178436b0 0x562417bff9c0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56241783b4d0 0x562417c0e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 --2- 172.21.15.94:0/1005175963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.508+0000 7f733179e740 1 -- 172.21.15.94:0/1005175963 >> 172.21.15.94:0/1005175963 conn(0x5624178584e0 msgr2=0x562417c0a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- 172.21.15.94:0/1005175963 shutdown_connections 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- 172.21.15.94:0/1005175963 wait complete. 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- start start 2024-04-25T14:03:07.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 Processor -- start 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c13a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73241c4070 con 0x56241783b4d0 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73241c39f0 con 0x562417822f60 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73241afbc0 con 0x5624178436b0 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f733179e740 10 client.0 _ll_register_callbacks cb 0x7f73140a6b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f73237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f73237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40316/0 (socket says 172.21.15.94:40316) 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.509+0000 7f73237fe640 1 -- 172.21.15.94:0/3976465117 learned_addr learned my addr 172.21.15.94:0/3976465117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f7323fff640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c13a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f7328e8b640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 -- 172.21.15.94:0/3976465117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 msgr2=0x562417c14e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c14e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 -- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 msgr2=0x562417c13a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c13a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 -- 172.21.15.94:0/3976465117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562417b2b4b0 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73237fe640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f73140a4660 tx=0x7f7314010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2526d9999ecf528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140a32b0 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f73140a39e0 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c036070 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73140aed30 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f733179e740 10 client.23046 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562417c16580 con 0x56241783b4d0 2024-04-25T14:03:07.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.510+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f731c036070 con 0x56241783b4d0 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f73140c1ce0 con 0x56241783b4d0 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 10 client.23046 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 10 client.23046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 20 client.23046 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417864390 0x562417c87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 4 client.23046 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562417b72800 con 0x562417864390 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.511+0000 7f7323fff640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417864390 0x562417c87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.512+0000 7f73037fe640 10 client.23046 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.512+0000 7f7323fff640 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417864390 0x562417c87c90 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca18fd2d34ca95cc server_cookie=af098fcefd5d3f92 in_seq=0 out_seq=0 2024-04-25T14:03:07.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.519+0000 7f73037fe640 1 -- 172.21.15.94:0/3976465117 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562417b72800 con 0x562417864390 2024-04-25T14:03:07.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.519+0000 7f73037fe640 10 client.23046 handle_command_reply: tid=0 2024-04-25T14:03:07.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.519+0000 7f733179e740 1 client.23046 shutdown 2024-04-25T14:03:07.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 msgr2=0x562417c13fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f73140a4660 tx=0x7f7314010860 comp rx=0 tx=0).stop 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 shutdown_connections 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562417864390 0x562417c87c90 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624178436b0 0x562417c14e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56241783b4d0 0x562417c13fd0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f73037fe640 0 client.23046 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 --2- 172.21.15.94:0/3976465117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562417822f60 0x562417c13a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 >> 172.21.15.94:0/3976465117 conn(0x5624178584e0 msgr2=0x562417c00c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 shutdown_connections 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 1 -- 172.21.15.94:0/3976465117 wait complete. 2024-04-25T14:03:07.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f733179e740 20 client.23046 trim_cache size 0 max 16384 2024-04-25T14:03:07.521 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:03:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:03:07.521 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:07.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 msgr2=0x7f730c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 0x7f730c0b21d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318008040 comp rx=0 tx=0).stop 2024-04-25T14:03:07.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 msgr2=0x7f73241028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f731400e9f0 tx=0x7f731400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 shutdown_connections 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f730c0afcc0 0x7f730c0b21d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7324105090 0x7f7324102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.520+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7324102240 0x7f73241028e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.521+0000 7f732a8ee640 1 --2- 172.21.15.94:0/2090094541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7324101740 0x7f7324104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.521+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 >> 172.21.15.94:0/2090094541 conn(0x7f73240fd340 msgr2=0x7f7324107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.521+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 shutdown_connections 2024-04-25T14:03:07.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.521+0000 7f732a8ee640 1 -- 172.21.15.94:0/2090094541 wait complete. 2024-04-25T14:03:07.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:03:07.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 -- 172.21.15.94:0/3901892777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 msgr2=0x7fa53410bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 --2- 172.21.15.94:0/3901892777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa53410bc80 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009ca0 tx=0x7fa52c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 -- 172.21.15.94:0/3901892777 shutdown_connections 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 --2- 172.21.15.94:0/3901892777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa53410bc80 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 --2- 172.21.15.94:0/3901892777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534106c00 0x7fa5341090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.659+0000 7fa53bf84640 1 --2- 172.21.15.94:0/3901892777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534101e20 0x7fa534102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.660+0000 7fa53bf84640 1 -- 172.21.15.94:0/3901892777 >> 172.21.15.94:0/3901892777 conn(0x7fa5340fd330 msgr2=0x7fa5340ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.660+0000 7fa53bf84640 1 -- 172.21.15.94:0/3901892777 shutdown_connections 2024-04-25T14:03:07.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.660+0000 7fa53bf84640 1 -- 172.21.15.94:0/3901892777 wait complete. 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 Processor -- start 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 -- start start 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 0x7fa5341042f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534106c00 0x7fa534102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa534102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa53410f5e0 con 0x7fa534109770 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa53410f460 con 0x7fa534106c00 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.661+0000 7fa53bf84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa53410f760 con 0x7fa534101e20 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa539d20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 0x7fa5341042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa539d20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 0x7fa5341042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:58998/0 (socket says 172.21.15.94:58998) 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa539d20640 1 -- 172.21.15.94:0/4060280326 learned_addr learned my addr 172.21.15.94:0/4060280326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa534102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53951f640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534106c00 0x7fa534102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 -- 172.21.15.94:0/4060280326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 msgr2=0x7fa5341042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 0x7fa5341042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 -- 172.21.15.94:0/4060280326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534106c00 msgr2=0x7fa534102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534106c00 0x7fa534102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.662+0000 7fa53a521640 1 -- 172.21.15.94:0/4060280326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa534103600 con 0x7fa534109770 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa53a521640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa534102ed0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fa52c033040 tx=0x7fa52c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c003e40 con 0x7fa534109770 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa52c040070 con 0x7fa534109770 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c030bd0 con 0x7fa534109770 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa534073700 con 0x7fa534109770 2024-04-25T14:03:07.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.663+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa534073c00 con 0x7fa534109770 2024-04-25T14:03:07.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.665+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa534069310 con 0x7fa534109770 2024-04-25T14:03:07.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.667+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa52c030d70 con 0x7fa534109770 2024-04-25T14:03:07.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.668+0000 7fa52affd640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 0x7fa5200b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:07.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.668+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa52c0ffef0 con 0x7fa534109770 2024-04-25T14:03:07.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.668+0000 7fa539d20640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 0x7fa5200b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:07.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.669+0000 7fa539d20640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 0x7fa5200b2150 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa5300058c0 tx=0x7fa530005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:07.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.675+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa52c0c8a30 con 0x7fa534109770 2024-04-25T14:03:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.944+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa534074620 con 0x7fa534109770 2024-04-25T14:03:07.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.945+0000 7fa52affd640 1 -- 172.21.15.94:0/4060280326 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7fa52c0c8330 con 0x7fa534109770 2024-04-25T14:03:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:03:07.944 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:03:07.944 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:03:07.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 msgr2=0x7fa5200b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 0x7fa5200b2150 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa5300058c0 tx=0x7fa530005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:07.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 msgr2=0x7fa534102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:07.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa534102ed0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fa52c033040 tx=0x7fa52c031e20 comp rx=0 tx=0).stop 2024-04-25T14:03:07.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 shutdown_connections 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa5200afc40 0x7fa5200b2150 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa534109770 0x7fa534102ed0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa534106c00 0x7fa534102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 --2- 172.21.15.94:0/4060280326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa534101e20 0x7fa5341042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 >> 172.21.15.94:0/4060280326 conn(0x7fa5340fd330 msgr2=0x7fa534107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 shutdown_connections 2024-04-25T14:03:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:07.947+0000 7fa53bf84640 1 -- 172.21.15.94:0/4060280326 wait complete. 2024-04-25T14:03:07.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- 172.21.15.94:0/1739674913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381045e0 msgr2=0x7f9938106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 --2- 172.21.15.94:0/1739674913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381045e0 0x7f9938106a00 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f9934009900 tx=0x7f993402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- 172.21.15.94:0/1739674913 shutdown_connections 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 --2- 172.21.15.94:0/1739674913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 0x7f993810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 --2- 172.21.15.94:0/1739674913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938106f40 0x7f9938109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 --2- 172.21.15.94:0/1739674913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99381045e0 0x7f9938106a00 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- 172.21.15.94:0/1739674913 >> 172.21.15.94:0/1739674913 conn(0x7f99380fdbe0 msgr2=0x7f9938100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- 172.21.15.94:0/1739674913 shutdown_connections 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- 172.21.15.94:0/1739674913 wait complete. 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 Processor -- start 2024-04-25T14:03:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.080+0000 7f9940382640 1 -- start start 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938106f40 0x7f993819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 0x7f99381a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9938102520 con 0x7f9938106f40 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99381023a0 con 0x7f99381098a0 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f9940382640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99381026a0 con 0x7f99381045e0 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59014/0 (socket says 172.21.15.94:59014) 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 -- 172.21.15.94:0/1852728949 learned_addr learned my addr 172.21.15.94:0/1852728949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993d91d640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938106f40 0x7f993819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e91f640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 0x7f99381a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 -- 172.21.15.94:0/1852728949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 msgr2=0x7f99381a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 0x7f99381a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 -- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938106f40 msgr2=0x7f993819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938106f40 0x7f993819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 -- 172.21.15.94:0/1852728949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99381a4000 con 0x7f99381045e0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.081+0000 7f993e11e640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9934006010 tx=0x7f9934004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.082+0000 7f992f7fe640 1 -- 172.21.15.94:0/1852728949 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993403f030 con 0x7f99381045e0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.082+0000 7f992f7fe640 1 -- 172.21.15.94:0/1852728949 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9934031910 con 0x7f99381045e0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.082+0000 7f992f7fe640 1 -- 172.21.15.94:0/1852728949 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934035070 con 0x7f99381045e0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.082+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381a4290 con 0x7f99381045e0 2024-04-25T14:03:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.082+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381a4730 con 0x7f99381045e0 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.084+0000 7f992f7fe640 1 -- 172.21.15.94:0/1852728949 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9934042030 con 0x7f99381045e0 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.085+0000 7f992f7fe640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 0x7f99240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 Processor -- start 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993d91d640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 0x7f99240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f992f7fe640 1 -- 172.21.15.94:0/1852728949 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99340fec60 con 0x7f99381045e0 2024-04-25T14:03:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 -- start start 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ebfed80 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ebb8a20 con 0x55ce0e8d76b0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f9947237740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ebfad40 con 0x55ce0e8f8390 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993d91d640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 0x7f99240b2000 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f993819d560 tx=0x7f9928005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59016/0 (socket says 172.21.15.94:59016) 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993e11e640 1 -- 172.21.15.94:0/2923702147 learned_addr learned my addr 172.21.15.94:0/2923702147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993d91d640 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.086+0000 7f993e91f640 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 -- 172.21.15.94:0/2923702147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 msgr2=0x55ce0ec9ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 -- 172.21.15.94:0/2923702147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 msgr2=0x55ce0ec9ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 -- 172.21.15.94:0/2923702147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0ebc3c30 con 0x55ce0e8f8390 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f993e11e640 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99340c6df0 tx=0x7f9934031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e0cd6692172a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f992cbfe640 1 -- 172.21.15.94:0/2923702147 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99340e1070 con 0x55ce0e8f8390 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f992cbfe640 1 -- 172.21.15.94:0/2923702147 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340cb3f0 con 0x55ce0e8f8390 2024-04-25T14:03:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.087+0000 7f992cbfe640 1 -- 172.21.15.94:0/2923702147 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934049040 con 0x55ce0e8f8390 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- 172.21.15.94:0/2923702147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 msgr2=0x55ce0ec9a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99340c6df0 tx=0x7f9934031400 comp rx=0 tx=0).stop 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- 172.21.15.94:0/2923702147 shutdown_connections 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 --2- 172.21.15.94:0/2923702147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0ec9a810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- 172.21.15.94:0/2923702147 >> 172.21.15.94:0/2923702147 conn(0x55ce0e8ec4e0 msgr2=0x55ce0ec96020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- 172.21.15.94:0/2923702147 shutdown_connections 2024-04-25T14:03:08.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- 172.21.15.94:0/2923702147 wait complete. 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 -- start start 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.088+0000 7f9947237740 1 Processor -- start 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0ecab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ec9c2f0 con 0x55ce0e8f8390 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ce0eb85dc0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ec9c470 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9947237740 10 client.0 _ll_register_callbacks cb 0x55ce0eba1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0ecab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e11e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59030/0 (socket says 172.21.15.94:59030) 2024-04-25T14:03:08.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e11e640 1 -- 172.21.15.94:0/1544001098 learned_addr learned my addr 172.21.15.94:0/1544001098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 -- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 msgr2=0x55ce0eca4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 -- 172.21.15.94:0/1544001098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 msgr2=0x55ce0ecab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0ecab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 -- 172.21.15.94:0/1544001098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0e9103f0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e91f640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0ecab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993e11e640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f993d91d640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca4870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99280118d0 tx=0x7f9928004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c7b3f5ded3abaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.089+0000 7f9921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992801db20 con 0x55ce0e8d76b0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99340cb3f0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f992801dcc0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9928004030 con 0x55ce0e8d76b0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 10 client.32450 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ce0e9da7d0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340cb3f0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9928003bc0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 10 client.32450 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 10 client.32450 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 20 client.32450 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e8b6f60 0x55ce0ecb2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 4 client.32450 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.090+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce0ec0c990 con 0x55ce0e8b6f60 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.091+0000 7f993e11e640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e8b6f60 0x55ce0ecb2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.091+0000 7f9921ffb640 10 client.32450 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.091+0000 7f993e11e640 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e8b6f60 0x55ce0ecb2050 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=955323a6b54927fd server_cookie=9b522aeff1bba43d in_seq=0 out_seq=0 2024-04-25T14:03:08.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9921ffb640 1 -- 172.21.15.94:0/1544001098 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce0ec0c990 con 0x55ce0e8b6f60 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9921ffb640 10 client.32450 handle_command_reply: tid=0 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 client.32450 shutdown 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 msgr2=0x55ce0eca4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca4870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99280118d0 tx=0x7f9928004730 comp rx=0 tx=0).stop 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 shutdown_connections 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e8b6f60 0x55ce0ecb2050 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0ecab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca4870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 --2- 172.21.15.94:0/1544001098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8cf4d0 0x55ce0eca4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9921ffb640 0 client.32450 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 >> 172.21.15.94:0/1544001098 conn(0x55ce0e8ec4e0 msgr2=0x55ce0ec95040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 shutdown_connections 2024-04-25T14:03:08.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 1 -- 172.21.15.94:0/1544001098 wait complete. 2024-04-25T14:03:08.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.092+0000 7f9947237740 20 client.32450 trim_cache size 0 max 16384 2024-04-25T14:03:08.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 Processor -- start 2024-04-25T14:03:08.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 -- start start 2024-04-25T14:03:08.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0eca2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99381ad860 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99381af3a0 con 0x55ce0e8d76b0 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f9947237740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99381adf30 con 0x55ce0e8f8390 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f993e11e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f993e11e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40378/0 (socket says 172.21.15.94:40378) 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f993e11e640 1 -- 172.21.15.94:0/3318507693 learned_addr learned my addr 172.21.15.94:0/3318507693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f993d91d640 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.112+0000 7f993e91f640 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0eca2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 -- 172.21.15.94:0/3318507693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 msgr2=0x55ce0eca2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0eca2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 -- 172.21.15.94:0/3318507693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 msgr2=0x55ce0ec9f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 -- 172.21.15.94:0/3318507693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0ec9a3d0 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f993e11e640 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f99340c3170 tx=0x7f99340c7270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a1f0051ca3a404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f9923fff640 1 -- 172.21.15.94:0/3318507693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934049040 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f9923fff640 1 -- 172.21.15.94:0/3318507693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340c3380 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.113+0000 7f9923fff640 1 -- 172.21.15.94:0/3318507693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99340e1040 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- 172.21.15.94:0/3318507693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 msgr2=0x55ce0ec9f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f99340c3170 tx=0x7f99340c7270 comp rx=0 tx=0).stop 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- 172.21.15.94:0/3318507693 shutdown_connections 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8f8390 0x55ce0eca2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8d76b0 0x55ce0ec9f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 --2- 172.21.15.94:0/3318507693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ec9f210 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- 172.21.15.94:0/3318507693 >> 172.21.15.94:0/3318507693 conn(0x55ce0e8ec4e0 msgr2=0x55ce0ec9a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- 172.21.15.94:0/3318507693 shutdown_connections 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- 172.21.15.94:0/3318507693 wait complete. 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.114+0000 7f9947237740 1 -- start start 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 Processor -- start 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99381aecd0 con 0x55ce0e8f8390 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99381ae600 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99381ab840 con 0x55ce0e8d76b0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9947237740 10 client.0 _ll_register_callbacks cb 0x7f992800e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39822/0 (socket says 172.21.15.94:39822) 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993d91d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 -- 172.21.15.94:0/90901874 learned_addr learned my addr 172.21.15.94:0/90901874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40392/0 (socket says 172.21.15.94:40392) 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 -- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8d76b0 msgr2=0x55ce0eca7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 -- 172.21.15.94:0/90901874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 msgr2=0x55ce0eca7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 -- 172.21.15.94:0/90901874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0eca8130 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e91f640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f993e11e640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f99340c38b0 tx=0x7f99340c6d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dffefcb43b0e429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.115+0000 7f9921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934049070 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99340cfc10 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99340c3380 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f99340e1040 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 10 client.32456 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ce0ed19340 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340c3380 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f99340e9050 con 0x55ce0e8cf4d0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 10 client.32456 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 10 client.32456 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 20 client.32456 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e88c3a0 0x55ce0ed1ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 4 client.32456 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce0ec06800 con 0x55ce0e88c3a0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.116+0000 7f993d91d640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e88c3a0 0x55ce0ed1ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9921ffb640 10 client.32456 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f993d91d640 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e88c3a0 0x55ce0ed1ba20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f7b4ec812f675f0 server_cookie=16ef03d5700db564 in_seq=0 out_seq=0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9921ffb640 1 -- 172.21.15.94:0/90901874 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce0ec06800 con 0x55ce0e88c3a0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9921ffb640 10 client.32456 handle_command_reply: tid=0 2024-04-25T14:03:08.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 client.32456 shutdown 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 msgr2=0x55ce0ecaaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f99340c38b0 tx=0x7f99340c6d60 comp rx=0 tx=0).stop 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 shutdown_connections 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55ce0e88c3a0 0x55ce0ed1ba20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ce0e8f8390 0x55ce0eca7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce0e8d76b0 0x55ce0eca7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 --2- 172.21.15.94:0/90901874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce0e8cf4d0 0x55ce0ecaaa40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9921ffb640 0 client.32456 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 >> 172.21.15.94:0/90901874 conn(0x55ce0e8ec4e0 msgr2=0x55ce0eca41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 shutdown_connections 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 1 -- 172.21.15.94:0/90901874 wait complete. 2024-04-25T14:03:08.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.117+0000 7f9947237740 20 client.32456 trim_cache size 0 max 16384 2024-04-25T14:03:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:08.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 msgr2=0x7f99240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 0x7f99240b2000 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f993819d560 tx=0x7f9928005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 msgr2=0x7f993819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9934006010 tx=0x7f9934004270 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 shutdown_connections 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99240afaf0 0x7f99240b2000 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99381098a0 0x7f99381a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9938106f40 0x7f993819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 --2- 172.21.15.94:0/1852728949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99381045e0 0x7f993819c4c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 >> 172.21.15.94:0/1852728949 conn(0x7f99380fdbe0 msgr2=0x7f9938109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 shutdown_connections 2024-04-25T14:03:08.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.118+0000 7f9940382640 1 -- 172.21.15.94:0/1852728949 wait complete. 2024-04-25T14:03:08.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:03:08.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 167616 18.97 MB/sec execute 350 sec latency 369.486 ms 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 -- 172.21.15.94:0/3530120974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c101740 msgr2=0x7f1d7c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3530120974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c101740 0x7f1d7c101bb0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d7002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 -- 172.21.15.94:0/3530120974 shutdown_connections 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3530120974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c102240 0x7f1d7c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3530120974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c101740 0x7f1d7c101bb0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3530120974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c105090 0x7f1d7c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 -- 172.21.15.94:0/3530120974 >> 172.21.15.94:0/3530120974 conn(0x7f1d7c0fd340 msgr2=0x7f1d7c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 -- 172.21.15.94:0/3530120974 shutdown_connections 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.255+0000 7f1d80db4640 1 -- 172.21.15.94:0/3530120974 wait complete. 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 Processor -- start 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 -- start start 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 0x7f1d7c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10f620 con 0x7f1d7c102240 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10f4a0 con 0x7f1d7c101740 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.256+0000 7f1d80db4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10f7a0 con 0x7f1d7c105090 2024-04-25T14:03:08.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39852/0 (socket says 172.21.15.94:39852) 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 0x7f1d7c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40424/0 (socket says 172.21.15.94:40424) 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7a59c640 1 -- 172.21.15.94:0/3463106608 learned_addr learned my addr 172.21.15.94:0/3463106608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 -- 172.21.15.94:0/3463106608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 msgr2=0x7f1d7c104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 -- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 msgr2=0x7f1d7c1028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 -- 172.21.15.94:0/3463106608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c103550 con 0x7f1d7c105090 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 0x7f1d7c102e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400e940 tx=0x7f1d7400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.257+0000 7f1d6b7fe640 1 -- 172.21.15.94:0/3463106608 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74004d20 con 0x7f1d7c105090 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.258+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c1a42f0 con 0x7f1d7c105090 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.258+0000 7f1d6b7fe640 1 -- 172.21.15.94:0/3463106608 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d74005020 con 0x7f1d7c105090 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.258+0000 7f1d6b7fe640 1 -- 172.21.15.94:0/3463106608 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d740116d0 con 0x7f1d7c105090 2024-04-25T14:03:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.258+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c1a47e0 con 0x7f1d7c105090 2024-04-25T14:03:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.260+0000 7f1d87c62740 1 Processor -- start 2024-04-25T14:03:08.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d6b7fe640 1 -- 172.21.15.94:0/3463106608 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1d74011870 con 0x7f1d7c105090 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 -- start start 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a2f3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a2f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b66a257d80 con 0x55b669f284d0 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b66a211a20 con 0x55b669f51390 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d87c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b66a253d40 con 0x55b669f306b0 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a2f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40438/0 (socket says 172.21.15.94:40438) 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d7a59c640 1 -- 172.21.15.94:0/3262238218 learned_addr learned my addr 172.21.15.94:0/3262238218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a2f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39866/0 (socket says 172.21.15.94:39866) 2024-04-25T14:03:08.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.261+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a2f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 -- 172.21.15.94:0/3262238218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 msgr2=0x55b66a2f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a2f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 -- 172.21.15.94:0/3262238218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 msgr2=0x55b66a2f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d6b7fe640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 0x7f1d640ba890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a2f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 -- 172.21.15.94:0/3262238218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66a21cc30 con 0x55b669f284d0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0115c0 tx=0x7f1d6c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1206b240c79dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 0x7f1d640ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d6b7fe640 1 -- 172.21.15.94:0/3463106608 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d740defd0 con 0x7f1d7c105090 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d68ff9640 1 -- 172.21.15.94:0/3262238218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c005020 con 0x55b669f284d0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d68ff9640 1 -- 172.21.15.94:0/3262238218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6c0047a0 con 0x55b669f284d0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.262+0000 7f1d68ff9640 1 -- 172.21.15.94:0/3262238218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00f8e0 con 0x55b669f284d0 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 -- 172.21.15.94:0/3262238218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 msgr2=0x55b66a2f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0115c0 tx=0x7f1d6c011a80 comp rx=0 tx=0).stop 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 0x7f1d640ba890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009230 tx=0x7f1d70031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 -- 172.21.15.94:0/3262238218 shutdown_connections 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a2f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a2f3d90 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3262238218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a2f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 -- 172.21.15.94:0/3262238218 >> 172.21.15.94:0/3262238218 conn(0x55b669f454e0 msgr2=0x55b66a2ef020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 -- 172.21.15.94:0/3262238218 shutdown_connections 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.263+0000 7f1d87c62740 1 -- 172.21.15.94:0/3262238218 wait complete. 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 -- start start 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 Processor -- start 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f284d0 0x55b66a2fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a304a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b66a2f5530 con 0x55b669f306b0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b66a1dedc0 con 0x55b669f51390 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b66a2f56b0 con 0x55b669f284d0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d87c62740 10 client.0 _ll_register_callbacks cb 0x55b66a1fa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40452/0 (socket says 172.21.15.94:40452) 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d79d9b640 1 -- 172.21.15.94:0/3564784321 learned_addr learned my addr 172.21.15.94:0/3564784321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.264+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f284d0 0x55b66a2fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 -- 172.21.15.94:0/3564784321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f284d0 msgr2=0x55b66a2fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a304a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f284d0 0x55b66a2fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 -- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 msgr2=0x55b66a304a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a304a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 -- 172.21.15.94:0/3564784321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b669f693f0 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a304a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4ad3ebcc15d69a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70035040 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d70003980 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d6c0047a0 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1d70003b80 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d87c62740 10 client.23082 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.265+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b66a0337d0 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6c0047a0 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1d7003b460 con 0x55b669f306b0 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 10 client.23082 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 10 client.23082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 20 client.23082 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f0ff60 0x55b66a30b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 4 client.23082 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66a265990 con 0x55b669f0ff60 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.266+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f0ff60 0x55b66a30b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.267+0000 7f1d61ffb640 10 client.23082 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.267+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f0ff60 0x55b66a30b360 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ffb9d7b405383b9 server_cookie=97c493b6f66bbf1e in_seq=0 out_seq=0 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d61ffb640 1 -- 172.21.15.94:0/3564784321 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b66a265990 con 0x55b669f0ff60 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d61ffb640 10 client.23082 handle_command_reply: tid=0 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 client.23082 shutdown 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 msgr2=0x55b66a2fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d70009260 comp rx=0 tx=0).stop 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 shutdown_connections 2024-04-25T14:03:08.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f0ff60 0x55b66a30b360 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d61ffb640 0 client.23082 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f51390 0x55b66a304a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f306b0 0x55b66a2fdaa0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 --2- 172.21.15.94:0/3564784321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f284d0 0x55b66a2fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.268+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 >> 172.21.15.94:0/3564784321 conn(0x55b669f454e0 msgr2=0x55b66a2f1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.269+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 shutdown_connections 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.269+0000 7f1d87c62740 1 -- 172.21.15.94:0/3564784321 wait complete. 2024-04-25T14:03:08.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.269+0000 7f1d87c62740 20 client.23082 trim_cache size 0 max 16384 2024-04-25T14:03:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 Processor -- start 2024-04-25T14:03:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 -- start start 2024-04-25T14:03:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 0x55b66a2f2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a310b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1a9410 con 0x55b669f284d0 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1aaf50 con 0x55b669f0ff60 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d87c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1a9ae0 con 0x55b669f306b0 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40464/0 (socket says 172.21.15.94:40464) 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d7a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 0x55b66a2f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d79d9b640 1 -- 172.21.15.94:0/768249147 learned_addr learned my addr 172.21.15.94:0/768249147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.286+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a310b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 -- 172.21.15.94:0/768249147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 msgr2=0x55b66a310b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a310b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 -- 172.21.15.94:0/768249147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 msgr2=0x55b66a2f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 0x55b66a2f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 -- 172.21.15.94:0/768249147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66a304e60 con 0x55b669f284d0 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 0x55b66a2f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f1d7003b8a0 tx=0x7f1d7002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443a829d29c164f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d63fff640 1 -- 172.21.15.94:0/768249147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70062070 con 0x55b669f284d0 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d63fff640 1 -- 172.21.15.94:0/768249147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d70050070 con 0x55b669f284d0 2024-04-25T14:03:08.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.287+0000 7f1d63fff640 1 -- 172.21.15.94:0/768249147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7003ba40 con 0x55b669f284d0 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 -- 172.21.15.94:0/768249147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 msgr2=0x55b66a304920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f1d7003b8a0 tx=0x7f1d7002ab90 comp rx=0 tx=0).stop 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 -- 172.21.15.94:0/768249147 shutdown_connections 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f306b0 0x55b66a310b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304920 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 --2- 172.21.15.94:0/768249147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f0ff60 0x55b66a2f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 -- 172.21.15.94:0/768249147 >> 172.21.15.94:0/768249147 conn(0x55b669f454e0 msgr2=0x55b66a2f1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 -- 172.21.15.94:0/768249147 shutdown_connections 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.288+0000 7f1d87c62740 1 -- 172.21.15.94:0/768249147 wait complete. 2024-04-25T14:03:08.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 -- start start 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 Processor -- start 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f0ff60 0x55b66a303bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f306b0 0x55b66a371820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1aa880 con 0x55b669f284d0 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1aa1b0 con 0x55b669f306b0 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c1a7410 con 0x55b669f0ff60 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d87c62740 10 client.0 _ll_register_callbacks cb 0x7f1d70009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40480/0 (socket says 172.21.15.94:40480) 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 -- 172.21.15.94:0/1575788713 learned_addr learned my addr 172.21.15.94:0/1575788713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d7ad9d640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f306b0 0x55b66a371820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f0ff60 0x55b66a303bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 -- 172.21.15.94:0/1575788713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f0ff60 msgr2=0x55b66a303bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f0ff60 0x55b66a303bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 -- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f306b0 msgr2=0x55b66a371820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f306b0 0x55b66a371820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.289+0000 7f1d79d9b640 1 -- 172.21.15.94:0/1575788713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66a371de0 con 0x55b669f284d0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d79d9b640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d70003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39be62211d6a0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70062070 con 0x55b669f284d0 2024-04-25T14:03:08.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1d7003ba10 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d70050070 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1d7006e050 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d87c62740 10 client.23088 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b66a373090 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.290+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d70050070 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1d70054410 con 0x55b669f284d0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 10 client.23088 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 10 client.23088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 20 client.23088 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f51390 0x55b66a375ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 4 client.23088 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b66a25f800 con 0x55b669f51390 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f51390 0x55b66a375ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d61ffb640 10 client.23088 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.291+0000 7f1d7a59c640 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f51390 0x55b66a375ab0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bad13ce712a89c97 server_cookie=b5e13dda55c5786a in_seq=0 out_seq=0 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d61ffb640 1 -- 172.21.15.94:0/1575788713 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b66a25f800 con 0x55b669f51390 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d61ffb640 10 client.23088 handle_command_reply: tid=0 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 client.23088 shutdown 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 msgr2=0x55b66a304100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d70003980 comp rx=0 tx=0).stop 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 shutdown_connections 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55b669f51390 0x55b66a375ab0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b669f306b0 0x55b66a371820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b669f284d0 0x55b66a304100 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d61ffb640 0 client.23088 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 --2- 172.21.15.94:0/1575788713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b669f0ff60 0x55b66a303bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 >> 172.21.15.94:0/1575788713 conn(0x55b669f454e0 msgr2=0x55b66a30fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 shutdown_connections 2024-04-25T14:03:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 1 -- 172.21.15.94:0/1575788713 wait complete. 2024-04-25T14:03:08.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.292+0000 7f1d87c62740 20 client.23088 trim_cache size 0 max 16384 2024-04-25T14:03:08.298 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 msgr2=0x7f1d640ba890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 0x7f1d640ba890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009230 tx=0x7f1d70031400 comp rx=0 tx=0).stop 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 msgr2=0x7f1d7c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 0x7f1d7c102e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400e940 tx=0x7f1d7400ee00 comp rx=0 tx=0).stop 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 shutdown_connections 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1d640b8380 0x7f1d640ba890 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d7c105090 0x7f1d7c102e20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c102240 0x7f1d7c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 --2- 172.21.15.94:0/3463106608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d7c101740 0x7f1d7c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 >> 172.21.15.94:0/3463106608 conn(0x7f1d7c0fd340 msgr2=0x7f1d7c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 shutdown_connections 2024-04-25T14:03:08.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.293+0000 7f1d80db4640 1 -- 172.21.15.94:0/3463106608 wait complete. 2024-04-25T14:03:08.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:03:08.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 -- 172.21.15.94:0/3402458111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 msgr2=0x7f61dc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 --2- 172.21.15.94:0/3402458111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc102e70 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009900 tx=0x7f61d002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:08.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 -- 172.21.15.94:0/3402458111 shutdown_connections 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 --2- 172.21.15.94:0/3402458111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 0x7f61dc10a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 --2- 172.21.15.94:0/3402458111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 0x7f61dc1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 --2- 172.21.15.94:0/3402458111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc102e70 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 -- 172.21.15.94:0/3402458111 >> 172.21.15.94:0/3402458111 conn(0x7f61dc0fd340 msgr2=0x7f61dc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 -- 172.21.15.94:0/3402458111 shutdown_connections 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.470+0000 7f61e2b84640 1 -- 172.21.15.94:0/3402458111 wait complete. 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 Processor -- start 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 -- start start 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 0x7f61dc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 0x7f61dc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10dc90 con 0x7f61dc102170 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10db10 con 0x7f61dc1033b0 2024-04-25T14:03:08.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e2b84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc10de10 con 0x7f61dc1080b0 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40496/0 (socket says 172.21.15.94:40496) 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 -- 172.21.15.94:0/4044096748 learned_addr learned my addr 172.21.15.94:0/4044096748 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61dbfff640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 0x7f61dc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e1121640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 0x7f61dc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 -- 172.21.15.94:0/4044096748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 msgr2=0x7f61dc1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 0x7f61dc1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 -- 172.21.15.94:0/4044096748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 msgr2=0x7f61dc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 0x7f61dc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.471+0000 7f61e0920640 1 -- 172.21.15.94:0/4044096748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a1dc0 con 0x7f61dc102170 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61e0920640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61d9ffb640 1 -- 172.21.15.94:0/4044096748 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d003f030 con 0x7f61dc102170 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61d9ffb640 1 -- 172.21.15.94:0/4044096748 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d0003aa0 con 0x7f61dc102170 2024-04-25T14:03:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a2050 con 0x7f61dc102170 2024-04-25T14:03:08.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61d9ffb640 1 -- 172.21.15.94:0/4044096748 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0035070 con 0x7f61dc102170 2024-04-25T14:03:08.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.472+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a2590 con 0x7f61dc102170 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.475+0000 7f61e9a34740 1 Processor -- start 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 -- start start 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61d9ffb640 1 -- 172.21.15.94:0/4044096748 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f61d0042030 con 0x7f61dc102170 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560860332d80 con 0x56086002c390 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608602eca20 con 0x5608600034d0 2024-04-25T14:03:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e9a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56086032ed40 con 0x56086000b6b0 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61e1121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59142/0 (socket says 172.21.15.94:59142) 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 -- 172.21.15.94:0/805812151 learned_addr learned my addr 172.21.15.94:0/805812151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 -- 172.21.15.94:0/805812151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 msgr2=0x5608603ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.476+0000 7f61dbfff640 1 -- 172.21.15.94:0/805812151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 msgr2=0x5608603ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61dbfff640 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61dbfff640 1 -- 172.21.15.94:0/805812151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608602f7c30 con 0x56086000b6b0 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61d9ffb640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 0x7f61cc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61e1121640 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61e1121640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 0x7f61cc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61dbfff640 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f61c800bd10 tx=0x7f61c80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363d8cb92cce550f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61d9ffb640 1 -- 172.21.15.94:0/4044096748 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f61d00ff150 con 0x7f61dc102170 2024-04-25T14:03:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61c6ffd640 1 -- 172.21.15.94:0/805812151 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8005220 con 0x56086000b6b0 2024-04-25T14:03:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61c6ffd640 1 -- 172.21.15.94:0/805812151 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8004c50 con 0x56086000b6b0 2024-04-25T14:03:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.477+0000 7f61c6ffd640 1 -- 172.21.15.94:0/805812151 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8011840 con 0x56086000b6b0 2024-04-25T14:03:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- 172.21.15.94:0/805812151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 msgr2=0x5608603cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f61c800bd10 tx=0x7f61c80107a0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e1121640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 0x7f61cc0b2080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5608603cf860 tx=0x7f61d40096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- 172.21.15.94:0/805812151 shutdown_connections 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603cfa20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 --2- 172.21.15.94:0/805812151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- 172.21.15.94:0/805812151 >> 172.21.15.94:0/805812151 conn(0x5608600204e0 msgr2=0x5608603ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- 172.21.15.94:0/805812151 shutdown_connections 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- 172.21.15.94:0/805812151 wait complete. 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.478+0000 7f61e9a34740 1 -- start start 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 Processor -- start 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 0x5608603d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 0x5608603dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5608603d0530 con 0x5608600034d0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608602b9dc0 con 0x56086000b6b0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608603d06b0 con 0x56086002c390 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e9a34740 10 client.0 _ll_register_callbacks cb 0x5608602d5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e0920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 0x5608603d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e0920640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 0x5608603d85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40518/0 (socket says 172.21.15.94:40518) 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e0920640 1 -- 172.21.15.94:0/348232900 learned_addr learned my addr 172.21.15.94:0/348232900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e1121640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 0x5608603dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61dbfff640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e1121640 1 -- 172.21.15.94:0/348232900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 msgr2=0x5608603d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e1121640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e1121640 1 -- 172.21.15.94:0/348232900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 msgr2=0x5608603d85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.479+0000 7f61e1121640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 0x5608603d85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61e1121640 1 -- 172.21.15.94:0/348232900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608600443f0 con 0x56086002c390 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61dbfff640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61e1121640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 0x5608603dfa40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005fd0 tx=0x7f61d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4cde8db01ed1ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004d20 con 0x56086002c390 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f61d400fb30 con 0x56086002c390 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c8004c50 con 0x56086002c390 2024-04-25T14:03:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61d400fd30 con 0x56086002c390 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61e9a34740 10 client.32143 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.480+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56086010e7d0 con 0x56086002c390 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c8004c50 con 0x56086002c390 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f61d4031420 con 0x56086002c390 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 10 client.32143 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 10 client.32143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 20 client.32143 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffeaf60 0x5608603e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 4 client.32143 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560860340990 con 0x56085ffeaf60 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61e0920640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffeaf60 0x5608603e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.481+0000 7f61bbfff640 10 client.32143 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61e0920640 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffeaf60 0x5608603e62f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65f2ae98a7803266 server_cookie=2e47a6cd8b1212ac in_seq=0 out_seq=0 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61bbfff640 1 -- 172.21.15.94:0/348232900 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560860340990 con 0x56085ffeaf60 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61bbfff640 10 client.32143 handle_command_reply: tid=0 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61e9a34740 1 client.32143 shutdown 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 msgr2=0x5608603dfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 0x5608603dfa40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005fd0 tx=0x7f61d4004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.482+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 shutdown_connections 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffeaf60 0x5608603e62f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086002c390 0x5608603dfa40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61bbfff640 0 client.32143 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 --2- 172.21.15.94:0/348232900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608600034d0 0x5608603d85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 >> 172.21.15.94:0/348232900 conn(0x5608600204e0 msgr2=0x5608603c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 shutdown_connections 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 1 -- 172.21.15.94:0/348232900 wait complete. 2024-04-25T14:03:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.483+0000 7f61e9a34740 20 client.32143 trim_cache size 0 max 16384 2024-04-25T14:03:08.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 Processor -- start 2024-04-25T14:03:08.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 -- start start 2024-04-25T14:03:08.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ad7d0 con 0x56086002c390 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1af310 con 0x56086000b6b0 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.497+0000 7f61e9a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1adea0 con 0x5608600034d0 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e1121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e1121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40526/0 (socket says 172.21.15.94:40526) 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39940/0 (socket says 172.21.15.94:39940) 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59150/0 (socket says 172.21.15.94:59150) 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e1121640 1 -- 172.21.15.94:0/149848961 learned_addr learned my addr 172.21.15.94:0/149848961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 -- 172.21.15.94:0/149848961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 msgr2=0x5608603df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 -- 172.21.15.94:0/149848961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 msgr2=0x5608603d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603d32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 -- 172.21.15.94:0/149848961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608603c57f0 con 0x5608600034d0 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61e0920640 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61d00c48b0 tx=0x7f61d00b0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703cf2e7a1ec68ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61c67fc640 1 -- 172.21.15.94:0/149848961 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0049040 con 0x5608600034d0 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61c67fc640 1 -- 172.21.15.94:0/149848961 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d00cb940 con 0x5608600034d0 2024-04-25T14:03:08.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.498+0000 7f61c67fc640 1 -- 172.21.15.94:0/149848961 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0003c50 con 0x5608600034d0 2024-04-25T14:03:08.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 -- 172.21.15.94:0/149848961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 msgr2=0x5608603c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61d00c48b0 tx=0x7f61d00b0040 comp rx=0 tx=0).stop 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 -- 172.21.15.94:0/149848961 shutdown_connections 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x5608603d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56086000b6b0 0x5608603df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 --2- 172.21.15.94:0/149848961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608600034d0 0x5608603c5190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 -- 172.21.15.94:0/149848961 >> 172.21.15.94:0/149848961 conn(0x5608600204e0 msgr2=0x5608603d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 -- 172.21.15.94:0/149848961 shutdown_connections 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.499+0000 7f61e9a34740 1 -- 172.21.15.94:0/149848961 wait complete. 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 -- start start 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 Processor -- start 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1aec40 con 0x56086002c390 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ae570 con 0x5608600034d0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1ab7b0 con 0x56086000b6b0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e9a34740 10 client.0 _ll_register_callbacks cb 0x7f61d4009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e0920640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e1121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e0920640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39950/0 (socket says 172.21.15.94:39950) 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e1121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40542/0 (socket says 172.21.15.94:40542) 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61e0920640 1 -- 172.21.15.94:0/263960405 learned_addr learned my addr 172.21.15.94:0/263960405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.500+0000 7f61dbfff640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 -- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 msgr2=0x5608603dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 -- 172.21.15.94:0/263960405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 msgr2=0x56086044c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 -- 172.21.15.94:0/263960405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086044cc10 con 0x5608600034d0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e1121640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e0920640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f61d00c36c0 tx=0x7f61d0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e035fc5fefc38a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0049070 con 0x5608600034d0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d00cb940 con 0x5608600034d0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f61d00c38d0 con 0x5608600034d0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61d00d03f0 con 0x5608600034d0 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e9a34740 10 client.32486 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56086044dee0 con 0x5608600034d0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.501+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d00cb940 con 0x5608600034d0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f61d00c78a0 con 0x5608600034d0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 10 client.32486 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 10 client.32486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 20 client.32486 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffc03a0 0x5608604508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 4 client.32486 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56086033a800 con 0x56085ffc03a0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.502+0000 7f61dbfff640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffc03a0 0x5608604508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61bbfff640 10 client.32486 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61dbfff640 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffc03a0 0x5608604508a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfd1f9b6257e0854 server_cookie=ef64c90d4607ce19 in_seq=0 out_seq=0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61bbfff640 1 -- 172.21.15.94:0/263960405 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56086033a800 con 0x56085ffc03a0 2024-04-25T14:03:08.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61bbfff640 10 client.32486 handle_command_reply: tid=0 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 client.32486 shutdown 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 msgr2=0x5608603dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f61d00c36c0 tx=0x7f61d0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 shutdown_connections 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x56085ffc03a0 0x5608604508a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56086002c390 0x56086044c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56086000b6b0 0x5608603dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 --2- 172.21.15.94:0/263960405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608600034d0 0x5608603dca10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61bbfff640 0 client.32486 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 >> 172.21.15.94:0/263960405 conn(0x5608600204e0 msgr2=0x5608603d84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 shutdown_connections 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 1 -- 172.21.15.94:0/263960405 wait complete. 2024-04-25T14:03:08.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.503+0000 7f61e9a34740 20 client.32486 trim_cache size 0 max 16384 2024-04-25T14:03:08.509 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:08.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 msgr2=0x7f61cc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 0x7f61cc0b2080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5608603cf860 tx=0x7f61d40096a0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 msgr2=0x7f61dc19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d0031a50 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 shutdown_connections 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f61cc0afb70 0x7f61cc0b2080 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61dc1080b0 0x7f61dc1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61dc1033b0 0x7f61dc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 --2- 172.21.15.94:0/4044096748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61dc102170 0x7f61dc19a280 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 >> 172.21.15.94:0/4044096748 conn(0x7f61dc0fd340 msgr2=0x7f61dc1097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 shutdown_connections 2024-04-25T14:03:08.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.504+0000 7f61e2b84640 1 -- 172.21.15.94:0/4044096748 wait complete. 2024-04-25T14:03:08.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:03:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:08 smithi137 ceph-mon[32568]: pgmap v991: 97 pgs: 97 active+clean; 277 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-04-25T14:03:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:08 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4060280326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:08.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.649+0000 7f1a5ee51640 1 -- 172.21.15.94:0/508310160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5810b9d0 msgr2=0x7f1a5810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.649+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/508310160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5810b9d0 0x7f1a5810deb0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009be0 tx=0x7f1a5002a6e0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 -- 172.21.15.94:0/508310160 shutdown_connections 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/508310160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5810b9d0 0x7f1a5810deb0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/508310160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58108ea0 0x7f1a5810b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/508310160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a58103fc0 0x7f1a581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 -- 172.21.15.94:0/508310160 >> 172.21.15.94:0/508310160 conn(0x7f1a580fd340 msgr2=0x7f1a580ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 -- 172.21.15.94:0/508310160 shutdown_connections 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.650+0000 7f1a5ee51640 1 -- 172.21.15.94:0/508310160 wait complete. 2024-04-25T14:03:08.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 Processor -- start 2024-04-25T14:03:08.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 -- start start 2024-04-25T14:03:08.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58103fc0 0x7f1a5819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a5810b9d0 0x7f1a581a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581118c0 con 0x7f1a58108ea0 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58111740 con 0x7f1a5810b9d0 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5ee51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58111a40 con 0x7f1a58103fc0 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40556/0 (socket says 172.21.15.94:40556) 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 -- 172.21.15.94:0/3100498847 learned_addr learned my addr 172.21.15.94:0/3100498847 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a5810b9d0 0x7f1a581a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 -- 172.21.15.94:0/3100498847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58103fc0 msgr2=0x7f1a5819c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58103fc0 0x7f1a5819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 -- 172.21.15.94:0/3100498847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a5810b9d0 msgr2=0x7f1a581a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a5810b9d0 0x7f1a581a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.651+0000 7f1a57fff640 1 -- 172.21.15.94:0/3100498847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a581a3fb0 con 0x7f1a58108ea0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a57fff640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b450 tx=0x7f1a4400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a55ffb640 1 -- 172.21.15.94:0/3100498847 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44015020 con 0x7f1a58108ea0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a55ffb640 1 -- 172.21.15.94:0/3100498847 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a44011d40 con 0x7f1a58108ea0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a55ffb640 1 -- 172.21.15.94:0/3100498847 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400f340 con 0x7f1a58108ea0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a58104af0 con 0x7f1a58108ea0 2024-04-25T14:03:08.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.652+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a58105070 con 0x7f1a58108ea0 2024-04-25T14:03:08.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 Processor -- start 2024-04-25T14:03:08.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 -- start start 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61532810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd6116f6b0 0x55cd61533a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd61496d80 con 0x55cd6116f6b0 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd61450a20 con 0x55cd61190390 2024-04-25T14:03:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a65de2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd61492d40 con 0x55cd611674d0 2024-04-25T14:03:08.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59174/0 (socket says 172.21.15.94:59174) 2024-04-25T14:03:08.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a5cbed640 1 -- 172.21.15.94:0/3739709432 learned_addr learned my addr 172.21.15.94:0/3739709432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.656+0000 7f1a55ffb640 1 -- 172.21.15.94:0/3100498847 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1a44018020 con 0x7f1a58108ea0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a55ffb640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 0x7f1a480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a57fff640 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd6116f6b0 0x55cd61533a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a5cbed640 1 -- 172.21.15.94:0/3739709432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 msgr2=0x55cd61532810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61532810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a5cbed640 1 -- 172.21.15.94:0/3739709432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd6116f6b0 msgr2=0x55cd61533a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd6116f6b0 0x55cd61533a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.657+0000 7f1a5cbed640 1 -- 172.21.15.94:0/3739709432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6145bc30 con 0x55cd611674d0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 0x7f1a480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009ad0 tx=0x7f1a4c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7be4bacddbdde9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a55ffb640 1 -- 172.21.15.94:0/3100498847 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1a440de910 con 0x7f1a58108ea0 2024-04-25T14:03:08.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a42ffd640 1 -- 172.21.15.94:0/3739709432 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00ed10 con 0x55cd611674d0 2024-04-25T14:03:08.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a42ffd640 1 -- 172.21.15.94:0/3739709432 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c0035f0 con 0x55cd611674d0 2024-04-25T14:03:08.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.658+0000 7f1a42ffd640 1 -- 172.21.15.94:0/3739709432 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c010690 con 0x55cd611674d0 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.659+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 0x7f1a480b2080 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006010 tx=0x7f1a500308a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.659+0000 7f1a65de2740 1 -- 172.21.15.94:0/3739709432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 msgr2=0x55cd61532d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.659+0000 7f1a65de2740 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009ad0 tx=0x7f1a4c009f90 comp rx=0 tx=0).stop 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 -- 172.21.15.94:0/3739709432 shutdown_connections 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd6116f6b0 0x55cd61533a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd61532d90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 --2- 172.21.15.94:0/3739709432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61532810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 -- 172.21.15.94:0/3739709432 >> 172.21.15.94:0/3739709432 conn(0x55cd611844e0 msgr2=0x55cd6152e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 -- 172.21.15.94:0/3739709432 shutdown_connections 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 -- 172.21.15.94:0/3739709432 wait complete. 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 -- start start 2024-04-25T14:03:08.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.660+0000 7f1a65de2740 1 Processor -- start 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 0x55cd6153c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 0x55cd6153cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61543af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd61534530 con 0x55cd611674d0 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cd6141ddc0 con 0x55cd61190390 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd615346b0 con 0x55cd6116f6b0 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a65de2740 10 client.0 _ll_register_callbacks cb 0x55cd61439600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 0x55cd6153c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 0x55cd6153c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40568/0 (socket says 172.21.15.94:40568) 2024-04-25T14:03:08.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5cbed640 1 -- 172.21.15.94:0/2608979686 learned_addr learned my addr 172.21.15.94:0/2608979686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a57fff640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 0x55cd6153cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61543af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 -- 172.21.15.94:0/2608979686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 msgr2=0x55cd6153cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 0x55cd6153cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 -- 172.21.15.94:0/2608979686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 msgr2=0x55cd6153c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 0x55cd6153c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.661+0000 7f1a5d3ee640 1 -- 172.21.15.94:0/2608979686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd611a83f0 con 0x55cd61190390 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.662+0000 7f1a57fff640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 0x55cd6153cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.664+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61543af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009b30 tx=0x7f1a500303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68a384bfec5f266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.664+0000 7f1a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:08.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.664+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5004c070 con 0x55cd61190390 2024-04-25T14:03:08.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.664+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a4c0035f0 con 0x55cd61190390 2024-04-25T14:03:08.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.664+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1a50003b70 con 0x55cd61190390 2024-04-25T14:03:08.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1a50041040 con 0x55cd61190390 2024-04-25T14:03:08.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 10 client.32495 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cd612727d0 con 0x55cd61190390 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50003d70 con 0x55cd61190390 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1a50047040 con 0x55cd61190390 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 10 client.32495 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 10 client.32495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 20 client.32495 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd6114ef60 0x55cd6154a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 4 client.32495 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:08.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.665+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1a4c0035f0 con 0x55cd6114ef60 2024-04-25T14:03:08.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.666+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd6114ef60 0x55cd6154a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.666+0000 7f1a37fff640 10 client.32495 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:08.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.666+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd6114ef60 0x55cd6154a330 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1fd0b0e2fd0c3ee server_cookie=f928ddaaef426624 in_seq=0 out_seq=0 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.667+0000 7f1a37fff640 1 -- 172.21.15.94:0/2608979686 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1a4c0035f0 con 0x55cd6114ef60 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.667+0000 7f1a37fff640 10 client.32495 handle_command_reply: tid=0 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 client.32495 shutdown 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 msgr2=0x55cd61543af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61543af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009b30 tx=0x7f1a500303a0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 shutdown_connections 2024-04-25T14:03:08.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd6114ef60 0x55cd6154a330 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd61190390 0x55cd61543af0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd6116f6b0 0x55cd6153cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 --2- 172.21.15.94:0/2608979686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd611674d0 0x55cd6153c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 >> 172.21.15.94:0/2608979686 conn(0x55cd611844e0 msgr2=0x55cd61530f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a37fff640 0 client.32495 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 shutdown_connections 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 1 -- 172.21.15.94:0/2608979686 wait complete. 2024-04-25T14:03:08.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.668+0000 7f1a65de2740 20 client.32495 trim_cache size 0 max 16384 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.687+0000 7f1a65de2740 1 Processor -- start 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.687+0000 7f1a65de2740 1 -- start start 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd61543a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b1c60 con 0x55cd61190390 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b37a0 con 0x55cd6116f6b0 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a65de2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b2330 con 0x55cd611674d0 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59194/0 (socket says 172.21.15.94:59194) 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 -- 172.21.15.94:0/1517615298 learned_addr learned my addr 172.21.15.94:0/1517615298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a57fff640 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd61543a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 -- 172.21.15.94:0/1517615298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 msgr2=0x55cd61543a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd61543a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 -- 172.21.15.94:0/1517615298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 msgr2=0x55cd615378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.688+0000 7f1a5cbed640 1 -- 172.21.15.94:0/1517615298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd61544ec0 con 0x55cd611674d0 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.689+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009550 tx=0x7f1a4c003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8b6e07c39e0166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.689+0000 7f1a427fc640 1 -- 172.21.15.94:0/1517615298 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00ec30 con 0x55cd611674d0 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.689+0000 7f1a427fc640 1 -- 172.21.15.94:0/1517615298 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c027070 con 0x55cd611674d0 2024-04-25T14:03:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.689+0000 7f1a427fc640 1 -- 172.21.15.94:0/1517615298 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c005490 con 0x55cd611674d0 2024-04-25T14:03:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 -- 172.21.15.94:0/1517615298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 msgr2=0x55cd615373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009550 tx=0x7f1a4c003d10 comp rx=0 tx=0).stop 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 -- 172.21.15.94:0/1517615298 shutdown_connections 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd61543a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 --2- 172.21.15.94:0/1517615298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd615373b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 -- 172.21.15.94:0/1517615298 >> 172.21.15.94:0/1517615298 conn(0x55cd611844e0 msgr2=0x55cd61532790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 -- 172.21.15.94:0/1517615298 shutdown_connections 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.690+0000 7f1a65de2740 1 -- 172.21.15.94:0/1517615298 wait complete. 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 -- start start 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 Processor -- start 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd615401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b30d0 con 0x55cd61190390 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b2a00 con 0x55cd6116f6b0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581afc40 con 0x55cd611674d0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a65de2740 10 client.0 _ll_register_callbacks cb 0x7f1a50034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.691+0000 7f1a5cbed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59204/0 (socket says 172.21.15.94:59204) 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 -- 172.21.15.94:0/4035812262 learned_addr learned my addr 172.21.15.94:0/4035812262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a57fff640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd615401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 -- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 msgr2=0x55cd615401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd615401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 -- 172.21.15.94:0/4035812262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 msgr2=0x55cd615430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 -- 172.21.15.94:0/4035812262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd615436b0 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5d3ee640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a5cbed640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0096c0 tx=0x7f1a4c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e36b1f078f972e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0037f0 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a4c027070 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1a4c003990 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.692+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c041050 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 10 client.32161 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cd615b1070 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c027070 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1a4c003d80 con 0x55cd611674d0 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 10 client.32161 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 10 client.32161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 20 client.32161 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd611243a0 0x55cd615b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 4 client.32161 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd614a4990 con 0x55cd611243a0 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.693+0000 7f1a57fff640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd611243a0 0x55cd615b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a37fff640 10 client.32161 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a57fff640 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd611243a0 0x55cd615b3820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46ca3c4ef3532b0c server_cookie=e062fc4383b6ab1 in_seq=0 out_seq=0 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a37fff640 1 -- 172.21.15.94:0/4035812262 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd614a4990 con 0x55cd611243a0 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a37fff640 10 client.32161 handle_command_reply: tid=0 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 client.32161 shutdown 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 msgr2=0x55cd6153fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0096c0 tx=0x7f1a4c00ecc0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 shutdown_connections 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55cd611243a0 0x55cd615b3820 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd61190390 0x55cd615430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd6116f6b0 0x55cd615401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 --2- 172.21.15.94:0/4035812262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd611674d0 0x55cd6153fc70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 >> 172.21.15.94:0/4035812262 conn(0x55cd611844e0 msgr2=0x55cd6153c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a37fff640 0 client.32161 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 shutdown_connections 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 1 -- 172.21.15.94:0/4035812262 wait complete. 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.694+0000 7f1a65de2740 20 client.32161 trim_cache size 0 max 16384 2024-04-25T14:03:08.699 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 msgr2=0x7f1a480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 0x7f1a480b2080 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006010 tx=0x7f1a500308a0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 msgr2=0x7f1a5819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b450 tx=0x7f1a4400b910 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 shutdown_connections 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1a480afb70 0x7f1a480b2080 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a5810b9d0 0x7f1a581a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a58108ea0 0x7f1a5819c9b0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 --2- 172.21.15.94:0/3100498847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58103fc0 0x7f1a5819c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 >> 172.21.15.94:0/3100498847 conn(0x7f1a580fd340 msgr2=0x7f1a581099f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 shutdown_connections 2024-04-25T14:03:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.695+0000 7f1a5ee51640 1 -- 172.21.15.94:0/3100498847 wait complete. 2024-04-25T14:03:08.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:08 smithi094 ceph-mon[29195]: pgmap v991: 97 pgs: 97 active+clean; 277 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-04-25T14:03:08.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:08 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4060280326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:08.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:03:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:08 smithi138 ceph-mon[32977]: pgmap v991: 97 pgs: 97 active+clean; 277 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-04-25T14:03:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4060280326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:08.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 -- 172.21.15.94:0/1033332355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 msgr2=0x7f0b4010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 --2- 172.21.15.94:0/1033332355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b4010be40 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009ca0 tx=0x7f0b3802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 -- 172.21.15.94:0/1033332355 shutdown_connections 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 --2- 172.21.15.94:0/1033332355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b4010be40 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 --2- 172.21.15.94:0/1033332355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40104d20 0x7f0b40105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 --2- 172.21.15.94:0/1033332355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40103ae0 0x7f0b401047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 -- 172.21.15.94:0/1033332355 >> 172.21.15.94:0/1033332355 conn(0x7f0b400fecd0 msgr2=0x7f0b40101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 -- 172.21.15.94:0/1033332355 shutdown_connections 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.827+0000 7f0b47a61640 1 -- 172.21.15.94:0/1033332355 wait complete. 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 Processor -- start 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 -- start start 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40103ae0 0x7f0b4019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 0x7f0b4019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4010f610 con 0x7f0b40109a20 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4010f490 con 0x7f0b40103ae0 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b47a61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4010f790 con 0x7f0b40104d20 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b45ffe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b45ffe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40608/0 (socket says 172.21.15.94:40608) 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b45ffe640 1 -- 172.21.15.94:0/940865715 learned_addr learned my addr 172.21.15.94:0/940865715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b457fd640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40103ae0 0x7f0b4019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.828+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 0x7f0b4019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 -- 172.21.15.94:0/940865715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 msgr2=0x7f0b4019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 0x7f0b4019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 -- 172.21.15.94:0/940865715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40103ae0 msgr2=0x7f0b4019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40103ae0 0x7f0b4019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 -- 172.21.15.94:0/940865715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b401a1d30 con 0x7f0b40109a20 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 0x7f0b4019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b45ffe640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0b38004af0 tx=0x7f0b38031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b367fc640 1 -- 172.21.15.94:0/940865715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38003b70 con 0x7f0b40109a20 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b367fc640 1 -- 172.21.15.94:0/940865715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b38003d10 con 0x7f0b40109a20 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b367fc640 1 -- 172.21.15.94:0/940865715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3802fb20 con 0x7f0b40109a20 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b401a1fc0 con 0x7f0b40109a20 2024-04-25T14:03:08.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.829+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b401a2480 con 0x7f0b40109a20 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.832+0000 7f0b4e9ef740 1 Processor -- start 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.832+0000 7f0b367fc640 1 -- 172.21.15.94:0/940865715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0b3802fcc0 con 0x7f0b40109a20 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.832+0000 7f0b4e9ef740 1 -- start start 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb447810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb447d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559bdb3abd80 con 0x559bdb0846b0 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559bdb365a20 con 0x559bdb0a5390 2024-04-25T14:03:08.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bdb3a7d40 con 0x559bdb07c4d0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40614/0 (socket says 172.21.15.94:40614) 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 -- 172.21.15.94:0/489429958 learned_addr learned my addr 172.21.15.94:0/489429958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb447810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b457fd640 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb447d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b367fc640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 0x7f0b2c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 -- 172.21.15.94:0/489429958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 msgr2=0x559bdb447d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb447d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 -- 172.21.15.94:0/489429958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 msgr2=0x559bdb447810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb447810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 -- 172.21.15.94:0/489429958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bdb370c30 con 0x559bdb0846b0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b45ffe640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 0x7f0b2c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b367fc640 1 -- 172.21.15.94:0/940865715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0b381002f0 con 0x7f0b40109a20 2024-04-25T14:03:08.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.833+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f0b30012810 tx=0x7f0b30009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed267f8b4068c492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.834+0000 7f0b2b7fe640 1 -- 172.21.15.94:0/489429958 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30012d90 con 0x559bdb0846b0 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.834+0000 7f0b2b7fe640 1 -- 172.21.15.94:0/489429958 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30010c80 con 0x559bdb0846b0 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.834+0000 7f0b2b7fe640 1 -- 172.21.15.94:0/489429958 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3001ebc0 con 0x559bdb0846b0 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/489429958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 msgr2=0x559bdb448a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b45ffe640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 0x7f0b2c0b2080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0b3804d000 tx=0x7f0b38097040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f0b30012810 tx=0x7f0b30009740 comp rx=0 tx=0).stop 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/489429958 shutdown_connections 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb448a20 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb447d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/489429958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb447810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/489429958 >> 172.21.15.94:0/489429958 conn(0x559bdb0994e0 msgr2=0x559bdb443020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/489429958 shutdown_connections 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/489429958 wait complete. 2024-04-25T14:03:08.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 -- start start 2024-04-25T14:03:08.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.835+0000 7f0b4e9ef740 1 Processor -- start 2024-04-25T14:03:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb451570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb451ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb458a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559bdb449530 con 0x559bdb0846b0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559bdb332dc0 con 0x559bdb0a5390 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bdb4496b0 con 0x559bdb07c4d0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b4e9ef740 10 client.0 _ll_register_callbacks cb 0x559bdb34e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb451ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb451ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40626/0 (socket says 172.21.15.94:40626) 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb451570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b457fd640 1 -- 172.21.15.94:0/1133908024 learned_addr learned my addr 172.21.15.94:0/1133908024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb458a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 -- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 msgr2=0x559bdb458a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb458a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 -- 172.21.15.94:0/1133908024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 msgr2=0x559bdb451ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb451ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.836+0000 7f0b44ffc640 1 -- 172.21.15.94:0/1133908024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bdb0bd3f0 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb451570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0b300096e0 tx=0x7f0b3001b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47723f001d7efdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30010aa0 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0b30004030 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b30010c80 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b30004230 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b4e9ef740 10 client.32173 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.837+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bdb1877d0 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30010c80 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0b30027020 con 0x559bdb07c4d0 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 10 client.32173 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 10 client.32173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 20 client.32173 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb063f60 0x559bdb45efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 4 client.32173 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bdb3b9990 con 0x559bdb063f60 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.838+0000 7f0b457fd640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb063f60 0x559bdb45efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.839+0000 7f0b457fd640 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb063f60 0x559bdb45efe0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44031c0b4078eff server_cookie=2f185aa2936c0e14 in_seq=0 out_seq=0 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.839+0000 7f0b28ff9640 10 client.32173 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b28ff9640 1 -- 172.21.15.94:0/1133908024 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559bdb3b9990 con 0x559bdb063f60 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b28ff9640 10 client.32173 handle_command_reply: tid=0 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 client.32173 shutdown 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 msgr2=0x559bdb451570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb451570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0b300096e0 tx=0x7f0b3001b040 comp rx=0 tx=0).stop 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 shutdown_connections 2024-04-25T14:03:08.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb063f60 0x559bdb45efe0 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb458a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b28ff9640 0 client.32173 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb451ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/1133908024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb451570 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 >> 172.21.15.94:0/1133908024 conn(0x559bdb0994e0 msgr2=0x559bdb445f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 shutdown_connections 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/1133908024 wait complete. 2024-04-25T14:03:08.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.841+0000 7f0b4e9ef740 20 client.32173 trim_cache size 0 max 16384 2024-04-25T14:03:08.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.858+0000 7f0b4e9ef740 1 Processor -- start 2024-04-25T14:03:08.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.858+0000 7f0b4e9ef740 1 -- start start 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb443160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0a5390 0x559bdb451140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ab4e0 con 0x559bdb0a5390 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ad020 con 0x559bdb0846b0 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401abbb0 con 0x559bdb07c4d0 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb443160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40060/0 (socket says 172.21.15.94:40060) 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b457fd640 1 -- 172.21.15.94:0/650350028 learned_addr learned my addr 172.21.15.94:0/650350028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0a5390 0x559bdb451140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 -- 172.21.15.94:0/650350028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 msgr2=0x559bdb4589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 -- 172.21.15.94:0/650350028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0a5390 msgr2=0x559bdb451140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0a5390 0x559bdb451140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 -- 172.21.15.94:0/650350028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bdb44cc80 con 0x559bdb07c4d0 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b457fd640 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.859+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb443160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009650 tx=0x7f0b30005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a103166f1f61fb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.860+0000 7f0b2affd640 1 -- 172.21.15.94:0/650350028 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30009b50 con 0x559bdb07c4d0 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.860+0000 7f0b2affd640 1 -- 172.21.15.94:0/650350028 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30003f20 con 0x559bdb07c4d0 2024-04-25T14:03:08.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.860+0000 7f0b2affd640 1 -- 172.21.15.94:0/650350028 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3001ebf0 con 0x559bdb07c4d0 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/650350028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 msgr2=0x559bdb443160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb443160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009650 tx=0x7f0b30005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/650350028 shutdown_connections 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0a5390 0x559bdb451140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0846b0 0x559bdb4589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/650350028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb443160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/650350028 >> 172.21.15.94:0/650350028 conn(0x559bdb0994e0 msgr2=0x559bdb442a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/650350028 shutdown_connections 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/650350028 wait complete. 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 -- start start 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.861+0000 7f0b4e9ef740 1 Processor -- start 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb455df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb457260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ac950 con 0x559bdb0846b0 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401ac280 con 0x559bdb0a5390 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b401a94c0 con 0x559bdb07c4d0 2024-04-25T14:03:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b4e9ef740 10 client.0 _ll_register_callbacks cb 0x7f0b30009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40642/0 (socket says 172.21.15.94:40642) 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 -- 172.21.15.94:0/4011530323 learned_addr learned my addr 172.21.15.94:0/4011530323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb457260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb455df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 -- 172.21.15.94:0/4011530323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 msgr2=0x559bdb455df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb455df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 -- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 msgr2=0x559bdb457260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb457260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 -- 172.21.15.94:0/4011530323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bdb457820 con 0x559bdb0846b0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb457260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.862+0000 7f0b457fd640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x559bdb45f740 tx=0x7f0b3c010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221528ae2def728e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c023070 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b30003f20 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0b3c004250 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b3c02f050 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 10 client.23139 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bdb458bf0 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b30003f20 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0b3c01f830 con 0x559bdb0846b0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 10 client.23139 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 10 client.23139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 20 client.23139 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb0393a0 0x559bdb4c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 4 client.23139 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.863+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bdb3b3800 con 0x559bdb0393a0 2024-04-25T14:03:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.864+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb0393a0 0x559bdb4c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.864+0000 7f0b28ff9640 10 client.23139 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.864+0000 7f0b44ffc640 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb0393a0 0x559bdb4c9a60 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97ac4144fd6ab68 server_cookie=870106f80cdf41b6 in_seq=0 out_seq=0 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b28ff9640 1 -- 172.21.15.94:0/4011530323 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bdb3b3800 con 0x559bdb0393a0 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b28ff9640 10 client.23139 handle_command_reply: tid=0 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 client.23139 shutdown 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 msgr2=0x559bdb456330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x559bdb45f740 tx=0x7f0b3c010b10 comp rx=0 tx=0).stop 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 shutdown_connections 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559bdb0393a0 0x559bdb4c9a60 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bdb0a5390 0x559bdb457260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b28ff9640 0 client.23139 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bdb0846b0 0x559bdb456330 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 --2- 172.21.15.94:0/4011530323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bdb07c4d0 0x559bdb455df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 >> 172.21.15.94:0/4011530323 conn(0x559bdb0994e0 msgr2=0x559bdb445ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 shutdown_connections 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.865+0000 7f0b4e9ef740 1 -- 172.21.15.94:0/4011530323 wait complete. 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b4e9ef740 20 client.23139 trim_cache size 0 max 16384 2024-04-25T14:03:08.871 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 msgr2=0x7f0b2c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 0x7f0b2c0b2080 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0b3804d000 tx=0x7f0b38097040 comp rx=0 tx=0).stop 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 msgr2=0x7f0b401a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0b38004af0 tx=0x7f0b38031700 comp rx=0 tx=0).stop 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 shutdown_connections 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0b2c0afb70 0x7f0b2c0b2080 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b40109a20 0x7f0b401a16c0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b40104d20 0x7f0b4019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 --2- 172.21.15.94:0/940865715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b40103ae0 0x7f0b4019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 >> 172.21.15.94:0/940865715 conn(0x7f0b400fecd0 msgr2=0x7f0b40105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 shutdown_connections 2024-04-25T14:03:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:08.866+0000 7f0b47a61640 1 -- 172.21.15.94:0/940865715 wait complete. 2024-04-25T14:03:09.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 167912 18.97 MB/sec execute 351 sec latency 68.450 ms 2024-04-25T14:03:09.877 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:03:09.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:09.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:03:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 -- 172.21.15.94:0/26527667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c103fc0 msgr2=0x7f849c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 --2- 172.21.15.94:0/26527667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c103fc0 0x7f849c1043d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c02a410 comp rx=0 tx=0).stop 2024-04-25T14:03:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 -- 172.21.15.94:0/26527667 shutdown_connections 2024-04-25T14:03:09.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 --2- 172.21.15.94:0/26527667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 --2- 172.21.15.94:0/26527667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c108e70 0x7f849c10b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 --2- 172.21.15.94:0/26527667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c103fc0 0x7f849c1043d0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 -- 172.21.15.94:0/26527667 >> 172.21.15.94:0/26527667 conn(0x7f849c0fd340 msgr2=0x7f849c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 -- 172.21.15.94:0/26527667 shutdown_connections 2024-04-25T14:03:09.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.997+0000 7f84a30f2640 1 -- 172.21.15.94:0/26527667 wait complete. 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 Processor -- start 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 -- start start 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 0x7f849c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c1a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f849c111800 con 0x7f849c108e70 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f849c111680 con 0x7f849c10b9a0 2024-04-25T14:03:09.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a30f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f849c111980 con 0x7f849c103fc0 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f849bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f849bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40672/0 (socket says 172.21.15.94:40672) 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f849bfff640 1 -- 172.21.15.94:0/3420063001 learned_addr learned my addr 172.21.15.94:0/3420063001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 0x7f849c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.998+0000 7f84a168f640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c1a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 -- 172.21.15.94:0/3420063001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 msgr2=0x7f849c19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 0x7f849c19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 -- 172.21.15.94:0/3420063001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 msgr2=0x7f849c1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c1a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 -- 172.21.15.94:0/3420063001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1a3f80 con 0x7f849c108e70 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 0x7f849c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f84a168f640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f849bfff640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f8499ffb640 1 -- 172.21.15.94:0/3420063001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000cd80 con 0x7f849c108e70 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f8499ffb640 1 -- 172.21.15.94:0/3420063001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84900054d0 con 0x7f849c108e70 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1a4270 con 0x7f849c108e70 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f8499ffb640 1 -- 172.21.15.94:0/3420063001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490010690 con 0x7f849c108e70 2024-04-25T14:03:10.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:09.999+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1a47c0 con 0x7f849c108e70 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.002+0000 7f8499ffb640 1 -- 172.21.15.94:0/3420063001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8490004e90 con 0x7f849c108e70 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.002+0000 7f84a9fa4740 1 Processor -- start 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.002+0000 7f84a9fa4740 1 -- start start 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.002+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.002+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 0x55baa3a3fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55baa39a4b00 con 0x55baa3675550 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55baa3940920 con 0x55baa367d730 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55baa39a0ac0 con 0x55baa369e410 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f8499ffb640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 0x7f84840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a0e8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 0x55baa3a3fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59310/0 (socket says 172.21.15.94:59310) 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a0e8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 0x55baa3a3fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40676/0 (socket says 172.21.15.94:40676) 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 -- 172.21.15.94:0/2791737521 learned_addr learned my addr 172.21.15.94:0/2791737521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 0x7f84840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f849bfff640 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f8499ffb640 1 -- 172.21.15.94:0/3420063001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f84900df050 con 0x7f849c108e70 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 -- 172.21.15.94:0/2791737521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 msgr2=0x55baa3a46920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a46920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 -- 172.21.15.94:0/2791737521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 msgr2=0x55baa3a3fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 0x55baa3a3fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 -- 172.21.15.94:0/2791737521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa377e7e0 con 0x55baa369e410 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f849bfff640 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a46920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8494011cf0 tx=0x7f849400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffea9b0e872d57fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.003+0000 7f84a168f640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 0x7f84840b2080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f84940099e0 tx=0x7f849401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.004+0000 7f8482ffd640 1 -- 172.21.15.94:0/2791737521 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401eda0 con 0x55baa369e410 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.004+0000 7f8482ffd640 1 -- 172.21.15.94:0/2791737521 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494022030 con 0x55baa369e410 2024-04-25T14:03:10.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.004+0000 7f8482ffd640 1 -- 172.21.15.94:0/2791737521 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940046b0 con 0x55baa369e410 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2791737521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 msgr2=0x55baa3a3f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8494011cf0 tx=0x7f849400f790 comp rx=0 tx=0).stop 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2791737521 shutdown_connections 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a46920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa3675550 0x55baa3a3fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2791737521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a3f4a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2791737521 >> 172.21.15.94:0/2791737521 conn(0x55baa3692560 msgr2=0x55baa3a3ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2791737521 shutdown_connections 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2791737521 wait complete. 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 -- start start 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.005+0000 7f84a9fa4740 1 Processor -- start 2024-04-25T14:03:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a49230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa367d730 0x55baa3a49770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55baa3a47310 con 0x55baa369e410 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55baa393e5d0 con 0x55baa3675550 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55baa3a47490 con 0x55baa367d730 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a9fa4740 10 client.0 _ll_register_callbacks cb 0x55baa392c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a168f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a49230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a168f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40686/0 (socket says 172.21.15.94:40686) 2024-04-25T14:03:10.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a168f640 1 -- 172.21.15.94:0/328429010 learned_addr learned my addr 172.21.15.94:0/328429010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f849bfff640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa367d730 0x55baa3a49770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 -- 172.21.15.94:0/328429010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa367d730 msgr2=0x55baa3a49770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa367d730 0x55baa3a49770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 -- 172.21.15.94:0/328429010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 msgr2=0x55baa3a506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a0e8e640 1 -- 172.21.15.94:0/328429010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa37def70 con 0x55baa3675550 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.006+0000 7f84a168f640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a49230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f848c02a930 tx=0x7f848c0317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d4cf9089bdbb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035040 con 0x55baa3675550 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f848c004450 con 0x55baa3675550 2024-04-25T14:03:10.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8494022030 con 0x55baa3675550 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f848c0035f0 con 0x55baa3675550 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f84a9fa4740 10 client.32531 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55baa3676170 con 0x55baa3675550 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494022030 con 0x55baa3675550 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f848c044070 con 0x55baa3675550 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.007+0000 7f84a9fa4740 10 client.32531 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f84a9fa4740 10 client.32531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f84a9fa4740 20 client.32531 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa365d1f0 0x55baa3a56b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f84a9fa4740 4 client.32531 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:10.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa39b2710 con 0x55baa365d1f0 2024-04-25T14:03:10.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f849bfff640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa365d1f0 0x55baa3a56b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f847bfff640 10 client.32531 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.008+0000 7f849bfff640 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa365d1f0 0x55baa3a56b00 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410e1c1d0678c41 server_cookie=b693cfd8ea18e832 in_seq=0 out_seq=0 2024-04-25T14:03:10.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f847bfff640 1 -- 172.21.15.94:0/328429010 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55baa39b2710 con 0x55baa365d1f0 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f847bfff640 10 client.32531 handle_command_reply: tid=0 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f84a9fa4740 1 client.32531 shutdown 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 msgr2=0x55baa3a49230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a49230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f848c02a930 tx=0x7f848c0317d0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 shutdown_connections 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.010+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa365d1f0 0x55baa3a56b00 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3a506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa367d730 0x55baa3a49770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/328429010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a49230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 >> 172.21.15.94:0/328429010 conn(0x55baa3692560 msgr2=0x55baa3a39ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f847bfff640 0 client.32531 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 shutdown_connections 2024-04-25T14:03:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 1 -- 172.21.15.94:0/328429010 wait complete. 2024-04-25T14:03:10.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.011+0000 7f84a9fa4740 20 client.32531 trim_cache size 0 max 16384 2024-04-25T14:03:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 Processor -- start 2024-04-25T14:03:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 -- start start 2024-04-25T14:03:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1ad6b0 con 0x55baa367d730 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1af1f0 con 0x55baa3675550 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1add80 con 0x55baa369e410 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f849bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f849bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40698/0 (socket says 172.21.15.94:40698) 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.031+0000 7f849bfff640 1 -- 172.21.15.94:0/731800755 learned_addr learned my addr 172.21.15.94:0/731800755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f84a168f640 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 -- 172.21.15.94:0/731800755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 msgr2=0x55baa3a5c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a5c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 -- 172.21.15.94:0/731800755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 msgr2=0x55baa3a3e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a3e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 -- 172.21.15.94:0/731800755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa3a46ad0 con 0x55baa367d730 2024-04-25T14:03:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f849bfff640 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f84900dcab0 tx=0x7f849001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc6eaf3f9bcf46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f84827fc640 1 -- 172.21.15.94:0/731800755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900c1070 con 0x55baa367d730 2024-04-25T14:03:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f84827fc640 1 -- 172.21.15.94:0/731800755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8490005680 con 0x55baa367d730 2024-04-25T14:03:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.032+0000 7f84827fc640 1 -- 172.21.15.94:0/731800755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900046c0 con 0x55baa367d730 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 -- 172.21.15.94:0/731800755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 msgr2=0x55baa3a3e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f84900dcab0 tx=0x7f849001ca20 comp rx=0 tx=0).stop 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 -- 172.21.15.94:0/731800755 shutdown_connections 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa369e410 0x55baa3a5c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa367d730 0x55baa3a3e880 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/731800755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa3675550 0x55baa3a3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 -- 172.21.15.94:0/731800755 >> 172.21.15.94:0/731800755 conn(0x55baa3692560 msgr2=0x55baa3a3dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 -- 172.21.15.94:0/731800755 shutdown_connections 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.033+0000 7f84a9fa4740 1 -- 172.21.15.94:0/731800755 wait complete. 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 -- start start 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 Processor -- start 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3abce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1aeb20 con 0x55baa369e410 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1ae450 con 0x55baa367d730 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1ab690 con 0x55baa3675550 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a9fa4740 10 client.0 _ll_register_callbacks cb 0x7f848c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a168f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3abce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a0e8e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a0e8e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59336/0 (socket says 172.21.15.94:59336) 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f84a0e8e640 1 -- 172.21.15.94:0/2965430668 learned_addr learned my addr 172.21.15.94:0/2965430668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.034+0000 7f849bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 -- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 msgr2=0x55baa3a50370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a50370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 -- 172.21.15.94:0/2965430668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 msgr2=0x55baa3abce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3abce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 -- 172.21.15.94:0/2965430668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa3abd3c0 con 0x55baa3675550 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a168f640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3abce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a0e8e640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c0314c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a454b1b6db002a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f847bfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:10.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c053070 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f848c046bf0 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8490005680 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f848c05f050 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a9fa4740 10 client.32197 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55baa3abe630 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.035+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8490005680 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f848c044070 con 0x55baa3675550 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 10 client.32197 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 10 client.32197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 20 client.32197 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa3632630 0x55baa3ac1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 4 client.32197 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:03:10.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55baa39ac580 con 0x55baa3632630 2024-04-25T14:03:10.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f849bfff640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa3632630 0x55baa3ac1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f847bfff640 10 client.32197 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.036+0000 7f849bfff640 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa3632630 0x55baa3ac1050 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e043f8c39bf527f server_cookie=13e3881aae6b3d36 in_seq=0 out_seq=0 2024-04-25T14:03:10.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f847bfff640 1 -- 172.21.15.94:0/2965430668 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55baa39ac580 con 0x55baa3632630 2024-04-25T14:03:10.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f847bfff640 10 client.32197 handle_command_reply: tid=0 2024-04-25T14:03:10.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 client.32197 shutdown 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 msgr2=0x55baa3a4fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c0314c0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 shutdown_connections 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55baa3632630 0x55baa3ac1050 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55baa369e410 0x55baa3abce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55baa367d730 0x55baa3a50370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 --2- 172.21.15.94:0/2965430668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55baa3675550 0x55baa3a4fe30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 >> 172.21.15.94:0/2965430668 conn(0x55baa3692560 msgr2=0x55baa3a3e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f847bfff640 0 client.32197 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 shutdown_connections 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 1 -- 172.21.15.94:0/2965430668 wait complete. 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.039+0000 7f84a9fa4740 20 client.32197 trim_cache size 0 max 16384 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "9c1c706e-ee1b-4395-b5b2-d2f8b397eac0", 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:03:10.043 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:10.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 msgr2=0x7f84840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 0x7f84840b2080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f84940099e0 tx=0x7f849401f000 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 msgr2=0x7f849c19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 shutdown_connections 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f84840afb70 0x7f84840b2080 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f849c10b9a0 0x7f849c1a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f849c108e70 0x7f849c19c980 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 --2- 172.21.15.94:0/3420063001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f849c103fc0 0x7f849c19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.040+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 >> 172.21.15.94:0/3420063001 conn(0x7f849c0fd340 msgr2=0x7f849c10af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.041+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 shutdown_connections 2024-04-25T14:03:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.041+0000 7f84a30f2640 1 -- 172.21.15.94:0/3420063001 wait complete. 2024-04-25T14:03:10.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:10.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:10.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 168177 18.95 MB/sec execute 352 sec latency 72.807 ms 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 -- 172.21.15.94:0/621970780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938106f40 msgr2=0x7f3938109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 --2- 172.21.15.94:0/621970780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938106f40 0x7f3938109360 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f3920009900 tx=0x7f392002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 -- 172.21.15.94:0/621970780 shutdown_connections 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 --2- 172.21.15.94:0/621970780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f393810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 --2- 172.21.15.94:0/621970780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3938106f40 0x7f3938109360 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 --2- 172.21.15.94:0/621970780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39381045e0 0x7f3938106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 -- 172.21.15.94:0/621970780 >> 172.21.15.94:0/621970780 conn(0x7f39380fdc00 msgr2=0x7f3938100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 -- 172.21.15.94:0/621970780 shutdown_connections 2024-04-25T14:03:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.174+0000 7f393d0bd640 1 -- 172.21.15.94:0/621970780 wait complete. 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 Processor -- start 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 -- start start 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 0x7f393819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f39381a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f620 con 0x7f39381045e0 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f4a0 con 0x7f3938106f40 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393d0bd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f7a0 con 0x7f39381098a0 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40718/0 (socket says 172.21.15.94:40718) 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f3936d9d640 1 -- 172.21.15.94:0/3999024320 learned_addr learned my addr 172.21.15.94:0/3999024320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f392ffff640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 0x7f393819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.175+0000 7f393759e640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f39381a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 -- 172.21.15.94:0/3999024320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 msgr2=0x7f39381a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f39381a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 -- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 msgr2=0x7f393819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 -- 172.21.15.94:0/3999024320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381a1d90 con 0x7f3938106f40 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f393759e640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f39381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f3936d9d640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f392ffff640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 0x7f393819a790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3920006010 tx=0x7f3920004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f3934d99640 1 -- 172.21.15.94:0/3999024320 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392003f070 con 0x7f3938106f40 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f3934d99640 1 -- 172.21.15.94:0/3999024320 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3920031910 con 0x7f3938106f40 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3938101fb0 con 0x7f3938106f40 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f3934d99640 1 -- 172.21.15.94:0/3999024320 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920035070 con 0x7f3938106f40 2024-04-25T14:03:10.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.176+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381024a0 con 0x7f3938106f40 2024-04-25T14:03:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.179+0000 7f3934d99640 1 -- 172.21.15.94:0/3999024320 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f39200037f0 con 0x7f3938106f40 2024-04-25T14:03:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3934d99640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 0x7f39240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 Processor -- start 2024-04-25T14:03:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 -- start start 2024-04-25T14:03:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3936d9d640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 0x7f39240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3934d99640 1 -- 172.21.15.94:0/3999024320 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f39200ff180 con 0x7f3938106f40 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5626dbd55d80 con 0x5626dba2e6b0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626dbd0f8b0 con 0x5626dba264d0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3943f71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5626dbd51d40 con 0x5626dba4f390 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f392ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f392ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40140/0 (socket says 172.21.15.94:40140) 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40730/0 (socket says 172.21.15.94:40730) 2024-04-25T14:03:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59368/0 (socket says 172.21.15.94:59368) 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.180+0000 7f392ffff640 1 -- 172.21.15.94:0/2612017810 learned_addr learned my addr 172.21.15.94:0/2612017810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 0x7f39240b2000 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3928011500 tx=0x7f3928005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 -- 172.21.15.94:0/2612017810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 msgr2=0x5626dbdf0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 -- 172.21.15.94:0/2612017810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 msgr2=0x5626dbdf1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 -- 172.21.15.94:0/2612017810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626dbce0a40 con 0x5626dba4f390 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f392ffff640 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f3936d9d640 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3928011d90 tx=0x7f392801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433a59c84ad0975e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f392d7fa640 1 -- 172.21.15.94:0/2612017810 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928004220 con 0x5626dba4f390 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f392d7fa640 1 -- 172.21.15.94:0/2612017810 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280043c0 con 0x5626dba4f390 2024-04-25T14:03:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.181+0000 7f392d7fa640 1 -- 172.21.15.94:0/2612017810 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928003850 con 0x5626dba4f390 2024-04-25T14:03:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.182+0000 7f3943f71740 1 -- 172.21.15.94:0/2612017810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 msgr2=0x5626dbdf0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.182+0000 7f3943f71740 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3928011d90 tx=0x7f392801c000 comp rx=0 tx=0).stop 2024-04-25T14:03:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- 172.21.15.94:0/2612017810 shutdown_connections 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- 172.21.15.94:0/2612017810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbdf0a60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- 172.21.15.94:0/2612017810 >> 172.21.15.94:0/2612017810 conn(0x5626dba434e0 msgr2=0x5626dbdec270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- 172.21.15.94:0/2612017810 shutdown_connections 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- 172.21.15.94:0/2612017810 wait complete. 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- start start 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 Processor -- start 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5626dbdf2780 con 0x5626dba264d0 2024-04-25T14:03:10.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626dbcdcd10 con 0x5626dba2e6b0 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5626dbdf2900 con 0x5626dba4f390 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.183+0000 7f3943f71740 10 client.0 _ll_register_callbacks cb 0x5626dbd09210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f393759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f392ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40734/0 (socket says 172.21.15.94:40734) 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f393759e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59384/0 (socket says 172.21.15.94:59384) 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 -- 172.21.15.94:0/747671909 learned_addr learned my addr 172.21.15.94:0/747671909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 -- 172.21.15.94:0/747671909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 msgr2=0x5626dbe01dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 -- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 msgr2=0x5626dbdfae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 -- 172.21.15.94:0/747671909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626dbb317d0 con 0x5626dba264d0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f393759e640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f392ffff640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3936d9d640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f392801cab0 tx=0x7f392801cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c62b06134ca460c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f391e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928031020 con 0x5626dba264d0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f39280046c0 con 0x5626dba264d0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.184+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39280043c0 con 0x5626dba264d0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f392803c050 con 0x5626dba264d0 2024-04-25T14:03:10.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 10 client.23169 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5626dba673f0 con 0x5626dba264d0 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280043c0 con 0x5626dba264d0 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3928023070 con 0x5626dba264d0 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 10 client.23169 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 10 client.23169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 20 client.23169 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba0df60 0x5626dbe07f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 4 client.23169 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.185+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626dbd63990 con 0x5626dba0df60 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.186+0000 7f392ffff640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba0df60 0x5626dbe07f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.186+0000 7f391e7fc640 10 client.23169 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.186+0000 7f392ffff640 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba0df60 0x5626dbe07f90 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6c1453e3a53e2a server_cookie=439a049fdae4408b in_seq=0 out_seq=0 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.189+0000 7f391e7fc640 1 -- 172.21.15.94:0/747671909 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626dbd63990 con 0x5626dba0df60 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.189+0000 7f391e7fc640 10 client.23169 handle_command_reply: tid=0 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 client.23169 shutdown 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 msgr2=0x5626dbdfa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f392801cab0 tx=0x7f392801cae0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 shutdown_connections 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba0df60 0x5626dbe07f90 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba4f390 0x5626dbe01dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f391e7fc640 0 client.23169 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 --2- 172.21.15.94:0/747671909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfa930 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 >> 172.21.15.94:0/747671909 conn(0x5626dba434e0 msgr2=0x5626dbdeb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 shutdown_connections 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 1 -- 172.21.15.94:0/747671909 wait complete. 2024-04-25T14:03:10.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.190+0000 7f3943f71740 20 client.23169 trim_cache size 0 max 16384 2024-04-25T14:03:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 Processor -- start 2024-04-25T14:03:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 -- start start 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbdfaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a9310 con 0x5626dba2e6b0 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aae50 con 0x5626dba264d0 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.213+0000 7f3943f71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a99e0 con 0x5626dba0df60 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbdfaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40748/0 (socket says 172.21.15.94:40748) 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 -- 172.21.15.94:0/348941683 learned_addr learned my addr 172.21.15.94:0/348941683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f392ffff640 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 -- 172.21.15.94:0/348941683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 msgr2=0x5626dbdfaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbdfaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 -- 172.21.15.94:0/348941683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 msgr2=0x5626dbdf5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 -- 172.21.15.94:0/348941683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626dba270f0 con 0x5626dba2e6b0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f393759e640 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f393001a390 tx=0x7f3930009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4156f320e18d3283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f392cff9640 1 -- 172.21.15.94:0/348941683 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930004d20 con 0x5626dba2e6b0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f392cff9640 1 -- 172.21.15.94:0/348941683 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39300054d0 con 0x5626dba2e6b0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.214+0000 7f392cff9640 1 -- 172.21.15.94:0/348941683 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393001c920 con 0x5626dba2e6b0 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 -- 172.21.15.94:0/348941683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 msgr2=0x5626dbdf0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f393001a390 tx=0x7f3930009500 comp rx=0 tx=0).stop 2024-04-25T14:03:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 -- 172.21.15.94:0/348941683 shutdown_connections 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba2e6b0 0x5626dbdf0a20 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba264d0 0x5626dbdf5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 --2- 172.21.15.94:0/348941683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbdfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 -- 172.21.15.94:0/348941683 >> 172.21.15.94:0/348941683 conn(0x5626dba434e0 msgr2=0x5626dbdf8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 -- 172.21.15.94:0/348941683 shutdown_connections 2024-04-25T14:03:10.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.215+0000 7f3943f71740 1 -- 172.21.15.94:0/348941683 wait complete. 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 -- start start 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 Processor -- start 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aa780 con 0x5626dba264d0 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aa0b0 con 0x5626dba2e6b0 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a72f0 con 0x5626dba0df60 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3943f71740 10 client.0 _ll_register_callbacks cb 0x7f3930009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:10.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f392ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59392/0 (socket says 172.21.15.94:59392) 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 -- 172.21.15.94:0/491042685 learned_addr learned my addr 172.21.15.94:0/491042685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f393759e640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 -- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 msgr2=0x5626dbdfe3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfe3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 -- 172.21.15.94:0/491042685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 msgr2=0x5626dbdfde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.216+0000 7f3936d9d640 1 -- 172.21.15.94:0/491042685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626dbd164b0 con 0x5626dba0df60 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f392ffff640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3936d9d640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3928009880 tx=0x7f392801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666ac2823c6670e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928031040 con 0x5626dba0df60 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39300054d0 con 0x5626dba0df60 2024-04-25T14:03:10.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f392802d530 con 0x5626dba0df60 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3928028040 con 0x5626dba0df60 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 10 client.32215 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5626dbe6f220 con 0x5626dba0df60 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39300054d0 con 0x5626dba0df60 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3928023040 con 0x5626dba0df60 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 10 client.32215 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 10 client.32215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 20 client.32215 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba4f390 0x5626dbe71b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 4 client.32215 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.217+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626dbd5d800 con 0x5626dba4f390 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.218+0000 7f392ffff640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba4f390 0x5626dbe71b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.218+0000 7f391e7fc640 10 client.32215 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.218+0000 7f392ffff640 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba4f390 0x5626dbe71b70 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9592a60a475dc381 server_cookie=1b3173da2a94aa35 in_seq=0 out_seq=0 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f391e7fc640 1 -- 172.21.15.94:0/491042685 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5626dbd5d800 con 0x5626dba4f390 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f391e7fc640 10 client.32215 handle_command_reply: tid=0 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 client.32215 shutdown 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 msgr2=0x5626dbe01280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3928009880 tx=0x7f392801c470 comp rx=0 tx=0).stop 2024-04-25T14:03:10.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 shutdown_connections 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5626dba4f390 0x5626dbe71b70 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626dba2e6b0 0x5626dbdfe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626dba264d0 0x5626dbdfde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f391e7fc640 0 client.32215 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 --2- 172.21.15.94:0/491042685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5626dba0df60 0x5626dbe01280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 >> 172.21.15.94:0/491042685 conn(0x5626dba434e0 msgr2=0x5626dbdfac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 shutdown_connections 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.221+0000 7f3943f71740 1 -- 172.21.15.94:0/491042685 wait complete. 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f3943f71740 20 client.32215 trim_cache size 0 max 16384 2024-04-25T14:03:10.224 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (495 inodes in the stack)", 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "9c1c706e-ee1b-4395-b5b2-d2f8b397eac0": { 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "9c1c706e-ee1b-4395-b5b2-d2f8b397eac0", 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:03:10.225 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:10.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 msgr2=0x7f39240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 0x7f39240b2000 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3928011500 tx=0x7f3928005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 msgr2=0x7f393819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 0x7f393819a790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3920006010 tx=0x7f3920004270 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 shutdown_connections 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f39240afaf0 0x7f39240b2000 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39381098a0 0x7f39381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3938106f40 0x7f393819a790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 --2- 172.21.15.94:0/3999024320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39381045e0 0x7f393819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 >> 172.21.15.94:0/3999024320 conn(0x7f39380fdc00 msgr2=0x7f39381051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.222+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 shutdown_connections 2024-04-25T14:03:10.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:10.223+0000 7f393d0bd640 1 -- 172.21.15.94:0/3999024320 wait complete. 2024-04-25T14:03:10.233 INFO:tasks.cephfs.filesystem:scrub status for tag:9c1c706e-ee1b-4395-b5b2-d2f8b397eac0 - {'path': '/', 'tag': '9c1c706e-ee1b-4395-b5b2-d2f8b397eac0', 'options': 'recursive,force'} 2024-04-25T14:03:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:10 smithi137 ceph-mon[32568]: pgmap v992: 97 pgs: 97 active+clean; 276 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 337 op/s 2024-04-25T14:03:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:10 smithi094 ceph-mon[29195]: pgmap v992: 97 pgs: 97 active+clean; 276 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 337 op/s 2024-04-25T14:03:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:10 smithi138 ceph-mon[32977]: pgmap v992: 97 pgs: 97 active+clean; 276 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 337 op/s 2024-04-25T14:03:11.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 168401 18.91 MB/sec execute 353 sec latency 292.537 ms 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:11 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:03:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:11 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:11.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:11 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:12.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 168731 18.89 MB/sec execute 354 sec latency 120.847 ms 2024-04-25T14:03:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:12 smithi137 ceph-mon[32568]: pgmap v993: 97 pgs: 97 active+clean; 266 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-04-25T14:03:12.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:12 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:12 smithi094 ceph-mon[29195]: pgmap v993: 97 pgs: 97 active+clean; 266 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-04-25T14:03:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:12 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:12 smithi138 ceph-mon[32977]: pgmap v993: 97 pgs: 97 active+clean; 266 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-04-25T14:03:12.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:12 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:13.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 169197 18.91 MB/sec execute 355 sec latency 168.150 ms 2024-04-25T14:03:14.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 169767 18.95 MB/sec execute 356 sec latency 53.317 ms 2024-04-25T14:03:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:14 smithi137 ceph-mon[32568]: pgmap v994: 97 pgs: 97 active+clean; 262 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-04-25T14:03:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:14 smithi094 ceph-mon[29195]: pgmap v994: 97 pgs: 97 active+clean; 262 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-04-25T14:03:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:14 smithi138 ceph-mon[32977]: pgmap v994: 97 pgs: 97 active+clean; 262 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 501 op/s 2024-04-25T14:03:15.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 170295 18.98 MB/sec execute 357 sec latency 51.476 ms 2024-04-25T14:03:16.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 170655 18.98 MB/sec execute 358 sec latency 198.151 ms 2024-04-25T14:03:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:16 smithi137 ceph-mon[32568]: pgmap v995: 97 pgs: 97 active+clean; 264 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 585 op/s 2024-04-25T14:03:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:16 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:16 smithi094 ceph-mon[29195]: pgmap v995: 97 pgs: 97 active+clean; 264 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 585 op/s 2024-04-25T14:03:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:16 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:16 smithi138 ceph-mon[32977]: pgmap v995: 97 pgs: 97 active+clean; 264 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 585 op/s 2024-04-25T14:03:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:16 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:17.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 170933 18.99 MB/sec execute 359 sec latency 152.803 ms 2024-04-25T14:03:17.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:17.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:17.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:17 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:17.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:17 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:17.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:17 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:18.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 171280 19.00 MB/sec execute 360 sec latency 68.925 ms 2024-04-25T14:03:18.625 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:18 smithi094 ceph-mon[29195]: pgmap v996: 97 pgs: 97 active+clean; 268 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 551 op/s 2024-04-25T14:03:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:18 smithi137 ceph-mon[32568]: pgmap v996: 97 pgs: 97 active+clean; 268 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 551 op/s 2024-04-25T14:03:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:18 smithi138 ceph-mon[32977]: pgmap v996: 97 pgs: 97 active+clean; 268 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 551 op/s 2024-04-25T14:03:19.154 INFO:tasks.workunit.client.0.smithi094.stdout: 10 171564 18.99 MB/sec execute 361 sec latency 78.000 ms 2024-04-25T14:03:20.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 171829 18.96 MB/sec execute 362 sec latency 106.812 ms 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- 172.21.15.94:0/3449505369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 msgr2=0x7f0cb810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/3449505369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb810a530 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009ca0 tx=0x7f0cb002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- 172.21.15.94:0/3449505369 shutdown_connections 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/3449505369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb810a530 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/3449505369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb8103410 0x7f0cb8103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/3449505369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb81021d0 0x7f0cb8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- 172.21.15.94:0/3449505369 >> 172.21.15.94:0/3449505369 conn(0x7f0cb80fd3c0 msgr2=0x7f0cb80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- 172.21.15.94:0/3449505369 shutdown_connections 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- 172.21.15.94:0/3449505369 wait complete. 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 Processor -- start 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.584+0000 7f0cc06f2640 1 -- start start 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb81021d0 0x7f0cb810f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb8103410 0x7f0cb819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb810dcf0 con 0x7f0cb8108110 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb810db70 con 0x7f0cb8103410 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb810de70 con 0x7f0cb81021d0 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57040/0 (socket says 172.21.15.94:57040) 2024-04-25T14:03:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 -- 172.21.15.94:0/955809416 learned_addr learned my addr 172.21.15.94:0/955809416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb81021d0 0x7f0cb810f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb8103410 0x7f0cb819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 -- 172.21.15.94:0/955809416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb81021d0 msgr2=0x7f0cb810f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb81021d0 0x7f0cb810f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 -- 172.21.15.94:0/955809416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb8103410 msgr2=0x7f0cb819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb8103410 0x7f0cb819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 -- 172.21.15.94:0/955809416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb81a1cc0 con 0x7f0cb8108110 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0004af0 tx=0x7f0cb0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0caf7fe640 1 -- 172.21.15.94:0/955809416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0003b70 con 0x7f0cb8108110 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb81a1f50 con 0x7f0cb8108110 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0caf7fe640 1 -- 172.21.15.94:0/955809416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb0003d10 con 0x7f0cb8108110 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.585+0000 7f0caf7fe640 1 -- 172.21.15.94:0/955809416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb002fb20 con 0x7f0cb8108110 2024-04-25T14:03:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.586+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a23c0 con 0x7f0cb8108110 2024-04-25T14:03:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.588+0000 7f0caf7fe640 1 -- 172.21.15.94:0/955809416 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0cb0004580 con 0x7f0cb8108110 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.588+0000 7f0caf7fe640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 0x7f0ca40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.588+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 0x7f0ca40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.588+0000 7f0cc75a6740 1 Processor -- start 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.588+0000 7f0caf7fe640 1 -- 172.21.15.94:0/955809416 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0cb0100280 con 0x7f0cb8108110 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 -- start start 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 0x7f0ca40b1ff0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8102dd0 tx=0x7f0cb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591502f0 0x55d259519ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d259520df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d25947f460 con 0x55d2591791b0 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d259439590 con 0x55d2591502f0 2024-04-25T14:03:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d25947b420 con 0x55d2591584d0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57052/0 (socket says 172.21.15.94:57052) 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 -- 172.21.15.94:0/4056883722 learned_addr learned my addr 172.21.15.94:0/4056883722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d259520df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591502f0 0x55d259519ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 -- 172.21.15.94:0/4056883722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 msgr2=0x55d259520df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d259520df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 -- 172.21.15.94:0/4056883722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591502f0 msgr2=0x55d259519ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591502f0 0x55d259519ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.589+0000 7f0cbe48e640 1 -- 172.21.15.94:0/4056883722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2594460d0 con 0x55d2591791b0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.590+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e630 tx=0x7f0cb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b126c6960f1e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.590+0000 7f0cacff9640 1 -- 172.21.15.94:0/4056883722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4026430 con 0x55d2591791b0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.590+0000 7f0cacff9640 1 -- 172.21.15.94:0/4056883722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb401c8c0 con 0x55d2591791b0 2024-04-25T14:03:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.590+0000 7f0cacff9640 1 -- 172.21.15.94:0/4056883722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb401d8f0 con 0x55d2591791b0 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- 172.21.15.94:0/4056883722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 msgr2=0x55d259519970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e630 tx=0x7f0cb4014000 comp rx=0 tx=0).stop 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- 172.21.15.94:0/4056883722 shutdown_connections 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d259520df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591502f0 0x55d259519ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/4056883722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591791b0 0x55d259519970 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- 172.21.15.94:0/4056883722 >> 172.21.15.94:0/4056883722 conn(0x55d25916d300 msgr2=0x55d259515300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- 172.21.15.94:0/4056883722 shutdown_connections 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- 172.21.15.94:0/4056883722 wait complete. 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 -- start start 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.591+0000 7f0cc75a6740 1 Processor -- start 2024-04-25T14:03:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259523640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d259521840 con 0x55d2591584d0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d259406b60 con 0x55d2591791b0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2595219c0 con 0x55d2591502f0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 10 client.0 _ll_register_callbacks cb 0x55d25942b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57060/0 (socket says 172.21.15.94:57060) 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/1722994298 learned_addr learned my addr 172.21.15.94:0/1722994298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259523640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/1722994298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 msgr2=0x55d259523640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259523640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 msgr2=0x55d25952aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/1722994298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d259191210 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80126e0 tx=0x7f0ca8012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee693d8e44ef3bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8012190 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ca800f920 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.592+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb401c8c0 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ca800fb20 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 10 client.23187 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d25925d620 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb401c8c0 con 0x55d2591584d0 2024-04-25T14:03:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0ca8026450 con 0x55d2591584d0 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 10 client.23187 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 10 client.23187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 20 client.23187 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d259137d80 0x55d259531370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 4 client.23187 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25948d070 con 0x55d259137d80 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.593+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d259137d80 0x55d259531370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.594+0000 7f0ca1ffb640 10 client.23187 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.594+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d259137d80 0x55d259531370 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5265bd77615e4629 server_cookie=8cdb74ef0448761d in_seq=0 out_seq=0 2024-04-25T14:03:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/1722994298 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d25948d070 con 0x55d259137d80 2024-04-25T14:03:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0ca1ffb640 10 client.23187 handle_command_reply: tid=0 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 client.23187 shutdown 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 msgr2=0x55d259523b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80126e0 tx=0x7f0ca8012ba0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 shutdown_connections 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d259137d80 0x55d259531370 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259523b80 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/1722994298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259523640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 >> 172.21.15.94:0/1722994298 conn(0x55d25916d300 msgr2=0x55d259514250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0ca1ffb640 0 client.23187 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 shutdown_connections 2024-04-25T14:03:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 1 -- 172.21.15.94:0/1722994298 wait complete. 2024-04-25T14:03:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.596+0000 7f0cc75a6740 20 client.23187 trim_cache size 0 max 16384 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 Processor -- start 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 -- start start 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d25951dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81ab500 con 0x55d2591502f0 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81ad040 con 0x55d2591791b0 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81abbd0 con 0x55d2591584d0 2024-04-25T14:03:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57068/0 (socket says 172.21.15.94:57068) 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 -- 172.21.15.94:0/363535593 learned_addr learned my addr 172.21.15.94:0/363535593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d25951dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 -- 172.21.15.94:0/363535593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 msgr2=0x55d25951dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d25951dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 -- 172.21.15.94:0/363535593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 msgr2=0x55d25952aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 -- 172.21.15.94:0/363535593 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25952af40 con 0x55d2591502f0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d25951dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.613+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f0cb401c6b0 tx=0x7f0cb400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb928f8cb7d94c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.614+0000 7f0ca3fff640 1 -- 172.21.15.94:0/363535593 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb401c410 con 0x55d2591502f0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.614+0000 7f0ca3fff640 1 -- 172.21.15.94:0/363535593 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb40054e0 con 0x55d2591502f0 2024-04-25T14:03:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.614+0000 7f0ca3fff640 1 -- 172.21.15.94:0/363535593 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40048c0 con 0x55d2591502f0 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.614+0000 7f0cc75a6740 1 -- 172.21.15.94:0/363535593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 msgr2=0x55d25951dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.614+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f0cb401c6b0 tx=0x7f0cb400eaf0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- 172.21.15.94:0/363535593 shutdown_connections 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d25952aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591584d0 0x55d25951dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/363535593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591502f0 0x55d25951dab0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- 172.21.15.94:0/363535593 >> 172.21.15.94:0/363535593 conn(0x55d25916d300 msgr2=0x55d259519930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- 172.21.15.94:0/363535593 shutdown_connections 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- 172.21.15.94:0/363535593 wait complete. 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- start start 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 Processor -- start 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259527920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d259597710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81ac970 con 0x55d2591584d0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81ac2a0 con 0x55d2591791b0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81a94e0 con 0x55d2591502f0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cc75a6740 10 client.0 _ll_register_callbacks cb 0x7f0ca8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cbdc8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cbdc8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57076/0 (socket says 172.21.15.94:57076) 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/214964950 learned_addr learned my addr 172.21.15.94:0/214964950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259527920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.615+0000 7f0cbec8f640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d259597710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/214964950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 msgr2=0x55d259527920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259527920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 msgr2=0x55d259597710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d259597710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 -- 172.21.15.94:0/214964950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d259597c50 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cbdc8d640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8026a10 tx=0x7f0ca8012380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2850802258d71c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8026e60 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ca800fcc0 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb40054e0 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ca8034050 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 10 client.23193 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d259598e50 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb40054e0 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0ca8026a40 con 0x55d2591584d0 2024-04-25T14:03:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 10 client.23193 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 10 client.23193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 20 client.23193 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d25910d1c0 0x55d25959b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 4 client.23193 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.616+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d259486ee0 con 0x55d25910d1c0 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.617+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d25910d1c0 0x55d25959b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.617+0000 7f0ca1ffb640 10 client.23193 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:20.617+0000 7f0cbe48e640 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d25910d1c0 0x55d25959b8c0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f03770d3ceec10e server_cookie=4c6147395b925811 in_seq=0 out_seq=0 2024-04-25T14:03:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:20 smithi137 ceph-mon[32568]: pgmap v997: 97 pgs: 97 active+clean; 272 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-04-25T14:03:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:20 smithi094 ceph-mon[29195]: pgmap v997: 97 pgs: 97 active+clean; 272 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-04-25T14:03:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:20 smithi138 ceph-mon[32977]: pgmap v997: 97 pgs: 97 active+clean; 272 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 577 op/s 2024-04-25T14:03:21.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 172008 18.92 MB/sec execute 363 sec latency 286.016 ms 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.483+0000 7f0ca1ffb640 1 -- 172.21.15.94:0/214964950 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d259486ee0 con 0x55d25910d1c0 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.483+0000 7f0ca1ffb640 10 client.23193 handle_command_reply: tid=0 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.483+0000 7f0cc75a6740 1 client.23193 shutdown 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 msgr2=0x55d259527e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8026a10 tx=0x7f0ca8012380 comp rx=0 tx=0).stop 2024-04-25T14:03:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 shutdown_connections 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d25910d1c0 0x55d25959b8c0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2591791b0 0x55d259597710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0ca1ffb640 0 client.23193 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2591584d0 0x55d259527e60 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 --2- 172.21.15.94:0/214964950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2591502f0 0x55d259527920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 >> 172.21.15.94:0/214964950 conn(0x55d25916d300 msgr2=0x55d25951dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 shutdown_connections 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 1 -- 172.21.15.94:0/214964950 wait complete. 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.484+0000 7f0cc75a6740 20 client.23193 trim_cache size 0 max 16384 2024-04-25T14:03:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:03:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:03:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:03:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:03:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 msgr2=0x7f0ca40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 0x7f0ca40b1ff0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8102dd0 tx=0x7f0cb4005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 msgr2=0x7f0cb81a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0004af0 tx=0x7f0cb0031700 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 shutdown_connections 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ca40afae0 0x7f0ca40b1ff0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb8108110 0x7f0cb81a14a0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb8103410 0x7f0cb819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 --2- 172.21.15.94:0/955809416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb81021d0 0x7f0cb810f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 >> 172.21.15.94:0/955809416 conn(0x7f0cb80fd3c0 msgr2=0x7f0cb80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 shutdown_connections 2024-04-25T14:03:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:21.485+0000 7f0cc06f2640 1 -- 172.21.15.94:0/955809416 wait complete. 2024-04-25T14:03:22.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 172241 18.90 MB/sec execute 364 sec latency 583.415 ms 2024-04-25T14:03:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:22 smithi137 ceph-mon[32568]: pgmap v998: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-04-25T14:03:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:22 smithi094 ceph-mon[29195]: pgmap v998: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-04-25T14:03:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:22 smithi138 ceph-mon[32977]: pgmap v998: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2024-04-25T14:03:23.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 172671 18.90 MB/sec execute 365 sec latency 81.261 ms 2024-04-25T14:03:24.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 173187 18.93 MB/sec execute 366 sec latency 50.955 ms 2024-04-25T14:03:24.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:24 smithi137 ceph-mon[32568]: pgmap v999: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-25T14:03:24.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:24 smithi094 ceph-mon[29195]: pgmap v999: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-25T14:03:24.750 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:24 smithi138 ceph-mon[32977]: pgmap v999: 97 pgs: 97 active+clean; 273 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-25T14:03:25.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 173722 18.96 MB/sec execute 367 sec latency 60.180 ms 2024-04-25T14:03:26.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 174218 18.99 MB/sec execute 368 sec latency 66.470 ms 2024-04-25T14:03:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:26 smithi137 ceph-mon[32568]: pgmap v1000: 97 pgs: 97 active+clean; 271 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-04-25T14:03:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:26 smithi094 ceph-mon[29195]: pgmap v1000: 97 pgs: 97 active+clean; 271 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-04-25T14:03:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:26 smithi138 ceph-mon[32977]: pgmap v1000: 97 pgs: 97 active+clean; 271 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-04-25T14:03:27.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 174419 18.97 MB/sec execute 369 sec latency 364.935 ms 2024-04-25T14:03:28.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 174737 18.98 MB/sec execute 370 sec latency 77.366 ms 2024-04-25T14:03:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:28 smithi137 ceph-mon[32568]: pgmap v1001: 97 pgs: 97 active+clean; 267 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-04-25T14:03:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:28 smithi094 ceph-mon[29195]: pgmap v1001: 97 pgs: 97 active+clean; 267 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-04-25T14:03:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:28 smithi138 ceph-mon[32977]: pgmap v1001: 97 pgs: 97 active+clean; 267 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 520 op/s 2024-04-25T14:03:29.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 174984 18.97 MB/sec execute 371 sec latency 219.916 ms 2024-04-25T14:03:30.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 175263 18.95 MB/sec execute 372 sec latency 220.087 ms 2024-04-25T14:03:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:30 smithi137 ceph-mon[32568]: pgmap v1002: 97 pgs: 97 active+clean; 269 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-25T14:03:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:30 smithi094 ceph-mon[29195]: pgmap v1002: 97 pgs: 97 active+clean; 269 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-25T14:03:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:30 smithi138 ceph-mon[32977]: pgmap v1002: 97 pgs: 97 active+clean; 269 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 484 op/s 2024-04-25T14:03:31.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 175521 18.93 MB/sec execute 373 sec latency 121.332 ms 2024-04-25T14:03:32.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 175783 18.91 MB/sec execute 374 sec latency 161.798 ms 2024-04-25T14:03:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:32 smithi137 ceph-mon[32568]: pgmap v1003: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 538 op/s 2024-04-25T14:03:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:32 smithi094 ceph-mon[29195]: pgmap v1003: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 538 op/s 2024-04-25T14:03:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:32 smithi138 ceph-mon[32977]: pgmap v1003: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.8 MiB/s wr, 538 op/s 2024-04-25T14:03:33.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 176185 18.91 MB/sec execute 375 sec latency 87.222 ms 2024-04-25T14:03:34.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 176661 18.92 MB/sec execute 376 sec latency 69.281 ms 2024-04-25T14:03:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 -- 172.21.15.94:0/2482693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c4101790 msgr2=0x7fa9c4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/2482693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c4101790 0x7fa9c4101c00 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009900 tx=0x7fa9b802a410 comp rx=0 tx=0).stop 2024-04-25T14:03:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 -- 172.21.15.94:0/2482693179 shutdown_connections 2024-04-25T14:03:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/2482693179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4102290 0x7fa9c410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/2482693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c4101790 0x7fa9c4101c00 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/2482693179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c41050e0 0x7fa9c41054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 -- 172.21.15.94:0/2482693179 >> 172.21.15.94:0/2482693179 conn(0x7fa9c40fd390 msgr2=0x7fa9c40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 -- 172.21.15.94:0/2482693179 shutdown_connections 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.590+0000 7fa9c9e20640 1 -- 172.21.15.94:0/2482693179 wait complete. 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.591+0000 7fa9c9e20640 1 Processor -- start 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.591+0000 7fa9c9e20640 1 -- start start 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.591+0000 7fa9c9e20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4101790 0x7fa9c419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4102290 0x7fa9c419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f670 con 0x7fa9c41050e0 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f4f0 con 0x7fa9c4101790 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410f7f0 con 0x7fa9c4102290 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54842/0 (socket says 172.21.15.94:54842) 2024-04-25T14:03:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 -- 172.21.15.94:0/301654980 learned_addr learned my addr 172.21.15.94:0/301654980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4101790 0x7fa9c419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4102290 0x7fa9c419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 -- 172.21.15.94:0/301654980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4102290 msgr2=0x7fa9c419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4102290 0x7fa9c419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 -- 172.21.15.94:0/301654980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4101790 msgr2=0x7fa9c419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4101790 0x7fa9c419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 -- 172.21.15.94:0/301654980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c41a1dc0 con 0x7fa9c41050e0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c3fff640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009fd0 tx=0x7fa9bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c0ff9640 1 -- 172.21.15.94:0/301654980 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc017070 con 0x7fa9c41050e0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c0ff9640 1 -- 172.21.15.94:0/301654980 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9bc003e20 con 0x7fa9c41050e0 2024-04-25T14:03:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41a20b0 con 0x7fa9c41050e0 2024-04-25T14:03:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.592+0000 7fa9c0ff9640 1 -- 172.21.15.94:0/301654980 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc010460 con 0x7fa9c41050e0 2024-04-25T14:03:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.593+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a25b0 con 0x7fa9c41050e0 2024-04-25T14:03:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 Processor -- start 2024-04-25T14:03:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c0ff9640 1 -- 172.21.15.94:0/301654980 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa9bc010640 con 0x7fa9c41050e0 2024-04-25T14:03:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 -- start start 2024-04-25T14:03:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 0x557516faad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 0x557516fab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516be54d0 0x557516fb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557516f0c460 con 0x557516c061b0 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557516ec6590 con 0x557516be54d0 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557516f08420 con 0x557516bdd2f0 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 0x557516faad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 0x557516faad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54858/0 (socket says 172.21.15.94:54858) 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/4240336631 learned_addr learned my addr 172.21.15.94:0/4240336631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 0x557516fab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.596+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516be54d0 0x557516fb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c0ff9640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 0x7fa9ac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 -- 172.21.15.94:0/4240336631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516be54d0 msgr2=0x557516fb2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516be54d0 0x557516fb2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 -- 172.21.15.94:0/4240336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 msgr2=0x557516faad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 0x557516faad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 -- 172.21.15.94:0/4240336631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557516ed30d0 con 0x557516bdd2f0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c3fff640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 0x7fa9ac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c0ff9640 1 -- 172.21.15.94:0/301654980 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9bc0dea40 con 0x7fa9c41050e0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 0x557516fab300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9c406a2f0 tx=0x7fa9b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a959ddaff0771b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9a9ffb640 1 -- 172.21.15.94:0/4240336631 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b401e070 con 0x557516bdd2f0 2024-04-25T14:03:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9a9ffb640 1 -- 172.21.15.94:0/4240336631 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b4009d50 con 0x557516bdd2f0 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.597+0000 7fa9a9ffb640 1 -- 172.21.15.94:0/4240336631 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4003e60 con 0x557516bdd2f0 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 -- 172.21.15.94:0/4240336631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 msgr2=0x557516fab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 0x557516fab300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9c406a2f0 tx=0x7fa9b4011810 comp rx=0 tx=0).stop 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9c3fff640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 0x7fa9ac0b1ff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0dc4f0 tx=0x7fa9bc0d55d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 -- 172.21.15.94:0/4240336631 shutdown_connections 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516be54d0 0x557516fb2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bdd2f0 0x557516fab300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/4240336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516c061b0 0x557516faad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 -- 172.21.15.94:0/4240336631 >> 172.21.15.94:0/4240336631 conn(0x557516bfa300 msgr2=0x557516fa2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 -- 172.21.15.94:0/4240336631 shutdown_connections 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.598+0000 7fa9d0daf740 1 -- 172.21.15.94:0/4240336631 wait complete. 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 -- start start 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 Processor -- start 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x557516fb4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516c061b0 0x557516fbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557516fb2c50 con 0x557516bdd2f0 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557516e93b60 con 0x557516c061b0 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557516fb2dd0 con 0x557516be54d0 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9d0daf740 10 client.0 _ll_register_callbacks cb 0x557516eb82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54862/0 (socket says 172.21.15.94:54862) 2024-04-25T14:03:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/557777603 learned_addr learned my addr 172.21.15.94:0/557777603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x557516fb4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.599+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516c061b0 0x557516fbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/557777603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 msgr2=0x557516fb4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x557516fb4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516c061b0 msgr2=0x557516fbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516c061b0 0x557516fbbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/557777603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557516c1e210 con 0x557516bdd2f0 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8006010 tx=0x7fa9b8005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed46d767d6c9f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8035070 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9b8004260 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b4009d50 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9b8053050 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9d0daf740 10 client.23205 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557516cea620 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.600+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b4009d50 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b803b460 con 0x557516bdd2f0 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 10 client.23205 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 10 client.23205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 20 client.23205 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516bc4d80 0x557516fc26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 4 client.23205 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557516f1a070 con 0x557516bc4d80 2024-04-25T14:03:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516bc4d80 0x557516fc26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9a2ffd640 10 client.23205 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.601+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516bc4d80 0x557516fc26a0 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e890b69047443650 server_cookie=6c668d494ff32b2f in_seq=0 out_seq=0 2024-04-25T14:03:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.604+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/557777603 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557516f1a070 con 0x557516bc4d80 2024-04-25T14:03:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.604+0000 7fa9a2ffd640 10 client.23205 handle_command_reply: tid=0 2024-04-25T14:03:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.604+0000 7fa9d0daf740 1 client.23205 shutdown 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 msgr2=0x557516fb4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8006010 tx=0x7fa9b8005e90 comp rx=0 tx=0).stop 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 shutdown_connections 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516bc4d80 0x557516fc26a0 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516c061b0 0x557516fbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x557516fb4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9a2ffd640 0 client.23205 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/557777603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bdd2f0 0x557516fb4a90 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 >> 172.21.15.94:0/557777603 conn(0x557516bfa300 msgr2=0x557516fa5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 shutdown_connections 2024-04-25T14:03:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 1 -- 172.21.15.94:0/557777603 wait complete. 2024-04-25T14:03:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.605+0000 7fa9d0daf740 20 client.23205 trim_cache size 0 max 16384 2024-04-25T14:03:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.635+0000 7fa9d0daf740 1 Processor -- start 2024-04-25T14:03:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 -- start start 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bc4d80 0x557516fa1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fa2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a92f0 con 0x557516be54d0 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41aae30 con 0x557516bdd2f0 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a99c0 con 0x557516bc4d80 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54872/0 (socket says 172.21.15.94:54872) 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/3446967120 learned_addr learned my addr 172.21.15.94:0/3446967120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fa2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bc4d80 0x557516fa1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/3446967120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bc4d80 msgr2=0x557516fa1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bc4d80 0x557516fa1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/3446967120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 msgr2=0x557516fa2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fa2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.636+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/3446967120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557516fb5b30 con 0x557516be54d0 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.637+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8031720 tx=0x7fa9b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9fc3f87d778fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.637+0000 7fa9a97fa640 1 -- 172.21.15.94:0/3446967120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b805c070 con 0x557516be54d0 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.637+0000 7fa9a97fa640 1 -- 172.21.15.94:0/3446967120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8031920 con 0x557516be54d0 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.637+0000 7fa9a97fa640 1 -- 172.21.15.94:0/3446967120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8050070 con 0x557516be54d0 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- 172.21.15.94:0/3446967120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 msgr2=0x557516fa5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8031720 tx=0x7fa9b8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- 172.21.15.94:0/3446967120 shutdown_connections 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516be54d0 0x557516fa5100 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fa2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/3446967120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516bc4d80 0x557516fa1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- 172.21.15.94:0/3446967120 >> 172.21.15.94:0/3446967120 conn(0x557516bfa300 msgr2=0x557516faeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- 172.21.15.94:0/3446967120 shutdown_connections 2024-04-25T14:03:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- 172.21.15.94:0/3446967120 wait complete. 2024-04-25T14:03:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.638+0000 7fa9d0daf740 1 -- start start 2024-04-25T14:03:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 Processor -- start 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fb8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x5575170289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41aa760 con 0x557516bc4d80 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41aa090 con 0x557516bdd2f0 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41a72d0 con 0x557516be54d0 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9d0daf740 10 client.0 _ll_register_callbacks cb 0x7fa9b80317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54882/0 (socket says 172.21.15.94:54882) 2024-04-25T14:03:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/555002514 learned_addr learned my addr 172.21.15.94:0/555002514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.639+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fb8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x5575170289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/555002514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 msgr2=0x5575170289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x5575170289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 msgr2=0x557516fb8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fb8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 -- 172.21.15.94:0/555002514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557517028f30 con 0x557516bc4d80 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fb8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9c2ffd640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005ed0 tx=0x7fa9b8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2037c71bdcaec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8050070 con 0x557516bc4d80 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9b80035f0 con 0x557516bc4d80 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b8031920 con 0x557516bc4d80 2024-04-25T14:03:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9b805c070 con 0x557516bc4d80 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9d0daf740 10 client.23211 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.640+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55751702a100 con 0x557516bc4d80 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8031920 con 0x557516bc4d80 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b804c7c0 con 0x557516bc4d80 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 10 client.23211 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 10 client.23211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 20 client.23211 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516c061b0 0x55751702c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 4 client.23211 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557516f13ee0 con 0x557516c061b0 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.641+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516c061b0 0x55751702c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.642+0000 7fa9a2ffd640 10 client.23211 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:34.642+0000 7fa9c37fe640 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516c061b0 0x55751702c9b0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1648cee6b2ec5c49 server_cookie=d061aa4d3664077e in_seq=0 out_seq=0 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: pgmap v1004: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:34 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: pgmap v1004: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:34 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: pgmap v1004: 97 pgs: 97 active+clean; 273 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:34 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 176924 18.90 MB/sec execute 377 sec latency 462.443 ms 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/555002514 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557516f13ee0 con 0x557516c061b0 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9a2ffd640 10 client.23211 handle_command_reply: tid=0 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 client.23211 shutdown 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 msgr2=0x557516fb8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005ed0 tx=0x7fa9b8032ec0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 shutdown_connections 2024-04-25T14:03:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557516c061b0 0x55751702c9b0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557516be54d0 0x5575170289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557516bdd2f0 0x557516fb8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 --2- 172.21.15.94:0/555002514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557516bc4d80 0x557516fb8540 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 >> 172.21.15.94:0/555002514 conn(0x557516bfa300 msgr2=0x557516fa5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9a2ffd640 0 client.23211 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 shutdown_connections 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 1 -- 172.21.15.94:0/555002514 wait complete. 2024-04-25T14:03:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.297+0000 7fa9d0daf740 20 client.23211 trim_cache size 0 max 16384 2024-04-25T14:03:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:03:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:03:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:03:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:03:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 msgr2=0x7fa9ac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 0x7fa9ac0b1ff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0dc4f0 tx=0x7fa9bc0d55d0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 msgr2=0x7fa9c41a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009fd0 tx=0x7fa9bc00b260 comp rx=0 tx=0).stop 2024-04-25T14:03:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 shutdown_connections 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9ac0afae0 0x7fa9ac0b1ff0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9c41050e0 0x7fa9c41a1750 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4102290 0x7fa9c419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 --2- 172.21.15.94:0/301654980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9c4101790 0x7fa9c419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 >> 172.21.15.94:0/301654980 conn(0x7fa9c40fd390 msgr2=0x7fa9c4105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 shutdown_connections 2024-04-25T14:03:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:35.298+0000 7fa9c9e20640 1 -- 172.21.15.94:0/301654980 wait complete. 2024-04-25T14:03:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:35 smithi094 ceph-mon[29195]: pgmap v1005: 97 pgs: 97 active+clean; 276 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-04-25T14:03:35.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:35 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:35 smithi138 ceph-mon[32977]: pgmap v1005: 97 pgs: 97 active+clean; 276 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-04-25T14:03:35.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:35.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:35 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:36.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:35 smithi137 ceph-mon[32568]: pgmap v1005: 97 pgs: 97 active+clean; 276 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 526 op/s 2024-04-25T14:03:36.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:36.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:36.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:36.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:35 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:36.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 177275 18.92 MB/sec execute 378 sec latency 624.189 ms 2024-04-25T14:03:37.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:37.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:37.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 177483 18.90 MB/sec execute 379 sec latency 362.671 ms 2024-04-25T14:03:37.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:37.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:37.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:37.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:37.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:37.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:03:38.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:37 smithi137 ceph-mon[32568]: pgmap v1006: 97 pgs: 97 active+clean; 279 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 400 op/s 2024-04-25T14:03:38.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 177902 18.92 MB/sec execute 380 sec latency 82.625 ms 2024-04-25T14:03:38.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:37 smithi094 ceph-mon[29195]: pgmap v1006: 97 pgs: 97 active+clean; 279 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 400 op/s 2024-04-25T14:03:38.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:37 smithi138 ceph-mon[32977]: pgmap v1006: 97 pgs: 97 active+clean; 279 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 400 op/s 2024-04-25T14:03:39.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 178262 18.94 MB/sec execute 381 sec latency 83.433 ms 2024-04-25T14:03:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:39 smithi137 ceph-mon[32568]: pgmap v1007: 97 pgs: 97 active+clean; 275 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-25T14:03:40.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 178548 18.93 MB/sec execute 382 sec latency 88.004 ms 2024-04-25T14:03:40.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:39 smithi094 ceph-mon[29195]: pgmap v1007: 97 pgs: 97 active+clean; 275 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-25T14:03:40.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:40.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:40.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:39 smithi138 ceph-mon[32977]: pgmap v1007: 97 pgs: 97 active+clean; 275 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.1 MiB/s wr, 369 op/s 2024-04-25T14:03:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 -- 172.21.15.94:0/2523392154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec101760 msgr2=0x7ff1ec101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 --2- 172.21.15.94:0/2523392154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec101760 0x7ff1ec101bd0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 -- 172.21.15.94:0/2523392154 shutdown_connections 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 --2- 172.21.15.94:0/2523392154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec102260 0x7ff1ec10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 --2- 172.21.15.94:0/2523392154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec101760 0x7ff1ec101bd0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 --2- 172.21.15.94:0/2523392154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 -- 172.21.15.94:0/2523392154 >> 172.21.15.94:0/2523392154 conn(0x7ff1ec0fd380 msgr2=0x7ff1ec0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 -- 172.21.15.94:0/2523392154 shutdown_connections 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.363+0000 7ff1f1696640 1 -- 172.21.15.94:0/2523392154 wait complete. 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 Processor -- start 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 -- start start 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec102260 0x7ff1ec19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10f640 con 0x7ff1ec1050b0 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10f4c0 con 0x7ff1ec102260 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1f1696640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10f7c0 con 0x7ff1ec101760 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34778/0 (socket says 172.21.15.94:34778) 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54894/0 (socket says 172.21.15.94:54894) 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1eaffd640 1 -- 172.21.15.94:0/602955304 learned_addr learned my addr 172.21.15.94:0/602955304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.364+0000 7ff1ea7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec102260 0x7ff1ec19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/602955304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 msgr2=0x7ff1ec19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/602955304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec102260 msgr2=0x7ff1ec19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec102260 0x7ff1ec19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/602955304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1a1d80 con 0x7ff1ec1050b0 2024-04-25T14:03:40.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009fd0 tx=0x7ff1e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1d3fff640 1 -- 172.21.15.94:0/602955304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400ce20 con 0x7ff1ec1050b0 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1d3fff640 1 -- 172.21.15.94:0/602955304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e4005020 con 0x7ff1ec1050b0 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1a2070 con 0x7ff1ec1050b0 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1d3fff640 1 -- 172.21.15.94:0/602955304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4010690 con 0x7ff1ec1050b0 2024-04-25T14:03:40.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.365+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1a25c0 con 0x7ff1ec1050b0 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.367+0000 7ff1f8624740 1 Processor -- start 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1d3fff640 1 -- 172.21.15.94:0/602955304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff1e4010830 con 0x7ff1ec1050b0 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 -- start start 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b5563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b18f6b0 0x56162b557080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56162b4b6d80 con 0x56162b1b0390 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56162b4708b0 con 0x56162b18f6b0 2024-04-25T14:03:40.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1f8624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56162b4b2d40 con 0x56162b1874d0 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54898/0 (socket says 172.21.15.94:54898) 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1ea7fc640 1 -- 172.21.15.94:0/4209457063 learned_addr learned my addr 172.21.15.94:0/4209457063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b5563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b18f6b0 0x56162b557080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1eaffd640 1 -- 172.21.15.94:0/4209457063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b18f6b0 msgr2=0x56162b557080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b18f6b0 0x56162b557080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1eaffd640 1 -- 172.21.15.94:0/4209457063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 msgr2=0x56162b555e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.368+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1eaffd640 1 -- 172.21.15.94:0/4209457063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162b441a40 con 0x56162b1874d0 2024-04-25T14:03:40.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1d3fff640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 0x7ff1d40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 0x7ff1d40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b5563f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0114c0 tx=0x7ff1dc011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a28876b4d2d0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1d3fff640 1 -- 172.21.15.94:0/602955304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1e40df360 con 0x7ff1ec1050b0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1d17fa640 1 -- 172.21.15.94:0/4209457063 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011250 con 0x56162b1874d0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1d17fa640 1 -- 172.21.15.94:0/4209457063 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc0035f0 con 0x56162b1874d0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.369+0000 7ff1d17fa640 1 -- 172.21.15.94:0/4209457063 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0044e0 con 0x56162b1874d0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- 172.21.15.94:0/4209457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 msgr2=0x56162b5563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b5563f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0114c0 tx=0x7ff1dc011980 comp rx=0 tx=0).stop 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 0x7ff1d40b2000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x56162b556ec0 tx=0x7ff1e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- 172.21.15.94:0/4209457063 shutdown_connections 2024-04-25T14:03:40.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b18f6b0 0x56162b557080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b5563f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 --2- 172.21.15.94:0/4209457063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b555e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- 172.21.15.94:0/4209457063 >> 172.21.15.94:0/4209457063 conn(0x56162b1a44e0 msgr2=0x56162b54d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- 172.21.15.94:0/4209457063 shutdown_connections 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- 172.21.15.94:0/4209457063 wait complete. 2024-04-25T14:03:40.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 -- start start 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.370+0000 7ff1f8624740 1 Processor -- start 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b1874d0 0x56162b55fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b560250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56162b557b90 con 0x56162b1b0390 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56162b43dd10 con 0x56162b1874d0 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56162b557d10 con 0x56162b18f6b0 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 10 client.0 _ll_register_callbacks cb 0x56162b46a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54900/0 (socket says 172.21.15.94:54900) 2024-04-25T14:03:40.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2860130381 learned_addr learned my addr 172.21.15.94:0/2860130381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b560250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b1874d0 0x56162b55fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2860130381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 msgr2=0x56162b560250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b560250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b1874d0 msgr2=0x56162b55fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b1874d0 0x56162b55fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2860130381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162b2927d0 con 0x56162b1b0390 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40a74a0 tx=0x7ff1e40a7f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9a34d32d9debe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40ca0d0 con 0x56162b1b0390 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1e40054d0 con 0x56162b1b0390 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.371+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1dc0035f0 con 0x56162b1b0390 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff1e40b0600 con 0x56162b1b0390 2024-04-25T14:03:40.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 10 client.23223 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56162b1c83f0 con 0x56162b1b0390 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc0035f0 con 0x56162b1b0390 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff1e4010b20 con 0x56162b1b0390 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 10 client.23223 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 10 client.23223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 20 client.23223 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b16ef60 0x56162b56d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 4 client.23223 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162b4c4990 con 0x56162b16ef60 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.372+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b16ef60 0x56162b56d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.373+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b16ef60 0x56162b56d7f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d36941bff92dc5d server_cookie=df4abc3fc9f2a008 in_seq=0 out_seq=0 2024-04-25T14:03:40.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.373+0000 7ff1ceff5640 10 client.23223 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.378+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2860130381 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56162b4c4990 con 0x56162b16ef60 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.378+0000 7ff1ceff5640 10 client.23223 handle_command_reply: tid=0 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.378+0000 7ff1f8624740 1 client.23223 shutdown 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.378+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 msgr2=0x56162b5671b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.378+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40a74a0 tx=0x7ff1e40a7f60 comp rx=0 tx=0).stop 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 shutdown_connections 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b16ef60 0x56162b56d7f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1b0390 0x56162b5671b0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1ceff5640 0 client.23223 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b560250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2860130381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b1874d0 0x56162b55fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 >> 172.21.15.94:0/2860130381 conn(0x56162b1a44e0 msgr2=0x56162b550180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 shutdown_connections 2024-04-25T14:03:40.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 1 -- 172.21.15.94:0/2860130381 wait complete. 2024-04-25T14:03:40.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.379+0000 7ff1f8624740 20 client.23223 trim_cache size 0 max 16384 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 Processor -- start 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 -- start start 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b54cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b54d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1a9370 con 0x56162b1874d0 2024-04-25T14:03:40.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1aaeb0 con 0x56162b16ef60 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1f8624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1a9a40 con 0x56162b18f6b0 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54914/0 (socket says 172.21.15.94:54914) 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 -- 172.21.15.94:0/3267774779 learned_addr learned my addr 172.21.15.94:0/3267774779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b54cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b54d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 -- 172.21.15.94:0/3267774779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 msgr2=0x56162b54d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b54d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 -- 172.21.15.94:0/3267774779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 msgr2=0x56162b54cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b54cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1ea7fc640 1 -- 172.21.15.94:0/3267774779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162b1880f0 con 0x56162b1874d0 2024-04-25T14:03:40.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.398+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b54d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.399+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005ab0 tx=0x7ff1e003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f171d9190d9fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.399+0000 7ff1d0ff9640 1 -- 172.21.15.94:0/3267774779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004240 con 0x56162b1874d0 2024-04-25T14:03:40.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.399+0000 7ff1d0ff9640 1 -- 172.21.15.94:0/3267774779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0050070 con 0x56162b1874d0 2024-04-25T14:03:40.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.399+0000 7ff1d0ff9640 1 -- 172.21.15.94:0/3267774779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00556d0 con 0x56162b1874d0 2024-04-25T14:03:40.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- 172.21.15.94:0/3267774779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 msgr2=0x56162b5670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0005ab0 tx=0x7ff1e003f040 comp rx=0 tx=0).stop 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- 172.21.15.94:0/3267774779 shutdown_connections 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b18f6b0 0x56162b54d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b1874d0 0x56162b5670d0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- 172.21.15.94:0/3267774779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b54cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- 172.21.15.94:0/3267774779 >> 172.21.15.94:0/3267774779 conn(0x56162b1a44e0 msgr2=0x56162b55aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- 172.21.15.94:0/3267774779 shutdown_connections 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- 172.21.15.94:0/3267774779 wait complete. 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 -- start start 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 Processor -- start 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b5670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.400+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b563c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1aa7e0 con 0x56162b18f6b0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1aa110 con 0x56162b16ef60 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1a7350 con 0x56162b1874d0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 10 client.0 _ll_register_callbacks cb 0x7ff1e40a3660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b5670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54926/0 (socket says 172.21.15.94:54926) 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2643238640 learned_addr learned my addr 172.21.15.94:0/2643238640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1ea7fc640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b563c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2643238640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 msgr2=0x56162b563c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b563c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 msgr2=0x56162b5670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b5670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 -- 172.21.15.94:0/2643238640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162b4774b0 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b5670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1eb7fe640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40a7730 tx=0x7ff1e40053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67877b4e414440fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40ca0d0 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1e40b0170 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0050070 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff1e40054d0 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 10 client.23229 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.401+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56162b5d4d00 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e0050070 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff1e40a3110 con 0x56162b18f6b0 2024-04-25T14:03:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 10 client.23229 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 10 client.23229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 20 client.23229 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b1b0390 0x56162b5d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 4 client.23229 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56162b4be800 con 0x56162b1b0390 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b1b0390 0x56162b5d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1ceff5640 10 client.23229 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.402+0000 7ff1eaffd640 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b1b0390 0x56162b5d74e0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5b3a5c57400834 server_cookie=dae36a0b2503fad7 in_seq=0 out_seq=0 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1ceff5640 1 -- 172.21.15.94:0/2643238640 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56162b4be800 con 0x56162b1b0390 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1ceff5640 10 client.23229 handle_command_reply: tid=0 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 client.23229 shutdown 2024-04-25T14:03:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 msgr2=0x56162b564330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40a7730 tx=0x7ff1e40053e0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 shutdown_connections 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56162b1b0390 0x56162b5d74e0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56162b18f6b0 0x56162b564330 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56162b1874d0 0x56162b563c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1ceff5640 0 client.23229 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 --2- 172.21.15.94:0/2643238640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56162b16ef60 0x56162b5670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 >> 172.21.15.94:0/2643238640 conn(0x56162b1a44e0 msgr2=0x56162b54cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 shutdown_connections 2024-04-25T14:03:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 1 -- 172.21.15.94:0/2643238640 wait complete. 2024-04-25T14:03:40.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.404+0000 7ff1f8624740 20 client.23229 trim_cache size 0 max 16384 2024-04-25T14:03:40.412 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:03:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:03:40.412 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:40.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 msgr2=0x7ff1d40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 0x7ff1d40b2000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x56162b556ec0 tx=0x7ff1e0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 msgr2=0x7ff1ec1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009fd0 tx=0x7ff1e400ef10 comp rx=0 tx=0).stop 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 shutdown_connections 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1d40afaf0 0x7ff1d40b2000 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1ec1050b0 0x7ff1ec1a1710 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec102260 0x7ff1ec19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 --2- 172.21.15.94:0/602955304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ec101760 0x7ff1ec19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 >> 172.21.15.94:0/602955304 conn(0x7ff1ec0fd380 msgr2=0x7ff1ec105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 shutdown_connections 2024-04-25T14:03:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.405+0000 7ff1f1696640 1 -- 172.21.15.94:0/602955304 wait complete. 2024-04-25T14:03:40.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:03:40.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:40.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 -- 172.21.15.94:0/1759499426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 msgr2=0x7fb918102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/1759499426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb918102e60 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fb90800aa10 tx=0x7fb90802b720 comp rx=0 tx=0).stop 2024-04-25T14:03:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 -- 172.21.15.94:0/1759499426 shutdown_connections 2024-04-25T14:03:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/1759499426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 0x7fb91810a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/1759499426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 0x7fb9181037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/1759499426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb918102e60 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 -- 172.21.15.94:0/1759499426 >> 172.21.15.94:0/1759499426 conn(0x7fb9180fd330 msgr2=0x7fb9180ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 -- 172.21.15.94:0/1759499426 shutdown_connections 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.571+0000 7fb91d9f3640 1 -- 172.21.15.94:0/1759499426 wait complete. 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.572+0000 7fb91d9f3640 1 Processor -- start 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.572+0000 7fb91d9f3640 1 -- start start 2024-04-25T14:03:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.572+0000 7fb91d9f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 0x7fb9180fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 0x7fb9180fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810d3f0 con 0x7fb918102160 2024-04-25T14:03:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810d270 con 0x7fb9181080a0 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810d570 con 0x7fb9181033a0 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54948/0 (socket says 172.21.15.94:54948) 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 -- 172.21.15.94:0/2423763440 learned_addr learned my addr 172.21.15.94:0/2423763440 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb9167fc640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 0x7fb9180fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb9177fe640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 0x7fb9180fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 -- 172.21.15.94:0/2423763440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 msgr2=0x7fb9180fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 0x7fb9180fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 -- 172.21.15.94:0/2423763440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 msgr2=0x7fb9180fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 0x7fb9180fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 -- 172.21.15.94:0/2423763440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9180fe4d0 con 0x7fb918102160 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb916ffd640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fb908006010 tx=0x7fb908004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90803f070 con 0x7fb918102160 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a43c0 con 0x7fb918102160 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb908003ca0 con 0x7fb918102160 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908036040 con 0x7fb918102160 2024-04-25T14:03:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.573+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a47c0 con 0x7fb918102160 2024-04-25T14:03:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.574+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918069310 con 0x7fb918102160 2024-04-25T14:03:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.576+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb908042070 con 0x7fb918102160 2024-04-25T14:03:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.576+0000 7fb91c9f1640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 0x7fb9000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.577+0000 7fb9167fc640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 0x7fb9000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.577+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb9080ff8f0 con 0x7fb918102160 2024-04-25T14:03:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.577+0000 7fb9167fc640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 0x7fb9000b2000 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb9180feb60 tx=0x7fb90c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.580+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9080c7de0 con 0x7fb918102160 2024-04-25T14:03:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.832+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9181a4ae0 con 0x7fb918102160 2024-04-25T14:03:40.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.834+0000 7fb91c9f1640 1 -- 172.21.15.94:0/2423763440 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7fb9080cc0e0 con 0x7fb918102160 2024-04-25T14:03:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:03:40.834 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:03:40.834 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 msgr2=0x7fb9000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 0x7fb9000b2000 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb9180feb60 tx=0x7fb90c005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 msgr2=0x7fb9180ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fb908006010 tx=0x7fb908004270 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 shutdown_connections 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb9000afaf0 0x7fb9000b2000 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9181080a0 0x7fb9180fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9181033a0 0x7fb9180fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 --2- 172.21.15.94:0/2423763440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb918102160 0x7fb9180ff150 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 >> 172.21.15.94:0/2423763440 conn(0x7fb9180fd330 msgr2=0x7fb918106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 shutdown_connections 2024-04-25T14:03:40.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.837+0000 7fb91d9f3640 1 -- 172.21.15.94:0/2423763440 wait complete. 2024-04-25T14:03:40.851 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- 172.21.15.94:0/2949623682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 msgr2=0x7fa1241090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 --2- 172.21.15.94:0/2949623682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa1241090e0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c02a410 comp rx=0 tx=0).stop 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- 172.21.15.94:0/2949623682 shutdown_connections 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 --2- 172.21.15.94:0/2949623682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124109770 0x7fa12410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 --2- 172.21.15.94:0/2949623682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa1241090e0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 --2- 172.21.15.94:0/2949623682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124104270 0x7fa124106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- 172.21.15.94:0/2949623682 >> 172.21.15.94:0/2949623682 conn(0x7fa1240fdbd0 msgr2=0x7fa124100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- 172.21.15.94:0/2949623682 shutdown_connections 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- 172.21.15.94:0/2949623682 wait complete. 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 Processor -- start 2024-04-25T14:03:40.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.969+0000 7fa129f1c640 1 -- start start 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104270 0x7fa12419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124109770 0x7fa1241a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410f5e0 con 0x7fa124106c60 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410f460 con 0x7fa124109770 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa129f1c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410f760 con 0x7fa124104270 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54966/0 (socket says 172.21.15.94:54966) 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 -- 172.21.15.94:0/649555777 learned_addr learned my addr 172.21.15.94:0/649555777 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa123fff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124109770 0x7fa1241a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa1237fe640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104270 0x7fa12419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 -- 172.21.15.94:0/649555777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104270 msgr2=0x7fa12419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104270 0x7fa12419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 -- 172.21.15.94:0/649555777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124109770 msgr2=0x7fa1241a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124109770 0x7fa1241a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 -- 172.21.15.94:0/649555777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1241a20b0 con 0x7fa124106c60 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa11bfff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006010 tx=0x7fa10c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.970+0000 7fa1217fa640 1 -- 172.21.15.94:0/649555777 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c03f030 con 0x7fa124106c60 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.971+0000 7fa1217fa640 1 -- 172.21.15.94:0/649555777 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c031910 con 0x7fa124106c60 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.971+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1241a2340 con 0x7fa124106c60 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.971+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241a2830 con 0x7fa124106c60 2024-04-25T14:03:40.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.971+0000 7fa1217fa640 1 -- 172.21.15.94:0/649555777 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c049060 con 0x7fa124106c60 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.973+0000 7fa1217fa640 1 -- 172.21.15.94:0/649555777 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa10c042030 con 0x7fa124106c60 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.973+0000 7fa130dcc740 1 Processor -- start 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.973+0000 7fa130dcc740 1 -- start start 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa1217fa640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 0x7fa1100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac213810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac214a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557eac177d80 con 0x557eabe506b0 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557eac131a20 con 0x557eabe484d0 2024-04-25T14:03:40.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa130dcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557eac173d40 con 0x557eabe71390 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa123fff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 0x7fa1100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa1217fa640 1 -- 172.21.15.94:0/649555777 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa10c0ff550 con 0x7fa124106c60 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36498/0 (socket says 172.21.15.94:36498) 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa123fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac214a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 -- 172.21.15.94:0/1037928708 learned_addr learned my addr 172.21.15.94:0/1037928708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa1237fe640 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac213810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 -- 172.21.15.94:0/1037928708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 msgr2=0x557eac213810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac213810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 -- 172.21.15.94:0/1037928708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 msgr2=0x557eac214a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac214a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 -- 172.21.15.94:0/1037928708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eac13cc30 con 0x557eabe484d0 2024-04-25T14:03:40.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa123fff640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 0x7fa1100b2000 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005fd0 tx=0x7fa11c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.974+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c4cb0 tx=0x7fa10c0c7810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acff689ab45e0acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa119ffb640 1 -- 172.21.15.94:0/1037928708 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c035030 con 0x557eabe484d0 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa119ffb640 1 -- 172.21.15.94:0/1037928708 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c0d0130 con 0x557eabe484d0 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa119ffb640 1 -- 172.21.15.94:0/1037928708 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0c7a40 con 0x557eabe484d0 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 -- 172.21.15.94:0/1037928708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 msgr2=0x557eac213d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c4cb0 tx=0x7fa10c0c7810 comp rx=0 tx=0).stop 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 -- 172.21.15.94:0/1037928708 shutdown_connections 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac214a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac213d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1037928708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac213810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.975+0000 7fa130dcc740 1 -- 172.21.15.94:0/1037928708 >> 172.21.15.94:0/1037928708 conn(0x557eabe654e0 msgr2=0x557eac20f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- 172.21.15.94:0/1037928708 shutdown_connections 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- 172.21.15.94:0/1037928708 wait complete. 2024-04-25T14:03:40.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- start start 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 Processor -- start 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 0x557eac21d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac224730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557eac2152f0 con 0x557eabe484d0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557eac0fedc0 con 0x557eabe506b0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557eac215470 con 0x557eabe71390 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa130dcc740 10 client.0 _ll_register_callbacks cb 0x557eac11a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54984/0 (socket says 172.21.15.94:54984) 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.976+0000 7fa1237fe640 1 -- 172.21.15.94:0/144381366 learned_addr learned my addr 172.21.15.94:0/144381366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa11bfff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 0x557eac21d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa123fff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac224730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 -- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 msgr2=0x557eac224730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac224730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 -- 172.21.15.94:0/144381366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 msgr2=0x557eac21d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 0x557eac21d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 -- 172.21.15.94:0/144381366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eabe893f0 con 0x557eabe484d0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa123fff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac224730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa11bfff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 0x557eac21d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa1237fe640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fa1140126e0 tx=0x7fa114012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a27ff1c071c752b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114012190 con 0x557eabe484d0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa11400f920 con 0x557eabe484d0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557eac185990 con 0x557eabe484d0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa11400fb20 con 0x557eabe484d0 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa130dcc740 10 client.23247 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:40.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.977+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557eabf537d0 con 0x557eabe484d0 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557eac185990 con 0x557eabe484d0 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa114010c90 con 0x557eabe484d0 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 10 client.23247 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 10 client.23247 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 20 client.23247 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe2ff60 0x557eac22b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 4 client.23247 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eac17f800 con 0x557eabe2ff60 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa11bfff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe2ff60 0x557eac22b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa102ffd640 10 client.23247 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.978+0000 7fa11bfff640 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe2ff60 0x557eac22b0b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d638d75a82a4fbe3 server_cookie=567c33f38e7dfe32 in_seq=0 out_seq=0 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa102ffd640 1 -- 172.21.15.94:0/144381366 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557eac17f800 con 0x557eabe2ff60 2024-04-25T14:03:40.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa102ffd640 10 client.23247 handle_command_reply: tid=0 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 client.23247 shutdown 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 msgr2=0x557eac21d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fa1140126e0 tx=0x7fa114012ba0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 shutdown_connections 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe2ff60 0x557eac22b0b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac224730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe506b0 0x557eac21d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa102ffd640 0 client.23247 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:40.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 --2- 172.21.15.94:0/144381366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe484d0 0x557eac21d290 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 >> 172.21.15.94:0/144381366 conn(0x557eabe654e0 msgr2=0x557eac20e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 shutdown_connections 2024-04-25T14:03:40.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 1 -- 172.21.15.94:0/144381366 wait complete. 2024-04-25T14:03:40.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.979+0000 7fa130dcc740 20 client.23247 trim_cache size 0 max 16384 2024-04-25T14:03:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.993+0000 7fa130dcc740 1 Processor -- start 2024-04-25T14:03:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.993+0000 7fa130dcc740 1 -- start start 2024-04-25T14:03:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac22f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ab520 con 0x557eabe506b0 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ad060 con 0x557eabe484d0 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa130dcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241abbf0 con 0x557eabe71390 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa1237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36510/0 (socket says 172.21.15.94:36510) 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54998/0 (socket says 172.21.15.94:54998) 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa1237fe640 1 -- 172.21.15.94:0/1664173388 learned_addr learned my addr 172.21.15.94:0/1664173388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa123fff640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac22f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 -- 172.21.15.94:0/1664173388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 msgr2=0x557eac22f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac22f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 -- 172.21.15.94:0/1664173388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 msgr2=0x557eac21d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 -- 172.21.15.94:0/1664173388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eac218ba0 con 0x557eabe506b0 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa123fff640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac22f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa1237fe640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa11bfff640 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x557eac213530 tx=0x7fa10c0ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad30f5fc1e0eeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.994+0000 7fa1197fa640 1 -- 172.21.15.94:0/1664173388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c035040 con 0x557eabe506b0 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa1197fa640 1 -- 172.21.15.94:0/1664173388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c0d07f0 con 0x557eabe506b0 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa1197fa640 1 -- 172.21.15.94:0/1664173388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0c3cd0 con 0x557eabe506b0 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 -- 172.21.15.94:0/1664173388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 msgr2=0x557eac224650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x557eac213530 tx=0x7fa10c0ea040 comp rx=0 tx=0).stop 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 -- 172.21.15.94:0/1664173388 shutdown_connections 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac22f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac224650 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 --2- 172.21.15.94:0/1664173388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac21d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 -- 172.21.15.94:0/1664173388 >> 172.21.15.94:0/1664173388 conn(0x557eabe654e0 msgr2=0x557eac21ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 -- 172.21.15.94:0/1664173388 shutdown_connections 2024-04-25T14:03:40.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.995+0000 7fa130dcc740 1 -- 172.21.15.94:0/1664173388 wait complete. 2024-04-25T14:03:40.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 -- start start 2024-04-25T14:03:40.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 Processor -- start 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac2216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac291410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ac990 con 0x557eabe506b0 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241ac2c0 con 0x557eabe484d0 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241a9500 con 0x557eabe71390 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa130dcc740 10 client.0 _ll_register_callbacks cb 0x7fa114009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa11bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55012/0 (socket says 172.21.15.94:55012) 2024-04-25T14:03:41.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa11bfff640 1 -- 172.21.15.94:0/3113133030 learned_addr learned my addr 172.21.15.94:0/3113133030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa1237fe640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac2216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.996+0000 7fa123fff640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac291410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 -- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 msgr2=0x557eac291410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac291410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 -- 172.21.15.94:0/3113133030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 msgr2=0x557eac2216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac2216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 -- 172.21.15.94:0/3113133030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eac291950 con 0x557eabe506b0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa11bfff640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c3a70 tx=0x7fa10c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bf44517d43446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c035040 con 0x557eabe506b0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa10c0c3cd0 con 0x557eabe506b0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa10c0d07f0 con 0x557eabe506b0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa10c0044d0 con 0x557eabe506b0 2024-04-25T14:03:41.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 10 client.23253 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557eac292a50 con 0x557eabe506b0 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c0d07f0 con 0x557eabe506b0 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa10c0c7d30 con 0x557eabe506b0 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 10 client.23253 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 10 client.23253 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 20 client.23253 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe053a0 0x557eac295020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 4 client.23253 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.997+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557eac0fa610 con 0x557eabe053a0 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa1237fe640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe053a0 0x557eac295020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa102ffd640 10 client.23253 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:41.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa1237fe640 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe053a0 0x557eac295020 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab066f75bae81515 server_cookie=b808be682bf94dac in_seq=0 out_seq=0 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa102ffd640 1 -- 172.21.15.94:0/3113133030 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557eac0fa610 con 0x557eabe053a0 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa102ffd640 10 client.23253 handle_command_reply: tid=0 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa130dcc740 1 client.23253 shutdown 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 msgr2=0x557eac221c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.998+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c3a70 tx=0x7fa10c005c90 comp rx=0 tx=0).stop 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 shutdown_connections 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x557eabe053a0 0x557eac295020 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557eabe71390 0x557eac291410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557eabe506b0 0x557eac221c10 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa102ffd640 0 client.23253 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 --2- 172.21.15.94:0/3113133030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eabe484d0 0x557eac2216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 >> 172.21.15.94:0/3113133030 conn(0x557eabe654e0 msgr2=0x557eac21cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 shutdown_connections 2024-04-25T14:03:41.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 1 -- 172.21.15.94:0/3113133030 wait complete. 2024-04-25T14:03:41.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa130dcc740 20 client.23253 trim_cache size 0 max 16384 2024-04-25T14:03:41.008 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 msgr2=0x7fa1100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 0x7fa1100b2000 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005fd0 tx=0x7fa11c009f10 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 msgr2=0x7fa12419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006010 tx=0x7fa10c0042f0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 shutdown_connections 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1100afaf0 0x7fa1100b2000 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa124109770 0x7fa1241a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa124106c60 0x7fa12419aab0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 --2- 172.21.15.94:0/649555777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa124104270 0x7fa12419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 >> 172.21.15.94:0/649555777 conn(0x7fa1240fdbd0 msgr2=0x7fa124104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 shutdown_connections 2024-04-25T14:03:41.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:40.999+0000 7fa129f1c640 1 -- 172.21.15.94:0/649555777 wait complete. 2024-04-25T14:03:41.014 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:03:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 -- 172.21.15.94:0/3418008093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c101740 msgr2=0x7fe68c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 --2- 172.21.15.94:0/3418008093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c101740 0x7fe68c101bb0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fe680009ca0 tx=0x7fe68002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 -- 172.21.15.94:0/3418008093 shutdown_connections 2024-04-25T14:03:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 --2- 172.21.15.94:0/3418008093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c102240 0x7fe68c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 --2- 172.21.15.94:0/3418008093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c101740 0x7fe68c101bb0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 --2- 172.21.15.94:0/3418008093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 0x7fe68c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 -- 172.21.15.94:0/3418008093 >> 172.21.15.94:0/3418008093 conn(0x7fe68c0fd340 msgr2=0x7fe68c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 -- 172.21.15.94:0/3418008093 shutdown_connections 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.132+0000 7fe690e1e640 1 -- 172.21.15.94:0/3418008093 wait complete. 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 Processor -- start 2024-04-25T14:03:41.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 -- start start 2024-04-25T14:03:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c101740 0x7fe68c104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 0x7fe68c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10f620 con 0x7fe68c102240 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10f4a0 con 0x7fe68c105090 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe690e1e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10f7a0 con 0x7fe68c101740 2024-04-25T14:03:41.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55032/0 (socket says 172.21.15.94:55032) 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 -- 172.21.15.94:0/2286721234 learned_addr learned my addr 172.21.15.94:0/2286721234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 0x7fe68c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe68a59c640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c101740 0x7fe68c104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 -- 172.21.15.94:0/2286721234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c101740 msgr2=0x7fe68c104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c101740 0x7fe68c104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 -- 172.21.15.94:0/2286721234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 msgr2=0x7fe68c102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 0x7fe68c102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 -- 172.21.15.94:0/2286721234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c103550 con 0x7fe68c102240 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.133+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fe68002ade0 tx=0x7fe6800317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.134+0000 7fe67b7fe640 1 -- 172.21.15.94:0/2286721234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680003c30 con 0x7fe68c102240 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.134+0000 7fe67b7fe640 1 -- 172.21.15.94:0/2286721234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe680003dd0 con 0x7fe68c102240 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.134+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c1a42f0 con 0x7fe68c102240 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.134+0000 7fe67b7fe640 1 -- 172.21.15.94:0/2286721234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68002fb60 con 0x7fe68c102240 2024-04-25T14:03:41.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.134+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a4750 con 0x7fe68c102240 2024-04-25T14:03:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.136+0000 7fe697da8740 1 Processor -- start 2024-04-25T14:03:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.136+0000 7fe697da8740 1 -- start start 2024-04-25T14:03:41.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.136+0000 7fe67b7fe640 1 -- 172.21.15.94:0/2286721234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe68002fd00 con 0x7fe68c102240 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 0x5596b99c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95fd6b0 0x5596b99c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5596b9924d80 con 0x5596b961e390 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596b98dea20 con 0x5596b95f54d0 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe697da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596b9920d40 con 0x5596b95fd6b0 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 0x5596b99c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 0x5596b99c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55038/0 (socket says 172.21.15.94:55038) 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36542/0 (socket says 172.21.15.94:36542) 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68ad9d640 1 -- 172.21.15.94:0/1477058132 learned_addr learned my addr 172.21.15.94:0/1477058132 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe689d9b640 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95fd6b0 0x5596b99c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 -- 172.21.15.94:0/1477058132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95fd6b0 msgr2=0x5596b99c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95fd6b0 0x5596b99c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 -- 172.21.15.94:0/1477058132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 msgr2=0x5596b99c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 0x5596b99c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 -- 172.21.15.94:0/1477058132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b98e9c30 con 0x5596b95f54d0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe68a59c640 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe68c069dc0 tx=0x7fe6840113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc1c806f76121e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe67b7fe640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 0x7fe6740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.137+0000 7fe678ff9640 1 -- 172.21.15.94:0/1477058132 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684011de0 con 0x5596b95f54d0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.138+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 0x7fe6740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.138+0000 7fe678ff9640 1 -- 172.21.15.94:0/1477058132 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684012900 con 0x5596b95f54d0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.138+0000 7fe678ff9640 1 -- 172.21.15.94:0/1477058132 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400f990 con 0x5596b95f54d0 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.138+0000 7fe67b7fe640 1 -- 172.21.15.94:0/2286721234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe680100270 con 0x7fe68c102240 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- 172.21.15.94:0/1477058132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 msgr2=0x5596b99c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe68c069dc0 tx=0x7fe6840113e0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 0x7fe6740b2000 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5596b99c1860 tx=0x7fe67c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- 172.21.15.94:0/1477058132 shutdown_connections 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95fd6b0 0x5596b99c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99c0d90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- 172.21.15.94:0/1477058132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b961e390 0x5596b99c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- 172.21.15.94:0/1477058132 >> 172.21.15.94:0/1477058132 conn(0x5596b96124e0 msgr2=0x5596b99bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- 172.21.15.94:0/1477058132 shutdown_connections 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- 172.21.15.94:0/1477058132 wait complete. 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- start start 2024-04-25T14:03:41.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 Processor -- start 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5596b99c2530 con 0x5596b95fd6b0 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596b98abdc0 con 0x5596b961e390 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596b99c26b0 con 0x5596b95f54d0 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.139+0000 7fe697da8740 10 client.0 _ll_register_callbacks cb 0x5596b98c7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55050/0 (socket says 172.21.15.94:55050) 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 -- 172.21.15.94:0/2999120021 learned_addr learned my addr 172.21.15.94:0/2999120021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 -- 172.21.15.94:0/2999120021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 msgr2=0x5596b99ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 -- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 msgr2=0x5596b99d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 -- 172.21.15.94:0/2999120021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b96363f0 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fe6800c4730 tx=0x7fe680103940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a319505c997344ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800c8480 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe680004610 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684012900 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6800c7e20 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe697da8740 10 client.23265 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.140+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596b97007d0 con 0x5596b95fd6b0 2024-04-25T14:03:41.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684012900 con 0x5596b95fd6b0 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe6800c8910 con 0x5596b95fd6b0 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 10 client.23265 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 10 client.23265 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 20 client.23265 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95dcf60 0x5596b99d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 4 client.23265 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596b9932990 con 0x5596b95dcf60 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.141+0000 7fe68a59c640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95dcf60 0x5596b99d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe671ffb640 10 client.23265 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe68a59c640 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95dcf60 0x5596b99d8250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd74acd7b82ff216 server_cookie=b2184650e7028a49 in_seq=0 out_seq=0 2024-04-25T14:03:41.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe671ffb640 1 -- 172.21.15.94:0/2999120021 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596b9932990 con 0x5596b95dcf60 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe671ffb640 10 client.23265 handle_command_reply: tid=0 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 client.23265 shutdown 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 msgr2=0x5596b99caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fe6800c4730 tx=0x7fe680103940 comp rx=0 tx=0).stop 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 shutdown_connections 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95dcf60 0x5596b99d8250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99caa50 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe671ffb640 0 client.23265 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 --2- 172.21.15.94:0/2999120021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.142+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 >> 172.21.15.94:0/2999120021 conn(0x5596b96124e0 msgr2=0x5596b99bef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.143+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 shutdown_connections 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.143+0000 7fe697da8740 1 -- 172.21.15.94:0/2999120021 wait complete. 2024-04-25T14:03:41.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.143+0000 7fe697da8740 20 client.23265 trim_cache size 0 max 16384 2024-04-25T14:03:41.155 INFO:tasks.workunit.client.0.smithi094.stdout: 10 178849 18.92 MB/sec execute 383 sec latency 66.964 ms 2024-04-25T14:03:41.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.157+0000 7fe697da8740 1 Processor -- start 2024-04-25T14:03:41.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.157+0000 7fe697da8740 1 -- start start 2024-04-25T14:03:41.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.157+0000 7fe697da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.157+0000 7fe697da8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe697da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe697da8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ad680 con 0x5596b95fd6b0 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe697da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1af1c0 con 0x5596b961e390 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe697da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1add50 con 0x5596b95f54d0 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:34934/0 (socket says 172.21.15.94:34934) 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe68a59c640 1 -- 172.21.15.94:0/2996292295 learned_addr learned my addr 172.21.15.94:0/2996292295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 -- 172.21.15.94:0/2996292295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 msgr2=0x5596b99c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 -- 172.21.15.94:0/2996292295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 msgr2=0x5596b99dc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99dc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 -- 172.21.15.94:0/2996292295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b99c5c20 con 0x5596b95fd6b0 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe689d9b640 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d18d0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fe680103af0 tx=0x7fe6800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e323dee503742a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe673fff640 1 -- 172.21.15.94:0/2996292295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800c8000 con 0x5596b95fd6b0 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe673fff640 1 -- 172.21.15.94:0/2996292295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe680036070 con 0x5596b95fd6b0 2024-04-25T14:03:41.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.158+0000 7fe673fff640 1 -- 172.21.15.94:0/2996292295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800d1510 con 0x5596b95fd6b0 2024-04-25T14:03:41.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 -- 172.21.15.94:0/2996292295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 msgr2=0x5596b99d18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d18d0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fe680103af0 tx=0x7fe6800095c0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 -- 172.21.15.94:0/2996292295 shutdown_connections 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b961e390 0x5596b99dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d18d0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 --2- 172.21.15.94:0/2996292295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b95f54d0 0x5596b99c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 -- 172.21.15.94:0/2996292295 >> 172.21.15.94:0/2996292295 conn(0x5596b96124e0 msgr2=0x5596b99c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 -- 172.21.15.94:0/2996292295 shutdown_connections 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.159+0000 7fe697da8740 1 -- 172.21.15.94:0/2996292295 wait complete. 2024-04-25T14:03:41.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 -- start start 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 Processor -- start 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b961e390 0x5596b9a3e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1aeaf0 con 0x5596b95fd6b0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ae420 con 0x5596b95f54d0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1ab660 con 0x5596b961e390 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe697da8740 10 client.0 _ll_register_callbacks cb 0x7fe680103a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe689d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55060/0 (socket says 172.21.15.94:55060) 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe68a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe689d9b640 1 -- 172.21.15.94:0/950819651 learned_addr learned my addr 172.21.15.94:0/950819651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.160+0000 7fe68ad9d640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b961e390 0x5596b9a3e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 -- 172.21.15.94:0/950819651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b961e390 msgr2=0x5596b9a3e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b961e390 0x5596b9a3e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 -- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 msgr2=0x5596b99d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 -- 172.21.15.94:0/950819651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b9a3ee50 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe68a59c640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe689d9b640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fe6800c5920 tx=0x7fe680103d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47502b64a3996ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800e1070 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe6800d1080 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe680036070 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6800c8000 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 10 client.23271 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596b9a3ee50 con 0x5596b95fd6b0 2024-04-25T14:03:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe680036070 con 0x5596b95fd6b0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe680004b40 con 0x5596b95fd6b0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 10 client.23271 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 10 client.23271 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.161+0000 7fe697da8740 20 client.23271 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95b23a0 0x5596b9a426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 4 client.23271 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596b992c800 con 0x5596b95b23a0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe68a59c640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95b23a0 0x5596b9a426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe671ffb640 10 client.23271 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe68a59c640 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95b23a0 0x5596b9a426b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7bf2413ecb9738 server_cookie=b3297dfb8578e22a in_seq=0 out_seq=0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe671ffb640 1 -- 172.21.15.94:0/950819651 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596b992c800 con 0x5596b95b23a0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe671ffb640 10 client.23271 handle_command_reply: tid=0 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 1 client.23271 shutdown 2024-04-25T14:03:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 msgr2=0x5596b99d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.162+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fe6800c5920 tx=0x7fe680103d50 comp rx=0 tx=0).stop 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 shutdown_connections 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5596b95b23a0 0x5596b9a426b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596b961e390 0x5596b9a3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe671ffb640 0 client.23271 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596b95fd6b0 0x5596b99d1210 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 --2- 172.21.15.94:0/950819651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596b95f54d0 0x5596b99d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 >> 172.21.15.94:0/950819651 conn(0x5596b96124e0 msgr2=0x5596b99bee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 shutdown_connections 2024-04-25T14:03:41.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 1 -- 172.21.15.94:0/950819651 wait complete. 2024-04-25T14:03:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe697da8740 20 client.23271 trim_cache size 0 max 16384 2024-04-25T14:03:41.175 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:41.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 msgr2=0x7fe6740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 0x7fe6740b2000 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5596b99c1860 tx=0x7fe67c005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:41.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 msgr2=0x7fe68c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fe68002ade0 tx=0x7fe6800317c0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 shutdown_connections 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe6740afaf0 0x7fe6740b2000 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe68c105090 0x7fe68c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe68c102240 0x7fe68c1028e0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 --2- 172.21.15.94:0/2286721234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe68c101740 0x7fe68c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 >> 172.21.15.94:0/2286721234 conn(0x7fe68c0fd340 msgr2=0x7fe68c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 shutdown_connections 2024-04-25T14:03:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.163+0000 7fe690e1e640 1 -- 172.21.15.94:0/2286721234 wait complete. 2024-04-25T14:03:41.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 -- 172.21.15.94:0/1008095107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 msgr2=0x7f91981036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 --2- 172.21.15.94:0/1008095107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f91981036e0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f9188009900 tx=0x7f918802a410 comp rx=0 tx=0).stop 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 -- 172.21.15.94:0/1008095107 shutdown_connections 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 --2- 172.21.15.94:0/1008095107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f919810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 --2- 172.21.15.94:0/1008095107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 0x7f9198104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 --2- 172.21.15.94:0/1008095107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f91981036e0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 -- 172.21.15.94:0/1008095107 >> 172.21.15.94:0/1008095107 conn(0x7f91980fdbd0 msgr2=0x7f9198100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 -- 172.21.15.94:0/1008095107 shutdown_connections 2024-04-25T14:03:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 -- 172.21.15.94:0/1008095107 wait complete. 2024-04-25T14:03:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.294+0000 7f919e172640 1 Processor -- start 2024-04-25T14:03:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 -- start start 2024-04-25T14:03:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 0x7f919819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f91981a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f919810e540 con 0x7f91981029e0 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f919810e3c0 con 0x7f9198108920 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f919e172640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f919810e6c0 con 0x7f9198103c20 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55074/0 (socket says 172.21.15.94:55074) 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 -- 172.21.15.94:0/1602825561 learned_addr learned my addr 172.21.15.94:0/1602825561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f9197fff640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f91981a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f9196ffd640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 0x7f919819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 -- 172.21.15.94:0/1602825561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 msgr2=0x7f919819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 0x7f919819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 -- 172.21.15.94:0/1602825561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 msgr2=0x7f91981a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f91981a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 -- 172.21.15.94:0/1602825561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91981a2120 con 0x7f91981029e0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f9197fff640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f91981a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.295+0000 7f91977fe640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f9188006010 tx=0x7f9188004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.296+0000 7f9194ff9640 1 -- 172.21.15.94:0/1602825561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918803f030 con 0x7f91981029e0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.296+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91981a23b0 con 0x7f91981029e0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.296+0000 7f9194ff9640 1 -- 172.21.15.94:0/1602825561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188003aa0 con 0x7f91981029e0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.296+0000 7f9194ff9640 1 -- 172.21.15.94:0/1602825561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035070 con 0x7f91981029e0 2024-04-25T14:03:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.296+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981a2970 con 0x7f91981029e0 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f9194ff9640 1 -- 172.21.15.94:0/1602825561 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9188042030 con 0x7f91981029e0 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 Processor -- start 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 -- start start 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af34d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 0x55c6af34ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c6af2b1d80 con 0x55c6aef824d0 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c6af26ba20 con 0x55c6aefab390 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91a501e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6af2add40 con 0x55c6aef8a6b0 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55078/0 (socket says 172.21.15.94:55078) 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f91977fe640 1 -- 172.21.15.94:0/3866017585 learned_addr learned my addr 172.21.15.94:0/3866017585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f9197fff640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af34d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.299+0000 7f9196ffd640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 0x55c6af34ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9194ff9640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 0x7f91800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 -- 172.21.15.94:0/3866017585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 msgr2=0x55c6af34ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 0x55c6af34ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 -- 172.21.15.94:0/3866017585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 msgr2=0x55c6af34d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af34d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 -- 172.21.15.94:0/3866017585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6af276c30 con 0x55c6aef824d0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9196ffd640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 0x55c6af34ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9197fff640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af34d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9197fff640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 0x7f91800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9194ff9640 1 -- 172.21.15.94:0/1602825561 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f91880ff2d0 con 0x7f91981029e0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f91977fe640 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f9188042ea0 tx=0x7f918804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1955058dd46c65b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f917dffb640 1 -- 172.21.15.94:0/3866017585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880c7060 con 0x55c6aef824d0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f9197fff640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 0x7f91800b2000 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55c6af34e860 tx=0x7f9190004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f917dffb640 1 -- 172.21.15.94:0/3866017585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918803b460 con 0x55c6aef824d0 2024-04-25T14:03:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.300+0000 7f917dffb640 1 -- 172.21.15.94:0/3866017585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188042ed0 con 0x55c6aef824d0 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 -- 172.21.15.94:0/3866017585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 msgr2=0x55c6af34dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f9188042ea0 tx=0x7f918804b000 comp rx=0 tx=0).stop 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 -- 172.21.15.94:0/3866017585 shutdown_connections 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef8a6b0 0x55c6af34ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef824d0 0x55c6af34dd90 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 --2- 172.21.15.94:0/3866017585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af34d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 -- 172.21.15.94:0/3866017585 >> 172.21.15.94:0/3866017585 conn(0x55c6aef9f4e0 msgr2=0x55c6af349020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 -- 172.21.15.94:0/3866017585 shutdown_connections 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.301+0000 7f91a501e740 1 -- 172.21.15.94:0/3866017585 wait complete. 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 -- start start 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 Processor -- start 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af357210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af35e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c6af34f2f0 con 0x55c6aef8a6b0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c6af238dc0 con 0x55c6aefab390 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6af34f470 con 0x55c6aef824d0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91a501e740 10 client.0 _ll_register_callbacks cb 0x55c6af254600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55086/0 (socket says 172.21.15.94:55086) 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f9196ffd640 1 -- 172.21.15.94:0/1795090394 learned_addr learned my addr 172.21.15.94:0/1795090394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f91977fe640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af357210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.302+0000 7f9197fff640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af35e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 -- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 msgr2=0x55c6af357210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af357210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 -- 172.21.15.94:0/1795090394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 msgr2=0x55c6af35e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af35e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 -- 172.21.15.94:0/1795090394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6aefc33f0 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9196ffd640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f918c011570 tx=0x7f918c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f251ff1e0146324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c013670 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f918c013810 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6af2bf990 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f918c013a10 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f91a501e740 10 client.23283 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c6af08d7d0 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.303+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c6af2bf990 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f918c027450 con 0x55c6aef8a6b0 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 10 client.23283 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 10 client.23283 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 20 client.23283 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef69f60 0x55c6af364f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 4 client.23283 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6af2b9800 con 0x55c6aef69f60 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91977fe640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef69f60 0x55c6af364f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f9176ffd640 10 client.23283 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.304+0000 7f91977fe640 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef69f60 0x55c6af364f30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41814ecd9ab539b1 server_cookie=74ff5143f86509e9 in_seq=0 out_seq=0 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f9176ffd640 1 -- 172.21.15.94:0/1795090394 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6af2b9800 con 0x55c6aef69f60 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f9176ffd640 10 client.23283 handle_command_reply: tid=0 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 client.23283 shutdown 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 msgr2=0x55c6af357750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f918c011570 tx=0x7f918c009f90 comp rx=0 tx=0).stop 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 shutdown_connections 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef69f60 0x55c6af364f30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aefab390 0x55c6af35e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aef8a6b0 0x55c6af357750 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 --2- 172.21.15.94:0/1795090394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af357210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f9176ffd640 0 client.23283 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 >> 172.21.15.94:0/1795090394 conn(0x55c6aef9f4e0 msgr2=0x55c6af34bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 shutdown_connections 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 1 -- 172.21.15.94:0/1795090394 wait complete. 2024-04-25T14:03:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.305+0000 7f91a501e740 20 client.23283 trim_cache size 0 max 16384 2024-04-25T14:03:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 Processor -- start 2024-04-25T14:03:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 -- start start 2024-04-25T14:03:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af3696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af369c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91981ab5f0 con 0x55c6aefab390 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91981ad130 con 0x55c6aef8a6b0 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91a501e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91981abcc0 con 0x55c6aef824d0 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f9197fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f9196ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af369c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f9197fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55096/0 (socket says 172.21.15.94:55096) 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f9197fff640 1 -- 172.21.15.94:0/511835310 learned_addr learned my addr 172.21.15.94:0/511835310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.319+0000 7f91977fe640 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af3696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 -- 172.21.15.94:0/511835310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 msgr2=0x55c6af3696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af3696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 -- 172.21.15.94:0/511835310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 msgr2=0x55c6af369c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af369c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 -- 172.21.15.94:0/511835310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6af352b20 con 0x55c6aefab390 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f9197fff640 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f9190011a60 tx=0x7f919001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83efada34924ecfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f917d7fa640 1 -- 172.21.15.94:0/511835310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919001d9e0 con 0x55c6aefab390 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f917d7fa640 1 -- 172.21.15.94:0/511835310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919002e070 con 0x55c6aefab390 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.320+0000 7f917d7fa640 1 -- 172.21.15.94:0/511835310 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000f3c0 con 0x55c6aefab390 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- 172.21.15.94:0/511835310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 msgr2=0x55c6af36a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f9190011a60 tx=0x7f919001a000 comp rx=0 tx=0).stop 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- 172.21.15.94:0/511835310 shutdown_connections 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af36a160 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af369c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 --2- 172.21.15.94:0/511835310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af3696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- 172.21.15.94:0/511835310 >> 172.21.15.94:0/511835310 conn(0x55c6aef9f4e0 msgr2=0x55c6af355830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- 172.21.15.94:0/511835310 shutdown_connections 2024-04-25T14:03:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- 172.21.15.94:0/511835310 wait complete. 2024-04-25T14:03:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.321+0000 7f91a501e740 1 -- start start 2024-04-25T14:03:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 Processor -- start 2024-04-25T14:03:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af35dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af3c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91981aca60 con 0x55c6aefab390 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91981ac390 con 0x55c6aef8a6b0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91981a95d0 con 0x55c6aef824d0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91a501e740 10 client.0 _ll_register_callbacks cb 0x7f918c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55110/0 (socket says 172.21.15.94:55110) 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 -- 172.21.15.94:0/1122912356 learned_addr learned my addr 172.21.15.94:0/1122912356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9196ffd640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af3c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91977fe640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af35dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 -- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 msgr2=0x55c6af35dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af35dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 -- 172.21.15.94:0/1122912356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 msgr2=0x55c6af3c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af3c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 -- 172.21.15.94:0/1122912356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6af3cc070 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f91977fe640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af35dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9197fff640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f9190009780 tx=0x7f91900097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9f1cc645ce53c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.322+0000 7f9176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190029800 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f91900323f0 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919002e070 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f91900325f0 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 10 client.23289 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c6af3cd320 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919002e070 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f919003a930 con 0x55c6aefab390 2024-04-25T14:03:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 10 client.23289 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 10 client.23289 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 20 client.23289 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef3f3a0 0x55c6af3cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 4 client.23289 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6af234610 con 0x55c6aef3f3a0 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.323+0000 7f91977fe640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef3f3a0 0x55c6af3cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f9176ffd640 10 client.23289 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91977fe640 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef3f3a0 0x55c6af3cfd70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2357105ec48bbe5d server_cookie=98d422b6bb181635 in_seq=0 out_seq=0 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f9176ffd640 1 -- 172.21.15.94:0/1122912356 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6af234610 con 0x55c6aef3f3a0 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f9176ffd640 10 client.23289 handle_command_reply: tid=0 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 client.23289 shutdown 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 msgr2=0x55c6af3cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f9190009780 tx=0x7f91900097b0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 shutdown_connections 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c6aef3f3a0 0x55c6af3cfd70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c6aefab390 0x55c6af3cb8e0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6aef8a6b0 0x55c6af3c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f9176ffd640 0 client.23289 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 --2- 172.21.15.94:0/1122912356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6aef824d0 0x55c6af35dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 >> 172.21.15.94:0/1122912356 conn(0x55c6aef9f4e0 msgr2=0x55c6af34be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 shutdown_connections 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 1 -- 172.21.15.94:0/1122912356 wait complete. 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.324+0000 7f91a501e740 20 client.23289 trim_cache size 0 max 16384 2024-04-25T14:03:41.332 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:41.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 msgr2=0x7f91800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 0x7f91800b2000 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55c6af34e860 tx=0x7f9190004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 msgr2=0x7f919819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f9188006010 tx=0x7f9188004270 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 shutdown_connections 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f91800afaf0 0x7f91800b2000 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198108920 0x7f91981a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9198103c20 0x7f919819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 --2- 172.21.15.94:0/1602825561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91981029e0 0x7f919819a5e0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 >> 172.21.15.94:0/1602825561 conn(0x7f91980fdbd0 msgr2=0x7f9198109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 shutdown_connections 2024-04-25T14:03:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.325+0000 7f919e172640 1 -- 172.21.15.94:0/1602825561 wait complete. 2024-04-25T14:03:41.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:03:41.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.484+0000 7f577b7c7640 1 -- 172.21.15.94:0/2043064607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774103ae0 msgr2=0x7f57741047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.484+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2043064607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774103ae0 0x7f57741047e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f576c009be0 tx=0x7f576c02eb20 comp rx=0 tx=0).stop 2024-04-25T14:03:41.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.487+0000 7f577b7c7640 1 -- 172.21.15.94:0/2043064607 shutdown_connections 2024-04-25T14:03:41.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.487+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2043064607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774109a20 0x7f577410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.487+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2043064607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774104d20 0x7f5774105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.487+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2043064607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774103ae0 0x7f57741047e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.487+0000 7f577b7c7640 1 -- 172.21.15.94:0/2043064607 >> 172.21.15.94:0/2043064607 conn(0x7f57740fecd0 msgr2=0x7f5774101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.492+0000 7f577b7c7640 1 -- 172.21.15.94:0/2043064607 shutdown_connections 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.494+0000 7f577b7c7640 1 -- 172.21.15.94:0/2043064607 wait complete. 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 Processor -- start 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 -- start start 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 0x7f5774074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774104d20 0x7f5774072a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 0x7f5774072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f577406cde0 con 0x7f5774103ae0 2024-04-25T14:03:41.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f577406cc60 con 0x7f5774109a20 2024-04-25T14:03:41.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.495+0000 7f577b7c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f577406cf60 con 0x7f5774104d20 2024-04-25T14:03:41.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.496+0000 7f5779563640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 0x7f5774074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.496+0000 7f5779563640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 0x7f5774074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55116/0 (socket says 172.21.15.94:55116) 2024-04-25T14:03:41.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.496+0000 7f5779563640 1 -- 172.21.15.94:0/2847231303 learned_addr learned my addr 172.21.15.94:0/2847231303 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.497+0000 7f5779d64640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 0x7f5774072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.497+0000 7f5778d62640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774104d20 0x7f5774072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.498+0000 7f5779d64640 1 -- 172.21.15.94:0/2847231303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774104d20 msgr2=0x7f5774072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.498+0000 7f5779d64640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774104d20 0x7f5774072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.498+0000 7f5779d64640 1 -- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 msgr2=0x7f5774074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.498+0000 7f5779d64640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 0x7f5774074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.498+0000 7f5779d64640 1 -- 172.21.15.94:0/2847231303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5774073500 con 0x7f5774109a20 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.501+0000 7f5779d64640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 0x7f5774072fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fd0 tx=0x7f576400eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.502+0000 7f576a7fc640 1 -- 172.21.15.94:0/2847231303 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764004230 con 0x7f5774109a20 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.502+0000 7f576a7fc640 1 -- 172.21.15.94:0/2847231303 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57640043d0 con 0x7f5774109a20 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.502+0000 7f576a7fc640 1 -- 172.21.15.94:0/2847231303 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640047c0 con 0x7f5774109a20 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.502+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57740737f0 con 0x7f5774109a20 2024-04-25T14:03:41.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.502+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a8b70 con 0x7f5774109a20 2024-04-25T14:03:41.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.506+0000 7f5782674740 1 Processor -- start 2024-04-25T14:03:41.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.508+0000 7f576a7fc640 1 -- 172.21.15.94:0/2847231303 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f576400c8b0 con 0x7f5774109a20 2024-04-25T14:03:41.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.509+0000 7f576a7fc640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 0x7f57600b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.509+0000 7f576a7fc640 1 -- 172.21.15.94:0/2847231303 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f57640dedf0 con 0x7f5774109a20 2024-04-25T14:03:41.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 -- start start 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5350810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a5350d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5584a52b4d80 con 0x5584a4fae390 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5584a526ea20 con 0x5584a4f8d6b0 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5782674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584a52b0d40 con 0x5584a4f854d0 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5779d64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5779d64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36634/0 (socket says 172.21.15.94:36634) 2024-04-25T14:03:41.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.510+0000 7f5779d64640 1 -- 172.21.15.94:0/112597494 learned_addr learned my addr 172.21.15.94:0/112597494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5778d62640 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a5350d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5779d64640 1 -- 172.21.15.94:0/112597494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 msgr2=0x5584a5350d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5779d64640 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a5350d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5778d62640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 0x7f57600b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5779d64640 1 -- 172.21.15.94:0/112597494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 msgr2=0x5584a5350810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5779d64640 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5350810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.511+0000 7f5779d64640 1 -- 172.21.15.94:0/112597494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a5279c30 con 0x5584a4f8d6b0 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.512+0000 7f5779d64640 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f57640a6ed0 tx=0x7f57640a6a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746a0e906c1de6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.512+0000 7f5778d62640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 0x7f57600b2210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5584a5350530 tx=0x7f5770005a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.512+0000 7f575f7fe640 1 -- 172.21.15.94:0/112597494 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640c0020 con 0x5584a4f8d6b0 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.512+0000 7f575f7fe640 1 -- 172.21.15.94:0/112597494 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57640ab580 con 0x5584a4f8d6b0 2024-04-25T14:03:41.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.512+0000 7f575f7fe640 1 -- 172.21.15.94:0/112597494 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640a3600 con 0x5584a4f8d6b0 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.513+0000 7f5782674740 1 -- 172.21.15.94:0/112597494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 msgr2=0x5584a5351a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.513+0000 7f5782674740 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f57640a6ed0 tx=0x7f57640a6a00 comp rx=0 tx=0).stop 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 -- 172.21.15.94:0/112597494 shutdown_connections 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a5351a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a5350d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 --2- 172.21.15.94:0/112597494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5350810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 -- 172.21.15.94:0/112597494 >> 172.21.15.94:0/112597494 conn(0x5584a4fa24e0 msgr2=0x5584a534c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 -- 172.21.15.94:0/112597494 shutdown_connections 2024-04-25T14:03:41.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.514+0000 7f5782674740 1 -- 172.21.15.94:0/112597494 wait complete. 2024-04-25T14:03:41.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.515+0000 7f5782674740 1 -- start start 2024-04-25T14:03:41.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.515+0000 7f5782674740 1 Processor -- start 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a535aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5361a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5584a5352530 con 0x5584a4fae390 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5584a523bdc0 con 0x5584a4f8d6b0 2024-04-25T14:03:41.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584a53526b0 con 0x5584a4f854d0 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5782674740 10 client.0 _ll_register_callbacks cb 0x5584a5257600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5779563640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5779563640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35014/0 (socket says 172.21.15.94:35014) 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5779563640 1 -- 172.21.15.94:0/4148456129 learned_addr learned my addr 172.21.15.94:0/4148456129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.516+0000 7f5778d62640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a535aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779d64640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5361a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 -- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 msgr2=0x5584a535aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a535aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 -- 172.21.15.94:0/4148456129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 msgr2=0x5584a5361a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5361a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 -- 172.21.15.94:0/4148456129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a4fc63f0 con 0x5584a4f854d0 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779d64640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5361a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f5779563640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f576c031b90 tx=0x7f576c010670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b00af1107caa80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f575cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.517+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c031650 con 0x5584a4f854d0 2024-04-25T14:03:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f576c0317f0 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57640ab580 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f576c044440 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f5782674740 10 client.32332 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5584a50907d0 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.518+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57640ab580 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f576c042070 con 0x5584a4f854d0 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 10 client.32332 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 10 client.32332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 20 client.32332 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f6cf60 0x5584a53682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 4 client.32332 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a52c2990 con 0x5584a4f6cf60 2024-04-25T14:03:41.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.519+0000 7f5778d62640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f6cf60 0x5584a53682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.520+0000 7f5778d62640 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f6cf60 0x5584a53682d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd7f8dd58c775806 server_cookie=6b66c72923a14343 in_seq=0 out_seq=0 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.520+0000 7f575cff9640 10 client.32332 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f575cff9640 1 -- 172.21.15.94:0/4148456129 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584a52c2990 con 0x5584a4f6cf60 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f575cff9640 10 client.32332 handle_command_reply: tid=0 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 client.32332 shutdown 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 msgr2=0x5584a535a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f576c031b90 tx=0x7f576c010670 comp rx=0 tx=0).stop 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 shutdown_connections 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f6cf60 0x5584a53682d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4fae390 0x5584a5361a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f8d6b0 0x5584a535aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 --2- 172.21.15.94:0/4148456129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a535a5a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 >> 172.21.15.94:0/4148456129 conn(0x5584a4fa24e0 msgr2=0x5584a534b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f575cff9640 0 client.32332 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 shutdown_connections 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 1 -- 172.21.15.94:0/4148456129 wait complete. 2024-04-25T14:03:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.521+0000 7f5782674740 20 client.32332 trim_cache size 0 max 16384 2024-04-25T14:03:41.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 Processor -- start 2024-04-25T14:03:41.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 -- start start 2024-04-25T14:03:41.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a534f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 0x5584a5358d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575800e7f0 con 0x5584a4f8d6b0 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5758010330 con 0x5584a4fae390 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5782674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575800eec0 con 0x5584a4f854d0 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5778d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5778d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55128/0 (socket says 172.21.15.94:55128) 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.536+0000 7f5778d62640 1 -- 172.21.15.94:0/3646833886 learned_addr learned my addr 172.21.15.94:0/3646833886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5779563640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a534f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5779d64640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 0x5584a5358d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 -- 172.21.15.94:0/3646833886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 msgr2=0x5584a534f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a534f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 -- 172.21.15.94:0/3646833886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 msgr2=0x5584a5358d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 0x5584a5358d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 -- 172.21.15.94:0/3646833886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a535aef0 con 0x5584a4f8d6b0 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5779d64640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 0x5584a5358d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f5778d62640 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f5770013040 tx=0x7f5770005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc4492ec0569b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f575effd640 1 -- 172.21.15.94:0/3646833886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770021510 con 0x5584a4f8d6b0 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f575effd640 1 -- 172.21.15.94:0/3646833886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5770018040 con 0x5584a4f8d6b0 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.537+0000 7f575effd640 1 -- 172.21.15.94:0/3646833886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577001f940 con 0x5584a4f8d6b0 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- 172.21.15.94:0/3646833886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 msgr2=0x5584a5361960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f5770013040 tx=0x7f5770005bd0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- 172.21.15.94:0/3646833886 shutdown_connections 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4fae390 0x5584a5358d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a5361960 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 --2- 172.21.15.94:0/3646833886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4f854d0 0x5584a534f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- 172.21.15.94:0/3646833886 >> 172.21.15.94:0/3646833886 conn(0x5584a4fa24e0 msgr2=0x5584a535a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- 172.21.15.94:0/3646833886 shutdown_connections 2024-04-25T14:03:41.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- 172.21.15.94:0/3646833886 wait complete. 2024-04-25T14:03:41.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.538+0000 7f5782674740 1 -- start start 2024-04-25T14:03:41.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 Processor -- start 2024-04-25T14:03:41.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a535f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 0x5584a535ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575800fc60 con 0x5584a4f8d6b0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575800f590 con 0x5584a4f854d0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575800c7d0 con 0x5584a4fae390 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5782674740 10 client.0 _ll_register_callbacks cb 0x7f5764005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779563640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5778d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a535f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779563640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36668/0 (socket says 172.21.15.94:36668) 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5778d62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a535f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55142/0 (socket says 172.21.15.94:55142) 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779563640 1 -- 172.21.15.94:0/1861313955 learned_addr learned my addr 172.21.15.94:0/1861313955 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779d64640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 0x5584a535ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779563640 1 -- 172.21.15.94:0/1861313955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 msgr2=0x5584a535ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.539+0000 7f5779563640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 0x5584a535ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5779563640 1 -- 172.21.15.94:0/1861313955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 msgr2=0x5584a535f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5779563640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a535f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5779563640 1 -- 172.21.15.94:0/1861313955 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a5361ea0 con 0x5584a4f854d0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5779d64640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 0x5584a535ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5779563640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f576c030860 tx=0x7f576c030d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1deded056710b471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f575cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c051020 con 0x5584a4f854d0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f576c0445e0 con 0x5584a4f854d0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5770018040 con 0x5584a4f854d0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f576c05c050 con 0x5584a4f854d0 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5782674740 10 client.32684 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.540+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5584a53615d0 con 0x5584a4f854d0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5770018040 con 0x5584a4f854d0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f576c042070 con 0x5584a4f854d0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 10 client.32684 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 10 client.32684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 20 client.32684 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f423a0 0x5584a53d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 4 client.32684 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584a52bc800 con 0x5584a4f423a0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5778d62640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f423a0 0x5584a53d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f575cff9640 10 client.32684 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.541+0000 7f5778d62640 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f423a0 0x5584a53d2830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=785395ca0dd16e0f server_cookie=d92fc132b7ae6103 in_seq=0 out_seq=0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f575cff9640 1 -- 172.21.15.94:0/1861313955 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584a52bc800 con 0x5584a4f423a0 2024-04-25T14:03:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f575cff9640 10 client.32684 handle_command_reply: tid=0 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 client.32684 shutdown 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 msgr2=0x5584a535eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f576c030860 tx=0x7f576c030d00 comp rx=0 tx=0).stop 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 shutdown_connections 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5584a4f423a0 0x5584a53d2830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584a4fae390 0x5584a535ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f575cff9640 0 client.32684 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5584a4f8d6b0 0x5584a535f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 --2- 172.21.15.94:0/1861313955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584a4f854d0 0x5584a535eac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 >> 172.21.15.94:0/1861313955 conn(0x5584a4fa24e0 msgr2=0x5584a535a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 shutdown_connections 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 1 -- 172.21.15.94:0/1861313955 wait complete. 2024-04-25T14:03:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.542+0000 7f5782674740 20 client.32684 trim_cache size 0 max 16384 2024-04-25T14:03:41.549 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 msgr2=0x7f57600b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 0x7f57600b2210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5584a5350530 tx=0x7f5770005a80 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 msgr2=0x7f5774072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 0x7f5774072fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5764009fd0 tx=0x7f576400eba0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 shutdown_connections 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f57600afd00 0x7f57600b2210 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774109a20 0x7f5774072fc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5774104d20 0x7f5774072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 --2- 172.21.15.94:0/2847231303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5774103ae0 0x7f5774074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 >> 172.21.15.94:0/2847231303 conn(0x7f57740fecd0 msgr2=0x7f5774105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 shutdown_connections 2024-04-25T14:03:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.543+0000 7f577b7c7640 1 -- 172.21.15.94:0/2847231303 wait complete. 2024-04-25T14:03:41.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:03:41.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2646158693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec1029e0 msgr2=0x7fb5ec1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2646158693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec1036e0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009900 tx=0x7fb5d402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:41.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2646158693 shutdown_connections 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2646158693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec108920 0x7fb5ec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2646158693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2646158693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec1036e0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2646158693 >> 172.21.15.94:0/2646158693 conn(0x7fb5ec0fdbd0 msgr2=0x7fb5ec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2646158693 shutdown_connections 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.670+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2646158693 wait complete. 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 Processor -- start 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 -- start start 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10e4e0 con 0x7fb5ec108920 2024-04-25T14:03:41.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10e360 con 0x7fb5ec103c20 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f2f5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec10e660 con 0x7fb5ec1029e0 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55160/0 (socket says 172.21.15.94:55160) 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 -- 172.21.15.94:0/2630621318 learned_addr learned my addr 172.21.15.94:0/2630621318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 -- 172.21.15.94:0/2630621318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec1029e0 msgr2=0x7fb5ec19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 -- 172.21.15.94:0/2630621318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 msgr2=0x7fb5ec19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 -- 172.21.15.94:0/2630621318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1a2060 con 0x7fb5ec108920 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.671+0000 7fb5f14fb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400eb10 tx=0x7fb5e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.672+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2630621318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4004b20 con 0x7fb5ec108920 2024-04-25T14:03:41.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.672+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1a2350 con 0x7fb5ec108920 2024-04-25T14:03:41.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.672+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2630621318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e4004cc0 con 0x7fb5ec108920 2024-04-25T14:03:41.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.672+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2630621318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4010690 con 0x7fb5ec108920 2024-04-25T14:03:41.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.672+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1a28f0 con 0x7fb5ec108920 2024-04-25T14:03:41.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.674+0000 7fb5f9e0f740 1 Processor -- start 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2630621318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb5e4005020 con 0x7fb5ec108920 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 -- start start 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe669c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 0x55bbe669da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bbe6600d80 con 0x55bbe62d14d0 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbe65baa20 con 0x55bbe62fa390 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbe65fcd40 con 0x55bbe62d96b0 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55164/0 (socket says 172.21.15.94:55164) 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 0x55bbe669da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1121988226 learned_addr learned my addr 172.21.15.94:0/1121988226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe669c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5e9ffb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 0x7fb5d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1121988226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 msgr2=0x55bbe669da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 0x55bbe669da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1121988226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 msgr2=0x55bbe669c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe669c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1121988226 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe65c5c30 con 0x55bbe62d14d0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.675+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 0x55bbe669da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5f14fb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 0x7fb5d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009230 tx=0x7fb5d4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deb417ee886423c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2630621318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb5e40df210 con 0x7fb5ec108920 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5e2ffd640 1 -- 172.21.15.94:0/1121988226 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4035030 con 0x55bbe62d14d0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5e2ffd640 1 -- 172.21.15.94:0/1121988226 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d4040070 con 0x55bbe62d14d0 2024-04-25T14:03:41.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.676+0000 7fb5e2ffd640 1 -- 172.21.15.94:0/1121988226 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40038c0 con 0x55bbe62d14d0 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1121988226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 msgr2=0x55bbe669cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f14fb640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 0x7fb5d80b2000 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40a7350 tx=0x7fb5e40761a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009230 tx=0x7fb5d4009620 comp rx=0 tx=0).stop 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1121988226 shutdown_connections 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d96b0 0x55bbe669da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d14d0 0x55bbe669cd90 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1121988226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe669c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1121988226 >> 172.21.15.94:0/1121988226 conn(0x55bbe62ee4e0 msgr2=0x55bbe6698020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1121988226 shutdown_connections 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1121988226 wait complete. 2024-04-25T14:03:41.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 -- start start 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.677+0000 7fb5f9e0f740 1 Processor -- start 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d14d0 0x55bbe66a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62fa390 0x55bbe66ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bbe669e4d0 con 0x55bbe62d96b0 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbe6587dc0 con 0x55bbe62d14d0 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbe669e650 con 0x55bbe62fa390 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f9e0f740 10 client.0 _ll_register_callbacks cb 0x55bbe65a3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55178/0 (socket says 172.21.15.94:55178) 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/965746021 learned_addr learned my addr 172.21.15.94:0/965746021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d14d0 0x55bbe66a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62fa390 0x55bbe66ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/965746021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62fa390 msgr2=0x55bbe66ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62fa390 0x55bbe66ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d14d0 msgr2=0x55bbe66a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d14d0 0x55bbe66a6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/965746021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe63123f0 con 0x55bbe62d96b0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4031500 tx=0x7fb5d402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780303f5bbe7ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.678+0000 7fb5cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4035070 con 0x55bbe62d96b0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb5d4003660 con 0x55bbe62d96b0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d4040070 con 0x55bbe62d96b0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5d40038a0 con 0x55bbe62d96b0 2024-04-25T14:03:41.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 10 client.23316 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bbe63dc7d0 con 0x55bbe62d96b0 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d4040070 con 0x55bbe62d96b0 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb5d40459e0 con 0x55bbe62d96b0 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 10 client.23316 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 10 client.23316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 20 client.23316 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe62b8f60 0x55bbe66b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 4 client.23316 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.679+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe660e990 con 0x55bbe62b8f60 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.680+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe62b8f60 0x55bbe66b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.680+0000 7fb5cbfff640 10 client.23316 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.680+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe62b8f60 0x55bbe66b3e10 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52d53ef71118ef2 server_cookie=bd38263aedce0d56 in_seq=0 out_seq=0 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5cbfff640 1 -- 172.21.15.94:0/965746021 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbe660e990 con 0x55bbe62b8f60 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5cbfff640 10 client.23316 handle_command_reply: tid=0 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 client.23316 shutdown 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 msgr2=0x55bbe66a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4031500 tx=0x7fb5d402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 shutdown_connections 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe62b8f60 0x55bbe66b3e10 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62fa390 0x55bbe66ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66a6a60 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5cbfff640 0 client.23316 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/965746021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d14d0 0x55bbe66a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 >> 172.21.15.94:0/965746021 conn(0x55bbe62ee4e0 msgr2=0x55bbe669af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 shutdown_connections 2024-04-25T14:03:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.683+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/965746021 wait complete. 2024-04-25T14:03:41.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.684+0000 7fb5f9e0f740 20 client.23316 trim_cache size 0 max 16384 2024-04-25T14:03:41.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 Processor -- start 2024-04-25T14:03:41.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 -- start start 2024-04-25T14:03:41.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ab500 con 0x55bbe62fa390 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ad040 con 0x55bbe62d96b0 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1abbd0 con 0x55bbe62d14d0 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55188/0 (socket says 172.21.15.94:55188) 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.698+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1814764745 learned_addr learned my addr 172.21.15.94:0/1814764745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1814764745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 msgr2=0x55bbe66a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1814764745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d96b0 msgr2=0x55bbe66ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 -- 172.21.15.94:0/1814764745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe66a5900 con 0x55bbe62fa390 2024-04-25T14:03:41.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x55bbe66b4480 tx=0x7fb5dc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b52097440bb4be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5e27fc640 1 -- 172.21.15.94:0/1814764745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc011310 con 0x55bbe62fa390 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5e27fc640 1 -- 172.21.15.94:0/1814764745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc00f850 con 0x55bbe62fa390 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.699+0000 7fb5e27fc640 1 -- 172.21.15.94:0/1814764745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc005280 con 0x55bbe62fa390 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1814764745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 msgr2=0x55bbe66b94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x55bbe66b4480 tx=0x7fb5dc013040 comp rx=0 tx=0).stop 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1814764745 shutdown_connections 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62fa390 0x55bbe66b94e0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1814764745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1814764745 >> 172.21.15.94:0/1814764745 conn(0x55bbe62ee4e0 msgr2=0x55bbe66a4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1814764745 shutdown_connections 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.700+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1814764745 wait complete. 2024-04-25T14:03:41.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 -- start start 2024-04-25T14:03:41.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 Processor -- start 2024-04-25T14:03:41.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66accf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe671a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ac970 con 0x55bbe62d96b0 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1ac2a0 con 0x55bbe62fa390 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1a94e0 con 0x55bbe62d14d0 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f9e0f740 10 client.0 _ll_register_callbacks cb 0x7fb5dc011630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f0cfa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55200/0 (socket says 172.21.15.94:55200) 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1116331756 learned_addr learned my addr 172.21.15.94:0/1116331756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.701+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe671a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1116331756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 msgr2=0x55bbe66accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 msgr2=0x55bbe671a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe671a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 -- 172.21.15.94:0/1116331756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe66ade20 con 0x55bbe62d96b0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe671a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f0cfa640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4031440 tx=0x7fb5d402aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec323e27fc89b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d405b070 con 0x55bbe62d96b0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb5d4003c40 con 0x55bbe62d96b0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5dc00f850 con 0x55bbe62d96b0 2024-04-25T14:03:41.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5d4067050 con 0x55bbe62d96b0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f9e0f740 10 client.23322 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bbe671b9e0 con 0x55bbe62d96b0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc00f850 con 0x55bbe62d96b0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.702+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb5d4060560 con 0x55bbe62d96b0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 10 client.23322 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 10 client.23322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 20 client.23322 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe628e3a0 0x55bbe671e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 4 client.23322 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbe6608800 con 0x55bbe628e3a0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe628e3a0 0x55bbe671e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5cbfff640 10 client.23322 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:41.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.703+0000 7fb5ebfff640 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe628e3a0 0x55bbe671e370 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91eb35858a632a4f server_cookie=a44d61dd1775169b in_seq=0 out_seq=0 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5cbfff640 1 -- 172.21.15.94:0/1116331756 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbe6608800 con 0x55bbe628e3a0 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5cbfff640 10 client.23322 handle_command_reply: tid=0 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 client.23322 shutdown 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 msgr2=0x55bbe66ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4031440 tx=0x7fb5d402aba0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 shutdown_connections 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55bbe628e3a0 0x55bbe671e370 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbe62fa390 0x55bbe671a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bbe62d96b0 0x55bbe66ad230 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5cbfff640 0 client.23322 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 --2- 172.21.15.94:0/1116331756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbe62d14d0 0x55bbe66accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.706+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 >> 172.21.15.94:0/1116331756 conn(0x55bbe62ee4e0 msgr2=0x55bbe669ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 shutdown_connections 2024-04-25T14:03:41.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f9e0f740 1 -- 172.21.15.94:0/1116331756 wait complete. 2024-04-25T14:03:41.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f9e0f740 20 client.23322 trim_cache size 0 max 16384 2024-04-25T14:03:41.711 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 msgr2=0x7fb5d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 0x7fb5d80b2000 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40a7350 tx=0x7fb5e40761a0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 msgr2=0x7fb5ec1a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400eb10 tx=0x7fb5e400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 shutdown_connections 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb5d80afaf0 0x7fb5d80b2000 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5ec108920 0x7fb5ec1a19f0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5ec103c20 0x7fb5ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 --2- 172.21.15.94:0/2630621318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5ec1029e0 0x7fb5ec19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:41.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.707+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 >> 172.21.15.94:0/2630621318 conn(0x7fb5ec0fdbd0 msgr2=0x7fb5ec069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:41.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.708+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 shutdown_connections 2024-04-25T14:03:41.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:41.708+0000 7fb5f2f5e640 1 -- 172.21.15.94:0/2630621318 wait complete. 2024-04-25T14:03:41.789 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:41 smithi138 ceph-mon[32977]: pgmap v1008: 97 pgs: 97 active+clean; 264 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-04-25T14:03:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:41 smithi094 ceph-mon[29195]: pgmap v1008: 97 pgs: 97 active+clean; 264 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-04-25T14:03:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:41 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2423763440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:42.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:41 smithi137 ceph-mon[32568]: pgmap v1008: 97 pgs: 97 active+clean; 264 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-04-25T14:03:42.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:41 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2423763440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:42.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 179040 18.88 MB/sec execute 384 sec latency 159.288 ms 2024-04-25T14:03:42.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:41 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2423763440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:42.719 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:03:42.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:42.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 -- 172.21.15.94:0/197367022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 msgr2=0x7f797010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 --2- 172.21.15.94:0/197367022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 0x7f797010ad40 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f7968009ca0 tx=0x7f796802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 -- 172.21.15.94:0/197367022 shutdown_connections 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 --2- 172.21.15.94:0/197367022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 0x7f797010ad40 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 --2- 172.21.15.94:0/197367022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7970103c20 0x7f7970104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 --2- 172.21.15.94:0/197367022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79701029e0 0x7f79701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 -- 172.21.15.94:0/197367022 >> 172.21.15.94:0/197367022 conn(0x7f79700fdbd0 msgr2=0x7f7970100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 -- 172.21.15.94:0/197367022 shutdown_connections 2024-04-25T14:03:42.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.844+0000 7f7977184640 1 -- 172.21.15.94:0/197367022 wait complete. 2024-04-25T14:03:42.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 Processor -- start 2024-04-25T14:03:42.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 -- start start 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7970103c20 0x7f79701987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 0x7f79701a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f797010f670 con 0x7f7970108920 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f797010f4f0 con 0x7f7970103c20 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.845+0000 7f7977184640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f797010f7f0 con 0x7f79701029e0 2024-04-25T14:03:42.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35094/0 (socket says 172.21.15.94:35094) 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 -- 172.21.15.94:0/4208524555 learned_addr learned my addr 172.21.15.94:0/4208524555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f796ffff640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7970103c20 0x7f79701987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 -- 172.21.15.94:0/4208524555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7970103c20 msgr2=0x7f79701987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7970103c20 0x7f79701987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 -- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 msgr2=0x7f79701a7f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:03:42.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 0x7f79701a7f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 -- 172.21.15.94:0/4208524555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79701a85e0 con 0x7f79701029e0 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.846+0000 7f7974f20640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f796400e9f0 tx=0x7f796400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.847+0000 7f796dffb640 1 -- 172.21.15.94:0/4208524555 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400ccc0 con 0x7f79701029e0 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.847+0000 7f796dffb640 1 -- 172.21.15.94:0/4208524555 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79640035f0 con 0x7f79701029e0 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.847+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79701a88d0 con 0x7f79701029e0 2024-04-25T14:03:42.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.847+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79701a8dc0 con 0x7f79701029e0 2024-04-25T14:03:42.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.847+0000 7f796dffb640 1 -- 172.21.15.94:0/4208524555 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400ce60 con 0x7f79701029e0 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.851+0000 7f796dffb640 1 -- 172.21.15.94:0/4208524555 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7964010690 con 0x7f79701029e0 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.852+0000 7f796dffb640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 0x7f79600b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.852+0000 7f796dffb640 1 -- 172.21.15.94:0/4208524555 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f79640e0000 con 0x7f79701029e0 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 Processor -- start 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 -- start start 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f796ffff640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 0x7f79600b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b388541b00 con 0x55b38821a730 2024-04-25T14:03:42.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b3884dd920 con 0x55b38823b410 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f797e039740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b38853dac0 con 0x55b388212550 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f7975721640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f796ffff640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 0x7f79600b21d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f79701a1b60 tx=0x7f795c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f7975721640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55220/0 (socket says 172.21.15.94:55220) 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f7975721640 1 -- 172.21.15.94:0/2549029330 learned_addr learned my addr 172.21.15.94:0/2549029330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.853+0000 7f796ffff640 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7974f20640 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 -- 172.21.15.94:0/2549029330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 msgr2=0x55b3885dca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885dca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 -- 172.21.15.94:0/2549029330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 msgr2=0x55b3885dc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885dc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 -- 172.21.15.94:0/2549029330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38831b7e0 con 0x55b38821a730 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f7975721640 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7968009800 tx=0x7f7968009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f05e7c15d82272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f795affd640 1 -- 172.21.15.94:0/2549029330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968031450 con 0x55b38821a730 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f795affd640 1 -- 172.21.15.94:0/2549029330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968036070 con 0x55b38821a730 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.854+0000 7f795affd640 1 -- 172.21.15.94:0/2549029330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968004460 con 0x55b38821a730 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- 172.21.15.94:0/2549029330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 msgr2=0x55b3885e3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7968009800 tx=0x7f7968009830 comp rx=0 tx=0).stop 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- 172.21.15.94:0/2549029330 shutdown_connections 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885e3920 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 --2- 172.21.15.94:0/2549029330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885dc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- 172.21.15.94:0/2549029330 >> 172.21.15.94:0/2549029330 conn(0x55b38822f560 msgr2=0x55b3885d7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- 172.21.15.94:0/2549029330 shutdown_connections 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- 172.21.15.94:0/2549029330 wait complete. 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.855+0000 7f797e039740 1 -- start start 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 Processor -- start 2024-04-25T14:03:42.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 0x55b3885e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3885e4310 con 0x55b388212550 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b3884db5d0 con 0x55b38823b410 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b3885e4490 con 0x55b38821a730 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f797e039740 10 client.0 _ll_register_callbacks cb 0x55b3884c9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f7974f20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f7974f20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55222/0 (socket says 172.21.15.94:55222) 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f7974f20640 1 -- 172.21.15.94:0/3437617031 learned_addr learned my addr 172.21.15.94:0/3437617031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f796ffff640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 0x55b3885e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.856+0000 7f7975721640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 -- 172.21.15.94:0/3437617031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 msgr2=0x55b3885e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 0x55b3885e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 -- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 msgr2=0x55b3885ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 -- 172.21.15.94:0/3437617031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38837bf70 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f796ffff640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 0x55b3885e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f7974f20640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f7964009250 tx=0x7f79640a7fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af10211fdbc2944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f794ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640c1070 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7964003aa0 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968036070 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.857+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79640ac790 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.858+0000 7f797e039740 10 client.23331 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.858+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b388213170 con 0x55b388212550 2024-04-25T14:03:42.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.858+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968036070 con 0x55b388212550 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.858+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f79640a7640 con 0x55b388212550 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 10 client.23331 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 10 client.23331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 20 client.23331 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881fa1f0 0x55b3885f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 4 client.23331 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b38854f710 con 0x55b3881fa1f0 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f796ffff640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881fa1f0 0x55b3885f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.859+0000 7f794ffff640 10 client.23331 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:42.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.860+0000 7f796ffff640 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881fa1f0 0x55b3885f3c50 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a097859df533460 server_cookie=d757f19c40125a1e in_seq=0 out_seq=0 2024-04-25T14:03:42.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.864+0000 7f794ffff640 1 -- 172.21.15.94:0/3437617031 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b38854f710 con 0x55b3881fa1f0 2024-04-25T14:03:42.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.864+0000 7f794ffff640 10 client.23331 handle_command_reply: tid=0 2024-04-25T14:03:42.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.864+0000 7f797e039740 1 client.23331 shutdown 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.864+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 msgr2=0x55b3885e61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.864+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f7964009250 tx=0x7f79640a7fb0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 shutdown_connections 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881fa1f0 0x55b3885f3c50 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b38821a730 0x55b3885e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 --2- 172.21.15.94:0/3437617031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b388212550 0x55b3885e61e0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 >> 172.21.15.94:0/3437617031 conn(0x55b38822f560 msgr2=0x55b3885d6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f794ffff640 0 client.23331 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 shutdown_connections 2024-04-25T14:03:42.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 1 -- 172.21.15.94:0/3437617031 wait complete. 2024-04-25T14:03:42.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.865+0000 7f797e039740 20 client.23331 trim_cache size 0 max 16384 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.892+0000 7f797e039740 1 Processor -- start 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.892+0000 7f797e039740 1 -- start start 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.892+0000 7f797e039740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.892+0000 7f797e039740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38821a730 0x55b3885ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.892+0000 7f797e039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 0x55b3885db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f797e039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f795400e730 con 0x55b38823b410 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f797e039740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7954010270 con 0x55b38821a730 2024-04-25T14:03:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f797e039740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f795400ee00 con 0x55b388212550 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 0x55b3885db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35122/0 (socket says 172.21.15.94:35122) 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7974f20640 1 -- 172.21.15.94:0/1922824046 learned_addr learned my addr 172.21.15.94:0/1922824046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f796ffff640 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38821a730 0x55b3885ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 -- 172.21.15.94:0/1922824046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 msgr2=0x55b3885e0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 -- 172.21.15.94:0/1922824046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38821a730 msgr2=0x55b3885ed610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38821a730 0x55b3885ed610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 -- 172.21.15.94:0/1922824046 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3881fa420 con 0x55b38823b410 2024-04-25T14:03:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7974f20640 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:42.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.893+0000 7f7975721640 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 0x55b3885db340 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f7968009ca0 tx=0x7f7968005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fa319b6db863c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.894+0000 7f795a7fc640 1 -- 172.21.15.94:0/1922824046 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968051020 con 0x55b38823b410 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.894+0000 7f795a7fc640 1 -- 172.21.15.94:0/1922824046 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968036070 con 0x55b38823b410 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.894+0000 7f795a7fc640 1 -- 172.21.15.94:0/1922824046 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968049040 con 0x55b38823b410 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- 172.21.15.94:0/1922824046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 msgr2=0x55b3885db340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 0x55b3885db340 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f7968009ca0 tx=0x7f7968005d80 comp rx=0 tx=0).stop 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- 172.21.15.94:0/1922824046 shutdown_connections 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38823b410 0x55b3885db340 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38821a730 0x55b3885ed610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 --2- 172.21.15.94:0/1922824046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- 172.21.15.94:0/1922824046 >> 172.21.15.94:0/1922824046 conn(0x55b38822f560 msgr2=0x55b3885dac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- 172.21.15.94:0/1922824046 shutdown_connections 2024-04-25T14:03:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- 172.21.15.94:0/1922824046 wait complete. 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 -- start start 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.895+0000 7f797e039740 1 Processor -- start 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f796ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f7974f20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35124/0 (socket says 172.21.15.94:35124) 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f796ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55240/0 (socket says 172.21.15.94:55240) 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f7974f20640 1 -- 172.21.15.94:0/2724202111 learned_addr learned my addr 172.21.15.94:0/2724202111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f795400fba0 con 0x55b38821a730 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f795400f4d0 con 0x55b38823b410 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f795400c710 con 0x55b388212550 2024-04-25T14:03:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f7975721640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.896+0000 7f797e039740 10 client.0 _ll_register_callbacks cb 0x7f79640a7b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 -- 172.21.15.94:0/2724202111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 msgr2=0x55b3885eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 -- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 msgr2=0x55b3885ec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3885ec9b0 con 0x55b38821a730 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f796ffff640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x55b3885f43f0 tx=0x7f795c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a71a3ec6bc9af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f794ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c02b380 con 0x55b38821a730 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f795c02b520 con 0x55b38821a730 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7968036070 con 0x55b38821a730 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f795c02b760 con 0x55b38821a730 2024-04-25T14:03:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f797e039740 10 client.23337 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b38865c190 con 0x55b38821a730 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.897+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795c02b900 con 0x55b38821a730 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f795c03c400 con 0x55b38821a730 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 10 client.23337 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 10 client.23337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 20 client.23337 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881cf630 0x55b38865eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 4 client.23337 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7968036070 con 0x55b3881cf630 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.898+0000 7f7974f20640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881cf630 0x55b38865eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.899+0000 7f7974f20640 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881cf630 0x55b38865eb30 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ba56d2384bdbcc server_cookie=4d0dcba3d5331f0b in_seq=0 out_seq=0 2024-04-25T14:03:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.899+0000 7f794ffff640 10 client.23337 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.923+0000 7f794ffff640 1 -- 172.21.15.94:0/2724202111 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7968036070 con 0x55b3881cf630 2024-04-25T14:03:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.923+0000 7f794ffff640 10 client.23337 handle_command_reply: tid=0 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 client.23337 shutdown 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 msgr2=0x55b3885eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x55b3885f43f0 tx=0x7f795c004730 comp rx=0 tx=0).stop 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 shutdown_connections 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b3881cf630 0x55b38865eb30 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b38823b410 0x55b3885ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b38821a730 0x55b3885eb4c0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f794ffff640 0 client.23337 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 --2- 172.21.15.94:0/2724202111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b388212550 0x55b3885eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 >> 172.21.15.94:0/2724202111 conn(0x55b38822f560 msgr2=0x55b3885db660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 shutdown_connections 2024-04-25T14:03:42.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 1 -- 172.21.15.94:0/2724202111 wait complete. 2024-04-25T14:03:42.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.924+0000 7f797e039740 20 client.23337 trim_cache size 0 max 16384 2024-04-25T14:03:42.928 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:42.928 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:03:42.928 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "c910b7ac-5719-4ed3-86c9-ff4fa7034880", 2024-04-25T14:03:42.929 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:03:42.929 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 msgr2=0x7f79600b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 0x7f79600b21d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f79701a1b60 tx=0x7f795c005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 msgr2=0x7f7970198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f796400e9f0 tx=0x7f796400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 shutdown_connections 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79600afcc0 0x7f79600b21d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7970108920 0x7f79701a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7970103c20 0x7f79701987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 --2- 172.21.15.94:0/4208524555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79701029e0 0x7f7970198290 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 >> 172.21.15.94:0/4208524555 conn(0x7f79700fdbd0 msgr2=0x7f79700fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 shutdown_connections 2024-04-25T14:03:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:42.925+0000 7f7977184640 1 -- 172.21.15.94:0/4208524555 wait complete. 2024-04-25T14:03:42.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:42.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:43.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 -- 172.21.15.94:0/397446301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 msgr2=0x7efdd0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 --2- 172.21.15.94:0/397446301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd0102e70 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009900 tx=0x7efdc402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 -- 172.21.15.94:0/397446301 shutdown_connections 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 --2- 172.21.15.94:0/397446301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 0x7efdd010a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 --2- 172.21.15.94:0/397446301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 --2- 172.21.15.94:0/397446301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd0102e70 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 -- 172.21.15.94:0/397446301 >> 172.21.15.94:0/397446301 conn(0x7efdd00fd340 msgr2=0x7efdd00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 -- 172.21.15.94:0/397446301 shutdown_connections 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 -- 172.21.15.94:0/397446301 wait complete. 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.056+0000 7efdd6d64640 1 Processor -- start 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 -- start start 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 0x7efdd01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efdd010d3c0 con 0x7efdd0102170 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efdd010d240 con 0x7efdd01033b0 2024-04-25T14:03:43.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd6d64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdd010d540 con 0x7efdd01080b0 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55260/0 (socket says 172.21.15.94:55260) 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 -- 172.21.15.94:0/3268215114 learned_addr learned my addr 172.21.15.94:0/3268215114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdcffff640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd5301640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 0x7efdd01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 -- 172.21.15.94:0/3268215114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 msgr2=0x7efdd01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 0x7efdd01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 -- 172.21.15.94:0/3268215114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 msgr2=0x7efdd019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 -- 172.21.15.94:0/3268215114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd01a1d60 con 0x7efdd0102170 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdcffff640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.057+0000 7efdd4b00640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.058+0000 7efdcdffb640 1 -- 172.21.15.94:0/3268215114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc403f030 con 0x7efdd0102170 2024-04-25T14:03:43.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.058+0000 7efdcdffb640 1 -- 172.21.15.94:0/3268215114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4031910 con 0x7efdd0102170 2024-04-25T14:03:43.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.058+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd01a1ff0 con 0x7efdd0102170 2024-04-25T14:03:43.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.058+0000 7efdcdffb640 1 -- 172.21.15.94:0/3268215114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4035070 con 0x7efdd0102170 2024-04-25T14:03:43.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.058+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd01a2450 con 0x7efdd0102170 2024-04-25T14:03:43.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.060+0000 7efdcdffb640 1 -- 172.21.15.94:0/3268215114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efdc4042030 con 0x7efdd0102170 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 Processor -- start 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 -- start start 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b524d0 0x55b224f1cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 0x55b224f1dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b224e81d80 con 0x55b224b7b390 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b224e3b8b0 con 0x55b224b5a6b0 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdddc12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b224e7dd40 con 0x55b224b524d0 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b524d0 0x55b224f1cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55272/0 (socket says 172.21.15.94:55272) 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 -- 172.21.15.94:0/1816245082 learned_addr learned my addr 172.21.15.94:0/1816245082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdcffff640 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 0x55b224f1dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdcdffb640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 0x7efdc00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 -- 172.21.15.94:0/1816245082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b524d0 msgr2=0x55b224f1cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b524d0 0x55b224f1cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 -- 172.21.15.94:0/1816245082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 msgr2=0x55b224f1dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 0x55b224f1dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdd5301640 1 -- 172.21.15.94:0/1816245082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b224e0ca40 con 0x55b224b7b390 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.061+0000 7efdcffff640 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 0x55b224f1dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdd5301640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 0x7efdc00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdcdffb640 1 -- 172.21.15.94:0/3268215114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efdc40fefb0 con 0x7efdd0102170 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdd5301640 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e480 tx=0x7efdc800a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbbe5f314ff9635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdbaffd640 1 -- 172.21.15.94:0/1816245082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc800dc00 con 0x55b224b7b390 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdbaffd640 1 -- 172.21.15.94:0/1816245082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc800dda0 con 0x55b224b7b390 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.062+0000 7efdbaffd640 1 -- 172.21.15.94:0/1816245082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80233a0 con 0x55b224b7b390 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- 172.21.15.94:0/1816245082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 msgr2=0x55b224f1ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdd5301640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 0x7efdc00b2080 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efdc8020000 tx=0x7efdc8023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e480 tx=0x7efdc800a610 comp rx=0 tx=0).stop 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- 172.21.15.94:0/1816245082 shutdown_connections 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b5a6b0 0x55b224f1dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b524d0 0x55b224f1cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 --2- 172.21.15.94:0/1816245082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b7b390 0x55b224f1ca60 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- 172.21.15.94:0/1816245082 >> 172.21.15.94:0/1816245082 conn(0x55b224b6f4e0 msgr2=0x55b224f18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- 172.21.15.94:0/1816245082 shutdown_connections 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- 172.21.15.94:0/1816245082 wait complete. 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.063+0000 7efdddc12740 1 -- start start 2024-04-25T14:03:43.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 Processor -- start 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b224f1e780 con 0x55b224b524d0 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b224e08d10 con 0x55b224b7b390 2024-04-25T14:03:43.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b224f1e900 con 0x55b224b5a6b0 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdddc12740 10 client.0 _ll_register_callbacks cb 0x55b224e35210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdcffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55286/0 (socket says 172.21.15.94:55286) 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdd5301640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdcffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35156/0 (socket says 172.21.15.94:35156) 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.064+0000 7efdd4b00640 1 -- 172.21.15.94:0/76610871 learned_addr learned my addr 172.21.15.94:0/76610871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:43.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 -- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 msgr2=0x55b224f2ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 -- 172.21.15.94:0/76610871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 msgr2=0x55b224f26990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 -- 172.21.15.94:0/76610871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b224c5d7d0 con 0x55b224b5a6b0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdd4b00640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdd5301640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdcffff640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdd019b2c0 tx=0x7efdbc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d77630afc5015d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdaffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc012110 con 0x55b224b5a6b0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efdbc004050 con 0x55b224b5a6b0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc800dda0 con 0x55b224b5a6b0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0122b0 con 0x55b224b5a6b0 2024-04-25T14:03:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdddc12740 10 client.32383 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b224b933f0 con 0x55b224b5a6b0 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.065+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc800dda0 con 0x55b224b5a6b0 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efdbc010430 con 0x55b224b5a6b0 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 10 client.32383 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 10 client.32383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 20 client.32383 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b39f60 0x55b224f34290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 4 client.32383 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b224e8f990 con 0x55b224b39f60 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdd4b00640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b39f60 0x55b224f34290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdaffff640 10 client.32383 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.066+0000 7efdd4b00640 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b39f60 0x55b224f34290 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b0e3f157c7d1ad server_cookie=63681235fd6893d2 in_seq=0 out_seq=0 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.070+0000 7efdaffff640 1 -- 172.21.15.94:0/76610871 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b224e8f990 con 0x55b224b39f60 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.070+0000 7efdaffff640 10 client.32383 handle_command_reply: tid=0 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.070+0000 7efdddc12740 1 client.32383 shutdown 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.070+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 msgr2=0x55b224f26ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.070+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdd019b2c0 tx=0x7efdbc00f790 comp rx=0 tx=0).stop 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 shutdown_connections 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b39f60 0x55b224f34290 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f26ed0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 --2- 172.21.15.94:0/76610871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f26990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdaffff640 0 client.32383 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 >> 172.21.15.94:0/76610871 conn(0x55b224b6f4e0 msgr2=0x55b224f1b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 shutdown_connections 2024-04-25T14:03:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 1 -- 172.21.15.94:0/76610871 wait complete. 2024-04-25T14:03:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.071+0000 7efdddc12740 20 client.32383 trim_cache size 0 max 16384 2024-04-25T14:03:43.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 Processor -- start 2024-04-25T14:03:43.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 -- start start 2024-04-25T14:03:43.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f2dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01ab510 con 0x55b224b524d0 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01ad050 con 0x55b224b7b390 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdddc12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01abbe0 con 0x55b224b5a6b0 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdd4b00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55290/0 (socket says 172.21.15.94:55290) 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdd4b00640 1 -- 172.21.15.94:0/2239241211 learned_addr learned my addr 172.21.15.94:0/2239241211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdcffff640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f2dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.091+0000 7efdd5301640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 -- 172.21.15.94:0/2239241211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 msgr2=0x55b224f2dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f2dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 -- 172.21.15.94:0/2239241211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 msgr2=0x55b224f2e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 -- 172.21.15.94:0/2239241211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b224b530f0 con 0x55b224b524d0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd5301640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdd4b00640 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7efdc40fcd80 tx=0x7efdc4102a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7af7547e1bf8f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdba7fc640 1 -- 172.21.15.94:0/2239241211 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4049040 con 0x55b224b524d0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdba7fc640 1 -- 172.21.15.94:0/2239241211 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc40c6fb0 con 0x55b224b524d0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.092+0000 7efdba7fc640 1 -- 172.21.15.94:0/2239241211 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4003d30 con 0x55b224b524d0 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- 172.21.15.94:0/2239241211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 msgr2=0x55b224f25680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7efdc40fcd80 tx=0x7efdc4102a90 comp rx=0 tx=0).stop 2024-04-25T14:03:43.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- 172.21.15.94:0/2239241211 shutdown_connections 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b7b390 0x55b224f2e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b5a6b0 0x55b224f2dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 --2- 172.21.15.94:0/2239241211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b524d0 0x55b224f25680 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- 172.21.15.94:0/2239241211 >> 172.21.15.94:0/2239241211 conn(0x55b224b6f4e0 msgr2=0x55b224f24f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- 172.21.15.94:0/2239241211 shutdown_connections 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- 172.21.15.94:0/2239241211 wait complete. 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 -- start start 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.093+0000 7efdddc12740 1 Processor -- start 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b524d0 0x55b224f2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b7b390 0x55b224f9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01ac980 con 0x55b224b5a6b0 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01ac2b0 con 0x55b224b524d0 2024-04-25T14:03:43.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01a94f0 con 0x55b224b7b390 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdddc12740 10 client.0 _ll_register_callbacks cb 0x7efdbc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55306/0 (socket says 172.21.15.94:55306) 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 -- 172.21.15.94:0/913129350 learned_addr learned my addr 172.21.15.94:0/913129350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdd4b00640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b524d0 0x55b224f2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdd5301640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b7b390 0x55b224f9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 -- 172.21.15.94:0/913129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b7b390 msgr2=0x55b224f9a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b7b390 0x55b224f9a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 -- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b524d0 msgr2=0x55b224f2b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b524d0 0x55b224f2b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 -- 172.21.15.94:0/913129350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b224e424b0 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdcffff640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x55b224f13c00 tx=0x7efdbc0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41103d0c1df012ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdaffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc005110 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.094+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efdbc0052b0 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc40c6fb0 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0054b0 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 10 client.23355 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b224f9b970 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc40c6fb0 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efdbc01c420 con 0x55b224b5a6b0 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 10 client.23355 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 10 client.23355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 20 client.23355 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b0f3a0 0x55b224f9e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 4 client.23355 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b224e89800 con 0x55b224b0f3a0 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.095+0000 7efdd4b00640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b0f3a0 0x55b224f9e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.096+0000 7efdaffff640 10 client.23355 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:43.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.096+0000 7efdd4b00640 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b0f3a0 0x55b224f9e3e0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5143fe7d48e322bc server_cookie=13a60a39a53ab6ed in_seq=0 out_seq=0 2024-04-25T14:03:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.101+0000 7efdaffff640 1 -- 172.21.15.94:0/913129350 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b224e89800 con 0x55b224b0f3a0 2024-04-25T14:03:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.101+0000 7efdaffff640 10 client.23355 handle_command_reply: tid=0 2024-04-25T14:03:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 client.23355 shutdown 2024-04-25T14:03:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 msgr2=0x55b224f2b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x55b224f13c00 tx=0x7efdbc0106e0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 shutdown_connections 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b224b0f3a0 0x55b224f9e3e0 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b224b7b390 0x55b224f9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b224b5a6b0 0x55b224f2b800 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdaffff640 0 client.23355 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 --2- 172.21.15.94:0/913129350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b224b524d0 0x55b224f2b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 >> 172.21.15.94:0/913129350 conn(0x55b224b6f4e0 msgr2=0x55b224f17c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 shutdown_connections 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 1 -- 172.21.15.94:0/913129350 wait complete. 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.102+0000 7efdddc12740 20 client.23355 trim_cache size 0 max 16384 2024-04-25T14:03:43.108 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (122 inodes in the stack)", 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "c910b7ac-5719-4ed3-86c9-ff4fa7034880": { 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "c910b7ac-5719-4ed3-86c9-ff4fa7034880", 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:03:43.109 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 msgr2=0x7efdc00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 0x7efdc00b2080 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efdc8020000 tx=0x7efdc8023000 comp rx=0 tx=0).stop 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 msgr2=0x7efdd019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006010 tx=0x7efdc4004270 comp rx=0 tx=0).stop 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 shutdown_connections 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efdc00afb70 0x7efdc00b2080 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd01080b0 0x7efdd01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdd01033b0 0x7efdd019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 --2- 172.21.15.94:0/3268215114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdd0102170 0x7efdd019a220 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 >> 172.21.15.94:0/3268215114 conn(0x7efdd00fd340 msgr2=0x7efdd0104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 shutdown_connections 2024-04-25T14:03:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:03:43.103+0000 7efdd6d64640 1 -- 172.21.15.94:0/3268215114 wait complete. 2024-04-25T14:03:43.114 INFO:tasks.cephfs.filesystem:scrub status for tag:c910b7ac-5719-4ed3-86c9-ff4fa7034880 - {'path': '/', 'tag': 'c910b7ac-5719-4ed3-86c9-ff4fa7034880', 'options': 'recursive,force'} 2024-04-25T14:03:43.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 179409 18.87 MB/sec execute 385 sec latency 87.881 ms 2024-04-25T14:03:44.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: pgmap v1009: 97 pgs: 97 active+clean; 267 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:44.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:43 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 179763 18.88 MB/sec execute 386 sec latency 97.249 ms 2024-04-25T14:03:44.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: pgmap v1009: 97 pgs: 97 active+clean; 267 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:44.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:43 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: pgmap v1009: 97 pgs: 97 active+clean; 267 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:44.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:44.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:43 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:03:45.143 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:44 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 180274 18.89 MB/sec execute 387 sec latency 107.801 ms 2024-04-25T14:03:45.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:03:45.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:45.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:45.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:45.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:03:45.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:44 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:03:45.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:03:45.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:44 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:46.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:45 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:46.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:45 smithi137 ceph-mon[32568]: pgmap v1010: 97 pgs: 97 active+clean; 273 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 580 op/s 2024-04-25T14:03:46.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 180685 18.90 MB/sec execute 388 sec latency 81.118 ms 2024-04-25T14:03:46.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:45 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:46.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:45 smithi094 ceph-mon[29195]: pgmap v1010: 97 pgs: 97 active+clean; 273 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 580 op/s 2024-04-25T14:03:46.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:45 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:46.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:45 smithi138 ceph-mon[32977]: pgmap v1010: 97 pgs: 97 active+clean; 273 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 580 op/s 2024-04-25T14:03:47.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 180960 18.90 MB/sec execute 389 sec latency 379.268 ms 2024-04-25T14:03:48.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 181396 18.91 MB/sec execute 390 sec latency 94.571 ms 2024-04-25T14:03:48.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:47 smithi094 ceph-mon[29195]: pgmap v1011: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-04-25T14:03:48.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:47 smithi138 ceph-mon[32977]: pgmap v1011: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-04-25T14:03:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:47 smithi137 ceph-mon[32568]: pgmap v1011: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-04-25T14:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.425+0000 7f6e3affd640 1 -- 172.21.15.94:0/3178949514 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3402f310 con 0x7f6e3c1120c0 2024-04-25T14:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 -- 172.21.15.94:0/3178949514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 msgr2=0x7f6e3c114560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 --2- 172.21.15.94:0/3178949514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c114560 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009ca0 tx=0x7f6e3402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 -- 172.21.15.94:0/3178949514 shutdown_connections 2024-04-25T14:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 --2- 172.21.15.94:0/3178949514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c114560 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 --2- 172.21.15.94:0/3178949514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 --2- 172.21.15.94:0/3178949514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.426+0000 7f6e42934640 1 -- 172.21.15.94:0/3178949514 >> 172.21.15.94:0/3178949514 conn(0x7f6e3c070540 msgr2=0x7f6e3c0729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.427+0000 7f6e42934640 1 -- 172.21.15.94:0/3178949514 shutdown_connections 2024-04-25T14:03:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.427+0000 7f6e42934640 1 -- 172.21.15.94:0/3178949514 wait complete. 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.427+0000 7f6e42934640 1 Processor -- start 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.427+0000 7f6e42934640 1 -- start start 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c11bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c121020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c115d40 con 0x7f6e3c10f760 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c115bc0 con 0x7f6e3c109fc0 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e42934640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c115ec0 con 0x7f6e3c1120c0 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58352/0 (socket says 172.21.15.94:58352) 2024-04-25T14:03:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/1075670969 learned_addr learned my addr 172.21.15.94:0/1075670969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c121020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c11bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.428+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1075670969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 msgr2=0x7f6e3c121020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c121020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 msgr2=0x7f6e3c11c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1075670969 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c121690 con 0x7f6e3c109fc0 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c11bb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9c0 tx=0x7f6e2c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e397fa640 1 -- 172.21.15.94:0/1075670969 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00ccc0 con 0x7f6e3c109fc0 2024-04-25T14:03:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c121980 con 0x7f6e3c109fc0 2024-04-25T14:03:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.429+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1b9de0 con 0x7f6e3c109fc0 2024-04-25T14:03:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.430+0000 7f6e397fa640 1 -- 172.21.15.94:0/1075670969 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c0108e0 con 0x7f6e3c109fc0 2024-04-25T14:03:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.430+0000 7f6e397fa640 1 -- 172.21.15.94:0/1075670969 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0187c0 con 0x7f6e3c109fc0 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 Processor -- start 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 -- start start 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1089d1b0 0x555a10c3d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c3def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555a10ba3460 con 0x555a1087c4d0 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a10b5d590 con 0x555a108742f0 2024-04-25T14:03:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e497e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a10b9f420 con 0x555a1089d1b0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58358/0 (socket says 172.21.15.94:58358) 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/3316588610 learned_addr learned my addr 172.21.15.94:0/3316588610 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.432+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1089d1b0 0x555a10c3d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e397fa640 1 -- 172.21.15.94:0/1075670969 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f6e2c010430 con 0x7f6e3c109fc0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/3316588610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1089d1b0 msgr2=0x555a10c3d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1089d1b0 0x555a10c3d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/3316588610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 msgr2=0x555a10c3def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c3def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/3316588610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a10b6a0d0 con 0x555a1087c4d0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f6e240115f0 tx=0x7f6e24011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4577bdd3dac1677d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e327fc640 1 -- 172.21.15.94:0/3316588610 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401e070 con 0x555a1087c4d0 2024-04-25T14:03:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e327fc640 1 -- 172.21.15.94:0/3316588610 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2400f660 con 0x555a1087c4d0 2024-04-25T14:03:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.433+0000 7f6e327fc640 1 -- 172.21.15.94:0/3316588610 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003850 con 0x555a1087c4d0 2024-04-25T14:03:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.434+0000 7f6e397fa640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 0x7f6e280b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.434+0000 7f6e397fa640 1 -- 172.21.15.94:0/1075670969 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6e2c0def50 con 0x7f6e3c109fc0 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.434+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 0x7f6e280b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.434+0000 7f6e497e6740 1 -- 172.21.15.94:0/3316588610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 msgr2=0x555a10c44df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.434+0000 7f6e497e6740 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f6e240115f0 tx=0x7f6e24011ab0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 -- 172.21.15.94:0/3316588610 shutdown_connections 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1087c4d0 0x555a10c44df0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c3def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 --2- 172.21.15.94:0/3316588610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1089d1b0 0x555a10c3d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 -- 172.21.15.94:0/3316588610 >> 172.21.15.94:0/3316588610 conn(0x555a10891300 msgr2=0x555a10c39300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 -- 172.21.15.94:0/3316588610 shutdown_connections 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 0x7f6e280b2070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6e340094e0 tx=0x7f6e34031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.435+0000 7f6e497e6740 1 -- 172.21.15.94:0/3316588610 wait complete. 2024-04-25T14:03:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.436+0000 7f6e497e6740 1 -- start start 2024-04-25T14:03:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.436+0000 7f6e497e6740 1 Processor -- start 2024-04-25T14:03:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.436+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c47c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555a10c45840 con 0x555a1089d1b0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555a10b2ab60 con 0x555a108742f0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e40ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e40ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58360/0 (socket says 172.21.15.94:58360) 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e40ed1640 1 -- 172.21.15.94:0/2105847028 learned_addr learned my addr 172.21.15.94:0/2105847028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c47c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a10c459c0 con 0x555a1087c4d0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e497e6740 10 client.0 _ll_register_callbacks cb 0x555a10b4f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e3bfff640 1 -- 172.21.15.94:0/2105847028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 msgr2=0x555a10c47c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.437+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c47c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e3bfff640 1 -- 172.21.15.94:0/2105847028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 msgr2=0x555a10c4eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e3bfff640 1 -- 172.21.15.94:0/2105847028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a108b5210 con 0x555a108742f0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c476e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x555a10c3ed50 tx=0x7f6e2c0a7a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8de786627eec07c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0a73e0 con 0x555a108742f0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2400f660 con 0x555a108742f0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.438+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6e2c0a2f60 con 0x555a108742f0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.439+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c0ab6e0 con 0x555a108742f0 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.439+0000 7f6e497e6740 10 client.32747 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.439+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555a10981620 con 0x555a108742f0 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.439+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2400f660 con 0x555a108742f0 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2c0ab120 con 0x555a108742f0 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e497e6740 10 client.32747 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e497e6740 10 client.32747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e497e6740 20 client.32747 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a1085bd80 0x555a10c553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.440+0000 7f6e497e6740 4 client.32747 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.441+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a10bb1070 con 0x555a1085bd80 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.441+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a1085bd80 0x555a10c553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.441+0000 7f6e1b7fe640 10 client.32747 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.441+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a1085bd80 0x555a10c553c0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596b07a91ddd55d1 server_cookie=14ec5cbfffc9669d in_seq=0 out_seq=0 2024-04-25T14:03:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.445+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/2105847028 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a10bb1070 con 0x555a1085bd80 2024-04-25T14:03:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.445+0000 7f6e1b7fe640 10 client.32747 handle_command_reply: tid=0 2024-04-25T14:03:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.445+0000 7f6e497e6740 1 client.32747 shutdown 2024-04-25T14:03:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.446+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 msgr2=0x555a10c476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.446+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c476e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x555a10c3ed50 tx=0x7f6e2c0a7a00 comp rx=0 tx=0).stop 2024-04-25T14:03:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.447+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 shutdown_connections 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.447+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a1085bd80 0x555a10c553c0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.447+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c47c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e497e6740 1 --2- 172.21.15.94:0/2105847028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c476e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 >> 172.21.15.94:0/2105847028 conn(0x555a10891300 msgr2=0x555a10c38250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e1b7fe640 0 client.32747 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 shutdown_connections 2024-04-25T14:03:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.448+0000 7f6e497e6740 1 -- 172.21.15.94:0/2105847028 wait complete. 2024-04-25T14:03:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.449+0000 7f6e497e6740 20 client.32747 trim_cache size 0 max 16384 2024-04-25T14:03:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.471+0000 7f6e497e6740 1 Processor -- start 2024-04-25T14:03:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 -- start start 2024-04-25T14:03:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a1087c4d0 0x555a10c3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00e7a0 con 0x555a1089d1b0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51776/0 (socket says 172.21.15.94:51776) 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1661582436 learned_addr learned my addr 172.21.15.94:0/1661582436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c0102e0 con 0x555a1087c4d0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00ee70 con 0x555a108742f0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a1087c4d0 0x555a10c3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1661582436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a1087c4d0 msgr2=0x555a10c3c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a1087c4d0 0x555a10c3c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1661582436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 msgr2=0x555a10c3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.472+0000 7f6e3bfff640 1 -- 172.21.15.94:0/1661582436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a10c42500 con 0x555a108742f0 2024-04-25T14:03:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.473+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0a4650 tx=0x7f6e2c0a6c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7dbc1c2f9af84bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.473+0000 7f6e31ffb640 1 -- 172.21.15.94:0/1661582436 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0b0190 con 0x555a108742f0 2024-04-25T14:03:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.473+0000 7f6e31ffb640 1 -- 172.21.15.94:0/1661582436 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c0a2f60 con 0x555a108742f0 2024-04-25T14:03:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.473+0000 7f6e31ffb640 1 -- 172.21.15.94:0/1661582436 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0acbf0 con 0x555a108742f0 2024-04-25T14:03:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 msgr2=0x555a10c392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0a4650 tx=0x7f6e2c0a6c80 comp rx=0 tx=0).stop 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 shutdown_connections 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a1087c4d0 0x555a10c3c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 --2- 172.21.15.94:0/1661582436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a108742f0 0x555a10c392b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 >> 172.21.15.94:0/1661582436 conn(0x555a10891300 msgr2=0x555a10c38ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 shutdown_connections 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.474+0000 7f6e497e6740 1 -- 172.21.15.94:0/1661582436 wait complete. 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 -- start start 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 Processor -- start 2024-04-25T14:03:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00fc10 con 0x555a1089d1b0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00f540 con 0x555a108742f0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e497e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00c780 con 0x555a1087c4d0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.475+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e40ed1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51784/0 (socket says 172.21.15.94:51784) 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/4217011591 learned_addr learned my addr 172.21.15.94:0/4217011591 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e497e6740 10 client.0 _ll_register_callbacks cb 0x7f6e2c0a2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 msgr2=0x555a10c4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c4b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/4217011591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 msgr2=0x555a10c4c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 -- 172.21.15.94:0/4217011591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a10c4cd90 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e40ed1640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e3b7fe640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x555a10c3a090 tx=0x7f6e2400f6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef3c9b752b4fcc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401e070 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f6e2400fd20 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c0a2f60 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e240050a0 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e497e6740 10 client.32407 fetch_fsmap learned FSMap version 39 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.476+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555a10c4e0a0 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c0a2f60 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6e240039e0 con 0x555a1087c4d0 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 10 client.32407 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:03:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 10 client.32407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 20 client.32407 populate_metadata read hostname 'smithi094' 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a108311c0 0x555a10cbf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 4 client.32407 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a10baaee0 con 0x555a108311c0 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.477+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a108311c0 0x555a10cbf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.478+0000 7f6e1b7fe640 10 client.32407 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:48.478+0000 7f6e3bfff640 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a108311c0 0x555a10cbf940 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1de90b9ae94d92 server_cookie=5816c0e81b4bdc04 in_seq=0 out_seq=0 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e1b7fe640 1 -- 172.21.15.94:0/4217011591 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a10baaee0 con 0x555a108311c0 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e1b7fe640 10 client.32407 handle_command_reply: tid=0 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 client.32407 shutdown 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 msgr2=0x555a10c4b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x555a10c3a090 tx=0x7f6e2400f6f0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 shutdown_connections 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x555a108311c0 0x555a10cbf940 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a1089d1b0 0x555a10c4c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1087c4d0 0x555a10c4b8a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e1b7fe640 0 client.32407 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 --2- 172.21.15.94:0/4217011591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a108742f0 0x555a10c4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 >> 172.21.15.94:0/4217011591 conn(0x555a10891300 msgr2=0x555a10c37820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 shutdown_connections 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 1 -- 172.21.15.94:0/4217011591 wait complete. 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.119+0000 7f6e497e6740 20 client.32407 trim_cache size 0 max 16384 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:03:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:03:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:03:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:03:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 msgr2=0x7f6e280b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 0x7f6e280b2070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6e340094e0 tx=0x7f6e34031af0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 msgr2=0x7f6e3c11bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c11bb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00e9c0 tx=0x7f6e2c00ee80 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 shutdown_connections 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f6e280afb60 0x7f6e280b2070 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e3c1120c0 0x7f6e3c121020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e3c10f760 0x7f6e3c11c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 --2- 172.21.15.94:0/1075670969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e3c109fc0 0x7f6e3c11bb60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 >> 172.21.15.94:0/1075670969 conn(0x7f6e3c070540 msgr2=0x7f6e3c1105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 shutdown_connections 2024-04-25T14:03:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:03:49.120+0000 7f6e42934640 1 -- 172.21.15.94:0/1075670969 wait complete. 2024-04-25T14:03:49.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 181559 18.90 MB/sec execute 391 sec latency 446.969 ms 2024-04-25T14:03:50.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 181875 18.90 MB/sec execute 392 sec latency 116.391 ms 2024-04-25T14:03:50.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:49 smithi094 ceph-mon[29195]: pgmap v1012: 97 pgs: 97 active+clean; 273 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-04-25T14:03:50.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:49 smithi138 ceph-mon[32977]: pgmap v1012: 97 pgs: 97 active+clean; 273 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-04-25T14:03:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:49 smithi137 ceph-mon[32568]: pgmap v1012: 97 pgs: 97 active+clean; 273 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-04-25T14:03:51.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 182193 18.89 MB/sec execute 393 sec latency 74.772 ms 2024-04-25T14:03:51.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:50 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:51.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:50 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:51.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:50 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:52.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 182425 18.87 MB/sec execute 394 sec latency 195.520 ms 2024-04-25T14:03:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:51 smithi094 ceph-mon[29195]: pgmap v1013: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-04-25T14:03:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:51 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:03:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:51 smithi138 ceph-mon[32977]: pgmap v1013: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-04-25T14:03:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:51 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:03:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:51 smithi137 ceph-mon[32568]: pgmap v1013: 97 pgs: 97 active+clean; 274 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2024-04-25T14:03:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:51 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:03:53.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 182738 18.86 MB/sec execute 395 sec latency 125.701 ms 2024-04-25T14:03:54.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 183175 18.87 MB/sec execute 396 sec latency 76.291 ms 2024-04-25T14:03:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:53 smithi094 ceph-mon[29195]: pgmap v1014: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:03:54.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:53 smithi138 ceph-mon[32977]: pgmap v1014: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:03:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:53 smithi137 ceph-mon[32568]: pgmap v1014: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 515 op/s 2024-04-25T14:03:55.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 183711 18.88 MB/sec execute 397 sec latency 69.014 ms 2024-04-25T14:03:56.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 184124 18.89 MB/sec execute 398 sec latency 80.663 ms 2024-04-25T14:03:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:55 smithi094 ceph-mon[29195]: pgmap v1015: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-04-25T14:03:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:55 smithi138 ceph-mon[32977]: pgmap v1015: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-04-25T14:03:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:55 smithi137 ceph-mon[32568]: pgmap v1015: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-04-25T14:03:57.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 184494 18.90 MB/sec execute 399 sec latency 229.393 ms 2024-04-25T14:03:58.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 184953 18.92 MB/sec execute 400 sec latency 200.875 ms 2024-04-25T14:03:58.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:57 smithi094 ceph-mon[29195]: pgmap v1016: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-04-25T14:03:58.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:57 smithi138 ceph-mon[32977]: pgmap v1016: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-04-25T14:03:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:57 smithi137 ceph-mon[32568]: pgmap v1016: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-04-25T14:03:59.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 185270 18.92 MB/sec execute 401 sec latency 74.274 ms 2024-04-25T14:04:00.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 185631 18.93 MB/sec execute 402 sec latency 72.436 ms 2024-04-25T14:04:00.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:03:59 smithi094 ceph-mon[29195]: pgmap v1017: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-04-25T14:04:00.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:03:59 smithi138 ceph-mon[32977]: pgmap v1017: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-04-25T14:04:00.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:03:59 smithi137 ceph-mon[32568]: pgmap v1017: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 508 op/s 2024-04-25T14:04:01.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 185953 18.93 MB/sec execute 403 sec latency 118.433 ms 2024-04-25T14:04:02.156 INFO:tasks.workunit.client.0.smithi094.stdout: 10 186195 18.91 MB/sec execute 404 sec latency 234.355 ms 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.224+0000 7fa9bf496640 1 -- 172.21.15.94:0/179740044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8102a30 msgr2=0x7fa9b8103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.224+0000 7fa9bf496640 1 --2- 172.21.15.94:0/179740044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8102a30 0x7fa9b8103730 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009ca0 tx=0x7fa9b002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.224+0000 7fa9bf496640 1 -- 172.21.15.94:0/179740044 shutdown_connections 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.224+0000 7fa9bf496640 1 --2- 172.21.15.94:0/179740044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8108970 0x7fa9b810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 --2- 172.21.15.94:0/179740044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8103c70 0x7fa9b8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 --2- 172.21.15.94:0/179740044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8102a30 0x7fa9b8103730 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 -- 172.21.15.94:0/179740044 >> 172.21.15.94:0/179740044 conn(0x7fa9b80fdc20 msgr2=0x7fa9b81000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 -- 172.21.15.94:0/179740044 shutdown_connections 2024-04-25T14:04:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 -- 172.21.15.94:0/179740044 wait complete. 2024-04-25T14:04:02.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:01 smithi094 ceph-mon[29195]: pgmap v1018: 97 pgs: 97 active+clean; 272 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-04-25T14:04:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 Processor -- start 2024-04-25T14:04:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 -- start start 2024-04-25T14:04:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8102a30 0x7fa9b8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.225+0000 7fa9bf496640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bf496640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 0x7fa9b81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bf496640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810e510 con 0x7fa9b8103c70 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bf496640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810e390 con 0x7fa9b8108970 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bf496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b810e690 con 0x7fa9b8102a30 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bda33640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 0x7fa9b81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60934/0 (socket says 172.21.15.94:60934) 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 -- 172.21.15.94:0/346720913 learned_addr learned my addr 172.21.15.94:0/346720913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bd232640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8102a30 0x7fa9b8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 -- 172.21.15.94:0/346720913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8102a30 msgr2=0x7fa9b8075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8102a30 0x7fa9b8075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 -- 172.21.15.94:0/346720913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 msgr2=0x7fa9b81a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 0x7fa9b81a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 -- 172.21.15.94:0/346720913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b81a1d60 con 0x7fa9b8103c70 2024-04-25T14:04:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bda33640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 0x7fa9b81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bca31640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800e9f0 tx=0x7fa9a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9ae7fc640 1 -- 172.21.15.94:0/346720913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800cd80 con 0x7fa9b8103c70 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9ae7fc640 1 -- 172.21.15.94:0/346720913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a80108e0 con 0x7fa9b8103c70 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b81a1ff0 con 0x7fa9b8103c70 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.226+0000 7fa9ae7fc640 1 -- 172.21.15.94:0/346720913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8018980 con 0x7fa9b8103c70 2024-04-25T14:04:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.227+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81a2500 con 0x7fa9b8103c70 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.229+0000 7fa9c6429740 1 Processor -- start 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.229+0000 7fa9c6429740 1 -- start start 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.229+0000 7fa9ae7fc640 1 -- 172.21.15.94:0/346720913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa9a8010430 con 0x7fa9b8103c70 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea0c6460 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea080590 con 0x55a7e9d972f0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9c6429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea0c2420 con 0x55a7e9dc01b0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60948/0 (socket says 172.21.15.94:60948) 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 -- 172.21.15.94:0/2699105652 learned_addr learned my addr 172.21.15.94:0/2699105652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bd232640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bda33640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 -- 172.21.15.94:0/2699105652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 msgr2=0x55a7ea160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 -- 172.21.15.94:0/2699105652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 msgr2=0x55a7ea160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 -- 172.21.15.94:0/2699105652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ea08d0d0 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bd232640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9ae7fc640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 0x7fa9a40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bda33640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 0x7fa9a40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9bca31640 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80e2fd0 tx=0x7fa9a80e75f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4fc37f801d7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.230+0000 7fa9ae7fc640 1 -- 172.21.15.94:0/346720913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9a80def40 con 0x7fa9b8103c70 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9a37fe640 1 -- 172.21.15.94:0/2699105652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80a32f0 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9a37fe640 1 -- 172.21.15.94:0/2699105652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a8010720 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9a37fe640 1 -- 172.21.15.94:0/2699105652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80a6ba0 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 -- 172.21.15.94:0/2699105652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 msgr2=0x55a7ea167df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9bda33640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 0x7fa9a40b2070 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55a7ea161b40 tx=0x7fa9b4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80e2fd0 tx=0x7fa9a80e75f0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 -- 172.21.15.94:0/2699105652 shutdown_connections 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea167df0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.231+0000 7fa9c6429740 1 --2- 172.21.15.94:0/2699105652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- 172.21.15.94:0/2699105652 >> 172.21.15.94:0/2699105652 conn(0x55a7e9db4300 msgr2=0x55a7ea15c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- 172.21.15.94:0/2699105652 shutdown_connections 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- 172.21.15.94:0/2699105652 wait complete. 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- start start 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 Processor -- start 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea16a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea168840 con 0x55a7e9dc01b0 2024-04-25T14:04:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea04db60 con 0x55a7e9d972f0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ea1689c0 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.232+0000 7fa9c6429740 10 client.0 _ll_register_callbacks cb 0x55a7ea0722b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bd232640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bca31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60152/0 (socket says 172.21.15.94:60152) 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bca31640 1 -- 172.21.15.94:0/562301436 learned_addr learned my addr 172.21.15.94:0/562301436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60952/0 (socket says 172.21.15.94:60952) 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 -- 172.21.15.94:0/562301436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 msgr2=0x55a7ea16abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 -- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 msgr2=0x55a7ea16a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea16a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 -- 172.21.15.94:0/562301436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e9dd8210 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9bda33640 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ed42c9bdc62bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b402c060 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9b4016030 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7ea0d4070 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9b401cd60 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9c6429740 10 client.23385 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.233+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a7e9ea4620 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a7ea0d4070 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b40278e0 con 0x55a7e9dc01b0 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 10 client.23385 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 10 client.23385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 20 client.23385 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d7ed80 0x55a7ea1783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 4 client.23385 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ea0cdee0 con 0x55a7e9d7ed80 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9bd232640 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d7ed80 0x55a7ea1783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9bd232640 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d7ed80 0x55a7ea1783f0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58aea978833707e5 server_cookie=1ebc3684b24e32a0 in_seq=0 out_seq=0 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.234+0000 7fa9a0ff9640 10 client.23385 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/562301436 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7ea0cdee0 con 0x55a7e9d7ed80 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9a0ff9640 10 client.23385 handle_command_reply: tid=0 2024-04-25T14:04:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 client.23385 shutdown 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 msgr2=0x55a7ea171b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b4004730 comp rx=0 tx=0).stop 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 shutdown_connections 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d7ed80 0x55a7ea1783f0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9dc01b0 0x55a7ea171b20 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9a0ff9640 0 client.23385 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 --2- 172.21.15.94:0/562301436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d972f0 0x55a7ea16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 >> 172.21.15.94:0/562301436 conn(0x55a7e9db4300 msgr2=0x55a7ea15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 shutdown_connections 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 1 -- 172.21.15.94:0/562301436 wait complete. 2024-04-25T14:04:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.236+0000 7fa9c6429740 20 client.23385 trim_cache size 0 max 16384 2024-04-25T14:04:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:01 smithi138 ceph-mon[32977]: pgmap v1018: 97 pgs: 97 active+clean; 272 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 Processor -- start 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 -- start start 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea171a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea15c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ab590 con 0x55a7e9d972f0 2024-04-25T14:04:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ad0d0 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.251+0000 7fa9c6429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81abc60 con 0x55a7e9dc01b0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60968/0 (socket says 172.21.15.94:60968) 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 -- 172.21.15.94:0/399169801 learned_addr learned my addr 172.21.15.94:0/399169801 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bda33640 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bca31640 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea171a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 -- 172.21.15.94:0/399169801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 msgr2=0x55a7ea15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 -- 172.21.15.94:0/399169801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d9f4d0 msgr2=0x55a7ea171a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea171a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 -- 172.21.15.94:0/399169801 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ea171f80 con 0x55a7e9d972f0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9bd232640 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0035000 tx=0x7fa9b0009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7710ad8773a0420b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/399169801 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0003b70 con 0x55a7e9d972f0 2024-04-25T14:04:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/399169801 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b004e030 con 0x55a7e9d972f0 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.252+0000 7fa9a2ffd640 1 -- 172.21.15.94:0/399169801 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b002f860 con 0x55a7e9d972f0 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- 172.21.15.94:0/399169801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 msgr2=0x55a7ea15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0035000 tx=0x7fa9b0009410 comp rx=0 tx=0).stop 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- 172.21.15.94:0/399169801 shutdown_connections 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9dc01b0 0x55a7ea15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea171a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 --2- 172.21.15.94:0/399169801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea15bcc0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- 172.21.15.94:0/399169801 >> 172.21.15.94:0/399169801 conn(0x55a7e9db4300 msgr2=0x55a7ea164b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- 172.21.15.94:0/399169801 shutdown_connections 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- 172.21.15.94:0/399169801 wait complete. 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 -- start start 2024-04-25T14:04:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.253+0000 7fa9c6429740 1 Processor -- start 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9dc01b0 0x55a7ea170ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81aca00 con 0x55a7e9d972f0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81ac330 con 0x55a7e9dc01b0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b81a9570 con 0x55a7e9d9f4d0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9c6429740 10 client.0 _ll_register_callbacks cb 0x7fa9b400ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60974/0 (socket says 172.21.15.94:60974) 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 -- 172.21.15.94:0/3401445849 learned_addr learned my addr 172.21.15.94:0/3401445849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bca31640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bda33640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9dc01b0 0x55a7ea170ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 -- 172.21.15.94:0/3401445849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 msgr2=0x55a7ea16fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 -- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9dc01b0 msgr2=0x55a7ea170ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9dc01b0 0x55a7ea170ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 -- 172.21.15.94:0/3401445849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ea171160 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9bd232640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009620 tx=0x7fa9b00314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d1e0f46cd5ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.254+0000 7fa9a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0057020 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa9b002f860 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b004e030 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9b0062050 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 10 client.23391 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a7ea1e0c10 con 0x55a7e9d972f0 2024-04-25T14:04:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b004e030 con 0x55a7e9d972f0 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa9b0030c40 con 0x55a7e9d972f0 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 10 client.23391 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 10 client.23391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 20 client.23391 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d541c0 0x55a7ea1e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 4 client.23391 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.255+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7ea0495f0 con 0x55a7e9d541c0 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.256+0000 7fa9bca31640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d541c0 0x55a7ea1e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.256+0000 7fa9a0ff9640 10 client.23391 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:02.256+0000 7fa9bca31640 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d541c0 0x55a7ea1e35c0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e228574406d40e server_cookie=22d3aca3e28838c8 in_seq=0 out_seq=0 2024-04-25T14:04:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:01 smithi137 ceph-mon[32568]: pgmap v1018: 97 pgs: 97 active+clean; 272 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-04-25T14:04:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.012+0000 7fa9a0ff9640 1 -- 172.21.15.94:0/3401445849 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7ea0495f0 con 0x55a7e9d541c0 2024-04-25T14:04:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9a0ff9640 10 client.23391 handle_command_reply: tid=0 2024-04-25T14:04:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 client.23391 shutdown 2024-04-25T14:04:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 msgr2=0x55a7ea16f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009620 tx=0x7fa9b00314d0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 shutdown_connections 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7e9d541c0 0x55a7ea1e35c0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7e9dc01b0 0x55a7ea170ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7e9d9f4d0 0x55a7ea16fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 --2- 172.21.15.94:0/3401445849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7e9d972f0 0x55a7ea16f730 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9a0ff9640 0 client.23391 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 >> 172.21.15.94:0/3401445849 conn(0x55a7e9db4300 msgr2=0x55a7ea164b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 shutdown_connections 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 1 -- 172.21.15.94:0/3401445849 wait complete. 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.013+0000 7fa9c6429740 20 client.23391 trim_cache size 0 max 16384 2024-04-25T14:04:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:04:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:04:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:04:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 msgr2=0x7fa9a40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 0x7fa9a40b2070 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55a7ea161b40 tx=0x7fa9b4005f60 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 msgr2=0x7fa9b819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800e9f0 tx=0x7fa9a800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 shutdown_connections 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa9a40afb60 0x7fa9a40b2070 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9b8108970 0x7fa9b81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b8103c70 0x7fa9b819efd0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 --2- 172.21.15.94:0/346720913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9b8102a30 0x7fa9b8075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 >> 172.21.15.94:0/346720913 conn(0x7fa9b80fdc20 msgr2=0x7fa9b8104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 shutdown_connections 2024-04-25T14:04:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:03.014+0000 7fa9bf496640 1 -- 172.21.15.94:0/346720913 wait complete. 2024-04-25T14:04:03.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 186303 18.87 MB/sec execute 405 sec latency 457.366 ms 2024-04-25T14:04:04.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 186664 18.87 MB/sec execute 406 sec latency 76.259 ms 2024-04-25T14:04:04.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:03 smithi094 ceph-mon[29195]: pgmap v1019: 97 pgs: 97 active+clean; 277 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-25T14:04:04.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:03 smithi138 ceph-mon[32977]: pgmap v1019: 97 pgs: 97 active+clean; 277 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-25T14:04:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:03 smithi137 ceph-mon[32568]: pgmap v1019: 97 pgs: 97 active+clean; 277 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-25T14:04:05.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 187005 18.86 MB/sec execute 407 sec latency 131.142 ms 2024-04-25T14:04:06.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 187537 18.87 MB/sec execute 408 sec latency 100.987 ms 2024-04-25T14:04:06.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:05 smithi094 ceph-mon[29195]: pgmap v1020: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 529 op/s 2024-04-25T14:04:06.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:05 smithi138 ceph-mon[32977]: pgmap v1020: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 529 op/s 2024-04-25T14:04:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:05 smithi137 ceph-mon[32568]: pgmap v1020: 97 pgs: 97 active+clean; 269 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 529 op/s 2024-04-25T14:04:07.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 187944 18.90 MB/sec execute 409 sec latency 117.686 ms 2024-04-25T14:04:08.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 188324 18.91 MB/sec execute 410 sec latency 269.722 ms 2024-04-25T14:04:08.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:07 smithi094 ceph-mon[29195]: pgmap v1021: 97 pgs: 97 active+clean; 266 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-04-25T14:04:08.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:07 smithi138 ceph-mon[32977]: pgmap v1021: 97 pgs: 97 active+clean; 266 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-04-25T14:04:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:07 smithi137 ceph-mon[32568]: pgmap v1021: 97 pgs: 97 active+clean; 266 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-04-25T14:04:09.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 188713 18.91 MB/sec execute 411 sec latency 81.060 ms 2024-04-25T14:04:10.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 189037 18.91 MB/sec execute 412 sec latency 82.841 ms 2024-04-25T14:04:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:09 smithi094 ceph-mon[29195]: pgmap v1022: 97 pgs: 97 active+clean; 270 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 518 op/s 2024-04-25T14:04:10.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:09 smithi138 ceph-mon[32977]: pgmap v1022: 97 pgs: 97 active+clean; 270 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 518 op/s 2024-04-25T14:04:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:09 smithi137 ceph-mon[32568]: pgmap v1022: 97 pgs: 97 active+clean; 270 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 518 op/s 2024-04-25T14:04:11.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 189349 18.91 MB/sec execute 413 sec latency 121.173 ms 2024-04-25T14:04:12.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 189640 18.91 MB/sec execute 414 sec latency 113.561 ms 2024-04-25T14:04:12.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:11 smithi094 ceph-mon[29195]: pgmap v1023: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-04-25T14:04:12.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:11 smithi138 ceph-mon[32977]: pgmap v1023: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-04-25T14:04:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:11 smithi137 ceph-mon[32568]: pgmap v1023: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-04-25T14:04:13.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:13.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:13.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 189922 18.88 MB/sec execute 415 sec latency 219.393 ms 2024-04-25T14:04:13.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 -- 172.21.15.94:0/4237553476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4103c20 msgr2=0x7f87f4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 --2- 172.21.15.94:0/4237553476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4103c20 0x7f87f4104030 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009900 tx=0x7f87f002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 -- 172.21.15.94:0/4237553476 shutdown_connections 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 --2- 172.21.15.94:0/4237553476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4108920 0x7f87f410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 --2- 172.21.15.94:0/4237553476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4103c20 0x7f87f4104030 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 --2- 172.21.15.94:0/4237553476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 -- 172.21.15.94:0/4237553476 >> 172.21.15.94:0/4237553476 conn(0x7f87f40fdbd0 msgr2=0x7f87f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 -- 172.21.15.94:0/4237553476 shutdown_connections 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.253+0000 7f87fb70d640 1 -- 172.21.15.94:0/4237553476 wait complete. 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 Processor -- start 2024-04-25T14:04:13.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 -- start start 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c20 0x7f87f419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e500 con 0x7f87f4108920 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e380 con 0x7f87f4103c20 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87fb70d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410e680 con 0x7f87f41029e0 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f9caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f94a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f9caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50674/0 (socket says 172.21.15.94:50674) 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f94a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:48712/0 (socket says 172.21.15.94:48712) 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f9caa640 1 -- 172.21.15.94:0/2759187999 learned_addr learned my addr 172.21.15.94:0/2759187999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.254+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c20 0x7f87f419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 -- 172.21.15.94:0/2759187999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 msgr2=0x7f87f406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 -- 172.21.15.94:0/2759187999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c20 msgr2=0x7f87f419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c20 0x7f87f419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 -- 172.21.15.94:0/2759187999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f41a1d70 con 0x7f87f4108920 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f94a9640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87f9caa640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00eb10 tx=0x7f87ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87ea7fc640 1 -- 172.21.15.94:0/2759187999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec004b20 con 0x7f87f4108920 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87ea7fc640 1 -- 172.21.15.94:0/2759187999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec004cc0 con 0x7f87f4108920 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f41a2060 con 0x7f87f4108920 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87ea7fc640 1 -- 172.21.15.94:0/2759187999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec010670 con 0x7f87f4108920 2024-04-25T14:04:13.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.255+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f41a2560 con 0x7f87f4108920 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 Processor -- start 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87ea7fc640 1 -- 172.21.15.94:0/2759187999 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f87ec010830 con 0x7f87f4108920 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 -- start start 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 0x556281147e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281149080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5562810a8d80 con 0x556280d816b0 2024-04-25T14:04:13.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5562810628b0 con 0x556280da2390 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f88025be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5562810a4d40 con 0x556280d794d0 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87f94a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 0x556281147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87f94a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:48714/0 (socket says 172.21.15.94:48714) 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87f94a9640 1 -- 172.21.15.94:0/1859739606 learned_addr learned my addr 172.21.15.94:0/1859739606 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.258+0000 7f87ea7fc640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 0x7f87e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 -- 172.21.15.94:0/1859739606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 msgr2=0x556281147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 0x556281147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 -- 172.21.15.94:0/1859739606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 msgr2=0x556281149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 -- 172.21.15.94:0/1859739606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556281033a40 con 0x556280d794d0 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 0x556281147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:13.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87ea7fc640 1 -- 172.21.15.94:0/2759187999 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87ec0df260 con 0x7f87f4108920 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f9caa640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 0x7f87e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f94a9640 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87e40114f0 tx=0x7f87e40119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5605d183d8139d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87df7fe640 1 -- 172.21.15.94:0/1859739606 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4011310 con 0x556280d794d0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87f9caa640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 0x7f87e00b2000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0dd030 tx=0x7f87ec00cee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87df7fe640 1 -- 172.21.15.94:0/1859739606 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e40035f0 con 0x556280d794d0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.259+0000 7f87df7fe640 1 -- 172.21.15.94:0/1859739606 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e400f7a0 con 0x556280d794d0 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 -- 172.21.15.94:0/1859739606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 msgr2=0x5562811483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87e40114f0 tx=0x7f87e40119b0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 -- 172.21.15.94:0/1859739606 shutdown_connections 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x5562811483f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 --2- 172.21.15.94:0/1859739606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280da2390 0x556281147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 -- 172.21.15.94:0/1859739606 >> 172.21.15.94:0/1859739606 conn(0x556280d964e0 msgr2=0x55628113f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 -- 172.21.15.94:0/1859739606 shutdown_connections 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.260+0000 7f88025be740 1 -- 172.21.15.94:0/1859739606 wait complete. 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 -- start start 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 Processor -- start 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280da2390 0x556281159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556281149b90 con 0x556280d794d0 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55628102fd10 con 0x556280d816b0 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556281149d10 con 0x556280da2390 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.261+0000 7f88025be740 10 client.0 _ll_register_callbacks cb 0x55628105c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f94a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50692/0 (socket says 172.21.15.94:50692) 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3334405333 learned_addr learned my addr 172.21.15.94:0/3334405333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f94a9640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33584/0 (socket says 172.21.15.94:33584) 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280da2390 0x556281159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3334405333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280da2390 msgr2=0x556281159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280da2390 0x556281159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 msgr2=0x556281152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3334405333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556280e847d0 con 0x556280d794d0 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f94a9640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f87f002a930 tx=0x7f87f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f942a33129542b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0035040 con 0x556280d794d0 2024-04-25T14:04:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87f0004040 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e40035f0 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87f0054050 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f88025be740 10 client.23403 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.262+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556280dba3f0 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e40035f0 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f87f003b3d0 con 0x556280d794d0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f88025be740 10 client.23403 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f88025be740 10 client.23403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f88025be740 20 client.23403 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.263+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280d60f60 0x55628115f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.264+0000 7f88025be740 4 client.23403 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.264+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562810b6990 con 0x556280d60f60 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.264+0000 7f87f94a9640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280d60f60 0x55628115f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.264+0000 7f87dcff9640 10 client.23403 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.264+0000 7f87f94a9640 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280d60f60 0x55628115f580 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f4ce28a8434e86 server_cookie=b154188ecae3096c in_seq=0 out_seq=0 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.266+0000 7f87dcff9640 1 -- 172.21.15.94:0/3334405333 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562810b6990 con 0x556280d60f60 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.266+0000 7f87dcff9640 10 client.23403 handle_command_reply: tid=0 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.266+0000 7f88025be740 1 client.23403 shutdown 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.266+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 msgr2=0x556281151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.266+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f87f002a930 tx=0x7f87f002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 shutdown_connections 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280d60f60 0x55628115f580 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280da2390 0x556281159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f87dcff9640 0 client.23403 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d816b0 0x556281152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 --2- 172.21.15.94:0/3334405333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d794d0 0x556281151ce0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 >> 172.21.15.94:0/3334405333 conn(0x556280d964e0 msgr2=0x556281142180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 shutdown_connections 2024-04-25T14:04:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 1 -- 172.21.15.94:0/3334405333 wait complete. 2024-04-25T14:04:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.267+0000 7f88025be740 20 client.23403 trim_cache size 0 max 16384 2024-04-25T14:04:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.291+0000 7f88025be740 1 Processor -- start 2024-04-25T14:04:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.291+0000 7f88025be740 1 -- start start 2024-04-25T14:04:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.291+0000 7f88025be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.291+0000 7f88025be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d794d0 0x55628114ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f88025be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d816b0 0x556281151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f88025be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41a9360 con 0x556280d60f60 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f88025be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41aaea0 con 0x556280d794d0 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f88025be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41a9a30 con 0x556280d816b0 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50698/0 (socket says 172.21.15.94:50698) 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3610059191 learned_addr learned my addr 172.21.15.94:0/3610059191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d816b0 0x556281151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3610059191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d816b0 msgr2=0x556281151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d816b0 0x556281151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3610059191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d794d0 msgr2=0x55628114ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d794d0 0x55628114ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 -- 172.21.15.94:0/3610059191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556280d7a0f0 con 0x556280d60f60 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f87f0001eb0 tx=0x7f87f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a76e4c71d70a561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.292+0000 7f87deffd640 1 -- 172.21.15.94:0/3610059191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0035070 con 0x556280d60f60 2024-04-25T14:04:13.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.293+0000 7f87deffd640 1 -- 172.21.15.94:0/3610059191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f003be30 con 0x556280d60f60 2024-04-25T14:04:13.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.293+0000 7f87deffd640 1 -- 172.21.15.94:0/3610059191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00035f0 con 0x556280d60f60 2024-04-25T14:04:13.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- 172.21.15.94:0/3610059191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 msgr2=0x556281148460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f87f0001eb0 tx=0x7f87f0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- 172.21.15.94:0/3610059191 shutdown_connections 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d816b0 0x556281151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d794d0 0x55628114ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 --2- 172.21.15.94:0/3610059191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d60f60 0x556281148460 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- 172.21.15.94:0/3610059191 >> 172.21.15.94:0/3610059191 conn(0x556280d964e0 msgr2=0x5562811518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- 172.21.15.94:0/3610059191 shutdown_connections 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- 172.21.15.94:0/3610059191 wait complete. 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 -- start start 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.294+0000 7f88025be740 1 Processor -- start 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 0x5562811586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x556281155230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281155770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 0x5562811586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f8ca8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 0x5562811586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33604/0 (socket says 172.21.15.94:33604) 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41aa7d0 con 0x556280d816b0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f8ca8640 1 -- 172.21.15.94:0/148199656 learned_addr learned my addr 172.21.15.94:0/148199656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41aa100 con 0x556280d60f60 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41a7340 con 0x556280d794d0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x556281155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f88025be740 10 client.0 _ll_register_callbacks cb 0x7f87f0031e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 -- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 msgr2=0x5562811586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 0x5562811586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 -- 172.21.15.94:0/148199656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 msgr2=0x556281155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.295+0000 7f87f94a9640 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562810694b0 con 0x556280d794d0 2024-04-25T14:04:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f87f94a9640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x556281155230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f87e401b040 tx=0x7f87e40096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd5534cdd4ceac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f87dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4030070 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f003be30 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f87e4005020 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87e4005220 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f88025be740 10 client.32440 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.296+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5562811c6890 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f003be30 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f87e4033030 con 0x556280d794d0 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 10 client.32440 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 10 client.32440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 20 client.32440 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280da2390 0x5562811c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 4 client.32440 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562810b0800 con 0x556280da2390 2024-04-25T14:04:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280da2390 0x5562811c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.297+0000 7f87dcff9640 10 client.32440 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.298+0000 7f87f8ca8640 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280da2390 0x5562811c90c0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cf277183112771 server_cookie=47de20b870f52f26 in_seq=0 out_seq=0 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f87dcff9640 1 -- 172.21.15.94:0/148199656 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562810b0800 con 0x556280da2390 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f87dcff9640 10 client.32440 handle_command_reply: tid=0 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 client.32440 shutdown 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 msgr2=0x556281155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x556281155230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f87e401b040 tx=0x7f87e40096a0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 shutdown_connections 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x556280da2390 0x5562811c90c0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556280d816b0 0x556281155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.302+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556280d794d0 0x556281155230 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f88025be740 1 --2- 172.21.15.94:0/148199656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556280d60f60 0x5562811586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 >> 172.21.15.94:0/148199656 conn(0x556280d964e0 msgr2=0x556281150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f87dcff9640 0 client.32440 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 shutdown_connections 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f88025be740 1 -- 172.21.15.94:0/148199656 wait complete. 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.303+0000 7f88025be740 20 client.32440 trim_cache size 0 max 16384 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:04:13.303 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 msgr2=0x7f87e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 0x7f87e00b2000 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0dd030 tx=0x7f87ec00cee0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 msgr2=0x7f87f41a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00eb10 tx=0x7f87ec00c2d0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 shutdown_connections 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f87e00afaf0 0x7f87e00b2000 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f4108920 0x7f87f41a1550 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87f4103c20 0x7f87f419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 --2- 172.21.15.94:0/2759187999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87f41029e0 0x7f87f406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 >> 172.21.15.94:0/2759187999 conn(0x7f87f40fdbd0 msgr2=0x7f87f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 shutdown_connections 2024-04-25T14:04:13.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.304+0000 7f87fb70d640 1 -- 172.21.15.94:0/2759187999 wait complete. 2024-04-25T14:04:13.318 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:04:13.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 -- 172.21.15.94:0/2430294747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068103ad0 msgr2=0x7f30681047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 --2- 172.21.15.94:0/2430294747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068103ad0 0x7f30681047d0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f3064009900 tx=0x7f306402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 -- 172.21.15.94:0/2430294747 shutdown_connections 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 --2- 172.21.15.94:0/2430294747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068109a10 0x7f306810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 --2- 172.21.15.94:0/2430294747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 0x7f3068105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 --2- 172.21.15.94:0/2430294747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068103ad0 0x7f30681047d0 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 -- 172.21.15.94:0/2430294747 >> 172.21.15.94:0/2430294747 conn(0x7f30680fece0 msgr2=0x7f3068101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 -- 172.21.15.94:0/2430294747 shutdown_connections 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.490+0000 7f30702f9640 1 -- 172.21.15.94:0/2430294747 wait complete. 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 Processor -- start 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 -- start start 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068103ad0 0x7f306819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 0x7f306819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f306810f5d0 con 0x7f3068109a10 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f306810f450 con 0x7f3068103ad0 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f30702f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f306810f750 con 0x7f3068104d10 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f306e896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f306e896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43410/0 (socket says 172.21.15.94:43410) 2024-04-25T14:04:13.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.491+0000 7f306e095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068103ad0 0x7f306819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 -- 172.21.15.94:0/100060941 learned_addr learned my addr 172.21.15.94:0/100060941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306d894640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 0x7f306819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 -- 172.21.15.94:0/100060941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 msgr2=0x7f306819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 0x7f306819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 -- 172.21.15.94:0/100060941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068103ad0 msgr2=0x7f306819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068103ad0 0x7f306819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 -- 172.21.15.94:0/100060941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30681a1d40 con 0x7f3068109a10 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f306e896640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f306000eb10 tx=0x7f306000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004b20 con 0x7f3068109a10 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3060004cc0 con 0x7f3068109a10 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30681a1fd0 con 0x7f3068109a10 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060010670 con 0x7f3068109a10 2024-04-25T14:04:13.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.492+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681a2440 con 0x7f3068109a10 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.493+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3068108820 con 0x7f3068109a10 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.495+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3060005000 con 0x7f3068109a10 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.495+0000 7f305f7fe640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 0x7f30540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.496+0000 7f306e095640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 0x7f30540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.496+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f30600df430 con 0x7f3068109a10 2024-04-25T14:04:13.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.496+0000 7f306e095640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 0x7f30540b2000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3064005ab0 tx=0x7f3064031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.500+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3060053ff0 con 0x7f3068109a10 2024-04-25T14:04:13.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.735+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f306802db50 con 0x7f3068109a10 2024-04-25T14:04:13.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.736+0000 7f305f7fe640 1 -- 172.21.15.94:0/100060941 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f30600a7f70 con 0x7f3068109a10 2024-04-25T14:04:13.735 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:04:13.736 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:04:13.736 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 msgr2=0x7f30540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 0x7f30540b2000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3064005ab0 tx=0x7f3064031400 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 msgr2=0x7f30681a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f306000eb10 tx=0x7f306000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 shutdown_connections 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30540afaf0 0x7f30540b2000 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3068109a10 0x7f30681a16d0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3068104d10 0x7f306819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 --2- 172.21.15.94:0/100060941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068103ad0 0x7f306819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 >> 172.21.15.94:0/100060941 conn(0x7f30680fece0 msgr2=0x7f30681059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 shutdown_connections 2024-04-25T14:04:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.738+0000 7f30702f9640 1 -- 172.21.15.94:0/100060941 wait complete. 2024-04-25T14:04:13.750 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:04:13.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 -- 172.21.15.94:0/560660272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 msgr2=0x7f11f410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 --2- 172.21.15.94:0/560660272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f410ad40 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 -- 172.21.15.94:0/560660272 shutdown_connections 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 --2- 172.21.15.94:0/560660272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f410ad40 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 --2- 172.21.15.94:0/560660272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f4103c20 0x7f11f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 --2- 172.21.15.94:0/560660272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f41029e0 0x7f11f41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 -- 172.21.15.94:0/560660272 >> 172.21.15.94:0/560660272 conn(0x7f11f40fdbd0 msgr2=0x7f11f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 -- 172.21.15.94:0/560660272 shutdown_connections 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.868+0000 7f11fa253640 1 -- 172.21.15.94:0/560660272 wait complete. 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 Processor -- start 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 -- start start 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f41029e0 0x7f11f41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f4103c20 0x7f11f4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410e4f0 con 0x7f11f4108920 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410e370 con 0x7f11f41029e0 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11fa253640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11f410e670 con 0x7f11f4103c20 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43422/0 (socket says 172.21.15.94:43422) 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11f3fff640 1 -- 172.21.15.94:0/1931764320 learned_addr learned my addr 172.21.15.94:0/1931764320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f4103c20 0x7f11f4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.869+0000 7f11f37fe640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f41029e0 0x7f11f41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 -- 172.21.15.94:0/1931764320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f4103c20 msgr2=0x7f11f4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f4103c20 0x7f11f4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 -- 172.21.15.94:0/1931764320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f41029e0 msgr2=0x7f11f41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f41029e0 0x7f11f41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 -- 172.21.15.94:0/1931764320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41a0e80 con 0x7f11f4108920 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f3fff640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f0ff9640 1 -- 172.21.15.94:0/1931764320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e403f030 con 0x7f11f4108920 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41a1110 con 0x7f11f4108920 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f0ff9640 1 -- 172.21.15.94:0/1931764320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e4031910 con 0x7f11f4108920 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11f0ff9640 1 -- 172.21.15.94:0/1931764320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4035070 con 0x7f11f4108920 2024-04-25T14:04:13.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.870+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41a1650 con 0x7f11f4108920 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.872+0000 7f11f0ff9640 1 -- 172.21.15.94:0/1931764320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f11e4042030 con 0x7f11f4108920 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 Processor -- start 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 -- start start 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211351810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211352a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e2112b5d80 con 0x55e210f864d0 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e21126fa20 con 0x55e210faf390 2024-04-25T14:04:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f12011df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2112b1d40 con 0x55e210f8e6b0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43430/0 (socket says 172.21.15.94:43430) 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 -- 172.21.15.94:0/2123134601 learned_addr learned my addr 172.21.15.94:0/2123134601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211351810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211352a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f0ff9640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 0x7f11e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 -- 172.21.15.94:0/2123134601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 msgr2=0x55e211352a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211352a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 -- 172.21.15.94:0/2123134601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 msgr2=0x55e211351810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211351810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 -- 172.21.15.94:0/2123134601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21127ac30 con 0x55e210f864d0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f3fff640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 0x7f11e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f11ec007f20 tx=0x7f11ec00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adfafb17b942fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.873+0000 7f11f0ff9640 1 -- 172.21.15.94:0/1931764320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f11e40ff4c0 con 0x7f11f4108920 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.874+0000 7f11e9ffb640 1 -- 172.21.15.94:0/2123134601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003b70 con 0x55e210f864d0 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.874+0000 7f11e9ffb640 1 -- 172.21.15.94:0/2123134601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec003d10 con 0x55e210f864d0 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.874+0000 7f11e9ffb640 1 -- 172.21.15.94:0/2123134601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec004520 con 0x55e210f864d0 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.874+0000 7f12011df740 1 -- 172.21.15.94:0/2123134601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 msgr2=0x55e211351d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.874+0000 7f11f3fff640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 0x7f11e00b2000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f11e40fd290 tx=0x7f11e40f6050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f11ec007f20 tx=0x7f11ec00f850 comp rx=0 tx=0).stop 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 -- 172.21.15.94:0/2123134601 shutdown_connections 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211352a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211351d90 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- 172.21.15.94:0/2123134601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211351810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 -- 172.21.15.94:0/2123134601 >> 172.21.15.94:0/2123134601 conn(0x55e210fa34e0 msgr2=0x55e21134d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 -- 172.21.15.94:0/2123134601 shutdown_connections 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 -- 172.21.15.94:0/2123134601 wait complete. 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 -- start start 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 Processor -- start 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210f8e6b0 0x55e21135fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.875+0000 7f12011df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210faf390 0x55e211366dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e211353530 con 0x55e210f864d0 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e21123cdc0 con 0x55e210f8e6b0 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2113536b0 con 0x55e210faf390 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 10 client.0 _ll_register_callbacks cb 0x55e211258600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43438/0 (socket says 172.21.15.94:43438) 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 -- 172.21.15.94:0/1542744485 learned_addr learned my addr 172.21.15.94:0/1542744485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f37fe640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210f8e6b0 0x55e21135fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210faf390 0x55e211366dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 -- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210faf390 msgr2=0x55e211366dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210faf390 0x55e211366dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 -- 172.21.15.94:0/1542744485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210f8e6b0 msgr2=0x55e21135fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210f8e6b0 0x55e21135fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 -- 172.21.15.94:0/1542744485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e210fc73f0 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009680 tx=0x7f11dc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7123594fcb2292b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0120f0 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f11dc012290 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ec003d10 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f11dc012990 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 10 client.23427 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.876+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e2110917d0 con 0x55e210f864d0 2024-04-25T14:04:13.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec003d10 con 0x55e210f864d0 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f11dc026070 con 0x55e210f864d0 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 10 client.23427 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 10 client.23427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 20 client.23427 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f6df60 0x55e21136d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 4 client.23427 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2112c3990 con 0x55e210f6df60 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.877+0000 7f11f37fe640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f6df60 0x55e21136d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.878+0000 7f11f37fe640 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f6df60 0x55e21136d690 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c5980d19ecc2871 server_cookie=39f873cd651460aa in_seq=0 out_seq=0 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.878+0000 7f11d2ffd640 10 client.23427 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.878+0000 7f11d2ffd640 1 -- 172.21.15.94:0/1542744485 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2112c3990 con 0x55e210f6df60 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.878+0000 7f11d2ffd640 10 client.23427 handle_command_reply: tid=0 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.878+0000 7f12011df740 1 client.23427 shutdown 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 msgr2=0x55e21135f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009680 tx=0x7f11dc009ac0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 shutdown_connections 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f6df60 0x55e21136d690 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210faf390 0x55e211366dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210f8e6b0 0x55e21135fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 --2- 172.21.15.94:0/1542744485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21135f930 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f11d2ffd640 0 client.23427 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 >> 172.21.15.94:0/1542744485 conn(0x55e210fa34e0 msgr2=0x55e21134ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 shutdown_connections 2024-04-25T14:04:13.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 1 -- 172.21.15.94:0/1542744485 wait complete. 2024-04-25T14:04:13.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.879+0000 7f12011df740 20 client.23427 trim_cache size 0 max 16384 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 Processor -- start 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 -- start start 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e21134cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e21135f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41a9240 con 0x55e210f864d0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41aad80 con 0x55e210faf390 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f12011df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41a9910 con 0x55e210f8e6b0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43442/0 (socket says 172.21.15.94:43442) 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f2ffd640 1 -- 172.21.15.94:0/829400370 learned_addr learned my addr 172.21.15.94:0/829400370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e21135f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f37fe640 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e21134cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.894+0000 7f11f2ffd640 1 -- 172.21.15.94:0/829400370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 msgr2=0x55e21134cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e21134cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11f2ffd640 1 -- 172.21.15.94:0/829400370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 msgr2=0x55e21135f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e21135f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11f2ffd640 1 -- 172.21.15.94:0/829400370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21135b240 con 0x55e210f864d0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5092847cc4279c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11e97fa640 1 -- 172.21.15.94:0/829400370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc02c070 con 0x55e210f864d0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11e97fa640 1 -- 172.21.15.94:0/829400370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc02d040 con 0x55e210f864d0 2024-04-25T14:04:13.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.895+0000 7f11e97fa640 1 -- 172.21.15.94:0/829400370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc01e610 con 0x55e210f864d0 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- 172.21.15.94:0/829400370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 msgr2=0x55e21134c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc0059b0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- 172.21.15.94:0/829400370 shutdown_connections 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e21135f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e21134cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 --2- 172.21.15.94:0/829400370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e21134c9e0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- 172.21.15.94:0/829400370 >> 172.21.15.94:0/829400370 conn(0x55e210fa34e0 msgr2=0x55e21135df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- 172.21.15.94:0/829400370 shutdown_connections 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- 172.21.15.94:0/829400370 wait complete. 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 -- start start 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.896+0000 7f12011df740 1 Processor -- start 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211362530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211365470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41aa6b0 con 0x55e210f864d0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41a9fe0 con 0x55e210faf390 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41a7220 con 0x55e210f8e6b0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f12011df740 10 client.0 _ll_register_callbacks cb 0x7f11dc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43454/0 (socket says 172.21.15.94:43454) 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 -- 172.21.15.94:0/2089697846 learned_addr learned my addr 172.21.15.94:0/2089697846 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211362530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211365470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 -- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 msgr2=0x55e211362530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211362530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 -- 172.21.15.94:0/2089697846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 msgr2=0x55e211365470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211365470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 -- 172.21.15.94:0/2089697846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e211365a30 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11f2ffd640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0058c0 tx=0x7f11dc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ea06c30b1f1aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.897+0000 7f11d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc02c040 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f11dc01e610 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11dc02d040 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f11dc012500 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 10 client.23433 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e2113d5cf0 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc02d040 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f11dc026040 con 0x55e210f864d0 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 10 client.23433 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 10 client.23433 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 20 client.23433 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f433a0 0x55e2113d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:13.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 4 client.23433 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2112bd800 con 0x55e210f433a0 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.898+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f433a0 0x55e2113d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f11d2ffd640 10 client.23433 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f11f37fe640 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f433a0 0x55e2113d8710 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea528f2afb2425ec server_cookie=b5d3ee5e023fc8df in_seq=0 out_seq=0 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f11d2ffd640 1 -- 172.21.15.94:0/2089697846 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2112bd800 con 0x55e210f433a0 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f11d2ffd640 10 client.23433 handle_command_reply: tid=0 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 client.23433 shutdown 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 msgr2=0x55e211361ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0058c0 tx=0x7f11dc009ac0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 shutdown_connections 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55e210f433a0 0x55e2113d8710 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e210faf390 0x55e211365470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e210f8e6b0 0x55e211362530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f11d2ffd640 0 client.23433 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 --2- 172.21.15.94:0/2089697846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e210f864d0 0x55e211361ff0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 >> 172.21.15.94:0/2089697846 conn(0x55e210fa34e0 msgr2=0x55e21135f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 shutdown_connections 2024-04-25T14:04:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.899+0000 7f12011df740 1 -- 172.21.15.94:0/2089697846 wait complete. 2024-04-25T14:04:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f12011df740 20 client.23433 trim_cache size 0 max 16384 2024-04-25T14:04:13.906 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 msgr2=0x7f11e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 0x7f11e00b2000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f11e40fd290 tx=0x7f11e40f6050 comp rx=0 tx=0).stop 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 msgr2=0x7f11f41a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e40042f0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 shutdown_connections 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f11e00afaf0 0x7f11e00b2000 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11f4108920 0x7f11f41a08c0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11f4103c20 0x7f11f4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 --2- 172.21.15.94:0/1931764320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11f41029e0 0x7f11f41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:13.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 >> 172.21.15.94:0/1931764320 conn(0x7f11f40fdbd0 msgr2=0x7f11f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:13.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 shutdown_connections 2024-04-25T14:04:13.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:13.900+0000 7f11fa253640 1 -- 172.21.15.94:0/1931764320 wait complete. 2024-04-25T14:04:13.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:04:14.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 -- 172.21.15.94:0/740328707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af8108920 msgr2=0x7f3af810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 --2- 172.21.15.94:0/740328707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af8108920 0x7f3af810ad40 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009ca0 tx=0x7f3af002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 -- 172.21.15.94:0/740328707 shutdown_connections 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 --2- 172.21.15.94:0/740328707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af8108920 0x7f3af810ad40 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 --2- 172.21.15.94:0/740328707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 --2- 172.21.15.94:0/740328707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af81029e0 0x7f3af81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 -- 172.21.15.94:0/740328707 >> 172.21.15.94:0/740328707 conn(0x7f3af80fdbd0 msgr2=0x7f3af8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 -- 172.21.15.94:0/740328707 shutdown_connections 2024-04-25T14:04:14.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.029+0000 7f3afed46640 1 -- 172.21.15.94:0/740328707 wait complete. 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 Processor -- start 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 -- start start 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af8108920 0x7f3af81a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3af810e510 con 0x7f3af81029e0 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3af810e390 con 0x7f3af8108920 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afed46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3af810e690 con 0x7f3af8103c20 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43482/0 (socket says 172.21.15.94:43482) 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afcae2640 1 -- 172.21.15.94:0/2018159586 learned_addr learned my addr 172.21.15.94:0/2018159586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3af7fff640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.030+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af8108920 0x7f3af81a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 -- 172.21.15.94:0/2018159586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af8108920 msgr2=0x7f3af81a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af8108920 0x7f3af81a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 -- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 msgr2=0x7f3af819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 -- 172.21.15.94:0/2018159586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af81a2040 con 0x7f3af8103c20 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3afcae2640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af7fff640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af819aa40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400e9f0 tx=0x7f3ae400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af5ffb640 1 -- 172.21.15.94:0/2018159586 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400ccc0 con 0x7f3af8103c20 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af81a2330 con 0x7f3af8103c20 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af5ffb640 1 -- 172.21.15.94:0/2018159586 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ae4005020 con 0x7f3af8103c20 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3af5ffb640 1 -- 172.21.15.94:0/2018159586 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004a00 con 0x7f3af8103c20 2024-04-25T14:04:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.031+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af81a2880 con 0x7f3af8103c20 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.033+0000 7f3af5ffb640 1 -- 172.21.15.94:0/2018159586 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3ae4004ba0 con 0x7f3af8103c20 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 Processor -- start 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 -- start start 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7d04390 0x55e0d80a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e0d800ad80 con 0x55e0d7ce36b0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0d7fc4a20 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0d8006d40 con 0x55e0d7d04390 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56808/0 (socket says 172.21.15.94:56808) 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 -- 172.21.15.94:0/188848698 learned_addr learned my addr 172.21.15.94:0/188848698 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7d04390 0x55e0d80a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3af5ffb640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 0x7f3ae80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3af7fff640 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 0x7f3ae80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 -- 172.21.15.94:0/188848698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7d04390 msgr2=0x55e0d80a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3af5ffb640 1 -- 172.21.15.94:0/2018159586 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3ae40dec80 con 0x7f3af8103c20 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7d04390 0x55e0d80a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 -- 172.21.15.94:0/188848698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 msgr2=0x55e0d80a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.034+0000 7f3afcae2640 1 -- 172.21.15.94:0/188848698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d7fcfc30 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.035+0000 7f3afcae2640 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0115c0 tx=0x7f3aec011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6216c1ec000ec91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.035+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 0x7f3ae80b2000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3af002f500 tx=0x7f3af002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.035+0000 7f3ae2ffd640 1 -- 172.21.15.94:0/188848698 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec005020 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.035+0000 7f3ae2ffd640 1 -- 172.21.15.94:0/188848698 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aec0047a0 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.035+0000 7f3ae2ffd640 1 -- 172.21.15.94:0/188848698 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00f8c0 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- 172.21.15.94:0/188848698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 msgr2=0x55e0d80a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0115c0 tx=0x7f3aec011a80 comp rx=0 tx=0).stop 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- 172.21.15.94:0/188848698 shutdown_connections 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80a6d90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/188848698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7d04390 0x55e0d80a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- 172.21.15.94:0/188848698 >> 172.21.15.94:0/188848698 conn(0x55e0d7cf84e0 msgr2=0x55e0d80a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- 172.21.15.94:0/188848698 shutdown_connections 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- 172.21.15.94:0/188848698 wait complete. 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 -- start start 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.036+0000 7f3b05bf7740 1 Processor -- start 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e0d80a8530 con 0x55e0d7d04390 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0d7f91dc0 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0d80a86b0 con 0x55e0d7ce36b0 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3b05bf7740 10 client.0 _ll_register_callbacks cb 0x55e0d7fad600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43508/0 (socket says 172.21.15.94:43508) 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56810/0 (socket says 172.21.15.94:56810) 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 -- 172.21.15.94:0/1849822587 learned_addr learned my addr 172.21.15.94:0/1849822587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3af7fff640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 -- 172.21.15.94:0/1849822587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7ce36b0 msgr2=0x55e0d80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 -- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 msgr2=0x55e0d80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 -- 172.21.15.94:0/1849822587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d7d1c3f0 con 0x55e0d7d04390 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afcae2640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f3af0035000 tx=0x7f3af00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73ee384e69e40b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.037+0000 7f3ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0004820 con 0x55e0d7d04390 2024-04-25T14:04:14.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3af002f660 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec0047a0 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3af005c050 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 10 client.23445 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0d7de67d0 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aec0047a0 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3af004a050 con 0x55e0d7d04390 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 10 client.23445 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 10 client.23445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 20 client.23445 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7cc2f60 0x55e0d80be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 4 client.23445 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0d8018990 con 0x55e0d7cc2f60 2024-04-25T14:04:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.038+0000 7f3afcae2640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7cc2f60 0x55e0d80be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.039+0000 7f3afcae2640 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7cc2f60 0x55e0d80be300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3cf30f6d7f283ff8 server_cookie=bf7ec1e737e05509 in_seq=0 out_seq=0 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.039+0000 7f3ad7fff640 10 client.23445 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.039+0000 7f3ad7fff640 1 -- 172.21.15.94:0/1849822587 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0d8018990 con 0x55e0d7cc2f60 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.039+0000 7f3ad7fff640 10 client.23445 handle_command_reply: tid=0 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.039+0000 7f3b05bf7740 1 client.23445 shutdown 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 msgr2=0x55e0d80b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f3af0035000 tx=0x7f3af00095c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 shutdown_connections 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7cc2f60 0x55e0d80be300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7d04390 0x55e0d80b7a30 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3ad7fff640 0 client.23445 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/1849822587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 >> 172.21.15.94:0/1849822587 conn(0x55e0d7cf84e0 msgr2=0x55e0d80a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 shutdown_connections 2024-04-25T14:04:14.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 1 -- 172.21.15.94:0/1849822587 wait complete. 2024-04-25T14:04:14.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.040+0000 7f3b05bf7740 20 client.23445 trim_cache size 0 max 16384 2024-04-25T14:04:14.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 Processor -- start 2024-04-25T14:04:14.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 -- start start 2024-04-25T14:04:14.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81ab5f0 con 0x55e0d7ce36b0 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81ad130 con 0x55e0d7cc2f60 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81abcc0 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3afcae2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56814/0 (socket says 172.21.15.94:56814) 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3afcae2640 1 -- 172.21.15.94:0/2883156233 learned_addr learned my addr 172.21.15.94:0/2883156233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.058+0000 7f3af7fff640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 -- 172.21.15.94:0/2883156233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 msgr2=0x55e0d80ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 -- 172.21.15.94:0/2883156233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 msgr2=0x55e0d80b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 -- 172.21.15.94:0/2883156233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d80b7e90 con 0x55e0d7cc2f60 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3afcae2640 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55e0d80a6530 tx=0x7f3aec004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f7964169775be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.059+0000 7f3ae27fc640 1 -- 172.21.15.94:0/2883156233 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec01d040 con 0x55e0d7cc2f60 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.060+0000 7f3ae27fc640 1 -- 172.21.15.94:0/2883156233 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aec02f030 con 0x55e0d7cc2f60 2024-04-25T14:04:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.060+0000 7f3ae27fc640 1 -- 172.21.15.94:0/2883156233 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00f660 con 0x55e0d7cc2f60 2024-04-25T14:04:14.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.060+0000 7f3b05bf7740 1 -- 172.21.15.94:0/2883156233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 msgr2=0x55e0d80ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.060+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55e0d80a6530 tx=0x7f3aec004cb0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 -- 172.21.15.94:0/2883156233 shutdown_connections 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d80b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/2883156233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80ab310 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 -- 172.21.15.94:0/2883156233 >> 172.21.15.94:0/2883156233 conn(0x55e0d7cf84e0 msgr2=0x55e0d80a6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 -- 172.21.15.94:0/2883156233 shutdown_connections 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 -- 172.21.15.94:0/2883156233 wait complete. 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 -- start start 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.061+0000 7f3b05bf7740 1 Processor -- start 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81aca60 con 0x55e0d7ce36b0 2024-04-25T14:04:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81ac390 con 0x55e0d7cc2f60 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81a95d0 con 0x55e0d7cdb4d0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3b05bf7740 10 client.0 _ll_register_callbacks cb 0x7f3af0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43522/0 (socket says 172.21.15.94:43522) 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3af7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59048/0 (socket says 172.21.15.94:59048) 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 -- 172.21.15.94:0/135129870 learned_addr learned my addr 172.21.15.94:0/135129870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afcae2640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 -- 172.21.15.94:0/135129870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 msgr2=0x55e0d80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 -- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 msgr2=0x55e0d80b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3afd2e3640 1 -- 172.21.15.94:0/135129870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d80b53c0 con 0x55e0d7ce36b0 2024-04-25T14:04:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.062+0000 7f3af7fff640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3afd2e3640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f3af0030b50 tx=0x7f3af0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed91c6da4520749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0065060 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3af0004d40 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec02f030 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3af00305c0 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3b05bf7740 10 client.23451 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e0d812e460 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.063+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aec02f030 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3af004a040 con 0x55e0d7ce36b0 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 10 client.23451 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 10 client.23451 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:04:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 20 client.23451 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7d04390 0x55e0d8130e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 4 client.23451 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0d8012800 con 0x55e0d7d04390 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3afcae2640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7d04390 0x55e0d8130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3ad7fff640 10 client.23451 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.064+0000 7f3afcae2640 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7d04390 0x55e0d8130e80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81bcedb62e43293b server_cookie=fcc6f5d9e1557470 in_seq=0 out_seq=0 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3ad7fff640 1 -- 172.21.15.94:0/135129870 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0d8012800 con 0x55e0d7d04390 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3ad7fff640 10 client.23451 handle_command_reply: tid=0 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 client.23451 shutdown 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 msgr2=0x55e0d812ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f3af0030b50 tx=0x7f3af0009c60 comp rx=0 tx=0).stop 2024-04-25T14:04:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 shutdown_connections 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55e0d7d04390 0x55e0d8130e80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0d7ce36b0 0x55e0d812ce70 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0d7cdb4d0 0x55e0d80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3ad7fff640 0 client.23451 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 --2- 172.21.15.94:0/135129870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0d7cc2f60 0x55e0d80b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 >> 172.21.15.94:0/135129870 conn(0x55e0d7cf84e0 msgr2=0x55e0d80ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 shutdown_connections 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 1 -- 172.21.15.94:0/135129870 wait complete. 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.065+0000 7f3b05bf7740 20 client.23451 trim_cache size 0 max 16384 2024-04-25T14:04:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.066+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 msgr2=0x7f3ae80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.066+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 0x7f3ae80b2000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3af002f500 tx=0x7f3af002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.066+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 msgr2=0x7f3af819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.066+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af819aa40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400e9f0 tx=0x7f3ae400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 shutdown_connections 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3ae80afaf0 0x7f3ae80b2000 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3af8108920 0x7f3af81a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af8103c20 0x7f3af819aa40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 --2- 172.21.15.94:0/2018159586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af81029e0 0x7f3af819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 >> 172.21.15.94:0/2018159586 conn(0x7f3af80fdbd0 msgr2=0x7f3af8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 shutdown_connections 2024-04-25T14:04:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.067+0000 7f3afed46640 1 -- 172.21.15.94:0/2018159586 wait complete. 2024-04-25T14:04:14.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:13 smithi094 ceph-mon[29195]: pgmap v1024: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-04-25T14:04:14.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/100060941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:14.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:04:14.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 190286 18.88 MB/sec execute 416 sec latency 75.954 ms 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 -- 172.21.15.94:0/3810294856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa0103c20 msgr2=0x7feaa0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 --2- 172.21.15.94:0/3810294856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa0103c20 0x7feaa0104030 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fea90009900 tx=0x7fea9002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 -- 172.21.15.94:0/3810294856 shutdown_connections 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 --2- 172.21.15.94:0/3810294856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0108920 0x7feaa010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 --2- 172.21.15.94:0/3810294856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa0103c20 0x7feaa0104030 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 --2- 172.21.15.94:0/3810294856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa01029e0 0x7feaa01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 -- 172.21.15.94:0/3810294856 >> 172.21.15.94:0/3810294856 conn(0x7feaa00fdbd0 msgr2=0x7feaa0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 -- 172.21.15.94:0/3810294856 shutdown_connections 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.204+0000 7feaa876c640 1 -- 172.21.15.94:0/3810294856 wait complete. 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 Processor -- start 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 -- start start 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0103c20 0x7feaa019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa0108920 0x7feaa01a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010e4e0 con 0x7feaa01029e0 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010e360 con 0x7feaa0103c20 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa876c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaa010e660 con 0x7feaa0108920 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43548/0 (socket says 172.21.15.94:43548) 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 -- 172.21.15.94:0/1199621790 learned_addr learned my addr 172.21.15.94:0/1199621790 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa5d07640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0103c20 0x7feaa019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6d09640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa0108920 0x7feaa01a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 -- 172.21.15.94:0/1199621790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa0108920 msgr2=0x7feaa01a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa0108920 0x7feaa01a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 -- 172.21.15.94:0/1199621790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0103c20 msgr2=0x7feaa019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0103c20 0x7feaa019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.205+0000 7feaa6508640 1 -- 172.21.15.94:0/1199621790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa01a2070 con 0x7feaa01029e0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7feaa6508640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9f0 tx=0x7fea9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7fea977fe640 1 -- 172.21.15.94:0/1199621790 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00ccc0 con 0x7feaa01029e0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa01a2360 con 0x7feaa01029e0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7fea977fe640 1 -- 172.21.15.94:0/1199621790 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x7feaa01029e0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7fea977fe640 1 -- 172.21.15.94:0/1199621790 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00ce60 con 0x7feaa01029e0 2024-04-25T14:04:14.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.206+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa01a28b0 con 0x7feaa01029e0 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.208+0000 7feaaf61b740 1 Processor -- start 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 -- start start 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 0x555d55322d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 0x555d55323a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d55286d80 con 0x555d54f574d0 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d55240a20 con 0x555d54f80390 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaaf61b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d55282d40 con 0x555d54f5f6b0 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:56854/0 (socket says 172.21.15.94:56854) 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa5d07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 0x555d55323a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 -- 172.21.15.94:0/3681784046 learned_addr learned my addr 172.21.15.94:0/3681784046 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7fea977fe640 1 -- 172.21.15.94:0/1199621790 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fea9c004020 con 0x7feaa01029e0 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6508640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 0x555d55322d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 -- 172.21.15.94:0/3681784046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 msgr2=0x555d55323a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 0x555d55323a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 -- 172.21.15.94:0/3681784046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 msgr2=0x555d55322d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 0x555d55322d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6d09640 1 -- 172.21.15.94:0/3681784046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5524bc30 con 0x555d54f80390 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa5d07640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 0x555d55323a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.209+0000 7feaa6508640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 0x555d55322d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7feaa6d09640 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea98009fd0 tx=0x7fea980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa1ed2c812dbd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7fea94ff9640 1 -- 172.21.15.94:0/3681784046 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98004d20 con 0x555d54f80390 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7fea94ff9640 1 -- 172.21.15.94:0/3681784046 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea98004050 con 0x555d54f80390 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7fea94ff9640 1 -- 172.21.15.94:0/3681784046 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98005280 con 0x555d54f80390 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7fea977fe640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 0x7fea8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7feaa6508640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 0x7fea8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.210+0000 7fea977fe640 1 -- 172.21.15.94:0/1199621790 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fea9c0def90 con 0x7feaa01029e0 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- 172.21.15.94:0/3681784046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 msgr2=0x555d55322810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaa6508640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 0x7fea8c0b2000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x555d55322530 tx=0x7fea9c003fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fea98009fd0 tx=0x7fea980118d0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- 172.21.15.94:0/3681784046 shutdown_connections 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f5f6b0 0x555d55323a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f574d0 0x555d55322d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 --2- 172.21.15.94:0/3681784046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d55322810 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- 172.21.15.94:0/3681784046 >> 172.21.15.94:0/3681784046 conn(0x555d54f744e0 msgr2=0x555d5531e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- 172.21.15.94:0/3681784046 shutdown_connections 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- 172.21.15.94:0/3681784046 wait complete. 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 -- start start 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.211+0000 7feaaf61b740 1 Processor -- start 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d5532c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d553339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555d55324530 con 0x555d54f5f6b0 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555d5520ddc0 con 0x555d54f80390 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d553246b0 con 0x555d54f574d0 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaaf61b740 10 client.0 _ll_register_callbacks cb 0x555d55229600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43566/0 (socket says 172.21.15.94:43566) 2024-04-25T14:04:14.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 -- 172.21.15.94:0/613150521 learned_addr learned my addr 172.21.15.94:0/613150521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa6d09640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d5532c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa6d09640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d553339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 -- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 msgr2=0x555d5532c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d5532c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 -- 172.21.15.94:0/613150521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 msgr2=0x555d553339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d553339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 -- 172.21.15.94:0/613150521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d54f983f0 con 0x555d54f5f6b0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa6d09640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d5532c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7feaa5d07640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a930 tx=0x7fea90031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434b45d97881c8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7fea89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.212+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90035040 con 0x555d54f5f6b0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fea90004040 con 0x555d54f5f6b0 2024-04-25T14:04:14.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea98004050 con 0x555d54f5f6b0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea90004240 con 0x555d54f5f6b0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7feaaf61b740 10 client.23460 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d550627d0 con 0x555d54f5f6b0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea98004050 con 0x555d54f5f6b0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fea90044020 con 0x555d54f5f6b0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7feaaf61b740 10 client.23460 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.213+0000 7feaaf61b740 10 client.23460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaaf61b740 20 client.23460 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f3ef60 0x555d5533a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaaf61b740 4 client.23460 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d55294990 con 0x555d54f3ef60 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaa6d09640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f3ef60 0x555d5533a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7feaa6d09640 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f3ef60 0x555d5533a610 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=850698bad411e967 server_cookie=41ab82f63f990711 in_seq=0 out_seq=0 2024-04-25T14:04:14.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.214+0000 7fea89ffb640 10 client.23460 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7fea89ffb640 1 -- 172.21.15.94:0/613150521 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d55294990 con 0x555d54f3ef60 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7fea89ffb640 10 client.23460 handle_command_reply: tid=0 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 client.23460 shutdown 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 msgr2=0x555d5532ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a930 tx=0x7fea90031db0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 shutdown_connections 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f3ef60 0x555d5533a610 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f80390 0x555d553339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f5f6b0 0x555d5532ca80 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7fea89ffb640 0 client.23460 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 --2- 172.21.15.94:0/613150521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d5532c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 >> 172.21.15.94:0/613150521 conn(0x555d54f744e0 msgr2=0x555d5531d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 shutdown_connections 2024-04-25T14:04:14.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 1 -- 172.21.15.94:0/613150521 wait complete. 2024-04-25T14:04:14.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.215+0000 7feaaf61b740 20 client.23460 trim_cache size 0 max 16384 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.229+0000 7feaaf61b740 1 Processor -- start 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 -- start start 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f3ef60 0x555d5533a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d553227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ab5c0 con 0x555d54f80390 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ad100 con 0x555d54f3ef60 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaaf61b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01abc90 con 0x555d54f574d0 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43582/0 (socket says 172.21.15.94:43582) 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 -- 172.21.15.94:0/421402612 learned_addr learned my addr 172.21.15.94:0/421402612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa5d07640 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d553227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f3ef60 0x555d5533a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 -- 172.21.15.94:0/421402612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 msgr2=0x555d553227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d553227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 -- 172.21.15.94:0/421402612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f3ef60 msgr2=0x555d5533a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f3ef60 0x555d5533a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 -- 172.21.15.94:0/421402612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5532d680 con 0x555d54f80390 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7feaa6d09640 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fea98009560 tx=0x7fea98000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f38300886d79dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.230+0000 7fea8bfff640 1 -- 172.21.15.94:0/421402612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea980042e0 con 0x555d54f80390 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.231+0000 7fea8bfff640 1 -- 172.21.15.94:0/421402612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea98004e30 con 0x555d54f80390 2024-04-25T14:04:14.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.231+0000 7fea8bfff640 1 -- 172.21.15.94:0/421402612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98005770 con 0x555d54f80390 2024-04-25T14:04:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.231+0000 7feaaf61b740 1 -- 172.21.15.94:0/421402612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 msgr2=0x555d55322d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.231+0000 7feaaf61b740 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fea98009560 tx=0x7fea98000f40 comp rx=0 tx=0).stop 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- 172.21.15.94:0/421402612 shutdown_connections 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d55322d10 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f574d0 0x555d553227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- 172.21.15.94:0/421402612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f3ef60 0x555d5533a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- 172.21.15.94:0/421402612 >> 172.21.15.94:0/421402612 conn(0x555d54f744e0 msgr2=0x555d5532c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- 172.21.15.94:0/421402612 shutdown_connections 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- 172.21.15.94:0/421402612 wait complete. 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- start start 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 Processor -- start 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f3ef60 0x555d5532ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f574d0 0x555d55330500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01aca30 con 0x555d54f80390 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ac360 con 0x555d54f574d0 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01a95a0 con 0x555d54f3ef60 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaaf61b740 10 client.0 _ll_register_callbacks cb 0x7fea90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaa6d09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43590/0 (socket says 172.21.15.94:43590) 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaa6d09640 1 -- 172.21.15.94:0/171817468 learned_addr learned my addr 172.21.15.94:0/171817468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaa5d07640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f574d0 0x555d55330500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.232+0000 7feaa6d09640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f3ef60 0x555d5532ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 -- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f3ef60 msgr2=0x555d5532ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f3ef60 0x555d5532ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 -- 172.21.15.94:0/171817468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f574d0 msgr2=0x555d55330500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f574d0 0x555d55330500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 -- 172.21.15.94:0/171817468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d553a0ef0 con 0x555d54f80390 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaa6d09640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fea98003eb0 tx=0x7fea98003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa00956310a0aa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98004980 con 0x555d54f80390 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fea98004b20 con 0x555d54f80390 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea98004e30 con 0x555d54f80390 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea980054f0 con 0x555d54f80390 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 10 client.23466 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d553a2020 con 0x555d54f80390 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea98004e30 con 0x555d54f80390 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fea9802e5c0 con 0x555d54f80390 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 10 client.23466 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 10 client.23466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 20 client.23466 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f143a0 0x555d553a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 4 client.23466 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.233+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d5528e800 con 0x555d54f143a0 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7feaa5d07640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f143a0 0x555d553a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7feaa5d07640 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f143a0 0x555d553a4800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6897819ec32146d8 server_cookie=f747824e125d6454 in_seq=0 out_seq=0 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7fea89ffb640 10 client.23466 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7fea89ffb640 1 -- 172.21.15.94:0/171817468 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d5528e800 con 0x555d54f143a0 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7fea89ffb640 10 client.23466 handle_command_reply: tid=0 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7feaaf61b740 1 client.23466 shutdown 2024-04-25T14:04:14.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 msgr2=0x555d553a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.234+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fea98003eb0 tx=0x7fea98003cd0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 shutdown_connections 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x555d54f143a0 0x555d553a4800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555d54f80390 0x555d553a09b0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555d54f574d0 0x555d55330500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 --2- 172.21.15.94:0/171817468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d54f3ef60 0x555d5532ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 >> 172.21.15.94:0/171817468 conn(0x555d54f744e0 msgr2=0x555d55327270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7fea89ffb640 0 client.23466 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 shutdown_connections 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 1 -- 172.21.15.94:0/171817468 wait complete. 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaaf61b740 20 client.23466 trim_cache size 0 max 16384 2024-04-25T14:04:14.241 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 msgr2=0x7fea8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 0x7fea8c0b2000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x555d55322530 tx=0x7fea9c003fb0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 msgr2=0x7feaa006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9f0 tx=0x7fea9c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 shutdown_connections 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fea8c0afaf0 0x7fea8c0b2000 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa0108920 0x7feaa01a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feaa0103c20 0x7feaa019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 --2- 172.21.15.94:0/1199621790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa01029e0 0x7feaa006a890 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 >> 172.21.15.94:0/1199621790 conn(0x7feaa00fdbd0 msgr2=0x7feaa0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 shutdown_connections 2024-04-25T14:04:14.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.235+0000 7feaa876c640 1 -- 172.21.15.94:0/1199621790 wait complete. 2024-04-25T14:04:14.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:04:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:13 smithi138 ceph-mon[32977]: pgmap v1024: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-04-25T14:04:14.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/100060941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 -- 172.21.15.94:0/1203822618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740101740 msgr2=0x7f7740101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 --2- 172.21.15.94:0/1203822618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740101740 0x7f7740101bb0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f7728009900 tx=0x7f772802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 -- 172.21.15.94:0/1203822618 shutdown_connections 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 --2- 172.21.15.94:0/1203822618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740102240 0x7f774010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 --2- 172.21.15.94:0/1203822618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740101740 0x7f7740101bb0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 --2- 172.21.15.94:0/1203822618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740105090 0x7f77401054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 -- 172.21.15.94:0/1203822618 >> 172.21.15.94:0/1203822618 conn(0x7f77400fd340 msgr2=0x7f77400ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 -- 172.21.15.94:0/1203822618 shutdown_connections 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 -- 172.21.15.94:0/1203822618 wait complete. 2024-04-25T14:04:14.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.363+0000 7f77455e3640 1 Processor -- start 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 -- start start 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740101740 0x7f774019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740102240 0x7f774019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f774010f600 con 0x7f7740105090 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f774010f480 con 0x7f7740102240 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f77455e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f774010f780 con 0x7f7740101740 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f7737fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740102240 0x7f774019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43612/0 (socket says 172.21.15.94:43612) 2024-04-25T14:04:14.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 -- 172.21.15.94:0/707685357 learned_addr learned my addr 172.21.15.94:0/707685357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773effd640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740101740 0x7f774019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 -- 172.21.15.94:0/707685357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740101740 msgr2=0x7f774019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740101740 0x7f774019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 -- 172.21.15.94:0/707685357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740102240 msgr2=0x7f774019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740102240 0x7f774019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 -- 172.21.15.94:0/707685357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77401a1d80 con 0x7f7740105090 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773f7fe640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f773800eb10 tx=0x7f773800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773cff9640 1 -- 172.21.15.94:0/707685357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738004b20 con 0x7f7740105090 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.364+0000 7f773cff9640 1 -- 172.21.15.94:0/707685357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7738004cc0 con 0x7f7740105090 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.365+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77401a2070 con 0x7f7740105090 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.365+0000 7f773cff9640 1 -- 172.21.15.94:0/707685357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738010670 con 0x7f7740105090 2024-04-25T14:04:14.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.365+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77401a2570 con 0x7f7740105090 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 Processor -- start 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f773cff9640 1 -- 172.21.15.94:0/707685357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7738005020 con 0x7f7740105090 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 -- start start 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 0x559b0a138d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a139a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559b0a09cd80 con 0x559b09d96390 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559b0a056a20 con 0x559b09d6d4d0 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.367+0000 7f774c56e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b0a098d40 con 0x559b09d756b0 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43624/0 (socket says 172.21.15.94:43624) 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 -- 172.21.15.94:0/868226144 learned_addr learned my addr 172.21.15.94:0/868226144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f773effd640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 0x559b0a138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f773cff9640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 0x7f772c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 -- 172.21.15.94:0/868226144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 msgr2=0x559b0a139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a139a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 -- 172.21.15.94:0/868226144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 msgr2=0x559b0a138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 0x559b0a138d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 -- 172.21.15.94:0/868226144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0a061c30 con 0x559b09d96390 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f773effd640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 0x559b0a138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f773f7fe640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 0x7f772c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f7737fff640 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7728031500 tx=0x7f7728009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29689a787f83b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f773cff9640 1 -- 172.21.15.94:0/707685357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f77380df280 con 0x7f7740105090 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f77357fa640 1 -- 172.21.15.94:0/868226144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728035030 con 0x559b09d96390 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f77357fa640 1 -- 172.21.15.94:0/868226144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728004040 con 0x559b09d96390 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.368+0000 7f77357fa640 1 -- 172.21.15.94:0/868226144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728003af0 con 0x559b09d96390 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 -- 172.21.15.94:0/868226144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 msgr2=0x559b0a138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7728031500 tx=0x7f7728009560 comp rx=0 tx=0).stop 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f773f7fe640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 0x7f772c0b2000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77380a4930 tx=0x7f773800efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 -- 172.21.15.94:0/868226144 shutdown_connections 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d6d4d0 0x559b0a138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 --2- 172.21.15.94:0/868226144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d96390 0x559b0a138810 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 -- 172.21.15.94:0/868226144 >> 172.21.15.94:0/868226144 conn(0x559b09d8a4e0 msgr2=0x559b0a134020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 -- 172.21.15.94:0/868226144 shutdown_connections 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.369+0000 7f774c56e740 1 -- 172.21.15.94:0/868226144 wait complete. 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 -- start start 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 Processor -- start 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a142af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559b0a13a4d0 con 0x559b09d6d4d0 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559b0a023dc0 con 0x559b09d96390 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b0a13a650 con 0x559b09d756b0 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f774c56e740 10 client.0 _ll_register_callbacks cb 0x559b0a03f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43634/0 (socket says 172.21.15.94:43634) 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 -- 172.21.15.94:0/3263457871 learned_addr learned my addr 172.21.15.94:0/3263457871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f773effd640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 -- 172.21.15.94:0/3263457871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 msgr2=0x559b0a142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.370+0000 7f7737fff640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f7737fff640 1 -- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 msgr2=0x559b0a1499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f7737fff640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f7737fff640 1 -- 172.21.15.94:0/3263457871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b09dae3f0 con 0x559b09d6d4d0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f7737fff640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7728006010 tx=0x7f77280035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2223e0d4ba6c613e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772804b020 con 0x559b09d6d4d0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7728003870 con 0x559b09d6d4d0 2024-04-25T14:04:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7728004040 con 0x559b09d6d4d0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7728035070 con 0x559b09d6d4d0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 10 client.23478 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b09e787d0 con 0x559b09d6d4d0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728004040 con 0x559b09d6d4d0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f772803b830 con 0x559b09d6d4d0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 10 client.23478 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 10 client.23478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 20 client.23478 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d54f60 0x559b0a14fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 4 client.23478 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.371+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0a0aa990 con 0x559b09d54f60 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.372+0000 7f773effd640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d54f60 0x559b0a14fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.372+0000 7f77267fc640 10 client.23478 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:14.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.372+0000 7f773effd640 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d54f60 0x559b0a14fe30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46da54cb3481b2f4 server_cookie=f26fe0ea5472ce1a in_seq=0 out_seq=0 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f77267fc640 1 -- 172.21.15.94:0/3263457871 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b0a0aa990 con 0x559b09d54f60 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f77267fc640 10 client.23478 handle_command_reply: tid=0 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 client.23478 shutdown 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 msgr2=0x559b0a1425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7728006010 tx=0x7f77280035f0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 shutdown_connections 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d54f60 0x559b0a14fe30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 --2- 172.21.15.94:0/3263457871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1425b0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f77267fc640 0 client.23478 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 >> 172.21.15.94:0/3263457871 conn(0x559b09d8a4e0 msgr2=0x559b0a136f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 shutdown_connections 2024-04-25T14:04:14.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 1 -- 172.21.15.94:0/3263457871 wait complete. 2024-04-25T14:04:14.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.373+0000 7f774c56e740 20 client.23478 trim_cache size 0 max 16384 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 Processor -- start 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 -- start start 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a149970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a133f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77401ab4f0 con 0x559b09d6d4d0 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77401ad030 con 0x559b09d96390 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f774c56e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77401abbc0 con 0x559b09d756b0 2024-04-25T14:04:14.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43650/0 (socket says 172.21.15.94:43650) 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f7737fff640 1 -- 172.21.15.94:0/1761057319 learned_addr learned my addr 172.21.15.94:0/1761057319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f773effd640 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.387+0000 7f7737fff640 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 -- 172.21.15.94:0/1761057319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 msgr2=0x559b0a149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 -- 172.21.15.94:0/1761057319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 msgr2=0x559b0a133f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a133f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 -- 172.21.15.94:0/1761057319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0a1344d0 con 0x559b09d6d4d0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7737fff640 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f77280095a0 tx=0x7f7728005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff82300b7a50a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7734ff9640 1 -- 172.21.15.94:0/1761057319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772805e070 con 0x559b09d6d4d0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7734ff9640 1 -- 172.21.15.94:0/1761057319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f772805f040 con 0x559b09d6d4d0 2024-04-25T14:04:14.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.388+0000 7f7734ff9640 1 -- 172.21.15.94:0/1761057319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772804b040 con 0x559b09d6d4d0 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- 172.21.15.94:0/1761057319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 msgr2=0x559b0a133a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f77280095a0 tx=0x7f7728005e50 comp rx=0 tx=0).stop 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- 172.21.15.94:0/1761057319 shutdown_connections 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a133f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 --2- 172.21.15.94:0/1761057319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a133a50 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- 172.21.15.94:0/1761057319 >> 172.21.15.94:0/1761057319 conn(0x559b09d8a4e0 msgr2=0x559b0a13d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- 172.21.15.94:0/1761057319 shutdown_connections 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- 172.21.15.94:0/1761057319 wait complete. 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.389+0000 7f774c56e740 1 -- start start 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 Processor -- start 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a146b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77401ac960 con 0x559b09d6d4d0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77401ac290 con 0x559b09d96390 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77401a94d0 con 0x559b09d756b0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f774c56e740 10 client.0 _ll_register_callbacks cb 0x7f77300097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f7737fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43656/0 (socket says 172.21.15.94:43656) 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f7737fff640 1 -- 172.21.15.94:0/2992055634 learned_addr learned my addr 172.21.15.94:0/2992055634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f7737fff640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 -- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 msgr2=0x559b0a1b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 -- 172.21.15.94:0/2992055634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 msgr2=0x559b0a1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f773effd640 1 -- 172.21.15.94:0/2992055634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0a149ac0 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.390+0000 7f7737fff640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f773effd640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a146b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559b0a150480 tx=0x7f7730010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6e2241fd606c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f77267fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730004460 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f773001f930 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f772805f040 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f773002f050 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f774c56e740 10 client.32509 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559b0a1b67c0 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.391+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f772805f040 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f77300047a0 con 0x559b09d756b0 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 10 client.32509 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 10 client.32509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 20 client.32509 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d2a3a0 0x559b0a1b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 4 client.32509 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b0a0a4800 con 0x559b09d2a3a0 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f7737fff640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d2a3a0 0x559b0a1b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f77267fc640 10 client.32509 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.392+0000 7f7737fff640 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d2a3a0 0x559b0a1b9130 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42c153a8b9b2b347 server_cookie=a5b563c1b87d9199 in_seq=0 out_seq=0 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f77267fc640 1 -- 172.21.15.94:0/2992055634 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b0a0a4800 con 0x559b09d2a3a0 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f77267fc640 10 client.32509 handle_command_reply: tid=0 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 client.32509 shutdown 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 msgr2=0x559b0a146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a146b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x559b0a150480 tx=0x7f7730010c50 comp rx=0 tx=0).stop 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 shutdown_connections 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x559b09d2a3a0 0x559b0a1b9130 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b09d96390 0x559b0a1b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f77267fc640 0 client.32509 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b09d756b0 0x559b0a146b00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 --2- 172.21.15.94:0/2992055634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b09d6d4d0 0x559b0a1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 >> 172.21.15.94:0/2992055634 conn(0x559b09d8a4e0 msgr2=0x559b0a142180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 shutdown_connections 2024-04-25T14:04:14.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 1 -- 172.21.15.94:0/2992055634 wait complete. 2024-04-25T14:04:14.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.393+0000 7f774c56e740 20 client.32509 trim_cache size 0 max 16384 2024-04-25T14:04:14.398 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 msgr2=0x7f772c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 0x7f772c0b2000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77380a4930 tx=0x7f773800efd0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 msgr2=0x7f77401a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f773800eb10 tx=0x7f773800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 shutdown_connections 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f772c0afaf0 0x7f772c0b2000 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7740105090 0x7f77401a1710 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7740102240 0x7f774019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 --2- 172.21.15.94:0/707685357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7740101740 0x7f774019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 >> 172.21.15.94:0/707685357 conn(0x7f77400fd340 msgr2=0x7f7740105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 shutdown_connections 2024-04-25T14:04:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.394+0000 7f77455e3640 1 -- 172.21.15.94:0/707685357 wait complete. 2024-04-25T14:04:14.400 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:13 smithi137 ceph-mon[32568]: pgmap v1024: 97 pgs: 97 active+clean; 270 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-04-25T14:04:14.400 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/100060941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:14.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 -- 172.21.15.94:0/2440724753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc109a30 msgr2=0x7f20fc10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 --2- 172.21.15.94:0/2440724753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc109a30 0x7f20fc10be50 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009ca0 tx=0x7f20f402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 -- 172.21.15.94:0/2440724753 shutdown_connections 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 --2- 172.21.15.94:0/2440724753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc109a30 0x7f20fc10be50 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 --2- 172.21.15.94:0/2440724753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc104d30 0x7f20fc105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 --2- 172.21.15.94:0/2440724753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc103af0 0x7f20fc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.524+0000 7f21044ee640 1 -- 172.21.15.94:0/2440724753 >> 172.21.15.94:0/2440724753 conn(0x7f20fc0fece0 msgr2=0x7f20fc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- 172.21.15.94:0/2440724753 shutdown_connections 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- 172.21.15.94:0/2440724753 wait complete. 2024-04-25T14:04:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 Processor -- start 2024-04-25T14:04:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- start start 2024-04-25T14:04:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc104d30 0x7f20fc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc109a30 0x7f20fc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc10f620 con 0x7f20fc103af0 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc10f4a0 con 0x7f20fc109a30 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f21044ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc10f7a0 con 0x7f20fc104d30 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.525+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43664/0 (socket says 172.21.15.94:43664) 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 -- 172.21.15.94:0/194821087 learned_addr learned my addr 172.21.15.94:0/194821087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f2101a89640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc104d30 0x7f20fc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f2102a8b640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc109a30 0x7f20fc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 -- 172.21.15.94:0/194821087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc104d30 msgr2=0x7f20fc076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc104d30 0x7f20fc076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 -- 172.21.15.94:0/194821087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc109a30 msgr2=0x7f20fc076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc109a30 0x7f20fc076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 -- 172.21.15.94:0/194821087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc076c70 con 0x7f20fc103af0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f210228a640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9f0 tx=0x7f20f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f20f37fe640 1 -- 172.21.15.94:0/194821087 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800ccc0 con 0x7f20fc103af0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f20f37fe640 1 -- 172.21.15.94:0/194821087 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f80035f0 con 0x7f20fc103af0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f20f37fe640 1 -- 172.21.15.94:0/194821087 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800ce60 con 0x7f20fc103af0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc1a42b0 con 0x7f20fc103af0 2024-04-25T14:04:14.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.526+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc1a4720 con 0x7f20fc103af0 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f210b39d740 1 Processor -- start 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f210b39d740 1 -- start start 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f210b39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0329810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f210b39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 0x55aee0329d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f20f37fe640 1 -- 172.21.15.94:0/194821087 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f20f8003aa0 con 0x7f20fc103af0 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.529+0000 7f210b39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210b39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aee028dd80 con 0x55aedff666b0 2024-04-25T14:04:14.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210b39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aee0247a20 con 0x55aedff87390 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210b39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aee0289d40 con 0x55aedff5e4d0 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f2101a89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 0x55aee0329d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f2101a89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43678/0 (socket says 172.21.15.94:43678) 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f2101a89640 1 -- 172.21.15.94:0/4060606275 learned_addr learned my addr 172.21.15.94:0/4060606275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f2102a8b640 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0329810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 -- 172.21.15.94:0/4060606275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 msgr2=0x55aee0329810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0329810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 -- 172.21.15.94:0/4060606275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 msgr2=0x55aee032aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 -- 172.21.15.94:0/4060606275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee0252c30 con 0x55aedff5e4d0 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f2101a89640 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f210228a640 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 0x55aee0329d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20f80dca40 tx=0x7f20f80e2f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b427d19bf3c83114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.530+0000 7f20f37fe640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 0x7f20e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.531+0000 7f20f0bfe640 1 -- 172.21.15.94:0/4060606275 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80f2070 con 0x55aedff5e4d0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.531+0000 7f2102a8b640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 0x7f20e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.531+0000 7f20f0bfe640 1 -- 172.21.15.94:0/4060606275 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f8003fd0 con 0x55aedff5e4d0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.531+0000 7f20f0bfe640 1 -- 172.21.15.94:0/4060606275 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80a6bc0 con 0x55aedff5e4d0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.531+0000 7f20f37fe640 1 -- 172.21.15.94:0/194821087 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f20f80df010 con 0x7f20fc103af0 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- 172.21.15.94:0/4060606275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 msgr2=0x55aee0329d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 0x55aee0329d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20f80dca40 tx=0x7f20f80e2f90 comp rx=0 tx=0).stop 2024-04-25T14:04:14.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f2102a8b640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 0x7f20e80b2000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55aee032a860 tx=0x7f20f4009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- 172.21.15.94:0/4060606275 shutdown_connections 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee032aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff5e4d0 0x55aee0329d90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- 172.21.15.94:0/4060606275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0329810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- 172.21.15.94:0/4060606275 >> 172.21.15.94:0/4060606275 conn(0x55aedff7b4e0 msgr2=0x55aee0325020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- 172.21.15.94:0/4060606275 shutdown_connections 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- 172.21.15.94:0/4060606275 wait complete. 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- start start 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 Processor -- start 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff666b0 0x55aee0333a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee033a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aee032b530 con 0x55aedff5e4d0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aee0214dc0 con 0x55aedff666b0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.532+0000 7f210b39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aee032b6b0 con 0x55aedff87390 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210b39d740 10 client.0 _ll_register_callbacks cb 0x55aee0230600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43688/0 (socket says 172.21.15.94:43688) 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 -- 172.21.15.94:0/895264789 learned_addr learned my addr 172.21.15.94:0/895264789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f2102a8b640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee033a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f2101a89640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff666b0 0x55aee0333a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 -- 172.21.15.94:0/895264789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 msgr2=0x55aee033a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee033a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 -- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff666b0 msgr2=0x55aee0333a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff666b0 0x55aee0333a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 -- 172.21.15.94:0/895264789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aedff9f3f0 con 0x55aedff5e4d0 2024-04-25T14:04:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210228a640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f20f80a73e0 tx=0x7f20f8003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d0609f0db11c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f20e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8004ca0 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f20f80e2340 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f8003fd0 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f20f8004e90 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210b39d740 10 client.23496 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.533+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55aee00697d0 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f8003fd0 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f20f8004180 con 0x55aedff5e4d0 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 10 client.23496 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 10 client.23496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 20 client.23496 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff45f60 0x55aee0340e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 4 client.23496 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee029b990 con 0x55aedff45f60 2024-04-25T14:04:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f2101a89640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff45f60 0x55aee0340e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f2101a89640 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff45f60 0x55aee0340e30 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cef6ad500e1988e server_cookie=34e64540a000bfb4 in_seq=0 out_seq=0 2024-04-25T14:04:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.534+0000 7f20e5ffb640 10 client.23496 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f20e5ffb640 1 -- 172.21.15.94:0/895264789 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aee029b990 con 0x55aedff45f60 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f20e5ffb640 10 client.23496 handle_command_reply: tid=0 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 client.23496 shutdown 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 msgr2=0x55aee0333540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f20f80a73e0 tx=0x7f20f8003d90 comp rx=0 tx=0).stop 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 shutdown_connections 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff45f60 0x55aee0340e30 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee033a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff666b0 0x55aee0333a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f20e5ffb640 0 client.23496 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 --2- 172.21.15.94:0/895264789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333540 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.536+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 >> 172.21.15.94:0/895264789 conn(0x55aedff7b4e0 msgr2=0x55aee0327f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.537+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 shutdown_connections 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.537+0000 7f210b39d740 1 -- 172.21.15.94:0/895264789 wait complete. 2024-04-25T14:04:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.537+0000 7f210b39d740 20 client.23496 trim_cache size 0 max 16384 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 Processor -- start 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 -- start start 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff666b0 0x55aee0338960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0338ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1ad7f0 con 0x55aedff5e4d0 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1af330 con 0x55aedff87390 2024-04-25T14:04:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210b39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1adec0 con 0x55aedff666b0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210228a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43700/0 (socket says 172.21.15.94:43700) 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f210228a640 1 -- 172.21.15.94:0/1211581997 learned_addr learned my addr 172.21.15.94:0/1211581997 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f2101a89640 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff666b0 0x55aee0338960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.551+0000 7f2102a8b640 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0338ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 -- 172.21.15.94:0/1211581997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff666b0 msgr2=0x55aee0338960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff666b0 0x55aee0338960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 -- 172.21.15.94:0/1211581997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 msgr2=0x55aee0338ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0338ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 -- 172.21.15.94:0/1211581997 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee0325bb0 con 0x55aedff5e4d0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f210228a640 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f20f80e2ee0 tx=0x7f20f80dcac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d7db3e8f9c175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f20e7fff640 1 -- 172.21.15.94:0/1211581997 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8005080 con 0x55aedff5e4d0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f20e7fff640 1 -- 172.21.15.94:0/1211581997 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f80a6f60 con 0x55aedff5e4d0 2024-04-25T14:04:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.552+0000 7f20e7fff640 1 -- 172.21.15.94:0/1211581997 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8005270 con 0x55aedff5e4d0 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- 172.21.15.94:0/1211581997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 msgr2=0x55aee0333820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f20f80e2ee0 tx=0x7f20f80dcac0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- 172.21.15.94:0/1211581997 shutdown_connections 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff87390 0x55aee0338ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff666b0 0x55aee0338960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 --2- 172.21.15.94:0/1211581997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff5e4d0 0x55aee0333820 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- 172.21.15.94:0/1211581997 >> 172.21.15.94:0/1211581997 conn(0x55aedff7b4e0 msgr2=0x55aee0333110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- 172.21.15.94:0/1211581997 shutdown_connections 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- 172.21.15.94:0/1211581997 wait complete. 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.553+0000 7f210b39d740 1 -- start start 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 Processor -- start 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee03a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee03a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1aec60 con 0x55aedff666b0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1ae590 con 0x55aedff5e4d0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1ab7d0 con 0x55aedff87390 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210b39d740 10 client.0 _ll_register_callbacks cb 0x7f20f80f0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee03a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210228a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2102a8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee03a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210228a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57002/0 (socket says 172.21.15.94:57002) 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f210228a640 1 -- 172.21.15.94:0/1401949405 learned_addr learned my addr 172.21.15.94:0/1401949405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 -- 172.21.15.94:0/1401949405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 msgr2=0x55aee03a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee03a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 -- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 msgr2=0x55aee039ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.554+0000 7f2101a89640 1 -- 172.21.15.94:0/1401949405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee03a79e0 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f2102a8b640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee03a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210228a640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f2101a89640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee03a04c0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x55aee032b7d0 tx=0x7f20ec0109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d4d96cc8c9f36d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004220 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f20ec0043c0 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f80a6f60 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f20ec020400 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 10 client.23502 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55aee03a8d30 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f80a6f60 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f20ec003bc0 con 0x55aedff666b0 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 10 client.23502 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:14.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 10 client.23502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 20 client.23502 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff1b3a0 0x55aee03ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 4 client.23502 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.555+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aee0295800 con 0x55aedff1b3a0 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.556+0000 7f210228a640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff1b3a0 0x55aee03ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.556+0000 7f20e5ffb640 10 client.23502 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:14.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.556+0000 7f210228a640 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff1b3a0 0x55aee03ab740 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc195934982768f server_cookie=bc295e860f56b2ee in_seq=0 out_seq=0 2024-04-25T14:04:14.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f20e5ffb640 1 -- 172.21.15.94:0/1401949405 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aee0295800 con 0x55aedff1b3a0 2024-04-25T14:04:14.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f20e5ffb640 10 client.23502 handle_command_reply: tid=0 2024-04-25T14:04:14.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 client.23502 shutdown 2024-04-25T14:04:14.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 msgr2=0x55aee03a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee03a04c0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x55aee032b7d0 tx=0x7f20ec0109b0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 shutdown_connections 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55aedff1b3a0 0x55aee03ab740 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aedff87390 0x55aee03a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aedff666b0 0x55aee03a04c0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 --2- 172.21.15.94:0/1401949405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aedff5e4d0 0x55aee039ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f20e5ffb640 0 client.23502 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 >> 172.21.15.94:0/1401949405 conn(0x55aedff7b4e0 msgr2=0x55aee0333080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 shutdown_connections 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 1 -- 172.21.15.94:0/1401949405 wait complete. 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.560+0000 7f210b39d740 20 client.23502 trim_cache size 0 max 16384 2024-04-25T14:04:14.563 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:14.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 msgr2=0x7f20e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 0x7f20e80b2000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55aee032a860 tx=0x7f20f4009380 comp rx=0 tx=0).stop 2024-04-25T14:04:14.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 msgr2=0x7f20fc077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:14.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9f0 tx=0x7f20f800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 shutdown_connections 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f20e80afaf0 0x7f20e80b2000 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20fc109a30 0x7f20fc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20fc104d30 0x7f20fc076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 --2- 172.21.15.94:0/194821087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20fc103af0 0x7f20fc077960 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 >> 172.21.15.94:0/194821087 conn(0x7f20fc0fece0 msgr2=0x7f20fc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 shutdown_connections 2024-04-25T14:04:14.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:14.561+0000 7f21044ee640 1 -- 172.21.15.94:0/194821087 wait complete. 2024-04-25T14:04:15.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 190732 18.89 MB/sec execute 417 sec latency 61.856 ms 2024-04-25T14:04:15.573 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:04:15.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:15.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:04:15.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 -- 172.21.15.94:0/1429493992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc1033b0 msgr2=0x7f62cc1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 --2- 172.21.15.94:0/1429493992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc1033b0 0x7f62cc1037c0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009900 tx=0x7f62c002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:15.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 -- 172.21.15.94:0/1429493992 shutdown_connections 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 --2- 172.21.15.94:0/1429493992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc10a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 --2- 172.21.15.94:0/1429493992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc1033b0 0x7f62cc1037c0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 --2- 172.21.15.94:0/1429493992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc102170 0x7f62cc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 -- 172.21.15.94:0/1429493992 >> 172.21.15.94:0/1429493992 conn(0x7f62cc0fd340 msgr2=0x7f62cc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 -- 172.21.15.94:0/1429493992 shutdown_connections 2024-04-25T14:04:15.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.696+0000 7f62d1138640 1 -- 172.21.15.94:0/1429493992 wait complete. 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 Processor -- start 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 -- start start 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc1033b0 0x7f62cc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10f610 con 0x7f62cc102170 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10f490 con 0x7f62cc1080b0 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62d1138640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc10f790 con 0x7f62cc1033b0 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62cad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43740/0 (socket says 172.21.15.94:43740) 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62cad9d640 1 -- 172.21.15.94:0/1862710831 learned_addr learned my addr 172.21.15.94:0/1862710831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.697+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc1033b0 0x7f62cc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 -- 172.21.15.94:0/1862710831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc1033b0 msgr2=0x7f62cc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc1033b0 0x7f62cc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 -- 172.21.15.94:0/1862710831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 msgr2=0x7f62cc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 -- 172.21.15.94:0/1862710831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1a1d80 con 0x7f62cc102170 2024-04-25T14:04:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e9f0 tx=0x7f62bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62b3fff640 1 -- 172.21.15.94:0/1862710831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00ccc0 con 0x7f62cc102170 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62b3fff640 1 -- 172.21.15.94:0/1862710831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62bc0035f0 con 0x7f62cc102170 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1a2070 con 0x7f62cc102170 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62b3fff640 1 -- 172.21.15.94:0/1862710831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00ce60 con 0x7f62cc102170 2024-04-25T14:04:15.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.698+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1a2570 con 0x7f62cc102170 2024-04-25T14:04:15.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.701+0000 7f62d7feb740 1 Processor -- start 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.701+0000 7f62d7feb740 1 -- start start 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.701+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 0x55879a0e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.701+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62d7feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55879a04db00 con 0x558799d26730 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62d7feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558799fe9920 con 0x558799d47410 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62d7feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55879a049ac0 con 0x558799d1e550 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62cb59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 0x55879a0e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43744/0 (socket says 172.21.15.94:43744) 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 -- 172.21.15.94:0/461462761 learned_addr learned my addr 172.21.15.94:0/461462761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62b3fff640 1 -- 172.21.15.94:0/1862710831 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f62bc010690 con 0x7f62cc102170 2024-04-25T14:04:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 -- 172.21.15.94:0/461462761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 msgr2=0x55879a0e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 -- 172.21.15.94:0/461462761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 msgr2=0x55879a0e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 0x55879a0e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 -- 172.21.15.94:0/461462761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558799e277e0 con 0x558799d26730 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62cb59e640 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 0x55879a0e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62ca59c640 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f62c0031500 tx=0x7f62c0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb860e0d4df4cc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62b17fa640 1 -- 172.21.15.94:0/461462761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035030 con 0x558799d26730 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62b17fa640 1 -- 172.21.15.94:0/461462761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0040070 con 0x558799d26730 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62b17fa640 1 -- 172.21.15.94:0/461462761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0003970 con 0x558799d26730 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.702+0000 7f62b3fff640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 0x7f62b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 0x7f62b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62b3fff640 1 -- 172.21.15.94:0/1862710831 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f62bc0defc0 con 0x7f62cc102170 2024-04-25T14:04:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 -- 172.21.15.94:0/461462761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 msgr2=0x55879a0ef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 0x7f62b40b2000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55879a0e9670 tx=0x7f62c4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f62c0031500 tx=0x7f62c0009640 comp rx=0 tx=0).stop 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 -- 172.21.15.94:0/461462761 shutdown_connections 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d26730 0x55879a0ef920 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 --2- 172.21.15.94:0/461462761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d47410 0x55879a0e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 -- 172.21.15.94:0/461462761 >> 172.21.15.94:0/461462761 conn(0x558799d3b560 msgr2=0x55879a0e3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 -- 172.21.15.94:0/461462761 shutdown_connections 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.703+0000 7f62d7feb740 1 -- 172.21.15.94:0/461462761 wait complete. 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 -- start start 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 Processor -- start 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 0x55879a0f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55879a0f0310 con 0x558799d47410 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558799fe75d0 con 0x558799d26730 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55879a0f0490 con 0x558799d1e550 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62d7feb740 10 client.0 _ll_register_callbacks cb 0x558799fd5080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62cb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 0x55879a0f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57056/0 (socket says 172.21.15.94:57056) 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62ca59c640 1 -- 172.21.15.94:0/1492939161 learned_addr learned my addr 172.21.15.94:0/1492939161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.704+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 -- 172.21.15.94:0/1492939161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 msgr2=0x55879a0f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 -- 172.21.15.94:0/1492939161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 msgr2=0x55879a0f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 0x55879a0f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 -- 172.21.15.94:0/1492939161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558799e87f70 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 0x55879a0f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693fc4c48e212281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035030 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f62c00036f0 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0040070 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62c00038f0 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62d7feb740 10 client.32894 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.705+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558799d1f170 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0040070 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f62c0045b90 con 0x558799d26730 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 10 client.32894 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 10 client.32894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:15.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 20 client.32894 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d061f0 0x55879a0ff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 4 client.32894 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55879a05b710 con 0x558799d061f0 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d061f0 0x55879a0ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d061f0 0x55879a0ff830 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51af078bad75eb35 server_cookie=2d68e31ccc8a2f43 in_seq=0 out_seq=0 2024-04-25T14:04:15.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.706+0000 7f62aeff5640 10 client.32894 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.709+0000 7f62aeff5640 1 -- 172.21.15.94:0/1492939161 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55879a05b710 con 0x558799d061f0 2024-04-25T14:04:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.709+0000 7f62aeff5640 10 client.32894 handle_command_reply: tid=0 2024-04-25T14:04:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.709+0000 7f62d7feb740 1 client.32894 shutdown 2024-04-25T14:04:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 msgr2=0x55879a0f2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c00042f0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 shutdown_connections 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d061f0 0x55879a0ff830 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d47410 0x55879a0f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62aeff5640 0 client.32894 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a0f2810 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1492939161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d1e550 0x55879a0f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 >> 172.21.15.94:0/1492939161 conn(0x558799d3b560 msgr2=0x55879a0e2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 shutdown_connections 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 1 -- 172.21.15.94:0/1492939161 wait complete. 2024-04-25T14:04:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.710+0000 7f62d7feb740 20 client.32894 trim_cache size 0 max 16384 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 Processor -- start 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 -- start start 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d061f0 0x55879a0f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 0x55879a0f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1ab520 con 0x558799d1e550 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1ad060 con 0x558799d061f0 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62d7feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1abbf0 con 0x558799d26730 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43760/0 (socket says 172.21.15.94:43760) 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 -- 172.21.15.94:0/2644316128 learned_addr learned my addr 172.21.15.94:0/2644316128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62cad9d640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d061f0 0x55879a0f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62cb59e640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 0x55879a0f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 -- 172.21.15.94:0/2644316128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 msgr2=0x55879a0f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 0x55879a0f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 -- 172.21.15.94:0/2644316128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d061f0 msgr2=0x55879a0f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d061f0 0x55879a0f0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62ca59c640 1 -- 172.21.15.94:0/2644316128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a0ed1b0 con 0x558799d1e550 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.730+0000 7f62cb59e640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 0x55879a0f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.731+0000 7f62ca59c640 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f62c003c4e0 tx=0x7f62c0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f18c7646b32d492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.731+0000 7f62b0ff9640 1 -- 172.21.15.94:0/2644316128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c005b070 con 0x558799d1e550 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.731+0000 7f62b0ff9640 1 -- 172.21.15.94:0/2644316128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0040070 con 0x558799d1e550 2024-04-25T14:04:15.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.731+0000 7f62b0ff9640 1 -- 172.21.15.94:0/2644316128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035040 con 0x558799d1e550 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- 172.21.15.94:0/2644316128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 msgr2=0x55879a0e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f62c003c4e0 tx=0x7f62c0005e10 comp rx=0 tx=0).stop 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- 172.21.15.94:0/2644316128 shutdown_connections 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d26730 0x55879a0f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0e3f00 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 --2- 172.21.15.94:0/2644316128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d061f0 0x55879a0f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- 172.21.15.94:0/2644316128 >> 172.21.15.94:0/2644316128 conn(0x558799d3b560 msgr2=0x55879a0e37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- 172.21.15.94:0/2644316128 shutdown_connections 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- 172.21.15.94:0/2644316128 wait complete. 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 -- start start 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 Processor -- start 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.732+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d061f0 0x55879a0f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a165a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1ac990 con 0x558799d1e550 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1ac2c0 con 0x558799d26730 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1a9500 con 0x558799d061f0 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 10 client.0 _ll_register_callbacks cb 0x7f62c002ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43772/0 (socket says 172.21.15.94:43772) 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 -- 172.21.15.94:0/1323165093 learned_addr learned my addr 172.21.15.94:0/1323165093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62cb59e640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d061f0 0x55879a0f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 -- 172.21.15.94:0/1323165093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d061f0 msgr2=0x55879a0f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d061f0 0x55879a0f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 -- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 msgr2=0x55879a165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 -- 172.21.15.94:0/1323165093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879a0fa170 con 0x558799d1e550 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62ca59c640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395a02eaa383d70f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035070 con 0x558799d1e550 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f62c003cdf0 con 0x558799d1e550 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.733+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0040070 con 0x558799d1e550 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62c005b070 con 0x558799d1e550 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 10 client.23520 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:15.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55879a166e40 con 0x558799d1e550 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0040070 con 0x558799d1e550 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f62c006a4e0 con 0x558799d1e550 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 10 client.23520 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 10 client.23520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 20 client.23520 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d47410 0x55879a1695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 4 client.23520 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55879a055580 con 0x558799d47410 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.734+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d47410 0x55879a1695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.735+0000 7f62aeff5640 10 client.23520 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.735+0000 7f62cad9d640 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d47410 0x55879a1695d0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d387eb9e32dfa0c9 server_cookie=33428009d8bed8c5 in_seq=0 out_seq=0 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62aeff5640 1 -- 172.21.15.94:0/1323165093 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55879a055580 con 0x558799d47410 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62aeff5640 10 client.23520 handle_command_reply: tid=0 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 client.23520 shutdown 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 msgr2=0x55879a0f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f62c0006010 tx=0x7f62c0003d40 comp rx=0 tx=0).stop 2024-04-25T14:04:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 shutdown_connections 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558799d47410 0x55879a1695d0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558799d26730 0x55879a165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558799d1e550 0x55879a0f71b0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62aeff5640 0 client.23520 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 --2- 172.21.15.94:0/1323165093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558799d061f0 0x55879a0f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 >> 172.21.15.94:0/1323165093 conn(0x558799d3b560 msgr2=0x55879a103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 shutdown_connections 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 1 -- 172.21.15.94:0/1323165093 wait complete. 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d7feb740 20 client.23520 trim_cache size 0 max 16384 2024-04-25T14:04:15.747 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:04:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "10af473f-c909-4092-b791-39f98b6fa19b", 2024-04-25T14:04:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:04:15.748 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 msgr2=0x7f62b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 0x7f62b40b2000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55879a0e9670 tx=0x7f62c4009500 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 msgr2=0x7f62cc19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.746+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e9f0 tx=0x7f62bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 shutdown_connections 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f62b40afaf0 0x7f62b40b2000 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc1080b0 0x7f62cc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62cc1033b0 0x7f62cc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 --2- 172.21.15.94:0/1862710831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62cc102170 0x7f62cc19a240 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 >> 172.21.15.94:0/1862710831 conn(0x7f62cc0fd340 msgr2=0x7f62cc104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 shutdown_connections 2024-04-25T14:04:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.747+0000 7f62d1138640 1 -- 172.21.15.94:0/1862710831 wait complete. 2024-04-25T14:04:15.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:15.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.874+0000 7fceb8dba640 1 -- 172.21.15.94:0/79969399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 msgr2=0x7fceb41040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.874+0000 7fceb8dba640 1 --2- 172.21.15.94:0/79969399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb41040a0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009900 tx=0x7fce9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 -- 172.21.15.94:0/79969399 shutdown_connections 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 --2- 172.21.15.94:0/79969399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4108990 0x7fceb410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 --2- 172.21.15.94:0/79969399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb41040a0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 --2- 172.21.15.94:0/79969399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4102a50 0x7fceb4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 -- 172.21.15.94:0/79969399 >> 172.21.15.94:0/79969399 conn(0x7fceb40fdc40 msgr2=0x7fceb41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 -- 172.21.15.94:0/79969399 shutdown_connections 2024-04-25T14:04:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 -- 172.21.15.94:0/79969399 wait complete. 2024-04-25T14:04:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 Processor -- start 2024-04-25T14:04:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.875+0000 7fceb8dba640 1 -- start start 2024-04-25T14:04:15.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4102a50 0x7fceb4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4108990 0x7fceb41a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410e510 con 0x7fceb4103c90 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410e390 con 0x7fceb4108990 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410e690 con 0x7fceb4102a50 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43778/0 (socket says 172.21.15.94:43778) 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 -- 172.21.15.94:0/3687673990 learned_addr learned my addr 172.21.15.94:0/3687673990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb259c640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4102a50 0x7fceb4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4108990 0x7fceb41a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 -- 172.21.15.94:0/3687673990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4102a50 msgr2=0x7fceb4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4102a50 0x7fceb4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 -- 172.21.15.94:0/3687673990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4108990 msgr2=0x7fceb41a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4108990 0x7fceb41a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 -- 172.21.15.94:0/3687673990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb41a1d30 con 0x7fceb4103c90 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceabfff640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006010 tx=0x7fce9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceab7fe640 1 -- 172.21.15.94:0/3687673990 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c03f030 con 0x7fceb4103c90 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb41a1fc0 con 0x7fceb4103c90 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceab7fe640 1 -- 172.21.15.94:0/3687673990 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c031910 con 0x7fceb4103c90 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceab7fe640 1 -- 172.21.15.94:0/3687673990 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c035070 con 0x7fceb4103c90 2024-04-25T14:04:15.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.876+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a23d0 con 0x7fceb4103c90 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.879+0000 7fcebfc6a740 1 Processor -- start 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.879+0000 7fcebfc6a740 1 -- start start 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceab7fe640 1 -- 172.21.15.94:0/3687673990 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fce9c042030 con 0x7fceb4103c90 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e6fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55855e664d80 con 0x55855e33d6b0 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55855e61e8b0 con 0x55855e3354d0 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55855e660d40 con 0x55855e35e390 2024-04-25T14:04:15.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceb259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e6fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43780/0 (socket says 172.21.15.94:43780) 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 -- 172.21.15.94:0/3623372372 learned_addr learned my addr 172.21.15.94:0/3623372372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 -- 172.21.15.94:0/3623372372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 msgr2=0x55855e6ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 -- 172.21.15.94:0/3623372372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 msgr2=0x55855e6fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e6fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 -- 172.21.15.94:0/3623372372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855e5efa40 con 0x55855e33d6b0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceab7fe640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 0x7fcea00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceabfff640 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fce9c107480 tx=0x7fce9c107570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99635f1c309dda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.880+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 0x7fcea00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.881+0000 7fceab7fe640 1 -- 172.21.15.94:0/3687673990 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fce9c0ff090 con 0x7fceb4103c90 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.881+0000 7fcea8ff9640 1 -- 172.21.15.94:0/3623372372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0c34e0 con 0x55855e33d6b0 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.881+0000 7fcea8ff9640 1 -- 172.21.15.94:0/3623372372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0c7130 con 0x55855e33d6b0 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.881+0000 7fcea8ff9640 1 -- 172.21.15.94:0/3623372372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0c6d90 con 0x55855e33d6b0 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- 172.21.15.94:0/3623372372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 msgr2=0x55855e700c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 0x7fcea00b2000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55855e700ab0 tx=0x7fceac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fce9c107480 tx=0x7fce9c107570 comp rx=0 tx=0).stop 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- 172.21.15.94:0/3623372372 shutdown_connections 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e700c70 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e6fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/3623372372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- 172.21.15.94:0/3623372372 >> 172.21.15.94:0/3623372372 conn(0x55855e3524e0 msgr2=0x55855e6fb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- 172.21.15.94:0/3623372372 shutdown_connections 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- 172.21.15.94:0/3623372372 wait complete. 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- start start 2024-04-25T14:04:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 Processor -- start 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e709600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e710aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55855e701540 con 0x55855e33d6b0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55855e5ebd10 con 0x55855e3354d0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55855e7016c0 con 0x55855e35e390 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.882+0000 7fcebfc6a740 10 client.0 _ll_register_callbacks cb 0x55855e618210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43796/0 (socket says 172.21.15.94:43796) 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 -- 172.21.15.94:0/667493381 learned_addr learned my addr 172.21.15.94:0/667493381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e710aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceb259c640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e709600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 -- 172.21.15.94:0/667493381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 msgr2=0x55855e710aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e710aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 -- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 msgr2=0x55855e709600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e709600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 -- 172.21.15.94:0/667493381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855e4407d0 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e710aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fceabfff640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fce9c03b610 tx=0x7fce9c102770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebd01b79a84085f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fce99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c049070 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fce9c003db0 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c0c7130 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fce9c0c6f70 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fcebfc6a740 10 client.23532 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55855e3763f0 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.883+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0c7130 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fce9c0c7880 con 0x55855e33d6b0 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 10 client.23532 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 10 client.23532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 20 client.23532 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e31cf60 0x55855e716d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 4 client.23532 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855e672990 con 0x55855e31cf60 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fceb259c640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e31cf60 0x55855e716d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fce99ffb640 10 client.23532 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.884+0000 7fceb259c640 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e31cf60 0x55855e716d10 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f8a0e51e367a11 server_cookie=6ffe5ba3886ce2e2 in_seq=0 out_seq=0 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fce99ffb640 1 -- 172.21.15.94:0/667493381 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55855e672990 con 0x55855e31cf60 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fce99ffb640 10 client.23532 handle_command_reply: tid=0 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 client.23532 shutdown 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 msgr2=0x55855e709b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fce9c03b610 tx=0x7fce9c102770 comp rx=0 tx=0).stop 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 shutdown_connections 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e31cf60 0x55855e716d10 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e710aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e33d6b0 0x55855e709b40 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fce99ffb640 0 client.23532 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/667493381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e3354d0 0x55855e709600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.887+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 >> 172.21.15.94:0/667493381 conn(0x55855e3524e0 msgr2=0x55855e6fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.888+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 shutdown_connections 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.888+0000 7fcebfc6a740 1 -- 172.21.15.94:0/667493381 wait complete. 2024-04-25T14:04:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.888+0000 7fcebfc6a740 20 client.23532 trim_cache size 0 max 16384 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 Processor -- start 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 -- start start 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e6fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ab540 con 0x55855e3354d0 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ad080 con 0x55855e33d6b0 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41abc10 con 0x55855e35e390 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fceb259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fceb259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43806/0 (socket says 172.21.15.94:43806) 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fceb259c640 1 -- 172.21.15.94:0/2984304564 learned_addr learned my addr 172.21.15.94:0/2984304564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fceabfff640 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e6fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.911+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 -- 172.21.15.94:0/2984304564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 msgr2=0x55855e6feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 -- 172.21.15.94:0/2984304564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 msgr2=0x55855e6fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e6fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 -- 172.21.15.94:0/2984304564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855e3360f0 con 0x55855e3354d0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fceb259c640 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fcea4000c00 tx=0x7fcea4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafb4b5c2437f1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fce9bfff640 1 -- 172.21.15.94:0/2984304564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400f890 con 0x55855e3354d0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fce9bfff640 1 -- 172.21.15.94:0/2984304564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea401e420 con 0x55855e3354d0 2024-04-25T14:04:15.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.912+0000 7fce9bfff640 1 -- 172.21.15.94:0/2984304564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea401f2b0 con 0x55855e3354d0 2024-04-25T14:04:15.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2984304564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 msgr2=0x55855e6fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fcea4000c00 tx=0x7fcea4005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:15.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2984304564 shutdown_connections 2024-04-25T14:04:15.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e6feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e6fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2984304564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e6fe070 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2984304564 >> 172.21.15.94:0/2984304564 conn(0x55855e3524e0 msgr2=0x55855e707c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2984304564 shutdown_connections 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2984304564 wait complete. 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 -- start start 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.913+0000 7fcebfc6a740 1 Processor -- start 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e70f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e70ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ac9b0 con 0x55855e3354d0 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ac2e0 con 0x55855e33d6b0 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41a9520 con 0x55855e35e390 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 10 client.0 _ll_register_callbacks cb 0x7fcea40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:15.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43808/0 (socket says 172.21.15.94:43808) 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 -- 172.21.15.94:0/2213741352 learned_addr learned my addr 172.21.15.94:0/2213741352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceabfff640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e70f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb2d9d640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e70ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 -- 172.21.15.94:0/2213741352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 msgr2=0x55855e70ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e70ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 -- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 msgr2=0x55855e70f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e70f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 -- 172.21.15.94:0/2213741352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855e6254b0 con 0x55855e3354d0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fceb259c640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009e50 tx=0x7fcea40104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f02c24d404bd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fce99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4010e10 con 0x55855e3354d0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcea401e7e0 con 0x55855e3354d0 2024-04-25T14:04:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.914+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea401e420 con 0x55855e3354d0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcea4038050 con 0x55855e3354d0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 10 client.23538 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55855e77f060 con 0x55855e3354d0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea401e420 con 0x55855e3354d0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcea400fcd0 con 0x55855e3354d0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 10 client.23538 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 10 client.23538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 20 client.23538 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e2f23a0 0x55855e7818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 4 client.23538 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55855e66c800 con 0x55855e2f23a0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.915+0000 7fceabfff640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e2f23a0 0x55855e7818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.916+0000 7fce99ffb640 10 client.23538 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.916+0000 7fceabfff640 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e2f23a0 0x55855e7818c0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02f91aac54d5e58 server_cookie=ac78b1509da230df in_seq=0 out_seq=0 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fce99ffb640 1 -- 172.21.15.94:0/2213741352 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55855e66c800 con 0x55855e2f23a0 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fce99ffb640 10 client.23538 handle_command_reply: tid=0 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 client.23538 shutdown 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 msgr2=0x55855e70ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009e50 tx=0x7fcea40104d0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 shutdown_connections 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55855e2f23a0 0x55855e7818c0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855e35e390 0x55855e70ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55855e33d6b0 0x55855e70f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fce99ffb640 0 client.23538 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 --2- 172.21.15.94:0/2213741352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855e3354d0 0x55855e70ebb0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 >> 172.21.15.94:0/2213741352 conn(0x55855e3524e0 msgr2=0x55855e6fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 shutdown_connections 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 1 -- 172.21.15.94:0/2213741352 wait complete. 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.920+0000 7fcebfc6a740 20 client.23538 trim_cache size 0 max 16384 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (476 inodes in the stack)", 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "10af473f-c909-4092-b791-39f98b6fa19b": { 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "10af473f-c909-4092-b791-39f98b6fa19b", 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:04:15.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:04:15.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:04:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:15.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 msgr2=0x7fcea00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 0x7fcea00b2000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55855e700ab0 tx=0x7fceac009500 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 msgr2=0x7fceb419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006010 tx=0x7fce9c0042f0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 shutdown_connections 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcea00afaf0 0x7fcea00b2000 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb4108990 0x7fceb41a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb4103c90 0x7fceb419efc0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 --2- 172.21.15.94:0/3687673990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4102a50 0x7fceb4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 >> 172.21.15.94:0/3687673990 conn(0x7fceb40fdc40 msgr2=0x7fceb4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 shutdown_connections 2024-04-25T14:04:15.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:15.921+0000 7fceb8dba640 1 -- 172.21.15.94:0/3687673990 wait complete. 2024-04-25T14:04:15.932 INFO:tasks.cephfs.filesystem:scrub status for tag:10af473f-c909-4092-b791-39f98b6fa19b - {'path': '/', 'tag': '10af473f-c909-4092-b791-39f98b6fa19b', 'options': 'recursive,force'} 2024-04-25T14:04:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 -- 172.21.15.94:0/4278044003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 msgr2=0x7f4fd810ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/4278044003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd810ad80 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009ca0 tx=0x7f4fd002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 -- 172.21.15.94:0/4278044003 shutdown_connections 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/4278044003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd810ad80 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/4278044003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/4278044003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 0x7f4fd8103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 -- 172.21.15.94:0/4278044003 >> 172.21.15.94:0/4278044003 conn(0x7f4fd80fdc10 msgr2=0x7f4fd81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 -- 172.21.15.94:0/4278044003 shutdown_connections 2024-04-25T14:04:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.120+0000 7f4fdcfce640 1 -- 172.21.15.94:0/4278044003 wait complete. 2024-04-25T14:04:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 Processor -- start 2024-04-25T14:04:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 -- start start 2024-04-25T14:04:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 0x7f4fd80ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd80fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810e5a0 con 0x7f4fd8108960 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810e420 con 0x7f4fd8102a20 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fdcfce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810e720 con 0x7f4fd8103c60 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fd6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 0x7f4fd80ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.121+0000 7f4fd6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43816/0 (socket says 172.21.15.94:43816) 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd80fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 -- 172.21.15.94:0/3023025750 learned_addr learned my addr 172.21.15.94:0/3023025750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 -- 172.21.15.94:0/3023025750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 msgr2=0x7f4fd80fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd80fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 -- 172.21.15.94:0/3023025750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 msgr2=0x7f4fd80ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 0x7f4fd80ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 -- 172.21.15.94:0/3023025750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd80fedb0 con 0x7f4fd8108960 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd80fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fd6d9d640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009430 tx=0x7f4fd00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fc77fe640 1 -- 172.21.15.94:0/3023025750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0003b70 con 0x7f4fd8108960 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a42a0 con 0x7f4fd8108960 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fc77fe640 1 -- 172.21.15.94:0/3023025750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0003d10 con 0x7f4fd8108960 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fc77fe640 1 -- 172.21.15.94:0/3023025750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd002fb40 con 0x7f4fd8108960 2024-04-25T14:04:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.122+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a4760 con 0x7f4fd8108960 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fc77fe640 1 -- 172.21.15.94:0/3023025750 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4fd0004610 con 0x7f4fd8108960 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 Processor -- start 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 -- start start 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 0x557a58bf7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557a58b5d460 con 0x557a588364d0 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a58b17590 con 0x557a588571b0 2024-04-25T14:04:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a58b59420 con 0x557a5882e2f0 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 0x557a58bf7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43828/0 (socket says 172.21.15.94:43828) 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.125+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/3392213535 learned_addr learned my addr 172.21.15.94:0/3392213535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fc77fe640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 0x7f4fc00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/3392213535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 msgr2=0x557a58bf7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 0x557a58bf7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/3392213535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 msgr2=0x557a58bf7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/3392213535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a58b240d0 con 0x557a588364d0 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd659c640 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 0x557a58bf7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd6d9d640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 0x7f4fc00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004630 tx=0x7f4fcc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d21328908766ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fc77fe640 1 -- 172.21.15.94:0/3023025750 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4fd01000b0 con 0x7f4fd8108960 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fc4ff9640 1 -- 172.21.15.94:0/3392213535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00fc50 con 0x557a588364d0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fc4ff9640 1 -- 172.21.15.94:0/3392213535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc00fdf0 con 0x557a588364d0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.126+0000 7f4fc4ff9640 1 -- 172.21.15.94:0/3392213535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00e5b0 con 0x557a588364d0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 -- 172.21.15.94:0/3392213535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 msgr2=0x557a58bfedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004630 tx=0x7f4fcc009b80 comp rx=0 tx=0).stop 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fd6d9d640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 0x7f4fc00b1ff0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00c8480 tx=0x7f4fd0093c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 -- 172.21.15.94:0/3392213535 shutdown_connections 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a588364d0 0x557a58bfedf0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a5882e2f0 0x557a58bf7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/3392213535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 -- 172.21.15.94:0/3392213535 >> 172.21.15.94:0/3392213535 conn(0x557a5884b300 msgr2=0x557a58bf3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 -- 172.21.15.94:0/3392213535 shutdown_connections 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.127+0000 7f4fe3e81740 1 -- 172.21.15.94:0/3392213535 wait complete. 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 -- start start 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 Processor -- start 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588364d0 0x557a58c01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588571b0 0x557a58c08b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557a58bff840 con 0x557a5882e2f0 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557a58ae4b60 con 0x557a588364d0 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a58bff9c0 con 0x557a588571b0 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fe3e81740 10 client.0 _ll_register_callbacks cb 0x557a58b092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43834/0 (socket says 172.21.15.94:43834) 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/760573269 learned_addr learned my addr 172.21.15.94:0/760573269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd659c640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588364d0 0x557a58c01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588571b0 0x557a58c08b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/760573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588571b0 msgr2=0x557a58c08b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588571b0 0x557a58c08b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588364d0 msgr2=0x557a58c01bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588364d0 0x557a58c01bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.128+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/760573269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a5886f210 con 0x557a5882e2f0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc01fa60 tx=0x7f4fcc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4983fa47c93a412c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc005050 con 0x557a5882e2f0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4fcc019040 con 0x557a5882e2f0 2024-04-25T14:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fcc00fdf0 con 0x557a5882e2f0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc035050 con 0x557a5882e2f0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 10 client.23550 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a5893b620 con 0x557a5882e2f0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc00fdf0 con 0x557a5882e2f0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4fcc01d020 con 0x557a5882e2f0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 10 client.23550 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 10 client.23550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 20 client.23550 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a58815d80 0x557a58c0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 4 client.23550 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.129+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a58b6b070 con 0x557a58815d80 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.130+0000 7f4fd659c640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a58815d80 0x557a58c0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.130+0000 7f4fd659c640 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a58815d80 0x557a58c0f100 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2801ec688cdfef server_cookie=7f2c99111539fe41 in_seq=0 out_seq=0 2024-04-25T14:04:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.130+0000 7f4fbdffb640 10 client.23550 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fbdffb640 1 -- 172.21.15.94:0/760573269 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a58b6b070 con 0x557a58815d80 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fbdffb640 10 client.23550 handle_command_reply: tid=0 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 client.23550 shutdown 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 msgr2=0x557a58c016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc01fa60 tx=0x7f4fcc005850 comp rx=0 tx=0).stop 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 shutdown_connections 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a58815d80 0x557a58c0f100 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588571b0 0x557a58c08b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588364d0 0x557a58c01bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/760573269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c016b0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fbdffb640 0 client.23550 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 >> 172.21.15.94:0/760573269 conn(0x557a5884b300 msgr2=0x557a58bf6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 shutdown_connections 2024-04-25T14:04:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 1 -- 172.21.15.94:0/760573269 wait complete. 2024-04-25T14:04:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.136+0000 7f4fe3e81740 20 client.23550 trim_cache size 0 max 16384 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 Processor -- start 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 -- start start 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58bf6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ad690 con 0x557a5882e2f0 2024-04-25T14:04:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81af1d0 con 0x557a588571b0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81add60 con 0x557a588364d0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43842/0 (socket says 172.21.15.94:43842) 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/852980867 learned_addr learned my addr 172.21.15.94:0/852980867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fd659c640 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58bf6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.152+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/852980867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 msgr2=0x557a58bf6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58bf6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/852980867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 msgr2=0x557a58bf2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/852980867 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a58bee9f0 con 0x557a5882e2f0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc00e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4904afb4817c9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fbffff640 1 -- 172.21.15.94:0/852980867 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc01fcd0 con 0x557a5882e2f0 2024-04-25T14:04:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fbffff640 1 -- 172.21.15.94:0/852980867 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc00fd40 con 0x557a5882e2f0 2024-04-25T14:04:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.153+0000 7f4fbffff640 1 -- 172.21.15.94:0/852980867 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00e990 con 0x557a5882e2f0 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- 172.21.15.94:0/852980867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 msgr2=0x557a58bf6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc00e430 comp rx=0 tx=0).stop 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- 172.21.15.94:0/852980867 shutdown_connections 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58bf2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58bf6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/852980867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58bf6100 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- 172.21.15.94:0/852980867 >> 172.21.15.94:0/852980867 conn(0x557a5884b300 msgr2=0x557a58bfbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- 172.21.15.94:0/852980867 shutdown_connections 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- 172.21.15.94:0/852980867 wait complete. 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 -- start start 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.154+0000 7f4fe3e81740 1 Processor -- start 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58c05b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58c06a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81aeb00 con 0x557a5882e2f0 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ae430 con 0x557a588571b0 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81ab670 con 0x557a588364d0 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fe3e81740 10 client.0 _ll_register_callbacks cb 0x7f4fcc0116f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43850/0 (socket says 172.21.15.94:43850) 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/123925222 learned_addr learned my addr 172.21.15.94:0/123925222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd659c640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58c05b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58c06a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/123925222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 msgr2=0x557a58c05b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58c05b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 msgr2=0x557a58c06a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58c06a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 -- 172.21.15.94:0/123925222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a58c06fc0 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fd5d9b640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005890 tx=0x7f4fcc0114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6703714c8a2009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.155+0000 7f4fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00e640 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4fcc00e7e0 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fcc00fd40 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc04d050 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 10 client.23556 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557a58c08250 con 0x557a5882e2f0 2024-04-25T14:04:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc00fd40 con 0x557a5882e2f0 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4fcc044070 con 0x557a5882e2f0 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 10 client.23556 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 10 client.23556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 20 client.23556 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a587eb1c0 0x557a58c799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 4 client.23556 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a58b64ee0 con 0x557a587eb1c0 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.156+0000 7f4fd659c640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a587eb1c0 0x557a58c799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.157+0000 7f4fd659c640 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a587eb1c0 0x557a58c799d0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b031005604374e server_cookie=910b223d9fbfbe99 in_seq=0 out_seq=0 2024-04-25T14:04:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.157+0000 7f4fbdffb640 10 client.23556 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:16.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 191221 18.91 MB/sec execute 418 sec latency 72.244 ms 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: pgmap v1025: 97 pgs: 97 active+clean; 277 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:16.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:15 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: pgmap v1025: 97 pgs: 97 active+clean; 277 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-04-25T14:04:16.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:16.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:15 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: pgmap v1025: 97 pgs: 97 active+clean; 277 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-04-25T14:04:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:15 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fbdffb640 1 -- 172.21.15.94:0/123925222 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a58b64ee0 con 0x557a587eb1c0 2024-04-25T14:04:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fbdffb640 10 client.23556 handle_command_reply: tid=0 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 client.23556 shutdown 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 msgr2=0x557a58c055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005890 tx=0x7f4fcc0114c0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 shutdown_connections 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x557a587eb1c0 0x557a58c799d0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557a588571b0 0x557a58c06a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a588364d0 0x557a58c05b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fbdffb640 0 client.23556 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 --2- 172.21.15.94:0/123925222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a5882e2f0 0x557a58c055e0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 >> 172.21.15.94:0/123925222 conn(0x557a5884b300 msgr2=0x557a58bf6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 shutdown_connections 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 1 -- 172.21.15.94:0/123925222 wait complete. 2024-04-25T14:04:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.887+0000 7f4fe3e81740 20 client.23556 trim_cache size 0 max 16384 2024-04-25T14:04:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:04:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:04:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:04:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 msgr2=0x7f4fc00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 0x7f4fc00b1ff0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00c8480 tx=0x7f4fd0093c40 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 msgr2=0x7f4fd80fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009430 tx=0x7f4fd00316f0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 shutdown_connections 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4fc00afae0 0x7f4fc00b1ff0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fd8108960 0x7f4fd80fe680 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fd8103c60 0x7f4fd80fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 --2- 172.21.15.94:0/3023025750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fd8102a20 0x7f4fd80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 >> 172.21.15.94:0/3023025750 conn(0x7f4fd80fdc10 msgr2=0x7f4fd8109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:16.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 shutdown_connections 2024-04-25T14:04:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:16.888+0000 7f4fdcfce640 1 -- 172.21.15.94:0/3023025750 wait complete. 2024-04-25T14:04:17.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 191316 18.87 MB/sec execute 419 sec latency 374.573 ms 2024-04-25T14:04:17.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:17.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:16 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:17.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:16 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:16 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:18.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 191635 18.88 MB/sec execute 420 sec latency 268.901 ms 2024-04-25T14:04:18.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:17 smithi138 ceph-mon[32977]: pgmap v1026: 97 pgs: 97 active+clean; 278 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 395 op/s 2024-04-25T14:04:18.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:17 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:17 smithi137 ceph-mon[32568]: pgmap v1026: 97 pgs: 97 active+clean; 278 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 395 op/s 2024-04-25T14:04:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:17 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:17 smithi094 ceph-mon[29195]: pgmap v1026: 97 pgs: 97 active+clean; 278 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 395 op/s 2024-04-25T14:04:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:17 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:19.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 192051 18.89 MB/sec execute 421 sec latency 107.109 ms 2024-04-25T14:04:20.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 192441 18.89 MB/sec execute 422 sec latency 91.618 ms 2024-04-25T14:04:20.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:19 smithi138 ceph-mon[32977]: pgmap v1027: 97 pgs: 97 active+clean; 275 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T14:04:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:19 smithi137 ceph-mon[32568]: pgmap v1027: 97 pgs: 97 active+clean; 275 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T14:04:20.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:19 smithi094 ceph-mon[29195]: pgmap v1027: 97 pgs: 97 active+clean; 275 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.4 MiB/s wr, 337 op/s 2024-04-25T14:04:21.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 192789 18.90 MB/sec execute 423 sec latency 114.679 ms 2024-04-25T14:04:22.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 193139 18.90 MB/sec execute 424 sec latency 59.960 ms 2024-04-25T14:04:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:21 smithi138 ceph-mon[32977]: pgmap v1028: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-04-25T14:04:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:21 smithi137 ceph-mon[32568]: pgmap v1028: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-04-25T14:04:22.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:21 smithi094 ceph-mon[29195]: pgmap v1028: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-04-25T14:04:23.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 193388 18.89 MB/sec execute 425 sec latency 220.442 ms 2024-04-25T14:04:24.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 193717 18.87 MB/sec execute 426 sec latency 82.716 ms 2024-04-25T14:04:24.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:23 smithi138 ceph-mon[32977]: pgmap v1029: 97 pgs: 97 active+clean; 267 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 553 op/s 2024-04-25T14:04:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:23 smithi137 ceph-mon[32568]: pgmap v1029: 97 pgs: 97 active+clean; 267 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 553 op/s 2024-04-25T14:04:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:23 smithi094 ceph-mon[29195]: pgmap v1029: 97 pgs: 97 active+clean; 267 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 553 op/s 2024-04-25T14:04:25.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 194073 18.87 MB/sec execute 427 sec latency 101.578 ms 2024-04-25T14:04:26.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 194448 18.86 MB/sec execute 428 sec latency 123.319 ms 2024-04-25T14:04:26.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:25 smithi138 ceph-mon[32977]: pgmap v1030: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-25T14:04:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:25 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:26.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:25 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:25 smithi137 ceph-mon[32568]: pgmap v1030: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-25T14:04:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:25 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:25 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:25 smithi094 ceph-mon[29195]: pgmap v1030: 97 pgs: 97 active+clean; 271 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-25T14:04:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:25 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:25 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:27.157 INFO:tasks.workunit.client.0.smithi094.stdout: 10 194871 18.87 MB/sec execute 429 sec latency 100.679 ms 2024-04-25T14:04:27.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:27.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:27.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:26 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:27.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:27.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:27.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:26 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:27.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:27.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:27.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:26 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:28.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 195160 18.88 MB/sec execute 430 sec latency 164.104 ms 2024-04-25T14:04:28.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:27 smithi138 ceph-mon[32977]: pgmap v1031: 97 pgs: 97 active+clean; 276 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-04-25T14:04:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:27 smithi137 ceph-mon[32568]: pgmap v1031: 97 pgs: 97 active+clean; 276 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-04-25T14:04:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:27 smithi094 ceph-mon[29195]: pgmap v1031: 97 pgs: 97 active+clean; 276 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-04-25T14:04:29.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 195542 18.89 MB/sec execute 431 sec latency 88.652 ms 2024-04-25T14:04:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 -- 172.21.15.94:0/2661123403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 msgr2=0x7f992010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 --2- 172.21.15.94:0/2661123403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f992010abf0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f9918009ca0 tx=0x7f991802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 -- 172.21.15.94:0/2661123403 shutdown_connections 2024-04-25T14:04:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 --2- 172.21.15.94:0/2661123403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f992010abf0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 --2- 172.21.15.94:0/2661123403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 0x7f9920103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 --2- 172.21.15.94:0/2661123403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 0x7f9920103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 -- 172.21.15.94:0/2661123403 >> 172.21.15.94:0/2661123403 conn(0x7f99200fd390 msgr2=0x7f99200ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 -- 172.21.15.94:0/2661123403 shutdown_connections 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.988+0000 7f99255a8640 1 -- 172.21.15.94:0/2661123403 wait complete. 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 Processor -- start 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 -- start start 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 0x7f9920102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 0x7f992019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f992010e5d0 con 0x7f99201044b0 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f992010e450 con 0x7f99201039d0 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f99255a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f992010e750 con 0x7f9920102ff0 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f991f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51316/0 (socket says 172.21.15.94:51316) 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f991f7fe640 1 -- 172.21.15.94:0/75851271 learned_addr learned my addr 172.21.15.94:0/75851271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f991effd640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 0x7f9920102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.989+0000 7f991e7fc640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 0x7f992019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 -- 172.21.15.94:0/75851271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 msgr2=0x7f9920102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 0x7f9920102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 -- 172.21.15.94:0/75851271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 msgr2=0x7f992019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 0x7f992019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 -- 172.21.15.94:0/75851271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99201a1d00 con 0x7f99201044b0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f991f7fe640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f9918009430 tx=0x7f99180316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f9907fff640 1 -- 172.21.15.94:0/75851271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918003b70 con 0x7f99201044b0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201a1f90 con 0x7f99201044b0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f9907fff640 1 -- 172.21.15.94:0/75851271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9918003d10 con 0x7f99201044b0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f9907fff640 1 -- 172.21.15.94:0/75851271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991802fb40 con 0x7f99201044b0 2024-04-25T14:04:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.990+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a23f0 con 0x7f99201044b0 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f9907fff640 1 -- 172.21.15.94:0/75851271 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9918004610 con 0x7f99201044b0 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 Processor -- start 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 -- start start 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513ee4d0 0x55b6517b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b651715460 con 0x55b65140f1b0 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b6516cf590 con 0x55b6513e62f0 2024-04-25T14:04:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f992c45b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b651711420 con 0x55b6513ee4d0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f9907fff640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 0x7f99080b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51326/0 (socket says 172.21.15.94:51326) 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991e7fc640 1 -- 172.21.15.94:0/2715680508 learned_addr learned my addr 172.21.15.94:0/2715680508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991f7fe640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 0x7f99080b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991effd640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.993+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513ee4d0 0x55b6517b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f9907fff640 1 -- 172.21.15.94:0/75851271 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99181000b0 con 0x7f99201044b0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 -- 172.21.15.94:0/2715680508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513ee4d0 msgr2=0x55b6517b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513ee4d0 0x55b6517b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 -- 172.21.15.94:0/2715680508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 msgr2=0x55b6517afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 -- 172.21.15.94:0/2715680508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6516dc0d0 con 0x55b65140f1b0 2024-04-25T14:04:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991effd640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991f7fe640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 0x7f99080b1ff0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f99180c5760 tx=0x7f9918092d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.994+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f9914013820 tx=0x7f99140097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ae01e7636b8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.996+0000 7f99057fa640 1 -- 172.21.15.94:0/2715680508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914013da0 con 0x55b65140f1b0 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.996+0000 7f99057fa640 1 -- 172.21.15.94:0/2715680508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914013170 con 0x55b65140f1b0 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.996+0000 7f99057fa640 1 -- 172.21.15.94:0/2715680508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914004d10 con 0x55b65140f1b0 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.996+0000 7f992c45b740 1 -- 172.21.15.94:0/2715680508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 msgr2=0x55b6517af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.996+0000 7f992c45b740 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f9914013820 tx=0x7f99140097a0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- 172.21.15.94:0/2715680508 shutdown_connections 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513ee4d0 0x55b6517b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- 172.21.15.94:0/2715680508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517af970 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- 172.21.15.94:0/2715680508 >> 172.21.15.94:0/2715680508 conn(0x55b651403300 msgr2=0x55b6517ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- 172.21.15.94:0/2715680508 shutdown_connections 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- 172.21.15.94:0/2715680508 wait complete. 2024-04-25T14:04:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- start start 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 Processor -- start 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513e62f0 0x55b6517b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b6517b77e0 con 0x55b65140f1b0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b65169cb60 con 0x55b6513ee4d0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6517b7960 con 0x55b6513e62f0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f992c45b740 10 client.0 _ll_register_callbacks cb 0x55b6516c12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51330/0 (socket says 172.21.15.94:51330) 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f991e7fc640 1 -- 172.21.15.94:0/2503818817 learned_addr learned my addr 172.21.15.94:0/2503818817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f991effd640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.997+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513e62f0 0x55b6517b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 -- 172.21.15.94:0/2503818817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513e62f0 msgr2=0x55b6517b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513e62f0 0x55b6517b95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 -- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 msgr2=0x55b6517b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 -- 172.21.15.94:0/2503818817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b651427210 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f991e7fc640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f9914013e20 tx=0x7f9914018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5410e0bcefe38dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f9902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914005460 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9914013170 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9914004020 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9914004220 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f992c45b740 10 client.23568 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b6514f3620 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.998+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914013170 con 0x55b65140f1b0 2024-04-25T14:04:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f99140035f0 con 0x55b65140f1b0 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 10 client.23568 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 10 client.23568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 20 client.23568 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513cdd80 0x55b6517c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 4 client.23568 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b651723070 con 0x55b6513cdd80 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f991effd640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513cdd80 0x55b6517c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f9902ff5640 10 client.23568 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:29.999+0000 7f991effd640 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513cdd80 0x55b6517c7090 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cd06bd85b7a0a2 server_cookie=7783308eadc6f92c in_seq=0 out_seq=0 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.005+0000 7f9902ff5640 1 -- 172.21.15.94:0/2503818817 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b651723070 con 0x55b6513cdd80 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.005+0000 7f9902ff5640 10 client.23568 handle_command_reply: tid=0 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 client.23568 shutdown 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 msgr2=0x55b6517c0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f9914013e20 tx=0x7f9914018040 comp rx=0 tx=0).stop 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 shutdown_connections 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513cdd80 0x55b6517c7090 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b65140f1b0 0x55b6517c0a80 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f9902ff5640 0 client.23568 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 --2- 172.21.15.94:0/2503818817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6513e62f0 0x55b6517b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 >> 172.21.15.94:0/2503818817 conn(0x55b651403300 msgr2=0x55b6517ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 shutdown_connections 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 1 -- 172.21.15.94:0/2503818817 wait complete. 2024-04-25T14:04:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.006+0000 7f992c45b740 20 client.23568 trim_cache size 0 max 16384 2024-04-25T14:04:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 Processor -- start 2024-04-25T14:04:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 -- start start 2024-04-25T14:04:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b6517aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ab440 con 0x55b6513e62f0 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99201acf80 con 0x55b6513ee4d0 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f992c45b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99201abb10 con 0x55b65140f1b0 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51342/0 (socket says 172.21.15.94:51342) 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 -- 172.21.15.94:0/1295345183 learned_addr learned my addr 172.21.15.94:0/1295345183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991effd640 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 -- 172.21.15.94:0/1295345183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 msgr2=0x55b6517aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b6517aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 -- 172.21.15.94:0/1295345183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 msgr2=0x55b6517c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991e7fc640 1 -- 172.21.15.94:0/1295345183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6517b4480 con 0x55b6513e62f0 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.021+0000 7f991effd640 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f991e7fc640 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f99140039e0 tx=0x7f9914013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6637b4770d74c07e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f9904ff9640 1 -- 172.21.15.94:0/1295345183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914013170 con 0x55b6513e62f0 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f9904ff9640 1 -- 172.21.15.94:0/1295345183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914013e50 con 0x55b6513e62f0 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f9904ff9640 1 -- 172.21.15.94:0/1295345183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914004620 con 0x55b6513e62f0 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 -- 172.21.15.94:0/1295345183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 msgr2=0x55b6517ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f99140039e0 tx=0x7f9914013720 comp rx=0 tx=0).stop 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 -- 172.21.15.94:0/1295345183 shutdown_connections 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b6517aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513ee4d0 0x55b6517c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 --2- 172.21.15.94:0/1295345183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513e62f0 0x55b6517ae810 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.022+0000 7f992c45b740 1 -- 172.21.15.94:0/1295345183 >> 172.21.15.94:0/1295345183 conn(0x55b651403300 msgr2=0x55b6517ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- 172.21.15.94:0/1295345183 shutdown_connections 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- 172.21.15.94:0/1295345183 wait complete. 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- start start 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 Processor -- start 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b65182d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ac8b0 con 0x55b6513ee4d0 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ac1e0 con 0x55b6513e62f0 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99201a9420 con 0x55b65140f1b0 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f992c45b740 10 client.0 _ll_register_callbacks cb 0x7f9910009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51344/0 (socket says 172.21.15.94:51344) 2024-04-25T14:04:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 -- 172.21.15.94:0/3433598633 learned_addr learned my addr 172.21.15.94:0/3433598633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991e7fc640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991e7fc640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b65182d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 -- 172.21.15.94:0/3433598633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 msgr2=0x55b65182d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b65182d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 -- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 msgr2=0x55b6517bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.023+0000 7f991effd640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f991effd640 1 -- 172.21.15.94:0/3433598633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65182d9a0 con 0x55b6513ee4d0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f991effd640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x55b6517b0d50 tx=0x7f9910010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932f6a83cf38fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100035f0 con 0x55b6513ee4d0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9910003790 con 0x55b6513ee4d0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9914013e50 con 0x55b6513ee4d0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9910003990 con 0x55b6513ee4d0 2024-04-25T14:04:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 10 client.23574 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b65182ecf0 con 0x55b6513ee4d0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914013e50 con 0x55b6513ee4d0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9910029470 con 0x55b6513ee4d0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 10 client.23574 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 10 client.23574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 20 client.23574 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513a31c0 0x55b651831670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 4 client.23574 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.024+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b65171cee0 con 0x55b6513a31c0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.025+0000 7f991e7fc640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513a31c0 0x55b651831670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.025+0000 7f991e7fc640 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513a31c0 0x55b651831670 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266f1eb878d7f58e server_cookie=29dc49bf536abc72 in_seq=0 out_seq=0 2024-04-25T14:04:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.025+0000 7f9902ff5640 10 client.23574 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:30.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 195919 18.89 MB/sec execute 432 sec latency 89.731 ms 2024-04-25T14:04:30.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:29 smithi138 ceph-mon[32977]: pgmap v1032: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 567 op/s 2024-04-25T14:04:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:29 smithi137 ceph-mon[32568]: pgmap v1032: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 567 op/s 2024-04-25T14:04:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:29 smithi094 ceph-mon[29195]: pgmap v1032: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 567 op/s 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f9902ff5640 1 -- 172.21.15.94:0/3433598633 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b65171cee0 con 0x55b6513a31c0 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f9902ff5640 10 client.23574 handle_command_reply: tid=0 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 client.23574 shutdown 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 msgr2=0x55b6517bdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x55b6517b0d50 tx=0x7f9910010aa0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 shutdown_connections 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b6513a31c0 0x55b651831670 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b65140f1b0 0x55b65182d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b6513ee4d0 0x55b6517bdf10 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 --2- 172.21.15.94:0/3433598633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b6513e62f0 0x55b6517bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f9902ff5640 0 client.23574 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 >> 172.21.15.94:0/3433598633 conn(0x55b651403300 msgr2=0x55b6517b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 shutdown_connections 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 1 -- 172.21.15.94:0/3433598633 wait complete. 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.833+0000 7f992c45b740 20 client.23574 trim_cache size 0 max 16384 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:04:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:04:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 msgr2=0x7f99080b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 0x7f99080b1ff0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f99180c5760 tx=0x7f9918092d10 comp rx=0 tx=0).stop 2024-04-25T14:04:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 msgr2=0x7f99201a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f9918009430 tx=0x7f99180316f0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 shutdown_connections 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99080afae0 0x7f99080b1ff0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99201044b0 0x7f99201a14e0 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99201039d0 0x7f992019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 --2- 172.21.15.94:0/75851271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9920102ff0 0x7f9920102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 >> 172.21.15.94:0/75851271 conn(0x7f99200fd390 msgr2=0x7f9920106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 shutdown_connections 2024-04-25T14:04:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:30.834+0000 7f99255a8640 1 -- 172.21.15.94:0/75851271 wait complete. 2024-04-25T14:04:31.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 196052 18.86 MB/sec execute 433 sec latency 613.552 ms 2024-04-25T14:04:32.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 196391 18.87 MB/sec execute 434 sec latency 58.371 ms 2024-04-25T14:04:32.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:31 smithi138 ceph-mon[32977]: pgmap v1033: 97 pgs: 97 active+clean; 276 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 559 op/s 2024-04-25T14:04:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:31 smithi137 ceph-mon[32568]: pgmap v1033: 97 pgs: 97 active+clean; 276 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 559 op/s 2024-04-25T14:04:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:31 smithi094 ceph-mon[29195]: pgmap v1033: 97 pgs: 97 active+clean; 276 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.8 MiB/s wr, 559 op/s 2024-04-25T14:04:33.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 196653 18.86 MB/sec execute 435 sec latency 179.071 ms 2024-04-25T14:04:34.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 196939 18.85 MB/sec execute 436 sec latency 79.639 ms 2024-04-25T14:04:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:33 smithi137 ceph-mon[32568]: pgmap v1034: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-04-25T14:04:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:33 smithi094 ceph-mon[29195]: pgmap v1034: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-04-25T14:04:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:33 smithi138 ceph-mon[32977]: pgmap v1034: 97 pgs: 97 active+clean; 277 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 392 op/s 2024-04-25T14:04:35.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 197285 18.84 MB/sec execute 437 sec latency 88.474 ms 2024-04-25T14:04:36.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 197690 18.85 MB/sec execute 438 sec latency 63.050 ms 2024-04-25T14:04:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:36 smithi137 ceph-mon[32568]: pgmap v1035: 97 pgs: 97 active+clean; 272 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 509 op/s 2024-04-25T14:04:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:36 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:36 smithi094 ceph-mon[29195]: pgmap v1035: 97 pgs: 97 active+clean; 272 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 509 op/s 2024-04-25T14:04:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:36 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:36 smithi138 ceph-mon[32977]: pgmap v1035: 97 pgs: 97 active+clean; 272 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 509 op/s 2024-04-25T14:04:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:36 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:37.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 198055 18.84 MB/sec execute 439 sec latency 83.720 ms 2024-04-25T14:04:37.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: pgmap v1036: 97 pgs: 97 active+clean; 274 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 495 op/s 2024-04-25T14:04:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:37.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:37 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: pgmap v1036: 97 pgs: 97 active+clean; 274 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 495 op/s 2024-04-25T14:04:37.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:37.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:37 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: pgmap v1036: 97 pgs: 97 active+clean; 274 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 495 op/s 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:37.997 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:37 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:04:38.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 198412 18.84 MB/sec execute 440 sec latency 355.104 ms 2024-04-25T14:04:39.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 198803 18.86 MB/sec execute 441 sec latency 288.099 ms 2024-04-25T14:04:39.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:39 smithi094 ceph-mon[29195]: pgmap v1037: 97 pgs: 97 active+clean; 275 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-04-25T14:04:39.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:39 smithi138 ceph-mon[32977]: pgmap v1037: 97 pgs: 97 active+clean; 275 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-04-25T14:04:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:39 smithi137 ceph-mon[32568]: pgmap v1037: 97 pgs: 97 active+clean; 275 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 449 op/s 2024-04-25T14:04:40.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 199210 18.86 MB/sec execute 442 sec latency 101.359 ms 2024-04-25T14:04:41.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 199598 18.87 MB/sec execute 443 sec latency 72.135 ms 2024-04-25T14:04:41.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:41 smithi094 ceph-mon[29195]: pgmap v1038: 97 pgs: 97 active+clean; 279 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-04-25T14:04:41.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:41 smithi138 ceph-mon[32977]: pgmap v1038: 97 pgs: 97 active+clean; 279 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-04-25T14:04:42.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:41 smithi137 ceph-mon[32568]: pgmap v1038: 97 pgs: 97 active+clean; 279 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-04-25T14:04:42.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 199908 18.87 MB/sec execute 444 sec latency 73.069 ms 2024-04-25T14:04:43.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 200215 18.87 MB/sec execute 445 sec latency 238.846 ms 2024-04-25T14:04:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.937+0000 7fee87b14640 1 -- 172.21.15.94:0/617902390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80102a20 msgr2=0x7fee80103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.937+0000 7fee87b14640 1 --2- 172.21.15.94:0/617902390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80102a20 0x7fee80103720 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009900 tx=0x7fee7c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 -- 172.21.15.94:0/617902390 shutdown_connections 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 --2- 172.21.15.94:0/617902390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 0x7fee8010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 --2- 172.21.15.94:0/617902390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80103c60 0x7fee80104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 --2- 172.21.15.94:0/617902390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80102a20 0x7fee80103720 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 -- 172.21.15.94:0/617902390 >> 172.21.15.94:0/617902390 conn(0x7fee800fdc10 msgr2=0x7fee801000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 -- 172.21.15.94:0/617902390 shutdown_connections 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 -- 172.21.15.94:0/617902390 wait complete. 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 Processor -- start 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 -- start start 2024-04-25T14:04:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80102a20 0x7fee80199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.938+0000 7fee87b14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 0x7fee801a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e520 con 0x7fee80103c60 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e3a0 con 0x7fee80108960 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010e6a0 con 0x7fee80102a20 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36720/0 (socket says 172.21.15.94:36720) 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 -- 172.21.15.94:0/3705323604 learned_addr learned my addr 172.21.15.94:0/3705323604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee860b1640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 0x7fee801a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee858b0640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80102a20 0x7fee80199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 -- 172.21.15.94:0/3705323604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80102a20 msgr2=0x7fee80199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80102a20 0x7fee80199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 -- 172.21.15.94:0/3705323604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 msgr2=0x7fee801a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 0x7fee801a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 -- 172.21.15.94:0/3705323604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801a0cb0 con 0x7fee80103c60 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee850af640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fee7000e9f0 tx=0x7fee7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee76ffd640 1 -- 172.21.15.94:0/3705323604 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7000cd80 con 0x7fee80103c60 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801a0fa0 con 0x7fee80103c60 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee76ffd640 1 -- 172.21.15.94:0/3705323604 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee700054d0 con 0x7fee80103c60 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee76ffd640 1 -- 172.21.15.94:0/3705323604 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70010690 con 0x7fee80103c60 2024-04-25T14:04:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.939+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a13b0 con 0x7fee80103c60 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.942+0000 7fee76ffd640 1 -- 172.21.15.94:0/3705323604 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fee70004e90 con 0x7fee80103c60 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.942+0000 7fee8e9ca740 1 Processor -- start 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 -- start start 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d218b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d220000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b70d17c460 con 0x55b70ce4d2f0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b70d136590 con 0x55b70ce554d0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b70d178420 con 0x55b70ce761b0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36724/0 (socket says 172.21.15.94:36724) 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 -- 172.21.15.94:0/869154461 learned_addr learned my addr 172.21.15.94:0/869154461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee860b1640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d218b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee850af640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d220000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 -- 172.21.15.94:0/869154461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 msgr2=0x55b70d218b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d218b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 -- 172.21.15.94:0/869154461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 msgr2=0x55b70d220000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d220000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee858b0640 1 -- 172.21.15.94:0/869154461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70d1430d0 con 0x55b70ce4d2f0 2024-04-25T14:04:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee76ffd640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 0x7fee6c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.943+0000 7fee850af640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d220000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee860b1640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 0x7fee6c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee858b0640 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009230 tx=0x7fee7c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b20f5821529d13d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee76ffd640 1 -- 172.21.15.94:0/3705323604 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fee700df0d0 con 0x7fee80103c60 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee6bfff640 1 -- 172.21.15.94:0/869154461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c035030 con 0x55b70ce4d2f0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee6bfff640 1 -- 172.21.15.94:0/869154461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c040070 con 0x55b70ce4d2f0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.944+0000 7fee6bfff640 1 -- 172.21.15.94:0/869154461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003950 con 0x55b70ce4d2f0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee860b1640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 0x7fee6c0b1ff0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55b70d219d50 tx=0x7fee78015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- 172.21.15.94:0/869154461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 msgr2=0x55b70d219100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009230 tx=0x7fee7c009620 comp rx=0 tx=0).stop 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- 172.21.15.94:0/869154461 shutdown_connections 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d220000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce4d2f0 0x55b70d219100 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/869154461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d218b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- 172.21.15.94:0/869154461 >> 172.21.15.94:0/869154461 conn(0x55b70ce6a300 msgr2=0x55b70d212300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- 172.21.15.94:0/869154461 shutdown_connections 2024-04-25T14:04:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- 172.21.15.94:0/869154461 wait complete. 2024-04-25T14:04:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.945+0000 7fee8e9ca740 1 -- start start 2024-04-25T14:04:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 Processor -- start 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d220aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d227a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b70d2167a0 con 0x55b70ce554d0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b70d103b60 con 0x55b70ce4d2f0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b70d216920 con 0x55b70ce761b0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee8e9ca740 10 client.0 _ll_register_callbacks cb 0x55b70d1282b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee858b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d220aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36734/0 (socket says 172.21.15.94:36734) 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 -- 172.21.15.94:0/3786290335 learned_addr learned my addr 172.21.15.94:0/3786290335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee860b1640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d227a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 -- 172.21.15.94:0/3786290335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 msgr2=0x55b70d227a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d227a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 -- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 msgr2=0x55b70d220aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d220aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee850af640 1 -- 172.21.15.94:0/3786290335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70ce8e210 con 0x55b70ce554d0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.946+0000 7fee858b0640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d220aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee850af640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fee70018000 tx=0x7fee700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b2c148d2cf0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee700a6c80 con 0x55b70ce554d0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fee700afee0 con 0x55b70ce554d0 2024-04-25T14:04:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7c040070 con 0x55b70ce554d0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fee70005180 con 0x55b70ce554d0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 10 client.23586 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b70cf5a620 con 0x55b70ce554d0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c040070 con 0x55b70ce554d0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fee700a2e80 con 0x55b70ce554d0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 10 client.23586 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 10 client.23586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 20 client.23586 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce34d80 0x55b70d22e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 4 client.23586 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.947+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b70d18a070 con 0x55b70ce34d80 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.948+0000 7fee858b0640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce34d80 0x55b70d22e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.948+0000 7fee697fa640 10 client.23586 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.948+0000 7fee858b0640 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce34d80 0x55b70d22e2a0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4788d1b3b0dcc85c server_cookie=eea2fcfe629723fc in_seq=0 out_seq=0 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee697fa640 1 -- 172.21.15.94:0/3786290335 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b70d18a070 con 0x55b70ce34d80 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee697fa640 10 client.23586 handle_command_reply: tid=0 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 client.23586 shutdown 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 msgr2=0x55b70d220fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fee70018000 tx=0x7fee700048f0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 shutdown_connections 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce34d80 0x55b70d22e2a0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce761b0 0x55b70d227a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d220fe0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee697fa640 0 client.23586 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/3786290335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d220aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 >> 172.21.15.94:0/3786290335 conn(0x55b70ce6a300 msgr2=0x55b70d215210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 shutdown_connections 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 1 -- 172.21.15.94:0/3786290335 wait complete. 2024-04-25T14:04:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.954+0000 7fee8e9ca740 20 client.23586 trim_cache size 0 max 16384 2024-04-25T14:04:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.969+0000 7fee8e9ca740 1 Processor -- start 2024-04-25T14:04:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.969+0000 7fee8e9ca740 1 -- start start 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.969+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce34d80 0x55b70d218b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d219040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a9300 con 0x55b70ce554d0 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aae40 con 0x55b70ce4d2f0 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a99d0 con 0x55b70ce34d80 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36742/0 (socket says 172.21.15.94:36742) 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 -- 172.21.15.94:0/1029061293 learned_addr learned my addr 172.21.15.94:0/1029061293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee850af640 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d219040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee858b0640 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce34d80 0x55b70d218b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 -- 172.21.15.94:0/1029061293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce34d80 msgr2=0x55b70d218b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce34d80 0x55b70d218b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 -- 172.21.15.94:0/1029061293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 msgr2=0x55b70d219040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d219040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 -- 172.21.15.94:0/1029061293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70d212890 con 0x55b70ce554d0 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee860b1640 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fee7800eda0 tx=0x7fee7801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ff161a3c0d4dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee6b7fe640 1 -- 172.21.15.94:0/1029061293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7801dd90 con 0x55b70ce554d0 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee6b7fe640 1 -- 172.21.15.94:0/1029061293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78030070 con 0x55b70ce554d0 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.970+0000 7fee6b7fe640 1 -- 172.21.15.94:0/1029061293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7802b3f0 con 0x55b70ce554d0 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 -- 172.21.15.94:0/1029061293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 msgr2=0x55b70d21b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fee7800eda0 tx=0x7fee7801b000 comp rx=0 tx=0).stop 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 -- 172.21.15.94:0/1029061293 shutdown_connections 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce554d0 0x55b70d21b2d0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce4d2f0 0x55b70d219040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/1029061293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce34d80 0x55b70d218b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 -- 172.21.15.94:0/1029061293 >> 172.21.15.94:0/1029061293 conn(0x55b70ce6a300 msgr2=0x55b70d21a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 -- 172.21.15.94:0/1029061293 shutdown_connections 2024-04-25T14:04:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.971+0000 7fee8e9ca740 1 -- 172.21.15.94:0/1029061293 wait complete. 2024-04-25T14:04:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 -- start start 2024-04-25T14:04:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 Processor -- start 2024-04-25T14:04:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 0x55b70d222680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d294640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aa770 con 0x55b70ce34d80 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aa0a0 con 0x55b70ce554d0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a72e0 con 0x55b70ce4d2f0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee8e9ca740 10 client.0 _ll_register_callbacks cb 0x7fee700e28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee858b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee858b0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36758/0 (socket says 172.21.15.94:36758) 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee850af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 0x55b70d222680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee858b0640 1 -- 172.21.15.94:0/2314291335 learned_addr learned my addr 172.21.15.94:0/2314291335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.972+0000 7fee860b1640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d294640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 -- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 msgr2=0x55b70d294640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d294640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 -- 172.21.15.94:0/2314291335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 msgr2=0x55b70d222140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 -- 172.21.15.94:0/2314291335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70d294b80 con 0x55b70ce4d2f0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee858b0640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee850af640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 0x55b70d222680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c6a0 tx=0x7fee700b0110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8376dc0952f687fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee697fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee700a6c80 con 0x55b70ce4d2f0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee78030070 con 0x55b70ce4d2f0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fee700a6980 con 0x55b70ce4d2f0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fee700a72e0 con 0x55b70ce4d2f0 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee8e9ca740 10 client.32611 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b70d295d50 con 0x55b70ce4d2f0 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.973+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78030070 con 0x55b70ce4d2f0 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fee700aca50 con 0x55b70ce4d2f0 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 10 client.32611 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 10 client.32611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 20 client.32611 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce761b0 0x55b70d298560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 4 client.32611 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b70d183ee0 con 0x55b70ce761b0 2024-04-25T14:04:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee858b0640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce761b0 0x55b70d298560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:43.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:43 smithi094 ceph-mon[29195]: pgmap v1039: 97 pgs: 97 active+clean; 283 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-04-25T14:04:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee697fa640 10 client.32611 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:43.974+0000 7fee858b0640 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce761b0 0x55b70d298560 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102f8a0e0163644d server_cookie=2e899ef5e0208825 in_seq=0 out_seq=0 2024-04-25T14:04:43.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:43 smithi138 ceph-mon[32977]: pgmap v1039: 97 pgs: 97 active+clean; 283 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-04-25T14:04:44.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:43 smithi137 ceph-mon[32568]: pgmap v1039: 97 pgs: 97 active+clean; 283 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-04-25T14:04:44.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 200498 18.86 MB/sec execute 446 sec latency 167.197 ms 2024-04-25T14:04:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee697fa640 1 -- 172.21.15.94:0/2314291335 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b70d183ee0 con 0x55b70ce761b0 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee697fa640 10 client.32611 handle_command_reply: tid=0 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 client.32611 shutdown 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 msgr2=0x55b70d222680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 0x55b70d222680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c6a0 tx=0x7fee700b0110 comp rx=0 tx=0).stop 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 shutdown_connections 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55b70ce761b0 0x55b70d298560 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70ce554d0 0x55b70d294640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70ce4d2f0 0x55b70d222680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 --2- 172.21.15.94:0/2314291335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b70ce34d80 0x55b70d222140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee697fa640 0 client.32611 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 >> 172.21.15.94:0/2314291335 conn(0x55b70ce6a300 msgr2=0x55b70d21b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 shutdown_connections 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 1 -- 172.21.15.94:0/2314291335 wait complete. 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.731+0000 7fee8e9ca740 20 client.32611 trim_cache size 0 max 16384 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:04:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 msgr2=0x7fee6c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 0x7fee6c0b1ff0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55b70d219d50 tx=0x7fee78015040 comp rx=0 tx=0).stop 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 msgr2=0x7fee801996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fee7000e9f0 tx=0x7fee7000edf0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 shutdown_connections 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fee6c0afae0 0x7fee6c0b1ff0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee80108960 0x7fee801a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee80103c60 0x7fee801996b0 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 --2- 172.21.15.94:0/3705323604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee80102a20 0x7fee80199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 >> 172.21.15.94:0/3705323604 conn(0x7fee800fdc10 msgr2=0x7fee80104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 shutdown_connections 2024-04-25T14:04:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:44.732+0000 7fee87b14640 1 -- 172.21.15.94:0/3705323604 wait complete. 2024-04-25T14:04:45.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 200728 18.84 MB/sec execute 447 sec latency 419.251 ms 2024-04-25T14:04:45.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:45.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:45.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:45 smithi094 ceph-mon[29195]: pgmap v1040: 97 pgs: 97 active+clean; 285 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-04-25T14:04:45.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:45 smithi138 ceph-mon[32977]: pgmap v1040: 97 pgs: 97 active+clean; 285 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 -- 172.21.15.94:0/2765136017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134102260 msgr2=0x7ff13410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 --2- 172.21.15.94:0/2765136017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134102260 0x7ff13410bcc0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009ca0 tx=0x7ff12c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 -- 172.21.15.94:0/2765136017 shutdown_connections 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 --2- 172.21.15.94:0/2765136017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134102260 0x7ff13410bcc0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 --2- 172.21.15.94:0/2765136017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134101760 0x7ff134101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 --2- 172.21.15.94:0/2765136017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 -- 172.21.15.94:0/2765136017 >> 172.21.15.94:0/2765136017 conn(0x7ff1340fd380 msgr2=0x7ff1340ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 -- 172.21.15.94:0/2765136017 shutdown_connections 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.054+0000 7ff13b91f640 1 -- 172.21.15.94:0/2765136017 wait complete. 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 Processor -- start 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 -- start start 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134102260 0x7ff13419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff13410f640 con 0x7ff134101760 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff13410f4c0 con 0x7ff134102260 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.055+0000 7ff13b91f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff13410f7c0 con 0x7ff1341050b0 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36780/0 (socket says 172.21.15.94:36780) 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 -- 172.21.15.94:0/3279562370 learned_addr learned my addr 172.21.15.94:0/3279562370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff139ebc640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff138eba640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134102260 0x7ff13419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 -- 172.21.15.94:0/3279562370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 msgr2=0x7ff1341a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 -- 172.21.15.94:0/3279562370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134102260 msgr2=0x7ff13419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134102260 0x7ff13419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 -- 172.21.15.94:0/3279562370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1341a1d00 con 0x7ff134101760 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff139ebc640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff1396bb640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7ff13000e9f0 tx=0x7ff13000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff12a7fc640 1 -- 172.21.15.94:0/3279562370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13000ccc0 con 0x7ff134101760 2024-04-25T14:04:46.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff12a7fc640 1 -- 172.21.15.94:0/3279562370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1300035f0 con 0x7ff134101760 2024-04-25T14:04:46.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1341a1ff0 con 0x7ff134101760 2024-04-25T14:04:46.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff12a7fc640 1 -- 172.21.15.94:0/3279562370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13000ce60 con 0x7ff134101760 2024-04-25T14:04:46.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.056+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341a2450 con 0x7ff134101760 2024-04-25T14:04:46.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff12a7fc640 1 -- 172.21.15.94:0/3279562370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff130003aa0 con 0x7ff134101760 2024-04-25T14:04:46.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 Processor -- start 2024-04-25T14:04:46.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 -- start start 2024-04-25T14:04:46.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e690fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55755e5f5d80 con 0x55755e2ce6b0 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55755e5af8b0 con 0x55755e2c64d0 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.059+0000 7ff1427ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55755e5f1d40 con 0x55755e2ef390 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff139ebc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36788/0 (socket says 172.21.15.94:36788) 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff139ebc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52722/0 (socket says 172.21.15.94:52722) 2024-04-25T14:04:46.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff1396bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e690fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 -- 172.21.15.94:0/4151986567 learned_addr learned my addr 172.21.15.94:0/4151986567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff12a7fc640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 0x7ff1200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 -- 172.21.15.94:0/4151986567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 msgr2=0x55755e690a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 -- 172.21.15.94:0/4151986567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 msgr2=0x55755e690fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e690fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 -- 172.21.15.94:0/4151986567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755e580a40 con 0x55755e2ce6b0 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff139ebc640 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff1396bb640 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e690fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff139ebc640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 0x7ff1200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff12a7fc640 1 -- 172.21.15.94:0/3279562370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1300deff0 con 0x7ff134101760 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff138eba640 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7ff124009c90 tx=0x7ff124011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab9d91233ce4873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff11f7fe640 1 -- 172.21.15.94:0/4151986567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124005220 con 0x55755e2ce6b0 2024-04-25T14:04:46.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff11f7fe640 1 -- 172.21.15.94:0/4151986567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff124004c50 con 0x55755e2ce6b0 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.060+0000 7ff11f7fe640 1 -- 172.21.15.94:0/4151986567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124019760 con 0x55755e2ce6b0 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 -- 172.21.15.94:0/4151986567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 msgr2=0x55755e691c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff139ebc640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 0x7ff1200b2000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55755e691ab0 tx=0x7ff12c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7ff124009c90 tx=0x7ff124011810 comp rx=0 tx=0).stop 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 -- 172.21.15.94:0/4151986567 shutdown_connections 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e691c70 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e690fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4151986567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e690a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 -- 172.21.15.94:0/4151986567 >> 172.21.15.94:0/4151986567 conn(0x55755e2e34e0 msgr2=0x55755e68c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 -- 172.21.15.94:0/4151986567 shutdown_connections 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.061+0000 7ff1427ce740 1 -- 172.21.15.94:0/4151986567 wait complete. 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 -- start start 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 Processor -- start 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2ce6b0 0x55755e69ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55755e692780 con 0x55755e2c64d0 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55755e57cd10 con 0x55755e2ce6b0 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55755e692900 con 0x55755e2ef390 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1427ce740 10 client.0 _ll_register_callbacks cb 0x55755e5a9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1396bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1396bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36804/0 (socket says 172.21.15.94:36804) 2024-04-25T14:04:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff1396bb640 1 -- 172.21.15.94:0/1374864240 learned_addr learned my addr 172.21.15.94:0/1374864240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff138eba640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2ce6b0 0x55755e69ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.062+0000 7ff139ebc640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 -- 172.21.15.94:0/1374864240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 msgr2=0x55755e6a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 -- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2ce6b0 msgr2=0x55755e69ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2ce6b0 0x55755e69ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 -- 172.21.15.94:0/1374864240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755e3d17d0 con 0x55755e2c64d0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1396bb640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7ff1300a73c0 tx=0x7ff1300a2bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138de8dac52db604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1300a6d80 con 0x55755e2c64d0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1300e2c80 con 0x55755e2c64d0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff124004c50 con 0x55755e2c64d0 2024-04-25T14:04:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff1300a6ba0 con 0x55755e2c64d0 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1427ce740 10 client.23604 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55755e3073f0 con 0x55755e2c64d0 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff124004c50 con 0x55755e2c64d0 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.063+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff1300e10d0 con 0x55755e2c64d0 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 10 client.23604 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 10 client.23604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 20 client.23604 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2adf60 0x55755e6a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 4 client.23604 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755e603990 con 0x55755e2adf60 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff138eba640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2adf60 0x55755e6a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff11cff9640 10 client.23604 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.064+0000 7ff138eba640 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2adf60 0x55755e6a8240 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a5aa0131e3fcf2 server_cookie=4f30b22ecf42e66a in_seq=0 out_seq=0 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.072+0000 7ff11cff9640 1 -- 172.21.15.94:0/1374864240 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55755e603990 con 0x55755e2adf60 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.072+0000 7ff11cff9640 10 client.23604 handle_command_reply: tid=0 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.072+0000 7ff1427ce740 1 client.23604 shutdown 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 msgr2=0x55755e69a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7ff1300a73c0 tx=0x7ff1300a2bd0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 shutdown_connections 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2adf60 0x55755e6a8240 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2ce6b0 0x55755e69ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff11cff9640 0 client.23604 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 --2- 172.21.15.94:0/1374864240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2c64d0 0x55755e69a8d0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 >> 172.21.15.94:0/1374864240 conn(0x55755e2e34e0 msgr2=0x55755e68f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 shutdown_connections 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 1 -- 172.21.15.94:0/1374864240 wait complete. 2024-04-25T14:04:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.073+0000 7ff1427ce740 20 client.23604 trim_cache size 0 max 16384 2024-04-25T14:04:46.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.092+0000 7ff1427ce740 1 Processor -- start 2024-04-25T14:04:46.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.092+0000 7ff1427ce740 1 -- start start 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e69a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ab590 con 0x55755e2ce6b0 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ad0d0 con 0x55755e2c64d0 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1427ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341abc60 con 0x55755e2ef390 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36814/0 (socket says 172.21.15.94:36814) 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 -- 172.21.15.94:0/3710789355 learned_addr learned my addr 172.21.15.94:0/3710789355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff1396bb640 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff139ebc640 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e69a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 -- 172.21.15.94:0/3710789355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 msgr2=0x55755e69a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e69a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 -- 172.21.15.94:0/3710789355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 msgr2=0x55755e69a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 -- 172.21.15.94:0/3710789355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755e2c70f0 con 0x55755e2ce6b0 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff138eba640 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7ff124009a50 tx=0x7ff124009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcb1b9af7d7b08f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff11effd640 1 -- 172.21.15.94:0/3710789355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124004a90 con 0x55755e2ce6b0 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff11effd640 1 -- 172.21.15.94:0/3710789355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1240044f0 con 0x55755e2ce6b0 2024-04-25T14:04:46.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.093+0000 7ff11effd640 1 -- 172.21.15.94:0/3710789355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124005640 con 0x55755e2ce6b0 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 -- 172.21.15.94:0/3710789355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 msgr2=0x55755e6a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7ff124009a50 tx=0x7ff124009c00 comp rx=0 tx=0).stop 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 -- 172.21.15.94:0/3710789355 shutdown_connections 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e69a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e6a1d00 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 --2- 172.21.15.94:0/3710789355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 -- 172.21.15.94:0/3710789355 >> 172.21.15.94:0/3710789355 conn(0x55755e2e34e0 msgr2=0x55755e6958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 -- 172.21.15.94:0/3710789355 shutdown_connections 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.094+0000 7ff1427ce740 1 -- 172.21.15.94:0/3710789355 wait complete. 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 -- start start 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 Processor -- start 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341aca00 con 0x55755e2ce6b0 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ac330 con 0x55755e2c64d0 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341a9570 con 0x55755e2ef390 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1427ce740 10 client.0 _ll_register_callbacks cb 0x7ff1300045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff138eba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36830/0 (socket says 172.21.15.94:36830) 2024-04-25T14:04:46.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff138eba640 1 -- 172.21.15.94:0/4066125594 learned_addr learned my addr 172.21.15.94:0/4066125594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff1396bb640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.095+0000 7ff139ebc640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 -- 172.21.15.94:0/4066125594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 msgr2=0x55755e6a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 -- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 msgr2=0x55755e69f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 -- 172.21.15.94:0/4066125594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755e5b64b0 con 0x55755e2ce6b0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff139ebc640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff138eba640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff124009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673fa1da68394ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12403a020 con 0x55755e2ce6b0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff124004040 con 0x55755e2ce6b0 2024-04-25T14:04:46.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1240044f0 con 0x55755e2ce6b0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff124045050 con 0x55755e2ce6b0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff1427ce740 10 client.23610 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55755e6a1d00 con 0x55755e2ce6b0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1240044f0 con 0x55755e2ce6b0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.096+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff1240038f0 con 0x55755e2ce6b0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 10 client.23610 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 10 client.23610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 20 client.23610 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2833a0 0x55755e712810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 4 client.23610 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55755e5fd800 con 0x55755e2833a0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1396bb640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2833a0 0x55755e712810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff11cff9640 10 client.23610 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.097+0000 7ff1396bb640 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2833a0 0x55755e712810 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac615d26a6d25ea3 server_cookie=94dd1fb7af28b76e in_seq=0 out_seq=0 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff11cff9640 1 -- 172.21.15.94:0/4066125594 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55755e5fd800 con 0x55755e2833a0 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff11cff9640 10 client.23610 handle_command_reply: tid=0 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 client.23610 shutdown 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 msgr2=0x55755e69f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7ff124000c00 tx=0x7ff124009820 comp rx=0 tx=0).stop 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 shutdown_connections 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55755e2833a0 0x55755e712810 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55755e2ef390 0x55755e6a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55755e2ce6b0 0x55755e69f750 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff11cff9640 0 client.23610 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 --2- 172.21.15.94:0/4066125594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55755e2c64d0 0x55755e69f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.099+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 >> 172.21.15.94:0/4066125594 conn(0x55755e2e34e0 msgr2=0x55755e69a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 shutdown_connections 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff1427ce740 1 -- 172.21.15.94:0/4066125594 wait complete. 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff1427ce740 20 client.23610 trim_cache size 0 max 16384 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:04:46.105 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 msgr2=0x7ff1200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 0x7ff1200b2000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55755e691ab0 tx=0x7ff12c0095c0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 msgr2=0x7ff13419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7ff13000e9f0 tx=0x7ff13000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 shutdown_connections 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff1200afaf0 0x7ff1200b2000 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341050b0 0x7ff1341a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff134102260 0x7ff13419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 --2- 172.21.15.94:0/3279562370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff134101760 0x7ff13419a1c0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 >> 172.21.15.94:0/3279562370 conn(0x7ff1340fd380 msgr2=0x7ff134107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 shutdown_connections 2024-04-25T14:04:46.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.100+0000 7ff13b91f640 1 -- 172.21.15.94:0/3279562370 wait complete. 2024-04-25T14:04:46.111 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:04:46.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:46.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:45 smithi137 ceph-mon[32568]: pgmap v1040: 97 pgs: 97 active+clean; 285 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-04-25T14:04:46.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 201074 18.83 MB/sec execute 448 sec latency 68.117 ms 2024-04-25T14:04:46.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.271+0000 7f3053f85640 1 -- 172.21.15.94:0/236723298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c104d10 msgr2=0x7f304c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.271+0000 7f3053f85640 1 --2- 172.21.15.94:0/236723298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c104d10 0x7f304c105120 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 -- 172.21.15.94:0/236723298 shutdown_connections 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 --2- 172.21.15.94:0/236723298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 0x7f304c10be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 --2- 172.21.15.94:0/236723298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c104d10 0x7f304c105120 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 --2- 172.21.15.94:0/236723298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c103ad0 0x7f304c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 -- 172.21.15.94:0/236723298 >> 172.21.15.94:0/236723298 conn(0x7f304c0fecc0 msgr2=0x7f304c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 -- 172.21.15.94:0/236723298 shutdown_connections 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 -- 172.21.15.94:0/236723298 wait complete. 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 Processor -- start 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.272+0000 7f3053f85640 1 -- start start 2024-04-25T14:04:46.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c104d10 0x7f304c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 0x7f304c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f600 con 0x7f304c103ad0 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f480 con 0x7f304c104d10 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f304c10f780 con 0x7f304c109a10 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36852/0 (socket says 172.21.15.94:36852) 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 -- 172.21.15.94:0/2363854529 learned_addr learned my addr 172.21.15.94:0/2363854529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051520640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c104d10 0x7f304c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3052522640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 0x7f304c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 -- 172.21.15.94:0/2363854529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 msgr2=0x7f304c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 0x7f304c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 -- 172.21.15.94:0/2363854529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c104d10 msgr2=0x7f304c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c104d10 0x7f304c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 -- 172.21.15.94:0/2363854529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c1a1d40 con 0x7f304c103ad0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3051d21640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9f0 tx=0x7f304800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800ccc0 con 0x7f304c103ad0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c1a1fd0 con 0x7f304c103ad0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30480035f0 con 0x7f304c103ad0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304800ce60 con 0x7f304c103ad0 2024-04-25T14:04:46.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.273+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1a24e0 con 0x7f304c103ad0 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.274+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c108820 con 0x7f304c103ad0 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.276+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3048003aa0 con 0x7f304c103ad0 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.276+0000 7f3042ffd640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 0x7f30380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.277+0000 7f3051520640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 0x7f30380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.277+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f30480df050 con 0x7f304c103ad0 2024-04-25T14:04:46.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.277+0000 7f3051520640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 0x7f30380b2000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f303c009230 tx=0x7f303c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.280+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30480a7c30 con 0x7f304c103ad0 2024-04-25T14:04:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.535+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f304c02db50 con 0x7f304c103ad0 2024-04-25T14:04:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.535+0000 7f3042ffd640 1 -- 172.21.15.94:0/2363854529 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f30480a7530 con 0x7f304c103ad0 2024-04-25T14:04:46.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:04:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:04:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:04:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.537+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 msgr2=0x7f30380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.537+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 0x7f30380b2000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f303c009230 tx=0x7f303c031400 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 msgr2=0x7f304c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f304800e9f0 tx=0x7f304800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 shutdown_connections 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f30380afaf0 0x7f30380b2000 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c109a10 0x7f304c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f304c104d10 0x7f304c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 --2- 172.21.15.94:0/2363854529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f304c103ad0 0x7f304c19a200 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 >> 172.21.15.94:0/2363854529 conn(0x7f304c0fecc0 msgr2=0x7f304c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 shutdown_connections 2024-04-25T14:04:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.538+0000 7f3053f85640 1 -- 172.21.15.94:0/2363854529 wait complete. 2024-04-25T14:04:46.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:04:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/1733014725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 msgr2=0x7fa1a410bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/1733014725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a410bc90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ca0 tx=0x7fa19c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/1733014725 shutdown_connections 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/1733014725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a410bc90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/1733014725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4104b70 0x7fa1a4104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/1733014725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4103930 0x7fa1a4104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/1733014725 >> 172.21.15.94:0/1733014725 conn(0x7fa1a40feb20 msgr2=0x7fa1a4100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/1733014725 shutdown_connections 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.717+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/1733014725 wait complete. 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.718+0000 7fa1a8f9d640 1 Processor -- start 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.718+0000 7fa1a8f9d640 1 -- start start 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.718+0000 7fa1a8f9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a8f9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 0x7fa1a419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a8f9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a41a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a8f9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410f4b0 con 0x7fa1a4103930 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a8f9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410f330 con 0x7fa1a4104b70 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a8f9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410f630 con 0x7fa1a4109870 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36868/0 (socket says 172.21.15.94:36868) 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2879138557 learned_addr learned my addr 172.21.15.94:0/2879138557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 0x7fa1a419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a41a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2879138557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 msgr2=0x7fa1a41a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a41a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2879138557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 msgr2=0x7fa1a419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 0x7fa1a419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2879138557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41a1b00 con 0x7fa1a4103930 2024-04-25T14:04:46.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 0x7fa1a419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a41a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.719+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00e9f0 tx=0x7fa18c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.720+0000 7fa1a0ff9640 1 -- 172.21.15.94:0/2879138557 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00ccc0 con 0x7fa1a4103930 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.720+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a1df0 con 0x7fa1a4103930 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.720+0000 7fa1a0ff9640 1 -- 172.21.15.94:0/2879138557 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c0035f0 con 0x7fa1a4103930 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.720+0000 7fa1a0ff9640 1 -- 172.21.15.94:0/2879138557 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00ce60 con 0x7fa1a4103930 2024-04-25T14:04:46.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.720+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a2250 con 0x7fa1a4103930 2024-04-25T14:04:46.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.723+0000 7fa1a0ff9640 1 -- 172.21.15.94:0/2879138557 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa18c003aa0 con 0x7fa1a4103930 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 Processor -- start 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 -- start start 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a0ff9640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 0x7fa1900b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 0x7fa1900b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8ae5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559bf8a39d80 con 0x559bf87126b0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559bf89f3a20 con 0x559bf8733390 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1b1184740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bf8a35d40 con 0x559bf870a4d0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a0ff9640 1 -- 172.21.15.94:0/2879138557 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa18c0def90 con 0x7fa1a4103930 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35536/0 (socket says 172.21.15.94:35536) 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.724+0000 7fa1a37fe640 1 -- 172.21.15.94:0/4152972324 learned_addr learned my addr 172.21.15.94:0/4152972324 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8ae5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 -- 172.21.15.94:0/4152972324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 msgr2=0x559bf8ae08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 -- 172.21.15.94:0/4152972324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 msgr2=0x559bf8ae5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8ae5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 -- 172.21.15.94:0/4152972324 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf89fec30 con 0x559bf870a4d0 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8ae5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 0x7fa1900b2070 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005e30 tx=0x7fa19c0099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae2e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa194009fa0 tx=0x7fa1940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8363aa21fae0e625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa199ffb640 1 -- 172.21.15.94:0/4152972324 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194004030 con 0x559bf870a4d0 2024-04-25T14:04:46.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.725+0000 7fa199ffb640 1 -- 172.21.15.94:0/4152972324 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1940054d0 con 0x559bf870a4d0 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.726+0000 7fa199ffb640 1 -- 172.21.15.94:0/4152972324 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194004c00 con 0x559bf870a4d0 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.726+0000 7fa1b1184740 1 -- 172.21.15.94:0/4152972324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 msgr2=0x559bf8ae2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.726+0000 7fa1b1184740 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae2e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa194009fa0 tx=0x7fa1940118d0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 -- 172.21.15.94:0/4152972324 shutdown_connections 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8ae5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae2e80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 --2- 172.21.15.94:0/4152972324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf8733390 0x559bf8ae08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 -- 172.21.15.94:0/4152972324 >> 172.21.15.94:0/4152972324 conn(0x559bf87274e0 msgr2=0x559bf8ad1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 -- 172.21.15.94:0/4152972324 shutdown_connections 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 -- 172.21.15.94:0/4152972324 wait complete. 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 -- start start 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.727+0000 7fa1b1184740 1 Processor -- start 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 0x559bf8ae83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 0x559bf8aef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559bf8ae6180 con 0x559bf8733390 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559bf89c0dc0 con 0x559bf87126b0 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bf8ae6300 con 0x559bf870a4d0 2024-04-25T14:04:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1b1184740 10 client.0 _ll_register_callbacks cb 0x559bf89dc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 0x559bf8ae83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 0x559bf8aef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 0x559bf8ae83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35552/0 (socket says 172.21.15.94:35552) 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52812/0 (socket says 172.21.15.94:52812) 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a2ffd640 1 -- 172.21.15.94:0/2961913979 learned_addr learned my addr 172.21.15.94:0/2961913979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2961913979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 msgr2=0x559bf8ae83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 0x559bf8ae83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2961913979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 msgr2=0x559bf8aef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 0x559bf8aef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2961913979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf874b3f0 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 0x559bf8aef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0a4640 tx=0x7fa18c0a7b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1220f60e1ecbf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.728+0000 7fa182ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0c0020 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa18c0affb0 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1940054d0 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa18c0b0230 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 10 client.32647 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bf88157d0 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1940054d0 con 0x559bf870a4d0 2024-04-25T14:04:46.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa18c0aeb00 con 0x559bf870a4d0 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 10 client.32647 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 10 client.32647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 20 client.32647 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf86f1f60 0x559bf8af5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 4 client.32647 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf8a47990 con 0x559bf86f1f60 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.729+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf86f1f60 0x559bf8af5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.730+0000 7fa182ffd640 10 client.32647 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.730+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf86f1f60 0x559bf8af5c90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3866b2c1db29af3b server_cookie=f640c65cd43ebc9d in_seq=0 out_seq=0 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.730+0000 7fa182ffd640 1 -- 172.21.15.94:0/2961913979 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559bf8a47990 con 0x559bf86f1f60 2024-04-25T14:04:46.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.730+0000 7fa182ffd640 10 client.32647 handle_command_reply: tid=0 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 client.32647 shutdown 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 msgr2=0x559bf8ae7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0a4640 tx=0x7fa18c0a7b90 comp rx=0 tx=0).stop 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 shutdown_connections 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf86f1f60 0x559bf8af5c90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf8733390 0x559bf8aef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa182ffd640 0 client.32647 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf87126b0 0x559bf8ae83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2961913979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae7e80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 >> 172.21.15.94:0/2961913979 conn(0x559bf87274e0 msgr2=0x559bf8acf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 shutdown_connections 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 1 -- 172.21.15.94:0/2961913979 wait complete. 2024-04-25T14:04:46.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.731+0000 7fa1b1184740 20 client.32647 trim_cache size 0 max 16384 2024-04-25T14:04:46.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.746+0000 7fa1b1184740 1 Processor -- start 2024-04-25T14:04:46.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.746+0000 7fa1b1184740 1 -- start start 2024-04-25T14:04:46.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 0x559bf8af5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 0x559bf8acf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a9150 con 0x559bf870a4d0 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41aac90 con 0x559bf86f1f60 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1b1184740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a9820 con 0x559bf87126b0 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 0x559bf8af5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 0x559bf8af5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36894/0 (socket says 172.21.15.94:36894) 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35560/0 (socket says 172.21.15.94:35560) 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a2ffd640 1 -- 172.21.15.94:0/701331063 learned_addr learned my addr 172.21.15.94:0/701331063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 0x559bf8acf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 -- 172.21.15.94:0/701331063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 msgr2=0x559bf8ae6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 -- 172.21.15.94:0/701331063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 msgr2=0x559bf8af5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 0x559bf8af5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 -- 172.21.15.94:0/701331063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf8ad1ad0 con 0x559bf87126b0 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.747+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 0x559bf8acf9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02aeb0 tx=0x7fa19c04a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01c1e8cabc56b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.748+0000 7fa1997fa640 1 -- 172.21.15.94:0/701331063 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c050070 con 0x559bf87126b0 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.748+0000 7fa1997fa640 1 -- 172.21.15.94:0/701331063 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x559bf87126b0 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.748+0000 7fa1997fa640 1 -- 172.21.15.94:0/701331063 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c02fbb0 con 0x559bf87126b0 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- 172.21.15.94:0/701331063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 msgr2=0x559bf8acf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 0x559bf8acf9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02aeb0 tx=0x7fa19c04a000 comp rx=0 tx=0).stop 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- 172.21.15.94:0/701331063 shutdown_connections 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf87126b0 0x559bf8acf9c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf870a4d0 0x559bf8af5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- 172.21.15.94:0/701331063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- 172.21.15.94:0/701331063 >> 172.21.15.94:0/701331063 conn(0x559bf87274e0 msgr2=0x559bf8ae6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- 172.21.15.94:0/701331063 shutdown_connections 2024-04-25T14:04:46.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- 172.21.15.94:0/701331063 wait complete. 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 -- start start 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 Processor -- start 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.749+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41aa5c0 con 0x559bf87126b0 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a9ef0 con 0x559bf86f1f60 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a7130 con 0x559bf870a4d0 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 10 client.0 _ll_register_callbacks cb 0x7fa18c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36900/0 (socket says 172.21.15.94:36900) 2024-04-25T14:04:46.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a3fff640 1 -- 172.21.15.94:0/2818120256 learned_addr learned my addr 172.21.15.94:0/2818120256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 msgr2=0x559bf8ae9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2818120256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 msgr2=0x559bf8b5bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 -- 172.21.15.94:0/2818120256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf8b5c530 con 0x559bf86f1f60 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a3fff640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1a37fe640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae9940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0aef20 tx=0x7fa18c0a7500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358fc13e0b715029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa182ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0c0070 con 0x559bf86f1f60 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa18c0a1b60 con 0x559bf86f1f60 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.750+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c036040 con 0x559bf86f1f60 2024-04-25T14:04:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa18c0a2d40 con 0x559bf86f1f60 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 10 client.33014 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bf8b5d610 con 0x559bf86f1f60 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x559bf86f1f60 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fa18c0041b0 con 0x559bf86f1f60 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 10 client.33014 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 10 client.33014 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 20 client.33014 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf8733390 0x559bf8b5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 4 client.33014 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.751+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bf8a41800 con 0x559bf8733390 2024-04-25T14:04:46.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.752+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf8733390 0x559bf8b5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.752+0000 7fa1a2ffd640 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf8733390 0x559bf8b5ffd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b51bbde98c3628f5 server_cookie=bcadc968784f806f in_seq=0 out_seq=0 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.752+0000 7fa182ffd640 10 client.33014 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.754+0000 7fa182ffd640 1 -- 172.21.15.94:0/2818120256 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bf8a41800 con 0x559bf8733390 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.754+0000 7fa182ffd640 10 client.33014 handle_command_reply: tid=0 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.754+0000 7fa1b1184740 1 client.33014 shutdown 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 msgr2=0x559bf8ae9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae9940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0aef20 tx=0x7fa18c0a7500 comp rx=0 tx=0).stop 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 shutdown_connections 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x559bf8733390 0x559bf8b5ffd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559bf87126b0 0x559bf8b5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bf870a4d0 0x559bf8ae9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa182ffd640 0 client.33014 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:04:46.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 --2- 172.21.15.94:0/2818120256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bf86f1f60 0x559bf8ae9940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 >> 172.21.15.94:0/2818120256 conn(0x559bf87274e0 msgr2=0x559bf8add6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 shutdown_connections 2024-04-25T14:04:46.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 1 -- 172.21.15.94:0/2818120256 wait complete. 2024-04-25T14:04:46.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1b1184740 20 client.33014 trim_cache size 0 max 16384 2024-04-25T14:04:46.759 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 msgr2=0x7fa1900b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 0x7fa1900b2070 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005e30 tx=0x7fa19c0099b0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 msgr2=0x7fa1a4199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00e9f0 tx=0x7fa18c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 shutdown_connections 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa1900afb60 0x7fa1900b2070 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4109870 0x7fa1a41a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1a4104b70 0x7fa1a419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 --2- 172.21.15.94:0/2879138557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1a4103930 0x7fa1a4199fc0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.755+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 >> 172.21.15.94:0/2879138557 conn(0x7fa1a40feb20 msgr2=0x7fa1a4105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.756+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 shutdown_connections 2024-04-25T14:04:46.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.756+0000 7fa1a8f9d640 1 -- 172.21.15.94:0/2879138557 wait complete. 2024-04-25T14:04:46.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.901+0000 7f560569f640 1 -- 172.21.15.94:0/1615569397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5600103c20 msgr2=0x7f5600104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.901+0000 7f560569f640 1 --2- 172.21.15.94:0/1615569397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5600103c20 0x7f5600104030 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009900 tx=0x7f55f402c620 comp rx=0 tx=0).stop 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.901+0000 7f560569f640 1 -- 172.21.15.94:0/1615569397 shutdown_connections 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.901+0000 7f560569f640 1 --2- 172.21.15.94:0/1615569397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 0x7f560010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 --2- 172.21.15.94:0/1615569397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5600103c20 0x7f5600104030 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 --2- 172.21.15.94:0/1615569397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56001029e0 0x7f56001036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 -- 172.21.15.94:0/1615569397 >> 172.21.15.94:0/1615569397 conn(0x7f56000fdbd0 msgr2=0x7f5600100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 -- 172.21.15.94:0/1615569397 shutdown_connections 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 -- 172.21.15.94:0/1615569397 wait complete. 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 Processor -- start 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.902+0000 7f560569f640 1 -- start start 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5600103c20 0x7f5600198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 0x7f560019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f560010d400 con 0x7f56001029e0 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f560010d280 con 0x7f5600108920 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f560569f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f560010d580 con 0x7f5600103c20 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36924/0 (socket says 172.21.15.94:36924) 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 -- 172.21.15.94:0/2657317537 learned_addr learned my addr 172.21.15.94:0/2657317537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5600103c20 0x7f5600198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 0x7f560019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 -- 172.21.15.94:0/2657317537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5600103c20 msgr2=0x7f5600198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5600103c20 0x7f5600198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 -- 172.21.15.94:0/2657317537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 msgr2=0x7f560019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 0x7f560019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 -- 172.21.15.94:0/2657317537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560019fe80 con 0x7f56001029e0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.903+0000 7f55feffd640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e9f0 tx=0x7f55f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.904+0000 7f55e7fff640 1 -- 172.21.15.94:0/2657317537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f000ccc0 con 0x7f56001029e0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.904+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a0170 con 0x7f56001029e0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.904+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a0710 con 0x7f56001029e0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.904+0000 7f55e7fff640 1 -- 172.21.15.94:0/2657317537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f00035f0 con 0x7f56001029e0 2024-04-25T14:04:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.904+0000 7f55e7fff640 1 -- 172.21.15.94:0/2657317537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f000ce60 con 0x7f56001029e0 2024-04-25T14:04:46.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.906+0000 7f560c62a740 1 Processor -- start 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.906+0000 7f560c62a740 1 -- start start 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 0x560ab4f5ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b8f4d0 0x560ab4f5f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ab4ebed80 con 0x560ab4b8f4d0 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ab4e78a20 con 0x560ab4b976b0 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f560c62a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ab4ebad40 con 0x560ab4bb8390 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 0x560ab4f5ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35598/0 (socket says 172.21.15.94:35598) 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 -- 172.21.15.94:0/215932278 learned_addr learned my addr 172.21.15.94:0/215932278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 0x560ab4f5ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52866/0 (socket says 172.21.15.94:52866) 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 -- 172.21.15.94:0/215932278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 msgr2=0x560ab4f5ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 0x560ab4f5ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 -- 172.21.15.94:0/215932278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b8f4d0 msgr2=0x560ab4f5f1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b8f4d0 0x560ab4f5f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 -- 172.21.15.94:0/215932278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab4e83c30 con 0x560ab4b976b0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55e7fff640 1 -- 172.21.15.94:0/2657317537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f55f0003aa0 con 0x7f56001029e0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.907+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55f4033500 tx=0x7f55f4005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67916a705981c6da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.908+0000 7f55e57fa640 1 -- 172.21.15.94:0/215932278 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4041070 con 0x560ab4b976b0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.908+0000 7f55e57fa640 1 -- 172.21.15.94:0/215932278 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f4033d90 con 0x560ab4b976b0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.908+0000 7f55e57fa640 1 -- 172.21.15.94:0/215932278 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4037070 con 0x560ab4b976b0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.908+0000 7f55e7fff640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 0x7f55e80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.908+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 0x7f55e80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f55e7fff640 1 -- 172.21.15.94:0/2657317537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55f00def10 con 0x7f56001029e0 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- 172.21.15.94:0/215932278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 msgr2=0x560ab4f5fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55f4033500 tx=0x7f55f4005e00 comp rx=0 tx=0).stop 2024-04-25T14:04:46.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 0x7f55e80b2080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x560ab4f5fc70 tx=0x7f55f80096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- 172.21.15.94:0/215932278 shutdown_connections 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f5fe30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b8f4d0 0x560ab4f5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 --2- 172.21.15.94:0/215932278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4bb8390 0x560ab4f5ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- 172.21.15.94:0/215932278 >> 172.21.15.94:0/215932278 conn(0x560ab4bac4e0 msgr2=0x560ab4f56020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- 172.21.15.94:0/215932278 shutdown_connections 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- 172.21.15.94:0/215932278 wait complete. 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.909+0000 7f560c62a740 1 -- start start 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 Processor -- start 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4f68e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4bb8390 0x560ab4f6fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ab4f60930 con 0x560ab4b976b0 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560ab4e45dc0 con 0x560ab4bb8390 2024-04-25T14:04:46.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560ab4f60ab0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f560c62a740 10 client.0 _ll_register_callbacks cb 0x560ab4e61600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55fe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4f68e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52882/0 (socket says 172.21.15.94:52882) 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55fe7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4f68e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36936/0 (socket says 172.21.15.94:36936) 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 -- 172.21.15.94:0/4154187428 learned_addr learned my addr 172.21.15.94:0/4154187428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4bb8390 0x560ab4f6fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 -- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4bb8390 msgr2=0x560ab4f6fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4bb8390 0x560ab4f6fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 -- 172.21.15.94:0/4154187428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 msgr2=0x560ab4f68e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4f68e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.910+0000 7f55feffd640 1 -- 172.21.15.94:0/4154187428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab4bd03f0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55feffd640 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55f00a72e0 tx=0x7f55f00a6b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45fb60d15d363dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00c0020 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55f00afd20 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f4033d90 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55f00c8050 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f560c62a740 10 client.32665 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ab4c9a7d0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f4033d90 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f55f0003fb0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f560c62a740 10 client.32665 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.911+0000 7f560c62a740 10 client.32665 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f560c62a740 20 client.32665 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4b76f60 0x560ab4f76b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f560c62a740 4 client.32665 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab4ecc990 con 0x560ab4b76f60 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4b76f60 0x560ab4f76b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f55e2ff5640 10 client.32665 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.912+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4b76f60 0x560ab4f76b10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=199d0592c13e316b server_cookie=c39d06a1e5cbe852 in_seq=0 out_seq=0 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f55e2ff5640 1 -- 172.21.15.94:0/4154187428 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ab4ecc990 con 0x560ab4b76f60 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f55e2ff5640 10 client.32665 handle_command_reply: tid=0 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 client.32665 shutdown 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 msgr2=0x560ab4f68930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55f00a72e0 tx=0x7f55f00a6b90 comp rx=0 tx=0).stop 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 shutdown_connections 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4b76f60 0x560ab4f76b10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4bb8390 0x560ab4f6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f55e2ff5640 0 client.32665 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:46.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4f68e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 --2- 172.21.15.94:0/4154187428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f68930 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 >> 172.21.15.94:0/4154187428 conn(0x560ab4bac4e0 msgr2=0x560ab4f55040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.913+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 shutdown_connections 2024-04-25T14:04:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.914+0000 7f560c62a740 1 -- 172.21.15.94:0/4154187428 wait complete. 2024-04-25T14:04:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.914+0000 7f560c62a740 20 client.32665 trim_cache size 0 max 16384 2024-04-25T14:04:46.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 Processor -- start 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 -- start start 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f54610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56001a9300 con 0x560ab4b76f60 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56001aae40 con 0x560ab4b976b0 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f560c62a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56001a99d0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f55feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.929+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f54610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36944/0 (socket says 172.21.15.94:36944) 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55feffd640 1 -- 172.21.15.94:0/1510237508 learned_addr learned my addr 172.21.15.94:0/1510237508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 -- 172.21.15.94:0/1510237508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 msgr2=0x560ab4f54610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f54610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 -- 172.21.15.94:0/1510237508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 msgr2=0x560ab4f677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 -- 172.21.15.94:0/1510237508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab4f640f0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55feffd640 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55fe7fc640 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6fd60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x560ab4f699a0 tx=0x7f55f402c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427cae9585624283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55e4ff9640 1 -- 172.21.15.94:0/1510237508 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4037040 con 0x560ab4b8f4d0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55e4ff9640 1 -- 172.21.15.94:0/1510237508 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f4056070 con 0x560ab4b8f4d0 2024-04-25T14:04:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.930+0000 7f55e4ff9640 1 -- 172.21.15.94:0/1510237508 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4041040 con 0x560ab4b8f4d0 2024-04-25T14:04:46.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 -- 172.21.15.94:0/1510237508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 msgr2=0x560ab4f6fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6fd60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x560ab4f699a0 tx=0x7f55f402c620 comp rx=0 tx=0).stop 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 -- 172.21.15.94:0/1510237508 shutdown_connections 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b976b0 0x560ab4f54610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6fd60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 --2- 172.21.15.94:0/1510237508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b76f60 0x560ab4f677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 -- 172.21.15.94:0/1510237508 >> 172.21.15.94:0/1510237508 conn(0x560ab4bac4e0 msgr2=0x560ab4f670c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 -- 172.21.15.94:0/1510237508 shutdown_connections 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.931+0000 7f560c62a740 1 -- 172.21.15.94:0/1510237508 wait complete. 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 -- start start 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 Processor -- start 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4fdceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56001aa770 con 0x560ab4b976b0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56001aa0a0 con 0x560ab4b8f4d0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56001a72e0 con 0x560ab4b76f60 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f560c62a740 10 client.0 _ll_register_callbacks cb 0x7f55f00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f55feffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f55fe7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4fdceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f55feffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52906/0 (socket says 172.21.15.94:52906) 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.932+0000 7f55feffd640 1 -- 172.21.15.94:0/805514890 learned_addr learned my addr 172.21.15.94:0/805514890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 -- 172.21.15.94:0/805514890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 msgr2=0x560ab4f6ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 -- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b8f4d0 msgr2=0x560ab4f6d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 -- 172.21.15.94:0/805514890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab4f67d10 con 0x560ab4b976b0 2024-04-25T14:04:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55feffd640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55ff7fe640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4fdceb0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f55f8004010 tx=0x7f55f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a7ee354f740d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.933+0000 7f55e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:46.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.935+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f802e060 con 0x560ab4b976b0 2024-04-25T14:04:46.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.935+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f4056070 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.935+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f55f801d410 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55f801d610 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 10 client.23649 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560ab4fde530 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f801dc60 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f55f8033b40 con 0x560ab4b976b0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 10 client.23649 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 10 client.23649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 20 client.23649 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4bb8390 0x560ab4fe0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 4 client.23649 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f55f4056070 con 0x560ab4bb8390 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.936+0000 7f55feffd640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4bb8390 0x560ab4fe0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f55e2ff5640 10 client.23649 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:46.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f55feffd640 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4bb8390 0x560ab4fe0f80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5eb75af8348edd16 server_cookie=156678c61f6c92d6 in_seq=0 out_seq=0 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f55e2ff5640 1 -- 172.21.15.94:0/805514890 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f55f4056070 con 0x560ab4bb8390 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f55e2ff5640 10 client.23649 handle_command_reply: tid=0 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 client.23649 shutdown 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 msgr2=0x560ab4fdceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4fdceb0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f55f8004010 tx=0x7f55f8004a40 comp rx=0 tx=0).stop 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 shutdown_connections 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x560ab4bb8390 0x560ab4fe0f80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ab4b976b0 0x560ab4fdceb0 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f55e2ff5640 0 client.23649 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ab4b8f4d0 0x560ab4f6d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 --2- 172.21.15.94:0/805514890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560ab4b76f60 0x560ab4f6ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 >> 172.21.15.94:0/805514890 conn(0x560ab4bac4e0 msgr2=0x560ab4f58e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 shutdown_connections 2024-04-25T14:04:46.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.937+0000 7f560c62a740 1 -- 172.21.15.94:0/805514890 wait complete. 2024-04-25T14:04:46.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560c62a740 20 client.23649 trim_cache size 0 max 16384 2024-04-25T14:04:46.942 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 msgr2=0x7f55e80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 0x7f55e80b2080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x560ab4f5fc70 tx=0x7f55f80096a0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 msgr2=0x7f5600198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e9f0 tx=0x7f55f000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 shutdown_connections 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f55e80afb70 0x7f55e80b2080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5600108920 0x7f560019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5600103c20 0x7f5600198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 --2- 172.21.15.94:0/2657317537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56001029e0 0x7f5600198340 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 >> 172.21.15.94:0/2657317537 conn(0x7f56000fdbd0 msgr2=0x7f5600104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 shutdown_connections 2024-04-25T14:04:46.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:46.938+0000 7f560569f640 1 -- 172.21.15.94:0/2657317537 wait complete. 2024-04-25T14:04:46.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:46 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2363854529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:46.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:04:46.996 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2363854529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:47.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 -- 172.21.15.94:0/1335548643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 msgr2=0x7f9814105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 --2- 172.21.15.94:0/1335548643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f9814105140 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f9804009900 tx=0x7f980402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 -- 172.21.15.94:0/1335548643 shutdown_connections 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 --2- 172.21.15.94:0/1335548643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 0x7f981410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 --2- 172.21.15.94:0/1335548643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f9814105140 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 --2- 172.21.15.94:0/1335548643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 0x7f98141047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 -- 172.21.15.94:0/1335548643 >> 172.21.15.94:0/1335548643 conn(0x7f98140fece0 msgr2=0x7f9814101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 -- 172.21.15.94:0/1335548643 shutdown_connections 2024-04-25T14:04:47.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.085+0000 7f981b8f9640 1 -- 172.21.15.94:0/1335548643 wait complete. 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 Processor -- start 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 -- start start 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 0x7f981419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 0x7f98141a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f981410f5f0 con 0x7f9814104d30 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f981410f470 con 0x7f9814109a30 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f981b8f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f981410f770 con 0x7f9814103af0 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f9818e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f9818e94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36968/0 (socket says 172.21.15.94:36968) 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f9818e94640 1 -- 172.21.15.94:0/932734709 learned_addr learned my addr 172.21.15.94:0/932734709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f9819695640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 0x7f981419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.086+0000 7f9819e96640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 0x7f98141a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 -- 172.21.15.94:0/932734709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 msgr2=0x7f981419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 0x7f981419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 -- 172.21.15.94:0/932734709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 msgr2=0x7f98141a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 0x7f98141a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 -- 172.21.15.94:0/932734709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141a1d90 con 0x7f9814104d30 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f9818e94640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f9804006010 tx=0x7f9804004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f980a7fc640 1 -- 172.21.15.94:0/932734709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980403f030 con 0x7f9814104d30 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f980a7fc640 1 -- 172.21.15.94:0/932734709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9804003cb0 con 0x7f9814104d30 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141a2020 con 0x7f9814104d30 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f980a7fc640 1 -- 172.21.15.94:0/932734709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804035070 con 0x7f9814104d30 2024-04-25T14:04:47.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.087+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141a2510 con 0x7f9814104d30 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.089+0000 7f98227ab740 1 Processor -- start 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.089+0000 7f98227ab740 1 -- start start 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f980a7fc640 1 -- 172.21.15.94:0/932734709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9804042030 con 0x7f9814104d30 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e903d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e904a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a39e867d80 con 0x55a39e561390 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a39e821a20 con 0x55a39e5384d0 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f98227ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a39e863d40 con 0x55a39e5406b0 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36972/0 (socket says 172.21.15.94:36972) 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 -- 172.21.15.94:0/343770058 learned_addr learned my addr 172.21.15.94:0/343770058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819695640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e903d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9818e94640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e904a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 -- 172.21.15.94:0/343770058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 msgr2=0x55a39e904a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e904a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 -- 172.21.15.94:0/343770058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 msgr2=0x55a39e903d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e903d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 -- 172.21.15.94:0/343770058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39e82cc30 con 0x55a39e561390 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819695640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f9819e96640 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f980c0039f0 tx=0x7f980c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757fafd2cde38cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f980a7fc640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 0x7f98000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.090+0000 7f97ff7fe640 1 -- 172.21.15.94:0/343770058 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c01a470 con 0x55a39e561390 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f97ff7fe640 1 -- 172.21.15.94:0/343770058 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c015070 con 0x55a39e561390 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f97ff7fe640 1 -- 172.21.15.94:0/343770058 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0107e0 con 0x55a39e561390 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f9819695640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 0x7f98000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f980a7fc640 1 -- 172.21.15.94:0/932734709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f98040ff2a0 con 0x7f9814104d30 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f98227ab740 1 -- 172.21.15.94:0/343770058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 msgr2=0x55a39e903810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f9819695640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 0x7f98000b2000 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55a39e903530 tx=0x7f981000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.091+0000 7f98227ab740 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f980c0039f0 tx=0x7f980c003eb0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- 172.21.15.94:0/343770058 shutdown_connections 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e904a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- 172.21.15.94:0/343770058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e903810 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- 172.21.15.94:0/343770058 >> 172.21.15.94:0/343770058 conn(0x55a39e5554e0 msgr2=0x55a39e8ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- 172.21.15.94:0/343770058 shutdown_connections 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- 172.21.15.94:0/343770058 wait complete. 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- start start 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 Processor -- start 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a39e905530 con 0x55a39e561390 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a39e7eedc0 con 0x55a39e5384d0 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a39e9056b0 con 0x55a39e5406b0 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f98227ab740 10 client.0 _ll_register_callbacks cb 0x55a39e80a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.092+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36978/0 (socket says 172.21.15.94:36978) 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 -- 172.21.15.94:0/3989210184 learned_addr learned my addr 172.21.15.94:0/3989210184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819695640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9818e94640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 -- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 msgr2=0x55a39e90dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 -- 172.21.15.94:0/3989210184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 msgr2=0x55a39e90d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 -- 172.21.15.94:0/3989210184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39e5793f0 con 0x55a39e561390 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f9819e96640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f980c0049a0 tx=0x7f980c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a386711710d22b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f97fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c0116d0 con 0x55a39e561390 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c015070 con 0x55a39e561390 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f980c011870 con 0x55a39e561390 2024-04-25T14:04:47.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f980c011a70 con 0x55a39e561390 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f98227ab740 10 client.23661 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a39e6437d0 con 0x55a39e561390 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.093+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c015070 con 0x55a39e561390 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f980c01a590 con 0x55a39e561390 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 10 client.23661 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 10 client.23661 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 20 client.23661 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e51ff60 0x55a39e91b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 4 client.23661 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a39e875990 con 0x55a39e51ff60 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f9819695640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e51ff60 0x55a39e91b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f9819695640 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e51ff60 0x55a39e91b060 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1074d2b689d5a72 server_cookie=1bb68aae6bbbc005 in_seq=0 out_seq=0 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.094+0000 7f97fcff9640 10 client.23661 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:47.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f97fcff9640 1 -- 172.21.15.94:0/3989210184 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a39e875990 con 0x55a39e51ff60 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f97fcff9640 10 client.23661 handle_command_reply: tid=0 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 client.23661 shutdown 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 msgr2=0x55a39e914b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f980c0049a0 tx=0x7f980c011de0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 shutdown_connections 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e51ff60 0x55a39e91b060 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e914b00 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f97fcff9640 0 client.23661 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 --2- 172.21.15.94:0/3989210184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 >> 172.21.15.94:0/3989210184 conn(0x55a39e5554e0 msgr2=0x55a39e8fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 shutdown_connections 2024-04-25T14:04:47.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.095+0000 7f98227ab740 1 -- 172.21.15.94:0/3989210184 wait complete. 2024-04-25T14:04:47.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.096+0000 7f98227ab740 20 client.23661 trim_cache size 0 max 16384 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.115+0000 7f98227ab740 1 Processor -- start 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.115+0000 7f98227ab740 1 -- start start 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98141ab590 con 0x55a39e561390 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98141ad0d0 con 0x55a39e5406b0 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f98227ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98141abc60 con 0x55a39e5384d0 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36988/0 (socket says 172.21.15.94:36988) 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 -- 172.21.15.94:0/253857960 learned_addr learned my addr 172.21.15.94:0/253857960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9818e94640 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819695640 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 -- 172.21.15.94:0/253857960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 msgr2=0x55a39e90d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 -- 172.21.15.94:0/253857960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 msgr2=0x55a39e90dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 -- 172.21.15.94:0/253857960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a39e905790 con 0x55a39e561390 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f9819e96640 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f980c005a20 tx=0x7f980c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be46ca0839db9053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f97feffd640 1 -- 172.21.15.94:0/253857960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c005ac0 con 0x55a39e561390 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.116+0000 7f97feffd640 1 -- 172.21.15.94:0/253857960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c015070 con 0x55a39e561390 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f97feffd640 1 -- 172.21.15.94:0/253857960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c02f810 con 0x55a39e561390 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 -- 172.21.15.94:0/253857960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 msgr2=0x55a39e91f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f980c005a20 tx=0x7f980c011c10 comp rx=0 tx=0).stop 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 -- 172.21.15.94:0/253857960 shutdown_connections 2024-04-25T14:04:47.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e91f720 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e90dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 --2- 172.21.15.94:0/253857960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e90d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.117+0000 7f98227ab740 1 -- 172.21.15.94:0/253857960 >> 172.21.15.94:0/253857960 conn(0x55a39e5554e0 msgr2=0x55a39e908390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- 172.21.15.94:0/253857960 shutdown_connections 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- 172.21.15.94:0/253857960 wait complete. 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- start start 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 Processor -- start 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e911410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e911950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e912880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98141aca00 con 0x55a39e561390 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98141ac330 con 0x55a39e5406b0 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98141a9570 con 0x55a39e5384d0 2024-04-25T14:04:47.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.118+0000 7f98227ab740 10 client.0 _ll_register_callbacks cb 0x7f981000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9819695640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e911410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e911950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9819e96640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e912880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9819695640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e911410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52942/0 (socket says 172.21.15.94:52942) 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9819695640 1 -- 172.21.15.94:0/1278876187 learned_addr learned my addr 172.21.15.94:0/1278876187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 -- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 msgr2=0x55a39e911410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e911410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 -- 172.21.15.94:0/1278876187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 msgr2=0x55a39e912880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e912880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 -- 172.21.15.94:0/1278876187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a39e912e40 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f9818e94640 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e911950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98040fcd50 tx=0x7f980403b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d0a70ef5cdaec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f97fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804049040 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c015070 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.119+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f980403bc20 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f98040c70e0 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 10 client.33050 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a39e914130 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f980c015070 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f98040b0070 con 0x55a39e5406b0 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 10 client.33050 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 10 client.33050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:04:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 20 client.33050 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e4f53a0 0x55a39e985800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 4 client.33050 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a39e86f800 con 0x55a39e4f53a0 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.120+0000 7f9819695640 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e4f53a0 0x55a39e985800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f97fcff9640 10 client.33050 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f9819695640 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e4f53a0 0x55a39e985800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b53078a42dd1fc0 server_cookie=7af2cebd49fc81a3 in_seq=0 out_seq=0 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f97fcff9640 1 -- 172.21.15.94:0/1278876187 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a39e86f800 con 0x55a39e4f53a0 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f97fcff9640 10 client.33050 handle_command_reply: tid=0 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 client.33050 shutdown 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 msgr2=0x55a39e911950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e911950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98040fcd50 tx=0x7f980403b6b0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 shutdown_connections 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55a39e4f53a0 0x55a39e985800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a39e561390 0x55a39e912880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a39e5406b0 0x55a39e911950 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f97fcff9640 0 client.33050 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 --2- 172.21.15.94:0/1278876187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a39e5384d0 0x55a39e911410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.121+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 >> 172.21.15.94:0/1278876187 conn(0x55a39e5554e0 msgr2=0x55a39e90d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 shutdown_connections 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f98227ab740 1 -- 172.21.15.94:0/1278876187 wait complete. 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f98227ab740 20 client.33050 trim_cache size 0 max 16384 2024-04-25T14:04:47.129 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 msgr2=0x7f98000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 0x7f98000b2000 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55a39e903530 tx=0x7f981000b390 comp rx=0 tx=0).stop 2024-04-25T14:04:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 msgr2=0x7f981419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f9804006010 tx=0x7f9804004340 comp rx=0 tx=0).stop 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 shutdown_connections 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f98000afaf0 0x7f98000b2000 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9814109a30 0x7f98141a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9814104d30 0x7f981419a790 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 --2- 172.21.15.94:0/932734709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9814103af0 0x7f981419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 >> 172.21.15.94:0/932734709 conn(0x7f98140fece0 msgr2=0x7f9814105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 shutdown_connections 2024-04-25T14:04:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.122+0000 7f981b8f9640 1 -- 172.21.15.94:0/932734709 wait complete. 2024-04-25T14:04:47.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:04:47.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:46 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2363854529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:47.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 201468 18.83 MB/sec execute 449 sec latency 76.039 ms 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 -- 172.21.15.94:0/2608710564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108920 msgr2=0x7fcfc810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2608710564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108920 0x7fcfc810ad40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009ca0 tx=0x7fcfc002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 -- 172.21.15.94:0/2608710564 shutdown_connections 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2608710564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8108920 0x7fcfc810ad40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2608710564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8103c20 0x7fcfc8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2608710564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc81029e0 0x7fcfc81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 -- 172.21.15.94:0/2608710564 >> 172.21.15.94:0/2608710564 conn(0x7fcfc80fdbd0 msgr2=0x7fcfc8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 -- 172.21.15.94:0/2608710564 shutdown_connections 2024-04-25T14:04:47.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.256+0000 7fcfcf442640 1 -- 172.21.15.94:0/2608710564 wait complete. 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 Processor -- start 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 -- start start 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 0x7fcfc819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 0x7fcfc81a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810f5f0 con 0x7fcfc8108920 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810f470 con 0x7fcfc8103c20 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcf442640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc810f770 con 0x7fcfc81029e0 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcd1de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcd9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 0x7fcfc81a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 0x7fcfc819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcd1de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:52960/0 (socket says 172.21.15.94:52960) 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2485804626 learned_addr learned my addr 172.21.15.94:0/2485804626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcd9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 0x7fcfc81a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37006/0 (socket says 172.21.15.94:37006) 2024-04-25T14:04:47.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/2485804626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 msgr2=0x7fcfc819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 msgr2=0x7fcfc81a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 0x7fcfc81a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.257+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/2485804626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc81a2000 con 0x7fcfc8103c20 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 0x7fcfc819aa00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e9f0 tx=0x7fcfb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfbe7fc640 1 -- 172.21.15.94:0/2485804626 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800ccc0 con 0x7fcfc8103c20 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfbe7fc640 1 -- 172.21.15.94:0/2485804626 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb80108e0 con 0x7fcfc8103c20 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc810cda0 con 0x7fcfc8103c20 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfbe7fc640 1 -- 172.21.15.94:0/2485804626 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8018880 con 0x7fcfc8103c20 2024-04-25T14:04:47.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.258+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc810d340 con 0x7fcfc8103c20 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.261+0000 7fcfd63c9740 1 Processor -- start 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.261+0000 7fcfd63c9740 1 -- start start 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfbe7fc640 1 -- 172.21.15.94:0/2485804626 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcfb8018a20 con 0x7fcfc8103c20 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x55632554f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x55632554fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5563254b3d80 con 0x55632518c6b0 2024-04-25T14:04:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55632546da20 con 0x5563251844d0 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563254afd40 con 0x5563251ad390 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37018/0 (socket says 172.21.15.94:37018) 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/2555290266 learned_addr learned my addr 172.21.15.94:0/2555290266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x55632554fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.262+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x55632554f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2555290266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 msgr2=0x55632554f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x55632554f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2555290266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 msgr2=0x556325550a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2555290266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556325478c30 con 0x5563251844d0 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfbe7fc640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 0x7fcfb40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x55632554fd90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c40 tx=0x7fcfc4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80c2e56cf41344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 0x7fcfb40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfb37fe640 1 -- 172.21.15.94:0/2555290266 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009d30 con 0x5563251844d0 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfb37fe640 1 -- 172.21.15.94:0/2555290266 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc40054d0 con 0x5563251844d0 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.263+0000 7fcfb37fe640 1 -- 172.21.15.94:0/2555290266 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004b60 con 0x5563251844d0 2024-04-25T14:04:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.264+0000 7fcfbe7fc640 1 -- 172.21.15.94:0/2485804626 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcfb80def70 con 0x7fcfc8103c20 2024-04-25T14:04:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2555290266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 msgr2=0x55632554fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 0x7fcfb40b2080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x556325550860 tx=0x7fcfc0009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x55632554fd90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c40 tx=0x7fcfc4005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2555290266 shutdown_connections 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325550a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x55632554fd90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2555290266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x55632554f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2555290266 >> 172.21.15.94:0/2555290266 conn(0x5563251a14e0 msgr2=0x55632554b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2555290266 shutdown_connections 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2555290266 wait complete. 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 -- start start 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.265+0000 7fcfd63c9740 1 Processor -- start 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x556325559450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x5563255608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556325551530 con 0x55632518c6b0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55632543adc0 con 0x5563251844d0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563255516b0 con 0x5563251ad390 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfd63c9740 10 client.0 _ll_register_callbacks cb 0x556325456600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37020/0 (socket says 172.21.15.94:37020) 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/3073271617 learned_addr learned my addr 172.21.15.94:0/3073271617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x556325559450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x5563255608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/3073271617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 msgr2=0x5563255608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x5563255608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 msgr2=0x556325559450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x556325559450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.266+0000 7fcfcc9dd640 1 -- 172.21.15.94:0/3073271617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563251c53f0 con 0x55632518c6b0 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x556325559450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88790ceae49c5476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a6bd0 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc40054d0 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcfb8010a90 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.267+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80a6f70 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 10 client.23679 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55632528f7d0 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc40054d0 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcfb80afd80 con 0x55632518c6b0 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 10 client.23679 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 10 client.23679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 20 client.23679 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55632516bf60 0x5563255672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 4 client.23679 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:47.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563254c1990 con 0x55632516bf60 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.268+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55632516bf60 0x5563255672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.269+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55632516bf60 0x5563255672f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1640da4e9e759eab server_cookie=f52e9850376d0667 in_seq=0 out_seq=0 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.269+0000 7fcfb0ff9640 10 client.23679 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/3073271617 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563254c1990 con 0x55632516bf60 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfb0ff9640 10 client.23679 handle_command_reply: tid=0 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 client.23679 shutdown 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 msgr2=0x556325559990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb800ce60 comp rx=0 tx=0).stop 2024-04-25T14:04:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 shutdown_connections 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55632516bf60 0x5563255672f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfb0ff9640 0 client.23679 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251ad390 0x5563255608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325559990 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/3073271617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563251844d0 0x556325559450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.270+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 >> 172.21.15.94:0/3073271617 conn(0x5563251a14e0 msgr2=0x55632554df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.271+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 shutdown_connections 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.271+0000 7fcfd63c9740 1 -- 172.21.15.94:0/3073271617 wait complete. 2024-04-25T14:04:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.271+0000 7fcfd63c9740 20 client.23679 trim_cache size 0 max 16384 2024-04-25T14:04:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.288+0000 7fcfd63c9740 1 Processor -- start 2024-04-25T14:04:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.288+0000 7fcfd63c9740 1 -- start start 2024-04-25T14:04:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.288+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x556325554130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.288+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x556325554670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81a92d0 con 0x55632518c6b0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81aae10 con 0x55632516bf60 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81a99a0 con 0x5563251844d0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37026/0 (socket says 172.21.15.94:37026) 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 -- 172.21.15.94:0/2291147094 learned_addr learned my addr 172.21.15.94:0/2291147094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x556325554670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x556325554130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 -- 172.21.15.94:0/2291147094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 msgr2=0x556325554670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x556325554670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 -- 172.21.15.94:0/2291147094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 msgr2=0x556325554130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x556325554130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 -- 172.21.15.94:0/2291147094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556325560d50 con 0x55632518c6b0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0035000 tx=0x7fcfc0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b562d2940060aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.289+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2291147094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0004040 con 0x55632518c6b0 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.290+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2291147094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc0036040 con 0x55632518c6b0 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.290+0000 7fcfb2ffd640 1 -- 172.21.15.94:0/2291147094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc002fd00 con 0x55632518c6b0 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2291147094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 msgr2=0x556325560810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0035000 tx=0x7fcfc0044000 comp rx=0 tx=0).stop 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2291147094 shutdown_connections 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x556325560810 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x556325554670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2291147094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x556325554130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2291147094 >> 172.21.15.94:0/2291147094 conn(0x5563251a14e0 msgr2=0x55632554f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2291147094 shutdown_connections 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2291147094 wait complete. 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 -- start start 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.291+0000 7fcfd63c9740 1 Processor -- start 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x55632555f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x55632555fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81aa740 con 0x55632518c6b0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81aa070 con 0x55632516bf60 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81a72b0 con 0x5563251844d0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfd63c9740 10 client.0 _ll_register_callbacks cb 0x7fcfb80e2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcc9dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x55632555f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35748/0 (socket says 172.21.15.94:35748) 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2342518580 learned_addr learned my addr 172.21.15.94:0/2342518580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd9df640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x55632555fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2342518580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 msgr2=0x55632555f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x55632555f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2342518580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 msgr2=0x55632555fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x55632555fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 -- 172.21.15.94:0/2342518580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563255cd690 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x55632555f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfcd1de640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4016c10 tx=0x7fcfc4003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2734822733ef3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.292+0000 7fcfb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4005c50 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc0036040 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcfc4005040 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfc4005240 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 10 client.33068 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5563255ce9d0 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc0036040 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcfc40045f0 con 0x55632516bf60 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 10 client.33068 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 10 client.33068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 20 client.33068 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5563251ad390 0x5563255d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 4 client.33068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563254bb800 con 0x5563251ad390 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.293+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5563251ad390 0x5563255d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfb0ff9640 10 client.33068 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfcc9dd640 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5563251ad390 0x5563255d10b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ee2b80cf1b942af server_cookie=13a4d1f20513dbc1 in_seq=0 out_seq=0 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfb0ff9640 1 -- 172.21.15.94:0/2342518580 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563254bb800 con 0x5563251ad390 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfb0ff9640 10 client.33068 handle_command_reply: tid=0 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 client.33068 shutdown 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 msgr2=0x55632555f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4016c10 tx=0x7fcfc4003d20 comp rx=0 tx=0).stop 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 shutdown_connections 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5563251ad390 0x5563255d10b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55632518c6b0 0x55632555fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfb0ff9640 0 client.33068 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563251844d0 0x55632555f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 --2- 172.21.15.94:0/2342518580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55632516bf60 0x55632555f090 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.294+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 >> 172.21.15.94:0/2342518580 conn(0x5563251a14e0 msgr2=0x556325559410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 shutdown_connections 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfd63c9740 1 -- 172.21.15.94:0/2342518580 wait complete. 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfd63c9740 20 client.33068 trim_cache size 0 max 16384 2024-04-25T14:04:47.301 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:47.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 msgr2=0x7fcfb40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 0x7fcfb40b2080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x556325550860 tx=0x7fcfc0009380 comp rx=0 tx=0).stop 2024-04-25T14:04:47.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 msgr2=0x7fcfc819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 0x7fcfc819aa00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e9f0 tx=0x7fcfb800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 shutdown_connections 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfb40afb70 0x7fcfb40b2080 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc8108920 0x7fcfc81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcfc8103c20 0x7fcfc819aa00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 --2- 172.21.15.94:0/2485804626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc81029e0 0x7fcfc819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.295+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 >> 172.21.15.94:0/2485804626 conn(0x7fcfc80fdbd0 msgr2=0x7fcfc8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.296+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 shutdown_connections 2024-04-25T14:04:47.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.296+0000 7fcfcf442640 1 -- 172.21.15.94:0/2485804626 wait complete. 2024-04-25T14:04:47.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:04:47.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.503+0000 7f5de2084640 1 -- 172.21.15.94:0/648828650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103c20 msgr2=0x7f5ddc104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.503+0000 7f5de2084640 1 --2- 172.21.15.94:0/648828650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103c20 0x7f5ddc104030 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009900 tx=0x7f5dd002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.505+0000 7f5de2084640 1 -- 172.21.15.94:0/648828650 shutdown_connections 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.505+0000 7f5de2084640 1 --2- 172.21.15.94:0/648828650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 0x7f5ddc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.505+0000 7f5de2084640 1 --2- 172.21.15.94:0/648828650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc103c20 0x7f5ddc104030 secure :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009900 tx=0x7f5dd002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.505+0000 7f5de2084640 1 --2- 172.21.15.94:0/648828650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.505+0000 7f5de2084640 1 -- 172.21.15.94:0/648828650 >> 172.21.15.94:0/648828650 conn(0x7f5ddc0fdbd0 msgr2=0x7f5ddc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.506+0000 7f5de2084640 1 -- 172.21.15.94:0/648828650 shutdown_connections 2024-04-25T14:04:47.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.506+0000 7f5de2084640 1 -- 172.21.15.94:0/648828650 wait complete. 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.507+0000 7f5de2084640 1 Processor -- start 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.507+0000 7f5de2084640 1 -- start start 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 0x7f5ddc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 0x7f5ddc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc10e510 con 0x7f5ddc19fa50 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc10e390 con 0x7f5ddc1029e0 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5de2084640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc10e690 con 0x7f5ddc108920 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5ddb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5ddb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35768/0 (socket says 172.21.15.94:35768) 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/3385648044 learned_addr learned my addr 172.21.15.94:0/3385648044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.508+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 0x7f5ddc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 0x7f5ddc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 -- 172.21.15.94:0/3385648044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 msgr2=0x7f5ddc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 0x7f5ddc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 -- 172.21.15.94:0/3385648044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 msgr2=0x7f5ddc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 -- 172.21.15.94:0/3385648044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a25a0 con 0x7f5ddc19fa50 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.509+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 0x7f5ddc1a1e70 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009fd0 tx=0x7f5dd400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.510+0000 7f5dd8ff9640 1 -- 172.21.15.94:0/3385648044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400ce20 con 0x7f5ddc19fa50 2024-04-25T14:04:47.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.510+0000 7f5dd8ff9640 1 -- 172.21.15.94:0/3385648044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd4005020 con 0x7f5ddc19fa50 2024-04-25T14:04:47.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.510+0000 7f5dd8ff9640 1 -- 172.21.15.94:0/3385648044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4010690 con 0x7f5ddc19fa50 2024-04-25T14:04:47.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.511+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc108d90 con 0x7f5ddc19fa50 2024-04-25T14:04:47.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.511+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1091a0 con 0x7f5ddc19fa50 2024-04-25T14:04:47.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.514+0000 7f5de9008740 1 Processor -- start 2024-04-25T14:04:47.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.515+0000 7f5dd8ff9640 1 -- 172.21.15.94:0/3385648044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5dd40054d0 con 0x7f5ddc19fa50 2024-04-25T14:04:47.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.515+0000 7f5de9008740 1 -- start start 2024-04-25T14:04:47.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.515+0000 7f5de9008740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.515+0000 7f5de9008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe8a4d0 0x5611c0255d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5de9008740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c0256a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5de9008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5611c01b9d80 con 0x5611bfe8a4d0 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5de9008740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5611c0173a20 con 0x5611bfe926b0 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5de9008740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611c01b5d40 con 0x5611bfeb3390 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53008/0 (socket says 172.21.15.94:53008) 2024-04-25T14:04:47.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2389168724 learned_addr learned my addr 172.21.15.94:0/2389168724 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c0256a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2389168724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 msgr2=0x5611c0256a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c0256a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2389168724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe8a4d0 msgr2=0x5611c0255d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe8a4d0 0x5611c0255d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2389168724 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c017ec30 con 0x5611bfeb3390 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0031460 tx=0x7f5dd0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c04850cc99251d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.516+0000 7f5dd8ff9640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 0x7f5dc40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.517+0000 7f5dd8ff9640 1 -- 172.21.15.94:0/3385648044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5dd40df400 con 0x7f5ddc19fa50 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.517+0000 7f5dc1ffb640 1 -- 172.21.15.94:0/2389168724 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035020 con 0x5611bfeb3390 2024-04-25T14:04:47.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.517+0000 7f5dc1ffb640 1 -- 172.21.15.94:0/2389168724 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00316d0 con 0x5611bfeb3390 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.517+0000 7f5dc1ffb640 1 -- 172.21.15.94:0/2389168724 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00044e0 con 0x5611bfeb3390 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.518+0000 7f5de9008740 1 -- 172.21.15.94:0/2389168724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 msgr2=0x5611c0255810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.518+0000 7f5de9008740 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0031460 tx=0x7f5dd0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.519+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 0x7f5dc40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 0x7f5dc40b2110 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x5611c0255530 tx=0x7f5dcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 -- 172.21.15.94:0/2389168724 shutdown_connections 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c0256a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe8a4d0 0x5611c0255d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 --2- 172.21.15.94:0/2389168724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfeb3390 0x5611c0255810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 -- 172.21.15.94:0/2389168724 >> 172.21.15.94:0/2389168724 conn(0x5611bfea74e0 msgr2=0x5611c0251020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 -- 172.21.15.94:0/2389168724 shutdown_connections 2024-04-25T14:04:47.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.520+0000 7f5de9008740 1 -- 172.21.15.94:0/2389168724 wait complete. 2024-04-25T14:04:47.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 -- start start 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 Processor -- start 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfeb3390 0x5611c0277a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5611c02574d0 con 0x5611bfe926b0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5611c0140dc0 con 0x5611bfeb3390 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611c0257650 con 0x5611bfe8a4d0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.521+0000 7f5de9008740 10 client.0 _ll_register_callbacks cb 0x5611c015c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37068/0 (socket says 172.21.15.94:37068) 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2072548866 learned_addr learned my addr 172.21.15.94:0/2072548866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfeb3390 0x5611c0277a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2072548866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 msgr2=0x5611c02705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfeb3390 msgr2=0x5611c0277a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfeb3390 0x5611c0277a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 -- 172.21.15.94:0/2072548866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611bfecb3f0 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.522+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00057c0 tx=0x7f5dd0031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a42d9030c610985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035040 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5dd003d680 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd00316d0 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5dd003d880 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5de9008740 10 client.23694 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.523+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5611bff957d0 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00316d0 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5dd004aaf0 con 0x5611bfe926b0 2024-04-25T14:04:47.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 10 client.23694 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 10 client.23694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 20 client.23694 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfe71f60 0x5611c027ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 4 client.23694 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.524+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611c01c7990 con 0x5611bfe71f60 2024-04-25T14:04:47.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.525+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfe71f60 0x5611c027ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.527+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfe71f60 0x5611c027ded0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec5abf42c234a66 server_cookie=7045d4e2b0212c61 in_seq=0 out_seq=0 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.528+0000 7f5dbaffd640 10 client.23694 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.530+0000 7f5dbaffd640 1 -- 172.21.15.94:0/2072548866 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611c01c7990 con 0x5611bfe71f60 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.530+0000 7f5dbaffd640 10 client.23694 handle_command_reply: tid=0 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.530+0000 7f5de9008740 1 client.23694 shutdown 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.531+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 msgr2=0x5611c0270b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.531+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00057c0 tx=0x7f5dd0031ab0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 shutdown_connections 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfe71f60 0x5611c027ded0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfeb3390 0x5611c0277a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfe926b0 0x5611c0270b20 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 --2- 172.21.15.94:0/2072548866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 >> 172.21.15.94:0/2072548866 conn(0x5611bfea74e0 msgr2=0x5611c0253f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5dbaffd640 0 client.23694 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 shutdown_connections 2024-04-25T14:04:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 1 -- 172.21.15.94:0/2072548866 wait complete. 2024-04-25T14:04:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.533+0000 7f5de9008740 20 client.23694 trim_cache size 0 max 16384 2024-04-25T14:04:47.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.554+0000 7f5de9008740 1 Processor -- start 2024-04-25T14:04:47.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.554+0000 7f5de9008740 1 -- start start 2024-04-25T14:04:47.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.554+0000 7f5de9008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5de9008740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5de9008740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c026ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5de9008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc073780 con 0x5611bfea74e0 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5de9008740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1b3c80 con 0x5611bfe926b0 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5de9008740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc073e50 con 0x5611bfe8a4d0 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37076/0 (socket says 172.21.15.94:37076) 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/2941287000 learned_addr learned my addr 172.21.15.94:0/2941287000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c026ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/2941287000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 msgr2=0x5611c02779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/2941287000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 msgr2=0x5611c026ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c026ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/2941287000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c025ad60 con 0x5611bfea74e0 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c026ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x5611c0251db0 tx=0x7f5dcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e66dc434cb15a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5dc17fa640 1 -- 172.21.15.94:0/2941287000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc028410 con 0x5611bfea74e0 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5dc17fa640 1 -- 172.21.15.94:0/2941287000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc028a60 con 0x5611bfea74e0 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.555+0000 7f5dc17fa640 1 -- 172.21.15.94:0/2941287000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc01d9b0 con 0x5611bfea74e0 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- 172.21.15.94:0/2941287000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 msgr2=0x5611c0251160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x5611c0251db0 tx=0x7f5dcc014000 comp rx=0 tx=0).stop 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- 172.21.15.94:0/2941287000 shutdown_connections 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe926b0 0x5611c026ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe8a4d0 0x5611c02779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 --2- 172.21.15.94:0/2941287000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0251160 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- 172.21.15.94:0/2941287000 >> 172.21.15.94:0/2941287000 conn(0x5611c0255fa0 msgr2=0x5611c0250a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- 172.21.15.94:0/2941287000 shutdown_connections 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- 172.21.15.94:0/2941287000 wait complete. 2024-04-25T14:04:47.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.557+0000 7f5de9008740 1 -- start start 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 Processor -- start 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 0x5611c0275120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe926b0 0x5611c0275660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc074bf0 con 0x5611bfea74e0 2024-04-25T14:04:47.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc074520 con 0x5611bfe8a4d0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc071760 con 0x5611bfe926b0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5de9008740 10 client.0 _ll_register_callbacks cb 0x7f5dd40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe926b0 0x5611c0275660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 0x5611c0275120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37078/0 (socket says 172.21.15.94:37078) 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddbfff640 1 -- 172.21.15.94:0/1700489340 learned_addr learned my addr 172.21.15.94:0/1700489340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/1700489340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe926b0 msgr2=0x5611c0275660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe926b0 0x5611c0275660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.558+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/1700489340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 msgr2=0x5611c0276590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5ddb7fe640 1 -- 172.21.15.94:0/1700489340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c0277ee0 con 0x5611bfe8a4d0 2024-04-25T14:04:47.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5ddbfff640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5ddb7fe640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 0x5611c0275120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5611c024c810 tx=0x7f5dcc01d550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddf4f378816fee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc028820 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5dcc005040 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dcc028a60 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5dcc005240 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5de9008740 10 client.33086 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.559+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5611c0277c00 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc028a60 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5dcc039070 con 0x5611bfe8a4d0 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 10 client.33086 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 10 client.33086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 20 client.33086 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfeb3390 0x5611c02e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:47.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 4 client.33086 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:47.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611c01c1800 con 0x5611bfeb3390 2024-04-25T14:04:47.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.560+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfeb3390 0x5611c02e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:47.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.561+0000 7f5dbaffd640 10 client.33086 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:47.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.561+0000 7f5ddaffd640 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfeb3390 0x5611c02e8760 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbee66dad18e9a44 server_cookie=bac3cd362e9f0205 in_seq=0 out_seq=0 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5dbaffd640 1 -- 172.21.15.94:0/1700489340 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611c01c1800 con 0x5611bfeb3390 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5dbaffd640 10 client.33086 handle_command_reply: tid=0 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 client.33086 shutdown 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 msgr2=0x5611c0275120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 0x5611c0275120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5611c024c810 tx=0x7f5dcc01d550 comp rx=0 tx=0).stop 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 shutdown_connections 2024-04-25T14:04:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5611bfeb3390 0x5611c02e8760 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5611bfea74e0 0x5611c0276590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5dbaffd640 0 client.33086 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611bfe926b0 0x5611c0275660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 --2- 172.21.15.94:0/1700489340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611bfe8a4d0 0x5611c0275120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 >> 172.21.15.94:0/1700489340 conn(0x5611c0255fa0 msgr2=0x5611c026ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.569+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 shutdown_connections 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de9008740 1 -- 172.21.15.94:0/1700489340 wait complete. 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de9008740 20 client.33086 trim_cache size 0 max 16384 2024-04-25T14:04:47.571 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:04:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 msgr2=0x7f5dc40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 0x7f5dc40b2110 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x5611c0255530 tx=0x7f5dcc005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 msgr2=0x7f5ddc1a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 0x7f5ddc1a1e70 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009fd0 tx=0x7f5dd400ef10 comp rx=0 tx=0).stop 2024-04-25T14:04:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 shutdown_connections 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5dc40afc00 0x7f5dc40b2110 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ddc19fa50 0x7f5ddc1a1e70 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc108920 0x7f5ddc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 --2- 172.21.15.94:0/3385648044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ddc1029e0 0x7f5ddc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.570+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 >> 172.21.15.94:0/3385648044 conn(0x7f5ddc0fdbd0 msgr2=0x7f5ddc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.571+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 shutdown_connections 2024-04-25T14:04:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:47.571+0000 7f5de2084640 1 -- 172.21.15.94:0/3385648044 wait complete. 2024-04-25T14:04:48.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 201898 18.84 MB/sec execute 450 sec latency 144.873 ms 2024-04-25T14:04:48.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:47 smithi094 ceph-mon[29195]: pgmap v1041: 97 pgs: 97 active+clean; 285 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-04-25T14:04:48.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:47 smithi138 ceph-mon[32977]: pgmap v1041: 97 pgs: 97 active+clean; 285 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-04-25T14:04:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:47 smithi137 ceph-mon[32568]: pgmap v1041: 97 pgs: 97 active+clean; 285 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-04-25T14:04:48.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:04:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:48.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 -- 172.21.15.94:0/222160909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 msgr2=0x7f06c8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 --2- 172.21.15.94:0/222160909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c8104030 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009900 tx=0x7f06b002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 -- 172.21.15.94:0/222160909 shutdown_connections 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 --2- 172.21.15.94:0/222160909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 0x7f06c810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 --2- 172.21.15.94:0/222160909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c8104030 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 --2- 172.21.15.94:0/222160909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 0x7f06c81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 -- 172.21.15.94:0/222160909 >> 172.21.15.94:0/222160909 conn(0x7f06c80fdbd0 msgr2=0x7f06c8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 -- 172.21.15.94:0/222160909 shutdown_connections 2024-04-25T14:04:48.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.710+0000 7f06ccb40640 1 -- 172.21.15.94:0/222160909 wait complete. 2024-04-25T14:04:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.711+0000 7f06ccb40640 1 Processor -- start 2024-04-25T14:04:48.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.711+0000 7f06ccb40640 1 -- start start 2024-04-25T14:04:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.711+0000 7f06ccb40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 0x7f06c806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 0x7f06c81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06c810e500 con 0x7f06c8103c20 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06c810e380 con 0x7f06c8108920 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c810e680 con 0x7f06c81029e0 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37092/0 (socket says 172.21.15.94:37092) 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 -- 172.21.15.94:0/3630223450 learned_addr learned my addr 172.21.15.94:0/3630223450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06c6d9d640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 0x7f06c81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06c659c640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 0x7f06c806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 -- 172.21.15.94:0/3630223450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 msgr2=0x7f06c806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 0x7f06c806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 -- 172.21.15.94:0/3630223450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 msgr2=0x7f06c81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 0x7f06c81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 -- 172.21.15.94:0/3630223450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c81a2080 con 0x7f06c8103c20 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bffff640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006010 tx=0x7f06b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bf7fe640 1 -- 172.21.15.94:0/3630223450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b003f030 con 0x7f06c8103c20 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c81a2310 con 0x7f06c8103c20 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bf7fe640 1 -- 172.21.15.94:0/3630223450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b0031910 con 0x7f06c8103c20 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06bf7fe640 1 -- 172.21.15.94:0/3630223450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0035070 con 0x7f06c8103c20 2024-04-25T14:04:48.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.712+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81a2850 con 0x7f06c8103c20 2024-04-25T14:04:48.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.715+0000 7f06bf7fe640 1 -- 172.21.15.94:0/3630223450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f06b0042030 con 0x7f06c8103c20 2024-04-25T14:04:48.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.715+0000 7f06d39f1740 1 Processor -- start 2024-04-25T14:04:48.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.715+0000 7f06d39f1740 1 -- start start 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d124b410 0x5586d15ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 0x5586d15f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5586d1551b00 con 0x5586d1222550 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586d14ed920 con 0x5586d124b410 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06d39f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586d154dac0 con 0x5586d122a730 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d124b410 0x5586d15ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37102/0 (socket says 172.21.15.94:37102) 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 -- 172.21.15.94:0/2645342950 learned_addr learned my addr 172.21.15.94:0/2645342950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06bffff640 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 0x5586d15f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06bf7fe640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 0x7f06b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 -- 172.21.15.94:0/2645342950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 msgr2=0x5586d15f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 0x5586d15f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 -- 172.21.15.94:0/2645342950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d124b410 msgr2=0x5586d15ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d124b410 0x5586d15ec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 -- 172.21.15.94:0/2645342950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d132b7e0 con 0x5586d1222550 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06bffff640 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 0x5586d15f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c6d9d640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 0x7f06b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06bf7fe640 1 -- 172.21.15.94:0/3630223450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f06b00ff160 con 0x7f06c8103c20 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.716+0000 7f06c659c640 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009c60 tx=0x7f06b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607bb6d5325383c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06bcff9640 1 -- 172.21.15.94:0/2645342950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004220 con 0x5586d1222550 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06bcff9640 1 -- 172.21.15.94:0/2645342950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b80043c0 con 0x5586d1222550 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06bcff9640 1 -- 172.21.15.94:0/2645342950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8019830 con 0x5586d1222550 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06d39f1740 1 -- 172.21.15.94:0/2645342950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 msgr2=0x5586d15eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009c60 tx=0x7f06b8011810 comp rx=0 tx=0).stop 2024-04-25T14:04:48.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.717+0000 7f06c6d9d640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 0x7f06b40b2000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009fd0 tx=0x7f06c0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- 172.21.15.94:0/2645342950 shutdown_connections 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d122a730 0x5586d15f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d1222550 0x5586d15eca20 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2645342950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d124b410 0x5586d15ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- 172.21.15.94:0/2645342950 >> 172.21.15.94:0/2645342950 conn(0x5586d123f560 msgr2=0x5586d15e7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- 172.21.15.94:0/2645342950 shutdown_connections 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- 172.21.15.94:0/2645342950 wait complete. 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- start start 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 Processor -- start 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5586d15f4370 con 0x5586d122a730 2024-04-25T14:04:48.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586d14eb5d0 con 0x5586d1222550 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586d15f44f0 con 0x5586d124b410 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06d39f1740 10 client.0 _ll_register_callbacks cb 0x5586d14d9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.718+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37118/0 (socket says 172.21.15.94:37118) 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 -- 172.21.15.94:0/13458497 learned_addr learned my addr 172.21.15.94:0/13458497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06c659c640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06c6d9d640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 -- 172.21.15.94:0/13458497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 msgr2=0x5586d15fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 -- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 msgr2=0x5586d15f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 -- 172.21.15.94:0/13458497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d138bf70 con 0x5586d122a730 2024-04-25T14:04:48.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06c659c640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06bffff640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f06b00c36d0 tx=0x7f06b0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9416617fc25f1272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06adffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0049070 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f06b00c38e0 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b80043c0 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06b00c6fa0 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06d39f1740 10 client.23712 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.719+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586d1223170 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b80043c0 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f06b00c78b0 con 0x5586d122a730 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 10 client.23712 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 10 client.23712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:48.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 20 client.23712 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d120a1f0 0x5586d1603cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 4 client.23712 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d155f710 con 0x5586d120a1f0 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06c659c640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d120a1f0 0x5586d1603cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06adffb640 10 client.23712 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.720+0000 7f06c659c640 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d120a1f0 0x5586d1603cc0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9724101324dac69 server_cookie=4d618f274a79436 in_seq=0 out_seq=0 2024-04-25T14:04:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06adffb640 1 -- 172.21.15.94:0/13458497 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586d155f710 con 0x5586d120a1f0 2024-04-25T14:04:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06adffb640 10 client.23712 handle_command_reply: tid=0 2024-04-25T14:04:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 client.23712 shutdown 2024-04-25T14:04:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 msgr2=0x5586d15f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f06b00c36d0 tx=0x7f06b0009620 comp rx=0 tx=0).stop 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 shutdown_connections 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d120a1f0 0x5586d1603cc0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f6780 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 --2- 172.21.15.94:0/13458497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 >> 172.21.15.94:0/13458497 conn(0x5586d123f560 msgr2=0x5586d15e6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06adffb640 0 client.23712 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 shutdown_connections 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 1 -- 172.21.15.94:0/13458497 wait complete. 2024-04-25T14:04:48.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.723+0000 7f06d39f1740 20 client.23712 trim_cache size 0 max 16384 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 Processor -- start 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 -- start start 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ab4f0 con 0x5586d122a730 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ad030 con 0x5586d1222550 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06d39f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81abbc0 con 0x5586d124b410 2024-04-25T14:04:48.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06c6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37134/0 (socket says 172.21.15.94:37134) 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 -- 172.21.15.94:0/2446148592 learned_addr learned my addr 172.21.15.94:0/2446148592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06c659c640 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 -- 172.21.15.94:0/2446148592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 msgr2=0x5586d15e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 -- 172.21.15.94:0/2446148592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 msgr2=0x5586d15f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.743+0000 7f06bffff640 1 -- 172.21.15.94:0/2446148592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d15f6920 con 0x5586d122a730 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.744+0000 7f06bffff640 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f06b00c48c0 tx=0x7f06b0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c07fd3690e4706f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.744+0000 7f06affff640 1 -- 172.21.15.94:0/2446148592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00e6070 con 0x5586d122a730 2024-04-25T14:04:48.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.744+0000 7f06affff640 1 -- 172.21.15.94:0/2446148592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00cb950 con 0x5586d122a730 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.744+0000 7f06affff640 1 -- 172.21.15.94:0/2446148592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0049070 con 0x5586d122a730 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- 172.21.15.94:0/2446148592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 msgr2=0x5586d15f0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f06b00c48c0 tx=0x7f06b0005cb0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- 172.21.15.94:0/2446148592 shutdown_connections 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d15e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15f0bd0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- 172.21.15.94:0/2446148592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- 172.21.15.94:0/2446148592 >> 172.21.15.94:0/2446148592 conn(0x5586d123f560 msgr2=0x5586d15f48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- 172.21.15.94:0/2446148592 shutdown_connections 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- 172.21.15.94:0/2446148592 wait complete. 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- start start 2024-04-25T14:04:48.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 Processor -- start 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d166a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ac960 con 0x5586d122a730 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81ac290 con 0x5586d1222550 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81a94d0 con 0x5586d124b410 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06d39f1740 10 client.0 _ll_register_callbacks cb 0x7f06b003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37150/0 (socket says 172.21.15.94:37150) 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.745+0000 7f06bffff640 1 -- 172.21.15.94:0/1515800532 learned_addr learned my addr 172.21.15.94:0/1515800532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06c659c640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06c6d9d640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d166a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 -- 172.21.15.94:0/1515800532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 msgr2=0x5586d166a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d166a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 -- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 msgr2=0x5586d15f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 -- 172.21.15.94:0/1515800532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d166a5a0 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06bffff640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ff130 tx=0x7f06b00d01a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c638e24e6bc6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06adffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a7060 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f06b003bd50 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b00cb950 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06b00e6070 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06d39f1740 10 client.23718 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586d166a5a0 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.746+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00cb950 con 0x5586d122a730 2024-04-25T14:04:48.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f06b00aeaf0 con 0x5586d122a730 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 10 client.23718 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 10 client.23718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 20 client.23718 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d11df630 0x5586d166dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 4 client.23718 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586d1559580 con 0x5586d11df630 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06c659c640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d11df630 0x5586d166dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06c659c640 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d11df630 0x5586d166dce0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8609b863b8592b2 server_cookie=ab9aed4b6238f78a in_seq=0 out_seq=0 2024-04-25T14:04:48.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.747+0000 7f06adffb640 10 client.23718 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.755+0000 7f06adffb640 1 -- 172.21.15.94:0/1515800532 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586d1559580 con 0x5586d11df630 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.755+0000 7f06adffb640 10 client.23718 handle_command_reply: tid=0 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.755+0000 7f06d39f1740 1 client.23718 shutdown 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 msgr2=0x5586d15fa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f06b00ff130 tx=0x7f06b00d01a0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 shutdown_connections 2024-04-25T14:04:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5586d11df630 0x5586d166dce0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586d124b410 0x5586d166a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06adffb640 0 client.23718 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586d122a730 0x5586d15fa120 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 --2- 172.21.15.94:0/1515800532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d1222550 0x5586d15f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 >> 172.21.15.94:0/1515800532 conn(0x5586d123f560 msgr2=0x5586d15f0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 shutdown_connections 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 1 -- 172.21.15.94:0/1515800532 wait complete. 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06d39f1740 20 client.23718 trim_cache size 0 max 16384 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:48.757 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:04:48.758 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "8c290d76-d6c4-4b96-bce8-eafbe174a606", 2024-04-25T14:04:48.758 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:04:48.758 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:48.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 msgr2=0x7f06b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 0x7f06b40b2000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009fd0 tx=0x7f06c0004a40 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 msgr2=0x7f06c819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006010 tx=0x7f06b00042f0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 shutdown_connections 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f06b40afaf0 0x7f06b40b2000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06c8108920 0x7f06c81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06c8103c20 0x7f06c819f2f0 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 --2- 172.21.15.94:0/3630223450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c81029e0 0x7f06c806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.756+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 >> 172.21.15.94:0/3630223450 conn(0x7f06c80fdbd0 msgr2=0x7f06c8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.757+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 shutdown_connections 2024-04-25T14:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.757+0000 7f06ccb40640 1 -- 172.21.15.94:0/3630223450 wait complete. 2024-04-25T14:04:48.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:48.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 -- 172.21.15.94:0/3388179612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088103af0 msgr2=0x7fd0881047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/3388179612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088103af0 0x7fd0881047f0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fd084009900 tx=0x7fd08402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 -- 172.21.15.94:0/3388179612 shutdown_connections 2024-04-25T14:04:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/3388179612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088109a30 0x7fd08810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/3388179612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 0x7fd088105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/3388179612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088103af0 0x7fd0881047f0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 -- 172.21.15.94:0/3388179612 >> 172.21.15.94:0/3388179612 conn(0x7fd0880fece0 msgr2=0x7fd088101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 -- 172.21.15.94:0/3388179612 shutdown_connections 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.888+0000 7fd08fcaa640 1 -- 172.21.15.94:0/3388179612 wait complete. 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 Processor -- start 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 -- start start 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088103af0 0x7fd08819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 0x7fd08819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810f5d0 con 0x7fd088109a30 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810f450 con 0x7fd088103af0 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08fcaa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810f750 con 0x7fd088104d30 2024-04-25T14:04:48.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08e247640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08e247640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37174/0 (socket says 172.21.15.94:37174) 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08e247640 1 -- 172.21.15.94:0/2770721025 learned_addr learned my addr 172.21.15.94:0/2770721025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08da46640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088103af0 0x7fd08819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.889+0000 7fd08d245640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 0x7fd08819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 -- 172.21.15.94:0/2770721025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 msgr2=0x7fd08819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 0x7fd08819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 -- 172.21.15.94:0/2770721025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088103af0 msgr2=0x7fd08819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088103af0 0x7fd08819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 -- 172.21.15.94:0/2770721025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0881a1d70 con 0x7fd088109a30 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08e247640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fd08000eb10 tx=0x7fd08000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd07effd640 1 -- 172.21.15.94:0/2770721025 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080004b20 con 0x7fd088109a30 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0881a2060 con 0x7fd088109a30 2024-04-25T14:04:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd07effd640 1 -- 172.21.15.94:0/2770721025 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd080004cc0 con 0x7fd088109a30 2024-04-25T14:04:48.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd07effd640 1 -- 172.21.15.94:0/2770721025 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080010670 con 0x7fd088109a30 2024-04-25T14:04:48.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.890+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0881a2560 con 0x7fd088109a30 2024-04-25T14:04:48.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.892+0000 7fd096c34740 1 Processor -- start 2024-04-25T14:04:48.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd07effd640 1 -- 172.21.15.94:0/2770721025 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd080005020 con 0x7fd088109a30 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 -- start start 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 0x55d712e79fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712ab76b0 0x55d712e7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d712dded80 con 0x55d712ad8390 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d712d988b0 con 0x55d712aaf4d0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd096c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d712ddad40 con 0x55d712ab76b0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37190/0 (socket says 172.21.15.94:37190) 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd08d245640 1 -- 172.21.15.94:0/3380634579 learned_addr learned my addr 172.21.15.94:0/3380634579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd08da46640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 0x55d712e79fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd07effd640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 0x7fd0740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.893+0000 7fd08d245640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712ab76b0 0x55d712e7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08e247640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 0x7fd0740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 -- 172.21.15.94:0/3380634579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712ab76b0 msgr2=0x55d712e7ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712ab76b0 0x55d712e7ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 -- 172.21.15.94:0/3380634579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 msgr2=0x55d712e79a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 -- 172.21.15.94:0/3380634579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d712d69a40 con 0x55d712aaf4d0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08d245640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd07effd640 1 -- 172.21.15.94:0/2770721025 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd0800df280 con 0x7fd088109a30 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08da46640 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 0x55d712e79fe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd08402a930 tx=0x7fd084000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2480633c437783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd073fff640 1 -- 172.21.15.94:0/3380634579 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084035030 con 0x55d712aaf4d0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd08e247640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 0x7fd0740b2000 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a73c0 tx=0x7fd08006e080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd073fff640 1 -- 172.21.15.94:0/3380634579 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd084040070 con 0x55d712aaf4d0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.894+0000 7fd073fff640 1 -- 172.21.15.94:0/3380634579 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084003850 con 0x55d712aaf4d0 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- 172.21.15.94:0/3380634579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 msgr2=0x55d712e79fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 0x55d712e79fe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd08402a930 tx=0x7fd084000bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- 172.21.15.94:0/3380634579 shutdown_connections 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712ab76b0 0x55d712e7ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712aaf4d0 0x55d712e79fe0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 --2- 172.21.15.94:0/3380634579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- 172.21.15.94:0/3380634579 >> 172.21.15.94:0/3380634579 conn(0x55d712acc4e0 msgr2=0x55d712e75270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- 172.21.15.94:0/3380634579 shutdown_connections 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- 172.21.15.94:0/3380634579 wait complete. 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.895+0000 7fd096c34740 1 -- start start 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 Processor -- start 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e83960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d712e7b720 con 0x55d712ad8390 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d712d65d10 con 0x55d712ab76b0 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d712e7b8a0 con 0x55d712aaf4d0 2024-04-25T14:04:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd096c34740 10 client.0 _ll_register_callbacks cb 0x55d712d92210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37206/0 (socket says 172.21.15.94:37206) 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 -- 172.21.15.94:0/3673251464 learned_addr learned my addr 172.21.15.94:0/3673251464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08da46640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e83960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 -- 172.21.15.94:0/3673251464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 msgr2=0x55d712e83960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e83960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 -- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 msgr2=0x55d712e83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 -- 172.21.15.94:0/3673251464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d712bba7d0 con 0x55d712ad8390 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd08d245640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fd078013f60 tx=0x7fd078005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62530a9f6a721c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.896+0000 7fd0717fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078005220 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd0780053c0 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd084040070 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd0780055c0 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 10 client.23730 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d712af03f0 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd084040070 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd07800f510 con 0x55d712ad8390 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 10 client.23730 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 10 client.23730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 20 client.23730 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a96f60 0x55d712e91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 4 client.23730 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.897+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d712dec990 con 0x55d712a96f60 2024-04-25T14:04:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.898+0000 7fd08da46640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a96f60 0x55d712e91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.898+0000 7fd0717fa640 10 client.23730 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.898+0000 7fd08da46640 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a96f60 0x55d712e91220 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e213f945125816fb server_cookie=43438397fddc1d88 in_seq=0 out_seq=0 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd0717fa640 1 -- 172.21.15.94:0/3673251464 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d712dec990 con 0x55d712a96f60 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd0717fa640 10 client.23730 handle_command_reply: tid=0 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 client.23730 shutdown 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 msgr2=0x55d712e8ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fd078013f60 tx=0x7fd078005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 shutdown_connections 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a96f60 0x55d712e91220 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e8ae00 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 --2- 172.21.15.94:0/3673251464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e83960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd0717fa640 0 client.23730 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 >> 172.21.15.94:0/3673251464 conn(0x55d712acc4e0 msgr2=0x55d712e78180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 shutdown_connections 2024-04-25T14:04:48.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 1 -- 172.21.15.94:0/3673251464 wait complete. 2024-04-25T14:04:48.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.901+0000 7fd096c34740 20 client.23730 trim_cache size 0 max 16384 2024-04-25T14:04:48.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.924+0000 7fd096c34740 1 Processor -- start 2024-04-25T14:04:48.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.924+0000 7fd096c34740 1 -- start start 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881ab4c0 con 0x55d712ad8390 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881ad000 con 0x55d712ab76b0 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd096c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881abb90 con 0x55d712aaf4d0 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37214/0 (socket says 172.21.15.94:37214) 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 -- 172.21.15.94:0/3489698868 learned_addr learned my addr 172.21.15.94:0/3489698868 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08da46640 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 -- 172.21.15.94:0/3489698868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 msgr2=0x55d712e700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 -- 172.21.15.94:0/3489698868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 msgr2=0x55d712e81f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e81f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 -- 172.21.15.94:0/3489698868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d712e84040 con 0x55d712ad8390 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd08d245640 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fd078015040 tx=0x7fd078010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b4596e0de7de09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.925+0000 7fd0737fe640 1 -- 172.21.15.94:0/3489698868 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07800fd20 con 0x55d712ad8390 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd0737fe640 1 -- 172.21.15.94:0/3489698868 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd078013170 con 0x55d712ad8390 2024-04-25T14:04:48.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd0737fe640 1 -- 172.21.15.94:0/3489698868 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078025c40 con 0x55d712ad8390 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 -- 172.21.15.94:0/3489698868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 msgr2=0x55d712e82440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fd078015040 tx=0x7fd078010f90 comp rx=0 tx=0).stop 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 -- 172.21.15.94:0/3489698868 shutdown_connections 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ad8390 0x55d712e82440 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ab76b0 0x55d712e81f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 --2- 172.21.15.94:0/3489698868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 -- 172.21.15.94:0/3489698868 >> 172.21.15.94:0/3489698868 conn(0x55d712acc4e0 msgr2=0x55d712e799e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 -- 172.21.15.94:0/3489698868 shutdown_connections 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.926+0000 7fd096c34740 1 -- 172.21.15.94:0/3489698868 wait complete. 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 -- start start 2024-04-25T14:04:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 Processor -- start 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e87250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ad8390 0x55d712ef7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881ac930 con 0x55d712ab76b0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881ac260 con 0x55d712ad8390 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0881a94a0 con 0x55d712aaf4d0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.927+0000 7fd096c34740 10 client.0 _ll_register_callbacks cb 0x7fd078013b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08d245640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e87250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37216/0 (socket says 172.21.15.94:37216) 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 -- 172.21.15.94:0/969629383 learned_addr learned my addr 172.21.15.94:0/969629383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08d245640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ad8390 0x55d712ef7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 -- 172.21.15.94:0/969629383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 msgr2=0x55d712e87250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e87250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 -- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ad8390 msgr2=0x55d712ef7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ad8390 0x55d712ef7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 -- 172.21.15.94:0/969629383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d712ab00f0 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd08da46640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fd084031440 tx=0x7fd08402a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98495cdf0fbd5d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd0717fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084055070 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd078013170 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd084004020 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd084035040 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd096c34740 10 client.23736 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.928+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d712d9f4b0 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd078013170 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd084058030 con 0x55d712ab76b0 2024-04-25T14:04:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 10 client.23736 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 10 client.23736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 20 client.23736 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a6c3a0 0x55d712efad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 4 client.23736 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d712de6800 con 0x55d712a6c3a0 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd08d245640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a6c3a0 0x55d712efad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.929+0000 7fd0717fa640 10 client.23736 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.930+0000 7fd08d245640 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a6c3a0 0x55d712efad40 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f84f3e2c53d6341 server_cookie=268f2d9769b85f25 in_seq=0 out_seq=0 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd0717fa640 1 -- 172.21.15.94:0/969629383 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d712de6800 con 0x55d712a6c3a0 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd0717fa640 10 client.23736 handle_command_reply: tid=0 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 client.23736 shutdown 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 msgr2=0x55d712e87790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fd084031440 tx=0x7fd08402a9b0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 shutdown_connections 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d712a6c3a0 0x55d712efad40 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d712ad8390 0x55d712ef7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d712ab76b0 0x55d712e87790 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 --2- 172.21.15.94:0/969629383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d712aaf4d0 0x55d712e87250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd0717fa640 0 client.23736 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.934+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 >> 172.21.15.94:0/969629383 conn(0x55d712acc4e0 msgr2=0x55d712e81f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 shutdown_connections 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd096c34740 1 -- 172.21.15.94:0/969629383 wait complete. 2024-04-25T14:04:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd096c34740 20 client.23736 trim_cache size 0 max 16384 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (545 inodes in the stack)", 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "8c290d76-d6c4-4b96-bce8-eafbe174a606": { 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "8c290d76-d6c4-4b96-bce8-eafbe174a606", 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:04:48.937 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:04:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 msgr2=0x7fd0740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 0x7fd0740b2000 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd0800a73c0 tx=0x7fd08006e080 comp rx=0 tx=0).stop 2024-04-25T14:04:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 msgr2=0x7fd0881a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fd08000eb10 tx=0x7fd08000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 shutdown_connections 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd0740afaf0 0x7fd0740b2000 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd088109a30 0x7fd0881a1700 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd088104d30 0x7fd08819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 --2- 172.21.15.94:0/2770721025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd088103af0 0x7fd08819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.935+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 >> 172.21.15.94:0/2770721025 conn(0x7fd0880fece0 msgr2=0x7fd088105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.936+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 shutdown_connections 2024-04-25T14:04:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:04:48.936+0000 7fd08fcaa640 1 -- 172.21.15.94:0/2770721025 wait complete. 2024-04-25T14:04:48.949 INFO:tasks.cephfs.filesystem:scrub status for tag:8c290d76-d6c4-4b96-bce8-eafbe174a606 - {'path': '/', 'tag': '8c290d76-d6c4-4b96-bce8-eafbe174a606', 'options': 'recursive,force'} 2024-04-25T14:04:49.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 202235 18.84 MB/sec execute 451 sec latency 218.169 ms 2024-04-25T14:04:49.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:48 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:04:49.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:48 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:49.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:48 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:49.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:48 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:04:49.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:48 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:49.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:48 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:49.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:48 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:04:49.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:48 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:49.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:48 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.158 INFO:tasks.workunit.client.0.smithi094.stdout: 10 202643 18.85 MB/sec execute 452 sec latency 99.568 ms 2024-04-25T14:04:50.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: pgmap v1042: 97 pgs: 97 active+clean; 286 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 424 op/s 2024-04-25T14:04:50.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:04:50.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:49 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: pgmap v1042: 97 pgs: 97 active+clean; 286 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 424 op/s 2024-04-25T14:04:50.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.248 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.249 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:49 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: pgmap v1042: 97 pgs: 97 active+clean; 286 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.0 MiB/s wr, 424 op/s 2024-04-25T14:04:50.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:50.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:49 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:51.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 203018 18.85 MB/sec execute 453 sec latency 88.701 ms 2024-04-25T14:04:52.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 203366 18.86 MB/sec execute 454 sec latency 88.139 ms 2024-04-25T14:04:52.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:51 smithi094 ceph-mon[29195]: pgmap v1043: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-04-25T14:04:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:51 smithi138 ceph-mon[32977]: pgmap v1043: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-04-25T14:04:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:51 smithi137 ceph-mon[32568]: pgmap v1043: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-04-25T14:04:53.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 203759 18.88 MB/sec execute 455 sec latency 74.479 ms 2024-04-25T14:04:54.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 203992 18.87 MB/sec execute 456 sec latency 322.463 ms 2024-04-25T14:04:54.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:53 smithi094 ceph-mon[29195]: pgmap v1044: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-04-25T14:04:54.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:53 smithi138 ceph-mon[32977]: pgmap v1044: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-04-25T14:04:54.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:53 smithi137 ceph-mon[32568]: pgmap v1044: 97 pgs: 97 active+clean; 280 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 542 op/s 2024-04-25T14:04:55.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 204321 18.84 MB/sec execute 457 sec latency 146.572 ms 2024-04-25T14:04:56.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 204688 18.84 MB/sec execute 458 sec latency 120.358 ms 2024-04-25T14:04:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:55 smithi094 ceph-mon[29195]: pgmap v1045: 97 pgs: 97 active+clean; 290 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-04-25T14:04:56.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:55 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:55 smithi138 ceph-mon[32977]: pgmap v1045: 97 pgs: 97 active+clean; 290 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-04-25T14:04:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:55 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:55 smithi137 ceph-mon[32568]: pgmap v1045: 97 pgs: 97 active+clean; 290 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.7 MiB/s wr, 626 op/s 2024-04-25T14:04:56.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:55 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:57.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 205087 18.85 MB/sec execute 459 sec latency 83.474 ms 2024-04-25T14:04:57.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:57.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:57.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:57.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:56 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:04:57.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:57.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:57.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:57.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:56 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:04:57.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:57.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:57.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:57.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:56 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:04:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 -- 172.21.15.94:0/238356209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8108780 msgr2=0x7f0ee810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 --2- 172.21.15.94:0/238356209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8108780 0x7f0ee810aba0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009ca0 tx=0x7f0ee002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 -- 172.21.15.94:0/238356209 shutdown_connections 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 --2- 172.21.15.94:0/238356209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8108780 0x7f0ee810aba0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 --2- 172.21.15.94:0/238356209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8103a80 0x7f0ee8103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 --2- 172.21.15.94:0/238356209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8102840 0x7f0ee8103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 -- 172.21.15.94:0/238356209 >> 172.21.15.94:0/238356209 conn(0x7f0ee80fda30 msgr2=0x7f0ee80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 -- 172.21.15.94:0/238356209 shutdown_connections 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.849+0000 7f0eed39d640 1 -- 172.21.15.94:0/238356209 wait complete. 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 Processor -- start 2024-04-25T14:04:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 -- start start 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8108780 0x7f0ee80fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.850+0000 7f0eed39d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810e370 con 0x7f0ee8102840 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0eed39d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810e1f0 con 0x7f0ee8108780 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0eed39d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810e4f0 con 0x7f0ee8103a80 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38066/0 (socket says 172.21.15.94:38066) 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0eecb9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8108780 0x7f0ee80fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee7fff640 1 -- 172.21.15.94:0/2787828439 learned_addr learned my addr 172.21.15.94:0/2787828439 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59048/0 (socket says 172.21.15.94:59048) 2024-04-25T14:04:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 -- 172.21.15.94:0/2787828439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8108780 msgr2=0x7f0ee80fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8108780 0x7f0ee80fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 -- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 msgr2=0x7f0ee80ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 -- 172.21.15.94:0/2787828439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee80febd0 con 0x7f0ee8103a80 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9f0 tx=0x7f0edc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee57fa640 1 -- 172.21.15.94:0/2787828439 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00cd80 con 0x7f0ee8103a80 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee57fa640 1 -- 172.21.15.94:0/2787828439 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc0035f0 con 0x7f0ee8103a80 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee81a4120 con 0x7f0ee8103a80 2024-04-25T14:04:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.851+0000 7f0ee57fa640 1 -- 172.21.15.94:0/2787828439 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc010670 con 0x7f0ee8103a80 2024-04-25T14:04:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.852+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee81a45e0 con 0x7f0ee8103a80 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.854+0000 7f0ee57fa640 1 -- 172.21.15.94:0/2787828439 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0edc0037a0 con 0x7f0ee8103a80 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.854+0000 7f0ef5584740 1 Processor -- start 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.854+0000 7f0ef5584740 1 -- start start 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2277f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d22785f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d226e4460 con 0x563d223bd4d0 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d2269e590 con 0x563d223de1b0 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ef5584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d226e0420 con 0x563d223b52f0 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53218/0 (socket says 172.21.15.94:53218) 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 -- 172.21.15.94:0/706512331 learned_addr learned my addr 172.21.15.94:0/706512331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2277f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d22785f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ee57fa640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 0x7f0ed00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 -- 172.21.15.94:0/706512331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 msgr2=0x563d2277f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2277f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 -- 172.21.15.94:0/706512331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 msgr2=0x563d22785f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d22785f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 -- 172.21.15.94:0/706512331 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d226ab0d0 con 0x563d223de1b0 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d22785f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 0x7f0ed00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0ee57fa640 1 -- 172.21.15.94:0/2787828439 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0edc0deef0 con 0x7f0ee8103a80 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.855+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0004af0 tx=0x7f0ee0009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9241b76de7deb15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ece7fc640 1 -- 172.21.15.94:0/706512331 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0042040 con 0x563d223de1b0 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 0x7f0ed00b1ff0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0ee004d000 tx=0x7f0ee004e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ece7fc640 1 -- 172.21.15.94:0/706512331 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0036070 con 0x563d223de1b0 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ece7fc640 1 -- 172.21.15.94:0/706512331 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee002fb20 con 0x563d223de1b0 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 -- 172.21.15.94:0/706512331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 msgr2=0x563d2277eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0004af0 tx=0x7f0ee0009800 comp rx=0 tx=0).stop 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 -- 172.21.15.94:0/706512331 shutdown_connections 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d22785f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2277f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 --2- 172.21.15.94:0/706512331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223de1b0 0x563d2277eb10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.856+0000 7f0ef5584740 1 -- 172.21.15.94:0/706512331 >> 172.21.15.94:0/706512331 conn(0x563d223d2300 msgr2=0x563d2277a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- 172.21.15.94:0/706512331 shutdown_connections 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- 172.21.15.94:0/706512331 wait complete. 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- start start 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 Processor -- start 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 0x563d22788db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 0x563d2278fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d227869e0 con 0x563d223b52f0 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d2266bb60 con 0x563d223bd4d0 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d22786b60 con 0x563d223de1b0 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ef5584740 10 client.0 _ll_register_callbacks cb 0x563d226902b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38092/0 (socket says 172.21.15.94:38092) 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ee7fff640 1 -- 172.21.15.94:0/4126485293 learned_addr learned my addr 172.21.15.94:0/4126485293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 0x563d22788db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.857+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 0x563d2278fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 -- 172.21.15.94:0/4126485293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 msgr2=0x563d2278fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 0x563d2278fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 -- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 msgr2=0x563d22788db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 0x563d22788db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 -- 172.21.15.94:0/4126485293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d223f6210 con 0x563d223b52f0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 0x563d2278fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 0x563d22788db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8012a10 tx=0x7f0ed8012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaffb91ec9cdc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800fa40 con 0x563d223b52f0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee0036070 con 0x563d223b52f0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed800fbe0 con 0x563d223b52f0 2024-04-25T14:04:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ed800fde0 con 0x563d223b52f0 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ef5584740 10 client.23748 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d224c2620 con 0x563d223b52f0 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.858+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0036070 con 0x563d223b52f0 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed8026450 con 0x563d223b52f0 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 10 client.23748 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 10 client.23748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 20 client.23748 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d2239cd80 0x563d22796590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 4 client.23748 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d226f2070 con 0x563d2239cd80 2024-04-25T14:04:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d2239cd80 0x563d22796590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d2239cd80 0x563d22796590 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e3af9aeeafd777 server_cookie=c7265335adabcabb in_seq=0 out_seq=0 2024-04-25T14:04:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.859+0000 7f0ec77fe640 10 client.23748 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ec77fe640 1 -- 172.21.15.94:0/4126485293 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d226f2070 con 0x563d2239cd80 2024-04-25T14:04:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ec77fe640 10 client.23748 handle_command_reply: tid=0 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 client.23748 shutdown 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 msgr2=0x563d22788870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8012a10 tx=0x7f0ed8012ed0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 shutdown_connections 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d2239cd80 0x563d22796590 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223de1b0 0x563d2278fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223bd4d0 0x563d22788db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ec77fe640 0 client.23748 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 --2- 172.21.15.94:0/4126485293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223b52f0 0x563d22788870 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 >> 172.21.15.94:0/4126485293 conn(0x563d223d2300 msgr2=0x563d22779500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 shutdown_connections 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.863+0000 7f0ef5584740 1 -- 172.21.15.94:0/4126485293 wait complete. 2024-04-25T14:04:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.864+0000 7f0ef5584740 20 client.23748 trim_cache size 0 max 16384 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.886+0000 7f0ef5584740 1 Processor -- start 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.886+0000 7f0ef5584740 1 -- start start 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 0x563d2278fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223bd4d0 0x563d22788980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ad4f0 con 0x563d2239cd80 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81af030 con 0x563d223b52f0 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ef5584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81adbc0 con 0x563d223bd4d0 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38108/0 (socket says 172.21.15.94:38108) 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0eecb9c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223bd4d0 0x563d22788980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 -- 172.21.15.94:0/2278681667 learned_addr learned my addr 172.21.15.94:0/2278681667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 0x563d2278fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 -- 172.21.15.94:0/2278681667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223bd4d0 msgr2=0x563d22788980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223bd4d0 0x563d22788980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 -- 172.21.15.94:0/2278681667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 msgr2=0x563d2278fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 0x563d2278fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee7fff640 1 -- 172.21.15.94:0/2278681667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d22790170 con 0x563d2239cd80 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.887+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 0x563d2278fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.888+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8026820 tx=0x7f0ed80124e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3975f4ab784d3a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.888+0000 7f0ecdffb640 1 -- 172.21.15.94:0/2278681667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8026450 con 0x563d2239cd80 2024-04-25T14:04:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.888+0000 7f0ecdffb640 1 -- 172.21.15.94:0/2278681667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed80265f0 con 0x563d2239cd80 2024-04-25T14:04:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.888+0000 7f0ecdffb640 1 -- 172.21.15.94:0/2278681667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8005310 con 0x563d2239cd80 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- 172.21.15.94:0/2278681667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 msgr2=0x563d22788440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8026820 tx=0x7f0ed80124e0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- 172.21.15.94:0/2278681667 shutdown_connections 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223bd4d0 0x563d22788980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d223b52f0 0x563d2278fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 --2- 172.21.15.94:0/2278681667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d2239cd80 0x563d22788440 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- 172.21.15.94:0/2278681667 >> 172.21.15.94:0/2278681667 conn(0x563d223d2300 msgr2=0x563d22782cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- 172.21.15.94:0/2278681667 shutdown_connections 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- 172.21.15.94:0/2278681667 wait complete. 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.889+0000 7f0ef5584740 1 -- start start 2024-04-25T14:04:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 Processor -- start 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2278d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d227fc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ae960 con 0x563d223bd4d0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ae290 con 0x563d2239cd80 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ab4d0 con 0x563d223b52f0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ef5584740 10 client.0 _ll_register_callbacks cb 0x7f0ed8012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0eecb9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d227fc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2278d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53238/0 (socket says 172.21.15.94:53238) 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1325850378 learned_addr learned my addr 172.21.15.94:0/1325850378 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:04:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1325850378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 msgr2=0x563d2278d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2278d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1325850378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 msgr2=0x563d227fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d227fc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee7fff640 1 -- 172.21.15.94:0/1325850378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d227fcec0 con 0x563d2239cd80 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2278d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.890+0000 7f0eecb9c640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d227fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800f890 tx=0x7f0ed8005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d21ed0688199962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8036060 con 0x563d2239cd80 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed80265f0 con 0x563d2239cd80 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ed8018040 con 0x563d2239cd80 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ed8026450 con 0x563d2239cd80 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ef5584740 10 client.33140 fetch_fsmap learned FSMap version 39 2024-04-25T14:04:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d227fdfb0 con 0x563d2239cd80 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.891+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed80265f0 con 0x563d2239cd80 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0ed80036c0 con 0x563d2239cd80 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 10 client.33140 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 10 client.33140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 20 client.33140 populate_metadata read hostname 'smithi094' 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d223de1b0 0x563d228008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 4 client.33140 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d226ebee0 con 0x563d223de1b0 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.892+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d223de1b0 0x563d228008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.893+0000 7f0ec77fe640 10 client.33140 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:57.893+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d223de1b0 0x563d228008e0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a593ed248f44a37 server_cookie=e9414b94d69097d2 in_seq=0 out_seq=0 2024-04-25T14:04:58.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 205478 18.85 MB/sec execute 460 sec latency 231.669 ms 2024-04-25T14:04:58.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:57 smithi094 ceph-mon[29195]: pgmap v1046: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-04-25T14:04:58.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:57 smithi138 ceph-mon[32977]: pgmap v1046: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-04-25T14:04:58.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:57 smithi137 ceph-mon[32568]: pgmap v1046: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.3 MiB/s wr, 575 op/s 2024-04-25T14:04:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.714+0000 7f0ec77fe640 1 -- 172.21.15.94:0/1325850378 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d226ebee0 con 0x563d223de1b0 2024-04-25T14:04:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.714+0000 7f0ec77fe640 10 client.33140 handle_command_reply: tid=0 2024-04-25T14:04:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 client.33140 shutdown 2024-04-25T14:04:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 msgr2=0x563d2278cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800f890 tx=0x7f0ed8005b20 comp rx=0 tx=0).stop 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 shutdown_connections 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d223de1b0 0x563d228008e0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d223bd4d0 0x563d227fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d223b52f0 0x563d2278d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 --2- 172.21.15.94:0/1325850378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d2239cd80 0x563d2278cad0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ec77fe640 0 client.33140 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 >> 172.21.15.94:0/1325850378 conn(0x563d223d2300 msgr2=0x563d2277d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 shutdown_connections 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 1 -- 172.21.15.94:0/1325850378 wait complete. 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.715+0000 7f0ef5584740 20 client.33140 trim_cache size 0 max 16384 2024-04-25T14:04:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 msgr2=0x7f0ed00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 0x7f0ed00b1ff0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0ee004d000 tx=0x7f0ee004e000 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 msgr2=0x7f0ee80fdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9f0 tx=0x7f0edc00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 shutdown_connections 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ed00afae0 0x7f0ed00b1ff0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ee8108780 0x7f0ee80fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee8103a80 0x7f0ee80fdf60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 --2- 172.21.15.94:0/2787828439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee8102840 0x7f0ee80ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 >> 172.21.15.94:0/2787828439 conn(0x7f0ee80fda30 msgr2=0x7f0ee8109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 shutdown_connections 2024-04-25T14:04:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:04:58.716+0000 7f0eed39d640 1 -- 172.21.15.94:0/2787828439 wait complete. 2024-04-25T14:04:59.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 205638 18.84 MB/sec execute 461 sec latency 602.838 ms 2024-04-25T14:05:00.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 206019 18.85 MB/sec execute 462 sec latency 73.891 ms 2024-04-25T14:05:00.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:04:59 smithi094 ceph-mon[29195]: pgmap v1047: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-04-25T14:05:00.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:04:59 smithi138 ceph-mon[32977]: pgmap v1047: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-04-25T14:05:00.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:04:59 smithi137 ceph-mon[32568]: pgmap v1047: 97 pgs: 97 active+clean; 291 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-04-25T14:05:01.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 206472 18.85 MB/sec execute 463 sec latency 117.596 ms 2024-04-25T14:05:02.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 206764 18.84 MB/sec execute 464 sec latency 124.575 ms 2024-04-25T14:05:02.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:01 smithi094 ceph-mon[29195]: pgmap v1048: 97 pgs: 97 active+clean; 279 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 650 op/s 2024-04-25T14:05:02.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:01 smithi138 ceph-mon[32977]: pgmap v1048: 97 pgs: 97 active+clean; 279 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 650 op/s 2024-04-25T14:05:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:01 smithi137 ceph-mon[32568]: pgmap v1048: 97 pgs: 97 active+clean; 279 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 650 op/s 2024-04-25T14:05:03.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 207104 18.85 MB/sec execute 465 sec latency 89.454 ms 2024-04-25T14:05:04.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 207408 18.86 MB/sec execute 466 sec latency 236.967 ms 2024-04-25T14:05:04.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:03 smithi094 ceph-mon[29195]: pgmap v1049: 97 pgs: 97 active+clean; 275 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-04-25T14:05:04.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:03 smithi138 ceph-mon[32977]: pgmap v1049: 97 pgs: 97 active+clean; 275 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-04-25T14:05:04.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:03 smithi137 ceph-mon[32568]: pgmap v1049: 97 pgs: 97 active+clean; 275 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 520 op/s 2024-04-25T14:05:05.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 207729 18.84 MB/sec execute 467 sec latency 91.135 ms 2024-04-25T14:05:06.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 208055 18.83 MB/sec execute 468 sec latency 120.245 ms 2024-04-25T14:05:06.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:05 smithi094 ceph-mon[29195]: pgmap v1050: 97 pgs: 97 active+clean; 277 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-04-25T14:05:06.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:05 smithi138 ceph-mon[32977]: pgmap v1050: 97 pgs: 97 active+clean; 277 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-04-25T14:05:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:05 smithi137 ceph-mon[32568]: pgmap v1050: 97 pgs: 97 active+clean; 277 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-04-25T14:05:07.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 208430 18.83 MB/sec execute 469 sec latency 70.101 ms 2024-04-25T14:05:08.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 208874 18.84 MB/sec execute 470 sec latency 125.174 ms 2024-04-25T14:05:08.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:07 smithi094 ceph-mon[29195]: pgmap v1051: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-04-25T14:05:08.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:07 smithi138 ceph-mon[32977]: pgmap v1051: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-04-25T14:05:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:07 smithi137 ceph-mon[32568]: pgmap v1051: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-04-25T14:05:09.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 209228 18.85 MB/sec execute 471 sec latency 287.323 ms 2024-04-25T14:05:10.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 209585 18.85 MB/sec execute 472 sec latency 91.223 ms 2024-04-25T14:05:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:09 smithi094 ceph-mon[29195]: pgmap v1052: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-04-25T14:05:10.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:09 smithi138 ceph-mon[32977]: pgmap v1052: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-04-25T14:05:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:09 smithi137 ceph-mon[32568]: pgmap v1052: 97 pgs: 97 active+clean; 281 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-04-25T14:05:11.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 210059 18.86 MB/sec execute 473 sec latency 72.118 ms 2024-04-25T14:05:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 -- 172.21.15.94:0/4013589225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91841050e0 msgr2=0x7f91841054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 --2- 172.21.15.94:0/4013589225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91841050e0 0x7f91841054f0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 -- 172.21.15.94:0/4013589225 shutdown_connections 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 --2- 172.21.15.94:0/4013589225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 --2- 172.21.15.94:0/4013589225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9184101790 0x7f9184101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 --2- 172.21.15.94:0/4013589225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91841050e0 0x7f91841054f0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 -- 172.21.15.94:0/4013589225 >> 172.21.15.94:0/4013589225 conn(0x7f91840fd390 msgr2=0x7f91840ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 -- 172.21.15.94:0/4013589225 shutdown_connections 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.833+0000 7f9189d57640 1 -- 172.21.15.94:0/4013589225 wait complete. 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 Processor -- start 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 -- start start 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91841050e0 0x7f91841a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f918410f650 con 0x7f9184101790 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f918410f4d0 con 0x7f9184102290 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9189d57640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f918410f7d0 con 0x7f91841050e0 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55010/0 (socket says 172.21.15.94:55010) 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f91837fe640 1 -- 172.21.15.94:0/2609688766 learned_addr learned my addr 172.21.15.94:0/2609688766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9183fff640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91841050e0 0x7f91841a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9182ffd640 1 -- 172.21.15.94:0/2609688766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91841050e0 msgr2=0x7f91841a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91841050e0 0x7f91841a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9182ffd640 1 -- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 msgr2=0x7f918419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.834+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9182ffd640 1 -- 172.21.15.94:0/2609688766 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841a1df0 con 0x7f9184102290 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f91837fe640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918419a7f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f917800e9f0 tx=0x7f917800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9180ff9640 1 -- 172.21.15.94:0/2609688766 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917800ccc0 con 0x7f9184102290 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841a20e0 con 0x7f9184102290 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9180ff9640 1 -- 172.21.15.94:0/2609688766 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9178005020 con 0x7f9184102290 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9180ff9640 1 -- 172.21.15.94:0/2609688766 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178004a00 con 0x7f9184102290 2024-04-25T14:05:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.835+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841a2630 con 0x7f9184102290 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.838+0000 7f9180ff9640 1 -- 172.21.15.94:0/2609688766 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9178004ba0 con 0x7f9184102290 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.838+0000 7f9190c07740 1 Processor -- start 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.838+0000 7f9190c07740 1 -- start start 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846ebd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559846e23460 con 0x559846af42f0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559846ddd590 con 0x559846b1d1b0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9190c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559846e1f420 con 0x559846afc4d0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55020/0 (socket says 172.21.15.94:55020) 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f91837fe640 1 -- 172.21.15.94:0/2176181902 learned_addr learned my addr 172.21.15.94:0/2176181902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 -- 172.21.15.94:0/2176181902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 msgr2=0x559846ebd970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846ebd970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 -- 172.21.15.94:0/2176181902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 msgr2=0x559846ebdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 -- 172.21.15.94:0/2176181902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559846dea0d0 con 0x559846afc4d0 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f91837fe640 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9180ff9640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 0x7f916c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec4df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f91780dc6b0 tx=0x7f91780e2ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2a3a064be78ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.839+0000 7f9183fff640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 0x7f916c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.840+0000 7f9180ff9640 1 -- 172.21.15.94:0/2609688766 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f91780dec80 con 0x7f9184102290 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.840+0000 7f9169ffb640 1 -- 172.21.15.94:0/2176181902 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917801c020 con 0x559846afc4d0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.840+0000 7f9169ffb640 1 -- 172.21.15.94:0/2176181902 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91780a6bd0 con 0x559846afc4d0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.840+0000 7f9169ffb640 1 -- 172.21.15.94:0/2176181902 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780a2f80 con 0x559846afc4d0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- 172.21.15.94:0/2176181902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 msgr2=0x559846ec4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9183fff640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 0x7f916c0b1ff0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x559846ebeb40 tx=0x7f917c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec4df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f91780dc6b0 tx=0x7f91780e2ee0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- 172.21.15.94:0/2176181902 shutdown_connections 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec4df0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ebdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 --2- 172.21.15.94:0/2176181902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846ebd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- 172.21.15.94:0/2176181902 >> 172.21.15.94:0/2176181902 conn(0x559846b11300 msgr2=0x559846eb9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- 172.21.15.94:0/2176181902 shutdown_connections 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- 172.21.15.94:0/2176181902 wait complete. 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 -- start start 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.841+0000 7f9190c07740 1 Processor -- start 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846eceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559846ec5840 con 0x559846af42f0 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559846daab60 con 0x559846b1d1b0 2024-04-25T14:05:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559846ec59c0 con 0x559846afc4d0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9190c07740 10 client.0 _ll_register_callbacks cb 0x559846dcf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55032/0 (socket says 172.21.15.94:55032) 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 -- 172.21.15.94:0/3564086284 learned_addr learned my addr 172.21.15.94:0/3564086284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9183fff640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846eceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9182ffd640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 -- 172.21.15.94:0/3564086284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 msgr2=0x559846ec7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 -- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 msgr2=0x559846eceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846eceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 -- 172.21.15.94:0/3564086284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559846b35210 con 0x559846af42f0 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9183fff640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846eceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f91837fe640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f9174031500 tx=0x7f9174031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd080a0f04d9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.842+0000 7f9162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174035040 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9174004450 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91780a6bd0 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f91740035f0 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 10 client.23766 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559846c01620 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91780a6bd0 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9174044070 con 0x559846af42f0 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 10 client.23766 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 10 client.23766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 20 client.23766 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846adbd80 0x559846ed52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 4 client.23766 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559846e31070 con 0x559846adbd80 2024-04-25T14:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.843+0000 7f9182ffd640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846adbd80 0x559846ed52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.844+0000 7f9162ffd640 10 client.23766 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.844+0000 7f9182ffd640 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846adbd80 0x559846ed52d0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7da7fc0148b8afd server_cookie=527a3c91645551c2 in_seq=0 out_seq=0 2024-04-25T14:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9162ffd640 1 -- 172.21.15.94:0/3564086284 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559846e31070 con 0x559846adbd80 2024-04-25T14:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9162ffd640 10 client.23766 handle_command_reply: tid=0 2024-04-25T14:05:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 client.23766 shutdown 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 msgr2=0x559846ec7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f9174031500 tx=0x7f9174031840 comp rx=0 tx=0).stop 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 shutdown_connections 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846adbd80 0x559846ed52d0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846b1d1b0 0x559846eceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9162ffd640 0 client.23766 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ec7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 --2- 172.21.15.94:0/3564086284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7670 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 >> 172.21.15.94:0/3564086284 conn(0x559846b11300 msgr2=0x559846eb8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 shutdown_connections 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 1 -- 172.21.15.94:0/3564086284 wait complete. 2024-04-25T14:05:11.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.848+0000 7f9190c07740 20 client.23766 trim_cache size 0 max 16384 2024-04-25T14:05:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 Processor -- start 2024-04-25T14:05:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 -- start start 2024-04-25T14:05:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 0x559846ecea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846b1d1b0 0x559846ebc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91841ab590 con 0x559846af42f0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91841ad0d0 con 0x559846afc4d0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.864+0000 7f9190c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91841abc60 con 0x559846b1d1b0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55044/0 (socket says 172.21.15.94:55044) 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 -- 172.21.15.94:0/2951166933 learned_addr learned my addr 172.21.15.94:0/2951166933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 0x559846ecea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f9183fff640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846b1d1b0 0x559846ebc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 -- 172.21.15.94:0/2951166933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846b1d1b0 msgr2=0x559846ebc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846b1d1b0 0x559846ebc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 -- 172.21.15.94:0/2951166933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 msgr2=0x559846ecea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 0x559846ecea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 -- 172.21.15.94:0/2951166933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559846ebe6d0 con 0x559846af42f0 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f9182ffd640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 0x559846ecea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91837fe640 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f9174009850 tx=0x7f917402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3810b64042186cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91697fa640 1 -- 172.21.15.94:0/2951166933 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174053030 con 0x559846af42f0 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91697fa640 1 -- 172.21.15.94:0/2951166933 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9174004250 con 0x559846af42f0 2024-04-25T14:05:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.865+0000 7f91697fa640 1 -- 172.21.15.94:0/2951166933 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174035040 con 0x559846af42f0 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 -- 172.21.15.94:0/2951166933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 msgr2=0x559846ec7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f9174009850 tx=0x7f917402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 -- 172.21.15.94:0/2951166933 shutdown_connections 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846b1d1b0 0x559846ebc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846afc4d0 0x559846ecea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 --2- 172.21.15.94:0/2951166933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846af42f0 0x559846ec7950 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 -- 172.21.15.94:0/2951166933 >> 172.21.15.94:0/2951166933 conn(0x559846b11300 msgr2=0x559846ec7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 -- 172.21.15.94:0/2951166933 shutdown_connections 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.866+0000 7f9190c07740 1 -- 172.21.15.94:0/2951166933 wait complete. 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 -- start start 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 Processor -- start 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 0x559846ecbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ecc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91841aca00 con 0x559846b1d1b0 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f91841ac330 con 0x559846af42f0 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91841a9570 con 0x559846afc4d0 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9190c07740 10 client.0 _ll_register_callbacks cb 0x7f9174009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9183fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f91837fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 0x559846ecbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9183fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55048/0 (socket says 172.21.15.94:55048) 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9183fff640 1 -- 172.21.15.94:0/191646745 learned_addr learned my addr 172.21.15.94:0/191646745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9182ffd640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ecc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9183fff640 1 -- 172.21.15.94:0/191646745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 msgr2=0x559846ecc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.867+0000 7f9183fff640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ecc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9183fff640 1 -- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 msgr2=0x559846ecbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9183fff640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 0x559846ecbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9183fff640 1 -- 172.21.15.94:0/191646745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559846ecef70 con 0x559846b1d1b0 2024-04-25T14:05:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f91837fe640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 0x559846ecbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9183fff640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f917c004830 tx=0x7f917c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8f326f663611f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c02d060 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f917c01d5c0 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9174004250 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f917c01d7c0 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9190c07740 10 client.23772 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559846eceb00 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9174004250 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.868+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f917c028970 con 0x559846b1d1b0 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 10 client.23772 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 10 client.23772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 20 client.23772 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846ab11c0 0x559846f3fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 4 client.23772 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559846e2aee0 con 0x559846ab11c0 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f91837fe640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846ab11c0 0x559846f3fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f9162ffd640 10 client.23772 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:11.869+0000 7f91837fe640 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846ab11c0 0x559846f3fd10 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4411d2a4cbf25d78 server_cookie=f58cac97f6ff79f2 in_seq=0 out_seq=0 2024-04-25T14:05:12.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 210338 18.86 MB/sec execute 474 sec latency 249.388 ms 2024-04-25T14:05:12.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:11 smithi094 ceph-mon[29195]: pgmap v1053: 97 pgs: 97 active+clean; 287 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-04-25T14:05:12.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:11 smithi138 ceph-mon[32977]: pgmap v1053: 97 pgs: 97 active+clean; 287 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-04-25T14:05:12.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:11 smithi137 ceph-mon[32568]: pgmap v1053: 97 pgs: 97 active+clean; 287 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-04-25T14:05:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9162ffd640 1 -- 172.21.15.94:0/191646745 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559846e2aee0 con 0x559846ab11c0 2024-04-25T14:05:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9162ffd640 10 client.23772 handle_command_reply: tid=0 2024-04-25T14:05:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 client.23772 shutdown 2024-04-25T14:05:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 msgr2=0x559846ecd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f917c004830 tx=0x7f917c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 shutdown_connections 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559846ab11c0 0x559846f3fd10 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559846b1d1b0 0x559846ecd3b0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559846afc4d0 0x559846ecc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9162ffd640 0 client.23772 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 --2- 172.21.15.94:0/191646745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559846af42f0 0x559846ecbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 >> 172.21.15.94:0/191646745 conn(0x559846b11300 msgr2=0x559846eb9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 shutdown_connections 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 1 -- 172.21.15.94:0/191646745 wait complete. 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.557+0000 7f9190c07740 20 client.23772 trim_cache size 0 max 16384 2024-04-25T14:05:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:05:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:05:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:05:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:05:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 msgr2=0x7f916c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 0x7f916c0b1ff0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x559846ebeb40 tx=0x7f917c009500 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 msgr2=0x7f918419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918419a7f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f917800e9f0 tx=0x7f917800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 shutdown_connections 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f916c0afae0 0x7f916c0b1ff0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91841050e0 0x7f91841a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9184102290 0x7f918419a7f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 --2- 172.21.15.94:0/2609688766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9184101790 0x7f918419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 >> 172.21.15.94:0/2609688766 conn(0x7f91840fd390 msgr2=0x7f9184105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 shutdown_connections 2024-04-25T14:05:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:12.558+0000 7f9189d57640 1 -- 172.21.15.94:0/2609688766 wait complete. 2024-04-25T14:05:13.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 210617 18.85 MB/sec execute 475 sec latency 418.144 ms 2024-04-25T14:05:14.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 210912 18.86 MB/sec execute 476 sec latency 152.865 ms 2024-04-25T14:05:14.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:13 smithi094 ceph-mon[29195]: pgmap v1054: 97 pgs: 97 active+clean; 287 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-04-25T14:05:14.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:13 smithi138 ceph-mon[32977]: pgmap v1054: 97 pgs: 97 active+clean; 287 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-04-25T14:05:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:13 smithi137 ceph-mon[32568]: pgmap v1054: 97 pgs: 97 active+clean; 287 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 343 op/s 2024-04-25T14:05:15.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 211227 18.85 MB/sec execute 477 sec latency 92.417 ms 2024-04-25T14:05:16.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 211558 18.83 MB/sec execute 478 sec latency 129.835 ms 2024-04-25T14:05:16.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:15 smithi094 ceph-mon[29195]: pgmap v1055: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-04-25T14:05:16.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:15 smithi138 ceph-mon[32977]: pgmap v1055: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-04-25T14:05:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:15 smithi137 ceph-mon[32568]: pgmap v1055: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 524 op/s 2024-04-25T14:05:17.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 211900 18.83 MB/sec execute 479 sec latency 161.993 ms 2024-04-25T14:05:18.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 212277 18.83 MB/sec execute 480 sec latency 113.978 ms 2024-04-25T14:05:18.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:17 smithi138 ceph-mon[32977]: pgmap v1056: 97 pgs: 97 active+clean; 278 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-04-25T14:05:18.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:17 smithi137 ceph-mon[32568]: pgmap v1056: 97 pgs: 97 active+clean; 278 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-04-25T14:05:18.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:17 smithi094 ceph-mon[29195]: pgmap v1056: 97 pgs: 97 active+clean; 278 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2024-04-25T14:05:18.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:18.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:19.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05f2ffd640 1 -- 172.21.15.94:0/1251715136 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e40318a0 con 0x7f05f41029e0 2024-04-25T14:05:19.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 -- 172.21.15.94:0/1251715136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 msgr2=0x7f05f41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 --2- 172.21.15.94:0/1251715136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f41036e0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009900 tx=0x7f05e402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 -- 172.21.15.94:0/1251715136 shutdown_connections 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 --2- 172.21.15.94:0/1251715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 0x7f05f410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 --2- 172.21.15.94:0/1251715136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 0x7f05f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.073+0000 7f05fa75e640 1 --2- 172.21.15.94:0/1251715136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f41036e0 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.074+0000 7f05fa75e640 1 -- 172.21.15.94:0/1251715136 >> 172.21.15.94:0/1251715136 conn(0x7f05f40fdbd0 msgr2=0x7f05f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.074+0000 7f05fa75e640 1 -- 172.21.15.94:0/1251715136 shutdown_connections 2024-04-25T14:05:19.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.074+0000 7f05fa75e640 1 -- 172.21.15.94:0/1251715136 wait complete. 2024-04-25T14:05:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.074+0000 7f05fa75e640 1 Processor -- start 2024-04-25T14:05:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.074+0000 7f05fa75e640 1 -- start start 2024-04-25T14:05:19.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 0x7f05f419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 0x7f05f41a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410d500 con 0x7f05f41029e0 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410d380 con 0x7f05f4103c20 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410d680 con 0x7f05f4108920 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34204/0 (socket says 172.21.15.94:34204) 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 -- 172.21.15.94:0/2314185535 learned_addr learned my addr 172.21.15.94:0/2314185535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f37fe640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 0x7f05f419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 0x7f05f41a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 -- 172.21.15.94:0/2314185535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 msgr2=0x7f05f41a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 0x7f05f41a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 -- 172.21.15.94:0/2314185535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 msgr2=0x7f05f419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 0x7f05f419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 -- 172.21.15.94:0/2314185535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f41a42e0 con 0x7f05f41029e0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f3fff640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006010 tx=0x7f05e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f17fa640 1 -- 172.21.15.94:0/2314185535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e403f030 con 0x7f05f41029e0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f17fa640 1 -- 172.21.15.94:0/2314185535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e4003aa0 con 0x7f05f41029e0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05f17fa640 1 -- 172.21.15.94:0/2314185535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4035070 con 0x7f05f41029e0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.075+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f41a4570 con 0x7f05f41029e0 2024-04-25T14:05:19.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.076+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a4a60 con 0x7f05f41029e0 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f17fa640 1 -- 172.21.15.94:0/2314185535 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f05e4042030 con 0x7f05f41029e0 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 Processor -- start 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 -- start start 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 0x560df2338a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2338fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 0x560df2339c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560df229dd80 con 0x560df1f97390 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df22578b0 con 0x560df1f766b0 2024-04-25T14:05:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f060160f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560df2299d40 con 0x560df1f6e4d0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 0x560df2338a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 0x560df2339c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 0x560df2338a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34216/0 (socket says 172.21.15.94:34216) 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f8cfb640 1 -- 172.21.15.94:0/2132338479 learned_addr learned my addr 172.21.15.94:0/2132338479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.078+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2338fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 -- 172.21.15.94:0/2132338479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 msgr2=0x560df2338fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2338fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 -- 172.21.15.94:0/2132338479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 msgr2=0x560df2338a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 0x560df2338a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f17fa640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 0x7f05dc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 -- 172.21.15.94:0/2132338479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df2228a40 con 0x560df1f766b0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f3fff640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 0x7f05dc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f37fe640 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 0x560df2339c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005d00 tx=0x7f05e80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee76fc155167a575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05f17fa640 1 -- 172.21.15.94:0/2314185535 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f05e40ff260 con 0x7f05f41029e0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05da7fc640 1 -- 172.21.15.94:0/2132338479 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e801c680 con 0x560df1f766b0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.079+0000 7f05da7fc640 1 -- 172.21.15.94:0/2132338479 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e800be20 con 0x560df1f766b0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f05da7fc640 1 -- 172.21.15.94:0/2132338479 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8011ba0 con 0x560df1f766b0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- 172.21.15.94:0/2132338479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 msgr2=0x560df2339c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 0x560df2339c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005d00 tx=0x7f05e80107a0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f05f3fff640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 0x7f05dc0b1fe0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x560df2338780 tx=0x7f05e4102d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- 172.21.15.94:0/2132338479 shutdown_connections 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f766b0 0x560df2339c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2338fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 --2- 172.21.15.94:0/2132338479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f97390 0x560df2338a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- 172.21.15.94:0/2132338479 >> 172.21.15.94:0/2132338479 conn(0x560df1f8b4e0 msgr2=0x560df2334270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- 172.21.15.94:0/2132338479 shutdown_connections 2024-04-25T14:05:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- 172.21.15.94:0/2132338479 wait complete. 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.080+0000 7f060160f740 1 -- start start 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 Processor -- start 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df2342ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f97390 0x560df2349e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560df233a820 con 0x560df1f6e4d0 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df2224d10 con 0x560df1f97390 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560df233a9a0 con 0x560df1f766b0 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f060160f740 10 client.0 _ll_register_callbacks cb 0x560df2251210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34220/0 (socket says 172.21.15.94:34220) 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 -- 172.21.15.94:0/766274654 learned_addr learned my addr 172.21.15.94:0/766274654 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f97390 0x560df2349e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f37fe640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df2342ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 -- 172.21.15.94:0/766274654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 msgr2=0x560df2342ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df2342ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 -- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f97390 msgr2=0x560df2349e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f97390 0x560df2349e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.081+0000 7f05f8cfb640 1 -- 172.21.15.94:0/766274654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df20797d0 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00f7f0 tx=0x7f05ec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bede09cacec804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00fe30 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f05ec003970 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e800be20 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f05ec027050 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f060160f740 10 client.23784 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560df1faf3f0 con 0x560df1f6e4d0 2024-04-25T14:05:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.082+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e800be20 con 0x560df1f6e4d0 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f05ec004020 con 0x560df1f6e4d0 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 10 client.23784 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 10 client.23784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 20 client.23784 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f55f60 0x560df2350330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 4 client.23784 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df22ab990 con 0x560df1f55f60 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f05f37fe640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f55f60 0x560df2350330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.083+0000 7f05d37fe640 10 client.23784 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.084+0000 7f05f37fe640 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f55f60 0x560df2350330 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707cbf65da02411d server_cookie=16587fac3eea85f0 in_seq=0 out_seq=0 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f05d37fe640 1 -- 172.21.15.94:0/766274654 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560df22ab990 con 0x560df1f55f60 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f05d37fe640 10 client.23784 handle_command_reply: tid=0 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 client.23784 shutdown 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 msgr2=0x560df23429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00f7f0 tx=0x7f05ec009500 comp rx=0 tx=0).stop 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 shutdown_connections 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f55f60 0x560df2350330 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f97390 0x560df2349e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f05d37fe640 0 client.23784 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df2342ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 --2- 172.21.15.94:0/766274654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f6e4d0 0x560df23429a0 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 >> 172.21.15.94:0/766274654 conn(0x560df1f8b4e0 msgr2=0x560df2337180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 shutdown_connections 2024-04-25T14:05:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 1 -- 172.21.15.94:0/766274654 wait complete. 2024-04-25T14:05:19.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.086+0000 7f060160f740 20 client.23784 trim_cache size 0 max 16384 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 Processor -- start 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 -- start start 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f55f60 0x560df2342570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2342ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41ad780 con 0x560df1f766b0 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41af2c0 con 0x560df1f55f60 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f060160f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41ade50 con 0x560df1f6e4d0 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34236/0 (socket says 172.21.15.94:34236) 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 -- 172.21.15.94:0/53922686 learned_addr learned my addr 172.21.15.94:0/53922686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f37fe640 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2342ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f55f60 0x560df2342570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 -- 172.21.15.94:0/53922686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 msgr2=0x560df2342ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2342ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 -- 172.21.15.94:0/53922686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f55f60 msgr2=0x560df2342570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f55f60 0x560df2342570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.109+0000 7f05f8cfb640 1 -- 172.21.15.94:0/53922686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df1f6f0f0 con 0x560df1f766b0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.110+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009690 tx=0x7f05ec009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71303367623e8ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.110+0000 7f05d9ffb640 1 -- 172.21.15.94:0/53922686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00eea0 con 0x560df1f766b0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.110+0000 7f05d9ffb640 1 -- 172.21.15.94:0/53922686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ec01b040 con 0x560df1f766b0 2024-04-25T14:05:19.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.110+0000 7f05d9ffb640 1 -- 172.21.15.94:0/53922686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec003970 con 0x560df1f766b0 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- 172.21.15.94:0/53922686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 msgr2=0x560df2349dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009690 tx=0x7f05ec009e50 comp rx=0 tx=0).stop 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- 172.21.15.94:0/53922686 shutdown_connections 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f766b0 0x560df2349dd0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f6e4d0 0x560df2342ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 --2- 172.21.15.94:0/53922686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f55f60 0x560df2342570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- 172.21.15.94:0/53922686 >> 172.21.15.94:0/53922686 conn(0x560df1f8b4e0 msgr2=0x560df233d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- 172.21.15.94:0/53922686 shutdown_connections 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- 172.21.15.94:0/53922686 wait complete. 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 -- start start 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.111+0000 7f060160f740 1 Processor -- start 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 0x560df23474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 0x560df2347a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df23b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41aebf0 con 0x560df1f55f60 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41ae520 con 0x560df1f6e4d0 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41ab760 con 0x560df1f766b0 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f060160f740 10 client.0 _ll_register_callbacks cb 0x7f05ec009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 0x560df23474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 0x560df2347a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f8cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 0x560df23474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34252/0 (socket says 172.21.15.94:34252) 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f8cfb640 1 -- 172.21.15.94:0/1660104259 learned_addr learned my addr 172.21.15.94:0/1660104259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df23b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 -- 172.21.15.94:0/1660104259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 msgr2=0x560df23b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df23b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 -- 172.21.15.94:0/1660104259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 msgr2=0x560df23474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 0x560df23474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 -- 172.21.15.94:0/1660104259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df225e4b0 con 0x560df1f6e4d0 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:05:19.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.112+0000 7f05f37fe640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 0x560df2347a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x560df2343a10 tx=0x7f05e800beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af654ecb0a9d68c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800b890 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ec01b040 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f05e8005250 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f05e803b050 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 10 client.33173 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560df23b7ab0 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ec01b040 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f05e8004660 con 0x560df1f6e4d0 2024-04-25T14:05:19.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 10 client.33173 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 10 client.33173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 20 client.33173 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f97390 0x560df23ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 4 client.33173 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.113+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560df22a5800 con 0x560df1f97390 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.114+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f97390 0x560df23ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.114+0000 7f05d37fe640 10 client.33173 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:19.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.114+0000 7f05f8cfb640 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f97390 0x560df23ba190 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb509c204568d02 server_cookie=9c59e61532bbeea3 in_seq=0 out_seq=0 2024-04-25T14:05:19.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f05d37fe640 1 -- 172.21.15.94:0/1660104259 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560df22a5800 con 0x560df1f97390 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f05d37fe640 10 client.33173 handle_command_reply: tid=0 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 client.33173 shutdown 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 msgr2=0x560df2347a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 0x560df2347a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x560df2343a10 tx=0x7f05e800beb0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 shutdown_connections 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x560df1f97390 0x560df23ba190 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560df1f766b0 0x560df23b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df1f6e4d0 0x560df2347a20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 --2- 172.21.15.94:0/1660104259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560df1f55f60 0x560df23474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f05d37fe640 0 client.33173 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 >> 172.21.15.94:0/1660104259 conn(0x560df1f8b4e0 msgr2=0x560df2342570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 shutdown_connections 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 1 -- 172.21.15.94:0/1660104259 wait complete. 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.118+0000 7f060160f740 20 client.33173 trim_cache size 0 max 16384 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:19.122 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:05:19.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:05:19.123 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:19.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 msgr2=0x7f05dc0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 0x7f05dc0b1fe0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x560df2338780 tx=0x7f05e4102d40 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 msgr2=0x7f05f419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006010 tx=0x7f05e4004270 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 shutdown_connections 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f05dc0afad0 0x7f05dc0b1fe0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4108920 0x7f05f41a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05f4103c20 0x7f05f419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 --2- 172.21.15.94:0/2314185535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f41029e0 0x7f05f419c7a0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 >> 172.21.15.94:0/2314185535 conn(0x7f05f40fdbd0 msgr2=0x7f05f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 shutdown_connections 2024-04-25T14:05:19.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.119+0000 7f05fa75e640 1 -- 172.21.15.94:0/2314185535 wait complete. 2024-04-25T14:05:19.131 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:05:19.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:19.159 INFO:tasks.workunit.client.0.smithi094.stdout: 10 212649 18.84 MB/sec execute 481 sec latency 295.920 ms 2024-04-25T14:05:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 -- 172.21.15.94:0/1411054741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 msgr2=0x7fb90c104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 --2- 172.21.15.94:0/1411054741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c104060 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fb900009900 tx=0x7fb90002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 -- 172.21.15.94:0/1411054741 shutdown_connections 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 --2- 172.21.15.94:0/1411054741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c108950 0x7fb90c10ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 --2- 172.21.15.94:0/1411054741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c104060 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 --2- 172.21.15.94:0/1411054741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c102a10 0x7fb90c103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 -- 172.21.15.94:0/1411054741 >> 172.21.15.94:0/1411054741 conn(0x7fb90c0fdc00 msgr2=0x7fb90c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 -- 172.21.15.94:0/1411054741 shutdown_connections 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.250+0000 7fb91141e640 1 -- 172.21.15.94:0/1411054741 wait complete. 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 Processor -- start 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 -- start start 2024-04-25T14:05:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 0x7fb90c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c108950 0x7fb90c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10e540 con 0x7fb90c103c50 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10e3c0 con 0x7fb90c108950 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb91141e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10e6c0 con 0x7fb90c102a10 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.251+0000 7fb90a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34278/0 (socket says 172.21.15.94:34278) 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 -- 172.21.15.94:0/4072650384 learned_addr learned my addr 172.21.15.94:0/4072650384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90affd640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 0x7fb90c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90b7fe640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c108950 0x7fb90c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 -- 172.21.15.94:0/4072650384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 msgr2=0x7fb90c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 0x7fb90c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 -- 172.21.15.94:0/4072650384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c108950 msgr2=0x7fb90c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c108950 0x7fb90c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 -- 172.21.15.94:0/4072650384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c1a1dd0 con 0x7fb90c103c50 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90affd640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 0x7fb90c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb90a7fc640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fb900006010 tx=0x7fb9000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90003f030 con 0x7fb90c103c50 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c1a2060 con 0x7fb90c103c50 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c1a25a0 con 0x7fb90c103c50 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb900031910 con 0x7fb90c103c50 2024-04-25T14:05:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.252+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900035070 con 0x7fb90c103c50 2024-04-25T14:05:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.254+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c1a2860 con 0x7fb90c103c50 2024-04-25T14:05:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.255+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb900042030 con 0x7fb90c103c50 2024-04-25T14:05:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.256+0000 7fb8f3fff640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 0x7fb8f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.256+0000 7fb90affd640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 0x7fb8f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.256+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb9000ff060 con 0x7fb90c103c50 2024-04-25T14:05:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.257+0000 7fb90affd640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 0x7fb8f40b2000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb90c103610 tx=0x7fb8fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.261+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9000c75f0 con 0x7fb90c103c50 2024-04-25T14:05:19.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.536+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb90c1a2a30 con 0x7fb90c103c50 2024-04-25T14:05:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.541+0000 7fb8f3fff640 1 -- 172.21.15.94:0/4072650384 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7fb9000cb8f0 con 0x7fb90c103c50 2024-04-25T14:05:19.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:05:19.541 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:05:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:05:19.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 msgr2=0x7fb8f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 0x7fb8f40b2000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb90c103610 tx=0x7fb8fc005f90 comp rx=0 tx=0).stop 2024-04-25T14:05:19.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 msgr2=0x7fb90c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fb900006010 tx=0x7fb9000042f0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 shutdown_connections 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb8f40afaf0 0x7fb8f40b2000 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb90c108950 0x7fb90c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb90c103c50 0x7fb90c19a840 secure :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fb900006010 tx=0x7fb9000042f0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 --2- 172.21.15.94:0/4072650384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb90c102a10 0x7fb90c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.544+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 >> 172.21.15.94:0/4072650384 conn(0x7fb90c0fdc00 msgr2=0x7fb90c109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.545+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 shutdown_connections 2024-04-25T14:05:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.545+0000 7fb91141e640 1 -- 172.21.15.94:0/4072650384 wait complete. 2024-04-25T14:05:19.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:05:19.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 -- 172.21.15.94:0/1620964721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18103c20 msgr2=0x7fbe18104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1620964721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18103c20 0x7fbe18104030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe0802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 -- 172.21.15.94:0/1620964721 shutdown_connections 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1620964721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18108920 0x7fbe1810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1620964721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18103c20 0x7fbe18104030 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1620964721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe181036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.746+0000 7fbe2038f640 1 -- 172.21.15.94:0/1620964721 >> 172.21.15.94:0/1620964721 conn(0x7fbe180fdbd0 msgr2=0x7fbe18100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.747+0000 7fbe2038f640 1 -- 172.21.15.94:0/1620964721 shutdown_connections 2024-04-25T14:05:19.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.750+0000 7fbe2038f640 1 -- 172.21.15.94:0/1620964721 wait complete. 2024-04-25T14:05:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.750+0000 7fbe2038f640 1 Processor -- start 2024-04-25T14:05:19.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 -- start start 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18103c20 0x7fbe1819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810e4e0 con 0x7fbe181029e0 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810e360 con 0x7fbe18103c20 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe2038f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810e660 con 0x7fbe18108920 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1e92c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34294/0 (socket says 172.21.15.94:34294) 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1e92c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40798/0 (socket says 172.21.15.94:40798) 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1e12b640 1 -- 172.21.15.94:0/1473005123 learned_addr learned my addr 172.21.15.94:0/1473005123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.751+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18103c20 0x7fbe1819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 -- 172.21.15.94:0/1473005123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 msgr2=0x7fbe181a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 -- 172.21.15.94:0/1473005123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18103c20 msgr2=0x7fbe1819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18103c20 0x7fbe1819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 -- 172.21.15.94:0/1473005123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181a2040 con 0x7fbe181029e0 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e980 tx=0x7fbe1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe0f7fe640 1 -- 172.21.15.94:0/1473005123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400cd80 con 0x7fbe181029e0 2024-04-25T14:05:19.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe181a2330 con 0x7fbe181029e0 2024-04-25T14:05:19.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.752+0000 7fbe0f7fe640 1 -- 172.21.15.94:0/1473005123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe140108e0 con 0x7fbe181029e0 2024-04-25T14:05:19.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.753+0000 7fbe0f7fe640 1 -- 172.21.15.94:0/1473005123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14018890 con 0x7fbe181029e0 2024-04-25T14:05:19.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.753+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a28d0 con 0x7fbe181029e0 2024-04-25T14:05:19.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.755+0000 7fbe0f7fe640 1 -- 172.21.15.94:0/1473005123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbe1401f020 con 0x7fbe181029e0 2024-04-25T14:05:19.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 Processor -- start 2024-04-25T14:05:19.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 -- start start 2024-04-25T14:05:19.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f024d0 0x55c3b92d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3b9231d80 con 0x55c3b8f2b390 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3b91eba20 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe27240740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3b922dd40 con 0x55c3b8f024d0 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe1e92c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe1e92c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34308/0 (socket says 172.21.15.94:34308) 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe1e92c640 1 -- 172.21.15.94:0/2549196038 learned_addr learned my addr 172.21.15.94:0/2549196038 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.756+0000 7fbe0f7fe640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 0x7fbe040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f024d0 0x55c3b92d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2549196038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f024d0 msgr2=0x55c3b92d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f024d0 0x55c3b92d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2549196038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 msgr2=0x55c3b92d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2549196038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b91f6c30 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 0x7fbe040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe0f7fe640 1 -- 172.21.15.94:0/1473005123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbe140dee40 con 0x7fbe181029e0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.757+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d2e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe1819b5a0 tx=0x7fbe08005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc5343e58bee0df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.758+0000 7fbe0cbfe640 1 -- 172.21.15.94:0/2549196038 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0803f070 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.758+0000 7fbe0cbfe640 1 -- 172.21.15.94:0/2549196038 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe08004380 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.758+0000 7fbe0cbfe640 1 -- 172.21.15.94:0/2549196038 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0803f070 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.758+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 0x7fbe040b2000 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55c3b92d2c70 tx=0x7fbe10004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 -- 172.21.15.94:0/2549196038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 msgr2=0x55c3b92d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d2e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe1819b5a0 tx=0x7fbe08005df0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 -- 172.21.15.94:0/2549196038 shutdown_connections 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d2e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f024d0 0x55c3b92d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 --2- 172.21.15.94:0/2549196038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 -- 172.21.15.94:0/2549196038 >> 172.21.15.94:0/2549196038 conn(0x55c3b8f1f4e0 msgr2=0x55c3b92c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 -- 172.21.15.94:0/2549196038 shutdown_connections 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.759+0000 7fbe27240740 1 -- 172.21.15.94:0/2549196038 wait complete. 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.760+0000 7fbe27240740 1 -- start start 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.760+0000 7fbe27240740 1 Processor -- start 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c3b92d38e0 con 0x55c3b8f2b390 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92db970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33352/0 (socket says 172.21.15.94:33352) 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1e12b640 1 -- 172.21.15.94:0/2405356766 learned_addr learned my addr 172.21.15.94:0/2405356766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c3b91b8dc0 con 0x55c3b8f024d0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3b92d3a60 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe27240740 10 client.0 _ll_register_callbacks cb 0x55c3b91d4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2405356766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 msgr2=0x55c3b92db970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92db970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2405356766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 msgr2=0x55c3b92e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.761+0000 7fbe1d92a640 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b8f433f0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92dbeb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe08003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b21027d2bd0f3fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0803f070 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbe08004180 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe08004380 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbe08035070 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe27240740 10 client.32833 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c3b900d7d0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.762+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe08009520 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbe08044aa0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 10 client.32833 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 10 client.32833 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 20 client.32833 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ee9f60 0x55c3b92e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 4 client.32833 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbe08004380 con 0x55c3b8ee9f60 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.763+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ee9f60 0x55c3b92e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.764+0000 7fbe01ffb640 10 client.32833 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:19.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.764+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ee9f60 0x55c3b92e9220 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8320fa7bc7b7ec3f server_cookie=2dc5394927539c59 in_seq=0 out_seq=0 2024-04-25T14:05:19.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe01ffb640 1 -- 172.21.15.94:0/2405356766 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbe08004380 con 0x55c3b8ee9f60 2024-04-25T14:05:19.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe01ffb640 10 client.32833 handle_command_reply: tid=0 2024-04-25T14:05:19.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 client.32833 shutdown 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 msgr2=0x55c3b92dbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92dbeb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe08003f00 comp rx=0 tx=0).stop 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 shutdown_connections 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ee9f60 0x55c3b92e9220 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f2b390 0x55c3b92e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92dbeb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 --2- 172.21.15.94:0/2405356766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92db970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 >> 172.21.15.94:0/2405356766 conn(0x55c3b8f1f4e0 msgr2=0x55c3b92c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe01ffb640 0 client.32833 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 shutdown_connections 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.765+0000 7fbe27240740 1 -- 172.21.15.94:0/2405356766 wait complete. 2024-04-25T14:05:19.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.766+0000 7fbe27240740 20 client.32833 trim_cache size 0 max 16384 2024-04-25T14:05:19.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.787+0000 7fbe27240740 1 Processor -- start 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.787+0000 7fbe27240740 1 -- start start 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f2b390 0x55c3b92da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a92f0 con 0x55c3b8f024d0 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aae30 con 0x55c3b8f2b390 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe27240740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a99c0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34324/0 (socket says 172.21.15.94:34324) 2024-04-25T14:05:19.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 -- 172.21.15.94:0/474553107 learned_addr learned my addr 172.21.15.94:0/474553107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f2b390 0x55c3b92da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 -- 172.21.15.94:0/474553107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 msgr2=0x55c3b92d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 -- 172.21.15.94:0/474553107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f2b390 msgr2=0x55c3b92da0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f2b390 0x55c3b92da0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.788+0000 7fbe1e12b640 1 -- 172.21.15.94:0/474553107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b92c32b0 con 0x55c3b8f024d0 2024-04-25T14:05:19.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.789+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c6a0 tx=0x7fbe14075d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c415ab8bc7be53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.789+0000 7fbe03fff640 1 -- 172.21.15.94:0/474553107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140ab5d0 con 0x55c3b8f024d0 2024-04-25T14:05:19.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.789+0000 7fbe03fff640 1 -- 172.21.15.94:0/474553107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe140b0080 con 0x55c3b8f024d0 2024-04-25T14:05:19.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.789+0000 7fbe03fff640 1 -- 172.21.15.94:0/474553107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140afbf0 con 0x55c3b8f024d0 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- 172.21.15.94:0/474553107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 msgr2=0x55c3b92d66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c6a0 tx=0x7fbe14075d70 comp rx=0 tx=0).stop 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- 172.21.15.94:0/474553107 shutdown_connections 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f2b390 0x55c3b92da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 --2- 172.21.15.94:0/474553107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f024d0 0x55c3b92d66c0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- 172.21.15.94:0/474553107 >> 172.21.15.94:0/474553107 conn(0x55c3b8f1f4e0 msgr2=0x55c3b92d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- 172.21.15.94:0/474553107 shutdown_connections 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- 172.21.15.94:0/474553107 wait complete. 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 -- start start 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.790+0000 7fbe27240740 1 Processor -- start 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f2b390 0x55c3b92df290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aa760 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aa090 con 0x55c3b8f024d0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a72d0 con 0x55c3b8f2b390 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 10 client.0 _ll_register_callbacks cb 0x7fbe10009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34326/0 (socket says 172.21.15.94:34326) 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 -- 172.21.15.94:0/1578217022 learned_addr learned my addr 172.21.15.94:0/1578217022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1e92c640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f2b390 0x55c3b92df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 -- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f2b390 msgr2=0x55c3b92df290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f2b390 0x55c3b92df290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 -- 172.21.15.94:0/1578217022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 msgr2=0x55c3b92e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92e21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 -- 172.21.15.94:0/1578217022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b92df8d0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe1d92a640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a450 tx=0x7fbe08058450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55950d7282f8d8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0803f040 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbe08004390 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.791+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe140b0080 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbe08068050 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 10 client.23814 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c3b9350510 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe140b0080 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbe080589f0 con 0x55c3b8f0a6b0 2024-04-25T14:05:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 10 client.23814 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 10 client.23814 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 20 client.23814 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ebf3a0 0x55c3b9352f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 4 client.23814 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3b923f990 con 0x55c3b8ebf3a0 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.792+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ebf3a0 0x55c3b9352f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.793+0000 7fbe01ffb640 10 client.23814 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.793+0000 7fbe1e12b640 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ebf3a0 0x55c3b9352f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2784560624469b91 server_cookie=6526e075221a70d2 in_seq=0 out_seq=0 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.793+0000 7fbe01ffb640 1 -- 172.21.15.94:0/1578217022 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3b923f990 con 0x55c3b8ebf3a0 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.793+0000 7fbe01ffb640 10 client.23814 handle_command_reply: tid=0 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.793+0000 7fbe27240740 1 client.23814 shutdown 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 msgr2=0x55c3b92ded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a450 tx=0x7fbe08058450 comp rx=0 tx=0).stop 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 shutdown_connections 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55c3b8ebf3a0 0x55c3b9352f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe01ffb640 0 client.23814 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b8f2b390 0x55c3b92df290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3b8f0a6b0 0x55c3b92ded50 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 --2- 172.21.15.94:0/1578217022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b8f024d0 0x55c3b92e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 >> 172.21.15.94:0/1578217022 conn(0x55c3b8f1f4e0 msgr2=0x55c3b92d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.794+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 shutdown_connections 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.795+0000 7fbe27240740 1 -- 172.21.15.94:0/1578217022 wait complete. 2024-04-25T14:05:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.795+0000 7fbe27240740 20 client.23814 trim_cache size 0 max 16384 2024-04-25T14:05:19.801 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.796+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 msgr2=0x7fbe040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.796+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 0x7fbe040b2000 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55c3b92d2c70 tx=0x7fbe10004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.796+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 msgr2=0x7fbe1819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e980 tx=0x7fbe1400ee40 comp rx=0 tx=0).stop 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 shutdown_connections 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbe040afaf0 0x7fbe040b2000 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe18108920 0x7fbe181a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe18103c20 0x7fbe1819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 --2- 172.21.15.94:0/1473005123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe181029e0 0x7fbe1819a500 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 >> 172.21.15.94:0/1473005123 conn(0x7fbe180fdbd0 msgr2=0x7fbe180ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 shutdown_connections 2024-04-25T14:05:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.797+0000 7fbe2038f640 1 -- 172.21.15.94:0/1473005123 wait complete. 2024-04-25T14:05:19.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.936+0000 7fcc0856c640 1 -- 172.21.15.94:0/2602220490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00103af0 msgr2=0x7fcc001047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.936+0000 7fcc0856c640 1 --2- 172.21.15.94:0/2602220490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00103af0 0x7fcc001047f0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009900 tx=0x7fcbfc02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- 172.21.15.94:0/2602220490 shutdown_connections 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- 172.21.15.94:0/2602220490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00109a30 0x7fcc0010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- 172.21.15.94:0/2602220490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 0x7fcc00105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- 172.21.15.94:0/2602220490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00103af0 0x7fcc001047f0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- 172.21.15.94:0/2602220490 >> 172.21.15.94:0/2602220490 conn(0x7fcc000fece0 msgr2=0x7fcc00101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- 172.21.15.94:0/2602220490 shutdown_connections 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- 172.21.15.94:0/2602220490 wait complete. 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 Processor -- start 2024-04-25T14:05:19.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- start start 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00103af0 0x7fcc0019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 0x7fcc0019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f5f0 con 0x7fcc00109a30 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f470 con 0x7fcc00103af0 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.937+0000 7fcc0856c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0010f770 con 0x7fcc00104d30 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34336/0 (socket says 172.21.15.94:34336) 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 -- 172.21.15.94:0/3081397246 learned_addr learned my addr 172.21.15.94:0/3081397246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06308640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00103af0 0x7fcc0019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc05b07640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 0x7fcc0019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 -- 172.21.15.94:0/3081397246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 msgr2=0x7fcc0019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 0x7fcc0019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 -- 172.21.15.94:0/3081397246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00103af0 msgr2=0x7fcc0019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00103af0 0x7fcc0019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 -- 172.21.15.94:0/3081397246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a1d90 con 0x7fcc00109a30 2024-04-25T14:05:19.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc06b09640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800eb10 tx=0x7fcbf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcbf77fe640 1 -- 172.21.15.94:0/3081397246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8004b20 con 0x7fcc00109a30 2024-04-25T14:05:19.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a2080 con 0x7fcc00109a30 2024-04-25T14:05:19.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcbf77fe640 1 -- 172.21.15.94:0/3081397246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf8004cc0 con 0x7fcc00109a30 2024-04-25T14:05:19.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcbf77fe640 1 -- 172.21.15.94:0/3081397246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8010670 con 0x7fcc00109a30 2024-04-25T14:05:19.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.938+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001a25d0 con 0x7fcc00109a30 2024-04-25T14:05:19.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.941+0000 7fcc0f417740 1 Processor -- start 2024-04-25T14:05:19.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcbf77fe640 1 -- 172.21.15.94:0/3081397246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcbf80108f0 con 0x7fcc00109a30 2024-04-25T14:05:19.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 -- start start 2024-04-25T14:05:19.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 0x555c36dd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36dd6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c36d39d80 con 0x555c36a0a4d0 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c36cf3a20 con 0x555c36a126b0 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc0f417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c36d35d40 con 0x555c36a33390 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc05b07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 0x555c36dd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34340/0 (socket says 172.21.15.94:34340) 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 -- 172.21.15.94:0/3935465139 learned_addr learned my addr 172.21.15.94:0/3935465139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc05b07640 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36dd6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcbf77fe640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 0x7fcbec0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 -- 172.21.15.94:0/3935465139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 msgr2=0x555c36dd5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 0x555c36dd5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.942+0000 7fcc06308640 1 -- 172.21.15.94:0/3935465139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 msgr2=0x555c36dd6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcc06308640 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36dd6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcc06308640 1 -- 172.21.15.94:0/3935465139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c36cfec30 con 0x555c36a0a4d0 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcc05b07640 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 0x555c36dd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcc06b09640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 0x7fcbec0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcc06308640 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009230 tx=0x7fcbfc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6182d55a230231e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcbf77fe640 1 -- 172.21.15.94:0/3081397246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcbf80df1d0 con 0x7fcc00109a30 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcbf4bfe640 1 -- 172.21.15.94:0/3935465139 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc035030 con 0x555c36a0a4d0 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcbf4bfe640 1 -- 172.21.15.94:0/3935465139 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc040070 con 0x555c36a0a4d0 2024-04-25T14:05:19.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.943+0000 7fcbf4bfe640 1 -- 172.21.15.94:0/3935465139 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0038c0 con 0x555c36a0a4d0 2024-04-25T14:05:19.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 -- 172.21.15.94:0/3935465139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 msgr2=0x555c36dd5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc06b09640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 0x7fcbec0b2080 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80dcc80 tx=0x7fcbf8016f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009230 tx=0x7fcbfc009620 comp rx=0 tx=0).stop 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 -- 172.21.15.94:0/3935465139 shutdown_connections 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36dd6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5d90 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 --2- 172.21.15.94:0/3935465139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a33390 0x555c36dd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 -- 172.21.15.94:0/3935465139 >> 172.21.15.94:0/3935465139 conn(0x555c36a274e0 msgr2=0x555c36dd1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 -- 172.21.15.94:0/3935465139 shutdown_connections 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.944+0000 7fcc0f417740 1 -- 172.21.15.94:0/3935465139 wait complete. 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 -- start start 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 Processor -- start 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36ddf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36de6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555c36dd74d0 con 0x555c36a126b0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555c36cc0dc0 con 0x555c36a33390 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c36dd7650 con 0x555c36a0a4d0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc0f417740 10 client.0 _ll_register_callbacks cb 0x555c36cdc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34346/0 (socket says 172.21.15.94:34346) 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 -- 172.21.15.94:0/15754795 learned_addr learned my addr 172.21.15.94:0/15754795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 -- 172.21.15.94:0/15754795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 msgr2=0x555c36ddf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36ddf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 -- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 msgr2=0x555c36de6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36de6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.945+0000 7fcc06308640 1 -- 172.21.15.94:0/15754795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c36a4b3f0 con 0x555c36a126b0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcc06308640 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc031500 tx=0x7fcbfc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55799736ed88392c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc035070 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcbfc003660 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc040070 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc003860 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcc0f417740 10 client.23826 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c36b157d0 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.946+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc040070 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcbfc045840 con 0x555c36a126b0 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 10 client.23826 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 10 client.23826 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 20 client.23826 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369f1f60 0x555c36dece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 4 client.23826 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c36d47990 con 0x555c369f1f60 2024-04-25T14:05:19.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.947+0000 7fcc05b07640 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369f1f60 0x555c36dece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcbe9ffb640 10 client.23826 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc05b07640 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369f1f60 0x555c36dece50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dec61913c41bf0c8 server_cookie=29563e0694abef14 in_seq=0 out_seq=0 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/15754795 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c36d47990 con 0x555c369f1f60 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcbe9ffb640 10 client.23826 handle_command_reply: tid=0 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 client.23826 shutdown 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 msgr2=0x555c36ddfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc031500 tx=0x7fcbfc02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 shutdown_connections 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369f1f60 0x555c36dece50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36de6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a126b0 0x555c36ddfaa0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 --2- 172.21.15.94:0/15754795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36ddf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcbe9ffb640 0 client.23826 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:19.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.948+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 >> 172.21.15.94:0/15754795 conn(0x555c36a274e0 msgr2=0x555c36dd3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.949+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 shutdown_connections 2024-04-25T14:05:19.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.949+0000 7fcc0f417740 1 -- 172.21.15.94:0/15754795 wait complete. 2024-04-25T14:05:19.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.949+0000 7fcc0f417740 20 client.23826 trim_cache size 0 max 16384 2024-04-25T14:05:19.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 Processor -- start 2024-04-25T14:05:19.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 -- start start 2024-04-25T14:05:19.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a126b0 0x555c36dd5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36ddf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ab5e0 con 0x555c36a0a4d0 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ad120 con 0x555c36a33390 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc0f417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001abcb0 con 0x555c36a126b0 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc05b07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc05b07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34348/0 (socket says 172.21.15.94:34348) 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc05b07640 1 -- 172.21.15.94:0/2949820450 learned_addr learned my addr 172.21.15.94:0/2949820450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.966+0000 7fcc06308640 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a126b0 0x555c36dd5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36ddf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 -- 172.21.15.94:0/2949820450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a126b0 msgr2=0x555c36dd5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a126b0 0x555c36dd5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 -- 172.21.15.94:0/2949820450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 msgr2=0x555c36ddf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36ddf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 -- 172.21.15.94:0/2949820450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c36ddaaa0 con 0x555c36a0a4d0 2024-04-25T14:05:19.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc837fb648612368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcbebfff640 1 -- 172.21.15.94:0/2949820450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0011b00 con 0x555c36a0a4d0 2024-04-25T14:05:19.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcbebfff640 1 -- 172.21.15.94:0/2949820450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0011ca0 con 0x555c36a0a4d0 2024-04-25T14:05:19.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.967+0000 7fcbebfff640 1 -- 172.21.15.94:0/2949820450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0004c30 con 0x555c36a0a4d0 2024-04-25T14:05:19.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 -- 172.21.15.94:0/2949820450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 msgr2=0x555c36dd5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0009ae0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 -- 172.21.15.94:0/2949820450 shutdown_connections 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a33390 0x555c36ddf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a126b0 0x555c36dd5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2949820450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a0a4d0 0x555c36dd5790 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 -- 172.21.15.94:0/2949820450 >> 172.21.15.94:0/2949820450 conn(0x555c36a274e0 msgr2=0x555c36dda390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 -- 172.21.15.94:0/2949820450 shutdown_connections 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.968+0000 7fcc0f417740 1 -- 172.21.15.94:0/2949820450 wait complete. 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 -- start start 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 Processor -- start 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36de2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36de3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 0x555c36e53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001aca50 con 0x555c36a33390 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001ac380 con 0x555c36a126b0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001a95c0 con 0x555c36a0a4d0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.969+0000 7fcc0f417740 10 client.0 _ll_register_callbacks cb 0x7fcbf0009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc06308640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36de3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36de2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc06308640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36de3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33418/0 (socket says 172.21.15.94:33418) 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc06308640 1 -- 172.21.15.94:0/2208766948 learned_addr learned my addr 172.21.15.94:0/2208766948 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 0x555c36e53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 -- 172.21.15.94:0/2208766948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 msgr2=0x555c36de2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36de2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 -- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 msgr2=0x555c36de3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36de3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 -- 172.21.15.94:0/2208766948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c36de62e0 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc05b07640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 0x555c36e53170 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005eb0 tx=0x7fcbf0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0ae8149c2eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0011b00 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcbf00035f0 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.970+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf0011ca0 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbf00037f0 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 10 client.23832 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c36e54640 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0011ca0 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcbf001d030 con 0x555c36a33390 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 10 client.23832 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 10 client.23832 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:05:19.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 20 client.23832 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369c73a0 0x555c36e56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 4 client.23832 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c36d41800 con 0x555c369c73a0 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.971+0000 7fcc06308640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369c73a0 0x555c36e56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.972+0000 7fcbe9ffb640 10 client.23832 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.972+0000 7fcc06308640 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369c73a0 0x555c36e56fa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3aa9448692ff61d9 server_cookie=2601dffdf22ae71 in_seq=0 out_seq=0 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.972+0000 7fcbe9ffb640 1 -- 172.21.15.94:0/2208766948 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c36d41800 con 0x555c369c73a0 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.972+0000 7fcbe9ffb640 10 client.23832 handle_command_reply: tid=0 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 client.23832 shutdown 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 msgr2=0x555c36e53170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 0x555c36e53170 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0005eb0 tx=0x7fcbf0005f90 comp rx=0 tx=0).stop 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 shutdown_connections 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x555c369c73a0 0x555c36e56fa0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c36a33390 0x555c36e53170 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c36a126b0 0x555c36de3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 --2- 172.21.15.94:0/2208766948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c36a0a4d0 0x555c36de2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcbe9ffb640 0 client.23832 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:19.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 >> 172.21.15.94:0/2208766948 conn(0x555c36a274e0 msgr2=0x555c36dda390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.973+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 shutdown_connections 2024-04-25T14:05:19.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0f417740 1 -- 172.21.15.94:0/2208766948 wait complete. 2024-04-25T14:05:19.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0f417740 20 client.23832 trim_cache size 0 max 16384 2024-04-25T14:05:19.979 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 msgr2=0x7fcbec0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 0x7fcbec0b2080 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80dcc80 tx=0x7fcbf8016f90 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 msgr2=0x7fcc001a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.974+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800eb10 tx=0x7fcbf800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 shutdown_connections 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcbec0afb70 0x7fcbec0b2080 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc00109a30 0x7fcc001a1720 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc00104d30 0x7fcc0019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 --2- 172.21.15.94:0/3081397246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc00103af0 0x7fcc0019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 >> 172.21.15.94:0/3081397246 conn(0x7fcc000fece0 msgr2=0x7fcc00105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 shutdown_connections 2024-04-25T14:05:19.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:19.975+0000 7fcc0856c640 1 -- 172.21.15.94:0/3081397246 wait complete. 2024-04-25T14:05:19.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 -- 172.21.15.94:0/2456311144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8109a20 msgr2=0x7fadf810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 --2- 172.21.15.94:0/2456311144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8109a20 0x7fadf810be40 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009ca0 tx=0x7fadf002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 -- 172.21.15.94:0/2456311144 shutdown_connections 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 --2- 172.21.15.94:0/2456311144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8109a20 0x7fadf810be40 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 --2- 172.21.15.94:0/2456311144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8104d20 0x7fadf8105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 --2- 172.21.15.94:0/2456311144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf81047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 -- 172.21.15.94:0/2456311144 >> 172.21.15.94:0/2456311144 conn(0x7fadf80fecd0 msgr2=0x7fadf8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.113+0000 7fadfed6c640 1 -- 172.21.15.94:0/2456311144 shutdown_connections 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 -- 172.21.15.94:0/2456311144 wait complete. 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 Processor -- start 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 -- start start 2024-04-25T14:05:20.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 0x7fadf8074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8109a20 0x7fadf8075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810f5f0 con 0x7fadf8104d20 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810f470 con 0x7fadf8103ae0 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.114+0000 7fadfed6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810f770 con 0x7fadf8109a20 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33436/0 (socket says 172.21.15.94:33436) 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 -- 172.21.15.94:0/1755175837 learned_addr learned my addr 172.21.15.94:0/1755175837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfd309640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8109a20 0x7fadf8075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 0x7fadf8074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 -- 172.21.15.94:0/1755175837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8109a20 msgr2=0x7fadf8075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8109a20 0x7fadf8075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 -- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 msgr2=0x7fadf8074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 0x7fadf8074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 -- 172.21.15.94:0/1755175837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf8075b60 con 0x7fadf8103ae0 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 0x7fadf8074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9f0 tx=0x7fade800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadf5ffb640 1 -- 172.21.15.94:0/1755175837 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800ccc0 con 0x7fadf8103ae0 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf81a4250 con 0x7fadf8103ae0 2024-04-25T14:05:20.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.115+0000 7fadf5ffb640 1 -- 172.21.15.94:0/1755175837 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade80035f0 con 0x7fadf8103ae0 2024-04-25T14:05:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.116+0000 7fadf5ffb640 1 -- 172.21.15.94:0/1755175837 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800ce60 con 0x7fadf8103ae0 2024-04-25T14:05:20.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.116+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a4740 con 0x7fadf8103ae0 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fadf5ffb640 1 -- 172.21.15.94:0/1755175837 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fade8004020 con 0x7fadf8103ae0 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fae05c1b740 1 Processor -- start 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fae05c1b740 1 -- start start 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 0x556b412e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.118+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fae05c1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b41246d80 con 0x556b40f40390 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fae05c1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b41200a20 con 0x556b40f174d0 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fae05c1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b41242d40 con 0x556b40f1f6b0 2024-04-25T14:05:20.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadf5ffb640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 0x7fade00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 0x556b412e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34374/0 (socket says 172.21.15.94:34374) 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 -- 172.21.15.94:0/1095526392 learned_addr learned my addr 172.21.15.94:0/1095526392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 0x7fade00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadf5ffb640 1 -- 172.21.15.94:0/1755175837 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fade80deff0 con 0x7fadf8103ae0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 -- 172.21.15.94:0/1095526392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 msgr2=0x556b412e3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412e3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 -- 172.21.15.94:0/1095526392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 msgr2=0x556b412e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 -- 172.21.15.94:0/1095526392 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4120bc30 con 0x556b40f174d0 2024-04-25T14:05:20.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfd309640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 0x7fade00b2000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x556b412e3860 tx=0x7fadf00095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 0x556b412e2d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fade8003fd0 tx=0x7fade8075f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2bfce6d7216cac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.119+0000 7faddeffd640 1 -- 172.21.15.94:0/1095526392 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80a6ee0 con 0x556b40f174d0 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7faddeffd640 1 -- 172.21.15.94:0/1095526392 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade80a7ad0 con 0x556b40f174d0 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7faddeffd640 1 -- 172.21.15.94:0/1095526392 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80a3660 con 0x556b40f174d0 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 -- 172.21.15.94:0/1095526392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 msgr2=0x556b412e2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 0x556b412e2d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fade8003fd0 tx=0x7fade8075f20 comp rx=0 tx=0).stop 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 -- 172.21.15.94:0/1095526392 shutdown_connections 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f174d0 0x556b412e2d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1095526392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.120+0000 7fae05c1b740 1 -- 172.21.15.94:0/1095526392 >> 172.21.15.94:0/1095526392 conn(0x556b40f344e0 msgr2=0x556b412de020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- 172.21.15.94:0/1095526392 shutdown_connections 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- 172.21.15.94:0/1095526392 wait complete. 2024-04-25T14:05:20.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- start start 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 Processor -- start 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 0x556b412ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b412e44d0 con 0x556b40f174d0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556b411cddc0 con 0x556b40f40390 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b412e4650 con 0x556b40f1f6b0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.121+0000 7fae05c1b740 10 client.0 _ll_register_callbacks cb 0x556b411e9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 0x556b412ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadfd309640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadfd309640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33448/0 (socket says 172.21.15.94:33448) 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 0x556b412ec5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34382/0 (socket says 172.21.15.94:34382) 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadfd309640 1 -- 172.21.15.94:0/3532002763 learned_addr learned my addr 172.21.15.94:0/3532002763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40902/0 (socket says 172.21.15.94:40902) 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 -- 172.21.15.94:0/3532002763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 msgr2=0x556b412f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 -- 172.21.15.94:0/3532002763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 msgr2=0x556b412ec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 0x556b412ec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 -- 172.21.15.94:0/3532002763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b40f583f0 con 0x556b40f1f6b0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadf7fff640 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadf80761f0 tx=0x7fadec011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8440f95c7a64bd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec010c30 con 0x556b40f1f6b0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fadec010dd0 con 0x556b40f1f6b0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.122+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade80a7ad0 con 0x556b40f1f6b0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fadec004020 con 0x556b40f1f6b0 2024-04-25T14:05:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 10 client.32866 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b410227d0 con 0x556b40f1f6b0 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade80a7ad0 con 0x556b40f1f6b0 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fadec0048c0 con 0x556b40f1f6b0 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 10 client.32866 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 10 client.32866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 20 client.32866 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40efef60 0x556b412fa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 4 client.32866 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b41254990 con 0x556b40efef60 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.123+0000 7fadfcb08640 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40efef60 0x556b412fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.124+0000 7fadd7fff640 10 client.32866 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.124+0000 7fadfcb08640 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40efef60 0x556b412fa380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6722fe38b85fa2f server_cookie=82e0c8f6dbd28c8b in_seq=0 out_seq=0 2024-04-25T14:05:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fadd7fff640 1 -- 172.21.15.94:0/3532002763 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b41254990 con 0x556b40efef60 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fadd7fff640 10 client.32866 handle_command_reply: tid=0 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 client.32866 shutdown 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 msgr2=0x556b412ecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadf80761f0 tx=0x7fadec011c90 comp rx=0 tx=0).stop 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 shutdown_connections 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40efef60 0x556b412fa380 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f1f6b0 0x556b412ecae0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fadd7fff640 0 client.32866 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 --2- 172.21.15.94:0/3532002763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f174d0 0x556b412ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 >> 172.21.15.94:0/3532002763 conn(0x556b40f344e0 msgr2=0x556b412e0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 shutdown_connections 2024-04-25T14:05:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 1 -- 172.21.15.94:0/3532002763 wait complete. 2024-04-25T14:05:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.125+0000 7fae05c1b740 20 client.32866 trim_cache size 0 max 16384 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 Processor -- start 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 -- start start 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 0x556b412f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81ad6c0 con 0x556b40f40390 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81af200 con 0x556b40f1f6b0 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fae05c1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81add90 con 0x556b40f174d0 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfcb08640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34388/0 (socket says 172.21.15.94:34388) 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 0x556b412f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 -- 172.21.15.94:0/374326514 learned_addr learned my addr 172.21.15.94:0/374326514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 -- 172.21.15.94:0/374326514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 msgr2=0x556b412de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 -- 172.21.15.94:0/374326514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 msgr2=0x556b412f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 0x556b412f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfd309640 1 -- 172.21.15.94:0/374326514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b412e3490 con 0x556b40f40390 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadf7fff640 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 0x556b412f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.140+0000 7fadfcb08640 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.141+0000 7fadfd309640 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fadf00315a0 tx=0x7fadf003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f762d94817c676b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.141+0000 7fadde7fc640 1 -- 172.21.15.94:0/374326514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0004040 con 0x556b40f40390 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.141+0000 7fadde7fc640 1 -- 172.21.15.94:0/374326514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf0053070 con 0x556b40f40390 2024-04-25T14:05:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.141+0000 7fadde7fc640 1 -- 172.21.15.94:0/374326514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf002fb80 con 0x556b40f40390 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 -- 172.21.15.94:0/374326514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 msgr2=0x556b412e0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fadf00315a0 tx=0x7fadf003b000 comp rx=0 tx=0).stop 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 -- 172.21.15.94:0/374326514 shutdown_connections 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f40390 0x556b412e0e20 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f1f6b0 0x556b412f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 --2- 172.21.15.94:0/374326514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 -- 172.21.15.94:0/374326514 >> 172.21.15.94:0/374326514 conn(0x556b40f344e0 msgr2=0x556b412dd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 -- 172.21.15.94:0/374326514 shutdown_connections 2024-04-25T14:05:20.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.142+0000 7fae05c1b740 1 -- 172.21.15.94:0/374326514 wait complete. 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 -- start start 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 Processor -- start 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412f1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81aeb30 con 0x556b40f1f6b0 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81ae460 con 0x556b40f40390 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81ab6a0 con 0x556b40f174d0 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fae05c1b740 10 client.0 _ll_register_callbacks cb 0x7fadec00bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34400/0 (socket says 172.21.15.94:34400) 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fadf7fff640 1 -- 172.21.15.94:0/1918708081 learned_addr learned my addr 172.21.15.94:0/1918708081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.143+0000 7fadfd309640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 -- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 msgr2=0x556b412f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412f1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 -- 172.21.15.94:0/1918708081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 msgr2=0x556b412f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 -- 172.21.15.94:0/1918708081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b412f3ea0 con 0x556b40f1f6b0 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadfd309640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadf7fff640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x556b412e2290 tx=0x7fadec005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0cfd4e6198372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec034060 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fadec0043d0 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf0053070 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.144+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fadec027070 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 10 client.23850 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b41362820 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf0053070 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fadec004f70 con 0x556b40f1f6b0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 10 client.23850 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 10 client.23850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 20 client.23850 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40ed43a0 0x556b41365270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 4 client.23850 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b4124e800 con 0x556b40ed43a0 2024-04-25T14:05:20.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.145+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40ed43a0 0x556b41365270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fadd7fff640 10 client.23850 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fadfcb08640 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40ed43a0 0x556b41365270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d77fdd7cb9537e8 server_cookie=452cb8bf617871c3 in_seq=0 out_seq=0 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fadd7fff640 1 -- 172.21.15.94:0/1918708081 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b4124e800 con 0x556b40ed43a0 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fadd7fff640 10 client.23850 handle_command_reply: tid=0 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fae05c1b740 1 client.23850 shutdown 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 msgr2=0x556b412f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.146+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x556b412e2290 tx=0x7fadec005c60 comp rx=0 tx=0).stop 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 shutdown_connections 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x556b40ed43a0 0x556b41365270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fadd7fff640 0 client.23850 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556b40f40390 0x556b412f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b40f1f6b0 0x556b412f19d0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 --2- 172.21.15.94:0/1918708081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b40f174d0 0x556b412f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 >> 172.21.15.94:0/1918708081 conn(0x556b40f344e0 msgr2=0x556b412dc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 shutdown_connections 2024-04-25T14:05:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 1 -- 172.21.15.94:0/1918708081 wait complete. 2024-04-25T14:05:20.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.147+0000 7fae05c1b740 20 client.23850 trim_cache size 0 max 16384 2024-04-25T14:05:20.153 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 msgr2=0x7fade00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 0x7fade00b2000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x556b412e3860 tx=0x7fadf00095c0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 msgr2=0x7fadf8076850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9f0 tx=0x7fade800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 shutdown_connections 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fade00afaf0 0x7fade00b2000 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadf8109a20 0x7fadf8075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf8104d20 0x7fadf8074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 --2- 172.21.15.94:0/1755175837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadf8103ae0 0x7fadf8076850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 >> 172.21.15.94:0/1755175837 conn(0x7fadf80fecd0 msgr2=0x7fadf8105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 shutdown_connections 2024-04-25T14:05:20.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.148+0000 7fadfed6c640 1 -- 172.21.15.94:0/1755175837 wait complete. 2024-04-25T14:05:20.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:19 smithi094 ceph-mon[29195]: pgmap v1057: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-04-25T14:05:20.158 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:19 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4072650384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:20.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:05:20.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 213082 18.84 MB/sec execute 482 sec latency 74.476 ms 2024-04-25T14:05:20.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:19 smithi138 ceph-mon[32977]: pgmap v1057: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-04-25T14:05:20.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:19 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4072650384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:20.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1773977086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84104d20 msgr2=0x7f7e84105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1773977086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84104d20 0x7f7e84105130 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009900 tx=0x7f7e7402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:20.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1773977086 shutdown_connections 2024-04-25T14:05:20.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1773977086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84109a20 0x7f7e8410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1773977086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84104d20 0x7f7e84105130 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1773977086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e841047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1773977086 >> 172.21.15.94:0/1773977086 conn(0x7f7e840fecd0 msgr2=0x7f7e84101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.280+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1773977086 shutdown_connections 2024-04-25T14:05:20.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.281+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1773977086 wait complete. 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.281+0000 7f7e8c0b1640 1 Processor -- start 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.281+0000 7f7e8c0b1640 1 -- start start 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84104d20 0x7f7e8419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 0x7f7e841a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8410f610 con 0x7f7e84109a20 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8410f490 con 0x7f7e84103ae0 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8c0b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8410f790 con 0x7f7e84104d20 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 0x7f7e841a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33488/0 (socket says 172.21.15.94:33488) 2024-04-25T14:05:20.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 -- 172.21.15.94:0/1894013601 learned_addr learned my addr 172.21.15.94:0/1894013601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 -- 172.21.15.94:0/1894013601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84104d20 msgr2=0x7f7e8419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8964c640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84104d20 0x7f7e8419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84104d20 0x7f7e8419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 -- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 msgr2=0x7f7e841a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 0x7f7e841a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e89e4d640 1 -- 172.21.15.94:0/1894013601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841a1d30 con 0x7f7e84103ae0 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.282+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 0x7f7e841a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.283+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e980 tx=0x7f7e8000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.283+0000 7f7e7affd640 1 -- 172.21.15.94:0/1894013601 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000cd80 con 0x7f7e84103ae0 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.283+0000 7f7e7affd640 1 -- 172.21.15.94:0/1894013601 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e800108e0 con 0x7f7e84103ae0 2024-04-25T14:05:20.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.283+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841a1fc0 con 0x7f7e84103ae0 2024-04-25T14:05:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.284+0000 7f7e7affd640 1 -- 172.21.15.94:0/1894013601 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80018990 con 0x7f7e84103ae0 2024-04-25T14:05:20.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.284+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a2480 con 0x7f7e84103ae0 2024-04-25T14:05:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.286+0000 7f7e7affd640 1 -- 172.21.15.94:0/1894013601 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7e8001f020 con 0x7f7e84103ae0 2024-04-25T14:05:20.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.287+0000 7f7e9303a740 1 Processor -- start 2024-04-25T14:05:20.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.287+0000 7f7e9303a740 1 -- start start 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.287+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.287+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e7affd640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 0x7f7e700b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e9303a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5638b0007d80 con 0x5638afcd84d0 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e9303a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5638affc1a20 con 0x5638afd01390 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e9303a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5638b0003d40 con 0x5638afce06b0 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33502/0 (socket says 172.21.15.94:33502) 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 -- 172.21.15.94:0/496832903 learned_addr learned my addr 172.21.15.94:0/496832903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 0x7f7e700b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e7affd640 1 -- 172.21.15.94:0/1894013601 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7e800def00 con 0x7f7e84103ae0 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 -- 172.21.15.94:0/496832903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 msgr2=0x5638b00a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8964c640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 -- 172.21.15.94:0/496832903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 msgr2=0x5638b00a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8a64e640 1 -- 172.21.15.94:0/496832903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638affccc30 con 0x5638afd01390 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.288+0000 7f7e8964c640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.289+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c011fd0 tx=0x7f7e7c00fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7801f3cdb2c930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.289+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 0x7f7e700b6410 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ff50 tx=0x7f7e7c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.289+0000 7f7e6ffff640 1 -- 172.21.15.94:0/496832903 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c02b070 con 0x5638afd01390 2024-04-25T14:05:20.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.289+0000 7f7e6ffff640 1 -- 172.21.15.94:0/496832903 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c011630 con 0x5638afd01390 2024-04-25T14:05:20.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.289+0000 7f7e6ffff640 1 -- 172.21.15.94:0/496832903 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c022070 con 0x5638afd01390 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 -- 172.21.15.94:0/496832903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 msgr2=0x5638b00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c011fd0 tx=0x7f7e7c00fa50 comp rx=0 tx=0).stop 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 -- 172.21.15.94:0/496832903 shutdown_connections 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 --2- 172.21.15.94:0/496832903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afd01390 0x5638b00a3810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 -- 172.21.15.94:0/496832903 >> 172.21.15.94:0/496832903 conn(0x5638afcf54e0 msgr2=0x5638b009f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 -- 172.21.15.94:0/496832903 shutdown_connections 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.290+0000 7f7e9303a740 1 -- 172.21.15.94:0/496832903 wait complete. 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 -- start start 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 Processor -- start 2024-04-25T14:05:20.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afd01390 0x5638b00b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5638b00a54d0 con 0x5638afcd84d0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5638aff8edc0 con 0x5638afce06b0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5638b00a5650 con 0x5638afd01390 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e9303a740 10 client.0 _ll_register_callbacks cb 0x5638affaa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e89e4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.291+0000 7f7e8964c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afd01390 0x5638b00b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e89e4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34430/0 (socket says 172.21.15.94:34430) 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33516/0 (socket says 172.21.15.94:33516) 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e89e4d640 1 -- 172.21.15.94:0/2560563881 learned_addr learned my addr 172.21.15.94:0/2560563881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 -- 172.21.15.94:0/2560563881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afd01390 msgr2=0x5638b00b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afd01390 0x5638b00b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 -- 172.21.15.94:0/2560563881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 msgr2=0x5638b00ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 -- 172.21.15.94:0/2560563881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638afd193f0 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c011630 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e8964c640 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5638b009fdb0 tx=0x7f7e740095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5ffff0c9b90ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e6d7fa640 10 client.33242 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/2560563881 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74035030 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/2560563881 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7e74004040 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.292+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/2560563881 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e74004280 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 10 client.33242 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5638afde37d0 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/2560563881 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7e7403a040 con 0x5638afce06b0 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 10 client.33242 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 10 client.33242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 20 client.33242 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afcbff60 0x5638b00bb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 4 client.33242 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b0015990 con 0x5638afcbff60 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afcbff60 0x5638b00bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e6d7fa640 10 client.33242 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.293+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afcbff60 0x5638b00bb060 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd0f7fef94f59a7f server_cookie=36f6322dded8c6f9 in_seq=0 out_seq=0 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/2560563881 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638b0015990 con 0x5638afcbff60 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e6d7fa640 10 client.33242 handle_command_reply: tid=0 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e9303a740 1 client.33242 shutdown 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 msgr2=0x5638b00ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5638b009fdb0 tx=0x7f7e740095e0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 shutdown_connections 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.294+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afcbff60 0x5638b00bb060 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afd01390 0x5638b00b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afce06b0 0x5638b00ada70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2560563881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e6d7fa640 0 client.33242 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 >> 172.21.15.94:0/2560563881 conn(0x5638afcf54e0 msgr2=0x5638b009e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 shutdown_connections 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 1 -- 172.21.15.94:0/2560563881 wait complete. 2024-04-25T14:05:20.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.295+0000 7f7e9303a740 20 client.33242 trim_cache size 0 max 16384 2024-04-25T14:05:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 Processor -- start 2024-04-25T14:05:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 -- start start 2024-04-25T14:05:20.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 0x5638b00a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841a92a0 con 0x5638afcd84d0 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841aade0 con 0x5638afcbff60 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.309+0000 7f7e9303a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841a9970 con 0x5638afce06b0 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e8964c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 0x5638b00a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e8964c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34440/0 (socket says 172.21.15.94:34440) 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e8964c640 1 -- 172.21.15.94:0/2130972023 learned_addr learned my addr 172.21.15.94:0/2130972023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 -- 172.21.15.94:0/2130972023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 msgr2=0x5638b00a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 -- 172.21.15.94:0/2130972023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 msgr2=0x5638b00b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 -- 172.21.15.94:0/2130972023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b00a4430 con 0x5638afcbff60 2024-04-25T14:05:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 0x5638b00a82b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7e800a72d0 tx=0x7f7e800dc830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f427d3caf70f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2130972023 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800b01a0 con 0x5638afcbff60 2024-04-25T14:05:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2130972023 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e800dca70 con 0x5638afcbff60 2024-04-25T14:05:20.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.310+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/2130972023 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800a35d0 con 0x5638afcbff60 2024-04-25T14:05:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- 172.21.15.94:0/2130972023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 msgr2=0x5638b00a82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 0x5638b00a82b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7e800a72d0 tx=0x7f7e800dc830 comp rx=0 tx=0).stop 2024-04-25T14:05:20.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- 172.21.15.94:0/2130972023 shutdown_connections 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afce06b0 0x5638b00a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afcd84d0 0x5638b00b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 --2- 172.21.15.94:0/2130972023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcbff60 0x5638b00a82b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- 172.21.15.94:0/2130972023 >> 172.21.15.94:0/2130972023 conn(0x5638afcf54e0 msgr2=0x5638b00ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- 172.21.15.94:0/2130972023 shutdown_connections 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- 172.21.15.94:0/2130972023 wait complete. 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.311+0000 7f7e9303a740 1 -- start start 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 Processor -- start 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afcbff60 0x5638b00b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 0x5638b00b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841aa710 con 0x5638afce06b0 2024-04-25T14:05:20.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841aa040 con 0x5638afcd84d0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841a7280 con 0x5638afcbff60 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e9303a740 10 client.0 _ll_register_callbacks cb 0x7f7e80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34456/0 (socket says 172.21.15.94:34456) 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8964c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 0x5638b00b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 -- 172.21.15.94:0/3256242596 learned_addr learned my addr 172.21.15.94:0/3256242596 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afcbff60 0x5638b00b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 -- 172.21.15.94:0/3256242596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afcbff60 msgr2=0x5638b00b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afcbff60 0x5638b00b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 -- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 msgr2=0x5638b00b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 0x5638b00b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8a64e640 1 -- 172.21.15.94:0/3256242596 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b00b2300 con 0x5638afce06b0 2024-04-25T14:05:20.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.312+0000 7f7e8964c640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 0x5638b00b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e8a64e640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005c40 tx=0x7f7e7c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ccdba3b99f2cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c034020 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7e7c0050b0 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e800dca70 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c03f050 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 10 client.23868 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5638b01229d0 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e800dca70 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7e7c0266b0 con 0x5638afce06b0 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 10 client.23868 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 10 client.23868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 20 client.23868 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afd01390 0x5638b01251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 4 client.23868 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.313+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638b000f800 con 0x5638afd01390 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afd01390 0x5638b01251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e6d7fa640 10 client.23868 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e89e4d640 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afd01390 0x5638b01251e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4095e40e5f085ee4 server_cookie=e988a1a6ec3af0f5 in_seq=0 out_seq=0 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e6d7fa640 1 -- 172.21.15.94:0/3256242596 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638b000f800 con 0x5638afd01390 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e6d7fa640 10 client.23868 handle_command_reply: tid=0 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 client.23868 shutdown 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 msgr2=0x5638b01213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005c40 tx=0x7f7e7c011850 comp rx=0 tx=0).stop 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 shutdown_connections 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5638afd01390 0x5638b01251e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638afce06b0 0x5638b01213c0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638afcd84d0 0x5638b00b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 --2- 172.21.15.94:0/3256242596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638afcbff60 0x5638b00b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e6d7fa640 0 client.23868 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.314+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 >> 172.21.15.94:0/3256242596 conn(0x5638afcf54e0 msgr2=0x5638b00ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 shutdown_connections 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e9303a740 1 -- 172.21.15.94:0/3256242596 wait complete. 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e9303a740 20 client.23868 trim_cache size 0 max 16384 2024-04-25T14:05:20.321 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 msgr2=0x7f7e700b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 0x7f7e700b6410 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00ff50 tx=0x7f7e7c01e000 comp rx=0 tx=0).stop 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 msgr2=0x7f7e8419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000e980 tx=0x7f7e8000ee40 comp rx=0 tx=0).stop 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 shutdown_connections 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e700b3f00 0x7f7e700b6410 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e84109a20 0x7f7e841a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e84104d20 0x7f7e8419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 --2- 172.21.15.94:0/1894013601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e84103ae0 0x7f7e8419a1f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 >> 172.21.15.94:0/1894013601 conn(0x7f7e840fecd0 msgr2=0x7f7e84105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 shutdown_connections 2024-04-25T14:05:20.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.315+0000 7f7e8c0b1640 1 -- 172.21.15.94:0/1894013601 wait complete. 2024-04-25T14:05:20.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:05:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:19 smithi137 ceph-mon[32568]: pgmap v1057: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-04-25T14:05:20.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:19 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4072650384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 -- 172.21.15.94:0/1815203178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 msgr2=0x7feac4109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 --2- 172.21.15.94:0/1815203178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac4109110 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7feab8009900 tx=0x7feab802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 -- 172.21.15.94:0/1815203178 shutdown_connections 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 --2- 172.21.15.94:0/1815203178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac41097a0 0x7feac410bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 --2- 172.21.15.94:0/1815203178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac4109110 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 --2- 172.21.15.94:0/1815203178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac4104240 0x7feac4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 -- 172.21.15.94:0/1815203178 >> 172.21.15.94:0/1815203178 conn(0x7feac40fdc00 msgr2=0x7feac4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 -- 172.21.15.94:0/1815203178 shutdown_connections 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.453+0000 7feaca340640 1 -- 172.21.15.94:0/1815203178 wait complete. 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 Processor -- start 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 -- start start 2024-04-25T14:05:20.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 0x7feac419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 0x7feac41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feac410f5e0 con 0x7feac4106c30 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feac410f460 con 0x7feac41097a0 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feaca340640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac410f760 con 0x7feac4104240 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feac37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feac37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34474/0 (socket says 172.21.15.94:34474) 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feac37fe640 1 -- 172.21.15.94:0/1397856389 learned_addr learned my addr 172.21.15.94:0/1397856389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feac88dd640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 0x7feac41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.454+0000 7feac3fff640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 0x7feac419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 -- 172.21.15.94:0/1397856389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 msgr2=0x7feac41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 0x7feac41a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 -- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 msgr2=0x7feac419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 -- 172.21.15.94:0/1397856389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41a1d60 con 0x7feac4104240 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac88dd640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 0x7feac41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac37fe640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac3fff640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 0x7feac419a220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9f0 tx=0x7feab400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac17fa640 1 -- 172.21.15.94:0/1397856389 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400ccc0 con 0x7feac4104240 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac4102090 con 0x7feac4104240 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac17fa640 1 -- 172.21.15.94:0/1397856389 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab40108e0 con 0x7feac4104240 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feac17fa640 1 -- 172.21.15.94:0/1397856389 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4018880 con 0x7feac4104240 2024-04-25T14:05:20.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.455+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac4102630 con 0x7feac4104240 2024-04-25T14:05:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac17fa640 1 -- 172.21.15.94:0/1397856389 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7feab4018a20 con 0x7feac4104240 2024-04-25T14:05:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 Processor -- start 2024-04-25T14:05:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 -- start start 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561445ed3d80 con 0x561445ba44d0 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561445e8da20 con 0x561445bac6b0 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7fead11f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561445ecfd40 con 0x561445bcd390 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34480/0 (socket says 172.21.15.94:34480) 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac3fff640 1 -- 172.21.15.94:0/352414513 learned_addr learned my addr 172.21.15.94:0/352414513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac37fe640 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.458+0000 7feac88dd640 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 -- 172.21.15.94:0/352414513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 msgr2=0x561445f6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac17fa640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 0x7feaac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 -- 172.21.15.94:0/352414513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 msgr2=0x561445f70a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f70a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 -- 172.21.15.94:0/352414513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561445e98c30 con 0x561445ba44d0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac88dd640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 0x7feaac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac17fa640 1 -- 172.21.15.94:0/1397856389 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7feab40df000 con 0x7feac4104240 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feac3fff640 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7feab40d17b0 tx=0x7feab40a3c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad589a6472cca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feaaa7fc640 1 -- 172.21.15.94:0/352414513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4026030 con 0x561445ba44d0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feaaa7fc640 1 -- 172.21.15.94:0/352414513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab40a7000 con 0x561445ba44d0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.459+0000 7feaaa7fc640 1 -- 172.21.15.94:0/352414513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab40a6c60 con 0x561445ba44d0 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 -- 172.21.15.94:0/352414513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 msgr2=0x561445f6fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7feac88dd640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 0x7feaac0b2000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feabc0049d0 tx=0x7feabc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7feab40d17b0 tx=0x7feab40a3c00 comp rx=0 tx=0).stop 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 -- 172.21.15.94:0/352414513 shutdown_connections 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f6fd90 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 --2- 172.21.15.94:0/352414513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.460+0000 7fead11f1740 1 -- 172.21.15.94:0/352414513 >> 172.21.15.94:0/352414513 conn(0x561445bc14e0 msgr2=0x561445f6b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- 172.21.15.94:0/352414513 shutdown_connections 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- 172.21.15.94:0/352414513 wait complete. 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- start start 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 Processor -- start 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f79560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f80a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561445f714d0 con 0x561445bac6b0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561445e5adc0 con 0x561445ba44d0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561445f71650 con 0x561445bcd390 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.461+0000 7fead11f1740 10 client.0 _ll_register_callbacks cb 0x561445e76600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34488/0 (socket says 172.21.15.94:34488) 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 -- 172.21.15.94:0/3981167127 learned_addr learned my addr 172.21.15.94:0/3981167127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac3fff640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f79560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac88dd640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f80a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 -- 172.21.15.94:0/3981167127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 msgr2=0x561445f80a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f80a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 -- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 msgr2=0x561445f79560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f79560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 -- 172.21.15.94:0/3981167127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561445be53f0 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac3fff640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f79560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feac37fe640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7feab8000c00 tx=0x7feab80317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4694855ce72e72e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feaa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8035050 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7feab8004450 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab40a7000 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.462+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feab80035f0 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 10 client.23880 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561445caf7d0 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab40a7000 con 0x561445bac6b0 2024-04-25T14:05:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7feab8043070 con 0x561445bac6b0 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 10 client.23880 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 10 client.23880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 20 client.23880 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b8bf60 0x561445f87240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 4 client.23880 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561445ee1990 con 0x561445b8bf60 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.463+0000 7feac3fff640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b8bf60 0x561445f87240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.464+0000 7feaa37fe640 10 client.23880 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:20.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.464+0000 7feac3fff640 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b8bf60 0x561445f87240 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f675a8c174dda7 server_cookie=cf9f6d7a991efd in_seq=0 out_seq=0 2024-04-25T14:05:20.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7feaa37fe640 1 -- 172.21.15.94:0/3981167127 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561445ee1990 con 0x561445b8bf60 2024-04-25T14:05:20.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7feaa37fe640 10 client.23880 handle_command_reply: tid=0 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 client.23880 shutdown 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 msgr2=0x561445f79aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7feab8000c00 tx=0x7feab80317d0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 shutdown_connections 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b8bf60 0x561445f87240 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f80a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bac6b0 0x561445f79aa0 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7feaa37fe640 0 client.23880 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 --2- 172.21.15.94:0/3981167127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f79560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 >> 172.21.15.94:0/3981167127 conn(0x561445bc14e0 msgr2=0x561445f6df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 shutdown_connections 2024-04-25T14:05:20.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 1 -- 172.21.15.94:0/3981167127 wait complete. 2024-04-25T14:05:20.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.466+0000 7fead11f1740 20 client.23880 trim_cache size 0 max 16384 2024-04-25T14:05:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 Processor -- start 2024-04-25T14:05:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 -- start start 2024-04-25T14:05:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f80920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ad750 con 0x561445ba44d0 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feac41af290 con 0x561445bac6b0 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7fead11f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ade20 con 0x561445bcd390 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7feac3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.481+0000 7feac88dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34500/0 (socket says 172.21.15.94:34500) 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41008/0 (socket says 172.21.15.94:41008) 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac3fff640 1 -- 172.21.15.94:0/1148314917 learned_addr learned my addr 172.21.15.94:0/1148314917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac37fe640 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f80920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 -- 172.21.15.94:0/1148314917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 msgr2=0x561445f80920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f80920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 -- 172.21.15.94:0/1148314917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 msgr2=0x561445f783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 -- 172.21.15.94:0/1148314917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561445f80e60 con 0x561445bcd390 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac3fff640 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feac88dd640 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6c2ecd5339712e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feaa9ffb640 1 -- 172.21.15.94:0/1148314917 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004b20 con 0x561445bcd390 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feaa9ffb640 1 -- 172.21.15.94:0/1148314917 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc004cc0 con 0x561445bcd390 2024-04-25T14:05:20.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.482+0000 7feaa9ffb640 1 -- 172.21.15.94:0/1148314917 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc005310 con 0x561445bcd390 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- 172.21.15.94:0/1148314917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 msgr2=0x561445f6a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc019000 comp rx=0 tx=0).stop 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- 172.21.15.94:0/1148314917 shutdown_connections 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bcd390 0x561445f6a7c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445bac6b0 0x561445f80920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 --2- 172.21.15.94:0/1148314917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445ba44d0 0x561445f783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- 172.21.15.94:0/1148314917 >> 172.21.15.94:0/1148314917 conn(0x561445bc14e0 msgr2=0x561445f77cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- 172.21.15.94:0/1148314917 shutdown_connections 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- 172.21.15.94:0/1148314917 wait complete. 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.483+0000 7fead11f1740 1 -- start start 2024-04-25T14:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 Processor -- start 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bac6b0 0x561445f7e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feac41aebc0 con 0x561445bcd390 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ae4f0 con 0x561445ba44d0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feac41ab730 con 0x561445bac6b0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7fead11f1740 10 client.0 _ll_register_callbacks cb 0x7feabc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34512/0 (socket says 172.21.15.94:34512) 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 -- 172.21.15.94:0/2856018389 learned_addr learned my addr 172.21.15.94:0/2856018389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac37fe640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bac6b0 0x561445f7e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 -- 172.21.15.94:0/2856018389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bac6b0 msgr2=0x561445f7e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bac6b0 0x561445f7e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 -- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 msgr2=0x561445f7dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f7dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.484+0000 7feac88dd640 1 -- 172.21.15.94:0/2856018389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561445f7f700 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feac88dd640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7feabc009540 tx=0x7feabc010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f894e4040bdbd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004e70 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc004cc0 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7feabc032480 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feabc032680 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 10 client.23886 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561445f809d0 con 0x561445bcd390 2024-04-25T14:05:20.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feabc004cc0 con 0x561445bcd390 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7feabc01a070 con 0x561445bcd390 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 10 client.23886 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 10 client.23886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 20 client.23886 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b613a0 0x561445ff1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 4 client.23886 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.485+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561445edb800 con 0x561445b613a0 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.486+0000 7feac3fff640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b613a0 0x561445ff1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.486+0000 7feaa37fe640 10 client.23886 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:20.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.486+0000 7feac3fff640 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b613a0 0x561445ff1af0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13f189485d4e205 server_cookie=f6ff174a6eabaa87 in_seq=0 out_seq=0 2024-04-25T14:05:20.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7feaa37fe640 1 -- 172.21.15.94:0/2856018389 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561445edb800 con 0x561445b613a0 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7feaa37fe640 10 client.23886 handle_command_reply: tid=0 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 client.23886 shutdown 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 msgr2=0x561445f7f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7feabc009540 tx=0x7feabc010960 comp rx=0 tx=0).stop 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 shutdown_connections 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x561445b613a0 0x561445ff1af0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561445bcd390 0x561445f7f140 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7feaa37fe640 0 client.23886 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561445bac6b0 0x561445f7e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 --2- 172.21.15.94:0/2856018389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561445ba44d0 0x561445f7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 >> 172.21.15.94:0/2856018389 conn(0x561445bc14e0 msgr2=0x561445f6a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 shutdown_connections 2024-04-25T14:05:20.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 1 -- 172.21.15.94:0/2856018389 wait complete. 2024-04-25T14:05:20.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.489+0000 7fead11f1740 20 client.23886 trim_cache size 0 max 16384 2024-04-25T14:05:20.494 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 msgr2=0x7feaac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 0x7feaac0b2000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feabc0049d0 tx=0x7feabc004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 msgr2=0x7feac419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 0x7feac419a220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9f0 tx=0x7feab400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 shutdown_connections 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7feaac0afaf0 0x7feaac0b2000 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feac41097a0 0x7feac41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac4106c30 0x7feac419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 --2- 172.21.15.94:0/1397856389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feac4104240 0x7feac419a220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 >> 172.21.15.94:0/1397856389 conn(0x7feac40fdc00 msgr2=0x7feac4105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:20.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 shutdown_connections 2024-04-25T14:05:20.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:20.490+0000 7feaca340640 1 -- 172.21.15.94:0/1397856389 wait complete. 2024-04-25T14:05:21.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 213560 18.85 MB/sec execute 483 sec latency 82.323 ms 2024-04-25T14:05:21.502 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:05:21.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:21.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:05:21.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 -- 172.21.15.94:0/1857968801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc444102170 msgr2=0x7fc444102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 --2- 172.21.15.94:0/1857968801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc444102170 0x7fc444102e70 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fc434009900 tx=0x7fc43402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 -- 172.21.15.94:0/1857968801 shutdown_connections 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 --2- 172.21.15.94:0/1857968801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4441080b0 0x7fc44410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 --2- 172.21.15.94:0/1857968801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441033b0 0x7fc4441037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 --2- 172.21.15.94:0/1857968801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc444102170 0x7fc444102e70 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 -- 172.21.15.94:0/1857968801 >> 172.21.15.94:0/1857968801 conn(0x7fc4440fd340 msgr2=0x7fc4440ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 -- 172.21.15.94:0/1857968801 shutdown_connections 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.647+0000 7fc449e84640 1 -- 172.21.15.94:0/1857968801 wait complete. 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.648+0000 7fc449e84640 1 Processor -- start 2024-04-25T14:05:21.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.648+0000 7fc449e84640 1 -- start start 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.648+0000 7fc449e84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc449e84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 0x7fc44419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41030/0 (socket says 172.21.15.94:41030) 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc449e84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441080b0 0x7fc4441a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc4437fe640 1 -- 172.21.15.94:0/373328742 learned_addr learned my addr 172.21.15.94:0/373328742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b3d0 con 0x7fc4441033b0 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b250 con 0x7fc4441080b0 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc442ffd640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 0x7fc44419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.649+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410b550 con 0x7fc444102170 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc443fff640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441080b0 0x7fc4441a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc4437fe640 1 -- 172.21.15.94:0/373328742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441080b0 msgr2=0x7fc4441a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc4437fe640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441080b0 0x7fc4441a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc4437fe640 1 -- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 msgr2=0x7fc44419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc4437fe640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 0x7fc44419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc4437fe640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441a1da0 con 0x7fc444102170 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.650+0000 7fc442ffd640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 0x7fc44419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.651+0000 7fc4437fe640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc434006010 tx=0x7fc434004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.651+0000 7fc440ff9640 1 -- 172.21.15.94:0/373328742 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43403f030 con 0x7fc444102170 2024-04-25T14:05:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.651+0000 7fc440ff9640 1 -- 172.21.15.94:0/373328742 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc434031910 con 0x7fc444102170 2024-04-25T14:05:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.651+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4441a2030 con 0x7fc444102170 2024-04-25T14:05:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.652+0000 7fc440ff9640 1 -- 172.21.15.94:0/373328742 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434035070 con 0x7fc444102170 2024-04-25T14:05:21.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.652+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441a2570 con 0x7fc444102170 2024-04-25T14:05:21.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc440ff9640 1 -- 172.21.15.94:0/373328742 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc434042030 con 0x7fc444102170 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 Processor -- start 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 -- start start 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e7532550 0x5595e7900e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e7907d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5595e7861b00 con 0x5595e755b410 2024-04-25T14:05:21.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5595e77fd920 con 0x5595e7532550 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.655+0000 7fc450e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595e785dac0 con 0x5595e753a730 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc440ff9640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 0x7fc42c0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34534/0 (socket says 172.21.15.94:34534) 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 -- 172.21.15.94:0/4189455070 learned_addr learned my addr 172.21.15.94:0/4189455070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc4437fe640 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e7532550 0x5595e7900e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc442ffd640 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e7907d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 0x7fc42c0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 -- 172.21.15.94:0/4189455070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 msgr2=0x5595e7907d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e7907d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 -- 172.21.15.94:0/4189455070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e7532550 msgr2=0x5595e7900e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e7532550 0x5595e7900e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 -- 172.21.15.94:0/4189455070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e763b7e0 con 0x5595e755b410 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc440ff9640 1 -- 172.21.15.94:0/373328742 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc4340fefa0 con 0x7fc444102170 2024-04-25T14:05:21.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.656+0000 7fc443fff640 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fc43c011e20 tx=0x7fc43c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8670bcfe0d4ef0de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.657+0000 7fc443fff640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 0x7fc42c0b2100 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00fc90 tx=0x7fc43c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.657+0000 7fc429ffb640 1 -- 172.21.15.94:0/4189455070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c01dcb0 con 0x5595e755b410 2024-04-25T14:05:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.657+0000 7fc429ffb640 1 -- 172.21.15.94:0/4189455070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43c01de50 con 0x5595e755b410 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.657+0000 7fc429ffb640 1 -- 172.21.15.94:0/4189455070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004280 con 0x5595e755b410 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 -- 172.21.15.94:0/4189455070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 msgr2=0x5595e79008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fc43c011e20 tx=0x7fc43c00f790 comp rx=0 tx=0).stop 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 -- 172.21.15.94:0/4189455070 shutdown_connections 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e7907d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e7532550 0x5595e7900e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 --2- 172.21.15.94:0/4189455070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e79008b0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 -- 172.21.15.94:0/4189455070 >> 172.21.15.94:0/4189455070 conn(0x5595e754f560 msgr2=0x5595e78f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 -- 172.21.15.94:0/4189455070 shutdown_connections 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.658+0000 7fc450e07740 1 -- 172.21.15.94:0/4189455070 wait complete. 2024-04-25T14:05:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 -- start start 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 Processor -- start 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e7532550 0x5595e790a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e753a730 0x5595e790ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5595e79086c0 con 0x5595e755b410 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc443fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc442ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e753a730 0x5595e790ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e7532550 0x5595e790a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc443fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34544/0 (socket says 172.21.15.94:34544) 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc443fff640 1 -- 172.21.15.94:0/661434230 learned_addr learned my addr 172.21.15.94:0/661434230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5595e77fb5d0 con 0x5595e753a730 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595e7908840 con 0x5595e7532550 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.659+0000 7fc450e07740 10 client.0 _ll_register_callbacks cb 0x5595e77e9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 -- 172.21.15.94:0/661434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e7532550 msgr2=0x5595e790a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e7532550 0x5595e790a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 -- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e753a730 msgr2=0x5595e790ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e753a730 0x5595e790ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e769bf70 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc443fff640 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00a010 tx=0x7fc43c005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fbf41b1cc2314f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c02a040 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43c01de50 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc43c003970 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc43c01dcb0 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc450e07740 10 client.23898 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5595e7533170 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.660+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43c01de50 con 0x5595e755b410 2024-04-25T14:05:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc43c030860 con 0x5595e755b410 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 10 client.23898 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 10 client.23898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 20 client.23898 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e751a1f0 0x5595e7917df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 4 client.23898 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e786f710 con 0x5595e751a1f0 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.661+0000 7fc4437fe640 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e751a1f0 0x5595e7917df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.662+0000 7fc422ffd640 10 client.23898 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.662+0000 7fc4437fe640 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e751a1f0 0x5595e7917df0 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78eae338262d5c1c server_cookie=e6b64282f4bc01a8 in_seq=0 out_seq=0 2024-04-25T14:05:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc422ffd640 1 -- 172.21.15.94:0/661434230 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595e786f710 con 0x5595e751a1f0 2024-04-25T14:05:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc422ffd640 10 client.23898 handle_command_reply: tid=0 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 client.23898 shutdown 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 msgr2=0x5595e7911af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00a010 tx=0x7fc43c005460 comp rx=0 tx=0).stop 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 shutdown_connections 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e751a1f0 0x5595e7917df0 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e755b410 0x5595e7911af0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc422ffd640 0 client.23898 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e753a730 0x5595e790ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 --2- 172.21.15.94:0/661434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e7532550 0x5595e790a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 >> 172.21.15.94:0/661434230 conn(0x5595e754f560 msgr2=0x5595e78f6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 shutdown_connections 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 1 -- 172.21.15.94:0/661434230 wait complete. 2024-04-25T14:05:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.666+0000 7fc450e07740 20 client.23898 trim_cache size 0 max 16384 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.684+0000 7fc450e07740 1 Processor -- start 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.684+0000 7fc450e07740 1 -- start start 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e78f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e78f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441a9480 con 0x5595e7532550 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aafc0 con 0x5595e755b410 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc450e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441a9b50 con 0x5595e753a730 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34548/0 (socket says 172.21.15.94:34548) 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 -- 172.21.15.94:0/1914479963 learned_addr learned my addr 172.21.15.94:0/1914479963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc443fff640 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e78f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc442ffd640 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e78f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 -- 172.21.15.94:0/1914479963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 msgr2=0x5595e78f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e78f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 -- 172.21.15.94:0/1914479963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 msgr2=0x5595e78f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e78f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 -- 172.21.15.94:0/1914479963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e7903c50 con 0x5595e7532550 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.685+0000 7fc4437fe640 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fc4340c7130 tx=0x7fc4340c7330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e7b5da290cd9e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc4297fa640 1 -- 172.21.15.94:0/1914479963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340e1070 con 0x5595e7532550 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc4297fa640 1 -- 172.21.15.94:0/1914479963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4340cb730 con 0x5595e7532550 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc4297fa640 1 -- 172.21.15.94:0/1914479963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340fce60 con 0x5595e7532550 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 -- 172.21.15.94:0/1914479963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 msgr2=0x5595e79093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fc4340c7130 tx=0x7fc4340c7330 comp rx=0 tx=0).stop 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 -- 172.21.15.94:0/1914479963 shutdown_connections 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e78f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e78f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.686+0000 7fc450e07740 1 --2- 172.21.15.94:0/1914479963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e79093b0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 -- 172.21.15.94:0/1914479963 >> 172.21.15.94:0/1914479963 conn(0x5595e754f560 msgr2=0x5595e7908ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 -- 172.21.15.94:0/1914479963 shutdown_connections 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 -- 172.21.15.94:0/1914479963 wait complete. 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 -- start start 2024-04-25T14:05:21.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 Processor -- start 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e790e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.687+0000 7fc450e07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e797e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc450e07740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aa8f0 con 0x5595e7532550 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc450e07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aa220 con 0x5595e755b410 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc450e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441a7460 con 0x5595e753a730 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc450e07740 10 client.0 _ll_register_callbacks cb 0x7fc43c011c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:21.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34554/0 (socket says 172.21.15.94:34554) 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 -- 172.21.15.94:0/3747183652 learned_addr learned my addr 172.21.15.94:0/3747183652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc443fff640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e797e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc442ffd640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e790e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 -- 172.21.15.94:0/3747183652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 msgr2=0x5595e790e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e790e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 -- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 msgr2=0x5595e797e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e797e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 -- 172.21.15.94:0/3747183652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e79055c0 con 0x5595e7532550 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc4437fe640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7fc4340c73c0 tx=0x7fc434000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ad7a66abc93088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340fce60 con 0x5595e7532550 2024-04-25T14:05:21.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc4340cfd50 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.688+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4340cb730 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4340e1040 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 10 client.23904 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5595e797f750 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4340cb730 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc4340c1bc0 con 0x5595e7532550 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 10 client.23904 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 10 client.23904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 20 client.23904 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e74ef630 0x5595e79820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 4 client.23904 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5595e7869580 con 0x5595e74ef630 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.689+0000 7fc442ffd640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e74ef630 0x5595e79820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.690+0000 7fc422ffd640 10 client.23904 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.690+0000 7fc442ffd640 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e74ef630 0x5595e79820c0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6258fe9278003af9 server_cookie=33195db9c04bfd7b in_seq=0 out_seq=0 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.693+0000 7fc422ffd640 1 -- 172.21.15.94:0/3747183652 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5595e7869580 con 0x5595e74ef630 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.693+0000 7fc422ffd640 10 client.23904 handle_command_reply: tid=0 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.693+0000 7fc450e07740 1 client.23904 shutdown 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.693+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 msgr2=0x5595e790e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.693+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7fc4340c73c0 tx=0x7fc434000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 shutdown_connections 2024-04-25T14:05:21.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5595e74ef630 0x5595e79820c0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595e755b410 0x5595e797e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595e753a730 0x5595e790e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc422ffd640 0 client.23904 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 --2- 172.21.15.94:0/3747183652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5595e7532550 0x5595e790e1f0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 >> 172.21.15.94:0/3747183652 conn(0x5595e754f560 msgr2=0x5595e78f75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 shutdown_connections 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 1 -- 172.21.15.94:0/3747183652 wait complete. 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.694+0000 7fc450e07740 20 client.23904 trim_cache size 0 max 16384 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:05:21.697 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "d4c3f296-a971-4230-b4e8-50a3cc3c3591", 2024-04-25T14:05:21.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:05:21.698 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:21.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 msgr2=0x7fc42c0b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 0x7fc42c0b2100 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00fc90 tx=0x7fc43c01e000 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 msgr2=0x7fc44419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc434006010 tx=0x7fc434004270 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 shutdown_connections 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc42c0afbf0 0x7fc42c0b2100 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4441080b0 0x7fc4441a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4441033b0 0x7fc44419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 --2- 172.21.15.94:0/373328742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc444102170 0x7fc44419a260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 >> 172.21.15.94:0/373328742 conn(0x7fc4440fd340 msgr2=0x7fc4440ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 shutdown_connections 2024-04-25T14:05:21.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.695+0000 7fc449e84640 1 -- 172.21.15.94:0/373328742 wait complete. 2024-04-25T14:05:21.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:21.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 -- 172.21.15.94:0/730549220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 msgr2=0x7ff1901036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 --2- 172.21.15.94:0/730549220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff1901036e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff180009900 tx=0x7ff18002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 -- 172.21.15.94:0/730549220 shutdown_connections 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 --2- 172.21.15.94:0/730549220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff19010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 --2- 172.21.15.94:0/730549220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 0x7ff190104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 --2- 172.21.15.94:0/730549220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff1901036e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 -- 172.21.15.94:0/730549220 >> 172.21.15.94:0/730549220 conn(0x7ff1900fdbd0 msgr2=0x7ff190100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 -- 172.21.15.94:0/730549220 shutdown_connections 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.831+0000 7ff195830640 1 -- 172.21.15.94:0/730549220 wait complete. 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 Processor -- start 2024-04-25T14:05:21.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 -- start start 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff190199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 0x7ff1901999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010e4c0 con 0x7ff190108920 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010e340 con 0x7ff1901029e0 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff195830640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010e640 con 0x7ff190103c20 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff190199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34570/0 (socket says 172.21.15.94:34570) 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff190199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:33678/0 (socket says 172.21.15.94:33678) 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 -- 172.21.15.94:0/2151402571 learned_addr learned my addr 172.21.15.94:0/2151402571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff187fff640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 0x7ff1901999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 -- 172.21.15.94:0/2151402571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 msgr2=0x7ff1901999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 0x7ff1901999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 -- 172.21.15.94:0/2151402571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 msgr2=0x7ff190199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff190199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.832+0000 7ff18f7fe640 1 -- 172.21.15.94:0/2151402571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1901a0fb0 con 0x7ff190108920 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff18f7fe640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff18800efd0 tx=0x7ff18800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff19482e640 1 -- 172.21.15.94:0/2151402571 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188003b70 con 0x7ff190108920 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1901a12a0 con 0x7ff190108920 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff19482e640 1 -- 172.21.15.94:0/2151402571 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff188003d10 con 0x7ff190108920 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff19482e640 1 -- 172.21.15.94:0/2151402571 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880108a0 con 0x7ff190108920 2024-04-25T14:05:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.833+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901a17a0 con 0x7ff190108920 2024-04-25T14:05:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.835+0000 7ff19c7bd740 1 Processor -- start 2024-04-25T14:05:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19482e640 1 -- 172.21.15.94:0/2151402571 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff188010a60 con 0x7ff190108920 2024-04-25T14:05:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 -- start start 2024-04-25T14:05:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa30ad80 con 0x55d2aa004390 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa2c48b0 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa306d40 con 0x55d2a9fe36b0 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff187fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff187fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34572/0 (socket says 172.21.15.94:34572) 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff187fff640 1 -- 172.21.15.94:0/2425699897 learned_addr learned my addr 172.21.15.94:0/2425699897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff187fff640 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff19482e640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 0x7ff17c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 -- 172.21.15.94:0/2425699897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 msgr2=0x55d2aa3a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 -- 172.21.15.94:0/2425699897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 msgr2=0x55d2aa3a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.836+0000 7ff18effd640 1 -- 172.21.15.94:0/2425699897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2aa295a40 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff18f7fe640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 0x7ff17c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff18effd640 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3a5fe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff18002a930 tx=0x7ff180000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd3ae49676027a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff185ffb640 1 -- 172.21.15.94:0/2425699897 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180035070 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff19482e640 1 -- 172.21.15.94:0/2151402571 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff1880df060 con 0x7ff190108920 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff185ffb640 1 -- 172.21.15.94:0/2425699897 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180031b90 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.837+0000 7ff185ffb640 1 -- 172.21.15.94:0/2425699897 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18003d9a0 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2425699897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 msgr2=0x55d2aa3a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff18f7fe640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 0x7ff17c0b2000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff1880c7780 tx=0x7ff1880b80a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3a5fe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff18002a930 tx=0x7ff180000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2425699897 shutdown_connections 2024-04-25T14:05:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3a5fe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2425699897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2425699897 >> 172.21.15.94:0/2425699897 conn(0x55d2a9ff84e0 msgr2=0x55d2aa3a1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2425699897 shutdown_connections 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2425699897 wait complete. 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.838+0000 7ff19c7bd740 1 -- start start 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 Processor -- start 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa3a7720 con 0x55d2aa004390 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa291d10 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2aa3a78a0 con 0x55d2a9fe36b0 2024-04-25T14:05:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff19c7bd740 10 client.0 _ll_register_callbacks cb 0x55d2aa2be210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff18effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34576/0 (socket says 172.21.15.94:34576) 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff18effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:41120/0 (socket says 172.21.15.94:41120) 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 -- 172.21.15.94:0/981254225 learned_addr learned my addr 172.21.15.94:0/981254225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 -- 172.21.15.94:0/981254225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 msgr2=0x55d2aa3afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 -- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 msgr2=0x55d2aa3af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff187fff640 1 -- 172.21.15.94:0/981254225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2aa0e67d0 con 0x55d2aa004390 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.839+0000 7ff18effd640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff187fff640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f440 tx=0x7ff178005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2773e50d35b68886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178011030 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff178009040 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff180031b90 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff1780111d0 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 10 client.23916 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d2aa01c3f0 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180031b90 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff178003ae0 con 0x55d2aa004390 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 10 client.23916 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 10 client.23916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:21.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 20 client.23916 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9fc2f60 0x55d2aa3bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 4 client.23916 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.840+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2aa318990 con 0x55d2a9fc2f60 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.841+0000 7ff18effd640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9fc2f60 0x55d2aa3bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.841+0000 7ff16effd640 10 client.23916 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.841+0000 7ff18effd640 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9fc2f60 0x55d2aa3bd290 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623aeba68b9ffea2 server_cookie=e2c3710d0f5c0f7f in_seq=0 out_seq=0 2024-04-25T14:05:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.844+0000 7ff16effd640 1 -- 172.21.15.94:0/981254225 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2aa318990 con 0x55d2a9fc2f60 2024-04-25T14:05:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.844+0000 7ff16effd640 10 client.23916 handle_command_reply: tid=0 2024-04-25T14:05:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.844+0000 7ff19c7bd740 1 client.23916 shutdown 2024-04-25T14:05:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 msgr2=0x55d2aa3b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f440 tx=0x7ff178005f90 comp rx=0 tx=0).stop 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 shutdown_connections 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9fc2f60 0x55d2aa3bd290 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2aa004390 0x55d2aa3b6d80 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/981254225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff16effd640 0 client.23916 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 >> 172.21.15.94:0/981254225 conn(0x55d2a9ff84e0 msgr2=0x55d2aa3a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 shutdown_connections 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 1 -- 172.21.15.94:0/981254225 wait complete. 2024-04-25T14:05:21.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.845+0000 7ff19c7bd740 20 client.23916 trim_cache size 0 max 16384 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 Processor -- start 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 -- start start 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2aa004390 0x55d2aa3a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a92f0 con 0x55d2a9fe36b0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901aae30 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a99c0 con 0x55d2aa004390 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff18effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff18effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34592/0 (socket says 172.21.15.94:34592) 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff18effd640 1 -- 172.21.15.94:0/2749443528 learned_addr learned my addr 172.21.15.94:0/2749443528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff187fff640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.865+0000 7ff187fff640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2aa004390 0x55d2aa3a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 -- 172.21.15.94:0/2749443528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2aa004390 msgr2=0x55d2aa3a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2aa004390 0x55d2aa3a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 -- 172.21.15.94:0/2749443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 msgr2=0x55d2aa3ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 -- 172.21.15.94:0/2749443528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a9fdc0f0 con 0x55d2a9fe36b0 2024-04-25T14:05:21.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff187fff640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:21.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff18effd640 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7ff18002a450 tx=0x7ff180005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9b1e13833d5d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff1857fa640 1 -- 172.21.15.94:0/2749443528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180035040 con 0x55d2a9fe36b0 2024-04-25T14:05:21.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff1857fa640 1 -- 172.21.15.94:0/2749443528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180004040 con 0x55d2a9fe36b0 2024-04-25T14:05:21.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.866+0000 7ff1857fa640 1 -- 172.21.15.94:0/2749443528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180052880 con 0x55d2a9fe36b0 2024-04-25T14:05:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2749443528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 msgr2=0x55d2aa3b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7ff18002a450 tx=0x7ff180005900 comp rx=0 tx=0).stop 2024-04-25T14:05:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2749443528 shutdown_connections 2024-04-25T14:05:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2aa004390 0x55d2aa3a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b6d10 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/2749443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2749443528 >> 172.21.15.94:0/2749443528 conn(0x55d2a9ff84e0 msgr2=0x55d2aa3adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2749443528 shutdown_connections 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.867+0000 7ff19c7bd740 1 -- 172.21.15.94:0/2749443528 wait complete. 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 -- start start 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 Processor -- start 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2aa004390 0x55d2aa4235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901aa760 con 0x55d2a9fe36b0 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901aa090 con 0x55d2aa004390 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a72d0 con 0x55d2a9fdb4d0 2024-04-25T14:05:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff19c7bd740 10 client.0 _ll_register_callbacks cb 0x7ff178012690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff18effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff18effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34594/0 (socket says 172.21.15.94:34594) 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff18effd640 1 -- 172.21.15.94:0/4021149168 learned_addr learned my addr 172.21.15.94:0/4021149168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff187fff640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff187fff640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2aa004390 0x55d2aa4235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff18effd640 1 -- 172.21.15.94:0/4021149168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fdb4d0 msgr2=0x55d2aa3b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.868+0000 7ff18effd640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff18effd640 1 -- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2aa004390 msgr2=0x55d2aa4235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff18effd640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2aa004390 0x55d2aa4235f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff18effd640 1 -- 172.21.15.94:0/4021149168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2aa2cb4b0 con 0x55d2a9fe36b0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff18effd640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7ff180005f70 tx=0x7ff180052470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324f0002f7d840c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180035040 con 0x55d2a9fe36b0 2024-04-25T14:05:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7ff18004c070 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff180004040 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff1800041f0 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 10 client.23922 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d2aa424b30 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180004040 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff18003dc80 con 0x55d2a9fe36b0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 10 client.23922 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 10 client.23922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 20 client.23922 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.869+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9f983a0 0x55d2aa4270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.870+0000 7ff19c7bd740 4 client.23922 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.870+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2aa312800 con 0x55d2a9f983a0 2024-04-25T14:05:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.870+0000 7ff187fff640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9f983a0 0x55d2aa4270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.870+0000 7ff16effd640 10 client.23922 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.870+0000 7ff187fff640 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9f983a0 0x55d2aa4270b0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4496b5bdf2920fe0 server_cookie=7734c11cf4136be7 in_seq=0 out_seq=0 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff16effd640 1 -- 172.21.15.94:0/4021149168 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d2aa312800 con 0x55d2a9f983a0 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff16effd640 10 client.23922 handle_command_reply: tid=0 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 client.23922 shutdown 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 msgr2=0x55d2aa3b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7ff180005f70 tx=0x7ff180052470 comp rx=0 tx=0).stop 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 shutdown_connections 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55d2a9f983a0 0x55d2aa4270b0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2aa004390 0x55d2aa4235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2a9fe36b0 0x55d2aa3b41c0 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 --2- 172.21.15.94:0/4021149168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2a9fdb4d0 0x55d2aa3b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 >> 172.21.15.94:0/4021149168 conn(0x55d2a9ff84e0 msgr2=0x55d2aa3a0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff16effd640 0 client.23922 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 shutdown_connections 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.875+0000 7ff19c7bd740 1 -- 172.21.15.94:0/4021149168 wait complete. 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff19c7bd740 20 client.23922 trim_cache size 0 max 16384 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (443 inodes in the stack)", 2024-04-25T14:05:21.877 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "d4c3f296-a971-4230-b4e8-50a3cc3c3591": { 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "d4c3f296-a971-4230-b4e8-50a3cc3c3591", 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:05:21.878 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 msgr2=0x7ff17c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 0x7ff17c0b2000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff1880c7780 tx=0x7ff1880b80a0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 msgr2=0x7ff1901a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff18800efd0 tx=0x7ff18800c6a0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 shutdown_connections 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7ff17c0afaf0 0x7ff17c0b2000 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff190108920 0x7ff1901a0940 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff190103c20 0x7ff1901999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 --2- 172.21.15.94:0/2151402571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1901029e0 0x7ff190199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 >> 172.21.15.94:0/2151402571 conn(0x7ff1900fdbd0 msgr2=0x7ff190104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 shutdown_connections 2024-04-25T14:05:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:21.876+0000 7ff195830640 1 -- 172.21.15.94:0/2151402571 wait complete. 2024-04-25T14:05:21.887 INFO:tasks.cephfs.filesystem:scrub status for tag:d4c3f296-a971-4230-b4e8-50a3cc3c3591 - {'path': '/', 'tag': 'd4c3f296-a971-4230-b4e8-50a3cc3c3591', 'options': 'recursive,force'} 2024-04-25T14:05:22.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 213894 18.86 MB/sec execute 484 sec latency 124.855 ms 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: pgmap v1058: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:22.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:22.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:21 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: pgmap v1058: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-04-25T14:05:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:05:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:22.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:22.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:22.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:21 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: pgmap v1058: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-04-25T14:05:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:05:22.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:22.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:21 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 214321 18.88 MB/sec execute 485 sec latency 63.028 ms 2024-04-25T14:05:23.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:22 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:22 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:23.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:22 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:24.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 214572 18.88 MB/sec execute 486 sec latency 203.192 ms 2024-04-25T14:05:24.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:23 smithi138 ceph-mon[32977]: pgmap v1059: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 495 op/s 2024-04-25T14:05:24.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:23 smithi137 ceph-mon[32568]: pgmap v1059: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 495 op/s 2024-04-25T14:05:24.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:23 smithi094 ceph-mon[29195]: pgmap v1059: 97 pgs: 97 active+clean; 279 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 495 op/s 2024-04-25T14:05:25.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 214868 18.85 MB/sec execute 487 sec latency 100.056 ms 2024-04-25T14:05:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 -- 172.21.15.94:0/3211970938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440103c60 msgr2=0x7fd440104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 --2- 172.21.15.94:0/3211970938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440103c60 0x7fd440104070 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009900 tx=0x7fd43c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 -- 172.21.15.94:0/3211970938 shutdown_connections 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 --2- 172.21.15.94:0/3211970938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440108960 0x7fd44010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 --2- 172.21.15.94:0/3211970938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440103c60 0x7fd440104070 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 --2- 172.21.15.94:0/3211970938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440102a20 0x7fd440103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 -- 172.21.15.94:0/3211970938 >> 172.21.15.94:0/3211970938 conn(0x7fd4400fdc10 msgr2=0x7fd4401000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 -- 172.21.15.94:0/3211970938 shutdown_connections 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 -- 172.21.15.94:0/3211970938 wait complete. 2024-04-25T14:05:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.660+0000 7fd44871f640 1 Processor -- start 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 -- start start 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440103c60 0x7fd440199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010e540 con 0x7fd440108960 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010e3c0 con 0x7fd440102a20 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd44871f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010e6c0 con 0x7fd440103c60 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46984/0 (socket says 172.21.15.94:46984) 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43078/0 (socket says 172.21.15.94:43078) 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 -- 172.21.15.94:0/379921104 learned_addr learned my addr 172.21.15.94:0/379921104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd445cba640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440103c60 0x7fd440199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 -- 172.21.15.94:0/379921104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440103c60 msgr2=0x7fd440199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440103c60 0x7fd440199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 -- 172.21.15.94:0/379921104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 msgr2=0x7fd440199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 -- 172.21.15.94:0/379921104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4401a0c90 con 0x7fd440108960 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd4464bb640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.661+0000 7fd446cbc640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fd43800eb10 tx=0x7fd43800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.662+0000 7fd4377fe640 1 -- 172.21.15.94:0/379921104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438004b20 con 0x7fd440108960 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.662+0000 7fd4377fe640 1 -- 172.21.15.94:0/379921104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd438004cc0 con 0x7fd440108960 2024-04-25T14:05:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.662+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4401a0f80 con 0x7fd440108960 2024-04-25T14:05:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.662+0000 7fd4377fe640 1 -- 172.21.15.94:0/379921104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438010670 con 0x7fd440108960 2024-04-25T14:05:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.662+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a1390 con 0x7fd440108960 2024-04-25T14:05:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.664+0000 7fd4377fe640 1 -- 172.21.15.94:0/379921104 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd438005020 con 0x7fd440108960 2024-04-25T14:05:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.664+0000 7fd44f5d4740 1 Processor -- start 2024-04-25T14:05:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 -- start start 2024-04-25T14:05:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 0x5635feabf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6fe4d0 0x5635feac6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635fea25460 con 0x5635fe6f62f0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635fe9df590 con 0x5635fe6fe4d0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635fea21420 con 0x5635fe71f1b0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4377fe640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 0x7fd42c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 0x5635feabf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46994/0 (socket says 172.21.15.94:46994) 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4464bb640 1 -- 172.21.15.94:0/3347077872 learned_addr learned my addr 172.21.15.94:0/3347077872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd446cbc640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 0x7fd42c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6fe4d0 0x5635feac6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4377fe640 1 -- 172.21.15.94:0/379921104 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd4380df280 con 0x7fd440108960 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 -- 172.21.15.94:0/3347077872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6fe4d0 msgr2=0x5635feac6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6fe4d0 0x5635feac6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 -- 172.21.15.94:0/3347077872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 msgr2=0x5635feabfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd445cba640 1 -- 172.21.15.94:0/3347077872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fe9ec0d0 con 0x5635fe71f1b0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.665+0000 7fd4464bb640 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.666+0000 7fd446cbc640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 0x7fd42c0b1ff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd4380a3740 tx=0x7fd4380ca050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.666+0000 7fd445cba640 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 0x5635feabf970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd6e453ccf768b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.666+0000 7fd434ff9640 1 -- 172.21.15.94:0/3347077872 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c035040 con 0x5635fe71f1b0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.666+0000 7fd434ff9640 1 -- 172.21.15.94:0/3347077872 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c031650 con 0x5635fe71f1b0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.666+0000 7fd434ff9640 1 -- 172.21.15.94:0/3347077872 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0035f0 con 0x5635fe71f1b0 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- 172.21.15.94:0/3347077872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 msgr2=0x5635feabf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 0x5635feabf970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- 172.21.15.94:0/3347077872 shutdown_connections 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6fe4d0 0x5635feac6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feabfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/3347077872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe71f1b0 0x5635feabf970 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- 172.21.15.94:0/3347077872 >> 172.21.15.94:0/3347077872 conn(0x5635fe713300 msgr2=0x5635feabb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- 172.21.15.94:0/3347077872 shutdown_connections 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- 172.21.15.94:0/3347077872 wait complete. 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- start start 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 Processor -- start 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6f62f0 0x5635feac9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635fead0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635feac77e0 con 0x5635fe6fe4d0 2024-04-25T14:05:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5635fe9acb60 con 0x5635fe71f1b0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635feac7960 con 0x5635fe6f62f0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.667+0000 7fd44f5d4740 10 client.0 _ll_register_callbacks cb 0x5635fe9d12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47000/0 (socket says 172.21.15.94:47000) 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 -- 172.21.15.94:0/753324929 learned_addr learned my addr 172.21.15.94:0/753324929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd445cba640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6f62f0 0x5635feac9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd445cba640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635fead0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 -- 172.21.15.94:0/753324929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6f62f0 msgr2=0x5635feac9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6f62f0 0x5635feac9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 -- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 msgr2=0x5635fead0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635fead0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 -- 172.21.15.94:0/753324929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fe737210 con 0x5635fe6fe4d0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd4464bb640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fd4300117e0 tx=0x7fd430011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d45bbbf053d5eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430010ba0 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd430010d40 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c031650 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd430005020 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.668+0000 7fd44f5d4740 10 client.23934 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5635fe803620 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c031650 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd430024430 con 0x5635fe6fe4d0 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 10 client.23934 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 10 client.23934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 20 client.23934 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6ddd80 0x5635fead73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 4 client.23934 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fea33070 con 0x5635fe6ddd80 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.669+0000 7fd445cba640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6ddd80 0x5635fead73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.670+0000 7fd429ffb640 10 client.23934 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.670+0000 7fd445cba640 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6ddd80 0x5635fead73c0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52567b440e14c8b6 server_cookie=b50ffceb850e657d in_seq=0 out_seq=0 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd429ffb640 1 -- 172.21.15.94:0/753324929 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635fea33070 con 0x5635fe6ddd80 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd429ffb640 10 client.23934 handle_command_reply: tid=0 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 client.23934 shutdown 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 msgr2=0x5635feac9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fd4300117e0 tx=0x7fd430011ca0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 shutdown_connections 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6ddd80 0x5635fead73c0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635fead0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6fe4d0 0x5635feac9c50 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/753324929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6f62f0 0x5635feac9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd429ffb640 0 client.23934 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 >> 172.21.15.94:0/753324929 conn(0x5635fe713300 msgr2=0x5635feabe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 shutdown_connections 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.672+0000 7fd44f5d4740 1 -- 172.21.15.94:0/753324929 wait complete. 2024-04-25T14:05:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.673+0000 7fd44f5d4740 20 client.23934 trim_cache size 0 max 16384 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.687+0000 7fd44f5d4740 1 Processor -- start 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 -- start start 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6f62f0 0x5635feac8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401aa400 con 0x5635fe71f1b0 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401abf40 con 0x5635fe6f62f0 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401aaad0 con 0x5635fe6fe4d0 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47012/0 (socket says 172.21.15.94:47012) 2024-04-25T14:05:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 -- 172.21.15.94:0/249367996 learned_addr learned my addr 172.21.15.94:0/249367996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6f62f0 0x5635feac8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd4464bb640 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 -- 172.21.15.94:0/249367996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 msgr2=0x5635fead0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 -- 172.21.15.94:0/249367996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6f62f0 msgr2=0x5635feac8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6f62f0 0x5635feac8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 -- 172.21.15.94:0/249367996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fead0fa0 con 0x5635fe71f1b0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.688+0000 7fd445cba640 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2265bfa80619be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd42bfff640 1 -- 172.21.15.94:0/249367996 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c035040 con 0x5635fe71f1b0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd42bfff640 1 -- 172.21.15.94:0/249367996 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c031d00 con 0x5635fe71f1b0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd42bfff640 1 -- 172.21.15.94:0/249367996 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c003e80 con 0x5635fe71f1b0 2024-04-25T14:05:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 -- 172.21.15.94:0/249367996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 msgr2=0x5635feadcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fd43c005ab0 tx=0x7fd43c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 -- 172.21.15.94:0/249367996 shutdown_connections 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe71f1b0 0x5635feadcbc0 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/249367996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe6f62f0 0x5635feac8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.689+0000 7fd44f5d4740 1 -- 172.21.15.94:0/249367996 >> 172.21.15.94:0/249367996 conn(0x5635fe713300 msgr2=0x5635feac7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- 172.21.15.94:0/249367996 shutdown_connections 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- 172.21.15.94:0/249367996 wait complete. 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- start start 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 Processor -- start 2024-04-25T14:05:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635feb3d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401ab870 con 0x5635fe6f62f0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401ab1a0 con 0x5635fe71f1b0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4401a83e0 con 0x5635fe6fe4d0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd44f5d4740 10 client.0 _ll_register_callbacks cb 0x7fd43000bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd445cba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd445cba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47018/0 (socket says 172.21.15.94:47018) 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd445cba640 1 -- 172.21.15.94:0/921437616 learned_addr learned my addr 172.21.15.94:0/921437616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd445cba640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635feb3d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.690+0000 7fd4464bb640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 -- 172.21.15.94:0/921437616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 msgr2=0x5635fead00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 -- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 msgr2=0x5635feb3d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635feb3d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 -- 172.21.15.94:0/921437616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635feb3ddb0 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd445cba640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0096c0 tx=0x7fd43c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ea789abd809113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c058020 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd43c003e00 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c031d00 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd43c035050 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 10 client.23940 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5635feb3f050 con 0x5635fe6f62f0 2024-04-25T14:05:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43c031d00 con 0x5635fe6f62f0 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd43c0586f0 con 0x5635fe6f62f0 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 10 client.23940 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 10 client.23940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 20 client.23940 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6b31c0 0x5635feb41500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 4 client.23940 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.691+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635fea2cee0 con 0x5635fe6b31c0 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.692+0000 7fd4464bb640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6b31c0 0x5635feb41500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.692+0000 7fd429ffb640 10 client.23940 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:25.692+0000 7fd4464bb640 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6b31c0 0x5635feb41500 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aae456743e61e91 server_cookie=6082a0113daca327 in_seq=0 out_seq=0 2024-04-25T14:05:26.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 215062 18.83 MB/sec execute 488 sec latency 342.100 ms 2024-04-25T14:05:26.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:25 smithi138 ceph-mon[32977]: pgmap v1060: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-04-25T14:05:26.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:25 smithi137 ceph-mon[32568]: pgmap v1060: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-04-25T14:05:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:25 smithi094 ceph-mon[29195]: pgmap v1060: 97 pgs: 97 active+clean; 283 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.6 MiB/s wr, 612 op/s 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd429ffb640 1 -- 172.21.15.94:0/921437616 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635fea2cee0 con 0x5635fe6b31c0 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd429ffb640 10 client.23940 handle_command_reply: tid=0 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 client.23940 shutdown 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 msgr2=0x5635feacfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0096c0 tx=0x7fd43c032ea0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 shutdown_connections 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5635fe6b31c0 0x5635feb41500 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635fe71f1b0 0x5635feb3d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635fe6fe4d0 0x5635fead00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 --2- 172.21.15.94:0/921437616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635fe6f62f0 0x5635feacfba0 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 >> 172.21.15.94:0/921437616 conn(0x5635fe713300 msgr2=0x5635feac7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd429ffb640 0 client.23940 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 shutdown_connections 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.492+0000 7fd44f5d4740 1 -- 172.21.15.94:0/921437616 wait complete. 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44f5d4740 20 client.23940 trim_cache size 0 max 16384 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:05:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:05:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 msgr2=0x7fd42c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 0x7fd42c0b1ff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd4380a3740 tx=0x7fd4380ca050 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 msgr2=0x7fd4401a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fd43800eb10 tx=0x7fd43800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 shutdown_connections 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd42c0afae0 0x7fd42c0b1ff0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd440108960 0x7fd4401a0620 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd440103c60 0x7fd440199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 --2- 172.21.15.94:0/379921104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd440102a20 0x7fd440199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 >> 172.21.15.94:0/379921104 conn(0x7fd4400fdc10 msgr2=0x7fd440104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 shutdown_connections 2024-04-25T14:05:26.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:26.493+0000 7fd44871f640 1 -- 172.21.15.94:0/379921104 wait complete. 2024-04-25T14:05:27.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 215321 18.83 MB/sec execute 489 sec latency 439.572 ms 2024-04-25T14:05:28.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 215730 18.82 MB/sec execute 490 sec latency 63.162 ms 2024-04-25T14:05:28.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:27 smithi138 ceph-mon[32977]: pgmap v1061: 97 pgs: 97 active+clean; 284 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 433 op/s 2024-04-25T14:05:28.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:27 smithi137 ceph-mon[32568]: pgmap v1061: 97 pgs: 97 active+clean; 284 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 433 op/s 2024-04-25T14:05:28.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:27 smithi094 ceph-mon[29195]: pgmap v1061: 97 pgs: 97 active+clean; 284 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 433 op/s 2024-04-25T14:05:29.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 216153 18.84 MB/sec execute 491 sec latency 193.651 ms 2024-04-25T14:05:30.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 216531 18.85 MB/sec execute 492 sec latency 250.715 ms 2024-04-25T14:05:30.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:29 smithi138 ceph-mon[32977]: pgmap v1062: 97 pgs: 97 active+clean; 277 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T14:05:30.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:29 smithi137 ceph-mon[32568]: pgmap v1062: 97 pgs: 97 active+clean; 277 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T14:05:30.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:29 smithi094 ceph-mon[29195]: pgmap v1062: 97 pgs: 97 active+clean; 277 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T14:05:31.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 217066 18.85 MB/sec execute 493 sec latency 52.521 ms 2024-04-25T14:05:31.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:05:31.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:31 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:05:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:05:31.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:31 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:05:31.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:05:31.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:31 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:05:32.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 217430 18.86 MB/sec execute 494 sec latency 120.069 ms 2024-04-25T14:05:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:32 smithi137 ceph-mon[32568]: pgmap v1063: 97 pgs: 97 active+clean; 272 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 571 op/s 2024-04-25T14:05:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:32 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:05:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:32 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:05:32.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:32 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:05:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:32 smithi094 ceph-mon[29195]: pgmap v1063: 97 pgs: 97 active+clean; 272 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 571 op/s 2024-04-25T14:05:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:32 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:05:32.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:32 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:05:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:32 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:05:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:32 smithi138 ceph-mon[32977]: pgmap v1063: 97 pgs: 97 active+clean; 272 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 571 op/s 2024-04-25T14:05:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:32 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:05:32.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:32 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:05:32.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:32 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:05:33.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 217878 18.88 MB/sec execute 495 sec latency 50.693 ms 2024-04-25T14:05:34.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 218210 18.88 MB/sec execute 496 sec latency 79.834 ms 2024-04-25T14:05:34.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:34 smithi137 ceph-mon[32568]: pgmap v1064: 97 pgs: 97 active+clean; 273 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-04-25T14:05:34.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:34 smithi094 ceph-mon[29195]: pgmap v1064: 97 pgs: 97 active+clean; 273 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-04-25T14:05:34.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:34 smithi138 ceph-mon[32977]: pgmap v1064: 97 pgs: 97 active+clean; 273 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-04-25T14:05:35.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 218465 18.86 MB/sec execute 497 sec latency 238.593 ms 2024-04-25T14:05:36.160 INFO:tasks.workunit.client.0.smithi094.stdout: 10 218703 18.85 MB/sec execute 498 sec latency 108.348 ms 2024-04-25T14:05:36.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:36 smithi137 ceph-mon[32568]: pgmap v1065: 97 pgs: 97 active+clean; 280 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 619 op/s 2024-04-25T14:05:36.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:36 smithi094 ceph-mon[29195]: pgmap v1065: 97 pgs: 97 active+clean; 280 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 619 op/s 2024-04-25T14:05:36.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:36 smithi138 ceph-mon[32977]: pgmap v1065: 97 pgs: 97 active+clean; 280 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 619 op/s 2024-04-25T14:05:37.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 219056 18.85 MB/sec execute 499 sec latency 101.623 ms 2024-04-25T14:05:38.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 219443 18.86 MB/sec execute 500 sec latency 122.066 ms 2024-04-25T14:05:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:38 smithi137 ceph-mon[32568]: pgmap v1066: 97 pgs: 97 active+clean; 284 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-04-25T14:05:38.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:38 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:38 smithi094 ceph-mon[29195]: pgmap v1066: 97 pgs: 97 active+clean; 284 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-04-25T14:05:38.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:38 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:38 smithi138 ceph-mon[32977]: pgmap v1066: 97 pgs: 97 active+clean; 284 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-04-25T14:05:38.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:38 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:39.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 219889 18.86 MB/sec execute 501 sec latency 97.041 ms 2024-04-25T14:05:39.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:39.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:39.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:05:39.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:39.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:39.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:05:39.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:39.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:39.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:05:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 -- 172.21.15.94:0/621246509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 msgr2=0x7f7028103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 --2- 172.21.15.94:0/621246509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028103750 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f7024009900 tx=0x7f702402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 -- 172.21.15.94:0/621246509 shutdown_connections 2024-04-25T14:05:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 --2- 172.21.15.94:0/621246509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028108990 0x7f702810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 --2- 172.21.15.94:0/621246509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028103c90 0x7f70281040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 --2- 172.21.15.94:0/621246509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028103750 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 -- 172.21.15.94:0/621246509 >> 172.21.15.94:0/621246509 conn(0x7f70280fdc60 msgr2=0x7f70281000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 -- 172.21.15.94:0/621246509 shutdown_connections 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.613+0000 7f702f74d640 1 -- 172.21.15.94:0/621246509 wait complete. 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 Processor -- start 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 -- start start 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028103c90 0x7f70281996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028108990 0x7f70281a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f702810e550 con 0x7f7028102a50 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f702810e3d0 con 0x7f7028103c90 2024-04-25T14:05:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702f74d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f702810e6d0 con 0x7f7028108990 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58652/0 (socket says 172.21.15.94:58652) 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 -- 172.21.15.94:0/3902961338 learned_addr learned my addr 172.21.15.94:0/3902961338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702cce8640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028103c90 0x7f70281996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702dcea640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028108990 0x7f70281a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 -- 172.21.15.94:0/3902961338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028108990 msgr2=0x7f70281a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028108990 0x7f70281a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 -- 172.21.15.94:0/3902961338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028103c90 msgr2=0x7f70281996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028103c90 0x7f70281996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 -- 172.21.15.94:0/3902961338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70281a0ca0 con 0x7f7028102a50 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.614+0000 7f702d4e9640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f7024006010 tx=0x7f7024004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.615+0000 7f701a7fc640 1 -- 172.21.15.94:0/3902961338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702403f030 con 0x7f7028102a50 2024-04-25T14:05:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.615+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70281a0f30 con 0x7f7028102a50 2024-04-25T14:05:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.615+0000 7f701a7fc640 1 -- 172.21.15.94:0/3902961338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7024031910 con 0x7f7028102a50 2024-04-25T14:05:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.615+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281a1340 con 0x7f7028102a50 2024-04-25T14:05:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.615+0000 7f701a7fc640 1 -- 172.21.15.94:0/3902961338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024035070 con 0x7f7028102a50 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.617+0000 7f70365fe740 1 Processor -- start 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.617+0000 7f701a7fc640 1 -- 172.21.15.94:0/3902961338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7024042030 con 0x7f7028102a50 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.617+0000 7f70365fe740 1 -- start start 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.617+0000 7f70365fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c88970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f70365fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c88ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f70365fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f70365fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5619d2bee460 con 0x5619d28bf2f0 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f70365fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5619d2ba8590 con 0x5619d28c74d0 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f70365fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5619d2bea420 con 0x5619d28e81b0 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c88ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c88ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58656/0 (socket says 172.21.15.94:58656) 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47732/0 (socket says 172.21.15.94:47732) 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702d4e9640 1 -- 172.21.15.94:0/4265872813 learned_addr learned my addr 172.21.15.94:0/4265872813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702dcea640 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c88970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f701a7fc640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 0x7f70140b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 -- 172.21.15.94:0/4265872813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 msgr2=0x5619d2c88970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c88970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 -- 172.21.15.94:0/4265872813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 msgr2=0x5619d2c88ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c88ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 -- 172.21.15.94:0/4265872813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d2bb50d0 con 0x5619d28c74d0 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702dcea640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 0x7f70140b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f701a7fc640 1 -- 172.21.15.94:0/3902961338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f70240fef80 con 0x7f7028102a50 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.618+0000 7f702cce8640 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f701c009bd0 tx=0x7f701c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d784ecb9d1930953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f70137fe640 1 -- 172.21.15.94:0/4265872813 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c004230 con 0x5619d28c74d0 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f70137fe640 1 -- 172.21.15.94:0/4265872813 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c009e40 con 0x5619d28c74d0 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f70137fe640 1 -- 172.21.15.94:0/4265872813 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c004d00 con 0x5619d28c74d0 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f702dcea640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 0x7f70140b1ff0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x5619d2c89b40 tx=0x7f7020014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f70365fe740 1 -- 172.21.15.94:0/4265872813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 msgr2=0x5619d2c8fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.619+0000 7f70365fe740 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f701c009bd0 tx=0x7f701c011810 comp rx=0 tx=0).stop 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 -- 172.21.15.94:0/4265872813 shutdown_connections 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c8fdf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c88ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 --2- 172.21.15.94:0/4265872813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c88970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 -- 172.21.15.94:0/4265872813 >> 172.21.15.94:0/4265872813 conn(0x5619d28dc300 msgr2=0x5619d2c84300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 -- 172.21.15.94:0/4265872813 shutdown_connections 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 -- 172.21.15.94:0/4265872813 wait complete. 2024-04-25T14:05:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 -- start start 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.620+0000 7f70365fe740 1 Processor -- start 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 0x5619d2c92be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c99b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5619d2c90840 con 0x5619d28c74d0 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5619d2b75b60 con 0x5619d28bf2f0 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5619d2c909c0 con 0x5619d28e81b0 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702cce8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 0x5619d2c92be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:47742/0 (socket says 172.21.15.94:47742) 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 -- 172.21.15.94:0/4022772774 learned_addr learned my addr 172.21.15.94:0/4022772774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f70365fe740 10 client.0 _ll_register_callbacks cb 0x5619d2b9a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702dcea640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c99b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 -- 172.21.15.94:0/4022772774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 msgr2=0x5619d2c99b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c99b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 -- 172.21.15.94:0/4022772774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 msgr2=0x5619d2c92be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 0x5619d2c92be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 -- 172.21.15.94:0/4022772774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d2900210 con 0x5619d28bf2f0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702cce8640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 0x5619d2c92be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f702d4e9640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70240c7400 tx=0x7f7024003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b4a988f0fc2b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.621+0000 7f7010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024049070 con 0x5619d28bf2f0 2024-04-25T14:05:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7024101110 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619d2bfc070 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f70240c7240 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 10 client.33332 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5619d29cc620 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5619d2bfc070 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7024102c10 con 0x5619d28bf2f0 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 10 client.33332 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 10 client.33332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 20 client.33332 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28a6d80 0x5619d2ca0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 4 client.33332 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.622+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d2bf5ee0 con 0x5619d28a6d80 2024-04-25T14:05:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.623+0000 7f702cce8640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28a6d80 0x5619d2ca0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.623+0000 7f702cce8640 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28a6d80 0x5619d2ca0400 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237bef1a2d2e1336 server_cookie=1923bfaa0f69c094 in_seq=0 out_seq=0 2024-04-25T14:05:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.623+0000 7f7010ff9640 10 client.33332 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f7010ff9640 1 -- 172.21.15.94:0/4022772774 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619d2bf5ee0 con 0x5619d28a6d80 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f7010ff9640 10 client.33332 handle_command_reply: tid=0 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 client.33332 shutdown 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 msgr2=0x5619d2c926a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70240c7400 tx=0x7f7024003f90 comp rx=0 tx=0).stop 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 shutdown_connections 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28a6d80 0x5619d2ca0400 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28e81b0 0x5619d2c99b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f7010ff9640 0 client.33332 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:39.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28c74d0 0x5619d2c92be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 --2- 172.21.15.94:0/4022772774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28bf2f0 0x5619d2c926a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 >> 172.21.15.94:0/4022772774 conn(0x5619d28dc300 msgr2=0x5619d2c87210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 shutdown_connections 2024-04-25T14:05:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 1 -- 172.21.15.94:0/4022772774 wait complete. 2024-04-25T14:05:39.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.628+0000 7f70365fe740 20 client.33332 trim_cache size 0 max 16384 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 Processor -- start 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 -- start start 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28bf2f0 0x5619d2c83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c84200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f70281a9260 con 0x5619d28a6d80 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70281aada0 con 0x5619d28c74d0 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f70365fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70281a9930 con 0x5619d28bf2f0 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58686/0 (socket says 172.21.15.94:58686) 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 -- 172.21.15.94:0/35632687 learned_addr learned my addr 172.21.15.94:0/35632687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702cce8640 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28bf2f0 0x5619d2c83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702dcea640 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c84200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 -- 172.21.15.94:0/35632687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28bf2f0 msgr2=0x5619d2c83cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28bf2f0 0x5619d2c83cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 -- 172.21.15.94:0/35632687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 msgr2=0x5619d2c84200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c84200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.644+0000 7f702d4e9640 1 -- 172.21.15.94:0/35632687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d2c8d550 con 0x5619d28a6d80 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.645+0000 7f702d4e9640 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f70240c46e0 tx=0x7f7024000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d334249a3bd10d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.645+0000 7f7012ffd640 1 -- 172.21.15.94:0/35632687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024049070 con 0x5619d28a6d80 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.645+0000 7f7012ffd640 1 -- 172.21.15.94:0/35632687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70240cb770 con 0x5619d28a6d80 2024-04-25T14:05:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.645+0000 7f7012ffd640 1 -- 172.21.15.94:0/35632687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70240c6be0 con 0x5619d28a6d80 2024-04-25T14:05:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- 172.21.15.94:0/35632687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 msgr2=0x5619d2c8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f70240c46e0 tx=0x7f7024000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- 172.21.15.94:0/35632687 shutdown_connections 2024-04-25T14:05:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2c84200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28bf2f0 0x5619d2c83cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- 172.21.15.94:0/35632687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28a6d80 0x5619d2c8c290 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- 172.21.15.94:0/35632687 >> 172.21.15.94:0/35632687 conn(0x5619d28dc300 msgr2=0x5619d2c8bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- 172.21.15.94:0/35632687 shutdown_connections 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- 172.21.15.94:0/35632687 wait complete. 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- start start 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 Processor -- start 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28a6d80 0x5619d2c96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2d067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f70281aa6d0 con 0x5619d28bf2f0 2024-04-25T14:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70281aa000 con 0x5619d28c74d0 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.646+0000 7f70365fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70281a7240 con 0x5619d28a6d80 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f70365fe740 10 client.0 _ll_register_callbacks cb 0x7f7024005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702d4e9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28a6d80 0x5619d2c96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58690/0 (socket says 172.21.15.94:58690) 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 -- 172.21.15.94:0/4052065738 learned_addr learned my addr 172.21.15.94:0/4052065738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702dcea640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2d067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 -- 172.21.15.94:0/4052065738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28a6d80 msgr2=0x5619d2c96190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28a6d80 0x5619d2c96190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 -- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 msgr2=0x5619d2d067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2d067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 -- 172.21.15.94:0/4052065738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d2d06ce0 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f702cce8640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f701c000c00 tx=0x7f701c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab11381442af2e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f7010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c037060 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f701c004650 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70240cb770 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f701c01e040 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f70365fe740 10 client.23958 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.647+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5619d2d07f70 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70240cb770 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f701c018070 con 0x5619d28bf2f0 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 10 client.23958 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 10 client.23958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 20 client.23958 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28e81b0 0x5619d2d0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 4 client.23958 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619d2b715f0 con 0x5619d28e81b0 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f702d4e9640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28e81b0 0x5619d2d0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f7010ff9640 10 client.23958 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:39.648+0000 7f702d4e9640 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28e81b0 0x5619d2d0a7d0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e8270cad185dd6 server_cookie=d9f4684e9209c23f in_seq=0 out_seq=0 2024-04-25T14:05:40.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 219986 18.83 MB/sec execute 502 sec latency 507.357 ms 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f7010ff9640 1 -- 172.21.15.94:0/4052065738 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619d2b715f0 con 0x5619d28e81b0 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f7010ff9640 10 client.23958 handle_command_reply: tid=0 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 client.23958 shutdown 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 msgr2=0x5619d2c966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f701c000c00 tx=0x7f701c005c50 comp rx=0 tx=0).stop 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 shutdown_connections 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5619d28e81b0 0x5619d2d0a7d0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619d28c74d0 0x5619d2d067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619d28bf2f0 0x5619d2c966d0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f7010ff9640 0 client.23958 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 --2- 172.21.15.94:0/4052065738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619d28a6d80 0x5619d2c96190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 >> 172.21.15.94:0/4052065738 conn(0x5619d28dc300 msgr2=0x5619d2c87100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 shutdown_connections 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 1 -- 172.21.15.94:0/4052065738 wait complete. 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.250+0000 7f70365fe740 20 client.23958 trim_cache size 0 max 16384 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:05:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:05:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 msgr2=0x7f70140b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 0x7f70140b1ff0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x5619d2c89b40 tx=0x7f7020014040 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 msgr2=0x7f7028199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f7024006010 tx=0x7f7024004270 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 shutdown_connections 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f70140afae0 0x7f70140b1ff0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7028108990 0x7f70281a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7028103c90 0x7f70281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 --2- 172.21.15.94:0/3902961338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7028102a50 0x7f7028199160 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 >> 172.21.15.94:0/3902961338 conn(0x7f70280fdc60 msgr2=0x7f7028104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 shutdown_connections 2024-04-25T14:05:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:40.251+0000 7f702f74d640 1 -- 172.21.15.94:0/3902961338 wait complete. 2024-04-25T14:05:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:40 smithi137 ceph-mon[32568]: pgmap v1067: 97 pgs: 97 active+clean; 285 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-04-25T14:05:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:40 smithi094 ceph-mon[29195]: pgmap v1067: 97 pgs: 97 active+clean; 285 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-04-25T14:05:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:40 smithi138 ceph-mon[32977]: pgmap v1067: 97 pgs: 97 active+clean; 285 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2024-04-25T14:05:41.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 220309 18.83 MB/sec execute 503 sec latency 594.562 ms 2024-04-25T14:05:42.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 220823 18.84 MB/sec execute 504 sec latency 61.216 ms 2024-04-25T14:05:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:42 smithi137 ceph-mon[32568]: pgmap v1068: 97 pgs: 97 active+clean; 288 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-04-25T14:05:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:42 smithi094 ceph-mon[29195]: pgmap v1068: 97 pgs: 97 active+clean; 288 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-04-25T14:05:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:42 smithi138 ceph-mon[32977]: pgmap v1068: 97 pgs: 97 active+clean; 288 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-04-25T14:05:43.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 221232 18.87 MB/sec execute 505 sec latency 77.900 ms 2024-04-25T14:05:44.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 221616 18.88 MB/sec execute 506 sec latency 58.558 ms 2024-04-25T14:05:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:44 smithi137 ceph-mon[32568]: pgmap v1069: 97 pgs: 97 active+clean; 287 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-04-25T14:05:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:44 smithi094 ceph-mon[29195]: pgmap v1069: 97 pgs: 97 active+clean; 287 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-04-25T14:05:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:44 smithi138 ceph-mon[32977]: pgmap v1069: 97 pgs: 97 active+clean; 287 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-04-25T14:05:45.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 221872 18.86 MB/sec execute 507 sec latency 227.961 ms 2024-04-25T14:05:46.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 222164 18.84 MB/sec execute 508 sec latency 93.768 ms 2024-04-25T14:05:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:46 smithi137 ceph-mon[32568]: pgmap v1070: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-04-25T14:05:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:46 smithi094 ceph-mon[29195]: pgmap v1070: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-04-25T14:05:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:46 smithi138 ceph-mon[32977]: pgmap v1070: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 540 op/s 2024-04-25T14:05:47.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 222516 18.84 MB/sec execute 509 sec latency 80.578 ms 2024-04-25T14:05:48.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 222905 18.85 MB/sec execute 510 sec latency 152.811 ms 2024-04-25T14:05:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:48 smithi137 ceph-mon[32568]: pgmap v1071: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-04-25T14:05:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:48 smithi094 ceph-mon[29195]: pgmap v1071: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-04-25T14:05:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:48 smithi138 ceph-mon[32977]: pgmap v1071: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 497 op/s 2024-04-25T14:05:49.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 223367 18.85 MB/sec execute 511 sec latency 73.055 ms 2024-04-25T14:05:50.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 223664 18.85 MB/sec execute 512 sec latency 295.950 ms 2024-04-25T14:05:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:50 smithi137 ceph-mon[32568]: pgmap v1072: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-04-25T14:05:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:50 smithi094 ceph-mon[29195]: pgmap v1072: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-04-25T14:05:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:50 smithi138 ceph-mon[32977]: pgmap v1072: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-04-25T14:05:51.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 224058 18.86 MB/sec execute 513 sec latency 78.802 ms 2024-04-25T14:05:51.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:51.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- 172.21.15.94:0/2922942608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288109890 msgr2=0x7f128810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 --2- 172.21.15.94:0/2922942608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288109890 0x7f128810bcb0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ca0 tx=0x7f128002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- 172.21.15.94:0/2922942608 shutdown_connections 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 --2- 172.21.15.94:0/2922942608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288109890 0x7f128810bcb0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 --2- 172.21.15.94:0/2922942608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288106f30 0x7f1288109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 --2- 172.21.15.94:0/2922942608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 0x7f12881069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- 172.21.15.94:0/2922942608 >> 172.21.15.94:0/2922942608 conn(0x7f12880fdbd0 msgr2=0x7f1288100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- 172.21.15.94:0/2922942608 shutdown_connections 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- 172.21.15.94:0/2922942608 wait complete. 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 Processor -- start 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.004+0000 7f128e270640 1 -- start start 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 0x7f128819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288109890 0x7f12881a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f128810f610 con 0x7f1288106f30 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f128810f490 con 0x7f1288109890 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128e270640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f128810f790 con 0x7f12881045d0 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48162/0 (socket says 172.21.15.94:48162) 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 -- 172.21.15.94:0/4140480473 learned_addr learned my addr 172.21.15.94:0/4140480473 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f1287fff640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 0x7f128819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f128c80d640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288109890 0x7f12881a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 -- 172.21.15.94:0/4140480473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 msgr2=0x7f128819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 0x7f128819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 -- 172.21.15.94:0/4140480473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288109890 msgr2=0x7f12881a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288109890 0x7f12881a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 -- 172.21.15.94:0/4140480473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12881a1dc0 con 0x7f1288106f30 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.005+0000 7f12877fe640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f127800e9f0 tx=0x7f127800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.006+0000 7f12857fa640 1 -- 172.21.15.94:0/4140480473 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127800cd80 con 0x7f1288106f30 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.006+0000 7f12857fa640 1 -- 172.21.15.94:0/4140480473 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12780108e0 con 0x7f1288106f30 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.006+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881a20b0 con 0x7f1288106f30 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.006+0000 7f12857fa640 1 -- 172.21.15.94:0/4140480473 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278018980 con 0x7f1288106f30 2024-04-25T14:05:52.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.006+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a2650 con 0x7f1288106f30 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.008+0000 7f12857fa640 1 -- 172.21.15.94:0/4140480473 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1278010430 con 0x7f1288106f30 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.008+0000 7f12951f3740 1 Processor -- start 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.008+0000 7f12951f3740 1 -- start start 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2ecad80 con 0x55f4a2bc4390 2024-04-25T14:05:52.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2e848b0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12951f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2ec6d40 con 0x55f4a2ba36b0 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48172/0 (socket says 172.21.15.94:48172) 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 -- 172.21.15.94:0/4111159583 learned_addr learned my addr 172.21.15.94:0/4111159583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f1287fff640 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12877fe640 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12857fa640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 0x7f12700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 -- 172.21.15.94:0/4111159583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 msgr2=0x55f4a2f66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f66c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 -- 172.21.15.94:0/4111159583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b9b4d0 msgr2=0x55f4a2f65fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f65fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 -- 172.21.15.94:0/4111159583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a2e55a40 con 0x55f4a2bc4390 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 0x7f12700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f12857fa640 1 -- 172.21.15.94:0/4140480473 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f12780def10 con 0x7f1288106f30 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.009+0000 7f128c80d640 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f1288075eb0 tx=0x7f12800094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eae86862d80f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f126e7fc640 1 -- 172.21.15.94:0/4111159583 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280042040 con 0x55f4a2bc4390 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f128c80d640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 0x7f12700b2080 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f128004d000 tx=0x7f128004e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f126e7fc640 1 -- 172.21.15.94:0/4111159583 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1280036070 con 0x55f4a2bc4390 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f126e7fc640 1 -- 172.21.15.94:0/4111159583 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128002f930 con 0x55f4a2bc4390 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f12951f3740 1 -- 172.21.15.94:0/4111159583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 msgr2=0x55f4a2f65a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.010+0000 7f12951f3740 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f1288075eb0 tx=0x7f12800094a0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- 172.21.15.94:0/4111159583 shutdown_connections 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- 172.21.15.94:0/4111159583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2bc4390 0x55f4a2f65a60 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- 172.21.15.94:0/4111159583 >> 172.21.15.94:0/4111159583 conn(0x55f4a2bb84e0 msgr2=0x55f4a2f61270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- 172.21.15.94:0/4111159583 shutdown_connections 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- 172.21.15.94:0/4111159583 wait complete. 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- start start 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 Processor -- start 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f6fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2bc4390 0x55f4a2f76e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2f67780 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2e51d10 con 0x55f4a2ba36b0 2024-04-25T14:05:52.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a2f67900 con 0x55f4a2bc4390 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f12951f3740 10 client.0 _ll_register_callbacks cb 0x55f4a2e7e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.011+0000 7f1287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48174/0 (socket says 172.21.15.94:48174) 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 -- 172.21.15.94:0/2712424039 learned_addr learned my addr 172.21.15.94:0/2712424039 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f128c80d640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2bc4390 0x55f4a2f76e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12877fe640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f6fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 -- 172.21.15.94:0/2712424039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2bc4390 msgr2=0x55f4a2f76e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2bc4390 0x55f4a2f76e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 -- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 msgr2=0x55f4a2f6fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f6fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 -- 172.21.15.94:0/2712424039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a2ca67d0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f1287fff640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f127c012700 tx=0x7f127c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa9c5db47565c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c004020 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f127c0041c0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1280036070 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f127c0043c0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12951f3740 10 client.23970 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.012+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4a2bdc3f0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1280036070 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f127c003a10 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 10 client.23970 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 10 client.23970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 20 client.23970 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2b82f60 0x55f4a2f7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 4 client.23970 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a2ed8990 con 0x55f4a2b82f60 2024-04-25T14:05:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.013+0000 7f12877fe640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2b82f60 0x55f4a2f7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.014+0000 7f12677fe640 10 client.23970 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:52.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.014+0000 7f12877fe640 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2b82f60 0x55f4a2f7d2f0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84228f41e177cb9 server_cookie=7ea518c29766bb85 in_seq=0 out_seq=0 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.017+0000 7f12677fe640 1 -- 172.21.15.94:0/2712424039 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4a2ed8990 con 0x55f4a2b82f60 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.017+0000 7f12677fe640 10 client.23970 handle_command_reply: tid=0 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.017+0000 7f12951f3740 1 client.23970 shutdown 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 msgr2=0x55f4a2f6f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f127c012700 tx=0x7f127c012bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 shutdown_connections 2024-04-25T14:05:52.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2b82f60 0x55f4a2f7d2f0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2bc4390 0x55f4a2f76e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f6fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12677fe640 0 client.23970 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 --2- 172.21.15.94:0/2712424039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f6f990 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 >> 172.21.15.94:0/2712424039 conn(0x55f4a2bb84e0 msgr2=0x55f4a2f603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 shutdown_connections 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 1 -- 172.21.15.94:0/2712424039 wait complete. 2024-04-25T14:05:52.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.018+0000 7f12951f3740 20 client.23970 trim_cache size 0 max 16384 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 Processor -- start 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 -- start start 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b82f60 0x55f4a2f6e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ab5a0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.036+0000 7f12951f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ad0e0 con 0x55f4a2b82f60 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12951f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12881abc70 con 0x55f4a2ba36b0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48178/0 (socket says 172.21.15.94:48178) 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 -- 172.21.15.94:0/3021375765 learned_addr learned my addr 172.21.15.94:0/3021375765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f128c80d640 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 -- 172.21.15.94:0/3021375765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 msgr2=0x55f4a2f64cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f64cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 -- 172.21.15.94:0/3021375765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b82f60 msgr2=0x55f4a2f6e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b82f60 0x55f4a2f6e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 -- 172.21.15.94:0/3021375765 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a2b9c0f0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f12877fe640 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f12780a7390 tx=0x7f12780e4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580386543ec372ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f126dffb640 1 -- 172.21.15.94:0/3021375765 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12780c1070 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f126dffb640 1 -- 172.21.15.94:0/3021375765 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12780b01b0 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.037+0000 7f126dffb640 1 -- 172.21.15.94:0/3021375765 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12780a1b90 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- 172.21.15.94:0/3021375765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 msgr2=0x55f4a2f64780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f12780a7390 tx=0x7f12780e4040 comp rx=0 tx=0).stop 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- 172.21.15.94:0/3021375765 shutdown_connections 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f64780 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 --2- 172.21.15.94:0/3021375765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2b82f60 0x55f4a2f6e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- 172.21.15.94:0/3021375765 >> 172.21.15.94:0/3021375765 conn(0x55f4a2bb84e0 msgr2=0x55f4a2f64070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- 172.21.15.94:0/3021375765 shutdown_connections 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- 172.21.15.94:0/3021375765 wait complete. 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.038+0000 7f12951f3740 1 -- start start 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 Processor -- start 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f74df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12881aca10 con 0x55f4a2b82f60 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12881ac340 con 0x55f4a2ba36b0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12881a9580 con 0x55f4a2b9b4d0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12951f3740 10 client.0 _ll_register_callbacks cb 0x7f127c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48194/0 (socket says 172.21.15.94:48194) 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 -- 172.21.15.94:0/2401475205 learned_addr learned my addr 172.21.15.94:0/2401475205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f128c80d640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f12877fe640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f74df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 -- 172.21.15.94:0/2401475205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2b9b4d0 msgr2=0x55f4a2f74df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f74df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 -- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 msgr2=0x55f4a2f75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.039+0000 7f1287fff640 1 -- 172.21.15.94:0/2401475205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a2e8b4b0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f1287fff640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f127c005eb0 tx=0x7f127c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866505ff0b478e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c01c6d0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f127c003fd0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12780b01b0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f127c036050 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 10 client.23976 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4a2f77470 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12780b01b0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f127c01cbe0 con 0x55f4a2b82f60 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 10 client.23976 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 10 client.23976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 20 client.23976 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2bc4390 0x55f4a2fe7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 4 client.23976 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.040+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4a2ed2800 con 0x55f4a2bc4390 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.041+0000 7f12877fe640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2bc4390 0x55f4a2fe7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.041+0000 7f12677fe640 10 client.23976 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:52.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.041+0000 7f12877fe640 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2bc4390 0x55f4a2fe7a30 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3956df06077a503 server_cookie=bd228b7f5963af7 in_seq=0 out_seq=0 2024-04-25T14:05:52.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12677fe640 1 -- 172.21.15.94:0/2401475205 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4a2ed2800 con 0x55f4a2bc4390 2024-04-25T14:05:52.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12677fe640 10 client.23976 handle_command_reply: tid=0 2024-04-25T14:05:52.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 client.23976 shutdown 2024-04-25T14:05:52.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 msgr2=0x55f4a2f748b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f127c005eb0 tx=0x7f127c00f810 comp rx=0 tx=0).stop 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 shutdown_connections 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f4a2bc4390 0x55f4a2fe7a30 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4a2ba36b0 0x55f4a2f75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a2b9b4d0 0x55f4a2f74df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12677fe640 0 client.23976 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 --2- 172.21.15.94:0/2401475205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f4a2b82f60 0x55f4a2f748b0 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 >> 172.21.15.94:0/2401475205 conn(0x55f4a2bb84e0 msgr2=0x55f4a2f6ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 shutdown_connections 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 1 -- 172.21.15.94:0/2401475205 wait complete. 2024-04-25T14:05:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.047+0000 7f12951f3740 20 client.23976 trim_cache size 0 max 16384 2024-04-25T14:05:52.049 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:52.049 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:05:52.049 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:05:52.049 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 msgr2=0x7f12700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 0x7f12700b2080 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f128004d000 tx=0x7f128004e000 comp rx=0 tx=0).stop 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 msgr2=0x7f128819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f127800e9f0 tx=0x7f127800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 shutdown_connections 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12700afb70 0x7f12700b2080 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1288109890 0x7f12881a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1288106f30 0x7f128819a7c0 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 --2- 172.21.15.94:0/4140480473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12881045d0 0x7f128819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 >> 172.21.15.94:0/4140480473 conn(0x7f12880fdbd0 msgr2=0x7f1288105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 shutdown_connections 2024-04-25T14:05:52.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.048+0000 7f128e270640 1 -- 172.21.15.94:0/4140480473 wait complete. 2024-04-25T14:05:52.060 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:05:52.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:52.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 224402 18.85 MB/sec execute 514 sec latency 89.864 ms 2024-04-25T14:05:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 -- 172.21.15.94:0/2674917326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d78101c50 msgr2=0x7f4d78102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/2674917326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d78101c50 0x7f4d78102060 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 -- 172.21.15.94:0/2674917326 shutdown_connections 2024-04-25T14:05:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/2674917326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d781095a0 0x7f4d7810bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/2674917326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78106a30 0x7f4d78108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/2674917326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d78101c50 0x7f4d78102060 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 -- 172.21.15.94:0/2674917326 >> 172.21.15.94:0/2674917326 conn(0x7f4d780fd1c0 msgr2=0x7f4d780ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 -- 172.21.15.94:0/2674917326 shutdown_connections 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.178+0000 7f4d7e79d640 1 -- 172.21.15.94:0/2674917326 wait complete. 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 Processor -- start 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 -- start start 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 0x7f4d7819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d78106a30 0x7f4d7819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810f400 con 0x7f4d781095a0 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810f280 con 0x7f4d78106a30 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7e79d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810f580 con 0x7f4d78101c50 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7df9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7d79b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 0x7f4d7819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7df9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48214/0 (socket says 172.21.15.94:48214) 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7df9c640 1 -- 172.21.15.94:0/909834712 learned_addr learned my addr 172.21.15.94:0/909834712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.179+0000 7f4d7cf9a640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d78106a30 0x7f4d7819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 -- 172.21.15.94:0/909834712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 msgr2=0x7f4d7819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 0x7f4d7819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 -- 172.21.15.94:0/909834712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d78106a30 msgr2=0x7f4d7819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d78106a30 0x7f4d7819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 -- 172.21.15.94:0/909834712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d781a1b70 con 0x7f4d781095a0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7d79b640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 0x7f4d7819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7df9c640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000eb10 tx=0x7f4d7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004b20 con 0x7f4d781095a0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70004cc0 con 0x7f4d781095a0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d781a1e60 con 0x7f4d781095a0 2024-04-25T14:05:52.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70010670 con 0x7f4d781095a0 2024-04-25T14:05:52.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.180+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a2270 con 0x7f4d781095a0 2024-04-25T14:05:52.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.181+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d780691e0 con 0x7f4d781095a0 2024-04-25T14:05:52.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.184+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4d70005000 con 0x7f4d781095a0 2024-04-25T14:05:52.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.184+0000 7f4d6e7fc640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 0x7f4d640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.184+0000 7f4d7d79b640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 0x7f4d640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.184+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4d700df440 con 0x7f4d781095a0 2024-04-25T14:05:52.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.185+0000 7f4d7d79b640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 0x7f4d640b1ff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.188+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d70053ff0 con 0x7f4d781095a0 2024-04-25T14:05:52.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.429+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d780fed30 con 0x7f4d781095a0 2024-04-25T14:05:52.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.430+0000 7f4d6e7fc640 1 -- 172.21.15.94:0/909834712 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f4d780fed30 con 0x7f4d781095a0 2024-04-25T14:05:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:05:52.430 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:05:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 msgr2=0x7f4d640b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 0x7f4d640b1ff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74031400 comp rx=0 tx=0).stop 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 msgr2=0x7f4d781a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000eb10 tx=0x7f4d7000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 shutdown_connections 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4d640afae0 0x7f4d640b1ff0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d781095a0 0x7f4d781a1500 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d78106a30 0x7f4d7819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 --2- 172.21.15.94:0/909834712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d78101c50 0x7f4d7819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 >> 172.21.15.94:0/909834712 conn(0x7f4d780fd1c0 msgr2=0x7f4d78107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 shutdown_connections 2024-04-25T14:05:52.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.432+0000 7f4d7e79d640 1 -- 172.21.15.94:0/909834712 wait complete. 2024-04-25T14:05:52.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:05:52.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 -- 172.21.15.94:0/3327862118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a00102260 msgr2=0x7f9a0010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3327862118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a00102260 0x7f9a0010bcc0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009ca0 tx=0x7f99f802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 -- 172.21.15.94:0/3327862118 shutdown_connections 2024-04-25T14:05:52.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3327862118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a00102260 0x7f9a0010bcc0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3327862118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 0x7f9a00101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3327862118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a001050b0 0x7f9a001054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 -- 172.21.15.94:0/3327862118 >> 172.21.15.94:0/3327862118 conn(0x7f9a000fd380 msgr2=0x7f9a000ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 -- 172.21.15.94:0/3327862118 shutdown_connections 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.560+0000 7f9a086aa640 1 -- 172.21.15.94:0/3327862118 wait complete. 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 Processor -- start 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 -- start start 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 0x7f9a0019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 0x7f9a0019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010f640 con 0x7f9a001050b0 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010f4c0 con 0x7f9a00102260 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a086aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010f7c0 con 0x7f9a00101760 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48244/0 (socket says 172.21.15.94:48244) 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 -- 172.21.15.94:0/3490132774 learned_addr learned my addr 172.21.15.94:0/3490132774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06446640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 0x7f9a0019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a05c45640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 0x7f9a0019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 -- 172.21.15.94:0/3490132774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 msgr2=0x7f9a0019a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 0x7f9a0019a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 -- 172.21.15.94:0/3490132774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 msgr2=0x7f9a0019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 0x7f9a0019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a06c47640 1 -- 172.21.15.94:0/3490132774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a1d20 con 0x7f9a001050b0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.561+0000 7f9a05c45640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 0x7f9a0019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f9a06c47640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f99f8004af0 tx=0x7f99f8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f99f77fe640 1 -- 172.21.15.94:0/3490132774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8003b70 con 0x7f9a001050b0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f99f77fe640 1 -- 172.21.15.94:0/3490132774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f8003d10 con 0x7f9a001050b0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a1fb0 con 0x7f9a001050b0 2024-04-25T14:05:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f99f77fe640 1 -- 172.21.15.94:0/3490132774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f802fb20 con 0x7f9a001050b0 2024-04-25T14:05:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.562+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a2470 con 0x7f9a001050b0 2024-04-25T14:05:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 Processor -- start 2024-04-25T14:05:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f99f77fe640 1 -- 172.21.15.94:0/3490132774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f99f802fcc0 con 0x7f9a001050b0 2024-04-25T14:05:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 -- start start 2024-04-25T14:05:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989cd390 0x558b98d6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 0x558b98d70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b98cd3d80 con 0x558b989a44d0 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b98c8da20 con 0x558b989ac6b0 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a0f635740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b98ccfd40 con 0x558b989cd390 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a06446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a06446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48250/0 (socket says 172.21.15.94:48250) 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.565+0000 7f9a06446640 1 -- 172.21.15.94:0/1464786260 learned_addr learned my addr 172.21.15.94:0/1464786260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a05c45640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 0x558b98d70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a05c45640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989cd390 0x558b98d6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 -- 172.21.15.94:0/1464786260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989cd390 msgr2=0x558b98d6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989cd390 0x558b98d6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 -- 172.21.15.94:0/1464786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 msgr2=0x558b98d70a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 0x558b98d70a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 -- 172.21.15.94:0/1464786260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b98c98c30 con 0x558b989a44d0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a05c45640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 0x558b98d70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f99f77fe640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 0x7f99ec0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a06446640 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009c60 tx=0x7f99fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be6d74230e2a597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f9a05c45640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 0x7f99ec0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f99f77fe640 1 -- 172.21.15.94:0/3490132774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99f81002b0 con 0x7f9a001050b0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f99f4bfe640 1 -- 172.21.15.94:0/1464786260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004220 con 0x558b989a44d0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f99f4bfe640 1 -- 172.21.15.94:0/1464786260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc0043c0 con 0x558b989a44d0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.566+0000 7f99f4bfe640 1 -- 172.21.15.94:0/1464786260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004d50 con 0x558b989a44d0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 -- 172.21.15.94:0/1464786260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 msgr2=0x558b98d6fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009c60 tx=0x7f99fc011810 comp rx=0 tx=0).stop 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a05c45640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 0x7f99ec0b2080 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x558b98d70860 tx=0x7f99f0014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 -- 172.21.15.94:0/1464786260 shutdown_connections 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989ac6b0 0x558b98d70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989a44d0 0x558b98d6fd90 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 --2- 172.21.15.94:0/1464786260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989cd390 0x558b98d6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 -- 172.21.15.94:0/1464786260 >> 172.21.15.94:0/1464786260 conn(0x558b989c14e0 msgr2=0x558b98d6b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 -- 172.21.15.94:0/1464786260 shutdown_connections 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.567+0000 7f9a0f635740 1 -- 172.21.15.94:0/1464786260 wait complete. 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 -- start start 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 Processor -- start 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989cd390 0x558b98d79c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b9898bf60 0x558b98d80bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b98d71530 con 0x558b989ac6b0 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b98c5adc0 con 0x558b989cd390 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b98d716b0 con 0x558b9898bf60 2024-04-25T14:05:52.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a0f635740 10 client.0 _ll_register_callbacks cb 0x558b98c76600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48264/0 (socket says 172.21.15.94:48264) 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 -- 172.21.15.94:0/2959456858 learned_addr learned my addr 172.21.15.94:0/2959456858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06c47640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b9898bf60 0x558b98d80bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a05c45640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989cd390 0x558b98d79c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 -- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b9898bf60 msgr2=0x558b98d80bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b9898bf60 0x558b98d80bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 -- 172.21.15.94:0/2959456858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989cd390 msgr2=0x558b98d79c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989cd390 0x558b98d79c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.568+0000 7f9a06446640 1 -- 172.21.15.94:0/2959456858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b989e53f0 con 0x558b989ac6b0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a06446640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0056a0 tx=0x7f99fc005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d62ed805e222ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc027030 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99fc019430 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc0043c0 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f99fc01d040 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 10 client.23994 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558b98aaf7d0 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99fc0043c0 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f99fc019c00 con 0x558b989ac6b0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 10 client.23994 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 10 client.23994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 20 client.23994 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989613a0 0x558b98d87170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.569+0000 7f9a0f635740 4 client.23994 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.570+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b98ce1990 con 0x558b989613a0 2024-04-25T14:05:52.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.570+0000 7f9a05c45640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989613a0 0x558b98d87170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.570+0000 7f99e9ffb640 10 client.23994 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:52.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.570+0000 7f9a05c45640 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989613a0 0x558b98d87170 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d35837113bcdbfd server_cookie=ee9f5dcb0713c495 in_seq=0 out_seq=0 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2959456858 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b98ce1990 con 0x558b989613a0 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f99e9ffb640 10 client.23994 handle_command_reply: tid=0 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 client.23994 shutdown 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 msgr2=0x558b98d79750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0056a0 tx=0x7f99fc005250 comp rx=0 tx=0).stop 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 shutdown_connections 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989613a0 0x558b98d87170 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b9898bf60 0x558b98d80bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989cd390 0x558b98d79c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f99e9ffb640 0 client.23994 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:52.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2959456858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d79750 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 >> 172.21.15.94:0/2959456858 conn(0x558b989c14e0 msgr2=0x558b98d6df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 shutdown_connections 2024-04-25T14:05:52.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 1 -- 172.21.15.94:0/2959456858 wait complete. 2024-04-25T14:05:52.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.571+0000 7f9a0f635740 20 client.23994 trim_cache size 0 max 16384 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 Processor -- start 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 -- start start 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989613a0 0x558b98d6b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989ac6b0 0x558b98d79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ab500 con 0x558b9898bf60 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ad040 con 0x558b989613a0 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a0f635740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001abbd0 con 0x558b989ac6b0 2024-04-25T14:05:52.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48278/0 (socket says 172.21.15.94:48278) 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 -- 172.21.15.94:0/954716971 learned_addr learned my addr 172.21.15.94:0/954716971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a06446640 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989613a0 0x558b98d6b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a06c47640 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989ac6b0 0x558b98d79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 -- 172.21.15.94:0/954716971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989ac6b0 msgr2=0x558b98d79860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989ac6b0 0x558b98d79860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 -- 172.21.15.94:0/954716971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989613a0 msgr2=0x558b98d6b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.587+0000 7f9a05c45640 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989613a0 0x558b98d6b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.588+0000 7f9a05c45640 1 -- 172.21.15.94:0/954716971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b98d77860 con 0x558b9898bf60 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.588+0000 7f9a05c45640 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009e00 tx=0x7f99f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e1e3e436ead822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.588+0000 7f99ebfff640 1 -- 172.21.15.94:0/954716971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0018030 con 0x558b9898bf60 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.588+0000 7f99ebfff640 1 -- 172.21.15.94:0/954716971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f001a050 con 0x558b9898bf60 2024-04-25T14:05:52.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.588+0000 7f99ebfff640 1 -- 172.21.15.94:0/954716971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0005240 con 0x558b9898bf60 2024-04-25T14:05:52.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- 172.21.15.94:0/954716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 msgr2=0x558b98d79320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009e00 tx=0x7f99f0017000 comp rx=0 tx=0).stop 2024-04-25T14:05:52.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- 172.21.15.94:0/954716971 shutdown_connections 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989ac6b0 0x558b98d79860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b9898bf60 0x558b98d79320 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 --2- 172.21.15.94:0/954716971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b989613a0 0x558b98d6b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- 172.21.15.94:0/954716971 >> 172.21.15.94:0/954716971 conn(0x558b989c14e0 msgr2=0x558b98d6aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- 172.21.15.94:0/954716971 shutdown_connections 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- 172.21.15.94:0/954716971 wait complete. 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 -- start start 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.589+0000 7f9a0f635740 1 Processor -- start 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 0x558b98d80070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b9898bf60 0x558b98d7cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac970 con 0x558b989ac6b0 2024-04-25T14:05:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac2a0 con 0x558b9898bf60 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a94e0 con 0x558b989613a0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 10 client.0 _ll_register_callbacks cb 0x7f99fc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48286/0 (socket says 172.21.15.94:48286) 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06446640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 0x558b98d80070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 -- 172.21.15.94:0/2026986763 learned_addr learned my addr 172.21.15.94:0/2026986763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a05c45640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b9898bf60 0x558b98d7cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 -- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 msgr2=0x558b98d80070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 0x558b98d80070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 -- 172.21.15.94:0/2026986763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b9898bf60 msgr2=0x558b98d7cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b9898bf60 0x558b98d7cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 -- 172.21.15.94:0/2026986763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b98d7d810 con 0x558b989ac6b0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06446640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 0x558b98d80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a06c47640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f99f80c58d0 tx=0x7f99f80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c086f98238b4fb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f99e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f80e1020 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f99f80c4c50 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.590+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f001a050 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.591+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f99f80c8df0 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.591+0000 7f9a0f635740 10 client.24000 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.591+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558b98df2a40 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f001a050 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f99f80c4910 con 0x558b989ac6b0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 10 client.24000 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 10 client.24000 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 20 client.24000 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989cd390 0x558b98df5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 4 client.24000 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b98cdb800 con 0x558b989cd390 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.592+0000 7f9a06446640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989cd390 0x558b98df5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f99e9ffb640 10 client.24000 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a06446640 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989cd390 0x558b98df5430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=171e87704e221163 server_cookie=4c780c2a88ee98d2 in_seq=0 out_seq=0 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f99e9ffb640 1 -- 172.21.15.94:0/2026986763 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b98cdb800 con 0x558b989cd390 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f99e9ffb640 10 client.24000 handle_command_reply: tid=0 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 client.24000 shutdown 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 msgr2=0x558b98d7d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f99f80c58d0 tx=0x7f99f80095c0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 shutdown_connections 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x558b989cd390 0x558b98df5430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b989ac6b0 0x558b98d7d1a0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b9898bf60 0x558b98d7cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 --2- 172.21.15.94:0/2026986763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b989613a0 0x558b98d80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 >> 172.21.15.94:0/2026986763 conn(0x558b989c14e0 msgr2=0x558b98d79290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f99e9ffb640 0 client.24000 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.593+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 shutdown_connections 2024-04-25T14:05:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a0f635740 1 -- 172.21.15.94:0/2026986763 wait complete. 2024-04-25T14:05:52.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a0f635740 20 client.24000 trim_cache size 0 max 16384 2024-04-25T14:05:52.598 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 msgr2=0x7f99ec0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 0x7f99ec0b2080 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x558b98d70860 tx=0x7f99f0014040 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 msgr2=0x7f9a001a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f99f8004af0 tx=0x7f99f8031700 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 shutdown_connections 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f99ec0afb70 0x7f99ec0b2080 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a001050b0 0x7f9a001a16b0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a00102260 0x7f9a0019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 --2- 172.21.15.94:0/3490132774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00101760 0x7f9a0019a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.594+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 >> 172.21.15.94:0/3490132774 conn(0x7f9a000fd380 msgr2=0x7f9a00107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.595+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 shutdown_connections 2024-04-25T14:05:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.595+0000 7f9a086aa640 1 -- 172.21.15.94:0/3490132774 wait complete. 2024-04-25T14:05:52.606 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:52 smithi094 ceph-mon[29195]: pgmap v1073: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-04-25T14:05:52.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:05:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:52 smithi137 ceph-mon[32568]: pgmap v1073: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-04-25T14:05:52.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 -- 172.21.15.94:0/2001163030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 msgr2=0x7f2778104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 --2- 172.21.15.94:0/2001163030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f2778104030 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f2768009900 tx=0x7f276802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:52.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 -- 172.21.15.94:0/2001163030 shutdown_connections 2024-04-25T14:05:52.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 --2- 172.21.15.94:0/2001163030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2778108920 0x7f277810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 --2- 172.21.15.94:0/2001163030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f2778104030 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 --2- 172.21.15.94:0/2001163030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27781029e0 0x7f27781036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 -- 172.21.15.94:0/2001163030 >> 172.21.15.94:0/2001163030 conn(0x7f27780fdbd0 msgr2=0x7f2778100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 -- 172.21.15.94:0/2001163030 shutdown_connections 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.727+0000 7f277da92640 1 -- 172.21.15.94:0/2001163030 wait complete. 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 Processor -- start 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 -- start start 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27781029e0 0x7f277806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2778108920 0x7f27781a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f277810e4e0 con 0x7f2778103c20 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f277810e360 con 0x7f27781029e0 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f277da92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f277810e660 con 0x7f2778108920 2024-04-25T14:05:52.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48308/0 (socket says 172.21.15.94:48308) 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 -- 172.21.15.94:0/3300098904 learned_addr learned my addr 172.21.15.94:0/3300098904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f27777fe640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27781029e0 0x7f277806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2777fff640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2778108920 0x7f27781a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 -- 172.21.15.94:0/3300098904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2778108920 msgr2=0x7f27781a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2778108920 0x7f27781a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 -- 172.21.15.94:0/3300098904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27781029e0 msgr2=0x7f277806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27781029e0 0x7f277806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.728+0000 7f2776ffd640 1 -- 172.21.15.94:0/3300098904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a2070 con 0x7f2778103c20 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f2776ffd640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f2768006010 tx=0x7f2768004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f2774ff9640 1 -- 172.21.15.94:0/3300098904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276803f030 con 0x7f2778103c20 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f2774ff9640 1 -- 172.21.15.94:0/3300098904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2768003cb0 con 0x7f2778103c20 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a2300 con 0x7f2778103c20 2024-04-25T14:05:52.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f2774ff9640 1 -- 172.21.15.94:0/3300098904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768035070 con 0x7f2778103c20 2024-04-25T14:05:52.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.729+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a27f0 con 0x7f2778103c20 2024-04-25T14:05:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.731+0000 7f2784a1b740 1 Processor -- start 2024-04-25T14:05:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.731+0000 7f2784a1b740 1 -- start start 2024-04-25T14:05:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.731+0000 7f2774ff9640 1 -- 172.21.15.94:0/3300098904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2768042030 con 0x7f2778103c20 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019eded4d0 0x56019f1b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56019f11cd80 con 0x56019ee16390 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56019f0d6a20 con 0x56019edf56b0 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2784a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56019f118d40 con 0x56019eded4d0 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48324/0 (socket says 172.21.15.94:48324) 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 -- 172.21.15.94:0/2880846042 learned_addr learned my addr 172.21.15.94:0/2880846042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f27777fe640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019eded4d0 0x56019f1b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2776ffd640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 -- 172.21.15.94:0/2880846042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019eded4d0 msgr2=0x56019f1b8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019eded4d0 0x56019f1b8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 -- 172.21.15.94:0/2880846042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 msgr2=0x56019f1b9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1b9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 -- 172.21.15.94:0/2880846042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019f0e1c30 con 0x56019ee16390 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2776ffd640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2774ff9640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 0x7f27600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.732+0000 7f2777fff640 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f2770009f20 tx=0x7f277000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dc6760bc9de0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.733+0000 7f27777fe640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 0x7f27600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.733+0000 7f2774ff9640 1 -- 172.21.15.94:0/3300098904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f27680ff350 con 0x7f2778103c20 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.733+0000 7f275dffb640 1 -- 172.21.15.94:0/2880846042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770015070 con 0x56019ee16390 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.733+0000 7f275dffb640 1 -- 172.21.15.94:0/2880846042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277001ab10 con 0x56019ee16390 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.733+0000 7f275dffb640 1 -- 172.21.15.94:0/2880846042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770011bc0 con 0x56019ee16390 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- 172.21.15.94:0/2880846042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 msgr2=0x56019f1b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f2770009f20 tx=0x7f277000f7f0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f27777fe640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 0x7f27600b2000 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x56019f1b8530 tx=0x7f276c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- 172.21.15.94:0/2880846042 shutdown_connections 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1b9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019eded4d0 0x56019f1b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2880846042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1b8810 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- 172.21.15.94:0/2880846042 >> 172.21.15.94:0/2880846042 conn(0x56019ee0a4e0 msgr2=0x56019f1b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- 172.21.15.94:0/2880846042 shutdown_connections 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- 172.21.15.94:0/2880846042 wait complete. 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 -- start start 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.734+0000 7f2784a1b740 1 Processor -- start 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 0x56019f1c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56019f1ba530 con 0x56019ee16390 2024-04-25T14:05:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56019f0a3dc0 con 0x56019eded4d0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56019f1ba6b0 con 0x56019edf56b0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 10 client.0 _ll_register_callbacks cb 0x56019f0bf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48328/0 (socket says 172.21.15.94:48328) 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f27777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 -- 172.21.15.94:0/962153598 learned_addr learned my addr 172.21.15.94:0/962153598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2776ffd640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 0x56019f1c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 -- 172.21.15.94:0/962153598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 msgr2=0x56019f1c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 0x56019f1c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 -- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 msgr2=0x56019f1c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 -- 172.21.15.94:0/962153598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019ee2e3f0 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2776ffd640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 0x56019f1c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2777fff640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f2770009370 tx=0x7f2770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb1b9e591833f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770028060 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f2770004dc0 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.735+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277001ab10 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2770015070 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 10 client.24012 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56019eef87d0 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277001ab10 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f2770005110 con 0x56019ee16390 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 10 client.24012 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 10 client.24012 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 20 client.24012 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edd4f60 0x56019f1d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 4 client.24012 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56019f12a990 con 0x56019edd4f60 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.736+0000 7f27777fe640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edd4f60 0x56019f1d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.737+0000 7f2756ffd640 10 client.24012 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.737+0000 7f27777fe640 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edd4f60 0x56019f1d0030 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69bd79a3060546d3 server_cookie=4b6ced1d8ad3f6fe in_seq=0 out_seq=0 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.737+0000 7f2756ffd640 1 -- 172.21.15.94:0/962153598 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56019f12a990 con 0x56019edd4f60 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.737+0000 7f2756ffd640 10 client.24012 handle_command_reply: tid=0 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.737+0000 7f2784a1b740 1 client.24012 shutdown 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 msgr2=0x56019f1c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f2770009370 tx=0x7f2770004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 shutdown_connections 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edd4f60 0x56019f1d0030 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019ee16390 0x56019f1c9ad0 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019edf56b0 0x56019f1c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2756ffd640 0 client.24012 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 --2- 172.21.15.94:0/962153598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 >> 172.21.15.94:0/962153598 conn(0x56019ee0a4e0 msgr2=0x56019f1b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 shutdown_connections 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 1 -- 172.21.15.94:0/962153598 wait complete. 2024-04-25T14:05:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.738+0000 7f2784a1b740 20 client.24012 trim_cache size 0 max 16384 2024-04-25T14:05:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:52 smithi138 ceph-mon[32977]: pgmap v1073: 97 pgs: 97 active+clean; 285 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 Processor -- start 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 -- start start 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27781ab570 con 0x56019edf56b0 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27781ad0b0 con 0x56019eded4d0 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2784a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27781abc40 con 0x56019ee16390 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48336/0 (socket says 172.21.15.94:48336) 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 -- 172.21.15.94:0/1020117592 learned_addr learned my addr 172.21.15.94:0/1020117592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2777fff640 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f27777fe640 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 -- 172.21.15.94:0/1020117592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 msgr2=0x56019f1b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 -- 172.21.15.94:0/1020117592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 msgr2=0x56019f1b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.752+0000 7f2776ffd640 1 -- 172.21.15.94:0/1020117592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019f1c23e0 con 0x56019edf56b0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.753+0000 7f2776ffd640 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f2768032af0 tx=0x7f27680d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc151ac12af5d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.753+0000 7f275d7fa640 1 -- 172.21.15.94:0/1020117592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768049040 con 0x56019edf56b0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.753+0000 7f275d7fa640 1 -- 172.21.15.94:0/1020117592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27680c72a0 con 0x56019edf56b0 2024-04-25T14:05:52.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.753+0000 7f275d7fa640 1 -- 172.21.15.94:0/1020117592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27680c7790 con 0x56019edf56b0 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- 172.21.15.94:0/1020117592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 msgr2=0x56019f1c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f2768032af0 tx=0x7f27680d0040 comp rx=0 tx=0).stop 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- 172.21.15.94:0/1020117592 shutdown_connections 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019edf56b0 0x56019f1c99f0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 --2- 172.21.15.94:0/1020117592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019eded4d0 0x56019f1b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- 172.21.15.94:0/1020117592 >> 172.21.15.94:0/1020117592 conn(0x56019ee0a4e0 msgr2=0x56019f1bd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- 172.21.15.94:0/1020117592 shutdown_connections 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- 172.21.15.94:0/1020117592 wait complete. 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 -- start start 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.754+0000 7f2784a1b740 1 Processor -- start 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27781ac9e0 con 0x56019eded4d0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27781ac310 con 0x56019edf56b0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27781a9550 con 0x56019ee16390 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 10 client.0 _ll_register_callbacks cb 0x7f276c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48348/0 (socket says 172.21.15.94:48348) 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 -- 172.21.15.94:0/2849749562 learned_addr learned my addr 172.21.15.94:0/2849749562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2777fff640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2776ffd640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 -- 172.21.15.94:0/2849749562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 msgr2=0x56019f1c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 -- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 msgr2=0x56019f1c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 -- 172.21.15.94:0/2849749562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019f236260 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f27777fe640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f276c005fd0 tx=0x7f276c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8eb4b4b1bf735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c02aa30 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f276c033420 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.755+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27680c72a0 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f276c033620 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 10 client.24018 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56019f237580 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27680c72a0 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f276c01e8d0 con 0x56019eded4d0 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 10 client.24018 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 10 client.24018 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:05:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 20 client.24018 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edaa3a0 0x56019f239f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 4 client.24018 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56019f124800 con 0x56019edaa3a0 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.756+0000 7f2776ffd640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edaa3a0 0x56019f239f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2776ffd640 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edaa3a0 0x56019f239f50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de532df954956b29 server_cookie=e9289719ab117b2b in_seq=0 out_seq=0 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2756ffd640 10 client.24018 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2756ffd640 1 -- 172.21.15.94:0/2849749562 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56019f124800 con 0x56019edaa3a0 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2756ffd640 10 client.24018 handle_command_reply: tid=0 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 client.24018 shutdown 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 msgr2=0x56019f1c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f276c005fd0 tx=0x7f276c0046c0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 shutdown_connections 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56019edaa3a0 0x56019f239f50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019ee16390 0x56019f1c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2756ffd640 0 client.24018 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56019edf56b0 0x56019f1c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 --2- 172.21.15.94:0/2849749562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56019eded4d0 0x56019f1c9950 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 >> 172.21.15.94:0/2849749562 conn(0x56019ee0a4e0 msgr2=0x56019f1c2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.757+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 shutdown_connections 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f2784a1b740 1 -- 172.21.15.94:0/2849749562 wait complete. 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f2784a1b740 20 client.24018 trim_cache size 0 max 16384 2024-04-25T14:05:52.763 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 msgr2=0x7f27600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 0x7f27600b2000 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x56019f1b8530 tx=0x7f276c00b410 comp rx=0 tx=0).stop 2024-04-25T14:05:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 msgr2=0x7f277819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f2768006010 tx=0x7f2768004340 comp rx=0 tx=0).stop 2024-04-25T14:05:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 shutdown_connections 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f27600afaf0 0x7f27600b2000 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2778108920 0x7f27781a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2778103c20 0x7f277819f350 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 --2- 172.21.15.94:0/3300098904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27781029e0 0x7f277806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 >> 172.21.15.94:0/3300098904 conn(0x7f27780fdbd0 msgr2=0x7f2778104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 shutdown_connections 2024-04-25T14:05:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.758+0000 7f277da92640 1 -- 172.21.15.94:0/3300098904 wait complete. 2024-04-25T14:05:52.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 -- 172.21.15.94:0/2327236597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c103c20 msgr2=0x7f1c6c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/2327236597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c103c20 0x7f1c6c104030 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009900 tx=0x7f1c6002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 -- 172.21.15.94:0/2327236597 shutdown_connections 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/2327236597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c108920 0x7f1c6c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/2327236597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c103c20 0x7f1c6c104030 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/2327236597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 -- 172.21.15.94:0/2327236597 >> 172.21.15.94:0/2327236597 conn(0x7f1c6c0fdbd0 msgr2=0x7f1c6c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 -- 172.21.15.94:0/2327236597 shutdown_connections 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 -- 172.21.15.94:0/2327236597 wait complete. 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.889+0000 7f1c71f9d640 1 Processor -- start 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 -- start start 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 0x7f1c6c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 0x7f1c6c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10e500 con 0x7f1c6c1029e0 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10e380 con 0x7f1c6c103c20 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c71f9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10e680 con 0x7f1c6c108920 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48360/0 (socket says 172.21.15.94:48360) 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/357826107 learned_addr learned my addr 172.21.15.94:0/357826107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6affd640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 0x7f1c6c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 0x7f1c6c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/357826107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 msgr2=0x7f1c6c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 0x7f1c6c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/357826107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 msgr2=0x7f1c6c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 0x7f1c6c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/357826107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c1a2080 con 0x7f1c6c1029e0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6affd640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 0x7f1c6c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 0x7f1c6c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.890+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00eb50 tx=0x7f1c5c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.891+0000 7f1c68ff9640 1 -- 172.21.15.94:0/357826107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c003b70 con 0x7f1c6c1029e0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.891+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c1a2370 con 0x7f1c6c1029e0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.891+0000 7f1c68ff9640 1 -- 172.21.15.94:0/357826107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5c00ee50 con 0x7f1c6c1029e0 2024-04-25T14:05:52.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.891+0000 7f1c68ff9640 1 -- 172.21.15.94:0/357826107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c010790 con 0x7f1c6c1029e0 2024-04-25T14:05:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.891+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a28c0 con 0x7f1c6c1029e0 2024-04-25T14:05:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 Processor -- start 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 -- start start 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c68ff9640 1 -- 172.21.15.94:0/357826107 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1c5c010930 con 0x7f1c6c1029e0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 0x5652c7cea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ceba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652c7c4ed80 con 0x5652c791f4d0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5652c7c08a20 con 0x5652c7948390 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5652c7c4ad40 con 0x5652c79276b0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.893+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 0x5652c7cea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48374/0 (socket says 172.21.15.94:48374) 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3270176024 learned_addr learned my addr 172.21.15.94:0/3270176024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6affd640 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ceba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3270176024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 msgr2=0x5652c7ceba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ceba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3270176024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 msgr2=0x5652c7cea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 0x5652c7cea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3270176024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c7c13c30 con 0x5652c791f4d0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c68ff9640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 0x5652c7cea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0e5750 tx=0x7f1c5c0e5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1267566fb31a5f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c68ff9640 1 -- 172.21.15.94:0/357826107 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c5c0df5e0 con 0x7f1c6c1029e0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c51ffb640 1 -- 172.21.15.94:0/3270176024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a77f0 con 0x5652c791f4d0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c51ffb640 1 -- 172.21.15.94:0/3270176024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5c0b01c0 con 0x5652c791f4d0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.894+0000 7f1c51ffb640 1 -- 172.21.15.94:0/3270176024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a7190 con 0x5652c791f4d0 2024-04-25T14:05:52.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3270176024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 msgr2=0x5652c7cead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x5652c7ceb860 tx=0x7f1c6400b550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0e5750 tx=0x7f1c5c0e5840 comp rx=0 tx=0).stop 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3270176024 shutdown_connections 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ceba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c791f4d0 0x5652c7cead90 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3270176024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c7948390 0x5652c7cea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3270176024 >> 172.21.15.94:0/3270176024 conn(0x5652c793c4e0 msgr2=0x5652c7ce6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3270176024 shutdown_connections 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.895+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3270176024 wait complete. 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 -- start start 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 Processor -- start 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cf4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652c7cec3b0 con 0x5652c7948390 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5652c7bd5dc0 con 0x5652c791f4d0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5652c7cec530 con 0x5652c79276b0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c78e4d740 10 client.0 _ll_register_callbacks cb 0x5652c7bf1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48380/0 (socket says 172.21.15.94:48380) 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c6bfff640 1 -- 172.21.15.94:0/2634042087 learned_addr learned my addr 172.21.15.94:0/2634042087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.896+0000 7f1c6affd640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cf4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 -- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 msgr2=0x5652c7cf4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cf4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 -- 172.21.15.94:0/2634042087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 msgr2=0x5652c7cf43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 -- 172.21.15.94:0/2634042087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c79603f0 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6affd640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cf4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401e7d0 tx=0x7f1c640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a719351a339a39c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64010b50 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c64010cf0 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652c7c5c990 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1c64005020 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c78e4d740 10 client.24030 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5652c7a2a7d0 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5652c7c5c990 con 0x5652c7948390 2024-04-25T14:05:52.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.897+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6401f420 con 0x5652c7948390 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 10 client.24030 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 10 client.24030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 20 client.24030 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7906f60 0x5652c7d021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 4 client.24030 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652c7c56800 con 0x5652c7906f60 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7906f60 0x5652c7d021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c4affd640 10 client.24030 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.898+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7906f60 0x5652c7d021b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2afe7eb428676b6e server_cookie=1546158303449857 in_seq=0 out_seq=0 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c4affd640 1 -- 172.21.15.94:0/2634042087 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652c7c56800 con 0x5652c7906f60 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c4affd640 10 client.24030 handle_command_reply: tid=0 2024-04-25T14:05:52.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 client.24030 shutdown 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 msgr2=0x5652c7cfb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401e7d0 tx=0x7f1c640046c0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 shutdown_connections 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7906f60 0x5652c7d021b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7948390 0x5652c7cfb860 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c4affd640 0 client.24030 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cf4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/2634042087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 >> 172.21.15.94:0/2634042087 conn(0x5652c793c4e0 msgr2=0x5652c7ce8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 shutdown_connections 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 1 -- 172.21.15.94:0/2634042087 wait complete. 2024-04-25T14:05:52.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.899+0000 7f1c78e4d740 20 client.24030 trim_cache size 0 max 16384 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 Processor -- start 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 -- start start 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cfb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ce4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ab560 con 0x5652c7906f60 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ad0a0 con 0x5652c791f4d0 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1abc30 con 0x5652c79276b0 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48382/0 (socket says 172.21.15.94:48382) 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3197244339 learned_addr learned my addr 172.21.15.94:0/3197244339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6affd640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cfb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ce4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3197244339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 msgr2=0x5652c7ce4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ce4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3197244339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 msgr2=0x5652c7cfb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cfb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/3197244339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c7cefc80 con 0x5652c7906f60 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ce4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.913+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0dd3b0 tx=0x7f1c5c0e5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc611883639fd30e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c517fa640 1 -- 172.21.15.94:0/3197244339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0c0030 con 0x5652c7906f60 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c517fa640 1 -- 172.21.15.94:0/3197244339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5c0a8120 con 0x5652c7906f60 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c517fa640 1 -- 172.21.15.94:0/3197244339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a7370 con 0x5652c7906f60 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3197244339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 msgr2=0x5652c7cf32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0dd3b0 tx=0x7f1c5c0e5840 comp rx=0 tx=0).stop 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3197244339 shutdown_connections 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7ce4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cfb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/3197244339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf32c0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3197244339 >> 172.21.15.94:0/3197244339 conn(0x5652c793c4e0 msgr2=0x5652c7cf2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3197244339 shutdown_connections 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.914+0000 7f1c78e4d740 1 -- 172.21.15.94:0/3197244339 wait complete. 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 -- start start 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 Processor -- start 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cfa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ac9d0 con 0x5652c7906f60 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1ac300 con 0x5652c791f4d0 2024-04-25T14:05:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1a9540 con 0x5652c79276b0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c78e4d740 10 client.0 _ll_register_callbacks cb 0x7f1c6400bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48398/0 (socket says 172.21.15.94:48398) 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/1801904510 learned_addr learned my addr 172.21.15.94:0/1801904510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c6bfff640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cfa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.915+0000 7f1c6affd640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 msgr2=0x5652c7cfa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cfa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/1801904510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 msgr2=0x5652c7cf90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 -- 172.21.15.94:0/1801904510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c7cfbcc0 con 0x5652c7906f60 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c6b7fe640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0a7870 tx=0x7f1c5c0e5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e93754a2baf57f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0c0020 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1c5c0e5d40 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5c0a8120 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c0a7370 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c78e4d740 10 client.24036 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5652c7cfb690 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5c0a8120 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1c5c00b2c0 con 0x5652c7906f60 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.916+0000 7f1c78e4d740 10 client.24036 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 10 client.24036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 20 client.24036 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7948390 0x5652c7d6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 4 client.24036 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652c7bd1610 con 0x5652c7948390 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c6affd640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7948390 0x5652c7d6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c4affd640 10 client.24036 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c6affd640 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7948390 0x5652c7d6c660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e0c4c13ba7cfcba server_cookie=9ff02d8222b95ed3 in_seq=0 out_seq=0 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c4affd640 1 -- 172.21.15.94:0/1801904510 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652c7bd1610 con 0x5652c7948390 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c4affd640 10 client.24036 handle_command_reply: tid=0 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 client.24036 shutdown 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 msgr2=0x5652c7cf8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0a7870 tx=0x7f1c5c0e5680 comp rx=0 tx=0).stop 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 shutdown_connections 2024-04-25T14:05:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.917+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x5652c7948390 0x5652c7d6c660 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5652c79276b0 0x5652c7cfa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c4affd640 0 client.24036 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5652c791f4d0 0x5652c7cf90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 --2- 172.21.15.94:0/1801904510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652c7906f60 0x5652c7cf8bb0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 >> 172.21.15.94:0/1801904510 conn(0x5652c793c4e0 msgr2=0x5652c7ce8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 shutdown_connections 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 1 -- 172.21.15.94:0/1801904510 wait complete. 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c78e4d740 20 client.24036 trim_cache size 0 max 16384 2024-04-25T14:05:52.925 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:52.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 msgr2=0x7f1c540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x5652c7ceb860 tx=0x7f1c6400b550 comp rx=0 tx=0).stop 2024-04-25T14:05:52.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 msgr2=0x7f1c6c06a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00eb50 tx=0x7f1c5c00c6a0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 shutdown_connections 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1c540afaf0 0x7f1c540b2000 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c6c108920 0x7f1c6c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c103c20 0x7f1c6c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 --2- 172.21.15.94:0/357826107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c6c1029e0 0x7f1c6c06a900 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 >> 172.21.15.94:0/357826107 conn(0x7f1c6c0fdbd0 msgr2=0x7f1c6c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 shutdown_connections 2024-04-25T14:05:52.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:52.918+0000 7f1c71f9d640 1 -- 172.21.15.94:0/357826107 wait complete. 2024-04-25T14:05:52.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:05:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 -- 172.21.15.94:0/2699597926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434103c30 msgr2=0x7fd434104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2699597926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434103c30 0x7fd434104040 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009900 tx=0x7fd41c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 -- 172.21.15.94:0/2699597926 shutdown_connections 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2699597926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434108930 0x7fd43410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2699597926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434103c30 0x7fd434104040 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2699597926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 0x7fd4341036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.043+0000 7fd43abc5640 1 -- 172.21.15.94:0/2699597926 >> 172.21.15.94:0/2699597926 conn(0x7fd4340fdbc0 msgr2=0x7fd434100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- 172.21.15.94:0/2699597926 shutdown_connections 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- 172.21.15.94:0/2699597926 wait complete. 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 Processor -- start 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- start start 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 0x7fd434199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 0x7fd434199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410e510 con 0x7fd434108930 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410e390 con 0x7fd434103c30 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd43abc5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd43410e690 con 0x7fd4341029f0 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.044+0000 7fd439162640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd438961640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 0x7fd434199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48420/0 (socket says 172.21.15.94:48420) 2024-04-25T14:05:53.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 -- 172.21.15.94:0/2397064728 learned_addr learned my addr 172.21.15.94:0/2397064728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd433fff640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 0x7fd434199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 -- 172.21.15.94:0/2397064728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 msgr2=0x7fd434199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 0x7fd434199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 -- 172.21.15.94:0/2397064728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 msgr2=0x7fd434199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 0x7fd434199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 -- 172.21.15.94:0/2397064728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341a0c80 con 0x7fd434108930 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd433fff640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 0x7fd434199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd439162640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d950 tx=0x7fd42400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd431ffb640 1 -- 172.21.15.94:0/2397064728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424017030 con 0x7fd434108930 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd431ffb640 1 -- 172.21.15.94:0/2397064728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd424004020 con 0x7fd434108930 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd431ffb640 1 -- 172.21.15.94:0/2397064728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424003b50 con 0x7fd434108930 2024-04-25T14:05:53.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341a0f70 con 0x7fd434108930 2024-04-25T14:05:53.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.045+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a13d0 con 0x7fd434108930 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd431ffb640 1 -- 172.21.15.94:0/2397064728 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd42401a070 con 0x7fd434108930 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 Processor -- start 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 -- start start 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b757fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3926b0 0x55c18b758c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c18b6b9d80 con 0x55c18b3b3390 2024-04-25T14:05:53.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c18b673a20 con 0x55c18b3926b0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd441a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c18b6b5d40 con 0x55c18b38a4d0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48430/0 (socket says 172.21.15.94:48430) 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd433fff640 1 -- 172.21.15.94:0/2178043002 learned_addr learned my addr 172.21.15.94:0/2178043002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd438961640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b757fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd433fff640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3926b0 0x55c18b758c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd438961640 1 -- 172.21.15.94:0/2178043002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3926b0 msgr2=0x55c18b758c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd438961640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3926b0 0x55c18b758c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.048+0000 7fd438961640 1 -- 172.21.15.94:0/2178043002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 msgr2=0x55c18b757a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd438961640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd438961640 1 -- 172.21.15.94:0/2178043002 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18b67ec30 con 0x55c18b38a4d0 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd433fff640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:53.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd431ffb640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 0x7fd4200b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd433fff640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 0x7fd4200b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd438961640 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b757fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd4341022c0 tx=0x7fd42c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab267ae02a165ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd431ffb640 1 -- 172.21.15.94:0/2397064728 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd4240de400 con 0x7fd434108930 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd42affd640 1 -- 172.21.15.94:0/2178043002 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c004b20 con 0x55c18b38a4d0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd42affd640 1 -- 172.21.15.94:0/2178043002 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42c003e20 con 0x55c18b38a4d0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.049+0000 7fd42affd640 1 -- 172.21.15.94:0/2178043002 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c004800 con 0x55c18b38a4d0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- 172.21.15.94:0/2178043002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 msgr2=0x55c18b757fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd433fff640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 0x7fd4200b2080 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd41c031500 tx=0x7fd41c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b757fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd4341022c0 tx=0x7fd42c0118d0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- 172.21.15.94:0/2178043002 shutdown_connections 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3926b0 0x55c18b758c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b757fa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 --2- 172.21.15.94:0/2178043002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b757a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- 172.21.15.94:0/2178043002 >> 172.21.15.94:0/2178043002 conn(0x55c18b3a74e0 msgr2=0x55c18b751020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- 172.21.15.94:0/2178043002 shutdown_connections 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- 172.21.15.94:0/2178043002 wait complete. 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 -- start start 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.050+0000 7fd441a79740 1 Processor -- start 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b7616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b768b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c18b759740 con 0x55c18b3926b0 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c18b640dc0 con 0x55c18b3b3390 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c18b7598c0 con 0x55c18b38a4d0 2024-04-25T14:05:53.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd441a79740 10 client.0 _ll_register_callbacks cb 0x55c18b65c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48434/0 (socket says 172.21.15.94:48434) 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 -- 172.21.15.94:0/4180651526 learned_addr learned my addr 172.21.15.94:0/4180651526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd438961640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b7616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd439162640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b768b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 -- 172.21.15.94:0/4180651526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 msgr2=0x55c18b7616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b7616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 -- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 msgr2=0x55c18b768b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b768b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 -- 172.21.15.94:0/4180651526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c18b3cb3f0 con 0x55c18b3926b0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd438961640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b7616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd433fff640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009230 tx=0x7fd41c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5895634d8d00be5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.051+0000 7fd413fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c04d020 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd41c005450 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42c003e20 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd41c058050 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 10 client.24048 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c18b4957d0 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42c003e20 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd41c0038c0 con 0x55c18b3926b0 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 10 client.24048 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 10 client.24048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 20 client.24048 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b371f60 0x55c18b76f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 4 client.24048 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.052+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c18b6c7990 con 0x55c18b371f60 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.053+0000 7fd438961640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b371f60 0x55c18b76f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.053+0000 7fd413fff640 10 client.24048 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.053+0000 7fd438961640 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b371f60 0x55c18b76f520 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89a8c38ca6dbc4dc server_cookie=b43607283d4a9140 in_seq=0 out_seq=0 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd413fff640 1 -- 172.21.15.94:0/4180651526 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c18b6c7990 con 0x55c18b371f60 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd413fff640 10 client.24048 handle_command_reply: tid=0 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 client.24048 shutdown 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 msgr2=0x55c18b761c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009230 tx=0x7fd41c005a20 comp rx=0 tx=0).stop 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 shutdown_connections 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b371f60 0x55c18b76f520 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b768b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3926b0 0x55c18b761c30 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 --2- 172.21.15.94:0/4180651526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b38a4d0 0x55c18b7616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd413fff640 0 client.24048 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 >> 172.21.15.94:0/4180651526 conn(0x55c18b3a74e0 msgr2=0x55c18b753f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 shutdown_connections 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 1 -- 172.21.15.94:0/4180651526 wait complete. 2024-04-25T14:05:53.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.054+0000 7fd441a79740 20 client.24048 trim_cache size 0 max 16384 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 Processor -- start 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 -- start start 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b38a4d0 0x55c18b754530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b754a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a92f0 con 0x55c18b3b3390 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341aae30 con 0x55c18b38a4d0 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd441a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a99c0 con 0x55c18b3926b0 2024-04-25T14:05:53.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48440/0 (socket says 172.21.15.94:48440) 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 -- 172.21.15.94:0/3643813092 learned_addr learned my addr 172.21.15.94:0/3643813092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd433fff640 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b754a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd438961640 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b38a4d0 0x55c18b754530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 -- 172.21.15.94:0/3643813092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 msgr2=0x55c18b754a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b754a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 -- 172.21.15.94:0/3643813092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b38a4d0 msgr2=0x55c18b754530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b38a4d0 0x55c18b754530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.068+0000 7fd439162640 1 -- 172.21.15.94:0/3643813092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18b768ff0 con 0x55c18b3b3390 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.069+0000 7fd439162640 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd4240dc1d0 tx=0x7fd424004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02ca882673da59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.069+0000 7fd42a7fc640 1 -- 172.21.15.94:0/3643813092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4240c0070 con 0x55c18b3b3390 2024-04-25T14:05:53.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.069+0000 7fd42a7fc640 1 -- 172.21.15.94:0/3643813092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4240a6840 con 0x55c18b3b3390 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.069+0000 7fd42a7fc640 1 -- 172.21.15.94:0/3643813092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424003cf0 con 0x55c18b3b3390 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- 172.21.15.94:0/3643813092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 msgr2=0x55c18b768ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd4240dc1d0 tx=0x7fd424004550 comp rx=0 tx=0).stop 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- 172.21.15.94:0/3643813092 shutdown_connections 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b3b3390 0x55c18b768ab0 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b754a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- 172.21.15.94:0/3643813092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b38a4d0 0x55c18b754530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- 172.21.15.94:0/3643813092 >> 172.21.15.94:0/3643813092 conn(0x55c18b3a74e0 msgr2=0x55c18b753e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- 172.21.15.94:0/3643813092 shutdown_connections 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- 172.21.15.94:0/3643813092 wait complete. 2024-04-25T14:05:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- start start 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 Processor -- start 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b768570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b7d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341aa760 con 0x55c18b38a4d0 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341aa090 con 0x55c18b3b3390 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.070+0000 7fd441a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341a72d0 con 0x55c18b3926b0 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd441a79740 10 client.0 _ll_register_callbacks cb 0x7fd41c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48452/0 (socket says 172.21.15.94:48452) 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 -- 172.21.15.94:0/808997816 learned_addr learned my addr 172.21.15.94:0/808997816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd433fff640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b768570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd439162640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b7d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 -- 172.21.15.94:0/808997816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 msgr2=0x55c18b768570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b768570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 -- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 msgr2=0x55c18b7d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b7d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 -- 172.21.15.94:0/808997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c18b7d62f0 con 0x55c18b38a4d0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd438961640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x55c18b757740 tx=0x7fd42c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2458b457e628cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd413fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c003a70 con 0x55c18b38a4d0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fd42c01e040 con 0x55c18b38a4d0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4240a6840 con 0x55c18b38a4d0 2024-04-25T14:05:53.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd42c035050 con 0x55c18b38a4d0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd441a79740 10 client.24054 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.071+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c18b7d75e0 con 0x55c18b38a4d0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4240a6840 con 0x55c18b38a4d0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd42c00f700 con 0x55c18b38a4d0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 10 client.24054 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 10 client.24054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 20 client.24054 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b3473a0 0x55c18b7d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 4 client.24054 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c18b6c1800 con 0x55c18b3473a0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd433fff640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b3473a0 0x55c18b7d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd413fff640 10 client.24054 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:53.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.072+0000 7fd433fff640 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b3473a0 0x55c18b7d9ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1f7c151798dfdec server_cookie=279898b2083fe55e in_seq=0 out_seq=0 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd413fff640 1 -- 172.21.15.94:0/808997816 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c18b6c1800 con 0x55c18b3473a0 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd413fff640 10 client.24054 handle_command_reply: tid=0 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 client.24054 shutdown 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 msgr2=0x55c18b768030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x55c18b757740 tx=0x7fd42c004470 comp rx=0 tx=0).stop 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 shutdown_connections 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x55c18b3473a0 0x55c18b7d9ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c18b3b3390 0x55c18b7d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c18b3926b0 0x55c18b768570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd413fff640 0 client.24054 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 --2- 172.21.15.94:0/808997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c18b38a4d0 0x55c18b768030 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 >> 172.21.15.94:0/808997816 conn(0x55c18b3a74e0 msgr2=0x55c18b754850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 shutdown_connections 2024-04-25T14:05:53.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 1 -- 172.21.15.94:0/808997816 wait complete. 2024-04-25T14:05:53.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.073+0000 7fd441a79740 20 client.24054 trim_cache size 0 max 16384 2024-04-25T14:05:53.081 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 msgr2=0x7fd4200b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 0x7fd4200b2080 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd41c031500 tx=0x7fd41c0058b0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 msgr2=0x7fd4341a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d950 tx=0x7fd42400de10 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 shutdown_connections 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fd4200afb70 0x7fd4200b2080 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd434108930 0x7fd4341a0610 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd434103c30 0x7fd434199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 --2- 172.21.15.94:0/2397064728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4341029f0 0x7fd434199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 >> 172.21.15.94:0/2397064728 conn(0x7fd4340fdbc0 msgr2=0x7fd434104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 shutdown_connections 2024-04-25T14:05:53.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.074+0000 7fd43abc5640 1 -- 172.21.15.94:0/2397064728 wait complete. 2024-04-25T14:05:53.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:05:53.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 224706 18.86 MB/sec execute 515 sec latency 120.390 ms 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- 172.21.15.94:0/4042669027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e54103c20 msgr2=0x7f0e54104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/4042669027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e54103c20 0x7f0e54104030 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009900 tx=0x7f0e4402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- 172.21.15.94:0/4042669027 shutdown_connections 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/4042669027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54108920 0x7f0e5410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/4042669027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e54103c20 0x7f0e54104030 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/4042669027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e541029e0 0x7f0e541036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- 172.21.15.94:0/4042669027 >> 172.21.15.94:0/4042669027 conn(0x7f0e540fdbd0 msgr2=0x7f0e54100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- 172.21.15.94:0/4042669027 shutdown_connections 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- 172.21.15.94:0/4042669027 wait complete. 2024-04-25T14:05:53.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 Processor -- start 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.198+0000 7f0e5b73f640 1 -- start start 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54103c20 0x7f0e5419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e54108920 0x7f0e541a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5410f670 con 0x7f0e541029e0 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5410f4f0 con 0x7f0e54103c20 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e5b73f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5410f7f0 con 0x7f0e54108920 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48470/0 (socket says 172.21.15.94:48470) 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 -- 172.21.15.94:0/2926070898 learned_addr learned my addr 172.21.15.94:0/2926070898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e58cda640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54103c20 0x7f0e5419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e54108920 0x7f0e541a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 -- 172.21.15.94:0/2926070898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e54108920 msgr2=0x7f0e541a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e54108920 0x7f0e541a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 -- 172.21.15.94:0/2926070898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54103c20 msgr2=0x7f0e5419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54103c20 0x7f0e5419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 -- 172.21.15.94:0/2926070898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e541a2040 con 0x7f0e541029e0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e594db640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e9f0 tx=0x7f0e5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e4a7fc640 1 -- 172.21.15.94:0/2926070898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000ccc0 con 0x7f0e541029e0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e4a7fc640 1 -- 172.21.15.94:0/2926070898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e500035f0 con 0x7f0e541029e0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.199+0000 7f0e4a7fc640 1 -- 172.21.15.94:0/2926070898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000ce60 con 0x7f0e541029e0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.200+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541a2330 con 0x7f0e541029e0 2024-04-25T14:05:53.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.200+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541a2850 con 0x7f0e541029e0 2024-04-25T14:05:53.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.202+0000 7f0e625f4740 1 Processor -- start 2024-04-25T14:05:53.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.202+0000 7f0e625f4740 1 -- start start 2024-04-25T14:05:53.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007897f390 0x560078d21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560078c85d80 con 0x56007895e6b0 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560078c3fa20 con 0x56007897f390 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e625f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560078c81d40 con 0x5600789564d0 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48484/0 (socket says 172.21.15.94:48484) 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 -- 172.21.15.94:0/998819619 learned_addr learned my addr 172.21.15.94:0/998819619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007897f390 0x560078d21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 -- 172.21.15.94:0/998819619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 msgr2=0x560078d21d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d21d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 -- 172.21.15.94:0/998819619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007897f390 msgr2=0x560078d21810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007897f390 0x560078d21810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 -- 172.21.15.94:0/998819619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560078c4ac30 con 0x56007895e6b0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e58cda640 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406bc9b2c75fedc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.203+0000 7f0e4a7fc640 1 -- 172.21.15.94:0/2926070898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0e50003aa0 con 0x7f0e541029e0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e3f7fe640 1 -- 172.21.15.94:0/998819619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44035030 con 0x56007895e6b0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e3f7fe640 1 -- 172.21.15.94:0/998819619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e44040070 con 0x56007895e6b0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e3f7fe640 1 -- 172.21.15.94:0/998819619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440319a0 con 0x56007895e6b0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e4a7fc640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 0x7f0e400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 0x7f0e400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.204+0000 7f0e4a7fc640 1 -- 172.21.15.94:0/2926070898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0e500defb0 con 0x7f0e541029e0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- 172.21.15.94:0/998819619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 msgr2=0x560078d22a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44004040 comp rx=0 tx=0).stop 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 0x7f0e400b2000 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x560078d22860 tx=0x7f0e4c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- 172.21.15.94:0/998819619 shutdown_connections 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d22a20 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d21d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 --2- 172.21.15.94:0/998819619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007897f390 0x560078d21810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- 172.21.15.94:0/998819619 >> 172.21.15.94:0/998819619 conn(0x5600789734e0 msgr2=0x560078d1d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- 172.21.15.94:0/998819619 shutdown_connections 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- 172.21.15.94:0/998819619 wait complete. 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.205+0000 7f0e625f4740 1 -- start start 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 Processor -- start 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d2b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d2bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560078d234d0 con 0x56007897f390 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560078c0cdc0 con 0x56007895e6b0 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560078d23650 con 0x5600789564d0 2024-04-25T14:05:53.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e625f4740 10 client.0 _ll_register_callbacks cb 0x560078c28600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e59cdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e59cdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48494/0 (socket says 172.21.15.94:48494) 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e58cda640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d2bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e59cdc640 1 -- 172.21.15.94:0/47611715 learned_addr learned my addr 172.21.15.94:0/47611715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.206+0000 7f0e594db640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d2b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 -- 172.21.15.94:0/47611715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 msgr2=0x560078d2b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d2b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 -- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 msgr2=0x560078d2bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d2bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 -- 172.21.15.94:0/47611715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600789973f0 con 0x56007897f390 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e58cda640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d2bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009cd0 tx=0x7f0e4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe22523d0604d372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00f830 con 0x56007897f390 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0e4c00f9d0 con 0x56007897f390 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e44040070 con 0x56007897f390 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c00fbd0 con 0x56007897f390 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e625f4740 10 client.24066 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.207+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560078a617d0 con 0x56007897f390 2024-04-25T14:05:53.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.208+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e44040070 con 0x56007897f390 2024-04-25T14:05:53.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.208+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0e4c026070 con 0x56007897f390 2024-04-25T14:05:53.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.208+0000 7f0e625f4740 10 client.24066 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e625f4740 10 client.24066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e625f4740 20 client.24066 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56007893df60 0x560078d39390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e625f4740 4 client.24066 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560078c93990 con 0x56007893df60 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e594db640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56007893df60 0x560078d39390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e594db640 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56007893df60 0x560078d39390 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54eb97c61ac58ef0 server_cookie=4d528f59193950a3 in_seq=0 out_seq=0 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.209+0000 7f0e3cff9640 10 client.24066 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e3cff9640 1 -- 172.21.15.94:0/47611715 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560078c93990 con 0x56007893df60 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e3cff9640 10 client.24066 handle_command_reply: tid=0 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 client.24066 shutdown 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 msgr2=0x560078d32a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009cd0 tx=0x7f0e4c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 shutdown_connections 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56007893df60 0x560078d39390 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d32a10 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d2bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e3cff9640 0 client.24066 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 --2- 172.21.15.94:0/47611715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d2b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 >> 172.21.15.94:0/47611715 conn(0x5600789734e0 msgr2=0x560078d1c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 shutdown_connections 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 1 -- 172.21.15.94:0/47611715 wait complete. 2024-04-25T14:05:53.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.214+0000 7f0e625f4740 20 client.24066 trim_cache size 0 max 16384 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 Processor -- start 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 -- start start 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d3db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d32930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a92e0 con 0x56007897f390 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541aae20 con 0x56007895e6b0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.228+0000 7f0e625f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a99b0 con 0x5600789564d0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e594db640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d3db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48496/0 (socket says 172.21.15.94:48496) 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 -- 172.21.15.94:0/1067788524 learned_addr learned my addr 172.21.15.94:0/1067788524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e58cda640 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d32930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 -- 172.21.15.94:0/1067788524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 msgr2=0x560078d3db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d3db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 -- 172.21.15.94:0/1067788524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 msgr2=0x560078d32930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d32930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 -- 172.21.15.94:0/1067788524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560078d25dc0 con 0x56007897f390 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c011f00 tx=0x7f0e4c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301102ed196ad41c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e3effd640 1 -- 172.21.15.94:0/1067788524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c035070 con 0x56007897f390 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e3effd640 1 -- 172.21.15.94:0/1067788524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c01e7a0 con 0x56007897f390 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.229+0000 7f0e3effd640 1 -- 172.21.15.94:0/1067788524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00fa30 con 0x56007897f390 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- 172.21.15.94:0/1067788524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 msgr2=0x560078d3e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c011f00 tx=0x7f0e4c003f80 comp rx=0 tx=0).stop 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- 172.21.15.94:0/1067788524 shutdown_connections 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007897f390 0x560078d3e050 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56007895e6b0 0x560078d32930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 --2- 172.21.15.94:0/1067788524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600789564d0 0x560078d3db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- 172.21.15.94:0/1067788524 >> 172.21.15.94:0/1067788524 conn(0x5600789734e0 msgr2=0x560078d262d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- 172.21.15.94:0/1067788524 shutdown_connections 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- 172.21.15.94:0/1067788524 wait complete. 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.230+0000 7f0e625f4740 1 -- start start 2024-04-25T14:05:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 Processor -- start 2024-04-25T14:05:53.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5600789564d0 0x560078d2fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56007897f390 0x560078d31160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541aa750 con 0x56007895e6b0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541aa080 con 0x5600789564d0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541a72c0 con 0x56007897f390 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e625f4740 10 client.0 _ll_register_callbacks cb 0x7f0e4c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48508/0 (socket says 172.21.15.94:48508) 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 -- 172.21.15.94:0/337424988 learned_addr learned my addr 172.21.15.94:0/337424988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e594db640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5600789564d0 0x560078d2fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e59cdc640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56007897f390 0x560078d31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 -- 172.21.15.94:0/337424988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56007897f390 msgr2=0x560078d31160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56007897f390 0x560078d31160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 -- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5600789564d0 msgr2=0x560078d2fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5600789564d0 0x560078d2fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 -- 172.21.15.94:0/337424988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560078d32e70 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e58cda640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009230 tx=0x7f0e4402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38c2e69a24932f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.231+0000 7f0e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44057060 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0e440037f0 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4c01e7a0 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e44035070 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 10 client.24072 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560078d327d0 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c01e7a0 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0e44031c90 con 0x56007895e6b0 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 10 client.24072 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 10 client.24072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 20 client.24072 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5600789133a0 0x560078da36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 4 client.24072 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:53.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560078c8d800 con 0x5600789133a0 2024-04-25T14:05:53.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.232+0000 7f0e594db640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5600789133a0 0x560078da36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.233+0000 7f0e3cff9640 10 client.24072 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.233+0000 7f0e594db640 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5600789133a0 0x560078da36e0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac82bec4c572566 server_cookie=57c68817f708ff92 in_seq=0 out_seq=0 2024-04-25T14:05:53.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e3cff9640 1 -- 172.21.15.94:0/337424988 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560078c8d800 con 0x5600789133a0 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e3cff9640 10 client.24072 handle_command_reply: tid=0 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e625f4740 1 client.24072 shutdown 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 msgr2=0x560078d30230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009230 tx=0x7f0e4402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 shutdown_connections 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.238+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5600789133a0 0x560078da36e0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56007897f390 0x560078d31160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56007895e6b0 0x560078d30230 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e3cff9640 0 client.24072 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 --2- 172.21.15.94:0/337424988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5600789564d0 0x560078d2fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 >> 172.21.15.94:0/337424988 conn(0x5600789734e0 msgr2=0x560078d1fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 shutdown_connections 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 1 -- 172.21.15.94:0/337424988 wait complete. 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e625f4740 20 client.24072 trim_cache size 0 max 16384 2024-04-25T14:05:53.242 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:05:53.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 msgr2=0x7f0e400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 0x7f0e400b2000 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x560078d22860 tx=0x7f0e4c009500 comp rx=0 tx=0).stop 2024-04-25T14:05:53.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 msgr2=0x7f0e5419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000e9f0 tx=0x7f0e5000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 shutdown_connections 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0e400afaf0 0x7f0e400b2000 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e54108920 0x7f0e541a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e54103c20 0x7f0e5419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 --2- 172.21.15.94:0/2926070898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e541029e0 0x7f0e5419a500 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 >> 172.21.15.94:0/2926070898 conn(0x7f0e540fdbd0 msgr2=0x7f0e54104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 shutdown_connections 2024-04-25T14:05:53.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:53.239+0000 7f0e5b73f640 1 -- 172.21.15.94:0/2926070898 wait complete. 2024-04-25T14:05:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 -- 172.21.15.94:0/561176095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c41046d0 msgr2=0x7f16c410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 --2- 172.21.15.94:0/561176095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c41046d0 0x7f16c410ae20 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009ca0 tx=0x7f16bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 -- 172.21.15.94:0/561176095 shutdown_connections 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 --2- 172.21.15.94:0/561176095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c41046d0 0x7f16c410ae20 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 --2- 172.21.15.94:0/561176095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103b70 0x7f16c4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 --2- 172.21.15.94:0/561176095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103010 0x7f16c4103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.355+0000 7f16caeb6640 1 -- 172.21.15.94:0/561176095 >> 172.21.15.94:0/561176095 conn(0x7f16c40fd390 msgr2=0x7f16c40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.356+0000 7f16caeb6640 1 -- 172.21.15.94:0/561176095 shutdown_connections 2024-04-25T14:05:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.356+0000 7f16caeb6640 1 -- 172.21.15.94:0/561176095 wait complete. 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.356+0000 7f16caeb6640 1 Processor -- start 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.356+0000 7f16caeb6640 1 -- start start 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.356+0000 7f16caeb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 0x7f16c4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103b70 0x7f16c419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410f8a0 con 0x7f16c419f4a0 2024-04-25T14:05:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410f720 con 0x7f16c4103b70 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410fa20 con 0x7f16c4103010 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48522/0 (socket says 172.21.15.94:48522) 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c8c52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 0x7f16c4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 -- 172.21.15.94:0/1105035719 learned_addr learned my addr 172.21.15.94:0/1105035719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103b70 0x7f16c419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 -- 172.21.15.94:0/1105035719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 msgr2=0x7f16c4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 0x7f16c4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 -- 172.21.15.94:0/1105035719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103b70 msgr2=0x7f16c419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103b70 0x7f16c419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 -- 172.21.15.94:0/1105035719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c41a1f30 con 0x7f16c419f4a0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 0x7f16c4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c9453640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f16bc004af0 tx=0x7f16bc031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c1ffb640 1 -- 172.21.15.94:0/1105035719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc003b70 con 0x7f16c419f4a0 2024-04-25T14:05:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41a21c0 con 0x7f16c419f4a0 2024-04-25T14:05:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c1ffb640 1 -- 172.21.15.94:0/1105035719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16bc003d10 con 0x7f16c419f4a0 2024-04-25T14:05:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.357+0000 7f16c1ffb640 1 -- 172.21.15.94:0/1105035719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc02fb20 con 0x7f16c419f4a0 2024-04-25T14:05:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.358+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a2760 con 0x7f16c419f4a0 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 Processor -- start 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c1ffb640 1 -- 172.21.15.94:0/1105035719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f16bc004580 con 0x7f16c419f4a0 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 -- start start 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363a32f0 0x564636771300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636778200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5646366d2460 con 0x5646363cc1b0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56463668c590 con 0x5646363ab4d0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646366ce420 con 0x5646363a32f0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48536/0 (socket says 172.21.15.94:48536) 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c3fff640 1 -- 172.21.15.94:0/1047124292 learned_addr learned my addr 172.21.15.94:0/1047124292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636778200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363a32f0 0x564636771300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.361+0000 7f16c1ffb640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 0x7f16b40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 -- 172.21.15.94:0/1047124292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363a32f0 msgr2=0x564636771300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363a32f0 0x564636771300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 -- 172.21.15.94:0/1047124292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 msgr2=0x564636778200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636778200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 -- 172.21.15.94:0/1047124292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646366990d0 con 0x5646363cc1b0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c9453640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 0x7f16b40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009710 tx=0x7f16b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978dd37be239fcd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16c1ffb640 1 -- 172.21.15.94:0/1105035719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f16bc100210 con 0x7f16c419f4a0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16aeffd640 1 -- 172.21.15.94:0/1047124292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0013e50 con 0x5646363cc1b0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16aeffd640 1 -- 172.21.15.94:0/1047124292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0013120 con 0x5646363cc1b0 2024-04-25T14:05:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.362+0000 7f16aeffd640 1 -- 172.21.15.94:0/1047124292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b00035f0 con 0x5646363cc1b0 2024-04-25T14:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1047124292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 msgr2=0x564636770d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16c9453640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 0x7f16b40b2070 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0c85e0 tx=0x7f16bc02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009710 tx=0x7f16b0009780 comp rx=0 tx=0).stop 2024-04-25T14:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1047124292 shutdown_connections 2024-04-25T14:05:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636778200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363a32f0 0x564636771300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1047124292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363cc1b0 0x564636770d80 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1047124292 >> 172.21.15.94:0/1047124292 conn(0x5646363c0300 msgr2=0x564636768300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1047124292 shutdown_connections 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1047124292 wait complete. 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 -- start start 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.363+0000 7f16d1d6c740 1 Processor -- start 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363ab4d0 0x56463677af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363cc1b0 0x564636781ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564636778bf0 con 0x5646363a32f0 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564636659b60 con 0x5646363cc1b0 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564636778d70 con 0x5646363ab4d0 2024-04-25T14:05:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16d1d6c740 10 client.0 _ll_register_callbacks cb 0x56463667e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48550/0 (socket says 172.21.15.94:48550) 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 -- 172.21.15.94:0/1076477979 learned_addr learned my addr 172.21.15.94:0/1076477979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363ab4d0 0x56463677af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363cc1b0 0x564636781ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 -- 172.21.15.94:0/1076477979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363ab4d0 msgr2=0x56463677af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363ab4d0 0x56463677af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 -- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363cc1b0 msgr2=0x564636781ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363cc1b0 0x564636781ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.364+0000 7f16c3fff640 1 -- 172.21.15.94:0/1076477979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646363e4210 con 0x5646363a32f0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f870 tx=0x7f16b000f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ec7c2c71e760df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b002d060 con 0x5646363a32f0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f16b00048c0 con 0x5646363a32f0 2024-04-25T14:05:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b0013120 con 0x5646363a32f0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f16b0014e60 con 0x5646363a32f0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 10 client.24084 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5646364b0620 con 0x5646363a32f0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0013120 con 0x5646363a32f0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f16b0021850 con 0x5646363a32f0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 10 client.24084 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 10 client.24084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 20 client.24084 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56463638ad80 0x564636788430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 4 client.24084 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.365+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646366e0070 con 0x56463638ad80 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.366+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56463638ad80 0x564636788430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.366+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56463638ad80 0x564636788430 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6b15f619328f11 server_cookie=57a3b07998a0901d in_seq=0 out_seq=0 2024-04-25T14:05:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.366+0000 7f16a3fff640 10 client.24084 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.368+0000 7f16a3fff640 1 -- 172.21.15.94:0/1076477979 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646366e0070 con 0x56463638ad80 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16a3fff640 10 client.24084 handle_command_reply: tid=0 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 client.24084 shutdown 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 msgr2=0x56463677aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f870 tx=0x7f16b000f7e0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 shutdown_connections 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56463638ad80 0x564636788430 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363cc1b0 0x564636781ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363ab4d0 0x56463677af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1076477979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677aa30 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16a3fff640 0 client.24084 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 >> 172.21.15.94:0/1076477979 conn(0x5646363c0300 msgr2=0x56463676b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 shutdown_connections 2024-04-25T14:05:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1076477979 wait complete. 2024-04-25T14:05:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.369+0000 7f16d1d6c740 20 client.24084 trim_cache size 0 max 16384 2024-04-25T14:05:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.385+0000 7f16d1d6c740 1 Processor -- start 2024-04-25T14:05:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.385+0000 7f16d1d6c740 1 -- start start 2024-04-25T14:05:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636781e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x56463676b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9250 con 0x5646363a32f0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aad90 con 0x5646363ab4d0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9920 con 0x5646363cc1b0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48554/0 (socket says 172.21.15.94:48554) 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 -- 172.21.15.94:0/1389298592 learned_addr learned my addr 172.21.15.94:0/1389298592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c8c52640 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636781e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x56463676b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 -- 172.21.15.94:0/1389298592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 msgr2=0x56463676b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x56463676b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 -- 172.21.15.94:0/1389298592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 msgr2=0x564636781e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636781e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.386+0000 7f16c3fff640 1 -- 172.21.15.94:0/1389298592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564636768de0 con 0x5646363a32f0 2024-04-25T14:05:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.387+0000 7f16c3fff640 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f920 tx=0x7f16b001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cb173799976abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.387+0000 7f16ae7fc640 1 -- 172.21.15.94:0/1389298592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b002d040 con 0x5646363a32f0 2024-04-25T14:05:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.387+0000 7f16ae7fc640 1 -- 172.21.15.94:0/1389298592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16b0005630 con 0x5646363a32f0 2024-04-25T14:05:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.387+0000 7f16ae7fc640 1 -- 172.21.15.94:0/1389298592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0021a10 con 0x5646363a32f0 2024-04-25T14:05:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1389298592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 msgr2=0x564636768360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f920 tx=0x7f16b001c040 comp rx=0 tx=0).stop 2024-04-25T14:05:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1389298592 shutdown_connections 2024-04-25T14:05:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x56463676b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x564636781e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/1389298592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x564636768360 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1389298592 >> 172.21.15.94:0/1389298592 conn(0x5646363c0300 msgr2=0x564636767c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1389298592 shutdown_connections 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.388+0000 7f16d1d6c740 1 -- 172.21.15.94:0/1389298592 wait complete. 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 -- start start 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 Processor -- start 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x56463677ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x564636780e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41aa6c0 con 0x5646363a32f0 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a9ff0 con 0x5646363ab4d0 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41a7230 con 0x5646363cc1b0 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16d1d6c740 10 client.0 _ll_register_callbacks cb 0x7f16b0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48560/0 (socket says 172.21.15.94:48560) 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16c3fff640 1 -- 172.21.15.94:0/4174769975 learned_addr learned my addr 172.21.15.94:0/4174769975 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.389+0000 7f16c8c52640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x56463677ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x564636780e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 -- 172.21.15.94:0/4174769975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 msgr2=0x564636780e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x564636780e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 -- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 msgr2=0x56463677ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x56463677ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 -- 172.21.15.94:0/4174769975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564636775910 con 0x5646363a32f0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16c3fff640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f820 tx=0x7f16b0013430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa22e6ed0a44c337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b002d070 con 0x5646363a32f0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f16b0009780 con 0x5646363a32f0 2024-04-25T14:05:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646366d9ee0 con 0x5646363a32f0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f16b0021440 con 0x5646363a32f0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16d1d6c740 10 client.24090 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.390+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564636778620 con 0x5646363a32f0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646366d9ee0 con 0x5646363a32f0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f16b0032cb0 con 0x5646363a32f0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 10 client.24090 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 10 client.24090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 20 client.24090 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646363601c0 0x5646367f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 4 client.24090 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646366555f0 con 0x5646363601c0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.391+0000 7f16c8c52640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646363601c0 0x5646367f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.392+0000 7f16a3fff640 10 client.24090 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:53.392+0000 7f16c8c52640 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646363601c0 0x5646367f3080 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b283cf6b0d4c47 server_cookie=bed49965f3f65a6b in_seq=0 out_seq=0 2024-04-25T14:05:53.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:53 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/909834712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:53.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:53 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/909834712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:53.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:53 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/909834712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16a3fff640 1 -- 172.21.15.94:0/4174769975 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646366555f0 con 0x5646363601c0 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16a3fff640 10 client.24090 handle_command_reply: tid=0 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 client.24090 shutdown 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 msgr2=0x56463677fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f16b000f820 tx=0x7f16b0013430 comp rx=0 tx=0).stop 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 shutdown_connections 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5646363601c0 0x5646367f3080 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16a3fff640 0 client.24090 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646363cc1b0 0x564636780e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5646363ab4d0 0x56463677ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 --2- 172.21.15.94:0/4174769975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5646363a32f0 0x56463677fa70 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 >> 172.21.15.94:0/4174769975 conn(0x5646363c0300 msgr2=0x56463677a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 shutdown_connections 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.108+0000 7f16d1d6c740 1 -- 172.21.15.94:0/4174769975 wait complete. 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16d1d6c740 20 client.24090 trim_cache size 0 max 16384 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:05:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:05:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:05:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 msgr2=0x7f16b40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 0x7f16b40b2070 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0c85e0 tx=0x7f16bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 msgr2=0x7f16c41a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f16bc004af0 tx=0x7f16bc031700 comp rx=0 tx=0).stop 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 shutdown_connections 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f16b40afb60 0x7f16b40b2070 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16c419f4a0 0x7f16c41a18c0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16c4103b70 0x7f16c419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 --2- 172.21.15.94:0/1105035719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4103010 0x7f16c4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 >> 172.21.15.94:0/1105035719 conn(0x7f16c40fd390 msgr2=0x7f16c40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.109+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 shutdown_connections 2024-04-25T14:05:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:05:54.110+0000 7f16caeb6640 1 -- 172.21.15.94:0/1105035719 wait complete. 2024-04-25T14:05:54.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 224828 18.84 MB/sec execute 516 sec latency 493.591 ms 2024-04-25T14:05:54.251 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:05:54.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:54.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2133480385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c1029e0 msgr2=0x7f7a1c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2133480385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c1036e0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a1802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2133480385 shutdown_connections 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2133480385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c108920 0x7f7a1c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2133480385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c103c20 0x7f7a1c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2133480385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c1036e0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.368+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2133480385 >> 172.21.15.94:0/2133480385 conn(0x7f7a1c0fdbd0 msgr2=0x7f7a1c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2133480385 shutdown_connections 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2133480385 wait complete. 2024-04-25T14:05:54.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 Processor -- start 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- start start 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c108920 0x7f7a1c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e4e0 con 0x7f7a1c103c20 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e360 con 0x7f7a1c108920 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a23ee2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10e660 con 0x7f7a1c1029e0 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a2147d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a2147d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50102/0 (socket says 172.21.15.94:50102) 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a2147d640 1 -- 172.21.15.94:0/2943402730 learned_addr learned my addr 172.21.15.94:0/2943402730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.369+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c108920 0x7f7a1c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 -- 172.21.15.94:0/2943402730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c1029e0 msgr2=0x7f7a1c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 -- 172.21.15.94:0/2943402730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c108920 msgr2=0x7f7a1c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c108920 0x7f7a1c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 -- 172.21.15.94:0/2943402730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c1a1da0 con 0x7f7a1c103c20 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00d950 tx=0x7f7a0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a12ffd640 1 -- 172.21.15.94:0/2943402730 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c017070 con 0x7f7a1c103c20 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a12ffd640 1 -- 172.21.15.94:0/2943402730 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c009c30 con 0x7f7a1c103c20 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a12ffd640 1 -- 172.21.15.94:0/2943402730 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00f620 con 0x7f7a1c103c20 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1a2090 con 0x7f7a1c103c20 2024-04-25T14:05:54.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.370+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a2590 con 0x7f7a1c103c20 2024-04-25T14:05:54.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.373+0000 7f7a2ad8d740 1 Processor -- start 2024-04-25T14:05:54.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.373+0000 7f7a12ffd640 1 -- 172.21.15.94:0/2943402730 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7a0c010460 con 0x7f7a1c103c20 2024-04-25T14:05:54.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 -- start start 2024-04-25T14:05:54.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 0x56402a4584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a45f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56402a3bdb00 con 0x56402a08e550 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56402a359920 con 0x56402a096730 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56402a3b9ac0 con 0x56402a0b7410 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50108/0 (socket says 172.21.15.94:50108) 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2135960123 learned_addr learned my addr 172.21.15.94:0/2135960123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a45f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 0x56402a4584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2135960123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 msgr2=0x56402a4584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 0x56402a4584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2135960123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 msgr2=0x56402a45f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a45f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a12ffd640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 0x7f7a080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2135960123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402a1977e0 con 0x56402a08e550 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.374+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 0x56402a4584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 0x7f7a080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a12ffd640 1 -- 172.21.15.94:0/2943402730 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7a0c0de800 con 0x7f7a1c103c20 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009230 tx=0x7f7a18032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb5148f8bf89ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a07fff640 1 -- 172.21.15.94:0/2135960123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18040070 con 0x56402a08e550 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a07fff640 1 -- 172.21.15.94:0/2135960123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a18004030 con 0x56402a08e550 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.375+0000 7f7a07fff640 1 -- 172.21.15.94:0/2135960123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18035040 con 0x56402a08e550 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2135960123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 msgr2=0x56402a458a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 0x7f7a080b2080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x56402a459670 tx=0x7f7a14009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009230 tx=0x7f7a18032bd0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2135960123 shutdown_connections 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a45f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a458a20 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2135960123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a0b7410 0x56402a4584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2135960123 >> 172.21.15.94:0/2135960123 conn(0x56402a0ab560 msgr2=0x56402a453e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2135960123 shutdown_connections 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.376+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2135960123 wait complete. 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 -- start start 2024-04-25T14:05:54.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 Processor -- start 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a462260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4627a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56402a460310 con 0x56402a0b7410 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56402a3575d0 con 0x56402a096730 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56402a460490 con 0x56402a08e550 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2ad8d740 10 client.0 _ll_register_callbacks cb 0x56402a345080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:54.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2247f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2147d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2247f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50116/0 (socket says 172.21.15.94:50116) 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a2247f640 1 -- 172.21.15.94:0/3486438121 learned_addr learned my addr 172.21.15.94:0/3486438121 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.377+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a462260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 -- 172.21.15.94:0/3486438121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 msgr2=0x56402a462260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a462260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 -- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 msgr2=0x56402a4627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4627a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 -- 172.21.15.94:0/3486438121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402a1f7f70 con 0x56402a0b7410 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a462260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:54.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2147d640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2247f640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f7a14003ce0 tx=0x7f7a14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfbe1c57b57b1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14010690 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7a14010830 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a18004030 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a14010a30 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2ad8d740 10 client.24102 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56402a08f170 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.378+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a18004030 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7a1400f040 con 0x56402a0b7410 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 10 client.24102 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 10 client.24102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 20 client.24102 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:54.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0761f0 0x56402a46fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 4 client.24102 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402a3cb710 con 0x56402a0761f0 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0761f0 0x56402a46fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a057fa640 10 client.24102 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.379+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0761f0 0x56402a46fd40 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a1ea03938e154f server_cookie=14408b8e6212991e in_seq=0 out_seq=0 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.382+0000 7f7a057fa640 1 -- 172.21.15.94:0/3486438121 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56402a3cb710 con 0x56402a0761f0 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.382+0000 7f7a057fa640 10 client.24102 handle_command_reply: tid=0 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.382+0000 7f7a2ad8d740 1 client.24102 shutdown 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.382+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 msgr2=0x56402a469700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.382+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f7a14003ce0 tx=0x7f7a14004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 shutdown_connections 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0761f0 0x56402a46fd40 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0b7410 0x56402a469700 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3486438121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a462260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a057fa640 0 client.24102 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 >> 172.21.15.94:0/3486438121 conn(0x56402a0ab560 msgr2=0x56402a452ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 shutdown_connections 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3486438121 wait complete. 2024-04-25T14:05:54.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.383+0000 7f7a2ad8d740 20 client.24102 trim_cache size 0 max 16384 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 Processor -- start 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 -- start start 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a0761f0 0x56402a460f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a096730 0x56402a469b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ab5c0 con 0x56402a08e550 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ad100 con 0x56402a0761f0 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1abc90 con 0x56402a096730 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2147d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2147d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50130/0 (socket says 172.21.15.94:50130) 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2147d640 1 -- 172.21.15.94:0/3596826467 learned_addr learned my addr 172.21.15.94:0/3596826467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a0761f0 0x56402a460f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.401+0000 7f7a2247f640 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a096730 0x56402a469b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 -- 172.21.15.94:0/3596826467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a096730 msgr2=0x56402a469b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a096730 0x56402a469b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 -- 172.21.15.94:0/3596826467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a0761f0 msgr2=0x56402a460f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a0761f0 0x56402a460f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 -- 172.21.15.94:0/3596826467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402a45d160 con 0x56402a08e550 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a2147d640 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6c80 tx=0x7f7a0c0e4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacbe79956774bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a077fe640 1 -- 172.21.15.94:0/3596826467 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c020040 con 0x56402a08e550 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a077fe640 1 -- 172.21.15.94:0/3596826467 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c0af3e0 con 0x56402a08e550 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.402+0000 7f7a077fe640 1 -- 172.21.15.94:0/3596826467 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0afaa0 con 0x56402a08e550 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3596826467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 msgr2=0x56402a469620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6c80 tx=0x7f7a0c0e4550 comp rx=0 tx=0).stop 2024-04-25T14:05:54.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3596826467 shutdown_connections 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a096730 0x56402a469b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a08e550 0x56402a469620 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/3596826467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a0761f0 0x56402a460f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3596826467 >> 172.21.15.94:0/3596826467 conn(0x56402a0ab560 msgr2=0x56402a460830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3596826467 shutdown_connections 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/3596826467 wait complete. 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 -- start start 2024-04-25T14:05:54.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.403+0000 7f7a2ad8d740 1 Processor -- start 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a4674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1aca30 con 0x56402a0761f0 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ac360 con 0x56402a096730 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1a95a0 con 0x56402a08e550 2024-04-25T14:05:54.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 10 client.0 _ll_register_callbacks cb 0x7f7a14003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50134/0 (socket says 172.21.15.94:50134) 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2979575354 learned_addr learned my addr 172.21.15.94:0/2979575354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a4674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2247f640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2979575354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 msgr2=0x56402a4674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a4674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 msgr2=0x56402a4d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 -- 172.21.15.94:0/2979575354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402a4d6660 con 0x56402a0761f0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a21c7e640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005f70 tx=0x7f7a18005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20de8a2d61099d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:54.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18057020 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7a180317a0 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.404+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56402a3c5580 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a18062050 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 10 client.24108 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56402a4d79a0 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56402a3c5580 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7a18003ae0 con 0x56402a0761f0 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 10 client.24108 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 10 client.24108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 20 client.24108 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0b7410 0x56402a4d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 4 client.24108 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:05:54.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56402a340ac0 con 0x56402a0b7410 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0b7410 0x56402a4d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.405+0000 7f7a057fa640 10 client.24108 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.406+0000 7f7a2147d640 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0b7410 0x56402a4d9fe0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e320ad85e0413433 server_cookie=3b00d8817d0999c1 in_seq=0 out_seq=0 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.408+0000 7f7a057fa640 1 -- 172.21.15.94:0/2979575354 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56402a340ac0 con 0x56402a0b7410 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.408+0000 7f7a057fa640 10 client.24108 handle_command_reply: tid=0 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.408+0000 7f7a2ad8d740 1 client.24108 shutdown 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 msgr2=0x56402a466f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005f70 tx=0x7f7a18005e40 comp rx=0 tx=0).stop 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 shutdown_connections 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56402a0b7410 0x56402a4d9fe0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56402a096730 0x56402a4d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56402a08e550 0x56402a4674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a057fa640 0 client.24108 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 --2- 172.21.15.94:0/2979575354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56402a0761f0 0x56402a466f70 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 >> 172.21.15.94:0/2979575354 conn(0x56402a0ab560 msgr2=0x56402a456c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 shutdown_connections 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 1 -- 172.21.15.94:0/2979575354 wait complete. 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a2ad8d740 20 client.24108 trim_cache size 0 max 16384 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "92d7fbf4-187a-4f86-addd-8c14c264b1e0", 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:05:54.413 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:54.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 msgr2=0x7f7a080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 0x7f7a080b2080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x56402a459670 tx=0x7f7a14009600 comp rx=0 tx=0).stop 2024-04-25T14:05:54.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 msgr2=0x7f7a1c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00d950 tx=0x7f7a0c00de10 comp rx=0 tx=0).stop 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.409+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 shutdown_connections 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7a080afb70 0x7f7a080b2080 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a1c108920 0x7f7a1c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a1c103c20 0x7f7a1c19f010 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 --2- 172.21.15.94:0/2943402730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c1029e0 0x7f7a1c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 >> 172.21.15.94:0/2943402730 conn(0x7f7a1c0fdbd0 msgr2=0x7f7a1c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 shutdown_connections 2024-04-25T14:05:54.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.410+0000 7f7a23ee2640 1 -- 172.21.15.94:0/2943402730 wait complete. 2024-04-25T14:05:54.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:54.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- 172.21.15.94:0/1331732874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0105cf0 msgr2=0x7fefa01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 --2- 172.21.15.94:0/1331732874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0105cf0 0x7fefa01069f0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009ca0 tx=0x7fef9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- 172.21.15.94:0/1331732874 shutdown_connections 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 --2- 172.21.15.94:0/1331732874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa010bc30 0x7fefa010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 --2- 172.21.15.94:0/1331732874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa0106f30 0x7fefa0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 --2- 172.21.15.94:0/1331732874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0105cf0 0x7fefa01069f0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- 172.21.15.94:0/1331732874 >> 172.21.15.94:0/1331732874 conn(0x7fefa0100f00 msgr2=0x7fefa0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- 172.21.15.94:0/1331732874 shutdown_connections 2024-04-25T14:05:54.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- 172.21.15.94:0/1331732874 wait complete. 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 Processor -- start 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.536+0000 7fefa8196640 1 -- start start 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0105cf0 0x7fefa019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa010bc30 0x7fefa01a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01117f0 con 0x7fefa0106f30 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0111670 con 0x7fefa010bc30 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa8196640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0111970 con 0x7fefa0105cf0 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50152/0 (socket says 172.21.15.94:50152) 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 -- 172.21.15.94:0/3850882970 learned_addr learned my addr 172.21.15.94:0/3850882970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5f32640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0105cf0 0x7fefa019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa6733640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa010bc30 0x7fefa01a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 -- 172.21.15.94:0/3850882970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0105cf0 msgr2=0x7fefa019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0105cf0 0x7fefa019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 -- 172.21.15.94:0/3850882970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa010bc30 msgr2=0x7fefa01a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa010bc30 0x7fefa01a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 -- 172.21.15.94:0/3850882970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01a3f40 con 0x7fefa0106f30 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.537+0000 7fefa5731640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9f0 tx=0x7fef9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.538+0000 7fef96ffd640 1 -- 172.21.15.94:0/3850882970 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000cd80 con 0x7fefa0106f30 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.538+0000 7fef96ffd640 1 -- 172.21.15.94:0/3850882970 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef900108e0 con 0x7fefa0106f30 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.538+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01a4230 con 0x7fefa0106f30 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.538+0000 7fef96ffd640 1 -- 172.21.15.94:0/3850882970 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90018980 con 0x7fefa0106f30 2024-04-25T14:05:54.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.538+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01a4780 con 0x7fefa0106f30 2024-04-25T14:05:54.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.540+0000 7fefaf04a740 1 Processor -- start 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.540+0000 7fef96ffd640 1 -- 172.21.15.94:0/3850882970 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fef90010430 con 0x7fefa0106f30 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.540+0000 7fefaf04a740 1 -- start start 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 0x55565fcf2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcf3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55565fc57d80 con 0x55565f951390 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55565fc118b0 con 0x55565f9284d0 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefaf04a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55565fc53d40 con 0x55565f9306b0 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50156/0 (socket says 172.21.15.94:50156) 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 -- 172.21.15.94:0/700799946 learned_addr learned my addr 172.21.15.94:0/700799946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa5f32640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 0x55565fcf2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa5731640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcf3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 -- 172.21.15.94:0/700799946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 msgr2=0x55565fcf3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcf3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 -- 172.21.15.94:0/700799946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 msgr2=0x55565fcf2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 0x55565fcf2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 -- 172.21.15.94:0/700799946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565fbe2a40 con 0x55565f951390 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa5f32640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 0x55565fcf2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fef96ffd640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 0x7fef8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa5f32640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 0x7fef8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fefa6733640 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fef98009ca0 tx=0x7fef98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b45010e317d1751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.541+0000 7fef96ffd640 1 -- 172.21.15.94:0/3850882970 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef900def90 con 0x7fefa0106f30 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fef8bfff640 1 -- 172.21.15.94:0/700799946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9801e070 con 0x55565f951390 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fef8bfff640 1 -- 172.21.15.94:0/700799946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98009e50 con 0x55565f951390 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fef8bfff640 1 -- 172.21.15.94:0/700799946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800ea10 con 0x55565f951390 2024-04-25T14:05:54.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fefaf04a740 1 -- 172.21.15.94:0/700799946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 msgr2=0x55565fcf2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fefaf04a740 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fef98009ca0 tx=0x7fef98011810 comp rx=0 tx=0).stop 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.542+0000 7fefa5f32640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 0x7fef8c0b2000 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f80 tx=0x7fef9c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 -- 172.21.15.94:0/700799946 shutdown_connections 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcf3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f9284d0 0x55565fcf2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 --2- 172.21.15.94:0/700799946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f951390 0x55565fcf2a60 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 -- 172.21.15.94:0/700799946 >> 172.21.15.94:0/700799946 conn(0x55565f9454e0 msgr2=0x55565fcee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 -- 172.21.15.94:0/700799946 shutdown_connections 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 -- 172.21.15.94:0/700799946 wait complete. 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 -- start start 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.543+0000 7fefaf04a740 1 Processor -- start 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcfcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd03e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55565fcf4780 con 0x55565f9284d0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55565fbded10 con 0x55565f951390 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55565fcf4900 con 0x55565f9306b0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 10 client.0 _ll_register_callbacks cb 0x55565fc0b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50162/0 (socket says 172.21.15.94:50162) 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 -- 172.21.15.94:0/935312168 learned_addr learned my addr 172.21.15.94:0/935312168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa6733640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd03e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5731640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcfcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 -- 172.21.15.94:0/935312168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 msgr2=0x55565fcfcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcfcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 -- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 msgr2=0x55565fd03e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd03e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 -- 172.21.15.94:0/935312168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565fa337d0 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefa5f32640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fefa01055c0 tx=0x7fef9c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f891702e46760a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fef897fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c003e40 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef9c004250 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.544+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef98009e50 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef9c004450 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 10 client.24120 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55565f9693f0 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98009e50 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fef9c051070 con 0x55565f9284d0 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 10 client.24120 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 10 client.24120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 20 client.24120 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:54.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f90ff60 0x55565fd0a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 4 client.24120 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:54.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55565fc65990 con 0x55565f90ff60 2024-04-25T14:05:54.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.545+0000 7fefa5731640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f90ff60 0x55565fd0a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.546+0000 7fef897fa640 10 client.24120 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.546+0000 7fefa5731640 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f90ff60 0x55565fd0a340 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272836d6e5b08a9d server_cookie=f70e01dc1503ed25 in_seq=0 out_seq=0 2024-04-25T14:05:54.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fef897fa640 1 -- 172.21.15.94:0/935312168 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55565fc65990 con 0x55565f90ff60 2024-04-25T14:05:54.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fef897fa640 10 client.24120 handle_command_reply: tid=0 2024-04-25T14:05:54.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 client.24120 shutdown 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 msgr2=0x55565fcfc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fefa01055c0 tx=0x7fef9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 shutdown_connections 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f90ff60 0x55565fd0a340 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd03e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9306b0 0x55565fcfcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fef897fa640 0 client.24120 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 --2- 172.21.15.94:0/935312168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9284d0 0x55565fcfc9d0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 >> 172.21.15.94:0/935312168 conn(0x55565f9454e0 msgr2=0x55565fcf1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 shutdown_connections 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 1 -- 172.21.15.94:0/935312168 wait complete. 2024-04-25T14:05:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.551+0000 7fefaf04a740 20 client.24120 trim_cache size 0 max 16384 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.569+0000 7fefaf04a740 1 Processor -- start 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.569+0000 7fefaf04a740 1 -- start start 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fcfb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fcfccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ad700 con 0x55565f9306b0 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01af240 con 0x55565f951390 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefaf04a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01addd0 con 0x55565f9284d0 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50178/0 (socket says 172.21.15.94:50178) 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 -- 172.21.15.94:0/2797257983 learned_addr learned my addr 172.21.15.94:0/2797257983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa6733640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fcfccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5f32640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fcfb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 -- 172.21.15.94:0/2797257983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 msgr2=0x55565fcfb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fcfb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 -- 172.21.15.94:0/2797257983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 msgr2=0x55565fcfccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fcfccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 -- 172.21.15.94:0/2797257983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565f9290f0 con 0x55565f9306b0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5f32640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fcfb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fefa5731640 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7fef900a3500 tx=0x7fef900dc8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c1ad8d59cca1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fef8b7fe640 1 -- 172.21.15.94:0/2797257983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900a6bf0 con 0x55565f9306b0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fef8b7fe640 1 -- 172.21.15.94:0/2797257983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef90010720 con 0x55565f9306b0 2024-04-25T14:05:54.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.570+0000 7fef8b7fe640 1 -- 172.21.15.94:0/2797257983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90010a90 con 0x55565f9306b0 2024-04-25T14:05:54.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.571+0000 7fefaf04a740 1 -- 172.21.15.94:0/2797257983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 msgr2=0x55565fcf7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.571+0000 7fefaf04a740 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7fef900a3500 tx=0x7fef900dc8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.572+0000 7fefaf04a740 1 -- 172.21.15.94:0/2797257983 shutdown_connections 2024-04-25T14:05:54.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.572+0000 7fefaf04a740 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fcfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.572+0000 7fefaf04a740 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fcf7780 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.572+0000 7fefaf04a740 1 --2- 172.21.15.94:0/2797257983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fcfb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.572+0000 7fefaf04a740 1 -- 172.21.15.94:0/2797257983 >> 172.21.15.94:0/2797257983 conn(0x55565f9454e0 msgr2=0x55565fcfc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.594+0000 7fefaf04a740 1 -- 172.21.15.94:0/2797257983 shutdown_connections 2024-04-25T14:05:54.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.598+0000 7fefaf04a740 1 -- 172.21.15.94:0/2797257983 wait complete. 2024-04-25T14:05:54.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.598+0000 7fefaf04a740 1 -- start start 2024-04-25T14:05:54.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.599+0000 7fefaf04a740 1 Processor -- start 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fd013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fd018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01aeb70 con 0x55565f9306b0 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ae4a0 con 0x55565f951390 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefaf04a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefa01ab6e0 con 0x55565f9284d0 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefa6733640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefa6733640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:41714/0 (socket says 172.21.15.94:41714) 2024-04-25T14:05:54.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.600+0000 7fefa6733640 1 -- 172.21.15.94:0/3044398720 learned_addr learned my addr 172.21.15.94:0/3044398720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:05:54.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefaf04a740 10 client.0 _ll_register_callbacks cb 0x7fef9c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:54.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa5731640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fd018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa6733640 1 -- 172.21.15.94:0/3044398720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 msgr2=0x55565fd013b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:05:54.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa6733640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fd013b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa6733640 1 -- 172.21.15.94:0/3044398720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 msgr2=0x55565fd018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:54.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa6733640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fd018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.601+0000 7fefa6733640 1 -- 172.21.15.94:0/3044398720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565fc184b0 con 0x55565f951390 2024-04-25T14:05:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.602+0000 7fefa6733640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef98009800 tx=0x7fef98009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e996ea333f7b8d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.602+0000 7fef897fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:05:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.602+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98028070 con 0x55565f951390 2024-04-25T14:05:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.603+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55565fc5f800 con 0x55565f951390 2024-04-25T14:05:54.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.603+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef98003a00 con 0x55565f951390 2024-04-25T14:05:54.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.608+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fef9801e070 con 0x55565f951390 2024-04-25T14:05:54.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.609+0000 7fefaf04a740 10 client.33503 fetch_fsmap learned FSMap version 39 2024-04-25T14:05:54.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.609+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55565fd71a00 con 0x55565f951390 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9801f040 con 0x55565f951390 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fef98019cd0 con 0x55565f951390 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 10 client.33503 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 10 client.33503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 20 client.33503 populate_metadata read hostname 'smithi094' 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f8e53a0 0x55565fd74370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:54.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 4 client.33503 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:54.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.611+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55565fc5f800 con 0x55565f8e53a0 2024-04-25T14:05:54.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.617+0000 7fefa5f32640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f8e53a0 0x55565fd74370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.636+0000 7fefa5f32640 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f8e53a0 0x55565fd74370 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba8cfcee1256a0c server_cookie=37029f32be39d978 in_seq=0 out_seq=0 2024-04-25T14:05:54.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.636+0000 7fef897fa640 10 client.33503 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:54 smithi137 ceph-mon[32568]: pgmap v1074: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-04-25T14:05:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.642+0000 7fef897fa640 1 -- 172.21.15.94:0/3044398720 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55565fc5f800 con 0x55565f8e53a0 2024-04-25T14:05:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.642+0000 7fef897fa640 10 client.33503 handle_command_reply: tid=0 2024-04-25T14:05:54.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.642+0000 7fefaf04a740 1 client.33503 shutdown 2024-04-25T14:05:54.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.649+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 msgr2=0x55565fd706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.649+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef98009800 tx=0x7fef98009e70 comp rx=0 tx=0).stop 2024-04-25T14:05:54.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 shutdown_connections 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55565f8e53a0 0x55565fd74370 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55565f951390 0x55565fd706e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55565f9306b0 0x55565fd018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 --2- 172.21.15.94:0/3044398720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55565f9284d0 0x55565fd013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 >> 172.21.15.94:0/3044398720 conn(0x55565f9454e0 msgr2=0x55565fcec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.650+0000 7fef897fa640 0 client.33503 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:05:54.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.656+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 shutdown_connections 2024-04-25T14:05:54.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.660+0000 7fefaf04a740 1 -- 172.21.15.94:0/3044398720 wait complete. 2024-04-25T14:05:54.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.662+0000 7fefaf04a740 20 client.33503 trim_cache size 0 max 16384 2024-04-25T14:05:54.669 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (229 inodes in the stack)", 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "92d7fbf4-187a-4f86-addd-8c14c264b1e0": { 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "92d7fbf4-187a-4f86-addd-8c14c264b1e0", 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:05:54.670 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.668+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 msgr2=0x7fef8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.668+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 0x7fef8c0b2000 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f80 tx=0x7fef9c005da0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.668+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 msgr2=0x7fefa019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.668+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e9f0 tx=0x7fef9000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 shutdown_connections 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef8c0afaf0 0x7fef8c0b2000 secure :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f80 tx=0x7fef9c005da0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa010bc30 0x7fefa01a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefa0106f30 0x7fefa019c940 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 --2- 172.21.15.94:0/3850882970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa0105cf0 0x7fefa019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:54.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.670+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 >> 172.21.15.94:0/3850882970 conn(0x7fefa0100f00 msgr2=0x7fefa0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.672+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 shutdown_connections 2024-04-25T14:05:54.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:05:54.672+0000 7fefa8196640 1 -- 172.21.15.94:0/3850882970 wait complete. 2024-04-25T14:05:54.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:54 smithi094 ceph-mon[29195]: pgmap v1074: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-04-25T14:05:54.693 INFO:tasks.cephfs.filesystem:scrub status for tag:92d7fbf4-187a-4f86-addd-8c14c264b1e0 - {'path': '/', 'tag': '92d7fbf4-187a-4f86-addd-8c14c264b1e0', 'options': 'recursive,force'} 2024-04-25T14:05:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:54 smithi138 ceph-mon[32977]: pgmap v1074: 97 pgs: 97 active+clean; 286 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 472 op/s 2024-04-25T14:05:55.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 225019 18.82 MB/sec execute 517 sec latency 219.607 ms 2024-04-25T14:05:55.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:55.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:55 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:05:55.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:55.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:55 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:05:55.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:55.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:55 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 225333 18.82 MB/sec execute 518 sec latency 144.814 ms 2024-04-25T14:05:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: pgmap v1075: 97 pgs: 97 active+clean; 283 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:56 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.647 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: pgmap v1075: 97 pgs: 97 active+clean; 283 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.648 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:56 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: pgmap v1075: 97 pgs: 97 active+clean; 283 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:56.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:56 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:57.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 225694 18.81 MB/sec execute 519 sec latency 70.456 ms 2024-04-25T14:05:58.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 226071 18.82 MB/sec execute 520 sec latency 69.136 ms 2024-04-25T14:05:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:05:58 smithi137 ceph-mon[32568]: pgmap v1076: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:05:58.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:05:58 smithi094 ceph-mon[29195]: pgmap v1076: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:05:58.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:05:58 smithi138 ceph-mon[32977]: pgmap v1076: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 501 op/s 2024-04-25T14:05:59.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 226514 18.83 MB/sec execute 521 sec latency 62.948 ms 2024-04-25T14:06:00.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 226792 18.81 MB/sec execute 522 sec latency 288.014 ms 2024-04-25T14:06:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:00 smithi137 ceph-mon[32568]: pgmap v1077: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-04-25T14:06:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:00 smithi094 ceph-mon[29195]: pgmap v1077: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-04-25T14:06:00.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:00 smithi138 ceph-mon[32977]: pgmap v1077: 97 pgs: 97 active+clean; 287 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-04-25T14:06:01.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 227143 18.81 MB/sec execute 523 sec latency 154.477 ms 2024-04-25T14:06:01.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:01.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:01 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:01 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:01 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:02.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 227522 18.83 MB/sec execute 524 sec latency 73.331 ms 2024-04-25T14:06:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:02 smithi137 ceph-mon[32568]: pgmap v1078: 97 pgs: 97 active+clean; 279 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-04-25T14:06:02.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:02 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:02 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:02.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:02 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:02 smithi094 ceph-mon[29195]: pgmap v1078: 97 pgs: 97 active+clean; 279 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-04-25T14:06:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:02 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:02 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:02 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:02 smithi138 ceph-mon[32977]: pgmap v1078: 97 pgs: 97 active+clean; 279 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.8 MiB/s wr, 630 op/s 2024-04-25T14:06:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:02 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:02 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:02 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:03.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 227966 18.82 MB/sec execute 525 sec latency 115.977 ms 2024-04-25T14:06:04.161 INFO:tasks.workunit.client.0.smithi094.stdout: 10 228355 18.84 MB/sec execute 526 sec latency 65.247 ms 2024-04-25T14:06:04.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:04 smithi137 ceph-mon[32568]: pgmap v1079: 97 pgs: 97 active+clean; 278 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 538 op/s 2024-04-25T14:06:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:04 smithi094 ceph-mon[29195]: pgmap v1079: 97 pgs: 97 active+clean; 278 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 538 op/s 2024-04-25T14:06:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:04 smithi138 ceph-mon[32977]: pgmap v1079: 97 pgs: 97 active+clean; 278 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 538 op/s 2024-04-25T14:06:05.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 228650 18.83 MB/sec execute 527 sec latency 257.810 ms 2024-04-25T14:06:06.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 229016 18.82 MB/sec execute 528 sec latency 270.984 ms 2024-04-25T14:06:06.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:06 smithi137 ceph-mon[32568]: pgmap v1080: 97 pgs: 97 active+clean; 285 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-04-25T14:06:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:06 smithi094 ceph-mon[29195]: pgmap v1080: 97 pgs: 97 active+clean; 285 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-04-25T14:06:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:06 smithi138 ceph-mon[32977]: pgmap v1080: 97 pgs: 97 active+clean; 285 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-04-25T14:06:07.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 229313 18.82 MB/sec execute 529 sec latency 51.938 ms 2024-04-25T14:06:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 -- 172.21.15.94:0/1924136765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 msgr2=0x7fab8c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 --2- 172.21.15.94:0/1924136765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c10be80 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7fab84009ca0 tx=0x7fab8402a8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 -- 172.21.15.94:0/1924136765 shutdown_connections 2024-04-25T14:06:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 --2- 172.21.15.94:0/1924136765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c10be80 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 --2- 172.21.15.94:0/1924136765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c104d60 0x7fab8c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 --2- 172.21.15.94:0/1924136765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c103b20 0x7fab8c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 -- 172.21.15.94:0/1924136765 >> 172.21.15.94:0/1924136765 conn(0x7fab8c0fed10 msgr2=0x7fab8c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 -- 172.21.15.94:0/1924136765 shutdown_connections 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.217+0000 7fab9359d640 1 -- 172.21.15.94:0/1924136765 wait complete. 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 Processor -- start 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 -- start start 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c103b20 0x7fab8c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c104d60 0x7fab8c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10f6a0 con 0x7fab8c109a60 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10f520 con 0x7fab8c103b20 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab9359d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10f820 con 0x7fab8c104d60 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab91b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab91b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50104/0 (socket says 172.21.15.94:50104) 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab91b3a640 1 -- 172.21.15.94:0/1786273988 learned_addr learned my addr 172.21.15.94:0/1786273988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab91339640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c103b20 0x7fab8c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.218+0000 7fab90b38640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c104d60 0x7fab8c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 -- 172.21.15.94:0/1786273988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c104d60 msgr2=0x7fab8c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c104d60 0x7fab8c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 -- 172.21.15.94:0/1786273988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c103b20 msgr2=0x7fab8c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c103b20 0x7fab8c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 -- 172.21.15.94:0/1786273988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c1a1d80 con 0x7fab8c109a60 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab91b3a640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fab84006010 tx=0x7fab84003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab827fc640 1 -- 172.21.15.94:0/1786273988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84004040 con 0x7fab8c109a60 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1a2010 con 0x7fab8c109a60 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab827fc640 1 -- 172.21.15.94:0/1786273988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84004690 con 0x7fab8c109a60 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab827fc640 1 -- 172.21.15.94:0/1786273988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84030dd0 con 0x7fab8c109a60 2024-04-25T14:06:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.219+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a2630 con 0x7fab8c109a60 2024-04-25T14:06:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 Processor -- start 2024-04-25T14:06:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab827fc640 1 -- 172.21.15.94:0/1786273988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fab840041e0 con 0x7fab8c109a60 2024-04-25T14:06:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 -- start start 2024-04-25T14:06:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec2270ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec2277df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558ec21d6460 con 0x558ec1ed01b0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ec2190590 con 0x558ec1eaf4d0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab9a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ec21d2420 con 0x558ec1ea72f0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50114/0 (socket says 172.21.15.94:50114) 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.222+0000 7fab91339640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec2270ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 -- 172.21.15.94:0/118017061 learned_addr learned my addr 172.21.15.94:0/118017061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec2277df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab827fc640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 0x7fab780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 -- 172.21.15.94:0/118017061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 msgr2=0x558ec2270ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec2270ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 -- 172.21.15.94:0/118017061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 msgr2=0x558ec2277df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec2277df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 -- 172.21.15.94:0/118017061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec219d0d0 con 0x558ec1ed01b0 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab91b3a640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 0x7fab780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab90b38640 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fab7c012790 tx=0x7fab7c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dc4019fc7995a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab827fc640 1 -- 172.21.15.94:0/1786273988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fab841000b0 con 0x7fab8c109a60 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab777fe640 1 -- 172.21.15.94:0/118017061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c012d10 con 0x558ec1ed01b0 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab777fe640 1 -- 172.21.15.94:0/118017061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c010c20 con 0x558ec1ed01b0 2024-04-25T14:06:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.223+0000 7fab777fe640 1 -- 172.21.15.94:0/118017061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c01e9d0 con 0x558ec1ed01b0 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 -- 172.21.15.94:0/118017061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 msgr2=0x558ec2270970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab91b3a640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 0x7fab780b1ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fab84031930 tx=0x7fab8402a8e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fab7c012790 tx=0x7fab7c00b790 comp rx=0 tx=0).stop 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 -- 172.21.15.94:0/118017061 shutdown_connections 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec2277df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec2270ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 --2- 172.21.15.94:0/118017061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2270970 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 -- 172.21.15.94:0/118017061 >> 172.21.15.94:0/118017061 conn(0x558ec1ec4300 msgr2=0x558ec226c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 -- 172.21.15.94:0/118017061 shutdown_connections 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.224+0000 7fab9a44e740 1 -- 172.21.15.94:0/118017061 wait complete. 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 -- start start 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 Processor -- start 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ea72f0 0x558ec227a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec227abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558ec2278840 con 0x558ec1ed01b0 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558ec215db60 con 0x558ec1ea72f0 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ec22789c0 con 0x558ec1eaf4d0 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab9a44e740 10 client.0 _ll_register_callbacks cb 0x558ec21822b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50120/0 (socket says 172.21.15.94:50120) 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab90b38640 1 -- 172.21.15.94:0/3438052591 learned_addr learned my addr 172.21.15.94:0/3438052591 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab91339640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec227abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.225+0000 7fab90b38640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ea72f0 0x558ec227a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 -- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ea72f0 msgr2=0x558ec227a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ea72f0 0x558ec227a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 -- 172.21.15.94:0/3438052591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 msgr2=0x558ec2281b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 -- 172.21.15.94:0/3438052591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec1ee8210 con 0x558ec1eaf4d0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab90b38640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab91339640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec227abc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558ec2271d50 tx=0x7fab880119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6a6b53a9af01af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88013670 con 0x558ec1eaf4d0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c010c20 con 0x558ec1eaf4d0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fab88013810 con 0x558ec1eaf4d0 2024-04-25T14:06:07.230 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab88013a10 con 0x558ec1eaf4d0 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab9a44e740 10 client.33145 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ec1fb4620 con 0x558ec1eaf4d0 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.226+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c010c20 con 0x558ec1eaf4d0 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fab88027cc0 con 0x558ec1eaf4d0 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 10 client.33145 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 10 client.33145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 20 client.33145 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e8ed80 0x558ec22883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 4 client.33145 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ec21e4070 con 0x558ec1e8ed80 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab90b38640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e8ed80 0x558ec22883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab74ff9640 10 client.33145 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.227+0000 7fab90b38640 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e8ed80 0x558ec22883b0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d9834aa1126b64 server_cookie=7565c6d7042d3fd0 in_seq=0 out_seq=0 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab74ff9640 1 -- 172.21.15.94:0/3438052591 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ec21e4070 con 0x558ec1e8ed80 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab74ff9640 10 client.33145 handle_command_reply: tid=0 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 client.33145 shutdown 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 msgr2=0x558ec227abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec227abc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x558ec2271d50 tx=0x7fab880119d0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 shutdown_connections 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e8ed80 0x558ec22883b0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec2281b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec227abc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3438052591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ea72f0 0x558ec227a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab74ff9640 0 client.33145 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 >> 172.21.15.94:0/3438052591 conn(0x558ec1ec4300 msgr2=0x558ec226f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 shutdown_connections 2024-04-25T14:06:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 1 -- 172.21.15.94:0/3438052591 wait complete. 2024-04-25T14:06:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.232+0000 7fab9a44e740 20 client.33145 trim_cache size 0 max 16384 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 Processor -- start 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 -- start start 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec2281a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ed01b0 0x558ec226c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ab610 con 0x558ec1ea72f0 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ad150 con 0x558ec1ed01b0 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab9a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1abce0 con 0x558ec1eaf4d0 2024-04-25T14:06:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50132/0 (socket says 172.21.15.94:50132) 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 -- 172.21.15.94:0/3605109496 learned_addr learned my addr 172.21.15.94:0/3605109496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab91339640 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec2281a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ed01b0 0x558ec226c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 -- 172.21.15.94:0/3605109496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 msgr2=0x558ec2281a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec2281a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 -- 172.21.15.94:0/3605109496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ed01b0 msgr2=0x558ec226c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.248+0000 7fab90b38640 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ed01b0 0x558ec226c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.249+0000 7fab90b38640 1 -- 172.21.15.94:0/3605109496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec2281f80 con 0x558ec1ea72f0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.249+0000 7fab90b38640 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x558ec2288b50 tx=0x7fab7c00bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdbcb6366a016dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.249+0000 7fab76ffd640 1 -- 172.21.15.94:0/3605109496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c024040 con 0x558ec1ea72f0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.249+0000 7fab76ffd640 1 -- 172.21.15.94:0/3605109496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c016040 con 0x558ec1ea72f0 2024-04-25T14:06:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.249+0000 7fab76ffd640 1 -- 172.21.15.94:0/3605109496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c01ec00 con 0x558ec1ea72f0 2024-04-25T14:06:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- 172.21.15.94:0/3605109496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 msgr2=0x558ec226bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x558ec2288b50 tx=0x7fab7c00bbf0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- 172.21.15.94:0/3605109496 shutdown_connections 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1ed01b0 0x558ec226c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1eaf4d0 0x558ec2281a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 --2- 172.21.15.94:0/3605109496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ea72f0 0x558ec226bc50 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- 172.21.15.94:0/3605109496 >> 172.21.15.94:0/3605109496 conn(0x558ec1ec4300 msgr2=0x558ec2274ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- 172.21.15.94:0/3605109496 shutdown_connections 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- 172.21.15.94:0/3605109496 wait complete. 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 -- start start 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.250+0000 7fab9a44e740 1 Processor -- start 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec227f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec227f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1aca80 con 0x558ec1ed01b0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ac3b0 con 0x558ec1eaf4d0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1a95f0 con 0x558ec1ea72f0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 10 client.0 _ll_register_callbacks cb 0x7fab880094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50138/0 (socket says 172.21.15.94:50138) 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 -- 172.21.15.94:0/2686571977 learned_addr learned my addr 172.21.15.94:0/2686571977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab91339640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec227f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec227f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 -- 172.21.15.94:0/2686571977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 msgr2=0x558ec227f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec227f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 -- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 msgr2=0x558ec227f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec227f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 -- 172.21.15.94:0/2686571977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec2280b60 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab90b38640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0124b0 tx=0x7fab7c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03e5e10bc2e611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c010740 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fab7c0108e0 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.251+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c016040 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab7c010ae0 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 10 client.24144 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558ec22f0760 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c016040 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fab7c02a040 con 0x558ec1ed01b0 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 10 client.24144 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 10 client.24144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 20 client.24144 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e641c0 0x558ec22f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 4 client.24144 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ec21ddee0 con 0x558ec1e641c0 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.252+0000 7fab91339640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e641c0 0x558ec22f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.253+0000 7fab74ff9640 10 client.24144 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.253+0000 7fab91339640 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e641c0 0x558ec22f2e00 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be83c8df01e4fe12 server_cookie=b3019fd69345bc4f in_seq=0 out_seq=0 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab74ff9640 1 -- 172.21.15.94:0/2686571977 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ec21ddee0 con 0x558ec1e641c0 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab74ff9640 10 client.24144 handle_command_reply: tid=0 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 client.24144 shutdown 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 msgr2=0x558ec22805a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0124b0 tx=0x7fab7c00bb40 comp rx=0 tx=0).stop 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 shutdown_connections 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558ec1e641c0 0x558ec22f2e00 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ec1ed01b0 0x558ec22805a0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ec1eaf4d0 0x558ec227f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab74ff9640 0 client.24144 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.943+0000 7fab9a44e740 1 --2- 172.21.15.94:0/2686571977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ec1ea72f0 0x558ec227f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.944+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 >> 172.21.15.94:0/2686571977 conn(0x558ec1ec4300 msgr2=0x558ec226f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.944+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 shutdown_connections 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.944+0000 7fab9a44e740 1 -- 172.21.15.94:0/2686571977 wait complete. 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.944+0000 7fab9a44e740 20 client.24144 trim_cache size 0 max 16384 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:06:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:06:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:06:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 msgr2=0x7fab780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 0x7fab780b1ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fab84031930 tx=0x7fab8402a8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 msgr2=0x7fab8c1a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fab84006010 tx=0x7fab84003ba0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 shutdown_connections 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fab780afae0 0x7fab780b1ff0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab8c109a60 0x7fab8c1a1710 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c104d60 0x7fab8c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 --2- 172.21.15.94:0/1786273988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab8c103b20 0x7fab8c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 >> 172.21.15.94:0/1786273988 conn(0x7fab8c0fed10 msgr2=0x7fab8c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 shutdown_connections 2024-04-25T14:06:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:07.945+0000 7fab9359d640 1 -- 172.21.15.94:0/1786273988 wait complete. 2024-04-25T14:06:08.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 229441 18.80 MB/sec execute 530 sec latency 396.914 ms 2024-04-25T14:06:08.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:08 smithi137 ceph-mon[32568]: pgmap v1081: 97 pgs: 97 active+clean; 288 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 497 op/s 2024-04-25T14:06:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:08 smithi094 ceph-mon[29195]: pgmap v1081: 97 pgs: 97 active+clean; 288 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 497 op/s 2024-04-25T14:06:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:08 smithi138 ceph-mon[32977]: pgmap v1081: 97 pgs: 97 active+clean; 288 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 497 op/s 2024-04-25T14:06:09.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 229855 18.81 MB/sec execute 531 sec latency 72.233 ms 2024-04-25T14:06:10.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 230136 18.81 MB/sec execute 532 sec latency 232.265 ms 2024-04-25T14:06:10.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:10 smithi137 ceph-mon[32568]: pgmap v1082: 97 pgs: 97 active+clean; 288 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 390 op/s 2024-04-25T14:06:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:10 smithi094 ceph-mon[29195]: pgmap v1082: 97 pgs: 97 active+clean; 288 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 390 op/s 2024-04-25T14:06:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:10 smithi138 ceph-mon[32977]: pgmap v1082: 97 pgs: 97 active+clean; 288 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 390 op/s 2024-04-25T14:06:11.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 230493 18.80 MB/sec execute 533 sec latency 234.387 ms 2024-04-25T14:06:12.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 230896 18.80 MB/sec execute 534 sec latency 79.653 ms 2024-04-25T14:06:12.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:12 smithi137 ceph-mon[32568]: pgmap v1083: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-04-25T14:06:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:12 smithi094 ceph-mon[29195]: pgmap v1083: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-04-25T14:06:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:12 smithi138 ceph-mon[32977]: pgmap v1083: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 587 op/s 2024-04-25T14:06:13.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 231319 18.81 MB/sec execute 535 sec latency 106.273 ms 2024-04-25T14:06:14.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 231822 18.83 MB/sec execute 536 sec latency 54.933 ms 2024-04-25T14:06:14.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:14 smithi137 ceph-mon[32568]: pgmap v1084: 97 pgs: 97 active+clean; 276 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-04-25T14:06:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:14 smithi094 ceph-mon[29195]: pgmap v1084: 97 pgs: 97 active+clean; 276 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-04-25T14:06:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:14 smithi138 ceph-mon[32977]: pgmap v1084: 97 pgs: 97 active+clean; 276 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-04-25T14:06:15.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 232188 18.84 MB/sec execute 537 sec latency 112.158 ms 2024-04-25T14:06:16.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 232460 18.83 MB/sec execute 538 sec latency 282.453 ms 2024-04-25T14:06:16.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:16 smithi137 ceph-mon[32568]: pgmap v1085: 97 pgs: 97 active+clean; 278 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-04-25T14:06:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:16 smithi094 ceph-mon[29195]: pgmap v1085: 97 pgs: 97 active+clean; 278 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-04-25T14:06:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:16 smithi138 ceph-mon[32977]: pgmap v1085: 97 pgs: 97 active+clean; 278 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-04-25T14:06:17.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 232790 18.83 MB/sec execute 539 sec latency 72.954 ms 2024-04-25T14:06:18.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 233142 18.83 MB/sec execute 540 sec latency 73.202 ms 2024-04-25T14:06:18.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:18 smithi137 ceph-mon[32568]: pgmap v1086: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T14:06:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:18 smithi094 ceph-mon[29195]: pgmap v1086: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T14:06:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:18 smithi138 ceph-mon[32977]: pgmap v1086: 97 pgs: 97 active+clean; 281 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-04-25T14:06:19.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 233542 18.84 MB/sec execute 541 sec latency 108.006 ms 2024-04-25T14:06:20.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 233876 18.82 MB/sec execute 542 sec latency 94.269 ms 2024-04-25T14:06:20.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:20 smithi137 ceph-mon[32568]: pgmap v1087: 97 pgs: 97 active+clean; 281 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:20 smithi094 ceph-mon[29195]: pgmap v1087: 97 pgs: 97 active+clean; 281 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:20 smithi138 ceph-mon[32977]: pgmap v1087: 97 pgs: 97 active+clean; 281 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 -- 172.21.15.94:0/3915156749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 msgr2=0x7f79cc10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3915156749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc10bcf0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009ca0 tx=0x7f79c402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 -- 172.21.15.94:0/3915156749 shutdown_connections 2024-04-25T14:06:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3915156749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc10bcf0 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3915156749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc101790 0x7f79cc101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3915156749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc1050e0 0x7f79cc1054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 -- 172.21.15.94:0/3915156749 >> 172.21.15.94:0/3915156749 conn(0x7f79cc0fd390 msgr2=0x7f79cc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 -- 172.21.15.94:0/3915156749 shutdown_connections 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.069+0000 7f79d1f64640 1 -- 172.21.15.94:0/3915156749 wait complete. 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.070+0000 7f79d1f64640 1 Processor -- start 2024-04-25T14:06:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 -- start start 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc101790 0x7f79cc1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc1050e0 0x7f79cc1a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10f670 con 0x7f79cc102290 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10f4f0 con 0x7f79cc101790 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79d1f64640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10f7f0 con 0x7f79cc1050e0 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33820/0 (socket says 172.21.15.94:33820) 2024-04-25T14:06:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 -- 172.21.15.94:0/3613181363 learned_addr learned my addr 172.21.15.94:0/3613181363 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79cbfff640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc1050e0 0x7f79cc1a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc101790 0x7f79cc1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 -- 172.21.15.94:0/3613181363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc1050e0 msgr2=0x7f79cc1a7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc1050e0 0x7f79cc1a7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 -- 172.21.15.94:0/3613181363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc101790 msgr2=0x7f79cc1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc101790 0x7f79cc1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.071+0000 7f79caffd640 1 -- 172.21.15.94:0/3613181363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc194e80 con 0x7f79cc102290 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79caffd640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d770 tx=0x7f79c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79c8ff9640 1 -- 172.21.15.94:0/3613181363 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0017070 con 0x7f79cc102290 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc195170 con 0x7f79cc102290 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79c8ff9640 1 -- 172.21.15.94:0/3613181363 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c00054d0 con 0x7f79cc102290 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79c8ff9640 1 -- 172.21.15.94:0/3613181363 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0004c30 con 0x7f79cc102290 2024-04-25T14:06:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.072+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1956c0 con 0x7f79cc102290 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 Processor -- start 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 -- start start 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a14096970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a14096ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cd54d0 0x558a1409ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a13ffc460 con 0x558a13cd54d0 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a13fb6590 con 0x558a13cf61b0 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79d8e15740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a13ff8420 con 0x558a13ccd2f0 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a14096ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79cbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a14096970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a14096ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50570/0 (socket says 172.21.15.94:50570) 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.075+0000 7f79cb7fe640 1 -- 172.21.15.94:0/2568555314 learned_addr learned my addr 172.21.15.94:0/2568555314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79c8ff9640 1 -- 172.21.15.94:0/3613181363 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f79c0010460 con 0x7f79cc102290 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 -- 172.21.15.94:0/2568555314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 msgr2=0x558a14096ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a14096ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 -- 172.21.15.94:0/2568555314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cd54d0 msgr2=0x558a1409ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cd54d0 0x558a1409ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 -- 172.21.15.94:0/2568555314 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a13fc30d0 con 0x558a13cf61b0 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79cbfff640 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a14096970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f79c4031b50 tx=0x7f79c402f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fbf048d0c8c2c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79b1ffb640 1 -- 172.21.15.94:0/2568555314 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4031650 con 0x558a13cf61b0 2024-04-25T14:06:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79b1ffb640 1 -- 172.21.15.94:0/2568555314 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c4036030 con 0x558a13cf61b0 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79b1ffb640 1 -- 172.21.15.94:0/2568555314 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40047a0 con 0x558a13cf61b0 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.076+0000 7f79c8ff9640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 0x7f79b40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.077+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 0x7f79b40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.077+0000 7f79c8ff9640 1 -- 172.21.15.94:0/3613181363 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f79c00de790 con 0x7f79cc102290 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.077+0000 7f79d8e15740 1 -- 172.21.15.94:0/2568555314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 msgr2=0x558a14096970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.077+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 0x7f79b40b2070 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x558a14097d50 tx=0x7f79bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.077+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a14096970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f79c4031b50 tx=0x7f79c402f4c0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 -- 172.21.15.94:0/2568555314 shutdown_connections 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cd54d0 0x558a1409ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a14096ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2568555314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a14096970 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 -- 172.21.15.94:0/2568555314 >> 172.21.15.94:0/2568555314 conn(0x558a13cea300 msgr2=0x558a14092300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 -- 172.21.15.94:0/2568555314 shutdown_connections 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 -- 172.21.15.94:0/2568555314 wait complete. 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 -- start start 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.078+0000 7f79d8e15740 1 Processor -- start 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cf61b0 0x558a140a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a1409e660 con 0x558a13cf61b0 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a13f83b60 con 0x558a13cd54d0 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a1409e7e0 con 0x558a13ccd2f0 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50582/0 (socket says 172.21.15.94:50582) 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 -- 172.21.15.94:0/2837653837 learned_addr learned my addr 172.21.15.94:0/2837653837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79d8e15740 10 client.0 _ll_register_callbacks cb 0x558a13fa82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 -- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 msgr2=0x558a140a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 -- 172.21.15.94:0/2837653837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cf61b0 msgr2=0x558a140a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cf61b0 0x558a140a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.079+0000 7f79cb7fe640 1 -- 172.21.15.94:0/2837653837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a13d0e210 con 0x558a13ccd2f0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009880 tx=0x7f79bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23be6deec74d7a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc02e060 con 0x558a13ccd2f0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f79bc018030 con 0x558a13ccd2f0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c4036030 con 0x558a13ccd2f0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79bc009e40 con 0x558a13ccd2f0 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79d8e15740 10 client.33163 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.080+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a13dda620 con 0x558a13ccd2f0 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c4036030 con 0x558a13ccd2f0 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f79bc01c560 con 0x558a13ccd2f0 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 10 client.33163 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 10 client.33163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 20 client.33163 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13cb4d80 0x558a140ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 4 client.33163 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a1400a070 con 0x558a13cb4d80 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.081+0000 7f79caffd640 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13cb4d80 0x558a140ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.082+0000 7f79caffd640 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13cb4d80 0x558a140ae7f0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742a5d5cc2075349 server_cookie=103ae426d3c63a93 in_seq=0 out_seq=0 2024-04-25T14:06:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.082+0000 7f79aaffd640 10 client.33163 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79aaffd640 1 -- 172.21.15.94:0/2837653837 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a1400a070 con 0x558a13cb4d80 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79aaffd640 10 client.33163 handle_command_reply: tid=0 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 client.33163 shutdown 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 msgr2=0x558a140a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009880 tx=0x7f79bc004730 comp rx=0 tx=0).stop 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 shutdown_connections 2024-04-25T14:06:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13cb4d80 0x558a140ae7f0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13cf61b0 0x558a140a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 --2- 172.21.15.94:0/2837653837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13ccd2f0 0x558a140a05f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 >> 172.21.15.94:0/2837653837 conn(0x558a13cea300 msgr2=0x558a14091250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79aaffd640 0 client.33163 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 shutdown_connections 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.088+0000 7f79d8e15740 1 -- 172.21.15.94:0/2837653837 wait complete. 2024-04-25T14:06:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.089+0000 7f79d8e15740 20 client.33163 trim_cache size 0 max 16384 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.113+0000 7f79d8e15740 1 Processor -- start 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.113+0000 7f79d8e15740 1 -- start start 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a1409b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 0x558a140a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a140a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ba260 con 0x558a13ccd2f0 2024-04-25T14:06:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1bbda0 con 0x558a13cf61b0 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 0x558a140a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79caffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 0x558a140a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50598/0 (socket says 172.21.15.94:50598) 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79caffd640 1 -- 172.21.15.94:0/1063262200 learned_addr learned my addr 172.21.15.94:0/1063262200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79cbfff640 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a140a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ba930 con 0x558a13cd54d0 2024-04-25T14:06:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.114+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a1409b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 -- 172.21.15.94:0/1063262200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 msgr2=0x558a140a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 0x558a140a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 -- 172.21.15.94:0/1063262200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 msgr2=0x558a140a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a140a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 -- 172.21.15.94:0/1063262200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1409dfe0 con 0x558a13ccd2f0 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a1409b290 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f79bc019040 tx=0x7f79bc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d4682c8cb58a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79b17fa640 1 -- 172.21.15.94:0/1063262200 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc01c420 con 0x558a13ccd2f0 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79b17fa640 1 -- 172.21.15.94:0/1063262200 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79bc01c5c0 con 0x558a13ccd2f0 2024-04-25T14:06:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.115+0000 7f79b17fa640 1 -- 172.21.15.94:0/1063262200 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc004c10 con 0x558a13ccd2f0 2024-04-25T14:06:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 msgr2=0x558a1409b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a1409b290 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f79bc019040 tx=0x7f79bc0092a0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 shutdown_connections 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cf61b0 0x558a140a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cd54d0 0x558a140a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 --2- 172.21.15.94:0/1063262200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a1409b290 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.116+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 >> 172.21.15.94:0/1063262200 conn(0x558a13cea300 msgr2=0x558a140a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.117+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 shutdown_connections 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.117+0000 7f79d8e15740 1 -- 172.21.15.94:0/1063262200 wait complete. 2024-04-25T14:06:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.117+0000 7f79d8e15740 1 -- start start 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.117+0000 7f79d8e15740 1 Processor -- start 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a140a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79d8e15740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cf61b0 0x558a14114b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79d8e15740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1bb6d0 con 0x558a13ccd2f0 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a140a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a140a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33858/0 (socket says 172.21.15.94:33858) 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79cb7fe640 1 -- 172.21.15.94:0/4022775629 learned_addr learned my addr 172.21.15.94:0/4022775629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79caffd640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.118+0000 7f79cbfff640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cf61b0 0x558a14114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1bb000 con 0x558a13cd54d0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc19c5c0 con 0x558a13cf61b0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79d8e15740 10 client.0 _ll_register_callbacks cb 0x7f79bc01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 -- 172.21.15.94:0/4022775629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cf61b0 msgr2=0x558a14114b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cf61b0 0x558a14114b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 -- 172.21.15.94:0/4022775629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 msgr2=0x558a140a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a140a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a141150d0 con 0x558a13cd54d0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79caffd640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a4ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79c00a6bb0 tx=0x7f79c00de9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1048c29270998834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.119+0000 7f79aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c008c020 con 0x558a13cd54d0 2024-04-25T14:06:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f79c00a2eb0 con 0x558a13cd54d0 2024-04-25T14:06:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79bc01c5c0 con 0x558a13cd54d0 2024-04-25T14:06:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79c0020040 con 0x558a13cd54d0 2024-04-25T14:06:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79d8e15740 10 client.33536 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.120+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a14116250 con 0x558a13cd54d0 2024-04-25T14:06:21.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c0005680 con 0x558a13cd54d0 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f79c00debc0 con 0x558a13cd54d0 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79d8e15740 10 client.33536 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79d8e15740 10 client.33536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79d8e15740 20 client.33536 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13c8a1c0 0x558a14118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.122+0000 7f79d8e15740 4 client.33536 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.123+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f79bc01c5c0 con 0x558a13c8a1c0 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.124+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13c8a1c0 0x558a14118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.125+0000 7f79aaffd640 10 client.33536 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.125+0000 7f79cb7fe640 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13c8a1c0 0x558a14118bf0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919184840ae23b04 server_cookie=47b5e4b3ce9d468a in_seq=0 out_seq=0 2024-04-25T14:06:21.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 234097 18.81 MB/sec execute 543 sec latency 326.618 ms 2024-04-25T14:06:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.887+0000 7f79aaffd640 1 -- 172.21.15.94:0/4022775629 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f79bc01c5c0 con 0x558a13c8a1c0 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.887+0000 7f79aaffd640 10 client.33536 handle_command_reply: tid=0 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.887+0000 7f79d8e15740 1 client.33536 shutdown 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.887+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 msgr2=0x558a140a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.887+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a4ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79c00a6bb0 tx=0x7f79c00de9e0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 shutdown_connections 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558a13c8a1c0 0x558a14118bf0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79aaffd640 0 client.33536 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a13cf61b0 0x558a14114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a13cd54d0 0x558a140a4ea0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 --2- 172.21.15.94:0/4022775629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a13ccd2f0 0x558a140a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 >> 172.21.15.94:0/4022775629 conn(0x558a13cea300 msgr2=0x558a140a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 shutdown_connections 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 1 -- 172.21.15.94:0/4022775629 wait complete. 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.888+0000 7f79d8e15740 20 client.33536 trim_cache size 0 max 16384 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:06:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.889+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 msgr2=0x7f79b40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.889+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 0x7f79b40b2070 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x558a14097d50 tx=0x7f79bc005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.889+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 msgr2=0x7f79cc1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.889+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d770 tx=0x7f79c000dc30 comp rx=0 tx=0).stop 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 shutdown_connections 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f79b40afb60 0x7f79b40b2070 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79cc1050e0 0x7f79cc1a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79cc102290 0x7f79cc1a0d00 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 --2- 172.21.15.94:0/3613181363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79cc101790 0x7f79cc1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 >> 172.21.15.94:0/3613181363 conn(0x7f79cc0fd390 msgr2=0x7f79cc105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 shutdown_connections 2024-04-25T14:06:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:21.890+0000 7f79d1f64640 1 -- 172.21.15.94:0/3613181363 wait complete. 2024-04-25T14:06:22.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 234255 18.79 MB/sec execute 544 sec latency 482.459 ms 2024-04-25T14:06:22.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:22 smithi137 ceph-mon[32568]: pgmap v1088: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-04-25T14:06:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:22 smithi094 ceph-mon[29195]: pgmap v1088: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-04-25T14:06:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:22 smithi138 ceph-mon[32977]: pgmap v1088: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-04-25T14:06:23.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 234647 18.80 MB/sec execute 545 sec latency 92.712 ms 2024-04-25T14:06:24.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 235038 18.80 MB/sec execute 546 sec latency 94.737 ms 2024-04-25T14:06:24.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:24 smithi137 ceph-mon[32568]: pgmap v1089: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-04-25T14:06:24.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:24.694 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:24.716 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:24 smithi094 ceph-mon[29195]: pgmap v1089: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-04-25T14:06:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:24 smithi138 ceph-mon[32977]: pgmap v1089: 97 pgs: 97 active+clean; 287 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 349 op/s 2024-04-25T14:06:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 -- 172.21.15.94:0/1949833730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b41029e0 msgr2=0x7f33b41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 --2- 172.21.15.94:0/1949833730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b41029e0 0x7f33b41036e0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009900 tx=0x7f33a402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 -- 172.21.15.94:0/1949833730 shutdown_connections 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 --2- 172.21.15.94:0/1949833730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b4108920 0x7f33b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 --2- 172.21.15.94:0/1949833730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b4103c20 0x7f33b4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 --2- 172.21.15.94:0/1949833730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b41029e0 0x7f33b41036e0 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 -- 172.21.15.94:0/1949833730 >> 172.21.15.94:0/1949833730 conn(0x7f33b40fdbd0 msgr2=0x7f33b4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 -- 172.21.15.94:0/1949833730 shutdown_connections 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.822+0000 7f33b9584640 1 -- 172.21.15.94:0/1949833730 wait complete. 2024-04-25T14:06:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 Processor -- start 2024-04-25T14:06:24.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 -- start start 2024-04-25T14:06:24.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b41029e0 0x7f33b419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b4103c20 0x7f33b419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33b410e4e0 con 0x7f33b4108920 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b410e360 con 0x7f33b4103c20 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b9584640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33b410e660 con 0x7f33b41029e0 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b41029e0 0x7f33b419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37544/0 (socket says 172.21.15.94:37544) 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.823+0000 7f33b37fe640 1 -- 172.21.15.94:0/964893592 learned_addr learned my addr 172.21.15.94:0/964893592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 -- 172.21.15.94:0/964893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b41029e0 msgr2=0x7f33b419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b41029e0 0x7f33b419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 -- 172.21.15.94:0/964893592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b4103c20 msgr2=0x7f33b419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b4103c20 0x7f33b419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 -- 172.21.15.94:0/964893592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b41a20f0 con 0x7f33b4108920 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b37fe640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00eb10 tx=0x7f33ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b0ff9640 1 -- 172.21.15.94:0/964893592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004b20 con 0x7f33b4108920 2024-04-25T14:06:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b41a23e0 con 0x7f33b4108920 2024-04-25T14:06:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b0ff9640 1 -- 172.21.15.94:0/964893592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ac004cc0 con 0x7f33b4108920 2024-04-25T14:06:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b0ff9640 1 -- 172.21.15.94:0/964893592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac010690 con 0x7f33b4108920 2024-04-25T14:06:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.824+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b41a2930 con 0x7f33b4108920 2024-04-25T14:06:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.827+0000 7f33c0435740 1 Processor -- start 2024-04-25T14:06:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33b0ff9640 1 -- 172.21.15.94:0/964893592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f33ac005020 con 0x7f33b4108920 2024-04-25T14:06:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 -- start start 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56044d41cd80 con 0x56044d0f56b0 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56044d3d68b0 con 0x56044d0ed4d0 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.828+0000 7f33c0435740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56044d418d40 con 0x56044d116390 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4bbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60664/0 (socket says 172.21.15.94:60664) 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 -- 172.21.15.94:0/1341045981 learned_addr learned my addr 172.21.15.94:0/1341045981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33b0ff9640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 0x7f33a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 -- 172.21.15.94:0/1341045981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 msgr2=0x56044d4bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 -- 172.21.15.94:0/1341045981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 msgr2=0x56044d4bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 -- 172.21.15.94:0/1341045981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044d3a7a40 con 0x56044d0f56b0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33b37fe640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 0x7f33a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4bd080 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f339c013800 tx=0x7f339c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e09fb75ca927fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.829+0000 7f33b0ff9640 1 -- 172.21.15.94:0/964893592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f33ac0df210 con 0x7f33b4108920 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.830+0000 7f33a97fa640 1 -- 172.21.15.94:0/1341045981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c013310 con 0x56044d0f56b0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.830+0000 7f33a97fa640 1 -- 172.21.15.94:0/1341045981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c005020 con 0x56044d0f56b0 2024-04-25T14:06:24.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.830+0000 7f33a97fa640 1 -- 172.21.15.94:0/1341045981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c010a30 con 0x56044d0f56b0 2024-04-25T14:06:24.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- 172.21.15.94:0/1341045981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 msgr2=0x56044d4bd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33b37fe640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 0x7f33a00b2000 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f33ac08eaf0 tx=0x7f33ac0d5da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4bd080 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f339c013800 tx=0x7f339c005850 comp rx=0 tx=0).stop 2024-04-25T14:06:24.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- 172.21.15.94:0/1341045981 shutdown_connections 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4bd080 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 --2- 172.21.15.94:0/1341045981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- 172.21.15.94:0/1341045981 >> 172.21.15.94:0/1341045981 conn(0x56044d10a4e0 msgr2=0x56044d4b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- 172.21.15.94:0/1341045981 shutdown_connections 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- 172.21.15.94:0/1341045981 wait complete. 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.831+0000 7f33c0435740 1 -- start start 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 Processor -- start 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d116390 0x56044d4cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56044d4bdb30 con 0x56044d0ed4d0 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56044d3a3d10 con 0x56044d116390 2024-04-25T14:06:24.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56044d4bdcb0 con 0x56044d0f56b0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33c0435740 10 client.0 _ll_register_callbacks cb 0x56044d3d0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37564/0 (socket says 172.21.15.94:37564) 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33aa7fc640 1 -- 172.21.15.94:0/4285961161 learned_addr learned my addr 172.21.15.94:0/4285961161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d116390 0x56044d4cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.832+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 -- 172.21.15.94:0/4285961161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 msgr2=0x56044d4c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 -- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d116390 msgr2=0x56044d4cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d116390 0x56044d4cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 -- 172.21.15.94:0/4285961161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044d1f87d0 con 0x56044d0ed4d0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f339c009cd0 tx=0x7f339c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277958da81a50a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f339a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c010a30 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f339c004250 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.833+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c005020 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f339c0051d0 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 10 client.24168 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56044d12e3f0 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c005020 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f339c0035f0 con 0x56044d0ed4d0 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 10 client.24168 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 10 client.24168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 20 client.24168 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0d4f60 0x56044d4b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 4 client.24168 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:24.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.834+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044d42a990 con 0x56044d0d4f60 2024-04-25T14:06:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.836+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0d4f60 0x56044d4b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.837+0000 7f339a7fc640 10 client.24168 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:24.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.837+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0d4f60 0x56044d4b5620 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc71a53740684918 server_cookie=2c6a244f27439cc in_seq=0 out_seq=0 2024-04-25T14:06:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.840+0000 7f339a7fc640 1 -- 172.21.15.94:0/4285961161 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56044d42a990 con 0x56044d0d4f60 2024-04-25T14:06:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.840+0000 7f339a7fc640 10 client.24168 handle_command_reply: tid=0 2024-04-25T14:06:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 client.24168 shutdown 2024-04-25T14:06:24.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 msgr2=0x56044d4c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f339c009cd0 tx=0x7f339c005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 shutdown_connections 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0d4f60 0x56044d4b5620 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d116390 0x56044d4cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 --2- 172.21.15.94:0/4285961161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0ed4d0 0x56044d4c5d90 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 >> 172.21.15.94:0/4285961161 conn(0x56044d10a4e0 msgr2=0x56044d4b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f339a7fc640 0 client.24168 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 shutdown_connections 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 1 -- 172.21.15.94:0/4285961161 wait complete. 2024-04-25T14:06:24.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.841+0000 7f33c0435740 20 client.24168 trim_cache size 0 max 16384 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.863+0000 7f33c0435740 1 Processor -- start 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.863+0000 7f33c0435740 1 -- start start 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 0x56044d4cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60682/0 (socket says 172.21.15.94:60682) 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 0x56044d4cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33aa7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 0x56044d4cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37574/0 (socket says 172.21.15.94:37574) 2024-04-25T14:06:24.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33b2ffd640 1 -- 172.21.15.94:0/1371649021 learned_addr learned my addr 172.21.15.94:0/1371649021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41a9480 con 0x56044d116390 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41aafc0 con 0x56044d0ed4d0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41a9b50 con 0x56044d0f56b0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.864+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 -- 172.21.15.94:0/1371649021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 msgr2=0x56044d4c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 -- 172.21.15.94:0/1371649021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 msgr2=0x56044d4cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 0x56044d4cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 -- 172.21.15.94:0/1371649021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044d0ee0f0 con 0x56044d0f56b0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005ab0 tx=0x7f33a4031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba9fdc97c01d6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33a8ff9640 1 -- 172.21.15.94:0/1371649021 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4035040 con 0x56044d0f56b0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33a8ff9640 1 -- 172.21.15.94:0/1371649021 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a404a070 con 0x56044d0f56b0 2024-04-25T14:06:24.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.865+0000 7f33a8ff9640 1 -- 172.21.15.94:0/1371649021 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40035f0 con 0x56044d0f56b0 2024-04-25T14:06:24.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 msgr2=0x56044d4cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005ab0 tx=0x7f33a4031400 comp rx=0 tx=0).stop 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 shutdown_connections 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d116390 0x56044d4cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d0f56b0 0x56044d4cd150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 --2- 172.21.15.94:0/1371649021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.866+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 >> 172.21.15.94:0/1371649021 conn(0x56044d10a4e0 msgr2=0x56044d4c4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 shutdown_connections 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- 172.21.15.94:0/1371649021 wait complete. 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- start start 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 Processor -- start 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41aa8f0 con 0x56044d0f56b0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41aa220 con 0x56044d0ed4d0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41a7460 con 0x56044d116390 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33c0435740 10 client.0 _ll_register_callbacks cb 0x7f33ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37582/0 (socket says 172.21.15.94:37582) 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33b2ffd640 1 -- 172.21.15.94:0/347827171 learned_addr learned my addr 172.21.15.94:0/347827171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.867+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 -- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 msgr2=0x56044d4c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 -- 172.21.15.94:0/347827171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 msgr2=0x56044d4c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 -- 172.21.15.94:0/347827171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044d3dd4b0 con 0x56044d116390 2024-04-25T14:06:24.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33aa7fc640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4cc610 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f339c003b10 tx=0x7f339c003f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da7a42f1b7a86c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f339a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0109a0 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f339c005370 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a404a070 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f339c045050 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33c0435740 10 client.33184 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.868+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56044d4cdc90 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a404a070 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f339c038040 con 0x56044d116390 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 10 client.33184 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 10 client.33184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 20 client.33184 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:24.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0aa3a0 0x56044d541200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 4 client.33184 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56044d424800 con 0x56044d0aa3a0 2024-04-25T14:06:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.869+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0aa3a0 0x56044d541200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.870+0000 7f339a7fc640 10 client.33184 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:24.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.870+0000 7f33b2ffd640 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0aa3a0 0x56044d541200 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb6f48f6a0ae169 server_cookie=70d1dde9013a1c52 in_seq=0 out_seq=0 2024-04-25T14:06:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.872+0000 7f339a7fc640 1 -- 172.21.15.94:0/347827171 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56044d424800 con 0x56044d0aa3a0 2024-04-25T14:06:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.872+0000 7f339a7fc640 10 client.33184 handle_command_reply: tid=0 2024-04-25T14:06:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.872+0000 7f33c0435740 1 client.33184 shutdown 2024-04-25T14:06:24.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.872+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 msgr2=0x56044d4cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.872+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4cc610 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f339c003b10 tx=0x7f339c003f40 comp rx=0 tx=0).stop 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 shutdown_connections 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56044d0aa3a0 0x56044d541200 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56044d116390 0x56044d4cc610 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56044d0f56b0 0x56044d4c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f339a7fc640 0 client.33184 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 --2- 172.21.15.94:0/347827171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56044d0ed4d0 0x56044d4c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 >> 172.21.15.94:0/347827171 conn(0x56044d10a4e0 msgr2=0x56044d4ba480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 shutdown_connections 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 1 -- 172.21.15.94:0/347827171 wait complete. 2024-04-25T14:06:24.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.873+0000 7f33c0435740 20 client.33184 trim_cache size 0 max 16384 2024-04-25T14:06:24.877 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:06:24.877 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:06:24.877 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:06:24.877 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:06:24.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 msgr2=0x7f33a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 0x7f33a00b2000 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f33ac08eaf0 tx=0x7f33ac0d5da0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 msgr2=0x7f33b41a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00eb10 tx=0x7f33ac00c2d0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 shutdown_connections 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f33a00afaf0 0x7f33a00b2000 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b4108920 0x7f33b41a1a80 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b4103c20 0x7f33b419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 --2- 172.21.15.94:0/964893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33b41029e0 0x7f33b419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 >> 172.21.15.94:0/964893592 conn(0x7f33b40fdbd0 msgr2=0x7f33b4109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 shutdown_connections 2024-04-25T14:06:24.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:24.874+0000 7f33b9584640 1 -- 172.21.15.94:0/964893592 wait complete. 2024-04-25T14:06:24.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:06:24.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 -- 172.21.15.94:0/2294832942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c103c00 msgr2=0x7f821c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 --2- 172.21.15.94:0/2294832942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c103c00 0x7f821c104010 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f820c009900 tx=0x7f820c02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 -- 172.21.15.94:0/2294832942 shutdown_connections 2024-04-25T14:06:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 --2- 172.21.15.94:0/2294832942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c108900 0x7f821c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 --2- 172.21.15.94:0/2294832942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c103c00 0x7f821c104010 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 --2- 172.21.15.94:0/2294832942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c1029c0 0x7f821c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 -- 172.21.15.94:0/2294832942 >> 172.21.15.94:0/2294832942 conn(0x7f821c0fdbd0 msgr2=0x7f821c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 -- 172.21.15.94:0/2294832942 shutdown_connections 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.046+0000 7f822349a640 1 -- 172.21.15.94:0/2294832942 wait complete. 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 Processor -- start 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 -- start start 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c103c00 0x7f821c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c108900 0x7f821c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10e4c0 con 0x7f821c1029c0 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10e340 con 0x7f821c103c00 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f822349a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f821c10e640 con 0x7f821c108900 2024-04-25T14:06:25.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37598/0 (socket says 172.21.15.94:37598) 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 -- 172.21.15.94:0/2602809152 learned_addr learned my addr 172.21.15.94:0/2602809152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8220a35640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c103c00 0x7f821c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221a37640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c108900 0x7f821c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 -- 172.21.15.94:0/2602809152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c108900 msgr2=0x7f821c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c108900 0x7f821c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 -- 172.21.15.94:0/2602809152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c103c00 msgr2=0x7f821c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c103c00 0x7f821c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.047+0000 7f8221236640 1 -- 172.21.15.94:0/2602809152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c1a1d60 con 0x7f821c1029c0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f8221236640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9f0 tx=0x7f821800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800ccc0 con 0x7f821c1029c0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c1a1ff0 con 0x7f821c1029c0 2024-04-25T14:06:25.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1a2500 con 0x7f821c1029c0 2024-04-25T14:06:25.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82180044d0 con 0x7f821c1029c0 2024-04-25T14:06:25.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.048+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218021430 con 0x7f821c1029c0 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.049+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82000052c0 con 0x7f821c1029c0 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.051+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8218004020 con 0x7f821c1029c0 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.051+0000 7f82127fc640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 0x7f82080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.051+0000 7f8220a35640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 0x7f82080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.051+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f82180def40 con 0x7f821c1029c0 2024-04-25T14:06:25.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.052+0000 7f8220a35640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 0x7f82080b2000 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f820c009230 tx=0x7f820c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.057+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82180a7a80 con 0x7f821c1029c0 2024-04-25T14:06:25.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 235470 18.81 MB/sec execute 547 sec latency 82.935 ms 2024-04-25T14:06:25.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.328+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82000055b0 con 0x7f821c1029c0 2024-04-25T14:06:25.337 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:06:25.337 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:06:25.337 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:06:25.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.336+0000 7f82127fc640 1 -- 172.21.15.94:0/2602809152 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f82180a7380 con 0x7f821c1029c0 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.340+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 msgr2=0x7f82080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.340+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 0x7f82080b2000 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f820c009230 tx=0x7f820c031400 comp rx=0 tx=0).stop 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.340+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 msgr2=0x7f821c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.340+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9f0 tx=0x7f821800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 shutdown_connections 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82080afaf0 0x7f82080b2000 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f821c108900 0x7f821c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f821c103c00 0x7f821c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 --2- 172.21.15.94:0/2602809152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c1029c0 0x7f821c19a220 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.341+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 >> 172.21.15.94:0/2602809152 conn(0x7f821c0fdbd0 msgr2=0x7f821c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.345+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 shutdown_connections 2024-04-25T14:06:25.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.345+0000 7f822349a640 1 -- 172.21.15.94:0/2602809152 wait complete. 2024-04-25T14:06:25.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:06:25.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 -- 172.21.15.94:0/4116231666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff8106f30 msgr2=0x7fcff8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/4116231666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff8106f30 0x7fcff8109350 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009900 tx=0x7fcfec02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:25.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 -- 172.21.15.94:0/4116231666 shutdown_connections 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/4116231666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8109890 0x7fcff810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/4116231666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff8106f30 0x7fcff8109350 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/4116231666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff81045d0 0x7fcff81069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 -- 172.21.15.94:0/4116231666 >> 172.21.15.94:0/4116231666 conn(0x7fcff80fdbd0 msgr2=0x7fcff8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 -- 172.21.15.94:0/4116231666 shutdown_connections 2024-04-25T14:06:25.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.514+0000 7fcffd1ce640 1 -- 172.21.15.94:0/4116231666 wait complete. 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.515+0000 7fcffd1ce640 1 Processor -- start 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.515+0000 7fcffd1ce640 1 -- start start 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 0x7fcff81020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8109890 0x7fcff8102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcff810f610 con 0x7fcff81045d0 2024-04-25T14:06:25.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8109890 0x7fcff8102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 0x7fcff81020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37624/0 (socket says 172.21.15.94:37624) 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff6d9d640 1 -- 172.21.15.94:0/2969693351 learned_addr learned my addr 172.21.15.94:0/2969693351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcff810f490 con 0x7fcff8109890 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcff810f790 con 0x7fcff8106f30 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 -- 172.21.15.94:0/2969693351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8109890 msgr2=0x7fcff8102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8109890 0x7fcff8102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 -- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 msgr2=0x7fcff8103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff659c640 1 -- 172.21.15.94:0/2969693351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff8102d30 con 0x7fcff8106f30 2024-04-25T14:06:25.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.516+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcff659c640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 0x7fcff81020c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcfdffff640 1 -- 172.21.15.94:0/2969693351 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec03f070 con 0x7fcff8106f30 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcfdffff640 1 -- 172.21.15.94:0/2969693351 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfec031910 con 0x7fcff8106f30 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a4630 con 0x7fcff8106f30 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcfdffff640 1 -- 172.21.15.94:0/2969693351 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec035070 con 0x7fcff8106f30 2024-04-25T14:06:25.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.517+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a4a30 con 0x7fcff8106f30 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 Processor -- start 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 -- start start 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 0x56194b58bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c86b0 0x56194b58ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56194b4efd80 con 0x56194b1e9390 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56194b4a9a20 con 0x56194b1c04d0 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fd00407e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56194b4ebd40 con 0x56194b1c86b0 2024-04-25T14:06:25.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcfdffff640 1 -- 172.21.15.94:0/2969693351 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcfec043400 con 0x7fcff8106f30 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37628/0 (socket says 172.21.15.94:37628) 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 -- 172.21.15.94:0/196826956 learned_addr learned my addr 172.21.15.94:0/196826956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 0x56194b58bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff659c640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c86b0 0x56194b58ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 -- 172.21.15.94:0/196826956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c86b0 msgr2=0x56194b58ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c86b0 0x56194b58ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 -- 172.21.15.94:0/196826956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 msgr2=0x56194b58bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 0x56194b58bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff759e640 1 -- 172.21.15.94:0/196826956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194b4b4c30 con 0x56194b1e9390 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.520+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 0x56194b58bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcff759e640 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009f90 tx=0x7fcff00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14e0c5cd0e63f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcfdffff640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 0x7fcfe00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcfdd7fa640 1 -- 172.21.15.94:0/196826956 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0004d20 con 0x56194b1e9390 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcfdd7fa640 1 -- 172.21.15.94:0/196826956 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0004050 con 0x56194b1e9390 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcfdd7fa640 1 -- 172.21.15.94:0/196826956 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0005240 con 0x56194b1e9390 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 0x7fcfe00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.521+0000 7fcfdffff640 1 -- 172.21.15.94:0/2969693351 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcfec0ff1f0 con 0x7fcff8106f30 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- 172.21.15.94:0/196826956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 msgr2=0x56194b58b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009f90 tx=0x7fcff00118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 0x7fcfe00b2000 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56194b58b530 tx=0x7fcfe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- 172.21.15.94:0/196826956 shutdown_connections 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c86b0 0x56194b58ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c04d0 0x56194b58bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 --2- 172.21.15.94:0/196826956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1e9390 0x56194b58b810 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- 172.21.15.94:0/196826956 >> 172.21.15.94:0/196826956 conn(0x56194b1dd4e0 msgr2=0x56194b587020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- 172.21.15.94:0/196826956 shutdown_connections 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- 172.21.15.94:0/196826956 wait complete. 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 -- start start 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.522+0000 7fd00407e740 1 Processor -- start 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b595ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1e9390 0x56194b59cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56194b58d530 con 0x56194b1c04d0 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56194b476dc0 con 0x56194b1c86b0 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56194b58d6b0 con 0x56194b1e9390 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 10 client.0 _ll_register_callbacks cb 0x56194b492600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37638/0 (socket says 172.21.15.94:37638) 2024-04-25T14:06:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 -- 172.21.15.94:0/2086099330 learned_addr learned my addr 172.21.15.94:0/2086099330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff659c640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b595ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff759e640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1e9390 0x56194b59cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 -- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1e9390 msgr2=0x56194b59cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1e9390 0x56194b59cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 -- 172.21.15.94:0/2086099330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 msgr2=0x56194b595ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b595ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 -- 172.21.15.94:0/2086099330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194b2013f0 con 0x56194b1c04d0 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004280 tx=0x7fcfe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cdf9abb24d9b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcfdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80044c0 con 0x56194b1c04d0 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcfe8003af0 con 0x56194b1c04d0 2024-04-25T14:06:25.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff0004050 con 0x56194b1c04d0 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfe8003cf0 con 0x56194b1c04d0 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 10 client.24192 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.523+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56194b2cb7d0 con 0x56194b1c04d0 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0004050 con 0x56194b1c04d0 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcfe8027070 con 0x56194b1c04d0 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 10 client.24192 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 10 client.24192 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 20 client.24192 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1a7f60 0x56194b5a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 4 client.24192 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56194b4fd990 con 0x56194b1a7f60 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fcff659c640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1a7f60 0x56194b5a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fcfdaff5640 10 client.24192 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.524+0000 7fcff659c640 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1a7f60 0x56194b5a34a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a4465441798b67e server_cookie=d7877bb8217c58c1 in_seq=0 out_seq=0 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fcfdaff5640 1 -- 172.21.15.94:0/2086099330 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56194b4fd990 con 0x56194b1a7f60 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fcfdaff5640 10 client.24192 handle_command_reply: tid=0 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 client.24192 shutdown 2024-04-25T14:06:25.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 msgr2=0x56194b595660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004280 tx=0x7fcfe8004730 comp rx=0 tx=0).stop 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 shutdown_connections 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1a7f60 0x56194b5a34a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1e9390 0x56194b59cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b595ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fcfdaff5640 0 client.24192 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 --2- 172.21.15.94:0/2086099330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b595660 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.525+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 >> 172.21.15.94:0/2086099330 conn(0x56194b1dd4e0 msgr2=0x56194b586040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.526+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 shutdown_connections 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.526+0000 7fd00407e740 1 -- 172.21.15.94:0/2086099330 wait complete. 2024-04-25T14:06:25.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.526+0000 7fd00407e740 20 client.24192 trim_cache size 0 max 16384 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.541+0000 7fd00407e740 1 Processor -- start 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 -- start start 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1a7f60 0x56194b595940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b585610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ad720 con 0x56194b1c04d0 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81af260 con 0x56194b1c86b0 2024-04-25T14:06:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fd00407e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81addf0 con 0x56194b1a7f60 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37644/0 (socket says 172.21.15.94:37644) 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 -- 172.21.15.94:0/3907217058 learned_addr learned my addr 172.21.15.94:0/3907217058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff759e640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b585610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1a7f60 0x56194b595940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 -- 172.21.15.94:0/3907217058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1a7f60 msgr2=0x56194b595940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1a7f60 0x56194b595940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 -- 172.21.15.94:0/3907217058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 msgr2=0x56194b585610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b585610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 -- 172.21.15.94:0/3907217058 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194b590d80 con 0x56194b1c04d0 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff759e640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b585610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.542+0000 7fcff659c640 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0c73e0 tx=0x7fcfec101700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d17f3b8caa48e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fcfdcff9640 1 -- 172.21.15.94:0/3907217058 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec048040 con 0x56194b1c04d0 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fcfdcff9640 1 -- 172.21.15.94:0/3907217058 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfec0c71f0 con 0x56194b1c04d0 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fcfdcff9640 1 -- 172.21.15.94:0/3907217058 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0d0490 con 0x56194b1c04d0 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 -- 172.21.15.94:0/3907217058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 msgr2=0x56194b59ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0c73e0 tx=0x7fcfec101700 comp rx=0 tx=0).stop 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 -- 172.21.15.94:0/3907217058 shutdown_connections 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1c86b0 0x56194b585610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c04d0 0x56194b59ca90 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 --2- 172.21.15.94:0/3907217058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1a7f60 0x56194b595940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.543+0000 7fd00407e740 1 -- 172.21.15.94:0/3907217058 >> 172.21.15.94:0/3907217058 conn(0x56194b1dd4e0 msgr2=0x56194b595230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- 172.21.15.94:0/3907217058 shutdown_connections 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- 172.21.15.94:0/3907217058 wait complete. 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- start start 2024-04-25T14:06:25.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 Processor -- start 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1a7f60 0x56194b599870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c04d0 0x56194b599db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81aeb90 con 0x56194b1c86b0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ae4c0 con 0x56194b1a7f60 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ab700 con 0x56194b1c04d0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fd00407e740 10 client.0 _ll_register_callbacks cb 0x7fcfe8004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37646/0 (socket says 172.21.15.94:37646) 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff759e640 1 -- 172.21.15.94:0/3665287676 learned_addr learned my addr 172.21.15.94:0/3665287676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1a7f60 0x56194b599870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff659c640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c04d0 0x56194b599db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.544+0000 7fcff759e640 1 -- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c04d0 msgr2=0x56194b599db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcff759e640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c04d0 0x56194b599db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcff759e640 1 -- 172.21.15.94:0/3665287676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1a7f60 msgr2=0x56194b599870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcff759e640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1a7f60 0x56194b599870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcff759e640 1 -- 172.21.15.94:0/3665287676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194b59a3a0 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcff759e640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009f90 tx=0x7fcff0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329a514a18351498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0005870 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcff000f4e0 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec0c71f0 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcff0009a50 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 10 client.24198 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56194b60adf0 con 0x56194b1c86b0 2024-04-25T14:06:25.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfec0c71f0 con 0x56194b1c86b0 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcff002d600 con 0x56194b1c86b0 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 10 client.24198 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 10 client.24198 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 20 client.24198 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.545+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1e9390 0x56194b60d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 4 client.24198 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56194b4f7800 con 0x56194b1e9390 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1e9390 0x56194b60d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcfdaff5640 10 client.24198 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcff6d9d640 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1e9390 0x56194b60d670 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c346d79db2f6ea0 server_cookie=e241a18bfca710b0 in_seq=0 out_seq=0 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcfdaff5640 1 -- 172.21.15.94:0/3665287676 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56194b4f7800 con 0x56194b1e9390 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcfdaff5640 10 client.24198 handle_command_reply: tid=0 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 client.24198 shutdown 2024-04-25T14:06:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 msgr2=0x56194b609800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009f90 tx=0x7fcff0005270 comp rx=0 tx=0).stop 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 shutdown_connections 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x56194b1e9390 0x56194b60d670 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56194b1c86b0 0x56194b609800 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56194b1c04d0 0x56194b599db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fcfdaff5640 0 client.24198 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 --2- 172.21.15.94:0/3665287676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56194b1a7f60 0x56194b599870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 >> 172.21.15.94:0/3665287676 conn(0x56194b1dd4e0 msgr2=0x56194b58a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 shutdown_connections 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.546+0000 7fd00407e740 1 -- 172.21.15.94:0/3665287676 wait complete. 2024-04-25T14:06:25.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fd00407e740 20 client.24198 trim_cache size 0 max 16384 2024-04-25T14:06:25.556 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:25.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 msgr2=0x7fcfe00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 0x7fcfe00b2000 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56194b58b530 tx=0x7fcfe8005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 msgr2=0x7fcff81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 0x7fcff81020c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006010 tx=0x7fcfec004270 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 shutdown_connections 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcfe00afaf0 0x7fcfe00b2000 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcff8109890 0x7fcff8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff8106f30 0x7fcff81020c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 --2- 172.21.15.94:0/2969693351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcff81045d0 0x7fcff8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 >> 172.21.15.94:0/2969693351 conn(0x7fcff80fdbd0 msgr2=0x7fcff8105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 shutdown_connections 2024-04-25T14:06:25.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.547+0000 7fcffd1ce640 1 -- 172.21.15.94:0/2969693351 wait complete. 2024-04-25T14:06:25.558 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:06:25.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 -- 172.21.15.94:0/2720354990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8108930 msgr2=0x7febe810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 --2- 172.21.15.94:0/2720354990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8108930 0x7febe810ad50 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7febe0009ca0 tx=0x7febe002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 -- 172.21.15.94:0/2720354990 shutdown_connections 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 --2- 172.21.15.94:0/2720354990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8108930 0x7febe810ad50 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 --2- 172.21.15.94:0/2720354990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8103c30 0x7febe8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 --2- 172.21.15.94:0/2720354990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 -- 172.21.15.94:0/2720354990 >> 172.21.15.94:0/2720354990 conn(0x7febe80fdc00 msgr2=0x7febe8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 -- 172.21.15.94:0/2720354990 shutdown_connections 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.677+0000 7febee16c640 1 -- 172.21.15.94:0/2720354990 wait complete. 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 Processor -- start 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 -- start start 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 0x7febe81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e520 con 0x7febe8103c30 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e3a0 con 0x7febe81029f0 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.678+0000 7febee16c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febe810e6a0 con 0x7febe8108930 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37662/0 (socket says 172.21.15.94:37662) 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 -- 172.21.15.94:0/157493377 learned_addr learned my addr 172.21.15.94:0/157493377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe7fff640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 0x7febe81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 -- 172.21.15.94:0/157493377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 msgr2=0x7febe81a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 0x7febe81a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 -- 172.21.15.94:0/157493377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 msgr2=0x7febe806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 -- 172.21.15.94:0/157493377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe81a1ce0 con 0x7febe8103c30 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe7fff640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 0x7febe81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe77fe640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe6ffd640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7febdc009ae0 tx=0x7febdc00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febe4ff9640 1 -- 172.21.15.94:0/157493377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc007920 con 0x7febe8103c30 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.679+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe8198d80 con 0x7febe8103c30 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.680+0000 7febe4ff9640 1 -- 172.21.15.94:0/157493377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febdc005020 con 0x7febe8103c30 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.680+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81992d0 con 0x7febe8103c30 2024-04-25T14:06:25.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.680+0000 7febe4ff9640 1 -- 172.21.15.94:0/157493377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc004cd0 con 0x7febe8103c30 2024-04-25T14:06:25.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.682+0000 7febe4ff9640 1 -- 172.21.15.94:0/157493377 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7febdc0051d0 con 0x7febe8103c30 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 Processor -- start 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 -- start start 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002624810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002625a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562002588d80 con 0x5620022594d0 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562002542a20 con 0x5620022616b0 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febf5016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562002584d40 con 0x562002282390 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe4ff9640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 0x7febd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37668/0 (socket says 172.21.15.94:37668) 2024-04-25T14:06:25.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 -- 172.21.15.94:0/2220183106 learned_addr learned my addr 172.21.15.94:0/2220183106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe6ffd640 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002625a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe7fff640 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002624810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe7fff640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 0x7febd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe4ff9640 1 -- 172.21.15.94:0/157493377 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7febdc0e22d0 con 0x7febe8103c30 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 -- 172.21.15.94:0/2220183106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 msgr2=0x562002624810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002624810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 -- 172.21.15.94:0/2220183106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 msgr2=0x562002625a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002625a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.683+0000 7febe77fe640 1 -- 172.21.15.94:0/2220183106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200254dc30 con 0x5620022594d0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.684+0000 7febe77fe640 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7febd80114f0 tx=0x7febd80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22beaf0b9eb222c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.684+0000 7febe7fff640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 0x7febd00b2000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x562002625860 tx=0x7febe002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.684+0000 7febcdffb640 1 -- 172.21.15.94:0/2220183106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8005020 con 0x5620022594d0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.684+0000 7febcdffb640 1 -- 172.21.15.94:0/2220183106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd80047a0 con 0x5620022594d0 2024-04-25T14:06:25.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.684+0000 7febcdffb640 1 -- 172.21.15.94:0/2220183106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd800f840 con 0x5620022594d0 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- 172.21.15.94:0/2220183106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 msgr2=0x562002624d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7febd80114f0 tx=0x7febd80119b0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- 172.21.15.94:0/2220183106 shutdown_connections 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002625a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002624d90 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 --2- 172.21.15.94:0/2220183106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002624810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- 172.21.15.94:0/2220183106 >> 172.21.15.94:0/2220183106 conn(0x5620022764e0 msgr2=0x562002620020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- 172.21.15.94:0/2220183106 shutdown_connections 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- 172.21.15.94:0/2220183106 wait complete. 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.685+0000 7febf5016740 1 -- start start 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 Processor -- start 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 0x56200262e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x56200262ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562002626530 con 0x562002282390 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56200250fdc0 con 0x5620022616b0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5620026266b0 con 0x5620022594d0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febf5016740 10 client.0 _ll_register_callbacks cb 0x56200252b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 0x56200262e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37682/0 (socket says 172.21.15.94:37682) 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 -- 172.21.15.94:0/1352676260 learned_addr learned my addr 172.21.15.94:0/1352676260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe6ffd640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x56200262ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 -- 172.21.15.94:0/1352676260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 msgr2=0x56200262e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 0x56200262e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 -- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 msgr2=0x56200262ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x56200262ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 -- 172.21.15.94:0/1352676260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200229a3f0 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe77fe640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 0x56200262e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febe7fff640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7febe0009ca0 tx=0x7febe00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df53ca94eee643a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.686+0000 7febc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0031570 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd80047a0 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7febe002f660 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7febe002f860 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febf5016740 10 client.24210 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5620023647d0 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.687+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd80047a0 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7febe0030a40 con 0x562002282390 2024-04-25T14:06:25.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 10 client.24210 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 10 client.24210 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 20 client.24210 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562002240f60 0x56200263c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 4 client.24210 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562002596990 con 0x562002240f60 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febe77fe640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562002240f60 0x56200263c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febc6ffd640 10 client.24210 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.688+0000 7febe77fe640 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562002240f60 0x56200263c4f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85eff95d83fb4275 server_cookie=4046f17f6217143d in_seq=0 out_seq=0 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febc6ffd640 1 -- 172.21.15.94:0/1352676260 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562002596990 con 0x562002240f60 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febc6ffd640 10 client.24210 handle_command_reply: tid=0 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 client.24210 shutdown 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 msgr2=0x5620026359e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7febe0009ca0 tx=0x7febe00095c0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 shutdown_connections 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x562002240f60 0x56200263c4f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562002282390 0x5620026359e0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x56200262ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febc6ffd640 0 client.24210 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 --2- 172.21.15.94:0/1352676260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022594d0 0x56200262e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 >> 172.21.15.94:0/1352676260 conn(0x5620022764e0 msgr2=0x562002622f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 shutdown_connections 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 1 -- 172.21.15.94:0/1352676260 wait complete. 2024-04-25T14:06:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.689+0000 7febf5016740 20 client.24210 trim_cache size 0 max 16384 2024-04-25T14:06:25.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 Processor -- start 2024-04-25T14:06:25.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 -- start start 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022616b0 0x562002635900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562002282390 0x562002622e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a9330 con 0x5620022594d0 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe81aae70 con 0x562002282390 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febf5016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a9a00 con 0x5620022616b0 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.703+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37698/0 (socket says 172.21.15.94:37698) 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 -- 172.21.15.94:0/2042716507 learned_addr learned my addr 172.21.15.94:0/2042716507 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe7fff640 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562002282390 0x562002622e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe6ffd640 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022616b0 0x562002635900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 -- 172.21.15.94:0/2042716507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022616b0 msgr2=0x562002635900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022616b0 0x562002635900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 -- 172.21.15.94:0/2042716507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562002282390 msgr2=0x562002622e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562002282390 0x562002622e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 -- 172.21.15.94:0/2042716507 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562002635e40 con 0x5620022594d0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febe77fe640 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7febd8011eb0 tx=0x7febd80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f711880563c2ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febcd7fa640 1 -- 172.21.15.94:0/2042716507 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd802e020 con 0x5620022594d0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febcd7fa640 1 -- 172.21.15.94:0/2042716507 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8004610 con 0x5620022594d0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.704+0000 7febcd7fa640 1 -- 172.21.15.94:0/2042716507 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004d80 con 0x5620022594d0 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- 172.21.15.94:0/2042716507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 msgr2=0x5620026200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7febd8011eb0 tx=0x7febd80119b0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- 172.21.15.94:0/2042716507 shutdown_connections 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562002282390 0x562002622e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620022616b0 0x562002635900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 --2- 172.21.15.94:0/2042716507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x5620026200f0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- 172.21.15.94:0/2042716507 >> 172.21.15.94:0/2042716507 conn(0x5620022764e0 msgr2=0x56200261f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- 172.21.15.94:0/2042716507 shutdown_connections 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- 172.21.15.94:0/2042716507 wait complete. 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.705+0000 7febf5016740 1 -- start start 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 Processor -- start 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002633480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002634360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febe81aa7a0 con 0x5620022594d0 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febe81aa0d0 con 0x5620022616b0 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febe81a7310 con 0x562002282390 2024-04-25T14:06:25.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febf5016740 10 client.0 _ll_register_callbacks cb 0x7febe0009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37714/0 (socket says 172.21.15.94:37714) 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 -- 172.21.15.94:0/3965056553 learned_addr learned my addr 172.21.15.94:0/3965056553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe6ffd640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002633480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe7fff640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002634360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 -- 172.21.15.94:0/3965056553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 msgr2=0x562002634360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002634360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 -- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 msgr2=0x562002633480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002633480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 -- 172.21.15.94:0/3965056553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562002634920 con 0x5620022594d0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febe77fe640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7febd8009f70 tx=0x7febd8009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bd8afa26e96866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.706+0000 7febc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:25.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd802a4d0 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7febd802a840 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd8004610 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7febd802aa40 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 10 client.24216 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562002635ba0 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd8004610 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7febd802ace0 con 0x5620022594d0 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 10 client.24216 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 10 client.24216 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 20 client.24216 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5620022163a0 0x5620026a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 4 client.24216 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562002590800 con 0x5620022163a0 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.707+0000 7febe6ffd640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5620022163a0 0x5620026a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febc6ffd640 10 client.24216 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febe6ffd640 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5620022163a0 0x5620026a6cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=298617649ebbde71 server_cookie=5a2dd3b104942743 in_seq=0 out_seq=0 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febc6ffd640 1 -- 172.21.15.94:0/3965056553 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562002590800 con 0x5620022163a0 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febc6ffd640 10 client.24216 handle_command_reply: tid=0 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 client.24216 shutdown 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 msgr2=0x562002632f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7febd8009f70 tx=0x7febd8009d00 comp rx=0 tx=0).stop 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 shutdown_connections 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x5620022163a0 0x5620026a6cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562002282390 0x562002634360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620022616b0 0x562002633480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 --2- 172.21.15.94:0/3965056553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620022594d0 0x562002632f40 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febc6ffd640 0 client.24216 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 >> 172.21.15.94:0/3965056553 conn(0x5620022764e0 msgr2=0x56200261e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 shutdown_connections 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 1 -- 172.21.15.94:0/3965056553 wait complete. 2024-04-25T14:06:25.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.708+0000 7febf5016740 20 client.24216 trim_cache size 0 max 16384 2024-04-25T14:06:25.717 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 msgr2=0x7febd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 0x7febd00b2000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x562002625860 tx=0x7febe002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 msgr2=0x7febe819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7febdc009ae0 tx=0x7febdc00ff80 comp rx=0 tx=0).stop 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 shutdown_connections 2024-04-25T14:06:25.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7febd00afaf0 0x7febd00b2000 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe8108930 0x7febe81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febe8103c30 0x7febe819ef50 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 --2- 172.21.15.94:0/157493377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febe81029f0 0x7febe806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 >> 172.21.15.94:0/157493377 conn(0x7febe80fdc00 msgr2=0x7febe8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 shutdown_connections 2024-04-25T14:06:25.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.709+0000 7febee16c640 1 -- 172.21.15.94:0/157493377 wait complete. 2024-04-25T14:06:25.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:06:25.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 -- 172.21.15.94:0/2985808998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80106c10 msgr2=0x7f3a801090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 --2- 172.21.15.94:0/2985808998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80106c10 0x7f3a801090f0 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009900 tx=0x7f3a7402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:25.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 -- 172.21.15.94:0/2985808998 shutdown_connections 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 --2- 172.21.15.94:0/2985808998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80109780 0x7f3a8010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 --2- 172.21.15.94:0/2985808998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80106c10 0x7f3a801090f0 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 --2- 172.21.15.94:0/2985808998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80101e30 0x7f3a80102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 -- 172.21.15.94:0/2985808998 >> 172.21.15.94:0/2985808998 conn(0x7f3a800fd340 msgr2=0x7f3a800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 -- 172.21.15.94:0/2985808998 shutdown_connections 2024-04-25T14:06:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.848+0000 7f3a84820640 1 -- 172.21.15.94:0/2985808998 wait complete. 2024-04-25T14:06:25.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.849+0000 7f3a84820640 1 Processor -- start 2024-04-25T14:06:25.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.849+0000 7f3a84820640 1 -- start start 2024-04-25T14:06:25.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 0x7f3a8019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 0x7f3a801a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8010f660 con 0x7f3a80109780 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8010f4e0 con 0x7f3a80101e30 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a84820640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8010f7e0 con 0x7f3a80106c10 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 0x7f3a801a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60840/0 (socket says 172.21.15.94:60840) 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 0x7f3a801a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37728/0 (socket says 172.21.15.94:37728) 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/3664799780 learned_addr learned my addr 172.21.15.94:0/3664799780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 0x7f3a8019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/3664799780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 msgr2=0x7f3a8019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 0x7f3a8019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 msgr2=0x7f3a801a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 0x7f3a801a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/3664799780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801a1d70 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 0x7f3a8019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.850+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a74031440 tx=0x7f3a74003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.851+0000 7f3a6f7fe640 1 -- 172.21.15.94:0/3664799780 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7403f070 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.851+0000 7f3a6f7fe640 1 -- 172.21.15.94:0/3664799780 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a74031c50 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.851+0000 7f3a6f7fe640 1 -- 172.21.15.94:0/3664799780 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74035030 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.851+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801a2000 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.851+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801a24a0 con 0x7f3a80106c10 2024-04-25T14:06:25.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.854+0000 7f3a8b7ab740 1 Processor -- start 2024-04-25T14:06:25.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.854+0000 7f3a8b7ab740 1 -- start start 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf08a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c99be6bd80 con 0x55c99bb65390 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c99be25a20 con 0x55c99bb446b0 2024-04-25T14:06:25.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c99be67d40 con 0x55c99bb3c4d0 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37734/0 (socket says 172.21.15.94:37734) 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 -- 172.21.15.94:0/163504992 learned_addr learned my addr 172.21.15.94:0/163504992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a6f7fe640 1 -- 172.21.15.94:0/3664799780 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3a74042070 con 0x7f3a80106c10 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 -- 172.21.15.94:0/163504992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 msgr2=0x55c99bf07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 -- 172.21.15.94:0/163504992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 msgr2=0x55c99bf08a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf08a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.855+0000 7f3a7ed9d640 1 -- 172.21.15.94:0/163504992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99be30c30 con 0x55c99bb65390 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a780118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a660cc6ba0a97ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a6cff9640 1 -- 172.21.15.94:0/163504992 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78004d20 con 0x55c99bb65390 2024-04-25T14:06:25.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a6cff9640 1 -- 172.21.15.94:0/163504992 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a78004050 con 0x55c99bb65390 2024-04-25T14:06:25.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a6cff9640 1 -- 172.21.15.94:0/163504992 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78005260 con 0x55c99bb65390 2024-04-25T14:06:25.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a6f7fe640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 0x7f3a680b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.856+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 0x7f3a680b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a6f7fe640 1 -- 172.21.15.94:0/3664799780 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a740ff310 con 0x7f3a80106c10 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/163504992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 msgr2=0x55c99bf07810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a780118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 0x7f3a680b2080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55c99bf07530 tx=0x7f3a70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/163504992 shutdown_connections 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf08a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/163504992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf07810 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/163504992 >> 172.21.15.94:0/163504992 conn(0x55c99bb594e0 msgr2=0x55c99bf03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/163504992 shutdown_connections 2024-04-25T14:06:25.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.857+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/163504992 wait complete. 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 -- start start 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 Processor -- start 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf11b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf18af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c99bf09530 con 0x55c99bb65390 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c99bdf2dc0 con 0x55c99bb446b0 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c99bf096b0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf11650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60854/0 (socket says 172.21.15.94:60854) 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a7e59c640 1 -- 172.21.15.94:0/1491073941 learned_addr learned my addr 172.21.15.94:0/1491073941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a8b7ab740 10 client.0 _ll_register_callbacks cb 0x55c99be0e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf18af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.858+0000 7f3a7dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf11b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 msgr2=0x55c99bf11650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf11650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/1491073941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 msgr2=0x55c99bf18af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf18af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 -- 172.21.15.94:0/1491073941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99bb7d3f0 con 0x55c99bb446b0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf11b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a740c4a70 tx=0x7f3a74005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb969e0ecc6e2af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74049030 con 0x55c99bb446b0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a740c3a90 con 0x55c99bb446b0 2024-04-25T14:06:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.859+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a78004050 con 0x55c99bb446b0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a740ea0b0 con 0x55c99bb446b0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 10 client.33602 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c99bc477d0 con 0x55c99bb446b0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a78004050 con 0x55c99bb446b0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3a740b0110 con 0x55c99bb446b0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 10 client.33602 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 10 client.33602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 20 client.33602 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99bb23f60 0x55c99bf1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 4 client.33602 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.860+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c99be79990 con 0x55c99bb23f60 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.861+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99bb23f60 0x55c99bf1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.861+0000 7f3a65ffb640 10 client.33602 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.861+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99bb23f60 0x55c99bf1f370 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d85877464e9efc49 server_cookie=4c5d9f767bab10af in_seq=0 out_seq=0 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a65ffb640 1 -- 172.21.15.94:0/1491073941 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c99be79990 con 0x55c99bb23f60 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a65ffb640 10 client.33602 handle_command_reply: tid=0 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 client.33602 shutdown 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 msgr2=0x55c99bf11b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf11b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a740c4a70 tx=0x7f3a74005e50 comp rx=0 tx=0).stop 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 shutdown_connections 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99bb23f60 0x55c99bf1f370 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf11b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/1491073941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 >> 172.21.15.94:0/1491073941 conn(0x55c99bb594e0 msgr2=0x55c99bf02040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a65ffb640 0 client.33602 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 shutdown_connections 2024-04-25T14:06:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/1491073941 wait complete. 2024-04-25T14:06:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.862+0000 7f3a8b7ab740 20 client.33602 trim_cache size 0 max 16384 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 Processor -- start 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 -- start start 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb65390 0x55c99bf029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801a9310 con 0x55c99bb3c4d0 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801aae50 con 0x55c99bb446b0 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801a99e0 con 0x55c99bb65390 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37748/0 (socket says 172.21.15.94:37748) 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a7e59c640 1 -- 172.21.15.94:0/2206428763 learned_addr learned my addr 172.21.15.94:0/2206428763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.880+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb65390 0x55c99bf029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 -- 172.21.15.94:0/2206428763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb65390 msgr2=0x55c99bf029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb65390 0x55c99bf029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 -- 172.21.15.94:0/2206428763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 msgr2=0x55c99bf18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 -- 172.21.15.94:0/2206428763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99befe7f0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.881+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f3a70021ed0 tx=0x7f3a7001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d76dfc21916e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.882+0000 7f3a67fff640 1 -- 172.21.15.94:0/2206428763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700200e0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.882+0000 7f3a67fff640 1 -- 172.21.15.94:0/2206428763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a70020730 con 0x55c99bb3c4d0 2024-04-25T14:06:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.882+0000 7f3a67fff640 1 -- 172.21.15.94:0/2206428763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700319b0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/2206428763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 msgr2=0x55c99befe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f3a70021ed0 tx=0x7f3a7001b000 comp rx=0 tx=0).stop 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/2206428763 shutdown_connections 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb65390 0x55c99bf029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/2206428763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb3c4d0 0x55c99befe190 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/2206428763 >> 172.21.15.94:0/2206428763 conn(0x55c99bb594e0 msgr2=0x55c99bf077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/2206428763 shutdown_connections 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.883+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/2206428763 wait complete. 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 -- start start 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 Processor -- start 2024-04-25T14:06:25.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf15480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a7e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60864/0 (socket says 172.21.15.94:60864) 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a7e59c640 1 -- 172.21.15.94:0/606851995 learned_addr learned my addr 172.21.15.94:0/606851995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801aa780 con 0x55c99bb65390 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf15480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.884+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801aa0b0 con 0x55c99bb446b0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801a72f0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a8b7ab740 10 client.0 _ll_register_callbacks cb 0x7f3a74005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7ed9d640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7e59c640 1 -- 172.21.15.94:0/606851995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 msgr2=0x55c99bf15480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf15480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7e59c640 1 -- 172.21.15.94:0/606851995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 msgr2=0x55c99bf159e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:25.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf159e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.885+0000 7f3a7e59c640 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99bf856f0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a7e59c640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a70022fd0 tx=0x7f3a70031120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146e741a6d5a065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700200e0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3a70020280 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a70020730 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3a70020480 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a8b7ab740 10 client.33244 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.886+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c99bf868b0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a70020730 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3a700053b0 con 0x55c99bb3c4d0 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 10 client.33244 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 10 client.33244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 20 client.33244 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99baf93a0 0x55c99bf891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 4 client.33244 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.887+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c99be73800 con 0x55c99baf93a0 2024-04-25T14:06:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.889+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99baf93a0 0x55c99bf891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a7dd9b640 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99baf93a0 0x55c99bf891d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81ccab1940b990e9 server_cookie=a54a04b6a7d176fd in_seq=0 out_seq=0 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a65ffb640 10 client.33244 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a65ffb640 1 -- 172.21.15.94:0/606851995 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c99be73800 con 0x55c99baf93a0 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a65ffb640 10 client.33244 handle_command_reply: tid=0 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a8b7ab740 1 client.33244 shutdown 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 msgr2=0x55c99bf18900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.890+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a70022fd0 tx=0x7f3a70031120 comp rx=0 tx=0).stop 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 shutdown_connections 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55c99baf93a0 0x55c99bf891d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c99bb65390 0x55c99bf159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c99bb446b0 0x55c99bf15480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 --2- 172.21.15.94:0/606851995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c99bb3c4d0 0x55c99bf18900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 >> 172.21.15.94:0/606851995 conn(0x55c99bb594e0 msgr2=0x55c99bf0c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a65ffb640 0 client.33244 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 shutdown_connections 2024-04-25T14:06:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 1 -- 172.21.15.94:0/606851995 wait complete. 2024-04-25T14:06:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a8b7ab740 20 client.33244 trim_cache size 0 max 16384 2024-04-25T14:06:25.893 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 msgr2=0x7f3a680b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.891+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 0x7f3a680b2080 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55c99bf07530 tx=0x7f3a70005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 msgr2=0x7f3a8019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a74031440 tx=0x7f3a74003c10 comp rx=0 tx=0).stop 2024-04-25T14:06:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 shutdown_connections 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3a680afb70 0x7f3a680b2080 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a80109780 0x7f3a801a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a80106c10 0x7f3a8019a770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 --2- 172.21.15.94:0/3664799780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a80101e30 0x7f3a8019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 >> 172.21.15.94:0/3664799780 conn(0x7f3a800fd340 msgr2=0x7f3a801078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 shutdown_connections 2024-04-25T14:06:25.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:25.892+0000 7f3a84820640 1 -- 172.21.15.94:0/3664799780 wait complete. 2024-04-25T14:06:25.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:06:26.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 -- 172.21.15.94:0/2824751140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34103af0 msgr2=0x7fcb341047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 --2- 172.21.15.94:0/2824751140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34103af0 0x7fcb341047f0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009900 tx=0x7fcb2402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 -- 172.21.15.94:0/2824751140 shutdown_connections 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 --2- 172.21.15.94:0/2824751140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34109a30 0x7fcb3410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 --2- 172.21.15.94:0/2824751140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb34105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 --2- 172.21.15.94:0/2824751140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34103af0 0x7fcb341047f0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 -- 172.21.15.94:0/2824751140 >> 172.21.15.94:0/2824751140 conn(0x7fcb340fece0 msgr2=0x7fcb34101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 -- 172.21.15.94:0/2824751140 shutdown_connections 2024-04-25T14:06:26.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.050+0000 7fcb39070640 1 -- 172.21.15.94:0/2824751140 wait complete. 2024-04-25T14:06:26.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.051+0000 7fcb39070640 1 Processor -- start 2024-04-25T14:06:26.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 -- start start 2024-04-25T14:06:26.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb3419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 0x7fcb341a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410f5f0 con 0x7fcb34109a30 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410f470 con 0x7fcb34104d30 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb39070640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3410f770 con 0x7fcb34103af0 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb3419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 0x7fcb341a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb3419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55608/0 (socket says 172.21.15.94:55608) 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60878/0 (socket says 172.21.15.94:60878) 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3259c640 1 -- 172.21.15.94:0/3756543257 learned_addr learned my addr 172.21.15.94:0/3756543257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 -- 172.21.15.94:0/3756543257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 msgr2=0x7fcb3419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 -- 172.21.15.94:0/3756543257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 msgr2=0x7fcb3419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb3419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb3359e640 1 -- 172.21.15.94:0/3756543257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb341a1bd0 con 0x7fcb34109a30 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.052+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb3359e640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 0x7fcb341a1560 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009fd0 tx=0x7fcb2c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb1bfff640 1 -- 172.21.15.94:0/3756543257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00ce20 con 0x7fcb34109a30 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb341a1ec0 con 0x7fcb34109a30 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb341a2380 con 0x7fcb34109a30 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb1bfff640 1 -- 172.21.15.94:0/3756543257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb2c005020 con 0x7fcb34109a30 2024-04-25T14:06:26.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.053+0000 7fcb1bfff640 1 -- 172.21.15.94:0/3756543257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c010690 con 0x7fcb34109a30 2024-04-25T14:06:26.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.056+0000 7fcb3fff3740 1 Processor -- start 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb1bfff640 1 -- 172.21.15.94:0/3756543257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fcb2c004b20 con 0x7fcb34109a30 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 -- start start 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 0x556cba81a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba4576b0 0x556cba81ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556cba77ed80 con 0x556cba44f4d0 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556cba738a20 con 0x556cba4576b0 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3fff3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cba77ad40 con 0x556cba478390 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 0x556cba81a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba4576b0 0x556cba81ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37798/0 (socket says 172.21.15.94:37798) 2024-04-25T14:06:26.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.057+0000 7fcb32d9d640 1 -- 172.21.15.94:0/528994706 learned_addr learned my addr 172.21.15.94:0/528994706 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb1bfff640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 0x7fcb1c0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 -- 172.21.15.94:0/528994706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 msgr2=0x556cba81a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 0x556cba81a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 -- 172.21.15.94:0/528994706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba4576b0 msgr2=0x556cba81ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba4576b0 0x556cba81ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 -- 172.21.15.94:0/528994706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cba743c30 con 0x556cba44f4d0 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb3259c640 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 0x556cba81a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb1bfff640 1 -- 172.21.15.94:0/3756543257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcb2c0df1f0 con 0x7fcb34109a30 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb3359e640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 0x7fcb1c0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fcb341033c0 tx=0x7fcb24032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c4bb1b189e424d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb197fa640 1 -- 172.21.15.94:0/528994706 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24040070 con 0x556cba44f4d0 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb3359e640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 0x7fcb1c0b2100 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c01a040 tx=0x7fcb2c005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb197fa640 1 -- 172.21.15.94:0/528994706 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24005040 con 0x556cba44f4d0 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.058+0000 7fcb197fa640 1 -- 172.21.15.94:0/528994706 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24035040 con 0x556cba44f4d0 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 -- 172.21.15.94:0/528994706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 msgr2=0x556cba81ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7fcb341033c0 tx=0x7fcb24032f90 comp rx=0 tx=0).stop 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 -- 172.21.15.94:0/528994706 shutdown_connections 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba4576b0 0x556cba81ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba44f4d0 0x556cba81ad90 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/528994706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba478390 0x556cba81a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.059+0000 7fcb3fff3740 1 -- 172.21.15.94:0/528994706 >> 172.21.15.94:0/528994706 conn(0x556cba46c4e0 msgr2=0x556cba816020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- 172.21.15.94:0/528994706 shutdown_connections 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- 172.21.15.94:0/528994706 wait complete. 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- start start 2024-04-25T14:06:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 Processor -- start 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 0x556cba828900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba828e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba828e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba828e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37800/0 (socket says 172.21.15.94:37800) 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb32d9d640 1 -- 172.21.15.94:0/4294496810 learned_addr learned my addr 172.21.15.94:0/4294496810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3259c640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 0x556cba828900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556cba81c4d0 con 0x556cba4576b0 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556cba705dc0 con 0x556cba478390 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.060+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cba81c650 con 0x556cba44f4d0 2024-04-25T14:06:26.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3fff3740 10 client.0 _ll_register_callbacks cb 0x556cba721600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 -- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 msgr2=0x556cba82fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 -- 172.21.15.94:0/4294496810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 msgr2=0x556cba828e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba828e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cba4903f0 con 0x556cba44f4d0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3259c640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 0x556cba828900 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcb3419b130 tx=0x7fcb28009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58c23c2e0cf250f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28019040 con 0x556cba44f4d0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb24005040 con 0x556cba44f4d0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcb28013370 con 0x556cba44f4d0 2024-04-25T14:06:26.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb28005020 con 0x556cba44f4d0 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3fff3740 10 client.33256 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.061+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556cba55a7d0 con 0x556cba44f4d0 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb24005040 con 0x556cba44f4d0 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcb2801f040 con 0x556cba44f4d0 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 10 client.33256 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 10 client.33256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 20 client.33256 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba436f60 0x556cba836710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 4 client.33256 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:26.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cba78c990 con 0x556cba436f60 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.062+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba436f60 0x556cba836710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.063+0000 7fcb16ff5640 10 client.33256 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.063+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba436f60 0x556cba836710 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91d65842d7222d81 server_cookie=7cd00053044c5943 in_seq=0 out_seq=0 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.063+0000 7fcb16ff5640 1 -- 172.21.15.94:0/4294496810 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cba78c990 con 0x556cba436f60 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.063+0000 7fcb16ff5640 10 client.33256 handle_command_reply: tid=0 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 client.33256 shutdown 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 msgr2=0x556cba828900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 0x556cba828900 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcb3419b130 tx=0x7fcb28009c00 comp rx=0 tx=0).stop 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 shutdown_connections 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba436f60 0x556cba836710 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb16ff5640 0 client.33256 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba828e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/4294496810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba44f4d0 0x556cba828900 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcb3419b130 tx=0x7fcb28009c00 comp rx=0 tx=0).stop 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.064+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 >> 172.21.15.94:0/4294496810 conn(0x556cba46c4e0 msgr2=0x556cba818f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.066+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 shutdown_connections 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.066+0000 7fcb3fff3740 1 -- 172.21.15.94:0/4294496810 wait complete. 2024-04-25T14:06:26.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.066+0000 7fcb3fff3740 20 client.33256 trim_cache size 0 max 16384 2024-04-25T14:06:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.086+0000 7fcb3fff3740 1 Processor -- start 2024-04-25T14:06:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.086+0000 7fcb3fff3740 1 -- start start 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba4576b0 0x556cba83af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba478390 0x556cba83b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55648/0 (socket says 172.21.15.94:55648) 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3259c640 1 -- 172.21.15.94:0/1590748115 learned_addr learned my addr 172.21.15.94:0/1590748115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2000e700 con 0x556cba478390 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20010240 con 0x556cba436f60 2024-04-25T14:06:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.087+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2000edd0 con 0x556cba4576b0 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba478390 0x556cba83b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba4576b0 0x556cba83af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 -- 172.21.15.94:0/1590748115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba4576b0 msgr2=0x556cba83af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba4576b0 0x556cba83af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 -- 172.21.15.94:0/1590748115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba478390 msgr2=0x556cba83b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba478390 0x556cba83b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 -- 172.21.15.94:0/1590748115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cba81a3d0 con 0x556cba436f60 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb3259c640 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009d40 tx=0x7fcb28009c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8f237576882e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb18ff9640 1 -- 172.21.15.94:0/1590748115 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28013170 con 0x556cba436f60 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb18ff9640 1 -- 172.21.15.94:0/1590748115 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb28013310 con 0x556cba436f60 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.088+0000 7fcb18ff9640 1 -- 172.21.15.94:0/1590748115 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28010bf0 con 0x556cba436f60 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 msgr2=0x556cba828be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009d40 tx=0x7fcb28009c70 comp rx=0 tx=0).stop 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 shutdown_connections 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba478390 0x556cba83b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba4576b0 0x556cba83af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/1590748115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba436f60 0x556cba828be0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 >> 172.21.15.94:0/1590748115 conn(0x556cba46c4e0 msgr2=0x556cba8284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.090+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 shutdown_connections 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.091+0000 7fcb3fff3740 1 -- 172.21.15.94:0/1590748115 wait complete. 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.091+0000 7fcb3fff3740 1 -- start start 2024-04-25T14:06:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.091+0000 7fcb3fff3740 1 Processor -- start 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.092+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.092+0000 7fcb3fff3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba82b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.092+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.092+0000 7fcb3259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60912/0 (socket says 172.21.15.94:60912) 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.092+0000 7fcb3259c640 1 -- 172.21.15.94:0/976044479 learned_addr learned my addr 172.21.15.94:0/976044479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba82b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2000fb70 con 0x556cba4576b0 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2000f4a0 con 0x556cba478390 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2000c6e0 con 0x556cba436f60 2024-04-25T14:06:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3fff3740 10 client.0 _ll_register_callbacks cb 0x7fcb28013b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 -- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 msgr2=0x556cba82e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 -- 172.21.15.94:0/976044479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 msgr2=0x556cba82b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba82b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.093+0000 7fcb3259c640 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cba81fc80 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb3259c640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006010 tx=0x7fcb28009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f388f8237d3846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28013170 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fcb28010b10 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb28013310 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb28047050 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb3fff3740 10 client.33262 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.095+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556cba82fb20 con 0x556cba436f60 2024-04-25T14:06:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb28013310 con 0x556cba436f60 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcb28004040 con 0x556cba436f60 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 10 client.33262 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 10 client.33262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 20 client.33262 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba40c3a0 0x556cba8a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 4 client.33262 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cba786800 con 0x556cba40c3a0 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.096+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba40c3a0 0x556cba8a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb16ff5640 10 client.33262 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb32d9d640 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba40c3a0 0x556cba8a5420 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44d7889d31372198 server_cookie=77f754a23dcf61c6 in_seq=0 out_seq=0 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb16ff5640 1 -- 172.21.15.94:0/976044479 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cba786800 con 0x556cba40c3a0 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb16ff5640 10 client.33262 handle_command_reply: tid=0 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 client.33262 shutdown 2024-04-25T14:06:26.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 msgr2=0x556cba82b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006010 tx=0x7fcb28009c00 comp rx=0 tx=0).stop 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 shutdown_connections 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x556cba40c3a0 0x556cba8a5420 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb16ff5640 0 client.33262 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cba478390 0x556cba82e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556cba4576b0 0x556cba82b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 --2- 172.21.15.94:0/976044479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cba436f60 0x556cba82b050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 >> 172.21.15.94:0/976044479 conn(0x556cba46c4e0 msgr2=0x556cba818e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 shutdown_connections 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.097+0000 7fcb3fff3740 1 -- 172.21.15.94:0/976044479 wait complete. 2024-04-25T14:06:26.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.098+0000 7fcb3fff3740 20 client.33262 trim_cache size 0 max 16384 2024-04-25T14:06:26.101 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.098+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 msgr2=0x7fcb1c0b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.098+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 0x7fcb1c0b2100 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c01a040 tx=0x7fcb2c005f70 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.098+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 msgr2=0x7fcb341a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.098+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 0x7fcb341a1560 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009fd0 tx=0x7fcb2c00ef10 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 shutdown_connections 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fcb1c0afbf0 0x7fcb1c0b2100 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb34109a30 0x7fcb341a1560 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb34104d30 0x7fcb3419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 --2- 172.21.15.94:0/3756543257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb34103af0 0x7fcb3419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 >> 172.21.15.94:0/3756543257 conn(0x7fcb340fece0 msgr2=0x7fcb34101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 shutdown_connections 2024-04-25T14:06:26.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.099+0000 7fcb39070640 1 -- 172.21.15.94:0/3756543257 wait complete. 2024-04-25T14:06:26.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:06:26.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 235725 18.80 MB/sec execute 548 sec latency 223.915 ms 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 -- 172.21.15.94:0/1107503500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 msgr2=0x7fc1d010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/1107503500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d010bcc0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009ca0 tx=0x7fc1c802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 -- 172.21.15.94:0/1107503500 shutdown_connections 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/1107503500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d010bcc0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/1107503500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0106f40 0x7fc1d0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/1107503500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d01045e0 0x7fc1d0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 -- 172.21.15.94:0/1107503500 >> 172.21.15.94:0/1107503500 conn(0x7fc1d00fdc00 msgr2=0x7fc1d0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 -- 172.21.15.94:0/1107503500 shutdown_connections 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.251+0000 7fc1d83b2640 1 -- 172.21.15.94:0/1107503500 wait complete. 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 Processor -- start 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 -- start start 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d010f620 con 0x7fc1d01098a0 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d010f4a0 con 0x7fc1d0106f40 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.252+0000 7fc1d83b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d010f7a0 con 0x7fc1d01045e0 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55682/0 (socket says 172.21.15.94:55682) 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60932/0 (socket says 172.21.15.94:60932) 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d594d640 1 -- 172.21.15.94:0/3517510442 learned_addr learned my addr 172.21.15.94:0/3517510442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d694f640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 -- 172.21.15.94:0/3517510442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 msgr2=0x7fc1d019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 -- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 msgr2=0x7fc1d01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 -- 172.21.15.94:0/3517510442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d01a1d30 con 0x7fc1d01045e0 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d594d640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e9c0 tx=0x7fc1cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1c77fe640 1 -- 172.21.15.94:0/3517510442 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00ccc0 con 0x7fc1d01045e0 2024-04-25T14:06:26.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1c77fe640 1 -- 172.21.15.94:0/3517510442 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0108e0 con 0x7fc1d01045e0 2024-04-25T14:06:26.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.253+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d01a1fc0 con 0x7fc1d01045e0 2024-04-25T14:06:26.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.254+0000 7fc1c77fe640 1 -- 172.21.15.94:0/3517510442 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0187c0 con 0x7fc1d01045e0 2024-04-25T14:06:26.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.254+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d01a24d0 con 0x7fc1d01045e0 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.256+0000 7fc1df266740 1 Processor -- start 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.256+0000 7fc1c77fe640 1 -- 172.21.15.94:0/3517510442 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc1cc018960 con 0x7fc1d01045e0 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.256+0000 7fc1df266740 1 -- start start 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559772413d80 con 0x5597720e44d0 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5597723cda20 con 0x5597720ec6b0 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1df266740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55977240fd40 con 0x55977210d390 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60936/0 (socket says 172.21.15.94:60936) 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 -- 172.21.15.94:0/893539431 learned_addr learned my addr 172.21.15.94:0/893539431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d594d640 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d614e640 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 -- 172.21.15.94:0/893539431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 msgr2=0x5597724b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 -- 172.21.15.94:0/893539431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 msgr2=0x5597724b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1d694f640 1 -- 172.21.15.94:0/893539431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597723d8c30 con 0x55977210d390 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.257+0000 7fc1c77fe640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 0x7fc1bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1d694f640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 0x7fc1bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1c77fe640 1 -- 172.21.15.94:0/3517510442 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1cc0def40 con 0x7fc1d01045e0 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1d694f640 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc1d006c310 tx=0x7fc1c802f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c36fb378aa6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1c4bfe640 1 -- 172.21.15.94:0/893539431 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80316d0 con 0x55977210d390 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1c4bfe640 1 -- 172.21.15.94:0/893539431 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c8036030 con 0x55977210d390 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.258+0000 7fc1c4bfe640 1 -- 172.21.15.94:0/893539431 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c802f940 con 0x55977210d390 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- 172.21.15.94:0/893539431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 msgr2=0x5597724b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc1d006c310 tx=0x7fc1c802f410 comp rx=0 tx=0).stop 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1d694f640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 0x7fc1bc0b2080 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc1c804a000 tx=0x7fc1c804b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- 172.21.15.94:0/893539431 shutdown_connections 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 --2- 172.21.15.94:0/893539431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55977210d390 0x5597724b3c20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- 172.21.15.94:0/893539431 >> 172.21.15.94:0/893539431 conn(0x5597721014e0 msgr2=0x5597724ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- 172.21.15.94:0/893539431 shutdown_connections 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- 172.21.15.94:0/893539431 wait complete. 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.259+0000 7fc1df266740 1 -- start start 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 Processor -- start 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 0x5597724c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597724b57c0 con 0x55977210d390 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55977239adc0 con 0x5597720ec6b0 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5597724b5940 con 0x5597720e44d0 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1df266740 10 client.0 _ll_register_callbacks cb 0x5597723b6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724bd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60938/0 (socket says 172.21.15.94:60938) 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d614e640 1 -- 172.21.15.94:0/1533239498 learned_addr learned my addr 172.21.15.94:0/1533239498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 0x5597724c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 -- 172.21.15.94:0/1533239498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 msgr2=0x5597724bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724bd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 -- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 msgr2=0x5597724bdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724bdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.260+0000 7fc1d694f640 1 -- 172.21.15.94:0/1533239498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597721253f0 con 0x55977210d390 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1d594d640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1d694f640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 0x5597724c4c80 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006010 tx=0x7fc1c802f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd181c5a7f7d50d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8009360 con 0x55977210d390 2024-04-25T14:06:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1c802fe50 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c8036030 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1c8003f00 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1df266740 10 client.24264 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.261+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597721ef7d0 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c805e070 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc1c8030b90 con 0x55977210d390 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 10 client.24264 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 10 client.24264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 20 client.24264 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597720cbf60 0x5597724cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 4 client.24264 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc1c8036030 con 0x5597720cbf60 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.262+0000 7fc1d614e640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597720cbf60 0x5597724cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.263+0000 7fc1b9ffb640 10 client.24264 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.263+0000 7fc1d614e640 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597720cbf60 0x5597724cb0d0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae85e8f1f8ebec29 server_cookie=2b972173efceb78b in_seq=0 out_seq=0 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/1533239498 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc1c8036030 con 0x5597720cbf60 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1b9ffb640 10 client.24264 handle_command_reply: tid=0 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 client.24264 shutdown 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 msgr2=0x5597724c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 0x5597724c4c80 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006010 tx=0x7fc1c802f970 comp rx=0 tx=0).stop 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 shutdown_connections 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5597720cbf60 0x5597724cb0d0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55977210d390 0x5597724c4c80 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1b9ffb640 0 client.24264 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 --2- 172.21.15.94:0/1533239498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 >> 172.21.15.94:0/1533239498 conn(0x5597721014e0 msgr2=0x5597724adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 shutdown_connections 2024-04-25T14:06:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 1 -- 172.21.15.94:0/1533239498 wait complete. 2024-04-25T14:06:26.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.264+0000 7fc1df266740 20 client.24264 trim_cache size 0 max 16384 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.281+0000 7fc1df266740 1 Processor -- start 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.281+0000 7fc1df266740 1 -- start start 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 0x5597724bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01a9340 con 0x5597720e44d0 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01aae80 con 0x5597720cbf60 2024-04-25T14:06:26.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1df266740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01a9a10 con 0x5597720ec6b0 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37850/0 (socket says 172.21.15.94:37850) 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 -- 172.21.15.94:0/2861280983 learned_addr learned my addr 172.21.15.94:0/2861280983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55710/0 (socket says 172.21.15.94:55710) 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d694f640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 0x5597724bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 -- 172.21.15.94:0/2861280983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 msgr2=0x5597724bbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 0x5597724bbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 -- 172.21.15.94:0/2861280983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 msgr2=0x5597724ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d594d640 1 -- 172.21.15.94:0/2861280983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597724b36b0 con 0x5597720e44d0 2024-04-25T14:06:26.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d694f640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 0x5597724bbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.282+0000 7fc1d614e640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.283+0000 7fc1d594d640 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fc1d019b290 tx=0x7fc1c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c9594c16e8683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.283+0000 7fc1bbfff640 1 -- 172.21.15.94:0/2861280983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00120d0 con 0x5597720e44d0 2024-04-25T14:06:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.283+0000 7fc1bbfff640 1 -- 172.21.15.94:0/2861280983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c0012270 con 0x5597720e44d0 2024-04-25T14:06:26.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.283+0000 7fc1bbfff640 1 -- 172.21.15.94:0/2861280983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004a00 con 0x5597720e44d0 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- 172.21.15.94:0/2861280983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 msgr2=0x5597724c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fc1d019b290 tx=0x7fc1c0005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- 172.21.15.94:0/2861280983 shutdown_connections 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720ec6b0 0x5597724bbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720e44d0 0x5597724c4ba0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 --2- 172.21.15.94:0/2861280983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720cbf60 0x5597724ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- 172.21.15.94:0/2861280983 >> 172.21.15.94:0/2861280983 conn(0x5597721014e0 msgr2=0x5597724aaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- 172.21.15.94:0/2861280983 shutdown_connections 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- 172.21.15.94:0/2861280983 wait complete. 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 -- start start 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.284+0000 7fc1df266740 1 Processor -- start 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01aa7b0 con 0x5597720cbf60 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01aa0e0 con 0x5597720ec6b0 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01a7320 con 0x5597720e44d0 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1df266740 10 client.0 _ll_register_callbacks cb 0x7fc1c8049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d594d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37856/0 (socket says 172.21.15.94:37856) 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 -- 172.21.15.94:0/3733567497 learned_addr learned my addr 172.21.15.94:0/3733567497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d694f640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 -- 172.21.15.94:0/3733567497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 msgr2=0x5597724c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 -- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 msgr2=0x5597724c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 -- 172.21.15.94:0/3733567497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597724b90f0 con 0x5597720cbf60 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d694f640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d594d640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:26.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1d614e640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009560 tx=0x7fc1cc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee0ad8b25c35dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.285+0000 7fc1b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0dce60 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc1cc010a90 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0012270 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc0ab730 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 10 client.24270 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597724c4c30 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c0012270 con 0x5597720cbf60 2024-04-25T14:06:26.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc1c80315d0 con 0x5597720cbf60 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 10 client.24270 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 10 client.24270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 20 client.24270 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55977210d390 0x5597725357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 4 client.24270 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559772421990 con 0x55977210d390 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.286+0000 7fc1d594d640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55977210d390 0x5597725357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.287+0000 7fc1d594d640 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55977210d390 0x5597725357e0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ee0a9a0e3c9d3c server_cookie=6928de30cf84bfa1 in_seq=0 out_seq=0 2024-04-25T14:06:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.287+0000 7fc1b9ffb640 10 client.24270 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1b9ffb640 1 -- 172.21.15.94:0/3733567497 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559772421990 con 0x55977210d390 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1b9ffb640 10 client.24270 handle_command_reply: tid=0 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 client.24270 shutdown 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 msgr2=0x5597724c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009560 tx=0x7fc1cc00ce60 comp rx=0 tx=0).stop 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 shutdown_connections 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55977210d390 0x5597725357e0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597720ec6b0 0x5597724c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1b9ffb640 0 client.24270 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597720e44d0 0x5597724c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 --2- 172.21.15.94:0/3733567497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597720cbf60 0x5597724c2080 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 >> 172.21.15.94:0/3733567497 conn(0x5597721014e0 msgr2=0x5597724ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 shutdown_connections 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 1 -- 172.21.15.94:0/3733567497 wait complete. 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1df266740 20 client.24270 trim_cache size 0 max 16384 2024-04-25T14:06:26.297 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 msgr2=0x7fc1bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 0x7fc1bc0b2080 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc1c804a000 tx=0x7fc1c804b000 comp rx=0 tx=0).stop 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 msgr2=0x7fc1d019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e9c0 tx=0x7fc1cc00ee80 comp rx=0 tx=0).stop 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 shutdown_connections 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc1bc0afb70 0x7fc1bc0b2080 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1d01098a0 0x7fc1d01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1d0106f40 0x7fc1d019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 --2- 172.21.15.94:0/3517510442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d01045e0 0x7fc1d019a1f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:26.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.291+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 >> 172.21.15.94:0/3517510442 conn(0x7fc1d00fdc00 msgr2=0x7fc1d01050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:26.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.292+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 shutdown_connections 2024-04-25T14:06:26.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:26.292+0000 7fc1d83b2640 1 -- 172.21.15.94:0/3517510442 wait complete. 2024-04-25T14:06:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:26 smithi094 ceph-mon[29195]: pgmap v1090: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 486 op/s 2024-04-25T14:06:26.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:26 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2602809152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:26 smithi137 ceph-mon[32568]: pgmap v1090: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 486 op/s 2024-04-25T14:06:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:26 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2602809152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:26 smithi138 ceph-mon[32977]: pgmap v1090: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 486 op/s 2024-04-25T14:06:26.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2602809152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:27.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 236174 18.81 MB/sec execute 549 sec latency 73.237 ms 2024-04-25T14:06:27.303 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:06:27.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:27.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:06:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 -- 172.21.15.94:0/2895193761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc109780 msgr2=0x7f71dc10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2895193761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc109780 0x7f71dc10bc90 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009ca0 tx=0x7f71d402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 -- 172.21.15.94:0/2895193761 shutdown_connections 2024-04-25T14:06:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2895193761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc109780 0x7f71dc10bc90 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2895193761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc106c50 0x7f71dc1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2895193761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 -- 172.21.15.94:0/2895193761 >> 172.21.15.94:0/2895193761 conn(0x7f71dc0fdc00 msgr2=0x7f71dc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 -- 172.21.15.94:0/2895193761 shutdown_connections 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.430+0000 7f71e47ce640 1 -- 172.21.15.94:0/2895193761 wait complete. 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 Processor -- start 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 -- start start 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc109780 0x7f71dc1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10f610 con 0x7f71dc106c50 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10f490 con 0x7f71dc104260 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e47ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10f790 con 0x7f71dc109780 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37876/0 (socket says 172.21.15.94:37876) 2024-04-25T14:06:27.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 -- 172.21.15.94:0/2284542695 learned_addr learned my addr 172.21.15.94:0/2284542695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e256a640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc109780 0x7f71dc1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 -- 172.21.15.94:0/2284542695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc109780 msgr2=0x7f71dc1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc109780 0x7f71dc1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 -- 172.21.15.94:0/2284542695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 msgr2=0x7f71dc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e1d69640 1 -- 172.21.15.94:0/2284542695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1a1d00 con 0x7f71dc106c50 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.431+0000 7f71e256a640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71e1d69640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00e9f0 tx=0x7f71cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71d37fe640 1 -- 172.21.15.94:0/2284542695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00cd80 con 0x7f71dc106c50 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71d37fe640 1 -- 172.21.15.94:0/2284542695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71cc0054d0 con 0x7f71dc106c50 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a1ff0 con 0x7f71dc106c50 2024-04-25T14:06:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71d37fe640 1 -- 172.21.15.94:0/2284542695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc010690 con 0x7f71dc106c50 2024-04-25T14:06:27.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.432+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a2450 con 0x7f71dc106c50 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.434+0000 7f71d37fe640 1 -- 172.21.15.94:0/2284542695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f71cc004e90 con 0x7f71dc106c50 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 Processor -- start 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 -- start start 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 0x55a7108c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a71082eb00 con 0x55a710507730 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7107ca920 con 0x55a7104ff550 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71eb67e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a71082aac0 con 0x55a710528410 2024-04-25T14:06:27.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e2d6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 0x55a7108c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71d37fe640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 0x7f71c80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e256a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e2d6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 0x55a7108c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60986/0 (socket says 172.21.15.94:60986) 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e2d6b640 1 -- 172.21.15.94:0/3480692989 learned_addr learned my addr 172.21.15.94:0/3480692989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.435+0000 7f71e256a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55758/0 (socket says 172.21.15.94:55758) 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 0x7f71c80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 -- 172.21.15.94:0/3480692989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 msgr2=0x55a7108c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 0x55a7108c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 -- 172.21.15.94:0/3480692989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 msgr2=0x55a7108d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 -- 172.21.15.94:0/3480692989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7106087e0 con 0x55a7104ff550 2024-04-25T14:06:27.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71d37fe640 1 -- 172.21.15.94:0/2284542695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f71cc0df050 con 0x7f71dc106c50 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e256a640 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009fa0 tx=0x7f71d80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55be3bb9218e4d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 0x7f71c80b2080 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55a7108ca670 tx=0x7f71d402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71d0ff9640 1 -- 172.21.15.94:0/3480692989 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004030 con 0x55a7104ff550 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71d0ff9640 1 -- 172.21.15.94:0/3480692989 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d80054d0 con 0x55a7104ff550 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.436+0000 7f71d0ff9640 1 -- 172.21.15.94:0/3480692989 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004c40 con 0x55a7104ff550 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- 172.21.15.94:0/3480692989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 msgr2=0x55a7108c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009fa0 tx=0x7f71d80118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- 172.21.15.94:0/3480692989 shutdown_connections 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108c9a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3480692989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710528410 0x55a7108c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- 172.21.15.94:0/3480692989 >> 172.21.15.94:0/3480692989 conn(0x55a71051c560 msgr2=0x55a7108c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- 172.21.15.94:0/3480692989 shutdown_connections 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- 172.21.15.94:0/3480692989 wait complete. 2024-04-25T14:06:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 -- start start 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.437+0000 7f71eb67e740 1 Processor -- start 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104ff550 0x55a7108d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 0x55a7108da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7108d1370 con 0x55a710507730 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7107c85d0 con 0x55a710528410 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7108d14f0 con 0x55a7104ff550 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71eb67e740 10 client.0 _ll_register_callbacks cb 0x55a7107b6080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37892/0 (socket says 172.21.15.94:37892) 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e2d6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 0x55a7108da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 -- 172.21.15.94:0/174501124 learned_addr learned my addr 172.21.15.94:0/174501124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 -- 172.21.15.94:0/174501124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104ff550 msgr2=0x55a7108d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104ff550 0x55a7108d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 -- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 msgr2=0x55a7108da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 0x55a7108da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 -- 172.21.15.94:0/174501124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a710668f70 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 0x55a7108da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71e1d69640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0dcab0 tx=0x7f71cc0a7260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8885ee867fc81df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.438+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc005180 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f71cc0e12f0 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d80054d0 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71cc0a6fa0 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 10 client.24282 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a710500170 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d80054d0 con 0x55a710507730 2024-04-25T14:06:27.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f71cc0afe60 con 0x55a710507730 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 10 client.24282 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 10 client.24282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 20 client.24282 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7104e71f0 0x55a7108e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 4 client.24282 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a71083c710 con 0x55a7104e71f0 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.439+0000 7f71e256a640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7104e71f0 0x55a7108e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.440+0000 7f71c5ffb640 10 client.24282 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.440+0000 7f71e256a640 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7104e71f0 0x55a7108e0cd0 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee53c3abc15b18ac server_cookie=2960d9588537cad3 in_seq=0 out_seq=0 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71c5ffb640 1 -- 172.21.15.94:0/174501124 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a71083c710 con 0x55a7104e71f0 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71c5ffb640 10 client.24282 handle_command_reply: tid=0 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 client.24282 shutdown 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 msgr2=0x55a7108d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0dcab0 tx=0x7f71cc0a7260 comp rx=0 tx=0).stop 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 shutdown_connections 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a7104e71f0 0x55a7108e0cd0 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a710528410 0x55a7108da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71c5ffb640 0 client.24282 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d3870 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 --2- 172.21.15.94:0/174501124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104ff550 0x55a7108d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 >> 172.21.15.94:0/174501124 conn(0x55a71051c560 msgr2=0x55a7108c3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 shutdown_connections 2024-04-25T14:06:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 1 -- 172.21.15.94:0/174501124 wait complete. 2024-04-25T14:06:27.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.442+0000 7f71eb67e740 20 client.24282 trim_cache size 0 max 16384 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 Processor -- start 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 -- start start 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ab510 con 0x55a710507730 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ad050 con 0x55a7104ff550 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.463+0000 7f71eb67e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1abbe0 con 0x55a7104e71f0 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e2d6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60998/0 (socket says 172.21.15.94:60998) 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 -- 172.21.15.94:0/3102987043 learned_addr learned my addr 172.21.15.94:0/3102987043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37894/0 (socket says 172.21.15.94:37894) 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e1d69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55770/0 (socket says 172.21.15.94:55770) 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 -- 172.21.15.94:0/3102987043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 msgr2=0x55a7108d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 -- 172.21.15.94:0/3102987043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 msgr2=0x55a7108d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 -- 172.21.15.94:0/3102987043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7108ce150 con 0x55a7104e71f0 2024-04-25T14:06:27.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71e256a640 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f71d80057c0 tx=0x7f71d8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6ab259f01e5a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.464+0000 7f71c7fff640 1 -- 172.21.15.94:0/3102987043 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d801d040 con 0x55a7104e71f0 2024-04-25T14:06:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71c7fff640 1 -- 172.21.15.94:0/3102987043 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d800fcb0 con 0x55a7104e71f0 2024-04-25T14:06:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71c7fff640 1 -- 172.21.15.94:0/3102987043 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8029940 con 0x55a7104e71f0 2024-04-25T14:06:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 -- 172.21.15.94:0/3102987043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 msgr2=0x55a7108c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f71d80057c0 tx=0x7f71d8009ed0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 -- 172.21.15.94:0/3102987043 shutdown_connections 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a710507730 0x55a7108d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 --2- 172.21.15.94:0/3102987043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7104e71f0 0x55a7108c4d30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.465+0000 7f71eb67e740 1 -- 172.21.15.94:0/3102987043 >> 172.21.15.94:0/3102987043 conn(0x55a71051c560 msgr2=0x55a7108c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 -- 172.21.15.94:0/3102987043 shutdown_connections 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 -- 172.21.15.94:0/3102987043 wait complete. 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 -- start start 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 Processor -- start 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71e256a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710507730 0x55a7108d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.466+0000 7f71e256a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37900/0 (socket says 172.21.15.94:37900) 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 -- 172.21.15.94:0/1360124628 learned_addr learned my addr 172.21.15.94:0/1360124628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ac980 con 0x55a7104e71f0 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ac2b0 con 0x55a7104ff550 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1a94f0 con 0x55a710507730 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 10 client.0 _ll_register_callbacks cb 0x7f71cc004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e1d69640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e2d6b640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710507730 0x55a7108d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 -- 172.21.15.94:0/1360124628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710507730 msgr2=0x55a7108d8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710507730 0x55a7108d8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 -- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 msgr2=0x55a7108d79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7108d8ed0 con 0x55a7104e71f0 2024-04-25T14:06:27.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71e256a640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fb0 tx=0x7f71d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b33f82715025fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8037020 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f71d8029100 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.467+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d800fcb0 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71d8042050 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 10 client.24288 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a710949040 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d800fcb0 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f71d80296d0 con 0x55a7104e71f0 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 10 client.24288 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 10 client.24288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 20 client.24288 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a710528410 0x55a71094b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 4 client.24288 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:06:27.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.468+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a710836580 con 0x55a710528410 2024-04-25T14:06:27.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.469+0000 7f71e1d69640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a710528410 0x55a71094b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.469+0000 7f71c5ffb640 10 client.24288 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.469+0000 7f71e1d69640 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a710528410 0x55a71094b7b0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35ef101eec875c2 server_cookie=b4ba5fcb1ce75840 in_seq=0 out_seq=0 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.478+0000 7f71c5ffb640 1 -- 172.21.15.94:0/1360124628 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a710836580 con 0x55a710528410 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.478+0000 7f71c5ffb640 10 client.24288 handle_command_reply: tid=0 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.478+0000 7f71eb67e740 1 client.24288 shutdown 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 msgr2=0x55a7108d74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fb0 tx=0x7f71d8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 shutdown_connections 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a710528410 0x55a71094b7b0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a710507730 0x55a7108d8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7104ff550 0x55a7108d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 --2- 172.21.15.94:0/1360124628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a7104e71f0 0x55a7108d74a0 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71c5ffb640 0 client.24288 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 >> 172.21.15.94:0/1360124628 conn(0x55a71051c560 msgr2=0x55a7108c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 shutdown_connections 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 1 -- 172.21.15.94:0/1360124628 wait complete. 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71eb67e740 20 client.24288 trim_cache size 0 max 16384 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:06:27.480 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "99f25c2d-5945-481b-8341-9fa0aa0bbbfe", 2024-04-25T14:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:06:27.481 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:06:27.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 msgr2=0x7f71c80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 0x7f71c80b2080 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55a7108ca670 tx=0x7f71d402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 msgr2=0x7f71dc19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00e9f0 tx=0x7f71cc00edf0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 shutdown_connections 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f71c80afb70 0x7f71c80b2080 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc109780 0x7f71dc1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71dc106c50 0x7f71dc19a700 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 --2- 172.21.15.94:0/2284542695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71dc104260 0x7f71dc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.479+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 >> 172.21.15.94:0/2284542695 conn(0x7f71dc0fdc00 msgr2=0x7f71dc1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.480+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 shutdown_connections 2024-04-25T14:06:27.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.480+0000 7f71e47ce640 1 -- 172.21.15.94:0/2284542695 wait complete. 2024-04-25T14:06:27.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:27.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:27.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/1946299673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 msgr2=0x7f3c081054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/1946299673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c081054c0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009900 tx=0x7f3bf802a410 comp rx=0 tx=0).stop 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/1946299673 shutdown_connections 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/1946299673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c0810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/1946299673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 0x7f3c08101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/1946299673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c081054c0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/1946299673 >> 172.21.15.94:0/1946299673 conn(0x7f3c080fd380 msgr2=0x7f3c080ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/1946299673 shutdown_connections 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.611+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/1946299673 wait complete. 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 Processor -- start 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 -- start start 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 0x7f3c08104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c08102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f600 con 0x7f3c081050b0 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f480 con 0x7f3c08101760 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c0ddb2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810f780 con 0x7f3c08102260 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37918/0 (socket says 172.21.15.94:37918) 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c07fff640 1 -- 172.21.15.94:0/4234023611 learned_addr learned my addr 172.21.15.94:0/4234023611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c08102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.612+0000 7f3c077fe640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 0x7f3c08104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 -- 172.21.15.94:0/4234023611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 msgr2=0x7f3c08102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c08102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 -- 172.21.15.94:0/4234023611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 msgr2=0x7f3c08104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 0x7f3c08104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 -- 172.21.15.94:0/4234023611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c081034e0 con 0x7f3c081050b0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c08102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c07fff640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000eb10 tx=0x7f3c0000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c04ff9640 1 -- 172.21.15.94:0/4234023611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000ce20 con 0x7f3c081050b0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c04ff9640 1 -- 172.21.15.94:0/4234023611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00005020 con 0x7f3c081050b0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a4350 con 0x7f3c081050b0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c04ff9640 1 -- 172.21.15.94:0/4234023611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00010690 con 0x7f3c081050b0 2024-04-25T14:06:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.613+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a4760 con 0x7f3c081050b0 2024-04-25T14:06:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.615+0000 7f3c14c66740 1 Processor -- start 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.615+0000 7f3c04ff9640 1 -- 172.21.15.94:0/4234023611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f3c00005510 con 0x7f3c081050b0 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.615+0000 7f3c14c66740 1 -- start start 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038a6390 0x562003c47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562003bacd80 con 0x5620038856b0 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562003b668b0 con 0x56200387d4d0 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c14c66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562003ba8d40 con 0x5620038a6390 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37928/0 (socket says 172.21.15.94:37928) 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 -- 172.21.15.94:0/449083040 learned_addr learned my addr 172.21.15.94:0/449083040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038a6390 0x562003c47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c04ff9640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 0x7f3bf00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 -- 172.21.15.94:0/449083040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038a6390 msgr2=0x562003c47a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038a6390 0x562003c47a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 -- 172.21.15.94:0/449083040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 msgr2=0x562003c47fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c47fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 -- 172.21.15.94:0/449083040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562003b37a40 con 0x5620038856b0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c07fff640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 0x7f3bf00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00a5c0 tx=0x7f3bfc00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894ac458f21a70b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3c04ff9640 1 -- 172.21.15.94:0/4234023611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3c000df2c0 con 0x7f3c081050b0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.616+0000 7f3bedffb640 1 -- 172.21.15.94:0/449083040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00fd50 con 0x5620038856b0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3bedffb640 1 -- 172.21.15.94:0/449083040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bfc01b7f0 con 0x5620038856b0 2024-04-25T14:06:27.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3bedffb640 1 -- 172.21.15.94:0/449083040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00d630 con 0x5620038856b0 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 -- 172.21.15.94:0/449083040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 msgr2=0x562003c48c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c07fff640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 0x7f3bf00b2000 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c760 tx=0x7f3c0006db70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00a5c0 tx=0x7f3bfc00ac00 comp rx=0 tx=0).stop 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 -- 172.21.15.94:0/449083040 shutdown_connections 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038856b0 0x562003c48c70 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c47fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 --2- 172.21.15.94:0/449083040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038a6390 0x562003c47a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.617+0000 7f3c14c66740 1 -- 172.21.15.94:0/449083040 >> 172.21.15.94:0/449083040 conn(0x56200389a4e0 msgr2=0x562003c43270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- 172.21.15.94:0/449083040 shutdown_connections 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- 172.21.15.94:0/449083040 wait complete. 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- start start 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 Processor -- start 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56200387d4d0 0x562003c518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038856b0 0x562003c51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562003c49780 con 0x5620038a6390 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562003b33d10 con 0x5620038856b0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562003c49900 con 0x56200387d4d0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c14c66740 10 client.0 _ll_register_callbacks cb 0x562003b60210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.618+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37936/0 (socket says 172.21.15.94:37936) 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 -- 172.21.15.94:0/3479598576 learned_addr learned my addr 172.21.15.94:0/3479598576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56200387d4d0 0x562003c518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c077fe640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038856b0 0x562003c51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 -- 172.21.15.94:0/3479598576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56200387d4d0 msgr2=0x562003c518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56200387d4d0 0x562003c518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 -- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038856b0 msgr2=0x562003c51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038856b0 0x562003c51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 -- 172.21.15.94:0/3479598576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620039887d0 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc001eb0 tx=0x7f3bfc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dddcff723956ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc01bbb0 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc01b7f0 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3bfc01d440 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bfc033050 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c14c66740 10 client.24300 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5620038be3f0 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.619+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bfc01b7f0 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3bfc01b020 con 0x5620038a6390 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 10 client.24300 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 10 client.24300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 20 client.24300 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562003864f60 0x562003c5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 4 client.24300 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.620+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562003bba990 con 0x562003864f60 2024-04-25T14:06:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.853+0000 7f3c077fe640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562003864f60 0x562003c5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.854+0000 7f3c077fe640 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562003864f60 0x562003c5eed0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf21eb823b54fe88 server_cookie=7409d5741313cecd in_seq=0 out_seq=0 2024-04-25T14:06:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.854+0000 7f3be6ffd640 10 client.24300 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.973+0000 7f3be6ffd640 1 -- 172.21.15.94:0/3479598576 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562003bba990 con 0x562003864f60 2024-04-25T14:06:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3be6ffd640 10 client.24300 handle_command_reply: tid=0 2024-04-25T14:06:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 client.24300 shutdown 2024-04-25T14:06:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 msgr2=0x562003c58d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc001eb0 tx=0x7f3bfc00a2d0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 shutdown_connections 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562003864f60 0x562003c5eed0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003c58d70 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038856b0 0x562003c51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3be6ffd640 0 client.24300 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 --2- 172.21.15.94:0/3479598576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56200387d4d0 0x562003c518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 >> 172.21.15.94:0/3479598576 conn(0x56200389a4e0 msgr2=0x562003c46180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 shutdown_connections 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 1 -- 172.21.15.94:0/3479598576 wait complete. 2024-04-25T14:06:27.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.974+0000 7f3c14c66740 20 client.24300 trim_cache size 0 max 16384 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.993+0000 7f3c14c66740 1 Processor -- start 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.993+0000 7f3c14c66740 1 -- start start 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038a6390 0x562003c4c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ad780 con 0x56200387d4d0 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081af2c0 con 0x5620038a6390 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c14c66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ade50 con 0x5620038856b0 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37938/0 (socket says 172.21.15.94:37938) 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1276092507 learned_addr learned my addr 172.21.15.94:0/1276092507 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038a6390 0x562003c4c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c077fe640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1276092507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 msgr2=0x562003c433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1276092507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038a6390 msgr2=0x562003c4c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038a6390 0x562003c4c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 -- 172.21.15.94:0/1276092507 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200387e0f0 con 0x56200387d4d0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c077fe640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.994+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d380 tx=0x7f3bfc00a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59da487aac5eecc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3bed7fa640 1 -- 172.21.15.94:0/1276092507 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc03b030 con 0x56200387d4d0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3bed7fa640 1 -- 172.21.15.94:0/1276092507 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bfc00fb50 con 0x56200387d4d0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3bed7fa640 1 -- 172.21.15.94:0/1276092507 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00ab10 con 0x56200387d4d0 2024-04-25T14:06:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 -- 172.21.15.94:0/1276092507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 msgr2=0x562003c48050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00d380 tx=0x7f3bfc00a550 comp rx=0 tx=0).stop 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 -- 172.21.15.94:0/1276092507 shutdown_connections 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620038a6390 0x562003c4c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.995+0000 7f3c14c66740 1 --2- 172.21.15.94:0/1276092507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56200387d4d0 0x562003c48050 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- 172.21.15.94:0/1276092507 >> 172.21.15.94:0/1276092507 conn(0x56200389a4e0 msgr2=0x562003c42ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- 172.21.15.94:0/1276092507 shutdown_connections 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- 172.21.15.94:0/1276092507 wait complete. 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- start start 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 Processor -- start 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c56b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003cc5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081aebf0 con 0x5620038a6390 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ae520 con 0x56200387d4d0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.996+0000 7f3c14c66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c081ab760 con 0x5620038856b0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c14c66740 10 client.0 _ll_register_callbacks cb 0x7f3c000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c56b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c06ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55834/0 (socket says 172.21.15.94:55834) 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c06ffd640 1 -- 172.21.15.94:0/2230070508 learned_addr learned my addr 172.21.15.94:0/2230070508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003cc5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 -- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 msgr2=0x562003c565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 -- 172.21.15.94:0/2230070508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 msgr2=0x562003cc5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:28.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003cc5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 -- 172.21.15.94:0/2230070508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562003b6d4b0 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c077fe640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c56b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3bf802a930 tx=0x7f3bf80042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b957a9b16f47867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8035040 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3bf8005240 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.997+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc00fb50 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bf8005440 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 10 client.33310 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562003cc6810 con 0x5620038856b0 2024-04-25T14:06:28.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bfc00fb50 con 0x5620038856b0 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf8004850 con 0x5620038856b0 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 10 client.33310 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 10 client.33310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 20 client.33310 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56200383a3a0 0x562003cc9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 4 client.33310 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562003bb4800 con 0x56200383a3a0 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.998+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56200383a3a0 0x562003cc9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.999+0000 7f3be6ffd640 10 client.33310 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:28.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:27.999+0000 7f3c06ffd640 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56200383a3a0 0x562003cc9200 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8e6fa7ae5314c9 server_cookie=9724f24898ab8909 in_seq=0 out_seq=0 2024-04-25T14:06:28.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3be6ffd640 1 -- 172.21.15.94:0/2230070508 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562003bb4800 con 0x56200383a3a0 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3be6ffd640 10 client.33310 handle_command_reply: tid=0 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 client.33310 shutdown 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 msgr2=0x562003c56b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c56b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3bf802a930 tx=0x7f3bf80042b0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 shutdown_connections 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x56200383a3a0 0x562003cc9200 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5620038a6390 0x562003cc5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3be6ffd640 0 client.33310 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5620038856b0 0x562003c56b00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 --2- 172.21.15.94:0/2230070508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56200387d4d0 0x562003c565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 >> 172.21.15.94:0/2230070508 conn(0x56200389a4e0 msgr2=0x562003c46070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 shutdown_connections 2024-04-25T14:06:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 1 -- 172.21.15.94:0/2230070508 wait complete. 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.004+0000 7f3c14c66740 20 client.33310 trim_cache size 0 max 16384 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (216 inodes in the stack)", 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "99f25c2d-5945-481b-8341-9fa0aa0bbbfe": { 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "99f25c2d-5945-481b-8341-9fa0aa0bbbfe", 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:06:28.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:06:28.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:06:28.008 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:06:28.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 msgr2=0x7f3bf00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 0x7f3bf00b2000 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c760 tx=0x7f3c0006db70 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 msgr2=0x7f3c08102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000eb10 tx=0x7f3c0000ef10 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 shutdown_connections 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f3bf00afaf0 0x7f3bf00b2000 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c081050b0 0x7f3c08102e40 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c08102260 0x7f3c08102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 --2- 172.21.15.94:0/4234023611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c08101760 0x7f3c08104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 >> 172.21.15.94:0/4234023611 conn(0x7f3c080fd380 msgr2=0x7f3c08105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 shutdown_connections 2024-04-25T14:06:28.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:28.005+0000 7f3c0ddb2640 1 -- 172.21.15.94:0/4234023611 wait complete. 2024-04-25T14:06:28.016 INFO:tasks.cephfs.filesystem:scrub status for tag:99f25c2d-5945-481b-8341-9fa0aa0bbbfe - {'path': '/', 'tag': '99f25c2d-5945-481b-8341-9fa0aa0bbbfe', 'options': 'recursive,force'} 2024-04-25T14:06:28.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 236351 18.80 MB/sec execute 550 sec latency 424.960 ms 2024-04-25T14:06:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: pgmap v1091: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 517 op/s 2024-04-25T14:06:28.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:06:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:28.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:28 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: pgmap v1091: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 517 op/s 2024-04-25T14:06:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:06:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:28.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:28 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: pgmap v1091: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.1 MiB/s wr, 517 op/s 2024-04-25T14:06:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:06:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:28.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:28.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:28 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 236644 18.80 MB/sec execute 551 sec latency 100.815 ms 2024-04-25T14:06:29.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:06:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.644 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:29 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:29 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:29.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:29 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:30.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 237061 18.81 MB/sec execute 552 sec latency 58.375 ms 2024-04-25T14:06:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:30 smithi137 ceph-mon[32568]: pgmap v1092: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-04-25T14:06:30.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:30 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:30 smithi094 ceph-mon[29195]: pgmap v1092: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-04-25T14:06:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:30 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:30 smithi138 ceph-mon[32977]: pgmap v1092: 97 pgs: 97 active+clean; 275 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-04-25T14:06:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:30 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:31.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 237260 18.79 MB/sec execute 553 sec latency 275.246 ms 2024-04-25T14:06:32.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 237612 18.78 MB/sec execute 554 sec latency 152.379 ms 2024-04-25T14:06:32.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:32 smithi137 ceph-mon[32568]: pgmap v1093: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 578 op/s 2024-04-25T14:06:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:32 smithi094 ceph-mon[29195]: pgmap v1093: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 578 op/s 2024-04-25T14:06:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:32 smithi138 ceph-mon[32977]: pgmap v1093: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 578 op/s 2024-04-25T14:06:33.162 INFO:tasks.workunit.client.0.smithi094.stdout: 10 238092 18.79 MB/sec execute 555 sec latency 84.981 ms 2024-04-25T14:06:34.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 238553 18.80 MB/sec execute 556 sec latency 72.189 ms 2024-04-25T14:06:34.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:34 smithi137 ceph-mon[32568]: pgmap v1094: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-04-25T14:06:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:34 smithi094 ceph-mon[29195]: pgmap v1094: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-04-25T14:06:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:34 smithi138 ceph-mon[32977]: pgmap v1094: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 -- 172.21.15.94:0/2237328832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4102a20 msgr2=0x7f0eb4103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 --2- 172.21.15.94:0/2237328832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4102a20 0x7f0eb4103720 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009900 tx=0x7f0eb002a410 comp rx=0 tx=0).stop 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 -- 172.21.15.94:0/2237328832 shutdown_connections 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 --2- 172.21.15.94:0/2237328832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 0x7f0eb410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 --2- 172.21.15.94:0/2237328832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4103c60 0x7f0eb4104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 --2- 172.21.15.94:0/2237328832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4102a20 0x7f0eb4103720 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.006+0000 7f0ebb896640 1 -- 172.21.15.94:0/2237328832 >> 172.21.15.94:0/2237328832 conn(0x7f0eb40fdc10 msgr2=0x7f0eb41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 -- 172.21.15.94:0/2237328832 shutdown_connections 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 -- 172.21.15.94:0/2237328832 wait complete. 2024-04-25T14:06:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 Processor -- start 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 -- start start 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4102a20 0x7f0eb406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.007+0000 7f0ebb896640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 0x7f0eb41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb410e550 con 0x7f0eb4103c60 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb410e3d0 con 0x7f0eb4108960 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb410e6d0 con 0x7f0eb4102a20 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40624/0 (socket says 172.21.15.94:40624) 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 -- 172.21.15.94:0/3490772331 learned_addr learned my addr 172.21.15.94:0/3490772331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb9632640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4102a20 0x7f0eb406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 0x7f0eb41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 -- 172.21.15.94:0/3490772331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4102a20 msgr2=0x7f0eb406ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4102a20 0x7f0eb406ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 -- 172.21.15.94:0/3490772331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 msgr2=0x7f0eb41a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 0x7f0eb41a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 -- 172.21.15.94:0/3490772331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb41a1d80 con 0x7f0eb4103c60 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400e9f0 tx=0x7f0ea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eaa7fc640 1 -- 172.21.15.94:0/3490772331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400cd80 con 0x7f0eb4103c60 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb41a2070 con 0x7f0eb4103c60 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eaa7fc640 1 -- 172.21.15.94:0/3490772331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea40108e0 con 0x7f0eb4103c60 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0eaa7fc640 1 -- 172.21.15.94:0/3490772331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4018980 con 0x7f0eb4103c60 2024-04-25T14:06:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.008+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a2610 con 0x7f0eb4103c60 2024-04-25T14:06:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 Processor -- start 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 -- start start 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597122722f0 0x55971263bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712642df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597125a1460 con 0x55971229b1b0 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55971255b590 con 0x55971227a4d0 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.011+0000 7f0ec2825740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55971259d420 con 0x5597122722f0 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40634/0 (socket says 172.21.15.94:40634) 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 -- 172.21.15.94:0/4256204609 learned_addr learned my addr 172.21.15.94:0/4256204609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9632640 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597122722f0 0x55971263bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 -- 172.21.15.94:0/4256204609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597122722f0 msgr2=0x55971263bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eaa7fc640 1 -- 172.21.15.94:0/3490772331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0ea4018b20 con 0x7f0eb4103c60 2024-04-25T14:06:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597122722f0 0x55971263bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 -- 172.21.15.94:0/4256204609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 msgr2=0x559712642df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712642df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 -- 172.21.15.94:0/4256204609 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597125680d0 con 0x55971229b1b0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fd0 tx=0x7f0eac0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902a7a4ae8de1a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0e9f7fe640 1 -- 172.21.15.94:0/4256204609 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004d20 con 0x55971229b1b0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0e9f7fe640 1 -- 172.21.15.94:0/4256204609 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac004050 con 0x55971229b1b0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.012+0000 7f0e9f7fe640 1 -- 172.21.15.94:0/4256204609 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac005260 con 0x55971229b1b0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0eaa7fc640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 0x7f0ea00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0eb9632640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 0x7f0ea00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0eaa7fc640 1 -- 172.21.15.94:0/3490772331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ea40deff0 con 0x7f0eb4103c60 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0ec2825740 1 -- 172.21.15.94:0/4256204609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 msgr2=0x55971263b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0eb9632640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 0x7f0ea00b1ff0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002aa00 tx=0x7f0eb00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.013+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fd0 tx=0x7f0eac0118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 -- 172.21.15.94:0/4256204609 shutdown_connections 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712642df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597122722f0 0x55971263bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4256204609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971229b1b0 0x55971263b970 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 -- 172.21.15.94:0/4256204609 >> 172.21.15.94:0/4256204609 conn(0x55971228f300 msgr2=0x559712637300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 -- 172.21.15.94:0/4256204609 shutdown_connections 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 -- 172.21.15.94:0/4256204609 wait complete. 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 -- start start 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 Processor -- start 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597122722f0 0x559712645730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.014+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x55971264cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0ec2825740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559712643840 con 0x55971227a4d0 2024-04-25T14:06:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0ec2825740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559712528b60 con 0x5597122722f0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0ec2825740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5597126439c0 con 0x55971229b1b0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0ec2825740 10 client.0 _ll_register_callbacks cb 0x55971254d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb9e33640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x55971264cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40640/0 (socket says 172.21.15.94:40640) 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 -- 172.21.15.94:0/4008466875 learned_addr learned my addr 172.21.15.94:0/4008466875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb9e33640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x55971264cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:59982/0 (socket says 172.21.15.94:59982) 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb9632640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597122722f0 0x559712645730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 -- 172.21.15.94:0/4008466875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 msgr2=0x55971264cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x55971264cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 -- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597122722f0 msgr2=0x559712645730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597122722f0 0x559712645730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 -- 172.21.15.94:0/4008466875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597122b3210 con 0x55971227a4d0 2024-04-25T14:06:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40a71e0 tx=0x7f0ea40a6d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6f95e5f1f9cb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0e9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40ab7e0 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0ea4010430 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.015+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eac004050 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ea40b0290 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 10 client.24318 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55971237f620 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eac004050 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0ea40a61b0 con 0x55971227a4d0 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 10 client.24318 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 10 client.24318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 20 client.24318 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559712259d80 0x559712653410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 4 client.24318 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597125af070 con 0x559712259d80 2024-04-25T14:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.016+0000 7f0eb9632640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559712259d80 0x559712653410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.017+0000 7f0e9cff9640 10 client.24318 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.017+0000 7f0eb9632640 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559712259d80 0x559712653410 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695e642fc28ef88e server_cookie=17a50b114a5f8177 in_seq=0 out_seq=0 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0e9cff9640 1 -- 172.21.15.94:0/4008466875 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5597125af070 con 0x559712259d80 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0e9cff9640 10 client.24318 handle_command_reply: tid=0 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 client.24318 shutdown 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 msgr2=0x559712645c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40a71e0 tx=0x7f0ea40a6d10 comp rx=0 tx=0).stop 2024-04-25T14:06:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 shutdown_connections 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x559712259d80 0x559712653410 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x55971264cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55971227a4d0 0x559712645c70 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0e9cff9640 0 client.24318 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 --2- 172.21.15.94:0/4008466875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597122722f0 0x559712645730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 >> 172.21.15.94:0/4008466875 conn(0x55971228f300 msgr2=0x559712636250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 shutdown_connections 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 1 -- 172.21.15.94:0/4008466875 wait complete. 2024-04-25T14:06:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.021+0000 7f0ec2825740 20 client.24318 trim_cache size 0 max 16384 2024-04-25T14:06:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 Processor -- start 2024-04-25T14:06:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 -- start start 2024-04-25T14:06:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712637190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x559712643d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ab550 con 0x5597122722f0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ad090 con 0x55971227a4d0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0ec2825740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41abc20 con 0x55971229b1b0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0eb9632640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0eb9632640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40650/0 (socket says 172.21.15.94:40650) 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.037+0000 7f0eb9632640 1 -- 172.21.15.94:0/2988226580 learned_addr learned my addr 172.21.15.94:0/2988226580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712637190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x559712643d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 -- 172.21.15.94:0/2988226580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 msgr2=0x559712643d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x559712643d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 -- 172.21.15.94:0/2988226580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 msgr2=0x559712637190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712637190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 -- 172.21.15.94:0/2988226580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971263c600 con 0x5597122722f0 2024-04-25T14:06:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0eb9632640 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0031440 tx=0x7f0eb0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4f257a6511f65b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0e9effd640 1 -- 172.21.15.94:0/2988226580 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035040 con 0x5597122722f0 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0e9effd640 1 -- 172.21.15.94:0/2988226580 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb0031b90 con 0x5597122722f0 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.038+0000 7f0e9effd640 1 -- 172.21.15.94:0/2988226580 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb003d6b0 con 0x5597122722f0 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- 172.21.15.94:0/2988226580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 msgr2=0x559712636c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0031440 tx=0x7f0eb0040000 comp rx=0 tx=0).stop 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- 172.21.15.94:0/2988226580 shutdown_connections 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971229b1b0 0x559712643d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971227a4d0 0x559712637190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 --2- 172.21.15.94:0/2988226580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712636c50 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- 172.21.15.94:0/2988226580 >> 172.21.15.94:0/2988226580 conn(0x55971228f300 msgr2=0x55971263fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- 172.21.15.94:0/2988226580 shutdown_connections 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- 172.21.15.94:0/2988226580 wait complete. 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 -- start start 2024-04-25T14:06:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.039+0000 7f0ec2825740 1 Processor -- start 2024-04-25T14:06:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971227a4d0 0x559712649ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971229b1b0 0x55971264abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ac9c0 con 0x5597122722f0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ac2f0 con 0x55971229b1b0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41a9530 con 0x55971227a4d0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0ec2825740 10 client.0 _ll_register_callbacks cb 0x7f0ea400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40666/0 (socket says 172.21.15.94:40666) 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 -- 172.21.15.94:0/1514831468 learned_addr learned my addr 172.21.15.94:0/1514831468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971227a4d0 0x559712649ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9e33640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971229b1b0 0x55971264abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 -- 172.21.15.94:0/1514831468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971227a4d0 msgr2=0x559712649ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971227a4d0 0x559712649ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 -- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971229b1b0 msgr2=0x55971264abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971229b1b0 0x55971264abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 -- 172.21.15.94:0/1514831468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559712643b00 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0eb9632640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a930 tx=0x7f0eb0004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec5ed57edae7ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.040+0000 7f0e9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035040 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f0eb0003a00 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0031b90 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0eb003d450 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 10 client.24324 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55971264c250 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb0031b90 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0eb0064070 con 0x5597122722f0 2024-04-25T14:06:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 10 client.24324 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 10 client.24324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 20 client.24324 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55971222f1c0 0x5597126bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 4 client.24324 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597125a8ee0 con 0x55971222f1c0 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.041+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55971222f1c0 0x5597126bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.042+0000 7f0e9cff9640 10 client.24324 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.042+0000 7f0eb8e31640 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55971222f1c0 0x5597126bdbe0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e124fc51046049 server_cookie=2a123071f80d0568 in_seq=0 out_seq=0 2024-04-25T14:06:35.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 238966 18.80 MB/sec execute 557 sec latency 91.262 ms 2024-04-25T14:06:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0e9cff9640 1 -- 172.21.15.94:0/1514831468 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597125a8ee0 con 0x55971222f1c0 2024-04-25T14:06:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0e9cff9640 10 client.24324 handle_command_reply: tid=0 2024-04-25T14:06:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0ec2825740 1 client.24324 shutdown 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 msgr2=0x559712649760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a930 tx=0x7f0eb0004070 comp rx=0 tx=0).stop 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 shutdown_connections 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.845+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55971222f1c0 0x5597126bdbe0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0e9cff9640 0 client.24324 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55971229b1b0 0x55971264abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55971227a4d0 0x559712649ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 --2- 172.21.15.94:0/1514831468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597122722f0 0x559712649760 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 >> 172.21.15.94:0/1514831468 conn(0x55971228f300 msgr2=0x559712636c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 shutdown_connections 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 1 -- 172.21.15.94:0/1514831468 wait complete. 2024-04-25T14:06:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ec2825740 20 client.24324 trim_cache size 0 max 16384 2024-04-25T14:06:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:06:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:06:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:06:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:06:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 msgr2=0x7f0ea00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 0x7f0ea00b1ff0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002aa00 tx=0x7f0eb00314a0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 msgr2=0x7f0eb419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.846+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400e9f0 tx=0x7f0ea400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 shutdown_connections 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f0ea00afae0 0x7f0ea00b1ff0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108960 0x7f0eb41a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4103c60 0x7f0eb419f010 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 --2- 172.21.15.94:0/3490772331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4102a20 0x7f0eb406ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 >> 172.21.15.94:0/3490772331 conn(0x7f0eb40fdc10 msgr2=0x7f0eb4104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 shutdown_connections 2024-04-25T14:06:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:35.847+0000 7f0ebb896640 1 -- 172.21.15.94:0/3490772331 wait complete. 2024-04-25T14:06:36.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 239067 18.79 MB/sec execute 558 sec latency 671.641 ms 2024-04-25T14:06:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:36 smithi137 ceph-mon[32568]: pgmap v1095: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-04-25T14:06:36.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:36.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:36 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:36 smithi094 ceph-mon[29195]: pgmap v1095: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-04-25T14:06:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:36 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:36 smithi138 ceph-mon[32977]: pgmap v1095: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 574 op/s 2024-04-25T14:06:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:36 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:37.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 239420 18.80 MB/sec execute 559 sec latency 83.464 ms 2024-04-25T14:06:37.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:37.643 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:37 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:06:37.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:37.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:37.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:37 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:06:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:37.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:37 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:06:38.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 239813 18.80 MB/sec execute 560 sec latency 79.954 ms 2024-04-25T14:06:38.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:38 smithi137 ceph-mon[32568]: pgmap v1096: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-04-25T14:06:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:38 smithi094 ceph-mon[29195]: pgmap v1096: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-04-25T14:06:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:38 smithi138 ceph-mon[32977]: pgmap v1096: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-04-25T14:06:39.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 240050 18.79 MB/sec execute 561 sec latency 100.296 ms 2024-04-25T14:06:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:39.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:06:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:39.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:06:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:39.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:06:40.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 240401 18.79 MB/sec execute 562 sec latency 66.946 ms 2024-04-25T14:06:40.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:40 smithi137 ceph-mon[32568]: pgmap v1097: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 395 op/s 2024-04-25T14:06:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:40 smithi094 ceph-mon[29195]: pgmap v1097: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 395 op/s 2024-04-25T14:06:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:40 smithi138 ceph-mon[32977]: pgmap v1097: 97 pgs: 97 active+clean; 283 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 395 op/s 2024-04-25T14:06:41.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 240714 18.79 MB/sec execute 563 sec latency 192.784 ms 2024-04-25T14:06:42.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 241071 18.77 MB/sec execute 564 sec latency 92.767 ms 2024-04-25T14:06:42.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:42 smithi137 ceph-mon[32568]: pgmap v1098: 97 pgs: 97 active+clean; 284 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-25T14:06:42.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:42 smithi094 ceph-mon[29195]: pgmap v1098: 97 pgs: 97 active+clean; 284 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-25T14:06:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:42 smithi138 ceph-mon[32977]: pgmap v1098: 97 pgs: 97 active+clean; 284 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-25T14:06:43.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 241488 18.78 MB/sec execute 565 sec latency 76.371 ms 2024-04-25T14:06:44.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 241843 18.79 MB/sec execute 566 sec latency 167.164 ms 2024-04-25T14:06:44.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:44 smithi137 ceph-mon[32568]: pgmap v1099: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:44 smithi094 ceph-mon[29195]: pgmap v1099: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:44.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:44 smithi138 ceph-mon[32977]: pgmap v1099: 97 pgs: 97 active+clean; 280 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 482 op/s 2024-04-25T14:06:45.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 242305 18.79 MB/sec execute 567 sec latency 97.875 ms 2024-04-25T14:06:46.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 242538 18.78 MB/sec execute 568 sec latency 375.974 ms 2024-04-25T14:06:46.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:46 smithi137 ceph-mon[32568]: pgmap v1100: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-04-25T14:06:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:46 smithi094 ceph-mon[29195]: pgmap v1100: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-04-25T14:06:46.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:46 smithi138 ceph-mon[32977]: pgmap v1100: 97 pgs: 97 active+clean; 285 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-04-25T14:06:47.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 242891 18.80 MB/sec execute 569 sec latency 235.093 ms 2024-04-25T14:06:48.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 243291 18.80 MB/sec execute 570 sec latency 67.863 ms 2024-04-25T14:06:48.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:48 smithi137 ceph-mon[32568]: pgmap v1101: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-04-25T14:06:48.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:48 smithi094 ceph-mon[29195]: pgmap v1101: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-04-25T14:06:48.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:48 smithi138 ceph-mon[32977]: pgmap v1101: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 511 op/s 2024-04-25T14:06:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.961+0000 7f1068af9640 1 -- 172.21.15.94:0/4049412440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 msgr2=0x7f106410ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.961+0000 7f1068af9640 1 --2- 172.21.15.94:0/4049412440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f106410ae80 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f105c009ca0 tx=0x7f105c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 -- 172.21.15.94:0/4049412440 shutdown_connections 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 --2- 172.21.15.94:0/4049412440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f106410ae80 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 --2- 172.21.15.94:0/4049412440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064104410 0x7f1064102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 --2- 172.21.15.94:0/4049412440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1064103f30 0x7f1064102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 -- 172.21.15.94:0/4049412440 >> 172.21.15.94:0/4049412440 conn(0x7f10640fd390 msgr2=0x7f10640ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 -- 172.21.15.94:0/4049412440 shutdown_connections 2024-04-25T14:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 -- 172.21.15.94:0/4049412440 wait complete. 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 Processor -- start 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.962+0000 7f1068af9640 1 -- start start 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106419e160 0x7f10641a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f106410e550 con 0x7f1064108a60 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f106410e3d0 con 0x7f1064103f30 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1068af9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f106410e6d0 con 0x7f106419e160 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f106259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44482/0 (socket says 172.21.15.94:44482) 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f106259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57796/0 (socket says 172.21.15.94:57796) 2024-04-25T14:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 -- 172.21.15.94:0/1966160894 learned_addr learned my addr 172.21.15.94:0/1966160894 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1062d9d640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106419e160 0x7f10641a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 -- 172.21.15.94:0/1966160894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106419e160 msgr2=0x7f10641a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106419e160 0x7f10641a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 -- 172.21.15.94:0/1966160894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 msgr2=0x7f1064198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 -- 172.21.15.94:0/1966160894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10641a0bf0 con 0x7f1064108a60 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f106259c640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.963+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f105800ea60 tx=0x7f105800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.964+0000 7f10537fe640 1 -- 172.21.15.94:0/1966160894 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105800cde0 con 0x7f1064108a60 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.964+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10641a0ee0 con 0x7f1064108a60 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.964+0000 7f10537fe640 1 -- 172.21.15.94:0/1966160894 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10580035f0 con 0x7f1064108a60 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.964+0000 7f10537fe640 1 -- 172.21.15.94:0/1966160894 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058010690 con 0x7f1064108a60 2024-04-25T14:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.964+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a13a0 con 0x7f1064108a60 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.966+0000 7f10537fe640 1 -- 172.21.15.94:0/1966160894 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f10580037a0 con 0x7f1064108a60 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 Processor -- start 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 -- start start 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f10537fe640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 0x7f104c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 0x5645b0ddd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0dddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 0x5645b0de7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5645b0d43460 con 0x5645b0a1c4d0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5645b0cfd590 con 0x5645b0a142f0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f106f9ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645b0d3f420 con 0x5645b0a3d1b0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f10537fe640 1 -- 172.21.15.94:0/1966160894 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f10580defb0 con 0x7f1064108a60 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f1062d9d640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 0x7f104c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 0x5645b0de7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 0x5645b0de7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44498/0 (socket says 172.21.15.94:44498) 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.967+0000 7f1062d9d640 1 -- 172.21.15.94:0/116960500 learned_addr learned my addr 172.21.15.94:0/116960500 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 0x5645b0ddd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f1061d9b640 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0dddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 -- 172.21.15.94:0/116960500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 msgr2=0x5645b0dddef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0dddef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 -- 172.21.15.94:0/116960500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 msgr2=0x5645b0de7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 0x5645b0de7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 -- 172.21.15.94:0/116960500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0d0a0d0 con 0x5645b0a3d1b0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f106259c640 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 0x5645b0ddd970 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f106406a1e0 tx=0x7f10540118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dec2db601abc624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f1062d9d640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 0x7f104c0b20a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f105c009ca0 tx=0x7f105c02f480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f1050ff9640 1 -- 172.21.15.94:0/116960500 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105401e070 con 0x5645b0a3d1b0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f1050ff9640 1 -- 172.21.15.94:0/116960500 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10540035f0 con 0x5645b0a3d1b0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.968+0000 7f1050ff9640 1 -- 172.21.15.94:0/116960500 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105400f760 con 0x5645b0a3d1b0 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.969+0000 7f106f9ac740 1 -- 172.21.15.94:0/116960500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 msgr2=0x5645b0ddd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.969+0000 7f106f9ac740 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 0x5645b0ddd970 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f106406a1e0 tx=0x7f10540118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 -- 172.21.15.94:0/116960500 shutdown_connections 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a1c4d0 0x5645b0de7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0dddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 --2- 172.21.15.94:0/116960500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a3d1b0 0x5645b0ddd970 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 -- 172.21.15.94:0/116960500 >> 172.21.15.94:0/116960500 conn(0x5645b0a31300 msgr2=0x5645b0dd9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 -- 172.21.15.94:0/116960500 shutdown_connections 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.970+0000 7f106f9ac740 1 -- 172.21.15.94:0/116960500 wait complete. 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 -- start start 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 Processor -- start 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0dec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5645b0de30e0 con 0x5645b0a3d1b0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5645b0ccab60 con 0x5645b0a142f0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645b0de3260 con 0x5645b0a1c4d0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106f9ac740 10 client.0 _ll_register_callbacks cb 0x5645b0cef2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f106259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:57818/0 (socket says 172.21.15.94:57818) 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44508/0 (socket says 172.21.15.94:44508) 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.971+0000 7f1062d9d640 1 -- 172.21.15.94:0/1140285607 learned_addr learned my addr 172.21.15.94:0/1140285607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0dec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 -- 172.21.15.94:0/1140285607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 msgr2=0x5645b0dec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0dec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 -- 172.21.15.94:0/1140285607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 msgr2=0x5645b0df2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 -- 172.21.15.94:0/1140285607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0a55210 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f1062d9d640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106259c640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5645b0dda090 tx=0x7f1054004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be60cee611de929a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f1049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105401e070 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1054005240 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.972+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10540035f0 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1054005440 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 10 client.33701 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645b0b21620 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10540035f0 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f105400f5d0 con 0x5645b0a142f0 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 10 client.33701 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 10 client.33701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 20 client.33701 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b09fbd80 0x5645b0df92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 4 client.33701 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b0d51070 con 0x5645b09fbd80 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.973+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b09fbd80 0x5645b0df92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.974+0000 7f1049ffb640 10 client.33701 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.974+0000 7f1061d9b640 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b09fbd80 0x5645b0df92c0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ed425bb58e0ff7 server_cookie=c3f7589c37daca98 in_seq=0 out_seq=0 2024-04-25T14:06:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f1049ffb640 1 -- 172.21.15.94:0/1140285607 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645b0d51070 con 0x5645b09fbd80 2024-04-25T14:06:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f1049ffb640 10 client.33701 handle_command_reply: tid=0 2024-04-25T14:06:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 client.33701 shutdown 2024-04-25T14:06:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 msgr2=0x5645b0debdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5645b0dda090 tx=0x7f1054004490 comp rx=0 tx=0).stop 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 shutdown_connections 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b09fbd80 0x5645b0df92c0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a3d1b0 0x5645b0df2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0dec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f1049ffb640 0 client.33701 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 --2- 172.21.15.94:0/1140285607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b0a142f0 0x5645b0debdf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 >> 172.21.15.94:0/1140285607 conn(0x5645b0a31300 msgr2=0x5645b0dd8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 shutdown_connections 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 1 -- 172.21.15.94:0/1140285607 wait complete. 2024-04-25T14:06:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.979+0000 7f106f9ac740 20 client.33701 trim_cache size 0 max 16384 2024-04-25T14:06:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.997+0000 7f106f9ac740 1 Processor -- start 2024-04-25T14:06:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.997+0000 7f106f9ac740 1 -- start start 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.997+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0dd93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0debb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106f9ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a93c0 con 0x5645b0a142f0 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106f9ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10641aaf00 con 0x5645b09fbd80 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106f9ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a9a90 con 0x5645b0a1c4d0 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44520/0 (socket says 172.21.15.94:44520) 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 -- 172.21.15.94:0/3636908365 learned_addr learned my addr 172.21.15.94:0/3636908365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106259c640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0dd93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1062d9d640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0debb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 -- 172.21.15.94:0/3636908365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 msgr2=0x5645b0debb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0debb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 -- 172.21.15.94:0/3636908365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 msgr2=0x5645b0dd93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0dd93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 -- 172.21.15.94:0/3636908365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0de6eb0 con 0x5645b0a142f0 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f106259c640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0dd93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f1061d9b640 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f105800d010 tx=0x7f10580aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cce770ffab142b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.998+0000 7f104bfff640 1 -- 172.21.15.94:0/3636908365 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580ab740 con 0x5645b0a142f0 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f104bfff640 1 -- 172.21.15.94:0/3636908365 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10580b01f0 con 0x5645b0a142f0 2024-04-25T14:06:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f104bfff640 1 -- 172.21.15.94:0/3636908365 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580a65f0 con 0x5645b0a142f0 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 -- 172.21.15.94:0/3636908365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 msgr2=0x5645b0df2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f105800d010 tx=0x7f10580aa040 comp rx=0 tx=0).stop 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 -- 172.21.15.94:0/3636908365 shutdown_connections 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0debb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2ed0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3636908365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0dd93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:48.999+0000 7f106f9ac740 1 -- 172.21.15.94:0/3636908365 >> 172.21.15.94:0/3636908365 conn(0x5645b0a31300 msgr2=0x5645b0dd8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- 172.21.15.94:0/3636908365 shutdown_connections 2024-04-25T14:06:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- 172.21.15.94:0/3636908365 wait complete. 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- start start 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 Processor -- start 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0df1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10641aa830 con 0x5645b0a142f0 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10641aa160 con 0x5645b09fbd80 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.000+0000 7f106f9ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10641a73a0 con 0x5645b0a1c4d0 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f106f9ac740 10 client.0 _ll_register_callbacks cb 0x7f105401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1061d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47314/0 (socket says 172.21.15.94:47314) 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 -- 172.21.15.94:0/3426431874 learned_addr learned my addr 172.21.15.94:0/3426431874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f106259c640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0df1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 -- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 msgr2=0x5645b0df1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0df1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 -- 172.21.15.94:0/3426431874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 msgr2=0x5645b0df2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 -- 172.21.15.94:0/3426431874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b0e5fb50 con 0x5645b0a1c4d0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1062d9d640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f105c003dc0 tx=0x7f105c049000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c168db0c906a0dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.001+0000 7f1049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c051040 con 0x5645b0a1c4d0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f105c004540 con 0x5645b0a1c4d0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10580b01f0 con 0x5645b0a1c4d0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f105c004740 con 0x5645b0a1c4d0 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 10 client.33346 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645b0e60d20 con 0x5645b0a1c4d0 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10580b01f0 con 0x5645b0a1c4d0 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f105c054070 con 0x5645b0a1c4d0 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 10 client.33346 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 10 client.33346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 20 client.33346 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b0a3d1b0 0x5645b0e637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 4 client.33346 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.002+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645b0d4aee0 con 0x5645b0a3d1b0 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.003+0000 7f106259c640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b0a3d1b0 0x5645b0e637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.003+0000 7f1049ffb640 10 client.33346 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.003+0000 7f106259c640 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b0a3d1b0 0x5645b0e637c0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d503b760f9ed1007 server_cookie=1b6003338b64a51e in_seq=0 out_seq=0 2024-04-25T14:06:49.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 243563 18.79 MB/sec execute 571 sec latency 122.942 ms 2024-04-25T14:06:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f1049ffb640 1 -- 172.21.15.94:0/3426431874 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645b0d4aee0 con 0x5645b0a3d1b0 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f1049ffb640 10 client.33346 handle_command_reply: tid=0 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 client.33346 shutdown 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 msgr2=0x5645b0e5f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f105c003dc0 tx=0x7f105c049000 comp rx=0 tx=0).stop 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 shutdown_connections 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x5645b0a3d1b0 0x5645b0e637c0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645b0a1c4d0 0x5645b0e5f610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f1049ffb640 0 client.33346 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645b0a142f0 0x5645b0df2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 --2- 172.21.15.94:0/3426431874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5645b09fbd80 0x5645b0df1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 >> 172.21.15.94:0/3426431874 conn(0x5645b0a31300 msgr2=0x5645b0debb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 shutdown_connections 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 1 -- 172.21.15.94:0/3426431874 wait complete. 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.716+0000 7f106f9ac740 20 client.33346 trim_cache size 0 max 16384 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:06:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 msgr2=0x7f104c0b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 0x7f104c0b20a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f105c009ca0 tx=0x7f105c02f480 comp rx=0 tx=0).stop 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 msgr2=0x7f10641991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f105800ea60 tx=0x7f105800ef20 comp rx=0 tx=0).stop 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 shutdown_connections 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f104c0afb90 0x7f104c0b20a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106419e160 0x7f10641a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064108a60 0x7f10641991f0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 --2- 172.21.15.94:0/1966160894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1064103f30 0x7f1064198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 >> 172.21.15.94:0/1966160894 conn(0x7f10640fd390 msgr2=0x7f10640ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 shutdown_connections 2024-04-25T14:06:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:06:49.717+0000 7f1068af9640 1 -- 172.21.15.94:0/1966160894 wait complete. 2024-04-25T14:06:50.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 243726 18.78 MB/sec execute 572 sec latency 406.270 ms 2024-04-25T14:06:50.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:50 smithi137 ceph-mon[32568]: pgmap v1102: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 449 op/s 2024-04-25T14:06:50.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:50 smithi094 ceph-mon[29195]: pgmap v1102: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 449 op/s 2024-04-25T14:06:50.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:50 smithi138 ceph-mon[32977]: pgmap v1102: 97 pgs: 97 active+clean; 288 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 449 op/s 2024-04-25T14:06:51.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 244020 18.77 MB/sec execute 573 sec latency 185.798 ms 2024-04-25T14:06:52.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 244388 18.77 MB/sec execute 574 sec latency 172.066 ms 2024-04-25T14:06:52.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:52 smithi137 ceph-mon[32568]: pgmap v1103: 97 pgs: 97 active+clean; 290 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-04-25T14:06:52.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:52 smithi094 ceph-mon[29195]: pgmap v1103: 97 pgs: 97 active+clean; 290 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-04-25T14:06:52.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:52 smithi138 ceph-mon[32977]: pgmap v1103: 97 pgs: 97 active+clean; 290 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-04-25T14:06:53.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 244677 18.75 MB/sec execute 575 sec latency 154.572 ms 2024-04-25T14:06:54.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 245110 18.77 MB/sec execute 576 sec latency 57.174 ms 2024-04-25T14:06:54.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:54 smithi137 ceph-mon[32568]: pgmap v1104: 97 pgs: 97 active+clean; 287 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 376 op/s 2024-04-25T14:06:54.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:54 smithi094 ceph-mon[29195]: pgmap v1104: 97 pgs: 97 active+clean; 287 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 376 op/s 2024-04-25T14:06:54.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:54 smithi138 ceph-mon[32977]: pgmap v1104: 97 pgs: 97 active+clean; 287 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 376 op/s 2024-04-25T14:06:55.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 245572 18.78 MB/sec execute 577 sec latency 66.644 ms 2024-04-25T14:06:56.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 245920 18.77 MB/sec execute 578 sec latency 162.364 ms 2024-04-25T14:06:56.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:56 smithi137 ceph-mon[32568]: pgmap v1105: 97 pgs: 97 active+clean; 281 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-25T14:06:56.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:56 smithi094 ceph-mon[29195]: pgmap v1105: 97 pgs: 97 active+clean; 281 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-25T14:06:56.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:56 smithi138 ceph-mon[32977]: pgmap v1105: 97 pgs: 97 active+clean; 281 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-25T14:06:57.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 246189 18.77 MB/sec execute 579 sec latency 255.286 ms 2024-04-25T14:06:58.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:58.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 -- 172.21.15.94:0/2712921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 msgr2=0x7f126c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 --2- 172.21.15.94:0/2712921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c1036f0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f125c009900 tx=0x7f125c02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 -- 172.21.15.94:0/2712921240 shutdown_connections 2024-04-25T14:06:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 --2- 172.21.15.94:0/2712921240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c108930 0x7f126c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 --2- 172.21.15.94:0/2712921240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c103c30 0x7f126c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 --2- 172.21.15.94:0/2712921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c1036f0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 -- 172.21.15.94:0/2712921240 >> 172.21.15.94:0/2712921240 conn(0x7f126c0fdbc0 msgr2=0x7f126c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 -- 172.21.15.94:0/2712921240 shutdown_connections 2024-04-25T14:06:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.135+0000 7f1271220640 1 -- 172.21.15.94:0/2712921240 wait complete. 2024-04-25T14:06:58.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 Processor -- start 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 -- start start 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 0x7f126c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c108930 0x7f126c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10e4f0 con 0x7f126c1029f0 2024-04-25T14:06:58.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10e370 con 0x7f126c108930 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.136+0000 7f1271220640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f126c10e670 con 0x7f126c103c30 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59170/0 (socket says 172.21.15.94:59170) 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126ad9d640 1 -- 172.21.15.94:0/2423947306 learned_addr learned my addr 172.21.15.94:0/2423947306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 0x7f126c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126b59e640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c108930 0x7f126c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 -- 172.21.15.94:0/2423947306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c108930 msgr2=0x7f126c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c108930 0x7f126c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 -- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 msgr2=0x7f126c0758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 -- 172.21.15.94:0/2423947306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1a1d70 con 0x7f126c103c30 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126ad9d640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f126a59c640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 0x7f126c19efa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f126000e9f0 tx=0x7f126000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f1253fff640 1 -- 172.21.15.94:0/2423947306 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126000ccc0 con 0x7f126c103c30 2024-04-25T14:06:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c198dd0 con 0x7f126c103c30 2024-04-25T14:06:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f1253fff640 1 -- 172.21.15.94:0/2423947306 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12600108e0 con 0x7f126c103c30 2024-04-25T14:06:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.137+0000 7f1253fff640 1 -- 172.21.15.94:0/2423947306 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260018880 con 0x7f126c103c30 2024-04-25T14:06:58.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.138+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c199320 con 0x7f126c103c30 2024-04-25T14:06:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f1253fff640 1 -- 172.21.15.94:0/2423947306 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1260018a20 con 0x7f126c103c30 2024-04-25T14:06:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 Processor -- start 2024-04-25T14:06:58.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 -- start start 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da603adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da60311d80 con 0x55da6000b390 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da602cb8b0 con 0x55da5ffea6b0 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f12781ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da6030dd40 con 0x55da5ffe24d0 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f126ad9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59184/0 (socket says 172.21.15.94:59184) 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f126b59e640 1 -- 172.21.15.94:0/3172986411 learned_addr learned my addr 172.21.15.94:0/3172986411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.141+0000 7f126a59c640 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da603adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f1253fff640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 0x7f12540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 -- 172.21.15.94:0/3172986411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 msgr2=0x55da603adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da603adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 -- 172.21.15.94:0/3172986411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 msgr2=0x55da603aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 -- 172.21.15.94:0/3172986411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6029ca40 con 0x55da5ffe24d0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126b59e640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 0x7f12540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f1253fff640 1 -- 172.21.15.94:0/2423947306 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f12600defe0 con 0x7f126c103c30 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f126ad9d640 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603acfe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f125c000c00 tx=0x7f125c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24875993d1f8d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f12517fa640 1 -- 172.21.15.94:0/3172986411 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c035070 con 0x55da5ffe24d0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f12517fa640 1 -- 172.21.15.94:0/3172986411 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125c031820 con 0x55da5ffe24d0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.142+0000 7f12517fa640 1 -- 172.21.15.94:0/3172986411 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c003e00 con 0x55da5ffe24d0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 -- 172.21.15.94:0/3172986411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 msgr2=0x55da603acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f126b59e640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 0x7f12540b2000 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1264009fd0 tx=0x7f1264004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603acfe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f125c000c00 tx=0x7f125c009580 comp rx=0 tx=0).stop 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 -- 172.21.15.94:0/3172986411 shutdown_connections 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da603adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603acfe0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 --2- 172.21.15.94:0/3172986411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 -- 172.21.15.94:0/3172986411 >> 172.21.15.94:0/3172986411 conn(0x55da5ffff4e0 msgr2=0x55da603a8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 -- 172.21.15.94:0/3172986411 shutdown_connections 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.143+0000 7f12781ab740 1 -- 172.21.15.94:0/3172986411 wait complete. 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 -- start start 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 Processor -- start 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffe24d0 0x55da603b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffea6b0 0x55da603b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55da603ae720 con 0x55da6000b390 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da60298d10 con 0x55da5ffe24d0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da603ae8a0 con 0x55da5ffea6b0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f12781ab740 10 client.0 _ll_register_callbacks cb 0x55da602c5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59196/0 (socket says 172.21.15.94:59196) 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffe24d0 0x55da603b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 -- 172.21.15.94:0/294851991 learned_addr learned my addr 172.21.15.94:0/294851991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126a59c640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffea6b0 0x55da603b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 -- 172.21.15.94:0/294851991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffea6b0 msgr2=0x55da603b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffea6b0 0x55da603b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 -- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffe24d0 msgr2=0x55da603b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffe24d0 0x55da603b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.144+0000 7f126b59e640 1 -- 172.21.15.94:0/294851991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da600ed7d0 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f126b59e640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f1264020950 tx=0x7f1264004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4919eef166ffaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264010dc0 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f126401b440 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c031820 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1264020e10 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 10 client.24354 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da600233f0 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125c031820 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1264027020 con 0x55da6000b390 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 10 client.24354 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 10 client.24354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 20 client.24354 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da5ffc9f60 0x55da603c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 4 client.24354 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.145+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da6031f990 con 0x55da5ffc9f60 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.146+0000 7f126ad9d640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da5ffc9f60 0x55da603c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.146+0000 7f124eff5640 10 client.24354 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:58.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.146+0000 7f126ad9d640 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da5ffc9f60 0x55da603c42f0 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e3589f601193af server_cookie=44957019cb90c0d1 in_seq=0 out_seq=0 2024-04-25T14:06:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.151+0000 7f124eff5640 1 -- 172.21.15.94:0/294851991 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da6031f990 con 0x55da5ffc9f60 2024-04-25T14:06:58.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.151+0000 7f124eff5640 10 client.24354 handle_command_reply: tid=0 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.151+0000 7f12781ab740 1 client.24354 shutdown 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.151+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 msgr2=0x55da603bdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f1264020950 tx=0x7f1264004ab0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 shutdown_connections 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da5ffc9f60 0x55da603c42f0 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da6000b390 0x55da603bdda0 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffea6b0 0x55da603b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 --2- 172.21.15.94:0/294851991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffe24d0 0x55da603b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f124eff5640 0 client.24354 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 >> 172.21.15.94:0/294851991 conn(0x55da5ffff4e0 msgr2=0x55da603ab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 shutdown_connections 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 1 -- 172.21.15.94:0/294851991 wait complete. 2024-04-25T14:06:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.152+0000 7f12781ab740 20 client.24354 trim_cache size 0 max 16384 2024-04-25T14:06:58.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 246521 18.78 MB/sec execute 580 sec latency 71.227 ms 2024-04-25T14:06:58.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.172+0000 7f12781ab740 1 Processor -- start 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.172+0000 7f12781ab740 1 -- start start 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffc9f60 0x55da603a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ad700 con 0x55da5ffea6b0 2024-04-25T14:06:58.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1af240 con 0x55da5ffc9f60 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f12781ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1addd0 con 0x55da5ffe24d0 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126b59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59208/0 (socket says 172.21.15.94:59208) 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126b59e640 1 -- 172.21.15.94:0/270089243 learned_addr learned my addr 172.21.15.94:0/270089243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126ad9d640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffc9f60 0x55da603a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 -- 172.21.15.94:0/270089243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffc9f60 msgr2=0x55da603a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffc9f60 0x55da603a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 -- 172.21.15.94:0/270089243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 msgr2=0x55da603a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 -- 172.21.15.94:0/270089243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5ffe30f0 con 0x55da5ffe24d0 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126b59e640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.173+0000 7f126a59c640 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bdcc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12600dca90 tx=0x7f1260009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd092d4856ffcc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.174+0000 7f1250ff9640 1 -- 172.21.15.94:0/270089243 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600a6c40 con 0x55da5ffe24d0 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.174+0000 7f1250ff9640 1 -- 172.21.15.94:0/270089243 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12600a6fe0 con 0x55da5ffe24d0 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.174+0000 7f1250ff9640 1 -- 172.21.15.94:0/270089243 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600e10b0 con 0x55da5ffe24d0 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.174+0000 7f12781ab740 1 -- 172.21.15.94:0/270089243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 msgr2=0x55da603bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.174+0000 7f12781ab740 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bdcc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12600dca90 tx=0x7f1260009700 comp rx=0 tx=0).stop 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- 172.21.15.94:0/270089243 shutdown_connections 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffea6b0 0x55da603a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bdcc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- 172.21.15.94:0/270089243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffc9f60 0x55da603a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- 172.21.15.94:0/270089243 >> 172.21.15.94:0/270089243 conn(0x55da5ffff4e0 msgr2=0x55da603b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- 172.21.15.94:0/270089243 shutdown_connections 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- 172.21.15.94:0/270089243 wait complete. 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- start start 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 Processor -- start 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffc9f60 0x55da603baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da6042a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1aeb70 con 0x55da5ffc9f60 2024-04-25T14:06:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ae4a0 con 0x55da5ffea6b0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ab6e0 con 0x55da5ffe24d0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f12781ab740 10 client.0 _ll_register_callbacks cb 0x7f1264009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f126a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.175+0000 7f126a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50954/0 (socket says 172.21.15.94:50954) 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 -- 172.21.15.94:0/2824127469 learned_addr learned my addr 172.21.15.94:0/2824127469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126ad9d640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffc9f60 0x55da603baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126b59e640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da6042a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 -- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 msgr2=0x55da6042a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da6042a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 -- 172.21.15.94:0/2824127469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffc9f60 msgr2=0x55da603baa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffc9f60 0x55da603baa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 -- 172.21.15.94:0/2824127469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da602d24b0 con 0x55da5ffe24d0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f126a59c640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12600a71d0 tx=0x7f12600a79e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844603c5102d392b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f124eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600a6c40 con 0x55da5ffe24d0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f12600e1250 con 0x55da5ffe24d0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.176+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12600a6fe0 con 0x55da5ffe24d0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f12600a74d0 con 0x55da5ffe24d0 2024-04-25T14:06:58.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 10 client.33364 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da6042b940 con 0x55da5ffe24d0 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12600a6fe0 con 0x55da5ffe24d0 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f12600a2e40 con 0x55da5ffe24d0 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 10 client.33364 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 10 client.33364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 20 client.33364 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da6000b390 0x55da6042e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 4 client.33364 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:58.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da60319800 con 0x55da6000b390 2024-04-25T14:06:58.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.177+0000 7f126ad9d640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da6000b390 0x55da6042e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.178+0000 7f124eff5640 10 client.33364 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:58.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.178+0000 7f126ad9d640 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da6000b390 0x55da6042e290 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d600b9143c8e1a server_cookie=b1797fa27971cfb5 in_seq=0 out_seq=0 2024-04-25T14:06:58.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.182+0000 7f124eff5640 1 -- 172.21.15.94:0/2824127469 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55da60319800 con 0x55da6000b390 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.182+0000 7f124eff5640 10 client.33364 handle_command_reply: tid=0 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.182+0000 7f12781ab740 1 client.33364 shutdown 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 msgr2=0x55da603bafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12600a71d0 tx=0x7f12600a79e0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 shutdown_connections 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55da6000b390 0x55da6042e290 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da5ffea6b0 0x55da6042a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5ffe24d0 0x55da603bafc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f124eff5640 0 client.33364 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 --2- 172.21.15.94:0/2824127469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da5ffc9f60 0x55da603baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 >> 172.21.15.94:0/2824127469 conn(0x55da5ffff4e0 msgr2=0x55da603a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 shutdown_connections 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 1 -- 172.21.15.94:0/2824127469 wait complete. 2024-04-25T14:06:58.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f12781ab740 20 client.33364 trim_cache size 0 max 16384 2024-04-25T14:06:58.194 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:06:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:06:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:06:58.194 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 msgr2=0x7f12540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 0x7f12540b2000 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1264009fd0 tx=0x7f1264004a40 comp rx=0 tx=0).stop 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 msgr2=0x7f126c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 0x7f126c19efa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f126000e9f0 tx=0x7f126000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 shutdown_connections 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f12540afaf0 0x7f12540b2000 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f126c108930 0x7f126c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f126c103c30 0x7f126c19efa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 --2- 172.21.15.94:0/2423947306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f126c1029f0 0x7f126c0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 >> 172.21.15.94:0/2423947306 conn(0x7f126c0fdbc0 msgr2=0x7f126c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 shutdown_connections 2024-04-25T14:06:58.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.183+0000 7f1271220640 1 -- 172.21.15.94:0/2423947306 wait complete. 2024-04-25T14:06:58.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:06:58.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:58.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 -- 172.21.15.94:0/708181101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70108940 msgr2=0x7f7e7010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 --2- 172.21.15.94:0/708181101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70108940 0x7f7e7010ad60 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009ca0 tx=0x7f7e6802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 -- 172.21.15.94:0/708181101 shutdown_connections 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 --2- 172.21.15.94:0/708181101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70108940 0x7f7e7010ad60 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 --2- 172.21.15.94:0/708181101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 0x7f7e70104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 --2- 172.21.15.94:0/708181101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70102a00 0x7f7e70103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 -- 172.21.15.94:0/708181101 >> 172.21.15.94:0/708181101 conn(0x7f7e700fdbf0 msgr2=0x7f7e701000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 -- 172.21.15.94:0/708181101 shutdown_connections 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.331+0000 7f7e75df3640 1 -- 172.21.15.94:0/708181101 wait complete. 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 Processor -- start 2024-04-25T14:06:58.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 -- start start 2024-04-25T14:06:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 0x7f7e7019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70108940 0x7f7e701a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f630 con 0x7f7e70102a00 2024-04-25T14:06:58.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f4b0 con 0x7f7e70103c40 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e75df3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010f7b0 con 0x7f7e70108940 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59240/0 (socket says 172.21.15.94:59240) 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/414658624 learned_addr learned my addr 172.21.15.94:0/414658624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e6effd640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 0x7f7e7019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.332+0000 7f7e6ffff640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70108940 0x7f7e701a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/414658624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70108940 msgr2=0x7f7e701a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70108940 0x7f7e701a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/414658624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 msgr2=0x7f7e7019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 0x7f7e7019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 -- 172.21.15.94:0/414658624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a1d10 con 0x7f7e70102a00 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6f7fe640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e9f0 tx=0x7f7e6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000ccc0 con 0x7f7e70102a00 2024-04-25T14:06:58.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e600035f0 con 0x7f7e70102a00 2024-04-25T14:06:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a1fa0 con 0x7f7e70102a00 2024-04-25T14:06:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000ce60 con 0x7f7e70102a00 2024-04-25T14:06:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.333+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a2410 con 0x7f7e70102a00 2024-04-25T14:06:58.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.334+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e70107750 con 0x7f7e70102a00 2024-04-25T14:06:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.336+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7e60003aa0 con 0x7f7e70102a00 2024-04-25T14:06:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.336+0000 7f7e6cff9640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.337+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7e600df070 con 0x7f7e70102a00 2024-04-25T14:06:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.337+0000 7f7e6effd640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.337+0000 7f7e6effd640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 0x7f7e580b2000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7e7019b270 tx=0x7f7e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.341+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e600a7bb0 con 0x7f7e70102a00 2024-04-25T14:06:58.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.577+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e7002db50 con 0x7f7e70102a00 2024-04-25T14:06:58.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.578+0000 7f7e6cff9640 1 -- 172.21.15.94:0/414658624 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11938 (secure 0 0 0) 0x7f7e600a74b0 con 0x7f7e70102a00 2024-04-25T14:06:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:06:58.578 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T13:58:46.617960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24557,24512,24475,24502,24488]},"id":1}]} 2024-04-25T14:06:58.578 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 39 2024-04-25T14:06:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 msgr2=0x7f7e580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 0x7f7e580b2000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7e7019b270 tx=0x7f7e64005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 msgr2=0x7f7e7019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e9f0 tx=0x7f7e6000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 shutdown_connections 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7e580afaf0 0x7f7e580b2000 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70108940 0x7f7e701a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e70103c40 0x7f7e7019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 --2- 172.21.15.94:0/414658624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e70102a00 0x7f7e7019a1d0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 >> 172.21.15.94:0/414658624 conn(0x7f7e700fdbf0 msgr2=0x7f7e70104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 shutdown_connections 2024-04-25T14:06:58.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.580+0000 7f7e75df3640 1 -- 172.21.15.94:0/414658624 wait complete. 2024-04-25T14:06:58.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:06:58.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:58 smithi137 ceph-mon[32568]: pgmap v1106: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-04-25T14:06:58.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3767653311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc0103f50 msgr2=0x7f1dc0102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3767653311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc0103f50 0x7f1dc0102840 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009900 tx=0x7f1db402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3767653311 shutdown_connections 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3767653311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0108a90 0x7f1dc010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3767653311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc01043f0 0x7f1dc0102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3767653311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc0103f50 0x7f1dc0102840 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3767653311 >> 172.21.15.94:0/3767653311 conn(0x7f1dc00fd340 msgr2=0x7f1dc00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3767653311 shutdown_connections 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.707+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3767653311 wait complete. 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 Processor -- start 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 -- start start 2024-04-25T14:06:58.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0103f50 0x7f1dc019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc0108a90 0x7f1dc01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc010f620 con 0x7f1dc01043f0 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc010f4a0 con 0x7f1dc0108a90 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc6bd5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc010f7a0 con 0x7f1dc0103f50 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59258/0 (socket says 172.21.15.94:59258) 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dbffff640 1 -- 172.21.15.94:0/3933500739 learned_addr learned my addr 172.21.15.94:0/3933500739 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc5172640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc0108a90 0x7f1dc01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dc4971640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0103f50 0x7f1dc019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.708+0000 7f1dbffff640 1 -- 172.21.15.94:0/3933500739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0103f50 msgr2=0x7f1dc019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbffff640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0103f50 0x7f1dc019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbffff640 1 -- 172.21.15.94:0/3933500739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc0108a90 msgr2=0x7f1dc01a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbffff640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc0108a90 0x7f1dc01a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbffff640 1 -- 172.21.15.94:0/3933500739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc01a1d80 con 0x7f1dc01043f0 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbffff640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbdffb640 1 -- 172.21.15.94:0/3933500739 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00cd80 con 0x7f1dc01043f0 2024-04-25T14:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbdffb640 1 -- 172.21.15.94:0/3933500739 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0108e0 con 0x7f1dc01043f0 2024-04-25T14:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc01a2070 con 0x7f1dc01043f0 2024-04-25T14:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dbdffb640 1 -- 172.21.15.94:0/3933500739 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac018980 con 0x7f1dc01043f0 2024-04-25T14:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.709+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc01a2610 con 0x7f1dc01043f0 2024-04-25T14:06:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 Processor -- start 2024-04-25T14:06:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 -- start start 2024-04-25T14:06:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250428d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250429a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25038cd80 con 0x55f250086390 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f250346a20 con 0x55f2500656b0 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dcd722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f250388d40 con 0x55f25005d4d0 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dc5172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dc5172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59270/0 (socket says 172.21.15.94:59270) 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dc5172640 1 -- 172.21.15.94:0/3836536776 learned_addr learned my addr 172.21.15.94:0/3836536776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.712+0000 7f1dc4971640 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250428d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dbdffb640 1 -- 172.21.15.94:0/3933500739 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1dac010430 con 0x7f1dc01043f0 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 -- 172.21.15.94:0/3836536776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 msgr2=0x55f250428d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250428d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 -- 172.21.15.94:0/3836536776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 msgr2=0x55f250429a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250429a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 -- 172.21.15.94:0/3836536776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f250351c30 con 0x55f250086390 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dc5172640 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f1db800ae90 tx=0x7f1db800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f98a5663ec9498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1daaffd640 1 -- 172.21.15.94:0/3836536776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8003b70 con 0x55f250086390 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1daaffd640 1 -- 172.21.15.94:0/3836536776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db8003d10 con 0x55f250086390 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1daaffd640 1 -- 172.21.15.94:0/3836536776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8010ba0 con 0x55f250086390 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.713+0000 7f1dbdffb640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 0x7f1db00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dc4971640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 0x7f1db00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dbdffb640 1 -- 172.21.15.94:0/3933500739 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1dac0defc0 con 0x7f1dc01043f0 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 -- 172.21.15.94:0/3836536776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 msgr2=0x55f250428810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dc4971640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 0x7f1db00b2080 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f1db800ae90 tx=0x7f1db800f790 comp rx=0 tx=0).stop 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 -- 172.21.15.94:0/3836536776 shutdown_connections 2024-04-25T14:06:58.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250429a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250428d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 --2- 172.21.15.94:0/3836536776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250428810 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 -- 172.21.15.94:0/3836536776 >> 172.21.15.94:0/3836536776 conn(0x55f25007a4e0 msgr2=0x55f250424020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.714+0000 7f1dcd722740 1 -- 172.21.15.94:0/3836536776 shutdown_connections 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.715+0000 7f1dcd722740 1 -- 172.21.15.94:0/3836536776 wait complete. 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.715+0000 7f1dcd722740 1 -- start start 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.715+0000 7f1dcd722740 1 Processor -- start 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.715+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.715+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250432bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f250086390 0x55f250439b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f25042a530 con 0x55f25005d4d0 2024-04-25T14:06:58.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f250313dc0 con 0x55f2500656b0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f25042a6b0 con 0x55f250086390 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 10 client.0 _ll_register_callbacks cb 0x55f25032f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250432bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59276/0 (socket says 172.21.15.94:59276) 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 -- 172.21.15.94:0/1498366875 learned_addr learned my addr 172.21.15.94:0/1498366875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc5172640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f250086390 0x55f250439b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 -- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f250086390 msgr2=0x55f250439b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f250086390 0x55f250439b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 -- 172.21.15.94:0/1498366875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 msgr2=0x55f250432bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250432bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 -- 172.21.15.94:0/1498366875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f25009e3f0 con 0x55f25005d4d0 2024-04-25T14:06:58.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dbffff640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250432bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dc4971640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f1db40096c0 tx=0x7f1db4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14005c05f76a2af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1d9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db404c030 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1db4005540 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.716+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db8003d10 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1db4035040 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 10 client.24378 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2501687d0 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db8003d10 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1db4048970 con 0x55f25005d4d0 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 10 client.24378 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 10 client.24378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 20 client.24378 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f250044f60 0x55f250440770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 4 client.24378 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f25039a990 con 0x55f250044f60 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.717+0000 7f1dbffff640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f250044f60 0x55f250440770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.718+0000 7f1d9ffff640 10 client.24378 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.718+0000 7f1dbffff640 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f250044f60 0x55f250440770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2d07c201fcba9fb server_cookie=9c281adba958c351 in_seq=0 out_seq=0 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1d9ffff640 1 -- 172.21.15.94:0/1498366875 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f25039a990 con 0x55f250044f60 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1d9ffff640 10 client.24378 handle_command_reply: tid=0 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 client.24378 shutdown 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 msgr2=0x55f250432690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f1db40096c0 tx=0x7f1db4009260 comp rx=0 tx=0).stop 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 shutdown_connections 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f250044f60 0x55f250440770 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f250086390 0x55f250439b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250432bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 --2- 172.21.15.94:0/1498366875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f25005d4d0 0x55f250432690 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1d9ffff640 0 client.24378 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 >> 172.21.15.94:0/1498366875 conn(0x55f25007a4e0 msgr2=0x55f250423040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 shutdown_connections 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 1 -- 172.21.15.94:0/1498366875 wait complete. 2024-04-25T14:06:58.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.719+0000 7f1dcd722740 20 client.24378 trim_cache size 0 max 16384 2024-04-25T14:06:58.730 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:58 smithi094 ceph-mon[29195]: pgmap v1106: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-04-25T14:06:58.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 Processor -- start 2024-04-25T14:06:58.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 -- start start 2024-04-25T14:06:58.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f25042d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f25042d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01ab580 con 0x55f250086390 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01ad0c0 con 0x55f2500656b0 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dcd722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01abc50 con 0x55f25005d4d0 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.739+0000 7f1dc5172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f25042d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59284/0 (socket says 172.21.15.94:59284) 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 -- 172.21.15.94:0/83926954 learned_addr learned my addr 172.21.15.94:0/83926954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc4971640 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f25042d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 -- 172.21.15.94:0/83926954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 msgr2=0x55f25042d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f25042d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 -- 172.21.15.94:0/83926954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 msgr2=0x55f25042d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f25042d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 -- 172.21.15.94:0/83926954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f25041ed30 con 0x55f250086390 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dbffff640 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f25042d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1dc5172640 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x55f250429860 tx=0x7f1db800aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7ffd12718209e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1daa7fc640 1 -- 172.21.15.94:0/83926954 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db801d040 con 0x55f250086390 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1daa7fc640 1 -- 172.21.15.94:0/83926954 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db8010d60 con 0x55f250086390 2024-04-25T14:06:58.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.740+0000 7f1daa7fc640 1 -- 172.21.15.94:0/83926954 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8018d90 con 0x55f250086390 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.741+0000 7f1dcd722740 1 -- 172.21.15.94:0/83926954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 msgr2=0x55f250432260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.741+0000 7f1dcd722740 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x55f250429860 tx=0x7f1db800aa10 comp rx=0 tx=0).stop 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.741+0000 7f1dcd722740 1 -- 172.21.15.94:0/83926954 shutdown_connections 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f250432260 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f25042d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 --2- 172.21.15.94:0/83926954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f25042d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 -- 172.21.15.94:0/83926954 >> 172.21.15.94:0/83926954 conn(0x55f25007a4e0 msgr2=0x55f250430d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 -- 172.21.15.94:0/83926954 shutdown_connections 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 -- 172.21.15.94:0/83926954 wait complete. 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 -- start start 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 Processor -- start 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.742+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250436240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f2504a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01ac9f0 con 0x55f250086390 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01ac320 con 0x55f2500656b0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc01a9560 con 0x55f25005d4d0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dc5172640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f2504a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37358/0 (socket says 172.21.15.94:37358) 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 -- 172.21.15.94:0/778274736 learned_addr learned my addr 172.21.15.94:0/778274736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 10 client.0 _ll_register_callbacks cb 0x7f1db4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dc4971640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250436240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 -- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 msgr2=0x55f250436240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250436240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 -- 172.21.15.94:0/778274736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 msgr2=0x55f2504a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f2504a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 -- 172.21.15.94:0/778274736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2504a7080 con 0x55f2500656b0 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dc5172640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f2504a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dbffff640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0a7100 tx=0x7f1dac0df2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aee1bb36103a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1d9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0a7400 con 0x55f2500656b0 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db8010d60 con 0x55f2500656b0 2024-04-25T14:06:58.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.743+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1dac0e1110 con 0x55f2500656b0 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1dac0ab7b0 con 0x55f2500656b0 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 10 client.33746 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2504a83b0 con 0x55f2500656b0 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db8010d60 con 0x55f2500656b0 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1dac0a1b90 con 0x55f2500656b0 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 10 client.33746 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 10 client.33746 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 20 client.33746 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f25001a3a0 0x55f2504aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 4 client.33746 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f250394800 con 0x55f25001a3a0 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.744+0000 7f1dc4971640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f25001a3a0 0x55f2504aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dc4971640 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f25001a3a0 0x55f2504aad50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ece1c2897fd25d5d server_cookie=8bcf99cca5b87591 in_seq=0 out_seq=0 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1d9ffff640 10 client.33746 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1d9ffff640 1 -- 172.21.15.94:0/778274736 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f250394800 con 0x55f25001a3a0 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1d9ffff640 10 client.33746 handle_command_reply: tid=0 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 client.33746 shutdown 2024-04-25T14:06:58.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 msgr2=0x55f250436780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0a7100 tx=0x7f1dac0df2c0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 shutdown_connections 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55f25001a3a0 0x55f2504aad50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f250086390 0x55f2504a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2500656b0 0x55f250436780 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 --2- 172.21.15.94:0/778274736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f25005d4d0 0x55f250436240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1d9ffff640 0 client.33746 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:06:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 >> 172.21.15.94:0/778274736 conn(0x55f25007a4e0 msgr2=0x55f250426e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.745+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 shutdown_connections 2024-04-25T14:06:58.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.746+0000 7f1dcd722740 1 -- 172.21.15.94:0/778274736 wait complete. 2024-04-25T14:06:58.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.746+0000 7f1dcd722740 20 client.33746 trim_cache size 0 max 16384 2024-04-25T14:06:58.755 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:58.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.746+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 msgr2=0x7f1db00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 0x7f1db00b2080 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db40314a0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 msgr2=0x7f1dc019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 shutdown_connections 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1db00afb70 0x7f1db00b2080 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dc0108a90 0x7f1dc01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dc01043f0 0x7f1dc019a780 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 --2- 172.21.15.94:0/3933500739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0103f50 0x7f1dc019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 >> 172.21.15.94:0/3933500739 conn(0x7f1dc00fd340 msgr2=0x7f1dc0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 shutdown_connections 2024-04-25T14:06:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.747+0000 7f1dc6bd5640 1 -- 172.21.15.94:0/3933500739 wait complete. 2024-04-25T14:06:58.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:58 smithi138 ceph-mon[32977]: pgmap v1106: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.8 MiB/s wr, 482 op/s 2024-04-25T14:06:58.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 -- 172.21.15.94:0/2359509339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 msgr2=0x7f5af8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 --2- 172.21.15.94:0/2359509339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af8104040 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009900 tx=0x7f5ae402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 -- 172.21.15.94:0/2359509339 shutdown_connections 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 --2- 172.21.15.94:0/2359509339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af8108930 0x7f5af810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 --2- 172.21.15.94:0/2359509339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af8104040 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 --2- 172.21.15.94:0/2359509339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af81029f0 0x7f5af81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 -- 172.21.15.94:0/2359509339 >> 172.21.15.94:0/2359509339 conn(0x7f5af80fdc00 msgr2=0x7f5af8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.894+0000 7f5afed27640 1 -- 172.21.15.94:0/2359509339 shutdown_connections 2024-04-25T14:06:58.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.895+0000 7f5afed27640 1 -- 172.21.15.94:0/2359509339 wait complete. 2024-04-25T14:06:58.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.895+0000 7f5afed27640 1 Processor -- start 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 -- start start 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8108930 0x7f5af81a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5af810d410 con 0x7f5af8103c30 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5af810d290 con 0x7f5af81029f0 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afed27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5af810d590 con 0x7f5af8108930 2024-04-25T14:06:58.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37388/0 (socket says 172.21.15.94:37388) 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 -- 172.21.15.94:0/2159566111 learned_addr learned my addr 172.21.15.94:0/2159566111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8108930 0x7f5af81a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 -- 172.21.15.94:0/2159566111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8108930 msgr2=0x7f5af81a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8108930 0x7f5af81a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 -- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 msgr2=0x7f5af81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.896+0000 7f5afcac3640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5afcac3640 1 -- 172.21.15.94:0/2159566111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af806c6f0 con 0x7f5af81029f0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5af7fff640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5afcac3640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009990 tx=0x7f5aec009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5af5ffb640 1 -- 172.21.15.94:0/2159566111 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01b070 con 0x7f5af81029f0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af806c9e0 con 0x7f5af81029f0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5af5ffb640 1 -- 172.21.15.94:0/2159566111 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec00de10 con 0x7f5af81029f0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5af5ffb640 1 -- 172.21.15.94:0/2159566111 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0039f0 con 0x7f5af81029f0 2024-04-25T14:06:58.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.897+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af806cf30 con 0x7f5af81029f0 2024-04-25T14:06:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 Processor -- start 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5af5ffb640 1 -- 172.21.15.94:0/2159566111 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5aec01e070 con 0x7f5af81029f0 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 -- start start 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 0x56440ab41810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 0x56440ab42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56440aaa5d80 con 0x56440a7764d0 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56440aa5fa20 con 0x56440a79f390 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.901+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56440aaa1d40 con 0x56440a77e6b0 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 0x56440ab42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59308/0 (socket says 172.21.15.94:59308) 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 0x56440ab42a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51054/0 (socket says 172.21.15.94:51054) 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 -- 172.21.15.94:0/1917639182 learned_addr learned my addr 172.21.15.94:0/1917639182 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5af5ffb640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 0x7f5ae80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 0x56440ab41810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 -- 172.21.15.94:0/1917639182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 msgr2=0x56440ab42a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 0x56440ab42a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 -- 172.21.15.94:0/1917639182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 msgr2=0x56440ab41810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 0x56440ab41810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afcac3640 1 -- 172.21.15.94:0/1917639182 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440aa6ac30 con 0x56440a7764d0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 0x56440ab41810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 0x7f5ae80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.902+0000 7f5af5ffb640 1 -- 172.21.15.94:0/2159566111 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5aec0de1e0 con 0x7f5af81029f0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.903+0000 7f5afcac3640 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f5aec04fff0 tx=0x7f5aec022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d0c5876d6dc8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.903+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 0x7f5ae80b2000 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5af00049d0 tx=0x7f5af0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.903+0000 7f5ae2ffd640 1 -- 172.21.15.94:0/1917639182 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0a63f0 con 0x56440a7764d0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.903+0000 7f5ae2ffd640 1 -- 172.21.15.94:0/1917639182 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec01a170 con 0x56440a7764d0 2024-04-25T14:06:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.903+0000 7f5ae2ffd640 1 -- 172.21.15.94:0/1917639182 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0a5d90 con 0x56440a7764d0 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 -- 172.21.15.94:0/1917639182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 msgr2=0x56440ab41d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f5aec04fff0 tx=0x7f5aec022000 comp rx=0 tx=0).stop 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 -- 172.21.15.94:0/1917639182 shutdown_connections 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a77e6b0 0x56440ab42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a7764d0 0x56440ab41d90 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/1917639182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a79f390 0x56440ab41810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 -- 172.21.15.94:0/1917639182 >> 172.21.15.94:0/1917639182 conn(0x56440a7934e0 msgr2=0x56440ab3d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 -- 172.21.15.94:0/1917639182 shutdown_connections 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.904+0000 7f5b05bd6740 1 -- 172.21.15.94:0/1917639182 wait complete. 2024-04-25T14:06:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 -- start start 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 Processor -- start 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440ab529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56440ab434d0 con 0x56440a77e6b0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56440aa2cdc0 con 0x56440a7764d0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56440ab43650 con 0x56440a79f390 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5b05bd6740 10 client.0 _ll_register_callbacks cb 0x56440aa48600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5afcac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.905+0000 7f5afd2c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440ab529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59324/0 (socket says 172.21.15.94:59324) 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 -- 172.21.15.94:0/3558963281 learned_addr learned my addr 172.21.15.94:0/3558963281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 -- 172.21.15.94:0/3558963281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 msgr2=0x56440ab529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440ab529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 -- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 msgr2=0x56440ab4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 -- 172.21.15.94:0/3558963281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440a7b73f0 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4031440 tx=0x7f5ae40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2760d955f3c573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.906+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4035040 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5ae4004450 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aec01a170 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ae4031920 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5b05bd6740 10 client.24396 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56440a8817d0 con 0x56440a77e6b0 2024-04-25T14:06:58.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec01a170 con 0x56440a77e6b0 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ae4044070 con 0x56440a77e6b0 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5b05bd6740 10 client.24396 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.907+0000 7f5b05bd6740 10 client.24396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.908+0000 7f5b05bd6740 20 client.24396 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.908+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a75df60 0x56440ab591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.908+0000 7f5b05bd6740 4 client.24396 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.908+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56440aab3990 con 0x56440a75df60 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.908+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a75df60 0x56440ab591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.909+0000 7f5ad7fff640 10 client.24396 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.909+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a75df60 0x56440ab591e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=617f063b2ee9aa4e server_cookie=4ee8d52ef4660420 in_seq=0 out_seq=0 2024-04-25T14:06:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3558963281 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56440aab3990 con 0x56440a75df60 2024-04-25T14:06:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5ad7fff640 10 client.24396 handle_command_reply: tid=0 2024-04-25T14:06:58.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 client.24396 shutdown 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 msgr2=0x56440ab4ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4031440 tx=0x7f5ae40317b0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 shutdown_connections 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a75df60 0x56440ab591e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440ab529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab4ba70 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3558963281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 >> 172.21.15.94:0/3558963281 conn(0x56440a7934e0 msgr2=0x56440ab3ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5ad7fff640 0 client.24396 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 shutdown_connections 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3558963281 wait complete. 2024-04-25T14:06:58.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.910+0000 7f5b05bd6740 20 client.24396 trim_cache size 0 max 16384 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 Processor -- start 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 -- start start 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 0x56440ab4a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a79f390 0x56440ab5d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81ba2a0 con 0x56440a79f390 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81bbde0 con 0x56440a77e6b0 2024-04-25T14:06:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81ba970 con 0x56440a7764d0 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37414/0 (socket says 172.21.15.94:37414) 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5af7fff640 1 -- 172.21.15.94:0/3335114498 learned_addr learned my addr 172.21.15.94:0/3335114498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a79f390 0x56440ab5d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.926+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 0x56440ab4a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 -- 172.21.15.94:0/3335114498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 msgr2=0x56440ab528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 -- 172.21.15.94:0/3335114498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a79f390 msgr2=0x56440ab5d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a79f390 0x56440ab5d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 -- 172.21.15.94:0/3335114498 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440ab4a900 con 0x56440a7764d0 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:58.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 0x56440ab4a3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0a6320 tx=0x7f5aec0a2610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b574101ed7d1ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5ae27fc640 1 -- 172.21.15.94:0/3335114498 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0a5f70 con 0x56440a7764d0 2024-04-25T14:06:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5ae27fc640 1 -- 172.21.15.94:0/3335114498 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec0a6130 con 0x56440a7764d0 2024-04-25T14:06:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.927+0000 7f5ae27fc640 1 -- 172.21.15.94:0/3335114498 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0e1950 con 0x56440a7764d0 2024-04-25T14:06:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3335114498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 msgr2=0x56440ab4a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 0x56440ab4a3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0a6320 tx=0x7f5aec0a2610 comp rx=0 tx=0).stop 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3335114498 shutdown_connections 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a79f390 0x56440ab5d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a77e6b0 0x56440ab528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3335114498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a7764d0 0x56440ab4a3c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3335114498 >> 172.21.15.94:0/3335114498 conn(0x56440a7934e0 msgr2=0x56440ab49cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3335114498 shutdown_connections 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.928+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3335114498 wait complete. 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 -- start start 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 Processor -- start 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab51c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440abbf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81bb710 con 0x56440a77e6b0 2024-04-25T14:06:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81bb040 con 0x56440a7764d0 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8073dd0 con 0x56440a79f390 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.929+0000 7f5b05bd6740 10 client.0 _ll_register_callbacks cb 0x7f5ae4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59344/0 (socket says 172.21.15.94:59344) 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 -- 172.21.15.94:0/3332466275 learned_addr learned my addr 172.21.15.94:0/3332466275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab51c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5afd2c4640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440abbf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 -- 172.21.15.94:0/3332466275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 msgr2=0x56440abbf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440abbf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 -- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 msgr2=0x56440ab51c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab51c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 -- 172.21.15.94:0/3332466275 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440abbfd40 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5af7fff640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402a930 tx=0x7f5ae4032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca23d7e445d973bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.930+0000 7f5ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4053070 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5ae4004450 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aec0a6130 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ae405f050 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5b05bd6740 10 client.24402 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.931+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56440abc1060 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4003aa0 con 0x56440a77e6b0 2024-04-25T14:06:58.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ae4044070 con 0x56440a77e6b0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 10 client.24402 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 10 client.24402 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 20 client.24402 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a7333a0 0x56440abc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 4 client.24402 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5aec0a6130 con 0x56440a7333a0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.932+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a7333a0 0x56440abc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5ad7fff640 10 client.24402 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5afcac3640 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a7333a0 0x56440abc3a70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34d0ef762eefd6f server_cookie=94407dd8db0ccf57 in_seq=0 out_seq=0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5ad7fff640 1 -- 172.21.15.94:0/3332466275 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5aec0a6130 con 0x56440a7333a0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5ad7fff640 10 client.24402 handle_command_reply: tid=0 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 client.24402 shutdown 2024-04-25T14:06:58.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 msgr2=0x56440ab52150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402a930 tx=0x7f5ae4032f70 comp rx=0 tx=0).stop 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 shutdown_connections 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x56440a7333a0 0x56440abc3a70 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5ad7fff640 0 client.24402 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56440a79f390 0x56440abbf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56440a77e6b0 0x56440ab52150 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 --2- 172.21.15.94:0/3332466275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56440a7764d0 0x56440ab51c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.933+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 >> 172.21.15.94:0/3332466275 conn(0x56440a7934e0 msgr2=0x56440ab3fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.934+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 shutdown_connections 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.934+0000 7f5b05bd6740 1 -- 172.21.15.94:0/3332466275 wait complete. 2024-04-25T14:06:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.934+0000 7f5b05bd6740 20 client.24402 trim_cache size 0 max 16384 2024-04-25T14:06:58.940 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 msgr2=0x7f5ae80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 0x7f5ae80b2000 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5af00049d0 tx=0x7f5af0004a40 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 msgr2=0x7f5af81a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009990 tx=0x7f5aec009d90 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 shutdown_connections 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5ae80afaf0 0x7f5ae80b2000 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8108930 0x7f5af81a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5af8103c30 0x7f5af81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 --2- 172.21.15.94:0/2159566111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af81029f0 0x7f5af81a0850 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 >> 172.21.15.94:0/2159566111 conn(0x7f5af80fdc00 msgr2=0x7f5af8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 shutdown_connections 2024-04-25T14:06:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:58.935+0000 7f5afed27640 1 -- 172.21.15.94:0/2159566111 wait complete. 2024-04-25T14:06:58.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:06:59.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 -- 172.21.15.94:0/1314378113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc1029e0 msgr2=0x7f68dc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/1314378113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc1029e0 0x7f68dc1036e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009860 tx=0x7f68cc02a340 comp rx=0 tx=0).stop 2024-04-25T14:06:59.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 -- 172.21.15.94:0/1314378113 shutdown_connections 2024-04-25T14:06:59.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/1314378113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc108920 0x7f68dc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/1314378113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc103c20 0x7f68dc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/1314378113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc1029e0 0x7f68dc1036e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.080+0000 7f68e0ee0640 1 -- 172.21.15.94:0/1314378113 >> 172.21.15.94:0/1314378113 conn(0x7f68dc0fdbd0 msgr2=0x7f68dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.081+0000 7f68e0ee0640 1 -- 172.21.15.94:0/1314378113 shutdown_connections 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.081+0000 7f68e0ee0640 1 -- 172.21.15.94:0/1314378113 wait complete. 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.081+0000 7f68e0ee0640 1 Processor -- start 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.081+0000 7f68e0ee0640 1 -- start start 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc103c20 0x7f68dc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc108920 0x7f68dc1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc10e4e0 con 0x7f68dc103c20 2024-04-25T14:06:59.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc10e360 con 0x7f68dc1029e0 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68e0ee0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc10e660 con 0x7f68dc108920 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc103c20 0x7f68dc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37442/0 (socket says 172.21.15.94:37442) 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 -- 172.21.15.94:0/2631494165 learned_addr learned my addr 172.21.15.94:0/2631494165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc108920 0x7f68dc1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 -- 172.21.15.94:0/2631494165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc108920 msgr2=0x7f68dc1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc108920 0x7f68dc1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 -- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc103c20 msgr2=0x7f68dc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc103c20 0x7f68dc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.082+0000 7f68da59c640 1 -- 172.21.15.94:0/2631494165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc1a0fa0 con 0x7f68dc1029e0 2024-04-25T14:06:59.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68da59c640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02a860 tx=0x7f68cc032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68cb7fe640 1 -- 172.21.15.94:0/2631494165 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc037040 con 0x7f68dc1029e0 2024-04-25T14:06:59.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc069e90 con 0x7f68dc1029e0 2024-04-25T14:06:59.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68cb7fe640 1 -- 172.21.15.94:0/2631494165 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc030c10 con 0x7f68dc1029e0 2024-04-25T14:06:59.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68cb7fe640 1 -- 172.21.15.94:0/2631494165 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0052b0 con 0x7f68dc1029e0 2024-04-25T14:06:59.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.083+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc06a320 con 0x7f68dc1029e0 2024-04-25T14:06:59.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.086+0000 7f68cb7fe640 1 -- 172.21.15.94:0/2631494165 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f68cc043020 con 0x7f68dc1029e0 2024-04-25T14:06:59.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.086+0000 7f68e7d90740 1 Processor -- start 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 -- start start 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68cb7fe640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 0x7f68c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 0x5585153a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68cb7fe640 1 -- 172.21.15.94:0/2631494165 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68cc0fe230 con 0x7f68dc1029e0 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558514fe36b0 0x5585153a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55851530ad80 con 0x558514fe36b0 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5585152c4a20 con 0x558515004390 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68e7d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558515306d40 con 0x558514fdb4d0 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.087+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 0x5585153a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51098/0 (socket says 172.21.15.94:51098) 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68d9d9b640 1 -- 172.21.15.94:0/4068708334 learned_addr learned my addr 172.21.15.94:0/4068708334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 0x7f68c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68dad9d640 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558514fe36b0 0x5585153a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 -- 172.21.15.94:0/4068708334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 msgr2=0x5585153a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 -- 172.21.15.94:0/4068708334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558514fe36b0 msgr2=0x5585153a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558514fe36b0 0x5585153a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 -- 172.21.15.94:0/4068708334 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585152cfc30 con 0x558515004390 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68da59c640 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 0x5585153a6810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0c63a0 tx=0x7f68cc0c6470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c694f70321794c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.088+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 0x7f68c40b2080 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f68d4005fd0 tx=0x7f68d4009e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68c8ff9640 1 -- 172.21.15.94:0/4068708334 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0c61b0 con 0x558515004390 2024-04-25T14:06:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68c8ff9640 1 -- 172.21.15.94:0/4068708334 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0c6da0 con 0x558515004390 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 -- 172.21.15.94:0/4068708334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 msgr2=0x5585153a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 0x5585153a6810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0c63a0 tx=0x7f68cc0c6470 comp rx=0 tx=0).stop 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 -- 172.21.15.94:0/4068708334 shutdown_connections 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558514fe36b0 0x5585153a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 --2- 172.21.15.94:0/4068708334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558515004390 0x5585153a6810 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.089+0000 7f68e7d90740 1 -- 172.21.15.94:0/4068708334 >> 172.21.15.94:0/4068708334 conn(0x558514ff84e0 msgr2=0x5585153a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 -- 172.21.15.94:0/4068708334 shutdown_connections 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 -- 172.21.15.94:0/4068708334 wait complete. 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 -- start start 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 Processor -- start 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.090+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68e7d90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5585153a82f0 con 0x558515004390 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68e7d90740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558515291dc0 con 0x558514fe36b0 2024-04-25T14:06:59.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68e7d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585153a8470 con 0x558514fdb4d0 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68e7d90740 10 client.0 _ll_register_callbacks cb 0x5585152ad600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68da59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37468/0 (socket says 172.21.15.94:37468) 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 -- 172.21.15.94:0/2878593627 learned_addr learned my addr 172.21.15.94:0/2878593627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 -- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 msgr2=0x5585153b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 -- 172.21.15.94:0/2878593627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 msgr2=0x5585153b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68d9d9b640 1 -- 172.21.15.94:0/2878593627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851501c3f0 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.091+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68d000f6a0 tx=0x7f68d000faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a88de1ae53820061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d001f060 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68cc0c6da0 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68d00035f0 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f68d00037f0 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68e7d90740 10 client.33773 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5585150e67d0 con 0x558514fe36b0 2024-04-25T14:06:59.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.092+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68cc0c6da0 con 0x558514fe36b0 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f68d000eb20 con 0x558514fe36b0 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 10 client.33773 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 10 client.33773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 20 client.33773 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514fc2f60 0x5585153be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 4 client.33773 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558515318990 con 0x558514fc2f60 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68da59c640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514fc2f60 0x5585153be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68c1ffb640 10 client.33773 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.093+0000 7f68da59c640 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514fc2f60 0x5585153be000 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d46906ab24c8d8cc server_cookie=652182a6e18ca79b in_seq=0 out_seq=0 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2878593627 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558515318990 con 0x558514fc2f60 2024-04-25T14:06:59.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68c1ffb640 10 client.33773 handle_command_reply: tid=0 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 client.33773 shutdown 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 msgr2=0x5585153b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68d000f6a0 tx=0x7f68d000faa0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 shutdown_connections 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514fc2f60 0x5585153be000 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153b07d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2878593627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68c1ffb640 0 client.33773 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 >> 172.21.15.94:0/2878593627 conn(0x558514ff84e0 msgr2=0x5585153a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 shutdown_connections 2024-04-25T14:06:59.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.094+0000 7f68e7d90740 1 -- 172.21.15.94:0/2878593627 wait complete. 2024-04-25T14:06:59.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.095+0000 7f68e7d90740 20 client.33773 trim_cache size 0 max 16384 2024-04-25T14:06:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 Processor -- start 2024-04-25T14:06:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 -- start start 2024-04-25T14:06:59.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1a9320 con 0x558515004390 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aae60 con 0x558514fe36b0 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68e7d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1a99f0 con 0x558514fdb4d0 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68dad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59392/0 (socket says 172.21.15.94:59392) 2024-04-25T14:06:59.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68d9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37484/0 (socket says 172.21.15.94:37484) 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68dad9d640 1 -- 172.21.15.94:0/3621185199 learned_addr learned my addr 172.21.15.94:0/3621185199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68da59c640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.111+0000 7f68d9d9b640 1 -- 172.21.15.94:0/3621185199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 msgr2=0x5585153ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68d9d9b640 1 -- 172.21.15.94:0/3621185199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 msgr2=0x5585153c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68d9d9b640 1 -- 172.21.15.94:0/3621185199 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585153b0e90 con 0x558514fe36b0 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68dad9d640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68da59c640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f68d001a520 tx=0x7f68d0009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c5c1f3dd69e20aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68c3fff640 1 -- 172.21.15.94:0/3621185199 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00055a0 con 0x558514fe36b0 2024-04-25T14:06:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68c3fff640 1 -- 172.21.15.94:0/3621185199 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0005740 con 0x558514fe36b0 2024-04-25T14:06:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.112+0000 7f68c3fff640 1 -- 172.21.15.94:0/3621185199 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d001a5a0 con 0x558514fe36b0 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 -- 172.21.15.94:0/3621185199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 msgr2=0x5585153aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f68d001a520 tx=0x7f68d0009c50 comp rx=0 tx=0).stop 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 -- 172.21.15.94:0/3621185199 shutdown_connections 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585153c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fe36b0 0x5585153aedf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 --2- 172.21.15.94:0/3621185199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fdb4d0 0x5585153ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 -- 172.21.15.94:0/3621185199 >> 172.21.15.94:0/3621185199 conn(0x558514ff84e0 msgr2=0x5585153ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 -- 172.21.15.94:0/3621185199 shutdown_connections 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.113+0000 7f68e7d90740 1 -- 172.21.15.94:0/3621185199 wait complete. 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 -- start start 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 Processor -- start 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fe36b0 0x5585153b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585154244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aa790 con 0x558515004390 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1aa0c0 con 0x558514fdb4d0 2024-04-25T14:06:59.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1a7300 con 0x558514fe36b0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68e7d90740 10 client.0 _ll_register_callbacks cb 0x7f68d0009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37494/0 (socket says 172.21.15.94:37494) 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68da59c640 1 -- 172.21.15.94:0/2372519417 learned_addr learned my addr 172.21.15.94:0/2372519417 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.114+0000 7f68dad9d640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585154244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fe36b0 0x5585153b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 -- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fe36b0 msgr2=0x5585153b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fe36b0 0x5585153b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 -- 172.21.15.94:0/2372519417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 msgr2=0x5585154244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585154244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 -- 172.21.15.94:0/2372519417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558515424ab0 con 0x558514fdb4d0 2024-04-25T14:06:59.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68da59c640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0303f0 tx=0x7f68cc0c65a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1377995dd47e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0e5060 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68cc004040 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.115+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d0005740 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f68cc0a6050 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 10 client.33779 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558515425d30 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0005740 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f68cc0a1ce0 con 0x558514fdb4d0 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 10 client.33779 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 10 client.33779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:06:59.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 20 client.33779 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514f983a0 0x558515428460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 4 client.33779 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558515312800 con 0x558514f983a0 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.116+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514f983a0 0x558515428460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.117+0000 7f68c1ffb640 10 client.33779 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.117+0000 7f68d9d9b640 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514f983a0 0x558515428460 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a2fc36562cdc521 server_cookie=906a8db4c552f500 in_seq=0 out_seq=0 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68c1ffb640 1 -- 172.21.15.94:0/2372519417 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558515312800 con 0x558514f983a0 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68c1ffb640 10 client.33779 handle_command_reply: tid=0 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 client.33779 shutdown 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 msgr2=0x5585153b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0303f0 tx=0x7f68cc0c65a0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 shutdown_connections 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x558514f983a0 0x558515428460 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558515004390 0x5585154244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558514fe36b0 0x5585153b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 --2- 172.21.15.94:0/2372519417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558514fdb4d0 0x5585153b5ff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68c1ffb640 0 client.33779 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 >> 172.21.15.94:0/2372519417 conn(0x558514ff84e0 msgr2=0x5585153ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 shutdown_connections 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 1 -- 172.21.15.94:0/2372519417 wait complete. 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e7d90740 20 client.33779 trim_cache size 0 max 16384 2024-04-25T14:06:59.125 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 msgr2=0x7f68c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 0x7f68c40b2080 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f68d4005fd0 tx=0x7f68d4009e10 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 msgr2=0x7f68dc199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f68cc02a860 tx=0x7f68cc032d50 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 shutdown_connections 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68c40afb70 0x7f68c40b2080 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc108920 0x7f68dc1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68dc103c20 0x7f68dc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 --2- 172.21.15.94:0/2631494165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68dc1029e0 0x7f68dc199460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.118+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 >> 172.21.15.94:0/2631494165 conn(0x7f68dc0fdbd0 msgr2=0x7f68dc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.119+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 shutdown_connections 2024-04-25T14:06:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.119+0000 7f68e0ee0640 1 -- 172.21.15.94:0/2631494165 wait complete. 2024-04-25T14:06:59.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:06:59.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 246878 18.77 MB/sec execute 581 sec latency 72.842 ms 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 -- 172.21.15.94:0/778538464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40781029e0 msgr2=0x7f40781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 --2- 172.21.15.94:0/778538464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40781029e0 0x7f40781036e0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f4068009900 tx=0x7f406802a410 comp rx=0 tx=0).stop 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 -- 172.21.15.94:0/778538464 shutdown_connections 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 --2- 172.21.15.94:0/778538464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078108920 0x7f407810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 --2- 172.21.15.94:0/778538464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4078103c20 0x7f4078104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 --2- 172.21.15.94:0/778538464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40781029e0 0x7f40781036e0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.262+0000 7f407cc27640 1 -- 172.21.15.94:0/778538464 >> 172.21.15.94:0/778538464 conn(0x7f40780fdbd0 msgr2=0x7f4078100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.263+0000 7f407cc27640 1 -- 172.21.15.94:0/778538464 shutdown_connections 2024-04-25T14:06:59.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.263+0000 7f407cc27640 1 -- 172.21.15.94:0/778538464 wait complete. 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.263+0000 7f407cc27640 1 Processor -- start 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 -- start start 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40781029e0 0x7f407810d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 0x7f407810b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f407810f610 con 0x7f4078108920 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f407810f490 con 0x7f4078103c20 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407cc27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f407810f790 con 0x7f40781029e0 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 0x7f407810b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59418/0 (socket says 172.21.15.94:59418) 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f4076d9d640 1 -- 172.21.15.94:0/3479541413 learned_addr learned my addr 172.21.15.94:0/3479541413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.264+0000 7f407659c640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40781029e0 0x7f407810d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 -- 172.21.15.94:0/3479541413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40781029e0 msgr2=0x7f407810d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40781029e0 0x7f407810d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 -- 172.21.15.94:0/3479541413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 msgr2=0x7f407810b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 0x7f407810b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 -- 172.21.15.94:0/3479541413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407810c3c0 con 0x7f4078108920 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4075d9b640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 0x7f407810b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f4070009fd0 tx=0x7f407000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f40677fe640 1 -- 172.21.15.94:0/3479541413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070017070 con 0x7f4078108920 2024-04-25T14:06:59.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40781a46a0 con 0x7f4078108920 2024-04-25T14:06:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f40677fe640 1 -- 172.21.15.94:0/3479541413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4070003e20 con 0x7f4078108920 2024-04-25T14:06:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.265+0000 7f40677fe640 1 -- 172.21.15.94:0/3479541413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070010460 con 0x7f4078108920 2024-04-25T14:06:59.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.266+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40781a4b00 con 0x7f4078108920 2024-04-25T14:06:59.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.269+0000 7f40677fe640 1 -- 172.21.15.94:0/3479541413 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4070003970 con 0x7f4078108920 2024-04-25T14:06:59.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.270+0000 7f40677fe640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 0x7f40600b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.271+0000 7f40677fe640 1 -- 172.21.15.94:0/3479541413 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f40700de8c0 con 0x7f4078108920 2024-04-25T14:06:59.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.271+0000 7f4083bb8740 1 Processor -- start 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f407659c640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 0x7f40600b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 -- start start 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 0x5604174c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560417427d80 con 0x560417121390 2024-04-25T14:06:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5604173e1a20 con 0x5604170f84d0 2024-04-25T14:06:59.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.273+0000 7f4083bb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560417423d40 con 0x5604171006b0 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f407659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 0x5604174c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f407659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 0x5604174c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59432/0 (socket says 172.21.15.94:59432) 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51152/0 (socket says 172.21.15.94:51152) 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f407659c640 1 -- 172.21.15.94:0/3552197972 learned_addr learned my addr 172.21.15.94:0/3552197972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f407659c640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 0x7f40600b20c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f406802aa00 tx=0x7f4068000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 -- 172.21.15.94:0/3552197972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 msgr2=0x5604174c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 -- 172.21.15.94:0/3552197972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 msgr2=0x5604174c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 0x5604174c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 -- 172.21.15.94:0/3552197972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604173ecc30 con 0x5604170f84d0 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.274+0000 7f4075d9b640 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174c3d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f406c009fa0 tx=0x7f406c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1daa100b195ee5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.275+0000 7f4064ff9640 1 -- 172.21.15.94:0/3552197972 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c004030 con 0x5604170f84d0 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.275+0000 7f4064ff9640 1 -- 172.21.15.94:0/3552197972 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406c0054d0 con 0x5604170f84d0 2024-04-25T14:06:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.275+0000 7f4064ff9640 1 -- 172.21.15.94:0/3552197972 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c004ce0 con 0x5604170f84d0 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 -- 172.21.15.94:0/3552197972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 msgr2=0x5604174c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174c3d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f406c009fa0 tx=0x7f406c0118d0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 -- 172.21.15.94:0/3552197972 shutdown_connections 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174c3d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3552197972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560417121390 0x5604174c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 -- 172.21.15.94:0/3552197972 >> 172.21.15.94:0/3552197972 conn(0x5604171154e0 msgr2=0x5604174bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 -- 172.21.15.94:0/3552197972 shutdown_connections 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.276+0000 7f4083bb8740 1 -- 172.21.15.94:0/3552197972 wait complete. 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 -- start start 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 Processor -- start 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f407659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f407659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59440/0 (socket says 172.21.15.94:59440) 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f407659c640 1 -- 172.21.15.94:0/4008895957 learned_addr learned my addr 172.21.15.94:0/4008895957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5604174c5530 con 0x5604170f84d0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5604173aedc0 con 0x560417121390 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5604174c56b0 con 0x5604171006b0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4083bb8740 10 client.0 _ll_register_callbacks cb 0x5604173ca600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4075d9b640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.277+0000 7f4076d9d640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 -- 172.21.15.94:0/4008895957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 msgr2=0x5604174cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 -- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 msgr2=0x5604174d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174d4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604171393f0 con 0x5604170f84d0 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f4076d9d640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f407659c640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f4068009850 tx=0x7f4068009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991a31e1291b3846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.278+0000 7f405dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.279+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068035040 con 0x5604170f84d0 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.279+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f406c0054d0 con 0x5604170f84d0 2024-04-25T14:06:59.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.279+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f40680315a0 con 0x5604170f84d0 2024-04-25T14:06:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f40680317a0 con 0x5604170f84d0 2024-04-25T14:06:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 10 client.24432 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5604172037d0 con 0x5604170f84d0 2024-04-25T14:06:59.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406c0054d0 con 0x5604170f84d0 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f406803cac0 con 0x5604170f84d0 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 10 client.24432 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 10 client.24432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 20 client.24432 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170dff60 0x5604174db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 4 client.24432 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.280+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560417435990 con 0x5604170dff60 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.281+0000 7f4075d9b640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170dff60 0x5604174db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.281+0000 7f4075d9b640 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170dff60 0x5604174db330 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bd121686e8d7613 server_cookie=9a1e40ef87fbbd5c in_seq=0 out_seq=0 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.281+0000 7f405dffb640 10 client.24432 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f405dffb640 1 -- 172.21.15.94:0/4008895957 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560417435990 con 0x5604170dff60 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f405dffb640 10 client.24432 handle_command_reply: tid=0 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 client.24432 shutdown 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 msgr2=0x5604174cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f4068009850 tx=0x7f4068009260 comp rx=0 tx=0).stop 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 shutdown_connections 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170dff60 0x5604174db330 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f405dffb640 0 client.24432 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604171006b0 0x5604174cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.282+0000 7f4083bb8740 1 --2- 172.21.15.94:0/4008895957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604170f84d0 0x5604174cd5f0 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.283+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 >> 172.21.15.94:0/4008895957 conn(0x5604171154e0 msgr2=0x5604174c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.283+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 shutdown_connections 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.283+0000 7f4083bb8740 1 -- 172.21.15.94:0/4008895957 wait complete. 2024-04-25T14:06:59.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.283+0000 7f4083bb8740 20 client.24432 trim_cache size 0 max 16384 2024-04-25T14:06:59.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.303+0000 7f4083bb8740 1 Processor -- start 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.303+0000 7f4083bb8740 1 -- start start 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 0x5604174cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59450/0 (socket says 172.21.15.94:59450) 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f407659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 0x5604174cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 -- 172.21.15.94:0/2079913307 learned_addr learned my addr 172.21.15.94:0/2079913307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40781ad760 con 0x5604171006b0 2024-04-25T14:06:59.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40781af2a0 con 0x560417121390 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40781ade30 con 0x5604170f84d0 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f407659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 0x5604174cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51168/0 (socket says 172.21.15.94:51168) 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 -- 172.21.15.94:0/2079913307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 msgr2=0x5604174cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 0x5604174cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 -- 172.21.15.94:0/2079913307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 msgr2=0x5604174c1e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174c1e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.304+0000 7f4075d9b640 1 -- 172.21.15.94:0/2079913307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604174c7dc0 con 0x5604171006b0 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.305+0000 7f4075d9b640 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f406c009d60 tx=0x7f406c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ef9ae501874ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.305+0000 7f405ffff640 1 -- 172.21.15.94:0/2079913307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c032070 con 0x5604171006b0 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.305+0000 7f405ffff640 1 -- 172.21.15.94:0/2079913307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406c01e040 con 0x5604171006b0 2024-04-25T14:06:59.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.305+0000 7f405ffff640 1 -- 172.21.15.94:0/2079913307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c01d040 con 0x5604171006b0 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 msgr2=0x5604174d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f406c009d60 tx=0x7f406c0051d0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 shutdown_connections 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560417121390 0x5604174c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d4a20 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 --2- 172.21.15.94:0/2079913307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5604170f84d0 0x5604174cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 >> 172.21.15.94:0/2079913307 conn(0x5604171154e0 msgr2=0x5604174cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 shutdown_connections 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.306+0000 7f4083bb8740 1 -- 172.21.15.94:0/2079913307 wait complete. 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.307+0000 7f4083bb8740 1 -- start start 2024-04-25T14:06:59.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.307+0000 7f4083bb8740 1 Processor -- start 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.307+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.307+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4083bb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 0x560417541690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4083bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40781aebd0 con 0x5604171006b0 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4075d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59452/0 (socket says 172.21.15.94:59452) 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4075d9b640 1 -- 172.21.15.94:0/3794086922 learned_addr learned my addr 172.21.15.94:0/3794086922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f407659c640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40781ae500 con 0x5604170f84d0 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40781ab740 con 0x560417121390 2024-04-25T14:06:59.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 0x560417541690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4083bb8740 10 client.0 _ll_register_callbacks cb 0x7f4068009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 -- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 msgr2=0x5604174d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 -- 172.21.15.94:0/3794086922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 msgr2=0x5604174d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.308+0000 7f4076d9d640 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560417541c50 con 0x560417121390 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f4076d9d640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 0x560417541690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40700dc370 tx=0x7f40700042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8711aa62154dbc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f405dffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070020040 con 0x560417121390 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f40700c1390 con 0x560417121390 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f406c01e040 con 0x560417121390 2024-04-25T14:06:59.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f40700afb60 con 0x560417121390 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f4083bb8740 10 client.33439 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560417542f20 con 0x560417121390 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.309+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406c01e040 con 0x560417121390 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f40700a5ba0 con 0x560417121390 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 10 client.33439 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 10 client.33439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 20 client.33439 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170b53a0 0x560417545920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 4 client.33439 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56041742f800 con 0x5604170b53a0 2024-04-25T14:06:59.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f407659c640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170b53a0 0x560417545920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f405dffb640 10 client.33439 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.310+0000 7f407659c640 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170b53a0 0x560417545920 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3d7eab6198853fc server_cookie=39712a0f833cd533 in_seq=0 out_seq=0 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f405dffb640 1 -- 172.21.15.94:0/3794086922 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56041742f800 con 0x5604170b53a0 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f405dffb640 10 client.33439 handle_command_reply: tid=0 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 client.33439 shutdown 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 msgr2=0x560417541690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 0x560417541690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40700dc370 tx=0x7f40700042e0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 shutdown_connections 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x5604170b53a0 0x560417545920 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560417121390 0x560417541690 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f405dffb640 0 client.33439 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604171006b0 0x5604174d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 --2- 172.21.15.94:0/3794086922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604170f84d0 0x5604174d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 >> 172.21.15.94:0/3794086922 conn(0x5604171154e0 msgr2=0x5604174cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 shutdown_connections 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 1 -- 172.21.15.94:0/3794086922 wait complete. 2024-04-25T14:06:59.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.311+0000 7f4083bb8740 20 client.33439 trim_cache size 0 max 16384 2024-04-25T14:06:59.318 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 msgr2=0x7f40600b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 0x7f40600b20c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f406802aa00 tx=0x7f4068000bc0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 msgr2=0x7f407810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f4070009fd0 tx=0x7f407000b260 comp rx=0 tx=0).stop 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 shutdown_connections 2024-04-25T14:06:59.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f40600afbb0 0x7f40600b20c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4078108920 0x7f407810bc90 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4078103c20 0x7f407810b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 --2- 172.21.15.94:0/3479541413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40781029e0 0x7f407810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 >> 172.21.15.94:0/3479541413 conn(0x7f40780fdbd0 msgr2=0x7f4078104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 shutdown_connections 2024-04-25T14:06:59.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.312+0000 7f407cc27640 1 -- 172.21.15.94:0/3479541413 wait complete. 2024-04-25T14:06:59.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:06:59.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 -- 172.21.15.94:0/1552770349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 msgr2=0x7efd08109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 --2- 172.21.15.94:0/1552770349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd08109370 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009900 tx=0x7efcfc02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:59.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 -- 172.21.15.94:0/1552770349 shutdown_connections 2024-04-25T14:06:59.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 --2- 172.21.15.94:0/1552770349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 0x7efd0810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 --2- 172.21.15.94:0/1552770349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd08109370 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 --2- 172.21.15.94:0/1552770349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd08106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 -- 172.21.15.94:0/1552770349 >> 172.21.15.94:0/1552770349 conn(0x7efd080fdbf0 msgr2=0x7efd081000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 -- 172.21.15.94:0/1552770349 shutdown_connections 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.510+0000 7efd0d825640 1 -- 172.21.15.94:0/1552770349 wait complete. 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 Processor -- start 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 -- start start 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd0819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 0x7efd081a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810f630 con 0x7efd08106f50 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810f4b0 con 0x7efd081045f0 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd0d825640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810f7b0 con 0x7efd081098b0 2024-04-25T14:06:59.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59470/0 (socket says 172.21.15.94:59470) 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd06ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd0819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd067fc640 1 -- 172.21.15.94:0/3257645416 learned_addr learned my addr 172.21.15.94:0/3257645416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd077fe640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 0x7efd081a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd067fc640 1 -- 172.21.15.94:0/3257645416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 msgr2=0x7efd081a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.511+0000 7efd067fc640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 0x7efd081a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd067fc640 1 -- 172.21.15.94:0/3257645416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 msgr2=0x7efd0819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd067fc640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd0819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd067fc640 1 -- 172.21.15.94:0/3257645416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd081a1d80 con 0x7efd08106f50 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd0819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd067fc640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006010 tx=0x7efcfc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd0c823640 1 -- 172.21.15.94:0/3257645416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc03f030 con 0x7efd08106f50 2024-04-25T14:06:59.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081a2010 con 0x7efd08106f50 2024-04-25T14:06:59.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd0c823640 1 -- 172.21.15.94:0/3257645416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc003aa0 con 0x7efd08106f50 2024-04-25T14:06:59.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd0c823640 1 -- 172.21.15.94:0/3257645416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc035070 con 0x7efd08106f50 2024-04-25T14:06:59.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.512+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a2550 con 0x7efd08106f50 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 Processor -- start 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd0c823640 1 -- 172.21.15.94:0/3257645416 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efcfc042030 con 0x7efd08106f50 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 -- start start 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e976d4d0 0x55f2e9b38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9a9cd80 con 0x55f2e976d4d0 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9a56a20 con 0x55f2e9796390 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd147aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9a98d40 con 0x55f2e97756b0 2024-04-25T14:06:59.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37584/0 (socket says 172.21.15.94:37584) 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd077fe640 1 -- 172.21.15.94:0/245815155 learned_addr learned my addr 172.21.15.94:0/245815155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd06ffd640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e976d4d0 0x55f2e9b38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.515+0000 7efd067fc640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 -- 172.21.15.94:0/245815155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 msgr2=0x55f2e9b39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 -- 172.21.15.94:0/245815155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e976d4d0 msgr2=0x55f2e9b38d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e976d4d0 0x55f2e9b38d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 -- 172.21.15.94:0/245815155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e9a61c30 con 0x55f2e9796390 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd0c823640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 0x7efcf00b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd067fc640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd077fe640 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd0000ae40 tx=0x7efd0000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82edf8e69f2a37f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd0c823640 1 -- 172.21.15.94:0/3257645416 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efcfc0ff230 con 0x7efd08106f50 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 0x7efcf00b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efcedffb640 1 -- 172.21.15.94:0/245815155 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00010840 con 0x55f2e9796390 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efcedffb640 1 -- 172.21.15.94:0/245815155 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd00018430 con 0x55f2e9796390 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.516+0000 7efcedffb640 1 -- 172.21.15.94:0/245815155 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00017640 con 0x55f2e9796390 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 -- 172.21.15.94:0/245815155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 msgr2=0x55f2e9b38810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd0000ae40 tx=0x7efd0000f790 comp rx=0 tx=0).stop 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 0x7efcf00b6410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55f2e9b38530 tx=0x7efcf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 -- 172.21.15.94:0/245815155 shutdown_connections 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e976d4d0 0x55f2e9b38d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 --2- 172.21.15.94:0/245815155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b38810 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.517+0000 7efd147aa740 1 -- 172.21.15.94:0/245815155 >> 172.21.15.94:0/245815155 conn(0x55f2e978a4e0 msgr2=0x55f2e9b34020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- 172.21.15.94:0/245815155 shutdown_connections 2024-04-25T14:06:59.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- 172.21.15.94:0/245815155 wait complete. 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- start start 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 Processor -- start 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e976d4d0 0x55f2e9b42680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b42bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9b3a530 con 0x55f2e97756b0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9a23dc0 con 0x55f2e9796390 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.518+0000 7efd147aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2e9b3a6b0 con 0x55f2e976d4d0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd147aa740 10 client.0 _ll_register_callbacks cb 0x55f2e9a3f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b42bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd06ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e976d4d0 0x55f2e9b42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37596/0 (socket says 172.21.15.94:37596) 2024-04-25T14:06:59.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b42bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59488/0 (socket says 172.21.15.94:59488) 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 -- 172.21.15.94:0/3684484519 learned_addr learned my addr 172.21.15.94:0/3684484519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 -- 172.21.15.94:0/3684484519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e976d4d0 msgr2=0x55f2e9b42680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e976d4d0 0x55f2e9b42680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 -- 172.21.15.94:0/3684484519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 msgr2=0x55f2e9b42bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b42bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 -- 172.21.15.94:0/3684484519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e97ae3f0 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efd077fe640 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd00018f00 tx=0x7efd00018f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa08cc2d470a2043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.519+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00026070 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd00018430 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efd00003d80 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efd00032050 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 10 client.33806 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2e98787d0 con 0x55f2e9796390 2024-04-25T14:06:59.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd000044f0 con 0x55f2e9796390 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efd00010bb0 con 0x55f2e9796390 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 10 client.33806 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 10 client.33806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 20 client.33806 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e9754f60 0x55f2e9b50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 4 client.33806 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:59.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.520+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd00018430 con 0x55f2e9754f60 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.521+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e9754f60 0x55f2e9b50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.521+0000 7efce6ffd640 10 client.33806 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.521+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e9754f60 0x55f2e9b50670 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf9ebaa23d772b2 server_cookie=47e05d3a969132bc in_seq=0 out_seq=0 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efce6ffd640 1 -- 172.21.15.94:0/3684484519 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7efd00018430 con 0x55f2e9754f60 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efce6ffd640 10 client.33806 handle_command_reply: tid=0 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 client.33806 shutdown 2024-04-25T14:06:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 msgr2=0x55f2e9b49ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd00018f00 tx=0x7efd00018f30 comp rx=0 tx=0).stop 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 shutdown_connections 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e9754f60 0x55f2e9b50670 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e9796390 0x55f2e9b49ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b42bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 --2- 172.21.15.94:0/3684484519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e976d4d0 0x55f2e9b42680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efce6ffd640 0 client.33806 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 >> 172.21.15.94:0/3684484519 conn(0x55f2e978a4e0 msgr2=0x55f2e9b36f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 shutdown_connections 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 1 -- 172.21.15.94:0/3684484519 wait complete. 2024-04-25T14:06:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.523+0000 7efd147aa740 20 client.33806 trim_cache size 0 max 16384 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 Processor -- start 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 -- start start 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b38e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a9370 con 0x55f2e9796390 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd081aaeb0 con 0x55f2e976d4d0 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd147aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a9a40 con 0x55f2e97756b0 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59496/0 (socket says 172.21.15.94:59496) 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd077fe640 1 -- 172.21.15.94:0/3892034444 learned_addr learned my addr 172.21.15.94:0/3892034444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.539+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 -- 172.21.15.94:0/3892034444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 msgr2=0x55f2e9b38e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b38e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 -- 172.21.15.94:0/3892034444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 msgr2=0x55f2e9b34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b34160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 -- 172.21.15.94:0/3892034444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2e9b3dde0 con 0x55f2e9796390 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efd077fe640 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7efd00005bc0 tx=0x7efd0000a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b4464384149ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efced7fa640 1 -- 172.21.15.94:0/3892034444 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0003b070 con 0x55f2e9796390 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efced7fa640 1 -- 172.21.15.94:0/3892034444 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd00018430 con 0x55f2e9796390 2024-04-25T14:06:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.540+0000 7efced7fa640 1 -- 172.21.15.94:0/3892034444 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00026070 con 0x55f2e9796390 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- 172.21.15.94:0/3892034444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 msgr2=0x55f2e9b499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7efd00005bc0 tx=0x7efd0000a930 comp rx=0 tx=0).stop 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- 172.21.15.94:0/3892034444 shutdown_connections 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e9796390 0x55f2e9b499d0 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e97756b0 0x55f2e9b38e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 --2- 172.21.15.94:0/3892034444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- 172.21.15.94:0/3892034444 >> 172.21.15.94:0/3892034444 conn(0x55f2e978a4e0 msgr2=0x55f2e9b33a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- 172.21.15.94:0/3892034444 shutdown_connections 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- 172.21.15.94:0/3892034444 wait complete. 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.541+0000 7efd147aa740 1 -- start start 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 Processor -- start 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 0x55f2e9b469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd081aa7e0 con 0x55f2e97756b0 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd081aa110 con 0x55f2e976d4d0 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a7350 con 0x55f2e9796390 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd067fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59506/0 (socket says 172.21.15.94:59506) 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd147aa740 10 client.0 _ll_register_callbacks cb 0x7efcf8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:59.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd067fc640 1 -- 172.21.15.94:0/3835343608 learned_addr learned my addr 172.21.15.94:0/3835343608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.542+0000 7efd077fe640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 0x55f2e9b469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 -- 172.21.15.94:0/3835343608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 msgr2=0x55f2e9b469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 0x55f2e9b469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 -- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 msgr2=0x55f2e9b49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 -- 172.21.15.94:0/3835343608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2e9b49f10 con 0x55f2e97756b0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd077fe640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 0x55f2e9b469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd067fc640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0fcc90 tx=0x7efcfc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfe335fabc5581e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc049040 con 0x55f2e97756b0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7efcfc0b00d0 con 0x55f2e97756b0 2024-04-25T14:06:59.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd00018430 con 0x55f2e97756b0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efcfc0c6e90 con 0x55f2e97756b0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd147aa740 10 client.24456 fetch_fsmap learned FSMap version 39 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.543+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2e9bb7a70 con 0x55f2e97756b0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd00018430 con 0x55f2e97756b0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7efcfc0c7980 con 0x55f2e97756b0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 10 client.24456 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 10 client.24456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 20 client.24456 populate_metadata read hostname 'smithi094' 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e972a3a0 0x55f2e9bba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 4 client.24456 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2e9aaa990 con 0x55f2e972a3a0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.544+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e972a3a0 0x55f2e9bba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:59.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.545+0000 7efce6ffd640 10 client.24456 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:59.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.545+0000 7efd06ffd640 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e972a3a0 0x55f2e9bba460 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89832400db5743e3 server_cookie=38629809404d8c62 in_seq=0 out_seq=0 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.548+0000 7efce6ffd640 1 -- 172.21.15.94:0/3835343608 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2e9aaa990 con 0x55f2e972a3a0 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.548+0000 7efce6ffd640 10 client.24456 handle_command_reply: tid=0 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.548+0000 7efd147aa740 1 client.24456 shutdown 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 msgr2=0x55f2e9b46480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0fcc90 tx=0x7efcfc003e80 comp rx=0 tx=0).stop 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 shutdown_connections 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55f2e972a3a0 0x55f2e9bba460 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2e9796390 0x55f2e9b469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efce6ffd640 0 client.24456 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:06:59.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f2e97756b0 0x55f2e9b46480 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 --2- 172.21.15.94:0/3835343608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2e976d4d0 0x55f2e9b49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 >> 172.21.15.94:0/3835343608 conn(0x55f2e978a4e0 msgr2=0x55f2e9b42510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 shutdown_connections 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 1 -- 172.21.15.94:0/3835343608 wait complete. 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd147aa740 20 client.24456 trim_cache size 0 max 16384 2024-04-25T14:06:59.551 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 msgr2=0x7efcf00b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 0x7efcf00b6410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55f2e9b38530 tx=0x7efcf8005f90 comp rx=0 tx=0).stop 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 msgr2=0x7efd0819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006010 tx=0x7efcfc031a50 comp rx=0 tx=0).stop 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 shutdown_connections 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7efcf00b3f00 0x7efcf00b6410 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd081098b0 0x7efd081a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd08106f50 0x7efd0819a780 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 --2- 172.21.15.94:0/3257645416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd081045f0 0x7efd0819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:59.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.549+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 >> 172.21.15.94:0/3257645416 conn(0x7efd080fdbf0 msgr2=0x7efd081051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:59.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.550+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 shutdown_connections 2024-04-25T14:06:59.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:06:59.550+0000 7efd0d825640 1 -- 172.21.15.94:0/3257645416 wait complete. 2024-04-25T14:06:59.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:06:59 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/414658624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:59.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:06:59 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/414658624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:59.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:06:59 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/414658624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:00.165 INFO:tasks.workunit.client.0.smithi094.stdout: 10 247159 18.76 MB/sec execute 582 sec latency 92.093 ms 2024-04-25T14:07:00.562 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:07:00.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:00.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:07:00.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:00 smithi137 ceph-mon[32568]: pgmap v1107: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-25T14:07:00.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 -- 172.21.15.94:0/2312001684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8102240 msgr2=0x7f1cc810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/2312001684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8102240 0x7f1cc810bca0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009ca0 tx=0x7f1cc002a8b0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 -- 172.21.15.94:0/2312001684 shutdown_connections 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/2312001684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8102240 0x7f1cc810bca0 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/2312001684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8101740 0x7f1cc8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/2312001684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8105090 0x7f1cc81054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 -- 172.21.15.94:0/2312001684 >> 172.21.15.94:0/2312001684 conn(0x7f1cc80fd340 msgr2=0x7f1cc80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 -- 172.21.15.94:0/2312001684 shutdown_connections 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.698+0000 7f1ccdba8640 1 -- 172.21.15.94:0/2312001684 wait complete. 2024-04-25T14:07:00.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 Processor -- start 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 -- start start 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8101740 0x7f1cc819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 0x7f1cc819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810f600 con 0x7f1cc8105090 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810f480 con 0x7f1cc8102240 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.699+0000 7f1ccdba8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc810f780 con 0x7f1cc8101740 2024-04-25T14:07:00.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 0x7f1cc819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59526/0 (socket says 172.21.15.94:59526) 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc7fff640 1 -- 172.21.15.94:0/3756573106 learned_addr learned my addr 172.21.15.94:0/3756573106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8101740 0x7f1cc819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/3756573106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8101740 msgr2=0x7f1cc819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8101740 0x7f1cc819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 msgr2=0x7f1cc81a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/3756573106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc81027c0 con 0x7f1cc8102240 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 0x7f1cc819a6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9f0 tx=0x7f1cbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.700+0000 7f1cc4ff9640 1 -- 172.21.15.94:0/3756573106 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00ccc0 con 0x7f1cc8102240 2024-04-25T14:07:00.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.701+0000 7f1cc4ff9640 1 -- 172.21.15.94:0/3756573106 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cbc0108e0 con 0x7f1cc8102240 2024-04-25T14:07:00.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.701+0000 7f1cc4ff9640 1 -- 172.21.15.94:0/3756573106 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc018880 con 0x7f1cc8102240 2024-04-25T14:07:00.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.701+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc8102ab0 con 0x7f1cc8102240 2024-04-25T14:07:00.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.701+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc8103000 con 0x7f1cc8102240 2024-04-25T14:07:00.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 Processor -- start 2024-04-25T14:07:00.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 -- start start 2024-04-25T14:07:00.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc4ff9640 1 -- 172.21.15.94:0/3756573106 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1cbc018a20 con 0x7f1cc8102240 2024-04-25T14:07:00.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x563237079a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237080920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563236fdeb00 con 0x563236cd8410 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563236f7a920 con 0x563236cb7730 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563236fdaac0 con 0x563236caf550 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59538/0 (socket says 172.21.15.94:59538) 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc7fff640 1 -- 172.21.15.94:0/2472901210 learned_addr learned my addr 172.21.15.94:0/2472901210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x563237079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.704+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237080920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc4ff9640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 0x7f1cb00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 -- 172.21.15.94:0/2472901210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 msgr2=0x563237079a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x563237079a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 -- 172.21.15.94:0/2472901210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 msgr2=0x563237080920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237080920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 -- 172.21.15.94:0/2472901210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563236db87e0 con 0x563236cd8410 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc4ff9640 1 -- 172.21.15.94:0/3756573106 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1cbc0deff0 con 0x7f1cc8102240 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0030db0 tx=0x7f1cc0030ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260ba50df4d48a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 0x7f1cb00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cadffb640 1 -- 172.21.15.94:0/2472901210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003970 con 0x563236cd8410 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cadffb640 1 -- 172.21.15.94:0/2472901210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc0036070 con 0x563236cd8410 2024-04-25T14:07:00.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.705+0000 7f1cadffb640 1 -- 172.21.15.94:0/2472901210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00466b0 con 0x563236cd8410 2024-04-25T14:07:00.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.706+0000 7f1cd4a54740 1 -- 172.21.15.94:0/2472901210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 msgr2=0x5632370794a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.706+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0030db0 tx=0x7f1cc0030ff0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.706+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 0x7f1cb00b2000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00306c0 tx=0x7f1cc00469b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.706+0000 7f1cd4a54740 1 -- 172.21.15.94:0/2472901210 shutdown_connections 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237080920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x563237079a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/2472901210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x5632370794a0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 -- 172.21.15.94:0/2472901210 >> 172.21.15.94:0/2472901210 conn(0x563236ccc560 msgr2=0x563237074e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 -- 172.21.15.94:0/2472901210 shutdown_connections 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 -- 172.21.15.94:0/2472901210 wait complete. 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.707+0000 7f1cd4a54740 1 -- start start 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 Processor -- start 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x5632370831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x56323708a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x5632370831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37670/0 (socket says 172.21.15.94:37670) 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x5632370831d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51266/0 (socket says 172.21.15.94:51266) 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/10534680 learned_addr learned my addr 172.21.15.94:0/10534680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563237081310 con 0x563236cd8410 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563236f785d0 con 0x563236cb7730 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563237081490 con 0x563236caf550 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.708+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x56323708a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/10534680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 msgr2=0x5632370831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x5632370831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/10534680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 msgr2=0x56323708a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x56323708a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563236e18f70 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0dca50 tx=0x7f1cbc009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2521b8aa7d3f4a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cd4a54740 10 client.0 _ll_register_callbacks cb 0x563236f66080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc0036070 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/10534680 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0a6e30 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/10534680 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1cbc0e10b0 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.709+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/10534680 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc0a6c50 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 10 client.33824 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=39}) -- 0x563236cb0170 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/10534680 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1cbc0ac470 con 0x563236cb7730 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 10 client.33824 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 10 client.33824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 20 client.33824 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c971f0 0x563237090a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 4 client.33824 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:00.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1cc0036070 con 0x563236c971f0 2024-04-25T14:07:00.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.710+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c971f0 0x563237090a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.711+0000 7f1ca6ffd640 10 client.33824 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.711+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c971f0 0x563237090a30 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b0cc17f18bf2f3 server_cookie=5ecd7e2c0ef8a635 in_seq=0 out_seq=0 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/10534680 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1cc0036070 con 0x563236c971f0 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1ca6ffd640 10 client.33824 handle_command_reply: tid=0 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 client.33824 shutdown 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 msgr2=0x563237083710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0dca50 tx=0x7f1cbc009700 comp rx=0 tx=0).stop 2024-04-25T14:07:00.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 shutdown_connections 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c971f0 0x563237090a30 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cd8410 0x56323708a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237083710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/10534680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236caf550 0x5632370831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 >> 172.21.15.94:0/10534680 conn(0x563236ccc560 msgr2=0x563237073ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1ca6ffd640 0 client.33824 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 shutdown_connections 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 1 -- 172.21.15.94:0/10534680 wait complete. 2024-04-25T14:07:00.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.717+0000 7f1cd4a54740 20 client.33824 trim_cache size 0 max 16384 2024-04-25T14:07:00.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:00 smithi094 ceph-mon[29195]: pgmap v1107: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-25T14:07:00.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 Processor -- start 2024-04-25T14:07:00.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 -- start start 2024-04-25T14:07:00.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 0x563237081ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x56323708a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ad640 con 0x563236cb7730 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81af180 con 0x563236caf550 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81add10 con 0x563236cd8410 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 0x563237081ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59554/0 (socket says 172.21.15.94:59554) 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.740+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/1068577044 learned_addr learned my addr 172.21.15.94:0/1068577044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x56323708a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 -- 172.21.15.94:0/1068577044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 msgr2=0x56323708a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x56323708a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 -- 172.21.15.94:0/1068577044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 msgr2=0x563237073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 -- 172.21.15.94:0/1068577044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632370815d0 con 0x563236caf550 2024-04-25T14:07:00.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:00.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 0x563237081ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x563237091160 tx=0x7f1cb8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d338a4b50ac0286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cad7fa640 1 -- 172.21.15.94:0/1068577044 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8004230 con 0x563236caf550 2024-04-25T14:07:00.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cad7fa640 1 -- 172.21.15.94:0/1068577044 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb80043d0 con 0x563236caf550 2024-04-25T14:07:00.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.741+0000 7f1cad7fa640 1 -- 172.21.15.94:0/1068577044 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8003ba0 con 0x563236caf550 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1068577044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 msgr2=0x563237081ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 0x563237081ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x563237091160 tx=0x7f1cb8005f90 comp rx=0 tx=0).stop 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1068577044 shutdown_connections 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x56323708a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236cb7730 0x563237073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1068577044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236caf550 0x563237081ec0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1068577044 >> 172.21.15.94:0/1068577044 conn(0x563236ccc560 msgr2=0x5632370734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1068577044 shutdown_connections 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.742+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1068577044 wait complete. 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 -- start start 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 Processor -- start 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 0x563237088150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x5632370895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81aeab0 con 0x563236caf550 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ae3e0 con 0x563236cb7730 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.743+0000 7f1cd4a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ab620 con 0x563236cd8410 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cd4a54740 10 client.0 _ll_register_callbacks cb 0x7f1cbc0a2c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 0x563237088150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37686/0 (socket says 172.21.15.94:37686) 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/1860216645 learned_addr learned my addr 172.21.15.94:0/1860216645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc7fff640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x5632370895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/1860216645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 msgr2=0x5632370895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x5632370895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/1860216645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 msgr2=0x563237088150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 0x563237088150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 -- 172.21.15.94:0/1860216645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323707e0e0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 0x563237088150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cc6ffd640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9c0 tx=0x7f1cbc0dc920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9225f09129141e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0b0290 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f1cbc0e12c0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb80043d0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc0a74e0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.744+0000 7f1cd4a54740 10 client.33830 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56323708ace0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb80043d0 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f1cbc0a6740 con 0x563236cb7730 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 10 client.33830 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 10 client.33830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 20 client.33830 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c6c630 0x5632370fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 4 client.33830 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563236fec710 con 0x563236c6c630 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.745+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c6c630 0x5632370fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.746+0000 7f1ca6ffd640 10 client.33830 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.746+0000 7f1cc77fe640 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c6c630 0x5632370fb120 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0434f7e85260bc5 server_cookie=bcf5ceb4e7d72e56 in_seq=0 out_seq=0 2024-04-25T14:07:00.751 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:00 smithi138 ceph-mon[32977]: pgmap v1107: 97 pgs: 97 active+clean; 280 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.750+0000 7f1ca6ffd640 1 -- 172.21.15.94:0/1860216645 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563236fec710 con 0x563236c6c630 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.750+0000 7f1ca6ffd640 10 client.33830 handle_command_reply: tid=0 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.750+0000 7f1cd4a54740 1 client.33830 shutdown 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 msgr2=0x563237088690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9c0 tx=0x7f1cbc0dc920 comp rx=0 tx=0).stop 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 shutdown_connections 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563236c6c630 0x5632370fb120 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563236cd8410 0x5632370895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563236cb7730 0x563237088690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ca6ffd640 0 client.33830 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 --2- 172.21.15.94:0/1860216645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563236caf550 0x563237088150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 >> 172.21.15.94:0/1860216645 conn(0x563236ccc560 msgr2=0x5632370734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 shutdown_connections 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 1 -- 172.21.15.94:0/1860216645 wait complete. 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1cd4a54740 20 client.33830 trim_cache size 0 max 16384 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "8ee9313f-2691-4f69-a190-f8c0c427d308", 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-04-25T14:07:00.753 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 msgr2=0x7f1cb00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 0x7f1cb00b2000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00306c0 tx=0x7f1cc00469b0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 msgr2=0x7f1cc819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 0x7f1cc819a6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00e9f0 tx=0x7f1cbc00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 shutdown_connections 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f1cb00afaf0 0x7f1cb00b2000 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cc8105090 0x7f1cc81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc8102240 0x7f1cc819a6e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 --2- 172.21.15.94:0/3756573106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8101740 0x7f1cc819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 >> 172.21.15.94:0/3756573106 conn(0x7f1cc80fd340 msgr2=0x7f1cc8107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 shutdown_connections 2024-04-25T14:07:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.751+0000 7f1ccdba8640 1 -- 172.21.15.94:0/3756573106 wait complete. 2024-04-25T14:07:00.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:00.762 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 -- 172.21.15.94:0/2440718404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8304103c20 msgr2=0x7f8304104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 --2- 172.21.15.94:0/2440718404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8304103c20 0x7f8304104030 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009900 tx=0x7f82f402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 -- 172.21.15.94:0/2440718404 shutdown_connections 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 --2- 172.21.15.94:0/2440718404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304108920 0x7f830410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 --2- 172.21.15.94:0/2440718404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8304103c20 0x7f8304104030 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 --2- 172.21.15.94:0/2440718404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83041029e0 0x7f83041036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 -- 172.21.15.94:0/2440718404 >> 172.21.15.94:0/2440718404 conn(0x7f83040fdbd0 msgr2=0x7f8304100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 -- 172.21.15.94:0/2440718404 shutdown_connections 2024-04-25T14:07:00.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.898+0000 7f830ba70640 1 -- 172.21.15.94:0/2440718404 wait complete. 2024-04-25T14:07:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 Processor -- start 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 -- start start 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 0x7f830419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8304108920 0x7f83041a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e500 con 0x7f83041029e0 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e380 con 0x7f8304103c20 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830ba70640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e680 con 0x7f8304108920 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830980c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830980c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59576/0 (socket says 172.21.15.94:59576) 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830900b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 0x7f830419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830980c640 1 -- 172.21.15.94:0/3667230859 learned_addr learned my addr 172.21.15.94:0/3667230859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.899+0000 7f830a00d640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8304108920 0x7f83041a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 -- 172.21.15.94:0/3667230859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8304108920 msgr2=0x7f83041a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8304108920 0x7f83041a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 -- 172.21.15.94:0/3667230859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 msgr2=0x7f830419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 0x7f830419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 -- 172.21.15.94:0/3667230859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83041a2080 con 0x7f83041029e0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830900b640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 0x7f830419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830980c640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f830000e9f0 tx=0x7f830000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f82faffd640 1 -- 172.21.15.94:0/3667230859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000ccc0 con 0x7f83041029e0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f82faffd640 1 -- 172.21.15.94:0/3667230859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000035f0 con 0x7f83041029e0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f82faffd640 1 -- 172.21.15.94:0/3667230859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000ce60 con 0x7f83041029e0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83041a2370 con 0x7f83041029e0 2024-04-25T14:07:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.900+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83041a28c0 con 0x7f83041029e0 2024-04-25T14:07:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 Processor -- start 2024-04-25T14:07:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 -- start start 2024-04-25T14:07:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99ee34d0 0x564b9a2adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99eeb6b0 0x564b9a2aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b9a212d80 con 0x564b99f0c390 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b9a1cc8b0 con 0x564b99eeb6b0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f83129fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b9a20ed40 con 0x564b99ee34d0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f82faffd640 1 -- 172.21.15.94:0/3667230859 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8300003aa0 con 0x7f83041029e0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f830980c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99ee34d0 0x564b9a2adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59580/0 (socket says 172.21.15.94:59580) 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f830a00d640 1 -- 172.21.15.94:0/2489667043 learned_addr learned my addr 172.21.15.94:0/2489667043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.904+0000 7f830900b640 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99eeb6b0 0x564b9a2aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f830a00d640 1 -- 172.21.15.94:0/2489667043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99ee34d0 msgr2=0x564b9a2adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f830a00d640 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99ee34d0 0x564b9a2adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f830a00d640 1 -- 172.21.15.94:0/2489667043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99eeb6b0 msgr2=0x564b9a2aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f830a00d640 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99eeb6b0 0x564b9a2aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f830a00d640 1 -- 172.21.15.94:0/2489667043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9a19da40 con 0x564b99f0c390 2024-04-25T14:07:00.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.905+0000 7f82faffd640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 0x7f82f00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f82faffd640 1 -- 172.21.15.94:0/3667230859 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f83000def50 con 0x7f83041029e0 2024-04-25T14:07:00.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f830a00d640 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009fd0 tx=0x7f82fc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee237c47fede8b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f830980c640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 0x7f82f00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f82effff640 1 -- 172.21.15.94:0/2489667043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc004d20 con 0x564b99f0c390 2024-04-25T14:07:00.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f82effff640 1 -- 172.21.15.94:0/2489667043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc004050 con 0x564b99f0c390 2024-04-25T14:07:00.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.906+0000 7f82effff640 1 -- 172.21.15.94:0/2489667043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc005280 con 0x564b99f0c390 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.907+0000 7f830980c640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 0x7f82f00b21d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f83000a33b0 tx=0x7f8300003a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.907+0000 7f83129fd740 1 -- 172.21.15.94:0/2489667043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 msgr2=0x564b9a2ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.907+0000 7f83129fd740 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009fd0 tx=0x7f82fc0118d0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 -- 172.21.15.94:0/2489667043 shutdown_connections 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99eeb6b0 0x564b9a2aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99ee34d0 0x564b9a2adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 --2- 172.21.15.94:0/2489667043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99f0c390 0x564b9a2ada60 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 -- 172.21.15.94:0/2489667043 >> 172.21.15.94:0/2489667043 conn(0x564b99f004e0 msgr2=0x564b9a2a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 -- 172.21.15.94:0/2489667043 shutdown_connections 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 -- 172.21.15.94:0/2489667043 wait complete. 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.908+0000 7f83129fd740 1 -- start start 2024-04-25T14:07:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 Processor -- start 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 0x564b9a2b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99f0c390 0x564b9a2beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b9a2af780 con 0x564b99ee34d0 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b9a199d10 con 0x564b99f0c390 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f83129fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b9a2af900 con 0x564b99eeb6b0 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 0x564b9a2b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 0x564b9a2b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59590/0 (socket says 172.21.15.94:59590) 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f830a00d640 1 -- 172.21.15.94:0/1301152048 learned_addr learned my addr 172.21.15.94:0/1301152048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.909+0000 7f830900b640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f83129fd740 10 client.0 _ll_register_callbacks cb 0x564b9a1c6210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830a00d640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99f0c390 0x564b9a2beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 -- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99f0c390 msgr2=0x564b9a2beb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99f0c390 0x564b9a2beb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 -- 172.21.15.94:0/1301152048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 msgr2=0x564b9a2b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 0x564b9a2b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 -- 172.21.15.94:0/1301152048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b99fee7d0 con 0x564b99eeb6b0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f830900b640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b7c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005ab0 tx=0x7f82f40317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e093576c5066e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f82ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4035050 con 0x564b99eeb6b0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc004050 con 0x564b99eeb6b0 2024-04-25T14:07:00.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.910+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f82f4004250 con 0x564b99eeb6b0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82f4004470 con 0x564b99eeb6b0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 10 client.33478 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b99f243f0 con 0x564b99eeb6b0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc004050 con 0x564b99eeb6b0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f82f4043070 con 0x564b99eeb6b0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 10 client.33478 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 10 client.33478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 20 client.33478 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99ecaf60 0x564b9a2c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 4 client.33478 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b9a220990 con 0x564b99ecaf60 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.911+0000 7f830a00d640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99ecaf60 0x564b9a2c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.912+0000 7f82ed7fa640 10 client.33478 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.912+0000 7f830a00d640 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99ecaf60 0x564b9a2c4fc0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf708993ecf15257 server_cookie=8470874bddc5951 in_seq=0 out_seq=0 2024-04-25T14:07:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.915+0000 7f82ed7fa640 1 -- 172.21.15.94:0/1301152048 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b9a220990 con 0x564b99ecaf60 2024-04-25T14:07:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.915+0000 7f82ed7fa640 10 client.33478 handle_command_reply: tid=0 2024-04-25T14:07:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.915+0000 7f83129fd740 1 client.33478 shutdown 2024-04-25T14:07:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 msgr2=0x564b9a2b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b7c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005ab0 tx=0x7f82f40317d0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 shutdown_connections 2024-04-25T14:07:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99ecaf60 0x564b9a2c4fc0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99f0c390 0x564b9a2beb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f82ed7fa640 0 client.33478 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b7c00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 --2- 172.21.15.94:0/1301152048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ee34d0 0x564b9a2b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.916+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 >> 172.21.15.94:0/1301152048 conn(0x564b99f004e0 msgr2=0x564b9a2a83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.917+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 shutdown_connections 2024-04-25T14:07:00.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.917+0000 7f83129fd740 1 -- 172.21.15.94:0/1301152048 wait complete. 2024-04-25T14:07:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.919+0000 7f83129fd740 20 client.33478 trim_cache size 0 max 16384 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 Processor -- start 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 -- start start 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83041a9330 con 0x564b99ecaf60 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83041aae70 con 0x564b99ee34d0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.945+0000 7f83129fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83041a9a00 con 0x564b99eeb6b0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59592/0 (socket says 172.21.15.94:59592) 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 -- 172.21.15.94:0/1927826400 learned_addr learned my addr 172.21.15.94:0/1927826400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830900b640 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 -- 172.21.15.94:0/1927826400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 msgr2=0x564b9a2b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 -- 172.21.15.94:0/1927826400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 msgr2=0x564b9a2bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 -- 172.21.15.94:0/1927826400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b99ee40f0 con 0x564b99ecaf60 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f830a00d640 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f82fc014040 tx=0x7f82fc004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2891cd55d638c820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f82ef7fe640 1 -- 172.21.15.94:0/1927826400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc009b30 con 0x564b99ecaf60 2024-04-25T14:07:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f82ef7fe640 1 -- 172.21.15.94:0/1927826400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc004e20 con 0x564b99ecaf60 2024-04-25T14:07:00.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.946+0000 7f82ef7fe640 1 -- 172.21.15.94:0/1927826400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc005850 con 0x564b99ecaf60 2024-04-25T14:07:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- 172.21.15.94:0/1927826400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 msgr2=0x564b9a2b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f82fc014040 tx=0x7f82fc004900 comp rx=0 tx=0).stop 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- 172.21.15.94:0/1927826400 shutdown_connections 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 --2- 172.21.15.94:0/1927826400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b1770 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- 172.21.15.94:0/1927826400 >> 172.21.15.94:0/1927826400 conn(0x564b99f004e0 msgr2=0x564b9a2ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- 172.21.15.94:0/1927826400 shutdown_connections 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- 172.21.15.94:0/1927826400 wait complete. 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 -- start start 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.948+0000 7f83129fd740 1 Processor -- start 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83041aa7a0 con 0x564b99ecaf60 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83041aa0d0 con 0x564b99ee34d0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83041a7310 con 0x564b99eeb6b0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f83129fd740 10 client.0 _ll_register_callbacks cb 0x7f8300005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f830a00d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59608/0 (socket says 172.21.15.94:59608) 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f830a00d640 1 -- 172.21.15.94:0/3284826420 learned_addr learned my addr 172.21.15.94:0/3284826420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f830900b640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.949+0000 7f830a00d640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 -- 172.21.15.94:0/3284826420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 msgr2=0x564b9a2bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 -- 172.21.15.94:0/3284826420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 msgr2=0x564b9a2b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 -- 172.21.15.94:0/3284826420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9a1d34b0 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830a00d640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f830900b640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2ba1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f82f402a930 tx=0x7f82f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e10421c02c9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4052030 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f82f403b430 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc004e20 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82f4035040 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f83129fd740 10 client.33848 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b9a2be7d0 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82f4003aa0 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.950+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f82f4043070 con 0x564b99ee34d0 2024-04-25T14:07:00.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 10 client.33848 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 10 client.33848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 20 client.33848 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99f0c390 0x564b9a337db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 4 client.33848 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f82fc004e20 con 0x564b99f0c390 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f830a00d640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99f0c390 0x564b9a337db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:00.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f82ed7fa640 10 client.33848 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.951+0000 7f830a00d640 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99f0c390 0x564b9a337db0 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ad25a526541286 server_cookie=2fc8ee3f21ffcf86 in_seq=0 out_seq=0 2024-04-25T14:07:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f82ed7fa640 1 -- 172.21.15.94:0/3284826420 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f82fc004e20 con 0x564b99f0c390 2024-04-25T14:07:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f82ed7fa640 10 client.33848 handle_command_reply: tid=0 2024-04-25T14:07:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 client.33848 shutdown 2024-04-25T14:07:00.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 msgr2=0x564b9a2ba1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2ba1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f82f402a930 tx=0x7f82f4000bc0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 shutdown_connections 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x564b99f0c390 0x564b9a337db0 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b99eeb6b0 0x564b9a2bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b99ee34d0 0x564b9a2ba1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 --2- 172.21.15.94:0/3284826420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b99ecaf60 0x564b9a2b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 >> 172.21.15.94:0/3284826420 conn(0x564b99f004e0 msgr2=0x564b9a2ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f82ed7fa640 0 client.33848 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 shutdown_connections 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 1 -- 172.21.15.94:0/3284826420 wait complete. 2024-04-25T14:07:00.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.952+0000 7f83129fd740 20 client.33848 trim_cache size 0 max 16384 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (432 inodes in the stack)", 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "8ee9313f-2691-4f69-a190-f8c0c427d308": { 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "8ee9313f-2691-4f69-a190-f8c0c427d308", 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:00.957 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 msgr2=0x7f82f00b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 0x7f82f00b21d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f83000a33b0 tx=0x7f8300003a10 comp rx=0 tx=0).stop 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 msgr2=0x7f830419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f830000e9f0 tx=0x7f830000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 shutdown_connections 2024-04-25T14:07:00.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82f00afcc0 0x7f82f00b21d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8304108920 0x7f83041a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8304103c20 0x7f830419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 --2- 172.21.15.94:0/3667230859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83041029e0 0x7f830419a540 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 >> 172.21.15.94:0/3667230859 conn(0x7f83040fdbd0 msgr2=0x7f8304109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 shutdown_connections 2024-04-25T14:07:00.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:00.953+0000 7f830ba70640 1 -- 172.21.15.94:0/3667230859 wait complete. 2024-04-25T14:07:00.967 INFO:tasks.cephfs.filesystem:scrub status for tag:8ee9313f-2691-4f69-a190-f8c0c427d308 - {'path': '/', 'tag': '8ee9313f-2691-4f69-a190-f8c0c427d308', 'options': 'recursive,force'} 2024-04-25T14:07:01.163 INFO:tasks.workunit.client.0.smithi094.stdout: 10 247461 18.77 MB/sec execute 583 sec latency 106.926 ms 2024-04-25T14:07:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub queued for path: / 2024-04-25T14:07:01.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:01.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:01 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub queued for path: / 2024-04-25T14:07:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:01.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:01.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:01.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:01 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub queued for path: / 2024-04-25T14:07:01.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:01.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:01 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 247756 18.76 MB/sec execute 584 sec latency 225.439 ms 2024-04-25T14:07:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: pgmap v1108: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-04-25T14:07:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: active paths [/] 2024-04-25T14:07:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.726 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:02 smithi094 ceph-mon[29195]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: pgmap v1108: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: active paths [/] 2024-04-25T14:07:02.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.748 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:02 smithi138 ceph-mon[32977]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 -- 172.21.15.94:0/1445189488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8108960 msgr2=0x7fe7c810ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 --2- 172.21.15.94:0/1445189488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8108960 0x7fe7c810ad80 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009ca0 tx=0x7fe7c002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 -- 172.21.15.94:0/1445189488 shutdown_connections 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 --2- 172.21.15.94:0/1445189488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8108960 0x7fe7c810ad80 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 --2- 172.21.15.94:0/1445189488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 0x7fe7c8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 --2- 172.21.15.94:0/1445189488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8102a20 0x7fe7c8103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 -- 172.21.15.94:0/1445189488 >> 172.21.15.94:0/1445189488 conn(0x7fe7c80fdc10 msgr2=0x7fe7c81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 -- 172.21.15.94:0/1445189488 shutdown_connections 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.835+0000 7fe7cce14640 1 -- 172.21.15.94:0/1445189488 wait complete. 2024-04-25T14:07:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 Processor -- start 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 -- start start 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 0x7fe7c819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8108960 0x7fe7c81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810e530 con 0x7fe7c8102a20 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810e3b0 con 0x7fe7c8103c60 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7cce14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c810e6b0 con 0x7fe7c8108960 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59626/0 (socket says 172.21.15.94:59626) 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7c659c640 1 -- 172.21.15.94:0/2484771480 learned_addr learned my addr 172.21.15.94:0/2484771480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8108960 0x7fe7c81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.836+0000 7fe7bffff640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 0x7fe7c819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 -- 172.21.15.94:0/2484771480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8108960 msgr2=0x7fe7c81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8108960 0x7fe7c81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 -- 172.21.15.94:0/2484771480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 msgr2=0x7fe7c819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 0x7fe7c819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 -- 172.21.15.94:0/2484771480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c81a1cd0 con 0x7fe7c8102a20 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7c659c640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9f0 tx=0x7fe7b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7bf7fe640 1 -- 172.21.15.94:0/2484771480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ccc0 con 0x7fe7c8102a20 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7bf7fe640 1 -- 172.21.15.94:0/2484771480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b80035f0 con 0x7fe7c8102a20 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81a1fc0 con 0x7fe7c8102a20 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7bf7fe640 1 -- 172.21.15.94:0/2484771480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ce60 con 0x7fe7c8102a20 2024-04-25T14:07:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.837+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a2480 con 0x7fe7c8102a20 2024-04-25T14:07:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.839+0000 7fe7d3d9c740 1 Processor -- start 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 -- start start 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a363300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 0x56267a36a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56267a2c4460 con 0x562679f952f0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56267a27e590 con 0x562679fbe1b0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56267a2c0420 con 0x562679f9d4d0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7bf7fe640 1 -- 172.21.15.94:0/2484771480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fe7b8010690 con 0x7fe7c8102a20 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 0x56267a36a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37758/0 (socket says 172.21.15.94:37758) 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a363300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 0x56267a36a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:51384/0 (socket says 172.21.15.94:51384) 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/3150030008 learned_addr learned my addr 172.21.15.94:0/3150030008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/3150030008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 msgr2=0x56267a36a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 0x56267a36a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/3150030008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 msgr2=0x56267a363300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a363300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.840+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/3150030008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267a28b0d0 con 0x562679fbe1b0 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7bf7fe640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 0x7fe7b40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0035000 tx=0x7fe7c002f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b2131dbd454842 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7c659c640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 0x7fe7b40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7bcff9640 1 -- 172.21.15.94:0/3150030008 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0031650 con 0x562679fbe1b0 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7bf7fe640 1 -- 172.21.15.94:0/2484771480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b80def30 con 0x7fe7c8102a20 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7bcff9640 1 -- 172.21.15.94:0/3150030008 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0036030 con 0x562679fbe1b0 2024-04-25T14:07:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.841+0000 7fe7bcff9640 1 -- 172.21.15.94:0/3150030008 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00047a0 con 0x562679fbe1b0 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3150030008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 msgr2=0x56267a362d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0035000 tx=0x7fe7c002f410 comp rx=0 tx=0).stop 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7c659c640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 0x7fe7b40b2070 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x56267a364160 tx=0x7fe7b80e2a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3150030008 shutdown_connections 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f9d4d0 0x56267a36a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a363300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3150030008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679fbe1b0 0x56267a362d80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3150030008 >> 172.21.15.94:0/3150030008 conn(0x562679fb2300 msgr2=0x56267a35a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3150030008 shutdown_connections 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3150030008 wait complete. 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 -- start start 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.842+0000 7fe7d3d9c740 1 Processor -- start 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a36c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 0x56267a373d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56267a36aa70 con 0x562679fbe1b0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56267a24bb60 con 0x562679f9d4d0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56267a36abf0 con 0x562679f952f0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 10 client.0 _ll_register_callbacks cb 0x56267a2702b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 0x56267a373d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37760/0 (socket says 172.21.15.94:37760) 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 0x56267a373d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59644/0 (socket says 172.21.15.94:59644) 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 -- 172.21.15.94:0/3356108655 learned_addr learned my addr 172.21.15.94:0/3356108655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a36c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 -- 172.21.15.94:0/3356108655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 msgr2=0x56267a36c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a36c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 -- 172.21.15.94:0/3356108655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 msgr2=0x56267a373d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 0x56267a373d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 -- 172.21.15.94:0/3356108655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562679fd6210 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7bffff640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0012720 tx=0x7fe7b0012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129507d3221590b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0004030 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0036030 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.843+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b00041d0 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7b0004410 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 10 client.33860 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56267a0a2620 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0036030 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b0003a80 con 0x562679f9d4d0 2024-04-25T14:07:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 10 client.33860 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 10 client.33860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 20 client.33860 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679f7cd80 0x56267a37a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 4 client.33860 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56267a2d2070 con 0x562679f7cd80 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.844+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679f7cd80 0x56267a37a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.845+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679f7cd80 0x56267a37a680 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc97982956b3704 server_cookie=f54038b441988f61 in_seq=0 out_seq=0 2024-04-25T14:07:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.845+0000 7fe7adffb640 10 client.33860 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7adffb640 1 -- 172.21.15.94:0/3356108655 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56267a2d2070 con 0x562679f7cd80 2024-04-25T14:07:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7adffb640 10 client.33860 handle_command_reply: tid=0 2024-04-25T14:07:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 client.33860 shutdown 2024-04-25T14:07:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 msgr2=0x56267a36cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0012720 tx=0x7fe7b0012be0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 shutdown_connections 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679f7cd80 0x56267a37a680 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679fbe1b0 0x56267a373d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7adffb640 0 client.33860 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a36cdf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/3356108655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a36c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 >> 172.21.15.94:0/3356108655 conn(0x562679fb2300 msgr2=0x56267a359250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 shutdown_connections 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/3356108655 wait complete. 2024-04-25T14:07:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.849+0000 7fe7d3d9c740 20 client.33860 trim_cache size 0 max 16384 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 Processor -- start 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 -- start start 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a373c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a35d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81a91d0 con 0x562679f7cd80 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81aad10 con 0x562679f9d4d0 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81a98a0 con 0x562679f952f0 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59648/0 (socket says 172.21.15.94:59648) 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/2729786883 learned_addr learned my addr 172.21.15.94:0/2729786883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7bffff640 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a373c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a35d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/2729786883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 msgr2=0x56267a373c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a373c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/2729786883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 msgr2=0x56267a35d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a35d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 -- 172.21.15.94:0/2729786883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267a3741b0 con 0x562679f7cd80 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.864+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002f480 tx=0x7fe7c002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f74bea8f574745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7affff640 1 -- 172.21.15.94:0/2729786883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c005b030 con 0x562679f7cd80 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7affff640 1 -- 172.21.15.94:0/2729786883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0036040 con 0x562679f7cd80 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7affff640 1 -- 172.21.15.94:0/2729786883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c004a040 con 0x562679f7cd80 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/2729786883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 msgr2=0x56267a35a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002f480 tx=0x7fe7c002f3a0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/2729786883 shutdown_connections 2024-04-25T14:07:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a35d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f952f0 0x56267a373c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/2729786883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f7cd80 0x56267a35a2b0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/2729786883 >> 172.21.15.94:0/2729786883 conn(0x562679fb2300 msgr2=0x56267a359ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/2729786883 shutdown_connections 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.865+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/2729786883 wait complete. 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 -- start start 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 Processor -- start 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f7cd80 0x56267a3710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a3724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81aa640 con 0x562679f952f0 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81a9f70 con 0x562679f9d4d0 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81a71b0 con 0x562679f7cd80 2024-04-25T14:07:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7d3d9c740 10 client.0 _ll_register_callbacks cb 0x7fe7b0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59654/0 (socket says 172.21.15.94:59654) 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 -- 172.21.15.94:0/4120829223 learned_addr learned my addr 172.21.15.94:0/4120829223 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a3724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f7cd80 0x56267a3710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 -- 172.21.15.94:0/4120829223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f7cd80 msgr2=0x56267a3710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f7cd80 0x56267a3710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 -- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 msgr2=0x56267a3724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a3724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.866+0000 7fe7bffff640 1 -- 172.21.15.94:0/4120829223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267a372ab0 con 0x562679f952f0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7bffff640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009f80 tx=0x7fe7b000f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cfe59d52cb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b001c530 con 0x562679f952f0 2024-04-25T14:07:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fe7b001c6d0 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0036040 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7b001c9d0 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 10 client.24510 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56267a373db0 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0036040 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b00035f0 con 0x562679f952f0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 10 client.24510 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 10 client.24510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 20 client.24510 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679fbe1b0 0x56267a3e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 4 client.24510 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.867+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56267a2cbee0 con 0x562679fbe1b0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.868+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679fbe1b0 0x56267a3e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.868+0000 7fe7adffb640 10 client.24510 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:02.868+0000 7fe7c6d9d640 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679fbe1b0 0x56267a3e5070 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c42239b835842ae server_cookie=8fd325319f1373e1 in_seq=0 out_seq=0 2024-04-25T14:07:02.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: pgmap v1108: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 525 op/s 2024-04-25T14:07:02.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: active paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:02.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:02 smithi137 ceph-mon[32568]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:03.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 248003 18.74 MB/sec execute 585 sec latency 148.711 ms 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7adffb640 1 -- 172.21.15.94:0/4120829223 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56267a2cbee0 con 0x562679fbe1b0 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7adffb640 10 client.24510 handle_command_reply: tid=0 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 client.24510 shutdown 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 msgr2=0x56267a371610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009f80 tx=0x7fe7b000f820 comp rx=0 tx=0).stop 2024-04-25T14:07:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 shutdown_connections 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x562679fbe1b0 0x56267a3e5070 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7adffb640 0 client.24510 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562679f9d4d0 0x56267a3724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562679f952f0 0x56267a371610 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 --2- 172.21.15.94:0/4120829223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562679f7cd80 0x56267a3710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 >> 172.21.15.94:0/4120829223 conn(0x562679fb2300 msgr2=0x56267a358820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.557+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 shutdown_connections 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7d3d9c740 1 -- 172.21.15.94:0/4120829223 wait complete. 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7d3d9c740 20 client.24510 trim_cache size 0 max 16384 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:07:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:07:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:07:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 msgr2=0x7fe7b40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 0x7fe7b40b2070 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x56267a364160 tx=0x7fe7b80e2a10 comp rx=0 tx=0).stop 2024-04-25T14:07:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 msgr2=0x7fe7c819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.558+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9f0 tx=0x7fe7b800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 shutdown_connections 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fe7b40afb60 0x7fe7b40b2070 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c8108960 0x7fe7c81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c8103c60 0x7fe7c819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 --2- 172.21.15.94:0/2484771480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7c8102a20 0x7fe7c819a190 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 >> 172.21.15.94:0/2484771480 conn(0x7fe7c80fdc10 msgr2=0x7fe7c8109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 shutdown_connections 2024-04-25T14:07:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:03.559+0000 7fe7cce14640 1 -- 172.21.15.94:0/2484771480 wait complete. 2024-04-25T14:07:04.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 248251 18.72 MB/sec execute 586 sec latency 475.129 ms 2024-04-25T14:07:04.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:04 smithi094 ceph-mon[29195]: pgmap v1109: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-04-25T14:07:04.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:04 smithi138 ceph-mon[32977]: pgmap v1109: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-04-25T14:07:04.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:04 smithi137 ceph-mon[32568]: pgmap v1109: 97 pgs: 97 active+clean; 285 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-04-25T14:07:05.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 248708 18.75 MB/sec execute 587 sec latency 82.971 ms 2024-04-25T14:07:06.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 249192 18.75 MB/sec execute 588 sec latency 56.736 ms 2024-04-25T14:07:06.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:06 smithi094 ceph-mon[29195]: pgmap v1110: 97 pgs: 97 active+clean; 279 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-04-25T14:07:06.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:06 smithi138 ceph-mon[32977]: pgmap v1110: 97 pgs: 97 active+clean; 279 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-04-25T14:07:06.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:06 smithi137 ceph-mon[32568]: pgmap v1110: 97 pgs: 97 active+clean; 279 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-04-25T14:07:07.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 249469 18.74 MB/sec execute 589 sec latency 300.469 ms 2024-04-25T14:07:08.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 249803 18.75 MB/sec execute 590 sec latency 82.815 ms 2024-04-25T14:07:08.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:08 smithi094 ceph-mon[29195]: pgmap v1111: 97 pgs: 97 active+clean; 274 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-04-25T14:07:08.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:08 smithi138 ceph-mon[32977]: pgmap v1111: 97 pgs: 97 active+clean; 274 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-04-25T14:07:08.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:08 smithi137 ceph-mon[32568]: pgmap v1111: 97 pgs: 97 active+clean; 274 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 508 op/s 2024-04-25T14:07:09.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 250197 18.76 MB/sec execute 591 sec latency 70.687 ms 2024-04-25T14:07:10.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 250555 18.76 MB/sec execute 592 sec latency 72.401 ms 2024-04-25T14:07:10.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:10 smithi094 ceph-mon[29195]: pgmap v1112: 97 pgs: 97 active+clean; 275 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 502 op/s 2024-04-25T14:07:10.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:10 smithi138 ceph-mon[32977]: pgmap v1112: 97 pgs: 97 active+clean; 275 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 502 op/s 2024-04-25T14:07:10.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:10 smithi137 ceph-mon[32568]: pgmap v1112: 97 pgs: 97 active+clean; 275 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 502 op/s 2024-04-25T14:07:11.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 250780 18.75 MB/sec execute 593 sec latency 81.841 ms 2024-04-25T14:07:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:07:11.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:11 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:07:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:07:11.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:11 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:07:11.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:07:11.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:11 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:07:12.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 251004 18.74 MB/sec execute 594 sec latency 199.859 ms 2024-04-25T14:07:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:12 smithi094 ceph-mon[29195]: pgmap v1113: 97 pgs: 97 active+clean; 280 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-04-25T14:07:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:07:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:07:12.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:12 smithi094 ceph-mon[29195]: scrub summary: idle 2024-04-25T14:07:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:12 smithi138 ceph-mon[32977]: pgmap v1113: 97 pgs: 97 active+clean; 280 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-04-25T14:07:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:07:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:07:12.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:12 smithi138 ceph-mon[32977]: scrub summary: idle 2024-04-25T14:07:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:12 smithi137 ceph-mon[32568]: pgmap v1113: 97 pgs: 97 active+clean; 280 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.7 MiB/s wr, 599 op/s 2024-04-25T14:07:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:07:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:07:12.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:12 smithi137 ceph-mon[32568]: scrub summary: idle 2024-04-25T14:07:13.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 251368 18.74 MB/sec execute 595 sec latency 144.503 ms 2024-04-25T14:07:14.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 251713 18.73 MB/sec execute 596 sec latency 80.314 ms 2024-04-25T14:07:14.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:14 smithi094 ceph-mon[29195]: pgmap v1114: 97 pgs: 97 active+clean; 279 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-04-25T14:07:14.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:14 smithi138 ceph-mon[32977]: pgmap v1114: 97 pgs: 97 active+clean; 279 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-04-25T14:07:14.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:14 smithi137 ceph-mon[32568]: pgmap v1114: 97 pgs: 97 active+clean; 279 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-04-25T14:07:15.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 252163 18.74 MB/sec execute 597 sec latency 103.282 ms 2024-04-25T14:07:16.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 252563 18.75 MB/sec execute 598 sec latency 105.516 ms 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 -- 172.21.15.94:0/3185711880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618102a30 msgr2=0x7f9618103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 --2- 172.21.15.94:0/3185711880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618102a30 0x7f9618103730 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f9614009900 tx=0x7f961402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 -- 172.21.15.94:0/3185711880 shutdown_connections 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 --2- 172.21.15.94:0/3185711880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618108970 0x7f961810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 --2- 172.21.15.94:0/3185711880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618103c70 0x7f9618104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 --2- 172.21.15.94:0/3185711880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618102a30 0x7f9618103730 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 -- 172.21.15.94:0/3185711880 >> 172.21.15.94:0/3185711880 conn(0x7f96180fdc00 msgr2=0x7f96181000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 -- 172.21.15.94:0/3185711880 shutdown_connections 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.698+0000 7f961f8d8640 1 -- 172.21.15.94:0/3185711880 wait complete. 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 Processor -- start 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 -- start start 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618102a30 0x7f9618199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 0x7f96181a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f961810e530 con 0x7f9618103c70 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f961810e3b0 con 0x7f9618108970 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961f8d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f961810e6b0 con 0x7f9618102a30 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33206/0 (socket says 172.21.15.94:33206) 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961de75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 0x7f96181a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 -- 172.21.15.94:0/1652182190 learned_addr learned my addr 172.21.15.94:0/1652182190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961d674640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618102a30 0x7f9618199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 -- 172.21.15.94:0/1652182190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618102a30 msgr2=0x7f9618199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618102a30 0x7f9618199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 -- 172.21.15.94:0/1652182190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 msgr2=0x7f96181a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 0x7f96181a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961ce73640 1 -- 172.21.15.94:0/1652182190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96181a0cb0 con 0x7f9618103c70 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.699+0000 7f961de75640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 0x7f96181a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f961ce73640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f960800ea60 tx=0x7f960800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f960e7fc640 1 -- 172.21.15.94:0/1652182190 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960800cde0 con 0x7f9618103c70 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181a0fa0 con 0x7f9618103c70 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f960e7fc640 1 -- 172.21.15.94:0/1652182190 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96080035f0 con 0x7f9618103c70 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f960e7fc640 1 -- 172.21.15.94:0/1652182190 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608010690 con 0x7f9618103c70 2024-04-25T14:07:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.700+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181a1410 con 0x7f9618103c70 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.702+0000 7f960e7fc640 1 -- 172.21.15.94:0/1652182190 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f96080037a0 con 0x7f9618103c70 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 Processor -- start 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 -- start start 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a416fffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a416f63460 con 0x55a416c3c4d0 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a416f1d590 con 0x55a416c342f0 2024-04-25T14:07:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f9626788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a416f5f420 con 0x55a416c5d1b0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961ce73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961d674640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961d674640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35956/0 (socket says 172.21.15.94:35956) 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961ce73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33218/0 (socket says 172.21.15.94:33218) 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961d674640 1 -- 172.21.15.94:0/3752953030 learned_addr learned my addr 172.21.15.94:0/3752953030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f961de75640 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a416fffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.703+0000 7f960e7fc640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 0x7f96040b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 -- 172.21.15.94:0/3752953030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 msgr2=0x55a416fffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961de75640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 0x7f96040b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a416fffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 -- 172.21.15.94:0/3752953030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 msgr2=0x55a417007000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 -- 172.21.15.94:0/3752953030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a416f2a0d0 con 0x55a416c342f0 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f960e7fc640 1 -- 172.21.15.94:0/1652182190 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f96080def50 con 0x7f9618103c70 2024-04-25T14:07:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961ce73640 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961d674640 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9614009230 tx=0x7f9614009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626352cf44178a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f961de75640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 0x7f96040b1ff0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55a417000d50 tx=0x7f9610004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f96037fe640 1 -- 172.21.15.94:0/3752953030 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614035030 con 0x55a416c342f0 2024-04-25T14:07:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f96037fe640 1 -- 172.21.15.94:0/3752953030 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614040070 con 0x55a416c342f0 2024-04-25T14:07:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.704+0000 7f96037fe640 1 -- 172.21.15.94:0/3752953030 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614003950 con 0x55a416c342f0 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 -- 172.21.15.94:0/3752953030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 msgr2=0x55a417000100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9614009230 tx=0x7f9614009620 comp rx=0 tx=0).stop 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 -- 172.21.15.94:0/3752953030 shutdown_connections 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a417007000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c342f0 0x55a417000100 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 --2- 172.21.15.94:0/3752953030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a416fffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 -- 172.21.15.94:0/3752953030 >> 172.21.15.94:0/3752953030 conn(0x55a416c51300 msgr2=0x55a416ff9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 -- 172.21.15.94:0/3752953030 shutdown_connections 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.705+0000 7f9626788740 1 -- 172.21.15.94:0/3752953030 wait complete. 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 -- start start 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 Processor -- start 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a417009d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a417010cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a4170079f0 con 0x55a416c342f0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a416eeab60 con 0x55a416c3c4d0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a417007b70 con 0x55a416c5d1b0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f9626788740 10 client.0 _ll_register_callbacks cb 0x55a416f0f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f961d674640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f961d674640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33226/0 (socket says 172.21.15.94:33226) 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f961d674640 1 -- 172.21.15.94:0/2861367616 learned_addr learned my addr 172.21.15.94:0/2861367616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.706+0000 7f961ce73640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a417009d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961de75640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a417010cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 -- 172.21.15.94:0/2861367616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 msgr2=0x55a417010cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a417010cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 -- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 msgr2=0x55a417009d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a417009d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 -- 172.21.15.94:0/2861367616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a416c75210 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961ce73640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a417009d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f961d674640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f9614031500 tx=0x7f961402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2931995c4a0a042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614035070 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f96140036d0 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9614040070 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f96140038d0 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 10 client.24522 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a416d41620 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614040070 con 0x55a416c342f0 2024-04-25T14:07:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f9614045920 con 0x55a416c342f0 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 10 client.24522 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 10 client.24522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 20 client.24522 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.707+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c1bd80 0x55a417017120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.708+0000 7f9626788740 4 client.24522 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.708+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a416f71070 con 0x55a416c1bd80 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.708+0000 7f961ce73640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c1bd80 0x55a417017120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.708+0000 7f9600ff9640 10 client.24522 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.708+0000 7f961ce73640 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c1bd80 0x55a417017120 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4041a64c2beebb server_cookie=7896d0b18864ebbf in_seq=0 out_seq=0 2024-04-25T14:07:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9600ff9640 1 -- 172.21.15.94:0/2861367616 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a416f71070 con 0x55a416c1bd80 2024-04-25T14:07:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9600ff9640 10 client.24522 handle_command_reply: tid=0 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 client.24522 shutdown 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 msgr2=0x55a417009830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f9614031500 tx=0x7f961402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 shutdown_connections 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c1bd80 0x55a417017120 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c5d1b0 0x55a417010cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9600ff9640 0 client.24522 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a417009d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 --2- 172.21.15.94:0/2861367616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c342f0 0x55a417009830 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 >> 172.21.15.94:0/2861367616 conn(0x55a416c51300 msgr2=0x55a416ffc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 shutdown_connections 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 1 -- 172.21.15.94:0/2861367616 wait complete. 2024-04-25T14:07:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.710+0000 7f9626788740 20 client.24522 trim_cache size 0 max 16384 2024-04-25T14:07:16.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:16 smithi094 ceph-mon[29195]: pgmap v1115: 97 pgs: 97 active+clean; 285 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-04-25T14:07:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.726+0000 7f9626788740 1 Processor -- start 2024-04-25T14:07:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.726+0000 7f9626788740 1 -- start start 2024-04-25T14:07:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c1bd80 0x55a416ff8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a417010bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96181a9360 con 0x55a416c3c4d0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aaea0 con 0x55a416c1bd80 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9626788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96181a9a30 con 0x55a416c342f0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33236/0 (socket says 172.21.15.94:33236) 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 -- 172.21.15.94:0/2440668942 learned_addr learned my addr 172.21.15.94:0/2440668942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961d674640 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c1bd80 0x55a416ff8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961ce73640 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a417010bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 -- 172.21.15.94:0/2440668942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 msgr2=0x55a417010bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a417010bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 -- 172.21.15.94:0/2440668942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c1bd80 msgr2=0x55a416ff8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c1bd80 0x55a416ff8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 -- 172.21.15.94:0/2440668942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a417017680 con 0x55a416c3c4d0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f961de75640 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f961000ff50 tx=0x7f9610009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866290b2c88b807c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9602ffd640 1 -- 172.21.15.94:0/2440668942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610013170 con 0x55a416c3c4d0 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9602ffd640 1 -- 172.21.15.94:0/2440668942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610026070 con 0x55a416c3c4d0 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.727+0000 7f9602ffd640 1 -- 172.21.15.94:0/2440668942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001f6a0 con 0x55a416c3c4d0 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- 172.21.15.94:0/2440668942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 msgr2=0x55a416ff9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f961000ff50 tx=0x7f9610009f90 comp rx=0 tx=0).stop 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- 172.21.15.94:0/2440668942 shutdown_connections 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c3c4d0 0x55a416ff9190 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a417010bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 --2- 172.21.15.94:0/2440668942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c1bd80 0x55a416ff8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- 172.21.15.94:0/2440668942 >> 172.21.15.94:0/2440668942 conn(0x55a416c51300 msgr2=0x55a417002d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- 172.21.15.94:0/2440668942 shutdown_connections 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- 172.21.15.94:0/2440668942 wait complete. 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.728+0000 7f9626788740 1 -- start start 2024-04-25T14:07:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 Processor -- start 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a416ffd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a416ffdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aa7d0 con 0x55a416c1bd80 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aa100 con 0x55a416c3c4d0 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96181a7340 con 0x55a416c342f0 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f9626788740 10 client.0 _ll_register_callbacks cb 0x7f96080a39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33246/0 (socket says 172.21.15.94:33246) 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 -- 172.21.15.94:0/52623869 learned_addr learned my addr 172.21.15.94:0/52623869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961de75640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a416ffdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961ce73640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a416ffd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 -- 172.21.15.94:0/52623869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 msgr2=0x55a416ffd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a416ffd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 -- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 msgr2=0x55a416ffdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a416ffdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961d674640 1 -- 172.21.15.94:0/52623869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a417011130 con 0x55a416c1bd80 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.729+0000 7f961de75640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a416ffdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f961d674640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f9614047010 tx=0x7f9614003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d74af547a9848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961405b070 con 0x55a416c1bd80 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f9614003c70 con 0x55a416c1bd80 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9610026070 con 0x55a416c1bd80 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9614067050 con 0x55a416c1bd80 2024-04-25T14:07:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 10 client.24528 fetch_fsmap learned FSMap version 39 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a417004740 con 0x55a416c1bd80 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610026070 con 0x55a416c1bd80 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f961403c560 con 0x55a416c1bd80 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 10 client.24528 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 10 client.24528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 20 client.24528 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c5d1b0 0x55a4170109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 4 client.24528 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a416f6aee0 con 0x55a416c5d1b0 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.730+0000 7f961ce73640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c5d1b0 0x55a4170109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.731+0000 7f9600ff9640 10 client.24528 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:16.731+0000 7f961ce73640 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c5d1b0 0x55a4170109d0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead7576c93ff8de3 server_cookie=ce799cdf6a711b63 in_seq=0 out_seq=0 2024-04-25T14:07:16.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:16 smithi138 ceph-mon[32977]: pgmap v1115: 97 pgs: 97 active+clean; 285 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-04-25T14:07:16.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:16 smithi137 ceph-mon[32568]: pgmap v1115: 97 pgs: 97 active+clean; 285 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-04-25T14:07:17.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 252669 18.73 MB/sec execute 599 sec latency 323.798 ms 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.556+0000 7f9600ff9640 1 -- 172.21.15.94:0/52623869 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a416f6aee0 con 0x55a416c5d1b0 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.556+0000 7f9600ff9640 10 client.24528 handle_command_reply: tid=0 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.556+0000 7f9626788740 1 client.24528 shutdown 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.556+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 msgr2=0x55a416ffefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.556+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f9614047010 tx=0x7f9614003820 comp rx=0 tx=0).stop 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 shutdown_connections 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x55a416c5d1b0 0x55a4170109d0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a416c3c4d0 0x55a416ffdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a416c342f0 0x55a416ffd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 --2- 172.21.15.94:0/52623869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a416c1bd80 0x55a416ffefe0 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9600ff9640 0 client.24528 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 >> 172.21.15.94:0/52623869 conn(0x55a416c51300 msgr2=0x55a416ffc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 shutdown_connections 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 1 -- 172.21.15.94:0/52623869 wait complete. 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.557+0000 7f9626788740 20 client.24528 trim_cache size 0 max 16384 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T14:07:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T14:07:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T14:07:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 msgr2=0x7f96040b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 0x7f96040b1ff0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55a417000d50 tx=0x7f9610004ab0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 msgr2=0x7f96181996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f960800ea60 tx=0x7f960800ef20 comp rx=0 tx=0).stop 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 shutdown_connections 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f96040afae0 0x7f96040b1ff0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9618108970 0x7f96181a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9618103c70 0x7f96181996b0 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 --2- 172.21.15.94:0/1652182190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9618102a30 0x7f9618199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 >> 172.21.15.94:0/1652182190 conn(0x7f96180fdc00 msgr2=0x7f9618104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 shutdown_connections 2024-04-25T14:07:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T14:07:17.558+0000 7f961f8d8640 1 -- 172.21.15.94:0/1652182190 wait complete. 2024-04-25T14:07:18.164 INFO:tasks.workunit.client.0.smithi094.stdout: 10 cleanup 600 sec 2024-04-25T14:07:18.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:18 smithi094 ceph-mon[29195]: pgmap v1116: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 362 op/s 2024-04-25T14:07:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:18 smithi138 ceph-mon[32977]: pgmap v1116: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 362 op/s 2024-04-25T14:07:18.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:18 smithi137 ceph-mon[32568]: pgmap v1116: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 362 op/s 2024-04-25T14:07:19.165 INFO:tasks.workunit.client.0.smithi094.stdout: 10 cleanup 601 sec 2024-04-25T14:07:19.855 INFO:tasks.workunit.client.0.smithi094.stdout: 0 cleanup 602 sec 2024-04-25T14:07:19.855 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Operation Count AvgLat MaxLat 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: ---------------------------------------- 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: NTCreateX 360262 7.019 671.634 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Close 264564 0.003 0.121 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Rename 15272 29.543 670.183 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Unlink 72803 10.671 549.959 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Qpathinfo 326791 5.422 638.049 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Qfileinfo 56984 0.001 0.030 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Qfsinfo 59883 0.149 13.215 2024-04-25T14:07:19.856 INFO:tasks.workunit.client.0.smithi094.stdout: Sfileinfo 29382 0.260 226.397 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: Find 126232 0.650 318.964 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: WriteX 178016 0.055 235.932 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: ReadX 565444 0.021 336.106 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: LockX 1176 11.675 142.773 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: UnlockX 1176 9.860 132.846 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: Flush 25262 12.724 535.165 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-04-25T14:07:19.857 INFO:tasks.workunit.client.0.smithi094.stdout:Throughput 18.7287 MB/sec 10 clients 10 procs max_latency=671.641 ms 2024-04-25T14:07:19.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:07:19.858 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:07:20.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:20 smithi094 ceph-mon[29195]: pgmap v1117: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 331 op/s 2024-04-25T14:07:20.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:20 smithi138 ceph-mon[32977]: pgmap v1117: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 331 op/s 2024-04-25T14:07:20.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:20 smithi137 ceph-mon[32568]: pgmap v1117: 97 pgs: 97 active+clean; 286 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 331 op/s 2024-04-25T14:07:20.968 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-25T14:07:20.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T14:07:21.244 DEBUG:teuthology.parallel:result is None 2024-04-25T14:07:21.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:07:21.282 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:07:21.282 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T14:07:21.296 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T14:07:21.332 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T14:07:21.341 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T14:07:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:22 smithi094 ceph-mon[29195]: pgmap v1118: 97 pgs: 97 active+clean; 267 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-04-25T14:07:22.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:22 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:22 smithi138 ceph-mon[32977]: pgmap v1118: 97 pgs: 97 active+clean; 267 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-04-25T14:07:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:22 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:22.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:22 smithi137 ceph-mon[32568]: pgmap v1118: 97 pgs: 97 active+clean; 267 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.4 MiB/s wr, 509 op/s 2024-04-25T14:07:22.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:22 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:23.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:23 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:23.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:23 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:23.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:23 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:24.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:24 smithi094 ceph-mon[29195]: pgmap v1119: 97 pgs: 97 active+clean; 252 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 456 op/s 2024-04-25T14:07:24.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:24 smithi138 ceph-mon[32977]: pgmap v1119: 97 pgs: 97 active+clean; 252 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 456 op/s 2024-04-25T14:07:24.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:24 smithi137 ceph-mon[32568]: pgmap v1119: 97 pgs: 97 active+clean; 252 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 456 op/s 2024-04-25T14:07:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:26 smithi094 ceph-mon[29195]: pgmap v1120: 97 pgs: 97 active+clean; 144 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-04-25T14:07:26.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:26 smithi094 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:26 smithi138 ceph-mon[32977]: pgmap v1120: 97 pgs: 97 active+clean; 144 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-04-25T14:07:26.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:26 smithi138 ceph-mon[32977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:26.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:26 smithi137 ceph-mon[32568]: pgmap v1120: 97 pgs: 97 active+clean; 144 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-04-25T14:07:26.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:26 smithi137 ceph-mon[32568]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T14:07:28.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:28 smithi094 ceph-mon[29195]: pgmap v1121: 97 pgs: 97 active+clean; 128 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-04-25T14:07:28.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:28 smithi138 ceph-mon[32977]: pgmap v1121: 97 pgs: 97 active+clean; 128 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-04-25T14:07:28.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:28 smithi137 ceph-mon[32568]: pgmap v1121: 97 pgs: 97 active+clean; 128 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-04-25T14:07:30.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:30 smithi094 ceph-mon[29195]: pgmap v1122: 97 pgs: 97 active+clean; 125 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T14:07:30.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:30 smithi138 ceph-mon[32977]: pgmap v1122: 97 pgs: 97 active+clean; 125 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T14:07:30.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:30 smithi137 ceph-mon[32568]: pgmap v1122: 97 pgs: 97 active+clean; 125 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-04-25T14:07:30.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:30.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:31.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 -- 172.21.15.94:0/2272583739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 msgr2=0x7f388810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 --2- 172.21.15.94:0/2272583739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f388810ad60 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ca0 tx=0x7f388002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 -- 172.21.15.94:0/2272583739 shutdown_connections 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 --2- 172.21.15.94:0/2272583739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f388810ad60 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 --2- 172.21.15.94:0/2272583739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888103c40 0x7f3888104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 --2- 172.21.15.94:0/2272583739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888102a00 0x7f3888103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 -- 172.21.15.94:0/2272583739 >> 172.21.15.94:0/2272583739 conn(0x7f38880fdbf0 msgr2=0x7f38881000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 -- 172.21.15.94:0/2272583739 shutdown_connections 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.094+0000 7f388fcf2640 1 -- 172.21.15.94:0/2272583739 wait complete. 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 Processor -- start 2024-04-25T14:07:31.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 -- start start 2024-04-25T14:07:31.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888102a00 0x7f3888199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 0x7f3888199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f388810e580 con 0x7f3888108940 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f388810e400 con 0x7f3888103c40 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388fcf2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f388810e700 con 0x7f3888102a00 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388e28f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388e28f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51968/0 (socket says 172.21.15.94:51968) 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.095+0000 7f388e28f640 1 -- 172.21.15.94:0/1645477299 learned_addr learned my addr 172.21.15.94:0/1645477299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388da8e640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888102a00 0x7f3888199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388d28d640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 0x7f3888199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 -- 172.21.15.94:0/1645477299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888102a00 msgr2=0x7f3888199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888102a00 0x7f3888199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 -- 172.21.15.94:0/1645477299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 msgr2=0x7f3888199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 0x7f3888199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 -- 172.21.15.94:0/1645477299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38881a0c60 con 0x7f3888108940 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388d28d640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 0x7f3888199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388e28f640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f38800097b0 tx=0x7f3880003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f387affd640 1 -- 172.21.15.94:0/1645477299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880004240 con 0x7f3888108940 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38881a0ef0 con 0x7f3888108940 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f387affd640 1 -- 172.21.15.94:0/1645477299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388002f380 con 0x7f3888108940 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f387affd640 1 -- 172.21.15.94:0/1645477299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388002fc30 con 0x7f3888108940 2024-04-25T14:07:31.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.096+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a13e0 con 0x7f3888108940 2024-04-25T14:07:31.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 Processor -- start 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f387affd640 1 -- 172.21.15.94:0/1645477299 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f38800043e0 con 0x7f3888108940 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 -- start start 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x55869708fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x55869708ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558696ff4d80 con 0x558696ccd6b0 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558696fae8b0 con 0x558696cc54d0 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.099+0000 7f3896ba2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558696ff0d40 con 0x558696cee390 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51972/0 (socket says 172.21.15.94:51972) 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 -- 172.21.15.94:0/2999718034 learned_addr learned my addr 172.21.15.94:0/2999718034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388da8e640 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x55869708ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f387affd640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 0x7f38740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 -- 172.21.15.94:0/2999718034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 msgr2=0x55869708fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x55869708fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 -- 172.21.15.94:0/2999718034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 msgr2=0x55869708ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x55869708ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 -- 172.21.15.94:0/2999718034 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558696f7fa40 con 0x558696ccd6b0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388e28f640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 0x7f38740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f387affd640 1 -- 172.21.15.94:0/1645477299 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f38801004b0 con 0x7f3888108940 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f388d28d640 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f388400af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b17fe967c6386e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f3873fff640 1 -- 172.21.15.94:0/2999718034 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388400cc90 con 0x558696ccd6b0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f3873fff640 1 -- 172.21.15.94:0/2999718034 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388400ce30 con 0x558696ccd6b0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.100+0000 7f3873fff640 1 -- 172.21.15.94:0/2999718034 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840107e0 con 0x558696ccd6b0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 -- 172.21.15.94:0/2999718034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 msgr2=0x558697090c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f388e28f640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 0x7f38740b2000 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f38800c5ad0 tx=0x7f38800b1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f388400af60 comp rx=0 tx=0).stop 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 -- 172.21.15.94:0/2999718034 shutdown_connections 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696ccd6b0 0x558697090c70 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x55869708ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2999718034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x55869708fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.101+0000 7f3896ba2740 1 -- 172.21.15.94:0/2999718034 >> 172.21.15.94:0/2999718034 conn(0x558696ce24e0 msgr2=0x55869708b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- 172.21.15.94:0/2999718034 shutdown_connections 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- 172.21.15.94:0/2999718034 wait complete. 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- start start 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 Processor -- start 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 0x558697099e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x5586970a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558697091780 con 0x558696cc54d0 2024-04-25T14:07:31.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558696f7bd10 con 0x558696ccd6b0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558697091900 con 0x558696cee390 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.102+0000 7f3896ba2740 10 client.0 _ll_register_callbacks cb 0x558696fa8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51980/0 (socket says 172.21.15.94:51980) 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 -- 172.21.15.94:0/2712622099 learned_addr learned my addr 172.21.15.94:0/2712622099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388da8e640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 0x558697099e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 -- 172.21.15.94:0/2712622099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 msgr2=0x5586970a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x5586970a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 -- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 msgr2=0x558697099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 0x558697099e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 -- 172.21.15.94:0/2712622099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558696dd07d0 con 0x558696cc54d0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388da8e640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 0x558697099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f388d28d640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f3884004810 tx=0x7f388400a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dd7ad9ffe46b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f38717fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884011be0 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f38840105a0 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388400ce30 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f38840107a0 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f3896ba2740 10 client.24540 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.103+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558696d063f0 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388400ce30 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f388402c070 con 0x558696cc54d0 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 10 client.24540 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 10 client.24540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 20 client.24540 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696cacf60 0x5586970a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 4 client.24540 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:31.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558697002990 con 0x558696cacf60 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.104+0000 7f388da8e640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696cacf60 0x5586970a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f38717fa640 10 client.24540 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f388da8e640 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696cacf60 0x5586970a9160 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca693bfc5c20ec9a server_cookie=b0a6516fabc25b1e in_seq=0 out_seq=0 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f38717fa640 1 -- 172.21.15.94:0/2712622099 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558697002990 con 0x558696cacf60 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f38717fa640 10 client.24540 handle_command_reply: tid=0 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f3896ba2740 1 client.24540 shutdown 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 msgr2=0x5586970998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.105+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f3884004810 tx=0x7f388400a7a0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 shutdown_connections 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696cacf60 0x5586970a9160 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696cee390 0x5586970a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696ccd6b0 0x558697099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 --2- 172.21.15.94:0/2712622099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x5586970998e0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f38717fa640 0 client.24540 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:31.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 >> 172.21.15.94:0/2712622099 conn(0x558696ce24e0 msgr2=0x55869708a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 shutdown_connections 2024-04-25T14:07:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 1 -- 172.21.15.94:0/2712622099 wait complete. 2024-04-25T14:07:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.106+0000 7f3896ba2740 20 client.24540 trim_cache size 0 max 16384 2024-04-25T14:07:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 Processor -- start 2024-04-25T14:07:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 -- start start 2024-04-25T14:07:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x558697098440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cee390 0x5586970994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.123+0000 7f3896ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38881a93c0 con 0x558696cc54d0 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f3896ba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f38881aaf60 con 0x558696cee390 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f3896ba2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38881a9ac0 con 0x558696ccd6b0 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51984/0 (socket says 172.21.15.94:51984) 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 -- 172.21.15.94:0/3285194319 learned_addr learned my addr 172.21.15.94:0/3285194319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388da8e640 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x558697098440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 -- 172.21.15.94:0/3285194319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 msgr2=0x558697098440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x558697098440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 -- 172.21.15.94:0/3285194319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cee390 msgr2=0x5586970994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cee390 0x5586970994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 -- 172.21.15.94:0/3285194319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558696cc60f0 con 0x558696cc54d0 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388da8e640 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x558697098440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f388d28d640 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f3884006de0 tx=0x7f388400a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afe32e7e188d2ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f38737fe640 1 -- 172.21.15.94:0/3285194319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884032070 con 0x558696cc54d0 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f38737fe640 1 -- 172.21.15.94:0/3285194319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3884020070 con 0x558696cc54d0 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.124+0000 7f38737fe640 1 -- 172.21.15.94:0/3285194319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884010940 con 0x558696cc54d0 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- 172.21.15.94:0/3285194319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 msgr2=0x558697097f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f3884006de0 tx=0x7f388400a350 comp rx=0 tx=0).stop 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- 172.21.15.94:0/3285194319 shutdown_connections 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cee390 0x5586970994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x558697098440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 --2- 172.21.15.94:0/3285194319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cc54d0 0x558697097f00 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- 172.21.15.94:0/3285194319 >> 172.21.15.94:0/3285194319 conn(0x558696ce24e0 msgr2=0x558697094640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- 172.21.15.94:0/3285194319 shutdown_connections 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- 172.21.15.94:0/3285194319 wait complete. 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 -- start start 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.125+0000 7f3896ba2740 1 Processor -- start 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x5586970a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x5586970a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38881aa890 con 0x558696cee390 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f38881aa1c0 con 0x558696cc54d0 2024-04-25T14:07:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38881a73a0 con 0x558696ccd6b0 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f3896ba2740 10 client.0 _ll_register_callbacks cb 0x7f388400a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51998/0 (socket says 172.21.15.94:51998) 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 -- 172.21.15.94:0/1175120107 learned_addr learned my addr 172.21.15.94:0/1175120107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388da8e640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x5586970a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x5586970a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 -- 172.21.15.94:0/1175120107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 msgr2=0x5586970a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x5586970a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 -- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 msgr2=0x5586970a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x5586970a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 -- 172.21.15.94:0/1175120107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558696fb54b0 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f388d28d640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f3884026040 tx=0x7f388401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a8a9665381bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.126+0000 7f38717fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884032050 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f3884011bf0 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884020070 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3884011df0 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 10 client.24546 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558697112a40 con 0x558696cee390 2024-04-25T14:07:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3884020070 con 0x558696cee390 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f388402c040 con 0x558696cee390 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 10 client.24546 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 10 client.24546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 20 client.24546 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696c823a0 0x558697115280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 4 client.24546 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.127+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558696ffc800 con 0x558696c823a0 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f388da8e640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696c823a0 0x558697115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f38717fa640 10 client.24546 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f388da8e640 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696c823a0 0x558697115280 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dab042f56954d61 server_cookie=8abf5e4e08958669 in_seq=0 out_seq=0 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f38717fa640 1 -- 172.21.15.94:0/1175120107 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558696ffc800 con 0x558696c823a0 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f38717fa640 10 client.24546 handle_command_reply: tid=0 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f3896ba2740 1 client.24546 shutdown 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 msgr2=0x558697111470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.128+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f3884026040 tx=0x7f388401b040 comp rx=0 tx=0).stop 2024-04-25T14:07:31.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 shutdown_connections 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x558696c823a0 0x558697115280 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558696cee390 0x558697111470 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558696ccd6b0 0x5586970a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 --2- 172.21.15.94:0/1175120107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558696cc54d0 0x5586970a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f38717fa640 0 client.24546 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 >> 172.21.15.94:0/1175120107 conn(0x558696ce24e0 msgr2=0x5586970994b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 shutdown_connections 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 1 -- 172.21.15.94:0/1175120107 wait complete. 2024-04-25T14:07:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f3896ba2740 20 client.24546 trim_cache size 0 max 16384 2024-04-25T14:07:31.140 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:31.140 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-04-25T14:07:31.140 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-04-25T14:07:31.140 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 msgr2=0x7f38740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 0x7f38740b2000 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f38800c5ad0 tx=0x7f38800b1040 comp rx=0 tx=0).stop 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 msgr2=0x7f38881a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f38800097b0 tx=0x7f3880003c20 comp rx=0 tx=0).stop 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 shutdown_connections 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f38740afaf0 0x7f38740b2000 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3888108940 0x7f38881a05f0 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3888103c40 0x7f3888199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 --2- 172.21.15.94:0/1645477299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3888102a00 0x7f3888199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 >> 172.21.15.94:0/1645477299 conn(0x7f38880fdbf0 msgr2=0x7f3888104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 shutdown_connections 2024-04-25T14:07:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.129+0000 7f388fcf2640 1 -- 172.21.15.94:0/1645477299 wait complete. 2024-04-25T14:07:31.145 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:07:31.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 -- 172.21.15.94:0/1523867453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754104d50 msgr2=0x7f5754105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 --2- 172.21.15.94:0/1523867453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754104d50 0x7f5754105160 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f5748009900 tx=0x7f574802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 -- 172.21.15.94:0/1523867453 shutdown_connections 2024-04-25T14:07:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 --2- 172.21.15.94:0/1523867453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754109a50 0x7f575410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 --2- 172.21.15.94:0/1523867453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754104d50 0x7f5754105160 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 --2- 172.21.15.94:0/1523867453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754103b10 0x7f5754104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 -- 172.21.15.94:0/1523867453 >> 172.21.15.94:0/1523867453 conn(0x7f57540fed40 msgr2=0x7f57541011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 -- 172.21.15.94:0/1523867453 shutdown_connections 2024-04-25T14:07:31.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.256+0000 7f575bce0640 1 -- 172.21.15.94:0/1523867453 wait complete. 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 Processor -- start 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 -- start start 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754103b10 0x7f575419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754104d50 0x7f575419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f575410f610 con 0x7f5754109a50 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575410f490 con 0x7f5754104d50 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575bce0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575410f790 con 0x7f5754103b10 2024-04-25T14:07:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52008/0 (socket says 172.21.15.94:52008) 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 -- 172.21.15.94:0/2839453326 learned_addr learned my addr 172.21.15.94:0/2839453326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575927b640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754104d50 0x7f575419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 -- 172.21.15.94:0/2839453326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754103b10 msgr2=0x7f575419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754103b10 0x7f575419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 -- 172.21.15.94:0/2839453326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754104d50 msgr2=0x7f575419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754104d50 0x7f575419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 -- 172.21.15.94:0/2839453326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57541a1d80 con 0x7f5754109a50 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.257+0000 7f575a27d640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f574400eb10 tx=0x7f574400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.258+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744004b20 con 0x7f5754109a50 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.258+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57541a2070 con 0x7f5754109a50 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.258+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5744004cc0 con 0x7f5754109a50 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.258+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744010670 con 0x7f5754109a50 2024-04-25T14:07:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.258+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57541a25c0 con 0x7f5754109a50 2024-04-25T14:07:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.259+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5754108860 con 0x7f5754109a50 2024-04-25T14:07:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.260+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5744005000 con 0x7f5754109a50 2024-04-25T14:07:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.261+0000 7f5752ffd640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 0x7f573c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.261+0000 7f5759a7c640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 0x7f573c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.261+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f57440df540 con 0x7f5754109a50 2024-04-25T14:07:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.262+0000 7f5759a7c640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 0x7f573c0b2000 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5754104710 tx=0x7f574c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.265+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5744053ff0 con 0x7f5754109a50 2024-04-25T14:07:31.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.559+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f575402db50 con 0x7f5754109a50 2024-04-25T14:07:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.560+0000 7f5752ffd640 1 -- 172.21.15.94:0/2839453326 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11942 (secure 0 0 0) 0x7f57440a8080 con 0x7f5754109a50 2024-04-25T14:07:31.560 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:07:31.560 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T14:07:24.561299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24502,24488,24475,24512,24557]},"id":1}]} 2024-04-25T14:07:31.561 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 42 2024-04-25T14:07:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 msgr2=0x7f573c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 0x7f573c0b2000 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5754104710 tx=0x7f574c005ed0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 msgr2=0x7f57541a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f574400eb10 tx=0x7f574400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 shutdown_connections 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f573c0afaf0 0x7f573c0b2000 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5754109a50 0x7f57541a1710 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5754104d50 0x7f575419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.563+0000 7f575bce0640 1 --2- 172.21.15.94:0/2839453326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754103b10 0x7f575419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.564+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 >> 172.21.15.94:0/2839453326 conn(0x7f57540fed40 msgr2=0x7f5754105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.564+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 shutdown_connections 2024-04-25T14:07:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.564+0000 7f575bce0640 1 -- 172.21.15.94:0/2839453326 wait complete. 2024-04-25T14:07:31.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 -- 172.21.15.94:0/3898533382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 msgr2=0x7f72c0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 --2- 172.21.15.94:0/3898533382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c0104030 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009900 tx=0x7f72a802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 -- 172.21.15.94:0/3898533382 shutdown_connections 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 --2- 172.21.15.94:0/3898533382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0108920 0x7f72c010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 --2- 172.21.15.94:0/3898533382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c0104030 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 --2- 172.21.15.94:0/3898533382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c01029e0 0x7f72c01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 -- 172.21.15.94:0/3898533382 >> 172.21.15.94:0/3898533382 conn(0x7f72c00fdbd0 msgr2=0x7f72c0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 -- 172.21.15.94:0/3898533382 shutdown_connections 2024-04-25T14:07:31.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.695+0000 7f72c6496640 1 -- 172.21.15.94:0/3898533382 wait complete. 2024-04-25T14:07:31.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 Processor -- start 2024-04-25T14:07:31.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 -- start start 2024-04-25T14:07:31.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 0x7f72c00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c0108920 0x7f72c01a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010e510 con 0x7f72c0103c20 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010e390 con 0x7f72c0108920 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.696+0000 7f72c6496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c010e690 con 0x7f72c01029e0 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52034/0 (socket says 172.21.15.94:52034) 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 -- 172.21.15.94:0/1895248825 learned_addr learned my addr 172.21.15.94:0/1895248825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72c4a33640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c0108920 0x7f72c01a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bffff640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 0x7f72c00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 -- 172.21.15.94:0/1895248825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 msgr2=0x7f72c00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 0x7f72c00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 -- 172.21.15.94:0/1895248825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c0108920 msgr2=0x7f72c01a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c0108920 0x7f72c01a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 -- 172.21.15.94:0/1895248825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c01a1ce0 con 0x7f72c0103c20 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bffff640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 0x7f72c00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f72a8006010 tx=0x7f72a8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bd7fa640 1 -- 172.21.15.94:0/1895248825 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a803f030 con 0x7f72c0103c20 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bd7fa640 1 -- 172.21.15.94:0/1895248825 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a8003cb0 con 0x7f72c0103c20 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c01a1f70 con 0x7f72c0103c20 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72bd7fa640 1 -- 172.21.15.94:0/1895248825 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8035070 con 0x7f72c0103c20 2024-04-25T14:07:31.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.697+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a23d0 con 0x7f72c0103c20 2024-04-25T14:07:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 Processor -- start 2024-04-25T14:07:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72bd7fa640 1 -- 172.21.15.94:0/1895248825 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f72a8042030 con 0x7f72c0103c20 2024-04-25T14:07:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 -- start start 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc935a4d0 0x55bcc9725d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc9726a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9689d80 con 0x55bcc9383390 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9643a20 con 0x55bcc93626b0 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72cd428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9685d40 con 0x55bcc935a4d0 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52048/0 (socket says 172.21.15.94:52048) 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 -- 172.21.15.94:0/1204000513 learned_addr learned my addr 172.21.15.94:0/1204000513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc9726a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72bffff640 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc935a4d0 0x55bcc9725d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 -- 172.21.15.94:0/1204000513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc935a4d0 msgr2=0x55bcc9725d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc935a4d0 0x55bcc9725d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 -- 172.21.15.94:0/1204000513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 msgr2=0x55bcc9726a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc9726a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72c4a33640 1 -- 172.21.15.94:0/1204000513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc964ec30 con 0x55bcc9383390 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.700+0000 7f72bd7fa640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 0x7f72ac0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72c4a33640 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f72b8010370 tx=0x7f72b8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67db0d471d1ede4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72bffff640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 0x7f72ac0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72bd7fa640 1 -- 172.21.15.94:0/1895248825 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f72a80ff600 con 0x7f72c0103c20 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72b67fc640 1 -- 172.21.15.94:0/1204000513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8016070 con 0x55bcc9383390 2024-04-25T14:07:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72b67fc640 1 -- 172.21.15.94:0/1204000513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8004d20 con 0x55bcc9383390 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.701+0000 7f72b67fc640 1 -- 172.21.15.94:0/1204000513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80053c0 con 0x55bcc9383390 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- 172.21.15.94:0/1204000513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 msgr2=0x55bcc9725810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72bffff640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 0x7f72ac0b2080 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f72b00058c0 tx=0x7f72b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f72b8010370 tx=0x7f72b8010830 comp rx=0 tx=0).stop 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- 172.21.15.94:0/1204000513 shutdown_connections 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc9726a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc935a4d0 0x55bcc9725d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 --2- 172.21.15.94:0/1204000513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc9383390 0x55bcc9725810 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- 172.21.15.94:0/1204000513 >> 172.21.15.94:0/1204000513 conn(0x55bcc93774e0 msgr2=0x55bcc9721020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- 172.21.15.94:0/1204000513 shutdown_connections 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- 172.21.15.94:0/1204000513 wait complete. 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 -- start start 2024-04-25T14:07:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 Processor -- start 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.702+0000 7f72cd428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc972fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc9736ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9727530 con 0x55bcc935a4d0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bcc9610dc0 con 0x55bcc93626b0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcc97276b0 con 0x55bcc9383390 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 10 client.0 _ll_register_callbacks cb 0x55bcc962c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52052/0 (socket says 172.21.15.94:52052) 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 -- 172.21.15.94:0/3705442459 learned_addr learned my addr 172.21.15.94:0/3705442459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc972fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72c4a33640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc9736ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 -- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 msgr2=0x55bcc9736ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc9736ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 -- 172.21.15.94:0/3705442459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 msgr2=0x55bcc972fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc972fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 -- 172.21.15.94:0/3705442459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc939b3f0 con 0x55bcc935a4d0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72c4a33640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc9736ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc972fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72bffff640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f72b0011a20 tx=0x7f72b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45537905410fd827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f729f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0022070 con 0x55bcc935a4d0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f72b0004460 con 0x55bcc935a4d0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8004d20 con 0x55bcc935a4d0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f72b002e050 con 0x55bcc935a4d0 2024-04-25T14:07:31.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 10 client.24564 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.703+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bcc94657d0 con 0x55bcc935a4d0 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8004d20 con 0x55bcc935a4d0 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f72b0003c40 con 0x55bcc935a4d0 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 10 client.24564 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 10 client.24564 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 20 client.24564 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc9341f60 0x55bcc973d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 4 client.24564 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc9697990 con 0x55bcc9341f60 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc9341f60 0x55bcc973d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f729f7fe640 10 client.24564 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.704+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc9341f60 0x55bcc973d380 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecee70608071a290 server_cookie=5ba57443a853cc43 in_seq=0 out_seq=0 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f729f7fe640 1 -- 172.21.15.94:0/3705442459 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcc9697990 con 0x55bcc9341f60 2024-04-25T14:07:31.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f729f7fe640 10 client.24564 handle_command_reply: tid=0 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 client.24564 shutdown 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 msgr2=0x55bcc972f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f72b0011a20 tx=0x7f72b00046c0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 shutdown_connections 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc9341f60 0x55bcc973d380 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc9736ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc93626b0 0x55bcc972fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f729f7fe640 0 client.24564 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 --2- 172.21.15.94:0/3705442459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc935a4d0 0x55bcc972f620 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 >> 172.21.15.94:0/3705442459 conn(0x55bcc93774e0 msgr2=0x55bcc9723f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 shutdown_connections 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.705+0000 7f72cd428740 1 -- 172.21.15.94:0/3705442459 wait complete. 2024-04-25T14:07:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.706+0000 7f72cd428740 20 client.24564 trim_cache size 0 max 16384 2024-04-25T14:07:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 Processor -- start 2024-04-25T14:07:31.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 -- start start 2024-04-25T14:07:31.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc9724530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc972dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ab4c0 con 0x55bcc93626b0 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ad060 con 0x55bcc935a4d0 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72cd428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01abbc0 con 0x55bcc9383390 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52066/0 (socket says 172.21.15.94:52066) 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 -- 172.21.15.94:0/3905443715 learned_addr learned my addr 172.21.15.94:0/3905443715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 -- 172.21.15.94:0/3905443715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 msgr2=0x55bcc972dd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc972dd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 -- 172.21.15.94:0/3905443715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 msgr2=0x55bcc9724530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc9724530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 -- 172.21.15.94:0/3905443715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc9721520 con 0x55bcc93626b0 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.719+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f72a80c3ac0 tx=0x7f72a80fcf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44384670847b075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72b5ffb640 1 -- 172.21.15.94:0/3905443715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8105040 con 0x55bcc93626b0 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72b5ffb640 1 -- 172.21.15.94:0/3905443715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a80d01e0 con 0x55bcc93626b0 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72b5ffb640 1 -- 172.21.15.94:0/3905443715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8049040 con 0x55bcc93626b0 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 -- 172.21.15.94:0/3905443715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 msgr2=0x55bcc9724a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f72a80c3ac0 tx=0x7f72a80fcf30 comp rx=0 tx=0).stop 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 -- 172.21.15.94:0/3905443715 shutdown_connections 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc972dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc9724a70 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 --2- 172.21.15.94:0/3905443715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc9724530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 -- 172.21.15.94:0/3905443715 >> 172.21.15.94:0/3905443715 conn(0x55bcc93774e0 msgr2=0x55bcc9723e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 -- 172.21.15.94:0/3905443715 shutdown_connections 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.720+0000 7f72cd428740 1 -- 172.21.15.94:0/3905443715 wait complete. 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 -- start start 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 Processor -- start 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc97a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc97a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ac990 con 0x55bcc93626b0 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01ac2c0 con 0x55bcc935a4d0 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01a94a0 con 0x55bcc9383390 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72cd428740 10 client.0 _ll_register_callbacks cb 0x7f72b000c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52076/0 (socket says 172.21.15.94:52076) 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bf7fe640 1 -- 172.21.15.94:0/591597341 learned_addr learned my addr 172.21.15.94:0/591597341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bffff640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc97a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bf7fe640 1 -- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 msgr2=0x55bcc97a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.721+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc97a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72bf7fe640 1 -- 172.21.15.94:0/591597341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 msgr2=0x55bcc97a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc97a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72bf7fe640 1 -- 172.21.15.94:0/591597341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc97a80c0 con 0x55bcc93626b0 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72bf7fe640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f72a8032af0 tx=0x7f72a8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc227d408836ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80ae060 con 0x55bcc93626b0 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f72a803bd00 con 0x55bcc93626b0 2024-04-25T14:07:31.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcc9691800 con 0x55bcc93626b0 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f72a8049070 con 0x55bcc93626b0 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 10 client.24570 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bcc97a9340 con 0x55bcc93626b0 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bcc9691800 con 0x55bcc93626b0 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f72a80c3cd0 con 0x55bcc93626b0 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 10 client.24570 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 10 client.24570 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 20 client.24570 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc93173a0 0x55bcc97abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 4 client.24570 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.722+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 --> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcc960c610 con 0x55bcc93173a0 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.723+0000 7f72bffff640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc93173a0 0x55bcc97abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f729f7fe640 10 client.24570 ms_handle_connect on v2:172.21.15.138:6836/356914816 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72bffff640 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc93173a0 0x55bcc97abce0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=279e7c588d71e705 server_cookie=bca830d3199d2658 in_seq=0 out_seq=0 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f729f7fe640 1 -- 172.21.15.94:0/591597341 <== mds.4 v2:172.21.15.138:6836/356914816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcc960c610 con 0x55bcc93173a0 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f729f7fe640 10 client.24570 handle_command_reply: tid=0 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 client.24570 shutdown 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 msgr2=0x55bcc97a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f72a8032af0 tx=0x7f72a8004530 comp rx=0 tx=0).stop 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 shutdown_connections 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:6836/356914816,v1:172.21.15.138:6837/356914816] conn(0x55bcc93173a0 0x55bcc97abce0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcc9383390 0x55bcc97a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcc93626b0 0x55bcc97a0ba0 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 --2- 172.21.15.94:0/591597341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bcc935a4d0 0x55bcc97a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f729f7fe640 0 client.24570 ms_handle_reset on v2:172.21.15.138:6836/356914816 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 >> 172.21.15.94:0/591597341 conn(0x55bcc93774e0 msgr2=0x55bcc97245b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 shutdown_connections 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 1 -- 172.21.15.94:0/591597341 wait complete. 2024-04-25T14:07:31.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.724+0000 7f72cd428740 20 client.24570 trim_cache size 0 max 16384 2024-04-25T14:07:31.733 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 msgr2=0x7f72ac0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 0x7f72ac0b2080 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f72b00058c0 tx=0x7f72b0005850 comp rx=0 tx=0).stop 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 msgr2=0x7f72c019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f72a8006010 tx=0x7f72a8004340 comp rx=0 tx=0).stop 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 shutdown_connections 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f72ac0afb70 0x7f72ac0b2080 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c0108920 0x7f72c01a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72c0103c20 0x7f72c019f0a0 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 --2- 172.21.15.94:0/1895248825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c01029e0 0x7f72c00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 >> 172.21.15.94:0/1895248825 conn(0x7f72c00fdbd0 msgr2=0x7f72c0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 shutdown_connections 2024-04-25T14:07:31.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.725+0000 7f72c6496640 1 -- 172.21.15.94:0/1895248825 wait complete. 2024-04-25T14:07:31.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T14:07:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.870+0000 7fbfd56fa640 1 -- 172.21.15.94:0/117998108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 msgr2=0x7fbfd010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.870+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/117998108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 0x7fbfd010ad40 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009ca0 tx=0x7fbfc802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.877+0000 7fbfd56fa640 1 -- 172.21.15.94:0/117998108 shutdown_connections 2024-04-25T14:07:31.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.877+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/117998108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 0x7fbfd010ad40 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.877+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/117998108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfd0103c20 0x7fbfd0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.877+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/117998108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd01029e0 0x7fbfd01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.877+0000 7fbfd56fa640 1 -- 172.21.15.94:0/117998108 >> 172.21.15.94:0/117998108 conn(0x7fbfd00fdbd0 msgr2=0x7fbfd0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.882+0000 7fbfd56fa640 1 -- 172.21.15.94:0/117998108 shutdown_connections 2024-04-25T14:07:31.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.884+0000 7fbfd56fa640 1 -- 172.21.15.94:0/117998108 wait complete. 2024-04-25T14:07:31.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.884+0000 7fbfd56fa640 1 Processor -- start 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 -- start start 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfd01029e0 0x7fbfd006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 0x7fbfd01a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd006ce20 con 0x7fbfd0108920 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd006cca0 con 0x7fbfd0103c20 2024-04-25T14:07:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfd56fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd006cfa0 con 0x7fbfd01029e0 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39918/0 (socket says 172.21.15.94:39918) 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.887+0000 7fbfc7fff640 1 -- 172.21.15.94:0/3464527709 learned_addr learned my addr 172.21.15.94:0/3464527709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 -- 172.21.15.94:0/3464527709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfd01029e0 msgr2=0x7fbfd006a920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfd01029e0 0x7fbfd006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 -- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 msgr2=0x7fbfd01a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 0x7fbfd01a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 -- 172.21.15.94:0/3464527709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd01a8660 con 0x7fbfd0103c20 2024-04-25T14:07:31.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.888+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e9f0 tx=0x7fbfb800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.892+0000 7fbfccff9640 1 -- 172.21.15.94:0/3464527709 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800cd80 con 0x7fbfd0103c20 2024-04-25T14:07:31.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.892+0000 7fbfccff9640 1 -- 172.21.15.94:0/3464527709 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80054d0 con 0x7fbfd0103c20 2024-04-25T14:07:31.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.893+0000 7fbfccff9640 1 -- 172.21.15.94:0/3464527709 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8010430 con 0x7fbfd0103c20 2024-04-25T14:07:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.893+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd01a8950 con 0x7fbfd0103c20 2024-04-25T14:07:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.893+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01a8e20 con 0x7fbfd0103c20 2024-04-25T14:07:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.902+0000 7fbfccff9640 1 -- 172.21.15.94:0/3464527709 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fbfb8004ee0 con 0x7fbfd0103c20 2024-04-25T14:07:31.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.904+0000 7fbfccff9640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 0x7fbfbc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.905+0000 7fbfccff9640 1 -- 172.21.15.94:0/3464527709 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbfb80df1b0 con 0x7fbfd0103c20 2024-04-25T14:07:31.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.905+0000 7fbfceffd640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 0x7fbfbc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.905+0000 7fbfdc5a8740 1 Processor -- start 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.905+0000 7fbfceffd640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 0x7fbfbc0b1fe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbfd01035e0 tx=0x7fbfc0009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.905+0000 7fbfdc5a8740 1 -- start start 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.906+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a785188810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.906+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785188d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.906+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785189a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.906+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a7850ecd80 con 0x55a784dbd4d0 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.907+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7850a6a20 con 0x55a784de6390 2024-04-25T14:07:31.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.907+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7850e8d40 con 0x55a784dc56b0 2024-04-25T14:07:31.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.911+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785188d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.911+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a785188810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.911+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785188d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52106/0 (socket says 172.21.15.94:52106) 2024-04-25T14:07:31.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.911+0000 7fbfc7fff640 1 -- 172.21.15.94:0/936762546 learned_addr learned my addr 172.21.15.94:0/936762546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.911+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785189a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.915+0000 7fbfceffd640 1 -- 172.21.15.94:0/936762546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 msgr2=0x55a785189a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.915+0000 7fbfceffd640 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785189a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.915+0000 7fbfceffd640 1 -- 172.21.15.94:0/936762546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 msgr2=0x55a785188d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.915+0000 7fbfceffd640 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785188d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.915+0000 7fbfceffd640 1 -- 172.21.15.94:0/936762546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7850b1c30 con 0x55a784de6390 2024-04-25T14:07:31.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.923+0000 7fbfceffd640 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a785188810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000ece0 tx=0x7fbfc0011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f4f441c5cc509fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.923+0000 7fbfc57fa640 1 -- 172.21.15.94:0/936762546 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0020070 con 0x55a784de6390 2024-04-25T14:07:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.923+0000 7fbfc57fa640 1 -- 172.21.15.94:0/936762546 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc0010e40 con 0x55a784de6390 2024-04-25T14:07:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.923+0000 7fbfc57fa640 1 -- 172.21.15.94:0/936762546 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc001c7e0 con 0x55a784de6390 2024-04-25T14:07:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.925+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/936762546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 msgr2=0x55a785188810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.925+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a785188810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000ece0 tx=0x7fbfc0011000 comp rx=0 tx=0).stop 2024-04-25T14:07:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.928+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/936762546 shutdown_connections 2024-04-25T14:07:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.928+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785189a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.928+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785188d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.928+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/936762546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a785188810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.929+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/936762546 >> 172.21.15.94:0/936762546 conn(0x55a784dda4e0 msgr2=0x55a785184020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.929+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/936762546 shutdown_connections 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.929+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/936762546 wait complete. 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.930+0000 7fbfdc5a8740 1 -- start start 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.930+0000 7fbfdc5a8740 1 Processor -- start 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dc56b0 0x55a785192ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785199a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:39942/0 (socket says 172.21.15.94:39942) 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfceffd640 1 -- 172.21.15.94:0/3322171521 learned_addr learned my addr 172.21.15.94:0/3322171521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a78518a530 con 0x55a784dc56b0 2024-04-25T14:07:31.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a785073dc0 con 0x55a784dbd4d0 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a78518a6b0 con 0x55a784de6390 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfdc5a8740 10 client.0 _ll_register_callbacks cb 0x55a78508f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.931+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dc56b0 0x55a785192ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfcf7fe640 1 -- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 msgr2=0x55a785192580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfcf7fe640 1 -- 172.21.15.94:0/3322171521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dc56b0 msgr2=0x55a785192ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dc56b0 0x55a785192ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.932+0000 7fbfcf7fe640 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a784dfe3f0 con 0x55a784de6390 2024-04-25T14:07:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.933+0000 7fbfceffd640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.934+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785199a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55a785184db0 tx=0x7fbfc802f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202586c77ed16ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.934+0000 7fbfb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T14:07:31.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.939+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8031860 con 0x55a784de6390 2024-04-25T14:07:31.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.939+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbfc8003970 con 0x55a784de6390 2024-04-25T14:07:31.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.939+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7850fa990 con 0x55a784de6390 2024-04-25T14:07:31.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.941+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbfc8003b70 con 0x55a784de6390 2024-04-25T14:07:31.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.941+0000 7fbfdc5a8740 10 client.33544 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.941+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a784ec87d0 con 0x55a784de6390 2024-04-25T14:07:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.945+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc8036050 con 0x55a784de6390 2024-04-25T14:07:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.945+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fbfc8042070 con 0x55a784de6390 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 10 client.33544 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 10 client.33544 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 20 client.33544 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784da4f60 0x55a7851c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 4 client.33544 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.946+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7850fa990 con 0x55a784da4f60 2024-04-25T14:07:31.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.947+0000 7fbfceffd640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784da4f60 0x55a7851c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.952+0000 7fbfceffd640 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784da4f60 0x55a7851c21b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ce15ae42aa9a004 server_cookie=69b8d6cc255a2fa8 in_seq=0 out_seq=0 2024-04-25T14:07:31.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.953+0000 7fbfb67fc640 10 client.33544 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:07:31.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.954+0000 7fbfb67fc640 1 -- 172.21.15.94:0/3322171521 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7850fa990 con 0x55a784da4f60 2024-04-25T14:07:31.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.954+0000 7fbfb67fc640 10 client.33544 handle_command_reply: tid=0 2024-04-25T14:07:31.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 client.33544 shutdown 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 msgr2=0x55a785199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785199a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55a785184db0 tx=0x7fbfc802f380 comp rx=0 tx=0).stop 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 shutdown_connections 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784da4f60 0x55a7851c21b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785199a20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dc56b0 0x55a785192ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.955+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/3322171521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dbd4d0 0x55a785192580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.956+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 >> 172.21.15.94:0/3322171521 conn(0x55a784dda4e0 msgr2=0x55a785183040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.956+0000 7fbfb67fc640 0 client.33544 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.956+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 shutdown_connections 2024-04-25T14:07:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.956+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/3322171521 wait complete. 2024-04-25T14:07:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.956+0000 7fbfdc5a8740 20 client.33544 trim_cache size 0 max 16384 2024-04-25T14:07:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 Processor -- start 2024-04-25T14:07:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 -- start start 2024-04-25T14:07:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785199940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a78518d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b1a40 con 0x55a784dbd4d0 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52116/0 (socket says 172.21.15.94:52116) 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfceffd640 1 -- 172.21.15.94:0/1052153834 learned_addr learned my addr 172.21.15.94:0/1052153834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b35e0 con 0x55a784de6390 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.975+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b2140 con 0x55a784dc56b0 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a78518d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 -- 172.21.15.94:0/1052153834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 msgr2=0x55a785199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 -- 172.21.15.94:0/1052153834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 msgr2=0x55a78518d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a78518d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 -- 172.21.15.94:0/1052153834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a784da5190 con 0x55a784dbd4d0 2024-04-25T14:07:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a78518d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfceffd640 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00046c0 tx=0x7fbfc000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4247d34bec6870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfc4ff9640 1 -- 172.21.15.94:0/1052153834 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0038070 con 0x55a784dbd4d0 2024-04-25T14:07:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfc4ff9640 1 -- 172.21.15.94:0/1052153834 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc000ee30 con 0x55a784dbd4d0 2024-04-25T14:07:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.976+0000 7fbfc4ff9640 1 -- 172.21.15.94:0/1052153834 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0014040 con 0x55a784dbd4d0 2024-04-25T14:07:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 msgr2=0x55a78518d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00046c0 tx=0x7fbfc000ec90 comp rx=0 tx=0).stop 2024-04-25T14:07:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 shutdown_connections 2024-04-25T14:07:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784de6390 0x55a78518d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784dc56b0 0x55a785199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/1052153834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a78518d390 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 >> 172.21.15.94:0/1052153834 conn(0x55a784dda4e0 msgr2=0x55a785190c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 shutdown_connections 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.977+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/1052153834 wait complete. 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 -- start start 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 Processor -- start 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 0x55a785196c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785228550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b2f10 con 0x55a784dbd4d0 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b2840 con 0x55a784dc56b0 2024-04-25T14:07:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01afa20 con 0x55a784de6390 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52124/0 (socket says 172.21.15.94:52124) 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfceffd640 1 -- 172.21.15.94:0/2587025393 learned_addr learned my addr 172.21.15.94:0/2587025393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfdc5a8740 10 client.0 _ll_register_callbacks cb 0x7fbfc0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.978+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 0x55a785196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfcf7fe640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785228550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 -- 172.21.15.94:0/2587025393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 msgr2=0x55a785228550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785228550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 -- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 msgr2=0x55a785196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 0x55a785196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 -- 172.21.15.94:0/2587025393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a785199c00 con 0x55a784dbd4d0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 0x55a785196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfceffd640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036040 tx=0x7fbfc000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e9889522680cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0038070 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fbfc0031a30 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc000ee30 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0014040 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfdc5a8740 10 client.24585 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.979+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a785229990 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc000ee30 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fbfc003d460 con 0x55a784dbd4d0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 10 client.24585 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 10 client.24585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 20 client.24585 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784d7a3a0 0x55a78522c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 4 client.24585 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.980+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 --> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7850f4800 con 0x55a784d7a3a0 2024-04-25T14:07:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784d7a3a0 0x55a78522c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfb67fc640 10 client.24585 ms_handle_connect on v2:172.21.15.137:6834/3929131029 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfc7fff640 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784d7a3a0 0x55a78522c1c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd1d4298c38cac77 server_cookie=db80a925c9c2c8a1 in_seq=0 out_seq=0 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfb67fc640 1 -- 172.21.15.94:0/2587025393 <== mds.3 v2:172.21.15.137:6834/3929131029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7850f4800 con 0x55a784d7a3a0 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfb67fc640 10 client.24585 handle_command_reply: tid=0 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 client.24585 shutdown 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 msgr2=0x55a785196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036040 tx=0x7fbfc000ea50 comp rx=0 tx=0).stop 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 shutdown_connections 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:6834/3929131029,v1:172.21.15.137:6835/3929131029] conn(0x55a784d7a3a0 0x55a78522c1c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a784de6390 0x55a785228550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a784dc56b0 0x55a785196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 --2- 172.21.15.94:0/2587025393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a784dbd4d0 0x55a785196700 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.981+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 >> 172.21.15.94:0/2587025393 conn(0x55a784dda4e0 msgr2=0x55a78518d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfb67fc640 0 client.24585 ms_handle_reset on v2:172.21.15.137:6834/3929131029 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 shutdown_connections 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfdc5a8740 1 -- 172.21.15.94:0/2587025393 wait complete. 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfdc5a8740 20 client.24585 trim_cache size 0 max 16384 2024-04-25T14:07:31.987 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 msgr2=0x7fbfbc0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 0x7fbfbc0b1fe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbfd01035e0 tx=0x7fbfc0009630 comp rx=0 tx=0).stop 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 msgr2=0x7fbfd01a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e9f0 tx=0x7fbfb800edf0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 shutdown_connections 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fbfbc0afad0 0x7fbfbc0b1fe0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd0108920 0x7fbfd01a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfd0103c20 0x7fbfd01a59e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 --2- 172.21.15.94:0/3464527709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfd01029e0 0x7fbfd006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.982+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 >> 172.21.15.94:0/3464527709 conn(0x7fbfd00fdbd0 msgr2=0x7fbfd0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.983+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 shutdown_connections 2024-04-25T14:07:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:31.983+0000 7fbfd56fa640 1 -- 172.21.15.94:0/3464527709 wait complete. 2024-04-25T14:07:31.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 -- 172.21.15.94:0/2260430150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d8101750 msgr2=0x7f01d8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 --2- 172.21.15.94:0/2260430150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d8101750 0x7f01d8101bc0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009900 tx=0x7f01c802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 -- 172.21.15.94:0/2260430150 shutdown_connections 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 --2- 172.21.15.94:0/2260430150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 0x7f01d810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 --2- 172.21.15.94:0/2260430150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d8101750 0x7f01d8101bc0 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 --2- 172.21.15.94:0/2260430150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d81050a0 0x7f01d81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.105+0000 7f01e0605640 1 -- 172.21.15.94:0/2260430150 >> 172.21.15.94:0/2260430150 conn(0x7f01d80fd350 msgr2=0x7f01d80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- 172.21.15.94:0/2260430150 shutdown_connections 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- 172.21.15.94:0/2260430150 wait complete. 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 Processor -- start 2024-04-25T14:07:32.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- start start 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 0x7f01d819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 0x7f01d819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01d810f610 con 0x7f01d81050a0 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01d810f490 con 0x7f01d8102250 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01e0605640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01d810f790 con 0x7f01d8101750 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01deba2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01deba2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52148/0 (socket says 172.21.15.94:52148) 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.106+0000 7f01deba2640 1 -- 172.21.15.94:0/1027482332 learned_addr learned my addr 172.21.15.94:0/1027482332 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01ddba0640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 0x7f01d819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01de3a1640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 0x7f01d819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 -- 172.21.15.94:0/1027482332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 msgr2=0x7f01d819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 0x7f01d819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 -- 172.21.15.94:0/1027482332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 msgr2=0x7f01d819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 0x7f01d819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 -- 172.21.15.94:0/1027482332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d81a1d30 con 0x7f01d81050a0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01de3a1640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 0x7f01d819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01deba2640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f01d000eaa0 tx=0x7f01d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01cf7fe640 1 -- 172.21.15.94:0/1027482332 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000cdb0 con 0x7f01d81050a0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01cf7fe640 1 -- 172.21.15.94:0/1027482332 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d0005020 con 0x7f01d81050a0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d81a1fc0 con 0x7f01d81050a0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01cf7fe640 1 -- 172.21.15.94:0/1027482332 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00116d0 con 0x7f01d81050a0 2024-04-25T14:07:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.107+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d81a2430 con 0x7f01d81050a0 2024-04-25T14:07:32.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 Processor -- start 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01cf7fe640 1 -- 172.21.15.94:0/1027482332 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f01d0011870 con 0x7f01d81050a0 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 -- start start 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f95a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6ef8d80 con 0x55b4a6bf2390 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6eb2a20 con 0x55b4a6bc94d0 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.110+0000 7f01e7591740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6ef4d40 con 0x55b4a6bd16b0 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52158/0 (socket says 172.21.15.94:52158) 2024-04-25T14:07:32.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ddba0640 1 -- 172.21.15.94:0/3041391083 learned_addr learned my addr 172.21.15.94:0/3041391083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01cf7fe640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 0x7f01c40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f95a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 -- 172.21.15.94:0/3041391083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bd16b0 msgr2=0x55b4a6f95a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f95a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 -- 172.21.15.94:0/3041391083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 msgr2=0x55b4a6f94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 -- 172.21.15.94:0/3041391083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a6ebdc30 con 0x55b4a6bc94d0 2024-04-25T14:07:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01deba2640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 0x7f01c40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01cf7fe640 1 -- 172.21.15.94:0/1027482332 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f01d00dee40 con 0x7f01d81050a0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f94d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01d806b4c0 tx=0x7f01d400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709d20ecf5468744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ccbfe640 1 -- 172.21.15.94:0/3041391083 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4005220 con 0x55b4a6bc94d0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ccbfe640 1 -- 172.21.15.94:0/3041391083 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d4004c50 con 0x55b4a6bc94d0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.111+0000 7f01ccbfe640 1 -- 172.21.15.94:0/3041391083 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4010690 con 0x55b4a6bc94d0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 -- 172.21.15.94:0/3041391083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 msgr2=0x55b4a6f94d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01deba2640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 0x7f01c40b2000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f01d00dcc10 tx=0x7f01d00d59d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f94d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01d806b4c0 tx=0x7f01d400e720 comp rx=0 tx=0).stop 2024-04-25T14:07:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 -- 172.21.15.94:0/3041391083 shutdown_connections 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f95a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f94d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 --2- 172.21.15.94:0/3041391083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6f94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 -- 172.21.15.94:0/3041391083 >> 172.21.15.94:0/3041391083 conn(0x55b4a6be64e0 msgr2=0x55b4a6f90020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 -- 172.21.15.94:0/3041391083 shutdown_connections 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.112+0000 7f01e7591740 1 -- 172.21.15.94:0/3041391083 wait complete. 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 -- start start 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 Processor -- start 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6f96530 con 0x55b4a6bf2390 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6e7fdc0 con 0x55b4a6bd16b0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4a6f966b0 con 0x55b4a6bc94d0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01e7591740 10 client.0 _ll_register_callbacks cb 0x55b4a6e9b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52166/0 (socket says 172.21.15.94:52166) 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01ddba0640 1 -- 172.21.15.94:0/3696193328 learned_addr learned my addr 172.21.15.94:0/3696193328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.113+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 -- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bc94d0 msgr2=0x55b4a6f9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f9e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 -- 172.21.15.94:0/3696193328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 msgr2=0x55b4a6f9eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f9eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 -- 172.21.15.94:0/3696193328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a6c0a3f0 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009230 tx=0x7f01c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d91b5fa9c3e6b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035040 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f01c8004250 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d4004c50 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f01c8004450 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 10 client.24597 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b4a6cd47d0 con 0x55b4a6bf2390 2024-04-25T14:07:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d4004c50 con 0x55b4a6bf2390 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f01c8003710 con 0x55b4a6bf2390 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 10 client.24597 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 10 client.24597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 20 client.24597 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.114+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6bb0f60 0x55b4a6fac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.115+0000 7f01e7591740 4 client.24597 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.115+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a6f06990 con 0x55b4a6bb0f60 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.115+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6bb0f60 0x55b4a6fac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.115+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6bb0f60 0x55b4a6fac2a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d913152303587445 server_cookie=71960d189e179fe in_seq=0 out_seq=0 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.115+0000 7f01c1ffb640 10 client.24597 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3696193328 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4a6f06990 con 0x55b4a6bb0f60 2024-04-25T14:07:32.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01c1ffb640 10 client.24597 handle_command_reply: tid=0 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 client.24597 shutdown 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 msgr2=0x55b4a6fa5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009230 tx=0x7f01c8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 shutdown_connections 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6bb0f60 0x55b4a6fac2a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa5a10 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6f9eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 --2- 172.21.15.94:0/3696193328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01c1ffb640 0 client.24597 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 >> 172.21.15.94:0/3696193328 conn(0x55b4a6be64e0 msgr2=0x55b4a6f92f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 shutdown_connections 2024-04-25T14:07:32.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 1 -- 172.21.15.94:0/3696193328 wait complete. 2024-04-25T14:07:32.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.116+0000 7f01e7591740 20 client.24597 trim_cache size 0 max 16384 2024-04-25T14:07:32.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 Processor -- start 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 -- start start 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6f8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81ab580 con 0x55b4a6bc94d0 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81ad120 con 0x55b4a6bd16b0 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01e7591740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81abc80 con 0x55b4a6bf2390 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52182/0 (socket says 172.21.15.94:52182) 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 -- 172.21.15.94:0/2024043946 learned_addr learned my addr 172.21.15.94:0/2024043946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01de3a1640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6f8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 -- 172.21.15.94:0/2024043946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 msgr2=0x55b4a6f8ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6f8ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 -- 172.21.15.94:0/2024043946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 msgr2=0x55b4a6fa59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01ddba0640 1 -- 172.21.15.94:0/2024043946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a6f98dc0 con 0x55b4a6bc94d0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.130+0000 7f01de3a1640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01ddba0640 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f01c8042040 tx=0x7f01c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed0584b656ac81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01c3fff640 1 -- 172.21.15.94:0/2024043946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035040 con 0x55b4a6bc94d0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01c3fff640 1 -- 172.21.15.94:0/2024043946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8043040 con 0x55b4a6bc94d0 2024-04-25T14:07:32.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01c3fff640 1 -- 172.21.15.94:0/2024043946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c803eb20 con 0x55b4a6bc94d0 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 -- 172.21.15.94:0/2024043946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 msgr2=0x55b4a6f8f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f01c8042040 tx=0x7f01c802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 -- 172.21.15.94:0/2024043946 shutdown_connections 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6f8ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 --2- 172.21.15.94:0/2024043946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6f8f9e0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.131+0000 7f01e7591740 1 -- 172.21.15.94:0/2024043946 >> 172.21.15.94:0/2024043946 conn(0x55b4a6be64e0 msgr2=0x55b4a6f992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- 172.21.15.94:0/2024043946 shutdown_connections 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- 172.21.15.94:0/2024043946 wait complete. 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- start start 2024-04-25T14:07:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 Processor -- start 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81aca50 con 0x55b4a6bc94d0 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81ac380 con 0x55b4a6bd16b0 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81a9560 con 0x55b4a6bf2390 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01e7591740 10 client.0 _ll_register_callbacks cb 0x7f01c8031810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52198/0 (socket says 172.21.15.94:52198) 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 -- 172.21.15.94:0/3050724939 learned_addr learned my addr 172.21.15.94:0/3050724939 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 -- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 msgr2=0x55b4a6fa4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 -- 172.21.15.94:0/3050724939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 msgr2=0x55b4a6fa34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.132+0000 7f01ddba0640 1 -- 172.21.15.94:0/3050724939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a6fa49c0 con 0x55b4a6bc94d0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01ddba0640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009850 tx=0x7f01c80315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e11aea1bd7ac7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035070 con 0x55b4a6bc94d0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f01c803eb00 con 0x55b4a6bc94d0 2024-04-25T14:07:32.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c8043040 con 0x55b4a6bc94d0 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f01c803b420 con 0x55b4a6bc94d0 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 10 client.24603 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b4a6fa5bb0 con 0x55b4a6bc94d0 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8043040 con 0x55b4a6bc94d0 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f01c804c4a0 con 0x55b4a6bc94d0 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 10 client.24603 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 10 client.24603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 20 client.24603 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6b863a0 0x55b4a7016d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 4 client.24603 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.133+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 --> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4a6f9d630 con 0x55b4a6b863a0 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6b863a0 0x55b4a7016d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01c1ffb640 10 client.24603 ms_handle_connect on v2:172.21.15.138:6838/2137998525 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01de3a1640 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6b863a0 0x55b4a7016d80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c01eca0bca4d9f61 server_cookie=d31385f18132b54f in_seq=0 out_seq=0 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01c1ffb640 1 -- 172.21.15.94:0/3050724939 <== mds.2 v2:172.21.15.138:6838/2137998525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4a6f9d630 con 0x55b4a6b863a0 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01c1ffb640 10 client.24603 handle_command_reply: tid=0 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 client.24603 shutdown 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 msgr2=0x55b4a6fa2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009850 tx=0x7f01c80315a0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 shutdown_connections 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:6838/2137998525,v1:172.21.15.138:6839/2137998525] conn(0x55b4a6b863a0 0x55b4a7016d80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4a6bf2390 0x55b4a6fa4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4a6bd16b0 0x55b4a6fa34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 --2- 172.21.15.94:0/3050724939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4a6bc94d0 0x55b4a6fa2f90 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 >> 172.21.15.94:0/3050724939 conn(0x55b4a6be64e0 msgr2=0x55b4a6f92e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01c1ffb640 0 client.24603 ms_handle_reset on v2:172.21.15.138:6838/2137998525 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 shutdown_connections 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 1 -- 172.21.15.94:0/3050724939 wait complete. 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.134+0000 7f01e7591740 20 client.24603 trim_cache size 0 max 16384 2024-04-25T14:07:32.144 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 msgr2=0x7f01c40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 0x7f01c40b2000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f01d00dcc10 tx=0x7f01d00d59d0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 msgr2=0x7f01d81a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f01d000eaa0 tx=0x7f01d000ef60 comp rx=0 tx=0).stop 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 shutdown_connections 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f01c40afaf0 0x7f01c40b2000 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01d81050a0 0x7f01d81a16c0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d8102250 0x7f01d819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 --2- 172.21.15.94:0/1027482332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01d8101750 0x7f01d819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 >> 172.21.15.94:0/1027482332 conn(0x7f01d80fd350 msgr2=0x7f01d8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 shutdown_connections 2024-04-25T14:07:32.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.135+0000 7f01e0605640 1 -- 172.21.15.94:0/1027482332 wait complete. 2024-04-25T14:07:32.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- 172.21.15.94:0/2280276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 msgr2=0x7fb6581047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/2280276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb6581047e0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fb648009ca0 tx=0x7fb64802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- 172.21.15.94:0/2280276542 shutdown_connections 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/2280276542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb65810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/2280276542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 0x7fb658105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/2280276542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb6581047e0 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- 172.21.15.94:0/2280276542 >> 172.21.15.94:0/2280276542 conn(0x7fb6580fed10 msgr2=0x7fb658101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- 172.21.15.94:0/2280276542 shutdown_connections 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- 172.21.15.94:0/2280276542 wait complete. 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 Processor -- start 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.259+0000 7fb65d6aa640 1 -- start start 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 0x7fb65819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb6581a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810f5c0 con 0x7fb658103ae0 2024-04-25T14:07:32.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810f440 con 0x7fb658109a20 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb65d6aa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810f740 con 0x7fb658104d20 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52218/0 (socket says 172.21.15.94:52218) 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 -- 172.21.15.94:0/3651743495 learned_addr learned my addr 172.21.15.94:0/3651743495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb6577fe640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb6581a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb6567fc640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 0x7fb65819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 -- 172.21.15.94:0/3651743495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 msgr2=0x7fb65819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 0x7fb65819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 -- 172.21.15.94:0/3651743495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 msgr2=0x7fb6581a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb6581a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 -- 172.21.15.94:0/3651743495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6581a1dc0 con 0x7fb658103ae0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb6577fe640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb6581a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb656ffd640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb64802ade0 tx=0x7fb6480316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb63ffff640 1 -- 172.21.15.94:0/3651743495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648003b70 con 0x7fb658103ae0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.260+0000 7fb63ffff640 1 -- 172.21.15.94:0/3651743495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb648003d10 con 0x7fb658103ae0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.261+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a2050 con 0x7fb658103ae0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.261+0000 7fb63ffff640 1 -- 172.21.15.94:0/3651743495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64802fa60 con 0x7fb658103ae0 2024-04-25T14:07:32.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.261+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a25a0 con 0x7fb658103ae0 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 Processor -- start 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb63ffff640 1 -- 172.21.15.94:0/3651743495 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fb648004580 con 0x7fb658103ae0 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 -- start start 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56342e427d80 con 0x56342e1006b0 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56342e3e1a20 con 0x56342e0f84d0 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb664633740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56342e423d40 con 0x56342e121390 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52224/0 (socket says 172.21.15.94:52224) 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb6567fc640 1 -- 172.21.15.94:0/1094847757 learned_addr learned my addr 172.21.15.94:0/1094847757 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.263+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 -- 172.21.15.94:0/1094847757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 msgr2=0x56342e4c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 -- 172.21.15.94:0/1094847757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 msgr2=0x56342e4c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb63ffff640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 0x7fb6400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 -- 172.21.15.94:0/1094847757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342e3ecc30 con 0x56342e1006b0 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:32.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6577fe640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 0x7fb6400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00bcb0 tx=0x7fb64c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e5a73030f34baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb63ffff640 1 -- 172.21.15.94:0/3651743495 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb6481000c0 con 0x7fb658103ae0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb63d7fa640 1 -- 172.21.15.94:0/1094847757 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01c480 con 0x56342e1006b0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb63d7fa640 1 -- 172.21.15.94:0/1094847757 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c01cad0 con 0x56342e1006b0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.264+0000 7fb63d7fa640 1 -- 172.21.15.94:0/1094847757 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c011c30 con 0x56342e1006b0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- 172.21.15.94:0/1094847757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 msgr2=0x56342e4c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00bcb0 tx=0x7fb64c0107a0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb6577fe640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 0x7fb6400b2000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x56342e4c4860 tx=0x7fb650005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- 172.21.15.94:0/1094847757 shutdown_connections 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4c4a20 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 --2- 172.21.15.94:0/1094847757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- 172.21.15.94:0/1094847757 >> 172.21.15.94:0/1094847757 conn(0x56342e1154e0 msgr2=0x56342e4bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- 172.21.15.94:0/1094847757 shutdown_connections 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- 172.21.15.94:0/1094847757 wait complete. 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 -- start start 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.265+0000 7fb664633740 1 Processor -- start 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e1006b0 0x56342e4d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56342e4c5530 con 0x56342e121390 2024-04-25T14:07:32.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56342e3aedc0 con 0x56342e0f84d0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56342e4c56b0 con 0x56342e1006b0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 10 client.0 _ll_register_callbacks cb 0x56342e3ca600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52228/0 (socket says 172.21.15.94:52228) 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 -- 172.21.15.94:0/1393421649 learned_addr learned my addr 172.21.15.94:0/1393421649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e1006b0 0x56342e4d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 -- 172.21.15.94:0/1393421649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e1006b0 msgr2=0x56342e4d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e1006b0 0x56342e4d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 -- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 msgr2=0x56342e4d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 -- 172.21.15.94:0/1393421649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342e1393f0 con 0x56342e121390 2024-04-25T14:07:32.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb650012e30 tx=0x7fb650004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ce052cd841160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb63aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500119c0 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb650011b60 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c01cad0 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb650011d60 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.266+0000 7fb664633740 10 client.24615 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56342e2037d0 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c01cad0 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fb650027070 con 0x56342e121390 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 10 client.24615 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 10 client.24615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 20 client.24615 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0dff60 0x56342e4df820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 4 client.24615 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342e435990 con 0x56342e0dff60 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0dff60 0x56342e4df820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb63aff5640 10 client.24615 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.267+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0dff60 0x56342e4df820 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=394d8f9340e4b937 server_cookie=709868c209ab49d1 in_seq=0 out_seq=0 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb63aff5640 1 -- 172.21.15.94:0/1393421649 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56342e435990 con 0x56342e0dff60 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb63aff5640 10 client.24615 handle_command_reply: tid=0 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 client.24615 shutdown 2024-04-25T14:07:32.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 msgr2=0x56342e4d8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb650012e30 tx=0x7fb650004730 comp rx=0 tx=0).stop 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 shutdown_connections 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0dff60 0x56342e4df820 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e121390 0x56342e4d8e40 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e1006b0 0x56342e4d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb63aff5640 0 client.24615 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 --2- 172.21.15.94:0/1393421649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 >> 172.21.15.94:0/1393421649 conn(0x56342e1154e0 msgr2=0x56342e4c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 shutdown_connections 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 1 -- 172.21.15.94:0/1393421649 wait complete. 2024-04-25T14:07:32.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.268+0000 7fb664633740 20 client.24615 trim_cache size 0 max 16384 2024-04-25T14:07:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 Processor -- start 2024-04-25T14:07:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 -- start start 2024-04-25T14:07:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4be9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581a93e0 con 0x56342e1006b0 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aaf80 con 0x56342e0f84d0 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb664633740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581a9ae0 con 0x56342e121390 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52234/0 (socket says 172.21.15.94:52234) 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 -- 172.21.15.94:0/1529222391 learned_addr learned my addr 172.21.15.94:0/1529222391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4be9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 -- 172.21.15.94:0/1529222391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 msgr2=0x56342e4bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 -- 172.21.15.94:0/1529222391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 msgr2=0x56342e4be9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4be9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6567fc640 1 -- 172.21.15.94:0/1529222391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342e4c33d0 con 0x56342e1006b0 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb656ffd640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.282+0000 7fb6577fe640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb6567fc640 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb64c010ca0 tx=0x7fb64c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb191b248e2e498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb63cff9640 1 -- 172.21.15.94:0/1529222391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c02b070 con 0x56342e1006b0 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb63cff9640 1 -- 172.21.15.94:0/1529222391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0035f0 con 0x56342e1006b0 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb63cff9640 1 -- 172.21.15.94:0/1529222391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01c6e0 con 0x56342e1006b0 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb664633740 1 -- 172.21.15.94:0/1529222391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 msgr2=0x56342e4d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.283+0000 7fb664633740 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb64c010ca0 tx=0x7fb64c00bda0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- 172.21.15.94:0/1529222391 shutdown_connections 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d8d60 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- 172.21.15.94:0/1529222391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- 172.21.15.94:0/1529222391 >> 172.21.15.94:0/1529222391 conn(0x56342e1154e0 msgr2=0x56342e4c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- 172.21.15.94:0/1529222391 shutdown_connections 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- 172.21.15.94:0/1529222391 wait complete. 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- start start 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 Processor -- start 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aa8b0 con 0x56342e1006b0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aa1e0 con 0x56342e0f84d0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581a73c0 con 0x56342e121390 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb664633740 10 client.0 _ll_register_callbacks cb 0x7fb650009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb6567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52246/0 (socket says 172.21.15.94:52246) 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb6567fc640 1 -- 172.21.15.94:0/413601301 learned_addr learned my addr 172.21.15.94:0/413601301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb656ffd640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.284+0000 7fb6577fe640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 -- 172.21.15.94:0/413601301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 msgr2=0x56342e4d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 -- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 msgr2=0x56342e4d6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 -- 172.21.15.94:0/413601301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342e4d92a0 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb6567fc640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fb64c011010 tx=0x7fb64c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c77e66df5cffaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c034060 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fb64c0052c0 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c0035f0 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb64c02b070 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 10 client.24621 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56342e4d8b60 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c0035f0 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fb64c011480 con 0x56342e1006b0 2024-04-25T14:07:32.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 10 client.24621 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 10 client.24621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 20 client.24621 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0b53a0 0x56342e549c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 4 client.24621 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.285+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 --> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56342e42f800 con 0x56342e0b53a0 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb656ffd640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0b53a0 0x56342e549c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb63aff5640 10 client.24621 ms_handle_connect on v2:172.21.15.137:6836/1274977027 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb656ffd640 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0b53a0 0x56342e549c20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78e3a9fd71cda006 server_cookie=c120350fce799b91 in_seq=0 out_seq=0 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb63aff5640 1 -- 172.21.15.94:0/413601301 <== mds.1 v2:172.21.15.137:6836/1274977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56342e42f800 con 0x56342e0b53a0 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb63aff5640 10 client.24621 handle_command_reply: tid=0 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 client.24621 shutdown 2024-04-25T14:07:32.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 msgr2=0x56342e4d6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fb64c011010 tx=0x7fb64c0039d0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 shutdown_connections 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:6836/1274977027,v1:172.21.15.137:6837/1274977027] conn(0x56342e0b53a0 0x56342e549c20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56342e121390 0x56342e4d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56342e1006b0 0x56342e4d6650 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 --2- 172.21.15.94:0/413601301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56342e0f84d0 0x56342e4d6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb63aff5640 0 client.24621 ms_handle_reset on v2:172.21.15.137:6836/1274977027 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 >> 172.21.15.94:0/413601301 conn(0x56342e1154e0 msgr2=0x56342e4c1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 shutdown_connections 2024-04-25T14:07:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 1 -- 172.21.15.94:0/413601301 wait complete. 2024-04-25T14:07:32.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.286+0000 7fb664633740 20 client.24621 trim_cache size 0 max 16384 2024-04-25T14:07:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:07:32.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 msgr2=0x7fb6400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 0x7fb6400b2000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x56342e4c4860 tx=0x7fb650005f90 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 msgr2=0x7fb65819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb64802ade0 tx=0x7fb6480316e0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 shutdown_connections 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fb6400afaf0 0x7fb6400b2000 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb658109a20 0x7fb6581a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb658104d20 0x7fb65819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 --2- 172.21.15.94:0/3651743495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb658103ae0 0x7fb65819a280 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 >> 172.21.15.94:0/3651743495 conn(0x7fb6580fed10 msgr2=0x7fb658105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 shutdown_connections 2024-04-25T14:07:32.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.287+0000 7fb65d6aa640 1 -- 172.21.15.94:0/3651743495 wait complete. 2024-04-25T14:07:32.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:07:32.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 -- 172.21.15.94:0/3354832512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518108920 msgr2=0x7f751810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 --2- 172.21.15.94:0/3354832512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518108920 0x7f751810ad40 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f7510009ca0 tx=0x7f751002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 -- 172.21.15.94:0/3354832512 shutdown_connections 2024-04-25T14:07:32.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 --2- 172.21.15.94:0/3354832512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518108920 0x7f751810ad40 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 --2- 172.21.15.94:0/3354832512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518103c20 0x7f7518104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 --2- 172.21.15.94:0/3354832512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 0x7f75181036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 -- 172.21.15.94:0/3354832512 >> 172.21.15.94:0/3354832512 conn(0x7f75180fdbd0 msgr2=0x7f7518100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 -- 172.21.15.94:0/3354832512 shutdown_connections 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.410+0000 7f751e6af640 1 -- 172.21.15.94:0/3354832512 wait complete. 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 Processor -- start 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 -- start start 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 0x7f751819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518108920 0x7f75181a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f751810f610 con 0x7f7518103c20 2024-04-25T14:07:32.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f751810f490 con 0x7f7518108920 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751e6af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f751810f790 con 0x7f75181029e0 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52272/0 (socket says 172.21.15.94:52272) 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 -- 172.21.15.94:0/2863097294 learned_addr learned my addr 172.21.15.94:0/2863097294 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f7517fff640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 0x7f751819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f751cc4c640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518108920 0x7f75181a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 -- 172.21.15.94:0/2863097294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 msgr2=0x7f751819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 0x7f751819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 -- 172.21.15.94:0/2863097294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518108920 msgr2=0x7f75181a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518108920 0x7f75181a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.411+0000 7f75177fe640 1 -- 172.21.15.94:0/2863097294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75181a1d80 con 0x7f7518103c20 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f75177fe640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d950 tx=0x7f750c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f75157fa640 1 -- 172.21.15.94:0/2863097294 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c017070 con 0x7f7518103c20 2024-04-25T14:07:32.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f75157fa640 1 -- 172.21.15.94:0/2863097294 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c009c30 con 0x7f7518103c20 2024-04-25T14:07:32.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75181a2070 con 0x7f7518103c20 2024-04-25T14:07:32.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f75157fa640 1 -- 172.21.15.94:0/2863097294 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c00f620 con 0x7f7518103c20 2024-04-25T14:07:32.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.412+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a25c0 con 0x7f7518103c20 2024-04-25T14:07:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.414+0000 7f75157fa640 1 -- 172.21.15.94:0/2863097294 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f750c010460 con 0x7f7518103c20 2024-04-25T14:07:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.414+0000 7f7525566740 1 Processor -- start 2024-04-25T14:07:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 -- start start 2024-04-25T14:07:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 0x563d4b24dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b24e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d4b1add80 con 0x563d4ae866b0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d4b167a20 con 0x563d4aea7390 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7525566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d4b1a9d40 con 0x563d4ae7e4d0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52276/0 (socket says 172.21.15.94:52276) 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 -- 172.21.15.94:0/4279376628 learned_addr learned my addr 172.21.15.94:0/4279376628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7517fff640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b24e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f751cc4c640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 0x563d4b24dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75157fa640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 0x7f75000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f751cc4c640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 0x7f75000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 -- 172.21.15.94:0/4279376628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 msgr2=0x563d4b24e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b24e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 -- 172.21.15.94:0/4279376628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 msgr2=0x563d4b24dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 0x563d4b24dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75157fa640 1 -- 172.21.15.94:0/2863097294 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f750c0de930 con 0x7f7518103c20 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f75177fe640 1 -- 172.21.15.94:0/4279376628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4b172c30 con 0x563d4ae866b0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f7517fff640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b24e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.415+0000 7f751cc4c640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 0x563d4b24dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f75177fe640 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a47f0 tx=0x7f750c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa37b9d8d876e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f751cc4c640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 0x7f75000b2000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x563d4b24ec70 tx=0x7f751002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f74fe7fc640 1 -- 172.21.15.94:0/4279376628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c020020 con 0x563d4ae866b0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f74fe7fc640 1 -- 172.21.15.94:0/4279376628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c0a6880 con 0x563d4ae866b0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f74fe7fc640 1 -- 172.21.15.94:0/4279376628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01de40 con 0x563d4ae866b0 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f7525566740 1 -- 172.21.15.94:0/4279376628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 msgr2=0x563d4b24ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.416+0000 7f7525566740 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a47f0 tx=0x7f750c010cb0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- 172.21.15.94:0/4279376628 shutdown_connections 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b24ee30 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b24e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- 172.21.15.94:0/4279376628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4aea7390 0x563d4b24dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- 172.21.15.94:0/4279376628 >> 172.21.15.94:0/4279376628 conn(0x563d4ae9b4e0 msgr2=0x563d4b245020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- 172.21.15.94:0/4279376628 shutdown_connections 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- 172.21.15.94:0/4279376628 wait complete. 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- start start 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 Processor -- start 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b257dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b25ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d4b24f880 con 0x563d4ae7e4d0 2024-04-25T14:07:32.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d4b134dc0 con 0x563d4ae866b0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d4b24fa00 con 0x563d4aea7390 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.417+0000 7f7525566740 10 client.0 _ll_register_callbacks cb 0x563d4b150600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52280/0 (socket says 172.21.15.94:52280) 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 -- 172.21.15.94:0/3659633617 learned_addr learned my addr 172.21.15.94:0/3659633617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f75177fe640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b257dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f751cc4c640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b25ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 -- 172.21.15.94:0/3659633617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 msgr2=0x563d4b25ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b25ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 -- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 msgr2=0x563d4b257dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b257dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 -- 172.21.15.94:0/3659633617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4aebf3f0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f75177fe640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b257dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7517fff640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f7508012700 tx=0x7f7508012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348b76348dcdc1df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f74f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508004030 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f75080041d0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c0a6880 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f75080043d0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7525566740 10 client.24633 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.418+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563d4af897d0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c0a6880 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f7508003ae0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 10 client.24633 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 10 client.24633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 20 client.24633 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae65f60 0x563d4b265580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 4 client.24633 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d4b1bb990 con 0x563d4ae65f60 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.419+0000 7f75177fe640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae65f60 0x563d4b265580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.420+0000 7f74f77fe640 10 client.24633 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.420+0000 7f75177fe640 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae65f60 0x563d4b265580 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbaf993b9c2fd3a6 server_cookie=bf1f3625b7bdea38 in_seq=0 out_seq=0 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.420+0000 7f74f77fe640 1 -- 172.21.15.94:0/3659633617 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d4b1bb990 con 0x563d4ae65f60 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.420+0000 7f74f77fe640 10 client.24633 handle_command_reply: tid=0 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.420+0000 7f7525566740 1 client.24633 shutdown 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 msgr2=0x563d4b257890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f7508012700 tx=0x7f7508012bc0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 shutdown_connections 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae65f60 0x563d4b265580 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b25ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b257dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 --2- 172.21.15.94:0/3659633617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae7e4d0 0x563d4b257890 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f74f77fe640 0 client.24633 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 >> 172.21.15.94:0/3659633617 conn(0x563d4ae9b4e0 msgr2=0x563d4b244040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 shutdown_connections 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 1 -- 172.21.15.94:0/3659633617 wait complete. 2024-04-25T14:07:32.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.421+0000 7f7525566740 20 client.24633 trim_cache size 0 max 16384 2024-04-25T14:07:32.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 Processor -- start 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 -- start start 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae7e4d0 0x563d4b257460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b2579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75181a9430 con 0x563d4ae866b0 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aafd0 con 0x563d4ae7e4d0 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7525566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75181a9b30 con 0x563d4aea7390 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52284/0 (socket says 172.21.15.94:52284) 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 -- 172.21.15.94:0/2050803931 learned_addr learned my addr 172.21.15.94:0/2050803931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f751cc4c640 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b2579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f7517fff640 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae7e4d0 0x563d4b257460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 -- 172.21.15.94:0/2050803931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 msgr2=0x563d4b2579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b2579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 -- 172.21.15.94:0/2050803931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae7e4d0 msgr2=0x563d4b257460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae7e4d0 0x563d4b257460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.435+0000 7f75177fe640 1 -- 172.21.15.94:0/2050803931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4b25f190 con 0x563d4ae866b0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f75177fe640 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a6f30 tx=0x7f750c0dc300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6256661b233b01eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f74fdffb640 1 -- 172.21.15.94:0/2050803931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c020040 con 0x563d4ae866b0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f74fdffb640 1 -- 172.21.15.94:0/2050803931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c009de0 con 0x563d4ae866b0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f74fdffb640 1 -- 172.21.15.94:0/2050803931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0af740 con 0x563d4ae866b0 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f7525566740 1 -- 172.21.15.94:0/2050803931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 msgr2=0x563d4b25ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.436+0000 7f7525566740 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a6f30 tx=0x7f750c0dc300 comp rx=0 tx=0).stop 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- 172.21.15.94:0/2050803931 shutdown_connections 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4aea7390 0x563d4b2579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4ae866b0 0x563d4b25ec50 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- 172.21.15.94:0/2050803931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae7e4d0 0x563d4b257460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- 172.21.15.94:0/2050803931 >> 172.21.15.94:0/2050803931 conn(0x563d4ae9b4e0 msgr2=0x563d4b2526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- 172.21.15.94:0/2050803931 shutdown_connections 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- 172.21.15.94:0/2050803931 wait complete. 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- start start 2024-04-25T14:07:32.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 Processor -- start 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b25b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b25be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aa900 con 0x563d4aea7390 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aa230 con 0x563d4ae866b0 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75181a7410 con 0x563d4ae7e4d0 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.437+0000 7f7525566740 10 client.0 _ll_register_callbacks cb 0x7f7508009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f7517fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b25b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52294/0 (socket says 172.21.15.94:52294) 2024-04-25T14:07:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 -- 172.21.15.94:0/1616751181 learned_addr learned my addr 172.21.15.94:0/1616751181 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f75177fe640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b25be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 -- 172.21.15.94:0/1616751181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 msgr2=0x563d4b25b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b25b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 -- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 msgr2=0x563d4b25be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b25be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 -- 172.21.15.94:0/1616751181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4b25ede0 con 0x563d4aea7390 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f7517fff640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b25b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f751cc4c640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f751002ade0 tx=0x7f75100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a89159911c7ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f74f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510043070 con 0x563d4aea7390 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7510004040 con 0x563d4aea7390 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c009de0 con 0x563d4aea7390 2024-04-25T14:07:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7510004240 con 0x563d4aea7390 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f7525566740 10 client.24639 fetch_fsmap learned FSMap version 42 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563d4b2ccd60 con 0x563d4aea7390 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.438+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c009de0 con 0x563d4aea7390 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f75100305c0 con 0x563d4aea7390 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 10 client.24639 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 10 client.24639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 20 client.24639 populate_metadata read hostname 'smithi094' 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae3b3a0 0x563d4b2cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 4 client.24639 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 --> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d4b1b5800 con 0x563d4ae3b3a0 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7517fff640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae3b3a0 0x563d4b2cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f74f77fe640 10 client.24639 ms_handle_connect on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:32.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7517fff640 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae3b3a0 0x563d4b2cf7d0 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a12b586dc683a0 server_cookie=e630c273fd9d4fc9 in_seq=0 out_seq=0 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f74f77fe640 1 -- 172.21.15.94:0/1616751181 <== mds.0 v2:172.21.15.137:6838/3143839143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d4b1b5800 con 0x563d4ae3b3a0 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f74f77fe640 10 client.24639 handle_command_reply: tid=0 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.439+0000 7f7525566740 1 client.24639 shutdown 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 msgr2=0x563d4b2cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f751002ade0 tx=0x7f75100095c0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 shutdown_connections 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:6838/3143839143,v1:172.21.15.137:6839/3143839143] conn(0x563d4ae3b3a0 0x563d4b2cf7d0 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d4aea7390 0x563d4b2cb8e0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4ae866b0 0x563d4b25be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 --2- 172.21.15.94:0/1616751181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4ae7e4d0 0x563d4b25b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f74f77fe640 0 client.24639 ms_handle_reset on v2:172.21.15.137:6838/3143839143 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 >> 172.21.15.94:0/1616751181 conn(0x563d4ae9b4e0 msgr2=0x563d4b247e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 shutdown_connections 2024-04-25T14:07:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 1 -- 172.21.15.94:0/1616751181 wait complete. 2024-04-25T14:07:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f7525566740 20 client.24639 trim_cache size 0 max 16384 2024-04-25T14:07:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 msgr2=0x7f75000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 0x7f75000b2000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x563d4b24ec70 tx=0x7f751002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 msgr2=0x7f751819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d950 tx=0x7f750c00de10 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 shutdown_connections 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f75000afaf0 0x7f75000b2000 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7518108920 0x7f75181a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7518103c20 0x7f751819a780 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 --2- 172.21.15.94:0/2863097294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75181029e0 0x7f751819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.440+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 >> 172.21.15.94:0/2863097294 conn(0x7f75180fdbd0 msgr2=0x7f7518104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.441+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 shutdown_connections 2024-04-25T14:07:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:32.441+0000 7f751e6af640 1 -- 172.21.15.94:0/2863097294 wait complete. 2024-04-25T14:07:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:32 smithi094 ceph-mon[29195]: pgmap v1123: 97 pgs: 97 active+clean; 131 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 374 op/s 2024-04-25T14:07:32.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:32 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2839453326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:32 smithi138 ceph-mon[32977]: pgmap v1123: 97 pgs: 97 active+clean; 131 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 374 op/s 2024-04-25T14:07:32.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:32 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2839453326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:32.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:32 smithi137 ceph-mon[32568]: pgmap v1123: 97 pgs: 97 active+clean; 131 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 374 op/s 2024-04-25T14:07:32.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:32 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2839453326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:33.453 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T14:07:33.453 INFO:tasks.fwd_scrub:done joining 2024-04-25T14:07:33.454 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T14:07:33.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/3033984140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 msgr2=0x7f7b9c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/3033984140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c10ad40 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b9402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/3033984140 shutdown_connections 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/3033984140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c10ad40 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/3033984140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 0x7f7b9c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/3033984140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 0x7f7b9c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/3033984140 >> 172.21.15.94:0/3033984140 conn(0x7f7b9c0fdbd0 msgr2=0x7f7b9c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/3033984140 shutdown_connections 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.588+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/3033984140 wait complete. 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 Processor -- start 2024-04-25T14:07:33.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 -- start start 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 0x7f7b9c19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 0x7f7b9c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10f610 con 0x7f7b9c108920 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10f490 con 0x7f7b9c103c20 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.589+0000 7f7ba2a0c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10f790 con 0x7f7b9c1029e0 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46858/0 (socket says 172.21.15.94:46858) 2024-04-25T14:07:33.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 -- 172.21.15.94:0/2899282333 learned_addr learned my addr 172.21.15.94:0/2899282333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7b9bfff640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 0x7f7b9c19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7b9b7fe640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 0x7f7b9c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 -- 172.21.15.94:0/2899282333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 msgr2=0x7f7b9c19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 0x7f7b9c19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 -- 172.21.15.94:0/2899282333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 msgr2=0x7f7b9c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 0x7f7b9c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 -- 172.21.15.94:0/2899282333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1a1cd0 con 0x7f7b9c108920 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba0fa9640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f7b940097b0 tx=0x7f7b94003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94004240 con 0x7f7b9c108920 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1a1f60 con 0x7f7b9c108920 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b9402f380 con 0x7f7b9c108920 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402fc30 con 0x7f7b9c108920 2024-04-25T14:07:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.590+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a2420 con 0x7f7b9c108920 2024-04-25T14:07:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.591+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c107730 con 0x7f7b9c108920 2024-04-25T14:07:33.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.593+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7b940043e0 con 0x7f7b9c108920 2024-04-25T14:07:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.593+0000 7f7b997fa640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 0x7f7b840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.593+0000 7f7b9bfff640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 0x7f7b840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.594+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f7b94100510 con 0x7f7b9c108920 2024-04-25T14:07:33.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.594+0000 7f7b9bfff640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 0x7f7b840b2000 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c1035e0 tx=0x7f7b8c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.597+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b940c9050 con 0x7f7b9c108920 2024-04-25T14:07:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.778+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b9c02dd20 con 0x7f7b9c108920 2024-04-25T14:07:33.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.779+0000 7f7b997fa640 1 -- 172.21.15.94:0/2899282333 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22958 (secure 0 0 0) 0x7f7b940c8950 con 0x7f7b9c108920 2024-04-25T14:07:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:07:33.780 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:58:45.755152+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:07:33.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 msgr2=0x7f7b840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 0x7f7b840b2000 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c1035e0 tx=0x7f7b8c00b390 comp rx=0 tx=0).stop 2024-04-25T14:07:33.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 msgr2=0x7f7b9c1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f7b940097b0 tx=0x7f7b94003c20 comp rx=0 tx=0).stop 2024-04-25T14:07:33.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 shutdown_connections 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f7b840afaf0 0x7f7b840b2000 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b9c108920 0x7f7b9c1a1660 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b9c103c20 0x7f7b9c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 --2- 172.21.15.94:0/2899282333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b9c1029e0 0x7f7b9c19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 >> 172.21.15.94:0/2899282333 conn(0x7f7b9c0fdbd0 msgr2=0x7f7b9c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 shutdown_connections 2024-04-25T14:07:33.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.781+0000 7f7ba2a0c640 1 -- 172.21.15.94:0/2899282333 wait complete. 2024-04-25T14:07:33.793 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:07:33.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- 172.21.15.94:0/2812357393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394108940 msgr2=0x7f239410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 --2- 172.21.15.94:0/2812357393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394108940 0x7f239410ad60 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f238c009ca0 tx=0x7f238c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- 172.21.15.94:0/2812357393 shutdown_connections 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 --2- 172.21.15.94:0/2812357393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394108940 0x7f239410ad60 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 --2- 172.21.15.94:0/2812357393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f2394104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 --2- 172.21.15.94:0/2812357393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394102a00 0x7f2394103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- 172.21.15.94:0/2812357393 >> 172.21.15.94:0/2812357393 conn(0x7f23940fdbf0 msgr2=0x7f23941000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- 172.21.15.94:0/2812357393 shutdown_connections 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- 172.21.15.94:0/2812357393 wait complete. 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 Processor -- start 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.905+0000 7f239af06640 1 -- start start 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f239419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394108940 0x7f23941a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f239410e530 con 0x7f2394102a00 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f239410e3b0 con 0x7f2394103c40 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f239410e6b0 con 0x7f2394108940 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46872/0 (socket says 172.21.15.94:46872) 2024-04-25T14:07:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 -- 172.21.15.94:0/1190002392 learned_addr learned my addr 172.21.15.94:0/1190002392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f23994a3640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394108940 0x7f23941a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2393fff640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f239419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 -- 172.21.15.94:0/1190002392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394108940 msgr2=0x7f23941a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394108940 0x7f23941a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 -- 172.21.15.94:0/1190002392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 msgr2=0x7f239419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f239419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 -- 172.21.15.94:0/1190002392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941a1d50 con 0x7f2394102a00 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2393fff640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f239419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2398ca2640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f238800c970 tx=0x7f238800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388004220 con 0x7f2394102a00 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880043c0 con 0x7f2394102a00 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a1fe0 con 0x7f2394102a00 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.906+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388004a00 con 0x7f2394102a00 2024-04-25T14:07:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.907+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a2450 con 0x7f2394102a00 2024-04-25T14:07:33.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.908+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2394107750 con 0x7f2394102a00 2024-04-25T14:07:33.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.909+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f23880079b0 con 0x7f2394102a00 2024-04-25T14:07:33.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.910+0000 7f2391ffb640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 0x7f23840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:33.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.910+0000 7f2393fff640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 0x7f23840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:33.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.910+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f23880deaa0 con 0x7f2394102a00 2024-04-25T14:07:33.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.910+0000 7f2393fff640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 0x7f23840b2000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f23940751e0 tx=0x7f2380005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:33.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:33.915+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2388014030 con 0x7f2394102a00 2024-04-25T14:07:34.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.093+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23940637a0 con 0x7f2394102a00 2024-04-25T14:07:34.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.094+0000 7f2391ffb640 1 -- 172.21.15.94:0/1190002392 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f23880a75e0 con 0x7f2394102a00 2024-04-25T14:07:34.094 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 msgr2=0x7f23840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 0x7f23840b2000 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f23940751e0 tx=0x7f2380005f90 comp rx=0 tx=0).stop 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 msgr2=0x7f2394075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f238800c970 tx=0x7f238800ce30 comp rx=0 tx=0).stop 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 shutdown_connections 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f23840afaf0 0x7f23840b2000 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2394108940 0x7f23941a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394103c40 0x7f239419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 --2- 172.21.15.94:0/1190002392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2394102a00 0x7f2394075840 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 >> 172.21.15.94:0/1190002392 conn(0x7f23940fdbf0 msgr2=0x7f2394104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 shutdown_connections 2024-04-25T14:07:34.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.096+0000 7f239af06640 1 -- 172.21.15.94:0/1190002392 wait complete. 2024-04-25T14:07:34.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:07:34.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 -- 172.21.15.94:0/399761655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c1029f0 msgr2=0x7fa64c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 --2- 172.21.15.94:0/399761655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c1029f0 0x7fa64c1036f0 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009900 tx=0x7fa63c02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 -- 172.21.15.94:0/399761655 shutdown_connections 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 --2- 172.21.15.94:0/399761655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c108930 0x7fa64c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 --2- 172.21.15.94:0/399761655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c103c30 0x7fa64c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 --2- 172.21.15.94:0/399761655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c1029f0 0x7fa64c1036f0 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 -- 172.21.15.94:0/399761655 >> 172.21.15.94:0/399761655 conn(0x7fa64c0fdbc0 msgr2=0x7fa64c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 -- 172.21.15.94:0/399761655 shutdown_connections 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.219+0000 7fa6515f3640 1 -- 172.21.15.94:0/399761655 wait complete. 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 Processor -- start 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 -- start start 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 0x7fa64c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 0x7fa64c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10e4f0 con 0x7fa64c108930 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10e370 con 0x7fa64c103c30 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa6515f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10e670 con 0x7fa64c1029f0 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46888/0 (socket says 172.21.15.94:46888) 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64b7fe640 1 -- 172.21.15.94:0/2359924245 learned_addr learned my addr 172.21.15.94:0/2359924245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64affd640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 0x7fa64c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa643fff640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 0x7fa64c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64b7fe640 1 -- 172.21.15.94:0/2359924245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 msgr2=0x7fa64c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.220+0000 7fa64b7fe640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 0x7fa64c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa64b7fe640 1 -- 172.21.15.94:0/2359924245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 msgr2=0x7fa64c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa64b7fe640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 0x7fa64c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa64b7fe640 1 -- 172.21.15.94:0/2359924245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c1a1d40 con 0x7fa64c108930 2024-04-25T14:07:34.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa64affd640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 0x7fa64c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa643fff640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 0x7fa64c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa64b7fe640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fa64400eaa0 tx=0x7fa64400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64400cdb0 con 0x7fa64c108930 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa644005020 con 0x7fa64c108930 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c1a1fd0 con 0x7fa64c108930 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440116d0 con 0x7fa64c108930 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.221+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a2440 con 0x7fa64c108930 2024-04-25T14:07:34.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.222+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c107740 con 0x7fa64c108930 2024-04-25T14:07:34.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.223+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fa6440051d0 con 0x7fa64c108930 2024-04-25T14:07:34.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.224+0000 7fa648ff9640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 0x7fa6380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.224+0000 7fa64affd640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 0x7fa6380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.224+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fa6440df280 con 0x7fa64c108930 2024-04-25T14:07:34.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.225+0000 7fa64affd640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 0x7fa6380b2000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009230 tx=0x7fa63c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.228+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa644016030 con 0x7fa64c108930 2024-04-25T14:07:34.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.407+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa64c0637a0 con 0x7fa64c108930 2024-04-25T14:07:34.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.408+0000 7fa648ff9640 1 -- 172.21.15.94:0/2359924245 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fa6440a7dc0 con 0x7fa64c108930 2024-04-25T14:07:34.408 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 msgr2=0x7fa6380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 0x7fa6380b2000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009230 tx=0x7fa63c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 msgr2=0x7fa64c1a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fa64400eaa0 tx=0x7fa64400ef60 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 shutdown_connections 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fa6380afaf0 0x7fa6380b2000 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64c108930 0x7fa64c1a1520 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa64c103c30 0x7fa64c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 --2- 172.21.15.94:0/2359924245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c1029f0 0x7fa64c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 >> 172.21.15.94:0/2359924245 conn(0x7fa64c0fdbc0 msgr2=0x7fa64c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 shutdown_connections 2024-04-25T14:07:34.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.410+0000 7fa6515f3640 1 -- 172.21.15.94:0/2359924245 wait complete. 2024-04-25T14:07:34.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:07:34.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 -- 172.21.15.94:0/341247440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258109a30 msgr2=0x7fc25810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 --2- 172.21.15.94:0/341247440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258109a30 0x7fc25810be50 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fc250009ca0 tx=0x7fc25002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 -- 172.21.15.94:0/341247440 shutdown_connections 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 --2- 172.21.15.94:0/341247440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258109a30 0x7fc25810be50 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 --2- 172.21.15.94:0/341247440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc258105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 --2- 172.21.15.94:0/341247440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258103af0 0x7fc2581047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.533+0000 7fc25eee2640 1 -- 172.21.15.94:0/341247440 >> 172.21.15.94:0/341247440 conn(0x7fc2580fece0 msgr2=0x7fc258101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- 172.21.15.94:0/341247440 shutdown_connections 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- 172.21.15.94:0/341247440 wait complete. 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 Processor -- start 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- start start 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc25819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258109a30 0x7fc2581a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810f620 con 0x7fc258103af0 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810f4a0 con 0x7fc258109a30 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25eee2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc25810f7a0 con 0x7fc258104d30 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25cc7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25cc7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46896/0 (socket says 172.21.15.94:46896) 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.534+0000 7fc25cc7e640 1 -- 172.21.15.94:0/581873074 learned_addr learned my addr 172.21.15.94:0/581873074 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25d47f640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258109a30 0x7fc2581a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc257fff640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc25819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 -- 172.21.15.94:0/581873074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 msgr2=0x7fc25819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc25819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 -- 172.21.15.94:0/581873074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258109a30 msgr2=0x7fc2581a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258109a30 0x7fc2581a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 -- 172.21.15.94:0/581873074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2581a1cf0 con 0x7fc258103af0 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc257fff640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc25819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:34.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25cc7e640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00d740 tx=0x7fc24c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c017070 con 0x7fc258103af0 2024-04-25T14:07:34.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc24c00de50 con 0x7fc258103af0 2024-04-25T14:07:34.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581a1fe0 con 0x7fc258103af0 2024-04-25T14:07:34.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c003c20 con 0x7fc258103af0 2024-04-25T14:07:34.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.535+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2581a24a0 con 0x7fc258103af0 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.536+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc258108840 con 0x7fc258103af0 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.537+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc24c003dc0 con 0x7fc258103af0 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.538+0000 7fc255ffb640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 0x7fc2480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.538+0000 7fc257fff640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 0x7fc2480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.538+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fc24c0de650 con 0x7fc258103af0 2024-04-25T14:07:34.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.538+0000 7fc257fff640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 0x7fc2480b2080 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc244005fd0 tx=0x7fc244005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.543+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc24c0a7190 con 0x7fc258103af0 2024-04-25T14:07:34.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.721+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2580637a0 con 0x7fc258103af0 2024-04-25T14:07:34.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.721+0000 7fc255ffb640 1 -- 172.21.15.94:0/581873074 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc24c0a6a90 con 0x7fc258103af0 2024-04-25T14:07:34.721 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 msgr2=0x7fc2480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 0x7fc2480b2080 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc244005fd0 tx=0x7fc244005850 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 msgr2=0x7fc25819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00d740 tx=0x7fc24c00dc00 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 shutdown_connections 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fc2480afb70 0x7fc2480b2080 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc258109a30 0x7fc2581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc258104d30 0x7fc25819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 --2- 172.21.15.94:0/581873074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc258103af0 0x7fc25819a1b0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 >> 172.21.15.94:0/581873074 conn(0x7fc2580fece0 msgr2=0x7fc258105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 shutdown_connections 2024-04-25T14:07:34.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.724+0000 7fc25eee2640 1 -- 172.21.15.94:0/581873074 wait complete. 2024-04-25T14:07:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:34 smithi094 ceph-mon[29195]: pgmap v1124: 97 pgs: 97 active+clean; 132 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 3.0 MiB/s wr, 182 op/s 2024-04-25T14:07:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2899282333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:07:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1190002392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:07:34.725 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:34 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2359924245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:07:34.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T14:07:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:34 smithi138 ceph-mon[32977]: pgmap v1124: 97 pgs: 97 active+clean; 132 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 3.0 MiB/s wr, 182 op/s 2024-04-25T14:07:34.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2899282333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:07:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1190002392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:07:34.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2359924245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:07:34.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- 172.21.15.94:0/3246810426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 msgr2=0x7fef741027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 --2- 172.21.15.94:0/3246810426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef741027d0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fef64009900 tx=0x7fef6402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:34.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- 172.21.15.94:0/3246810426 shutdown_connections 2024-04-25T14:07:34.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 --2- 172.21.15.94:0/3246810426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74108a20 0x7fef7410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 --2- 172.21.15.94:0/3246810426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef741043d0 0x7fef74102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 --2- 172.21.15.94:0/3246810426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef741027d0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- 172.21.15.94:0/3246810426 >> 172.21.15.94:0/3246810426 conn(0x7fef740fd370 msgr2=0x7fef740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- 172.21.15.94:0/3246810426 shutdown_connections 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- 172.21.15.94:0/3246810426 wait complete. 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 Processor -- start 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.845+0000 7fef79f00640 1 -- start start 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74108a20 0x7fef7419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef7419f2b0 0x7fef741a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410f630 con 0x7fef74103ef0 2024-04-25T14:07:34.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410f4b0 con 0x7fef74108a20 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410f7b0 con 0x7fef7419f2b0 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46912/0 (socket says 172.21.15.94:46912) 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 -- 172.21.15.94:0/999543209 learned_addr learned my addr 172.21.15.94:0/999543209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 -- 172.21.15.94:0/999543209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef7419f2b0 msgr2=0x7fef741a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef7419f2b0 0x7fef741a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 -- 172.21.15.94:0/999543209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74108a20 msgr2=0x7fef7419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74108a20 0x7fef7419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 -- 172.21.15.94:0/999543209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741a1c90 con 0x7fef74103ef0 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef737fe640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef64031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6403f030 con 0x7fef74103ef0 2024-04-25T14:07:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef64003aa0 con 0x7fef74103ef0 2024-04-25T14:07:34.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741a1f20 con 0x7fef74103ef0 2024-04-25T14:07:34.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64035070 con 0x7fef74103ef0 2024-04-25T14:07:34.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.846+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef741a2380 con 0x7fef74103ef0 2024-04-25T14:07:34.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.847+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef74107120 con 0x7fef74103ef0 2024-04-25T14:07:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.849+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fef64042030 con 0x7fef74103ef0 2024-04-25T14:07:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.850+0000 7fef717fa640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 0x7fef600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.850+0000 7fef6bfff640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 0x7fef600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.850+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fef640ff750 con 0x7fef74103ef0 2024-04-25T14:07:34.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.850+0000 7fef6bfff640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 0x7fef600b2000 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005fd0 tx=0x7fef5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:34.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:34.854+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef640c8290 con 0x7fef74103ef0 2024-04-25T14:07:34.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:34 smithi137 ceph-mon[32568]: pgmap v1124: 97 pgs: 97 active+clean; 132 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 3.0 MiB/s wr, 182 op/s 2024-04-25T14:07:34.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2899282333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:07:34.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1190002392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:07:34.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:34 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2359924245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:07:35.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.081+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fef7402dd20 con 0x7fef74103ef0 2024-04-25T14:07:35.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.090+0000 7fef717fa640 1 -- 172.21.15.94:0/999543209 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+803124 (secure 0 0 0) 0x7fef640c7b90 con 0x7fef74103ef0 2024-04-25T14:07:35.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:07:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:35.102 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 20, 2024-04-25T14:07:35.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 14152, 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 449370799 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 449370799 2024-04-25T14:07:35.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.94:6801/449370799", 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-04-25T13:32:40.168036+0000", 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24106, 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "z", 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:07:35.105 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.106 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.110 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.114 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.120 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:07:35.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T14:07:35.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:35.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T14:07:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:35.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T14:07:35.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:35.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T14:07:35.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T14:07:35.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T14:07:35.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T14:07:35.151 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T14:07:35.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T14:07:35.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T14:07:35.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.156 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.157 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.158 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.159 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.160 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.165 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.172 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.173 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T14:07:35.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.175 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T14:07:35.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T14:07:35.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.178 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.179 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:07:35.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:07:35.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T14:07:35.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T14:07:35.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.193 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.197 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:07:35.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:07:35.205 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:07:35.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.208 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T14:07:35.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:07:35.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:07:35.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:07:35.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:07:35.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:07:35.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:07:35.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:07:35.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:07:35.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:07:35.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.230 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:07:35.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:07:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.235 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.236 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.240 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.241 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.242 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:07:35.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:07:35.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:07:35.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:07:35.246 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:07:35.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.248 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.249 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.250 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:07:35.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:07:35.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:07:35.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:35.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.269 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.273 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:35.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T14:07:35.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:35.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.286 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.287 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.289 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T14:07:35.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T14:07:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:07:35.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T14:07:35.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T14:07:35.300 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.301 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:35.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T14:07:35.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.319 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.328 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.329 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.337 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.338 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:07:35.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T14:07:35.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T14:07:35.349 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.350 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.351 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.353 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.354 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T14:07:35.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T14:07:35.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T14:07:35.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.358 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T14:07:35.360 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.362 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.371 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.374 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.375 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.379 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.384 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.385 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.390 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.396 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.399 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:07:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T14:07:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T14:07:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T14:07:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T14:07:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:07:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T14:07:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T14:07:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T14:07:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T14:07:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T14:07:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T14:07:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:07:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24113, 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:07:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:07:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T14:07:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T14:07:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:07:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:07:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T14:07:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T14:07:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T14:07:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T14:07:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T14:07:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:07:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T14:07:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T14:07:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T14:07:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T14:07:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:07:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:07:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T14:07:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:07:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T14:07:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:07:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:07:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:07:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:07:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:07:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:07:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:07:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:07:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:07:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:07:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:07:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:07:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:07:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:07:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:07:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:07:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T14:07:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T14:07:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T14:07:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T14:07:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T14:07:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T14:07:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T14:07:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T14:07:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T14:07:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T14:07:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:07:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:07:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:07:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:07:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:07:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T14:07:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T14:07:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T14:07:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T14:07:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:07:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T14:07:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T14:07:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T14:07:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T14:07:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard", 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T14:07:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T14:07:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:07:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:07:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T14:07:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:07:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:07:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:07:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:07:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:07:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T14:07:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:07:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T14:07:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T14:07:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:07:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:07:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:07:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:07:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T14:07:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T14:07:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T14:07:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T14:07:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T14:07:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T14:07:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T14:07:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T14:07:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T14:07:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:07:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T14:07:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T14:07:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T14:07:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:07:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:07:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:07:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:07:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:07:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:07:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:07:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T14:07:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T14:07:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:07:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T14:07:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:07:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T14:07:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T14:07:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T14:07:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T14:07:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T14:07:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:07:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T14:07:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T14:07:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.387 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.393 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.396 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T14:07:36.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.404 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.409 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.411 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.413 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.417 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.418 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.422 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.423 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.424 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T14:07:36.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.426 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.427 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.428 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.429 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.430 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T14:07:36.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T14:07:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:07:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:07:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T14:07:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T14:07:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T14:07:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T14:07:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T14:07:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T14:07:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T14:07:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "services": { 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard": "https://172.21.15.94:8443/" 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T14:07:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T14:07:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T14:07:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 3, 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T14:07:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3351813029 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T14:07:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1155370168 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3124826398 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2188866588 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:07:36.524 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:36.525 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/581873074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:07:36.525 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:35 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/999543209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:07:36.525 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/581873074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:07:36.525 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/999543209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 msgr2=0x7fef600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 0x7fef600b2000 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005fd0 tx=0x7fef5c005850 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 msgr2=0x7fef74199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef64031a50 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 shutdown_connections 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fef600afaf0 0x7fef600b2000 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef7419f2b0 0x7fef741a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74108a20 0x7fef7419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 --2- 172.21.15.94:0/999543209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef74103ef0 0x7fef74199df0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.095+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 >> 172.21.15.94:0/999543209 conn(0x7fef740fd370 msgr2=0x7fef740ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:36.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.096+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 shutdown_connections 2024-04-25T14:07:36.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:07:35.096+0000 7fef79f00640 1 -- 172.21.15.94:0/999543209 wait complete. 2024-04-25T14:07:36.534 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T14:07:36.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T14:07:36.534 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/581873074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:07:36.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:35 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/999543209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:07:36.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:36 smithi094 ceph-mon[29195]: pgmap v1125: 97 pgs: 97 active+clean; 127 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-04-25T14:07:36.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:36 smithi138 ceph-mon[32977]: pgmap v1125: 97 pgs: 97 active+clean; 127 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-04-25T14:07:36.774 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:07:36.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:36 smithi137 ceph-mon[32568]: pgmap v1125: 97 pgs: 97 active+clean; 127 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 3.6 MiB/s wr, 162 op/s 2024-04-25T14:07:38.595 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3051, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 3053, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 9307771, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 970219, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T14:07:38.596 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.239081523, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.076292441, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.065322211, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.077400466, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3051, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.082490559, 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000027037 2024-04-25T14:07:38.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:38.598 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1893, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2232, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 16379311, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1289052, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.169956642, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.053948594, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.049447992, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.052753103, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2230, 2024-04-25T14:07:38.599 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.072007283, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000032290 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 687328 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:38.600 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 763, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 738, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 5273614, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 366744, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.081320116, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.030294189, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.036611975, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.014274426, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 738, 2024-04-25T14:07:38.601 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.020825447, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000028218 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1009680, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 278976 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T14:07:38.602 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 8842, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.200826713, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000022712 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:38.603 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.604 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 3071, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 2, 2024-04-25T14:07:38.605 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 3069, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 3068, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 6, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:38.606 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-25T14:07:38.607 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:38.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T14:07:38.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T14:07:38.610 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T14:07:38.611 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 21995, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 28, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 3, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 32683147, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 32681072, 2024-04-25T14:07:38.612 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 21995 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 87, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 3022, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 2935, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 20, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 36, 2024-04-25T14:07:38.613 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 16, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 3075, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 3094, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 19, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 125540, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215440 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T14:07:38.614 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.616 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.617 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.619 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.620 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.620 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T14:07:38.620 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.620 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.622 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.623 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.624 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.625 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:38.626 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T14:07:38.627 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.628 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.629 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 1, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 0, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-25T14:07:38.631 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 2657, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:38.632 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:38.633 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:38.634 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 218679, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 141, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:38.635 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 170, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 2400737, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 21554, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T14:07:38.636 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 5219, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 5219, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5219, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 5.004918651, 2024-04-25T14:07:38.637 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000958980 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5219, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 5220 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 5219, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 5219, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 3112, 2024-04-25T14:07:38.638 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 2105, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-25T14:07:38.639 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T14:07:38.640 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 88, 2024-04-25T14:07:38.641 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 9, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 8, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 10, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 2, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-25T14:07:38.642 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T14:07:38.643 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 5703, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 30419728, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 5703, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:38.644 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5703, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 30419728, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:38.645 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 5219, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 28489316, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5219, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 28489316, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.646 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 5219, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 5219, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5219, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 5219, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.647 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:38.648 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x555b909668f0": { 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:38.650 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:38.651 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:38.651 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:38.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:38.651 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:38.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:38 smithi094 ceph-mon[29195]: pgmap v1126: 97 pgs: 97 active+clean; 124 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 3.0 MiB/s wr, 111 op/s 2024-04-25T14:07:38.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:38 smithi138 ceph-mon[32977]: pgmap v1126: 97 pgs: 97 active+clean; 124 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 3.0 MiB/s wr, 111 op/s 2024-04-25T14:07:38.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:38 smithi137 ceph-mon[32568]: pgmap v1126: 97 pgs: 97 active+clean; 124 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 3.0 MiB/s wr, 111 op/s 2024-04-25T14:07:39.299 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T14:07:39.299 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T14:07:39.299 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T14:07:39.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T14:07:39.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:39 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:39.600 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:07:39.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:39 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:39.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:39 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:40.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:40 smithi094 ceph-mon[29195]: pgmap v1127: 97 pgs: 97 active+clean; 122 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-04-25T14:07:40.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:40 smithi138 ceph-mon[32977]: pgmap v1127: 97 pgs: 97 active+clean; 122 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-04-25T14:07:40.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:40 smithi137 ceph-mon[32568]: pgmap v1127: 97 pgs: 97 active+clean; 122 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001161014, 2024-04-25T14:07:41.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000250835, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:07:41.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 345, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 218000, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.021715418, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.017386678, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001582472, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 343, 2024-04-25T14:07:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013900947, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000040527 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 218000 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 460, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 453, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 873072, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 279760, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.057711253, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.023111974, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.026935013, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008089770, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 453, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.016028474, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000035382 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 873072, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 279760 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T14:07:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148744, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T14:07:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T14:07:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 458, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 805608, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 458, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 458, 2024-04-25T14:07:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 805608, 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:41.446 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:41 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:07:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:41 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:07:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:41 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' 2024-04-25T14:07:42.158 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T14:07:42.159 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T14:07:42.159 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T14:07:42.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T14:07:42.442 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:07:42.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:42 smithi138 ceph-mon[32977]: pgmap v1128: 97 pgs: 97 active+clean; 110 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.4 MiB/s wr, 126 op/s 2024-04-25T14:07:42.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:42 smithi094 ceph-mon[29195]: pgmap v1128: 97 pgs: 97 active+clean; 110 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.4 MiB/s wr, 126 op/s 2024-04-25T14:07:42.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:42 smithi137 ceph-mon[32568]: pgmap v1128: 97 pgs: 97 active+clean; 110 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.4 MiB/s wr, 126 op/s 2024-04-25T14:07:43.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:43 smithi094 ceph-mon[29195]: pgmap v1129: 97 pgs: 97 active+clean; 109 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 109 op/s 2024-04-25T14:07:43.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:43 smithi138 ceph-mon[32977]: pgmap v1129: 97 pgs: 97 active+clean; 109 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 109 op/s 2024-04-25T14:07:43.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:43 smithi137 ceph-mon[32568]: pgmap v1129: 97 pgs: 97 active+clean; 109 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 109 op/s 2024-04-25T14:07:44.112 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:44.112 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2064, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2064, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 4856495, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 655529, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.182211116, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.071192069, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.042814606, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.052290440, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2063, 2024-04-25T14:07:44.113 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.068801101, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000033350 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3128, 2024-04-25T14:07:44.114 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 3468, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 16097036, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1681947, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.208072795, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.064036264, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.061364872, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.067034002, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3466, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.082559029, 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000023819 2024-04-25T14:07:44.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 687504 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 619, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 596, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1040806, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 322431, 2024-04-25T14:07:44.116 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.070677866, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.026176022, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.033658095, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.012271721, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 595, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.021246662, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000035708 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.117 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1007232, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 279808 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 9041, 2024-04-25T14:07:44.118 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.192976498, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000021344 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-25T14:07:44.119 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T14:07:44.120 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 2081, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 2081, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 2081, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 5, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:44.121 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-25T14:07:44.122 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:44.123 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:44.124 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T14:07:44.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 14790, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T14:07:44.126 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 20, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 3, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 23689792, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 23689193, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.127 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 14790 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 65, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 2044, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 1979, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 18, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 32, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 14, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 2084, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 2100, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 16, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-25T14:07:44.128 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 124064, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215440 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.129 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.130 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.131 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:44.132 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.134 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.138 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.139 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.140 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 2, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 3761, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:44.141 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:44.142 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:44.143 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 200208, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 109, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:44.144 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 170, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 1654106, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 15194, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:44.145 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 5321, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 5321, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5321, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 6.562893286, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001233394 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.146 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5321, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 5322 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 5321, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 5321, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 3221, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 2098, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-25T14:07:44.147 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:44.148 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T14:07:44.149 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 88, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 9, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 6, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 2, 2024-04-25T14:07:44.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T14:07:44.151 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 5807, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 21443905, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 5807, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5807, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 21443905, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 5321, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 19497319, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5321, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 19497319, 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:44.154 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 5321, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 5321, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5321, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 5321, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:44.156 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:44.157 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x555e49c68c30": { 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:44.158 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:44.159 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:44.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:44.159 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:44.901 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T14:07:44.901 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T14:07:44.901 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T14:07:44.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T14:07:45.139 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:07:45.974 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:45 smithi094 ceph-mon[29195]: pgmap v1130: 97 pgs: 97 active+clean; 94 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-04-25T14:07:46.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:45 smithi137 ceph-mon[32568]: pgmap v1130: 97 pgs: 97 active+clean; 94 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-04-25T14:07:46.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:45 smithi138 ceph-mon[32977]: pgmap v1130: 97 pgs: 97 active+clean; 94 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.5 MiB/s wr, 140 op/s 2024-04-25T14:07:47.305 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2992, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001488407, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000033635, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000621538, 2024-04-25T14:07:47.306 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000789606, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000263202 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:47.307 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 347, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 218352, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.021906605, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.017470470, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001719744, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 345, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013479832, 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000039071 2024-04-25T14:07:47.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 218352 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 458, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 453, 2024-04-25T14:07:47.309 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 870624, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 280592, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.059523203, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.023743791, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.028605849, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008174209, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 452, 2024-04-25T14:07:47.310 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.016254718, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000035961 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 870624, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 280592 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-25T14:07:47.311 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:47.312 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:47.313 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148704, 2024-04-25T14:07:47.314 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 29, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:47.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 460, 2024-04-25T14:07:47.316 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 805868, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 460, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 460, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 805868, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:07:47.317 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:07:48.016 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T14:07:48.016 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T14:07:48.016 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T14:07:48.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T14:07:48.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:47 smithi094 ceph-mon[29195]: pgmap v1131: 97 pgs: 97 active+clean; 93 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-04-25T14:07:48.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:47 smithi138 ceph-mon[32977]: pgmap v1131: 97 pgs: 97 active+clean; 93 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-04-25T14:07:48.305 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:48.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:47 smithi137 ceph-mon[32568]: pgmap v1131: 97 pgs: 97 active+clean; 93 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2024-04-25T14:07:50.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:49 smithi137 ceph-mon[32568]: pgmap v1132: 97 pgs: 97 active+clean; 90 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T14:07:50.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:49 smithi138 ceph-mon[32977]: pgmap v1132: 97 pgs: 97 active+clean; 90 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T14:07:50.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:49 smithi094 ceph-mon[29195]: pgmap v1132: 97 pgs: 97 active+clean; 90 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 332724, 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 544186, 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 69328254, 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 494022201, 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 43, 2024-04-25T14:07:50.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 27.685148750, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 11.448126541, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 12.892604546, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 3.885255700, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 544188, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 74.047213484, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000136069 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 663891, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.138172519, 2024-04-25T14:07:50.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000208 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1054720, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 344080 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1368239, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2065310, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 413490361, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 2251238241, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 16, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 7, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 77.498149959, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 32.585923975, 2024-04-25T14:07:50.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 47.568997707, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 7.308843504, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2065304, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 206.941313348, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000100198 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2877070, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.452611516, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000157 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 54032, 2024-04-25T14:07:50.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 730608 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 391612, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 604430, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 93623617, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 761664446, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 584, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 10, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 35.412394470, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 15.992304854, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 14.341517193, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 6.088471189, 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:50.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 604430, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 92.185927148, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000152517 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 708784, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.137205341, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000193 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 45536, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 4368 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-25T14:07:50.593 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 298206, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 633.733987639, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002125155 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 266669, 2024-04-25T14:07:50.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 83.641573722, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000313653 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 862956, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 858545, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 858545, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6973.006419352, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008121888 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 4411, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:50.595 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 12986, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 47, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 63, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 4014, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 1722, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 2292, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 1554, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 141, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 35, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 35, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 15, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 1267235, 2024-04-25T14:07:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 735987, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 4411, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 8, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 9160, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 9, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 94, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:50.597 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 18785, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 18785, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 3147, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 48, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 3099, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 15638, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 19, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 41652, 2024-04-25T14:07:50.598 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 541648, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 238643, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 111153, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 51236, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 531403 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 48, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 8, 2024-04-25T14:07:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 128, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 285, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 142338, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 142053, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:50.600 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 48, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 10, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 110, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 668671, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 663959, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 663959, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 4712, 2024-04-25T14:07:50.601 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 6935, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 818, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 812, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 812, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 6, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 1, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 9, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 1421051907, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 1429160437, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-25T14:07:50.602 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 182192, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1927.186613667, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.010577778 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 525, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 142496, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 141971, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 110, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 574, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 464, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 4014, 2024-04-25T14:07:50.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 337106, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 333092, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 35, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 155118, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 155083, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 176404, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 231824 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 1072760, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 13223, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 862956, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 20392, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 26468, 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-25T14:07:50.604 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 142302, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 982.668780550, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006905516 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10619, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 361.507551915, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.034043464 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.605 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 311053, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1739.174003486, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005591246 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000376989, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000188494 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.607 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.608 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 119, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.045805832, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008788284 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016331449, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001633144 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 218008, 2024-04-25T14:07:50.610 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2666.961769227, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.012233320 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 29201, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 209.450949686, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007172732 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.611 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 124, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.876460280, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007068228 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:50.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 126, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.991965195, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.039618771 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4748, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 30.697210270, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006465292 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.018113719, 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002264214 2024-04-25T14:07:50.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 142225, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 975.597100754, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006859533 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 3, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 55777, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 55777, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1564, 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:50.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:50.618 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 4871818, 2024-04-25T14:07:50.619 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 1984, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 170, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:50.620 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 5742612, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 170346, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 437380, 2024-04-25T14:07:50.621 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 1538169360, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 31, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 437380, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 437380, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 16119.372426380, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.036854388 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 437380, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 613138 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 437380, 2024-04-25T14:07:50.622 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 21956, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 415424, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 12418, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 70060, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 3123, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 185267, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 1812, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-25T14:07:50.623 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 45, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 143580, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 18833, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 140120, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-25T14:07:50.624 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-25T14:07:50.625 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 88, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 10, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 27160, 2024-04-25T14:07:50.626 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 10720 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-25T14:07:50.627 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 142053, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 142053, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2092571, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 388065988, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:50.628 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2092571, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2092571, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 388065988, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:50.629 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 437380, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1552516713, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 437380, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1552516713, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.630 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 437380, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 437380, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 437380, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 437380, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.631 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 142053, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 14347353, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 142053, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3075, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 14347353, 2024-04-25T14:07:50.632 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55c0661708f0": { 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 600341, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 668671, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1424966133, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 668671, 2024-04-25T14:07:50.633 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 182192, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1424365792, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:50.634 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T14:07:51.582 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-04-25T14:07:51.583 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T14:07:51.583 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T14:07:51.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T14:07:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:52.238 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:51 smithi137 ceph-mon[32568]: pgmap v1133: 97 pgs: 97 active+clean; 86 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-04-25T14:07:52.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:51 smithi138 ceph-mon[32977]: pgmap v1133: 97 pgs: 97 active+clean; 86 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-04-25T14:07:52.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:51 smithi094 ceph-mon[29195]: pgmap v1133: 97 pgs: 97 active+clean; 86 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.2 MiB/s wr, 131 op/s 2024-04-25T14:07:53.838 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 62471, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 77941, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 18106890, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 55130462, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 62, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 3.848781328, 2024-04-25T14:07:53.839 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 1.498915223, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 2.759738535, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.101811313, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 77940, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.755458012, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000022523 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 127501, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.023654099, 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000185 2024-04-25T14:07:53.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 5131, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 5476, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1407370, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 19470139, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 58, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 7, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.436286583, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.154297343, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.106114615, 2024-04-25T14:07:53.841 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.144966174, 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5474, 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.211669587, 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000038668 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 51, 2024-04-25T14:07:53.842 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000005845, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000114 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 782592 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 244561, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 138778, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 66791177, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 29953694, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-04-25T14:07:53.843 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 13.599451649, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 5.837292674, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 10.477062123, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.086134356, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 138777, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.772933935, 2024-04-25T14:07:53.844 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000019981 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 375896, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.066425683, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000176 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1031696, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 329888 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T14:07:53.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11874, 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 8.191329406, 2024-04-25T14:07:53.846 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000689854 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6548, 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.362203294, 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000055315 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-25T14:07:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 46188, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 17661, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 17661, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 94.975528364, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005377698 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 28527, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:53.848 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 42, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 2, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 40, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 38, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 76, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 7, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 74004, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 20188, 2024-04-25T14:07:53.849 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 28527, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 17, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 913, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 4, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 40, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 816, 2024-04-25T14:07:53.850 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 138, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 33223382, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 816, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 816, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:07:53.851 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 891, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 14501, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 7931, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 3275, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 1688, 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 16646 2024-04-25T14:07:53.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 3, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 43, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 28, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 36, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 8, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 5, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 2988, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 2983, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-25T14:07:53.853 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:53.854 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 22121, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 22121, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 28, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 28, 2024-04-25T14:07:53.855 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 3, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 32852095, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5556, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 40.249870932, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007244397 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-25T14:07:53.856 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 47, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 3047, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 3000, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 29, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 68, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 39, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 46, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 5799, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 5753, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 3752, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 3752, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 131336, 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223632 2024-04-25T14:07:53.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 64773, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 7, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 46188, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 489, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 7040, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2957, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 46.632829297, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.015770317 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-04-25T14:07:53.858 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010573221, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003524407 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-25T14:07:53.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5823, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 8.223546802, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001412252 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.860 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.862 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-25T14:07:53.862 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.862 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.862 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.863 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4974, 2024-04-25T14:07:53.864 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3.504088620, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000704481 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 913, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 14.120152596, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.015465665 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:53.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016501995, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005500665 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2988, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 22.467835833, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007519356 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.870 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 2115, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 2115, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1469, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T14:07:53.871 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:53.872 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:53.873 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 618472, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 257, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 1408, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 16, 2024-04-25T14:07:53.874 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 170, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 260526, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 7720, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:53.875 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 11069, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 29396060, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 33, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 11069, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11069, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 70.263749571, 2024-04-25T14:07:53.876 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006347795 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11069, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 11787 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 11069, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 1263, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 9806, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 447, 2024-04-25T14:07:53.877 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 6003, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 286, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 3001, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 816, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:53.878 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-25T14:07:53.879 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 88, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 9, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 17, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 5, 2024-04-25T14:07:53.880 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 935, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 269 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-25T14:07:53.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 2983, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 2983, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:53.882 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 312159, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 58179428, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 312159, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 312159, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 58179428, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 11069, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 29697343, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 11069, 2024-04-25T14:07:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 29697343, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 11069, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 11069, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 11069, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 11069, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2983, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 301283, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2983, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 447, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 301283, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55fdf9fe4820": { 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 22345, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 22121, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 28657791, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 22121, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5556, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 28635446, 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:53.889 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T14:07:54.133 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:53 smithi137 ceph-mon[32568]: pgmap v1134: 97 pgs: 97 active+clean; 82 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-04-25T14:07:54.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:53 smithi138 ceph-mon[32977]: pgmap v1134: 97 pgs: 97 active+clean; 82 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-04-25T14:07:54.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:53 smithi094 ceph-mon[29195]: pgmap v1134: 97 pgs: 97 active+clean; 82 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-04-25T14:07:54.764 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T14:07:54.764 INFO:tasks.check_counter:Daemon mds.e mds.imported=4 2024-04-25T14:07:54.764 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T14:07:54.764 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T14:07:55.006 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:56.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:55 smithi137 ceph-mon[32568]: pgmap v1135: 97 pgs: 97 active+clean; 71 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-04-25T14:07:56.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:55 smithi138 ceph-mon[32977]: pgmap v1135: 97 pgs: 97 active+clean; 71 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-04-25T14:07:56.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:55 smithi094 ceph-mon[29195]: pgmap v1135: 97 pgs: 97 active+clean; 71 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 215143, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 106956, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 57984610, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 17979479, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-04-25T14:07:56.695 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 11.592508177, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 4.937784115, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 9.195308279, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.014541047, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 106954, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.102711898, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000019659 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 320743, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.060875877, 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000189 2024-04-25T14:07:56.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 28877, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 29226, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 7719396, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 3735868, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 20, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 1.660275554, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.587652025, 2024-04-25T14:07:56.697 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 1.389444849, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.001535094, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 29224, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.434398338, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000014864 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 57593, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010864904, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000188 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 696624 2024-04-25T14:07:56.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 683, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 652, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1068131, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1353990, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 95, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.094199554, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.025696823, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.059711303, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.009670426, 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:56.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 652, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.033118868, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000050795 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 91, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000010772, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000118 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1042832, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 306736 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.700 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 233, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.024846511, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000106637 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 18, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000480738, 2024-04-25T14:07:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000026707 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 28535, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 8, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.033562652, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.004195331 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 28527, 2024-04-25T14:07:56.702 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 15, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 15, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 14, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 75, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 4, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 28580, 2024-04-25T14:07:56.703 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 31, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 28527, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 15, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 1, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 3, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:56.704 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 66, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 138, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 33223382, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 66, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 66, 2024-04-25T14:07:56.705 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 1, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 7, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 71, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 66, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 4, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 7 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 46, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 30, 2024-04-25T14:07:56.706 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 36, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 3, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 3, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:56.707 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 108, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 108, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 2, 2024-04-25T14:07:56.708 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4319110, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 57, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.498203111, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008740405 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 20, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 60, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 40, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 26, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 66, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 40, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 16, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 98, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 82, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-04-25T14:07:56.710 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 4, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 4, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 124736, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223632 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 28540, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 5, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 28535, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 24, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 45, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.712 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001122671, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000280667 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-25T14:07:56.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000352286, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000352286 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.718 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.719 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-25T14:07:56.721 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.032087695, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.010695898 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-04-25T14:07:56.722 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 3, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 3, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1472, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:56.723 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:56.724 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:56.725 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 302281, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 122, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:56.726 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 170, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 91222, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 394, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-25T14:07:56.727 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 202, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 139918, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 33, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 202, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 202, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.197978582, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005930587 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 202, 2024-04-25T14:07:56.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 230 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 202, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 67, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 135, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 1, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 58, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 33, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-25T14:07:56.729 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 15, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 66, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-25T14:07:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-25T14:07:56.731 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 88, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 9, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 11, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 13, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 2, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-25T14:07:56.732 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 24, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 3 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.733 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 3, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 3, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 3, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 3, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 244699, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 44717443, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 244699, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 244699, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 44717443, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.735 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 202, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 140221, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 202, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 140221, 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:56.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 202, 2024-04-25T14:07:56.737 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 202, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 202, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 202, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:56.738 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 303, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 303, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:56.739 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5602684fc680": { 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 108, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 124806, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 108, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 57, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 124806, 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:56.740 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:56.741 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:56.741 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:56.741 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.741 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:56.741 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T14:07:57.453 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T14:07:57.454 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-04-25T14:07:57.454 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T14:07:57.454 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T14:07:57.694 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:07:58.088 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:57 smithi137 ceph-mon[32568]: pgmap v1136: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-04-25T14:07:58.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:57 smithi138 ceph-mon[32977]: pgmap v1136: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-04-25T14:07:58.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:57 smithi094 ceph-mon[29195]: pgmap v1136: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-04-25T14:07:59.432 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T14:07:59.432 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:07:59.432 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2178, 2024-04-25T14:07:59.432 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2155, 2024-04-25T14:07:59.432 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1575504, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 821080, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.201955599, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.072130649, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.066212800, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.054124444, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2154, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.064759785, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000030064 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:59.433 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1009680, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 285568 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 8, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 354, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3868, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 784794, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:59.434 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.016226712, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.012028922, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001766708, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000046175, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 353, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.013083086, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000037062 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 784096 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3343, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3345, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 974773, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1063081, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.179495882, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.045039916, 2024-04-25T14:07:59.436 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.046810161, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.077142148, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3343, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.076086097, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000022759 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:07:59.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8423, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.191333754, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000022715 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-25T14:07:59.438 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 8, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 8, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 8, 2024-04-25T14:07:59.440 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 3, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-25T14:07:59.441 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:07:59.442 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.443 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-25T14:07:59.444 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-25T14:07:59.445 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 108, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-25T14:07:59.446 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4319110, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 4317972, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 108 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 7, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 19, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 12, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 5, 2024-04-25T14:07:59.447 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 15, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 10, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 9, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 19, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 10, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 122132, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215440 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-04-25T14:07:59.448 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-25T14:07:59.449 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.450 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.453 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.455 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.456 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.457 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-25T14:07:59.458 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.459 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-25T14:07:59.460 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2658, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:07:59.461 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:07:59.462 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:07:59.463 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 182164, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 50, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8104, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 170, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:07:59.464 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 29041, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 108, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 5029, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-04-25T14:07:59.465 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 5029, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5029, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5.453902390, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001084490 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5029, 2024-04-25T14:07:59.466 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5030 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 5029, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 5029, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 3289, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 1738, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-25T14:07:59.467 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-25T14:07:59.468 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-25T14:07:59.469 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 88, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 9, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 4, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 5, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 1, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-04-25T14:07:59.470 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:07:59.471 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5525, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2028582, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:59.472 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5525, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5525, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2028582, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:59.473 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 5029, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 124811, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5029, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 124811, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 5029, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 5029, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5029, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5029, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.475 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:59.476 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55e3a63e68f0": { 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T14:07:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T14:07:59.478 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T14:08:00.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:07:59 smithi137 ceph-mon[32568]: pgmap v1137: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-04-25T14:08:00.141 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T14:08:00.141 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T14:08:00.141 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T14:08:00.141 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T14:08:00.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:07:59 smithi138 ceph-mon[32977]: pgmap v1137: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-04-25T14:08:00.392 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:00.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:07:59 smithi094 ceph-mon[29195]: pgmap v1137: 97 pgs: 97 active+clean; 67 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-04-25T14:08:02.033 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:01 smithi138 ceph-mon[32977]: pgmap v1138: 97 pgs: 97 active+clean; 66 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 900 KiB/s wr, 76 op/s 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 229163, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 125893, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 61989796, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 34201508, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 14, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 11.708780384, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 4.224763823, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 10.259716029, 2024-04-25T14:08:02.180 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.151460280, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 125891, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.053739213, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000016313 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 344943, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.058930724, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000170 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.181 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1803, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1941, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1484201, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 1680508, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 59, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.154903373, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.050009630, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.065268120, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.034692570, 2024-04-25T14:08:02.182 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1941, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.068817464, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000035454 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 73, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000010571, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000144 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1028768, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 701456 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.183 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 23676, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 34371, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 7178991, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 46980762, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 58, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 9, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 1.595856081, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.664752429, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.981707818, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.093857399, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 34369, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.743638574, 2024-04-25T14:08:02.184 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000021636 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 47705, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.008749634, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000183 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 402752 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.185 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8774, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.379787058, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000499177 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4963, 2024-04-25T14:08:02.186 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.231506263, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000046646 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 10930, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 10929, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 10929, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 57.562102968, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005266913 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-25T14:08:02.187 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 1, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 15, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 15, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 14, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 75, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-04-25T14:08:02.188 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 6, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 27258, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 10799, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 16, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 349, 2024-04-25T14:08:02.189 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 3, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 34, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 888, 2024-04-25T14:08:02.190 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 1, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 138, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 33223382, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 888, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:08:02.191 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 888, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 824, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 10487, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 5836, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 2154, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 1305, 2024-04-25T14:08:02.192 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 10370 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 9, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 39, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 26, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 31, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 9, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:08:02.193 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 2013, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 2013, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:08:02.194 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:08:02.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.196 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 14860, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 14860, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 20, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 20, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 3, 2024-04-25T14:08:02.197 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 23764974, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3923, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 36.552075353, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.009317378 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-25T14:08:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 22, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 2071, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 2049, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 28, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 65, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 37, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 18, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 3730, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 3712, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 2771, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 2771, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 131120, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223632 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.199 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 22712, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 6, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 10930, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 212, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 4500, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1979, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 30.922046194, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.015625086 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.011865850, 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003955283 2024-04-25T14:08:02.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3131, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 3.148919696, 2024-04-25T14:08:02.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001005723 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-25T14:08:02.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.204 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3452, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.633023869, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000762753 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 351, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 5.369681465, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.015298237 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.207 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.208 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.210 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2013, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 15.476565894, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007688308 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:08:02.211 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-25T14:08:02.212 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 2, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 1296, 2024-04-25T14:08:02.213 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 1296, 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1573, 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T14:08:02.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:08:02.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:08:02.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T14:08:02.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 308948, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 138, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 528, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 6, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:08:02.218 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8104, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 170, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:08:02.219 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 100696, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 534, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 8530, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 20124598, 2024-04-25T14:08:02.220 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 8530, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8530, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 61.572755048, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007218376 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8530, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 9239 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 8530, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 1322, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 7208, 2024-04-25T14:08:02.221 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 434, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 4357, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 289, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 2029, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-25T14:08:02.222 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 888, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-25T14:08:02.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-25T14:08:02.224 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 88, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-25T14:08:02.225 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 9, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 15, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 3, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 949, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 263 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-25T14:08:02.226 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 6, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 6, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 2013, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 2013, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:08:02.227 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 254638, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 47703919, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 254638, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:02.228 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 254638, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 47703919, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:02.229 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 8530, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 20327911, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 8530, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 20327911, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:02.230 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 8530, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 8530, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 8530, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 8530, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2013, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 203313, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2013, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 434, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 203313, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.232 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55d4ff8f8680": { 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 14860, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 19570670, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 14860, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3923, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 19570670, 2024-04-25T14:08:02.233 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:02.234 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T14:08:02.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:01 smithi137 ceph-mon[32568]: pgmap v1138: 97 pgs: 97 active+clean; 66 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 900 KiB/s wr, 76 op/s 2024-04-25T14:08:02.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:01 smithi094 ceph-mon[29195]: pgmap v1138: 97 pgs: 97 active+clean; 66 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 900 KiB/s wr, 76 op/s 2024-04-25T14:08:03.142 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T14:08:03.143 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-04-25T14:08:03.143 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T14:08:03.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T14:08:03.396 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:04.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:03 smithi138 ceph-mon[32977]: pgmap v1139: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 283 KiB/s wr, 48 op/s 2024-04-25T14:08:04.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:03 smithi137 ceph-mon[32568]: pgmap v1139: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 283 KiB/s wr, 48 op/s 2024-04-25T14:08:04.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:03 smithi094 ceph-mon[29195]: pgmap v1139: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 283 KiB/s wr, 48 op/s 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1238, 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1240, 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 184038719, 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 393694, 2024-04-25T14:08:05.174 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 8, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.191200303, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.027192816, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.122137601, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.032754272, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1238, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.033138813, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000026768 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.175 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 888, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 882, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 189171818, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 323950, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-04-25T14:08:05.176 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.145712186, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.022264248, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.100362533, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.018834158, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 882, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.029440105, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000033378 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.177 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 232912, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 94288 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 767, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 881, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 222448788, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 532565, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.161321703, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.018249911, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.120237337, 2024-04-25T14:08:05.178 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.017393169, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 879, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.021703278, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000024690 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 288976 2024-04-25T14:08:05.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4247, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.585536593, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000137870 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-25T14:08:05.180 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.181 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 60427, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 41318, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 19109, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 2358, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-04-25T14:08:05.182 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 11, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-04-25T14:08:05.183 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-04-25T14:08:05.184 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-04-25T14:08:05.185 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-25T14:08:05.186 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 279917, 2024-04-25T14:08:05.187 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 6, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 38, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 1421051907, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 1429387008, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 1429385870, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 279917 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 44, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 60013, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 59969, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 104, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 341, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 237, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 60427, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 61620, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 1193, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-04-25T14:08:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 175752, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 215416 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.190 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.191 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.192 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.193 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-25T14:08:05.195 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.196 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-25T14:08:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.198 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.200 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.201 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 0, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 0, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 2674, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.202 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:08:05.203 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T14:08:05.204 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 240830, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 135, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:08:05.205 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8104, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 170, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 42862324, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 249294, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-25T14:08:05.206 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 2726, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 26, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 4, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 2726, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2726, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 7.757844307, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002845870 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2726, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2727 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.207 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 2726, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 2726, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 1561, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 1162, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-04-25T14:08:05.208 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 2, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-25T14:08:05.209 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-25T14:08:05.210 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 11, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 11, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-25T14:08:05.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:05.212 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2889, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 595387995, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2889, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2889, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 595387995, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.213 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2726, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 594675739, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2726, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 594675739, 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:05.214 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2726, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 2726, 2024-04-25T14:08:05.215 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2726, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2726, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:05.216 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x558cbcd208f0": { 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:05.217 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:05.219 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T14:08:06.019 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T14:08:06.020 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T14:08:06.020 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T14:08:06.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T14:08:06.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:06 smithi138 ceph-mon[32977]: pgmap v1140: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-04-25T14:08:06.257 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:06.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:06 smithi137 ceph-mon[32568]: pgmap v1140: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-04-25T14:08:06.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:06 smithi094 ceph-mon[29195]: pgmap v1140: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 178 KiB/s wr, 43 op/s 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 222595, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 115475, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 59972913, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 24606205, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 51, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 11.302237591, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 4.101972697, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 10.116163938, 2024-04-25T14:08:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.059080753, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 115478, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.830186422, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000015848 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 334051, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.057016966, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000170 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:08:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1348, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1693, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 368556, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 3386692, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 67, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 8, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.139357907, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.047822844, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.042388820, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.037914765, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1691, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.069393254, 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000041036 2024-04-25T14:08:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 56, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000006593, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000117 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 785664 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 35563, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 39113, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 10648747, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 10395187, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 13, 2024-04-25T14:08:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 2.527363698, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.911390642, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 2.129435466, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.030964977, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 39108, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.654823050, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000016743 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 70878, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.013299308, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000187 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1036256, 2024-04-25T14:08:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 307408 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3431, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.650869424, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000481162 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-25T14:08:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2455, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.089018920, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000036260 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 32391, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 4231, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4231, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 22.066204089, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005215363 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-25T14:08:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 28160, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 18, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 2, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 16, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 15, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 75, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 4, 2024-04-25T14:08:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 40081, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 6822, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 28159, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 16, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 420, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 1, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 17, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 109, 2024-04-25T14:08:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 138, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 33223382, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 109, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 109, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 23, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 2724, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:08:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 910, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 425, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 378, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 4444 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 46, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 30, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 37, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 3, 2024-04-25T14:08:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 635, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 635, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-04-25T14:08:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 4478, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4478, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 7, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-25T14:08:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 7, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 9767012, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1462, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 7.622109214, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005213480 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-25T14:08:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 20, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 692, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 672, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 26, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 66, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 40, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 19, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 1026, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 1007, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 649, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 648, 2024-04-25T14:08:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 127960, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223632 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 37220, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 5, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 32391, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 21, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 1752, 2024-04-25T14:08:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 630, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 9.231191734, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.014652685 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000918249, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000918249 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1262, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.199834910, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001743133 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1283, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.617445210, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000481251 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 420, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.212394361, 2024-04-25T14:08:08.021 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.014791415 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.022 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.024 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.025 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 635, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 3.804419625, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005991212 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-04-25T14:08:08.026 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 513, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 513, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1586, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:08:08.027 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:08:08.028 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:08:08.029 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 285338, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 128, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 440, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 5, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8104, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 170, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:08:08.030 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 94198, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 450, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 3506, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 5766339, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 32, 2024-04-25T14:08:08.031 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 3506, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3506, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 18.588625174, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005301946 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3506, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4006 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 3506, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 435, 2024-04-25T14:08:08.032 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 3071, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 326, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 1788, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 245, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 648, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-25T14:08:08.033 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 109, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-25T14:08:08.034 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-25T14:08:08.035 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 88, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 9, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 16, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 4, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 664, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 196 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-25T14:08:08.036 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 6, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 6, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 635, 2024-04-25T14:08:08.037 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 635, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 259502, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 47603413, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 259502, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 259502, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 47603413, 2024-04-25T14:08:08.038 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 3506, 2024-04-25T14:08:08.039 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 5830474, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3506, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 5830474, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:08.040 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 3506, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 3506, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3506, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 3506, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 635, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 64135, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 635, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 326, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 64135, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x559217cd28f0": { 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:08.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 4478, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 5572708, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 4478, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1462, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 5572708, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:08.043 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T14:08:08.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:08 smithi138 ceph-mon[32977]: pgmap v1141: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T14:08:08.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:08 smithi137 ceph-mon[32568]: pgmap v1141: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T14:08:08.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:08 smithi094 ceph-mon[29195]: pgmap v1141: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T14:08:08.863 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T14:08:08.863 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-25T14:08:08.863 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T14:08:08.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f9021454-0307-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T14:08:09.107 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:08:10.226 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:10 smithi138 ceph-mon[32977]: pgmap v1142: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T14:08:10.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:10 smithi137 ceph-mon[32568]: pgmap v1142: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T14:08:10.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:10 smithi094 ceph-mon[29195]: pgmap v1142: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T14:08:10.769 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T14:08:10.769 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:08:10.769 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1715, 2024-04-25T14:08:10.769 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1717, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 4640561, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 545434, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 7, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.125445975, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.037716693, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.037040861, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.038701578, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1715, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.047250116, 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000027551 2024-04-25T14:08:10.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1673, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2022, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1795490, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 1230013, 2024-04-25T14:08:10.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 5, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.143452615, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.045976730, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.032739398, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.051502556, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2021, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.062636388, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000030992 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 699584 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2402, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2379, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1631432, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 893600, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-04-25T14:08:10.773 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.243638728, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.077823725, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.077649780, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.074061919, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2378, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.100634848, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000042319 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1010768, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 288128 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8829, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.224646419, 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000025444 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.776 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 659, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 2, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 657, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 656, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-04-25T14:08:10.777 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 3, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-04-25T14:08:10.778 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:08:10.779 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-04-25T14:08:10.780 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:08:10.781 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-25T14:08:10.782 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4478, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 7, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 9767012, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 9765874, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.783 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 4478 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 12, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 656, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 644, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 10, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 20, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 10, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 660, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 670, 2024-04-25T14:08:10.784 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 10, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 120592, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 215440 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.785 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.786 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.787 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.788 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-25T14:08:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.790 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.791 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.792 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-25T14:08:10.793 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-25T14:08:10.794 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-25T14:08:10.795 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-04-25T14:08:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 2683, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:08:10.797 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:08:10.798 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:08:10.799 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 199068, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 59, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T14:08:10.800 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 8104, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 170, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T14:08:10.801 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 504825, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2762, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 5287, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 21, 2024-04-25T14:08:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 3, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 5287, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5287, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 9.459807565, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001789258 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5287, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 5288 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 5287, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 5287, 2024-04-25T14:08:10.803 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 3274, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 2011, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-25T14:08:10.804 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 1, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-25T14:08:10.805 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 88, 2024-04-25T14:08:10.806 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 9, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 6, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 7, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 1, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-25T14:08:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:10.808 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 5786, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 7518265, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 5786, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 5786, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 7518265, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 5287, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 5572713, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 5287, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 5572713, 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:10.810 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 5287, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 5287, 2024-04-25T14:08:10.811 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 5287, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 5287, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:10.812 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55cac78f0750": { 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T14:08:10.813 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T14:08:10.814 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T14:08:11.598 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T14:08:11.598 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T14:08:11.599 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T14:08:11.609 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:08:11.609 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:08:11.609 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:11.629 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:08:11.629 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:08:11.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- 172.21.15.94:0/1700180910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 msgr2=0x7f8278105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 --2- 172.21.15.94:0/1700180910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f8278105160 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f826c009900 tx=0x7f826c02a410 comp rx=0 tx=0).stop 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- 172.21.15.94:0/1700180910 shutdown_connections 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 --2- 172.21.15.94:0/1700180910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278109a50 0x7f827810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 --2- 172.21.15.94:0/1700180910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f8278105160 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 --2- 172.21.15.94:0/1700180910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278103b10 0x7f8278104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- 172.21.15.94:0/1700180910 >> 172.21.15.94:0/1700180910 conn(0x7f82780fed40 msgr2=0x7f82781011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- 172.21.15.94:0/1700180910 shutdown_connections 2024-04-25T14:08:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- 172.21.15.94:0/1700180910 wait complete. 2024-04-25T14:08:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 Processor -- start 2024-04-25T14:08:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.852+0000 7f827e726640 1 -- start start 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 0x7f827819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 0x7f82781a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f827810f610 con 0x7f8278104d50 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f827810f490 con 0x7f8278109a50 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827e726640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f827810f790 con 0x7f8278103b10 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59968/0 (socket says 172.21.15.94:59968) 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 -- 172.21.15.94:0/2101820085 learned_addr learned my addr 172.21.15.94:0/2101820085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827ccc3640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 0x7f82781a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f8277fff640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 0x7f827819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 -- 172.21.15.94:0/2101820085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 msgr2=0x7f827819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 0x7f827819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 -- 172.21.15.94:0/2101820085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 msgr2=0x7f82781a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 0x7f82781a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 -- 172.21.15.94:0/2101820085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781a1dc0 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f8277fff640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 0x7f827819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f827ccc3640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 0x7f82781a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82777fe640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f826c006010 tx=0x7f826c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.853+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c03f030 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.854+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826c003cb0 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.854+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781a2050 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.854+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c035070 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.854+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a2540 con 0x7f8278104d50 2024-04-25T14:08:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.855+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8278108860 con 0x7f8278104d50 2024-04-25T14:08:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.856+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f826c042030 con 0x7f8278104d50 2024-04-25T14:08:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.857+0000 7f82757fa640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 0x7f82600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.857+0000 7f8277fff640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 0x7f82600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.857+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f826c0ffad0 con 0x7f8278104d50 2024-04-25T14:08:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.858+0000 7f8277fff640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 0x7f82600b2000 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8268005d00 tx=0x7f8268005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:11.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:11.861+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f826c0c8610 con 0x7f8278104d50 2024-04-25T14:08:12.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.041+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82780637a0 con 0x7f8278104d50 2024-04-25T14:08:12.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.042+0000 7f82757fa640 1 -- 172.21.15.94:0/2101820085 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22958 (secure 0 0 0) 0x7f826c0c7f10 con 0x7f8278104d50 2024-04-25T14:08:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:08:12.042 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:58:45.755152+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 msgr2=0x7f82600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 0x7f82600b2000 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8268005d00 tx=0x7f8268005ef0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 msgr2=0x7f827819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f826c006010 tx=0x7f826c004340 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 shutdown_connections 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f82600afaf0 0x7f82600b2000 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8278109a50 0x7f82781a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8278104d50 0x7f827819a7c0 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 --2- 172.21.15.94:0/2101820085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278103b10 0x7f827819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 >> 172.21.15.94:0/2101820085 conn(0x7f82780fed40 msgr2=0x7f8278105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 shutdown_connections 2024-04-25T14:08:12.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.045+0000 7f827e726640 1 -- 172.21.15.94:0/2101820085 wait complete. 2024-04-25T14:08:12.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:35:08.260364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:39:42.984423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:39:43.934444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:08:12.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 -- 172.21.15.94:0/2158435755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8104480 msgr2=0x7f07d810abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 --2- 172.21.15.94:0/2158435755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8104480 0x7f07d810abd0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009ca0 tx=0x7f07d002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 -- 172.21.15.94:0/2158435755 shutdown_connections 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 --2- 172.21.15.94:0/2158435755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8104480 0x7f07d810abd0 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 --2- 172.21.15.94:0/2158435755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 0x7f07d8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 --2- 172.21.15.94:0/2158435755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8102f80 0x7f07d8103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.167+0000 7f07df385640 1 -- 172.21.15.94:0/2158435755 >> 172.21.15.94:0/2158435755 conn(0x7f07d80fd380 msgr2=0x7f07d80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- 172.21.15.94:0/2158435755 shutdown_connections 2024-04-25T14:08:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- 172.21.15.94:0/2158435755 wait complete. 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 Processor -- start 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- start start 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 0x7f07d819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 0x7f07d81a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810f730 con 0x7f07d8102f80 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810f5b0 con 0x7f07d8104480 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07df385640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07d810f8b0 con 0x7f07d8103960 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.168+0000 7f07dd121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59990/0 (socket says 172.21.15.94:59990) 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 -- 172.21.15.94:0/1107200251 learned_addr learned my addr 172.21.15.94:0/1107200251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dc920640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 0x7f07d819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd922640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 0x7f07d81a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 -- 172.21.15.94:0/1107200251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 msgr2=0x7f07d819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 0x7f07d819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 -- 172.21.15.94:0/1107200251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 msgr2=0x7f07d81a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 0x7f07d81a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 -- 172.21.15.94:0/1107200251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d81a1e50 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd922640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 0x7f07d81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07dd121640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f07d400b450 tx=0x7f07d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4015020 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d4011d40 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d81a2140 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400f340 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.169+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81a2710 con 0x7f07d8102f80 2024-04-25T14:08:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.170+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d81099a0 con 0x7f07d8102f80 2024-04-25T14:08:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.171+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f07d4018020 con 0x7f07d8102f80 2024-04-25T14:08:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.172+0000 7f07ce7fc640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 0x7f07c40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.172+0000 7f07dc920640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 0x7f07c40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.172+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f07d40deda0 con 0x7f07d8102f80 2024-04-25T14:08:12.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.173+0000 7f07dc920640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 0x7f07c40b2000 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f07d819b3b0 tx=0x7f07c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.176+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d401f020 con 0x7f07d8102f80 2024-04-25T14:08:12.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.354+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07d80637a0 con 0x7f07d8102f80 2024-04-25T14:08:12.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.354+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1107200251 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f07d400d040 con 0x7f07d8102f80 2024-04-25T14:08:12.354 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T14:08:12.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 msgr2=0x7f07c40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 0x7f07c40b2000 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f07d819b3b0 tx=0x7f07c8008040 comp rx=0 tx=0).stop 2024-04-25T14:08:12.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 msgr2=0x7f07d819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f07d400b450 tx=0x7f07d400b910 comp rx=0 tx=0).stop 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 shutdown_connections 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f07c40afaf0 0x7f07c40b2000 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d8104480 0x7f07d81a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d8103960 0x7f07d819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 --2- 172.21.15.94:0/1107200251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d8102f80 0x7f07d819a310 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 >> 172.21.15.94:0/1107200251 conn(0x7f07d80fd380 msgr2=0x7f07d8106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 shutdown_connections 2024-04-25T14:08:12.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.356+0000 7f07df385640 1 -- 172.21.15.94:0/1107200251 wait complete. 2024-04-25T14:08:12.366 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:12 smithi094 ceph-mon[29195]: pgmap v1143: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-25T14:08:12.367 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:08:12.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:12 smithi137 ceph-mon[32568]: pgmap v1143: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-25T14:08:12.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 -- 172.21.15.94:0/1356057614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc1029e0 msgr2=0x7f4dcc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 --2- 172.21.15.94:0/1356057614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1036e0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009900 tx=0x7f4dc802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 -- 172.21.15.94:0/1356057614 shutdown_connections 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 --2- 172.21.15.94:0/1356057614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc108920 0x7f4dcc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 --2- 172.21.15.94:0/1356057614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc103c20 0x7f4dcc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 --2- 172.21.15.94:0/1356057614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1036e0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 -- 172.21.15.94:0/1356057614 >> 172.21.15.94:0/1356057614 conn(0x7f4dcc0fdbd0 msgr2=0x7f4dcc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 -- 172.21.15.94:0/1356057614 shutdown_connections 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.478+0000 7f4dd4534640 1 -- 172.21.15.94:0/1356057614 wait complete. 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 Processor -- start 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 -- start start 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc103c20 0x7f4dcc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10e540 con 0x7f4dcc108920 2024-04-25T14:08:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10e3c0 con 0x7f4dcc1029e0 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd4534640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10e6c0 con 0x7f4dcc103c20 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd2ad1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd2ad1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60020/0 (socket says 172.21.15.94:60020) 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd2ad1640 1 -- 172.21.15.94:0/4163926059 learned_addr learned my addr 172.21.15.94:0/4163926059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd1acf640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc103c20 0x7f4dcc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.479+0000 7f4dd22d0640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 -- 172.21.15.94:0/4163926059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc103c20 msgr2=0x7f4dcc1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc103c20 0x7f4dcc1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 -- 172.21.15.94:0/4163926059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 msgr2=0x7f4dcc1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 -- 172.21.15.94:0/4163926059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc1a0f40 con 0x7f4dcc108920 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd22d0640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:12.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd2ad1640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b970 tx=0x7f4dc400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4019070 con 0x7f4dcc108920 2024-04-25T14:08:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc1a1230 con 0x7f4dcc108920 2024-04-25T14:08:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4014440 con 0x7f4dcc108920 2024-04-25T14:08:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40075d0 con 0x7f4dcc108920 2024-04-25T14:08:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.480+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a1800 con 0x7f4dcc108920 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.481+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc107730 con 0x7f4dcc108920 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.482+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4dc401c070 con 0x7f4dcc108920 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.483+0000 7f4dc37fe640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 0x7f4db80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.483+0000 7f4dd22d0640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 0x7f4db80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.483+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f4dc40de1c0 con 0x7f4dcc108920 2024-04-25T14:08:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.484+0000 7f4dd22d0640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 0x7f4db80b2000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005f70 tx=0x7f4dc8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.488+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dc40a6d00 con 0x7f4dcc108920 2024-04-25T14:08:12.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:12 smithi138 ceph-mon[32977]: pgmap v1143: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-25T14:08:12.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.667+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4dcc02dd50 con 0x7f4dcc108920 2024-04-25T14:08:12.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.668+0000 7f4dc37fe640 1 -- 172.21.15.94:0/4163926059 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f4dc40a6600 con 0x7f4dcc108920 2024-04-25T14:08:12.667 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 msgr2=0x7f4db80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 0x7f4db80b2000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005f70 tx=0x7f4dc8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 msgr2=0x7f4dcc1a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b970 tx=0x7f4dc400bd70 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 shutdown_connections 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f4db80afaf0 0x7f4db80b2000 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dcc108920 0x7f4dcc1a0980 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dcc103c20 0x7f4dcc1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 --2- 172.21.15.94:0/4163926059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dcc1029e0 0x7f4dcc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 >> 172.21.15.94:0/4163926059 conn(0x7f4dcc0fdbd0 msgr2=0x7f4dcc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 shutdown_connections 2024-04-25T14:08:12.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.670+0000 7f4dd4534640 1 -- 172.21.15.94:0/4163926059 wait complete. 2024-04-25T14:08:12.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 -- 172.21.15.94:0/621751604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01098b0 msgr2=0x7f50a010bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 --2- 172.21.15.94:0/621751604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01098b0 0x7f50a010bcd0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f5098009ca0 tx=0x7f509802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 -- 172.21.15.94:0/621751604 shutdown_connections 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 --2- 172.21.15.94:0/621751604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01098b0 0x7f50a010bcd0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 --2- 172.21.15.94:0/621751604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a0106f50 0x7f50a0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 --2- 172.21.15.94:0/621751604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a01045f0 0x7f50a0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 -- 172.21.15.94:0/621751604 >> 172.21.15.94:0/621751604 conn(0x7f50a00fdbf0 msgr2=0x7f50a01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 -- 172.21.15.94:0/621751604 shutdown_connections 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.792+0000 7f50a77ce640 1 -- 172.21.15.94:0/621751604 wait complete. 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 Processor -- start 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 -- start start 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 0x7f50a019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 0x7f50a01a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010f630 con 0x7f50a01045f0 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010f4b0 con 0x7f50a0106f50 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a77ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010f7b0 con 0x7f50a01098b0 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60046/0 (socket says 172.21.15.94:60046) 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 -- 172.21.15.94:0/1142604708 learned_addr learned my addr 172.21.15.94:0/1142604708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:12.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a4d69640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 0x7f50a019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a5d6b640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 0x7f50a01a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 -- 172.21.15.94:0/1142604708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 msgr2=0x7f50a01a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 0x7f50a01a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 -- 172.21.15.94:0/1142604708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 msgr2=0x7f50a019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 0x7f50a019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 -- 172.21.15.94:0/1142604708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a01a1d10 con 0x7f50a01045f0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a4d69640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 0x7f50a019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a5d6b640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 0x7f50a01a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50a556a640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c970 tx=0x7f509c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.793+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c007bb0 con 0x7f50a01045f0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.794+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c007d50 con 0x7f50a01045f0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.794+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c00f450 con 0x7f50a01045f0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.794+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a01a1fa0 con 0x7f50a01045f0 2024-04-25T14:08:12.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.794+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a01a2460 con 0x7f50a01045f0 2024-04-25T14:08:12.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.795+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a00ff710 con 0x7f50a01045f0 2024-04-25T14:08:12.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.796+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f509c004f20 con 0x7f50a01045f0 2024-04-25T14:08:12.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.797+0000 7f50967fc640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 0x7f508c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:12.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.797+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f509c0ded70 con 0x7f50a01045f0 2024-04-25T14:08:12.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.797+0000 7f50a4d69640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 0x7f508c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:12.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.797+0000 7f50a4d69640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 0x7f508c0b2000 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5090005fd0 tx=0x7f5090005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:12.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.801+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f509c014030 con 0x7f50a01045f0 2024-04-25T14:08:12.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.979+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50a00637a0 con 0x7f50a01045f0 2024-04-25T14:08:12.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.979+0000 7f50967fc640 1 -- 172.21.15.94:0/1142604708 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f509c0a78b0 con 0x7f50a01045f0 2024-04-25T14:08:12.978 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T14:08:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 msgr2=0x7f508c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 0x7f508c0b2000 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5090005fd0 tx=0x7f5090005850 comp rx=0 tx=0).stop 2024-04-25T14:08:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 msgr2=0x7f50a019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:12.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c970 tx=0x7f509c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 shutdown_connections 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f508c0afaf0 0x7f508c0b2000 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50a01098b0 0x7f50a01a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50a0106f50 0x7f50a019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 --2- 172.21.15.94:0/1142604708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50a01045f0 0x7f50a019a1d0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 >> 172.21.15.94:0/1142604708 conn(0x7f50a00fdbf0 msgr2=0x7f50a0105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 shutdown_connections 2024-04-25T14:08:12.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:12.981+0000 7f50a77ce640 1 -- 172.21.15.94:0/1142604708 wait complete. 2024-04-25T14:08:12.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.102+0000 7fdb5d805640 1 -- 172.21.15.94:0/1761594234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 msgr2=0x7fdb581037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1761594234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb581037e0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009ca0 tx=0x7fdb4c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 -- 172.21.15.94:0/1761594234 shutdown_connections 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1761594234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 0x7fdb5810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1761594234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb581037e0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1761594234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 0x7fdb58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 -- 172.21.15.94:0/1761594234 >> 172.21.15.94:0/1761594234 conn(0x7fdb580fd380 msgr2=0x7fdb580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 -- 172.21.15.94:0/1761594234 shutdown_connections 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 -- 172.21.15.94:0/1761594234 wait complete. 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 Processor -- start 2024-04-25T14:08:13.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.103+0000 7fdb5d805640 1 -- start start 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 0x7fdb5819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 0x7fdb581a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5810f630 con 0x7fdb581033d0 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5810f4b0 con 0x7fdb581080d0 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5810f7b0 con 0x7fdb58102190 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60058/0 (socket says 172.21.15.94:60058) 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 -- 172.21.15.94:0/1350652915 learned_addr learned my addr 172.21.15.94:0/1350652915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb577fe640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 0x7fdb581a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb56ffd640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 0x7fdb5819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 -- 172.21.15.94:0/1350652915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 msgr2=0x7fdb5819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 0x7fdb5819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 -- 172.21.15.94:0/1350652915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 msgr2=0x7fdb581a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 0x7fdb581a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 -- 172.21.15.94:0/1350652915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb581a1ca0 con 0x7fdb581033d0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb567fc640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c02ade0 tx=0x7fdb4c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0317c0 con 0x7fdb581033d0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c0044f0 con 0x7fdb581033d0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb581a1f30 con 0x7fdb581033d0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.104+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c02fd10 con 0x7fdb581033d0 2024-04-25T14:08:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.105+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb581a23a0 con 0x7fdb581033d0 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.105+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb58106ee0 con 0x7fdb581033d0 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.107+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fdb4c004040 con 0x7fdb581033d0 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.107+0000 7fdb5c803640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 0x7fdb400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.107+0000 7fdb56ffd640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 0x7fdb400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.107+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7fdb4c100460 con 0x7fdb581033d0 2024-04-25T14:08:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.108+0000 7fdb56ffd640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 0x7fdb400b2000 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdb58102d90 tx=0x7fdb5000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.112+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb4c0c8fa0 con 0x7fdb581033d0 2024-04-25T14:08:13.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2101820085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:13.246 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1107200251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4163926059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1142604708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.301+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb5802db50 con 0x7fdb581033d0 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.301+0000 7fdb5c803640 1 -- 172.21.15.94:0/1350652915 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+591 (secure 0 0 0) 0x7fdb4c0c88a0 con 0x7fdb581033d0 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1021554684 2024-04-26T13:32:40.167913+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1911980702 2024-04-26T13:32:40.167913+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1021554684 2024-04-26T13:32:40.167913+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2063913042 2024-04-26T13:32:40.167913+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2471492892 2024-04-26T13:32:13.498920+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1740247749 2024-04-26T13:32:13.498920+0000 2024-04-25T14:08:13.300 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2945875633 2024-04-26T13:32:13.498920+0000 2024-04-25T14:08:13.301 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2945875633 2024-04-26T13:32:13.498920+0000 2024-04-25T14:08:13.301 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/315514889 2024-04-26T13:32:13.498920+0000 2024-04-25T14:08:13.301 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2862296884 2024-04-26T13:32:40.167913+0000 2024-04-25T14:08:13.301 INFO:teuthology.orchestra.run.smithi094.stderr:listed 10 entries 2024-04-25T14:08:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 msgr2=0x7fdb400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 0x7fdb400b2000 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdb58102d90 tx=0x7fdb5000b390 comp rx=0 tx=0).stop 2024-04-25T14:08:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 msgr2=0x7fdb5819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:13.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c02ade0 tx=0x7fdb4c003ba0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 shutdown_connections 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7fdb400afaf0 0x7fdb400b2000 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb581080d0 0x7fdb581a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb581033d0 0x7fdb5819a750 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 --2- 172.21.15.94:0/1350652915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb58102190 0x7fdb5819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 >> 172.21.15.94:0/1350652915 conn(0x7fdb580fd380 msgr2=0x7fdb581040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 shutdown_connections 2024-04-25T14:08:13.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:13.303+0000 7fdb5d805640 1 -- 172.21.15.94:0/1350652915 wait complete. 2024-04-25T14:08:13.313 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2101820085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:13.313 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1107200251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.313 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4163926059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.313 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:13 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1142604708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.314 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:08:13.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:13.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:08:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2101820085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1107200251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4163926059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:13 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1142604708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:08:13.480 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T14:08:13.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:13.480 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:08:13.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:13.540 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T14:08:13.598 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:08:13.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:13.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:08:13.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:13.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T14:08:13.784 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T14:08:13.794 INFO:tasks.cephadm:Teardown begin 2024-04-25T14:08:13.795 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:08:13.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:14.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.010+0000 7f32f7413640 1 -- 172.21.15.94:0/35998614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f0110030 msgr2=0x7f32f0110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.010+0000 7f32f7413640 1 --2- 172.21.15.94:0/35998614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f0110030 0x7f32f0110440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f32e001a910 tx=0x7f32e003b470 comp rx=0 tx=0).stop 2024-04-25T14:08:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.019+0000 7f32f7413640 1 -- 172.21.15.94:0/35998614 shutdown_connections 2024-04-25T14:08:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.019+0000 7f32f7413640 1 --2- 172.21.15.94:0/35998614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f0110030 0x7f32f0110440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.019+0000 7f32f7413640 1 --2- 172.21.15.94:0/35998614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f010b310 0x7f32f010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.019+0000 7f32f7413640 1 --2- 172.21.15.94:0/35998614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f010a420 0x7f32f010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.019+0000 7f32f7413640 1 -- 172.21.15.94:0/35998614 >> 172.21.15.94:0/35998614 conn(0x7f32f00704e0 msgr2=0x7f32f0072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:14.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.028+0000 7f32f7413640 1 -- 172.21.15.94:0/35998614 shutdown_connections 2024-04-25T14:08:14.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.032+0000 7f32f7413640 1 -- 172.21.15.94:0/35998614 wait complete. 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.032+0000 7f32f7413640 1 Processor -- start 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 -- start start 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 0x7f32f01a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f010b310 0x7f32f01a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 0x7f32f01c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32f0113c70 con 0x7f32f010b310 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32f0113af0 con 0x7f32f010a420 2024-04-25T14:08:14.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.033+0000 7f32f7413640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32f0113df0 con 0x7f32f0110030 2024-04-25T14:08:14.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.034+0000 7f32f51af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 0x7f32f01a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.034+0000 7f32f51af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 0x7f32f01a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:35530/0 (socket says 172.21.15.94:35530) 2024-04-25T14:08:14.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.034+0000 7f32f51af640 1 -- 172.21.15.94:0/3236634632 learned_addr learned my addr 172.21.15.94:0/3236634632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:14.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.036+0000 7f32f59b0640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 0x7f32f01c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.037+0000 7f32f59b0640 1 -- 172.21.15.94:0/3236634632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 msgr2=0x7f32f01a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.037+0000 7f32f59b0640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 0x7f32f01a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.037+0000 7f32f59b0640 1 -- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f010b310 msgr2=0x7f32f01a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.037+0000 7f32f59b0640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f010b310 0x7f32f01a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.037+0000 7f32f59b0640 1 -- 172.21.15.94:0/3236634632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f01ca2f0 con 0x7f32f0110030 2024-04-25T14:08:14.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.038+0000 7f32f59b0640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 0x7f32f01c9c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e0044040 tx=0x7f32e0005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.042+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0013070 con 0x7f32f0110030 2024-04-25T14:08:14.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.042+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0040e50 con 0x7f32f0110030 2024-04-25T14:08:14.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.042+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0042440 con 0x7f32f0110030 2024-04-25T14:08:14.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.043+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f01ca580 con 0x7f32f0110030 2024-04-25T14:08:14.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.044+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f01caa80 con 0x7f32f0110030 2024-04-25T14:08:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.051+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f01cada0 con 0x7f32f0110030 2024-04-25T14:08:14.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.055+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f32e0016070 con 0x7f32f0110030 2024-04-25T14:08:14.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.055+0000 7f32e67fc640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 0x7f32dc0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.056+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f32e00ffdf0 con 0x7f32f0110030 2024-04-25T14:08:14.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.060+0000 7f32f51af640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 0x7f32dc0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.067+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32e00c88d0 con 0x7f32f0110030 2024-04-25T14:08:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.070+0000 7f32f51af640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 0x7f32dc0b2230 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f32f01a1bd0 tx=0x7f32e8007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:14 smithi137 ceph-mon[32568]: pgmap v1144: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T14:08:14.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:14 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1350652915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:08:14.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.399+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32f01caf70 con 0x7f32f0110030 2024-04-25T14:08:14.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.400+0000 7f32e67fc640 1 -- 172.21.15.94:0/3236634632 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11942 (secure 0 0 0) 0x7f32e00c81d0 con 0x7f32f0110030 2024-04-25T14:08:14.400 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:08:14.400 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T14:07:24.561299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24502,24488,24475,24512,24557]},"id":1}]} 2024-04-25T14:08:14.400 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 42 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 msgr2=0x7f32dc0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 0x7f32dc0b2230 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f32f01a1bd0 tx=0x7f32e8007600 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 msgr2=0x7f32f01c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 0x7f32f01c9c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e0044040 tx=0x7f32e0005770 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 shutdown_connections 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f32dc0afd20 0x7f32dc0b2230 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f0110030 0x7f32f01c9c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32f010b310 0x7f32f01a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 --2- 172.21.15.94:0/3236634632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32f010a420 0x7f32f01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 >> 172.21.15.94:0/3236634632 conn(0x7f32f00704e0 msgr2=0x7f32f010e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:14.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 shutdown_connections 2024-04-25T14:08:14.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.403+0000 7f32f7413640 1 -- 172.21.15.94:0/3236634632 wait complete. 2024-04-25T14:08:14.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:14 smithi094 ceph-mon[29195]: pgmap v1144: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T14:08:14.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:14 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1350652915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:08:14.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:14 smithi138 ceph-mon[32977]: pgmap v1144: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-25T14:08:14.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:14 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1350652915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- 172.21.15.94:0/2213011268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c109a20 msgr2=0x7f5f6c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 --2- 172.21.15.94:0/2213011268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c109a20 0x7f5f6c10be40 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009ca0 tx=0x7f5f6402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- 172.21.15.94:0/2213011268 shutdown_connections 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 --2- 172.21.15.94:0/2213011268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c109a20 0x7f5f6c10be40 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 --2- 172.21.15.94:0/2213011268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c104d20 0x7f5f6c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 --2- 172.21.15.94:0/2213011268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- 172.21.15.94:0/2213011268 >> 172.21.15.94:0/2213011268 conn(0x7f5f6c0fecd0 msgr2=0x7f5f6c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- 172.21.15.94:0/2213011268 shutdown_connections 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- 172.21.15.94:0/2213011268 wait complete. 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 Processor -- start 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.523+0000 7f5f728f2640 1 -- start start 2024-04-25T14:08:14.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c104d20 0x7f5f6c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 0x7f5f6c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10f610 con 0x7f5f6c103ae0 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10f490 con 0x7f5f6c104d20 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10f790 con 0x7f5f6c109a20 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49604/0 (socket says 172.21.15.94:49604) 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 -- 172.21.15.94:0/3960602367 learned_addr learned my addr 172.21.15.94:0/3960602367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6b7fe640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c104d20 0x7f5f6c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f70e8f640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 0x7f5f6c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 -- 172.21.15.94:0/3960602367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 msgr2=0x7f5f6c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 0x7f5f6c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 -- 172.21.15.94:0/3960602367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c104d20 msgr2=0x7f5f6c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c104d20 0x7f5f6c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 -- 172.21.15.94:0/3960602367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c1a1c90 con 0x7f5f6c103ae0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f70e8f640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 0x7f5f6c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f6bfff640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00d740 tx=0x7f5f5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c017070 con 0x7f5f6c103ae0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c00de50 con 0x7f5f6c103ae0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c1a1f80 con 0x7f5f6c103ae0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.524+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c003c20 con 0x7f5f6c103ae0 2024-04-25T14:08:14.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.525+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c1a23e0 con 0x7f5f6c103ae0 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.525+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c06afa0 con 0x7f5f6c103ae0 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.527+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5f5c003dc0 con 0x7f5f6c103ae0 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.528+0000 7f5f697fa640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.528+0000 7f5f6b7fe640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.528+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5f5c0deb50 con 0x7f5f6c103ae0 2024-04-25T14:08:14.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.528+0000 7f5f6b7fe640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 0x7f5f540b2000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5f600058c0 tx=0x7f5f6000a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.532+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f5c0a7630 con 0x7f5f6c103ae0 2024-04-25T14:08:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.745+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f6c02dc10 con 0x7f5f6c103ae0 2024-04-25T14:08:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.746+0000 7f5f697fa640 1 -- 172.21.15.94:0/3960602367 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11942 (secure 0 0 0) 0x7f5f5c0a6f30 con 0x7f5f6c103ae0 2024-04-25T14:08:14.745 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:08:14.745 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4024796429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4024796429},{"type":"v1","addr":"172.21.15.94:6835","nonce":4024796429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2786321466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2786321466},{"type":"v1","addr":"172.21.15.94:6837","nonce":2786321466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:39:44.966108+0000","modified":"2024-04-25T14:07:24.561299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24512,"mds_2":24502,"mds_3":24488,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3883623467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3883623467},{"type":"v1","addr":"172.21.15.94:6839","nonce":3883623467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/2301018309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2301018309},{"type":"v1","addr":"172.21.15.94:6841","nonce":2301018309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/4012763239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4012763239},{"type":"v1","addr":"172.21.15.138:6833","nonce":4012763239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/3830518540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3830518540},{"type":"v1","addr":"172.21.15.137:6833","nonce":3830518540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.138:6837/356914816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":356914816},{"type":"v1","addr":"172.21.15.138:6837","nonce":356914816}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.137:6835/3929131029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3929131029},{"type":"v1","addr":"172.21.15.137:6835","nonce":3929131029}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.138:6839/2137998525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2137998525},{"type":"v1","addr":"172.21.15.138:6839","nonce":2137998525}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/1274977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1274977027},{"type":"v1","addr":"172.21.15.137:6837","nonce":1274977027}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3143839143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3143839143},{"type":"v1","addr":"172.21.15.137:6839","nonce":3143839143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_30730":{"gid":30730,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/677168211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":677168211},{"type":"v1","addr":"172.21.15.138:6835","nonce":677168211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24502,24488,24475,24512,24557]},"id":1}]} 2024-04-25T14:08:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 42 2024-04-25T14:08:14.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 msgr2=0x7f5f540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 0x7f5f540b2000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5f600058c0 tx=0x7f5f6000a250 comp rx=0 tx=0).stop 2024-04-25T14:08:14.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 msgr2=0x7f5f6c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00d740 tx=0x7f5f5c00dc00 comp rx=0 tx=0).stop 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 shutdown_connections 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f6c109a20 0x7f5f6c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5f6c104d20 0x7f5f6c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 --2- 172.21.15.94:0/3960602367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c103ae0 0x7f5f6c19a150 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 >> 172.21.15.94:0/3960602367 conn(0x7f5f6c0fecd0 msgr2=0x7f5f6c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 shutdown_connections 2024-04-25T14:08:14.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.748+0000 7f5f728f2640 1 -- 172.21.15.94:0/3960602367 wait complete. 2024-04-25T14:08:14.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 -- 172.21.15.94:0/3975491614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14104d50 msgr2=0x7f5b14105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3975491614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14104d50 0x7f5b14105160 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009900 tx=0x7f5b0002a410 comp rx=0 tx=0).stop 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 -- 172.21.15.94:0/3975491614 shutdown_connections 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3975491614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14109a50 0x7f5b1410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3975491614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14104d50 0x7f5b14105160 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3975491614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14103b10 0x7f5b14104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 -- 172.21.15.94:0/3975491614 >> 172.21.15.94:0/3975491614 conn(0x7f5b140fed40 msgr2=0x7f5b141011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:14.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 -- 172.21.15.94:0/3975491614 shutdown_connections 2024-04-25T14:08:14.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.869+0000 7f5b1abce640 1 -- 172.21.15.94:0/3975491614 wait complete. 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 Processor -- start 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 -- start start 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 0x7f5b1419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 0x7f5b141a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410f690 con 0x7f5b14103b10 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410f510 con 0x7f5b14109a50 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1abce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410f810 con 0x7f5b14104d50 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49626/0 (socket says 172.21.15.94:49626) 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 -- 172.21.15.94:0/3416400421 learned_addr learned my addr 172.21.15.94:0/3416400421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1916b640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 0x7f5b141a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b13fff640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 0x7f5b1419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 -- 172.21.15.94:0/3416400421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 msgr2=0x7f5b1419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 0x7f5b1419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 -- 172.21.15.94:0/3416400421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 msgr2=0x7f5b141a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 0x7f5b141a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1896a640 1 -- 172.21.15.94:0/3416400421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b141a1da0 con 0x7f5b14103b10 2024-04-25T14:08:14.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b1916b640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 0x7f5b141a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.870+0000 7f5b13fff640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 0x7f5b1419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b1896a640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c970 tx=0x7f5b0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004220 con 0x7f5b14103b10 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b080043c0 con 0x7f5b14103b10 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b141a2090 con 0x7f5b14103b10 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800f450 con 0x7f5b14103b10 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.871+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b141a2660 con 0x7f5b14103b10 2024-04-25T14:08:14.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.872+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14108860 con 0x7f5b14103b10 2024-04-25T14:08:14.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.873+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5b080079b0 con 0x7f5b14103b10 2024-04-25T14:08:14.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.874+0000 7f5b11ffb640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 0x7f5b040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:14.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.874+0000 7f5b13fff640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 0x7f5b040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:14.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.874+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f5b080deea0 con 0x7f5b14103b10 2024-04-25T14:08:14.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.874+0000 7f5b13fff640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 0x7f5b040b2000 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:14.879+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b08014030 con 0x7f5b14103b10 2024-04-25T14:08:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.061+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b140637a0 con 0x7f5b14103b10 2024-04-25T14:08:15.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.063+0000 7f5b11ffb640 1 -- 172.21.15.94:0/3416400421 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22958 (secure 0 0 0) 0x7f5b080a79e0 con 0x7f5b14103b10 2024-04-25T14:08:15.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:08:15.062 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f9021454-0307-11ef-bc93-c7b262605968","created":"2024-04-25T13:31:45.992646+0000","modified":"2024-04-25T13:58:45.755152+0000","last_up_change":"2024-04-25T13:38:54.369426+0000","last_in_change":"2024-04-25T13:38:34.076236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:35:08.260364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:39:42.984423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:39:43.934444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"71e93888-dbdc-44cc-b0c5-bd211fc1fbed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6803","nonce":2637923455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6805","nonce":2637923455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6809","nonce":2637923455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2637923455},{"type":"v1","addr":"172.21.15.94:6807","nonce":2637923455}]},"public_addr":"172.21.15.94:6803/2637923455","cluster_addr":"172.21.15.94:6805/2637923455","heartbeat_back_addr":"172.21.15.94:6809/2637923455","heartbeat_front_addr":"172.21.15.94:6807/2637923455","state":["exists","up"]},{"osd":1,"uuid":"814d5805-282b-4231-9570-1ff9e7e41fec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6801","nonce":1683710608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6803","nonce":1683710608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6807","nonce":1683710608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1683710608},{"type":"v1","addr":"172.21.15.137:6805","nonce":1683710608}]},"public_addr":"172.21.15.137:6801/1683710608","cluster_addr":"172.21.15.137:6803/1683710608","heartbeat_back_addr":"172.21.15.137:6807/1683710608","heartbeat_front_addr":"172.21.15.137:6805/1683710608","state":["exists","up"]},{"osd":2,"uuid":"14d09b14-8c4c-48b2-b295-b6581d524d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6801","nonce":4136773447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6803","nonce":4136773447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6807","nonce":4136773447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4136773447},{"type":"v1","addr":"172.21.15.138:6805","nonce":4136773447}]},"public_addr":"172.21.15.138:6801/4136773447","cluster_addr":"172.21.15.138:6803/4136773447","heartbeat_back_addr":"172.21.15.138:6807/4136773447","heartbeat_front_addr":"172.21.15.138:6805/4136773447","state":["exists","up"]},{"osd":3,"uuid":"046f0d40-5505-42d2-961f-4cf5250b1eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6811","nonce":3561678692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6813","nonce":3561678692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6817","nonce":3561678692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3561678692},{"type":"v1","addr":"172.21.15.94:6815","nonce":3561678692}]},"public_addr":"172.21.15.94:6811/3561678692","cluster_addr":"172.21.15.94:6813/3561678692","heartbeat_back_addr":"172.21.15.94:6817/3561678692","heartbeat_front_addr":"172.21.15.94:6815/3561678692","state":["exists","up"]},{"osd":4,"uuid":"0d740f16-7559-45f6-bd33-f834183b06f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6809","nonce":3127701916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6811","nonce":3127701916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6815","nonce":3127701916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3127701916},{"type":"v1","addr":"172.21.15.137:6813","nonce":3127701916}]},"public_addr":"172.21.15.137:6809/3127701916","cluster_addr":"172.21.15.137:6811/3127701916","heartbeat_back_addr":"172.21.15.137:6815/3127701916","heartbeat_front_addr":"172.21.15.137:6813/3127701916","state":["exists","up"]},{"osd":5,"uuid":"eeda2336-9862-4bd4-b776-454204034ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6809","nonce":699823626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6811","nonce":699823626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6815","nonce":699823626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":699823626},{"type":"v1","addr":"172.21.15.138:6813","nonce":699823626}]},"public_addr":"172.21.15.138:6809/699823626","cluster_addr":"172.21.15.138:6811/699823626","heartbeat_back_addr":"172.21.15.138:6815/699823626","heartbeat_front_addr":"172.21.15.138:6813/699823626","state":["exists","up"]},{"osd":6,"uuid":"a884764f-45af-4c83-b7d3-65ebd314f012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6819","nonce":1177969162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6821","nonce":1177969162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6825","nonce":1177969162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1177969162},{"type":"v1","addr":"172.21.15.94:6823","nonce":1177969162}]},"public_addr":"172.21.15.94:6819/1177969162","cluster_addr":"172.21.15.94:6821/1177969162","heartbeat_back_addr":"172.21.15.94:6825/1177969162","heartbeat_front_addr":"172.21.15.94:6823/1177969162","state":["exists","up"]},{"osd":7,"uuid":"73b89099-7f29-4114-b29f-c0b33124e04e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6817","nonce":1075953971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6819","nonce":1075953971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6823","nonce":1075953971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1075953971},{"type":"v1","addr":"172.21.15.137:6821","nonce":1075953971}]},"public_addr":"172.21.15.137:6817/1075953971","cluster_addr":"172.21.15.137:6819/1075953971","heartbeat_back_addr":"172.21.15.137:6823/1075953971","heartbeat_front_addr":"172.21.15.137:6821/1075953971","state":["exists","up"]},{"osd":8,"uuid":"47e5d4bf-c849-4b0c-bb6b-f4c14675072a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6817","nonce":3084581661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6819","nonce":3084581661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6823","nonce":3084581661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3084581661},{"type":"v1","addr":"172.21.15.138:6821","nonce":3084581661}]},"public_addr":"172.21.15.138:6817/3084581661","cluster_addr":"172.21.15.138:6819/3084581661","heartbeat_back_addr":"172.21.15.138:6823/3084581661","heartbeat_front_addr":"172.21.15.138:6821/3084581661","state":["exists","up"]},{"osd":9,"uuid":"d4cec5f6-fc9d-4166-9979-a7dad7ce3585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6827","nonce":1188484868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6829","nonce":1188484868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6833","nonce":1188484868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1188484868},{"type":"v1","addr":"172.21.15.94:6831","nonce":1188484868}]},"public_addr":"172.21.15.94:6827/1188484868","cluster_addr":"172.21.15.94:6829/1188484868","heartbeat_back_addr":"172.21.15.94:6833/1188484868","heartbeat_front_addr":"172.21.15.94:6831/1188484868","state":["exists","up"]},{"osd":10,"uuid":"110b4c5b-fa58-48a4-9567-c88bd908afa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6825","nonce":3471315400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6827","nonce":3471315400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6831","nonce":3471315400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3471315400},{"type":"v1","addr":"172.21.15.137:6829","nonce":3471315400}]},"public_addr":"172.21.15.137:6825/3471315400","cluster_addr":"172.21.15.137:6827/3471315400","heartbeat_back_addr":"172.21.15.137:6831/3471315400","heartbeat_front_addr":"172.21.15.137:6829/3471315400","state":["exists","up"]},{"osd":11,"uuid":"3af033e4-3bf8-4a1d-a355-55edad2c8226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6825","nonce":2554799218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6827","nonce":2554799218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6831","nonce":2554799218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2554799218},{"type":"v1","addr":"172.21.15.138:6829","nonce":2554799218}]},"public_addr":"172.21.15.138:6825/2554799218","cluster_addr":"172.21.15.138:6827/2554799218","heartbeat_back_addr":"172.21.15.138:6831/2554799218","heartbeat_front_addr":"172.21.15.138:6829/2554799218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:16.327980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:34:41.527347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:06.179325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:30.386774+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:35:55.499538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:20.507513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:36:46.154253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:10.984239+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:37:36.017352+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:01.540361+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:27.135675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:38:52.684119+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2e","mappings":[{"from":1,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6800/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/1911980702":"2024-04-26T13:32:40.167913+0000","172.21.15.94:6801/1021554684":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2063913042":"2024-04-26T13:32:40.167913+0000","172.21.15.94:0/2471492892":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/1740247749":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6800/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:6801/2945875633":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/315514889":"2024-04-26T13:32:13.498920+0000","172.21.15.94:0/2862296884":"2024-04-26T13:32:40.167913+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 msgr2=0x7f5b040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 0x7f5b040b2000 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b00000bc0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 msgr2=0x7f5b1419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c970 tx=0x7f5b0800ce30 comp rx=0 tx=0).stop 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 shutdown_connections 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5b040afaf0 0x7f5b040b2000 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b14109a50 0x7f5b141a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b14104d50 0x7f5b1419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 --2- 172.21.15.94:0/3416400421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b14103b10 0x7f5b1419a260 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 >> 172.21.15.94:0/3416400421 conn(0x7f5b140fed40 msgr2=0x7f5b14105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:15.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 shutdown_connections 2024-04-25T14:08:15.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.065+0000 7f5b1abce640 1 -- 172.21.15.94:0/3416400421 wait complete. 2024-04-25T14:08:15.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 -- 172.21.15.94:0/1886583745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4103c20 msgr2=0x7f68b4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 --2- 172.21.15.94:0/1886583745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4103c20 0x7f68b4104030 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009900 tx=0x7f68a402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 -- 172.21.15.94:0/1886583745 shutdown_connections 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 --2- 172.21.15.94:0/1886583745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4108920 0x7f68b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 --2- 172.21.15.94:0/1886583745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4103c20 0x7f68b4104030 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 --2- 172.21.15.94:0/1886583745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 0x7f68b41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 -- 172.21.15.94:0/1886583745 >> 172.21.15.94:0/1886583745 conn(0x7f68b40fdbd0 msgr2=0x7f68b4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 -- 172.21.15.94:0/1886583745 shutdown_connections 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.189+0000 7f68bbe51640 1 -- 172.21.15.94:0/1886583745 wait complete. 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 Processor -- start 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 -- start start 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 0x7f68b4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4103c20 0x7f68b4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68b410e560 con 0x7f68b4108920 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68b410e3e0 con 0x7f68b41029e0 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68bbe51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68b410e6e0 con 0x7f68b4103c20 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68ba3ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68ba3ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49632/0 (socket says 172.21.15.94:49632) 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68ba3ee640 1 -- 172.21.15.94:0/4103198398 learned_addr learned my addr 172.21.15.94:0/4103198398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68b9bed640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 0x7f68b4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68b93ec640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4103c20 0x7f68b4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68ba3ee640 1 -- 172.21.15.94:0/4103198398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4103c20 msgr2=0x7f68b4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.190+0000 7f68ba3ee640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4103c20 0x7f68b4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68ba3ee640 1 -- 172.21.15.94:0/4103198398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 msgr2=0x7f68b4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68ba3ee640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 0x7f68b4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68ba3ee640 1 -- 172.21.15.94:0/4103198398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b41a0f40 con 0x7f68b4108920 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68ba3ee640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eaa0 tx=0x7f68ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00cdb0 con 0x7f68b4108920 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ac005020 con 0x7f68b4108920 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b41a1230 con 0x7f68b4108920 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0116d0 con 0x7f68b4108920 2024-04-25T14:08:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.191+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a1780 con 0x7f68b4108920 2024-04-25T14:08:15.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.192+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b4107730 con 0x7f68b4108920 2024-04-25T14:08:15.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.194+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f68ac0051d0 con 0x7f68b4108920 2024-04-25T14:08:15.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.195+0000 7f68aaffd640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 0x7f68a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.195+0000 7f68b9bed640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 0x7f68a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.195+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8385+0+0 (secure 0 0 0) 0x7f68ac0df230 con 0x7f68b4108920 2024-04-25T14:08:15.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.196+0000 7f68b9bed640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 0x7f68a00b2000 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f68b41035e0 tx=0x7f68b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:15.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.199+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68ac016030 con 0x7f68b4108920 2024-04-25T14:08:15.357 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3236634632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.357 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3960602367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3236634632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3960602367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.426+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f68b402dd30 con 0x7f68b4108920 2024-04-25T14:08:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.428+0000 7f68aaffd640 1 -- 172.21.15.94:0/4103198398 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f68ac0a7d70 con 0x7f68b4108920 2024-04-25T14:08:15.427 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:08:15.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.431+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 msgr2=0x7f68a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.431+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 0x7f68a00b2000 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f68b41035e0 tx=0x7f68b000b390 comp rx=0 tx=0).stop 2024-04-25T14:08:15.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.431+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 msgr2=0x7f68b41a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.431+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eaa0 tx=0x7f68ac00ef60 comp rx=0 tx=0).stop 2024-04-25T14:08:15.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 shutdown_connections 2024-04-25T14:08:15.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f68a00afaf0 0x7f68a00b2000 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b4108920 0x7f68b41a08d0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4103c20 0x7f68b4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 --2- 172.21.15.94:0/4103198398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68b41029e0 0x7f68b4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.432+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 >> 172.21.15.94:0/4103198398 conn(0x7f68b40fdbd0 msgr2=0x7f68b4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:15.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.433+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 shutdown_connections 2024-04-25T14:08:15.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.433+0000 7f68bbe51640 1 -- 172.21.15.94:0/4103198398 wait complete. 2024-04-25T14:08:15.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:08:15.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3236634632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3960602367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:15.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 -- 172.21.15.94:0/710130734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce8106f40 msgr2=0x7f2ce8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 --2- 172.21.15.94:0/710130734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce8106f40 0x7f2ce8109360 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009900 tx=0x7f2cd802a410 comp rx=0 tx=0).stop 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 -- 172.21.15.94:0/710130734 shutdown_connections 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 --2- 172.21.15.94:0/710130734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81098a0 0x7f2ce810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 --2- 172.21.15.94:0/710130734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce8106f40 0x7f2ce8109360 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 --2- 172.21.15.94:0/710130734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce81045e0 0x7f2ce8106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 -- 172.21.15.94:0/710130734 >> 172.21.15.94:0/710130734 conn(0x7f2ce80fdbc0 msgr2=0x7f2ce8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 -- 172.21.15.94:0/710130734 shutdown_connections 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.571+0000 7f2cf0005640 1 -- 172.21.15.94:0/710130734 wait complete. 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 Processor -- start 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 -- start start 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81045e0 0x7f2ce819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 0x7f2ce819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810f650 con 0x7f2ce81098a0 2024-04-25T14:08:15.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810f4d0 con 0x7f2ce81045e0 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cf0005640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810f7d0 con 0x7f2ce8106f40 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cee5a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2ced5a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 0x7f2ce819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cee5a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49688/0 (socket says 172.21.15.94:49688) 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.572+0000 7f2cee5a2640 1 -- 172.21.15.94:0/2714886489 learned_addr learned my addr 172.21.15.94:0/2714886489 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cedda1640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81045e0 0x7f2ce819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 -- 172.21.15.94:0/2714886489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81045e0 msgr2=0x7f2ce819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81045e0 0x7f2ce819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 -- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 msgr2=0x7f2ce81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:15.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 -- 172.21.15.94:0/2714886489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce81a1d90 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cee5a2640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2ced5a0640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 0x7f2ce819a790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2cd802aa00 tx=0x7f2cd8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803f070 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd8031c50 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce81a2020 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8035030 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.573+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce81a24e0 con 0x7f2ce8106f40 2024-04-25T14:08:15.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.574+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce80ff6e0 con 0x7f2ce8106f40 2024-04-25T14:08:15.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.576+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2cd8042070 con 0x7f2ce8106f40 2024-04-25T14:08:15.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.576+0000 7f2cdeffd640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 0x7f2cd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:15.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.577+0000 7f2cedda1640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 0x7f2cd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:15.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.577+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8815+0+0 (secure 0 0 0) 0x7f2cd80fff90 con 0x7f2ce8106f40 2024-04-25T14:08:15.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.577+0000 7f2cedda1640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 0x7f2cd40b2000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40095f0 tx=0x7f2ce4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:15.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.580+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cd80c8930 con 0x7f2ce8106f40 2024-04-25T14:08:15.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T14:08:15.426+0000 7f09e6381700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:08:15.724 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.724 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-mds[70516]: -- 172.21.15.94:0/3294204051 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dc9c91a40 con 0x555dc9cd9180 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: starting mds.g at 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-mds[70876]: -- 172.21.15.94:0/2316268410 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fd504ba40 con 0x557fd5093180 2024-04-25T14:08:15.725 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:08:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: starting mds.a at 2024-04-25T14:08:15.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:15.804+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ce80637a0 con 0x7f2ce8106f40 2024-04-25T14:08:15.892 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.892 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mds[62886]: -- 172.21.15.137:0/653703754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c149a27a40 con 0x55c149a6f180 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: starting mds.e at 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mds[63500]: -- 172.21.15.137:0/3239955544 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2d8abfa40 con 0x55d2d8b07600 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: starting mds.b at 2024-04-25T14:08:15.893 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mds[63196]: -- 172.21.15.137:0/1970168610 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558f11e5a40 con 0x5558f122d180 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: starting mds.h at 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-mds[62571]: -- 172.21.15.137:0/2655368727 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626754b5a40 con 0x5626754fda80 2024-04-25T14:08:15.894 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:08:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: starting mds.k at 2024-04-25T14:08:15.996 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.996 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.996 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mds[63643]: -- 172.21.15.138:0/2069885773 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55923ba27a40 con 0x55923ba6f600 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: starting mds.c at 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mds[63334]: -- 172.21.15.138:0/1081034389 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577aafcda40 con 0x5577ab015600 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: starting mds.f at 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.997 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mds[63953]: -- 172.21.15.138:0/1056767938 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7f7b4ba40 con 0x55a7f7b93180 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: starting mds.i at 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: ignoring --setuser ceph since I am not root 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: ignoring --setgroup ceph since I am not root 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-mds[63013]: -- 172.21.15.138:0/1200542851 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563297929a40 con 0x563297971a80 2024-04-25T14:08:15.998 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:08:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: starting mds.l at 2024-04-25T14:08:16.334 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: pgmap v1145: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/3416400421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/2714886489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.335 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:16 smithi094 ceph-mon[29195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: pgmap v1145: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T14:08:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/3416400421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:16.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/2714886489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:16 smithi137 ceph-mon[32568]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.434+0000 7f2cdeffd640 1 -- 172.21.15.94:0/2714886489 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f2cd80c8230 con 0x7f2ce8106f40 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.437+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 msgr2=0x7f2cd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.437+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 0x7f2cd40b2000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40095f0 tx=0x7f2ce4005f90 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.437+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 msgr2=0x7f2ce819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.437+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 0x7f2ce819a790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2cd802aa00 tx=0x7f2cd8003c10 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.437+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 shutdown_connections 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f2cd40afaf0 0x7f2cd40b2000 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce81098a0 0x7f2ce81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce8106f40 0x7f2ce819a790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 --2- 172.21.15.94:0/2714886489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce81045e0 0x7f2ce819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 >> 172.21.15.94:0/2714886489 conn(0x7f2ce80fdbc0 msgr2=0x7f2ce8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 shutdown_connections 2024-04-25T14:08:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.438+0000 7f2cf0005640 1 -- 172.21.15.94:0/2714886489 wait complete. 2024-04-25T14:08:16.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:08:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: pgmap v1145: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T14:08:16.496 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/3416400421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/4103198398' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/2714886489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.497 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:16 smithi138 ceph-mon[32977]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- 172.21.15.94:0/304843243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 msgr2=0x7f8120104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 --2- 172.21.15.94:0/304843243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f8120104030 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f8114009900 tx=0x7f811402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- 172.21.15.94:0/304843243 shutdown_connections 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 --2- 172.21.15.94:0/304843243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 0x7f812010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 --2- 172.21.15.94:0/304843243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f8120104030 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 --2- 172.21.15.94:0/304843243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f81201036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- 172.21.15.94:0/304843243 >> 172.21.15.94:0/304843243 conn(0x7f81200fdbd0 msgr2=0x7f8120100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- 172.21.15.94:0/304843243 shutdown_connections 2024-04-25T14:08:16.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- 172.21.15.94:0/304843243 wait complete. 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 Processor -- start 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.563+0000 7f8126614640 1 -- start start 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f812006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 0x7f81201a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f812010e4e0 con 0x7f8120103c20 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f812010e360 con 0x7f8120108920 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f812010e660 con 0x7f81201029e0 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49706/0 (socket says 172.21.15.94:49706) 2024-04-25T14:08:16.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 -- 172.21.15.94:0/49659747 learned_addr learned my addr 172.21.15.94:0/49659747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8124bb1640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 0x7f81201a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811ffff640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f812006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 -- 172.21.15.94:0/49659747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 msgr2=0x7f812006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f812006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 -- 172.21.15.94:0/49659747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 msgr2=0x7f81201a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 0x7f81201a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 -- 172.21.15.94:0/49659747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81201a2070 con 0x7f8120103c20 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811ffff640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f812006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811f7fe640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f8114006010 tx=0x7f8114004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811403f030 con 0x7f8120103c20 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81201a2300 con 0x7f8120103c20 2024-04-25T14:08:16.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8114003cb0 con 0x7f8120103c20 2024-04-25T14:08:16.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.564+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114035070 con 0x7f8120103c20 2024-04-25T14:08:16.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.565+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a27f0 con 0x7f8120103c20 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.565+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8120107730 con 0x7f8120103c20 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.567+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8114042030 con 0x7f8120103c20 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.567+0000 7f811d7fa640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 0x7f81080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.568+0000 7f811ffff640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 0x7f81080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.568+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8815+0+0 (secure 0 0 0) 0x7f81140ffc00 con 0x7f8120103c20 2024-04-25T14:08:16.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.568+0000 7f811ffff640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 0x7f81080b2000 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8110005d00 tx=0x7f8110005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.571+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81140c85a0 con 0x7f8120103c20 2024-04-25T14:08:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:16.751+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f812006c2a0 con 0x7f8120103c20 2024-04-25T14:08:17.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: Cluster is now healthy 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6838/616748912,v1:172.21.15.94:6839/616748912] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.94:6840/3639115892,v1:172.21.15.94:6841/3639115892] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6833/3697575313,v1:172.21.15.138:6836/3697575313] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6832/172873671,v1:172.21.15.138:6834/172873671] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6835/344293727,v1:172.21.15.138:6837/344293727] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.138:6838/3819254298,v1:172.21.15.138:6839/3819254298] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6834/798940130,v1:172.21.15.137:6838/798940130] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6833/832416891,v1:172.21.15.137:6837/832416891] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6832/477198727,v1:172.21.15.137:6836/477198727] up:boot 2024-04-25T14:08:17.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: mds.? [v2:172.21.15.137:6835/4020411595,v1:172.21.15.137:6839/4020411595] up:boot 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: fsmap 12 up:standby 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:08:17.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:17 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:17.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.436+0000 7f811d7fa640 1 -- 172.21.15.94:0/49659747 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f8114003e60 con 0x7f8120103c20 2024-04-25T14:08:17.435 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.438+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 msgr2=0x7f81080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.438+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 0x7f81080b2000 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8110005d00 tx=0x7f8110005ef0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.438+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 msgr2=0x7f812019f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.438+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f8114006010 tx=0x7f8114004340 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 shutdown_connections 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f81080afaf0 0x7f81080b2000 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8120108920 0x7f81201a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8120103c20 0x7f812019f350 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 --2- 172.21.15.94:0/49659747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81201029e0 0x7f812006a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 >> 172.21.15.94:0/49659747 conn(0x7f81200fdbd0 msgr2=0x7f8120104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 shutdown_connections 2024-04-25T14:08:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.439+0000 7f8126614640 1 -- 172.21.15.94:0/49659747 wait complete. 2024-04-25T14:08:17.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: Cluster is now healthy 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6838/616748912,v1:172.21.15.94:6839/616748912] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.94:6840/3639115892,v1:172.21.15.94:6841/3639115892] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6833/3697575313,v1:172.21.15.138:6836/3697575313] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6832/172873671,v1:172.21.15.138:6834/172873671] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6835/344293727,v1:172.21.15.138:6837/344293727] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.138:6838/3819254298,v1:172.21.15.138:6839/3819254298] up:boot 2024-04-25T14:08:17.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6834/798940130,v1:172.21.15.137:6838/798940130] up:boot 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6833/832416891,v1:172.21.15.137:6837/832416891] up:boot 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6832/477198727,v1:172.21.15.137:6836/477198727] up:boot 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: mds.? [v2:172.21.15.137:6835/4020411595,v1:172.21.15.137:6839/4020411595] up:boot 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:08:17.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:17 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:17.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:08:17.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:08:17.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T14:08:17.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6838/616748912,v1:172.21.15.94:6839/616748912] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.94:6840/3639115892,v1:172.21.15.94:6841/3639115892] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6833/3697575313,v1:172.21.15.138:6836/3697575313] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6832/172873671,v1:172.21.15.138:6834/172873671] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6835/344293727,v1:172.21.15.138:6837/344293727] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6838/3819254298,v1:172.21.15.138:6839/3819254298] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6834/798940130,v1:172.21.15.137:6838/798940130] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6833/832416891,v1:172.21.15.137:6837/832416891] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6832/477198727,v1:172.21.15.137:6836/477198727] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.137:6835/4020411595,v1:172.21.15.137:6839/4020411595] up:boot 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: fsmap 12 up:standby 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:08:17.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='mgr.14152 172.21.15.94:0/810472332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:08:17.702 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:17.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:08:17.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 -- 172.21.15.94:0/883609146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24108900 msgr2=0x7f5e2410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:17.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 --2- 172.21.15.94:0/883609146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24108900 0x7f5e2410ad20 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009ca0 tx=0x7f5e1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 -- 172.21.15.94:0/883609146 shutdown_connections 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 --2- 172.21.15.94:0/883609146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e24108900 0x7f5e2410ad20 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 --2- 172.21.15.94:0/883609146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24103c00 0x7f5e24104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 --2- 172.21.15.94:0/883609146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e241029c0 0x7f5e241036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 -- 172.21.15.94:0/883609146 >> 172.21.15.94:0/883609146 conn(0x7f5e240fdbd0 msgr2=0x7f5e24100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 -- 172.21.15.94:0/883609146 shutdown_connections 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.820+0000 7f5e2a451640 1 -- 172.21.15.94:0/883609146 wait complete. 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 Processor -- start 2024-04-25T14:08:17.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 -- start start 2024-04-25T14:08:17.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:17.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e24103c00 0x7f5e2419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:17.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24108900 0x7f5e241a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:17.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2410e4d0 con 0x7f5e241029c0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2410e350 con 0x7f5e24103c00 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e2a451640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2410e650 con 0x7f5e24108900 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49716/0 (socket says 172.21.15.94:49716) 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 -- 172.21.15.94:0/1459819764 learned_addr learned my addr 172.21.15.94:0/1459819764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e237fe640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e24103c00 0x7f5e2419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e289ee640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24108900 0x7f5e241a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 -- 172.21.15.94:0/1459819764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24108900 msgr2=0x7f5e241a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24108900 0x7f5e241a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 -- 172.21.15.94:0/1459819764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e24103c00 msgr2=0x7f5e2419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.821+0000 7f5e23fff640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e24103c00 0x7f5e2419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e23fff640 1 -- 172.21.15.94:0/1459819764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e241a1c90 con 0x7f5e241029c0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e23fff640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9f0 tx=0x7f5e1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:17.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ccc0 con 0x7f5e241029c0 2024-04-25T14:08:17.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e140035f0 con 0x7f5e241029c0 2024-04-25T14:08:17.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ce60 con 0x7f5e241029c0 2024-04-25T14:08:17.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e241a1f80 con 0x7f5e241029c0 2024-04-25T14:08:17.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.822+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e241a2440 con 0x7f5e241029c0 2024-04-25T14:08:17.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.823+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e24107710 con 0x7f5e241029c0 2024-04-25T14:08:17.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.824+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f5e14003aa0 con 0x7f5e241029c0 2024-04-25T14:08:17.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.825+0000 7f5e217fa640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 0x7f5e0c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:17.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.825+0000 7f5e237fe640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 0x7f5e0c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:17.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.825+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8261+0+0 (secure 0 0 0) 0x7f5e140df010 con 0x7f5e241029c0 2024-04-25T14:08:17.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.826+0000 7f5e237fe640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 0x7f5e0c0b2080 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5e2406a2c0 tx=0x7f5e1800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:17.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:17.829+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e140a7bd0 con 0x7f5e241029c0 2024-04-25T14:08:18.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.057+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e2402db50 con 0x7f5e241029c0 2024-04-25T14:08:18.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.444+0000 7f5e217fa640 1 -- 172.21.15.94:0/1459819764 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f5e140a74d0 con 0x7f5e241029c0 2024-04-25T14:08:18.443 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-04-25T14:08:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.447+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 msgr2=0x7f5e0c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.447+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 0x7f5e0c0b2080 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5e2406a2c0 tx=0x7f5e1800b390 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.447+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 msgr2=0x7f5e2406a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.447+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9f0 tx=0x7f5e1400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 shutdown_connections 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:6800/449370799,v1:172.21.15.94:6801/449370799] conn(0x7f5e0c0afb70 0x7f5e0c0b2080 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e24108900 0x7f5e241a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e24103c00 0x7f5e2419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 --2- 172.21.15.94:0/1459819764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e241029c0 0x7f5e2406a920 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 >> 172.21.15.94:0/1459819764 conn(0x7f5e240fdbd0 msgr2=0x7f5e241048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 shutdown_connections 2024-04-25T14:08:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:08:18.448+0000 7f5e2a451640 1 -- 172.21.15.94:0/1459819764 wait complete. 2024-04-25T14:08:18.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:08:18.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-mon[29195]: pgmap v1147: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T14:08:18.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:08:18.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-mon[29195]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:08:18.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-mon[29195]: from='client.? 172.21.15.94:0/1459819764' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:18.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:08:18.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:08:18.559 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T14:08:18.559 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:08:18.574 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:08:18.597 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:08:18.614 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T14:08:18.614 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T14:08:18.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:08:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:18 smithi138 ceph-mon[32977]: pgmap v1147: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T14:08:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:18 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:08:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:18 smithi138 ceph-mon[32977]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:08:18.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:18 smithi138 ceph-mon[32977]: from='client.? 172.21.15.94:0/1459819764' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:18.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:18 smithi137 ceph-mon[32568]: pgmap v1147: 97 pgs: 97 active+clean; 65 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T14:08:18.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:18 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/49659747' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:08:18.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:18 smithi137 ceph-mon[32568]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T14:08:18.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:18 smithi137 ceph-mon[32568]: from='client.? 172.21.15.94:0/1459819764' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:08:18.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 systemd[1]: Stopping Ceph mon.a for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:19.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T14:08:18.920+0000 7f09ebb8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:19.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T14:08:18.920+0000 7f09ebb8c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T14:08:19.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:18 smithi094 podman[121838]: 2024-04-25 14:08:18.9822687 +0000 UTC m=+0.187464141 container died 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:19.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:19 smithi094 podman[121838]: 2024-04-25 14:08:19.403414767 +0000 UTC m=+0.608610192 container cleanup 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T14:08:19.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:19 smithi094 bash[121838]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:08:19.907 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:08:19 smithi094 podman[121859]: 2024-04-25 14:08:19.630064716 +0000 UTC m=+0.649783850 container remove 5f7c39663efe94fb178667c51dc5444c5da0e5b705580a90b7ba8949c593e5ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T14:08:20.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T14:08:20.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:20.867 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T14:08:20.867 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T14:08:20.867 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.b 2024-04-25T14:08:21.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:20 smithi137 systemd[1]: Stopping Ceph mon.b for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:21.563 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b[32563]: 2024-04-25T14:08:21.221+0000 7fdb51a9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:21.563 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b[32563]: 2024-04-25T14:08:21.221+0000 7fdb51a9a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:08:21.564 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 podman[70344]: 2024-04-25 14:08:21.303929588 +0000 UTC m=+0.200920937 container died d7eb3cd3aa0ab3efd1bcdeb0c66162953b610db6ae09df5e0af1e9e9c14873f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:08:21.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 podman[70344]: 2024-04-25 14:08:21.64732724 +0000 UTC m=+0.544318589 container cleanup d7eb3cd3aa0ab3efd1bcdeb0c66162953b610db6ae09df5e0af1e9e9c14873f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T14:08:21.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 bash[70344]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b 2024-04-25T14:08:22.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T14:08:22.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:21 smithi137 podman[70359]: 2024-04-25 14:08:21.873759331 +0000 UTC m=+0.566187474 container remove d7eb3cd3aa0ab3efd1bcdeb0c66162953b610db6ae09df5e0af1e9e9c14873f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T14:08:22.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:22 smithi137 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-25T14:08:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:22 smithi137 systemd[1]: Stopped Ceph mon.b for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T14:08:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:08:22 smithi137 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.b.service: Consumed 20.696s CPU time. 2024-04-25T14:08:22.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:22.148 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T14:08:22.148 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T14:08:22.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.c 2024-04-25T14:08:22.449 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:22 smithi138 systemd[1]: Stopping Ceph mon.c for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c[32973]: 2024-04-25T14:08:22.448+0000 7f0d24ab9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:22.746 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c[32973]: 2024-04-25T14:08:22.448+0000 7f0d24ab9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:08:22.747 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:22 smithi138 podman[72670]: 2024-04-25 14:08:22.571528835 +0000 UTC m=+0.226432265 container died 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:08:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:23 smithi138 podman[72670]: 2024-04-25 14:08:23.03850453 +0000 UTC m=+0.693407988 container cleanup 0a48be0527eadc7be11f44655f0fcfdd23db0a7c37cc3698416f7a02ae994f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:08:23.247 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:08:23 smithi138 bash[72670]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mon-c 2024-04-25T14:08:23.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T14:08:23.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:23.556 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T14:08:23.556 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T14:08:23.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:08:23.858 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:23 smithi094 systemd[1]: Stopping Ceph mgr.x for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:24.224 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:23 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T14:08:23.857+0000 7f3f29fe3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:24.224 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:23 smithi094 podman[124403]: 2024-04-25 14:08:23.888714216 +0000 UTC m=+0.136515417 container died fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:08:24.541 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 podman[124403]: 2024-04-25 14:08:24.324880362 +0000 UTC m=+0.572681562 container cleanup fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:08:24.541 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 bash[124403]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x 2024-04-25T14:08:24.792 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 podman[124417]: 2024-04-25 14:08:24.542749311 +0000 UTC m=+0.648474002 container remove fdb23ed7978a77febb9404e56708fb6afb1a9260a2c167e240b81f3982d05ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T14:08:24.792 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T14:08:24.793 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 systemd[1]: Stopped Ceph mgr.x for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T14:08:24.793 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:08:24 smithi094 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 33.148s CPU time. 2024-04-25T14:08:24.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T14:08:24.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:24.831 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T14:08:24.831 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T14:08:24.831 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.y 2024-04-25T14:08:25.142 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:08:24 smithi137 systemd[1]: Stopping Ceph mgr.y for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:25.485 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:08:25 smithi137 podman[70467]: 2024-04-25 14:08:25.181312488 +0000 UTC m=+0.153828878 container died 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T14:08:25.860 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:08:25 smithi137 podman[70467]: 2024-04-25 14:08:25.585830196 +0000 UTC m=+0.558346583 container cleanup 2e733b5a3d35e9f97c2ed8ed4e9d45b9ced24148dc166aa3668cd63dd1cc5440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T14:08:25.861 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:08:25 smithi137 bash[70467]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-y 2024-04-25T14:08:26.001 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T14:08:26.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:26.034 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T14:08:26.034 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T14:08:26.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.z 2024-04-25T14:08:26.476 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:08:26 smithi138 systemd[1]: Stopping Ceph mgr.z for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:26.736 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:08:26 smithi138 podman[72791]: 2024-04-25 14:08:26.476627825 +0000 UTC m=+0.172806526 container died 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T14:08:26.996 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:08:26 smithi138 podman[72791]: 2024-04-25 14:08:26.882175705 +0000 UTC m=+0.578354395 container cleanup 744714fd3aeac4db5eed3a773c2a884fd79df1fd1ac17212633555032aeb63a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T14:08:26.996 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:08:26 smithi138 bash[72791]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mgr-z 2024-04-25T14:08:27.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T14:08:27.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:27.295 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T14:08:27.295 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T14:08:27.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.0 2024-04-25T14:08:27.603 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:27 smithi094 systemd[1]: Stopping Ceph osd.0 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:27.974 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0[44411]: 2024-04-25T14:08:27.603+0000 7fc406f55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:27.974 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0[44411]: 2024-04-25T14:08:27.603+0000 7fc406f55700 -1 osd.0 91 *** Got signal Terminated *** 2024-04-25T14:08:27.974 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0[44411]: 2024-04-25T14:08:27.603+0000 7fc406f55700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:08:33.470 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:33 smithi094 podman[124525]: 2024-04-25 14:08:33.18903223 +0000 UTC m=+5.706761387 container died 200c596276c58cb24d61a54963d34b83bfd8ef71ab40616cdba2737cde3ce753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T14:08:33.724 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:33 smithi094 podman[124525]: 2024-04-25 14:08:33.545497993 +0000 UTC m=+6.063227149 container cleanup 200c596276c58cb24d61a54963d34b83bfd8ef71ab40616cdba2737cde3ce753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:33.724 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:33 smithi094 bash[124525]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0 2024-04-25T14:08:34.162 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:33 smithi094 podman[124542]: 2024-04-25 14:08:33.821653282 +0000 UTC m=+0.630931454 container remove 200c596276c58cb24d61a54963d34b83bfd8ef71ab40616cdba2737cde3ce753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:34.474 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.066657375 +0000 UTC m=+0.018098657 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:08:34.474 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.164120503 +0000 UTC m=+0.115561786 container create bbced2c6f5a3e6b9aec016555f16fadb2e23bd8c545834eb245c57c955b98c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:34.474 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.391614417 +0000 UTC m=+0.343055686 container init bbced2c6f5a3e6b9aec016555f16fadb2e23bd8c545834eb245c57c955b98c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:34.474 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.394725206 +0000 UTC m=+0.346166487 container start bbced2c6f5a3e6b9aec016555f16fadb2e23bd8c545834eb245c57c955b98c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T14:08:34.474 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.474834886 +0000 UTC m=+0.426276168 container attach bbced2c6f5a3e6b9aec016555f16fadb2e23bd8c545834eb245c57c955b98c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:34.724 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:08:34 smithi094 podman[124631]: 2024-04-25 14:08:34.687837136 +0000 UTC m=+0.639278410 container died bbced2c6f5a3e6b9aec016555f16fadb2e23bd8c545834eb245c57c955b98c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:08:35.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T14:08:35.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:35.336 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T14:08:35.337 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T14:08:35.337 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.1 2024-04-25T14:08:35.642 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:35 smithi137 systemd[1]: Stopping Ceph osd.1 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:36.142 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1[37198]: 2024-04-25T14:08:35.680+0000 7fa3c3551700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:36.143 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1[37198]: 2024-04-25T14:08:35.680+0000 7fa3c3551700 -1 osd.1 91 *** Got signal Terminated *** 2024-04-25T14:08:36.143 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1[37198]: 2024-04-25T14:08:35.680+0000 7fa3c3551700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:08:41.523 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:41 smithi137 podman[70605]: 2024-04-25 14:08:41.238105993 +0000 UTC m=+5.698772026 container died 8207e9eeee0473c77ba5f3cec649f09de26dab56e0e1c02a75b44013b5b3e9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T14:08:41.892 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:41 smithi137 podman[70605]: 2024-04-25 14:08:41.606798212 +0000 UTC m=+6.067464245 container cleanup 8207e9eeee0473c77ba5f3cec649f09de26dab56e0e1c02a75b44013b5b3e9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:41.892 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:41 smithi137 bash[70605]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1 2024-04-25T14:08:42.209 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:41 smithi137 podman[70624]: 2024-04-25 14:08:41.916756816 +0000 UTC m=+0.664819863 container remove 8207e9eeee0473c77ba5f3cec649f09de26dab56e0e1c02a75b44013b5b3e9dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T14:08:42.459 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:42 smithi137 podman[70717]: 2024-04-25 14:08:42.113685758 +0000 UTC m=+0.018526796 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:08:42.460 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:42 smithi137 podman[70717]: 2024-04-25 14:08:42.36762846 +0000 UTC m=+0.272469500 container create 91f7b064fbbfc77e8a112afc077860a551db11f34f611141e467f6145ea87f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:42.892 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:42 smithi137 podman[70717]: 2024-04-25 14:08:42.570497546 +0000 UTC m=+0.475338587 container init 91f7b064fbbfc77e8a112afc077860a551db11f34f611141e467f6145ea87f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:08:42.892 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:42 smithi137 podman[70717]: 2024-04-25 14:08:42.573587185 +0000 UTC m=+0.478428231 container start 91f7b064fbbfc77e8a112afc077860a551db11f34f611141e467f6145ea87f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-25T14:08:42.893 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:42 smithi137 podman[70717]: 2024-04-25 14:08:42.700183424 +0000 UTC m=+0.605024469 container attach 91f7b064fbbfc77e8a112afc077860a551db11f34f611141e467f6145ea87f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:43.393 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:08:43 smithi137 podman[70717]: 2024-04-25 14:08:43.128657312 +0000 UTC m=+1.033498359 container died 91f7b064fbbfc77e8a112afc077860a551db11f34f611141e467f6145ea87f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:08:43.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T14:08:43.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:43.792 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T14:08:43.792 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T14:08:43.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.2 2024-04-25T14:08:44.095 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:43 smithi138 systemd[1]: Stopping Ceph osd.2 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:44.496 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T14:08:44.095+0000 7f0b22646700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:44.496 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T14:08:44.095+0000 7f0b22646700 -1 osd.2 91 *** Got signal Terminated *** 2024-04-25T14:08:44.497 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2[38119]: 2024-04-25T14:08:44.095+0000 7f0b22646700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:08:49.970 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:49 smithi138 podman[72929]: 2024-04-25 14:08:49.704897278 +0000 UTC m=+5.720145814 container died cd5d55b4f86df44c875ddf7081eafc998674a483be170eadd4c024d4c03d9d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T14:08:50.246 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:50 smithi138 podman[72929]: 2024-04-25 14:08:50.054139231 +0000 UTC m=+6.069387757 container cleanup cd5d55b4f86df44c875ddf7081eafc998674a483be170eadd4c024d4c03d9d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T14:08:50.246 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:50 smithi138 bash[72929]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2 2024-04-25T14:08:50.708 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:50 smithi138 podman[72946]: 2024-04-25 14:08:50.35592967 +0000 UTC m=+0.654483467 container remove cd5d55b4f86df44c875ddf7081eafc998674a483be170eadd4c024d4c03d9d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T14:08:50.996 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:50 smithi138 podman[73038]: 2024-04-25 14:08:50.612558568 +0000 UTC m=+0.017895421 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:08:50.996 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:50 smithi138 podman[73038]: 2024-04-25 14:08:50.774051536 +0000 UTC m=+0.179388428 container create 7bcabeedf5fb680810cdda7f12d849047672d53cf1f73c31f5cd1c1675c39bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T14:08:51.308 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:51 smithi138 podman[73038]: 2024-04-25 14:08:51.018345455 +0000 UTC m=+0.423682300 container init 7bcabeedf5fb680810cdda7f12d849047672d53cf1f73c31f5cd1c1675c39bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T14:08:51.308 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:51 smithi138 podman[73038]: 2024-04-25 14:08:51.021454777 +0000 UTC m=+0.426791625 container start 7bcabeedf5fb680810cdda7f12d849047672d53cf1f73c31f5cd1c1675c39bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T14:08:51.308 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:51 smithi138 podman[73038]: 2024-04-25 14:08:51.119629733 +0000 UTC m=+0.524966588 container attach 7bcabeedf5fb680810cdda7f12d849047672d53cf1f73c31f5cd1c1675c39bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:08:51.582 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:08:51 smithi138 podman[73038]: 2024-04-25 14:08:51.309622881 +0000 UTC m=+0.714959738 container died 7bcabeedf5fb680810cdda7f12d849047672d53cf1f73c31f5cd1c1675c39bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T14:08:51.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T14:08:51.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:51.966 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T14:08:51.966 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T14:08:51.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.3 2024-04-25T14:08:52.281 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:52 smithi094 systemd[1]: Stopping Ceph osd.3 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:08:52.535 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:52 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T14:08:52.281+0000 7f31ea789700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:08:52.536 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:52 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T14:08:52.281+0000 7f31ea789700 -1 osd.3 91 *** Got signal Terminated *** 2024-04-25T14:08:52.536 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:52 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T14:08:52.281+0000 7f31ea789700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:08:54.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:53.937+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:54.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:54 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:54.684+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:55.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:54 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:54.969+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:55.967 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:55 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T14:08:55.784+0000 7f31e2d37700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:32.219569+0000 front 2024-04-25T14:08:32.219538+0000 (oldest deadline 2024-04-25T14:08:55.620448+0000) 2024-04-25T14:08:55.967 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:55 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:55.644+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:56.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:08:55 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:08:55.967+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:08:56.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:55 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:55.970+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:56.974 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3[51106]: 2024-04-25T14:08:56.775+0000 7f31e2d37700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:32.219569+0000 front 2024-04-25T14:08:32.219538+0000 (oldest deadline 2024-04-25T14:08:55.620448+0000) 2024-04-25T14:08:56.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:56 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:56.652+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:57.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:08:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:08:56.995+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:08:57.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:57.006+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:57.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:08:57 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:08:57.010+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:08:57.974 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:57 smithi094 podman[124737]: 2024-04-25 14:08:57.813541993 +0000 UTC m=+5.652579692 container died da4ab22445971fb247556a29b0e4a0b069890ce5479b74141af64e3b294d5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:08:57.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:57 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:57.658+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:57.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:08:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:08:57.704+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:08:58.259 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:08:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:08:57.998+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:08:58.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:57 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:57.986+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:58.458 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:08:58 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:08:58.046+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:08:58.458 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:58 smithi094 podman[124737]: 2024-04-25 14:08:58.166005281 +0000 UTC m=+6.005042977 container cleanup da4ab22445971fb247556a29b0e4a0b069890ce5479b74141af64e3b294d5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T14:08:58.458 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:58 smithi094 bash[124737]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3 2024-04-25T14:08:58.724 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:58 smithi094 podman[124766]: 2024-04-25 14:08:58.459372399 +0000 UTC m=+0.645779636 container remove da4ab22445971fb247556a29b0e4a0b069890ce5479b74141af64e3b294d5779 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:58.724 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:58 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:58.646+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:58.965 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:08:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:08:58.670+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:08:59.066 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:58 smithi094 podman[124853]: 2024-04-25 14:08:58.681386561 +0000 UTC m=+0.018123226 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:08:59.066 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:58 smithi094 podman[124853]: 2024-04-25 14:08:58.785327991 +0000 UTC m=+0.122064642 container create a71b22be6f514529aeaf9fc90639a47786e9d743d59bd75f7219bc2307265889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:08:59.066 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:59 smithi094 podman[124853]: 2024-04-25 14:08:59.012616996 +0000 UTC m=+0.349353645 container init a71b22be6f514529aeaf9fc90639a47786e9d743d59bd75f7219bc2307265889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T14:08:59.067 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:59 smithi094 podman[124853]: 2024-04-25 14:08:59.015631453 +0000 UTC m=+0.352368107 container start a71b22be6f514529aeaf9fc90639a47786e9d743d59bd75f7219bc2307265889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:08:59.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:08:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:08:58.965+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:08:59.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:58 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:58.946+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:08:59.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:08:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:08:59.066+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:08:59.475 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:59 smithi094 podman[124853]: 2024-04-25 14:08:59.112758101 +0000 UTC m=+0.449494750 container attach a71b22be6f514529aeaf9fc90639a47786e9d743d59bd75f7219bc2307265889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD) 2024-04-25T14:08:59.475 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 14:08:59 smithi094 podman[124853]: 2024-04-25 14:08:59.308310937 +0000 UTC m=+0.645047591 container died a71b22be6f514529aeaf9fc90639a47786e9d743d59bd75f7219bc2307265889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T14:08:59.873 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:08:59 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:08:59.661+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:08:59.895 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T14:08:59.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:08:59.933 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T14:08:59.933 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T14:08:59.933 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.4 2024-04-25T14:08:59.946 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:08:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:08:59.690+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:00.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:00 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:00.099+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:00.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:08:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:08:59.947+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:00.250 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:08:59 smithi137 systemd[1]: Stopping Ceph osd.4 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:00.250 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:08:59 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:08:59.939+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:00.642 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:00 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T14:09:00.251+0000 7fcbc1183700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:00.642 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:00 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T14:09:00.251+0000 7fcbc1183700 -1 osd.4 91 *** Got signal Terminated *** 2024-04-25T14:09:00.643 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:00 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T14:09:00.251+0000 7fcbc1183700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:00.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:00 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:00.643+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:00.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:00.685+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:00.931+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:01.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:00 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:00.934+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:01.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:01 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:01.109+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:01.965 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:01 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:01.699+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:01.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:01 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:01.685+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:02.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:01 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:01.966+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:02.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:02 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:02.114+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:02.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:01 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:01.915+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:02.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:02 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:02.090+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:02.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:02 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:02.658+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:02.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:02 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:02.919+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:02.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:02 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:02.919+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:02.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:02 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:02.701+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:02.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:02 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:02.701+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:03.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:02 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:02.758+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:03.142 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:02 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:02.911+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:03.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:03 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:03.113+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:03.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:03 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:03.109+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:03.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:03 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:03.653+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:03.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:03 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:03.888+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:03.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:03 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:03.888+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:03.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:03 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:03.717+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:03.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:03 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:03.717+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:04.044 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:03 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T14:09:03.851+0000 7fcbb9731700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.111807+0000 front 2024-04-25T14:08:38.111835+0000 (oldest deadline 2024-04-25T14:09:03.411963+0000) 2024-04-25T14:09:04.044 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:03 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:03.782+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:04.044 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:03 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:03.949+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:04.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:04 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:04.069+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:04.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:04 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:04.101+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:04.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:04 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:04.628+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:04.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:04 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:04.700+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:04.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:04 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:04.700+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:04.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:04 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:04.886+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:04.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:04 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:04.886+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:05.056 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:04 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4[44132]: 2024-04-25T14:09:04.808+0000 7fcbb9731700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.111807+0000 front 2024-04-25T14:08:38.111835+0000 (oldest deadline 2024-04-25T14:09:03.411963+0000) 2024-04-25T14:09:05.056 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:04 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:04.733+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:05.056 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:04 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:04.960+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:05.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:05 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:05.089+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:05.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:05.106+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:05.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:05 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:05.637+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:05.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:05.684+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:05.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:05.685+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:05.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:05.878+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:05.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:05 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:05.878+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:06.000 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:05 smithi137 podman[70824]: 2024-04-25 14:09:05.764151102 +0000 UTC m=+5.639140286 container died 200bc586b1d6c92d70d9107034d21af808f6fcd9ff30e59d08551908197a3bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:09:06.000 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:05 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:05.728+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:06.001 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:05 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:05.920+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:06.001 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:05 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:05.920+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:06.278 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:06 smithi137 podman[70824]: 2024-04-25 14:09:06.092405768 +0000 UTC m=+5.967394960 container cleanup 200bc586b1d6c92d70d9107034d21af808f6fcd9ff30e59d08551908197a3bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:09:06.278 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:06 smithi137 bash[70824]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4 2024-04-25T14:09:06.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:06.070+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:06.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:06.070+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:06.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:06 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:06.146+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:06.642 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:06 smithi137 podman[70852]: 2024-04-25 14:09:06.361266266 +0000 UTC m=+0.592716524 container remove 200bc586b1d6c92d70d9107034d21af808f6fcd9ff30e59d08551908197a3bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:09:06.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:06.651+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:06.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:06 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:06.651+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:06.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:06 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:06.710+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:06.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:06 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:06.710+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:06.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:06 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:06.906+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:06.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:06 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:06.906+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:07.046 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:06 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:06.735+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:07.046 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:06 smithi137 podman[70946]: 2024-04-25 14:09:06.748925133 +0000 UTC m=+0.017980907 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:07.046 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:06 smithi137 podman[70946]: 2024-04-25 14:09:06.853822398 +0000 UTC m=+0.122878199 container create c1bf86e59ab4b9b7d292805469dfb04ec910ad78d91244a229595bfc8ba41814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T14:09:07.046 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:06 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:06.928+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:07.047 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:06 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:06.928+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:07.336 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:07 smithi137 podman[70946]: 2024-04-25 14:09:07.047472508 +0000 UTC m=+0.316528270 container init c1bf86e59ab4b9b7d292805469dfb04ec910ad78d91244a229595bfc8ba41814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:09:07.336 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:07 smithi137 podman[70946]: 2024-04-25 14:09:07.050713891 +0000 UTC m=+0.319769651 container start c1bf86e59ab4b9b7d292805469dfb04ec910ad78d91244a229595bfc8ba41814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:09:07.336 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:07 smithi137 podman[70946]: 2024-04-25 14:09:07.139510626 +0000 UTC m=+0.408566397 container attach c1bf86e59ab4b9b7d292805469dfb04ec910ad78d91244a229595bfc8ba41814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T14:09:07.336 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:09:07 smithi137 podman[70946]: 2024-04-25 14:09:07.33741441 +0000 UTC m=+0.606470170 container died c1bf86e59ab4b9b7d292805469dfb04ec910ad78d91244a229595bfc8ba41814 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T14:09:07.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:07.060+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:07.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:07.060+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:07.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:07.186+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:07.924 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:07 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:07.722+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:07.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T14:09:07.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:07.642+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:07.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:07 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:07.642+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:07.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:07.744+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:07.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:07.744+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:07.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:07.888+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:07.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:07 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:07.888+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:08.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:08.008 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T14:09:08.008 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T14:09:08.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.5 2024-04-25T14:09:08.290 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 systemd[1]: Stopping Ceph osd.5 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:08.290 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:08.180+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:08.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:07 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:07.924+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:08.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:07 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:07.924+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:08.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:08.094+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:08.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:08.094+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:08.746 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:08.326+0000 7f5d94bff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:08.746 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:08.326+0000 7f5d94bff700 -1 osd.5 91 *** Got signal Terminated *** 2024-04-25T14:09:08.746 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:08.326+0000 7f5d94bff700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:08.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:08.613+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:08.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:08.613+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:09.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:08 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:08.746+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:09.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:08 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:08.951+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:09.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:08 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:08.951+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:09.225 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:08.839+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:09.225 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:08.839+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:09.225 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:08.753+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:09.225 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:08 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:08.753+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:09.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:09.083+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:09.475 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:09.083+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:09.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:09.225+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:09.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:09.627+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:09.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:09 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:09.627+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:09.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:09.739+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:09.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:09.739+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:09.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:09.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:09.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:09 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:09.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:10.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:09 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:09.786+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:10.142 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:09 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:09.978+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:10.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:09 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:09.978+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:10.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:10.097+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:10.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:10.097+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:10.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:10.218+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:10.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:10.602+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:10.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:10 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:10.602+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:10.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:10.831+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:10.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:10.831+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:10.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:10.692+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:10.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:10.692+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:10.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:10 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:10.692+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:44.036650+0000 front 2024-04-25T14:08:44.036611+0000 (oldest deadline 2024-04-25T14:09:09.936762+0000) 2024-04-25T14:09:11.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:10 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:10.767+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:11.142 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:10 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:10.940+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:11.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:10 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:10.940+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:11.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:10 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:10.940+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:11.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:11.061+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:11.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:11.061+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:11.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:11.174+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:11.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:11.174+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:11.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:11.639+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:11.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:11.639+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:11.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:11.639+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:11.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:11.835+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:11.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:11.835+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:11.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:11.680+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:11.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:11.680+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:11.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:11 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:11.680+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:44.036650+0000 front 2024-04-25T14:08:44.036611+0000 (oldest deadline 2024-04-25T14:09:09.936762+0000) 2024-04-25T14:09:12.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:11.771+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:12.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:11.946+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:12.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:11.946+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:12.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:11 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:11.946+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:12.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.028+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:12.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.028+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:12.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.028+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:12.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:12.176+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:12.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:12.176+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:12.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:12.627+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:12.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:12.627+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:12.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:12.627+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:12.996 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:12.643+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.933579+0000 front 2024-04-25T14:08:31.933610+0000 (oldest deadline 2024-04-25T14:08:57.134431+0000) 2024-04-25T14:09:12.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:12.643+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:38.235899+0000 front 2024-04-25T14:08:38.235632+0000 (oldest deadline 2024-04-25T14:09:02.335606+0000) 2024-04-25T14:09:12.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5[44617]: 2024-04-25T14:09:12.643+0000 7f5d8d1ad700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:44.036650+0000 front 2024-04-25T14:08:44.036611+0000 (oldest deadline 2024-04-25T14:09:09.936762+0000) 2024-04-25T14:09:12.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:12.794+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:12.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:12.794+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:12.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:12 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:12.794+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:13.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:12.782+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:13.143 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:12.782+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:13.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:12.925+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:13.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:12.925+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:13.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:12 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:12.925+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:13.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.994+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:13.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.994+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:13.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:12 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:12.994+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:13.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:13.196+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:13.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:13.196+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:13.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:13.671+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:13.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:13.671+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:13.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:13.671+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:14.052 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:13 smithi138 podman[73156]: 2024-04-25 14:09:13.777504727 +0000 UTC m=+5.571717090 container died 185b6b25d175763c9fe1a2a7667de0d39b39bbbf9b41d339ea1bf11d116369c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:09:14.052 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:13.761+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:14.052 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:13.761+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:14.052 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:13 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:13.761+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:14.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:13 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:13.748+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:14.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:13 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:13.748+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:14.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:13 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:13.911+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:14.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:13 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:13.911+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:14.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:13 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:13.911+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:14.404 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73156]: 2024-04-25 14:09:14.169999689 +0000 UTC m=+5.964212038 container cleanup 185b6b25d175763c9fe1a2a7667de0d39b39bbbf9b41d339ea1bf11d116369c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T14:09:14.404 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 bash[73156]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5 2024-04-25T14:09:14.404 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:14 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:14.232+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:14.404 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:14 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:14.232+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:14.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:13.982+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:14.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:13.982+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:14.474 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:13 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:13.982+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:14.729 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73172]: 2024-04-25 14:09:14.4050612 +0000 UTC m=+0.630065738 container remove 185b6b25d175763c9fe1a2a7667de0d39b39bbbf9b41d339ea1bf11d116369c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:09:14.933 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:14.676+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:14.934 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:14.676+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:14.934 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:14.676+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:14.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:14 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:14.796+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:14.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:14 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:14.796+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:14.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:14 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:14.796+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:14.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73262]: 2024-04-25 14:09:14.730905662 +0000 UTC m=+0.113486751 container create 5045be91c2d34c4ed398dd40b2f15445d9783d84da1dd938392b4257d8f9d69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T14:09:14.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73262]: 2024-04-25 14:09:14.635237709 +0000 UTC m=+0.017818814 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:14.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73262]: 2024-04-25 14:09:14.975375829 +0000 UTC m=+0.357956913 container init 5045be91c2d34c4ed398dd40b2f15445d9783d84da1dd938392b4257d8f9d69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T14:09:14.997 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:14 smithi138 podman[73262]: 2024-04-25 14:09:14.978533525 +0000 UTC m=+0.361114618 container start 5045be91c2d34c4ed398dd40b2f15445d9783d84da1dd938392b4257d8f9d69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:09:15.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:14 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:14.740+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:15.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:14 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:14.740+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:15.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:14 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:14.933+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:15.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:14 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:14.933+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:15.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:14 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:14.933+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:15.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:14.934+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:15.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:14.934+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:15.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:14 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:14.934+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:15.497 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:15 smithi138 podman[73262]: 2024-04-25 14:09:15.08407973 +0000 UTC m=+0.466660831 container attach 5045be91c2d34c4ed398dd40b2f15445d9783d84da1dd938392b4257d8f9d69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:09:15.497 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:09:15 smithi138 podman[73262]: 2024-04-25 14:09:15.277696796 +0000 UTC m=+0.660277882 container died 5045be91c2d34c4ed398dd40b2f15445d9783d84da1dd938392b4257d8f9d69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:09:15.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:15.227+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:15.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:15.227+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:15.871 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:15.811+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:15.871 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:15.811+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:15.871 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:15 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:15.811+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:15.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T14:09:15.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:15.936 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T14:09:15.936 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T14:09:15.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.6 2024-04-25T14:09:15.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:15.919+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:15.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:15.919+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:15.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:15.919+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:15.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:15.723+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:15.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:15.723+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:15.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:15 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:15.723+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:16.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:15.734+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:16.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:15.734+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:16.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:15.924+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:16.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:15.924+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:16.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:15 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:15.924+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:16.474 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:15 smithi094 systemd[1]: Stopping Ceph osd.6 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:16.475 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.219+0000 7f910d0a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:16.475 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.219+0000 7f910d0a6700 -1 osd.6 91 *** Got signal Terminated *** 2024-04-25T14:09:16.475 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.219+0000 7f910d0a6700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:16.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:16 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:16.201+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:16.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:16 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:16.201+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:16.969 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:16 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:16.713+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:16.969 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:16 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:16.713+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:16.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:16.907+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:16.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:16.907+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:16.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:16.907+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:16.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.688+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:16.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.688+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:16.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:16.688+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:17.169 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:16 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:16.784+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:17.169 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:16 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:16.784+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:17.169 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:16 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:16.784+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:17.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:16 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:16.969+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:17.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:16 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:16.969+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:17.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:16 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:16.969+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:17.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:17 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:17.170+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:17.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:17 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:17.170+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:17.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:17.879+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:17.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:17.879+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:17.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:17.879+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:17.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:17.672+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:17.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:17.672+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:17.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:17 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:17.672+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:18.001 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:17 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:17.667+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:18.001 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:17 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:17.667+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:18.122 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:17 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:17.753+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:18.123 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:17 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:17.753+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:18.123 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:17 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:17.753+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:18.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:18.001+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:18.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:18.001+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:18.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:18.001+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:18.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:18 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:18.123+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:18.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:18 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:18.123+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:18.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:18.852+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:18.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:18.852+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:18.975 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:18.852+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:18.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:18.711+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:18.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:18.711+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:18.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:18 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:18.711+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:19.040 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:18.688+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:19.040 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:18.688+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:19.040 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:18 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:18.688+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:19.132 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:18 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:18.792+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:19.132 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:18 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:18.792+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:19.132 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:18 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:18.792+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:19.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:19.041+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:19.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:19.041+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:19.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:19.041+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:19.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:19.041+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:19.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:19.132+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:19.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:19.132+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:19.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:19.132+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:19.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:19.641+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:19.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:19.641+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:19.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:19 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:19.641+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:20.107 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:19.809+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:20.107 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:19.809+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:20.107 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:19 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:19.809+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:20.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:19.893+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:20.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:19.893+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:20.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:19.893+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:20.224 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:19.752+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:20.225 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:19.752+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:20.225 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:19 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:19.752+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:20.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:20.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:20.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:20.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:20.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:20.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:20.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:20.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:20.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:20.107+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:20.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:20.107+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:20.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:20.107+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:20.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:20.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:20.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:20.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:20.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:20.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:20.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:20.898+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:20.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:20.898+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:20.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:20.898+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:20.974 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:20.717+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.241319+0000 front 2024-04-25T14:08:28.241345+0000 (oldest deadline 2024-04-25T14:08:54.141648+0000) 2024-04-25T14:09:20.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:20.717+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.444212+0000 front 2024-04-25T14:08:40.444135+0000 (oldest deadline 2024-04-25T14:09:05.744168+0000) 2024-04-25T14:09:20.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:20.717+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.045349+0000 front 2024-04-25T14:08:48.045426+0000 (oldest deadline 2024-04-25T14:09:11.545425+0000) 2024-04-25T14:09:20.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:20 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6[57643]: 2024-04-25T14:09:20.717+0000 7f9105654700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.846339+0000 front 2024-04-25T14:08:56.846398+0000 (oldest deadline 2024-04-25T14:09:20.346534+0000) 2024-04-25T14:09:21.102 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:20.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:21.102 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:20.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:21.102 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:20.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:21.102 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:20 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:20.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:21.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:21.028+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:21.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:21.029+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:21.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:21.029+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:21.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:21.029+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:21.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:21.102+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:21.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:21.102+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:21.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:21.102+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:21.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:21.581+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:21.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:21.581+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:21.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:21 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:21.581+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:22.018 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:21 smithi094 podman[124959]: 2024-04-25 14:09:21.756493966 +0000 UTC m=+5.625777786 container died 32eb9cb47d9bae6f79a36c4beca468731573ed468e7be5b6adbdb50940f8f33d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:09:22.018 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:21 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:21.896+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:22.019 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:21 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:21.896+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:22.019 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:21 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:21.896+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:22.083 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:21.818+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:22.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:21.818+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:22.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:21.818+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:22.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:21 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:21.818+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:22.368 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[124959]: 2024-04-25 14:09:22.135437386 +0000 UTC m=+6.004721208 container cleanup 32eb9cb47d9bae6f79a36c4beca468731573ed468e7be5b6adbdb50940f8f33d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:09:22.369 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 bash[124959]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6 2024-04-25T14:09:22.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:22.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:22.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:22.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:22.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:22.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:22.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:22.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:22.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:22.083+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:22.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:22.083+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:22.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:22.083+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:22.701 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[124979]: 2024-04-25 14:09:22.369773091 +0000 UTC m=+0.615576964 container remove 32eb9cb47d9bae6f79a36c4beca468731573ed468e7be5b6adbdb50940f8f33d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T14:09:22.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:22.612+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:22.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:22.612+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:22.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:22 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:22.612+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:22.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:22 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:22.940+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:22.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:22 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:22.940+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:22 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:22.940+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:22 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:22.940+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[125073]: 2024-04-25 14:09:22.605139775 +0000 UTC m=+0.018101643 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[125073]: 2024-04-25 14:09:22.720973214 +0000 UTC m=+0.133935070 container create 5cd7e692585ef500d15cad4e68b4c0118c438728a977a8dbc066ad3702de848a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[125073]: 2024-04-25 14:09:22.931427004 +0000 UTC m=+0.344388859 container init 5cd7e692585ef500d15cad4e68b4c0118c438728a977a8dbc066ad3702de848a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:09:22.975 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:22 smithi094 podman[125073]: 2024-04-25 14:09:22.934922116 +0000 UTC m=+0.347883975 container start 5cd7e692585ef500d15cad4e68b4c0118c438728a977a8dbc066ad3702de848a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:09:23.083 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:22.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:23.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:22.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:23.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:22.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:23.084 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:22 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:22.830+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:23.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:23.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:23.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:23.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:23.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:23.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:23.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:23.010+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:23.474 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:23 smithi094 podman[125073]: 2024-04-25 14:09:23.031610311 +0000 UTC m=+0.444572175 container attach 5cd7e692585ef500d15cad4e68b4c0118c438728a977a8dbc066ad3702de848a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T14:09:23.474 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:09:23 smithi094 podman[125073]: 2024-04-25 14:09:23.226539821 +0000 UTC m=+0.639501683 container died 5cd7e692585ef500d15cad4e68b4c0118c438728a977a8dbc066ad3702de848a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:09:23.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:23.084+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:23.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:23.084+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:23.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:23.084+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:23.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T14:09:23.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:23.888 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T14:09:23.888 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T14:09:23.888 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.7 2024-04-25T14:09:23.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:23.609+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:23.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:23.609+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:23.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:23 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:23.609+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:24.041 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:23.789+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:24.041 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:23.789+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:24.041 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:23.789+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:24.041 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:23 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:23.789+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:24.186 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:23 smithi137 systemd[1]: Stopping Ceph osd.7 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:24.186 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:24.060+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:24.186 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:24.060+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:24.186 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:24.060+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:24.187 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:24.060+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:24.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:23 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:23.942+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:24.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:23 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:23.942+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:24.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:23 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:23.942+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:24.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:23 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:23.942+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:24.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:24.041+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:24.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:24.041+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:24.497 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:24.041+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.186+0000 7feae0b40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.186+0000 7feae0b40700 -1 osd.7 91 *** Got signal Terminated *** 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.186+0000 7feae0b40700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.594+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.594+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:24.594 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:24.594+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:25.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:24 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:24.908+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:25.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:24 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:24.908+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:25.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:24 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:24.908+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:25.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:24 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:24.908+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.036+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.036+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.036+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:24.812+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:24.812+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:25.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:24.812+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:25.248 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:24 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:24.812+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:25.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:25.053+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:25.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:25.053+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:25.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:25.053+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:25.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:25.053+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:25.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:25.620+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:25.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:25.620+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:25.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:25 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:25.620+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:26.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:25 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:25.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:26.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:25 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:25.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:26.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:25 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:25.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:26.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:25 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:25.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:26.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:25 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:25.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:26.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.991+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:26.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.991+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:25.991+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:25.829+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:25.829+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:25.829+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:25.829+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:26.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:25 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:25.829+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:26.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:26.021+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:26.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:26.021+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:26.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:26.021+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:26.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:26.021+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:26.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:26.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:26.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:26.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:26.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:26.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:26.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:26 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:26.603+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:00.631851+0000 front 2024-04-25T14:09:00.631827+0000 (oldest deadline 2024-04-25T14:09:25.931949+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:26.992+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:26.992+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:26.992+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:26.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:26.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:26.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:27.148 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:26.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:27.149 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:26 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:26.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:27.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:26 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:26.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:27.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:26 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:26.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:27.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:26 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:26.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:27.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:26 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:26.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:27.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:26 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:26.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:27.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:27.054+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:27.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:27.054+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:27.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:27.054+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:27.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:27.054+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:27.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:27.600+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:27.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:27.600+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:27.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:27.600+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:27.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:27 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:27.600+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:00.631851+0000 front 2024-04-25T14:09:00.631827+0000 (oldest deadline 2024-04-25T14:09:25.931949+0000) 2024-04-25T14:09:28.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:27.832+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:28.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:27.832+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:28.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:27.832+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:28.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:27.832+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:28.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:27 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:27.832+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:28.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:27.964+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:28.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:27.964+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:28.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:27.964+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:28.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:27.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:28.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:27.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:28.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:27.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:28.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:27.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:28.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:27 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:27.808+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:28.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.022+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:28.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.022+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:28.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.022+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:28.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.022+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:28.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.022+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:28.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:28.583+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.326339+0000 front 2024-04-25T14:08:39.326373+0000 (oldest deadline 2024-04-25T14:09:02.127175+0000) 2024-04-25T14:09:28.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:28.583+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:46.828449+0000 front 2024-04-25T14:08:46.828335+0000 (oldest deadline 2024-04-25T14:09:12.128359+0000) 2024-04-25T14:09:28.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:28.583+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.429581+0000 front 2024-04-25T14:08:54.429629+0000 (oldest deadline 2024-04-25T14:09:18.529487+0000) 2024-04-25T14:09:28.893 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7[51109]: 2024-04-25T14:09:28.583+0000 7feadd0f6700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:00.631851+0000 front 2024-04-25T14:09:00.631827+0000 (oldest deadline 2024-04-25T14:09:25.931949+0000) 2024-04-25T14:09:29.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:28 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:28.844+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:29.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:28 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:28.844+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:29.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:28 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:28.844+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:29.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:28 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:28.844+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:29.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:28 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:28.844+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:29.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:28.960+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:28.960+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:28.960+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:28.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:28.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:28.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:28.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:29.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:28 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:28.804+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:29.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.979+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:29.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.979+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:29.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.979+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:29.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.979+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:29.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:28.979+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:29.892 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:29 smithi137 podman[71053]: 2024-04-25 14:09:29.62729531 +0000 UTC m=+5.549965887 container died 1890b6d09e9ab6c2f8020f30397f272e5810b08530ceaf65154f9f88309118ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:09:30.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:29 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:29.804+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:30.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:29 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:29.804+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:30.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:29 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:29.804+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:30.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:29 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:29.804+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:30.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:29 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:29.804+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71053]: 2024-04-25 14:09:30.00316436 +0000 UTC m=+5.925834992 container cleanup 1890b6d09e9ab6c2f8020f30397f272e5810b08530ceaf65154f9f88309118ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 bash[71053]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:30 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:30.007+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:30 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:30.007+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:30 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:30.007+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:30.245 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:30 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:30.007+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:30.246 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:30 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:30.007+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:30.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:29.797+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:30.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:29.797+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:30.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:29.797+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:29.797+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:29.797+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:29.948+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:29.948+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:29.948+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:30.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:29 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:29.948+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:30.575 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71070]: 2024-04-25 14:09:30.245945326 +0000 UTC m=+0.618179361 container remove 1890b6d09e9ab6c2f8020f30397f272e5810b08530ceaf65154f9f88309118ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:09:30.889 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71160]: 2024-04-25 14:09:30.478856629 +0000 UTC m=+0.017852445 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:30.889 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71160]: 2024-04-25 14:09:30.580028757 +0000 UTC m=+0.119024616 container create 8c7f1116fd0e63dd7d82e6cc13ad263b2b081ac5470c1f69f7aecee31a8e8712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:09:30.889 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71160]: 2024-04-25 14:09:30.807566998 +0000 UTC m=+0.346562815 container init 8c7f1116fd0e63dd7d82e6cc13ad263b2b081ac5470c1f69f7aecee31a8e8712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T14:09:30.890 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71160]: 2024-04-25 14:09:30.811798433 +0000 UTC m=+0.350794251 container start 8c7f1116fd0e63dd7d82e6cc13ad263b2b081ac5470c1f69f7aecee31a8e8712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T14:09:30.890 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:30 smithi137 podman[71160]: 2024-04-25 14:09:30.890618361 +0000 UTC m=+0.429614178 container attach 8c7f1116fd0e63dd7d82e6cc13ad263b2b081ac5470c1f69f7aecee31a8e8712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T14:09:31.142 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:09:31 smithi137 podman[71160]: 2024-04-25 14:09:31.111641758 +0000 UTC m=+0.650637583 container died 8c7f1116fd0e63dd7d82e6cc13ad263b2b081ac5470c1f69f7aecee31a8e8712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:09:31.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:31 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:31.005+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:31.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:31 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:31.005+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:31.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:31 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:31.005+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:31.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:31 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:31.005+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:31.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:31 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:31.005+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:31.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:30 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:30.769+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:31.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:30 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:30.769+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:31.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:30 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:30.769+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:31.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:30 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:30.769+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:31.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:30 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:30.769+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:31.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:30.950+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:31.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:30.950+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:31.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:30.950+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:31.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:30.950+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:31.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:30.788+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:31.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:30.788+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:31.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:30.788+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:31.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:30.788+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:31.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:30 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:30.788+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:31.703 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T14:09:31.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:31.781 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T14:09:31.781 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T14:09:31.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.8 2024-04-25T14:09:32.091 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:31 smithi138 systemd[1]: Stopping Ceph osd.8 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:32.091 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:31.937+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:32.091 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:31.937+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:32.091 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:31.937+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:31.937+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:31.758+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:31.758+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:31.758+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:31.758+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:32.092 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:31 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:31.758+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:32.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:31.791+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:32.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:31.791+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:32.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:31.791+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:32.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:31.791+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:32.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:31 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:31.791+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:32.331 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:32.031+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:32.331 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:32.031+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:32.332 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:32.031+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:32.332 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:32.031+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:32.332 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:32.031+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:32.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.091+0000 7fea5e451700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:32.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.091+0000 7fea5e451700 -1 osd.8 91 *** Got signal Terminated *** 2024-04-25T14:09:32.496 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.091+0000 7fea5e451700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:32.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.965+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:32.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.965+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:32.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.965+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:32.965+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:32.716+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:32.716+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:32.716+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:32.716+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:32.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:32 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:32.716+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:33.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:32.823+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:33.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:32.823+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:33.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:32.823+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:33.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:32.823+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:33.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:32 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:32.823+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:33.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:33 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:33.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:33.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:33 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:33.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:33.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:33 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:33.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:33.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:33 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:33.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:33.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:33 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:33.000+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:34.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:33.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:34.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:33.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:34.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:33.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:34.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:33.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:34.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:33 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:33.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:34.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:33.961+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:34.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:33.961+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:34.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:33.961+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:34.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:33.961+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:34.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:33.759+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:34.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:33.759+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:34.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:33.759+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:34.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:33.759+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:34.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:33 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:33.759+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:34.294 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:34 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:34.027+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:34.295 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:34 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:34.027+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:34.295 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:34 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:34.027+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:34.295 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:34 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:34.027+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:34.295 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:34 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:34.027+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:34.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:34.762+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:34.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:34.762+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:34.974 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:34.762+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:34.975 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:34.762+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:34.975 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:34 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:34.762+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:34.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:34.939+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:34.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:34.939+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:34.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:34.939+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:34.939+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:34.739+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:34.739+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:34.739+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:34.739+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:34.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:34 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:34.739+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:35.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:35.062+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:35.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:35.062+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:35.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:35.062+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:35.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:35.062+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:35.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:35 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:35.062+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:35.932+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:35.932+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:35.932+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:35.932+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:35.698+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:35.698+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:35.698+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:35.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:35.698+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:35.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:35 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:35.698+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:36.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:35 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:35.798+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:36.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:36 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:36.058+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:36.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:36 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:36.058+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:36.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:36 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:36.058+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:36.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:36 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:36.058+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:36.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:36 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:36.058+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:36.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:36.894+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:35.862230+0000 front 2024-04-25T14:08:35.862199+0000 (oldest deadline 2024-04-25T14:09:01.762435+0000) 2024-04-25T14:09:36.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:36.894+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.064757+0000 front 2024-04-25T14:08:48.064858+0000 (oldest deadline 2024-04-25T14:09:10.964965+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:36.894+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.766651+0000 front 2024-04-25T14:08:56.766633+0000 (oldest deadline 2024-04-25T14:09:19.066392+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8[51480]: 2024-04-25T14:09:36.894+0000 7fea569ff700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.268393+0000 front 2024-04-25T14:09:04.268307+0000 (oldest deadline 2024-04-25T14:09:29.568352+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:36.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:36.700+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:37.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:37.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:37.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:37.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:37.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:37.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:36 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:36.812+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:37.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:37.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:37.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:37.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:37.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:37.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:37 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:37.067+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:37.996 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:37 smithi138 podman[73379]: 2024-04-25 14:09:37.745798836 +0000 UTC m=+5.771433233 container died d0a11f6b1a3b4131c98b06e81837cb4b209e591ed244ead67a4e1ccd531c3647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:37.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:37 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:37.665+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:38.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:38.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:38.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:38.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:38.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:38.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:37 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:37.771+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:38.378 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 podman[73379]: 2024-04-25 14:09:38.109095248 +0000 UTC m=+6.134729642 container cleanup d0a11f6b1a3b4131c98b06e81837cb4b209e591ed244ead67a4e1ccd531c3647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:09:38.379 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 bash[73379]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8 2024-04-25T14:09:38.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:38.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:38.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:38.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:38.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:38.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:38 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:38.097+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:38.630 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 podman[73398]: 2024-04-25 14:09:38.37968661 +0000 UTC m=+0.635744449 container remove d0a11f6b1a3b4131c98b06e81837cb4b209e591ed244ead67a4e1ccd531c3647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 podman[73488]: 2024-04-25 14:09:38.633363276 +0000 UTC m=+0.018356517 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 podman[73488]: 2024-04-25 14:09:38.763019486 +0000 UTC m=+0.148012759 container create cac97d9128c539f60f9046db1da3bb8d15306c2d3ae0def1094c5acc7efcea75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:38.889 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:38.890 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:38 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:38.630+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:39.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:39.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:39.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:39.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:39.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:39.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:38 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:38.799+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:39.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:38 smithi138 podman[73488]: 2024-04-25 14:09:38.99824163 +0000 UTC m=+0.383234865 container init cac97d9128c539f60f9046db1da3bb8d15306c2d3ae0def1094c5acc7efcea75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T14:09:39.246 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:39 smithi138 podman[73488]: 2024-04-25 14:09:39.001389531 +0000 UTC m=+0.386382759 container start cac97d9128c539f60f9046db1da3bb8d15306c2d3ae0def1094c5acc7efcea75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:09:39.247 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:39 smithi138 podman[73488]: 2024-04-25 14:09:39.131488925 +0000 UTC m=+0.516482155 container attach cac97d9128c539f60f9046db1da3bb8d15306c2d3ae0def1094c5acc7efcea75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:39.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:39 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:39.116+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:39.564 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:09:39 smithi138 podman[73488]: 2024-04-25 14:09:39.298380449 +0000 UTC m=+0.683373676 container died cac97d9128c539f60f9046db1da3bb8d15306c2d3ae0def1094c5acc7efcea75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-25T14:09:39.918 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:39.918 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:39.918 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:39.918 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:39.918 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:39.919 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:39 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:39.617+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:39.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T14:09:39.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:39.982 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T14:09:39.983 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T14:09:39.983 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.9 2024-04-25T14:09:40.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:39 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:39.836+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:40.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 systemd[1]: Stopping Ceph osd.9 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:40.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:40.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:40.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:40.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:40.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:40.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:40 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:40.103+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:40.724 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.287+0000 7fafbc982700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:40.724 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.287+0000 7fafbc982700 -1 osd.9 91 *** Got signal Terminated *** 2024-04-25T14:09:40.724 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.287+0000 7fafbc982700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:40.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:40.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:40.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:40.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:40.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:40.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:40.586+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:41.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:41.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:41.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:41.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:41.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:41.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:40 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:40.871+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:41.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:41.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:41.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:41.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:41.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:41.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:41 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:41.108+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:41.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:41 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:41.627+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:42.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:42.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:42.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:42.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:42.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:42.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:41 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:41.842+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:42.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:42.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:42.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:42.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:42.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:42.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:42 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:42.091+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:42.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:42.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:42.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:42.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:42.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:42.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:42 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:42.625+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:43.224 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:43.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:43.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:43.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:43.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:43.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:42 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:42.878+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:43.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:43.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:43.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:43.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:43.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:43.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:43 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:43.079+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:43.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:43 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:43.585+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.890+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:44.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:43 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:43.891+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:19.906701+0000 front 2024-04-25T14:09:19.906732+0000 (oldest deadline 2024-04-25T14:09:43.406765+0000) 2024-04-25T14:09:44.392 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:44.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:44.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:44.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:44.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:44.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:44 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:44.128+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:44.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:44.593+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:31.700551+0000 front 2024-04-25T14:08:31.700578+0000 (oldest deadline 2024-04-25T14:08:55.800751+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.902075+0000 front 2024-04-25T14:08:39.901909+0000 (oldest deadline 2024-04-25T14:09:05.802019+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:45.802901+0000 front 2024-04-25T14:08:45.802833+0000 (oldest deadline 2024-04-25T14:09:11.102793+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:56.403844+0000 front 2024-04-25T14:08:56.403877+0000 (oldest deadline 2024-04-25T14:09:22.304073+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:02.304532+0000 front 2024-04-25T14:09:02.304562+0000 (oldest deadline 2024-04-25T14:09:25.804608+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:11.105776+0000 front 2024-04-25T14:09:11.105851+0000 (oldest deadline 2024-04-25T14:09:35.205612+0000) 2024-04-25T14:09:45.225 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:44 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9[64269]: 2024-04-25T14:09:44.938+0000 7fafb4f30700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:19.906701+0000 front 2024-04-25T14:09:19.906732+0000 (oldest deadline 2024-04-25T14:09:43.406765+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:45.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:45 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:45.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:45.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:45 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:45.579+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:46.095 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:45 smithi094 podman[125189]: 2024-04-25 14:09:45.829960688 +0000 UTC m=+5.648143615 container died b8aece4a841b6a5b3ce57afec5473a0385ff053be7f5859775a33f1debaada0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T14:09:46.453 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:46 smithi094 podman[125189]: 2024-04-25 14:09:46.18733282 +0000 UTC m=+6.005515746 container cleanup b8aece4a841b6a5b3ce57afec5473a0385ff053be7f5859775a33f1debaada0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T14:09:46.454 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:46 smithi094 bash[125189]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9 2024-04-25T14:09:46.642 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:46.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:46 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:46.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:46.724 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:46 smithi094 podman[125206]: 2024-04-25 14:09:46.454886663 +0000 UTC m=+0.619471562 container remove b8aece4a841b6a5b3ce57afec5473a0385ff053be7f5859775a33f1debaada0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:46.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:46 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:46.623+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:47.101 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:46 smithi094 podman[125297]: 2024-04-25 14:09:46.701535131 +0000 UTC m=+0.017510527 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:47.101 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:46 smithi094 podman[125297]: 2024-04-25 14:09:46.822425309 +0000 UTC m=+0.138400692 container create 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:09:47.101 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:47 smithi094 podman[125297]: 2024-04-25 14:09:47.102193557 +0000 UTC m=+0.418168943 container init 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:09:47.391 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:47 smithi094 podman[125297]: 2024-04-25 14:09:47.105242885 +0000 UTC m=+0.421218268 container start 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:09:47.391 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:47 smithi094 podman[125297]: 2024-04-25 14:09:47.241179095 +0000 UTC m=+0.557154478 container attach 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T14:09:47.642 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:47.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:47 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:47.154+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:47.724 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:47 smithi094 podman[125297]: 2024-04-25 14:09:47.392862635 +0000 UTC m=+0.708838017 container died 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:47.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:47 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:47.607+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:48.004 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:09:48 smithi094 podman[125323]: 2024-04-25 14:09:48.003254696 +0000 UTC m=+0.605158181 container remove 9de65c1957e7a253508b403f7ff361a17c8eef9a85339b3b03962fd518a2c771 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:09:48.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T14:09:48.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:48.063 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T14:09:48.063 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T14:09:48.063 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.10 2024-04-25T14:09:48.370 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 systemd[1]: Stopping Ceph osd.10 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:48.370 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:48.370 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:48.371 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:48.371 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:48.371 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:48.371 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:48.371 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.129+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:48.642 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.371+0000 7f5529d61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:48.642 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.371+0000 7f5529d61700 -1 osd.10 91 *** Got signal Terminated *** 2024-04-25T14:09:48.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:48 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:48.371+0000 7f5529d61700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:48.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:48.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:48.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:48.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:48.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:48.998 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:48.569+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:49.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:49 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:49.146+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:49.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:49.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:49.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:49.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:49.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:49.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:49 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:49.566+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:50.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:50.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:50.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:50.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:50.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:50.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:50.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:50 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:50.143+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:50.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:50.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:50.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:50.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:50.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:50.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:50 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:50.546+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:51.393 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:51 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:51.126+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:51.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:51.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:51.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:51.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:51.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:51.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:51 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:51.513+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:52.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:52.644 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:52 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:52.147+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:52.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:52.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:52.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:52.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:52.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:52.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:52 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:52.530+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:53.642 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:30.903067+0000 front 2024-04-25T14:08:30.903088+0000 (oldest deadline 2024-04-25T14:08:53.803051+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:39.104183+0000 front 2024-04-25T14:08:39.104112+0000 (oldest deadline 2024-04-25T14:09:05.004265+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:48.505457+0000 front 2024-04-25T14:08:48.505351+0000 (oldest deadline 2024-04-25T14:09:10.205199+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.206702+0000 front 2024-04-25T14:08:54.206742+0000 (oldest deadline 2024-04-25T14:09:18.906812+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:03.607829+0000 front 2024-04-25T14:09:03.607861+0000 (oldest deadline 2024-04-25T14:09:27.107985+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.608926+0000 front 2024-04-25T14:09:10.608973+0000 (oldest deadline 2024-04-25T14:09:36.508894+0000) 2024-04-25T14:09:53.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10[57828]: 2024-04-25T14:09:53.180+0000 7f552230f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T14:09:18.810081+0000 front 2024-04-25T14:09:18.810094+0000 (oldest deadline 2024-04-25T14:09:44.610450+0000) 2024-04-25T14:09:53.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:53.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:53.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:53.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:53.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:53.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:53 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:53.577+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:54.171 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:53 smithi137 podman[71277]: 2024-04-25 14:09:53.902011228 +0000 UTC m=+5.639639880 container died 5b107141e1fc43c54aba315329162af8ccf26a94004c997f1b19f83e035c5696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T14:09:54.554 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:54 smithi137 podman[71277]: 2024-04-25 14:09:54.270791684 +0000 UTC m=+6.008420334 container cleanup 5b107141e1fc43c54aba315329162af8ccf26a94004c997f1b19f83e035c5696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T14:09:54.554 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:54 smithi137 bash[71277]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10 2024-04-25T14:09:54.888 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:54 smithi137 podman[71294]: 2024-04-25 14:09:54.555082909 +0000 UTC m=+0.650088303 container remove 5b107141e1fc43c54aba315329162af8ccf26a94004c997f1b19f83e035c5696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T14:09:54.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:54.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:54.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:54.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:54.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:54.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:54 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:54.587+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:55.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:54 smithi137 podman[71386]: 2024-04-25 14:09:54.889257915 +0000 UTC m=+0.112522185 container create 163bc032c1f344bd77a0ca1c3f6d89c8b116ba5f01205275de52835877f056ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:09:55.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:54 smithi137 podman[71386]: 2024-04-25 14:09:54.794618192 +0000 UTC m=+0.017882476 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:09:55.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:55 smithi137 podman[71386]: 2024-04-25 14:09:55.083128779 +0000 UTC m=+0.306393050 container init 163bc032c1f344bd77a0ca1c3f6d89c8b116ba5f01205275de52835877f056ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T14:09:55.143 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:55 smithi137 podman[71386]: 2024-04-25 14:09:55.086300581 +0000 UTC m=+0.309564852 container start 163bc032c1f344bd77a0ca1c3f6d89c8b116ba5f01205275de52835877f056ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T14:09:55.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:55 smithi137 podman[71386]: 2024-04-25 14:09:55.166323442 +0000 UTC m=+0.389587725 container attach 163bc032c1f344bd77a0ca1c3f6d89c8b116ba5f01205275de52835877f056ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T14:09:55.643 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:09:55 smithi137 podman[71386]: 2024-04-25 14:09:55.379971369 +0000 UTC m=+0.603235641 container died 163bc032c1f344bd77a0ca1c3f6d89c8b116ba5f01205275de52835877f056ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T14:09:55.976 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T14:09:55.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:55.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:55 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:55.633+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:09:56.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:09:56.015 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T14:09:56.015 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T14:09:56.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.11 2024-04-25T14:09:56.331 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 systemd[1]: Stopping Ceph osd.11 for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:09:56.332 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.331+0000 7fefb6995700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:09:56.332 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.331+0000 7fefb6995700 -1 osd.11 91 *** Got signal Terminated *** 2024-04-25T14:09:56.332 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.331+0000 7fefb6995700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:09:56.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:56.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:56 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:56.645+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:09:57.898 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:57.898 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:57.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:57.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:57.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:57.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:57.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:57 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:57.638+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:09:58.996 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:58.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:58 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:58.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:09:59.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:09:59 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:09:59.632+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T14:08:28.470811+0000 front 2024-04-25T14:08:28.470795+0000 (oldest deadline 2024-04-25T14:08:56.671415+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T14:08:40.172851+0000 front 2024-04-25T14:08:40.172971+0000 (oldest deadline 2024-04-25T14:09:02.472927+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T14:08:47.774108+0000 front 2024-04-25T14:08:47.774200+0000 (oldest deadline 2024-04-25T14:09:12.374774+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T14:08:54.675792+0000 front 2024-04-25T14:08:54.675896+0000 (oldest deadline 2024-04-25T14:09:19.975761+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T14:09:04.076981+0000 front 2024-04-25T14:09:04.077005+0000 (oldest deadline 2024-04-25T14:09:25.776764+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T14:09:10.477503+0000 front 2024-04-25T14:09:10.477564+0000 (oldest deadline 2024-04-25T14:09:35.777563+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T14:09:29.180364+0000 front 2024-04-25T14:09:29.180430+0000 (oldest deadline 2024-04-25T14:09:55.080179+0000) 2024-04-25T14:10:00.997 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:00 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11[58498]: 2024-04-25T14:10:00.621+0000 7fefb2f4b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.138:6820 osd.8 since back 2024-04-25T14:09:35.080585+0000 front 2024-04-25T14:09:35.080618+0000 (oldest deadline 2024-04-25T14:09:59.780597+0000) 2024-04-25T14:10:01.990 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:01 smithi138 podman[73606]: 2024-04-25 14:10:01.72373427 +0000 UTC m=+5.512461735 container died 6bda862c67890e9c3f39ed8439e817bbddb747fcfb85137c84a4687c77d46b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T14:10:02.246 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73606]: 2024-04-25 14:10:02.06471189 +0000 UTC m=+5.853439347 container cleanup 6bda862c67890e9c3f39ed8439e817bbddb747fcfb85137c84a4687c77d46b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T14:10:02.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 bash[73606]: ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11 2024-04-25T14:10:02.676 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73637]: 2024-04-25 14:10:02.249719894 +0000 UTC m=+0.521585884 container remove 6bda862c67890e9c3f39ed8439e817bbddb747fcfb85137c84a4687c77d46b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T14:10:02.985 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73727]: 2024-04-25 14:10:02.579897415 +0000 UTC m=+0.018212727 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:10:02.986 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73727]: 2024-04-25 14:10:02.684069487 +0000 UTC m=+0.122384828 container create ff94c0e2b6483337364dec610152d6287d122640c778ca47f3280aae95e5c260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T14:10:02.986 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73727]: 2024-04-25 14:10:02.903324607 +0000 UTC m=+0.341639902 container init ff94c0e2b6483337364dec610152d6287d122640c778ca47f3280aae95e5c260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T14:10:02.986 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73727]: 2024-04-25 14:10:02.906337255 +0000 UTC m=+0.344652550 container start ff94c0e2b6483337364dec610152d6287d122640c778ca47f3280aae95e5c260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:02.986 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:02 smithi138 podman[73727]: 2024-04-25 14:10:02.986839329 +0000 UTC m=+0.425154627 container attach ff94c0e2b6483337364dec610152d6287d122640c778ca47f3280aae95e5c260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:10:03.247 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:10:03 smithi138 podman[73727]: 2024-04-25 14:10:03.200122803 +0000 UTC m=+0.638438096 container died ff94c0e2b6483337364dec610152d6287d122640c778ca47f3280aae95e5c260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T14:10:03.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T14:10:03.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:03.844 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T14:10:03.844 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T14:10:03.845 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.a 2024-04-25T14:10:04.133 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:03 smithi094 systemd[1]: Stopping Ceph mds.a for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:04.474 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:04 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: 2024-04-25T14:10:04.130+0000 7f6d6908f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:04.474 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:04 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a[70872]: 2024-04-25T14:10:04.130+0000 7f6d6908f700 -1 mds.a *** got signal Terminated *** 2024-04-25T14:10:07.855 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:07 smithi094 podman[125422]: 2024-04-25 14:10:07.523550726 +0000 UTC m=+3.489318062 container died d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:10:08.163 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:07 smithi094 podman[125422]: 2024-04-25 14:10:07.972323826 +0000 UTC m=+3.938091164 container cleanup d5fdd1d08f3e0f565c06546dc2f367f119589585411d45fcc4f2a50796fa48eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T14:10:08.164 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:10:07 smithi094 bash[125422]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-a 2024-04-25T14:10:08.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T14:10:08.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:08.410 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T14:10:08.411 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T14:10:08.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.d 2024-04-25T14:10:08.724 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:08 smithi094 systemd[1]: Stopping Ceph mds.d for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:09.224 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d[69909]: 2024-04-25T14:10:08.724+0000 7f6ab68a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:09.224 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:08 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d[69909]: 2024-04-25T14:10:08.724+0000 7f6ab68a6700 -1 mds.d *** got signal Terminated *** 2024-04-25T14:10:10.875 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:10 smithi094 podman[125543]: 2024-04-25 14:10:10.618112161 +0000 UTC m=+1.994722841 container died 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:10:11.224 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:10 smithi094 podman[125543]: 2024-04-25 14:10:10.966996796 +0000 UTC m=+2.343607475 container cleanup 29f43754b6a81fb7ed63b03f63484f86c81486e9b67bf1045509c13a8c106601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T14:10:11.224 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:10:10 smithi094 bash[125543]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-d 2024-04-25T14:10:11.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T14:10:11.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:11.410 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T14:10:11.410 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T14:10:11.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.g 2024-04-25T14:10:11.724 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:11 smithi094 systemd[1]: Stopping Ceph mds.g for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:12.224 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: 2024-04-25T14:10:11.745+0000 7f1438ea8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:12.224 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:11 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g[70512]: 2024-04-25T14:10:11.745+0000 7f1438ea8700 -1 mds.g *** got signal Terminated *** 2024-04-25T14:10:15.937 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:15 smithi094 podman[125664]: 2024-04-25 14:10:15.517918316 +0000 UTC m=+3.890896110 container died 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:10:16.224 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:16 smithi094 podman[125664]: 2024-04-25 14:10:16.020463578 +0000 UTC m=+4.393441372 container cleanup 8a1744e7809cabc2ea73fdd4840a479ded246adca6afd6260ff51957c6ad5ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:10:16.224 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:10:16 smithi094 bash[125664]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-g 2024-04-25T14:10:16.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T14:10:16.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:16.474 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T14:10:16.474 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T14:10:16.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.j 2024-04-25T14:10:16.724 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:16 smithi094 systemd[1]: Stopping Ceph mds.j for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:17.224 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j[69561]: 2024-04-25T14:10:16.807+0000 7f7e693fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:17.224 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:16 smithi094 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j[69561]: 2024-04-25T14:10:16.807+0000 7f7e693fd700 -1 mds.j *** got signal Terminated *** 2024-04-25T14:10:19.974 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:19 smithi094 podman[125785]: 2024-04-25 14:10:19.666142262 +0000 UTC m=+2.975074942 container died abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T14:10:20.290 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 podman[125785]: 2024-04-25 14:10:20.107761111 +0000 UTC m=+3.416693779 container cleanup abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:20.291 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 bash[125785]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j 2024-04-25T14:10:20.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T14:10:20.544 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 podman[125799]: 2024-04-25 14:10:20.29204284 +0000 UTC m=+0.623947276 container remove abaab5d7e28eab8eff9311dd4c852d1895b92b6add69846b65d9c16841bff688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:10:20.545 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T14:10:20.545 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 systemd[1]: Stopped Ceph mds.j for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T14:10:20.545 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:10:20 smithi094 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.216s CPU time. 2024-04-25T14:10:20.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:20.555 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T14:10:20.555 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T14:10:20.555 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.b 2024-04-25T14:10:20.861 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:20 smithi137 systemd[1]: Stopping Ceph mds.b for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:21.142 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T14:10:20.860+0000 7f73aae59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:21.143 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:20 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b[63496]: 2024-04-25T14:10:20.860+0000 7f73aae59700 -1 mds.b *** got signal Terminated *** 2024-04-25T14:10:23.892 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:23 smithi137 podman[71496]: 2024-04-25 14:10:23.578709183 +0000 UTC m=+2.834221643 container died a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T14:10:24.202 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:23 smithi137 podman[71496]: 2024-04-25 14:10:23.977553109 +0000 UTC m=+3.233065583 container cleanup a6e3ee9a6b3a81949c50015de9d5d684defa652c733f4bb3ccf7580e37e1c175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:10:24.202 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:10:23 smithi137 bash[71496]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-b 2024-04-25T14:10:24.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T14:10:24.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:24.382 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T14:10:24.382 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T14:10:24.382 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.e 2024-04-25T14:10:24.892 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:24 smithi137 systemd[1]: Stopping Ceph mds.e for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:24.892 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: 2024-04-25T14:10:24.688+0000 7ff04fa96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:24.893 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:24 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e[62882]: 2024-04-25T14:10:24.688+0000 7ff04fa96700 -1 mds.e *** got signal Terminated *** 2024-04-25T14:10:27.864 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:27 smithi137 podman[71627]: 2024-04-25 14:10:27.577585416 +0000 UTC m=+2.978824555 container died 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T14:10:28.142 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:27 smithi137 podman[71627]: 2024-04-25 14:10:27.956014456 +0000 UTC m=+3.357253596 container cleanup 6524bf848e749ca55cda6b637fe67df3bb66a30eb751fe4174db7ef20f9b0176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T14:10:28.143 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:10:27 smithi137 bash[71627]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-e 2024-04-25T14:10:28.380 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T14:10:28.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:28.413 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T14:10:28.413 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T14:10:28.413 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.h 2024-04-25T14:10:28.768 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:28 smithi137 systemd[1]: Stopping Ceph mds.h for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:29.142 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: 2024-04-25T14:10:28.767+0000 7fb02ce3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:29.142 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:28 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h[63192]: 2024-04-25T14:10:28.767+0000 7fb02ce3f700 -1 mds.h *** got signal Terminated *** 2024-04-25T14:10:31.884 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:31 smithi137 podman[71747]: 2024-04-25 14:10:31.575930769 +0000 UTC m=+2.943627452 container died 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T14:10:32.142 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:31 smithi137 podman[71747]: 2024-04-25 14:10:31.984679155 +0000 UTC m=+3.352375846 container cleanup 64b97700661840a020b83deff3a48f9e7baf1666a53d7896608d8ead0e17ec75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:32.142 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:10:31 smithi137 bash[71747]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-h 2024-04-25T14:10:32.354 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T14:10:32.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:32.384 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T14:10:32.385 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T14:10:32.385 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.k 2024-04-25T14:10:32.892 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:32 smithi137 systemd[1]: Stopping Ceph mds.k for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:32.892 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: 2024-04-25T14:10:32.704+0000 7f10a1458700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:32.893 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:32 smithi137 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k[62567]: 2024-04-25T14:10:32.704+0000 7f10a1458700 -1 mds.k *** got signal Terminated *** 2024-04-25T14:10:35.887 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:35 smithi137 podman[71867]: 2024-04-25 14:10:35.597542236 +0000 UTC m=+2.998066982 container died c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:10:35.887 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:35 smithi137 podman[71867]: 2024-04-25 14:10:35.888395455 +0000 UTC m=+3.288920199 container cleanup c5ae7ce5f129c634350a452370d4d09c28a82d10f32c47d58ec03bc28a17a549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:10:36.138 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:10:35 smithi137 bash[71867]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-k 2024-04-25T14:10:36.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T14:10:36.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:36.343 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T14:10:36.343 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T14:10:36.343 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.c 2024-04-25T14:10:36.724 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:36 smithi138 systemd[1]: Stopping Ceph mds.c for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:36.996 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: 2024-04-25T14:10:36.724+0000 7f7ff1561700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:36.996 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:36 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c[63639]: 2024-04-25T14:10:36.724+0000 7f7ff1561700 -1 mds.c *** got signal Terminated *** 2024-04-25T14:10:39.858 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:39 smithi138 podman[73832]: 2024-04-25 14:10:39.524371276 +0000 UTC m=+2.991583141 container died 0cf929995c85c4c43d0d3e726bcd6345626477426cfc11f7c77146a58cb73caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:40.167 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:39 smithi138 podman[73832]: 2024-04-25 14:10:39.958823922 +0000 UTC m=+3.426035792 container cleanup 0cf929995c85c4c43d0d3e726bcd6345626477426cfc11f7c77146a58cb73caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:10:40.167 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:39 smithi138 bash[73832]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c 2024-04-25T14:10:40.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T14:10:40.418 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:40 smithi138 podman[73858]: 2024-04-25 14:10:40.168530921 +0000 UTC m=+0.640442194 container remove 0cf929995c85c4c43d0d3e726bcd6345626477426cfc11f7c77146a58cb73caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:40.418 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:40 smithi138 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-25T14:10:40.418 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:40 smithi138 systemd[1]: Stopped Ceph mds.c for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T14:10:40.418 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:10:40 smithi138 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.c.service: Consumed 1min 23.439s CPU time. 2024-04-25T14:10:40.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:40.431 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T14:10:40.431 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T14:10:40.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.f 2024-04-25T14:10:40.746 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:40 smithi138 systemd[1]: Stopping Ceph mds.f for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:40.996 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: 2024-04-25T14:10:40.746+0000 7efe6cff8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:40.996 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:40 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f[63330]: 2024-04-25T14:10:40.746+0000 7efe6cff8700 -1 mds.f *** got signal Terminated *** 2024-04-25T14:10:43.805 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:43 smithi138 podman[73966]: 2024-04-25 14:10:43.530515754 +0000 UTC m=+2.885564756 container died c586f1c3852b78988bd680a513d26f34f9aa92370b330e590195c1c93bedfae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:10:44.163 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:43 smithi138 podman[73966]: 2024-04-25 14:10:43.888463703 +0000 UTC m=+3.243512712 container cleanup c586f1c3852b78988bd680a513d26f34f9aa92370b330e590195c1c93bedfae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T14:10:44.163 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:10:43 smithi138 bash[73966]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-f 2024-04-25T14:10:44.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T14:10:44.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:44.346 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T14:10:44.346 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T14:10:44.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.i 2024-04-25T14:10:44.746 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:44 smithi138 systemd[1]: Stopping Ceph mds.i for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:44.746 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: 2024-04-25T14:10:44.666+0000 7f2c7faa3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:44.747 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:44 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i[63949]: 2024-04-25T14:10:44.667+0000 7f2c7faa3700 -1 mds.i *** got signal Terminated *** 2024-04-25T14:10:47.785 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:47 smithi138 podman[74088]: 2024-04-25 14:10:47.523954188 +0000 UTC m=+2.963242124 container died a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True) 2024-04-25T14:10:48.152 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:47 smithi138 podman[74088]: 2024-04-25 14:10:47.927525994 +0000 UTC m=+3.366813911 container cleanup a10d3a91f64b5318b2f42937f32b5432c5560b4d07854051a9d7fd1d2b0489cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T14:10:48.153 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:10:47 smithi138 bash[74088]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-i 2024-04-25T14:10:48.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T14:10:48.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:48.321 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T14:10:48.321 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T14:10:48.321 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.l 2024-04-25T14:10:48.496 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:48 smithi138 systemd[1]: Stopping Ceph mds.l for f9021454-0307-11ef-bc93-c7b262605968... 2024-04-25T14:10:48.996 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: 2024-04-25T14:10:48.631+0000 7f031720b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:10:48.996 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:48 smithi138 ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l[63009]: 2024-04-25T14:10:48.631+0000 7f031720b700 -1 mds.l *** got signal Terminated *** 2024-04-25T14:10:51.883 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:51 smithi138 podman[74208]: 2024-04-25 14:10:51.531684708 +0000 UTC m=+2.992594548 container died a002a84e219607ec4d0a0ef667bf7493809d98528a372e135055d3cafb2b45fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T14:10:52.143 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:51 smithi138 podman[74208]: 2024-04-25 14:10:51.958772577 +0000 UTC m=+3.419682456 container cleanup a002a84e219607ec4d0a0ef667bf7493809d98528a372e135055d3cafb2b45fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:10:52.143 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:51 smithi138 bash[74208]: ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l 2024-04-25T14:10:52.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T14:10:52.404 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:52 smithi138 podman[74222]: 2024-04-25 14:10:52.144174356 +0000 UTC m=+0.614914582 container remove a002a84e219607ec4d0a0ef667bf7493809d98528a372e135055d3cafb2b45fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f9021454-0307-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T14:10:52.404 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:52 smithi138 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-25T14:10:52.404 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:52 smithi138 systemd[1]: Stopped Ceph mds.l for f9021454-0307-11ef-bc93-c7b262605968. 2024-04-25T14:10:52.404 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:10:52 smithi138 systemd[1]: ceph-f9021454-0307-11ef-bc93-c7b262605968@mds.l.service: Consumed 3.984s CPU time. 2024-04-25T14:10:52.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:10:52.415 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T14:10:52.415 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:10:52.609 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:10:55.023 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:10:55.213 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:10:57.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:10:57.880 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:11:00.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:00.328 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:00.359 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:00.384 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T14:11:00.386 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi094/crash 2024-04-25T14:11:00.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:11:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:11:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:11:00.427 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi137/crash 2024-04-25T14:11:00.428 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:11:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:11:00.468 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:11:00.470 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi138/crash 2024-04-25T14:11:00.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:11:00.506 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/f9021454-0307-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:11:00.506 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:11:00.507 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T14:11:00.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T14:11:00.541 INFO:tasks.cephadm:Compressing logs... 2024-04-25T14:11:00.541 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:11:00.583 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:11:00.585 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:11:00.605 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:11:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-04-25T14:11:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:11:00.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T14:11:00.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T14:11:00.617 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:11:00.617 INFO:teuthology.orchestra.run.smithi137.stderr:: No such file or directory 2024-04-25T14:11:00.617 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T14:11:00.618 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr: 94.7% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:11:00.623 INFO:teuthology.orchestra.run.smithi094.stderr: 83.7% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr: /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T14:11:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.6% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.i.log: 87.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.11.log: 93.5% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T14:11:00.625 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:11:00.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T14:11:00.631 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T14:11:00.639 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:11:00.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:11:00.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T14:11:00.651 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T14:11:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:11:00.663 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:11:00.669 INFO:teuthology.orchestra.run.smithi138.stderr: 91.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:11:00.671 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 83.6% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:11:00.682 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:11:00.753 INFO:teuthology.orchestra.run.smithi137.stderr: 93.5% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T14:11:00.915 INFO:teuthology.orchestra.run.smithi137.stderr: 90.5% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:11:00.972 INFO:teuthology.orchestra.run.smithi137.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:11:01.002 INFO:teuthology.orchestra.run.smithi137.stderr: 87.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:11:01.046 INFO:teuthology.orchestra.run.smithi094.stderr: 90.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T14:11:01.122 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T14:11:01.430 INFO:teuthology.orchestra.run.smithi138.stderr: 92.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T14:11:01.533 INFO:teuthology.orchestra.run.smithi094.stderr: 94.8% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T14:11:01.697 INFO:teuthology.orchestra.run.smithi137.stderr: 91.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:11:01.909 INFO:teuthology.orchestra.run.smithi094.stderr: 95.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T14:11:02.794 INFO:teuthology.orchestra.run.smithi137.stderr: 94.5% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T14:11:03.828 INFO:teuthology.orchestra.run.smithi094.stderr: 92.4% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi138.stderr: 95.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T14:11:14.055 INFO:teuthology.orchestra.run.smithi137.stderr: 92.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T14:11:14.475 INFO:teuthology.orchestra.run.smithi138.stderr: 94.6% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T14:11:15.499 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T14:11:18.154 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T14:11:19.588 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T14:11:21.487 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T14:11:21.489 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T14:11:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m20.893s 2024-04-25T14:11:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:user 1m15.216s 2024-04-25T14:11:21.489 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m2.826s 2024-04-25T14:11:22.581 INFO:teuthology.orchestra.run.smithi138.stderr: 94.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T14:11:24.865 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T14:11:24.925 INFO:teuthology.orchestra.run.smithi138.stderr: 94.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T14:11:28.581 INFO:teuthology.orchestra.run.smithi138.stderr: 94.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T14:11:38.202 INFO:teuthology.orchestra.run.smithi138.stderr: 94.5% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T14:11:38.204 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T14:11:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m37.605s 2024-04-25T14:11:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:user 2m37.207s 2024-04-25T14:11:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m5.095s 2024-04-25T14:12:44.741 INFO:teuthology.orchestra.run.smithi137.stderr: 96.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T14:15:11.885 INFO:teuthology.orchestra.run.smithi137.stderr: 94.0% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T14:15:48.467 INFO:teuthology.orchestra.run.smithi137.stderr: 93.9% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T14:16:35.470 INFO:teuthology.orchestra.run.smithi137.stderr: 94.2% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T14:17:01.738 INFO:teuthology.orchestra.run.smithi137.stderr: 94.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T14:17:42.174 INFO:teuthology.orchestra.run.smithi137.stderr: 94.1% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T14:29:13.799 INFO:teuthology.orchestra.run.smithi137.stderr: 93.2% -- replaced with /var/log/ceph/f9021454-0307-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T14:29:13.925 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-25T14:29:13.925 INFO:teuthology.orchestra.run.smithi137.stderr:real 18m13.326s 2024-04-25T14:29:13.925 INFO:teuthology.orchestra.run.smithi137.stderr:user 12m13.221s 2024-04-25T14:29:13.925 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m30.815s 2024-04-25T14:29:13.926 INFO:tasks.cephadm:Archiving logs... 2024-04-25T14:29:13.926 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi094/log 2024-04-25T14:29:13.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:29:24.991 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi137/log 2024-04-25T14:29:24.992 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:31:26.753 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi138/log 2024-04-25T14:31:26.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:31:45.224 INFO:tasks.cephadm:Removing cluster... 2024-04-25T14:31:45.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force 2024-04-25T14:31:45.417 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:31:45.784 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force 2024-04-25T14:31:46.372 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:31:47.217 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f9021454-0307-11ef-bc93-c7b262605968 --force 2024-04-25T14:31:47.409 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: f9021454-0307-11ef-bc93-c7b262605968 2024-04-25T14:31:47.892 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T14:31:47.892 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:31:47.911 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:31:47.928 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:31:47.944 INFO:tasks.cephadm:Teardown complete 2024-04-25T14:31:47.944 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T14:31:47.956 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T14:31:47.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:31:47.959 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:31:47.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:31:48.110 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T14:31:48.121 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T14:31:48.121 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:31:48.124 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:31:48.127 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:31:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:31:48.139 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:31:48.140 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:31:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:31:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T14:31:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 7 377 93 +968us[ +947us] +/- 54ms 2024-04-25T14:31:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1652us[-1652us] +/- 76ms 2024-04-25T14:31:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +204us[ +204us] +/- 69ms 2024-04-25T14:31:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:31:48.145 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:31:48.145 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T14:31:48.146 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1015us[+1014us] +/- 55ms 2024-04-25T14:31:48.146 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1597us[-1597us] +/- 76ms 2024-04-25T14:31:48.146 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +281us[ +281us] +/- 69ms 2024-04-25T14:31:48.146 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:31:48.232 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:31:48.232 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-25T14:31:48.232 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +951us[ +947us] +/- 55ms 2024-04-25T14:31:48.233 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1641us[-1645us] +/- 76ms 2024-04-25T14:31:48.233 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +234us[ +230us] +/- 69ms 2024-04-25T14:31:48.233 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:31:48.245 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T14:31:48.255 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T14:31:48.256 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T14:31:48.305 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:31:48.427 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:31:48.540 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:31:48.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:31:48.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:48.637 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:31:48.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:48.668 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:31:48.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:48.694 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T14:31:48.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T14:31:48.748 INFO:teuthology.task.internal:Duration was 4017.207553 seconds 2024-04-25T14:31:48.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T14:31:48.839 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T14:31:48.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:31:48.842 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:31:48.845 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:31:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:31:48.881 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:31:48.911 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:31:49.287 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T14:31:49.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T14:31:49.288 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:31:49.309 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-04-25T14:31:49.309 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:31:49.333 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T14:31:49.333 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:31:49.358 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T14:31:49.358 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:31:49.361 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:31:49.376 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:31:49.432 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T14:31:49.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:31:49.435 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:31:49.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:31:49.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T14:31:49.861 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T14:31:49.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:31:49.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:31:49.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:31:49.936 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T14:31:49.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:49.953 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:49.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:49.972 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-25T14:31:49.984 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-04-25T14:31:49.999 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-25T14:31:50.026 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:50.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:50.040 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:50.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:50.054 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:31:50.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:31:50.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T14:31:50.093 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T14:31:50.095 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi094 2024-04-25T14:31:50.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:31:50.138 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi137 2024-04-25T14:31:50.139 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:31:50.176 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673159/remote/smithi138 2024-04-25T14:31:50.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:31:50.212 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T14:31:50.212 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:31:50.215 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:31:50.217 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:31:50.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T14:31:50.274 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T14:31:50.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T14:31:50.338 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T14:31:50.339 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:31:50.341 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:31:50.343 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:31:50.353 INFO:teuthology.orchestra.run.smithi094.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:31 /home/ubuntu/cephtest 2024-04-25T14:31:50.355 INFO:teuthology.orchestra.run.smithi137.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:31 /home/ubuntu/cephtest 2024-04-25T14:31:50.357 INFO:teuthology.orchestra.run.smithi138.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:31 /home/ubuntu/cephtest 2024-04-25T14:31:50.358 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T14:31:50.370 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T14:31:50.397 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 4017.207552909851 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T14:31:50.397 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:31:50.482 INFO:teuthology.run:pass